File size: 2,330 Bytes
c6901b7 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 | ---
license: mit
tags:
- audio
- breath-detection
- onnx
- speech
library_name: onnx
---
# respiro-en-onnx
ONNX conversion of **Respiro-en**, a per-frame breath detector for English speech.
- **Original work:** [Respiro-en](https://github.com/ydqmkkx/Respiro-en) by DongYANG (GitHub: `ydqmkkx`)
- **License:** MIT (see `LICENSE`, preserving the original copyright)
- **This repo** only redistributes an ONNX export of the original `respiro-en.pt`
checkpoint. No retraining or architectural change was performed; the network is
numerically identical to the original within float tolerance (max abs diff of the
per-frame probabilities vs. the original PyTorch model ≈ 3e-6 across the upstream
sample clips).
## Model
- **File:** `respiro-en.onnx` (opset 17)
- **Input:** `feat` — float32 `[1, 3, 128, T]`
- channel 0: log-mel spectrogram in dB (`librosa.power_to_db(melspectrogram(...), ref=np.max)`)
- channel 1: variance of the log-mel across the frequency axis (unbiased), broadcast over mels
- channel 2: zero-crossing rate, broadcast over mels
- feature params: 16 kHz, `n_fft=400`, `hop_length=160` (10 ms hop), `n_mels=128`,
Slaney mel scale (`htk=False`), `power=2.0`, `power_to_db(ref=np.max, top_db=80)`
- **Output:** `prob` — float32 `[1, T]`, per-frame breath probability in `[0, 1]` (10 ms hop)
- **Dynamic axis:** `T` (sequence length) is dynamic.
> Note on the export: the original network feeds a `length` tensor to a torchaudio
> Conformer purely to build a self-attention padding mask. For a single,
> full-length (`batch=1`, unpadded) input that mask is all-valid, so it has no effect
> on the output. To make the exported graph robust to arbitrary `T`, the
> `torch.nn.MultiheadAttention` modules were replaced with an equivalent
> dynamic-shape implementation (same weights) before export, which removed the need
> for the `length` input. The exported model therefore has a single input, `feat`.
## Conversion
Produced by `Tools/convert_respiro_onnx.py` in the pbCosmo repository (one-time,
offline). The C++ application (pbCosmo) replicates the exact librosa/torch feature
extraction natively and runs this ONNX via ONNX Runtime.
## Attribution
If you use this model, please cite / credit the original authors of
[Respiro-en](https://github.com/ydqmkkx/Respiro-en).
|