| --- |
| license: mit |
| tags: |
| - audio |
| - breath-detection |
| - onnx |
| - speech |
| library_name: onnx |
| --- |
| |
| # respiro-en-onnx |
|
|
| ONNX conversion of **Respiro-en**, a per-frame breath detector for English speech. |
|
|
| - **Original work:** [Respiro-en](https://github.com/ydqmkkx/Respiro-en) by DongYANG (GitHub: `ydqmkkx`) |
| - **License:** MIT (see `LICENSE`, preserving the original copyright) |
| - **This repo** only redistributes an ONNX export of the original `respiro-en.pt` |
| checkpoint. No retraining or architectural change was performed; the network is |
| numerically identical to the original within float tolerance (max abs diff of the |
| per-frame probabilities vs. the original PyTorch model ≈ 3e-6 across the upstream |
| sample clips). |
|
|
| ## Model |
|
|
| - **File:** `respiro-en.onnx` (opset 17) |
| - **Input:** `feat` — float32 `[1, 3, 128, T]` |
| - channel 0: log-mel spectrogram in dB (`librosa.power_to_db(melspectrogram(...), ref=np.max)`) |
| - channel 1: variance of the log-mel across the frequency axis (unbiased), broadcast over mels |
| - channel 2: zero-crossing rate, broadcast over mels |
| - feature params: 16 kHz, `n_fft=400`, `hop_length=160` (10 ms hop), `n_mels=128`, |
| Slaney mel scale (`htk=False`), `power=2.0`, `power_to_db(ref=np.max, top_db=80)` |
| - **Output:** `prob` — float32 `[1, T]`, per-frame breath probability in `[0, 1]` (10 ms hop) |
| - **Dynamic axis:** `T` (sequence length) is dynamic. |
| |
| > Note on the export: the original network feeds a `length` tensor to a torchaudio |
| > Conformer purely to build a self-attention padding mask. For a single, |
| > full-length (`batch=1`, unpadded) input that mask is all-valid, so it has no effect |
| > on the output. To make the exported graph robust to arbitrary `T`, the |
| > `torch.nn.MultiheadAttention` modules were replaced with an equivalent |
| > dynamic-shape implementation (same weights) before export, which removed the need |
| > for the `length` input. The exported model therefore has a single input, `feat`. |
|
|
| ## Conversion |
|
|
| Produced by `Tools/convert_respiro_onnx.py` in the pbCosmo repository (one-time, |
| offline). The C++ application (pbCosmo) replicates the exact librosa/torch feature |
| extraction natively and runs this ONNX via ONNX Runtime. |
|
|
| ## Attribution |
|
|
| If you use this model, please cite / credit the original authors of |
| [Respiro-en](https://github.com/ydqmkkx/Respiro-en). |
|
|
