twmht/FireRedVAD-VAD-onnx

FireRedVAD non-streaming VAD (ONNX, float32 and float16)

This is an unofficial ONNX export of the non-streaming VAD model (VAD/) from FireRedTeam/FireRedVAD. It runs with ONNX Runtime and does not need PyTorch. It is not published or endorsed by the FireRedTeam. The float32 file has the original weights. The float16 file has the same weights rounded to float16. Both keep the original Apache-2.0 license.

Only the non-streaming VAD is included. The streaming VAD (Stream-VAD/) and the audio event detection model (AED/) were not exported.

Files

File	Description
fireredvad_vad.onnx	The float32 model (2.4 MB, opset 17, 0.59M parameters)
fireredvad_vad_fp16.onnx	The same model with float16 weights (1.2 MB). Input and output are still float32
cmvn.ark	Feature normalization statistics, copied from the original VAD/cmvn.ark
fireredvad_onnx.py	Example that runs the whole pipeline with ONNX Runtime (features, model, post-processing)
fireredvad_core/	The original post-processor, copied without changes from the FireRedVAD code (Apache-2.0)
export_onnx.py	The export script (needs PyTorch and a clone of the original code)

Model interface

• Input feat: float32, shape [batch, T, 80]. It is the 80-bin Kaldi log-mel filterbank (25 ms window, 10 ms shift, no dither, snip_edges=True, samples on the int16 scale), then normalized with cmvn.ark. batch and T are dynamic.
• Output probs: float32, shape [batch, T, 1]. It is the speech probability for each 10 ms frame, already passed through a sigmoid.
• The model is not streaming. It sees the whole input at once. The original code splits input longer than 30000 frames (300 s) into chunks, and the example does the same.
• The probabilities must still be turned into segments with the post-processor (smoothing, threshold, minimum lengths). The original defaults are smoothing window 5, threshold 0.4, minimum speech 20 frames, minimum silence 20 frames, maximum speech 2000 frames.

Usage

pip install onnxruntime kaldi_native_fbank kaldiio soundfile numpy
python fireredvad_onnx.py audio.wav   # 16 kHz, 16-bit, mono

from fireredvad_onnx import FireRedVadOnnx

vad = FireRedVadOnnx(".")            # folder with fireredvad_vad.onnx and cmvn.ark
print(vad.detect("audio.wav"))       # {'dur': 2.32, 'timestamps': [(0.44, 1.82)]}

To use the float16 file, pass model_file: FireRedVadOnnx(".", model_file="fireredvad_vad_fp16.onnx").

Convert other audio first: ffmpeg -i in.mp3 -ar 16000 -ac 1 -acodec pcm_s16le out.wav.

Verification

Compared with the original PyTorch model (torch 2.14.1 CPU, onnxruntime 1.23.2):

• On random features with lengths from 1 to 30000 frames, the largest probability difference was 7.2e-7.
• On 8 test utterances (2 to 12 s) and one 299 s file that uses the chunking path, the features from fireredvad_onnx.py were bit-identical to the original, and the final speech segments were identical in 9 of 9 files.
• I did not re-run the original benchmark (FLEURS-VAD-102). The original model card reports an F1 of 97.57% for this model. The outputs are numerically the same, so that result should carry over, but I did not check it.

Speed

Measured on a 32-thread laptop CPU, with audio made by repeating test speech. The values are the median of 2 to 15 runs, and other programs were running, so treat them as rough.

Threads	Audio	PyTorch model	ONNX Runtime model	Feature extraction	End to end, PyTorch → ONNX Runtime
1	60 s	161 ms	91 ms	69 ms	232 → 162 ms
1	1 h	9.9 s	6.2 s	4.2 s	14.2 → 10.5 s
4	60 s	45 ms	30 ms	69 ms	116 → 101 ms
4	1 h	3.7 s	2.4 s	4.4 s	8.3 → 7.0 s

• The model alone is 1.25 to 2.2 times faster with ONNX Runtime across all settings tested (1, 4 and 16 threads).
• End to end, it is only 1.1 to 1.5 times faster, because the Kaldi feature extraction takes 40% to 70% of the ONNX Runtime end-to-end time. It is single-threaded, so more threads do not help it.
• More than 4 threads gave no real gain, because the model is small.

Float16

fireredvad_vad_fp16.onnx was made from the float32 file with onnxconverter_common.float16.convert_float_to_float16(model, keep_io_types=True). The converter clamps weights smaller than about 6e-8 in magnitude to ±1e-7, and the numbers below include that effect. The graph still takes and returns float32.

Compared with the PyTorch float32 model on the same 9 files (8 short utterances and one 299 s file), RTX 4090, onnxruntime-gpu 1.23.2:

Run	Max probability difference	Segments identical	60 s of audio	1 h of audio
ONNX Runtime CPU, float32	2.4e-7	9/9	30 ms	1735 ms
ONNX Runtime CPU, float16	5.0e-4	9/9	64 ms	1721 ms
ONNX Runtime CUDA, float32 (TF32 on, the default)	1.6e-3	9/9	3.8 ms	157 ms
ONNX Runtime CUDA, float16	1.6e-3	9/9	1.8 ms	126 ms

• The times are for the model only, without feature extraction. On 1 h of audio, feature extraction takes more than 4 s on the CPU, so float16 does not change the total time much.
• float16 only helps on a GPU, and the gain is small. On the CPU it is slower for short audio and equal for long audio. Use the float32 file there.
• ONNX Runtime's CUDA provider enables TF32 by default, which already gives an error of about 1e-3 with the float32 file. With use_tf32=0 it falls to 4.8e-7 on random input. So the float16 error is about the same as the default GPU float32 error.
• In my environment the CUDA provider only loaded when torch was imported first, because that provides cuDNN. Without it, ONNX Runtime logged a warning and silently ran on the CPU, so check session.get_providers().
• I checked only these 9 files. I did not measure F1 on a labelled data set for float16.

Not tested

int8 quantization, and any other opset or ONNX Runtime version.

Credits

Original model and code: FireRedTeam/FireRedVAD (Apache-2.0), introduced in FireRedASR2S. Please cite the original work:

@article{xu2026fireredasr2s,
  title={FireRedASR2S: A State-of-the-Art Industrial-Grade All-in-One Automatic Speech Recognition System},
  author={Xu, Kaituo and Jia, Yan and Huang, Kai and Chen, Junjie and Li, Wenpeng and Liu, Kun and Xie, Feng-Long and Tang, Xu and Hu, Yao},
  journal={arXiv preprint arXiv:2603.10420},
  year={2026}
}