VAD Multi Speaker
Detect speech activity from any speaker in the original audio.
VAD Multi Speaker is a production voice activity detection model by ai-coustics that decides when anyone is speaking directly on noisy, multi-speaker, far-field microphone input β the default choice for real-time, noise-robust turn-taking. Designed to work without a separate denoiser, in parallel with or independent of speech enhancement. Outperforms Silero VAD in accuracy, balance, and reliability; 30 ms latency, no GPU needed, no ONNX dependency. Real-time audio intelligence that makes Voice AI work in production. Not just in the lab. Inference runs through the aic-sdk Python package (also available for Rust, C/C++, Node.js, and WebAssembly).
Setup
- Install the SDK:
pip install aic-sdk - Create a free SDK key on the ai-coustics playground β 30-day trial included, no credit card required.
Usage
# initialize the voice activity detector
import aic_sdk as aic
model = aic.Model.from_file(aic.Model.download("vad-ms-2.1-xxs-16khz", "./models"))
vad = aic.Vad(model, "{aic-sdk-key}", aic.ProcessorConfig.optimal(model)) # get a free SDK key: https://developers.ai-coustics.com/login
# feed mono float32 blocks β the audio is not modified, only analyzed
vad.process(audio_block)
# enjoy real-time, noise-robust turn-taking
speech_detected = vad.get_context().is_speech_detected()
Tuning turn-taking behavior
ctx = vad.get_context()
ctx.set_parameter(aic.VadParameter.Sensitivity, 0.5) # 0.0β1.0 probability threshold
ctx.set_parameter(aic.VadParameter.MinimumSpeechDuration, 0.1) # seconds before "speech" triggers
ctx.set_parameter(aic.VadParameter.SpeechHoldDuration, 0.3) # seconds speech is held after it stops
ctx.raw_vad_probability() exposes the unprocessed per-block probability; ctx.get_prediction_delay() returns the prediction delay in samples for aligning decisions with the input timeline.
Running VAD alongside speech enhancement
Feed the VAD the original input audio, not the enhanced output β run both on the same block:
vad.process(block) # reads the block, does not modify it
enhanced = processor.process(block) # enhances the same original block
Align predictions with your audio timeline via ctx.get_prediction_delay() (delay in samples, like ProcessorContext.get_audio_delay() for enhancement).
Which model should I use?
| Model | Use it for |
|---|---|
| VAD Multi Speaker | Default choice. Standalone VAD for noisy, multi-speaker, far-field audio β no separate denoiser required. |
| VAD Voice Focus | Primary-speaker turn-taking in voice agents. Preferred over chaining Voice Focus enhancement into VAD Multi Speaker. |
Related ai-coustics models
- π Quail Voice Focus β primary-speaker isolation for speech-to-text and voice agents
- π Quail Multi Speaker β multi-speaker speech enhancement for speech-to-text
- π§ Rook Multi Speaker β speech enhancement for human listening
- ποΈ VAD Voice Focus β primary-speaker VAD for voice agents
- π Tyto β predict and diagnose downstream failures in Voice AI stacks
Resources
- π VAD Multi Speaker docs Β· SDK quickstart Β· docs.ai-coustics.com
- π§© Framework quickstarts: LiveKit, Pipecat
- ποΈ All model versions & checksums: artifacts.ai-coustics.io
- π¦ Python SDK: pypi.org/project/aic-sdk
- π Free SDK key: developers.ai-coustics.com
- π’ Enterprise licensing: ai-coustics.com
- Downloads last month
- 71