File size: 7,236 Bytes
1beae7c | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 | ---
license: apache-2.0
library_name: inworld-sava
pipeline_tag: voice-activity-detection
tags: [audio, speech, voice-activity-detection, vad, semantic-vad, turn-detection, end-of-turn, streaming, realtime]
---
<p align="center">
<picture>
<source media="(prefers-color-scheme: dark)" srcset="https://huggingface.co/inworld/sava/resolve/main/docs/assets/inworld-logo-white.svg">
<img src="https://huggingface.co/inworld/sava/resolve/main/docs/assets/inworld-logo-black.svg" alt="Inworld" width="220">
</picture>
</p>
<h1 align="center">SAVA</h1>
<p align="center">
Semantic acoustic voice activity: streaming speech and end-of-turn detection in one model with an open license.
</p>
<p align="center">
<a href="https://github.com/inworld-ai/sava"><img src="https://img.shields.io/badge/GitHub-inworld--ai%2Fsava-181717?logo=github" alt="GitHub: inworld-ai/sava"></a>
</p>
---
## TL;DR
SAVA returns two probabilities for a live audio stream:
* `speech` → whether a frame holds speech
* `turn` → whether the speaker's turn has ended
The model's shared network uses acoustic and sentence context to distinguish speech from coughs, laughs, music and keyboard clatter.
| | SAVA |
|---|---|
| made for | voice activity and conversational turn taking |
| input | 16 kHz mono audio |
| output | two probabilities every 40 ms |
| timing | speech probability for a frame is emitted 120 ms later (lookahead); end-of-turn probability has no lookahead |
| parameters | 25.5M |
| execution | plain PyTorch, CPU or an explicitly selected device |
Use it to open a turn when speech starts and close it when the sentence is complete.
The [reference turn policy](https://github.com/inworld-ai/sava/blob/main/docs/USAGE.md) turns the probabilities into segments; applications can supply their own policy.
SAVA responds to any voice. It does not pick out a target speaker, and it leaves interruptions and backchannels to the application's dialogue policy.
## Quick start
Using a coding agent? Point it at the [setup section of AGENTS.md](https://github.com/inworld-ai/sava/blob/main/AGENTS.md#setting-up-sava-for-a-user): it installs the package, checks the machine with `sava-doctor` and runs SAVA on a first recording.
With Python 3.10 or later:
```bash
python -m pip install inworld-sava
# Or from a clone: git clone https://github.com/inworld-ai/sava && cd sava && python -m pip install -e .
sava run recording.wav
sava run recording.wav --turns_dir turns/ --curves curves.json
```
```python
from inworld import sava
model = sava.load("inworld/sava")
curves = sava.run(model, sava.load_audio("recording.wav"))
turns = sava.segment(curves, sava.Policy(), model.lookahead_frames)
session = sava.Session(model)
for piece in pieces_of_16k_audio: # One-dimensional floating-point mono samples.
frames = session.push(piece)
frames = session.flush()
```
`sava.load("inworld/sava")` and the CLI's default model download the weights on first use and verify their checksum.
The CLI defaults to CPU; `--device` selects another device.
For microphone input, install `python -m pip install "inworld-sava[mic]"` and run `sava mic`.
The code, [the integration guide](https://github.com/inworld-ai/sava/blob/main/docs/USAGE.md) and the [benchmarks](https://github.com/inworld-ai/sava/blob/main/bench/README.md) are on GitHub: [inworld-ai/sava](https://github.com/inworld-ai/sava).
## Hardware
Tested on NVIDIA Ampere, Hopper and Blackwell GPUs and on CPU.
On an Apple M3 Max CPU one stream takes 3.8 ms per 40 ms frame; [the integration guide](https://github.com/inworld-ai/sava/blob/main/docs/USAGE.md) lists the measured configurations.
## The benchmarks
<p align="center">
<picture>
<source media="(prefers-color-scheme: dark)" srcset="https://huggingface.co/inworld/sava/resolve/main/docs/assets/vad-dark.svg">
<img src="https://huggingface.co/inworld/sava/resolve/main/docs/assets/vad-light.svg" alt="Voice-activity frame AUC for SAVA and Silero VAD on nine public dataset readings" width="100%">
</picture>
</p>
<p align="center">
<picture>
<source media="(prefers-color-scheme: dark)" srcset="https://huggingface.co/inworld/sava/resolve/main/docs/assets/turn-dark.svg">
<img src="https://huggingface.co/inworld/sava/resolve/main/docs/assets/turn-light.svg" alt="End-of-turn clip AUC for SAVA and smart-turn on public datasets" width="100%">
</picture>
</p>
Voice activity is compared with Silero VAD; end of turn is compared with pipecat smart-turn v3.2.
These are per-dataset measurements, with no combined ranking.
Speech scores use each model's own output timing, so SAVA's 120 ms lookahead makes the comparison unequal in latency.
Quiet speech-like noise can trigger SAVA; the ESC-50 results measure that failure mode.
[The complete tables, confidence intervals, noise tests, turn latency and reproduction steps](https://github.com/inworld-ai/sava/blob/main/bench/README.md) are on GitHub.
## Architecture
<p align="center">
<picture>
<source media="(prefers-color-scheme: dark)" srcset="https://huggingface.co/inworld/sava/resolve/main/docs/assets/architecture-dark.svg">
<img src="https://huggingface.co/inworld/sava/resolve/main/docs/assets/architecture-light.svg" alt="SAVA: causal log-mel features, frame mixer, Mamba and attention trunk, speech and turn heads" width="100%">
</picture>
</p>
| part | detail |
|---|---|
| front end | causal log-mel, 128 Slaney mel bins of a 25 ms Hann DFT every 10 ms, log10 with a 1e-10 floor, `(x + 4) / 4`; four hops per frame plus the frame's log RMS |
| mixer | the current and previous frame's features, one linear projection to 512, RMSNorm |
| trunk | three Mamba-2 blocks (expand 2, head dim 64, state 128, conv kernel 4) and one attention block (8 query heads, 2 key/value heads, head dim 64, 512-frame causal window, rotary positions, RMS-normed q and k, exclusive self attention), each with a SwiGLU FFN of 2048 |
| voice-activity head | RMSNorm, down-projection to 512, one attention block (window 512, FFN 1536), RMSNorm, two linear readouts: on time, and the released three-frame look-ahead |
| end-of-turn head | the trunk output with a 64-wide embedding of the previous frame's on-time speech probability, the same block shape, a linear readout with four horizon columns (0, 200, 400, 800 ms); column 0 is the released output |
## Training data
SAVA was trained on approximately 5,000 hours of unique audio.
Inworld built training examples from real and synthetic conversational speech
mixed with non-verbal vocal sounds, noise, music and room acoustics, with a
speech label and an end-of-turn label for every frame. The training recipe,
code and data are not part of this release.
Third-party datasets used in training are credited in [NOTICE](NOTICE).
## License
Code and weights are Apache-2.0 ([LICENSE](LICENSE)).
Third-party notices are in [NOTICE](NOTICE).
## How to contribute
Everyone is welcome to use the model however they like and to make it better in any way they see fit!
Development happens on GitHub: see [CONTRIBUTING.md](https://github.com/inworld-ai/sava/blob/main/CONTRIBUTING.md) for setup, tests and bug reports.
---
<p align="center">Built for people with ❤️ by Inworld.</p>
|