sava / README.md
ipoletaev's picture
Initial release
1beae7c
|
Raw History Blame Contribute Delete
7.24 kB
---
license: apache-2.0
library_name: inworld-sava
pipeline_tag: voice-activity-detection
tags: [audio, speech, voice-activity-detection, vad, semantic-vad, turn-detection, end-of-turn, streaming, realtime]
---
<p align="center">
<picture>
<source media="(prefers-color-scheme: dark)" srcset="https://huggingface.co/inworld/sava/resolve/main/docs/assets/inworld-logo-white.svg">
<img src="https://huggingface.co/inworld/sava/resolve/main/docs/assets/inworld-logo-black.svg" alt="Inworld" width="220">
</picture>
</p>
<h1 align="center">SAVA</h1>
<p align="center">
Semantic acoustic voice activity: streaming speech and end-of-turn detection in one model with an open license.
</p>
<p align="center">
<a href="https://github.com/inworld-ai/sava"><img src="https://img.shields.io/badge/GitHub-inworld--ai%2Fsava-181717?logo=github" alt="GitHub: inworld-ai/sava"></a>
</p>
---
## TL;DR
SAVA returns two probabilities for a live audio stream:
* `speech` → whether a frame holds speech
* `turn` → whether the speaker's turn has ended
The model's shared network uses acoustic and sentence context to distinguish speech from coughs, laughs, music and keyboard clatter.
| | SAVA |
|---|---|
| made for | voice activity and conversational turn taking |
| input | 16 kHz mono audio |
| output | two probabilities every 40 ms |
| timing | speech probability for a frame is emitted 120 ms later (lookahead); end-of-turn probability has no lookahead |
| parameters | 25.5M |
| execution | plain PyTorch, CPU or an explicitly selected device |
Use it to open a turn when speech starts and close it when the sentence is complete.
The [reference turn policy](https://github.com/inworld-ai/sava/blob/main/docs/USAGE.md) turns the probabilities into segments; applications can supply their own policy.
SAVA responds to any voice. It does not pick out a target speaker, and it leaves interruptions and backchannels to the application's dialogue policy.
## Quick start
Using a coding agent? Point it at the [setup section of AGENTS.md](https://github.com/inworld-ai/sava/blob/main/AGENTS.md#setting-up-sava-for-a-user): it installs the package, checks the machine with `sava-doctor` and runs SAVA on a first recording.
With Python 3.10 or later:
```bash
python -m pip install inworld-sava
# Or from a clone: git clone https://github.com/inworld-ai/sava && cd sava && python -m pip install -e .
sava run recording.wav
sava run recording.wav --turns_dir turns/ --curves curves.json
```
```python
from inworld import sava
model = sava.load("inworld/sava")
curves = sava.run(model, sava.load_audio("recording.wav"))
turns = sava.segment(curves, sava.Policy(), model.lookahead_frames)
session = sava.Session(model)
for piece in pieces_of_16k_audio: # One-dimensional floating-point mono samples.
frames = session.push(piece)
frames = session.flush()
```
`sava.load("inworld/sava")` and the CLI's default model download the weights on first use and verify their checksum.
The CLI defaults to CPU; `--device` selects another device.
For microphone input, install `python -m pip install "inworld-sava[mic]"` and run `sava mic`.
The code, [the integration guide](https://github.com/inworld-ai/sava/blob/main/docs/USAGE.md) and the [benchmarks](https://github.com/inworld-ai/sava/blob/main/bench/README.md) are on GitHub: [inworld-ai/sava](https://github.com/inworld-ai/sava).
## Hardware
Tested on NVIDIA Ampere, Hopper and Blackwell GPUs and on CPU.
On an Apple M3 Max CPU one stream takes 3.8 ms per 40 ms frame; [the integration guide](https://github.com/inworld-ai/sava/blob/main/docs/USAGE.md) lists the measured configurations.
## The benchmarks
<p align="center">
<picture>
<source media="(prefers-color-scheme: dark)" srcset="https://huggingface.co/inworld/sava/resolve/main/docs/assets/vad-dark.svg">
<img src="https://huggingface.co/inworld/sava/resolve/main/docs/assets/vad-light.svg" alt="Voice-activity frame AUC for SAVA and Silero VAD on nine public dataset readings" width="100%">
</picture>
</p>
<p align="center">
<picture>
<source media="(prefers-color-scheme: dark)" srcset="https://huggingface.co/inworld/sava/resolve/main/docs/assets/turn-dark.svg">
<img src="https://huggingface.co/inworld/sava/resolve/main/docs/assets/turn-light.svg" alt="End-of-turn clip AUC for SAVA and smart-turn on public datasets" width="100%">
</picture>
</p>
Voice activity is compared with Silero VAD; end of turn is compared with pipecat smart-turn v3.2.
These are per-dataset measurements, with no combined ranking.
Speech scores use each model's own output timing, so SAVA's 120 ms lookahead makes the comparison unequal in latency.
Quiet speech-like noise can trigger SAVA; the ESC-50 results measure that failure mode.
[The complete tables, confidence intervals, noise tests, turn latency and reproduction steps](https://github.com/inworld-ai/sava/blob/main/bench/README.md) are on GitHub.
## Architecture
<p align="center">
<picture>
<source media="(prefers-color-scheme: dark)" srcset="https://huggingface.co/inworld/sava/resolve/main/docs/assets/architecture-dark.svg">
<img src="https://huggingface.co/inworld/sava/resolve/main/docs/assets/architecture-light.svg" alt="SAVA: causal log-mel features, frame mixer, Mamba and attention trunk, speech and turn heads" width="100%">
</picture>
</p>
| part | detail |
|---|---|
| front end | causal log-mel, 128 Slaney mel bins of a 25 ms Hann DFT every 10 ms, log10 with a 1e-10 floor, `(x + 4) / 4`; four hops per frame plus the frame's log RMS |
| mixer | the current and previous frame's features, one linear projection to 512, RMSNorm |
| trunk | three Mamba-2 blocks (expand 2, head dim 64, state 128, conv kernel 4) and one attention block (8 query heads, 2 key/value heads, head dim 64, 512-frame causal window, rotary positions, RMS-normed q and k, exclusive self attention), each with a SwiGLU FFN of 2048 |
| voice-activity head | RMSNorm, down-projection to 512, one attention block (window 512, FFN 1536), RMSNorm, two linear readouts: on time, and the released three-frame look-ahead |
| end-of-turn head | the trunk output with a 64-wide embedding of the previous frame's on-time speech probability, the same block shape, a linear readout with four horizon columns (0, 200, 400, 800 ms); column 0 is the released output |
## Training data
SAVA was trained on approximately 5,000 hours of unique audio.
Inworld built training examples from real and synthetic conversational speech
mixed with non-verbal vocal sounds, noise, music and room acoustics, with a
speech label and an end-of-turn label for every frame. The training recipe,
code and data are not part of this release.
Third-party datasets used in training are credited in [NOTICE](NOTICE).
## License
Code and weights are Apache-2.0 ([LICENSE](LICENSE)).
Third-party notices are in [NOTICE](NOTICE).
## How to contribute
Everyone is welcome to use the model however they like and to make it better in any way they see fit!
Development happens on GitHub: see [CONTRIBUTING.md](https://github.com/inworld-ai/sava/blob/main/CONTRIBUTING.md) for setup, tests and bug reports.
---
<p align="center">Built for people with ❤️ by Inworld.</p>