|
Download README.md from inworld/sava: direct link, hf CLI and curl.
- Browser
- Download file 7.24 kB
-
https://huggingface.co/inworld/sava/resolve/main/README.md
- Command line
-
hf download hf://inworld/sava/README.md
-
curl -L -o README.md https://huggingface.co/inworld/sava/resolve/main/README.md
7.24 kB
| license: apache-2.0 | |
| library_name: inworld-sava | |
| pipeline_tag: voice-activity-detection | |
| tags: [audio, speech, voice-activity-detection, vad, semantic-vad, turn-detection, end-of-turn, streaming, realtime] | |
| <p align="center"> | |
| <picture> | |
| <source media="(prefers-color-scheme: dark)" srcset="https://huggingface.co/inworld/sava/resolve/main/docs/assets/inworld-logo-white.svg"> | |
| <img src="https://huggingface.co/inworld/sava/resolve/main/docs/assets/inworld-logo-black.svg" alt="Inworld" width="220"> | |
| </picture> | |
| </p> | |
| <h1 align="center">SAVA</h1> | |
| <p align="center"> | |
| Semantic acoustic voice activity: streaming speech and end-of-turn detection in one model with an open license. | |
| </p> | |
| <p align="center"> | |
| <a href="https://github.com/inworld-ai/sava"><img src="https://img.shields.io/badge/GitHub-inworld--ai%2Fsava-181717?logo=github" alt="GitHub: inworld-ai/sava"></a> | |
| </p> | |
| --- | |
| ## TL;DR | |
| SAVA returns two probabilities for a live audio stream: | |
| * `speech` → whether a frame holds speech | |
| * `turn` → whether the speaker's turn has ended | |
| The model's shared network uses acoustic and sentence context to distinguish speech from coughs, laughs, music and keyboard clatter. | |
| | | SAVA | | |
| |---|---| | |
| | made for | voice activity and conversational turn taking | | |
| | input | 16 kHz mono audio | | |
| | output | two probabilities every 40 ms | | |
| | timing | speech probability for a frame is emitted 120 ms later (lookahead); end-of-turn probability has no lookahead | | |
| | parameters | 25.5M | | |
| | execution | plain PyTorch, CPU or an explicitly selected device | | |
| Use it to open a turn when speech starts and close it when the sentence is complete. | |
| The [reference turn policy](https://github.com/inworld-ai/sava/blob/main/docs/USAGE.md) turns the probabilities into segments; applications can supply their own policy. | |
| SAVA responds to any voice. It does not pick out a target speaker, and it leaves interruptions and backchannels to the application's dialogue policy. | |
| ## Quick start | |
| Using a coding agent? Point it at the [setup section of AGENTS.md](https://github.com/inworld-ai/sava/blob/main/AGENTS.md#setting-up-sava-for-a-user): it installs the package, checks the machine with `sava-doctor` and runs SAVA on a first recording. | |
| With Python 3.10 or later: | |
| ```bash | |
| python -m pip install inworld-sava | |
| # Or from a clone: git clone https://github.com/inworld-ai/sava && cd sava && python -m pip install -e . | |
| sava run recording.wav | |
| sava run recording.wav --turns_dir turns/ --curves curves.json | |
| ``` | |
| ```python | |
| from inworld import sava | |
| model = sava.load("inworld/sava") | |
| curves = sava.run(model, sava.load_audio("recording.wav")) | |
| turns = sava.segment(curves, sava.Policy(), model.lookahead_frames) | |
| session = sava.Session(model) | |
| for piece in pieces_of_16k_audio: # One-dimensional floating-point mono samples. | |
| frames = session.push(piece) | |
| frames = session.flush() | |
| ``` | |
| `sava.load("inworld/sava")` and the CLI's default model download the weights on first use and verify their checksum. | |
| The CLI defaults to CPU; `--device` selects another device. | |
| For microphone input, install `python -m pip install "inworld-sava[mic]"` and run `sava mic`. | |
| The code, [the integration guide](https://github.com/inworld-ai/sava/blob/main/docs/USAGE.md) and the [benchmarks](https://github.com/inworld-ai/sava/blob/main/bench/README.md) are on GitHub: [inworld-ai/sava](https://github.com/inworld-ai/sava). | |
| ## Hardware | |
| Tested on NVIDIA Ampere, Hopper and Blackwell GPUs and on CPU. | |
| On an Apple M3 Max CPU one stream takes 3.8 ms per 40 ms frame; [the integration guide](https://github.com/inworld-ai/sava/blob/main/docs/USAGE.md) lists the measured configurations. | |
| ## The benchmarks | |
| <p align="center"> | |
| <picture> | |
| <source media="(prefers-color-scheme: dark)" srcset="https://huggingface.co/inworld/sava/resolve/main/docs/assets/vad-dark.svg"> | |
| <img src="https://huggingface.co/inworld/sava/resolve/main/docs/assets/vad-light.svg" alt="Voice-activity frame AUC for SAVA and Silero VAD on nine public dataset readings" width="100%"> | |
| </picture> | |
| </p> | |
| <p align="center"> | |
| <picture> | |
| <source media="(prefers-color-scheme: dark)" srcset="https://huggingface.co/inworld/sava/resolve/main/docs/assets/turn-dark.svg"> | |
| <img src="https://huggingface.co/inworld/sava/resolve/main/docs/assets/turn-light.svg" alt="End-of-turn clip AUC for SAVA and smart-turn on public datasets" width="100%"> | |
| </picture> | |
| </p> | |
| Voice activity is compared with Silero VAD; end of turn is compared with pipecat smart-turn v3.2. | |
| These are per-dataset measurements, with no combined ranking. | |
| Speech scores use each model's own output timing, so SAVA's 120 ms lookahead makes the comparison unequal in latency. | |
| Quiet speech-like noise can trigger SAVA; the ESC-50 results measure that failure mode. | |
| [The complete tables, confidence intervals, noise tests, turn latency and reproduction steps](https://github.com/inworld-ai/sava/blob/main/bench/README.md) are on GitHub. | |
| ## Architecture | |
| <p align="center"> | |
| <picture> | |
| <source media="(prefers-color-scheme: dark)" srcset="https://huggingface.co/inworld/sava/resolve/main/docs/assets/architecture-dark.svg"> | |
| <img src="https://huggingface.co/inworld/sava/resolve/main/docs/assets/architecture-light.svg" alt="SAVA: causal log-mel features, frame mixer, Mamba and attention trunk, speech and turn heads" width="100%"> | |
| </picture> | |
| </p> | |
| | part | detail | | |
| |---|---| | |
| | front end | causal log-mel, 128 Slaney mel bins of a 25 ms Hann DFT every 10 ms, log10 with a 1e-10 floor, `(x + 4) / 4`; four hops per frame plus the frame's log RMS | | |
| | mixer | the current and previous frame's features, one linear projection to 512, RMSNorm | | |
| | trunk | three Mamba-2 blocks (expand 2, head dim 64, state 128, conv kernel 4) and one attention block (8 query heads, 2 key/value heads, head dim 64, 512-frame causal window, rotary positions, RMS-normed q and k, exclusive self attention), each with a SwiGLU FFN of 2048 | | |
| | voice-activity head | RMSNorm, down-projection to 512, one attention block (window 512, FFN 1536), RMSNorm, two linear readouts: on time, and the released three-frame look-ahead | | |
| | end-of-turn head | the trunk output with a 64-wide embedding of the previous frame's on-time speech probability, the same block shape, a linear readout with four horizon columns (0, 200, 400, 800 ms); column 0 is the released output | | |
| ## Training data | |
| SAVA was trained on approximately 5,000 hours of unique audio. | |
| Inworld built training examples from real and synthetic conversational speech | |
| mixed with non-verbal vocal sounds, noise, music and room acoustics, with a | |
| speech label and an end-of-turn label for every frame. The training recipe, | |
| code and data are not part of this release. | |
| Third-party datasets used in training are credited in [NOTICE](NOTICE). | |
| ## License | |
| Code and weights are Apache-2.0 ([LICENSE](LICENSE)). | |
| Third-party notices are in [NOTICE](NOTICE). | |
| ## How to contribute | |
| Everyone is welcome to use the model however they like and to make it better in any way they see fit! | |
| Development happens on GitHub: see [CONTRIBUTING.md](https://github.com/inworld-ai/sava/blob/main/CONTRIBUTING.md) for setup, tests and bug reports. | |
| --- | |
| <p align="center">Built for people with ❤️ by Inworld.</p> | |