File size: 7,236 Bytes
5eb9d57
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
---
license: apache-2.0
library_name: inworld-sava
pipeline_tag: voice-activity-detection
tags: [audio, speech, voice-activity-detection, vad, semantic-vad, turn-detection, end-of-turn, streaming, realtime]
---

<p align="center">
  <picture>
    <source media="(prefers-color-scheme: dark)" srcset="https://huggingface.co/inworld/sava/resolve/main/docs/assets/inworld-logo-white.svg">
    <img src="https://huggingface.co/inworld/sava/resolve/main/docs/assets/inworld-logo-black.svg" alt="Inworld" width="220">
  </picture>
</p>

<h1 align="center">SAVA</h1>

<p align="center">
  Semantic acoustic voice activity: streaming speech and end-of-turn detection in one model with an open license.
</p>

<p align="center">
  <a href="https://github.com/inworld-ai/sava"><img src="https://img.shields.io/badge/GitHub-inworld--ai%2Fsava-181717?logo=github" alt="GitHub: inworld-ai/sava"></a>
</p>

---

## TL;DR

SAVA returns two probabilities for a live audio stream:
* `speech` → whether a frame holds speech
* `turn` → whether the speaker's turn has ended

The model's shared network uses acoustic and sentence context to distinguish speech from coughs, laughs, music and keyboard clatter.

| | SAVA |
|---|---|
| made for | voice activity and conversational turn taking |
| input | 16 kHz mono audio |
| output | two probabilities every 40 ms |
| timing | speech probability for a frame is emitted 120 ms later (lookahead); end-of-turn probability has no lookahead |
| parameters | 25.5M |
| execution | plain PyTorch, CPU or an explicitly selected device |

Use it to open a turn when speech starts and close it when the sentence is complete.
The [reference turn policy](https://github.com/inworld-ai/sava/blob/main/docs/USAGE.md) turns the probabilities into segments; applications can supply their own policy.
SAVA responds to any voice. It does not pick out a target speaker, and it leaves interruptions and backchannels to the application's dialogue policy.

## Quick start

Using a coding agent? Point it at the [setup section of AGENTS.md](https://github.com/inworld-ai/sava/blob/main/AGENTS.md#setting-up-sava-for-a-user): it installs the package, checks the machine with `sava-doctor` and runs SAVA on a first recording.

With Python 3.10 or later:

```bash
python -m pip install inworld-sava
# Or from a clone: git clone https://github.com/inworld-ai/sava && cd sava && python -m pip install -e .
sava run recording.wav
sava run recording.wav --turns_dir turns/ --curves curves.json
```

```python
from inworld import sava

model = sava.load("inworld/sava")
curves = sava.run(model, sava.load_audio("recording.wav"))
turns = sava.segment(curves, sava.Policy(), model.lookahead_frames)

session = sava.Session(model)
for piece in pieces_of_16k_audio:  # One-dimensional floating-point mono samples.
    frames = session.push(piece)
frames = session.flush()
```

`sava.load("inworld/sava")` and the CLI's default model download the weights on first use and verify their checksum.
The CLI defaults to CPU; `--device` selects another device.
For microphone input, install `python -m pip install "inworld-sava[mic]"` and run `sava mic`.

The code, [the integration guide](https://github.com/inworld-ai/sava/blob/main/docs/USAGE.md) and the [benchmarks](https://github.com/inworld-ai/sava/blob/main/bench/README.md) are on GitHub: [inworld-ai/sava](https://github.com/inworld-ai/sava).

## Hardware

Tested on NVIDIA Ampere, Hopper and Blackwell GPUs and on CPU.
On an Apple M3 Max CPU one stream takes 3.8 ms per 40 ms frame; [the integration guide](https://github.com/inworld-ai/sava/blob/main/docs/USAGE.md) lists the measured configurations.

## The benchmarks

<p align="center">
  <picture>
    <source media="(prefers-color-scheme: dark)" srcset="https://huggingface.co/inworld/sava/resolve/main/docs/assets/vad-dark.svg">
    <img src="https://huggingface.co/inworld/sava/resolve/main/docs/assets/vad-light.svg" alt="Voice-activity frame AUC for SAVA and Silero VAD on nine public dataset readings" width="100%">
  </picture>
</p>

<p align="center">
  <picture>
    <source media="(prefers-color-scheme: dark)" srcset="https://huggingface.co/inworld/sava/resolve/main/docs/assets/turn-dark.svg">
    <img src="https://huggingface.co/inworld/sava/resolve/main/docs/assets/turn-light.svg" alt="End-of-turn clip AUC for SAVA and smart-turn on public datasets" width="100%">
  </picture>
</p>

Voice activity is compared with Silero VAD; end of turn is compared with pipecat smart-turn v3.2.
These are per-dataset measurements, with no combined ranking.
Speech scores use each model's own output timing, so SAVA's 120 ms lookahead makes the comparison unequal in latency.
Quiet speech-like noise can trigger SAVA; the ESC-50 results measure that failure mode.

[The complete tables, confidence intervals, noise tests, turn latency and reproduction steps](https://github.com/inworld-ai/sava/blob/main/bench/README.md) are on GitHub.

## Architecture

<p align="center">
  <picture>
    <source media="(prefers-color-scheme: dark)" srcset="https://huggingface.co/inworld/sava/resolve/main/docs/assets/architecture-dark.svg">
    <img src="https://huggingface.co/inworld/sava/resolve/main/docs/assets/architecture-light.svg" alt="SAVA: causal log-mel features, frame mixer, Mamba and attention trunk, speech and turn heads" width="100%">
  </picture>
</p>

| part | detail |
|---|---|
| front end | causal log-mel, 128 Slaney mel bins of a 25 ms Hann DFT every 10 ms, log10 with a 1e-10 floor, `(x + 4) / 4`; four hops per frame plus the frame's log RMS |
| mixer | the current and previous frame's features, one linear projection to 512, RMSNorm |
| trunk | three Mamba-2 blocks (expand 2, head dim 64, state 128, conv kernel 4) and one attention block (8 query heads, 2 key/value heads, head dim 64, 512-frame causal window, rotary positions, RMS-normed q and k, exclusive self attention), each with a SwiGLU FFN of 2048 |
| voice-activity head | RMSNorm, down-projection to 512, one attention block (window 512, FFN 1536), RMSNorm, two linear readouts: on time, and the released three-frame look-ahead |
| end-of-turn head | the trunk output with a 64-wide embedding of the previous frame's on-time speech probability, the same block shape, a linear readout with four horizon columns (0, 200, 400, 800 ms); column 0 is the released output |

## Training data

SAVA was trained on approximately 5,000 hours of unique audio.
Inworld built training examples from real and synthetic conversational speech
mixed with non-verbal vocal sounds, noise, music and room acoustics, with a
speech label and an end-of-turn label for every frame. The training recipe,
code and data are not part of this release.

Third-party datasets used in training are credited in [NOTICE](NOTICE).

## License

Code and weights are Apache-2.0 ([LICENSE](LICENSE)).
Third-party notices are in [NOTICE](NOTICE).

## How to contribute

Everyone is welcome to use the model however they like and to make it better in any way they see fit!
Development happens on GitHub: see [CONTRIBUTING.md](https://github.com/inworld-ai/sava/blob/main/CONTRIBUTING.md) for setup, tests and bug reports.

---

<p align="center">Built for people with ❤️ by Inworld.</p>