File size: 24,291 Bytes
63dd946
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
---
license: cc-by-nc-4.0
base_model: BUT-FIT/diarizen-wavlm-large-s80-md-v2
pipeline_tag: voice-activity-detection
language:
- as
- bn
- brx
- doi
- en
- gu
- hi
- kn
- ks
- kok
- mai
- ml
- mni
- mr
- ne
- or
- pa
- sa
- sat
- sd
- ta
- te
- ur
tags:
- speaker-diarization
- audio
- indic
- diarizen
- wavlm
- speech
- speaker
- speaker-segmentation
- overlapped-speech-detection
- conformer
- wespeaker
- vbx
- pytorch
- rust
metrics:
- der
model-index:
- name: Anuvani-Diarization
  results:
  - task:
      name: Speaker Diarization
      type: speaker-diarization
    dataset:
      name: IndicDiarBench, all 1,164 files (zero-shot)
      type: sarvamai/indic-diarbench
      revision: 92877bad8aab6e598167d91c6ee02aa8ca6ede09
    metrics:
    - name: DER (micro, zero collar, overlap scored)
      type: der
      value: 17.11
    - name: DER (duration-weighted, zero collar, overlap scored)
      type: der
      value: 16.78
---

<!-- markdownlint-disable first-line-h1 -->
<!-- markdownlint-disable html -->
<!-- markdownlint-disable no-duplicate-header -->

<div align="center">
  <img class="dark:hidden" src="figures/hero_light.png" width="100%" alt="Anuvani-Diarization" />
  <img class="hidden dark:block" src="figures/hero_dark.png" width="100%" alt="Anuvani-Diarization" />
</div>
<hr>
<div align="center" style="line-height: 1;">
  <a href="https://www.ciil.org/" target="_blank" style="margin: 2px;">
    <img alt="Homepage" src="figures/badge.svg" style="display: inline-block; vertical-align: middle;"/>
  </a>
  <a href="https://colab.research.google.com/#fileId=https://huggingface.co/CIIL/Anuvani-Diarization/resolve/main/notebook.ipynb" target="_blank" style="margin: 2px;">
    <img alt="Open in Colab" src="https://img.shields.io/badge/Colab-Quick%20start-1a88ff?logo=googlecolab&logoColor=F9AB00&color=1a88ff" style="display: inline-block; vertical-align: middle;"/>
  </a>
  <a href="https://huggingface.co/CIIL" target="_blank" style="margin: 2px;">
    <img alt="Hugging Face" src="https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-CIIL-ffc107?color=ffc107&logoColor=white" style="display: inline-block; vertical-align: middle;"/>
  </a>
  <a href="https://arxiv.org/abs/2610.20327" target="_blank" style="margin: 2px;">
    <img alt="arXiv paper" src="https://img.shields.io/badge/arXiv-Paper-b31b1b?logo=arxiv&logoColor=white&color=b31b1b" style="display: inline-block; vertical-align: middle;"/>
  </a>
</div>

<div align="center" style="line-height: 1;">
  <a href="https://huggingface.co/CIIL/Anuvani-Diarization/blob/main/triton_inference/README.md" target="_blank" style="margin: 2px;">
    <img alt="Serving kit" src="https://img.shields.io/badge/Triton-Serving%20kit-76b900?logo=nvidia&logoColor=white&color=76b900" style="display: inline-block; vertical-align: middle;"/>
  </a>
</div>

<div align="center" style="line-height: 1;">
  <a href="#licence-and-attribution" style="margin: 2px;">
    <img alt="Code License" src="https://img.shields.io/badge/Code_License-MIT%20OR%20Apache--2.0-f5de53?&color=f5de53" style="display: inline-block; vertical-align: middle;"/>
  </a>
  <a href="#licence-and-attribution" style="margin: 2px;">
    <img alt="Model License" src="https://img.shields.io/badge/Model_License-CC%20BY--NC%204.0-f5de53?&color=f5de53" style="display: inline-block; vertical-align: middle;"/>
  </a>
</div>


<p align="center" style="text-wrap: balance;"><b>Who spoke when</b>, for multi-speaker audio in the 22 scheduled languages of India and English.</p>

Anuvani-Diarization finds who spoke when in recordings of conversations, calls and meetings: it splits the audio into
speaker turns and labels each turn with an anonymous speaker, in a standard RTTM file. It is built for the 22 scheduled
languages of India and English. On Indic DiarBench, the public benchmark for those languages, it has the lowest
error of every system we ran, evaluated zero-shot: nothing in it was trained, tuned or selected on the benchmark.

**Release 1.0.0** comes with:

* **`anuvani_diar`**, a pip package that runs the model in a few lines of Python on a Rust engine, on every GPU you
  give it (or the CPU).
* **`triton_inference/`**, a serving kit for many users: Triton Inference Server and a FastAPI service, on any number
  of NVIDIA GPUs on one machine or several.
* **The original Python program** (`reference/`), whose outputs the engine reproduces.

Key capabilities:

* **The lowest error of every system we ran on Indic DiarBench:** 16.78 DER (duration-weighted) and 17.11 (pooled) on
  all 1,164 recordings, zero-shot, against 17.18 to 39.41 for the nine other systems we ran; the most accurate
  measured system in 14 of the 22 languages.
* **Any length, any number of speakers:** from seconds to many hours; up to 4 people talking at once within each
  16-second window, and no limit on the total number of speakers.
* **Runs on what you have:** any NVIDIA GPU from the Volta generation to Blackwell, as many as you have, on one
  machine or several; the CPU works too, more slowly. By default it uses every GPU, and you can choose which.
* **Fast:** decoding, windowing, batching across GPUs and clustering run in Rust; only the two neural
  networks run in PyTorch.
* **Reproducible:** on the setup the release's outputs were produced with, the output is byte-identical to the
  original program; on any other GPU or PyTorch version every GPU checks its networks at start-up against the
  release's own outputs.
* **Open source code** under MIT OR Apache-2.0; the weights are for non-commercial use (CC BY-NC 4.0).

## Model overview

<div align="center">

| Property | Value |
|:---|:---|
| **Task** | Speaker diarization: who spoke when; one RTTM file per recording |
| **Languages** | Built and evaluated for the 22 scheduled languages of India and English |
| **Segmentation network** | DiariZen's pruned WavLM-Large plus Conformer end-to-end network, fine-tuned (69.5M parameters) |
| **Voiceprint network** | WeSpeaker ResNet34 (6.6M parameters) |
| **Clustering** | VBx clustering with the base model's PLDA and default settings (AHC threshold 0.6, Fa 0.07, Fb 0.8); nothing after the network is tuned |
| **Total parameters** | 76.1M |
| **Analysis window** | 16 seconds, one every 1.6 seconds; up to 4 overlapping speakers per window |
| **Input** | any audio file libsndfile reads (WAV, FLAC, OGG, MP3), any length; converted to 16 kHz mono |
| **Output** | RTTM (speaker label, start, duration); in Python also a list of (start, end, speaker) |
| **Hardware** | any NVIDIA GPU from Volta to Blackwell with about 12 GB of memory free, any number of them, on one machine or several; or the CPU |
| **Licence** | code MIT OR Apache-2.0; weights CC BY-NC 4.0 (segmentation) and CC BY 4.0 (PLDA, voiceprints) |
| **Paper** | [arXiv 2610.20327](https://arxiv.org/abs/2610.20327) · [Hugging Face paper page](https://huggingface.co/papers/2610.20327) |

</div>

## Benchmark results

All numbers are diarization error rates (DER, %): lower is better. Zero collar, overlap scored, NIST md-eval
semantics (identical to dscore `e02f949`).

### Indic DiarBench

All 1,164 recordings (107.7 hours, 22 Indian languages), dataset revision `92877bad`.

<div align="center">

| # | System | Maker | DER, duration-weighted | DER, pooled | Source |
|---:|:---|:---|---:|---:|:---|
| 1 | Sarvam AI | Sarvam AI | 16.0 | | paper |
| 2 | **Anuvani-Diarization 1.0** | **CIIL** | **16.78** | **17.11** | **this model, zero-shot** |
| 3 | Precision-2 | pyannoteAI | 17.18 | 17.58 | run by us |
| 4 | DiariZen WavLM-Large s80 md v2 (the base model) | BUT Speech@FIT | 17.99 | 18.31 | run by us |
| 5 | Nemotron 3 Diarization | NVIDIA | 18.38 | 18.71 | run by us |
| 6 | speaker-diarization-community-1 | pyannote | 21.22 | 21.65 | run by us |
| 7 | Streaming Sortformer 4spk v2.1 | NVIDIA | 22.14 | 22.34 | run by us |
| 8 | DiCoW v3.3 | BUT Speech@FIT | 22.62 | 22.76 | run by us |
| 9 | AWS Transcribe | Amazon | 23.5 | | paper |
| 10 | 3D-Speaker CAM++ diarization | Alibaba Tongyi | 23.67 | 24.39 | run by us; 2 recordings failed |
| 11 | Deepgram Nova-3 | Deepgram | 32.0 | | paper |
| 12 | MOSS-Transcribe-Diarize 0.9B | OpenMOSS | 33.76 | 34.52 | run by us; 14 recordings failed |
| 13 | Azure STT | Microsoft | 34.8 | | paper |
| 14 | ElevenLabs Scribe | ElevenLabs | 35.0 | | paper |
| 15 | GPT-4o Transcribe | OpenAI | 36.2 | | paper |
| 16 | Sortformer 4spk v1 | NVIDIA | 39.41 | 39.66 | run by us; 40 recordings failed |
| 17 | AssemblyAI Universal-2 | AssemblyAI | 40.5 | | paper |
| 18 | Gemini 3 Pro | Google | 74.0 | | paper |

</div>

- **Run by us:** every system on the same 1,164 recordings with one scorer: 0 s collar, overlap scored, no
  speaker-count hint; recordings a system failed on are scored as empty output.
- **Paper:** Indic DiarBench (Sarvam AI and AI4Bharat, arXiv 2607.23808), Table 3, duration-weighted, run as joint
  transcription and diarization services. The paper does not name Sarvam AI's model.
- **This model:** evaluated zero-shot. Nothing in it was trained, tuned or selected on Indic DiarBench: the
  segmentation network was adapted on other recordings, its recipe and checkpoint were fixed on non-benchmark
  recordings, and clustering uses the base model's default settings. The system was frozen before it was first
  scored on the benchmark. Earlier versions of this system were also scored on Indic DiarBench.
- **Pooled 95% confidence interval:** 15.80 to 18.45 (bootstrap over the 590 source recordings). The paired
  differences against the three strongest systems we ran all exclude zero: -0.47 points against Precision-2, -1.21
  against DiariZen md-v2 and -1.60 against Nemotron 3 Diarization.
- **Scorer choice changes the number.** pyannote.metrics reads about 0.20 higher on Indic DiarBench, because it does
  not merge the benchmark's overlapping same-speaker reference segments (13.7 minutes in 196 recordings).

### Public benchmarks

This model against its base model on six international test sets, both run by us under the same conditions with the
base model's clustering.

<div align="center">

| Test set | Files | DiariZen WavLM-Large s80 md v2 (the base model) | Anuvani-Diarization 1.0 |
|:---|---:|---:|---:|
| VoxConverse v0.3 test | 232 | **9.17** | 9.32 |
| NOTSOFAR-1 eval (one single-channel device per meeting) | 129 | 18.26 | **17.41** |
| MagicData-RAMC test | 43 | **10.97** | 11.13 |
| AMI test, single distant mic | 16 | **13.93** | 14.00 |
| AISHELL-4 test | 20 | 10.06 | **9.85** |
| AliMeeting test, far-field | 20 | 10.74 | **10.44** |
| **Mean** | | 12.19 | **12.03** |

</div>

The Indic adaptation costs no accuracy on these sets on average: better on three, worse on three.

### This model on every benchmark

<div align="center">

| Benchmark | DER |
|:---|---:|
| Indic DiarBench, all 1,164 recordings, duration-weighted | **16.78** |
| Indic DiarBench, all 1,164 recordings, pooled | **17.11** |
| Indic DiarBench, near-field meetings, pooled | 19.05 |
| Indic DiarBench, far-field meetings, pooled | 18.64 |
| Indic DiarBench, in-the-wild video, pooled | 11.51 |
| VoxConverse v0.3 test | 9.32 |
| AISHELL-4 test | 9.85 |
| AliMeeting test, far-field | 10.44 |
| MagicData-RAMC test | 11.13 |
| AMI test, single distant mic | 14.00 |
| NOTSOFAR-1 eval (one single-channel device per meeting) | 17.41 |
| Average of the six public sets | 12.03 |

</div>

## Getting started

<a href="https://colab.research.google.com/#fileId=https://huggingface.co/CIIL/Anuvani-Diarization/resolve/main/notebook.ipynb" target="_blank"><img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open in Colab"></a>
opens the quick-start notebook (`notebook.ipynb`) in Google Colab: choose a GPU runtime and run every cell.

On your own machine you need Linux x86_64 with glibc 2.28 or newer, Python 3.10 or newer and PyTorch 2.1 or newer; an
NVIDIA GPU is strongly recommended (the CPU works, much more slowly).

```bash
pip install -U huggingface_hub
hf download CIIL/Anuvani-Diarization wheels/anuvani_diar-1.0.0-cp310-abi3-manylinux_2_28_x86_64.whl --local-dir .
pip install wheels/anuvani_diar-1.0.0-cp310-abi3-manylinux_2_28_x86_64.whl
```

```python
from anuvani_diar import Diarizer

# every visible GPU, or the CPU when there is none
diarize = Diarizer()
# any length; WAV, FLAC, OGG, MP3
result = diarize("meeting.wav")
result.save_rttm("meeting.rttm")
# [(start, end, speaker), ...]
print(result.speakers, result.segments[:3])

# many files; results arrive as each one finishes
for r in diarize.run(["a.wav", "b.flac"], out_dir="rttm/"):
    print(r.uri, r.ok)
```

Choosing devices: `Diarizer(devices=2)` uses the first two GPUs, `Diarizer(devices=[0, 3])` GPUs 0 and 3,
`Diarizer(devices="cpu")` the CPU. The engine spreads every file's windows over all of them.

- **Command line:** `anuvani-diar --out rttm/ meeting.wav`, or `--in-dir recordings/`; `--devices 2`, `--devices 0,3`
  or `--devices cpu` choose the devices.
- **What it downloads:** the first `Diarizer()` fetches `model/` (307 MB) once from this repository, at the release's
  tag, and checks every file's sha256 against the package's own list. No account is needed. It also fetches the small
  `config.json`, which the Hub counts as a download of the model; nothing else is sent.
- **Install check:** `anuvani-diar-check` diarizes the 12 check recordings and compares the results with the
  release's fingerprints.
- **PyTorch:** installing the wheel into an environment without PyTorch pulls PyPI's default build; for PyTorch 2.14.1
  that is a CUDA 13.0 build, which needs a recent NVIDIA driver. With an older driver, install a CUDA 12 build first,
  e.g. `pip install torch --index-url https://download.pytorch.org/whl/cu126`. Google Colab already has PyTorch.

## Serving at scale

`triton_inference/` deploys the model for many users with Docker Compose:
- one Triton Inference Server per GPU holds the two networks;
- a FastAPI service holds the Rust engine, and it is the only thing clients talk to (uploads, job ids, RTTMs, a
  token, limits, metrics);
- it uses every GPU of the machine by default (`./diar up 2` or `./diar up --gpus 0,3` to choose), and the GPUs of other
  machines too (`./diar up --tritons-only` on those, `./diar up --remote ...` on the service's machine).

```bash
cd triton_inference && chmod +x diar
# the Triton image this machine's GPUs need, the service, every GPU
./diar doctor && ./diar build && ./diar up
# the 12 check recordings through the service
./diar gate
curl -s -H "Authorization: Bearer $(./diar token)" --data-binary @meeting.wav "localhost:8180/v1/diarize?name=meeting.wav"
```

The kit's default lane runs part of the networks in FP16 and TF32. It is not byte-identical to the release, and its
output can differ slightly from run to run. `./diar up --exact` runs the exact lane.
Details: `triton_inference/README.md`.

## Best practices

### 1. Devices

Leave `devices` on `"auto"` to use every GPU, or pass a number or a list to share a machine. One process per machine
is best: the engine already keeps every GPU busy and spreads each file over all of them.

### 2. Many files

Give `run()` all the files at once rather than calling the model once per file: the engine decodes, schedules and
clusters them in parallel, and results arrive as each file finishes.

### 3. Long recordings

Any length works, from seconds to many hours. A single long file finishes faster with more GPUs, because its windows
spread over all of them; decoding and the clustering run on the CPU before and after the networks.

### 4. Speed and exactness

The pip package runs the exact lane (the release's FP32 networks). The serving kit defaults to the faster ffn16
lane, which serves about 2.1 times as much audio per second as the exact lane on the same GPUs; `--exact` keeps the
exact lane.

### 5. Audio formats

WAV, FLAC, OGG and MP3 are read directly; convert M4A, AAC and MP4 audio to WAV or FLAC first.

### 6. Checking an installation

`anuvani-diar-check` (and `./diar gate` in the serving kit) diarize the 12 check recordings. Identical results
confirm the setup the release's outputs were produced with; anywhere else the networks pass a near-identical check
and the results may differ slightly.

## Model data

### Training data

- **Segmentation network:** DiariZen's WavLM-Large and Conformer network (`BUT-FIT/diarizen-wavlm-large-s80-md-v2`),
  fine-tuned on Indian telephone conversations (2-person calls in 8 languages: as, bn, gu, hi, ml, mr, pa, ta) and
  international meeting corpora.
- **Voiceprint network:** `pyannote/wespeaker-voxceleb-resnet34-LM`, unchanged.
- **PLDA model and clustering settings:** BUT Speech@FIT's, unchanged.
- **Indic DiarBench:** not used for training, tuning or selection; audio-fingerprint screening found no audio shared
  between the training data and the benchmark.

### Evaluation data

Indic DiarBench (Sarvam AI and AI4Bharat): 1,164 recordings, 107.7 hours, in 22 Indian languages and three recording
conditions, at the pinned dataset revision `92877bad`. The public sets are listed in the tables above.

## Exactness and determinism

- **FP32, TF32 off, deterministic kernels,** seed 3407 reset per file.
- **Byte-identical to the research pipeline** the published numbers come from.
- **The package's default (exact) lane and the kit's exact lane** give RTTMs byte-identical to the original program
  on the setup the release's outputs were produced with (PyTorch 2.1.1 with CUDA 12.1): all 1,945 benchmark files from
  cached outputs, and 1,973 audio inputs, including 1-, 3- and 6-hour files and damaged files, both locally and
  through Triton.
- **Other setups** (other GPUs, other PyTorch versions) run the same FP32 networks with the same settings, but
  cannot be byte-identical. At start-up, each GPU's outputs are checked against the release's own and must agree
  within fixed bounds; the package then warns that its output is near-identical, and `Diarizer(strict=True)` refuses
  instead. On other GPU models, rare near-ties in clustering could flip a file; the install checks show whether a
  setup reproduces the release byte for byte.

## Usage and limitations

### Intended usage

- **Conversation and meeting analysis:** speaker turns for calls, interviews, panels, classrooms and meetings in
  Indian languages and English.
- **Transcription pipelines:** attributing an ASR transcript's words to speakers, by aligning them with the RTTM.
- **Media and archives:** indexing radio, television and oral-history recordings by speaker turn.
- **Research:** a strong, reproducible baseline for speaker diarization in Indian languages.

### Limitations

- **Overlap cap:** at most 4 people talking at the same moment within a 16-second window. There is no limit on the
  total number of speakers.
- **Indian fine-tuning data is narrow:** 2-person telephone calls in 8 languages (as, bn, gu, hi, ml, mr, pa, ta). There
  are no Indian multi-person meetings, and 14 of Indic DiarBench's 22 languages are unseen in fine-tuning.
- **Languages and conditions:** it trails the best other system we ran most in Maithili (+1.98 points), Sindhi (+1.80)
  and Bodo (+1.56), and on in-the-wild video, where Precision-2 leads (10.78 against 11.51 pooled).
- **Annotation convention:** Indic DiarBench's references mark whole utterances, while this model, like other
  diarizers, splits speech at pauses; part of its error on the benchmark comes from that difference.
- **Speaker identity** inside overlaps and very short turns is the largest remaining error source.
- **Single training seed.**
- **Evaluation history:** earlier versions of this system were also scored on Indic DiarBench.
- **Formats:** M4A/AAC/MP4 audio is not decoded; convert it to WAV or FLAC first.

### Ethical considerations and risks

- **Privacy and consent:** diarization tells apart the voices in a recording. Use it on recordings whose speakers have
  agreed to it, and follow the data protection law that applies to you.
- **No identities:** the speaker labels are anonymous clusters within one recording; the model does not identify
  people and does not link speakers across recordings. Using its output to identify or track people needs separate
  consent and a separate, appropriately evaluated system.
- **Uneven accuracy:** error rates vary with language, recording condition and the number of people talking; check
  the results on your own audio before relying on them.
- **Licence:** the weights are for non-commercial use only (CC BY-NC 4.0, inherited from DiariZen).

### Benefits

At release, the most accurate openly available speaker diarization for Indian languages on Indic DiarBench, with
reproducible results, open source code and a deployment that scales from one GPU to many machines.

## Repository layout

<div align="center">

| Path | What |
|:---|:---|
| `model/` | the weights (safetensors) and `MANIFEST.json` (every file's sha256) |
| `model.safetensors.index.json` | the index of the tensors in `model/`'s two safetensors files (the Hub reads the parameter count from it) |
| `config.json` | the release's description (version, wheel, window, parameters, licences); the package fetches it with `model/` |
| `wheels/` | the pip package (`anuvani_diar-1.0.0-cp310-abi3-manylinux_2_28_x86_64.whl`) |
| `notebook.ipynb`, `notebooks/quickstart.ipynb` | install and diarize in a notebook (the same notebook; the first opens in Colab from this page) |
| `triton_inference/` | the serving kit: the `diar` command, the FastAPI service, the Triton images, Docker Compose, checks, a load test |
| `golden/audio/` | the 12 Indic DiarBench recordings of the install checks |
| `reference/` | the original Python program (`diarize.py`), its pickled weights and its install check |
| `src/` | the source of the pip package and the serving kit (Rust and Python), its tests and developer documents |
| `CHANGELOG.md` | release notes |
| `LICENSE-MIT`, `LICENSE-APACHE` | the code's licence (MIT OR Apache-2.0) |
| `THIRD_PARTY_NOTICES.md`, `LICENSES/` | third-party code, bundled libraries and Rust crates, with their licence texts |
| `third_party_sources/` | the source of the LGPL libraries the wheel bundles, and the script that built libsndfile |

</div>

## Licence and attribution

- **Segmentation network:** a fine-tuned derivative of `BUT-FIT/diarizen-wavlm-large-s80-md-v2`, under **CC BY-NC 4.0
  (non-commercial)**. DiariZen's weights carry that licence because some of their training data is research-only.
- **PLDA model:** BUT Speech@FIT, CC BY 4.0.
- **Speaker-embedding model:** `pyannote/wespeaker-voxceleb-resnet34-LM`, CC BY 4.0.
- **Indic DiarBench audio** (`golden/audio`): derived from Sarvam AI's Indic DiarBench
  (CC BY 4.0). Cite Mehendale et al., *Indic DiarBench*, Interspeech 2026 (arXiv 2607.23808).
- **DiariZen:** cite Han et al., ICASSP 2025, and arXiv 2505.24111 / 2506.18623.
- **Code:** MIT OR Apache-2.0, at your option ([`LICENSE-MIT`](LICENSE-MIT), [`LICENSE-APACHE`](LICENSE-APACHE)).
  - It contains or ports code from DiariZen, torchaudio, pyannote.audio, WeSpeaker, VBx, SciPy, NumPy and
    python-soundfile.
  - The wheel bundles numpy 1.26.4's OpenBLAS (BSD-3) with libgfortran (GPL-3 with the GCC Runtime Library
    Exception) and libquadmath (LGPL-2.1+), and libsndfile 1.2.2 (LGPL-2.1) with its codecs.
  - Their licences and notices are in `THIRD_PARTY_NOTICES.md` and `LICENSES/`, and in the wheel.
  - The source of the LGPL ones (libsndfile with its codecs, libquadmath) is in `third_party_sources/`.

## Acknowledgement

Anuvani-Diarization was conceived and developed by Stephen Fernandes under the leadership and guidance of Dr. Narayan Kumar Choudhary. We
thank Dr. Choudhary and the Central Institute of Indian Languages (CIIL) for their assistance, for the compute they
provided and for their continued support.

## Citation

If you use this model, please cite the paper:

```bibtex
@misc{fernandes2026anuvanidiarization,
  title={Anuvani-Diarization: Zero Shot Speaker Diarization and Annotation Conventions Across 22 Languages},
  author={Stephen Fernandes and Narayan Choudhary},
  year={2026},
  eprint={2610.20327},
  archivePrefix={arXiv},
  primaryClass={cs.CL},
  url={https://arxiv.org/abs/2610.20327}
}
```