File size: 4,366 Bytes
1c68fa2 35dcbf1 1c68fa2 30d0277 1c68fa2 30d0277 1c68fa2 30d0277 1c68fa2 30d0277 1c68fa2 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 | ---
license: apache-2.0
library_name: transformers
base_model:
- nvidia/Nemotron-3-Diarization
- Wespeaker/wespeaker-voxceleb-resnet34-LM
tags:
- speaker-diarization
- streaming
- open-speaker
- clustering
- e-means
language:
- en
---
# eDIAR: training-free open-speaker streaming diarization for long meetings and large archives
**Training-free.** This repository contains code only and stores no weights. It lifts the speaker limit of a fixed-slot
diarizer ([Nemotron-3 Diarization](https://huggingface.co/nvidia/Nemotron-3-Diarization), 8 slots) by clustering speaker
embeddings (WeSpeaker ResNet34-LM) *inside a slot*, and only after every slot has been used. The clustering engine is the
official [e-means](https://github.com/facebookresearch/faiss/tree/main/contrib/e_means) algorithm.
The streaming system re-clusters a bounded 90 s window every chunk, may revise labels for 10 s, and anchors permanent
speaker ids to the slots of the head.
Demo: https://kyudan1.github.io/eDIAR/
## Usage: streaming or non-streaming
One class, one switch. Everything else (models, clustering, speaker ids) is the same.
| | `streaming=True` (default) | `streaming=False` |
|---|---|---|
| Audio | arrives in chunks, causal | whole recording at once |
| Latency | 1.04 s (`latency="low_latency"`) or 0.64 s (`"very_low_latency"`) | none, runs after the recording |
| Labels | provisional, revised for 10 s, then fixed | final |
| Clustering | bounded 90 s window, re-run every chunk | all embeddings of the session at once |
| Use it for | live meetings, long recordings, bounded cost | files, highest accuracy |
| Paper setting | streaming tables | non-streaming tables |
```python
import soundfile as sf
from emeans_diar import SlotEMeansDiarizer
wav, sr = sf.read("meeting.wav", dtype="float32") # mono
# 1) streaming (default): audio is fed in 100 ms chunks like a live system
dia = SlotEMeansDiarizer(device="cuda", streaming=True, latency="low_latency")
for start, end, speaker in dia.diarize(wav, sr):
print(f"{start:7.2f} {end:7.2f} spk{speaker}")
# 2) non-streaming: the whole file at once
dia = SlotEMeansDiarizer(device="cuda", streaming=False)
segments = dia.diarize(wav, sr)
# 3) real streaming loop (live audio): incremental interface
dia = SlotEMeansDiarizer(streaming=True)
session = dia.new_session()
for pcm in microphone_chunks(): # 16 kHz int16 bytes, any chunk size
msgs = session.feed(pcm) # provisional + revised labels
session.feed(None, flush=True)
```
Command line, RTTM on stdout:
```bash
python examples/diarize.py meeting.wav # streaming, 1.04 s
python examples/diarize.py meeting.wav --latency very_low_latency
python examples/diarize.py meeting.wav --offline # non-streaming
```
The two upstream models are downloaded from their own repositories on first use (`HF_TOKEN` if they are gated for you).
Install with `pip install -r requirements.txt`.
## Results (streaming, 1.04 s input buffer, DER %, collar 0.25 s, overlap scored)
| System | VoxConverse | AMI | ICSI | LibriSpeech-OpenSpk |
|---|---|---|---|---|
| Streaming Sortformer 4spk v2.1 | 14.1 | 8.3 | 27.7 | 41.4 |
| Nemotron-3 head (8 slots) | 6.4 | 9.3 | 27.4 | 19.8 |
| **This repository** (after revision) | 5.8 | 9.1 | 27.4 | 16.4 |
Numbers are generated from the result files of the accompanying paper (cropped sessions of 7-8 minutes; 44 / 16 / 15 / 18 sessions).
What this gives you:
- **No cap on the number of speakers.** The 8-slot Nemotron-3 head becomes open-speaker without retraining; the paper shows the same for the 4-slot Streaming Sortformer.
- **Robust on long meetings.** Bounded-window re-clustering and slot-anchored ids keep the error flat over hour-long meetings.
- **Linking speakers across recordings.** Frame-level linking with e-means finds the same person across an archive of more than a thousand speakers, reaching the loss of Lloyd's k-means several times faster.
- **Low and constant compute.** The cost per update does not grow with the meeting (real-time factor about 0.09 on a shared CPU host).
## Licenses
Code: Apache-2.0 (see `LICENSE`, `NOTICE`). Vendored e-means: MIT (Meta). The models it downloads keep their own licenses
(Nemotron-3 Diarization: openmdw-1.1; WeSpeaker ResNet34-LM: see its model card).
|