| --- |
| license: apache-2.0 |
| library_name: transformers |
| base_model: |
| - nvidia/Nemotron-3-Diarization |
| - Wespeaker/wespeaker-voxceleb-resnet34-LM |
| tags: |
| - speaker-diarization |
| - streaming |
| - open-speaker |
| - clustering |
| - e-means |
| language: |
| - en |
| --- |
| |
| # eDIAR: training-free open-speaker streaming diarization for long meetings and large archives |
|
|
| **Training-free.** This repository contains code only and stores no weights. It lifts the speaker limit of a fixed-slot |
| diarizer ([Nemotron-3 Diarization](https://huggingface.co/nvidia/Nemotron-3-Diarization), 8 slots) by clustering speaker |
| embeddings (WeSpeaker ResNet34-LM) *inside a slot*, and only after every slot has been used. The clustering engine is the |
| official [e-means](https://github.com/facebookresearch/faiss/tree/main/contrib/e_means) algorithm. |
| The streaming system re-clusters a bounded 90 s window every chunk, may revise labels for 10 s, and anchors permanent |
| speaker ids to the slots of the head. |
|
|
| Demo: https://kyudan1.github.io/eDIAR/ |
|
|
| ## Usage: streaming or non-streaming |
|
|
| One class, one switch. Everything else (models, clustering, speaker ids) is the same. |
|
|
| | | `streaming=True` (default) | `streaming=False` | |
| |---|---|---| |
| | Audio | arrives in chunks, causal | whole recording at once | |
| | Latency | 1.04 s (`latency="low_latency"`) or 0.64 s (`"very_low_latency"`) | none, runs after the recording | |
| | Labels | provisional, revised for 10 s, then fixed | final | |
| | Clustering | bounded 90 s window, re-run every chunk | all embeddings of the session at once | |
| | Use it for | live meetings, long recordings, bounded cost | files, highest accuracy | |
| | Paper setting | streaming tables | non-streaming tables | |
|
|
| ```python |
| import soundfile as sf |
| from emeans_diar import SlotEMeansDiarizer |
| |
| wav, sr = sf.read("meeting.wav", dtype="float32") # mono |
| |
| # 1) streaming (default): audio is fed in 100 ms chunks like a live system |
| dia = SlotEMeansDiarizer(device="cuda", streaming=True, latency="low_latency") |
| for start, end, speaker in dia.diarize(wav, sr): |
| print(f"{start:7.2f} {end:7.2f} spk{speaker}") |
| |
| # 2) non-streaming: the whole file at once |
| dia = SlotEMeansDiarizer(device="cuda", streaming=False) |
| segments = dia.diarize(wav, sr) |
| |
| # 3) real streaming loop (live audio): incremental interface |
| dia = SlotEMeansDiarizer(streaming=True) |
| session = dia.new_session() |
| for pcm in microphone_chunks(): # 16 kHz int16 bytes, any chunk size |
| msgs = session.feed(pcm) # provisional + revised labels |
| session.feed(None, flush=True) |
| ``` |
|
|
| Command line, RTTM on stdout: |
|
|
| ```bash |
| python examples/diarize.py meeting.wav # streaming, 1.04 s |
| python examples/diarize.py meeting.wav --latency very_low_latency |
| python examples/diarize.py meeting.wav --offline # non-streaming |
| ``` |
|
|
| The two upstream models are downloaded from their own repositories on first use (`HF_TOKEN` if they are gated for you). |
| Install with `pip install -r requirements.txt`. |
|
|
| ## Results (streaming, 1.04 s input buffer, DER %, collar 0.25 s, overlap scored) |
|
|
| | System | VoxConverse | AMI | ICSI | LibriSpeech-OpenSpk | |
| |---|---|---|---|---| |
| | Streaming Sortformer 4spk v2.1 | 14.1 | 8.3 | 27.7 | 41.4 | |
| | Nemotron-3 head (8 slots) | 6.4 | 9.3 | 27.4 | 19.8 | |
| | **This repository** (after revision) | 5.8 | 9.1 | 27.4 | 16.4 | |
|
|
| Numbers are generated from the result files of the accompanying paper (cropped sessions of 7-8 minutes; 44 / 16 / 15 / 18 sessions). |
|
|
| What this gives you: |
|
|
| - **No cap on the number of speakers.** The 8-slot Nemotron-3 head becomes open-speaker without retraining; the paper shows the same for the 4-slot Streaming Sortformer. |
| - **Robust on long meetings.** Bounded-window re-clustering and slot-anchored ids keep the error flat over hour-long meetings. |
| - **Linking speakers across recordings.** Frame-level linking with e-means finds the same person across an archive of more than a thousand speakers, reaching the loss of Lloyd's k-means several times faster. |
| - **Low and constant compute.** The cost per update does not grow with the meeting (real-time factor about 0.09 on a shared CPU host). |
|
|
| ## Licenses |
|
|
| Code: Apache-2.0 (see `LICENSE`, `NOTICE`). Vendored e-means: MIT (Meta). The models it downloads keep their own licenses |
| (Nemotron-3 Diarization: openmdw-1.1; WeSpeaker ResNet34-LM: see its model card). |
|
|
