license: apache-2.0
library_name: transformers
base_model:
- nvidia/Nemotron-3-Diarization
- Wespeaker/wespeaker-voxceleb-resnet34-LM
tags:
- speaker-diarization
- streaming
- open-speaker
- clustering
- e-means
language:
- en
eDIAR: training-free open-speaker streaming diarization for long meetings and large archives
Training-free. This repository contains code only and stores no weights. It lifts the speaker limit of a fixed-slot diarizer (Nemotron-3 Diarization, 8 slots) by clustering speaker embeddings (WeSpeaker ResNet34-LM) inside a slot, and only after every slot has been used. The clustering engine is the official e-means algorithm. The streaming system re-clusters a bounded 90 s window every chunk, may revise labels for 10 s, and anchors permanent speaker ids to the slots of the head.
Demo: https://kyudan1.github.io/eDIAR/
Usage: streaming or non-streaming
One class, one switch. Everything else (models, clustering, speaker ids) is the same.
streaming=True (default) |
streaming=False |
|
|---|---|---|
| Audio | arrives in chunks, causal | whole recording at once |
| Latency | 1.04 s (latency="low_latency") or 0.64 s ("very_low_latency") |
none, runs after the recording |
| Labels | provisional, revised for 10 s, then fixed | final |
| Clustering | bounded 90 s window, re-run every chunk | all embeddings of the session at once |
| Use it for | live meetings, long recordings, bounded cost | files, highest accuracy |
| Paper setting | streaming tables | non-streaming tables |
import soundfile as sf
from emeans_diar import SlotEMeansDiarizer
wav, sr = sf.read("meeting.wav", dtype="float32") # mono
# 1) streaming (default): audio is fed in 100 ms chunks like a live system
dia = SlotEMeansDiarizer(device="cuda", streaming=True, latency="low_latency")
for start, end, speaker in dia.diarize(wav, sr):
print(f"{start:7.2f} {end:7.2f} spk{speaker}")
# 2) non-streaming: the whole file at once
dia = SlotEMeansDiarizer(device="cuda", streaming=False)
segments = dia.diarize(wav, sr)
# 3) real streaming loop (live audio): incremental interface
dia = SlotEMeansDiarizer(streaming=True)
session = dia.new_session()
for pcm in microphone_chunks(): # 16 kHz int16 bytes, any chunk size
msgs = session.feed(pcm) # provisional + revised labels
session.feed(None, flush=True)
Command line, RTTM on stdout:
python examples/diarize.py meeting.wav # streaming, 1.04 s
python examples/diarize.py meeting.wav --latency very_low_latency
python examples/diarize.py meeting.wav --offline # non-streaming
The two upstream models are downloaded from their own repositories on first use (HF_TOKEN if they are gated for you).
Install with pip install -r requirements.txt.
Results (streaming, 1.04 s input buffer, DER %, collar 0.25 s, overlap scored)
| System | VoxConverse | AMI | ICSI | LibriSpeech-OpenSpk |
|---|---|---|---|---|
| Streaming Sortformer 4spk v2.1 | 14.1 | 8.3 | 27.7 | 41.4 |
| Nemotron-3 head (8 slots) | 6.4 | 9.3 | 27.4 | 19.8 |
| This repository (after revision) | 5.8 | 9.1 | 27.4 | 16.4 |
Numbers are generated from the result files of the accompanying paper (cropped sessions of 7-8 minutes; 44 / 16 / 15 / 18 sessions).
What this gives you:
- No cap on the number of speakers. The 8-slot Nemotron-3 head becomes open-speaker without retraining; the paper shows the same for the 4-slot Streaming Sortformer.
- Robust on long meetings. Bounded-window re-clustering and slot-anchored ids keep the error flat over hour-long meetings.
- Linking speakers across recordings. Frame-level linking with e-means finds the same person across an archive of more than a thousand speakers, reaching the loss of Lloyd's k-means several times faster.
- Low and constant compute. The cost per update does not grow with the meeting (real-time factor about 0.09 on a shared CPU host).
Licenses
Code: Apache-2.0 (see LICENSE, NOTICE). Vendored e-means: MIT (Meta). The models it downloads keep their own licenses
(Nemotron-3 Diarization: openmdw-1.1; WeSpeaker ResNet34-LM: see its model card).