# anuvani_diar: the source of the package and of the serving kit The source of Anuvani-Diarization 1.0's Python package `anuvani_diar` and of its serving kit (`triton_inference/`). Everything except the two networks runs in Rust: decoding, windowing, batching across every GPU, masks, clustering, the decoder and the RTTM text; the networks run the release's own PyTorch code, in your Python process or on Triton, on any NVIDIA GPU (or the CPU), as many as you give it. The output is the release's original Python program's (`reference/diarize.py` in the model repository): byte-identical on the setup the release's outputs were produced with, near-identical, checked at start-up, on any other GPU or PyTorch version. To use the model, install the wheel or run the serving kit (see the model card). Build from here to change the code. ## Building The release wheel (CPython 3.10 and newer, manylinux_2_28, x86_64), built in a pinned container (needs Docker): ```bash # artifacts/wheels/anuvani_diar--cp310-abi3-manylinux_2_28_x86_64.whl tools/build_wheel.sh --site-packages SITE ``` The wheel bundles numpy 1.26.4's OpenBLAS and soundfile 0.14.0's libsndfile, which `tools/stage_wheel_files.py` copies from SITE, the site-packages folder of any Python environment with `numpy==1.26.4` and `soundfile==0.14.0` installed, and checks by sha256 (`docs/WHEEL.md`). For development: a Rust toolchain (1.80 or newer), maturin 1.15 and the reference Python environment (Python 3.10, PyTorch 2.1.1, the packages of `env/pip_freeze_reference.txt` in the model repository's `reference/`): ```bash # stages the bundled libraries and assets, builds the extension into .venv (maturin) make build source env/canonical.sh && source .venv/bin/activate ``` The test suites (`make test-fast`, `test-cache`, `test-gpu`, `test-gpu-all`) compare every output with the release's own, regenerated by `tools/oracle_from_audio.py` and `tools/oracle_from_cache.py` from the reference program; that generated data is not part of the repository (`docs/TESTING.md`). ## Using it ```python from anuvani_diar import Diarizer # every visible GPU (the CPU if there is none) d = Diarizer() d("meeting.wav").save_rttm("meeting.rttm") # many files, results as each one finishes for r in d.run(["a.wav", "b.flac"], out_dir="rttm/"): print(r.uri, r.speakers, r.timing["wall_s"]) ``` `Diarizer(devices=2)` uses the first two GPUs, `Diarizer(devices=[0, 3])` GPUs 0 and 3, `Diarizer(devices="cpu")` the CPU; the default, `"auto"`, is every visible GPU. Command line, with the reference program's flags (there is one model, so no `--mode`): ```bash anuvani-diar --out OUT a.wav b.flac anuvani-diar --out OUT --in-dir AUDIO_DIR # the first two GPUs; --devices 0,3 or --devices cpu also work anuvani-diar --out OUT --devices 2 a.wav ``` Checking an installation (the release's install check; `--long-dir` adds the 1 h, 3 h and 6 h files that `tools/make_test_audio.py` makes): ```bash # downloads the 12 check recordings from the model repository anuvani-diar-check anuvani-diar-check --audio-dir AUDIO_DIR --long-dir LONG_DIR ``` It prints PASS or DIFFERS per output; before that, every GPU checks both networks against the release's own outputs. Exit status 0 means every RTTM is identical to the release's; 3 means a setup where the networks passed the near-identical check, so identical bytes are not expected. Serving many users: the kit in `triton_inference/` (Triton, the FastAPI service on this engine, Docker Compose, any number of GPUs on one or several machines; its README). `triton/` holds the two Triton images and the model files the kit builds on, and the Rust gateway (`crates/diar-gateway`) is the reference implementation of the same API that the serving identity checks and measurements ran through. ## What is guaranteed The RTTMs (and the front-end records behind them) are byte-identical to the release's on everything tested: all 1,945 benchmark files from cached records, all 1,973 audio inputs including 1 h / 3 h / 6 h files and 25 robustness files through the local engine on several GPUs at once, and all 1,973 again through Triton, the gateway and the client with concurrent shuffled jobs, also with another GPU assignment and with Triton killed and restarted in the middle of the run. `docs/TESTING.md` lists the checks. The conditions: the setup the release's outputs were produced with (`env/canonical.sh`: PyTorch 2.1.1 with CUDA 12.1 and cuBLAS 12.1.0.26, on the same kind of GPU), one network stream per GPU. Every network backend checks itself at start-up against the release's own outputs (`assets/nn_golden.json`, `assets/nn_reference.npz`): where those bytes can be reproduced it gets identical ones; on any other GPU or PyTorch version the networks must agree within fixed near-identical bounds, and the package says that its output is near-identical, not identical. Strict mode (`strict=True`, the serving kit's `--strict`) refuses to run unless every batch shape gives identical bytes. ## Where things are | Path | What | |---|---| | `crates/diar-exact` | numpy / scipy / pyannote / CPython numerics reproduced exactly | | `crates/diar-core` | the front-end around the networks, clustering, the decoder, the RTTM writer | | `crates/diar-audio` | libsndfile decoding as soundfile does it | | `crates/diar-engine` | the engine: pools, blocks, scheduling over GPUs, limits (`docs/ENGINE.md`) | | `crates/diar-triton`, `crates/diar-gateway` | the Triton backend and the HTTP gateway | | `crates/diar-py`, `python/anuvani_diar` | the Python API, CLI, client and the network backend | | `triton/` | the two Triton images (cu121, cu128), the model files, the scripts of the gateway stack | | `triton_inference/` | the serving kit: `diar`, the FastAPI service, Docker Compose, checks, load test, reference library | | `tools/` | wheel build, staging, licence headers and notices, the oracles, probes | | `tests/` | the suites (`make test-fast`, `test-cache`, `test-gpu`, `test-gpu-all`, `triton/identity_check.sh`) | | `docs/` | ARCHITECTURE (start here), ENGINE, DETERMINISM, TESTING, SECURITY, WHEEL | ## Licence The code is licensed under MIT OR Apache-2.0, at your option (`LICENSE-MIT`, `LICENSE-APACHE`). Code it copies or ports from other projects (DiariZen, torchaudio, pyannote.audio, WeSpeaker, VBx, SciPy, NumPy, python-soundfile), the native libraries the wheel bundles (OpenBLAS, libsndfile and their runtime libraries) and the Rust crates it compiles are listed with their licences in `THIRD_PARTY_NOTICES.md`; the texts are in `LICENSES/`. The model weights are not covered by the code's licence: the segmentation network is CC BY-NC 4.0 (non-commercial; a fine-tuned derivative of DiariZen's weights), the PLDA model and the speaker-embedding network CC BY 4.0.