|
Download src/docs/DETERMINISM.md from CIIL/Anuvani-Diarization: direct link, hf CLI and curl.
- Browser
- Download file 4.84 kB
-
https://huggingface.co/CIIL/Anuvani-Diarization/resolve/main/src/docs/DETERMINISM.md
- Command line
-
hf download hf://CIIL/Anuvani-Diarization/src/docs/DETERMINISM.md
-
curl -L -o DETERMINISM.md https://huggingface.co/CIIL/Anuvani-Diarization/resolve/main/src/docs/DETERMINISM.md
4.84 kB
Determinism: why the outputs are byte-identical, and where that stops
The engine reproduces the release's arithmetic, not just its results. Each piece below is pinned for a measured
reason; docs/TESTING.md lists the checks that hold it in place, and tests/test_matrix.py runs the same 39 files
under changing conditions (each GPU alone and all of them together, 1 to 64 threads, 1 to 64 files at once, other arrival orders,
a warm repeat, a fresh process with every BLAS thread variable at 64): every record and RTTM identical in all of them.
The networks (GPU)
- Same code and weights. The release's own model classes (DiariZen and its pyannote fork at commit 844f555), the
bundle's weights, built the way
load_bundlebuilds them. - Same numeric flags, set after the imports. FP32, TF32 off, deterministic algorithms, cuDNN benchmark off,
CUBLAS_WORKSPACE_CONFIG=:4096:8. Importing DiariZen switches TF32 on for the whole process, so the flags are set after it, asload_bundledoes, and re-asserted before every network call. - Same kernels. torch 2.1.1+cu121, its cuDNN 8.9.2, and cuBLAS 12.1.0.26 (
env/canonical.shloads it from a CUDA 12.1 installation; the cu121 Triton image carries the same files and searches them first). - Same batch shapes. Segmentation: batches of 8, the remainder, the zero-padded last window alone (other sizes change bits). Voiceprints: the trunk once per window in batches of 8, pooling and the last layer at the release's rows (identical bits), the odd last window as the release runs it.
- One CUDA stream per GPU. Every GPU of a kind gives the same bits as the others, but two streams sharing one model on one GPU changed about 1 batch in 1,000. Locally there is one worker thread per GPU; on Triton one process per GPU.
- Models never cross GPUs. Moving several model copies from GPU to GPU in one process gave all-NaN weights on one GPU; copies go through host memory, and every Triton instance sees only its own GPU.
- Checked before use. Every backend runs every batch shape at start-up and compares with the release's own
outputs (
assets/nn_golden.json, written fromload_bundle): identical bytes where the setup is reproduced; otherwise the outputs must stay within fixed near-identical bounds ofassets/nn_reference.npz(or, in strict mode, the backend stops).
Everything else (CPU, Rust)
- numpy, scipy, pyannote and CPython reproduced operation by operation (crate
diar-exact): pairwise summation with numpy's 8 accumulators and 8,192-element buffers, numpy's BLAS dispatch (gemm / gemv / dot) with numpy's own OpenBLAS binary at one thread, glibc'sexp/log, introsort tie order, scipy's centroid linkage and heap, cosine distance with its two-lane sum, rectangular assignment, pyannote's sliding-window arithmetic and aggregation, CPython'sround,sum, string sort and.3fformatting. - Parallel only where each element's arithmetic is unchanged. Distances by row, VBx products by blocks of result rows (each its own single-threaded OpenBLAS call; tested bitwise against the one-call product), VBx and aggregation by rows or output frames with every element's terms in the release's order.
- The engine's OpenBLAS is private (its own linker namespace) and pinned to one thread whatever the environment says.
- Decoding is libsndfile 1.2.2, the binary soundfile loads, called in soundfile's order; non-16 kHz audio is
resampled by the release's own torchaudio resampler (copied unchanged into
anuvani_diar.nn.resample) with 2 threads.
Where the guarantee stops
- Identical bytes need the same software (
env/canonical.sh) on the same kind of GPU as the release's outputs were produced with. Another GPU model, or another PyTorch, CUDA or cuDNN version, picks other kernels whose last bits differ: the start-up check then measures the networks against the release's outputs and runs only if they stay within the near-identical bounds (maximum posterior difference 0.25, frame decisions at least 99% equal, voiceprint cosine at least 0.995), and says so; strict mode refuses to run instead. - The CPU side is pinned to AVX2 code paths by
env/canonical.sh(numpy, ATen and oneDNN features, andOPENBLAS_CORETYPE=Zen). The engine's private OpenBLAS no longer depends on that variable: it setsOPENBLAS_CORETYPE=Zen(and one thread) itself for its own load when the CPU reports AVX2 and FMA, checks that it runs core Zen, and refuses to run otherwise (docs/WHEEL.md). On an AVX2 CPU the pins change nothing; on a CPU with AVX-512, running without them could select other kernels in numpy, ATen and oneDNN. - Inputs the release itself would treat differently (a different bundle, batch size or package versions) are outside the contract.