src/docs/WHEEL.md

# The wheel: anuvani_diar as one self-contained manylinux package

`tools/build_wheel.sh` builds `anuvani_diar-<version>-cp310-abi3-manylinux_2_28_x86_64.whl`: one wheel for CPython
3.10 and later (abi3) on x86_64 Linux with glibc 2.28 or newer. It carries the Rust extension, the Python package, the
four native libraries the engine loads at run time, the licence texts and the clustering assets, so the CPU side
needs nothing from the build machine.

Release 1.0.0 was built on 10 October 2026 from commit 2ce08e60 (clean tree): 14,910,918 bytes, 245 files (189 of them
licence files in `.dist-info/licenses/`), sha256 `84d7b5e71eec083a3835f0cc3340f846f3e03562cf2ed6f054476f4107fdd3b1`;
the extension `_core.abi3.so` is 2,571,976 bytes after stripping. The first build (9 October, commit 7c648240, 36
files) was 13,657,026 bytes. Each build writes `<wheel>.build.json` (inputs, versions, checks) and each check
`<wheel>.check.json` beside the wheel.

## What is bundled

`anuvani_diar/_libs/`: unmodified copies of the binaries the release's own Python stack uses, sha256-checked when staged,
in the built wheel, and again by the engine before it loads them.

| File | What, from where | sha256 | Licence (text in `_libs/LICENSES/`) |
|---|---|---|---|
| `libopenblas64_p-r0-0cf96a72.3.23.dev.so` | OpenBLAS 0.3.23.dev, ILP64, with LAPACK; numpy 1.26.4 wheel (`numpy.libs/`) | `9254d085…` | BSD-3-Clause (LAPACK: BSD-3-Clause-Attribution); `numpy-1.26.4-LICENSE.txt` |
| `libgfortran-040039e1.so.5.0.0` | GCC Fortran runtime; same wheel | `14afb312…` | GPL-3.0 with the GCC Runtime Library Exception 3.1; same file (both full texts) |
| `libquadmath-96973f99.so.0.0.0` | GCC quad-precision maths; same wheel | `934c22de…` | LGPL-2.1-or-later; same file, full text in `libsndfile-1.2.2-COPYING.txt` |
| `libsndfile_x86_64.so` | libsndfile 1.2.2 with libogg 1.3, libvorbis 1.3.7, FLAC 1.4.3, opus 1.4, mpg123 1.32.3, LAME 3.100 linked in; soundfile 0.14.0 wheel (`_soundfile_data/`) | `c872fb84…` | LGPL-2.1 (`libsndfile-1.2.2-COPYING.txt`, soundfile's copy, identical to libsndfile's own); codecs: BSD-3-Clause (ogg, vorbis, FLAC, opus), LGPL-2.1 (mpg123), LGPL-2.0-or-later (LAME), one file each |

`_libs/LICENSES/README.txt` says which file covers which library and where each comes from. The numpy and soundfile
texts are copied from the reference environment; the codec texts (`tools/wheel_licences/`) were taken once from each
project's source at the version in the binary (FLAC, Vorbis, Opus and mpg123 report their versions in the binary;
LAME's is pinned by the libsndfile-binaries build script; libogg's COPYING is the same in 1.3.4, 1.3.5 and 1.3.6).

`anuvani_diar/assets/`: the PLDA and x-vector arrays (`*.bin`), `MANIFEST.json` (their sha256), `nn_golden.json`,
`long_golden.json`; about 270 KB.

**Staged, not committed.** Both folders are written by `tools/stage_wheel_files.py` at build time and are git-ignored.
The libraries are 41.7 MB of third-party binaries (and must not be in git); the assets already live in `assets/`, where
the Rust tests and the gateway read them, and a second committed copy could drift. The script copies the libraries from
the reference environment's site-packages and the codec licence texts from `tools/wheel_licences/`, checks every file
against `tools/wheel_files.json` (the committed sha256 manifest) and the arrays against `assets/MANIFEST.json`, and
leaves exactly the listed files. `make build`, `make test-rust` and `tools/build_wheel.sh` run it.

## How the engine finds them

`anuvani_diar/__init__.py` sets three variables to the package's own copies before `_core` loads, unless they are set
(non-empty) already:

| Variable | Default (set by the package) | Used by |
|---|---|---|
| `ANUVANI_DIAR_OPENBLAS_DIR` | `<package>/_libs` | `crates/diar-exact/src/blas.rs` |
| `ANUVANI_DIAR_SNDFILE` | `<package>/_libs/libsndfile_x86_64.so` | `crates/diar-audio/src/sndfile.rs` |
| `ANUVANI_DIAR_ASSETS` | `<package>/assets` | `crates/diar-core/src/assets.rs` |
| `ANUVANI_DIAR_SNDFILE_UNCHECKED` | unset | `1`: load the libsndfile named by `ANUVANI_DIAR_SNDFILE` without the sha256 and version checks |

No path is compiled into `_core`. Without the variables (`cargo test`, the gateway under `target/`, a source checkout)
the loaders fall back, for development only, to the source tree found at run time (from cargo's `CARGO_MANIFEST_DIR`,
the running executable, or the extension's own file): its staged `python/anuvani_diar/_libs` and its `assets/`.

**OpenBLAS.** The three files must have their sha256 before anything is loaded. OpenBLAS reads its settings from the
environment while it loads, so the loader sets `OPENBLAS_NUM_THREADS=1` and, if the CPU reports AVX2 and FMA,
`OPENBLAS_CORETYPE=Zen` just before the first `dlmopen`, restores both afterwards, and then requires 1 thread and core
"Zen". On a CPU without AVX2 or FMA the core is not forced and loading fails with "exact mode unavailable" (the Zen
kernels need both; other kernels would not reproduce the release bit for bit). The private namespace keeps its own
copy of the environment, so a later `setenv` elsewhere cannot leave it reading freed memory. If glibc has no static TLS
space left for the namespace ("cannot allocate memory in static TLS block"), the error says so and names the remedy:
start the process with `GLIBC_TUNABLES=glibc.rtld.optional_static_tls=4096`, or load the engine's OpenBLAS first
(`anuvani_diar._core.openblas_info()`). `_core.openblas_info()` reports the file, threads, core and whether the core
was forced; `_core.sndfile_info()` the libsndfile file, version and whether it passed the checks.

**libsndfile.** Checked by sha256 and version (`libsndfile-1.2.2`). The LGPL lets users replace it:
`ANUVANI_DIAR_SNDFILE=/path/to/libsndfile.so.1 ANUVANI_DIAR_SNDFILE_UNCHECKED=1` loads another build, any version,
and the engine says on stderr that decoding is no longer claimed to be identical to the release's.

## Building

```bash
# about 20 s once the cache is warm; the first build also fetches the toolchain and crates
tools/build_wheel.sh
```

Pinned: the image `quay.io/pypa/manylinux_2_28_x86_64@sha256:39df0042d5cc900b085aa25a0659368b42a0006c54c474299b785b44c1b4ff82`
(AlmaLinux 8, glibc 2.28; pulled 9 October 2026, created 3 October 2026), rustup-init 1.29.1 (by sha256), rustc 1.94.1
(minimal profile), maturin 1.15.0, the crates by `Cargo.lock` (`--locked`), and file times by `SOURCE_DATE_EPOCH`
(the commit's time). The build context is the tracked build files plus the staged ones, copied to
`artifacts/wheel_build/`; the target directory starts empty. maturin runs with `--strip`, every source path remapped
(`/build/anuvani_diar_rs` to `/anuvani_diar_rs`, the crate registry to `/cargo`), `--compatibility manylinux_2_28` and
`--auditwheel check` (it verifies the tag and changes nothing). `dlmopen` and `dlinfo` are declared with
`#[link(name = "dl")]` in `blas.rs`, so on glibc 2.28 they come from `libdl.so.2`; on newer glibc the link finds only
an empty `libdl.a` and nothing changes. Strip and remapping are set in the script, not in `pyproject.toml`, so that
`maturin develop` keeps its symbols. `pyproject.toml` names the staged folders in `include` (maturin leaves git-ignored
files out otherwise). Caches (rustup, cargo, maturin) stay in `artifacts/wheel_build/cache`.

**Reproducible:** two builds from empty target directories at the same commit gave the same wheel, byte for byte
(`acd8a022…` twice, 9 October 2026); a new commit changes only the file times.

**Checks made by the build** (all passed for the wheel above):

- the bundled files in the wheel have the sha256 of `tools/wheel_files.json` and `assets/MANIFEST.json`;
- `strings` on `_core.abi3.so`: no path of the build machine or of the build. What remains is the Rust toolchain's own
  (`/rustc/<hash>/library/...`, `/rust/deps/...`, `/checkout/src`), the remapped registry (`/cargo/registry/src/...`),
  the project's relative `crates/...` and system files the standard library reads (`/proc/self/...`, `/dev/urandom`);
- `_core` needs glibc symbols up to 2.28 and only `libdl.so.2`, `libgcc_s.so.1`, `libpthread.so.0`, `libm.so.6`,
  `libc.so.6`, `ld-linux-x86-64.so.2`;
- `auditwheel show` (never `repair`, which would rewrite the bundled libraries and break their sha256): the symbol
  versions allow `manylinux_2_28_x86_64`, but its overall verdict is `linux_x86_64`, for one reason: OpenBLAS needs
  `libgfortran-040039e1.so.5.0.0` and has no RPATH, so auditwheel cannot see that the file is in the wheel (the engine
  loads it first, by path, into the same namespace). Every other library the bundled files need is on the manylinux
  list (`libz.so.1`, `libmvec.so.1`, `libgcc_s.so.1`, glibc's). Making auditwheel agree would mean editing OpenBLAS.
  The SBOM maturin adds (`.dist-info/sboms/diar-py.cyclonedx.json`) names the build-container path
  `/build/anuvani_diar_rs`, not a host path.

## What was tested

`tools/wheel_check.py` (CPU only, `CUDA_VISIBLE_DEVICES` empty), on this wheel, 9 October 2026: **PASS in 32.6 s**
((a) and (b) side by side; 42.9 s on the first run, which also fetched Python 3.12 and torch).

(a) In `.venv` with the canonical environment, the existing tests that cover the bundled libraries and assets:
decoding against soundfile over every format and 2,750 damaged files, the release's `load_audio`, numpy's memory-error
text, the VBx row-parallel products against numpy's own (shapes up to 5,000 rows), the engine with every BLAS thread
variable at 64, and 40 cached records' RTTMs against the canonical oracle through `diarize_record` and through the
engine: **121 passed** (24 of the 20,774-row product shapes left to `make test-fast`). From a git worktree the main
checkout's untracked data are bound read-only over the worktree's in a private mount namespace.

(b) The wheel in a fresh venv: Python 3.12.11, torch 2.11.0+cpu, numpy 2.5.3, no variable of `env/canonical.sh` and
no `ANUVANI_DIAR_*` variable set; torch and numpy imported before `anuvani_diar`.

| Check | Result |
|---|---|
| private OpenBLAS | from the package's `_libs`; 1 thread, core Zen (forced), 0 threads started by the load (0.075 s) |
| libsndfile | from the package's `_libs`; `libsndfile-1.2.2`, checks passed |
| bengali_009.flac, telugu_020.flac (golden inputs, 3,200,512 samples each at 16 kHz) | sha256 of the decoded float32 samples identical to soundfile 0.14.0's in the reference environment |
| long_1h record, 2,241 chunks | RTTM sha256 `0f74418f…` = `assets/long_golden.json` (1.07 s) |
| long_3h record, 6,741 chunks | `982bbb38…` = golden (6.24 s) |
| long_6h record, 13,491 chunks | `651635aa…` = golden (19.30 s) |

Also checked while making the change (before and after it, on the build machine):

- the whole `make test-fast` from this worktree (data bound read-only as above), at commit 5ed8a2fd: 462 passed in
  7 min 23 s, the Rust tests included;
- the same test files without the 106 s channel-mean property test and the slow sets: 247 passed before the change
  and 247 after;
- `cargo test --workspace`: all pass, with new tests for the hash refusal (OpenBLAS and libsndfile), the static-TLS
  message, the core pin (forcing Haswell shows up as core "Haswell", so the pin reaches OpenBLAS), the namespace's own
  environment copy, and the source-tree fallback (also from a test binary run with an empty environment);
- loaded without any OpenBLAS variable set, the private OpenBLAS starts no thread and leaves both variables unset
  again; the same binary loaded without the pin (numpy importing it) starts 63 extra threads on this 64-CPU host;
- static TLS used up on purpose (`glibc.rtld.nns=1`, `optional_static_tls=0`, two libgomp copies loaded first): the
  readable error appears; with `optional_static_tls=4096` the same process loads;
- two gateway tests with the stand-in Triton (every RTTM against the release, the resample limit).

## Not covered here

- The networks: since 9 October 2026 the wheel declares torch, safetensors and huggingface_hub as well (the networks,
  their weights, the download).
- Licences: the code is MIT OR Apache-2.0 (decided 10 October 2026). The wheel carries `LICENSE-MIT`,
  `LICENSE-APACHE`, `THIRD_PARTY_NOTICES.md` and the licence files of every Rust crate compiled into `_core`
  (`LICENSES/crates/`, from `tools/third_party_notices.py`) in its `.dist-info/licenses/`, beside the bundled
  libraries' texts in `_libs/LICENSES/`. The source of the LGPL libraries is in the model repository's
  `third_party_sources/`.
- Other CPUs and older glibc: tested on one AVX2 CPU with glibc 2.35 only.