Validation

The catalog includes an individual JSON report for each of its seven models, linked below. The reports record input and output shapes, inference settings, output peaks, file-writing results, and conversion comparisons where available.

Catalog checks

All seven models passed a fresh-process inference check on the same 14-second stereo music excerpt at 44.1 kHz, with overlapping chunks. Each produced the expected number of finite, full-length stereo stems and wrote its output audio successfully. Network access was disabled, and PyTorch, ONNX, and ONNX Runtime imports were blocked during inference.

Model	Stems checked	Additional verification	Report
BS-RoFormer-SW	6	Bit-identical unquantized output versus the original checkpoint in MLX	JSON
Mel-RoFormer InstV7n	2	Bit-identical unquantized output versus the original checkpoint in MLX	JSON
UVR-MDX-NET Inst HQ3	2	Bit-identical unquantized output versus the original checkpoint in MLX	JSON
HTDemucs	4	Weights and config match the pinned published files	JSON
HTDemucs_FT	4	Weights and config match the pinned published files	JSON
HTDemucs_6s	6	Weights and config match the pinned published files	JSON
ZFTurbo Vocals v1	2	Weights and config match the pinned published files	JSON

The three new conversions were compared before output quantization, across all ten emitted stems. The four reused models were downloaded into an empty cache and verified by size and SHA-256 before inference. Their weights retain the original precision: float32 for Demucs, float16 for ZFTurbo.

RoFormer checks used two overlapping windows. Demucs checks used two shifts, a fixed seed of zero, and 25% overlap. The fixed seed makes the validation runs repeatable; normal package use leaves shift selection random unless a seed is supplied. The source audio and separated music are not distributed with the models.

Automatic downloads and offline loading

The public API integration report covers all seven models through normal Separator.load_model() and Separator.separate() calls. Each test:

Starts with an empty model directory and downloads the exact repository files from a local HTTP mirror.
Loads the model using its existing package name and separates the 14-second fixture.
Blocks all network connections, creates a new separator, and loads the same cache using the short catalog ID.
Verifies full-length, finite output and bit-identical unquantized stems between both runs.

PyTorch, ONNX, and ONNX Runtime imports are blocked throughout. The test checks actual audio writes and confirms that each model file is requested exactly once. This exercises download and cache behavior independently of Hugging Face availability. The report records the tested implementation's source hashes and identifies the local mirror as the download source.

CPU tests additionally cover corrupt downloads and caches, interrupted transfers, concurrent first use, offline cache misses, older cache reuse, and unchanged routing for other models.

Earlier reference comparisons

The following evidence comes from the separator projects' reference tests, independently of the catalog packaging checks. PyTorch references use the original python-audio-separator.

BS-RoFormer-SW: the standard core passes real-checkpoint tests against native python-audio-separator references and cross-repository consistency checks. Earlier full-track PCM comparisons include near-silent relative-error exceptions bounded by one PCM16 step.
InstV7n and Inst HQ3: a twelve-track comparison on two hosts passed relative L2 ≤ 0.05 for every compared stem. The worst values were 0.0120522 and 0.0047207, respectively.
HTDemucs and HTDemucs_FT: full-track controls with identical shift offsets, following the normalization correction, had worst relative L2 errors of 0.000775014 and 0.00106586. These are single-track controls. Other corpus comparisons include stochastic shift differences.
HTDemucs_6s: the existing Demucs model publication reports source-checkpoint parity. The catalog check adds offline six-stem inference and exact file verification; it does not rerun the PyTorch reference comparison.
ZFTurbo Vocals v1: earlier release tests cover both stems, overlap, input sample rates, and mixture reconstruction. This catalog uses the same published float16 conversion.

Detailed methods and results are in the mlx-audio-separator performance report, release validation, and RoFormer core documentation.

Interpreting the results

The catalog checks verify that the packaged files load and preserve the tested MLX behavior. They do not provide a new ground-truth SDR score, guarantee separation quality on every song, or establish a speedup from the recorded validation wall times. Bit-identical conversion results describe original-MLX versus safetensors-MLX outputs, not MLX versus PyTorch.

The manifest records source identities, runtime versions, and conversion source fingerprints. The model cards list file hashes, and SHA256SUMS covers the distributed files.