model card

README.md

@@ -0,0 +1,55 @@
+---
+license: mit
+tags:
+  - music-source-separation
+  - stems
+  - demucs
+  - core-ai
+  - aimodel
+  - apple-silicon
+  - macos
+library_name: swift-vocal-isolation
+---
+
+# Hybrid Transformer Demucs for Core AI (`.aimodel`)
+
+Apple Core AI conversion of **HTDemucs** (`htdemucs`) — Simon Rouard, Francisco Massa, Alexandre
+Défossez, *Hybrid Transformers for Music Source Separation*, ICASSP 2023 — from
+[github.com/facebookresearch/demucs](https://github.com/facebookresearch/demucs) (MIT, 42M
+parameters). Four stems: drums, bass, other, vocals.
+
+`stems-htdemucs-float32.aimodel` (168 MB) holds the network between the complex spectrogram and
+the mask at the model's 7.8-second training segment. Inputs: the normalised mix `[1, 2, 343980]`,
+its complex-as-channels spectrogram `[1, 4, 2048, 336]` and the four per-segment statistics the
+model normalises by. Outputs: the spectral stems `[1, 16, 2048, 336]` and the time-branch stems
+`[1, 8, 343980]`. Core AI has no STFT and no variance op, so those run in the host — in
+[swift-vocal-isolation](https://github.com/arraypress/swift-vocal-isolation) (MIT), together with
+upstream's chunking, overlap-add weights, centred padding and global normalisation, line for line.
+Nothing was re-authored, retrained or pruned. The `stems` CLI exposes it as `--engine demucs`.
+
+## Faithfulness
+
+Held to upstream's Python with `shifts=0` (random shifts make upstream itself non-deterministic):
+
+- The exported network is asserted equal to `model(mix)` before export.
+- One training segment through Core AI on the GPU: 136–147 dB PSNR against upstream's output.
+- Every one of 63 chunks of a 6-minute mix: worst stem 110 dB.
+- Whole clips end to end (10 s and 6 min): 127–148 dB on every stem.
+
+## Use
+
+```sh
+hf download arraypress/stems-demucs --local-dir models
+stems model install models/stems-htdemucs-float32.aimodel
+stems song.wav --engine demucs      # song-drums.wav, -bass, -other, -vocals
+```
+
+Requirements: macOS 27, Apple silicon. Reproduce with `uv run Tools/export_demucs.py` in the
+library repo (fetches the checkpoint through the `demucs` package). `htdemucs_ft` (a bag of four)
+and `htdemucs_6s` (adds guitar and piano) export the same way but are not verified here.
+
+## Licence and citation
+
+MIT, as upstream. Please cite:
+
+> S. Rouard, F. Massa, A. Défossez. "Hybrid Transformers for Music Source Separation." ICASSP 2023.