| --- |
| license: mit |
| library_name: coreml |
| tags: |
| - audio |
| - music |
| - source-separation |
| - coreml |
| - scnet |
| --- |
| |
| # SCNet (base) – Core ML |
|
|
| Core ML conversion of **SCNet**, a music source separation model that splits a stereo mix into |
| four stems: drums, bass, other, vocals. These files are downloaded on first use by the iOS app |
| **Preso!**, which runs the separation entirely on device. |
|
|
| ## Attribution |
|
|
| The model and its pretrained weights are the work of the SCNet authors: |
|
|
| - Paper: Weinan Tong et al., *SCNet: Sparse Compression Network for Music Source Separation* (2024) |
| - Original code and weights: <https://github.com/starrytong/SCNet> |
| - License: MIT. The author confirmed that the released pretrained weights are MIT licensed and |
| may be redistributed, including format-converted versions, with attribution: |
| <https://github.com/starrytong/SCNet/issues/35> |
|
|
| This repository only contains a format conversion of the original checkpoint. No retraining or |
| fine-tuning was done. The original license text is in `LICENSE-SCNet`. |
|
|
| ## Files |
|
|
| | Folder | Precision | Size | |
| |---|---|---| |
| | `SCNet_fp16.mlpackage` | float16 | about 23 MB | |
| | `SCNet_fp32.mlpackage` | float32 | about 45 MB | |
|
|
| Each `.mlpackage` is a folder with three files: `Manifest.json`, |
| `Data/com.apple.CoreML/model.mlmodel` and `Data/com.apple.CoreML/weights/weight.bin`. |
|
|
| ## What is inside the model |
|
|
| Only the network. The STFT and inverse STFT, input normalisation and chunking are done outside, |
| by the caller: |
|
|
| - Sample rate 44.1 kHz, stereo. |
| - STFT: 4096 points, hop 1024, rectangular window, centred with reflect padding, normalised. |
| - Fixed segments of 11 s (476 frames), 25% overlap, triangular cross-fade. |
| - Input `spec`: shape `(1, 4, 2049, 476)`, laid out as `[channel][real/imag]`. |
| - Output `stems`: shape `(1, 16, 2049, 476)`, laid out as `[source][channel][real/imag]`, |
| sources in the order drums, bass, other, vocals. |
| - The FFTs along the time axis inside the original network are replaced by constant DFT |
| matrices, which is why the segment length is fixed. |
|
|
| ## Measured |
|
|
| - Conversion fidelity against the PyTorch original: 109 dB (fp32), 59 dB (fp16). |
| - Quality on 12 seven-second MUSDB18 test excerpts (global SDR, not comparable one to one with |
| the BSSEval figures in the paper): drums 9.2, bass 7.6, other 5.3, vocals 10.0 dB. |
| - Speed on a Mac with Apple M1, GPU, fp16: about 2 s for 30 s of audio. |
|
|
| Not measured: speed and memory on an iPhone, and the Neural Engine. In the iOS Simulator the |
| fp16 variant returned all zeros while fp32 worked; on macOS both work. |
|
|
| ## Training data |
|
|
| SCNet was trained by its authors on MUSDB18-HQ, a dataset released for research and educational |
| use. Check whether that matters for your use before relying on these weights commercially. |
|
|
