File size: 2,767 Bytes
018240c | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 | ---
license: mit
library_name: coreml
tags:
- audio
- music
- source-separation
- coreml
- scnet
---
# SCNet (base) – Core ML
Core ML conversion of **SCNet**, a music source separation model that splits a stereo mix into
four stems: drums, bass, other, vocals. These files are downloaded on first use by the iOS app
**Preso!**, which runs the separation entirely on device.
## Attribution
The model and its pretrained weights are the work of the SCNet authors:
- Paper: Weinan Tong et al., *SCNet: Sparse Compression Network for Music Source Separation* (2024)
- Original code and weights: <https://github.com/starrytong/SCNet>
- License: MIT. The author confirmed that the released pretrained weights are MIT licensed and
may be redistributed, including format-converted versions, with attribution:
<https://github.com/starrytong/SCNet/issues/35>
This repository only contains a format conversion of the original checkpoint. No retraining or
fine-tuning was done. The original license text is in `LICENSE-SCNet`.
## Files
| Folder | Precision | Size |
|---|---|---|
| `SCNet_fp16.mlpackage` | float16 | about 23 MB |
| `SCNet_fp32.mlpackage` | float32 | about 45 MB |
Each `.mlpackage` is a folder with three files: `Manifest.json`,
`Data/com.apple.CoreML/model.mlmodel` and `Data/com.apple.CoreML/weights/weight.bin`.
## What is inside the model
Only the network. The STFT and inverse STFT, input normalisation and chunking are done outside,
by the caller:
- Sample rate 44.1 kHz, stereo.
- STFT: 4096 points, hop 1024, rectangular window, centred with reflect padding, normalised.
- Fixed segments of 11 s (476 frames), 25% overlap, triangular cross-fade.
- Input `spec`: shape `(1, 4, 2049, 476)`, laid out as `[channel][real/imag]`.
- Output `stems`: shape `(1, 16, 2049, 476)`, laid out as `[source][channel][real/imag]`,
sources in the order drums, bass, other, vocals.
- The FFTs along the time axis inside the original network are replaced by constant DFT
matrices, which is why the segment length is fixed.
## Measured
- Conversion fidelity against the PyTorch original: 109 dB (fp32), 59 dB (fp16).
- Quality on 12 seven-second MUSDB18 test excerpts (global SDR, not comparable one to one with
the BSSEval figures in the paper): drums 9.2, bass 7.6, other 5.3, vocals 10.0 dB.
- Speed on a Mac with Apple M1, GPU, fp16: about 2 s for 30 s of audio.
Not measured: speed and memory on an iPhone, and the Neural Engine. In the iOS Simulator the
fp16 variant returned all zeros while fp32 worked; on macOS both work.
## Training data
SCNet was trained by its authors on MUSDB18-HQ, a dataset released for research and educational
use. Check whether that matters for your use before relying on these weights commercially.
|