MuS3D: Music Source Separation via Stem Discovery

Model checkpoints accompanying the paper "Music Source Separation via Stem Discovery".

Contents

Each folder contains the weights and a config.yaml describing the architecture and training setup:

• SEG_mixemb: SEG, the stem embedding generator. A diffusion transformer (37M parameters) trained with conditional flow matching on PCA-reduced PaSST embeddings; generates stem queries and residual embeddings.
• SEG_resemb: SEG variant additionally conditioned on the residual embedding from the previous iteration.
• DiT: DiT-Sep, the conditional flow matching separator based on SAM-Audio (large, ~2.2B parameters), conditioned on query embeddings and operating on DAC-VAE latents; fine-tuned from the public SAM-Audio weights.
• BS-Locoformer_query: BS-Locoformer (medium band-split TF-Locoformer) with query conditioning via SwiGLU bias modulation.
• BS-Locoformer_static: the same architecture trained as a static 6-stem (VDBGPO) separator, used as a baseline.
• classifiers/pca_params_train.pkl: fitted PCA parameters used to reduce the 1024-dimensional PaSST embeddings to 128 dimensions.
• classifiers/stem_classifier_pcatrain.pkl: hierarchical stem classifier over the PCA-reduced PaSST embeddings, used to assign taxonomy labels to generated queries during evaluation.
• classifiers/multilabel_classifier.pkl: multi-label mixture classifier, the stem-discovery baseline from Table 1 of the paper.

Code

Code and training details: github.com/REMUS-Aalto/Mus3D

Sound examples: research.spa.aalto.fi/publications/papers/mus3d

Citation

@article{kallinen2026mus3d,
  title   = {Music Source Separation via Stem Discovery},
  author  = {Kallinen, V. Valtteri and Moliner, Eloi and Juvela, Lauri and V{\"a}lim{\"a}ki, Vesa},
  journal = {arXiv preprint arXiv:2609.30912},
  year    = {2026},
}

License

The model weights are released under CC BY 4.0.

Acknowledgments

This research was funded by the Research Council of Finland (grant no. 371845). Computational resources were provided by the Aalto Science-IT project.