Upload README.md with huggingface_hub

README.md

@@ -0,0 +1,136 @@
+---
+license: cc-by-nc-4.0
+pipeline_tag: audio-classification
+base_model:
+  - m-a-p/MERT-v1-95M
+  - m-a-p/MERT-v1-330M
+base_model_relation: finetune
+tags:
+  - pytorch
+  - Transformers
+  - music
+  - streaming
+  - music-information-retrieval
+  - feature-extraction
+  - onnx
+---
+
+# MuSTream: A Streaming Music Representation for Online Music Understanding
+
+MuSTream is a streaming music representation for online music understanding. A
+two-stage distillation (capacity, then causality) turns the non-causal
+[MERT](https://huggingface.co/m-a-p/MERT-v1-95M) into causal streaming models of 26M,
+95M and 330M parameters. They process audio as a stream, give one prediction every
+0.2 s with a block-wise KV cache, and reach near-offline performance on chord
+recognition, beat and downbeat tracking and key detection, in real time on a CPU.
+
+- **Code:** <https://github.com/yamaha-research/MuSTream>
+- **Real-time demo:** <https://yamaha-research-mustream.static.hf.space>
+
+## What is included
+
+Six streaming backbones and, for each, three task probes, in PyTorch and ONNX:
+
+| Model | Parameters | Teacher |
+|---|---|---|
+| MuSTream-26M | 26M | 26M bidirectional student distilled from MERT-v1-95M |
+| MuSTream-95M | 95M | MERT-v1-95M |
+| MuSTream-330M | 330M | MERT-v1-330M |
+
+On an Apple M1 CPU, MuSTream-26M runs at a real-time factor of 0.09 with 0.22 s
+latency, and MuSTream-330M at 0.35 with 0.29 s.
+
+- **Variants `r0` / `r15`.** The local-interval teacher used in distillation sees
+  $p = 0$ or $p = 15$ frames past each supervised block.
+- **Streaming setup.** 24 kHz mono input, 75 fps features. Each frame attends to the
+  past plus the rest of its 15-frame (0.2 s) block, over a 750-frame (10 s) KV cache.
+- **Probes.** LSTM probes for chord recognition, beat / downbeat tracking and key
+  detection on top of each backbone.
+- **Training data.** The released backbones are distilled on the public
+  [MTG-Jamendo](https://github.com/MTG/mtg-jamendo-dataset) audio (no labels).
+  The paper's results use backbones distilled on internal data; see
+  [`docs/understanding_performance.md`](https://github.com/yamaha-research/MuSTream/blob/main/docs/understanding_performance.md)
+  for a comparison of the two.
+
+```
+pytorch/
+├── backbone/                   # MuSTream backbones
+│   └── streaming_{26M,95M,330M}_{r0,r15}.ckpt
+└── {chord,key,beat}/           # LSTM probes, one folder per backbone
+    └── streaming_{26M,95M,330M}_{r0,r15}/{best.pt, config.yaml}
+onnx/                           # the same models in ONNX
+├── backbone/
+│   └── streaming_{26M,95M,330M}_{r0,r15}.streaming.{onnx,json}
+└── {chord,key,beat}/
+    └── streaming_{26M,95M,330M}_{r0,r15}.streaming.{onnx,json}
+```
+
+## How to use
+
+### Real-time demo in the browser
+
+Open **<https://yamaha-research-mustream.static.hf.space>**, choose a task (chord,
+beat, key or all), upload a song and press *Start streaming*, or press *Try a sample*.
+The ONNX models run on your own device with ONNX Runtime Web: nothing to install, and
+your audio is never uploaded. It uses MuSTream-26M by default. To run larger models or
+the demo locally, see
+[`streaming-demo/README.md`](https://github.com/yamaha-research/MuSTream/blob/main/streaming-demo/README.md).
+
+### Streaming feature extraction
+
+Set up the code with [uv](https://docs.astral.sh/uv/) and download this repository
+into `mustream/ckpt/`:
+
+```bash
+git clone https://github.com/yamaha-research/MuSTream.git && cd MuSTream
+uv sync && source .venv/bin/activate
+hf download yamaha-research/MuSTream --local-dir mustream/ckpt
+```
+
+Extract streaming features from your own audio (`--model` 26M, 95M or 330M;
+`--variant` `r0` or `r15`):
+
+```bash
+python mustream/extract_features.py song.wav --model 95M --variant r15
+# -> features/song.npz: `features` [L, T, D] at 75 fps (L = 13, D = 768 for 95M)
+```
+
+From Python:
+
+```python
+import sys; sys.path.insert(0, "mustream")
+from extract_features import load_backbone, load_audio, extract
+
+model = load_backbone("mustream/ckpt/pytorch/backbone/streaming_95M_r15.ckpt", device="cuda")  # or "cpu"
+features = extract(model, load_audio("song.wav"))     # torch.Tensor [L, T, D], 75 fps
+```
+
+### Chord, beat and key predictions
+
+Each probe's `config.yaml` already points at its backbone, so the probes run as
+downloaded. For example, streaming chord recognition:
+
+```bash
+cd mustream/chord_recognition
+python infer_streaming.py ../ckpt/pytorch/chord/streaming_95M_r15/config.yaml \
+                          ../ckpt/pytorch/chord/streaming_95M_r15/best.pt --audio song.wav
+```
+
+See the [GitHub repository](https://github.com/yamaha-research/MuSTream) for beat and
+key detection, ONNX inference without PyTorch, training and evaluation.
+
+## License
+
+Copyright 2026 Yamaha Corporation and the MuSTream authors.
+
+The model weights in this repository (backbones and probes, in PyTorch and ONNX) are
+licensed under the
+[Creative Commons Attribution-NonCommercial 4.0 International License](https://creativecommons.org/licenses/by-nc/4.0/)
+(CC BY-NC 4.0). They are granted for non-commercial research purposes only.
+
+The MuSTream backbones are distilled from [MERT](https://huggingface.co/m-a-p/MERT-v1-95M),
+whose weights are also released under CC BY-NC 4.0, using audio from
+[MTG-Jamendo](https://github.com/MTG/mtg-jamendo-dataset).
+
+The code in the [GitHub repository](https://github.com/yamaha-research/MuSTream) is
+licensed separately under the GNU Affero General Public License v3.0 (AGPL-3.0).