omr-synth — a small OMR model for Western staff, Jianpu (简谱) & Sargam
A ~26M-parameter image→sequence Transformer (CNN encoder + autoregressive decoder) that reads a page of music notation and writes out a compact symbolic score (OMRDSL). Trained entirely on synthetic data, overnight, on a Mac mini (M4, Apple-Silicon MPS) — no cloud GPU. A weekend learning project: early and honest, not production OMR.
- 💻 Code: https://github.com/abhid1234/omr-synth
- ▶️ Interactive demo: https://omr-synth.vercel.app
- 📚 Dataset: https://huggingface.co/datasets/abhid1234/omr-synth
What it does
Reads Western staff, Jianpu, and Sargam — three ways of writing music — into ONE shared symbolic target. General vision models can't read Jianpu/Sargam and there's almost no labeled data for them; a render-forward engine manufactures it (write the score first — that's the label — then render it into an image).
Honest held-out results (pages it never saw)
Scoped to level-0 (clean, monophonic) to be learnable in one overnight run:
| Notation | Token accuracy | Content-only (the actual notes) |
|---|---|---|
| Western | ~30% | ~11% |
| Jianpu | ~28% | ~9% |
| Sargam | ~28% | ~8% |
| Overall | ~29% | ~9% |
Token accuracy counts every output token, including the easy structural scaffolding (clef, meter, bar/voice layout) it almost always gets right. Content-only counts just the actual notes and durations — the honest "can it read the music" number: about one note in ten. Exact-match ~0%. Balanced across all three.
Architecture
CNN stem (1→64→128→384) + 6-layer Transformer encoder (d=384) reading a grayscale, ink-positive,
aspect-padded page; 6-layer autoregressive Transformer decoder emitting OMRDSL. ~26M params
(src/model/omr.py). This checkpoint is the best of the overnight run (gentler-LR rerun, epoch 7).
Usage
git clone https://github.com/abhid1234/omr-synth && cd omr-synth
pip install torch torchvision
python predict.py --checkpoint omr-synth-e7.pt --image page.png
Limitations
Level-0 only (no polyphony/dynamics/ornaments, no real manuscripts). Reads content partially. Training was unstable early (fixed with a warmup+cosine schedule). The reusable contribution is the render-forward data engine, not this early model. MIT · Abhi Das.