| --- |
| license: cc-by-nc-sa-4.0 |
| pipeline_tag: audio-classification |
| library_name: pytorch |
| tags: |
| - piano-transcription |
| - automatic-music-transcription |
| - music |
| - mir |
| metrics: |
| - note-f1 |
| --- |
| |
| # miditizer v1 — polyphonic piano transcription |
|
|
| miditizer transcribes solo-piano audio (wav, mp3, flac) into notes |
| (pitch, onset, offset, velocity) and writes MIDI and JSON. The model is an |
| original architecture trained from scratch on MAESTRO v3. It ships with |
| its full training recipe, its evaluation protocol, and the error |
| characterization behind every number reported here. |
|
|
| **Checkpoint:** `miditizer-v1.pt` (385 MB), sha256 |
| `fc5a037dcbaab3705d4d8c0ee4193c452f9b7280984f3a07ae705f2b550ff62f` |
|
|
| ## Results |
|
|
| MAESTRO v3 validation (137 held-out pieces), sustain-extended label |
| convention, frozen harness v1.1.0. "note-F1@0.2" = a note counts as |
| correct when pitch matches, onset is within 50 ms, and the offset is |
| within max(20 % of the reference note's duration, 50 ms) (mir_eval-style |
| matching, comparable to Transkun's reporting convention). "onset-F1" uses the same 50 ms tolerance without the offset |
| condition. |
| |
| | metric | value | |
| |---|---| |
| | note-F1@0.2 | **0.7999** (95 % CI [0.787, 0.813], 137-piece bootstrap) | |
| | onset-F1 | **0.9213** | |
| | frame-F1 | 0.9039 | |
| |
| Ablations (each row is one controlled change, measured as a paired |
| per-piece difference with a 10,000-resample bootstrap CI on the same |
| 137 pieces; ¢ = hundredths of note-F1@0.2): |
| |
| | change | Δ note-F1@0.2 | 95 % CI | |
| |---|---|---| |
| | focal → ASL onset loss | +1.33¢ | [+1.09, +1.58] | |
| | SemiCRF loss weight 0.5 → 2.0 (the released setting) | +1.48¢ | [+1.28, +1.69], 126/137 pieces improve | |
| | SemiCRF loss weight 2.0 → 4.0 | +0.28¢ | [+0.14, +0.43] | |
| |
| The structured-loss weight saturates above 2.0; the released setting sits |
| near the optimum of that curve. |
| |
| Characterization (measured on the released checkpoint): |
| |
| - Recipe re-execution (n=1: one fresh seed and execution path, |
| identical recipe): Δ = −0.03¢, CI spans zero. |
| - Decode operating point: a 9-point onset-logit bias sweep puts the |
| shipped θ=0 at the top of the grid; decode-scale sweeps find no setting |
| beating the shipped one by more than bootstrap noise (best alternative |
| +0.015¢). |
| - Known deficit: soft-note onsets (the quietest velocity third) score |
| onset-F1 0.626 vs 0.921 overall, and 77 % of "too long" offset errors |
| sit on reference notes shorter than 0.35 s. Loss-side interventions on |
| the soft-note slice were measured and did not move it; we classify this |
| deficit as architectural. |
| |
| Context for these numbers: on comparable conventions, published systems |
| include Kong 2020 (0.8247, raw note-off), Hawthorne 2021 seq2seq (0.8394, |
| sustain-extended), and Transkun (0.9348 on the MAESTRO *test* split, |
| sustain-extended; ISMIR 2024, arXiv:2404.09466 — our numbers are |
| validation). |
| |
| ## Architecture (19.26M parameters) |
| |
| - **Front-end:** 229-bin log-mel spectrogram, 62.5 fps, 16 kHz mono. |
| - **Trunk:** 5-block CNN funnel (64→512 ch, residual, stride-2 frequency) |
| into an 8-layer / 8-head Transformer encoder (d_model 384, dff 1536, |
| RoPE positional encoding). |
| - **Heads:** onset / frame / offset / release (binary logits) + |
| velocity (sigmoid-scaled 1–127). |
| - **Decoding:** Semi-CRF segment model — Viterbi over note segments with a |
| learned duration prior, onset/frame/offset probabilities as potentials |
| (offset scaled 0.5), sliding 10 s windows at 50 % overlap (max-stitching, |
| mean velocity), dedup and duration floors. The SemiCRF is both a training |
| loss (its gradients shape the heads) and the decoder at inference. |
|
|
| ## Training |
|
|
| - **Data:** MAESTRO v3.0.0 train split (962 pieces, ~159 h of the dataset's |
| ~199 h; Hawthorne et al., 2019). Model selection used the validation split only; the released |
| checkpoint has never been evaluated on the test split. |
| - **Recipe:** ASL onset loss (γ−=2, m=0.1), focal frame loss, SemiCRF |
| structured offset loss (weight 2.0), masked-L1 velocity; frequency-only |
| SpecAugment; 45 epochs cosine schedule, batch 16 × 10 s windows, |
| seed 20260728; one RTX 5090, ~14 GPU-hours per run (training and |
| in-training validation). |
| - **Provenance:** every training run in the program is logged (config |
| hash, code commit, metrics); the released checkpoint is epoch 39 of the training run whose |
| MLflow id is embedded in the file (internal run `8354c373…`; not in |
| this repository). |
|
|
| ## Intended use |
|
|
| Offline transcription of solo piano recordings — musicology, practice |
| tooling, dataset labeling, creative MIDI manipulation. Evaluation is |
| MAESTRO-only (Disklavier pianos in controlled competition conditions); |
| expect degradation on non-piano instruments, heavily produced mixes, or |
| noisy field recordings. This is an offline system; streaming/causal use |
| is future work. |
|
|
| ## Data |
|
|
| Trained exclusively on **MAESTRO v3.0.0** (Hawthorne et al., 2019; |
| CC-BY-NC-SA 4.0): ~200 h of paired piano audio and aligned MIDI from |
| International Piano-e-Competition performances. See *Intended use* for |
| scope notes. |
|
|
| ## License |
|
|
| **CC-BY-NC-SA 4.0** — non-commercial use with attribution and |
| share-alike. We apply the dataset's restrictions to the checkpoint as |
| policy (whether trained weights legally inherit dataset licenses is |
| unsettled; we take the conservative reading). |
|
|
| ## Limitations |
|
|
| - Quiet notes are under-detected (see *Known deficit* above); dense pedal |
| textures accumulate offset errors on short notes. |
| - Sustain-pedal convention: labels and outputs reflect *acoustic* note |
| sustain including pedal, not raw MIDI note-off. Comparisons against |
| systems using a different convention are not apples-to-apples. |
| - Velocity is a relative loudness estimate, not a calibrated strike force. |
|
|
| ## Citation |
|
|
| ``` |
| @software{miditizer_v1_2026, |
| title = {miditizer v1: a piano transcriber with a RoPE Transformer |
| trunk and SemiCRF-as-loss}, |
| author = {Krystian Lewandowski}, |
| year = {2026}, |
| url = {https://github.com/elewarr/miditizer}, |
| note = {Note-F1@0.2 0.7999 / onset-F1 0.9213 on MAESTRO v3 |
| validation, sustain-extended convention}, |
| } |
| ``` |
|
|
| ## Usage |
|
|
| ```bash |
| pip install torch numpy librosa soundfile mido pretty_midi |
| git clone https://github.com/elewarr/miditizer && cd miditizer && pip install . |
| python examples/transcribe.py input.wav output.mid \ |
| --ckpt miditizer-v1.pt --device cuda --json-out output.json |
| ``` |
|
|
| The example is self-contained: it loads the checkpoint, runs the |
| sliding-window + SemiCRF decode, and writes type-1 MIDI plus a JSON note |
| list. On the released checkpoint it produces the same notes the frozen |
| evaluation harness scored on held-out pieces. |
|
|
