dettato
dettato is the transcriber of KobiMusic's dettato, an audio to MIDI model. It takes any recording (a band, an orchestra, a piano or a voice) and finds the notes. For each note it gives the start, the end and the pitch on the 88 piano keys. It can also hold 2 notes on the same key, for when 2 instruments play the same pitch. It is 15.3M parameters.
With dettato-drums for the drums, it gets 59.2 composite F1 on nine public benchmark sets. MuScriptor 1.3B is the next best open model and it scores 45.1. So dettato is 14 points ahead, and it is 65 times smaller. On the three sets that Mirelo publishes it scores 60.8. That is a bit higher than Mirelo A2M v1.1 Pro, which is a closed model, at 56.1.
What to expect. The transcriber only writes notes. The grouper puts them into instrument tracks and the drum model adds the drums. The dettato package runs all four models. The notes are performance MIDI, so they are not quantized to a beat. I would try it on your music before you rely on the benchmarks, since a benchmark could be very different from your songs.
It is part of the dettato collection, with dettato-evidence, dettato-grouper and dettato-drums.
How It Works
The transcriber is a pitch-equivariant lane network. Each of the 88 keys is a lane, and each lane runs the same layers. This way a pattern it learns on one key will work on the other keys too. For instance, a piano note at C4 and at C5 would look the same to the lane blocks.
- Front end: a harmonic log-frequency spectrogram of 16 kHz audio. It has 3 bins per semitone from A0. It also has one channel per harmonic from 0.5 to 7, so each harmonic of a note lands on the bin of that note. A short-window channel helps with sharp onsets.
- Stem: two 3x3 convolutions, then a strided convolution down to 20 ms frames and 1 lane per semitone.
- Trunk: 368 channels and 15 ConvNeXt-style lane blocks at 40 ms. Each block has a dilated convolution in time and sparse pitch mixing at 1, 2, 12, 19, 24, 28 and 31 semitones. After blocks 5, 10 and 15 there is axial attention on pitch and on time, plus a bidirectional GRU for each lane. Then 2 refine blocks bring it back to 20 ms.
- Heads: onset, frame and offset probabilities for each of the 88 keys, with 2 voices per key.
It reads a recording in 8 s windows that start 2 s apart. The windows are averaged using a Hann weight. Then it picks the notes from the peaks. The onset threshold is 0.25, and it is 0.30 for voice 2. The offset is 0.30 and the frame is 0.40. Each note keeps its onset probability as its confidence.
Use
pip install git+https://github.com/kobimusic/dettato
dettato song.mp3 -o out/ # the full system: out/song.mid (instrument tracks + drums) and out/song.json
This model alone, from Python:
from dettato.audio import load
from dettato.transcriber import Transcriber
t = Transcriber.from_pretrained("cuda") # or "cpu"; downloads this repo on first use
maps, _ = t.run(load("song.mp3").at(16000)) # six probability maps, each (frames, 88), 20 ms frames
notes = t.notes(maps) # dict of arrays: onset, offset (s), pitch (MIDI), conf, slot (voice 1 / 2)
The files are model.safetensors and config.json. The config has the architecture and the decode thresholds.
Results
The benchmark is the eight sets from Table 2 of the MuScriptor paper, plus the Slakh2100 test set. The eight sets are Bach10, Dagstuhl ChoirSet, PHENICX-Anechoic, RWC Popular, RWC Classical, RWC Genre, RWC Jazz and RWC Royalty-Free. I scored them with the rules from that paper, which use mir_eval and a 50-cent pitch tolerance. Each score is F1 x 100 for each recording. It is averaged for each set, and then the sets are averaged.
- onset: pitched notes with the onset within 50 ms. The instrument does not count.
- offset: an onset match, and the end within 50 ms or 20% of the note's length. The bigger one is used.
- frame: pitch-only piano rolls at 62.5 ms steps, scored cell by cell.
- drums: onsets within 50 ms on the exact GM key. Six of the sets have drums.
- composite: the mean of the four. The 95% interval is a bootstrap on the recordings.
These are the transcriber's notes, with dettato-drums for the drum score:
| system | parameters | composite | onset | frame | offset | drums |
|---|---|---|---|---|---|---|
| dettato + dettato-drums | 15.3M + 5.0M | 59.2 (57.9-60.6) | 67.2 | 77.9 | 44.8 | 47.1 |
| MuScriptor | 1.3B | 45.1 | 48.6 | 70.2 | 27.8 | 34.0 |
On the eight MuScriptor sets alone it scores 58.4. MuScriptor scores 44.6 and YourMT3+ scores 41.1.
| set | onset | frame | offset | drums | MuScriptor onset / frame / offset / drums |
|---|---|---|---|---|---|
| Bach10 | 91.9 | 94.2 | 87.7 | - | 43.1 / 85.0 / 36.0 / - |
| Dagstuhl ChoirSet | 55.9 | 82.0 | 44.4 | - | 14.4 / 80.7 / 11.5 / - |
| PHENICX-Anechoic | 71.2 | 83.3 | 40.0 | - | 56.1 / 74.6 / 32.6 / - |
| RWC Popular | 57.1 | 71.8 | 33.1 | 50.6 | 46.1 / 61.2 / 25.6 / 42.1 |
| RWC Classical | 77.9 | 73.8 | 41.6 | 39.6 | 67.7 / 70.5 / 36.9 / 23.7 |
| RWC Genre | 51.0 | 65.7 | 29.3 | 36.3 | 44.7 / 58.7 / 24.4 / 29.2 |
| RWC Jazz | 63.6 | 67.4 | 36.0 | 36.3 | 59.4 / 62.7 / 33.9 / 31.3 |
| RWC Royalty-Free | 63.3 | 80.2 | 41.3 | 65.4 | 47.1 / 68.6 / 24.8 / 36.5 |
| Slakh2100 test | 73.5 | 82.3 | 49.4 | 54.4 | 58.9 / 70.2 / 24.3 / 41.0 |
Mirelo A2M v1.1 Pro reports three of the sets: PHENICX-Anechoic, RWC Classical and RWC Royalty-Free. On those three dettato gets 60.8. Mirelo A2M gets 56.1 (Mirelo published that number), MuScriptor gets 47.4 and YourMT3+ gets 45.7.
The MuScriptor and YourMT3+ numbers come from the MuScriptor paper (Rouard et al. 2026). On Slakh2100 I ran the MuScriptor weights myself. These scores are from the transcriber's notes alone. The evidence repairer runs after it, and I measured it by itself on the dettato-evidence card.
If you would like to check these numbers, the MIDI files and the scripts I used are in dettato-benchmark. I think you should get the same composite if you run them again.
In my opinion the onsets and the note ends are the biggest win here. dettato gets 67.2 on onsets and 44.8 on offsets. MuScriptor gets 48.6 and 27.8. The frames are closer, since it is 77.9 against 70.2.
Limits
- It only writes notes. The grouper adds the instrument tracks and the drum model adds the drums.
- The range is the 88 piano keys (A0 to C8), with at most 2 notes on one key at a time.
Training Data
I trained the transcriber on a mix of synthetic instrument recordings and CC solo instruments from the URMP dataset mixed in. The rest of the instruments morph together around 20 soundfonts, using the soundfont babies technique I published. The vocals and the other augmentation are CC data too. No copyrighted recordings were used. I made the training audio in-house, and I will not be releasing it. No benchmark recording was used for training or for picking thresholds. None of the 151 Slakh2100 test MIDI files are in the training material either (I checked for the exact files).
Licence
Apache-2.0.
Citation
@software{dettato2026,
title = {Dettato: multi-instrument audio-to-MIDI transcription},
author = {Mills, Nathan},
year = {2026},
url = {https://github.com/kobimusic/dettato}
}
- Downloads last month
- 602



