dettato-drums
dettato-drums is the drum model of dettato. It writes the drum hits of a recording on General MIDI drum keys. It covers drum kits and orchestral percussion, and it writes timpani at their pitch. It is 2 models and a router, with 5.0M parameters in all. A 189K-parameter drum-kit model is for pop, rock, jazz and electronic music. An orchestral model is for concert music. The router listens to the recording and picks one of them.
On the six public benchmark sets with drums it scores 47.1 drum F1. MuScriptor 1.3B scores 34.0. On RWC Classical, which is mostly timpani, it scores 39.5. Mirelo A2M v1.1 Pro scores 32.5 there and MuScriptor scores 23.7.
What to expect. It needs a source that was recorded at 40 kHz or more. Audio at a lower rate gets no drums. The router picks one model for each recording. So a piece that has a drum kit and orchestral percussion would be transcribed by only one model.
It is part of the dettato collection, with dettato, dettato-evidence and dettato-grouper.
What It Is
| part | what it does | parameters |
|---|---|---|
| drum-kit model | 2-D CNN over the mel bands, a band GRU per frame, temporal conv / BiGRU / song attention, one small BiGRU head per drum family, with onset + velocity for GM keys 27-87 | 188,764 |
| orchestral model (3 averaged) | its own copy of the drum-kit network + a pitch-resolving low band (6144-point STFT, 120 log-frequency bands, 24 per octave) + an adapter (BiGRU + self-attention) that corrects the drum logits for concert percussion and adds a timpani head | 1,606,407 each |
| router | part of the orchestral model: one value per recording, from the adapter's states pooled over the whole recording | (included) |
If the router says a recording is orchestral (0.5 or more), the orchestral model transcribes it. If not, the drum-kit model transcribes it as it is. The hits are the peaks of the onset probability for each key, at a threshold of 0.5 and 50 frames per second. The orchestral model also adds its timpani hits. It skips a timpani hit if the same key was already hit within 40 ms.
Input: mono audio at 44.1 kHz from a source that was recorded at 40 kHz or more. The model reads the 7.8 to 20 kHz band, so dettato skips drums for sources at a lower rate.
Output: a list of hits {"onset", "offset", "pitch", "is_drum", "velocity"}. The onset is in seconds and each hit is 100 ms
long. pitch is the GM drum key and the velocity goes from 0 to 1. In orchestral mode, keys 41 to 53 are timpani at the pitch
they sound. This follows the GS Orchestra kit, where key 43 is G2 and key 48 is C3. Concert snare, bass drum, cymbals and
triangle use their normal GM keys.
Use
pip install git+https://github.com/kobimusic/dettato
from dettato.audio import load
from dettato.drums import DrumTranscriber, MIN_SOURCE_RATE
audio = load("song.wav")
drums = DrumTranscriber.from_pretrained("cuda") # or "cpu"; downloads this repo on first use
if audio.rate >= MIN_SOURCE_RATE:
hits = drums.transcribe(audio.at(44100))
print(len(hits), drums.last_info) # e.g. {"router": 0.0004, "model": "drum-kit"}
The dettato command line tool runs this model for the drum track of the full transcription. The package README has more.
The files are model.safetensors and config.json. The drum-kit model is under drum_kit.* and the 3 orchestral models are
under orchestral.{0,1,2}.*. The config has the architecture and the thresholds.
Results
The score is drum onsets within 50 ms on the exact GM key. It is F1 for each recording, and it is averaged for each set on the recordings that have drum labels. I picked the thresholds and the orchestral ensemble on held-out synthetic audio only. No benchmark audio or labels were used for training, or for picking thresholds and checkpoints.
| set | drum-kit model alone | dettato drums | Mirelo A2M v1.1 (self-reported) | MuScriptor 1.3B |
|---|---|---|---|---|
| RWC Classical | 11.20 | 39.52 | 32.5 | 23.7 |
| RWC Royalty-Free | 65.40 | 65.40 | 50.3 | 36.5 |
| RWC Popular | 50.57 | 50.57 | - | 42.1 |
| Slakh2100 test | 54.43 | 54.43 | - | 41.0 |
| RWC Jazz | 35.54 | 36.26 | - | 31.3 |
| RWC Genre | 34.24 | 36.29 | - | 29.2 |
The MIDI files and the scripts to make them again are in dettato-benchmark. When I ran it all again with the package, each set came out within 0.1 of this table.
Each RWC Popular and Slakh recording goes to the drum-kit model, so those rows are the same as its numbers. 14 of the 35 RWC Jazz recordings and 14 of the 76 RWC Genre recordings go to the orchestral model.
The RWC Classical drum score is the average of 4 recordings. C003 (Beethoven 5) and C008 (the Figaro overture) are timpani, and they went from 0 to 54.3 and 63.1. C010 (the Blue Danube) has concert snare, bass drum and triangle, and it went from 44.8 to 40.7. The orchestral model also writes the timpani for that piece, which RWC labels as a pitched part there. C004 has one cymbal crash in about ten minutes, and each model scores 0 on it.
Limits
- It needs a source that was recorded at 40 kHz or more. Audio at a lower rate gets no drums (for instance, many MP3s from video sites and 16 kHz files).
- The router picks one model for each recording. A piece that has a drum kit and orchestral percussion is transcribed by one model.
- Timpani are only covered on keys 41 to 53 (F2 to F3).
- Slakh2100: 12 of the 151 test-set MIDI files were part of the orchestral model's training material. The Slakh score is from the drum-kit model, since each Slakh test song goes to it. So that row is not affected.
Training Data
I trained it on synthetic multi-instrument audio with exact drum labels. I made that audio in-house. I am not releasing it. Its orchestral percussion uses the sample libraries Sonatina Symphonic Orchestra, VSCO-2 Community Edition and VCSL. The orchestral model also saw drum-free recordings from MUSDB18-HQ and MusicNet. The drum-kit model was trained on synthetic audio only. No benchmark recording was used.
Training data licences: Sonatina Symphonic Orchestra is Creative Commons Sampling Plus 1.0. VSCO-2 Community Edition and VCSL are CC0. MUSDB18-HQ has the research and non-commercial terms of its tracks' licences. MusicNet has the licences that the dataset lists.
Licence
Apache-2.0.
Citation
@software{dettato2026,
title = {Dettato: multi-instrument audio-to-MIDI transcription},
author = {Mills, Nathan},
year = {2026},
url = {https://github.com/kobimusic/dettato}
}
- Downloads last month
- 572

