lvndry commited on
Commit
89fcce4
·
verified ·
1 Parent(s): e305870

Release drone-v2 and rewrite the model card

Browse files
Files changed (5) hide show
  1. README.md +192 -66
  2. drone-v2.json +0 -0
  3. drone-v2.onnx +3 -0
  4. drone-v2.pt +3 -0
  5. drone-v2.safetensors +3 -0
README.md CHANGED
@@ -1,99 +1,225 @@
1
  ---
2
  license: mit
3
- library_name: torch
 
4
  tags:
5
  - audio
6
  - music
7
  - key-detection
 
 
8
  - constant-q-transform
9
  - cnn
 
 
10
  model-index:
11
- - name: drone-v1
12
  results:
13
  - task:
14
  name: Musical key detection
15
- type: classification
16
  dataset:
17
- name: GiantSteps (held-out, 604 tracks)
18
- type: other
19
  metrics:
20
- - name: MIREX (weighted)
21
- type: other
22
- value: 73.58
23
- - name: exact
 
 
 
 
 
 
 
 
 
 
 
 
24
  type: other
25
- value: 66.8
 
26
  ---
27
 
28
- # drone-v1
29
-
30
- A 109,128-parameter key classifier: 24 classes (12 tonics x major/minor) from a
31
- constant-Q spectrogram. K10 (MIREX 2026) architecture, retrained on 1,363
32
- Beatport EDM tracks. The smallest model to beat the native libkeyfinder
33
- reference (69.35% MIREX) on the held-out GiantSteps 604.
34
-
35
- ## Results
36
-
37
- Held-out GiantSteps 604, single evaluation per seed:
38
-
39
- | seed | exact | MIREX |
40
- | --- | --- | --- |
41
- | 13 | 68.5% | 74.8% |
42
- | 37 | 66.1% | 73.0% |
43
- | 73 | 65.7% | 72.9% |
44
- | **mean** | **66.8%** | **73.58%** |
45
-
46
- The shipped `drone-v1.pt` is the seed-13 checkpoint. `drone-v1.onnx` is its
47
- export (opset 18, dynamic batch, CPU parity verified).
48
-
49
- ## Input contract
50
-
51
- - Audio: mono, 22,050 Hz PCM (decode with any standard decoder, e.g. ffmpeg).
52
- - Features: `librosa.cqt`, 24 bins per octave, 208 bins from two semitones
53
- below C1 (fmin ~ 25.957 Hz), hop 4096, `log1p` amplitude, shape
54
- `(208, frames)`. The 192 core bins are rows 8..199; the 16 outer rows are
55
- the +/-(4) semitone shift pad.
56
- - Model input: `(batch, 192, frames)` — the core window of the CQT, any length
57
- >= 1 frame (the head pools over time).
58
- - Output: 24 logits. Class `i` = tonic `i // 12` semitones from C, mode
59
- `i % 2` (0 = major, 1 = minor).
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
60
 
61
- ## Usage
62
 
63
- Python (PyTorch):
 
 
64
 
65
  ```python
66
  from drone import Predictor
67
 
68
- predictor = Predictor() # resolves drone-v1.pt by default
69
- print(predictor.predict("track.mp3")) # (tonic, mode, softmax probability)
 
70
  ```
71
 
72
- Or raw, with the ONNX file:
 
 
 
 
 
 
 
73
 
74
  ```python
 
75
  import numpy as np
76
  import onnxruntime as ort
77
- import librosa
78
 
79
- y, _ = librosa.load("track.mp3", sr=22050, mono=True)
80
- cqt = np.log1p(np.abs(librosa.cqt(
81
- y, sr=22050, hop_length=4096, fmin=25.95661061576629,
82
- n_bins=208, bins_per_octave=24)))
83
- x = cqt[8:200].astype(np.float32)[None]
84
- logits = ort.InferenceSession("drone-v1.onnx").run(["logits"], {"features": x})[0]
85
- ```
86
 
87
- ## Training
 
 
 
 
 
 
88
 
89
- Beatport EDM Key Dataset previews (1,363 of 1,486 usable; 933 minor / 430
90
- major), K10 protocol: 320-frame crops, +/-(4) semitone shift as an exact
91
- re-slice with a transposed label, AdamW 1e-3, cosine 120 epochs, batch 32,
92
- plain cross-entropy. Audio is (c) Beatport; annotations are CC BY-SA 4.0.
93
- Full story in [`docs/training.md`](https://github.com/lvndry/drone/blob/main/docs/training.md)
94
- in the source repository.
95
-
96
- ## Caveats
97
 
98
- Trained on 2-minute EDM previews; expect lower accuracy on long, genre-diverse
99
- recordings. Softmax values are uncalibrated.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  license: mit
3
+ library_name: pytorch
4
+ pipeline_tag: audio-classification
5
  tags:
6
  - audio
7
  - music
8
  - key-detection
9
+ - music-information-retrieval
10
+ - dj
11
  - constant-q-transform
12
  - cnn
13
+ - onnx
14
+ - safetensors
15
  model-index:
16
+ - name: drone-v2
17
  results:
18
  - task:
19
  name: Musical key detection
20
+ type: audio-classification
21
  dataset:
22
+ name: GiantSteps Key with GiantSteps+ expert labels, single-key tracks (457)
23
+ type: giantsteps-plus
24
  metrics:
25
+ - name: Exact accuracy
26
+ type: accuracy
27
+ value: 87.96
28
+ verified: false
29
+ - task:
30
+ name: Musical key detection
31
+ type: audio-classification
32
+ dataset:
33
+ name: GiantSteps Key, original labels (604)
34
+ type: giantsteps-key
35
+ metrics:
36
+ - name: Exact accuracy
37
+ type: accuracy
38
+ value: 69.87
39
+ verified: false
40
+ - name: MIREX weighted score
41
  type: other
42
+ value: 75.45
43
+ verified: false
44
  ---
45
 
46
+ # drone-v2: musical key detection on a CPU
47
+
48
+ drone-v2 tells you the musical key of a song: one of 24 keys, the 12 tonics
49
+ (C, C#, D ... B) each in major or minor. It is a 109,128-parameter
50
+ convolutional network that reads a constant-Q spectrogram of the whole track
51
+ and runs in about 60 ms on one CPU core for a two-minute track. It was built for DJ software
52
+ ([Euphoria](https://github.com/lvndry/euphoria)) and works anywhere you can
53
+ run PyTorch, ONNX Runtime, or a few hundred lines of native code.
54
+
55
+ | | |
56
+ |---|---|
57
+ | Output | One of 24 keys, with a softmax score for each |
58
+ | Size | 109,128 parameters; 0.44 MB of float32 weights |
59
+ | Speed | About 0.3 s per two-minute track on one core of an Apple M5, of which 57 ms is the network |
60
+ | Source | [github.com/lvndry/drone](https://github.com/lvndry/drone) (MIT) |
61
+ | Previous version | `drone-v1`, still in this repository |
62
+
63
+ ## How accurate is it?
64
+
65
+ All detectors below were run on the same 604 GiantSteps previews (two-minute
66
+ excerpts of electronic dance music) and scored against the same labels.
67
+ **Exact** means the predicted key equals the reference key.
68
+
69
+ | Detector | Original GiantSteps labels (604) | Expert labels (571) | Expert labels, single-key tracks (457) |
70
+ |---|---:|---:|---:|
71
+ | libkeyfinder 2.2.8 | 60.4% | 69.9% | 76.2% |
72
+ | madmom KEY_CNN | 67.9% | 77.4% | 83.4% |
73
+ | drone-v1 | 68.5% | 77.4% | 83.4% |
74
+ | **drone-v2** | **69.9%** | **80.7%** | **88.0%** |
75
+
76
+ Why three columns? The original GiantSteps labels agree with the
77
+ [GiantSteps+ expert re-annotation](https://zenodo.org/records/4153506) on only
78
+ 82% of tracks, so part of every detector's "error" on the original labels is
79
+ label error. The single-key column keeps only tracks the experts marked with
80
+ one key throughout, the cleanest reference available.
81
+
82
+ On the expert single-key tracks, drone-v2 is ahead of libkeyfinder by 11.8
83
+ points (paired 95% interval 8.1 to 15.5) and of drone-v1 by 4.6 points
84
+ (2.0 to 7.4). On the half of those tracks never used for any development
85
+ decision, the gaps are 12.4 and 6.2 points.
86
+
87
+ Outside dance music, on 500 tracks of mixed genres from the Free Music Archive
88
+ ([FMAKv2](https://zenodo.org/records/12759100)), drone-v2 scores 75.0% exact
89
+ against drone-v1's 65.4%: 72.4% and 77.6% on the two 250-track panels, a gain
90
+ of 9 to 10 points on each.
91
+
92
+ ### What these numbers do not say
93
+
94
+ - **Not compared with Mixed In Key.** Published Mixed In Key results (85 to
95
+ 89% exact) come from other tracks, labels and scoring, so they are not
96
+ comparable with this table.
97
+ - **GiantSteps is not a blind test for this model family.** It was used while
98
+ developing Drone, though never to train it. The 604 previews share no track
99
+ with the training set.
100
+ - **On the original GiantSteps labels**, a larger published model (KeyMyna,
101
+ 22M parameters) reports 72.0% exact, above drone-v2's 69.9%.
102
+ - **Previews, not full mixes.** GiantSteps tracks are two-minute excerpts and
103
+ most training audio is 30-second clips or previews. Full-length tracks work
104
+ (the network averages over time), but accuracy on 6 to 10 minute club tracks
105
+ has not been measured separately.
106
+ - **One key per track.** A track that changes key gets the key the network
107
+ hears most.
108
+ - **The scores are not probabilities.** The softmax output ranks keys; a score
109
+ of 0.8 does not mean the key is right 80% of the time.
110
+
111
+ ## Use it
112
+
113
+ ### Python
114
+
115
+ In a project managed by [uv](https://docs.astral.sh/uv/):
116
+
117
+ ```bash
118
+ uv add "drone-key @ git+https://github.com/lvndry/drone"
119
+ ```
120
 
121
+ With pip:
122
 
123
+ ```bash
124
+ pip install "drone-key @ git+https://github.com/lvndry/drone"
125
+ ```
126
 
127
  ```python
128
  from drone import Predictor
129
 
130
+ predictor = Predictor() # loads drone-v2; reuse it for many files
131
+ tonic, mode, score = predictor.predict("track.mp3")
132
+ print(tonic, mode) # e.g. "A minor"
133
  ```
134
 
135
+ `ffmpeg` must be installed: the package decodes audio with it. From a shell:
136
+ `python -m drone track.mp3`.
137
+
138
+ ### Any language, with ONNX
139
+
140
+ `drone-v2.onnx` takes the spectrogram and returns 24 logits. You compute the
141
+ spectrogram yourself; this is the exact recipe, checked against the Python
142
+ package on 20 of 20 test tracks:
143
 
144
  ```python
145
+ import librosa
146
  import numpy as np
147
  import onnxruntime as ort
 
148
 
149
+ TONICS = ["C", "C#", "D", "D#", "E", "F", "F#", "G", "G#", "A", "A#", "B"]
 
 
 
 
 
 
150
 
151
+ samples, _ = librosa.load("track.mp3", sr=22050, mono=True)
152
+ lowest = librosa.note_to_hz("C1") * 2 ** (-4 / 12) # four semitones below C1
153
+ transform = librosa.cqt(
154
+ samples, sr=22050, hop_length=4096, fmin=lowest, n_bins=208, bins_per_octave=24
155
+ )
156
+ spectrogram = np.log1p(np.abs(transform))
157
+ features = spectrogram[8:200].astype(np.float32)[None] # (1, 192, frames)
158
 
159
+ session = ort.InferenceSession("drone-v2.onnx")
160
+ logits = session.run(["logits"], {"features": features})[0][0]
161
+ key_class = int(logits.argmax())
162
+ print(TONICS[key_class // 2], "minor" if key_class % 2 else "major")
163
+ ```
 
 
 
164
 
165
+ ### The contract, for a native port
166
+
167
+ | Step | Specification |
168
+ |---|---|
169
+ | Audio | Mono (average the channels), 22,050 Hz |
170
+ | Spectrogram | Constant-Q transform as in `librosa.cqt` 1.0: 208 bins, 24 per octave, lowest bin four semitones below C1 (25.9565 Hz), hop 4,096 samples, default Hann window and filter scale |
171
+ | Values | `log1p` of the magnitude, float32 |
172
+ | Network input | Rows 8 to 199 of the spectrogram, shape `(batch, 192, frames)`, at least 8 frames (about 1.5 s) |
173
+ | Output | 24 logits. Class `k` is tonic `TONICS[k // 2]`, minor when `k` is odd: 0 is C major, 1 C minor, 2 C# major ... 23 B minor |
174
+
175
+ `drone-v2.safetensors` holds every weight and BatchNorm statistic in float32
176
+ under its PyTorch name, for hosts without PyTorch or ONNX Runtime. The
177
+ architecture is three blocks of two 3x3 convolutions (16, 32, 64 channels), each
178
+ convolution followed by BatchNorm and ELU, each block ending in 2x2 max pooling;
179
+ then an average over time only, which keeps the 24 frequency rows, and one
180
+ linear layer from 64 x 24 features to 24 keys. See
181
+ [`drone/model.py`](https://github.com/lvndry/drone/blob/main/drone/model.py).
182
+ Euphoria's [Rust implementation](https://github.com/lvndry/euphoria) is a
183
+ complete example.
184
+
185
+ ### Files
186
+
187
+ | File | For |
188
+ |---|---|
189
+ | `drone-v2.pt` | The Python package and PyTorch. Includes training provenance in its metadata |
190
+ | `drone-v2.onnx` | ONNX Runtime in any language. Opset 18, dynamic batch and length; matches PyTorch within 2e-5 |
191
+ | `drone-v2.json` | The ONNX file's contracts (features, labels, input layout) |
192
+ | `drone-v2.safetensors` | Native implementations |
193
+ | `drone-v1.*` | The previous release, unchanged |
194
+
195
+ ## How it was trained
196
+
197
+ - **Data:** 6,263 labelled tracks: 4,988 from FMAKv2 (Free Music Archive tracks
198
+ of many genres with key labels) and 1,275 Beatport dance-music previews from
199
+ GiantSteps-MTG-Key. The training set shares no track ID with the 604
200
+ GiantSteps Key previews.
201
+ - **Distillation:** three networks first trained on the labels alone were
202
+ averaged into a "teacher". drone-v2 learned half from the human labels and
203
+ half from the teacher's softer predictions, and also from the teacher's
204
+ predictions on 24,827 unlabelled 30-second clips (FMA medium).
205
+ - **Pitch shifting:** every training excerpt was shifted by up to 12
206
+ semitones either way, with its key label moved to match, so the network
207
+ learns every key from every song.
208
+ - **Recipe:** AdamW (learning rate 1e-3, weight decay 1e-4), cosine schedule,
209
+ 120 epochs, batch 32, 60-second crops. Seed 42; the checkpoint from epoch 88
210
+ was chosen on a held-back 10% of the training data.
211
+
212
+ The full study, including 13 experiments and everything that did not work
213
+ (larger networks, more unlabelled audio, other spectrogram views, label
214
+ cleaning), is in
215
+ [`experiments/studies/frontier-distillation`](https://github.com/lvndry/drone/blob/main/experiments/studies/frontier-distillation/REPORT.md).
216
+ The release evaluation is in
217
+ [`experiments/studies/drone-v2-release`](https://github.com/lvndry/drone/blob/main/experiments/studies/drone-v2-release/REPORT.md).
218
+
219
+ ## License
220
+
221
+ The weights and code are MIT-licensed. The training audio and labels keep their
222
+ own terms: Free Music Archive audio under each track's Creative Commons
223
+ licence, FMAKv2 labels under CC BY 4.0, GiantSteps+ annotations under
224
+ CC BY-SA 4.0, and GiantSteps-MTG-Key, whose publisher states no licence. The
225
+ K10 architecture follows the MIT-licensed MIREX 2026 submission.
drone-v2.json ADDED
The diff for this file is too large to render. See raw diff
 
drone-v2.onnx ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d0f0f1dff52150b8c3fe1b7d70958a6fe17cdb862ba921421fecefeb4d302d0f
3
+ size 462056
drone-v2.pt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f069fc37004062e5f7ac931dcf5f7414b2da805eadaa2c1b28b82f32eef6acb5
3
+ size 578449
drone-v2.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f3badc870fd1f8c79fc0cabb4e1819d55e38a3b724ea0158a63d5b781765af97
3
+ size 441456