redline-v1

AI timing for osu! — a 45M-parameter frame-level model that turns raw audio into osu! timing (redlines: BPM / offset / meter), built for the songs that are hard to time by hand: live drummers, metal, rubato, accelerandos. Output is a starting point to verify by ear.

Try it in your browser: https://redline-kohl.vercel.app/ Code and solver: https://github.com/Tiger14n/Redline

Model

Encoder-only transformer (rotary attention, conv frontend over log-mel + onset flux), per-frame outputs at 50 fps:

head	output
beat	beat-here logit
down	downbeat (measure start) logit
off	sub-frame offset (recovers ~2-5 ms precision from 20 ms frames)
tempo	BPM-bin logits (per-frame tempo curve)
change	major (audible) tempo/meter change logit
red	notation logit — where a human mapper writes a redline, including convention re-anchors

d_model 512, depth 14, 8 heads, FFN 2048. Trained on ranked osu! beatmaps (three-stage lineage; configs in the repo) on a single RTX 3080.

Results (held-out data)

Ranked mapsets excluded from training by song-level dedup, scored against the ranked map's human timing.

Model heads, 1000 held-out mapsets (frame-level, ±40 ms):

genre	maps	beat F1	downbeat F1	offset MAE
Pop	285	0.966	0.885	4.7 ms
Anime	89	0.969	0.895	5.2 ms
Rock	192	0.963	0.863	4.8 ms
Electronic	224	0.962	0.886	5.1 ms
Metal	50	0.947	0.855	5.2 ms
Video game	100	0.944	0.863	5.1 ms
other	60	0.916	0.801	5.6 ms
all	1000	0.959	0.873	5.0 ms

End-to-end (audio → redlines), 366 held-out mapsets, ~50+ per genre:

genre	maps	beats within 10 ms	typical map	redline match P / R
Pop	55	89.3 %	100.0 %	0.51 / 0.54
Metal	48	88.2 %	96.9 %	0.55 / 0.43
Rock	51	87.3 %	95.2 %	0.42 / 0.42
Electronic	49	85.0 %	100.0 %	0.57 / 0.57
Video game	54	82.7 %	98.7 %	0.47 / 0.50
Anime	55	80.9 %	98.8 %	0.65 / 0.50
other	54	80.2 %	92.7 %	0.40 / 0.52
all	366	84.7 %	98.7 %	0.51 / 0.50

Genre gaps are mostly sample size: on the 1000-map run the bootstrap intervals overlap (metal − anime = +3.2 pt, 95 % CI [−4.1, +10.5]). Read the "all" row.

The typical song comes out on-grid (median 93–100 % per genre; 240 of 366 maps above 90 %). Of the 185 songs a human timed with one redline, the model writes exactly one on 147.

Known failure modes

~7 % of maps score below 50 %. Classifying those by the shape of the timing error gives four distinct causes:

1. The reference is offset, not the model (~27 %). A perfectly constant displacement, ~12 ms, so a 10 ms tolerance scores ~0 % while the grid is ~2 ms from right. Checked against the audio itself (spectral-flux onsets, detector latency calibrated out): on 19 of 20 such maps the model is 2.0 ms from the audio-true beat and the mapper 16.0 ms away — outside tolerance from their own song. Not an MP3-delay artifact.
2. Metrical-level disagreement (~17 %). ×2, ÷2 or ×2⁄3 of the mapper's BPM — the same pulse at a different metrical level, a notation convention rather than an acoustic fact.
3. Genuinely hard audio (~40 %). Live recordings, solo-piano rubato, jazz swing. Beat activation drops to 0.650 vs 0.942 on healthy maps; neither grid sits on strong onsets.
4. Odd tempo ramps (~4 %). Too few maps to conclude anything.

Excluding only the proven-offset maps moves the 1000-map benchmark from 88.5 % to 90.2 % and the failure rate from 7.0 % to 5.1 %. The reference set is not hand-cleaned, so numbers are reported as measured.

Usage

pip install -r requirements.txt   # from the repo
hf download Tiger14n/redline-v1 redline-v1.pt --local-dir models/
python -m redline.generate_timing song.mp3 --osz

The checkpoint contains model (state dict) and args (data representation + architecture), which redline.infer.Model2Beats uses to reconstruct the exact training-time pipeline.