README.md

metadata

license: other
tags:
  - music
  - midi
  - melody-generation
  - symbolic-music
  - safetensors

Apollo-M

Apollo-M generates monophonic melodies as MIDI from structured controls (key, number of bars, tempo, time signature, density, register, genre, mood, instrument family). It is a small decoder-only Transformer (27.5M parameters) trained from scratch on melodies extracted from MIDI collections, then finetuned in two stages. The output is MIDI; turning MIDI into audio is a separate step (any synth or soundfont).

Model details

Architecture	Decoder-only Transformer, RoPE, RMSNorm, SwiGLU, tied input/output embeddings
Size	27.5M parameters (8 layers, d_model 512, 8 heads, FFN 1536)
Context	1024 tokens (a 16-bar melody fits comfortably)
Vocabulary	352 tokens (including reserved ids)
Precision	trained with fp16 mixed precision; weights stored in fp16
Hardware	Google Colab free tier (single T4)
Author	viknesh100

Representation

Each sequence is BOS, a 9-token header, then the melody, then EOS. The melody is a series of bars: a BAR token starts each bar, followed by note events made of three tokens (POS, PITCH, DUR) on a 16th-note grid. One note takes about 3 tokens.

Header field	Values	Meaning
TEMPO	24 bins	10-BPM bins covering 40 to 279 BPM
TS	5	4/4, 3/4, 2/4, 6/8, 12/8
KEY	24	12 tonics × major/minor
BARS	16	melody length, 1 to 16 bars
DENS	8	note density (notes per bar, binned)
REG	8	register (mean pitch, bins of 8 semitones from MIDI 36)
GENRE	24	electronic, pop, classical, rock, soundtrack, jazz, ambient, folk, instrumentalpop, world, reggae, easylistening, latin, country, experimental, alternative, popfolk, metal, poprock, blues, dance, newage
MOOD	16	melodic, happy, film, christmas, relaxing, energetic, meditative, dark, love, space, slow, corporate, epic, game, motivational, inspiring
INSTR	16	General MIDI instrument family (program // 8) of the source melody track

Every header field also has an unknown token. The model saw randomly hidden fields during training, so any control can be left unspecified and the model chooses.

Training

All three stages used the same melody extraction: choose the most melody-like track of a MIDI file (monophony, pitch range, track-name hints; bass and pad programs penalised), reduce it to a monophonic line with a skyline rule, quantise to a 16th-note grid, and cut 4/8/16-bar windows. Files with multiple time signatures or a poor key fit were dropped, melodies were de-duplicated by an interval/rhythm hash, and train/validation were split by melody (never by window). Augmentation: transposition up to ±5 semitones (key and register tokens updated), small tempo jitter, 15% random hiding of header fields.

1. Pretraining ("Apollo", viknesh100/Apollo). Lakh MIDI, 178,561 source files scanned, 90,095 unique melodies kept. Header: the first 6 fields only. AdamW, lr 5e-4 → 5e-5 (cosine), warmup 500, weight decay 0.1, 128 sequences per step (32 × 4 accumulation), grad-clip 1.0. Best checkpoint at step 10000 (validation loss 0.9240).

2. Stage A: control-token finetune. Same Lakh melodies joined with MidiCaps genre and mood tags (top-1 tag per file; the 24 most frequent genres and 16 most frequent moods are used, the rest become unknown), plus the instrument family of the extracted track. 49,714 unique melodies, 415,747 training windows, 8,024 validation windows. Three new header fields were added using ids reserved in the pretrained vocabulary. lr 2e-4 → 2e-5 (cosine), warmup 200, weight decay 0.05, up to 4000 steps. Best checkpoint at step 4000.

3. Stage B: polish. Clean, human-made melodies: POP909 (2,108 windows, lead-vocal MELODY track) and the Nottingham folk database (11,783 windows). Stricter filters (key fit, large-leap fraction, bar fill). Header: genre set to pop/folk where that label exists, mood and instrument left unknown. lr 2e-5 → 2e-6, 2 epochs over the polish data, 25% of every batch replayed from the stage A data to avoid forgetting. A checkpoint counted as best only if the polish-set loss improved while the stage A validation loss stayed within +0.05 of where stage A ended. Final checkpoint: step 275 (model.safetensors).

Evaluation

Validation loss is the average cross-entropy per body token (header tokens are not scored), so it is comparable across the rows below but not with other models.

Checkpoint	Val loss, MidiCaps windows (ppl)	Val loss, polish windows (ppl)
Stage A (before polish)	0.8319 (2.30)	0.6355 (1.89)
Final (stage B)	0.8392 (2.31)	0.5868 (1.80)

Key-adherence results were not recorded when this card was generated.

Loss does not measure musicality. The checkpoints were also compared by listening to paired samples (same prompt and seed) from stage A and stage B.

Intended use

• Generating melody ideas and sketches in a requested key, length and tempo, to be edited or arranged further.
• A starting point for research on controllable symbolic music generation.

Not intended as a replacement for composers, and not evaluated for any use beyond melody sketching.

How to use

The model is a custom PyTorch architecture (not a transformers model). Everything needed (architecture, vocab, prompt parser, constrained decoder, MIDI writer) is in apollo_m.py in this repo.

# pip install torch huggingface_hub safetensors pretty_midi
import sys, os
from huggingface_hub import hf_hub_download
repo = "viknesh100/Apollo-M"
sys.path.insert(0, os.path.dirname(hf_hub_download(repo, "apollo_m.py")))   # add token="hf_..." if the repo is private
from apollo_m import load

m = load(repo)
m.make("8 bars in D minor at 90 bpm, sad piano melody", n=3, seed=1)       # writes MIDI files to ./apollo_out/

Generation uses grammar-constrained sampling: at each step only tokens that keep the sequence valid are allowed (BAR then POS then PITCH then DUR, increasing positions inside a bar, exactly the requested number of bars), so every output parses to valid MIDI. Defaults: temperature 0.9, nucleus p 0.92. Override fields directly if you prefer, e.g. m.make("8 bars", key=("A","minor"), bpm=100, program=40).

A small rule-based parser turns the prompt into header tokens (key, bars, bpm or tempo words, time signature, genre and mood words from labels.json, instrument words). The model itself does not read free text.

Files

• model.safetensors: final weights (stage B, fp16). head.weight is not stored because it is tied to emb.weight.
• stageA/model.safetensors: weights before the polish stage.
• config.json: architecture, lineage, final step and validation losses.
• vocab.json, labels.json: token list and the genre / mood label lists.
• best/, last/, stageB/: training checkpoints (.pt, pickle-based; they include optimizer state). For resuming training only. Load pickle files only from sources you trust.

Limitations

• Monophonic single-line melodies only: no chords, accompaniment, dynamics or articulation (all notes have the same velocity). Chords and arrangement are out of scope.
• 16th-note grid: triplets and fine expressive timing are quantised away. Supported meters: 4/4, 3/4, 2/4, 6/8, 12/8. Melodies are 1 to 16 bars.
• Training melodies were extracted automatically from multi-track MIDI. The chosen track is sometimes not the real melody, and noise from that extraction is in the data.
• Genre and mood tags come from MidiCaps' automatic labelling and are noisy; use them as soft style hints, not guarantees. The instrument control picks a General MIDI family from the source track and does not model the timbre of a specific instrument.
• Requested key is followed approximately; relative major/minor keys can be confused. Requested bars and meter are enforced by the decoder.
• The data is dominated by Western tonal music (pop, rock, folk, classical); other traditions are poorly covered.
• The model can reproduce phrases from its training melodies. Check outputs before publishing them.

Licensing and data provenance

This card is not legal advice. The model weights were trained on melodies derived from the sources below. The source compositions may be under copyright even where the dataset packaging carries an open licence; check each source's terms before any commercial use or redistribution of generated material.

Source	Used for	Note
Lakh MIDI Dataset (Raffel, 2016)	pretraining, stage A melodies	scraped MIDI of existing songs
MidiCaps (Melechovsky et al., 2024)	genre and mood tags	captions and attributes for Lakh files
POP909 (Wang et al., ISMIR 2020)	stage B polish	piano arrangements and lead-melody transcriptions of existing pop songs
Nottingham Music Database (E. Foxley; cleaned MIDI release by Jukedeck)	stage B polish	traditional folk tunes

Citations

• Raffel, C. Learning-Based Methods for Comparing Sequences, with Applications to Audio-to-MIDI Alignment and Matching. PhD thesis, Columbia University, 2016 (Lakh MIDI Dataset).
• Melechovsky, J. et al. MidiCaps: A large-scale MIDI dataset with text captions. arXiv:2406.02255, 2024.
• Wang, Z. et al. POP909: A Pop-song Dataset for Music Arrangement Generation. ISMIR 2020 (arXiv:2008.07142).