# Apollo

Decoder-only Transformer (RoPE, RMSNorm, SwiGLU, tied embeddings) that generates monophonic melodies as symbolic tokens (BAR / POS / PITCH / DUR) conditioned on a header (tempo, time signature, key, bars, density, register, ...).

- `model.safetensors`: fp16 weights. `lm_head` is tied to `emb.weight`, so `head.weight` is not stored; build the model, load with `strict=False`, and the tie fills it.
- `config.json`: architecture (`cfg`), context length, training step and validation loss.
- `vocab.json`: token list in id order.
- `best/` and `last/`: training checkpoints (`.pt`), used for resuming; load only from sources you trust.

Trained on melodies extracted from the Lakh MIDI dataset. Check the licensing of the source data before any commercial use.