---
license: apache-2.0
library_name: transformers
pipeline_tag: text-generation
tags:
- slmkit
- llama
- from-scratch
- abc-music
- music
- abc-notation
- folk-music
- symbolic-music
---

# abc-folk:3

A 864,256-parameter Llama-style language model trained from scratch with
[slmkit](https://github.com/m87techlabs/slmkit) for the `abc_music` project. It is a **fine-tuned (SFT) model**: it answers requests in the format it was fine-tuned on.

## How to prompt it

Generation starts from the prompt and stops at `<eos>` (token ID 1). An example prompt, exactly
as the evaluation used it:

```
% a reel in D major, 2/2 time
```

Serve it and ask over HTTP. A copy downloaded from the Hugging Face Hub goes in
`$SLM_HOME/models/abc-folk/3/` first; that is where `slm serve` looks for `abc-folk:3`.

```bash
uv run slm serve --model abc-folk:3
curl -s localhost:8000/generate -H 'content-type: application/json' \
  -d '{"prompt": "% a reel in D major, 2/2 time\n", "seed": 0}'
```

Or load it with Hugging Face `transformers`; the directory is a standard `LlamaForCausalLM`
checkpoint:

```python
from transformers import AutoModelForCausalLM, AutoTokenizer
path = "$SLM_HOME/models/abc-folk/3"  # or its Hugging Face Hub repo ID, if it was published there
tok = AutoTokenizer.from_pretrained(path)
model = AutoModelForCausalLM.from_pretrained(path)
```

## Model

| | |
|---|---|
| architecture | `LlamaForCausalLM` (RMSNorm, RoPE, SwiGLU, tied embeddings) |
| parameters | 864,256 (853,120 non-embedding) |
| vocabulary | 87 tokens |
| context | 512 tokens |
| weights | `model.safetensors`, fp32, 3.5 MB |

## Training

| | |
|---|---|
| run | `run-9b890db6dced` (abc-baseline-sft), checkpoint `best` at step 1,000 |
| tokens seen (this stage) | 16,384,000 |
| validation loss | 1.1092 |
| GPU-hours (this stage) | 0.006 |
| tokenizer | `tk-1030590a89ca` |
| code | slmkit `c34aed6` |

Full lineage back to the raw data: `uv run slm lineage abc-folk:3`.

## Evaluation

From eval report `ev-0ddc32e2a9`: 3 sampling seeds × 200 samples, prompted with the project's **sft** prompts. Mean ± standard deviation across seeds.

| metric | score |
|---|---|
| ended | 0.982 ± 0.006 |
| length | 259.345 ± 4.707 |
| plays | 0.730 ± 0.048 |
| bar_accuracy | 0.851 ± 0.023 |
| ends_on_tonic | 0.340 ± 0.026 |
| novelty | 0.998 ± 0.001 |

Metric definitions: the project's README and `docs/concepts/evaluation.md`.

## Verification

Checked before it was published, over 94 tokens: slmkit reloads the files with max
|Δlogit| = 0.0e+00 (identical). `transformers` 5.17.0 loads them with max |Δlogit| = 0.0e+00 and identical token IDs.

## Data and licence

Trained on 4,248 tunes in ABC notation from three **public-domain** tune books: Ryan's Mammoth
Collection (1883), O'Neill's Music of Ireland (1903) and Aird's Selection of Airs (1778–1803), as
transcribed by volunteers and shipped in the [`music21`](https://github.com/cuthbertLab/music21)
package (BSD-3-Clause).

**One open question, stated plainly.** The printed books are public domain, but the transcription
files carry no licence of their own. Faithful transcriptions of public-domain scores are generally
not treated as new creative work, but that is an assumption, not a grant. If a transcriber or
rights-holder objects, the collection is removed, the model retrained, and this version withdrawn.
The Session's data was deliberately *not* used: its licence forbids training language models.

The weights are released under Apache-2.0, the same licence as slmkit.

More detail: [`projects/abc_music/README.md`](https://github.com/m87techlabs/slmkit/blob/main/projects/abc_music/README.md) in the slmkit repository.

## Limitations

- **Context of 512 tokens.** slmkit's sampler keeps only the last 512 tokens as
  context; positions beyond that were never trained.
- **Fixed vocabulary of 87 tokens.** Text the tokenizer has no token for becomes
  `<unk>`, which the model learned nothing about. `slm serve` reports how many prompt tokens that
  affected.
- **Small and narrow.** It knows only its training domain. The scores above are averages;
  individual samples can be wrong, so check them with the project's graders.