---
license: mit
language: en
datasets:
- roneneldan/TinyStoriesInstruct
base_model: singhpratech/tiny-storyteller
tags:
- tiny
- instruction-tuning
- chat-template
- prompt-masking
- pytorch
- educational
- text-generation
- emc2
---
# e=mc² Lesson 05: Instruction tuning from scratch

Lesson page: https://theaivibe.org/emc2/instruction-tuning-from-scratch-tiny-storyteller

Lesson 05 of e=mc², a learn-by-building course by Prateek Singh. It takes `tiny-storyteller-v2`
(Lesson 04, repo `singhpratech/emc2-lesson-04-fine-tune`) and teaches it to follow a request, with two new
special tokens, a chat template and prompt masking.

## Contents
- `lesson_05.ipynb`: the lesson, unexecuted (Colab T4, Apple-silicon Mac or CPU).
- `RESULTS.md`: every number from the run on 2026-10-11 (Apple Silicon Mac).
- `tiny-storyteller-instruct/`: the instruction-tuned model (vocab 4,098) with `chat_template.json`.
- `pairs/`: the instruction pairs as JSONL (`train`, `heldout`, `test50`, `familiar50`, `crossworld20`).
- `story_pairs.py`: builds instruction pairs from the Momo and Zuri story generators.

## Result (string match: character AND place in the generated story, 50 held-out instructions)
| | both | character | place |
|---|---|---|---|
| before tuning | 12% | 68% | 14% |
| after 500 steps | 44% | 100% | 44% |

Held-out instructions ask for character + scene + place combinations that never appear together in training.
On 50 familiar combinations the same model scores 86%; across the two characters' worlds, 5% of 20.

## Template
`<|endoftext|><|instruction|>{instruction}<|story|>{story}<|endoftext|>` - generate after `<|story|>`
until `<|endoftext|>`. Model class: Lesson 01's `GPT`, unchanged (in the notebook).

Licences: code and weights MIT. Pairs derived from TinyStories-Instruct (CDLA-Sharing-1.0) keep that licence;
generator pairs are MIT.