| --- |
| license: mit |
| language: en |
| datasets: |
| - roneneldan/TinyStoriesInstruct |
| base_model: singhpratech/tiny-storyteller |
| tags: |
| - tiny |
| - instruction-tuning |
| - chat-template |
| - prompt-masking |
| - pytorch |
| - educational |
| - text-generation |
| - emc2 |
| --- |
| # e=mc² Lesson 05: Instruction tuning from scratch |
|
|
| Lesson page: https://theaivibe.org/emc2/instruction-tuning-from-scratch-tiny-storyteller |
|
|
| Lesson 05 of e=mc², a learn-by-building course by Prateek Singh. It takes `tiny-storyteller-v2` |
| (Lesson 04, repo `singhpratech/emc2-lesson-04-fine-tune`) and teaches it to follow a request, with two new |
| special tokens, a chat template and prompt masking. |
|
|
| ## Contents |
| - `lesson_05.ipynb`: the lesson, unexecuted (Colab T4, Apple-silicon Mac or CPU). |
| - `RESULTS.md`: every number from the run on 2026-10-11 (Apple Silicon Mac). |
| - `tiny-storyteller-instruct/`: the instruction-tuned model (vocab 4,098) with `chat_template.json`. |
| - `pairs/`: the instruction pairs as JSONL (`train`, `heldout`, `test50`, `familiar50`, `crossworld20`). |
| - `story_pairs.py`: builds instruction pairs from the Momo and Zuri story generators. |
|
|
| ## Result (string match: character AND place in the generated story, 50 held-out instructions) |
| | | both | character | place | |
| |---|---|---|---| |
| | before tuning | 12% | 68% | 14% | |
| | after 500 steps | 44% | 100% | 44% | |
|
|
| Held-out instructions ask for character + scene + place combinations that never appear together in training. |
| On 50 familiar combinations the same model scores 86%; across the two characters' worlds, 5% of 20. |
|
|
| ## Template |
| `<|endoftext|><|instruction|>{instruction}<|story|>{story}<|endoftext|>` - generate after `<|story|>` |
| until `<|endoftext|>`. Model class: Lesson 01's `GPT`, unchanged (in the notebook). |
|
|
| Licences: code and weights MIT. Pairs derived from TinyStories-Instruct (CDLA-Sharing-1.0) keep that licence; |
| generator pairs are MIT. |
|
|
