singhpratech/emc2-lesson-04-fine-tune

e=mc² Lesson 04: Fine-tune a small LLM, add a new character, measure the forgetting

Lesson page: https://theaivibe.org/emc2/fine-tune-small-llm-measure-catastrophic-forgetting

This repo holds the notebook, the results and the model from Lesson 04 of e=mc², a learn-by-building course by Prateek Singh. The lesson starts from Lesson 01's public model singhpratech/tiny-storyteller (5.85M parameters) and fine-tunes it on 300 stories about a new character, Zuri the zebra, then measures what it forgets.

Contents

• lesson_04.ipynb: the lesson, unexecuted. Runs on a free Colab T4, an Apple-silicon Mac or CPU.
• RESULTS.md: every number from the run on 2026-10-11 (Apple Silicon Mac).
• tiny-storyteller-v2/: the model fine-tuned with 10% replay (model.pt, config.json, tokenizer). Lesson 05 starts from it.
• zuri_stories.py: the generator for the new character's stories.

Results (perplexity, lower is better)

	Lesson 01 quiz (200 TinyStories)	100 fresh Momo stories	100 held-out Zuri stories
before	9.09	1.24	68.32
after 300 steps, Zuri only	156.08	17.35	1.17
after 300 steps, 10% replay (this model)	11.75	1.39	1.15

The Zuri and Momo quizzes are new combinations of the generators' sentence pools, so the model has seen their sentences, not their stories; numbers near 1 there mean recitation. With 10% replay the model still sometimes names the monkey "Zuri" (see RESULTS.md).

Load

Use the GPT class from the notebook (Lesson 01's, unchanged):

cfg = json.load(open(hf_hub_download(REPO, "tiny-storyteller-v2/config.json")))
state = torch.load(hf_hub_download(REPO, "tiny-storyteller-v2/model.pt"), map_location="cpu")

Data: TinyStories (CDLA-Sharing-1.0) plus generated Momo and Zuri stories. Code and weights: MIT.