Author line: drop co-author mention

tiny_llm_from_scratch.ipynb

@@ -2,12 +2,12 @@
  "cells": [
   {
    "cell_type": "markdown",
-   "id": "100683d2",
+   "id": "d6dbef8d",
    "metadata": {},
    "source": [
     "# Tiny Storyteller: building a language model from scratch\n",
     "\n",
-    "**Author:** singhpratech, with Claude. **Runs on:** Google Colab (free T4 GPU), or a Mac with Apple Silicon, or CPU.\n",
+    "**Author:** singhpratech. **Runs on:** Google Colab (free T4 GPU), or a Mac with Apple Silicon, or CPU.\n",
     "\n",
     "## Purpose\n",
     "Learn how a language model (an \"LLM\") is really built, by building one from nothing.\n",
@@ -47,7 +47,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "d5bdeee4",
+   "id": "52bce3d0",
    "metadata": {},
    "source": [
     "## 0. Before any code: Momo, Ella, and what we are about to build\n",
@@ -59,7 +59,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "10284f1b",
+   "id": "18badc67",
    "metadata": {},
    "source": [
     "### Meet the characters\n",
@@ -82,7 +82,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "1ba6140a",
+   "id": "c6b124e5",
    "metadata": {},
    "source": [
     "### 0.1 What is PyTorch?\n",
@@ -108,7 +108,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "f4e250e6",
+   "id": "a8557a5e",
    "metadata": {},
    "source": [
     "### 0.2 What is a model, really?\n",
@@ -123,7 +123,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "c09d3c35",
+   "id": "f05878b3",
    "metadata": {},
    "source": [
     "### 0.3 What does a language model actually do?\n",
@@ -142,7 +142,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "0cfee0ff",
+   "id": "8774b740",
    "metadata": {},
    "source": [
     "### 0.4 What happens when you ask Momo a question?\n",
@@ -181,7 +181,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "7726ccdd",
+   "id": "5a0ae3aa",
    "metadata": {},
    "source": [
     "### 0.5 How attention looks things up: the story circle\n",
@@ -213,7 +213,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "2c0721dd",
+   "id": "e786fabb",
    "metadata": {},
    "source": [
     "### 0.6 Ella's shelf: the KV cache\n",
@@ -240,7 +240,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "805418a3",
+   "id": "b4e0da60",
    "metadata": {},
    "source": [
     "### 0.7 The strange names, decoded\n",
@@ -292,7 +292,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "1c616475",
+   "id": "91659f55",
    "metadata": {},
    "source": [
     "### The one-sentence version\n",
@@ -306,7 +306,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "30ffb0ee",
+   "id": "eabf2c36",
    "metadata": {},
    "source": [
     "## Live flow diagram\n",
@@ -323,7 +323,7 @@
   {
    "cell_type": "code",
    "execution_count": null,
-   "id": "54d3386e",
+   "id": "1baea7bb",
    "metadata": {},
    "outputs": [],
    "source": [
@@ -448,7 +448,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "eaf77536",
+   "id": "0c0f4f61",
    "metadata": {},
    "source": [
     "### Setup (still Part 0)\n",
@@ -460,7 +460,7 @@
   {
    "cell_type": "code",
    "execution_count": null,
-   "id": "ef405d64",
+   "id": "478042b4",
    "metadata": {},
    "outputs": [],
    "source": [
@@ -475,7 +475,7 @@
   {
    "cell_type": "code",
    "execution_count": null,
-   "id": "ccc7f98a",
+   "id": "00b89e95",
    "metadata": {},
    "outputs": [],
    "source": [
@@ -492,7 +492,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "e951d497",
+   "id": "33dd01e2",
    "metadata": {},
    "source": [
     "## 1. Tensors and gradients: the two building blocks\n",
@@ -511,7 +511,7 @@
   {
    "cell_type": "code",
    "execution_count": null,
-   "id": "093f532c",
+   "id": "0b5c0de4",
    "metadata": {},
    "outputs": [],
    "source": [
@@ -554,7 +554,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "d6380281",
+   "id": "2dfe6dad",
    "metadata": {},
    "source": [
     "### How matrix multiplication actually happens\n",
@@ -578,7 +578,7 @@
   {
    "cell_type": "code",
    "execution_count": null,
-   "id": "fde92e0a",
+   "id": "c94b0555",
    "metadata": {},
    "outputs": [],
    "source": [
@@ -635,7 +635,7 @@
   {
    "cell_type": "code",
    "execution_count": null,
-   "id": "1453c25f",
+   "id": "e39e048b",
    "metadata": {},
    "outputs": [],
    "source": [
@@ -648,7 +648,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "f3f3aa44",
+   "id": "3febc443",
    "metadata": {},
    "source": [
     "### The gradient: which way to turn each dial\n",
@@ -671,7 +671,7 @@
   {
    "cell_type": "code",
    "execution_count": null,
-   "id": "6711ff33",
+   "id": "4e72fa2e",
    "metadata": {},
    "outputs": [],
    "source": [
@@ -711,7 +711,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "20984aa8",
+   "id": "d8c2b93f",
    "metadata": {},
    "source": [
     "### Momo in the foggy jungle: thorn bushes, marks, and the slope\n",
@@ -759,7 +759,7 @@
   {
    "cell_type": "code",
    "execution_count": null,
-   "id": "059a673c",
+   "id": "2c8b20dd",
    "metadata": {},
    "outputs": [],
    "source": [
@@ -870,7 +870,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "71aef56e",
+   "id": "ea803726",
    "metadata": {},
    "source": [
     "### How does Momo feel the slope without walking? The blame game\n",
@@ -906,7 +906,7 @@
   {
    "cell_type": "code",
    "execution_count": null,
-   "id": "fb4448d7",
+   "id": "dc8e8406",
    "metadata": {},
    "outputs": [],
    "source": [
@@ -1007,7 +1007,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "641405e9",
+   "id": "fbbc8f63",
    "metadata": {},
    "source": [
     "Now the blame game in three lines of PyTorch. Three dials, one mistake, `backward()`, and\n",
@@ -1021,7 +1021,7 @@
   {
    "cell_type": "code",
    "execution_count": null,
-   "id": "8728bb4c",
+   "id": "db819628",
    "metadata": {},
    "outputs": [],
    "source": [
@@ -1039,7 +1039,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "e4ba1c30",
+   "id": "99aa0e23",
    "metadata": {},
    "source": [
     "## 2. Data: Ella's bedtime stories (TinyStories)\n",
@@ -1055,7 +1055,7 @@
   {
    "cell_type": "code",
    "execution_count": null,
-   "id": "5f3df28e",
+   "id": "fc46895d",
    "metadata": {},
    "outputs": [],
    "source": [
@@ -1073,7 +1073,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "1b4619b9",
+   "id": "d1eca3fe",
    "metadata": {},
    "source": [
     "### Ella's own stories about Momo\n",
@@ -1096,7 +1096,7 @@
   {
    "cell_type": "code",
    "execution_count": null,
-   "id": "2bd4ede4",
+   "id": "215e5fa4",
    "metadata": {},
    "outputs": [],
    "source": [
@@ -1207,7 +1207,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "21d005a6",
+   "id": "550278c3",
    "metadata": {},
    "source": [
     "## 3. Momo's first trick: the bigram\n",
@@ -1233,7 +1233,7 @@
   {
    "cell_type": "code",
    "execution_count": null,
-   "id": "0cb96cb7",
+   "id": "0b4022c7",
    "metadata": {},
    "outputs": [],
    "source": [
@@ -1263,7 +1263,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "3ed9f401",
+   "id": "fc2334a9",
    "metadata": {},
    "source": [
     "Now the same thing in code: first the letter vocabulary and the encode/decode pair."
@@ -1272,7 +1272,7 @@
   {
    "cell_type": "code",
    "execution_count": null,
-   "id": "485e4285",
+   "id": "0ebe181d",
    "metadata": {},
    "outputs": [],
    "source": [
@@ -1291,7 +1291,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "6a2f2003",
+   "id": "b5cc8528",
    "metadata": {},
    "source": [
     "Next, Momo's first lessons. Every lesson in this notebook, here and for the big model in\n",
@@ -1312,7 +1312,7 @@
   {
    "cell_type": "code",
    "execution_count": null,
-   "id": "fd96c9d6",
+   "id": "c1c9d084",
    "metadata": {},
    "outputs": [],
    "source": [
@@ -1361,7 +1361,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "dab65e7f",
+   "id": "be9bfbb8",
    "metadata": {},
    "source": [
     "Momo learned *something* (the loss dropped, and he produces word-ish shapes with spaces), but\n",
@@ -1371,7 +1371,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "3793556f",
+   "id": "2f160514",
    "metadata": {},
    "source": [
     "## 4. A real tokenizer: how Momo decides where to bite\n",
@@ -1398,7 +1398,7 @@
   {
    "cell_type": "code",
    "execution_count": null,
-   "id": "cefde3e3",
+   "id": "026dd481",
    "metadata": {},
    "outputs": [],
    "source": [
@@ -1428,7 +1428,7 @@
   {
    "cell_type": "code",
    "execution_count": null,
-   "id": "614ca18c",
+   "id": "4f80df23",
    "metadata": {},
    "outputs": [],
    "source": [
@@ -1455,7 +1455,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "06ddcccd",
+   "id": "48a8b12f",
    "metadata": {},
    "source": [
     "Now every story becomes numbers. We glue all the lesson stories into one long stream of token\n",
@@ -1466,7 +1466,7 @@
   {
    "cell_type": "code",
    "execution_count": null,
-   "id": "81094684",
+   "id": "d96b25f1",
    "metadata": {},
    "outputs": [],
    "source": [
@@ -1480,7 +1480,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "2f94f049",
+   "id": "2c7ab127",
    "metadata": {},
    "source": [
     "## 5. Attention, by hand: the story circle in numbers\n",
@@ -1506,7 +1506,7 @@
   {
    "cell_type": "code",
    "execution_count": null,
-   "id": "07e576a3",
+   "id": "69abace2",
    "metadata": {},
    "outputs": [],
    "source": [
@@ -1531,7 +1531,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "6843eaec",
+   "id": "c24308af",
    "metadata": {},
    "source": [
     "### Ella's shelf (the KV cache), in about a dozen lines\n",
@@ -1545,7 +1545,7 @@
   {
    "cell_type": "code",
    "execution_count": null,
-   "id": "e87ea92c",
+   "id": "84949030",
    "metadata": {},
    "outputs": [],
    "source": [
@@ -1575,7 +1575,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "39d7754c",
+   "id": "626f6cd8",
    "metadata": {},
    "source": [
     "## 6. A tiny GPT: the whole of Momo\n",
@@ -1610,7 +1610,7 @@
   {
    "cell_type": "code",
    "execution_count": null,
-   "id": "b6cebe94",
+   "id": "af4363f8",
    "metadata": {},
    "outputs": [],
    "source": [
@@ -1654,7 +1654,7 @@
   {
    "cell_type": "code",
    "execution_count": null,
-   "id": "32a22e9f",
+   "id": "c54e5e5e",
    "metadata": {},
    "outputs": [],
    "source": [
@@ -1765,7 +1765,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "85705754",
+   "id": "246a4e91",
    "metadata": {},
    "source": [
     "**Four things in that code we have not talked about.**\n",
@@ -1783,7 +1783,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "c0e60a25",
+   "id": "31bb615e",
    "metadata": {},
    "source": [
     "## 7. Training: Ella's bedtime lessons\n",
@@ -1818,7 +1818,7 @@
   {
    "cell_type": "code",
    "execution_count": null,
-   "id": "ea11fdfd",
+   "id": "0edd20df",
    "metadata": {},
    "outputs": [],
    "source": [
@@ -1861,7 +1861,7 @@
   {
    "cell_type": "code",
    "execution_count": null,
-   "id": "9aa473c0",
+   "id": "7ed19704",
    "metadata": {},
    "outputs": [],
    "source": [
@@ -1929,7 +1929,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "87eb0edd",
+   "id": "ce14f79b",
    "metadata": {},
    "source": [
     "**How to read the numbers.** The loss is Ella's average \"how surprised were you\" per token.\n",
@@ -1943,7 +1943,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "d65d3fda",
+   "id": "13ba6e8b",
    "metadata": {},
    "source": [
     "## 8. Momo tells a story: the use case\n",
@@ -1962,7 +1962,7 @@
   {
    "cell_type": "code",
    "execution_count": null,
-   "id": "a7c4007b",
+   "id": "b8b00a38",
    "metadata": {},
    "outputs": [],
    "source": [
@@ -1980,7 +1980,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "6e5636c2",
+   "id": "fc77f782",
    "metadata": {},
    "source": [
     "**Look closely at what we saw.** The Momo stories are almost word for word the sentences we\n",
@@ -1994,7 +1994,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "bf995053",
+   "id": "24859a73",
    "metadata": {},
    "source": [
     "## 9. Put Momo's book in the library: the Hugging Face Hub\n",
@@ -2019,7 +2019,7 @@
   {
    "cell_type": "code",
    "execution_count": null,
-   "id": "56b44a10",
+   "id": "7e55d046",
    "metadata": {},
    "outputs": [],
    "source": [
@@ -2063,7 +2063,7 @@
   {
    "cell_type": "code",
    "execution_count": null,
-   "id": "52610e5a",
+   "id": "6cacb9a8",
    "metadata": {},
    "outputs": [],
    "source": [
@@ -2084,7 +2084,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "6f624dcb",
+   "id": "77a4c8d7",
    "metadata": {},
    "source": [
     "## 10. Loading it back later\n",
@@ -2096,7 +2096,7 @@
   {
    "cell_type": "code",
    "execution_count": null,
-   "id": "9d454e34",
+   "id": "59455da3",
    "metadata": {},
    "outputs": [],
    "source": [
@@ -2119,7 +2119,7 @@
   },
   {
    "cell_type": "markdown",
-   "id": "755c1fb3",
+   "id": "bef94f79",
    "metadata": {},
    "source": [
     "## What we built, in one paragraph\n",