Correct the card: the model is now trained on the 19.2 MB TinyStories validation set (the actual #17 request). Fixes param count (100,936 unique; the earlier 106,568 double-counted the tied head) and reports the honest memorization result (ratio 1.02x, verified reproducible).

README.md

@@ -18,13 +18,13 @@ metrics:
 
 # Testr-100K
 
-A **106,568-parameter** character-level GPT trained from scratch.
+A **100,936-parameter** character-level GPT trained from scratch. Fulfils model request [#17](https://huggingface.co/spaces/Compactbot/model-requests/discussions/17) from @GGUFGuy.
 
-**Relationship to model request [#17](https://huggingface.co/spaces/Compactbot/model-requests/discussions/17) from @GGUFGuy — read this first:** that request was a *deliberate experiment* — train a model **only on the TinyStories validation set** and see what happens. This model does **not** do that: it was trained on **191 MB of web text**. It is published as a from-scratch character-level training demonstration, **not** as the requested TinyStories-validation-only experiment. If you were waiting on the actual experiment, this is not it — that one is still to be done.
+> **Correction (2026-09-30):** the weights originally published here were trained on the **wrong data** (191 MB of web text). The request asked for a model trained **only** on the TinyStories *validation* set, as a deliberate memorization test. This commit swaps in the correct model (trained on the 19.2 MB TinyStories validation set) and fixes the parameter count (106,568 → 100,936; the earlier figure double-counted the tied head). The eval numbers below are from the correct model.
 
 ## What it is
 
-A minimal nanoGPT-style causal transformer operating at the **character level** (128-char vocabulary). This is a demonstration of training a working language model from absolute scratch with a very small parameter budget — not a tool for generating coherent text.
+A minimal nanoGPT-style causal transformer operating at the **character level** (128-char vocabulary). The point of this specific model is a **memorization experiment**: train a very small model on a held-out validation set and measure how much it actually memorizes versus generalizing.
 
 ## Architecture
 
@@ -39,11 +39,13 @@ A minimal nanoGPT-style causal transformer operating at the **character level**
 | Normalization | LayerNorm (pre-norm) |
 | Vocab | 128 chars + 1 UNK |
 | Tied embeddings | Yes (head = embedding) |
-| **Total params** | **106,568** |
+| **Total params (unique)** | **100,936** |
+
+(52 tensors in the file; the head is tied to the embedding, so counting it once gives 100,936. The earlier "106,568" double-counted the tied head.)
 
 ## Training
 
-- **Data:** 191 MB web text (character-level, uint8) — **not** the TinyStories validation set that request #17 asked for
+- **Data:** 19.2 MB **TinyStories validation set** (character-level, uint8) — *deliberately* the set the request asked to train on
 - **Steps:** 8,000
 - **Batch size:** 32 × seq 512
 - **Optimizer:** Muon (0.02) + AdamW (1e-4) for biases/norms
@@ -51,45 +53,47 @@ A minimal nanoGPT-style causal transformer operating at the **character level**
 - **Hardware:** RTX 5090 (32 GB)
 - **dtype:** float32
 
-## Quality (honest)
+## The experiment (honest result)
 
-This is a **character-level** model at 106K params. It learns English character statistics and produces text that is *grammatical in shape* but **degenerates into repetition loops** within 2–3 sentences. It is not a coherent text generator.
+The question: does a 100K-parameter model trained *only* on a 19 MB validation set actually memorize it, or does it just learn character statistics?
 
-**Val perplexity:** 39.77 (1M held-out chars from the same corpus)
+Measured perplexity (char-level, 512-char blocks, 20 random windows, seed 1337):
 
-### Sample outputs (greedy, temp=0)
+| Set | Size | Perplexity |
+|-----|------|-----------|
+| **Memorized** (the 19.2 MB set it trained on) | 19,212,307 B | **3.1540** |
+| **Never seen** (191 MB web text) | 191,283,131 B | **3.2203** |
 
-> `The` → ` mean to the box and said, "I don't know what the boy was so happy to the box and said, "I want to t`
+**Ratio (never-seen / memorized) = 1.02×.** At 100K parameters the model **barely memorizes** the set it was trained on — its perplexity on the memorized data is only 2% lower than on data it never saw. It has essentially learned general English character statistics, not the specific stories. That is the honest answer to the request: a 100K-param char model is far too small to memorize 19 MB of text.
 
-> `Once upon a time` → `, there was a little girl named Lily. He was so happy to the box and said, "I want to the box and sa`
+## Sample outputs (greedy, temp=0)
 
-> `I think that` → ` the boy was so happy to the box and said, "I want to the box and said. "I want to the`
+> `Once upon a time` → `, there was a little girl named Lily. She liked to play with her mom and said, "I want to the bird was so happy and said`
 
-### Sample outputs (temp=0.7)
+> `The sun was` → ` so happy and said, "I want to the boy named Lily. They were so happy and said, "I want to the ball and said, "I want to`
 
-> `The` → ` little girl named Lily was hands, "It was not want to be and field. It could go him to play. She pu`
+### Sample outputs (temp=0.8)
 
-> `Once upon a time` → ` there was a mommy was playing and the grandma happy.\n\nLily was very happy thast and pretty was so d`
+> `A little` → ` girl named Spot around of friends.\nOnce upon a time, there was a praye was so like in room that it was angry, the sweet`
 
-The first sentence or two is often grammatical; after that the model locks into a phrase and repeats it. This is expected at this scale and vocabulary size.
+The first sentence or two is often grammatical; after that the model locks into a phrase and repeats it. Expected at this scale and vocabulary size.
 
 ## What it is NOT
 
-- Not the TinyStories-validation-only experiment that request #17 asked for
 - Not a subword/token-level model (it cannot "read" word-level benchmarks like BLiMP or ARC)
 - Not a coherent paragraph generator
-- Not comparable to sub-1M subword models on any word-level metric
+- Not evidence of memorization at this scale (see ratio above)
 
 ## Reproducing
 
-The model is a standard nanoGPT-style architecture. The training script is a standard causal transformer with Muon optimizer. The checkpoint is saved as safetensors with tied embeddings.
+Standard nanoGPT-style causal transformer with Muon optimizer. The checkpoint is saved as safetensors with tied embeddings (the file stores `tok.weight` and an identical `head.weight`; they are the same tensor).
 
-To generate: load `model.safetensors` into a GPT class matching the config, map characters to token IDs via `vocab.json`, and sample.
+To generate: load `model.safetensors` into a GPT class matching `config.json`, map characters to token IDs via `vocab.json`, and sample.
 
 ## Files
 
 | File | Description |
 |------|-------------|
-| `model.safetensors` | Model weights (408 KB) |
+| `model.safetensors` | Model weights (431 KB, 52 tensors, tied head) |
 | `config.json` | Architecture config |
 | `vocab.json` | Character → token ID mapping (128 chars + UNK) |