Fix false 'fulfils request #17' claim: the model was trained on 191 MB web text, not the requested TinyStories-validation-only experiment. Published as a from-scratch demonstration, not the requested experiment.
Browse files
README.md
CHANGED
|
@@ -18,7 +18,9 @@ metrics:
|
|
| 18 |
|
| 19 |
# Testr-100K
|
| 20 |
|
| 21 |
-
A **106,568-parameter** character-level GPT trained from scratch.
|
|
|
|
|
|
|
| 22 |
|
| 23 |
## What it is
|
| 24 |
|
|
@@ -41,7 +43,7 @@ A minimal nanoGPT-style causal transformer operating at the **character level**
|
|
| 41 |
|
| 42 |
## Training
|
| 43 |
|
| 44 |
-
- **Data:** 191 MB web text (character-level, uint8)
|
| 45 |
- **Steps:** 8,000
|
| 46 |
- **Batch size:** 32 × seq 512
|
| 47 |
- **Optimizer:** Muon (0.02) + AdamW (1e-4) for biases/norms
|
|
@@ -73,6 +75,7 @@ The first sentence or two is often grammatical; after that the model locks into
|
|
| 73 |
|
| 74 |
## What it is NOT
|
| 75 |
|
|
|
|
| 76 |
- Not a subword/token-level model (it cannot "read" word-level benchmarks like BLiMP or ARC)
|
| 77 |
- Not a coherent paragraph generator
|
| 78 |
- Not comparable to sub-1M subword models on any word-level metric
|
|
|
|
| 18 |
|
| 19 |
# Testr-100K
|
| 20 |
|
| 21 |
+
A **106,568-parameter** character-level GPT trained from scratch.
|
| 22 |
+
|
| 23 |
+
**Relationship to model request [#17](https://huggingface.co/spaces/Compactbot/model-requests/discussions/17) from @GGUFGuy — read this first:** that request was a *deliberate experiment* — train a model **only on the TinyStories validation set** and see what happens. This model does **not** do that: it was trained on **191 MB of web text**. It is published as a from-scratch character-level training demonstration, **not** as the requested TinyStories-validation-only experiment. If you were waiting on the actual experiment, this is not it — that one is still to be done.
|
| 24 |
|
| 25 |
## What it is
|
| 26 |
|
|
|
|
| 43 |
|
| 44 |
## Training
|
| 45 |
|
| 46 |
+
- **Data:** 191 MB web text (character-level, uint8) — **not** the TinyStories validation set that request #17 asked for
|
| 47 |
- **Steps:** 8,000
|
| 48 |
- **Batch size:** 32 × seq 512
|
| 49 |
- **Optimizer:** Muon (0.02) + AdamW (1e-4) for biases/norms
|
|
|
|
| 75 |
|
| 76 |
## What it is NOT
|
| 77 |
|
| 78 |
+
- Not the TinyStories-validation-only experiment that request #17 asked for
|
| 79 |
- Not a subword/token-level model (it cannot "read" word-level benchmarks like BLiMP or ARC)
|
| 80 |
- Not a coherent paragraph generator
|
| 81 |
- Not comparable to sub-1M subword models on any word-level metric
|