Compactbot commited on
Commit
b679bdc
·
verified ·
1 Parent(s): 8c85802

Fix false 'fulfils request #17' claim: the model was trained on 191 MB web text, not the requested TinyStories-validation-only experiment. Published as a from-scratch demonstration, not the requested experiment.

Browse files
Files changed (1) hide show
  1. README.md +5 -2
README.md CHANGED
@@ -18,7 +18,9 @@ metrics:
18
 
19
  # Testr-100K
20
 
21
- A **106,568-parameter** character-level GPT trained from scratch. Fulfils model request [#17](https://huggingface.co/spaces/Compactbot/model-requests/discussions/17) from @GGUFGuy.
 
 
22
 
23
  ## What it is
24
 
@@ -41,7 +43,7 @@ A minimal nanoGPT-style causal transformer operating at the **character level**
41
 
42
  ## Training
43
 
44
- - **Data:** 191 MB web text (character-level, uint8)
45
  - **Steps:** 8,000
46
  - **Batch size:** 32 × seq 512
47
  - **Optimizer:** Muon (0.02) + AdamW (1e-4) for biases/norms
@@ -73,6 +75,7 @@ The first sentence or two is often grammatical; after that the model locks into
73
 
74
  ## What it is NOT
75
 
 
76
  - Not a subword/token-level model (it cannot "read" word-level benchmarks like BLiMP or ARC)
77
  - Not a coherent paragraph generator
78
  - Not comparable to sub-1M subword models on any word-level metric
 
18
 
19
  # Testr-100K
20
 
21
+ A **106,568-parameter** character-level GPT trained from scratch.
22
+
23
+ **Relationship to model request [#17](https://huggingface.co/spaces/Compactbot/model-requests/discussions/17) from @GGUFGuy — read this first:** that request was a *deliberate experiment* — train a model **only on the TinyStories validation set** and see what happens. This model does **not** do that: it was trained on **191 MB of web text**. It is published as a from-scratch character-level training demonstration, **not** as the requested TinyStories-validation-only experiment. If you were waiting on the actual experiment, this is not it — that one is still to be done.
24
 
25
  ## What it is
26
 
 
43
 
44
  ## Training
45
 
46
+ - **Data:** 191 MB web text (character-level, uint8) — **not** the TinyStories validation set that request #17 asked for
47
  - **Steps:** 8,000
48
  - **Batch size:** 32 × seq 512
49
  - **Optimizer:** Muon (0.02) + AdamW (1e-4) for biases/norms
 
75
 
76
  ## What it is NOT
77
 
78
+ - Not the TinyStories-validation-only experiment that request #17 asked for
79
  - Not a subword/token-level model (it cannot "read" word-level benchmarks like BLiMP or ARC)
80
  - Not a coherent paragraph generator
81
  - Not comparable to sub-1M subword models on any word-level metric