Replace table images with markdown tables; list all datasets incl. later runs

README.md

@@ -75,7 +75,7 @@ A separate local cooldown (170 steps from step 13,000) was an experiment. It is
 
 ## Datasets
 
-Every dataset used to train or evaluate this checkpoint is listed here. Each stage after the first used documents the earlier stages had not seen; shards were excluded between corpora. Training data is not re-released. It was built from the public datasets below.
+**In total: 16 training datasets and 8 evaluation datasets.** All 16 training datasets are also in this page's metadata; the Hugging Face sidebar shows the first three and folds the rest behind "+13". Every dataset used to train or evaluate this checkpoint is listed here. Each stage after the first used documents the earlier stages had not seen; shards were excluded between corpora. Training data is not re-released. It was built from the public datasets below.
 
 ### Pretraining, growth, expansion and cooldown (stages 1, 2, 4, 5)
 
@@ -107,10 +107,23 @@ Shares are of each stage's tokens. The tokenizer was trained on a sample drawn f
 | [hieunguyenminh/roleplay](https://huggingface.co/datasets/hieunguyenminh/roleplay) | Character roleplay |
 | [PygmalionAI/PIPPA](https://huggingface.co/datasets/PygmalionAI/PIPPA) | Character roleplay |
 
-The earlier 75M chat models (not in this checkpoint's weights; the 100M was grown from the 75M base before any chat tuning) used a subset of these: smol-smoltalk, everyday-conversations, oasst2, airoboros, kalo, Sonnet3.5 Charcard, hieunguyenminh/roleplay and PIPPA. No other datasets were used anywhere in the MP-01 line.
+The earlier 75M chat models (not in this checkpoint's weights; the 100M was grown from the 75M base before any chat tuning) used a subset of these: smol-smoltalk, everyday-conversations, oasst2, airoboros, kalo, Sonnet3.5 Charcard, hieunguyenminh/roleplay and PIPPA.
 
 Chat data was filtered: refusals, boilerplate, other models' identities and over-length chats were dropped, and any sexual content involving minors was removed.
 
+### Used in later MP-01 runs (not in this checkpoint)
+
+These datasets were used in training runs that continue from this model. They are listed for completeness: **none of them is in the weights released here.**
+
+| Dataset | Used for |
+|---|---|
+| [SlayerLab/plwiki-history-rag-20260920](https://huggingface.co/datasets/SlayerLab/plwiki-history-rag-20260920) | Polish history text |
+| [open-web-math/open-web-math](https://huggingface.co/datasets/open-web-math/open-web-math) | Mathematical web pages |
+| [sedthh/gutenberg_english](https://huggingface.co/datasets/sedthh/gutenberg_english) | Public-domain books |
+| [RealTimeData/bbc_news_alltime](https://huggingface.co/datasets/RealTimeData/bbc_news_alltime) | Recent business and economics news |
+| [meta-math/MetaMathQA](https://huggingface.co/datasets/meta-math/MetaMathQA) | Step-by-step math answers (chat data) |
+| [sinhal/Charcard-Roleplay-ShareGPT-NSFW](https://huggingface.co/datasets/sinhal/Charcard-Roleplay-ShareGPT-NSFW) | Adult character roleplay (chat data) |
+
 ### Evaluation benchmarks
 
 | Benchmark | Dataset |
@@ -141,15 +154,19 @@ This is a custom architecture, not a `transformers` model. Load it with the `vbq
 
 Scored by this repository's harness (`scripts/eval_checkpoint.py`, `scripts/eval_hf.py`): zero-shot, 500 questions per task (the same sample every time). BLiMP is zero-shot minimal pairs (67 paradigms). With 500 questions a task, differences under about 2 points are within noise. Models use their own tokenizers, so validation loss is not comparable across models and is not compared.
 
-### Average score
-
-![Average benchmark score](charts/average_table.png)
-
-### Per-task results
+### Comparison with other small models
 
-The highlighted cell in each column is the best score in that column, whichever model it is. On BLiMP the best is GPT-2.
+Sorted by the 7-task average. Bold is the best score in each column, whichever model it is.
 
-![Benchmark results](charts/benchmark_table.png)
+| Rank | Model | Params | ARC-C | ARC-E | HellaSwag | OBQA | PIQA | SciQ | Wino | Avg (7) | BLiMP |
+|---|---|---|---:|---:|---:|---:|---:|---:|---:|---:|---:|
+| 1 | MP-01 100M base (this model) | 100.6M | **29.2** | **54.0** | 29.4 | **29.4** | 62.8 | 72.2 | **54.0** | **47.29** | 78.70 |
+| 2 | GPT-neo-125M | 125.2M | 25.2 | 41.8 | 27.8 | 25.6 | 63.4 | **73.4** | 50.4 | 43.94 | 79.26 |
+| 3 | OPT-125M | 125.2M | 25.4 | 39.8 | **30.6** | 27.8 | 61.0 | 70.0 | 50.8 | 43.63 | 81.41 |
+| 4 | GPT-2 | 124.4M | 24.8 | 39.6 | 29.6 | 25.8 | **64.2** | 67.6 | 50.2 | 43.11 | **82.51** |
+| 5 | Pythia-160M | 162.3M | 25.4 | 41.8 | 26.0 | 26.4 | 62.8 | 69.0 | 47.8 | 42.74 | 78.53 |
+| 6 | DistilGPT2 | 81.9M | 25.2 | 36.0 | 25.4 | 26.0 | 59.0 | 64.8 | 50.2 | 40.94 | 79.53 |
+| 7 | GPT-1 | 116.5M | 25.8 | 34.4 | 22.8 | 26.6 | 55.8 | 59.0 | 53.6 | 39.71 | 77.59 |
 
 ### Where this model stands
 
@@ -160,13 +177,27 @@ The highlighted cell in each column is the best score in that column, whichever
 
 ### Progress across checkpoints of this model
 
-![Progress across checkpoints](charts/progress_table.png)
+| Checkpoint | Val loss | ARC-C | ARC-E | HellaSwag | OBQA | PIQA | SciQ | Wino | Avg (7) | BLiMP |
+|---|---:|---:|---:|---:|---:|---:|---:|---:|---:|---:|
+| Expansion step 9,500 | 2.7780 | 27.6 | 52.8 | **29.8** | 27.6 | 62.4 | 71.0 | 51.6 | 46.11 | 76.88 |
+| Expansion step 13,000 | 2.7722 | 28.8 | 49.8 | 28.4 | 29.0 | 61.6 | 70.4 | 51.8 | 45.69 | 78.03 |
+| Expansion step 15,000 (cooldown start) | 2.7730 | 26.8 | 52.6 | 29.0 | **30.0** | 62.0 | 69.8 | 53.0 | 46.17 | 77.66 |
+| **Cooldown step 4,000 (this release)** | **2.6531** | **29.2** | **54.0** | 29.4 | 29.4 | **62.8** | **72.2** | **54.0** | **47.29** | **78.70** |
 
-The highlighted cell in each column is the best value (lowest for val loss). Validation loss is measured on the held-out split of the first expansion corpus, which the model trained on. It is not a clean test, so the cooldown's lower loss should not be read as general improvement on its own. The benchmark average rose after the cooldown and is the more useful signal.
+Bold is the best value in each column (lowest for val loss). Validation loss is measured on the held-out split of the first expansion corpus, which the model trained on. It is not a clean test, so the cooldown's lower loss should not be read as general improvement on its own. The benchmark average rose after the cooldown and is the more useful signal.
 
 ## Compute
 
-![Compute used for this release](charts/compute_table.png)
+| Stage | Hardware | GPU hours | Tokens |
+|---|---|---:|---:|
+| Pretraining, 75M base | Local AMD Radeon RX 7600 (8 GB) | not totalled from logs | 4.0B |
+| Growth to 100M, first stage | Local AMD Radeon RX 7600 (8 GB) | 3.94 (one segment) | 0.61B |
+| Chat fine-tune | Local AMD Radeon RX 7600 (8 GB) | 2.78 | 0.18B |
+| Expansion, chunk 1 (steps 2,314 to 14,230) | Cloud A100 40 GB | 4.00 | 3.1B new |
+| Expansion, chunk 2 (steps 14,231 to 15,000) | Cloud A100 40 GB | ~0.35 | 0.2B new |
+| Cooldown, 4,000 steps | Cloud H200 141 GB | 0.68 | 1.05B |
+| Benchmarks of released checkpoints | Local GPU | a few minutes each | |
+| Comparison benchmarks (6 models) | Cloud H200 | a few minutes each, in parallel | |
 
 Hours are wall-clock GPU time from the run logs. Costs in credits are not listed. The local GPU's times are per recorded segment; long runs were restarted, so the 75M pretraining total is not reliable from the logs. Expansion work after step 15,000 on the cloud is not part of this checkpoint.
 
@@ -201,7 +232,6 @@ tok = Tokenizer.from_file("tokenizer.json")
 - `vbq/`: model code
 - `eval_results.json`: full evaluation output for this checkpoint and the comparison models
 - `manprot-logo.png`: logo
-- `charts/`: the prerendered tables (PNG)
 
 ## Author and citation