MP-01 100M base: model card with comparison charts, average and benchmark tables, logo

.gitattributes

@@ -34,3 +34,6 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
 manprot-logo.png filter=lfs diff=lfs merge=lfs -text
+charts/average_table.png filter=lfs diff=lfs merge=lfs -text
+charts/avg_7tasks.png filter=lfs diff=lfs merge=lfs -text
+charts/benchmark_table.png filter=lfs diff=lfs merge=lfs -text

README.md

@@ -69,6 +69,17 @@ Scored by this repository's harness (`scripts/eval_checkpoint.py`, `scripts/eval
 
 ### Against other models
 
+![Average benchmark score](charts/average_table.png)
+
+![Benchmark results](charts/benchmark_table.png)
+
+![Average across 7 benchmarks](charts/avg_7tasks.png)
+
+![BLiMP grammar accuracy](charts/blimp.png)
+
+In the table, the highlighted cell in each column is the best score in that column, whichever model it is. On BLiMP the best is GPT-2.
+
+
 Same harness and the same 500 questions for every model. Models are measured with their own tokenizers. Validation loss is not comparable across tokenizers, so it is not shown here.
 
 | Model | Params | arc_challenge | arc_easy | hellaswag | openbookqa | piqa | sciq | winogrande | Avg (7) | BLiMP |
@@ -129,3 +140,4 @@ tok = Tokenizer.from_file("tokenizer.json")
 - `vbq/`: model code
 - `eval_results.json`: full evaluation output for this checkpoint
 - `manprot-logo.png`: logo
+- `charts/`: benchmark tables and charts (PNG)