Card 1.08b: standard benchmark panel (regression in both arms vs pre-registered guards)

README.md

@@ -17,7 +17,7 @@ datasets:
 
 # Slayer1B 1.08b (experimental)
 
-**Status: trained and evaluated — result inconclusive, weights are not published yet.** Slayer1B 1.08b is an
+**Status: trained and evaluated — reasoning result inconclusive, worse than 1.07 on standard English benchmarks; weights are not published yet.** Slayer1B 1.08b is an
 experimental continuation of [Slayer1B 1.07](https://huggingface.co/SlayerLab/Slayer1B-1.07). The suffix "b" marks an
 experiment: the model would be released as Slayer1B 1.08 only if the evaluation showed it is better than 1.07. It did
 not show that (see Evaluation), so the name stays 1.08b.
@@ -84,6 +84,25 @@ cannot be scored across the two library versions are excluded, which gives **363
 - The p-values of the degenerate pass@1 contrasts (all zeros) were clamped to 1.0 after a boundary bug fix; no other
   field changed.
 
+**Standard benchmark panel (measured after the verdict).** Same calibrated instrument as the 1.07 results; accuracies
+in percent.
+
+| | 1.07 (22327) | 1.08b | 1.08b control |
+|---|---:|---:|---:|
+| ARC-Easy | 58.67 | 52.10 | 52.06 |
+| BLiMP | 77.90 | 74.64 | 74.44 |
+| WikiText-2 byte perplexity (lower is better) | 2.184 | 2.299 | 2.263 |
+| ARC-Easy PL | 46.93 | 46.17 | 45.92 |
+| MultiBLiMP PL | 98.84 | 98.23 | 98.59 |
+
+The method note pre-registered guards on this panel (written for the planned run with learning-rate decay; the actual
+run used a constant rate): at most 1.0 pp drop on ARC-Easy and BLiMP, at most +0.02 byte perplexity, and at most 0.5 pp
+drop on each Polish benchmark. By those guards the result is a regression in both arms. English: ARC-Easy −6.6 pp,
+BLiMP −3.3 pp (1.08b) and −3.5 pp (control), byte perplexity +0.115 and +0.080. Polish: −0.6 pp (MultiBLiMP PL) and
+−0.8 pp (ARC-Easy PL) for 1.08b; −0.3 pp and −1.0 pp for the control. Because the control dropped as much as 1.08b,
+the drop is not specific to the reasoning data; its cause (for example the constant learning rate, or the absence of
+general English text in both mixtures) has not been tested.
+
 A new held-out set with a pre-registered short answer format (and the condition that 1.07 scores above zero before
 the arms are run) and a comparison after instruction tuning of both arms are in preparation.
 
@@ -91,6 +110,7 @@ the arms are run) and a comparison after instruction tuning of both arms are in
 
 - Base model, not instruction-tuned; responses may be inconsistent.
 - Experimental: constant learning rate, one run per arm, one seed. Reasoning ability is not measured yet (see Evaluation).
+- Worse than 1.07 on the standard English benchmarks (see the panel above); use 1.07 as the general-purpose base.
 
 ## Authorship and license