Use verified float32 harness results in model card

README.md

@@ -18,7 +18,7 @@ tags:
 
 Sol Pro is a **138,037,594-parameter** language model from Sol Labs. It combines recurrent transformer blocks with tensorized n-gram memory and a 2,048-token context.
 
-The released checkpoint scores **29.37 on the normalized Intelligence Index** and **24.01 on the raw accuracy Index**.
+The released checkpoint scores **28.63 on the normalized Intelligence Index** and **24.13 on the raw accuracy Index**.
 
 ## Model
 
@@ -77,18 +77,18 @@ The repository includes standalone PyTorch model code. Generation recomputes the
 
 | Benchmark | Examples | Normalized accuracy | Raw accuracy |
 |---|---:|---:|---:|
-| HellaSwag | 10,042 | 47.64% | 36.70% |
-| ARC Easy | 2,376 | 55.51% | 54.63% |
-| ARC Challenge | 1,172 | 32.42% | 28.92% |
-| PIQA | 1,838 | 71.22% | 69.59% |
-| Arithmark3 | 1,000 | 35.70% | 37.10% |
-| **Intelligence Index** | | **29.37** | **24.01** |
+| HellaSwag | 10,042 | 46.76% | 36.88% |
+| ARC Easy | 2,376 | 54.08% | 55.35% |
+| ARC Challenge | 1,172 | 32.00% | 29.35% |
+| PIQA | 1,838 | 70.95% | 69.26% |
+| Arithmark3 | 1,000 | 36.00% | 37.20% |
+| **Intelligence Index** | | **28.63** | **24.13** |
 
-These are local measurements on complete evaluation splits, using the existing Sol likelihood evaluator and chance-adjusted Index calculation. Normalized accuracy divides each choice's continuation log-likelihood by its token count.
+These are zero-shot float32 measurements on complete evaluation splits. HellaSwag, ARC Easy, ARC Challenge, and PIQA use lm-evaluation-harness 0.4.12 with batch size 64. Arithmark3 uses the official AxiomicLabs script with its default batch size 32 and 1,024-token context, explicitly set to float32. The Index uses the leaderboard's chance-adjusted formula and each task's normalized accuracy.
 
 This is a task-adapted checkpoint using public benchmark training splits. Evaluation used separate held-out splits, with matching evaluation contexts excluded from the adaptation data; Arithmark evaluation examples were not used for adaptation. The scores have not been independently verified and do not establish a leaderboard position.
 
-WikiText-2 validation cross-entropy is **2.7677** over **366,592 tokens**. Exact results and evaluation details are in [evaluation.json](evaluation.json).
+WikiText-2 validation cross-entropy is **2.7677** over **366,592 tokens**. Exact results, package versions, hashes, and commands are in [the float32 evaluation summary](evaluation/standard_float32/summary.json). Raw outputs are available for [lm-eval](evaluation/standard_float32/lm_eval.json) and [ArithMark-3](evaluation/standard_float32/arithmark.json).
 
 ## Use and limits
 
@@ -102,7 +102,7 @@ Sol Pro supports text-completion and small-model research. Multiple-choice score
 | `modeling_sol_pro.py` | Architecture, loader, and text generator |
 | `config.json` | Architecture configuration |
 | `tokenizer.json` | Original tokenizer |
-| `evaluation.json` | Exact evaluation results |
+| `evaluation/standard_float32/` | Standard-harness evaluation results and reproducibility details |
 | `banner.png` | Sol Pro artwork |
 
 ## License