Show benchmark accuracy and normalized accuracy as paired values

Compare published OpenSML-150M and SmolLM2 results, include IFEval, simplify metric tables, and add training details, recorded examples and reproducible figures. Model weights and inference runtime are unchanged.

FILE_MANIFEST.json

@@ -23,7 +23,7 @@
     },
     "README.md": {
       "bytes": 13155,
-      "sha256": "894e1200eb78c3afd6617410b209bfd389f77c45301c3b6cfb3faf8b9fd380cb"
+      "sha256": "9ae223eaf33f9dfb4aa1563dc5f74ce535e92903e47ba7644abe227a5306e9d5"
     },
     "REPOSITORIES.json": {
       "bytes": 716,
@@ -131,7 +131,7 @@
     },
     "docs/EVALUATION.md": {
       "bytes": 8582,
-      "sha256": "79c7190a634b9008502242133bf76e246ab69700814049b5ac39f49ad51c57c3"
+      "sha256": "e890892eb870896a8f5d585bde6ab748820f91b2ff49e8d70434bc14140767a7"
     },
     "docs/EXAMPLES.md": {
       "bytes": 1117,

README.md

@@ -55,17 +55,14 @@ Solstice-1 has a separate weight lineage from OpenSML-150M and OpenSML-JSONLite-
 
 Zero-shot likelihood scoring on the complete ARC Easy, ARC Challenge, PIQA and HellaSwag evaluation splits. Accuracy ranks the total answer log-likelihood; `acc_norm` ranks it after dividing by answer length in characters. Higher is better.
 
-| Benchmark | Metric | Solstice-1 | OpenSML-150M | SmolLM2-135M Instruct |
-| --- | --- | --- | --- | --- |
-| ARC Easy | Accuracy | 62.58% | 56.65% | — |
-| ARC Easy | acc_norm | 62.33% | 55.43% | — |
-| ARC Challenge | Accuracy | 29.35% | 26.02% | — |
-| ARC Challenge | acc_norm | 30.89% | 29.52% | — |
-| PIQA | Accuracy | 66.49% | 64.53% | — |
-| PIQA | acc_norm | 66.21% | 64.09% | 66.3% |
-| HellaSwag | Accuracy | 32.85% | 30.55% | — |
-| HellaSwag | acc_norm | 39.40% | 33.94% | 40.9% |
-| ARC (publisher average) | acc_norm | — | — | 37.3% |
+| Benchmark | Solstice-1 acc / acc_norm | OpenSML-150M acc / acc_norm | SmolLM2-135M Instruct acc / acc_norm |
+| --- | --- | --- | --- |
+| ARC Easy | 62.58 / 62.33 | 56.65 / 55.43 | — / — |
+| ARC Challenge | 29.35 / 30.89 | 26.02 / 29.52 | — / — |
+| PIQA | 66.49 / 66.21 | 64.53 / 64.09 | — / 66.3 |
+| HellaSwag | 32.85 / 39.40 | 30.55 / 33.94 | — / 40.9 |
+
+All values are percentages, shown as **acc / acc_norm**. SmolLM2 publishes ARC only as a combined average (**37.3% acc_norm**); separate Easy/Challenge scores are unavailable in its card.
 
 ![Normalized benchmark accuracy for Solstice-1 and published small-model references](docs/figures/benchmark-comparison.png)
 

SHA256SUMS

@@ -3,7 +3,7 @@ f09ed498cc42b482dae88d9fb5026bf74e1d8b9187f9ea301aeda0db76d6c859  BACKUP_AND_CLE
 af646a190d135ec25a6ee2c76d1594c3f9ba8005cd3ca6df908606a15023273e  CITATION.cff
 cfc7749b96f63bd31c3c42b5c471bf756814053e847c10f3eb003417bc523d30  LICENSE
 0e6865f1a81cfd69ca48e6987658854ce0c0d23e91d72ae545393cd5fa032375  PACKAGING_VERIFICATION.json
-894e1200eb78c3afd6617410b209bfd389f77c45301c3b6cfb3faf8b9fd380cb  README.md
+9ae223eaf33f9dfb4aa1563dc5f74ce535e92903e47ba7644abe227a5306e9d5  README.md
 152a87d0098752350d54fdd3cde3861a77eb497c53dc973ee1a8370d12a13ff6  REPOSITORIES.json
 fd05e1595bcdc52942e9425b6efb2c7e89553e6b0ad7d5f93d9075969cd75dba  RUN_GUIDE.md
 87fa33719fe1182d467a05ab417be2a362f50eea59f0ac6496d40e677544e27c  SOURCE_NOTICE.md
@@ -30,7 +30,7 @@ fb204be1d9e72eb1cae140bfdb03dbcadc1843c47b1894a731396ceeedb9a22f  configs/sft1/p
 51d66108416988bad1015d7b52cf63db0571f87f202b2b18c7e98e67a29ec62b  configs/sft2/plan.json
 ae708eca3c09d5a27eab3fddbe4106d0658787beebc9393de57f75234480a284  configs/sft3/plan.json
 e0b0c730321360c40f20e53a22890c52cd5294bad2a5777a0a891096557a2f0a  docs/DATA.md
-79c7190a634b9008502242133bf76e246ab69700814049b5ac39f49ad51c57c3  docs/EVALUATION.md
+e890892eb870896a8f5d585bde6ab748820f91b2ff49e8d70434bc14140767a7  docs/EVALUATION.md
 7a818c365f1c047b349479829396ba5cd841ca8d2f04fa14fc1489083a1fd4b3  docs/EXAMPLES.md
 f98d45b39b9595510fa80d495a3758b211d8c770adc739d381763d3cfb2a2a0b  docs/TECHNICAL_REPORT.md
 37438bfe291e2cc5b7513ec912cab9eaf7ce3d6352543f8e63d44da3a4f7d5b3  docs/figures/benchmark-comparison.png

docs/EVALUATION.md

@@ -4,17 +4,14 @@ All results identify the same release weight hash, `7b2d8777040f05d8e0849acac4e4
 
 ## Likelihood scoring
 
-| Benchmark | Metric | Solstice-1 | OpenSML-150M | SmolLM2-135M Instruct |
-| --- | --- | --- | --- | --- |
-| ARC Easy | Accuracy | 62.58% | 56.65% | — |
-| ARC Easy | acc_norm | 62.33% | 55.43% | — |
-| ARC Challenge | Accuracy | 29.35% | 26.02% | — |
-| ARC Challenge | acc_norm | 30.89% | 29.52% | — |
-| PIQA | Accuracy | 66.49% | 64.53% | — |
-| PIQA | acc_norm | 66.21% | 64.09% | 66.3% |
-| HellaSwag | Accuracy | 32.85% | 30.55% | — |
-| HellaSwag | acc_norm | 39.40% | 33.94% | 40.9% |
-| ARC (publisher average) | acc_norm | — | — | 37.3% |
+| Benchmark | Solstice-1 acc / acc_norm | OpenSML-150M acc / acc_norm | SmolLM2-135M Instruct acc / acc_norm |
+| --- | --- | --- | --- |
+| ARC Easy | 62.58 / 62.33 | 56.65 / 55.43 | — / — |
+| ARC Challenge | 29.35 / 30.89 | 26.02 / 29.52 | — / — |
+| PIQA | 66.49 / 66.21 | 64.53 / 64.09 | — / 66.3 |
+| HellaSwag | 32.85 / 39.40 | 30.55 / 33.94 | — / 40.9 |
+
+All values are percentages, shown as **acc / acc_norm**. SmolLM2 publishes ARC only as a combined average (**37.3% acc_norm**); separate Easy/Challenge scores are unavailable in its card.
 
 Native MLX evaluates complete ARC Easy/Challenge test splits, PIQA validation and HellaSwag validation. It uses zero-shot raw completion prompts, no chat template or inserted BOS/EOS, FP32 weights, vanilla attention, batch size four and chunked scoring. Raw accuracy ranks summed continuation log-likelihood; normalized accuracy divides by choice Unicode-character count. No generated explanation or LLM judge determines the answer. No choices were context-truncated. Dataset revisions and scorer settings are preserved in [the result record](../results/public/choices/results.json).