model card: Sol image reference scores

README.md

@@ -55,8 +55,8 @@ Sev vs Jev pooled: **+0.003 [−0.008, +0.017]** (95% paired group bootstrap). T
 ### Image decisions (Jev does not accept images)
 | Suite | **Sev-9B** | Sol (reference) |
 |---|---|---|
-| Held-out image final (2,451 rows) | 87.6 | *scoring* |
-| COCO (394 rows) | 96.7 | *scoring* |
+| Held-out image final (2,451 rows) | 87.6 | 90.1 |
+| COCO (394 rows) | **96.7** | 94.9 |
 
 ### Decision Index 0.2.1 (Cloudflare's suite, rebuilt from the public kit on the official case ids)
 Scoring is in progress. Early rows:
@@ -89,7 +89,7 @@ Each claim below is tied to a measured number. Clef and Jev latency figures come
 | **Speed** | p50 **33 ms**, p95 **41 ms** on one GPU; **88 ms** on a MacBook (MLX). Cloudflare reports Jev at 524 / 536 ms, Clef (27B) at 209 / 239 ms, and Clef-flash at 38.8 / 122 ms. That makes Sev ~16× faster than Jev at the median, with the lowest p95 of any model on that page. |
 | **Jev-level accuracy in a model you run yourself** | Pooled sealed general **94.6 vs Jev 94.3**. Clear leads: SNLI yes/no **+4.0**, SNLI 3-way **+4.7**, Amazon stars **+3.4**, HellaSwag **+2.3**, counterfactual **+2.3**, structured rules **+1.8**. |
 | **Better ordinal scores** | Score-type normalized MAE **0.046 vs Jev 0.050**; score accuracy 92.6 vs 92.3; choice accuracy 96.2 vs 94.4. |
-| **Images** | Decides over photos, documents and charts: image final **87.6**, COCO **96.7**. In our tests Jev rejects image inputs (HTTP 400). |
+| **Images** | Decides over photos, documents and charts: image final **87.6** (Sol 90.1), COCO **96.7**, ahead of Sol's 94.9. In our tests Jev rejects image inputs (HTTP 400). |
 | **Trading** | Trading suite **74.0 vs Jev 71.5**. |
 | **One pass, calibrated, no generation** | Returns a full probability distribution over every option in a single forward pass. No sampled text to parse, and outputs are deterministic for a given input. |
 | **Local and private** | Runs on a single GPU: no per-call API cost, no data leaves your infrastructure. |
@@ -157,7 +157,7 @@ For a smaller 8-bit copy (8.9 GB, same answers in our test), run `python -m mlx_
 
 ## Limitations
 - **Not confirmed on a fresh human-labelled final.** About 45 candidates were scored on the sealed general suite, so expect some selection optimism in the general numbers.
-- **Trading and image accuracy trail Sol** (reference).
+- **Trading accuracy trails Sol** (74.0 vs 90.0), and so does the held-out image final (87.6 vs 90.1). Sev leads Sol on COCO (96.7 vs 94.9).
 - **Forecast-type trading questions** are near chance for every model tested, including Sol.
 - **CommonsenseQA** is the weakest general suite vs Jev (−3 points).
 - **Keep this repository private and for internal use.**