File size: 988 Bytes
4649798 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 | # Benchmark summaries
All numbers here were measured locally on one RTX 5060 Ti 16GB using the same
rows and scorer for every model.
## Typed Decisions
- Full independent test split: 400 held-out cases, 2,000 probabilistic decisions.
- Scored with the benchmark metric code: argmax accuracy, KL, Brier, ECE.
- Models: Pluto-2.1, Pluto-2, d1-2B, d1-3B. d1 checkpoints are zero-shot.
## DecisionBench
- Public `Hanno-Labs/decision-bench` evaluation split.
- Matched accessibility subset: 1,565 of 23,900 rows, limited to ≤16 candidates
and ≤2,048-token prompts.
- Models: Pluto-2.1, Pluto-2, Bosun 3.1 1.7B, d1-2B, d1-3B.
- This is not the full 23,900-row suite. No full-suite score is claimed.
## Files
- `pluto21_*.json` — Pluto-2.1 step 2,000
- `pluto2_*.json` — Pluto-2 400M
- `bosun_base_decisionbench_1565.json` — Bosun 3.1 1.7B initializer
- `d1_2b_*.json` — Liquid d1-2B
- `d1_3b_*.json` — Liquid d1-3B
The model card charts are generated from these values.
|