# Pluto-2.1 results

## General-model comparison

Same rows and same scorer for every locally measured model. `Mean` is the mean
of Typed Decisions and DecisionBench matched accuracy.

| Model | Params | Typed Decisions | DecisionBench matched | Mean |
|---|---:|---:|---:|---:|
| **Pluto-2.1** | **1.738B** | **0.6695** | **0.7208** | **0.6952** |
| Pluto-2 400M | 0.397B | **0.7835** | 0.5789 | 0.6812 |
| Bosun 3.1 1.7B (initializer) | 1.738B | 0.4980 | **0.8288** | 0.6634 |
| d1-3B | 3.123B | 0.6495 | 0.6486 | 0.6490 |
| d1-2B | 1.918B | 0.6325 | 0.6339 | 0.6332 |

Pluto-2.1 clears d1-2B by **+3.7** Typed and **+8.7** DecisionBench points, and
clears d1-3B by **+2.0** and **+7.2** points. It has the highest mean accuracy
of the five models. Bosun retains the broad-only lead; Pluto-2 400M retains the
Typed-only lead. Pluto-2.1 owns the balance.

## Distribution quality

| Benchmark | Model | Brier | KL / NLL |
|---|---|---:|---:|
| Typed Decisions | **Pluto-2.1** | **0.1261** | **0.2139** |
| Typed Decisions | d1-3B | 0.1416 | 0.2854 |
| Typed Decisions | d1-2B | 0.1440 | 0.2958 |
| DecisionBench matched | **Pluto-2.1** | **0.3730** | **0.7439** |
| DecisionBench matched | d1-3B | 0.4937 | 1.0345 |
| DecisionBench matched | d1-2B | 0.4510 | 0.9333 |

## Typed Decisions by primitive

| Primitive | n | Accuracy | Brier | ECE |
|---|---:|---:|---:|---:|
| choice | 600 | 0.7250 | 0.1024 | 0.1011 |
| noul | 600 | 0.7350 | 0.1241 | 0.0581 |
| score | 800 | 0.5788 | 0.1455 | 0.0368 |
| **all** | **2,000** | **0.6695** | **0.1261** | **0.0556** |

## DecisionBench by primitive

| Primitive | n | Accuracy | NLL | ECE |
|---|---:|---:|---:|---:|
| binary classification | 530 | 0.7868 | 0.4489 | 0.0489 |
| candidate selection | 893 | 0.7492 | 0.7902 | 0.0928 |
| ordinal scoring | 142 | 0.2958 | 1.5537 | 0.1550 |
| **all** | **1,565** | **0.7208** | **0.7439** | **0.0586** |

## Evaluation scope

Typed Decisions is the full 2,000-decision test split. DecisionBench is a
1,565-row matched subset limited to ≤16 candidates and ≤2,048-token prompts; it
is not the full 23,900-row suite. The full suite is not claimed.

## Checkpoint selection

| Step | Typed Decisions | DecisionBench 1,565 |
|---:|---:|---:|
| 1,000 | 0.6095 | 0.7534 |
| 1,500 | 0.5550 | 0.7655 |
| **2,000** | **0.6695** | **0.7208** |

The 2,000-step checkpoint is shipped because it is the only tested checkpoint
that clears both d1-2B and d1-3B on both measured benchmarks.