# typed-decisions: single-question protocol vs full-context stress tests

- **single-question**: one prompt per question (the trained format). Specialist protocol: the model was trained on typed-decisions train.
- **full-case context**: every prompt holds the state and ALL the case questions and ends with "Answer question k"; one forward per question (batched as rows of one prefill).
- **full-case joint**: one prompt with all questions; answers read in sequence as "k: letter", each conditioned on the model own previous answers (one prefill + n-1 incremental steps).
- Neither full-case readout answers five independent decisions in one forward, and neither reproduces the public (DecisionEval) protocol: they are stress tests of request shape, not comparable scores.
- Canonical inference: bf16 weights under bf16 autocast. CI = bootstrap over cases. Brier and KL against the teacher distribution (dataset-card definitions). our-ECE: 10 equal-width bins of top-1 confidence, not comparable with published ECE. Score MAE: our definition (|E_p[level] - E_gold[level]|), no public code to match.

| model | test | protocol | correct/n | acc | 95% CI | choice | score | noul | log loss | KL | Brier | our-ECE | score MAE |
|---|---|---|---:|---:|---|---:|---:|---:|---:|---:|---:|---:|---:|
| Decision-FT Stage 1 | typed_en | single-question | 1510/2000 | 0.755 | 0.734–0.776 | 0.685 | 0.745 | 0.838 | 0.646 | 0.133 | 0.070 | 0.123 | 0.242 |
| Decision-FT Stage 1 | typed_en | full-case context | 857/2000 | 0.428 | 0.407–0.450 | 0.330 | 0.451 | 0.497 | 1.210 | 0.496 | 0.269 | 0.111 | 0.507 |
| Decision-FT Stage 1 | typed_en | full-case joint | 706/2000 | 0.353 | 0.332–0.374 | 0.322 | 0.284 | 0.477 | 1.217 | 0.458 | 0.255 | 0.124 | 0.727 |
| Decision-FT Stage 1 | typed_es | single-question | 1482/1985 | 0.747 | 0.725–0.768 | 0.675 | 0.736 | 0.832 | 0.655 | 0.133 | 0.070 | 0.115 | 0.239 |
| Decision-FT Stage 1 | typed_es | full-case context | 832/1985 | 0.419 | 0.398–0.440 | 0.397 | 0.379 | 0.494 | 1.200 | 0.475 | 0.261 | 0.107 | 0.552 |
| Decision-FT Stage 1 | typed_es | full-case joint | 673/1985 | 0.339 | 0.317–0.360 | 0.281 | 0.280 | 0.476 | 1.204 | 0.445 | 0.247 | 0.112 | 0.690 |
| Decision-FT Stage 1 | telepatia_es | single-question | 1492/2000 | 0.746 | 0.725–0.768 | 0.692 | 0.719 | 0.837 | 0.677 | 0.137 | 0.072 | 0.130 | 0.261 |
| Decision-FT Stage 1 | telepatia_es | full-case context | 823/2000 | 0.411 | 0.390–0.433 | 0.332 | 0.416 | 0.485 | 1.190 | 0.467 | 0.260 | 0.104 | 0.530 |
| Decision-FT Stage 1 | telepatia_es | full-case joint | 636/2000 | 0.318 | 0.296–0.340 | 0.208 | 0.266 | 0.497 | 1.233 | 0.467 | 0.260 | 0.151 | 0.687 |
| Ines-1 (release) | typed_en | single-question | 1528/2000 | 0.764 | 0.742–0.785 | 0.715 | 0.745 | 0.838 | 0.635 | 0.121 | 0.065 | 0.132 | 0.234 |
| Ines-1 (release) | typed_en | full-case context | 831/2000 | 0.415 | 0.395–0.436 | 0.255 | 0.479 | 0.492 | 1.195 | 0.487 | 0.266 | 0.118 | 0.540 |
| Ines-1 (release) | typed_en | full-case joint | 742/2000 | 0.371 | 0.349–0.393 | 0.287 | 0.367 | 0.460 | 1.182 | 0.437 | 0.246 | 0.089 | 0.618 |
| Ines-1 (release) | typed_es | single-question | 1496/1985 | 0.754 | 0.732–0.774 | 0.699 | 0.739 | 0.827 | 0.639 | 0.119 | 0.065 | 0.123 | 0.236 |
| Ines-1 (release) | typed_es | full-case context | 859/1985 | 0.433 | 0.411–0.454 | 0.355 | 0.443 | 0.496 | 1.162 | 0.450 | 0.251 | 0.099 | 0.546 |
| Ines-1 (release) | typed_es | full-case joint | 763/1985 | 0.384 | 0.362–0.407 | 0.379 | 0.317 | 0.479 | 1.172 | 0.421 | 0.233 | 0.077 | 0.678 |
| Ines-1 (release) | telepatia_es | single-question | 1526/2000 | 0.763 | 0.742–0.785 | 0.730 | 0.738 | 0.830 | 0.639 | 0.119 | 0.064 | 0.134 | 0.236 |
| Ines-1 (release) | telepatia_es | full-case context | 878/2000 | 0.439 | 0.418–0.460 | 0.372 | 0.449 | 0.493 | 1.147 | 0.441 | 0.247 | 0.092 | 0.539 |
| Ines-1 (release) | telepatia_es | full-case joint | 745/2000 | 0.372 | 0.350–0.395 | 0.383 | 0.295 | 0.465 | 1.175 | 0.424 | 0.236 | 0.083 | 0.672 |

Dataset reference (card): Prior 0.470 accuracy, Brier 0.189.

## Paired (exact McNemar, same questions)

| model | test | comparison | n | only first right | only second right | p |
|---|---|---|---:|---:|---:|---:|
| Decision-FT Stage 1 | typed_en | single-question vs full-case context | 2000 | 797 | 144 | 3.6e-110 |
| Decision-FT Stage 1 | typed_en | full-case context vs full-case joint | 2000 | 453 | 302 | 4.3e-08 |
| Decision-FT Stage 1 | typed_es | single-question vs full-case context | 1985 | 769 | 119 | 3.8e-117 |
| Decision-FT Stage 1 | typed_es | full-case context vs full-case joint | 1985 | 413 | 254 | 7.9e-10 |
| Decision-FT Stage 1 | telepatia_es | single-question vs full-case context | 2000 | 795 | 126 | 2.3e-119 |
| Decision-FT Stage 1 | telepatia_es | full-case context vs full-case joint | 2000 | 448 | 261 | 2.1e-12 |
| Ines-1 (release) | typed_en | single-question vs full-case context | 2000 | 847 | 150 | 1.5e-118 |
| Ines-1 (release) | typed_en | full-case context vs full-case joint | 2000 | 474 | 385 | 0.0027 |
| Ines-1 (release) | typed_es | single-question vs full-case context | 1985 | 774 | 137 | 1.7e-108 |
| Ines-1 (release) | typed_es | full-case context vs full-case joint | 1985 | 416 | 320 | 0.00045 |
| Ines-1 (release) | telepatia_es | single-question vs full-case context | 2000 | 784 | 136 | 2.4e-111 |
| Ines-1 (release) | telepatia_es | full-case context vs full-case joint | 2000 | 463 | 330 | 2.6e-06 |