nanodiff-350m-typed-decisions

Experiment code for calibrated single-pass typed decisions with a masked diffusion LM — a reproduction of the mechanism behind TypeSafe's "System One" / Jev, using the LLaDA recipe (nanoDiff) as the base.

Contents

File	Purpose
code/build_dataset.py	Builds pngwn/typed-decisions — Noul / Choice / Score domains with a closed-form Bayes-optimal synthetic generator
code/decision_format.py	Serialization + option-token map (GPT-2 BPE) for decision slots
code/sft_decision.py	LLaDA SFT with a λ-weighted decision-slot (proper-scoring) term — the λ=0 vs λ=1.0 ablation
code/eval_calibration.py	Calibration eval: ECE hard/soft, AUROC, AURC, selective accuracy, temperature scaling, Bayes-oracle distance

Results in one line

The model's own masked-slot distribution is calibrated (overall ECE 0.065 → 0.036 with the λ term, vs 0.333 for the accuracy-trained "confidence = 1.0" reading of the same predictions, at identical accuracy ≈ 0.67), and k=4 multi-question scoring in one pass stays the best-calibrated block (ECE 0.020, acc@0.5 = 0.998).

• Full report: REPORT.md
• Dataset: pngwn/typed-decisions
• Checkpoints: lam0 (control) · lam1 (proper-scoring)

Base model: Sebasdi/nanodiff-350m-base (MIT).