code/README.md

Pluto

A compact, calibrated, typed decision model. Give Pluto a state and typed questions; it returns a probability distribution for every option in a single forward pass. No generation, no JSON parsing, no retries.

• 160.6M parameter general model (ModernBERT-base encoder + a decision tower)
• 149.1M parameter Typed Decisions specialist
• One forward pass for choice, noul and score questions
• Calibrated probabilities and measurable ECE
• Fast: 26.9 ms p50 for five questions on an RTX 5060 Ti
• Apache 2.0, small enough to run on a 16GB consumer GPU

Headline results

Independent LocalLLaMA/typed-decisions benchmark, 400 held-out cases / 2,000 decisions, scored with the benchmark's own metric definition. Laya numbers were measured by us on the same rows.

Model	Params	Accuracy	KL	Brier	ECE	p50
Pluto-Ensemble	332.8M	0.7515	0.118	0.0649	0.140	54.2 ms
Pluto-typed	149.1M	0.7275	0.139	0.0765	0.107	26.9 ms
Laya-typed (measured)	421.3M	0.7680	0.117	0.0615	0.215	66.2 ms
Jev 1.13 (published)	~27B API	0.727	1.442	0.148	0.144	~710 ms
Liquid d1 (published)	API	0.742	0.475	0.155	0.124	~525 ms

Pluto-Ensemble beats Jev 1.13 and d1 on accuracy while being 9-81x smaller and 10-13x faster. It is 1.65 points behind Laya-typed on raw accuracy, but has 35% lower ECE, comparable KL, 18% lower latency, and 21% fewer parameters.

On the general 66-domain held-out corpus (29,955 decisions, never trained on): 0.694 accuracy, 0.081 ECE. Full breakdowns and methodology: RESULTS.md.

Quickstart

pip install -e ".[serve]"

from pluto import PlutoEngine

engine = PlutoEngine("artifacts/pluto-149m")

state = "Hi, we were billed twice for March. Refund the duplicate or we cancel."
questions = {
    "department": {
        "type": "choice",
        "instructions": "Which department should handle this?",
        "criteria": {
            "billing": "invoices, payments and refunds",
            "technical": "bugs and outages",
            "other": "everything else",
        },
    },
    "churn_risk": {
        "type": "noul",
        "instructions": "Does the customer threaten to cancel?",
    },
    "urgency": {
        "type": "score",
        "instructions": "How urgent is this?",
        "criteria": ["not urgent", "soon", "blocking"],
    },
}

result = engine.predict(state, questions)
print(result["answers"]["department"]["choice"])
print(result["answers"]["churn_risk"]["noul"])       # P(true)
print(result["answers"]["urgency"]["score"])          # expected level
print(result["escalate"])

CLI:

pluto --model artifacts/pluto-149m predict \
  --state "Invoice #4411 was charged twice." \
  --questions questions.json

Jev-compatible server:

pluto --model artifacts/pluto-149m serve --port 8000
curl -s localhost:8000/v1/systemone -H 'Content-Type: application/json' \
  -d '{"state":"refund my duplicate charge","questions":{"billing":{"type":"noul","instructions":"Is this about billing?"}}}'

Specialist checkpoints

from pluto.task_specialist import TaskSpecialistEngine
from pluto.ensemble import EnsembleEngine

single = TaskSpecialistEngine("artifacts/pluto-typed-task")
best = EnsembleEngine("artifacts/pluto-typed-ensemble/config.json")

Specialist model card: MODEL_CARD_TYPED.md.

The specialist supports the Typed Decisions question names (action, needs_review, outcome, risk, urgency, category, ...). The general model supports arbitrary typed questions.

Typed questions

Type	Input	Output
choice	criteria: keyed or ordered option descriptions	chosen key + distribution
noul	no criteria	P(true) + distribution over false/true
score	criteria: ordered levels (2-12)	expected level + distribution

The answer space is defined at request time. New schemas need no retraining for the general model.

Architecture

state + question + options
        |
  ModernBERT-base (149M, bidirectional, 640-token budget)
        |
  one hidden state per option marker
        |
  2-layer permutation-equivariant option tower
        |
  scalar scorer --> option distribution
  confidence head --> escalation signal

The decision tower is trained from scratch. The backbone is initialized from answerdotai/ModernBERT-base and fine-tuned end to end. The training objective combines KL distillation from calibrated soft labels, top-1 cross-entropy, ordinal RPS for score, and a confidence BCE auxiliary.

The Typed Decisions specialist adds 13 per-question linear heads on the same backbone; the ensemble averages the specialist with a DeBERTa-v3-base sibling. Both are shipped as separate checkpoints.

Calibration and abstention

• The general checkpoint is calibrated on 30k held-out decisions (ECE 0.081).
• The Typed Decisions specialist is temperature-calibrated per primitive (ECE 0.107, versus Laya-typed's 0.215 on the same rows).
• The general checkpoint has a confidence head but its broad-corpus gate is not certified at 5% risk (AUROC 0.590); do not treat escalate as a guarantee.
• Validate calibration and escalation thresholds on your own domain before high-stakes automation.

Training

python -m train.prepare --source data/jev-distill-v3 --out data/pluto-corpus
python -m train.train --data data/pluto-corpus --out artifacts/pluto-149m \
  --max-steps 4000 --batch-size 32 --freeze-layers 8

python scripts/make_typed_dataset.py --replay 0 --out data/pluto-typed-only
python -m train.task_specialist --data data/pluto-typed-only \
  --out artifacts/pluto-typed-task --init artifacts/pluto-149m

Full commands and hardware notes are in RESULTS.md.

Repository layout

pluto/        inference engine, schema, general model, specialist, ensemble, server, CLI
train/        data preparation, general/specialist training, metrics
eval/         held-out evaluator, Typed Decisions harness, latency, Laya comparison
scripts/      dataset builders, calibration/finalization, HF publisher, launch copy
space/        Gradio demo for Hugging Face Spaces
tests/        schema and calibration tests

Limitations

• English only.
• A System-1 decision model, not a reasoning model; it is not evaluated on MMLU-Pro, GPQA, HLE or the full Decision Index.
• Laya-typed remains ahead on raw accuracy on the Typed Decisions test split.
• The general broad-corpus abstention head is weak; use the specialist for risk-sensitive workflows and validate the gate on production data.
• Benchmark results are synthetic and public; validate on your own domain.

License

Apache 2.0. The distillation corpus and the Typed Decisions benchmark are Apache 2.0. See NOTICE for attribution.

Built by Omida.