Add Bosun-style percentage matchup tables

86214d1 verified 1 day ago

9.26 kB

license: apache-2.0
language:
  - en
library_name: pytorch
pipeline_tag: text-classification
base_model:
  - Qwen/Qwen3-1.7B
  - Hanno-Labs/bosun-v3.1-1.7b
datasets:
  - LocalLLaMA/typed-decisions
  - SargeDev/jev-distill-corpus-v3
tags:
  - decision-model
  - system-one
  - calibrated-decisions
  - typed-decisions
  - choice
  - noul
  - score
  - qwen3
  - bosun
  - safetensors
  - pytorch
model-index:
  - name: Pluto-2.1-1.7B
    results:
      - task:
          type: text-classification
          name: Typed Decisions
        dataset:
          name: LocalLLaMA/typed-decisions
          type: LocalLLaMA/typed-decisions
        metrics:
          - type: accuracy
            name: Accuracy
            value: 0.6695
          - type: brier
            name: Brier
            value: 0.1261
          - type: kl
            name: KL
            value: 0.2139
      - task:
          type: text-classification
          name: DecisionBench matched subset
        dataset:
          name: Hanno-Labs/decision-bench
          type: Hanno-Labs/decision-bench
        metrics:
          - type: accuracy
            name: Accuracy
            value: 0.7208
          - type: brier
            name: Brier
            value: 0.373
          - type: nll
            name: NLL
            value: 0.7439

Pluto-2.1

A 1.7B decision model that clears both d1 checkpoints and owns the balance.

Pluto-2.1 reads a state and typed questions, then returns a calibrated probability distribution for every candidate. It does not generate an answer, so there is nothing to parse, nothing to retry, and nothing to hallucinate.

Built solo on one RTX 5060 Ti 16GB.

Headline: 0.6695 on Typed Decisions and 0.7208 on the matched DecisionBench comparison. It clears both d1-2B and d1-3B on both suites and posts the highest mean accuracy in the comparison at 0.6952 — from a 1.738B standalone checkpoint with 2–255 candidates, choice / noul / score, and Apache-2.0 weights.

The receipts

Same rows and same scorer for every locally measured model. Mean is the mean of Typed Decisions and DecisionBench matched accuracy.

Model	Params	Typed Decisions	DecisionBench matched	Mean
Pluto-2.1	1.738B	0.6695	0.7208	0.6952
Pluto-2 400M	0.397B	0.7835	0.5789	0.6812
Bosun 3.1 1.7B (initializer)	1.738B	0.4980	0.8288	0.6634
d1-3B	3.123B	0.6495	0.6486	0.6490
d1-2B	1.918B	0.6325	0.6339	0.6332

What this means

+3.7 points over d1-2B and +2.0 points over d1-3B on Typed Decisions.
+8.7 points over d1-2B and +7.2 points over d1-3B on DecisionBench.
Highest mean accuracy across the two suites.
1.738B: 9.4% smaller than d1-2B and 44% smaller than d1-3B.
Sharper distributions than both d1 checkpoints: lower Typed Brier and KL, lower DecisionBench NLL and Brier.
One forward pass. No autoregressive decoding and no token bill.
Bosun leads broad-only; Pluto-2 leads Typed-only; Pluto-2.1 owns the balance.

Typed Decisions — full 2,000-decision board

Model	Params	Mode	Primary accuracy	ECE	Brier	KL	Coverage
Pluto-2 Typed	395.8M	fitted	80.30%	7.48%	6.72%	0.119	100%
OpenDecider-nano	394.8M	fitted	79.30%	16.09%	4.28%	0.079	100%
Pluto-2	397.4M	fitted	78.35%	11.80%	5.32%	0.097	100%
Laya-typed	421.3M	fitted	76.80%	21.52%	6.15%	0.117	100%
Pluto-1	332.8M	fitted	75.15%	14.00%	6.49%	0.118	100%
Jev 1.13	~27B	published	72.70%	14.40%	14.80%	1.442	—
Pluto-2.1	1.738B	fine-tuned	66.95%	5.56%	12.61%	0.214	100%
d1-3B	3.123B	zero-shot	64.95%	3.43%	14.16%	0.285	100%
d1-2B	1.918B	zero-shot	63.25%	3.90%	14.40%	0.296	100%
Bosun 3.1 1.7B	1.738B	zero-shot	49.80%	18.48%	30.15%	0.590	100%

DecisionBench — 1,565-row matched board

Model	Params	Primary accuracy	ECE	NLL	Brier	Coverage
Bosun 3.1 1.7B	1.738B	82.88%	6.10%	0.468	22.88%	100%
Pluto-2.1	1.738B	72.08%	5.86%	0.744	37.30%	100%
d1-3B	3.123B	64.86%	11.74%	1.034	49.37%	100%
d1-2B	1.918B	63.39%	4.48%	0.933	45.10%	100%
Pluto-2 400M	397.4M	57.89%	3.04%	1.087	54.77%	100%

Coverage is 2,000/2,000 on Typed Decisions and 1,565/1,565 of the DecisionBench matched subset, which is 6.55% of the full 23,900-row suite. Jev's published card does not report coverage, so it is marked -. The matched subset is the shared accessibility slice, not the full suite.

Two tracks. Pluto-2.1 is the balanced generalist. The 400M Pluto-2 Typed checkpoint is the raw Typed specialist and beats every fitted specialist above.

Comparison hygiene

The Typed Decisions result is the full 2,000-decision test split. The DecisionBench result is the 1,565-row matched subset used for every model in the table: ≤16 candidates and ≤2,048-token prompts, not the full 23,900-row suite. Bosun, Pluto-2, d1-2B, and d1-3B were all run locally on the same rows and scored with the same code. Raw summaries are in benchmarks/. The full DecisionBench suite is not claimed.

Architecture

Qwen3-1.7B backbone with 256 stable decision-token slots.
The fine-tuned adapter is merged into the base; this repository is self-contained.
One forward pass scores every presented candidate.
Up to 255 runtime choices per question.
No d1 weights, no d1-generated training targets, no ensemble, and no runtime dependency on another model service.

Lineage: Qwen3-1.7B → Hanno-Labs Bosun 3.1 1.7B → Pluto-2.1 2,000-step fine-tune → merged standalone checkpoint. Pluto-2.1 is self-contained at inference: no adapter files are needed and no base model is downloaded at runtime.

Quickstart

git clone https://huggingface.co/omida-ai/pluto-2.1
pip install -e "pluto-2.1/code"

from pluto21 import Pluto21

model = Pluto21("pluto-2.1")
out = model.predict(
    state={"transaction": "Duplicate charge confirmed by the bank."},
    instructions="Was the duplicate charge confirmed?",
    candidates=[
        {"id": "yes", "label": "Yes"},
        {"id": "no", "label": "No"},
    ],
    decision_type="noul",
    row_id="demo-1",
)
print(out["probabilities"])

out = model.predict(
    state={"ticket": "Customer was charged twice and the parcel is late."},
    instructions="Which team should handle this?",
    candidates=[
        {"id": "billing", "label": "Billing", "description": "Charges and refunds"},
        {"id": "shipping", "label": "Shipping", "description": "Delivery and parcels"},
        {"id": "technical", "label": "Technical", "description": "App or site faults"},
    ],
    decision_type="choice",
    row_id="demo-2",
)
print(out["probabilities"])

The runtime returns candidate-aligned probabilities. It never generates text.

Model details


Parameters	1,737,985,024 (1.738B)
Backbone	Qwen3-1.7B, 28 layers, 40,960-token context
Decision interface	256 stable decision-token slots
Candidates	2–255 per question
Outputs	`choice`, `noul`, `score` distributions
Weights	BF16, 3.44GB
License	Apache-2.0
Initialization	Bosun 3.1 1.7B (Apache-2.0)
Base	Qwen3-1.7B
Runtime	`pluto21`, PyTorch + Transformers

Training

2,000 LoRA steps on Pluto’s Typed Decisions and JEV decision data.
Selected checkpoint: 2,000 steps, the only tested checkpoint that clears both d1-2B and d1-3B on both measured benchmarks.
Trained and merged on one 16GB consumer GPU.
The adapter is merged into the base; no PEFT dependency at inference.
The Typed Decisions test split was never used for training or checkpoint selection.

Scope

Pluto-2.1 is a decision model, not a chat or reasoning model. The DecisionBench comparison is the matched accessibility subset; the full 23,900-row suite is not claimed. As with any benchmark, validate on your own data before high-stakes use.

License and attribution

Pluto-2.1 weights and runtime are released under Apache-2.0. Upstream components remain under their original licenses:

Qwen3-1.7B, Apache-2.0
Hanno-Labs Bosun 3.1 1.7B, Apache-2.0

See LICENSE, NOTICE, and serving.json. No d1 weights or outputs are included, and d1 is not required at inference.

Citation

@misc{omida2026pluto21,
  title  = {Pluto-2.1: A 1.7B Standalone Decision Model},
  author = {Omida AI},
  year   = {2026},
  url    = {https://huggingface.co/omida-ai/pluto-2.1}
}

Built by Omida AI on one 16GB consumer GPU.