elythros-decider-1.7b

A decision model for Apple Silicon. You give it some text (the state) and one or more typed questions, and it returns a probability for every allowed answer. It doesn't generate text, so it can't produce an answer outside the options you supply.

Built on Qwen3-1.7B, trained and run entirely with MLX.
Many questions, one state: the state is read once, then every question is scored in parallel on a copy of its KV cache, so questions can't influence each other.
Question types: choice (2–255 options) and noul (yes/no). score (ordered levels) is experimental and wasn't trained explicitly.
Calibrated: one temperature per question type, plus contextual calibration for yes/no questions (see below). This is what fixed v0.1's habit of answering unfamiliar yes/no questions with the same answer every time.

This is an independent v0.2 research release, the successor to elythros-decider-0.6b. It is not affiliated with or endorsed by TypeSafe (makers of Jev) or Qwen. Read Limitations and Data licences before relying on it.

Quickstart

pip install mlx mlx-lm huggingface_hub
hf download Elythros/elythros-decider-1.7b --local-dir elythros-decider-1.7b

import sys; sys.path.insert(0, "elythros-decider-1.7b")
from decider import Decider

d = Decider.load("elythros-decider-1.7b")
d.decide(
    state="My card was charged twice for one coffee and I'm furious. Please fix it today.",
    questions={
        "topic":   {"type": "choice", "instructions": "What is the ticket about?",
                    "options": ["billing", "shipping", "login", "other"]},
        "angry":   {"type": "noul", "instructions": "Is the customer angry?"},
        "urgency": {"type": "score", "instructions": "How urgent is it?",
                    "levels": ["low", "medium", "high"]},
    },
)
# {"topic": {"type": "choice", "choice": "billing", "probabilities": {...}, "confidence": ...},
#  "angry": {"type": "noul", "noul": <P(yes)>},
#  "urgency": {"type": "score", "choice": ..., "score": <expected level index>, ...}}

decide(..., calibrated=False) returns raw probabilities. decide(..., contextual=False) turns off the yes/no contextual calibration. python decider.py runs a demo.

How it works

State:\n{state}\n                                       <- encoded once
\nQuestion: {q}\nOptions: {opt1}<OPT> {opt2}<OPT> ...   <- one branch per question, batched

<OPT> is a learned embedding inserted after every option (not a vocabulary token).
Readout: the final hidden state at each <OPT> goes through RMSNorm and a linear head, giving one logit per option, then a softmax over that question's options.
Yes/no questions are 2-option choices over yes/no.

Contextual calibration (yes/no only). On a yes/no question the model hasn't seen before, it usually ranks cases well but puts the yes/no cut-off in the wrong place, because it doesn't know how common "yes" is for that question.

In our held-out tests, the ranking quality (AUC) was 0.94 on spam and 0.95 on subjectivity, yet the plain model said "yes" to only 17% of spam emails.
decider.py therefore first asks each yes/no question about three content-free states ("N/A", "", "[MASK]"), then divides P(yes) by that baseline (Zhao et al. 2021).
The baseline is cached per question text, so it costs three extra short passes the first time a question is seen.
It has no tunable parameters.
It is not applied to choice questions, where it hurt: the label priors there are useful.

Files

model.safetensors: the backbone, with the LoRA fused in (bf16, standard mlx-lm Qwen3 format, 3.4 GB).
decision_head.safetensors: the <OPT> embedding, readout norm and head.
decider_config.json: prompt format, temperatures and contextual-calibration settings.
decider.py: the whole inference code.

Latency (M4 Pro):

State of 148 tokens, median of 10 runs:

Request	Latency
1 choice question × 4 options	~105 ms
10 choice questions × 4 options	~195 ms
1 choice question × 255 options	~710 ms
1 yes/no question (content-free baseline cached)	~100 ms
1 yes/no question, first time its text is seen (3 extra passes)	~200 ms
10 yes/no questions (cached)	~170 ms

Roughly 2× the 0.6B model.

Training

Starting point: frozen Qwen3-1.7B base weights.
Trained parameters: LoRA (rank 16, all 28 layers) plus the <OPT> embedding and head, about 17M in total.
Loss: cross-entropy over each question's options.
Run: 8,000 steps at up to 2,048 tokens per step, AdamW at lr 2e-5 with 300 warmup steps and cosine decay.
- Took about 5.5 hours on one M4 Pro (24 GB), peaking at 12.8 GB.
- The best checkpoint by validation NLL was at step 4,500.
Option shuffling: option order is reshuffled every time a request is used.
Calibration: temperature scaling on the validation splits (choice T = 1.17, yes/no T = 1.79), plus contextual calibration for yes/no at inference time.

Training data: 23 sources, sampled by weight (not in proportion to size). Many classification sets are also recast as yes/no questions (for example "Is this article about sports?"), so the same text appears with both answers. That is how yes/no became a general skill rather than being tied to two tasks.

Group	Sources	Decision taught
v0.1 tasks	Banking77, CLINC150 (+ out-of-scope), MNLI, SNLI, BoolQ, SST-2, MMLU auxiliary train	intent, NLI, yes/no passage QA, sentiment, 4-way multiple choice
Topic / entity / language	AG News, DBpedia-14, Yahoo Answers topics, language identification (20 languages), Banking77/CLINC intents as yes/no	category choice + "is this about X?" yes/no
Reviews	IMDB, Yelp (5-star)	sentiment yes/no in both polarities, star rating, rating thresholds
Tweets / comments	TweetEval (hate, irony, offensive), Civil Comments (toxicity + sub-attributes)	content yes/no
Sentence pairs	PAWS, QQP, QNLI, CoLA	paraphrase, duplicate question, answer contained, grammaticality
Reading comprehension	SQuAD v2 (answerable?), MultiRC (is this answer correct?)	several yes/no questions per passage
Synthetic: elythros-decider-synthetic	5,856 requests (1,688 minimal pairs) written by Qwen3-8B (Apache-2.0) and checked by Phi-4 (MIT); split into ≤1,500-token chunks for training	~6 questions per realistic text (tickets, logs, emails, ...) plus "minimal-pair twins": the same text with one edit that flips one answer

Synthetic data quality: about 10% of the synthetic labels are questionable on hand inspection. The typical case is a question the text doesn't actually answer, where both the teacher and the checker read the silence as "no".
Held-out hygiene: no emotion, spam, question-type, subjectivity, counterfactual or financial-sentiment data was used. Every training text that also appears in the held-out sets was removed.

Evaluation

All numbers below are for these released files, run with calibrated probabilities (temperatures plus yes/no contextual calibration). ECE is expected calibration error: lower means the stated confidence is more honest.

1. Test splits of the training datasets (500 requests each, options shuffled)

Task	Zero-shot Qwen3-0.6B	Decision-2.0-Kai-0.6B (acc / ECE)	elythros-decider-0.6b (acc / ECE)	This model (acc / ECE)
Banking77	0.286	0.788 / 0.111	0.834 / 0.046	0.806 / 0.088
CLINC intent	0.330	0.738 / 0.079	0.838 / 0.080	0.834 / 0.035
CLINC out-of-scope (F1)	0.211	0.449	0.717	0.670
MNLI	0.386	0.740 / 0.186	0.844 / 0.070	0.860 / 0.042
SNLI	0.310	0.718 / 0.172	0.858 / 0.051	0.886 / 0.030
BoolQ	0.612	0.768 / 0.052	0.824 / 0.067	0.846 / 0.068
SST-2	0.794	0.856 / 0.159	0.904 / 0.022	0.934 / 0.017
MMLU	0.342	0.420 / 0.061	0.436 / 0.113	0.548 / 0.055
Mean accuracy (8 questions, incl. CLINC out-of-scope accuracy)	0.410	0.724	0.802	0.819

These are this model's training domains, and Kai was not trained on them as far as we know, so this comparison favours this model. Banking77 and the CLINC out-of-scope F1 are slightly below the 0.6B release, which trained on fewer sources.

2. Held-out tasks this model never trained on (200 requests each)

Task	Chance	Decision-2.0-Kai-0.6B	mpuig/system-one-qwen3-0.6b	elythros-decider-0.6b	This model
Emotion (6-way)	0.17	0.520	0.495	0.440	0.480
TREC question type (6-way)	0.17	0.585	0.815	0.645	0.685
Subjectivity (yes/no)	0.50	0.690	0.595	0.560	0.855
Enron spam (yes/no)	0.50	0.495	0.515	0.510	0.800
Mean accuracy		0.573	0.605	0.539	0.705
Mean ECE		0.178	0.105	0.181	0.128

The two yes/no tasks are where the release improves most. "Yes" rates now track the truth: spam 36% against a true 53% (v0.1: 4%); subjectivity 65% against a true 56% (v0.1: 98.5%).
TREC: still behind mpuig.
Emotion: still behind both baselines.
Baselines were run with their official runtimes on the same requests. The held-out tasks avoid this model's and mpuig's training sets; Kai's training data isn't public.

3. Fresh held-out tasks (200 requests each)

Added later and never used for any tuning decision.

Task	Majority-class baseline	This model (acc / macro-F1 / ECE)
Amazon counterfactual statements (yes/no, 17.5% yes)	0.825	0.820 / 0.521 / 0.079
Financial tweet sentiment (bearish / bullish / neutral)	0.675	0.745 / 0.745 / 0.072

Counterfactual is not solved. The model says "yes" only 4% of the time, against a true 17.5%. Its accuracy (0.820) is no better than always answering "no" (0.825).

Limitations

Contextual calibration was chosen after looking at held-out results. We found the yes/no cut-off problem by analysing the held-out tasks, so they are no longer completely blind for this release. Mitigations:
- the method has no tunable parameters, and the content-free states were fixed in advance;
- the decision to apply it only to yes/no questions is supported by the in-domain test sets, not just held-out data.
- Even so, read the held-out numbers as somewhat optimistic.
Counterfactual-style yes/no questions remain hard: about 4% "yes" against 17.5% true. Its accuracy there mostly reflects the majority answer (macro-F1 about 0.5).
Rare-label choice tasks: emotion (6-way) is still weak at about 0.48.
Language and length: English only. Training requests were short (under ~2k tokens); long states are untested.
Score questions: score is untrained and treated as a choice over the levels.
Calibration under domain shift: temperatures were fitted on in-domain validation data. Refit on about 100 labelled examples from your own workload if confidence matters.
Fusion rounding: merging the LoRA into bf16 weights shifts probabilities slightly versus the unfused checkpoint. The scores above are for the fused release.
Size and speed: about 2–3× slower than the 0.6B model, and needs about 4 GB of memory to run.

Data licences

The weights and code are released under Apache-2.0, as far as we are able to grant that. Several training datasets carry their own terms, and some restrict commercial use of the data: