elythros-decider-4b-exp (experimental)

A larger, experimental sibling of elythros-decider-1.7b. Same decision-model design: you give it some text (the state) and one or more typed questions, and it returns a probability for every allowed answer without generating any text. This version is built on Qwen3-4B, using a 4-bit base with a LoRA adapter.

Recommended for multiple-choice and knowledge-heavy questions, where it is clearly better than the 1.7B. For yes/no questions, prefer the 1.7B, or test this model on your own data first: on our blind benchmark it was worse at yes/no, and much worse on one task where most answers are "yes" (see Limitations).

This is an independent research release. It is not affiliated with or endorsed by TypeSafe (makers of Jev) or Qwen.

Which model should I use?

	elythros-decider-1.7b	elythros-decider-4b-exp
Multiple-choice / knowledge / commonsense	good	better
Yes/no questions on new tasks	better	weaker, especially when one answer is much more common
Calibration (honest confidence)	better	somewhat worse
Speed	~2.5–3× faster	slower (~255 ms for 1 question × 4 options)
Download	3.4 GB	2.4 GB (4-bit)

On our 9-task blind benchmark (never used for any tuning decision), the 4B averages 0.817 against 0.804 for the 1.7B. That difference is within noise overall, but splits clearly:

• multiple choice: 0.881 against 0.832;
• yes/no: 0.736 against 0.770.

Quickstart

pip install mlx mlx-lm huggingface_hub
hf download Elythros/elythros-decider-4b-exp --local-dir elythros-decider-4b-exp

import sys; sys.path.insert(0, "elythros-decider-4b-exp")
from decider import Decider

d = Decider.load("elythros-decider-4b-exp")
d.decide(
    state="A student pours salt on an icy sidewalk and the ice melts even though it is below 0 °C.",
    questions={
        "why":   {"type": "choice", "instructions": "Why does the ice melt?",
                  "options": ["salt lowers the freezing point of water", "salt is warm",
                              "salt absorbs sunlight", "salt makes ice heavier"]},
        "topic": {"type": "choice", "instructions": "Which school subject is this about?",
                  "options": ["Biology", "Chemistry", "Maths", "Physics"]},
    },
)

decide(..., calibrated=False) returns raw probabilities. decide(..., contextual=False) turns off the yes/no contextual calibration (see Limitations). python decider.py runs a demo.

How it works

The design is the same as the 1.7B model; see its card for details.

• The state is encoded once, and every question is scored in parallel on a copy of its KV cache.
• A learned <OPT> embedding follows every option. Its final hidden state goes through RMSNorm and a linear head, giving one logit per option.
• Yes/no questions are additionally contextually calibrated: P(yes) is divided by the model's answer for content-free states ("N/A", "", "[MASK]") (Zhao et al. 2021).

4-bit base + adapter. The base is mlx-community/Qwen3-4B-4bit, unchanged. The trained LoRA is shipped separately (adapters.safetensors) and applied by mlx-lm at load time. We don't fuse it into the weights, because re-quantizing to 4 bits after adding the LoRA update rounded most of it away and changed answers badly in our tests.

Files

• model.safetensors and config.json: the untouched 4-bit Qwen3-4B base (2.3 GB).
• adapters.safetensors and adapter_config.json: the trained LoRA (rank 16, 28 of 36 layers, 103 MB).
• decision_head.safetensors: the <OPT> embedding, readout norm and head.
• decider_config.json: prompt format, temperatures and contextual-calibration settings.
• decider.py: the whole inference code.

Latency (M4 Pro):

State of 148 tokens, median of 10 runs; the 1.7B for comparison:

Request	This model	elythros-decider-1.7b
1 choice question × 4 options	~255 ms	~105 ms
10 choice questions × 4 options	~545 ms	~195 ms
1 choice question × 255 options	~2.2 s	~0.71 s
1 yes/no question (content-free baseline cached)	~255 ms	~100 ms
1 yes/no question, first time its text is seen	~460 ms	~200 ms
10 yes/no questions (cached)	~470 ms	~170 ms

About 2.4–3× slower than the 1.7B.

Training

• Starting point: Qwen3-4B with 4-bit weights (QLoRA), frozen.
• Trained parameters: LoRA (rank 16) on 28 of 36 layers, plus the <OPT> embedding and head, about 26M in total.
• Run: 8,000 steps at up to 1,536 tokens per step, AdamW at lr 2e-5 with 300 warmup steps and cosine decay.
  • Took about 7.8 hours on one M4 Pro (24 GB), peaking at 13.9 GB.
  • The best checkpoint by validation NLL was at step 4,000.
• Data: identical to elythros-decider-1.7b: 23 sources, including the synthetic minimal-pair set. See that card for the full table and licence notes.
• Calibration: temperature scaling on validation data (choice T = 1.32, yes/no T = 1.85, fitted on 100 requests per source), plus yes/no contextual calibration at inference.

Evaluation

All numbers are for these released files (calibrated, contextual calibration on yes/no), next to elythros-decider-1.7b scored the same way.

1. Blind benchmark (200 requests each, never used for any tuning decision)

Task	Type	elythros-decider-1.7b	This model
ARC-Challenge (science, 4-way)*	choice	0.795	0.850
CommonsenseQA (5-way)	choice	0.629	0.747
COPA (cause / effect, 2-way)	choice	0.905	0.940
SciQ (science, 4-way)	choice	0.905	0.925
School-subject classification (4-way)	choice	0.925	0.945
Climate-topic detection (79% "yes")	yes/no	0.865	0.655
Environmental-claim detection	yes/no	0.765	0.790
Insincere Quora questions (sampled 1:1)	yes/no	0.860	0.855
Word in context (same meaning?)	yes/no	0.590	0.645
Mean, choice (5)		0.832	0.881
Mean, yes/no (4)		0.770	0.736
Mean, all 9		0.804	0.817
Mean ECE (lower is better)		0.090	0.107

* ARC test questions were never trained on, but ARC training questions are part of MMLU's auxiliary training set.

2. Test splits of the training datasets (500 requests each)

Task	elythros-decider-1.7b (acc / ECE)	This model (acc / ECE)
Banking77	0.806 / 0.088	0.820 / 0.049
CLINC intent	0.834 / 0.035	0.880 / 0.064
CLINC out-of-scope (F1)	0.670	0.576
MNLI	0.860 / 0.042	0.874 / 0.109
SNLI	0.886 / 0.030	0.910 / 0.127
BoolQ	0.846 / 0.068	0.844 / 0.072
SST-2	0.934 / 0.017	0.938 / 0.023
MMLU	0.548 / 0.055	0.640 / 0.064
Mean accuracy (8 questions)	0.819	0.843
Mean ECE	0.048	0.069

3. Earlier held-out tasks (200 requests each)

These tasks were analysed while developing the 1.7B, so they are no longer blind. Indicative only.

Task	elythros-decider-1.7b	This model
Emotion (6-way)	0.480	0.530
TREC question type (6-way)	0.685	0.790
Enron spam (yes/no)	0.800	0.800
Subjectivity (yes/no)	0.855	0.755
Amazon counterfactual (yes/no, 17.5% yes; macro-F1)	0.521	0.668
Financial tweet sentiment (3-way; majority baseline 0.675)	0.745	0.545

Limitations

• Yes/no on tasks with a lopsided base rate. Contextual calibration assumes that, before reading the text, "yes" and "no" are about equally likely. When one answer is much more common, it pushes toward 50/50 and hurts.
  • Climate-topic detection (79% "yes") dropped to 0.655 with this model, against 0.865 for the 1.7B.
  • If you know your task is lopsided, try contextual=False and check on your own data. We have not measured that setting on the blind benchmark.
• Other yes/no and sentiment weak spots: subjectivity 0.755 (says "yes" too often) and financial-tweet sentiment 0.545 (below always answering "neutral"). These tasks were analysed earlier, so treat those numbers as indicative only.
• Calibration is somewhat worse than the 1.7B (higher ECE), partly because the temperatures were fitted on only 100 validation requests per source.
• Experimental: only one training run, not tuned further. The 4-bit base costs a little quality compared with a full-precision base.
• Benchmark caveat: ARC-Challenge test questions were never trained on, but MMLU's auxiliary training data (which we use) includes ARC training questions.
• Language and length: English only. Requests up to about 1.5k tokens were trained; longer ones are untested.
• Data licences: as for the 1.7B, some training datasets restrict commercial use of the data. Check each dataset card.

License

Apache-2.0 (see LICENSE). Built on Qwen/Qwen3-4B (Apache-2.0), via the 4-bit MLX conversion mlx-community/Qwen3-4B-4bit. Training datasets are under their own licences.