AnkitAI/TinyJev-4B

Typed decisions, on your laptop, in one forward pass. The bigger one.

GitHub · PyPI · TinyJev 0.6B · Examples

Send this model some state, a ticket or a record or a log line, plus questions with the answers you will accept. It returns a probability for every option you offered. It cannot answer with anything else, because it never generates text; it scores the options you gave it and stops.

TinyJev 4B is the same head and the same training data as TinyJev 0.6B on a Qwen3-4B-Base backbone. It is the accurate one: 474 of 500 never-seen decisions against 440, and at confidence ≥ 0.85 it answers 87% of the queue on its own at 99.1%. It costs 8 GB (4.5 GB at 8 bits) and about 630 ms per decision on a base M1, seven times the 0.6B's.

• Choice picks one option from a list, with a probability for each.
• Noul measures whether a statement is true.
• Score places state on an ordered scale.
• Calibrated confidence (ECE 0.022), so a threshold means something and you can decide what to automate.
• A Python API, a local HTTP server, and a System One compatible endpoint.

Models

Two models so far, same head, same training data, scored on the same 500 never-seen cases from 25 domains (benchmarks/opendecision, every case and probability logged). Latency is a base M1 (16 GB) via MLX, one forward pass per case.

Model	Params	OD-500	Gate 0.85	ms / case	Weights
TinyJev 0.6B	596M, 1.2 GB	440 (88.0%)	59% @ 98.0%	85	🤗 AnkitAI/TinyJev-0.6B
TinyJev 4B	4.0B, 8.0 GB	474 (94.8%)	87% @ 99.1%	628	🤗 AnkitAI/TinyJev-4B

OD-500 is correct answers out of 500. Gate 0.85 is the share of decisions answered on its own at confidence ≥ 0.85, and how often those were right. Calibration (ECE 0.071 vs 0.022), coverage at 2% error (63% vs 92%) and transfer-v4 dev (0.625 vs 0.762) are on the benchmark page. Load either with tinyjev.load("TinyJev-0.6B") or tinyjev.load("TinyJev-4B").

Both rows are fp16. Loading with quantize=8 keeps the same weights in half the memory and changes almost nothing: the 0.6B scores 440 at 90 ms, the 4B 473 at 845 ms, one answer in 500 different from fp16. The gate is the number that matters in production; the rest of the queue goes to a person or a bigger model. On the same 500: Kev-0.8B 463, Claude Opus 5.5 496, the same Qwen3-0.6B weights read through letter logits with no head 354.

Measured

On OpenDecision's Original Choice 500, a suite of 25 domains that was not in the training data, with the same 500 inputs for every model:

Model	Correct / 500	Handled alone at confidence ≥ 0.85
Claude Opus 5.5 (cloud, self-reported probabilities)	496	477 at 100.0%
TinyJev-4B (fp16)	474	437 at 99.1%
TinyJev-4B at quantize=8	473	437 at 99.1%
Kev-0.8B (raw logits)	463	186 at 100.0%
TinyJev-0.6B	440	296 at 98.0%

353/375 on dev, 121/125 on holdout, 95% CI 0.928–0.966, ECE 0.022, Brier 0.071, coverage at 2% error 92%. On Kev's transfer-v4 dev it scores 0.762 against 0.625 for the 0.6B (Kev-4B, a full fine-tune, 0.790). Every case, every probability, the coverage curves and all the baseline rows: benchmarks/opendecision.

Known weakness, measured: statement-form yes/no checks. On a 21-item check list about one support email it scores 18 (the 0.6B: 14, it says yes to everything); on hard negatives, statements about a topic the text mentions but does not support, 81%. Training data for that is the next experiment.

Use it

pip install 'tinyjev[mlx]'     # Apple Silicon
pip install 'tinyjev[torch]'   # everything else

import tinyjev
agent = tinyjev.load("TinyJev-4B", quantize=8)   # 4.5 GB in memory; drop quantize for fp16, 8 GB

agent.predict({
    "state": "Shoes arrived two weeks late and in the wrong size. Also I see two charges on my card.",
    "questions": {
        "team":     {"type": "choice", "instructions": "Which team should handle this?",
                     "criteria": {"returns": "Exchanges, refunds, wrong or damaged items",
                                  "shipping": "Delivery status, delays, lost packages",
                                  "billing":  "Charges, invoices, payment problems"}},
        "escalate": {"type": "noul",   "instructions": "Does this need urgent human attention?"},
        "anger":    {"type": "score",  "instructions": "How angry is the customer?",
                     "criteria": ["calm", "frustrated", "very angry"]},
    }})

Eight bits changed one answer in 500 on the held-out suite. Serve it over HTTP, speaking the System One request shape:

tinyjev serve TinyJev-4B --quantize 8     # POST /v1/systemone on 127.0.0.1:8077

What is in this repo

AutoModel.from_pretrained("AnkitAI/TinyJev-4B") loads the backbone on its own, a standard Qwen3Model in fp16 with the LoRA already merged. The decision head lives in head.safetensors, and tinyjev is what turns hidden states into calibrated answers.

How it was built, and how it scores

Qwen3-4B-Base with a pointer head, LoRA r16 at lr 5e-5, seed 2, two epochs on the public jaredpalmer/kev-suites decision-v7 training split (12,576 records), Kev's study runner on one H100 for 45 minutes, the adapter merged back into the base. No held-out transfer source was used in training. Temperature 1.0 at inference; the calibration figures above are at raw logits.

	transfer-v4 dev	decision-v7 dev	OpenDecision 500
TinyJev-4B	0.762	0.859	474 / 500
TinyJev-0.6B	0.625		440 / 500

Support the Project

If this model is useful in your work, you can support independent research:

Credits

Built on Qwen3-4B-Base (Apache-2.0). The training data, evaluation suites and the pointer-head design come from Kev by Jared Palmer (Apache-2.0). The typed-decision interface follows TypeSafe's Jev. MIT licensed.