AnkitAI/TinyJev-0.6B

Typed decisions, on your laptop, in one forward pass.

GitHub · PyPI · Examples

English · 简体中文 · 日本語 · 한국어

Send this model some state, a ticket or a record or a log line, plus questions with the answers you will accept. It returns a probability for every option you offered. It cannot answer with anything else, because it never generates text; it scores the options you gave it and stops.

• Choice picks one option from a list, with a probability for each.
• Noul measures whether a statement is true.
• Score places state on an ordered scale.
• Confidence is calibrated, so a threshold means something.

596M parameters, about 1.2 GB. MLX on Apple Silicon, PyTorch everywhere else, fully offline.

Watch it decide

Six decisions from six domains it never trained on, the same question to both models at the same instant. TinyJev answers in one forward pass: 6 of 6 right, 86 ms a question on a base M1. GPT-6 Sol writes the answer as JSON, token by token: 6 of 6 right, 2,042 ms a question. Both lanes are real runs replayed at real speed; the API lane is a recording with wall-clock timestamps. The questions, the answers and both timings are in assets/recordings.

pip install 'tinyjev[mlx,demo]'
python demos/race.py --recording assets/recordings/gpt-6-sol.jsonl --llm-name "GPT-6 Sol" --gif demo.gif

Twelve decisions about one ticket, in one pass — click to expand

One support ticket, twelve typed decisions: which team, what priority, what the customer wants. TinyJev scores every option of every question in a single forward pass and returns all twelve together, 596 ms, 10 of 12 right. GPT-6 Sol writes the same twelve as JSON: 2,198 ms, 12 of 12 right. The ticket, the questions, the expected answers and both measurements are in demos/cases and assets/recordings.

python demos/batch_race.py --data assets/recordings/batch-support-ticket-gpt-6-sol.json --gif demo.gif

Eight tickets, three questions each — click to expand

Eight real support tickets, one after another, on a base M1. Three questions per ticket in a single forward pass, about 110 ms each. Every number in that recording came from a live run.

python demos/triage_desk.py --gif demo.gif

And, for fun, Doom — click to expand

TinyJev is text-only, so it never sees the game pixels. VizDoom supplies health, ammo, enemy positions, recent damage and the location of the goal. A small rules-based router picks the tactical mode; TinyJev chooses a tactic and returns its probabilities; ordinary code handles aiming and key presses. In this fixed-seed run it kills all six enemies and reaches the goal.

pip install 'tinyjev[mlx,doom]'
python demos/doom_corridor.py --gif tinyjev_doom.gif

This is a demo, not a benchmark: on structured numeric state the answer barely moves with the input, and the router does the game's work.

Models

Two models so far, same head, same training data, scored on the same 500 never-seen cases from 25 domains (benchmarks/opendecision, every case and probability logged). Latency is a base M1 (16 GB) via MLX, one forward pass per case.

Model	Params	OD-500	Gate 0.85	ms / case	Weights
TinyJev 0.6B	596M, 1.2 GB	440 (88.0%)	59% @ 98.0%	85	🤗 AnkitAI/TinyJev-0.6B
TinyJev 4B	4.0B, 8.0 GB	474 (94.8%)	87% @ 99.1%	628	🤗 AnkitAI/TinyJev-4B

OD-500 is correct answers out of 500. Gate 0.85 is the share of decisions answered on its own at confidence ≥ 0.85, and how often those were right. Calibration (ECE 0.071 vs 0.022), coverage at 2% error (63% vs 92%) and transfer-v4 dev (0.625 vs 0.762) are on the benchmark page. Load either with tinyjev.load("TinyJev-0.6B") or tinyjev.load("TinyJev-4B").

Both rows are fp16. Loading with quantize=8 keeps the same weights in half the memory and changes almost nothing: the 0.6B scores 440 at 90 ms, the 4B 473 at 845 ms, one answer in 500 different from fp16. The gate is the number that matters in production; the rest of the queue goes to a person or a bigger model. On the same 500: Kev-0.8B 463, Claude Opus 5.5 496, the same Qwen3-0.6B weights read through letter logits with no head 354.

Measured

On OpenDecision's Original Choice 500, a suite of 25 domains that was not in the training data, with the same 500 inputs for every model:

Model	Correct / 500	Handled alone at confidence ≥ 0.85
Claude Opus 5.5 (cloud, self-reported probabilities)	496	477 at 100.0%
Kev-0.6B, the checkpoint this reproduces (raw logits)	441	378 at 98.2%
TinyJev-0.6B	440	296 at 98.0%
Same Qwen3-0.6B weights, no head, letter logits	354	79 at 97.5%

330/375 on dev, 110/125 on holdout, 95% CI 0.850–0.908. The shipped temperature (1.464, fitted in-distribution) flattens confidence out of distribution: at raw logits the same gate covers 76% of the queue, matching Kev-0.6B. Reported, not refit. Every case, every probability, the coverage curves and all the baseline rows: benchmarks/opendecision.

Use it

pip install 'tinyjev[mlx]'     # Apple Silicon
pip install 'tinyjev[torch]'   # everything else

import tinyjev
agent = tinyjev.load("TinyJev-0.6B")

agent.predict({
    "state": "Shoes arrived two weeks late and in the wrong size. Also I see two charges on my card.",
    "questions": {
        "team":     {"type": "choice", "instructions": "Which team should handle this?",
                     "criteria": {"returns": "Exchanges, refunds, wrong or damaged items",
                                  "shipping": "Delivery status, delays, lost packages",
                                  "billing":  "Charges, invoices, payment problems"}},
        "escalate": {"type": "noul",   "instructions": "Does this need urgent human attention?"},
        "anger":    {"type": "score",  "instructions": "How angry is the customer?",
                     "criteria": ["calm", "frustrated", "very angry"]},
    }})

On Apple Silicon you can quantize as it loads. Eight bits is free: half the memory, slightly faster, and it scored identically to full precision on our held-out set.

agent = tinyjev.load("TinyJev-0.6B", quantize=8)

Serve it over HTTP, speaking the System One request shape:

tinyjev serve TinyJev-0.6B        # POST /v1/systemone on 127.0.0.1:8077

What is in this repo

AutoModel.from_pretrained("AnkitAI/TinyJev-0.6B") loads the backbone on its own, a standard Qwen3Model in fp16. The decision head lives in head.safetensors, and tinyjev is what turns hidden states into calibrated answers.

How it was built, and how it scores

Qwen3-0.6B-Base with a pointer head, LoRA r16 at lr 5e-5 merged back into the base, trained on the public jaredpalmer/kev-suites decision-v7 split. No held-out transfer source was used in training. A fitted temperature of 1.46 is applied at inference.

	transfer-v4 dev	transfer-v4 test, read once	ECE on test
TinyJev-0.6B	0.625	0.663	0.082
Same-size public anchor (Kev-0.6B)	0.620	0.642	0.128

Scored with the upstream harness on its frozen held-out suite. This matches the same-size public anchor and edges ahead on the locked test with lower calibration error. It is not 4B-class, and it is not meant to be. Full fine-tuning, distillation from a 4B teacher, and a 149M encoder were all tried and all lost to the configuration above.

TinyJev-0.6B is done and published. Next is a smaller one, around 0.15B.

Support the Project

If this model is useful in your work, you can support independent research:

Credits

Built on Qwen3-0.6B-Base (Apache-2.0). The training data, evaluation suites and the pointer-head design come from Kev by Jared Palmer (Apache-2.0). The typed-decision interface follows TypeSafe's Jev. MIT licensed.