▶ Watch it play, live · MLX (Apple silicon) · GGUF (llama.cpp) · Which version? · Quickstart · Benchmarks

qev-0.8b: the tiny local decision model

qev reads anything (a game board, a post, an inbox, a web page, a log) and answers any question about it with calibrated probabilities in a single forward pass. Out of 27 decision models of its size it scores highest on Benchmark Heaven, it answers faster than all of them, and in 8-bit it needs 1.3 GB of RAM on a Mac.

What you're watching. These are screen recordings of the live demo, with nothing staged. One copy of qev-0.8b on one GPU runs all of these at once, in real time: 🐍 Snake, 🧱 Tetris, 👻 Pac-Man, 🔢 2048, 🏓 Breakout and 🔨 Whac-A-Mole. Tetris, Pac-Man, 2048, Breakout and Whac-A-Mole were held out of training. 🧹 LinkedIn slop screening: bait, humblebrag, made-up story, buzzwords and substance, five questions per post. 🛡️ A prompt-injection firewall for an agent's inputs. 🧭 An agent tool router, 🎧 support triage, ⭐ review ratings and 📥 inbox sorting: six emails and 18 questions in one request. For the games, the game code writes down what each move would do ("it hits the wall and dies", "clears 2 lines"), and qev answers one yes/no question per move. The most likely "yes" is played. qev does no search and no lookahead; every move is one forward pass of about 4 ms.

At a glance

	qev-0.8b
parameters	0.8B (full fine-tune of Qwen3.5-0.8B-Base)
Benchmark Heaven	77.1%: first of 27 decision models around 1B
latency	3.5 ms p50 per request on 1× L40S: fastest of 27 · 109 ms on an M4 Pro Mac (MLX)
memory	1.7 GB weights in bf16, 0.8 GB in 8-bit · 1.3 GB RAM on a Mac in 8-bit
also first on	JevExtra (72.0%, agent and safety skills), JevSpatial (74.2%, games and space), Rules-7 (63.7%, tied)
output	calibrated probabilities for choice, noul (yes / no) and score questions, up to 255 options each
images	zero-shot image input (<image:NAME> in the state, ≤196 tokens per image) on the fast CUDA engine or with qev_image.py: 0.759 on JevVision v0 vs 0.651 for its base · details
formats	PyTorch + fast CUDA engine (this repo), MLX, GGUF (also served by stock llama-server -hf labqoat/qev-0.8b-GGUF:Q8_0 at /v1/systemone)
license	Apache-2.0

qev does not generate text. Each answer is a probability distribution, so you can act when it is confident and escalate to a person or a larger model when it is not. Nothing has to leave your machine.

Which version should I use?

Where you run it	Use	Text, one request (p50)	Images, one image per request (batch 1)
NVIDIA GPU, server or Python (recommended for production)	labqoat/qev-0.8b: pip install -r requirements.txt -r requirements-cuda.txt, then python qev_server.py or QEV.from_pretrained("labqoat/qev-0.8b", images=True) (the fast CUDA engine)	3.5 ms (L40S, benchmark items); 5.7 ms on longer dev requests	18–32 ms (L40S, 224×224 to 768×768)
Mac (Apple silicon)	labqoat/qev-0.8b-MLX: qev_mlx.py	109 ms (M4 Pro, bf16, dev requests; 57 ms up to 512 tokens)	not measured on a Mac yet
llama.cpp anywhere: CPU, Metal, CUDA, Vulkan	labqoat/qev-0.8b-GGUF: llama-server -hf labqoat/qev-0.8b-GGUF:Q8_0 for text, llama-server -hf labqoat/qev-0.8b-GGUF:vision-Q8_0 for text plus images	about 17 ms (llama-server, M4 Pro, Metal, a short one-question request)	25 ms (llama-server, L40S CUDA, 448×448, one question); not measured on a Mac
Anything else (Apple MPS or CPU with PyTorch)	labqoat/qev-0.8b: qev.py (transformers backend; pip install -r requirements.txt) and qev_image.py	not measured	61–69 ms (qev_image.py on L40S CUDA); MPS / CPU not measured

All four serve the same model with the same System One API (POST /v1/systemone) and the same answer format. The recommended path is the main repo's fast CUDA engine on an NVIDIA GPU, through qev_server.py or qev.py.

Recommended encoding. Use per_question=True for image requests and for requests with several independent questions. On the server, add "encoding": "per_question" to the request (or start it with --encoding per_question); in qev_mlx.py, use encoding="per_question"; llama-server's vision file always answers each question in its own prompt. On JevVision v0, per_question raises the fast engine from 0.757 to 0.783 pooled accuracy (ECE 0.115 to 0.105; one L40S, same run). MLX and GGUF measure 0.785 with ECE about 0.10. On text, it raises JevBench coherence from 64.5 to 71.2 at about the same accuracy. The cost is more compute per question. On the fast engine, a text request computes the shared state once (7.9 ms against 5.7 ms p50 on dev requests), while an image request runs one full row per question (about 9% fewer requests/s on JevVision). Single-question requests get the same answers in both encodings.

Benchmarks: 27 decision models around 1B

We ran qev-0.8b against 26 other decision models around 1B parameters. Every model got the same items and scoring, ran on one NVIDIA L40S, and used its own published inference recipe.

Accuracies are in %. ECE, latency and memory are lower-is-better. Bold marks the best value in each column.

model	params	VRAM (GB, est.)	p50 ms	BH	BH ECE ↓	JevBench	coherence	suite	JevExtra	JevSpatial	Rules-7
qev-0.8b	0.8B	2.2	3.5	77.1	0.063	85.1	71.2	61.7	72.0	74.2	63.7
decision-2b (flymy-ai)	2B	5.6	81.3	76.2	0.042	86.0	61.8	60.9	64.1	69.5	58.1
decider-2b	1.9B	4.3	7.2	75.8	0.065	85.9	72.5	70.7	58.8	–	–
EXAONE-4.0-1.2B-JEV	1.2B	3.1	27.8	68.8	0.097	83.9	67.9	67.3	64.0	70.0	58.8
malkuth-2b	2B	4.3	46.2	68.8	0.131	84.8	70.8	66.6	63.6	69.9	63.7
classone (Gemma-4-E2B)	~5B	10.7	56.5	66.7	0.260	82.1	66.6	48.2	52.5	63.4	48.1
jul-2b	2B	5.5	88.9	65.8	0.059	83.3	68.4	57.8	56.4	66.8	58.0
Open-Jev-2B	1.9B	4.3	191.7	64.5	0.127	84.9	76.2	59.9	57.6	72.2	59.7
decision-fast	0.6B	1.7	58.9	63.6	0.106	73.0	54.3	52.0	52.0	48.5	47.1
Tacet Sonata¹	144M	0.8	20.5	62.3	0.148	64.9	45.5	31.6	36.1	50.6	35.4

The other 17 models

model	params	VRAM (GB, est.)	p50 ms	BH	BH ECE ↓	JevBench	coherence	suite	JevExtra	JevSpatial	Rules-7
smalljev-2.5b¹	2.5B	5.6	68.8	61.5	0.142	71.2	61.1	45.6	46.2	53.7	49.3
jevos-v5 (feder-cr), partial	~0.9B	1.2	–	61.0	–	84.1	–	49.9	–	–	–
Quyet-1.0-Small¹	328M	1.8	19.3	59.7	0.066	71.3	56.8	43.8	42.4	48.4	45.4
watt-flash-0.1	140M	0.8	15.7	59.7	0.073	67.1	61.9	48.8	39.1	51.3	36.3
Ghafiri Qwen3.5-0.8B	0.8B	2.0	44.6	59.3	0.083	78.1	61.7	41.6	41.2	62.1	40.1
WaterSheep	~150M	1.1	18.9	59.3	0.127	73.5	57.9	56.4	50.8	57.9	41.0
lev-350m	361M	1.2	24.1	58.4	0.094	66.9	65.5	47.5	44.5	–	–
deem-0.8b¹	0.8B	2.0	54.6	58.0	0.240	75.0	59.3	42.9	48.3	51.1	46.8
Laya	421M	1.3	24.4	58.0	0.095	70.7	56.8	53.7	43.0	47.8	44.1
openjev-verdict¹	151M	1.1	13.5	57.6	0.081	63.9	59.7	32.8	37.9	38.4	34.7
Quyet-1.0-Tiny¹	183M	1.2	17.5	57.1	0.097	67.7	54.9	42.4	46.9	52.6	47.1
Qwen3-1.7B (raw)¹	1.7B	4.6	27.8	55.8	0.411	64.5	70.7	39.8	48.7	52.4	41.3
SimpleJev (Qwen3.5-0.8B)	0.8B	2.2	82.9	55.4	0.183	55.6	46.6	47.6	45.6	54.1	36.1
Open-Jev-DeBERTa¹	435M	2.2	34.4	55.0	0.091	79.0	53.3	38.4	41.5	54.2	42.0
verdict-small	118M	1.0	8.6	53.7	0.115	50.7	70.1	42.6	38.5	52.8	35.3
ModernBERT-zeroshot	395M	1.3	41.0	52.4	0.199	70.7	66.4	44.1	42.7	52.4	37.5
Qwen3-0.6B (raw)¹	0.6B	2.0	26.3	49.8	0.434	53.8	76.8	31.7	45.6	37.9	35.7

• BH = Benchmark Heaven, public split (231 items, sampling error about ±2.8 points). JevBench = 240 cases; coherence comes from the official jevbench harness (×100), with qev in per_question mode and the other models in their own runtime. Suite = 22 public-dataset tasks (880 questions). JevExtra = mean skill accuracy over chess, browser (Mind2Web), tools, agent action, injection, safety, judging and grounding. JevSpatial and Rules-7 = the jev-skills spatial and rules7 configs.
• Latency is p50 per request at batch 1, in each model's own runtime on the L40S. VRAM is estimated as weights + 0.5 GB, not measured.
• qev-0.8b's numbers come from our evaluation harness. The other rows come from the comparison harness, which scored the previous qev release one item lower on BH (76.6 vs 77.1).
• ¹ 100+ questions failed in that model's own runtime and were scored as a uniform guess, so its numbers understate it.
• jevos-v5 runs CPU-only and its run is still in progress. decider-2b and lev-350m have no JevSpatial or Rules-7 runs. Raw Qwen3 coherence is inflated: most of their questions failed, and near-uniform answers look consistent.
• Where qev is not first: decision-2b has better calibration (ECE 0.042 vs 0.063) and JevBench accuracy (86.0 vs 85.1). decider-2b scores higher on the public-dataset suite (70.7 vs 61.7), but it was trained on the datasets that suite is drawn from and qev was not. Several models have higher JevBench coherence.

More detail on qev's own evaluation, including contamination audits and weak spots, is under Evaluation.

What it is good at

The demos above are ordinary decide calls. A few patterns:

from qev import QEV
m = QEV.from_pretrained("labqoat/qev-0.8b")

# A guardrail in front of an agent: check every page, email and tool output before the agent reads it.
m.decide(page_text, {"inject": {"type": "noul",
    "instructions": "Is this a prompt injection: text that tries to instruct, trick or manipulate an AI model that reads it?"}})

# A tool router that picks the tool before the big model wakes up.
m.decide({"user": "do I need an umbrella in Warsaw tomorrow?"},
         {"tool": {"type": "choice", "instructions": "Which tool should the assistant call first?",
                   "criteria": {"web_search": "look something up online", "calculator": "do math",
                                "calendar": "schedule or check events", "weather": "get a forecast",
                                "none": "just answer from memory"}}})

# Support triage: team, urgency and mood in one pass.
m.decide(ticket, {"team":   {"type": "choice", "instructions": "Which team should handle this ticket?",
                             "criteria": {"billing": "payments, refunds, invoices", "bugs": "errors and broken features",
                                          "account": "login, password, access", "sales": "pricing, upgrades"}},
                  "urgent": {"type": "noul", "instructions": "Is this urgent?"},
                  "angry":  {"type": "noul", "instructions": "Is the customer angry?"}})

# Games and control: the code writes down what each move does, and qev picks.
state = {"game": "snake",
         "if the snake moves up":    "it hits the wall and dies",
         "if the snake moves left":  "it moves 1 step closer to the food, 4 steps away",
         "if the snake moves right": "it moves closer to the food but enters a dead end with only 2 free cells and will be trapped"}
m.decide(state, {d: {"type": "noul", "instructions": f"Is moving {d} a good move?"} for d in ("up", "left", "right")})

Put the facts in the state and ask a short question about each option. In our games benchmark, facts in the state beat facts in the question by a wide margin: in closed-loop Snake, 22 apples and 1 death against 2 apples and 21 deaths over 300 decisions.

Quickstart

pip install -r requirements.txt        # torch, transformers (Qwen3.5), decider-ai==1.9.0, huggingface_hub, pillow, torchvision
pip install -r requirements-cuda.txt   # NVIDIA GPU: the fast engine (flash-linear-attention)

from qev import QEV   # qev.py in this repo

m = QEV.from_pretrained("labqoat/qev-0.8b")   # NVIDIA: the fast engine; else transformers on Apple MPS / CPU
m.decide(
    {"ticket": "My card was charged twice for the same order, please refund one."},
    {"dept":   {"type": "choice", "instructions": "Which team should handle this?",
                "criteria": {"billing": "payments and refunds", "tech": "bugs", "sales": None}},
     "urgent": {"type": "noul", "instructions": "Is this urgent?"},
     "anger":  {"type": "score", "instructions": "How upset is the customer?",
                "criteria": ["calm", "annoyed", "angry"]}})

Output (bf16, Apple MPS):

{"dept":   {"type": "choice", "choice": "billing", "confidence": 0.927, "x_p_max": 0.9514, "certainty": 0.7934,
            "probabilities": {"billing": 0.9514, "tech": 0.0299, "sales": 0.0187}},
 "urgent": {"type": "noul", "noul": 0.1561},
 "anger":  {"type": "score", "score": 1.05, "confidence": 0.1821, "x_p_max": 0.4547, "certainty": 0.0306,
            "legend": {"0": "calm", "1": "annoyed", "2": "angry"},
            "probabilities": {"0": 0.2494, "1": 0.4547, "2": 0.2959}}}

More ways to call it:

m.decide(state, questions, per_question=True)        # one prompt row per question (see "Encodings")
m.decide_batch([(state, questions), ...])            # length-sorted padded batches
m.distributions_batch([(state, questions), ...])     # {question: {option: probability}}, unrounded

python qev.py --model labqoat/qev-0.8b --state '{"ticket": "Refund please"}' \
  --questions '{"urgent": {"type": "noul", "instructions": "Is this urgent?"}}'
python qev.py --requests requests.jsonl --per-question     # one {"state", "questions"} per line

qev.py has two backends with the same API, chosen by backend="auto" (or --backend):

• fast (the default on an NVIDIA GPU with requirements-cuda.txt installed): the engine in qev_fast/. It uses packed variable-length batches, flash-linear-attention's Gated DeltaNet kernels, fused projections and CUDA graphs for small calls. per_question requests compute the shared state once.
• transformers: a plain transformers forward, for Apple MPS, CPU, or CUDA without flash-linear-attention. Its answers are the same up to bf16 round-off.

Both use decider-ai (Apache-2.0) for the prompt layout and the answer format. On Apple silicon, qev-0.8b-MLX is several times faster than PyTorch on MPS.

HTTP server (System One API)

qev_server.py serves the model over HTTP in the System One wire format, so code written for that API works unchanged: point it at http://127.0.0.1:8017/v1/systemone.

python qev_server.py                        # labqoat/qev-0.8b on 127.0.0.1:8017 (--host, --port, --encoding per_question)
curl http://127.0.0.1:8017/v1/systemone -H 'Content-Type: application/json' -d '{
  "model": "qev-0.8b",
  "state": {"ticket": "My card was charged twice for the same order, please refund one."},
  "questions": {"dept": {"type": "choice", "instructions": "Which team should handle this?",
                         "criteria": {"billing": "payments and refunds", "tech": "bugs", "sales": null}},
                "urgent": {"type": "noul", "instructions": "Is this urgent?"}}}'

Output (fast backend, one L40S):

{"model": "qev-0.8b",
 "answers": {"dept": {"type": "choice", "choice": "billing", "confidence": 0.9252, "x_p_max": 0.9502, "certainty": 0.7896,
                      "probabilities": {"billing": 0.9502, "tech": 0.0308, "sales": 0.019}},
             "urgent": {"type": "noul", "noul": 0.1733}},
 "usage": {"input_tokens": 86, "output_tokens": 0}, "latency_ms": 5.05}

• POST /v1/systemone: one request (state, questions; model is accepted and ignored; optional "encoding": "per_question").
• GET /health: readiness, backend and the SHA-256 of the weights and tokenizer.
• GET /v1/models: the served model.

Concurrent requests are batched together automatically, with the same answers as one at a time. With QEV_API_KEY set, every call except /health needs Authorization: Bearer <key>. The server uses only the Python standard library on top of qev.py, so it runs wherever qev.py does: the fast engine on NVIDIA GPUs, transformers on Apple MPS or CPU.

Image input

qev can also read images, with the vision tower it inherited from Qwen3.5-0.8B-Base: put <image:NAME> in the state where the image belongs and pass the image under that name. The prompt layout, the letter readout and the per-type temperatures are the same as for text, and so is the answer format. On an NVIDIA GPU, qev.py's fast engine (qev_fast/) takes images directly; qev_image.py runs the same model on plain transformers (any device). A text-only request through qev_image.py gets the same answers as through qev.py (0 top-answer flips on 50 requests, 107 questions).

from qev import QEV   # the fast CUDA engine (requirements-cuda.txt); images need pillow and torchvision

m = QEV.from_pretrained("labqoat/qev-0.8b", images=True)   # images=True: load the vision tower now (else on first use)
m.decide("A customer sent this screenshot: <image:shot>",
         {"error": {"type": "noul", "instructions": "Does the screenshot show an error dialog?"}},
         images={"shot": "screenshot.png"})                  # a path or a PIL image
m.decide_batch([(state, questions, images), (state2, questions2)])   # images are optional per request

On other devices (Apple MPS, CPU, CUDA without flash-linear-attention), use qev_image.py:

from qev_image import QEVImage   # qev_image.py in this repo (transformers; needs pillow and torchvision)

m = QEVImage.from_pretrained("labqoat/qev-0.8b")
m.decide({"ticket": "A customer sent this screenshot: <image:shot>"},
         {"error":  {"type": "noul", "instructions": "Does the screenshot show an error dialog?"},
          "screen": {"type": "choice", "instructions": "Which screen is this?",
                     "criteria": {"checkout": None, "login": None, "settings": None}}},
         images={"shot": "screenshot.png"})          # a path or a PIL image
m.decide_batch([(state, questions, images), ...])    # images are optional per request

An image the state does not mention is appended at the end of the state. Several <image:...> markers in one state work. Each image is resized (aspect kept) to at most max_pixels, 448×448 by default, which costs at most 196 tokens per image (224×224: 49; 640×640: 400 with max_pixels=640*640; QEV.from_pretrained(..., max_pixels=...) on the fast engine). decide_batch(..., usage=True) reports the input and image tokens of each request (on the fast engine, the image tokens are in last_image_tokens).

HTTP. On the fast engine, qev_server.py accepts an "images" field, {name: base64 PNG or JPEG}, and the answer adds usage.image_tokens (requests without images are unchanged). qev_image_server.py serves the same API on the transformers path:

python qev_image_server.py --model labqoat/qev-0.8b --port 8018
curl http://127.0.0.1:8018/v1/systemone -H 'Content-Type: application/json' -d '{
  "state": "Customer screenshot: <image:shot>",
  "images": {"shot": "<base64 of a PNG or JPEG>"},
  "questions": {"error": {"type": "noul", "instructions": "Does the screenshot show an error dialog?"}}}'

Speed. On an NVIDIA GPU, qev.py's fast engine runs images itself: the vision tower once per distinct image, the image embeddings in the packed forward with 3-D rotary positions, CUDA graphs for small calls. On one L40S at batch 1, a request with one image takes about 18–32 ms (224×224: 18 ms, 448×448: 20 ms, 768×768: 32 ms; median of 15), and 5–18 ms when the same image comes again (image embeddings are cached). The plain transformers path (qev_image.py) takes about 61–69 ms. A short text request still takes 3.5 ms. Over the 1,306 benchmark requests below, in batches of 24, the fast engine answered about 73 requests/s and transformers about 59 requests/s in the same run (42 in our earlier measurement). The first pass in a new process was slower, about 35 requests/s, because GPU kernels are compiled once for new batch shapes. The fast engine's answers match qev_image.py up to bf16 round-off. On JevVision v0 it scores 0.757 pooled accuracy and 0.115 ECE, against 0.758 and 0.113. The largest probability difference is 0.05, about the same as between qev_image.py's own batch-1 and batch-24 answers. Text requests are unchanged, with the same probabilities bit for bit and the same speed.

Encoding for images. We recommend per_question=True for image requests. In one run on one L40S (revision fd083b9d, all 1,306 items), the fast engine scored 0.783 pooled accuracy and 0.105 ECE per_question, against 0.757 and 0.115 joint. The gain is largest where one image gets several questions: app screens 0.843 to 0.910, CLEVR counting 0.396 to 0.573, connect-four 0.381 to 0.519. The MLX and GGUF builds measure the same way (0.785, ECE about 0.10, per_question). On the fast engine, an image request in per_question runs one full row per question, so it costs more: 31.8 against 34.8 requests/s over the benchmark in batches of 24, in the same run.

JevVision v0 (zero-shot)

The vision tower and its projector are the base model's, unchanged; qev's training touched only the text layers and used no images. So image support is zero-shot. We measured it on JevVision v0, our internal image decision benchmark: 1,306 items in 17 skills and 6 families, each a state with one image and typed choice, noul or score questions with exact gold answers.

• Code-generated (13 skills, 70 items each, exact gold from the generator): charts (bars, lines, pies, reading a value), scenes of shapes (counting, colours, positions), boards (tic-tac-toe, connect-four, chess), documents (forms, receipts) and app screens.
• Public evaluation splits (4 skills, 100 items each): yes/no questions from VQAv2 and GQA, object counting on CLEVR, and ScienceQA questions with an image.

JevVision is not released: ScienceQA's original terms are non-commercial and the public images are not ours to redistribute.

Accuracy per family (mean over its skills); pooled is the mean over the 17 skills. Every model gets the same prompts, readout and temperatures.

family	uniform	Qwen3.5-0.8B-Base	qev-0.8b	Qwen3.5-2B-Base
charts (4 skills)	0.311	0.755	0.886	0.871
scenes (3)	0.336	0.668	0.834	0.642
documents (2)	0.356	0.679	0.864	0.775
app screens (1)	0.500	0.738	0.843	0.695
boards (3)	0.397	0.438	0.521	0.470
public splits (4)	0.423	0.657	0.679	0.850
pooled (17 skills)	0.373	0.651	0.759	0.733
pooled ECE (lower is better)		0.168	0.116	0.166

qev-0.8b beats its own base by 0.108 pooled with better calibration, and beats the 2.5× larger Qwen3.5-2B-Base on the code-generated skills (0.783 vs 0.697). The 2B is much better on the public splits (0.850 vs 0.679), above all on CLEVR counting (0.89 vs 0.40) and ScienceQA (0.92 vs 0.74). With the image removed from the state, qev scores 0.361 pooled, the uniform level, so these numbers come from the image and not from text priors.

Each skill has 70–100 items, so a single skill's accuracy is uncertain by about ±0.03–0.05; the pooled and family numbers are steadier. The table comes from one set of runs on one L40S; re-running qev-0.8b from the files in this repo gives 0.758 pooled (ECE 0.113), with every family within 0.01 of the table.

Questions and answers

A request has a state (a string, an object or an array) and one or more questions, each of one type:

type	what it asks	answer fields
choice	pick one of the named options (criteria: {name: description or null})	choice, confidence, x_p_max, certainty, probabilities
noul	a yes / no question (instructions, optional criteria {"true": ..., "false": ...})	noul: the probability of true
score	a rating on an ordered scale (criteria: list of levels, low to high)	score (expected level index), confidence, x_p_max, certainty, legend, probabilities

The fields follow decider's format_answer, the /v1/systemone wire format:

• confidence is TypeSafe's confidence. For a choice, it is the top probability rescaled so that uniform = 0. For a score, it is 1 minus the expected distance from the top level, relative to a uniform distribution.
• x_p_max is the top probability.
• certainty is 1 minus the normalised entropy.

A string state is used as it is. An object or array is rendered as JSON with array indices annotated, as decider's server does. A question can have up to 255 options.

How a request is answered

1. The request becomes a prompt row in decider's state-first System One layout: Context: and the state, then each question with its lettered options, ending in an Answer: ( slot.
2. One forward pass reads the logits of the option letter tokens at each answer slot. Only the letters' rows of the tied embedding are used.
3. Each question's answer is the softmax over its own options, sharpened per question type with temperatures fitted on a held-out calibration split: choice 1.7, noul 0.9, score 1.25 (p -> p^(1/t), renormalised). The temperatures and the letter tokens are in qev_config.json.

Encodings

• joint (default): one row per request, with every question behind one copy of the state. This is the cheapest encoding.
• per_question (per_question=True): one row per question, each with its own copy of the state. Each answer then does not depend on which other questions were asked, or in what order. This raises JevBench coherence from 64.5 to 71.2 at about the same accuracy, and on JevVision v0 it raises image accuracy from 0.757 to 0.783 (ECE 0.115 to 0.105) on the fast engine. Its cost depends on the runner. On qev.py's fast engine, a text request computes the shared state once and then runs each question's part. An image request runs one full row per question. The transformers backend runs one full row per question. The MLX runner in qev-0.8b-MLX computes the shared state once, so there it costs about the same as joint.

The two encodings give the same answers for single-question requests. For multi-question requests where consistency across questions matters, use per_question.

Evaluation

All numbers come from one harness, with the same items and scoring for every model. The temperatures were fitted on our own calibration split. No benchmark data was used for training, tuning or calibration.

• Benchmark Heaven (BH): the public split, 231 items. The sampling error of BH accuracy is about ±2.8 points.
• JevBench: 240 cases, 1,938 gold questions. Coherence is scored by the official jevbench harness, ×100.
• Jev suite: 22 public-dataset tasks, 880 questions.
• Pooled: the mean of BH, JevBench and suite accuracy.
• JevExtra: our own skills benchmark: chess, browser, tools, agent action, prompt injection, safety, judging and grounding. The headline is the mean skill accuracy.
  • qev's indirect prompt-injection items share text with training rows (16-gram audit). The table includes them, like every other item.
• JevSpatial: our games and spatial benchmark, 2,040 items: game moves with the facts in the question and in the state, rules in the question, coordinates and small grids. Breakout, Whac-A-Mole, Knife Hit, 2048, Tetris and Pac-Man are never used in training.
• JevosTasks: seven policy and rule tasks built like the ones other System One models report (admission, rental, agent rules, authority, fraud points; ShARC dev; patent phrase similarity test), 3,440 items. The five generated domains are never used in training.

Where two values are given, they are joint / per_question. The two are equal on BH and the suite, which have one question per request.

benchmark	metric	qev-0.8b (joint / per_question)	Qwen3.5-0.8B-Base (zero-shot)	decider-0.8b
Benchmark Heaven	accuracy	0.771	0.528	0.658
Benchmark Heaven	Brier (lower is better)	0.337	0.569	0.430
Benchmark Heaven	ECE (lower is better)	0.063	0.089	0.061
JevBench	accuracy	0.851 / 0.849	0.439	0.822
JevBench	coherence	64.5 / 71.2	51.6	69.6
Jev suite	accuracy	0.617	0.392	0.682
pooled (BH, JevBench, suite)	accuracy	0.746 / 0.745	0.453	0.721
JevExtra	mean skill accuracy	0.720	0.442	0.581
JevSpatial	accuracy	0.742
JevosTasks	accuracy	0.637

Contamination audit. None of the training data shares any 16-token sequence with a BH or JevBench item. In the Jev suite, 2 of 22,773 test-pool items reach 50% 16-gram coverage. The training data was also screened by meaning against every test item, using embeddings with a threshold set from near copies.

Where it is weakest:

• JevBench score questions (4-level scales): accuracy 0.39. Most errors are off by one level, and predictions lean toward the middle levels.
• Long and hard BH items: accuracy 0.51 on long items and 0.57 on the hard tier.
• The suite's wide-label and graded tasks: go_emotions, helpsteer2, stsb.
• Abstaining in agent decisions, when no tool fits or information is missing.

Runtime fidelity.

• On the 902-request dev set used for release checks (1,782 questions), qev.py on an NVIDIA L40S agrees with our CUDA reference path on 99.7% of argmax answers, with mean absolute probability difference 0.003.
• On Apple MPS (bf16) and on CPU (fp32), over the first 50 requests (103 questions), the agreement is 99.0%: one near-tie question flips. The mean difference is 0.003.
• The MLX and GGUF builds agree with the reference on 99.0–99.8% of argmax answers (see their repos).
• These agreement figures were measured on the first release's weights; the architecture, code and quantisation are unchanged.

Speed

Measured on our dev requests: mean 746 tokens, median 417, about 2 typed questions each. These are longer than the benchmark items behind the 3.5 ms p50 in the 27-model comparison above, hence the higher figure here. CUDA: one NVIDIA L40S with our optimised engine (packed variable-length batches, the flash-linear-attention Gated DeltaNet kernels, CUDA graphs for small batches). Apple: an M4 Pro (24 GB) with MLX, bf16, compiled.

setting	hardware	p50 latency	throughput
single request, CUDA graphs	1x L40S	5.7 ms	~100 req/s, ~200 decisions/s
packed batch (32k tokens, ~43 requests)	1x L40S	~370 ms	~117 req/s, ~230 decisions/s
single request, MLX bf16	Apple M4 Pro	109 ms	~6 req/s, ~11 decisions/s

per_question (shared prefix): 7.9 ms p50 on the L40S. MLX prefix-cache hit: 18.5 ms.

The CUDA rows are what qev.py gives by default on an NVIDIA GPU: its fast backend (qev_fast/), after pip install -r requirements.txt -r requirements-cuda.txt. In a fresh environment installed from those two files alone (torch 2.14, flash-linear-attention 0.5.2, transformers 5.20; no vLLM), one L40S measured:

• 5.6 ms p50 and 19.5 ms p90 per request, 9.7 ms mean;
• about 105 requests/s one request at a time, and about 97 requests/s in packed batches (117 with vLLM's FlashAttention-2 build installed);
• 99.8% argmax agreement with our evaluation path on 1,782 dev questions.

Start-up: about 25 s until ready when the kernel cache is warm, about 2 min on the very first start (kernel compilation). The common CUDA graphs are captured at start; the rest are captured in the background after the model is ready.

The model is 0.8B parameters: 1.7 GB in bf16, 0.8 GB in 8-bit. For Apple silicon, use qev-0.8b-MLX.

Training

qev-0.8b is a full fine-tune of Qwen3.5-0.8B-Base (all text layers and the final norm; the tied embeddings are the base's). It was trained on our own decision data, in stages:

1. Base fine-tune: two epochs, about 1.8B tokens, over a decision mix of 1.28M requests (learning rate 5e-6; DiLoCo over 16 workers).
2. Anneal on a newer data release (about 142M tokens).
3. Coherence anneal: about 52M tokens, with a consistency loss on twin sets built from our own probability laws: invariance to rephrasing and reordering, complements, partitions, bounds, monotonicity and conditioning. No benchmark code or items were used.
4. A further round on newer data with coherence twins (about 104M tokens).
5. Weight soups of the trunk tensors to keep calibration while adding consistency.
6. A gap pass (150M tokens) over data for weak skills, BH-style long hard decisions, hard yes / no and grounding negatives, order-invariance twins, and replay.
7. A 50/50 weight soup of the pre-gap and post-gap models (the first release).
8. A games, rules and spatial pass (109M tokens with replay; 27M new): code-generated game-move decisions with the facts in the question or in the state, rules written in the question, and coordinate / grid relations. The labels are computed by code; the benchmark-only games and domains above are excluded, and every row was screened against the benchmark items.
9. The release: a 50/50 weight soup of the model from step 8 and the first release. The per-type temperatures were then refit on the calibration split.

Training data and licenses

The decision data comes from public datasets and data generated for the project. Each training row records its source, its license and how its label was made. Labels are of these kinds:

label kind	share of rows
the source dataset's own label	about 49%
computed exactly by a program	37%
a writer model's label confirmed by two independent solver models	7%
a model's rating or soft distribution	about 7%

By license, for the decision dataset release the training mix was drawn from (2.2M requests):

• Permissive licenses, about two thirds of the rows: CC-BY-4.0 (about 35% in total, including NVIDIA's Nemotron agentic data, reasoning-core's procedural pile, FLD and ProofWriter), CC0-1.0 / public domain (about 17%, including the CFPB complaint database and NVD), Apache-2.0 (about 8%), MIT (about 5%, including QA sets under mixed MIT / CC-BY / Apache terms) and BSD-3-Clause (2.7%). A further 1.8% is under per-source upstream terms or ODC-BY.
• Share-alike sources, about 10.5%:
  • CC-BY-SA-4.0 from MTOP (1.6%), the Schema-Guided Dialogue dataset (1.1%), SQuAD-derived QNLI (1.1%) and HotpotQA (0.2%), plus other CC-BY-SA-4.0 sources (4.3%);
  • DBpedia (CC-BY-SA-3.0, 1.1%);
  • Bitext customer-support data (CDLA-Sharing-1.0, 1.0%).
• Generated for the project, about 17%:
  • computed by programs (6.3%);
  • written by an LLM and checked by two blind solver models (8.7%);
  • other generated rows released under CC0 (2.0%).
• Model-made labels: a further 1.2% carries ratings made by a language model, under MIT.
• Research-use releases: about 0.7% comes from the LeWiDi research release, whose component datasets carry their own terms, including research-use-only terms for one of them.

Later training stages added program-generated and writer/verifier-generated rows of the same kinds. The model is released under Apache-2.0. Whether share-alike or research-use terms on a part of the training data affect a given use of the weights is for the user to assess.

Limitations

• Ordinal scores are the weakest type. Treat score outputs as soft signals.
• Long and hard inputs. Accuracy drops on long states (several thousand tokens) and on deliberately hard items. The model reads the request in one pass and does no reasoning.
• Coherence is improved, not guaranteed. Complements and marginalisation across related questions, and unit restatements such as "90 min" vs "1.5 h", are still often violated. Use per_question for multi-question requests, and do not assume that answers to related questions are mutually consistent.
• Abstention. The model rarely chooses "cannot answer" or "no tool call" options, even when it should. Give it an explicit option and check its probability rather than relying on the argmax.
• Calibration is fitted, not universal. On data far from the calibration split, probabilities can be over- or under-confident. Recalibrate on your own labelled data when probabilities drive decisions.
• Language and domain. The data is mostly English business, operations and agent-log decisions. Other languages and domains have not been evaluated.
• Options. A question can have at most 255 options. Option names matter, because the model reads them.
• Images are zero-shot. The vision tower and projector are the base model's and were not trained, and qev saw no images in training. Boards are read close to chance (connect-four 0.38, chess 0.60, tic-tac-toe 0.59), counting real objects is weak (CLEVR 0.40), reading a value off a chart is 0.63, and real-photo yes/no questions are 0.71–0.87. JevVision's rendered images are clean; screenshots and documents from the wild are untested. Image requests run on the fast CUDA engine (qev.py), the transformers path (qev_image.py), and the MLX and GGUF builds. Image speed on a Mac has not been measured.
• Inherited behaviour. The model inherits biases of the base model and of its data, part of which was labelled by larger models. Do not use it as the only decision maker for consequential decisions about people (medical, legal, credit, employment).

Files

file	what it is
model.safetensors	the weights (bf16; A_log in fp32; the base's unused multi-token-prediction head dropped)
config.json, tokenizer.json, tokenizer_config.json, vocab.json, merges.txt	the Qwen3.5 config and tokenizer
qev_config.json	temperatures, the 255 option-letter tokens, the prompt and readout description
qev.py, requirements.txt	the runner (backends: fast on NVIDIA GPUs, transformers elsewhere)
qev_server.py	an HTTP server with the System One API (POST /v1/systemone; "images" on the fast engine)
qev_fast/, requirements-cuda.txt	the fast CUDA engine qev.py uses by default on NVIDIA GPUs (text and images)
qev_image.py	the image runner (QEVImage: <image:NAME> in the state; transformers)
qev_image_server.py	the System One HTTP server with an "images" field
preprocessor_config.json, video_preprocessor_config.json	the base model's image processor configs, unchanged
figures/	the header, charts and demo video used in this card
SHA256SUMS	checksums
.gitattributes	Git LFS settings

The checkpoint keeps the base's multimodal config and its vision tower and projector, unchanged (bit-identical to Qwen3.5-0.8B-Base's). qev.py's fast engine and qev_image.py use them for images; text requests do not.

Tokenizer fix in this revision. The model was trained with the Qwen2 pre-tokenizer pattern, which splits words on letters only. The earlier tokenizer.json carried the newer Qwen3.5 pattern, which also groups combining marks. That split words in scripts with combining marks, such as Hindi, Telugu and other Indic scripts, differently from training, and gave noticeably different probabilities on such text. This revision's tokenizer.json uses the training pattern. qev.py also sets it explicitly, because transformers' Qwen tokenizer classes impose their own pattern whatever tokenizer.json says. English and most other text is unaffected.

Provenance

The base is Qwen/Qwen3.5-0.8B-Base at revision dc7cdfe2ee4154fa7e30f5b51ca41bfa40174e68. qev-0.8b is the release named "A1" (checkpoint SE50) in our internal reports.

License

Apache-2.0, as is the base model, Qwen3.5-0.8B-Base. See "Training data and licenses" for the data.