Model card: the full qev-0.8b card (demo, benchmarks, evaluation, training, data, limitations) with the MLX parts

README.md

@@ -9,26 +9,187 @@ tags:
   - decision-making
   - classification
   - calibration
+  - routing
+  - guardrails
+  - prompt-injection
+  - agents
+  - games
+  - on-device
   - system-one
   - jev
   - mlx
 ---
 
-# qev-0.8b-MLX
+<p align="center"><img src="https://huggingface.co/labqoat/qev-0.8b/resolve/main/figures/header.jpg" alt="qev-0.8b: the tiny local decision model, by Labqoat" width="100%"></p>
+
+<p align="center">
+  <a href="https://app.labqoat.com/e/qev/qev/"><b>▶ Watch it play, live</b></a> ·
+  <a href="https://huggingface.co/labqoat/qev-0.8b">PyTorch + CUDA (main repo)</a> ·
+  <a href="https://huggingface.co/labqoat/qev-0.8b-GGUF">GGUF (llama.cpp)</a> ·
+  <a href="#quickstart">Quickstart</a> ·
+  <a href="#benchmarks-27-decision-models-around-1b">Benchmarks</a>
+</p>
+
+# qev-0.8b-MLX: the tiny local decision model, for Apple silicon
 
 [qev-0.8b](https://huggingface.co/labqoat/qev-0.8b) for Apple silicon with [MLX](https://github.com/ml-explore/mlx).
-qev is a 0.8B System One decision model: it answers typed questions about a piece of state with calibrated
-probabilities, in one forward pass. See the main repo for the model card, the evaluation and the training data.
 
 | weights | folder | size | argmax agreement with the CUDA reference¹ |
 |---|---|---|---|
 | bf16 | repo root | 1.4 GB | 99.3% |
 | 8-bit (affine, group size 64) | `q8/` | 0.78 GB | 99.0% |
 
-¹ Measured on the first release's weights; the conversion and runner are unchanged.
+¹ Measured on the first release's weights; the conversion and runner are unchanged. A 4-bit build is not provided:
+it changes about 6% of the argmax answers, which is too much for calibrated probabilities.
+
+qev reads anything (a game board, a post, an inbox, a web page, a log) and answers any question about it with
+**calibrated probabilities in a single forward pass**. Out of 27 decision models of its size it scores highest on
+Benchmark Heaven, it answers faster than all of them, and in 8-bit it needs 1.3 GB of RAM on a Mac.
+
+<table>
+<tr>
+<td width="38%" align="center">
+<video src="https://huggingface.co/labqoat/qev-0.8b/resolve/main/figures/qev-demo-mobile.mp4" poster="https://huggingface.co/labqoat/qev-0.8b/resolve/main/figures/qev-demo-mobile-poster.jpg" autoplay loop muted playsinline controls width="100%"></video>
+</td>
+<td>
+
+**What you're watching.** These are screen recordings of the [live demo](https://app.labqoat.com/e/qev/qev/), with nothing staged. One copy of qev-0.8b
+on one GPU runs all of these at once, in real time:
+
+- 🐍 **Snake**, 🧱 **Tetris**, 👻 **Pac-Man**, 🔢 **2048**, 🏓 **Breakout** and 🔨 **Whac-A-Mole**. Tetris, Pac-Man,
+  2048, Breakout and Whac-A-Mole were held out of training.
+- 🧹 **LinkedIn slop** screening: bait, humblebrag, made-up story, buzzwords and substance, five questions per post.
+- 🛡️ A **prompt-injection firewall** for an agent's inputs.
+- 🧭 An agent **tool router**, 🎧 **support triage**, ⭐ **review ratings** and 📥 **inbox sorting**: six emails and 18
+  questions in one request.
+
+For the games, the game code writes down what each move would do ("it hits the wall and dies", "clears 2 lines"),
+and qev answers one yes/no question per move. The most likely "yes" is played. qev does no search and no lookahead;
+every move is one forward pass of about 4 ms.
+
+</td>
+</tr>
+</table>
+
+## At a glance
+
+| | qev-0.8b |
+|---|---|
+| parameters | 0.8B (full fine-tune of Qwen3.5-0.8B-Base) |
+| Benchmark Heaven | **77.1%**: first of 27 decision models around 1B |
+| latency | **3.5 ms** p50 per request on 1× L40S: fastest of 27 · 109 ms on an M4 Pro Mac (MLX) |
+| memory | 1.7 GB weights in bf16, 0.8 GB in 8-bit · **1.3 GB RAM** on a Mac in 8-bit |
+| also first on | JevExtra (72.0%, agent and safety skills), JevSpatial (74.2%, games and space), Rules-7 (63.7%, tied) |
+| output | calibrated probabilities for `choice`, `noul` (yes / no) and `score` questions, up to 255 options each |
+| formats | MLX bf16 and 8-bit for Apple silicon (this repo), [PyTorch + fast CUDA engine](https://huggingface.co/labqoat/qev-0.8b), [GGUF](https://huggingface.co/labqoat/qev-0.8b-GGUF) (also stock `llama-server` at `/v1/systemone`) |
+| license | Apache-2.0 |
+
+qev does not generate text. Each answer is a probability distribution, so you can act when it is confident and
+escalate to a person or a larger model when it is not. Nothing has to leave your machine.
+
+## Benchmarks: 27 decision models around 1B
+
+We ran qev-0.8b against 26 other decision models around 1B parameters. Every model got the same items and scoring,
+ran on one NVIDIA L40S, and used its own published inference recipe.
+
+<p align="center"><img src="https://huggingface.co/labqoat/qev-0.8b/resolve/main/figures/ranks.png" alt="Most accurate, fastest, and smallest of the most accurate: qev-0.8b leads all three" width="100%"></p>
+
+Accuracies are in %. ECE, latency and memory are lower-is-better. **Bold** marks the best value in each column.
+
+| model | params | VRAM (GB, est.) | p50 ms | BH | BH ECE ↓ | JevBench | coherence | suite | JevExtra | JevSpatial | Rules-7 |
+|---|---|---|---|---|---|---|---|---|---|---|---|
+| **qev-0.8b** | 0.8B | 2.2 | **3.5** | **77.1** | 0.063 | 85.1 | 71.2 | 61.7 | **72.0** | **74.2** | **63.7** |
+| decision-2b (flymy-ai) | 2B | 5.6 | 81.3 | 76.2 | **0.042** | **86.0** | 61.8 | 60.9 | 64.1 | 69.5 | 58.1 |
+| decider-2b | 1.9B | 4.3 | 7.2 | 75.8 | 0.065 | 85.9 | 72.5 | **70.7** | 58.8 | – | – |
+| EXAONE-4.0-1.2B-JEV | 1.2B | 3.1 | 27.8 | 68.8 | 0.097 | 83.9 | 67.9 | 67.3 | 64.0 | 70.0 | 58.8 |
+| malkuth-2b | 2B | 4.3 | 46.2 | 68.8 | 0.131 | 84.8 | 70.8 | 66.6 | 63.6 | 69.9 | **63.7** |
+| classone (Gemma-4-E2B) | ~5B | 10.7 | 56.5 | 66.7 | 0.260 | 82.1 | 66.6 | 48.2 | 52.5 | 63.4 | 48.1 |
+| jul-2b | 2B | 5.5 | 88.9 | 65.8 | 0.059 | 83.3 | 68.4 | 57.8 | 56.4 | 66.8 | 58.0 |
+| Open-Jev-2B | 1.9B | 4.3 | 191.7 | 64.5 | 0.127 | 84.9 | 76.2 | 59.9 | 57.6 | 72.2 | 59.7 |
+| decision-fast | 0.6B | 1.7 | 58.9 | 63.6 | 0.106 | 73.0 | 54.3 | 52.0 | 52.0 | 48.5 | 47.1 |
+| Tacet Sonata¹ | 144M | 0.8 | 20.5 | 62.3 | 0.148 | 64.9 | 45.5 | 31.6 | 36.1 | 50.6 | 35.4 |
+
+<details>
+<summary>The other 17 models</summary>
+
+| model | params | VRAM (GB, est.) | p50 ms | BH | BH ECE ↓ | JevBench | coherence | suite | JevExtra | JevSpatial | Rules-7 |
+|---|---|---|---|---|---|---|---|---|---|---|---|
+| smalljev-2.5b¹ | 2.5B | 5.6 | 68.8 | 61.5 | 0.142 | 71.2 | 61.1 | 45.6 | 46.2 | 53.7 | 49.3 |
+| jevos-v5 (feder-cr), partial | ~0.9B | 1.2 | – | 61.0 | – | 84.1 | – | 49.9 | – | – | – |
+| Quyet-1.0-Small¹ | 328M | 1.8 | 19.3 | 59.7 | 0.066 | 71.3 | 56.8 | 43.8 | 42.4 | 48.4 | 45.4 |
+| watt-flash-0.1 | 140M | 0.8 | 15.7 | 59.7 | 0.073 | 67.1 | 61.9 | 48.8 | 39.1 | 51.3 | 36.3 |
+| Ghafiri Qwen3.5-0.8B | 0.8B | 2.0 | 44.6 | 59.3 | 0.083 | 78.1 | 61.7 | 41.6 | 41.2 | 62.1 | 40.1 |
+| WaterSheep | ~150M | 1.1 | 18.9 | 59.3 | 0.127 | 73.5 | 57.9 | 56.4 | 50.8 | 57.9 | 41.0 |
+| lev-350m | 361M | 1.2 | 24.1 | 58.4 | 0.094 | 66.9 | 65.5 | 47.5 | 44.5 | – | – |
+| deem-0.8b¹ | 0.8B | 2.0 | 54.6 | 58.0 | 0.240 | 75.0 | 59.3 | 42.9 | 48.3 | 51.1 | 46.8 |
+| Laya | 421M | 1.3 | 24.4 | 58.0 | 0.095 | 70.7 | 56.8 | 53.7 | 43.0 | 47.8 | 44.1 |
+| openjev-verdict¹ | 151M | 1.1 | 13.5 | 57.6 | 0.081 | 63.9 | 59.7 | 32.8 | 37.9 | 38.4 | 34.7 |
+| Quyet-1.0-Tiny¹ | 183M | 1.2 | 17.5 | 57.1 | 0.097 | 67.7 | 54.9 | 42.4 | 46.9 | 52.6 | 47.1 |
+| Qwen3-1.7B (raw)¹ | 1.7B | 4.6 | 27.8 | 55.8 | 0.411 | 64.5 | 70.7 | 39.8 | 48.7 | 52.4 | 41.3 |
+| SimpleJev (Qwen3.5-0.8B) | 0.8B | 2.2 | 82.9 | 55.4 | 0.183 | 55.6 | 46.6 | 47.6 | 45.6 | 54.1 | 36.1 |
+| Open-Jev-DeBERTa¹ | 435M | 2.2 | 34.4 | 55.0 | 0.091 | 79.0 | 53.3 | 38.4 | 41.5 | 54.2 | 42.0 |
+| verdict-small | 118M | 1.0 | 8.6 | 53.7 | 0.115 | 50.7 | 70.1 | 42.6 | 38.5 | 52.8 | 35.3 |
+| ModernBERT-zeroshot | 395M | 1.3 | 41.0 | 52.4 | 0.199 | 70.7 | 66.4 | 44.1 | 42.7 | 52.4 | 37.5 |
+| Qwen3-0.6B (raw)¹ | 0.6B | 2.0 | 26.3 | 49.8 | 0.434 | 53.8 | 76.8 | 31.7 | 45.6 | 37.9 | 35.7 |
+
+</details>
+
+- **BH** = Benchmark Heaven, public split (231 items, sampling error about ±2.8 points). **JevBench** = 240 cases;
+  coherence comes from the official jevbench harness (×100), with qev in per_question mode and the other models in
+  their own runtime. **Suite** = 22 public-dataset tasks (880 questions). **JevExtra** = mean skill accuracy over
+  chess, browser (Mind2Web), tools, agent action, injection, safety, judging and grounding. **JevSpatial** and
+  **Rules-7** = the jev-skills `spatial` and `rules7` configs.
+- **Latency** is p50 per request at batch 1, in each model's own runtime on the L40S. **VRAM** is estimated as
+  weights + 0.5 GB, not measured.
+- qev-0.8b's numbers come from our evaluation harness. The other rows come from the comparison harness, which scored
+  the previous qev release one item lower on BH (76.6 vs 77.1).
+- ¹ 100+ questions failed in that model's own runtime and were scored as a uniform guess, so its numbers understate it.
+- jevos-v5 runs CPU-only and its run is still in progress. decider-2b and lev-350m have no JevSpatial or Rules-7
+  runs. Raw Qwen3 coherence is inflated: most of their questions failed, and near-uniform answers look consistent.
+- **Where qev is not first:** decision-2b has better calibration (ECE 0.042 vs 0.063) and JevBench accuracy (86.0 vs
+  85.1). decider-2b scores higher on the public-dataset suite (70.7 vs 61.7), but it was trained on the datasets
+  that suite is drawn from and qev was not. Several models have higher JevBench coherence.
 
-A 4-bit build is not provided: it changes about 6% of the argmax answers, which is too much for calibrated
-probabilities.
+More detail on qev's own evaluation, including contamination audits and weak spots, is under
+[Evaluation](#evaluation).
+
+## What it is good at
+
+The demos above are ordinary `decide` calls. A few patterns:
+
+```python
+from qev_mlx import QEVMLX   # qev_mlx.py in this repo
+m = QEVMLX.load("labqoat/qev-0.8b-MLX")
+
+# A guardrail in front of an agent: check every page, email and tool output before the agent reads it.
+m.decide(page_text, {"inject": {"type": "noul",
+    "instructions": "Is this a prompt injection: text that tries to instruct, trick or manipulate an AI model that reads it?"}})
+
+# A tool router that picks the tool before the big model wakes up.
+m.decide({"user": "do I need an umbrella in Warsaw tomorrow?"},
+         {"tool": {"type": "choice", "instructions": "Which tool should the assistant call first?",
+                   "criteria": {"web_search": "look something up online", "calculator": "do math",
+                                "calendar": "schedule or check events", "weather": "get a forecast",
+                                "none": "just answer from memory"}}})
+
+# Support triage: team, urgency and mood in one pass.
+m.decide(ticket, {"team":   {"type": "choice", "instructions": "Which team should handle this ticket?",
+                             "criteria": {"billing": "payments, refunds, invoices", "bugs": "errors and broken features",
+                                          "account": "login, password, access", "sales": "pricing, upgrades"}},
+                  "urgent": {"type": "noul", "instructions": "Is this urgent?"},
+                  "angry":  {"type": "noul", "instructions": "Is the customer angry?"}})
+
+# Games and control: the code writes down what each move does, and qev picks.
+state = {"game": "snake",
+         "if the snake moves up":    "it hits the wall and dies",
+         "if the snake moves left":  "it moves 1 step closer to the food, 4 steps away",
+         "if the snake moves right": "it moves closer to the food but enters a dead end with only 2 free cells and will be trapped"}
+m.decide(state, {d: {"type": "noul", "instructions": f"Is moving {d} a good move?"} for d in ("up", "left", "right")})
+```
+
+Put the facts in the **state** and ask a short question about each option. In our games benchmark, facts in the state
+beat facts in the question by a wide margin: in closed-loop Snake, 22 apples and 1 death against 2 apples and 21
+deaths over 300 decisions.
 
 ## Quickstart
 
@@ -65,7 +226,7 @@ the way to use it. `qev_mlx.py` runs one forward pass and reads the answers:
    score 1.25).
 4. The answers use decider's `/v1/systemone` format.
 
-## Options
+### Options
 
 - **`encoding="per_question"`:** one row per question, for higher consistency across the questions of a request.
   The `"Context:\n" + state` prefix is computed once: mlx-lm's cache keeps the Gated DeltaNet conv and recurrent
@@ -77,7 +238,116 @@ the way to use it. `qev_mlx.py` runs one forward pass and reads the answers:
 - **`compile=True`** (the default): `mx.compile` of the joint forward per 64-token length bucket. The median is about
   13% lower; each new bucket compiles once, in about 0.4 s.
 
-## Speed (M4 Pro, 24 GB, bf16)
+## Questions and answers
+
+A request has a **state** (a string, an object or an array) and one or more **questions**, each of one type:
+
+| type | what it asks | answer fields |
+|---|---|---|
+| `choice` | pick one of the named options (`criteria`: `{name: description or null}`) | `choice`, `confidence`, `x_p_max`, `certainty`, `probabilities` |
+| `noul` | a yes / no question (`instructions`, optional `criteria` `{"true": ..., "false": ...}`) | `noul`: the probability of true |
+| `score` | a rating on an ordered scale (`criteria`: list of levels, low to high) | `score` (expected level index), `confidence`, `x_p_max`, `certainty`, `legend`, `probabilities` |
+
+The fields follow decider's `format_answer`, the `/v1/systemone` wire format:
+
+- `confidence` is TypeSafe's confidence. For a choice, it is the top probability rescaled so that uniform = 0. For a
+  score, it is 1 minus the expected distance from the top level, relative to a uniform distribution.
+- `x_p_max` is the top probability.
+- `certainty` is 1 minus the normalised entropy.
+
+A string state is used as it is. An object or array is rendered as JSON with array indices annotated, as decider's
+server does. A question can have up to 255 options.
+
+### How a request is answered
+
+1. The request becomes a prompt row in decider's state-first System One layout: `Context:` and the state, then each
+   question with its lettered options, ending in an `Answer: (` slot.
+2. One forward pass reads the logits of the option letter tokens at each answer slot. Only the letters' rows of the
+   tied embedding are used.
+3. Each question's answer is the softmax over its own options, sharpened per question type with temperatures fitted
+   on a held-out calibration split: **choice 1.7, noul 0.9, score 1.25** (`p -> p^(1/t)`, renormalised). The
+   temperatures and the letter tokens are in `qev_config.json`.
+
+### Encodings
+
+- **joint** (default): one row per request, with every question behind one copy of the state. This is the cheapest
+  encoding.
+- **per_question** (`per_question=True`): one row per question, each with its own copy of the state. Each answer
+  then does not depend on which other questions were asked, or in what order. This raises JevBench coherence from
+  64.5 to 71.2 at about the same accuracy. It costs one row per question in `qev.py`. The MLX runner in
+  [qev-0.8b-MLX](https://huggingface.co/labqoat/qev-0.8b-MLX) computes the shared state once, so there it costs about
+  the same as joint.
+
+The two encodings give the same answers for single-question requests. For multi-question requests where consistency
+across questions matters, use per_question.
+
+**In this repo**, choose the encoding with `QEVMLX.load(..., encoding="per_question")`. The shared
+state is computed once, so per_question costs about the same as joint here.
+
+## Evaluation
+
+All numbers come from one harness, with the same items and scoring for every model. The temperatures were fitted on
+our own calibration split. No benchmark data was used for training, tuning or calibration.
+
+- **Benchmark Heaven (BH):** the public split, 231 items. The sampling error of BH accuracy is about ±2.8 points.
+- **JevBench:** 240 cases, 1,938 gold questions. Coherence is scored by the official `jevbench` harness, ×100.
+- **Jev suite:** 22 public-dataset tasks, 880 questions.
+- **Pooled:** the mean of BH, JevBench and suite accuracy.
+- **JevExtra:** our own skills benchmark: chess, browser, tools, agent action, prompt injection, safety, judging and
+  grounding. The headline is the mean skill accuracy.
+  - qev's indirect prompt-injection items share text with training rows (16-gram audit). The table includes them,
+    like every other item.
+- **JevSpatial:** our games and spatial benchmark, 2,040 items: game moves with the facts in the question and in the
+  state, rules in the question, coordinates and small grids. Breakout, Whac-A-Mole, Knife Hit, 2048, Tetris and Pac-Man
+  are never used in training.
+- **JevosTasks:** seven policy and rule tasks built like the ones other System One models report (admission, rental,
+  agent rules, authority, fraud points; ShARC dev; patent phrase similarity test), 3,440 items. The five generated
+  domains are never used in training.
+
+Where two values are given, they are **joint / per_question**. The two are equal on BH and the suite, which have one
+question per request.
+
+| benchmark | metric | **qev-0.8b** (joint / per_question) | Qwen3.5-0.8B-Base (zero-shot) | decider-0.8b |
+|---|---|---|---|---|
+| Benchmark Heaven | accuracy | **0.771** | 0.528 | 0.658 |
+| Benchmark Heaven | Brier (lower is better) | **0.337** | 0.569 | 0.430 |
+| Benchmark Heaven | ECE (lower is better) | 0.063 | 0.089 | **0.061** |
+| JevBench | accuracy | **0.851** / 0.849 | 0.439 | 0.822 |
+| JevBench | coherence | 64.5 / **71.2** | 51.6 | 69.6 |
+| Jev suite | accuracy | 0.617 | 0.392 | **0.682** |
+| pooled (BH, JevBench, suite) | accuracy | **0.746** / 0.745 | 0.453 | 0.721 |
+| JevExtra | mean skill accuracy | **0.720** | 0.442 | 0.581 |
+| JevSpatial | accuracy | **0.742** | | |
+| JevosTasks | accuracy | **0.637** | | |
+
+**Contamination audit.** None of the training data shares any 16-token sequence with a BH or JevBench item. In the
+Jev suite, 2 of 22,773 test-pool items reach 50% 16-gram coverage. The training data was also screened by meaning
+against every test item, using embeddings with a threshold set from near copies.
+
+**Where it is weakest:**
+
+- JevBench `score` questions (4-level scales): accuracy 0.39. Most errors are off by one level, and predictions lean
+  toward the middle levels.
+- Long and hard BH items: accuracy 0.51 on long items and 0.57 on the hard tier.
+- The suite's wide-label and graded tasks: go_emotions, helpsteer2, stsb.
+- Abstaining in agent decisions, when no tool fits or information is missing.
+
+**Runtime fidelity.**
+
+- On the 902-request dev set used for release checks (1,782 questions), `qev.py` on an NVIDIA L40S agrees with our
+  CUDA reference path on 99.7% of argmax answers, with mean absolute probability difference 0.003.
+- On Apple MPS (bf16) and on CPU (fp32), over the first 50 requests (103 questions), the agreement is 99.0%: one
+  near-tie question flips. The mean difference is 0.003.
+- The MLX and GGUF builds agree with the reference on 99.0–99.8% of argmax answers (see their repos).
+- These agreement figures were measured on the first release's weights; the architecture, code and quantisation are
+  unchanged.
+
+**This repo.** `qev_mlx.py` agrees with the CUDA reference on 99.3% (bf16) and 99.0% (8-bit) of argmax
+answers. per_question with the shared state prefix agrees with running each row on its own on 99.7%.
+
+## Speed
+
+### qev_mlx.py (M4 Pro, 24 GB, bf16)
 
 Steady-state batch-1 latency over 200 dev requests (mean 746 tokens, median 417).
 
@@ -90,6 +360,133 @@ Steady-state batch-1 latency over 200 dev requests (mean 746 tokens, median 417)
 Memory is 2.0 GB RSS and 2.6 GB Metal for bf16 (1.3 / 2.0 GB for 8-bit), plus about 25 MB per cached state. Prefill
 is compute-bound on the Mac GPU, so 8-bit saves memory, not time.
 
+### The CUDA engine in the main repo, for comparison
+
+Measured on our dev requests: mean 746 tokens, median 417, about 2 typed questions each. These are longer
+than the benchmark items behind the 3.5 ms p50 in the 27-model comparison above, hence the higher figure here. CUDA: one NVIDIA L40S with
+our optimised engine (packed variable-length batches, the flash-linear-attention Gated DeltaNet kernels, CUDA graphs
+for small batches). Apple: an M4 Pro (24 GB) with MLX, bf16, compiled.
+
+| setting | hardware | p50 latency | throughput |
+|---|---|---|---|
+| single request, CUDA graphs | 1x L40S | 5.7 ms | ~100 req/s, ~200 decisions/s |
+| packed batch (32k tokens, ~43 requests) | 1x L40S | ~370 ms | ~117 req/s, ~230 decisions/s |
+| single request, MLX bf16 | Apple M4 Pro | 109 ms | ~6 req/s, ~11 decisions/s |
+
+`per_question` (shared prefix): 7.9 ms p50 on the L40S. MLX prefix-cache hit: 18.5 ms.
+
+The CUDA rows are what `qev.py` gives by default on an NVIDIA GPU: its fast backend (`qev_fast/`), after
+`pip install -r requirements.txt -r requirements-cuda.txt`. In a fresh environment installed from those two files
+alone (torch 2.14, flash-linear-attention 0.5.2, transformers 5.20; no vLLM), one L40S measured:
+
+- 5.6 ms p50 and 19.5 ms p90 per request, 9.7 ms mean;
+- about 105 requests/s one request at a time, and about 97 requests/s in packed batches (117 with vLLM's
+  FlashAttention-2 build installed);
+- 99.8% argmax agreement with our evaluation path on 1,782 dev questions.
+
+Start-up: about 25 s until ready when the kernel cache is warm, about 2 min on the very first start (kernel
+compilation). The common CUDA graphs are captured at start; the rest are captured in the background after the
+model is ready.
+
+The model is 0.8B parameters: 1.7 GB in bf16, 0.8 GB in 8-bit. For Apple silicon, use
+[qev-0.8b-MLX](https://huggingface.co/labqoat/qev-0.8b-MLX).
+
+## Training
+
+qev-0.8b is a full fine-tune of Qwen3.5-0.8B-Base (all text layers and the final norm; the tied embeddings are the
+base's). It was trained on our own decision data, in stages:
+
+1. **Base fine-tune:** two epochs, about 1.8B tokens, over a decision mix of 1.28M requests (learning rate 5e-6;
+   DiLoCo over 16 workers).
+2. **Anneal** on a newer data release (about 142M tokens).
+3. **Coherence anneal:** about 52M tokens, with a consistency loss on twin sets built from our own probability laws:
+   invariance to rephrasing and reordering, complements, partitions, bounds, monotonicity and conditioning. No
+   benchmark code or items were used.
+4. A further round on newer data with coherence twins (about 104M tokens).
+5. **Weight soups** of the trunk tensors to keep calibration while adding consistency.
+6. **A gap pass** (150M tokens) over data for weak skills, BH-style long hard decisions, hard yes / no and grounding
+   negatives, order-invariance twins, and replay.
+7. A 50/50 weight soup of the pre-gap and post-gap models (the first release).
+8. **A games, rules and spatial pass** (109M tokens with replay; 27M new): code-generated game-move decisions with the
+   facts in the question or in the state, rules written in the question, and coordinate / grid relations. The labels
+   are computed by code; the benchmark-only games and domains above are excluded, and every row was screened against
+   the benchmark items.
+9. **The release:** a 50/50 weight soup of the model from step 8 and the first release. The per-type temperatures
+   were then refit on the calibration split.
+
+## Training data and licenses
+
+The decision data comes from public datasets and data generated for the project. Each training row records its
+source, its license and how its label was made. Labels are of these kinds:
+
+| label kind | share of rows |
+|---|---|
+| the source dataset's own label | about 49% |
+| computed exactly by a program | 37% |
+| a writer model's label confirmed by two independent solver models | 7% |
+| a model's rating or soft distribution | about 7% |
+
+By license, for the decision dataset release the training mix was drawn from (2.2M requests):
+
+- **Permissive licenses**, about two thirds of the rows: CC-BY-4.0 (about 35% in total, including NVIDIA's Nemotron
+  agentic data, reasoning-core's procedural pile, FLD and ProofWriter), CC0-1.0 / public domain (about 17%, including
+  the CFPB complaint database and NVD), Apache-2.0 (about 8%), MIT (about 5%, including QA sets under mixed MIT /
+  CC-BY / Apache terms) and BSD-3-Clause (2.7%). A further 1.8% is under per-source upstream terms or ODC-BY.
+- **Share-alike sources**, about 10.5%:
+  - CC-BY-SA-4.0 from MTOP (1.6%), the Schema-Guided Dialogue dataset (1.1%), SQuAD-derived QNLI (1.1%) and HotpotQA
+    (0.2%), plus other CC-BY-SA-4.0 sources (4.3%);
+  - DBpedia (CC-BY-SA-3.0, 1.1%);
+  - Bitext customer-support data (CDLA-Sharing-1.0, 1.0%).
+- **Generated for the project**, about 17%:
+  - computed by programs (6.3%);
+  - written by an LLM and checked by two blind solver models (8.7%);
+  - other generated rows released under CC0 (2.0%).
+- **Model-made labels:** a further 1.2% carries ratings made by a language model, under MIT.
+- **Research-use releases:** about 0.7% comes from the LeWiDi research release, whose component datasets carry their
+  own terms, including research-use-only terms for one of them.
+
+Later training stages added program-generated and writer/verifier-generated rows of the same kinds. The model is
+released under Apache-2.0. Whether share-alike or research-use terms on a part of the training data affect a given
+use of the weights is for the user to assess.
+
+## Limitations
+
+- **Ordinal scores are the weakest type.** Treat `score` outputs as soft signals.
+- **Long and hard inputs.** Accuracy drops on long states (several thousand tokens) and on deliberately hard items.
+  The model reads the request in one pass and does no reasoning.
+- **Coherence is improved, not guaranteed.** Complements and marginalisation across related questions, and unit
+  restatements such as "90 min" vs "1.5 h", are still often violated. Use per_question for multi-question requests,
+  and do not assume that answers to related questions are mutually consistent.
+- **Abstention.** The model rarely chooses "cannot answer" or "no tool call" options, even when it should. Give it an
+  explicit option and check its probability rather than relying on the argmax.
+- **Calibration is fitted, not universal.** On data far from the calibration split, probabilities can be over- or
+  under-confident. Recalibrate on your own labelled data when probabilities drive decisions.
+- **Language and domain.** The data is mostly English business, operations and agent-log decisions. Other languages
+  and domains have not been evaluated.
+- **Options.** A question can have at most 255 options. Option names matter, because the model reads them.
+- **Inherited behaviour.** The model inherits biases of the base model and of its data, part of which was labelled
+  by larger models. Do not use it as the only decision maker for consequential decisions about people (medical,
+  legal, credit, employment).
+
+## Files
+
+| file | what it is |
+|---|---|
+| `model.safetensors`, `config.json` | the weights in bf16 and mlx-lm's Qwen3.5 config |
+| `q8/` | the same model with 8-bit affine weights (group size 64) |
+| `tokenizer.json`, `tokenizer_config.json`, `chat_template.jinja` | the tokenizer |
+| `qev_config.json` | temperatures, the 255 option-letter tokens, the prompt and readout description |
+| `qev_mlx.py` | the MLX runner |
+| `SHA256SUMS` | checksums |
+
+## Provenance
+
+The base is `Qwen/Qwen3.5-0.8B-Base` at revision `dc7cdfe2ee4154fa7e30f5b51ca41bfa40174e68`. qev-0.8b is the release
+named "A1" (checkpoint SE50) in our internal reports.
+
+The MLX weights are qev-0.8b's weights in mlx-lm's Qwen3.5 layout (`mlx_lm.utils.load_model` loads
+them); `q8/` holds the same model with 8-bit affine weights, group size 64.
+
 ## License
 
-Apache-2.0. See [qev-0.8b](https://huggingface.co/labqoat/qev-0.8b) for the training data and its licenses.
+Apache-2.0, as is the base model, Qwen3.5-0.8B-Base. See "Training data and licenses" for the data.

SHA256SUMS

@@ -1,4 +1,4 @@
-2df4901181a2a2bd9c0642d15c19d7d8ecfcb26941e5dfbc799a60456968278e  README.md
+7d655921ca9ef6e324c6c38d4915f71925c24307687e3dcd988478021be14b77  README.md
 273d8e0e683b885071fb17e08d71e5f2a5ddfb5309756181681de4f5a1822d80  chat_template.jinja
 711035379ef578e900004acaf8de9130f12189438ee721dd935dcb3080b077af  config.json
 aa600fc9b70a8830ef6c05da7b3228c1b8e73fa3c4810a707eb8df304817ebc1  model.safetensors