Release Qev-9B v0.3.0 with 4K web-action and rule-reasoning training

README.md

@@ -29,7 +29,9 @@ metrics:
 
 [Source and documentation](https://github.com/QiqianFu/Qev) · [中文说明](https://github.com/QiqianFu/Qev/blob/main/README.zh-CN.md) · [Model weights](https://huggingface.co/AustinFu/Qev-9B)
 
-Qev-9B v0.2.0 combines general decision training with HelpSteer3 Principle judgments and 600 synthetic boundary questions in the main set. Additional alignment and document-rule examples are repeated three times during the second half of training. It uses the full candidate-interaction architecture shown below.
+Qev-9B v0.3.0 combines general decisions, Principle judgments and web-action training with 4K context. Alignment, document-rule, boundary and additional rule/reasoning examples are repeated three times during the second half of training. It uses the full candidate-interaction architecture shown below.
+
+This checkpoint is also the teacher used to train Qev-4B v0.1.0. The previous 9B weights remain available at [v0.2.0](https://huggingface.co/AustinFu/Qev-9B/tree/v0.2.0), and the original Qev-2B teacher at [v0.1.0](https://huggingface.co/AustinFu/Qev-9B/tree/v0.1.0).
 
 ## Quick start
 
@@ -45,7 +47,7 @@ python -m pip install -e .
 from qev import Qev
 
 model = Qev.from_pretrained(
-    "AustinFu/Qev-9B", revision="main", device="cuda"
+    "AustinFu/Qev-9B", revision="v0.3.0", device="cuda"
 )
 answers = model.predict({
     "state": "I was charged twice. Please help immediately.",
@@ -67,7 +69,7 @@ For JSONL inference:
 
 ```bash
 python -m qev.predict \
-  --checkpoint AustinFu/Qev-9B@main \
+  --checkpoint AustinFu/Qev-9B@v0.3.0 \
   --input examples/requests.jsonl --out runs/predictions.jsonl \
   --device cuda --weights-dtype checkpoint
 ```
@@ -75,7 +77,7 @@ python -m qev.predict \
 ## Architecture and training
 
 <p align="center">
-  <img src="assets/architecture.svg" alt="Qev encodes context, questions and answer options, then scores the options with its decision head." width="100%">
+  <a href="assets/method.pdf"><img src="assets/method.svg" alt="Qev branch encoding, final-layer candidate interaction and set decision head." width="100%"></a>
 </p>
 
 | Field | Released model |
@@ -86,34 +88,38 @@ python -m qev.predict \
 | Backbone interaction | `last-full-attention` |
 | Computation | BF16 backbone; FP32 decision head and key reductions |
 | Stored adaptation tensors | FP32 |
-| Model release | v0.2.0 |
-| Main training partition | 39,605 records, including 4,459 Principle judgments and 600 synthetic boundary questions |
-| Late partition | 1,419 alignment records + 364 rule-compliance judgments |
+| Model release | v0.3.0 |
+| Input limits | State 4,096; question 512; candidate 256; complete path 4,096 tokens |
+| Main training partition | 38,198 records, including 2,230 Principle judgments and 1,422 web-action records |
+| Late partition | 2,883 records: 1,419 alignment, 364 document rules, 600 boundary and 500 additional rule/reasoning questions |
 | Training schedule | Two epochs, global batch 32, seed 17 |
 | Late mixing | Starts halfway through main training; late examples repeat three times |
-| Selected checkpoint | Step 2643 |
+| Selected checkpoint | Step 2658 |
 
-The main and late partitions intentionally share 249 replay records. [Qev-train](https://huggingface.co/datasets/AustinFu/Qev-train) publishes 2,442 synthetic alignment, rule-compliance, world-knowledge and HelpSteer3-derived boundary examples, with generation methods and source-specific licenses. The complete mixed training corpus is not distributed. [Training guide](https://github.com/QiqianFu/Qev/blob/main/docs/training.md) · [Data recipe](https://github.com/QiqianFu/Qev/blob/main/docs/data.md).
+The main and late partitions intentionally share 249 replay records. [Qev-train](https://huggingface.co/datasets/AustinFu/Qev-train) publishes 2,442 synthetic alignment, rule-compliance, world-knowledge and HelpSteer3-derived boundary examples, with generation methods and source-specific licenses. The complete mixed training corpus, including the added web-action and rule/reasoning records, is not distributed. [Training guide](https://github.com/QiqianFu/Qev/blob/main/docs/training.md) · [Data recipe](https://github.com/QiqianFu/Qev/blob/main/docs/data.md).
 
 ## Evaluation
 
 **Qev-9B uses BF16 backbone computation; Kev-9B uses FP32.**
 
 <p align="center">
-  <img src="assets/evaluation.svg" alt="Qev-9B and Qev-2B accuracy compared with their Qwen3.5 base models on seven benchmarks." width="100%">
+  <img src="assets/evaluation-matrix.svg" alt="Qev models, native bases and reference models across ten benchmarks." width="100%">
 </p>
 
 | Benchmark | Jev (reference) | Qwen3.5-9B-Base | Qev-9B | Kev-9B |
 |---|---:|---:|---:|---:|
-| Decision development · clean | 84.49 | 77.69 | **87.42** | 87.18 |
-| Transfer development · clean | 85.67 | 74.39 | **83.99** | 82.16 |
-| MMLU-Pro · 1,000 | 83.50 | 50.40 | **57.40** | 51.10 |
-| SemIf · 144 handwritten | 96.53 | 90.28 | **93.06** | 90.97 |
-| scienthoon · 873 | 75.26 | 68.84 | 71.02 | **75.49** |
-| WANLI · 256 | 75.78 | 67.97 | **71.09** | 70.31 |
-| JevBench public · 231 | 85.71 | 75.76 | **83.12** | 75.76 |
-
-Accuracy (%). Bold compares Qev with Kev.
+| JevBench public · 231 | 85.71 | 75.76 | **80.95** | 75.76 |
+| Decision development · clean | 84.49 | 77.69 | **87.58** | 87.18 |
+| Transfer development · clean | 85.67 | 74.39 | **83.69** | 82.16 |
+| MMLU-Pro · 1,000 | 83.50 | 50.40 | **56.50** | 51.10 |
+| SemIf · 144 handwritten | 96.53 | 90.28 | **93.75** | 90.97 |
+| scienthoon · 873 | 75.26 | 68.84 | 74.80 | **75.49** |
+| WANLI · 256 | 75.78 | 67.97 | **75.00** | 70.31 |
+| GSM8K · multiple choice | 79.87 | 55.53 | **61.75** | 46.36 |
+| ChessBench | 17.22 | 13.22 | 10.34 | **11.76** |
+| BPoMP · variant mean | 90.92 | 59.39 | **81.03** | 66.93 |
+
+Scores (%). The new GSM8K, ChessBench and BPoMP rows use [Decision Index raw scores](https://github.com/QiqianFu/Qev/blob/main/docs/decision-index.md); the original rows use accuracy. Bold compares Qev with Kev.
 
 These are the recorded results for the released checkpoint, using full causal reference execution. The selected model is a single seed, and public benchmarks were observed during research iteration. The comparisons do not isolate architecture gains. [Results, sources, and reproduction commands](https://github.com/QiqianFu/Qev/blob/main/docs/evaluation.md).
 

THIRD_PARTY_NOTICES.md

@@ -4,7 +4,7 @@
 
 Copyright 2026 Qiqian Fu and Qev contributors.
 
-Qev's code, documentation, original SVG illustrations, example requests, and generated prediction results are provided under the [Apache License 2.0](LICENSE). The Qev adaptation weights published at [AustinFu/Qev-9B](https://huggingface.co/AustinFu/Qev-9B), including its LoRA adapter, decision head, and interaction gate, use the same license. Contributions are accepted under these terms unless explicitly stated otherwise.
+Qev's code, documentation, original SVG illustrations, example requests, and generated prediction results are provided under the [Apache License 2.0](LICENSE). The Qev adaptation weights published at [AustinFu/Qev-2B](https://huggingface.co/AustinFu/Qev-2B), [AustinFu/Qev-4B](https://huggingface.co/AustinFu/Qev-4B) and [AustinFu/Qev-9B](https://huggingface.co/AustinFu/Qev-9B), including their LoRA adapters, decision heads and interaction gates, use the same license. Contributions are accepted under these terms unless explicitly stated otherwise.
 
 Third-party materials retain the terms listed below. The Qev license does not relicense training datasets, external services, or third-party recordings.
 
@@ -13,7 +13,7 @@ Third-party materials retain the terms listed below. The Qev license does not re
 | Material | Source and copyright | License and scope |
 |---|---|---|
 | Kev conventions and adapted implementation | [Jared Palmer's Kev](https://github.com/jaredpalmer/kev), Copyright 2026 Jared Palmer | [Apache-2.0, original notice retained](licenses/Kev-Apache-2.0.txt). Qev adapts delimiter and rendering conventions, LoRA targets, and cache forking. Modified source files identify the Qev adaptation. |
-| Qwen models and tokenizers | [Qwen3.5-9B-Base](https://huggingface.co/Qwen/Qwen3.5-9B-Base) and [Qwen3.5-2B-Base](https://huggingface.co/Qwen/Qwen3.5-2B-Base), Copyright 2026 Alibaba Cloud | [Apache-2.0](licenses/Qwen3.5-Apache-2.0.txt). Tokenizers accompany the corresponding Qev checkpoints; base weights are downloaded separately. |
+| Qwen models and tokenizers | [Qwen3.5-9B-Base](https://huggingface.co/Qwen/Qwen3.5-9B-Base), [Qwen3.5-4B-Base](https://huggingface.co/Qwen/Qwen3.5-4B-Base) and [Qwen3.5-2B-Base](https://huggingface.co/Qwen/Qwen3.5-2B-Base), Copyright 2026 Alibaba Cloud | [Apache-2.0](licenses/Qwen3.5-Apache-2.0.txt). Tokenizers accompany the corresponding Qev checkpoints; base weights are downloaded separately. |
 | JevBench public evaluation tasks | [JevBench](https://github.com/fstandhartinger/jevbench), Copyright 2026 Florian Standhartinger and contributors | [MIT](licenses/JevBench-MIT.txt), with [upstream third-party notices](licenses/JevBench-THIRD-PARTY.md). Tasks are downloaded by the optional preparation script. The original license, third-party notice, and per-question provenance accompany the prepared dataset. |
 | Crafter visuals in the gameplay recording | [Crafter](https://github.com/danijar/crafter), Copyright 2021 Danijar Hafner | [MIT](licenses/Crafter-MIT.txt). The gameplay recording contains the environment's visual assets; the recording and Qev decision overlay were produced for this project. |
 
@@ -35,7 +35,7 @@ Their source code and dependency distributions are not vendored in Qev's Python
 
 The [Qev-train dataset](https://huggingface.co/datasets/AustinFu/Qev-train) publishes 2,442 synthetic training examples separately from this code repository. Its 1,534 original alignment and rule-compliance examples use Apache-2.0 where copyright applies. Its 308 Wikipedia-grounded knowledge examples retain CC BY-SA 4.0, with article versions, contributor links and changes recorded in `ATTRIBUTION.jsonl`. Its 600 boundary examples adapt NVIDIA HelpSteer3 Preference contexts and retain [CC BY 4.0](licenses/HelpSteer3-CC-BY-4.0.txt), with source lineage and changes in `HELPSTEER3_ATTRIBUTION.jsonl`. The [dataset license](https://huggingface.co/datasets/AustinFu/Qev-train/blob/main/LICENSE.md) identifies the scope of each component; combining them does not relicense the derived components under Apache-2.0.
 
-Qev-9B v0.2.0 also uses 4,459 Principle judgments from [NVIDIA HelpSteer3](https://huggingface.co/datasets/nvidia/HelpSteer3), by Zhilin Wang and collaborators, under CC BY 4.0. The source revision is `f6d145777bcbde96137596340fab89793acd1031`; those original judgments are not part of the synthetic Qev-train release. No endorsement by NVIDIA or the source authors is implied.
+Qev-9B v0.3.0 also uses 2,230 Principle judgments from [NVIDIA HelpSteer3](https://huggingface.co/datasets/nvidia/HelpSteer3), by Zhilin Wang and collaborators, under CC BY 4.0. The source revision is `f6d145777bcbde96137596340fab89793acd1031`; those original judgments are not part of the synthetic Qev-train release. No endorsement by NVIDIA or the source authors is implied.
 
 The complete research training mixture and distillation data are not distributed. Other source datasets retain their own terms.
 

adapter/adapter_config.json

@@ -32,18 +32,18 @@
   "rank_pattern": {},
   "revision": "68c46c4b3498877f3ef123c856ecfde50c39f404",
   "target_modules": [
-    "down_proj",
-    "in_proj_b",
-    "out_proj",
-    "v_proj",
-    "k_proj",
-    "in_proj_z",
     "o_proj",
+    "gate_proj",
+    "in_proj_z",
     "in_proj_qkv",
-    "q_proj",
+    "v_proj",
     "in_proj_a",
-    "gate_proj",
-    "up_proj"
+    "in_proj_b",
+    "k_proj",
+    "q_proj",
+    "up_proj",
+    "down_proj",
+    "out_proj"
   ],
   "target_parameters": null,
   "task_type": "FEATURE_EXTRACTION",

adapter/adapter_model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:9554dd117e73c4f8ee34805f9c3223fa1a7d9e35ea050e45116600ecbc5f5217
+oid sha256:d4edcdc2aa6c4ec78969939ca19d259d631ad17410aaae01ed0a01d67597bc11
 size 692518608

benchmarks.json

@@ -1,5 +1,5 @@
 {
-  "date": "2026-09-30",
+  "date": "2026-10-03",
   "metric": "argmax_accuracy",
   "models": {
     "jev": {
@@ -14,10 +14,10 @@
     },
     "qev_9b": {
       "repo_id": "AustinFu/Qev-9B",
-      "revision": "v0.2.0",
+      "revision": "v0.3.0",
       "base": "Qwen/Qwen3.5-9B-Base",
       "base_revision": "68c46c4b3498877f3ef123c856ecfde50c39f404",
-      "step": 2643,
+      "step": 2658,
       "dtype": "bf16",
       "execution": "reference",
       "seed": 17,
@@ -41,6 +41,48 @@
       "dtype": "bf16",
       "base": "Qwen/Qwen3.5-2B-Base",
       "base_revision": "b1485b2fa6dfa1287294f269f5fb618e03d52d7c"
+    },
+    "qev_4b": {
+      "repo_id": "AustinFu/Qev-4B",
+      "revision": "v0.1.0",
+      "base": "Qwen/Qwen3.5-4B-Base",
+      "base_revision": "710fd005d44d55ee27b7ad5147e318e546efdbfe",
+      "step": 2786,
+      "dtype": "bf16",
+      "execution": "reference",
+      "seed": 17,
+      "temperature": 1.0,
+      "description": "Initialized directly from the Qwen base; teacher-probability cross entropy",
+      "teacher_repo_id": "AustinFu/Qev-9B",
+      "teacher_revision": "v0.3.0"
+    },
+    "qwen35_4b_base": {
+      "repo_id": "Qwen/Qwen3.5-4B-Base",
+      "revision": "710fd005d44d55ee27b7ad5147e318e546efdbfe",
+      "dtype": "bf16",
+      "description": "Frozen native LM head; zero-shot candidate code probabilities"
+    },
+    "kev_4b": {
+      "repo_id": "jaredpalmer/kev-4b",
+      "revision": "139fdd9",
+      "code_revision": "5920c5f",
+      "dtype": "fp32",
+      "temperature": 1.0,
+      "description": "Author inference code, unmerged adapter; Qwen3.5-4B-Base"
+    },
+    "jevany_4b_pointer": {
+      "repo_id": "SimpleJev/JevAny-Qwen3.5-4B-LoRA",
+      "code_revision": "33cb677",
+      "dtype": "fp32",
+      "temperature": 1.0,
+      "description": "Pointer head on post-trained Qwen3.5-4B, author exact inference path"
+    },
+    "jevany_4b_direct": {
+      "repo_id": "SimpleJev/JevAny-Qwen3.5-4B-Direct-Token-LoRA",
+      "code_revision": "33cb677",
+      "dtype": "fp32",
+      "temperature": 1.0,
+      "description": "Direct-token readout on post-trained Qwen3.5-4B, author exact inference path"
     }
   },
   "results": {
@@ -56,8 +98,8 @@
           "accuracy": 0.8780653950953679
         },
         "qev_9b": {
-          "correct": 1295,
-          "accuracy": 0.8821525885558583
+          "correct": 1296,
+          "accuracy": 0.8828337874659401
         },
         "qwen35_9b_base": {
           "correct": 1112,
@@ -70,6 +112,26 @@
         "qwen35_2b_base": {
           "correct": 927,
           "accuracy": 0.6314713896457765
+        },
+        "qev_4b": {
+          "correct": 1286,
+          "accuracy": 0.8760217983651226
+        },
+        "qwen35_4b_base": {
+          "correct": 1050,
+          "accuracy": 0.715258855585831
+        },
+        "kev_4b": {
+          "correct": 1293,
+          "accuracy": 0.8807901907356949
+        },
+        "jevany_4b_pointer": {
+          "correct": 1281,
+          "accuracy": 0.8726158038147139
+        },
+        "jevany_4b_direct": {
+          "correct": 1278,
+          "accuracy": 0.8705722070844687
         }
       }
     },
@@ -85,8 +147,8 @@
           "accuracy": 0.8718354430379747
         },
         "qev_9b": {
-          "correct": 1105,
-          "accuracy": 0.8742088607594937
+          "correct": 1107,
+          "accuracy": 0.8757911392405063
         },
         "qwen35_9b_base": {
           "correct": 982,
@@ -99,6 +161,26 @@
         "qwen35_2b_base": {
           "correct": 827,
           "accuracy": 0.6542721518987342
+        },
+        "qev_4b": {
+          "correct": 1099,
+          "accuracy": 0.8694620253164557
+        },
+        "qwen35_4b_base": {
+          "correct": 932,
+          "accuracy": 0.7373417721518988
+        },
+        "kev_4b": {
+          "correct": 1103,
+          "accuracy": 0.872626582278481
+        },
+        "jevany_4b_pointer": {
+          "correct": 1095,
+          "accuracy": 0.8662974683544303
+        },
+        "jevany_4b_direct": {
+          "correct": 1090,
+          "accuracy": 0.8623417721518988
         }
       }
     },
@@ -114,8 +196,8 @@
           "accuracy": 0.8115183246073299
         },
         "qev_9b": {
-          "correct": 630,
-          "accuracy": 0.824607329842932
+          "correct": 629,
+          "accuracy": 0.8232984293193717
         },
         "qwen35_9b_base": {
           "correct": 561,
@@ -128,6 +210,26 @@
         "qwen35_2b_base": {
           "correct": 493,
           "accuracy": 0.6452879581151832
+        },
+        "qev_4b": {
+          "correct": 618,
+          "accuracy": 0.8089005235602095
+        },
+        "qwen35_4b_base": {
+          "correct": 537,
+          "accuracy": 0.7028795811518325
+        },
+        "kev_4b": {
+          "correct": 615,
+          "accuracy": 0.8049738219895288
+        },
+        "jevany_4b_pointer": {
+          "correct": 654,
+          "accuracy": 0.856020942408377
+        },
+        "jevany_4b_direct": {
+          "correct": 660,
+          "accuracy": 0.8638743455497382
         }
       }
     },
@@ -143,8 +245,8 @@
           "accuracy": 0.8216463414634146
         },
         "qev_9b": {
-          "correct": 551,
-          "accuracy": 0.8399390243902439
+          "correct": 549,
+          "accuracy": 0.836890243902439
         },
         "qwen35_9b_base": {
           "correct": 488,
@@ -157,6 +259,26 @@
         "qwen35_2b_base": {
           "correct": 427,
           "accuracy": 0.6509146341463414
+        },
+        "qev_4b": {
+          "correct": 538,
+          "accuracy": 0.8201219512195121
+        },
+        "qwen35_4b_base": {
+          "correct": 469,
+          "accuracy": 0.7149390243902439
+        },
+        "kev_4b": {
+          "correct": 536,
+          "accuracy": 0.8170731707317073
+        },
+        "jevany_4b_pointer": {
+          "correct": 555,
+          "accuracy": 0.8460365853658537
+        },
+        "jevany_4b_direct": {
+          "correct": 561,
+          "accuracy": 0.8551829268292683
         }
       }
     },
@@ -172,8 +294,8 @@
           "accuracy": 0.511
         },
         "qev_9b": {
-          "correct": 574,
-          "accuracy": 0.574
+          "correct": 565,
+          "accuracy": 0.565
         },
         "qwen35_9b_base": {
           "correct": 504,
@@ -186,6 +308,26 @@
         "qwen35_2b_base": {
           "correct": 312,
           "accuracy": 0.312
+        },
+        "qev_4b": {
+          "correct": 503,
+          "accuracy": 0.503
+        },
+        "qwen35_4b_base": {
+          "correct": 435,
+          "accuracy": 0.435
+        },
+        "kev_4b": {
+          "correct": 524,
+          "accuracy": 0.524
+        },
+        "jevany_4b_pointer": {
+          "correct": 523,
+          "accuracy": 0.523
+        },
+        "jevany_4b_direct": {
+          "correct": 511,
+          "accuracy": 0.511
         }
       }
     },
@@ -201,8 +343,8 @@
           "accuracy": 0.9097222222222222
         },
         "qev_9b": {
-          "correct": 134,
-          "accuracy": 0.9305555555555556
+          "correct": 135,
+          "accuracy": 0.9375
         },
         "qwen35_9b_base": {
           "correct": 130,
@@ -215,6 +357,26 @@
         "qwen35_2b_base": {
           "correct": 92,
           "accuracy": 0.6388888888888888
+        },
+        "qev_4b": {
+          "correct": 130,
+          "accuracy": 0.9027777777777778
+        },
+        "qwen35_4b_base": {
+          "correct": 111,
+          "accuracy": 0.7708333333333334
+        },
+        "kev_4b": {
+          "correct": 128,
+          "accuracy": 0.8888888888888888
+        },
+        "jevany_4b_pointer": {
+          "correct": 130,
+          "accuracy": 0.9027777777777778
+        },
+        "jevany_4b_direct": {
+          "correct": 130,
+          "accuracy": 0.9027777777777778
         }
       }
     },
@@ -230,8 +392,8 @@
           "accuracy": 0.7548682703321878
         },
         "qev_9b": {
-          "correct": 620,
-          "accuracy": 0.7101947308132875
+          "correct": 653,
+          "accuracy": 0.7479954180985109
         },
         "qwen35_9b_base": {
           "correct": 601,
@@ -244,6 +406,26 @@
         "qwen35_2b_base": {
           "correct": 470,
           "accuracy": 0.5383734249713631
+        },
+        "qev_4b": {
+          "correct": 670,
+          "accuracy": 0.7674684994272624
+        },
+        "qwen35_4b_base": {
+          "correct": 649,
+          "accuracy": 0.7434135166093929
+        },
+        "kev_4b": {
+          "correct": 631,
+          "accuracy": 0.722794959908362
+        },
+        "jevany_4b_pointer": {
+          "correct": 605,
+          "accuracy": 0.693012600229095
+        },
+        "jevany_4b_direct": {
+          "correct": 601,
+          "accuracy": 0.6884306987399771
         }
       }
     },
@@ -259,8 +441,8 @@
           "accuracy": 0.703125
         },
         "qev_9b": {
-          "correct": 182,
-          "accuracy": 0.7109375
+          "correct": 192,
+          "accuracy": 0.75
         },
         "qwen35_9b_base": {
           "correct": 174,
@@ -273,6 +455,26 @@
         "qwen35_2b_base": {
           "correct": 129,
           "accuracy": 0.50390625
+        },
+        "qev_4b": {
+          "correct": 188,
+          "accuracy": 0.734375
+        },
+        "qwen35_4b_base": {
+          "correct": 155,
+          "accuracy": 0.60546875
+        },
+        "kev_4b": {
+          "correct": 176,
+          "accuracy": 0.6875
+        },
+        "jevany_4b_pointer": {
+          "correct": 182,
+          "accuracy": 0.7109375
+        },
+        "jevany_4b_direct": {
+          "correct": 182,
+          "accuracy": 0.7109375
         }
       }
     },
@@ -288,8 +490,8 @@
           "accuracy": 0.7575757575757576
         },
         "qev_9b": {
-          "correct": 192,
-          "accuracy": 0.8311688311688312
+          "correct": 187,
+          "accuracy": 0.8095238095238095
         },
         "qwen35_9b_base": {
           "correct": 175,
@@ -302,6 +504,26 @@
         "qwen35_2b_base": {
           "correct": 146,
           "accuracy": 0.6320346320346321
+        },
+        "qev_4b": {
+          "correct": 190,
+          "accuracy": 0.8225108225108225
+        },
+        "qwen35_4b_base": {
+          "correct": 155,
+          "accuracy": 0.670995670995671
+        },
+        "kev_4b": {
+          "correct": 175,
+          "accuracy": 0.7575757575757576
+        },
+        "jevany_4b_pointer": {
+          "correct": 181,
+          "accuracy": 0.7835497835497836
+        },
+        "jevany_4b_direct": {
+          "correct": 184,
+          "accuracy": 0.7965367965367965
         }
       }
     }
@@ -312,10 +534,15 @@
       "correct": {
         "jev": 71,
         "kev_9b": 65,
-        "qev_9b": 68,
+        "qev_9b": 67,
         "qwen35_9b_base": 59,
         "qev_2b": 64,
-        "qwen35_2b_base": 44
+        "qwen35_2b_base": 44,
+        "qev_4b": 69,
+        "qwen35_4b_base": 53,
+        "kev_4b": 67,
+        "jevany_4b_pointer": 69,
+        "jevany_4b_direct": 71
       }
     },
     "easy": {
@@ -326,7 +553,12 @@
         "qev_9b": 48,
         "qwen35_9b_base": 48,
         "qev_2b": 48,
-        "qwen35_2b_base": 48
+        "qwen35_2b_base": 48,
+        "qev_4b": 48,
+        "qwen35_4b_base": 48,
+        "kev_4b": 48,
+        "jevany_4b_pointer": 48,
+        "jevany_4b_direct": 48
       }
     },
     "hard": {
@@ -334,10 +566,15 @@
       "correct": {
         "jev": 79,
         "kev_9b": 62,
-        "qev_9b": 76,
+        "qev_9b": 72,
         "qwen35_9b_base": 68,
         "qev_2b": 60,
-        "qwen35_2b_base": 54
+        "qwen35_2b_base": 54,
+        "qev_4b": 73,
+        "qwen35_4b_base": 54,
+        "kev_4b": 60,
+        "jevany_4b_pointer": 64,
+        "jevany_4b_direct": 65
       }
     }
   },

decision-index.json

@@ -0,0 +1,824 @@
+{
+  "date": "2026-10-03",
+  "edition": "0.2.1",
+  "metric": "official_raw_score",
+  "code_revision": "87d4650b42b377c0291a89c1f1a879f9b31082bf",
+  "source_url": "https://github.com/apolinario/decision-index/tree/87d4650b42b377c0291a89c1f1a879f9b31082bf",
+  "jev_reference_url": "https://github.com/apolinario/decision-index/blob/87d4650b42b377c0291a89c1f1a879f9b31082bf/tests/fixtures/board-0.2.1.json",
+  "results": {
+    "gsm8k": {
+      "catalog_id": 30,
+      "title": "GSM8K",
+      "metric": "mean track accuracy",
+      "total": 2638,
+      "models": {
+        "qwen35_2b_base": {
+          "raw": 0.304,
+          "skill": 0.1609,
+          "coverage": 1.0,
+          "answered": 2638,
+          "native_score": 0.30401819560272936,
+          "tracks": [
+            {
+              "track": "GSM8K-4choice",
+              "raw": 0.40788476118271416,
+              "skill": 0.21051301491028554,
+              "random": 0.25,
+              "coverage": 1
+            },
+            {
+              "track": "GSM8K-10choice",
+              "raw": 0.2001516300227445,
+              "skill": 0.11127958891416054,
+              "random": 0.1,
+              "coverage": 1
+            }
+          ],
+          "source": "local_evaluation"
+        },
+        "qev_2b": {
+          "raw": 0.3776,
+          "skill": 0.2531,
+          "coverage": 1.0,
+          "answered": 2638,
+          "native_score": 0.3775587566338135,
+          "tracks": [
+            {
+              "track": "GSM8K-4choice",
+              "raw": 0.5026535253980288,
+              "skill": 0.3368713671973717,
+              "random": 0.25,
+              "coverage": 1
+            },
+            {
+              "track": "GSM8K-10choice",
+              "raw": 0.25246398786959817,
+              "skill": 0.16940443096622018,
+              "random": 0.1,
+              "coverage": 1
+            }
+          ],
+          "source": "local_evaluation"
+        },
+        "qwen35_4b_base": {
+          "raw": 0.37,
+          "skill": 0.2381,
+          "coverage": 1.0,
+          "answered": 2638,
+          "native_score": 0.36997725549658833,
+          "tracks": [
+            {
+              "track": "GSM8K-4choice",
+              "raw": 0.44275966641395,
+              "skill": 0.2570128885519333,
+              "random": 0.25,
+              "coverage": 1
+            },
+            {
+              "track": "GSM8K-10choice",
+              "raw": 0.2971948445792267,
+              "skill": 0.2191053828658074,
+              "random": 0.1,
+              "coverage": 1
+            }
+          ],
+          "source": "local_evaluation"
+        },
+        "qev_4b": {
+          "raw": 0.5459,
+          "skill": 0.4585,
+          "coverage": 1.0,
+          "answered": 2638,
+          "native_score": 0.5458680818802123,
+          "tracks": [
+            {
+              "track": "GSM8K-4choice",
+              "raw": 0.6679302501895376,
+              "skill": 0.5572403335860501,
+              "random": 0.25,
+              "coverage": 1
+            },
+            {
+              "track": "GSM8K-10choice",
+              "raw": 0.42380591357088704,
+              "skill": 0.3597843484120967,
+              "random": 0.1,
+              "coverage": 1
+            }
+          ],
+          "source": "local_evaluation"
+        },
+        "qwen35_9b_base": {
+          "raw": 0.5553,
+          "skill": 0.4642,
+          "coverage": 1.0,
+          "answered": 2638,
+          "native_score": 0.5553449583017438,
+          "tracks": [
+            {
+              "track": "GSM8K-4choice",
+              "raw": 0.624715693707354,
+              "skill": 0.4996209249431387,
+              "random": 0.25,
+              "coverage": 1
+            },
+            {
+              "track": "GSM8K-10choice",
+              "raw": 0.48597422289613346,
+              "skill": 0.4288602476623705,
+              "random": 0.1,
+              "coverage": 1
+            }
+          ],
+          "source": "local_evaluation"
+        },
+        "qev_9b": {
+          "raw": 0.6175,
+          "skill": 0.5442,
+          "coverage": 1.0,
+          "answered": 2638,
+          "native_score": 0.6175132676269901,
+          "tracks": [
+            {
+              "track": "GSM8K-4choice",
+              "raw": 0.7225170583775588,
+              "skill": 0.6300227445034117,
+              "random": 0.25,
+              "coverage": 1
+            },
+            {
+              "track": "GSM8K-10choice",
+              "raw": 0.5125094768764216,
+              "skill": 0.458343863196024,
+              "random": 0.1,
+              "coverage": 1
+            }
+          ],
+          "source": "local_evaluation",
+          "model_revision": "v0.3.0"
+        },
+        "kev_9b": {
+          "raw": 0.4636,
+          "skill": 0.357,
+          "coverage": 1.0,
+          "answered": 2638,
+          "native_score": 0.4636087945413192,
+          "tracks": [
+            {
+              "track": "GSM8K-4choice",
+              "raw": 0.5769522365428355,
+              "skill": 0.43593631539044725,
+              "random": 0.25,
+              "coverage": 1
+            },
+            {
+              "track": "GSM8K-10choice",
+              "raw": 0.3502653525398029,
+              "skill": 0.2780726139331143,
+              "random": 0.1,
+              "coverage": 1
+            }
+          ],
+          "source": "local_evaluation"
+        },
+        "kev_4b": {
+          "raw": 0.5849,
+          "skill": 0.5003,
+          "coverage": 1.0,
+          "answered": 2638,
+          "native_score": 0.5849128127369219,
+          "tracks": [
+            {
+              "track": "GSM8K-4choice",
+              "raw": 0.6535253980288097,
+              "skill": 0.5380338640384129,
+              "random": 0.25,
+              "coverage": 1
+            },
+            {
+              "track": "GSM8K-10choice",
+              "raw": 0.5163002274450341,
+              "skill": 0.46255580827226017,
+              "random": 0.1,
+              "coverage": 1
+            }
+          ],
+          "source": "local_evaluation"
+        },
+        "jevany_4b_pointer": {
+          "raw": 0.4761,
+          "skill": 0.3736,
+          "coverage": 1.0,
+          "answered": 2638,
+          "native_score": 0.4761182714177407,
+          "tracks": [
+            {
+              "track": "GSM8K-4choice",
+              "raw": 0.601213040181956,
+              "skill": 0.46828405357594133,
+              "random": 0.25,
+              "coverage": 1
+            },
+            {
+              "track": "GSM8K-10choice",
+              "raw": 0.3510235026535254,
+              "skill": 0.2789150029483615,
+              "random": 0.1,
+              "coverage": 1
+            }
+          ],
+          "source": "local_evaluation"
+        },
+        "jevany_4b_direct": {
+          "raw": 0.4503,
+          "skill": 0.3407,
+          "coverage": 1.0,
+          "answered": 2638,
+          "native_score": 0.45034116755117515,
+          "tracks": [
+            {
+              "track": "GSM8K-4choice",
+              "raw": 0.5625473843821076,
+              "skill": 0.41672984584281014,
+              "random": 0.25,
+              "coverage": 1
+            },
+            {
+              "track": "GSM8K-10choice",
+              "raw": 0.33813495072024263,
+              "skill": 0.2645943896891585,
+              "random": 0.1,
+              "coverage": 1
+            }
+          ],
+          "source": "local_evaluation"
+        },
+        "jev": {
+          "raw": 0.7987,
+          "skill": 0.7565,
+          "coverage": 1,
+          "source": "published_leaderboard",
+          "version": "Jev 1.13.0"
+        }
+      }
+    },
+    "chessbench": {
+      "catalog_id": 31,
+      "title": "ChessBench",
+      "metric": "best-move accuracy, accepting tied optima",
+      "total": 5000,
+      "models": {
+        "qwen35_2b_base": {
+          "raw": 0.0884,
+          "skill": 0.0071,
+          "coverage": 1.0,
+          "answered": 5000,
+          "native_score": 0.0884,
+          "tracks": [
+            {
+              "track": "all",
+              "raw": 0.0884,
+              "skill": 0.00712784287150035,
+              "random": 0.0818556110623024,
+              "coverage": 1
+            }
+          ],
+          "source": "local_evaluation"
+        },
+        "qev_2b": {
+          "raw": 0.1098,
+          "skill": 0.0304,
+          "coverage": 1.0,
+          "answered": 5000,
+          "native_score": 0.1098,
+          "tracks": [
+            {
+              "track": "all",
+              "raw": 0.1098,
+              "skill": 0.03043572369922071,
+              "random": 0.0818556110623024,
+              "coverage": 1
+            }
+          ],
+          "source": "local_evaluation"
+        },
+        "qwen35_4b_base": {
+          "raw": 0.1178,
+          "skill": 0.0391,
+          "coverage": 1.0,
+          "answered": 5000,
+          "native_score": 0.1178,
+          "tracks": [
+            {
+              "track": "all",
+              "raw": 0.1178,
+              "skill": 0.0391489501768732,
+              "random": 0.0818556110623024,
+              "coverage": 1
+            }
+          ],
+          "source": "local_evaluation"
+        },
+        "qev_4b": {
+          "raw": 0.1242,
+          "skill": 0.0461,
+          "coverage": 1.0,
+          "answered": 5000,
+          "native_score": 0.1242,
+          "tracks": [
+            {
+              "track": "all",
+              "raw": 0.1242,
+              "skill": 0.04611953135899518,
+              "random": 0.0818556110623024,
+              "coverage": 1
+            }
+          ],
+          "source": "local_evaluation"
+        },
+        "qwen35_9b_base": {
+          "raw": 0.1322,
+          "skill": 0.0548,
+          "coverage": 1.0,
+          "answered": 5000,
+          "native_score": 0.1322,
+          "tracks": [
+            {
+              "track": "all",
+              "raw": 0.1322,
+              "skill": 0.054832757836647664,
+              "random": 0.0818556110623024,
+              "coverage": 1
+            }
+          ],
+          "source": "local_evaluation"
+        },
+        "qev_9b": {
+          "raw": 0.1034,
+          "skill": 0.0235,
+          "coverage": 1.0,
+          "answered": 5000,
+          "native_score": 0.1034,
+          "tracks": [
+            {
+              "track": "all",
+              "raw": 0.1034,
+              "skill": 0.023465142517098744,
+              "random": 0.0818556110623024,
+              "coverage": 1
+            }
+          ],
+          "source": "local_evaluation",
+          "model_revision": "v0.3.0"
+        },
+        "kev_9b": {
+          "raw": 0.1176,
+          "skill": 0.0389,
+          "coverage": 1.0,
+          "answered": 5000,
+          "native_score": 0.1176,
+          "tracks": [
+            {
+              "track": "all",
+              "raw": 0.1176,
+              "skill": 0.038931119514931875,
+              "random": 0.0818556110623024,
+              "coverage": 1
+            }
+          ],
+          "source": "local_evaluation"
+        },
+        "kev_4b": {
+          "raw": 0.088,
+          "skill": 0.0067,
+          "coverage": 1.0,
+          "answered": 5000,
+          "native_score": 0.088,
+          "tracks": [
+            {
+              "track": "all",
+              "raw": 0.088,
+              "skill": 0.0066921815476177135,
+              "random": 0.0818556110623024,
+              "coverage": 1
+            }
+          ],
+          "source": "local_evaluation"
+        },
+        "jevany_4b_pointer": {
+          "raw": 0.1132,
+          "skill": 0.0341,
+          "coverage": 1.0,
+          "answered": 5000,
+          "native_score": 0.1132,
+          "tracks": [
+            {
+              "track": "all",
+              "raw": 0.1132,
+              "skill": 0.03413884495222302,
+              "random": 0.0818556110623024,
+              "coverage": 1
+            }
+          ],
+          "source": "local_evaluation"
+        },
+        "jevany_4b_direct": {
+          "raw": 0.1172,
+          "skill": 0.0385,
+          "coverage": 1.0,
+          "answered": 5000,
+          "native_score": 0.1172,
+          "tracks": [
+            {
+              "track": "all",
+              "raw": 0.1172,
+              "skill": 0.038495458191049256,
+              "random": 0.0818556110623024,
+              "coverage": 1
+            }
+          ],
+          "source": "local_evaluation"
+        },
+        "jev": {
+          "raw": 0.1722,
+          "skill": 0.0984,
+          "coverage": 1,
+          "source": "published_leaderboard",
+          "version": "Jev 1.13.0"
+        }
+      }
+    },
+    "esci": {
+      "catalog_id": 37,
+      "title": "Amazon ESCI",
+      "metric": "macro-F1",
+      "total": 5000,
+      "models": {
+        "qwen35_2b_base": {
+          "raw": 0.281,
+          "skill": 0.0982,
+          "coverage": 1.0,
+          "answered": 5000,
+          "native_score": 0.281041285029981,
+          "tracks": [
+            {
+              "track": "all",
+              "raw": 0.281041285029981,
+              "skill": 0.09821544691532137,
+              "random": 0.20273782411694521,
+              "coverage": 1
+            }
+          ],
+          "source": "local_evaluation"
+        },
+        "qev_2b": {
+          "raw": 0.3634,
+          "skill": 0.2015,
+          "coverage": 1.0,
+          "answered": 5000,
+          "native_score": 0.3634004876980141,
+          "tracks": [
+            {
+              "track": "all",
+              "raw": 0.3634004876980141,
+              "skill": 0.2015179804599628,
+              "random": 0.20273782411694521,
+              "coverage": 1
+            }
+          ],
+          "source": "local_evaluation"
+        },
+        "qwen35_4b_base": {
+          "raw": 0.2918,
+          "skill": 0.1117,
+          "coverage": 1.0,
+          "answered": 5000,
+          "native_score": 0.2918028420723953,
+          "tracks": [
+            {
+              "track": "all",
+              "raw": 0.2918028420723953,
+              "skill": 0.11171358763734261,
+              "random": 0.20273782411694521,
+              "coverage": 1
+            }
+          ],
+          "source": "local_evaluation"
+        },
+        "qev_4b": {
+          "raw": 0.4249,
+          "skill": 0.2786,
+          "coverage": 1.0,
+          "answered": 5000,
+          "native_score": 0.4248617285215504,
+          "tracks": [
+            {
+              "track": "all",
+              "raw": 0.4248617285215504,
+              "skill": 0.2786083563522611,
+              "random": 0.20273782411694521,
+              "coverage": 1
+            }
+          ],
+          "source": "local_evaluation"
+        },
+        "qwen35_9b_base": {
+          "raw": 0.3056,
+          "skill": 0.129,
+          "coverage": 1.0,
+          "answered": 5000,
+          "native_score": 0.3055619197954184,
+          "tracks": [
+            {
+              "track": "all",
+              "raw": 0.3055619197954184,
+              "skill": 0.12897149618892217,
+              "random": 0.20273782411694521,
+              "coverage": 1
+            }
+          ],
+          "source": "local_evaluation"
+        },
+        "qev_9b": {
+          "raw": 0.4708,
+          "skill": 0.3363,
+          "coverage": 1.0,
+          "answered": 5000,
+          "native_score": 0.4708360372719961,
+          "tracks": [
+            {
+              "track": "all",
+              "raw": 0.4708360372719961,
+              "skill": 0.336273588870691,
+              "random": 0.20273782411694521,
+              "coverage": 1
+            }
+          ],
+          "source": "local_evaluation",
+          "model_revision": "v0.3.0"
+        },
+        "kev_9b": {
+          "raw": 0.4809,
+          "skill": 0.3489,
+          "coverage": 1.0,
+          "answered": 5000,
+          "native_score": 0.4809184305561307,
+          "tracks": [
+            {
+              "track": "all",
+              "raw": 0.4809184305561307,
+              "skill": 0.3489198595569521,
+              "random": 0.20273782411694521,
+              "coverage": 1
+            }
+          ],
+          "source": "local_evaluation"
+        },
+        "kev_4b": {
+          "raw": 0.3956,
+          "skill": 0.2419,
+          "coverage": 1.0,
+          "answered": 5000,
+          "native_score": 0.39561788161121436,
+          "tracks": [
+            {
+              "track": "all",
+              "raw": 0.39561788161121436,
+              "skill": 0.24192801731830993,
+              "random": 0.20273782411694521,
+              "coverage": 1
+            }
+          ],
+          "source": "local_evaluation"
+        },
+        "jevany_4b_pointer": {
+          "raw": 0.5494,
+          "skill": 0.4348,
+          "coverage": 1.0,
+          "answered": 5000,
+          "native_score": 0.5493592773788667,
+          "tracks": [
+            {
+              "track": "all",
+              "raw": 0.5493592773788667,
+              "skill": 0.4347647031893874,
+              "random": 0.20273782411694521,
+              "coverage": 1
+            }
+          ],
+          "source": "local_evaluation"
+        },
+        "jevany_4b_direct": {
+          "raw": 0.5425,
+          "skill": 0.4262,
+          "coverage": 1.0,
+          "answered": 5000,
+          "native_score": 0.5424928991743364,
+          "tracks": [
+            {
+              "track": "all",
+              "raw": 0.5424928991743364,
+              "skill": 0.42615225622747677,
+              "random": 0.20273782411694521,
+              "coverage": 1
+            }
+          ],
+          "source": "local_evaluation"
+        },
+        "jev": {
+          "raw": 0.5521,
+          "skill": 0.4382,
+          "coverage": 1,
+          "source": "published_leaderboard",
+          "version": "Jev 1.13.0"
+        }
+      }
+    },
+    "bpomp": {
+      "catalog_id": 20,
+      "title": "BPoMP",
+      "metric": "mean accuracy across poem variants",
+      "total": 5000,
+      "models": {
+        "qwen35_2b_base": {
+          "raw": 0.505,
+          "skill": 0.0101,
+          "coverage": 1.0,
+          "answered": 5000,
+          "native_score": 0.5026,
+          "tracks": [
+            {
+              "track": "all",
+              "raw": 0.5050301177080375,
+              "skill": 0.010060235416075036,
+              "random": 0.5,
+              "coverage": 1
+            }
+          ],
+          "source": "local_evaluation"
+        },
+        "qev_2b": {
+          "raw": 0.7381,
+          "skill": 0.4763,
+          "coverage": 1.0,
+          "answered": 5000,
+          "native_score": 0.7346,
+          "tracks": [
+            {
+              "track": "all",
+              "raw": 0.7381339972176438,
+              "skill": 0.47626799443528767,
+              "random": 0.5,
+              "coverage": 1
+            }
+          ],
+          "source": "local_evaluation"
+        },
+        "qwen35_4b_base": {
+          "raw": 0.6895,
+          "skill": 0.3791,
+          "coverage": 1.0,
+          "answered": 5000,
+          "native_score": 0.6942,
+          "tracks": [
+            {
+              "track": "all",
+              "raw": 0.689541386353874,
+              "skill": 0.37908277270774793,
+              "random": 0.5,
+              "coverage": 1
+            }
+          ],
+          "source": "local_evaluation"
+        },
+        "qev_4b": {
+          "raw": 0.7819,
+          "skill": 0.5638,
+          "coverage": 1.0,
+          "answered": 5000,
+          "native_score": 0.779,
+          "tracks": [
+            {
+              "track": "all",
+              "raw": 0.7818951793165829,
+              "skill": 0.5637903586331658,
+              "random": 0.5,
+              "coverage": 1
+            }
+          ],
+          "source": "local_evaluation"
+        },
+        "qwen35_9b_base": {
+          "raw": 0.5939,
+          "skill": 0.1877,
+          "coverage": 1.0,
+          "answered": 5000,
+          "native_score": 0.5954,
+          "tracks": [
+            {
+              "track": "all",
+              "raw": 0.593868528281851,
+              "skill": 0.187737056563702,
+              "random": 0.5,
+              "coverage": 1
+            }
+          ],
+          "source": "local_evaluation"
+        },
+        "qev_9b": {
+          "raw": 0.8103,
+          "skill": 0.6205,
+          "coverage": 1.0,
+          "answered": 5000,
+          "native_score": 0.8068,
+          "tracks": [
+            {
+              "track": "all",
+              "raw": 0.8102618917009247,
+              "skill": 0.6205237834018493,
+              "random": 0.5,
+              "coverage": 1
+            }
+          ],
+          "source": "local_evaluation",
+          "model_revision": "v0.3.0"
+        },
+        "kev_9b": {
+          "raw": 0.6693,
+          "skill": 0.3387,
+          "coverage": 1.0,
+          "answered": 5000,
+          "native_score": 0.6688,
+          "tracks": [
+            {
+              "track": "all",
+              "raw": 0.6693391049629296,
+              "skill": 0.3386782099258592,
+              "random": 0.5,
+              "coverage": 1
+            }
+          ],
+          "source": "local_evaluation"
+        },
+        "kev_4b": {
+          "raw": 0.6528,
+          "skill": 0.3056,
+          "coverage": 1.0,
+          "answered": 5000,
+          "native_score": 0.6502,
+          "tracks": [
+            {
+              "track": "all",
+              "raw": 0.6528226564034934,
+              "skill": 0.3056453128069867,
+              "random": 0.5,
+              "coverage": 1
+            }
+          ],
+          "source": "local_evaluation"
+        },
+        "jevany_4b_pointer": {
+          "raw": 0.7387,
+          "skill": 0.4773,
+          "coverage": 1.0,
+          "answered": 5000,
+          "native_score": 0.731,
+          "tracks": [
+            {
+              "track": "all",
+              "raw": 0.7386594480630367,
+              "skill": 0.4773188961260735,
+              "random": 0.5,
+              "coverage": 1
+            }
+          ],
+          "source": "local_evaluation"
+        },
+        "jevany_4b_direct": {
+          "raw": 0.8129,
+          "skill": 0.6258,
+          "coverage": 1.0,
+          "answered": 5000,
+          "native_score": 0.8142,
+          "tracks": [
+            {
+              "track": "all",
+              "raw": 0.8128861556266215,
+              "skill": 0.625772311253243,
+              "random": 0.5,
+              "coverage": 1
+            }
+          ],
+          "source": "local_evaluation"
+        },
+        "jev": {
+          "raw": 0.9092,
+          "skill": 0.8184,
+          "coverage": 1,
+          "source": "published_leaderboard",
+          "version": "Jev 1.13.0"
+        }
+      }
+    }
+  }
+}

evaluation.json

@@ -1,85 +1,85 @@
 {
   "model": "Qev-9B",
-  "revision": "v0.2.0",
+  "revision": "v0.3.0",
   "full_suite": {
     "decision_dev": {
-      "correct": 1295,
+      "correct": 1296,
       "total": 1468,
-      "accuracy": 0.8821525885558583,
+      "accuracy": 0.8828337874659401,
       "rejected": 0
     },
     "transfer_dev": {
-      "correct": 630,
+      "correct": 629,
       "total": 764,
-      "accuracy": 0.824607329842932,
+      "accuracy": 0.8232984293193717,
       "rejected": 0
     },
     "mmlupro": {
-      "correct": 574,
+      "correct": 565,
       "total": 1000,
-      "accuracy": 0.574,
+      "accuracy": 0.565,
       "rejected": 0
     },
     "semif": {
-      "correct": 237,
+      "correct": 243,
       "total": 252,
-      "accuracy": 0.9404761904761905,
+      "accuracy": 0.9642857142857143,
       "rejected": 0
     },
     "scienthoon": {
-      "correct": 620,
+      "correct": 653,
       "total": 873,
-      "accuracy": 0.7101947308132875,
+      "accuracy": 0.7479954180985109,
       "rejected": 0
     },
     "wanli": {
-      "correct": 182,
+      "correct": 192,
       "total": 256,
-      "accuracy": 0.7109375,
+      "accuracy": 0.75,
       "rejected": 0
     },
     "public": {
-      "correct": 192,
+      "correct": 187,
       "total": 231,
-      "accuracy": 0.8311688311688312,
+      "accuracy": 0.8095238095238095,
       "rejected": 0
     }
   },
   "matched_subsets": {
     "decision_dev_all": {
-      "correct": 1295,
+      "correct": 1296,
       "total": 1468
     },
     "decision_dev_clean": {
-      "correct": 1105,
+      "correct": 1107,
       "total": 1264
     },
     "transfer_dev_all": {
-      "correct": 630,
+      "correct": 629,
       "total": 764
     },
     "transfer_dev_clean": {
-      "correct": 551,
+      "correct": 549,
       "total": 656
     },
     "mmlupro": {
-      "correct": 574,
+      "correct": 565,
       "total": 1000
     },
-    "semif_handwritten": {
-      "correct": 134,
-      "total": 144
-    },
     "scienthoon": {
-      "correct": 620,
+      "correct": 653,
       "total": 873
     },
     "wanli": {
-      "correct": 182,
+      "correct": 192,
       "total": 256
     },
+    "semif_handwritten": {
+      "correct": 135,
+      "total": 144
+    },
     "jevbench_public": {
-      "correct": 192,
+      "correct": 187,
       "total": 231
     }
   }

finetuning_config.json

@@ -0,0 +1,51 @@
+{
+  "model": {
+    "base": "Qwen/Qwen3.5-9B-Base",
+    "revision": "68c46c4b3498877f3ef123c856ecfde50c39f404",
+    "head_dim": 256,
+    "head_heads": 4,
+    "head_layers": 2,
+    "lora_rank": 64,
+    "lora_dropout": 0.0,
+    "weights_dtype": "bf16",
+    "attention": "sdpa",
+    "rows_per_forward": 32,
+    "layout": "state-question-candidate.v1",
+    "max_padding_ratio": 1.25,
+    "max_padded_tokens": 8192,
+    "choice_none_policy": "as-provided",
+    "candidate_interaction": "last-full-attention"
+  },
+  "limits": {
+    "max_state": 4096,
+    "max_question": 512,
+    "max_candidate": 256,
+    "max_path": 4096,
+    "max_candidates": 128
+  },
+  "training": {
+    "epochs": 2,
+    "batch_size": 1,
+    "accum": 32,
+    "seed": 17,
+    "lr": 1e-05,
+    "head_lr": 0.0001,
+    "weight_decay": 0.01,
+    "warmup_steps": 20,
+    "head_warmup_steps": 0,
+    "gradient_checkpointing": true,
+    "save_every": 100,
+    "autocast": "bf16",
+    "wall_hours": 16,
+    "batch_assignment": "leaf-balanced-v1",
+    "prefix_execution": "tree-batched",
+    "joint_gate_lr": 0.01,
+    "none_insert_prob": 0.2,
+    "none_insert_absent_frac": 0.5,
+    "none_insert_exempt_sources": [
+      "synthetic/hs3_preference_boundary/",
+      "synthetic/codex_gap/",
+      "synthetic/codex_web/"
+    ]
+  }
+}

head.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:61460922e8e45f8a1be335e2845553cbcd8c2228f209654ea67087b2ef45c035
+oid sha256:4666834ae9c82dc6e6de7a2af39da5fa7437bf3134f6c52b5b8f2378a9599148
 size 11080748

joint.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:8e9b05c40c52c311c98ce5d2b851acb38fe567e7d6f98c64aada18c8a24b65a9
+oid sha256:e99d5d8d3e6363fc73585fef502a14f600b1340995c2bd6f6d6ef2d5f5edb410
 size 136

model.json

@@ -19,10 +19,10 @@
     "candidate_interaction": "last-full-attention"
   },
   "limits": {
-    "max_state": 1024,
+    "max_state": 4096,
     "max_question": 512,
     "max_candidate": 256,
-    "max_path": 2048,
+    "max_path": 4096,
     "max_candidates": 128
   },
   "adapter": true,

provenance.json

@@ -1,10 +1,10 @@
 {
   "checkpoint": {
     "repo_id": "AustinFu/Qev-9B",
-    "revision": "v0.2.0",
+    "revision": "v0.3.0",
     "base": "Qwen/Qwen3.5-9B-Base",
     "base_revision": "68c46c4b3498877f3ef123c856ecfde50c39f404",
-    "step": 2643,
+    "step": 2658,
     "dtype": "bf16",
     "execution": "reference",
     "seed": 17,
@@ -12,12 +12,16 @@
   },
   "config": "configs/qev-9b.json",
   "training_recipe": {
-    "main_records": 39605,
-    "main_questions": 45887,
-    "principle_records": 4459,
+    "main_records": 38198,
+    "main_questions": 45503,
+    "principle_records": 2230,
+    "web_records": 1422,
+    "web_questions": 2445,
+    "late_records": 2883,
     "boundary_records": 600,
-    "boundary_partition": "main",
-    "late_records": 1783,
+    "boundary_partition": "late",
+    "additional_rule_reasoning_records": 500,
+    "additional_rule_reasoning_partition": "late",
     "late_fraction": 0.5,
     "late_repeats": 3,
     "epochs": 2,
@@ -29,48 +33,48 @@
     "jevbench_version": "v1.4.2",
     "jevbench_upstream_revision": "1df665e3956d7aab7fa0208ff6c4f2d8557f9f90",
     "trained_limits": {
-      "max_state": 1024,
+      "max_state": 4096,
       "max_question": 512,
       "max_candidate": 256,
-      "max_path": 2048,
+      "max_path": 4096,
       "max_candidates": 128
     }
   },
   "results": {
     "decision_dev_all": {
-      "correct": 1295,
+      "correct": 1296,
       "total": 1468
     },
     "decision_dev_clean": {
-      "correct": 1105,
+      "correct": 1107,
       "total": 1264
     },
     "transfer_dev_all": {
-      "correct": 630,
+      "correct": 629,
       "total": 764
     },
     "transfer_dev_clean": {
-      "correct": 551,
+      "correct": 549,
       "total": 656
     },
     "mmlupro": {
-      "correct": 574,
+      "correct": 565,
       "total": 1000
     },
-    "semif_handwritten": {
-      "correct": 134,
-      "total": 144
-    },
     "scienthoon": {
-      "correct": 620,
+      "correct": 653,
       "total": 873
     },
     "wanli": {
-      "correct": 182,
+      "correct": 192,
       "total": 256
     },
+    "semif_handwritten": {
+      "correct": 135,
+      "total": 144
+    },
     "jevbench_public": {
-      "correct": 192,
+      "correct": 187,
       "total": 231
     }
   }

training_config.json

@@ -17,10 +17,10 @@
     "candidate_interaction": "last-full-attention"
   },
   "limits": {
-    "max_state": 1024,
+    "max_state": 4096,
     "max_question": 512,
     "max_candidate": 256,
-    "max_path": 2048,
+    "max_path": 4096,
     "max_candidates": 128
   },
   "training": {
@@ -44,7 +44,9 @@
     "none_insert_absent_frac": 0.5,
     "none_insert_exempt_sources": [
       "jev_distill/",
-      "synthetic/hs3_preference_boundary/"
+      "synthetic/hs3_preference_boundary/",
+      "synthetic/codex_gap/",
+      "synthetic/codex_web/"
     ],
     "late_split": "late_train",
     "late_fraction": 0.5,