yev0-35b-a3b

An open System One decision model: a LoRA adapter for Qwen/Qwen3.6-35B-A3B (Apache-2.0), a mixture-of-experts model with about 35B parameters, of which about 3B are active per token. Give it a state, a question and its options; it returns a calibrated probability per option from one forward pass, read from the logits of the option letters. It trained on questions with 2–6 options and also handles more (see Benchmarks).

yev0-35b-a3b is yev0-4b's training data (yev-data-short, 44k rows) and recipe on yev2's larger base. It is the fourth cell of the yev experiment on whether better data or a better base model matters more; the yev README has the full comparison.

	yev-data-short (44k rows)	yev-data-long (81k rows)
Qwen3.5-4B-Base	yev0-4b	yev1-4b
Qwen3.6-35B-A3B	yev0-35b-a3b	yev2-35b-a3b

This repo holds the adapter only, not merged weights.

Hardware

The base model must be in GPU memory: 65 GB in bf16, so one 80 GB GPU (A100 80 GB, H100, RTX PRO 6000).

How to serve

transformers, adapter on the base:

hf download choyiny/yev0-35b-a3b --local-dir yev0-35b-a3b
pip install torch peft 'yev[serve,train] @ git+https://github.com/xyspacedev/yev'
yev serve --model yev0-35b-a3b --base Qwen/Qwen3.6-35B-A3B --calibration yev0-35b-a3b/calibration.json \
  --model-name yev0-35b-a3b

vLLM (faster): merge the adapter into the base once, then serve the merged folder as for yev2-35b-a3b, including --max-num-seqs 64:

yev export-merged --model yev0-35b-a3b --base Qwen/Qwen3.6-35B-A3B --out yev0-35b-a3b-merged
cp yev0-35b-a3b/calibration.json yev0-35b-a3b-merged/

curl -s localhost:8000/v1/systemone -H 'content-type: application/json' -d '{
  "state": "I was charged twice. Please help.",
  "questions": {"billing": {"instructions": "Is this message about billing?", "type": "noul"}}}'

Benchmarks

Benchmark	yev0-4b	yev1-4b	yev0-35b-a3b	yev2-35b-a3b
DecideBench v2.0 test (450), accuracy	78.2	79.3	86.0	88.0
DecideBench v2.0 test, pair accuracy	60.0	62.7	74.7	78.2
DecideBench v1.0 test (400), accuracy, with examples	94.5	95.25	98.0	97.75
DecideBench v1.0 test, accuracy, zero-shot	95.0	94.75	96.0	96.75
JevBench public (231), accuracy	78.4	82.7	87.4	90.5
DynaBench test (543), F1	55.7	67.5	83.7	75.9

DecideBench test was never inspected or used for training or model selection. The 95 % interval on the DecideBench v2 accuracy is 82.7–89.1. Although it trained on questions with up to 6 options, it scores higher on DecideBench v2's 100 questions with 8–10 options (92 %) than on the rest (84 %).

Development set (1,898 rows, held out from training): accuracy 95.3 (yev0-4b: 94.1), group accuracy 88.4 (83.9), ECE 0.030 (0.037).

Speed and cost

DecideBench v2, one NVIDIA RTX PRO 6000 Blackwell (96 GB), merged weights on vLLM 0.31.0 behind yev serve, 4 requests in flight: p50 0.38 s, p95 0.72 s, 9.6 req/s, $95 per 1M decisions (wall-clock GPU time at $3.36 per hour).

Training

LoRA r 64, α 128, dropout 0.05 on the attention, linear-attention and shared-expert projections; routed experts frozen. yev0-4b's config (configs/stage0/lc100.json) with only the base model changed: 1 epoch, learning rate 1e-4, 44,228 rows (28.2M tokens), 1,218 steps, 2.6 h on one RTX PRO 6000. Per-type temperatures in calibration.json were fitted on 2,000 held-out rows. Training code: xyspacedev/yev, configs/yev2/yev0-35b-a3b.json.

Licence

Adapter: Apache-2.0. Training-data credits are in NOTICE and ATTRIBUTION.json. Recipe after Together's "How to train your own Jev".