EliovpAI/Qwen3.8-27B-W3Rot-INT3-Paiton-RDNA4

Qwen3.8 27B rotated INT3 weights for Paiton on RDNA4

These are 3-bit replacement weights for the decoder projections of Qwen/Qwen3.8-27B, calibrated with our own GPTQ run on permissively licensed data. They are an add-on to the current Paiton Qwen3.8 release on one AMD Radeon AI PRO R9700, not a standalone checkpoint. The same target checkpoint supplies all other tensors, and the same DFlash2 drafter is used. Against the previous MXFP4 image, weighted decode rises 19.9 %, prefill rises 5–13 %, and model memory falls from 19.18 to 15.89 GiB. The cost was about 3 points of MMLU-Pro knowledge recall; the 12 October update recovers an estimated 0.5 to 1 point of it.

7 November 2026: agentic scale update. The MLP gate and up scales of layers 32 to 63 were distilled toward bf16 again, this time mostly on long coding-agent conversations (bf16 SWE-bench Verified trajectories). The format, the 3-bit codes, the size and the launcher are unchanged; 1.3 % of the scale values changed. On 80 held-out SWE-bench Verified tasks the agent looped on 7 tasks instead of 16 (exact McNemar p 0.078, borderline) and solved 49 instead of 41 (p 0.12, not significant on 80 tasks); GSM8K, HumanEval and MMLU-Pro show no significant change. On general text the new weights are slightly further from bf16 (decode top-1 agreement −0.32 points). On all 500 SWE-bench Verified tasks (8 November) they solved 323 against 284 (p 0.0006), and 165 against 134 on the 250 tasks held out from training (p 0.0002); bf16 solved 390 (details). The distillation conversations quote code from the SWE-bench repositories under their own licenses, one of them (pylint) GPL-2.0; see CALIBRATION.md. The 24 October weights stay available at revision 383fa4b221a992fc2437500328056ef05355c8eb. Use them with the new image r2 below (native drafter kernels, a prompt_logprobs fix); the 5 October image also runs them.

24 October 2026: second accuracy update. The group scales were distilled toward bf16 again, starting from the 12 October weights, on 4.2 million new tokens. The format, the 3-bit codes, the image and the launcher are unchanged; only scale values differ, and the speed is the same. Against the 12 October weights, the served KL divergence to bf16 on the R9700 falls by 3.0 %, the gap in next-token log-likelihood to bf16 by 17–22 %, and long documents (8K to 131K tokens) come 1–2 % closer to bf16 in every length bucket (details). The 12 October weights stay available at revision 3ef7eb149ec0bd9a0f7927491bf30f53f54ced01.

12 October 2026: accuracy update. The BF16 group scales were distilled toward the bf16 model. The format, the 3-bit codes, the runtime image and the launcher are unchanged; only scale values differ. Against the previous weights, the KL divergence to bf16 falls by 20.9 % (decode) and 23.8 % (prefill), and by 19.5 % when served on the R9700, at unchanged speed (details). The previous weights stay available at revision 7a2e3d702144f744c1c5a46eaa42c7a2309d5cf7.

GPU: Radeon AI PRO R9700 (gfx1201, 32 GB) · Size: 9.55 GB · Context: 65,536 by default, up to 262,144 (524,288 opt-in) · Cache: up to 628,877 tokens, 555,128 with images · Text weights

Requirements

Component	Requirement
GPU	One AMD Radeon AI PRO R9700 (gfx1201, RDNA4, 32 GB)
Host	Linux x86-64, Python 3, Docker, AMD GPU device access (/dev/kfd, /dev/dri)
Runtime image	ghcr.io/eliovp/paiton-vllm-plugin:qwen38-rocm10-vllm029-20261107-r2@sha256:efe8c3f3242330009822e97afa10c10fb4a83ec250eb5e5468abc6a6650278c8
Launcher	models/Qwen3.8-MXFP4-DFlash2/run-3bit.sh in the public plugin repository
Target checkpoint	unsloth/Qwen3.8-27B-NVFP4 at f0b7c9e722f5565102fff8481c99e4d86ae099c7
Draft checkpoint	tcclaviger/Qwen3.8-27B-DFlash2-FP8 at ee0cb26a8279b7910cc28d82a8a3e15e4728d56f
These weights	This repository at cbf5b2a05c7f0b9f37a2e34783fcc199db3672b8

Use exactly these revisions. The rotated Gated DeltaNet b/a rows were derived from the target revision's bytes.

Quick start

git clone --depth 1 https://github.com/Eliovp-BV/paiton-vllm-plugin.git
cd paiton-vllm-plugin

export PAITON_TARGET_DIR="$PWD/model-cache/qwen38-nvfp4"
export PAITON_DRAFT_DIR="$PWD/model-cache/qwen38-dflash2"
export PAITON_W3ROT_DIR="$PWD/model-cache/qwen38-w3rot-int3"
export PAITON_CACHE_DIR="$PWD/runtime-cache/qwen38-rocm10-20261107"
mkdir -p "$PAITON_TARGET_DIR" "$PAITON_DRAFT_DIR" "$PAITON_W3ROT_DIR" "$PAITON_CACHE_DIR"

hf download unsloth/Qwen3.8-27B-NVFP4 \
  --revision f0b7c9e722f5565102fff8481c99e4d86ae099c7 --local-dir "$PAITON_TARGET_DIR"
hf download tcclaviger/Qwen3.8-27B-DFlash2-FP8 \
  --revision ee0cb26a8279b7910cc28d82a8a3e15e4728d56f --local-dir "$PAITON_DRAFT_DIR"
hf download EliovpAI/Qwen3.8-27B-W3Rot-INT3-Paiton-RDNA4 \
  --revision cbf5b2a05c7f0b9f37a2e34783fcc199db3672b8 --local-dir "$PAITON_W3ROT_DIR"
(cd "$PAITON_W3ROT_DIR" && sha256sum -c SHA256SUMS)

bash models/Qwen3.8-MXFP4-DFlash2/run-3bit.sh

Already cloned it? Run git pull first; the launcher pins the image above. run-3bit.sh serves these 3-bit weights from PAITON_W3ROT_DIR, and run-mxfp4.sh serves the MXFP4 path on the same image. Pick one mode by adding its flag, e.g. bash models/Qwen3.8-MXFP4-DFlash2/run-3bit.sh --mode long:

What you get with these weights (one R9700; all commands start with bash models/Qwen3.8-MXFP4-DFlash2/):

You run	Per request (prompt + output)	Shared cache (text)	Cache with --vision	Full-length requests that fit together	Prefix caching
run-3bit.sh --mode long-kv4: coding agents, long documents, screenshots	262,144	628,877	555,128	2 (plus one of 100K)	yes
run-3bit.sh --mode long-kv4 --extend-cache	262,144	628,877 on the GPU, plus about 0.7M to 2.8M in system RAM or on an NVMe/SSD	–	2	yes, also after a restart
run-3bit.sh --mode long	262,144 (245,000 with images)	316,325	253,298	1, plus short ones	yes
run-3bit.sh --mode long-512k (experimental)	524,288	594,290	–	1, plus short ones	yes
run-3bit.sh (the default, --mode 65k): fast everyday chat	65,536	393,216	278,050	6	no

Every mode takes up to eight requests at once and uses the checkpoint's FP8 output head (since the 11 October launcher update). --mode long-kv4 keeps the embedding table (2.4 GiB) in pinned system RAM, uses a 4-bit KV cache, and reads long text-only prompts in 2,048-token steps so short questions from other users are answered meanwhile (about 0.7 s instead of 5.9 s during a 64K read). --mode long-512k applies the model's official long-context scaling to every request. Images up to 4K are read at full resolution. The plugin README has the same table for the MXFP4 path (run-mxfp4.sh), what each row gives up, and the measurements. Earlier images stay runnable with --image; the plugin README lists their exact references. Once the server is ready, it accepts OpenAI-compatible requests at http://127.0.0.1:18982/v1 with API model name Qwen3.8.

Method

The MLP, attention and Gated DeltaNet projections of all 64 layers (24.3 billion weights) are symmetric INT3 with one BF16 scale per 128 input elements (3.125 bits per weight). MLP gate/up, attention q/k/v and Gated DeltaNet in_proj are stored in a block-diagonal Hadamard-128 basis, and the runtime rotates their inputs to match.

We ran sequential GPTQ in the rotated basis with an MSE scale search on 292,864 tokens of permissively licensed data. CALIBRATION.md lists the sources.

In the first scale distillation (12 October 2026), starting from the GPTQ result, the BF16 group scales of all 64 layers were trained end to end so that the 3-bit model's next-token distributions match bf16 Qwen3.8-27B: KL divergence on assistant and plain-text tokens, under both serving policies (FP8 activations for decode, 4-bit for prefill). Each scale could move by −18 % to +22 %, and no 3-bit code changed. The data were 5.16 million tokens (57 % code) from seven of the permissively licensed sources in CALIBRATION.md, decontaminated against the evaluation sets; the manifest's finetune block lists them with revisions and licenses.

The second scale distillation (24 October 2026) used the same objective, starting from the 12 October scales, on 4.2 million new tokens (about a third code; also math, chat, multilingual, science and textbook text) from the permissively licensed sources in CALIBRATION.md, none shared with the 12 October data and decontaminated against the evaluation sets. Each scale could again move by −18 % to +22 % from its 12 October value, and no 3-bit code changed.

The agentic scale distillation (7 November 2026) used the same objective, starting from the 24 October scales, with only the MLP gate and up scales of layers 32 to 63 trainable (where the 3-bit model's KL to bf16 on agent conversations was largest). It trained on 177 bf16 agent trajectories on SWE-bench Verified tasks (2.09 million tokens, up to 32K tokens each; assistant positions only) mixed with 300 rows (0.68 million tokens) of the 24 October data; these weights are the snapshot after 48 of the 119 steps of one epoch. 2.41 million of 190.1 million scales changed, no 3-bit code. The SWE-bench tasks used for training and the 80 test tasks are disjoint. CALIBRATION.md lists the data and its licenses.

During prefill, the rotated projections take 4-bit activations, scaled per token and per 128-element group. The other projections, and all projections during decode, take 8-bit (FP8) activations. Native RDNA4 kernels in the Paiton image run these weights. All other tensors come from the pinned target checkpoint.

Accuracy

Agentic update, 7 November 2026

SWE-bench Verified, 80 held-out tasks (random, seed 20261105, chosen without looking at outcomes; disjoint from the training tasks). mini-swe-agent 2.4.6, 250-step limit, temperature 0.7, thinking off, one R9700 with run-3bit.sh --mode long-kv4 on the 5 October image; graded with the official SWE-bench harness. Both 3-bit columns were run on the same tasks in the same period; paired per task, exact McNemar test.

	24 October weights	These weights	Paired test
Tasks solved	41	49	14 only here, 6 only 24 Oct; p 0.12, not significant
Tasks where the agent looped (the same command 3+ times in a row)	16	7	p 0.078, borderline
Tasks stopped by the 250-step limit	20	12	p 0.13
For scale: bf16 Qwen3.8-27B (datacenter GPU, same agent settings)	58

Agent runs at temperature 0.7 vary between runs: an earlier run of the 24 October weights on the same 80 tasks solved 39.

SWE-bench Verified, all 500 tasks (8 November). The same agent settings and mode, one R9700 per run: these weights on the 7 November image r2, the 24 October weights on the 5 October image (run of 3-5 November), bf16 Qwen3.8-27B on a datacenter GPU. One run each, paired per task, exact McNemar test. The held-out 250 are the second half of the task list (by instance id); none of them was used for training.

	bf16	24 October weights	These weights	These vs 24 October	These vs bf16
Solved, all 500	390	284	323	81 only here, 42 only 24 Oct; p 0.0006	25 vs 92; p < 0.0001
Solved, held-out 250	191	134	165	48 vs 17; p 0.0002	17 vs 43; p 0.001
Looped, all 500	12	89	67	p 0.03	p < 0.0001
Stopped by the step limit, all 500	8	107	69	p 0.0003

These weights solve more tasks than the 24 October weights and loop less; both differences are significant. They still solve significantly fewer tasks than bf16. The 80 tasks above, which we used to choose these weights, lie inside the held-out 250; on the other 170 these weights solved 110 against 95 (p 0.04). On the 197 training tasks the two 3-bit weights are not significantly different (137 vs 145, p 0.26). The two 3-bit runs used different images, so they compare the weights on solved tasks and loops, but not on time or energy.

Served tasks (one R9700, the 65K default mode, thinking on; these weights on the 7 November image r2, the 24 October weights on the 5 October image; paired on the same questions and seeds):

Task	bf16 (datacenter GPU)	24 October weights	These weights	Change against 24 October [95 % CI], exact McNemar p
GSM8K (1,319)	95.5 %	95.8 %	95.8 %	−0.1 [−1.0, +0.9] points, p 1.00
HumanEval (164)	95.7 %	93.3 %	97.0 %	+3.7 [+0.3, +7.0] points, p 0.07
MMLU-Pro (1,400)	81.5 %	79.3 %	79.5 %	+0.2 [−1.3, +1.7] points, p 0.85

No change against the 24 October weights is significant. Against bf16 these weights are within noise on GSM8K and HumanEval and 2.0 points lower on MMLU-Pro (p 0.02). Until 8 November the bf16 column showed the bf16 scores of Qwen3.8 Flash Next by mistake; it now shows Qwen3.8 27B, measured with the same questions and seeds. HumanEval with these weights scored 92.1 % in an earlier run on the 5 October image and 97.0 % here (sampling diverges between images on long reasoning paths), so we read it as run-to-run variance, not a gain.

Closeness to bf16 (emulated serving arithmetic, both policies; change against the 24 October weights):

Metric	Change
KL to bf16 on 20 held-out agent trajectories, prefill / decode	−3.0 % [−3.9, −2.2] / −4.1 %
KL to bf16 on general text, decode / prefill	+3.5 % [−0.1, +7.8] / +1.8 % (not significant)
Decode top-1 agreement with bf16	−0.32 points (significant)
Long documents 8K-32K, decode / prefill	+0.06 % / −0.5 %
131K-token documents, decode / prefill	−0.11 % / +0.08 %

So these weights trade a little general-text closeness for closeness on agent conversations.

Accuracy update, 24 October 2026

Each row pairs these weights with the 12 October weights (3ef7eb14) on identical inputs, with a 95 % bootstrap interval over documents, defined as in the 12 October table below. The R9700 rows come from the Paiton image served with run-3bit.sh --mode long-kv4.

Metric	12 October weights	These weights	Change [95 % CI]
R9700, served: KL to bf16 (top-20 log-probabilities, 40 texts)	0.1395	0.1353	−3.0 % [−6.4, −0.4]
R9700, served: top-1 agreement with bf16	88.14 %	88.29 %	+0.15 points [−0.13, +0.42]
R9700, served: 131K-token documents (5), log-likelihood in each length bucket	–	–	none worse, 64K+ better
KL to bf16, decode (115 held-out texts)	0.0673	0.0654	−2.9 % [−5.1, −1.0]
KL to bf16, prefill (4-bit activations)	0.0819	0.0807	−1.5 % [−4.0, +1.0]
Gap in log-likelihood to bf16, decode / prefill	0.036 / 0.046 nats	0.028 / 0.038 nats	−22 % / −17 %
Long documents (22, 8K to 32K tokens), KL decode / prefill	0.0746 / 0.0909	0.0728 / 0.0888	−2.4 / −2.2 %, every bucket better
131K-token documents (5), KL decode / prefill	0.249 / 0.282	0.246 / 0.278	−1.3 / −1.4 %, every bucket better
MMLU-Pro subset (1,400), answer log-likelihood, prefill / decode	62.93 / 63.00 %	62.36 / 63.00 %	−0.57 [−2.07, +0.93] / 0.00 points

Knowledge (−8 %), chat (−6 %) and tool-calling agents (−6 %) gain most in decode KL; code is unchanged within noise (+1 %).

GSM8K 5-shot, HumanEval and the MMLU-Pro subset, with the same prompts and seeds as below, run on a bf16 copy of each set of weights (weight-only, no activation quantisation) give 95.45 / 94.51 / 63.50 % against 95.53 / 93.90 / 62.86 % for the 12 October weights. No difference is significant (McNemar p 1.0 / 1.0 / 0.11).

Speed is unchanged: same image, launcher and kernels. With --mode long-kv4 on one R9700, BetterBench 0.6.0 weighted decode reads 169.1 against 169.3 tok/s, and 484.3 against 479.1 tok/s at eight requests. The decode step is 28.8–29.0 ms with either weights. In the default mode the weighted decode reads 155.5 against 155.9 tok/s, and 565.9 against 564.0 tok/s at eight requests. LocalMaxxing runs the same number of decode steps per second (about 34 at one request). Its single-prompt c1 figure moves with draft acceptance on that one prompt (116.7 against 125.5 tok/s in the median runs).

Accuracy update, 12 October 2026

Each row pairs the 12 October weights with the 26 September weights (7a2e3d70) on identical inputs, with a 95 % bootstrap interval over documents. KL is KL(bf16 ‖ 3-bit) of the next-token distribution, averaged over assistant and plain-text tokens; top-1 is how often the 3-bit model's most likely next token is also bf16's. The first rows reproduce the runtime's arithmetic under both activation policies on MI355X GPUs; the R9700 rows come from the Paiton image itself.

Metric	26 September weights (7a2e3d70)	12 October weights (3ef7eb14)	Change [95 % CI]
KL to bf16, decode (115 held-out chats, code, math, tool calls, documents)	0.0851	0.0673	−20.9 % [−29.8, −14.1]
KL to bf16, prefill (4-bit activations, same texts)	0.1074	0.0818	−23.8 % [−39.5, −13.1]
Top-1 agreement with bf16, decode / prefill	88.66 / 87.36 %	89.70 / 88.38 %	+1.0 / +1.0 points
Long documents (22, 8K to 32K tokens), KL decode / prefill	0.0806 / 0.0975	0.0746 / 0.0909	−7.5 / −6.8 %
the same, positions beyond 8K tokens	0.0632 / 0.0770	0.0578 / 0.0710	−8.6 / −7.9 %
131K-token documents (5), KL in each length bucket up to 128K	0.20 to 0.37	0.19 to 0.36	−2.3 to −4.0 %, every bucket better
MMLU-Pro subset (1,400), answer log-likelihood, prefill / decode; bf16: 65.29 %	61.36 / 62.43 %	62.93 / 63.00 %	about +0.5 to +1 point (see below)
R9700, served: KL to bf16 (top-20 log-probabilities, 40 texts)	0.1732	0.1395	−19.5 % [−36.8, −8.5]
R9700, served: top-1 agreement with bf16	86.90 %	88.14 %	+1.2 points [+0.6, +2.1]

No category gets worse under either policy. Chat, code and math gain the most (decode KL −41 %, −35 % and −22 %); knowledge questions (−1 % decode, −6 % prefill) and tool calls under prefill (−0.7 %) gain least, within noise.

On long context, the gain is smaller at 131K tokens than on shorter texts, but every bucket improves, from the first 8K tokens to 128K. On the R9700, the served 131K check found the new weights better or equal in every bucket.

On MMLU-Pro, this training run scores +1.57 / +0.57 points (prefill / decode policy), a second run with another seed +0.57 / +0.79 points; the paired intervals include zero. We read this as about +0.5 to +1 point of the 2.9 to 3.9 points by which the previous weights trailed bf16 in this scoring.

Speed is unchanged, as the format and kernels are the same. On one R9700 with the current launcher and image (--mode long-kv4), the decode step takes 28.7–28.9 ms and the first token 66 ms (p50) with either weights, and prefill from 2K to 64K tokens is within 0.3 %. BetterBench weighted decode reads 170.1 against 172.8 tok/s, and 482.2 against 481.9 tok/s at eight requests; the single-stream difference comes from speculative-decoding acceptance on replayed conversations, not from kernel speed.

Outputs differ from the previous weights, greedy ones included. The GSM8K, HumanEval, needle and long-session results below were measured with the 26 September weights and were not re-run for this update.

Previous weights, 26 September 2026

These results were measured in-model on 2026-09-26 with the Paiton image and the previous weights, using greedy decoding with thinking off. Each benchmark is paired against the MXFP4 image on identical items. Δ is in points with a normal-approximation 95 % CI, and p is the exact McNemar test.

Benchmark	MXFP4	Previous weights (W3A4)	Δ [95 % CI]	McNemar p
GSM8K 5-shot (1,319)	95.68 %	95.30 %	−0.38 [−1.44, +0.68]	0.58
HumanEval pass@1 (164)	95.12 %	93.90 %	−1.22 [−4.99, +2.56]	0.75
MMLU-Pro subset, 0-shot direct answer (14 × 100)	62.57 %	59.71 %	−2.86 [−4.81, −0.90]	0.005
Needle at 61,440 tokens (80)	100 %	100 %	0	1

DFlash2 mean acceptance length relative to MXFP4 changes by +0.1 % on 38 chat prompts, −0.4 % on GSM8K, +2.8 % on HumanEval, +1.8 % on MMLU-Pro and +2.0 % on the needle test.

Math, code and long-context retrieval stay within noise. Knowledge recall drops by about 3 MMLU-Pro points, which is the trade-off of 3-bit weights. The MXFP4 path (run-mxfp4.sh) remains the choice for maximum knowledge accuracy.

Long coding sessions. In 20-turn coding sessions that grow from about 100K to 181K tokens, with a new HumanEval+ task every turn (8 sessions, 160 tasks), --mode long-kv4 with the previous weights solved 87.5 % of the tasks with no decline from the first turns to the last (change per turn 0.00, 95 % interval −0.71 to +0.64), against 83.75 % for MXFP4 in --mode long. Cached turns started in 3.1 to 4.1 s and decoded code at about 210 tok/s (full table).

Performance

The benchmark is BetterBench 0.6.0 (quick preset) with the 65K release profile on one R9700, using two fresh server processes per image. The reference is the previous MXFP4 image, qwen38-rocm10-vllm029-65k-20260924-r3. These runs used the earlier v3 calibration of these weights. Speed does not depend on the calibration data or on the 12 October scale update, because the tensor format and runtime are identical. Changes are computed from unrounded values.

Metric	MXFP4	These weights	Change
Weighted decode, tok/s	153.8	184.4	+19.9 %
chat / code / file edit	121.2 / 179.9 / 179.9	135.8 / 226.0 / 195.0	+12.0 / +25.6 / +8.5 %
json / math / prose	217.5 / 183.8 / 78.5	269.0 / 228.9 / 94.7	+23.7 / +24.5 / +20.7 %
reasoning / summarization	117.8 / 138.4	133.5 / 158.4	+13.3 / +14.4 %
Aggregate at 1 / 2 / 4 / 8 requests, tok/s	122.0 / 204.2 / 308.2 / 425.3	148.8 / 249.3 / 368.3 / 492.1	+22.0 / +22.1 / +19.5 / +15.7 %
Prefill at 2K / 8K / 16K, input tok/s	3,689 / 3,834 / 3,871	4,156 / 4,165 / 4,103	+12.7 / +8.6 / +6.0 %
Prefill at 32K / 64K, input tok/s	3,751 / 3,455	3,958 / 3,629	+5.5 / +5.0 %

Time to first token shortens by the same factors as prefill throughput. Model memory falls from 19.18 GiB to 15.89 GiB, and the launcher gives the freed VRAM to the KV cache: 250,578 cached tokens instead of 174,634, or 3.8 concurrent 65K-token requests instead of 2.7. Since 28 September, the 65K mode stores that cache in 4 bits by default with these weights: vLLM reports 393,216 tokens, and six 61K-token requests run together. Its accuracy and long-session checks are in the plugin reference.

Files and verification

The repository holds 64 layer shards (w3rot-L00.safetensors … w3rot-L63.safetensors, 1,296 tensors), w3rot-index.json, w3rot-manifest.json, CALIBRATION.md, LICENSE, THIRD_PARTY_NOTICES.md and SHA256SUMS. The manifest records the format, per-shard SHA-256, b/a source hashes and calibration provenance, including 33 dataset entries with revisions and licenses. Since the 12 October update it also records the new shards' SHA-256, weights_version and a finetune block: the base revision and manifest hash, the objective, the data sources with revisions and licenses, and per-shard counts of changed values. The 24 October update records the same block against the 12 October weights (31.9 million of 190.1 million scales changed, no 3-bit code), with the 12 October block kept as previous_finetune. The 7 November update records its block against the 24 October weights (2.41 million scales changed in the 32 files of layers 32 to 63, no 3-bit code). Everything else in it is unchanged. The paths in its sources section refer to the calibration environment.

sha256sum -c SHA256SUMS
sha256sum w3rot-manifest.json   # fe158c7bcc4e64b5160f68c222702bb7a6296460212d8109f0b70295ca9af5c8

The 24 October weights' manifest (revision ae09a11f) is 04ace57455f8f54b7cd330f0cfe6741e641a63bbeaf47f43ac58a037db2eaec4; the 12 October weights' manifest (revision 3ef7eb14) is 2d1f6c312a253397ad5c293d4d7a5dc53c6e50de64e58f05f4fa50d59d211f67; the 26 September weights' manifest (revision 7a2e3d70) is 335c5c834fb3f455203908d2b5aff8131aca2fe64ad73d59feddf32247d41a70.

Limitations

• The 7 November update was tested on SWE-bench Verified (80 tasks, then all 500), GSM8K, HumanEval, MMLU-Pro and the KL sets above. Needles, long coding sessions, vision and the long modes other than --mode long-kv4 were not re-run with it.
• The weights are lossy. With the previous weights, MMLU-Pro scored about 3 points below MXFP4 (the 12 October update recovers an estimated 0.5 to 1 point). Outputs differ from the MXFP4 path, and greedy generations can diverge.
• The served task benchmarks were measured with the 26 September weights. The 12 and 24 October updates were checked with KL to bf16 on held-out texts, long documents up to 131K tokens, MMLU-Pro log-likelihood and a served check on the R9700. The 24 October update also ran GSM8K, HumanEval and MMLU-Pro on a weight-only bf16 copy. Served GSM8K, HumanEval, needles and the long coding sessions were not re-run.
• The evaluation covers GSM8K, HumanEval, a 1,400-question MMLU-Pro subset, 38 chat prompts and needle retrieval at 61,440 tokens. Multilingual and tool-calling quality are not measured. For the long modes, the plugin reports GSM8K, HumanEval and MMLU-Pro with every question placed after a 257K-token document in --mode long (95.83 / 91.46 / 60.29 against 95.30 / 93.90 / 59.71 at short context, within the paired 95 % intervals), and, in --mode long-kv4, that the 4-bit cache predicts five 131K-token documents as well as the FP8 cache. With prefix caching (--mode long-kv4), GSM8K, HumanEval, MMLU-Pro and the 61,440-token needle stay within noise of the 2 October mode; --mode long-512k stays within noise of --mode long-kv4 on the same tests and found 4 of 4 planted facts at 300K and at 500K tokens (long-mode results).
• The 7 November image r2 (and the 5 October image) serves these weights in all four modes. They cover the language model; with --vision (not in --mode long-512k), image input uses the target checkpoint's vision encoder. One R9700 only; Transformers, stock vLLM and llama.cpp cannot load them.
• The default 65K mode uses the 4-bit KV cache. The long modes use the FP8 cache (--mode long) or the 4-bit cache (--mode long-kv4 and --mode long-512k), all three with prefix caching. The MXFP4 path and other settings use the FP8 cache.
• --mode long-kv4 reads a new long prompt about 3 % slower than without prefix caching (7 to 13 % for short prompts) and keeps 2.4 GiB of system RAM pinned; a host that cannot pin it gets a 451,879-token cache instead. --mode long-512k is opt-in: its long-context scaling applies to every request, it needs that system RAM, and a 500K-token document takes about 6 minutes to read the first time.
• The weights are tied to the pinned target revision and to Paiton's rotation and activation formats.

License and attribution

These weights are released under Apache-2.0. They are a quantized derivative of Qwen/Qwen3.8-27B (Apache-2.0), and the rotated b/a rows derive from unsloth/Qwen3.8-27B-NVFP4 (Apache-2.0). The method builds on GPTQ and GSQ (Apache-2.0). The target checkpoint and the DFlash2 drafter (tcclaviger/Qwen3.8-27B-DFlash2-FP8, Apache-2.0) are not redistributed here. The calibration and scale-distillation data require these attributions:

QASC

Tushar Khot, Peter Clark, Michal Guerquin, Peter Jansen and Ashish Sabharwal (Allen Institute for AI), allenai/qasc. It is licensed under CC BY 4.0 and provided as is. We reformatted it into calibration prompts, and it is not redistributed.

SWE-bench Verified

The agentic scale distillation used bf16 agent trajectories on tasks from SWE-bench/SWE-bench_Verified (MIT); the trajectories quote code from the tasks' repositories, under those repositories' licenses (CALIBRATION.md). They are not redistributed.

ODC-By datasets

The calibration contains information from FineWeb-Edu, FineWeb-2 and FinePDFs-Edu (Hugging Face), and the scale distillation contains information from FineWeb-Edu. They are made available under the ODC Attribution License 1.0, and their use is also subject to Common Crawl's Terms of Use.

Notices and links

THIRD_PARTY_NOTICES.md has the full notices · Paiton · Public Paiton plugin