StillDeadcode/qwen3.8-next-flash-iqr-v4

Qwen3.8-Flash-Next, IQR-V4 — radiance container

Qwen/Qwen3.8-Flash-Next quantised from its bf16 checkpoint into a single .rad container for the radiance inference engine (AMD RDNA4, ROCm), with the model's MTP head for speculative decoding and its vision tower. It follows qwen3.8-next-flash-fp8-iq4r-moe: closer to the bf16 model and a shorter decode step on the same cards.

Engine version. The container needs a radiance build newer than 1.17.0. No release reads it at the time of upload.

File	qwen3.8-next-flash-iqr-v4.rad — 113.3 GiB
Routed experts, gate_up	4-bit trellis codes after a 128-point Walsh–Hadamard rotation, chosen by GPTQ on Hessians recorded from the bf16 model
Routed experts, down	4-bit codes into a 16-level non-uniform codebook after the same rotation, one E4M3 scale per 64 weights, chosen by GPTQ and refined by three sweeps of coordinate descent
Protected experts	ten experts that carry most of their layer's down-projection energy, kept bf16
Trunk	the attention's query-gate and the delta net's input projection as 5-bit trellis codes (GPTQ); the other attention, delta-net and shared-expert linears and the n-gram layer's projections rotated int8, a scale per 128 columns searched for least error; the lm_head int8; hyper-connection mixing matrices E4M3
Speculator	the model's MTP head (depth 3), its block as 3- and 4-bit trellis codes beside codebook experts
Vision	the vision tower, bf16: images and video in chat requests
Context	262,144 tokens trained; served at 200K

Quality

Full-vocabulary KL divergence against the bf16 model, teacher-forced over 83,053 positions of chat, tool-use and code transcripts, 66,187 of them written by the model. Both containers were measured by the same engine build on two cards with the exact cross-card wire:

	mean KL	median	p99	top-1 agreement
this container, all positions	0.0680	0.0031	1.43	93.38%
this container, model-written positions	0.0224	0.0025	0.29	94.93%
qwen3.8-next-flash-fp8-iq4r-moe, all positions	0.0730	0.0035	1.49	93.04%
qwen3.8-next-flash-fp8-iq4r-moe, model-written positions	0.0250	0.0029	0.34	94.54%

With the lossy cross-card wire of the serve settings below (tp-wire: wht6) the model-written mean is 0.0242 for this container and 0.0279 for the earlier one.

Serve

The routed experts do not fit two 32 GB cards; the engine keeps the hottest in VRAM and streams the rest from a pinned host pool, so the host needs about 32 GB of free RAM. The n-gram table is read from the container on disk as tokens need it, so the container belongs on an NVMe drive.

# radiance.yaml -- run with: radiance --config radiance.yaml
server:
  host: 0.0.0.0
  port: 8000
memory:
  gpu-headroom-mib: 96
models:
  flashnext:
    model: qwen3.8-next-flash-iqr-v4.rad
    tp: 2
    tp-wire: wht6
    max-num-seqs: 8
    max-model-len: 200000
    placement: expert_tiered
    host-pool-mib: 12288
    kv-cache-dtype: i8
    prefix-cache-mib: 4096
    num-speculative-tokens: 3
    max-num-batched-tokens: 2048

On 2× Radeon AI PRO R9700 (gfx1201) with these settings, one stream: about 12.5 ms per decode step at ~2.4 accepted tokens per step (13.2 ms behind a 1K-token prompt, 15.2 ms behind 178K), four streams 23 ms per step at ~9.3 tokens, eight streams 32 ms at ~17.7 tokens, and ~6,700 tokens/s prefill on a 24K-token prompt. draft-window: adaptive lets the engine shorten the draft per sequence and measured about 1.5% less time per token on average.

The server speaks the OpenAI API (/v1/chat/completions, /v1/completions), with tool calls, structured output, and image_url / video parts in chat messages.

How this file was made

CALIB=calib/w4nl-calib rad-convert Qwen/Qwen3.8-Flash-Next \
    --recipe qwen4exp-iqr-v4.recipe -o qwen3.8-next-flash-iqr-v4.rad

The recipe (qwen4exp-iqr-v4.recipe in this repository). $CALIB is the calibration data of the codebook experts and calib/qc-fn-bf16 the Hessians recorded from the bf16 model over 3.4M tokens of chat, tool-use and code documents; neither is published. The first rule that matches a weight decides it, and everything no rule names is the checkpoint's bf16:

blk.*.ple_key.weight rtn codes=i8 group=128 scale=bf16 transform=fwht128 clamp=sym rule=search
blk.*.ple_value.weight rtn codes=i8 group=128 scale=bf16 transform=fwht128 clamp=sym rule=search
blk.48.attn_qg.weight gptq codes=u3 tcq=16 tcq_gain=1.08 tcq_code=mad tcq_rows=256 block=1x* scale=f32 transform=fwht128 damp=0.01 tcq_order=1
blk.48.attn_k.weight gptq codes=u3 tcq=16 tcq_gain=1.08 tcq_code=mad tcq_rows=256 block=1x* scale=f32 transform=fwht128 damp=0.01 tcq_order=1
blk.48.attn_v.weight gptq codes=u3 tcq=16 tcq_gain=1.08 tcq_code=mad tcq_rows=256 block=1x* scale=f32 transform=fwht128 damp=0.01 tcq_order=1
blk.48.attn_output.weight gptq codes=u4 tcq=16 tcq_code=mad tcq_rows=256 group=128 scale=bf16 dyn=1 transform=fwht128 damp=0.01 tcq_gain=1.09
blk.48.ffn_gate_up_shexp.weight gptq codes=u3 tcq=16 tcq_gain=1.08 tcq_code=mad tcq_rows=128 block=1x* scale=f32 transform=fwht128 damp=0.01 tcq_order=1
blk.48.ffn_*_exps.*.weight gptq table=w4nl group=64 scale=fp8_e4m3 scale2=f32 block2=*x* scale2_value=0.0001220703125 transform=fwht128 rule=search cd=3 calib=$CALIB
blk.*.attn_qg.weight gptq codes=u5 tcq=16 tcq_code=mad tcq_rows=256 block=1x* scale=f32 transform=fwht128 tcq_gain=1.16 damp=0.01 calib=calib/qc-fn-bf16 tcq_order=1
blk.*.attn_k.weight rtn codes=i8 group=128 scale=bf16 transform=fwht128 clamp=sym rule=search
blk.*.attn_v.weight rtn codes=i8 group=128 scale=bf16 transform=fwht128 clamp=sym rule=search
blk.*.attn_output.weight rtn codes=i8 group=128 scale=bf16 transform=fwht128 clamp=sym rule=search
blk.*.ssm_inz.weight gptq codes=u5 tcq=16 tcq_code=mad tcq_rows=256 block=1x* scale=f32 transform=fwht128 tcq_gain=1.16 damp=0.01 calib=calib/qc-fn-bf16 tcq_order=1
blk.*.ssm_out.weight rtn codes=i8 group=128 scale=bf16 transform=fwht128 clamp=sym rule=search
blk.*.ffn_gate_up_shexp.weight rtn codes=i8 group=128 scale=bf16 transform=fwht128 clamp=sym rule=search
blk.*.ffn_down_shexp.weight rtn codes=i8 group=128 scale=bf16 transform=fwht128 clamp=sym rule=search
blk.34.ffn_*_exps.407.weight cast dtype=bf16
blk.34.ffn_*_exps.496.weight cast dtype=bf16
blk.44.ffn_*_exps.292.weight cast dtype=bf16
blk.44.ffn_*_exps.350.weight cast dtype=bf16
blk.46.ffn_*_exps.290.weight cast dtype=bf16
blk.46.ffn_*_exps.392.weight cast dtype=bf16
blk.47.ffn_*_exps.122.weight cast dtype=bf16
blk.47.ffn_*_exps.143.weight cast dtype=bf16
blk.47.ffn_*_exps.399.weight cast dtype=bf16
blk.47.ffn_*_exps.445.weight cast dtype=bf16
blk.*.ffn_gate_up_exps.*.weight gptq codes=u4 tcq=16 tcq_code=mad tcq_rows=128 block=1x* scale=f32 transform=fwht128 tcq_gain=1.12 damp=0.01 calib=calib/qc-fn-bf16 tcq_order=1
blk.*.ffn_down_exps.*.weight gptq table=w4nl group=64 scale=fp8_e4m3 scale2=f32 block2=*x* scale2_value=0.0001220703125 transform=fwht128 rule=search cd=3 calib=$CALIB
*_hc_down.weight rtn codes=fp8_e4m3 group=128 scale=f32
*_hc_up.weight rtn codes=fp8_e4m3 group=80 scale=f32
mtp.fc_hidden.weight rtn codes=fp8_e4m3 group=128 scale=f32
mtp.fc_embedding.weight rtn codes=fp8_e4m3 group=128 scale=f32
output.weight rtn codes=i8 group=128 scale=bf16 clamp=sym rule=search
blk.*.ple_ngram.weight rtn codes=fp8_e4m3 block=*x* scale=bf16 rule=fixed scale_value=1.99317932128906e-4
mtp.draft_head.weight rtn codes=u2 zero=u8 group=128 scale=f16

License

The Qwen community license of the base model; see LICENSE.