StillDeadcode/qwen3.8-27b-iq8r

Qwen3.8-27B IQ8R (GPTQ int8, Hadamard-rotated) — radiance container

Qwen/Qwen3.8-27B with an int8 trunk fitted by GPTQ, each 128 of K rotated by a Hadamard transform, as a single .rad container for the radiance inference engine (AMD RDNA4, ROCm). It is radiance's IQ8R entry in the Qwen3.8-27B quantization challenge.

File	qwen3.8-27b-iq8r.rad — 27.95 GiB (text weights 28.59 GB, 8.50 bits per weight)
Weights	every trunk linear int8 (codes −127..127) with a bf16 scale per row and 128 of K, the 128 rotated by the unnormalised Hadamard; codes chosen by GPTQ against per-linear Hessians. The lm_head int8 by round to nearest
Activations	int8 under the same rotation (W8A8)
MTP head	Qwen's one-layer multi-token-prediction head, int8 by round to nearest
Vision	the 27-block vision tower, bf16: images and video in chat requests
Context	262,144 tokens trained

Quality

KL divergence from the bf16 model's next-token distribution, over the positions a model writes (assistant turns including tool calls, and plain text; not system or user turns or tool results), on scoring tier T1 of the challenge's private split (155,666 positions):

mean KLD	KLD 99%	KLD 99.9%	top-1 agreement
0.00055	0.0033	0.0232	99.18%

Serve

radiance --model qwen3.8-27b-iq8r.rad --tp 2 --max-model-len 200000 --max-num-seqs 8 \
    --host 0.0.0.0 --port 8000

The server speaks the OpenAI API (/v1/chat/completions, /v1/completions), with tool calls and structured output, and image_url / video_url parts in chat messages.

How this file was made

The Hessians are the input Gram matrices of every trunk linear, recorded by radiance running the bf16 model over the calib split of StillDeadcode/qwen3.8-27b-quant-challenge (the recording tap is not in a radiance release yet). CALIB names the directory that holds them:

CALIB=DIR rad-convert Qwen/Qwen3.8-27B --recipe q38-27b-iq8r.recipe \
    --set quantization_config.quant_method=w8a8 --set quantization_config.fmt=fwht128 \
    -o qwen3.8-27b-iq8r.rad

The recipe (q38-27b-iq8r.recipe in this repository); everything it does not name is the checkpoint's own:

blk.*.attn_qg.weight      gptq codes=i8 group=128 scale=bf16 transform=fwht128 clamp=sym rule=search calib=$CALIB
blk.*.attn_k.weight       gptq codes=i8 group=128 scale=bf16 transform=fwht128 clamp=sym rule=search calib=$CALIB
blk.*.attn_v.weight       gptq codes=i8 group=128 scale=bf16 transform=fwht128 clamp=sym rule=search calib=$CALIB
blk.*.attn_output.weight  gptq codes=i8 group=128 scale=bf16 transform=fwht128 clamp=sym rule=search calib=$CALIB
blk.*.ssm_inz.weight      gptq codes=i8 group=128 scale=bf16 transform=fwht128 clamp=sym rule=search calib=$CALIB
blk.*.ssm_out.weight      gptq codes=i8 group=128 scale=bf16 transform=fwht128 clamp=sym rule=search calib=$CALIB
blk.*.ffn_gate_up.weight  gptq codes=i8 group=128 scale=bf16 transform=fwht128 clamp=sym rule=search calib=$CALIB
blk.*.ffn_down.weight     gptq codes=i8 group=128 scale=bf16 transform=fwht128 clamp=sym rule=search calib=$CALIB
output.weight             rtn  codes=i8 group=128 scale=bf16 clamp=sym rule=search

License

Apache 2.0, as Qwen/Qwen3.8-27B; its LICENSE is in this repository.