basecompute

/

Qwen3.8-27B-BaseQuant-V1

Like 28

Follow

Base Compute 35

Adds an XL tier, the same size as unsloth UD-Q6_K_XL (25,299,061,664 B):

• Qwen3.8-27B-BaseQuant-V1-XL.gguf: 25,295,066,592 B, sha256 d76eb0892ce5d63a28b3337a6bd6a786d5fe7bff7d9a1bb4b9e9d87c05b46501. It includes the MTP layer (Q6_K), like UD-Q6_K_XL.
• Qwen3.8-27B-BaseQuant-V1-XL.base: 26,975,551,488 B, sha256 94a577b439d72ad6d66090ab57475137e567187971d42ecc3beec870a0a0b7dc. It includes the MTP head and the vision tower.
• README.md: an XL row and XL columns in the result tables. It also fixes "four sizes" to "five sizes".

Results (KL to BF16, unsloth / XL):

• Held-out model continuations: 0.00126 / 0.00089 (-30%), 95% CI of the difference [-0.00058, -0.00004]. Top-1 99.27% / 99.38%.
• Held-out written text: 0.0886 / 0.0558 (-37%), 95% CI of the difference [-0.0440, -0.0233]. Top-1 96.35% / 96.96%.
• Public prompts, all six sources: 0.00106 / 0.00088 (-17%), 95% CI [-22%, -12%]. Top-1 99.56% / 99.59%. Five of the six sources are significantly better; OpenHermes is lower but not significantly.

The held-out rows come from the XL campaign's validation split. The tool-call rows were not measured for XL. The plots were not changed.

baseRT bundle:

• 175 tensors use the GGUF's own Q6_K/Q5_K blocks, with the linear-attention V-head rows of 42 qkv/z tensors moved back into HF order.
• baseRT loads no Q8_0, so the GGUF's 184 Q8_0 tensors are stored as base_q8 (groups of 64, bf16 scale and offset), the same 8.5 bits per weight.
• 43 more tensors are also base_q8:
  • 16 linear-attention out_proj, because their GGUF blocks are in llama.cpp's column order;
  • 23 that share a fused kernel with a Q8_0 tensor;
  • 4 in layer 3, which the MTP-head loader uses as its shape reference.
• MTP head: decoder layer base_q8, fc and norms f16 (basert convert --quantize-drafter).

Tested on an M5 Max with baseRT 0.4.1:

• Over 50 public prompts (SWE-agent, OpenHands and 10 others, 7,880 positions), KL is 0.00072. The XL GGUF on the same Mac in llama.cpp (Metal, llama-cpp-python 0.3.36) scores 0.00069 on the same prompts. Top-1 is 99.78% / 99.87%.
• The same GGUF scores 0.00098 in llama.cpp on CUDA. At this KL level the backend's arithmetic is a large share of the total, so compare only numbers measured on the same backend.
• With superfluid 0.1.4, greedy generations are correct and identical with --speculate mtp-head --spec-bitexact and without it.

Card additions (second commit): a Recommended sampling section (Qwen's settings for thinking and instruct modes), and a Speed table giving prefill (pp512) and decode (tg128) tokens per second for every size, GGUF in llama.cpp b10360 against bundle in baseRT 0.4.1, on an M5 Max.