Qwen3.8-27B AlphaDflash MTP head (GGUF sidecar)

A retrained multi-token-prediction (MTP) draft head for Qwen3.8-27B, for AlphaLlama 2.0.10 and later. It replaces the draft head inside your Qwen3.8-27B GGUF at load time; the model file itself stays unchanged. Drafts are always verified by the full model, so the head changes speed only, not what the model can answer.

What it is

  • The file: the 15 tensors of the model's MTP block (blk.64.*), 351 MB, in the same tensor types as Qwen3.8-27B-UD-Q4_K_M.gguf. AlphaLlama loads them in place of that file's own blk.64.*.
  • What AlphaChat changed: the MTP block was retrained (fine-tuned) by AlphaChat on text generated by Qwen3.8-27B itself, so its drafts match the model's own next tokens more often. The rest of the model is untouched. The training text is mostly English; prompts in other languages gain less.
  • Base model: Qwen/Qwen3.8-27B, Apache-2.0. Tested with Qwen3.8-27B-UD-Q4_K_M.gguf from unsloth/Qwen3.8-27B-GGUF.
  • Licence: Apache-2.0, as the base model. See LICENSE.
File Size SHA-256
alphadflash-27b-head.gguf 351,009,888 bytes (351 MB) 33e6a004d0fe9eff4fab5f6b58001c2415887c21f7167939b796f06dfa814350

Use with AlphaLlama

Requirements: Qwen3.8-27B (Qwen3.8-27B-UD-Q4_K_M.gguf), AlphaLlama 2.0.10 or later with an NVIDIA GPU.

  1. Put the file in one of these places:
    • beside the model: <folder of your 27B .gguf>/alphadflash-27b-head.gguf, or
    • in a sibling folder: <parent folder>/Qwen3.8-27B-AlphaDflash/alphadflash-27b-head.gguf, or
    • anywhere, then point to it with ALPHA_AI_MTP_HEAD=/path/to/alphadflash-27b-head.gguf.
  2. Start as usual:
    alphallama-serve -m Qwen3.8-27B-UD-Q4_K_M.gguf
    
  3. Check that it is active. The startup log shows:
    [auto] MTP head: AlphaDflash 27B MTP head <path>, depth 7
    [ai] MTP head sidecar <path>: 15 tensors (334.8 MiB) replace the model's
    
    If the file isn't found, the server says so and uses the model's own head.

On an RTX 3090 or 5090, default (fast) decoding then drafts 7 tokens deep at 2 x 98,304 context instead of 3 deep at 2 x 131,072. Pass -c to choose the context yourself. On cards where AlphaLlama uses its own tuned drafting (16-20 GB cards), the head is loaded but that tuning is kept.

Quality and speed

Quality (RTX 3090, temperature 0, thinking off; paired against plain decoding of the same model, same prompts):

Benchmark Plain decoding With this head, 7 deep
GSM8K (CoT, flexible extract) 96.06% 95.91% (-0.15 pt, 95% CI -0.75 to +0.44)
MMLU (generative) 83.79% 83.79%
CMMLU (generative) 81.34% 81.34%
HumanEval 156 / 164 158 / 164

--decode-mode exact with this head gives text identical to plain decoding (25 / 25 prompts).

Speed (RTX 3090, AlphaLlama 2.0.10 candidate, board sets A / V / R, 25 prompts x 2 passes, decode tok/s, geomean per set):

A V R
Default fast mode, the model's own head 104.2 109.6 122.2
Default fast mode, this head 124.8 147.2 195.3
Gain +20% +34% +60%

Speed mode (ALPHA_SPEED_MODE=1, with the DFlash2 drafter) is unchanged by this head (within 1%).

Downloads last month
72
GGUF
Model size
0.4B params
Architecture
qwen35
Hardware compatibility
Log In to add your hardware

We're not able to determine the quantization variants.

Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for AlphaChat-dotcom/Qwen3.8-27B-AlphaDflash-GGUF

Base model

Qwen/Qwen3.8-27B
Quantized
(1,907)
this model