Qwen3.8-27B AlphaDflash MTP head (GGUF sidecar)
A retrained multi-token-prediction (MTP) draft head for Qwen3.8-27B, for AlphaLlama 2.0.10 and later. It replaces the draft head inside your Qwen3.8-27B GGUF at load time; the model file itself stays unchanged. Drafts are always verified by the full model, so the head changes speed only, not what the model can answer.
What it is
- The file: the 15 tensors of the model's MTP block (
blk.64.*), 351 MB, in the same tensor types asQwen3.8-27B-UD-Q4_K_M.gguf. AlphaLlama loads them in place of that file's ownblk.64.*. - What AlphaChat changed: the MTP block was retrained (fine-tuned) by AlphaChat on text generated by Qwen3.8-27B itself, so its drafts match the model's own next tokens more often. The rest of the model is untouched. The training text is mostly English; prompts in other languages gain less.
- Base model:
Qwen/Qwen3.8-27B, Apache-2.0. Tested withQwen3.8-27B-UD-Q4_K_M.gguffromunsloth/Qwen3.8-27B-GGUF. - Licence: Apache-2.0, as the base model. See
LICENSE.
| File | Size | SHA-256 |
|---|---|---|
alphadflash-27b-head.gguf |
351,009,888 bytes (351 MB) | 33e6a004d0fe9eff4fab5f6b58001c2415887c21f7167939b796f06dfa814350 |
Use with AlphaLlama
Requirements: Qwen3.8-27B (Qwen3.8-27B-UD-Q4_K_M.gguf), AlphaLlama 2.0.10 or later with an NVIDIA GPU.
- Put the file in one of these places:
- beside the model:
<folder of your 27B .gguf>/alphadflash-27b-head.gguf, or - in a sibling folder:
<parent folder>/Qwen3.8-27B-AlphaDflash/alphadflash-27b-head.gguf, or - anywhere, then point to it with
ALPHA_AI_MTP_HEAD=/path/to/alphadflash-27b-head.gguf.
- beside the model:
- Start as usual:
alphallama-serve -m Qwen3.8-27B-UD-Q4_K_M.gguf - Check that it is active. The startup log shows:
If the file isn't found, the server says so and uses the model's own head.[auto] MTP head: AlphaDflash 27B MTP head <path>, depth 7 [ai] MTP head sidecar <path>: 15 tensors (334.8 MiB) replace the model's
On an RTX 3090 or 5090, default (fast) decoding then drafts 7 tokens deep at 2 x 98,304 context instead of 3 deep at
2 x 131,072. Pass -c to choose the context yourself. On cards where AlphaLlama uses its own tuned drafting (16-20 GB
cards), the head is loaded but that tuning is kept.
Quality and speed
Quality (RTX 3090, temperature 0, thinking off; paired against plain decoding of the same model, same prompts):
| Benchmark | Plain decoding | With this head, 7 deep |
|---|---|---|
| GSM8K (CoT, flexible extract) | 96.06% | 95.91% (-0.15 pt, 95% CI -0.75 to +0.44) |
| MMLU (generative) | 83.79% | 83.79% |
| CMMLU (generative) | 81.34% | 81.34% |
| HumanEval | 156 / 164 | 158 / 164 |
--decode-mode exact with this head gives text identical to plain decoding (25 / 25 prompts).
Speed (RTX 3090, AlphaLlama 2.0.10 candidate, board sets A / V / R, 25 prompts x 2 passes, decode tok/s, geomean per set):
| A | V | R | |
|---|---|---|---|
| Default fast mode, the model's own head | 104.2 | 109.6 | 122.2 |
| Default fast mode, this head | 124.8 | 147.2 | 195.3 |
| Gain | +20% | +34% | +60% |
Speed mode (ALPHA_SPEED_MODE=1, with the DFlash2 drafter) is unchanged by this head (within 1%).
- Downloads last month
- 72
We're not able to determine the quantization variants.
Model tree for AlphaChat-dotcom/Qwen3.8-27B-AlphaDflash-GGUF
Base model
Qwen/Qwen3.8-27B