Qwen3.8-27B DFlash2 drafter (GGUF, q5_K)
A GGUF copy of the DFlash2 parallel drafter for Qwen3.8-27B, for AlphaLlama 2.0.9 and later. It is used by
speed mode (ALPHA_SPEED_MODE=1) to draft several tokens at once.
Origin and changes
- Original model:
z-lab/Qwen3.8-27B-DFlash2, revision50307d4c4cde6860d4eee73e2547cd786fe8e8a4, which is also mirrored atincoai/Qwen3.8-27B-DFlash2. Blog: inco.ai/blog/dflash2; code: github.com/z-lab/dflash. All credit for the model goes to its authors. - Licence: Apache-2.0, as declared by the original model card. See
LICENSE. - What AlphaChat changed: only the format and precision. This is not an AlphaChat-trained model: nothing was
retrained, fine-tuned, merged or pruned.
- Converted from the original BF16 safetensors to GGUF with
alphainfer/tools/convert_dflash2.py(in the AlphaLlama repository). Every tensor stays BF16 and was checked bit-exact against the source. - Quantised the projection matrices to q5_K; the selector codebooks stay BF16. Norms and conv kernels stay F32.
- Converted from the original BF16 safetensors to GGUF with
- The file contains only the drafter. It uses the embedding and output head of your own Qwen3.8-27B GGUF at run time.
| File | Size | SHA-256 |
|---|---|---|
dflash2-q5_K.gguf |
1,490,791,072 bytes (1.49 GB) | 848c4bba967fa88129206aa4c96cee703b5a706bd73bafb636adbb2f19a4d53b |
Use with AlphaLlama
Requirements:
- Qwen3.8-27B only (tested with
Qwen3.8-27B-UD-Q4_K_M.gguf); - the AlphaLlama 2.0.9+ Linux package with an NVIDIA GPU;
- tested on an RTX 3090 (24 GB). The Windows and macOS packages don't run speed mode.
- Put the file in one of these places:
- beside the model:
<folder of your 27B .gguf>/dflash2-q5_K.gguf, or - in a sibling folder:
<parent folder>/Qwen3.8-27B-DFlash2/dflash2-q5_K.gguf, or - anywhere, then point to it with
ALPHA_AI_DRAFTER=/path/to/dflash2-q5_K.gguf.
- beside the model:
- Start in speed mode:
ALPHA_SPEED_MODE=1 alphallama-serve -m Qwen3.8-27B-UD-Q4_K_M.gguf - Check that it is active. The startup log shows:
If the file isn't found, the server still starts with sequential drafting instead and says so in the log, and[auto] ALPHA_SPEED_MODE: best-speed DFlash2 drafter <path>, 1 stream -- output is NOT token-identical to plain greedy/healthreports"speed_drafter": "missing".
Things to know
- Not token-identical: output is not token-identical to plain greedy decoding. Use
--decode-mode exactorplainwithout speed mode if you need that. - One conversation at a time: speed mode with this drafter serves one stream.
- Which prompts it drafts: it drafts English / Latin-script prompts and code. Chinese prompts are routed to sequential drafting plus prompt lookup, so the drafter doesn't speed them up.
- Measured speed, for reference: RTX 3090, AlphaLlama 2.0.9, board sets A / V / R, decode tok/s. Default fast mode: 104.3 / 109.5 / 122.0. Speed mode with this drafter: 161.7 / 171.0 / 506.3.
- Downloads last month
- 95
Hardware compatibility
Log In to add your hardware
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐ Ask for provider support