Qwen3.8-27B DFlash2 drafter (GGUF, q5_K)

A GGUF copy of the DFlash2 parallel drafter for Qwen3.8-27B, for AlphaLlama 2.0.9 and later. It is used by speed mode (ALPHA_SPEED_MODE=1) to draft several tokens at once.

Origin and changes

  • Original model: z-lab/Qwen3.8-27B-DFlash2, revision 50307d4c4cde6860d4eee73e2547cd786fe8e8a4, which is also mirrored at incoai/Qwen3.8-27B-DFlash2. Blog: inco.ai/blog/dflash2; code: github.com/z-lab/dflash. All credit for the model goes to its authors.
  • Licence: Apache-2.0, as declared by the original model card. See LICENSE.
  • What AlphaChat changed: only the format and precision. This is not an AlphaChat-trained model: nothing was retrained, fine-tuned, merged or pruned.
    1. Converted from the original BF16 safetensors to GGUF with alphainfer/tools/convert_dflash2.py (in the AlphaLlama repository). Every tensor stays BF16 and was checked bit-exact against the source.
    2. Quantised the projection matrices to q5_K; the selector codebooks stay BF16. Norms and conv kernels stay F32.
  • The file contains only the drafter. It uses the embedding and output head of your own Qwen3.8-27B GGUF at run time.
File Size SHA-256
dflash2-q5_K.gguf 1,490,791,072 bytes (1.49 GB) 848c4bba967fa88129206aa4c96cee703b5a706bd73bafb636adbb2f19a4d53b

Use with AlphaLlama

Requirements:

  • Qwen3.8-27B only (tested with Qwen3.8-27B-UD-Q4_K_M.gguf);
  • the AlphaLlama 2.0.9+ Linux package with an NVIDIA GPU;
  • tested on an RTX 3090 (24 GB). The Windows and macOS packages don't run speed mode.
  1. Put the file in one of these places:
    • beside the model: <folder of your 27B .gguf>/dflash2-q5_K.gguf, or
    • in a sibling folder: <parent folder>/Qwen3.8-27B-DFlash2/dflash2-q5_K.gguf, or
    • anywhere, then point to it with ALPHA_AI_DRAFTER=/path/to/dflash2-q5_K.gguf.
  2. Start in speed mode:
    ALPHA_SPEED_MODE=1 alphallama-serve -m Qwen3.8-27B-UD-Q4_K_M.gguf
    
  3. Check that it is active. The startup log shows:
    [auto] ALPHA_SPEED_MODE: best-speed DFlash2 drafter <path>, 1 stream -- output is NOT token-identical to plain greedy
    
    If the file isn't found, the server still starts with sequential drafting instead and says so in the log, and /health reports "speed_drafter": "missing".

Things to know

  • Not token-identical: output is not token-identical to plain greedy decoding. Use --decode-mode exact or plain without speed mode if you need that.
  • One conversation at a time: speed mode with this drafter serves one stream.
  • Which prompts it drafts: it drafts English / Latin-script prompts and code. Chinese prompts are routed to sequential drafting plus prompt lookup, so the drafter doesn't speed them up.
  • Measured speed, for reference: RTX 3090, AlphaLlama 2.0.9, board sets A / V / R, decode tok/s. Default fast mode: 104.3 / 109.5 / 122.0. Speed mode with this drafter: 161.7 / 171.0 / 506.3.
Downloads last month
95
GGUF
Model size
2B params
Architecture
dflash2
Hardware compatibility
Log In to add your hardware
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for AlphaChat-dotcom/Qwen3.8-27B-DFlash2-GGUF

Base model

Qwen/Qwen3.8-27B
Quantized
(24)
this model