Qwen3.8-27B-Quark-DFlash2

A DFlash2 speculative-decoding draft model for Qwen3.8-27B, trained with AMD Quark.

• Target model: Qwen3.8-27B (hybrid; served with --language-model-only)
• Training data: the whole Tulu-3 SFT mixture, thinking disabled, shuffled, 3 epochs (17,118 steps)
• This checkpoint: end of epoch 3
• Draft architecture: 5 non-causal layers, hidden 5120, intermediate 8192, 40 heads / 8 KV heads, head dim 128, block size 8 (7 speculative tokens), grouped conv (kernel 2, group 16), candidate selector (rank 256, top-k 16)
• Target layers tapped: 3, 15, 31, 47, 59. Mask token id: 248077
• Parameters: about 1.27B, bfloat16

Usage with vLLM

Serve with the speculative method dflash and num_speculative_tokens = 7:

vllm serve Qwen/Qwen3.8-27B --language-model-only \
  --speculative-config '{"method": "dflash", "model": "ZachWangAMD/Qwen3.8-27B-Quark-DFlash2", "num_speculative_tokens": 7}'

This needs a vLLM build with DFlash2 support (DFlash2DraftModel).

Spec-Bench (K=7)

Metric	Value
Acceptance length (AL)	3.67
Speedup vs. target alone	2.31x

Per-position acceptance: position 0 = 0.781, position 1 = 0.568, position 2 = 0.415, position 3 = 0.312.

The conv layers are part of the trained weights. Setting conv_kernel_size to 1 at inference breaks the draft (AL falls to 1.79), so do not change it.