Qwen3.5-4B-DFlash2

DFlash2 speculative drafter for Qwen/Qwen3.5-4B. One recipe, one checkpoint: the end-of-training drafter (epoch 6 of 6), selected at the serving temperature.

Acceptance (τ — interval-weighted, 20-prompt chat ladder; GB10 serving)

config	τ, greedy (t=0)	τ, t=0.7 (serving)	τ, t=1.0
Qwen3.5-4B-W4A16-GPTQ verifier (training verifier)	4.05 (n=2)	3.79 (n=4: 3.71–3.85)	3.25 (n=2)
Qwen3.5-4B bf16 base	—	3.75	3.32
native MTP k=3 (baseline, same ladder)	2.99	2.86	2.66

Beats the model's native MTP by ~+1 τ at every temperature in every measured config, and drafts against the bf16 base as well as against the W4A16-GPTQ quant it was trained on.

Measurement honesty: τ depends on sampling temperature and carries run-to-run spread (t=1.0 is heavy-tailed: 0.34 observed on identical inputs; t=0.7 spread ~0.1). Numbers above are reported with n and ranges, never as single draws. Ranking is temperature-dependent — see Selection.

Selection

All six epochs' checkpoints were ranked at three temperatures with repeats. At the serving point (t=0.7 — the same sampling config the training corpus was generated with, temp 0.7 / top_p 0.95 / top_k 20), τ rises monotonically to the end of training: 3.28 → 3.49 → 3.68 → 3.79 (n=4 on the top two). The shipped artifact is the end-of-training checkpoint — no early stop.

Under greedy decoding the picture inverts: a mid-training snapshot (epoch 4) peaks at 4.20 vs end-of-training 4.05. Greedy-heavy workloads (agentic, extraction) may prefer an earlier snapshot; for sampled serving the end-of-training drafter is the measured winner.

Category profile (RedHatAI speculator_benchmarks, temp 0, n=2)

subset	MTP k=3	this drafter	Δ
qa	2.94	3.61	+0.67
HumanEval	3.35	4.85	+1.51
math_reasoning	3.51	6.64	+3.13
tool_call	3.24	4.65	+1.42
rag	3.35	5.03	+1.68
question	3.12	4.47	+1.36
writing	3.11	4.47	+1.36
summarization	3.15	4.31	+1.16
translation	2.96	3.04	+0.08
weighted	3.21	4.65	+1.43
Beats native MTP on every category; weakest margin on translation
(consistent with a chat-heavy training corpus), strongest on math, code,
and rag.

Recipe

• Corpus: 260M tokens — 100k Nemotron-Cascade-2 conversations with assistant turns regenerated by the deployed W4A16-GPTQ quant at the serving sampling config (t=0.7 / top_p 0.95 / top_k 20, seeded), so the drafter learns exactly what its verifier emits where it matters; 6-tap hidden states
• Training: lr 6e-4, cosine schedule, 4% warmup, 6 epochs; block 16 / 15 spec tokens
• Data lineage: slice ids frozen in warp-3090-quant-lab-frozen-data; corpus rebuildable via the warp-3090-quant-lab pipeline

Requirements

speculators 0.8.1 · vllm 0.31.0 · transformers 5.16.1 · torch 2.13.0 (compressed-tensors 0.17.0). Ships config.py — load with trust in remote code. If speculative serving wedges during CUDA-graph capture, serve with --enforce-eager.

Integrity

model.safetensors sha256 c919a019f84c8be86fcfc24a3e694dd2e90f4f616ac6aa103f27f5308b5f5e6c