Qwen3.8-27B-MMFlash

MMFlash draft model for speculative decoding of Qwen/Qwen3.8-27B on image and text inputs. It is not a standalone language model; it is paired with the target model in a speculative-decoding server (SGLang DFLASH).

MMFlash builds on DFlash block-diffusion drafting: the draft reads hidden states from five layers of the target and proposes a block of 16 tokens in one forward pass, and the target verifies the block in one forward pass. MMFlash keeps the DFlash architecture and is trained on multimodal data with a loss weighted by per-token visual-grounding (visual-KL) scores.

_{Qwen3.5-9B on a MathVista question, recorded in real time on one NVIDIA H200 with SGLang (greedy decoding).}

Need a drafter for another model? Email us at fscnkucs@gmail.com and we can help train one. We will keep updating these releases, and a Qwen4 checkpoint is planned.

Model details

Target model	Qwen/Qwen3.8-27B
Parameters	1.73B (bf16)
Block size	16
Target hidden states	layers 1, 16, 31, 46, 61 (of 64)

Usage

SGLang

SGLANG_ENABLE_OVERLAP_PLAN_STREAM=1 python -m sglang.launch_server \
  --model-path Qwen/Qwen3.8-27B \
  --speculative-algorithm DFLASH \
  --speculative-draft-model-path FSCCS/Qwen3.8-27B-MMFlash \
  --speculative-dflash-block-size 16 \
  --attention-backend fa3 \
  --mm-attention-backend sdpa \
  --disable-radix-cache \
  --max-running-requests 16 \
  --reasoning-parser qwen3 \
  --trust-remote-code

--max-running-requests 16 is required with --disable-radix-cache: SGLang then reserves about 2.3 GB of GDN verification states per allowed request at startup, and the default of 48 requests does not fit on one H200.

vLLM

vllm serve Qwen/Qwen3.8-27B \
  --dtype bfloat16 \
  --max-model-len 32768 \
  --attention-backend FLASH_ATTN \
  --reasoning-parser qwen3 \
  --speculative-config '{"method": "dflash", "model": "FSCCS/Qwen3.8-27B-MMFlash", "num_speculative_tokens": 15}'

Tested with vLLM 0.32.0, which serves DFlash drafts natively. num_speculative_tokens is the block size minus one.

Results

Each cell is τ / speedup: τ is the average acceptance length (tokens accepted per verification step), and the speedup is the decoding throughput over Qwen3.8-27B without speculative decoding.

Temperature 0

Method	ChartQA	MMStar	DynaMath
MTP (15)	6.36 / 2.47×	5.25 / 2.16×	6.63 / 2.68×
MMFlash (16)	6.37 / 4.18×	5.36 / 3.54×	7.57 / 4.71×

Values in parentheses are the number of draft steps for MTP and the block size for MMFlash. MTP is Qwen3.8-27B's built-in multi-token-prediction head.

Concurrency (temperature 0)

MMFlash at 1, 8 and 32 requests in flight, on a single NVIDIA H200. Each cell is the decode throughput in tokens/s, with the speedup over Qwen3.8-27B alone at the same concurrency in parentheses.

Concurrency	ChartQA	CharXiv	MMStar	MMBench	DynaMath	MathVista	MathVerse	Avg.
1	277 (4.18×)	286 (4.33×)	234 (3.54×)	209 (3.16×)	311 (4.71×)	278 (4.20×)	296 (4.48×)	270 (4.09×)
8	762 (1.86×)	907 (2.22×)	927 (2.14×)	606 (1.43×)	1,300 (3.01×)	1,032 (2.39×)	1,169 (2.66×)	958 (2.24×)
32	932 (1.15×)	1,320 (1.40×)	1,211 (1.11×)	946 (1.48×)	1,627 (1.59×)	1,405 (1.32×)	1,628 (1.22×)	1,295 (1.32×)

For Qwen3.8-27B, concurrency 8 and 32 were served with --disable-radix-cache --mem-fraction-static 0.9. With the radix cache, SGLang keeps five state slots per request, and the MMFlash server held only 6 requests. At 32 in flight, the MMFlash server still holds at most 24 requests, because the GDN verification states take about 2.3 GB per request; the target alone holds 32.

Citation

Coming soon.