Qwen3.5-4B-MMFlash

MMFlash draft model for speculative decoding of Qwen/Qwen3.5-4B on image and text inputs. It is not a standalone language model; it is paired with the target model in a speculative-decoding server (SGLang DFLASH).

MMFlash builds on DFlash block-diffusion drafting: the draft reads hidden states from five layers of the target and proposes a block of 16 tokens in one forward pass, and the target verifies the block in one forward pass. MMFlash keeps the DFlash architecture and is trained on multimodal data with a loss weighted by per-token visual-grounding (visual-KL) scores.

_{Qwen3.5-9B on a MathVista question, recorded in real time on one NVIDIA H200 with SGLang (greedy decoding).}

Need a drafter for another model? Email us at fscnkucs@gmail.com and we can help train one. We will keep updating these releases, and a Qwen4 checkpoint is planned.

Model details

Target model	Qwen/Qwen3.5-4B
Parameters	537M (bf16)
Block size	16
Target hidden states	layers 1, 8, 15, 22, 29 (of 32)

Usage

SGLang

SGLANG_ENABLE_OVERLAP_PLAN_STREAM=1 python -m sglang.launch_server \
  --model-path Qwen/Qwen3.5-4B \
  --speculative-algorithm DFLASH \
  --speculative-draft-model-path FSCCS/Qwen3.5-4B-MMFlash \
  --speculative-dflash-block-size 16 \
  --attention-backend fa3 \
  --mm-attention-backend sdpa \
  --disable-radix-cache \
  --reasoning-parser qwen3 \
  --trust-remote-code

vLLM

vllm serve Qwen/Qwen3.5-4B \
  --dtype bfloat16 \
  --max-model-len 32768 \
  --attention-backend FLASH_ATTN \
  --reasoning-parser qwen3 \
  --speculative-config '{"method": "dflash", "model": "FSCCS/Qwen3.5-4B-MMFlash", "num_speculative_tokens": 15}'

Tested with vLLM 0.32.0, which serves DFlash drafts natively. num_speculative_tokens is the block size minus one.

Results

Each cell is τ / speedup: τ is the average acceptance length (tokens accepted per verification step), and the speedup is the decoding throughput over Qwen3.5-4B without speculative decoding.

Temperature 0

Method	ChartQA	CharXiv	MMStar	MMBench	DynaMath	MathVista	MathVerse	Avg.
MTP (3)	3.18 / 1.69×	3.27 / 1.76×	2.95 / 1.61×	2.43 / 1.40×	3.27 / 1.69×	3.18 / 1.68×	3.17 / 1.69×	3.06 / 1.64×
MTP (7)	4.26 / 1.68×	4.64 / 1.87×	3.75 / 1.57×	2.58 / 1.24×	4.42 / 1.69×	4.29 / 1.67×	4.27 / 1.69×	4.03 / 1.63×
MTP (15)	4.67 / 1.26×	5.37 / 1.43×	3.99 / 1.14×	2.60 / 0.87×	4.76 / 1.23×	4.65 / 1.21×	4.61 / 1.22×	4.38 / 1.19×
EAGLE-3 (16)	4.31 / 1.74×	4.38 / 1.86×	3.49 / 1.49×	2.61 / 1.08×	4.29 / 1.67×	4.02 / 1.56×	3.95 / 1.68×	3.86 / 1.58×
DFlash-z (16)	4.92 / 2.76×	5.63 / 3.10×	3.81 / 2.35×	5.01 / 2.62×	4.82 / 2.56×	4.43 / 2.45×	4.34 / 2.50×	4.71 / 2.62×
DFlash (16)	5.43 / 2.97×	5.60 / 3.13×	4.05 / 2.31×	5.17 / 2.93×	5.57 / 2.57×	5.16 / 2.53×	4.84 / 2.62×	5.12 / 2.72×
MMFlash (16)	5.97 / 3.28×	6.17 / 3.46×	4.49 / 2.55×	5.68 / 3.19×	6.26 / 2.87×	6.00 / 2.78×	5.35 / 2.88×	5.70 / 3.00×

Values in parentheses are the number of draft steps for MTP, the draft tree size for EAGLE-3, and the block size for DFlash and MMFlash. MTP is Qwen3.5-4B's built-in multi-token-prediction head. DFlash-z is the public z-lab/Qwen3.5-4B-DFlash draft. DFlash is our DFlash draft trained on the same data without the visual-KL weighting.

Concurrency (temperature 0)

MMFlash at 1, 8 and 32 requests in flight, on a single NVIDIA H200. Each cell is the decode throughput in tokens/s, with the speedup over Qwen3.5-4B alone at the same concurrency in parentheses.

Concurrency	ChartQA	CharXiv	MMStar	MMBench	DynaMath	MathVista	MathVerse	Avg.
1	783 (3.28×)	826 (3.46×)	607 (2.55×)	772 (3.19×)	685 (2.87×)	664 (2.78×)	686 (2.88×)	718 (3.00×)
8	2,937 (2.23×)	3,102 (2.37×)	2,347 (1.73×)	1,451 (2.16×)	3,081 (2.12×)	1,963 (1.60×)	2,956 (2.02×)	2,548 (2.03×)
32	3,799 (1.59×)	4,395 (1.61×)	3,296 (1.19×)	2,364 (1.35×)	4,643 (1.35×)	2,654 (1.16×)	4,237 (1.30×)	3,627 (1.37×)

Citation

Coming soon.