--- license: apache-2.0 base_model: - Qwen/Qwen3.5-9B inference: false tags: - mmflash - dflash - speculative-decoding - speculative-decoding-draft - block-diffusion - draft-model - multimodal - sglang - qwen - qwen3.5 ---

# Qwen3.5-9B-MMFlash MMFlash draft model for speculative decoding of [`Qwen/Qwen3.5-9B`](https://huggingface.co/Qwen/Qwen3.5-9B) on image and text inputs. It is not a standalone language model; it is paired with the target model in a speculative-decoding server (SGLang `DFLASH`). MMFlash builds on [DFlash](https://github.com/z-lab/dflash) block-diffusion drafting: the draft reads hidden states from five layers of the target and proposes a block of 16 tokens in one forward pass, and the target verifies the block in one forward pass. MMFlash keeps the DFlash architecture and is trained on multimodal data with a loss weighted by per-token visual-grounding (visual-KL) scores.

_{Qwen3.5-9B on a MathVista question, recorded in real time on one NVIDIA H200 with SGLang (greedy decoding).}

> [!TIP] > Need a drafter for another model? Email us at [fscnkucs@gmail.com](mailto:fscnkucs@gmail.com) and we can help train one. We will keep updating these releases, and a Qwen4 checkpoint is planned. ## Model details | | | |---|---| | Target model | [`Qwen/Qwen3.5-9B`](https://huggingface.co/Qwen/Qwen3.5-9B) | | Parameters | 1.05B (bf16) | | Block size | 16 | | Target hidden states | layers 1, 8, 15, 22, 29 (of 32) | ## Usage ### SGLang ```bash SGLANG_ENABLE_OVERLAP_PLAN_STREAM=1 python -m sglang.launch_server \ --model-path Qwen/Qwen3.5-9B \ --speculative-algorithm DFLASH \ --speculative-draft-model-path FSCCS/Qwen3.5-9B-MMFlash \ --speculative-dflash-block-size 16 \ --attention-backend fa3 \ --mm-attention-backend sdpa \ --disable-radix-cache \ --reasoning-parser qwen3 \ --trust-remote-code ``` ### vLLM ```bash vllm serve Qwen/Qwen3.5-9B \ --dtype bfloat16 \ --max-model-len 32768 \ --attention-backend FLASH_ATTN \ --reasoning-parser qwen3 \ --speculative-config '{"method": "dflash", "model": "FSCCS/Qwen3.5-9B-MMFlash", "num_speculative_tokens": 15}' ``` Tested with vLLM 0.32.0, which serves DFlash drafts natively. `num_speculative_tokens` is the block size minus one. ## Results Each cell is **τ / speedup**: τ is the average acceptance length (tokens accepted per verification step), and the speedup is the decoding throughput over Qwen3.5-9B without speculative decoding. ### Temperature 0 | Method | ChartQA | CharXiv | MMStar | MMBench | DynaMath | MathVista | MathVerse | Avg. | |---|---|---|---|---|---|---|---|---| | MTP (3) | 3.32 / 1.83× | 3.28 / 1.87× | 3.01 / 1.73× | 2.88 / 1.57× | 3.38 / 1.80× | 3.26 / 1.80× | 3.24 / 1.80× | 3.20 / 1.77× | | MTP (7) | 4.66 / 1.88× | 4.60 / 1.94× | 3.91 / 1.66× | 3.55 / 1.41× | 4.74 / 1.77× | 4.52 / 1.78× | 4.43 / 1.77× | 4.34 / 1.74× | | MTP (15) | 5.27 / 1.38× | 5.27 / 1.47× | 4.20 / 1.18× | 3.75 / 0.99× | 5.25 / 1.30× | 5.08 / 1.32× | 4.90 / 1.30× | 4.81 / 1.27× | | EAGLE-3 (16) | 3.93 / 1.81× | 3.85 / 1.93× | 3.06 / 1.55× | 2.72 / 1.32× | 3.60 / 1.76× | 3.50 / 1.63× | 3.48 / 1.75× | 3.45 / 1.68× | | DFlash-z (16) | 5.22 / 3.03× | 5.27 / 3.15× | 3.93 / 2.46× | 3.67 / 2.11× | 4.96 / 2.75× | 4.77 / 2.75× | 4.52 / 2.67× | 4.62 / 2.70× | | DFlash (16) | 5.59 / 3.20× | 5.17 / 3.09× | 4.15 / 2.46× | 3.79 / 2.20× | 5.62 / 2.71× | 5.38 / 2.83× | 4.92 / 2.79× | 4.95 / 2.75× | | **MMFlash (16)** | **6.19 / 3.57×** | **5.67 / 3.42×** | **4.58 / 2.67×** | **4.20 / 2.39×** | **6.44 / 3.02×** | **6.49 / 3.17×** | **5.60 / 3.10×** | **5.60 / 3.05×** | Values in parentheses are the number of draft steps for MTP, the draft tree size for EAGLE-3, and the block size for DFlash and MMFlash. MTP is Qwen3.5-9B's built-in multi-token-prediction head. DFlash-z is the public [`z-lab/Qwen3.5-9B-DFlash`](https://huggingface.co/z-lab/Qwen3.5-9B-DFlash) draft. DFlash is our DFlash draft trained on the same data without the visual-KL weighting. ### Concurrency (temperature 0) MMFlash at 1, 8 and 32 requests in flight, on a single NVIDIA H200. Each cell is the decode throughput in tokens/s, with the speedup over Qwen3.5-9B alone at the same concurrency in parentheses. | Concurrency | ChartQA | CharXiv | MMStar | MMBench | DynaMath | MathVista | MathVerse | Avg. | |---|---|---|---|---|---|---|---|---| | 1 | 620 (3.57×) | 582 (3.42×) | 459 (2.67×) | 421 (2.39×) | 519 (3.02×) | 544 (3.17×) | 530 (3.10×) | 525 (3.05×) | | 8 | 2,438 (2.33×) | 2,534 (2.29×) | 1,964 (1.85×) | 1,869 (1.69×) | 2,541 (2.20×) | 1,675 (1.84×) | 2,435 (2.13×) | 2,208 (2.05×) | | 32 | 3,685 (1.89×) | 3,771 (1.53×) | 2,814 (1.14×) | 2,599 (1.07×) | 3,792 (1.42×) | 2,170 (1.28×) | 3,326 (1.16×) | 3,165 (1.36×) | ## Citation Coming soon.