Upload MMFlash draft and model card

MMFlash draft weights (config.json, model.safetensors) and the first model card.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LrUG2MPyVgRWHb7y6iKMf4

README.md

@@ -0,0 +1,100 @@
+---
+license: apache-2.0
+base_model:
+  - Qwen/Qwen3.5-9B
+inference: false
+tags:
+  - mmflash
+  - dflash
+  - speculative-decoding
+  - speculative-decoding-draft
+  - block-diffusion
+  - draft-model
+  - multimodal
+  - sglang
+  - qwen
+  - qwen3.5
+---
+
+# Qwen3.5-9B-MMFlash
+
+MMFlash draft model for speculative decoding of [`Qwen/Qwen3.5-9B`](https://huggingface.co/Qwen/Qwen3.5-9B) on image and text inputs. It is not a standalone language model; it is paired with the target model in a speculative-decoding server (SGLang `DFLASH`).
+
+MMFlash builds on [DFlash](https://arxiv.org/abs/2602.06036) block-diffusion drafting: the draft reads hidden states from five layers of the target and proposes a block of 16 tokens in one forward pass, and the target verifies the block in one forward pass. MMFlash keeps the DFlash architecture and is trained on multimodal data, with responses regenerated by the target and a loss weighted by per-token visual-grounding (visual-KL) scores.
+
+## Model details
+
+| | |
+|---|---|
+| Target model | [`Qwen/Qwen3.5-9B`](https://huggingface.co/Qwen/Qwen3.5-9B) |
+| Draft architecture | `DFlashDraftModel`: 5 decoder layers, hidden size 4096, 32 attention heads, 8 KV heads |
+| Parameters | 1.05B (bf16). The embedding and LM head are shared with the target and are not stored here. |
+| Block size | 16 |
+| Target hidden states | layers 1, 8, 15, 22, 29 (of 32) |
+| Mask token id | 248070 |
+| Training data | 1M samples of [`mvp-lab/LLaVA-OneVision-1.5-Instruct-Data`](https://huggingface.co/datasets/mvp-lab/LLaVA-OneVision-1.5-Instruct-Data), responses regenerated by Qwen3.5-9B, sequences up to 4096 tokens |
+
+## Usage
+
+```bash
+python -m sglang.launch_server \
+  --model-path Qwen/Qwen3.5-9B \
+  --speculative-algorithm DFLASH \
+  --speculative-draft-model-path FSCCS/Qwen3.5-9B-MMFlash \
+  --speculative-dflash-block-size 16 \
+  --attention-backend fa3 \
+  --mm-attention-backend sdpa \
+  --reasoning-parser qwen3 \
+  --trust-remote-code
+```
+
+The results below were measured with SGLang 0.5.14 plus the SpecForge patches.
+
+## Results
+
+Each cell is **τ / speedup**: τ is the average acceptance length (tokens accepted per verification step), and the speedup is the decoding throughput over Qwen3.5-9B without speculative decoding.
+
+### Temperature 0
+
+| Method | ChartQA | CharXiv | MMStar | MMBench | DynaMath | MathVista | MathVerse | Avg. |
+|---|---|---|---|---|---|---|---|---|
+| MTP (3) | 3.32 / 1.83× | 3.28 / 1.87× | 3.01 / 1.73× | 2.88 / 1.57× | 3.38 / 1.80× | 3.26 / 1.80× | 3.24 / 1.80× | 3.20 / 1.77× |
+| MTP (7) | 4.66 / 1.88× | 4.60 / 1.94× | 3.91 / 1.66× | 3.55 / 1.41× | 4.74 / 1.77× | 4.52 / 1.78× | 4.43 / 1.77× | 4.34 / 1.74× |
+| MTP (15) | 5.27 / 1.38× | 5.27 / 1.47× | 4.20 / 1.18× | 3.75 / 0.99× | 5.25 / 1.30× | 5.08 / 1.32× | 4.90 / 1.30× | 4.81 / 1.27× |
+| EAGLE-3 (16) | 3.93 / 1.81× | 3.85 / 1.93× | 3.06 / 1.55× | 2.72 / 1.32× | 3.60 / 1.76× | 3.50 / 1.63× | 3.48 / 1.75× | 3.45 / 1.68× |
+| DFlash-z (16) | 5.22 / 3.03× | 5.27 / 3.15× | 3.93 / 2.46× | 3.67 / 2.11× | 4.96 / 2.75× | 4.77 / 2.75× | 4.52 / 2.67× | 4.62 / 2.70× |
+| DFlash (16) | 5.59 / 3.20× | 5.17 / 3.09× | 4.15 / 2.46× | 3.79 / 2.20× | 5.62 / 2.71× | 5.38 / 2.83× | 4.92 / 2.79× | 4.95 / 2.75× |
+| **MMFlash (16)** | **6.19 / 3.57×** | **5.67 / 3.42×** | **4.58 / 2.67×** | **4.20 / 2.39×** | **6.44 / 3.02×** | **6.49 / 3.17×** | **5.60 / 3.10×** | **5.60 / 3.05×** |
+
+### Temperature 1
+
+| Method | ChartQA | CharXiv | MMStar | MMBench | DynaMath | MathVista | MathVerse | Avg. |
+|---|---|---|---|---|---|---|---|---|
+| MTP (3) | 3.19 / 1.72× | 3.08 / 1.72× | 2.86 / 1.61× | 2.78 / 1.49× | 3.25 / 1.70× | 3.11 / 1.69× | 3.11 / 1.71× | 3.05 / 1.66× |
+| MTP (7) | 4.35 / 1.72× | 4.15 / 1.71× | 3.60 / 1.49× | 3.37 / 1.34× | 4.46 / 1.64× | 4.26 / 1.65× | 4.17 / 1.66× | 4.05 / 1.60× |
+| MTP (15) | 4.84 / 1.24× | 4.57 / 1.27× | 3.81 / 1.06× | 3.53 / 0.93× | 4.92 / 1.22× | 4.65 / 1.18× | 4.56 / 1.21× | 4.41 / 1.16× |
+| EAGLE-3 (16) | 3.60 / 1.67× | 3.56 / 1.72× | 2.84 / 1.42× | 2.59 / 1.23× | 3.46 / 1.61× | 3.32 / 1.50× | 3.29 / 1.62× | 3.24 / 1.54× |
+| DFlash-z (16) | 4.77 / 2.64× | 4.70 / 2.76× | 3.66 / 2.25× | 3.42 / 1.97× | 4.71 / 2.57× | 4.49 / 2.51× | 4.26 / 2.49× | 4.29 / 2.46× |
+| DFlash (16) | 4.94 / 2.75× | 4.50 / 2.65× | 3.74 / 2.18× | 3.55 / 2.02× | 5.24 / 2.52× | 4.85 / 2.52× | 4.53 / 2.57× | 4.48 / 2.46× |
+| **MMFlash (16)** | **5.54 / 3.22×** | **4.85 / 2.85×** | **4.13 / 2.37×** | **3.82 / 2.17×** | **6.07 / 2.77×** | **5.71 / 2.74×** | **5.10 / 2.81×** | **5.03 / 2.70×** |
+
+Values in parentheses are the number of draft steps for MTP, the draft tree size for EAGLE-3, and the block size for DFlash and MMFlash. MTP is Qwen3.5-9B's built-in multi-token-prediction head. DFlash-z is the public [`z-lab/Qwen3.5-9B-DFlash`](https://huggingface.co/z-lab/Qwen3.5-9B-DFlash) draft. DFlash is our DFlash draft trained on the same 1M LLaVA-OneVision-1.5 samples without the visual-KL weighting.
+
+### Concurrency (temperature 0)
+
+MMFlash at 1, 8 and 32 requests in flight. Each cell is τ / speedup over Qwen3.5-9B at the same concurrency.
+
+| Concurrency | ChartQA | MMStar | DynaMath |
+|---|---|---|---|
+| 1 | 6.19 / 3.57× | 4.58 / 2.67× | 6.44 / 3.02× |
+| 8 | 6.20 / 2.33× | 4.57 / 1.85× | 6.50 / 2.20× |
+| 32 | 6.20 / 1.89× | 4.66 / 1.14× | 6.52 / 1.42× |
+
+### Setup
+
+- 200 questions per benchmark, thinking off, at most 4096 new tokens, one NVIDIA H200, SGLang.
+- At concurrency 1, throughput is decode tokens over decode time (first token to the last), summed over the requests. At concurrency 8 and 32, it is decode tokens over the time the server spent running decode batches.
+
+## Citation
+
+Coming soon.

config.json

@@ -0,0 +1,53 @@
+{
+  "architectures": [
+    "DFlashDraftModel"
+  ],
+  "attention_bias": false,
+  "attention_dropout": 0.0,
+  "block_size": 16,
+  "bos_token_id": 248043,
+  "dflash_config": {
+    "mask_token_id": 248070,
+    "target_layer_ids": [
+      1,
+      8,
+      15,
+      22,
+      29
+    ]
+  },
+  "draft_vocab_size": 248320,
+  "dtype": "bfloat16",
+  "eos_token_id": 248044,
+  "head_dim": 128,
+  "hidden_act": "silu",
+  "hidden_size": 4096,
+  "initializer_range": 0.02,
+  "intermediate_size": 12288,
+  "layer_types": [
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention"
+  ],
+  "max_position_embeddings": 262144,
+  "max_window_layers": 5,
+  "model_type": "qwen3",
+  "num_attention_heads": 32,
+  "num_hidden_layers": 5,
+  "num_key_value_heads": 8,
+  "num_target_layers": 32,
+  "pad_token_id": null,
+  "rms_norm_eps": 1e-06,
+  "rope_parameters": {
+    "rope_theta": 10000000,
+    "rope_type": "default"
+  },
+  "sliding_window": null,
+  "tie_word_embeddings": false,
+  "transformers_version": "5.8.1",
+  "use_cache": true,
+  "use_sliding_window": false,
+  "vocab_size": 248320
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b2c47162245735da9a2d32ec61ec2bd7798737d4902a56368b5f4922f5614493
+size 2097259104