FSCCS Claude Opus 5.5 commited on
Commit
fc5a5a3
·
verified ·
1 Parent(s): 935910e

README: drop the training-data details

Browse files

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LrUG2MPyVgRWHb7y6iKMf4

Files changed (1) hide show
  1. README.md +2 -3
README.md CHANGED
@@ -20,7 +20,7 @@ tags:
20
 
21
  MMFlash draft model for speculative decoding of [`Qwen/Qwen3.5-4B`](https://huggingface.co/Qwen/Qwen3.5-4B) on image and text inputs. It is not a standalone language model; it is paired with the target model in a speculative-decoding server (SGLang `DFLASH`).
22
 
23
- MMFlash builds on [DFlash](https://github.com/z-lab/dflash) block-diffusion drafting: the draft reads hidden states from five layers of the target and proposes a block of 16 tokens in one forward pass, and the target verifies the block in one forward pass. MMFlash keeps the DFlash architecture and is trained on multimodal data, with responses regenerated by the target and a loss weighted by per-token visual-grounding (visual-KL) scores.
24
 
25
  ## Model details
26
 
@@ -32,7 +32,6 @@ MMFlash builds on [DFlash](https://github.com/z-lab/dflash) block-diffusion draf
32
  | Block size | 16 |
33
  | Target hidden states | layers 1, 8, 15, 22, 29 (of 32) |
34
  | Mask token id | 248070 |
35
- | Training data | 1M samples of [`mvp-lab/LLaVA-OneVision-1.5-Instruct-Data`](https://huggingface.co/datasets/mvp-lab/LLaVA-OneVision-1.5-Instruct-Data), responses regenerated by Qwen3.5-4B, sequences up to 4096 tokens |
36
 
37
  ## Usage
38
 
@@ -93,7 +92,7 @@ Each cell is **τ / speedup**: τ is the average acceptance length (tokens accep
93
  | DFlash (16) | 4.87 / 2.56× | 4.95 / 2.66× | 3.71 / 2.04× | 4.92 / 2.48× | 5.07 / 2.31× | 4.66 / 2.22× | 4.47 / 2.37× | 4.67 / 2.38× |
94
  | **MMFlash (16)** | **5.23 / 2.74×** | **5.39 / 2.86×** | **3.96 / 2.18×** | **5.25 / 2.58×** | **5.75 / 2.56×** | **5.23 / 2.37×** | **4.69 / 2.49×** | **5.07 / 2.54×** |
95
 
96
- Values in parentheses are the number of draft steps for MTP, the draft tree size for EAGLE-3, and the block size for DFlash and MMFlash. MTP is Qwen3.5-4B's built-in multi-token-prediction head. DFlash-z is the public [`z-lab/Qwen3.5-4B-DFlash`](https://huggingface.co/z-lab/Qwen3.5-4B-DFlash) draft. DFlash is our DFlash draft trained on the same 1M LLaVA-OneVision-1.5 samples without the visual-KL weighting.
97
 
98
  ### Concurrency (temperature 0)
99
 
 
20
 
21
  MMFlash draft model for speculative decoding of [`Qwen/Qwen3.5-4B`](https://huggingface.co/Qwen/Qwen3.5-4B) on image and text inputs. It is not a standalone language model; it is paired with the target model in a speculative-decoding server (SGLang `DFLASH`).
22
 
23
+ MMFlash builds on [DFlash](https://github.com/z-lab/dflash) block-diffusion drafting: the draft reads hidden states from five layers of the target and proposes a block of 16 tokens in one forward pass, and the target verifies the block in one forward pass. MMFlash keeps the DFlash architecture and is trained on multimodal data with a loss weighted by per-token visual-grounding (visual-KL) scores.
24
 
25
  ## Model details
26
 
 
32
  | Block size | 16 |
33
  | Target hidden states | layers 1, 8, 15, 22, 29 (of 32) |
34
  | Mask token id | 248070 |
 
35
 
36
  ## Usage
37
 
 
92
  | DFlash (16) | 4.87 / 2.56× | 4.95 / 2.66× | 3.71 / 2.04× | 4.92 / 2.48× | 5.07 / 2.31× | 4.66 / 2.22× | 4.47 / 2.37× | 4.67 / 2.38× |
93
  | **MMFlash (16)** | **5.23 / 2.74×** | **5.39 / 2.86×** | **3.96 / 2.18×** | **5.25 / 2.58×** | **5.75 / 2.56×** | **5.23 / 2.37×** | **4.69 / 2.49×** | **5.07 / 2.54×** |
94
 
95
+ Values in parentheses are the number of draft steps for MTP, the draft tree size for EAGLE-3, and the block size for DFlash and MMFlash. MTP is Qwen3.5-4B's built-in multi-token-prediction head. DFlash-z is the public [`z-lab/Qwen3.5-4B-DFlash`](https://huggingface.co/z-lab/Qwen3.5-4B-DFlash) draft. DFlash is our DFlash draft trained on the same data without the visual-KL weighting.
96
 
97
  ### Concurrency (temperature 0)
98