mango-dflash2 / README.md
isilicon's picture
Publish detailed Mango model card, measured results and visual analysis
fe2e918 verified
|
Raw History Blame Contribute Delete
13.3 kB
---
license: apache-2.0
base_model: z-lab/Qwen3.8-27B-DFlash2
inference: false
tags:
- mango
- preservation-mirror
- qwen-27b
- auxiliary-component
---
![Mango · Qwen DFlash2 · Block-diffusion draft · z-lab](assets/mango-header.png)
# Qwen DFlash2
**Block-diffusion draft · z-lab** · Runtime companion
z-lab's Qwen3.8 27B DFlash2 draft model, containing its config and safetensors weights. It drafts tokens for a target model to verify and is not a standalone language model.
[Benchmark results](#mango-evaluation) · [Files](#files-and-integrity) · [Download](#download-and-use) · [Runtime](#runtime-and-compatibility) · [Provenance](#provenance-and-license) · [Full collection](https://huggingface.co/collections/isilicon/mango-tested-models-and-preservation-mirrors-6ae3b73994c895ab42e360d2)
<!-- mango-community:start -->
> **Join Infinity on Discord**
>
> Get model support, request models and quants, discuss benchmarks, and follow new releases and model improvements.
>
> [Join the Infinity community](https://discord.gg/5n5YDbQCMy)
>
> [All model tests, per-question scores, experiment history and scoring methods](https://huggingface.co/datasets/isilicon/mango-evaluation-results)
<!-- mango-community:end -->
<!-- mango-qwen-effort:start -->
**Qwen effort study — separate full-404 cohort:** medium 82.87/100 versus xhigh 84.41/100; paired difference +1.54 points (95% interval [-1.71, +4.82]). 404 collected questions, 403 valid questions graded per effort; one invalid reference annotation excluded symmetrically. The historical 202-question cross-model comparison below is unchanged. [Study, timing and blind subjective results](https://huggingface.co/isilicon/mango-qwen38-27b-uncensored-mlx4#qwen-medium-versus-xhigh).
<!-- mango-qwen-effort:end -->
Preserved and documented by **isilicon / Mango**. The original model, modification and quantization remain the upstream creators' work. This repository adds attributed preservation and exact-configuration evaluation, with the selected source bytes unchanged.
## At a glance
| Property | Recorded value |
| --- | --- |
| Artifact | safetensors draft |
| Selected source payload | 3.85 GB / 3.58 GiB |
| Primary selected files | 2 |
| Evaluation status | Runtime companion |
| Source publisher | z-lab |
| Source revision | `50307d4c4cde6860d4eee73e2547cd786fe8e8a4` |
| Declared license | apache-2.0 |
GB is decimal and GiB is binary. Payload size is disk/storage size, not a claim that the model fits in the same amount of VRAM. Runtime, KV cache, working memory and companions add their own requirements.
## Mango evaluation
**No standalone Mango quality score has been published for these exact files.** This is a companion artifact; the consuming model's result belongs to the full target/runtime configuration.
### Related configurations measured separately
These results describe the linked configurations, **not this artifact**. A change of quant, modification lineage, draft, engine or tokenizer can change the result.
| Different configuration | Its weighted score | Its workload min |
| --- | --- | --- |
| [Qwen3.8 27B Cyber · IQ4_XS imatrix · embedded MTP](https://huggingface.co/isilicon/mango-qwen27-cyber-iq4xs-mtp) | 70.27 | 29.74 |
| [Qwen3.8 27B Cyber · Q6_K · Q8 MTP + projector](https://huggingface.co/isilicon/mango-qwen27-cyber-q6k) | 66.98 | 45.60 |
| [Qwen3.8 27B · MLX affine 4-bit · OrcaRouter](https://huggingface.co/isilicon/mango-qwen38-27b-uncensored-mlx4) | 80.64 | 23.98 |
A completed score requires a native preflight, matching frozen prompts, 202 suite answers, 12 agent cases, archive verification and the same reference graders. The [benchmark methodology](BENCHMARK_METHODOLOGY.md) describes the protocol; publisher-side results are retained separately below.
## Runtime and compatibility
A speculative-decoding runtime must support DFlash2 and the target model. The measured Mango workload combines the target, draft and engine; this mirror alone has no separate task-quality or speed score.
Context and output behavior belong to the target/runtime configuration.
### Companion artifacts
| Consuming model | Connection |
| --- | --- |
| [Qwen3.8 27B · MLX affine 4-bit · OrcaRouter](https://huggingface.co/isilicon/mango-qwen38-27b-uncensored-mlx4) | Use the compatible parent runtime and exact target revision |
## Files and integrity
| Selected file · original path | GB | Identity prefix |
| --- | --- | --- |
| [config.json](https://huggingface.co/isilicon/mango-dflash2/resolve/main/config.json) | 0.00000 | `79279cc5665bced6…` (Git blob) |
| [model.safetensors](https://huggingface.co/isilicon/mango-dflash2/resolve/main/model.safetensors) | 3.84882 | `67fc76d68dc5a941…` (SHA-256) |
Full paths, byte sizes and complete SHA-256/Git blob identities are in [SOURCE_MANIFEST.json](SOURCE_MANIFEST.json). Available original README/license/notice files are also preserved. [MIRROR_STATUS.json](MIRROR_STATUS.json) records the earlier source-file verification; MODEL_CARD_STATUS.json records this documentation refresh separately.
## Download and use
The commands pin the already-verified mirror weight revision. Documentation can advance on main without changing that preserved weight snapshot. Keep original subfolders and all shards.
```bash
python -m pip install -U huggingface_hub
hf download isilicon/mango-dflash2 \
config.json \
model.safetensors \
--revision e383e96a1ac8822a9012a57d43c40d488993a7ad \
--local-dir ./weights/dflash2
```
### Load example and setup scope
This component has no independent chat launch command. Install it through the consuming model's compatible runtime and companion layout.
## Provenance and license
The source credits the DFlash2 work and points to the incoai release. Its acceptance-length and SGLang/vLLM throughput tables concern the publisher's configurations. Mango used this companion with its Qwen MLX4 target through TensorFold.
| Responsibility / identity | Attribution |
| --- | --- |
| Mirror / local evaluation | [isilicon](https://huggingface.co/isilicon) · Mango |
| Immediate source | [z-lab/Qwen3.8-27B-DFlash2](https://huggingface.co/z-lab/Qwen3.8-27B-DFlash2/tree/50307d4c4cde6860d4eee73e2547cd786fe8e8a4) |
| Pinned source commit | `50307d4c4cde6860d4eee73e2547cd786fe8e8a4` |
| Declared upstream base(s) | [Qwen/Qwen3.8-27B](https://huggingface.co/Qwen/Qwen3.8-27B) |
| Source-declared license | apache-2.0 |
Original notices, base-model conditions and the publisher's stated limitations still apply. Downloadable preservation does not add new license rights. Source-described refusal modification and publisher-side benchmark claims have not been independently re-measured by this Mango task cohort.
### Original publisher documentation
The [unchanged upstream README](UPSTREAM_README.md) and [pinned source card](https://huggingface.co/z-lab/Qwen3.8-27B-DFlash2/blob/50307d4c4cde6860d4eee73e2547cd786fe8e8a4/README.md) retain the original usage guidance, technical discussion, attribution, caveats and citations.
<details>
<summary>Read the complete original publisher card</summary>
**Source reference.** The following is the original publisher's documentation. Its other quants, full-model files, context claims and benchmark tables can exceed the selection in this mirror. Relative asset/file URLs are qualified to the pinned original source; publisher measurements remain separate from Mango measurements.
# Qwen3.8-27B-DFlash2
[Blog](https://inco.ai/blog/dflash2/) | [GitHub](https://github.com/z-lab/dflash)
This repository contains the DFlash 2 draft model for
[`Qwen/Qwen3.8-27B`](https://huggingface.co/Qwen/Qwen3.8-27B).
It is not a standalone language model: it runs inside a speculative
decoding server and drafts tokens for the target model to verify. This repository is a mirror
of [`incoai/Qwen3.8-27B-DFlash2`](https://huggingface.co/incoai/Qwen3.8-27B-DFlash2).
DFlash 2 is a block-diffusion drafter for speculative decoding. It predicts
a whole block of tokens in a single pass and keeps the top candidates at
every position. A lightweight selector then traces one coherent path through them.
Two-tap dynamic convolutions in the backbone keep the draft from decaying
toward the end of the block. Decoding is lossless: greedy output
matches the target model exactly, and sampling preserves its distribution.
<div align="center">
<img src="https://huggingface.co/z-lab/Qwen3.8-27B-DFlash2/resolve/50307d4c4cde6860d4eee73e2547cd786fe8e8a4/assets/dflash2-figure.png" alt="DFlash 2: parallel block drafting with a candidate path selector" width="100%">
</div>
## Quick Start
Serve with [SGLang](https://github.com/sgl-project/sglang):
```bash
pip install "sglang[all] @ git+https://github.com/sgl-project/sglang.git#subdirectory=python"
python -m sglang.launch_server \
--model-path Qwen/Qwen3.8-27B \
--speculative-algorithm DFLASH \
--speculative-draft-model-path incoai/Qwen3.8-27B-DFlash2 \
--speculative-num-draft-tokens 8
```
Or with [vLLM](https://github.com/vllm-project/vllm):
```bash
pip install -U "vllm @ git+https://github.com/vllm-project/vllm.git@refs/pull/52816/head"
vllm serve Qwen/Qwen3.8-27B \
--speculative-config '{
"method": "dflash",
"model": "incoai/Qwen3.8-27B-DFlash2",
"num_speculative_tokens": 7
}'
```
See the [blog post](https://inco.ai/blog/dflash2/) for other engines and more details.
## Evaluation
- Runtime: SGLang on one NVIDIA H200, with FlashAttention 3 for target and draft attention
- Speculation block size: 8 (7 draft tokens per verification step)
- Sampling: Qwen3.8's officially recommended parameters (temperature 1.0, top-p 0.95, top-k 20), with `xhigh` reasoning effort
- Maximum new tokens: 4096
- Prompts: benchmark formatting from [`z-lab/dflash`](https://github.com/z-lab/dflash)
We compare autoregressive decoding, Qwen3.8's built-in seven-token MTP,
a community DSpark drafter
([`RadixArk/Qwen3.8-27B-DSpark`](https://huggingface.co/RadixArk/Qwen3.8-27B-DSpark)),
and DFlash 2. All speculative methods propose seven draft tokens per
verification step.
### Acceptance Length
Acceptance length is the per-request mean of completion tokens divided by verification steps.
Higher is better.
| Task | MTP | DSpark | DFlash 2 |
| :--- | ---: | ---: | ---: |
| GSM8K | 5.02 | 4.36 | **5.46** |
| MATH-500 | 4.72 | 3.92 | **5.28** |
| HumanEval | 3.91 | 3.30 | **4.39** |
| MBPP | 3.99 | 3.51 | **4.79** |
| MT-Bench | 3.74 | 3.01 | **4.10** |
### Throughput
Throughput is total output tokens divided by end-to-end wall time.
Each cell shows `output tok/s (speedup vs. autoregressive)`.
#### Concurrency 1
| Task | Autoregressive | MTP | DSpark | DFlash 2 |
| :--- | ---: | ---: | ---: | ---: |
| GSM8K | 68.9 | 178.5 (2.59×) | 185.3 (2.69×) | **236.1 (3.43×)** |
| MATH-500 | 69.0 | 172.8 (2.51×) | 174.5 (2.53×) | **230.7 (3.34×)** |
| HumanEval | 69.0 | 151.9 (2.20×) | 159.9 (2.32×) | **214.6 (3.11×)** |
| MBPP | 69.0 | 153.1 (2.22×) | 163.3 (2.37×) | **226.9 (3.29×)** |
| MT-Bench | 68.9 | 134.9 (1.96×) | 137.6 (2.00×) | **184.0 (2.67×)** |
#### Concurrency 8
| Task | Autoregressive | MTP | DSpark | DFlash 2 |
| :--- | ---: | ---: | ---: | ---: |
| GSM8K | 467.2 | 1,022.1 (2.19×) | 1,040.8 (2.23×) | **1,328.7 (2.84×)** |
| MATH-500 | 480.0 | 1,023.5 (2.13×) | 1,025.8 (2.14×) | **1,368.3 (2.85×)** |
| HumanEval | 483.4 | 934.2 (1.93×) | 956.5 (1.98×) | **1,291.5 (2.67×)** |
| MBPP | 478.0 | 938.1 (1.96×) | 974.1 (2.04×) | **1,328.0 (2.78×)** |
| MT-Bench | 480.5 | 835.2 (1.74×) | 802.3 (1.67×) | **1,090.2 (2.27×)** |
#### Concurrency 32
| Task | Autoregressive | MTP | DSpark | DFlash 2 |
| :--- | ---: | ---: | ---: | ---: |
| GSM8K | 1,329.8 | 1,381.1 (1.04×) | 1,506.5 (1.13×) | **1,922.5 (1.45×)** |
| MATH-500 | 1,505.8 | 1,415.6 (0.94×) | 1,429.0 (0.95×) | **1,951.8 (1.30×)** |
| HumanEval | 1,546.5 | 1,296.8 (0.84×) | 1,330.1 (0.86×) | **1,799.0 (1.16×)** |
| MBPP | 1,507.7 | 1,314.9 (0.87×) | 1,361.3 (0.90×) | **1,886.8 (1.25×)** |
| MT-Bench | 1,507.4 | 1,159.7 (0.77×) | 1,115.5 (0.74×) | **1,525.3 (1.01×)** |
## Citation
If you find DFlash 2 useful, please cite:
```bibtex
@misc{inco2026dflash2,
title = {{DFlash 2: Keep Drafting Parallel}},
author = {{Inco AI}},
year = {2026},
month = {August},
url = {https://inco.ai/blog/dflash2/}
}
```
Please also cite the original DFlash paper:
```bibtex
@inproceedings{chen2026dflash,
title = {{DFlash: Block Diffusion for Flash Speculative Decoding}},
author = {Chen, Jian and Liang, Yesheng and Liu, Zhijian},
booktitle = {International Conference on Machine Learning (ICML)},
year = {2026}
}
```
</details>
## Preservation record
Selected source bytes are verified against pinned metadata; the original eight graded mirrors also passed the previously recorded anonymous 1 MiB weight-prefix checks. This is not a full independent-device restore/load test. Hugging Face repository copies share a provider and do not guarantee permanent storage.
[Collection](https://huggingface.co/collections/isilicon/mango-tested-models-and-preservation-mirrors-6ae3b73994c895ab42e360d2) · [Source manifest](SOURCE_MANIFEST.json) · [Original documentation](UPSTREAM_README.md)