|
Download README.md from isilicon/mango-dflash2: direct link, hf CLI and curl.
- Browser
- Download file 13.3 kB
-
https://huggingface.co/isilicon/mango-dflash2/resolve/main/README.md
- Command line
-
hf download hf://isilicon/mango-dflash2/README.md
-
curl -L -o README.md https://huggingface.co/isilicon/mango-dflash2/resolve/main/README.md
13.3 kB
| license: apache-2.0 | |
| base_model: z-lab/Qwen3.8-27B-DFlash2 | |
| inference: false | |
| tags: | |
| - mango | |
| - preservation-mirror | |
| - qwen-27b | |
| - auxiliary-component | |
|  | |
| # Qwen DFlash2 | |
| **Block-diffusion draft · z-lab** · Runtime companion | |
| z-lab's Qwen3.8 27B DFlash2 draft model, containing its config and safetensors weights. It drafts tokens for a target model to verify and is not a standalone language model. | |
| [Benchmark results](#mango-evaluation) · [Files](#files-and-integrity) · [Download](#download-and-use) · [Runtime](#runtime-and-compatibility) · [Provenance](#provenance-and-license) · [Full collection](https://huggingface.co/collections/isilicon/mango-tested-models-and-preservation-mirrors-6ae3b73994c895ab42e360d2) | |
| <!-- mango-community:start --> | |
| > **Join Infinity on Discord** | |
| > | |
| > Get model support, request models and quants, discuss benchmarks, and follow new releases and model improvements. | |
| > | |
| > [Join the Infinity community](https://discord.gg/5n5YDbQCMy) | |
| > | |
| > [All model tests, per-question scores, experiment history and scoring methods](https://huggingface.co/datasets/isilicon/mango-evaluation-results) | |
| <!-- mango-community:end --> | |
| <!-- mango-qwen-effort:start --> | |
| **Qwen effort study — separate full-404 cohort:** medium 82.87/100 versus xhigh 84.41/100; paired difference +1.54 points (95% interval [-1.71, +4.82]). 404 collected questions, 403 valid questions graded per effort; one invalid reference annotation excluded symmetrically. The historical 202-question cross-model comparison below is unchanged. [Study, timing and blind subjective results](https://huggingface.co/isilicon/mango-qwen38-27b-uncensored-mlx4#qwen-medium-versus-xhigh). | |
| <!-- mango-qwen-effort:end --> | |
| Preserved and documented by **isilicon / Mango**. The original model, modification and quantization remain the upstream creators' work. This repository adds attributed preservation and exact-configuration evaluation, with the selected source bytes unchanged. | |
| ## At a glance | |
| | Property | Recorded value | | |
| | --- | --- | | |
| | Artifact | safetensors draft | | |
| | Selected source payload | 3.85 GB / 3.58 GiB | | |
| | Primary selected files | 2 | | |
| | Evaluation status | Runtime companion | | |
| | Source publisher | z-lab | | |
| | Source revision | `50307d4c4cde6860d4eee73e2547cd786fe8e8a4` | | |
| | Declared license | apache-2.0 | | |
| GB is decimal and GiB is binary. Payload size is disk/storage size, not a claim that the model fits in the same amount of VRAM. Runtime, KV cache, working memory and companions add their own requirements. | |
| ## Mango evaluation | |
| **No standalone Mango quality score has been published for these exact files.** This is a companion artifact; the consuming model's result belongs to the full target/runtime configuration. | |
| ### Related configurations measured separately | |
| These results describe the linked configurations, **not this artifact**. A change of quant, modification lineage, draft, engine or tokenizer can change the result. | |
| | Different configuration | Its weighted score | Its workload min | | |
| | --- | --- | --- | | |
| | [Qwen3.8 27B Cyber · IQ4_XS imatrix · embedded MTP](https://huggingface.co/isilicon/mango-qwen27-cyber-iq4xs-mtp) | 70.27 | 29.74 | | |
| | [Qwen3.8 27B Cyber · Q6_K · Q8 MTP + projector](https://huggingface.co/isilicon/mango-qwen27-cyber-q6k) | 66.98 | 45.60 | | |
| | [Qwen3.8 27B · MLX affine 4-bit · OrcaRouter](https://huggingface.co/isilicon/mango-qwen38-27b-uncensored-mlx4) | 80.64 | 23.98 | | |
| A completed score requires a native preflight, matching frozen prompts, 202 suite answers, 12 agent cases, archive verification and the same reference graders. The [benchmark methodology](BENCHMARK_METHODOLOGY.md) describes the protocol; publisher-side results are retained separately below. | |
| ## Runtime and compatibility | |
| A speculative-decoding runtime must support DFlash2 and the target model. The measured Mango workload combines the target, draft and engine; this mirror alone has no separate task-quality or speed score. | |
| Context and output behavior belong to the target/runtime configuration. | |
| ### Companion artifacts | |
| | Consuming model | Connection | | |
| | --- | --- | | |
| | [Qwen3.8 27B · MLX affine 4-bit · OrcaRouter](https://huggingface.co/isilicon/mango-qwen38-27b-uncensored-mlx4) | Use the compatible parent runtime and exact target revision | | |
| ## Files and integrity | |
| | Selected file · original path | GB | Identity prefix | | |
| | --- | --- | --- | | |
| | [config.json](https://huggingface.co/isilicon/mango-dflash2/resolve/main/config.json) | 0.00000 | `79279cc5665bced6…` (Git blob) | | |
| | [model.safetensors](https://huggingface.co/isilicon/mango-dflash2/resolve/main/model.safetensors) | 3.84882 | `67fc76d68dc5a941…` (SHA-256) | | |
| Full paths, byte sizes and complete SHA-256/Git blob identities are in [SOURCE_MANIFEST.json](SOURCE_MANIFEST.json). Available original README/license/notice files are also preserved. [MIRROR_STATUS.json](MIRROR_STATUS.json) records the earlier source-file verification; MODEL_CARD_STATUS.json records this documentation refresh separately. | |
| ## Download and use | |
| The commands pin the already-verified mirror weight revision. Documentation can advance on main without changing that preserved weight snapshot. Keep original subfolders and all shards. | |
| ```bash | |
| python -m pip install -U huggingface_hub | |
| hf download isilicon/mango-dflash2 \ | |
| config.json \ | |
| model.safetensors \ | |
| --revision e383e96a1ac8822a9012a57d43c40d488993a7ad \ | |
| --local-dir ./weights/dflash2 | |
| ``` | |
| ### Load example and setup scope | |
| This component has no independent chat launch command. Install it through the consuming model's compatible runtime and companion layout. | |
| ## Provenance and license | |
| The source credits the DFlash2 work and points to the incoai release. Its acceptance-length and SGLang/vLLM throughput tables concern the publisher's configurations. Mango used this companion with its Qwen MLX4 target through TensorFold. | |
| | Responsibility / identity | Attribution | | |
| | --- | --- | | |
| | Mirror / local evaluation | [isilicon](https://huggingface.co/isilicon) · Mango | | |
| | Immediate source | [z-lab/Qwen3.8-27B-DFlash2](https://huggingface.co/z-lab/Qwen3.8-27B-DFlash2/tree/50307d4c4cde6860d4eee73e2547cd786fe8e8a4) | | |
| | Pinned source commit | `50307d4c4cde6860d4eee73e2547cd786fe8e8a4` | | |
| | Declared upstream base(s) | [Qwen/Qwen3.8-27B](https://huggingface.co/Qwen/Qwen3.8-27B) | | |
| | Source-declared license | apache-2.0 | | |
| Original notices, base-model conditions and the publisher's stated limitations still apply. Downloadable preservation does not add new license rights. Source-described refusal modification and publisher-side benchmark claims have not been independently re-measured by this Mango task cohort. | |
| ### Original publisher documentation | |
| The [unchanged upstream README](UPSTREAM_README.md) and [pinned source card](https://huggingface.co/z-lab/Qwen3.8-27B-DFlash2/blob/50307d4c4cde6860d4eee73e2547cd786fe8e8a4/README.md) retain the original usage guidance, technical discussion, attribution, caveats and citations. | |
| <details> | |
| <summary>Read the complete original publisher card</summary> | |
| **Source reference.** The following is the original publisher's documentation. Its other quants, full-model files, context claims and benchmark tables can exceed the selection in this mirror. Relative asset/file URLs are qualified to the pinned original source; publisher measurements remain separate from Mango measurements. | |
| # Qwen3.8-27B-DFlash2 | |
| [Blog](https://inco.ai/blog/dflash2/) | [GitHub](https://github.com/z-lab/dflash) | |
| This repository contains the DFlash 2 draft model for | |
| [`Qwen/Qwen3.8-27B`](https://huggingface.co/Qwen/Qwen3.8-27B). | |
| It is not a standalone language model: it runs inside a speculative | |
| decoding server and drafts tokens for the target model to verify. This repository is a mirror | |
| of [`incoai/Qwen3.8-27B-DFlash2`](https://huggingface.co/incoai/Qwen3.8-27B-DFlash2). | |
| DFlash 2 is a block-diffusion drafter for speculative decoding. It predicts | |
| a whole block of tokens in a single pass and keeps the top candidates at | |
| every position. A lightweight selector then traces one coherent path through them. | |
| Two-tap dynamic convolutions in the backbone keep the draft from decaying | |
| toward the end of the block. Decoding is lossless: greedy output | |
| matches the target model exactly, and sampling preserves its distribution. | |
| <div align="center"> | |
| <img src="https://huggingface.co/z-lab/Qwen3.8-27B-DFlash2/resolve/50307d4c4cde6860d4eee73e2547cd786fe8e8a4/assets/dflash2-figure.png" alt="DFlash 2: parallel block drafting with a candidate path selector" width="100%"> | |
| </div> | |
| ## Quick Start | |
| Serve with [SGLang](https://github.com/sgl-project/sglang): | |
| ```bash | |
| pip install "sglang[all] @ git+https://github.com/sgl-project/sglang.git#subdirectory=python" | |
| python -m sglang.launch_server \ | |
| --model-path Qwen/Qwen3.8-27B \ | |
| --speculative-algorithm DFLASH \ | |
| --speculative-draft-model-path incoai/Qwen3.8-27B-DFlash2 \ | |
| --speculative-num-draft-tokens 8 | |
| ``` | |
| Or with [vLLM](https://github.com/vllm-project/vllm): | |
| ```bash | |
| pip install -U "vllm @ git+https://github.com/vllm-project/vllm.git@refs/pull/52816/head" | |
| vllm serve Qwen/Qwen3.8-27B \ | |
| --speculative-config '{ | |
| "method": "dflash", | |
| "model": "incoai/Qwen3.8-27B-DFlash2", | |
| "num_speculative_tokens": 7 | |
| }' | |
| ``` | |
| See the [blog post](https://inco.ai/blog/dflash2/) for other engines and more details. | |
| ## Evaluation | |
| - Runtime: SGLang on one NVIDIA H200, with FlashAttention 3 for target and draft attention | |
| - Speculation block size: 8 (7 draft tokens per verification step) | |
| - Sampling: Qwen3.8's officially recommended parameters (temperature 1.0, top-p 0.95, top-k 20), with `xhigh` reasoning effort | |
| - Maximum new tokens: 4096 | |
| - Prompts: benchmark formatting from [`z-lab/dflash`](https://github.com/z-lab/dflash) | |
| We compare autoregressive decoding, Qwen3.8's built-in seven-token MTP, | |
| a community DSpark drafter | |
| ([`RadixArk/Qwen3.8-27B-DSpark`](https://huggingface.co/RadixArk/Qwen3.8-27B-DSpark)), | |
| and DFlash 2. All speculative methods propose seven draft tokens per | |
| verification step. | |
| ### Acceptance Length | |
| Acceptance length is the per-request mean of completion tokens divided by verification steps. | |
| Higher is better. | |
| | Task | MTP | DSpark | DFlash 2 | | |
| | :--- | ---: | ---: | ---: | | |
| | GSM8K | 5.02 | 4.36 | **5.46** | | |
| | MATH-500 | 4.72 | 3.92 | **5.28** | | |
| | HumanEval | 3.91 | 3.30 | **4.39** | | |
| | MBPP | 3.99 | 3.51 | **4.79** | | |
| | MT-Bench | 3.74 | 3.01 | **4.10** | | |
| ### Throughput | |
| Throughput is total output tokens divided by end-to-end wall time. | |
| Each cell shows `output tok/s (speedup vs. autoregressive)`. | |
| #### Concurrency 1 | |
| | Task | Autoregressive | MTP | DSpark | DFlash 2 | | |
| | :--- | ---: | ---: | ---: | ---: | | |
| | GSM8K | 68.9 | 178.5 (2.59×) | 185.3 (2.69×) | **236.1 (3.43×)** | | |
| | MATH-500 | 69.0 | 172.8 (2.51×) | 174.5 (2.53×) | **230.7 (3.34×)** | | |
| | HumanEval | 69.0 | 151.9 (2.20×) | 159.9 (2.32×) | **214.6 (3.11×)** | | |
| | MBPP | 69.0 | 153.1 (2.22×) | 163.3 (2.37×) | **226.9 (3.29×)** | | |
| | MT-Bench | 68.9 | 134.9 (1.96×) | 137.6 (2.00×) | **184.0 (2.67×)** | | |
| #### Concurrency 8 | |
| | Task | Autoregressive | MTP | DSpark | DFlash 2 | | |
| | :--- | ---: | ---: | ---: | ---: | | |
| | GSM8K | 467.2 | 1,022.1 (2.19×) | 1,040.8 (2.23×) | **1,328.7 (2.84×)** | | |
| | MATH-500 | 480.0 | 1,023.5 (2.13×) | 1,025.8 (2.14×) | **1,368.3 (2.85×)** | | |
| | HumanEval | 483.4 | 934.2 (1.93×) | 956.5 (1.98×) | **1,291.5 (2.67×)** | | |
| | MBPP | 478.0 | 938.1 (1.96×) | 974.1 (2.04×) | **1,328.0 (2.78×)** | | |
| | MT-Bench | 480.5 | 835.2 (1.74×) | 802.3 (1.67×) | **1,090.2 (2.27×)** | | |
| #### Concurrency 32 | |
| | Task | Autoregressive | MTP | DSpark | DFlash 2 | | |
| | :--- | ---: | ---: | ---: | ---: | | |
| | GSM8K | 1,329.8 | 1,381.1 (1.04×) | 1,506.5 (1.13×) | **1,922.5 (1.45×)** | | |
| | MATH-500 | 1,505.8 | 1,415.6 (0.94×) | 1,429.0 (0.95×) | **1,951.8 (1.30×)** | | |
| | HumanEval | 1,546.5 | 1,296.8 (0.84×) | 1,330.1 (0.86×) | **1,799.0 (1.16×)** | | |
| | MBPP | 1,507.7 | 1,314.9 (0.87×) | 1,361.3 (0.90×) | **1,886.8 (1.25×)** | | |
| | MT-Bench | 1,507.4 | 1,159.7 (0.77×) | 1,115.5 (0.74×) | **1,525.3 (1.01×)** | | |
| ## Citation | |
| If you find DFlash 2 useful, please cite: | |
| ```bibtex | |
| @misc{inco2026dflash2, | |
| title = {{DFlash 2: Keep Drafting Parallel}}, | |
| author = {{Inco AI}}, | |
| year = {2026}, | |
| month = {August}, | |
| url = {https://inco.ai/blog/dflash2/} | |
| } | |
| ``` | |
| Please also cite the original DFlash paper: | |
| ```bibtex | |
| @inproceedings{chen2026dflash, | |
| title = {{DFlash: Block Diffusion for Flash Speculative Decoding}}, | |
| author = {Chen, Jian and Liang, Yesheng and Liu, Zhijian}, | |
| booktitle = {International Conference on Machine Learning (ICML)}, | |
| year = {2026} | |
| } | |
| ``` | |
| </details> | |
| ## Preservation record | |
| Selected source bytes are verified against pinned metadata; the original eight graded mirrors also passed the previously recorded anonymous 1 MiB weight-prefix checks. This is not a full independent-device restore/load test. Hugging Face repository copies share a provider and do not guarantee permanent storage. | |
| [Collection](https://huggingface.co/collections/isilicon/mango-tested-models-and-preservation-mirrors-6ae3b73994c895ab42e360d2) · [Source manifest](SOURCE_MANIFEST.json) · [Original documentation](UPSTREAM_README.md) | |