Publish detailed Mango model card, measured results and visual analysis
Browse files- BENCHMARK_METHODOLOGY.md +68 -0
- MODEL_CARD_STATUS.json +30 -0
- README.md +245 -10
- assets/mango-header.png +0 -0
BENCHMARK_METHODOLOGY.md
ADDED
|
@@ -0,0 +1,68 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Mango benchmark methodology
|
| 2 |
+
|
| 3 |
+
## Scope and hardware
|
| 4 |
+
|
| 5 |
+
These are local measurements of exact checkpoint/quant/engine configurations. The matched cohort uses
|
| 6 |
+
202 questions selected with stride 2 within each task from a frozen 404-item suite, plus 12 authored
|
| 7 |
+
agent-loop cases. Generation seed is 41. Runs used a single NVIDIA RTX PRO 6000 Blackwell Server Edition
|
| 8 |
+
(reported 97,887 MiB VRAM), with CPU/RAM offload where needed. Some cohorts span restored MoLab hosts.
|
| 9 |
+
This is a convenience sample with local graders, not an official full-benchmark leaderboard.
|
| 10 |
+
|
| 11 |
+
## Tasks and weighting
|
| 12 |
+
|
| 13 |
+
| Domain | Weight | Tasks in this cohort |
|
| 14 |
+
| --- | ---: | --- |
|
| 15 |
+
| Social / emotional understanding | 30% | EQ-Bench v2 (30), OpenToM labels (18) |
|
| 16 |
+
| Cybersecurity | 20% | CTI MCQ (20), CTI root cause/CWE (15), CyberMetric (20) |
|
| 17 |
+
| Coding | 10% | HumanEval+ local tests (20) |
|
| 18 |
+
| Agentic | 30% | BFCL-style local function-call checks (40), agent loop (12) |
|
| 19 |
+
| Instruction following | 5% | IFEval strict prompt-level checks (20) |
|
| 20 |
+
| General knowledge / math | 5% | MMLU-Pro (14), MATH-500 levels 4–5 (5) |
|
| 21 |
+
|
| 22 |
+
First average scores within each task. Then equally average the task means inside each domain.
|
| 23 |
+
The final score is the sum of domain means multiplied by their weights. Unequal task sizes therefore
|
| 24 |
+
do not make larger tasks dominate a domain. EQ is a continuous upstream-formula score; it is not an
|
| 25 |
+
accuracy or a count of correct answers. The agentic domain is not identical to the 12-case pass rate.
|
| 26 |
+
|
| 27 |
+
## Graders and intervals
|
| 28 |
+
|
| 29 |
+
Multiple-choice, math, label and CWE answers use deterministic extraction/checks. Coding runs the
|
| 30 |
+
repository's local HumanEval+ test subset; BFCL uses its local AST-style matcher, not the full official
|
| 31 |
+
pipeline. IFEval uses Google's strict reference checker at commit
|
| 32 |
+
700de9630a39cf1c0ded1ea7d1092ec0f467c7a7 with tokenizer resources restored.
|
| 33 |
+
|
| 34 |
+
Agent passes require completion, full evidence coverage, no unauthorized calls, correct pricing totals
|
| 35 |
+
where applicable, and no infrastructure error. These are mechanical checks; broader final-answer
|
| 36 |
+
semantics, open-ended writing, live web behavior and GUI competence are outside this result set.
|
| 37 |
+
|
| 38 |
+
Binary task intervals are Wilson 95% intervals. Continuous EQ intervals bootstrap the mean. Overall
|
| 39 |
+
model differences use 10,000 paired bootstrap resamples within each task, seed 41, with fixed task/domain
|
| 40 |
+
weights. An interval crossing zero does not establish a quality winner. These intervals describe the
|
| 41 |
+
released sample; they are not population-wide guarantees.
|
| 42 |
+
|
| 43 |
+
## Performance definitions
|
| 44 |
+
|
| 45 |
+
Workload minutes sum the saved suite-request durations and all 12 agent-case durations. They exclude
|
| 46 |
+
model loading, restarts, host-wipe downtime and idle gaps. This measures recorded task time, not wall
|
| 47 |
+
time from first launch to final archive. Tasks/minute is 214 divided by workload minutes.
|
| 48 |
+
|
| 49 |
+
Latency and time-to-first-token (TTFT) summaries below cover suite requests only. P50/P95 use linear
|
| 50 |
+
interpolation at (n−1)×quantile; missing TTFT values are excluded and coverage is reported. The median
|
| 51 |
+
reported output-token/request-second rate uses completion-token usage divided by request duration for
|
| 52 |
+
requests reporting more than 50 completion tokens. It includes request overhead and is not native decode
|
| 53 |
+
throughput. Token accounting and reasoning/output lengths differ between engines; task time is the
|
| 54 |
+
more useful cross-engine comparison. Single-request results do not establish concurrent-serving capacity.
|
| 55 |
+
|
| 56 |
+
## Configuration limits
|
| 57 |
+
|
| 58 |
+
Thinking was enabled. Native effort vocabularies and vendor sampling differ. The historical mixed GLM
|
| 59 |
+
used an internal 8192-token thinking cap and is retained as a capped, unavailable historical baseline.
|
| 60 |
+
The newer Bolding GLM uses an uncapped native setting. Streamed context headers were absent in these
|
| 61 |
+
archives; per-card context descriptions rely on recorded native configuration/preflight evidence.
|
| 62 |
+
Context ceilings, vision/audio quality, refusal behavior, safety retention, GPQA, expanded planning,
|
| 63 |
+
search, sandbox and browser results are not established by these 202+12 measurements.
|
| 64 |
+
|
| 65 |
+
The frozen suite SHA-256 is
|
| 66 |
+
6ba848464b4685e496b5ac83a115e7efb3cf0c8f7a6afcdc49553af49a5df421.
|
| 67 |
+
BENCHMARK_REPORT.json records exact scores, uncertainty, raw artifact hashes, grader hashes and timing
|
| 68 |
+
coverage without publishing prompts, generated answers, credentials or real user traffic.
|
MODEL_CARD_STATUS.json
ADDED
|
@@ -0,0 +1,30 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"schema": 1,
|
| 3 |
+
"state": "documentation_refresh",
|
| 4 |
+
"source_repo": "z-lab/Qwen3.8-27B-DFlash2",
|
| 5 |
+
"source_revision": "50307d4c4cde6860d4eee73e2547cd786fe8e8a4",
|
| 6 |
+
"prior_repository_commit": "c8dea2bcc59f56c68cc1374ccf0b7b20d95d763c",
|
| 7 |
+
"reviewed_plan_sha256": "24549c3245fa5a4c7c8d74fb542e5f9d516388828dd4d178e0c27df3b70f98ff",
|
| 8 |
+
"created_utc": "2026-10-29T23:50:41.206319+00:00",
|
| 9 |
+
"scope": "Card Markdown, aggregate benchmark data, methodology and original visual assets only.",
|
| 10 |
+
"files": [
|
| 11 |
+
{
|
| 12 |
+
"path": "BENCHMARK_METHODOLOGY.md",
|
| 13 |
+
"bytes": 4369,
|
| 14 |
+
"sha256": "6bf76af964a956c9a61d27665766bc2ed014b99a21cd6a340a64950a32f2da6d",
|
| 15 |
+
"git_blob": "52d04ab158058ac6799686d69f77a088777383b6"
|
| 16 |
+
},
|
| 17 |
+
{
|
| 18 |
+
"path": "README.md",
|
| 19 |
+
"bytes": 12034,
|
| 20 |
+
"sha256": "da0e39044a27985386281377b3b3a89f72826e4a5aefbdfea126aab4289d7c75",
|
| 21 |
+
"git_blob": "219f14b1c1c8413da3e3daffbe25d2aa3fb3340d"
|
| 22 |
+
},
|
| 23 |
+
{
|
| 24 |
+
"path": "assets/mango-header.png",
|
| 25 |
+
"bytes": 26933,
|
| 26 |
+
"sha256": "c703d76b3600687e0ddd2c4aaaa030affb45acde310833c98ac23caf28bafa80",
|
| 27 |
+
"git_blob": "2095857408d34d49a92719eacaadacd71b97b7dd"
|
| 28 |
+
}
|
| 29 |
+
]
|
| 30 |
+
}
|
README.md
CHANGED
|
@@ -1,22 +1,257 @@
|
|
| 1 |
---
|
| 2 |
license: apache-2.0
|
| 3 |
-
base_model:
|
|
|
|
| 4 |
tags:
|
| 5 |
-
- preservation-mirror
|
| 6 |
- mango
|
|
|
|
|
|
|
|
|
|
| 7 |
---
|
| 8 |
-
# Preservation mirror: z-lab/Qwen3.8-27B-DFlash2
|
| 9 |
|
| 10 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 11 |
|
| 12 |
-
|
| 13 |
|
| 14 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 15 |
|
| 16 |
-
|
| 17 |
|
| 18 |
-
|
| 19 |
|
| 20 |
-
|
| 21 |
|
| 22 |
-
|
|
|
|
| 1 |
---
|
| 2 |
license: apache-2.0
|
| 3 |
+
base_model: z-lab/Qwen3.8-27B-DFlash2
|
| 4 |
+
inference: false
|
| 5 |
tags:
|
|
|
|
| 6 |
- mango
|
| 7 |
+
- preservation-mirror
|
| 8 |
+
- qwen-27b
|
| 9 |
+
- auxiliary-component
|
| 10 |
---
|
|
|
|
| 11 |
|
| 12 |
+

|
| 13 |
+
|
| 14 |
+
# Qwen DFlash2
|
| 15 |
+
|
| 16 |
+
**Block-diffusion draft · z-lab** · Runtime companion
|
| 17 |
+
|
| 18 |
+
z-lab's Qwen3.8 27B DFlash2 draft model, containing its config and safetensors weights. It drafts tokens for a target model to verify and is not a standalone language model.
|
| 19 |
+
|
| 20 |
+
[Benchmark results](#mango-evaluation) · [Files](#files-and-integrity) · [Download](#download-and-use) · [Runtime](#runtime-and-compatibility) · [Provenance](#provenance-and-license) · [Full collection](https://huggingface.co/collections/isilicon/mango-tested-models-and-preservation-mirrors-6ae3b73994c895ab42e360d2)
|
| 21 |
+
|
| 22 |
+
Preserved and documented by **isilicon / Mango**. The original model, modification and quantization remain the upstream creators' work. This repository adds attributed preservation and exact-configuration evaluation, with the selected source bytes unchanged.
|
| 23 |
+
|
| 24 |
+
## At a glance
|
| 25 |
+
|
| 26 |
+
| Property | Recorded value |
|
| 27 |
+
| --- | --- |
|
| 28 |
+
| Artifact | safetensors draft |
|
| 29 |
+
| Selected source payload | 3.85 GB / 3.58 GiB |
|
| 30 |
+
| Primary selected files | 2 |
|
| 31 |
+
| Evaluation status | Runtime companion |
|
| 32 |
+
| Source publisher | z-lab |
|
| 33 |
+
| Source revision | `50307d4c4cde6860d4eee73e2547cd786fe8e8a4` |
|
| 34 |
+
| Declared license | apache-2.0 |
|
| 35 |
+
|
| 36 |
+
GB is decimal and GiB is binary. Payload size is disk/storage size, not a claim that the model fits in the same amount of VRAM. Runtime, KV cache, working memory and companions add their own requirements.
|
| 37 |
+
|
| 38 |
+
## Mango evaluation
|
| 39 |
+
|
| 40 |
+
**No standalone Mango quality score has been published for these exact files.** This is a companion artifact; the consuming model's result belongs to the full target/runtime configuration.
|
| 41 |
+
|
| 42 |
+
### Related configurations measured separately
|
| 43 |
+
|
| 44 |
+
These results describe the linked configurations, **not this artifact**. A change of quant, modification lineage, draft, engine or tokenizer can change the result.
|
| 45 |
+
|
| 46 |
+
| Different configuration | Its weighted score | Its workload min |
|
| 47 |
+
| --- | --- | --- |
|
| 48 |
+
| [Qwen3.8 27B · MLX affine 4-bit · OrcaRouter](https://huggingface.co/isilicon/mango-qwen38-27b-uncensored-mlx4) | 80.64 | 23.98 |
|
| 49 |
+
|
| 50 |
+
A completed score requires a native preflight, matching frozen prompts, 202 suite answers, 12 agent cases, archive verification and the same reference graders. The [benchmark methodology](BENCHMARK_METHODOLOGY.md) describes the protocol; publisher-side results are retained separately below.
|
| 51 |
+
|
| 52 |
+
## Runtime and compatibility
|
| 53 |
+
|
| 54 |
+
A speculative-decoding runtime must support DFlash2 and the target model. The measured Mango workload combines the target, draft and engine; this mirror alone has no separate task-quality or speed score.
|
| 55 |
+
|
| 56 |
+
Context and output behavior belong to the target/runtime configuration.
|
| 57 |
+
|
| 58 |
+
### Companion artifacts
|
| 59 |
+
|
| 60 |
+
| Consuming model | Connection |
|
| 61 |
+
| --- | --- |
|
| 62 |
+
| [Qwen3.8 27B · MLX affine 4-bit · OrcaRouter](https://huggingface.co/isilicon/mango-qwen38-27b-uncensored-mlx4) | Use the compatible parent runtime and exact target revision |
|
| 63 |
+
|
| 64 |
+
## Files and integrity
|
| 65 |
+
|
| 66 |
+
| Selected file · original path | GB | Identity prefix |
|
| 67 |
+
| --- | --- | --- |
|
| 68 |
+
| [config.json](https://huggingface.co/isilicon/mango-dflash2/resolve/main/config.json) | 0.00000 | `79279cc5665bced6…` (Git blob) |
|
| 69 |
+
| [model.safetensors](https://huggingface.co/isilicon/mango-dflash2/resolve/main/model.safetensors) | 3.84882 | `67fc76d68dc5a941…` (SHA-256) |
|
| 70 |
+
|
| 71 |
+
Full paths, byte sizes and complete SHA-256/Git blob identities are in [SOURCE_MANIFEST.json](SOURCE_MANIFEST.json). Available original README/license/notice files are also preserved. [MIRROR_STATUS.json](MIRROR_STATUS.json) records the earlier source-file verification; MODEL_CARD_STATUS.json records this documentation refresh separately.
|
| 72 |
+
|
| 73 |
+
## Download and use
|
| 74 |
+
|
| 75 |
+
The commands pin the already-verified mirror weight revision. Documentation can advance on main without changing that preserved weight snapshot. Keep original subfolders and all shards.
|
| 76 |
+
|
| 77 |
+
```bash
|
| 78 |
+
python -m pip install -U huggingface_hub
|
| 79 |
+
|
| 80 |
+
hf download isilicon/mango-dflash2 \
|
| 81 |
+
config.json \
|
| 82 |
+
model.safetensors \
|
| 83 |
+
--revision c8dea2bcc59f56c68cc1374ccf0b7b20d95d763c \
|
| 84 |
+
--local-dir ./weights/dflash2
|
| 85 |
+
```
|
| 86 |
+
|
| 87 |
+
### Load example and setup scope
|
| 88 |
+
|
| 89 |
+
This component has no independent chat launch command. Install it through the consuming model's compatible runtime and companion layout.
|
| 90 |
+
|
| 91 |
+
## Provenance and license
|
| 92 |
+
|
| 93 |
+
The source credits the DFlash2 work and points to the incoai release. Its acceptance-length and SGLang/vLLM throughput tables concern the publisher's configurations. Mango used this companion with its Qwen MLX4 target through TensorFold.
|
| 94 |
+
|
| 95 |
+
| Responsibility / identity | Attribution |
|
| 96 |
+
| --- | --- |
|
| 97 |
+
| Mirror / local evaluation | [isilicon](https://huggingface.co/isilicon) · Mango |
|
| 98 |
+
| Immediate source | [z-lab/Qwen3.8-27B-DFlash2](https://huggingface.co/z-lab/Qwen3.8-27B-DFlash2/tree/50307d4c4cde6860d4eee73e2547cd786fe8e8a4) |
|
| 99 |
+
| Pinned source commit | `50307d4c4cde6860d4eee73e2547cd786fe8e8a4` |
|
| 100 |
+
| Declared upstream base(s) | [Qwen/Qwen3.8-27B](https://huggingface.co/Qwen/Qwen3.8-27B) |
|
| 101 |
+
| Source-declared license | apache-2.0 |
|
| 102 |
+
|
| 103 |
+
Original notices, base-model conditions and the publisher's stated limitations still apply. Downloadable preservation does not add new license rights. Source-described refusal modification and publisher-side benchmark claims have not been independently re-measured by this Mango task cohort.
|
| 104 |
+
|
| 105 |
+
### Original publisher documentation
|
| 106 |
+
|
| 107 |
+
The [unchanged upstream README](UPSTREAM_README.md) and [pinned source card](https://huggingface.co/z-lab/Qwen3.8-27B-DFlash2/blob/50307d4c4cde6860d4eee73e2547cd786fe8e8a4/README.md) retain the original usage guidance, technical discussion, attribution, caveats and citations.
|
| 108 |
+
|
| 109 |
+
<details>
|
| 110 |
+
<summary>Read the complete original publisher card</summary>
|
| 111 |
+
|
| 112 |
+
**Source reference.** The following is the original publisher's documentation. Its other quants, full-model files, context claims and benchmark tables can exceed the selection in this mirror. Relative asset/file URLs are qualified to the pinned original source; publisher measurements remain separate from Mango measurements.
|
| 113 |
+
|
| 114 |
+
# Qwen3.8-27B-DFlash2
|
| 115 |
+
|
| 116 |
+
[Blog](https://inco.ai/blog/dflash2/) | [GitHub](https://github.com/z-lab/dflash)
|
| 117 |
+
|
| 118 |
+
This repository contains the DFlash 2 draft model for
|
| 119 |
+
[`Qwen/Qwen3.8-27B`](https://huggingface.co/Qwen/Qwen3.8-27B).
|
| 120 |
+
It is not a standalone language model: it runs inside a speculative
|
| 121 |
+
decoding server and drafts tokens for the target model to verify. This repository is a mirror
|
| 122 |
+
of [`incoai/Qwen3.8-27B-DFlash2`](https://huggingface.co/incoai/Qwen3.8-27B-DFlash2).
|
| 123 |
+
|
| 124 |
+
DFlash 2 is a block-diffusion drafter for speculative decoding. It predicts
|
| 125 |
+
a whole block of tokens in a single pass and keeps the top candidates at
|
| 126 |
+
every position. A lightweight selector then traces one coherent path through them.
|
| 127 |
+
Two-tap dynamic convolutions in the backbone keep the draft from decaying
|
| 128 |
+
toward the end of the block. Decoding is lossless: greedy output
|
| 129 |
+
matches the target model exactly, and sampling preserves its distribution.
|
| 130 |
+
|
| 131 |
+
<div align="center">
|
| 132 |
+
<img src="https://huggingface.co/z-lab/Qwen3.8-27B-DFlash2/resolve/50307d4c4cde6860d4eee73e2547cd786fe8e8a4/assets/dflash2-figure.png" alt="DFlash 2: parallel block drafting with a candidate path selector" width="100%">
|
| 133 |
+
</div>
|
| 134 |
+
|
| 135 |
+
## Quick Start
|
| 136 |
+
|
| 137 |
+
Serve with [SGLang](https://github.com/sgl-project/sglang):
|
| 138 |
+
|
| 139 |
+
```bash
|
| 140 |
+
pip install "sglang[all] @ git+https://github.com/sgl-project/sglang.git#subdirectory=python"
|
| 141 |
+
|
| 142 |
+
python -m sglang.launch_server \
|
| 143 |
+
--model-path Qwen/Qwen3.8-27B \
|
| 144 |
+
--speculative-algorithm DFLASH \
|
| 145 |
+
--speculative-draft-model-path incoai/Qwen3.8-27B-DFlash2 \
|
| 146 |
+
--speculative-num-draft-tokens 8
|
| 147 |
+
```
|
| 148 |
+
|
| 149 |
+
Or with [vLLM](https://github.com/vllm-project/vllm):
|
| 150 |
+
|
| 151 |
+
```bash
|
| 152 |
+
pip install -U "vllm @ git+https://github.com/vllm-project/vllm.git@refs/pull/52816/head"
|
| 153 |
+
|
| 154 |
+
vllm serve Qwen/Qwen3.8-27B \
|
| 155 |
+
--speculative-config '{
|
| 156 |
+
"method": "dflash",
|
| 157 |
+
"model": "incoai/Qwen3.8-27B-DFlash2",
|
| 158 |
+
"num_speculative_tokens": 7
|
| 159 |
+
}'
|
| 160 |
+
```
|
| 161 |
+
|
| 162 |
+
See the [blog post](https://inco.ai/blog/dflash2/) for other engines and more details.
|
| 163 |
+
|
| 164 |
+
## Evaluation
|
| 165 |
+
|
| 166 |
+
- Runtime: SGLang on one NVIDIA H200, with FlashAttention 3 for target and draft attention
|
| 167 |
+
- Speculation block size: 8 (7 draft tokens per verification step)
|
| 168 |
+
- Sampling: Qwen3.8's officially recommended parameters (temperature 1.0, top-p 0.95, top-k 20), with `xhigh` reasoning effort
|
| 169 |
+
- Maximum new tokens: 4096
|
| 170 |
+
- Prompts: benchmark formatting from [`z-lab/dflash`](https://github.com/z-lab/dflash)
|
| 171 |
+
|
| 172 |
+
We compare autoregressive decoding, Qwen3.8's built-in seven-token MTP,
|
| 173 |
+
a community DSpark drafter
|
| 174 |
+
([`RadixArk/Qwen3.8-27B-DSpark`](https://huggingface.co/RadixArk/Qwen3.8-27B-DSpark)),
|
| 175 |
+
and DFlash 2. All speculative methods propose seven draft tokens per
|
| 176 |
+
verification step.
|
| 177 |
+
|
| 178 |
+
### Acceptance Length
|
| 179 |
+
|
| 180 |
+
Acceptance length is the per-request mean of completion tokens divided by verification steps.
|
| 181 |
+
Higher is better.
|
| 182 |
+
|
| 183 |
+
| Task | MTP | DSpark | DFlash 2 |
|
| 184 |
+
| :--- | ---: | ---: | ---: |
|
| 185 |
+
| GSM8K | 5.02 | 4.36 | **5.46** |
|
| 186 |
+
| MATH-500 | 4.72 | 3.92 | **5.28** |
|
| 187 |
+
| HumanEval | 3.91 | 3.30 | **4.39** |
|
| 188 |
+
| MBPP | 3.99 | 3.51 | **4.79** |
|
| 189 |
+
| MT-Bench | 3.74 | 3.01 | **4.10** |
|
| 190 |
+
|
| 191 |
+
### Throughput
|
| 192 |
+
|
| 193 |
+
Throughput is total output tokens divided by end-to-end wall time.
|
| 194 |
+
Each cell shows `output tok/s (speedup vs. autoregressive)`.
|
| 195 |
+
|
| 196 |
+
#### Concurrency 1
|
| 197 |
+
|
| 198 |
+
| Task | Autoregressive | MTP | DSpark | DFlash 2 |
|
| 199 |
+
| :--- | ---: | ---: | ---: | ---: |
|
| 200 |
+
| GSM8K | 68.9 | 178.5 (2.59×) | 185.3 (2.69×) | **236.1 (3.43×)** |
|
| 201 |
+
| MATH-500 | 69.0 | 172.8 (2.51×) | 174.5 (2.53×) | **230.7 (3.34×)** |
|
| 202 |
+
| HumanEval | 69.0 | 151.9 (2.20×) | 159.9 (2.32×) | **214.6 (3.11×)** |
|
| 203 |
+
| MBPP | 69.0 | 153.1 (2.22×) | 163.3 (2.37×) | **226.9 (3.29×)** |
|
| 204 |
+
| MT-Bench | 68.9 | 134.9 (1.96×) | 137.6 (2.00×) | **184.0 (2.67×)** |
|
| 205 |
+
|
| 206 |
+
#### Concurrency 8
|
| 207 |
+
|
| 208 |
+
| Task | Autoregressive | MTP | DSpark | DFlash 2 |
|
| 209 |
+
| :--- | ---: | ---: | ---: | ---: |
|
| 210 |
+
| GSM8K | 467.2 | 1,022.1 (2.19×) | 1,040.8 (2.23×) | **1,328.7 (2.84×)** |
|
| 211 |
+
| MATH-500 | 480.0 | 1,023.5 (2.13×) | 1,025.8 (2.14×) | **1,368.3 (2.85×)** |
|
| 212 |
+
| HumanEval | 483.4 | 934.2 (1.93×) | 956.5 (1.98×) | **1,291.5 (2.67×)** |
|
| 213 |
+
| MBPP | 478.0 | 938.1 (1.96×) | 974.1 (2.04×) | **1,328.0 (2.78×)** |
|
| 214 |
+
| MT-Bench | 480.5 | 835.2 (1.74×) | 802.3 (1.67×) | **1,090.2 (2.27×)** |
|
| 215 |
+
|
| 216 |
+
#### Concurrency 32
|
| 217 |
+
|
| 218 |
+
| Task | Autoregressive | MTP | DSpark | DFlash 2 |
|
| 219 |
+
| :--- | ---: | ---: | ---: | ---: |
|
| 220 |
+
| GSM8K | 1,329.8 | 1,381.1 (1.04×) | 1,506.5 (1.13×) | **1,922.5 (1.45×)** |
|
| 221 |
+
| MATH-500 | 1,505.8 | 1,415.6 (0.94×) | 1,429.0 (0.95×) | **1,951.8 (1.30×)** |
|
| 222 |
+
| HumanEval | 1,546.5 | 1,296.8 (0.84×) | 1,330.1 (0.86×) | **1,799.0 (1.16×)** |
|
| 223 |
+
| MBPP | 1,507.7 | 1,314.9 (0.87×) | 1,361.3 (0.90×) | **1,886.8 (1.25×)** |
|
| 224 |
+
| MT-Bench | 1,507.4 | 1,159.7 (0.77×) | 1,115.5 (0.74×) | **1,525.3 (1.01×)** |
|
| 225 |
+
|
| 226 |
+
## Citation
|
| 227 |
+
|
| 228 |
+
If you find DFlash 2 useful, please cite:
|
| 229 |
+
|
| 230 |
+
```bibtex
|
| 231 |
+
@misc{inco2026dflash2,
|
| 232 |
+
title = {{DFlash 2: Keep Drafting Parallel}},
|
| 233 |
+
author = {{Inco AI}},
|
| 234 |
+
year = {2026},
|
| 235 |
+
month = {August},
|
| 236 |
+
url = {https://inco.ai/blog/dflash2/}
|
| 237 |
+
}
|
| 238 |
+
```
|
| 239 |
|
| 240 |
+
Please also cite the original DFlash paper:
|
| 241 |
|
| 242 |
+
```bibtex
|
| 243 |
+
@inproceedings{chen2026dflash,
|
| 244 |
+
title = {{DFlash: Block Diffusion for Flash Speculative Decoding}},
|
| 245 |
+
author = {Chen, Jian and Liang, Yesheng and Liu, Zhijian},
|
| 246 |
+
booktitle = {International Conference on Machine Learning (ICML)},
|
| 247 |
+
year = {2026}
|
| 248 |
+
}
|
| 249 |
+
```
|
| 250 |
|
| 251 |
+
</details>
|
| 252 |
|
| 253 |
+
## Preservation record
|
| 254 |
|
| 255 |
+
Selected source bytes are verified against pinned metadata; the eight graded mirrors also passed the previously recorded anonymous 1 MiB weight-prefix checks. This is not a full independent-device restore/load test. Hugging Face repository copies share a provider and do not guarantee permanent storage.
|
| 256 |
|
| 257 |
+
[Collection](https://huggingface.co/collections/isilicon/mango-tested-models-and-preservation-mirrors-6ae3b73994c895ab42e360d2) · [Source manifest](SOURCE_MANIFEST.json) · [Original documentation](UPSTREAM_README.md)
|
assets/mango-header.png
ADDED
|
|