Arete-Dflash2
DFlash2 draft model for speculative decoding, compatible with Arete-27B, Arete-27B-Uncensored, Qwen3.8-27B, and Qwen3.8-27B-Uncensored β meant to be used with Arete-27B / Qwen3.8-27B variants. Trained by SargeDev on the GB10 (2x GB10 cluster, vLLM 0.28).
What this is
A DFlash2 drafter: a small (1.92B) draft model that proposes tokens for a larger verifier (target) model. The verifier scores the drafts in one forward pass and accepts a prefix β lossless speedups, identical output.
In z-lab export format (DFlash2DraftModel + dflash_config), drop-in for
vLLM --speculative-config.
| repo | base | training |
|---|---|---|
SargeDev/Arete-Dflash2 |
incoai/Qwen3.8-27B-DFlash2 (Z-Lab DFlash2, 1.92B) | 7 epochs finetune on Arete-27B-Uncensored regen data |
Targets
Both drafters are drop-in compatible with all three (greedy losslessness 5/5 on every combo β verified):
| target | repo |
|---|---|
| Arete-27B-Uncensored (our uncensored build, bf16) | SargeDev/Arete-27B-Uncensored-V2 |
| Arete-27B (censored, original) | NousResearch/Arete-27B |
| Qwen3.8-27B | Qwen/Qwen3.8-27B |
Benchmarks
Accept length = mean accepted tokens per draft block (7-token blocks, greedy). Greedy losslessness: 5/5 prompts identical to the un-drafted target output.
Bench matrix (vLLM 0.28, GB10, single target serve)
| target | accept_len | draft rate |
|---|---|---|
| Arete-27B-Uncensored | 3.00 | 28.6% |
| Arete-27B (censored) | 2.97 | 28.2% |
| Qwen3.8-27B | 3.80 | 40.0% |
Live testing β Arete-27B-Uncensored bf16 TP2 (262k ctx, pen-testing workload)
Real-world serving with structured security content (pen-testing prompts): accept length up to 4.71 at 53.0% draft rate on structured windows, 2.6β3.9 accept length / 23β41% on general windows. The drafter's accept rate tracks how close the prompt is to its training style β structured technical content drafts markedly better than free-form prose.
Usage (vLLM)
vllm serve <target-model> \
--speculative-config '{"method": "dflash", "model": "SargeDev/Arete-Dflash2", "num_speculative_tokens": 7}'
Training data
6,903 rows generated on-policy from Arete-27B-Uncensored (loop rows
quarantined), hidden states extracted from layers 6/20/34/48/62/64 of the
target. The exact training rows are published:
SargeDev/arete-dflash2-trainset
(97.6 MB, jsonl: id, primary_id, source, input_ids, loss_mask,
text, metadata). Corpus lineage: SargeDev/jev-distill-corpus-v3.
Provenance
- Z-Lab drafter base: z-lab/Qwen3.8-27B-DFlash2
- incoai finetuned variant: incoai/Qwen3.8-27B-DFlash2
- Target: SargeDev/Arete-27B-Uncensored-V2 (our uncensored build of NousResearch/Arete-27B)
- Training corpus: SargeDev/arete-dflash2-trainset (
SargeDev/jev-distill-corpus-v3lineage) - Trainer: vllm-project/speculators 0.9.0a0, DFlash2 (dflash2) speculator type
License
Apache-2.0 (base drafter). Arete-27B is cc-by-nc-4.0; the uncensored SargeDev build follows the same license as its base.
- Downloads last month
- 3