Arete-Dflash2 / README.md
SargeDev's picture
Upload README.md with huggingface_hub
4f287ea verified
|
Raw History Blame Contribute Delete
3.84 kB
metadata
license: apache-2.0
tags:
  - dflash2
  - speculative-decoding
  - draft-model
  - qwen3
base_model:
  - incoai/Qwen3.8-27B-DFlash2

Arete-Dflash2

DFlash2 draft model for speculative decoding, compatible with Arete-27B, Arete-27B-Uncensored, Qwen3.8-27B, and Qwen3.8-27B-Uncensored — meant to be used with Arete-27B / Qwen3.8-27B variants. Trained by SargeDev on the GB10 (2x GB10 cluster, vLLM 0.28).

What this is

A DFlash2 drafter: a small (1.92B) draft model that proposes tokens for a larger verifier (target) model. The verifier scores the drafts in one forward pass and accepts a prefix — lossless speedups, identical output.

In z-lab export format (DFlash2DraftModel + dflash_config), drop-in for vLLM --speculative-config.

repo base training
SargeDev/Arete-Dflash2 incoai/Qwen3.8-27B-DFlash2 (Z-Lab DFlash2, 1.92B) 7 epochs finetune on Arete-27B-Uncensored regen data

Targets

Both drafters are drop-in compatible with all three (greedy losslessness 5/5 on every combo — verified):

target repo
Arete-27B-Uncensored (our uncensored build, bf16) SargeDev/Arete-27B-Uncensored-V2
Arete-27B (censored, original) NousResearch/Arete-27B
Qwen3.8-27B Qwen/Qwen3.8-27B

Benchmarks

Accept length = mean accepted tokens per draft block (7-token blocks, greedy). Greedy losslessness: 5/5 prompts identical to the un-drafted target output.

Bench matrix (vLLM 0.28, GB10, single target serve)

target accept_len draft rate
Arete-27B-Uncensored 3.00 28.6%
Arete-27B (censored) 2.97 28.2%
Qwen3.8-27B 3.80 40.0%

Live testing — Arete-27B-Uncensored bf16 TP2 (262k ctx, pen-testing workload)

Real-world serving with structured security content (pen-testing prompts): accept length up to 4.71 at 53.0% draft rate on structured windows, 2.6–3.9 accept length / 23–41% on general windows. The drafter's accept rate tracks how close the prompt is to its training style — structured technical content drafts markedly better than free-form prose.

Usage (vLLM)

vllm serve <target-model> \
  --speculative-config '{"method": "dflash", "model": "SargeDev/Arete-Dflash2", "num_speculative_tokens": 7}'

Training data

6,903 rows generated on-policy from Arete-27B-Uncensored (loop rows quarantined), hidden states extracted from layers 6/20/34/48/62/64 of the target. The exact training rows are published: SargeDev/arete-dflash2-trainset (97.6 MB, jsonl: id, primary_id, source, input_ids, loss_mask, text, metadata). Corpus lineage: SargeDev/jev-distill-corpus-v3.

Provenance

License

Apache-2.0 (base drafter). Arete-27B is cc-by-nc-4.0; the uncensored SargeDev build follows the same license as its base.