File size: 3,838 Bytes
baf8dad
 
 
 
 
 
 
 
 
 
 
72e0303
baf8dad
72e0303
 
 
 
baf8dad
 
 
 
 
 
 
14b30a9
 
baf8dad
 
 
72e0303
baf8dad
 
 
 
 
 
 
 
 
 
 
 
72e0303
baf8dad
 
 
 
72e0303
baf8dad
72e0303
 
 
 
 
 
 
 
 
 
 
 
 
baf8dad
 
 
 
 
72e0303
baf8dad
 
 
 
 
 
4f287ea
 
 
 
baf8dad
 
 
 
 
 
4f287ea
baf8dad
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
---
license: apache-2.0
tags:
  - dflash2
  - speculative-decoding
  - draft-model
  - qwen3
base_model:
  - incoai/Qwen3.8-27B-DFlash2
---

# Arete-Dflash2

DFlash2 draft model for speculative decoding, compatible with **Arete-27B**,
**Arete-27B-Uncensored**, **Qwen3.8-27B**, and **Qwen3.8-27B-Uncensored** —
meant to be used with Arete-27B / Qwen3.8-27B variants. Trained by SargeDev
on the GB10 (2x GB10 cluster, vLLM 0.28).

## What this is

A DFlash2 drafter: a small (1.92B) draft model that proposes tokens for a
larger verifier (target) model. The verifier scores the drafts in one forward
pass and accepts a prefix — lossless speedups, identical output.

In z-lab export format (`DFlash2DraftModel` + `dflash_config`), drop-in for
vLLM `--speculative-config`.

| repo | base | training |
|---|---|---|
| `SargeDev/Arete-Dflash2` | [incoai/Qwen3.8-27B-DFlash2](https://huggingface.co/incoai/Qwen3.8-27B-DFlash2) (Z-Lab DFlash2, 1.92B) | 7 epochs finetune on Arete-27B-Uncensored regen data |

## Targets

Both drafters are **drop-in compatible** with all three (greedy losslessness
5/5 on every combo — verified):

| target | repo |
|---|---|
| Arete-27B-Uncensored (our uncensored build, bf16) | [SargeDev/Arete-27B-Uncensored-V2](https://huggingface.co/SargeDev/Arete-27B-Uncensored-V2) |
| Arete-27B (censored, original) | [NousResearch/Arete-27B](https://huggingface.co/NousResearch/Arete-27B) |
| Qwen3.8-27B | [Qwen/Qwen3.8-27B](https://huggingface.co/Qwen/Qwen3.8-27B) |

## Benchmarks

Accept length = mean accepted tokens per draft block (7-token blocks, greedy).
Greedy losslessness: 5/5 prompts identical to the un-drafted target output.

### Bench matrix (vLLM 0.28, GB10, single target serve)

| target | accept_len | draft rate |
|---|---|---|
| **Arete-27B-Uncensored** | **3.00** | **28.6%** |
| **Arete-27B (censored)** | **2.97** | **28.2%** |
| **Qwen3.8-27B** | **3.80** | **40.0%** |

### Live testing — Arete-27B-Uncensored bf16 TP2 (262k ctx, pen-testing workload)

Real-world serving with structured security content (pen-testing prompts):
accept length up to **4.71** at **53.0%** draft rate on structured windows,
2.6–3.9 accept length / 23–41% on general windows. The drafter's accept rate
tracks how close the prompt is to its training style — structured technical
content drafts markedly better than free-form prose.

## Usage (vLLM)

```bash
vllm serve <target-model> \
  --speculative-config '{"method": "dflash", "model": "SargeDev/Arete-Dflash2", "num_speculative_tokens": 7}'
```

## Training data

6,903 rows generated on-policy from Arete-27B-Uncensored (loop rows
quarantined), hidden states extracted from layers 6/20/34/48/62/64 of the
target. The exact training rows are published:
**[SargeDev/arete-dflash2-trainset](https://huggingface.co/datasets/SargeDev/arete-dflash2-trainset)**
(97.6 MB, jsonl: `id`, `primary_id`, `source`, `input_ids`, `loss_mask`,
`text`, `metadata`). Corpus lineage: `SargeDev/jev-distill-corpus-v3`.

## Provenance

- Z-Lab drafter base: [z-lab/Qwen3.8-27B-DFlash2](https://huggingface.co/z-lab/Qwen3.8-27B-DFlash2)
- incoai finetuned variant: [incoai/Qwen3.8-27B-DFlash2](https://huggingface.co/incoai/Qwen3.8-27B-DFlash2)
- Target: [SargeDev/Arete-27B-Uncensored-V2](https://huggingface.co/SargeDev/Arete-27B-Uncensored-V2) (our uncensored build of [NousResearch/Arete-27B](https://huggingface.co/NousResearch/Arete-27B))
- Training corpus: [SargeDev/arete-dflash2-trainset](https://huggingface.co/datasets/SargeDev/arete-dflash2-trainset) (`SargeDev/jev-distill-corpus-v3` lineage)
- Trainer: [vllm-project/speculators](https://github.com/vllm-project/speculators) 0.9.0a0, DFlash2 (dflash2) speculator type

## License

Apache-2.0 (base drafter). Arete-27B is cc-by-nc-4.0; the uncensored
SargeDev build follows the same license as its base.