SargeDev commited on
Commit
72e0303
·
verified ·
1 Parent(s): 1b75a33

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +22 -15
README.md CHANGED
@@ -9,11 +9,12 @@ base_model:
9
  - incoai/Qwen3.8-27B-DFlash2
10
  ---
11
 
12
- # Arete DFlash2 Drafter
13
 
14
- DFlash2 draft model for speculative decoding, built for **Arete-27B** and
15
- **Arete-27B-Uncensored** — meant to be used with Arete-27B variants. Trained
16
- by SargeDev on the GB10 (2x GB10 cluster, vLLM 0.28).
 
17
 
18
  ## What this is
19
 
@@ -26,7 +27,7 @@ vLLM `--speculative-config`.
26
 
27
  | repo | base | training |
28
  |---|---|---|
29
- | `SargeDev/arete-dflash2-incoai` | [incoai/Qwen3.8-27B-DFlash2](https://huggingface.co/incoai/Qwen3.8-27B-DFlash2) (Z-Lab DFlash2, 1.92B) | 7 epochs finetune on Arete-27B-Uncensored regen data |
30
 
31
  ## Targets
32
 
@@ -39,26 +40,32 @@ Both drafters are **drop-in compatible** with all three (greedy losslessness
39
  | Arete-27B (censored, original) | [NousResearch/Arete-27B](https://huggingface.co/NousResearch/Arete-27B) |
40
  | Qwen3.8-27B | [Qwen/Qwen3.8-27B](https://huggingface.co/Qwen/Qwen3.8-27B) |
41
 
42
- ## Benchmarks (vLLM 0.28, GB10, single target serve)
43
 
44
  Accept length = mean accepted tokens per draft block (7-token blocks, greedy).
45
  Greedy losslessness: 5/5 prompts identical to the un-drafted target output.
46
 
47
- | drafter | target | accept_len | draft rate |
48
- |---|---|---|---|
49
- | **incoai** | **Arete-27B-Uncensored** | **3.00** | **28.6%** |
50
- | **incoai** | **Arete-27B (censored)** | **2.97** | **28.2%** |
51
- | **incoai** | **Qwen3.8-27B** | **3.80** | **40.0%** |
52
 
53
- The incoai drafter (Z-Lab base finetuned on our regen data) is the strongest
54
- combo everywhere — including on the base Qwen3.8-27B, supporting cross-model
55
- transfer of DFlash2 drafters.
 
 
 
 
 
 
 
 
 
 
56
 
57
  ## Usage (vLLM)
58
 
59
  ```bash
60
  vllm serve <target-model> \
61
- --speculative-config '{"method": "dflash", "model": "SargeDev/arete-dflash2-incoai", "num_speculative_tokens": 7}'
62
  ```
63
 
64
  ## Training data
 
9
  - incoai/Qwen3.8-27B-DFlash2
10
  ---
11
 
12
+ # Arete-Dflash2
13
 
14
+ DFlash2 draft model for speculative decoding, compatible with **Arete-27B**,
15
+ **Arete-27B-Uncensored**, **Qwen3.8-27B**, and **Qwen3.8-27B-Uncensored** —
16
+ meant to be used with Arete-27B / Qwen3.8-27B variants. Trained by SargeDev
17
+ on the GB10 (2x GB10 cluster, vLLM 0.28).
18
 
19
  ## What this is
20
 
 
27
 
28
  | repo | base | training |
29
  |---|---|---|
30
+ | `SargeDev/Arete-Dflash2` | [incoai/Qwen3.8-27B-DFlash2](https://huggingface.co/incoai/Qwen3.8-27B-DFlash2) (Z-Lab DFlash2, 1.92B) | 7 epochs finetune on Arete-27B-Uncensored regen data |
31
 
32
  ## Targets
33
 
 
40
  | Arete-27B (censored, original) | [NousResearch/Arete-27B](https://huggingface.co/NousResearch/Arete-27B) |
41
  | Qwen3.8-27B | [Qwen/Qwen3.8-27B](https://huggingface.co/Qwen/Qwen3.8-27B) |
42
 
43
+ ## Benchmarks
44
 
45
  Accept length = mean accepted tokens per draft block (7-token blocks, greedy).
46
  Greedy losslessness: 5/5 prompts identical to the un-drafted target output.
47
 
48
+ ### Bench matrix (vLLM 0.28, GB10, single target serve)
 
 
 
 
49
 
50
+ | target | accept_len | draft rate |
51
+ |---|---|---|
52
+ | **Arete-27B-Uncensored** | **3.00** | **28.6%** |
53
+ | **Arete-27B (censored)** | **2.97** | **28.2%** |
54
+ | **Qwen3.8-27B** | **3.80** | **40.0%** |
55
+
56
+ ### Live testing — Arete-27B-Uncensored bf16 TP2 (262k ctx, pen-testing workload)
57
+
58
+ Real-world serving with structured security content (pen-testing prompts):
59
+ accept length up to **4.71** at **53.0%** draft rate on structured windows,
60
+ 2.6–3.9 accept length / 23–41% on general windows. The drafter's accept rate
61
+ tracks how close the prompt is to its training style — structured technical
62
+ content drafts markedly better than free-form prose.
63
 
64
  ## Usage (vLLM)
65
 
66
  ```bash
67
  vllm serve <target-model> \
68
+ --speculative-config '{"method": "dflash", "model": "SargeDev/Arete-Dflash2", "num_speculative_tokens": 7}'
69
  ```
70
 
71
  ## Training data