SargeDev commited on
Commit
baf8dad
·
verified ·
1 Parent(s): 0c0f9e0

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +84 -0
README.md ADDED
@@ -0,0 +1,84 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ tags:
4
+ - dflash2
5
+ - speculative-decoding
6
+ - draft-model
7
+ - qwen3
8
+ base_model:
9
+ - incoai/Qwen3.8-27B-DFlash2
10
+ ---
11
+
12
+ # Arete DFlash2 Drafter (Z-Lab base finetuned)
13
+
14
+ DFlash2 draft model for speculative decoding, trained by SargeDev on the
15
+ GB10 (2x GB10 cluster, vLLM 0.28).
16
+
17
+ ## What this is
18
+
19
+ A DFlash2 drafter: a small (1.92B) draft model that proposes tokens for a
20
+ larger verifier (target) model. The verifier scores the drafts in one forward
21
+ pass and accepts a prefix — lossless speedups, identical output.
22
+
23
+ Two drafters ship, both in z-lab export format (`DFlash2DraftModel` +
24
+ `dflash_config`), drop-in for vLLM `--speculative-config`:
25
+
26
+ | repo | base | training |
27
+ |---|---|---|
28
+ | `SargeDev/arete-dflash2-scratch` | random init | 5 epochs on Arete-27B-Uncensored regen data |
29
+ | `SargeDev/arete-dflash2-incoai` | [incoai/Qwen3.8-27B-DFlash2](https://huggingface.co/incoai/Qwen3.8-27B-DFlash2) (Z-Lab DFlash2, 1.92B) | 7 epochs finetune on the same data |
30
+
31
+ ## Targets
32
+
33
+ Both drafters are **drop-in compatible** with all three (greedy losslessness
34
+ 5/5 on every combo — verified):
35
+
36
+ | target | repo |
37
+ |---|---|
38
+ | Arete-27B-Uncensored (our uncensored build, bf16) | [SargeDev/Arete-27B-Uncensored-V2](https://huggingface.co/SargeDev/Arete-27B-Uncensored-V2) |
39
+ | Arete-27B (censored, original) | [NousResearch/Arete-27B](https://huggingface.co/NousResearch/Arete-27B) |
40
+ | Qwen3.8-27B | [Qwen/Qwen3.8-27B](https://huggingface.co/Qwen/Qwen3.8-27B) |
41
+
42
+ ## Benchmarks (vLLM 0.28, GB10, single target serve)
43
+
44
+ Accept length = mean accepted tokens per draft block (7-token blocks, greedy).
45
+ Greedy losslessness: 5/5 prompts identical to the un-drafted target output.
46
+
47
+ | drafter | target | accept_len | draft rate |
48
+ |---|---|---|---|
49
+ | scratch | Arete-27B-Uncensored | 2.03 | 14.7% |
50
+ | scratch | Arete-27B (censored) | 1.91 | 13.1% |
51
+ | scratch | Qwen3.8-27B | 1.78 | 11.2% |
52
+ | **incoai** | **Arete-27B-Uncensored** | **3.00** | **28.6%** |
53
+ | **incoai** | **Arete-27B (censored)** | **2.97** | **28.2%** |
54
+ | **incoai** | **Qwen3.8-27B** | **3.80** | **40.0%** |
55
+
56
+ The incoai drafter (Z-Lab base finetuned on our regen data) is the strongest
57
+ combo everywhere — including on the base Qwen3.8-27B, supporting cross-model
58
+ transfer of DFlash2 drafters.
59
+
60
+ ## Usage (vLLM)
61
+
62
+ ```bash
63
+ vllm serve <target-model> \
64
+ --speculative-config '{"method": "dflash", "model": "SargeDev/arete-dflash2-incoai", "num_speculative_tokens": 7}'
65
+ ```
66
+
67
+ ## Training data
68
+
69
+ 6,903 rows generated on-policy from Arete-27B-Uncensored (loop rows
70
+ quarantined), hidden states extracted from layers 6/20/34/48/62/64 of the
71
+ target. Corpus: `SargeDev/jev-distill-corpus-v3` lineage.
72
+
73
+ ## Provenance
74
+
75
+ - Z-Lab drafter base: [z-lab/Qwen3.8-27B-DFlash2](https://huggingface.co/z-lab/Qwen3.8-27B-DFlash2)
76
+ - incoai finetuned variant: [incoai/Qwen3.8-27B-DFlash2](https://huggingface.co/incoai/Qwen3.8-27B-DFlash2)
77
+ - Target: [SargeDev/Arete-27B-Uncensored-V2](https://huggingface.co/SargeDev/Arete-27B-Uncensored-V2) (our uncensored build of [NousResearch/Arete-27B](https://huggingface.co/NousResearch/Arete-27B))
78
+ - Training corpus: `SargeDev/jev-distill-corpus-v3`
79
+ - Trainer: [vllm-project/speculators](https://github.com/vllm-project/speculators) 0.9.0a0, DFlash2 (dflash2) speculator type
80
+
81
+ ## License
82
+
83
+ Apache-2.0 (base drafter). Arete-27B is cc-by-nc-4.0; the uncensored
84
+ SargeDev build follows the same license as its base.