Upload README.md with huggingface_hub
Browse files
README.md
CHANGED
|
@@ -9,11 +9,12 @@ base_model:
|
|
| 9 |
- incoai/Qwen3.8-27B-DFlash2
|
| 10 |
---
|
| 11 |
|
| 12 |
-
# Arete
|
| 13 |
|
| 14 |
-
DFlash2 draft model for speculative decoding,
|
| 15 |
-
**Arete-27B-Uncensored**
|
| 16 |
-
|
|
|
|
| 17 |
|
| 18 |
## What this is
|
| 19 |
|
|
@@ -26,7 +27,7 @@ vLLM `--speculative-config`.
|
|
| 26 |
|
| 27 |
| repo | base | training |
|
| 28 |
|---|---|---|
|
| 29 |
-
| `SargeDev/
|
| 30 |
|
| 31 |
## Targets
|
| 32 |
|
|
@@ -39,26 +40,32 @@ Both drafters are **drop-in compatible** with all three (greedy losslessness
|
|
| 39 |
| Arete-27B (censored, original) | [NousResearch/Arete-27B](https://huggingface.co/NousResearch/Arete-27B) |
|
| 40 |
| Qwen3.8-27B | [Qwen/Qwen3.8-27B](https://huggingface.co/Qwen/Qwen3.8-27B) |
|
| 41 |
|
| 42 |
-
## Benchmarks
|
| 43 |
|
| 44 |
Accept length = mean accepted tokens per draft block (7-token blocks, greedy).
|
| 45 |
Greedy losslessness: 5/5 prompts identical to the un-drafted target output.
|
| 46 |
|
| 47 |
-
|
| 48 |
-
|---|---|---|---|
|
| 49 |
-
| **incoai** | **Arete-27B-Uncensored** | **3.00** | **28.6%** |
|
| 50 |
-
| **incoai** | **Arete-27B (censored)** | **2.97** | **28.2%** |
|
| 51 |
-
| **incoai** | **Qwen3.8-27B** | **3.80** | **40.0%** |
|
| 52 |
|
| 53 |
-
|
| 54 |
-
|
| 55 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 56 |
|
| 57 |
## Usage (vLLM)
|
| 58 |
|
| 59 |
```bash
|
| 60 |
vllm serve <target-model> \
|
| 61 |
-
--speculative-config '{"method": "dflash", "model": "SargeDev/
|
| 62 |
```
|
| 63 |
|
| 64 |
## Training data
|
|
|
|
| 9 |
- incoai/Qwen3.8-27B-DFlash2
|
| 10 |
---
|
| 11 |
|
| 12 |
+
# Arete-Dflash2
|
| 13 |
|
| 14 |
+
DFlash2 draft model for speculative decoding, compatible with **Arete-27B**,
|
| 15 |
+
**Arete-27B-Uncensored**, **Qwen3.8-27B**, and **Qwen3.8-27B-Uncensored** —
|
| 16 |
+
meant to be used with Arete-27B / Qwen3.8-27B variants. Trained by SargeDev
|
| 17 |
+
on the GB10 (2x GB10 cluster, vLLM 0.28).
|
| 18 |
|
| 19 |
## What this is
|
| 20 |
|
|
|
|
| 27 |
|
| 28 |
| repo | base | training |
|
| 29 |
|---|---|---|
|
| 30 |
+
| `SargeDev/Arete-Dflash2` | [incoai/Qwen3.8-27B-DFlash2](https://huggingface.co/incoai/Qwen3.8-27B-DFlash2) (Z-Lab DFlash2, 1.92B) | 7 epochs finetune on Arete-27B-Uncensored regen data |
|
| 31 |
|
| 32 |
## Targets
|
| 33 |
|
|
|
|
| 40 |
| Arete-27B (censored, original) | [NousResearch/Arete-27B](https://huggingface.co/NousResearch/Arete-27B) |
|
| 41 |
| Qwen3.8-27B | [Qwen/Qwen3.8-27B](https://huggingface.co/Qwen/Qwen3.8-27B) |
|
| 42 |
|
| 43 |
+
## Benchmarks
|
| 44 |
|
| 45 |
Accept length = mean accepted tokens per draft block (7-token blocks, greedy).
|
| 46 |
Greedy losslessness: 5/5 prompts identical to the un-drafted target output.
|
| 47 |
|
| 48 |
+
### Bench matrix (vLLM 0.28, GB10, single target serve)
|
|
|
|
|
|
|
|
|
|
|
|
|
| 49 |
|
| 50 |
+
| target | accept_len | draft rate |
|
| 51 |
+
|---|---|---|
|
| 52 |
+
| **Arete-27B-Uncensored** | **3.00** | **28.6%** |
|
| 53 |
+
| **Arete-27B (censored)** | **2.97** | **28.2%** |
|
| 54 |
+
| **Qwen3.8-27B** | **3.80** | **40.0%** |
|
| 55 |
+
|
| 56 |
+
### Live testing — Arete-27B-Uncensored bf16 TP2 (262k ctx, pen-testing workload)
|
| 57 |
+
|
| 58 |
+
Real-world serving with structured security content (pen-testing prompts):
|
| 59 |
+
accept length up to **4.71** at **53.0%** draft rate on structured windows,
|
| 60 |
+
2.6–3.9 accept length / 23–41% on general windows. The drafter's accept rate
|
| 61 |
+
tracks how close the prompt is to its training style — structured technical
|
| 62 |
+
content drafts markedly better than free-form prose.
|
| 63 |
|
| 64 |
## Usage (vLLM)
|
| 65 |
|
| 66 |
```bash
|
| 67 |
vllm serve <target-model> \
|
| 68 |
+
--speculative-config '{"method": "dflash", "model": "SargeDev/Arete-Dflash2", "num_speculative_tokens": 7}'
|
| 69 |
```
|
| 70 |
|
| 71 |
## Training data
|