|
Download README.md from SargeDev/Arete-Dflash2: direct link, hf CLI and curl.
- Browser
- Download file 3.84 kB
-
https://huggingface.co/SargeDev/Arete-Dflash2/resolve/main/README.md
- Command line
-
hf download hf://SargeDev/Arete-Dflash2/README.md
-
curl -L -o README.md https://huggingface.co/SargeDev/Arete-Dflash2/resolve/main/README.md
3.84 kB
| license: apache-2.0 | |
| tags: | |
| - dflash2 | |
| - speculative-decoding | |
| - draft-model | |
| - qwen3 | |
| base_model: | |
| - incoai/Qwen3.8-27B-DFlash2 | |
| # Arete-Dflash2 | |
| DFlash2 draft model for speculative decoding, compatible with **Arete-27B**, | |
| **Arete-27B-Uncensored**, **Qwen3.8-27B**, and **Qwen3.8-27B-Uncensored** — | |
| meant to be used with Arete-27B / Qwen3.8-27B variants. Trained by SargeDev | |
| on the GB10 (2x GB10 cluster, vLLM 0.28). | |
| ## What this is | |
| A DFlash2 drafter: a small (1.92B) draft model that proposes tokens for a | |
| larger verifier (target) model. The verifier scores the drafts in one forward | |
| pass and accepts a prefix — lossless speedups, identical output. | |
| In z-lab export format (`DFlash2DraftModel` + `dflash_config`), drop-in for | |
| vLLM `--speculative-config`. | |
| | repo | base | training | | |
| |---|---|---| | |
| | `SargeDev/Arete-Dflash2` | [incoai/Qwen3.8-27B-DFlash2](https://huggingface.co/incoai/Qwen3.8-27B-DFlash2) (Z-Lab DFlash2, 1.92B) | 7 epochs finetune on Arete-27B-Uncensored regen data | | |
| ## Targets | |
| Both drafters are **drop-in compatible** with all three (greedy losslessness | |
| 5/5 on every combo — verified): | |
| | target | repo | | |
| |---|---| | |
| | Arete-27B-Uncensored (our uncensored build, bf16) | [SargeDev/Arete-27B-Uncensored-V2](https://huggingface.co/SargeDev/Arete-27B-Uncensored-V2) | | |
| | Arete-27B (censored, original) | [NousResearch/Arete-27B](https://huggingface.co/NousResearch/Arete-27B) | | |
| | Qwen3.8-27B | [Qwen/Qwen3.8-27B](https://huggingface.co/Qwen/Qwen3.8-27B) | | |
| ## Benchmarks | |
| Accept length = mean accepted tokens per draft block (7-token blocks, greedy). | |
| Greedy losslessness: 5/5 prompts identical to the un-drafted target output. | |
| ### Bench matrix (vLLM 0.28, GB10, single target serve) | |
| | target | accept_len | draft rate | | |
| |---|---|---| | |
| | **Arete-27B-Uncensored** | **3.00** | **28.6%** | | |
| | **Arete-27B (censored)** | **2.97** | **28.2%** | | |
| | **Qwen3.8-27B** | **3.80** | **40.0%** | | |
| ### Live testing — Arete-27B-Uncensored bf16 TP2 (262k ctx, pen-testing workload) | |
| Real-world serving with structured security content (pen-testing prompts): | |
| accept length up to **4.71** at **53.0%** draft rate on structured windows, | |
| 2.6–3.9 accept length / 23–41% on general windows. The drafter's accept rate | |
| tracks how close the prompt is to its training style — structured technical | |
| content drafts markedly better than free-form prose. | |
| ## Usage (vLLM) | |
| ```bash | |
| vllm serve <target-model> \ | |
| --speculative-config '{"method": "dflash", "model": "SargeDev/Arete-Dflash2", "num_speculative_tokens": 7}' | |
| ``` | |
| ## Training data | |
| 6,903 rows generated on-policy from Arete-27B-Uncensored (loop rows | |
| quarantined), hidden states extracted from layers 6/20/34/48/62/64 of the | |
| target. The exact training rows are published: | |
| **[SargeDev/arete-dflash2-trainset](https://huggingface.co/datasets/SargeDev/arete-dflash2-trainset)** | |
| (97.6 MB, jsonl: `id`, `primary_id`, `source`, `input_ids`, `loss_mask`, | |
| `text`, `metadata`). Corpus lineage: `SargeDev/jev-distill-corpus-v3`. | |
| ## Provenance | |
| - Z-Lab drafter base: [z-lab/Qwen3.8-27B-DFlash2](https://huggingface.co/z-lab/Qwen3.8-27B-DFlash2) | |
| - incoai finetuned variant: [incoai/Qwen3.8-27B-DFlash2](https://huggingface.co/incoai/Qwen3.8-27B-DFlash2) | |
| - Target: [SargeDev/Arete-27B-Uncensored-V2](https://huggingface.co/SargeDev/Arete-27B-Uncensored-V2) (our uncensored build of [NousResearch/Arete-27B](https://huggingface.co/NousResearch/Arete-27B)) | |
| - Training corpus: [SargeDev/arete-dflash2-trainset](https://huggingface.co/datasets/SargeDev/arete-dflash2-trainset) (`SargeDev/jev-distill-corpus-v3` lineage) | |
| - Trainer: [vllm-project/speculators](https://github.com/vllm-project/speculators) 0.9.0a0, DFlash2 (dflash2) speculator type | |
| ## License | |
| Apache-2.0 (base drafter). Arete-27B is cc-by-nc-4.0; the uncensored | |
| SargeDev build follows the same license as its base. | |