Cyronius commited on
Commit
edb776b
·
verified ·
1 Parent(s): 6130d86

Adapter relation, Qwen3.5 size tags, Strix Halo hardware line

Browse files
Files changed (1) hide show
  1. README.md +10 -2
README.md CHANGED
@@ -4,11 +4,18 @@ library_name: llama.cpp
4
  pipeline_tag: text-generation
5
  base_model:
6
  - Qwen/Qwen3.5-9B
 
7
  tags:
8
  - speculative-decoding
9
- - llama.cpp
10
  - drafter
 
11
  - code
 
 
 
 
 
12
  ---
13
 
14
  # MemDraft
@@ -30,7 +37,8 @@ One drafter, built from Qwen3.5-9B's own text, used unchanged on four Qwen3.5 si
30
  | Qwen3.5-4B | 65 tok/s | 110 tok/s (1.69×) | 199 tok/s (3.04×) |
31
  | Qwen3.5-9B | 38 tok/s | 70 tok/s (1.84×) | 113 tok/s (2.96×) |
32
 
33
- - **Setup:** `ngram-simple`, then MemDraft with the reranker, drafting 3 tokens. Q4_K_M models on an AMD Radeon 8060S through Vulkan, greedy decoding, 2026-11-21.
 
34
  - **Prompts:** 12 held-out OSS-Instruct coding prompts (2 rounds) and HumanEval problems 0–11. No table holds answers to them. The speedup is over plain decoding on the same prompts; plain decoding runs within about 1% of the same speed on both sets.
35
  - **Output:** the target model checks every drafted token, so the text is the target's own. See [Quality](#quality).
36
  - **Limits:** Qwen3.5 only, code only, one GPU. The 27B and 35B-A3B are not measured yet. On chat, an earlier table gave only 1.09×.
 
4
  pipeline_tag: text-generation
5
  base_model:
6
  - Qwen/Qwen3.5-9B
7
+ base_model_relation: adapter
8
  tags:
9
  - speculative-decoding
10
+ - draft-model
11
  - drafter
12
+ - llama.cpp
13
  - code
14
+ - qwen3.5
15
+ - qwen3.5-0.8b
16
+ - qwen3.5-2b
17
+ - qwen3.5-4b
18
+ - qwen3.5-9b
19
  ---
20
 
21
  # MemDraft
 
37
  | Qwen3.5-4B | 65 tok/s | 110 tok/s (1.69×) | 199 tok/s (3.04×) |
38
  | Qwen3.5-9B | 38 tok/s | 70 tok/s (1.84×) | 113 tok/s (2.96×) |
39
 
40
+ - **Setup:** `ngram-simple`, then MemDraft with the reranker, drafting 3 tokens. Q4_K_M models through llama.cpp's Vulkan backend, greedy decoding, 2026-11-21.
41
+ - **Hardware:** an AMD Strix Halo mini PC (Ryzen AI Max+ 395, Radeon 8060S integrated GPU, 128 GB of memory shared by CPU and GPU) on Linux.
42
  - **Prompts:** 12 held-out OSS-Instruct coding prompts (2 rounds) and HumanEval problems 0–11. No table holds answers to them. The speedup is over plain decoding on the same prompts; plain decoding runs within about 1% of the same speed on both sets.
43
  - **Output:** the target model checks every drafted token, so the text is the target's own. See [Quality](#quality).
44
  - **Limits:** Qwen3.5 only, code only, one GPU. The 27B and 35B-A3B are not measured yet. On chat, an earlier table gave only 1.09×.