Paris99 commited on
Commit
e5ef5e6
·
verified ·
1 Parent(s): 4a42b6e

Add Splash speeds

Browse files
Files changed (1) hide show
  1. README.md +15 -0
README.md CHANGED
@@ -81,6 +81,21 @@ so treat the speeds as indicative.
81
  | Q6_K | 22 tok/s | 46 tok/s (×2.07) | 73% |
82
  | Q8_0 | 18 tok/s | 43 tok/s (×2.36) | 74% |
83
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
84
  ## What was checked
85
 
86
  On a MacBook Pro M5 Max (128 GB), 18 November 2026.
 
81
  | Q6_K | 22 tok/s | 46 tok/s (×2.07) | 73% |
82
  | Q8_0 | 18 tok/s | 43 tok/s (×2.36) | 74% |
83
 
84
+ ### With Splash
85
+
86
+ These builds also run in Splash (1.7.0), which uses its own DFlash2 draft model for speculative decoding. Measured on the same 5 prompts × 256 tokens, thinking off, on 19 November 2026 with other models loaded on the Mac, so treat the speeds as indicative; text, tool calling and image input passed on every build:
87
+
88
+ | File | Decode speed | End to end, incl. prompt | Tokens per cycle |
89
+ |---|---|---|---|
90
+ | Q4_K_M | 144 tok/s | 111 tok/s | 6.6 |
91
+ | Q6_K | 129 tok/s | 98 tok/s | 6.9 |
92
+ | Q8_0 | 105 tok/s | 85 tok/s | 6.6 |
93
+
94
+ ```bash
95
+ splash serve --model-dir Qwen3.8-27B-opencode-Q6_K.gguf --mmproj mmproj-Qwen3.8-27B-opencode-F16.gguf \
96
+ --model local/qwen3.8-opencode --reasoning-effort none
97
+ ```
98
+
99
  ## What was checked
100
 
101
  On a MacBook Pro M5 Max (128 GB), 18 November 2026.