Card: datasheet/neo-brutalist redesign — monospace, hard square borders, numbered sections, symmetric spec grid, theme-adaptive (currentColor)
Browse files
README.md
CHANGED
|
@@ -21,64 +21,57 @@ language:
|
|
| 21 |
base_model_relation: quantized
|
| 22 |
---
|
| 23 |
|
| 24 |
-
<div style="font-family:
|
| 25 |
-
<div style="
|
| 26 |
-
|
| 27 |
-
|
| 28 |
-
|
| 29 |
-
</div>
|
| 30 |
-
<p style="margin: 8px 0 0 0; font-size: 14px; color: #dbeafe; font-weight: 500;">4-bit ROCmFP4 quant of Jackrong's Qwopus-Coder, with a built-in MTP head — self-speculative decode + vision on a single AMD APU.</p>
|
| 31 |
-
</div>
|
| 32 |
-
<div style="display: flex; gap: 8px; flex-wrap: wrap; padding: 12px 24px; background: rgba(128,128,128,0.06); border-bottom: 1px solid rgba(128,128,128,0.18);">
|
| 33 |
-
<span style="background:#dbeafe; color:#1e40af; font-size:11px; font-weight:700; padding:4px 10px; border-radius:20px; border:1px solid #bfdbfe;">⚙️ ROCmFP4 4-bit · ~4.5 bpw</span>
|
| 34 |
-
<span style="background:#e0e7ff; color:#3730a3; font-size:11px; font-weight:700; padding:4px 10px; border-radius:20px; border:1px solid #c7d2fe;">📦 16 GB</span>
|
| 35 |
-
<span style="background:#d1fae5; color:#065f46; font-size:11px; font-weight:700; padding:4px 10px; border-radius:20px; border:1px solid #a7f3d0;">⚡ MTP self-speculative</span>
|
| 36 |
-
<span style="background:#f3e8ff; color:#6b21a8; font-size:11px; font-weight:700; padding:4px 10px; border-radius:20px; border:1px solid #e9d5ff;">👁 Qwen3-VL vision</span>
|
| 37 |
-
<span style="background:#ffedd5; color:#9a3412; font-size:11px; font-weight:700; padding:4px 10px; border-radius:20px; border:1px solid #fed7aa;">🔥 Vulkan · gfx1151</span>
|
| 38 |
-
<span style="background:#f1f5f9; color:#334155; font-size:11px; font-weight:700; padding:4px 10px; border-radius:20px; border:1px solid #e2e8f0;">📜 Apache-2.0</span>
|
| 39 |
</div>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 40 |
</div>
|
| 41 |
|
| 42 |
-
|
| 43 |
-
|
| 44 |
-
|
| 45 |
-
>
|
| 46 |
-
> Ignore HuggingFace's auto-detected **"F16"** badge — its parser can't read ROCmFP4 and mislabels by the f16 embeddings. These are **~4.4–4.5 bpw 4-bit** files; pick by filename.
|
| 47 |
-
|
| 48 |
-
## ⚡ At a glance
|
| 49 |
|
| 50 |
-
<
|
| 51 |
-
<
|
| 52 |
-
<
|
| 53 |
-
<tr><td style="padding:9px 12px; border-bottom:1px solid rgba(128,128,128,0.15); font-weight:700;">Format</td><td style="padding:9px 12px; border-bottom:1px solid rgba(128,128,128,0.15);">ROCmFP4 4-bit (~4.4–4.5 bpw), <b>genuine f16 embeddings</b> (from BF16)</td></tr>
|
| 54 |
-
<tr><td style="padding:9px 12px; border-bottom:1px solid rgba(128,128,128,0.15); font-weight:700;">MTP</td><td style="padding:9px 12px; border-bottom:1px solid rgba(128,128,128,0.15);">Built-in next-token head → self-speculative decode (output-lossless)</td></tr>
|
| 55 |
-
<tr><td style="padding:9px 12px; border-bottom:1px solid rgba(128,128,128,0.15); font-weight:700;">Vision</td><td style="padding:9px 12px; border-bottom:1px solid rgba(128,128,128,0.15);">Qwen3-VL — add an <code>--mmproj</code> projector (see below)</td></tr>
|
| 56 |
-
<tr><td style="padding:9px 12px; border-bottom:1px solid rgba(128,128,128,0.15); font-weight:700;">Decode</td><td style="padding:9px 12px; border-bottom:1px solid rgba(128,128,128,0.15);"><b>~37 t/s</b> thinking-off, MTP acceptance <b>~0.8</b> on code (Vulkan / Strix Halo)</td></tr>
|
| 57 |
-
<tr><td style="padding:9px 12px; border-bottom:1px solid rgba(128,128,128,0.15); font-weight:700;">Context</td><td style="padding:9px 12px; border-bottom:1px solid rgba(128,128,128,0.15);">262K</td></tr>
|
| 58 |
-
<tr><td style="padding:9px 12px; font-weight:700;">Hardware</td><td style="padding:9px 12px;">AMD Ryzen AI Max+ 395 (gfx1151), 128 GB unified</td></tr>
|
| 59 |
-
</tbody>
|
| 60 |
-
</table>
|
| 61 |
|
| 62 |
-
|
| 63 |
|
| 64 |
-
<table style="width:100%; border-collapse:collapse; font-size:
|
| 65 |
-
<thead><tr
|
| 66 |
-
<th style="
|
| 67 |
-
<th style="
|
| 68 |
-
<th style="
|
| 69 |
-
<th style="
|
| 70 |
</tr></thead>
|
| 71 |
<tbody>
|
| 72 |
-
<tr><td style="
|
| 73 |
-
<tr><td style="padding:
|
| 74 |
</tbody>
|
| 75 |
</table>
|
| 76 |
|
| 77 |
-
Both share the ROCmFP4 4-bit body + genuine f16 embeddings + MTP head, **no imatrix** (see
|
| 78 |
|
| 79 |
-
|
| 80 |
|
| 81 |
-
|
| 82 |
|
| 83 |
```bash
|
| 84 |
env HSA_OVERRIDE_GFX_VERSION=11.5.1 GGML_HIP_ENABLE_UNIFIED_MEMORY=1 \
|
|
@@ -123,40 +116,38 @@ llama-server \
|
|
| 123 |
--no-mmap
|
| 124 |
```
|
| 125 |
|
| 126 |
-
|
| 127 |
-
|
| 128 |
-
<
|
| 129 |
-
<
|
| 130 |
-
<th style="padding:7px 11px; border-bottom:2px solid #4f46e5; text-align:left; color:#4f46e5; width:38%;">Flag</th>
|
| 131 |
-
<th style="padding:7px 11px; border-bottom:2px solid #4f46e5; text-align:left; color:#4f46e5;">What it does</th>
|
| 132 |
</tr></thead>
|
| 133 |
<tbody>
|
| 134 |
-
<tr><td style="
|
| 135 |
-
<tr><td style="
|
| 136 |
-
<tr><td style="
|
| 137 |
-
<tr><td style="
|
| 138 |
-
<tr><td style="
|
| 139 |
-
<tr><td style="
|
| 140 |
-
<tr><td style="
|
| 141 |
-
<tr><td style="
|
| 142 |
-
<tr><td style="
|
| 143 |
-
<tr><td style="
|
| 144 |
-
<tr><td style="
|
| 145 |
-
<tr><td style="
|
| 146 |
-
<tr><td style="
|
| 147 |
-
<tr><td style="padding:
|
| 148 |
</tbody>
|
| 149 |
</table>
|
| 150 |
|
| 151 |
-
|
| 152 |
|
| 153 |
-
Run
|
| 154 |
|
| 155 |
```
|
| 156 |
--reasoning-format deepseek --chat-template-kwargs '{"enable_thinking": false, "preserve_thinking": true}'
|
| 157 |
```
|
| 158 |
|
| 159 |
-
**OpenCode** —
|
| 160 |
|
| 161 |
```json
|
| 162 |
{
|
|
@@ -172,32 +163,33 @@ Run it **thinking-off** — it commits straight to the tool call instead of over
|
|
| 172 |
}
|
| 173 |
```
|
| 174 |
|
| 175 |
-
|
| 176 |
|
| 177 |
-
Qwen3-VL lineage — add a Qwen3-VL `mmproj` projector with `--mmproj` (no different LLM GGUF
|
| 178 |
|
| 179 |
```bash
|
| 180 |
--mmproj mmproj-Qwen3.6-27B-f16.gguf \
|
| 181 |
--image-min-tokens 1024 # REQUIRED — Qwen-VL needs >=1024 image tokens or it misreads fine detail
|
| 182 |
```
|
| 183 |
|
| 184 |
-
|
| 185 |
-
>
|
|
|
|
| 186 |
|
| 187 |
-
|
| 188 |
|
| 189 |
-
<table style="width:100%; border-collapse:collapse; font-size:
|
| 190 |
<tbody>
|
| 191 |
-
<tr><td style="
|
| 192 |
-
<tr><td style="
|
| 193 |
-
<tr><td style="
|
| 194 |
-
<tr><td style="
|
| 195 |
</tbody>
|
| 196 |
</table>
|
| 197 |
|
| 198 |
-
**Why no imatrix (we measured it):** a code-weighted importance matrix improved fidelity-to-BF16 (median KL **−15%**, top-token +0.6 pp) but **measurably worsened held-out-code perplexity (+2.6%, significant)**. For a coder, code-prediction is the task-relevant metric, so we shipped the **non-imatrix** quant. (On
|
| 199 |
|
| 200 |
-
|
| 201 |
|
| 202 |
```bash
|
| 203 |
# from Jackrong's BF16+MTP GGUF -> ROCmFP4, genuine f16 embeddings, no imatrix
|
|
@@ -211,13 +203,16 @@ llama-quantize --token-embedding-type f16 --output-tensor-type q6_K \
|
|
| 211 |
Qwopus3.6-27B-Coder-MTP-ROCmFP4-STRIX-embF16-headQ6.gguf Q4_0_ROCMFP4_STRIX
|
| 212 |
```
|
| 213 |
|
| 214 |
-
> [!NOTE]
|
| 215 |
> Experimental research build for AMD Strix Halo — hardware/driver/prompt-sensitive, may not reproduce elsewhere. Not native FP4 tensor-core execution.
|
| 216 |
|
| 217 |
-
|
| 218 |
|
| 219 |
-
|
| 220 |
-
|
| 221 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
| 222 |
|
| 223 |
*Derivative quantization — verify the base model's license before redistribution / use.*
|
|
|
|
| 21 |
base_model_relation: quantized
|
| 22 |
---
|
| 23 |
|
| 24 |
+
<div style="border:2px solid currentColor; font-family:ui-monospace,'SF Mono','Cascadia Mono',Consolas,'Liberation Mono',monospace;">
|
| 25 |
+
<div style="border-bottom:1px solid currentColor; padding:6px 12px; font-size:11px; letter-spacing:3px; text-transform:uppercase; opacity:0.7;">PLUNDERSTRUCK // ROCmFP4 QUANTIZED MODEL // STRIX HALO · gfx1151</div>
|
| 26 |
+
<div style="padding:16px 14px 12px 14px;">
|
| 27 |
+
<div style="font-size:23px; font-weight:800; letter-spacing:1px;">QWOPUS3.6-27B-CODER-MTP</div>
|
| 28 |
+
<div style="font-size:12.5px; letter-spacing:1px; opacity:0.8; margin-top:5px;">4-BIT ROCmFP4 · MTP SELF-SPECULATIVE DECODE · VISION-CAPABLE · SINGLE AMD APU</div>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 29 |
</div>
|
| 30 |
+
<table style="width:100%; border-collapse:collapse; font-family:ui-monospace,'SF Mono',Consolas,monospace; font-size:12px;">
|
| 31 |
+
<tr>
|
| 32 |
+
<td style="border-top:1px solid currentColor; border-right:1px solid currentColor; padding:8px 12px;"><div style="font-size:10px; letter-spacing:1px; opacity:0.6;">FORMAT</div><div style="font-weight:700;">ROCmFP4 4-BIT</div></td>
|
| 33 |
+
<td style="border-top:1px solid currentColor; border-right:1px solid currentColor; padding:8px 12px;"><div style="font-size:10px; letter-spacing:1px; opacity:0.6;">PRECISION</div><div style="font-weight:700;">~4.5 BPW</div></td>
|
| 34 |
+
<td style="border-top:1px solid currentColor; border-right:1px solid currentColor; padding:8px 12px;"><div style="font-size:10px; letter-spacing:1px; opacity:0.6;">SIZE</div><div style="font-weight:700;">16 GB</div></td>
|
| 35 |
+
<td style="border-top:1px solid currentColor; padding:8px 12px;"><div style="font-size:10px; letter-spacing:1px; opacity:0.6;">CONTEXT</div><div style="font-weight:700;">262 K</div></td>
|
| 36 |
+
</tr>
|
| 37 |
+
<tr>
|
| 38 |
+
<td style="border-top:1px solid currentColor; border-right:1px solid currentColor; padding:8px 12px;"><div style="font-size:10px; letter-spacing:1px; opacity:0.6;">DRAFT</div><div style="font-weight:700;">MTP n-max 5</div></td>
|
| 39 |
+
<td style="border-top:1px solid currentColor; border-right:1px solid currentColor; padding:8px 12px;"><div style="font-size:10px; letter-spacing:1px; opacity:0.6;">VISION</div><div style="font-weight:700;">QWEN3-VL</div></td>
|
| 40 |
+
<td style="border-top:1px solid currentColor; border-right:1px solid currentColor; padding:8px 12px;"><div style="font-size:10px; letter-spacing:1px; opacity:0.6;">BACKEND</div><div style="font-weight:700;">VULKAN0</div></td>
|
| 41 |
+
<td style="border-top:1px solid currentColor; padding:8px 12px;"><div style="font-size:10px; letter-spacing:1px; opacity:0.6;">LICENSE</div><div style="font-weight:700;">APACHE-2.0</div></td>
|
| 42 |
+
</tr>
|
| 43 |
+
</table>
|
| 44 |
</div>
|
| 45 |
|
| 46 |
+
<div style="border:2px solid currentColor; padding:10px 13px; font-family:ui-monospace,'SF Mono',Consolas,monospace; font-size:12.5px; margin:14px 0;">
|
| 47 |
+
<b style="letter-spacing:1px;">⚠ REQUIRES THE ROCmFP4 FORK</b><br>
|
| 48 |
+
The custom <code>q4_0_rocmfp4</code> tensor types <b>will not load in stock llama.cpp, LM Studio, or Ollama</b>. Build/run with <a href="https://github.com/charlie12345/rocmfp4-llama">charlie12345/rocmfp4-llama</a> · branch <code>mtp-rocmfp4-strix</code>.
|
| 49 |
+
</div>
|
|
|
|
|
|
|
|
|
|
| 50 |
|
| 51 |
+
<div style="border:1px solid currentColor; padding:8px 13px; font-family:ui-monospace,'SF Mono',Consolas,monospace; font-size:12px; margin:14px 0; opacity:0.85;">
|
| 52 |
+
<b>NOTE //</b> Ignore HuggingFace's auto-detected "F16" badge — its parser can't read ROCmFP4 and mislabels by the f16 embeddings. These are <b>~4.4–4.5 bpw 4-bit</b> files; pick by filename.
|
| 53 |
+
</div>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 54 |
|
| 55 |
+
<div style="font-family:ui-monospace,'SF Mono',Consolas,monospace; font-weight:800; font-size:14px; letter-spacing:2px; text-transform:uppercase; border-bottom:2px solid currentColor; padding-bottom:5px; margin:26px 0 12px;">01 · FILES</div>
|
| 56 |
|
| 57 |
+
<table style="width:100%; border-collapse:collapse; font-family:ui-monospace,'SF Mono',Consolas,monospace; font-size:12.5px;">
|
| 58 |
+
<thead><tr>
|
| 59 |
+
<th style="border:1px solid currentColor; padding:7px 10px; text-align:left; text-transform:uppercase; font-size:10px; letter-spacing:1px;">File</th>
|
| 60 |
+
<th style="border:1px solid currentColor; padding:7px 10px; text-align:left; text-transform:uppercase; font-size:10px; letter-spacing:1px;">Size</th>
|
| 61 |
+
<th style="border:1px solid currentColor; padding:7px 10px; text-align:left; text-transform:uppercase; font-size:10px; letter-spacing:1px;">Output head</th>
|
| 62 |
+
<th style="border:1px solid currentColor; padding:7px 10px; text-align:left; text-transform:uppercase; font-size:10px; letter-spacing:1px;">Pick if</th>
|
| 63 |
</tr></thead>
|
| 64 |
<tbody>
|
| 65 |
+
<tr><td style="border:1px solid currentColor; padding:7px 10px;"><code>…-STRIX-embF16.gguf</code></td><td style="border:1px solid currentColor; padding:7px 10px;">16 GB</td><td style="border:1px solid currentColor; padding:7px 10px;">ROCmFP4 4-bit</td><td style="border:1px solid currentColor; padding:7px 10px;">fastest decode (default)</td></tr>
|
| 66 |
+
<tr><td style="border:1px solid currentColor; padding:7px 10px;"><code>…-STRIX-embF16-headQ6.gguf</code></td><td style="border:1px solid currentColor; padding:7px 10px;">16 GB</td><td style="border:1px solid currentColor; padding:7px 10px;">Q6_K</td><td style="border:1px solid currentColor; padding:7px 10px;">more output fidelity (small decode cost)</td></tr>
|
| 67 |
</tbody>
|
| 68 |
</table>
|
| 69 |
|
| 70 |
+
Both share the ROCmFP4 4-bit body + genuine f16 embeddings + MTP head, **no imatrix** (see §05). Repo also bundles **[`chat_template.jinja`](./chat_template.jinja)** — froggeric's unified Qwen3.6 template (tool calls + inline `<|think_off|>` + vision).
|
| 71 |
|
| 72 |
+
<div style="font-family:ui-monospace,'SF Mono',Consolas,monospace; font-weight:800; font-size:14px; letter-spacing:2px; text-transform:uppercase; border-bottom:2px solid currentColor; padding-bottom:5px; margin:26px 0 12px;">02 · QUICK START</div>
|
| 73 |
|
| 74 |
+
Run from the folder holding the `.gguf` + `chat_template.jinja`:
|
| 75 |
|
| 76 |
```bash
|
| 77 |
env HSA_OVERRIDE_GFX_VERSION=11.5.1 GGML_HIP_ENABLE_UNIFIED_MEMORY=1 \
|
|
|
|
| 116 |
--no-mmap
|
| 117 |
```
|
| 118 |
|
| 119 |
+
<table style="width:100%; border-collapse:collapse; font-family:ui-monospace,'SF Mono',Consolas,monospace; font-size:12px;">
|
| 120 |
+
<thead><tr>
|
| 121 |
+
<th style="border:1px solid currentColor; padding:6px 10px; text-align:left; text-transform:uppercase; font-size:10px; letter-spacing:1px; width:40%;">Flag</th>
|
| 122 |
+
<th style="border:1px solid currentColor; padding:6px 10px; text-align:left; text-transform:uppercase; font-size:10px; letter-spacing:1px;">Function</th>
|
|
|
|
|
|
|
| 123 |
</tr></thead>
|
| 124 |
<tbody>
|
| 125 |
+
<tr><td style="border:1px solid currentColor; padding:6px 10px;"><code>HSA_OVERRIDE_GFX_VERSION=11.5.1</code></td><td style="border:1px solid currentColor; padding:6px 10px;">treat the APU as gfx1151 (Strix Halo)</td></tr>
|
| 126 |
+
<tr><td style="border:1px solid currentColor; padding:6px 10px;"><code>GGML_HIP_ENABLE_UNIFIED_MEMORY=1</code></td><td style="border:1px solid currentColor; padding:6px 10px;">allow use of the full 128 GB unified memory</td></tr>
|
| 127 |
+
<tr><td style="border:1px solid currentColor; padding:6px 10px;"><code>-dev Vulkan0</code></td><td style="border:1px solid currentColor; padding:6px 10px;">run on Vulkan — fastest backend for ROCmFP4 on Strix Halo</td></tr>
|
| 128 |
+
<tr><td style="border:1px solid currentColor; padding:6px 10px;"><code>-ngl 999 · -fa on</code></td><td style="border:1px solid currentColor; padding:6px 10px;">offload all layers · flash attention</td></tr>
|
| 129 |
+
<tr><td style="border:1px solid currentColor; padding:6px 10px;"><code>-c 262144</code></td><td style="border:1px solid currentColor; padding:6px 10px;">context length (256K)</td></tr>
|
| 130 |
+
<tr><td style="border:1px solid currentColor; padding:6px 10px;"><code>-b 2048 · -ub 256 · -t/-tb 16</code></td><td style="border:1px solid currentColor; padding:6px 10px;">prefill batch / micro-batch · CPU threads</td></tr>
|
| 131 |
+
<tr><td style="border:1px solid currentColor; padding:6px 10px;"><code>-ctk f16 · -ctv f16</code></td><td style="border:1px solid currentColor; padding:6px 10px;">KV cache in f16 — dominant MTP-acceptance lever (use q8_0 on less RAM)</td></tr>
|
| 132 |
+
<tr><td style="border:1px solid currentColor; padding:6px 10px;"><code>-cpent · -ctxcp · --cache-reuse · --cache-ram 65536</code></td><td style="border:1px solid currentColor; padding:6px 10px;">cross-turn KV checkpointing + 64 GB resident reuse cache</td></tr>
|
| 133 |
+
<tr><td style="border:1px solid currentColor; padding:6px 10px;"><code>--temp 0.6 --top-p 0.95 --top-k 20 --min-p 0.0</code></td><td style="border:1px solid currentColor; padding:6px 10px;">Qwen3.6 recommended sampling</td></tr>
|
| 134 |
+
<tr><td style="border:1px solid currentColor; padding:6px 10px;"><code>--spec-type draft-mtp · --spec-draft-n-max 5</code></td><td style="border:1px solid currentColor; padding:6px 10px;">built-in MTP head, self-speculative; draft depth 5 (measured optimum)</td></tr>
|
| 135 |
+
<tr><td style="border:1px solid currentColor; padding:6px 10px;"><code>--spec-draft-device Vulkan0 · -ngl all · type-k/v f16</code></td><td style="border:1px solid currentColor; padding:6px 10px;">draft head on Vulkan, fully offloaded, f16 KV</td></tr>
|
| 136 |
+
<tr><td style="border:1px solid currentColor; padding:6px 10px;"><code>--chat-template-file chat_template.jinja</code></td><td style="border:1px solid currentColor; padding:6px 10px;">bundled froggeric template (tool calls + think-toggle + vision)</td></tr>
|
| 137 |
+
<tr><td style="border:1px solid currentColor; padding:6px 10px;"><code>--reasoning-format deepseek + kwargs {enable_thinking:false, preserve_thinking:true}</code></td><td style="border:1px solid currentColor; padding:6px 10px;">thinking-off for agentic use, but keep cross-turn cache (~86% reuse)</td></tr>
|
| 138 |
+
<tr><td style="border:1px solid currentColor; padding:6px 10px;"><code>--jinja --parallel 1 --metrics --no-mmap</code></td><td style="border:1px solid currentColor; padding:6px 10px;">apply template · single slot · metrics · weights in RAM</td></tr>
|
| 139 |
</tbody>
|
| 140 |
</table>
|
| 141 |
|
| 142 |
+
<div style="font-family:ui-monospace,'SF Mono',Consolas,monospace; font-weight:800; font-size:14px; letter-spacing:2px; text-transform:uppercase; border-bottom:2px solid currentColor; padding-bottom:5px; margin:26px 0 12px;">03 · CODING AGENT</div>
|
| 143 |
|
| 144 |
+
Run **thinking-off** — it commits straight to the tool call instead of over-planning in `<think>`. Naive `enable_thinking:false` **breaks cross-turn prompt-cache reuse** (measured 0% → full re-prefill); add **`preserve_thinking:true`** and the cache stays (**~86% reuse measured**):
|
| 145 |
|
| 146 |
```
|
| 147 |
--reasoning-format deepseek --chat-template-kwargs '{"enable_thinking": false, "preserve_thinking": true}'
|
| 148 |
```
|
| 149 |
|
| 150 |
+
**OpenCode** — via my fork [`PlunderStruck/opencode`](https://github.com/PlunderStruck/opencode) (compaction doesn't rewrite the leading prompt → cache survives long sessions). The model **must** be `tool_call: true`, or OpenCode won't send tools natively and the model just narrates code instead of calling tools:
|
| 151 |
|
| 152 |
```json
|
| 153 |
{
|
|
|
|
| 163 |
}
|
| 164 |
```
|
| 165 |
|
| 166 |
+
<div style="font-family:ui-monospace,'SF Mono',Consolas,monospace; font-weight:800; font-size:14px; letter-spacing:2px; text-transform:uppercase; border-bottom:2px solid currentColor; padding-bottom:5px; margin:26px 0 12px;">04 · VISION</div>
|
| 167 |
|
| 168 |
+
Qwen3-VL lineage — add a Qwen3-VL `mmproj` projector with `--mmproj` (no different LLM GGUF). Any projector whose `projection_dim` is **5120** fits (the Qwen3.6-27B one; f16 or f32):
|
| 169 |
|
| 170 |
```bash
|
| 171 |
--mmproj mmproj-Qwen3.6-27B-f16.gguf \
|
| 172 |
--image-min-tokens 1024 # REQUIRED — Qwen-VL needs >=1024 image tokens or it misreads fine detail
|
| 173 |
```
|
| 174 |
|
| 175 |
+
<div style="border:1px solid currentColor; padding:8px 13px; font-family:ui-monospace,'SF Mono',Consolas,monospace; font-size:12px; margin:12px 0; opacity:0.85;">
|
| 176 |
+
<b>NOTE //</b> thinking model → for one-shot image Q&A use <code><|think_off|></code> or allow enough tokens, else the answer can come back empty. Loading <code>--mmproj</code> also disables cross-turn cache reuse — use vision for one-shot image tasks.
|
| 177 |
+
</div>
|
| 178 |
|
| 179 |
+
<div style="font-family:ui-monospace,'SF Mono',Consolas,monospace; font-weight:800; font-size:14px; letter-spacing:2px; text-transform:uppercase; border-bottom:2px solid currentColor; padding-bottom:5px; margin:26px 0 12px;">05 · PERFORMANCE & QUALITY</div>
|
| 180 |
|
| 181 |
+
<table style="width:100%; border-collapse:collapse; font-family:ui-monospace,'SF Mono',Consolas,monospace; font-size:12.5px;">
|
| 182 |
<tbody>
|
| 183 |
+
<tr><td style="border:1px solid currentColor; padding:8px 11px; width:42%;">DECODE · thinking-off</td><td style="border:1px solid currentColor; padding:8px 11px; font-weight:700;">~37 t/s (Vulkan / Strix Halo)</td></tr>
|
| 184 |
+
<tr><td style="border:1px solid currentColor; padding:8px 11px;">MTP DRAFT ACCEPTANCE · code</td><td style="border:1px solid currentColor; padding:8px 11px; font-weight:700;">~0.8</td></tr>
|
| 185 |
+
<tr><td style="border:1px solid currentColor; padding:8px 11px;">BIGCODEBENCH HARD · instruct · pass@1</td><td style="border:1px solid currentColor; padding:8px 11px; font-style:italic;">measuring — result posted here</td></tr>
|
| 186 |
+
<tr><td style="border:1px solid currentColor; padding:8px 11px;">QUANTIZATION</td><td style="border:1px solid currentColor; padding:8px 11px;">non-imatrix (measured better for code)</td></tr>
|
| 187 |
</tbody>
|
| 188 |
</table>
|
| 189 |
|
| 190 |
+
**Why no imatrix (we measured it):** a code-weighted importance matrix improved fidelity-to-BF16 (median KL **−15%**, top-token +0.6 pp) but **measurably worsened held-out-code perplexity (+2.6%, significant)**. For a coder, code-prediction is the task-relevant metric, so we shipped the **non-imatrix** quant. (On [Qwen3-Coder-Next](https://huggingface.co/plunderstruck/Qwen3-Coder-Next-ROCmFP4-GGUF) the imatrix *was* a clean win — it's model-dependent.)
|
| 191 |
|
| 192 |
+
<div style="font-family:ui-monospace,'SF Mono',Consolas,monospace; font-weight:800; font-size:14px; letter-spacing:2px; text-transform:uppercase; border-bottom:2px solid currentColor; padding-bottom:5px; margin:26px 0 12px;">06 · BUILD (REPRODUCIBLE)</div>
|
| 193 |
|
| 194 |
```bash
|
| 195 |
# from Jackrong's BF16+MTP GGUF -> ROCmFP4, genuine f16 embeddings, no imatrix
|
|
|
|
| 203 |
Qwopus3.6-27B-Coder-MTP-ROCmFP4-STRIX-embF16-headQ6.gguf Q4_0_ROCMFP4_STRIX
|
| 204 |
```
|
| 205 |
|
|
|
|
| 206 |
> Experimental research build for AMD Strix Halo — hardware/driver/prompt-sensitive, may not reproduce elsewhere. Not native FP4 tensor-core execution.
|
| 207 |
|
| 208 |
+
<div style="font-family:ui-monospace,'SF Mono',Consolas,monospace; font-weight:800; font-size:14px; letter-spacing:2px; text-transform:uppercase; border-bottom:2px solid currentColor; padding-bottom:5px; margin:26px 0 12px;">07 · LINEAGE & CREDITS</div>
|
| 209 |
|
| 210 |
+
<table style="width:100%; border-collapse:collapse; font-family:ui-monospace,'SF Mono',Consolas,monospace; font-size:12.5px;">
|
| 211 |
+
<tbody>
|
| 212 |
+
<tr><td style="border:1px solid currentColor; padding:8px 11px; width:26%;">BASE MODEL</td><td style="border:1px solid currentColor; padding:8px 11px;"><a href="https://huggingface.co/Jackrong/Qwopus3.6-27B-Coder-MTP-GGUF">Jackrong/Qwopus3.6-27B-Coder-MTP</a> (Apache-2.0) · from Qwen3.6-27B (Qwen)</td></tr>
|
| 213 |
+
<tr><td style="border:1px solid currentColor; padding:8px 11px;">FORMAT + RUNTIME</td><td style="border:1px solid currentColor; padding:8px 11px;"><a href="https://github.com/charlie12345/rocmfp4-llama">charlie12345/rocmfp4-llama</a> (llama.cpp, MIT)</td></tr>
|
| 214 |
+
<tr><td style="border:1px solid currentColor; padding:8px 11px;">CHAT TEMPLATE</td><td style="border:1px solid currentColor; padding:8px 11px;"><a href="https://huggingface.co/froggeric/Qwen-Fixed-Chat-Templates">froggeric/Qwen-Fixed-Chat-Templates</a></td></tr>
|
| 215 |
+
</tbody>
|
| 216 |
+
</table>
|
| 217 |
|
| 218 |
*Derivative quantization — verify the base model's license before redistribution / use.*
|