README.md · kingjones777/OxCoder-9B-MTP-ROCmFP4-GGUF at main

Model card: measured results, reproduction, known issues
5053215 verified about 1 month ago
44 kB
---
license: apache-2.0
base_model: OrionLLM/OxCoder-9B
base_model_relation: quantized
pipeline_tag: image-text-to-text
library_name: gguf
tags:
  - gguf
  - llama.cpp
  - rocm
  - amd
  - rocmfp4
  - rocmfpx
  - strix-halo
  - amd-strix-halo
  - gfx1151
  - ryzen-ai-max
  - ryzen-ai-max-395
  - radeon-8060s
  - mtp
  - speculative-decoding
  - code
  - coding
  - multimodal
  - vision
  - oxcoder
  - qwen3.5
  - quantized
---
# OxCoder-9B — ROCmFP4 for AMD Strix Halo (gfx1151)

ROCmFP4 / ROCmFPX quantizations of **[OrionLLM/OxCoder-9B](https://huggingface.co/OrionLLM/OxCoder-9B)** —
9,409,813,744 parameters (BF16), a 9B coding model (32-layer dense Qwen3.5 hybrid: 24 Gated DeltaNet linear-attention and 8 full-attention layers, every 4th being full attention; hidden 4096, FFN 12288, 16 attention / 4 KV heads, 262,144-token context), text + image — built and measured on an AMD Ryzen AI
Max+ 395 (Radeon 8060S, `gfx1151`). The checkpoint ships **no** `mtp.*` tensors (it declares
`mtp_num_hidden_layers: 1`); every file here carries a grafted in-file MTP head from the declared base model.

- **Grafted MTP head inside every file.** Measured on ROCm0 at n-max 3: **64.36 tok/s vs 38.50 without it (1.67×)**, draft acceptance 0.743 ([Speed](#speed-making-it-fast)).
- **Prompt caching with MTP needs the included server patch** in `recipe/patches/` — without it, a build of this ROCmFPX line reuses nothing while the head is loaded ([measured](#prompt-caching-with-mtp)).
- **Vision projector included.**
- Importance-matrix builds of the same three 4-bit tiers: **[kingjones777/OxCoder-9B-MTP-ROCmFP4-imatrix-GGUF](https://huggingface.co/kingjones777/OxCoder-9B-MTP-ROCmFP4-imatrix-GGUF)**.

## Which file should I use?

AMD Ryzen AI Max+ 395 (Radeon 8060S) (MAX-1), ROCm 7.2.4, patched `llama-server` (see [Quick start](#quick-start)), `-c 65536`, one request at a time (`--parallel 1`), greedy (`temp 0`, `top_k 1`), `ignore_eos` so every arm generates exactly 256 tokens after a code prompt of 7,094–7,102 tokens (the first 30,000 characters of `convert_hf_to_gguf.py` plus an instruction), a unique nonce per request and `cache_prompt: false` (`cache_n = 0` asserted on every timed request), 1 warm-up then the median of 3. Decode numbers are the server's own `predicted_per_second`. Box iced: no other model loaded (except the rows marked †, see [Vulkan0 prefill and CPU activity](#vulkan0-prefill-and-cpu-activity)).

| File | ftype | Size | BPW⁴ | KLD vs BF16 ↓² | Same top-1 ↑ | PPL (× BF16) | TG MTP ROCm0 | TG MTP Vulkan0 | PP ROCm0 |
| --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
| `OxCoder-9B-MTP-Q4_0-ROCmFP4-STRIX_LEAN.gguf` | 106 | 5.03 GiB | 4.69 | 0.0879 ± 0.0034 | 89.29 % | 6.7713 ± 0.0829 (×1.0185) | 65.80 | 60.98‡ | 898 |
| `OxCoder-9B-MTP-Q4_0-ROCmFP4-COHERENT.gguf` | 102 | 5.34 GiB | 4.97 | 0.0857 ± 0.0033 | 89.31 % | 6.6389 ± 0.0802 (×0.9986) | 57.40† | 60.53‡ | 803† |
| `OxCoder-9B-MTP-Q4_0-ROCmFP4-FAST.gguf` | 103 | 4.85 GiB | 4.52 | 0.0890 ± 0.0033 | 88.94 % | 6.8166 ± 0.0835 (×1.0253) | 64.81† | 61.72 | 899† |
| *BF16 reference* | 32 | 17.14 GiB³ | 16.00 | 0 | 100 % | 6.6482 ± 0.0810¹ | — | — | — |

TG = decode tokens/s after the 7,094–7,102-token code prompt, grafted MTP head, `--spec-draft-n-max 3`. PP = prefill tokens/s on ROCm0.
¹ The BF16 PPL shown is the paired base every "×" ratio is computed against (averaged over the same scored tokens in the KL-divergence runs). The standalone BF16 run's own summary line reads 6.6532 ± 0.0813.
² Quality columns: see *Where the quality numbers come from* below.
³ BF16 conversion of the grafted checkpoint (MTP block included); not published.
⁴ BPW as printed by `llama-quantize`: bits per weight over the 9,197,093,888 weights in each GGUF — the checkpoint's 8,953,803,264 text weights plus the grafted 243,290,624-weight MTP block. The 9,409,813,744-parameter count above is the checkpoint's: it includes the 456,010,480-weight vision tower, which ships in the projector file, and no MTP block.
† Quiet re-run: the first measurement had requests inside a window when an unrelated job ran on this box; re-run later with other model servers loaded but idle — see [Vulkan0 prefill and CPU activity](#vulkan0-prefill-and-cpu-activity).
‡ Has requests inside a window when an unrelated job ran on this box; kept as measured — see [Vulkan0 prefill and CPU activity](#vulkan0-prefill-and-cpu-activity).

**Start with `STRIX_LEAN`.** `COHERENT`'s KLD is within noise of it (-2.4 %, 0.4σ) for 315 MiB more; `STRIX_LEAN` decodes 14.6 % faster than `COHERENT` on ROCm0 (65.80 vs 57.40† tok/s; per-run ranges 64.36–65.83 and 55.67–58.10) and within 0.7 % of it on Vulkan0 (60.98‡ vs 60.53‡), and prefills 11.8 % faster than it on ROCm0 (897.8 vs 803.4† tok/s).
`FAST` does not buy a clear speed gain here: `FAST` decodes within 1.5 % of `STRIX_LEAN` on ROCm0 (64.81† vs 65.80 tok/s; per-run ranges 62.61–68.03 and 64.36–65.83) and within 1.2 % of it on Vulkan0 (61.72 vs 60.98‡), and prefills within 0.1 % of it on ROCm0 (898.6† vs 897.8 tok/s); its KLD is within noise of `STRIX_LEAN`'s (0.0890 vs 0.0879, +1.3 %, 0.2σ).
KLD order (lower is closer to BF16; KLD gaps under 2σ are called within noise): `COHERENT` 0.0857, `STRIX_LEAN` 0.0879 (+2.5 % vs `COHERENT`), `FAST` 0.0890 (+3.8 % vs `COHERENT`). `STRIX_LEAN` is 315 MiB smaller than `COHERENT`; `FAST` is 178 MiB smaller than `STRIX_LEAN`. Speed gaps are called a tie below 10.3 % for decode and below 3.0 % (ROCm0) / 3.0 % (Vulkan0) for prefill (3.0 % when the two backends are compared): the larger of 3 % and the widest gap measured between two files that run identical tensors — each imatrix file and its standard twin at the MTP knee (decode 10.3 %; prefill 1.3 % on ROCm0, 0.2 % on Vulkan0). The decode floor is the widest because each twin's drafts are accepted at a different rate (STRIX_LEAN on ROCm0: 0.750 standard vs 0.641 imatrix): the imatrix changes the main model's weights, so the head sees different hidden states and is checked against different target tokens.
**The [imatrix build](https://huggingface.co/kingjones777/OxCoder-9B-MTP-ROCmFP4-imatrix-GGUF) is measurably closer to BF16 at the same size:** `STRIX_LEAN` 0.0879 → 0.0610 (-30.6 %, 6.2σ); `COHERENT` 0.0857 → 0.0550 (-35.9 %, 7.4σ); `FAST` 0.0890 → 0.0637 (-28.4 %, 5.8σ).

Quality is graded against the **BF16 GGUF** (reference logits computed on **CPU**) on a **held-out** corpus (wikitext-2 *test*, `-c 2048`, 40 chunks × 1,023 scored tokens each — the second half of every window, less its first token — = 40,920), never on the imatrix calibration text. **KLD** is the per-token KL divergence of each quant's next-token distribution from BF16's on the same tokens — far more sensitive than perplexity.

**Where the quality numbers come from.** Measured directly on these files, against BF16 logits computed **on CPU** in the same session. The BF16 model's own first-window perplexity was CPU 6.6290, Vulkan0 6.6346 (+0.08 %), ROCm0 6.6463 (+0.26 %): the two GPU backends disagree with CPU and with each other, so the reference logits come from pure CPU (`-dev none -ngl 0 --no-op-offload`), and that run's own first-window value (6.6290) equals the CPU control's. Every file was graded on ROCm0 (the columns above); the standard STRIX_LEAN file was also graded on Vulkan0. The standard STRIX_LEAN grade was run twice on ROCm0 and all 40 per-chunk rows matched.

**Same file, same reference, graded on each backend** — standard STRIX_LEAN KLD ROCm0 / Vulkan0: 0.0879 / 0.0886 (+0.9 %, 0.2σ). The two backends agree within noise on this file.
## Quick start
**1. Download**
```bash
hf download kingjones777/OxCoder-9B-MTP-ROCmFP4-GGUF --local-dir ~/models/oxcoder
```
**2. Build `llama-server` with the prompt-cache patch** — ROCmFPX at the measured commit (ROCm and Vulkan
prerequisites: the project's [build guide](https://github.com/charlie12345/ROCmFPX/blob/d3ca53726109b46fd5cd6fa42f688c707079b44e/docs/build.md))
```bash
git clone https://github.com/charlie12345/ROCmFPX.git && cd ROCmFPX
git checkout d3ca53726109b46fd5cd6fa42f688c707079b44e
git apply ~/models/oxcoder/recipe/patches/mtp-prompt-cache-fix.patch
HIPCXX="$(hipconfig -l)/clang" HIP_PATH="$(hipconfig -R)" \
cmake -S . -B build -DCMAKE_BUILD_TYPE=Release \
  -DGGML_HIP=ON -DGGML_VULKAN=ON -DGPU_TARGETS=gfx1151 \
  -DGGML_HIP_GRAPHS=ON -DGGML_HIP_NO_VMM=ON -DLLAMA_CURL=OFF
cmake --build build --target llama-server -j
```
(The CMake options of the measured build are listed in [Reproduction](#reproduction).)
`d3ca537` is also in the history of the official [ROCmFPX/ROCmFPX](https://github.com/ROCmFPX/ROCmFPX) repository; the
patch was verified against `d3ca53726109b46fd5cd6fa42f688c707079b44e` only.
**3. Serve** — the MTP head is inside the model file, so there is no `--model-draft`:
Text and tool calls, with MTP:
```bash
env LD_LIBRARY_PATH=$PWD/build/bin:/opt/rocm/lib HSA_OVERRIDE_GFX_VERSION=11.5.1 GGML_HIP_ENABLE_UNIFIED_MEMORY=1 \
build/bin/llama-server \
  -m ~/models/oxcoder/OxCoder-9B-MTP-Q4_0-ROCmFP4-STRIX_LEAN.gguf \
  --spec-type draft-mtp \
  --spec-draft-n-max 3 --spec-draft-n-min 0 --spec-draft-p-min 0.0 \
  -ngl 999 -fa on -dio --jinja -fit off --parallel 1 -dev ROCm0 \
  -c 65536 --host 127.0.0.1 --port 8080
```
Images (`--mmproj`), **without** MTP:
```bash
env LD_LIBRARY_PATH=$PWD/build/bin:/opt/rocm/lib HSA_OVERRIDE_GFX_VERSION=11.5.1 GGML_HIP_ENABLE_UNIFIED_MEMORY=1 \
build/bin/llama-server \
  -m ~/models/oxcoder/OxCoder-9B-MTP-Q4_0-ROCmFP4-STRIX_LEAN.gguf \
  --mmproj ~/models/oxcoder/mmproj-OxCoder-9B-BF16.gguf \
  -ngl 999 -fa on -dio --jinja -fit off --parallel 1 -dev ROCm0 \
  -c 65536 --host 127.0.0.1 --port 8080
```
> ⛔ **Images:** with the MTP head loaded, image requests fail on this build ([measured](#vision)). For image input, drop the two `--spec-*` lines.
(`LD_LIBRARY_PATH` avoids a soname clash on machines that also have a Vulkan-only llama.cpp build.) The exact measured
argv is in [Reproduction](#reproduction).
**4. Call** — thinking off (OxCoder's SWE-bench sampling: temperature 1.0, top_p 0.95):

```bash
curl http://127.0.0.1:8080/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{
    "messages": [{"role": "user", "content": "Hello"}],
    "temperature": 1.0,
    "top_p": 0.95,
    "chat_template_kwargs": {"enable_thinking": false}
  }'
```

| Flag | Why |
| --- | --- |
| `--spec-type draft-mtp` | Uses the grafted MTP layer inside the model file; no second file to download or place. |
| `--spec-draft-n-max 3` | llama.cpp's default is 16. On this model the measured knee is n-max 3 — see [Speed](#speed-making-it-fast). |
| `--spec-draft-p-min 0.0` | The default 0.75 gate throttles drafting. |
| `--jinja` | Already on by default in this build; keep it on — the thinking controls (`chat_template_kwargs`) and tool calling rely on the Jinja chat template. |
| `-fit off` | Autofit reads `MemAvailable` on integrated GPUs and can silently shrink context or push tensors to CPU. |
| `-cram <MiB>` | Not set above (default 8 GiB of host RAM for saved prompts). Set it on a shared box — see [Known issues](#known-issues-and-limits). |
| `--mmproj` | Loads the 27-layer vision tower. Drop the flag for text-only. |

> ⚠️ **On an unpatched build the in-file head still drafts, but prompt caching turns off while it is loaded** — every turn re-prefills the whole conversation ([measured](#prompt-caching-with-mtp)). For multi-turn / agentic use there, drop the two `--spec-*` lines.
Requires a llama.cpp build with ROCmFP4 / ROCmFPX tensor-type support; stock llama.cpp rejects these tensor types.
## Reasoning controls
OxCoder ships its own chat template (`template_version` `qwen3.6-froggeric-v21.3`, in the source repo's
`chat_template.jinja`). It reads these `chat_template_kwargs`:
| kwarg | default | effect |
| --- | --- | --- |
| `enable_thinking` | `true` | `true`: the generation prompt opens `<think>\n`; `false`: it emits an empty `<think>\n\n</think>` block (no reasoning) |
| `preserve_thinking` | `true` | earlier assistant turns are re-rendered **with** their reasoning, so multi-turn contexts grow faster than with templates that drop it |
| `auto_disable_thinking_with_tools` | `false` | `true` switches thinking off whenever `tools` are passed |
| `tool_call_format` | `xml` | the tool-call format the system prompt asks for: `<tool_call><function=…><parameter=…>`; `json` asks for a JSON object inside `<tool_call>` instead (not tested here) |
| `max_tool_arg_chars` / `max_tool_response_chars` | `0` (off) | truncate long arguments of earlier tool calls / long tool results, noting the original length |
A `<|think_off|>` or `<|think_on|>` tag inside a system or user message overrides `enable_thinking` (the last tag
wins) and is removed from the text.
A short tool result that starts like an error (`error:`, `Traceback`, `fatal:`, `command not found` …) gets a
"⚠️ SYSTEM WARNING" appended by the template; after two such results in a row (no user turn in between), the next
generation prompt also gets an empty think block, regardless of `enable_thinking`. A `developer` message is treated as
`system`; any other unknown role is rendered as a user turn `[role]: content`.
With thinking on and a small `max_tokens`, the whole budget can go to reasoning and `content` comes back empty —
raise `max_tokens` before concluding the model is broken.
Tool calls in the default format are parsed by llama.cpp natively through the Jinja chat template (on by default).
OxCoder's own card reports SWE-bench at temperature 1.0 / top_p 0.95 and Terminal-Bench / NL2Repo at temperature 1.0 /
top_p 1.0. The tools gate on this card used temperature 1.0, top_p 0.95, top_k disabled, max_tokens 8192.
## Speed: making it fast
Measured on `OxCoder-9B-MTP-Q4_0-ROCmFP4-STRIX_LEAN.gguf` (patched server; 7,094–7,102-token code prompt):
| Config | Decode tok/s (min–max) | Draft acceptance | Prefill tok/s |
| --- | ---: | ---: | ---: |
| ROCm0 · no draft head | 38.50 (38.14–38.54) | — | 1089 |
| Vulkan0 · no draft head | 38.65 (38.63–38.65) | — | 668 |
| ROCm0 · grafted MTP, `n-max 1` | 55.06 (54.99–55.15) | 0.937 | 906 |
| ROCm0 · grafted MTP, `n-max 2` | 62.32 (60.82–62.40) | 0.826 | 904 |
| ROCm0 · grafted MTP, `n-max 3` | **64.36** (64.32–65.00) | 0.743 | 901 |
| ROCm0 · grafted MTP, `n-max 4` | 60.91 (60.53–61.13) | 0.651 | 898 |
| ROCm0 · grafted MTP, `n-max 5` | 58.31 (58.19–58.79) | 0.584 | 896 |
| ROCm0 · grafted MTP, `n-max 6` | 55.45 (55.26–56.73) | 0.526 | 896 |
| Vulkan0 · grafted MTP, `n-max 2` | 60.98 (60.95–61.10) | 0.770 | 605 |
| Vulkan0 · grafted MTP, `n-max 3` | 61.95 (61.89–61.97) | 0.684 | 615 |
| Vulkan0 · grafted MTP, `n-max 4` | 60.82 (59.47–62.03) | 0.628 | 620 |
| ROCm0 · DFlash drafter, `n-max 4` | 48.99 (47.81–49.24) | 0.615 | 982 |
| ROCm0 · DFlash drafter, `n-max 8` | 41.26 (40.14–41.90) | 0.341 | 981 |
| ROCm0 · DFlash drafter, `n-max 15` | 37.74 (34.47–38.74) | 0.168 | 977 |
On ROCm0, `n-max 3` is the fastest depth of the n-max 1–6 sweep: **64.36 tok/s, 1.67× the 38.50 tok/s no-draft baseline**, with 0.743 of drafted tokens accepted. On Vulkan0 (n-max 2–4 measured, around the ROCm0 knee) the fastest is n-max 3: 61.95 tok/s, 1.60× its own 38.65 tok/s no-draft baseline, acceptance 0.684 — within 3.7 % of ROCm0 at its knee, a tie. The separate DFlash drafter ([z-lab/Qwen3.5-9B-DFlash](https://huggingface.co/z-lab/Qwen3.5-9B-DFlash), Q8_0; n-max 4, 8, 15 measured on ROCm0) peaks at n-max 4: 48.99 tok/s, 1.27× the no-draft baseline, acceptance 0.615 — 23.9 % slower than the grafted MTP head at its knee. Prefill with the head loaded vs without it, same prompt: ROCm0 897.8 vs 1089.2 tok/s; Vulkan0 614.2‡ vs 668.3 tok/s. Speed gaps are called a tie below 10.3 % for decode and below 3.0 % (ROCm0) / 3.0 % (Vulkan0) for prefill (3.0 % when the two backends are compared): the larger of 3 % and the widest gap measured between two files that run identical tensors — each imatrix file and its standard twin at the MTP knee (decode 10.3 %; prefill 1.3 % on ROCm0, 0.2 % on Vulkan0). The decode floor is the widest because each twin's drafts are accepted at a different rate (STRIX_LEAN on ROCm0: 0.750 standard vs 0.641 imatrix): the imatrix changes the main model's weights, so the head sees different hidden states and is checked against different target tokens.
Draft acceptance depends on how predictable the text is, so the same file was also run on prose. STRIX_LEAN, code prompt (7,095–7,101 tokens: the first 30,000 characters of `convert_hf_to_gguf.py` plus a coding instruction) vs prose prompt (7,450–7,458 tokens: the first 34,000 characters of wikitext-2 *train* plus a writing instruction) — ROCm0 with the head: code 65.80 tok/s vs prose 48.00 (prose decodes 27.1 % slower than code); Vulkan0 with the head: code 60.98‡ tok/s vs prose 49.75 (prose decodes 18.4 % slower than code); ROCm0 without the head: code 38.50 tok/s vs prose 38.46 (prose decodes within 0.1 % of code); Vulkan0 without the head: code 38.65 tok/s vs prose 38.55 (prose decodes within 0.3 % of code). (Same tie rule as above.)

Every 4-bit file at the MTP knee (`n-max 3`), both backends — separate runs from the depth sweep above (`STRIX_LEAN` at n-max 3 read 64.36 tok/s on ROCm0 and 61.95 on Vulkan0 there):

| File | Backend | Decode tok/s (min–max) | Acceptance | Prefill tok/s |
| --- | --- | ---: | ---: | ---: |
| `OxCoder-9B-MTP-Q4_0-ROCmFP4-STRIX_LEAN.gguf` | ROCm0 | 65.80 (64.36–65.83) | 0.750 | 898 |
| `OxCoder-9B-MTP-Q4_0-ROCmFP4-STRIX_LEAN.gguf` | Vulkan0‡ | 60.98 (60.86–61.24) | 0.672 | 614 |
| `OxCoder-9B-MTP-Q4_0-ROCmFP4-COHERENT.gguf` | ROCm0† | 57.40 (55.67–58.10) | 0.628 | 803 |
| `OxCoder-9B-MTP-Q4_0-ROCmFP4-COHERENT.gguf` | Vulkan0‡ | 60.53 (58.45–60.74) | 0.673 | 709 |
| `OxCoder-9B-MTP-Q4_0-ROCmFP4-FAST.gguf` | ROCm0† | 64.81 (62.61–68.03) | 0.738 | 899 |
| `OxCoder-9B-MTP-Q4_0-ROCmFP4-FAST.gguf` | Vulkan0 | 61.72 (61.72–61.81) | 0.672 | 618 |
| `OxCoder-9B-MTP-imatrix-Q4_0-ROCmFP4-STRIX_LEAN.gguf` | ROCm0 | 59.03 (57.78–59.12) | 0.641 | 898 |
| `OxCoder-9B-MTP-imatrix-Q4_0-ROCmFP4-STRIX_LEAN.gguf` | Vulkan0 | 59.18 (59.18–59.80) | 0.639 | 613 |
| `OxCoder-9B-MTP-imatrix-Q4_0-ROCmFP4-COHERENT.gguf` | ROCm0 | 59.28 (58.16–59.53) | 0.665 | 813 |
| `OxCoder-9B-MTP-imatrix-Q4_0-ROCmFP4-COHERENT.gguf` | Vulkan0 | 63.83 (63.62–64.40) | 0.737 | 609 |
| `OxCoder-9B-MTP-imatrix-Q4_0-ROCmFP4-FAST.gguf` | ROCm0 | 60.01 (58.76–60.11) | 0.656 | 910 |
| `OxCoder-9B-MTP-imatrix-Q4_0-ROCmFP4-FAST.gguf` | Vulkan0 | 61.13 (60.50–61.13) | 0.659 | 617 |

`STRIX_LEAN`: ROCm0 decodes within 7.9 % of Vulkan0 (65.80 vs 60.98‡ tok/s; ranges 64.36–65.83 / 60.86–61.24) and prefills 46.2 % faster than it (897.8 vs 614.2‡ tok/s). `COHERENT`: ROCm0 decodes within 5.2 % of Vulkan0 (57.40† vs 60.53‡ tok/s; ranges 55.67–58.10 / 58.45–60.74) and prefills 13.3 % faster than it (803.4† vs 709.3‡ tok/s; all three Vulkan0 runs overlapped the unrelated job — a quiet re-run read 621.6 tok/s prefill). `FAST`: ROCm0 decodes within 5.0 % of Vulkan0 (64.81† vs 61.72 tok/s; ranges 62.61–68.03 / 61.72–61.81) and prefills 45.4 % faster than it (898.6† vs 618.0 tok/s). (Same tie rule as above.)

### Vulkan0 prefill and CPU activity

While the per-file rows were being measured, an unrelated job ran on this box from 01:50:50 to 01:53:40 UTC: one niced, single-threaded process re-rendering another model's cards, no GPU use (`recipe/logs/O6_background_job_journal.txt`). Rows with requests inside that window, each against the closest like-for-like requests outside it: `STRIX_LEAN` on Vulkan0, run 3: 718 tok/s prefill vs 610–614 for runs 1 and 2 of the same row; `COHERENT` on Vulkan0, all three runs: 709 tok/s prefill vs 604–609 for its imatrix twin (identical tensor types, outside the window); `COHERENT` on ROCm0, all three runs: 809–811 tok/s prefill vs 810–815 for its imatrix twin (identical tensor types, outside the window); `FAST` on ROCm0, run 1: 910 tok/s prefill vs 906–910 for runs 2 and 3 of the same row.

Re-run with the same harness and flags, quiet vs with one niced busy-loop thread (whole-box CPU busy 1.2–3.6 % quiet, 4.4–6.7 % with the thread; this re-run was not on an iced box: 5 other model servers were loaded, all idle before and after every arm):

| file | backend | quiet: prefill / decode tok/s (acceptance) | busy thread: prefill / decode tok/s (acceptance) | prefill change |
| --- | --- | ---: | ---: | ---: |
| `STRIX_LEAN` | Vulkan0 | 625 / 62.46 (0.703) | 721 / 62.58 (0.684) | +15.3 % |
| `COHERENT` | Vulkan0 | 622 / 59.36 (0.662) | 716 / 60.78 (0.669) | +15.1 % |
| `COHERENT` | ROCm0 | 803 / 57.40 (0.628) | 801 / 59.34 (0.644) | -0.2 % |
| imatrix `STRIX_LEAN` (control) | Vulkan0 | 632 / 60.39 (0.646) | 720 / 60.37 (0.642) | +14.0 % |

**On this box a single busy CPU thread made Vulkan0 prefill with the MTP head 14–15 % faster; ROCm0 prefill moved -0.2 %.** Vulkan0 prefill numbers therefore depend on what else the CPU is doing. A second quiet `COHERENT` Vulkan0 run after the busy-thread arms read 620 tok/s (first quiet run: 622). The ROCm0 quiet control (imatrix `COHERENT`, from an undisturbed window) reproduced its original row (prefill 812.9 → 807.4 tok/s, -0.7 %; decode +0.6 %; tolerances ±3 % / ±5 %, set before the re-run), so the `COHERENT` and `FAST` ROCm0 rows show the quiet re-run (marked †). The Vulkan0 quiet control (imatrix `STRIX_LEAN`, from an undisturbed window) did not reproduce its original row within the tolerances set before the re-run (prefill 612.9 → 631.6 tok/s, +3.05 % against ±3 %; decode +2.0 % against ±5 %; the re-run's box state differed: other model servers were loaded), so the `STRIX_LEAN` and `COHERENT` Vulkan0 rows keep the original values (marked ‡): `STRIX_LEAN`'s reported medians are values from its runs outside the window (its in-window run 3 is the extreme of the three in both prefill and decode); all three `COHERENT` runs were inside the window, so its reported values were measured with the job running (its quiet re-run read 622 tok/s prefill and 59.36 tok/s decode). The replaced rows stay in `recipe/results_summary.json` under `bench_superseded`.

## The grafted MTP head

The OxCoder checkpoint has no `mtp.*` tensors. The 15 `mtp.*` tensors of
**[Qwen/Qwen3.5-9B](https://huggingface.co/Qwen/Qwen3.5-9B)** (the declared `base_model`) were fetched by HTTP range
read and written byte-for-byte as an extra safetensors shard next to the untouched OxCoder file; the converter keeps
`mtp.*` as the in-file nextn block. The head was trained for the base model, not for OxCoder.

| | |
| --- | --- |
| donor | `Qwen/Qwen3.5-9B` |
| donor revision | `c202236235762e1c871ad0ccb60c8ee5ba337b9a` |
| tensors | 15 |
| total bytes | 486,581,248 |
| graft file | `model-mtp-graft.safetensors` |
| graft sha256 | `6a1197089cf65461ac0ebec179e172f922884f13c3fe10e565e520ab4bd76051` |
| every tensor BF16 and shape-checked before writing (the script refuses to write otherwise) | yes |

| tensor | shape | bytes | sha256 | shape checked against |
| --- | --- | ---: | --- | --- |
| `mtp.layers.0.mlp.down_proj.weight` | 4096×12288 | 100,663,296 | `52c72564f7da59c25233b2194a79239cc1e69dd6694130aae87d5fffc472707c` | `model.language_model.layers.0.mlp.down_proj.weight` |
| `mtp.layers.0.mlp.gate_proj.weight` | 12288×4096 | 100,663,296 | `dd3e6d05e9c519ebb16c5eee64b4d4d217d7efb8ae7c88e8dfa96f6e6f7d3eac` | `model.language_model.layers.0.mlp.gate_proj.weight` |
| `mtp.layers.0.mlp.up_proj.weight` | 12288×4096 | 100,663,296 | `79e02e92d4cb775120f48cd523577345b277dacb749e56e2d52532583f92800e` | `model.language_model.layers.0.mlp.up_proj.weight` |
| `mtp.fc.weight` | 4096×8192 | 67,108,864 | `a4639d8f4b81cbdc65c61f1cba82816ae1534ec494a95a0b677dba98f04f4017` | `2*hidden -> hidden` |
| `mtp.layers.0.self_attn.q_proj.weight` | 8192×4096 | 67,108,864 | `17d3aac05cb017e9ef98f93fe567f1c1ffae08fbe5001b80e67829bda189a584` | `model.language_model.layers.3.self_attn.q_proj.weight` |
| `mtp.layers.0.input_layernorm.weight` | 4096 | 8,192 | `c8913bfe7cef186fb59b7f1ca80d391ba5eaec45c9c8db54f54e63f6410ae633` | `model.language_model.layers.0.input_layernorm.weight` |
| `mtp.layers.0.post_attention_layernorm.weight` | 4096 | 8,192 | `63500aa0ed93918f6f3d5f1bfd1656921543931392a06267f81c0b3ad680fdcf` | `model.language_model.layers.0.post_attention_layernorm.weight` |
| `mtp.layers.0.self_attn.k_norm.weight` | 256 | 512 | `8f5a8071649fa9a7b6b6afcea6d5404cd849640be2c4f90e6759c4c0a60120ac` | `model.language_model.layers.3.self_attn.k_norm.weight` |
| `mtp.layers.0.self_attn.k_proj.weight` | 1024×4096 | 8,388,608 | `1b0129a3c6f4add6c8417d67e84050f8595f96f7693511beaace9ef56d1ad30a` | `model.language_model.layers.3.self_attn.k_proj.weight` |
| `mtp.layers.0.self_attn.o_proj.weight` | 4096×4096 | 33,554,432 | `31140d974d94190127d0f3d908a18bafe5fba0d8c3d7f522e012013795c89924` | `model.language_model.layers.3.self_attn.o_proj.weight` |
| `mtp.layers.0.self_attn.q_norm.weight` | 256 | 512 | `10e6c9fa42ceb72373c207422c305663a89dfd08f5332cbbe6d1057301fe57d1` | `model.language_model.layers.3.self_attn.q_norm.weight` |
| `mtp.layers.0.self_attn.v_proj.weight` | 1024×4096 | 8,388,608 | `c21332947c44203ca197d014de23559bf6e88d7c0debca8d4ecad735d717e2c9` | `model.language_model.layers.3.self_attn.v_proj.weight` |
| `mtp.norm.weight` | 4096 | 8,192 | `7e4daf06ad25b834b3b95c9592fa690b75a4a90fb9f4128ccee8537d7f15988e` | `hidden-size norm` |
| `mtp.pre_fc_norm_embedding.weight` | 4096 | 8,192 | `8b9bca62497def4783b20dfd58ddf32573a255b44a79f0acf8ca9c611b7bea15` | `hidden-size norm` |
| `mtp.pre_fc_norm_hidden.weight` | 4096 | 8,192 | `a7410736f9962dd6ebd8ac2b4c355898d057296a03c47795223b7298cac8988c` | `hidden-size norm` |

| File | nextn tensors (read-back) | MTP block (`blk.32.*`) | MTP block bytes |
| --- | ---: | ---: | ---: |
| *BF16* | 4 | 15 | 486,623,232 |
| `OxCoder-9B-MTP-Q4_0-ROCmFP4-STRIX_LEAN.gguf` | 4 | 15 | 147,408,896 |
| `OxCoder-9B-MTP-Q4_0-ROCmFP4-COHERENT.gguf` | 4 | 15 | 153,700,352 |
| `OxCoder-9B-MTP-Q4_0-ROCmFP4-FAST.gguf` | 4 | 15 | 147,146,752 |
| `OxCoder-9B-MTP-imatrix-Q4_0-ROCmFP4-STRIX_LEAN.gguf` | 4 | 15 | 147,408,896 |
| `OxCoder-9B-MTP-imatrix-Q4_0-ROCmFP4-COHERENT.gguf` | 4 | 15 | 153,700,352 |
| `OxCoder-9B-MTP-imatrix-Q4_0-ROCmFP4-FAST.gguf` | 4 | 15 | 147,146,752 |

At n-max 3 on ROCm0, the grafted head's measured acceptance was 0.743 and STRIX_LEAN decoded at 64.36 tok/s — 1.67× the 38.50 tok/s no-draft baseline on the same backend and the same prompt. The head was trained for Qwen3.5-9B, not for OxCoder; those figures are measurements on the benchmark prompts above, not a claim about other prompts.
## Is the output the same as without MTP?
**Greedy output with MTP matched greedy output without a draft head on 2 of 6 prompts** (every pair run cold at temperature 0). The 6 prompts are the identity gate's fixed prompt (1) plus the 5 prompts the aligned cache gate also ran without a draft head (`o-c3-*` rows; cold reply compared with cold reply). Without a draft head, the unpatched and patched servers produced identical output on the fixed prompt. A mismatch is not a quality regression: with a draft head loaded the server evaluates the same math in different batch shapes (drafts are verified several tokens at a time, and the head needs the hidden state of every prompt position), so float rounding can differ and a near-tie can break the other way; from that token on, the two replies diverge. If you need output that is byte-for-byte reproducible against a no-MTP run, serve without `--spec-type`.
## Prompt caching with MTP
Measured: a second turn that shares the first turn's long prefix and differs only in its closing instruction, each
exchange run warm (`cache_prompt: true`) and then cold (`cache_prompt: false`) and the two replies compared byte for byte.
Every prompt in the patched gate is padded to one token length so warm and cold see identical chunking.
| server | draft head | second-turn prompt tokens reused | processed | warm reply = cold reply |
| --- | --- | ---: | ---: | :---: |
| d3ca537, unpatched | grafted MTP | **0** of 7,067 | 7,067 | — |
| d3ca537 + patch | grafted MTP | **6,011** of 7,039 (at the aligned checkpoint in 5/5 exchanges) | 1,028 | 5/5 |
| d3ca537 + patch | none (control) | **6,011** of 7,039 (at the aligned checkpoint in 5/5 exchanges) | 1,028 | 5/5 |
⛔ **Unpatched, loading the draft head turns prompt reuse off.** On the unmodified d3ca537 server with the grafted head loaded, a second turn sharing a long prefix reused **0** of its 7,067 prompt tokens.
✅ **Patched, reuse works.** In 5 two-turn exchanges at a fixed prompt length of 7,039 tokens, every second turn resumed from the checkpoint 1,028 tokens before its end — **6,011 tokens reused (85 %), 1,028 processed** — median prefill **1.3 s instead of 7.8 s** cold (6.2× faster), with the draft head still loaded.
**Resuming does not change the output:** each warm reply was byte-identical to a cold run of the same prompt in **5/5** exchanges with the grafted head. Patched with no draft head (control): warm = cold in **5/5** exchanges.
One behaviour to know about, which belongs to llama-server rather than to this patch: the server processes the last `n_ubatch` + 4 tokens of every prompt as two separate batches so it can checkpoint there (1,024 + 4 = 1,028 tokens with the `-ub 1024` used in these measurements, where the server default is `-ub 512`; [upstream PR #20288](https://github.com/ggml-org/llama.cpp/pull/20288)). A turn that resumes from a checkpoint left by a prompt of a *different* length therefore splits its tail differently from a cold run, and float rounding can flip a greedy token. Both replies are ordinary greedy decodes; they are just not bit-identical to each other. That is why every prompt in this gate has the same length.

## Tool calling

The template emits the XML-style `<tool_call><function=…><parameter=…>` format, which llama.cpp parses natively
through the Jinja chat template (on by default). Suite run through `llama-server` in the deployed configuration
(grafted MTP on):

**14/14** on native `tool_calls` (raw XML left in `content` counts as a failure), run on `OxCoder-9B-MTP-Q4_0-ROCmFP4-STRIX_LEAN.gguf` with the grafted head on, at OxCoder's SWE-bench sampling (temperature 1.0, top_p 0.95, top_k disabled, max_tokens 8192).
| check | thinking ON (`enable_thinking: true`) | thinking OFF (`enable_thinking: false`) |
| --- | :---: | :---: |
| multi-arg | ✅ | ✅ |
| nested-object | ✅ | ✅ |
| enum | ✅ | ✅ |
| correct-decline | ✅ | ✅ |
| multi-turn | ✅ | ✅ |
| streaming | ✅ | ✅ |
| parallel | ✅ | ✅ |
Every check passed in both thinking modes (`enable_thinking: true` and `false`).

## Vision

`mmproj-OxCoder-9B-BF16.gguf` is the 27-layer vision tower (width 1152), loaded with `--mmproj`.
Its attention follows the server's `-fa` setting, so both settings were checked, with and without the MTP head.

Probe: a synthetic 448×448 image with a red circle and a blue square (a model that ignores the image cannot name both), sent to `OxCoder-9B-MTP-Q4_0-ROCmFP4-STRIX_LEAN.gguf` on the patched server with `--mmproj`, temperature 0. Pass = the reply names every expected term (`red`, `blue`, `circle`, `square`).

| MTP head | `-fa on` | `-fa off` |
| --- | :---: | :---: |
| loaded | ❌ server stopped | ❌ server stopped |
| not loaded | ✅ 4/4 terms | ✅ 4/4 terms |
| loaded, **unpatched** server (control) | ❌ server stopped | — |

⛔ **With the MTP head loaded, image requests fail** with `-fa on` and with `-fa off` — the server aborted (`process: missing MTP boundary for seq_id=0 pos=17 (current=3/1 previous=2/1)` → `update_slots: failed to process speculative batch`). Without the head they work. **For image input, serve without `--spec-type`.** The unmodified `d3ca537` server aborts the same way, so this is not caused by the prompt-cache patch; the server log points to the MTP state not being advanced across an image's positions, which jump (M-RoPE).

Reply (no head, `-fa on`; first 300 characters as stored by the harness):

> The image contains two distinct geometric shapes placed side by side on a white background:
>
> 1. **Left Shape**: A solid red circle. It is perfectly round with no visible outline or gradient — just a flat, uniform red fill.
>
> 2. **Right Shape**: A solid blue square. It has sharp, straight edges and ri

## Memory

Measured footprint (drop in `MemAvailable`) of STRIX_LEAN on ROCm0 with the grafted MTP head loaded (n-max 3, `--spec-draft-p-min 0.0`), the vision projector, q8_0 KV cache, `--no-mmap`, `-cram 512`, one slot, patched server. *After one request* is after a single request whose prompt is the first 30,000 characters of `convert_hf_to_gguf.py` plus a one-line instruction, with 64 tokens generated.

| context | after load | after one request |
| ---: | ---: | ---: |
| 65,536 | 10.82 GiB | 16.46 GiB |
| 262,144 | 17.37 GiB | 22.45 GiB |

No row was decode-benchmarked beyond that one request. Nothing beyond these rows was measured.

## Quantization methodology

```bash
# 0. graft the 15 mtp.* tensors of Qwen/Qwen3.5-9B onto the OxCoder checkpoint (byte-for-byte extra shard):
#    hf/ is the untouched OxCoder download, graft/ a hard-linked copy of it plus the new shard
cp -al hf graft && rm -rf graft/.cache
python3 oxcoder_graft.py hf graft/model-mtp-graft.safetensors
# 1. convert: grafted checkpoint WITH its MTP layer (the converter's default), and the vision projector
python convert_hf_to_gguf.py graft --outtype bf16 --model-name OxCoder-9B --outfile OxCoder-9B-MTP-BF16.gguf
python convert_hf_to_gguf.py graft --outtype bf16 --mmproj --model-name OxCoder-9B --outfile mmproj-OxCoder-9B-BF16.gguf
# 2. quantize from BF16 only; the LM head is forced up on every tier and read back by exact tensor name
llama-quantize --output-tensor-type q6_K                            OxCoder-9B-MTP-BF16.gguf OUT Q4_0_ROCMFP4_STRIX_LEAN 8
llama-quantize --output-tensor-type q6_K --token-embedding-type q6_K OxCoder-9B-MTP-BF16.gguf OUT Q4_0_ROCMFP4_COHERENT 8
llama-quantize --output-tensor-type q6_K                            OxCoder-9B-MTP-BF16.gguf OUT Q4_0_ROCMFP4_FAST 8
# 3. BF16 reference logits on CPU (chunk-1 control, BF16 first-window PPL: CPU 6.6290, Vulkan0 6.6346, ROCm0 6.6463)
llama-perplexity -m OxCoder-9B-MTP-BF16.gguf -f wikitext-2-raw/wiki.test.raw -c 2048 -b 2048 --chunks 40 --kl-divergence-base bf16.kld \
  -dev none -ngl 0 --no-op-offload -t 16
# 4. grade each shipped file against those logits on ROCm0 (STRIX_LEAN also on Vulkan0)
llama-perplexity -m OUT --kl-divergence-base bf16.kld --kl-divergence -c 2048 -b 2048 -ngl 999 -fa on -dio -dev ROCm0
llama-perplexity -m OUT --kl-divergence-base bf16.kld --kl-divergence -c 2048 -b 2048 -ngl 999 -fa on -dio -dev Vulkan0
# 5. DFlash drafter (not redistributed): this fork's converter, then Q8_0
python convert_hf_to_gguf.py dflash-hf --outtype bf16 --target-model-dir graft \
  --outfile gguf/dflash-Qwen3.5-9B-BF16.gguf
llama-quantize gguf/dflash-Qwen3.5-9B-BF16.gguf out/dflash-Qwen3.5-9B-Q8_0.gguf Q8_0 16
```

Receipts (the built file is the receipt — exact tensor names, never a substring match; `recipe/logs/`):

| File | `output.weight` | `token_embd.weight` | tensors | nextn |
| --- | --- | --- | ---: | ---: |
| `OxCoder-9B-MTP-Q4_0-ROCmFP4-STRIX_LEAN.gguf` | Q6_K | Q5_K | 442 | 4 |
| `OxCoder-9B-MTP-Q4_0-ROCmFP4-COHERENT.gguf` | Q6_K | Q6_K | 442 | 4 |
| `OxCoder-9B-MTP-Q4_0-ROCmFP4-FAST.gguf` | Q6_K | Q4_0_ROCMFP4_FAST | 442 | 4 |

Where the three tiers differ, read from the standard files (`blk.N` = every layer that has the tensor, the MTP block included; families not listed have the same type in all three):

| tensor family | tensors | STRIX_LEAN | COHERENT | FAST |
| --- | ---: | --- | --- | --- |
| `blk.N.attn_gate.weight` | 24 | Q4_0_ROCMFP4_FAST | Q4_0_ROCMFP4 | Q4_0_ROCMFP4_FAST |
| `blk.N.attn_k.weight` | 9 | Q4_0_ROCMFP4 | Q4_0_ROCMFP4 | Q4_0_ROCMFP4_FAST |
| `blk.N.attn_output.weight` | 9 | Q4_0_ROCMFP4_FAST | Q4_0_ROCMFP4 | Q4_0_ROCMFP4_FAST |
| `blk.N.attn_q.weight` | 9 | Q4_0_ROCMFP4_FAST | Q4_0_ROCMFP4 | Q4_0_ROCMFP4_FAST |
| `blk.N.attn_qkv.weight` | 24 | Q4_0_ROCMFP4 | Q4_0_ROCMFP4 | Q4_0_ROCMFP4_FAST |
| `blk.N.attn_v.weight` | 9 | Q4_0_ROCMFP4 | Q4_0_ROCMFP4 | Q4_0_ROCMFP4_FAST |
| `blk.N.ffn_down.weight` | 33 | Q4_0_ROCMFP4_FAST | Q4_0_ROCMFP4 | Q4_0_ROCMFP4_FAST |
| `blk.N.ffn_gate.weight` | 33 | Q4_0_ROCMFP4_FAST | Q4_0_ROCMFP4 | Q4_0_ROCMFP4_FAST |
| `blk.N.ffn_up.weight` | 33 | Q4_0_ROCMFP4_FAST | Q4_0_ROCMFP4 | Q4_0_ROCMFP4_FAST |
| `blk.N.ssm_alpha.weight` | 24 | Q4_0_ROCMFP4_FAST | Q4_0_ROCMFP4 | Q4_0_ROCMFP4_FAST |
| `blk.N.ssm_beta.weight` | 24 | Q4_0_ROCMFP4_FAST | Q4_0_ROCMFP4 | Q4_0_ROCMFP4_FAST |
| `blk.N.ssm_out.weight` | 24 | Q4_0_ROCMFP4_FAST | Q4_0_ROCMFP4 | Q4_0_ROCMFP4_FAST |
| `output.weight` | 1 | Q6_K | Q6_K | Q6_K |
| `token_embd.weight` | 1 | Q5_K | Q6_K | Q4_0_ROCMFP4_FAST |
`tie_word_embeddings` is false, so the output head is a real tensor and `--output-tensor-type q6_K` does real work. All three tiers pin `output.weight` to `q6_K`; COHERENT also pins `token_embd.weight` to `q6_K`, while STRIX_LEAN and FAST keep their tier's own embedding type (shown in the receipts).

## Reproduction

```
server   : https://github.com/charlie12345/ROCmFPX @ d3ca53726109b46fd5cd6fa42f688c707079b44e
           + recipe/patches/mtp-prompt-cache-fix.patch (sha256 412a0173f60faa937018f0153efe9b5d9230c1606735502485e7e520fffb68dd)
           built target llama-server with the same CMake options as below; sha256 14e9f1299240c891478c820b8c20811ca5721044d2e63e23799eaa9be26b30bd
tools    : same commit, unpatched (quantize / imatrix / perplexity; the patch touches only common/ and tools/server/)
           build dir /opt/llama-rocm/rocmfpx-724/build-hipvk, Release, Unix Makefiles, GGML_HIP=ON GGML_VULKAN=ON
           GGML_HIP_GRAPHS=ON GGML_HIP_NO_VMM=ON GGML_NATIVE=ON AMDGPU_TARGETS=gfx1151 LLAMA_CURL=OFF
           (this CMake forwards AMDGPU_TARGETS to GPU_TARGETS, the spelling used in Quick start)
           CMAKE_HIP_COMPILER=/opt/rocm-7.2.4/lib/llvm/bin/clang
           sha256 llama-quantize   1559189968a30781f837e4d25ca3cdf26c8eb0351023675406535eddcbe0db13
           sha256 llama-imatrix    bfdce5c21a01bdc8043d69335b74a7b68b8335ae989991a1e10e21d5faeaeb8e
           sha256 llama-perplexity c2175925bb0c61df89ff787845d8e03a35403f684be52719ac2987b8ca636a53
           sha256 llama-server     d077b98f3f04df4b482a5826c8922e39cda3aa8d2e2b2fbe1870f36d245cebe7   (unpatched; used for the unpatched control rows: prompt cache and vision)
source   : OrionLLM/OxCoder-9B  revision 07ba04e2149bace1b43c66448471a70378a7a708
model    : OxCoder-9B-MTP-Q4_0-ROCmFP4-STRIX_LEAN.gguf (the argv below; every file was measured the same way)
argv     : /opt/llama-rocm/rocmfpx-724-mtpcache/build-hipvk/bin/llama-server -m /mnt/models/oxcoder-9b/out/OxCoder-9B-MTP-Q4_0_ROCMFP4_STRIX_LEAN.gguf -dev ROCm0 -ngl 999 -fa on -dio --jinja -fit off --parallel 1 -c 65536 -b 2048 -ub 1024 --host 127.0.0.1 --port 18600 --no-webui --spec-type draft-mtp --spec-draft-n-max 3 --spec-draft-n-min 0 --spec-draft-p-min 0.0
env      : LD_LIBRARY_PATH=<patched build>/bin:/opt/rocm-7.2.4/lib
           HSA_OVERRIDE_GFX_VERSION=11.5.1 GGML_HIP_ENABLE_UNIFIED_MEMORY=1
box      : aimax — AMD RYZEN AI MAX+ 395 w/ Radeon 8060S (gfx1151), 124 GiB RAM, GTT 131072 MiB,
           kernel 6.17.6-061706-generic, ROCm 7.2.4
protocol : 7,094–7,102-token code prompt, 256 generated tokens, temp 0 / top_k 1, ignore_eos, cache_prompt false,
           1 warm-up + median of 3, no co-resident models (box iced)
           (rows marked † were re-run later, quiet, with other model servers loaded but idle)
measured : 2026-09-16 to 2026-09-17, by the pipeline in recipe/ (every raw number in recipe/results_summary.json and recipe/raw/)
```

## Files

| File | Size | sha256 |
| --- | ---: | --- |
| `OxCoder-9B-MTP-Q4_0-ROCmFP4-STRIX_LEAN.gguf` | 5.03 GiB | `1af457fc6f292b98cdaa072295fe3cb6c11293d275914fd706146fbbd23ae5fb` |
| `OxCoder-9B-MTP-Q4_0-ROCmFP4-COHERENT.gguf` | 5.34 GiB | `46c811b654aba00c1b4d1f89ee09f39868a892d2aa344950205123bdea03f817` |
| `OxCoder-9B-MTP-Q4_0-ROCmFP4-FAST.gguf` | 4.85 GiB | `5b96e01da43361c21f56d37d121e448f2715aa082b9c6b9bc0d128fbaa65bf39` |
| `mmproj-OxCoder-9B-BF16.gguf` | 0.86 GiB | `5c3e4e46a7d7d6fc860ba3e044100aac852f61bbfdefee8a6b4cc60fe520d1ee` |

The DFlash drafter (`dflash-Qwen3.5-9B-Q8_0.gguf`, from [z-lab/Qwen3.5-9B-DFlash](https://huggingface.co/z-lab/Qwen3.5-9B-DFlash) @ `5fc3b3d474760f18c516db87d84c37edbfd3ede6`) is **not
redistributed**. The methodology block shows how to convert it with this fork (`--target-model-dir`) and quantize Q8_0.
`SHA256SUMS` covers every model file. `recipe/` holds the measurement pipeline (`recipe/pipeline/`), the prompt-cache
patch (`recipe/patches/`), the graft provenance (`recipe/graft/`), raw per-run results (`recipe/raw/`), build and
receipt logs (`recipe/logs/`), and `results_summary.json` with every measured value on this card. Architecture facts
(layer counts, vocabulary, vision depth) come from the checkpoint's `config.json` at revision `07ba04e2149bace1b43c66448471a70378a7a708`.

## Known issues and limits

- **The MTP head is grafted from the base model.** The 15 `mtp.*` tensors come from [Qwen/Qwen3.5-9B](https://huggingface.co/Qwen/Qwen3.5-9B) at revision `c20223623576` (`recipe/graft/GRAFT_PROVENANCE.json`). They were trained for that checkpoint, not for OxCoder's fine-tune; their usefulness here is the measured acceptance and speed on this box, not a task-accuracy claim.
- **MTP + image requests fail** on this build (they work with the head unloaded) — see [Vision](#vision). For image input, drop the two `--spec-*` lines.
- **Prompt caching with MTP needs the included server patch.** On the unpatched server the in-file head still drafts (40 of 62 drafted tokens accepted in that control), but the measured second turn reused 0 of its 7,067 prompt tokens; for multi-turn use on an unpatched build, drop the two `--spec-*` lines.
- **`preserve_thinking` defaults to true** in this chat template: earlier assistant turns are re-rendered with their reasoning, so multi-turn contexts grow faster than with templates that drop it. Pass `"preserve_thinking": false` in `chat_template_kwargs` to drop it.
- Tool calling was measured only with the template's default `tool_call_format` (XML parameters); the JSON variant (`"tool_call_format": "json"`) was not tested.
- **`llama-server`'s host-RAM prompt cache defaults to 8 GiB** (`-cram 8192`). On a shared box, set `-cram` explicitly.
- Measured on Linux only (AMD RYZEN AI MAX+ 395 w/ Radeon 8060S, ROCm 7.2.4, patched `d3ca537`).
- **Not measured:** decode beyond a 7,458-token prompt (a 262,144-token context was loaded in the memory test, not benchmarked), long-context quality, video input, concurrency above 1, and task-level accuracy. Perplexity/KLD measure next-token fidelity to BF16 on prose, not reasoning or code correctness. OxCoder's own SWE-bench / Terminal-Bench / NL2Repo numbers are for the unquantized checkpoint, not these files.

## License and attribution

Apache-2.0, inherited from the base model. Weights: **OrionLLM**
([OrionLLM/OxCoder-9B](https://huggingface.co/OrionLLM/OxCoder-9B)). MTP head weights: **Qwen**
([Qwen/Qwen3.5-9B](https://huggingface.co/Qwen/Qwen3.5-9B), Apache-2.0). DFlash drafter: z-lab (not redistributed).
ROCmFP4 / ROCmFPX quantization format and runtime: the ROCmFPX project. Patch, graft, quantization and measurements:
kingjones777.