philtheriver commited on
Commit
b101fa3
·
verified ·
1 Parent(s): d138f17

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +9 -9
README.md CHANGED
@@ -1,7 +1,6 @@
1
  ---
2
  license: apache-2.0
3
- base_model:
4
- - Jackrong/Qwopus3.6-27B-Coder-MTP-GGUF
5
  base_model_relation: quantized
6
  pipeline_tag: text-generation
7
  library_name: llama.cpp
@@ -27,8 +26,8 @@ tags:
27
 
28
  A **ROCmFPX** quantization of [`Jackrong/Qwopus3.6-27B-Coder-MTP-GGUF`](https://huggingface.co/Jackrong/Qwopus3.6-27B-Coder-MTP-GGUF) built for **AMD Strix Halo** (Ryzen AI MAX, `gfx1151`, 128 GB unified). 27B dense, Apache-2.0, MTP head intact (`draft=2`), agent/tool-use tuned, multimodal (Qwen3-VL).
29
 
30
- > ⚠️ **Requires the [ROCmFPX llama.cpp fork](https://github.com/charlie12345/ROCmFPX).**
31
- > `ROCmFPX` is a family of **custom AMD quant types** (enum IDs 110–115). This file will **NOT load** in stock `llama.cpp`, LM Studio, Ollama, or any other runtime — only the fork's `llama-quantize`/`llama-server` understand it. Build the fork first (see *Running* below).
32
 
33
  > ℹ️ **The Hugging Face precision badge is wrong.** HF's GGUF parser can't read ROCmFPX block layouts and will mislabel this (e.g. "F16"). Pick the file by **filename**; the real footprint is **~6.5 bpw** (+ f16 token embeddings + Q6_K output head).
34
 
@@ -61,7 +60,7 @@ llama-quantize \
61
 
62
  ## Quant tiers (see the **Files** tab)
63
 
64
- Every tier carries the MTP head (`draft=2`) and pairs with the bundled `mmproj-F32.gguf` (vision).
65
  f16 token embeddings throughout; `_AGENT` presets route attn/FFN at higher precision for tool-call /
66
  JSON coherence. Filenames are prefixed `Qwopus3.6-27B-Coder-MTP-STRIX-`.
67
 
@@ -79,8 +78,9 @@ JSON coherence. Filenames are prefixed `Qwopus3.6-27B-Coder-MTP-STRIX-`.
79
  ## Running (ROCmFPX fork required)
80
 
81
  ```bash
82
- # 1) build the fork once (ROCm/HIP toolchain, gfx1151)
83
  git clone https://github.com/charlie12345/ROCmFPX.git && cd ROCmFPX
 
84
  JOBS=16 scripts/build-strix-rocmfp4-mtp.sh # -> build-strix-rocmfp4/bin/
85
 
86
  # 2) serve with MTP speculative decoding + vision
@@ -90,12 +90,12 @@ build-strix-rocmfp4/bin/llama-server \
90
  --alias qwopus-coder \
91
  -dev ROCm0 -ngl 999 -fa on -c 32768 \
92
  --spec-type draft-mtp --spec-draft-ngl all --spec-draft-n-max 2 \
93
- --jinja --mmproj mmproj-F32.gguf \
94
  --host 0.0.0.0 --port 8080
95
  ```
96
 
97
  - **MTP:** the head is built in (`draft=2`). Run with `--spec-type draft-mtp`; `--spec-draft-n-max 2` was optimal in testing.
98
- - **Vision (Qwen3-VL):** bundled `mmproj-F32.gguf`; pass `--mmproj` (add `--image-min-tokens 1024` for high-detail images).
99
  - **Tool calls:** point your client at the **`qwen3_coder`** tool-call parser, or the model narrates code instead of emitting structured calls.
100
  - The chat template is embedded — `--jinja` reads it.
101
 
@@ -130,7 +130,7 @@ build-strix-rocmfp4/bin/llama-server \
130
 
131
  - **Prompt processing ~30% faster than stock `Q6_K`** (despite the larger file) — the gfx1151-tuned ROCmFPX kernels win the compute-bound prefill; a real payoff for long-context / agentic workloads.
132
  - **Decode (`tg`) is bandwidth-bound** — ≈ `Q6_K` at equal size; the format doesn't change it. ¹ Raw decode — **MTP (`--spec-type draft-mtp`) roughly doubles it** in serving.
133
- - Benched on the 27B Coder build; v2 is identical arch/format/size so figures are equivalent. The FP4 row is the v2 STRIX_LEAN as a bit-width reference.
134
 
135
  ## Credits & license
136
 
 
1
  ---
2
  license: apache-2.0
3
+ base_model: Jackrong/Qwopus3.6-27B-Coder-MTP-GGUF
 
4
  base_model_relation: quantized
5
  pipeline_tag: text-generation
6
  library_name: llama.cpp
 
26
 
27
  A **ROCmFPX** quantization of [`Jackrong/Qwopus3.6-27B-Coder-MTP-GGUF`](https://huggingface.co/Jackrong/Qwopus3.6-27B-Coder-MTP-GGUF) built for **AMD Strix Halo** (Ryzen AI MAX, `gfx1151`, 128 GB unified). 27B dense, Apache-2.0, MTP head intact (`draft=2`), agent/tool-use tuned, multimodal (Qwen3-VL).
28
 
29
+ > ⚠️ **Requires the [ROCmFPX llama.cpp fork](https://github.com/charlie12345/ROCmFPX), built at commit [`553504b`](https://github.com/charlie12345/ROCmFPX/commit/553504bd5d16fe1374e9b1508a1f395c9a35d746).**
30
+ > `ROCmFPX` is a family of **custom AMD quant types**; this build uses **enum IDs 110–115**. Those IDs are **not upstream-stable** — a newer fork revision could renumber them and silently mis-decode these weights, so **build from the pinned commit above**. The file will **NOT load** in stock `llama.cpp`, LM Studio, Ollama, or any other runtime — only the fork's `llama-quantize`/`llama-server` understand it.
31
 
32
  > ℹ️ **The Hugging Face precision badge is wrong.** HF's GGUF parser can't read ROCmFPX block layouts and will mislabel this (e.g. "F16"). Pick the file by **filename**; the real footprint is **~6.5 bpw** (+ f16 token embeddings + Q6_K output head).
33
 
 
60
 
61
  ## Quant tiers (see the **Files** tab)
62
 
63
+ Every tier carries the MTP head (`draft=2`) and pairs with the bundled `mmproj/mmproj-F32.gguf` (vision).
64
  f16 token embeddings throughout; `_AGENT` presets route attn/FFN at higher precision for tool-call /
65
  JSON coherence. Filenames are prefixed `Qwopus3.6-27B-Coder-MTP-STRIX-`.
66
 
 
78
  ## Running (ROCmFPX fork required)
79
 
80
  ```bash
81
+ # 1) build the fork at the pinned commit (ROCm/HIP toolchain, gfx1151)
82
  git clone https://github.com/charlie12345/ROCmFPX.git && cd ROCmFPX
83
+ git checkout 553504b # enum IDs 110-115 match this revision
84
  JOBS=16 scripts/build-strix-rocmfp4-mtp.sh # -> build-strix-rocmfp4/bin/
85
 
86
  # 2) serve with MTP speculative decoding + vision
 
90
  --alias qwopus-coder \
91
  -dev ROCm0 -ngl 999 -fa on -c 32768 \
92
  --spec-type draft-mtp --spec-draft-ngl all --spec-draft-n-max 2 \
93
+ --jinja --mmproj mmproj/mmproj-F32.gguf \
94
  --host 0.0.0.0 --port 8080
95
  ```
96
 
97
  - **MTP:** the head is built in (`draft=2`). Run with `--spec-type draft-mtp`; `--spec-draft-n-max 2` was optimal in testing.
98
+ - **Vision (Qwen3-VL):** bundled `mmproj/mmproj-F32.gguf`; pass `--mmproj` (add `--image-min-tokens 1024` for high-detail images).
99
  - **Tool calls:** point your client at the **`qwen3_coder`** tool-call parser, or the model narrates code instead of emitting structured calls.
100
  - The chat template is embedded — `--jinja` reads it.
101
 
 
130
 
131
  - **Prompt processing ~30% faster than stock `Q6_K`** (despite the larger file) — the gfx1151-tuned ROCmFPX kernels win the compute-bound prefill; a real payoff for long-context / agentic workloads.
132
  - **Decode (`tg`) is bandwidth-bound** — ≈ `Q6_K` at equal size; the format doesn't change it. ¹ Raw decode — **MTP (`--spec-type draft-mtp`) roughly doubles it** in serving.
133
+ - Measured on **this** (Coder) build. The FP4 row is the v2 STRIX_LEAN — the only 4-bit on hand — as a bit-width reference.
134
 
135
  ## Credits & license
136