kingjones777/Xing4.0-29B-Intent-ROCmFP4-GGUF

Xing4.0-29B-Intent — ROCmFP4 GGUF (Strix Halo / gfx1151)

4-bit ROCmFP4 builds of XingChen-AGI/Xing4.0-29B-Intent, the schema-guided intent classification and structured-extraction finetune of Xing4.0-29B-A4B (29B total, ~4B active; MLA attention + 64-expert MoE + manifold-constrained hyper-connections, with an MTP layer), built and measured on one AMD Ryzen AI Max+ 395 "Strix Halo" (gfx1151, ROCm 7.2.4, 128 GiB unified memory).

Every number on this page was measured on these files. Nothing is estimated or carried over from the base model.

⛔ These files need the ROCmFPX fork — stock llama.cpp cannot load them

1. ROCmFP4 tensor types are ≥ 100, past stock llama.cpp's GGML_TYPE_COUNT; the file is rejected outright.
2. xing4 is its own architecture (MLA with key width 192 / value width 128 plus hyper-connections), and it lives on a branch of the fork, not main:

git clone -b xing4-port https://github.com/kingjones30/ROCmFPX.git
cd ROCmFPX && cmake -B build -DGGML_HIP=ON -DGPU_TARGETS=gfx1151 \
  -DGGML_HIP_GRAPHS=ON -DGGML_HIP_NO_VMM=ON -DLLAMA_CURL=OFF -DLLAMA_BUILD_WEBUI=OFF
cmake --build build -j --target llama-server llama-bench llama-perplexity

Pin commit cbc2cd5. The build behind every number below has byte-identical xing4, model-loader, server and speculative-decoding sources.

Files

file	tier	size	BPW	PPL long-context ↓	PPL wikitext-2 ↓	pp512	tg128	pp512 @ 4K	tg128 @ 4K	intent	tool calls
…-COHERENT-PAS.gguf ⭐	COHERENT + Q8_0 attention, hc and shared expert	18.76 GB	4.81	7.5294 ± 0.266	9.5500 ± 0.143	889.3	30.17	455.4	7.71	10/10	6/8
…-COHERENT.gguf	COHERENT (ftype 102)	17.85 GB	4.57	7.7713 ± 0.275	9.9729 ± 0.150	871.4	35.33	448.4	8.00	10/10	6/8
…-STRIX_LEAN-imatrix.gguf	STRIX_LEAN (ftype 106) + imatrix	16.78 GB	4.30	8.0411 ± 0.290	9.9830 ± 0.149	879.2	35.91	450.7	8.02	9/10	5/8
xing4-intent-agentic.imatrix	the importance matrix used for STRIX_LEAN	88 MB	—	—	—	—	—	—	—	—	—

Speed is tokens/s from llama-bench -fa 0 (the faster setting for this model on this box), 3 repetitions, ± below 0.2 %. PPL "long-context" = our long mixed-text corpus scored as 3 windows of 4096 tokens; wikitext-2 = wiki.test.raw, 128 chunks at -c 512. Intent = 10 cases in the vendor SDK's own prompt format, scored automatically on the expected category and argument values. Tool calls = 8 OpenAI-style cases with tool_choice required and auto (same result both ways); every miss is a prompt answered in text instead of a tool call (a greeting, a request for facts, and for STRIX_LEAN also a two-tool request).

Which one

• COHERENT-PAS — take this for intent and extraction work. Best quality: wikitext PPL −4.2 % vs COHERENT (about three times the error bar) and long-context −3.1 % (inside its ±0.27, same direction; both are paired runs on identical text), and the fastest prefill. Intent traffic is prompt-heavy (the schema goes in every request) with short JSON answers, so prefill and prompt caching matter more than decode. It decodes ~15 % slower than COHERENT because its attention, hyper-connection and shared-expert weights are Q8_0.
• COHERENT — the general-purpose pick: faster decode for long generations at slightly lower quality.
• STRIX_LEAN-imatrix — the smallest file and the fastest decode. On wikitext it ties COHERENT (9.983 vs 9.973, inside the error bars) but it is 3.5 % worse on the long-context corpus, misses one intent case (it drops the play_music half of "Set an alarm for 6am and play some jazz.") and one tool case. The imatrix is real: the same tier without it scores 8.2080 / 10.0947, so the imatrix recovers about 40 % of the long-context gap.

Usage

llama-server -m Xing4.0-29B-Intent-Q4_0-ROCmFP4-COHERENT-PAS.gguf \
  -ngl 999 -fa off -fit off --jinja -c 16384 --parallel 1

• KV memory: this arch stores decompressed K/V: the server saved a 400-token prompt state of 320.3 MiB, i.e. 0.80 MiB per token (16K context ≈ 13 GiB, 32K ≈ 26 GiB) on top of the weights. Size -c to your traffic; Intent prompts with a large schema are 1K–20K tokens.
• --jinja is required for tool calling. -fit off with an explicit -c keeps auto-fit from silently shrinking the context on unified memory.

The intent prompt

Use the vendor SDK's format (system prompt, the task template, the schema as json.dumps(..., ensure_ascii=False), the user input last), temperature 0, and turn thinking off — the Intent model thinks by default:

{"messages": [{"role": "system", "content": "<the SDK's default system prompt>"},
              {"role": "user", "content": "<task template with the schema and the input>"}],
 "temperature": 0, "chat_template_kwargs": {"enable_thinking": false}}

Answer shape: [{"category": "...", "args": [{"name": "value"}]}].

Prompt caching: on by default, and it matters here — but not with MTP

Every request with the same schema shares everything up to the user input. llama-server's prompt cache (on by default) reuses that prefix. Measured with an 835-token Intent prompt and five requests sharing one schema (COHERENT-PAS):

request	prompt tokens	reused from cache	prompt processing
first (191 tokens reused from the warm-up request's system prompt)	835	191	1002 ms
same schema, new input	832	822	101 ms
exact repeat of the first	835	822	115 ms — answer byte-identical to the cold run

So a warm schema costs about 0.1 s instead of 1.0 s of prefill per request (all three files behave the same).

⚠️ With --spec-type draft-mtp the prompt cache is never reused on this build. Every request logs prompt cache cold fallback: reason=spec-boundary-mismatch and processes the full prompt again. Do not enable MTP for intent/extraction serving.

MTP: present, accurate, but no speed-up on this build

Layer 40 is the MTP block and is included in every file. With --spec-type draft-mtp --spec-draft-n-max 1 the draft is almost always right (acceptance 0.96–0.99, mean accepted length 1.96–1.99), yet generation speed on our code-writing prompts did not improve: 23.8 vs 24.4 tok/s (COHERENT), 21.7 vs 21.9 (PAS), 23.6 vs 24.8 (STRIX_LEAN-imatrix), MTP vs off, at -c 4096. Combined with the prompt-cache loss above, leave MTP off for these files on the current build.

Verification and reproduction

box      AMD Ryzen AI Max+ 395 "Strix Halo", gfx1151, ROCm 7.2.4, 128 GiB unified; other models resident but idle
build    github.com/kingjones30/ROCmFPX, branch xing4-port (xing4 sources identical to cbc2cd5)
source   XingChen-AGI/Xing4.0-29B-Intent, revision 070c34b10a7cb8dab87c9f602e38cd67826ab48f (bf16 safetensors)
convert  the fork's convert_hf_to_gguf.py -> BF16 GGUF (62.45 GB)
quant    COHERENT        llama-quantize <bf16> <out> Q4_0_ROCMFP4_COHERENT     (ftype 102, no imatrix)
         COHERENT-PAS    COHERENT with Q8_0 for attn_q_a, attn_q_b, attn_kv_a_mqa, attn_kv_b, attn_output, hc_attn_fn,
                         hc_ffn_fn, ffn_{gate,up,down}_shexp and the dense FFN of blocks 0-1   (no imatrix)
         STRIX_LEAN      llama-imatrix on a Q8_0 copy over an agentic calibration text (-c 512, 575 chunks, 552 entries),
                         then llama-quantize --imatrix xing4-intent-agentic.imatrix <bf16> <out> Q4_0_ROCMFP4_STRIX_LEAN 16
ppl      llama-perplexity -m <f> -f wiki.test.raw -c 512 --chunks 128 -ngl 999 -fa off -fit off
         llama-perplexity -m <f> -f <long corpus> -c 4096 -ub 64 -ngl 999 -fa off -fit off
speed    llama-bench -m <f> -p 512 -n 128 -d 0,4096 -fa 0 -ngl 999 -r 3
serving  llama-server ... -fa off -fit off --jinja -c 4096 --parallel 1   (intent, tool, cache and MTP legs)

⚠️ llama-perplexity scores only the second half of each window. Match the scoring window before comparing these numbers with another tool.

Credits

Base model © China Telecom Artificial Intelligence Technology Co., Ltd. (XingChen-AGI/Xing4.0-29B-Intent), Apache-2.0. Quantized, measured and arch-ported by kingjones777; these GGUFs inherit Apache-2.0. Base-model ROCmFP4 builds: Xing4.0-29B-A4B-ROCmFP4-GGUF.