Spark-X2.5-4B (Q4_K_M) for AMD Loom

A Loom package (.loom programs and a model.json card) for the unmodified GGUF checkpoint XHToken/Spark-X2.5-4B-GGUF (Spark-X2.5-4B-Q4_K_M.gguf), for the AMD Ryzen AI Max+ 395 "Strix Halo", AMD Radeon AI PRO R9700 and AMD Radeon gfx1201 16 GB class (RX 9070 XT, RX 9070) GPUs, generated with the loom-skills workflow. Learn more on the Loom ecosystem website.

Quick start

Open your GPU and copy its commands.

AMD Ryzen AI Max+ 395 "Strix Halo" (gfx1151)
pipx install "hrx-loom[hf]==0.2.0"
loom-fetch huggingface:loom-fun/XHToken-Spark-X2.5-4B-loom:chat --materialize=copy --output=Spark-X2.5-4B
loom-serve --model=Spark-X2.5-4B '--device=amdgpu://?completion_spin_ns=4294967295' --pool_backing=fixed --parameter_mode=preload
  • --materialize=copy puts copies of the fetched package files in the output directory.
  • completion_spin_ns=4294967295 enables active waiting for GPU completions (uses CPU while GPU work is in flight); the card's performance measurements use this setting.
  • --pool_backing=fixed commits the preset's memory pool up front instead of growing elastic backing.
  • --parameter_mode=preload only changes how the weights are loaded on the host, so the performance rows were measured without it unless their job served with it.
AMD Radeon AI PRO R9700 (gfx1201, 32 GB)
pipx install "hrx-loom[hf]==0.2.0"
loom-fetch huggingface:loom-fun/XHToken-Spark-X2.5-4B-loom:chat-r9700 --materialize=copy --output=Spark-X2.5-4B
loom-serve --model=Spark-X2.5-4B '--device=amdgpu://?completion_spin_ns=4294967295' --pool_backing=fixed --parameter_mode=preload
  • --materialize=copy puts copies of the fetched package files in the output directory.
  • completion_spin_ns=4294967295 enables active waiting for GPU completions (uses CPU while GPU work is in flight); the card's performance measurements use this setting.
  • --pool_backing=fixed commits the preset's memory pool up front instead of growing elastic backing.
  • --parameter_mode=preload only changes how the weights are loaded on the host, so the performance rows were measured without it unless their job served with it.

On this GPU, the package's preset sizes the memory pool (--pool_backing=fixed commits it up front): this package serves up to 195,584 tokens per request (its native context is 1,048,576).

Radeon gfx1201, 16 GB (RX 9070 XT, RX 9070)
pipx install "hrx-loom[hf]==0.2.0"
loom-fetch huggingface:loom-fun/XHToken-Spark-X2.5-4B-loom:chat-16gb --materialize=copy --output=Spark-X2.5-4B
loom-serve --model=Spark-X2.5-4B '--device=amdgpu://?completion_spin_ns=4294967295' --memory_bytes=15000000000 --state_slab_bytes=4mib --weight_slab_bytes=64mib --parameter_mode=preload
  • --materialize=copy puts copies of the fetched package files in the output directory.
  • completion_spin_ns=4294967295 enables active waiting for GPU completions (uses CPU while GPU work is in flight); the card's performance measurements use this setting.
  • --memory_bytes sets the serving memory budget for this GPU.
  • --state_slab_bytes sets the allocation slab size for state memory.
  • --weight_slab_bytes sets the allocation slab size for weight memory.
  • --parameter_mode=preload only changes how the weights are loaded on the host, so the performance rows were measured without it unless their job served with it.

On this GPU, --memory_bytes=15000000000 bounds the memory pool: this package serves up to 75,776 tokens per request (its native context is 1,048,576).

Send a request

In another terminal (optional thinking is turned off):

curl http://127.0.0.1:8080/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{"model": "XHToken/Spark-X2.5-4B", "messages": [{"role": "user", "content": "Hello"}], "chat_template_kwargs": {"enable_thinking": false}}'

Performance vs llama.cpp

Strix Halo (gfx1151)

metric llama.cpp Loom margin vs best llama.cpp roofline Loom % of roofline llama.cpp Vulkan
decode tok/s, short prompt 61.1 72.2 +18 % 83.2 87 % 61.3
decode tok/s, 2k context 55.2 68.2 +17 % 79.3 86 % 58.1
decode tok/s, 32k context 41.5 50.0 +20 % 56.4 89 % 41.4
prefill tok/s, 1k document 1615.0 2829.8 +75 % 7518.2 38 % 1425.1
prefill tok/s, 2k document 1723.4 2843.8 +65 % 7443.3 38 % 1507.6
prefill tok/s, 8k document 1697.1 2817.9 +66 % 7049.8 40 % 1517.1
prefill tok/s, 16k document 1538.4 2548.7 +66 % 6585.7 39 % 1378.7
prefill tok/s, 32k document 1273.5 2173.9 +71 % 5819.4 37 % 1130.6
4 streams, aggregate decode tok/s 152.7 246.7 +53 % 296.9 83 % 161.5
speculative decoding, drafted tokens accepted n/a n/a n/a n/a n/a n/a

Methodology

  • Compared: Loom against llama.cpp ROCm llamacpp-rocm/b1374, llama.cpp Vulkan llamacpp-vulkan/b12211; the same GGUF, the same DevLab runner type, different jobs, the same prompts; Loom served with the active wait. Values are the median of 3 sessions (4 streams: median of 6); Loom is timed by the client, llama.cpp by its server.
  • llama.cpp measured on 2026-11-13 (DevLab Dispatch run 43390795404), Loom on 2026-11-17 (DevLab Dispatch run 44088653570): same runner type, different jobs (llama.cpp was not re-measured for this package).
  • Out of the box on both engines: Loom serves the preset shipped in this package; llama.cpp runs its build defaults plus only the server arguments listed below (offload, context and slots). No tuning sweep on either engine.
  • Drafting, 1-stream rows (decode, decode_2k, decode_32k, prefill): no drafting on either engine (no MTP head in the GGUF and no drafter published for this exact model). Each engine is loaded once for all these rows.
  • 4 streams: both engines loaded with drafting off, 4 parallel requests, aggregate tokens per second.
  • Rows: decode = short-context decode; decode_2k / decode_32k = decode after a ~2k / ~32k-token context; prefill_Nk = prompt processing of an N k-token prompt; streams4 = 4-stream aggregate decode. A margin within +-3 % (the measured same-runner noise) is a tie.
  • llama.cpp ROCm server arguments: 1-stream rows: -ngl 99 -cram 0 -c 65536 -np 1; 4 streams: -ngl 99 -cram 0 -c 65536 -kvu -np 4.
  • llama.cpp Vulkan server arguments: 1-stream rows: -ngl 99 -cram 0 -c 36864 -np 1; 4 streams: -ngl 99 -cram 0 -c 147456 -np 4.
  • Measured on AMD DevLab Dispatch Strix Halo runners (devlab-perf jobs); runners differ between sessions.
  • Margin vs best llama.cpp: (Loom / max(ROCm, Vulkan) - 1) Γ— 100 %, using the available measured tokens/s for each row on this GPU. With only one valid backend, that backend is used; with neither, the margin is not measured. Raw backend values and their measurement sources are retained below and in the table.
  • Measured 2026-11-17; device AMD Strix Halo (DevLab Dispatch runner X2-LA03-S01-D03), Linux; hrx-loom 0.2.0 (hrx-system 0cd370f, recovered); source: DevLab Dispatch run 44088653570.
  • llama.cpp Vulkan: measured 2026-11-13; source: DevLab Dispatch run 43390795404. Archive sha256 9759acd.
  • Roofline: the hardware ceiling for this GGUF on AMD Ryzen AI Max+ 395 "Strix Halo" (gfx1151) at 216 GB/s and 59 TFLOP/s, from loom-roofline roofline.py (loom-skills 8fef91d).
  • β€” not measured; n/a not applicable.

Radeon AI PRO R9700 (gfx1201, 32 GB)

metric llama.cpp Loom margin vs best llama.cpp roofline Loom % of roofline llama.cpp Vulkan
decode tok/s, short prompt 131.0 165.8 +27 % 236.2 70 % 127.6
decode tok/s, 2k context 123.3 154.7 +17 % 224.9 69 % 132.4
decode tok/s, 32k context 102.0 116.6 +12 % 160.0 73 % 104.3
prefill tok/s, 1k document 4960.2 9165.6 +85 % 27269.5 34 % 4146.0
prefill tok/s, 2k document 5578.3 9386.1 +68 % 26997.6 35 % 4577.5
prefill tok/s, 8k document 5847.4 9309.7 +59 % 25570.5 36 % 4620.1
prefill tok/s, 16k document 5465.9 8512.4 +56 % 23887.0 36 % 4323.9
prefill tok/s, 32k document 4727.4 7152.8 +51 % 21107.7 34 % 3702.5
4 streams, aggregate decode tok/s 295.3 572.5 +94 % 842.7 68 % 250.1
speculative decoding, drafted tokens accepted n/a n/a n/a n/a n/a n/a

Methodology

  • Compared: Loom against llama.cpp ROCm llamacpp-rocm/b1375, llama.cpp Vulkan llamacpp-vulkan/b12211; the same GGUF, the same DevLab runner type, different jobs, the same prompts; Loom served with the active wait. Values are the median of 3 sessions (4 streams: median of 6); Loom is timed by the client, llama.cpp by its server.
  • llama.cpp measured on 2026-11-15 (DevLab Dispatch run 43633464433), Loom on 2026-11-17 (DevLab Dispatch run 43993211976): same runner type, different jobs (llama.cpp was not re-measured for this package).
  • Out of the box on both engines: Loom serves the preset shipped in this package; llama.cpp runs its build defaults plus only the server arguments listed below (offload, context and slots). No tuning sweep on either engine.
  • Drafting, 1-stream rows (decode, decode_2k, decode_32k, prefill): no drafting on either engine (no MTP head in the GGUF and no drafter published for this exact model). Each engine is loaded once for all these rows.
  • 4 streams: both engines loaded with drafting off, 4 parallel requests, aggregate tokens per second.
  • Rows: decode = short-context decode; decode_2k / decode_32k = decode after a ~2k / ~32k-token context; prefill_Nk = prompt processing of an N k-token prompt; streams4 = 4-stream aggregate decode. A margin within +-3 % (the measured same-runner noise) is a tie.
  • llama.cpp ROCm server arguments: -ngl 99 -cram 0 -c 65536 -np 1.
  • llama.cpp Vulkan server arguments: -ngl 99 -cram 0 -c 36864 -np 1.
  • Measured on AMD DevLab Dispatch R9700 runners (devlab-perf jobs); runners differ between sessions.
  • Margin vs best llama.cpp: (Loom / max(ROCm, Vulkan) - 1) Γ— 100 %, using the available measured tokens/s for each row on this GPU. With only one valid backend, that backend is used; with neither, the margin is not measured. Raw backend values and their measurement sources are retained below and in the table.
  • Measured 2026-11-17; device DevLab Dispatch runner ephemeral-devlab-sm-01-02-r9700s-7-133738799762, AMD Radeon AI PRO R9700S (gfx1201), Linux; hrx-loom 0.2.0 (hrx-system commit unknown, recovered); source: DevLab Dispatch run 43993211976.
  • llama.cpp Vulkan: measured 2026-11-15; source: DevLab Dispatch run 43633464433. Archive sha256 9759acd.
  • Roofline: the hardware ceiling for this GGUF on AMD Radeon AI PRO R9700S (gfx1201, 32 GB, 2.35 GHz) at 613 GB/s and 214 TFLOP/s, from loom-roofline roofline.py (loom-skills 203cf33).
  • β€” not measured; n/a not applicable.

Radeon RX 9070 XT

metric llama.cpp Loom margin vs best llama.cpp roofline Loom % of roofline llama.cpp Vulkan
decode tok/s, short prompt 130.4 164.3 +17 % 246.6 67 % 140.5
decode tok/s, 2k context 125.4 153.1 +12 % 234.8 65 % 136.2
decode tok/s, 32k context 100.2 115.2 +8 % 167.1 69 % 106.7
prefill tok/s, 1k document 5019.6 8678.5 +73 % 24797.4 35 % 4724.4
prefill tok/s, 2k document 5657.7 8919.4 +58 % 24550.1 36 % 5391.6
prefill tok/s, 8k document 5882.7 8829.0 +50 % 23252.4 38 % 5777.8
prefill tok/s, 16k document 5485.8 7892.8 +44 % 21721.6 36 % 5381.2
prefill tok/s, 32k document 4758.9 6578.5 +38 % 19194.2 34 % 4591.3
4 streams, aggregate decode tok/s 310.5 569.4 +83 % 879.8 65 % 253.5
speculative decoding, drafted tokens accepted n/a n/a n/a n/a n/a n/a

Methodology

  • Compared: Loom against llama.cpp ROCm llamacpp-rocm/b1375, llama.cpp Vulkan llamacpp-vulkan/b12211; the same GGUF, the same machine and session, the same prompts; Loom served with the active wait. Values are the median of 3 sessions (4 streams: median of 3); Loom is timed by the client, llama.cpp by its server.
  • Out of the box on both engines: Loom serves the preset shipped in this package; llama.cpp runs its build defaults plus only the server arguments listed below (offload, context and slots). No tuning sweep on either engine.
  • Drafting, 1-stream rows (decode, decode_2k, decode_32k, prefill): no drafting on either engine (no MTP head in the GGUF and no drafter published for this exact model). Each engine is loaded once for all these rows.
  • 4 streams: both engines loaded with drafting off, 4 parallel requests, aggregate tokens per second.
  • Rows: decode = short-context decode; decode_2k / decode_32k = decode after a ~2k / ~32k-token context; prefill_Nk = prompt processing of an N k-token prompt; streams4 = 4-stream aggregate decode. A margin within +-3 % (the measured same-runner noise) is a tie.
  • llama.cpp ROCm server arguments: -ngl 99 -cram 0 -c 65536 -np 1.
  • llama.cpp Vulkan server arguments: -ngl 99 -cram 0 -c 36864 -np 1.
  • Measured on AMD Ryzen 7 9800X3D 8-Core Processor, AMD Radeon RX 9070 XT, Ubuntu 26.04, a named machine (this GPU has no DevLab runner); both engines ran on it.
  • Margin vs best llama.cpp: (Loom / max(ROCm, Vulkan) - 1) Γ— 100 %, using the available measured tokens/s for each row on this GPU. With only one valid backend, that backend is used; with neither, the margin is not measured. Raw backend values and their measurement sources are retained below and in the table.
  • Measured 2026-11-17; device AMD Ryzen 7 9800X3D 8-Core Processor, AMD Radeon RX 9070 XT, Ubuntu 26.04; hrx-loom 0.2.0 (hrx-system 0cd370f); source: named-machine run.
  • llama.cpp Vulkan: measured 2026-11-17; source: named-machine run. Archive sha256 9759acd.
  • Roofline: the hardware ceiling for this GGUF on AMD Radeon RX 9070 XT (gfx1201, 16 GB) at 640 GB/s and 195 TFLOP/s, from loom-roofline roofline.py (loom-skills 963ead9).
  • β€” not measured; n/a not applicable.

Features

feature status detail
API βœ… OpenAI-style /v1/chat/completions
tool calling βœ… OpenAI-style tools and tool_calls
reasoning / thinking βœ… on by default; chat_template_kwargs: {"enable_thinking": false} turns it off
reasoning effort β€” not applicable
speculative decoding β€” not applicable
image input β€” not applicable
audio input / output β€” not applicable
sampling βœ… temperature, top_p, top_k, seed
context size βœ… 262,144 tokens per request; Radeon AI PRO R9700 up to 195,584 (its memory); Radeon gfx1201 16 GB up to 75,776 (its memory)
concurrency βœ… 4 concurrent requests
max output tokens βœ… 4,096 (preset max_tokens)

Correctness

gate description Strix Halo Radeon AI PRO R9700 Radeon RX 9070 XT
kernel_tests Unit tests of the generated kernels against reference implementations βœ… pass (65/65) ΒΉ βœ… pass (65/65) Β² βœ… pass (65/65) ⁡
golden llama.cpp reference recorded on the same GGUF: greedy tokens and top-5 log-probs, short prompts and 1k-32k documents βœ… pass Β³ βœ… pass ⁴ βœ… pass ⁢
compare Greedy output vs the llama.cpp reference, token by token; every divergence is a near-tie βœ… pass Β³ βœ… pass ⁴ βœ… pass ⁢
vision Image input vs llama.cpp with the same projector: identical prompt tokens, replies equal up to near-ties β€” not applicable β€” not applicable β€” not applicable
consistency Concurrent requests give the same output as the same requests run alone βœ… pass Β³ βœ… pass ⁴ βœ… pass ⁢
spec Speculative decoding on gives exactly the output of speculative decoding off β€” not applicable β€” not applicable β€” not applicable
agent Multi-turn tool-calling loop: exact prompt rendering, history reuse, valid tool calls, output vs llama.cpp βœ… pass Β³ βœ… pass ⁴ βœ… pass ⁢
agent_think_off The tool-calling loop with thinking off βœ… pass Β³ βœ… pass ⁴ βœ… pass ⁢
sampling temperature, top_k and seed behave as specified; first-token distribution matches llama.cpp βœ… pass Β³ βœ… pass ⁴ βœ… pass ⁢
logits Host logits readback (llm.logits on): at T=0 every returned row's argmax is the device's token, the top-k matches llama.cpp, greedy text is unchanged ? not re-run on this package (the last pass was on a previous build, sha256 52aa103; the nightly gates re-run it) ? not re-run on this package (the last pass was on a previous build, sha256 52aa103; the nightly gates re-run it) ❌ no evidence
decision Decision answers (choice, score, noul probabilities) vs llama.cpp on the same prompts β€” not applicable β€” not applicable β€” not applicable
  • ΒΉ DevLab Dispatch run 44088653570, 2026-11-17; device AMD Strix Halo (DevLab Dispatch runner X2-LA03-S01-D03), Linux; hrx-loom 0.2.0 (hrx-system 0cd370f, recovered).
  • Β² DevLab Dispatch run 43993211976, 2026-11-17; device DevLab Dispatch runner ephemeral-devlab-sm-01-02-r9700s-7-133738799762, AMD Radeon AI PRO R9700S (gfx1201), Linux; hrx-loom 0.2.0 (hrx-system commit unknown, recovered).
  • Β³ DevLab Dispatch run 44088653570, LEVEL=dev, 2026-11-17; device AMD Strix Halo (DevLab Dispatch runner X2-LA03-S01-D03), Linux; hrx-loom 0.2.0 (hrx-system 0cd370f, recovered).
  • ⁴ DevLab Dispatch run 43993211976, LEVEL=dev, 2026-11-17; device DevLab Dispatch runner ephemeral-devlab-sm-01-02-r9700s-7-133738799762, AMD Radeon AI PRO R9700S (gfx1201), Linux; hrx-loom 0.2.0 (hrx-system commit unknown, recovered).
  • ⁡ named-machine run, 2026-11-17; device AMD Ryzen 7 9800X3D 8-Core Processor, AMD Radeon RX 9070 XT, Ubuntu 26.04; hrx-loom 0.2.0 (hrx-system 0cd370f).
  • ⁢ named-machine run, LEVEL=dev, 2026-11-17; device AMD Ryzen 7 9800X3D 8-Core Processor, AMD Radeon RX 9070 XT, Ubuntu 26.04; hrx-loom 0.2.0 (hrx-system 0cd370f).
  • Gates are the loom-benchmark correctness gates (gates.sh); a gate without a recorded run shows no evidence.

Files

  • model.json / model.json.lock: the card and the pinned source files (sha256 checked by loom-fetch).
  • program/*.loom, weights.loom: the generated Loom programs.
  • tokenizer.json: the tokenizer.
  • llm_config.json: the configuration the programs were generated from.
  • info.json: the measurements, gate evidence and versions this card shows; everything else comes from the files above.
  • REPORT.md: how the values above were measured: setup, commands and the value of every run.

Versions: card schema loom-card/3; loom-skills unknown; hrx-loom unknown; card generator 926f475.

Source files, fetched by loom-fetch from their own repositories:

Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
This model isn't deployed by any Inference Provider. πŸ™‹ Ask for provider support

Model tree for loom-fun/XHToken-Spark-X2.5-4B-loom

Finetuned
(3)
this model