Nanbeige_Nanbeige4.3-33B (Q4_K_M) for AMD Loom
A Loom package (.loom programs and a model.json card) for the unmodified GGUF checkpoint bartowski/Nanbeige_Nanbeige4.3-33B-GGUF (Nanbeige_Nanbeige4.3-33B-Q4_K_M.gguf), for the AMD Ryzen AI Max+ 395 "Strix Halo" and AMD Radeon AI PRO R9700 GPUs, generated with the loom-skills workflow. Learn more on the Loom ecosystem website.
Quick start
pipx install "hrx-loom[hf]==0.1.0"
Fetch and serve on the AMD Ryzen AI Max+ 395 "Strix Halo" (gfx1151):
loom-fetch https://huggingface.co/loom-fun/bartowski-Nanbeige4.3-33B-loom/resolve/main/model.json --preset=chat --materialize=copy --output=Nanbeige4.3-33B
loom-serve --model=Nanbeige4.3-33B/model.json '--device=amdgpu://?completion_spin_ns=4294967295' --memory_bytes=81000000000
Fetch and serve on the AMD Radeon AI PRO R9700 (gfx1201, 32 GB):
loom-fetch https://huggingface.co/loom-fun/bartowski-Nanbeige4.3-33B-loom/resolve/main/model.json --preset=chat-r9700 --materialize=copy --output=Nanbeige4.3-33B
loom-serve --model=Nanbeige4.3-33B/model.json '--device=amdgpu://?completion_spin_ns=4294967295' --pool_backing=fixed
The server listens on 127.0.0.1:8080 and answers OpenAI-style /v1/chat/completions requests.
On the AMD Radeon AI PRO R9700, the package's preset sizes the memory pool (--pool_backing=fixed commits it up front): this package serves up to 37,888 tokens per request there (its native context is 524,288); the package files are the same on every GPU.
On the AMD Radeon AI PRO R9700, the Quick start fetches the chat-r9700 preset of model.json: the default preset with context_capacity 37,888, pool_capacity 37,888 for its 32 GB. The programs, weights and tokenizer are the same.
Performance vs llama.cpp
Strix Halo (gfx1151)
| metric | llama.cpp | Loom | margin | roofline | Loom % of roofline | llama.cpp Vulkan |
|---|---|---|---|---|---|---|
| decode tok/s, short prompt | 10.6 | 11.2 | +6 % | 10.7 | 105 % β‘ | 10.7 |
| decode tok/s, 2k context | 9.7 | 10.7 | +10 % | 10.5 | 102 % β‘ | 10.4 |
| decode tok/s, 32k context | 2.4 | 8.1 | +238 % | 8.0 | 101 % β‘ | 7.8 |
| prefill tok/s, 1k document | 322.6 | 420.3 | +30 % | 948.6 | 44 % | 260.4 |
| prefill tok/s, 2k document | 323.0 | 418.5 | +30 % | 937.4 | 45 % | 258.7 |
| prefill tok/s, 8k document | 287.6 | 397.5 | +38 % | 875.2 | 45 % | 227.8 |
| prefill tok/s, 16k document | 244.7 | 345.2 | +41 % | 804.1 | 43 % | 197.7 |
| prefill tok/s, 32k document | 186.2 | 271.1 | +46 % | 691.6 | 39 % | 154.0 |
| 4 streams, aggregate decode tok/s | 34.1 | 42.6 | +25 % | 41.9 | 102 % β‘ | 38.9 |
| speculative decoding, drafted tokens accepted | n/a | n/a | n/a | n/a | n/a | n/a |
Methodology
- Compared: Loom against llama.cpp ROCm
llamacpp-rocm/b1374, llama.cpp Vulkanllamacpp-vulkan/b12211; the same GGUF, the same DevLab runner and job, the same prompts; Loom served with the active wait. Values are the mean of 2β3 rounds (4 streams: median of 3); Loom is timed by the client, llama.cpp by its server. - Out of the box on both engines: Loom serves the preset shipped in this package; llama.cpp runs its build defaults plus only the server arguments listed below (offload, context and slots). No tuning sweep on either engine.
- Drafting, 1-stream rows (decode, decode_2k, decode_32k, prefill): no drafting on either engine (no MTP head in the GGUF and no drafter published for this exact model). Each engine is loaded once for all these rows.
- 4 streams: both engines loaded with drafting off, 4 parallel requests, aggregate tokens per second.
- Rows: decode = short-context decode; decode_2k / decode_32k = decode after a ~2k / ~32k-token context; prefill_Nk = prompt processing of an N k-token prompt; streams4 = 4-stream aggregate decode. A margin within +-3 % (the measured same-runner noise) is a tie.
- llama.cpp ROCm server arguments:
1-stream rows: -ngl 99 -cram 0 -c 65536 -np 1; 4 streams: -ngl 99 -cram 0 -c 65536 -kvu -np 4. - llama.cpp Vulkan server arguments:
1-stream rows: -ngl 99 -cram 0 -c 36864 -np 1; 4 streams: -ngl 99 -cram 0 -c 147456 -np 4. - Measured on AMD DevLab Dispatch Strix Halo runners (
devlab-perfjobs); runners differ between sessions. - Measured 2026-11-13; device AMD Strix Halo (DevLab Dispatch runner X2-LA03-S03-D06), Linux; hrx-loom
0.1.0(hrx-systemd01d1c1, recovered); source: DevLab Dispatch run 43376548212. - Strix Halo performance measured on programs
60ec93a; the default path of this package (sha2562a64e2b) is identical after canonicalization (fleet/stage_equiv.py at d2be0384d7 against the measured revision 61ee2b0ad3; storage, prepare and control differ only inside the logits blocks; checked 2026-11-15). - Strix Halo re-measured on programs
60ec93aon 2026-11-15: within the per-row bands (methodology, 2-4 %) of the published values except prefill_32k (faster, confirmed in 2 runs; new values shown), and prefill_16k +2.1 % (not repeated in the same direction by the first run) (DevLab Dispatch run 43670695792, DevLab Dispatch run 43670695792). - prefill_32k: the card value is the mean of the two re-measure runs' medians (each the median of 3 sessions; prefill_32k 274.1 and 268.2); the published value was prefill_32k 258.4; llama.cpp values are as published (not re-measured).
- llama.cpp Vulkan: measured 2026-11-13; source: DevLab Dispatch run 43376548212. Archive sha256
9759acd. - β‘ measured above the modeled ceiling; the roofline is an estimate (216 GB/s effective bandwidth).
- Roofline: the hardware ceiling for this GGUF on AMD Ryzen AI Max+ 395 "Strix Halo" (gfx1151) at 216 GB/s and 59 TFLOP/s, from loom-roofline
roofline.py(loom-skillsacd9ac7). - β not measured; n/a not applicable.
Radeon AI PRO R9700 (gfx1201, 32 GB)
| metric | llama.cpp | Loom | margin | roofline | Loom % of roofline | llama.cpp Vulkan |
|---|---|---|---|---|---|---|
| decode tok/s, short prompt | 27.0 | 29.4 | +9 % | 30.5 | 96 % | 28.7 |
| decode tok/s, 2k context | 25.9 | 27.8 | +7 % | 29.8 | 93 % | 28.0 |
| decode tok/s, 32k context | 17.3 | 19.9 | +15 % | 22.8 | 87 % | 21.3 |
| prefill tok/s, 1k document | 1098.1 | 1446.4 | +32 % | 3440.7 | 42 % | 744.0 |
| prefill tok/s, 2k document | 1090.7 | 1428.9 | +31 % | 3400.0 | 42 % | 739.4 |
| prefill tok/s, 8k document | 990.5 | 1379.7 | +39 % | 3174.4 | 43 % | 692.3 |
| prefill tok/s, 16k document | 857.4 | 1212.1 | +41 % | 2916.4 | 42 % | 610.4 |
| prefill tok/s, 32k document | 665.9 | 946.1 | +42 % | 2508.6 | 38 % | 493.4 |
| 4 streams, aggregate decode tok/s | 77.3 | 109.2 | +41 % | 118.9 | 92 % | β |
| speculative decoding, drafted tokens accepted | n/a | n/a | n/a | n/a | n/a | n/a |
Methodology
- Compared: Loom against llama.cpp ROCm
llamacpp-rocm/b1375, llama.cpp Vulkanllamacpp-vulkan/b12211; the same GGUF, the same DevLab runner and job, the same prompts; Loom served with the active wait. Values are the median of 3 sessions (4 streams: median of 6); Loom is timed by the client, llama.cpp by its server. - Out of the box on both engines: Loom serves the preset shipped in this package; llama.cpp runs its build defaults plus only the server arguments listed below (offload, context and slots). No tuning sweep on either engine.
- Drafting, 1-stream rows (decode, decode_2k, decode_32k, prefill): no drafting on either engine (no MTP head in the GGUF and no drafter published for this exact model). Each engine is loaded once for all these rows.
- 4 streams: both engines loaded with drafting off, 4 parallel requests, aggregate tokens per second.
- Rows: decode = short-context decode; decode_2k / decode_32k = decode after a ~2k / ~32k-token context; prefill_Nk = prompt processing of an N k-token prompt; streams4 = 4-stream aggregate decode. A margin within +-3 % (the measured same-runner noise) is a tie.
- llama.cpp ROCm server arguments:
-ngl 99 -cram 0 -c 36864 -np 1. - llama.cpp Vulkan server arguments:
-ngl 99 -cram 0 -c 36864 -np 1. - Measured on AMD DevLab Dispatch R9700 runners (
devlab-perfjobs); runners differ between sessions. - Measured 2026-11-15; device DevLab Dispatch runner ephemeral-devlab-sm-01-02-r9700s-7-132600067371, AMD Radeon AI PRO R9700S (gfx1201), Linux; hrx-loom
0.1.0(hrx-system commit unknown, recovered); source: DevLab Dispatch run 43655173096. - Radeon AI PRO R9700 performance measured on programs
60ec93a; the default path of this package (sha2562a64e2b) is identical after canonicalization (fleet/stage_equiv.py at d2be0384d7 against the measured revision 61ee2b0ad3; storage, prepare and control differ only inside the logits blocks; checked 2026-11-15). - llama.cpp Vulkan: measured 2026-11-15; source: DevLab Dispatch run 43655173096. Archive sha256
9759acd. - Roofline: the hardware ceiling for this GGUF on AMD Radeon AI PRO R9700S (gfx1201, 32 GB, 2.35 GHz) at 613 GB/s and 214 TFLOP/s, from loom-roofline
roofline.py(loom-skills2ef98d2). - β not measured; n/a not applicable.
Features
| feature | status | detail |
|---|---|---|
| API | β | OpenAI-style /v1/chat/completions |
| tool calling | β | OpenAI-style tools and tool_calls |
| reasoning / thinking | β | on by default; chat_template_kwargs: {"enable_thinking": false} turns it off |
| reasoning effort | β | not applicable |
| speculative decoding | β | not applicable |
| image input | β | not applicable |
| audio input / output | β | not applicable |
| sampling | β | temperature, top_p, top_k, seed |
| context size | β | 242,688 tokens per request; Radeon AI PRO R9700 up to 37,888 (its memory) |
| concurrency | β | 4 concurrent requests |
| max output tokens | β | 4,096 (preset max_tokens) |
Correctness
| gate | description | Strix Halo | Radeon AI PRO R9700 |
|---|---|---|---|
kernel_tests |
Unit tests of the generated kernels against reference implementations | β pass (77/77) ΒΉ | β pass (77/77) Β² |
golden |
llama.cpp reference recorded on the same GGUF: greedy tokens and top-5 log-probs, short prompts and 1k-32k documents | β pass Β³ | β pass β΄ |
compare |
Greedy output vs the llama.cpp reference, token by token; every divergence is a near-tie | β pass Β³ | β pass β΄ |
vision |
Image input vs llama.cpp with the same projector: identical prompt tokens, replies equal up to near-ties | β not applicable | β not applicable |
consistency |
Concurrent requests give the same output as the same requests run alone | β pass Β³ | β pass β΄ |
spec |
Speculative decoding on gives exactly the output of speculative decoding off | β not applicable | β not applicable |
agent |
Multi-turn tool-calling loop: exact prompt rendering, history reuse, valid tool calls, output vs llama.cpp | β pass Β³ | β pass β΄ |
agent_think_off |
The tool-calling loop with thinking off | β pass Β³ | β pass β΄ |
sampling |
temperature, top_k and seed behave as specified; first-token distribution matches llama.cpp | β pass Β³ | β pass β΄ |
logits |
Host logits readback (llm.logits on): at T=0 every returned row's argmax is the device's token, the top-k matches llama.cpp, greedy text is unchanged | β pass (argmax = device token; top-5 = llama.cpp's) β΅ | β pass (argmax = device token; top-5 = llama.cpp's) βΆ |
decision |
Decision answers (choice, score, noul probabilities) vs llama.cpp on the same prompts | β not applicable | β not applicable |
- ΒΉ DevLab Dispatch run 43721362070, 2026-11-15; device AMD Strix Halo (DevLab Dispatch runner X2-LA03-S01-D04), Linux; hrx-loom
0.1.0(hrx-systemd01d1c1, recovered). - Β² DevLab Dispatch run 43721368522, 2026-11-15; device DevLab Dispatch runner ephemeral-devlab-sm-01-02-r9700s-7-132835931145, AMD Radeon AI PRO R9700S (gfx1201), Linux; hrx-loom
0.1.0(hrx-system commit unknown, recovered). - Β³ DevLab Dispatch run 43670695792, LEVEL=dev, 2026-11-15; device AMD Strix Halo (DevLab Dispatch runner X2-LA03-S03-D06), Linux; hrx-loom
0.1.0(hrx-systemd01d1c1, recovered). - β΄ DevLab Dispatch run 43655173096, LEVEL=dev, 2026-11-15; device DevLab Dispatch runner ephemeral-devlab-sm-01-02-r9700s-7-132600067371, AMD Radeon AI PRO R9700S (gfx1201), Linux; hrx-loom
0.1.0(hrx-system commit unknown, recovered). - β΅ DevLab Dispatch run 43721362070, LEVEL=dev, 2026-11-15; device AMD Strix Halo (DevLab Dispatch runner X2-LA03-S01-D04), Linux; hrx-loom
0.1.0(hrx-systemd01d1c1, recovered). - βΆ DevLab Dispatch run 43721368522, LEVEL=dev, 2026-11-15; device DevLab Dispatch runner ephemeral-devlab-sm-01-02-r9700s-7-132835931145, AMD Radeon AI PRO R9700S (gfx1201), Linux; hrx-loom
0.1.0(hrx-system commit unknown, recovered). - Gates are the loom-benchmark correctness gates (
gates.sh); a gate without a recorded run shows no evidence.
Files
model.json/model.json.lock: the card and the pinned source files (sha256 checked byloom-fetch).program/*.loom,weights.loom: the generated Loom programs.tokenizer.json: the tokenizer.llm_config.json: the configuration the programs were generated from.info.json: the measurements, gate evidence and versions this card shows; everything else comes from the files above.REPORT.md: how the values above were measured: setup, commands and the value of every run.
Versions: card schema loom-card/3; loom-skills d2be038; hrx-loom 0.1.0 (install fingerprint 4358bf6e19ba3594); card generator d2be038.
Source files, fetched by loom-fetch from their own repositories:
- checkpoint: bartowski/Nanbeige_Nanbeige4.3-33B-GGUF @
9beb684,Nanbeige_Nanbeige4.3-33B-Q4_K_M.gguf
Model tree for loom-fun/bartowski-Nanbeige4.3-33B-loom
Base model
Nanbeige/Nanbeige4.3-33B-Base