Sage-A1.5-Q4_K_M (Q4_K_M) for AMD Loom
A Loom package (.loom programs and a model.json card) for the unmodified GGUF checkpoint InternScience/Sage-A1.5-Q4_K_M-GGUF (Sage-A1.5-Q4_K_M.gguf), for the AMD Ryzen AI Max+ 395 "Strix Halo" and AMD Radeon AI PRO R9700 GPUs, generated with the loom-skills workflow. Learn more on the Loom ecosystem website.
Quick start
Open your GPU and copy its commands.
AMD Ryzen AI Max+ 395 "Strix Halo" (gfx1151)
pipx install "hrx-loom[hf]==0.2.0"
loom-fetch huggingface:loom-fun-staging/InternScience-Sage-A1.5-loom:chat --materialize=copy --output=Sage-A1.5
loom-serve --model=Sage-A1.5 '--device=amdgpu://?completion_spin_ns=4294967295' --pool_backing=fixed --parameter_mode=preload
--materialize=copyputs copies of the fetched package files in the output directory.completion_spin_ns=4294967295enables active waiting for GPU completions (uses CPU while GPU work is in flight); the card's performance measurements use this setting.--pool_backing=fixedcommits the preset's memory pool up front instead of growing elastic backing.--parameter_mode=preloadonly changes how the weights are loaded on the host, so the performance rows were measured without it unless their job served with it.
In another terminal, send a request (optional thinking is turned off):
curl http://127.0.0.1:8080/v1/chat/completions \
-H 'Content-Type: application/json' \
-d '{"model": "InternScience/Sage-A1.5", "messages": [{"role": "user", "content": "Hello"}], "chat_template_kwargs": {"enable_thinking": false}}'
AMD Radeon AI PRO R9700 (gfx1201, 32 GB)
pipx install "hrx-loom[hf]==0.2.0"
loom-fetch huggingface:loom-fun-staging/InternScience-Sage-A1.5-loom:chat-r9700 --materialize=copy --output=Sage-A1.5
loom-serve --model=Sage-A1.5 '--device=amdgpu://?completion_spin_ns=4294967295' --pool_backing=fixed --parameter_mode=preload
--materialize=copyputs copies of the fetched package files in the output directory.completion_spin_ns=4294967295enables active waiting for GPU completions (uses CPU while GPU work is in flight); the card's performance measurements use this setting.--pool_backing=fixedcommits the preset's memory pool up front instead of growing elastic backing.--parameter_mode=preloadonly changes how the weights are loaded on the host, so the performance rows were measured without it unless their job served with it.
On this GPU, the package's preset sizes the memory pool (--pool_backing=fixed commits it up front): this package serves up to 138,240 tokens per request (its native context is 262,144).
In another terminal, send a request (optional thinking is turned off):
curl http://127.0.0.1:8080/v1/chat/completions \
-H 'Content-Type: application/json' \
-d '{"model": "InternScience/Sage-A1.5", "messages": [{"role": "user", "content": "Hello"}], "chat_template_kwargs": {"enable_thinking": false}}'
The server listens on 127.0.0.1:8080 and answers OpenAI-style /v1/chat/completions requests.
Performance vs llama.cpp
Strix Halo (gfx1151)
| metric | llama.cpp | Loom | margin vs best llama.cpp | roofline | Loom % of roofline | llama.cpp Vulkan |
|---|---|---|---|---|---|---|
| decode tok/s, short prompt | 71.2 | 87.9 | +23 % | 146.8 | 60 % | 70.3 |
| decode tok/s, 2k context | 69.5 | 79.7 | +15 % | 138.6 | 58 % | 63.1 |
| decode tok/s, 32k context | 76.3 | 70.3 | -8 % | 150.9 | 47 % | 65.1 |
| prefill tok/s, 1k document | 1200.6 | 2209.4 | +84 % | 11600.2 | 19 % | 902.6 |
| prefill tok/s, 2k document | 1284.3 | 2472.6 | +93 % | 11416.3 | 22 % | 1000.8 |
| prefill tok/s, 8k document | 1291.5 | 2680.9 | +108 % | 10424.8 | 26 % | 1013.2 |
| prefill tok/s, 16k document | 1175.5 | 2423.4 | +106 % | 9343.0 | 26 % | 929.1 |
| prefill tok/s, 32k document | 993.6 | 2102.6 | +112 % | 7737.1 | 27 % | 793.5 |
| 4 streams, aggregate decode tok/s | 140.4 | 172.2 | +23 % | 418.3 | 41 % | 132.6 |
| speculative decoding, drafted tokens accepted | β | 46 % | β | n/a | n/a | n/a |
Methodology
- Compared: Loom against llama.cpp ROCm
llamacpp-rocm/b1374, llama.cpp Vulkanllamacpp-vulkan/b12211; the same GGUF, the same DevLab runner type, different jobs, the same prompts; Loom served with the active wait. Values are the median of 3 sessions (4 streams: median of 6); Loom is timed by the client, llama.cpp by its server. - llama.cpp measured on 2026-11-13 (DevLab Dispatch run 43376548212), Loom on 2026-11-17 (DevLab Dispatch run 44088653570): same runner type, different jobs (llama.cpp was not re-measured for this package).
- Out of the box on both engines: Loom serves the preset shipped in this package; llama.cpp runs its build defaults plus only the server arguments listed below (offload, context and slots, and on the 1-stream rows the drafting flags that match Loom's drafting). No tuning sweep on either engine.
- Drafting, 1-stream rows (decode, decode_2k, decode_32k, prefill): MTP on both engines; draft depth: Loom 3, llama.cpp --spec-draft-n-max llama.cpp default (each engine its own). Each engine is loaded once for all these rows.
- 4 streams: both engines loaded with drafting off, 4 parallel requests, aggregate tokens per second.
- Rows: decode = short-context decode; decode_2k / decode_32k = decode after a ~2k / ~32k-token context; prefill_Nk = prompt processing of an N k-token prompt; streams4 = 4-stream aggregate decode. A margin within +-3 % (the measured same-runner noise) is a tie.
- llama.cpp ROCm server arguments:
1-stream rows: -ngl 99 -cram 0 --spec-type draft-mtp -c 65536 -np 1; 4 streams: -ngl 99 -cram 0 -c 65536 -kvu -np 4. - llama.cpp Vulkan server arguments:
1-stream rows: -ngl 99 -cram 0 --spec-type draft-mtp -c 36864 -np 1; 4 streams: -ngl 99 -cram 0 -c 147456 -np 4. - Measured on AMD DevLab Dispatch Strix Halo runners (
devlab-perfjobs); runners differ between sessions. - Margin vs best llama.cpp: (Loom / max(ROCm, Vulkan) - 1) Γ 100 %, using the available measured tokens/s for each row on this GPU. With only one valid backend, that backend is used; with neither, the margin is not measured. Raw backend values and their measurement sources are retained below and in the table.
- Measured 2026-11-17; device AMD Strix Halo (DevLab Dispatch runner X2-LA03-S01-D04), Linux; hrx-loom
0.2.0(hrx-system0cd370f, recovered); source: DevLab Dispatch run 44088653570. - llama.cpp Vulkan: measured 2026-11-13; source: DevLab Dispatch run 43376548212. Archive sha256
9759acd. - Drafted tokens accepted: the share of the drafter's proposed tokens the model accepted during the
specgate's requests. - Roofline: the hardware ceiling for this GGUF on AMD Ryzen AI Max+ 395 "Strix Halo" (gfx1151) at 216 GB/s and 59 TFLOP/s, from loom-roofline
roofline.py(loom-skills6d3ca93); decode rows with speculative decoding: verify passes Γ (1 + acceptance Γ depth 3) tokens per pass, the drafter's cost not counted, with the acceptance measured in the row's own session: decode 52 %, decode at 2k 48 %, decode at 32k 70 %; 4 streams run without drafting, so its ceiling is single-pass; the 4-stream ceiling of this mixture-of-experts model assumes the four streams route to the same experts (the bound a run may not exceed). - β not measured; n/a not applicable.
Radeon AI PRO R9700 (gfx1201, 32 GB)
| metric | llama.cpp | Loom | margin vs best llama.cpp | roofline | Loom % of roofline | llama.cpp Vulkan |
|---|---|---|---|---|---|---|
| decode tok/s, short prompt | 122.6 | 189.8 | +24 % | 399.7 | 47 % | 152.6 |
| decode tok/s, 2k context | 121.3 | 170.4 | +21 % | 375.2 | 45 % | 141.2 |
| decode tok/s, 32k context | 143.8 | 167.1 | -2 % | 428.1 | 39 % | 170.3 |
| prefill tok/s, 1k document | 3152.8 | 6392.0 | +103 % | 42075.3 | 15 % | 2610.9 |
| prefill tok/s, 2k document | 3569.0 | 6664.8 | +87 % | 41408.4 | 16 % | 2896.1 |
| prefill tok/s, 8k document | 3763.7 | 7282.2 | +93 % | 37812.1 | 19 % | 2996.7 |
| prefill tok/s, 16k document | 3520.6 | 6648.5 | +89 % | 33888.0 | 20 % | 2792.7 |
| prefill tok/s, 32k document | 3131.4 | 5780.7 | +85 % | 28063.2 | 21 % | 2476.8 |
| 4 streams, aggregate decode tok/s | 275.8 | 386.0 | +26 % | 1187.0 | 33 % | 306.0 |
| speculative decoding, drafted tokens accepted | β | 46 % | β | n/a | n/a | n/a |
Methodology
- Compared: Loom against llama.cpp ROCm
llamacpp-rocm/b1375, llama.cpp Vulkanllamacpp-vulkan/b12211; the same GGUF, the same DevLab runner type, different jobs, the same prompts; Loom served with the active wait. Values are the median of 3 sessions (4 streams: median of 6); Loom is timed by the client, llama.cpp by its server. - llama.cpp measured on 2026-11-15 (DevLab Dispatch run 43652305001), Loom on 2026-11-17 (DevLab Dispatch run 43993211976): same runner type, different jobs (llama.cpp was not re-measured for this package).
- Out of the box on both engines: Loom serves the preset shipped in this package; llama.cpp runs its build defaults plus only the server arguments listed below (offload, context and slots, and on the 1-stream rows the drafting flags that match Loom's drafting). No tuning sweep on either engine.
- Drafting, 1-stream rows (decode, decode_2k, decode_32k, prefill): MTP on both engines; draft depth: Loom 3, llama.cpp --spec-draft-n-max llama.cpp default (each engine its own). Each engine is loaded once for all these rows.
- 4 streams: both engines loaded with drafting off, 4 parallel requests, aggregate tokens per second.
- Rows: decode = short-context decode; decode_2k / decode_32k = decode after a ~2k / ~32k-token context; prefill_Nk = prompt processing of an N k-token prompt; streams4 = 4-stream aggregate decode. A margin within +-3 % (the measured same-runner noise) is a tie.
- llama.cpp ROCm server arguments:
-ngl 99 -cram 0 --spec-type draft-mtp -c 65536 -np 1. - llama.cpp Vulkan server arguments:
-ngl 99 -cram 0 --spec-type draft-mtp -c 36864 -np 1. - Measured on AMD DevLab Dispatch R9700 runners (
devlab-perfjobs); runners differ between sessions. - Margin vs best llama.cpp: (Loom / max(ROCm, Vulkan) - 1) Γ 100 %, using the available measured tokens/s for each row on this GPU. With only one valid backend, that backend is used; with neither, the margin is not measured. Raw backend values and their measurement sources are retained below and in the table.
- Measured 2026-11-17; device DevLab Dispatch runner ephemeral-devlab-sm-01-02-r9700s-3-133738799762, AMD Radeon AI PRO R9700S (gfx1201), Linux; hrx-loom
0.2.0(hrx-system commit unknown, recovered); source: DevLab Dispatch run 43993211976. - llama.cpp Vulkan: measured 2026-11-15; source: DevLab Dispatch run 43652305001. Archive sha256
9759acd. - Drafted tokens accepted: the share of the drafter's proposed tokens the model accepted during the
specgate's requests. - Roofline: the hardware ceiling for this GGUF on AMD Radeon AI PRO R9700S (gfx1201, 32 GB, 2.35 GHz) at 613 GB/s and 214 TFLOP/s, from loom-roofline
roofline.py(loom-skills203cf33); decode rows with speculative decoding: verify passes Γ (1 + acceptance Γ depth 3) tokens per pass, the drafter's cost not counted, with the acceptance measured in the row's own session: decode 49 %, decode at 2k 45 %, decode at 32k 70 %; 4 streams run without drafting, so its ceiling is single-pass; the 4-stream ceiling of this mixture-of-experts model assumes the four streams route to the same experts (the bound a run may not exceed). - β not measured; n/a not applicable.
Features
| feature | status | detail |
|---|---|---|
| API | β | OpenAI-style /v1/chat/completions |
| tool calling | β | OpenAI-style tools and tool_calls |
| reasoning / thinking | β | on by default; chat_template_kwargs: {"enable_thinking": false} turns it off |
| reasoning effort | β | not applicable |
| speculative decoding | β | MTP, depth 3, on by default |
| image input | β | OpenAI-style image_url parts; projector Sage-A1.5-mmproj.gguf |
| audio input / output | β | not applicable |
| sampling | β | temperature, top_p, top_k, seed |
| context size | β | 262,144 tokens per request; Radeon AI PRO R9700 up to 138,240 (its memory) |
| concurrency | β | 4 concurrent requests |
| max output tokens | β | 4,096 (preset max_tokens) |
Correctness
| gate | description | Strix Halo | Radeon AI PRO R9700 |
|---|---|---|---|
kernel_tests |
Unit tests of the generated kernels against reference implementations | β pass (64/64) ΒΉ | β pass (64/64) Β² |
golden |
llama.cpp reference recorded on the same GGUF: greedy tokens and top-5 log-probs, short prompts and 1k-32k documents | β pass Β³ | β pass β΄ |
compare |
Greedy output vs the llama.cpp reference, token by token; every divergence is a near-tie | β pass Β³ | β pass β΄ |
vision |
Image input vs llama.cpp with the same projector: identical prompt tokens, replies equal up to near-ties | β pass Β³ | β pass β΄ |
consistency |
Concurrent requests give the same output as the same requests run alone | β pass Β³ | β pass β΄ |
spec |
Speculative decoding on gives exactly the output of speculative decoding off | β pass Β³ | β pass β΄ |
agent |
Multi-turn tool-calling loop: exact prompt rendering, history reuse, valid tool calls, output vs llama.cpp | β pass Β³ | β pass β΄ |
agent_think_off |
The tool-calling loop with thinking off | β pass Β³ | β pass β΄ |
sampling |
temperature, top_k and seed behave as specified; first-token distribution matches llama.cpp | β pass Β³ | β pass β΄ |
logits |
Host logits readback (llm.logits on): at T=0 every returned row's argmax is the device's token, the top-k matches llama.cpp, greedy text is unchanged | ? not re-run on this package (the last pass was on a previous build, sha256 c54b814; the nightly gates re-run it) |
? not re-run on this package (the last pass was on a previous build, sha256 c54b814; the nightly gates re-run it) |
decision |
Decision answers (choice, score, noul probabilities) vs llama.cpp on the same prompts | β not applicable | β not applicable |
- ΒΉ DevLab Dispatch run 44088653570, 2026-11-17; device AMD Strix Halo (DevLab Dispatch runner X2-LA03-S01-D04), Linux; hrx-loom
0.2.0(hrx-system0cd370f, recovered). - Β² DevLab Dispatch run 43993211976, 2026-11-17; device DevLab Dispatch runner ephemeral-devlab-sm-01-02-r9700s-3-133738799762, AMD Radeon AI PRO R9700S (gfx1201), Linux; hrx-loom
0.2.0(hrx-system commit unknown, recovered). - Β³ DevLab Dispatch run 44088653570, LEVEL=dev, 2026-11-17; device AMD Strix Halo (DevLab Dispatch runner X2-LA03-S01-D04), Linux; hrx-loom
0.2.0(hrx-system0cd370f, recovered). - β΄ DevLab Dispatch run 43993211976, LEVEL=dev, 2026-11-17; device DevLab Dispatch runner ephemeral-devlab-sm-01-02-r9700s-3-133738799762, AMD Radeon AI PRO R9700S (gfx1201), Linux; hrx-loom
0.2.0(hrx-system commit unknown, recovered). - Gates are the loom-benchmark correctness gates (
gates.sh); a gate without a recorded run shows no evidence.
Files
model.json/model.json.lock: the card and the pinned source files (sha256 checked byloom-fetch).program/*.loom,weights.loom: the generated Loom programs.tokenizer.json: the tokenizer.llm_config.json: the configuration the programs were generated from.info.json: the measurements, gate evidence and versions this card shows; everything else comes from the files above.REPORT.md: how the values above were measured: setup, commands and the value of every run.
Versions: card schema loom-card/3; loom-skills unknown; hrx-loom unknown; card generator 926f475.
Source files, fetched by loom-fetch from their own repositories:
- checkpoint: InternScience/Sage-A1.5-Q4_K_M-GGUF @
10b179b,Sage-A1.5-Q4_K_M.gguf - projector: InternScience/Sage-A1.5-Q4_K_M-GGUF @
10b179b,Sage-A1.5-mmproj.gguf
Model tree for loom-fun-staging/InternScience-Sage-A1.5-loom
Base model
InternScience/Sage-A1.5