How to use from
llama.cpp
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf AtomicChat/SenseNova-Loop-9B-GGUF:
# Run inference directly in the terminal:
llama cli -hf AtomicChat/SenseNova-Loop-9B-GGUF:
Install from WinGet (Windows)
winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf AtomicChat/SenseNova-Loop-9B-GGUF:
# Run inference directly in the terminal:
llama cli -hf AtomicChat/SenseNova-Loop-9B-GGUF:
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-app
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama serve -hf AtomicChat/SenseNova-Loop-9B-GGUF:
# Run inference directly in the terminal:
./build/bin/llama cli -hf AtomicChat/SenseNova-Loop-9B-GGUF:
Use Docker
docker model run hf.co/AtomicChat/SenseNova-Loop-9B-GGUF:
Quick Links

How to Run SenseNova-Loop-9B Locally

Importance-matrix GGUF files of SenseNova-Loop-9B in the vendor's own format, with the MTP head and the vision projector.

Atomic Chat Discord GitHub X
  • Built by the team behind Atomic Chat, the app that runs open-weight models locally and offline.
  • Needs SenseNova's llama.cpp fork at 24ea7bb7, the commit SenseNova's own GGUF card names. Stock llama.cpp, LM Studio and Ollama do not load SenseNova-Loop yet (ggml-org/llama.cpp#31571).

SenseNova-Loop-9B is SenseNova's 9B looped multimodal model, Apache-2.0. It stores 32 decoder layers and runs 56: layers 4-27 execute twice on the same weights, with two-stream manifold-constrained hyper-connections (mHC) around the looped layers. It reads text and images and supports a 512K-token context. This repo has GGUF files of it for llama.cpp, quantized with an importance matrix, in the same sensenova_loop format as SenseNova's own GGUF, so they run on the same build.

Pick a file

AD (Atomic Dynamic) files quantize the feed-forward and DeltaNet projections with an importance matrix and keep the parts the model is sensitive to higher: attention K/V at Q8_0, the head at Q6_K or above, the edge layers one step up, the mHC routers in F32. File names are SenseNova-Loop-9B-<name>.gguf. Every file carries the MTP draft head.

file GB on disk bpw suggested hardware mean KLD 99th pct KLD same top token
AD-Q6_K 8.76 7.61 12 GB GPUs, 16 GB Macs 0.0092 0.055 96.8%
AD-Q5_K_M 6.86 5.95 10 GB GPUs 0.0231 0.192 94.1%
AD-Q4_K_M 5.96 5.17 8 GB GPUs (our pick) 0.0445 0.393 91.3%
AD-IQ4_XS 5.81 5.05 8 GB GPUs, the size of SenseNova's Q4_K_M 0.0535 0.490 90.5%
AD-IQ3_S 4.86 4.22 6 GB GPUs with a short context 0.1367 1.234 84.2%

The suggested hardware leaves room for about 16K tokens of context (text only; the vision projector adds 0.9 GB). The KV cache is small for a 9B: only 14 of the 56 layer passes are full attention, about 57 KB per token at F16.

"Same top token" is how often the file picks the same next token as the BF16 original; "mean KLD" is the KL divergence of its next-token distribution from BF16's. Both are measured with llama-perplexity on our 30-language neutral corpus (87 chunks of 4096 tokens) against SenseNova's BF16 GGUF, on SenseNova's llama.cpp fork at 24ea7bb7, 1x H100 80 GB. All numbers, logs and the importance matrix are in AtomicChat/SenseNova-Loop-9B-GGUF-metrics.

SenseNova-Loop loses more to quantization than most models of its size: even SenseNova's own Q8_0 picks the BF16 token 97.4% of the time, where most models stay above 98%. One likely reason is the loop itself: layers 4-27 run twice on the same rounded weights. Read the "same top token" column with that in mind; it ranks the files fairly, but sits lower than for other models. Our 4.35 GB layout (AD-IQ3_XXS) measured 79.1% and is not published.

How these compare to SenseNova's GGUF

SenseNova publishes BF16, Q8_0, Q5_K_M and Q4_K_M, quantized with llama.cpp's presets and no importance matrix. We measured them on the same stand as our files (same corpus, reference, build and GPU):

SenseNova's file GB mean KLD same top token our file GB mean KLD same top token
Q4_K_M 5.82 0.1014 86.6% AD-IQ4_XS 5.81 0.0535 (โˆ’47%) 90.5%
Q4_K_M 5.82 0.1014 86.6% AD-Q4_K_M 5.96 0.0445 (โˆ’56%) 91.3%
Q5_K_M 6.67 0.0376 92.1% AD-Q5_K_M 6.86 0.0231 (โˆ’39%) 94.1%
Q8_0 9.80 0.0064 97.4% AD-Q6_K 8.76 0.0092 (+44%) 96.8%

At the same 5.8 GB, AD-IQ4_XS halves the divergence of SenseNova's Q4_K_M and picks the original's token 3.9 points more often. SenseNova's Q8_0 stays the closest file to BF16; AD-Q6_K is 1.0 GB smaller and gives up 0.65 points of same top token for it. Our files use an importance matrix and a layout tuned for this model; SenseNova's use llama.cpp's presets without an importance matrix.

Run it

Build SenseNova's fork at 24ea7bb7. The commit lives on the sensenova-loop branch of blueswhen/llama.cpp, which OpenSenseNova/llama.cpp forks; a plain clone of OpenSenseNova/llama.cpp does not have it. Keep -DLLAMA_USE_PREBUILT_UI=OFF: at this commit the build fetches llama-server's web UI for a release number that does not exist, and stops (the server then runs without the web page; the API works).

git clone -b sensenova-loop https://github.com/blueswhen/llama.cpp llama.cpp-sensenova
cd llama.cpp-sensenova
git checkout 24ea7bb78410e0f0f7bbb362e37c60cd9ecfc761
cmake -B build -DGGML_CUDA=ON -DLLAMA_USE_PREBUILT_UI=OFF   # Mac: drop -DGGML_CUDA=ON (Metal is on by default)
cmake --build build -j --config Release --target llama-server llama-cli llama-mtmd-cli

When upstream llama.cpp adds SenseNova-Loop, its file layout may differ from this fork's; we will re-check these files then.

Then serve a file with the vision projector. The chat template (thinking, tools) is embedded, so pass --jinja:

./build/bin/llama-server -hf AtomicChat/SenseNova-Loop-9B-GGUF:AD-Q4_K_M --jinja -c 32768 -ngl 99

-hf also fetches mmproj-SenseNova-Loop-9B-F16.gguf from this repo; with a local file pass -m SenseNova-Loop-9B-AD-Q4_K_M.gguf --mmproj mmproj-SenseNova-Loop-9B-F16.gguf. Add --spec-type draft-mtp for speculative decoding with the built-in MTP head. Thinking is on by default; send "chat_template_kwargs": {"enable_thinking": false} to turn it off for a request.

Speed on SenseNova's fork, 1x H100 PCIe, llama-bench with flash attention: AD-Q4_K_M processes a 512-token prompt at about 1,150 tokens/s and generates 15 tokens/s; SenseNova's own Q4_K_M measures the same on this build. That is slow for a 9B model, for their files and ours alike; we have not profiled why. With --spec-type draft-mtp the MTP head drafted tokens that were accepted 60% of the time, but generation did not get faster on this build. Checked by hand in llama-server --jinja with AD-Q4_K_M: a reasoning answer, thinking off, a tool call, and an image through the projector (llama-mtmd-cli).

How we built them

  • Base: SenseNova's BF16 GGUF (SenseNova-Loop-9B-BF16.gguf in sensenova/SenseNova-Loop-9B-GGUF, revision 5889ddb1). We did not re-convert the safetensors: SenseNova's converter for this architecture exists only in their fork, and their BF16 file is the one their runtime is built against. The vision projector in this repo is SenseNova's mmproj F16, unchanged.
  • Importance matrix: computed on that BF16 over our public calibration build (4.96M tokens: agentic, code, reasoning, 30 languages, long context, a sweep of the vocabulary), tokenized with --parse-special. The build was rendered for a Qwen3.5 model with the same 248,320-token vocabulary and, byte for byte, the same chat template as SenseNova-Loop. Both passes of every looped layer feed the same matrix entry, since they share the weights.
  • Layout: profile dense-sensenova-loop in our pipeline. Feed-forward weights carry most of the bytes; the first two blocks and the last six (26-27 and the four after the loop) get them one step higher, and below Q6_K blocks 2-5 too. The fused DeltaNet input projection and the DeltaNet output sit one step above the full-attention Q projection (level on the 3-bit file), attention K/V and the small DeltaNet gates stay Q8_0, the head Q6_K or Q8_0 (Q5_K on the 3-bit file), the token embedding IQ4_XS or above, the MTP block Q5_K, the mHC routers F32. We tried two other band choices of the same size at AD-IQ4_XS: the blocks with the most activation energy in the importance matrix (all in the tail) measured 0.0563 (5% worse), the blocks outside the loop as the edge 0.0512 (4% better, and 0.1357 against 0.1367 at AD-IQ3_S). Neither is a clear win, so the files keep the default.

Model facts

parameters ~9B stored, ~14.6B per forward pass (SenseNova's figure)
layers 32 stored, 56 executed: 0-3, then 4-27 twice, then 28-31
attention 3 Gated DeltaNet layers per full-attention layer; 16 query heads, 4 KV heads, head dim 256
hidden / FFN 4096 / 12288
mHC 2 residual streams over the looped layers, Sinkhorn-normalized mixing (20 iterations)
MTP 1 draft layer (blk.56 in the GGUF)
context 512K (YaRN x2 over 262,144)
vocabulary 248,320
license Apache-2.0 (LICENSE in this repo, from the base model)

Quantized by AtomicChat.

Downloads last month
1,202
GGUF
Model size
9B params
Architecture
sensenova_loop
Hardware compatibility
Log In to add your hardware

3-bit

4-bit

5-bit

6-bit

Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for AtomicChat/SenseNova-Loop-9B-GGUF

Quantized
(42)
this model

Collection including AtomicChat/SenseNova-Loop-9B-GGUF