Instructions to use AtomicChat/SenseNova-Loop-9B-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use AtomicChat/SenseNova-Loop-9B-GGUF with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf AtomicChat/SenseNova-Loop-9B-GGUF:Q4_K_M # Run inference directly in the terminal: llama cli -hf AtomicChat/SenseNova-Loop-9B-GGUF:Q4_K_M
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf AtomicChat/SenseNova-Loop-9B-GGUF:Q4_K_M # Run inference directly in the terminal: llama cli -hf AtomicChat/SenseNova-Loop-9B-GGUF:Q4_K_M
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf AtomicChat/SenseNova-Loop-9B-GGUF:Q4_K_M # Run inference directly in the terminal: ./build/bin/llama cli -hf AtomicChat/SenseNova-Loop-9B-GGUF:Q4_K_M
Use Docker
docker model run hf.co/AtomicChat/SenseNova-Loop-9B-GGUF:Q4_K_M
- Llama new
- LM Studio
- Jan
- vLLM
How to use AtomicChat/SenseNova-Loop-9B-GGUF with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "AtomicChat/SenseNova-Loop-9B-GGUF" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "AtomicChat/SenseNova-Loop-9B-GGUF", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker
docker model run hf.co/AtomicChat/SenseNova-Loop-9B-GGUF:Q4_K_M
- Ollama
How to use AtomicChat/SenseNova-Loop-9B-GGUF with Ollama:
ollama run hf.co/AtomicChat/SenseNova-Loop-9B-GGUF:Q4_K_M
- Unsloth Desktop
- Pi
How to use AtomicChat/SenseNova-Loop-9B-GGUF with Pi:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf AtomicChat/SenseNova-Loop-9B-GGUF:Q4_K_M
Configure the model in Pi
# Install Pi: npm install -g @earendil-works/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "llama-cpp": { "baseUrl": "http://localhost:9931/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "AtomicChat/SenseNova-Loop-9B-GGUF:Q4_K_M" } ] } } }Run Pi
# Start Pi in your project directory: pi
- Docker Model Runner
How to use AtomicChat/SenseNova-Loop-9B-GGUF with Docker Model Runner:
docker model run hf.co/AtomicChat/SenseNova-Loop-9B-GGUF:Q4_K_M
- Lemonade
How to use AtomicChat/SenseNova-Loop-9B-GGUF with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull AtomicChat/SenseNova-Loop-9B-GGUF:Q4_K_M
Run and chat with the model
lemonade run user.SenseNova-Loop-9B-GGUF-Q4_K_M
List all available models
lemonade list
- Hermes Agent
How to use AtomicChat/SenseNova-Loop-9B-GGUF with Hermes Agent:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf AtomicChat/SenseNova-Loop-9B-GGUF:Q4_K_M
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:9931/v1 hermes config set model.default AtomicChat/SenseNova-Loop-9B-GGUF:Q4_K_M
Run Hermes
hermes
- Atomic Chat
- OpenClaw
How to use AtomicChat/SenseNova-Loop-9B-GGUF with OpenClaw:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf AtomicChat/SenseNova-Loop-9B-GGUF:Q4_K_M
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:9931/v1 \ --custom-model-id "AtomicChat/SenseNova-Loop-9B-GGUF:Q4_K_M" \ --custom-provider-id llama-cpp \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
How to Run SenseNova-Loop-9B Locally
Importance-matrix GGUF files of SenseNova-Loop-9B in the vendor's own format, with the MTP head and the vision projector.
- Built by the team behind Atomic Chat, the app that runs open-weight models locally and offline.
- Needs SenseNova's llama.cpp fork at
24ea7bb7, the commit SenseNova's own GGUF card names. Stock llama.cpp, LM Studio and Ollama do not load SenseNova-Loop yet (ggml-org/llama.cpp#31571).
SenseNova-Loop-9B is SenseNova's 9B looped multimodal model, Apache-2.0. It stores 32 decoder layers and runs 56: layers 4-27 execute twice on the same weights, with two-stream manifold-constrained hyper-connections (mHC) around the looped layers. It reads text and images and supports a 512K-token context. This repo has GGUF files of it for llama.cpp, quantized with an importance matrix, in the same sensenova_loop format as SenseNova's own GGUF, so they run on the same build.
Pick a file
AD (Atomic Dynamic) files quantize the feed-forward and DeltaNet projections with an importance matrix and keep the parts the model is sensitive to higher: attention K/V at Q8_0, the head at Q6_K or above, the edge layers one step up, the mHC routers in F32. File names are SenseNova-Loop-9B-<name>.gguf. Every file carries the MTP draft head.
| file | GB on disk | bpw | suggested hardware | mean KLD | 99th pct KLD | same top token |
|---|---|---|---|---|---|---|
AD-Q6_K |
8.76 | 7.61 | 12 GB GPUs, 16 GB Macs | 0.0092 | 0.055 | 96.8% |
AD-Q5_K_M |
6.86 | 5.95 | 10 GB GPUs | 0.0231 | 0.192 | 94.1% |
AD-Q4_K_M |
5.96 | 5.17 | 8 GB GPUs (our pick) | 0.0445 | 0.393 | 91.3% |
AD-IQ4_XS |
5.81 | 5.05 | 8 GB GPUs, the size of SenseNova's Q4_K_M | 0.0535 | 0.490 | 90.5% |
AD-IQ3_S |
4.86 | 4.22 | 6 GB GPUs with a short context | 0.1367 | 1.234 | 84.2% |
The suggested hardware leaves room for about 16K tokens of context (text only; the vision projector adds 0.9 GB). The KV cache is small for a 9B: only 14 of the 56 layer passes are full attention, about 57 KB per token at F16.
"Same top token" is how often the file picks the same next token as the BF16 original; "mean KLD" is the KL divergence of its next-token distribution from BF16's. Both are measured with llama-perplexity on our 30-language neutral corpus (87 chunks of 4096 tokens) against SenseNova's BF16 GGUF, on SenseNova's llama.cpp fork at 24ea7bb7, 1x H100 80 GB. All numbers, logs and the importance matrix are in AtomicChat/SenseNova-Loop-9B-GGUF-metrics.
SenseNova-Loop loses more to quantization than most models of its size: even SenseNova's own Q8_0 picks the BF16 token 97.4% of the time, where most models stay above 98%. One likely reason is the loop itself: layers 4-27 run twice on the same rounded weights. Read the "same top token" column with that in mind; it ranks the files fairly, but sits lower than for other models. Our 4.35 GB layout (AD-IQ3_XXS) measured 79.1% and is not published.
How these compare to SenseNova's GGUF
SenseNova publishes BF16, Q8_0, Q5_K_M and Q4_K_M, quantized with llama.cpp's presets and no importance matrix. We measured them on the same stand as our files (same corpus, reference, build and GPU):
| SenseNova's file | GB | mean KLD | same top token | our file | GB | mean KLD | same top token |
|---|---|---|---|---|---|---|---|
Q4_K_M |
5.82 | 0.1014 | 86.6% | AD-IQ4_XS |
5.81 | 0.0535 (โ47%) | 90.5% |
Q4_K_M |
5.82 | 0.1014 | 86.6% | AD-Q4_K_M |
5.96 | 0.0445 (โ56%) | 91.3% |
Q5_K_M |
6.67 | 0.0376 | 92.1% | AD-Q5_K_M |
6.86 | 0.0231 (โ39%) | 94.1% |
Q8_0 |
9.80 | 0.0064 | 97.4% | AD-Q6_K |
8.76 | 0.0092 (+44%) | 96.8% |
At the same 5.8 GB, AD-IQ4_XS halves the divergence of SenseNova's Q4_K_M and picks the original's token 3.9 points more often. SenseNova's Q8_0 stays the closest file to BF16; AD-Q6_K is 1.0 GB smaller and gives up 0.65 points of same top token for it. Our files use an importance matrix and a layout tuned for this model; SenseNova's use llama.cpp's presets without an importance matrix.
Run it
Build SenseNova's fork at 24ea7bb7. The commit lives on the sensenova-loop branch of blueswhen/llama.cpp, which OpenSenseNova/llama.cpp forks; a plain clone of OpenSenseNova/llama.cpp does not have it. Keep -DLLAMA_USE_PREBUILT_UI=OFF: at this commit the build fetches llama-server's web UI for a release number that does not exist, and stops (the server then runs without the web page; the API works).
git clone -b sensenova-loop https://github.com/blueswhen/llama.cpp llama.cpp-sensenova
cd llama.cpp-sensenova
git checkout 24ea7bb78410e0f0f7bbb362e37c60cd9ecfc761
cmake -B build -DGGML_CUDA=ON -DLLAMA_USE_PREBUILT_UI=OFF # Mac: drop -DGGML_CUDA=ON (Metal is on by default)
cmake --build build -j --config Release --target llama-server llama-cli llama-mtmd-cli
When upstream llama.cpp adds SenseNova-Loop, its file layout may differ from this fork's; we will re-check these files then.
Then serve a file with the vision projector. The chat template (thinking, tools) is embedded, so pass --jinja:
./build/bin/llama-server -hf AtomicChat/SenseNova-Loop-9B-GGUF:AD-Q4_K_M --jinja -c 32768 -ngl 99
-hf also fetches mmproj-SenseNova-Loop-9B-F16.gguf from this repo; with a local file pass -m SenseNova-Loop-9B-AD-Q4_K_M.gguf --mmproj mmproj-SenseNova-Loop-9B-F16.gguf. Add --spec-type draft-mtp for speculative decoding with the built-in MTP head. Thinking is on by default; send "chat_template_kwargs": {"enable_thinking": false} to turn it off for a request.
Speed on SenseNova's fork, 1x H100 PCIe, llama-bench with flash attention: AD-Q4_K_M processes a 512-token prompt at about 1,150 tokens/s and generates 15 tokens/s; SenseNova's own Q4_K_M measures the same on this build. That is slow for a 9B model, for their files and ours alike; we have not profiled why. With --spec-type draft-mtp the MTP head drafted tokens that were accepted 60% of the time, but generation did not get faster on this build. Checked by hand in llama-server --jinja with AD-Q4_K_M: a reasoning answer, thinking off, a tool call, and an image through the projector (llama-mtmd-cli).
How we built them
- Base: SenseNova's BF16 GGUF (
SenseNova-Loop-9B-BF16.ggufin sensenova/SenseNova-Loop-9B-GGUF, revision5889ddb1). We did not re-convert the safetensors: SenseNova's converter for this architecture exists only in their fork, and their BF16 file is the one their runtime is built against. The vision projector in this repo is SenseNova'smmprojF16, unchanged. - Importance matrix: computed on that BF16 over our public calibration build (4.96M tokens: agentic, code, reasoning, 30 languages, long context, a sweep of the vocabulary), tokenized with
--parse-special. The build was rendered for a Qwen3.5 model with the same 248,320-token vocabulary and, byte for byte, the same chat template as SenseNova-Loop. Both passes of every looped layer feed the same matrix entry, since they share the weights. - Layout: profile
dense-sensenova-loopin our pipeline. Feed-forward weights carry most of the bytes; the first two blocks and the last six (26-27 and the four after the loop) get them one step higher, and below Q6_K blocks 2-5 too. The fused DeltaNet input projection and the DeltaNet output sit one step above the full-attention Q projection (level on the 3-bit file), attention K/V and the small DeltaNet gates stay Q8_0, the head Q6_K or Q8_0 (Q5_K on the 3-bit file), the token embedding IQ4_XS or above, the MTP block Q5_K, the mHC routers F32. We tried two other band choices of the same size atAD-IQ4_XS: the blocks with the most activation energy in the importance matrix (all in the tail) measured 0.0563 (5% worse), the blocks outside the loop as the edge 0.0512 (4% better, and 0.1357 against 0.1367 atAD-IQ3_S). Neither is a clear win, so the files keep the default.
Model facts
| parameters | ~9B stored, ~14.6B per forward pass (SenseNova's figure) |
| layers | 32 stored, 56 executed: 0-3, then 4-27 twice, then 28-31 |
| attention | 3 Gated DeltaNet layers per full-attention layer; 16 query heads, 4 KV heads, head dim 256 |
| hidden / FFN | 4096 / 12288 |
| mHC | 2 residual streams over the looped layers, Sinkhorn-normalized mixing (20 iterations) |
| MTP | 1 draft layer (blk.56 in the GGUF) |
| context | 512K (YaRN x2 over 262,144) |
| vocabulary | 248,320 |
| license | Apache-2.0 (LICENSE in this repo, from the base model) |
Quantized by AtomicChat.
- Downloads last month
- 1,202
3-bit
4-bit
5-bit
6-bit
Model tree for AtomicChat/SenseNova-Loop-9B-GGUF
Base model
sensenova/SenseNova-Loop-9B


