Image-Text-to-Text
GGUF
llama.cpp
sensenova
looped-transformer
imatrix
quantized
conversational
atomic-chat
Instructions to use AtomicChat/SenseNova-Loop-9B-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use AtomicChat/SenseNova-Loop-9B-GGUF with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf AtomicChat/SenseNova-Loop-9B-GGUF:Q4_K_M # Run inference directly in the terminal: llama cli -hf AtomicChat/SenseNova-Loop-9B-GGUF:Q4_K_M
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf AtomicChat/SenseNova-Loop-9B-GGUF:Q4_K_M # Run inference directly in the terminal: llama cli -hf AtomicChat/SenseNova-Loop-9B-GGUF:Q4_K_M
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf AtomicChat/SenseNova-Loop-9B-GGUF:Q4_K_M # Run inference directly in the terminal: ./build/bin/llama cli -hf AtomicChat/SenseNova-Loop-9B-GGUF:Q4_K_M
Use Docker
docker model run hf.co/AtomicChat/SenseNova-Loop-9B-GGUF:Q4_K_M
- Llama new
- LM Studio
- Jan
- vLLM
How to use AtomicChat/SenseNova-Loop-9B-GGUF with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "AtomicChat/SenseNova-Loop-9B-GGUF" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "AtomicChat/SenseNova-Loop-9B-GGUF", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker
docker model run hf.co/AtomicChat/SenseNova-Loop-9B-GGUF:Q4_K_M
- Ollama
How to use AtomicChat/SenseNova-Loop-9B-GGUF with Ollama:
ollama run hf.co/AtomicChat/SenseNova-Loop-9B-GGUF:Q4_K_M
- Unsloth Desktop
- Pi
How to use AtomicChat/SenseNova-Loop-9B-GGUF with Pi:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf AtomicChat/SenseNova-Loop-9B-GGUF:Q4_K_M
Configure the model in Pi
# Install Pi: npm install -g @earendil-works/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "llama-cpp": { "baseUrl": "http://localhost:9931/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "AtomicChat/SenseNova-Loop-9B-GGUF:Q4_K_M" } ] } } }Run Pi
# Start Pi in your project directory: pi
- Docker Model Runner
How to use AtomicChat/SenseNova-Loop-9B-GGUF with Docker Model Runner:
docker model run hf.co/AtomicChat/SenseNova-Loop-9B-GGUF:Q4_K_M
- Lemonade
How to use AtomicChat/SenseNova-Loop-9B-GGUF with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull AtomicChat/SenseNova-Loop-9B-GGUF:Q4_K_M
Run and chat with the model
lemonade run user.SenseNova-Loop-9B-GGUF-Q4_K_M
List all available models
lemonade list
- Hermes Agent
How to use AtomicChat/SenseNova-Loop-9B-GGUF with Hermes Agent:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf AtomicChat/SenseNova-Loop-9B-GGUF:Q4_K_M
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:9931/v1 hermes config set model.default AtomicChat/SenseNova-Loop-9B-GGUF:Q4_K_M
Run Hermes
hermes
- Atomic Chat
- OpenClaw
How to use AtomicChat/SenseNova-Loop-9B-GGUF with OpenClaw:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf AtomicChat/SenseNova-Loop-9B-GGUF:Q4_K_M
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:9931/v1 \ --custom-model-id "AtomicChat/SenseNova-Loop-9B-GGUF:Q4_K_M" \ --custom-provider-id llama-cpp \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
model card from results.json
Browse files
README.md
ADDED
|
@@ -0,0 +1,215 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
base_model: sensenova/SenseNova-Loop-9B
|
| 3 |
+
base_model_relation: quantized
|
| 4 |
+
quantized_by: AtomicChat
|
| 5 |
+
pipeline_tag: image-text-to-text
|
| 6 |
+
library_name: gguf
|
| 7 |
+
license: apache-2.0
|
| 8 |
+
tags:
|
| 9 |
+
- gguf
|
| 10 |
+
- llama.cpp
|
| 11 |
+
- sensenova
|
| 12 |
+
- imatrix
|
| 13 |
+
- quantized
|
| 14 |
+
- conversational
|
| 15 |
+
- atomic-chat
|
| 16 |
+
---
|
| 17 |
+
|
| 18 |
+
# How to Run SenseNova Loop 9B Locally
|
| 19 |
+
<p style="margin-top: 0; margin-bottom: 0;">
|
| 20 |
+
<em>Built from sensenova's original weights with our own importance matrix. The <a href="https://huggingface.co/datasets/AtomicChat/calib-corpora">calibration corpora</a> behind our builds are public.</em>
|
| 21 |
+
</p>
|
| 22 |
+
<div style="display: flex; gap: 8px; align-items: center; margin-top: 10px; margin-bottom: 10px;">
|
| 23 |
+
<a href="https://atomic.chat/?utm_source=huggingface&utm_medium=referral&utm_campaign=hf_sensenova_loop_9b&utm_content=btn_atomic"><img src="https://huggingface.co/AtomicChat/Qwen3.8-27B-GGUF/resolve/main/btn_atomic.png" width="162" alt="Atomic Chat"></a>
|
| 24 |
+
<a href="https://discord.gg/8wGSsvmg4V"><img src="https://huggingface.co/AtomicChat/Qwen3.8-27B-GGUF/resolve/main/btn_discord.png" width="119" alt="Discord"></a>
|
| 25 |
+
<a href="https://github.com/AtomicBot-ai/Atomic-Chat"><img src="https://huggingface.co/AtomicChat/Qwen3.8-27B-GGUF/resolve/main/btn_github.png" width="115" alt="GitHub"></a>
|
| 26 |
+
<a href="https://x.com/atomic_chat_hq"><img src="https://huggingface.co/AtomicChat/Qwen3.8-27B-GGUF/resolve/main/btn_x.png" width="108" alt="X"></a>
|
| 27 |
+
</div>
|
| 28 |
+
<ul style="margin: 0 0 12px 0;">
|
| 29 |
+
<!-- TODO(editorial): guide bullet: link to the atomic.chat blog guide for this model, if there is one -->
|
| 30 |
+
<li>Built by the team behind <a href="https://atomic.chat/?utm_source=huggingface&utm_medium=referral&utm_campaign=hf_sensenova_loop_9b&utm_content=bullet_app">Atomic Chat</a>, the app that runs open-weight models locally and offline.</li>
|
| 31 |
+
<li>See our quantization analysis below for measurements and instructions.</li>
|
| 32 |
+
</ul>
|
| 33 |
+
<hr style="margin: 0 0 16px 0;">
|
| 34 |
+
|
| 35 |
+
<!-- TODO(editorial): the KLD-vs-size chart (ours and other publishers) and the sentence under it -->
|
| 36 |
+
|
| 37 |
+
## Pick a file
|
| 38 |
+
|
| 39 |
+
Every number below is measured, not estimated. How we measured it is at the
|
| 40 |
+
bottom, and the raw logs are in
|
| 41 |
+
[the metrics repo](https://huggingface.co/datasets/AtomicChat/SenseNova-Loop-9B-GGUF-metrics)
|
| 42 |
+
so you can check any of it yourself.
|
| 43 |
+
|
| 44 |
+
`KL divergence` is how far the quantized model's predictions drift from the
|
| 45 |
+
original weights. Lower is better, and zero means identical. `top-1` is how
|
| 46 |
+
often it picks the same next word the original would have picked.
|
| 47 |
+
|
| 48 |
+
| File | Size | KL divergence | top-1 |
|
| 49 |
+
|---|---:|---:|---:|
|
| 50 |
+
| `AD-Q6_K` | 8.8 GB | 0.00920 | 96.79% |
|
| 51 |
+
| `AD-Q5_K_M` | 6.9 GB | 0.02308 | 94.13% |
|
| 52 |
+
| `AD-Q4_K_M` | 6.0 GB | 0.04451 | 91.33% |
|
| 53 |
+
| `AD-IQ4_XS` | 5.8 GB | 0.05350 | 90.55% |
|
| 54 |
+
| `AD-IQ3_S` | 4.9 GB | 0.13673 | 84.17% |
|
| 55 |
+
| `AD-IQ3_XXS` | 4.3 GB | 0.23782 | 79.13% |
|
| 56 |
+
|
| 57 |
+
`AD-` marks an **Atomic Dynamic** layout. The name says what the two largest
|
| 58 |
+
tensor groups got: `AD-<ffn_down>-<ffn_up>`, collapsed to one name when both
|
| 59 |
+
match. Nothing is named after a type it does not contain.
|
| 60 |
+
|
| 61 |
+
## Which one fits your card
|
| 62 |
+
|
| 63 |
+
The file has to fit, and so does the context. This model keeps 32 KB of
|
| 64 |
+
attention cache per token, which is 0.2 GB at 8k context and 1.0 GB at 32k. Budget
|
| 65 |
+
for both.
|
| 66 |
+
|
| 67 |
+
| Your card | File | Leaves room for |
|
| 68 |
+
|---|---|---|
|
| 69 |
+
| 8 GB | `AD-Q5_K_M` | around 16k context |
|
| 70 |
+
| 12 GB | `AD-Q6_K` | around 88k context |
|
| 71 |
+
| 32 GB and up | `AD-Q6_K` | full context |
|
| 72 |
+
|
| 73 |
+
> [!TIP]
|
| 74 |
+
> If you are choosing between two neighbouring files, take the larger one. The
|
| 75 |
+
> steps between them cost one or two gigabytes and buy noticeably fewer wrong words.
|
| 76 |
+
|
| 77 |
+
## Running it
|
| 78 |
+
|
| 79 |
+
```bash
|
| 80 |
+
llama-server -m SenseNova-Loop-9B-AD-Q4_K_M.gguf -ngl 99 -c 8192
|
| 81 |
+
```
|
| 82 |
+
|
| 83 |
+
The chat template is inside every file, llama.cpp applies it on its own.
|
| 84 |
+
<!-- TODO(editorial): prompt format block and the thinking toggle, if the family has one -->
|
| 85 |
+
|
| 86 |
+
The model ships a multi token prediction head. It is inside every file here
|
| 87 |
+
and needs no extra download:
|
| 88 |
+
|
| 89 |
+
```bash
|
| 90 |
+
llama-cli -m SenseNova-Loop-9B-AD-Q4_K_M.gguf --spec-type draft-mtp -ngl 99 -c 8192
|
| 91 |
+
```
|
| 92 |
+
|
| 93 |
+
## Speed
|
| 94 |
+
|
| 95 |
+
<!-- TODO(editorial): llama-bench prompt and generation t/s at 8k and 32k for the recommended file -->
|
| 96 |
+
|
| 97 |
+
## Images and video
|
| 98 |
+
|
| 99 |
+
This model reads images, not only text. That needs one extra file, the vision
|
| 100 |
+
projector, which is separate from the quant and shared by all of them:
|
| 101 |
+
|
| 102 |
+
- `mmproj-SenseNova-Loop-9B-F16.gguf` for most people
|
| 103 |
+
|
| 104 |
+
Download it once alongside whichever quant you picked:
|
| 105 |
+
|
| 106 |
+
```bash
|
| 107 |
+
llama-mtmd-cli \
|
| 108 |
+
-m SenseNova-Loop-9B-AD-Q4_K_M.gguf \
|
| 109 |
+
--mmproj mmproj-SenseNova-Loop-9B-F16.gguf \
|
| 110 |
+
--image your-photo.jpg -ngl 99 -c 8192 \
|
| 111 |
+
-p "What is in this image?"
|
| 112 |
+
```
|
| 113 |
+
|
| 114 |
+
<!-- TODO(editorial): family specific vision flags (e.g. --image-min-tokens) and the demo image test -->
|
| 115 |
+
|
| 116 |
+
## How these compare to other builds
|
| 117 |
+
|
| 118 |
+
<!-- TODO(editorial): other publishers measured against the same reference (kld_ext), same-name table -->
|
| 119 |
+
|
| 120 |
+
## What we found while building these
|
| 121 |
+
|
| 122 |
+
<!-- TODO(editorial): findings from this release: layout experiments, bands, groups that mattered -->
|
| 123 |
+
|
| 124 |
+
## The calibration data
|
| 125 |
+
|
| 126 |
+
Our importance matrix comes from a corpus we built for this model and
|
| 127 |
+
published: [AtomicChat/calib-corpora](https://huggingface.co/datasets/AtomicChat/calib-corpora),
|
| 128 |
+
recipe `ornith-1.5-9b`. It is 4,958,490 tokens across 3,079 documents:
|
| 129 |
+
|
| 130 |
+
| Part | Share |
|
| 131 |
+
|---|---:|
|
| 132 |
+
| agentic and tool use | 24.7% |
|
| 133 |
+
| code | 17.8% |
|
| 134 |
+
| reasoning | 14.8% |
|
| 135 |
+
| multilingual | 13.8% |
|
| 136 |
+
| long context | 11.9% |
|
| 137 |
+
| vocabulary sweep | 9.9% |
|
| 138 |
+
| structured data | 3.9% |
|
| 139 |
+
| graphics | 3.0% |
|
| 140 |
+
|
| 141 |
+
Every conversation is rendered through **this model's own chat template**, so
|
| 142 |
+
the chat markup appears as the single tokens the model actually sees.
|
| 143 |
+
|
| 144 |
+
> [!IMPORTANT]
|
| 145 |
+
> If you build your own importance matrix from this corpus, pass
|
| 146 |
+
> `--parse-special` to `llama-imatrix`. Without it the chat markup is read as
|
| 147 |
+
> ordinary punctuation.
|
| 148 |
+
|
| 149 |
+
## How we measured
|
| 150 |
+
|
| 151 |
+
Reference: the original BF16 weights, converted to GGUF and run unquantized.
|
| 152 |
+
Perplexity on the held-out set is 8.7987 plus or minus 0.05818.
|
| 153 |
+
|
| 154 |
+
Held-out text: `eval_neutral` from our calibration dataset, never used for
|
| 155 |
+
calibration. 87 chunks at 4096 context.
|
| 156 |
+
|
| 157 |
+
Metric: per-token KL divergence of each file's predictions against the
|
| 158 |
+
reference, plus top-1 agreement.
|
| 159 |
+
|
| 160 |
+
Hardware: 1x H100 PCIe.
|
| 161 |
+
|
| 162 |
+
The raw logits of the reference are published in the metrics repo, split into
|
| 163 |
+
parts because of the file size limit. With them you can measure your own build
|
| 164 |
+
against exactly the same point we did:
|
| 165 |
+
|
| 166 |
+
```bash
|
| 167 |
+
cat base-neutral.kld.*.part > base-neutral.kld
|
| 168 |
+
|
| 169 |
+
llama-perplexity -m your-quant.gguf -f eval_neutral.txt \
|
| 170 |
+
--kl-divergence-base base-neutral.kld --kl-divergence -c 4096 -ngl 99
|
| 171 |
+
```
|
| 172 |
+
|
| 173 |
+
## Reproducing a file
|
| 174 |
+
|
| 175 |
+
The importance matrix was collected on the BF16 weights rather than on a
|
| 176 |
+
quantized copy, split across 8 workers and merged. Splitting by chunk
|
| 177 |
+
range gives the same result as one long run, because the statistic is a sum:
|
| 178 |
+
|
| 179 |
+
```bash
|
| 180 |
+
llama-imatrix -m SenseNova-Loop-9B-BF16.gguf -f calib_train.txt -o shard-0.gguf \
|
| 181 |
+
-ngl 99 -c 512 -b 8192 -ub 8192 --parse-special --from-chunk 0 --chunks 1211
|
| 182 |
+
|
| 183 |
+
llama-imatrix -m SenseNova-Loop-9B-BF16.gguf --in-file shard-0.gguf,shard-1.gguf,... \
|
| 184 |
+
-o imatrix.gguf
|
| 185 |
+
```
|
| 186 |
+
|
| 187 |
+
Then, for `AD-Q4_K_M`:
|
| 188 |
+
|
| 189 |
+
```bash
|
| 190 |
+
llama-quantize \
|
| 191 |
+
--imatrix imatrix.gguf \
|
| 192 |
+
--tensor-type '^blk\.56\.=q5_k' \
|
| 193 |
+
--tensor-type '^blk\.(0|1|26|27|52|53|54|55)\.ffn_(down|gate|up)\.weight$=q5_k' \
|
| 194 |
+
--tensor-type '^blk\.(2|3|4|5)\.ffn_(down|gate|up)\.weight$=q5_k' \
|
| 195 |
+
--tensor-type '^blk\.\d+\.ffn_down\.weight$=q4_k' \
|
| 196 |
+
--tensor-type '^blk\.\d+\.ffn_gate\.weight$=q4_k' \
|
| 197 |
+
--tensor-type '^blk\.\d+\.ffn_up\.weight$=q4_k' \
|
| 198 |
+
--tensor-type '^blk\.\d+\.attn_q\.weight$=q4_k' \
|
| 199 |
+
--tensor-type '^blk\.\d+\.attn_qkv_gate\.weight$=q5_k' \
|
| 200 |
+
--tensor-type '^blk\.\d+\.ssm_out\.weight$=q5_k' \
|
| 201 |
+
--tensor-type '^output\.weight$=q6_k' \
|
| 202 |
+
--tensor-type '^blk\.\d+\.attn_output\.weight$=q6_k' \
|
| 203 |
+
--tensor-type '^token_embd\.weight$=iq4_xs' \
|
| 204 |
+
--override-kv general.file_type=int:15 \
|
| 205 |
+
SenseNova-Loop-9B-BF16.gguf SenseNova-Loop-9B-AD-Q4_K_M.gguf Q8_0
|
| 206 |
+
```
|
| 207 |
+
|
| 208 |
+
The rules for every other file are in `ladder/` in the metrics repo, next to
|
| 209 |
+
its quantization log. The first rule that matches a tensor wins.
|
| 210 |
+
|
| 211 |
+
## Model details
|
| 212 |
+
|
| 213 |
+
56 layers, hidden size 4096, feed-forward size 12288, vocabulary 248,320, context 524,288. One multi token prediction head.
|
| 214 |
+
|
| 215 |
+
Needs a recent llama.cpp with `sensenova_loop` architecture support. Built and tested with llama.cpp b12467 (24ea7bb78).
|