SenseNova-Loop-9B — NVFP4-MLP (W4A16, quality-first)
A 4-bit NVFP4 build of sensenova/SenseNova-Loop-9B (the final release) that quantizes only the MLPs and leaves attention and the DeltaNet projections in BF16. It is made for vLLM on NVIDIA Blackwell and the DGX Spark / GB10, for when you want to stay as close to the original as an NVFP4 build allows.
On a single GB10 it decodes chat answers at 14 tok/s per stream, and at 25 tok/s with MTP speculative decoding: 2× BF16 in the same engine. It loads in 10.0 GiB (vision tower included), and in our cross-engine comparison it is closer to BF16 than the official Q4_K_M GGUF. Images work too: the vision tower stays in BF16 and is served through the plugin.
🆕 2026-11-19: the LM head is now NVFP4 too. The 248K × 4096 output layer is read in full at every decoded token; quantizing it made this build 14 % faster (22 % with MTP) and 1.5 GB smaller. Mean KLD went from 0.086 to 0.094, so it is still our most accurate build. The previous version with a BF16 LM head is revision
e2a005e0.
💚 Who this is for: Blackwell / DGX Spark owners who serve with vLLM and put accuracy first. Want more speed? The sibling SenseNova-Loop-9B-NVFP4 decodes at 26 tok/s (37 with MTP). On llama.cpp or a Mac, the community GGUF and MLX builds listed in the sibling card are the better fit.
Quick facts
| Base model | sensenova/SenseNova-Loop-9B, revision 3e1ba6e8 |
| Size | 10.9 GB on disk (BF16: 19 GB) |
| Format | NVFP4, nvidia-modelopt 0.43 checkpoint, quant_algo: W4A16_NVFP4 |
| Quantized | 96 linears: mlp.{gate,up,down}_proj in all 32 layers, plus lm_head (weight-only, no calibration needed) |
| Kept as in the source | all attention and DeltaNet projections, embed_tokens, norms, mHC routers (FP32), loop_gate, conv1d, A_log/dt_bias (FP32), vision tower, MTP head (all byte-identical to the source) |
| Calibration | 64 × 1,024 tokens of mixed text, through the real looped forward (layers 4–27 visited twice) |
| Input | text and images (vision tower in BF16) |
| MTP | the source's MTP head is included; use it with --speculative-config '{"method": "mtp", "num_speculative_tokens": 1}' |
Speed on DGX Spark (GB10)
vLLM 0.31.0 with the vllm-sensenova-loop plugin, BF16 KV cache. Six chat tasks (EN/DE), thinking off, up to 384 new tokens, greedy. Decode tok/s for one stream / total across four streams (plugin's tests/bench_mtp.py).
| Build | Weights in memory | No MTP | MTP, 1 draft token |
|---|---|---|---|
| BF16 (source) | 17.8 GiB | 7.5 / 22 | 12.2 / 31 |
| this build | 10.0 GiB | 14.0 / 53 | 24.6 / 76 |
| this build before 2026-11-19 (BF16 LM head) | 11.3 GiB | 12.3 / 50 | 20.1 / 67 |
| NVFP4 (full decoder) | 7.2 GiB | 25.9 / 90 | 36.6 / 126 |
| SenseNova AWQ (official) | 5.5 GiB | 27.5 / 77 | 39.4 / 104 |
Memory includes the vision tower; MTP adds 0.5 GiB. The draft token is accepted 69 % of the time.
Quality
All builds were scored on the same tokens in llama-perplexity style (2,048-token chunks, second half scored, full-vocabulary KLD against SenseNova's BF16 GGUF; 72,633 positions on one GB10). The sibling card describes the method in full.
| Build | Size | Mean KLD ↓ | Same top token ↑ |
|---|---|---|---|
| AtomicChat AD-Q4_K_M (GGUF, imatrix) | 5.96 GB | 0.054 | 92.6 % |
| this build | 10.9 GB | 0.094 * | 88.7 % * |
| this build before 2026-11-19 (BF16 LM head) | 12.4 GB | 0.087 | 90.0 % |
| SenseNova Q4_K_M (GGUF, official) | 5.82 GB | 0.111 | 89.2 % |
| balllvin NVFP4 (QAD) | 8.9 GB | 0.139 | 87.3 % |
| NVFP4 (full decoder) | 7.9 GB | 0.160 * | 85.5 % * |
| SenseNova AWQ (official) | 5.7 GB | 0.167 * | 85.1 % * |
* Measured later on the same tokens against the BF16 model in transformers instead of the BF16 GGUF (71,610 positions). The references agree closely: on the new one the previous version of this build scores 0.086 / 89.8 % (0.087 / 90.0 % above).
Per text type (mean KLD, this build): news 0.207 · wiki EN 0.085 · wiki DE 0.104 · code 0.089 · chat 0.043 · role-play 0.064.
To be fair to the GGUFs: AtomicChat's importance-matrix file is more accurate at half the size. Use it on llama.cpp. NVFP4's advantage is that it runs natively on Blackwell tensor cores in vLLM, which on a GB10 currently means about 3× the decode speed of the llama.cpp fork.
Images: on seven test prompts (shapes, OCR, chart, photo, banner, two images, text before image) all answers are correct, and 88–100 % of the answer tokens match BF16's top token in SenseNova's transformers code. Details in the sibling card.
Running it with vLLM
Needs the out-of-tree plugin vllm-sensenova-loop, which packages SenseTime's vLLM implementation from PR #64832 and adds ModelOpt NVFP4 support. It was tested with vllm/vllm-openai:v0.31.0 on GB10.
pip install --no-deps https://github.com/KaletoAI/vllm-sensenova-loop/archive/refs/heads/master.tar.gz # inside your vLLM 0.31 environment or image
vllm serve Kaleto/SenseNova-Loop-9B-NVFP4-MLP \
--served-model-name sensenova-loop-9b-nvfp4-mlp \
--reasoning-parser qwen3 \
--enable-auto-tool-choice --tool-call-parser qwen3_coder \
--max-model-len 32768 --max-num-seqs 3 \
--enable-chunked-prefill --max-num-batched-tokens 8192 \
--speculative-config '{"method": "mtp", "num_speculative_tokens": 1}'
Needs plugin commit 2778586 or newer for the NVFP4 LM head.
The sibling card covers thinking on/off, images, memory and the current plugin limits (video untested). On sampling, add top_k: 20 to SenseNova's temperature 1.0 / top_p 0.95 to keep the long tail clean. min_p works too, but vLLM 0.31 rejects it (HTTP 400) while MTP is on.
Credits & license
Thanks to SenseNova / SenseTime for the model and the vLLM implementation, to NVIDIA for modelopt, and to the community builders whose work is compared above.
Apache-2.0, inherited from sensenova/SenseNova-Loop-9B. This is an independent community build, not affiliated with SenseNova.
- Downloads last month
- 95
Model tree for Kaleto/SenseNova-Loop-9B-NVFP4-MLP
Base model
sensenova/SenseNova-Loop-9B