SenseNova-Loop-9B — NVFP4-MLP (W4A16, quality-first)

A 4-bit NVFP4 build of sensenova/SenseNova-Loop-9B (the final release) that quantizes only the MLPs and leaves attention and the DeltaNet projections in BF16. It is made for vLLM on NVIDIA Blackwell and the DGX Spark / GB10, for when you want to stay as close to the original as an NVFP4 build allows.

On a single GB10 it decodes chat answers at 14 tok/s per stream, and at 25 tok/s with MTP speculative decoding: 2× BF16 in the same engine. It loads in 10.0 GiB (vision tower included), and in our cross-engine comparison it is closer to BF16 than the official Q4_K_M GGUF. Images work too: the vision tower stays in BF16 and is served through the plugin.

🆕 2026-11-19: the LM head is now NVFP4 too. The 248K × 4096 output layer is read in full at every decoded token; quantizing it made this build 14 % faster (22 % with MTP) and 1.5 GB smaller. Mean KLD went from 0.086 to 0.094, so it is still our most accurate build. The previous version with a BF16 LM head is revision e2a005e0.

💚 Who this is for: Blackwell / DGX Spark owners who serve with vLLM and put accuracy first. Want more speed? The sibling SenseNova-Loop-9B-NVFP4 decodes at 26 tok/s (37 with MTP). On llama.cpp or a Mac, the community GGUF and MLX builds listed in the sibling card are the better fit.


Quick facts

Base model sensenova/SenseNova-Loop-9B, revision 3e1ba6e8
Size 10.9 GB on disk (BF16: 19 GB)
Format NVFP4, nvidia-modelopt 0.43 checkpoint, quant_algo: W4A16_NVFP4
Quantized 96 linears: mlp.{gate,up,down}_proj in all 32 layers, plus lm_head (weight-only, no calibration needed)
Kept as in the source all attention and DeltaNet projections, embed_tokens, norms, mHC routers (FP32), loop_gate, conv1d, A_log/dt_bias (FP32), vision tower, MTP head (all byte-identical to the source)
Calibration 64 × 1,024 tokens of mixed text, through the real looped forward (layers 4–27 visited twice)
Input text and images (vision tower in BF16)
MTP the source's MTP head is included; use it with --speculative-config '{"method": "mtp", "num_speculative_tokens": 1}'

Speed on DGX Spark (GB10)

vLLM 0.31.0 with the vllm-sensenova-loop plugin, BF16 KV cache. Six chat tasks (EN/DE), thinking off, up to 384 new tokens, greedy. Decode tok/s for one stream / total across four streams (plugin's tests/bench_mtp.py).

Build Weights in memory No MTP MTP, 1 draft token
BF16 (source) 17.8 GiB 7.5 / 22 12.2 / 31
this build 10.0 GiB 14.0 / 53 24.6 / 76
this build before 2026-11-19 (BF16 LM head) 11.3 GiB 12.3 / 50 20.1 / 67
NVFP4 (full decoder) 7.2 GiB 25.9 / 90 36.6 / 126
SenseNova AWQ (official) 5.5 GiB 27.5 / 77 39.4 / 104

Memory includes the vision tower; MTP adds 0.5 GiB. The draft token is accepted 69 % of the time.

Quality

All builds were scored on the same tokens in llama-perplexity style (2,048-token chunks, second half scored, full-vocabulary KLD against SenseNova's BF16 GGUF; 72,633 positions on one GB10). The sibling card describes the method in full.

Build Size Mean KLD ↓ Same top token ↑
AtomicChat AD-Q4_K_M (GGUF, imatrix) 5.96 GB 0.054 92.6 %
this build 10.9 GB 0.094 * 88.7 % *
this build before 2026-11-19 (BF16 LM head) 12.4 GB 0.087 90.0 %
SenseNova Q4_K_M (GGUF, official) 5.82 GB 0.111 89.2 %
balllvin NVFP4 (QAD) 8.9 GB 0.139 87.3 %
NVFP4 (full decoder) 7.9 GB 0.160 * 85.5 % *
SenseNova AWQ (official) 5.7 GB 0.167 * 85.1 % *

* Measured later on the same tokens against the BF16 model in transformers instead of the BF16 GGUF (71,610 positions). The references agree closely: on the new one the previous version of this build scores 0.086 / 89.8 % (0.087 / 90.0 % above).

Per text type (mean KLD, this build): news 0.207 · wiki EN 0.085 · wiki DE 0.104 · code 0.089 · chat 0.043 · role-play 0.064.

To be fair to the GGUFs: AtomicChat's importance-matrix file is more accurate at half the size. Use it on llama.cpp. NVFP4's advantage is that it runs natively on Blackwell tensor cores in vLLM, which on a GB10 currently means about 3× the decode speed of the llama.cpp fork.

Images: on seven test prompts (shapes, OCR, chart, photo, banner, two images, text before image) all answers are correct, and 88–100 % of the answer tokens match BF16's top token in SenseNova's transformers code. Details in the sibling card.

Running it with vLLM

Needs the out-of-tree plugin vllm-sensenova-loop, which packages SenseTime's vLLM implementation from PR #64832 and adds ModelOpt NVFP4 support. It was tested with vllm/vllm-openai:v0.31.0 on GB10.

pip install --no-deps https://github.com/KaletoAI/vllm-sensenova-loop/archive/refs/heads/master.tar.gz   # inside your vLLM 0.31 environment or image

vllm serve Kaleto/SenseNova-Loop-9B-NVFP4-MLP \
  --served-model-name sensenova-loop-9b-nvfp4-mlp \
  --reasoning-parser qwen3 \
  --enable-auto-tool-choice --tool-call-parser qwen3_coder \
  --max-model-len 32768 --max-num-seqs 3 \
  --enable-chunked-prefill --max-num-batched-tokens 8192 \
  --speculative-config '{"method": "mtp", "num_speculative_tokens": 1}'

Needs plugin commit 2778586 or newer for the NVFP4 LM head.

The sibling card covers thinking on/off, images, memory and the current plugin limits (video untested). On sampling, add top_k: 20 to SenseNova's temperature 1.0 / top_p 0.95 to keep the long tail clean. min_p works too, but vLLM 0.31 rejects it (HTTP 400) while MTP is on.

Credits & license

Thanks to SenseNova / SenseTime for the model and the vLLM implementation, to NVIDIA for modelopt, and to the community builders whose work is compared above.

Apache-2.0, inherited from sensenova/SenseNova-Loop-9B. This is an independent community build, not affiliated with SenseNova.

Downloads last month
95
Safetensors
Model size
10B params
Tensor type
F32
·
BF16
·
U8
·
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for Kaleto/SenseNova-Loop-9B-NVFP4-MLP

Quantized
(44)
this model