Munin 2.0 NVFP4

This is Munin 2.0 quantized to NVFP4: 80 GB of weights instead of 247 GB. We used the same mixed-precision layer map as NVIDIA's own Nemotron 3 Super NVFP4 release. Most expert and linear weights are 4-bit; the sensitive layers stay in higher precision, and the multi-token-prediction (MTP) layers stay in BF16. The quantization config also sets the KV cache to FP8.

This is the version to use on Blackwell GPUs, which run FP4 natively. On Hopper GPUs you save memory but gain no speed, so the BF16 repo with online FP8 quantization is usually the better choice unless memory is tight. For what the model is, how it was trained and how it scores, see the Munin 2.0 card.

Quality vs BF16

Munin 2.0 BF16 Munin 2.0 NVFP4
Mean KL divergence to BF16 (held-out Danish and agentic text) โ€” 0.0097
Top-1 token agreement with BF16, in vLLM โ€” 98.7%
GPQA-Diamond, quick single run 69.7 67.2
Terminal-Bench Lite, bash harness, single run 63.6 65.0
EuroEval Danish, 6-dataset subset โ€” within noise of BF16
MTP draft acceptance rate 0.65 0.58

We quantized it with NVIDIA ModelOpt 0.47.1 and MSE-based scale search, and calibrated it on about 2.5M tokens: rollout traces from Munin 2.0's RL run plus Danish text prompts.

Serving

The quantization format is detected automatically from hf_quant_config.json. We tested vLLM 0.28.0; set --tensor-parallel-size to your GPU count.

vllm serve danish-foundation-models/munin-2.0-nvfp4 \
  --served-model-name munin-2.0 \
  --trust-remote-code \
  --tensor-parallel-size 8 --enable-expert-parallel \
  --max-model-len 131072 \
  --kv-cache-dtype fp8 \
  --mamba-ssm-cache-dtype float32 \
  --async-scheduling --enable-chunked-prefill \
  --reasoning-parser nemotron_v3 \
  --enable-auto-tool-choice --tool-call-parser qwen3_coder \
  --speculative-config '{"method": "mtp", "num_speculative_tokens": 3}'
  • Hopper (H100, H200): vLLM runs the weights as 4-bit with 16-bit activations (Marlin kernels).
  • Blackwell (B200, GB200, RTX PRO 6000 Blackwell): runs FP4 natively.

Sampling and chat-template options are the same as for Munin 2.0: temperature=0.8, top_p=0.95, and reasoning on by default.

License

NVIDIA Nemotron Open Model License. Licensed by NVIDIA Corporation under the NVIDIA Nemotron Model License.

Downloads last month
84
Safetensors
Model size
124B params
Tensor type
F32
ยท
BF16
ยท
F8_E4M3
ยท
U8
ยท
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for danish-foundation-models/munin-2.0-nvfp4

Collection including danish-foundation-models/munin-2.0-nvfp4