Instructions to use danish-foundation-models/munin-2.0-nvfp4 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use danish-foundation-models/munin-2.0-nvfp4 with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="danish-foundation-models/munin-2.0-nvfp4", trust_remote_code=True) messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("danish-foundation-models/munin-2.0-nvfp4", trust_remote_code=True) model = AutoModelForCausalLM.from_pretrained("danish-foundation-models/munin-2.0-nvfp4", trust_remote_code=True, device_map="auto") messages = [ {"role": "user", "content": "Who are you?"}, ] inputs = tokenizer.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=256) print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use danish-foundation-models/munin-2.0-nvfp4 with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "danish-foundation-models/munin-2.0-nvfp4" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "danish-foundation-models/munin-2.0-nvfp4", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/danish-foundation-models/munin-2.0-nvfp4
- SGLang
How to use danish-foundation-models/munin-2.0-nvfp4 with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "danish-foundation-models/munin-2.0-nvfp4" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "danish-foundation-models/munin-2.0-nvfp4", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "danish-foundation-models/munin-2.0-nvfp4" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "danish-foundation-models/munin-2.0-nvfp4", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Docker Model Runner
How to use danish-foundation-models/munin-2.0-nvfp4 with Docker Model Runner:
docker model run hf.co/danish-foundation-models/munin-2.0-nvfp4
Munin 2.0 NVFP4
This is Munin 2.0 quantized to NVFP4: 80 GB of weights instead of 247 GB. We used the same mixed-precision layer map as NVIDIA's own Nemotron 3 Super NVFP4 release. Most expert and linear weights are 4-bit; the sensitive layers stay in higher precision, and the multi-token-prediction (MTP) layers stay in BF16. The quantization config also sets the KV cache to FP8.
This is the version to use on Blackwell GPUs, which run FP4 natively. On Hopper GPUs you save memory but gain no speed, so the BF16 repo with online FP8 quantization is usually the better choice unless memory is tight. For what the model is, how it was trained and how it scores, see the Munin 2.0 card.
Quality vs BF16
| Munin 2.0 BF16 | Munin 2.0 NVFP4 | |
|---|---|---|
| Mean KL divergence to BF16 (held-out Danish and agentic text) | โ | 0.0097 |
| Top-1 token agreement with BF16, in vLLM | โ | 98.7% |
| GPQA-Diamond, quick single run | 69.7 | 67.2 |
| Terminal-Bench Lite, bash harness, single run | 63.6 | 65.0 |
| EuroEval Danish, 6-dataset subset | โ | within noise of BF16 |
| MTP draft acceptance rate | 0.65 | 0.58 |
We quantized it with NVIDIA ModelOpt 0.47.1 and MSE-based scale search, and calibrated it on about 2.5M tokens: rollout traces from Munin 2.0's RL run plus Danish text prompts.
Serving
The quantization format is detected automatically from hf_quant_config.json. We tested vLLM 0.28.0; set
--tensor-parallel-size to your GPU count.
vllm serve danish-foundation-models/munin-2.0-nvfp4 \
--served-model-name munin-2.0 \
--trust-remote-code \
--tensor-parallel-size 8 --enable-expert-parallel \
--max-model-len 131072 \
--kv-cache-dtype fp8 \
--mamba-ssm-cache-dtype float32 \
--async-scheduling --enable-chunked-prefill \
--reasoning-parser nemotron_v3 \
--enable-auto-tool-choice --tool-call-parser qwen3_coder \
--speculative-config '{"method": "mtp", "num_speculative_tokens": 3}'
- Hopper (H100, H200): vLLM runs the weights as 4-bit with 16-bit activations (Marlin kernels).
- Blackwell (B200, GB200, RTX PRO 6000 Blackwell): runs FP4 natively.
Sampling and chat-template options are the same as for Munin 2.0: temperature=0.8, top_p=0.95, and
reasoning on by default.
License
NVIDIA Nemotron Open Model License. Licensed by NVIDIA Corporation under the NVIDIA Nemotron Model License.
- Downloads last month
- 84