Instructions to use rodrigoramosrs/veriloop-coder-e2-nvfp4 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use rodrigoramosrs/veriloop-coder-e2-nvfp4 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="rodrigoramosrs/veriloop-coder-e2-nvfp4")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("rodrigoramosrs/veriloop-coder-e2-nvfp4")
model = AutoModelForCausalLM.from_pretrained("rodrigoramosrs/veriloop-coder-e2-nvfp4", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use rodrigoramosrs/veriloop-coder-e2-nvfp4 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "rodrigoramosrs/veriloop-coder-e2-nvfp4"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "rodrigoramosrs/veriloop-coder-e2-nvfp4",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/rodrigoramosrs/veriloop-coder-e2-nvfp4

SGLang

How to use rodrigoramosrs/veriloop-coder-e2-nvfp4 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "rodrigoramosrs/veriloop-coder-e2-nvfp4" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "rodrigoramosrs/veriloop-coder-e2-nvfp4",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "rodrigoramosrs/veriloop-coder-e2-nvfp4" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "rodrigoramosrs/veriloop-coder-e2-nvfp4",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use rodrigoramosrs/veriloop-coder-e2-nvfp4 with Docker Model Runner:
```
docker model run hf.co/rodrigoramosrs/veriloop-coder-e2-nvfp4
```

VeriLoop Coder E2 · NVFP4

Coding-Optimized Quantized Model (NVIDIA NVFP4)

Original Model ↗ · GitHub · Apache-2.0

Overview

This repository contains an NVFP4 quantization of VeriLoop E2, an open 27B post-trained model built on Qwen3.8-27B for code, mathematics, and physics. Its core reasoning discipline is VeriLoop-Governed Recurrence (VGR): candidate states are recursively proposed, externally checked, and retained only when the protected evidence state improves without regression.

Quantized by Rodrigo Ramos.

Quantization Approach

Produced with NVIDIA Model Optimizer using the canonical NVFP4_W4A4_WEIGHT_LOCAL_HESSIAN_CFG recipe, verbatim: static per-block 4-bit weights (group size 16) + dynamic 4-bit activations, FP8 attention, local-Hessian calibration with MSE and fp8 scale sweep.

Calibration ran on 512 samples × 512 tokens (262144 tokens) of competitive-programming code from nvidia/Nemotron-Competitive-Programming-v1, keeping quantization faithful on the distributions that matter most for coding tasks.

The result retains the original model's strong software-engineering capabilities at roughly one third of the BF16 footprint, ready for NVFP4-capable inference stacks on Blackwell GPUs.

Files

File	Size	Notes
`model-00001-of-00005.safetensors` … `model-00005-of-00005.safetensors`	~17.5 GB total	NVFP4 weights (packed U8 + FP8 block scales)
`model.safetensors.index.json`	—	Shard index (2051 tensors)
`hf_quant_config.json`	—	`quant_algo: NVFP4`, group size 16
`config.json`, `tokenizer.*`, `merges.txt`, `vocab.json`	—	Config + tokenizer from the base model
`scripts/quantize_veriloop.py`	—	Full reproduction script (multi-GPU PTQ)
`scripts/validate_nvfp4.py`	—	Structural shard/index validator

Usage

This checkpoint targets NVFP4-capable runtimes (Blackwell GPUs). Plain transformers cannot load modelopt NVFP4 checkpoints directly.

Compatibility note: per the NVIDIA recipe, a small set of projections (linear_attn.in_proj_a/b, conv1d), plus the embeddings and lm_head, intentionally ship in BF16. The exclude_modules list in hf_quant_config.json (mirrored in config.json) declares exactly which modules those are — inference loaders must honor it and serve those modules unquantized. If a loader errors with "weight is torch.bfloat16 but the quant config declares NVFP4", it is ignoring the exclusion list, not a checkpoint defect.

vLLM

pip install 'vllm>=0.17.0'

vllm serve rodrigoramosrs/veriloop-coder-e2-nvfp4 \
    --tensor-parallel-size 1 \
    --max-model-len 131072 \
    --kv-cache-dtype fp8_e4m3 \
    --gpu-memory-utilization 0.92

SGLang

pip install 'sglang[all]'

python -m sglang.launch_server \
    --model-path rodrigoramosrs/veriloop-coder-e2-nvfp4 \
    --trust-remote-code \
    --mem-fraction-static 0.88

TensorRT-LLM

Build the engine from this checkpoint with trtllm-build using an NVFP4 + FP8-KV configuration, then serve with trtllm-serve. See the TensorRT-LLM documentation for the exact flags for your GPU.

Reproducing

pip install "nvidia-modelopt[all]" torch transformers accelerate datasets safetensors

# needs ~55 GB VRAM across CUDA GPUs for the 27B BF16 source model
python scripts/quantize_veriloop.py \
    --model ./model-bf16 \
    --output ./model-nvfp4 \
    --calib-size 512 \
    --calib-seq-len 512

python scripts/validate_nvfp4.py ./model-nvfp4

On Windows the Triton JIT needs a C compiler: run from a Visual Studio Native Tools prompt (vcvars64) with CC pointing at cl.exe. See the header of scripts/quantize_veriloop.py for multi-GPU layout tuning (--layers-split, --gpu-order).

Acknowledgements

Libo Wang and the Intelligent Robotics Laboratory, Tsinghua SIGS for developing the original VeriLoop E2 model.
NVIDIA for Model Optimizer and the NVFP4 quantization tooling.
The original model repository: tsinghua-sigs-robot-lab/VeriLoop-E2

License

Apache-2.0. The weights are quantized from the original Apache-2.0 licensed model. See the original repository for full licensing details and third-party notices.

Downloads last month: -

Safetensors

Model size

15B params

Tensor type

BF16

Model tree for rodrigoramosrs/veriloop-coder-e2-nvfp4

Base model

Qwen/Qwen3.8-27B

Finetuned

tsinghua-sigs-robot-lab/VeriLoop-E2

Quantized

(4)

this model