GaMS3-12B-Instruct-Lex-NVFP4

This repository contains GaMS3-12B-Instruct-Lex quantized to NVIDIA NVFP4 (W4A4/FP4 E2M1) format using NVIDIA ModelOpt (nvidia-modelopt). The model is optimized for high-efficiency, low-latency inference on NVIDIA Blackwell architecture GPUs (such as B200 and newer) and is fully compatible with vLLM and TensorRT-LLM.

Model Summary

  • Original Model: cjvt/GaMS3-12B-Instruct-Lex
  • Architecture: Gemma3ForCausalLM
  • Quantization Format: NVFP4 (Weights 4-bit E2M1 with block_size=16 and per-block FP8 E4M3 scaling)
  • Model Size: ~9.39 GB (down from ~25.5 GB in BF16, ~3.5x compression)
  • Producer: NVIDIA ModelOpt

Serving with vLLM

Run with vLLM directly:

vllm serve tknez/GaMS3-12B-Instruct-Lex-NVFP4 --dtype bfloat16 --max-model-len 16384
Downloads last month
51
Safetensors
Model size
13B params
Tensor type
BF16
ยท
U8
ยท
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for tknez/GaMS3-12B-Instruct-Lex-NVFP4

Finetuned
cjvt/GaMS3-12B
Quantized
(1)
this model