GaMS3-12B-Instruct-Lex-NVFP4
This repository contains GaMS3-12B-Instruct-Lex quantized to NVIDIA NVFP4 (W4A4/FP4 E2M1) format using NVIDIA ModelOpt (nvidia-modelopt).
The model is optimized for high-efficiency, low-latency inference on NVIDIA Blackwell architecture GPUs (such as B200 and newer) and is fully compatible with vLLM and TensorRT-LLM.
Model Summary
- Original Model:
cjvt/GaMS3-12B-Instruct-Lex - Architecture:
Gemma3ForCausalLM - Quantization Format: NVFP4 (Weights 4-bit E2M1 with block_size=16 and per-block FP8 E4M3 scaling)
- Model Size: ~9.39 GB (down from ~25.5 GB in BF16, ~3.5x compression)
- Producer: NVIDIA ModelOpt
Serving with vLLM
Run with vLLM directly:
vllm serve tknez/GaMS3-12B-Instruct-Lex-NVFP4 --dtype bfloat16 --max-model-len 16384
- Downloads last month
- 51