QDA-GLM-5.3-Flash-W4A4

NVFP4 quantization (4-bit weights and 4-bit activations) of zai-org/GLM-5.3-Flash for NVIDIA Blackwell GPUs. 197.2 GB vs. 328.3 GB for the original FP8 checkpoint, and 98.3 % of its accuracy (average of four benchmarks).

Evaluation

Benchmark FP8 (original)
328.3 GB
QDA W4A4
197.2 GB
NVIDIA NVFP4
204.4 GB
RedHatAI NVFP4
197.8 GB
GPQA-Diamond
198 problems × 12 draws
86.0783.8883.3384.01
Math
93 problems × 16 draws
84.8183.2780.9178.83
LiveCodeBench v6
454 problems × 4 draws
71.3170.3769.3369.16
SuperGPQA
9,000 problems × 1 draw
68.6667.9167.7767.38
Average77.776.475.374.8
Share of FP8100 %98.3 %96.9 %96.3 %

Thinking mode, pass@1 averaged over the draws; same problems, draws and seeds for every checkpoint. Math is AIME 2026, HMMT Feb 2026 and HMMT Nov 2025. Served with vLLM 0.31.0 on 4× GB300 (tensor parallel 4).

Usage

Needs 4 NVIDIA Blackwell GPUs and a CUDA 13.0 toolkit: vLLM compiles DeepGEMM kernels at start (the MTP layer's FP8 experts and the sparse indexer), so set CUDA_HOME to the toolkit before vllm serve.

vllm serve qda-lab/QDA-GLM-5.3-Flash-W4A4 --tensor-parallel-size 4 --reasoning-parser glm47 \
  --speculative-config '{"method": "mtp", "num_speculative_tokens": 5}'

Format

The routed experts of the MoE layers (3–44) and the dense MLPs of layers 0–2 are NVFP4: E2M1 weights with one E4M3 scale per 16 inputs and an FP32 tensor scale, and activations quantized the same way at run time. The MTP layer's experts are FP8 (block-scaled weights, dynamic activation scales). Attention (linear and sparse MLA), shared experts, routers, embeddings, lm_head and the vision tower stay in BF16. The checkpoint is in the compressed-tensors format. The quantization recipe is not published.

License

MIT, the same as the base model.

Citation

QDA: Closed-Loop LLM Quantization Design with Agents

@article{li2026qda,
  title   = {QDA: Closed-Loop LLM Quantization Design with Agents},
  author  = {Li, Xingyang and Zhao, Tianlang and Han, Song and Zhu, Ligeng},
  journal = {arXiv preprint arXiv:2611.23388},
  year    = {2026}
}
Downloads last month
30
Safetensors
Model size
169B params
Tensor type
F32
·
BF16
·
U8
·
F8_E4M3
·
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for qda-lab/QDA-GLM-5.3-Flash-W4A4

Quantized
(251)
this model

Paper for qda-lab/QDA-GLM-5.3-Flash-W4A4