GLM-5.3-W4AFP8-EP-GPTQ

A W4AFP8 quantization of GLM-5.3 by Bitdeer AI. Routed experts are 4-bit integer (group size 128, GPTQ-calibrated); attention, shared experts and the dense layers are block FP8. The checkpoint is about 400 GB, so the full 753B-parameter model serves on a single 8×H100 node, and it matches the published scores of the unquantized model on the benchmarks below.

This is the recommended of our two GLM-5.3 W4AFP8 checkpoints. Its sibling, GLM-5.3-W4AFP8-AWQ, scores the same but generates more output tokens per answer.

Model overview

Base model	zai-org/GLM-5.3-BF16 (revision 304b8051)
Architecture	GlmMoeDsaForCausalLM, 78 layers + 1 MTP layer, 256 routed experts per MoE layer
Parameters	753B
Quantization	W4AFP8: INT4 routed experts, FP8 activations and non-expert weights
Method	GPTQ (in-house expert-parallel implementation)
Checkpoint size	399.7 GB (including the 5.3 GB MTP draft layer)
Context length	1,048,576 tokens (model maximum)
Runtime	SGLang ≥ 0.5.17, --quantization w4afp8

Quantization

Routed experts are quantized to INT4 with GPTQ, using an in-house expert-parallel implementation.

Component	Precision
Routed experts, layers 3–77	INT4, group size 128, GPTQ
Attention (MLA) projections	Block FP8 (128×128)
DSA indexer (wk, wq_b)	Block FP8 (SGLang does not load it in BF16)
Shared experts; dense MLP layers 0–2	Block FP8
Router, lm_head, embeddings, norms	Unchanged (BF16)
MTP draft layer 78	INT4 experts + FP8, round-to-nearest (not calibrated)

• Calibration data: 256 samples × 2,048 tokens from UltraChat 200k, with all 256 experts calibrated in every layer.
• Expert parallelism: each GPU owns a subset of the experts and holds only their Hessians, which takes the per-GPU Hessian workspace for one MoE layer from about 76 GiB to about 9.5 GiB on 8 GPUs. IST-DASLab's MoE-Quant uses a similar approach for DeepSeek-V3.
• The tokenizer and generation config are unchanged from the base model. The chat template is the current upstream GLM-5.3 template (see Notes).

Evaluation

Three W4AFP8 checkpoints evaluated with the same harness, prompts and serving setup: our EP-GPTQ and AWQ checkpoints, and PhalaCloud/GLM-5.3-W4AFP8.

Following Artificial Analysis' methodology

Benchmark	This model	Ours (AWQ)	PhalaCloud	AA published (unquantized)
GPQA Diamond, pass@1	91.21 ± 0.88	91.52 ± 0.87	91.11 ± 0.85	~91.7
AA-LCR v1.1, pass@1	78.33 ± 2.4	79.00 ± 2.4	80.00 ± 2.3	80.0

± is one standard error. These are our own reproductions of Artificial Analysis' published methodology, not official Artificial Analysis runs. GPQA Diamond: 198 questions × 5 repeats. AA-LCR: 100 questions × 3 repeats, around 97k input tokens each, LLM-judged. On AA-LCR all three checkpoints answered the same 300 items; paired McNemar tests find no difference between any pair (p = 0.55–0.87).

Output tokens at equal accuracy

Mean completion tokens per answer	This model	Ours (AWQ)	PhalaCloud
GPQA Diamond	13,944	14,757	16,332
AA-LCR v1.1	5,203	6,363	5,656

This model reaches the same scores with shorter reasoning: 5.5% and 18.2% fewer output tokens than our AWQ checkpoint, and 14.6% and 8.0% fewer than PhalaCloud's. On a short-answer suite the difference disappears.

lm-evaluation-harness

Task	This model	Ours (AWQ)	PhalaCloud
GSM8K	97.57 ± 0.42	97.35 ± 0.44	96.82 ± 0.48
IFEval	88.91 ± 1.35	90.76 ± 1.25	90.20 ± 1.28
MMLU	86.84 ± 0.27	86.63 ± 0.28	86.81 ± 0.27
ARC Challenge	68.86 ± 1.35	69.37 ± 1.35	70.31 ± 1.34
HellaSwag	88.97 ± 0.31	89.20 ± 0.31	89.35 ± 0.31
TruthfulQA MC2	61.77 ± 1.45	62.88 ± 1.46	62.54 ± 1.46

Full test sets, lm-evaluation-harness 0.4.10 against the SGLang endpoint. No gap between the checkpoints reaches 2 points.

All evaluations used SGLang 0.5.17, FP8 KV cache, temperature 1.0, top_p 0.95, and a 131,072-token output limit for the reasoning benchmarks.

Usage

Serving on one 8×H100 node

python -m sglang.launch_server \
  --model-path BitdeerAI/GLM-5.3-W4AFP8-EP-GPTQ \
  --quantization w4afp8 \
  --tp 8 \
  --kv-cache-dtype fp8_e4m3 \
  --disable-shared-experts-fusion \
  --context-length 155000 \
  --mem-fraction-static 0.75 \
  --chunked-prefill-size 2048 \
  --reasoning-parser glm45 \
  --tool-call-parser glm47 \
  --trust-remote-code

On 8×H100-80GB this leaves room for a KV cache of about 155k tokens. For the full 1M-token context, serve across two 8×H100 nodes (--tp 16 --nnodes 2; we measured a 598,848-token KV pool at --mem-fraction-static 0.80), or use a node with more memory per GPU, such as 8×H200.

Speculative decoding (MTP)

The checkpoint includes GLM-5.3's MTP draft layer. To enable EAGLE speculative decoding, add:

  --speculative-algorithm EAGLE \
  --speculative-num-steps 3 \
  --speculative-eagle-topk 1 \
  --speculative-num-draft-tokens 4

We validated these flags on the two-node serve. We have not measured the speedup.

Sending requests

Use the base model's recommended sampling: temperature 1.0, top_p 0.95.

from openai import OpenAI

client = OpenAI(base_url="http://localhost:30000/v1", api_key="EMPTY")
response = client.chat.completions.create(
    model="BitdeerAI/GLM-5.3-W4AFP8-EP-GPTQ",
    messages=[{"role": "user", "content": "Explain expert parallelism in two sentences."}],
    temperature=1.0,
    top_p=0.95,
    max_tokens=32768,
)
print(response.choices[0].message.reasoning_content)
print(response.choices[0].message.content)

Notes

• These are quantized weights of GLM-5.3 and inherit the base model's capabilities, limitations and biases.
• The MTP draft layer is quantized with round-to-nearest, not calibrated. The main model verifies every drafted token, so this affects speculative-decoding acceptance, not outputs.
• Benchmark results are our reproductions and may differ from official leaderboard figures.
• Our evaluations ran with the base model's chat template as of 2026-08-28. This repository ships the current upstream template, which adds a fix for tool-calling conversations (tool-result reordering and messages with empty content); plain chat prompts render identically.

License

This model is a derivative of GLM-5.3 and is released under the same GLM-5.3 License. Copyright belongs to Z.ai; see LICENSE for the full terms.

Citation

If you use this model, please cite GLM-5:

@misc{glm5team2026glm5vibecodingagentic,
      title={GLM-5: from Vibe Coding to Agentic Engineering},
      author={GLM-5-Team and : and Aohan Zeng and Xin Lv and Zhenyu Hou and Zhengxiao Du and Qinkai Zheng and Bin Chen and Da Yin and Chendi Ge and Chenghua Huang and Chengxing Xie and Chenzheng Zhu and Congfeng Yin and Cunxiang Wang and Gengzheng Pan and Hao Zeng and Haoke Zhang and Haoran Wang and Huilong Chen and Jiajie Zhang and Jian Jiao and Jiaqi Guo and Jingsen Wang and Jingzhao Du and Jinzhu Wu and Kedong Wang and Lei Li and Lin Fan and Lucen Zhong and Mingdao Liu and Mingming Zhao and Pengfan Du and Qian Dong and Rui Lu and Shuang-Li and Shulin Cao and Song Liu and Ting Jiang and Xiaodong Chen and Xiaohan Zhang and Xuancheng Huang and Xuezhen Dong and Yabo Xu and Yao Wei and Yifan An and Yilin Niu and Yitong Zhu and Yuanhao Wen and Yukuo Cen and Yushi Bai and Zhongpei Qiao and Zihan Wang and Zikang Wang and Zilin Zhu and Ziqiang Liu and Zixuan Li and Bojie Wang and Bosi Wen and Can Huang and Changpeng Cai and Chao Yu and Chen Li and Chengwei Hu and Chenhui Zhang and Dan Zhang and Daoyan Lin and Dayong Yang and Di Wang and Ding Ai and Erle Zhu and Fangzhou Yi and Feiyu Chen and Guohong Wen and Hailong Sun and Haisha Zhao and Haiyi Hu and Hanchen Zhang and Hanrui Liu and Hanyu Zhang and Hao Peng and Hao Tai and Haobo Zhang and He Liu and Hongwei Wang and Hongxi Yan and Hongyu Ge and Huan Liu and Huanpeng Chu and Jia'ni Zhao and Jiachen Wang and Jiajing Zhao and Jiamin Ren and Jiapeng Wang and Jiaxin Zhang and Jiayi Gui and Jiayue Zhao and Jijie Li and Jing An and Jing Li and Jingwei Yuan and Jinhua Du and Jinxin Liu and Junkai Zhi and Junwen Duan and Kaiyue Zhou and Kangjian Wei and Ke Wang and Keyun Luo and Laiqiang Zhang and Leigang Sha and Liang Xu and Lindong Wu and Lintao Ding and Lu Chen and Minghao Li and Nianyi Lin and Pan Ta and Qiang Zou and Rongjun Song and Ruiqi Yang and Shangqing Tu and Shangtong Yang and Shaoxiang Wu and Shengyan Zhang and Shijie Li and Shuang Li and Shuyi Fan and Wei Qin and Wei Tian and Weining Zhang and Wenbo Yu and Wenjie Liang and Xiang Kuang and Xiangmeng Cheng and Xiangyang Li and Xiaoquan Yan and Xiaowei Hu and Xiaoying Ling and Xing Fan and Xingye Xia and Xinyuan Zhang and Xinze Zhang and Xirui Pan and Xu Zou and Xunkai Zhang and Yadi Liu and Yandong Wu and Yanfu Li and Yidong Wang and Yifan Zhu and Yijun Tan and Yilin Zhou and Yiming Pan and Ying Zhang and Yinpei Su and Yipeng Geng and Yong Yan and Yonglin Tan and Yuean Bi and Yuhan Shen and Yuhao Yang and Yujiang Li and Yunan Liu and Yunqing Wang and Yuntao Li and Yurong Wu and Yutao Zhang and Yuxi Duan and Yuxuan Zhang and Zezhen Liu and Zhengtao Jiang and Zhenhe Yan and Zheyu Zhang and Zhixiang Wei and Zhuo Chen and Zhuoer Feng and Zijun Yao and Ziwei Chai and Ziyuan Wang and Zuzhou Zhang and Bin Xu and Minlie Huang and Hongning Wang and Juanzi Li and Yuxiao Dong and Jie Tang},
      year={2026},
      eprint={2602.15763},
      archivePrefix={arXiv},
      primaryClass={cs.LG},
      url={https://arxiv.org/abs/2602.15763},
}

Acknowledgements

Thanks to the GLM team at Z.ai for releasing GLM-5.3. Served with SGLang.