GLM-OCR-NVFP4

NVFP4 (4-bit) quantization of zai-org/GLM-OCR, produced with NVIDIA Model Optimizer. Built and validated by VLM Run for document OCR serving on Blackwell GPUs.

The language decoder is quantized; the vision encoder is not. What the model can read off a page is bounded by the vision tower, so it stays in bfloat16 — only the decoder's linear layers move to 4-bit.

Component	Precision
Decoder linear layers	NVFP4 (W4A4, E2M1, block size 16)
Vision encoder, vision-language projector	bfloat16
Token embeddings, output head	bfloat16

Checkpoint: 2.7 GB → 1.5 GB.

Calibration

Calibrated on several hundred real document pages — scanned and born-digital, multilingual, including tables and figures — drawn from public OCR corpora. Each page was calibrated through a complete multimodal forward pass using the model's own serving prompt, so the activation ranges reflect real page traffic rather than a text-only proxy.

Quality vs the bf16 parent

Measured on held-out document pages that the calibration set never saw:

Pages with identical output	9 / 20
Next-token agreement with bf16	99.54%
KL divergence from bf16	0.0054
Change in content recall	-0.004
Independent vision-model judge (blind A/B)	4 prefer 4-bit · 12 tie · 4 prefer bf16

Requirements

NVFP4 kernels need a Blackwell GPU (SM100/SM120) — RTX 5090, RTX PRO 6000, B200 — and vLLM 0.25 or newer.

Usage

uv pip install "vllm>=0.25.0"
vllm serve vlm-run/GLM-OCR-NVFP4 --quantization modelopt_fp4 \
  --limit-mm-per-prompt '{"image": 1}' --no-enable-prefix-caching

Prompt the model exactly as upstream ("Text Recognition:", one page image per request).

Notes and limitations

• Not developed by VLM Run. See the upstream model card; its license and terms govern use of this model.
• On a minority of pages the output differs from the parent — usually Markdown heading or spacing style, occasionally a changed word or digit. Validate on your own documents before relying on it.
• Quantization was tuned and validated for document OCR. Other tasks the parent supports were not measured.
• Served and validated under vLLM only.