Instructions to use zai-org/GLM-5.3-Flash-BF16 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use zai-org/GLM-5.3-Flash-BF16 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="zai-org/GLM-5.3-Flash-BF16")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("zai-org/GLM-5.3-Flash-BF16")
model = AutoModelForMultimodalLM.from_pretrained("zai-org/GLM-5.3-Flash-BF16", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Inference
HuggingChat
Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use zai-org/GLM-5.3-Flash-BF16 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "zai-org/GLM-5.3-Flash-BF16"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "zai-org/GLM-5.3-Flash-BF16",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/zai-org/GLM-5.3-Flash-BF16

SGLang

How to use zai-org/GLM-5.3-Flash-BF16 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "zai-org/GLM-5.3-Flash-BF16" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "zai-org/GLM-5.3-Flash-BF16",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "zai-org/GLM-5.3-Flash-BF16" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "zai-org/GLM-5.3-Flash-BF16",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use zai-org/GLM-5.3-Flash-BF16 with Docker Model Runner:
```
docker model run hf.co/zai-org/GLM-5.3-Flash-BF16
```

chat template: speed up tool-result validation and reordering (GLM-5.3-Flash)

by ShijinZhang - opened 3 days ago

base: refs/heads/main

←

from: refs/pr/2

Discussion Files changed

+51

-55

ShijinZhang

Z.ai org 3 days ago

•

edited 3 days ago

Summary

Syncs the sort-merge tool-result validation and reordering optimization from GLM-5.3-Flash (Discussion #63 port, merged in the FP8 repo) to the BF16 repo. The template is otherwise identical to this repo's current chat_template.jinja.

The existing chat template uses nested loops and repeated macro calls to validate and reorder tool results, which introduces significant rendering overhead for large numbers of parallel tool calls. The updated implementation uses a sort-merge approach, preserving the original ordering and fallback behavior, and avoids Python-specific list.index() calls.

Correctness Validation

Compared this repo's current template against the optimized implementation using randomized conversations and targeted edge cases.

Suite 1 — Jinja2 differential (Python 3.13, Jinja2 3.1.6, ImmutableSandboxedEnvironment with loopcontrols and a Transformers-compatible tojson adapter), run under all four trim_blocks/lstrip_blocks configurations (2,000 cases each): byte-identical outputs / identical error types and messages, 0 behavioral mismatches.

Suite 2 — MiniJinja 2.24.0 differential (Python binding, pycompat=True, HF-compatible tojson): 15,000-case seeded corpus under chainable-undefined semantics — 11,586 byte-identical successes + 3,414 identical-error cases, 0 mismatches.

Suite 3 — NVIDIA Dynamo production renderer: the real Rust dynamo-renderer 8.0.1 (crates.io) with minijinja + minijinja-contrib 2.24.0 — the exact versions in Dynamo main's Cargo.lock (verified via cargo tree) — rendering both templates through the full production path on the identical 15,000-case corpus, 0 mismatches.

chat template: speed up tool-result validation and reordering (GLM-5.3-Flash)950ccb22

ShijinZhang changed pull request status to open 3 days ago

ZHANGYUXUAN-zR changed pull request status to merged 3 days ago

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment