Instructions to use tiny-random/glm-5.3-flash with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use tiny-random/glm-5.3-flash with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="tiny-random/glm-5.3-flash")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("tiny-random/glm-5.3-flash")
model = AutoModelForMultimodalLM.from_pretrained("tiny-random/glm-5.3-flash", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use tiny-random/glm-5.3-flash with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "tiny-random/glm-5.3-flash"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "tiny-random/glm-5.3-flash",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/tiny-random/glm-5.3-flash

SGLang

How to use tiny-random/glm-5.3-flash with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "tiny-random/glm-5.3-flash" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "tiny-random/glm-5.3-flash",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "tiny-random/glm-5.3-flash" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "tiny-random/glm-5.3-flash",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use tiny-random/glm-5.3-flash with Docker Model Runner:
```
docker model run hf.co/tiny-random/glm-5.3-flash
```

yujiepan commited on 7 days ago

Commit

1e7065c

verified ·

1 Parent(s): b3efe4a

Upload folder using huggingface_hub

Browse files

Files changed (1) hide show

README.md +1 -1

README.md CHANGED Viewed

@@ -23,7 +23,7 @@ Structural comparison with `zai-org/GLM-5.3-Flash`:
 | Vision depth / hidden / output size | 24 / 1,024 / 4,096 | 2 / 128 / 128 |
 | Checkpoint precision | BF16 plus E4M3 block-FP8 and FP32 scales | Same; 32x32 blocks for reduced axes |
-The KDA and DSA indexer heads remain 128-wide because vLLM's GLM5-Next KDA and Hadamard-128 indexer kernels require that ABI. The source E4M3 format is retained with reduced 32x32 blocks; the TP=4 example selects vLLM's Triton block-FP8 GEMM because H20's automatic Humming kernel requires an output scale group of at least 64.
 Tokenizer, processor, chat template and generation assets are copied unchanged from the pinned source revision.
 | File path | Size |

 | Vision depth / hidden / output size | 24 / 1,024 / 4,096 | 2 / 128 / 128 |
 | Checkpoint precision | BF16 plus E4M3 block-FP8 and FP32 scales | Same; 32x32 blocks for reduced axes |
+The KDA and DSA indexer heads remain 128-wide because vLLM's GLM5-Next KDA and Hadamard-128 indexer kernels require that ABI. The source E4M3 format is retained with reduced 32x32 blocks; the TP=4 example selects vLLM's Triton block-FP8 GEMM because H20's automatic Humming kernel requires an output scale group of at least 64. With the unchanged 32-wide FP8 K block, vLLM TP=4 partitions residual-width linear inputs by four. The 32-wide candidate reached GLM5-Next model loading but was rejected with `input_size_per_partition = 8 is not divisible by weight quantization block_k = 32`; a 64-wide residual would likewise partition to 16. The tested 128-wide checkpoint is therefore the smallest executable residual width for this required checkpoint/backend/TP combination.
 Tokenizer, processor, chat template and generation assets are copied unchanged from the pinned source revision.
 | File path | Size |