Instructions to use tiny-random/glm-5.3 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use tiny-random/glm-5.3 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="tiny-random/glm-5.3")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("tiny-random/glm-5.3")
model = AutoModelForCausalLM.from_pretrained("tiny-random/glm-5.3", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use tiny-random/glm-5.3 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "tiny-random/glm-5.3"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "tiny-random/glm-5.3",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/tiny-random/glm-5.3

SGLang

How to use tiny-random/glm-5.3 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "tiny-random/glm-5.3" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "tiny-random/glm-5.3",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "tiny-random/glm-5.3" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "tiny-random/glm-5.3",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use tiny-random/glm-5.3 with Docker Model Runner:
```
docker model run hf.co/tiny-random/glm-5.3
```

glm-5.3 / README.md

yujiepan

Move size summary to top of comparison

b0430c6 verified 3 days ago

preview code

Raw History Blame Contribute Delete

18.1 kB

---
library_name: transformers
base_model:
- zai-org/GLM-5.3
---
This tiny model is intended for debugging. It is randomly initialized using the configuration adapted from [zai-org/GLM-5.3](https://huggingface.co/zai-org/GLM-5.3).

Structural comparison with `zai-org/GLM-5.3`:

| Feature | Source | Tiny |
|---|---:|---:|
| Checkpoint tensor elements | 753.33B | 146.57M, including 104.21K FP32 scale elements |
| Safetensors weight files | 755.63 GB across shards | 186.88 MB (`model.safetensors`) |
| Architecture / input | GLM-MoE-DSA; text | Same |
| Vocabulary rows / tied embeddings | 154,880 / false | Same |
| Backbone layers | 78 | 11: three dense, then two complete shared/shared/shared/full cycles |
| Hidden size | 6,144 | 128 |
| Attention heads / NoPE / RoPE / value dimensions | 64 / 192 / 64 / 256 | 64 / 32 / 32 / 256 |
| Query / KV LoRA rank | 2,048 / 512 | 128 / 256 |
| Indexer heads / dimension / top-k | 32 / 128 / 2,048 | 32 / 128 / 128 |
| Routed experts / selected experts / shared experts | 256 / 8 / 1 | 32 / 8 / 1 |
| Expert intermediate size | 2,048 | 128 |
| MTP | One DSA/MoE layer with shared embedding/head | Same |
| Checkpoint precision | BF16 plus E4M3 block-FP8 and FP32 scales | Same; 32x32 blocks for reduced axes |

The indexer head remains 128-wide: vLLM's GLM DSA cache and fused-indexer path use a 128-wide cache layout. The source E4M3 format is retained with reduced 32x32 blocks; the TP=4 example selects vLLM's Triton block-FP8 GEMM because H20's automatic Humming kernel requires an output scale group of at least 64. With that unchanged 32-wide FP8 K block, vLLM TP=4 partitions residual-width linear inputs by four. Both 32-wide and 64-wide candidates were generated and rejected at model load with `input_size_per_partition = 8` and `16` respectively, neither divisible by `block_k = 32`; 128 is therefore the smallest executable residual width for this required checkpoint/backend/TP combination.
Tokenizer and generation assets are copied unchanged from the pinned source revision.

| File path | Size |
|------|------|
| model.safetensors | 186.9MB |


### Example usage:

- vLLM (✅ verified)

```bash
vllm serve tiny-random/glm-5.3 \
  --tensor-parallel-size 4 \
  --speculative-config '{"method":"mtp","num_speculative_tokens":1}' \
  --max-model-len 4096 \
  --max-num-batched-tokens 512 \
  --max-num-seqs 4 \
  --gpu-memory-utilization 0.3 \
  --kernel-config '{"moe_backend":"triton","linear_backend":"triton"}' \
  --compilation-config '{"cudagraph_capture_sizes":[1,2,4,8]}'
```

- Transformers (✅ verified)

```python
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from transformers.integrations.hub_kernels import allow_all_hub_kernels
model_id = "tiny-random/glm-5.3"
# The GLM-5.3 Transformers implementation supports the DSA sparse mask on
# SDPA; eager attention is deliberately not selected.
# Transformers resolves the official fine-grained FP8 kernel from the Hub.
with allow_all_hub_kernels():
    model = AutoModelForCausalLM.from_pretrained(
        model_id, dtype=torch.bfloat16, attn_implementation="sdpa",
    ).to("cuda")
    tokenizer = AutoTokenizer.from_pretrained(model_id)
    inputs = tokenizer.apply_chat_template(
        [{"role": "user", "content": "Hello"}], tokenize=True,
        add_generation_prompt=True, return_dict=True, return_tensors="pt",
    ).to(model.device)
    output = model.generate(**inputs, do_sample=False, max_new_tokens=10)
print(tokenizer.decode(output[0, inputs["input_ids"].shape[1]:], skip_special_tokens=True))
```

### Codes to create this repo (✅ verified):

<details><summary>Click to expand</summary>

```python
"""Create the GLM-MoE-DSA checkpoint using the public FP8 checkpoint schema."""
import json
from pathlib import Path
from shutil import copyfile
import torch
from huggingface_hub import hf_hub_download, list_repo_files
from safetensors.torch import save_file
from transformers import AutoConfig, AutoModelForCausalLM, set_seed
source_model_id = "zai-org/GLM-5.3"
save_folder = "/tmp/tiny-random/glm-53"
source_revision = "ba32f8c19c8738e2c57c71798c601d477a7bf844"
set_seed(42)
folder = Path(save_folder)
folder.mkdir(parents=True, exist_ok=True)
raw = json.loads(Path(hf_hub_download(
    source_model_id, "config.json", revision=source_revision)).read_text())
# The source uses block-FP8.  32x32 is the smallest source-compatible
# block geometry.  TP=4 therefore needs every sharded FP8 axis to be at
# least 128 wide, so each rank owns one complete 32-wide block.
raw["quantization_config"]["weight_block_size"] = [32, 32]
raw.update(
    hidden_size=128,
    num_hidden_layers=11,
    # Retain the source's 64 global attention heads.  The regular MLA
    # path shards these to 16 query heads per TP=4 rank; its 32-wide
    # reduced NoPE/RoPE projections and source-sized values keep every
    # backend-required head dimension valid.
    num_attention_heads=64,
    num_key_value_heads=64,
    intermediate_size=128,
    moe_intermediate_size=128,
    n_routed_experts=32,
    num_experts_per_tok=8,
    q_lora_rank=128,
    # Sparse-MQA passes this latent as FA3's q_v input.  The optimized
    # CUDA kernel requires it to be at least 256 wide; source uses 512.
    kv_lora_rank=256,
    qk_nope_head_dim=32,
    qk_rope_head_dim=32,
    # vLLM's sparse-MQA FlashAttention path requires q_v >= 256; this
    # therefore retains the source value-head dimension.
    v_head_dim=256,
    index_head_dim=128,
    # This is the DSA indexer head count, distinct from normal attention.
    # vLLM's fused sparse-indexer kernel accepts 32 or 64; retain source 32.
    index_n_heads=32,
    index_topk=128,
    first_k_dense_replace=3,
)
# The original three shared-indexer layers followed by one full indexer
# are retained twice after the dense prefix.
raw["indexer_types"] = ["full", "full", "full"] + ["shared", "shared", "shared", "full"] * 2
raw["mlp_layer_types"] = ["dense"] * 3 + ["sparse"] * 8
# Build the Transformer graph from the reduced config before serializing
# the source checkpoint layout.
config_for_build = dict(raw)
config_for_build.pop("quantization_config")
(folder / "config.json").write_text(json.dumps(config_for_build, indent=2) + "\n")
config = AutoConfig.from_pretrained(folder)
model = AutoModelForCausalLM.from_config(config, dtype=torch.bfloat16)
with torch.no_grad():
    for name, value in model.named_parameters():
        if name.endswith("e_score_correction_bias"):
            value.zero_()
        elif name.endswith("weight") and "layernorm" in name:
            value.fill_(1)
        else:
            value.normal_(mean=0.0, std=0.02)
state = {name: value.detach().cpu().contiguous() for name, value in model.state_dict().items()}
block_out, block_in = raw["quantization_config"]["weight_block_size"]
def pack(name: str, weight: torch.Tensor) -> None:
    rows, columns = weight.shape
    assert rows % block_out == 0 and columns % block_in == 0, (name, weight.shape)
    blocks = weight.float().reshape(rows // block_out, block_out, columns // block_in, block_in)
    scales = (blocks.abs().amax(dim=(1, 3)) / 448).clamp_min(1e-12).to(torch.float32)
    state[name] = (blocks / scales[:, None, :, None]).clamp(-448, 448).reshape(weight.shape).to(
        torch.float8_e4m3fn
    ).contiguous()
    state[name.removesuffix("weight") + "weight_scale_inv"] = scales.contiguous()
def keep_bf16(name: str) -> bool:
    return (
        name in {"model.embed_tokens.weight", "lm_head.weight"}
        or ".layernorm." in name
        or ".indexer.k_norm." in name
        or name.endswith(".indexer.weights_proj.weight")
        or name.endswith(".mlp.gate.weight")
        or name.endswith(".mlp.gate.e_score_correction_bias")
    )
# Source exclusion paths are tied to its 78-layer tree.  Rebuild the list
# for the reduced graph so Transformers does not turn BF16 norms, routers
# or indexer projections into FP8 modules while loading this checkpoint.
exclusions = {"model.embed_tokens", "lm_head"}
for module_name, module in model.named_modules():
    if isinstance(module, torch.nn.Linear) and keep_bf16(module_name + ".weight"):
        exclusions.add(module_name)
raw["quantization_config"]["modules_to_not_convert"] = sorted(exclusions)
# The public checkpoint stores routed experts one expert per tensor; the
# Transformers graph stores its expert matrices as compact three-dimensional
# tensors.  Convert only at serialization time.
for name in list(state):
    if not name.endswith(".mlp.experts.gate_up_proj"):
        continue
    prefix = name.removesuffix(".gate_up_proj")
    gate_up = state.pop(name)
    down = state.pop(prefix + ".down_proj")
    for expert in range(gate_up.shape[0]):
        gate, up = gate_up[expert].chunk(2, dim=0)
        pack(f"{prefix}.{expert}.gate_proj.weight", gate)
        pack(f"{prefix}.{expert}.up_proj.weight", up)
        pack(f"{prefix}.{expert}.down_proj.weight", down[expert])
for name in list(state):
    value = state[name]
    if not (name.endswith(".weight") and value.ndim == 2) or keep_bf16(name):
        continue
    pack(name, value)
# GLM's embedded one-layer MTP head is represented as the source's extra
# decoder layer.  It shares the checkpoint's global embedding and LM head.
base_prefix = "model.layers.10."
mtp_prefix = "model.layers.11."
for name, value in list(state.items()):
    if name.startswith(base_prefix):
        state[mtp_prefix + name.removeprefix(base_prefix)] = value.clone()
state[mtp_prefix + "eh_proj.weight"] = torch.empty(128, 256, dtype=torch.bfloat16).normal_(0, 0.02)
for name in ("enorm.weight", "hnorm.weight", "shared_head.norm.weight"):
    state[mtp_prefix + name] = torch.ones(128, dtype=torch.bfloat16)
save_file(state, folder / "model.safetensors", metadata={"format": "pt"})
(folder / "config.json").write_text(json.dumps(raw, indent=2) + "\n")
assets = {
    "tokenizer.json", "tokenizer_config.json", "special_tokens_map.json",
    "chat_template.jinja", "generation_config.json", "LICENSE",
}
for name in sorted(set(list_repo_files(source_model_id, revision=source_revision)) & assets):
    copyfile(hf_hub_download(source_model_id, name, revision=source_revision), folder / name)
```

</details>

### Printing the model:

<details><summary>Click to expand</summary>

```text
GlmMoeDsaForCausalLM(
  (model): GlmMoeDsaModel(
    (embed_tokens): Embedding(154880, 128, padding_idx=154820)
    (layers): ModuleList(
      (0-2): 3 x GlmMoeDsaDecoderLayer(
        (self_attn): GlmMoeDsaAttention(
          (q_a_proj): Linear(in_features=128, out_features=128, bias=False)
          (q_a_layernorm): GlmMoeDsaRMSNorm((128,), eps=1e-06)
          (q_b_proj): Linear(in_features=128, out_features=4096, bias=False)
          (kv_a_proj_with_mqa): Linear(in_features=128, out_features=288, bias=False)
          (kv_a_layernorm): GlmMoeDsaRMSNorm((256,), eps=1e-06)
          (kv_b_proj): Linear(in_features=256, out_features=18432, bias=False)
          (o_proj): Linear(in_features=16384, out_features=128, bias=False)
          (indexer): GlmMoeDsaIndexer(
            (wq_b): Linear(in_features=128, out_features=4096, bias=False)
            (wk): Linear(in_features=128, out_features=128, bias=False)
            (k_norm): LayerNorm((128,), eps=1e-06, elementwise_affine=True, bias=True)
            (weights_proj): Linear(in_features=128, out_features=32, bias=False)
          )
        )
        (mlp): GlmMoeDsaMLP(
          (gate_proj): Linear(in_features=128, out_features=128, bias=False)
          (up_proj): Linear(in_features=128, out_features=128, bias=False)
          (down_proj): Linear(in_features=128, out_features=128, bias=False)
          (act_fn): SiLUActivation()
        )
        (input_layernorm): GlmMoeDsaRMSNorm((128,), eps=1e-05)
        (post_attention_layernorm): GlmMoeDsaRMSNorm((128,), eps=1e-05)
      )
      (3-5): 3 x GlmMoeDsaDecoderLayer(
        (self_attn): GlmMoeDsaAttention(
          (q_a_proj): Linear(in_features=128, out_features=128, bias=False)
          (q_a_layernorm): GlmMoeDsaRMSNorm((128,), eps=1e-06)
          (q_b_proj): Linear(in_features=128, out_features=4096, bias=False)
          (kv_a_proj_with_mqa): Linear(in_features=128, out_features=288, bias=False)
          (kv_a_layernorm): GlmMoeDsaRMSNorm((256,), eps=1e-06)
          (kv_b_proj): Linear(in_features=256, out_features=18432, bias=False)
          (o_proj): Linear(in_features=16384, out_features=128, bias=False)
        )
        (mlp): GlmMoeDsaMoE(
          (experts): GlmMoeDsaExperts(
            (act_fn): SiLUActivation()
          )
          (gate): GlmMoeDsaTopkRouter()
          (shared_experts): GlmMoeDsaMLP(
            (gate_proj): Linear(in_features=128, out_features=128, bias=False)
            (up_proj): Linear(in_features=128, out_features=128, bias=False)
            (down_proj): Linear(in_features=128, out_features=128, bias=False)
            (act_fn): SiLUActivation()
          )
        )
        (input_layernorm): GlmMoeDsaRMSNorm((128,), eps=1e-05)
        (post_attention_layernorm): GlmMoeDsaRMSNorm((128,), eps=1e-05)
      )
      (6): GlmMoeDsaDecoderLayer(
        (self_attn): GlmMoeDsaAttention(
          (q_a_proj): Linear(in_features=128, out_features=128, bias=False)
          (q_a_layernorm): GlmMoeDsaRMSNorm((128,), eps=1e-06)
          (q_b_proj): Linear(in_features=128, out_features=4096, bias=False)
          (kv_a_proj_with_mqa): Linear(in_features=128, out_features=288, bias=False)
          (kv_a_layernorm): GlmMoeDsaRMSNorm((256,), eps=1e-06)
          (kv_b_proj): Linear(in_features=256, out_features=18432, bias=False)
          (o_proj): Linear(in_features=16384, out_features=128, bias=False)
          (indexer): GlmMoeDsaIndexer(
            (wq_b): Linear(in_features=128, out_features=4096, bias=False)
            (wk): Linear(in_features=128, out_features=128, bias=False)
            (k_norm): LayerNorm((128,), eps=1e-06, elementwise_affine=True, bias=True)
            (weights_proj): Linear(in_features=128, out_features=32, bias=False)
          )
        )
        (mlp): GlmMoeDsaMoE(
          (experts): GlmMoeDsaExperts(
            (act_fn): SiLUActivation()
          )
          (gate): GlmMoeDsaTopkRouter()
          (shared_experts): GlmMoeDsaMLP(
            (gate_proj): Linear(in_features=128, out_features=128, bias=False)
            (up_proj): Linear(in_features=128, out_features=128, bias=False)
            (down_proj): Linear(in_features=128, out_features=128, bias=False)
            (act_fn): SiLUActivation()
          )
        )
        (input_layernorm): GlmMoeDsaRMSNorm((128,), eps=1e-05)
        (post_attention_layernorm): GlmMoeDsaRMSNorm((128,), eps=1e-05)
      )
      (7-9): 3 x GlmMoeDsaDecoderLayer(
        (self_attn): GlmMoeDsaAttention(
          (q_a_proj): Linear(in_features=128, out_features=128, bias=False)
          (q_a_layernorm): GlmMoeDsaRMSNorm((128,), eps=1e-06)
          (q_b_proj): Linear(in_features=128, out_features=4096, bias=False)
          (kv_a_proj_with_mqa): Linear(in_features=128, out_features=288, bias=False)
          (kv_a_layernorm): GlmMoeDsaRMSNorm((256,), eps=1e-06)
          (kv_b_proj): Linear(in_features=256, out_features=18432, bias=False)
          (o_proj): Linear(in_features=16384, out_features=128, bias=False)
        )
        (mlp): GlmMoeDsaMoE(
          (experts): GlmMoeDsaExperts(
            (act_fn): SiLUActivation()
          )
          (gate): GlmMoeDsaTopkRouter()
          (shared_experts): GlmMoeDsaMLP(
            (gate_proj): Linear(in_features=128, out_features=128, bias=False)
            (up_proj): Linear(in_features=128, out_features=128, bias=False)
            (down_proj): Linear(in_features=128, out_features=128, bias=False)
            (act_fn): SiLUActivation()
          )
        )
        (input_layernorm): GlmMoeDsaRMSNorm((128,), eps=1e-05)
        (post_attention_layernorm): GlmMoeDsaRMSNorm((128,), eps=1e-05)
      )
      (10): GlmMoeDsaDecoderLayer(
        (self_attn): GlmMoeDsaAttention(
          (q_a_proj): Linear(in_features=128, out_features=128, bias=False)
          (q_a_layernorm): GlmMoeDsaRMSNorm((128,), eps=1e-06)
          (q_b_proj): Linear(in_features=128, out_features=4096, bias=False)
          (kv_a_proj_with_mqa): Linear(in_features=128, out_features=288, bias=False)
          (kv_a_layernorm): GlmMoeDsaRMSNorm((256,), eps=1e-06)
          (kv_b_proj): Linear(in_features=256, out_features=18432, bias=False)
          (o_proj): Linear(in_features=16384, out_features=128, bias=False)
          (indexer): GlmMoeDsaIndexer(
            (wq_b): Linear(in_features=128, out_features=4096, bias=False)
            (wk): Linear(in_features=128, out_features=128, bias=False)
            (k_norm): LayerNorm((128,), eps=1e-06, elementwise_affine=True, bias=True)
            (weights_proj): Linear(in_features=128, out_features=32, bias=False)
          )
        )
        (mlp): GlmMoeDsaMoE(
          (experts): GlmMoeDsaExperts(
            (act_fn): SiLUActivation()
          )
          (gate): GlmMoeDsaTopkRouter()
          (shared_experts): GlmMoeDsaMLP(
            (gate_proj): Linear(in_features=128, out_features=128, bias=False)
            (up_proj): Linear(in_features=128, out_features=128, bias=False)
            (down_proj): Linear(in_features=128, out_features=128, bias=False)
            (act_fn): SiLUActivation()
          )
        )
        (input_layernorm): GlmMoeDsaRMSNorm((128,), eps=1e-05)
        (post_attention_layernorm): GlmMoeDsaRMSNorm((128,), eps=1e-05)
      )
    )
    (norm): GlmMoeDsaRMSNorm((128,), eps=1e-05)
    (rotary_emb): GlmMoeDsaRotaryEmbedding()
  )
  (lm_head): Linear(in_features=128, out_features=154880, bias=False)
)
```

</details>

### Test environment:

- accelerate: 1.13.0
- huggingface_hub: 2.3.0
- kernels: 0.17.2
- safetensors: 0.8.0
- tokenizers: 0.23.3
- torch: 2.13.0+cu126
- transformers: 5.20.2
- vllm: 0.1.dev1+g5a3eba034.d20260730.cu126
### Change log:
- 2026-10-24: Initial version.
This repo is generated by GPT-6.1-Sol

	---
	library_name: transformers
	base_model:
	- zai-org/GLM-5.3
	---

	This tiny model is intended for debugging. It is randomly initialized using the configuration adapted from [zai-org/GLM-5.3](https://huggingface.co/zai-org/GLM-5.3).

	Structural comparison with `zai-org/GLM-5.3`:

	\| Feature \| Source \| Tiny \|
	\|---\|---:\|---:\|
	\| Checkpoint tensor elements \| 753.33B \| 146.57M, including 104.21K FP32 scale elements \|
	\| Safetensors weight files \| 755.63 GB across shards \| 186.88 MB (`model.safetensors`) \|
	\| Architecture / input \| GLM-MoE-DSA; text \| Same \|
	\| Vocabulary rows / tied embeddings \| 154,880 / false \| Same \|
	\| Backbone layers \| 78 \| 11: three dense, then two complete shared/shared/shared/full cycles \|
	\| Hidden size \| 6,144 \| 128 \|
	\| Attention heads / NoPE / RoPE / value dimensions \| 64 / 192 / 64 / 256 \| 64 / 32 / 32 / 256 \|
	\| Query / KV LoRA rank \| 2,048 / 512 \| 128 / 256 \|
	\| Indexer heads / dimension / top-k \| 32 / 128 / 2,048 \| 32 / 128 / 128 \|
	\| Routed experts / selected experts / shared experts \| 256 / 8 / 1 \| 32 / 8 / 1 \|
	\| Expert intermediate size \| 2,048 \| 128 \|
	\| MTP \| One DSA/MoE layer with shared embedding/head \| Same \|
	\| Checkpoint precision \| BF16 plus E4M3 block-FP8 and FP32 scales \| Same; 32x32 blocks for reduced axes \|

	The indexer head remains 128-wide: vLLM's GLM DSA cache and fused-indexer path use a 128-wide cache layout. The source E4M3 format is retained with reduced 32x32 blocks; the TP=4 example selects vLLM's Triton block-FP8 GEMM because H20's automatic Humming kernel requires an output scale group of at least 64. With that unchanged 32-wide FP8 K block, vLLM TP=4 partitions residual-width linear inputs by four. Both 32-wide and 64-wide candidates were generated and rejected at model load with `input_size_per_partition = 8` and `16` respectively, neither divisible by `block_k = 32`; 128 is therefore the smallest executable residual width for this required checkpoint/backend/TP combination.
	Tokenizer and generation assets are copied unchanged from the pinned source revision.

	\| File path \| Size \|
	\|------\|------\|
	\| model.safetensors \| 186.9MB \|


	### Example usage:

	- vLLM (✅ verified)

	```bash
	vllm serve tiny-random/glm-5.3 \
	--tensor-parallel-size 4 \
	--speculative-config '{"method":"mtp","num_speculative_tokens":1}' \
	--max-model-len 4096 \
	--max-num-batched-tokens 512 \
	--max-num-seqs 4 \
	--gpu-memory-utilization 0.3 \
	--kernel-config '{"moe_backend":"triton","linear_backend":"triton"}' \
	--compilation-config '{"cudagraph_capture_sizes":[1,2,4,8]}'
	```

	- Transformers (✅ verified)

	```python
	import torch
	from transformers import AutoModelForCausalLM, AutoTokenizer
	from transformers.integrations.hub_kernels import allow_all_hub_kernels

	model_id = "tiny-random/glm-5.3"
	# The GLM-5.3 Transformers implementation supports the DSA sparse mask on
	# SDPA; eager attention is deliberately not selected.
	# Transformers resolves the official fine-grained FP8 kernel from the Hub.
	with allow_all_hub_kernels():
	model = AutoModelForCausalLM.from_pretrained(
	model_id, dtype=torch.bfloat16, attn_implementation="sdpa",
	).to("cuda")
	tokenizer = AutoTokenizer.from_pretrained(model_id)
	inputs = tokenizer.apply_chat_template(
	[{"role": "user", "content": "Hello"}], tokenize=True,
	add_generation_prompt=True, return_dict=True, return_tensors="pt",
	).to(model.device)
	output = model.generate(**inputs, do_sample=False, max_new_tokens=10)
	print(tokenizer.decode(output[0, inputs["input_ids"].shape[1]:], skip_special_tokens=True))
	```

	### Codes to create this repo (✅ verified):

	<details><summary>Click to expand</summary>

	```python
	"""Create the GLM-MoE-DSA checkpoint using the public FP8 checkpoint schema."""
	import json
	from pathlib import Path
	from shutil import copyfile

	import torch
	from huggingface_hub import hf_hub_download, list_repo_files
	from safetensors.torch import save_file
	from transformers import AutoConfig, AutoModelForCausalLM, set_seed

	source_model_id = "zai-org/GLM-5.3"
	save_folder = "/tmp/tiny-random/glm-53"
	source_revision = "ba32f8c19c8738e2c57c71798c601d477a7bf844"
	set_seed(42)
	folder = Path(save_folder)
	folder.mkdir(parents=True, exist_ok=True)
	raw = json.loads(Path(hf_hub_download(
	source_model_id, "config.json", revision=source_revision)).read_text())
	# The source uses block-FP8. 32x32 is the smallest source-compatible
	# block geometry. TP=4 therefore needs every sharded FP8 axis to be at
	# least 128 wide, so each rank owns one complete 32-wide block.
	raw["quantization_config"]["weight_block_size"] = [32, 32]
	raw.update(
	hidden_size=128,
	num_hidden_layers=11,
	# Retain the source's 64 global attention heads. The regular MLA
	# path shards these to 16 query heads per TP=4 rank; its 32-wide
	# reduced NoPE/RoPE projections and source-sized values keep every
	# backend-required head dimension valid.
	num_attention_heads=64,
	num_key_value_heads=64,
	intermediate_size=128,
	moe_intermediate_size=128,
	n_routed_experts=32,
	num_experts_per_tok=8,
	q_lora_rank=128,
	# Sparse-MQA passes this latent as FA3's q_v input. The optimized
	# CUDA kernel requires it to be at least 256 wide; source uses 512.
	kv_lora_rank=256,
	qk_nope_head_dim=32,
	qk_rope_head_dim=32,
	# vLLM's sparse-MQA FlashAttention path requires q_v >= 256; this
	# therefore retains the source value-head dimension.
	v_head_dim=256,
	index_head_dim=128,
	# This is the DSA indexer head count, distinct from normal attention.
	# vLLM's fused sparse-indexer kernel accepts 32 or 64; retain source 32.
	index_n_heads=32,
	index_topk=128,
	first_k_dense_replace=3,
	)
	# The original three shared-indexer layers followed by one full indexer
	# are retained twice after the dense prefix.
	raw["indexer_types"] = ["full", "full", "full"] + ["shared", "shared", "shared", "full"] * 2
	raw["mlp_layer_types"] = ["dense"] * 3 + ["sparse"] * 8

	# Build the Transformer graph from the reduced config before serializing
	# the source checkpoint layout.
	config_for_build = dict(raw)
	config_for_build.pop("quantization_config")
	(folder / "config.json").write_text(json.dumps(config_for_build, indent=2) + "\n")
	config = AutoConfig.from_pretrained(folder)
	model = AutoModelForCausalLM.from_config(config, dtype=torch.bfloat16)

	with torch.no_grad():
	for name, value in model.named_parameters():
	if name.endswith("e_score_correction_bias"):
	value.zero_()
	elif name.endswith("weight") and "layernorm" in name:
	value.fill_(1)
	else:
	value.normal_(mean=0.0, std=0.02)

	state = {name: value.detach().cpu().contiguous() for name, value in model.state_dict().items()}
	block_out, block_in = raw["quantization_config"]["weight_block_size"]

	def pack(name: str, weight: torch.Tensor) -> None:
	rows, columns = weight.shape
	assert rows % block_out == 0 and columns % block_in == 0, (name, weight.shape)
	blocks = weight.float().reshape(rows // block_out, block_out, columns // block_in, block_in)
	scales = (blocks.abs().amax(dim=(1, 3)) / 448).clamp_min(1e-12).to(torch.float32)
	state[name] = (blocks / scales[:, None, :, None]).clamp(-448, 448).reshape(weight.shape).to(
	torch.float8_e4m3fn
	).contiguous()
	state[name.removesuffix("weight") + "weight_scale_inv"] = scales.contiguous()

	def keep_bf16(name: str) -> bool:
	return (
	name in {"model.embed_tokens.weight", "lm_head.weight"}
	or ".layernorm." in name
	or ".indexer.k_norm." in name
	or name.endswith(".indexer.weights_proj.weight")
	or name.endswith(".mlp.gate.weight")
	or name.endswith(".mlp.gate.e_score_correction_bias")
	)

	# Source exclusion paths are tied to its 78-layer tree. Rebuild the list
	# for the reduced graph so Transformers does not turn BF16 norms, routers
	# or indexer projections into FP8 modules while loading this checkpoint.
	exclusions = {"model.embed_tokens", "lm_head"}
	for module_name, module in model.named_modules():
	if isinstance(module, torch.nn.Linear) and keep_bf16(module_name + ".weight"):
	exclusions.add(module_name)
	raw["quantization_config"]["modules_to_not_convert"] = sorted(exclusions)

	# The public checkpoint stores routed experts one expert per tensor; the
	# Transformers graph stores its expert matrices as compact three-dimensional
	# tensors. Convert only at serialization time.
	for name in list(state):
	if not name.endswith(".mlp.experts.gate_up_proj"):
	continue
	prefix = name.removesuffix(".gate_up_proj")
	gate_up = state.pop(name)
	down = state.pop(prefix + ".down_proj")
	for expert in range(gate_up.shape[0]):
	gate, up = gate_up[expert].chunk(2, dim=0)
	pack(f"{prefix}.{expert}.gate_proj.weight", gate)
	pack(f"{prefix}.{expert}.up_proj.weight", up)
	pack(f"{prefix}.{expert}.down_proj.weight", down[expert])

	for name in list(state):
	value = state[name]
	if not (name.endswith(".weight") and value.ndim == 2) or keep_bf16(name):
	continue
	pack(name, value)

	# GLM's embedded one-layer MTP head is represented as the source's extra
	# decoder layer. It shares the checkpoint's global embedding and LM head.
	base_prefix = "model.layers.10."
	mtp_prefix = "model.layers.11."
	for name, value in list(state.items()):
	if name.startswith(base_prefix):
	state[mtp_prefix + name.removeprefix(base_prefix)] = value.clone()
	state[mtp_prefix + "eh_proj.weight"] = torch.empty(128, 256, dtype=torch.bfloat16).normal_(0, 0.02)
	for name in ("enorm.weight", "hnorm.weight", "shared_head.norm.weight"):
	state[mtp_prefix + name] = torch.ones(128, dtype=torch.bfloat16)

	save_file(state, folder / "model.safetensors", metadata={"format": "pt"})
	(folder / "config.json").write_text(json.dumps(raw, indent=2) + "\n")

	assets = {
	"tokenizer.json", "tokenizer_config.json", "special_tokens_map.json",
	"chat_template.jinja", "generation_config.json", "LICENSE",
	}
	for name in sorted(set(list_repo_files(source_model_id, revision=source_revision)) & assets):
	copyfile(hf_hub_download(source_model_id, name, revision=source_revision), folder / name)
	```

	</details>

	### Printing the model:

	<details><summary>Click to expand</summary>

	```text
	GlmMoeDsaForCausalLM(
	(model): GlmMoeDsaModel(
	(embed_tokens): Embedding(154880, 128, padding_idx=154820)
	(layers): ModuleList(
	(0-2): 3 x GlmMoeDsaDecoderLayer(
	(self_attn): GlmMoeDsaAttention(
	(q_a_proj): Linear(in_features=128, out_features=128, bias=False)
	(q_a_layernorm): GlmMoeDsaRMSNorm((128,), eps=1e-06)
	(q_b_proj): Linear(in_features=128, out_features=4096, bias=False)
	(kv_a_proj_with_mqa): Linear(in_features=128, out_features=288, bias=False)
	(kv_a_layernorm): GlmMoeDsaRMSNorm((256,), eps=1e-06)
	(kv_b_proj): Linear(in_features=256, out_features=18432, bias=False)
	(o_proj): Linear(in_features=16384, out_features=128, bias=False)
	(indexer): GlmMoeDsaIndexer(
	(wq_b): Linear(in_features=128, out_features=4096, bias=False)
	(wk): Linear(in_features=128, out_features=128, bias=False)
	(k_norm): LayerNorm((128,), eps=1e-06, elementwise_affine=True, bias=True)
	(weights_proj): Linear(in_features=128, out_features=32, bias=False)
	)
	)
	(mlp): GlmMoeDsaMLP(
	(gate_proj): Linear(in_features=128, out_features=128, bias=False)
	(up_proj): Linear(in_features=128, out_features=128, bias=False)
	(down_proj): Linear(in_features=128, out_features=128, bias=False)
	(act_fn): SiLUActivation()
	)
	(input_layernorm): GlmMoeDsaRMSNorm((128,), eps=1e-05)
	(post_attention_layernorm): GlmMoeDsaRMSNorm((128,), eps=1e-05)
	)
	(3-5): 3 x GlmMoeDsaDecoderLayer(
	(self_attn): GlmMoeDsaAttention(
	(q_a_proj): Linear(in_features=128, out_features=128, bias=False)
	(q_a_layernorm): GlmMoeDsaRMSNorm((128,), eps=1e-06)
	(q_b_proj): Linear(in_features=128, out_features=4096, bias=False)
	(kv_a_proj_with_mqa): Linear(in_features=128, out_features=288, bias=False)
	(kv_a_layernorm): GlmMoeDsaRMSNorm((256,), eps=1e-06)
	(kv_b_proj): Linear(in_features=256, out_features=18432, bias=False)
	(o_proj): Linear(in_features=16384, out_features=128, bias=False)
	)
	(mlp): GlmMoeDsaMoE(
	(experts): GlmMoeDsaExperts(
	(act_fn): SiLUActivation()
	)
	(gate): GlmMoeDsaTopkRouter()
	(shared_experts): GlmMoeDsaMLP(
	(gate_proj): Linear(in_features=128, out_features=128, bias=False)
	(up_proj): Linear(in_features=128, out_features=128, bias=False)
	(down_proj): Linear(in_features=128, out_features=128, bias=False)
	(act_fn): SiLUActivation()
	)
	)
	(input_layernorm): GlmMoeDsaRMSNorm((128,), eps=1e-05)
	(post_attention_layernorm): GlmMoeDsaRMSNorm((128,), eps=1e-05)
	)
	(6): GlmMoeDsaDecoderLayer(
	(self_attn): GlmMoeDsaAttention(
	(q_a_proj): Linear(in_features=128, out_features=128, bias=False)
	(q_a_layernorm): GlmMoeDsaRMSNorm((128,), eps=1e-06)
	(q_b_proj): Linear(in_features=128, out_features=4096, bias=False)
	(kv_a_proj_with_mqa): Linear(in_features=128, out_features=288, bias=False)
	(kv_a_layernorm): GlmMoeDsaRMSNorm((256,), eps=1e-06)
	(kv_b_proj): Linear(in_features=256, out_features=18432, bias=False)
	(o_proj): Linear(in_features=16384, out_features=128, bias=False)
	(indexer): GlmMoeDsaIndexer(
	(wq_b): Linear(in_features=128, out_features=4096, bias=False)
	(wk): Linear(in_features=128, out_features=128, bias=False)
	(k_norm): LayerNorm((128,), eps=1e-06, elementwise_affine=True, bias=True)
	(weights_proj): Linear(in_features=128, out_features=32, bias=False)
	)
	)
	(mlp): GlmMoeDsaMoE(
	(experts): GlmMoeDsaExperts(
	(act_fn): SiLUActivation()
	)
	(gate): GlmMoeDsaTopkRouter()
	(shared_experts): GlmMoeDsaMLP(
	(gate_proj): Linear(in_features=128, out_features=128, bias=False)
	(up_proj): Linear(in_features=128, out_features=128, bias=False)
	(down_proj): Linear(in_features=128, out_features=128, bias=False)
	(act_fn): SiLUActivation()
	)
	)
	(input_layernorm): GlmMoeDsaRMSNorm((128,), eps=1e-05)
	(post_attention_layernorm): GlmMoeDsaRMSNorm((128,), eps=1e-05)
	)
	(7-9): 3 x GlmMoeDsaDecoderLayer(
	(self_attn): GlmMoeDsaAttention(
	(q_a_proj): Linear(in_features=128, out_features=128, bias=False)
	(q_a_layernorm): GlmMoeDsaRMSNorm((128,), eps=1e-06)
	(q_b_proj): Linear(in_features=128, out_features=4096, bias=False)
	(kv_a_proj_with_mqa): Linear(in_features=128, out_features=288, bias=False)
	(kv_a_layernorm): GlmMoeDsaRMSNorm((256,), eps=1e-06)
	(kv_b_proj): Linear(in_features=256, out_features=18432, bias=False)
	(o_proj): Linear(in_features=16384, out_features=128, bias=False)
	)
	(mlp): GlmMoeDsaMoE(
	(experts): GlmMoeDsaExperts(
	(act_fn): SiLUActivation()
	)
	(gate): GlmMoeDsaTopkRouter()
	(shared_experts): GlmMoeDsaMLP(
	(gate_proj): Linear(in_features=128, out_features=128, bias=False)
	(up_proj): Linear(in_features=128, out_features=128, bias=False)
	(down_proj): Linear(in_features=128, out_features=128, bias=False)
	(act_fn): SiLUActivation()
	)
	)
	(input_layernorm): GlmMoeDsaRMSNorm((128,), eps=1e-05)
	(post_attention_layernorm): GlmMoeDsaRMSNorm((128,), eps=1e-05)
	)
	(10): GlmMoeDsaDecoderLayer(
	(self_attn): GlmMoeDsaAttention(
	(q_a_proj): Linear(in_features=128, out_features=128, bias=False)
	(q_a_layernorm): GlmMoeDsaRMSNorm((128,), eps=1e-06)
	(q_b_proj): Linear(in_features=128, out_features=4096, bias=False)
	(kv_a_proj_with_mqa): Linear(in_features=128, out_features=288, bias=False)
	(kv_a_layernorm): GlmMoeDsaRMSNorm((256,), eps=1e-06)
	(kv_b_proj): Linear(in_features=256, out_features=18432, bias=False)
	(o_proj): Linear(in_features=16384, out_features=128, bias=False)
	(indexer): GlmMoeDsaIndexer(
	(wq_b): Linear(in_features=128, out_features=4096, bias=False)
	(wk): Linear(in_features=128, out_features=128, bias=False)
	(k_norm): LayerNorm((128,), eps=1e-06, elementwise_affine=True, bias=True)
	(weights_proj): Linear(in_features=128, out_features=32, bias=False)
	)
	)
	(mlp): GlmMoeDsaMoE(
	(experts): GlmMoeDsaExperts(
	(act_fn): SiLUActivation()
	)
	(gate): GlmMoeDsaTopkRouter()
	(shared_experts): GlmMoeDsaMLP(
	(gate_proj): Linear(in_features=128, out_features=128, bias=False)
	(up_proj): Linear(in_features=128, out_features=128, bias=False)
	(down_proj): Linear(in_features=128, out_features=128, bias=False)
	(act_fn): SiLUActivation()
	)
	)
	(input_layernorm): GlmMoeDsaRMSNorm((128,), eps=1e-05)
	(post_attention_layernorm): GlmMoeDsaRMSNorm((128,), eps=1e-05)
	)
	)
	(norm): GlmMoeDsaRMSNorm((128,), eps=1e-05)
	(rotary_emb): GlmMoeDsaRotaryEmbedding()
	)
	(lm_head): Linear(in_features=128, out_features=154880, bias=False)
	)
	```

	</details>

	### Test environment:

	- accelerate: 1.13.0
	- huggingface_hub: 2.3.0
	- kernels: 0.17.2
	- safetensors: 0.8.0
	- tokenizers: 0.23.3
	- torch: 2.13.0+cu126
	- transformers: 5.20.2
	- vllm: 0.1.dev1+g5a3eba034.d20260730.cu126

	### Change log:

	- 2026-10-24: Initial version.

	This repo is generated by GPT-6.1-Sol