GLM-5.3-DFlash2

This repository releases a DFlash2 draft model for GLM-5.3. It is a speculator, not a standalone LLM: pair it with a GLM-5.3 target (BF16 or FP8) in SGLang or vLLM.

• Draft model: AlayaNeW/GLM-5.3-DFlash2
• Target model: zai-org/GLM-5.3 / zai-org/GLM-5.3-FP8
• Algorithm: DFlash2 (parallel block draft + path selector + dynamic convolution)

Architecture

Field	Value
Architecture	DFlash2DraftModel
Draft layers	5 (4× sliding attention + 1× full attention)
Hidden size / head dim	6144 / 128
Attention	GQA, 64 Q heads, 16 KV heads
Sliding window	2048
Block size $\gamma$	12
Target layers (KV inject)	[1, 20, 38, 56, 75] (GLM-5.3 has 78 layers)
Selector	rank 256, top-k 16
Dynamic conv	kernel 2, group 16
Anchors	128
Vocab	154880 (same as GLM-5.3)
Precision	bfloat16

The draft is trained with block size 12. The commands and tables below propose 8 draft tokens per verification step.

Quick Start

Requires a recent SGLang or vLLM build with native DFlash support. This draft is for GLM-5.3 only.

Serve with SGLang

export SGLANG_ENABLE_SPEC_V2=1

sglang serve \
  --model-path zai-org/GLM-5.3-FP8 \
  --trust-remote-code \
  --tp-size 8 \
  --reasoning-parser glm45 \
  --tool-call-parser glm47 \
  --context-length 202752 \
  --mem-fraction-static 0.83 \
  --cuda-graph-max-bs 256 \
  --max-running-requests 128 \
  --disable-shared-experts-fusion \
  --host 0.0.0.0 \
  --port 8100 \
  --speculative-algorithm DFLASH \
  --speculative-draft-model-path AlayaNeW/GLM-5.3-DFlash2 \
  --speculative-num-draft-tokens 8

Serve with vLLM

export VLLM_USE_V2_MODEL_RUNNER=1

vllm serve zai-org/GLM-5.3-FP8 \
  --trust-remote-code \
  --kv-cache-dtype fp8 \
  --tensor-parallel-size 8 \
  --enable-expert-parallel \
  --tool-call-parser glm47 \
  --enable-auto-tool-choice \
  --reasoning-parser glm45 \
  --enable-prefix-caching \
  --max-model-len 202752 \
  --max-num-seqs 256 \
  --gpu-memory-utilization 0.90 \
  --max-cudagraph-capture-size 256 \
  --host 0.0.0.0 \
  --port 8100 \
  --served-model-name GLM-5.3 \
  --speculative-config '{
    "method": "dflash",
    "model": "AlayaNeW/GLM-5.3-DFlash2",
    "num_speculative_tokens": 8,
    "kv_cache_dtype": "auto"
  }'

The draft is bf16 dense. Keep the draft KV cache dtype on auto so it does not inherit the target's FP8 MLA KV cache.

Call the server

from openai import OpenAI

client = OpenAI(base_url="http://localhost:8100/v1", api_key="EMPTY")
resp = client.chat.completions.create(
    model="GLM-5.3",
    messages=[{"role": "user", "content": "Write a Python function that checks if a number is prime."}],
    max_tokens=1024,
    extra_body={"chat_template_kwargs": {"enable_thinking": False}},
)
print(resp.choices[0].message.content)

Evaluation

• Runtime: vLLM on 8× NVIDIA H200 (TP8)
• Target: zai-org/GLM-5.3-FP8
• Speculation: DFlash, num_speculative_tokens 8
• Sampling: temperature 0.7, concurrency 10, max generation 4096
• Context-length numbers are the mean of 3 runs (50 prompts, SPEED-Bench low-entropy throughput)
• Qualitative numbers are one run on SPEED-Bench categories (80 prompts each)

Acceptance length is the per-request mean of completion tokens divided by verification steps. Higher is better. Per-position rates are percent.

Speed-Bench: Low Entropy

Context Len	AL	Pos 0	Pos 1	Pos 2	Pos 3	Pos 4	Pos 5	Pos 6	Pos 7
1k	4.74	80.58	65.03	53.27	44.89	38.83	34.07	30.25	27.25
2k	4.97	82.11	67.65	56.82	48.60	42.22	37.14	32.86	29.41
8k	5.28	85.12	72.71	62.94	54.95	47.83	41.01	34.67	29.11
16k	5.50	86.37	74.74	65.64	57.86	50.89	44.25	37.96	32.26
32k	5.39	86.08	74.49	65.02	56.89	49.43	42.24	35.56	29.51

Speed-Bench: Qualitative

Category	AL	Pos 0	Pos 1	Pos 2	Pos 3	Pos 4	Pos 5	Pos 6	Pos 7
coding	4.49	80.97	64.24	51.59	41.88	34.64	29.17	24.95	21.58
stem	3.47	74.89	53.88	38.50	27.67	19.94	14.45	10.46	7.58
humanities	3.33	73.24	51.32	35.87	25.47	18.07	12.70	9.32	6.71
math	3.52	75.03	54.23	38.95	28.03	20.57	15.12	11.27	8.55
multilingual	4.76	81.01	65.52	54.15	45.53	38.90	34.02	30.11	26.66
qa	3.40	73.74	52.05	37.02	26.17	18.88	13.89	10.32	7.75
rag	4.50	79.92	63.83	51.72	42.44	35.42	29.84	25.32	21.41
roleplay	2.73	65.58	39.91	24.41	15.64	10.39	7.41	5.52	4.27
reasoning	3.59	73.14	52.50	38.49	28.88	22.39	17.67	14.32	11.58
summarization	4.82	84.17	69.65	57.15	47.81	39.46	32.91	27.60	23.57
writing	2.95	67.50	43.58	28.46	19.28	13.62	9.86	7.33	5.62

Citation

@misc{glm53dflash22026,
  title={GLM-5.3-DFlash2},
  year={2026},
  howpublished={\url{https://huggingface.co/AlayaNeW/GLM-5.3-DFlash2}}
}