sionic-ai/GLM-5.3-ultrafast-Draft

GLM-5.3 ultrafast Draft

Sionic AI's DFlash 2 draft model for accelerating GLM-5.3 across Korean conversations, coding, mathematics, and agent tasks.

Trained on responses generated by GLM-5.3 itself and evaluated in a live SGLang serving environment, the latest v2 checkpoint brings together 1.69 million training examples and 2.46 billion parameters for Korean and multi-domain speculative decoding.

Highlights

• 3.573 mean acceptance length on Korean queries — an average of 3.57 output tokens per verification step in our Korean evaluation suite.
• 305.3 output tokens/s median across the full suite — measured per request across Korean, code, math, chat, and agent queries.
• 4.009 acceptance length for code, 4.628 for math, and 3.614 for agent tasks — multiple output tokens processed in each verification step across practical workloads.
• Evaluated with inputs up to 235,398 tokens — a mean acceptance length of 3.013 across eight long-context queries.
• A complete training-to-serving pipeline — 25,669 additional optimizer steps, a BF16 serving export, and evaluation through SGLang.

Serving Results

Results below come from our internal evaluation on September 19, 2026. Mean acceptance length is the arithmetic mean of each request's completion_tokens / spec_verify_ct, including output tokens produced during verification.

Domain	Queries	Mean acceptance length	Median output tokens/s
Korean	4	3.573	305.3
Code	4	4.009	308.0
Math	3	4.628	362.7
Agent	2	3.614	314.4
Chat	3	2.288	201.3
Overall	16	3.644	305.3

Evaluation setup: eight NVIDIA B300 GPUs, GLM-5.3 FP8 target, TP8, SGLang DFLASH, FA4 draft attention, block size 8, concurrency 1, temperature 0, top-p 0.95, Low reasoning effort, and up to 1,024 output tokens. At temperature 0.6, two runs of the same 16 queries achieved a mean acceptance length of 3.395.

Long-context evaluation used a separate fixed set of eight queries, two per context bucket, with temperature 0 and up to 512 output tokens. Reasoning effort followed each query's source-data setting.

Input context	Mean acceptance length
8K–32K	3.173
32K–64K	2.463
64K–128K	3.103
128K–256K	3.313

These measurements describe a small internal query suite and vary with hardware, serving configuration, output length, and traffic. Acceptance length measures decoding efficiency; it does not represent answer accuracy or a speedup multiplier. Original measurement records are included in evaluation/.

Model and Training

DFlash 2 proposes multiple tokens in parallel for verification by the target model. This draft conditions on GLM-5.3 hidden states and runs alongside GLM-5.3 in a speculative decoding server.

Specification	Configuration
Target model	zai-org/GLM-5.3
Release	v2 · epoch_1_step_25669
Draft	6 layers · 2,459,424,256 parameters · BF16
Hidden / FFN size	6,144 / 12,288
Attention	64 query heads · 8 KV heads · head dimension 128
Target hidden-state taps	5, 19, 33, 47, 61, 75
Block size / sliding window	8 / 2,048
Candidate selector	rank 256 · top-k 16
Training data	1,686,121 examples · self-distillation from GLM-5.3 responses
Continued training	1 epoch from v1d weights · 25,669 optimizer steps
Training sequence length / anchors	4,096 / 512
Optimizer	AdamW · peak LR 3e-4 · 3% warmup · cosine schedule
Training stack	Custom NeMo Automodel trainer · FSDP2 · EP8 · DeepEP

The training corpus combines 1,114,858 existing examples with 571,263 newly generated responses. For v2, draft weights were restored from v1d while the optimizer and scheduler were initialized afresh. Training completed on September 19, 2026.

Usage

Use a SGLang build with DFlash 2 support and the GLM-5.3 target weights. Authenticate with a Hugging Face account that has access to this private organization repository, then download the draft.

hf auth login
hf download sionic-ai/GLM-5.3-ultrafast-Draft \
  --local-dir ./GLM-5.3-ultrafast-Draft

python -m sglang.launch_server \
  --model-path /path/to/GLM-5.3 \
  --served-model-name glm-5.3-ultrafast \
  --tp-size 8 \
  --speculative-algorithm DFLASH \
  --speculative-draft-model-path ./GLM-5.3-ultrafast-Draft \
  --speculative-draft-attention-backend fa4 \
  --reasoning-parser glm45 \
  --tool-call-parser glm47 \
  --sampling-defaults model \
  --trust-remote-code \
  --host 0.0.0.0 --port 30000

Set the target path to GLM-5.3. Compatibility with GLM-5.3-Flash has not been evaluated for this release. FA4 requires a compatible GPU and runtime.

Repository Contents

model.safetensors contains the trained draft weights. GLM-5.3 target weights, embeddings, and the LM head are not included. config.json, tokenizer.json, and tokenizer_config.json provide the serving configuration. provenance.json records the checkpoint identifier and the weights' SHA-256 checksum.

License and Commercial Use

The Sionic AI draft weights in model.safetensors and Sionic AI-authored documentation are licensed under Apache License 2.0, permitting commercial use, modification, and redistribution subject to the license terms. See LICENSE for the full license and LICENSE.md for its scope.

The GLM-5.3 target model and accompanying tokenizer retain their original GLM-5.3 license, reproduced in LICENSE.glm53. Apache 2.0 does not replace those upstream terms. Serving and training software retain their respective upstream licenses.

Acknowledgements

Our training and serving pipeline builds on the research and implementations from Z.ai GLM-5.3, DFlash, NeMo Automodel, and SGLang.