Instructions to use patrickbdevaney/GLM-5.3-Flash-REAP50-FP8 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use patrickbdevaney/GLM-5.3-Flash-REAP50-FP8 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="patrickbdevaney/GLM-5.3-Flash-REAP50-FP8")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("patrickbdevaney/GLM-5.3-Flash-REAP50-FP8")
model = AutoModelForMultimodalLM.from_pretrained("patrickbdevaney/GLM-5.3-Flash-REAP50-FP8", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use patrickbdevaney/GLM-5.3-Flash-REAP50-FP8 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "patrickbdevaney/GLM-5.3-Flash-REAP50-FP8"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "patrickbdevaney/GLM-5.3-Flash-REAP50-FP8",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/patrickbdevaney/GLM-5.3-Flash-REAP50-FP8

SGLang

How to use patrickbdevaney/GLM-5.3-Flash-REAP50-FP8 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "patrickbdevaney/GLM-5.3-Flash-REAP50-FP8" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "patrickbdevaney/GLM-5.3-Flash-REAP50-FP8",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "patrickbdevaney/GLM-5.3-Flash-REAP50-FP8" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "patrickbdevaney/GLM-5.3-Flash-REAP50-FP8",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use patrickbdevaney/GLM-5.3-Flash-REAP50-FP8 with Docker Model Runner:
```
docker model run hf.co/patrickbdevaney/GLM-5.3-Flash-REAP50-FP8
```

A newer version of this model is available: patrickbdevaney/GLM-5.3-Flash-REAP50-FP8-v2

GLM-5.3-Flash REAP-50 (FP8 — Pass-1 Archive)

Pass-1 (v1) 50% REAP expert-pruned FP8 (157.0 GiB) checkpoint of zai-org/GLM-5.3-Flash — superseded by GLM-5.3-Flash-REAP50-FP8-v2.

Newer Version Available (v2): This is the initial Pass-1 (v1) single-pool REAP-50 checkpoint, retained for archival reproducibility. For all new deployments, please use patrickbdevaney/GLM-5.3-Flash-REAP50-FP8-v2, which adds a 5-domain balanced keep-set (general, code, math, zh, tool), union top-16 specialist protection, a frequency floor (f_j >= 40), and preserves the upstream Layer-78 Multi-Token Prediction (MTP) speculative decoding head (num_nextn_predict_layers = 1).

50% expert-pruned derivative of zai-org/GLM-5.3-Flash produced with REAP (Router-weighted Expert Activation Pruning; arXiv:2510.13999) on a single NVIDIA Thor (128 GB unified memory).

Need to run on a single 128 GB device (NVIDIA Thor / DGX Spark / Strix Halo)? See the companion NVFP4 repository patrickbdevaney/GLM-5.3-Flash-REAP50-NVFP4-v2 (~94.6 GiB) or GGUF repository patrickbdevaney/GLM-5.3-Flash-REAP50-GGUF (72.9–94.8 GiB).

Summary

Property	Original (`zai-org/GLM-5.3-Flash`)	This model (`REAP-50`)
Total parameters	~301 B	~157 B
Active parameters / token	~12.5 B	~12.5 B (unchanged)
Layers	78 (3 dense + 75 MoE)	78 (3 dense + 75 MoE)
Routed experts / MoE layer	256	128 (50% pruned)
Active experts / token (`num_experts_per_tok`)	8	8
Shared expert / MoE layer	1	1 (untouched)
Attention	MLA + DSA (`deepseek_v32`, `kv_lora_rank=512`, `q_lora_rank=2048`, `index_topk=2048`)	Untouched
Context length	202,752	202,752
Size	296 GiB (FP8)	157.0 GiB (FP8)
License	MIT	MIT

Pruning methodology

Each MoE layer l in [3..77] has 256 routed SwiGLU experts (gate_proj, up_proj, down_proj, intermediate size 1536) routed by a sigmoid gate (routed_scaling_factor = 2.5, norm_topk_prob = true, e_score_correction_bias).

Calibration corpus: 320 sequences x 2048 tokens (655,360 tokens) drawn from fineweb-edu, Wikipedia, Python/C++/Rust code, GSM8K/math, reasoning traces, and Chinese/English prompts (calibration_tokens.pt, SHA-256 recorded in reap_plan.json).
Exact teacher forward: Layers 0..77 were executed sequentially on GPU in native bfloat16 using the exact deepseek_v32 MLA + rotary + lightning-indexer attention implementation and exact block-scale FP8 (F8_E4M3 x F8_E8M0 UE8M0) dequantization (saliency_pass.py).
REAP saliency metric (Equation 6 of arXiv:2510.13999): For each routed expert j in layer l, over the set T_j of calibration tokens where expert j is in the router's top-8:
```
S_j = (1 / |T_j|) * sum_{t in T_j} g_{t,j} * || down_proj_j( silu(gate_proj_j(h_t)) * up_proj_j(h_t) ) ||_2
```
where g_{t,j} is the normalized top-8 routing weight (prior to routed_scaling_factor, which is a constant multiplier across experts).
Bit-exact expert selection & weight remapping: In each MoE layer, the 128 experts with highest S_j are retained in ascending original-index order kept = [k_0 < k_1 < ... < k_127]:
- mlp.gate.weight ([256, 5120] -> [128, 5120]) and mlp.gate.e_score_correction_bias ([256] -> [128]) are sliced to kept.
- Routed experts mlp.experts.{kept[new_idx]}.* are renamed to mlp.experts.{new_idx}.* (0..127) with zero modification to their F8_E4M3 weights or F8_E8M0 weight_scale_inv blocks.
- All attention projections, DSA indexer weights, dense layers (0..2), shared experts, norms, embeddings, and lm_head are copied bit-for-bit from zai-org/GLM-5.3-Flash.
- The training-only MTP head (model.layers.78, num_nextn_predict_layers) is omitted (num_nextn_predict_layers = 0).

Full per-layer keep/prune metadata and first-moment gains are included in ./reap_metadata.json and ./adapters/first_moment_gains.json, with the full pipeline source in patrickbdevaney/glm-5.3-reap.

Quantization format

Identical to zai-org/GLM-5.3-Flash: fine-grained FP8 (fp8 block [1, 128] with F8_E4M3 weights and F8_E8M0 weight_scale_inv), ready for drop-in serving with vLLM or SGLang (deepseek_v32 / GlmMoeDsaForCausalLM).

Usage (vLLM / SGLang)

# vLLM (2x 80GB+ GPUs, or 1x 192GB B200)
vllm serve patrickbdevaney/GLM-5.3-Flash-REAP50-FP8 \
  --tensor-parallel-size 2 \
  --trust-remote-code \
  --max-model-len 65536

# SGLang
python3 -m sglang.launch_server \
  --model-path patrickbdevaney/GLM-5.3-Flash-REAP50-FP8 \
  --tp 2 \
  --trust-remote-code

🔎 Provenance & Verification Ledger

Claim / Artifact	Primary Verification Source
Upstream Base Model (`301B` FP8, MIT)	`zai-org/GLM-5.3-Flash` · `./LICENSE`
Superseded By (v2 Domain-Balanced + MTP Layer 78)	`patrickbdevaney/GLM-5.3-Flash-REAP50-FP8-v2`
In-Repo REAP Metadata & First-Moment Gains	`./reap_metadata.json` · `./adapters/first_moment_gains.json`
Model Config & 62-Shard Safetensors Index	`./config.json` · `./model.safetensors.index.json` · `./chat_template.jinja`
REAP-50 Pipeline & Integrity Audit	`patrickbdevaney/glm-5.3-reap` (`main` @ `8540a0a`) · `EVAL_INTEGRITY.md`
Pure-CUDA Thor Engine (`sm_110a`)	`patrickbdevaney/glm-5.3-flash-cuda-server` (`main` @ `399a837`)

Attribution & License

Base model: zai-org/GLM-5.3-Flash by Z.ai (MIT License, ./LICENSE).
Pruning method: REAP the Experts: Why Pruning Prevails for One-Shot MoE Compression (Lasby et al., 2025, arXiv:2510.13999).

Downloads last month: 68

Safetensors

Model size

162B params

Tensor type

BF16

F32

F8_E4M3

Model tree for patrickbdevaney/GLM-5.3-Flash-REAP50-FP8

Base model

zai-org/GLM-5.3-Flash

Quantized

(219)

this model

Paper for patrickbdevaney/GLM-5.3-Flash-REAP50-FP8

REAP the Experts: Why Pruning Prevails for One-Shot MoE compression

Paper • 2510.13999 • Published Oct 15, 2025 • 23