---
license: mit
base_model: zai-org/GLM-5.3-Flash
tags: [reap, moe, pruning, glm5_next, jetson, thor]
---

# GLM-5.3-Flash REAP-50 (FP8)

> ## Known issue — carries a superseded healing scalar (2026-08-28)
>
> This checkpoint was derived from the FP8 master **before** its healing gain was re-fit, so it
> inherits a systematic under-scaling of the MoE pathway: a first-moment gain of median
> **0.6933** where the measured value is **0.9111**, i.e. every retained expert's output
> is scaled ~0.76x too small relative to attention, the shared experts and the residual stream.
>
> The cause: the first-moment estimator ignores that `norm_topk_prob` renormalises the surviving
> top-8, so the router already returns the pruned gate mass by itself (measured: 2.5000 before,
> 2.5000 after).
>
> The FP8 master [`patrickbdevaney/GLM-5.3-Flash-REAP50-FP8`](https://huggingface.co/patrickbdevaney/GLM-5.3-Flash-REAP50-FP8)
> **has been corrected**; prefer it. Unlike the FP8 fix, this one cannot be applied by scaling
> block scales — the correction predates the FP4 packing — so it requires re-deriving from the
> corrected master. A fully re-run, corrected NVFP4 is in progress.


50% of routed experts removed with **REAP**
(Router-weighted Expert Activation Pruning, arXiv:2510.13999), calibrated on a
permissively-licensed multi-domain corpus that includes real image-text pairs.

| | |
|---|---|
| Base | `zai-org/GLM-5.3-Flash` (MIT, FP8 E4M3, 128x128 block scales) |
| Experts | 288 -> 144 per layer, top-8 routing unchanged |
| Size | 157.0 GiB (FP8) |
| Healed | yes |
| MTP block | excluded (see below) |

## Why FP8 and not BF16

The upstream release is **FP8**, not BF16. Routed experts are stored per-expert with their own
`weight_scale_inv` block scales, so pruning is deleting whole tensors - **lossless on every
retained weight**. The 642 GB BF16 repo elsewhere on the Hub is a dequantised upcast carrying
no additional information.

## Calibration

Mixture weighted for a coding/agentic model that stays empirically grounded: agentic 24%,
code 21%, math 15%, multimodal 15%, science+bio 10%, finance 8%, ballast 7%.
Permissive licences only, so this checkpoint keeps the base model's MIT lineage.

Vision is first-class: the vision tower contains no MoE and is untouched, but image tokens
route through the same expert pool as text, so text-only calibration would have deleted
vision-serving experts with certainty. Real image-text pairs were asserted present.

## Evaluation status: NONE

**This checkpoint has not been evaluated.** No benchmark has been run against it - not coding,
not agentic, not vision, not knowledge. What has been verified is *structural*: expert counts
match the config, routers are sliced to the retained set, every tensor loads, the vision tower
is untouched, and the MTP block is cleanly absent.

The pruning itself measured **1.29x better than random** at retaining expert output
contribution (saliency mass 0.643 against 0.50 for random pruning at the same ratio). That says
the criterion selected well. It does **not** say the model is good.

Treat this as a research artifact pending evaluation, not a drop-in replacement.

## Known limitations

- **The MTP (multi-token-prediction) block at layer 45 is excluded.** `transformers`'
  `Glm5NextForConditionalGeneration` does not instantiate it, so the pruning path cannot see
  it. Its original tensors are archived unmodified rather than inconsistently pruned.
- REAP has no published data above 50% compression; this checkpoint sits at the validated
  ceiling, not beyond it.
- Expect **factual-recall** regression before reasoning or coding regression. That is the
  measured failure mode of expert pruning on this architecture family: the closest published
  analogue (`cerebras/Kimi-Linear-REAP-35B-A3B`, same KDA + full-attention stack) loses 3.4
  points on FRAMES at only 30% pruning while code and maths hold flat.
- Healing is a **first-moment output-scale correction** derived from the calibration saliency
  (median gain 0.696, applied exactly to the F32 block scales). It is *not* distillation and
  does not attempt to recover lost knowledge.
- Routing is disrupted more than expert count suggests: the retained experts carry ~0.90x the
  routing mass an average expert would, because REAP preserves rare-but-strong experts over
  common-but-weak ones.

## Serving on Jetson Thor

Use the **cutlass** fused-MoE backend (the Marlin FP4 MoE kernel faults at >=256 experts) and
`TRITON_MLA` for the 11 MLA+DSA layers (FLASHINFER is invalid for MLA).