README.md

---
license: mit
base_model: LibertAIDAI/GLM-5.3-Flash-NVFP4
library_name: transformers
pipeline_tag: image-text-to-text
tags:
  - glm5-next
  - glm-5.3-flash
  - tiny-random
  - nvfp4
  - vllm
  - gb10
---

# GLM-5.3 Flash tiny-random NVFP4 fixture

This is a deterministic, randomly initialized test checkpoint derived from the
configuration and tensor schema of
[`LibertAIDAI/GLM-5.3-Flash-NVFP4`](https://huggingface.co/LibertAIDAI/GLM-5.3-Flash-NVFP4),
pinned at revision `9e0d74e3cef17f634e84fb8e2223707e02616290`.

It is intended to reproduce vLLM loader, cache-layout, sparse-attention, NVFP4
MoE, and MTP compatibility problems on GB10/SM121 without loading the full
checkpoint. It is **not** useful for language or vision tasks, and its output is
deliberately meaningless.

The companion GB10 runtime and source patch live at
[`cyijun/glm-5.3-flash-nvfp4-gb10`](https://github.com/cyijun/glm-5.3-flash-nvfp4-gb10).

## What remains architecture-faithful

The fixture keeps dimensions that select execution branches, kernels, packed
weight layouts, or cache ABIs:

| Contract | Fixture and target value |
| --- | ---: |
| layer pattern | 3 KDA + 1 sparse MLA |
| attention / KV heads | 64 / 64 |
| Q LoRA / KV LoRA rank | 1536 / 512 |
| QK NoPE / RoPE dimension | 256 / 0 |
| V head dimension | 256 |
| KDA heads / head dimension / convolution | 64 / 128 / 4 |
| index heads / head dimension | 32 / 128 |
| index top-k / k-pool | 2048 / 4 |
| routed / selected / shared experts | 288 / 8 / 1 |
| MoE intermediate size | 2048 |
| dense MLP intermediate size | 12288 |
| dense prefix | first 3 layers |
| MTP layers | 1 |
| vocabulary and tokenizer | 154880 |

The routed-expert tensors use the same ModelOpt NVFP4 storage ABI as the source
checkpoint: packed `U8` weights, `F8_E4M3` per-block scales, and scalar `F32`
global scales. All 1,728 expert projections are present.

To keep the artifact under 1 GiB, `hidden_size` is reduced from 4096 to 256,
the 45-layer stack is reduced to its first four-layer cycle, maximum positions
are reduced to 8192, and the vision tower is reduced to one small layer. The
four base layers plus one copied MTP layer still cover KDA, dense MLP, sparse
MLA, routed NVFP4 MoE, shared experts, and speculative decoding.

## What the fixture exposed on GB10

The pinned base image
`vllm/vllm-openai:glm53-flash-arm64-cu130@sha256:905c02933be6021301db2dc284e24e3727467aa3a0f63b41d609885778a07bce`
loads the checkpoint but needs three compatibility changes:

1. GLM-5.3 is logically NoPE (`qk_rope_head_dim=0`), while the SM120
   `fp8_ds_mla` cache writer requires a physical 64-BF16 positional region.
   The adapter writes zeros into that region and pads the absorbed 512-wide
   query to 576. This preserves the NoPE dot product.
2. `index_topk=2048`, `index_kpool=4`, the always-selected tail, and
   `BLOCK_N=128` alignment produce a physical sparse-index buffer of 2176
   columns. The bundled FlashInfer AOT module only instantiates 2048. The
   adapter adds the exact `num_heads=64, topk=2176` decode specialization.
3. Requests with more than 64 query tokens use the prefill orchestrator, so the
   matching `GLM_NSA, num_heads=64, topk=2176` prefill specialization is also
   required. The patched module is rebuilt for `sm_121a`; editing Python or JIT
   source alone does not override the precompiled AOT `.so` in the base image.

The physical sparse-MLA row remains the native GLM_NSA 656-byte layout:

```text
query = [absorbed NoPE 512 | zero padding 64]
KV    = [FP8 latent 512 | four FP32 scales | 64 BF16 zeros]
```

The vLLM CLI still requires `--block-size 256` because the GLM index cache must
be divisible by `index_kpool * 32`. Internally the compressed FlashInfer sparse
cache uses 64-token physical pages; these are different levels of the cache
layout and should not be conflated.

## Run with the GB10 image

```bash
docker run --rm --gpus all --ipc=host -p 8000:8000 \
  ghcr.io/cyijun/glm-5.3-flash-nvfp4-gb10:vllm-glm53-sm121 \
  cyijun2k/glm-5.3-flash-tiny-random-nvfp4 \
  --served-model-name glm53-tiny \
  --max-model-len 512 \
  --max-num-seqs 2 \
  --gpu-memory-utilization 0.10 \
  --block-size 256 \
  --moe-backend marlin \
  --enforce-eager
```

Optional one-token MTP smoke test:

```text
--speculative-config '{"method":"mtp","num_speculative_tokens":1}'
```

## Validation evidence

Validation was performed on an NVIDIA GB10 (SM121) with vLLM
`0.1.dev20051+g487ecf187`, FlashInfer `0.6.17`, Transformers `5.15.1`, and
PyTorch `2.13.0+cu130`.

| Check | Result |
| --- | --- |
| critical configuration fields against pinned source revision | 21/21 match |
| fixture tensor names with an analogous source tensor | 5342/5342 |
| NVFP4 weight/scale/global-scale dtypes | `U8` / `F8_E4M3` / `F32` |
| model load and `/health` | pass |
| OpenAI-compatible short chat + continuous decode | pass |
| 491-token prefill + 16-token decode | pass |
| MTP with one speculative token | pass |
| public companion GHCR manifest | `sha256:4251b561d111d817765ed4097512ce36811deac071a4a7411d20242df5c74a47` |

See `VALIDATION.json` for the exact image digest and machine-readable results.

## Integrity

`model.safetensors` is 1,010,842,392 bytes and contains 5,342 tensors.

```text
sha256 d0f8a670c6d96be8984db205a4bfb32c947f09182d6b327fab8d4f1088b66810
```

The checkpoint is deterministic with model seed 42 and expert payload seed
4242. `BUILD_INFO.json` records the source revision and fixture profile.