README.md

---
base_model:
  - zai-org/GLM-5.3-Flash
tags:
  - glm5_next
  - fixture
  - testing
pipeline_tag: text-generation
---

# GLM-5.3-Flash tiny architecture fixture (glm5_next)

A ~0.27M-parameter random-init TEXT checkpoint plus a reduced, nested config that
reproduces the **GLM-5.3-Flash** `glm5_next` schema, so loaders, quant planners,
and CI jobs can exercise the new multimodal-MoE config parsing, expert-table
sizing, and safetensors load path without the real weights.




## What this is
- Base architecture: `zai-org/GLM-5.3-Flash` (released 2026-08-25),
  `model_type: glm5_next`, `architectures: ["Glm5NextForConditionalGeneration"]`.
  The base is a large multimodal MoE that cannot be instantiated in a unit test,
  in CI, or on a laptop.
- What this fixture is: a byte-reproducible **random-init** text checkpoint plus a
  reduced config that keeps the real top-level wrapper fields (`model_type`,
  `architectures`, `image_token_id`, `language_model_only`, `text_config`,
  `vision_config`) and the GLM MoE field names inside `text_config`, including the
  dense-to-MoE layer schedule (`first_k_dense_replace`, routed + shared experts).
- What it is **not**: not trained, not distilled, not a quality or benchmark claim,
  and not a quantization of anything.
- Why it is useful: it lets you test config parsing, weight-name mapping,
  expert-table sizing, router/top-k bookkeeping, and safetensors load paths in
  milliseconds, and it exercises the `glm5_next` wrapper shape that the plain
  `glm_moe_dsa` text fixture does not.

## Fixture geometry (what this checkpoint actually contains)
| Field | Value |
|---|---|
| num_hidden_layers | 4 |
| hidden_size | 64 |
| num_attention_heads / num_key_value_heads | 4 / 4 |
| head_dim | 16 |
| n_routed_experts | 8 |
| num_experts_per_tok | 2 |
| n_shared_experts | 1 |
| n_group | 1 |
| first_k_dense_replace | 1 |
| moe_intermediate_size | 32 |
| intermediate_size (dense) | 128 |
| num_nextn_predict_layers | 0 |
| dtype | float32 |
| vocab_size | 256 |

Total: 113 tensors, 1,097,984 data bytes = 274,496 float32 parameters.

## Intentional omissions (documented, not silent)
- **Text-only.** No vision encoder or projector tensors; `vision_config` is a
  placeholder. A full multimodal loader must supply vision/projector tensors and
  remap names.
- No MTP / next-n-predictor head (`num_nextn_predict_layers: 0`).
- No `lm_head` tensor; a loader must tie to `model.embed_tokens.weight` or supply
  its own head.
- Tokenizer metadata files are placeholders (no vocab file); use your own tokenizer.
- Weight names are a reduced text-only convention; they are not guaranteed to match
  every tensor a full `Glm5NextForConditionalGeneration` loader expects.

## Verification actually performed (stdlib only, no torch in this environment)
- safetensors header parses: 113 tensors, contiguous `data_offsets`, header padded
  to 8-byte alignment; `__metadata__` records the generator string.
- `checksums.txt` records the SHA-256 of every tensor blob.
- Deterministic regeneration: SplitMix64 seed 20260902, Box-Muller normals, scale
  0.02, float32 row-major, consumed in sorted-name order.
- Generator script is included as `build_fixture.py` so the folder can be rebuilt
  and diffed.

**Not yet verified:** loading under a specific `transformers` version (no
torch/transformers in the build environment), and whether
`Glm5NextForConditionalGeneration` accepts this reduced text-only geometry without
extra fields. Treat those as open until run against a real install.

## How to use
Read the tensors with the standard library (no torch needed, matching how this was
built):

```python
import json, struct
with open("model.safetensors", "rb") as f:
    n = struct.unpack("<Q", f.read(8))[0]
    header = json.loads(f.read(n))
    # header[name] = {"dtype", "shape", "data_offsets"}; data starts at byte 8+n
```

Or with the `safetensors` package:

```python
from safetensors.torch import load_file
tensors = load_file("model.safetensors")   # {name: tensor}
```

To exercise a real loader, build a config from `config.json` (the `glm5_next`
model type; use `AutoConfig.from_pretrained(..., trust_remote_code=True)` where
needed) and feed these weights in. There is no `lm_head` tensor, no vision stack,
and the tokenizer files are placeholders, so supply your own head/tokenizer/vision.

## License
The generated fixture content (random weights, config, scripts) is released under
MIT (see `LICENSE`). The `glm5_next` architecture and config schema belong to the
base model `zai-org/GLM-5.3-Flash` under its own terms, which were **not**
independently re-verified this cycle — check the base repository before
redistribution.

## Citation
Zhipu AI / Z.ai, GLM-5.3-Flash, 2026.

Buy me a beer:
**BTC** `bc1q5ayht3fxhj0v95fk0z8l2f6900g3awdsw5842p`