| --- |
| library_name: mlx |
| license: mit |
| license_link: LICENSE |
| base_model: zai-org/GLM-5.3-Flash |
| base_model_relation: quantized |
| pipeline_tag: image-text-to-text |
| language: |
| - en |
| - zh |
| tags: |
| - mlx |
| - mlx-vlm |
| - omlx |
| - glm |
| - glm5-next |
| - native-mtp |
| - speculative-decoding |
| - mixed-precision |
| - mixture-of-experts |
| - multimodal |
| - vision-language |
| - quantized |
| - apple-silicon |
| - 2-bit |
| --- |
| |
| <p align="center"> |
| <a href="https://tensorfold.dev"> |
| <img src="https://huggingface.co/spaces/TensorFold/README/resolve/main/tensorfold-logo.png" alt="TensorFold" width="160"> |
| </a> |
| </p> |
| |
| <div align="center"> |
| <a href="https://huggingface.co/zai-org/GLM-5.3-Flash"> |
| <img src="zai-logo.png" width="72" height="72" alt="Z.ai logo"> |
| </a> |
| </div> |
| |
|
|
| <p align="center"> |
| <img src="https://img.shields.io/badge/Z.ai-GLM--5.3--Flash-111827?style=flat-square" alt="Z.ai GLM-5.3-Flash"> |
| <img src="https://img.shields.io/badge/Apple_Silicon-MLX-000000?style=flat-square&logo=apple&logoColor=white" alt="Apple silicon MLX"> |
| <img src="https://img.shields.io/badge/Native_MTP-Depth_1-2563EB?style=flat-square" alt="Native MTP depth 1"> |
| <img src="https://img.shields.io/badge/TensorFold-Hybrid_2--bit-6E56CF?style=flat-square&logo=huggingface&logoColor=white" alt="TensorFold hybrid 2-bit"> |
| </p> |
|
|
|
|
| <h1 align="center">GLM-5.3-Flash, hybrid MLX 2-bit with native MTP</h1> |
|
|
|
|
| <p align="center"> |
| A deterministic quality-protected MLX conversion of <a href="https://huggingface.co/zai-org/GLM-5.3-Flash">zai-org/GLM-5.3-Flash</a>, retaining the model's matching native next-token prediction layer. |
| </p> |
|
|
|
|
| <p align="center"> |
| <a href="https://huggingface.co/zai-org/GLM-5.3-Flash">Original model</a> 路 |
| <a href="https://z.ai/blog/glm-5.3-flash">Z.ai overview</a> 路 |
| <a href="https://arxiv.org/abs/2602.15763">Technical report</a> 路 |
| <a href="https://github.com/ml-explore/mlx">Apple MLX</a> 路 |
| <a href="LICENSE">MIT licence</a> |
| </p> |
|
|
|
|
| > [!IMPORTANT] |
| > This is a hybrid 2-bit checkpoint, not a uniform Q2 build. Routed trunk experts use Q2, non-expert language projections use Q8, native-MTP expert projections use Q4, and sensitive components remain BF16. This allocation passed live generation where broader Q2 recipes did not. |
|
|
|
|
| ## At a glance |
|
|
|
|
| | Item | Value | |
| | --- | --- | |
| | Repository | `TensorFold/GLM-5.3-Flash-MLX-2bit-MTP` | |
| | Base model | [`zai-org/GLM-5.3-Flash`](https://huggingface.co/zai-org/GLM-5.3-Flash) | |
| | Source revision | [`04c4e9e95c5da8862dced7e5056455116f83a7e0`](https://huggingface.co/zai-org/GLM-5.3-Flash/commit/04c4e9e95c5da8862dced7e5056455116f83a7e0) | |
| | Source weight format | FP8 E4M3 with 128x128 block scaling | |
| | Output format | MLX safetensors, affine weight quantisation, group size 64 | |
| | Quantisation | Deterministic hybrid 2-bit recipe | |
| | Native MTP | Preserved, one upstream prediction layer, runtime depth 1 | |
| | Indexed tensors | 114,154 | |
| | Weight shards | 26 | |
| | Tensor payload | 111,317,859,192 bytes, 111.318 GB / 103.673 GiB | |
| | Configured context | 1,048,576 tokens | |
| | Architecture | `glm5_next`, multimodal sparse MoE | |
|
|
|
|
| ## Precision recipe |
|
|
|
|
| | Component | Treatment | |
| | --- | --- | |
| | Routed trunk experts | Q2 affine, group size 64; 36,288 source matrices stacked into 126 runtime modules | |
| | Non-expert language projections | Q8 affine, group size 64; 540 source matrices mapped to 552 runtime modules | |
| | Native-MTP routed experts | Q4 affine, group size 64; 864 source matrices stacked into 3 runtime modules | |
| | Sparse indexer projections | All 36 at Q8 affine, group size 64 | |
| | Token embedding and output head | BF16 | |
| | Vision encoder and projector | BF16 | |
| | MTP fusion projection | BF16 | |
| | Routers, hyper-connections, norms and other non-quantisable tensors | Source precision | |
|
|
|
|
| This is weight-only post-training quantisation. It does not retrain or fine-tune the upstream model. |
|
|
|
|
| ## Runtime compatibility |
|
|
|
|
| GLM-5.3-Flash uses the `glm5_next` multimodal architecture and a native NextN/MTP block. Use a runtime that supports this architecture and per-module MLX quantisation metadata. |
|
|
|
|
| | Component | Tested version | |
| | --- | --- | |
| | oMLX | 0.6.3rc3, build 2475 | |
| | MLX | 0.32.0 | |
| | mlx-lm | 0.31.3 | |
| | mlx-vlm | 0.6.3 | |
| | Native-MTP draft depth | 1 | |
|
|
|
|
| The current upstream chat template defaults to maximum reasoning effort. For short, direct answers, pass `reasoning_effort: low` through the chat-template arguments. |
|
|
|
|
| ## Download and use |
|
|
|
|
| ```bash |
| hf download TensorFold/GLM-5.3-Flash-MLX-2bit-MTP \ |
| --local-dir ./GLM-5.3-Flash-MLX-2bit-MTP |
| ``` |
|
|
|
|
| Add the downloaded directory to a compatible oMLX model directory, refresh the model registry, and select the model. Native MTP is optional and uses draft depth 1. |
|
|
|
|
| For an OpenAI-compatible request through oMLX, include the upstream low-reasoning template option when you want a concise answer: |
|
|
|
|
| ```json |
| { |
| "model": "GLM-5.3-Flash-MLX-2bit-MTP", |
| "messages": [{"role": "user", "content": "What is the capital of France?"}], |
| "chat_template_kwargs": {"reasoning_effort": "low"} |
| } |
| ``` |
|
|
|
|
| ## Apple M3 Studio performance |
|
|
|
|
| Each result is the median of three 512-token text-generation runs after a separate warm-up. Both modes used the same checkpoint, prompt, deterministic sampling settings, and current upstream chat template. |
|
|
|
|
| | Runtime mode | Runs | Output per run | Median decode speed | Long-run output parity | |
| | --- | ---: | ---: | ---: | --- | |
| | Native MTP disabled | 3 | 512 tokens | **6.063 tok/s** | Reference | |
| | Native MTP enabled, depth 1 | 3 | 512 tokens | **6.257 tok/s** | Exact match | |
|
|
|
|
| Native MTP improved median decode throughput by **3.21%** in this test. All six 512-token runs produced the same output hash. MTP gains depend on the prompt and draft acceptance, so treat this as a practical reference for the tested Studio rather than a universal result. |
|
|
|
|
| ## Architecture |
|
|
|
|
| GLM-5.3-Flash combines KDA linear-attention layers with periodic sparse-attention layers, a sparse mixture-of-experts feed-forward stack, manifold-constrained hyper-connections, a vision encoder, and one native next-token prediction layer. |
|
|
|
|
| | Architecture detail | Upstream value | |
| | --- | ---: | |
| | Parameters | 320B total / 18B active | |
| | Language layers | 45 | |
| | Routed / active experts | 288 / 8, plus 1 shared expert | |
| | Hidden size | 4,096 | |
| | Attention heads | 64 | |
| | Native MTP layers | 1 | |
| | Configured maximum context | 1,048,576 tokens | |
|
|
|
|
| See the [official model card](https://huggingface.co/zai-org/GLM-5.3-Flash), [Z.ai overview](https://z.ai/blog/glm-5.3-flash), and [GLM-5 technical report](https://arxiv.org/abs/2602.15763) for upstream training, evaluations, intended uses, and safety guidance. |
|
|
|
|
| ## Validation |
|
|
|
|
| | Check | Result | |
| | --- | --- | |
| | Official source structure | 76,108 tensors, 62 shards and 37,338 FP8 weight-scale pairs validated before conversion | |
| | Safetensors index and shard resolution | 114,154 entries resolve to 26 final shards | |
| | Saved precision layout | 36,288 Q2, 864 Q4 and 540 Q8 source matrices; all quantised matrices have matching weight, scale and bias tensors | |
| | Vision payload | 347 BF16 source tensors preserved | |
| | Native MTP structure | Complete upstream prediction layer preserved; runtime reports native-MTP compatibility | |
| | MTP disabled generation | Deterministic factual, arithmetic, instruction and coherence checks passed | |
| | MTP enabled generation | The same checks passed with exact output parity | |
| | Sustained generation | Three 512-token runs per mode; 6.063 tok/s off and 6.257 tok/s on | |
|
|
|
|
| ## Limitations |
|
|
|
|
| - Hybrid quantisation can reduce quality relative to the official checkpoint. The effect depends on the workload. |
| - The configured one-million-token context does not mean every Apple silicon system has enough memory for a full-context request. |
| - Image and video prefill have different memory and throughput characteristics from text-only generation. The published throughput numbers are text-only. |
| - Native MTP may be neutral or slower on prompts with low draft acceptance. |
| - Runtime support for `glm5_next`, mixed per-module quantisation, and native MTP is version-sensitive. |
|
|
|
|
| This is a community quantisation, not an official Z.ai release. |
|
|
|
|
| ## Licence and attribution |
|
|
|
|
| The upstream model uses the **MIT License**. The official licence text is included as `LICENSE`. |
|
|
|
|
| Model design, training, upstream evaluations, and documentation belong to Z.ai and the GLM-5 contributors. The MLX conversion, native-MTP preservation, validation, and packaging are provided by [TensorFold](https://huggingface.co/TensorFold). |
|
|
|
|
| If you use this model in research, cite the upstream report: |
|
|
|
|
| ```bibtex |
| @misc{glm5team2026glm5, |
| title = {GLM-5: from Vibe Coding to Agentic Engineering}, |
| author = {GLM-5-Team and others}, |
| year = {2026}, |
| eprint = {2602.15763}, |
| archivePrefix= {arXiv}, |
| primaryClass = {cs.LG}, |
| url = {https://arxiv.org/abs/2602.15763} |
| } |
| ``` |
|
|
|
|
|
|
| <!-- TensorFold-chooser-start --> |
| ## Choose for your Mac |
|
|
| [64GB Macs](https://huggingface.co/collections/TensorFold/mlx-models-for-64gb-macs-6a9fefda17932216ec9ab457) 路 [128GB Macs](https://huggingface.co/collections/TensorFold/mlx-models-for-128gb-macs-6a9ff0abd31bc9abbe7922d7) 路 [256GB Macs](https://huggingface.co/collections/TensorFold/mlx-models-for-256gb-macs-6a9ff0ef9fed7c5bdca15e9b) |
|
|
| No measured memory tier is assigned here. The collections use published M3 Studio peaks with at least 25% nominal headroom; fit on other Macs is an estimate, and full context is not guaranteed. Start with short context and one request. |
|
|
| ### Runtime and evidence |
|
|
| The exact tested oMLX application version is not recorded here; a library version is not an app version. The original performance tables retain their benchmark conditions and speed figures; this documentation update adds no new test results. |
|
|
| ### Quick start and demo prompt |
|
|
| ```bash |
| hf download TensorFold/GLM-5.3-Flash-MLX-2bit-MTP --local-dir ./models/GLM-5.3-Flash-MLX-2bit-MTP |
| ``` |
|
|
| Add the downloaded folder to oMLX model directories, refresh the list, and follow this card's architecture and MTP compatibility requirements before loading. |
|
|
| Try this in a new chat with a 128-token output limit: |
|
|
| ```text |
| Explain why the sky looks blue in three short sentences. |
| ``` |
|
|
| This is a demo prompt to try, not a recorded successful run; a captured demonstration for this documentation update is not yet available. |
|
|
| [Follow TensorFold for new Apple Silicon releases and fixes.](https://huggingface.co/TensorFold) |
| <!-- TensorFold-chooser-end --> |
|
|
