A deterministic quality-protected MLX conversion of zai-org/GLM-5.3-Flash, retaining the model's matching native next-token prediction layer.
Original model · Z.ai overview · Technical report · Apple MLX · MIT licence
> [!IMPORTANT] > This is a hybrid 2-bit checkpoint, not a uniform Q2 build. Routed trunk experts use Q2, non-expert language projections use Q8, native-MTP expert projections use Q4, and sensitive components remain BF16. This allocation passed live generation where broader Q2 recipes did not. ## At a glance | Item | Value | | --- | --- | | Repository | `TensorFold/GLM-5.3-Flash-MLX-2bit-MTP` | | Base model | [`zai-org/GLM-5.3-Flash`](https://huggingface.co/zai-org/GLM-5.3-Flash) | | Source revision | [`04c4e9e95c5da8862dced7e5056455116f83a7e0`](https://huggingface.co/zai-org/GLM-5.3-Flash/commit/04c4e9e95c5da8862dced7e5056455116f83a7e0) | | Source weight format | FP8 E4M3 with 128x128 block scaling | | Output format | MLX safetensors, affine weight quantisation, group size 64 | | Quantisation | Deterministic hybrid 2-bit recipe | | Native MTP | Preserved, one upstream prediction layer, runtime depth 1 | | Indexed tensors | 114,154 | | Weight shards | 26 | | Tensor payload | 111,317,859,192 bytes, 111.318 GB / 103.673 GiB | | Configured context | 1,048,576 tokens | | Architecture | `glm5_next`, multimodal sparse MoE | ## Precision recipe | Component | Treatment | | --- | --- | | Routed trunk experts | Q2 affine, group size 64; 36,288 source matrices stacked into 126 runtime modules | | Non-expert language projections | Q8 affine, group size 64; 540 source matrices mapped to 552 runtime modules | | Native-MTP routed experts | Q4 affine, group size 64; 864 source matrices stacked into 3 runtime modules | | Sparse indexer projections | All 36 at Q8 affine, group size 64 | | Token embedding and output head | BF16 | | Vision encoder and projector | BF16 | | MTP fusion projection | BF16 | | Routers, hyper-connections, norms and other non-quantisable tensors | Source precision | This is weight-only post-training quantisation. It does not retrain or fine-tune the upstream model. ## Runtime compatibility GLM-5.3-Flash uses the `glm5_next` multimodal architecture and a native NextN/MTP block. Use a runtime that supports this architecture and per-module MLX quantisation metadata. | Component | Tested version | | --- | --- | | oMLX | 0.6.3rc3, build 2475 | | MLX | 0.32.0 | | mlx-lm | 0.31.3 | | mlx-vlm | 0.6.3 | | Native-MTP draft depth | 1 | The current upstream chat template defaults to maximum reasoning effort. For short, direct answers, pass `reasoning_effort: low` through the chat-template arguments. ## Download and use ```bash hf download TensorFold/GLM-5.3-Flash-MLX-2bit-MTP \ --local-dir ./GLM-5.3-Flash-MLX-2bit-MTP ``` Add the downloaded directory to a compatible oMLX model directory, refresh the model registry, and select the model. Native MTP is optional and uses draft depth 1. For an OpenAI-compatible request through oMLX, include the upstream low-reasoning template option when you want a concise answer: ```json { "model": "GLM-5.3-Flash-MLX-2bit-MTP", "messages": [{"role": "user", "content": "What is the capital of France?"}], "chat_template_kwargs": {"reasoning_effort": "low"} } ``` ## Apple M3 Studio performance Each result is the median of three 512-token text-generation runs after a separate warm-up. Both modes used the same checkpoint, prompt, deterministic sampling settings, and current upstream chat template. | Runtime mode | Runs | Output per run | Median decode speed | Long-run output parity | | --- | ---: | ---: | ---: | --- | | Native MTP disabled | 3 | 512 tokens | **6.063 tok/s** | Reference | | Native MTP enabled, depth 1 | 3 | 512 tokens | **6.257 tok/s** | Exact match | Native MTP improved median decode throughput by **3.21%** in this test. All six 512-token runs produced the same output hash. MTP gains depend on the prompt and draft acceptance, so treat this as a practical reference for the tested Studio rather than a universal result. ## Architecture GLM-5.3-Flash combines KDA linear-attention layers with periodic sparse-attention layers, a sparse mixture-of-experts feed-forward stack, manifold-constrained hyper-connections, a vision encoder, and one native next-token prediction layer. | Architecture detail | Upstream value | | --- | ---: | | Parameters | 320B total / 18B active | | Language layers | 45 | | Routed / active experts | 288 / 8, plus 1 shared expert | | Hidden size | 4,096 | | Attention heads | 64 | | Native MTP layers | 1 | | Configured maximum context | 1,048,576 tokens | See the [official model card](https://huggingface.co/zai-org/GLM-5.3-Flash), [Z.ai overview](https://z.ai/blog/glm-5.3-flash), and [GLM-5 technical report](https://arxiv.org/abs/2602.15763) for upstream training, evaluations, intended uses, and safety guidance. ## Validation | Check | Result | | --- | --- | | Official source structure | 76,108 tensors, 62 shards and 37,338 FP8 weight-scale pairs validated before conversion | | Safetensors index and shard resolution | 114,154 entries resolve to 26 final shards | | Saved precision layout | 36,288 Q2, 864 Q4 and 540 Q8 source matrices; all quantised matrices have matching weight, scale and bias tensors | | Vision payload | 347 BF16 source tensors preserved | | Native MTP structure | Complete upstream prediction layer preserved; runtime reports native-MTP compatibility | | MTP disabled generation | Deterministic factual, arithmetic, instruction and coherence checks passed | | MTP enabled generation | The same checks passed with exact output parity | | Sustained generation | Three 512-token runs per mode; 6.063 tok/s off and 6.257 tok/s on | ## Limitations - Hybrid quantisation can reduce quality relative to the official checkpoint. The effect depends on the workload. - The configured one-million-token context does not mean every Apple silicon system has enough memory for a full-context request. - Image and video prefill have different memory and throughput characteristics from text-only generation. The published throughput numbers are text-only. - Native MTP may be neutral or slower on prompts with low draft acceptance. - Runtime support for `glm5_next`, mixed per-module quantisation, and native MTP is version-sensitive. This is a community quantisation, not an official Z.ai release. ## Licence and attribution The upstream model uses the **MIT License**. The official licence text is included as `LICENSE`. Model design, training, upstream evaluations, and documentation belong to Z.ai and the GLM-5 contributors. The MLX conversion, native-MTP preservation, validation, and packaging are provided by [TensorFold](https://huggingface.co/TensorFold). If you use this model in research, cite the upstream report: ```bibtex @misc{glm5team2026glm5, title = {GLM-5: from Vibe Coding to Agentic Engineering}, author = {GLM-5-Team and others}, year = {2026}, eprint = {2602.15763}, archivePrefix= {arXiv}, primaryClass = {cs.LG}, url = {https://arxiv.org/abs/2602.15763} } ``` ## Choose for your Mac [64GB Macs](https://huggingface.co/collections/TensorFold/mlx-models-for-64gb-macs-6a9fefda17932216ec9ab457) · [128GB Macs](https://huggingface.co/collections/TensorFold/mlx-models-for-128gb-macs-6a9ff0abd31bc9abbe7922d7) · [256GB Macs](https://huggingface.co/collections/TensorFold/mlx-models-for-256gb-macs-6a9ff0ef9fed7c5bdca15e9b) No measured memory tier is assigned here. The collections use published M3 Studio peaks with at least 25% nominal headroom; fit on other Macs is an estimate, and full context is not guaranteed. Start with short context and one request. ### Runtime and evidence The exact tested oMLX application version is not recorded here; a library version is not an app version. The original performance tables retain their benchmark conditions and speed figures; this documentation update adds no new test results. ### Quick start and demo prompt ```bash hf download TensorFold/GLM-5.3-Flash-MLX-2bit-MTP --local-dir ./models/GLM-5.3-Flash-MLX-2bit-MTP ``` Add the downloaded folder to oMLX model directories, refresh the list, and follow this card's architecture and MTP compatibility requirements before loading. Try this in a new chat with a 128-token output limit: ```text Explain why the sky looks blue in three short sentences. ``` This is a demo prompt to try, not a recorded successful run; a captured demonstration for this documentation update is not yet available. [Follow TensorFold for new Apple Silicon releases and fixes.](https://huggingface.co/TensorFold)