Add Mac memory chooser and reproducible demo prompt

README.md

@@ -24,12 +24,14 @@ tags:
   - apple-silicon
 ---
 
+
 <div align="center">
   <a href="https://huggingface.co/zai-org/GLM-5.3-Flash">
     <img src="zai-logo.png" width="96" height="96" alt="Z.ai logo">
   </a>
 </div>
 
+
 <p align="center">
   <img src="https://img.shields.io/badge/Z.ai-GLM--5.3--Flash-111827?style=for-the-badge" alt="Z.ai GLM-5.3-Flash">
   <img src="https://img.shields.io/badge/Apple_Silicon-MLX-000000?style=for-the-badge&logo=apple&logoColor=white" alt="Apple silicon MLX">
@@ -37,12 +39,15 @@ tags:
   <img src="https://img.shields.io/badge/Vontra-MLX-6E56CF?style=for-the-badge&logo=huggingface&logoColor=white" alt="Vontra MLX">
 </p>
 
+
 <h1 align="center">GLM-5.3-Flash — MLX 4-bit with native MTP</h1>
 
+
 <p align="center">
   A native Apple-silicon conversion of <a href="https://huggingface.co/zai-org/GLM-5.3-Flash">zai-org/GLM-5.3-Flash</a>, quantised from the official checkpoint with its matching next-token prediction block preserved.
 </p>
 
+
 <p align="center">
   <a href="https://huggingface.co/zai-org/GLM-5.3-Flash">Original model</a> ·
   <a href="https://z.ai/blog/glm-5.3-flash">Z.ai overview</a> ·
@@ -51,8 +56,10 @@ tags:
   <a href="LICENSE">MIT licence</a>
 </p>
 
+
 ## At a glance
 
+
 | Item | Value |
 | --- | --- |
 | Base model | [`zai-org/GLM-5.3-Flash`](https://huggingface.co/zai-org/GLM-5.3-Flash) |
@@ -70,10 +77,13 @@ tags:
 | Configured context | 1,048,576 tokens |
 | Architecture | `glm5_next`, multimodal sparse MoE |
 
+
 Modules that are not eligible for MLX quantisation remain at their source-compatible precision. The checkpoint preserves the upstream tokenizer, chat template, multimodal processor, generation configuration, and native MTP metadata.
 
+
 ## Quantisation recipe
 
+
 | Component | Treatment |
 | --- | --- |
 | Eligible language projections | 4-bit affine, group size 64 |
@@ -84,12 +94,16 @@ Modules that are not eligible for MLX quantisation remain at their source-compat
 | Other non-quantisable tensors | Preserved at source-compatible precision |
 | Converter | Vontra streamed MLX affine converter using MLX 0.32.0 |
 
+
 This is a weight-only post-training quantisation. It does not retrain or fine-tune the upstream model.
 
+
 ## Runtime compatibility
 
+
 GLM-5.3-Flash uses the new `glm5_next` multimodal architecture, hybrid linear and sparse attention, manifold-constrained hyper-connections, and an embedded next-token prediction layer. All four features need explicit runtime support.
 
+
 | Runtime path | Status |
 | --- | --- |
 | oMLX `0.6.3rc3` | Validated for baseline generation; native MTP tensors recognised |
@@ -97,24 +111,31 @@ GLM-5.3-Flash uses the new `glm5_next` multimodal architecture, hybrid linear an
 | MLX-VLM `0.6.3 with GLM5 Next runtime integration` | Required for `glm5_next` model and vision loading |
 | Older or stock builds without GLM5 Next MTP integration | Unsupported for native MTP |
 
+
 > [!IMPORTANT]
 > A runtime that can load `glm5_next` is not automatically able to use its embedded MTP layer. Native MTP requires the GLM5 Next-specific draft, verification, and cache-reconciliation path. If that integration is absent, keep MTP disabled or use a compatible build.
 
+
 ## Download and use
 
+
 ```bash
 hf download Vontra/GLM-5.3-Flash-MLX-4bit-MTP \
   --local-dir GLM-5.3-Flash-MLX-4bit-MTP
 ```
 
+
 In a compatible oMLX build:
 
+
 1. Add the downloaded model directory.
 2. Keep **Native MTP** disabled for the currently validated serving path.
 3. Start a new conversation after changing model settings so the runtime rebuilds the model and caches.
 
+
 For baseline generation without native MTP, a compatible MLX-VLM installation can use:
 
+
 ```bash
 python -m mlx_vlm.generate \
   --model Vontra/GLM-5.3-Flash-MLX-4bit-MTP \
@@ -122,14 +143,18 @@ python -m mlx_vlm.generate \
   --max-tokens 512
 ```
 
+
 ## Apple M3 Studio performance
 
+
 Measured on an Apple M3 Studio with a warmed model state:
 
+
 | Mode | Prompt tokens | Output tokens | Decode | Model memory |
 | --- | ---: | ---: | ---: | ---: |
 | Baseline, MTP off | 63 | 512 | 6.2657 tokens/s | 173.39 GB |
 
+
 | Benchmark detail | Value |
 | --- | --- |
 | Chip | Apple M3 Ultra |
@@ -139,10 +164,13 @@ Measured on an Apple M3 Studio with a warmed model state:
 | Warm-up | one 64-token greedy request discarded before measurement |
 | Sampling | greedy decoding, temperature 0, top-p 1, top-k 0, fixed seed 6330 |
 
+
 These figures are a local reference, not a cross-platform guarantee. Prompt shape, image input, context growth, draft acceptance, sampling, memory pressure, thermal state, and runtime revisions can materially change performance.
 
+
 ### Native MTP status
 
+
 The checkpoint contains the official matching NextN layer. A short deterministic
 MTP smoke test produced the exact requested output and recorded an accepted
 draft. Sustained testing on the current oMLX build exposed repeated speculative
@@ -151,10 +179,13 @@ therefore preserved for compatible future runtimes but is **not recommended for
 sustained serving on the validated build**. Keep MTP disabled unless your
 runtime has corrected that scheduler behaviour.
 
+
 ## Architecture
 
+
 GLM-5.3-Flash is Z.ai's first natively multimodal GLM-5 model. It combines KDA linear-attention layers with periodic sparse-attention layers, a sparse mixture-of-experts feed-forward stack, manifold-constrained hyper-connections, and a native next-token prediction layer.
 
+
 | Architecture detail | Upstream value |
 | --- | ---: |
 | Parameters | 320B total / 18B active |
@@ -170,10 +201,13 @@ GLM-5.3-Flash is Z.ai's first natively multimodal GLM-5 model. It combines KDA l
 | Vision encoder | 24 layers, 1,024 hidden size, 448-pixel input |
 | Configured maximum context | 1,048,576 tokens |
 
+
 See the [official model card](https://huggingface.co/zai-org/GLM-5.3-Flash), [Z.ai overview](https://z.ai/blog/glm-5.3-flash), and [GLM-5 technical report](https://arxiv.org/abs/2602.15763) for upstream training, evaluations, intended uses, and safety guidance.
 
+
 ## Validation
 
+
 | Check | Result |
 | --- | --- |
 | Config, processor, tokenizer, and chat-template files present | Passed — official tokenizer, updated chat template, processor, generation config, and MIT licence included |
@@ -186,10 +220,13 @@ See the [official model card](https://huggingface.co/zai-org/GLM-5.3-Flash), [Z.
 | Vision tensor structure | Passed structurally — all 347 official vision tensors are preserved; image generation was not benchmarked |
 | Long-context generation | Not claimed — configured context metadata is preserved, but no long-context generation result is reported |
 
+
 Validation used oMLX `0.6.3rc3`, MLX `0.32.0`, and MLX-VLM `0.6.3 with GLM5 Next runtime integration`.
 
+
 ## Limitations
 
+
 - Quantisation can reduce quality relative to the official checkpoint, especially on difficult reasoning, tool-use, multilingual, vision, and long-context workloads.
 - Native MTP improves speed only when draft acceptance offsets drafting and verification cost. It may be neutral or slower for unpredictable output.
 - The configured one-million-token context does not imply that every Apple-silicon system has enough memory for a full-context request.
@@ -197,16 +234,22 @@ Validation used oMLX `0.6.3rc3`, MLX `0.32.0`, and MLX-VLM `0.6.3 with GLM5 Next
 - Runtime support for `glm5_next` and its native MTP path is evolving. Use the exact validated versions listed in this card.
 - Validate numerical behaviour, output quality, safety, and tool use on representative workloads before production deployment.
 
+
 This is a community quantisation and is not an official Z.ai release.
 
+
 ## Licence and attribution
 
+
 The upstream model is released under the **MIT License**. The required licence text is included in this repository.
 
+
 Model design, training, upstream evaluations, and documentation belong to Z.ai and the GLM-5 contributors. The MLX conversion, Apple-silicon validation, native-MTP integration work, and packaging are provided by [Vontra](https://huggingface.co/Vontra).
 
+
 If you use this model in research, cite the upstream report:
 
+
 ```bibtex
 @misc{glm5team2026glm5,
   title        = {GLM-5: from Vibe Coding to Agentic Engineering},
@@ -218,3 +261,35 @@ If you use this model in research, cite the upstream report:
   url          = {https://arxiv.org/abs/2602.15763}
 }
 ```
+
+
+
+<!-- vontra-chooser-start -->
+## Choose for your Mac
+
+[64GB Macs](https://huggingface.co/collections/Vontra/mlx-models-for-64gb-macs-6a9fefda17932216ec9ab457) · [128GB Macs](https://huggingface.co/collections/Vontra/mlx-models-for-128gb-macs-6a9ff0abd31bc9abbe7922d7) · [256GB Macs](https://huggingface.co/collections/Vontra/mlx-models-for-256gb-macs-6a9ff0ef9fed7c5bdca15e9b)
+
+No measured memory tier is assigned here. The collections use published M3 Studio peaks with at least 25% nominal headroom; fit on other Macs is an estimate, and full context is not guaranteed. Start with short context and one request.
+
+### Runtime and evidence
+
+oMLX version mentioned in the existing card: **0.6.3rc3**; consult its compatibility notes for whether this was tested and any required integrations. The original performance tables retain their benchmark conditions and speed figures; this documentation update adds no new test results.
+
+### Quick start and demo prompt
+
+```bash
+hf download Vontra/GLM-5.3-Flash-MLX-4bit-MTP --local-dir ./models/GLM-5.3-Flash-MLX-4bit-MTP
+```
+
+Add the downloaded folder to oMLX model directories, refresh the list, and follow this card's architecture and MTP compatibility requirements before loading.
+
+Try this in a new chat with a 128-token output limit:
+
+```text
+Explain why the sky looks blue in three short sentences.
+```
+
+This is a demo prompt to try, not a recorded successful run; a captured demonstration for this documentation update is not yet available.
+
+[Follow Vontra for new Apple Silicon releases and fixes.](https://huggingface.co/Vontra)
+<!-- vontra-chooser-end -->