| --- |
| license: mit |
| base_model: zai-org/GLM-5.2 |
| library_name: basert |
| pipeline_tag: text-generation |
| tags: |
| - basert |
| - apple-silicon |
| - quantized |
| --- |
| |
| # GLM-5.2 |
|
|
| [BaseRT](https://github.com/basecompute/baseRT) `.base` build of [`zai-org/GLM-5.2`](https://huggingface.co/zai-org/GLM-5.2) for local inference on Apple Silicon (Metal). |
|
|
| A 78-layer mixture-of-experts reasoning model: 256 routed experts with 8 active per token plus one always-on shared expert (sigmoid gating), Multi-head Latent Attention (MLA, 64 heads over a single 576-wide latent KV row) with DeepSeek-style sparse attention (DSA), a 155k vocabulary and a 1M-token context window. The first three layers use a dense FFN. |
|
|
| Needs 512 GB of unified memory: the Q4 weights alone are ~418 GB resident. Apple Silicon only — the MLA/DSA kernels have no CUDA build. |
|
|
| ## Files |
|
|
| Q4 is 432 GB, past the Hub's per-file ceiling, so it ships as 9 parts of ≤45 GiB under `parts/`. |
|
|
| | File | Precision | Size | |
| |------|-----------|------| |
| | `parts/GLM-5.2-Q4.base.part-000` … `part-008` | 4-bit (group-quantized, f16 attention norms / MLA projections) | 432 GB total | |
| | `parts/GLM-5.2-Q4.base.manifest.json` | part count, per-part sizes and sha256s, whole-file sha256 | 2 KB | |
|
|
| `basert pull` (BaseRT 0.2.4 or newer) downloads the parts and reassembles them on install, checking each part against the manifest as it lands. The parts are streamed into place one at a time and each is deleted once appended, so the pull needs the bundle plus one part of free disk (about 480 GB), not two bundles. A pull that is interrupted resumes at the next part. The repo is private, so the token has to be in the environment: |
|
|
| ```sh |
| export HF_TOKEN=hf_... |
| basert pull basecompute/GLM-5.2 |
| ``` |
|
|
| Without `basert pull`, the parts concatenate in order into one file; stream them so the download never needs a second 432 GB of free space: |
|
|
| ```sh |
| for i in 0 1 2 3 4 5 6 7 8; do |
| curl -L -H "Authorization: Bearer $HF_TOKEN" \ |
| "https://huggingface.co/basecompute/GLM-5.2/resolve/main/parts/GLM-5.2-Q4.base.part-00$i" >> GLM-5.2-Q4.base |
| done |
| shasum -a 256 GLM-5.2-Q4.base # expect e7cf483188806908f290a6eed1d96ec6a974cb3d4751a867d5c67a317646c230 |
| ``` |
|
|
| ## Usage |
|
|
| GLM 5.2 needs BaseRT 0.2.4 or newer. |
|
|
| ```bash |
| curl -LsSf https://basecompute.co/install.sh | sh |
| basert chat basecompute/GLM-5.2 # after basert pull |
| basert serve basecompute/GLM-5.2 # OpenAI-compatible HTTP server |
| basert chat ./GLM-5.2-Q4.base # or a hand-reassembled file |
| ``` |
|
|
| DSA sparse attention is opt-in. Dense MLA is the default and matches llama.cpp's glm-dsa implementation bitwise; set `BASERT_DSA=1` to decode on the sparse top-k path once the context is deeper than the indexer's selection width (2048 tokens): |
|
|
| ```bash |
| BASERT_DSA=1 basert serve ./GLM-5.2-Q4.base |
| ``` |
|
|
| ## Measured |
|
|
| Mac Studio M3 Ultra (80-core GPU, 512 GB), BaseRT 0.2.4 (staging build of commit 36bd7e24), greedy, single stream, `basert bench` (pp = prompt tokens/s, tg = generated tokens/s): |
|
|
| | test | dense MLA | `BASERT_DSA=1` | |
| |---|---:|---:| |
| | pp512 | 236 t/s | — | |
| | tg128 | 22.9 t/s | — | |
| | pp4096 | 178 t/s | 188 t/s | |
| | tg128 after 4096 | 22.7 t/s | 22.4 t/s | |
|
|
| Weights resident: ~418 GB. First load after a reboot streams the whole file from disk (about 3.5 minutes on the Studio's internal SSD); a warm load is under 10 seconds. At a 4k context the sparse path is a wash — its selection width is 2048 tokens, so it only starts to pay past that, and generation is weight-bandwidth-bound at this size either way. |
|
|
| Released under the mit license, inherited from the base model. |
|
|
