Instructions to use kernelpool/GLM-5.3-5bit-UVMAX with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use kernelpool/GLM-5.3-5bit-UVMAX with MLX:

# Make sure mlx-lm is installed
# pip install --upgrade mlx-lm

# Generate text with mlx-lm
from mlx_lm import load, generate

model, tokenizer = load("kernelpool/GLM-5.3-5bit-UVMAX")

prompt = "Write a story about Einstein"
messages = [{"role": "user", "content": prompt}]
prompt = tokenizer.apply_chat_template(
    messages, add_generation_prompt=True
)

text = generate(model, tokenizer, prompt=prompt, verbose=True)

Notebooks
Google Colab
Kaggle
Local Apps Settings
LM Studio

How to use kernelpool/GLM-5.3-5bit-UVMAX with Pi:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "kernelpool/GLM-5.3-5bit-UVMAX"

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "mlx-lm": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "kernelpool/GLM-5.3-5bit-UVMAX"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

MLX LM

How to use kernelpool/GLM-5.3-5bit-UVMAX with MLX LM:

Generate or start a chat session

# Install MLX LM
uv tool install mlx-lm
# Interactive chat REPL
mlx_lm.chat --model "kernelpool/GLM-5.3-5bit-UVMAX"

Run an OpenAI-compatible server

# Install MLX LM
uv tool install mlx-lm
# Start the server
mlx_lm.server --model "kernelpool/GLM-5.3-5bit-UVMAX"
# Calling the OpenAI-compatible server with curl
curl -X POST "http://localhost:8000/v1/chat/completions" \
   -H "Content-Type: application/json" \
   --data '{
     "model": "kernelpool/GLM-5.3-5bit-UVMAX",
     "messages": [
       {"role": "user", "content": "Hello"}
     ]
   }'

Hermes Agent

How to use kernelpool/GLM-5.3-5bit-UVMAX with Hermes Agent:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "kernelpool/GLM-5.3-5bit-UVMAX"

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default kernelpool/GLM-5.3-5bit-UVMAX

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use kernelpool/GLM-5.3-5bit-UVMAX with OpenClaw:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "kernelpool/GLM-5.3-5bit-UVMAX"

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "kernelpool/GLM-5.3-5bit-UVMAX" \
  --custom-provider-id mlx-lm \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

GLM-5.3-5bit-UVMAX / README.md

kernelpool

Add files using upload-large-folder tool

a30a53a verified 2 months ago

preview code

Raw History Blame Contribute Delete

2.51 kB

---
language:
- en
- zh
library_name: mlx
pipeline_tag: text-generation
tags:
- mlx
license: other
license_name: glm-5.3
license_link: https://huggingface.co/zai-org/GLM-5.3-BF16/blob/main/LICENSE
base_model: zai-org/GLM-5.3-BF16
base_model_relation: quantized
---
# kernelpool/GLM-5.3-5bit-UVMAX

Mixed-precision (UVMAX) quantization of
[zai-org/GLM-5.3-BF16](https://huggingface.co/zai-org/GLM-5.3-BF16),
converted with mlx-lm from the bf16 release.

## What is UVMAX?

UVMAX assigns bit widths per tensor class instead of quantizing uniformly.

| tensor class | precision | parameters | size | share |
|---|---|---|---|---|
| Expert FFN gate/up | 4-bit, group 128 | 483B | 239.1 GiB | 59.1% |
| Expert FFN down | 5-bit, group 128 | 242B | 147.7 GiB | 36.5% |
| Attention + DSA indexer + dense MLP | 8-bit, group 64 | 13.7B | 13.6 GiB | 3.4% |
| Shared experts | 8-bit, group 64 | 2.8B | 2.8 GiB | 0.7% |
| Embeddings, LM head | 4-bit, group 64 | 1.9B | 1.0 GiB | 0.2% |
| Routers | bf16 | 0.1B | 0.2 GiB | <0.1% |
| Norms | bf16 | — | — | <0.1% |
| total | 4.67 bits/weight | 743B | 404 GiB | |

## Quality

Teacher-forced against the bf16 release on identical tokens, 48 windows of
1025 tokens. KLD is `KL(bf16 ‖ quant)` over the full output distribution and
is corpus-specific.

**UVMAX (4.67 bits/weight)**

| corpus | ppl bf16 | ppl UVMAX | ratio | mean KLD | median KLD | top-1 agreement |
|---|---|---|---|---|---|---|
| Linux kernel C | 1.434 | 1.459 | 1.02× | 0.040 | 0.0002 | 96.8% |
| XNU kernel C | 2.435 | 2.472 | 1.02× | 0.056 | 0.0027 | 93.4% |
| JavaScriptCore C++ | 1.757 | 1.791 | 1.02× | 0.055 | 0.0007 | 95.1% |
| English prose | 2.687 | 2.750 | 1.02× | 0.063 | 0.0076 | 92.4% |
| all | 2.015 | 2.053 | 1.02× | 0.053 | 0.0012 | 94.4% |

**Uniform 4-bit (4.50 bits/weight)**

| corpus | ppl bf16 | ppl 4-bit | ratio | mean KLD | median KLD | top-1 agreement |
|---|---|---|---|---|---|---|
| Linux kernel C | 1.434 | 1.492 | 1.04× | 0.076 | 0.0004 | 95.3% |
| XNU kernel C | 2.435 | 2.522 | 1.04× | 0.109 | 0.0063 | 91.0% |
| JavaScriptCore C++ | 1.757 | 1.841 | 1.05× | 0.102 | 0.0014 | 93.4% |
| English prose | 2.687 | 2.909 | 1.08× | 0.143 | 0.0209 | 88.6% |
| all | 2.015 | 2.119 | 1.05× | 0.107 | 0.0028 | 92.1% |

## Use with mlx

Requires a recent mlx-lm with GLM-5.3 (`glm_moe_dsa`) support.

The model fits on a single 512 GB machine:

```bash
mlx_lm.server --model kernelpool/GLM-5.3-5bit-UVMAX
```

Sampling follows the base model: temperature 1.0, top-p 0.95.

	---
	language:
	- en
	- zh
	library_name: mlx
	pipeline_tag: text-generation
	tags:
	- mlx
	license: other
	license_name: glm-5.3
	license_link: https://huggingface.co/zai-org/GLM-5.3-BF16/blob/main/LICENSE
	base_model: zai-org/GLM-5.3-BF16
	base_model_relation: quantized
	---

	# kernelpool/GLM-5.3-5bit-UVMAX

	Mixed-precision (UVMAX) quantization of
	[zai-org/GLM-5.3-BF16](https://huggingface.co/zai-org/GLM-5.3-BF16),
	converted with mlx-lm from the bf16 release.

	## What is UVMAX?

	UVMAX assigns bit widths per tensor class instead of quantizing uniformly.

	\| tensor class \| precision \| parameters \| size \| share \|
	\|---\|---\|---\|---\|---\|
	\| Expert FFN gate/up \| 4-bit, group 128 \| 483B \| 239.1 GiB \| 59.1% \|
	\| Expert FFN down \| 5-bit, group 128 \| 242B \| 147.7 GiB \| 36.5% \|
	\| Attention + DSA indexer + dense MLP \| 8-bit, group 64 \| 13.7B \| 13.6 GiB \| 3.4% \|
	\| Shared experts \| 8-bit, group 64 \| 2.8B \| 2.8 GiB \| 0.7% \|
	\| Embeddings, LM head \| 4-bit, group 64 \| 1.9B \| 1.0 GiB \| 0.2% \|
	\| Routers \| bf16 \| 0.1B \| 0.2 GiB \| <0.1% \|
	\| Norms \| bf16 \| — \| — \| <0.1% \|
	\| total \| 4.67 bits/weight \| 743B \| 404 GiB \| \|

	## Quality

	Teacher-forced against the bf16 release on identical tokens, 48 windows of
	1025 tokens. KLD is `KL(bf16 ‖ quant)` over the full output distribution and
	is corpus-specific.

	UVMAX (4.67 bits/weight)

	\| corpus \| ppl bf16 \| ppl UVMAX \| ratio \| mean KLD \| median KLD \| top-1 agreement \|
	\|---\|---\|---\|---\|---\|---\|---\|
	\| Linux kernel C \| 1.434 \| 1.459 \| 1.02× \| 0.040 \| 0.0002 \| 96.8% \|
	\| XNU kernel C \| 2.435 \| 2.472 \| 1.02× \| 0.056 \| 0.0027 \| 93.4% \|
	\| JavaScriptCore C++ \| 1.757 \| 1.791 \| 1.02× \| 0.055 \| 0.0007 \| 95.1% \|
	\| English prose \| 2.687 \| 2.750 \| 1.02× \| 0.063 \| 0.0076 \| 92.4% \|
	\| all \| 2.015 \| 2.053 \| 1.02× \| 0.053 \| 0.0012 \| 94.4% \|

	Uniform 4-bit (4.50 bits/weight)

	\| corpus \| ppl bf16 \| ppl 4-bit \| ratio \| mean KLD \| median KLD \| top-1 agreement \|
	\|---\|---\|---\|---\|---\|---\|---\|
	\| Linux kernel C \| 1.434 \| 1.492 \| 1.04× \| 0.076 \| 0.0004 \| 95.3% \|
	\| XNU kernel C \| 2.435 \| 2.522 \| 1.04× \| 0.109 \| 0.0063 \| 91.0% \|
	\| JavaScriptCore C++ \| 1.757 \| 1.841 \| 1.05× \| 0.102 \| 0.0014 \| 93.4% \|
	\| English prose \| 2.687 \| 2.909 \| 1.08× \| 0.143 \| 0.0209 \| 88.6% \|
	\| all \| 2.015 \| 2.119 \| 1.05× \| 0.107 \| 0.0028 \| 92.1% \|

	## Use with mlx

	Requires a recent mlx-lm with GLM-5.3 (`glm_moe_dsa`) support.

	The model fits on a single 512 GB machine:

	```bash
	mlx_lm.server --model kernelpool/GLM-5.3-5bit-UVMAX
	```

	Sampling follows the base model: temperature 1.0, top-p 0.95.