Instructions to use Manprot/MP-02-16B-Preview with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Manprot/MP-02-16B-Preview with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="Manprot/MP-02-16B-Preview")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("Manprot/MP-02-16B-Preview")
model = AutoModelForMultimodalLM.from_pretrained("Manprot/MP-02-16B-Preview", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

llama.cpp

How to use Manprot/MP-02-16B-Preview with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf Manprot/MP-02-16B-Preview:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf Manprot/MP-02-16B-Preview:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf Manprot/MP-02-16B-Preview:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf Manprot/MP-02-16B-Preview:Q4_K_M

Build from source code

git clone https://github.com/ggml-org/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-app
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama serve -hf Manprot/MP-02-16B-Preview:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama cli -hf Manprot/MP-02-16B-Preview:Q4_K_M

Use Docker

docker model run hf.co/Manprot/MP-02-16B-Preview:Q4_K_M

Llama new
LM Studio
Jan

vLLM

How to use Manprot/MP-02-16B-Preview with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Manprot/MP-02-16B-Preview"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Manprot/MP-02-16B-Preview",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/Manprot/MP-02-16B-Preview:Q4_K_M

SGLang

How to use Manprot/MP-02-16B-Preview with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Manprot/MP-02-16B-Preview" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Manprot/MP-02-16B-Preview",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Manprot/MP-02-16B-Preview" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Manprot/MP-02-16B-Preview",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Ollama
How to use Manprot/MP-02-16B-Preview with Ollama:
```
ollama run hf.co/Manprot/MP-02-16B-Preview:Q4_K_M
```
Unsloth Desktop

How to use Manprot/MP-02-16B-Preview with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf Manprot/MP-02-16B-Preview:Q4_K_M

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "Manprot/MP-02-16B-Preview:Q4_K_M"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use Manprot/MP-02-16B-Preview with Docker Model Runner:
```
docker model run hf.co/Manprot/MP-02-16B-Preview:Q4_K_M
```

Lemonade

How to use Manprot/MP-02-16B-Preview with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull Manprot/MP-02-16B-Preview:Q4_K_M

Run and chat with the model

lemonade run user.MP-02-16B-Preview-Q4_K_M

List all available models

lemonade list

Hermes Agent

How to use Manprot/MP-02-16B-Preview with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf Manprot/MP-02-16B-Preview:Q4_K_M

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default Manprot/MP-02-16B-Preview:Q4_K_M

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use Manprot/MP-02-16B-Preview with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf Manprot/MP-02-16B-Preview:Q4_K_M

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "Manprot/MP-02-16B-Preview:Q4_K_M" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

MP-02-16B-Preview / README.md

SURELYNOTVAIN

Add bf16 safetensors, card, eval results

e8c28b6 verified 4 days ago

preview code

Raw History Blame Contribute Delete

15.3 kB

---
license: mit
base_model: zai-org/GLM-5.3-Flash
base_model_relation: finetune
library_name: transformers
pipeline_tag: image-text-to-text
language:
- en
tags:
- glm
- moe
- expert-pruning
- reap
- gguf
- preview
- roleplay
datasets:
- Zek-Takai/glm53-flash-harvest
- shanjiaz/glm-5.3-flash-open-perfectblend-regen-25pct
- best-distill/glm-5.3-flash-distillation-chat
- open-athena/GLM-5.3-RLVR1-Training-Rollouts-2026.09.25
- sollamon/ox-alpha-glm-5.3-flash-distillation-coding-17k-raw
- open-athena/GLM-5.3-tmax-swesmith-k8-traces-2026.10
- liangzhidanta/claude-code-glm53-swesmith-trajectories
- liangzhidanta/claude-code-glm53-swesmith-compact-trajectories
- kaivoss/UltraData-SFT-Agent-2609-GLM53-20k
- Rallex3/glm-5.3-flash-function-calling
- closestfriend/glm-5.3-flash-function-calling
- FranckAbgrall/glm53-flash-function-calling
- flaviojoshua/glm53-flash-function-calling
- Offlin33er/glm-5.3-flash-function-calling
- Gryphe/Sonnet3.5-Charcard-Roleplay
- anthracite-org/c2_logs_32k_llama3_qwen2_v1.3
---
<p align="center"><img src="manprot-logo.png" alt="Manprot" width="360"></p>

<h1 align="center">MP-02 16B Preview</h1>

<p align="center">
A pruned and healed version of <a href="https://huggingface.co/zai-org/GLM-5.3-Flash">GLM-5.3-Flash</a>, made by the <a href="https://huggingface.co/Manprot">Manprot</a> team
</p>

| | |
|---|---|
| **Made by** | The [Manprot](https://huggingface.co/Manprot) team |
| **Family** | MP-02 (see also [MP-01 100M Preview](https://huggingface.co/Manprot/MP-01-100M-Preview)) |
| **Status** | **Preview**: it works, but it's rough (see [Limitations](#limitations)) |
| **Type** | Chat model with thinking, healed on GLM-5.3 conversations |
| **Parameters** | 16.06B in the transformer layers; 18.28B in total with embeddings and LM head (1.27B), MTP head (0.39B) and vision encoder (0.56B) |
| **Architecture** | GLM-5.3-Flash (`glm5_next`) cut from 288 to 8 routed experts per MoE layer |
| **Built from** | [zai-org/GLM-5.3-Flash](https://huggingface.co/zai-org/GLM-5.3-Flash) (MIT) |
| **Tokens trained** | 22.9M (healing only; no pretraining of our own) |
| **7-task average** | 51.29, about 3 points above MP-01 100M Preview |
| **License** | MIT, same as the base model (Z.ai copyright notice kept in `LICENSE`) |

> **Note:** this is a **preview**. It is somewhat decent at chat and roleplay, but cutting 97% of the experts cost it much of GLM-5.3-Flash's knowledge and reasoning. It is **not** a replacement for GLM-5.3-Flash. Expect wrong facts, broken arithmetic, over-long thinking and the occasional loop.

## Contents

- [Training lineage](#training-lineage)
- [Datasets](#datasets)
- [Architecture](#architecture)
- [Evaluation](#evaluation)
- [Compute](#compute)
- [Running it](#running-it)
- [Limitations](#limitations)
- [Repository contents](#repository-contents)
- [Citation](#citation)

## Training lineage

1. **Expert pruning (REAP-style).** Each of the 42 MoE layers of GLM-5.3-Flash (FP8) kept the 8 of its 288 routed experts with the highest router-weighted output norm (gate × ‖expert output‖), measured on a 102k-token calibration set. Router rows were cut to match. The shared expert, attention, the 3 dense layers, the MTP head and the vision encoder are unchanged. Held-out loss right after pruning: 3.725.
2. **Healing, stage 1.** LoRA (rank 64, alpha 128, on every attention and MLP projection, routed experts included) plus fully trained routers. AdamW, learning rate 1e-4, warmup and cosine decay, 2,048-token chunks, loss on assistant turns only. 174 steps, **10.9M tokens**. Held-out loss: 3.725 → **2.105**.
3. **Healing, stage 2.** Continued from stage 1 with a fresh optimizer on **12.1M new tokens** (no conversation from stage 1 repeated), learning rate 8e-5, 193 steps. Held-out loss on a new held-out set: 2.449 → **2.218**. **This is the checkpoint released here.**

The two held-out sets differ, so their losses are not comparable with each other.

## Datasets

**In total: 13 healing datasets and 3 calibration datasets** (one used for both). Every dataset that shaped these weights is listed here and in this page's metadata. Healing data is **only text written by GLM-5.3 or GLM-5.3-Flash**, with one exception noted below. Training data is not re-released.

### Healing

| Dataset | What it is | Stage 1 | Stage 2 |
|---|---|---:|---:|
| [Zek-Takai/glm53-flash-harvest](https://huggingface.co/datasets/Zek-Takai/glm53-flash-harvest) | Prose, chat, multi-turn, reasoning, code, structured output, agentic (GLM-5.3-Flash) | 65% | 69% |
| [shanjiaz/glm-5.3-flash-open-perfectblend-regen-25pct](https://huggingface.co/datasets/shanjiaz/glm-5.3-flash-open-perfectblend-regen-25pct) | Mixed instructions | 15% | 2% |
| [best-distill/glm-5.3-flash-distillation-chat](https://huggingface.co/datasets/best-distill/glm-5.3-flash-distillation-chat) | Instructions with reasoning | 7% | 3% |
| [open-athena/GLM-5.3-RLVR1-Training-Rollouts-2026.09.25](https://huggingface.co/datasets/open-athena/GLM-5.3-RLVR1-Training-Rollouts-2026.09.25) | Mixed tasks (GLM-5.3) | 5% | 10% |
| [sollamon/ox-alpha-glm-5.3-flash-distillation-coding-17k-raw](https://huggingface.co/datasets/sollamon/ox-alpha-glm-5.3-flash-distillation-coding-17k-raw) | Coding | 3% | 3% |
| [open-athena/GLM-5.3-tmax-swesmith-k8-traces-2026.10](https://huggingface.co/datasets/open-athena/GLM-5.3-tmax-swesmith-k8-traces-2026.10) | Terminal and bug-fix agent runs, passed only (GLM-5.3) | 1% | 7% |
| [liangzhidanta/claude-code-glm53-swesmith-trajectories](https://huggingface.co/datasets/liangzhidanta/claude-code-glm53-swesmith-trajectories) and [its compact-aware version](https://huggingface.co/datasets/liangzhidanta/claude-code-glm53-swesmith-compact-trajectories) | Coding-agent runs, some with context compaction (GLM-5.3) | – | 3% |
| [kaivoss/UltraData-SFT-Agent-2609-GLM53-20k](https://huggingface.co/datasets/kaivoss/UltraData-SFT-Agent-2609-GLM53-20k) | Agent runs in GLM-5.3's tool format. **Exception:** the text was written by the upstream models, not GLM | 2% | – |
| [Rallex3](https://huggingface.co/datasets/Rallex3/glm-5.3-flash-function-calling), [closestfriend](https://huggingface.co/datasets/closestfriend/glm-5.3-flash-function-calling), [FranckAbgrall](https://huggingface.co/datasets/FranckAbgrall/glm53-flash-function-calling), [flaviojoshua](https://huggingface.co/datasets/flaviojoshua/glm53-flash-function-calling), [Offlin33er](https://huggingface.co/datasets/Offlin33er/glm-5.3-flash-function-calling) function-calling sets | Tool calls (GLM-5.3-Flash) | 2% | 3% |

Shares are of each stage's tokens. Stage 1 drew 10.9M tokens from a 40M-token mix; stage 2 used 12.1M tokens from an 18M-token mix (the time budget ended the run first). In both stages, conversations where the model refused as an AI were dropped, and any sexual content involving minors was removed.

### Calibration for pruning (stage 1 only)

These decided which experts were kept. The model was **not trained** on them.

| Dataset | Share |
|---|---:|
| [Gryphe/Sonnet3.5-Charcard-Roleplay](https://huggingface.co/datasets/Gryphe/Sonnet3.5-Charcard-Roleplay) | 30% |
| [anthracite-org/c2_logs_32k_llama3_qwen2_v1.3](https://huggingface.co/datasets/anthracite-org/c2_logs_32k_llama3_qwen2_v1.3) | 15% |
| [Zek-Takai/glm53-flash-harvest](https://huggingface.co/datasets/Zek-Takai/glm53-flash-harvest) (multi-turn, chat, prose, reasoning) | 55% |

### Evaluation benchmarks

| Benchmark | Dataset |
|---|---|
| ARC-Challenge, ARC-Easy | [allenai/ai2_arc](https://huggingface.co/datasets/allenai/ai2_arc) |
| HellaSwag | [Rowan/hellaswag](https://huggingface.co/datasets/Rowan/hellaswag) |
| OpenBookQA | [allenai/openbookqa](https://huggingface.co/datasets/allenai/openbookqa) |
| PIQA | [baber/piqa](https://huggingface.co/datasets/baber/piqa) |
| SciQ | [allenai/sciq](https://huggingface.co/datasets/allenai/sciq) |
| WinoGrande | [allenai/winogrande](https://huggingface.co/datasets/allenai/winogrande) |

## Architecture

| Component | Value |
|---|---|
| Layers | 45 (3 dense + 42 MoE) |
| Width (hidden size) | 4096 |
| Experts | 8 routed + 1 shared per MoE layer, **all 8 routed experts active on every token** (from 288, top-8 of 288 in the base) |
| Expert hidden | 2048 |
| Attention | GLM-5.3-Flash hybrid sparse / linear attention (unchanged) |
| Extra heads | 1 MTP (multi-token prediction) layer, kept |
| Vision | GLM-5.3-Flash vision encoder (unchanged, not evaluated after pruning) |
| Vocabulary | 154,880 (GLM-5.3 tokenizer) |
| Context length | healed at 2,048 tokens; the base supports far more, but long context is untested after pruning |
| Parameters | 18.28B total: 16.06B layers, 1.27B embeddings and LM head, 0.39B MTP, 0.56B vision |

Because all 8 kept experts fire on every token, MP-02 runs at the speed of a dense 16B model, not a sparse MoE.

## Evaluation

Scored with [lm-evaluation-harness](https://github.com/EleutherAI/lm-evaluation-harness): zero-shot, the first 500 questions of each task, bf16 on one H100. The metric per task is the same as MP-01's harness: accuracy for ARC-Easy and WinoGrande, length-normalised accuracy for the rest. With 500 questions a task, differences under about 2 points are within noise. MP-01 was scored by its own harness on the same tasks, metrics and sample size, so treat small differences between the two as noise.

### Comparison with MP-01

| Rank | Model | Params | ARC-C | ARC-E | HellaSwag | OBQA | PIQA | SciQ | Wino | Avg (7) |
|---|---|---|---:|---:|---:|---:|---:|---:|---:|---:|
| 1 | MP-02 16B Preview (this model) | 18.28B | 27.0 | 48.4 | **47.0** | 29.2 | 63.2 | **89.0** | **55.2** | **51.29** |
| 2 | [MP-01 100M Preview](https://huggingface.co/Manprot/MP-01-100M-Preview) | 100.6M | **29.4** | **56.4** | 31.8 | **30.2** | **64.4** | 73.0 | 53.8 | 48.43 |

Bold is the best score in each column.

### Where this model stands

- **7-task average:** about 3 points above MP-01, mostly from HellaSwag (+15.2) and SciQ (+16.0). It is behind MP-01 on ARC-Easy (−8.0) and slightly on ARC-Challenge, OpenBookQA and PIQA.
- **That is a low result for its size.** ARC-Challenge is near chance (25% for 4 choices). GLM-5.3-Flash itself was not run on this setup, but its published scores are far higher, so the pruning loss is large.
- These are multiple-choice log-likelihood tests. They don't measure chat, roleplay or thinking, which is what the healing focused on.

### Progress across checkpoints of this model

| Checkpoint | Held-out loss | ARC-C | ARC-E | HellaSwag | OBQA | PIQA | SciQ | Wino | Avg (7) |
|---|---:|---:|---:|---:|---:|---:|---:|---:|---:|
| Stage 1, step 174 | 2.105 (set A) | 27.0 | 48.0 | **47.2** | 27.4 | **64.4** | **89.4** | **56.8** | **51.46** |
| Stage 2, step 193 (this release) | 2.218 (set B) | 27.0 | **48.4** | 47.0 | **29.2** | 63.2 | 89.0 | 55.2 | 51.29 |

Stage 2 left the benchmarks where they were (all changes are within noise) and lowered loss on its own held-out set from 2.449 to 2.218.

## Compute

| Stage | Hardware | GPU hours | Tokens |
|---|---|---:|---:|
| Expert selection and pruning | Local consumer GPU (8 GB) | about 1.9 | 102k (calibration) |
| Healing, stage 1 (3 runs) | Cloud B200 | 2.85 | 10.9M |
| Healing, stage 2 (this release) | Cloud B200 | 2.60 | 12.1M |
| Benchmarks | Cloud H100 | 0.12 per checkpoint | |

Hours are wall-clock GPU time from the run logs, including model loading. Costs in credits are not listed.

## Running it

### llama.cpp (GGUF)

```bash
hf download Manprot/MP-02-16B-Preview MP-02-16B-Preview-Q4_K_M.gguf --local-dir MP-02
llama-server -m MP-02/MP-02-16B-Preview-Q4_K_M.gguf -ngl 99 --n-cpu-moe 99 -c 8192
```

Needs a llama.cpp build with `glm5-next` support. On a desktop with an 8 GB GPU and DDR5 memory, keeping the experts on the CPU (`--n-cpu-moe 99`), it runs at about 6–7 tokens/s. On a single 24 GB GPU with everything on the GPU, it runs at about 28 tokens/s. For images, add `--mmproj mmproj-MP-02-16B-Preview-BF16.gguf`.

### transformers (safetensors)

```python
import torch
from transformers import AutoTokenizer, Glm5NextForConditionalGeneration
tok = AutoTokenizer.from_pretrained("Manprot/MP-02-16B-Preview")
model = Glm5NextForConditionalGeneration.from_pretrained("Manprot/MP-02-16B-Preview", dtype=torch.bfloat16, device_map="auto")
msgs = [{"role": "user", "content": "Write a short poem about the sea."}]
ids = tok.apply_chat_template(msgs, add_generation_prompt=True, reasoning_effort="low", return_tensors="pt").to(model.device)
print(tok.decode(model.generate(ids, max_new_tokens=400)[0][ids.shape[1]:]))
```

Needs transformers 5.17 or newer and about 37 GB of GPU memory in bf16.

### Thinking and the chat template

MP-02 uses GLM-5.3's chat template, which **always** starts a thinking block and, with no `reasoning_effort`, sends `Reasoning Effort: Max`. All healing data was rendered at Max, so by default it thinks long on everything, even simple requests. For normal chat, pass `reasoning_effort: "low"` (for example `"chat_template_kwargs": {"reasoning_effort": "low"}` on llama-server). A thinking on/off switch in a chat UI has no effect with this template.

Suggested sampling: temperature 0.7, top-p 0.8, repeat penalty 1.05.

## Limitations

- **Preview quality.** Most of GLM-5.3-Flash's knowledge is gone. It invents facts, gets simple arithmetic wrong and can talk confidently about things it does not know.
- **Over-thinking.** It thinks at maximum effort by default (see above) and can spend its whole token budget thinking before it answers.
- **Agent habits.** Agent traces in the healing data sometimes leak into chat: invented commands, sandbox paths such as `/home/user` or `/testbed`, and fake tool output.
- **Loops.** It can repeat a phrase until the token limit. A repeat penalty helps.
- **Identity.** It was trained on GLM's own conversations and may say it is GLM, made by Z.ai.
- **Few refusals.** AI refusals were removed from the healing data, so it follows most requests. It has not been checked for safety; filter outputs yourself.
- **Untested after pruning:** vision, tool calling, long context and languages other than English.

## Repository contents

- `model-*.safetensors`, `model.safetensors.index.json`: merged weights in bf16 (norms in fp32)
- `config.json`, `generation_config.json`, `tokenizer.json`, `tokenizer_config.json`, `chat_template.jinja`: model configuration and tokenizer
- `MP-02-16B-Preview-Q4_K_M.gguf`: 4-bit GGUF for llama.cpp (12.9 GB)
- `mmproj-MP-02-16B-Preview-BF16.gguf`: vision projector for llama.cpp (optional)
- `eval_results.json`: full evaluation output for this checkpoint
- `LICENSE`: MIT license with Z.ai's copyright notice
- `manprot-logo.png`: logo

## Citation

Made by the **[Manprot](https://huggingface.co/Manprot) team**, based on [GLM-5.3-Flash](https://huggingface.co/zai-org/GLM-5.3-Flash) by Z.ai. All credit for the original architecture and weights goes to Z.ai.

```bibtex
@misc{manprot2026mp02,
  title        = {MP-02 16B Preview},
  author       = {{Manprot team}},
  year         = {2026},
  publisher    = {Hugging Face},
  howpublished = {\url{https://huggingface.co/Manprot/MP-02-16B-Preview}}
}
```

The datasets listed above belong to their authors and keep their own licenses.

	---
	license: mit
	base_model: zai-org/GLM-5.3-Flash
	base_model_relation: finetune
	library_name: transformers
	pipeline_tag: image-text-to-text
	language:
	- en
	tags:
	- glm
	- moe
	- expert-pruning
	- reap
	- gguf
	- preview
	- roleplay
	datasets:
	- Zek-Takai/glm53-flash-harvest
	- shanjiaz/glm-5.3-flash-open-perfectblend-regen-25pct
	- best-distill/glm-5.3-flash-distillation-chat
	- open-athena/GLM-5.3-RLVR1-Training-Rollouts-2026.09.25
	- sollamon/ox-alpha-glm-5.3-flash-distillation-coding-17k-raw
	- open-athena/GLM-5.3-tmax-swesmith-k8-traces-2026.10
	- liangzhidanta/claude-code-glm53-swesmith-trajectories
	- liangzhidanta/claude-code-glm53-swesmith-compact-trajectories
	- kaivoss/UltraData-SFT-Agent-2609-GLM53-20k
	- Rallex3/glm-5.3-flash-function-calling
	- closestfriend/glm-5.3-flash-function-calling
	- FranckAbgrall/glm53-flash-function-calling
	- flaviojoshua/glm53-flash-function-calling
	- Offlin33er/glm-5.3-flash-function-calling
	- Gryphe/Sonnet3.5-Charcard-Roleplay
	- anthracite-org/c2_logs_32k_llama3_qwen2_v1.3
	---

	<p align="center"><img src="manprot-logo.png" alt="Manprot" width="360"></p>

	<h1 align="center">MP-02 16B Preview</h1>

	<p align="center">
	A pruned and healed version of <a href="https://huggingface.co/zai-org/GLM-5.3-Flash">GLM-5.3-Flash</a>, made by the <a href="https://huggingface.co/Manprot">Manprot</a> team
	</p>

	\| \| \|
	\|---\|---\|
	\| Made by \| The [Manprot](https://huggingface.co/Manprot) team \|
	\| Family \| MP-02 (see also [MP-01 100M Preview](https://huggingface.co/Manprot/MP-01-100M-Preview)) \|
	\| Status \| Preview: it works, but it's rough (see [Limitations](#limitations)) \|
	\| Type \| Chat model with thinking, healed on GLM-5.3 conversations \|
	\| Parameters \| 16.06B in the transformer layers; 18.28B in total with embeddings and LM head (1.27B), MTP head (0.39B) and vision encoder (0.56B) \|
	\| Architecture \| GLM-5.3-Flash (`glm5_next`) cut from 288 to 8 routed experts per MoE layer \|
	\| Built from \| [zai-org/GLM-5.3-Flash](https://huggingface.co/zai-org/GLM-5.3-Flash) (MIT) \|
	\| Tokens trained \| 22.9M (healing only; no pretraining of our own) \|
	\| 7-task average \| 51.29, about 3 points above MP-01 100M Preview \|
	\| License \| MIT, same as the base model (Z.ai copyright notice kept in `LICENSE`) \|

	> Note: this is a preview. It is somewhat decent at chat and roleplay, but cutting 97% of the experts cost it much of GLM-5.3-Flash's knowledge and reasoning. It is not a replacement for GLM-5.3-Flash. Expect wrong facts, broken arithmetic, over-long thinking and the occasional loop.

	## Contents

	- [Training lineage](#training-lineage)
	- [Datasets](#datasets)
	- [Architecture](#architecture)
	- [Evaluation](#evaluation)
	- [Compute](#compute)
	- [Running it](#running-it)
	- [Limitations](#limitations)
	- [Repository contents](#repository-contents)
	- [Citation](#citation)

	## Training lineage

	1. Expert pruning (REAP-style). Each of the 42 MoE layers of GLM-5.3-Flash (FP8) kept the 8 of its 288 routed experts with the highest router-weighted output norm (gate × ‖expert output‖), measured on a 102k-token calibration set. Router rows were cut to match. The shared expert, attention, the 3 dense layers, the MTP head and the vision encoder are unchanged. Held-out loss right after pruning: 3.725.
	2. Healing, stage 1. LoRA (rank 64, alpha 128, on every attention and MLP projection, routed experts included) plus fully trained routers. AdamW, learning rate 1e-4, warmup and cosine decay, 2,048-token chunks, loss on assistant turns only. 174 steps, 10.9M tokens. Held-out loss: 3.725 → 2.105.
	3. Healing, stage 2. Continued from stage 1 with a fresh optimizer on 12.1M new tokens (no conversation from stage 1 repeated), learning rate 8e-5, 193 steps. Held-out loss on a new held-out set: 2.449 → 2.218. This is the checkpoint released here.

	The two held-out sets differ, so their losses are not comparable with each other.

	## Datasets

	In total: 13 healing datasets and 3 calibration datasets (one used for both). Every dataset that shaped these weights is listed here and in this page's metadata. Healing data is only text written by GLM-5.3 or GLM-5.3-Flash, with one exception noted below. Training data is not re-released.

	### Healing

	\| Dataset \| What it is \| Stage 1 \| Stage 2 \|
	\|---\|---\|---:\|---:\|
	\| [Zek-Takai/glm53-flash-harvest](https://huggingface.co/datasets/Zek-Takai/glm53-flash-harvest) \| Prose, chat, multi-turn, reasoning, code, structured output, agentic (GLM-5.3-Flash) \| 65% \| 69% \|
	\| [shanjiaz/glm-5.3-flash-open-perfectblend-regen-25pct](https://huggingface.co/datasets/shanjiaz/glm-5.3-flash-open-perfectblend-regen-25pct) \| Mixed instructions \| 15% \| 2% \|
	\| [best-distill/glm-5.3-flash-distillation-chat](https://huggingface.co/datasets/best-distill/glm-5.3-flash-distillation-chat) \| Instructions with reasoning \| 7% \| 3% \|
	\| [open-athena/GLM-5.3-RLVR1-Training-Rollouts-2026.09.25](https://huggingface.co/datasets/open-athena/GLM-5.3-RLVR1-Training-Rollouts-2026.09.25) \| Mixed tasks (GLM-5.3) \| 5% \| 10% \|
	\| [sollamon/ox-alpha-glm-5.3-flash-distillation-coding-17k-raw](https://huggingface.co/datasets/sollamon/ox-alpha-glm-5.3-flash-distillation-coding-17k-raw) \| Coding \| 3% \| 3% \|
	\| [open-athena/GLM-5.3-tmax-swesmith-k8-traces-2026.10](https://huggingface.co/datasets/open-athena/GLM-5.3-tmax-swesmith-k8-traces-2026.10) \| Terminal and bug-fix agent runs, passed only (GLM-5.3) \| 1% \| 7% \|
	\| [liangzhidanta/claude-code-glm53-swesmith-trajectories](https://huggingface.co/datasets/liangzhidanta/claude-code-glm53-swesmith-trajectories) and [its compact-aware version](https://huggingface.co/datasets/liangzhidanta/claude-code-glm53-swesmith-compact-trajectories) \| Coding-agent runs, some with context compaction (GLM-5.3) \| – \| 3% \|
	\| [kaivoss/UltraData-SFT-Agent-2609-GLM53-20k](https://huggingface.co/datasets/kaivoss/UltraData-SFT-Agent-2609-GLM53-20k) \| Agent runs in GLM-5.3's tool format. Exception: the text was written by the upstream models, not GLM \| 2% \| – \|
	\| [Rallex3](https://huggingface.co/datasets/Rallex3/glm-5.3-flash-function-calling), [closestfriend](https://huggingface.co/datasets/closestfriend/glm-5.3-flash-function-calling), [FranckAbgrall](https://huggingface.co/datasets/FranckAbgrall/glm53-flash-function-calling), [flaviojoshua](https://huggingface.co/datasets/flaviojoshua/glm53-flash-function-calling), [Offlin33er](https://huggingface.co/datasets/Offlin33er/glm-5.3-flash-function-calling) function-calling sets \| Tool calls (GLM-5.3-Flash) \| 2% \| 3% \|

	Shares are of each stage's tokens. Stage 1 drew 10.9M tokens from a 40M-token mix; stage 2 used 12.1M tokens from an 18M-token mix (the time budget ended the run first). In both stages, conversations where the model refused as an AI were dropped, and any sexual content involving minors was removed.

	### Calibration for pruning (stage 1 only)

	These decided which experts were kept. The model was not trained on them.

	\| Dataset \| Share \|
	\|---\|---:\|
	\| [Gryphe/Sonnet3.5-Charcard-Roleplay](https://huggingface.co/datasets/Gryphe/Sonnet3.5-Charcard-Roleplay) \| 30% \|
	\| [anthracite-org/c2_logs_32k_llama3_qwen2_v1.3](https://huggingface.co/datasets/anthracite-org/c2_logs_32k_llama3_qwen2_v1.3) \| 15% \|
	\| [Zek-Takai/glm53-flash-harvest](https://huggingface.co/datasets/Zek-Takai/glm53-flash-harvest) (multi-turn, chat, prose, reasoning) \| 55% \|

	### Evaluation benchmarks

	\| Benchmark \| Dataset \|
	\|---\|---\|
	\| ARC-Challenge, ARC-Easy \| [allenai/ai2_arc](https://huggingface.co/datasets/allenai/ai2_arc) \|
	\| HellaSwag \| [Rowan/hellaswag](https://huggingface.co/datasets/Rowan/hellaswag) \|
	\| OpenBookQA \| [allenai/openbookqa](https://huggingface.co/datasets/allenai/openbookqa) \|
	\| PIQA \| [baber/piqa](https://huggingface.co/datasets/baber/piqa) \|
	\| SciQ \| [allenai/sciq](https://huggingface.co/datasets/allenai/sciq) \|
	\| WinoGrande \| [allenai/winogrande](https://huggingface.co/datasets/allenai/winogrande) \|

	## Architecture

	\| Component \| Value \|
	\|---\|---\|
	\| Layers \| 45 (3 dense + 42 MoE) \|
	\| Width (hidden size) \| 4096 \|
	\| Experts \| 8 routed + 1 shared per MoE layer, all 8 routed experts active on every token (from 288, top-8 of 288 in the base) \|
	\| Expert hidden \| 2048 \|
	\| Attention \| GLM-5.3-Flash hybrid sparse / linear attention (unchanged) \|
	\| Extra heads \| 1 MTP (multi-token prediction) layer, kept \|
	\| Vision \| GLM-5.3-Flash vision encoder (unchanged, not evaluated after pruning) \|
	\| Vocabulary \| 154,880 (GLM-5.3 tokenizer) \|
	\| Context length \| healed at 2,048 tokens; the base supports far more, but long context is untested after pruning \|
	\| Parameters \| 18.28B total: 16.06B layers, 1.27B embeddings and LM head, 0.39B MTP, 0.56B vision \|

	Because all 8 kept experts fire on every token, MP-02 runs at the speed of a dense 16B model, not a sparse MoE.

	## Evaluation

	Scored with [lm-evaluation-harness](https://github.com/EleutherAI/lm-evaluation-harness): zero-shot, the first 500 questions of each task, bf16 on one H100. The metric per task is the same as MP-01's harness: accuracy for ARC-Easy and WinoGrande, length-normalised accuracy for the rest. With 500 questions a task, differences under about 2 points are within noise. MP-01 was scored by its own harness on the same tasks, metrics and sample size, so treat small differences between the two as noise.

	### Comparison with MP-01

	\| Rank \| Model \| Params \| ARC-C \| ARC-E \| HellaSwag \| OBQA \| PIQA \| SciQ \| Wino \| Avg (7) \|
	\|---\|---\|---\|---:\|---:\|---:\|---:\|---:\|---:\|---:\|---:\|
	\| 1 \| MP-02 16B Preview (this model) \| 18.28B \| 27.0 \| 48.4 \| 47.0 \| 29.2 \| 63.2 \| 89.0 \| 55.2 \| 51.29 \|
	\| 2 \| [MP-01 100M Preview](https://huggingface.co/Manprot/MP-01-100M-Preview) \| 100.6M \| 29.4 \| 56.4 \| 31.8 \| 30.2 \| 64.4 \| 73.0 \| 53.8 \| 48.43 \|

	Bold is the best score in each column.

	### Where this model stands

	- 7-task average: about 3 points above MP-01, mostly from HellaSwag (+15.2) and SciQ (+16.0). It is behind MP-01 on ARC-Easy (−8.0) and slightly on ARC-Challenge, OpenBookQA and PIQA.
	- That is a low result for its size. ARC-Challenge is near chance (25% for 4 choices). GLM-5.3-Flash itself was not run on this setup, but its published scores are far higher, so the pruning loss is large.
	- These are multiple-choice log-likelihood tests. They don't measure chat, roleplay or thinking, which is what the healing focused on.

	### Progress across checkpoints of this model

	\| Checkpoint \| Held-out loss \| ARC-C \| ARC-E \| HellaSwag \| OBQA \| PIQA \| SciQ \| Wino \| Avg (7) \|
	\|---\|---:\|---:\|---:\|---:\|---:\|---:\|---:\|---:\|---:\|
	\| Stage 1, step 174 \| 2.105 (set A) \| 27.0 \| 48.0 \| 47.2 \| 27.4 \| 64.4 \| 89.4 \| 56.8 \| 51.46 \|
	\| Stage 2, step 193 (this release) \| 2.218 (set B) \| 27.0 \| 48.4 \| 47.0 \| 29.2 \| 63.2 \| 89.0 \| 55.2 \| 51.29 \|

	Stage 2 left the benchmarks where they were (all changes are within noise) and lowered loss on its own held-out set from 2.449 to 2.218.

	## Compute

	\| Stage \| Hardware \| GPU hours \| Tokens \|
	\|---\|---\|---:\|---:\|
	\| Expert selection and pruning \| Local consumer GPU (8 GB) \| about 1.9 \| 102k (calibration) \|
	\| Healing, stage 1 (3 runs) \| Cloud B200 \| 2.85 \| 10.9M \|
	\| Healing, stage 2 (this release) \| Cloud B200 \| 2.60 \| 12.1M \|
	\| Benchmarks \| Cloud H100 \| 0.12 per checkpoint \| \|

	Hours are wall-clock GPU time from the run logs, including model loading. Costs in credits are not listed.

	## Running it

	### llama.cpp (GGUF)

	```bash
	hf download Manprot/MP-02-16B-Preview MP-02-16B-Preview-Q4_K_M.gguf --local-dir MP-02
	llama-server -m MP-02/MP-02-16B-Preview-Q4_K_M.gguf -ngl 99 --n-cpu-moe 99 -c 8192
	```

	Needs a llama.cpp build with `glm5-next` support. On a desktop with an 8 GB GPU and DDR5 memory, keeping the experts on the CPU (`--n-cpu-moe 99`), it runs at about 6–7 tokens/s. On a single 24 GB GPU with everything on the GPU, it runs at about 28 tokens/s. For images, add `--mmproj mmproj-MP-02-16B-Preview-BF16.gguf`.

	### transformers (safetensors)

	```python
	import torch
	from transformers import AutoTokenizer, Glm5NextForConditionalGeneration

	tok = AutoTokenizer.from_pretrained("Manprot/MP-02-16B-Preview")
	model = Glm5NextForConditionalGeneration.from_pretrained("Manprot/MP-02-16B-Preview", dtype=torch.bfloat16, device_map="auto")
	msgs = [{"role": "user", "content": "Write a short poem about the sea."}]
	ids = tok.apply_chat_template(msgs, add_generation_prompt=True, reasoning_effort="low", return_tensors="pt").to(model.device)
	print(tok.decode(model.generate(ids, max_new_tokens=400)[0][ids.shape[1]:]))
	```

	Needs transformers 5.17 or newer and about 37 GB of GPU memory in bf16.

	### Thinking and the chat template

	MP-02 uses GLM-5.3's chat template, which always starts a thinking block and, with no `reasoning_effort`, sends `Reasoning Effort: Max`. All healing data was rendered at Max, so by default it thinks long on everything, even simple requests. For normal chat, pass `reasoning_effort: "low"` (for example `"chat_template_kwargs": {"reasoning_effort": "low"}` on llama-server). A thinking on/off switch in a chat UI has no effect with this template.

	Suggested sampling: temperature 0.7, top-p 0.8, repeat penalty 1.05.

	## Limitations

	- Preview quality. Most of GLM-5.3-Flash's knowledge is gone. It invents facts, gets simple arithmetic wrong and can talk confidently about things it does not know.
	- Over-thinking. It thinks at maximum effort by default (see above) and can spend its whole token budget thinking before it answers.
	- Agent habits. Agent traces in the healing data sometimes leak into chat: invented commands, sandbox paths such as `/home/user` or `/testbed`, and fake tool output.
	- Loops. It can repeat a phrase until the token limit. A repeat penalty helps.
	- Identity. It was trained on GLM's own conversations and may say it is GLM, made by Z.ai.
	- Few refusals. AI refusals were removed from the healing data, so it follows most requests. It has not been checked for safety; filter outputs yourself.
	- Untested after pruning: vision, tool calling, long context and languages other than English.

	## Repository contents

	- `model-*.safetensors`, `model.safetensors.index.json`: merged weights in bf16 (norms in fp32)
	- `config.json`, `generation_config.json`, `tokenizer.json`, `tokenizer_config.json`, `chat_template.jinja`: model configuration and tokenizer
	- `MP-02-16B-Preview-Q4_K_M.gguf`: 4-bit GGUF for llama.cpp (12.9 GB)
	- `mmproj-MP-02-16B-Preview-BF16.gguf`: vision projector for llama.cpp (optional)
	- `eval_results.json`: full evaluation output for this checkpoint
	- `LICENSE`: MIT license with Z.ai's copyright notice
	- `manprot-logo.png`: logo

	## Citation

	Made by the [Manprot](https://huggingface.co/Manprot) team, based on [GLM-5.3-Flash](https://huggingface.co/zai-org/GLM-5.3-Flash) by Z.ai. All credit for the original architecture and weights goes to Z.ai.

	```bibtex
	@misc{manprot2026mp02,
	title = {MP-02 16B Preview},
	author = {{Manprot team}},
	year = {2026},
	publisher = {Hugging Face},
	howpublished = {\url{https://huggingface.co/Manprot/MP-02-16B-Preview}}
	}
	```

	The datasets listed above belong to their authors and keep their own licenses.