---
library_name: transformers
license: mit
tags:
- llm-compressor
- tiny-model
- mtp
- pr-3225
- test-fixture
---

# GLM-5-0.88B-MTP

Small BF16 test model using [zai-org/GLM-5](https://huggingface.co/zai-org/GLM-5), with a toy-trained backbone and synthetic MTP weights.

## Quantization recipe

Uses [LLM Compressor](https://github.com/vllm-project/llm-compressor/pull/3225) model-free quantization. MTP inference has not been verified for this fixture.

```python
from llmcompressor import model_free_ptq

MODEL_ID = "inference-optimization/GLM-5-0.88B-MTP"
SAVE_DIR = "GLM-5-0.88B-MTP-FP8-Dynamic-MFPTQ"

model_free_ptq(
    model_stub=MODEL_ID,
    save_directory=SAVE_DIR,
    scheme="FP8_DYNAMIC",
    ignore=[
        "lm_head",
        "model.embed_tokens",
        r"re:.*\.mlp\.gate$",
        r"re:.*\.eh_proj$",
        r"re:.*\.indexer\..*",
    ],
    max_workers=2,
    device="cuda",
)
```