README.md

---
license: apache-2.0
language:
  - en
  - zh
library_name: gguf
pipeline_tag: text-generation
base_model: Caldalis/MiniCPM5-2B-Math
base_model_relation: quantized
tags:
  - minicpm
  - minicpm5
  - math
  - reasoning
  - theorem-proving
  - gguf
  - llama.cpp
---

<div align="center">
<img src="https://huggingface.co/Caldalis/MiniCPM5-2B-Math/resolve/main/assets/banner.png" alt="MiniCPM5-2B-Math: a 2.5B model for math and proofs, with a proof swarm you can run yourself" width="100%" />
</div>

<p align="center">
English |
<a href="https://huggingface.co/Caldalis/MiniCPM5-2B-Math-GGUF/blob/main/README-cn.md" target="_blank">中文</a>
</p>

# MiniCPM5-2B-Math-GGUF

GGUF builds of [MiniCPM5-2B-Math](https://huggingface.co/Caldalis/MiniCPM5-2B-Math), a 2.5B math and proof
specialist, for llama.cpp, Ollama and LM Studio. Pair it with the
[MiniCPM-Math Harness](https://github.com/Caldalis/MiniCPM-Math-harness) to run a small proof swarm on a laptop.

| File | Size | Notes |
|---|---:|---|
| `MiniCPM5-2B-Math-Q4_K_M.gguf` | 1.6 GB | smallest; quality loss likely on long proofs |
| `MiniCPM5-2B-Math-Q8_0.gguf` | 2.7 GB | recommended: usually close to BF16, and used for our laptop tests |
| `MiniCPM5-2B-Math-BF16.gguf` | 5.0 GB | full precision |

Converted with llama.cpp b10964 (`convert_hf_to_gguf.py`, tokenizer pre-type `minicpm5`) and quantized with
`llama-quantize`. The recommended sampling settings (temperature 0.9, top-p 0.95, min-p 0, top-k off) are embedded in
the GGUF metadata, so llama.cpp uses them by default. All results on the
[model card](https://huggingface.co/Caldalis/MiniCPM5-2B-Math) were measured with the BF16 safetensors on vLLM.

| File | SHA-256 |
|---|---|
| `MiniCPM5-2B-Math-Q4_K_M.gguf` | `9ff4f86fd8eb3a16b1ff6b29d86366f3c963320e44a7325e6fb90c0142a7cf0b` |
| `MiniCPM5-2B-Math-Q8_0.gguf` | `db251717a4d11b3221e7e2f1d096c3aebec94b1f25e8e24b4ebc56c185d32b9d` |
| `MiniCPM5-2B-Math-BF16.gguf` | `0703334ee732e110ef18dee427a8736fae52fbdd8a6ce3510de0df45dcaac72c` |

## llama.cpp

```bash
llama-server -hf Caldalis/MiniCPM5-2B-Math-GGUF:Q8_0 --jinja -ngl 99 -c 98304 -np 4 -kvu --port 8000
```

- `--jinja` uses the embedded chat template (thinking mode, `<think>` … `</think>`).
- `-np 4 -kvu` gives 4 parallel slots that share a 96K-token KV cache: one slot per agent for the harness. Run the
  harness with `--profile quick` against this server.
- `--port 8000` is where the harness looks by default; without it, `llama-server` listens on 8080.
- The embedded defaults already set `min_p=0`; llama.cpp's usual 0.05 can encourage repetition with this model.

## Ollama

```bash
ollama run hf.co/Caldalis/MiniCPM5-2B-Math-GGUF:Q8_0
```

Start the server with a larger context window (for example `OLLAMA_CONTEXT_LENGTH=65536 ollama serve`), because
proofs routinely need tens of thousands of tokens. With the harness, pass
`--base-url http://127.0.0.1:11434/v1 --model hf.co/Caldalis/MiniCPM5-2B-Math-GGUF:Q8_0 --no-raw`.

## Recommended settings

`temperature=0.9, top_p=0.95, min_p=0.0`, thinking enabled, and a large `max_tokens`. The published benchmark
numbers let every call use the full 131,072-token context: a quarter of HMMT samples think past 64K tokens, and many
of those still end with the right answer. On a laptop, 16K–32K is a practical compromise.

## License

Apache-2.0 (see `LICENSE`).