Text Generation
Transformers
Safetensors
English
Chinese
llama
minicpm
minicpm5
math
reasoning
theorem-proving
long-context
on-device
conversational
text-generation-inference
Instructions to use Caldalis/MiniCPM5-2B-Math with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use Caldalis/MiniCPM5-2B-Math with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="Caldalis/MiniCPM5-2B-Math") messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("Caldalis/MiniCPM5-2B-Math") model = AutoModelForCausalLM.from_pretrained("Caldalis/MiniCPM5-2B-Math", device_map="auto") messages = [ {"role": "user", "content": "Who are you?"}, ] inputs = tokenizer.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=256) print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use Caldalis/MiniCPM5-2B-Math with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "Caldalis/MiniCPM5-2B-Math" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Caldalis/MiniCPM5-2B-Math", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/Caldalis/MiniCPM5-2B-Math
- SGLang
How to use Caldalis/MiniCPM5-2B-Math with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "Caldalis/MiniCPM5-2B-Math" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Caldalis/MiniCPM5-2B-Math", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "Caldalis/MiniCPM5-2B-Math" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Caldalis/MiniCPM5-2B-Math", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Docker Model Runner
How to use Caldalis/MiniCPM5-2B-Math with Docker Model Runner:
docker model run hf.co/Caldalis/MiniCPM5-2B-Math
Update README
Browse files- README-cn.md +3 -6
- README.md +8 -9
README-cn.md
CHANGED
|
@@ -22,7 +22,7 @@
|
|
| 22 |
2.5B 稠密模型。单张消费级显卡或一台笔记本就能运行,保留原生 128k 上下文。
|
| 23 |
|
| 24 |
- **同尺寸下很强。** 作为最多思考 128K token 的普通单次调用,AIME 2025 92.3、AIME 2026 89.8、HMMT 2026 年 2 月赛 81.8
|
| 25 |
-
(avg@16),16 样本投票为 100 / 93.3 / 90.9。
|
| 26 |
- **能写出可核查的证明。** 在 IMO-ProofBench Basic 上,一次最多 128K token 的普通调用平均能为 30 题中的 21 题写出完整证明,
|
| 27 |
证明蜂群为 23 题(AI 评审盲评)。
|
| 28 |
- **自带"证明蜂群"。** 开源的 [MiniCPM-Math harness](https://github.com/Caldalis/MiniCPM-Math-harness) 可以在任意兼容
|
|
@@ -94,8 +94,7 @@ curl http://localhost:8000/v1/chat/completions \
|
|
| 94 |
}'
|
| 95 |
```
|
| 96 |
|
| 97 |
-
vLLM 0.24.0 + RTX 4090 实测:单卡、短输入时单流解码约 155 tok/s
|
| 98 |
-
eager 模式只有约 48 tok/s)。在五张 4090、最多 10 路并发的长证明任务中,单请求解码速度中位数约 118 tok/s,总吞吐约 900 tok/s。
|
| 99 |
|
| 100 |
### SGLang
|
| 101 |
|
|
@@ -110,9 +109,7 @@ python -m sglang.launch_server --model-path Caldalis/MiniCPM5-2B-Math --context-
|
|
| 110 |
llama-server -hf Caldalis/MiniCPM5-2B-Math-GGUF:Q8_0 --jinja -ngl 99 -c 98304 -np 4 -kvu --port 8000
|
| 111 |
```
|
| 112 |
|
| 113 |
-
8000 是 harness 默认连接的端口(不加 `--port` 时 llama-server 监听 8080);配合这个服务运行 harness 时请用 `--profile quick`。
|
| 114 |
-
请使用 llama.cpp b9833 或更新版本:b9354 加入了 MiniCPM5 分词器,更早的版本加载不了这个 GGUF;b9833 加入了把推理和答案分开的
|
| 115 |
-
chat parser。Apple M5 笔记本(16 GB)+ Q8_0 实测:单流约 46 tok/s,4 个 slot 并发总计约 125 tok/s。llama.cpp 默认的
|
| 116 |
`min_p=0.05` 容易引发复读,请在请求中设置 `min_p=0.0`(harness 会自动设置)。
|
| 117 |
|
| 118 |
### Transformers
|
|
|
|
| 22 |
2.5B 稠密模型。单张消费级显卡或一台笔记本就能运行,保留原生 128k 上下文。
|
| 23 |
|
| 24 |
- **同尺寸下很强。** 作为最多思考 128K token 的普通单次调用,AIME 2025 92.3、AIME 2026 89.8、HMMT 2026 年 2 月赛 81.8
|
| 25 |
+
(avg@16),16 样本投票为 100 / 93.3 / 90.9。整体 AIME,HMMT 测评上超越原版通用模型 MiniCPM5-2B。
|
| 26 |
- **能写出可核查的证明。** 在 IMO-ProofBench Basic 上,一次最多 128K token 的普通调用平均能为 30 题中的 21 题写出完整证明,
|
| 27 |
证明蜂群为 23 题(AI 评审盲评)。
|
| 28 |
- **自带"证明蜂群"。** 开源的 [MiniCPM-Math harness](https://github.com/Caldalis/MiniCPM-Math-harness) 可以在任意兼容
|
|
|
|
| 94 |
}'
|
| 95 |
```
|
| 96 |
|
| 97 |
+
vLLM 0.24.0 + RTX 4090 实测:单卡、短输入时单流解码约 155 tok/s。在五张 4090、最多 10 路并发的长证明任务中,单请求解码速度中位数约 118 tok/s,总吞吐约 900 tok/s。
|
|
|
|
| 98 |
|
| 99 |
### SGLang
|
| 100 |
|
|
|
|
| 109 |
llama-server -hf Caldalis/MiniCPM5-2B-Math-GGUF:Q8_0 --jinja -ngl 99 -c 98304 -np 4 -kvu --port 8000
|
| 110 |
```
|
| 111 |
|
| 112 |
+
8000 是 harness 默认连接的端口(不加 `--port` 时 llama-server 监听 8080);配合这个服务运行 harness 时请用 `--profile quick`。Apple M5 笔记本(16 GB)+ Q8_0 实测:单流约 46 tok/s,4 个 slot 并发总计约 125 tok/s。llama.cpp 默认的
|
|
|
|
|
|
|
| 113 |
`min_p=0.05` 容易引发复读,请在请求中设置 `min_p=0.0`(harness 会自动设置)。
|
| 114 |
|
| 115 |
### Transformers
|
README.md
CHANGED
|
@@ -41,8 +41,8 @@ English |
|
|
| 41 |
writing. It runs on a single consumer GPU or a laptop and keeps the native 128K context.
|
| 42 |
|
| 43 |
- **Strong for its size.** As a plain single call with up to 128K tokens of thinking, 92.3 on AIME 2025, 89.8 on
|
| 44 |
-
AIME 2026 and 81.8 on HMMT February 2026 (avg@16), and 100 / 93.3 / 90.9 with a 16-sample vote.
|
| 45 |
-
general-purpose MiniCPM5-2B
|
| 46 |
- **Writes checkable proofs.** On IMO-ProofBench Basic, an average single call with up to 128K tokens writes a
|
| 47 |
complete proof for 21 of the 30 problems, and the proof swarm for 23, as graded blind by an AI grader.
|
| 48 |
- **Comes with a proof swarm.** The open-source [MiniCPM-Math Harness](https://github.com/Caldalis/MiniCPM-Math-harness) runs
|
|
@@ -119,9 +119,9 @@ curl http://localhost:8000/v1/chat/completions \
|
|
| 119 |
}'
|
| 120 |
```
|
| 121 |
|
| 122 |
-
Measured with vLLM 0.24.0 on RTX 4090s: about 155 tok/s single-stream decode on one GPU with short inputs
|
| 123 |
-
|
| 124 |
-
|
| 125 |
|
| 126 |
### SGLang
|
| 127 |
|
|
@@ -137,10 +137,9 @@ llama-server -hf Caldalis/MiniCPM5-2B-Math-GGUF:Q8_0 --jinja -ngl 99 -c 98304 -n
|
|
| 137 |
```
|
| 138 |
|
| 139 |
Port 8000 is where the harness looks by default (`llama-server` alone listens on 8080); with this server, run the
|
| 140 |
-
harness with `--profile quick`.
|
| 141 |
-
|
| 142 |
-
|
| 143 |
-
`min_p=0.05` can encourage repetition, so send `min_p=0.0` (the harness does this for you).
|
| 144 |
|
| 145 |
### Transformers
|
| 146 |
|
|
|
|
| 41 |
writing. It runs on a single consumer GPU or a laptop and keeps the native 128K context.
|
| 42 |
|
| 43 |
- **Strong for its size.** As a plain single call with up to 128K tokens of thinking, 92.3 on AIME 2025, 89.8 on
|
| 44 |
+
AIME 2026 and 81.8 on HMMT February 2026 (avg@16), and 100 / 93.3 / 90.9 with a 16-sample vote. Overall, it
|
| 45 |
+
outperforms the original general-purpose MiniCPM5-2B on AIME and HMMT.
|
| 46 |
- **Writes checkable proofs.** On IMO-ProofBench Basic, an average single call with up to 128K tokens writes a
|
| 47 |
complete proof for 21 of the 30 problems, and the proof swarm for 23, as graded blind by an AI grader.
|
| 48 |
- **Comes with a proof swarm.** The open-source [MiniCPM-Math Harness](https://github.com/Caldalis/MiniCPM-Math-harness) runs
|
|
|
|
| 119 |
}'
|
| 120 |
```
|
| 121 |
|
| 122 |
+
Measured with vLLM 0.24.0 on RTX 4090s: about 155 tok/s single-stream decode on one GPU with short inputs. On
|
| 123 |
+
long proof runs across five 4090s with up to 10 concurrent requests, the median request decoded at about 118 tok/s,
|
| 124 |
+
about 900 tok/s in total.
|
| 125 |
|
| 126 |
### SGLang
|
| 127 |
|
|
|
|
| 137 |
```
|
| 138 |
|
| 139 |
Port 8000 is where the harness looks by default (`llama-server` alone listens on 8080); with this server, run the
|
| 140 |
+
harness with `--profile quick`. On an Apple M5 laptop (16 GB) with Q8_0: about 46 tok/s for one stream, about
|
| 141 |
+
125 tok/s total with 4 slots. llama.cpp's default `min_p=0.05` can encourage repetition, so send `min_p=0.0` (the
|
| 142 |
+
harness does this for you).
|
|
|
|
| 143 |
|
| 144 |
### Transformers
|
| 145 |
|