Instructions to use openbmb/MiniCPM5-1B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use openbmb/MiniCPM5-1B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="openbmb/MiniCPM5-1B")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("openbmb/MiniCPM5-1B")
model = AutoModelForCausalLM.from_pretrained("openbmb/MiniCPM5-1B", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
AMD Developer Cloud
Local Apps Settings

vLLM

How to use openbmb/MiniCPM5-1B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "openbmb/MiniCPM5-1B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "openbmb/MiniCPM5-1B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/openbmb/MiniCPM5-1B

SGLang

How to use openbmb/MiniCPM5-1B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "openbmb/MiniCPM5-1B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "openbmb/MiniCPM5-1B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "openbmb/MiniCPM5-1B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "openbmb/MiniCPM5-1B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use openbmb/MiniCPM5-1B with Docker Model Runner:
```
docker model run hf.co/openbmb/MiniCPM5-1B
```

Why's this model #1 beating GPT Sol, Fable, Opus 5 & K3 with very close to having 0% hallucinations?

#25

by SuperPauly - opened 11 days ago

Discussion

SuperPauly

11 days ago

Is this a mistake?
https://artificialanalysis.ai/models/minicpm5-1b-non-reasoning

Why is it not hallucinating barley at all?

This is almost the ideal Database/Knowledge Retrieval Engine. Are these. Results real, or could their be a mistake?

This is SOTA level of truth telling but I do not understand why this isn't mentioned on the README.md benchmarks?

If these are valid results I would be more than happy to push an update to reflect the SOTA status on the README.md since it seems no one else has discovered or is talking about it, from what I can see anyway.

xcjthu

OpenBMB org 3 days ago

Thanks for pointing this out and for taking the time to look into the results.

The Omni-Science questions on Artificial Analysis are generally very challenging, especially for small models. For models at the 1B scale, many of these questions are simply beyond what the model can reliably answer.

Because of that, a small model can sometimes achieve a high “non-hallucination” rate by frequently refusing to answer or stating that it does not know, rather than by actually retrieving or reasoning through the correct knowledge. So the result is not necessarily a mistake, but it should not be interpreted as meaning the model has SOTA-level knowledge retrieval ability.

This reflects a limitation of our evaluation: high non-hallucination can sometimes reward conservative behavior, especially for smaller models. It is still useful, but it needs to be interpreted together with answer rate, accuracy, and other capability benchmarks.

For this reason, we have not highlighted it as a SOTA claim in the README. That said, improving the model’s ability to recognize its own knowledge boundaries is an important direction for us, and we will continue working on it.

Thanks again for raising this — it’s a valuable observation.

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment