Text Generation
Transformers
GGUF
English
code
agentic
tool-use
agent
minicpm
full-fine-tune
on-cpu
text-generation-inference
unsloth
llama
conversational
Instructions to use ITLL/MiniCPM5-1B-Agent-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use ITLL/MiniCPM5-1B-Agent-GGUF with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="ITLL/MiniCPM5-1B-Agent-GGUF")# pip install -U transformers accelerate # Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("ITLL/MiniCPM5-1B-Agent-GGUF", device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use ITLL/MiniCPM5-1B-Agent-GGUF with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf ITLL/MiniCPM5-1B-Agent-GGUF:Q8_0 # Run inference directly in the terminal: llama cli -hf ITLL/MiniCPM5-1B-Agent-GGUF:Q8_0
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf ITLL/MiniCPM5-1B-Agent-GGUF:Q8_0 # Run inference directly in the terminal: llama cli -hf ITLL/MiniCPM5-1B-Agent-GGUF:Q8_0
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf ITLL/MiniCPM5-1B-Agent-GGUF:Q8_0 # Run inference directly in the terminal: ./build/bin/llama cli -hf ITLL/MiniCPM5-1B-Agent-GGUF:Q8_0
Use Docker
docker model run hf.co/ITLL/MiniCPM5-1B-Agent-GGUF:Q8_0
- Llama new
- LM Studio
- Jan
- vLLM
How to use ITLL/MiniCPM5-1B-Agent-GGUF with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "ITLL/MiniCPM5-1B-Agent-GGUF" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "ITLL/MiniCPM5-1B-Agent-GGUF", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/ITLL/MiniCPM5-1B-Agent-GGUF:Q8_0
- SGLang
How to use ITLL/MiniCPM5-1B-Agent-GGUF with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "ITLL/MiniCPM5-1B-Agent-GGUF" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "ITLL/MiniCPM5-1B-Agent-GGUF", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "ITLL/MiniCPM5-1B-Agent-GGUF" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "ITLL/MiniCPM5-1B-Agent-GGUF", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Ollama
How to use ITLL/MiniCPM5-1B-Agent-GGUF with Ollama:
ollama run hf.co/ITLL/MiniCPM5-1B-Agent-GGUF:Q8_0
- Unsloth Desktop
- Pi
How to use ITLL/MiniCPM5-1B-Agent-GGUF with Pi:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf ITLL/MiniCPM5-1B-Agent-GGUF:Q8_0
Configure the model in Pi
# Install Pi: npm install -g @earendil-works/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "llama-cpp": { "baseUrl": "http://localhost:9931/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "ITLL/MiniCPM5-1B-Agent-GGUF:Q8_0" } ] } } }Run Pi
# Start Pi in your project directory: pi
- Docker Model Runner
How to use ITLL/MiniCPM5-1B-Agent-GGUF with Docker Model Runner:
docker model run hf.co/ITLL/MiniCPM5-1B-Agent-GGUF:Q8_0
- Lemonade
How to use ITLL/MiniCPM5-1B-Agent-GGUF with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull ITLL/MiniCPM5-1B-Agent-GGUF:Q8_0
Run and chat with the model
lemonade run user.MiniCPM5-1B-Agent-GGUF-Q8_0
List all available models
lemonade list
- Hermes Agent
How to use ITLL/MiniCPM5-1B-Agent-GGUF with Hermes Agent:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf ITLL/MiniCPM5-1B-Agent-GGUF:Q8_0
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:9931/v1 hermes config set model.default ITLL/MiniCPM5-1B-Agent-GGUF:Q8_0
Run Hermes
hermes
- Atomic Chat
- OpenClaw
How to use ITLL/MiniCPM5-1B-Agent-GGUF with OpenClaw:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf ITLL/MiniCPM5-1B-Agent-GGUF:Q8_0
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:9931/v1 \ --custom-model-id "ITLL/MiniCPM5-1B-Agent-GGUF:Q8_0" \ --custom-provider-id llama-cpp \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
File size: 5,657 Bytes
0467408 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 | """Unified content-block -> canonical {messages, tools} converter for agent SESSION logs whose turns use
content-blocks. Handles every variant seen across the recent-teacher pi-trace / Claude-Code dumps:
block types : text | thinking/reasoning | toolCall/tool_use/tool_call/function_call | tool_result
roles : system | developer(->system) | user | assistant | toolResult/tool(->our tool role)
tool names : bash/read/edit/write (pi-harness, already ours) | Bash/Edit/Read/Glob/Grep/WebSearch
(Claude-Code) | functions.Write (Kimi) -> all mapped to our served vocabulary.
Thinking blocks -> reasoning_content; toolCall -> structured tool_calls; toolResult/tool_result -> role:tool.
Tools are synthesized from the tool names actually called (arg-key union), since pi-harness has no tools list.
`session_to_example(raw_msgs)` takes an ORDERED list of raw message dicts (each {role, content, [toolName]})
and returns a validated {messages, tools} (or None). Session grouping lives in build_keepadds3.py.
"""
import json
import schema
TOOLMAP = {"bash": "bash", "read": "read", "edit": "edit", "write": "write", "glob": "glob", "grep": "grep",
"Bash": "bash", "Edit": "edit", "MultiEdit": "edit", "Read": "read", "Glob": "glob", "Grep": "grep",
"Write": "write", "WebSearch": "web_search", "WebFetch": "web_fetch", "NotebookEdit": "edit"}
def _norm_tool(name):
if not name:
return None
name = str(name)
if "." in name: # Kimi: "functions.Write" -> "Write"
name = name.split(".")[-1]
name = name.split(":")[0] # "functions.Write:0" style -> "Write"
return TOOLMAP.get(name, name)
def _extract(content):
"""-> (text, reasoning, calls[(name,args,id)], tool_results[(id,content)])"""
if isinstance(content, str):
return content, "", [], []
text, reasoning, calls, tres = [], [], [], []
if isinstance(content, list):
for b in content:
if not isinstance(b, dict):
continue
t = b.get("type")
if t == "text":
text.append(b.get("text", ""))
elif t in ("thinking", "reasoning"):
reasoning.append(b.get("thinking") or b.get("reasoning") or b.get("text") or "")
elif t in ("toolCall", "tool_use", "tool_call", "function_call"):
fn = b.get("function", {}) if isinstance(b.get("function"), dict) else {}
name = b.get("name") or b.get("toolName") or fn.get("name")
args = b.get("arguments")
if args is None:
args = b.get("input")
if args is None:
args = fn.get("arguments", {})
if isinstance(args, str):
try:
args = json.loads(args)
except Exception:
args = {"_raw": args}
calls.append((name, args if isinstance(args, dict) else {"_raw": args}, b.get("id")))
elif t == "tool_result":
c = b.get("content")
if isinstance(c, list):
c = "\n".join(bb.get("text", "") for bb in c if isinstance(bb, dict))
tres.append((b.get("tool_use_id"), c or ""))
return "\n".join(t for t in text if t), "\n".join(r for r in reasoning if r), calls, tres
def session_to_example(raw_msgs):
out = []
toolkeys = {}
id2name = {}
last_call = None
for m in raw_msgs:
if not isinstance(m, dict):
continue
role = m.get("role")
text, reasoning, calls, tres = _extract(m.get("content"))
if role in ("system", "developer"):
out.append({"role": "system", "content": text})
elif role == "user":
if tres: # Claude-Code: tool results ride in a user turn
for tid, c in tres:
out.append({"role": "tool", "name": id2name.get(tid, last_call or "tool"), "content": c})
if text.strip():
out.append({"role": "user", "content": text})
else:
out.append({"role": "user", "content": text})
elif role == "assistant":
tcs = []
for name, args, cid in calls:
nm = _norm_tool(name)
if not nm:
continue
tcs.append({"type": "function", "function": {"name": nm, "arguments": args}})
toolkeys.setdefault(nm, set()).update(args.keys() if isinstance(args, dict) else [])
if cid:
id2name[cid] = nm
last_call = nm
a = {"role": "assistant", "content": text}
if reasoning.strip():
a["reasoning_content"] = reasoning.strip()
if tcs:
a["tool_calls"] = tcs
if a["content"] or a.get("tool_calls") or a.get("reasoning_content"):
out.append(a)
elif role in ("toolResult", "tool", "toolresult", "tool_result"):
nm = _norm_tool(m.get("toolName")) or last_call or "tool"
out.append({"role": "tool", "name": nm, "content": text or ""})
tools = []
for nm, keys in toolkeys.items():
tools.append({"type": "function", "function": {
"name": nm, "description": nm,
"parameters": {"type": "object", "properties": {k: {"type": "string"} for k in sorted(keys)}, "required": []}}})
ex = {"messages": out}
if tools:
ex["tools"] = tools
ok, _ = schema.validate(ex)
return ex if ok else None
|