Instructions to use jj90/MiniCPM-V-4.7-1B-RKNN3-RK1828 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use jj90/MiniCPM-V-4.7-1B-RKNN3-RK1828 with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf jj90/MiniCPM-V-4.7-1B-RKNN3-RK1828 # Run inference directly in the terminal: llama cli -hf jj90/MiniCPM-V-4.7-1B-RKNN3-RK1828
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf jj90/MiniCPM-V-4.7-1B-RKNN3-RK1828 # Run inference directly in the terminal: llama cli -hf jj90/MiniCPM-V-4.7-1B-RKNN3-RK1828
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf jj90/MiniCPM-V-4.7-1B-RKNN3-RK1828 # Run inference directly in the terminal: ./build/bin/llama cli -hf jj90/MiniCPM-V-4.7-1B-RKNN3-RK1828
Use Docker
docker model run hf.co/jj90/MiniCPM-V-4.7-1B-RKNN3-RK1828
- Llama new
- LM Studio
- Jan
- vLLM
How to use jj90/MiniCPM-V-4.7-1B-RKNN3-RK1828 with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "jj90/MiniCPM-V-4.7-1B-RKNN3-RK1828" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "jj90/MiniCPM-V-4.7-1B-RKNN3-RK1828", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker
docker model run hf.co/jj90/MiniCPM-V-4.7-1B-RKNN3-RK1828
- Ollama
How to use jj90/MiniCPM-V-4.7-1B-RKNN3-RK1828 with Ollama:
ollama run hf.co/jj90/MiniCPM-V-4.7-1B-RKNN3-RK1828
- Unsloth Desktop
- Pi
How to use jj90/MiniCPM-V-4.7-1B-RKNN3-RK1828 with Pi:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf jj90/MiniCPM-V-4.7-1B-RKNN3-RK1828
Configure the model in Pi
# Install Pi: npm install -g @earendil-works/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "llama-cpp": { "baseUrl": "http://localhost:9931/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "jj90/MiniCPM-V-4.7-1B-RKNN3-RK1828" } ] } } }Run Pi
# Start Pi in your project directory: pi
- Docker Model Runner
How to use jj90/MiniCPM-V-4.7-1B-RKNN3-RK1828 with Docker Model Runner:
docker model run hf.co/jj90/MiniCPM-V-4.7-1B-RKNN3-RK1828
- Lemonade
How to use jj90/MiniCPM-V-4.7-1B-RKNN3-RK1828 with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull jj90/MiniCPM-V-4.7-1B-RKNN3-RK1828
Run and chat with the model
lemonade run user.MiniCPM-V-4.7-1B-RKNN3-RK1828-{{QUANT_TAG}}List all available models
lemonade list
- Hermes Agent
How to use jj90/MiniCPM-V-4.7-1B-RKNN3-RK1828 with Hermes Agent:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf jj90/MiniCPM-V-4.7-1B-RKNN3-RK1828
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:9931/v1 hermes config set model.default jj90/MiniCPM-V-4.7-1B-RKNN3-RK1828
Run Hermes
hermes
- Atomic Chat
- OpenClaw
How to use jj90/MiniCPM-V-4.7-1B-RKNN3-RK1828 with OpenClaw:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf jj90/MiniCPM-V-4.7-1B-RKNN3-RK1828
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:9931/v1 \ --custom-model-id "jj90/MiniCPM-V-4.7-1B-RKNN3-RK1828" \ --custom-provider-id llama-cpp \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
Download docs/DEPLOY.md from jj90/MiniCPM-V-4.7-1B-RKNN3-RK1828: direct link, hf CLI and curl.
- Browser
- Download file 6.35 kB
-
https://huggingface.co/jj90/MiniCPM-V-4.7-1B-RKNN3-RK1828/resolve/main/docs/DEPLOY.md
- Command line
-
hf download hf://jj90/MiniCPM-V-4.7-1B-RKNN3-RK1828/docs/DEPLOY.md
-
curl -L -o DEPLOY.md https://huggingface.co/jj90/MiniCPM-V-4.7-1B-RKNN3-RK1828/resolve/main/docs/DEPLOY.md
Deployment guide — MiniCPM-V 4.7 1B on RK1828
0. What you need
| Item | Requirement |
|---|---|
| NPU module | Rockchip RK1828 / RK1820 (tested on RM1828MC0-F, firmware 1.1.0) |
| Host | Windows PC, or RK3588 / RK3576 board. Ubuntu x86 cannot drive the module directly (conversion / simulator only) |
| Runtime | rknn3_110 — rkllm3-server.exe, matching toolkit / proxy / runtime / firmware versions |
| Session | Windows Session 1 (interactive logon or an InteractiveToken scheduled task) |
Component versions must match. A mismatched rknn3_110 typically fails with
a DLL-load error that prints nothing at all.
1. Layout
Put the runtime next to rknn/:
<root>\
rknn\ <- this repo's rknn/ folder (6 artifacts + start.cmd)
runtime\rknn3_110\bin\rkllm3-server.exe
runtime\rknn3_110\lib\*.dll
rknn/start.cmd looks for the runtime in ..\runtime\rknn3_110, then
..\..\runtime\rknn3_110, then the RK_RUNTIME environment variable.
If none is found it exits with an error rather than silently falling back
to a hard-coded drive.
Override the listen address / port with RK_HOST / RK_PORT
(defaults 0.0.0.0 / 8899).
cd /d <root>\rknn
start.cmd
Wait until the server is actually ready. The log can sit at
Weight sync chunk size resolved for a couple of minutes — do not treat that
as a hang, and do not run rknn-smi while weights are loading (it can wedge
the sync). Readiness is proven only by a real inference request.
2. Start the communication proxy
One rknn3_transfer_proxy process per machine, reuse it. Repeatedly
tearing it down and recreating it can drop the module off the USB bus.
If you use the RK1828 web console, let the console own the proxy.
3. Send an image request
The server exposes an OpenAI-compatible endpoint. messages[].content must be
an array, and the image goes in as a base64 data URI:
{
"messages": [{
"role": "user",
"content": [
{ "type": "text", "text": "图里是什么?" },
{ "type": "image_url", "image_url": { "url": "data:image/png;base64,<BASE64>" } }
]
}],
"max_tokens": 512
}
Do not put <image> / <|image_pad|> placeholder tokens into the text.
The server constructs them itself. Ready-made bodies are in host/:
cd /d <root>\host
host\make_test_images.py :: generates test_images.json (pure stdlib)
send_vlm_req.cmd orange :: -> 127.0.0.1:8899
send_vlm_req.cmd text 8888 :: override port
Use curl.exe --max-time. Windows PowerShell Invoke-WebRequest /
Invoke-RestMethod hang indefinitely against this server (no response, no
-OutFile), and -TimeoutSec does not always rescue it.
Expect roughly HTTP=200 TIME=0.6s for a simple image. A correct response
contains timings.prompt_n ≈ 218 — that is 196 visual tokens plus text tokens,
and is the evidence that vision embeddings really entered the LLM.
4. Verified reference results
| Check | Result |
|---|---|
| Orange image, "what colour" | 这张图是橙色。 |
| Black square image | 黑色正方形 |
| Cross-checks (orange asked for shape, square asked for colour) | correct, no language-prior guessing |
| Pure text "中国的首都是哪里" | 北京 |
finish_reason |
stop on simple cases |
| NPU, VLM alone | 1269 MB (exclusive differential; console shows 1303 incl. 34 MB baseline) |
| NPU, VLM + Qwen3-ASR-0.6B | 3458 MB |
| NPU, VLM + Qwen3-ASR-1.7B | 3353 MB (A/B/A reproducible) |
| Generation | ~90 tok/s, prefill ~1700 tok/s |
Five request/response pairs plus the full console acceptance run are documented in the upstream project docs; the ready-made request bodies here reproduce the two simplest ones.
5. Pitfalls (all hit in practice)
- Session 0 breaks the USB bulk channel. Starting the server from an SSH
session lands it in Session 0;
rknn3_initsucceeds but weight sync never completes. UseschtaskswithInteractiveToken(Session 1). Diagnose with a different, known-good model — if both fail identically, suspect the session, not the USB port or the firmware. tie_word_embeddingsat the top level. A converted graph without a reallm_headmakesload_llmsegfault silently. Seeconvert/README.md.- Vision output shape and token count. 3D
[1,N,1024]andN == image_ph * image_pw == 196. Seeconvert/README.md. - Placeholder tokens must match this model's own vocabulary. MiniCPM-V 4.7
uses bare
<image>/</image>/<|image_pad|>. Its GGUF vocabulary is a superset that also contains Qwen-VL's<|vision_start|>/<|vision_end|>; a detector that checks Qwen-VL first will silently rewrite a correctstart.cmdinto a broken one. -cis context length, not core mask. Use--core-mask/--core-mask2; 8 cores =0xFF. A zero core mask makesrknn3_model_initreturn-2..cmd/.bat/.ps1must be pure ASCII + CRLF, andPATHmust be set before launching the child process (the launch script does this).- Check
finish_reason. This is a 1B model with a<think>chain. Complex images can fillmax_tokensand returnfinish_reason=lengthwith the answer truncated. Increasingmax_tokensis not monotonically safer (256 worked for one OCR case where 512 stalled inside<think>). - Residual processes hold the port → "I changed the code but behaviour did
not change". Check
Get-NetTCPConnectionbefore restarting. - ASR device errors can leak NPU memory (
mem_sync ret=-10retry loop). Workaround:taskkill /F /IM rknn_qwen3_asr_demo.exe. Not fixed upstream. - Power loss may land the module in MaskROM (
2207:180B);upgrade_tool ufrecovers it in ~2 s. A genuine hard hang needs a physical power cycle.
6. Known limitations — read before filing issues
- The 196-token resample is a runtime-contract adapter. This build is not equivalent to the Hugging Face original MiniCPM-V 4.7 1B.
- canvas mRoPE, multi-resolution tiling, high-resolution OCR and dense multi-object counting were not verified.
- OCR is "characters readable" at best:
RK1828may come back asRK₁₈₂₈. - Only 448x448 single-image input was tested.