Image-Text-to-Text
GGUF
English
Chinese
rknn
rknn3
rk1828
rk1820
rockchip
npu
w4a16
vision-language-model
multimodal
on-device
edge-ai
conversational
Instructions to use jj90/MiniCPM-V-4.7-1B-RKNN3-RK1828 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use jj90/MiniCPM-V-4.7-1B-RKNN3-RK1828 with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf jj90/MiniCPM-V-4.7-1B-RKNN3-RK1828 # Run inference directly in the terminal: llama cli -hf jj90/MiniCPM-V-4.7-1B-RKNN3-RK1828
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf jj90/MiniCPM-V-4.7-1B-RKNN3-RK1828 # Run inference directly in the terminal: llama cli -hf jj90/MiniCPM-V-4.7-1B-RKNN3-RK1828
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf jj90/MiniCPM-V-4.7-1B-RKNN3-RK1828 # Run inference directly in the terminal: ./build/bin/llama cli -hf jj90/MiniCPM-V-4.7-1B-RKNN3-RK1828
Use Docker
docker model run hf.co/jj90/MiniCPM-V-4.7-1B-RKNN3-RK1828
- Llama new
- LM Studio
- Jan
- vLLM
How to use jj90/MiniCPM-V-4.7-1B-RKNN3-RK1828 with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "jj90/MiniCPM-V-4.7-1B-RKNN3-RK1828" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "jj90/MiniCPM-V-4.7-1B-RKNN3-RK1828", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker
docker model run hf.co/jj90/MiniCPM-V-4.7-1B-RKNN3-RK1828
- Ollama
How to use jj90/MiniCPM-V-4.7-1B-RKNN3-RK1828 with Ollama:
ollama run hf.co/jj90/MiniCPM-V-4.7-1B-RKNN3-RK1828
- Unsloth Desktop
- Pi
How to use jj90/MiniCPM-V-4.7-1B-RKNN3-RK1828 with Pi:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf jj90/MiniCPM-V-4.7-1B-RKNN3-RK1828
Configure the model in Pi
# Install Pi: npm install -g @earendil-works/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "llama-cpp": { "baseUrl": "http://localhost:9931/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "jj90/MiniCPM-V-4.7-1B-RKNN3-RK1828" } ] } } }Run Pi
# Start Pi in your project directory: pi
- Docker Model Runner
How to use jj90/MiniCPM-V-4.7-1B-RKNN3-RK1828 with Docker Model Runner:
docker model run hf.co/jj90/MiniCPM-V-4.7-1B-RKNN3-RK1828
- Lemonade
How to use jj90/MiniCPM-V-4.7-1B-RKNN3-RK1828 with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull jj90/MiniCPM-V-4.7-1B-RKNN3-RK1828
Run and chat with the model
lemonade run user.MiniCPM-V-4.7-1B-RKNN3-RK1828-{{QUANT_TAG}}List all available models
lemonade list
- Hermes Agent
How to use jj90/MiniCPM-V-4.7-1B-RKNN3-RK1828 with Hermes Agent:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf jj90/MiniCPM-V-4.7-1B-RKNN3-RK1828
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:9931/v1 hermes config set model.default jj90/MiniCPM-V-4.7-1B-RKNN3-RK1828
Run Hermes
hermes
- Atomic Chat
- OpenClaw
How to use jj90/MiniCPM-V-4.7-1B-RKNN3-RK1828 with OpenClaw:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf jj90/MiniCPM-V-4.7-1B-RKNN3-RK1828
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:9931/v1 \ --custom-model-id "jj90/MiniCPM-V-4.7-1B-RKNN3-RK1828" \ --custom-provider-id llama-cpp \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
|
Download docs/DEPLOY.md from jj90/MiniCPM-V-4.7-1B-RKNN3-RK1828: direct link, hf CLI and curl.
- Browser
- Download file 6.35 kB
-
https://huggingface.co/jj90/MiniCPM-V-4.7-1B-RKNN3-RK1828/resolve/main/docs/DEPLOY.md
- Command line
-
hf download hf://jj90/MiniCPM-V-4.7-1B-RKNN3-RK1828/docs/DEPLOY.md
-
curl -L -o DEPLOY.md https://huggingface.co/jj90/MiniCPM-V-4.7-1B-RKNN3-RK1828/resolve/main/docs/DEPLOY.md
6.35 kB
| # Deployment guide — MiniCPM-V 4.7 1B on RK1828 | |
| ## 0. What you need | |
| | Item | Requirement | | |
| | --- | --- | | |
| | NPU module | Rockchip **RK1828 / RK1820** (tested on `RM1828MC0-F`, firmware 1.1.0) | | |
| | Host | Windows PC, or RK3588 / RK3576 board. **Ubuntu x86 cannot drive the module directly** (conversion / simulator only) | | |
| | Runtime | `rknn3_110` — `rkllm3-server.exe`, matching `toolkit / proxy / runtime / firmware` versions | | |
| | Session | Windows **Session 1** (interactive logon or an `InteractiveToken` scheduled task) | | |
| Component versions **must match**. A mismatched `rknn3_110` typically fails with | |
| a DLL-load error that prints **nothing at all**. | |
| ## 1. Layout | |
| Put the runtime next to `rknn/`: | |
| ```text | |
| <root>\ | |
| rknn\ <- this repo's rknn/ folder (6 artifacts + start.cmd) | |
| runtime\rknn3_110\bin\rkllm3-server.exe | |
| runtime\rknn3_110\lib\*.dll | |
| ``` | |
| `rknn/start.cmd` looks for the runtime in `..\runtime\rknn3_110`, then | |
| `..\..\runtime\rknn3_110`, then the `RK_RUNTIME` environment variable. | |
| If none is found it **exits with an error** rather than silently falling back | |
| to a hard-coded drive. | |
| Override the listen address / port with `RK_HOST` / `RK_PORT` | |
| (defaults `0.0.0.0` / `8899`). | |
| ```cmd | |
| cd /d <root>\rknn | |
| start.cmd | |
| ``` | |
| Wait until the server is actually ready. The log can sit at | |
| `Weight sync chunk size resolved` for a couple of minutes — **do not treat that | |
| as a hang**, and do not run `rknn-smi` while weights are loading (it can wedge | |
| the sync). Readiness is proven only by a real inference request. | |
| ## 2. Start the communication proxy | |
| One `rknn3_transfer_proxy` process per machine, **reuse it**. Repeatedly | |
| tearing it down and recreating it can drop the module off the USB bus. | |
| If you use the RK1828 web console, let the console own the proxy. | |
| ## 3. Send an image request | |
| The server exposes an OpenAI-compatible endpoint. `messages[].content` must be | |
| an **array**, and the image goes in as a base64 data URI: | |
| ```json | |
| { | |
| "messages": [{ | |
| "role": "user", | |
| "content": [ | |
| { "type": "text", "text": "图里是什么?" }, | |
| { "type": "image_url", "image_url": { "url": "data:image/png;base64,<BASE64>" } } | |
| ] | |
| }], | |
| "max_tokens": 512 | |
| } | |
| ``` | |
| **Do not put `<image>` / `<|image_pad|>` placeholder tokens into the text.** | |
| The server constructs them itself. Ready-made bodies are in `host/`: | |
| ```cmd | |
| cd /d <root>\host | |
| host\make_test_images.py :: generates test_images.json (pure stdlib) | |
| send_vlm_req.cmd orange :: -> 127.0.0.1:8899 | |
| send_vlm_req.cmd text 8888 :: override port | |
| ``` | |
| Use **`curl.exe --max-time`**. Windows PowerShell `Invoke-WebRequest` / | |
| `Invoke-RestMethod` hang indefinitely against this server (no response, no | |
| `-OutFile`), and `-TimeoutSec` does not always rescue it. | |
| Expect roughly `HTTP=200 TIME=0.6s` for a simple image. A correct response | |
| contains `timings.prompt_n ≈ 218` — that is 196 visual tokens plus text tokens, | |
| and is the evidence that vision embeddings really entered the LLM. | |
| ## 4. Verified reference results | |
| | Check | Result | | |
| | --- | --- | | |
| | Orange image, "what colour" | `这张图是橙色。` | | |
| | Black square image | `黑色正方形` | | |
| | Cross-checks (orange asked for shape, square asked for colour) | correct, no language-prior guessing | | |
| | Pure text "中国的首都是哪里" | `北京` | | |
| | `finish_reason` | `stop` on simple cases | | |
| | NPU, VLM alone | **1269 MB** (exclusive differential; console shows 1303 incl. 34 MB baseline) | | |
| | NPU, VLM + Qwen3-ASR-0.6B | 3458 MB | | |
| | NPU, VLM + Qwen3-ASR-1.7B | 3353 MB (A/B/A reproducible) | | |
| | Generation | ~90 tok/s, prefill ~1700 tok/s | | |
| Five request/response pairs plus the full console acceptance run are documented | |
| in the upstream project docs; the ready-made request bodies here reproduce the | |
| two simplest ones. | |
| ## 5. Pitfalls (all hit in practice) | |
| 1. **Session 0 breaks the USB bulk channel.** Starting the server from an SSH | |
| session lands it in Session 0; `rknn3_init` succeeds but weight sync never | |
| completes. Use `schtasks` with `InteractiveToken` (Session 1). Diagnose with | |
| a *different, known-good* model — if both fail identically, suspect the | |
| session, not the USB port or the firmware. | |
| 2. **`tie_word_embeddings` at the top level.** A converted graph without a real | |
| `lm_head` makes `load_llm` **segfault silently**. See `convert/README.md`. | |
| 3. **Vision output shape and token count.** 3D `[1,N,1024]` and | |
| `N == image_ph * image_pw == 196`. See `convert/README.md`. | |
| 4. **Placeholder tokens must match this model's own vocabulary.** MiniCPM-V 4.7 | |
| uses bare `<image>` / `</image>` / `<|image_pad|>`. Its GGUF vocabulary is a | |
| **superset** that also contains Qwen-VL's `<|vision_start|>` / | |
| `<|vision_end|>`; a detector that checks Qwen-VL first will silently rewrite | |
| a correct `start.cmd` into a broken one. | |
| 5. **`-c` is context length, not core mask.** Use `--core-mask` / | |
| `--core-mask2`; 8 cores = `0xFF`. A zero core mask makes | |
| `rknn3_model_init` return `-2`. | |
| 6. **`.cmd` / `.bat` / `.ps1` must be pure ASCII + CRLF**, and `PATH` must be | |
| set before launching the child process (the launch script does this). | |
| 7. **Check `finish_reason`.** This is a 1B model with a `<think>` chain. Complex | |
| images can fill `max_tokens` and return `finish_reason=length` with the | |
| answer truncated. Increasing `max_tokens` is **not** monotonically safer | |
| (256 worked for one OCR case where 512 stalled inside `<think>`). | |
| 8. **Residual processes hold the port** → "I changed the code but behaviour did | |
| not change". Check `Get-NetTCPConnection` before restarting. | |
| 9. **ASR device errors can leak NPU memory** (`mem_sync ret=-10` retry loop). | |
| Workaround: `taskkill /F /IM rknn_qwen3_asr_demo.exe`. Not fixed upstream. | |
| 10. **Power loss may land the module in MaskROM** (`2207:180B`); `upgrade_tool uf` | |
| recovers it in ~2 s. A genuine hard hang needs a physical power cycle. | |
| ## 6. Known limitations — read before filing issues | |
| - The 196-token resample is a **runtime-contract adapter**. This build is | |
| **not equivalent to the Hugging Face original MiniCPM-V 4.7 1B**. | |
| - canvas mRoPE, multi-resolution tiling, high-resolution OCR and dense | |
| multi-object counting were **not verified**. | |
| - OCR is "characters readable" at best: `RK1828` may come back as `RK₁₈₂₈`. | |
| - Only 448x448 single-image input was tested. | |