# Deployment guide — MiniCPM-V 4.7 1B on RK1828 ## 0. What you need | Item | Requirement | | --- | --- | | NPU module | Rockchip **RK1828 / RK1820** (tested on `RM1828MC0-F`, firmware 1.1.0) | | Host | Windows PC, or RK3588 / RK3576 board. **Ubuntu x86 cannot drive the module directly** (conversion / simulator only) | | Runtime | `rknn3_110` — `rkllm3-server.exe`, matching `toolkit / proxy / runtime / firmware` versions | | Session | Windows **Session 1** (interactive logon or an `InteractiveToken` scheduled task) | Component versions **must match**. A mismatched `rknn3_110` typically fails with a DLL-load error that prints **nothing at all**. ## 1. Layout Put the runtime next to `rknn/`: ```text \ rknn\ <- this repo's rknn/ folder (6 artifacts + start.cmd) runtime\rknn3_110\bin\rkllm3-server.exe runtime\rknn3_110\lib\*.dll ``` `rknn/start.cmd` looks for the runtime in `..\runtime\rknn3_110`, then `..\..\runtime\rknn3_110`, then the `RK_RUNTIME` environment variable. If none is found it **exits with an error** rather than silently falling back to a hard-coded drive. Override the listen address / port with `RK_HOST` / `RK_PORT` (defaults `0.0.0.0` / `8899`). ```cmd cd /d \rknn start.cmd ``` Wait until the server is actually ready. The log can sit at `Weight sync chunk size resolved` for a couple of minutes — **do not treat that as a hang**, and do not run `rknn-smi` while weights are loading (it can wedge the sync). Readiness is proven only by a real inference request. ## 2. Start the communication proxy One `rknn3_transfer_proxy` process per machine, **reuse it**. Repeatedly tearing it down and recreating it can drop the module off the USB bus. If you use the RK1828 web console, let the console own the proxy. ## 3. Send an image request The server exposes an OpenAI-compatible endpoint. `messages[].content` must be an **array**, and the image goes in as a base64 data URI: ```json { "messages": [{ "role": "user", "content": [ { "type": "text", "text": "图里是什么?" }, { "type": "image_url", "image_url": { "url": "data:image/png;base64," } } ] }], "max_tokens": 512 } ``` **Do not put `` / `<|image_pad|>` placeholder tokens into the text.** The server constructs them itself. Ready-made bodies are in `host/`: ```cmd cd /d \host host\make_test_images.py :: generates test_images.json (pure stdlib) send_vlm_req.cmd orange :: -> 127.0.0.1:8899 send_vlm_req.cmd text 8888 :: override port ``` Use **`curl.exe --max-time`**. Windows PowerShell `Invoke-WebRequest` / `Invoke-RestMethod` hang indefinitely against this server (no response, no `-OutFile`), and `-TimeoutSec` does not always rescue it. Expect roughly `HTTP=200 TIME=0.6s` for a simple image. A correct response contains `timings.prompt_n ≈ 218` — that is 196 visual tokens plus text tokens, and is the evidence that vision embeddings really entered the LLM. ## 4. Verified reference results | Check | Result | | --- | --- | | Orange image, "what colour" | `这张图是橙色。` | | Black square image | `黑色正方形` | | Cross-checks (orange asked for shape, square asked for colour) | correct, no language-prior guessing | | Pure text "中国的首都是哪里" | `北京` | | `finish_reason` | `stop` on simple cases | | NPU, VLM alone | **1269 MB** (exclusive differential; console shows 1303 incl. 34 MB baseline) | | NPU, VLM + Qwen3-ASR-0.6B | 3458 MB | | NPU, VLM + Qwen3-ASR-1.7B | 3353 MB (A/B/A reproducible) | | Generation | ~90 tok/s, prefill ~1700 tok/s | Five request/response pairs plus the full console acceptance run are documented in the upstream project docs; the ready-made request bodies here reproduce the two simplest ones. ## 5. Pitfalls (all hit in practice) 1. **Session 0 breaks the USB bulk channel.** Starting the server from an SSH session lands it in Session 0; `rknn3_init` succeeds but weight sync never completes. Use `schtasks` with `InteractiveToken` (Session 1). Diagnose with a *different, known-good* model — if both fail identically, suspect the session, not the USB port or the firmware. 2. **`tie_word_embeddings` at the top level.** A converted graph without a real `lm_head` makes `load_llm` **segfault silently**. See `convert/README.md`. 3. **Vision output shape and token count.** 3D `[1,N,1024]` and `N == image_ph * image_pw == 196`. See `convert/README.md`. 4. **Placeholder tokens must match this model's own vocabulary.** MiniCPM-V 4.7 uses bare `` / `` / `<|image_pad|>`. Its GGUF vocabulary is a **superset** that also contains Qwen-VL's `<|vision_start|>` / `<|vision_end|>`; a detector that checks Qwen-VL first will silently rewrite a correct `start.cmd` into a broken one. 5. **`-c` is context length, not core mask.** Use `--core-mask` / `--core-mask2`; 8 cores = `0xFF`. A zero core mask makes `rknn3_model_init` return `-2`. 6. **`.cmd` / `.bat` / `.ps1` must be pure ASCII + CRLF**, and `PATH` must be set before launching the child process (the launch script does this). 7. **Check `finish_reason`.** This is a 1B model with a `` chain. Complex images can fill `max_tokens` and return `finish_reason=length` with the answer truncated. Increasing `max_tokens` is **not** monotonically safer (256 worked for one OCR case where 512 stalled inside ``). 8. **Residual processes hold the port** → "I changed the code but behaviour did not change". Check `Get-NetTCPConnection` before restarting. 9. **ASR device errors can leak NPU memory** (`mem_sync ret=-10` retry loop). Workaround: `taskkill /F /IM rknn_qwen3_asr_demo.exe`. Not fixed upstream. 10. **Power loss may land the module in MaskROM** (`2207:180B`); `upgrade_tool uf` recovers it in ~2 s. A genuine hard hang needs a physical power cycle. ## 6. Known limitations — read before filing issues - The 196-token resample is a **runtime-contract adapter**. This build is **not equivalent to the Hugging Face original MiniCPM-V 4.7 1B**. - canvas mRoPE, multi-resolution tiling, high-resolution OCR and dense multi-object counting were **not verified**. - OCR is "characters readable" at best: `RK1828` may come back as `RK₁₈₂₈`. - Only 448x448 single-image input was tested.