---
license: apache-2.0
base_model: openbmb/MiniCPM5-2B
pipeline_tag: text-generation
library_name: rknn3
language:
- en
- zh
tags:
- rknn3
- rkllm
- rkllm3
- rkllm3-server
- llm
- rockchip
- rk1820
- rk1828
- minicpm
- minicpm5
- edge-ai
- on-device
- w4a16
---

# MiniCPM5-2B for RKNN3 / RKLLM3 Server

RKNN3-converted [openbmb/MiniCPM5-2B](https://huggingface.co/openbmb/MiniCPM5-2B) LLM artifacts for the Rockchip **RK1820** and **RK1828** AI coprocessors, ready for deployment with Rockchip's **RKLLM3 Server** (`rkllm3-server`) or the RKNN3 Runtime examples.

These are hardware-specific deployment artifacts, not a Transformers checkpoint and not an RKLLM model. They were produced with **rknn3-toolkit v1.1.0** for the new RKNN3 toolchain and are intended for use with a compatible RKNN3 Runtime, firmware, demo, or `rkllm3-server`.

## Base model

- Upstream model: [openbmb/MiniCPM5-2B](https://huggingface.co/openbmb/MiniCPM5-2B)
- Model type: causal language model (`LlamaForCausalLM`)
- Parameters: 2.52B total, 1.98B excluding embeddings
- Upstream context length: 131,072 tokens
- Upstream license: Apache-2.0

The usable context length of an RKNN3 deployment is determined by the converted model and runtime configuration. Do not assume that it is identical to the upstream checkpoint's maximum context length.

## Conversion

| Item | Value |
| --- | --- |
| Toolkit | [rknn3-toolkit v1.1.0](https://github.com/airockchip/rknn3-toolkit) |
| Conversion reference | [rknn3-model-zoo MiniCPM5 example](https://github.com/airockchip/rknn3-model-zoo/tree/main/examples/MiniCPM5) |
| Targets | RK1820 and RK1828 |
| Quantization | W4A16, `normal`, `group32` |
| Output format | Weight-separated RKNN3 (`.rknn` + `.weight`) |

RKNN3 is a separate toolchain from RKLLM Toolkit, RKNN-Toolkit, and RKNN-Toolkit2. Use RKNN3 v1.1.0 runtime components and RK182x firmware that are compatible with these converted artifacts.

## Files

```text
.
├── MiniCPM5-2B-w4a16.weight
├── MiniCPM5-2B.embed.bin
├── MiniCPM5-2B.tokenizer.gguf
├── RK1820/
│   └── MiniCPM5-2B-RK1820-w4a16.rknn
└── RK1828/
    └── MiniCPM5-2B-RK1828-w4a16.rknn
```

The tokenizer, FP16 embedding table, and W4A16 weight file are shared by both targets. The RK1820 and RK1828 conversions produced byte-identical `.weight` files, so this repository stores one copy at the root. The `.rknn` files differ and you must select the one matching your coprocessor.

| Target | RKNN graph | Shared weight | Tokenizer | Embedding |
| --- | --- | --- | --- | --- |
| RK1820 | [MiniCPM5-2B-RK1820-w4a16.rknn](RK1820/MiniCPM5-2B-RK1820-w4a16.rknn) | [MiniCPM5-2B-w4a16.weight](MiniCPM5-2B-w4a16.weight) | [MiniCPM5-2B.tokenizer.gguf](MiniCPM5-2B.tokenizer.gguf) | [MiniCPM5-2B.embed.bin](MiniCPM5-2B.embed.bin) |
| RK1828 | [MiniCPM5-2B-RK1828-w4a16.rknn](RK1828/MiniCPM5-2B-RK1828-w4a16.rknn) | [MiniCPM5-2B-w4a16.weight](MiniCPM5-2B-w4a16.weight) | [MiniCPM5-2B.tokenizer.gguf](MiniCPM5-2B.tokenizer.gguf) | [MiniCPM5-2B.embed.bin](MiniCPM5-2B.embed.bin) |

Keep all four files from this repository revision together. A model graph or weight from a different conversion may be incompatible even when its filename looks similar.

## Download

Install the current Hugging Face CLI and download the complete repository:

```bash
hf download HanzoHuang/MiniCPM5-2B-RKNN3 \
  --local-dir MiniCPM5-2B-RKNN3
```

To download only the RK1820 runtime set:

```bash
hf download HanzoHuang/MiniCPM5-2B-RKNN3 \
  RK1820/MiniCPM5-2B-RK1820-w4a16.rknn \
  MiniCPM5-2B-w4a16.weight \
  MiniCPM5-2B.tokenizer.gguf \
  MiniCPM5-2B.embed.bin \
  --local-dir MiniCPM5-2B-RKNN3
```

Replace `RK1820` with `RK1828` and use the corresponding filename for an RK1828 deployment.

## Run with rkllm3-server

The official `rkllm3-server` accepts the graph, separated weight, tokenizer, and embedding as explicit paths. For RK1820:

```bash
/usr/bin/rkllm3-server \
  -m MiniCPM5-2B-RKNN3/RK1820/MiniCPM5-2B-RK1820-w4a16.rknn \
  --weight MiniCPM5-2B-RKNN3/MiniCPM5-2B-w4a16.weight \
  --vocab MiniCPM5-2B-RKNN3/MiniCPM5-2B.tokenizer.gguf \
  --embed MiniCPM5-2B-RKNN3/MiniCPM5-2B.embed.bin
```

For RK1828, change only the graph path:

```bash
-m MiniCPM5-2B-RKNN3/RK1828/MiniCPM5-2B-RK1828-w4a16.rknn
```

Use the server binary, RKNN3 Runtime libraries, transfer proxy, and RK182x firmware supplied for the same compatible RKNN3 release. RK1820/RK1828 operate as coprocessors and require a supported host platform and connection method.

## Run the rknn3-model-zoo C++ demo

Build and deploy the official MiniCPM5 example as described by the [rknn3-model-zoo](https://github.com/airockchip/rknn3-model-zoo/tree/main/examples/MiniCPM5). From the deployed demo directory, an RK1820 invocation is:

```bash
export LD_LIBRARY_PATH=./lib:$LD_LIBRARY_PATH

./rknn_minicpm5_demo \
  /path/to/MiniCPM5-2B-RKNN3/RK1820/MiniCPM5-2B-RK1820-w4a16.rknn \
  /path/to/MiniCPM5-2B-RKNN3/MiniCPM5-2B-w4a16.weight \
  /path/to/MiniCPM5-2B-RKNN3/MiniCPM5-2B.tokenizer.gguf \
  /path/to/MiniCPM5-2B-RKNN3/MiniCPM5-2B.embed.bin \
  0xff \
  "Explain the theory of relativity"
```

For RK1828, select the graph under `RK1828/`. The `0xff` argument is the NPU core mask used in Rockchip's example; adjust it for your deployment when necessary.

## Compatibility and limitations

- These files require the RKNN3 software stack. They cannot be loaded by Transformers, llama.cpp, the legacy RKLLM Runtime, RKNN-Toolkit, or RKNN-Toolkit2.
- The `.rknn` graph is target-specific. Do not use the RK1820 graph on RK1828 or the RK1828 graph on RK1820.
- The root `.weight`, `.tokenizer.gguf`, and `.embed.bin` files are shared by the two graphs in this repository revision.
- Quantization can change generation quality relative to the upstream BF16 model. Validate accuracy and performance for your application.
- Runtime behavior depends on the installed RKNN3 Runtime, host software, transfer method, and RK182x firmware.

## References and acknowledgements

- [MiniCPM5-2B by OpenBMB](https://huggingface.co/openbmb/MiniCPM5-2B)
- [Rockchip rknn3-toolkit](https://github.com/airockchip/rknn3-toolkit)
- [Rockchip rknn3-model-zoo](https://github.com/airockchip/rknn3-model-zoo)
- [MiniCPM5 conversion and deployment example](https://github.com/airockchip/rknn3-model-zoo/tree/main/examples/MiniCPM5)

Thanks to OpenBMB, Rockchip, and the RKNN community.