| --- |
| license: apache-2.0 |
| base_model: Qwen/Qwen3.5-2B |
| pipeline_tag: image-text-to-text |
| library_name: rkllm |
| tags: |
| - rkllm |
| - rockchip |
| - rk3576 |
| - rk3588 |
| - qwen |
| - qwen3.5 |
| - multimodal |
| --- |
| |
| # Qwen3.5-2B · RKLLM |
|
|
| * Max context length: 4096 |
|
|
| Image and text on Rockchip RK3576 and RK3588, converted from [Qwen/Qwen3.5-2B](https://huggingface.co/Qwen/Qwen3.5-2B). |
|
|
| - **Conversion toolkit:** RKLLM Toolkit 1.3.0. |
| - **Inputs / output:** text or image + text → text. |
| - **License:** `apache-2.0`; the [upstream model card](https://huggingface.co/Qwen/Qwen3.5-2B) describes its terms and intended use. |
|
|
| ## Choose your files |
|
|
| Use the files for your board’s exact SoC. `w4a16` uses 4-bit weights and 16-bit activations; `w4a16_g128` adds a weight group size of 128. `w8a8` uses 8-bit weights and activations. File size below is download size, not total runtime RAM. |
|
|
| | Board | Quantization | Size | Language model | |
| | --- | --- | --- | --- | |
| | RK3576 | `w4a16_g128` | 2.21 GiB | [Qwen3.5-2B_RK3576_w4a16_g128.rkllm](RK3576/Qwen3.5-2B_RK3576_w4a16_g128.rkllm) | |
| | RK3576 | `w8a8` | 2.76 GiB | [Qwen3.5-2B_RK3576_w8a8.rkllm](RK3576/Qwen3.5-2B_RK3576_w8a8.rkllm) | |
| | RK3588 | `w8a8` | 2.74 GiB | [Qwen3.5-2B_RK3588_w8a8.rkllm](RK3588/Qwen3.5-2B_RK3588_w8a8.rkllm) | |
|
|
| For image input, also download the **vision encoder for the same SoC**. Keep the language model and vision encoder from this release together. |
|
|
| | Board | Size | Vision encoder | |
| | --- | --- | --- | |
| | RK3576 | 0.67 GiB | [Qwen3.5-2B_vision_RK3576.rknn](RK3576/Qwen3.5-2B_vision_RK3576.rknn) | |
| | RK3588 | 0.65 GiB | [Qwen3.5-2B_vision_RK3588.rknn](RK3588/Qwen3.5-2B_vision_RK3588.rknn) | |
|
|
| ONNX files are export intermediates; they are not needed to run these RKNN/RKLLM models. |
|
|
| ## Run on your board |
|
|
| With Docker installed on a matching Rockchip board, this example starts the **RK3576 w4a16_g128** model and its API on port 8001: |
| |
| ```bash |
| sudo docker run --rm -d --name rkllm \ |
| --privileged -v /dev:/dev -p 8001:8001 \ |
| ghcr.io/hanzo-huang/rkllm-docker/vlm/qwen3.5-2b:rk3576-w4a16-g128 |
| ``` |
| |
| The image includes its model files. See [rkllm-docker](https://github.com/Hanzo-Huang/rkllm-docker#readme) for API requests, image inputs, other board variants, and configuration. |
| |
| To download the files separately instead: |
| |
| ```bash |
| hf download HanzoHuang/Qwen3.5-2B-RKLLM \ |
| RK3576/Qwen3.5-2B_RK3576_w4a16_g128.rkllm \ |
| RK3576/Qwen3.5-2B_vision_RK3576.rknn \ |
| --local-dir ./models |
| ``` |
| |
| These are native RKLLM/RKNN artifacts. Use a compatible RKLLM runtime and the model’s correct chat template when integrating them in another application. |
| |
| ## Release notes |
| |
| No per-variant board benchmark is published here. Quantized output may differ from the original model; check responses on your own workloads. Use the toolkit version listed above when checking runtime compatibility. |
| |
| <details> |
| <summary>SHA-256 checksums</summary> |
| |
| ```text |
| 76f40bbbc559615767e49a9408d28a1afc16c3a12aa3764a11e13f52acd868e9 RK3576/Qwen3.5-2B_RK3576_w4a16_g128.rkllm |
| 95f4c10bf5d8880a4697c9c4e06508559930a9e379c4b154012daa94018ce8ae RK3576/Qwen3.5-2B_RK3576_w8a8.rkllm |
| 946daf4377bbf5913d8d5226180991b8e6b4a3068d8b5959bdc81a9492311a09 RK3588/Qwen3.5-2B_RK3588_w8a8.rkllm |
| c752c9ddec5ad0415cff7d5b2cd645c00273d7f75cfb034182126309911c051b RK3576/Qwen3.5-2B_vision_RK3576.rknn |
| 9077bfd3f4a0a3846af7fd1609175aaaba07858d00acdc31459ae9f2dc66716a RK3588/Qwen3.5-2B_vision_RK3588.rknn |
| ``` |
| |
| </details> |
| |
| ## Credits |
| |
| Original model: [Qwen/Qwen3.5-2B](https://huggingface.co/Qwen/Qwen3.5-2B). Conversion tools: [Rockchip RKLLM](https://github.com/airockchip/rknn-llm). |
| |
