README.md

---
license: apache-2.0
base_model: Qwen/Qwen3.5-4B
pipeline_tag: image-text-to-text
library_name: rkllm
tags:
- rkllm
- rockchip
- rk3576
- rk3588
- qwen
- qwen3.5
- multimodal
---

# Qwen3.5-4B · RKLLM

* Max context length: 4096

Image and text on Rockchip RK3576 and RK3588, converted from [Qwen/Qwen3.5-4B](https://huggingface.co/Qwen/Qwen3.5-4B).

- **Conversion toolkit:** RKLLM Toolkit 1.3.0.
- **Inputs / output:** text or image + text → text.
- **License:** `apache-2.0`; the [upstream model card](https://huggingface.co/Qwen/Qwen3.5-4B) describes its terms and intended use.

## Choose your files

Use the files for your board’s exact SoC. `w4a16` uses 4-bit weights and 16-bit activations; `w4a16_g128` adds a weight group size of 128. `w8a8` uses 8-bit weights and activations. File size below is download size, not total runtime RAM.

| Board | Quantization | Size | Language model |
| --- | --- | --- | --- |
| RK3576 | `w4a16_g128` | 3.71 GiB | [Qwen3.5-4B_RK3576_w4a16_g128.rkllm](RK3576/Qwen3.5-4B_RK3576_w4a16_g128.rkllm) |
| RK3576 | `w8a8` | 5.18 GiB | [Qwen3.5-4B_RK3576_w8a8.rkllm](RK3576/Qwen3.5-4B_RK3576_w8a8.rkllm) |
| RK3588 | `w8a8` | 5.16 GiB | [Qwen3.5-4B_RK3588_w8a8.rkllm](RK3588/Qwen3.5-4B_RK3588_w8a8.rkllm) |

For image input, also download the **vision encoder for the same SoC**. Keep the language model and vision encoder from this release together.

| Board | Size | Vision encoder |
| --- | --- | --- |
| RK3576 | 0.68 GiB | [Qwen3.5-4B_vision_RK3576.rknn](RK3576/Qwen3.5-4B_vision_RK3576.rknn) |
| RK3588 | 0.66 GiB | [Qwen3.5-4B_vision_RK3588.rknn](RK3588/Qwen3.5-4B_vision_RK3588.rknn) |

ONNX files are export intermediates; they are not needed to run these RKNN/RKLLM models.

## Run on your board

With Docker installed on a matching Rockchip board, this example starts the **RK3576 w4a16_g128** model and its API on port 8001:

```bash
sudo docker run --rm -d --name rkllm \
  --privileged -v /dev:/dev -p 8001:8001 \
  ghcr.io/hanzo-huang/rkllm-docker/vlm/qwen3.5-4b:rk3576-w4a16-g128
```

The image includes its model files. See [rkllm-docker](https://github.com/Hanzo-Huang/rkllm-docker#readme) for API requests, image inputs, other board variants, and configuration.

To download the files separately instead:

```bash
hf download HanzoHuang/Qwen3.5-4B-RKLLM \
  RK3576/Qwen3.5-4B_RK3576_w4a16_g128.rkllm \
  RK3576/Qwen3.5-4B_vision_RK3576.rknn \
  --local-dir ./models
```

These are native RKLLM/RKNN artifacts. Use a compatible RKLLM runtime and the model’s correct chat template when integrating them in another application.

## Release notes

No per-variant board benchmark is published here. Quantized output may differ from the original model; check responses on your own workloads. Use the toolkit version listed above when checking runtime compatibility.

<details>
<summary>SHA-256 checksums</summary>

```text
aa4d34b42752a0e491ed891ce4b7f32a745631601a65b175f743f971bdb33482  RK3576/Qwen3.5-4B_RK3576_w4a16_g128.rkllm
736f4b1065b481bae9cbd86e6c9ed30222abba9c5a3cb2d8d68dd92037d82dfe  RK3576/Qwen3.5-4B_RK3576_w8a8.rkllm
715566bbee72b25d8c4912f6cf6256a8ac264a754511573a99a541487bbc06b4  RK3588/Qwen3.5-4B_RK3588_w8a8.rkllm
6692f52adaa7ee0ba892c7cbba01979750cb82cff8061a571f56e9d84c6d98a2  RK3576/Qwen3.5-4B_vision_RK3576.rknn
c286ef69266c11a2a2cedd881ad6a40415c17c8924962f07454359bb6d60e2ba  RK3588/Qwen3.5-4B_vision_RK3588.rknn
```

</details>

## Credits

Original model: [Qwen/Qwen3.5-4B](https://huggingface.co/Qwen/Qwen3.5-4B). Conversion tools: [Rockchip RKLLM](https://github.com/airockchip/rknn-llm).