File size: 2,942 Bytes
100b7ac 926e4ab eacf8ac 926e4ab 100b7ac 926e4ab 6028918 926e4ab 6028918 926e4ab 6028918 926e4ab 6028918 926e4ab 6028918 926e4ab 6028918 902b366 6028918 902b366 6028918 926e4ab 6028918 926e4ab 6028918 926e4ab 6028918 926e4ab eacf8ac 6028918 926e4ab 6028918 926e4ab 6028918 926e4ab 6028918 926e4ab 6028918 926e4ab 6028918 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 | ---
license: apache-2.0
base_model: Qwen/Qwen3-4B
pipeline_tag: text-generation
library_name: rkllm
tags:
- rkllm
- rockchip
- rk3576
- rk3588
- qwen
- qwen3
---
# Qwen3-4B · RKLLM
Text generation on Rockchip RK3576 and RK3588, converted from [Qwen/Qwen3-4B](https://huggingface.co/Qwen/Qwen3-4B).
- **Conversion toolkit:** RKLLM Toolkit 1.2.3.
- **Inputs / output:** text → text.
- **License:** `apache-2.0`; the [upstream model card](https://huggingface.co/Qwen/Qwen3-4B) describes its terms and intended use.
## Choose your files
Use the files for your board’s exact SoC. `w4a16` uses 4-bit weights and 16-bit activations; `w4a16_g128` adds a weight group size of 128. `w8a8` uses 8-bit weights and activations. File size below is download size, not total runtime RAM.
| Board | Quantization | Size | Language model |
| --- | --- | --- | --- |
| RK3576 | `w4a16` | 2.84 GiB | [Qwen3-4B_RK3576_w4a16.rkllm](RK3576/Qwen3-4B_RK3576_w4a16.rkllm) |
| RK3576 | `w4a16_g128` | 3.06 GiB | [Qwen3-4B_RK3576_w4a16_g128.rkllm](RK3576/Qwen3-4B_RK3576_w4a16_g128.rkllm) |
| RK3576 | `w8a8` | 4.53 GiB | [Qwen3-4B_RK3576_w8a8.rkllm](RK3576/Qwen3-4B_RK3576_w8a8.rkllm) |
| RK3588 | `w8a8` | 4.51 GiB | [Qwen3-4B_RK3588_w8a8.rkllm](RK3588/Qwen3-4B_RK3588_w8a8.rkllm) |
## Run on your board
With Docker installed on a matching Rockchip board, this example starts the **RK3576 w4a16_g128** model and its API on port 8001:
```bash
sudo docker run --rm -d --name rkllm \
--privileged -v /dev:/dev -p 8001:8001 \
ghcr.io/hanzo-huang/rkllm-docker/llm/qwen3-4b:rk3576-w4a16-g128
```
The image includes its model file. See [rkllm-docker](https://github.com/Hanzo-Huang/rkllm-docker#readme) for API requests, other board variants, and configuration.
To download the files separately instead:
```bash
hf download HanzoHuang/Qwen3-4B-RKLLM \
RK3576/Qwen3-4B_RK3576_w4a16_g128.rkllm \
--local-dir ./models
```
These are native RKLLM/RKNN artifacts. Use a compatible RKLLM runtime and the model’s correct chat template when integrating them in another application.
## Release notes
No per-variant board benchmark is published here. Quantized output may differ from the original model; check responses on your own workloads. Use the toolkit version listed above when checking runtime compatibility.
<details>
<summary>SHA-256 checksums</summary>
```text
07b25e01a16b50c0f76c777826aea602e485395a77df4c62428cf9fa532f509e RK3576/Qwen3-4B_RK3576_w4a16.rkllm
66b8dbc941a7fc9864583082013a2d765be9839f19e98763abfd5ec2bca166a2 RK3576/Qwen3-4B_RK3576_w4a16_g128.rkllm
7f40a9c638360b1ce51dc03d2fe5005c4ded9804d13304de3c52244634e7ed26 RK3576/Qwen3-4B_RK3576_w8a8.rkllm
6f53a7b5ec8430f95a9fb6e6279e8a4744c9952d8ec59c4a7303b92ebc8b50d5 RK3588/Qwen3-4B_RK3588_w8a8.rkllm
```
</details>
## Credits
Original model: [Qwen/Qwen3-4B](https://huggingface.co/Qwen/Qwen3-4B). Conversion tools: [Rockchip RKLLM](https://github.com/airockchip/rknn-llm).
|