HanzoHuang/Qwen3-4B-RKLLM

English | 简体中文

🚀 Deploy with reComputer AI Lab

Explore and deploy AI models on Rockchip RK3576 and RK3588 devices with reComputer AI Lab.

⚡ Run with one command

With Docker installed, choose your model and board in reComputer AI Lab, copy the deployment command, and run it in your device’s terminal.

Explore Models & Deployment Guides →

---

Qwen3-4B · RKLLM

• Max context length: 4096

Text generation on Rockchip RK3576 and RK3588, converted from Qwen/Qwen3-4B.

• Conversion toolkit: RKLLM Toolkit 1.2.3.
• Inputs / output: text → text.
• License: apache-2.0; the upstream model card describes its terms and intended use.

Choose your files

Use the files for your board’s exact SoC. w4a16 uses 4-bit weights and 16-bit activations; w4a16_g128 adds a weight group size of 128. w8a8 uses 8-bit weights and activations. File size below is download size, not total runtime RAM.

Board	Quantization	Size	Language model
RK3576	w4a16	2.84 GiB	Qwen3-4B_RK3576_w4a16.rkllm
RK3576	w4a16_g128	3.06 GiB	Qwen3-4B_RK3576_w4a16_g128.rkllm
RK3576	w8a8	4.53 GiB	Qwen3-4B_RK3576_w8a8.rkllm
RK3588	w8a8	4.51 GiB	Qwen3-4B_RK3588_w8a8.rkllm

Run on your board

With Docker installed on a matching Rockchip board, this example starts the RK3576 w4a16_g128 model and its API on port 8001:

sudo docker run --rm -d --name rkllm \
  --privileged -v /dev:/dev -p 8001:8001 \
  ghcr.io/hanzo-huang/rkllm-docker/llm/qwen3-4b:rk3576-w4a16-g128

The image includes its model file. See rkllm-docker for API requests, other board variants, and configuration.

To download the files separately instead:

hf download HanzoHuang/Qwen3-4B-RKLLM \
  RK3576/Qwen3-4B_RK3576_w4a16_g128.rkllm \
  --local-dir ./models

These are native RKLLM/RKNN artifacts. Use a compatible RKLLM runtime and the model’s correct chat template when integrating them in another application.

Release notes

No per-variant board benchmark is published here. Quantized output may differ from the original model; check responses on your own workloads. Use the toolkit version listed above when checking runtime compatibility.

SHA-256 checksums

07b25e01a16b50c0f76c777826aea602e485395a77df4c62428cf9fa532f509e  RK3576/Qwen3-4B_RK3576_w4a16.rkllm
66b8dbc941a7fc9864583082013a2d765be9839f19e98763abfd5ec2bca166a2  RK3576/Qwen3-4B_RK3576_w4a16_g128.rkllm
7f40a9c638360b1ce51dc03d2fe5005c4ded9804d13304de3c52244634e7ed26  RK3576/Qwen3-4B_RK3576_w8a8.rkllm
6f53a7b5ec8430f95a9fb6e6279e8a4744c9952d8ec59c4a7303b92ebc8b50d5  RK3588/Qwen3-4B_RK3588_w8a8.rkllm

Credits

Original model: Qwen/Qwen3-4B. Conversion tools: Rockchip RKLLM.