Document exported max context length

271342b verified 4 days ago

3.58 kB

license: apache-2.0
base_model: Qwen/Qwen3.5-4B
pipeline_tag: image-text-to-text
library_name: rkllm
tags:
  - rkllm
  - rockchip
  - rk3576
  - rk3588
  - qwen
  - qwen3.5
  - multimodal

Qwen3.5-4B · RKLLM

Max context length: 4096

Image and text on Rockchip RK3576 and RK3588, converted from Qwen/Qwen3.5-4B.

Conversion toolkit: RKLLM Toolkit 1.3.0.
Inputs / output: text or image + text → text.
License: apache-2.0; the upstream model card describes its terms and intended use.

Choose your files

Use the files for your board’s exact SoC. w4a16 uses 4-bit weights and 16-bit activations; w4a16_g128 adds a weight group size of 128. w8a8 uses 8-bit weights and activations. File size below is download size, not total runtime RAM.

Board	Quantization	Size	Language model
RK3576	`w4a16_g128`	3.71 GiB	Qwen3.5-4B_RK3576_w4a16_g128.rkllm
RK3576	`w8a8`	5.18 GiB	Qwen3.5-4B_RK3576_w8a8.rkllm
RK3588	`w8a8`	5.16 GiB	Qwen3.5-4B_RK3588_w8a8.rkllm

For image input, also download the vision encoder for the same SoC. Keep the language model and vision encoder from this release together.

Board	Size	Vision encoder
RK3576	0.68 GiB	Qwen3.5-4B_vision_RK3576.rknn
RK3588	0.66 GiB	Qwen3.5-4B_vision_RK3588.rknn

ONNX files are export intermediates; they are not needed to run these RKNN/RKLLM models.

Run on your board

With Docker installed on a matching Rockchip board, this example starts the RK3576 w4a16_g128 model and its API on port 8001:

sudo docker run --rm -d --name rkllm \
  --privileged -v /dev:/dev -p 8001:8001 \
  ghcr.io/hanzo-huang/rkllm-docker/vlm/qwen3.5-4b:rk3576-w4a16-g128

The image includes its model files. See rkllm-docker for API requests, image inputs, other board variants, and configuration.

To download the files separately instead:

hf download HanzoHuang/Qwen3.5-4B-RKLLM \
  RK3576/Qwen3.5-4B_RK3576_w4a16_g128.rkllm \
  RK3576/Qwen3.5-4B_vision_RK3576.rknn \
  --local-dir ./models

These are native RKLLM/RKNN artifacts. Use a compatible RKLLM runtime and the model’s correct chat template when integrating them in another application.

Release notes

No per-variant board benchmark is published here. Quantized output may differ from the original model; check responses on your own workloads. Use the toolkit version listed above when checking runtime compatibility.

SHA-256 checksums

aa4d34b42752a0e491ed891ce4b7f32a745631601a65b175f743f971bdb33482  RK3576/Qwen3.5-4B_RK3576_w4a16_g128.rkllm
736f4b1065b481bae9cbd86e6c9ed30222abba9c5a3cb2d8d68dd92037d82dfe  RK3576/Qwen3.5-4B_RK3576_w8a8.rkllm
715566bbee72b25d8c4912f6cf6256a8ac264a754511573a99a541487bbc06b4  RK3588/Qwen3.5-4B_RK3588_w8a8.rkllm
6692f52adaa7ee0ba892c7cbba01979750cb82cff8061a571f56e9d84c6d98a2  RK3576/Qwen3.5-4B_vision_RK3576.rknn
c286ef69266c11a2a2cedd881ad6a40415c17c8924962f07454359bb6d60e2ba  RK3588/Qwen3.5-4B_vision_RK3588.rknn

Credits

Original model: Qwen/Qwen3.5-4B. Conversion tools: Rockchip RKLLM.