Document exported max context length

d3e64e6 verified 5 days ago

3.58 kB

license: apache-2.0
base_model: Qwen/Qwen3.5-2B
pipeline_tag: image-text-to-text
library_name: rkllm
tags:
  - rkllm
  - rockchip
  - rk3576
  - rk3588
  - qwen
  - qwen3.5
  - multimodal

Qwen3.5-2B · RKLLM

Max context length: 4096

Image and text on Rockchip RK3576 and RK3588, converted from Qwen/Qwen3.5-2B.

Conversion toolkit: RKLLM Toolkit 1.3.0.
Inputs / output: text or image + text → text.
License: apache-2.0; the upstream model card describes its terms and intended use.

Choose your files

Use the files for your board’s exact SoC. w4a16 uses 4-bit weights and 16-bit activations; w4a16_g128 adds a weight group size of 128. w8a8 uses 8-bit weights and activations. File size below is download size, not total runtime RAM.

Board	Quantization	Size	Language model
RK3576	`w4a16_g128`	2.21 GiB	Qwen3.5-2B_RK3576_w4a16_g128.rkllm
RK3576	`w8a8`	2.76 GiB	Qwen3.5-2B_RK3576_w8a8.rkllm
RK3588	`w8a8`	2.74 GiB	Qwen3.5-2B_RK3588_w8a8.rkllm

For image input, also download the vision encoder for the same SoC. Keep the language model and vision encoder from this release together.

Board	Size	Vision encoder
RK3576	0.67 GiB	Qwen3.5-2B_vision_RK3576.rknn
RK3588	0.65 GiB	Qwen3.5-2B_vision_RK3588.rknn

ONNX files are export intermediates; they are not needed to run these RKNN/RKLLM models.

Run on your board

With Docker installed on a matching Rockchip board, this example starts the RK3576 w4a16_g128 model and its API on port 8001:

sudo docker run --rm -d --name rkllm \
  --privileged -v /dev:/dev -p 8001:8001 \
  ghcr.io/hanzo-huang/rkllm-docker/vlm/qwen3.5-2b:rk3576-w4a16-g128

The image includes its model files. See rkllm-docker for API requests, image inputs, other board variants, and configuration.

To download the files separately instead:

hf download HanzoHuang/Qwen3.5-2B-RKLLM \
  RK3576/Qwen3.5-2B_RK3576_w4a16_g128.rkllm \
  RK3576/Qwen3.5-2B_vision_RK3576.rknn \
  --local-dir ./models

These are native RKLLM/RKNN artifacts. Use a compatible RKLLM runtime and the model’s correct chat template when integrating them in another application.

Release notes

No per-variant board benchmark is published here. Quantized output may differ from the original model; check responses on your own workloads. Use the toolkit version listed above when checking runtime compatibility.

SHA-256 checksums

76f40bbbc559615767e49a9408d28a1afc16c3a12aa3764a11e13f52acd868e9  RK3576/Qwen3.5-2B_RK3576_w4a16_g128.rkllm
95f4c10bf5d8880a4697c9c4e06508559930a9e379c4b154012daa94018ce8ae  RK3576/Qwen3.5-2B_RK3576_w8a8.rkllm
946daf4377bbf5913d8d5226180991b8e6b4a3068d8b5959bdc81a9492311a09  RK3588/Qwen3.5-2B_RK3588_w8a8.rkllm
c752c9ddec5ad0415cff7d5b2cd645c00273d7f75cfb034182126309911c051b  RK3576/Qwen3.5-2B_vision_RK3576.rknn
9077bfd3f4a0a3846af7fd1609175aaaba07858d00acdc31459ae9f2dc66716a  RK3588/Qwen3.5-2B_vision_RK3588.rknn

Credits

Original model: Qwen/Qwen3.5-2B. Conversion tools: Rockchip RKLLM.