license: gemma
base_model: google/gemma-3-4b-it
pipeline_tag: text-generation
library_name: rkllm
tags:
- rkllm
- rockchip
- rk3576
- rk3588
- gemma
- gemma-3
gemma-3-4b-it · RKLLM
Text generation on Rockchip RK3576 and RK3588, converted from google/gemma-3-4b-it.
The files currently published here are text-only. The source model supports images, but this conversion does not include a vision encoder.
- Conversion toolkit: RKLLM Toolkit 1.2.3.
- Inputs / output: text → text.
- License:
gemma; the upstream model card describes its terms and intended use.
Choose your files
Use the files for your board’s exact SoC. w4a16 uses 4-bit weights and 16-bit activations; w4a16_g128 adds a weight group size of 128. w8a8 uses 8-bit weights and activations. File size below is download size, not total runtime RAM.
| Board | Quantization | Size | Language model |
|---|---|---|---|
| RK3576 | w4a16 |
3.42 GiB | gemma-3-4b-it_RK3576_w4a16.rkllm |
| RK3576 | w8a8 |
4.92 GiB | gemma-3-4b-it_RK3576_w8a8.rkllm |
| RK3588 | w8a8 |
4.90 GiB | gemma-3-4b-it_RK3588_w8a8.rkllm |
Run on your board
With Docker installed on a matching Rockchip board, this example starts the RK3576 w4a16 model and its API on port 8001:
sudo docker run --rm -d --name rkllm \
--privileged -v /dev:/dev -p 8001:8001 \
ghcr.io/hanzo-huang/rkllm-docker/llm/gemma-3-4b-it:rk3576-w4a16
The image includes its model file. See rkllm-docker for API requests, other board variants, and configuration.
To download the files separately instead:
hf download HanzoHuang/gemma-3-4b-it-RKLLM \
RK3576/gemma-3-4b-it_RK3576_w4a16.rkllm \
--local-dir ./models
These are native RKLLM/RKNN artifacts. Use a compatible RKLLM runtime and the model’s correct chat template when integrating them in another application.
Release notes
No per-variant board benchmark is published here. Quantized output may differ from the original model; check responses on your own workloads. Use the toolkit version listed above when checking runtime compatibility.
SHA-256 checksums
0e6d6298453f483e832a7982bcdf1acdf895a47c6777e93eb3c1e43e82cf8fc2 RK3576/gemma-3-4b-it_RK3576_w4a16.rkllm
b077e9ee75f837230dbf1be948a03d39942abd90ab23343399fa13b7d19526cf RK3576/gemma-3-4b-it_RK3576_w8a8.rkllm
aa04fce6ca2e418cd946814f1563fe8ed7e56dfbf5cf99ced4e753739287ee80 RK3588/gemma-3-4b-it_RK3588_w8a8.rkllm
Credits
Original model: google/gemma-3-4b-it. Conversion tools: Rockchip RKLLM.