File size: 2,950 Bytes
c7b1122 4d3024f 94a57b0 4d3024f c7b1122 4d3024f 70e7fd3 4d3024f 70e7fd3 4d3024f 70e7fd3 4d3024f 70e7fd3 4d3024f 70e7fd3 4d3024f 70e7fd3 4d3024f 70e7fd3 6a147ed 70e7fd3 6a147ed 70e7fd3 4d3024f 70e7fd3 4d3024f 70e7fd3 4d3024f 70e7fd3 4d3024f 94a57b0 70e7fd3 4d3024f 70e7fd3 4d3024f 70e7fd3 4d3024f 70e7fd3 4d3024f 70e7fd3 4d3024f 70e7fd3 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 | ---
license: gemma
base_model: google/gemma-3-4b-it
pipeline_tag: text-generation
library_name: rkllm
tags:
- rkllm
- rockchip
- rk3576
- rk3588
- gemma
- gemma-3
---
# gemma-3-4b-it · RKLLM
Text generation on Rockchip RK3576 and RK3588, converted from [google/gemma-3-4b-it](https://huggingface.co/google/gemma-3-4b-it).
**The files currently published here are text-only.** The source model supports images, but this conversion does not include a vision encoder.
- **Conversion toolkit:** RKLLM Toolkit 1.2.3.
- **Inputs / output:** text → text.
- **License:** `gemma`; the [upstream model card](https://huggingface.co/google/gemma-3-4b-it) describes its terms and intended use.
## Choose your files
Use the files for your board’s exact SoC. `w4a16` uses 4-bit weights and 16-bit activations; `w4a16_g128` adds a weight group size of 128. `w8a8` uses 8-bit weights and activations. File size below is download size, not total runtime RAM.
| Board | Quantization | Size | Language model |
| --- | --- | --- | --- |
| RK3576 | `w4a16` | 3.42 GiB | [gemma-3-4b-it_RK3576_w4a16.rkllm](RK3576/gemma-3-4b-it_RK3576_w4a16.rkllm) |
| RK3576 | `w8a8` | 4.92 GiB | [gemma-3-4b-it_RK3576_w8a8.rkllm](RK3576/gemma-3-4b-it_RK3576_w8a8.rkllm) |
| RK3588 | `w8a8` | 4.90 GiB | [gemma-3-4b-it_RK3588_w8a8.rkllm](RK3588/gemma-3-4b-it_RK3588_w8a8.rkllm) |
## Run on your board
With Docker installed on a matching Rockchip board, this example starts the **RK3576 w4a16** model and its API on port 8001:
```bash
sudo docker run --rm -d --name rkllm \
--privileged -v /dev:/dev -p 8001:8001 \
ghcr.io/hanzo-huang/rkllm-docker/llm/gemma-3-4b-it:rk3576-w4a16
```
The image includes its model file. See [rkllm-docker](https://github.com/Hanzo-Huang/rkllm-docker#readme) for API requests, other board variants, and configuration.
To download the files separately instead:
```bash
hf download HanzoHuang/gemma-3-4b-it-RKLLM \
RK3576/gemma-3-4b-it_RK3576_w4a16.rkllm \
--local-dir ./models
```
These are native RKLLM/RKNN artifacts. Use a compatible RKLLM runtime and the model’s correct chat template when integrating them in another application.
## Release notes
No per-variant board benchmark is published here. Quantized output may differ from the original model; check responses on your own workloads. Use the toolkit version listed above when checking runtime compatibility.
<details>
<summary>SHA-256 checksums</summary>
```text
0e6d6298453f483e832a7982bcdf1acdf895a47c6777e93eb3c1e43e82cf8fc2 RK3576/gemma-3-4b-it_RK3576_w4a16.rkllm
b077e9ee75f837230dbf1be948a03d39942abd90ab23343399fa13b7d19526cf RK3576/gemma-3-4b-it_RK3576_w8a8.rkllm
aa04fce6ca2e418cd946814f1563fe8ed7e56dfbf5cf99ced4e753739287ee80 RK3588/gemma-3-4b-it_RK3588_w8a8.rkllm
```
</details>
## Credits
Original model: [google/gemma-3-4b-it](https://huggingface.co/google/gemma-3-4b-it). Conversion tools: [Rockchip RKLLM](https://github.com/airockchip/rknn-llm).
|