| --- |
| license: gemma |
| base_model: google/gemma-3-4b-it |
| pipeline_tag: text-generation |
| library_name: rkllm |
| tags: |
| - rkllm |
| - rockchip |
| - rk3576 |
| - rk3588 |
| - gemma |
| - gemma-3 |
| --- |
| |
| # gemma-3-4b-it · RKLLM |
|
|
| Text generation on Rockchip RK3576 and RK3588, converted from [google/gemma-3-4b-it](https://huggingface.co/google/gemma-3-4b-it). |
|
|
| **The files currently published here are text-only.** The source model supports images, but this conversion does not include a vision encoder. |
|
|
| - **Conversion toolkit:** RKLLM Toolkit 1.2.3. |
| - **Inputs / output:** text → text. |
| - **License:** `gemma`; the [upstream model card](https://huggingface.co/google/gemma-3-4b-it) describes its terms and intended use. |
|
|
| ## Choose your files |
|
|
| Use the files for your board’s exact SoC. `w4a16` uses 4-bit weights and 16-bit activations; `w4a16_g128` adds a weight group size of 128. `w8a8` uses 8-bit weights and activations. File size below is download size, not total runtime RAM. |
|
|
| | Board | Quantization | Size | Language model | |
| | --- | --- | --- | --- | |
| | RK3576 | `w4a16` | 3.42 GiB | [gemma-3-4b-it_RK3576_w4a16.rkllm](RK3576/gemma-3-4b-it_RK3576_w4a16.rkllm) | |
| | RK3576 | `w8a8` | 4.92 GiB | [gemma-3-4b-it_RK3576_w8a8.rkllm](RK3576/gemma-3-4b-it_RK3576_w8a8.rkllm) | |
| | RK3588 | `w8a8` | 4.90 GiB | [gemma-3-4b-it_RK3588_w8a8.rkllm](RK3588/gemma-3-4b-it_RK3588_w8a8.rkllm) | |
|
|
| ## Run on your board |
|
|
| With Docker installed on a matching Rockchip board, this example starts the **RK3576 w4a16** model and its API on port 8001: |
|
|
| ```bash |
| sudo docker run --rm -d --name rkllm \ |
| --privileged -v /dev:/dev -p 8001:8001 \ |
| ghcr.io/hanzo-huang/rkllm-docker/llm/gemma-3-4b-it:rk3576-w4a16 |
| ``` |
|
|
| The image includes its model file. See [rkllm-docker](https://github.com/Hanzo-Huang/rkllm-docker#readme) for API requests, other board variants, and configuration. |
|
|
| To download the files separately instead: |
|
|
| ```bash |
| hf download HanzoHuang/gemma-3-4b-it-RKLLM \ |
| RK3576/gemma-3-4b-it_RK3576_w4a16.rkllm \ |
| --local-dir ./models |
| ``` |
|
|
| These are native RKLLM/RKNN artifacts. Use a compatible RKLLM runtime and the model’s correct chat template when integrating them in another application. |
|
|
| ## Release notes |
|
|
| No per-variant board benchmark is published here. Quantized output may differ from the original model; check responses on your own workloads. Use the toolkit version listed above when checking runtime compatibility. |
|
|
| <details> |
| <summary>SHA-256 checksums</summary> |
|
|
| ```text |
| 0e6d6298453f483e832a7982bcdf1acdf895a47c6777e93eb3c1e43e82cf8fc2 RK3576/gemma-3-4b-it_RK3576_w4a16.rkllm |
| b077e9ee75f837230dbf1be948a03d39942abd90ab23343399fa13b7d19526cf RK3576/gemma-3-4b-it_RK3576_w8a8.rkllm |
| aa04fce6ca2e418cd946814f1563fe8ed7e56dfbf5cf99ced4e753739287ee80 RK3588/gemma-3-4b-it_RK3588_w8a8.rkllm |
| ``` |
|
|
| </details> |
|
|
| ## Credits |
|
|
| Original model: [google/gemma-3-4b-it](https://huggingface.co/google/gemma-3-4b-it). Conversion tools: [Rockchip RKLLM](https://github.com/airockchip/rknn-llm). |
|
|
