File size: 3,025 Bytes
c1264ae d5c5e40 c1264ae d5c5e40 746807a d5c5e40 746807a d5c5e40 746807a d5c5e40 746807a d5c5e40 746807a d5c5e40 746807a 67adf08 746807a 67adf08 746807a d5c5e40 746807a d5c5e40 746807a d5c5e40 746807a d5c5e40 746807a d5c5e40 746807a d5c5e40 746807a d5c5e40 746807a d5c5e40 746807a d5c5e40 746807a | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 | ---
license: llama3.2
base_model: meta-llama/Llama-3.2-1B-Instruct
pipeline_tag: text-generation
library_name: rkllm
tags:
- rkllm
- rockchip
- rk3576
- rk3588
- llama
- llama-3.2
---
# Llama-3.2-1B-Instruct · RKLLM
Text generation on Rockchip RK3576 and RK3588, converted from [meta-llama/Llama-3.2-1B-Instruct](https://huggingface.co/meta-llama/Llama-3.2-1B-Instruct).
- **Conversion toolkit:** RKLLM Toolkit 1.3.0.
- **Inputs / output:** text → text.
- **License:** `llama3.2`; the [upstream model card](https://huggingface.co/meta-llama/Llama-3.2-1B-Instruct) describes its terms and intended use.
## Choose your files
Use the files for your board’s exact SoC. `w4a16` uses 4-bit weights and 16-bit activations; `w4a16_g128` adds a weight group size of 128. `w8a8` uses 8-bit weights and activations. File size below is download size, not total runtime RAM.
| Board | Quantization | Size | Language model |
| --- | --- | --- | --- |
| RK3576 | `w4a16_g128` | 1.30 GiB | [Llama-3.2-1B-Instruct_RK3576_w4a16_g128.rkllm](RK3576/Llama-3.2-1B-Instruct_RK3576_w4a16_g128.rkllm) |
| RK3576 | `w8a8` | 1.68 GiB | [Llama-3.2-1B-Instruct_RK3576_w8a8.rkllm](RK3576/Llama-3.2-1B-Instruct_RK3576_w8a8.rkllm) |
| RK3588 | `w8a8` | 1.67 GiB | [Llama-3.2-1B-Instruct_RK3588_w8a8.rkllm](RK3588/Llama-3.2-1B-Instruct_RK3588_w8a8.rkllm) |
## Run on your board
With Docker installed on a matching Rockchip board, this example starts the **RK3576 w4a16_g128** model and its API on port 8001:
```bash
sudo docker run --rm -d --name rkllm \
--privileged -v /dev:/dev -p 8001:8001 \
ghcr.io/hanzo-huang/rkllm-docker/llm/llama-3.2-1b-instruct:rk3576-w4a16-g128
```
The image includes its model file. See [rkllm-docker](https://github.com/Hanzo-Huang/rkllm-docker#readme) for API requests, other board variants, and configuration.
To download the files separately instead:
```bash
hf download HanzoHuang/Llama-3.2-1B-Instruct-RKLLM \
RK3576/Llama-3.2-1B-Instruct_RK3576_w4a16_g128.rkllm \
--local-dir ./models
```
These are native RKLLM/RKNN artifacts. Use a compatible RKLLM runtime and the model’s correct chat template when integrating them in another application.
## Release notes
No per-variant board benchmark is published here. Quantized output may differ from the original model; check responses on your own workloads. Use the toolkit version listed above when checking runtime compatibility.
<details>
<summary>SHA-256 checksums</summary>
```text
bd6cbbf84e90b63d451e672a542858a5a6aa15ad2b45f83e1b5dbcd944ac3018 RK3576/Llama-3.2-1B-Instruct_RK3576_w4a16_g128.rkllm
9c4d103f7c486740ba75605c61a51b80d170df067645360ffad8464b50c764df RK3576/Llama-3.2-1B-Instruct_RK3576_w8a8.rkllm
4403aec1c018c7ebf29c95730336e848aff8824e3be3bd6be7db44b4a4973bad RK3588/Llama-3.2-1B-Instruct_RK3588_w8a8.rkllm
```
</details>
## Credits
Original model: [meta-llama/Llama-3.2-1B-Instruct](https://huggingface.co/meta-llama/Llama-3.2-1B-Instruct). Conversion tools: [Rockchip RKLLM](https://github.com/airockchip/rknn-llm).
|