File size: 3,185 Bytes
a7d73c0 1b3d534 a7d73c0 1b3d534 37c3eca 1b3d534 37c3eca 1b3d534 37c3eca 1b3d534 37c3eca 1b3d534 37c3eca 1b3d534 37c3eca 1b3d534 37c3eca 1b3d534 37c3eca 1b3d534 37c3eca 1b3d534 37c3eca 1b3d534 37c3eca 1b3d534 37c3eca 1b3d534 37c3eca 1b3d534 37c3eca 1b3d534 37c3eca 1b3d534 37c3eca 1b3d534 37c3eca | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 | ---
license: mit
base_model: deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B
pipeline_tag: text-generation
library_name: rkllm
tags:
- rkllm
- rockchip
- rk3576
- rk3588
- deepseek
- deepseek-r1
- qwen
---
# DeepSeek-R1-Distill-Qwen-1.5B · RKLLM
Text generation on Rockchip RK3576 and RK3588, converted from [deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B](https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B).
- **Conversion toolkit:** RKLLM Toolkit 1.3.0.
- **Inputs / output:** text → text.
- **License:** `mit`; the [upstream model card](https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B) describes its terms and intended use.
## Choose your files
Use the files for your board’s exact SoC. `w4a16` uses 4-bit weights and 16-bit activations; `w4a16_g128` adds a weight group size of 128. `w8a8` uses 8-bit weights and activations. File size below is download size, not total runtime RAM.
| Board | Quantization | Size | Language model |
| --- | --- | --- | --- |
| RK3576 | `w4a16_g128` | 1.43 GiB | [DeepSeek-R1-Distill-Qwen-1.5B_RK3576_w4a16_g128.rkllm](RK3576/DeepSeek-R1-Distill-Qwen-1.5B_RK3576_w4a16_g128.rkllm) |
| RK3576 | `w8a8` | 1.93 GiB | [DeepSeek-R1-Distill-Qwen-1.5B_RK3576_w8a8.rkllm](RK3576/DeepSeek-R1-Distill-Qwen-1.5B_RK3576_w8a8.rkllm) |
| RK3588 | `w8a8` | 1.91 GiB | [DeepSeek-R1-Distill-Qwen-1.5B_RK3588_w8a8.rkllm](RK3588/DeepSeek-R1-Distill-Qwen-1.5B_RK3588_w8a8.rkllm) |
## Run on your board
With Docker installed on a matching Rockchip board, this example starts the **RK3576 w4a16_g128** model and its API on port 8001:
```bash
sudo docker run --rm -d --name rkllm \
--privileged -v /dev:/dev -p 8001:8001 \
ghcr.io/hanzo-huang/rkllm-docker/llm/deepseek-r1-distill-qwen-1.5b:rk3576-w4a16-g128
```
The image includes its model file. See [rkllm-docker](https://github.com/Hanzo-Huang/rkllm-docker#readme) for API requests, other board variants, and configuration.
To download the files separately instead:
```bash
hf download HanzoHuang/DeepSeek-R1-Distill-Qwen-1.5B-RKLLM \
RK3576/DeepSeek-R1-Distill-Qwen-1.5B_RK3576_w4a16_g128.rkllm \
--local-dir ./models
```
These are native RKLLM/RKNN artifacts. Use a compatible RKLLM runtime and the model’s correct chat template when integrating them in another application.
## Release notes
No per-variant board benchmark is published here. Quantized output may differ from the original model; check responses on your own workloads. Use the toolkit version listed above when checking runtime compatibility.
<details>
<summary>SHA-256 checksums</summary>
```text
e737c9aa8cfbfd0216e547ed67bc2325d5add2f8b2aa21037389e296ac3b8d80 RK3576/DeepSeek-R1-Distill-Qwen-1.5B_RK3576_w4a16_g128.rkllm
6ceab2c6b93a8b55837ab75e4050bbce888030153c0bc83a465bd73ead44120f RK3576/DeepSeek-R1-Distill-Qwen-1.5B_RK3576_w8a8.rkllm
d00038d619c1905ac66ab5d499019d80889d37cc1f81be491cc503e284950c2d RK3588/DeepSeek-R1-Distill-Qwen-1.5B_RK3588_w8a8.rkllm
```
</details>
## Credits
Original model: [deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B](https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B). Conversion tools: [Rockchip RKLLM](https://github.com/airockchip/rknn-llm).
|