File size: 3,987 Bytes
6b725cb
 
 
 
 
 
 
 
 
 
 
c307d04
6b725cb
 
e0e99d6
 
569278f
 
 
 
 
 
 
 
 
 
 
 
 
 
 
c307d04
6b725cb
db0a519
 
c307d04
6b725cb
c307d04
 
 
6b725cb
c307d04
6b725cb
c307d04
6b725cb
c307d04
 
 
 
 
6b725cb
c307d04
 
 
6b725cb
 
c307d04
 
fe37b7d
 
6b725cb
 
c307d04
6b725cb
c307d04
6b725cb
 
c307d04
 
 
6b725cb
 
c307d04
6b725cb
c307d04
3f859da
c307d04
3f859da
c307d04
3f859da
c307d04
 
3f859da
c307d04
 
 
 
3f859da
 
c307d04
 
 
3f859da
c307d04
6b725cb
c307d04
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
---
license: apache-2.0
base_model: XHToken/Spark-X2.5-1.7B
pipeline_tag: text-generation
tags:
- rkllm
- rockchip
- rk3576
- rk3588
- quantized
- spark2_5
library_name: rkllm
---

**English** | [简体中文](README_zh-CN.md)

## 🚀 Deploy with reComputer AI Lab

Explore and deploy AI models on Rockchip **RK3576** and **RK3588** devices with **reComputer AI Lab**.

[![reComputer AI Lab — explore models and deployment guides](https://files.seeedstudio.com/Edge_box/reComputer_Lab.webp)](https://sensecraft.seeed.cc/ai-lab/en/models?device=rk3588)

### ⚡ Run with one command

With Docker installed, choose your model and board in reComputer AI Lab, copy the deployment command, and run it in your device’s terminal.

**[Explore Models & Deployment Guides →](https://sensecraft.seeed.cc/ai-lab/en/models?device=rk3588)**

---


# Spark-X2.5-1.7B · RKLLM

* Max context length: 4096

Text generation on Rockchip RK3576 and RK3588, converted from [XHToken/Spark-X2.5-1.7B](https://huggingface.co/XHToken/Spark-X2.5-1.7B).

- **Conversion toolkit:** RKLLM Toolkit 1.3.1.
- **Inputs / output:** text → text.
- **License:** `apache-2.0`; the [upstream model card](https://huggingface.co/XHToken/Spark-X2.5-1.7B) describes its terms and intended use.

## Choose your files

Use the files for your board’s exact SoC. `w4a16` uses 4-bit weights and 16-bit activations; `w4a16_g128` adds a weight group size of 128. `w8a8` uses 8-bit weights and activations. File size below is download size, not total runtime RAM.

| Board | Quantization | Size | Language model |
| --- | --- | --- | --- |
| RK3576 | `w4a16_g128` | 1.56 GiB | [Spark-X2.5-1.7B_RK3576_w4a16_g128.rkllm](RK3576/Spark-X2.5-1.7B_RK3576_w4a16_g128.rkllm) |
| RK3576 | `w8a8` | 2.14 GiB | [Spark-X2.5-1.7B_RK3576_w8a8.rkllm](RK3576/Spark-X2.5-1.7B_RK3576_w8a8.rkllm) |
| RK3588 | `w8a8` | 2.12 GiB | [Spark-X2.5-1.7B_RK3588_w8a8.rkllm](RK3588/Spark-X2.5-1.7B_RK3588_w8a8.rkllm) |

## Run on your board

With Docker installed on a matching Rockchip board, this example starts the **RK3576 w4a16_g128** model and its API on port 8001:

```bash
sudo docker run --rm -d --name rkllm \
  --privileged -v /dev:/dev -p 8001:8001 \
  ghcr.io/hanzo-huang/rkllm-docker/llm/spark-x2.5-1.7b:rk3576-w4a16-g128 \
  serve --max_context_len 4096
```

The image includes its model file. See [rkllm-docker](https://github.com/Hanzo-Huang/rkllm-docker#readme) for API requests, other board variants, and configuration.

To download the files separately instead:

```bash
hf download HanzoHuang/Spark-X2.5-1.7B-RKLLM \
  RK3576/Spark-X2.5-1.7B_RK3576_w4a16_g128.rkllm \
  --local-dir ./models
```

These are native RKLLM/RKNN artifacts. Use a compatible RKLLM runtime and the model’s correct chat template when integrating them in another application.

## Release notes

All three variants passed functional checks on their matching RK3576/RK3588 boards using RKLLM 1.3.1 and the packaged Docker images: arithmetic, a factual question, streamed output, and conversation recall. These are smoke tests, not a quality benchmark.

The exported context length is 4,096 tokens. Conversion used ModelScope source weights and SDK-generated calibration samples. Spark calibration required an isolated Transformers 4.57.1 environment; other conversion environments were left unchanged.

<details>
<summary>SHA-256 checksums</summary>

```text
725bed8bbd4e6c682b542920dd43081a7f1c29ce5e38b69eceb3e9e8c710e5b4  RK3576/Spark-X2.5-1.7B_RK3576_w4a16_g128.rkllm
fe47ea39dd7968f0cfd4673e9ed189dbec768600dd35f068ca3f33c38a9b8267  RK3576/Spark-X2.5-1.7B_RK3576_w8a8.rkllm
4c87d982c70424ce493a96f2c06bfb49ca9f5e7fd9f805c70f5cf7eb14f64577  RK3588/Spark-X2.5-1.7B_RK3588_w8a8.rkllm
```

</details>

A downloadable checksum list is also available in [SHA256SUMS](SHA256SUMS).

## Credits

Original model: [XHToken/Spark-X2.5-1.7B](https://huggingface.co/XHToken/Spark-X2.5-1.7B). Conversion tools: [Rockchip RKLLM](https://github.com/airockchip/rknn-llm).