Instructions to use HanzoHuang/Qwen3-1.7B-RKLLM with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- RKLLM
How to use HanzoHuang/Qwen3-1.7B-RKLLM with RKLLM:
# No code snippets available yet for this library. # To use this model, check the repository files and the library's documentation. # Want to help? PRs adding snippets are welcome at: # https://github.com/huggingface/huggingface.js
- Notebooks
- Google Colab
- Kaggle
Add Simplified Chinese README and language navigation
Browse files- README.md +2 -0
- README_zh-CN.md +93 -0
README.md
CHANGED
|
@@ -12,6 +12,8 @@ tags:
|
|
| 12 |
- qwen3
|
| 13 |
---
|
| 14 |
|
|
|
|
|
|
|
| 15 |
## 🚀 Deploy with reComputer AI Lab
|
| 16 |
|
| 17 |
Explore and deploy AI models on Rockchip **RK3576** and **RK3588** devices with **reComputer AI Lab**.
|
|
|
|
| 12 |
- qwen3
|
| 13 |
---
|
| 14 |
|
| 15 |
+
**English** | [简体中文](README_zh-CN.md)
|
| 16 |
+
|
| 17 |
## 🚀 Deploy with reComputer AI Lab
|
| 18 |
|
| 19 |
Explore and deploy AI models on Rockchip **RK3576** and **RK3588** devices with **reComputer AI Lab**.
|
README_zh-CN.md
ADDED
|
@@ -0,0 +1,93 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: apache-2.0
|
| 3 |
+
base_model: Qwen/Qwen3-1.7B
|
| 4 |
+
pipeline_tag: text-generation
|
| 5 |
+
library_name: rkllm
|
| 6 |
+
tags:
|
| 7 |
+
- rkllm
|
| 8 |
+
- rockchip
|
| 9 |
+
- rk3576
|
| 10 |
+
- rk3588
|
| 11 |
+
- qwen
|
| 12 |
+
- qwen3
|
| 13 |
+
---
|
| 14 |
+
|
| 15 |
+
[English](README.md) | **简体中文**
|
| 16 |
+
|
| 17 |
+
## 🚀 使用 reComputer AI Lab 部署
|
| 18 |
+
|
| 19 |
+
通过 **reComputer AI Lab**,在 Rockchip **RK3576** 和 **RK3588** 设备上探索并部署 AI 模型。
|
| 20 |
+
|
| 21 |
+
[](https://sensecraft.seeed.cc/ai-lab/zh/models?device=rk3588)
|
| 22 |
+
|
| 23 |
+
### ⚡ 一条命令即可运行
|
| 24 |
+
|
| 25 |
+
安装 Docker 后,在 reComputer AI Lab 中选择模型和开发板,复制部署命令,并在设备终端中运行即可。
|
| 26 |
+
|
| 27 |
+
**[探索模型与部署指南 →](https://sensecraft.seeed.cc/ai-lab/zh/models?device=rk3588)**
|
| 28 |
+
|
| 29 |
+
---
|
| 30 |
+
|
| 31 |
+
|
| 32 |
+
# Qwen3-1.7B · RKLLM
|
| 33 |
+
|
| 34 |
+
* 最大上下文长度: 4096
|
| 35 |
+
|
| 36 |
+
在 Rockchip RK3576 和 RK3588 上进行文本生成,转换自 [Qwen/Qwen3-1.7B](https://huggingface.co/Qwen/Qwen3-1.7B)。
|
| 37 |
+
|
| 38 |
+
- **转换工具包:** RKLLM Toolkit 1.3.0。
|
| 39 |
+
- **输入 / 输出:** 文本 → 文本。
|
| 40 |
+
- **许可证:** `apache-2.0`;条款及预期用途请参阅[上游模型卡](https://huggingface.co/Qwen/Qwen3-1.7B)。
|
| 41 |
+
|
| 42 |
+
## 选择模型文件
|
| 43 |
+
|
| 44 |
+
请选择与开发板 SoC 完全匹配的文件。`w4a16` 使用 4 位权重和 16 位激活;`w4a16_g128` 的权重分组大小为 128。`w8a8` 使用 8 位权重和 8 位激活。下表列出的是下载大小,并非运行时总内存占用。
|
| 45 |
+
|
| 46 |
+
| 开发板 | 量化方式 | 大小 | 语言模型 |
|
| 47 |
+
| --- | --- | --- | --- |
|
| 48 |
+
| RK3576 | `w4a16` | 1.57 GiB | [Qwen3-1.7B_RK3576_w4a16.rkllm](RK3576/Qwen3-1.7B_RK3576_w4a16.rkllm) |
|
| 49 |
+
| RK3576 | `w4a16_g128` | 1.69 GiB | [Qwen3-1.7B_RK3576_w4a16_g128.rkllm](RK3576/Qwen3-1.7B_RK3576_w4a16_g128.rkllm) |
|
| 50 |
+
| RK3576 | `w8a8` | 2.22 GiB | [Qwen3-1.7B_RK3576_w8a8.rkllm](RK3576/Qwen3-1.7B_RK3576_w8a8.rkllm) |
|
| 51 |
+
| RK3588 | `w8a8` | 2.21 GiB | [Qwen3-1.7B_RK3588_w8a8.rkllm](RK3588/Qwen3-1.7B_RK3588_w8a8.rkllm) |
|
| 52 |
+
|
| 53 |
+
## 在开发板上运行
|
| 54 |
+
|
| 55 |
+
在匹配的 Rockchip 开发板上安装 Docker 后,以下示例将启动 **RK3576 w4a16_g128** 模型,并在端口 8001 提供 API:
|
| 56 |
+
|
| 57 |
+
```bash
|
| 58 |
+
sudo docker run --rm -d --name rkllm \
|
| 59 |
+
--privileged -v /dev:/dev -p 8001:8001 \
|
| 60 |
+
ghcr.io/hanzo-huang/rkllm-docker/llm/qwen3-1.7b:rk3576-w4a16-g128
|
| 61 |
+
```
|
| 62 |
+
|
| 63 |
+
镜像已包含模型文件。API 请求、其他开发板版本和配置说明请参阅 [rkllm-docker](https://github.com/Hanzo-Huang/rkllm-docker#readme)。
|
| 64 |
+
|
| 65 |
+
如需单独下载文件:
|
| 66 |
+
|
| 67 |
+
```bash
|
| 68 |
+
hf download HanzoHuang/Qwen3-1.7B-RKLLM \
|
| 69 |
+
RK3576/Qwen3-1.7B_RK3576_w4a16_g128.rkllm \
|
| 70 |
+
--local-dir ./models
|
| 71 |
+
```
|
| 72 |
+
|
| 73 |
+
这些文件是原生 RKLLM/RKNN 模型产物。集成到其他应用时,请使用兼容的 RKLLM 运行时及该模型对应的对话模板。
|
| 74 |
+
|
| 75 |
+
## 发布说明
|
| 76 |
+
|
| 77 |
+
此处未发布各配置在开发板上的基准测试。量化后的输出可能与原始模型不同,请使用自己的实际任务检查回答效果。检查运行时兼容性时,请参考上方列出的工具包版本。
|
| 78 |
+
|
| 79 |
+
<details>
|
| 80 |
+
<summary>SHA-256 校验和</summary>
|
| 81 |
+
|
| 82 |
+
```text
|
| 83 |
+
9f5f2facdef5f1da27756524f0284b307d5e4cee45ae9d9c8a02b6e9d8888ce2 RK3576/Qwen3-1.7B_RK3576_w4a16.rkllm
|
| 84 |
+
c72954ad5bd155aa49f89a86cb2133b79fafd502685d345973e26bb07a9c14a7 RK3576/Qwen3-1.7B_RK3576_w4a16_g128.rkllm
|
| 85 |
+
2d1c1ec4e50a9edb85b801f546d935943481e7671d3a73d19a2df9f18d4da2ca RK3576/Qwen3-1.7B_RK3576_w8a8.rkllm
|
| 86 |
+
384294b54bf68b61590c4d818c33e27d67dbeced51fb9f1f93014f0ad06d2545 RK3588/Qwen3-1.7B_RK3588_w8a8.rkllm
|
| 87 |
+
```
|
| 88 |
+
|
| 89 |
+
</details>
|
| 90 |
+
|
| 91 |
+
## 致谢
|
| 92 |
+
|
| 93 |
+
原始模型:[Qwen/Qwen3-1.7B](https://huggingface.co/Qwen/Qwen3-1.7B)。转换工具:[Rockchip RKLLM](https://github.com/airockchip/rknn-llm)。
|