Update README.md
Browse files
README.md
CHANGED
|
@@ -1,3 +1,48 @@
|
|
| 1 |
---
|
| 2 |
license: apache-2.0
|
| 3 |
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
---
|
| 2 |
license: apache-2.0
|
| 3 |
---
|
| 4 |
+
# RWKV-7 60M Mobile Pretrained (English)
|
| 5 |
+
|
| 6 |
+
This is a pretrained language model based on the cutting-edge **RWKV-7** architecture, optimized specifically for local fine-tuning and inference directly on your **iPhone**.
|
| 7 |
+
|
| 8 |
+
Thanks to its ultra-lightweight design and the linear complexity of the RNN-like RWKV architecture, it delivers high performance, low latency, and minimal power consumption, making it an ideal choice for edge computing on mobile devices.
|
| 9 |
+
|
| 10 |
+
---
|
| 11 |
+
|
| 12 |
+
## 📊 Model Specifications
|
| 13 |
+
|
| 14 |
+
Below are the key technical parameters used by the application to initialize the network and tokenizer:
|
| 15 |
+
|
| 16 |
+
| Parameter | Value |
|
| 17 |
+
| :--- | :--- |
|
| 18 |
+
| **Architecture** | RWKV-7 |
|
| 19 |
+
| **Total Parameters** | 60M |
|
| 20 |
+
| **Layers** | 18 |
|
| 21 |
+
| **Hidden Size** | 448 |
|
| 22 |
+
| **Vocab Size** | 16,000 (16k) |
|
| 23 |
+
| **Tokenizer** | BPE (Byte-Pair Encoding) |
|
| 24 |
+
| **Primary Language** | English |
|
| 25 |
+
| **License** | Apache 2.0 |
|
| 26 |
+
|
| 27 |
+
---
|
| 28 |
+
|
| 29 |
+
## 📱 On-Device Fine-Tuning & Inference on iPhone
|
| 30 |
+
|
| 31 |
+
This model is tailored to fit within the strict RAM constraints of iOS. A footprint of ~60 million parameters allows you to perform local fine-tuning and text generation without overwhelming the device's available memory.
|
| 32 |
+
|
| 33 |
+
### Key Mobile Features:
|
| 34 |
+
* **Low Memory Footprint:** Easily fits into RAM, leaving plenty of headroom for the application's UI and system processes.
|
| 35 |
+
* **Efficient BPE Tokenizer:** A 16k vocabulary optimized specifically for fast, low-overhead English text processing on mobile processors.
|
| 36 |
+
* **RWKV-7 Architecture Advantage:** Combines the generation quality of Transformers with the computational efficiency of RNNs, preserving your iPhone's battery life during inference and training.
|
| 37 |
+
|
| 38 |
+
> 💡 **Fine-Tuning Tip:** When training inside the app, we recommend using compact text datasets (such as personal notes, specific documentation, or custom dialogue logs). Local training ensures absolute privacy — your data never leaves your device.
|
| 39 |
+
|
| 40 |
+
---
|
| 41 |
+
|
| 42 |
+
## 🔒 License
|
| 43 |
+
|
| 44 |
+
This model and its weights are distributed under the **Apache 2.0** license. You are free to use, modify, and distribute it for both personal and commercial applications.
|
| 45 |
+
|
| 46 |
+
**ImpulseLeap / Alexei Goncharov**
|
| 47 |
+
|
| 48 |
+
www.impulseleap.com
|