Update README.md
Browse files
README.md
CHANGED
|
@@ -16,24 +16,25 @@ library_name: PyTorch
|
|
| 16 |
|
| 17 |
# SmallLM
|
| 18 |
|
| 19 |
-
SmallLM is a specialized transformer-based language model with fewer than 50 million parameters. It utilizes a tokenizer
|
| 20 |
|
| 21 |
## Features
|
| 22 |
|
| 23 |
- Can be specialized for specific tasks
|
| 24 |
- Compact model size (less than 50M parameters)
|
| 25 |
-
- Custom tokenizer
|
| 26 |
- Trained on custom datasets
|
| 27 |
-
- Can run
|
| 28 |
|
| 29 |
## Limitations
|
| 30 |
|
| 31 |
-
Due to its relatively small size, limited training data, and shorter training duration, the model may occasionally underperform compared to
|
| 32 |
|
| 33 |
## Contact
|
| 34 |
|
| 35 |
For any questions or feedback, please reach out to us through Discussions.
|
| 36 |
|
| 37 |
## Resources
|
| 38 |
-
|
|
|
|
| 39 |
|
|
|
|
| 16 |
|
| 17 |
# SmallLM
|
| 18 |
|
| 19 |
+
SmallLM is a specialized transformer-based language model with fewer than 50 million parameters. It utilizes a tokenizer based on turboBPE, and has been trained on custom datasets. This model is currently in active development, and we are continuously working to enhance its performance. The model can be used on small end devices without any issues. The model has not been released yet, but we are working to make it available soon. Stay tuned\!
|
| 20 |
|
| 21 |
## Features
|
| 22 |
|
| 23 |
- Can be specialized for specific tasks
|
| 24 |
- Compact model size (less than 50M parameters)
|
| 25 |
+
- Custom tokenizer based on turboBPE
|
| 26 |
- Trained on custom datasets
|
| 27 |
+
- Can run on CPU-based devices
|
| 28 |
|
| 29 |
## Limitations
|
| 30 |
|
| 31 |
+
Due to its relatively small size, limited training data, and shorter training duration, the model may occasionally underperform compared to larger language models.
|
| 32 |
|
| 33 |
## Contact
|
| 34 |
|
| 35 |
For any questions or feedback, please reach out to us through Discussions.
|
| 36 |
|
| 37 |
## Resources
|
| 38 |
+
|
| 39 |
+
- Tokenizer Source: [turboBPE](<https://github.com/Amrendra-gupta/turbobpe>)
|
| 40 |
|