Update README.md
Browse files
README.md
CHANGED
|
@@ -17,7 +17,7 @@ pipeline_tag: text-generation
|
|
| 17 |
|
| 18 |
# 🚀 Fortaki Multi Mega BPE Tokenizer Suite (120K & 65K)
|
| 19 |
|
| 20 |
-
**BPEF** is a family of ultra-optimized, custom **Byte-Level BPE tokenizers** available in two configurations: a massive flagship **120,000 token vocabulary** and a lean, compute-efficient **65,000 token vocabulary**.
|
| 21 |
|
| 22 |
Engineered from scratch by `fortik11`, this suite is specifically designed to eliminate the heavy "tokenization tax" on Cyrillic and highly agglutinative Central Asian languages, redefining compression standards for **Russian and Kyrgyz** NLP.
|
| 23 |
|
|
|
|
| 17 |
|
| 18 |
# 🚀 Fortaki Multi Mega BPE Tokenizer Suite (120K & 65K)
|
| 19 |
|
| 20 |
+
**BPEF** - is a family of ultra-optimized, custom **Byte-Level BPE tokenizers** available in two configurations: a massive flagship **120,000 token vocabulary** and a lean, compute-efficient **65,000 token vocabulary**.
|
| 21 |
|
| 22 |
Engineered from scratch by `fortik11`, this suite is specifically designed to eliminate the heavy "tokenization tax" on Cyrillic and highly agglutinative Central Asian languages, redefining compression standards for **Russian and Kyrgyz** NLP.
|
| 23 |
|