Update README.md
Browse files
README.md
CHANGED
|
@@ -16,7 +16,7 @@ Custom multimodal tokenizer family for the NeonTech model series.
|
|
| 16 |
## Available Versions
|
| 17 |
|
| 18 |
- v1 — 131,072 vocab, DeepSeek V4 base + Igbo/Japanese/Arabic extensions, 80 special tokens
|
| 19 |
-
- See the [v1 branch](https://huggingface.co/Neon-tech/Tokenizers/
|
| 20 |
|
| 21 |
|
| 22 |
## Usage
|
|
|
|
| 16 |
## Available Versions
|
| 17 |
|
| 18 |
- v1 — 131,072 vocab, DeepSeek V4 base + Igbo/Japanese/Arabic extensions, 80 special tokens
|
| 19 |
+
- See the [v1 branch](https://huggingface.co/Neon-tech/Tokenizers/blob/v1/README.md) for full details on special tokens and architecture.
|
| 20 |
|
| 21 |
|
| 22 |
## Usage
|