Download v6_byte_tokenizer.json from quachtohai/mini-code-generator-v6: direct link, hf CLI and curl.
- Browser
- Download file 111 Bytes
-
https://huggingface.co/quachtohai/mini-code-generator-v6/resolve/main/v6_byte_tokenizer.json
- Command line
-
hf download hf://quachtohai/mini-code-generator-v6/v6_byte_tokenizer.json
-
curl -L -o v6_byte_tokenizer.json https://huggingface.co/quachtohai/mini-code-generator-v6/resolve/main/v6_byte_tokenizer.json
111 Bytes
| { | |
| "type": "utf8_byte", | |
| "pad_id": 0, | |
| "bos_id": 1, | |
| "eos_id": 2, | |
| "byte_offset": 3, | |
| "vocab_size": 259 | |
| } |