Spaces:
Running
Running
Download weights/tokenizer.json from Tarul/pxg-tiny-site: direct link, hf CLI and curl.
- Browser
- Download file 228 Bytes
-
https://huggingface.co/spaces/Tarul/pxg-tiny-site/resolve/main/weights/tokenizer.json
- Command line
-
hf download hf://spaces/Tarul/pxg-tiny-site/weights/tokenizer.json
-
curl -L -o tokenizer.json https://huggingface.co/spaces/Tarul/pxg-tiny-site/resolve/main/weights/tokenizer.json
228 Bytes
| { | |
| "pad_id": 0, | |
| "bos_id": 1, | |
| "sep_id": 2, | |
| "caps_len": 32, | |
| "type": "char+byte-fallback-buckets", | |
| "charset": "abcdefghijklmnopqrstuvwxyz ,:-:'", | |
| "note": "ids: 0 pad,1 bos,2 sep,3-6 punct,7-32 a-z,33-62 byte buckets" | |
| } |