Hugging Face's logo

openeurollm
/
tokenizer-256k

tokenizer
sentencepiece
bpe
multilingual
european-languages
Model card Files
xet
 Community
1
tokenizer-256k / tokenizer.model
timpal0l's picture
timpal0l
Upload 262k vocab SentencePiece BPE tokenizer (trained on 175GB)
f1d2f3d verified
History Blame Contribute Delete
4.88 MB
This file is stored with Xet . It is too big to display, but you can still download it.

Xet Pointer Details

( Raw pointer file )
Xet hash:
cf3c8d57733c14529cc6e6eb1f6a49d7cf3e66c2dd236694b1008dd73685b082
Size of remote file:
4.88 MB
·
SHA256:
ac214401227404105817141c21c6693e3b17c878465a42765d2c345310e003ce

Xet efficiently stores Large Files inside Git, intelligently splitting files into unique chunks and accelerating uploads and downloads. More info.