openeurollm
/

tokenizer-128k

european-languages

Model card Files Files and versions

tokenizer-128k / tokenizer.model

Commit History

Feb 20

16:55
Upload 128k vocab SentencePiece BPE tokenizer (trained on 175GB)

timpal0l

Verified signature
43f50d5