openeurollm
/

tokenizer-256k

european-languages

Model card Files Files and versions

tokenizer-256k / tokenizer.model

Commit History

Feb 20

16:55
Upload 262k vocab SentencePiece BPE tokenizer (trained on 175GB)

timpal0l

Verified signature
f1d2f3d