The purpose of this model is to experiment BPE tokenization with a very large vocabulary (2m tokens).
Applications include: text compression, language models, search engines.
I recommend gigatoken to run this model: https://github.com/marcelroed/gigatoken .

Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support