Instructions to use goryden/mn-en-tokenizer with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use goryden/mn-en-tokenizer with Transformers:
# pip install -U transformers accelerate # Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("goryden/mn-en-tokenizer", device_map="auto") - Notebooks
- Google Colab
- Kaggle
Download tokenizer.model from goryden/mn-en-tokenizer: direct link, hf CLI and curl.
- Browser
- Download file 888 kB
-
https://huggingface.co/goryden/mn-en-tokenizer/resolve/main/tokenizer.model
- Command line
-
hf download hf://goryden/mn-en-tokenizer/tokenizer.model
-
curl -L -o tokenizer.model https://huggingface.co/goryden/mn-en-tokenizer/resolve/main/tokenizer.model
888 kB
- Xet hash:
- c60c8edbf30d548a567e68a03576221db37ffe0ef77216df83e0af0f8fb69394
- Size of remote file:
- 888 kB
- SHA256:
- 0689d494ebc22c2b6460b939ee1ef4b07630a837e7f035f5e360e6cef3e8d513
·
Xet efficiently stores Large Files inside Git, intelligently splitting files into unique chunks and accelerating uploads and downloads. More info.