Instructions to use wellflat/o200k_base_tokenizer with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use wellflat/o200k_base_tokenizer with Transformers:
# Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("wellflat/o200k_base_tokenizer", device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Xet hash:
- a692c081a43a9ee2bfe37e3b6cda5c1e97dc6e81a3ac07f74785faee5b96eba2
- Size of remote file:
- 27.9 MB
- SHA256:
- 27f8a5a5997a1aa0e6ac889e213b24a99c342831ab59cccc1638b878370c20f0
·
Xet efficiently stores Large Files inside Git, intelligently splitting files into unique chunks and accelerating uploads and downloads. More info.