Instructions to use wellflat/o200k_base_tokenizer with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use wellflat/o200k_base_tokenizer with Transformers:
# Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("wellflat/o200k_base_tokenizer", device_map="auto") - Notebooks
- Google Colab
- Kaggle
File size: 133 Bytes
6284f55 | 1 2 3 4 | version https://git-lfs.github.com/spec/v1
oid sha256:27f8a5a5997a1aa0e6ac889e213b24a99c342831ab59cccc1638b878370c20f0
size 27864518
|