Instructions to use crumb/only_tokenizer_youll_ever_need with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use crumb/only_tokenizer_youll_ever_need with Transformers:
# pip install -U transformers accelerate # Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("crumb/only_tokenizer_youll_ever_need", device_map="auto") - Notebooks
- Google Colab
- Kaggle
Commit History
-
space
Verified signature75d8248 -
space
Verified signatureb7c920b -
Update README.md
Verified signature459ca5a -
Update README.md
Verified signaturea079b6f -
Update README.md
Verified signature9d43720 -
Update README.md
Verified signature9331038 -
Update README.md
Verified signaturef975670 -
Upload tokenizer
Verified signature45a6b05 -
initial commit
Verified signature4ed4fed