Instructions to use pietrolesci/tokenizers with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use pietrolesci/tokenizers with Transformers:
# pip install -U transformers accelerate # Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("pietrolesci/tokenizers", device_map="auto") - Notebooks
- Google Colab
- Kaggle
Commit History
-
Upload folder using huggingface_hub
Verified signaturef634b8a -
Upload folder using huggingface_hub
Verified signature5181567 -
Upload folder using huggingface_hub
Verified signature4fee282 -
Upload tokenizer
Verified signatureefcbeec -
initial commit
Verified signatureec02ac4