Instructions to use Joycean0301/test-tokenizer with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use Joycean0301/test-tokenizer with Transformers:
# pip install -U transformers accelerate # Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("Joycean0301/test-tokenizer", device_map="auto") - Notebooks
- Google Colab
- Kaggle
Commit History
-
Update added_tokens.json
Verified signature2f127a6 -
Update tokenizer_config.json
Verified signatureff189f8 -
Update tokenizer_config.json
Verified signaturee741b31 -
Update added_tokens.json
Verified signature26a26d7 -
Update tokenizer_config.json
Verified signaturef0449ed -
Upload tokenizer
Verified signature5a07af4 -
initial commit
Verified signaturee9e8ae2