Instructions to use DIAL-TFM/TFM-tokenizer with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use DIAL-TFM/TFM-tokenizer with Transformers:
# pip install -U transformers accelerate # Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("DIAL-TFM/TFM-tokenizer", device_map="auto") - Notebooks
- Google Colab
- Kaggle
Commit History
-
Add special token for table tokenization
AUGUST-GKDfa6d40e
-
Update README.md
Verified signature6108270 -
Update README.md
Verified signature1741cf2 -
Update chat_template.jinja
Verified signature4b707f1 -
Upload tokenizer
Verified signature75cdaf0 -
initial commit
Verified signatureed16a20