Instructions to use DIAL-TFM/TFM-tokenizer with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use DIAL-TFM/TFM-tokenizer with Transformers:
# pip install -U transformers accelerate # Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("DIAL-TFM/TFM-tokenizer", device_map="auto") - Notebooks
- Google Colab
- Kaggle
Commit History
-
Upload 7 files
Verified signature037092e
-
Add special token for table tokenization (#1)
Verified signature- Add special token for table tokenization (fa6d40e859b2a2e7d1580cd38d93471e0188e3c4)1fc2308
-
Update README.md
Verified signature6108270 -
Update README.md
Verified signature1741cf2 -
Update chat_template.jinja
Verified signature4b707f1 -
Upload tokenizer
Verified signature75cdaf0 -
initial commit
Verified signatureed16a20