Transformers How to use youliangtan/fast_tokenizer-test with Transformers:
# pip install -U transformers accelerate
# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("youliangtan/fast_tokenizer-test", device_map="auto") 
