TeluguTranslatica / tokenizer.json
hima06varshini's picture
Upload 7 files
9e1f23b verified
Raw
History Blame Contribute Delete
148 Bytes
Invalid JSON:Unexpected token 'r', "from transf"... is not valid JSON
from transformers import AutoTokenizer
tokenizer = AutoTokenizer.from_pretrained("your-model-name")
tokenizer.save_pretrained("model_directory")