system-test / tokenizer_config.json
gdsfbvdf's picture
Create tokenizer_config.json
075cd29 verified
Raw
History Blame Contribute Delete
67 Bytes
{
"tokenizer_class": "BertTokenizer",
"model_max_length": 512
}