Neo / tokenizer /tokenizer_config.json
assemsabry's picture
docs: organize tokenizer and remove release numbering
97305e4 verified
Raw History Blame Contribute Delete
48 Bytes
{"do_lower_case": true, "model_max_length": 512}