jintinghou
/

hugging-face-test

hugging-face-test / tokenizer_config.json

Upload tokenizer_config.json with huggingface_hub

5955321 verified over 1 year ago

99 Bytes

{
  "do_lower_case": true,
  "model_max_length": 512,
  "tokenizer_class": "DistilBertTokenizer"
}

	{
	"do_lower_case": true,
	"model_max_length": 512,
	"tokenizer_class": "DistilBertTokenizer"
	}