Tasfiya025
/

shorif-textgen-basic

shorif-textgen-basic / tokenizer_config.json

Create tokenizer_config.json

a40a9bb verified 11 months ago

124 Bytes

{
  "tokenizer_class": "GPT2Tokenizer",
  "model_max_length": 256,
  "padding_side": "left",
  "truncation_side": "right"
}

	{
	"tokenizer_class": "GPT2Tokenizer",
	"model_max_length": 256,
	"padding_side": "left",
	"truncation_side": "right"
	}