FlyGPT / tokenizer_config.json

Model card: inference and training examples (both executed), five-seed result; fused connectome-kernels path in modeling code

f7c186c verified 17 days ago

152 Bytes

	{
	"tokenizer_class": "PreTrainedTokenizerFast",
	"model_max_length": 10000000,
	"clean_up_tokenization_spaces": false,
	"add_prefix_space": false
	}