simanto848/maya2-native-bucket / backbone /tokenizer_config.json
simanto848's picture
download
raw
159 Bytes
{
"backend": "tokenizers",
"eos_token": "<|endoftext|>",
"model_max_length": 1000000000000000019884624838656,
"tokenizer_class": "TokenizersBackend"
}

Xet Storage Details

Size:
159 Bytes
·
Xet hash:
f98d4cb64600eb6422c81b88f94028a487697c8e84415ab11455afd75ce67101

Xet efficiently stores files, intelligently splitting them into unique chunks and accelerating uploads and downloads. More info.