vtuber / tokenizer_config.json
Junping0645's picture
QLoRA adapter: 사투리 시드 + 멀티턴 보강 (eval_loss 1.366)
323253c verified
Raw History Blame Contribute Delete
370 Bytes
{
"backend": "tokenizers",
"eos_token": "<|endoftext|>",
"extra_special_tokens": [
"<|endoftext|>",
"<|sep|>",
"<|acc|>",
"<|tel|>",
"<|rrn|>"
],
"is_local": false,
"local_files_only": false,
"model_max_length": 1000000000000000019884624838656,
"pad_token": "<|endoftext|>",
"tokenizer_class": "TokenizersBackend"
}