picur-tokenizer / tokenizer_config.json

Commit History

  1. picur-tokenizer a2x: 49,152-piece Hungarian Unigram vocabulary

    Verified signature
    a7cf8a7