xscript-eval / tokenizers

Commit History

  1. tokenizer unigram_bi_zh

    Verified signature
    a0e05b4
  2. tokenizer unigram_bi_ar

    Verified signature
    c9dda21
  3. tokenizer unigram_bi_fr

    Verified signature
    3029eb0
  4. tokenizer unigram_bi_de

    Verified signature
    b688209

  1. tokenizer unigram_20lang

    Verified signature
    ec73726

  1. Upload tokenizers/unigram_destarved/meta.json with huggingface_hub

    Verified signature
    ae2e4c1
  2. Upload tokenizers/unigram_destarved/sp.model with huggingface_hub

    Verified signature
    7c0d0ff
  3. Upload tokenizers/unigram_starved/meta.json with huggingface_hub

    Verified signature
    257520a
  4. Upload tokenizers/unigram_starved/sp.model with huggingface_hub

    Verified signature
    6d9c6d2