Hugging Face's logo

ByteSpanTokenisers
/
tokenizers

Model card Files
xet
 Community
tokenizers

Commit History

  1. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    e59664c

  2. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    da96c89

  3. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    442fcdc

  4. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    6fbaf12

  5. Fix normalizer

    Zeb
    b4a21a8

  6. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    86ff86d

  7. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    a2b66e6

  8. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    0c1a225

  9. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    1d2f9ce

  10. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    65f8a9f

  11. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    b924598

  12. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    de15262

  13. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    11c071d

  14. Fix normalizer

    Zeb
    715480f

  15. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    251f9ae

  1. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    d459522

  2. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    ba09252

  3. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    39e93f3

  4. Fix normalizer

    Zeb
    36cd135

  5. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    565cc8c

  6. Fix normalizer

    Zeb
    d557553

  7. Fix normalizer

    Zeb
    bdf2d38

  8. Remove old merge tokenisers

    Zeb
    bb7d1d2

  9. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    884154b

  10. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    b215c5e

  11. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    8c571b6

  12. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    0c66f52

  1. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    cdd9e7b

  2. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    0290e96

  3. Bytelevel without normalizer

    Zeb
    d842b80

  4. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    30b14ac

  5. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    f1c1fba

  6. Remove normalizers

    Zeb
    64b3309

  7. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    38928ba

  8. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    847dbbd

  9. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    625b01f

  1. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    4637696

  2. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    aa577b2

  3. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    bfc2a21

  4. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    f1e8d54

  5. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    fa83c21

  6. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    4ab1b41

  7. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    4b9bf06

  8. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    46e77f8

  9. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    2527407

  10. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    4ed123d

  11. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    7e062f7

  12. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    a02f082

  13. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    6e5822f

  1. Upload folder using huggingface_hub

    codebyzeb
    Verified signature
    038fc30