Hugging Face's logo

flexitok
/
maddition_fas_Arab_2000

Persian
tokenizer
bpe
flexitok
fineweb2
Model card Files
xet
 Community
maddition_fas_Arab_2000

Commit History

  1. Upload maddition/flexitok--maddition_fas_Arab_2000_vocab.json with huggingface_hub

    gsaltintas
    Verified signature
    739073f

  2. Upload maddition/flexitok--maddition_fas_Arab_2000_super_mapping.json with huggingface_hub

    gsaltintas
    Verified signature
    32a2ece

  3. Upload folder using huggingface_hub

    gsaltintas
    Verified signature
    45f8197

  1. Upload maddition/flexitok--maddition_fas_Arab_2000_overlap.json with huggingface_hub

    gsaltintas
    Verified signature
    091f7ab

  2. Upload maddition/participating_tokenizers.json with huggingface_hub

    gsaltintas
    Verified signature
    64eab85

  3. Upload maddition/flexitok--maddition_fas_Arab_2000.yaml with huggingface_hub

    gsaltintas
    Verified signature
    76beb90

  4. Upload maddition/flexitok--maddition_fas_Arab_2000_info.json with huggingface_hub

    gsaltintas
    Verified signature
    fab87d0

  5. Upload maddition/flexitok--maddition_fas_Arab_2000_vocab.json with huggingface_hub

    gsaltintas
    Verified signature
    8b48634

  6. Upload maddition/flexitok--maddition_fas_Arab_2000_super_mapping.json with huggingface_hub

    gsaltintas
    Verified signature
    f118766

  7. Upload folder using huggingface_hub

    gsaltintas
    Verified signature
    e21cba0

  8. Upload folder using huggingface_hub

    gsaltintas
    Verified signature
    99c7cf4

  9. Upload folder using huggingface_hub

    gsaltintas
    Verified signature
    7d36c99

  10. initial commit

    gsaltintas
    Verified signature
    54828af