Hugging Face's logo

OiQ
/
daa-tokenizers

Arabic
Latin
tokenizer
moroccan-darija
arabic
bpe
unigram
wordpiece
bbpe
benchmark
Model card Files
xet
 Community
daa-tokenizers / transformers_tokenizers /shared_bbpe_8000

Commit History

  1. Upload transformers_tokenizers/shared_bbpe_8000/tokenizer.json with huggingface_hub

    Ouaill
    Verified signature
    18e6260

  2. Upload transformers_tokenizers/shared_bbpe_8000/tokenizer_config.json with huggingface_hub

    Ouaill
    Verified signature
    19831e4

  3. Upload transformers_tokenizers/shared_bbpe_8000/special_tokens_map.json with huggingface_hub

    Ouaill
    Verified signature
    875c37d