Hugging Face's logo

OiQ
/
daa-tokenizers

Arabic
Latin
tokenizer
moroccan-darija
arabic
bpe
unigram
wordpiece
bbpe
benchmark
Model card Files
xet
 Community
daa-tokenizers / plots

Commit History

  1. Upload plots/external_comparison.png with huggingface_hub

    Ouaill
    Verified signature
    7b89ea4

  2. Upload plots/fertility_disparity_heatmap_v2.png with huggingface_hub

    Ouaill
    Verified signature
    e65894a

  3. Upload plots/fertility_disparity_comparison_v2.png with huggingface_hub

    Ouaill
    Verified signature
    9934a1e

  4. Upload plots/fertility_overall_trends.png with huggingface_hub

    Ouaill
    Verified signature
    e6441c9

  5. Upload plots/fertility_overall_comparison_v2.png with huggingface_hub

    Ouaill
    Verified signature
    08b4403

  1. Upload plots/fertility_overall_faceted.png with huggingface_hub

    Ouaill
    Verified signature
    6ee779e

  2. Upload plots/morph_edit_distance_ar_comparison_v2.png with huggingface_hub

    Ouaill
    Verified signature
    79e19cc

  3. Upload plots/cpt_overall_comparison_v2.png with huggingface_hub

    Ouaill
    Verified signature
    13fbc56

  4. Upload plots/exact_match_rate_comparison_v2.png with huggingface_hub

    Ouaill
    Verified signature
    75e957e

  5. Upload plots/fertility_disparity_heatmap_v2.png with huggingface_hub

    Ouaill
    Verified signature
    5d31685

  6. Upload plots/fertility_disparity_trends.png with huggingface_hub

    Ouaill
    Verified signature
    83c46df

  7. Upload plots/fertility_disparity_comparison_v2.png with huggingface_hub

    Ouaill
    Verified signature
    7059b0c

  8. Upload plots/morph_consistency_f1_comparison_v2.png with huggingface_hub

    Ouaill
    Verified signature
    bfa6f98

  9. Upload plots/morph_consistency_f1_heatmap_v2.png with huggingface_hub

    Ouaill
    Verified signature
    9eb984a

  10. Upload plots/fertility_overall_comparison_v2.png with huggingface_hub

    Ouaill
    Verified signature
    79fac54

  11. Upload plots/morph_edit_distance_ar_heatmap_v2.png with huggingface_hub

    Ouaill
    Verified signature
    d760658

  12. Upload plots/morph_edit_distance_ar_trends.png with huggingface_hub

    Ouaill
    Verified signature
    0b5bb9d

  13. Upload plots/exact_match_rate_heatmap_v2.png with huggingface_hub

    Ouaill
    Verified signature
    77fbb17

  14. Upload plots/cpt_overall_trends.png with huggingface_hub

    Ouaill
    Verified signature
    7338ed2

  15. Upload plots/fertility_overall_heatmap_v2.png with huggingface_hub

    Ouaill
    Verified signature
    d465e60

  16. Upload plots/exact_match_rate_trends.png with huggingface_hub

    Ouaill
    Verified signature
    1d04332

  17. Upload plots/fertility_overall_trends.png with huggingface_hub

    Ouaill
    Verified signature
    68b4b4b

  18. Upload plots/bootstrap_ci_forest.png with huggingface_hub

    Ouaill
    Verified signature
    7ce868c

  19. Upload plots/external_comparison.png with huggingface_hub

    Ouaill
    Verified signature
    0e4e973

  20. Upload plots/morph_consistency_f1_trends.png with huggingface_hub

    Ouaill
    Verified signature
    308a054

  21. Upload plots/cpt_overall_faceted.png with huggingface_hub

    Ouaill
    Verified signature
    87aaa10

  22. Upload plots/fertility_disparity_faceted.png with huggingface_hub

    Ouaill
    Verified signature
    b61125f

  23. Upload plots/cpt_overall_heatmap_v2.png with huggingface_hub

    Ouaill
    Verified signature
    7362657

  24. Upload plots/script_comparison_v2.png with huggingface_hub

    Ouaill
    Verified signature
    a87618b

  25. Upload plots/oov_disparity_comparison_v2.png with huggingface_hub

    Ouaill
    Verified signature
    2f636a1

  1. Update comparison plot with 9 external tokenizers

    Ouaill
    Verified signature
    1f947e5

  2. Upload plots/external_comparison.png with huggingface_hub

    Ouaill
    Verified signature
    02a40b6

  3. Upload plots/script_comparison_v2.png with huggingface_hub

    Ouaill
    Verified signature
    296f159

  4. Upload plots/oov_disparity_comparison_v2.png with huggingface_hub

    Ouaill
    Verified signature
    7130cb2

  5. Upload plots/morph_edit_distance_ar_trends.png with huggingface_hub

    Ouaill
    Verified signature
    64a1a4d

  6. Upload plots/morph_edit_distance_ar_heatmap_v2.png with huggingface_hub

    Ouaill
    Verified signature
    80b7629

  7. Upload plots/morph_edit_distance_ar_comparison_v2.png with huggingface_hub

    Ouaill
    Verified signature
    07b5369

  8. Upload plots/morph_consistency_f1_trends.png with huggingface_hub

    Ouaill
    Verified signature
    aeb25f3

  9. Upload plots/morph_consistency_f1_heatmap_v2.png with huggingface_hub

    Ouaill
    Verified signature
    af60f52

  10. Upload plots/morph_consistency_f1_comparison_v2.png with huggingface_hub

    Ouaill
    Verified signature
    61bf1f5

  11. Upload plots/fertility_overall_trends.png with huggingface_hub

    Ouaill
    Verified signature
    1bc1ca8

  12. Upload plots/fertility_overall_heatmap_v2.png with huggingface_hub

    Ouaill
    Verified signature
    6ee6157

  13. Upload plots/fertility_overall_faceted.png with huggingface_hub

    Ouaill
    Verified signature
    15905d6

  14. Upload plots/fertility_overall_comparison_v2.png with huggingface_hub

    Ouaill
    Verified signature
    1796080

  15. Upload plots/fertility_disparity_trends.png with huggingface_hub

    Ouaill
    Verified signature
    f5a9b3c

  16. Upload plots/fertility_disparity_heatmap_v2.png with huggingface_hub

    Ouaill
    Verified signature
    cba721e

  17. Upload plots/fertility_disparity_faceted.png with huggingface_hub

    Ouaill
    Verified signature
    eacdfc1

  18. Upload plots/fertility_disparity_comparison_v2.png with huggingface_hub

    Ouaill
    Verified signature
    d4ef29c

  19. Upload plots/exact_match_rate_trends.png with huggingface_hub

    Ouaill
    Verified signature
    9e83d1e

  20. Upload plots/exact_match_rate_heatmap_v2.png with huggingface_hub

    Ouaill
    Verified signature
    8b1a88e