Arabic
arabic
tokenizer
sentencepiece
unigram
aratoken

Commit History

  1. Add model card

    Verified signature
    6feb597
  2. Add extended tokenizer fw2hq-alif-unified-150k

    Verified signature
    2c108c6
  3. Add extended tokenizer fw2hq-alif-preserve-150k

    Verified signature
    b4bb41b
  4. Add fw2hq_wp_keep_norm

    Verified signature
    415d0ed
  5. Add fw2hq_wp_keep

    Verified signature
    d360a54
  6. Add fw2hq_wp_drop_norm

    Verified signature
    ebb2fac
  7. Add fw2hq_wp_drop

    Verified signature
    b296804
  8. Add fw2hq_sp_keep_norm

    Verified signature
    81be153
  9. Add fw2hq_sp_keep

    Verified signature
    eca30dc
  10. Add fw2hq_sp_drop_norm

    Verified signature
    cbd57e9
  11. Add fw2hq_sp_drop

    Verified signature
    8d87cde
  12. Add fw2hq_sp_alif_unified_0_99

    Verified signature
    b2b4a3e
  13. Add fw2hq_sp_alif_unified_0_95

    Verified signature
    b857855
  14. Add fw2hq_sp_alif_unified

    Verified signature
    33085cc
  15. Add fw2hq_sp_alif_preserve_bf

    Verified signature
    ebf2c0a
  16. Add fw2hq_sp_alif_preserve_0_99

    Verified signature
    76c8e18
  17. Add fw2hq_sp_alif_preserve_0_95

    Verified signature
    9f19048
  18. Add fw2hq_sp_alif_preserve

    Verified signature
    2f0f97e
  19. Add fw2hq_bpe_keep_norm

    Verified signature
    de866ac
  20. Add fw2hq_bpe_keep

    Verified signature
    bac62eb
  21. Add fw2hq_bpe_drop_norm

    Verified signature
    df772d9
  22. Add fw2hq_bpe_drop

    Verified signature
    6be5c6a
  23. initial commit

    Verified signature
    5cb312e