Hugging Face's logo

balochiml
/
balochi-tokenizer

Baluchi
balochi
tokenizer
Model card Files
xet
 Community
balochi-tokenizer / src

Commit History

  1. more spacy

    strickvl
    e938b17

  2. add spacy trainer

    strickvl
    b1d540a

  3. update trainer script

    strickvl
    e8868b3

  4. update tokenizer script

    strickvl
    57c829b

  5. add tokenization process

    strickvl
    24469e0

  6. process and clean files

    strickvl
    b3db2b8

  7. Get all text paths

    strickvl
    470e696

  8. add data folder to gitignore

    strickvl
    c9036ae