tokenizer-fineweb_ar / tokenizer.json

Commit History

  1. Register tokenizer as HF SentencePiece tokenizer

    Verified signature
    8d82b02