tinysms_tokenizer / tokenizer.json

Commit History

  1. byte-level BPE, vocab 8192, trained on the TinyGSM train split

    Verified signature
    93f2b7d