maxcushion / tokenizer

Commit History

  1. Create tokenizer/special_tokens_map.json

    Verified signature
    9989ac0
  2. Create tokenizer/merges.txt

    Verified signature
    402b7e3
  3. Create tokenizer/vocab.json

    Verified signature
    0564d98
  4. Create tokenizer/vocab.json

    Verified signature
    ce4d6ab
  5. Create tokenizer/tokenizer_config.json

    Verified signature
    1c32642