mistral3-tokenizer / tokenizer.json

Commit History

3 days ago

  1. Add tokenizer with cls/sep/mask mapped to reserved placeholder tokens

    Verified signature
    6d9be6a