phi-2 / tokenizer.tkz

Commit History

  1. Regenerate tokenizer.tkz as format v13 (self-contained: added/special tokens stored in-file)

    Verified signature
    052bdc1

  1. Upload tokenizer.tkz with huggingface_hub

    Verified signature
    ba4e378

  1. Update tokenizer.tkz to v11 (with pad_token_id)

    Verified signature
    6bad8c0