CodeLlama-7b-hf / tokenizer.tkz

Commit History

  1. Regenerate tokenizer.tkz as format v13 (self-contained: added/special tokens stored in-file)

    Verified signature
    6872c26

  1. Update tokenizer.tkz to v11 (with pad_token_id)

    Verified signature
    325b489