Commit History
11 days ago
-
rebuild tokenizer.tokie from tokenizer.json
Verified signature370a6d9
18 days ago
-
Add the tokie 0.2 tokenizer file
Verified signaturebcea4e4
-
Regenerate tokenizer.tkz as format v13 (self-contained: added/special tokens stored in-file)
Verified signature22150e9
-
Add model card README
Verified signature84f7e21 -
Add tokenizer_config.json from bert-base-uncased
Verified signaturee6b117d -
Add tokenizer.json from bert-base-uncased
Verified signaturedbc493c -
Add tokie banner
Verified signature530049b -
Update tokenizer.tkz to v11 (with pad_token_id)
Verified signature1aee37d -
initial commit
Verified signatureb135905