Delete tokenizer.json

Removing stale tokenizer.json (32K vocab) as discussed in transformers PR #48973 so AutoTokenizer relies on vocab.txt (100K vocab) via tokenizer_config.json.

Files changed (1) hide show

tokenizer.json +0 -0

tokenizer.json DELETED Viewed

The diff for this file is too large to render. See raw diff