compactlm-5m / tokenizer.json

Commit History

Add gollem_eval byte-level BPE tokenizer (12,288 vocab)
1b61941
verified

Compactbot commited on