Hugging Face's logo

flexitok
/
parity-aware-bpe-ratio

Model card Files
xet
 Community
parity-aware-bpe-ratio / tokenizer.json

Commit History

  1. Upload folder using huggingface_hub

    gsaltintas
    Verified signature
    019176d

  2. Rename parity_bpe_ratio.json to tokenizer.json

    gsaltintas
    Verified signature
    0ebc340