Hugging Face's logo

mjbommar
/
ogbert-tokenizer-32768

Transformers
English
tokenizer
bpe
ogbert
modernbert
opengloss
Model card Files
xet
 Community
ogbert-tokenizer-32768

Commit History

  1. Upload OGBERT tokenizer (vocab_size=32768)

    mjbommar
    Verified signature
    862a162

  2. Upload OGBERT tokenizer (vocab_size=32768)

    mjbommar
    Verified signature
    11977f1

  1. Add README with YAML metadata

    mjbommar
    Verified signature
    9b1c363

  2. Add special_tokens_map

    mjbommar
    Verified signature
    a86dd28

  3. Add tokenizer_config with special tokens

    mjbommar
    Verified signature
    4bfd63e

  1. Upload folder using huggingface_hub

    mjbommar
    Verified signature
    01a2d64

  2. initial commit

    mjbommar
    Verified signature
    7c88a1c