ASTERIZER
/

LUNA-Tokenizer

Token Classification

text-generation

Model card Files Files and versions

LUNA-Tokenizer / tokenizer.json

REZIRETSA's picture

Add shared LUNA tokenizer (GPT-NeoX, vocab 50304)

31bc007 verified about 1 month ago

Raw History Contribute Delete

2.11 MB

File too large to display, you can check the raw version instead.