amphora
/

L16-V65536-chat-tokenizer

Model card Files Files and versions

L16-V65536-chat-tokenizer / tokenizer.json

amphora's picture

L16-V65536 tokenizer with Llama-3 chat specials in reserved ids 4-8

1e037db verified about 1 month ago

Raw History Contribute Delete

6.25 MB

File too large to display, you can check the raw version instead.