amphora
/

L16-V65536-chat-tokenizer

Model card Files Files and versions

L16-V65536-chat-tokenizer / tokenizer.model

amphora's picture

L16-V65536 tokenizer with Llama-3 chat specials in reserved ids 4-8

1e037db verified 27 days ago

History Blame Contribute Delete

1.12 MB

This file is stored with Xet . It is too big to display, but you can still download it.

Xet Pointer Details

( Raw pointer file )

Xet hash:: e2388fcc7329cd4bdf8e3c713ce7cc87e69b876b0ba354a0eb8a66b52a6dd726
Size of remote file:: 1.12 MB
SHA256:: 08070d325bea4a6f4ee0a924f4f9019aa89bc0e4066b55b0cde52a82d3ec4966

Xet efficiently stores Large Files inside Git, intelligently splitting files into unique chunks and accelerating uploads and downloads. More info.