Duplicated from hf-internal-testing/llama-code-tokenizer

osanseviero
/

llama-code-tokenizer

Model card Files Files and versions

llama-code-tokenizer / tokenizer_config.json

Commit History

Nov 23, 2023

13:08
Upload tokenizer

osanseviero

4aa29f7
13:06
Duplicate from hf-internal-testing/llama-code-tokenizer⁠

Omar Sanseviero +1: ArthurZ

cc6d6bc