Duplicated from hf-internal-testing/llama-code-tokenizer

osanseviero
/

llama-code-tokenizer

Model card Files Files and versions

llama-code-tokenizer / tokenizer_config.json

Commit History

Nov 23, 2023

13:07
Upload tokenizer

osanseviero

2b2c68f
13:06
Duplicate from hf-internal-testing/llama-code-tokenizer⁠

Omar Sanseviero +1: ArthurZ

cc6d6bc