Commit History
-
Update tokenizer_class to PreTrainedTokenizerFast for GGUF compatibility
Verified signature5d34c66 -
Add standard contractions to pre-tokenizer Regex for GGUF compatibility
Verified signaturef15cc49 -
Update tokenizer_class to LlamaTokenizerFast for GGUF compatibility
Verified signaturedb101bf
-
Upload 4 files
Verified signaturec2b3af0 -
initial commit
Verified signature70fa015