Hugging Face's logo

lsmpp
/
qwenillustrious

Model card Files
xet
 Community
1
qwenillustrious / transformers /utils /test_module /custom_tokenization.py
lsmpp's picture
lsmpp
Add files using upload-large-folder tool
d477207 verified
Raw
History Blame Contribute Delete
88 Bytes
from transformers import BertTokenizer
class CustomTokenizer(BertTokenizer):
 pass