Commit History
-
Fix tokenizer config for transformers v4 compatibility
Verified signaturetokenizer_class was 'TokenizersBackend', a name that only exists in transformers v5, so loading failed on v4 with 'Unrecognized processing class'. Set it to PreTrainedTokenizerFast and restrict model_input_names to input_ids/attention_mask, since ModernBertModel.forward() does not accept token_type_ids. Verified on transformers 4.57.6 and 5.17.0; embeddings unchanged.d956956
-
Add model weights, tokenizer and bilingual model card
Verified signature24b8beb
