Commit History
-
Add `"add_prefix_space": true,`; this allows for much stronger token-level performance (e.g. NER, ColBERT)
Verified signatureHello! ## Pull Request overview * Add `"add_prefix_space": true,` to the tokenizer config ## Details This allows for much stronger token-level performance (e.g. NER, ColBERT), because otherwise each token will not be prepended by a space, while our model is trained with data where each token is prepended by a space. We will need to explain that users can set `add_prefix_space` to False in the model card somewhere. cc @bclavie @bwarner @NohTow could one of you take care of that? P.s. feel free to hold off on merging for now, this PR can also be used to run some tests first (with `revision="refs/pr/..."`). Note that you need to use `transformers` after https://github.com/huggingface/transformers/pull/35593 was merged. - Tom Aarsen8ae8af3
-
Update README.md
Verified signaturee829787
-
Bump `max_position_embeddings` to 8192
Verified signaturealso harmonize `layer_norm_eps` with `norm_eps` although the former isn't usedf87846c -
Add links to answer.ai and lighton.ai
Verified signature52efb96 -
Update README.md
Verified signaturee278db6 -
Add "If your GPU supports it, "
Verified signaturea494d8e -
Upload ONNX weights (#1)
Verified signature- Upload ONNX weights (8a783c9c2ced8d98e0c89d2db641d2ed74b27ddb) Co-authored-by: Joshua <Xenova@users.noreply.huggingface.co>7235292 -
Update snippet slightly
Verified signaturee0e1696 -
Strongly suggest -> Recommend
Verified signature2706a3e -
Remove now unused unpad_no_grad
Tom Aarsenba750de -
Update README.md
Verified signaturebc1758e -
Update README.md
Verified signaturefd923f4 -
Update README.md
Verified signature8ae8825
-
Update README.md
Verified signaturec72b2bd -
Update README.md
Verified signature0480fd1 -
Create README.md
Verified signature2097f35
-
Purge duplicate "decoder.weight", rely on tied weights instead
Tom Aarsen9459040 -
Also update tokenizer/special_tokens_map
Tom Aarsen7fe32b5 -
Update tokenizer: Set lstrip=True for [MASK]
Tom AarsenThis allows mask filling with e.g. "The dog [MASK]", whereas that otherwise didn't work as most tokens start with a space in our tokenizer. The model doesn't use double spaces, so it would give odd results like "urn".b1cadbc
-
Remove now unused parameters
Tom Aarsencb6a582 -
Remove the classifier norm config arg
Tom Aarsen55af33d
-
Remove unpad_inputs for now
Tom AarsenShould depending on user's attn_implementation insteadb725c93
-
Upload ModernBERT-large model
Tom AarsenNote: this currently gives different results than the sourcedca61cc -
initial commit
Verified signature36cb71d