Instructions to use answerdotai/ModernBERT-large with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use answerdotai/ModernBERT-large with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("fill-mask", model="answerdotai/ModernBERT-large")# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForMaskedLM tokenizer = AutoTokenizer.from_pretrained("answerdotai/ModernBERT-large") model = AutoModelForMaskedLM.from_pretrained("answerdotai/ModernBERT-large", device_map="auto") - Notebooks
- Google Colab
- Kaggle
Commit History
-
Upload re.zip
Verified signaturea6deca6
-
Bump `max_position_embeddings` to 8192
Verified signaturealso harmonize `layer_norm_eps` with `norm_eps` although the former isn't usedf87846c -
Add links to answer.ai and lighton.ai
Verified signature52efb96 -
Update README.md
Verified signaturee278db6 -
Add "If your GPU supports it, "
Verified signaturea494d8e -
Upload ONNX weights (#1)
Verified signature- Upload ONNX weights (8a783c9c2ced8d98e0c89d2db641d2ed74b27ddb) Co-authored-by: Joshua <Xenova@users.noreply.huggingface.co>7235292 -
Update snippet slightly
Verified signaturee0e1696 -
Strongly suggest -> Recommend
Verified signature2706a3e -
Remove now unused unpad_no_grad
Tom Aarsenba750de -
Update README.md
Verified signaturebc1758e -
Update README.md
Verified signaturefd923f4 -
Update README.md
Verified signature8ae8825
-
Update README.md
Verified signaturec72b2bd -
Update README.md
Verified signature0480fd1 -
Create README.md
Verified signature2097f35
-
Purge duplicate "decoder.weight", rely on tied weights instead
Tom Aarsen9459040 -
Also update tokenizer/special_tokens_map
Tom Aarsen7fe32b5 -
Update tokenizer: Set lstrip=True for [MASK]
Tom AarsenThis allows mask filling with e.g. "The dog [MASK]", whereas that otherwise didn't work as most tokens start with a space in our tokenizer. The model doesn't use double spaces, so it would give odd results like "urn".b1cadbc
-
Remove now unused parameters
Tom Aarsencb6a582 -
Remove the classifier norm config arg
Tom Aarsen55af33d
-
Remove unpad_inputs for now
Tom AarsenShould depending on user's attn_implementation insteadb725c93
-
Upload ModernBERT-large model
Tom AarsenNote: this currently gives different results than the sourcedca61cc -
initial commit
Verified signature36cb71d