Instructions to use jadermcs/lexsplade-bert-large-silver with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- sentence-transformers
How to use jadermcs/lexsplade-bert-large-silver with sentence-transformers:
from sentence_transformers import SparseEncoder model = SparseEncoder("jadermcs/lexsplade-bert-large-silver") queries = ["Which planet is known as the Red Planet?"] documents = [ "Venus is often called Earth's twin because of its similar size and proximity.", "Mars, known for its reddish appearance, is often referred to as the Red Planet.", "Jupiter, the largest planet in our solar system, has a prominent red spot.", ] query_embeddings = model.encode_query(queries) document_embeddings = model.encode_document(documents) similarities = model.similarity(query_embeddings, document_embeddings) print(similarities) - Notebooks
- Google Colab
- Kaggle
LexSPLADE (bert-large-uncased, silver)
A sparse contextual word encoder. Given a sentence with one word marked, it returns a sparse vector over the 30,524-token BERT vocabulary describing that word in that context: a word embedding, not a sentence embedding. Because the dimensions are vocabulary entries, a vector can be read directly as a list of words.
Usage
from sentence_transformers import SparseEncoder
import torch
model = SparseEncoder("jadermcs/lexsplade-bert-large-silver", trust_remote_code=True)
sents = [
"He sat on the <t>bank</t> of the river and watched the water.",
"She withdrew the money from her <t>bank</t> on Monday morning.",
]
v = model.encode(sents, convert_to_tensor=True).to_dense().float()
print(torch.nn.functional.cosine_similarity(v[0:1], v[1:2], dim=1)) # ~0.19
# Read a vector as words
top = v[1].topk(8).indices
print([model.tokenizer.convert_ids_to_tokens(int(i)) for i in top])
# ['bank', 'money', 'banking', 'cash', 'security', 'financial', 'account', 'atm']
- Wrap the target in
<t>…</t>. Pooling runs over the tokens between the markers only. trust_remote_code=Trueis required: the target-span pooling, which applies the MLM head only to the marked tokens, is a custom module shipped in this repo asmodeling_lexsplade.py.- Keep the marked word inside
max_seq_length. If the markers are truncated away, the pooling falls back to the whole sentence and returns a sentence embedding.
Training
| base model | google-bert/bert-large-uncased |
| training data | 31,914 same-lemma word-in-context pairs (LLM-verified silver + MCL-WiC train) |
| objective | AnglE loss on the sparse target-word vectors, SPLADE FLOPS regularizer (1e-3), Barlow Twins term on the dense span representation |
| masked outputs | special tokens, stop words and tokens without a letter or digit |
| seed | 43 (best MCL-WiC dev AP of seeds 42–44) |
Evaluation
AP is average precision of the cosine similarity; accuracy is at the best threshold on that split.
| set | accuracy | AP |
|---|---|---|
| MCL-WiC test (en) | 91.10 | 96.42 |
| MCL-WiC dev (en) | 90.60 | 95.73 |
| crosslemma dev | 88.91 | 95.79 |
Over three seeds the same recipe gives MCL-WiC test AP 96.50 ± 0.07, dev 95.49 ± 0.22, crosslemma 95.84 ± 0.15.
Mean nonzero dimensions per usage on MCL-WiC test: 480 of 30,524.
Versions
Updated 2026-11-09 to the current recipe, which masks stop words, punctuation
and special tokens out of the output. The first upload (old recipe, seed 42,
test AP 96.30, top words could include , and the) is still available as
revision 6c03ccb27b9b7642b8223553f04be7999772f540:
SparseEncoder("jadermcs/lexsplade-bert-large-silver", trust_remote_code=True,
revision="6c03ccb27b9b7642b8223553f04be7999772f540")
Variants
jadermcs/lexsplade-bert-large-silver-ldv: output restricted to the Longman Defining Vocabulary.jadermcs/lexsplade-roberta-large-silver-wic-wnbags: RoBERTa-large.jadermcs/lexsplade-xlm-roberta-large-silver: XLM-R-large (multilingual vocabulary).jadermcs/lexsplade-xlm-roberta-xl-silver: XLM-R-XL, the strongest on English MCL-WiC.
- Downloads last month
- 338
Model tree for jadermcs/lexsplade-bert-large-silver
Base model
google-bert/bert-large-uncased