PyLate model based on answerdotai/ModernBERT-large

This is a PyLate model finetuned from answerdotai/ModernBERT-large on the train dataset. It maps sentences & paragraphs to sequences of 128-dimensional dense vectors and can be used for semantic textual similarity using the MaxSim operator.

Model Details

Model Description

Model Type: PyLate model
Base model: answerdotai/ModernBERT-large
Document Length: 180 tokens
Query Length: 32 tokens
Output Dimensionality: 128 tokens
Similarity Function: MaxSim
Training Dataset:
- train
Language: en

Model Sources

Documentation: PyLate Documentation
Repository: PyLate on GitHub
Hugging Face: PyLate models on Hugging Face

Full Model Architecture

ColBERT(
  (0): Transformer({'max_seq_length': 31, 'do_lower_case': False, 'architecture': 'ModernBertModel'})
  (1): Dense({'in_features': 1024, 'out_features': 128, 'bias': False, 'activation_function': 'torch.nn.modules.linear.Identity', 'use_residual': False})
)

Usage

First install the PyLate library:

pip install -U pylate

Retrieval

Use this model with PyLate to index and retrieve documents. The index uses FastPLAID for efficient similarity search.

Indexing documents

Load the ColBERT model and initialize the PLAID index, then encode and index your documents:

from pylate import indexes, models, retrieve

# Step 1: Load the ColBERT model
model = models.ColBERT(
    model_name_or_path="pylate_model_id",
)

# Step 2: Initialize the PLAID index
index = indexes.PLAID(
    index_folder="pylate-index",
    index_name="index",
    override=True,  # This overwrites the existing index if any
)

# Step 3: Encode the documents
documents_ids = ["1", "2", "3"]
documents = ["document 1 text", "document 2 text", "document 3 text"]

documents_embeddings = model.encode(
    documents,
    batch_size=32,
    is_query=False,  # Ensure that it is set to False to indicate that these are documents, not queries
    show_progress_bar=True,
)

# Step 4: Add document embeddings to the index by providing embeddings and corresponding ids
index.add_documents(
    documents_ids=documents_ids,
    documents_embeddings=documents_embeddings,
)

Note that you do not have to recreate the index and encode the documents every time. Once you have created an index and added the documents, you can re-use the index later by loading it:

# To load an index, simply instantiate it with the correct folder/name and without overriding it
index = indexes.PLAID(
    index_folder="pylate-index",
    index_name="index",
)

Retrieving top-k documents for queries

Once the documents are indexed, you can retrieve the top-k most relevant documents for a given set of queries. To do so, initialize the ColBERT retriever with the index you want to search in, encode the queries and then retrieve the top-k documents to get the top matches ids and relevance scores:

# Step 1: Initialize the ColBERT retriever
retriever = retrieve.ColBERT(index=index)

# Step 2: Encode the queries
queries_embeddings = model.encode(
    ["query for document 3", "query for document 1"],
    batch_size=32,
    is_query=True,  #  # Ensure that it is set to False to indicate that these are queries
    show_progress_bar=True,
)

# Step 3: Retrieve top-k documents
scores = retriever.retrieve(
    queries_embeddings=queries_embeddings,
    k=10,  # Retrieve the top 10 matches for each query
)

Reranking

If you only want to use the ColBERT model to perform reranking on top of your first-stage retrieval pipeline without building an index, you can simply use rank function and pass the queries and documents to rerank:

from pylate import rank, models

queries = [
    "query A",
    "query B",
]

documents = [
    ["document A", "document B"],
    ["document 1", "document C", "document B"],
]

documents_ids = [
    [1, 2],
    [1, 3, 2],
]

model = models.ColBERT(
    model_name_or_path="pylate_model_id",
)

queries_embeddings = model.encode(
    queries,
    is_query=True,
)

documents_embeddings = model.encode(
    documents,
    is_query=False,
)

reranked_documents = rank.rerank(
    documents_ids=documents_ids,
    queries_embeddings=queries_embeddings,
    documents_embeddings=documents_embeddings,
)

Training Details

Training Dataset

train

Dataset: train at ad24729
Size: 808,728 training samples
Columns: query_id, document_ids, scores, query, and documents

Approximate statistics based on the first 1000 samples:

	query_id	document_ids	scores	query	documents
type	string	list	list	string	list
details	min: 5 tokens mean: 5.59 tokens max: 6 tokens	size: 32 elements	size: 32 elements	min: 5 tokens mean: 10.51 tokens max: 30 tokens	size: 32 elements

Samples:

query_id	document_ids	scores	query	documents
`121352`	`['2259784', '4923159', '40211', '1545154', '8527175', ...]`	`[0.2343463897705078, 0.639204204082489, 0.3806908428668976, 0.5623092651367188, 0.8051995635032654, ...]`	`define extreme`	['extremist. 1 AN EXTREMIST IS SOMEONE WHO SUPPORTS AN IDEA, CAUSE, OR SET OF VALUES SO ADAMANTLY AND WITHOUT COMPROMISE THAT SAID PERSON WILL USE THEIR IDEAS TO JUSTIFY ANYTHING THEY DO.', "at the extreme end meaning, at the extreme end definition \| English Cobuild dictionary. extreme. 1 adj Extreme means very great in degree or intensity. The girls were afraid of snakes and picked their way along with extreme caution., ...people living in extreme poverty., ...the author's extreme reluctance to generalise.", 'extremity (plural extremities) 1 The most extreme or furthest point of something. 2 An extreme measure. 3 A hand or foot. A limb (major appendage of human or animal such as a leg an arm or a wing)', ': extreme in a way that is not normal or that shows an illness or mental problem. medical: relating to or caused by disease.: of or relating to the study of diseases: relating to pathology. extreme in a way that is not normal or that shows an illness or mental problem. medical: ...
`634306`	`['7723525', '1874779', '379307', '2738583', '7599583', ...]`	`[0.7124203443527222, 0.7379189729690552, 0.5786551237106323, 0.6142299175262451, 0.6755089163780212, ...]`	`what does chattel mean on credit history`	["Duhaime's Law Dictionary. Chattel Mortgage Definition: Related Terms: Chattel, Mortgage. When a lien is given on goods, chattels, moveable or personal property (other than real property in which case it is referred to as just a mortgage), in writing, to guarantee the payment of a debt or the execution of some action.", 'From Wikipedia, the free encyclopedia. Chattel mortgage, sometimes abbreviated CM, is the legal term for a type of loan contract used in some states with legal systems derived from English law. Under a typical chattel mortgage, the purchaser borrows funds for the purchase of movable personal property (the chattel) from the lender. The lender then secures the loan with a mortgage over the chattel.', 'Chattel Mortgages In Detail. A Chattel Mortgage uses your vehicle or some other (non-real estate) property as the security on the loan meaning you can access a low interest rate.ncidentally, these loans can be used for other purposes such as business equipment. If you have...
`920825`	`['5976297', '2866112', '3560294', '3285659', '4706740', ...]`	`[0.6462352871894836, 0.7880821228027344, 0.791019856929779, 0.7709633111953735, 0.8284491300582886, ...]`	`what was the great leap forward brainly`	['It was a clever scheme that was hatched soon after the 1949 revolution. The first phase was to send spies to the west during the great leap forward in the 1950s to plant falsified basic science into our western understanding of physics.', 'Great Leap Forward Devolution Into the Great Famine . Yang Jisheng, the author of Tombstone , wrote in the New York Times, â\x80\x9cThe Great Leap Forward that Mao began in 1958 set ambitious goals without the means to meet them. A vicious cycle ensued; exaggerated production reports from below emboldened the higher-ups to set even loftier targets.', 'In 1958 Mao introduced a second five year plan which became known as the â\x80\x98Great Leap Forwardâ\x80\x99 (GLF). He believed it was possible for China to overtake Britain as a leading industrial power within seven years and the USA soon after.n 1958 Mao introduced a second five year plan which became known as the â\x80\x98Great Leap Forwardâ\x80\x99 (GLF). He believed it was possible for China to ...

Loss: pylate.losses.distillation.Distillation

Training Hyperparameters

Non-Default Hyperparameters

per_device_train_batch_size: 4
learning_rate: 3e-05
max_grad_norm: 2.0
num_train_epochs: 1
warmup_ratio: 0.05
bf16: True
dataloader_num_workers: 4
ddp_find_unused_parameters: False

All Hyperparameters

Click to expand

overwrite_output_dir: False
do_predict: False
eval_strategy: no
prediction_loss_only: True
per_device_train_batch_size: 4
per_device_eval_batch_size: 8
per_gpu_train_batch_size: None
per_gpu_eval_batch_size: None
gradient_accumulation_steps: 1
eval_accumulation_steps: None
torch_empty_cache_steps: None
learning_rate: 3e-05
weight_decay: 0.0
adam_beta1: 0.9
adam_beta2: 0.999
adam_epsilon: 1e-08
max_grad_norm: 2.0
num_train_epochs: 1
max_steps: -1
lr_scheduler_type: linear
lr_scheduler_kwargs: {}
warmup_ratio: 0.05
warmup_steps: 0
log_level: passive
log_level_replica: warning
log_on_each_node: True
logging_nan_inf_filter: True
save_safetensors: True
save_on_each_node: False
save_only_model: False
restore_callback_states_from_checkpoint: False
no_cuda: False
use_cpu: False
use_mps_device: False
seed: 42
data_seed: None
jit_mode_eval: False
use_ipex: False
bf16: True
fp16: False
fp16_opt_level: O1
half_precision_backend: auto
bf16_full_eval: False
fp16_full_eval: False
tf32: None
local_rank: 0
ddp_backend: None
tpu_num_cores: None
tpu_metrics_debug: False
debug: []
dataloader_drop_last: True
dataloader_num_workers: 4
dataloader_prefetch_factor: None
past_index: -1
disable_tqdm: False
remove_unused_columns: True
label_names: None
load_best_model_at_end: False
ignore_data_skip: False
fsdp: []
fsdp_min_num_params: 0
fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
fsdp_transformer_layer_cls_to_wrap: None
accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
deepspeed: None
label_smoothing_factor: 0.0
optim: adamw_torch
optim_args: None
adafactor: False
group_by_length: False
length_column_name: length
ddp_find_unused_parameters: False
ddp_bucket_cap_mb: None
ddp_broadcast_buffers: False
dataloader_pin_memory: True
dataloader_persistent_workers: False
skip_memory_metrics: True
use_legacy_prediction_loop: False
push_to_hub: False
resume_from_checkpoint: None
hub_model_id: None
hub_strategy: every_save
hub_private_repo: None
hub_always_push: False
gradient_checkpointing: False
gradient_checkpointing_kwargs: None
include_inputs_for_metrics: False
include_for_metrics: []
eval_do_concat_batches: True
fp16_backend: auto
push_to_hub_model_id: None
push_to_hub_organization: None
mp_parameters:
auto_find_batch_size: False
full_determinism: False
torchdynamo: None
ray_scope: last
ddp_timeout: 1800
torch_compile: False
torch_compile_backend: None
torch_compile_mode: None
dispatch_batches: None
split_batches: None
include_tokens_per_second: False
include_num_input_tokens_seen: False
neftune_noise_alpha: None
optim_target_modules: None
batch_eval_metrics: False
eval_on_start: False
use_liger_kernel: False
eval_use_gather_object: False
average_tokens_across_devices: False
prompts: None
batch_sampler: batch_sampler
multi_dataset_batch_sampler: proportional
router_mapping: {}
learning_rate_mapping: {}

Training Logs

Click to expand

Epoch	Step	Training Loss
0.0010	50	0.0584
0.0020	100	0.0496
0.0030	150	0.0494
0.0040	200	0.0484
0.0049	250	0.048
0.0059	300	0.0478
0.0069	350	0.0487
0.0079	400	0.0481
0.0089	450	0.0474
0.0099	500	0.0476
0.0109	550	0.0475
0.0119	600	0.0478
0.0129	650	0.0463
0.0138	700	0.0476
0.0148	750	0.0479
0.0158	800	0.0475
0.0168	850	0.0469
0.0178	900	0.0461
0.0188	950	0.0475
0.0198	1000	0.0473
0.0208	1050	0.047
0.0218	1100	0.047
0.0228	1150	0.0465
0.0237	1200	0.0454
0.0247	1250	0.0466
0.0257	1300	0.0469
0.0267	1350	0.0462
0.0277	1400	0.0464
0.0287	1450	0.046
0.0297	1500	0.0501
0.0307	1550	0.0469
0.0317	1600	0.0473
0.0326	1650	0.0456
0.0336	1700	0.0465
0.0346	1750	0.0462
0.0356	1800	0.0471
0.0366	1850	0.0458
0.0376	1900	0.0462
0.0386	1950	0.0465
0.0396	2000	0.0456
0.0406	2050	0.0471
0.0415	2100	0.0482
0.0425	2150	0.0454
0.0435	2200	0.0456
0.0445	2250	0.0446
0.0455	2300	0.0378
0.0465	2350	0.0349
0.0475	2400	0.0318
0.0485	2450	0.0321
0.0495	2500	0.0315
0.0505	2550	0.0302
0.0514	2600	0.0304
0.0524	2650	0.0291
0.0534	2700	0.0294
0.0544	2750	0.0287
0.0554	2800	0.0278
0.0564	2850	0.0281
0.0574	2900	0.0285
0.0584	2950	0.0279
0.0594	3000	0.0264
0.0603	3050	0.0268
0.0613	3100	0.0265
0.0623	3150	0.0266
0.0633	3200	0.0265
0.0643	3250	0.0272
0.0653	3300	0.0267
0.0663	3350	0.0269
0.0673	3400	0.0265
0.0683	3450	0.0263
0.0692	3500	0.0253
0.0702	3550	0.0257
0.0712	3600	0.0258
0.0722	3650	0.0253
0.0732	3700	0.0256
0.0742	3750	0.0255
0.0752	3800	0.0248
0.0762	3850	0.0247
0.0772	3900	0.026
0.0781	3950	0.0248
0.0791	4000	0.0251
0.0801	4050	0.0242
0.0811	4100	0.0251
0.0821	4150	0.0246
0.0831	4200	0.0242
0.0841	4250	0.0241
0.0851	4300	0.0243
0.0861	4350	0.0244
0.0871	4400	0.0251
0.0880	4450	0.0246
0.0890	4500	0.0232
0.0900	4550	0.024
0.0910	4600	0.0235
0.0920	4650	0.0245
0.0930	4700	0.0228
0.0940	4750	0.0239
0.0950	4800	0.0241
0.0960	4850	0.0239
0.0969	4900	0.0228
0.0979	4950	0.0233
0.0989	5000	0.0233
0.0999	5050	0.0236
0.1009	5100	0.024
0.1019	5150	0.0232
0.1029	5200	0.0237
0.1039	5250	0.0229
0.1049	5300	0.0236
0.1058	5350	0.0234
0.1068	5400	0.0224
0.1078	5450	0.0227
0.1088	5500	0.024
0.1098	5550	0.0233
0.1108	5600	0.023
0.1118	5650	0.0221
0.1128	5700	0.0227
0.1138	5750	0.0235
0.1147	5800	0.0228
0.1157	5850	0.0228
0.1167	5900	0.0231
0.1177	5950	0.022
0.1187	6000	0.0225
0.1197	6050	0.0214
0.1207	6100	0.0218
0.1217	6150	0.0221
0.1227	6200	0.0222
0.1237	6250	0.0221
0.1246	6300	0.0214
0.1256	6350	0.0225
0.1266	6400	0.0219
0.1276	6450	0.0218
0.1286	6500	0.0223
0.1296	6550	0.0216
0.1306	6600	0.0206
0.1316	6650	0.0215
0.1326	6700	0.0223
0.1335	6750	0.0222
0.1345	6800	0.0212
0.1355	6850	0.0213
0.1365	6900	0.0212
0.1375	6950	0.0206
0.1385	7000	0.0204
0.1395	7050	0.0212
0.1405	7100	0.0214
0.1415	7150	0.0218
0.1424	7200	0.0212
0.1434	7250	0.0199
0.1444	7300	0.0215
0.1454	7350	0.0215
0.1464	7400	0.0212
0.1474	7450	0.0211
0.1484	7500	0.0208
0.1494	7550	0.0211
0.1504	7600	0.0219
0.1514	7650	0.0207
0.1523	7700	0.0204
0.1533	7750	0.0209
0.1543	7800	0.0209
0.1553	7850	0.0206
0.1563	7900	0.0206
0.1573	7950	0.0204
0.1583	8000	0.0206
0.1593	8050	0.0199
0.1603	8100	0.0197
0.1612	8150	0.0206
0.1622	8200	0.0201
0.1632	8250	0.0191
0.1642	8300	0.0201
0.1652	8350	0.0211
0.1662	8400	0.02
0.1672	8450	0.0201
0.1682	8500	0.0194
0.1692	8550	0.0205
0.1701	8600	0.0199
0.1711	8650	0.0202
0.1721	8700	0.02
0.1731	8750	0.0205
0.1741	8800	0.0193
0.1751	8850	0.02
0.1761	8900	0.0191
0.1771	8950	0.0195
0.1781	9000	0.0201
0.1790	9050	0.0192
0.1800	9100	0.0203
0.1810	9150	0.0191
0.1820	9200	0.0207
0.1830	9250	0.0205
0.1840	9300	0.0192
0.1850	9350	0.0193
0.1860	9400	0.0193
0.1870	9450	0.0189
0.1880	9500	0.0196
0.1889	9550	0.019
0.1899	9600	0.0199
0.1909	9650	0.0191
0.1919	9700	0.0189
0.1929	9750	0.0184
0.1939	9800	0.0185
0.1949	9850	0.0181
0.1959	9900	0.0183
0.1969	9950	0.0187
0.1978	10000	0.0187
0.1988	10050	0.0188
0.1998	10100	0.0181
0.2008	10150	0.0183
0.2018	10200	0.0179
0.2028	10250	0.019
0.2038	10300	0.0186
0.2048	10350	0.0187
0.2058	10400	0.0183
0.2067	10450	0.0181
0.2077	10500	0.0184
0.2087	10550	0.018
0.2097	10600	0.0188
0.2107	10650	0.0185
0.2117	10700	0.0185
0.2127	10750	0.0176
0.2137	10800	0.0176
0.2147	10850	0.018
0.2156	10900	0.0181
0.2166	10950	0.0177
0.2176	11000	0.0181
0.2186	11050	0.018
0.2196	11100	0.0187
0.2206	11150	0.0173
0.2216	11200	0.0171
0.2226	11250	0.0184
0.2236	11300	0.0173
0.2246	11350	0.0178
0.2255	11400	0.0173
0.2265	11450	0.0177
0.2275	11500	0.0171
0.2285	11550	0.0171
0.2295	11600	0.0175
0.2305	11650	0.0169
0.2315	11700	0.0164
0.2325	11750	0.0163
0.2335	11800	0.0177
0.2344	11850	0.017
0.2354	11900	0.0164
0.2364	11950	0.017
0.2374	12000	0.0176
0.2384	12050	0.0167
0.2394	12100	0.0163
0.2404	12150	0.0168
0.2414	12200	0.0161
0.2424	12250	0.0169
0.2433	12300	0.0171
0.2443	12350	0.0163
0.2453	12400	0.0167
0.2463	12450	0.0162
0.2473	12500	0.0156
0.2483	12550	0.0159
0.2493	12600	0.0161
0.2503	12650	0.0161
0.2513	12700	0.0163
0.2523	12750	0.0156
0.2532	12800	0.0158
0.2542	12850	0.0157
0.2552	12900	0.0159
0.2562	12950	0.0159
0.2572	13000	0.0159
0.2582	13050	0.0155
0.2592	13100	0.0152
0.2602	13150	0.0151
0.2612	13200	0.0151
0.2621	13250	0.0143
0.2631	13300	0.0151
0.2641	13350	0.0151
0.2651	13400	0.0152
0.2661	13450	0.0147
0.2671	13500	0.0147
0.2681	13550	0.015
0.2691	13600	0.015
0.2701	13650	0.0145
0.2710	13700	0.0156
0.2720	13750	0.0151
0.2730	13800	0.0143
0.2740	13850	0.0142
0.2750	13900	0.0147
0.2760	13950	0.0145
0.2770	14000	0.0141
0.2780	14050	0.0138
0.2790	14100	0.0145
0.2799	14150	0.0144
0.2809	14200	0.0142
0.2819	14250	0.0139
0.2829	14300	0.0143
0.2839	14350	0.0139
0.2849	14400	0.0136
0.2859	14450	0.015
0.2869	14500	0.0133
0.2879	14550	0.0142
0.2889	14600	0.0137
0.2898	14650	0.0139
0.2908	14700	0.014
0.2918	14750	0.0134
0.2928	14800	0.0138
0.2938	14850	0.0134
0.2948	14900	0.014
0.2958	14950	0.0133
0.2968	15000	0.0138
0.2978	15050	0.0132
0.2987	15100	0.0131
0.2997	15150	0.0134
0.3007	15200	0.0142
0.3017	15250	0.0128
0.3027	15300	0.0134
0.3037	15350	0.013
0.3047	15400	0.0132
0.3057	15450	0.0134
0.3067	15500	0.0128
0.3076	15550	0.0136
0.3086	15600	0.0134
0.3096	15650	0.013
0.3106	15700	0.0128
0.3116	15750	0.0133
0.3126	15800	0.0132
0.3136	15850	0.0128
0.3146	15900	0.0124
0.3156	15950	0.0131
0.3165	16000	0.0128
0.3175	16050	0.0125
0.3185	16100	0.0126
0.3195	16150	0.0127
0.3205	16200	0.0124
0.3215	16250	0.0131
0.3225	16300	0.0122
0.3235	16350	0.0129
0.3245	16400	0.0126
0.3255	16450	0.0125
0.3264	16500	0.0122
0.3274	16550	0.0123
0.3284	16600	0.0123
0.3294	16650	0.0126
0.3304	16700	0.0123
0.3314	16750	0.0135
0.3324	16800	0.0125
0.3334	16850	0.0121
0.3344	16900	0.012
0.3353	16950	0.0125
0.3363	17000	0.0121
0.3373	17050	0.0118
0.3383	17100	0.0123
0.3393	17150	0.0115
0.3403	17200	0.0126
0.3413	17250	0.0115
0.3423	17300	0.0115
0.3433	17350	0.0119
0.3442	17400	0.0121
0.3452	17450	0.0116
0.3462	17500	0.0119
0.3472	17550	0.0116
0.3482	17600	0.0116
0.3492	17650	0.0119
0.3502	17700	0.0118
0.3512	17750	0.0119
0.3522	17800	0.0116
0.3532	17850	0.0119
0.3541	17900	0.0118
0.3551	17950	0.0117
0.3561	18000	0.0113
0.3571	18050	0.0118
0.3581	18100	0.0117
0.3591	18150	0.0114
0.3601	18200	0.0118
0.3611	18250	0.0113
0.3621	18300	0.0113
0.3630	18350	0.0116
0.3640	18400	0.0124
0.3650	18450	0.0115
0.3660	18500	0.0115
0.3670	18550	0.011
0.3680	18600	0.0115
0.3690	18650	0.0115
0.3700	18700	0.011
0.3710	18750	0.0112
0.3719	18800	0.0113
0.3729	18850	0.0113
0.3739	18900	0.0115
0.3749	18950	0.0109
0.3759	19000	0.0112
0.3769	19050	0.0112
0.3779	19100	0.011
0.3789	19150	0.0116
0.3799	19200	0.0115
0.3808	19250	0.0114
0.3818	19300	0.011
0.3828	19350	0.0112
0.3838	19400	0.011
0.3848	19450	0.0107
0.3858	19500	0.0113
0.3868	19550	0.0115
0.3878	19600	0.0113
0.3888	19650	0.0112
0.3898	19700	0.0114
0.3907	19750	0.0111
0.3917	19800	0.0107
0.3927	19850	0.0108
0.3937	19900	0.0112
0.3947	19950	0.011
0.3957	20000	0.0111
0.3967	20050	0.0108
0.3977	20100	0.0107
0.3987	20150	0.0104
0.3996	20200	0.0104
0.4006	20250	0.0104
0.4016	20300	0.0109
0.4026	20350	0.011
0.4036	20400	0.0109
0.4046	20450	0.0107
0.4056	20500	0.011
0.4066	20550	0.0108
0.4076	20600	0.0109
0.4085	20650	0.0105
0.4095	20700	0.0107
0.4105	20750	0.0106
0.4115	20800	0.0106
0.4125	20850	0.0106
0.4135	20900	0.011
0.4145	20950	0.0108
0.4155	21000	0.0102
0.4165	21050	0.0106
0.4174	21100	0.0103
0.4184	21150	0.0108
0.4194	21200	0.0098
0.4204	21250	0.0106
0.4214	21300	0.0101
0.4224	21350	0.0111
0.4234	21400	0.0103
0.4244	21450	0.0104
0.4254	21500	0.0103
0.4264	21550	0.0103
0.4273	21600	0.0099
0.4283	21650	0.01
0.4293	21700	0.0099
0.4303	21750	0.01
0.4313	21800	0.0104
0.4323	21850	0.0103
0.4333	21900	0.0105
0.4343	21950	0.0103
0.4353	22000	0.01
0.4362	22050	0.0102
0.4372	22100	0.0103
0.4382	22150	0.0102
0.4392	22200	0.01
0.4402	22250	0.0106
0.4412	22300	0.0101
0.4422	22350	0.01
0.4432	22400	0.0098
0.4442	22450	0.0106
0.4451	22500	0.01
0.4461	22550	0.0103
0.4471	22600	0.0101
0.4481	22650	0.0103
0.4491	22700	0.0099
0.4501	22750	0.0105
0.4511	22800	0.0096
0.4521	22850	0.0099
0.4531	22900	0.01
0.4541	22950	0.01
0.4550	23000	0.0101
0.4560	23050	0.0095
0.4570	23100	0.0101
0.4580	23150	0.0099
0.4590	23200	0.01
0.4600	23250	0.0097
0.4610	23300	0.0097
0.4620	23350	0.0098
0.4630	23400	0.0097
0.4639	23450	0.0096
0.4649	23500	0.0099
0.4659	23550	0.0097
0.4669	23600	0.0095
0.4679	23650	0.0098
0.4689	23700	0.0096
0.4699	23750	0.0098
0.4709	23800	0.0094
0.4719	23850	0.0093
0.4728	23900	0.0094
0.4738	23950	0.0092
0.4748	24000	0.0096
0.4758	24050	0.0095
0.4768	24100	0.01
0.4778	24150	0.01
0.4788	24200	0.0103
0.4798	24250	0.0094
0.4808	24300	0.0097
0.4817	24350	0.0096
0.4827	24400	0.0096
0.4837	24450	0.0099
0.4847	24500	0.0099
0.4857	24550	0.0092
0.4867	24600	0.0092
0.4877	24650	0.0097
0.4887	24700	0.0096
0.4897	24750	0.0096
0.4907	24800	0.01
0.4916	24850	0.0096
0.4926	24900	0.0098
0.4936	24950	0.0094
0.4946	25000	0.0096
0.4956	25050	0.0099
0.4966	25100	0.0094
0.4976	25150	0.0094
0.4986	25200	0.0103
0.4996	25250	0.0098
0.5005	25300	0.0096
0.5015	25350	0.0099
0.5025	25400	0.0092
0.5035	25450	0.0099
0.5045	25500	0.0092
0.5055	25550	0.0094
0.5065	25600	0.0094
0.5075	25650	0.0093
0.5085	25700	0.0093
0.5094	25750	0.0095
0.5104	25800	0.0086
0.5114	25850	0.009
0.5124	25900	0.0094
0.5134	25950	0.0091
0.5144	26000	0.0091
0.5154	26050	0.0089
0.5164	26100	0.0095
0.5174	26150	0.0092
0.5183	26200	0.0091
0.5193	26250	0.0092
0.5203	26300	0.009
0.5213	26350	0.0091
0.5223	26400	0.0089
0.5233	26450	0.0092
0.5243	26500	0.0088
0.5253	26550	0.0093
0.5263	26600	0.0093
0.5273	26650	0.0097
0.5282	26700	0.0092
0.5292	26750	0.0091
0.5302	26800	0.0091
0.5312	26850	0.0088
0.5322	26900	0.0089
0.5332	26950	0.0093
0.5342	27000	0.009
0.5352	27050	0.0091
0.5362	27100	0.0092
0.5371	27150	0.0093
0.5381	27200	0.0092
0.5391	27250	0.0091
0.5401	27300	0.0087
0.5411	27350	0.0091
0.5421	27400	0.009
0.5431	27450	0.0088
0.5441	27500	0.0089
0.5451	27550	0.0091
0.5460	27600	0.0091
0.5470	27650	0.0088
0.5480	27700	0.0088
0.5490	27750	0.0092
0.5500	27800	0.0094
0.5510	27850	0.0087
0.5520	27900	0.0092
0.5530	27950	0.0088
0.5540	28000	0.0093
0.5550	28050	0.0089
0.5559	28100	0.009
0.5569	28150	0.0088
0.5579	28200	0.0082
0.5589	28250	0.0087
0.5599	28300	0.0086
0.5609	28350	0.0094
0.5619	28400	0.0089
0.5629	28450	0.0089
0.5639	28500	0.009
0.5648	28550	0.0086
0.5658	28600	0.0089
0.5668	28650	0.0091
0.5678	28700	0.0092
0.5688	28750	0.0089
0.5698	28800	0.0085
0.5708	28850	0.0088
0.5718	28900	0.0093
0.5728	28950	0.0086
0.5737	29000	0.0093
0.5747	29050	0.0089
0.5757	29100	0.0087
0.5767	29150	0.0086
0.5777	29200	0.0084
0.5787	29250	0.0086
0.5797	29300	0.0085
0.5807	29350	0.0084
0.5817	29400	0.0086
0.5826	29450	0.0086
0.5836	29500	0.0089
0.5846	29550	0.0085
0.5856	29600	0.0086
0.5866	29650	0.0087
0.5876	29700	0.0085
0.5886	29750	0.0091
0.5896	29800	0.0084
0.5906	29850	0.0086
0.5916	29900	0.0087
0.5925	29950	0.0088
0.5935	30000	0.0086
0.5945	30050	0.009
0.5955	30100	0.0085
0.5965	30150	0.0084
0.5975	30200	0.0086
0.5985	30250	0.0083
0.5995	30300	0.0085
0.6005	30350	0.0087
0.6014	30400	0.0086
0.6024	30450	0.0088
0.6034	30500	0.0084
0.6044	30550	0.0086
0.6054	30600	0.0088
0.6064	30650	0.0085
0.6074	30700	0.0084
0.6084	30750	0.0085
0.6094	30800	0.0084
0.6103	30850	0.0084
0.6113	30900	0.0087
0.6123	30950	0.0085
0.6133	31000	0.0087
0.6143	31050	0.0087
0.6153	31100	0.0084
0.6163	31150	0.0085
0.6173	31200	0.0085
0.6183	31250	0.0085
0.6193	31300	0.008
0.6202	31350	0.0083
0.6212	31400	0.0085
0.6222	31450	0.0084
0.6232	31500	0.0081
0.6242	31550	0.0087
0.6252	31600	0.0081
0.6262	31650	0.0081
0.6272	31700	0.0085
0.6282	31750	0.0083
0.6291	31800	0.0086
0.6301	31850	0.0089
0.6311	31900	0.0082
0.6321	31950	0.0084
0.6331	32000	0.008
0.6341	32050	0.0084
0.6351	32100	0.0084
0.6361	32150	0.0082
0.6371	32200	0.0081
0.6380	32250	0.0084
0.6390	32300	0.008
0.6400	32350	0.008
0.6410	32400	0.0083
0.6420	32450	0.0083
0.6430	32500	0.0082
0.6440	32550	0.008
0.6450	32600	0.0085
0.6460	32650	0.0083
0.6469	32700	0.0084
0.6479	32750	0.0083
0.6489	32800	0.0078
0.6499	32850	0.0085
0.6509	32900	0.0079
0.6519	32950	0.008
0.6529	33000	0.008
0.6539	33050	0.0082
0.6549	33100	0.0077
0.6559	33150	0.0082
0.6568	33200	0.0086
0.6578	33250	0.008
0.6588	33300	0.0083
0.6598	33350	0.0083
0.6608	33400	0.008
0.6618	33450	0.008
0.6628	33500	0.008
0.6638	33550	0.0078
0.6648	33600	0.0082
0.6657	33650	0.0082
0.6667	33700	0.0082
0.6677	33750	0.0084
0.6687	33800	0.0083
0.6697	33850	0.0078
0.6707	33900	0.0078
0.6717	33950	0.0079
0.6727	34000	0.0078
0.6737	34050	0.008
0.6746	34100	0.008
0.6756	34150	0.0073
0.6766	34200	0.0079
0.6776	34250	0.0083
0.6786	34300	0.0077
0.6796	34350	0.0082
0.6806	34400	0.008
0.6816	34450	0.0083
0.6826	34500	0.0078
0.6835	34550	0.0077
0.6845	34600	0.0081
0.6855	34650	0.0079
0.6865	34700	0.0077
0.6875	34750	0.0077
0.6885	34800	0.0076
0.6895	34850	0.0078
0.6905	34900	0.0076
0.6915	34950	0.008
0.6925	35000	0.008
0.6934	35050	0.0077
0.6944	35100	0.0079
0.6954	35150	0.0076
0.6964	35200	0.0077
0.6974	35250	0.0082
0.6984	35300	0.008
0.6994	35350	0.0079
0.7004	35400	0.0077
0.7014	35450	0.0076
0.7023	35500	0.0082
0.7033	35550	0.0079
0.7043	35600	0.0077
0.7053	35650	0.0079
0.7063	35700	0.008
0.7073	35750	0.0082
0.7083	35800	0.0077
0.7093	35850	0.0076
0.7103	35900	0.0082
0.7112	35950	0.0082
0.7122	36000	0.0078
0.7132	36050	0.0081
0.7142	36100	0.0081
0.7152	36150	0.0078
0.7162	36200	0.0079
0.7172	36250	0.0076
0.7182	36300	0.0078
0.7192	36350	0.0076
0.7202	36400	0.0078
0.7211	36450	0.0078
0.7221	36500	0.0079
0.7231	36550	0.0075
0.7241	36600	0.008
0.7251	36650	0.0075
0.7261	36700	0.0078
0.7271	36750	0.0081
0.7281	36800	0.0081
0.7291	36850	0.0078
0.7300	36900	0.0074
0.7310	36950	0.008
0.7320	37000	0.0078
0.7330	37050	0.0076
0.7340	37100	0.0073
0.7350	37150	0.0079
0.7360	37200	0.0072
0.7370	37250	0.008
0.7380	37300	0.0077
0.7389	37350	0.0077
0.7399	37400	0.0079
0.7409	37450	0.0075
0.7419	37500	0.0082
0.7429	37550	0.0077
0.7439	37600	0.0077
0.7449	37650	0.0075
0.7459	37700	0.008
0.7469	37750	0.0079
0.7478	37800	0.0078
0.7488	37850	0.0076
0.7498	37900	0.0079
0.7508	37950	0.0073
0.7518	38000	0.0075
0.7528	38050	0.0074
0.7538	38100	0.0081
0.7548	38150	0.0078
0.7558	38200	0.0073
0.7568	38250	0.0081
0.7577	38300	0.0074
0.7587	38350	0.0074
0.7597	38400	0.0074
0.7607	38450	0.0074
0.7617	38500	0.0077
0.7627	38550	0.0073
0.7637	38600	0.0079
0.7647	38650	0.0074
0.7657	38700	0.0074
0.7666	38750	0.0075
0.7676	38800	0.0076
0.7686	38850	0.0076
0.7696	38900	0.0075
0.7706	38950	0.0072
0.7716	39000	0.0074
0.7726	39050	0.0076
0.7736	39100	0.0074
0.7746	39150	0.0074
0.7755	39200	0.0077
0.7765	39250	0.008
0.7775	39300	0.0077
0.7785	39350	0.0075
0.7795	39400	0.0076
0.7805	39450	0.0075
0.7815	39500	0.0073
0.7825	39550	0.0075
0.7835	39600	0.0076
0.7844	39650	0.0076
0.7854	39700	0.0074
0.7864	39750	0.0075
0.7874	39800	0.0074
0.7884	39850	0.0075
0.7894	39900	0.0079
0.7904	39950	0.0072
0.7914	40000	0.0074
0.7924	40050	0.0076
0.7934	40100	0.0077
0.7943	40150	0.0078
0.7953	40200	0.0077
0.7963	40250	0.0069
0.7973	40300	0.0076
0.7983	40350	0.0076
0.7993	40400	0.0078
0.8003	40450	0.0075
0.8013	40500	0.0072
0.8023	40550	0.0073
0.8032	40600	0.0071
0.8042	40650	0.0073
0.8052	40700	0.0073
0.8062	40750	0.0073
0.8072	40800	0.0074
0.8082	40850	0.007
0.8092	40900	0.0075
0.8102	40950	0.0075
0.8112	41000	0.0073
0.8121	41050	0.0078
0.8131	41100	0.0074
0.8141	41150	0.0074
0.8151	41200	0.0072
0.8161	41250	0.0076
0.8171	41300	0.0073
0.8181	41350	0.0075
0.8191	41400	0.0076
0.8201	41450	0.0074
0.8211	41500	0.0076
0.8220	41550	0.0073
0.8230	41600	0.0074
0.8240	41650	0.0072
0.8250	41700	0.0074
0.8260	41750	0.0074
0.8270	41800	0.0074
0.8280	41850	0.0075
0.8290	41900	0.0074
0.8300	41950	0.0075
0.8309	42000	0.0076
0.8319	42050	0.0078
0.8329	42100	0.0074
0.8339	42150	0.0077
0.8349	42200	0.0073
0.8359	42250	0.0076
0.8369	42300	0.0077
0.8379	42350	0.0077
0.8389	42400	0.0071
0.8398	42450	0.0076
0.8408	42500	0.0071
0.8418	42550	0.0075
0.8428	42600	0.0073
0.8438	42650	0.0073
0.8448	42700	0.0073
0.8458	42750	0.0068
0.8468	42800	0.0072
0.8478	42850	0.007
0.8487	42900	0.0073
0.8497	42950	0.0074
0.8507	43000	0.0073
0.8517	43050	0.0075
0.8527	43100	0.0073
0.8537	43150	0.0071
0.8547	43200	0.0069
0.8557	43250	0.0076
0.8567	43300	0.0076
0.8577	43350	0.0075
0.8586	43400	0.0076
0.8596	43450	0.0072
0.8606	43500	0.0072
0.8616	43550	0.0073
0.8626	43600	0.007
0.8636	43650	0.007
0.8646	43700	0.007
0.8656	43750	0.0075
0.8666	43800	0.0073
0.8675	43850	0.0071
0.8685	43900	0.0076
0.8695	43950	0.0073
0.8705	44000	0.0072
0.8715	44050	0.0076
0.8725	44100	0.0071
0.8735	44150	0.0072
0.8745	44200	0.0073
0.8755	44250	0.0071
0.8764	44300	0.0071
0.8774	44350	0.007
0.8784	44400	0.0074
0.8794	44450	0.0073
0.8804	44500	0.0071
0.8814	44550	0.0072
0.8824	44600	0.0069
0.8834	44650	0.0072
0.8844	44700	0.0073
0.8853	44750	0.0073
0.8863	44800	0.007
0.8873	44850	0.0073
0.8883	44900	0.0071
0.8893	44950	0.0069
0.8903	45000	0.0069
0.8913	45050	0.0072
0.8923	45100	0.0076
0.8933	45150	0.0073
0.8943	45200	0.0075
0.8952	45250	0.0074
0.8962	45300	0.007
0.8972	45350	0.0071
0.8982	45400	0.0069
0.8992	45450	0.007
0.9002	45500	0.0071
0.9012	45550	0.0072
0.9022	45600	0.0074
0.9032	45650	0.0071
0.9041	45700	0.0073
0.9051	45750	0.007
0.9061	45800	0.0069
0.9071	45850	0.0067
0.9081	45900	0.007
0.9091	45950	0.0071
0.9101	46000	0.0073
0.9111	46050	0.0073
0.9121	46100	0.0071
0.9130	46150	0.0072
0.9140	46200	0.007
0.9150	46250	0.0073
0.9160	46300	0.0072
0.9170	46350	0.0072
0.9180	46400	0.0074
0.9190	46450	0.0071
0.9200	46500	0.007
0.9210	46550	0.0073
0.9220	46600	0.0069
0.9229	46650	0.0071
0.9239	46700	0.0073
0.9249	46750	0.0069
0.9259	46800	0.0072
0.9269	46850	0.0068
0.9279	46900	0.0072
0.9289	46950	0.0069
0.9299	47000	0.0068
0.9309	47050	0.007
0.9318	47100	0.007
0.9328	47150	0.0071
0.9338	47200	0.0071
0.9348	47250	0.0071
0.9358	47300	0.007
0.9368	47350	0.0072
0.9378	47400	0.0073
0.9388	47450	0.0071
0.9398	47500	0.007
0.9407	47550	0.0072
0.9417	47600	0.0072
0.9427	47650	0.0068
0.9437	47700	0.007
0.9447	47750	0.0072
0.9457	47800	0.007
0.9467	47850	0.0069
0.9477	47900	0.0069
0.9487	47950	0.007
0.9496	48000	0.0069
0.9506	48050	0.0069
0.9516	48100	0.0071
0.9526	48150	0.0072
0.9536	48200	0.0071
0.9546	48250	0.0073
0.9556	48300	0.0068
0.9566	48350	0.007
0.9576	48400	0.0073
0.9586	48450	0.007
0.9595	48500	0.0072
0.9605	48550	0.007
0.9615	48600	0.0069
0.9625	48650	0.0068
0.9635	48700	0.007
0.9645	48750	0.0066
0.9655	48800	0.0068
0.9665	48850	0.0072
0.9675	48900	0.007
0.9684	48950	0.0072
0.9694	49000	0.007
0.9704	49050	0.0064
0.9714	49100	0.007
0.9724	49150	0.007
0.9734	49200	0.0072
0.9744	49250	0.0069
0.9754	49300	0.007
0.9764	49350	0.0068
0.9773	49400	0.0068
0.9783	49450	0.0066
0.9793	49500	0.0069
0.9803	49550	0.0074
0.9813	49600	0.007
0.9823	49650	0.0071
0.9833	49700	0.0068
0.9843	49750	0.007
0.9853	49800	0.0071
0.9862	49850	0.0071
0.9872	49900	0.0069
0.9882	49950	0.0069
0.9892	50000	0.0068
0.9902	50050	0.0072
0.9912	50100	0.0069
0.9922	50150	0.0067
0.9932	50200	0.0072
0.9942	50250	0.0068
0.9952	50300	0.0069
0.9961	50350	0.0074
0.9971	50400	0.0068
0.9981	50450	0.0072
0.9991	50500	0.007

Framework Versions

Python: 3.10.13
Sentence Transformers: 5.3.0
PyLate: 1.5.0
Transformers: 4.48.3
PyTorch: 2.4.0+cu124
Accelerate: 1.13.0
Datasets: 2.21.0
Tokenizers: 0.21.4

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084"
}

PyLate

@inproceedings{DBLP:conf/cikm/ChaffinS25,
  author       = {Antoine Chaffin and
                  Rapha{"{e}}l Sourty},
  editor       = {Meeyoung Cha and
                  Chanyoung Park and
                  Noseong Park and
                  Carl Yang and
                  Senjuti Basu Roy and
                  Jessie Li and
                  Jaap Kamps and
                  Kijung Shin and
                  Bryan Hooi and
                  Lifang He},
  title        = {PyLate: Flexible Training and Retrieval for Late Interaction Models},
  booktitle    = {Proceedings of the 34th {ACM} International Conference on Information
                  and Knowledge Management, {CIKM} 2025, Seoul, Republic of Korea, November
                  10-14, 2025},
  pages        = {6334--6339},
  publisher    = {{ACM}},
  year         = {2025},
  url          = {https://github.com/lightonai/pylate},
  doi          = {10.1145/3746252.3761608},
}