SentenceTransformer based on BAAI/bge-base-zh-v1.5

This is a sentence-transformers model finetuned from BAAI/bge-base-zh-v1.5. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, classification, clustering, and more.

Model Details

Model Description

Model Type: Sentence Transformer
Base model: BAAI/bge-base-zh-v1.5
Maximum Sequence Length: 256 tokens
Output Dimensionality: 768 dimensions
Similarity Function: Cosine Similarity
Supported Modality: Text

Model Sources

Documentation: Sentence Transformers Documentation
Repository: Sentence Transformers on GitHub
Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'transformer_task': 'feature-extraction', 'modality_config': {'text': {'method': 'forward', 'method_output_name': 'last_hidden_state'}}, 'module_output_name': 'token_embeddings', 'architecture': 'BertModel'})
  (1): Pooling({'embedding_dimension': 768, 'pooling_mode': 'cls', 'include_prompt': True})
  (2): Normalize({})
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("liudev/novel-embedding-gpl-base-v1")
# Run inference
sentences = [
    '在异能觉醒爆发时安抚对方并带走宝物',
    '宴席上有人当众示爱引发全场震惊',
    '凶暴巨兽冲破防线引发骚乱',
    '山贼讲述当年的恐惧往事，结果再次遭遇强敌被瞬杀',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [4, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [4, 4]

Training Details

Training Dataset

Unnamed Dataset

Size: 435,262 training samples
Columns: anchor, sentence1, sentence2, and label

Approximate statistics based on the first 100 samples:

	anchor	sentence1	sentence2	label
type	string	string	string	float
modality	text	text	text
details	min: 8 tokens mean: 15.5 tokens max: 25 tokens	min: 47 tokens mean: 73.19 tokens max: 256 tokens	min: 29 tokens mean: 69.23 tokens max: 92 tokens	min: 0.02 mean: 0.37 max: 0.84

Samples:

anchor	sentence1	sentence2	label
`看破敌人的隐身手段`	`周昂在围攻强敌时敏锐察觉到异样，通过观察灵气流动识破并破除隐身法术，最终果断出手将藏匿在墙头的施法者格杀。`	`李破在马场以赏罚分明的手段收拢人心并大摆酒宴后，在李碧的私下召见中，意外得知其欲随军出征辽东的计划并被征询意见。`	`0.759356061900818`
`在仓库意外得到防身宝物`	`柳渊在医馆仓库意外获得防御法器佛珠及攻击真言，随后通过定星罗盘锁定义务星邪，并凭借缜密的推理引导葛老与辛捕头共同前往后院枯井探查。`	`沈河在烂尾楼工地通过系统指引接连开启宝箱，获得了具备强控效果的“大力飞砖”与防御性的“安全员安全帽”，在确认工地无剩余宝箱后，随即根据新坐标向十公里外进发。`	`0.09672797019024215`
`进宫面圣前，太监好心提醒忌讳规矩`	`宦官郭穰在权衡利弊后决定拉拢入觐的天子待诏秀才张毅，不仅对其礼遇有加，更在入宫途中悉心传授面圣禁忌与生存经验，双方达成了初步的盟友共识。`	`在正月初一太子加冠的重大典礼上，南风费尽心思哄劝赖床的太子完成繁琐仪式并接手庞大的东宫官僚体系，同时担忧自己微服出游时的糗事在稍后的公开亮相中被路人认出。`	`0.5887360227245302`

Loss: MarginMSELoss with these parameters:

{
    "similarity_fct": "pairwise_dot_score"
}

Training Hyperparameters

Non-Default Hyperparameters

per_device_train_batch_size: 32
num_train_epochs: 150
learning_rate: 2e-05
warmup_steps: 0.02
fp16: True
push_to_hub: True
hub_model_id: liudev/novel-embedding-gpl-base-v1
hub_strategy: checkpoint
dataloader_num_workers: 4

All Hyperparameters

Click to expand

per_device_train_batch_size: 32
num_train_epochs: 150
max_steps: -1
learning_rate: 2e-05
lr_scheduler_type: linear
lr_scheduler_kwargs: None
warmup_steps: 0.02
optim: adamw_torch_fused
optim_args: None
weight_decay: 0.0
adam_beta1: 0.9
adam_beta2: 0.999
adam_epsilon: 1e-08
optim_target_modules: None
gradient_accumulation_steps: 1
average_tokens_across_devices: True
max_grad_norm: 1.0
label_smoothing_factor: 0.0
bf16: False
fp16: True
bf16_full_eval: False
fp16_full_eval: False
tf32: None
gradient_checkpointing: False
gradient_checkpointing_kwargs: None
torch_compile: False
torch_compile_backend: None
torch_compile_mode: None
use_liger_kernel: False
liger_kernel_config: None
use_cache: False
neftune_noise_alpha: None
torch_empty_cache_steps: None
auto_find_batch_size: False
log_on_each_node: True
logging_nan_inf_filter: True
include_num_input_tokens_seen: no
log_level: passive
log_level_replica: warning
disable_tqdm: False
project: huggingface
trackio_space_id: None
trackio_bucket_id: None
trackio_static_space_id: None
per_device_eval_batch_size: 8
prediction_loss_only: True
eval_on_start: False
eval_do_concat_batches: True
eval_use_gather_object: False
eval_accumulation_steps: None
include_for_metrics: []
batch_eval_metrics: False
save_only_model: False
save_on_each_node: False
enable_jit_checkpoint: False
push_to_hub: True
hub_private_repo: None
hub_model_id: liudev/novel-embedding-gpl-base-v1
hub_strategy: checkpoint
hub_always_push: False
hub_revision: None
load_best_model_at_end: False
ignore_data_skip: False
restore_callback_states_from_checkpoint: False
full_determinism: False
seed: 42
data_seed: None
use_cpu: False
accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
parallelism_config: None
dataloader_drop_last: True
dataloader_num_workers: 4
dataloader_pin_memory: True
dataloader_persistent_workers: False
dataloader_prefetch_factor: None
dataloader_multiprocessing_context: None
dataloader_in_order: True
remove_unused_columns: True
label_names: None
train_sampling_strategy: random
length_column_name: length
ddp_find_unused_parameters: None
ddp_bucket_cap_mb: None
ddp_broadcast_buffers: False
ddp_static_graph: None
ddp_backend: None
ddp_timeout: 1800
fsdp: None
fsdp_config: None
deepspeed: None
debug: []
skip_memory_metrics: True
do_predict: False
resume_from_checkpoint: None
local_rank: -1
prompts: None
batch_sampler: batch_sampler
multi_dataset_batch_sampler: proportional
router_mapping: {}
learning_rate_mapping: {}
warmup_ratio: None

Training Logs

Click to expand

Epoch	Step	Training Loss
94.125	640050	0.0090
94.1324	640100	0.0089
94.1397	640150	0.0085
94.1471	640200	0.0089
94.1544	640250	0.0087
94.1618	640300	0.0090
94.1691	640350	0.0090
94.1765	640400	0.0090
94.1838	640450	0.0090
94.1912	640500	0.0090
94.1985	640550	0.0088
94.2059	640600	0.0095
94.2132	640650	0.0088
94.2206	640700	0.0092
94.2279	640750	0.0090
94.2353	640800	0.0091
94.2426	640850	0.0086
94.25	640900	0.0088
94.2574	640950	0.0091
94.2647	641000	0.0087
94.2721	641050	0.0089
94.2794	641100	0.0089
94.2868	641150	0.0093
94.2941	641200	0.0092
94.3015	641250	0.0090
94.3088	641300	0.0092
94.3162	641350	0.0090
94.3235	641400	0.0091
94.3309	641450	0.0093
94.3382	641500	0.0093
94.3456	641550	0.0090
94.3529	641600	0.0093
94.3603	641650	0.0092
94.3676	641700	0.0094
94.375	641750	0.0091
94.3824	641800	0.0091
94.3897	641850	0.0087
94.3971	641900	0.0091
94.4044	641950	0.0088
94.4118	642000	0.0089
94.4191	642050	0.0089
94.4265	642100	0.0090
94.4338	642150	0.0089
94.4412	642200	0.0093
94.4485	642250	0.0088
94.4559	642300	0.0093
94.4632	642350	0.0086
94.4706	642400	0.0089
94.4779	642450	0.0090
94.4853	642500	0.0092
94.4926	642550	0.0093
94.5	642600	0.0089
94.5074	642650	0.0093
94.5147	642700	0.0093
94.5221	642750	0.0086
94.5294	642800	0.0089
94.5368	642850	0.0090
94.5441	642900	0.0089
94.5515	642950	0.0092
94.5588	643000	0.0092
94.5662	643050	0.0092
94.5735	643100	0.0094
94.5809	643150	0.0091
94.5882	643200	0.0087
94.5956	643250	0.0092
94.6029	643300	0.0090
94.6103	643350	0.0091
94.6176	643400	0.0091
94.625	643450	0.0088
94.6324	643500	0.0089
94.6397	643550	0.0086
94.6471	643600	0.0094
94.6544	643650	0.0091
94.6618	643700	0.0091
94.6691	643750	0.0088
94.6765	643800	0.0090
94.6838	643850	0.0087
94.6912	643900	0.0093
94.6985	643950	0.0085
94.7059	644000	0.0090
94.7132	644050	0.0095
94.7206	644100	0.0087
94.7279	644150	0.0091
94.7353	644200	0.0090
94.7426	644250	0.0091
94.75	644300	0.0090
94.7574	644350	0.0086
94.7647	644400	0.0087
94.7721	644450	0.0090
94.7794	644500	0.0088
94.7868	644550	0.0091
94.7941	644600	0.0088
94.8015	644650	0.0091
94.8088	644700	0.0090
94.8162	644750	0.0087
94.8235	644800	0.0092
94.8309	644850	0.0089
94.8382	644900	0.0094
94.8456	644950	0.0091
94.8529	645000	0.0090
94.8603	645050	0.0090
94.8676	645100	0.0091
94.875	645150	0.0091
94.8824	645200	0.0090
94.8897	645250	0.0090
94.8971	645300	0.0087
94.9044	645350	0.0090
94.9118	645400	0.0088
94.9191	645450	0.0092
94.9265	645500	0.0089
94.9338	645550	0.0088
94.9412	645600	0.0087
94.9485	645650	0.0088
94.9559	645700	0.0089
94.9632	645750	0.0088
94.9706	645800	0.0089
94.9779	645850	0.0090
94.9853	645900	0.0089
94.9926	645950	0.0092
95.0	646000	0.0087
95.0074	646050	0.0090
95.0147	646100	0.0091
95.0221	646150	0.0087
95.0294	646200	0.0091
95.0368	646250	0.0091
95.0441	646300	0.0092
95.0515	646350	0.0089
95.0588	646400	0.0086
95.0662	646450	0.0089
95.0735	646500	0.0088
95.0809	646550	0.0090
95.0882	646600	0.0091
95.0956	646650	0.0098
95.1029	646700	0.0088
95.1103	646750	0.0088
95.1176	646800	0.0089
95.125	646850	0.0086
95.1324	646900	0.0089
95.1397	646950	0.0093
95.1471	647000	0.0091
95.1544	647050	0.0089
95.1618	647100	0.0092
95.1691	647150	0.0089
95.1765	647200	0.0090
95.1838	647250	0.0091
95.1912	647300	0.0091
95.1985	647350	0.0083
95.2059	647400	0.0093
95.2132	647450	0.0088
95.2206	647500	0.0085
95.2279	647550	0.0087
95.2353	647600	0.0090
95.2426	647650	0.0089
95.25	647700	0.0089
95.2574	647750	0.0091
95.2647	647800	0.0092
95.2721	647850	0.0092
95.2794	647900	0.0090
95.2868	647950	0.0088
95.2941	648000	0.0090
95.3015	648050	0.0088
95.3088	648100	0.0090
95.3162	648150	0.0093
95.3235	648200	0.0088
95.3309	648250	0.0088
95.3382	648300	0.0086
95.3456	648350	0.0089
95.3529	648400	0.0089
95.3603	648450	0.0088
95.3676	648500	0.0088
95.375	648550	0.0088
95.3824	648600	0.0091
95.3897	648650	0.0089
95.3971	648700	0.0086
95.4044	648750	0.0090
95.4118	648800	0.0087
95.4191	648850	0.0091
95.4265	648900	0.0092
95.4338	648950	0.0093
95.4412	649000	0.0089
95.4485	649050	0.0089
95.4559	649100	0.0090
95.4632	649150	0.0086
95.4706	649200	0.0094
95.4779	649250	0.0091
95.4853	649300	0.0088
95.4926	649350	0.0091
95.5	649400	0.0091
95.5074	649450	0.0089
95.5147	649500	0.0090
95.5221	649550	0.0091
95.5294	649600	0.0087
95.5368	649650	0.0092
95.5441	649700	0.0091
95.5515	649750	0.0092
95.5588	649800	0.0090
95.5662	649850	0.0088
95.5735	649900	0.0085
95.5809	649950	0.0093
95.5882	650000	0.0091
95.5956	650050	0.0091
95.6029	650100	0.0087
95.6103	650150	0.0088
95.6176	650200	0.0094
95.625	650250	0.0087
95.6324	650300	0.0090
95.6397	650350	0.0091
95.6471	650400	0.0091
95.6544	650450	0.0088
95.6618	650500	0.0087
95.6691	650550	0.0089
95.6765	650600	0.0089
95.6838	650650	0.0086
95.6912	650700	0.0086
95.6985	650750	0.0091
95.7059	650800	0.0088
95.7132	650850	0.0091
95.7206	650900	0.0094
95.7279	650950	0.0092
95.7353	651000	0.0087
95.7426	651050	0.0088
95.75	651100	0.0085
95.7574	651150	0.0089
95.7647	651200	0.0091
95.7721	651250	0.0090
95.7794	651300	0.0088
95.7868	651350	0.0091
95.7941	651400	0.0090
95.8015	651450	0.0089
95.8088	651500	0.0089
95.8162	651550	0.0088
95.8235	651600	0.0088
95.8309	651650	0.0088
95.8382	651700	0.0091
95.8456	651750	0.0089
95.8529	651800	0.0090
95.8603	651850	0.0085
95.8676	651900	0.0092
95.875	651950	0.0093
95.8824	652000	0.0088
95.8897	652050	0.0091
95.8971	652100	0.0089
95.9044	652150	0.0092
95.9118	652200	0.0089
95.9191	652250	0.0088
95.9265	652300	0.0091
95.9338	652350	0.0086
95.9412	652400	0.0089
95.9485	652450	0.0088
95.9559	652500	0.0088
95.9632	652550	0.0090
95.9706	652600	0.0089
95.9779	652650	0.0088
95.9853	652700	0.0086
95.9926	652750	0.0092
96.0	652800	0.0091
96.0074	652850	0.0088
96.0147	652900	0.0090
96.0221	652950	0.0086
96.0294	653000	0.0085
96.0368	653050	0.0090
96.0441	653100	0.0088
96.0515	653150	0.0093
96.0588	653200	0.0089
96.0662	653250	0.0087
96.0735	653300	0.0085
96.0809	653350	0.0088
96.0882	653400	0.0087
96.0956	653450	0.0088
96.1029	653500	0.0090
96.1103	653550	0.0089
96.1176	653600	0.0090
96.125	653650	0.0089
96.1324	653700	0.0086
96.1397	653750	0.0088
96.1471	653800	0.0088
96.1544	653850	0.0085
96.1618	653900	0.0092
96.1691	653950	0.0090
96.1765	654000	0.0089
96.1838	654050	0.0092
96.1912	654100	0.0086
96.1985	654150	0.0091
96.2059	654200	0.0088
96.2132	654250	0.0089
96.2206	654300	0.0090
96.2279	654350	0.0090
96.2353	654400	0.0089
96.2426	654450	0.0092
96.25	654500	0.0088
96.2574	654550	0.0094
96.2647	654600	0.0087
96.2721	654650	0.0084
96.2794	654700	0.0085
96.2868	654750	0.0088
96.2941	654800	0.0087
96.3015	654850	0.0087
96.3088	654900	0.0083
96.3162	654950	0.0094
96.3235	655000	0.0092
96.3309	655050	0.0090
96.3382	655100	0.0089
96.3456	655150	0.0085
96.3529	655200	0.0085
96.3603	655250	0.0090
96.3676	655300	0.0089
96.375	655350	0.0087
96.3824	655400	0.0089
96.3897	655450	0.0094
96.3971	655500	0.0089
96.4044	655550	0.0089
96.4118	655600	0.0085
96.4191	655650	0.0091
96.4265	655700	0.0088
96.4338	655750	0.0088
96.4412	655800	0.0089
96.4485	655850	0.0085
96.4559	655900	0.0090
96.4632	655950	0.0092
96.4706	656000	0.0089
96.4779	656050	0.0086
96.4853	656100	0.0091
96.4926	656150	0.0092
96.5	656200	0.0085
96.5074	656250	0.0089
96.5147	656300	0.0090
96.5221	656350	0.0089
96.5294	656400	0.0091
96.5368	656450	0.0087
96.5441	656500	0.0085
96.5515	656550	0.0087
96.5588	656600	0.0093
96.5662	656650	0.0089
96.5735	656700	0.0086
96.5809	656750	0.0088
96.5882	656800	0.0088
96.5956	656850	0.0093
96.6029	656900	0.0089
96.6103	656950	0.0090
96.6176	657000	0.0088
96.625	657050	0.0092
96.6324	657100	0.0087
96.6397	657150	0.0091
96.6471	657200	0.0091
96.6544	657250	0.0088
96.6618	657300	0.0086
96.6691	657350	0.0089
96.6765	657400	0.0090
96.6838	657450	0.0087
96.6912	657500	0.0088
96.6985	657550	0.0088
96.7059	657600	0.0089
96.7132	657650	0.0091
96.7206	657700	0.0090
96.7279	657750	0.0086
96.7353	657800	0.0086
96.7426	657850	0.0085
96.75	657900	0.0092
96.7574	657950	0.0089
96.7647	658000	0.0089
96.7721	658050	0.0092
96.7794	658100	0.0088
96.7868	658150	0.0088
96.7941	658200	0.0087
96.8015	658250	0.0088
96.8088	658300	0.0091
96.8162	658350	0.0089
96.8235	658400	0.0087
96.8309	658450	0.0086
96.8382	658500	0.0092
96.8456	658550	0.0088
96.8529	658600	0.0096
96.8603	658650	0.0088
96.8676	658700	0.0091
96.875	658750	0.0088
96.8824	658800	0.0089
96.8897	658850	0.0094
96.8971	658900	0.0090
96.9044	658950	0.0088
96.9118	659000	0.0091
96.9191	659050	0.0091
96.9265	659100	0.0086
96.9338	659150	0.0090
96.9412	659200	0.0090
96.9485	659250	0.0089
96.9559	659300	0.0089
96.9632	659350	0.0086
96.9706	659400	0.0086
96.9779	659450	0.0090
96.9853	659500	0.0087
96.9926	659550	0.0086
97.0	659600	0.0091
97.0074	659650	0.0091
97.0147	659700	0.0088
97.0221	659750	0.0086
97.0294	659800	0.0088
97.0368	659850	0.0086
97.0441	659900	0.0090
97.0515	659950	0.0091
97.0588	660000	0.0086
97.0662	660050	0.0087
97.0735	660100	0.0089
97.0809	660150	0.0087
97.0882	660200	0.0090
97.0956	660250	0.0088
97.1029	660300	0.0087
97.1103	660350	0.0090
97.1176	660400	0.0087
97.125	660450	0.0086
97.1324	660500	0.0087
97.1397	660550	0.0089
97.1471	660600	0.0088
97.1544	660650	0.0084
97.1618	660700	0.0091
97.1691	660750	0.0086
97.1765	660800	0.0088
97.1838	660850	0.0084
97.1912	660900	0.0083
97.1985	660950	0.0087
97.2059	661000	0.0089
97.2132	661050	0.0090
97.2206	661100	0.0088
97.2279	661150	0.0089
97.2353	661200	0.0082
97.2426	661250	0.0086
97.25	661300	0.0092
97.2574	661350	0.0089
97.2647	661400	0.0091
97.2721	661450	0.0091
97.2794	661500	0.0091
97.2868	661550	0.0086
97.2941	661600	0.0091
97.3015	661650	0.0089
97.3088	661700	0.0083
97.3162	661750	0.0087
97.3235	661800	0.0088
97.3309	661850	0.0091
97.3382	661900	0.0089
97.3456	661950	0.0088
97.3529	662000	0.0086
97.3603	662050	0.0090
97.3676	662100	0.0085
97.375	662150	0.0088
97.3824	662200	0.0087
97.3897	662250	0.0089
97.3971	662300	0.0089
97.4044	662350	0.0090
97.4118	662400	0.0089
97.4191	662450	0.0089
97.4265	662500	0.0094
97.4338	662550	0.0087
97.4412	662600	0.0092
97.4485	662650	0.0090
97.4559	662700	0.0091
97.4632	662750	0.0091
97.4706	662800	0.0088
97.4779	662850	0.0092
97.4853	662900	0.0095
97.4926	662950	0.0088
97.5	663000	0.0089
97.5074	663050	0.0091
97.5147	663100	0.0091
97.5221	663150	0.0092
97.5294	663200	0.0089
97.5368	663250	0.0089
97.5441	663300	0.0089
97.5515	663350	0.0089
97.5588	663400	0.0092
97.5662	663450	0.0085
97.5735	663500	0.0086
97.5809	663550	0.0085
97.5882	663600	0.0089
97.5956	663650	0.0085
97.6029	663700	0.0089
97.6103	663750	0.0089
97.6176	663800	0.0089
97.625	663850	0.0085
97.6324	663900	0.0085
97.6397	663950	0.0090
97.6471	664000	0.0082
97.6544	664050	0.0090
97.6618	664100	0.0090
97.6691	664150	0.0089
97.6765	664200	0.0088
97.6838	664250	0.0088
97.6912	664300	0.0092
97.6985	664350	0.0087
97.7059	664400	0.0092
97.7132	664450	0.0092
97.7206	664500	0.0087
97.7279	664550	0.0087
97.7353	664600	0.0091
97.7426	664650	0.0092
97.75	664700	0.0090
97.7574	664750	0.0091
97.7647	664800	0.0091
97.7721	664850	0.0093
97.7794	664900	0.0092
97.7868	664950	0.0088
97.7941	665000	0.0088
97.8015	665050	0.0087
97.8088	665100	0.0089
97.8162	665150	0.0086
97.8235	665200	0.0087
97.8309	665250	0.0089
97.8382	665300	0.0090
97.8456	665350	0.0087
97.8529	665400	0.0084
97.8603	665450	0.0089
97.8676	665500	0.0087
97.875	665550	0.0090
97.8824	665600	0.0089
97.8897	665650	0.0084
97.8971	665700	0.0093
97.9044	665750	0.0090
97.9118	665800	0.0091
97.9191	665850	0.0090
97.9265	665900	0.0093
97.9338	665950	0.0089
97.9412	666000	0.0091
97.9485	666050	0.0091
97.9559	666100	0.0083
97.9632	666150	0.0086
97.9706	666200	0.0086
97.9779	666250	0.0089
97.9853	666300	0.0087
97.9926	666350	0.0093
98.0	666400	0.0091
98.0074	666450	0.0084
98.0147	666500	0.0087
98.0221	666550	0.0089
98.0294	666600	0.0087
98.0368	666650	0.0092
98.0441	666700	0.0084
98.0515	666750	0.0082
98.0588	666800	0.0086
98.0662	666850	0.0087
98.0735	666900	0.0084
98.0809	666950	0.0090
98.0882	667000	0.0089
98.0956	667050	0.0088
98.1029	667100	0.0090
98.1103	667150	0.0090
98.1176	667200	0.0087
98.125	667250	0.0088
98.1324	667300	0.0087
98.1397	667350	0.0087
98.1471	667400	0.0091
98.1544	667450	0.0085
98.1618	667500	0.0088
98.1691	667550	0.0089
98.1765	667600	0.0082
98.1838	667650	0.0087
98.1912	667700	0.0085
98.1985	667750	0.0086
98.2059	667800	0.0089
98.2132	667850	0.0085
98.2206	667900	0.0088
98.2279	667950	0.0082
98.2353	668000	0.0090
98.2426	668050	0.0086
98.25	668100	0.0084
98.2574	668150	0.0092
98.2647	668200	0.0087
98.2721	668250	0.0089
98.2794	668300	0.0087
98.2868	668350	0.0088
98.2941	668400	0.0091
98.3015	668450	0.0088
98.3088	668500	0.0088
98.3162	668550	0.0092
98.3235	668600	0.0089
98.3309	668650	0.0087
98.3382	668700	0.0090
98.3456	668750	0.0089
98.3529	668800	0.0089
98.3603	668850	0.0086
98.3676	668900	0.0086
98.375	668950	0.0090
98.3824	669000	0.0085
98.3897	669050	0.0085
98.3971	669100	0.0088
98.4044	669150	0.0088
98.4118	669200	0.0092
98.4191	669250	0.0088
98.4265	669300	0.0092
98.4338	669350	0.0085
98.4412	669400	0.0088
98.4485	669450	0.0092
98.4559	669500	0.0086
98.4632	669550	0.0082
98.4706	669600	0.0094
98.4779	669650	0.0093
98.4853	669700	0.0086
98.4926	669750	0.0089
98.5	669800	0.0091
98.5074	669850	0.0092
98.5147	669900	0.0091
98.5221	669950	0.0087
98.5294	670000	0.0089
98.5368	670050	0.0086
98.5441	670100	0.0088
98.5515	670150	0.0091
98.5588	670200	0.0090
98.5662	670250	0.0091
98.5735	670300	0.0087
98.5809	670350	0.0092
98.5882	670400	0.0085
98.5956	670450	0.0087
98.6029	670500	0.0088
98.6103	670550	0.0087
98.6176	670600	0.0087
98.625	670650	0.0084
98.6324	670700	0.0091
98.6397	670750	0.0090
98.6471	670800	0.0089
98.6544	670850	0.0089
98.6618	670900	0.0088
98.6691	670950	0.0089
98.6765	671000	0.0091
98.6838	671050	0.0086
98.6912	671100	0.0087
98.6985	671150	0.0092
98.7059	671200	0.0086
98.7132	671250	0.0086
98.7206	671300	0.0089
98.7279	671350	0.0089
98.7353	671400	0.0088
98.7426	671450	0.0081
98.75	671500	0.0087
98.7574	671550	0.0088
98.7647	671600	0.0088
98.7721	671650	0.0091
98.7794	671700	0.0087
98.7868	671750	0.0087
98.7941	671800	0.0086
98.8015	671850	0.0085
98.8088	671900	0.0090
98.8162	671950	0.0088
98.8235	672000	0.0092
98.8309	672050	0.0092
98.8382	672100	0.0087
98.8456	672150	0.0086
98.8529	672200	0.0089
98.8603	672250	0.0089
98.8676	672300	0.0088
98.875	672350	0.0086
98.8824	672400	0.0088
98.8897	672450	0.0086
98.8971	672500	0.0083
98.9044	672550	0.0090
98.9118	672600	0.0090
98.9191	672650	0.0090
98.9265	672700	0.0086
98.9338	672750	0.0091
98.9412	672800	0.0091
98.9485	672850	0.0087
98.9559	672900	0.0088
98.9632	672950	0.0089
98.9706	673000	0.0087
98.9779	673050	0.0091
98.9853	673100	0.0089
98.9926	673150	0.0092
99.0	673200	0.0088
99.0074	673250	0.0088
99.0147	673300	0.0087
99.0221	673350	0.0089
99.0294	673400	0.0087
99.0368	673450	0.0087
99.0441	673500	0.0087
99.0515	673550	0.0086
99.0588	673600	0.0085
99.0662	673650	0.0088
99.0735	673700	0.0090
99.0809	673750	0.0088
99.0882	673800	0.0087
99.0956	673850	0.0086
99.1029	673900	0.0086
99.1103	673950	0.0086
99.1176	674000	0.0085
99.125	674050	0.0092
99.1324	674100	0.0087
99.1397	674150	0.0083
99.1471	674200	0.0089
99.1544	674250	0.0084
99.1618	674300	0.0088
99.1691	674350	0.0088
99.1765	674400	0.0086
99.1838	674450	0.0090
99.1912	674500	0.0089
99.1985	674550	0.0091
99.2059	674600	0.0093
99.2132	674650	0.0087
99.2206	674700	0.0088
99.2279	674750	0.0086
99.2353	674800	0.0082
99.2426	674850	0.0090
99.25	674900	0.0087
99.2574	674950	0.0083
99.2647	675000	0.0086
99.2721	675050	0.0085
99.2794	675100	0.0088
99.2868	675150	0.0084
99.2941	675200	0.0089
99.3015	675250	0.0090
99.3088	675300	0.0087
99.3162	675350	0.0089
99.3235	675400	0.0089
99.3309	675450	0.0090
99.3382	675500	0.0089
99.3456	675550	0.0089
99.3529	675600	0.0088
99.3603	675650	0.0087
99.3676	675700	0.0087
99.375	675750	0.0092
99.3824	675800	0.0092
99.3897	675850	0.0089
99.3971	675900	0.0088
99.4044	675950	0.0089
99.4118	676000	0.0094
99.4191	676050	0.0090
99.4265	676100	0.0091
99.4338	676150	0.0090
99.4412	676200	0.0088
99.4485	676250	0.0089
99.4559	676300	0.0086
99.4632	676350	0.0085
99.4706	676400	0.0085
99.4779	676450	0.0086
99.4853	676500	0.0091
99.4926	676550	0.0092
99.5	676600	0.0086
99.5074	676650	0.0093
99.5147	676700	0.0089
99.5221	676750	0.0089
99.5294	676800	0.0087
99.5368	676850	0.0090
99.5441	676900	0.0090
99.5515	676950	0.0092
99.5588	677000	0.0085
99.5662	677050	0.0088
99.5735	677100	0.0087
99.5809	677150	0.0088
99.5882	677200	0.0087
99.5956	677250	0.0086
99.6029	677300	0.0087
99.6103	677350	0.0093
99.6176	677400	0.0091
99.625	677450	0.0089
99.6324	677500	0.0091
99.6397	677550	0.0092
99.6471	677600	0.0085
99.6544	677650	0.0093
99.6618	677700	0.0088
99.6691	677750	0.0087
99.6765	677800	0.0092
99.6838	677850	0.0086
99.6912	677900	0.0087
99.6985	677950	0.0085
99.7059	678000	0.0087
99.7132	678050	0.0088
99.7206	678100	0.0087
99.7279	678150	0.0086
99.7353	678200	0.0083
99.7426	678250	0.0090
99.75	678300	0.0087
99.7574	678350	0.0089
99.7647	678400	0.0088
99.7721	678450	0.0089
99.7794	678500	0.0089
99.7868	678550	0.0094
99.7941	678600	0.0086
99.8015	678650	0.0085
99.8088	678700	0.0088
99.8162	678750	0.0088
99.8235	678800	0.0087
99.8309	678850	0.0090
99.8382	678900	0.0088
99.8456	678950	0.0090
99.8529	679000	0.0085
99.8603	679050	0.0085
99.8676	679100	0.0094
99.875	679150	0.0088
99.8824	679200	0.0085
99.8897	679250	0.0085
99.8971	679300	0.0084
99.9044	679350	0.0084
99.9118	679400	0.0094
99.9191	679450	0.0089
99.9265	679500	0.0088
99.9338	679550	0.0088
99.9412	679600	0.0084
99.9485	679650	0.0091
99.9559	679700	0.0091
99.9632	679750	0.0087
99.9706	679800	0.0085
99.9779	679850	0.0088
99.9853	679900	0.0084
99.9926	679950	0.0090
100.0	680000	0.0092
100.0074	680050	0.0088
100.0147	680100	0.0090
100.0221	680150	0.0088
100.0294	680200	0.0084
100.0368	680250	0.0087
100.0441	680300	0.0088
100.0515	680350	0.0088
100.0588	680400	0.0085
100.0662	680450	0.0089
100.0735	680500	0.0093
100.0809	680550	0.0088
100.0882	680600	0.0088
100.0956	680650	0.0090
100.1029	680700	0.0084
100.1103	680750	0.0089
100.1176	680800	0.0086
100.125	680850	0.0082
100.1324	680900	0.0085
100.1397	680950	0.0090
100.1471	681000	0.0086
100.1544	681050	0.0088
100.1618	681100	0.0084
100.1691	681150	0.0088
100.1765	681200	0.0090
100.1838	681250	0.0085
100.1912	681300	0.0086
100.1985	681350	0.0085
100.2059	681400	0.0088
100.2132	681450	0.0088
100.2206	681500	0.0086
100.2279	681550	0.0087
100.2353	681600	0.0087
100.2426	681650	0.0087
100.25	681700	0.0088
100.2574	681750	0.0088
100.2647	681800	0.0085
100.2721	681850	0.0084
100.2794	681900	0.0085
100.2868	681950	0.0083
100.2941	682000	0.0086
100.3015	682050	0.0087
100.3088	682100	0.0089
100.3162	682150	0.0083
100.3235	682200	0.0085
100.3309	682250	0.0092
100.3382	682300	0.0087
100.3456	682350	0.0088
100.3529	682400	0.0087
100.3603	682450	0.0087
100.3676	682500	0.0087
100.375	682550	0.0091
100.3824	682600	0.0090
100.3897	682650	0.0090
100.3971	682700	0.0089
100.4044	682750	0.0088
100.4118	682800	0.0086
100.4191	682850	0.0086
100.4265	682900	0.0085
100.4338	682950	0.0084
100.4412	683000	0.0085
100.4485	683050	0.0086
100.4559	683100	0.0088
100.4632	683150	0.0083
100.4706	683200	0.0088
100.4779	683250	0.0084
100.4853	683300	0.0088
100.4926	683350	0.0085
100.5	683400	0.0089
100.5074	683450	0.0087
100.5147	683500	0.0087
100.5221	683550	0.0085
100.5294	683600	0.0089
100.5368	683650	0.0086
100.5441	683700	0.0086
100.5515	683750	0.0085
100.5588	683800	0.0092
100.5662	683850	0.0091
100.5735	683900	0.0087
100.5809	683950	0.0087
100.5882	684000	0.0088
100.5956	684050	0.0088
100.6029	684100	0.0083
100.6103	684150	0.0084
100.6176	684200	0.0090
100.625	684250	0.0093
100.6324	684300	0.0088
100.6397	684350	0.0089
100.6471	684400	0.0085
100.6544	684450	0.0092
100.6618	684500	0.0088
100.6691	684550	0.0089
100.6765	684600	0.0085
100.6838	684650	0.0090
100.6912	684700	0.0087
100.6985	684750	0.0086
100.7059	684800	0.0086
100.7132	684850	0.0089
100.7206	684900	0.0086
100.7279	684950	0.0085
100.7353	685000	0.0088
100.7426	685050	0.0087
100.75	685100	0.0085
100.7574	685150	0.0085
100.7647	685200	0.0083
100.7721	685250	0.0089
100.7794	685300	0.0093
100.7868	685350	0.0089
100.7941	685400	0.0084
100.8015	685450	0.0087
100.8088	685500	0.0084
100.8162	685550	0.0089
100.8235	685600	0.0088
100.8309	685650	0.0085
100.8382	685700	0.0090
100.8456	685750	0.0086
100.8529	685800	0.0089
100.8603	685850	0.0086
100.8676	685900	0.0086
100.875	685950	0.0086
100.8824	686000	0.0087
100.8897	686050	0.0089
100.8971	686100	0.0088
100.9044	686150	0.0089
100.9118	686200	0.0088
100.9191	686250	0.0086
100.9265	686300	0.0082
100.9338	686350	0.0086
100.9412	686400	0.0084
100.9485	686450	0.0092
100.9559	686500	0.0087
100.9632	686550	0.0088
100.9706	686600	0.0088
100.9779	686650	0.0086
100.9853	686700	0.0088
100.9926	686750	0.0088
101.0	686800	0.0088
101.0074	686850	0.0087
101.0147	686900	0.0089
101.0221	686950	0.0085
101.0294	687000	0.0085
101.0368	687050	0.0088
101.0441	687100	0.0089
101.0515	687150	0.0086
101.0588	687200	0.0092
101.0662	687250	0.0089
101.0735	687300	0.0086
101.0809	687350	0.0086
101.0882	687400	0.0087
101.0956	687450	0.0088
101.1029	687500	0.0086
101.1103	687550	0.0091
101.1176	687600	0.0090
101.125	687650	0.0087
101.1324	687700	0.0086
101.1397	687750	0.0085
101.1471	687800	0.0087
101.1544	687850	0.0089
101.1618	687900	0.0089
101.1691	687950	0.0083
101.1765	688000	0.0088
101.1838	688050	0.0085
101.1912	688100	0.0087
101.1985	688150	0.0090
101.2059	688200	0.0088
101.2132	688250	0.0089
101.2206	688300	0.0086
101.2279	688350	0.0090
101.2353	688400	0.0088
101.2426	688450	0.0088
101.25	688500	0.0086
101.2574	688550	0.0086
101.2647	688600	0.0088
101.2721	688650	0.0081
101.2794	688700	0.0087
101.2868	688750	0.0087
101.2941	688800	0.0089
101.3015	688850	0.0092
101.3088	688900	0.0080
101.3162	688950	0.0088
101.3235	689000	0.0084
101.3309	689050	0.0090
101.3382	689100	0.0084
101.3456	689150	0.0085
101.3529	689200	0.0087
101.3603	689250	0.0087
101.3676	689300	0.0091
101.375	689350	0.0087
101.3824	689400	0.0088
101.3897	689450	0.0087
101.3971	689500	0.0088
101.4044	689550	0.0085
101.4118	689600	0.0087
101.4191	689650	0.0088
101.4265	689700	0.0088
101.4338	689750	0.0089
101.4412	689800	0.0088
101.4485	689850	0.0085
101.4559	689900	0.0082
101.4632	689950	0.0085
101.4706	690000	0.0088

Training Time

Training: 7.4 hours

Framework Versions

Python: 3.13.15
Sentence Transformers: 5.7.0
Transformers: 5.16.1
PyTorch: 2.11.0+cu128
Accelerate: 1.14.0
Datasets: 4.8.5
Tokenizers: 0.23.1

Additional Resources

Training and Finetuning Embedding Models with Sentence Transformers: the end-to-end guide for training or finetuning Sentence Transformer models.
Introduction to Matryoshka Embedding Models: variable-size embeddings that can be truncated with minimal quality loss.
Binary and Scalar Embedding Quantization for Significantly Faster & Cheaper Retrieval: post-training compression of embedding vectors.
Multimodal Embedding & Reranker Models with Sentence Transformers: use text, image, audio, and video models through the same API.
Training and Finetuning Multimodal Embedding & Reranker Models with Sentence Transformers: train multimodal embedding models, with a Visual Document Retrieval walkthrough.

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MarginMSELoss

@misc{hofstätter2021improving,
    title={Improving Efficient Neural Ranking Models with Cross-Architecture Knowledge Distillation},
    author={Sebastian Hofstätter and Sophia Althammer and Michael Schröder and Mete Sertkan and Allan Hanbury},
    year={2021},
    eprint={2010.02666},
    archivePrefix={arXiv},
    primaryClass={cs.IR}
}