Sentence Similarity
sentence-transformers
Safetensors
bert
feature-extraction
dense
Generated from Trainer
dataset_size:86450
loss:DenoisingAutoEncoderLoss
Eval Results (legacy)
text-embeddings-inference
Instructions to use SwarajSolanke-turtle/make-model-mapping-TSDAE with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- sentence-transformers
How to use SwarajSolanke-turtle/make-model-mapping-TSDAE with sentence-transformers:
from sentence_transformers import SentenceTransformer model = SentenceTransformer("SwarajSolanke-turtle/make-model-mapping-TSDAE") sentences = [ "mini", "mini contryman e electric 1", "tata safari xm diesel 1956", "daewoo cielo glx petrol 1498" ] embeddings = model.encode(sentences) similarities = model.similarity(embeddings, embeddings) print(similarities.shape) # [4, 4] - Notebooks
- Google Colab
- Kaggle
|
Download README.md from SwarajSolanke-turtle/make-model-mapping-TSDAE: direct link, hf CLI and curl.
- Browser
- Download file 39 kB
-
https://huggingface.co/SwarajSolanke-turtle/make-model-mapping-TSDAE/resolve/main/README.md
- Command line
-
hf download hf://SwarajSolanke-turtle/make-model-mapping-TSDAE/README.md
-
curl -L -o README.md https://huggingface.co/SwarajSolanke-turtle/make-model-mapping-TSDAE/resolve/main/README.md
39 kB
metadata
tags:
- sentence-transformers
- sentence-similarity
- feature-extraction
- dense
- generated_from_trainer
- dataset_size:86450
- loss:DenoisingAutoEncoderLoss
base_model: google-bert/bert-base-uncased
widget:
- source_sentence: mini
sentences:
- mini contryman e electric 1
- tata safari xm diesel 1956
- daewoo cielo glx petrol 1498
- source_sentence: '1462'
sentences:
- maruti alto 800 lxi cng 796
- toyota rumion g at petrol 1462
- honda city v mt petrol 1498
- source_sentence: go.t cng
sentences:
- datsun go plus .t cng 1198
- bmw m5 standard petrol 4999
- volvo xc60 d4 diesel 1984
- source_sentence: tesla y l
sentences:
- tesla model y l premium awd
- toyota townace townace petrol 2200
- maruti 1000 ac petrol 970
- source_sentence: subaru -
sentences:
- subaru impreza - petrol 2000
- honda br - v s diesel 1498
- tata indica gls cng 1193
pipeline_tag: sentence-similarity
library_name: sentence-transformers
metrics:
- cosine_accuracy
model-index:
- name: SentenceTransformer based on google-bert/bert-base-uncased
results:
- task:
type: triplet
name: Triplet
dataset:
name: fw variant dev
type: fw-variant-dev
metrics:
- type: cosine_accuracy
value: 0.9660000205039978
name: Cosine Accuracy
SentenceTransformer based on google-bert/bert-base-uncased
This is a sentence-transformers model finetuned from google-bert/bert-base-uncased. It maps inputs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, classification, clustering, and more.
Model Details
Model Description
- Model Type: Sentence Transformer
- Base model: google-bert/bert-base-uncased
- Maximum Sequence Length: 32 tokens
- Output Dimensionality: 768 dimensions
- Similarity Function: Cosine Similarity
- Supported Modality: Text
Model Sources
- Documentation: Sentence Transformers Documentation
- Repository: Sentence Transformers on GitHub
- Hugging Face: Sentence Transformers on Hugging Face
Full Model Architecture
SentenceTransformer(
(0): Transformer({'transformer_task': 'feature-extraction', 'modality_config': {'text': {'method': 'forward', 'method_output_name': 'last_hidden_state'}}, 'module_output_name': 'token_embeddings', 'architecture': 'BertModel'})
(1): Pooling({'embedding_dimension': 768, 'pooling_mode': 'cls', 'include_prompt': True})
)
Usage
Direct Usage (Sentence Transformers)
First install the Sentence Transformers library:
pip install -U sentence-transformers
Then you can load this model and run inference.
from sentence_transformers import SentenceTransformer
# Download from the 🤗 Hub
model = SentenceTransformer("SwarajSolanke-turtle/bert-base-uncased-tsdae-fw")
# Run inference
sentences = [
'subaru -',
'subaru impreza - petrol 2000',
'honda br - v s diesel 1498',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]
# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities)
# tensor([[1.0000, 0.9939, 0.9908],
# [0.9939, 1.0000, 0.9881],
# [0.9908, 0.9881, 1.0000]])
Evaluation
Metrics
Triplet
- Dataset:
fw-variant-dev - Evaluated with
TripletEvaluator
| Metric | Value |
|---|---|
| cosine_accuracy | 0.966 |
Training Details
Training Dataset
Unnamed Dataset
- Size: 86,450 training samples
- Columns:
damagedandoriginal - Approximate statistics based on the first 100 samples:
damaged original type string string modality text text details - min: 4 tokens
- mean: 11.63 tokens
- max: 22 tokens
- min: 8 tokens
- mean: 16.18 tokens
- max: 25 tokens
- Samples:
damaged original 1.5 petrol 1482hyundai alcazar 1.5 executive mt petrol 1482mahindra. 300 1.5 w8 tone diesel 1497mahindra. xuv 300 1.5 w8 dual tone diesel 1497mahindra scorpio vlx mhawk bs iv airbag diesel 2179mahindra scorpio vlx 2.2 mhawk bs iv airbag diesel 2179 - Loss:
DenoisingAutoEncoderLosswith these parameters:{ "decoder_name_or_path": "google-bert/bert-base-uncased", "need_retokenization": false }
Evaluation Dataset
Unnamed Dataset
- Size: 1,765 evaluation samples
- Columns:
damagedandoriginal - Approximate statistics based on the first 100 samples:
damaged original type string string modality text text details - min: 5 tokens
- mean: 11.19 tokens
- max: 20 tokens
- min: 9 tokens
- mean: 15.5 tokens
- max: 29 tokens
- Samples:
damaged original bentley continental gt 6.0 speed convertible 4 petrolbentley continental gt 6.0 speed convertible 4 seater petrol 5998tata 1.2 rtn bs-vi 1199tata altroz xt 1.2 rtn bs-vi petrol 1199maruti s-cross hybrid diesel 1248maruti s-cross hybrid diesel 1248 - Loss:
DenoisingAutoEncoderLosswith these parameters:{ "decoder_name_or_path": "google-bert/bert-base-uncased", "need_retokenization": false }
Training Hyperparameters
Non-Default Hyperparameters
per_device_train_batch_size: 64num_train_epochs: 10learning_rate: 1e-05warmup_steps: 0.1bf16: Trueper_device_eval_batch_size: 64seed: 46
All Hyperparameters
Click to expand
per_device_train_batch_size: 64num_train_epochs: 10max_steps: -1learning_rate: 1e-05lr_scheduler_type: linearlr_scheduler_kwargs: Nonewarmup_steps: 0.1optim: adamw_torch_fusedoptim_args: Noneweight_decay: 0.0adam_beta1: 0.9adam_beta2: 0.999adam_epsilon: 1e-08optim_target_modules: Nonegradient_accumulation_steps: 1average_tokens_across_devices: Truemax_grad_norm: 1.0label_smoothing_factor: 0.0bf16: Truefp16: Falsebf16_full_eval: Falsefp16_full_eval: Falsetf32: Nonegradient_checkpointing: Falsegradient_checkpointing_kwargs: Nonetorch_compile: Falsetorch_compile_backend: Nonetorch_compile_mode: Noneuse_liger_kernel: Falseliger_kernel_config: Noneuse_cache: Falseneftune_noise_alpha: Nonetorch_empty_cache_steps: Noneauto_find_batch_size: Falselog_on_each_node: Truelogging_nan_inf_filter: Trueinclude_num_input_tokens_seen: nolog_level: passivelog_level_replica: warningdisable_tqdm: Falseproject: huggingfacetrackio_space_id: Nonetrackio_bucket_id: Nonetrackio_static_space_id: Noneper_device_eval_batch_size: 64prediction_loss_only: Trueeval_on_start: Falseeval_do_concat_batches: Trueeval_use_gather_object: Falseeval_accumulation_steps: Noneinclude_for_metrics: []batch_eval_metrics: Falsesave_only_model: Falsesave_on_each_node: Falseenable_jit_checkpoint: Falsepush_to_hub: Falsehub_private_repo: Nonehub_model_id: Nonehub_strategy: every_savehub_always_push: Falsehub_revision: Noneload_best_model_at_end: Falseignore_data_skip: Falserestore_callback_states_from_checkpoint: Falsefull_determinism: Falseseed: 46data_seed: Noneuse_cpu: Falseaccelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}parallelism_config: Nonedataloader_drop_last: Falsedataloader_num_workers: 0dataloader_pin_memory: Truedataloader_persistent_workers: Falsedataloader_prefetch_factor: Nonedataloader_multiprocessing_context: Nonedataloader_in_order: Trueremove_unused_columns: Truelabel_names: Nonetrain_sampling_strategy: randomlength_column_name: lengthddp_find_unused_parameters: Noneddp_bucket_cap_mb: Noneddp_broadcast_buffers: Falseddp_static_graph: Noneddp_backend: Noneddp_timeout: 1800fsdp: Nonefsdp_config: Nonedeepspeed: Nonedebug: []skip_memory_metrics: Truedo_predict: Falseresume_from_checkpoint: Nonelocal_rank: -1prompts: Nonebatch_sampler: batch_samplermulti_dataset_batch_sampler: proportionalrouter_mapping: {}learning_rate_mapping: {}warmup_ratio: None
Training Logs
Click to expand
| Epoch | Step | Training Loss | Validation Loss | fw-variant-dev_cosine_accuracy |
|---|---|---|---|---|
| -1 | -1 | - | - | 0.8845 |
| 0.0370 | 50 | 10.9495 | - | - |
| 0.0740 | 100 | 8.5518 | - | - |
| 0.1110 | 150 | 7.2139 | - | - |
| 0.1480 | 200 | 6.5304 | - | - |
| 0.1850 | 250 | 5.8942 | - | - |
| 0.2221 | 300 | 5.0836 | - | - |
| 0.2591 | 350 | 4.4827 | - | - |
| 0.2961 | 400 | 4.0965 | - | - |
| 0.3331 | 450 | 3.7756 | - | - |
| 0.3701 | 500 | 3.5197 | 3.3155 | 0.8885 |
| 0.4071 | 550 | 3.2203 | - | - |
| 0.4441 | 600 | 3.0355 | - | - |
| 0.4811 | 650 | 2.8250 | - | - |
| 0.5181 | 700 | 2.6778 | - | - |
| 0.5551 | 750 | 2.4930 | - | - |
| 0.5922 | 800 | 2.3938 | - | - |
| 0.6292 | 850 | 2.2241 | - | - |
| 0.6662 | 900 | 2.1228 | - | - |
| 0.7032 | 950 | 2.0260 | - | - |
| 0.7402 | 1000 | 1.9064 | 1.8352 | 0.9680 |
| 0.7772 | 1050 | 1.8245 | - | - |
| 0.8142 | 1100 | 1.7392 | - | - |
| 0.8512 | 1150 | 1.6638 | - | - |
| 0.8882 | 1200 | 1.6003 | - | - |
| 0.9252 | 1250 | 1.5114 | - | - |
| 0.9623 | 1300 | 1.4978 | - | - |
| 0.9993 | 1350 | 1.4336 | - | - |
| 1.0363 | 1400 | 1.3469 | - | - |
| 1.0733 | 1450 | 1.3045 | - | - |
| 1.1103 | 1500 | 1.2647 | 1.2372 | 0.9670 |
| 1.1473 | 1550 | 1.2286 | - | - |
| 1.1843 | 1600 | 1.2249 | - | - |
| 1.2213 | 1650 | 1.1645 | - | - |
| 1.2583 | 1700 | 1.1292 | - | - |
| 1.2953 | 1750 | 1.1289 | - | - |
| 1.3323 | 1800 | 1.0791 | - | - |
| 1.3694 | 1850 | 1.1114 | - | - |
| 1.4064 | 1900 | 1.0625 | - | - |
| 1.4434 | 1950 | 1.0366 | - | - |
| 1.4804 | 2000 | 1.0352 | 0.9793 | 0.9680 |
| 1.5174 | 2050 | 1.0102 | - | - |
| 1.5544 | 2100 | 0.9745 | - | - |
| 1.5914 | 2150 | 0.9928 | - | - |
| 1.6284 | 2200 | 0.9608 | - | - |
| 1.6654 | 2250 | 0.9365 | - | - |
| 1.7024 | 2300 | 0.9378 | - | - |
| 1.7395 | 2350 | 0.9319 | - | - |
| 1.7765 | 2400 | 0.8982 | - | - |
| 1.8135 | 2450 | 0.8977 | - | - |
| 1.8505 | 2500 | 0.9045 | 0.8560 | 0.9635 |
| 1.8875 | 2550 | 0.9055 | - | - |
| 1.9245 | 2600 | 0.8841 | - | - |
| 1.9615 | 2650 | 0.8696 | - | - |
| 1.9985 | 2700 | 0.8483 | - | - |
| 2.0355 | 2750 | 0.8080 | - | - |
| 2.0725 | 2800 | 0.8050 | - | - |
| 2.1095 | 2850 | 0.7986 | - | - |
| 2.1466 | 2900 | 0.7870 | - | - |
| 2.1836 | 2950 | 0.7727 | - | - |
| 2.2206 | 3000 | 0.7667 | 0.7722 | 0.9645 |
| 2.2576 | 3050 | 0.7718 | - | - |
| 2.2946 | 3100 | 0.7570 | - | - |
| 2.3316 | 3150 | 0.7626 | - | - |
| 2.3686 | 3200 | 0.7846 | - | - |
| 2.4056 | 3250 | 0.7362 | - | - |
| 2.4426 | 3300 | 0.7447 | - | - |
| 2.4796 | 3350 | 0.7596 | - | - |
| 2.5167 | 3400 | 0.7430 | - | - |
| 2.5537 | 3450 | 0.7309 | - | - |
| 2.5907 | 3500 | 0.7371 | 0.7235 | 0.9650 |
| 2.6277 | 3550 | 0.7318 | - | - |
| 2.6647 | 3600 | 0.7213 | - | - |
| 2.7017 | 3650 | 0.7066 | - | - |
| 2.7387 | 3700 | 0.7074 | - | - |
| 2.7757 | 3750 | 0.7072 | - | - |
| 2.8127 | 3800 | 0.6965 | - | - |
| 2.8497 | 3850 | 0.7158 | - | - |
| 2.8868 | 3900 | 0.7055 | - | - |
| 2.9238 | 3950 | 0.7028 | - | - |
| 2.9608 | 4000 | 0.6974 | 0.6875 | 0.9660 |
| 2.9978 | 4050 | 0.6916 | - | - |
| 3.0348 | 4100 | 0.6541 | - | - |
| 3.0718 | 4150 | 0.6413 | - | - |
| 3.1088 | 4200 | 0.6396 | - | - |
| 3.1458 | 4250 | 0.6463 | - | - |
| 3.1828 | 4300 | 0.6338 | - | - |
| 3.2198 | 4350 | 0.6310 | - | - |
| 3.2568 | 4400 | 0.6276 | - | - |
| 3.2939 | 4450 | 0.6183 | - | - |
| 3.3309 | 4500 | 0.6227 | 0.6574 | 0.9640 |
| 3.3679 | 4550 | 0.6204 | - | - |
| 3.4049 | 4600 | 0.6309 | - | - |
| 3.4419 | 4650 | 0.6304 | - | - |
| 3.4789 | 4700 | 0.6149 | - | - |
| 3.5159 | 4750 | 0.6114 | - | - |
| 3.5529 | 4800 | 0.6018 | - | - |
| 3.5899 | 4850 | 0.6260 | - | - |
| 3.6269 | 4900 | 0.6206 | - | - |
| 3.6640 | 4950 | 0.6125 | - | - |
| 3.7010 | 5000 | 0.6178 | 0.6358 | 0.9645 |
| 3.7380 | 5050 | 0.6171 | - | - |
| 3.7750 | 5100 | 0.5940 | - | - |
| 3.8120 | 5150 | 0.6041 | - | - |
| 3.8490 | 5200 | 0.5918 | - | - |
| 3.8860 | 5250 | 0.6052 | - | - |
| 3.9230 | 5300 | 0.5833 | - | - |
| 3.9600 | 5350 | 0.5919 | - | - |
| 3.9970 | 5400 | 0.5953 | - | - |
| 4.0340 | 5450 | 0.5496 | - | - |
| 4.0711 | 5500 | 0.5613 | 0.6063 | 0.9650 |
| 4.1081 | 5550 | 0.5507 | - | - |
| 4.1451 | 5600 | 0.5444 | - | - |
| 4.1821 | 5650 | 0.5467 | - | - |
| 4.2191 | 5700 | 0.5590 | - | - |
| 4.2561 | 5750 | 0.5520 | - | - |
| 4.2931 | 5800 | 0.5634 | - | - |
| 4.3301 | 5850 | 0.5425 | - | - |
| 4.3671 | 5900 | 0.5593 | - | - |
| 4.4041 | 5950 | 0.5281 | - | - |
| 4.4412 | 6000 | 0.5457 | 0.5993 | 0.9675 |
| 4.4782 | 6050 | 0.5426 | - | - |
| 4.5152 | 6100 | 0.5353 | - | - |
| 4.5522 | 6150 | 0.5282 | - | - |
| 4.5892 | 6200 | 0.5553 | - | - |
| 4.6262 | 6250 | 0.5390 | - | - |
| 4.6632 | 6300 | 0.5279 | - | - |
| 4.7002 | 6350 | 0.5438 | - | - |
| 4.7372 | 6400 | 0.5378 | - | - |
| 4.7742 | 6450 | 0.5395 | - | - |
| 4.8113 | 6500 | 0.5429 | 0.5766 | 0.9645 |
| 4.8483 | 6550 | 0.5333 | - | - |
| 4.8853 | 6600 | 0.5329 | - | - |
| 4.9223 | 6650 | 0.5273 | - | - |
| 4.9593 | 6700 | 0.5200 | - | - |
| 4.9963 | 6750 | 0.5304 | - | - |
| 5.0333 | 6800 | 0.4883 | - | - |
| 5.0703 | 6850 | 0.4996 | - | - |
| 5.1073 | 6900 | 0.4942 | - | - |
| 5.1443 | 6950 | 0.4987 | - | - |
| 5.1813 | 7000 | 0.4914 | 0.5701 | 0.9655 |
| 5.2184 | 7050 | 0.4953 | - | - |
| 5.2554 | 7100 | 0.4942 | - | - |
| 5.2924 | 7150 | 0.4792 | - | - |
| 5.3294 | 7200 | 0.4825 | - | - |
| 5.3664 | 7250 | 0.4894 | - | - |
| 5.4034 | 7300 | 0.4950 | - | - |
| 5.4404 | 7350 | 0.4933 | - | - |
| 5.4774 | 7400 | 0.4789 | - | - |
| 5.5144 | 7450 | 0.4901 | - | - |
| 5.5514 | 7500 | 0.4782 | 0.5646 | 0.9640 |
| 5.5885 | 7550 | 0.4881 | - | - |
| 5.6255 | 7600 | 0.4883 | - | - |
| 5.6625 | 7650 | 0.4712 | - | - |
| 5.6995 | 7700 | 0.4925 | - | - |
| 5.7365 | 7750 | 0.4812 | - | - |
| 5.7735 | 7800 | 0.4849 | - | - |
| 5.8105 | 7850 | 0.4895 | - | - |
| 5.8475 | 7900 | 0.4922 | - | - |
| 5.8845 | 7950 | 0.4830 | - | - |
| 5.9215 | 8000 | 0.4814 | 0.5485 | 0.9675 |
| 5.9585 | 8050 | 0.4705 | - | - |
| 5.9956 | 8100 | 0.4926 | - | - |
| 6.0326 | 8150 | 0.4516 | - | - |
| 6.0696 | 8200 | 0.4440 | - | - |
| 6.1066 | 8250 | 0.4471 | - | - |
| 6.1436 | 8300 | 0.4525 | - | - |
| 6.1806 | 8350 | 0.4415 | - | - |
| 6.2176 | 8400 | 0.4440 | - | - |
| 6.2546 | 8450 | 0.4438 | - | - |
| 6.2916 | 8500 | 0.4527 | 0.5425 | 0.9655 |
| 6.3286 | 8550 | 0.4530 | - | - |
| 6.3657 | 8600 | 0.4457 | - | - |
| 6.4027 | 8650 | 0.4347 | - | - |
| 6.4397 | 8700 | 0.4426 | - | - |
| 6.4767 | 8750 | 0.4549 | - | - |
| 6.5137 | 8800 | 0.4475 | - | - |
| 6.5507 | 8850 | 0.4416 | - | - |
| 6.5877 | 8900 | 0.4426 | - | - |
| 6.6247 | 8950 | 0.4496 | - | - |
| 6.6617 | 9000 | 0.4473 | 0.5347 | 0.9655 |
| 6.6987 | 9050 | 0.4571 | - | - |
| 6.7358 | 9100 | 0.4449 | - | - |
| 6.7728 | 9150 | 0.4440 | - | - |
| 6.8098 | 9200 | 0.4438 | - | - |
| 6.8468 | 9250 | 0.4498 | - | - |
| 6.8838 | 9300 | 0.4387 | - | - |
| 6.9208 | 9350 | 0.4459 | - | - |
| 6.9578 | 9400 | 0.4389 | - | - |
| 6.9948 | 9450 | 0.4489 | - | - |
| 7.0318 | 9500 | 0.4162 | 0.5264 | 0.9670 |
| 7.0688 | 9550 | 0.4080 | - | - |
| 7.1058 | 9600 | 0.4086 | - | - |
| 7.1429 | 9650 | 0.4139 | - | - |
| 7.1799 | 9700 | 0.4108 | - | - |
| 7.2169 | 9750 | 0.4104 | - | - |
| 7.2539 | 9800 | 0.4087 | - | - |
| 7.2909 | 9850 | 0.4200 | - | - |
| 7.3279 | 9900 | 0.4085 | - | - |
| 7.3649 | 9950 | 0.3993 | - | - |
| 7.4019 | 10000 | 0.4196 | 0.5217 | 0.9665 |
| 7.4389 | 10050 | 0.4242 | - | - |
| 7.4759 | 10100 | 0.4132 | - | - |
| 7.5130 | 10150 | 0.4064 | - | - |
| 7.5500 | 10200 | 0.4182 | - | - |
| 7.5870 | 10250 | 0.4140 | - | - |
| 7.6240 | 10300 | 0.4266 | - | - |
| 7.6610 | 10350 | 0.4160 | - | - |
| 7.6980 | 10400 | 0.4192 | - | - |
| 7.7350 | 10450 | 0.4140 | - | - |
| 7.7720 | 10500 | 0.4216 | 0.5198 | 0.9670 |
| 7.8090 | 10550 | 0.4114 | - | - |
| 7.8460 | 10600 | 0.4192 | - | - |
| 7.8830 | 10650 | 0.4032 | - | - |
| 7.9201 | 10700 | 0.4100 | - | - |
| 7.9571 | 10750 | 0.4123 | - | - |
| 7.9941 | 10800 | 0.4018 | - | - |
| 8.0311 | 10850 | 0.3920 | - | - |
| 8.0681 | 10900 | 0.3777 | - | - |
| 8.1051 | 10950 | 0.3943 | - | - |
| 8.1421 | 11000 | 0.3795 | 0.5113 | 0.9665 |
| 8.1791 | 11050 | 0.3890 | - | - |
| 8.2161 | 11100 | 0.3941 | - | - |
| 8.2531 | 11150 | 0.3896 | - | - |
| 8.2902 | 11200 | 0.3861 | - | - |
| 8.3272 | 11250 | 0.3837 | - | - |
| 8.3642 | 11300 | 0.3879 | - | - |
| 8.4012 | 11350 | 0.3901 | - | - |
| 8.4382 | 11400 | 0.3815 | - | - |
| 8.4752 | 11450 | 0.3909 | - | - |
| 8.5122 | 11500 | 0.3780 | 0.5074 | 0.9670 |
| 8.5492 | 11550 | 0.3905 | - | - |
| 8.5862 | 11600 | 0.3900 | - | - |
| 8.6232 | 11650 | 0.3854 | - | - |
| 8.6603 | 11700 | 0.3873 | - | - |
| 8.6973 | 11750 | 0.3825 | - | - |
| 8.7343 | 11800 | 0.3884 | - | - |
| 8.7713 | 11850 | 0.3996 | - | - |
| 8.8083 | 11900 | 0.3847 | - | - |
| 8.8453 | 11950 | 0.3811 | - | - |
| 8.8823 | 12000 | 0.3716 | 0.5050 | 0.9655 |
| 8.9193 | 12050 | 0.3829 | - | - |
| 8.9563 | 12100 | 0.3823 | - | - |
| 8.9933 | 12150 | 0.3884 | - | - |
| 9.0303 | 12200 | 0.3662 | - | - |
| 9.0674 | 12250 | 0.3702 | - | - |
| 9.1044 | 12300 | 0.3541 | - | - |
| 9.1414 | 12350 | 0.3674 | - | - |
| 9.1784 | 12400 | 0.3639 | - | - |
| 9.2154 | 12450 | 0.3817 | - | - |
| 9.2524 | 12500 | 0.3649 | 0.5002 | 0.9660 |
| 9.2894 | 12550 | 0.3733 | - | - |
| 9.3264 | 12600 | 0.3744 | - | - |
| 9.3634 | 12650 | 0.3719 | - | - |
| 9.4004 | 12700 | 0.3587 | - | - |
| 9.4375 | 12750 | 0.3685 | - | - |
| 9.4745 | 12800 | 0.3621 | - | - |
| 9.5115 | 12850 | 0.3699 | - | - |
| 9.5485 | 12900 | 0.3674 | - | - |
| 9.5855 | 12950 | 0.3607 | - | - |
| 9.6225 | 13000 | 0.3623 | 0.4984 | 0.9665 |
| 9.6595 | 13050 | 0.3675 | - | - |
| 9.6965 | 13100 | 0.3667 | - | - |
| 9.7335 | 13150 | 0.3612 | - | - |
| 9.7705 | 13200 | 0.3644 | - | - |
| 9.8075 | 13250 | 0.3673 | - | - |
| 9.8446 | 13300 | 0.3653 | - | - |
| 9.8816 | 13350 | 0.3665 | - | - |
| 9.9186 | 13400 | 0.3603 | - | - |
| 9.9556 | 13450 | 0.3536 | - | - |
| 9.9926 | 13500 | 0.3704 | 0.4973 | 0.9655 |
| 10.0 | 13510 | - | 0.4974 | 0.9660 |
| -1 | -1 | - | - | 0.9660 |
Training Time
- Training: 54.8 minutes
Framework Versions
- Python: 3.12.0
- Sentence Transformers: 6.1.0
- Transformers: 5.20.1
- PyTorch: 2.14.1+cu130
- Accelerate: 1.15.0
- Datasets: 5.1.0
- Tokenizers: 0.23.3
Additional Resources
- Training and Finetuning Embedding Models with Sentence Transformers: the end-to-end guide for training or finetuning Sentence Transformer models.
- Introduction to Matryoshka Embedding Models: variable-size embeddings that can be truncated with minimal quality loss.
- Binary and Scalar Embedding Quantization for Significantly Faster & Cheaper Retrieval: post-training compression of embedding vectors.
- Multimodal Embedding & Reranker Models with Sentence Transformers: use text, image, audio, and video models through the same API.
- Training and Finetuning Multimodal Embedding & Reranker Models with Sentence Transformers: train multimodal embedding models, with a Visual Document Retrieval walkthrough.
Citation
BibTeX
Sentence Transformers
@inproceedings{reimers-2019-sentence-bert,
title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
author = "Reimers, Nils and Gurevych, Iryna",
booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
month = "11",
year = "2019",
publisher = "Association for Computational Linguistics",
url = "https://arxiv.org/abs/1908.10084",
}
DenoisingAutoEncoderLoss
@inproceedings{wang-2021-TSDAE,
title = "TSDAE: Using Transformer-based Sequential Denoising Auto-Encoderfor Unsupervised Sentence Embedding Learning",
author = "Wang, Kexin and Reimers, Nils and Gurevych, Iryna",
booktitle = "Findings of the Association for Computational Linguistics: EMNLP 2021",
month = nov,
year = "2021",
address = "Punta Cana, Dominican Republic",
publisher = "Association for Computational Linguistics",
pages = "671--688",
url = "https://arxiv.org/abs/2104.06979",
}