--- tags: - sentence-transformers - sentence-similarity - feature-extraction - dense - generated_from_trainer - dataset_size:86450 - loss:DenoisingAutoEncoderLoss base_model: google-bert/bert-base-uncased widget: - source_sentence: mini sentences: - mini contryman e electric 1 - tata safari xm diesel 1956 - daewoo cielo glx petrol 1498 - source_sentence: '1462' sentences: - maruti alto 800 lxi cng 796 - toyota rumion g at petrol 1462 - honda city v mt petrol 1498 - source_sentence: go.t cng sentences: - datsun go plus .t cng 1198 - bmw m5 standard petrol 4999 - volvo xc60 d4 diesel 1984 - source_sentence: tesla y l sentences: - tesla model y l premium awd - toyota townace townace petrol 2200 - maruti 1000 ac petrol 970 - source_sentence: subaru - sentences: - subaru impreza - petrol 2000 - honda br - v s diesel 1498 - tata indica gls cng 1193 pipeline_tag: sentence-similarity library_name: sentence-transformers metrics: - cosine_accuracy model-index: - name: SentenceTransformer based on google-bert/bert-base-uncased results: - task: type: triplet name: Triplet dataset: name: fw variant dev type: fw-variant-dev metrics: - type: cosine_accuracy value: 0.9660000205039978 name: Cosine Accuracy --- # SentenceTransformer based on google-bert/bert-base-uncased This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [google-bert/bert-base-uncased](https://huggingface.co/google-bert/bert-base-uncased). It maps inputs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, classification, clustering, and more. ## Model Details ### Model Description - **Model Type:** Sentence Transformer - **Base model:** [google-bert/bert-base-uncased](https://huggingface.co/google-bert/bert-base-uncased) - **Maximum Sequence Length:** 32 tokens - **Output Dimensionality:** 768 dimensions - **Similarity Function:** Cosine Similarity - **Supported Modality:** Text ### Model Sources - **Documentation:** [Sentence Transformers Documentation](https://sbert.net) - **Repository:** [Sentence Transformers on GitHub](https://github.com/huggingface/sentence-transformers) - **Hugging Face:** [Sentence Transformers on Hugging Face](https://huggingface.co/models?library=sentence-transformers) ### Full Model Architecture ``` SentenceTransformer( (0): Transformer({'transformer_task': 'feature-extraction', 'modality_config': {'text': {'method': 'forward', 'method_output_name': 'last_hidden_state'}}, 'module_output_name': 'token_embeddings', 'architecture': 'BertModel'}) (1): Pooling({'embedding_dimension': 768, 'pooling_mode': 'cls', 'include_prompt': True}) ) ``` ## Usage ### Direct Usage (Sentence Transformers) First install the Sentence Transformers library: ```bash pip install -U sentence-transformers ``` Then you can load this model and run inference. ```python from sentence_transformers import SentenceTransformer # Download from the 🤗 Hub model = SentenceTransformer("SwarajSolanke-turtle/bert-base-uncased-tsdae-fw") # Run inference sentences = [ 'subaru -', 'subaru impreza - petrol 2000', 'honda br - v s diesel 1498', ] embeddings = model.encode(sentences) print(embeddings.shape) # [3, 768] # Get the similarity scores for the embeddings similarities = model.similarity(embeddings, embeddings) print(similarities) # tensor([[1.0000, 0.9939, 0.9908], # [0.9939, 1.0000, 0.9881], # [0.9908, 0.9881, 1.0000]]) ``` ## Evaluation ### Metrics #### Triplet * Dataset: `fw-variant-dev` * Evaluated with [TripletEvaluator](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.sentence_transformer.evaluation.TripletEvaluator) | Metric | Value | |:--------------------|:----------| | **cosine_accuracy** | **0.966** | ## Training Details ### Training Dataset #### Unnamed Dataset * Size: 86,450 training samples * Columns: damaged and original * Approximate statistics based on the first 100 samples: | | damaged | original | |:---------|:----------------------------------------------------------------------------------|:----------------------------------------------------------------------------------| | type | string | string | | modality | text | text | | details | | | * Samples: | damaged | original | |:-----------------------------------------------------------------|:---------------------------------------------------------------------| | 1.5 petrol 1482 | hyundai alcazar 1.5 executive mt petrol 1482 | | mahindra. 300 1.5 w8 tone diesel 1497 | mahindra. xuv 300 1.5 w8 dual tone diesel 1497 | | mahindra scorpio vlx mhawk bs iv airbag diesel 2179 | mahindra scorpio vlx 2.2 mhawk bs iv airbag diesel 2179 | * Loss: [DenoisingAutoEncoderLoss](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#denoisingautoencoderloss) with these parameters: ```json { "decoder_name_or_path": "google-bert/bert-base-uncased", "need_retokenization": false } ``` ### Evaluation Dataset #### Unnamed Dataset * Size: 1,765 evaluation samples * Columns: damaged and original * Approximate statistics based on the first 100 samples: | | damaged | original | |:---------|:----------------------------------------------------------------------------------|:---------------------------------------------------------------------------------| | type | string | string | | modality | text | text | | details | | | * Samples: | damaged | original | |:-------------------------------------------------------------------|:-------------------------------------------------------------------------------| | bentley continental gt 6.0 speed convertible 4 petrol | bentley continental gt 6.0 speed convertible 4 seater petrol 5998 | | tata 1.2 rtn bs-vi 1199 | tata altroz xt 1.2 rtn bs-vi petrol 1199 | | maruti s-cross hybrid diesel 1248 | maruti s-cross hybrid diesel 1248 | * Loss: [DenoisingAutoEncoderLoss](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#denoisingautoencoderloss) with these parameters: ```json { "decoder_name_or_path": "google-bert/bert-base-uncased", "need_retokenization": false } ``` ### Training Hyperparameters #### Non-Default Hyperparameters - `per_device_train_batch_size`: 64 - `num_train_epochs`: 10 - `learning_rate`: 1e-05 - `warmup_steps`: 0.1 - `bf16`: True - `per_device_eval_batch_size`: 64 - `seed`: 46 #### All Hyperparameters
Click to expand - `per_device_train_batch_size`: 64 - `num_train_epochs`: 10 - `max_steps`: -1 - `learning_rate`: 1e-05 - `lr_scheduler_type`: linear - `lr_scheduler_kwargs`: None - `warmup_steps`: 0.1 - `optim`: adamw_torch_fused - `optim_args`: None - `weight_decay`: 0.0 - `adam_beta1`: 0.9 - `adam_beta2`: 0.999 - `adam_epsilon`: 1e-08 - `optim_target_modules`: None - `gradient_accumulation_steps`: 1 - `average_tokens_across_devices`: True - `max_grad_norm`: 1.0 - `label_smoothing_factor`: 0.0 - `bf16`: True - `fp16`: False - `bf16_full_eval`: False - `fp16_full_eval`: False - `tf32`: None - `gradient_checkpointing`: False - `gradient_checkpointing_kwargs`: None - `torch_compile`: False - `torch_compile_backend`: None - `torch_compile_mode`: None - `use_liger_kernel`: False - `liger_kernel_config`: None - `use_cache`: False - `neftune_noise_alpha`: None - `torch_empty_cache_steps`: None - `auto_find_batch_size`: False - `log_on_each_node`: True - `logging_nan_inf_filter`: True - `include_num_input_tokens_seen`: no - `log_level`: passive - `log_level_replica`: warning - `disable_tqdm`: False - `project`: huggingface - `trackio_space_id`: None - `trackio_bucket_id`: None - `trackio_static_space_id`: None - `per_device_eval_batch_size`: 64 - `prediction_loss_only`: True - `eval_on_start`: False - `eval_do_concat_batches`: True - `eval_use_gather_object`: False - `eval_accumulation_steps`: None - `include_for_metrics`: [] - `batch_eval_metrics`: False - `save_only_model`: False - `save_on_each_node`: False - `enable_jit_checkpoint`: False - `push_to_hub`: False - `hub_private_repo`: None - `hub_model_id`: None - `hub_strategy`: every_save - `hub_always_push`: False - `hub_revision`: None - `load_best_model_at_end`: False - `ignore_data_skip`: False - `restore_callback_states_from_checkpoint`: False - `full_determinism`: False - `seed`: 46 - `data_seed`: None - `use_cpu`: False - `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None} - `parallelism_config`: None - `dataloader_drop_last`: False - `dataloader_num_workers`: 0 - `dataloader_pin_memory`: True - `dataloader_persistent_workers`: False - `dataloader_prefetch_factor`: None - `dataloader_multiprocessing_context`: None - `dataloader_in_order`: True - `remove_unused_columns`: True - `label_names`: None - `train_sampling_strategy`: random - `length_column_name`: length - `ddp_find_unused_parameters`: None - `ddp_bucket_cap_mb`: None - `ddp_broadcast_buffers`: False - `ddp_static_graph`: None - `ddp_backend`: None - `ddp_timeout`: 1800 - `fsdp`: None - `fsdp_config`: None - `deepspeed`: None - `debug`: [] - `skip_memory_metrics`: True - `do_predict`: False - `resume_from_checkpoint`: None - `local_rank`: -1 - `prompts`: None - `batch_sampler`: batch_sampler - `multi_dataset_batch_sampler`: proportional - `router_mapping`: {} - `learning_rate_mapping`: {} - `warmup_ratio`: None
### Training Logs
Click to expand | Epoch | Step | Training Loss | Validation Loss | fw-variant-dev_cosine_accuracy | |:------:|:-----:|:-------------:|:---------------:|:------------------------------:| | -1 | -1 | - | - | 0.8845 | | 0.0370 | 50 | 10.9495 | - | - | | 0.0740 | 100 | 8.5518 | - | - | | 0.1110 | 150 | 7.2139 | - | - | | 0.1480 | 200 | 6.5304 | - | - | | 0.1850 | 250 | 5.8942 | - | - | | 0.2221 | 300 | 5.0836 | - | - | | 0.2591 | 350 | 4.4827 | - | - | | 0.2961 | 400 | 4.0965 | - | - | | 0.3331 | 450 | 3.7756 | - | - | | 0.3701 | 500 | 3.5197 | 3.3155 | 0.8885 | | 0.4071 | 550 | 3.2203 | - | - | | 0.4441 | 600 | 3.0355 | - | - | | 0.4811 | 650 | 2.8250 | - | - | | 0.5181 | 700 | 2.6778 | - | - | | 0.5551 | 750 | 2.4930 | - | - | | 0.5922 | 800 | 2.3938 | - | - | | 0.6292 | 850 | 2.2241 | - | - | | 0.6662 | 900 | 2.1228 | - | - | | 0.7032 | 950 | 2.0260 | - | - | | 0.7402 | 1000 | 1.9064 | 1.8352 | 0.9680 | | 0.7772 | 1050 | 1.8245 | - | - | | 0.8142 | 1100 | 1.7392 | - | - | | 0.8512 | 1150 | 1.6638 | - | - | | 0.8882 | 1200 | 1.6003 | - | - | | 0.9252 | 1250 | 1.5114 | - | - | | 0.9623 | 1300 | 1.4978 | - | - | | 0.9993 | 1350 | 1.4336 | - | - | | 1.0363 | 1400 | 1.3469 | - | - | | 1.0733 | 1450 | 1.3045 | - | - | | 1.1103 | 1500 | 1.2647 | 1.2372 | 0.9670 | | 1.1473 | 1550 | 1.2286 | - | - | | 1.1843 | 1600 | 1.2249 | - | - | | 1.2213 | 1650 | 1.1645 | - | - | | 1.2583 | 1700 | 1.1292 | - | - | | 1.2953 | 1750 | 1.1289 | - | - | | 1.3323 | 1800 | 1.0791 | - | - | | 1.3694 | 1850 | 1.1114 | - | - | | 1.4064 | 1900 | 1.0625 | - | - | | 1.4434 | 1950 | 1.0366 | - | - | | 1.4804 | 2000 | 1.0352 | 0.9793 | 0.9680 | | 1.5174 | 2050 | 1.0102 | - | - | | 1.5544 | 2100 | 0.9745 | - | - | | 1.5914 | 2150 | 0.9928 | - | - | | 1.6284 | 2200 | 0.9608 | - | - | | 1.6654 | 2250 | 0.9365 | - | - | | 1.7024 | 2300 | 0.9378 | - | - | | 1.7395 | 2350 | 0.9319 | - | - | | 1.7765 | 2400 | 0.8982 | - | - | | 1.8135 | 2450 | 0.8977 | - | - | | 1.8505 | 2500 | 0.9045 | 0.8560 | 0.9635 | | 1.8875 | 2550 | 0.9055 | - | - | | 1.9245 | 2600 | 0.8841 | - | - | | 1.9615 | 2650 | 0.8696 | - | - | | 1.9985 | 2700 | 0.8483 | - | - | | 2.0355 | 2750 | 0.8080 | - | - | | 2.0725 | 2800 | 0.8050 | - | - | | 2.1095 | 2850 | 0.7986 | - | - | | 2.1466 | 2900 | 0.7870 | - | - | | 2.1836 | 2950 | 0.7727 | - | - | | 2.2206 | 3000 | 0.7667 | 0.7722 | 0.9645 | | 2.2576 | 3050 | 0.7718 | - | - | | 2.2946 | 3100 | 0.7570 | - | - | | 2.3316 | 3150 | 0.7626 | - | - | | 2.3686 | 3200 | 0.7846 | - | - | | 2.4056 | 3250 | 0.7362 | - | - | | 2.4426 | 3300 | 0.7447 | - | - | | 2.4796 | 3350 | 0.7596 | - | - | | 2.5167 | 3400 | 0.7430 | - | - | | 2.5537 | 3450 | 0.7309 | - | - | | 2.5907 | 3500 | 0.7371 | 0.7235 | 0.9650 | | 2.6277 | 3550 | 0.7318 | - | - | | 2.6647 | 3600 | 0.7213 | - | - | | 2.7017 | 3650 | 0.7066 | - | - | | 2.7387 | 3700 | 0.7074 | - | - | | 2.7757 | 3750 | 0.7072 | - | - | | 2.8127 | 3800 | 0.6965 | - | - | | 2.8497 | 3850 | 0.7158 | - | - | | 2.8868 | 3900 | 0.7055 | - | - | | 2.9238 | 3950 | 0.7028 | - | - | | 2.9608 | 4000 | 0.6974 | 0.6875 | 0.9660 | | 2.9978 | 4050 | 0.6916 | - | - | | 3.0348 | 4100 | 0.6541 | - | - | | 3.0718 | 4150 | 0.6413 | - | - | | 3.1088 | 4200 | 0.6396 | - | - | | 3.1458 | 4250 | 0.6463 | - | - | | 3.1828 | 4300 | 0.6338 | - | - | | 3.2198 | 4350 | 0.6310 | - | - | | 3.2568 | 4400 | 0.6276 | - | - | | 3.2939 | 4450 | 0.6183 | - | - | | 3.3309 | 4500 | 0.6227 | 0.6574 | 0.9640 | | 3.3679 | 4550 | 0.6204 | - | - | | 3.4049 | 4600 | 0.6309 | - | - | | 3.4419 | 4650 | 0.6304 | - | - | | 3.4789 | 4700 | 0.6149 | - | - | | 3.5159 | 4750 | 0.6114 | - | - | | 3.5529 | 4800 | 0.6018 | - | - | | 3.5899 | 4850 | 0.6260 | - | - | | 3.6269 | 4900 | 0.6206 | - | - | | 3.6640 | 4950 | 0.6125 | - | - | | 3.7010 | 5000 | 0.6178 | 0.6358 | 0.9645 | | 3.7380 | 5050 | 0.6171 | - | - | | 3.7750 | 5100 | 0.5940 | - | - | | 3.8120 | 5150 | 0.6041 | - | - | | 3.8490 | 5200 | 0.5918 | - | - | | 3.8860 | 5250 | 0.6052 | - | - | | 3.9230 | 5300 | 0.5833 | - | - | | 3.9600 | 5350 | 0.5919 | - | - | | 3.9970 | 5400 | 0.5953 | - | - | | 4.0340 | 5450 | 0.5496 | - | - | | 4.0711 | 5500 | 0.5613 | 0.6063 | 0.9650 | | 4.1081 | 5550 | 0.5507 | - | - | | 4.1451 | 5600 | 0.5444 | - | - | | 4.1821 | 5650 | 0.5467 | - | - | | 4.2191 | 5700 | 0.5590 | - | - | | 4.2561 | 5750 | 0.5520 | - | - | | 4.2931 | 5800 | 0.5634 | - | - | | 4.3301 | 5850 | 0.5425 | - | - | | 4.3671 | 5900 | 0.5593 | - | - | | 4.4041 | 5950 | 0.5281 | - | - | | 4.4412 | 6000 | 0.5457 | 0.5993 | 0.9675 | | 4.4782 | 6050 | 0.5426 | - | - | | 4.5152 | 6100 | 0.5353 | - | - | | 4.5522 | 6150 | 0.5282 | - | - | | 4.5892 | 6200 | 0.5553 | - | - | | 4.6262 | 6250 | 0.5390 | - | - | | 4.6632 | 6300 | 0.5279 | - | - | | 4.7002 | 6350 | 0.5438 | - | - | | 4.7372 | 6400 | 0.5378 | - | - | | 4.7742 | 6450 | 0.5395 | - | - | | 4.8113 | 6500 | 0.5429 | 0.5766 | 0.9645 | | 4.8483 | 6550 | 0.5333 | - | - | | 4.8853 | 6600 | 0.5329 | - | - | | 4.9223 | 6650 | 0.5273 | - | - | | 4.9593 | 6700 | 0.5200 | - | - | | 4.9963 | 6750 | 0.5304 | - | - | | 5.0333 | 6800 | 0.4883 | - | - | | 5.0703 | 6850 | 0.4996 | - | - | | 5.1073 | 6900 | 0.4942 | - | - | | 5.1443 | 6950 | 0.4987 | - | - | | 5.1813 | 7000 | 0.4914 | 0.5701 | 0.9655 | | 5.2184 | 7050 | 0.4953 | - | - | | 5.2554 | 7100 | 0.4942 | - | - | | 5.2924 | 7150 | 0.4792 | - | - | | 5.3294 | 7200 | 0.4825 | - | - | | 5.3664 | 7250 | 0.4894 | - | - | | 5.4034 | 7300 | 0.4950 | - | - | | 5.4404 | 7350 | 0.4933 | - | - | | 5.4774 | 7400 | 0.4789 | - | - | | 5.5144 | 7450 | 0.4901 | - | - | | 5.5514 | 7500 | 0.4782 | 0.5646 | 0.9640 | | 5.5885 | 7550 | 0.4881 | - | - | | 5.6255 | 7600 | 0.4883 | - | - | | 5.6625 | 7650 | 0.4712 | - | - | | 5.6995 | 7700 | 0.4925 | - | - | | 5.7365 | 7750 | 0.4812 | - | - | | 5.7735 | 7800 | 0.4849 | - | - | | 5.8105 | 7850 | 0.4895 | - | - | | 5.8475 | 7900 | 0.4922 | - | - | | 5.8845 | 7950 | 0.4830 | - | - | | 5.9215 | 8000 | 0.4814 | 0.5485 | 0.9675 | | 5.9585 | 8050 | 0.4705 | - | - | | 5.9956 | 8100 | 0.4926 | - | - | | 6.0326 | 8150 | 0.4516 | - | - | | 6.0696 | 8200 | 0.4440 | - | - | | 6.1066 | 8250 | 0.4471 | - | - | | 6.1436 | 8300 | 0.4525 | - | - | | 6.1806 | 8350 | 0.4415 | - | - | | 6.2176 | 8400 | 0.4440 | - | - | | 6.2546 | 8450 | 0.4438 | - | - | | 6.2916 | 8500 | 0.4527 | 0.5425 | 0.9655 | | 6.3286 | 8550 | 0.4530 | - | - | | 6.3657 | 8600 | 0.4457 | - | - | | 6.4027 | 8650 | 0.4347 | - | - | | 6.4397 | 8700 | 0.4426 | - | - | | 6.4767 | 8750 | 0.4549 | - | - | | 6.5137 | 8800 | 0.4475 | - | - | | 6.5507 | 8850 | 0.4416 | - | - | | 6.5877 | 8900 | 0.4426 | - | - | | 6.6247 | 8950 | 0.4496 | - | - | | 6.6617 | 9000 | 0.4473 | 0.5347 | 0.9655 | | 6.6987 | 9050 | 0.4571 | - | - | | 6.7358 | 9100 | 0.4449 | - | - | | 6.7728 | 9150 | 0.4440 | - | - | | 6.8098 | 9200 | 0.4438 | - | - | | 6.8468 | 9250 | 0.4498 | - | - | | 6.8838 | 9300 | 0.4387 | - | - | | 6.9208 | 9350 | 0.4459 | - | - | | 6.9578 | 9400 | 0.4389 | - | - | | 6.9948 | 9450 | 0.4489 | - | - | | 7.0318 | 9500 | 0.4162 | 0.5264 | 0.9670 | | 7.0688 | 9550 | 0.4080 | - | - | | 7.1058 | 9600 | 0.4086 | - | - | | 7.1429 | 9650 | 0.4139 | - | - | | 7.1799 | 9700 | 0.4108 | - | - | | 7.2169 | 9750 | 0.4104 | - | - | | 7.2539 | 9800 | 0.4087 | - | - | | 7.2909 | 9850 | 0.4200 | - | - | | 7.3279 | 9900 | 0.4085 | - | - | | 7.3649 | 9950 | 0.3993 | - | - | | 7.4019 | 10000 | 0.4196 | 0.5217 | 0.9665 | | 7.4389 | 10050 | 0.4242 | - | - | | 7.4759 | 10100 | 0.4132 | - | - | | 7.5130 | 10150 | 0.4064 | - | - | | 7.5500 | 10200 | 0.4182 | - | - | | 7.5870 | 10250 | 0.4140 | - | - | | 7.6240 | 10300 | 0.4266 | - | - | | 7.6610 | 10350 | 0.4160 | - | - | | 7.6980 | 10400 | 0.4192 | - | - | | 7.7350 | 10450 | 0.4140 | - | - | | 7.7720 | 10500 | 0.4216 | 0.5198 | 0.9670 | | 7.8090 | 10550 | 0.4114 | - | - | | 7.8460 | 10600 | 0.4192 | - | - | | 7.8830 | 10650 | 0.4032 | - | - | | 7.9201 | 10700 | 0.4100 | - | - | | 7.9571 | 10750 | 0.4123 | - | - | | 7.9941 | 10800 | 0.4018 | - | - | | 8.0311 | 10850 | 0.3920 | - | - | | 8.0681 | 10900 | 0.3777 | - | - | | 8.1051 | 10950 | 0.3943 | - | - | | 8.1421 | 11000 | 0.3795 | 0.5113 | 0.9665 | | 8.1791 | 11050 | 0.3890 | - | - | | 8.2161 | 11100 | 0.3941 | - | - | | 8.2531 | 11150 | 0.3896 | - | - | | 8.2902 | 11200 | 0.3861 | - | - | | 8.3272 | 11250 | 0.3837 | - | - | | 8.3642 | 11300 | 0.3879 | - | - | | 8.4012 | 11350 | 0.3901 | - | - | | 8.4382 | 11400 | 0.3815 | - | - | | 8.4752 | 11450 | 0.3909 | - | - | | 8.5122 | 11500 | 0.3780 | 0.5074 | 0.9670 | | 8.5492 | 11550 | 0.3905 | - | - | | 8.5862 | 11600 | 0.3900 | - | - | | 8.6232 | 11650 | 0.3854 | - | - | | 8.6603 | 11700 | 0.3873 | - | - | | 8.6973 | 11750 | 0.3825 | - | - | | 8.7343 | 11800 | 0.3884 | - | - | | 8.7713 | 11850 | 0.3996 | - | - | | 8.8083 | 11900 | 0.3847 | - | - | | 8.8453 | 11950 | 0.3811 | - | - | | 8.8823 | 12000 | 0.3716 | 0.5050 | 0.9655 | | 8.9193 | 12050 | 0.3829 | - | - | | 8.9563 | 12100 | 0.3823 | - | - | | 8.9933 | 12150 | 0.3884 | - | - | | 9.0303 | 12200 | 0.3662 | - | - | | 9.0674 | 12250 | 0.3702 | - | - | | 9.1044 | 12300 | 0.3541 | - | - | | 9.1414 | 12350 | 0.3674 | - | - | | 9.1784 | 12400 | 0.3639 | - | - | | 9.2154 | 12450 | 0.3817 | - | - | | 9.2524 | 12500 | 0.3649 | 0.5002 | 0.9660 | | 9.2894 | 12550 | 0.3733 | - | - | | 9.3264 | 12600 | 0.3744 | - | - | | 9.3634 | 12650 | 0.3719 | - | - | | 9.4004 | 12700 | 0.3587 | - | - | | 9.4375 | 12750 | 0.3685 | - | - | | 9.4745 | 12800 | 0.3621 | - | - | | 9.5115 | 12850 | 0.3699 | - | - | | 9.5485 | 12900 | 0.3674 | - | - | | 9.5855 | 12950 | 0.3607 | - | - | | 9.6225 | 13000 | 0.3623 | 0.4984 | 0.9665 | | 9.6595 | 13050 | 0.3675 | - | - | | 9.6965 | 13100 | 0.3667 | - | - | | 9.7335 | 13150 | 0.3612 | - | - | | 9.7705 | 13200 | 0.3644 | - | - | | 9.8075 | 13250 | 0.3673 | - | - | | 9.8446 | 13300 | 0.3653 | - | - | | 9.8816 | 13350 | 0.3665 | - | - | | 9.9186 | 13400 | 0.3603 | - | - | | 9.9556 | 13450 | 0.3536 | - | - | | 9.9926 | 13500 | 0.3704 | 0.4973 | 0.9655 | | 10.0 | 13510 | - | 0.4974 | 0.9660 | | -1 | -1 | - | - | 0.9660 |
### Training Time - **Training**: 54.8 minutes ### Framework Versions - Python: 3.12.0 - Sentence Transformers: 6.1.0 - Transformers: 5.20.1 - PyTorch: 2.14.1+cu130 - Accelerate: 1.15.0 - Datasets: 5.1.0 - Tokenizers: 0.23.3 ## Additional Resources - [Training and Finetuning Embedding Models with Sentence Transformers](https://huggingface.co/blog/train-sentence-transformers): the end-to-end guide for training or finetuning Sentence Transformer models. - [Introduction to Matryoshka Embedding Models](https://huggingface.co/blog/matryoshka): variable-size embeddings that can be truncated with minimal quality loss. - [Binary and Scalar Embedding Quantization for Significantly Faster & Cheaper Retrieval](https://huggingface.co/blog/embedding-quantization): post-training compression of embedding vectors. - [Multimodal Embedding & Reranker Models with Sentence Transformers](https://huggingface.co/blog/multimodal-sentence-transformers): use text, image, audio, and video models through the same API. - [Training and Finetuning Multimodal Embedding & Reranker Models with Sentence Transformers](https://huggingface.co/blog/train-multimodal-sentence-transformers): train multimodal embedding models, with a Visual Document Retrieval walkthrough. ## Citation ### BibTeX #### Sentence Transformers ```bibtex @inproceedings{reimers-2019-sentence-bert, title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks", author = "Reimers, Nils and Gurevych, Iryna", booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing", month = "11", year = "2019", publisher = "Association for Computational Linguistics", url = "https://arxiv.org/abs/1908.10084", } ``` #### DenoisingAutoEncoderLoss ```bibtex @inproceedings{wang-2021-TSDAE, title = "TSDAE: Using Transformer-based Sequential Denoising Auto-Encoderfor Unsupervised Sentence Embedding Learning", author = "Wang, Kexin and Reimers, Nils and Gurevych, Iryna", booktitle = "Findings of the Association for Computational Linguistics: EMNLP 2021", month = nov, year = "2021", address = "Punta Cana, Dominican Republic", publisher = "Association for Computational Linguistics", pages = "671--688", url = "https://arxiv.org/abs/2104.06979", } ```