--- tags: - sentence-transformers - sentence-similarity - feature-extraction - dense - generated_from_trainer - dataset_size:8089600 - loss:MultipleNegativesRankingLoss base_model: abhinand/MedEmbed-small-v0.1 widget: - source_sentence: hypoxic ischaemic encephalopathy in diseases classified elsewhere sentences: - other specified hypoxic-ischaemic encephalopathy - monocytopenia - isolated benign cerebral vasculitis - source_sentence: congenital laryngomalacia sentences: - floppy epiglottis - fructose-1,6-bisphosphatase deficiency - structural developmental anomalies of the nose or cavum, unspecified - source_sentence: molluscum contagiosum sentences: - laceration of bronchus nos - water warts - inflammation of bartholin gland - source_sentence: contusion of heart without haemopericardium sentences: - contusion of heart - phlebotomus fever - rectovaginal fistula - source_sentence: multiple delivery by combination of methods sentences: - rectal spasm - multiple delivery by combination of methods without caesarean - bipolar or related disorders, unspecified pipeline_tag: sentence-similarity library_name: sentence-transformers metrics: - cosine_accuracy@1 - cosine_accuracy@5 - cosine_accuracy@10 - cosine_precision@1 - cosine_precision@3 - cosine_precision@5 - cosine_precision@10 - cosine_recall@1 - cosine_recall@3 - cosine_recall@5 - cosine_recall@10 - cosine_ndcg@10 - cosine_mrr@10 - cosine_map@10 model-index: - name: SentenceTransformer based on abhinand/MedEmbed-small-v0.1 results: - task: type: information-retrieval name: Information Retrieval dataset: name: icd11 val type: icd11-val metrics: - type: cosine_accuracy@1 value: 0.7649122807017544 name: Cosine Accuracy@1 - type: cosine_accuracy@5 value: 0.8421052631578947 name: Cosine Accuracy@5 - type: cosine_accuracy@10 value: 0.8690058479532163 name: Cosine Accuracy@10 - type: cosine_precision@1 value: 0.7649122807017544 name: Cosine Precision@1 - type: cosine_precision@3 value: 0.6775828460038987 name: Cosine Precision@3 - type: cosine_precision@5 value: 0.5796491228070175 name: Cosine Precision@5 - type: cosine_precision@10 value: 0.42701754385964913 name: Cosine Precision@10 - type: cosine_recall@1 value: 0.16621564142709158 name: Cosine Recall@1 - type: cosine_recall@3 value: 0.4102644606511703 name: Cosine Recall@3 - type: cosine_recall@5 value: 0.5203490784736678 name: Cosine Recall@5 - type: cosine_recall@10 value: 0.6468261230695532 name: Cosine Recall@10 - type: cosine_ndcg@10 value: 0.724241553378169 name: Cosine Ndcg@10 - type: cosine_mrr@10 value: 0.7982117330362947 name: Cosine Mrr@10 - type: cosine_map@10 value: 0.6694373149953072 name: Cosine Map@10 --- # SentenceTransformer based on abhinand/MedEmbed-small-v0.1 This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [abhinand/MedEmbed-small-v0.1](https://huggingface.co/abhinand/MedEmbed-small-v0.1). It maps sentences & paragraphs to a 384-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, classification, clustering, and more. ## Model Details ### Model Description - **Model Type:** Sentence Transformer - **Base model:** [abhinand/MedEmbed-small-v0.1](https://huggingface.co/abhinand/MedEmbed-small-v0.1) - **Maximum Sequence Length:** 30 tokens - **Output Dimensionality:** 384 dimensions - **Similarity Function:** Cosine Similarity - **Supported Modality:** Text ### Model Sources - **Documentation:** [Sentence Transformers Documentation](https://sbert.net) - **Repository:** [Sentence Transformers on GitHub](https://github.com/huggingface/sentence-transformers) - **Hugging Face:** [Sentence Transformers on Hugging Face](https://huggingface.co/models?library=sentence-transformers) ### Full Model Architecture ``` SentenceTransformer( (0): Transformer({'transformer_task': 'feature-extraction', 'modality_config': {'text': {'method': 'forward', 'method_output_name': 'last_hidden_state'}}, 'module_output_name': 'token_embeddings', 'architecture': 'BertModel'}) (1): Pooling({'embedding_dimension': 384, 'pooling_mode': 'cls', 'include_prompt': True}) (2): Normalize({}) ) ``` ## Usage ### Direct Usage (Sentence Transformers) First install the Sentence Transformers library: ```bash pip install -U sentence-transformers ``` Then you can load this model and run inference. ```python from sentence_transformers import SentenceTransformer # Download from the 🤗 Hub model = SentenceTransformer("seanfarrell/icd11_ner_model_v2") # Run inference queries = [ 'multiple delivery by combination of methods', ] documents = [ 'multiple delivery by combination of methods without caesarean', 'bipolar or related disorders, unspecified', 'rectal spasm', ] query_embeddings = model.encode_query(queries) document_embeddings = model.encode_document(documents) print(query_embeddings.shape, document_embeddings.shape) # [1, 384] [3, 384] # Get the similarity scores for the embeddings similarities = model.similarity(query_embeddings, document_embeddings) print(similarities) # tensor([[ 0.9466, -0.0751, -0.0982]]) ``` ## Evaluation ### Metrics #### Information Retrieval * Dataset: `icd11-val` * Evaluated with [InformationRetrievalEvaluator](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.sentence_transformer.evaluation.InformationRetrievalEvaluator) | Metric | Value | |:--------------------|:-----------| | cosine_accuracy@1 | 0.7649 | | cosine_accuracy@5 | 0.8421 | | cosine_accuracy@10 | 0.869 | | cosine_precision@1 | 0.7649 | | cosine_precision@3 | 0.6776 | | cosine_precision@5 | 0.5796 | | cosine_precision@10 | 0.427 | | cosine_recall@1 | 0.1662 | | cosine_recall@3 | 0.4103 | | cosine_recall@5 | 0.5203 | | cosine_recall@10 | 0.6468 | | **cosine_ndcg@10** | **0.7242** | | cosine_mrr@10 | 0.7982 | | cosine_map@10 | 0.6694 | ## Training Details ### Training Dataset #### Unnamed Dataset * Size: 8,089,600 training samples * Columns: anchor, positive, and negative * Approximate statistics based on the first 100 samples: | | anchor | positive | negative | |:---------|:---------------------------------------------------------------------------------|:--------------------------------------------------------------------------------|:---------------------------------------------------------------------------------| | type | string | string | string | | modality | text | text | text | | details | | | | * Samples: | anchor | positive | negative | |:---------------------------------------------------------------|:---------------------------------------------------------------------------------|:------------------------------------------------------------| | rheumatic myocardiopathy | chronic rheumatic heart | abnormal liver lobulation | | chronic rheumatic coronary arteritis | inactive or quiescent rheumatic fever with left ventricular failure | other specified impairment of visual functions | | chronic rheumatic heart disease with endocarditis | rheumatic endocarditis | listeriosis, unspecified | * Loss: [MultipleNegativesRankingLoss](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#multiplenegativesrankingloss) with these parameters: ```json { "scale": 20.0, "similarity_fct": "cos_sim", "gather_across_devices": false, "directions": [ "query_to_doc" ], "partition_mode": "joint", "hardness_mode": null, "hardness_strength": 0.0 } ``` ### Training Hyperparameters #### Non-Default Hyperparameters - `per_device_train_batch_size`: 512 - `num_train_epochs`: 1 - `learning_rate`: 2e-05 - `lr_scheduler_type`: cosine - `warmup_steps`: 1580 - `weight_decay`: 0.01 - `bf16`: True - `per_device_eval_batch_size`: 256 - `eval_on_start`: True - `load_best_model_at_end`: True - `dataloader_drop_last`: True #### All Hyperparameters
Click to expand - `per_device_train_batch_size`: 512 - `num_train_epochs`: 1 - `max_steps`: -1 - `learning_rate`: 2e-05 - `lr_scheduler_type`: cosine - `lr_scheduler_kwargs`: None - `warmup_steps`: 1580 - `optim`: adamw_torch_fused - `optim_args`: None - `weight_decay`: 0.01 - `adam_beta1`: 0.9 - `adam_beta2`: 0.999 - `adam_epsilon`: 1e-08 - `optim_target_modules`: None - `gradient_accumulation_steps`: 1 - `average_tokens_across_devices`: True - `max_grad_norm`: 1.0 - `label_smoothing_factor`: 0.0 - `bf16`: True - `fp16`: False - `bf16_full_eval`: False - `fp16_full_eval`: False - `tf32`: None - `gradient_checkpointing`: False - `gradient_checkpointing_kwargs`: None - `torch_compile`: False - `torch_compile_backend`: None - `torch_compile_mode`: None - `use_liger_kernel`: False - `liger_kernel_config`: None - `use_cache`: False - `neftune_noise_alpha`: None - `torch_empty_cache_steps`: None - `auto_find_batch_size`: False - `log_on_each_node`: True - `logging_nan_inf_filter`: True - `include_num_input_tokens_seen`: no - `log_level`: passive - `log_level_replica`: warning - `disable_tqdm`: False - `project`: huggingface - `trackio_space_id`: None - `trackio_bucket_id`: None - `trackio_static_space_id`: None - `per_device_eval_batch_size`: 256 - `prediction_loss_only`: True - `eval_on_start`: True - `eval_do_concat_batches`: True - `eval_use_gather_object`: False - `eval_accumulation_steps`: None - `include_for_metrics`: [] - `batch_eval_metrics`: False - `save_only_model`: False - `save_on_each_node`: False - `enable_jit_checkpoint`: False - `push_to_hub`: False - `hub_private_repo`: None - `hub_model_id`: None - `hub_strategy`: every_save - `hub_always_push`: False - `hub_revision`: None - `load_best_model_at_end`: True - `ignore_data_skip`: False - `restore_callback_states_from_checkpoint`: False - `full_determinism`: False - `seed`: 42 - `data_seed`: None - `use_cpu`: False - `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None} - `parallelism_config`: None - `dataloader_drop_last`: True - `dataloader_num_workers`: 0 - `dataloader_pin_memory`: True - `dataloader_persistent_workers`: False - `dataloader_prefetch_factor`: None - `dataloader_multiprocessing_context`: None - `dataloader_in_order`: True - `remove_unused_columns`: True - `label_names`: None - `train_sampling_strategy`: random - `length_column_name`: length - `ddp_find_unused_parameters`: None - `ddp_bucket_cap_mb`: None - `ddp_broadcast_buffers`: False - `ddp_static_graph`: None - `ddp_backend`: None - `ddp_timeout`: 1800 - `fsdp`: None - `fsdp_config`: None - `deepspeed`: None - `debug`: [] - `skip_memory_metrics`: True - `do_predict`: False - `resume_from_checkpoint`: None - `local_rank`: -1 - `prompts`: None - `batch_sampler`: batch_sampler - `multi_dataset_batch_sampler`: proportional - `router_mapping`: {} - `learning_rate_mapping`: {} - `warmup_ratio`: None
### Training Logs
Click to expand | Epoch | Step | Training Loss | icd11-val_cosine_ndcg@10 | |:----------:|:---------:|:-------------:|:------------------------:| | 0 | 0 | - | 0.5035 | | 0.0032 | 50 | 3.3034 | - | | 0.0063 | 100 | 3.0570 | - | | 0.0095 | 150 | 2.6980 | - | | 0.0127 | 200 | 2.3748 | - | | 0.0158 | 250 | 2.1693 | - | | 0.0190 | 300 | 2.0670 | - | | 0.0222 | 350 | 2.0079 | - | | 0.0253 | 400 | 1.9661 | - | | 0.0285 | 450 | 1.9317 | - | | 0.0316 | 500 | 1.9053 | 0.5508 | | 0.0348 | 550 | 1.8528 | - | | 0.0380 | 600 | 1.8550 | - | | 0.0411 | 650 | 1.7966 | - | | 0.0443 | 700 | 1.7828 | - | | 0.0475 | 750 | 1.7346 | - | | 0.0506 | 800 | 1.6919 | - | | 0.0538 | 850 | 1.6980 | - | | 0.0570 | 900 | 1.6606 | - | | 0.0601 | 950 | 1.6058 | - | | 0.0633 | 1000 | 1.5859 | 0.5926 | | 0.0665 | 1050 | 1.5658 | - | | 0.0696 | 1100 | 1.5588 | - | | 0.0728 | 1150 | 1.5080 | - | | 0.0759 | 1200 | 1.4797 | - | | 0.0791 | 1250 | 1.4802 | - | | 0.0823 | 1300 | 1.4226 | - | | 0.0854 | 1350 | 1.4006 | - | | 0.0886 | 1400 | 1.3809 | - | | 0.0918 | 1450 | 1.3675 | - | | 0.0949 | 1500 | 1.3573 | 0.6178 | | 0.0981 | 1550 | 1.3354 | - | | 0.1013 | 1600 | 1.3037 | - | | 0.1044 | 1650 | 1.2889 | - | | 0.1076 | 1700 | 1.2706 | - | | 0.1108 | 1750 | 1.2340 | - | | 0.1139 | 1800 | 1.2057 | - | | 0.1171 | 1850 | 1.1862 | - | | 0.1203 | 1900 | 1.1948 | - | | 0.1234 | 1950 | 1.1557 | - | | 0.1266 | 2000 | 1.1571 | 0.6386 | | 0.1297 | 2050 | 1.1288 | - | | 0.1329 | 2100 | 1.1162 | - | | 0.1361 | 2150 | 1.0780 | - | | 0.1392 | 2200 | 1.0784 | - | | 0.1424 | 2250 | 1.0607 | - | | 0.1456 | 2300 | 1.0435 | - | | 0.1487 | 2350 | 1.0256 | - | | 0.1519 | 2400 | 1.0248 | - | | 0.1551 | 2450 | 0.9990 | - | | 0.1582 | 2500 | 0.9757 | 0.6572 | | 0.1614 | 2550 | 0.9813 | - | | 0.1646 | 2600 | 0.9761 | - | | 0.1677 | 2650 | 0.9549 | - | | 0.1709 | 2700 | 0.9369 | - | | 0.1741 | 2750 | 0.9127 | - | | 0.1772 | 2800 | 0.9223 | - | | 0.1804 | 2850 | 0.9096 | - | | 0.1835 | 2900 | 0.8837 | - | | 0.1867 | 2950 | 0.8843 | - | | 0.1899 | 3000 | 0.8898 | 0.6649 | | 0.1930 | 3050 | 0.8656 | - | | 0.1962 | 3100 | 0.8796 | - | | 0.1994 | 3150 | 0.8464 | - | | 0.2025 | 3200 | 0.8261 | - | | 0.2057 | 3250 | 0.8280 | - | | 0.2089 | 3300 | 0.8319 | - | | 0.2120 | 3350 | 0.8292 | - | | 0.2152 | 3400 | 0.7955 | - | | 0.2184 | 3450 | 0.8011 | - | | 0.2215 | 3500 | 0.7908 | 0.6771 | | 0.2247 | 3550 | 0.7760 | - | | 0.2278 | 3600 | 0.7667 | - | | 0.2310 | 3650 | 0.7683 | - | | 0.2342 | 3700 | 0.7523 | - | | 0.2373 | 3750 | 0.7344 | - | | 0.2405 | 3800 | 0.7471 | - | | 0.2437 | 3850 | 0.7250 | - | | 0.2468 | 3900 | 0.7163 | - | | 0.25 | 3950 | 0.7003 | - | | 0.2532 | 4000 | 0.7103 | 0.6836 | | 0.2563 | 4050 | 0.7073 | - | | 0.2595 | 4100 | 0.7046 | - | | 0.2627 | 4150 | 0.6804 | - | | 0.2658 | 4200 | 0.6806 | - | | 0.2690 | 4250 | 0.6792 | - | | 0.2722 | 4300 | 0.6708 | - | | 0.2753 | 4350 | 0.6678 | - | | 0.2785 | 4400 | 0.6475 | - | | 0.2816 | 4450 | 0.6444 | - | | 0.2848 | 4500 | 0.6395 | 0.6841 | | 0.2880 | 4550 | 0.6399 | - | | 0.2911 | 4600 | 0.6415 | - | | 0.2943 | 4650 | 0.6407 | - | | 0.2975 | 4700 | 0.6220 | - | | 0.3006 | 4750 | 0.6198 | - | | 0.3038 | 4800 | 0.6145 | - | | 0.3070 | 4850 | 0.6029 | - | | 0.3101 | 4900 | 0.6042 | - | | 0.3133 | 4950 | 0.5962 | - | | 0.3165 | 5000 | 0.5979 | 0.6933 | | 0.3196 | 5050 | 0.5811 | - | | 0.3228 | 5100 | 0.5738 | - | | 0.3259 | 5150 | 0.5731 | - | | 0.3291 | 5200 | 0.5787 | - | | 0.3323 | 5250 | 0.5770 | - | | 0.3354 | 5300 | 0.5539 | - | | 0.3386 | 5350 | 0.5573 | - | | 0.3418 | 5400 | 0.5619 | - | | 0.3449 | 5450 | 0.5387 | - | | 0.3481 | 5500 | 0.5357 | 0.6979 | | 0.3513 | 5550 | 0.5502 | - | | 0.3544 | 5600 | 0.5316 | - | | 0.3576 | 5650 | 0.5386 | - | | 0.3608 | 5700 | 0.5257 | - | | 0.3639 | 5750 | 0.5032 | - | | 0.3671 | 5800 | 0.5197 | - | | 0.3703 | 5850 | 0.5147 | - | | 0.3734 | 5900 | 0.5194 | - | | 0.3766 | 5950 | 0.5195 | - | | 0.3797 | 6000 | 0.5241 | 0.6999 | | 0.3829 | 6050 | 0.5032 | - | | 0.3861 | 6100 | 0.4955 | - | | 0.3892 | 6150 | 0.5055 | - | | 0.3924 | 6200 | 0.4951 | - | | 0.3956 | 6250 | 0.5022 | - | | 0.3987 | 6300 | 0.4916 | - | | 0.4019 | 6350 | 0.4840 | - | | 0.4051 | 6400 | 0.4684 | - | | 0.4082 | 6450 | 0.4569 | - | | 0.4114 | 6500 | 0.4737 | 0.7041 | | 0.4146 | 6550 | 0.4609 | - | | 0.4177 | 6600 | 0.4694 | - | | 0.4209 | 6650 | 0.4588 | - | | 0.4241 | 6700 | 0.4710 | - | | 0.4272 | 6750 | 0.4526 | - | | 0.4304 | 6800 | 0.4605 | - | | 0.4335 | 6850 | 0.4720 | - | | 0.4367 | 6900 | 0.4594 | - | | 0.4399 | 6950 | 0.4497 | - | | 0.4430 | 7000 | 0.4475 | 0.7087 | | 0.4462 | 7050 | 0.4415 | - | | 0.4494 | 7100 | 0.4275 | - | | 0.4525 | 7150 | 0.4349 | - | | 0.4557 | 7200 | 0.4375 | - | | 0.4589 | 7250 | 0.4340 | - | | 0.4620 | 7300 | 0.4346 | - | | 0.4652 | 7350 | 0.4281 | - | | 0.4684 | 7400 | 0.4277 | - | | 0.4715 | 7450 | 0.4188 | - | | 0.4747 | 7500 | 0.4179 | 0.7109 | | 0.4778 | 7550 | 0.4129 | - | | 0.4810 | 7600 | 0.4137 | - | | 0.4842 | 7650 | 0.4136 | - | | 0.4873 | 7700 | 0.4091 | - | | 0.4905 | 7750 | 0.4068 | - | | 0.4937 | 7800 | 0.4089 | - | | 0.4968 | 7850 | 0.4277 | - | | 0.5 | 7900 | 0.3952 | - | | 0.5032 | 7950 | 0.4009 | - | | 0.5063 | 8000 | 0.3889 | 0.7134 | | 0.5095 | 8050 | 0.4008 | - | | 0.5127 | 8100 | 0.4031 | - | | 0.5158 | 8150 | 0.3950 | - | | 0.5190 | 8200 | 0.3807 | - | | 0.5222 | 8250 | 0.3891 | - | | 0.5253 | 8300 | 0.3940 | - | | 0.5285 | 8350 | 0.3929 | - | | 0.5316 | 8400 | 0.3935 | - | | 0.5348 | 8450 | 0.3755 | - | | 0.5380 | 8500 | 0.3788 | 0.7176 | | 0.5411 | 8550 | 0.3770 | - | | 0.5443 | 8600 | 0.3882 | - | | 0.5475 | 8650 | 0.3747 | - | | 0.5506 | 8700 | 0.3785 | - | | 0.5538 | 8750 | 0.3689 | - | | 0.5570 | 8800 | 0.3706 | - | | 0.5601 | 8850 | 0.3756 | - | | 0.5633 | 8900 | 0.3679 | - | | 0.5665 | 8950 | 0.3696 | - | | 0.5696 | 9000 | 0.3722 | 0.7176 | | 0.5728 | 9050 | 0.3647 | - | | 0.5759 | 9100 | 0.3616 | - | | 0.5791 | 9150 | 0.3635 | - | | 0.5823 | 9200 | 0.3447 | - | | 0.5854 | 9250 | 0.3612 | - | | 0.5886 | 9300 | 0.3609 | - | | 0.5918 | 9350 | 0.3368 | - | | 0.5949 | 9400 | 0.3509 | - | | 0.5981 | 9450 | 0.3631 | - | | 0.6013 | 9500 | 0.3422 | 0.7188 | | 0.6044 | 9550 | 0.3537 | - | | 0.6076 | 9600 | 0.3507 | - | | 0.6108 | 9650 | 0.3460 | - | | 0.6139 | 9700 | 0.3527 | - | | 0.6171 | 9750 | 0.3441 | - | | 0.6203 | 9800 | 0.3509 | - | | 0.6234 | 9850 | 0.3396 | - | | 0.6266 | 9900 | 0.3350 | - | | 0.6297 | 9950 | 0.3493 | - | | 0.6329 | 10000 | 0.3299 | 0.7220 | | 0.6361 | 10050 | 0.3377 | - | | 0.6392 | 10100 | 0.3401 | - | | 0.6424 | 10150 | 0.3414 | - | | 0.6456 | 10200 | 0.3385 | - | | 0.6487 | 10250 | 0.3408 | - | | 0.6519 | 10300 | 0.3450 | - | | 0.6551 | 10350 | 0.3482 | - | | 0.6582 | 10400 | 0.3261 | - | | 0.6614 | 10450 | 0.3350 | - | | 0.6646 | 10500 | 0.3335 | 0.7215 | | 0.6677 | 10550 | 0.3458 | - | | 0.6709 | 10600 | 0.3194 | - | | 0.6741 | 10650 | 0.3224 | - | | 0.6772 | 10700 | 0.3289 | - | | 0.6804 | 10750 | 0.3239 | - | | 0.6835 | 10800 | 0.3283 | - | | 0.6867 | 10850 | 0.3277 | - | | 0.6899 | 10900 | 0.3243 | - | | 0.6930 | 10950 | 0.3196 | - | | 0.6962 | 11000 | 0.3204 | 0.7220 | | 0.6994 | 11050 | 0.3334 | - | | 0.7025 | 11100 | 0.3216 | - | | 0.7057 | 11150 | 0.3286 | - | | 0.7089 | 11200 | 0.3119 | - | | 0.7120 | 11250 | 0.3185 | - | | 0.7152 | 11300 | 0.3058 | - | | 0.7184 | 11350 | 0.3157 | - | | 0.7215 | 11400 | 0.3136 | - | | 0.7247 | 11450 | 0.3179 | - | | 0.7278 | 11500 | 0.3187 | 0.7235 | | 0.7310 | 11550 | 0.3144 | - | | 0.7342 | 11600 | 0.3156 | - | | 0.7373 | 11650 | 0.3139 | - | | 0.7405 | 11700 | 0.3104 | - | | 0.7437 | 11750 | 0.3187 | - | | 0.7468 | 11800 | 0.3154 | - | | 0.75 | 11850 | 0.3132 | - | | 0.7532 | 11900 | 0.3110 | - | | 0.7563 | 11950 | 0.3143 | - | | 0.7595 | 12000 | 0.3119 | 0.7236 | | 0.7627 | 12050 | 0.3068 | - | | 0.7658 | 12100 | 0.3227 | - | | 0.7690 | 12150 | 0.3170 | - | | 0.7722 | 12200 | 0.3046 | - | | 0.7753 | 12250 | 0.3085 | - | | 0.7785 | 12300 | 0.3135 | - | | 0.7816 | 12350 | 0.3042 | - | | 0.7848 | 12400 | 0.3152 | - | | 0.7880 | 12450 | 0.3122 | - | | 0.7911 | 12500 | 0.3201 | 0.7235 | | 0.7943 | 12550 | 0.3148 | - | | 0.7975 | 12600 | 0.2993 | - | | 0.8006 | 12650 | 0.3040 | - | | 0.8038 | 12700 | 0.2973 | - | | 0.8070 | 12750 | 0.3074 | - | | 0.8101 | 12800 | 0.3074 | - | | 0.8133 | 12850 | 0.2971 | - | | 0.8165 | 12900 | 0.3060 | - | | 0.8196 | 12950 | 0.3136 | - | | 0.8228 | 13000 | 0.3095 | 0.7244 | | 0.8259 | 13050 | 0.3032 | - | | 0.8291 | 13100 | 0.3070 | - | | 0.8323 | 13150 | 0.3014 | - | | 0.8354 | 13200 | 0.2965 | - | | 0.8386 | 13250 | 0.2977 | - | | 0.8418 | 13300 | 0.3077 | - | | 0.8449 | 13350 | 0.3145 | - | | 0.8481 | 13400 | 0.3019 | - | | 0.8513 | 13450 | 0.3201 | - | | **0.8544** | **13500** | **0.2986** | **0.7248** | | 0.8576 | 13550 | 0.3001 | - | | 0.8608 | 13600 | 0.3047 | - | | 0.8639 | 13650 | 0.3040 | - | | 0.8671 | 13700 | 0.2923 | - | | 0.8703 | 13750 | 0.3029 | - | | 0.8734 | 13800 | 0.2942 | - | | 0.8766 | 13850 | 0.2970 | - | | 0.8797 | 13900 | 0.2971 | - | | 0.8829 | 13950 | 0.2927 | - | | 0.8861 | 14000 | 0.3006 | 0.7239 | | 0.8892 | 14050 | 0.2930 | - | | 0.8924 | 14100 | 0.3001 | - | | 0.8956 | 14150 | 0.3124 | - | | 0.8987 | 14200 | 0.3083 | - | | 0.9019 | 14250 | 0.3059 | - | | 0.9051 | 14300 | 0.2926 | - | | 0.9082 | 14350 | 0.2905 | - | | 0.9114 | 14400 | 0.2959 | - | | 0.9146 | 14450 | 0.3038 | - | | 0.9177 | 14500 | 0.3045 | 0.7243 | | 0.9209 | 14550 | 0.2908 | - | | 0.9241 | 14600 | 0.3043 | - | | 0.9272 | 14650 | 0.3007 | - | | 0.9304 | 14700 | 0.2933 | - | | 0.9335 | 14750 | 0.2951 | - | | 0.9367 | 14800 | 0.2874 | - | | 0.9399 | 14850 | 0.3041 | - | | 0.9430 | 14900 | 0.3019 | - | | 0.9462 | 14950 | 0.3073 | - | | 0.9494 | 15000 | 0.3064 | 0.7245 | | 0.9525 | 15050 | 0.3038 | - | | 0.9557 | 15100 | 0.3043 | - | | 0.9589 | 15150 | 0.2990 | - | | 0.9620 | 15200 | 0.3016 | - | | 0.9652 | 15250 | 0.3073 | - | | 0.9684 | 15300 | 0.2959 | - | | 0.9715 | 15350 | 0.2936 | - | | 0.9747 | 15400 | 0.2928 | - | | 0.9778 | 15450 | 0.2945 | - | | 0.9810 | 15500 | 0.2930 | 0.7247 | | 0.9842 | 15550 | 0.3092 | - | | 0.9873 | 15600 | 0.3079 | - | | 0.9905 | 15650 | 0.2975 | - | | 0.9937 | 15700 | 0.2969 | - | | 0.9968 | 15750 | 0.2890 | - | | 1.0 | 15800 | 0.2956 | 0.7242 | * The bold row denotes the saved checkpoint.
### Training Time - **Training**: 26.3 minutes - **Evaluation**: 1.5 minutes - **Total**: 27.8 minutes ### Framework Versions - Python: 3.13.15 - Sentence Transformers: 5.7.0 - Transformers: 5.18.0 - PyTorch: 2.11.0+cu130 - Accelerate: 1.15.0 - Datasets: 4.8.5 - Tokenizers: 0.23.2 ## Additional Resources - [Training and Finetuning Embedding Models with Sentence Transformers](https://huggingface.co/blog/train-sentence-transformers): the end-to-end guide for training or finetuning Sentence Transformer models. - [Introduction to Matryoshka Embedding Models](https://huggingface.co/blog/matryoshka): variable-size embeddings that can be truncated with minimal quality loss. - [Binary and Scalar Embedding Quantization for Significantly Faster & Cheaper Retrieval](https://huggingface.co/blog/embedding-quantization): post-training compression of embedding vectors. - [Multimodal Embedding & Reranker Models with Sentence Transformers](https://huggingface.co/blog/multimodal-sentence-transformers): use text, image, audio, and video models through the same API. - [Training and Finetuning Multimodal Embedding & Reranker Models with Sentence Transformers](https://huggingface.co/blog/train-multimodal-sentence-transformers): train multimodal embedding models, with a Visual Document Retrieval walkthrough. ## Citation ### BibTeX #### Sentence Transformers ```bibtex @inproceedings{reimers-2019-sentence-bert, title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks", author = "Reimers, Nils and Gurevych, Iryna", booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing", month = "11", year = "2019", publisher = "Association for Computational Linguistics", url = "https://arxiv.org/abs/1908.10084", } ``` #### MultipleNegativesRankingLoss ```bibtex @misc{oord2019representationlearningcontrastivepredictive, title={Representation Learning with Contrastive Predictive Coding}, author={Aaron van den Oord and Yazhe Li and Oriol Vinyals}, year={2019}, eprint={1807.03748}, archivePrefix={arXiv}, primaryClass={cs.LG}, url={https://arxiv.org/abs/1807.03748}, } ```