rishhh/verisci-scifact-e5-retriever

SentenceTransformer based on intfloat/e5-base-v2

This is a sentence-transformers model finetuned from intfloat/e5-base-v2. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: intfloat/e5-base-v2
• Maximum Sequence Length: 512 tokens
• Output Dimensionality: 768 dimensions
• Similarity Function: Cosine Similarity

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: BertModel 
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("sentence_transformers_model_id")
# Run inference
sentences = [
    'query: Upregulation of dynein in Drosophila neurons increases the number of microtubule plus-ends growing toward the cell body of each neuron.',
    'passage: Dynein is required for polarized dendritic transport and uniform microtubule orientation in axons Axons and dendrites differ in both microtubule organization and in the organelles and proteins they contain. Here we show that the microtubule motor dynein has a crucial role in polarized transport and in controlling the orientation of axonal microtubules in Drosophila melanogaster dendritic arborization (da) neurons. Changes in organelle distribution within the dendritic arbors of dynein mutant neurons correlate with a proximal shift in dendritic branch position. Dynein is also necessary for the dendrite-specific localization of Golgi outposts and the ion channel Pickpocket. Axonal microtubules are normally oriented uniformly plus-end-distal; however, without dynein, axons contain both plus- and minus-end distal microtubules. These data suggest that dynein is required for the distinguishing properties of the axon and dendrites: without dynein, dendritic organelles and proteins enter the axon and the axonal microtubules are no longer uniform in polarity.',
    'passage: Tyr26 phosphorylation of PGAM1 provides a metabolic advantage to tumours by stabilizing the active conformation How oncogenic signalling coordinates glycolysis and anabolic biosynthesis in cancer cells remains unclear. We recently reported that the glycolytic enzyme phosphoglycerate mutase 1 (PGAM1) regulates anabolic biosynthesis by controlling intracellular levels of its substrate 3-phosphoglycerate and product 2-phosphoglycerate. Here we report a novel mechanism in which Y26 phosphorylation enhances PGAM1 activation through release of inhibitory E19 that blocks the active site, stabilising cofactor 2,3-bisphosphoglycerate binding and H11 phosphorylation. We also report the crystal structure of H11-phosphorylated PGAM1 and find that phospho-H11 activates PGAM1 at least in part by promoting substrate 3-phosphoglycerate binding. Moreover, Y26 phosphorylation of PGAM1 is common in human cancer cells and contributes to regulation of 3-phosphoglycerate and 2-phosphoglycerate levels, promoting cancer cell proliferation and tumour growth. As PGAM1 is a negative transcriptional target of TP53, and is therefore commonly upregulated in human cancers, these findings suggest that Y26 phosphorylation represents an additional acute mechanism underlying phosphoglycerate mutase 1 upregulation.',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

Training Details

Training Dataset

Unnamed Dataset

• Size: 919 training samples
• Columns: sentence_0 and sentence_1
• Approximate statistics based on the first 919 samples:

  	sentence_0	sentence_1
  type	string	string
  details	min: 9 tokens mean: 24.18 tokens max: 81 tokens	min: 84 tokens mean: 344.62 tokens max: 512 tokens

• Samples:

  sentence_0	sentence_1
  query: Cancer-associated fibroblasts (CAFs) have no known interaction with cancer cells in the formation and activation of CAFs.	passage: Microenvironmental regulation of tumor progression and metastasis Cancers develop in complex tissue environments, which they depend on for sustained growth, invasion and metastasis. Unlike tumor cells, stromal cell types within the tumor microenvironment (TME) are genetically stable and thus represent an attractive therapeutic target with reduced risk of resistance and tumor recurrence. However, specifically disrupting the pro-tumorigenic TME is a challenging undertaking, as the TME has diverse capacities to induce both beneficial and adverse consequences for tumorigenesis. Furthermore, many studies have shown that the microenvironment is capable of normalizing tumor cells, suggesting that re-education of stromal cells, rather than targeted ablation per se, may be an effective strategy for treating cancer. Here we discuss the paradoxical roles of the TME during specific stages of cancer progression and metastasis, as well as recent therapeutic attempts to re-educate stromal ce...
  query: Eilat virus (EILV) produced in mosquitos elicits rapid and long-lasting neutralizing antibodies in nonhuman primates.	passage: A chikungunya fever vaccine utilizing an insect-specific virus platform Traditionally, vaccine development involves tradeoffs between immunogenicity and safety. Live-attenuated vaccines typically offer rapid and durable immunity but have reduced safety when compared to inactivated vaccines. In contrast, the inability of inactivated vaccines to replicate enhances safety at the expense of immunogenicity, often necessitating multiple doses and boosters. To overcome these tradeoffs, we developed the insect-specific alphavirus, Eilat virus (EILV), as a vaccine platform. To address the chikungunya fever (CHIKF) pandemic, we used an EILV cDNA clone to design a chimeric virus containing the chikungunya virus (CHIKV) structural proteins. The recombinant EILV/CHIKV was structurally identical at 10 Å to wild-type CHIKV, as determined by single-particle cryo-electron microscopy, and it mimicked the early stages of CHIKV replication in vertebrate cells from attachment and entry to viral RN...
  query: Glial calcium waves influence seizures.	passage: An Excitatory Loop with Astrocytes Contributes to Drive Neurons to Seizure Threshold Seizures in focal epilepsies are sustained by a highly synchronous neuronal discharge that arises at restricted brain sites and subsequently spreads to large portions of the brain. Despite intense experimental research in this field, the earlier cellular events that initiate and sustain a focal seizure are still not well defined. Their identification is central to understand the pathophysiology of focal epilepsies and to develop new pharmacological therapies for drug-resistant forms of epilepsy. The prominent involvement of astrocytes in ictogenesis was recently proposed. We test here whether a cooperation between astrocytes and neurons is a prerequisite to support ictal (seizure-like) and interictal epileptiform events. Simultaneous patch-clamp recording and Ca2+ imaging techniques were performed in a new in vitro model of focal seizures induced by local applications of N-methyl-D-aspartic ac...

• Loss: MultipleNegativesRankingLoss with these parameters:

  {
      "scale": 20.0,
      "similarity_fct": "cos_sim"
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• per_device_train_batch_size: 32
• per_device_eval_batch_size: 32
• multi_dataset_batch_sampler: round_robin

All Hyperparameters

Click to expand

• overwrite_output_dir: False
• do_predict: False
• eval_strategy: no
• prediction_loss_only: True
• per_device_train_batch_size: 32
• per_device_eval_batch_size: 32
• per_gpu_train_batch_size: None
• per_gpu_eval_batch_size: None
• gradient_accumulation_steps: 1
• eval_accumulation_steps: None
• torch_empty_cache_steps: None
• learning_rate: 5e-05
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1
• num_train_epochs: 3
• max_steps: -1
• lr_scheduler_type: linear
• lr_scheduler_kwargs: None
• warmup_ratio: 0.0
• warmup_steps: 0
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• save_safetensors: True
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• no_cuda: False
• use_cpu: False
• use_mps_device: False
• seed: 42
• data_seed: None
• jit_mode_eval: False
• bf16: False
• fp16: False
• fp16_opt_level: O1
• half_precision_backend: auto
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• local_rank: 0
• ddp_backend: None
• tpu_num_cores: None
• tpu_metrics_debug: False
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_prefetch_factor: None
• past_index: -1
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: False
• ignore_data_skip: False
• fsdp: []
• fsdp_min_num_params: 0
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• fsdp_transformer_layer_cls_to_wrap: None
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• parallelism_config: None
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch_fused
• optim_args: None
• adafactor: False
• group_by_length: False
• length_column_name: length
• project: huggingface
• trackio_space_id: trackio
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• use_legacy_prediction_loop: False
• push_to_hub: False
• resume_from_checkpoint: None
• hub_model_id: None
• hub_strategy: every_save
• hub_private_repo: None
• hub_always_push: False
• hub_revision: None
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• include_inputs_for_metrics: False
• include_for_metrics: []
• eval_do_concat_batches: True
• fp16_backend: auto
• push_to_hub_model_id: None
• push_to_hub_organization: None
• mp_parameters:
• auto_find_batch_size: False
• full_determinism: False
• torchdynamo: None
• ray_scope: last
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• include_tokens_per_second: False
• include_num_input_tokens_seen: no
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• eval_on_start: False
• use_liger_kernel: False
• liger_kernel_config: None
• eval_use_gather_object: False
• average_tokens_across_devices: True
• prompts: None
• batch_sampler: batch_sampler
• multi_dataset_batch_sampler: round_robin

Framework Versions

• Python: 3.12.12
• Sentence Transformers: 3.4.1
• Transformers: 4.57.6
• PyTorch: 2.12.0+cu130
• Accelerate: 1.13.0
• Datasets: 2.21.0
• Tokenizers: 0.22.2

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MultipleNegativesRankingLoss

@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply},
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}