File size: 1,405 Bytes
c2f37c1 f4e57b7 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 | ---
library_name: peft
base_model: sentence-transformers/all-mpnet-base-v2
tags:
- medical
- cardiology
- embeddings
- domain-adaptation
- lora
- sentence-transformers
- sentence-similarity
language:
- en
license: apache-2.0
---
# CardioEmbed-MPNet-base
**Domain-specialized cardiology text embeddings using LoRA-adapted MPNet-base**
Part of a comparative study of 10 embedding architectures for clinical cardiology.
## Performance
| Metric | Score |
|--------|-------|
| Separation Score | **0.386** |
## Usage
```python
from transformers import AutoModel, AutoTokenizer
from peft import PeftModel
base_model = AutoModel.from_pretrained("sentence-transformers/all-mpnet-base-v2")
tokenizer = AutoTokenizer.from_pretrained("sentence-transformers/all-mpnet-base-v2")
model = PeftModel.from_pretrained(base_model, "richardyoung/CardioEmbed-MPNet-base")
```
## Training
- **Training Data**: 106,535 cardiology text pairs from medical textbooks
- **Method**: LoRA fine-tuning (r=16, alpha=32)
- **Loss**: Multiple Negatives Ranking Loss (InfoNCE)
## Citation
```bibtex
@article{young2024comparative,
title={Comparative Analysis of LoRA-Adapted Embedding Models for Clinical Cardiology Text Representation},
author={Young, Richard J and Matthews, Alice M},
journal={arXiv preprint},
year={2024}
}
```
---
*Built & maintained by [Richard Young](https://deepneuro.ai/richard) · DeepNeuro*
|