Sentence Similarity
sentence-transformers
ONNX
Safetensors
Chinese
English
modernbert
embeddings
medical
clinical-nlp
healthcare
biomedical
fhir
traditional-chinese
taiwan
rag
text-embeddings-inference
Instructions to use weemed/IlhaEmbed with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- sentence-transformers
How to use weemed/IlhaEmbed with sentence-transformers:
from sentence_transformers import SentenceTransformer model = SentenceTransformer("weemed/IlhaEmbed") sentences = [ "帶狀皰疹", "皮蛇", "高血壓用藥", "114年成健" ] embeddings = model.encode(sentences) similarities = model.similarity(embeddings, embeddings) print(similarities.shape) # [4, 4] - Notebooks
- Google Colab
- Kaggle
fix(style): remove block img from headings to ensure clean typography and proper heading alignment
Browse files
README.md
CHANGED
|
@@ -45,7 +45,7 @@ widget:
|
|
| 45 |
|
| 46 |
---
|
| 47 |
|
| 48 |
-
##
|
| 49 |
|
| 50 |
| Metric / Specification | IlhaEmbed-311M (Flagship) | IlhaEmbed-97M (Ultra-Lightweight) | Release Gate / Baseline |
|
| 51 |
|---|---:|---:|---:|
|
|
@@ -63,7 +63,7 @@ widget:
|
|
| 63 |
|
| 64 |
---
|
| 65 |
|
| 66 |
-
###
|
| 67 |
Evaluated using official **MTEB (Massive Text Embedding Benchmark v2.21.0)** on Chinese biomedical and semantic tasks, benchmarked head-to-head against the leading open-source model **BAAI/bge-small-zh-v1.5**:
|
| 68 |
|
| 69 |
| Task | Metric | **IlhaEmbed (97M / 37MB)** | **BAAI/bge-small-zh** | Analysis |
|
|
@@ -78,13 +78,13 @@ Evaluated using official **MTEB (Massive Text Embedding Benchmark v2.21.0)** on
|
|
| 78 |
|
| 79 |
> **Score Interpretation**: In information retrieval benchmarks, MAP/NDCG@10 in the 0.45–0.52 range represents top-tier open-source performance (e.g. BGE-small, OpenAI text-embedding-3-large). IlhaEmbed delivers competitive retrieval at a fraction of the computational footprint.
|
| 80 |
|
| 81 |
-
###
|
| 82 |
To verify real-world robustness beyond curated single terms, IlhaEmbed was tested against 20 dirty, unstructured, multi-clause clinical notes (spanning emergency triage, Taigi spoken complaints, nursing discharge summaries, and ICU notes):
|
| 83 |
- **Top-1 Dominant Intent Accuracy**: **95.0% (19/20)**
|
| 84 |
- **Top-3 Intent Coverage**: **100.0% (20/20)**
|
| 85 |
- **Boundary Edge Case**: Past checkup completion (*"114年成健已做"*) scored 0.703 for `checkup_intent` vs 0.692 for `care_event`. The small 0.011 margin demonstrates why pure semantic embeddings must be paired with downstream temporal validation rules and clinical human-in-the-loop oversight.
|
| 86 |
|
| 87 |
-
##
|
| 88 |
|
| 89 |
- **Intended Use**: Assistive terminology alignment, semantic routing, and candidate recommendation (Suggest-with-Review).
|
| 90 |
- **Non-SaMD Posture**: Under Taiwan TFDA / international SaMD regulatory guidance, IlhaEmbed does not diagnose, treat, or autonomously formulate clinical care decisions. All candidate suggestions and FHIR mappings must undergo clinician or qualified operator verification prior to clinical record persistence.
|
|
@@ -92,7 +92,7 @@ To verify real-world robustness beyond curated single terms, IlhaEmbed was teste
|
|
| 92 |
|
| 93 |
---
|
| 94 |
|
| 95 |
-
##
|
| 96 |
|
| 97 |
### 1. Flagship (311M) — Python / Sentence-Transformers
|
| 98 |
```python
|
|
@@ -135,7 +135,7 @@ print(normed.shape) # (2, 384)
|
|
| 135 |
|
| 136 |
---
|
| 137 |
|
| 138 |
-
##
|
| 139 |
|
| 140 |
1. **Zero Protected Health Information (PHI)**: No patient records, electronic medical records (EMR), or private institutional data are included in training datasets or model checkpoints.
|
| 141 |
2. **Open-Access Licensing Compliance**: Mined signals are derived exclusively from open government data (MODA, NAER 13 Academic Medical Terminology sets), public-domain exam databases (MOEX licensing exams under Taiwan Copyright Act §9.1.5), and licensed terminology descriptions.
|
|
@@ -156,7 +156,7 @@ print(normed.shape) # (2, 384)
|
|
| 156 |
|
| 157 |
---
|
| 158 |
|
| 159 |
-
##
|
| 160 |
|
| 161 |
台灣各級醫療院所的電子病歷、護理交班、社區健檢表上充滿高度在地化的行話與臨床縮寫:
|
| 162 |
- `L-CT`:代表低劑量胸部電腦斷層(肺癌早期篩檢)。
|
|
@@ -183,7 +183,7 @@ print(normed.shape) # (2, 384)
|
|
| 183 |
|
| 184 |
---
|
| 185 |
|
| 186 |
-
##
|
| 187 |
|
| 188 |
| 評測維度/指標 | **IlhaEmbed-311M (Flagship)** | **IlhaEmbed-97M (Edge / Kiosk)** | 歷史開源基準 (jina/ckip/bge) | 發布門禁要求 |
|
| 189 |
|---|---:|---:|---:|---:|
|
|
@@ -201,7 +201,7 @@ print(normed.shape) # (2, 384)
|
|
| 201 |
|
| 202 |
---
|
| 203 |
|
| 204 |
-
###
|
| 205 |
使用官方最新版 MTEB v2.21.0 針對中文醫療與語意任務進行同機同卡嚴謹對照,對比開源主流標竿模型 **BAAI/bge-small-zh-v1.5**:
|
| 206 |
|
| 207 |
| MTEB 官方評測任務 | 核心指標 | **IlhaEmbed (97M / 37MB)** | **BAAI/bge-small-zh** | 實測結論與意義 |
|
|
@@ -216,13 +216,13 @@ print(normed.shape) # (2, 384)
|
|
| 216 |
|
| 217 |
> **評測數值常識說明**:在資訊檢索(IR)領域中,MAP / NDCG@10 達 0.45 ~ 0.52 即為全球開源與商業模型(如 BGE-small, OpenAI)之頂尖梯隊。IlhaEmbed 僅以 37MB 體積即在醫療檢索任務超越 BGE-small。
|
| 218 |
|
| 219 |
-
###
|
| 220 |
使用正式發布版模型對 20 筆包含門診、急診、台語主訴、護理給藥等真實複合長句進行檢驗:
|
| 221 |
- **主意圖命中率 (Top-1 Dominant Hit)**:**95.0% (19/20)**
|
| 222 |
- **前三候選覆蓋率 (Top-3 Intent Coverage)**:**100.0% (20/20)**
|
| 223 |
- **邊界失效案例分析**:`「114年成健已做,糞檢MIF未交,建議下次門診補交檢體」` 預測 `checkup_intent` (0.703) 與 `care_event` (0.692) 僅差 0.011。「已做」與「安排」的時態語意在純向量空間邊界模糊,證明臨床落地時必須保留下游規則過濾與醫事人員覆核(Fail-Closed 殘差機制)。
|
| 224 |
|
| 225 |
-
##
|
| 226 |
|
| 227 |
### 1. Python / Sentence-Transformers(支援雙版本)
|
| 228 |
|
|
@@ -271,7 +271,7 @@ print("ONNX 輸出維度:", normed.shape) # (2, 384)
|
|
| 271 |
|
| 272 |
---
|
| 273 |
|
| 274 |
-
##
|
| 275 |
|
| 276 |
- **預期用途(Intended Use)**:臨床輔助建議、攝取分流與候選重排(Suggest-with-Review Candidate Ranker)。
|
| 277 |
- **非 SaMD 宣告(Non-SaMD Posture)**:依據台灣衛生福利部食品藥物管理署(TFDA)與國際醫療器材軟體(SaMD)法規指引,IlhaEmbed 不具備自主診斷、疾病處方或獨立醫療決策功能。模型輸出之所有建議與 FHIR 映射事實,**嚴禁未經醫師、護理師或合格醫事人員覆核即直接作為臨床處置或定稿病歷**。
|
|
@@ -279,7 +279,7 @@ print("ONNX 輸出維度:", normed.shape) # (2, 384)
|
|
| 279 |
|
| 280 |
---
|
| 281 |
|
| 282 |
-
##
|
| 283 |
|
| 284 |
1. **零受保護健康資訊(Zero PHI)**:模型訓練與評測全流程不包含任何真實病患姓名、身分證號、病歷號或可識別隱私個資。
|
| 285 |
2. **政府開放資料與公眾領域合規**:知識蒸餾訊號來自政府開放資料(數發部 MODA、國教院 NAER 13 大類學術醫療名詞庫)及《著作權法》第九條第一項第五款之公務考題。
|
|
|
|
| 45 |
|
| 46 |
---
|
| 47 |
|
| 48 |
+
## Technical Specifications & Benchmark Comparison
|
| 49 |
|
| 50 |
| Metric / Specification | IlhaEmbed-311M (Flagship) | IlhaEmbed-97M (Ultra-Lightweight) | Release Gate / Baseline |
|
| 51 |
|---|---:|---:|---:|
|
|
|
|
| 63 |
|
| 64 |
---
|
| 65 |
|
| 66 |
+
### Official MTEB Chinese Benchmark Comparison
|
| 67 |
Evaluated using official **MTEB (Massive Text Embedding Benchmark v2.21.0)** on Chinese biomedical and semantic tasks, benchmarked head-to-head against the leading open-source model **BAAI/bge-small-zh-v1.5**:
|
| 68 |
|
| 69 |
| Task | Metric | **IlhaEmbed (97M / 37MB)** | **BAAI/bge-small-zh** | Analysis |
|
|
|
|
| 78 |
|
| 79 |
> **Score Interpretation**: In information retrieval benchmarks, MAP/NDCG@10 in the 0.45–0.52 range represents top-tier open-source performance (e.g. BGE-small, OpenAI text-embedding-3-large). IlhaEmbed delivers competitive retrieval at a fraction of the computational footprint.
|
| 80 |
|
| 81 |
+
### Realistic Clinical Shorthand Evaluation (20 Arbitrary Multi-Clause Notes)
|
| 82 |
To verify real-world robustness beyond curated single terms, IlhaEmbed was tested against 20 dirty, unstructured, multi-clause clinical notes (spanning emergency triage, Taigi spoken complaints, nursing discharge summaries, and ICU notes):
|
| 83 |
- **Top-1 Dominant Intent Accuracy**: **95.0% (19/20)**
|
| 84 |
- **Top-3 Intent Coverage**: **100.0% (20/20)**
|
| 85 |
- **Boundary Edge Case**: Past checkup completion (*"114年成健已做"*) scored 0.703 for `checkup_intent` vs 0.692 for `care_event`. The small 0.011 margin demonstrates why pure semantic embeddings must be paired with downstream temporal validation rules and clinical human-in-the-loop oversight.
|
| 86 |
|
| 87 |
+
## Safety & Regulatory Boundary (SaMD Exemption)
|
| 88 |
|
| 89 |
- **Intended Use**: Assistive terminology alignment, semantic routing, and candidate recommendation (Suggest-with-Review).
|
| 90 |
- **Non-SaMD Posture**: Under Taiwan TFDA / international SaMD regulatory guidance, IlhaEmbed does not diagnose, treat, or autonomously formulate clinical care decisions. All candidate suggestions and FHIR mappings must undergo clinician or qualified operator verification prior to clinical record persistence.
|
|
|
|
| 92 |
|
| 93 |
---
|
| 94 |
|
| 95 |
+
## Usage
|
| 96 |
|
| 97 |
### 1. Flagship (311M) — Python / Sentence-Transformers
|
| 98 |
```python
|
|
|
|
| 135 |
|
| 136 |
---
|
| 137 |
|
| 138 |
+
## Data Governance & Open-Source Principles
|
| 139 |
|
| 140 |
1. **Zero Protected Health Information (PHI)**: No patient records, electronic medical records (EMR), or private institutional data are included in training datasets or model checkpoints.
|
| 141 |
2. **Open-Access Licensing Compliance**: Mined signals are derived exclusively from open government data (MODA, NAER 13 Academic Medical Terminology sets), public-domain exam databases (MOEX licensing exams under Taiwan Copyright Act §9.1.5), and licensed terminology descriptions.
|
|
|
|
| 156 |
|
| 157 |
---
|
| 158 |
|
| 159 |
+
## 臨床痛點與核心升級亮點
|
| 160 |
|
| 161 |
台灣各級醫療院所的電子病歷、護理交班、社區健檢表上充滿高度在地化的行話與臨床縮寫:
|
| 162 |
- `L-CT`:代表低劑量胸部電腦斷層(肺癌早期篩檢)。
|
|
|
|
| 183 |
|
| 184 |
---
|
| 185 |
|
| 186 |
+
## 雙版本評測成效對照 (Benchmark Comparison)
|
| 187 |
|
| 188 |
| 評測維度/指標 | **IlhaEmbed-311M (Flagship)** | **IlhaEmbed-97M (Edge / Kiosk)** | 歷史開源基準 (jina/ckip/bge) | 發布門禁要求 |
|
| 189 |
|---|---:|---:|---:|---:|
|
|
|
|
| 201 |
|
| 202 |
---
|
| 203 |
|
| 204 |
+
### MTEB 官方基準實測對照 (Massive Text Embedding Benchmark)
|
| 205 |
使用官方最新版 MTEB v2.21.0 針對中文醫療與語意任務進行同機同卡嚴謹對照,對比開源主流標竿模型 **BAAI/bge-small-zh-v1.5**:
|
| 206 |
|
| 207 |
| MTEB 官方評測任務 | 核心指標 | **IlhaEmbed (97M / 37MB)** | **BAAI/bge-small-zh** | 實測結論與意義 |
|
|
|
|
| 216 |
|
| 217 |
> **評測數值常識說明**:在資訊檢索(IR)領域中,MAP / NDCG@10 達 0.45 ~ 0.52 即為全球開源與商業模型(如 BGE-small, OpenAI)之頂尖梯隊。IlhaEmbed 僅以 37MB 體積即在醫療檢索任務超越 BGE-small。
|
| 218 |
|
| 219 |
+
### 隨意病歷速記檢驗 (真實未美化 20 筆複合速記)
|
| 220 |
使用正式發布版模型對 20 筆包含門診、急診、台語主訴、護理給藥等真實複合長句進行檢驗:
|
| 221 |
- **主意圖命中率 (Top-1 Dominant Hit)**:**95.0% (19/20)**
|
| 222 |
- **前三候選覆蓋率 (Top-3 Intent Coverage)**:**100.0% (20/20)**
|
| 223 |
- **邊界失效案例分析**:`「114年成健已做,糞檢MIF未交,建議下次門診補交檢體」` 預測 `checkup_intent` (0.703) 與 `care_event` (0.692) 僅差 0.011。「已做」與「安排」的時態語意在純向量空間邊界模糊,證明臨床落地時必須保留下游規則過濾與醫事人員覆核(Fail-Closed 殘差機制)。
|
| 224 |
|
| 225 |
+
## 快速開始 (Quick Start)
|
| 226 |
|
| 227 |
### 1. Python / Sentence-Transformers(支援雙版本)
|
| 228 |
|
|
|
|
| 271 |
|
| 272 |
---
|
| 273 |
|
| 274 |
+
## 醫療法規、SaMD 豁免與安全邊界
|
| 275 |
|
| 276 |
- **預期用途(Intended Use)**:臨床輔助建議、攝取分流與候選重排(Suggest-with-Review Candidate Ranker)。
|
| 277 |
- **非 SaMD 宣告(Non-SaMD Posture)**:依據台灣衛生福利部食品藥物管理署(TFDA)與國際醫療器材軟體(SaMD)法規指引,IlhaEmbed 不具備自主診斷、疾病處方或獨立醫療決策功能。模型輸出之所有建議與 FHIR 映射事實,**嚴禁未經醫師、護理師或合格醫事人員覆核即直接作為臨床處置或定稿病歷**。
|
|
|
|
| 279 |
|
| 280 |
---
|
| 281 |
|
| 282 |
+
## 資料治理與開源邊界原則
|
| 283 |
|
| 284 |
1. **零受保護健康資訊(Zero PHI)**:模型訓練與評測全流程不包含任何真實病患姓名、身分證號、病歷號或可識別隱私個資。
|
| 285 |
2. **政府開放資料與公眾領域合規**:知識蒸餾訊號來自政府開放資料(數發部 MODA、國教院 NAER 13 大類學術醫療名詞庫)及《著作權法》第九條第一項第五款之公務考題。
|