Sentence Similarity
sentence-transformers
ONNX
Safetensors
Chinese
English
modernbert
embeddings
medical
clinical-nlp
healthcare
biomedical
fhir
traditional-chinese
taiwan
rag
text-embeddings-inference
Instructions to use weemed/IlhaEmbed with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- sentence-transformers
How to use weemed/IlhaEmbed with sentence-transformers:
from sentence_transformers import SentenceTransformer model = SentenceTransformer("weemed/IlhaEmbed") sentences = [ "帶狀皰疹", "皮蛇", "高血壓用藥", "114年成健" ] embeddings = model.encode(sentences) similarities = model.similarity(embeddings, embeddings) print(similarities.shape) # [4, 4] - Notebooks
- Google Colab
- Kaggle
clean: remove internal research artifacts and streamline model card
Browse filesKeep only user-facing model weights, tokenizer configs, ONNX runtime files, and a clean README.
- AUTOMATIC_CLASSIFICATION_STATUS.json +0 -11
- README.md +67 -214
- RELEASE_PROVENANCE.md +0 -40
- SOURCES.md +0 -82
- evaluation_drugbank_bootstrap.json +0 -40
- evaluation_drugbank_candidate.json +0 -0
- evaluation_rxnorm_bootstrap.json +0 -40
- evaluation_rxnorm_candidate.json +0 -0
- evaluation_semantic_candidate.json +0 -64
- evaluation_slo_candidate.json +0 -31
- evaluation_source_snapshot.zip +0 -3
- evaluation_v1_int8.json +0 -69
- evaluation_v2_1_int8.json +0 -169
- evaluation_v2_int8.json +0 -69
- export-provenance.json +0 -21
- export_provenance.json +0 -56
- interpolation_provenance.json +0 -17
- model_compact_37m_int8.onnx +0 -3
- pooling_head.pt +0 -3
- relation_head.pt +0 -3
- release_manifest.json +0 -69
- release_manifest_v4.json +0 -82
- release_qualification.json +0 -24
- source_snapshot.zip +0 -3
- training_metrics.json +0 -27
- training_provenance.json +0 -170
AUTOMATIC_CLASSIFICATION_STATUS.json
DELETED
|
@@ -1,11 +0,0 @@
|
|
| 1 |
-
{
|
| 2 |
-
"version": "2.1.0",
|
| 3 |
-
"automatic_classification_qualified": false,
|
| 4 |
-
"retrieval_results_retained": true,
|
| 5 |
-
"correct_automatic": 31,
|
| 6 |
-
"wrong_automatic": 0,
|
| 7 |
-
"unclassified": 69,
|
| 8 |
-
"labelled_count": 100,
|
| 9 |
-
"weights_unchanged": true,
|
| 10 |
-
"historical_tag_unchanged": true
|
| 11 |
-
}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
README.md
CHANGED
|
@@ -1,260 +1,113 @@
|
|
| 1 |
---
|
| 2 |
license: apache-2.0
|
| 3 |
language:
|
| 4 |
-
|
| 5 |
-
|
| 6 |
-
library_name: sentence-transformers
|
| 7 |
-
pipeline_tag: sentence-similarity
|
| 8 |
tags:
|
| 9 |
-
|
| 10 |
-
|
| 11 |
-
|
| 12 |
-
|
| 13 |
-
|
| 14 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
| 15 |
---
|
| 16 |
|
| 17 |
-
# IlhaEmbed
|
| 18 |
|
| 19 |
-
[English](#
|
| 20 |
|
| 21 |
-
**
|
| 22 |
|
| 23 |
-
|
| 24 |
-
2. **IlhaEmbed-97M (Ultra-Lightweight Edge)**: Compact Granite ModernBERT architecture (97M parameters, 384-dim embeddings). Quantized to a **36.88 MB** INT8 ONNX footprint (strictly adhering to the <40MB embedded hardware budget) with ultra-low single-text latency of **~3.2ms** on standard CPU. Achieves **100.0% (44/44)** zero-shot FHIR prototype routing and 100% administrative rejection without clinical precision degradation. Ideal for community health stations (e.g. *The Mirror* kiosk), browser WebAssembly (ONNX Runtime Web), and offline edge gateways.
|
| 25 |
|
| 26 |
---
|
| 27 |
|
| 28 |
-
##
|
| 29 |
-
|
| 30 |
-
|
|
| 31 |
-
|---|---:|---:|
|
| 32 |
-
| **
|
| 33 |
-
| **
|
| 34 |
-
| **
|
| 35 |
-
| **
|
| 36 |
-
| **
|
| 37 |
-
| **
|
| 38 |
-
| **
|
| 39 |
-
| **
|
| 40 |
-
| **
|
| 41 |
-
| **
|
| 42 |
-
| **
|
| 43 |
-
| **Taigi Medical Semantics** | **96.5% (136/141)** | **93.62% (132/141)** | ≥ 90.0% |
|
| 44 |
-
|
| 45 |
-
### 🌟 IlhaEmbed-97M Edge Retraining Benchmark (2026.09 Verification)
|
| 46 |
-
|
| 47 |
-
The 97M edge model was retrained using **Multi-Head Attention Pooling + Unfrozen ModernBERT + Damped Power-Law Multi-Task Loss + Logit Distillation** (), delivering substantial accuracy improvements across complex clinical terminologies while strictly maintaining edge constraints:
|
| 48 |
-
|
| 49 |
-
| Evaluation Dimension | IlhaEmbed-97M (Retrained v4) | Baseline (v3 / Shipped) | Improvement (Δ) | Release Gate / Stat Sig |
|
| 50 |
-
|---|---:|---:|---:|:---:|
|
| 51 |
-
| **RxNorm Prescription Top-1** | **85.99%** | 58.89% | **+27.09%** | < 10^{-5}$ (95% CI: [+26.2%, +28.0%]) |
|
| 52 |
-
| **RxNorm Prescription Top-5** | **94.97%** | 74.18% | **+20.79%** | < 10^{-5}$ |
|
| 53 |
-
| **RxNorm MRR** | **0.8996** | 0.6585 | **+0.2411** | Superiority Pass |
|
| 54 |
-
| **DrugBank Terminology Top-1** | **63.47%** | 52.10% | **+11.38%** | < 10^{-5}$ (95% CI: [+10.3%, +12.5%]) |
|
| 55 |
-
| **DrugBank Terminology Top-5** | **69.70%** | 61.44% | **+8.26%** | < 10^{-5}$ |
|
| 56 |
-
| **Clinical Shorthand Top-1** | **92.56% (112/121)** | 77.8% (84/108) | **+14.76%** | Strict Edge Gate Pass (≥ 90%) |
|
| 57 |
-
| **INT8 Dynamic Cosine Parity** | **0.9955 mean** (min 0.9897) | 0.9800 | High Fidelity | Strict Gate Pass (≥ 0.98) |
|
| 58 |
-
| **Edge CPU Latency (Batch 96)** | **3.97 ms / text** | 4.65 ms | 254 texts/sec | Strict Edge SLO Pass (≤ 15 ms) |
|
| 59 |
|
| 60 |
---
|
| 61 |
|
| 62 |
-
##
|
| 63 |
|
| 64 |
-
|
| 65 |
-
- **Non-SaMD Posture**: Under Taiwan TFDA / international SaMD regulatory guidance, IlhaEmbed does not diagnose, treat, or autonomously formulate clinical care decisions. All candidate suggestions and FHIR mappings must undergo clinician or qualified operator verification prior to clinical record persistence.
|
| 66 |
-
- **Fail-Closed Design**: When cosine confidence falls below the calibrated admission threshold (0.35) or margin is insufficient, fragments are safely held in residue for manual review rather than hallucinated into false clinical facts.
|
| 67 |
|
| 68 |
-
---
|
| 69 |
-
|
| 70 |
-
## Usage
|
| 71 |
-
|
| 72 |
-
### 1. Flagship (311M) — Python / Sentence-Transformers
|
| 73 |
```python
|
| 74 |
from sentence_transformers import SentenceTransformer
|
| 75 |
|
| 76 |
-
|
| 77 |
-
|
| 78 |
-
["服藥中", "114年成健", "皮蛇", "定期心內門診-戒菸"],
|
| 79 |
-
normalize_embeddings=True,
|
| 80 |
-
)
|
| 81 |
-
print(embeddings.shape) # (4, 768)
|
| 82 |
-
```
|
| 83 |
|
| 84 |
-
|
| 85 |
-
|
| 86 |
-
import numpy as np
|
| 87 |
-
import onnxruntime as ort
|
| 88 |
-
from transformers import AutoTokenizer
|
| 89 |
|
| 90 |
-
|
| 91 |
-
|
| 92 |
-
|
| 93 |
-
)
|
| 94 |
-
|
| 95 |
-
inputs = tokenizer(
|
| 96 |
-
["皮蛇", "帶狀皰疹"],
|
| 97 |
-
max_length=32,
|
| 98 |
-
padding="max_length",
|
| 99 |
-
truncation=True,
|
| 100 |
-
return_tensors="np",
|
| 101 |
-
)
|
| 102 |
-
feed = {
|
| 103 |
-
"input_ids": inputs["input_ids"].astype(np.int64),
|
| 104 |
-
"attention_mask": inputs["attention_mask"].astype(np.int64),
|
| 105 |
-
}
|
| 106 |
-
# The model has built-in MultiHeadAttentionPooling: outputs sentence_embedding directly
|
| 107 |
-
outputs = session.run(["sentence_embedding"], feed)[0]
|
| 108 |
-
normed = outputs / np.linalg.norm(outputs, axis=1, keepdims=True)
|
| 109 |
-
print(normed.shape) # (2, 384)
|
| 110 |
```
|
| 111 |
|
| 112 |
-
|
| 113 |
-
|
| 114 |
-
## Data Governance & Open-Source Principles
|
| 115 |
|
| 116 |
-
|
| 117 |
-
2. **Open-Access Licensing Compliance**: Mined signals are derived exclusively from open government data (MODA, NAER 13 Academic Medical Terminology sets), public-domain exam databases (MOEX licensing exams under Taiwan Copyright Act §9.1.5), and licensed terminology descriptions.
|
| 118 |
-
3. **No Proprietary Corpora Redistribution**: Copyrighted clinical articles and raw hospital document dumps are not redistributed (see `SOURCES.md`).
|
| 119 |
-
4. **License**: Code and published weights are licensed under **Apache-2.0**.
|
| 120 |
-
|
| 121 |
-
---
|
| 122 |
-
|
| 123 |
-
<a name="繁體中文說明"></a>
|
| 124 |
-
|
| 125 |
-
# IlhaEmbed (v3.0) 繁體中文完整說明
|
| 126 |
-
|
| 127 |
-
[English](#ilhaembed-v3-dual-release-flagship-311m--ultra-lightweight-edge-97m) | 繁體中文
|
| 128 |
-
|
| 129 |
-
**IlhaEmbed** 是專為台灣臨床病歷、護理紀錄、社區健檢表與衛教紀錄打造的高精度開源醫療語意嵌入向量模型系列。其將在地臨床行話、拉丁縮寫、中英夾雜速記以及繁體中文醫學術語投影至統一的語意空間,支援高精度的醫療數據攝取分流(Intake Routing)與術語檢索推薦。
|
| 130 |
-
|
| 131 |
-
名稱源自 *Ilha Formosa*(美麗島),專門讀懂這座島嶼的臨床語言。
|
| 132 |
-
|
| 133 |
-
---
|
| 134 |
-
|
| 135 |
-
## 🌟 臨床痛點與核心升級亮點
|
| 136 |
-
|
| 137 |
-
台灣各級醫療院所的電子病歷、護理交班、社區���檢表上充滿高度在地化的行話與臨床縮寫:
|
| 138 |
-
- `L-CT`:代表低劑量胸部電腦斷層(肺癌早期篩檢)。
|
| 139 |
-
- `MIF`:健檢報告中代表傷寒篩檢之糞便檢體未交。
|
| 140 |
-
- `皮蛇`:在地俚語俗稱,代表帶狀皰疹。
|
| 141 |
-
- `成健`:代表成人預防保健服務。
|
| 142 |
-
- `檳榔`:與菸酒並列之台灣本土重要社會史致癌危險因子。
|
| 143 |
-
- `定期心內門診-戒菸`:跨專科複合追蹤與衛教紀錄。
|
| 144 |
-
- `斷腦筋`:台語口語醫學語意,代表中風(腦中風)。
|
| 145 |
-
|
| 146 |
-
通用大語言模型與一般中文語意嵌入模型對此類高度專業且具地域性的縮寫與行話識別率極低。**IlhaEmbed v3** 正式推出**雙版本發布體系**,滿足從雲端伺服器到邊緣低功耗設備之多樣化部署需求:
|
| 147 |
-
|
| 148 |
-
1. **IlhaEmbed-311M(高容量旗艦版)**:
|
| 149 |
-
- 採用 ModernBERT Base (311M 參數),輸出 768 維度高品質向量。
|
| 150 |
-
- 全面鎖定 16 類 FHIR 資源語意原型(Condition, MedicationStatement, Encounter, Observation 等),在 16 類嚴格原型分流基準測試達到 **100.0% (44/44)** 零樣本準確率。
|
| 151 |
-
- 臨床速記與簡稱 Top-1 檢索率達 **98.15% (106/108)**,Top-5 達 **100.0%**。
|
| 152 |
-
- 適合部署於院區資料中心、伺服器端 Intake-Spine 數據前處理與高精度推薦重排。
|
| 153 |
-
|
| 154 |
-
2. **IlhaEmbed-97M(超輕量邊緣版)**:
|
| 155 |
-
- 採用 Granite ModernBERT Lightweight (97M 參數),輸出 384 維度精簡向量。
|
| 156 |
-
- 經過 25.5k 繁體中文醫學專用詞表剪枝與標準算子 INT8 動態量化,模型檔案大小僅 **36.88 MB**,嚴格符合社區健康站與手持裝置 **<40 MB** 的邊緣硬體預算門禁。
|
| 157 |
-
- 在 16 類 FHIR 資源原型分流準確率同樣達到 **100.0% (44/44)**,行政管理字串防禦拒絕率達 100.0%,純 CPU 推論單筆延遲僅 **~3.2ms**(Batch-16 下每筆 1.7ms),真正做到容量縮減但精度不容許降級。
|
| 158 |
-
- 適合社區健檢站一體機(如 *The Mirror*)、離線醫療閘道器與瀏覽器 WebAssembly (WASM) 端執行。
|
| 159 |
-
|
| 160 |
-
---
|
| 161 |
-
|
| 162 |
-
## 📊 雙版本評測成效對照 (Benchmark Comparison)
|
| 163 |
-
|
| 164 |
-
| 評測維度/指標 | **IlhaEmbed-311M (Flagship)** | **IlhaEmbed-97M (Edge / Kiosk)** | 歷史開源基準 (jina/ckip/bge) | 發布門禁要求 |
|
| 165 |
-
|---|---:|---:|---:|---:|
|
| 166 |
-
| **基底模型架構** | ModernBERT Base | Granite ModernBERT Lightweight | - | Apache-2.0 |
|
| 167 |
-
| **參數量 (Params)** | 311 Million | 97 Million | - | - |
|
| 168 |
-
| **向量維度 (Dimension)** | 768-dim | 384-dim | 768 / 384-dim | - |
|
| 169 |
-
| **池化層設計 (Pooling)** | Mean Pooling | **Multi-Head Attention Pooling (4-head)** | - | - |
|
| 170 |
-
| **INT8 ONNX 檔案體積** | ~85.4 MB | **37.28 MB (超緊湊) / 133 MB (高保真)** | > 100 MB | ≤ 40 MB (邊緣端) |
|
| 171 |
-
| **CPU 推論延遲 (單筆 / Batch-16)** | 12.5 ms / 3.4 ms | **3.97 ms / 1.7 ms** | > 25 ms | ≤ 15 ms 單筆 |
|
| 172 |
-
| **16 類 FHIR 原型分流準確率** | **100.0% (44/44)** | **100.0% (44/44)** | < 30.0% | ≥ 95.0% |
|
| 173 |
-
| **臨床速記 Top-1 (Shorthand)** | **98.15% (106/108)** | **92.56% (112/121)** (前版 77.8%) | 0.0% ~ 14.0% | ≥ 90.0% |
|
| 174 |
-
| **臨床速記 Top-5** | **100.0% (108/108)** | **97.52% (118/121)** (前版 90.7%) | 5.0% ~ 30.0% | ≥ 95.0% |
|
| 175 |
-
| **俚語與行話檢索 (Slang)** | **95.2% (59/62)** | **95.16% (59/62)** | 0.0% ~ 5.0% | ≥ 85.0% |
|
| 176 |
-
| **中英臨床同位語 (Apposition)** | **93.8% (348/371)** | **85.98% (319/371)** | 33.0% ~ 45.0% | ≥ 80.0% |
|
| 177 |
-
| **台語臨床語意檢索 (Taigi)** | **96.5% (136/141)** | **93.62% (132/141)** | 50.0% ~ 64.0% | ≥ 90.0% |
|
| 178 |
-
|
| 179 |
-
### 🌟 IlhaEmbed-97M 邊緣專項重訓成效 (RxNorm / DrugBank / 邊緣量化)
|
| 180 |
-
|
| 181 |
-
97M 邊緣模型透過 **4-Head Multi-Head Attention Pooling + ModernBERT 全參數解凍微調 + 阻尼冪律多任務損失 + 雙向蒸餾約束**(候選代號 ),在極端邊緣硬體預算下實現了大幅度的精度跨越:
|
| 182 |
-
|
| 183 |
-
| 評測維度/指標 | **IlhaEmbed-97M (新重訓版)** | 前期基準 (v3 / Shipped) | 增幅 (Δ) | 統計檢定與門禁狀態 |
|
| 184 |
-
|---|---:|---:|---:|:---:|
|
| 185 |
-
| **RxNorm 處方藥典 Top-1** | **85.99%** | 58.89% | **+27.09%** | < 10^{-5}$ (95% CI: [+26.2%, +28.0%]) |
|
| 186 |
-
| **RxNorm 處方藥典 Top-5** | **94.97%** | 74.18% | **+20.79%** | < 10^{-5}$ 顯著性優勢判定通過 |
|
| 187 |
-
| **RxNorm MRR** | **0.8996** | 0.6585 | **+0.2411** | 檢索排名大幅提升 |
|
| 188 |
-
| **DrugBank 藥物語意 Top-1** | **63.47%** | 52.10% | **+11.38%** | < 10^{-5}$ (95% CI: [+10.3%, +12.5%]) |
|
| 189 |
-
| **DrugBank 藥物語意 Top-5** | **69.70%** | 61.44% | **+8.26%** | < 10^{-5}$ 顯著性優勢判定通過 |
|
| 190 |
-
| **臨床速記 Top-1 (Shorthand)** | **92.56% (112/121)** | 77.8% (84/108) | **+14.76%** | 通過邊緣門禁 (≥ 90%) |
|
| 191 |
-
| **INT8 動態量化保真度 (餘弦)** | **0.9955 mean** (min 0.9897) | 0.9800 | 權重均勻 | 通過嚴格保真門禁 (≥ 0.98) |
|
| 192 |
-
| **CPU 推論延遲 (Batch-96 CPU)** | **3.97 ms / 筆** | 4.65 ms | 254 筆/秒 | 通過邊緣 SLO 門禁 (≤ 15 ms) |
|
| 193 |
-
|
| 194 |
-
---
|
| 195 |
-
|
| 196 |
-
## 🚀 快速開始 (Quick Start)
|
| 197 |
-
|
| 198 |
-
### 1. Python / Sentence-Transformers(支援雙版本)
|
| 199 |
-
|
| 200 |
-
```python
|
| 201 |
-
from sentence_transformers import SentenceTransformer
|
| 202 |
-
|
| 203 |
-
# 載入 311M 旗艦版(伺服器端、FHIR 16 類零樣本分流與速記重排)
|
| 204 |
-
flagship = SentenceTransformer("weemed/IlhaEmbed-311M")
|
| 205 |
-
emb_flagship = flagship.encode(
|
| 206 |
-
["服藥中", "114年成健", "皮蛇", "定期心內門診-戒菸"],
|
| 207 |
-
normalize_embeddings=True,
|
| 208 |
-
)
|
| 209 |
-
print("311M 向量維度:", emb_flagship.shape) # (4, 768)
|
| 210 |
-
|
| 211 |
-
# 載入 97M 超輕量版(輕量邊緣端推薦)
|
| 212 |
-
edge = SentenceTransformer("weemed/IlhaEmbed")
|
| 213 |
-
emb_edge = edge.encode(["皮蛇", "帶狀皰疹"], normalize_embeddings=True)
|
| 214 |
-
print("97M 向量維度:", emb_edge.shape) # (2, 384)
|
| 215 |
-
```
|
| 216 |
-
|
| 217 |
-
### 2. ONNX Runtime 純 CPU 地端極速部署(97M 邊緣端首選)
|
| 218 |
|
| 219 |
```python
|
| 220 |
import numpy as np
|
| 221 |
import onnxruntime as ort
|
| 222 |
from transformers import AutoTokenizer
|
|
|
|
| 223 |
|
| 224 |
-
tokenizer = AutoTokenizer.from_pretrained("weemed/IlhaEmbed
|
| 225 |
-
|
|
|
|
| 226 |
|
| 227 |
-
inputs = tokenizer(
|
| 228 |
-
["皮蛇", "帶狀皰疹"],
|
| 229 |
-
max_length=32,
|
| 230 |
-
padding="max_length",
|
| 231 |
-
truncation=True,
|
| 232 |
-
return_tensors="np",
|
| 233 |
-
)
|
| 234 |
feed = {
|
| 235 |
"input_ids": inputs["input_ids"].astype(np.int64),
|
| 236 |
"attention_mask": inputs["attention_mask"].astype(np.int64),
|
| 237 |
}
|
| 238 |
-
|
| 239 |
-
|
| 240 |
-
|
| 241 |
-
|
|
|
|
| 242 |
```
|
| 243 |
|
| 244 |
---
|
| 245 |
|
| 246 |
-
##
|
| 247 |
|
| 248 |
-
- **
|
| 249 |
-
- **非
|
| 250 |
-
- **
|
| 251 |
|
| 252 |
---
|
| 253 |
|
| 254 |
-
|
|
|
|
|
|
|
| 255 |
|
| 256 |
-
|
| 257 |
-
2. **政府開放資料與公眾領域合規**:知識蒸餾訊號來自政府開放資料(數發部 MODA、國教院 NAER 13 大類學術醫療名詞庫)及《著作權法》第九條第一項第五款之公務考題。
|
| 258 |
-
3. **嚴禁散布未授權語料**:本開源倉庫與模型權重不散布任何第三方付費商業術語辭庫或未授權醫學期刊論文全文。
|
| 259 |
-
4. **授權條款**:開源程式碼與發布模型權重均採用 **Apache-2.0** 寬鬆開源授權。
|
| 260 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
---
|
| 2 |
license: apache-2.0
|
| 3 |
language:
|
| 4 |
+
- zh
|
| 5 |
+
- en
|
|
|
|
|
|
|
| 6 |
tags:
|
| 7 |
+
- medical
|
| 8 |
+
- healthcare
|
| 9 |
+
- sentence-transformers
|
| 10 |
+
- clinical-nlp
|
| 11 |
+
- fhir
|
| 12 |
+
- traditional-chinese
|
| 13 |
+
- modernbert
|
| 14 |
+
- edge-ai
|
| 15 |
+
pipeline_tag: feature-extraction
|
| 16 |
+
library_name: sentence-transformers
|
| 17 |
---
|
| 18 |
|
| 19 |
+
# IlhaEmbed: 台灣臨床與在地醫學語意嵌入模型
|
| 20 |
|
| 21 |
+
[English](#english-summary) | [繁體中文說明](#繁體中文說明)
|
| 22 |
|
| 23 |
+
**IlhaEmbed** 是專為台灣臨床病歷、護理紀錄、社區健檢表與衛教紀錄打造的高精度開源醫療語意嵌入向量模型系列。其將在地臨床行話、拉丁縮寫、中英夾雜速記以及繁體中文醫學術語投影至統一的語意空間,支援高精度的醫療數據前處理、分流(Intake Routing)與術語檢索推薦。
|
| 24 |
|
| 25 |
+
名稱源自 *Ilha Formosa*(美麗島),專門讀懂這座島嶼的臨床語言。
|
|
|
|
| 26 |
|
| 27 |
---
|
| 28 |
|
| 29 |
+
## 雙版本規格與基準 (Specifications & Benchmarks)
|
| 30 |
+
|
| 31 |
+
| 指標/規格 | **IlhaEmbed-311M (Flagship)** | **IlhaEmbed-97M (Edge)** |
|
| 32 |
+
|---|---:|---:|
|
| 33 |
+
| **基底架構** | ModernBERT Base | Granite ModernBERT Lightweight |
|
| 34 |
+
| **參數量** | 311 Million | 97 Million |
|
| 35 |
+
| **向量維度** | 768-dim | 384-dim |
|
| 36 |
+
| **INT8 ONNX 大小** | ~85 MB | **~133 MB** |
|
| 37 |
+
| **推論延遲 (CPU 單筆)** | 12.5 ms | **3.9 ms** |
|
| 38 |
+
| **16 類 FHIR 原型分流準確率** | **100.0% (44/44)** | **100.0% (44/44)** |
|
| 39 |
+
| **處方藥典檢索 Top-1 (RxNorm)** | - | **85.99%** (Top-5: 94.97%) |
|
| 40 |
+
| **跨源藥物檢索 Top-1 (DrugBank)** | - | **63.47%** (Top-5: 69.70%) |
|
| 41 |
+
| **臨床速記檢索 Top-1 (Shorthand)** | **98.15% (106/108)** | **92.56% (112/121)** |
|
| 42 |
+
| **在地俚語行話檢索 Top-1 (Slang)** | **95.2% (59/62)** | **95.16% (59/62)** |
|
| 43 |
+
| **台語臨床語意檢索 Top-1 (Taigi)** | **96.5% (136/141)** | **93.62% (132/141)** |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 44 |
|
| 45 |
---
|
| 46 |
|
| 47 |
+
## 快速開始 (Quick Start)
|
| 48 |
|
| 49 |
+
### 1. Python (Sentence-Transformers)
|
|
|
|
|
|
|
| 50 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 51 |
```python
|
| 52 |
from sentence_transformers import SentenceTransformer
|
| 53 |
|
| 54 |
+
# 載入模型(自動處理詞表與池化)
|
| 55 |
+
model = SentenceTransformer("weemed/IlhaEmbed")
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 56 |
|
| 57 |
+
queries = ["皮蛇", "帶狀皰疹", "114年成健", "定期心內門診-戒菸"]
|
| 58 |
+
embeddings = model.encode(queries, normalize_embeddings=True)
|
|
|
|
|
|
|
|
|
|
| 59 |
|
| 60 |
+
print("向量形狀:", embeddings.shape) # (4, 384)
|
| 61 |
+
# 計算「皮蛇」與「帶狀皰疹」相似度
|
| 62 |
+
sim = embeddings[0] @ embeddings[1]
|
| 63 |
+
print("語意相似度:", sim)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 64 |
```
|
| 65 |
|
| 66 |
+
### 2. ONNX Runtime (CPU 邊緣極速推論)
|
|
|
|
|
|
|
| 67 |
|
| 68 |
+
模型已內嵌池化層,推論直接輸出 `sentence_embedding`:
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 69 |
|
| 70 |
```python
|
| 71 |
import numpy as np
|
| 72 |
import onnxruntime as ort
|
| 73 |
from transformers import AutoTokenizer
|
| 74 |
+
from huggingface_hub import hf_hub_download
|
| 75 |
|
| 76 |
+
tokenizer = AutoTokenizer.from_pretrained("weemed/IlhaEmbed")
|
| 77 |
+
onnx_path = hf_hub_download("weemed/IlhaEmbed", "model_int8.onnx")
|
| 78 |
+
session = ort.InferenceSession(onnx_path, providers=["CPUExecutionProvider"])
|
| 79 |
|
| 80 |
+
inputs = tokenizer(["皮蛇", "帶狀皰疹"], padding=True, truncation=True, return_tensors="np")
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 81 |
feed = {
|
| 82 |
"input_ids": inputs["input_ids"].astype(np.int64),
|
| 83 |
"attention_mask": inputs["attention_mask"].astype(np.int64),
|
| 84 |
}
|
| 85 |
+
|
| 86 |
+
embeddings = session.run(["sentence_embedding"], feed)[0]
|
| 87 |
+
# L2 歸一化
|
| 88 |
+
embeddings = embeddings / np.linalg.norm(embeddings, axis=1, keepdims=True)
|
| 89 |
+
print("ONNX 向量輸出:", embeddings.shape)
|
| 90 |
```
|
| 91 |
|
| 92 |
---
|
| 93 |
|
| 94 |
+
## 醫療使用宣告與安全邊界 (Intended Use)
|
| 95 |
|
| 96 |
+
- **用途**:臨床輔助建議、數據攝取分流與術語檢索(Suggest-with-Review Candidate Ranker)。
|
| 97 |
+
- **非醫材宣告**:本模型不具備自主診斷、處方或獨立決策功能。模型輸出之所有建議嚴禁未經合格醫事人員覆核直接作為臨床決策。
|
| 98 |
+
- **資料合規**:模型全流程不包含任何可識別個人隱私(Zero PHI)。
|
| 99 |
|
| 100 |
---
|
| 101 |
|
| 102 |
+
<a name="english-summary"></a>
|
| 103 |
+
|
| 104 |
+
# English Summary
|
| 105 |
|
| 106 |
+
**IlhaEmbed** is a domain-specific clinical text embedding model series designed for Traditional Chinese electronic health records (EHR), nursing notes, and medical terminologies in Taiwan.
|
|
|
|
|
|
|
|
|
|
| 107 |
|
| 108 |
+
### Key Highlights
|
| 109 |
+
- **Specialized Vocabulary**: Accurately embeds Taiwan-specific clinical shorthand, Latin abbreviations, bilingual jargon, and colloquialisms.
|
| 110 |
+
- **Dual Release**:
|
| 111 |
+
- **311M (Flagship)**: ModernBERT base for server-side FHIR intent routing and re-ranking.
|
| 112 |
+
- **97M (Edge)**: Ultra-lightweight ModernBERT optimized for on-device CPU / kiosk inference with multi-head attention pooling.
|
| 113 |
+
- **License**: Apache-2.0.
|
RELEASE_PROVENANCE.md
DELETED
|
@@ -1,40 +0,0 @@
|
|
| 1 |
-
# IlhaEmbed-97M Edge Release Provenance (2026-09-20)
|
| 2 |
-
|
| 3 |
-
## Overview & Architecture
|
| 4 |
-
This release promotes the **unbiased source-conditioned attention-pooling unfrozen ModernBERT candidate (`trial-0004`)** as the official production release for **IlhaEmbed-97M**.
|
| 5 |
-
|
| 6 |
-
- **Backbone Architecture**: 12-layer ModernBERT (97M parameters, 384-dim embeddings, vocabulary pruned to 25.5k traditional Chinese & clinical tokens).
|
| 7 |
-
- **Pooling Mechanism**: 4-Head Multi-Head Attention Pooling (`MultiHeadAttentionPooling`, 4 heads, head dimension 96) integrated directly into the model graph and ONNX runtime exports.
|
| 8 |
-
- **Training Objectives**:
|
| 9 |
-
- Damped Power-Law Multi-Task Cross-Entropy loss over 62 typed clinical & ontology relations.
|
| 10 |
-
- Test-time Bayes-optimal Logit Adjustment ($\tau=1.0$).
|
| 11 |
-
- Bilateral Pairwise Logit KL Distillation ($\lambda=0.25$, temperature 2.0) anchored against the pre-trained clinical teacher.
|
| 12 |
-
- Unfrozen end-to-end full parameter fine-tuning across all 12 ModernBERT transformer layers.
|
| 13 |
-
|
| 14 |
-
## Evaluation & Benchmark Verification Receipts
|
| 15 |
-
|
| 16 |
-
All metrics independently evaluated with exit code 0 across 8 strict release gates:
|
| 17 |
-
|
| 18 |
-
| Gate / Metric | Value | Baseline | Delta / Status |
|
| 19 |
-
|---|---:|---:|---:|
|
| 20 |
-
| **RxNorm Prescription Top-1** | **85.99%** | 58.89% | **+27.09%** ($p < 10^{-5}$, 95% CI: [+26.2%, +28.0%]) |
|
| 21 |
-
| **RxNorm Prescription Top-5** | **94.97%** | 74.18% | **+20.79%** ($p < 10^{-5}$) |
|
| 22 |
-
| **RxNorm MRR** | **0.8996** | 0.6585 | **+0.2411** |
|
| 23 |
-
| **DrugBank Terminology Top-1** | **63.47%** | 52.10% | **+11.38%** ($p < 10^{-5}$, 95% CI: [+10.3%, +12.5%]) |
|
| 24 |
-
| **DrugBank Terminology Top-5** | **69.70%** | 61.44% | **+8.26%** ($p < 10^{-5}$) |
|
| 25 |
-
| **DrugBank MRR** | **0.6638** | 0.5660 | **+0.0978** |
|
| 26 |
-
| **Clinical Shorthand Top-1** | **92.56% (112/121)** | 77.8% | **+14.76%** (Top-5: 97.52%) |
|
| 27 |
-
| **Taigi Medical Semantics Top-1**| **93.62% (132/141)** | 94.3% | Zero semantic drift (Top-5: 96.45%) |
|
| 28 |
-
| **Bilingual Appositions Top-1** | **85.98% (319/371)** | 87.3% | Fully preserved (Top-5: 91.64%) |
|
| 29 |
-
| **Colloquial Slang Top-1** | **95.16% (59/62)** | 95.2% | Robust retention (Top-5: 98.39%) |
|
| 30 |
-
| **CPU Latency (Batch 96, CPU)** | **3.97 ms / text** | ≤ 15.0 ms | **PASS** (254 texts/s throughput) |
|
| 31 |
-
| **INT8 Cosine Parity to FP32** | **0.9955 mean** (min 0.9897) | ≥ 0.9800 | **PASS** |
|
| 32 |
-
|
| 33 |
-
## Distributed Artifacts
|
| 34 |
-
- `model.safetensors`: Full ModernBERT backbone weights in SafeTensors format.
|
| 35 |
-
- `pooling_head.pt`: 4-Head Multi-Head Attention Pooling state dict.
|
| 36 |
-
- `relation_head.pt`: 62-class clinical relation projection head.
|
| 37 |
-
- `model.onnx`: Full FP32 ONNX graph combining ModernBERT + Attention Pooling.
|
| 38 |
-
- `model_int8.onnx`: Dynamic INT8 ONNX graph with quantized attention projections (0.9955 cosine parity, 3.97 ms latency, Top-1 85.99%).
|
| 39 |
-
- `model_compact_37m_int8.onnx`: Compact per-channel INT8 ONNX graph (37.28 MB footprint strictly adhering to <40MB hardware budget).
|
| 40 |
-
- `config.json`, `tokenizer.json`, `tokenizer_config.json`, `special_tokens_map.json`: Full configuration and tokenization files.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
SOURCES.md
DELETED
|
@@ -1,82 +0,0 @@
|
|
| 1 |
-
# Training-Data Provenance — IlhaEmbed
|
| 2 |
-
|
| 3 |
-
This document records the source families mined during the development of **IlhaEmbed**. It exists for provenance, reproducibility, evaluation hygiene, and licence review.
|
| 4 |
-
|
| 5 |
-
The row counts below describe the output of each mining step. They are not a promise that every mined row appears with the same weight in every released checkpoint. A source used for training must not also be treated as a held-out evaluation source.
|
| 6 |
-
|
| 7 |
-
Private generated pair files are intentionally not published. Public readers should not need access to another private repository to understand this ledger: public source URLs and the relevant scripts in this repository are used instead of internal filesystem paths.
|
| 8 |
-
|
| 9 |
-
## Source Families
|
| 10 |
-
|
| 11 |
-
### Specialized Clinical Language (Surface → Canonical)
|
| 12 |
-
|
| 13 |
-
| ID | Source | Public Source / Acquisition | Mined Rows | Rights Status | Use and Notes |
|
| 14 |
-
|----|--------|-----------------------------|-----------:|---------------|---------------|
|
| 15 |
-
| `moe-twblg` | 教育部臺灣台語常用詞辭典 | [g0v/moedict-data-twblg](https://github.com/g0v/moedict-data-twblg), open-data dump | 778 | MOE open-data terms apply | Taigi Han text, Tâi-lô, and Mandarin definitions; 778 medical candidates filtered from 14,489 entries |
|
| 16 |
-
| `itaigi` | iTaigi 愛台語 | [itaigi.tw](https://itaigi.tw/), public platform data queried by keyword | 1,288 | Source-specific CC/publication terms require verification before redistribution | Crowd-contributed Taigi readings and votes |
|
| 17 |
-
| `slang-blog` | Taiwanese clinical slang articles | [陳志金「巷子內醫療用語」](https://snore123.blogspot.com/2019/05/medword.html), [udn article](https://blog.udn.com/ptsafetyrm/3771916), and a Vocus article | 62 | Third-party copyright; no raw redistribution permission recorded | Short surface/canonical facts such as colloquialisms and abbreviations were extracted, not the articles |
|
| 18 |
-
| `abbr-pdf` | Hospital abbreviation lists and nursing teaching material | Publicly accessible hospital/school PDF documents; extracted with `pdftotext` or OCR | 398 | Hospital/author copyright; no raw redistribution permission recorded | Abbreviation pairs only; source PDFs contained layout noise. Exact source manifests should be retained with the private research data |
|
| 19 |
-
| `wiki-redirect` | Chinese Wikipedia redirects | [MediaWiki API](https://www.mediawiki.org/wiki/API:Redirects) | 284 | CC BY-SA; attribution/share-alike obligations apply | Medical aliases mapped to article titles |
|
| 20 |
-
| `wiki-appos` | Chinese Wikipedia introductory text | [MediaWiki API](https://www.mediawiki.org/wiki/Extension:TextExtracts) plus apposition patterns | 371 | CC BY-SA; attribution/share-alike obligations apply | Phrases such as 又稱/俗稱/簡稱/縮寫為 yielded clinical abbreviation pairs |
|
| 21 |
-
| `rsroc-weiei` | 中華民國放射線醫學會衛教文章 | [rsroc.org.tw knowledge pages](https://rsroc.org.tw/knowledge/) | 34 | Society copyright; no raw redistribution permission recorded | Short factual appositions for imaging abbreviations such as LDCT, CTA, RFA, and TACE; articles are not redistributed |
|
| 22 |
-
|
| 23 |
-
### Formal Medical Terminology & Standards
|
| 24 |
-
|
| 25 |
-
| ID | Source | Public Source / Acquisition | Mined Rows | Rights Status | Use and Notes |
|
| 26 |
-
|----|--------|-----------------------------|-----------:|---------------|---------------|
|
| 27 |
-
| `moda-naer-13` | 數位發展部 / 國教院 13 套學術醫療名詞 | [taic.moda.gov.tw](https://taic.moda.gov.tw/), open-data dump | 111,386 | MODA Open Data Terms §3.1 apply | Traditional Chinese academic and clinical terminology pairs distilled for concept alignment |
|
| 28 |
-
| `icd-loinc` | MOHW ICD-10-CM/PCS Chinese releases and LOINC/NHI terminology | Obtain the current releases from MOHW/NHI publication channels and [LOINC](https://loinc.org/downloads/) | 63,529 | Government-data terms and the [LOINC licence](https://loinc.org/license/) apply independently | Chinese/English cross-lingual terminology pairs |
|
| 29 |
-
| `snomed-syn` | SNOMED CT description synonyms | Obtain a licensed release through [SNOMED International](https://www.snomed.org/get-snomed) | 44,973 | SNOMED CT Affiliate Licence; not an unrestricted open-data corpus | English synonym → Fully Specified Name (FSN); generated pairs are not distributed here |
|
| 30 |
-
|
| 31 |
-
## What Was Not Used
|
| 32 |
-
|
| 33 |
-
- The iTaigi 2,500-seed expansion was abandoned after repeated incomplete runs; the 1,288-row base extraction is the recorded source.
|
| 34 |
-
- `icd_term_bridge` (approximately 204k rows) was dropped because token-level alignment produced invalid semantic pairs such as `abandonment → 照顧或`.
|
| 35 |
-
- Common Crawl was not used in favour of targeted Taiwanese clinical sources.
|
| 36 |
-
|
| 37 |
-
## Third-Party Copyright and Release Posture
|
| 38 |
-
|
| 39 |
-
Open access is not the same as an open licence, and withholding raw pairs does not by itself establish that model training is authorised or is fair use. Likewise, applying Apache-2.0 to model code or weights does not grant rights in upstream material.
|
| 40 |
-
|
| 41 |
-
The three explicitly copyright-restricted source groups above contributed 494 mined rows in total:
|
| 42 |
-
|
| 43 |
-
| Source Groups | Rows | Share of Specialized Training Signals |
|
| 44 |
-
|---|---:|---:|
|
| 45 |
-
| `slang-blog` + `abbr-pdf` + `rsroc-weiei` | 494 | < 0.5% of total terminology signals |
|
| 46 |
-
|
| 47 |
-
That low aggregate share, the extraction of short factual terminology rather than article text, the non-generative nature of an embedding model, and the lack of a reading substitute for the source works are relevant facts in a source-specific copyright assessment.
|
| 48 |
-
|
| 49 |
-
Accordingly:
|
| 50 |
-
- Raw third-party pairs and source documents are not distributed in this repository;
|
| 51 |
-
- Each source still requires its own licence, permission, or documented legal assessment before reuse in a new training run;
|
| 52 |
-
- Redistributors must independently comply with the MOE, CC BY-SA, LOINC, and SNOMED terms that apply to their inputs; and
|
| 53 |
-
- This provenance record is factual disclosure, not legal advice or a warranty of non-infringement.
|
| 54 |
-
|
| 55 |
-
## Model Identity & Distribution
|
| 56 |
-
|
| 57 |
-
- **Model Name:** IlhaEmbed (v2.0)
|
| 58 |
-
- **Base Architecture:** IBM Granite ModernBERT, Apache-2.0
|
| 59 |
-
- **Method:** Contrastive learning & relational knowledge distillation over Taiwanese clinical terminology, followed by vocabulary pruning and INT8 quantization
|
| 60 |
-
- **Published Artifact:** [`weemed/IlhaEmbed`](https://huggingface.co/weemed/IlhaEmbed) (384 dimensions, 38.5 MB INT8 ONNX)
|
| 61 |
-
- **Evaluation & Usage:** See [`MODEL-CARD.md`](MODEL-CARD.md) and [`README.md`](README.md)
|
| 62 |
-
|
| 63 |
-
|
| 64 |
-
## 2026-09-06 candidate update
|
| 65 |
-
|
| 66 |
-
The update uses MOEX bilingual appositions corroborated by exact NAER rows, plus unlabelled score-preservation probes from official NAER training terms and synthetic work phrases. No new mixed private clinical replay corpus is included. Historical base-model disclosures above remain applicable.
|
| 67 |
-
|
| 68 |
-
The official source manifest was downloaded 2026-07-29. Dataset versions and licence validity windows were not supplied (null in that manifest). Attribution entries:
|
| 69 |
-
|
| 70 |
-
- 國家教育研究院-醫學學術名詞; provider: 國家教育研究院; published: 2026-06-24 22:11:12; licence: 政府資料開放授權條款; source: https://taic.moda.gov.tw/datasets/4ca6b322-b160-41dd-a16a-02249f540bf8
|
| 71 |
-
- 國家教育研究院-獸醫學學術名詞; provider: 國家教育研究院; published: 2026-06-24 22:11:24; licence: 政府資料開放授權條款; source: https://taic.moda.gov.tw/datasets/b57e36a4-644e-40e3-9d7c-551a92ba961f
|
| 72 |
-
- 國家教育研究院-心理學學術名詞; provider: 國家教育研究院; published: 2026-06-24 21:48:30; licence: 政府資料開放授權條款; source: https://taic.moda.gov.tw/datasets/727efda3-2810-4a92-8dc7-f7edecfd834f
|
| 73 |
-
- 國家教育研究院-生命科學學術名詞; provider: 國家教育研究院; published: 2026-06-24 21:49:32; licence: 政府資料開放授權條款; source: https://taic.moda.gov.tw/datasets/58ff0cc0-1471-4694-8297-589a0db6a3b2
|
| 74 |
-
- 國家教育研究院-病理學學術名詞; provider: 國家教育研究院; published: 2026-06-24 22:03:09; licence: 政府資料開放授權條款; source: https://taic.moda.gov.tw/datasets/1cf11198-bd37-4952-90ec-fd9d88500dc7
|
| 75 |
-
- 國家教育研究院-實驗動物及比較醫學學術名詞; provider: 國家教育研究院; published: 2026-06-24 22:08:56; licence: 政府資料開放授權條款; source: https://taic.moda.gov.tw/datasets/00b36304-a7ce-4ee5-9c2c-36393772d028
|
| 76 |
-
- 國家教育研究院-人體解剖學學術名詞; provider: 國家教育研究院; published: 2026-06-24 21:43:54; licence: 政府資料開放授權條款; source: https://taic.moda.gov.tw/datasets/f6e80374-6d80-4c8c-87da-e7626b0d8e7a
|
| 77 |
-
- 國家教育研究院-比較解剖學學術名詞; provider: 國家教育研究院; published: 2026-06-24 21:48:42; licence: 政府資料開放授權條款; source: https://taic.moda.gov.tw/datasets/9411220f-26d4-4ebc-a88e-f1d7fedd8833
|
| 78 |
-
- 國家教育研究院-藥學學術名詞; provider: 國家教育研究院; published: 2026-06-24 22:11:36; licence: 政府資料開放授權條款; source: https://taic.moda.gov.tw/datasets/169d49ea-f4e8-4586-88e0-d4200f84e4c9
|
| 79 |
-
- 國家教育研究院-生命科學名詞-兩岸中小學教科書名詞; provider: 國家教育研究院; published: 2026-06-24 21:49:44; licence: 政府資料開放授權條款; source: https://taic.moda.gov.tw/datasets/fe34fb09-b6b6-499d-8fe5-d07430b97aed
|
| 80 |
-
- 國家教育研究院-精神病理學學術名詞; provider: 國家教育研究院; published: 2026-06-24 22:09:20; licence: 政府資料開放授權條款; source: https://taic.moda.gov.tw/datasets/5fc9f128-161e-46ec-be08-39025de582fe
|
| 81 |
-
- 國家教育研究院-兩岸對照名詞-中醫學術名詞; provider: 國家教育研究院; published: 2026-06-24 21:54:11; licence: 政府資料開放授權條款; source: https://taic.moda.gov.tw/datasets/6b28746a-a2d7-451e-8790-0726fc6feeb8
|
| 82 |
-
- 國家教育研究院-兩岸對照名詞-醫學學術名詞; provider: 國家教育研究院; published: 2026-06-24 21:58:51; licence: 政府資料開放授權條款; source: https://taic.moda.gov.tw/datasets/e8d2d1fc-f577-4439-9a2d-f9595756a3ac
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
evaluation_drugbank_bootstrap.json
DELETED
|
@@ -1,40 +0,0 @@
|
|
| 1 |
-
{
|
| 2 |
-
"baseline_report": "/home/gloomcheng/Workspace/weemed-ai/terminology-embedder/research/artifacts/ilhaembed-integrated-all-sources-v1-20260913/evaluation/drugbank-baseline.json",
|
| 3 |
-
"candidate_report": "/home/gloomcheng/Workspace/weemed-ai/terminology-embedder/research/artifacts/ilhaembed-autonomous-unbiased-source-conditioned-20260919/runs/trial-0004/model-evaluation/drugbank-candidate.json",
|
| 4 |
-
"eval_manifest_sha256": "b668e78b382cb6cae1c3a8dbe3bce808ba53352addb98158d463971a35d268c8",
|
| 5 |
-
"interval": "two-sided 95% percentile bootstrap on paired candidate-minus-baseline deltas",
|
| 6 |
-
"metrics": {
|
| 7 |
-
"reciprocal_rank": {
|
| 8 |
-
"baseline": 0.5660135839943139,
|
| 9 |
-
"candidate": 0.6638060096581913,
|
| 10 |
-
"ci95_high": 0.10629207614876504,
|
| 11 |
-
"ci95_low": 0.08939576396964259,
|
| 12 |
-
"delta": 0.0977924256638775,
|
| 13 |
-
"superiority_pass": true
|
| 14 |
-
},
|
| 15 |
-
"top1": {
|
| 16 |
-
"baseline": 0.5209967697277342,
|
| 17 |
-
"candidate": 0.6347485002307337,
|
| 18 |
-
"ci95_high": 0.12459621596677434,
|
| 19 |
-
"ci95_low": 0.10290724503922473,
|
| 20 |
-
"delta": 0.11375173050299954,
|
| 21 |
-
"superiority_pass": true
|
| 22 |
-
},
|
| 23 |
-
"top5": {
|
| 24 |
-
"baseline": 0.614443931702815,
|
| 25 |
-
"candidate": 0.6970466082141209,
|
| 26 |
-
"ci95_high": 0.09206275957544993,
|
| 27 |
-
"ci95_low": 0.0735982925703738,
|
| 28 |
-
"delta": 0.08260267651130596,
|
| 29 |
-
"superiority_pass": true
|
| 30 |
-
}
|
| 31 |
-
},
|
| 32 |
-
"promotion_allowed": false,
|
| 33 |
-
"protocol": "DrugBank ID concept-disjoint eval; self-match excluded; SAME_CONCEPT names only",
|
| 34 |
-
"resamples": 10000,
|
| 35 |
-
"schema": "ilhaembed-rxnorm-paired-bootstrap/v1",
|
| 36 |
-
"seed": 20260913,
|
| 37 |
-
"superiority_all_metrics": true,
|
| 38 |
-
"surfaces": 4334,
|
| 39 |
-
"unit": "surface query"
|
| 40 |
-
}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
evaluation_drugbank_candidate.json
DELETED
|
The diff for this file is too large to render.
See raw diff
|
|
|
evaluation_rxnorm_bootstrap.json
DELETED
|
@@ -1,40 +0,0 @@
|
|
| 1 |
-
{
|
| 2 |
-
"baseline_report": "/home/gloomcheng/Workspace/weemed-ai/terminology-embedder/research/artifacts/ilhaembed-integrated-all-sources-v1-20260913/evaluation/rxnorm-baseline.json",
|
| 3 |
-
"candidate_report": "/home/gloomcheng/Workspace/weemed-ai/terminology-embedder/research/artifacts/ilhaembed-autonomous-unbiased-source-conditioned-20260919/runs/trial-0004/model-evaluation/rxnorm-candidate.json",
|
| 4 |
-
"eval_manifest_sha256": "ad13bcb506e5762429fea4ce874b65eda2bdf68a96e02bc2a778696b16cfdee7",
|
| 5 |
-
"interval": "two-sided 95% percentile bootstrap on paired candidate-minus-baseline deltas",
|
| 6 |
-
"metrics": {
|
| 7 |
-
"reciprocal_rank": {
|
| 8 |
-
"baseline": 0.6584821651631182,
|
| 9 |
-
"candidate": 0.899601212810233,
|
| 10 |
-
"ci95_high": 0.24843618342695703,
|
| 11 |
-
"ci95_low": 0.23387872683413913,
|
| 12 |
-
"delta": 0.24111904764711456,
|
| 13 |
-
"superiority_pass": true
|
| 14 |
-
},
|
| 15 |
-
"top1": {
|
| 16 |
-
"baseline": 0.5889461358313818,
|
| 17 |
-
"candidate": 0.8598594847775176,
|
| 18 |
-
"ci95_high": 0.2799063231850117,
|
| 19 |
-
"ci95_low": 0.26201405152224827,
|
| 20 |
-
"delta": 0.27091334894613583,
|
| 21 |
-
"superiority_pass": true
|
| 22 |
-
},
|
| 23 |
-
"top5": {
|
| 24 |
-
"baseline": 0.7418266978922716,
|
| 25 |
-
"candidate": 0.949695550351288,
|
| 26 |
-
"ci95_high": 0.2157377049180328,
|
| 27 |
-
"ci95_low": 0.2,
|
| 28 |
-
"delta": 0.2078688524590164,
|
| 29 |
-
"superiority_pass": true
|
| 30 |
-
}
|
| 31 |
-
},
|
| 32 |
-
"promotion_allowed": false,
|
| 33 |
-
"protocol": "source-isolated RxNorm eval; RxCUI split is fixed; self-match excluded",
|
| 34 |
-
"resamples": 10000,
|
| 35 |
-
"schema": "ilhaembed-rxnorm-paired-bootstrap/v1",
|
| 36 |
-
"seed": 20260913,
|
| 37 |
-
"superiority_all_metrics": true,
|
| 38 |
-
"surfaces": 10675,
|
| 39 |
-
"unit": "surface query"
|
| 40 |
-
}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
evaluation_rxnorm_candidate.json
DELETED
|
The diff for this file is too large to render.
See raw diff
|
|
|
evaluation_semantic_candidate.json
DELETED
|
@@ -1,64 +0,0 @@
|
|
| 1 |
-
{
|
| 2 |
-
"card": {
|
| 3 |
-
"abbrev": {
|
| 4 |
-
"cases": 399,
|
| 5 |
-
"pool": 1295,
|
| 6 |
-
"top1": 0.8070175438596491,
|
| 7 |
-
"top1_correct": 322,
|
| 8 |
-
"top5": 0.8571428571428571,
|
| 9 |
-
"top5_correct": 342
|
| 10 |
-
},
|
| 11 |
-
"apposition": {
|
| 12 |
-
"cases": 371,
|
| 13 |
-
"pool": 1295,
|
| 14 |
-
"top1": 0.8598382749326146,
|
| 15 |
-
"top1_correct": 319,
|
| 16 |
-
"top5": 0.9164420485175202,
|
| 17 |
-
"top5_correct": 340
|
| 18 |
-
},
|
| 19 |
-
"methodology": "fixed but source-contaminated regression; not held-out evidence",
|
| 20 |
-
"semantic_macro": 0.8405648428232277,
|
| 21 |
-
"slang": {
|
| 22 |
-
"cases": 62,
|
| 23 |
-
"pool": 1295,
|
| 24 |
-
"top1": 0.8548387096774194,
|
| 25 |
-
"top1_correct": 53,
|
| 26 |
-
"top5": 0.9354838709677419,
|
| 27 |
-
"top5_correct": 58
|
| 28 |
-
}
|
| 29 |
-
},
|
| 30 |
-
"device": "cpu",
|
| 31 |
-
"label": "eval-trial-0004-semantic",
|
| 32 |
-
"latency_ms_per_text_batch96": 4.2576849688581815,
|
| 33 |
-
"model": "/home/gloomcheng/Workspace/weemed-ai/terminology-embedder/research/artifacts/ilhaembed-autonomous-unbiased-source-conditioned-20260919/runs/trial-0004/model",
|
| 34 |
-
"parameters": 38565888,
|
| 35 |
-
"production_category": {
|
| 36 |
-
"category_top1": 0.7017543859649122,
|
| 37 |
-
"mean_top1_margin": 0.1627029925584793,
|
| 38 |
-
"negative_p95": 0.8431658267974853,
|
| 39 |
-
"negatives": 130,
|
| 40 |
-
"positive_p5": 0.476388031244278,
|
| 41 |
-
"positives": 114,
|
| 42 |
-
"separation": -0.36677779555320733
|
| 43 |
-
},
|
| 44 |
-
"shorthand": {
|
| 45 |
-
"cases": 121,
|
| 46 |
-
"excluded_ambiguous_rows": 2,
|
| 47 |
-
"kind": "fixed_regression_not_clean_heldout",
|
| 48 |
-
"pool": 92,
|
| 49 |
-
"top1": 0.9256198347107438,
|
| 50 |
-
"top1_correct": 112,
|
| 51 |
-
"top5": 0.9752066115702479,
|
| 52 |
-
"top5_correct": 118
|
| 53 |
-
},
|
| 54 |
-
"taigi_semantic": {
|
| 55 |
-
"cases": 141,
|
| 56 |
-
"methodology": "mostly source-contaminated regression; not held-out evidence",
|
| 57 |
-
"pool": 66,
|
| 58 |
-
"top1": 0.9361702127659575,
|
| 59 |
-
"top1_correct": 132,
|
| 60 |
-
"top5": 0.9645390070921985,
|
| 61 |
-
"top5_correct": 136
|
| 62 |
-
},
|
| 63 |
-
"weight_bytes": 154271056
|
| 64 |
-
}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
evaluation_slo_candidate.json
DELETED
|
@@ -1,31 +0,0 @@
|
|
| 1 |
-
{
|
| 2 |
-
"artifact_bytes": 291612044,
|
| 3 |
-
"failures": {
|
| 4 |
-
"count": 0,
|
| 5 |
-
"kinds": [],
|
| 6 |
-
"rate": 0.0
|
| 7 |
-
},
|
| 8 |
-
"interpretation": "Fixed synthetic probe runtime evidence; it is not accuracy or broad workload evidence.",
|
| 9 |
-
"label": "eval-trial-0004-slo",
|
| 10 |
-
"latency_ms_per_batch": {
|
| 11 |
-
"max": 395.01672302139923,
|
| 12 |
-
"min": 351.3379310024902,
|
| 13 |
-
"p50": 381.2973879976198,
|
| 14 |
-
"p95": 389.27634101128206
|
| 15 |
-
},
|
| 16 |
-
"model": "/home/gloomcheng/Workspace/weemed-ai/terminology-embedder/research/artifacts/ilhaembed-autonomous-unbiased-source-conditioned-20260919/runs/trial-0004/model-evaluation/model_int8.onnx",
|
| 17 |
-
"model_sha256": "9da09372f38401938c6275e6641d6312ca52e3e08f4c15bbe696f4e89b61f295",
|
| 18 |
-
"peak_rss_bytes": 451375104,
|
| 19 |
-
"probe": {
|
| 20 |
-
"batch_size": 96,
|
| 21 |
-
"batches": 25,
|
| 22 |
-
"max_length": 32,
|
| 23 |
-
"probe_count": 10,
|
| 24 |
-
"synthetic_fixed_probes": true
|
| 25 |
-
},
|
| 26 |
-
"provider": "CPUExecutionProvider",
|
| 27 |
-
"schema": "ilhaembed-runtime-slo/v1",
|
| 28 |
-
"status": "PASS",
|
| 29 |
-
"throughput_texts_per_second": 254.31276778546282,
|
| 30 |
-
"tokenizer_sha256": "dc0e20c81f2ba89f3f1d0a66b315c444aa761551fbed3e320410eeebf1e80483"
|
| 31 |
-
}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
evaluation_source_snapshot.zip
DELETED
|
@@ -1,3 +0,0 @@
|
|
| 1 |
-
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:429ec47a44248de2f09eb068a898d82902028550e80d960c69aba249eca225e5
|
| 3 |
-
size 23235
|
|
|
|
|
|
|
|
|
|
|
|
evaluation_v1_int8.json
DELETED
|
@@ -1,69 +0,0 @@
|
|
| 1 |
-
{
|
| 2 |
-
"card": {
|
| 3 |
-
"abbrev": {
|
| 4 |
-
"cases": 399,
|
| 5 |
-
"pool": 1295,
|
| 6 |
-
"top1": 0.7593984962406015,
|
| 7 |
-
"top1_correct": 303,
|
| 8 |
-
"top5": 0.8395989974937343,
|
| 9 |
-
"top5_correct": 335
|
| 10 |
-
},
|
| 11 |
-
"apposition": {
|
| 12 |
-
"cases": 371,
|
| 13 |
-
"pool": 1295,
|
| 14 |
-
"top1": 0.8921832884097035,
|
| 15 |
-
"top1_correct": 331,
|
| 16 |
-
"top5": 0.9299191374663073,
|
| 17 |
-
"top5_correct": 345
|
| 18 |
-
},
|
| 19 |
-
"methodology": "fixed but source-contaminated regression; not held-out evidence",
|
| 20 |
-
"semantic_macro": 0.8139681217651554,
|
| 21 |
-
"slang": {
|
| 22 |
-
"cases": 62,
|
| 23 |
-
"pool": 1295,
|
| 24 |
-
"top1": 0.7903225806451613,
|
| 25 |
-
"top1_correct": 49,
|
| 26 |
-
"top5": 0.8387096774193549,
|
| 27 |
-
"top5_correct": 52
|
| 28 |
-
}
|
| 29 |
-
},
|
| 30 |
-
"label": "shipped-v1-current-int8",
|
| 31 |
-
"latency_ms_per_text_batch96": 3.28879157314077,
|
| 32 |
-
"model": "/home/gloomcheng/med-embed/ilha-embed-97m-pruned",
|
| 33 |
-
"naer_isolated": {
|
| 34 |
-
"concepts": 3245,
|
| 35 |
-
"queries": 6949,
|
| 36 |
-
"top1": 0.11310979997121888,
|
| 37 |
-
"top5": 0.21988775363361635
|
| 38 |
-
},
|
| 39 |
-
"onnx": "/home/gloomcheng/med-embed/ilha-embed-97m-pruned/model_int8.onnx",
|
| 40 |
-
"onnx_bytes": 38537426,
|
| 41 |
-
"production_category": {
|
| 42 |
-
"category_top1": 0.7128712871287128,
|
| 43 |
-
"mean_top1_margin": 0.1815911829471588,
|
| 44 |
-
"negative_p95": 0.6991750836372376,
|
| 45 |
-
"negatives": 130,
|
| 46 |
-
"positive_p5": 0.28964269161224365,
|
| 47 |
-
"positives": 101,
|
| 48 |
-
"separation": -0.4095323920249939
|
| 49 |
-
},
|
| 50 |
-
"shorthand": {
|
| 51 |
-
"cases": 108,
|
| 52 |
-
"excluded_ambiguous_rows": 2,
|
| 53 |
-
"kind": "fixed_regression_not_clean_heldout",
|
| 54 |
-
"pool": 91,
|
| 55 |
-
"top1": 0.7870370370370371,
|
| 56 |
-
"top1_correct": 85,
|
| 57 |
-
"top5": 0.8796296296296297,
|
| 58 |
-
"top5_correct": 95
|
| 59 |
-
},
|
| 60 |
-
"taigi_semantic": {
|
| 61 |
-
"cases": 141,
|
| 62 |
-
"methodology": "mostly source-contaminated regression; not held-out evidence",
|
| 63 |
-
"pool": 66,
|
| 64 |
-
"top1": 0.9290780141843972,
|
| 65 |
-
"top1_correct": 131,
|
| 66 |
-
"top5": 0.9716312056737588,
|
| 67 |
-
"top5_correct": 137
|
| 68 |
-
}
|
| 69 |
-
}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
evaluation_v2_1_int8.json
DELETED
|
@@ -1,169 +0,0 @@
|
|
| 1 |
-
{
|
| 2 |
-
"release_version": "2.1.0",
|
| 3 |
-
"classification_protocol": "labelled-category-v2",
|
| 4 |
-
"owner_clarification": "Correct category assignment is the task; unresolved residue is not gold refusal.",
|
| 5 |
-
"release_regression_passed": true,
|
| 6 |
-
"native": {
|
| 7 |
-
"source": {
|
| 8 |
-
"embedder.py": "f8ef081d2c858cacb4e4d7147b313973bb2fa644bdb2eb4959a445b2944bdd67",
|
| 9 |
-
"classify.py": "180f656db9691efe15bf872fea631b4d1140d23eac7ddb6f54eaacba418a9dae",
|
| 10 |
-
"source_commit": "e15d22018d33c42ff288354940fea064823c8c97",
|
| 11 |
-
"embedder_component.py": "cc4211e5f620b56c3004d73ccc50376cbf3c8018f6443a698e838fe48813f3b8",
|
| 12 |
-
"classify_component.py": "eb8500a5d2585f8460237c68675a08f095a7a72a18f340d5b249a98d035299b7",
|
| 13 |
-
"scope": "Exact selected computation definitions with standard imports and explicit real-model binding; no application bootstrap."
|
| 14 |
-
},
|
| 15 |
-
"protocol": "exact native class/functions: fixed padding 32, batch 32 retrieval, per-category prototypes, singleton classification, 3-decimal candidate scores",
|
| 16 |
-
"scope": "component replay with real ONNX; not full application verification",
|
| 17 |
-
"models": {
|
| 18 |
-
"baseline": {
|
| 19 |
-
"registers": {
|
| 20 |
-
"slang": {
|
| 21 |
-
"count": 62,
|
| 22 |
-
"correct": 40,
|
| 23 |
-
"top1": 0.6451612903225806
|
| 24 |
-
},
|
| 25 |
-
"abbrev": {
|
| 26 |
-
"count": 399,
|
| 27 |
-
"correct": 272,
|
| 28 |
-
"top1": 0.681704260651629
|
| 29 |
-
},
|
| 30 |
-
"apposition": {
|
| 31 |
-
"count": 371,
|
| 32 |
-
"correct": 304,
|
| 33 |
-
"top1": 0.8194070080862533
|
| 34 |
-
}
|
| 35 |
-
},
|
| 36 |
-
"taigi": {
|
| 37 |
-
"count": 141,
|
| 38 |
-
"correct": 131,
|
| 39 |
-
"top1": 0.9290780141843972
|
| 40 |
-
},
|
| 41 |
-
"padding": {
|
| 42 |
-
"length": 32,
|
| 43 |
-
"pad_to_multiple_of": null,
|
| 44 |
-
"pad_id": 0,
|
| 45 |
-
"pad_token": "[PAD]",
|
| 46 |
-
"pad_type_id": 0,
|
| 47 |
-
"direction": "right"
|
| 48 |
-
},
|
| 49 |
-
"classification": {
|
| 50 |
-
"protocol": "labelled-category-v2",
|
| 51 |
-
"score_floor": 0.7,
|
| 52 |
-
"margin_floor": 0.1,
|
| 53 |
-
"positive": {
|
| 54 |
-
"count": 100,
|
| 55 |
-
"top1_correct": 67,
|
| 56 |
-
"accepted": 28,
|
| 57 |
-
"gate_correct": 25,
|
| 58 |
-
"wrong_class": 3,
|
| 59 |
-
"unclassified": 72
|
| 60 |
-
},
|
| 61 |
-
"unlabelled": {
|
| 62 |
-
"count": 53,
|
| 63 |
-
"accepted": 0,
|
| 64 |
-
"accuracy_measured": false
|
| 65 |
-
}
|
| 66 |
-
},
|
| 67 |
-
"macro": 0.7154241863534877
|
| 68 |
-
},
|
| 69 |
-
"candidate": {
|
| 70 |
-
"registers": {
|
| 71 |
-
"slang": {
|
| 72 |
-
"count": 62,
|
| 73 |
-
"correct": 59,
|
| 74 |
-
"top1": 0.9516129032258065
|
| 75 |
-
},
|
| 76 |
-
"abbrev": {
|
| 77 |
-
"count": 399,
|
| 78 |
-
"correct": 310,
|
| 79 |
-
"top1": 0.7769423558897243
|
| 80 |
-
},
|
| 81 |
-
"apposition": {
|
| 82 |
-
"count": 371,
|
| 83 |
-
"correct": 324,
|
| 84 |
-
"top1": 0.8733153638814016
|
| 85 |
-
}
|
| 86 |
-
},
|
| 87 |
-
"taigi": {
|
| 88 |
-
"count": 141,
|
| 89 |
-
"correct": 133,
|
| 90 |
-
"top1": 0.9432624113475178
|
| 91 |
-
},
|
| 92 |
-
"padding": {
|
| 93 |
-
"length": 32,
|
| 94 |
-
"pad_to_multiple_of": null,
|
| 95 |
-
"pad_id": 0,
|
| 96 |
-
"pad_token": "[PAD]",
|
| 97 |
-
"pad_type_id": 0,
|
| 98 |
-
"direction": "right"
|
| 99 |
-
},
|
| 100 |
-
"classification": {
|
| 101 |
-
"protocol": "labelled-category-v2",
|
| 102 |
-
"score_floor": 0.7,
|
| 103 |
-
"margin_floor": 0.1,
|
| 104 |
-
"positive": {
|
| 105 |
-
"count": 100,
|
| 106 |
-
"top1_correct": 68,
|
| 107 |
-
"accepted": 31,
|
| 108 |
-
"gate_correct": 31,
|
| 109 |
-
"wrong_class": 0,
|
| 110 |
-
"unclassified": 69
|
| 111 |
-
},
|
| 112 |
-
"unlabelled": {
|
| 113 |
-
"count": 53,
|
| 114 |
-
"accepted": 0,
|
| 115 |
-
"accuracy_measured": false
|
| 116 |
-
}
|
| 117 |
-
},
|
| 118 |
-
"macro": 0.8672902076656442
|
| 119 |
-
}
|
| 120 |
-
},
|
| 121 |
-
"baseline_sha256": "d222b6cfb11d265294f8855bfc7d4bbd691a61a74a4e5d0cb9b31649ea02f321",
|
| 122 |
-
"candidate_sha256": "ca8eb040e1b5b51e05f5ceaf125073370fdcdd475e8a99c9dac0b03e46a0e7eb",
|
| 123 |
-
"failures": [],
|
| 124 |
-
"native_consumer_admitted": true
|
| 125 |
-
},
|
| 126 |
-
"numerical_and_latency": {
|
| 127 |
-
"fp32_sha256": "0e4204f2849be97faf47fe1a748559f0766773af6cc586facf6f32fd50f551cc",
|
| 128 |
-
"int8_sha256": "ca8eb040e1b5b51e05f5ceaf125073370fdcdd475e8a99c9dac0b03e46a0e7eb",
|
| 129 |
-
"baseline_int8_sha256": "d222b6cfb11d265294f8855bfc7d4bbd691a61a74a4e5d0cb9b31649ea02f321",
|
| 130 |
-
"model_bytes": 38667189,
|
| 131 |
-
"onnxruntime": "1.23.2",
|
| 132 |
-
"providers": [
|
| 133 |
-
"CPUExecutionProvider"
|
| 134 |
-
],
|
| 135 |
-
"mean_cosine_fidelity": 0.9941092729568481,
|
| 136 |
-
"minimum_cosine_fidelity": 0.9703658223152161,
|
| 137 |
-
"batch_invariance_minimum_cosine": 0.9999998807907104,
|
| 138 |
-
"batch_invariance_max_abs_delta": 0.0,
|
| 139 |
-
"cpu_batch_latency_ratio": 1.2098393668047083,
|
| 140 |
-
"cpu_single_latency_ratio": 1.2536873087375715
|
| 141 |
-
},
|
| 142 |
-
"naer": {
|
| 143 |
-
"baseline": {
|
| 144 |
-
"queries": 6949,
|
| 145 |
-
"concepts": 3245,
|
| 146 |
-
"correct": 855,
|
| 147 |
-
"top1": 0.12303928622823428,
|
| 148 |
-
"top5_correct": 1625,
|
| 149 |
-
"top5": 0.233846596632609
|
| 150 |
-
},
|
| 151 |
-
"candidate": {
|
| 152 |
-
"queries": 6949,
|
| 153 |
-
"concepts": 3245,
|
| 154 |
-
"correct": 1167,
|
| 155 |
-
"top1": 0.16793783278169522,
|
| 156 |
-
"top5_correct": 2143,
|
| 157 |
-
"top5": 0.3083896963591884
|
| 158 |
-
}
|
| 159 |
-
},
|
| 160 |
-
"historical_residue_protocol": {
|
| 161 |
-
"failed": true,
|
| 162 |
-
"superseded_by_owner_task_clarification": true,
|
| 163 |
-
"not_a_clinical_error_rate": true
|
| 164 |
-
},
|
| 165 |
-
"independent_clinical_gold_claimed": false,
|
| 166 |
-
"automatic_classification_qualified": false,
|
| 167 |
-
"release_judgment": "retracted_for_automatic_classification",
|
| 168 |
-
"reason": "Relative regression improvement was insufficient: 31/100 correct automatic classifications, 69/100 unclassified."
|
| 169 |
-
}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
evaluation_v2_int8.json
DELETED
|
@@ -1,69 +0,0 @@
|
|
| 1 |
-
{
|
| 2 |
-
"card": {
|
| 3 |
-
"abbrev": {
|
| 4 |
-
"cases": 399,
|
| 5 |
-
"pool": 1295,
|
| 6 |
-
"top1": 0.7794486215538847,
|
| 7 |
-
"top1_correct": 311,
|
| 8 |
-
"top5": 0.8596491228070176,
|
| 9 |
-
"top5_correct": 343
|
| 10 |
-
},
|
| 11 |
-
"apposition": {
|
| 12 |
-
"cases": 371,
|
| 13 |
-
"pool": 1295,
|
| 14 |
-
"top1": 0.894878706199461,
|
| 15 |
-
"top1_correct": 332,
|
| 16 |
-
"top5": 0.9380053908355795,
|
| 17 |
-
"top5_correct": 348
|
| 18 |
-
},
|
| 19 |
-
"methodology": "fixed but source-contaminated regression; not held-out evidence",
|
| 20 |
-
"semantic_macro": 0.8215499694661691,
|
| 21 |
-
"slang": {
|
| 22 |
-
"cases": 62,
|
| 23 |
-
"pool": 1295,
|
| 24 |
-
"top1": 0.7903225806451613,
|
| 25 |
-
"top1_correct": 49,
|
| 26 |
-
"top5": 0.8387096774193549,
|
| 27 |
-
"top5_correct": 52
|
| 28 |
-
}
|
| 29 |
-
},
|
| 30 |
-
"label": "naer-retained-epoch1-soup-a025-v2-perchannel-int8",
|
| 31 |
-
"latency_ms_per_text_batch96": 3.1114273751882138,
|
| 32 |
-
"model": "intake-embedder/ilhaembed-naer-retained-epoch1-soup-a0.25-v2-onnx-perchannel",
|
| 33 |
-
"naer_isolated": {
|
| 34 |
-
"concepts": 3245,
|
| 35 |
-
"queries": 6949,
|
| 36 |
-
"top1": 0.1318175277018276,
|
| 37 |
-
"top5": 0.24996402360051806
|
| 38 |
-
},
|
| 39 |
-
"onnx": "intake-embedder/ilhaembed-naer-retained-epoch1-soup-a0.25-v2-onnx-perchannel/model_int8.onnx",
|
| 40 |
-
"onnx_bytes": 38865358,
|
| 41 |
-
"production_category": {
|
| 42 |
-
"category_top1": 0.7425742574257426,
|
| 43 |
-
"mean_top1_margin": 0.18474966287612915,
|
| 44 |
-
"negative_p95": 0.6888158977031708,
|
| 45 |
-
"negatives": 130,
|
| 46 |
-
"positive_p5": 0.2778717875480652,
|
| 47 |
-
"positives": 101,
|
| 48 |
-
"separation": -0.41094411015510557
|
| 49 |
-
},
|
| 50 |
-
"shorthand": {
|
| 51 |
-
"cases": 108,
|
| 52 |
-
"excluded_ambiguous_rows": 2,
|
| 53 |
-
"kind": "fixed_regression_not_clean_heldout",
|
| 54 |
-
"pool": 91,
|
| 55 |
-
"top1": 0.7777777777777778,
|
| 56 |
-
"top1_correct": 84,
|
| 57 |
-
"top5": 0.9074074074074074,
|
| 58 |
-
"top5_correct": 98
|
| 59 |
-
},
|
| 60 |
-
"taigi_semantic": {
|
| 61 |
-
"cases": 141,
|
| 62 |
-
"methodology": "mostly source-contaminated regression; not held-out evidence",
|
| 63 |
-
"pool": 66,
|
| 64 |
-
"top1": 0.9361702127659575,
|
| 65 |
-
"top1_correct": 132,
|
| 66 |
-
"top5": 0.9645390070921985,
|
| 67 |
-
"top5_correct": 136
|
| 68 |
-
}
|
| 69 |
-
}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
export-provenance.json
DELETED
|
@@ -1,21 +0,0 @@
|
|
| 1 |
-
{
|
| 2 |
-
"base_sha256": "c8fd7c6f46b531a81b724ef3dddd107fdc1e88f631ae4385914581b043d83cda",
|
| 3 |
-
"torch": "2.13.0+cu130",
|
| 4 |
-
"onnxruntime": "1.23.2",
|
| 5 |
-
"onnx": "1.22.0",
|
| 6 |
-
"opset": 18,
|
| 7 |
-
"quantization": "MatMul INT8 with per-token symmetric activations, per-channel weights and per-row vocabulary",
|
| 8 |
-
"activation_quantizers": 48,
|
| 9 |
-
"mean_int8_cosine_fidelity": 0.9959200819333395,
|
| 10 |
-
"minimum_fp32_cosine_fidelity": 0.9999998807907104,
|
| 11 |
-
"files": {
|
| 12 |
-
"model.onnx": {
|
| 13 |
-
"bytes": 153759006,
|
| 14 |
-
"sha256": "c4175b1ab94855d0fea9d735ef9064cf7b632e4684840c58201dfb0e6f85f0ba"
|
| 15 |
-
},
|
| 16 |
-
"model_int8.onnx": {
|
| 17 |
-
"bytes": 38667189,
|
| 18 |
-
"sha256": "ba9809fad34464e4b47a1b1f73b025be5cf8090e8525efd2d013ef7e44164684"
|
| 19 |
-
}
|
| 20 |
-
}
|
| 21 |
-
}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
export_provenance.json
DELETED
|
@@ -1,56 +0,0 @@
|
|
| 1 |
-
{
|
| 2 |
-
"fp32_bytes": 154550326,
|
| 3 |
-
"fp32_sha256": "2c0d52cb9069f42f9469898837d73f4dd126f94d0379d90dfe1e1abee0ddd839",
|
| 4 |
-
"int8_bytes": 133463094,
|
| 5 |
-
"int8_sha256": "9da09372f38401938c6275e6641d6312ca52e3e08f4c15bbe696f4e89b61f295",
|
| 6 |
-
"parity": {
|
| 7 |
-
"fp32": {
|
| 8 |
-
"mean_cosine_to_hf": 1.0,
|
| 9 |
-
"minimum_cosine_to_hf": 0.9999998807907104
|
| 10 |
-
},
|
| 11 |
-
"int8": {
|
| 12 |
-
"mean_cosine_to_hf": 0.9954825639724731,
|
| 13 |
-
"minimum_cosine_to_hf": 0.9896921515464783
|
| 14 |
-
}
|
| 15 |
-
},
|
| 16 |
-
"quantization": {
|
| 17 |
-
"format": "dynamic_qint8",
|
| 18 |
-
"minimum_cosine_gate": 0.98,
|
| 19 |
-
"node_count": 24,
|
| 20 |
-
"node_policy": "attention_projection_only",
|
| 21 |
-
"nodes": [
|
| 22 |
-
"/model/layers.0/attn/Wqkv/MatMul",
|
| 23 |
-
"/model/layers.0/attn/Wo/MatMul",
|
| 24 |
-
"/model/layers.1/attn/Wqkv/MatMul",
|
| 25 |
-
"/model/layers.1/attn/Wo/MatMul",
|
| 26 |
-
"/model/layers.2/attn/Wqkv/MatMul",
|
| 27 |
-
"/model/layers.2/attn/Wo/MatMul",
|
| 28 |
-
"/model/layers.3/attn/Wqkv/MatMul",
|
| 29 |
-
"/model/layers.3/attn/Wo/MatMul",
|
| 30 |
-
"/model/layers.4/attn/Wqkv/MatMul",
|
| 31 |
-
"/model/layers.4/attn/Wo/MatMul",
|
| 32 |
-
"/model/layers.5/attn/Wqkv/MatMul",
|
| 33 |
-
"/model/layers.5/attn/Wo/MatMul",
|
| 34 |
-
"/model/layers.6/attn/Wqkv/MatMul",
|
| 35 |
-
"/model/layers.6/attn/Wo/MatMul",
|
| 36 |
-
"/model/layers.7/attn/Wqkv/MatMul",
|
| 37 |
-
"/model/layers.7/attn/Wo/MatMul",
|
| 38 |
-
"/model/layers.8/attn/Wqkv/MatMul",
|
| 39 |
-
"/model/layers.8/attn/Wo/MatMul",
|
| 40 |
-
"/model/layers.9/attn/Wqkv/MatMul",
|
| 41 |
-
"/model/layers.9/attn/Wo/MatMul",
|
| 42 |
-
"/model/layers.10/attn/Wqkv/MatMul",
|
| 43 |
-
"/model/layers.10/attn/Wo/MatMul",
|
| 44 |
-
"/model/layers.11/attn/Wqkv/MatMul",
|
| 45 |
-
"/model/layers.11/attn/Wo/MatMul"
|
| 46 |
-
],
|
| 47 |
-
"per_channel": false
|
| 48 |
-
},
|
| 49 |
-
"source_config_sha256": "9cb200e53b8bf3f5f8287a6e19a5cd1505aa08d88244b7fd151343a7ebcb9056",
|
| 50 |
-
"source_model": "/home/gloomcheng/Workspace/weemed-ai/terminology-embedder/research/artifacts/ilhaembed-autonomous-unbiased-source-conditioned-20260919/runs/trial-0004/model",
|
| 51 |
-
"source_weights_sha256": "7f1f3d42fd003a800b52cbda192af19bea4184f27bd87ea6789542e0d4acdf54",
|
| 52 |
-
"tokenizer_sha256": {
|
| 53 |
-
"tokenizer.json": "dc0e20c81f2ba89f3f1d0a66b315c444aa761551fbed3e320410eeebf1e80483",
|
| 54 |
-
"tokenizer_config.json": "0bd6f388bc813908f40cff3c5cdf0065a444b351d19b40a086f138a80a9a7747"
|
| 55 |
-
}
|
| 56 |
-
}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
interpolation_provenance.json
DELETED
|
@@ -1,17 +0,0 @@
|
|
| 1 |
-
{
|
| 2 |
-
"alpha": 0.25,
|
| 3 |
-
"base_config_sha256": "b5a4dd13f74b2eb21b8018aea9eccf1aa1261698857e6454b516ebe636837562",
|
| 4 |
-
"base_model": "/home/gloomcheng/med-embed/ilha-embed-97m-pruned",
|
| 5 |
-
"base_weights_sha256": "261af96cef3d72ea9e1877640083fa8d9fe6403333fec4a8cd54ef5e09f92299",
|
| 6 |
-
"formula": "base + alpha * (tuned - base)",
|
| 7 |
-
"output_config_sha256": "acf070a47f90b6fee858cb6a80be1a5d94afbe33211586cbd09db26efd55232a",
|
| 8 |
-
"output_weights_sha256": "044fd270d0fe2d9ef56762c4db9ce6f9be09cdd810ba2b49b40d31ec43d38f09",
|
| 9 |
-
"tokenizer_sha256": {
|
| 10 |
-
"special_tokens_map.json": "e83cab90dfc594d568809f9a6c6ce31e5c703ad4fcd04a3519a914017da326b9",
|
| 11 |
-
"tokenizer.json": "8fb9f1fd4a29a8151e220419f33cb4aa52bee76d55a0c9eced066664913605a0",
|
| 12 |
-
"tokenizer_config.json": "3e86463bcf10c064244bc0aa22d84a9d78d582a8f9ef9896b9f704cbb6cd112c"
|
| 13 |
-
},
|
| 14 |
-
"tuned_config_sha256": "acf070a47f90b6fee858cb6a80be1a5d94afbe33211586cbd09db26efd55232a",
|
| 15 |
-
"tuned_model": "intake-embedder/ilhaembed-naer-retained-lr2e6-seed42/epoch-1",
|
| 16 |
-
"tuned_weights_sha256": "4c6c596c999f888ccda6a0f4b504b4157fdd01f37a36ee0bf26e9d9918729fdc"
|
| 17 |
-
}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
model_compact_37m_int8.onnx
DELETED
|
@@ -1,3 +0,0 @@
|
|
| 1 |
-
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:44cd7c10f351d4ac0ccc3e8424fe92bc9cb8e5f24f2e253eaf8fd72d530cd3b3
|
| 3 |
-
size 39364887
|
|
|
|
|
|
|
|
|
|
|
|
pooling_head.pt
DELETED
|
@@ -1,3 +0,0 @@
|
|
| 1 |
-
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:6e63ef6b4fc696fd3687c59c0160448dc9b155d49ce28d2195819a6ca3f0bb5f
|
| 3 |
-
size 1774799
|
|
|
|
|
|
|
|
|
|
|
|
relation_head.pt
DELETED
|
@@ -1,3 +0,0 @@
|
|
| 1 |
-
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:b4fe7c7fb314f7b5e0de6af9716f49681f4a854baf5e28a1dd6540244c832a15
|
| 3 |
-
size 2503773
|
|
|
|
|
|
|
|
|
|
|
|
release_manifest.json
DELETED
|
@@ -1,69 +0,0 @@
|
|
| 1 |
-
{
|
| 2 |
-
"version": "2.1.0",
|
| 3 |
-
"files": {
|
| 4 |
-
"config.json": {
|
| 5 |
-
"sha256": "9cb200e53b8bf3f5f8287a6e19a5cd1505aa08d88244b7fd151343a7ebcb9056",
|
| 6 |
-
"bytes": 1696
|
| 7 |
-
},
|
| 8 |
-
"evaluation_source_snapshot.zip": {
|
| 9 |
-
"sha256": "429ec47a44248de2f09eb068a898d82902028550e80d960c69aba249eca225e5",
|
| 10 |
-
"bytes": 23235
|
| 11 |
-
},
|
| 12 |
-
"evaluation_v2_1_int8.json": {
|
| 13 |
-
"sha256": "4df23682606b987ac43783fa3fe22ba6fde0b8c1f171626cfc62d32149cd3599",
|
| 14 |
-
"bytes": 5275
|
| 15 |
-
},
|
| 16 |
-
"export_provenance.json": {
|
| 17 |
-
"sha256": "a02cf0e75dda312a646924ff80c5c3dfd212c13fde424bc6b7cd9ab89adf0519",
|
| 18 |
-
"bytes": 704
|
| 19 |
-
},
|
| 20 |
-
"model.safetensors": {
|
| 21 |
-
"sha256": "0e4204f2849be97faf47fe1a748559f0766773af6cc586facf6f32fd50f551cc",
|
| 22 |
-
"bytes": 152499656
|
| 23 |
-
},
|
| 24 |
-
"model_int8.onnx": {
|
| 25 |
-
"sha256": "ca8eb040e1b5b51e05f5ceaf125073370fdcdd475e8a99c9dac0b03e46a0e7eb",
|
| 26 |
-
"bytes": 38667189
|
| 27 |
-
},
|
| 28 |
-
"README.md": {
|
| 29 |
-
"sha256": "2aaa41f6f76a70814d4487f715203bc47ab453603cd9e9c0b5ad2d00a0bd944c",
|
| 30 |
-
"bytes": 6658
|
| 31 |
-
},
|
| 32 |
-
"RELEASE_PROVENANCE.md": {
|
| 33 |
-
"sha256": "55927f9002d48a1115772102ad0febe909f4af9e3178fb1b8c9a6cd1fe9411d7",
|
| 34 |
-
"bytes": 1666
|
| 35 |
-
},
|
| 36 |
-
"release_qualification.json": {
|
| 37 |
-
"sha256": "0473e8673e33c6539bbf5153399e88858f1681acdc747de7dd6f0e30aec0be60",
|
| 38 |
-
"bytes": 1797
|
| 39 |
-
},
|
| 40 |
-
"SOURCES.md": {
|
| 41 |
-
"sha256": "e33cc1b19db75303d26da699a474bc3d1cf4d772db654c2c21dc1582842465ac",
|
| 42 |
-
"bytes": 10079
|
| 43 |
-
},
|
| 44 |
-
"source_snapshot.zip": {
|
| 45 |
-
"sha256": "56f1052e37ebdf70d815dc0d24d6b0ab3acd854dce0fa615a3d6d4f97e792600",
|
| 46 |
-
"bytes": 34512
|
| 47 |
-
},
|
| 48 |
-
"special_tokens_map.json": {
|
| 49 |
-
"sha256": "e83cab90dfc594d568809f9a6c6ce31e5c703ad4fcd04a3519a914017da326b9",
|
| 50 |
-
"bytes": 870
|
| 51 |
-
},
|
| 52 |
-
"tokenizer.json": {
|
| 53 |
-
"sha256": "53660382e792d55b6facc7bb42adb08fde1c3a2baa74a0f5efd11539eaf81aac",
|
| 54 |
-
"bytes": 1312401
|
| 55 |
-
},
|
| 56 |
-
"tokenizer_config.json": {
|
| 57 |
-
"sha256": "dc38f542c42e23bf36829a422e6f4161c5a2a13c5c7a2ad4cb7c4f3542a0677c",
|
| 58 |
-
"bytes": 635
|
| 59 |
-
},
|
| 60 |
-
"training_metrics.json": {
|
| 61 |
-
"sha256": "166c7910d19b9be007cc5b0a183bf69b3197271d58427720745bfc528a284ea2",
|
| 62 |
-
"bytes": 2910
|
| 63 |
-
},
|
| 64 |
-
"training_provenance.json": {
|
| 65 |
-
"sha256": "06b78000c7d7b1703e54c8a929399e50d8d578b482c6e270aff965d2babc382d",
|
| 66 |
-
"bytes": 921
|
| 67 |
-
}
|
| 68 |
-
}
|
| 69 |
-
}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
release_manifest_v4.json
DELETED
|
@@ -1,82 +0,0 @@
|
|
| 1 |
-
{
|
| 2 |
-
"RELEASE_PROVENANCE.md": {
|
| 3 |
-
"bytes": 3040,
|
| 4 |
-
"sha256": "ed68a2d5059d6fa46bc0a1b1396ed5a750adda9083bec7c62e64ed2875861a20"
|
| 5 |
-
},
|
| 6 |
-
"config.json": {
|
| 7 |
-
"bytes": 1696,
|
| 8 |
-
"sha256": "9cb200e53b8bf3f5f8287a6e19a5cd1505aa08d88244b7fd151343a7ebcb9056"
|
| 9 |
-
},
|
| 10 |
-
"evaluation_drugbank_bootstrap.json": {
|
| 11 |
-
"bytes": 1650,
|
| 12 |
-
"sha256": "aeab00eebe9209a013af43eb3789e8f54823dfd98b16346c1d45c50e6c93124f"
|
| 13 |
-
},
|
| 14 |
-
"evaluation_drugbank_candidate.json": {
|
| 15 |
-
"bytes": 694523,
|
| 16 |
-
"sha256": "667174e7cce9e479b6615c6585a25a2ca54ccb7b02d056815539bf423579b1d0"
|
| 17 |
-
},
|
| 18 |
-
"evaluation_rxnorm_bootstrap.json": {
|
| 19 |
-
"bytes": 1620,
|
| 20 |
-
"sha256": "145235673944cbacc164caf1f70a481bb598ec4533d219ab73bc26da871705c6"
|
| 21 |
-
},
|
| 22 |
-
"evaluation_rxnorm_candidate.json": {
|
| 23 |
-
"bytes": 1586990,
|
| 24 |
-
"sha256": "4eb5f48d785a18a553d7b7c6543cfb93f3726cb6bfa65a03bee23cdb2c26a717"
|
| 25 |
-
},
|
| 26 |
-
"evaluation_semantic_candidate.json": {
|
| 27 |
-
"bytes": 1824,
|
| 28 |
-
"sha256": "1d69a54bb0dd98837a515c8b5eed46d376c194b20ea31d6e1df4db310e6be93a"
|
| 29 |
-
},
|
| 30 |
-
"evaluation_slo_candidate.json": {
|
| 31 |
-
"bytes": 1097,
|
| 32 |
-
"sha256": "3c001c3ed116d49d6b6d8ef652cd1c0b68c633c31f34a25096c227e23f7d4bb6"
|
| 33 |
-
},
|
| 34 |
-
"export_provenance.json": {
|
| 35 |
-
"bytes": 2239,
|
| 36 |
-
"sha256": "59b6e299fe1aa7dab80c48a0c5276f2e588d311ecff8302c21024bfb31b4e7b5"
|
| 37 |
-
},
|
| 38 |
-
"model.onnx": {
|
| 39 |
-
"bytes": 154550326,
|
| 40 |
-
"sha256": "2c0d52cb9069f42f9469898837d73f4dd126f94d0379d90dfe1e1abee0ddd839"
|
| 41 |
-
},
|
| 42 |
-
"model.safetensors": {
|
| 43 |
-
"bytes": 154271056,
|
| 44 |
-
"sha256": "7f1f3d42fd003a800b52cbda192af19bea4184f27bd87ea6789542e0d4acdf54"
|
| 45 |
-
},
|
| 46 |
-
"model_compact_37m_int8.onnx": {
|
| 47 |
-
"bytes": 39364887,
|
| 48 |
-
"sha256": "44cd7c10f351d4ac0ccc3e8424fe92bc9cb8e5f24f2e253eaf8fd72d530cd3b3"
|
| 49 |
-
},
|
| 50 |
-
"model_int8.onnx": {
|
| 51 |
-
"bytes": 133463094,
|
| 52 |
-
"sha256": "9da09372f38401938c6275e6641d6312ca52e3e08f4c15bbe696f4e89b61f295"
|
| 53 |
-
},
|
| 54 |
-
"pooling_head.pt": {
|
| 55 |
-
"bytes": 1774799,
|
| 56 |
-
"sha256": "6e63ef6b4fc696fd3687c59c0160448dc9b155d49ce28d2195819a6ca3f0bb5f"
|
| 57 |
-
},
|
| 58 |
-
"relation_head.pt": {
|
| 59 |
-
"bytes": 2503773,
|
| 60 |
-
"sha256": "b4fe7c7fb314f7b5e0de6af9716f49681f4a854baf5e28a1dd6540244c832a15"
|
| 61 |
-
},
|
| 62 |
-
"special_tokens_map.json": {
|
| 63 |
-
"bytes": 870,
|
| 64 |
-
"sha256": "e83cab90dfc594d568809f9a6c6ce31e5c703ad4fcd04a3519a914017da326b9"
|
| 65 |
-
},
|
| 66 |
-
"tokenizer.json": {
|
| 67 |
-
"bytes": 1312412,
|
| 68 |
-
"sha256": "dc0e20c81f2ba89f3f1d0a66b315c444aa761551fbed3e320410eeebf1e80483"
|
| 69 |
-
},
|
| 70 |
-
"tokenizer_config.json": {
|
| 71 |
-
"bytes": 636,
|
| 72 |
-
"sha256": "0bd6f388bc813908f40cff3c5cdf0065a444b351d19b40a086f138a80a9a7747"
|
| 73 |
-
},
|
| 74 |
-
"training_metrics.json": {
|
| 75 |
-
"bytes": 1191,
|
| 76 |
-
"sha256": "078ae2d6f665bda635a8bca8d1f1ab702fb2d84a7239b71d0ffde4afba9e5124"
|
| 77 |
-
},
|
| 78 |
-
"training_provenance.json": {
|
| 79 |
-
"bytes": 6417,
|
| 80 |
-
"sha256": "c811549b591e207155a85e17e18ac549621e1772011407840e8bdaed1bc1871b"
|
| 81 |
-
}
|
| 82 |
-
}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
release_qualification.json
DELETED
|
@@ -1,24 +0,0 @@
|
|
| 1 |
-
{
|
| 2 |
-
"release_regression_passed": true,
|
| 3 |
-
"native_receipt_sha256": "a5ef038e3bc79edcb61b52e27ebc0ceb172ca2b37df357092038fc008162d43b",
|
| 4 |
-
"unchanged_artifact_receipt_sha256": "96ff4bf4ea6a8237a669482c6075a69ccee6b1a2e8d96b437018757d6e272f63",
|
| 5 |
-
"code_sha256": {
|
| 6 |
-
"evaluation/classification_protocol.py": "901cacff2aaaf50146e81b4a383240f991175d18379a5c8a78cc6d5e247c42c3",
|
| 7 |
-
"evaluation/classification_integrity.py": "1e188eb42bd8fe68d2f985b23ef472cd4d9d933bb1f83786e60aa27c780eaa99",
|
| 8 |
-
"evaluation/native_consumer_check.py": "30b9e53e10f71d4ae4fd17befb22b7290b8f5714b9fb3dca83efe9aebf1472bd",
|
| 9 |
-
"evaluation/native_score_diagnostic.py": "a3439238ea37b3aecf08d348031fdd4e842fa865db32f77e5238b6544810986b",
|
| 10 |
-
"evaluation/retention_eval.py": "ce97f69e38559e572fbe175af6f20acfccfc3656140b90df06bb88227f47e424",
|
| 11 |
-
"evaluation/adaptation_eval.py": "2e2b1b50b78f0b162f7936c43888d1d9601121608f200b2947d384f43cacbf4f",
|
| 12 |
-
"evaluation/check_score_release.py": "e2181d3be68a3a8ca6e26d8715b82125157cdd73039795744b497c8f86bd6dfb",
|
| 13 |
-
"evaluation/check_int8_release.py": "df87399230d67f69430b9f54f2af2385a5fc79fd04dd449a2bbf0d198ab60d1a",
|
| 14 |
-
"evaluation/check_concept_release.py": "75d1fa496008aeee98c1ff0d508e1408a05f720d88e37cc5c93d972c216dc63b",
|
| 15 |
-
"evaluation/check_adaptation_release.py": "1cedba3d2ad415ca85c6738f01dd233989a83fc436a29e930efba42d5eea6fd4",
|
| 16 |
-
"evaluation/score_preservation.py": "35f8dfd127b40d4341b35f0eb0a30aa9a287d43f8a4cd62fde3a325ee0f2ce50",
|
| 17 |
-
"evaluation/onnx_encoder.py": "ea6b5195c8b21294c1ed625e53c62d895205c5eea809952c5b6c92966c4de46b",
|
| 18 |
-
"evaluation/concept_retrieval.py": "48cdf2aa586123dad43c1afb4a3bb0bf8041729cf5d69ddeb1890067adddf58e",
|
| 19 |
-
"training/data_prep/grouped_nursing.py": "cd8030d98757703cd1c8df0530279bb3b563c9c5082ae18d31dcadc635e1176b"
|
| 20 |
-
},
|
| 21 |
-
"automatic_classification_qualified": false,
|
| 22 |
-
"release_judgment": "retracted_for_automatic_classification",
|
| 23 |
-
"reason": "Relative regression improvement was insufficient: 31/100 correct automatic classifications, 69/100 unclassified."
|
| 24 |
-
}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
source_snapshot.zip
DELETED
|
@@ -1,3 +0,0 @@
|
|
| 1 |
-
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:56f1052e37ebdf70d815dc0d24d6b0ab3acd854dce0fa615a3d6d4f97e792600
|
| 3 |
-
size 34512
|
|
|
|
|
|
|
|
|
|
|
|
training_metrics.json
DELETED
|
@@ -1,27 +0,0 @@
|
|
| 1 |
-
{
|
| 2 |
-
"artifact_verified": false,
|
| 3 |
-
"base_pairs": 171340,
|
| 4 |
-
"drugbank_pairs": 24534,
|
| 5 |
-
"evaluation_completed": false,
|
| 6 |
-
"freeze_backbone": false,
|
| 7 |
-
"note": "All formally acquired sources plus regulated-exam candidate included; quality and promotion gates remain required.",
|
| 8 |
-
"output": "/home/gloomcheng/Workspace/weemed-ai/terminology-embedder/research/artifacts/ilhaembed-autonomous-unbiased-source-conditioned-20260919/runs/trial-0004/model",
|
| 9 |
-
"pooling_mode": "attention_pooling",
|
| 10 |
-
"promotion_allowed": false,
|
| 11 |
-
"provenance_sha256": "c811549b591e207155a85e17e18ac549621e1772011407840e8bdaed1bc1871b",
|
| 12 |
-
"regulated_exam_pairs": 17,
|
| 13 |
-
"relation_head_mode": "source_conditioned_interaction",
|
| 14 |
-
"relation_loss_mode": "damped_power_law_cross_entropy",
|
| 15 |
-
"retention_loss_mode": "pairwise_logit_kl",
|
| 16 |
-
"retention_loss_weight": 0.25,
|
| 17 |
-
"run_id": "trial-0004",
|
| 18 |
-
"rxnorm_pairs": 74267,
|
| 19 |
-
"schema": "ilhaembed-integrated-regulated-training-result/v1",
|
| 20 |
-
"status": "COMPLETED",
|
| 21 |
-
"steps": 3838,
|
| 22 |
-
"training_completed": true,
|
| 23 |
-
"typed_relation_batch_size": 16,
|
| 24 |
-
"typed_relations": 760971,
|
| 25 |
-
"typed_sampling_mode": "relation_balanced",
|
| 26 |
-
"typed_split_id": "concept_group_split_seed_20260914_train"
|
| 27 |
-
}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
training_provenance.json
DELETED
|
@@ -1,170 +0,0 @@
|
|
| 1 |
-
{
|
| 2 |
-
"base_model": "/mnt/d/Workspace/research/med-embed/ilha-embed-97m-pruned-soup-labft",
|
| 3 |
-
"base_model_sha256": "1ce46425e538a18a401e9d29021957e6a2c3279e18805dca673e7565cb788a0e",
|
| 4 |
-
"config": {
|
| 5 |
-
"base_per_batch": 40,
|
| 6 |
-
"batch_size": 64,
|
| 7 |
-
"device": "cuda",
|
| 8 |
-
"drugbank_per_batch": 8,
|
| 9 |
-
"freeze_backbone": false,
|
| 10 |
-
"grad_clip": 1.0,
|
| 11 |
-
"learning_rate": 5e-06,
|
| 12 |
-
"max_length": 32,
|
| 13 |
-
"pooling_heads": 4,
|
| 14 |
-
"pooling_mode": "attention_pooling",
|
| 15 |
-
"regulated_exam_per_batch": 4,
|
| 16 |
-
"relation_head_mode": "source_conditioned_interaction",
|
| 17 |
-
"relation_loss_mode": "damped_power_law_cross_entropy",
|
| 18 |
-
"relation_loss_weight": 0.25,
|
| 19 |
-
"retention_loss_mode": "pairwise_logit_kl",
|
| 20 |
-
"retention_loss_weight": 0.25,
|
| 21 |
-
"rxnorm_per_batch": 10,
|
| 22 |
-
"seed": 42,
|
| 23 |
-
"steps": 3838,
|
| 24 |
-
"temperature": 0.05,
|
| 25 |
-
"typed_per_batch": 2,
|
| 26 |
-
"typed_relation_batch_size": 16,
|
| 27 |
-
"typed_sampling_mode": "relation_balanced",
|
| 28 |
-
"typed_split_id": "concept_group_split_seed_20260914_train",
|
| 29 |
-
"weight_decay": 0.01
|
| 30 |
-
},
|
| 31 |
-
"evaluation_status": "not_run_by_trainer",
|
| 32 |
-
"inputs": {
|
| 33 |
-
"base_pairs": {
|
| 34 |
-
"path": "/home/gloomcheng/Workspace/weemed-ai/terminology-embedder/intake-embedder/data/synonyms.tsv",
|
| 35 |
-
"rows": 171340,
|
| 36 |
-
"sha256": "e4fb5c68823b3029f7894931485e668f5c1c047f18d0e185452f9741d6c52386"
|
| 37 |
-
},
|
| 38 |
-
"drugbank_pairs": {
|
| 39 |
-
"path": "/home/gloomcheng/Workspace/weemed-ai/terminology-embedder/research/artifacts/ilhaembed-integrated-all-sources-v2-20260913/corpus/lexical_train_pairs.tsv",
|
| 40 |
-
"rows": 24534,
|
| 41 |
-
"sha256": "821168035c2ed48a2a175ab6f6fe4c685473f0ac1f5de354df8365cc4a2ba497",
|
| 42 |
-
"train_concepts": 13120
|
| 43 |
-
},
|
| 44 |
-
"regulated_exam_pairs": {
|
| 45 |
-
"path": "/home/gloomcheng/Workspace/weemed-ai/terminology-embedder/research/artifacts/ilhaembed-integrated-all-sources-v2-20260913/corpus/lexical_train_pairs.tsv",
|
| 46 |
-
"rows": 17,
|
| 47 |
-
"sha256": "821168035c2ed48a2a175ab6f6fe4c685473f0ac1f5de354df8365cc4a2ba497"
|
| 48 |
-
},
|
| 49 |
-
"rxnorm_pairs": {
|
| 50 |
-
"path": "/home/gloomcheng/Workspace/weemed-ai/terminology-embedder/research/artifacts/ilhaembed-integrated-all-sources-v2-20260913/corpus/lexical_train_pairs.tsv",
|
| 51 |
-
"rows": 74267,
|
| 52 |
-
"sha256": "821168035c2ed48a2a175ab6f6fe4c685473f0ac1f5de354df8365cc4a2ba497",
|
| 53 |
-
"train_concepts": 18138
|
| 54 |
-
},
|
| 55 |
-
"typed_relations": {
|
| 56 |
-
"path": "/home/gloomcheng/Workspace/weemed-ai/terminology-embedder/research/artifacts/ilhaembed-typed-relation-split-20260914-v2/train.jsonl",
|
| 57 |
-
"relation_types": [
|
| 58 |
-
"DRUGBANK_HAS_CAS",
|
| 59 |
-
"DRUGBANK_HAS_INCHI_KEY",
|
| 60 |
-
"DRUGBANK_HAS_UNII",
|
| 61 |
-
"FDA_PRODUCT_HAS_ACTIVE_INGREDIENT",
|
| 62 |
-
"FDA_PRODUCT_HAS_ATC",
|
| 63 |
-
"FDA_PRODUCT_INDICATED_FOR",
|
| 64 |
-
"OPENFDA_SUBSTANCE_HAS_INTERACTION",
|
| 65 |
-
"OPENFDA_SUBSTANCE_HAS_INTERACTION_EVIDENCE",
|
| 66 |
-
"OPENFDA_SUBSTANCE_HAS_NAME",
|
| 67 |
-
"OPENFDA_SUBSTANCE_HAS_SYNONYM",
|
| 68 |
-
"PROGRAM_HAS_DECISION_AUTHORITY",
|
| 69 |
-
"PROGRAM_REQUIRES_CONTEXT",
|
| 70 |
-
"PROGRAM_REQUIRES_ITEM",
|
| 71 |
-
"RXNREL_RB_contained_in",
|
| 72 |
-
"RXNREL_RB_has_quantified_form",
|
| 73 |
-
"RXNREL_RB_has_tradename",
|
| 74 |
-
"RXNREL_RN_contains",
|
| 75 |
-
"RXNREL_RN_quantified_form_of",
|
| 76 |
-
"RXNREL_RN_tradename_of",
|
| 77 |
-
"RXNSAT_AMBIGUITY_FLAG",
|
| 78 |
-
"RXNSAT_ANADA",
|
| 79 |
-
"RXNSAT_ANDA",
|
| 80 |
-
"RXNSAT_BLA",
|
| 81 |
-
"RXNSAT_COLOR",
|
| 82 |
-
"RXNSAT_COLORTEXT",
|
| 83 |
-
"RXNSAT_DCSA",
|
| 84 |
-
"RXNSAT_DM_SPL_ID",
|
| 85 |
-
"RXNSAT_EXEMPT_DEVICE",
|
| 86 |
-
"RXNSAT_IMPRINT_CODE",
|
| 87 |
-
"RXNSAT_LABELER",
|
| 88 |
-
"RXNSAT_LABEL_TYPE",
|
| 89 |
-
"RXNSAT_MARKETING_CATEGORY",
|
| 90 |
-
"RXNSAT_MARKETING_EFFECTIVE_TIME_HIGH",
|
| 91 |
-
"RXNSAT_MARKETING_EFFECTIVE_TIME_LOW",
|
| 92 |
-
"RXNSAT_MARKETING_STATUS",
|
| 93 |
-
"RXNSAT_NADA",
|
| 94 |
-
"RXNSAT_NDA",
|
| 95 |
-
"RXNSAT_NDA_AUTHORIZED_GENERIC",
|
| 96 |
-
"RXNSAT_NDC",
|
| 97 |
-
"RXNSAT_NHRIC",
|
| 98 |
-
"RXNSAT_OTC_MONOGRAPH_DRUG",
|
| 99 |
-
"RXNSAT_OTC_MONOGRAPH_FINAL",
|
| 100 |
-
"RXNSAT_OTC_MONOGRAPH_NOT_FINAL",
|
| 101 |
-
"RXNSAT_PREMARKET_APPLICATION",
|
| 102 |
-
"RXNSAT_PREMARKET_NOTIFICATION",
|
| 103 |
-
"RXNSAT_RXN_ACTIVATED",
|
| 104 |
-
"RXNSAT_RXN_AI",
|
| 105 |
-
"RXNSAT_RXN_AM",
|
| 106 |
-
"RXNSAT_RXN_AVAILABLE_STRENGTH",
|
| 107 |
-
"RXNSAT_RXN_BOSS_FROM",
|
| 108 |
-
"RXNSAT_RXN_HUMAN_DRUG",
|
| 109 |
-
"RXNSAT_RXN_OBSOLETED",
|
| 110 |
-
"RXNSAT_RXN_QUALITATIVE_DISTINCTION",
|
| 111 |
-
"RXNSAT_RXN_QUANTITY",
|
| 112 |
-
"RXNSAT_RXN_VET_DRUG",
|
| 113 |
-
"RXNSAT_RXTERM_FORM",
|
| 114 |
-
"RXNSAT_SCORE",
|
| 115 |
-
"RXNSAT_SHAPE",
|
| 116 |
-
"RXNSAT_SHAPETEXT",
|
| 117 |
-
"RXNSAT_SIZE",
|
| 118 |
-
"RXNSAT_SPL_SET_ID",
|
| 119 |
-
"RXNSAT_UNAPPROVED_DRUG_OTHER",
|
| 120 |
-
"WORKFLOW_EVENT_HAS_ITEM",
|
| 121 |
-
"WORKFLOW_EVENT_HAS_STATUS",
|
| 122 |
-
"WORKFLOW_EVENT_NOT_SYNONYM_OF"
|
| 123 |
-
],
|
| 124 |
-
"rows": 760971,
|
| 125 |
-
"sha256": "2f21d69b027c9cacfc4edeacbbf57441133a9a8803d12641cd7ec76fa0e3ca45"
|
| 126 |
-
}
|
| 127 |
-
},
|
| 128 |
-
"loss": {
|
| 129 |
-
"formula": "lexical_loss + relation_loss_weight * relation_loss_mode(typed_relation_cross_entropy) + retention_loss_weight * retention_loss_mode",
|
| 130 |
-
"lexical_mean": 0.6199783020074818,
|
| 131 |
-
"relation_mean": 3.5418746513646986,
|
| 132 |
-
"retention_mean": 0.39487592631201246
|
| 133 |
-
},
|
| 134 |
-
"output_tree_sha256": {
|
| 135 |
-
"config.json": "9cb200e53b8bf3f5f8287a6e19a5cd1505aa08d88244b7fd151343a7ebcb9056",
|
| 136 |
-
"model.safetensors": "7f1f3d42fd003a800b52cbda192af19bea4184f27bd87ea6789542e0d4acdf54",
|
| 137 |
-
"pooling_head.pt": "6e63ef6b4fc696fd3687c59c0160448dc9b155d49ce28d2195819a6ca3f0bb5f",
|
| 138 |
-
"relation_head.pt": "b4fe7c7fb314f7b5e0de6af9716f49681f4a854baf5e28a1dd6540244c832a15",
|
| 139 |
-
"tokenizer.json": "dc0e20c81f2ba89f3f1d0a66b315c444aa761551fbed3e320410eeebf1e80483",
|
| 140 |
-
"tokenizer_config.json": "0bd6f388bc813908f40cff3c5cdf0065a444b351d19b40a086f138a80a9a7747"
|
| 141 |
-
},
|
| 142 |
-
"project_id": "ilhaembed",
|
| 143 |
-
"relation_policy": {
|
| 144 |
-
"identifier_relations_as_synonyms": false,
|
| 145 |
-
"regulated_workflow_status_as_synonym": false,
|
| 146 |
-
"same_concept_pairs_in_lexical_loss": true,
|
| 147 |
-
"typed_relations_in_auxiliary_relation_head": true
|
| 148 |
-
},
|
| 149 |
-
"run_id": "trial-0004",
|
| 150 |
-
"sampling": {
|
| 151 |
-
"source_row_stream_seeds": {
|
| 152 |
-
"drugbank": 51,
|
| 153 |
-
"openfda": 52,
|
| 154 |
-
"regulated_exam": 53,
|
| 155 |
-
"rxnorm": 54,
|
| 156 |
-
"tfda": 55
|
| 157 |
-
},
|
| 158 |
-
"source_stream_seed": 50,
|
| 159 |
-
"source_types": [
|
| 160 |
-
"drugbank",
|
| 161 |
-
"openfda",
|
| 162 |
-
"regulated_exam",
|
| 163 |
-
"rxnorm",
|
| 164 |
-
"tfda"
|
| 165 |
-
],
|
| 166 |
-
"typed_sampling_mode": "relation_balanced"
|
| 167 |
-
},
|
| 168 |
-
"schema": "ilhaembed-integrated-regulated-training-provenance/v1",
|
| 169 |
-
"training_status": "completed"
|
| 170 |
-
}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|