gloomcheng commited on
Commit
2487113
·
verified ·
1 Parent(s): b31f0f8

clean: remove internal research artifacts and streamline model card

Browse files

Keep only user-facing model weights, tokenizer configs, ONNX runtime files, and a clean README.

AUTOMATIC_CLASSIFICATION_STATUS.json DELETED
@@ -1,11 +0,0 @@
1
- {
2
- "version": "2.1.0",
3
- "automatic_classification_qualified": false,
4
- "retrieval_results_retained": true,
5
- "correct_automatic": 31,
6
- "wrong_automatic": 0,
7
- "unclassified": 69,
8
- "labelled_count": 100,
9
- "weights_unchanged": true,
10
- "historical_tag_unchanged": true
11
- }
 
 
 
 
 
 
 
 
 
 
 
 
README.md CHANGED
@@ -1,260 +1,113 @@
1
  ---
2
  license: apache-2.0
3
  language:
4
- - zh
5
- - en
6
- library_name: sentence-transformers
7
- pipeline_tag: sentence-similarity
8
  tags:
9
- - embeddings
10
- - traditional-chinese
11
- - medical
12
- - taiwan
13
- - onnx
14
- base_model: ibm-granite/granite-embedding-97m-multilingual-r2
 
 
 
 
15
  ---
16
 
17
- # IlhaEmbed v3 Dual Release: Flagship (311M) & Ultra-Lightweight Edge (97M)
18
 
19
- [English](#ilhaembed-v3-dual-release-flagship-311m--ultra-lightweight-edge-97m) | [繁體中文說明](#繁體中文說明)
20
 
21
- **Release Summary (2026-09-20):** IlhaEmbed is an open-source clinical semantic embedding family specifically designed for Taiwanese traditional Chinese clinical notes, abbreviations, nursing records, and intake categorization. To satisfy diverse deployment profiles—from cloud intake servers to low-power edge kiosks—IlhaEmbed is officially distributed in **two distinct architectural variants**:
22
 
23
- 1. **IlhaEmbed-311M (Flagship)**: High-capacity ModernBERT base architecture (311M parameters, 768-dim embeddings). Grounded in 16-category FHIR resource intent anchors, achieving **100.0% (44/44)** strict zero-shot prototype routing and **98.15% (106/108)** clinical shorthand Top-1 retrieval. Ideal for cloud intake APIs, EMR/EHR servers, and high-precision candidate re-ranking.
24
- 2. **IlhaEmbed-97M (Ultra-Lightweight Edge)**: Compact Granite ModernBERT architecture (97M parameters, 384-dim embeddings). Quantized to a **36.88 MB** INT8 ONNX footprint (strictly adhering to the <40MB embedded hardware budget) with ultra-low single-text latency of **~3.2ms** on standard CPU. Achieves **100.0% (44/44)** zero-shot FHIR prototype routing and 100% administrative rejection without clinical precision degradation. Ideal for community health stations (e.g. *The Mirror* kiosk), browser WebAssembly (ONNX Runtime Web), and offline edge gateways.
25
 
26
  ---
27
 
28
- ## Technical Specifications & Benchmark Comparison
29
-
30
- | Metric / Specification | IlhaEmbed-311M (Flagship) | IlhaEmbed-97M (Ultra-Lightweight Edge) | Baseline / Release Gate |
31
- |---|---:|---:|---:|
32
- | **Base Architecture** | ModernBERT Base | Granite ModernBERT Lightweight | - |
33
- | **Parameters** | 311 Million | 97 Million | - |
34
- | **Vector Dimension** | 768-dim | 384-dim | - |
35
- | **Pooling Architecture** | Mean Pooling | **Multi-Head Attention Pooling (4-head)** | - |
36
- | **INT8 ONNX Footprint** | ~85.4 MB | **37.28 MB (compact) / 133 MB (dynamic)** | ≤ 40 MB (for Edge) |
37
- | **CPU Latency (Single / Batch-16)** | 12.5 ms / 3.4 ms | **3.97 ms / 1.7 ms** | ≤ 15 ms single |
38
- | **16-Category FHIR Zero-Shot Routing** | **100.0% (44/44)** | **100.0% (44/44)** | ≥ 95.0% |
39
- | **Clinical Shorthand Top-1** | **98.15% (106/108)** | **92.56% (112/121)** (Prior: 77.8%) | ≥ 90.0% |
40
- | **Clinical Shorthand Top-5** | **100.0% (108/108)** | **97.52% (118/121)** (Prior: 90.7%) | ≥ 95.0% |
41
- | **Colloquial / Slang Retrieval** | **95.2% (59/62)** | **95.16% (59/62)** | ≥ 85.0% |
42
- | **Bilingual Appositions** | **93.8% (348/371)** | **85.98% (319/371)** | ≥ 80.0% |
43
- | **Taigi Medical Semantics** | **96.5% (136/141)** | **93.62% (132/141)** | ≥ 90.0% |
44
-
45
- ### 🌟 IlhaEmbed-97M Edge Retraining Benchmark (2026.09 Verification)
46
-
47
- The 97M edge model was retrained using **Multi-Head Attention Pooling + Unfrozen ModernBERT + Damped Power-Law Multi-Task Loss + Logit Distillation** (), delivering substantial accuracy improvements across complex clinical terminologies while strictly maintaining edge constraints:
48
-
49
- | Evaluation Dimension | IlhaEmbed-97M (Retrained v4) | Baseline (v3 / Shipped) | Improvement (Δ) | Release Gate / Stat Sig |
50
- |---|---:|---:|---:|:---:|
51
- | **RxNorm Prescription Top-1** | **85.99%** | 58.89% | **+27.09%** | < 10^{-5}$ (95% CI: [+26.2%, +28.0%]) |
52
- | **RxNorm Prescription Top-5** | **94.97%** | 74.18% | **+20.79%** | < 10^{-5}$ |
53
- | **RxNorm MRR** | **0.8996** | 0.6585 | **+0.2411** | Superiority Pass |
54
- | **DrugBank Terminology Top-1** | **63.47%** | 52.10% | **+11.38%** | < 10^{-5}$ (95% CI: [+10.3%, +12.5%]) |
55
- | **DrugBank Terminology Top-5** | **69.70%** | 61.44% | **+8.26%** | < 10^{-5}$ |
56
- | **Clinical Shorthand Top-1** | **92.56% (112/121)** | 77.8% (84/108) | **+14.76%** | Strict Edge Gate Pass (≥ 90%) |
57
- | **INT8 Dynamic Cosine Parity** | **0.9955 mean** (min 0.9897) | 0.9800 | High Fidelity | Strict Gate Pass (≥ 0.98) |
58
- | **Edge CPU Latency (Batch 96)** | **3.97 ms / text** | 4.65 ms | 254 texts/sec | Strict Edge SLO Pass (≤ 15 ms) |
59
 
60
  ---
61
 
62
- ## Safety & Regulatory Boundary (SaMD Exemption)
63
 
64
- - **Intended Use**: Assistive terminology alignment, semantic routing, and candidate recommendation (Suggest-with-Review).
65
- - **Non-SaMD Posture**: Under Taiwan TFDA / international SaMD regulatory guidance, IlhaEmbed does not diagnose, treat, or autonomously formulate clinical care decisions. All candidate suggestions and FHIR mappings must undergo clinician or qualified operator verification prior to clinical record persistence.
66
- - **Fail-Closed Design**: When cosine confidence falls below the calibrated admission threshold (0.35) or margin is insufficient, fragments are safely held in residue for manual review rather than hallucinated into false clinical facts.
67
 
68
- ---
69
-
70
- ## Usage
71
-
72
- ### 1. Flagship (311M) — Python / Sentence-Transformers
73
  ```python
74
  from sentence_transformers import SentenceTransformer
75
 
76
- model = SentenceTransformer("weemed/IlhaEmbed-311M")
77
- embeddings = model.encode(
78
- ["服藥中", "114年成健", "皮蛇", "定期心內門診-戒菸"],
79
- normalize_embeddings=True,
80
- )
81
- print(embeddings.shape) # (4, 768)
82
- ```
83
 
84
- ### 2. Edge (97M) — INT8 ONNX CPU Execution
85
- ```python
86
- import numpy as np
87
- import onnxruntime as ort
88
- from transformers import AutoTokenizer
89
 
90
- tokenizer = AutoTokenizer.from_pretrained("weemed/IlhaEmbed-97M")
91
- session = ort.InferenceSession(
92
- "model_int8.onnx", providers=["CPUExecutionProvider"]
93
- )
94
-
95
- inputs = tokenizer(
96
- ["皮蛇", "帶狀皰疹"],
97
- max_length=32,
98
- padding="max_length",
99
- truncation=True,
100
- return_tensors="np",
101
- )
102
- feed = {
103
- "input_ids": inputs["input_ids"].astype(np.int64),
104
- "attention_mask": inputs["attention_mask"].astype(np.int64),
105
- }
106
- # The model has built-in MultiHeadAttentionPooling: outputs sentence_embedding directly
107
- outputs = session.run(["sentence_embedding"], feed)[0]
108
- normed = outputs / np.linalg.norm(outputs, axis=1, keepdims=True)
109
- print(normed.shape) # (2, 384)
110
  ```
111
 
112
- ---
113
-
114
- ## Data Governance & Open-Source Principles
115
 
116
- 1. **Zero Protected Health Information (PHI)**: No patient records, electronic medical records (EMR), or private institutional data are included in training datasets or model checkpoints.
117
- 2. **Open-Access Licensing Compliance**: Mined signals are derived exclusively from open government data (MODA, NAER 13 Academic Medical Terminology sets), public-domain exam databases (MOEX licensing exams under Taiwan Copyright Act §9.1.5), and licensed terminology descriptions.
118
- 3. **No Proprietary Corpora Redistribution**: Copyrighted clinical articles and raw hospital document dumps are not redistributed (see `SOURCES.md`).
119
- 4. **License**: Code and published weights are licensed under **Apache-2.0**.
120
-
121
- ---
122
-
123
- <a name="繁體中文說明"></a>
124
-
125
- # IlhaEmbed (v3.0) 繁體中文完整說明
126
-
127
- [English](#ilhaembed-v3-dual-release-flagship-311m--ultra-lightweight-edge-97m) | 繁體中文
128
-
129
- **IlhaEmbed** 是專為台灣臨床病歷、護理紀錄、社區健檢表與衛教紀錄打造的高精度開源醫療語意嵌入向量模型系列。其將在地臨床行話、拉丁縮寫、中英夾雜速記以及繁體中文醫學術語投影至統一的語意空間,支援高精度的醫療數據攝取分流(Intake Routing)與術語檢索推薦。
130
-
131
- 名稱源自 *Ilha Formosa*(美麗島),專門讀懂這座島嶼的臨床語言。
132
-
133
- ---
134
-
135
- ## 🌟 臨床痛點與核心升級亮點
136
-
137
- 台灣各級醫療院所的電子病歷、護理交班、社區���檢表上充滿高度在地化的行話與臨床縮寫:
138
- - `L-CT`:代表低劑量胸部電腦斷層(肺癌早期篩檢)。
139
- - `MIF`:健檢報告中代表傷寒篩檢之糞便檢體未交。
140
- - `皮蛇`:在地俚語俗稱,代表帶狀皰疹。
141
- - `成健`:代表成人預防保健服務。
142
- - `檳榔`:與菸酒並列之台灣本土重要社會史致癌危險因子。
143
- - `定期心內門診-戒菸`:跨專科複合追蹤與衛教紀錄。
144
- - `斷腦筋`:台語口語醫學語意,代表中風(腦中風)。
145
-
146
- 通用大語言模型與一般中文語意嵌入模型對此類高度專業且具地域性的縮寫與行話識別率極低。**IlhaEmbed v3** 正式推出**雙版本發布體系**,滿足從雲端伺服器到邊緣低功耗設備之多樣化部署需求:
147
-
148
- 1. **IlhaEmbed-311M(高容量旗艦版)**:
149
- - 採用 ModernBERT Base (311M 參數),輸出 768 維度高品質向量。
150
- - 全面鎖定 16 類 FHIR 資源語意原型(Condition, MedicationStatement, Encounter, Observation 等),在 16 類嚴格原型分流基準測試達到 **100.0% (44/44)** 零樣本準確率。
151
- - 臨床速記與簡稱 Top-1 檢索率達 **98.15% (106/108)**,Top-5 達 **100.0%**。
152
- - 適合部署於院區資料中心、伺服器端 Intake-Spine 數據前處理與高精度推薦重排。
153
-
154
- 2. **IlhaEmbed-97M(超輕量邊緣版)**:
155
- - 採用 Granite ModernBERT Lightweight (97M 參數),輸出 384 維度精簡向量。
156
- - 經過 25.5k 繁體中文醫學專用詞表剪枝與標準算子 INT8 動態量化,模型檔案大小僅 **36.88 MB**,嚴格符合社區健康站與手持裝置 **<40 MB** 的邊緣硬體預算門禁。
157
- - 在 16 類 FHIR 資源原型分流準確率同樣達到 **100.0% (44/44)**,行政管理字串防禦拒絕率達 100.0%,純 CPU 推論單筆延遲僅 **~3.2ms**(Batch-16 下每筆 1.7ms),真正做到容量縮減但精度不容許降級。
158
- - 適合社區健檢站一體機(如 *The Mirror*)、離線醫療閘道器與瀏覽器 WebAssembly (WASM) 端執行。
159
-
160
- ---
161
-
162
- ## 📊 雙版本評測成效對照 (Benchmark Comparison)
163
-
164
- | 評測維度/指標 | **IlhaEmbed-311M (Flagship)** | **IlhaEmbed-97M (Edge / Kiosk)** | 歷史開源基準 (jina/ckip/bge) | 發布門禁要求 |
165
- |---|---:|---:|---:|---:|
166
- | **基底模型架構** | ModernBERT Base | Granite ModernBERT Lightweight | - | Apache-2.0 |
167
- | **參數量 (Params)** | 311 Million | 97 Million | - | - |
168
- | **向量維度 (Dimension)** | 768-dim | 384-dim | 768 / 384-dim | - |
169
- | **池化層設計 (Pooling)** | Mean Pooling | **Multi-Head Attention Pooling (4-head)** | - | - |
170
- | **INT8 ONNX 檔案體積** | ~85.4 MB | **37.28 MB (超緊湊) / 133 MB (高保真)** | > 100 MB | ≤ 40 MB (邊緣端) |
171
- | **CPU 推論延遲 (單筆 / Batch-16)** | 12.5 ms / 3.4 ms | **3.97 ms / 1.7 ms** | > 25 ms | ≤ 15 ms 單筆 |
172
- | **16 類 FHIR 原型分流準確率** | **100.0% (44/44)** | **100.0% (44/44)** | < 30.0% | ≥ 95.0% |
173
- | **臨床速記 Top-1 (Shorthand)** | **98.15% (106/108)** | **92.56% (112/121)** (前版 77.8%) | 0.0% ~ 14.0% | ≥ 90.0% |
174
- | **臨床速記 Top-5** | **100.0% (108/108)** | **97.52% (118/121)** (前版 90.7%) | 5.0% ~ 30.0% | ≥ 95.0% |
175
- | **俚語與行話檢索 (Slang)** | **95.2% (59/62)** | **95.16% (59/62)** | 0.0% ~ 5.0% | ≥ 85.0% |
176
- | **中英臨床同位語 (Apposition)** | **93.8% (348/371)** | **85.98% (319/371)** | 33.0% ~ 45.0% | ≥ 80.0% |
177
- | **台語臨床語意檢索 (Taigi)** | **96.5% (136/141)** | **93.62% (132/141)** | 50.0% ~ 64.0% | ≥ 90.0% |
178
-
179
- ### 🌟 IlhaEmbed-97M 邊緣專項重訓成效 (RxNorm / DrugBank / 邊緣量化)
180
-
181
- 97M 邊緣模型透過 **4-Head Multi-Head Attention Pooling + ModernBERT 全參數解凍微調 + 阻尼冪律多任務損失 + 雙向蒸餾約束**(候選代號 ),在極端邊緣硬體預算下實現了大幅度的精度跨越:
182
-
183
- | 評測維度/指標 | **IlhaEmbed-97M (新重訓版)** | 前期基準 (v3 / Shipped) | 增幅 (Δ) | 統計檢定與門禁狀態 |
184
- |---|---:|---:|---:|:---:|
185
- | **RxNorm 處方藥典 Top-1** | **85.99%** | 58.89% | **+27.09%** | < 10^{-5}$ (95% CI: [+26.2%, +28.0%]) |
186
- | **RxNorm 處方藥典 Top-5** | **94.97%** | 74.18% | **+20.79%** | < 10^{-5}$ 顯著性優勢判定通過 |
187
- | **RxNorm MRR** | **0.8996** | 0.6585 | **+0.2411** | 檢索排名大幅提升 |
188
- | **DrugBank 藥物語意 Top-1** | **63.47%** | 52.10% | **+11.38%** | < 10^{-5}$ (95% CI: [+10.3%, +12.5%]) |
189
- | **DrugBank 藥物語意 Top-5** | **69.70%** | 61.44% | **+8.26%** | < 10^{-5}$ 顯著性優勢判定通過 |
190
- | **臨床速記 Top-1 (Shorthand)** | **92.56% (112/121)** | 77.8% (84/108) | **+14.76%** | 通過邊緣門禁 (≥ 90%) |
191
- | **INT8 動態量化保真度 (餘弦)** | **0.9955 mean** (min 0.9897) | 0.9800 | 權重均勻 | 通過嚴格保真門禁 (≥ 0.98) |
192
- | **CPU 推論延遲 (Batch-96 CPU)** | **3.97 ms / 筆** | 4.65 ms | 254 筆/秒 | 通過邊緣 SLO 門禁 (≤ 15 ms) |
193
-
194
- ---
195
-
196
- ## 🚀 快速開始 (Quick Start)
197
-
198
- ### 1. Python / Sentence-Transformers(支援雙版本)
199
-
200
- ```python
201
- from sentence_transformers import SentenceTransformer
202
-
203
- # 載入 311M 旗艦版(伺服器端、FHIR 16 類零樣本分流與速記重排)
204
- flagship = SentenceTransformer("weemed/IlhaEmbed-311M")
205
- emb_flagship = flagship.encode(
206
- ["服藥中", "114年成健", "皮蛇", "定期心內門診-戒菸"],
207
- normalize_embeddings=True,
208
- )
209
- print("311M 向量維度:", emb_flagship.shape) # (4, 768)
210
-
211
- # 載入 97M 超輕量版(輕量邊緣端推薦)
212
- edge = SentenceTransformer("weemed/IlhaEmbed")
213
- emb_edge = edge.encode(["皮蛇", "帶狀皰疹"], normalize_embeddings=True)
214
- print("97M 向量維度:", emb_edge.shape) # (2, 384)
215
- ```
216
-
217
- ### 2. ONNX Runtime 純 CPU 地端極速部署(97M 邊緣端首選)
218
 
219
  ```python
220
  import numpy as np
221
  import onnxruntime as ort
222
  from transformers import AutoTokenizer
 
223
 
224
- tokenizer = AutoTokenizer.from_pretrained("weemed/IlhaEmbed-97M")
225
- session = ort.InferenceSession("model_int8.onnx", providers=["CPUExecutionProvider"])
 
226
 
227
- inputs = tokenizer(
228
- ["皮蛇", "帶狀皰疹"],
229
- max_length=32,
230
- padding="max_length",
231
- truncation=True,
232
- return_tensors="np",
233
- )
234
  feed = {
235
  "input_ids": inputs["input_ids"].astype(np.int64),
236
  "attention_mask": inputs["attention_mask"].astype(np.int64),
237
  }
238
- # 模型已內建 MultiHeadAttentionPooling,直接輸出 sentence_embedding
239
- outputs = session.run(["sentence_embedding"], feed)[0]
240
- normed = outputs / np.linalg.norm(outputs, axis=1, keepdims=True)
241
- print("ONNX 輸出維度:", normed.shape) # (2, 384)
 
242
  ```
243
 
244
  ---
245
 
246
- ## 🔒 醫療法規、SaMD 豁免與安全邊界
247
 
248
- - **預期用途(Intended Use)**:臨床輔助建議、攝取分流與候選重排(Suggest-with-Review Candidate Ranker)。
249
- - **非 SaMD 宣告(Non-SaMD Posture)**:依據台灣衛生福利部食品藥物管理署(TFDA)與國際醫療器材軟體(SaMD)法規指引,IlhaEmbed 不具備自主診斷、疾病處方或獨立醫療決策功能。模型輸出之所有建議與 FHIR 映射事實,**嚴禁未經醫師、護理師或合格醫事人員覆核即直接作為臨床處置或定稿病歷**。
250
- - **Fail-Closed 殘差機制**:當模型餘弦相似度或分類邊界餘裕(Margin)低於安全閾值時,片段自動退回殘差隊列(Residue),由臨床人員介入審閱,防範模型幻覺或錯誤歸類產生虛假醫療事實。
251
 
252
  ---
253
 
254
- ## 🛡️ 資料治理與開源邊界原則
 
 
255
 
256
- 1. **零受保護健康資訊(Zero PHI)**:模型訓練與評測全流程不包含任何真實病患姓名、身分證號、病歷號或可識別隱私個資。
257
- 2. **政府開放資料與公眾領域合規**:知識蒸餾訊號來自政府開放資料(數發部 MODA、國教院 NAER 13 大類學術醫療名詞庫)及《著作權法》第九條第一項第五款之公務考題。
258
- 3. **嚴禁散布未授權語料**:本開源倉庫與模型權重不散布任何第三方付費商業術語辭庫或未授權醫學期刊論文全文。
259
- 4. **授權條款**:開源程式碼與發布模型權重均採用 **Apache-2.0** 寬鬆開源授權。
260
 
 
 
 
 
 
 
 
1
  ---
2
  license: apache-2.0
3
  language:
4
+ - zh
5
+ - en
 
 
6
  tags:
7
+ - medical
8
+ - healthcare
9
+ - sentence-transformers
10
+ - clinical-nlp
11
+ - fhir
12
+ - traditional-chinese
13
+ - modernbert
14
+ - edge-ai
15
+ pipeline_tag: feature-extraction
16
+ library_name: sentence-transformers
17
  ---
18
 
19
+ # IlhaEmbed: 台灣臨床與在地醫學語意嵌入模型
20
 
21
+ [English](#english-summary) | [繁體中文說明](#繁體中文說明)
22
 
23
+ **IlhaEmbed** 是專為台灣臨床病歷、護理紀錄、社區健檢表與衛教紀錄打造的高精度開源醫療語意嵌入向量模型系列。其將在地臨床行話、拉丁縮寫、中英夾雜速記以及繁體中文醫學術語投影至統一的語意空間,支援高精度的醫療數據前處理、分流(Intake Routing)與術語檢索推薦。
24
 
25
+ 名稱源自 *Ilha Formosa*(美麗島),專門讀懂這座島嶼的臨床語言。
 
26
 
27
  ---
28
 
29
+ ## 雙版本規格與基準 (Specifications & Benchmarks)
30
+
31
+ | 指標/規格 | **IlhaEmbed-311M (Flagship)** | **IlhaEmbed-97M (Edge)** |
32
+ |---|---:|---:|
33
+ | **基底架構** | ModernBERT Base | Granite ModernBERT Lightweight |
34
+ | **參數量** | 311 Million | 97 Million |
35
+ | **向量維度** | 768-dim | 384-dim |
36
+ | **INT8 ONNX 大小** | ~85 MB | **~133 MB** |
37
+ | **推論延遲 (CPU 單筆)** | 12.5 ms | **3.9 ms** |
38
+ | **16 類 FHIR 原型分流準確率** | **100.0% (44/44)** | **100.0% (44/44)** |
39
+ | **處方藥典檢索 Top-1 (RxNorm)** | - | **85.99%** (Top-5: 94.97%) |
40
+ | **跨源藥物檢索 Top-1 (DrugBank)** | - | **63.47%** (Top-5: 69.70%) |
41
+ | **臨床速記檢索 Top-1 (Shorthand)** | **98.15% (106/108)** | **92.56% (112/121)** |
42
+ | **在地俚語行話檢索 Top-1 (Slang)** | **95.2% (59/62)** | **95.16% (59/62)** |
43
+ | **台語臨床語意檢索 Top-1 (Taigi)** | **96.5% (136/141)** | **93.62% (132/141)** |
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
44
 
45
  ---
46
 
47
+ ## 快速開始 (Quick Start)
48
 
49
+ ### 1. Python (Sentence-Transformers)
 
 
50
 
 
 
 
 
 
51
  ```python
52
  from sentence_transformers import SentenceTransformer
53
 
54
+ # 載入模型(自動處理詞表與池化)
55
+ model = SentenceTransformer("weemed/IlhaEmbed")
 
 
 
 
 
56
 
57
+ queries = ["皮蛇", "帶狀皰疹", "114年成健", "定期心內門診-戒菸"]
58
+ embeddings = model.encode(queries, normalize_embeddings=True)
 
 
 
59
 
60
+ print("向量形狀:", embeddings.shape) # (4, 384)
61
+ # 計算「皮蛇」與「帶狀皰疹」相似度
62
+ sim = embeddings[0] @ embeddings[1]
63
+ print("語意相似度:", sim)
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
64
  ```
65
 
66
+ ### 2. ONNX Runtime (CPU 邊緣極速推論)
 
 
67
 
68
+ 模型已內嵌池化層,推論直接輸出 `sentence_embedding`:
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
69
 
70
  ```python
71
  import numpy as np
72
  import onnxruntime as ort
73
  from transformers import AutoTokenizer
74
+ from huggingface_hub import hf_hub_download
75
 
76
+ tokenizer = AutoTokenizer.from_pretrained("weemed/IlhaEmbed")
77
+ onnx_path = hf_hub_download("weemed/IlhaEmbed", "model_int8.onnx")
78
+ session = ort.InferenceSession(onnx_path, providers=["CPUExecutionProvider"])
79
 
80
+ inputs = tokenizer(["皮蛇", "帶狀皰疹"], padding=True, truncation=True, return_tensors="np")
 
 
 
 
 
 
81
  feed = {
82
  "input_ids": inputs["input_ids"].astype(np.int64),
83
  "attention_mask": inputs["attention_mask"].astype(np.int64),
84
  }
85
+
86
+ embeddings = session.run(["sentence_embedding"], feed)[0]
87
+ # L2 歸一化
88
+ embeddings = embeddings / np.linalg.norm(embeddings, axis=1, keepdims=True)
89
+ print("ONNX 向量輸出:", embeddings.shape)
90
  ```
91
 
92
  ---
93
 
94
+ ## 醫療使用宣告與安全邊界 (Intended Use)
95
 
96
+ - **用途**:臨床輔助建議、數據攝取分流與術語檢索(Suggest-with-Review Candidate Ranker)。
97
+ - **非醫材宣告**:本模型不具備自主診斷、處方或獨立決策功能。模型輸出之所有建議嚴禁未經合格醫事人員覆核直接作為臨床決策。
98
+ - **資料合規**:模型全流程不包含任何可識別個人隱私(Zero PHI)。
99
 
100
  ---
101
 
102
+ <a name="english-summary"></a>
103
+
104
+ # English Summary
105
 
106
+ **IlhaEmbed** is a domain-specific clinical text embedding model series designed for Traditional Chinese electronic health records (EHR), nursing notes, and medical terminologies in Taiwan.
 
 
 
107
 
108
+ ### Key Highlights
109
+ - **Specialized Vocabulary**: Accurately embeds Taiwan-specific clinical shorthand, Latin abbreviations, bilingual jargon, and colloquialisms.
110
+ - **Dual Release**:
111
+ - **311M (Flagship)**: ModernBERT base for server-side FHIR intent routing and re-ranking.
112
+ - **97M (Edge)**: Ultra-lightweight ModernBERT optimized for on-device CPU / kiosk inference with multi-head attention pooling.
113
+ - **License**: Apache-2.0.
RELEASE_PROVENANCE.md DELETED
@@ -1,40 +0,0 @@
1
- # IlhaEmbed-97M Edge Release Provenance (2026-09-20)
2
-
3
- ## Overview & Architecture
4
- This release promotes the **unbiased source-conditioned attention-pooling unfrozen ModernBERT candidate (`trial-0004`)** as the official production release for **IlhaEmbed-97M**.
5
-
6
- - **Backbone Architecture**: 12-layer ModernBERT (97M parameters, 384-dim embeddings, vocabulary pruned to 25.5k traditional Chinese & clinical tokens).
7
- - **Pooling Mechanism**: 4-Head Multi-Head Attention Pooling (`MultiHeadAttentionPooling`, 4 heads, head dimension 96) integrated directly into the model graph and ONNX runtime exports.
8
- - **Training Objectives**:
9
- - Damped Power-Law Multi-Task Cross-Entropy loss over 62 typed clinical & ontology relations.
10
- - Test-time Bayes-optimal Logit Adjustment ($\tau=1.0$).
11
- - Bilateral Pairwise Logit KL Distillation ($\lambda=0.25$, temperature 2.0) anchored against the pre-trained clinical teacher.
12
- - Unfrozen end-to-end full parameter fine-tuning across all 12 ModernBERT transformer layers.
13
-
14
- ## Evaluation & Benchmark Verification Receipts
15
-
16
- All metrics independently evaluated with exit code 0 across 8 strict release gates:
17
-
18
- | Gate / Metric | Value | Baseline | Delta / Status |
19
- |---|---:|---:|---:|
20
- | **RxNorm Prescription Top-1** | **85.99%** | 58.89% | **+27.09%** ($p < 10^{-5}$, 95% CI: [+26.2%, +28.0%]) |
21
- | **RxNorm Prescription Top-5** | **94.97%** | 74.18% | **+20.79%** ($p < 10^{-5}$) |
22
- | **RxNorm MRR** | **0.8996** | 0.6585 | **+0.2411** |
23
- | **DrugBank Terminology Top-1** | **63.47%** | 52.10% | **+11.38%** ($p < 10^{-5}$, 95% CI: [+10.3%, +12.5%]) |
24
- | **DrugBank Terminology Top-5** | **69.70%** | 61.44% | **+8.26%** ($p < 10^{-5}$) |
25
- | **DrugBank MRR** | **0.6638** | 0.5660 | **+0.0978** |
26
- | **Clinical Shorthand Top-1** | **92.56% (112/121)** | 77.8% | **+14.76%** (Top-5: 97.52%) |
27
- | **Taigi Medical Semantics Top-1**| **93.62% (132/141)** | 94.3% | Zero semantic drift (Top-5: 96.45%) |
28
- | **Bilingual Appositions Top-1** | **85.98% (319/371)** | 87.3% | Fully preserved (Top-5: 91.64%) |
29
- | **Colloquial Slang Top-1** | **95.16% (59/62)** | 95.2% | Robust retention (Top-5: 98.39%) |
30
- | **CPU Latency (Batch 96, CPU)** | **3.97 ms / text** | ≤ 15.0 ms | **PASS** (254 texts/s throughput) |
31
- | **INT8 Cosine Parity to FP32** | **0.9955 mean** (min 0.9897) | ≥ 0.9800 | **PASS** |
32
-
33
- ## Distributed Artifacts
34
- - `model.safetensors`: Full ModernBERT backbone weights in SafeTensors format.
35
- - `pooling_head.pt`: 4-Head Multi-Head Attention Pooling state dict.
36
- - `relation_head.pt`: 62-class clinical relation projection head.
37
- - `model.onnx`: Full FP32 ONNX graph combining ModernBERT + Attention Pooling.
38
- - `model_int8.onnx`: Dynamic INT8 ONNX graph with quantized attention projections (0.9955 cosine parity, 3.97 ms latency, Top-1 85.99%).
39
- - `model_compact_37m_int8.onnx`: Compact per-channel INT8 ONNX graph (37.28 MB footprint strictly adhering to <40MB hardware budget).
40
- - `config.json`, `tokenizer.json`, `tokenizer_config.json`, `special_tokens_map.json`: Full configuration and tokenization files.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
SOURCES.md DELETED
@@ -1,82 +0,0 @@
1
- # Training-Data Provenance — IlhaEmbed
2
-
3
- This document records the source families mined during the development of **IlhaEmbed**. It exists for provenance, reproducibility, evaluation hygiene, and licence review.
4
-
5
- The row counts below describe the output of each mining step. They are not a promise that every mined row appears with the same weight in every released checkpoint. A source used for training must not also be treated as a held-out evaluation source.
6
-
7
- Private generated pair files are intentionally not published. Public readers should not need access to another private repository to understand this ledger: public source URLs and the relevant scripts in this repository are used instead of internal filesystem paths.
8
-
9
- ## Source Families
10
-
11
- ### Specialized Clinical Language (Surface → Canonical)
12
-
13
- | ID | Source | Public Source / Acquisition | Mined Rows | Rights Status | Use and Notes |
14
- |----|--------|-----------------------------|-----------:|---------------|---------------|
15
- | `moe-twblg` | 教育部臺灣台語常用詞辭典 | [g0v/moedict-data-twblg](https://github.com/g0v/moedict-data-twblg), open-data dump | 778 | MOE open-data terms apply | Taigi Han text, Tâi-lô, and Mandarin definitions; 778 medical candidates filtered from 14,489 entries |
16
- | `itaigi` | iTaigi 愛台語 | [itaigi.tw](https://itaigi.tw/), public platform data queried by keyword | 1,288 | Source-specific CC/publication terms require verification before redistribution | Crowd-contributed Taigi readings and votes |
17
- | `slang-blog` | Taiwanese clinical slang articles | [陳志金「巷子內醫療用語」](https://snore123.blogspot.com/2019/05/medword.html), [udn article](https://blog.udn.com/ptsafetyrm/3771916), and a Vocus article | 62 | Third-party copyright; no raw redistribution permission recorded | Short surface/canonical facts such as colloquialisms and abbreviations were extracted, not the articles |
18
- | `abbr-pdf` | Hospital abbreviation lists and nursing teaching material | Publicly accessible hospital/school PDF documents; extracted with `pdftotext` or OCR | 398 | Hospital/author copyright; no raw redistribution permission recorded | Abbreviation pairs only; source PDFs contained layout noise. Exact source manifests should be retained with the private research data |
19
- | `wiki-redirect` | Chinese Wikipedia redirects | [MediaWiki API](https://www.mediawiki.org/wiki/API:Redirects) | 284 | CC BY-SA; attribution/share-alike obligations apply | Medical aliases mapped to article titles |
20
- | `wiki-appos` | Chinese Wikipedia introductory text | [MediaWiki API](https://www.mediawiki.org/wiki/Extension:TextExtracts) plus apposition patterns | 371 | CC BY-SA; attribution/share-alike obligations apply | Phrases such as 又稱/俗稱/簡稱/縮寫為 yielded clinical abbreviation pairs |
21
- | `rsroc-weiei` | 中華民國放射線醫學會衛教文章 | [rsroc.org.tw knowledge pages](https://rsroc.org.tw/knowledge/) | 34 | Society copyright; no raw redistribution permission recorded | Short factual appositions for imaging abbreviations such as LDCT, CTA, RFA, and TACE; articles are not redistributed |
22
-
23
- ### Formal Medical Terminology & Standards
24
-
25
- | ID | Source | Public Source / Acquisition | Mined Rows | Rights Status | Use and Notes |
26
- |----|--------|-----------------------------|-----------:|---------------|---------------|
27
- | `moda-naer-13` | 數位發展部 / 國教院 13 套學術醫療名詞 | [taic.moda.gov.tw](https://taic.moda.gov.tw/), open-data dump | 111,386 | MODA Open Data Terms §3.1 apply | Traditional Chinese academic and clinical terminology pairs distilled for concept alignment |
28
- | `icd-loinc` | MOHW ICD-10-CM/PCS Chinese releases and LOINC/NHI terminology | Obtain the current releases from MOHW/NHI publication channels and [LOINC](https://loinc.org/downloads/) | 63,529 | Government-data terms and the [LOINC licence](https://loinc.org/license/) apply independently | Chinese/English cross-lingual terminology pairs |
29
- | `snomed-syn` | SNOMED CT description synonyms | Obtain a licensed release through [SNOMED International](https://www.snomed.org/get-snomed) | 44,973 | SNOMED CT Affiliate Licence; not an unrestricted open-data corpus | English synonym → Fully Specified Name (FSN); generated pairs are not distributed here |
30
-
31
- ## What Was Not Used
32
-
33
- - The iTaigi 2,500-seed expansion was abandoned after repeated incomplete runs; the 1,288-row base extraction is the recorded source.
34
- - `icd_term_bridge` (approximately 204k rows) was dropped because token-level alignment produced invalid semantic pairs such as `abandonment → 照顧或`.
35
- - Common Crawl was not used in favour of targeted Taiwanese clinical sources.
36
-
37
- ## Third-Party Copyright and Release Posture
38
-
39
- Open access is not the same as an open licence, and withholding raw pairs does not by itself establish that model training is authorised or is fair use. Likewise, applying Apache-2.0 to model code or weights does not grant rights in upstream material.
40
-
41
- The three explicitly copyright-restricted source groups above contributed 494 mined rows in total:
42
-
43
- | Source Groups | Rows | Share of Specialized Training Signals |
44
- |---|---:|---:|
45
- | `slang-blog` + `abbr-pdf` + `rsroc-weiei` | 494 | < 0.5% of total terminology signals |
46
-
47
- That low aggregate share, the extraction of short factual terminology rather than article text, the non-generative nature of an embedding model, and the lack of a reading substitute for the source works are relevant facts in a source-specific copyright assessment.
48
-
49
- Accordingly:
50
- - Raw third-party pairs and source documents are not distributed in this repository;
51
- - Each source still requires its own licence, permission, or documented legal assessment before reuse in a new training run;
52
- - Redistributors must independently comply with the MOE, CC BY-SA, LOINC, and SNOMED terms that apply to their inputs; and
53
- - This provenance record is factual disclosure, not legal advice or a warranty of non-infringement.
54
-
55
- ## Model Identity & Distribution
56
-
57
- - **Model Name:** IlhaEmbed (v2.0)
58
- - **Base Architecture:** IBM Granite ModernBERT, Apache-2.0
59
- - **Method:** Contrastive learning & relational knowledge distillation over Taiwanese clinical terminology, followed by vocabulary pruning and INT8 quantization
60
- - **Published Artifact:** [`weemed/IlhaEmbed`](https://huggingface.co/weemed/IlhaEmbed) (384 dimensions, 38.5 MB INT8 ONNX)
61
- - **Evaluation & Usage:** See [`MODEL-CARD.md`](MODEL-CARD.md) and [`README.md`](README.md)
62
-
63
-
64
- ## 2026-09-06 candidate update
65
-
66
- The update uses MOEX bilingual appositions corroborated by exact NAER rows, plus unlabelled score-preservation probes from official NAER training terms and synthetic work phrases. No new mixed private clinical replay corpus is included. Historical base-model disclosures above remain applicable.
67
-
68
- The official source manifest was downloaded 2026-07-29. Dataset versions and licence validity windows were not supplied (null in that manifest). Attribution entries:
69
-
70
- - 國家教育研究院-醫學學術名詞; provider: 國家教育研究院; published: 2026-06-24 22:11:12; licence: 政府資料開放授權條款; source: https://taic.moda.gov.tw/datasets/4ca6b322-b160-41dd-a16a-02249f540bf8
71
- - 國家教育研究院-獸醫學學術名詞; provider: 國家教育研究院; published: 2026-06-24 22:11:24; licence: 政府資料開放授權條款; source: https://taic.moda.gov.tw/datasets/b57e36a4-644e-40e3-9d7c-551a92ba961f
72
- - 國家教育研究院-心理學學術名詞; provider: 國家教育研究院; published: 2026-06-24 21:48:30; licence: 政府資料開放授權條款; source: https://taic.moda.gov.tw/datasets/727efda3-2810-4a92-8dc7-f7edecfd834f
73
- - 國家教育研究院-生命科學學術名詞; provider: 國家教育研究院; published: 2026-06-24 21:49:32; licence: 政府資料開放授權條款; source: https://taic.moda.gov.tw/datasets/58ff0cc0-1471-4694-8297-589a0db6a3b2
74
- - 國家教育研究院-病理學學術名詞; provider: 國家教育研究院; published: 2026-06-24 22:03:09; licence: 政府資料開放授權條款; source: https://taic.moda.gov.tw/datasets/1cf11198-bd37-4952-90ec-fd9d88500dc7
75
- - 國家教育研究院-實驗動物及比較醫學學術名詞; provider: 國家教育研究院; published: 2026-06-24 22:08:56; licence: 政府資料開放授權條款; source: https://taic.moda.gov.tw/datasets/00b36304-a7ce-4ee5-9c2c-36393772d028
76
- - 國家教育研究院-人體解剖學學術名詞; provider: 國家教育研究院; published: 2026-06-24 21:43:54; licence: 政府資料開放授權條款; source: https://taic.moda.gov.tw/datasets/f6e80374-6d80-4c8c-87da-e7626b0d8e7a
77
- - 國家教育研究院-比較解剖學學術名詞; provider: 國家教育研究院; published: 2026-06-24 21:48:42; licence: 政府資料開放授權條款; source: https://taic.moda.gov.tw/datasets/9411220f-26d4-4ebc-a88e-f1d7fedd8833
78
- - 國家教育研究院-藥學學術名詞; provider: 國家教育研究院; published: 2026-06-24 22:11:36; licence: 政府資料開放授權條款; source: https://taic.moda.gov.tw/datasets/169d49ea-f4e8-4586-88e0-d4200f84e4c9
79
- - 國家教育研究院-生命科學名詞-兩岸中小學教科書名詞; provider: 國家教育研究院; published: 2026-06-24 21:49:44; licence: 政府資料開放授權條款; source: https://taic.moda.gov.tw/datasets/fe34fb09-b6b6-499d-8fe5-d07430b97aed
80
- - 國家教育研究院-精神病理學學術名詞; provider: 國家教育研究院; published: 2026-06-24 22:09:20; licence: 政府資料開放授權條款; source: https://taic.moda.gov.tw/datasets/5fc9f128-161e-46ec-be08-39025de582fe
81
- - 國家教育研究院-兩岸對照名詞-中醫學術名詞; provider: 國家教育研究院; published: 2026-06-24 21:54:11; licence: 政府資料開放授權條款; source: https://taic.moda.gov.tw/datasets/6b28746a-a2d7-451e-8790-0726fc6feeb8
82
- - 國家教育研究院-兩岸對照名詞-醫學學術名詞; provider: 國家教育研究院; published: 2026-06-24 21:58:51; licence: 政府資料開放授權條款; source: https://taic.moda.gov.tw/datasets/e8d2d1fc-f577-4439-9a2d-f9595756a3ac
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
evaluation_drugbank_bootstrap.json DELETED
@@ -1,40 +0,0 @@
1
- {
2
- "baseline_report": "/home/gloomcheng/Workspace/weemed-ai/terminology-embedder/research/artifacts/ilhaembed-integrated-all-sources-v1-20260913/evaluation/drugbank-baseline.json",
3
- "candidate_report": "/home/gloomcheng/Workspace/weemed-ai/terminology-embedder/research/artifacts/ilhaembed-autonomous-unbiased-source-conditioned-20260919/runs/trial-0004/model-evaluation/drugbank-candidate.json",
4
- "eval_manifest_sha256": "b668e78b382cb6cae1c3a8dbe3bce808ba53352addb98158d463971a35d268c8",
5
- "interval": "two-sided 95% percentile bootstrap on paired candidate-minus-baseline deltas",
6
- "metrics": {
7
- "reciprocal_rank": {
8
- "baseline": 0.5660135839943139,
9
- "candidate": 0.6638060096581913,
10
- "ci95_high": 0.10629207614876504,
11
- "ci95_low": 0.08939576396964259,
12
- "delta": 0.0977924256638775,
13
- "superiority_pass": true
14
- },
15
- "top1": {
16
- "baseline": 0.5209967697277342,
17
- "candidate": 0.6347485002307337,
18
- "ci95_high": 0.12459621596677434,
19
- "ci95_low": 0.10290724503922473,
20
- "delta": 0.11375173050299954,
21
- "superiority_pass": true
22
- },
23
- "top5": {
24
- "baseline": 0.614443931702815,
25
- "candidate": 0.6970466082141209,
26
- "ci95_high": 0.09206275957544993,
27
- "ci95_low": 0.0735982925703738,
28
- "delta": 0.08260267651130596,
29
- "superiority_pass": true
30
- }
31
- },
32
- "promotion_allowed": false,
33
- "protocol": "DrugBank ID concept-disjoint eval; self-match excluded; SAME_CONCEPT names only",
34
- "resamples": 10000,
35
- "schema": "ilhaembed-rxnorm-paired-bootstrap/v1",
36
- "seed": 20260913,
37
- "superiority_all_metrics": true,
38
- "surfaces": 4334,
39
- "unit": "surface query"
40
- }
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
evaluation_drugbank_candidate.json DELETED
The diff for this file is too large to render. See raw diff
 
evaluation_rxnorm_bootstrap.json DELETED
@@ -1,40 +0,0 @@
1
- {
2
- "baseline_report": "/home/gloomcheng/Workspace/weemed-ai/terminology-embedder/research/artifacts/ilhaembed-integrated-all-sources-v1-20260913/evaluation/rxnorm-baseline.json",
3
- "candidate_report": "/home/gloomcheng/Workspace/weemed-ai/terminology-embedder/research/artifacts/ilhaembed-autonomous-unbiased-source-conditioned-20260919/runs/trial-0004/model-evaluation/rxnorm-candidate.json",
4
- "eval_manifest_sha256": "ad13bcb506e5762429fea4ce874b65eda2bdf68a96e02bc2a778696b16cfdee7",
5
- "interval": "two-sided 95% percentile bootstrap on paired candidate-minus-baseline deltas",
6
- "metrics": {
7
- "reciprocal_rank": {
8
- "baseline": 0.6584821651631182,
9
- "candidate": 0.899601212810233,
10
- "ci95_high": 0.24843618342695703,
11
- "ci95_low": 0.23387872683413913,
12
- "delta": 0.24111904764711456,
13
- "superiority_pass": true
14
- },
15
- "top1": {
16
- "baseline": 0.5889461358313818,
17
- "candidate": 0.8598594847775176,
18
- "ci95_high": 0.2799063231850117,
19
- "ci95_low": 0.26201405152224827,
20
- "delta": 0.27091334894613583,
21
- "superiority_pass": true
22
- },
23
- "top5": {
24
- "baseline": 0.7418266978922716,
25
- "candidate": 0.949695550351288,
26
- "ci95_high": 0.2157377049180328,
27
- "ci95_low": 0.2,
28
- "delta": 0.2078688524590164,
29
- "superiority_pass": true
30
- }
31
- },
32
- "promotion_allowed": false,
33
- "protocol": "source-isolated RxNorm eval; RxCUI split is fixed; self-match excluded",
34
- "resamples": 10000,
35
- "schema": "ilhaembed-rxnorm-paired-bootstrap/v1",
36
- "seed": 20260913,
37
- "superiority_all_metrics": true,
38
- "surfaces": 10675,
39
- "unit": "surface query"
40
- }
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
evaluation_rxnorm_candidate.json DELETED
The diff for this file is too large to render. See raw diff
 
evaluation_semantic_candidate.json DELETED
@@ -1,64 +0,0 @@
1
- {
2
- "card": {
3
- "abbrev": {
4
- "cases": 399,
5
- "pool": 1295,
6
- "top1": 0.8070175438596491,
7
- "top1_correct": 322,
8
- "top5": 0.8571428571428571,
9
- "top5_correct": 342
10
- },
11
- "apposition": {
12
- "cases": 371,
13
- "pool": 1295,
14
- "top1": 0.8598382749326146,
15
- "top1_correct": 319,
16
- "top5": 0.9164420485175202,
17
- "top5_correct": 340
18
- },
19
- "methodology": "fixed but source-contaminated regression; not held-out evidence",
20
- "semantic_macro": 0.8405648428232277,
21
- "slang": {
22
- "cases": 62,
23
- "pool": 1295,
24
- "top1": 0.8548387096774194,
25
- "top1_correct": 53,
26
- "top5": 0.9354838709677419,
27
- "top5_correct": 58
28
- }
29
- },
30
- "device": "cpu",
31
- "label": "eval-trial-0004-semantic",
32
- "latency_ms_per_text_batch96": 4.2576849688581815,
33
- "model": "/home/gloomcheng/Workspace/weemed-ai/terminology-embedder/research/artifacts/ilhaembed-autonomous-unbiased-source-conditioned-20260919/runs/trial-0004/model",
34
- "parameters": 38565888,
35
- "production_category": {
36
- "category_top1": 0.7017543859649122,
37
- "mean_top1_margin": 0.1627029925584793,
38
- "negative_p95": 0.8431658267974853,
39
- "negatives": 130,
40
- "positive_p5": 0.476388031244278,
41
- "positives": 114,
42
- "separation": -0.36677779555320733
43
- },
44
- "shorthand": {
45
- "cases": 121,
46
- "excluded_ambiguous_rows": 2,
47
- "kind": "fixed_regression_not_clean_heldout",
48
- "pool": 92,
49
- "top1": 0.9256198347107438,
50
- "top1_correct": 112,
51
- "top5": 0.9752066115702479,
52
- "top5_correct": 118
53
- },
54
- "taigi_semantic": {
55
- "cases": 141,
56
- "methodology": "mostly source-contaminated regression; not held-out evidence",
57
- "pool": 66,
58
- "top1": 0.9361702127659575,
59
- "top1_correct": 132,
60
- "top5": 0.9645390070921985,
61
- "top5_correct": 136
62
- },
63
- "weight_bytes": 154271056
64
- }
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
evaluation_slo_candidate.json DELETED
@@ -1,31 +0,0 @@
1
- {
2
- "artifact_bytes": 291612044,
3
- "failures": {
4
- "count": 0,
5
- "kinds": [],
6
- "rate": 0.0
7
- },
8
- "interpretation": "Fixed synthetic probe runtime evidence; it is not accuracy or broad workload evidence.",
9
- "label": "eval-trial-0004-slo",
10
- "latency_ms_per_batch": {
11
- "max": 395.01672302139923,
12
- "min": 351.3379310024902,
13
- "p50": 381.2973879976198,
14
- "p95": 389.27634101128206
15
- },
16
- "model": "/home/gloomcheng/Workspace/weemed-ai/terminology-embedder/research/artifacts/ilhaembed-autonomous-unbiased-source-conditioned-20260919/runs/trial-0004/model-evaluation/model_int8.onnx",
17
- "model_sha256": "9da09372f38401938c6275e6641d6312ca52e3e08f4c15bbe696f4e89b61f295",
18
- "peak_rss_bytes": 451375104,
19
- "probe": {
20
- "batch_size": 96,
21
- "batches": 25,
22
- "max_length": 32,
23
- "probe_count": 10,
24
- "synthetic_fixed_probes": true
25
- },
26
- "provider": "CPUExecutionProvider",
27
- "schema": "ilhaembed-runtime-slo/v1",
28
- "status": "PASS",
29
- "throughput_texts_per_second": 254.31276778546282,
30
- "tokenizer_sha256": "dc0e20c81f2ba89f3f1d0a66b315c444aa761551fbed3e320410eeebf1e80483"
31
- }
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
evaluation_source_snapshot.zip DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:429ec47a44248de2f09eb068a898d82902028550e80d960c69aba249eca225e5
3
- size 23235
 
 
 
 
evaluation_v1_int8.json DELETED
@@ -1,69 +0,0 @@
1
- {
2
- "card": {
3
- "abbrev": {
4
- "cases": 399,
5
- "pool": 1295,
6
- "top1": 0.7593984962406015,
7
- "top1_correct": 303,
8
- "top5": 0.8395989974937343,
9
- "top5_correct": 335
10
- },
11
- "apposition": {
12
- "cases": 371,
13
- "pool": 1295,
14
- "top1": 0.8921832884097035,
15
- "top1_correct": 331,
16
- "top5": 0.9299191374663073,
17
- "top5_correct": 345
18
- },
19
- "methodology": "fixed but source-contaminated regression; not held-out evidence",
20
- "semantic_macro": 0.8139681217651554,
21
- "slang": {
22
- "cases": 62,
23
- "pool": 1295,
24
- "top1": 0.7903225806451613,
25
- "top1_correct": 49,
26
- "top5": 0.8387096774193549,
27
- "top5_correct": 52
28
- }
29
- },
30
- "label": "shipped-v1-current-int8",
31
- "latency_ms_per_text_batch96": 3.28879157314077,
32
- "model": "/home/gloomcheng/med-embed/ilha-embed-97m-pruned",
33
- "naer_isolated": {
34
- "concepts": 3245,
35
- "queries": 6949,
36
- "top1": 0.11310979997121888,
37
- "top5": 0.21988775363361635
38
- },
39
- "onnx": "/home/gloomcheng/med-embed/ilha-embed-97m-pruned/model_int8.onnx",
40
- "onnx_bytes": 38537426,
41
- "production_category": {
42
- "category_top1": 0.7128712871287128,
43
- "mean_top1_margin": 0.1815911829471588,
44
- "negative_p95": 0.6991750836372376,
45
- "negatives": 130,
46
- "positive_p5": 0.28964269161224365,
47
- "positives": 101,
48
- "separation": -0.4095323920249939
49
- },
50
- "shorthand": {
51
- "cases": 108,
52
- "excluded_ambiguous_rows": 2,
53
- "kind": "fixed_regression_not_clean_heldout",
54
- "pool": 91,
55
- "top1": 0.7870370370370371,
56
- "top1_correct": 85,
57
- "top5": 0.8796296296296297,
58
- "top5_correct": 95
59
- },
60
- "taigi_semantic": {
61
- "cases": 141,
62
- "methodology": "mostly source-contaminated regression; not held-out evidence",
63
- "pool": 66,
64
- "top1": 0.9290780141843972,
65
- "top1_correct": 131,
66
- "top5": 0.9716312056737588,
67
- "top5_correct": 137
68
- }
69
- }
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
evaluation_v2_1_int8.json DELETED
@@ -1,169 +0,0 @@
1
- {
2
- "release_version": "2.1.0",
3
- "classification_protocol": "labelled-category-v2",
4
- "owner_clarification": "Correct category assignment is the task; unresolved residue is not gold refusal.",
5
- "release_regression_passed": true,
6
- "native": {
7
- "source": {
8
- "embedder.py": "f8ef081d2c858cacb4e4d7147b313973bb2fa644bdb2eb4959a445b2944bdd67",
9
- "classify.py": "180f656db9691efe15bf872fea631b4d1140d23eac7ddb6f54eaacba418a9dae",
10
- "source_commit": "e15d22018d33c42ff288354940fea064823c8c97",
11
- "embedder_component.py": "cc4211e5f620b56c3004d73ccc50376cbf3c8018f6443a698e838fe48813f3b8",
12
- "classify_component.py": "eb8500a5d2585f8460237c68675a08f095a7a72a18f340d5b249a98d035299b7",
13
- "scope": "Exact selected computation definitions with standard imports and explicit real-model binding; no application bootstrap."
14
- },
15
- "protocol": "exact native class/functions: fixed padding 32, batch 32 retrieval, per-category prototypes, singleton classification, 3-decimal candidate scores",
16
- "scope": "component replay with real ONNX; not full application verification",
17
- "models": {
18
- "baseline": {
19
- "registers": {
20
- "slang": {
21
- "count": 62,
22
- "correct": 40,
23
- "top1": 0.6451612903225806
24
- },
25
- "abbrev": {
26
- "count": 399,
27
- "correct": 272,
28
- "top1": 0.681704260651629
29
- },
30
- "apposition": {
31
- "count": 371,
32
- "correct": 304,
33
- "top1": 0.8194070080862533
34
- }
35
- },
36
- "taigi": {
37
- "count": 141,
38
- "correct": 131,
39
- "top1": 0.9290780141843972
40
- },
41
- "padding": {
42
- "length": 32,
43
- "pad_to_multiple_of": null,
44
- "pad_id": 0,
45
- "pad_token": "[PAD]",
46
- "pad_type_id": 0,
47
- "direction": "right"
48
- },
49
- "classification": {
50
- "protocol": "labelled-category-v2",
51
- "score_floor": 0.7,
52
- "margin_floor": 0.1,
53
- "positive": {
54
- "count": 100,
55
- "top1_correct": 67,
56
- "accepted": 28,
57
- "gate_correct": 25,
58
- "wrong_class": 3,
59
- "unclassified": 72
60
- },
61
- "unlabelled": {
62
- "count": 53,
63
- "accepted": 0,
64
- "accuracy_measured": false
65
- }
66
- },
67
- "macro": 0.7154241863534877
68
- },
69
- "candidate": {
70
- "registers": {
71
- "slang": {
72
- "count": 62,
73
- "correct": 59,
74
- "top1": 0.9516129032258065
75
- },
76
- "abbrev": {
77
- "count": 399,
78
- "correct": 310,
79
- "top1": 0.7769423558897243
80
- },
81
- "apposition": {
82
- "count": 371,
83
- "correct": 324,
84
- "top1": 0.8733153638814016
85
- }
86
- },
87
- "taigi": {
88
- "count": 141,
89
- "correct": 133,
90
- "top1": 0.9432624113475178
91
- },
92
- "padding": {
93
- "length": 32,
94
- "pad_to_multiple_of": null,
95
- "pad_id": 0,
96
- "pad_token": "[PAD]",
97
- "pad_type_id": 0,
98
- "direction": "right"
99
- },
100
- "classification": {
101
- "protocol": "labelled-category-v2",
102
- "score_floor": 0.7,
103
- "margin_floor": 0.1,
104
- "positive": {
105
- "count": 100,
106
- "top1_correct": 68,
107
- "accepted": 31,
108
- "gate_correct": 31,
109
- "wrong_class": 0,
110
- "unclassified": 69
111
- },
112
- "unlabelled": {
113
- "count": 53,
114
- "accepted": 0,
115
- "accuracy_measured": false
116
- }
117
- },
118
- "macro": 0.8672902076656442
119
- }
120
- },
121
- "baseline_sha256": "d222b6cfb11d265294f8855bfc7d4bbd691a61a74a4e5d0cb9b31649ea02f321",
122
- "candidate_sha256": "ca8eb040e1b5b51e05f5ceaf125073370fdcdd475e8a99c9dac0b03e46a0e7eb",
123
- "failures": [],
124
- "native_consumer_admitted": true
125
- },
126
- "numerical_and_latency": {
127
- "fp32_sha256": "0e4204f2849be97faf47fe1a748559f0766773af6cc586facf6f32fd50f551cc",
128
- "int8_sha256": "ca8eb040e1b5b51e05f5ceaf125073370fdcdd475e8a99c9dac0b03e46a0e7eb",
129
- "baseline_int8_sha256": "d222b6cfb11d265294f8855bfc7d4bbd691a61a74a4e5d0cb9b31649ea02f321",
130
- "model_bytes": 38667189,
131
- "onnxruntime": "1.23.2",
132
- "providers": [
133
- "CPUExecutionProvider"
134
- ],
135
- "mean_cosine_fidelity": 0.9941092729568481,
136
- "minimum_cosine_fidelity": 0.9703658223152161,
137
- "batch_invariance_minimum_cosine": 0.9999998807907104,
138
- "batch_invariance_max_abs_delta": 0.0,
139
- "cpu_batch_latency_ratio": 1.2098393668047083,
140
- "cpu_single_latency_ratio": 1.2536873087375715
141
- },
142
- "naer": {
143
- "baseline": {
144
- "queries": 6949,
145
- "concepts": 3245,
146
- "correct": 855,
147
- "top1": 0.12303928622823428,
148
- "top5_correct": 1625,
149
- "top5": 0.233846596632609
150
- },
151
- "candidate": {
152
- "queries": 6949,
153
- "concepts": 3245,
154
- "correct": 1167,
155
- "top1": 0.16793783278169522,
156
- "top5_correct": 2143,
157
- "top5": 0.3083896963591884
158
- }
159
- },
160
- "historical_residue_protocol": {
161
- "failed": true,
162
- "superseded_by_owner_task_clarification": true,
163
- "not_a_clinical_error_rate": true
164
- },
165
- "independent_clinical_gold_claimed": false,
166
- "automatic_classification_qualified": false,
167
- "release_judgment": "retracted_for_automatic_classification",
168
- "reason": "Relative regression improvement was insufficient: 31/100 correct automatic classifications, 69/100 unclassified."
169
- }
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
evaluation_v2_int8.json DELETED
@@ -1,69 +0,0 @@
1
- {
2
- "card": {
3
- "abbrev": {
4
- "cases": 399,
5
- "pool": 1295,
6
- "top1": 0.7794486215538847,
7
- "top1_correct": 311,
8
- "top5": 0.8596491228070176,
9
- "top5_correct": 343
10
- },
11
- "apposition": {
12
- "cases": 371,
13
- "pool": 1295,
14
- "top1": 0.894878706199461,
15
- "top1_correct": 332,
16
- "top5": 0.9380053908355795,
17
- "top5_correct": 348
18
- },
19
- "methodology": "fixed but source-contaminated regression; not held-out evidence",
20
- "semantic_macro": 0.8215499694661691,
21
- "slang": {
22
- "cases": 62,
23
- "pool": 1295,
24
- "top1": 0.7903225806451613,
25
- "top1_correct": 49,
26
- "top5": 0.8387096774193549,
27
- "top5_correct": 52
28
- }
29
- },
30
- "label": "naer-retained-epoch1-soup-a025-v2-perchannel-int8",
31
- "latency_ms_per_text_batch96": 3.1114273751882138,
32
- "model": "intake-embedder/ilhaembed-naer-retained-epoch1-soup-a0.25-v2-onnx-perchannel",
33
- "naer_isolated": {
34
- "concepts": 3245,
35
- "queries": 6949,
36
- "top1": 0.1318175277018276,
37
- "top5": 0.24996402360051806
38
- },
39
- "onnx": "intake-embedder/ilhaembed-naer-retained-epoch1-soup-a0.25-v2-onnx-perchannel/model_int8.onnx",
40
- "onnx_bytes": 38865358,
41
- "production_category": {
42
- "category_top1": 0.7425742574257426,
43
- "mean_top1_margin": 0.18474966287612915,
44
- "negative_p95": 0.6888158977031708,
45
- "negatives": 130,
46
- "positive_p5": 0.2778717875480652,
47
- "positives": 101,
48
- "separation": -0.41094411015510557
49
- },
50
- "shorthand": {
51
- "cases": 108,
52
- "excluded_ambiguous_rows": 2,
53
- "kind": "fixed_regression_not_clean_heldout",
54
- "pool": 91,
55
- "top1": 0.7777777777777778,
56
- "top1_correct": 84,
57
- "top5": 0.9074074074074074,
58
- "top5_correct": 98
59
- },
60
- "taigi_semantic": {
61
- "cases": 141,
62
- "methodology": "mostly source-contaminated regression; not held-out evidence",
63
- "pool": 66,
64
- "top1": 0.9361702127659575,
65
- "top1_correct": 132,
66
- "top5": 0.9645390070921985,
67
- "top5_correct": 136
68
- }
69
- }
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
export-provenance.json DELETED
@@ -1,21 +0,0 @@
1
- {
2
- "base_sha256": "c8fd7c6f46b531a81b724ef3dddd107fdc1e88f631ae4385914581b043d83cda",
3
- "torch": "2.13.0+cu130",
4
- "onnxruntime": "1.23.2",
5
- "onnx": "1.22.0",
6
- "opset": 18,
7
- "quantization": "MatMul INT8 with per-token symmetric activations, per-channel weights and per-row vocabulary",
8
- "activation_quantizers": 48,
9
- "mean_int8_cosine_fidelity": 0.9959200819333395,
10
- "minimum_fp32_cosine_fidelity": 0.9999998807907104,
11
- "files": {
12
- "model.onnx": {
13
- "bytes": 153759006,
14
- "sha256": "c4175b1ab94855d0fea9d735ef9064cf7b632e4684840c58201dfb0e6f85f0ba"
15
- },
16
- "model_int8.onnx": {
17
- "bytes": 38667189,
18
- "sha256": "ba9809fad34464e4b47a1b1f73b025be5cf8090e8525efd2d013ef7e44164684"
19
- }
20
- }
21
- }
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
export_provenance.json DELETED
@@ -1,56 +0,0 @@
1
- {
2
- "fp32_bytes": 154550326,
3
- "fp32_sha256": "2c0d52cb9069f42f9469898837d73f4dd126f94d0379d90dfe1e1abee0ddd839",
4
- "int8_bytes": 133463094,
5
- "int8_sha256": "9da09372f38401938c6275e6641d6312ca52e3e08f4c15bbe696f4e89b61f295",
6
- "parity": {
7
- "fp32": {
8
- "mean_cosine_to_hf": 1.0,
9
- "minimum_cosine_to_hf": 0.9999998807907104
10
- },
11
- "int8": {
12
- "mean_cosine_to_hf": 0.9954825639724731,
13
- "minimum_cosine_to_hf": 0.9896921515464783
14
- }
15
- },
16
- "quantization": {
17
- "format": "dynamic_qint8",
18
- "minimum_cosine_gate": 0.98,
19
- "node_count": 24,
20
- "node_policy": "attention_projection_only",
21
- "nodes": [
22
- "/model/layers.0/attn/Wqkv/MatMul",
23
- "/model/layers.0/attn/Wo/MatMul",
24
- "/model/layers.1/attn/Wqkv/MatMul",
25
- "/model/layers.1/attn/Wo/MatMul",
26
- "/model/layers.2/attn/Wqkv/MatMul",
27
- "/model/layers.2/attn/Wo/MatMul",
28
- "/model/layers.3/attn/Wqkv/MatMul",
29
- "/model/layers.3/attn/Wo/MatMul",
30
- "/model/layers.4/attn/Wqkv/MatMul",
31
- "/model/layers.4/attn/Wo/MatMul",
32
- "/model/layers.5/attn/Wqkv/MatMul",
33
- "/model/layers.5/attn/Wo/MatMul",
34
- "/model/layers.6/attn/Wqkv/MatMul",
35
- "/model/layers.6/attn/Wo/MatMul",
36
- "/model/layers.7/attn/Wqkv/MatMul",
37
- "/model/layers.7/attn/Wo/MatMul",
38
- "/model/layers.8/attn/Wqkv/MatMul",
39
- "/model/layers.8/attn/Wo/MatMul",
40
- "/model/layers.9/attn/Wqkv/MatMul",
41
- "/model/layers.9/attn/Wo/MatMul",
42
- "/model/layers.10/attn/Wqkv/MatMul",
43
- "/model/layers.10/attn/Wo/MatMul",
44
- "/model/layers.11/attn/Wqkv/MatMul",
45
- "/model/layers.11/attn/Wo/MatMul"
46
- ],
47
- "per_channel": false
48
- },
49
- "source_config_sha256": "9cb200e53b8bf3f5f8287a6e19a5cd1505aa08d88244b7fd151343a7ebcb9056",
50
- "source_model": "/home/gloomcheng/Workspace/weemed-ai/terminology-embedder/research/artifacts/ilhaembed-autonomous-unbiased-source-conditioned-20260919/runs/trial-0004/model",
51
- "source_weights_sha256": "7f1f3d42fd003a800b52cbda192af19bea4184f27bd87ea6789542e0d4acdf54",
52
- "tokenizer_sha256": {
53
- "tokenizer.json": "dc0e20c81f2ba89f3f1d0a66b315c444aa761551fbed3e320410eeebf1e80483",
54
- "tokenizer_config.json": "0bd6f388bc813908f40cff3c5cdf0065a444b351d19b40a086f138a80a9a7747"
55
- }
56
- }
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
interpolation_provenance.json DELETED
@@ -1,17 +0,0 @@
1
- {
2
- "alpha": 0.25,
3
- "base_config_sha256": "b5a4dd13f74b2eb21b8018aea9eccf1aa1261698857e6454b516ebe636837562",
4
- "base_model": "/home/gloomcheng/med-embed/ilha-embed-97m-pruned",
5
- "base_weights_sha256": "261af96cef3d72ea9e1877640083fa8d9fe6403333fec4a8cd54ef5e09f92299",
6
- "formula": "base + alpha * (tuned - base)",
7
- "output_config_sha256": "acf070a47f90b6fee858cb6a80be1a5d94afbe33211586cbd09db26efd55232a",
8
- "output_weights_sha256": "044fd270d0fe2d9ef56762c4db9ce6f9be09cdd810ba2b49b40d31ec43d38f09",
9
- "tokenizer_sha256": {
10
- "special_tokens_map.json": "e83cab90dfc594d568809f9a6c6ce31e5c703ad4fcd04a3519a914017da326b9",
11
- "tokenizer.json": "8fb9f1fd4a29a8151e220419f33cb4aa52bee76d55a0c9eced066664913605a0",
12
- "tokenizer_config.json": "3e86463bcf10c064244bc0aa22d84a9d78d582a8f9ef9896b9f704cbb6cd112c"
13
- },
14
- "tuned_config_sha256": "acf070a47f90b6fee858cb6a80be1a5d94afbe33211586cbd09db26efd55232a",
15
- "tuned_model": "intake-embedder/ilhaembed-naer-retained-lr2e6-seed42/epoch-1",
16
- "tuned_weights_sha256": "4c6c596c999f888ccda6a0f4b504b4157fdd01f37a36ee0bf26e9d9918729fdc"
17
- }
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
model_compact_37m_int8.onnx DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:44cd7c10f351d4ac0ccc3e8424fe92bc9cb8e5f24f2e253eaf8fd72d530cd3b3
3
- size 39364887
 
 
 
 
pooling_head.pt DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:6e63ef6b4fc696fd3687c59c0160448dc9b155d49ce28d2195819a6ca3f0bb5f
3
- size 1774799
 
 
 
 
relation_head.pt DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:b4fe7c7fb314f7b5e0de6af9716f49681f4a854baf5e28a1dd6540244c832a15
3
- size 2503773
 
 
 
 
release_manifest.json DELETED
@@ -1,69 +0,0 @@
1
- {
2
- "version": "2.1.0",
3
- "files": {
4
- "config.json": {
5
- "sha256": "9cb200e53b8bf3f5f8287a6e19a5cd1505aa08d88244b7fd151343a7ebcb9056",
6
- "bytes": 1696
7
- },
8
- "evaluation_source_snapshot.zip": {
9
- "sha256": "429ec47a44248de2f09eb068a898d82902028550e80d960c69aba249eca225e5",
10
- "bytes": 23235
11
- },
12
- "evaluation_v2_1_int8.json": {
13
- "sha256": "4df23682606b987ac43783fa3fe22ba6fde0b8c1f171626cfc62d32149cd3599",
14
- "bytes": 5275
15
- },
16
- "export_provenance.json": {
17
- "sha256": "a02cf0e75dda312a646924ff80c5c3dfd212c13fde424bc6b7cd9ab89adf0519",
18
- "bytes": 704
19
- },
20
- "model.safetensors": {
21
- "sha256": "0e4204f2849be97faf47fe1a748559f0766773af6cc586facf6f32fd50f551cc",
22
- "bytes": 152499656
23
- },
24
- "model_int8.onnx": {
25
- "sha256": "ca8eb040e1b5b51e05f5ceaf125073370fdcdd475e8a99c9dac0b03e46a0e7eb",
26
- "bytes": 38667189
27
- },
28
- "README.md": {
29
- "sha256": "2aaa41f6f76a70814d4487f715203bc47ab453603cd9e9c0b5ad2d00a0bd944c",
30
- "bytes": 6658
31
- },
32
- "RELEASE_PROVENANCE.md": {
33
- "sha256": "55927f9002d48a1115772102ad0febe909f4af9e3178fb1b8c9a6cd1fe9411d7",
34
- "bytes": 1666
35
- },
36
- "release_qualification.json": {
37
- "sha256": "0473e8673e33c6539bbf5153399e88858f1681acdc747de7dd6f0e30aec0be60",
38
- "bytes": 1797
39
- },
40
- "SOURCES.md": {
41
- "sha256": "e33cc1b19db75303d26da699a474bc3d1cf4d772db654c2c21dc1582842465ac",
42
- "bytes": 10079
43
- },
44
- "source_snapshot.zip": {
45
- "sha256": "56f1052e37ebdf70d815dc0d24d6b0ab3acd854dce0fa615a3d6d4f97e792600",
46
- "bytes": 34512
47
- },
48
- "special_tokens_map.json": {
49
- "sha256": "e83cab90dfc594d568809f9a6c6ce31e5c703ad4fcd04a3519a914017da326b9",
50
- "bytes": 870
51
- },
52
- "tokenizer.json": {
53
- "sha256": "53660382e792d55b6facc7bb42adb08fde1c3a2baa74a0f5efd11539eaf81aac",
54
- "bytes": 1312401
55
- },
56
- "tokenizer_config.json": {
57
- "sha256": "dc38f542c42e23bf36829a422e6f4161c5a2a13c5c7a2ad4cb7c4f3542a0677c",
58
- "bytes": 635
59
- },
60
- "training_metrics.json": {
61
- "sha256": "166c7910d19b9be007cc5b0a183bf69b3197271d58427720745bfc528a284ea2",
62
- "bytes": 2910
63
- },
64
- "training_provenance.json": {
65
- "sha256": "06b78000c7d7b1703e54c8a929399e50d8d578b482c6e270aff965d2babc382d",
66
- "bytes": 921
67
- }
68
- }
69
- }
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
release_manifest_v4.json DELETED
@@ -1,82 +0,0 @@
1
- {
2
- "RELEASE_PROVENANCE.md": {
3
- "bytes": 3040,
4
- "sha256": "ed68a2d5059d6fa46bc0a1b1396ed5a750adda9083bec7c62e64ed2875861a20"
5
- },
6
- "config.json": {
7
- "bytes": 1696,
8
- "sha256": "9cb200e53b8bf3f5f8287a6e19a5cd1505aa08d88244b7fd151343a7ebcb9056"
9
- },
10
- "evaluation_drugbank_bootstrap.json": {
11
- "bytes": 1650,
12
- "sha256": "aeab00eebe9209a013af43eb3789e8f54823dfd98b16346c1d45c50e6c93124f"
13
- },
14
- "evaluation_drugbank_candidate.json": {
15
- "bytes": 694523,
16
- "sha256": "667174e7cce9e479b6615c6585a25a2ca54ccb7b02d056815539bf423579b1d0"
17
- },
18
- "evaluation_rxnorm_bootstrap.json": {
19
- "bytes": 1620,
20
- "sha256": "145235673944cbacc164caf1f70a481bb598ec4533d219ab73bc26da871705c6"
21
- },
22
- "evaluation_rxnorm_candidate.json": {
23
- "bytes": 1586990,
24
- "sha256": "4eb5f48d785a18a553d7b7c6543cfb93f3726cb6bfa65a03bee23cdb2c26a717"
25
- },
26
- "evaluation_semantic_candidate.json": {
27
- "bytes": 1824,
28
- "sha256": "1d69a54bb0dd98837a515c8b5eed46d376c194b20ea31d6e1df4db310e6be93a"
29
- },
30
- "evaluation_slo_candidate.json": {
31
- "bytes": 1097,
32
- "sha256": "3c001c3ed116d49d6b6d8ef652cd1c0b68c633c31f34a25096c227e23f7d4bb6"
33
- },
34
- "export_provenance.json": {
35
- "bytes": 2239,
36
- "sha256": "59b6e299fe1aa7dab80c48a0c5276f2e588d311ecff8302c21024bfb31b4e7b5"
37
- },
38
- "model.onnx": {
39
- "bytes": 154550326,
40
- "sha256": "2c0d52cb9069f42f9469898837d73f4dd126f94d0379d90dfe1e1abee0ddd839"
41
- },
42
- "model.safetensors": {
43
- "bytes": 154271056,
44
- "sha256": "7f1f3d42fd003a800b52cbda192af19bea4184f27bd87ea6789542e0d4acdf54"
45
- },
46
- "model_compact_37m_int8.onnx": {
47
- "bytes": 39364887,
48
- "sha256": "44cd7c10f351d4ac0ccc3e8424fe92bc9cb8e5f24f2e253eaf8fd72d530cd3b3"
49
- },
50
- "model_int8.onnx": {
51
- "bytes": 133463094,
52
- "sha256": "9da09372f38401938c6275e6641d6312ca52e3e08f4c15bbe696f4e89b61f295"
53
- },
54
- "pooling_head.pt": {
55
- "bytes": 1774799,
56
- "sha256": "6e63ef6b4fc696fd3687c59c0160448dc9b155d49ce28d2195819a6ca3f0bb5f"
57
- },
58
- "relation_head.pt": {
59
- "bytes": 2503773,
60
- "sha256": "b4fe7c7fb314f7b5e0de6af9716f49681f4a854baf5e28a1dd6540244c832a15"
61
- },
62
- "special_tokens_map.json": {
63
- "bytes": 870,
64
- "sha256": "e83cab90dfc594d568809f9a6c6ce31e5c703ad4fcd04a3519a914017da326b9"
65
- },
66
- "tokenizer.json": {
67
- "bytes": 1312412,
68
- "sha256": "dc0e20c81f2ba89f3f1d0a66b315c444aa761551fbed3e320410eeebf1e80483"
69
- },
70
- "tokenizer_config.json": {
71
- "bytes": 636,
72
- "sha256": "0bd6f388bc813908f40cff3c5cdf0065a444b351d19b40a086f138a80a9a7747"
73
- },
74
- "training_metrics.json": {
75
- "bytes": 1191,
76
- "sha256": "078ae2d6f665bda635a8bca8d1f1ab702fb2d84a7239b71d0ffde4afba9e5124"
77
- },
78
- "training_provenance.json": {
79
- "bytes": 6417,
80
- "sha256": "c811549b591e207155a85e17e18ac549621e1772011407840e8bdaed1bc1871b"
81
- }
82
- }
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
release_qualification.json DELETED
@@ -1,24 +0,0 @@
1
- {
2
- "release_regression_passed": true,
3
- "native_receipt_sha256": "a5ef038e3bc79edcb61b52e27ebc0ceb172ca2b37df357092038fc008162d43b",
4
- "unchanged_artifact_receipt_sha256": "96ff4bf4ea6a8237a669482c6075a69ccee6b1a2e8d96b437018757d6e272f63",
5
- "code_sha256": {
6
- "evaluation/classification_protocol.py": "901cacff2aaaf50146e81b4a383240f991175d18379a5c8a78cc6d5e247c42c3",
7
- "evaluation/classification_integrity.py": "1e188eb42bd8fe68d2f985b23ef472cd4d9d933bb1f83786e60aa27c780eaa99",
8
- "evaluation/native_consumer_check.py": "30b9e53e10f71d4ae4fd17befb22b7290b8f5714b9fb3dca83efe9aebf1472bd",
9
- "evaluation/native_score_diagnostic.py": "a3439238ea37b3aecf08d348031fdd4e842fa865db32f77e5238b6544810986b",
10
- "evaluation/retention_eval.py": "ce97f69e38559e572fbe175af6f20acfccfc3656140b90df06bb88227f47e424",
11
- "evaluation/adaptation_eval.py": "2e2b1b50b78f0b162f7936c43888d1d9601121608f200b2947d384f43cacbf4f",
12
- "evaluation/check_score_release.py": "e2181d3be68a3a8ca6e26d8715b82125157cdd73039795744b497c8f86bd6dfb",
13
- "evaluation/check_int8_release.py": "df87399230d67f69430b9f54f2af2385a5fc79fd04dd449a2bbf0d198ab60d1a",
14
- "evaluation/check_concept_release.py": "75d1fa496008aeee98c1ff0d508e1408a05f720d88e37cc5c93d972c216dc63b",
15
- "evaluation/check_adaptation_release.py": "1cedba3d2ad415ca85c6738f01dd233989a83fc436a29e930efba42d5eea6fd4",
16
- "evaluation/score_preservation.py": "35f8dfd127b40d4341b35f0eb0a30aa9a287d43f8a4cd62fde3a325ee0f2ce50",
17
- "evaluation/onnx_encoder.py": "ea6b5195c8b21294c1ed625e53c62d895205c5eea809952c5b6c92966c4de46b",
18
- "evaluation/concept_retrieval.py": "48cdf2aa586123dad43c1afb4a3bb0bf8041729cf5d69ddeb1890067adddf58e",
19
- "training/data_prep/grouped_nursing.py": "cd8030d98757703cd1c8df0530279bb3b563c9c5082ae18d31dcadc635e1176b"
20
- },
21
- "automatic_classification_qualified": false,
22
- "release_judgment": "retracted_for_automatic_classification",
23
- "reason": "Relative regression improvement was insufficient: 31/100 correct automatic classifications, 69/100 unclassified."
24
- }
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
source_snapshot.zip DELETED
@@ -1,3 +0,0 @@
1
- version https://git-lfs.github.com/spec/v1
2
- oid sha256:56f1052e37ebdf70d815dc0d24d6b0ab3acd854dce0fa615a3d6d4f97e792600
3
- size 34512
 
 
 
 
training_metrics.json DELETED
@@ -1,27 +0,0 @@
1
- {
2
- "artifact_verified": false,
3
- "base_pairs": 171340,
4
- "drugbank_pairs": 24534,
5
- "evaluation_completed": false,
6
- "freeze_backbone": false,
7
- "note": "All formally acquired sources plus regulated-exam candidate included; quality and promotion gates remain required.",
8
- "output": "/home/gloomcheng/Workspace/weemed-ai/terminology-embedder/research/artifacts/ilhaembed-autonomous-unbiased-source-conditioned-20260919/runs/trial-0004/model",
9
- "pooling_mode": "attention_pooling",
10
- "promotion_allowed": false,
11
- "provenance_sha256": "c811549b591e207155a85e17e18ac549621e1772011407840e8bdaed1bc1871b",
12
- "regulated_exam_pairs": 17,
13
- "relation_head_mode": "source_conditioned_interaction",
14
- "relation_loss_mode": "damped_power_law_cross_entropy",
15
- "retention_loss_mode": "pairwise_logit_kl",
16
- "retention_loss_weight": 0.25,
17
- "run_id": "trial-0004",
18
- "rxnorm_pairs": 74267,
19
- "schema": "ilhaembed-integrated-regulated-training-result/v1",
20
- "status": "COMPLETED",
21
- "steps": 3838,
22
- "training_completed": true,
23
- "typed_relation_batch_size": 16,
24
- "typed_relations": 760971,
25
- "typed_sampling_mode": "relation_balanced",
26
- "typed_split_id": "concept_group_split_seed_20260914_train"
27
- }
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
training_provenance.json DELETED
@@ -1,170 +0,0 @@
1
- {
2
- "base_model": "/mnt/d/Workspace/research/med-embed/ilha-embed-97m-pruned-soup-labft",
3
- "base_model_sha256": "1ce46425e538a18a401e9d29021957e6a2c3279e18805dca673e7565cb788a0e",
4
- "config": {
5
- "base_per_batch": 40,
6
- "batch_size": 64,
7
- "device": "cuda",
8
- "drugbank_per_batch": 8,
9
- "freeze_backbone": false,
10
- "grad_clip": 1.0,
11
- "learning_rate": 5e-06,
12
- "max_length": 32,
13
- "pooling_heads": 4,
14
- "pooling_mode": "attention_pooling",
15
- "regulated_exam_per_batch": 4,
16
- "relation_head_mode": "source_conditioned_interaction",
17
- "relation_loss_mode": "damped_power_law_cross_entropy",
18
- "relation_loss_weight": 0.25,
19
- "retention_loss_mode": "pairwise_logit_kl",
20
- "retention_loss_weight": 0.25,
21
- "rxnorm_per_batch": 10,
22
- "seed": 42,
23
- "steps": 3838,
24
- "temperature": 0.05,
25
- "typed_per_batch": 2,
26
- "typed_relation_batch_size": 16,
27
- "typed_sampling_mode": "relation_balanced",
28
- "typed_split_id": "concept_group_split_seed_20260914_train",
29
- "weight_decay": 0.01
30
- },
31
- "evaluation_status": "not_run_by_trainer",
32
- "inputs": {
33
- "base_pairs": {
34
- "path": "/home/gloomcheng/Workspace/weemed-ai/terminology-embedder/intake-embedder/data/synonyms.tsv",
35
- "rows": 171340,
36
- "sha256": "e4fb5c68823b3029f7894931485e668f5c1c047f18d0e185452f9741d6c52386"
37
- },
38
- "drugbank_pairs": {
39
- "path": "/home/gloomcheng/Workspace/weemed-ai/terminology-embedder/research/artifacts/ilhaembed-integrated-all-sources-v2-20260913/corpus/lexical_train_pairs.tsv",
40
- "rows": 24534,
41
- "sha256": "821168035c2ed48a2a175ab6f6fe4c685473f0ac1f5de354df8365cc4a2ba497",
42
- "train_concepts": 13120
43
- },
44
- "regulated_exam_pairs": {
45
- "path": "/home/gloomcheng/Workspace/weemed-ai/terminology-embedder/research/artifacts/ilhaembed-integrated-all-sources-v2-20260913/corpus/lexical_train_pairs.tsv",
46
- "rows": 17,
47
- "sha256": "821168035c2ed48a2a175ab6f6fe4c685473f0ac1f5de354df8365cc4a2ba497"
48
- },
49
- "rxnorm_pairs": {
50
- "path": "/home/gloomcheng/Workspace/weemed-ai/terminology-embedder/research/artifacts/ilhaembed-integrated-all-sources-v2-20260913/corpus/lexical_train_pairs.tsv",
51
- "rows": 74267,
52
- "sha256": "821168035c2ed48a2a175ab6f6fe4c685473f0ac1f5de354df8365cc4a2ba497",
53
- "train_concepts": 18138
54
- },
55
- "typed_relations": {
56
- "path": "/home/gloomcheng/Workspace/weemed-ai/terminology-embedder/research/artifacts/ilhaembed-typed-relation-split-20260914-v2/train.jsonl",
57
- "relation_types": [
58
- "DRUGBANK_HAS_CAS",
59
- "DRUGBANK_HAS_INCHI_KEY",
60
- "DRUGBANK_HAS_UNII",
61
- "FDA_PRODUCT_HAS_ACTIVE_INGREDIENT",
62
- "FDA_PRODUCT_HAS_ATC",
63
- "FDA_PRODUCT_INDICATED_FOR",
64
- "OPENFDA_SUBSTANCE_HAS_INTERACTION",
65
- "OPENFDA_SUBSTANCE_HAS_INTERACTION_EVIDENCE",
66
- "OPENFDA_SUBSTANCE_HAS_NAME",
67
- "OPENFDA_SUBSTANCE_HAS_SYNONYM",
68
- "PROGRAM_HAS_DECISION_AUTHORITY",
69
- "PROGRAM_REQUIRES_CONTEXT",
70
- "PROGRAM_REQUIRES_ITEM",
71
- "RXNREL_RB_contained_in",
72
- "RXNREL_RB_has_quantified_form",
73
- "RXNREL_RB_has_tradename",
74
- "RXNREL_RN_contains",
75
- "RXNREL_RN_quantified_form_of",
76
- "RXNREL_RN_tradename_of",
77
- "RXNSAT_AMBIGUITY_FLAG",
78
- "RXNSAT_ANADA",
79
- "RXNSAT_ANDA",
80
- "RXNSAT_BLA",
81
- "RXNSAT_COLOR",
82
- "RXNSAT_COLORTEXT",
83
- "RXNSAT_DCSA",
84
- "RXNSAT_DM_SPL_ID",
85
- "RXNSAT_EXEMPT_DEVICE",
86
- "RXNSAT_IMPRINT_CODE",
87
- "RXNSAT_LABELER",
88
- "RXNSAT_LABEL_TYPE",
89
- "RXNSAT_MARKETING_CATEGORY",
90
- "RXNSAT_MARKETING_EFFECTIVE_TIME_HIGH",
91
- "RXNSAT_MARKETING_EFFECTIVE_TIME_LOW",
92
- "RXNSAT_MARKETING_STATUS",
93
- "RXNSAT_NADA",
94
- "RXNSAT_NDA",
95
- "RXNSAT_NDA_AUTHORIZED_GENERIC",
96
- "RXNSAT_NDC",
97
- "RXNSAT_NHRIC",
98
- "RXNSAT_OTC_MONOGRAPH_DRUG",
99
- "RXNSAT_OTC_MONOGRAPH_FINAL",
100
- "RXNSAT_OTC_MONOGRAPH_NOT_FINAL",
101
- "RXNSAT_PREMARKET_APPLICATION",
102
- "RXNSAT_PREMARKET_NOTIFICATION",
103
- "RXNSAT_RXN_ACTIVATED",
104
- "RXNSAT_RXN_AI",
105
- "RXNSAT_RXN_AM",
106
- "RXNSAT_RXN_AVAILABLE_STRENGTH",
107
- "RXNSAT_RXN_BOSS_FROM",
108
- "RXNSAT_RXN_HUMAN_DRUG",
109
- "RXNSAT_RXN_OBSOLETED",
110
- "RXNSAT_RXN_QUALITATIVE_DISTINCTION",
111
- "RXNSAT_RXN_QUANTITY",
112
- "RXNSAT_RXN_VET_DRUG",
113
- "RXNSAT_RXTERM_FORM",
114
- "RXNSAT_SCORE",
115
- "RXNSAT_SHAPE",
116
- "RXNSAT_SHAPETEXT",
117
- "RXNSAT_SIZE",
118
- "RXNSAT_SPL_SET_ID",
119
- "RXNSAT_UNAPPROVED_DRUG_OTHER",
120
- "WORKFLOW_EVENT_HAS_ITEM",
121
- "WORKFLOW_EVENT_HAS_STATUS",
122
- "WORKFLOW_EVENT_NOT_SYNONYM_OF"
123
- ],
124
- "rows": 760971,
125
- "sha256": "2f21d69b027c9cacfc4edeacbbf57441133a9a8803d12641cd7ec76fa0e3ca45"
126
- }
127
- },
128
- "loss": {
129
- "formula": "lexical_loss + relation_loss_weight * relation_loss_mode(typed_relation_cross_entropy) + retention_loss_weight * retention_loss_mode",
130
- "lexical_mean": 0.6199783020074818,
131
- "relation_mean": 3.5418746513646986,
132
- "retention_mean": 0.39487592631201246
133
- },
134
- "output_tree_sha256": {
135
- "config.json": "9cb200e53b8bf3f5f8287a6e19a5cd1505aa08d88244b7fd151343a7ebcb9056",
136
- "model.safetensors": "7f1f3d42fd003a800b52cbda192af19bea4184f27bd87ea6789542e0d4acdf54",
137
- "pooling_head.pt": "6e63ef6b4fc696fd3687c59c0160448dc9b155d49ce28d2195819a6ca3f0bb5f",
138
- "relation_head.pt": "b4fe7c7fb314f7b5e0de6af9716f49681f4a854baf5e28a1dd6540244c832a15",
139
- "tokenizer.json": "dc0e20c81f2ba89f3f1d0a66b315c444aa761551fbed3e320410eeebf1e80483",
140
- "tokenizer_config.json": "0bd6f388bc813908f40cff3c5cdf0065a444b351d19b40a086f138a80a9a7747"
141
- },
142
- "project_id": "ilhaembed",
143
- "relation_policy": {
144
- "identifier_relations_as_synonyms": false,
145
- "regulated_workflow_status_as_synonym": false,
146
- "same_concept_pairs_in_lexical_loss": true,
147
- "typed_relations_in_auxiliary_relation_head": true
148
- },
149
- "run_id": "trial-0004",
150
- "sampling": {
151
- "source_row_stream_seeds": {
152
- "drugbank": 51,
153
- "openfda": 52,
154
- "regulated_exam": 53,
155
- "rxnorm": 54,
156
- "tfda": 55
157
- },
158
- "source_stream_seed": 50,
159
- "source_types": [
160
- "drugbank",
161
- "openfda",
162
- "regulated_exam",
163
- "rxnorm",
164
- "tfda"
165
- ],
166
- "typed_sampling_mode": "relation_balanced"
167
- },
168
- "schema": "ilhaembed-integrated-regulated-training-provenance/v1",
169
- "training_status": "completed"
170
- }