Clinical SapBERT Tri-Linker: Unified SNOMED CT, RxNorm & LOINC Entity Linker

A clinical and biomedical SapBERT representation model and Stage-2 Supervised Cross-Encoder Reranker pre-trained, self-aligned, and calibrated across the three universal medical vocabularies:

SNOMED CT: Clinical findings, disorders, surgical procedures, and body structures (638,238 active concepts).
RxNorm: Medications, clinical drugs, branded formulations, active ingredients, and dosages (316,330 active concepts).
LOINC: Laboratory observations, diagnostic panels, and physiological measurements (287,811 active concepts).

Total Knowledge Base: Over 1,242,379 clean clinical concepts indexed in exact 768-dimensional metric space.

Default Release: v1.0 is the default version served on the main branch. When calling AutoModel.from_pretrained("gitmodelmujtaba/sapbert-snomed-loinc-rxnorm") without revision arguments, you automatically receive v1.0.

📌 Available Versions & Revisions

Version	Branch / Tag	Status	Description	Macro-IoU (Competition)
`v1.0`	`main` / `v1.0`	Default / Production	Base SapBERT + Stage-2 Cross-Encoder Reranker (Gold Challenge Data)	`0.4427`
`v2.0`	`v2.0`	Experimental	Base SapBERT + MIMIC-IV Contrastive Projection Adapter & Overrides	`0.5646`
`v2.1`	`v2.1`	Experimental	Self-Healing Metric Retraining (NVIDIA A40 GPU)	`0.5646`

To load a specific version:

from transformers import AutoTokenizer, AutoModel

# Default (v1.0):
tokenizer = AutoTokenizer.from_pretrained("gitmodelmujtaba/sapbert-snomed-loinc-rxnorm")
model = AutoModel.from_pretrained("gitmodelmujtaba/sapbert-snomed-loinc-rxnorm")

# Or explicitly specifying revision:
model_v1 = AutoModel.from_pretrained("gitmodelmujtaba/sapbert-snomed-loinc-rxnorm", revision="v1.0")
model_v2 = AutoModel.from_pretrained("gitmodelmujtaba/sapbert-snomed-loinc-rxnorm", revision="v2.0")

🚀 Quickstart: Python Inference (Embedding & Similarity)

import torch
import torch.nn.functional as F
from transformers import AutoTokenizer, AutoModel

# 1. Load model and tokenizer (v1.0 is default)
repo_id = "gitmodelmujtaba/sapbert-snomed-loinc-rxnorm"
tokenizer = AutoTokenizer.from_pretrained(repo_id)
model = AutoModel.from_pretrained(repo_id)
model.eval()

# 2. Clinical terms (EHR mentions vs Official SNOMED / RxNorm concepts)
mentions = [
    "heart attack", 
    "lap chole", 
    "elevated blood sugar", 
    "tylenol extra strength"
]
targets = [
    "Acute myocardial infarction (disorder)", 
    "Laparoscopic cholecystectomy (procedure)", 
    "Hyperglycemia (finding)", 
    "Acetaminophen 500 MG Oral Tablet"
]

def encode(texts):
    inputs = tokenizer(texts, padding=True, truncation=True, max_length=64, return_tensors="pt")
    with torch.no_grad():
        outputs = model(**inputs)
        # Use [CLS] representation (first token)
        embeddings = outputs.last_hidden_state[:, 0, :]
        embeddings = F.normalize(embeddings, p=2, dim=1)
    return embeddings

mention_emb = encode(mentions)
target_emb = encode(targets)

# Compute cosine similarity matrix
similarity_matrix = torch.mm(mention_emb, target_emb.t())

for i, mention in enumerate(mentions):
    best_idx = similarity_matrix[i].argmax().item()
    best_score = similarity_matrix[i, best_idx].item()
    print(f"Mention: '{mention}' -> Target: '{targets[best_idx]}' (Score: {best_score:.4f})")

⚡ Google Colab & Kaggle Ready-to-Run Starter Code

Copy and paste this directly into a Google Colab or Kaggle Notebook cell (CPU or GPU):

# ==============================================================================
# 🩺 Clinical SapBERT Tri-Linker Starter (Google Colab / Kaggle)
# ==============================================================================
# Step 1: Install dependencies
!pip install -q transformers torch huggingface_hub

import torch
import torch.nn.functional as F
from transformers import AutoTokenizer, AutoModel

# Step 2: Select Device (GPU if available, else CPU)
device = "cuda" if torch.cuda.is_available() else "cpu"
print(f"Running on device: {device}")

# Step 3: Load Model & Tokenizer (v1.0 is Default)
repo_id = "gitmodelmujtaba/sapbert-snomed-loinc-rxnorm"
print(f"Loading {repo_id} (v1.0 Default)...")
tokenizer = AutoTokenizer.from_pretrained(repo_id)
model = AutoModel.from_pretrained(repo_id).to(device)
model.eval()

# Step 4: Batch of Clinical EHR Query Phrases
queries = [
    "acute myocardial infarction",
    "cholecystectomy, laparoscopic",
    "biliary pancreatitis",
    "lisinopril 20 mg oral tablet",
    "serum potassium level"
]

# Standard Medical Ontology Concepts (SNOMED CT, RxNorm, LOINC)
candidates = [
    {"code": "22298006", "ontology": "SNOMED CT", "name": "Myocardial infarction (disorder)"},
    {"code": "45595009", "ontology": "SNOMED CT", "name": "Laparoscopic cholecystectomy (procedure)"},
    {"code": "197454002", "ontology": "SNOMED CT", "name": "Acute pancreatitis (disorder)"},
    {"code": "314076", "ontology": "RxNorm", "name": "Lisinopril 20 MG Oral Tablet"},
    {"code": "2823-3", "ontology": "LOINC", "name": "Potassium [Moles/volume] in Serum or Plasma"}
]

# Step 5: Embed Queries & Candidates
def embed_terms(text_list):
    inputs = tokenizer(text_list, padding=True, truncation=True, max_length=64, return_tensors="pt").to(device)
    with torch.no_grad():
        out = model(**inputs)
        # [CLS] representation normalized
        cls_rep = out.last_hidden_state[:, 0, :]
        return F.normalize(cls_rep, p=2, dim=1)

candidate_names = [c["name"] for c in candidates]
query_vectors = embed_terms(queries)
candidate_vectors = embed_terms(candidate_names)

# Step 6: Similarity Matrix (Cosine Distance)
sim_matrix = torch.mm(query_vectors, candidate_vectors.t())

# Step 7: Print Top Matched Concepts
print("\n" + "=" * 98)
print(f"{'QUERY MENTION':<30} | {'MATCHED ONTOLOGY CONCEPT':<44} | {'SCORE':<7} | {'CODE'}")
print("=" * 98)
for q_idx, q_text in enumerate(queries):
    top_c_idx = sim_matrix[q_idx].argmax().item()
    matched = candidates[top_c_idx]
    score = sim_matrix[q_idx, top_c_idx].item()
    print(f"{q_text:<30} | {matched['name']:<44} | {score:<7.4f} | [{matched['ontology']} {matched['code']}]")
print("=" * 98)

📊 Tri-Vocabulary Retrieval Benchmarks

Vocabulary	Concepts Evaluated	Recall@1	Recall@5	Recall@10	MRR
SNOMED CT	638,238	88.42%	94.18%	96.05%	0.9084
RxNorm	316,330	91.20%	96.45%	97.80%	0.9328
LOINC	287,811	86.75%	92.89%	94.90%	0.8931
Overall Micro Avg	1,242,379	88.72%	94.42%	96.19%	0.9108

Pairwise Cross-Encoder Val Accuracy: 96.83% (evaluated on 226,470 hard negative pairs).
ECE Calibration Error: 3.87% (Clinical-grade probabilistic calibration).

📄 License & Attribution

License: Apache 2.0
Author: Mujtaba Hussain (gitmodelmujtaba)
Live Space: Clinical SapBERT & GLiNER Tri-Linker

Downloads last month: 174

Safetensors

Model size

0.1B params

Tensor type

F32