annetadd/medsiglip_app

MedSigLIP On-Device Skin Lesion Classifier (v2.0)

An on-device dermatology classification system based on Google MedSigLIP-448, optimized for real-time mobile inference via TensorFlow Lite in React Native.

🌟 Overview & Updates in v2.0

Version 2.0 introduces a major architectural correction and optimization suite:

1. Attention Pooling Head Parity Fix: Corrected pre-LayerNorm residual connection in the TFLite conversion, attaining 1.000000 mean cosine similarity with PyTorch google/medsiglip-448.
2. Linear Classification Head: Trained a 6x1152 linear projection head on frozen 1152-d ViT embeddings, initialized from clinical prompt ensembles.
3. Temperature Calibration & Bias Balancing: Reduced Expected Calibration Error (ECE) from 19.77% to 5.56% ($NLL = 0.8755$).
4. Enhanced Out-of-Distribution Skin Gate: Integrated dual skin/non-skin centroid projection to reject non-cutaneous photographs.
5. Over-The-Air (OTA) Delivery: All weights and biases are packaged in model/text_bundle.json (~35 KB), enabling instant deployment without vision model re-downloads.

---

📈 Benchmark Results

Evaluated on an untouched, patient-stratified held-out test split from PAD-UFES-20 ($n = 446$ images across 262 patients, 0% patient leakage; total dataset 2,227 images across 1,309 unique patients):

Metric	Score	95% Bootstrap CI	Baseline (v1.0)	Absolute Gain
Top-1 Overall Accuracy	67.49%	[63.68%, 71.75%]	52.46%	+15.03%
Top-2 Differential Accuracy	89.01%	[86.32%, 91.70%]	77.52%	+11.49%
Balanced Macro Accuracy	67.75%	[61.54%, 73.52%]	~55.0%	+12.75%
Cancer Sensitivity (TPR)	84.11%	[79.15%, 88.99%]	90.8%	Preserved high
Benign Specificity (TNR)	80.60%	[75.33%, 85.53%]	46.2%	+34.40%
Positive Predictive Value (PPV)	80.00%	[74.65%, 85.00%]	59.6%	+20.40%
Expected Calibration Error (ECE)	5.56%	—	19.77%	-14.21%

Per-Class Recall (Test Split)

• MEL (Melanoma - High Risk): 77.8% (7/9)
• BCC (Basal Cell Carcinoma - Moderate Risk): 66.1% (111/168)
• SCC (Squamous Cell Carcinoma - Moderate Risk): 43.2% (16/37)
• ACK (Actinic Keratosis - Pre-cancerous): 70.1% (96/137)
• SEK (Seborrheic Keratosis - Benign): 73.3% (33/45)
• NEV (Nevus / Mole - Benign): 76.0% (38/50)

6x6 Diagnostic Confusion Matrix

True \ Pred  |   MEL |   BCC |   SCC |   ACK |   SEK |   NEV | Total
MEL          |     7 |     0 |     0 |     0 |     2 |     0 |     9
BCC          |     2 |   111 |    27 |    24 |     3 |     1 |   168
SCC          |     2 |    15 |    16 |     3 |     1 |     0 |    37
ACK          |     1 |    16 |    18 |    96 |     5 |     1 |   137
SEK          |     4 |     1 |     0 |     2 |    33 |     5 |    45
NEV          |     3 |     2 |     0 |     0 |     7 |    38 |    50

---

⚙️ Scoring Formulation

For an on-device image embedding $v \in \mathbb{R}^{1152}$ normalized such that $|v|_2 = 1$:

$${\text{logit}}_i=\sum _{j=1}^{1152}{W}_{i,j}\cdot v_j+b_i\backslash text\{logit\}\_i\; =\; \backslash sum\_\{j=1\}^\{1152\}\; W\_\{i,j\}\; \backslash cdot\; v\_j\; +\; b\_i$$ $$p_i=\frac{\exp ({\text{logit}}_i)}{\sum _{k=1}^6\exp ({\text{logit}}_k)}p\_i\; =\; \backslash frac\{\backslash exp(\backslash text\{logit\}\_i)\}\{\backslash sum\_\{k=1\}^6\; \backslash exp(\backslash text\{logit\}\_k)\}$$

Where $W \in \mathbb{R}^{6 \times 1152}$ is stored under head_weights and $b \in \mathbb{R}^6$ under head_bias in text_bundle.json.

---

⚠️ Clinical & Dataset Limitations

1. Prevalence Skew: PAD-UFES-20 is collected in specialty dermatology settings where BCC and ACK are over-represented relative to general primary care.
2. Skin Phototype Representation: Over 85% of samples reflect Fitzpatrick types I, II, and III. Diagnostic reliability on Fitzpatrick types V and VI requires further dedicated validation.
3. Clinical Use Notice: Softmax scores represent temperature-calibrated class differential rankings and must NOT be used as standalone diagnostic confirmation without clinical examination or biopsy.