MetPredict Tumor Segmentation (DPT)
Dense semantic segmentation for lung H&E pathology (tumor).
- Encoder (frozen): H-optimus-0 ViT backbone (pretrained on histopathology data).
- Decoder (trained): custom DPT head with multi-scale feature fusion.
Classes (2): 0 = background, 1 = tumor
Input tile: 224x224 @ 0.5 MPP, ImageNet-normalized RGB.
Preprocessing
from torchvision.transforms import ToTensor, Normalize, Resize, Compose
transform = Compose([
ToTensor(),
Resize((224, 224)),
Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225]),
])
# pixel_values = transform(pil_rgb_image).unsqueeze(0) # (1, 3, 224, 224)
Usage
Option A — Transformers (safetensors). Needs transformers with
trust_remote_code=True, and access to the gated bioptimus/H-optimus-0
backbone (re-instantiated at load).
import torch
from transformers import AutoModel
model = AutoModel.from_pretrained("RendeiroLab/metpredict-tumor-seg", trust_remote_code=True).eval()
with torch.inference_mode():
out = model(pixel_values)
logits = out.logits # (B, 2, H, W)
pred = logits.argmax(dim=1) # (B, H, W)
Option B — torch.export (model.pt2): torch-only, self-contained. No
transformers, no custom code, no gated-backbone download — the weights are
baked into the exported program.
import torch
from huggingface_hub import hf_hub_download
path = hf_hub_download("RendeiroLab/metpredict-tumor-seg", "model.pt2")
model = torch.export.load(path).module()
with torch.inference_mode():
logits = model(pixel_values) # (B, 2, H, W)
pred = logits.argmax(dim=1)
Validation metrics
Held-out validation split of the 16-PDX reported cohort, all figures from the single exported epoch (epoch 43).
| Class | Precision | Recall | F1 | IoU |
|---|---|---|---|---|
| background | 0.923 | 0.915 | 0.919 | 0.850 |
| tumor | 0.808 | 0.824 | 0.816 | 0.689 |
- Mean foreground IoU: 0.689 (primary metric)
- Mean IoU incl. background: 0.770
- Mean foreground Dice: 0.614
- Scope: trained on all annotated PDX lines; metrics reported on the 16-PDX reported cohort only.
- Per-PDX foreground IoU (n=8 lines with adequate validation data): min 0.414 / median 0.608 / max 0.761
- PDX-macro foreground IoU, n=8: 0.603 (lines weighted equally, not by tile count)
- Excluded from the per-PDX figures above (8 of 16 reported lines): H3204, H4013, H4272, HCI005, J53353, J55454, J67762, J74968 — each lands fewer than 50 validation tiles. The validation split is per-slide, so a line whose held-out slides carry sparse annotation yields too few tiles for a stable per-line IoU. Where a line's validation tiles are mostly annotated-background regions, foreground IoU there measures false-positive suppression rather than segmentation accuracy, and is not comparable to the other lines.
- Downloads last month
- 24