How to use from the
Use from the
Transformers library
# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-feature-extraction", model="AI4PATH/XMAG", trust_remote_code=True)
# pip install -U transformers accelerate
# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("AI4PATH/XMAG", trust_remote_code=True, device_map="auto")
Quick Links

XMag

XMag is a pathology image encoder distilled from a high-magnification foundation model to a low-magnification student model.

  • Teacher: frozen UNIv2
  • Student: DINOv2 ViT-B/14
  • Input: RGB pathology patch, 224 x 224@5x or 2um mpp
  • Output: CLS embedding and patch embeddings
  • Training objective: global and local cosine feature distillation from high-magnification teacher features

Usage

import torch
from PIL import Image
from torchvision import transforms
from transformers import AutoModel

eval_transform = transforms.Compose([
    transforms.Resize((224, 224)),
    transforms.ToTensor(),
    transforms.Normalize(mean=(0.485, 0.456, 0.406), std=(0.229, 0.224, 0.225)),
])

model = AutoModel.from_pretrained("AI4PATH/XMAG", trust_remote_code=True)
model.eval()

image = Image.open("patch.png").convert("RGB")
pixel_values = eval_transform(image).unsqueeze(0)

with torch.no_grad():
    outputs = model(pixel_values)

cls_embedding = outputs["cls_embedding"]          # (1, 768)
patch_embeddings = outputs["patch_embeddings"]    # (1, 256, 768)

pixel_values should be normalized RGB tensors with shape (B, 3, 224, 224). The model does not apply preprocessing internally. Resize, ToTensor(), and ImageNet mean/std normalization should be done before calling the model.

Model Details

The student sees a low-magnification 224 x 224 view of the tissue region. During training, the frozen UNIv2 teacher sees the corresponding high-magnification 896 x 896 region split into 4 x 4 subpatches. The student is trained to match both:

  • A global teacher representation, computed by averaging the 16 teacher local features.
  • Local teacher representations, aligned to pooled student patch-token blocks.

This release contains the student EMA backbone weights only. Projection heads used during distillation are not included.

Requirements

pip install torch torchvision transformers huggingface_hub

The model code uses torch.hub to instantiate the DINOv2 ViT-B/14 backbone. The first load may need internet access to fetch the DINOv2 hub code, unless it is already cached.

Citation

If you use this model, please cite:

Su, Z., Akbar, A. R., & Niazi, M. K. K. (2027). Streamline pathology foundation model by cross-magnification distillation. Medical Image Analysis. Available at https://doi.org/10.1016/j.media.2026.104352.

@article{SU2027104352,
title = {Streamline pathology foundation model by cross-magnification distillation},
journal = {Medical Image Analysis},
volume = {115},
pages = {104352},
year = {2027},
issn = {1361-8415},
doi = {https://doi.org/10.1016/j.media.2026.104352},
url = {https://www.sciencedirect.com/science/article/pii/S1361841526004214},
author = {Ziyu Su and Abdul Rehman Akbar and Usama Sajjad and Anil V. Parwani and Muhammad Khalid Khan Niazi}}

License

This code is licensed under the PolyForm Noncommercial License 1.0.0. You may use it for research, academic, or other noncommercial purposes.

Downloads last month
783
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support