| --- |
| license: cc-by-nc-nd-4.0 |
| language: |
| - en |
| pipeline_tag: feature-extraction |
| library_name: timm |
| datasets: |
| - AI4Pathology/pathorchestra-image-features |
| --- |
| |
| # PathOrchestra V1.0.0: Foundation Model for Computational Pathology |
|
|
| <p align="left"> |
| <a href="https://www.nature.com/articles/s41746-025-02027-w">📄 Paper</a> • |
| <a href="#usage-load-as-a-vision-encoder">🤗 Model Weights</a> • |
| <a href="https://huggingface.co/datasets/AI4Pathology/pathorchestra-image-features">🧬 Pre-extracted Embeddings</a> • |
| <a href="#citation">📚 Citation</a> |
| </p> |
|
|
| --- |
|
|
|
|
| ## Access Policy |
|
|
| The pretrained weights of **PathOrchestra V1.0.0** are available for **academic research purposes only**. |
|
|
| To request access, please ensure that your Hugging Face account is associated with an **official institutional email address**. |
|
|
| > **License:** CC BY-NC-ND 4.0 |
| > Non-commercial use only. Modification and redistribution are not permitted. |
|
|
| --- |
|
|
|
|
| ## Model Overview |
|
|
| We present **PathOrchestra**, a versatile pathology foundation model pretrained on **287,424 whole-slide images (WSIs)** spanning **21 tissue types** collected from **three medical centers**. The model was comprehensively evaluated across **112 clinical tasks** using **61 private** and **51 public datasets**, covering a broad spectrum of applications including digital slide preprocessing, pan-cancer classification, lesion identification, multi-cancer subtype classification, biomarker assessment, gene expression prediction, and structured report generation. |
|
|
| Across **27,755 whole-slide images** and **9,415,729 region-of-interest images**, PathOrchestra achieved an accuracy exceeding **0.950** in **47 tasks**, including pan-cancer classification, lymphoma subtyping, and bladder cancer screening. Notably, it is the first pathology foundation model capable of generating structured reports for colorectal cancer and lymphoma. |
|
|
| Overall, PathOrchestra demonstrates the clinical potential of large-scale self-supervised pathology foundation models, achieving strong performance across diverse clinical scenarios and supporting the integration of AI into digital medicine workflows. |
|
|
| --- |
|
|
|
|
| ## Usage: Load as a Vision Encoder |
|
|
| Load the pretrained model directly from the Hugging Face Hub using `timm`: |
|
|
| ```python |
| import timm |
| from huggingface_hub import login |
| |
| # Authenticate with your Hugging Face access token |
| # https://huggingface.co/settings/tokens |
| login(token=your_hf_token) |
| |
| model = timm.create_model( |
| "hf-hub:AI4Pathology/PathOrchestra", |
| pretrained=True, |
| init_values=1e-5, |
| dynamic_img_size=True, |
| ) |
| |
| model.eval() |
| ``` |
|
|
| --- |
|
|
| ## Feature Extraction Example |
|
|
| ```python |
| import torch |
| from PIL import Image |
| from torchvision import transforms |
| from huggingface_hub import hf_hub_download |
| |
| # Define preprocessing transform |
| transform = transforms.Compose([ |
| transforms.Resize(224), |
| transforms.ToTensor(), |
| transforms.Normalize( |
| mean=(0.485, 0.456, 0.406), |
| std=(0.229, 0.224, 0.225) |
| ) |
| ]) |
| |
| # Download example image |
| image_path = hf_hub_download( |
| repo_id="AI4Pathology/PathOrchestra", |
| filename="example.png" |
| ) |
| |
| image = Image.open(image_path).convert("RGB") |
| image = transform(image).unsqueeze(0) |
| |
| with torch.inference_mode(): |
| features = model(image) |
| |
| print(features.shape) |
| ``` |
|
|
| --- |
|
|
| ## Contact |
|
|
| For access requests, collaboration inquiries, or academic use, please contact the corresponding authors listed in the official repository. |
|
|
| --- |
|
|
| ## Acknowledgements |
|
|
| We gratefully acknowledge the authors of DINOv2 and UNI for their foundational contributions to vision representation learning and computational pathology. |
|
|
| --- |
|
|
| ## Citation |
|
|
| If you use PathOrchestra in your research, please cite: |
|
|
| ```bibtex |
| @article{yan2025pathorchestra, |
| title={PathOrchestra: A comprehensive foundation model for computational pathology with over 100 diverse clinical-grade tasks}, |
| author={Fang Yan, Jianfeng Wu, Jiawen Li, Wei Wang, Yirong Chen, Linda Wei, Jiaxuan Lu, Wen Chen, Zizhao Gao, Jianan Li, Heng Li, Hong Yan, Jiabo Ma, Minda Chen, Yang Lu, Qing Chen, Yizhi Wang, Xitong Ling, Xuenian Wang, Zihan Wang, Qiang Huang, Shengyi Hua, Mianxin Liu, Lei Ma, Tian Shen, Xiaofan Zhang, Yonghong He, Hao Chen, Shaoting Zhang, Zhe Wang}, |
| journal={npj Digital Medicine}, |
| year={2025} |
| } |
| ``` |
