metadata
license: cc-by-nc-nd-4.0
tags:
- pathology
- histopathology
- virtual-staining
- multiplex-immunofluorescence
- image-to-image
pipeline_tag: image-to-image
extra_gated_heading: Request access to The QUEST models
extra_gated_prompt: >-
The QUEST models is released under CC BY-NC-ND 4.0 and may only be used for
non-commercial research with proper attribution. Any commercial use, sale or
other monetisation of it, or of models or datasets derived from it, is
prohibited and requires prior approval. The primary email of your Hugging Face
account must match your institutional email: requests from free providers
(@gmail, @outlook, @hotmail, @qq, @163, @foxmail and the like) are denied. By
downloading you attest that all information given below is correct and
up-to-date, and you agree not to distribute, publish, redistribute or
reproduce a copy in whole or in part. If another user in your organisation
wishes to use it, they must register as an individual user and agree to these
terms. Commercial entities should contact the corresponding author.
extra_gated_fields:
Full name (first and last): text
Current affiliation (no abbreviations): text
Type of affiliation:
type: select
options:
- Academia
- Industry
- Hospital / clinical
- Government / non-profit
- label: Other
value: other
Current and official institutional email (**this must match the primary email of your Hugging Face account; @gmail/@outlook/@qq/@163/@foxmail domains will be denied**): text
Position / role: text
Country: country
Please explain your intended research use: text
I agree to all terms outlined above: checkbox
I agree to use this for non-commercial research purposes only: checkbox
I agree not to distribute, publish or reproduce it; another user in my organisation must register individually: checkbox
I agree not to use the models, or anything trained on their outputs, commercially: checkbox
extra_gated_button_content: Request access
QUEST
QUEry-based virtual STaining: multiplex immunofluorescence predicted from H&E, with dynamic output panel. The model reads a frozen encoding of the H&E plus a semantic embedding of each marker name.
Quick start
import matplotlib.pyplot as plt
from quest import QuestGenerator
he = plt.imread("assets/example_he.png")[..., :3] # (224, 224, 3)
mif = QuestGenerator("quest-semantic")([he], ["PanCK", "CD68", "DAPI"])
Tutorials
We prepared tutorials from virtual staining to downstream applications.
| Tutorial | What for |
|---|---|
| 📦 00_data | what to download, where it goes, and the format of each cohort |
| 🔬 01_predict_mif | predict a panel from one H&E patch, and score it against the real MIF |
| 🧫 02_cell_typing | cell types from a virtual stained expression |
| 📐 03_distance_entropy | cell-cell distance and neighbourhood entropy from based on cell typing |
| 🗺️ 04_cn_annotation | cell states and cellular neighbourhoods discovered from H&E alone |
| 🔎 05_retrieval | search a stained archive with an H&E query |
| 📈 06_patient_aggregation | patches to a patient survival prediction, and which patch contribution analyses |
Set the two constants at the top of tutorials/_common.py, get the data with Tutorial 0, then explore them step-by-step.
Weights
| repo | |
|---|---|
| yandrewl/QUEST | three QUEST models, cell typer, the Eva marker table |
| yandrewl/Eva | Eva_model.ckpt, the MIF foundation encoder the retrieval benchmark embeds in |
| MahmoodLab/UNI2-h | the frozen H&E encoder |
| model | |
|---|---|
quest-semantic |
marker queries from a semantic embedding of the marker's name |
quest-id |
the same architecture with a fixed learned per-marker table instead |
quest-eva |
a masked autoencoder that inpaints the marker channels of a partly observed stack |
Check quest.zoo for detailed description of model architectures and configurations.
Data quick start
huggingface-cli login first.
import shutil
from huggingface_hub import snapshot_download, hf_hub_download
snapshot_download("yandrewl/QUEST", local_dir="checkpoints", # weights, 1.5 GB
allow_patterns=["*.ckpt", "*.npz"])
snapshot_download("yandrewl/QUEST-tutorial-data", repo_type="dataset", # cohorts, 1.7 GB
local_dir="data",
allow_patterns=["crc-metu/*", # 73 MB tutorial 6
"bog-86337/*", # 36 MB tutorial 4
"stanford-pc/*"]) # 1.6 GB tutorials 1, 5
shutil.copy(hf_hub_download("yandrewl/Eva", "Eva_model.ckpt"), # tutorial 5
"checkpoints/Eva_model.ckpt")
from questkit import cohort # tutorials 2, 3
cohort.fetch_pathocell(["reg016_B", "reg032_B"]) # ~350 MB / region
QUEST_DATA points elsewhere if you keep the cohorts outside the repo. Tutorial 0 has the file formats.
License
Code, weights and staged cohorts are released under
CC BY-NC-ND 4.0. See LICENSE.