You need to agree to share your Hugging Face information to access this model

This repository is publicly accessible, but you have to accept the conditions to access its files and content.

Access to the CellPlex weights is granted on request. The CELLO models contain fine-tuned copies of UNI2-h, Virchow2 or H-optimus-0 and are subject to the licences and terms of use of these models. The weights may only be used for non-commercial academic research.

CellPlex

CellPlex is a benchmark for predicting single-cell spatial proteomics from H&E. This repository holds its trained models: each one predicts the protein levels of every cell in an H&E image from the image and the nucleus centroids, for the protein panel of the cohort it was trained on. Training code, data sources and processing scripts: github.com/zjgao02/CellPlex.

Models

Folder	Model
`cello/{encoder}/{cohort}/fold{k}/`	Pathology foundation model adapted with CELLO: its token map is sampled at each nucleus, refined by a distance-decay cross-attention over the tile and decoded into the panel; trained end to end, so the folder holds the fine-tuned encoder
`linear_probe/{encoder}/{cohort}/fold{k}/`	Ridge regression on the frozen encoder's token at each nucleus; the folder holds the regression only, and the encoder is loaded from its own repository
`eva_ft/{cohort}/fold{k}/`	Eva fine-tuned on the cohort with its own H&E-to-multiplex objective, every panel protein requested by its marker name; run with the code of the GitHub repository
`gigatime_ft/{cohort}/fold{k}/`	GigaTIME fine-tuned on the cohort with its own pixel-level objective, one output channel per panel protein; run with the code of the GitHub repository

Encoders: hoptimus0 (H-optimus-0), uni2 (UNI2-h), virchow2 (Virchow2). CELLO with H-optimus-0 is the most accurate model on every cohort.

Each cohort has one model per fold: fold0 to fold4 for the five-fold cohorts, and fold0, the fixed split, for Orion-CRC and glioma. The model of fold k was trained without the test units of fold k, so every sample of the benchmark is predicted by the model of the fold that holds it out. The splits are in data/splits/ of the GitHub repository. For new data, any fold can be used; fold 0 is the default in the examples below.

Cohorts

Cohort	Tissue	Platform	µm/px	Proteins	Folds	Test samples	PCC: CELLO H-optimus-0 / UNI2-h / Virchow2	Linear probe H-optimus-0	Eva fine-tuned	GigaTIME fine-tuned
`orion_crc`	Colorectal cancer	Orion	0.325	16	1	8	0.620 / 0.615 / 0.595	0.590	0.500	0.426
`crc_tma`	Colorectal cancer	CODEX	0.377	56	5	130	0.530 / 0.512 / 0.508	0.432	0.491	0.356
`multitumor`	Multiple tumor types	CODEX	0.377	57	5	66	0.443 / 0.426 / 0.431	0.339	0.419	0.297
`bc_crc`	Breast, colorectal cancer	CODEX	0.507	36	5	45	0.531 / 0.514 / 0.514	0.421	0.456	0.317
`hnscc_ln`	HNSCC lymph nodes	CODEX	0.377	57	5	388	0.479 / 0.466 / 0.455	0.389	0.441	0.333
`bone_marrow`	Bone marrow	CODEX	0.249	52	5	11	0.620 / 0.607 / 0.602	0.538	0.548	0.358
`glioma`	Glioma	CODEX	0.249	68	1	9	0.503 / 0.492 / 0.480	0.413	0.365	0.323

PCC: per test sample, the Pearson correlation between predicted and measured levels across the sample's cells, averaged over the proteins of the panel, then over the test samples of all folds.

Files

modeling_cellplex.py                      CELLO, LinearProbe, load_model
inference.py                              prediction for a whole image and a table of nucleus centroids
cello/{encoder}/{cohort}/fold{k}/
    model.safetensors                     weights, including the fine-tuned encoder
    config.json                           architecture, encoder, cohort, fold and resolution
    proteins.txt                          the panel's markers, in output order
linear_probe/{encoder}/{cohort}/fold{k}/
    probe.safetensors                     ridge weight (D, P) and bias (P,)
    config.json, proteins.txt
eva_ft/{cohort}/fold{k}/
    model.safetensors                     Eva state dict
    config.json, proteins.txt
gigatime_ft/{cohort}/fold{k}/
    model.safetensors                     GigaTIME state dict with one output channel per panel protein
    config.json, proteins.txt

Quick start

Access requires approval: request it on this page, then log in with huggingface-cli login.

pip install torch timm huggingface_hub safetensors numpy pandas pillow opencv-python-headless openslide-python openslide-bin

Predict every cell of a slide (only the chosen model is downloaded):

python inference.py --model cello/hoptimus0/orion_crc/fold0 --image slide.tif --mpp 0.325 --cells nuclei.csv --out pred.csv

nuclei.csv has one row per cell with its centroid x, y in level-0 pixels; the output adds one column per protein. From Python:

import sys
from huggingface_hub import snapshot_download

repo = snapshot_download("gaozijun/CellPlex", allow_patterns=["*.py", "cello/hoptimus0/orion_crc/fold0/*"])
sys.path.insert(0, repo)
from modeling_cellplex import CELLO
from inference import predict_cells

model = CELLO.from_pretrained(repo, subfolder="cello/hoptimus0/orion_crc/fold0", device="cuda")
pred = predict_cells(model, "slide.tif", cx, cy, image_mpp=0.325)   # (cells, proteins); columns: model.proteins

model.predict works on a batch of 224×224 tiles at the model's resolution and the cell positions inside each tile:

import torch
images = torch.rand(2, 3, 224, 224)                      # RGB in [0, 1]
cell_xy = [torch.tensor([[50.0, 60.0], [120.5, 33.0]]),  # (x, y) pixel positions per tile
           torch.tensor([[10.0, 200.0]])]
with torch.autocast("cuda", dtype=torch.float16):
    preds = model.predict(images.cuda(), cell_xy)        # list of (n_cells, n_proteins) tensors

A linear probe is loaded the same way with LinearProbe.from_pretrained(repo, subfolder="linear_probe/hoptimus0/orion_crc/fold0"); it downloads the original encoder from its own repository, whose terms have to be accepted there first.

Inputs and outputs

Image: RGB H&E. Each model works at the resolution of its cohort (µm/px in the table); inference.py resamples the image when --mpp is given.
Cells: nucleus centroids. CellPlex segments nuclei on the H&E with CellViT++; see the GitHub repository.
Output: per cell and protein of the cohort's panel, the predicted log1p of the mean intensity over the nucleus and a 2 µm rim, on the intensity scale of that cohort's multiplexed images. Levels are not comparable across cohorts.

Accuracy drops on tissues, stains and scanners unlike the training cohort. Use the model whose cohort is closest to your data and, where possible, check it against paired measurements.

Licence

The weights are released for non-commercial academic research under CC BY-NC-ND 4.0. The CELLO models contain fine-tuned copies of their encoders and are also subject to the encoder's licence and terms of use: UNI2-h (CC BY-NC-ND 4.0), Virchow2 (CC BY-NC-ND 4.0) and H-optimus-0 (Apache 2.0). The linear probes use the original encoders, which are obtained from their own repositories under their terms. The gigatime_ft models are fine-tuned from GigaTIME (Apache 2.0, gigatime_ft/LICENSE): every layer was trained further on the cohort, and the output layer was replaced by one channel per panel protein. As for GigaTIME, they are intended for research only and not for clinical or other deployed use. The eva_ft models are fine-tuned from Eva (Eva_ft.ckpt; every parameter trained further on the cohort) and remain subject to the terms under which Eva is distributed.

Downloads last month: -; Downloads are not tracked for this model. How to track

Inference Providers NEW

This model isn't deployed by any Inference Provider. 🙋 Ask for provider support