You need to agree to share your contact information to access this model
This repository is publicly accessible, but you have to accept the conditions to access its files and content.
Access to the CELLO weights is granted on request. The model contains a fine-tuned copy of Virchow2 and may only be used for non-commercial academic research, under the Virchow2 licence (CC BY-NC-ND 4.0) and terms of use.
Log in or Sign Up to review the conditions and access this model content.
CELLO
CELLO predicts the gene expression of every cell in an H&E image from the image and the cell
locations. It runs one Virchow2 forward pass per 224×224 tile, queries each cell's feature at its
position on the token map by grid sampling, and refines it with a distance-decay cross-attention
over the tile. The output is the expression of 1,915 genes per cell, as log1p of counts
normalised to 100 per cell.
This checkpoint was trained on the 10X-Xenium-52 training split (gaozijun/cello_data). Training code: github.com/zjgao02/CELLO.
Files
model.safetensors weights, including the fine-tuned Virchow2 encoder
config.json architecture
genes.txt the 1,915 output genes, in output order
modeling_cello.py model definition and CELLO.from_pretrained
inference.py prediction for one sample of gaozijun/cello_data, and gene-wise PCC
tutorial.py end-to-end example
Quick start
Access requires approval: request it on this page, then log in with huggingface-cli login.
pip install torch timm huggingface_hub safetensors pillow numpy
python tutorial.py --sample NCBI879
tutorial.py downloads this model and one test sample, predicts every cell, and prints the
correlation between predicted and measured expression. The core of it:
import sys, tarfile
from huggingface_hub import hf_hub_download, snapshot_download
model_dir = snapshot_download("gaozijun/CELLO")
sys.path.insert(0, model_dir)
from modeling_cello import CELLO
from inference import predict_sample, gene_pcc
model = CELLO.from_pretrained(model_dir, device="cuda")
tar = hf_hub_download("gaozijun/cello_data", "tiles/NCBI879.tar", repo_type="dataset")
tarfile.open(tar).extractall("cello_demo")
pred, true, genes = predict_sample(model, "cello_demo/NCBI879") # (cells, genes) arrays
pcc = gene_pcc(pred, true, genes) # {gene: PCC}
Your own images
model.predict takes a batch of 224×224 tiles and the cell positions inside each tile:
import torch
images = torch.rand(2, 3, 224, 224) # RGB in [0, 1]
cell_xy = [torch.tensor([[50.0, 60.0], [120.5, 33.0]]), # (x, y) pixel positions per tile
torch.tensor([[10.0, 200.0]])]
with torch.autocast("cuda"):
preds = model.predict(images.cuda(), cell_xy) # list of (n_cells, 1915) tensors
To predict a whole slide, cut it into non-overlapping 224×224 tiles and pass the cell centroids of
each tile. scripts/predict_wsi.py in the GitHub repository does this for a WSI and a centroid
table.
Licence
The model contains a fine-tuned copy of Virchow2 and is subject to the Virchow2 licence (CC BY-NC-ND 4.0) and terms of use: non-commercial academic research only.
- Downloads last month
- 2