VOICE-23M

Predict spatial single-cell gene expression from H&E images using a LoRA-adapted image encoder and a spatial decoder. This model predicts 6,029 genes and can also extract 1,536-dimensional cell embeddings.

Resources

• Code: GitHub
• Paper: arXiv:2608.08366

Installation

git clone https://github.com/Luosanj/VOICE_HE_ST_Pred.git
cd VOICE_HE_ST_Pred
pip install -r requirements.txt

For automatic segmentation, also install cellpose>=4.0 and opencv-python-headless. SVS, NDPI, and MRXS images require the OpenSlide system library and openslide-python.

Download the model

Download the model files from this repository. Obtain access to the image encoder separately through MahmoodLab/UNI2-h and authenticate with your own Hugging Face account.

Choose your own Hugging Face cache location before downloading. Use the same VOICE_HF_HOME setting when running prediction:

export VOICE_HF_HOME=/path/to/huggingface/cache
export HF_HOME="$VOICE_HF_HOME"
hf auth login

python - <<'PYTHON'
from huggingface_hub import snapshot_download

snapshot_download("zchx/VOICE-23M", local_dir="models/voice23m",
                  allow_patterns=["stage*.safetensors", "config.json", "genes.tsv", "LICENSE"])
snapshot_download("MahmoodLab/UNI2-h")
PYTHON

File	Contents
stage1.safetensors	Image adapters, image–RNA projection towers, and RNA feature normalization
stage2.safetensors	Expression-trained image adapters, spatial decoder, and expression head
config.json	Model architecture and image geometry
genes.tsv	Ordered output gene symbols and indices
bank/TISSUE/	Tissue-pooled reference cells in compressed HDF5 shards
beta/TISSUE.json	Existing gene-wise fusion weights for supported tissues

The UNI2-h base weights are downloaded separately under their own terms. Prediction loads the Stage-1 adapters and overlays the trained Stage-2 adapters.

Prediction

Use your own image, model, and output paths. For a whole-slide image, first segment the cells or provide existing cell boundaries:

python predict/segment.py --image /path/to/slide.svs --mpp 0.25 \
  --out /path/to/cells.npz
python predict/predict.py --image /path/to/slide.svs \
  --cells /path/to/cells.npz --mpp 0.25 \
  --release models/voice23m --out /path/to/prediction.h5ad \
  --save_embeddings

0.25 is an example resolution in microns per pixel; supply the actual resolution of your image. Cell crops cover 55 µm and are resized to 224×224 pixels. Existing cell boundaries must use the source image's pixel coordinates.

For a prepared slide containing cell crops and boundaries:

python predict/predict.py --prepared /path/to/prepared_slide \
  --release models/voice23m --out /path/to/prediction.h5ad \
  --save_embeddings

Prediction requires no measured expression for the target slide. CUDA is the default device. Use --workers to control image-loading workers and --embeddings_only to extract cell embeddings without expression predictions.

Reference retrieval and fusion

The bank contains all 23,232,915 training reference cells, pooled and shuffled within tissue types. Features and expression remain paired. Features are L2-normalized and expression was standardized within each reference before pooling; no additional z-score is applied at prediction. Original slide names, cell identifiers, coordinates and source paths are omitted.

Tissue	Cells	Bank size (GiB)	Genes with fusion weights
bone	344,181	1.05	477
brain	816,519	2.60	Not supplied
breast	7,656,449	24.61	241
cervix	836,989	3.18	Not supplied
colon	2,169,152	6.86	438
heart	26,364	0.08	Not supplied
kidney	154,068	0.48	377
liver	401,890	1.26	377
lung	6,188,966	23.67	364
lymph_node	708,981	3.35	Not supplied
ovary	813,009	3.90	339
pancreas	331,666	1.04	98
prostate	193,000	0.84	Not supplied
skin	377,692	1.38	456
tonsil	2,213,989	6.88	377

Download only the tissue you need, for example lung:

python - <<'PYTHON'
from huggingface_hub import snapshot_download

snapshot_download("zchx/VOICE-23M", local_dir="models/voice23m",
                  allow_patterns=["bank/lung/*", "beta/lung.json"])
PYTHON
python predict/predict.py --image /path/to/slide.svs \
  --cells /path/to/cells.npz --mpp 0.25 --release models/voice23m \
  --bank models/voice23m/bank/lung --gate models/voice23m/beta/lung.json \
  --out /path/to/prediction.h5ad --save_embeddings

Fusion weights are supplied for bone, breast, colon, kidney, liver, lung, ovary, pancreas, skin and tonsil. Each beta/TISSUE.json maps global gene IDs to fusion weights. Weights are fitted on same-tissue reference slides using leave-one-reference-out retrieval and averaged per gene. Use the bank and fusion weights for the same tissue. Genes without fitted fusion weights use the direct prediction.

Brain, cervix, heart, lymph_node and prostate each have one training reference slide, so same-tissue leave-one-reference-out fitting is unavailable and no fusion weights are supplied. These tissues can be used with --bank alone to save retrieval scores; X then contains direct predictions. Downloading the whole repository includes all tissues, so use allow_patterns for smaller downloads.

Use the released banks with independent query slides. Source-level exclusions are unavailable after pooling. To fit fusion weights or exclude particular reference slides, build a separate slide-resolved bank from your own references using the code repository.

Output

import anndata as ad

prediction = ad.read_h5ad("/path/to/prediction.h5ad")

Field	Contents
X	Direct expression predictions in log1p(mu) scale
var_names	Gene symbols in model-head order
obsm['spatial']	Cell centroids as x,y coordinates in source-image pixels
obsm['X_voice']	Cell embeddings, when --save_embeddings is supplied

With a bank, layers['R'] contains retrieved standardized reference expression. With both a bank and fusion weights, layers['A'] contains direct predictions, var['beta'] contains applied gene weights, and X contains beta*A + (1-beta)*R. These fused values are gene-wise scores.

See the code repository for input formats, training, and optional reference retrieval and fusion.

License

See LICENSE for code terms and BANK_DATA_TERMS.md for reference-bank terms and source credits. UNI2-h is subject to the separate terms of its authors.