BioMod / README.md
beiken1201's picture
Remove new-data reconstruction evaluation utility and README references
07bb78a verified
|
Raw History Blame
15.1 kB
metadata
license: other
tags:
  - single-cell
  - transcriptomics
  - foundation-model
  - circulating-tumor-cells
library_name: pytorch

BioMod

BioMod is a single-cell transcriptomics foundation model built around gene modules and a Performer-based encoder. This repository provides pretrained model assets, pretraining and fine-tuning code, downstream task tutorials, and interpretability tools.

BioMod is released by CTC-Atlas, the joint laboratory of Raymind Biotechnology Co., Ltd and the Shenzhen Research Institute of Big Data (SRIBD).

Installation · BioMod model weights · Tasks and examples · Usage notes · Repository layout

Installation

Download the project from Hugging Face. Run the following commands from the parent of your intended project directory:

python -m pip install --upgrade huggingface_hub
hf auth login
hf download ctcatlas/BioMod --local-dir ./BioMod
cd BioMod

The download includes source code, model dependencies, the pretrained checkpoint, and datasets. To omit datasets, add --exclude "data/**" to the download command. If you already have the project, start from its root directory.

Create and activate the supplied Conda environment:

conda env create -f BioMod/environment.yml
conda activate performer_module

The environment specification uses Python 3.9, PyTorch 2.6, CUDA 12.4, and PyTorch Geometric 2.6. See environment.yml and requirements.txt for the complete dependency lists. Match the GPU packages to your system when adapting the environment.

Pretraining uses CUDA/NCCL. The zero-shot runner selects CUDA when available and otherwise uses CPU. Native Windows execution has not been validated for the supplied environment.

BioMod model weights

This release includes the BioMod model weights and their matching configuration. Use these files together; the configuration records the model architecture and gene-module definitions.

Asset File
BioMod model weights mae_tost_70M_75mask_ec2_epoch0.pth
Matching model configuration mae_tost_70M_75mask_ec2.pkl
Model gene order selected_genes_27k.txt
Gene identifier mapping gene2ens.csv
Gene2Vec embeddings selected_gene2vec_27k.npy
Shared module definitions merged_modules.json

The model components are bundled under model_dependencies/performer_pytorch_cont/. Task-specific checkpoint arguments are listed under Usage notes.

Tasks and examples

The following tasks have implementations in this repository. The links identify available source code and examples; these workflows have not been validated end to end with the bundled checkpoint.

Task Implementation Notebook
Cell-type annotation Fine-tuning script Notebook
Zero-shot cell embeddings and batch-effect evaluation Embedding and evaluation script Notebook
Single-drug response regression (IC50) Training script Notebook
Drug-pair synergy classification Training script Notebook
Gene perturbation response prediction Training script Notebook

Pretraining and analysis scripts

The repository also includes the following supporting workflows.

Workflow Implementation Documentation
Pretraining and continued pretraining Pretraining script Source guide, launcher reference
Module interpretability analysis Module geometry, module importance, text alignment Overview, attention, module importance guide

Tutorials and launcher references retain some historical checkpoint paths, server paths, and arguments. Adapt them to the current assets above and check the selected script's --help before running. Some notebook plotting steps also refer to external scripts or reference results that are not bundled here.

Usage notes

Project paths and notebooks

The project root contains BioMod/, model_dependencies/, weight/, and data/. The inner BioMod/ directory contains the source code. Task commands may assume different working directories; adjust relative paths when following a guide.

Before running a notebook, update its configuration cells:

Setting Location
BASE or ASSET_ROOT Project root
PROJECT_ROOT Inner BioMod/ source directory
MODULE_MODEL or LEGACY_MODULE_MODEL model_dependencies/ under the project root
Weight and configuration paths The BioMod model weights and matching configuration listed above
MODULE_DATA_DIR BioMod/module_data_all/ under the project root
Data and output paths Your task inputs and a writable output directory

Checkpoint variable names differ by notebook: CKPT_PATH / CONFIG_PKL for cell annotation, drug response, and zero-shot integration; SCFM_CKPT / SCFM_HYPER for gene perturbation; and CHECKPOINT_PATH / HYPER_PARAMS_PATH for drug combination. Updating environment variables alone does not replace hard-coded notebook paths.

Checkpoint arguments

Task Weights argument Configuration argument
Drug response / drug combination --checkpoint_path --hyper_params_path
Zero-shot / gene perturbation --scfm_ckpt_path --scfm_hyper_params_path
Cell annotation / backbone interpretation --performer_checkpoint --config_pkl
Continued pretraining --model_path --model_config_path or paired-file discovery

Data and evaluation

Datasets are stored under data/. Input formats, metadata fields, preprocessing, and train/validation splits depend on the task; follow the linked implementation and tutorial.

For zero-shot embeddings, provide an h5ad expression matrix and specify the gene, batch, and cell-type metadata fields. The runner aligns genes to the model gene list and fills missing genes with zeros. It does not normalize or log-transform the input matrix, so prepare expression values according to the intended protocol. Resolve duplicate gene identifiers and inspect gene overlap before evaluating embeddings.

The drug-response and drug-combination workflows use validation metrics for checkpoint selection. Their validation results should not be presented as performance on an independent final test set.

For checks of the shared training utilities, see Training utility checks.

Repository layout

The tree below includes source files, model dependencies, and the pretrained checkpoint. The top-level data/ and .cache/ directories are omitted for readability. Folder descriptions explain where to find each component.

Expand the full repository structure
BioMod/                                                   # Project root: source code, model dependencies and weights
├── BioMod/                                               # Task code: training, evaluation, interpretability and tutorials
│   ├── cell_annotation/                                  # Cell-type annotation and model fine-tuning
│   │   ├── module_data/                                  # Hallmark and Reactome modules with loading utilities
│   │   │   ├── load_module.py
│   │   │   ├── module1_hallmarks.json
│   │   │   └── module2_reactome_2022_filtered.json
│   │   ├── cmd_finetune.md
│   │   └── performer_module_finetune.py
│   ├── drug_combination/                                 # Drug-combination synergy prediction
│   │   ├── README.md
│   │   └── train_drug_combination_performer_module.py
│   ├── drug_response/                                    # Single-drug response prediction and training
│   │   ├── cmd.txt
│   │   └── train_drug_performer_module.py
│   ├── gene_perturbation/                                # Perturbation response prediction: processing, training and inference
│   │   ├── data_utils_local.py
│   │   ├── gears_reduce_local.py
│   │   ├── inference_local.py
│   │   ├── model_module.py
│   │   ├── pertdata_local.py
│   │   ├── README.md
│   │   ├── scfm_utils_local.py
│   │   ├── train_module.py
│   │   └── utils.py
│   ├── interpretability/                                 # Module representations and biological interpretability
│   │   ├── attention/                                    # Module crosstalk and within-module gene attention
│   │   │   ├── __init__.py
│   │   │   ├── module_crosstalk.py
│   │   │   ├── module_gene_attention.py
│   │   │   ├── README.md
│   │   │   └── README_gene_attention.md
│   │   ├── module_importance/                            # Module importance, ablation studies and marker-gene validation
│   │   │   ├── __init__.py
│   │   │   ├── cross_class_similarity.py
│   │   │   ├── family_ablation.py
│   │   │   ├── family_importance_analysis.py
│   │   │   ├── marker_gene_verification.py
│   │   │   ├── masking_ablation.py
│   │   │   ├── module_importance.py
│   │   │   ├── module_importance_publication.py
│   │   │   ├── pub_utils.py
│   │   │   ├── random_baseline.py
│   │   │   ├── README.md
│   │   │   ├── RESULTS.md
│   │   │   ├── tcell_reactome_analysis.py
│   │   │   └── TODO.md
│   │   ├── module_text_alignment/                        # Semantic alignment of module representations and pathway text
│   │   │   ├── __init__.py
│   │   │   ├── DESIGN.md
│   │   │   ├── fetch_reactome_descriptions.py
│   │   │   ├── MANUSCRIPT_SECTION.md
│   │   │   ├── module_text_alignment.py
│   │   │   ├── module_text_alignment_extended.py
│   │   │   ├── module_text_alignment_publication.py
│   │   │   └── reactome_descriptions.json
│   │   ├── __init__.py
│   │   ├── module_geometry.py
│   │   ├── README.md
│   │   └── utils.py
│   ├── module_data_all/                                  # Module definitions, merging and uncovered-gene tools
│   │   ├── create_uncovered_modules.py
│   │   ├── merge_modules.py
│   │   └── merged_modules.json
│   ├── tests/                                            # CPU tests for shared training utilities
│   │   └── test_training_utils.py
│   ├── toturials/                                        # Jupyter tutorials for five downstream tasks
│   │   ├── cell_annotation.ipynb
│   │   ├── drug_combination.ipynb
│   │   ├── drug_response_prediction.ipynb
│   │   ├── gene_perturbation.ipynb
│   │   └── zero_shot_batch_integration.ipynb
│   ├── zero_shot_batch/                                  # Zero-shot batch integration, evaluation and visualization
│   │   ├── sequencing_depth/                             # Low-depth data generation and zero-shot evaluation
│   │   │   ├── cmd.txt
│   │   │   ├── make_low_depth_h5ad.py
│   │   │   └── run_zero_shot_low_depth.py
│   │   ├── performer_module_zero_shot.py
│   │   ├── README.md
│   │   └── visualize_results.py
│   ├── .env.example
│   ├── .gitattributes
│   ├── .gitignore
│   ├── command.txt
│   ├── environment.yml
│   ├── models_performer_module.py
│   ├── module_dataset.py
│   ├── performer_module.py
│   ├── README.md
│   ├── requirements.txt
│   └── training_utils.py
├── model_dependencies/                                   # Shared gene mappings, gene lists, embeddings and model components
│   ├── module_data_all/                                  # Shared module definitions and construction tools
│   │   ├── create_uncovered_modules.py
│   │   ├── merge_modules.py
│   │   └── merged_modules.json
│   ├── performer_pytorch_cont/                           # Performer attention and model components
│   │   ├── performer_pytorch_cont/                       # Nested package containing copies of the model component source files
│   │   │   ├── __init__.py
│   │   │   ├── ding_models.py
│   │   │   ├── performer_pytorch.py
│   │   │   ├── performer_pytorch_old.py
│   │   │   └── reversible.py
│   │   ├── __init__.py
│   │   ├── ding_models.py
│   │   ├── performer_pytorch.py
│   │   ├── performer_pytorch_old.py
│   │   └── reversible.py
│   ├── gene2ens.csv
│   ├── selected_gene2vec_27k.npy
│   └── selected_genes_27k.txt
├── weight/                                               # Pretrained weights and matching model configurations
│   └── 0810model/                                        # BioMod model weights and matching configuration
│       ├── mae_tost_70M_75mask_ec2.pkl
│       └── mae_tost_70M_75mask_ec2_epoch0.pth
├── .gitattributes
├── .gitignore
└── README.md

Partners