kaichenxu/scCAFM

Building a causality-aware single-cell RNA-seq foundation model via context-specific causal regulation modeling

scCAFM is a foundation model for large-scale single-cell RNA-seq analysis that jointly learns context-specific gene-regulatory structure and transferable gene and cell representations. Its two-stage design combines a Structure Foundation Module (SFM) for causal regulatory modeling with an Embedding Foundation Module (EFM) guided by the structure learned by SFM.

Model repository: kaichenxu/scCAFM
Source code: Catchxu/scCAFM

Overview

Many single-cell foundation models primarily capture associative gene relationships or summarize regulation at the dataset or cell-type level. scCAFM is designed to model regulatory context at cellular resolution while learning representations that can transfer to downstream biological analyses.

Highlights

• Context-specific regulatory modeling: represents gene-regulatory structure in a compact latent factor space.
• Mixture-of-Experts structure learning: allows distinct experts to capture heterogeneous regulatory contexts.
• Structure-guided representation learning: uses SFM-derived causal ordering to guide EFM pretraining.
• Cell- and gene-level outputs: supports downstream analyses that require transferable embeddings and regulatory structure.
• Cross-species resources: includes shared assets for Homo sapiens and Mus musculus workflows.

Model architecture

Structure Foundation Module (SFM)

SFM learns context-aware structural representations of gene regulation. A Mixture-of-Experts architecture captures distinct regulatory contexts without fitting an independent causal model for every cell. The resulting structural factors and causal gene ordering guide EFM pretraining.

Embedding Foundation Module (EFM)

EFM learns gene and cell embeddings using the causal ordering produced by a frozen SFM. These representations are intended for downstream supervised and unsupervised analyses, including perturbation-response modeling, trajectory and lineage analysis, and phenotype-associated prediction tasks.

Single-cell expression + biological priors
                    │
                    ▼
         Shared vocabulary and tokenization
                    │
                    ▼
     Structure Foundation Module (SFM)
                    │
     Context-specific regulatory structure
                    │
                    ▼
     Embedding Foundation Module (EFM)
                    │
                    ▼
          Gene and cell representations

Checkpoint summary

Component	Tensors	Parameters	Precision	Weight file
SFM	199	135,393,926	F32	models/sfm/sfm_model.safetensors
EFM	181	149,029,941	F32	models/efm/efm_model.safetensors
Shared vocabulary	1	15,571,200	F32	tokenizer/vocab.safetensors
Total	381	299,995,067	F32	model.safetensors.index.json

The root Safetensors index describes the complete release while preserving the modular SFM, EFM, and tokenizer layout.

Installation

scCAFM requires Python 3.10–3.14; Python 3.12.9 is recommended. Install the package from the source repository:

git clone https://github.com/Catchxu/scCAFM.git
cd scCAFM
pip install .

For the pinned Python 3.12 environment:

pip install ".[py312]"

GPU-specific packages, including FlashAttention, must be selected for the local CUDA, PyTorch, compiler, and GPU environment. See the source repository for FA2/FA4 configuration and validation instructions.

Download the model assets

Install or update the Hugging Face CLI, then download this repository into the package's assets directory:

pip install -U huggingface_hub
hf download kaichenxu/scCAFM --local-dir assets

The resulting layout is:

assets/
├── release.json
├── model.safetensors.index.json
├── resources/
│   ├── human_tfs.csv
│   ├── mouse_tfs.csv
│   ├── OmniPath.csv
│   └── homologous.csv
├── tokenizer/
│   ├── cond_dict.json
│   ├── vocab.json
│   └── vocab.safetensors
└── models/
    ├── sfm/
    │   ├── sfm_config.json
    │   └── sfm_model.safetensors
    └── efm/
        ├── efm_config.json
        └── efm_model.safetensors

release.json is the machine-readable manifest for resolving shared resources, tokenizer assets, module configurations, and checkpoints.

Repository contents

Path	Description
model.safetensors.index.json	Root index for Safetensors discovery and tensor-to-checkpoint mapping
release.json	Versioned manifest describing the release layout
resources/human_tfs.csv	Human transcription-factor resource
resources/mouse_tfs.csv	Mouse transcription-factor resource
resources/OmniPath.csv	Prior-knowledge interactions derived from OmniPath
resources/homologous.csv	Cross-species homology resource
tokenizer/cond_dict.json	Condition vocabulary
tokenizer/vocab.json	Shared gene vocabulary
tokenizer/vocab.safetensors	Tensor representation of the shared vocabulary
models/sfm/	SFM configuration and pretrained weights
models/efm/	EFM configuration and pretrained weights

Intended use

scCAFM is intended for research applications involving single-cell transcriptomics, including:

• context-specific gene-regulatory network analysis;
• gene and cell representation learning;
• perturbation-response and phenotype-associated modeling;
• developmental trajectory and lineage analysis;
• comparative analyses across supported human and mouse resources.

Limitations and responsible use

• scCAFM is a research model and is not intended for clinical diagnosis or treatment decisions.
• Model-derived causal relationships are computational hypotheses and should be validated with appropriate experimental or orthogonal evidence.
• Performance may vary across tissues, technologies, preprocessing pipelines, species, and biological contexts that differ from the training setting.
• Downstream users are responsible for evaluating robustness, calibration, batch effects, and potential biases for their specific application.

Resources and support

• Code and documentation: github.com/Catchxu/scCAFM
• Model assets: huggingface.co/kaichenxu/scCAFM
• Issues and questions: GitHub Issues
• License: GNU General Public License v3.0