--- license: cc-by-4.0 library_name: pytorch tags: - single-cell - gene-expression --- # MetacellFormer **MetacellFormer** is a transcriptomic foundation model pretrained on metacell profiles to learn gene and cell representations. ## Usage Install the companion MetacellFormer GitHub package with `pip install .` from its repository root. Load this downloaded model directory with the supplied API: ```python import torch from metacellformer import GeneRankTokenizer, load_encoder, cell_embeddings model_dir = "MODEL_DIRECTORY" tokenizer = GeneRankTokenizer(model_dir) model = load_encoder(model_dir, device="cpu") # Small illustrative profile; replace with your gene IDs and raw counts. gene_ids = tokenizer.genes[:4] encoded = tokenizer.encode(gene_ids, [10, 5, 2, 1]) ids = torch.as_tensor(encoded["input_ids"]).unsqueeze(0) mask = torch.as_tensor(encoded["attention_mask"]).unsqueeze(0) embedding = cell_embeddings(model, ids, mask) print(embedding.shape) # torch.Size([1, 1024]) ``` Provide all measured genes and their raw counts so normalization uses the whole profile. When supplying a feature subset, pass the full-profile total as `total_counts_all_features` to `encode`. Paper evaluation uses token type 0 and non-padding final-layer mean pooling, including CLS. The GitHub `examples/02_embeddings.ipynb` demonstrates inference on an evaluation matrix. ## Files `model.safetensors`, `config.json`, `gene_vocabulary.json` and `gene_reference_medians.npy` provide the model and inference resources. `gene_reference_support.npy` and `training_resources/` support pretraining reproduction. Evaluation inputs and fixed contracts are supplied in MetacellFormer-evaluation. Checkpoint details are described in the manuscript Methods. ## License Model materials: CC BY 4.0. The GitHub code is licensed separately under Apache License 2.0.