File size: 1,855 Bytes
1f7bfd7
 
a4dcf32
 
 
 
1f7bfd7
a4dcf32
 
 
94efbd2
a4dcf32
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
8c0de50
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
---
license: cc-by-4.0
library_name: pytorch
tags:
- single-cell
- gene-expression
---

# MetacellFormer

**MetacellFormer** is a transcriptomic foundation model pretrained on metacell profiles to learn gene and cell representations.

## Usage

Install the companion MetacellFormer GitHub package with `pip install .` from its repository root. Load this downloaded model directory with the supplied API:

```python
import torch
from metacellformer import GeneRankTokenizer, load_encoder, cell_embeddings

model_dir = "MODEL_DIRECTORY"
tokenizer = GeneRankTokenizer(model_dir)
model = load_encoder(model_dir, device="cpu")
# Small illustrative profile; replace with your gene IDs and raw counts.
gene_ids = tokenizer.genes[:4]
encoded = tokenizer.encode(gene_ids, [10, 5, 2, 1])
ids = torch.as_tensor(encoded["input_ids"]).unsqueeze(0)
mask = torch.as_tensor(encoded["attention_mask"]).unsqueeze(0)
embedding = cell_embeddings(model, ids, mask)
print(embedding.shape)  # torch.Size([1, 1024])
```

Provide all measured genes and their raw counts so normalization uses the whole profile. When supplying a feature subset, pass the full-profile total as `total_counts_all_features` to `encode`. Paper evaluation uses token type 0 and non-padding final-layer mean pooling, including CLS. The GitHub `examples/02_embeddings.ipynb` demonstrates inference on an evaluation matrix.

## Files

`model.safetensors`, `config.json`, `gene_vocabulary.json` and `gene_reference_medians.npy` provide the model and inference resources. `gene_reference_support.npy` and `training_resources/` support pretraining reproduction. Evaluation inputs and fixed contracts are supplied in MetacellFormer-evaluation. Checkpoint details are described in the manuscript Methods.

## License

Model materials: CC BY 4.0. The GitHub code is licensed separately under Apache License 2.0.