|
Download README.md from altoslabs/scRM: direct link, hf CLI and curl.
- Browser
- Download file 3.86 kB
-
https://huggingface.co/altoslabs/scRM/resolve/main/README.md
- Command line
-
hf download hf://altoslabs/scRM/README.md
-
curl -L -o README.md https://huggingface.co/altoslabs/scRM/resolve/main/README.md
3.86 kB
| license: mit | |
| library_name: pytorch | |
| pipeline_tag: feature-extraction | |
| tags: | |
| - single-cell | |
| - genomics | |
| - rna-seq | |
| - jepa | |
| - foundation-model | |
| # scRM S-96 | |
| scRM is a single-cell foundation model trained directly on sequencing reads. Its input is raw | |
| per-(cell barcode, UMI) sequence: there is no aligner, no reference genome, no gene annotation and no | |
| count matrix anywhere on the input side. A cell embedding is read out through a hierarchy of | |
| attention pools over bytes, contigs, UMIs and windows. The model holds 1,182,720 parameters. | |
| Preprint: [He et al., *scRM: A single-cell foundation model trained on scRNA-seq reads*, bioRxiv | |
| 2026](https://www.biorxiv.org/content/10.64898/2026.10.24.761590v1). | |
| Code: [github.com/altoslabs/scRM](https://github.com/altoslabs/scRM). | |
| ## Checkpoints | |
| | name | trained on | steps | parameters | | |
| | --- | --- | --- | --- | | |
| | `scRM-S96` | 3,000 cells from four public mouse 10x 3′ datasets | 8,947 | 1,182,720 | | |
| | `scRM-S96-sc_mixology` | `sc_mixology`'s own cells at the same 3,000-cell cap | 8,947 | 1,182,720 | | |
| Both are the same architecture at the same recipe and differ only in what they read. `scRM-S96` is | |
| the reported model, benchmarked zero-shot. `scRM-S96-sc_mixology` is the dataset-specific arm, used | |
| only for the gene-detection comparison on `sc_mixology`. | |
| ## Files | |
| Four files per checkpoint: | |
| - `<name>.safetensors` — 103 weight tensors. | |
| - `<name>.pt` — the same weights as a torch `state_dict`. | |
| - `<name>.eval.pt` — that `state_dict` together with the `arch`, `readout` and `arch_sha` fields the | |
| evaluation loader requires. This is the file to load. | |
| - `<name>.config.json` — the `arch` and `readout` dicts, the step, the seed and the parameter count. | |
| ## Loading | |
| `embed()` takes no architecture arguments; the architecture comes from the checkpoint, and | |
| `Checkpoint.load` recomputes `arch_sha` over it and refuses a checkpoint whose weights and declared | |
| shape disagree. | |
| ```python | |
| from scrm_eval.embed import Checkpoint | |
| from scrm_model.evalio import model_factory | |
| ckpt = Checkpoint.load("scRM-S96.eval.pt", model_factory=model_factory) | |
| ``` | |
| Run with `PYTHONPATH=model:eval:target/py` from a checkout of the code repository. | |
| ## Architecture | |
| A local trunk of 4 layers and a global trunk of 6 layers, both at width 96 with 4 heads, rotary | |
| position embeddings at `theta = 10000` over a maximum of 1,600 positions, and a projector of hidden | |
| width 256 onto 96 dimensions. The readout pools bytes into contigs, contigs into UMIs, UMIs into | |
| windows, and two views into a cell, each pool a masked seed-attention pool. Both checkpoints carry | |
| `arch_sha = 7c4e4ffc9d304902eaaaeacc4ebac858edb48d5f5078d7f7245d71e7f7359bba`, which gates | |
| evaluation compatibility. | |
| ## Training | |
| Trained as a JEPA. The pretraining corpora are four public mouse 10x 3′ datasets — `mouse_brain_5k`, | |
| `neuron_1k_v3`, `neurons_5k` and `sod1_spinal_cord` — all downloadable without registration and | |
| released under CC BY 4.0. The reported model draws its 8,947 steps from 3,000 of those cells. Seed | |
| 20260903, torch 2.11.0+cu128. | |
| ## Scope of the evidence | |
| The reported numbers are zero-shot on held-out human and fly data; none of the four pretraining | |
| datasets appears in any evaluation. A read-level representation retains nucleotide content and can | |
| therefore carry germline variation, so these embeddings are better governed as genotype data than as | |
| expression matrices. | |
| ## Citation | |
| ```bibtex | |
| @article{he2026scrm, | |
| title = {scRM: A single-cell foundation model trained on scRNA-seq reads}, | |
| author = {He, D. and Patro, R. and Nassar, M. and Wu, H. and Yan, Q. and | |
| Xu, G. and Torkenczy, K. and Liu, J. and Tyanova, S.}, | |
| journal = {bioRxiv}, | |
| year = {2026}, | |
| doi = {10.64898/2026.10.24.761590}, | |
| url = {https://www.biorxiv.org/content/10.64898/2026.10.24.761590v1} | |
| } | |
| ``` | |