---
license: cc-by-4.0
tags:
  - biology
  - proteomics
  - transcriptomics
  - single-cell
library_name: omicsfm
---

# OmicsFM

A foundation model over paired proteomic and transcriptomic expression.
Attention between measured features yields a protein-protein association
network without task-specific training.

## Checkpoints

| name | modality | feature ID embedding | parameters |
|---|---|---|---|
| `proteomics` | proteomics | learned | 4.94 M |
| `proteomics_esmc` | proteomics | ESM-C | 5.75 M |
| `bulk_transcriptomics` | bulk transcriptomics | learned | 4.94 M |
| `bulk_transcriptomics_esmc` | bulk transcriptomics | ESM-C | 5.75 M |
| `sc_transcriptomics` | single-cell transcriptomics | learned | 4.94 M |
| `sc_transcriptomics_esmc` | single-cell transcriptomics | ESM-C | 5.75 M |
| `sc_transcriptomics_ensembl` | single-cell transcriptomics | learned | 4.94 M |

All but one are mapped onto a shared UniProt accession vocabulary of 20,272
accessions. `sc_transcriptomics_ensembl` is the exception: it is trained
directly on 61,497 ENSEMBL gene IDs, with no gene-to-protein mapping applied.
It exists so the cost of that mapping can be measured. Mapping genes to
proteins drops features that fail to map, and drops ground-truth edges
between them; comparing this checkpoint against `sc_transcriptomics` isolates
that effect, because the two are identical in architecture and training and
differ only in vocabulary.

**Feature ID embedding.** The `learned` variants train an embedding table over
the vocabulary from scratch. The `ESM-C` variants instead look each feature up
in a frozen table of precomputed ESM-C sequence embeddings and pass it through
a small trained projection, so identity is grounded in protein sequence rather
than learned from co-expression alone.

**Parameter counts** are the trained model excluding the feature ID embedding,
which scales with vocabulary rather than model capacity and would otherwise
dominate. The transformer is identical across all seven at 4.94 M parameters; the
ESM-C variants add 0.81 M for the projection. Excluded on top of that: the
5.19 M learned embedding table (`learned` variants) and the 23.35 M frozen
ESM-C lookup (`ESM-C` variants), which is not trained at all.
`sc_transcriptomics_ensembl` carries a larger 15.74 M table, since its
vocabulary is three times the size -- which is why the exclusion matters:
it is the same model, not a bigger one.

## Repository contents

Each checkpoint folder holds `best_model.ckpt` and a `config.yaml` recording the
architecture, hyperparameters, training epoch and validation loss, readable
without loading the checkpoint.

`esmc_human_cache.pt` (95 MB) sits at the repository root and is required by the
three `_esmc` checkpoints, which use it as a frozen lookup table instead of
learning their feature embeddings. It holds one 1152-dimensional vector per
UniProt accession: ESM-C 600M run over the canonical human proteome, mean-pooled
per sequence. It can be recomputed, but that means downloading a
600M-parameter model and running inference over 20,000 sequences. Shipping it
keeps the embeddings bit-identical to what the models saw during training, and
lets anyone run inference with the `_esmc` checkpoints straight away. The
`learned` checkpoints do not need it.

## Use

```python
from omicsfm.hub import get_checkpoint
from omicsfm.api import attention_map

ckpt = get_checkpoint("sc_transcriptomics")   # downloaded and cached on first use
network = attention_map(str(ckpt), data_path=...)
```

The ESM-C variants additionally need the canonical human proteome FASTA and
the precomputed embedding cache; see the repository README.

## Citation

Publication in preparation.