| ---
|
| license: cc-by-4.0
|
| tags:
|
| - biology
|
| - proteomics
|
| - transcriptomics
|
| - single-cell
|
| library_name: omicsfm
|
| ---
|
|
|
| # OmicsFM
|
|
|
| A foundation model over paired proteomic and transcriptomic expression.
|
| Attention between measured features yields a protein-protein association
|
| network without task-specific training.
|
|
|
| ## Checkpoints
|
|
|
| | name | modality | feature ID embedding | parameters |
|
| |---|---|---|---|
|
| | `proteomics` | proteomics | learned | 4.94 M |
|
| | `proteomics_esmc` | proteomics | ESM-C | 5.75 M |
|
| | `bulk_transcriptomics` | bulk transcriptomics | learned | 4.94 M |
|
| | `bulk_transcriptomics_esmc` | bulk transcriptomics | ESM-C | 5.75 M |
|
| | `sc_transcriptomics` | single-cell transcriptomics | learned | 4.94 M |
|
| | `sc_transcriptomics_esmc` | single-cell transcriptomics | ESM-C | 5.75 M |
|
| | `sc_transcriptomics_ensembl` | single-cell transcriptomics | learned | 4.94 M |
|
|
|
| All but one are mapped onto a shared UniProt accession vocabulary of 20,272
|
| accessions. `sc_transcriptomics_ensembl` is the exception: it is trained
|
| directly on 61,497 ENSEMBL gene IDs, with no gene-to-protein mapping applied.
|
| It exists so the cost of that mapping can be measured. Mapping genes to
|
| proteins drops features that fail to map, and drops ground-truth edges
|
| between them; comparing this checkpoint against `sc_transcriptomics` isolates
|
| that effect, because the two are identical in architecture and training and
|
| differ only in vocabulary.
|
|
|
| **Feature ID embedding.** The `learned` variants train an embedding table over
|
| the vocabulary from scratch. The `ESM-C` variants instead look each feature up
|
| in a frozen table of precomputed ESM-C sequence embeddings and pass it through
|
| a small trained projection, so identity is grounded in protein sequence rather
|
| than learned from co-expression alone.
|
|
|
| **Parameter counts** are the trained model excluding the feature ID embedding,
|
| which scales with vocabulary rather than model capacity and would otherwise
|
| dominate. The transformer is identical across all seven at 4.94 M parameters; the
|
| ESM-C variants add 0.81 M for the projection. Excluded on top of that: the
|
| 5.19 M learned embedding table (`learned` variants) and the 23.35 M frozen
|
| ESM-C lookup (`ESM-C` variants), which is not trained at all.
|
| `sc_transcriptomics_ensembl` carries a larger 15.74 M table, since its
|
| vocabulary is three times the size -- which is why the exclusion matters:
|
| it is the same model, not a bigger one.
|
|
|
| ## Repository contents
|
|
|
| Each checkpoint folder holds `best_model.ckpt` and a `config.yaml` recording the
|
| architecture, hyperparameters, training epoch and validation loss, readable
|
| without loading the checkpoint.
|
|
|
| `esmc_human_cache.pt` (95 MB) sits at the repository root and is required by the
|
| three `_esmc` checkpoints, which use it as a frozen lookup table instead of
|
| learning their feature embeddings. It holds one 1152-dimensional vector per
|
| UniProt accession: ESM-C 600M run over the canonical human proteome, mean-pooled
|
| per sequence. It can be recomputed, but that means downloading a
|
| 600M-parameter model and running inference over 20,000 sequences. Shipping it
|
| keeps the embeddings bit-identical to what the models saw during training, and
|
| lets anyone run inference with the `_esmc` checkpoints straight away. The
|
| `learned` checkpoints do not need it.
|
|
|
| ## Use
|
|
|
| ```python
|
| from omicsfm.hub import get_checkpoint
|
| from omicsfm.api import attention_map
|
|
|
| ckpt = get_checkpoint("sc_transcriptomics") # downloaded and cached on first use
|
| network = attention_map(str(ckpt), data_path=...)
|
| ```
|
|
|
| The ESM-C variants additionally need the canonical human proteome FASTA and
|
| the precomputed embedding cache; see the repository README.
|
|
|
| ## Citation
|
|
|
| Publication in preparation.
|
|
|
