| --- |
| license: mit |
| library_name: cell-fm |
| tags: |
| - biology |
| - microscopy |
| - protein |
| - condensate |
| - flow-matching |
| --- |
| |
| # CELL-FM weights |
|
|
| Checkpoints of CELL-FM. |
|
|
| | File | Model | |
| |---|---| |
| | `condenseq/cellfm_seq2img.bin` | CELL-FM CS sequence-to-image generator, trained with IDPs held out (includes the ESM-C 600M encoder) | |
| | `condenseq/vae.bin` | Image VAE, 160x160, 3 down blocks, 4 latent channels | |
| | `condenseq/vit_cls.bin` | ViT condensed/diffuse classifier, 2-channel 160x160 input | |
| | `condenseq/reference_nucleus.npy` | The DAPI channel every CondenSeq generation is conditioned on: `(1, 160, 160)` float32 in [-1, 1]. CondenSeq protein index 12626, image 0 — the same one the offline seq2img runs used | |
| | `hpa/cellfm_seq2img.bin` | CELL-FM sequence-to-image generator for HPA, trained with test proteins held out; 256x256, 3-channel conditioning (includes the ESM-C 600M encoder) | |
| | `hpa/cellfm_vs.bin` | CELL-FM virtual-staining generator for HPA, trained on all HPA proteins; 256x256, 3-channel conditioning (includes the ESM-C 600M encoder) | |
| | `hpa/vae.bin` | Image VAE at 256x256, the one both `hpa/cellfm_seq2img.bin` and `hpa/cellfm_vs.bin` were trained against | |
| | `hpa/vit.bin` | Image-embedding ViT for HPA: 12 layers, 512 hidden, 2048 MLP, 8 heads, patch 4, 256x256 input. Trained to identify the protein in an image over 13,908 classes; the embedding is the representation, not the prediction. Four input channels — protein, nucleus, ER, microtubules | |
| | `hpa/cellfm_img2seq.bin` | CELL-FM image-to-sequence model for HPA, 512x512, 3-channel conditioning (includes the ESM-C 600M encoder) | |
| | `hpa/vae_512.bin` | Image VAE at 512x512, the one `hpa/cellfm_img2seq.bin` was trained against — a different model from `hpa/vae.bin`, not a rename | |
| | `opencell/cellfm_vs.bin` | CELL-FM virtual-staining generator for OpenCell, fine-tuned on all OpenCell proteins and meant for those proteins only; 256x256, single nucleus conditioning channel (includes the ESM-C 600M encoder) | |
| | `opencell/vae.bin` | Image VAE at 256x256, the one `opencell/cellfm_vs.bin` was trained against — OpenCell-finetuned, so not interchangeable with `hpa/vae.bin` despite the matching shape | |
| | `opencell/vit.bin` | The same ViT fine-tuned on OpenCell, 1,311 classes. Identical backbone to `hpa/vit.bin`, but its input stem takes **two** channels — protein and nucleus — where the HPA one takes four, so the two cannot be swapped: the count is fixed in `conv_proj` and a mismatch fails there | |
| | `opencell/vs_anchor_nucleus.npy` | The nucleus every OpenCell generation is conditioned on: `(1, 256, 256)` float32 in [-1, 1]. Gene ATG7, crop `CID001813_FID00035838_proj_11` — bit-for-bit the conditioning channel of the published offline run | |
| | `opencell/vs_genes.csv` | OpenCell's 1,311 genes: name, protein name, UniProt accession, Ensembl id, localization annotation and sequence. The metadata table minus its image paths | |
| | `opencell/vs_reference_cells.npz` | Four proteins in two matched pairs (POLR1A/SNRPF nuclear, LSM14A/DDX6 both P-body), each with one real OpenCell crop as a `(nucleus, protein)` float16 pair — the image shown beside the generated one | |
| | `hpa/anchor_cell.npy` | The fixed cell every NLS-screening image is conditioned on: `(3, 256, 256)` float32 in [-1, 1], channels nucleus, ER, microtubules. HPA gene H3C13, cell crop `1194_B2_2_4` | |
| | `hpa/anchor_masks.npz` | Two 256x256 boolean masks over that cell, `nucleus` and `cell`; cytoplasm is `cell & ~nucleus` | |
| | `hpa/pls_anchor_nls.npz` | The cell PLS generation conditions on for nuclear signals: `cell` `(3, 512, 512)` nucleus/ER/microtubules and `protein` `(1, 512, 512)`, float32 in [-1, 1]. HPA gene PPM1G (Nucleoplasm), crop `392_B9_1_11` | |
| | `hpa/pls_anchor_nes.npz` | The same for export signals. HPA gene DIAPH1 (Cytosol, Plasma membrane), crop `1608_B3_1_1` | |
| | `hpa/proteome_aa_counts.json` | Residue counts over the 12,894 HPA proteins (7,940,784 residues), the proteome baseline the frequency analysis compares against | |
| | `hpa/pls_reference_nls.csv` | The 320 published NLS signals: 20 independent draws at each of 16 tail lengths, 10-25 aa | |
| | `hpa/pls_reference_nes.csv` | The same 320 for export signals | |
|
|
| Hyperparameters for the CondenSeq models are set in `pipeline.py` in the Space and mirror |
| `scripts/cell_fm_cs/evaluate_seq2img.sh` and |
| `scripts/vit_cls_condenseq_img/pretrain.sh` in the CELL-FM repository. The HPA |
| hyperparameters are spelled out in `notebooks/nls_screening.ipynb` and mirror |
| `scripts/cell_fm/evaluate_virtual_staining_hpa_dict.sh`. The img2seq pair mirrors |
| `scripts_local/cell_fm/evaluate_img2seq_hpa_v2.sh`: 512 px, `sample_size` 128, |
| `encoder_patch_size` 8, `img_generator_patch_size` 4, 8 attention heads. The OpenCell pair |
| mirrors `scripts/cell_fm/evaluate_virtual_staining_opencell.sh`: 256 px, `sample_size` 64, |
| `encoder_patch_size` 4, `img_generator_patch_size` 2, 18 attention heads, `cell_image` `nucl` |
| — spelled out in `notebooks/opencell_vs.ipynb`. That single conditioning channel is the one |
| architectural difference a caller can see: the HPA generator takes three. |
|
|
| Each generator must be loaded with the VAE it was trained against — pairing |
| `cellfm_img2seq.bin` with the 256 px `vae.bin` gives a latent-size mismatch. |
|
|
