README.md

---
license: mit
title: CELL-FM
emoji: 🧬
colorFrom: blue
colorTo: indigo
sdk: gradio
sdk_version: 5.9.1
app_file: app.py
pinned: false
short_description: Cell microscopy images from protein sequence
---

# CELL-FM

A virtual microscopy model that bridges microscopy images and protein sequences.

Each tab in the app is one **application** of the model. More will be added; the
sections below document the ones that exist.

---

## Condensate Titration

Predict how an Intrinsically Disordered Peptide (IDP) behaves as its
concentration rises, directly from sequence.

![CELL-FM condensate titration pipeline](images/cellfm_flow_diagram.png)

The only input is the sequence, and it must be **exactly 66 amino acids** — the
entire CondenSeq library the model was trained on is 66-mers (all 14,578 of them),
so other lengths are off-distribution. Sampling is fixed at 512 concentration
levels and 100 ODE steps, seed 6, matching the offline runs.

Enter a sequence and this section runs the three stages of the CELL-FM condensate
pipeline end to end:

1. **Generate.** CELL-FM CS samples one microscopy image per protein-intensity
   level, over a log-spaced concentration ladder from 20 to 2048 a.u. Every image
   is conditioned on the same fixed reference nucleus, so curves are comparable
   across sequences.
2. **Classify.** A ViT calls each generated image **condensed** or **diffuse**.
3. **Summarise.** A centred moving average over those binary calls is the
   **titration curve**; integrating it against log-intensity gives the two numbers
   the demo reports.

### What AUC and AAC mean

**AUC** — the area under the titration curve. How much of the scanned concentration
range the sequence spends in the condensed state. Higher means it condenses more
readily, over a wider range.

**AAC** — the area *above* the curve. The curve is compared against the one it would
have followed without **reentrant dissolution**: everything past the curve's maximum
is held flat at that maximum, and AAC is the area between the two. It is zero when
the sequence condenses and stays condensed, and grows the more the sequence
re-dissolves at high concentration.

Both are normalised by the width of the log-intensity range, so both lie in [0, 1]
and are directly comparable between sequences.

The curve also marks **c_sat**, the first intensity at which it reaches probability 0.8,
and the raw per-image calls download as a CSV.

### Layout

The tab is laid out like the Virtual OpenCell one, in the same OpenCell style:

- **Left** — the input alone, any 66-aa IDR sequence and **Run analysis**; then the
  run's AUC and AAC, and the titration curve.
- **Right** — the generated images in OpenCell's colours, nucleus blue and the IDP grey,
  with the same channel switch and per-channel intensity range and gamma. The slider
  steps through all 512 images along the ladder and the strip below holds 64 evenly
  spaced ones; the viewer opens on the image where condensation peaks, captioned with
  its expression level and call. Before a run it shows the reference nucleus alone.

Only generation and classification run on the GPU; the curve, viewer and strip are
drawn afterwards on the CPU, so none of that spends the ZeroGPU budget.

### Default sequence

The box is pre-filled with the **NUP98 IDP**, wild type, unnamed in the UI. Its F→S
panel — the wild type and 91 random F→S mutants, ten draws for each of 1 to 9
substitutions and the one way to replace all ten F — ships in
`assets/nup98_mutation_seqs.csv`; pasting a mutant in and comparing its AUC against WT is
the quickest way to see the model respond to sequence grammar, as in the paper's Fig. 5b.

### Amino acid substitution scan

Below the titration, the tab shows the paper's two substitution matrices side by side:
the mean change in **AUC** (Fig. 5d) and in **AAC** (Supplementary Fig. 5c) when every
residue of one type (row, *Mutate FROM*) is replaced by another (column, *Mutate TO*).
They come from an in silico scan of 100 held-out CondenSeq IDPs — 380 substitutions,
512 images per sequence on the same ladder as above — and each cell averages over the
IDPs that contain the FROM residue (58–95 of them, median 89). An IDP without it is left
out, not counted as a zero change.

**Click any cell** to see the distribution behind its mean: a histogram of the per-IDP
change in AUC and in AAC, with mean and median, and a table of every IDP's wild-type and
mutant values. The scan is far past a demo's GPU budget, so it is precomputed:
`condenseq/substitution_scan.npz` in the weights repo, built by
`notebooks/tools/build_substitution_scan_assets.py`, which recomputes every value from the
stored per-image calls and checks the means against the matrices
`ana_all_mutation_log_scale.py` wrote for the paper (they agree to 1e-16). This section
runs on CPU.

---

## Virtual OpenCell

Browse CELL-FM's virtual staining of 1,276 OpenCell proteins, laid out like an
[OpenCell](https://opencell.sf.czbiohub.org/) target page. Search for a protein:

- **Left** is OpenCell's annotation of the *real* cell line: localization grades
  (bar height is the grade, 3 the most prominent), UniProt and Ensembl IDs, sequence
  length, HEK293 abundance, and a link to the real images on OpenCell. None of it is
  predicted.
- **Right** are CELL-FM's images, in OpenCell's colours: the nucleus blue and the
  generated protein (the *target*) gray. OpenCell's channel switch and per-channel
  intensity range and gamma adjust the view; the strip below holds every sample of
  the protein, and **Download TIFF** saves the selected one as the raw 2-channel image.

Below the annotations, the **virtual staining map** places every protein's samples by
UMAP of their image embeddings (`opencell/vit.bin`), coloured by OpenCell localization in
the palette of the paper figures; the selected protein's samples are outlined and
labelled. Zoom, pan and hover work as in any Plotly chart, and clicking a legend entry
hides that category. The map is `opencell/vs_umap_embedding.npz` in the weights repo;
the backdrop draws 12 evenly spaced samples per protein to keep each update light.

Every image is generated from the protein sequence in **one shared cell**: the nucleus
channel is the same real nucleus (ATG7, crop 1) for every protein, so differences
between proteins come from the sequence, and a protein's samples are independent
draws for that cell.

The images are the dataset
[`BoHuangLab/CELL-FM`](https://huggingface.co/datasets/BoHuangLab/CELL-FM):
`OpenCell/virtual_staining/<GENE>/<NNNN>.tif`, 93,510 samples, each (2, 256, 256)
uint16 with the nucleus first. A protein's samples are fetched the first time it is
viewed (`VIRTUAL_OPENCELL_LOCAL_DIR` reads a local copy instead); while that repo is
private, the Space needs an `HF_TOKEN` secret that can read it. The protein table,
`assets/virtual_opencell_proteins.csv`, is built by
`notebooks/tools/build_virtual_opencell_assets.py`. This section runs on CPU.

---

## Hardware

**A GPU is required in practice.** The code itself runs on CPU — `flash-attn` is
deliberately left out of `requirements.txt`, so ESM-C uses its pure-torch rotary
fallback (verified bit-identical to the Triton path: same AUC, same AAC, same
256/256 per-image calls) — but a CPU run of the defaults would take hours and
time out. Use ZeroGPU or a GPU hardware tier.

The fixed settings — 512 images at 100 ODE steps — take **~85 s on an A40**
(measured: 11.5 s per batch of 64, plus ~9 s of one-off model loading). Runtime
scales linearly in both, so a section that needs to be faster should lower them in
`pipeline.py` (`NUM_IMAGES`, `NUM_STEPS`).

On ZeroGPU the per-call budget is set by `ZEROGPU_DURATION` (default 75 s). Keep it
close to what a run needs: `spaces` requests 1.5× it on the default GPU size, and ZeroGPU
admits a run only when the visitor's remaining daily quota covers the whole request, while
charging only the time used. A titration takes ~56 s on ZeroGPU (measured 2026-10-16), so
75 s requests ~113 s — twice the run, and two runs a day on an anonymous visitor's ~180 s;
signed-in visitors get more. At 300 s (450 s requested) every visitor without PRO was
turned away with "The requested GPU duration (450s) is larger than the maximum allowed".

---

## Running it without a Space

Creating a Gradio Space under an organization now needs a Team or Enterprise plan.
Until `HuangLab` has one, `run_local.sh` serves the identical app from a cluster GPU
over a temporary public URL:

```bash
sbatch run_local.sh      # prints a *.gradio.live URL, valid 72 h
```

## Weights

The Space downloads ~2.1 GB of checkpoints at startup from
[`BoHuangLab/CELL-FM`](https://huggingface.co/BoHuangLab/CELL-FM), overridable with
`CELLFM_MODEL_REPO`. That repo keeps one subfolder per model family:

| File | Model |
|---|---|
| `condenseq/cellfm_seq2img.bin` | CELL-FM CS generator, ESM-C 600M encoder included |
| `condenseq/vae.bin` | 160×160 image VAE |
| `condenseq/vit_cls.bin` | Condensed/diffuse ViT classifier |
| `condenseq/substitution_scan.npz` | The precomputed substitution scan: per-IDP AUC and AAC, wild type and every mutant |

To run against a local copy instead, set `CELLFM_LOCAL_WEIGHTS` to a **flat**
directory holding the three `.bin` files and `substitution_scan.npz` — the prefix is
stripped for local paths.
`upload_weights.py` pushes them from the cluster.

## Provenance

Ported from the CELL-FM repository, one module per stage, with behaviour preserved:

| Stage | Origin |
|---|---|
| Generation | `cell_fm/tasks_local/cell_fm_cs/evaluate_seq2img_dict.py` |
| Classification | `cell_fm/tasks_local/vit_cls_condenseq_img/evaluate_single_img.py` |
| Moving average | `cell_fm/tasks_local/vit_cls_condenseq_img/ma_plot.py` |
| AUC / AAC | `cell_diff/tasks/analysis/{utils,ana_all_mutation_log_scale}.py` |
| Substitution scan | `cell_diff/tasks/analysis/ana_all_mutation_log_scale.py`, precomputed by `notebooks/tools/build_substitution_scan_assets.py` |

Model hyperparameters mirror `scripts/cell_fm_cs/evaluate_seq2img.sh` and
`scripts/vit_cls_condenseq_img/pretrain.sh`. The metric port in `metrics.py` was
checked against the original on 40 stored curves and agrees to 0.0 in both log and
linear mode. Generated images make the same round-trip through uint16 that the
offline pipeline's TIFF write imposed, so demo numbers stay comparable to published
ones.

### Reference values, and how noisy they are

Measured at the fixed settings (512 images, 100 steps, **seed 6**) on an A40:

| | AUC | AAC | c_sat |
|---|---|---|---|
| NUP98 WT | 0.4912 | 0.0034 | 360 |
| 1F→S mutant | 0.4794 | 0.0035 | 412 |

These are single draws, and how much that matters depends on whether you pair.

**Taken alone, the numbers move a lot.** Over seeds 6–9 the WT AUC ranges
0.4660–0.5087 (sd 0.019) and c_sat ranges 289–531 (sd 102). Treat any individual
c_sat or AAC as indicative rather than precise.

**Compared at a matched seed, small differences hold up.** The 1F→S effect is
−0.0123 ± 0.0042 across four seeds and negative in every one, even though it is
smaller than the marginal spread above — driving both sequences from the same seed
cancels most of the sampling noise, so the comparison is far tighter than either
number is on its own. Compare sequences at the same seed and a single-residue
change is resolvable; the demo does this by construction, with the seed fixed at 6.

Measured on the cluster stack (torch 2.4.1); the Space runs torch 2.8.0, which is
not expected to change these but has not been compared directly.

---

## Citation

If you use CELL-FM, please cite:

```bibtex
@article{zheng2026virtual,
  title   = {Virtual experiments bridge sequence and microscopy with generative models},
  author  = {Zheng, Dihan and Hong, Kibeom and Huang, Bo},
  journal = {bioRxiv},
  year    = {2026},
  doi     = {10.64898/2026.09.13.751243},
  url     = {https://www.biorxiv.org/content/10.64898/2026.09.13.751243v1}
}
```