---
license: mit
tags:
- genomics
- dna
- transcription-factor
- tbx5
- evo2
- cardiomyopathy
base_model: arcinstitute/evo2_7b
---

# Supervised Evo 2 classifier for TBX5 binding

Classifier of TBX5-bound motif sites from Evo 2 7B embeddings, trained with whole chromosomes held out. It is the
supervised Evo 2 classifier of *Evo 2 Predicts Cardiomyopathy-Associated Variants and Elucidates Their Underlying
Mechanisms* (Kurozumi, Otsuka et al.; bioRxiv doi:10.64898/2026.05.15.725304), Figure 4.

## Input

For an 8-bp site, the Evo 2 7B embeddings (`blocks.26.mlp.l3`, 4,096 dimensions) of its eight nucleotides are taken
with an 8,192-bp context centred on the site, for the forward and the reverse-complement strands, and concatenated:
8 × 4,096 × 2 = 65,536 values. They are standardised with `scaler.pt` (`mu`, `sd`, from the training set).

## Model and training

- Input (65,536) → Linear (8,192) → ReLU → BatchNorm → Dropout (0.3) → Linear (2,048) → ReLU → BatchNorm → Dropout
  (0.3) → Linear (512) → ReLU → BatchNorm → Dropout (0.3) → Linear (128) → ReLU → BatchNorm → Dropout (0.15) →
  Linear (1); the output is a logit.
- Label 1: TBX5_motif_peak (`motif_chip`); label 0: TBX5_nonmotif_nonpeak, TBX5_nonmotif_peak, TBX5_motif_nonpeak.
- BCEWithLogitsLoss with class weighting; Adam (learning rate 1 × 10⁻³, weight decay 1 × 10⁻⁵, batch 256); early
  stopping (patience 7) and learning-rate decay (factor 0.5, patience 3) on the validation loss; the weights with the
  lowest validation loss (epoch 2) are given here.
- Chromosome-level split: training chr1, 3, 4, 5, 8, 10, 16, 17, 18, 20, 21, 22 (99,256 sites); validation chr2, 11,
  12, 15, X, Y (49,084); test chr6, 7, 9, 13, 14, 19 (51,370). `split_assignment.tsv` lists every site.

## Test performance (chromosomes 6, 7, 9, 13, 14, 19)

| TBX5_motif_peak vs | AUROC |
|---|---|
| TBX5_nonmotif_nonpeak | 0.993 |
| TBX5_nonmotif_peak | 0.980 |
| TBX5_motif_nonpeak | 0.824 |

## Usage

```python
import torch, torch.nn as nn

dims, drops = [65536, 8192, 2048, 512, 128], [0.3, 0.3, 0.3, 0.15]
layers = []
for i in range(4):
    layers += [nn.Linear(dims[i], dims[i + 1]), nn.ReLU(), nn.BatchNorm1d(dims[i + 1]), nn.Dropout(drops[i])]
net = nn.Sequential(*layers, nn.Linear(128, 1))

class Net(nn.Module):
    def __init__(self):
        super().__init__()
        self.net = net
    def forward(self, x):
        return self.net(x).squeeze(-1)

model = Net()
model.load_state_dict(torch.load("best_model.pth", map_location="cpu", weights_only=True))
model.eval()
sc = torch.load("scaler.pt", map_location="cpu")
prob = torch.sigmoid(model((x - sc["mu"]) / sc["sd"]))   # x: (n, 65,536) as described above
```

## Files

- `best_model.pth`, `scaler.pt`: weights and input standardisation
- `results.json`: test AUROCs and the training history
- `split_assignment.tsv`: train/validation/test assignment of every site

Also archived on Zenodo with the analysis code: doi:10.5281/zenodo.23824204.

## Citation

```
@article{kurozumi2026evo2,
  title={Evo 2 Predicts Cardiomyopathy-Associated Variants and Elucidates Their Underlying Mechanisms},
  author={Kurozumi, Atsumasa and Otsuka, Naoto and Ito, Masamichi and Kawakami, Toshinaru and Isagawa, Takayuki and Kodera, Satoshi and Takeda, Norihiko},
  journal={bioRxiv}, year={2026}, doi={10.64898/2026.05.15.725304}
}
```