---
license: cc-by-nc-sa-4.0
library_name: pytorch
tags:
  - biology
  - genomics
  - drug-discovery
  - target-prioritization
  - variant-prioritization
---

# Varformer

Varformer is a multimodal transformer for population-scale drug target and variant prioritization. It learns contextual representations of the missense variants observed in a population and integrates them with gene-level biological features through cross-modal attention. It is trained with a non-negative positive-unlabeled objective for the gene score, and an allele-frequency objective for a per-variant score.

Code, documentation and installation: https://github.com/aaronwtr/Varformer

## Models in this repository

The 25 models of the paper. Each was trained with five seeds (13, 23, 42, 85, 482), with the variant head (`variant_loss_weight` 0.1, pooled binary cross-entropy) and 32-bit precision.

| Folder | Cohort | Variant source | Gene inputs | Use in the paper |
|---|---|---|---|---|
| `afr/` | African / African American | gnomAD v4.1 | without `geneticConstraint` | gene prioritization |
| `amr/` | Admixed American | gnomAD v4.1 | without `geneticConstraint` | gene prioritization |
| `nfe/` | Non-Finnish European | gnomAD v4.1 | without `geneticConstraint` | gene prioritization |
| `sas/` | South Asian | Genes & Health | without `geneticConstraint` | gene prioritization |
| `sas-gnh/` | South Asian | Genes & Health | with `geneticConstraint` | variant scores in the Genes & Health analyses |

Each model has two files:

- `seed<N>.safetensors`: the model weights only. The files contain no training data, no genotypes and no individual-level information.
- `seed<N>.json`: the training settings, the input dimensions, the class prior, the decision threshold, the selected epoch, the validation Spearman correlation, and SHA-256 checksums of the weights and of the source checkpoint.

`index.json` lists all 25 models.

## Use

```python
from varformer import Varformer

model = Varformer.from_pretrained("nfe", seed="best")
predictions = model.predict(genes=["ENSG00000141510"])
variant_scores = model.score_variants(genes=["ENSG00000141510"])
```

`from_pretrained` checks the weights against their checksum. The model inputs are built from local copies of the input data (gnomAD or Genes & Health variants, AlphaMissense scores, Open Targets, Human Protein Atlas and GTEx features); see the data sources in the code repository. Genes & Health data are available only to approved researchers.

The gene score ranks genes by target-prioritization evidence; it is not a calibrated probability of clinical success. The variant score ranks variants for research prioritization; it is not validated for clinical variant classification.

## License

The weights are released under [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/): attribution, non-commercial use only, and the same license for derived works. The models take AlphaMissense scores as input, and AlphaMissense is released under the same license. The code is MIT-licensed. For commercial use, train a model with another pathogenicity predictor; the code repository describes how.

## Citation

The paper citation will be added when the preprint is available. Until then, please cite the software release described in the code repository.