| --- |
| license: cc-by-nc-sa-4.0 |
| library_name: pytorch |
| tags: |
| - biology |
| - genomics |
| - drug-discovery |
| - target-prioritization |
| - variant-prioritization |
| --- |
| |
| # Varformer |
|
|
| Varformer is a multimodal transformer for population-scale drug target and variant prioritization. It learns contextual representations of the missense variants observed in a population and integrates them with gene-level biological features through cross-modal attention. It is trained with a non-negative positive-unlabeled objective for the gene score, and an allele-frequency objective for a per-variant score. |
|
|
| Code, documentation and installation: https://github.com/aaronwtr/Varformer |
|
|
| ## Models in this repository |
|
|
| The 25 models of the paper. Each was trained with five seeds (13, 23, 42, 85, 482), with the variant head (`variant_loss_weight` 0.1, pooled binary cross-entropy) and 32-bit precision. |
|
|
| | Folder | Cohort | Variant source | Gene inputs | Use in the paper | |
| |---|---|---|---|---| |
| | `afr/` | African / African American | gnomAD v4.1 | without `geneticConstraint` | gene prioritization | |
| | `amr/` | Admixed American | gnomAD v4.1 | without `geneticConstraint` | gene prioritization | |
| | `nfe/` | Non-Finnish European | gnomAD v4.1 | without `geneticConstraint` | gene prioritization | |
| | `sas/` | South Asian | Genes & Health | without `geneticConstraint` | gene prioritization | |
| | `sas-gnh/` | South Asian | Genes & Health | with `geneticConstraint` | variant scores in the Genes & Health analyses | |
|
|
| Each model has two files: |
|
|
| - `seed<N>.safetensors`: the model weights only. The files contain no training data, no genotypes and no individual-level information. |
| - `seed<N>.json`: the training settings, the input dimensions, the class prior, the decision threshold, the selected epoch, the validation Spearman correlation, and SHA-256 checksums of the weights and of the source checkpoint. |
|
|
| `index.json` lists all 25 models. |
|
|
| ## Use |
|
|
| ```python |
| from varformer import Varformer |
| |
| model = Varformer.from_pretrained("nfe", seed="best") |
| predictions = model.predict(genes=["ENSG00000141510"]) |
| variant_scores = model.score_variants(genes=["ENSG00000141510"]) |
| ``` |
|
|
| `from_pretrained` checks the weights against their checksum. The model inputs are built from local copies of the input data (gnomAD or Genes & Health variants, AlphaMissense scores, Open Targets, Human Protein Atlas and GTEx features); see the data sources in the code repository. Genes & Health data are available only to approved researchers. |
|
|
| The gene score ranks genes by target-prioritization evidence; it is not a calibrated probability of clinical success. The variant score ranks variants for research prioritization; it is not validated for clinical variant classification. |
|
|
| ## License |
|
|
| The weights are released under [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/): attribution, non-commercial use only, and the same license for derived works. The models take AlphaMissense scores as input, and AlphaMissense is released under the same license. The code is MIT-licensed. For commercial use, train a model with another pathogenicity predictor; the code repository describes how. |
|
|
| ## Citation |
|
|
| The paper citation will be added when the preprint is available. Until then, please cite the software release described in the code repository. |
|
|
