Spaces:
Running
title: NTv3 Benchmark
emoji: 🚀
colorFrom: red
colorTo: red
sdk: docker
app_port: 8501
tags:
- streamlit
pinned: false
short_description: NTv3 Benchmark
license: apache-2.0
NTv3 Benchmark
A standardized comparison of sequence-to-function models on 155 functional-track prediction tasks across 11 species. Every model is given the same 32-kb input, the same task-specific 1-bp prediction head and the same per-nucleotide Poisson-multinomial loss, so what is compared is the representation rather than the training recipe.
What is measured
Metric. Pearson correlation on the held-out test set, per track, averaged over three seeds where available.
Species (11). Animals: human, cattle, rat. Plants: A. thaliana, rice, maize, tomato. Fungi: S. cerevisiae, S. pombe. Protists: P. falciparum, D. discoideum.
Assays (7). Chromatin accessibility (ATAC-seq), histone modifications and transcription factor binding (ChIP-seq), transcription initiation (PRO-cap), RNA binding (eCLIP), gene expression (RNA-seq) and translation (Ribo-seq).
The benchmark tests two kinds of generalization. For human, A. thaliana, rice and maize, which NTv3-tracks saw during post-training, the assays and tracks were chosen to be absent from that post-training set. The remaining seven genomes were absent from post-training entirely, so those species measure transfer to an unseen genome.
Models
Ten models: two convolutional baselines trained from scratch, five pretrained genomic foundation models, two pretrained sequence-to-function models, and NTv3.
| Model | Training |
|---|---|
| BPNet, Residual CNN | from scratch |
| HyenaDNA, Caduceus, NTv2, Evo2, PlantCAD2 | pretrained |
| Borzoi, AlphaGenome | pretrained sequence-to-function |
| NTv3 | post-trained |
AlphaGenome is fine-tuned from its fold-1 checkpoint on human, because the all-folds checkpoint saw those test regions, and from its all-folds checkpoint on every other species.
Reading the leaderboard
Scores are never averaged across species. Not every model was run on every species, so a single cross-species number would compare models over different task sets. Pick a species in the sidebar for its own leaderboard, where every model shown was evaluated on all of that species' tracks.
Contents
data/ntv3_benchmark_results.csv, one row per model, species, track and seed, carrying the score and the training-cost columns the app reports. This is the data behind every view; nothing is computed elsewhere.src/streamlit_app.py, the app.
Links
- 📚 Benchmark dataset:
InstaDeepAI/NTv3_benchmark_dataset_revised, the genomes, splits and tracks to reproduce or extend the benchmark. - 🧬 NTv3 tutorials and models:
InstaDeepAI/ntv3. - 🗄️ Initial release (2025): the leaderboard of the initial NTv3 release is the last page of this Space.
Citation
TBC, to be added on publication of the NTv3 paper.
License
The app code and the results table here are Apache-2.0. The NTv3 model weights are not. They are released under the InstaDeep Open Model Licence, which permits non-commercial use only and also places conditions on model outputs. See each model repository for its terms.