Nucleotide Transformer v3
Collection
10 items • Updated
Request access to the NTv3 models
One approval grants access to every gated NTv3 model. Requests are reviewed manually.
The NTv3 weights are released under the InstaDeep Open Model Licence for non-commercial use only, and its terms also cover the models' outputs. For commercial use, contact legal@instadeep.com.
Log in or Sign Up to review the conditions and access this model content.
NTv3 Annotate — mammals
NTv3 gene-annotation model for mammalian genomes: 703M parameters, predicting one of 15
gene-structure classes at every base of a 1 Mb window. Modelling code is not in this
repository: config.json resolves it from
InstaDeepAI/NTv3_pretrained
through auto_map, so loading requires trust_remote_code=True.
Use 1,048,576 bp windows only. This model was trained only on 1 Mb windows. Shorter windows run without error, but annotation quality drops as the window gets shorter. Tile long sequences with 1 Mb windows, keeping the central 75% of each, and pad sequences shorter than 1 Mb with
Nup to 1,048,576 bp. The scores below were measured this way.
Test-species annotations
Whole-genome predictions on the 2 held-out test species are in
annot_results/; its README gives provenance and scoring.
| species | CDS F1 | gene F1 | GFF3 |
|---|---|---|---|
| Bos taurus (cow) | 90.6 | 58.1 | annot_results/ntv3_bos_taurus_GCF_002263795.3.gff3.gz |
| Delphinapterus leucas (beluga whale) | 89.7 | 54.7 | annot_results/ntv3_delphinapterus_leucas_GCF_002288925.2.gff3.gz |
Data and notebooks
- Benchmark data. The test species' soft-masked genomes and reference
annotations are in
InstaDeepAI/NTv3_annot_datasetundermammals/test_species/: the longest-isoform reference the scores above use (reference_longest_isoform.gff3.gz) and every annotated isoform (reference_all_isoforms.gff3.gz). - Notebooks in the
ntv3Space: 03a trains a plant annotation model from NTv3-tracks, 03b decodes per-base predictions into gene models, and 03c scores a GFF3 against a reference the way the benchmark does.
Model information
- Soft-masked input. The tokenizer has 16 tokens:
A C G T N, their lowercase (repeat-masked) forms, and special tokens. Lowercase bases carry their own token ids, so feed a soft-masked assembly (e.g. NCBI_genomic.fna, Ensembldna_sm) for best results. An uppercase-only sequence runs without error but loses the repeat signal.AutoTokenizer.from_pretrained(..., trust_remote_code=True)loads it. - Window. 1,048,576 bp windows, keeping the central 0.75 of each
(
ntv3.keep_target_center_fraction, 0.75 by default inconfig.json). Keep both values: other settings give a plausible but wrong annotation, not an error. Pad sequences shorter than 1 Mb withN. - No species conditioning. Pass an empty list:
conditions_tokens=[]. - Precision. Weights are fp32. For bf16, use
torch.autocast(device_type="cuda", dtype=torch.bfloat16); casting the model itself (torch_dtype="bfloat16",model.to(torch.bfloat16)) fails. - Decoding. The model outputs per-base class logits, not genes.
config.jsonholds the decoding HMM underannotation(states, transitions, splice-motif rules). Notebook 03b decodes the logits into gene models (GFF3) with it, and the NTv3-annotate pipeline (Home tab of thentv3Space) annotates a genome FASTA end to end.
Licence
Released under the InstaDeep Open Model Licence. The licence restricts output as well as weights, so predicted annotations carry the same terms. Read it before building on the model commercially.
Citation
Citation to be added on publication.
- Downloads last month
- 17