NTv3 Models (gated)
Collection
NTv3 models: pretrained, tracks, annotation, generative. One approval covers all. Non-commercial use only (InstaDeep Open Model Licence). β’ 7 items β’ Updated
Request access to the NTv3 models
One approval grants access to every gated NTv3 model. Requests are reviewed manually.
The NTv3 weights are released under the InstaDeep Open Model Licence for non-commercial use only, and its terms also cover the models' outputs. For commercial use, contact legal@instadeep.com.
Log in or Sign Up to review the conditions and access this model content.
NTv3 Generative Plant Promoter
Designs plant core promoters of a requested strength. A masked discrete diffusion model (MDLM) that fills the 170 bp promoter slot of a STARR-seq plasmid, conditioned on an activity level and on the assay system the strength is measured in: tobacco leaves or maize protoplasts.
It is an example of NTv3's generative post-training: NTv3 after tracks post-training, fine-tuned on the plant promoter STARR-seq data of Jores et al. (2021), 170 bp core promoters from Arabidopsis thaliana, maize and sorghum.
Architecture
The NTv3 U-Net trunk, with FiLM conditioning in the trunk for each condition and the
language-model head on top. It starts from the tracks model at the 131 kb stage of its
post-training, the checkpoint that NTv3_tracks
was then extended to 1 Mb from. The track heads are removed.
| Parameters | 721,379,955 |
| Resolution | 1 bp |
| Input | a 6,016 bp plasmid, promoter slot masked |
| Generated | 170 bp |
| Vocabulary | 16 tokens (only A C G T are sampled) |
| Conditions | species (always masked), activity (5 bins), assay system (leaf, proto) |
| Precision | float32 |
Usage
Generation is iterative: notebook 06 of the NTv3 Space has the complete sampler with classifier-free guidance, and scores the designs with the Jores et al. CNN oracles. One forward pass looks like this:
import json
import torch
from huggingface_hub import hf_hub_download
from transformers import AutoModel, AutoTokenizer
repo = "InstaDeepAI/NTv3_generative_plant_promoter"
model = AutoModel.from_pretrained(repo, trust_remote_code=True).eval()
tokenizer = AutoTokenizer.from_pretrained(repo, trust_remote_code=True)
backbones = json.load(open(hf_hub_download(repo, "backbone_metadata.json")))
backbone = backbones["backbones"]["pPSup_H3_UTR_At_Zm"]
tokens = tokenizer(backbone["sequence"], add_special_tokens=False, return_tensors="pt")["input_ids"]
tokens[0, backbone["promoter_start"]:backbone["promoter_end"]] = tokenizer.mask_token_id
padding = torch.full((1, backbones["sequence_length"] - tokens.shape[1]), tokenizer.pad_token_id)
tokens = torch.cat([tokens, padding], dim=1)
meta = model.config.generation_metadata
vocab = meta["conditions_vocabulary"]
conditions = [
torch.tensor([meta["species_token_id"]]), # always the species mask token
torch.tensor([vocab["activity"]["4"]]), # strongest bin; vocab["activity"]["<mask>"] = unconditional
torch.tensor([vocab["condition"]["leaf"]]), # or "proto"
]
with torch.no_grad():
out = model(tokens, conditions_tokens=conditions)
out["logits"].shape # (1, 6016, 16)
trust_remote_code=True is required. The modelling code is not in this repository:
config.json resolves it from InstaDeepAI/NTv3_pretrained, which therefore has to be
reachable as well. The model class is NTv3Tracks with no track head: the output is the
language-model logits.
Outputs
| Key | Shape | What it is |
|---|---|---|
logits |
(batch, 6016, 16) |
Token logits; sample the promoter slot from them |
activity_logits |
(batch, 11) |
Auxiliary head from training, predicting the activity token |
condition_logits |
(batch, 8) |
Auxiliary head from training, predicting the assay system token |
species_logits |
(batch, 21) |
Auxiliary head; not trained to predict species, do not use |
Conditioning
Token ids for each condition are in config.generation_metadata["conditions_vocabulary"].
- Species. The model was trained on the promoters of the three species together
without being told which species a promoter came from: the species token is always the
species mask token,
generation_metadata["species_token_id"]. Passing a real species id gives the model an input it never saw in training. - Activity. Five bins of STARR-seq log2 enrichment, with edges
[-2.5, 0, 2.5, 5](backbone_metadata.json). Its mask token is the unconditional input for classifier-free guidance. - Assay system.
leaf(tobacco leaves) orproto(maize protoplasts).
backbone_metadata.json holds the two plasmid backbones the promoters were measured in:
the pPSup STARR-seq vector (Addgene #149416) with a 35S enhancer upstream of the promoter
slot and a histone H3 5' UTR downstream, one UTR for Arabidopsis and maize, one for
sorghum. Coordinates are 0-based and end-exclusive. Right-pad the plasmid with the pad
token to 6,016 bp, the length the model was trained on.
Training
| Data | Jores et al. (2021) promoter STARR-seq, A. thaliana, maize, sorghum, both assay systems; 80/10/10 splits per species |
| Objective | MDLM on the promoter slot only, linear noise schedule |
| Condition dropout | 10% for activity and for assay system |
| Tokens | β23.7 B |
| EMA | 0.9999; these are the EMA weights |
Evaluation
3,072 designs per activity bin and assay system (50 diffusion steps, guidance Ξ³ = 3), scored with the Jores et al. CNN of the matching system. Mean predicted log2 enrichment:
| System | bin 0 | bin 1 | bin 2 | bin 3 | bin 4 |
|---|---|---|---|---|---|
| leaf | β2.18 | β1.18 | 1.52 | 3.42 | 3.19 |
| proto | β1.67 | β0.98 | 1.05 | 2.32 | 2.24 |
Predicted strength rises with the requested bin from bin 0 to bin 3. Bin 4 lands at about the level of bin 3, so the top bin does not extend the range: the bin edges were inherited from an earlier run rather than derived from this data, and the top bin holds few training promoters.
Limitations
Not species-aware. The species token is always masked, so the model cannot be steered to a species. Designs are drawn from the pooled distribution of the three species it was trained on, whichever token you pass.
Asking for the strongest bin does not give the strongest promoters (see Evaluation).
The strengths above are oracle predictions, not measurements. These designs have not been tested experimentally, and the two oracles are not on a common scale, so a leaf score and a protoplast score are not directly comparable.
Generation is tied to this plasmid. The model fills a 170 bp slot in the backbones it was trained with; the flanking enhancer and 5' UTR are part of its input, not decoration.
An example model, not a benchmark result. It is published to show what NTv3's generative post-training produces and to make the notebook runnable.
Related models
NTv3_pretrainedβ the pretrained backbone, and the repository this model's code is loaded fromNTv3_tracksβ functional-track predictionNTv3_annot_mammals/_plants/_insectsβ gene annotation
License
Released under the InstaDeep Open Model Licence: non-commercial use only, with conditions
on the outputs of the model. See
MODEL-LICENSE.md
for the authoritative terms.
- Downloads last month
- 28