How to use from the
Use from the
Transformers library
# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("fill-mask", model="ATLASBIOINFO/PlantDNA-FM")
# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForMaskedLM

tokenizer = AutoTokenizer.from_pretrained("ATLASBIOINFO/PlantDNA-FM")
model = AutoModelForMaskedLM.from_pretrained("ATLASBIOINFO/PlantDNA-FM", device_map="auto")
Quick Links

PlantDNA-FM

PlantDNA-FM is a DNA foundation model for plant genomes. It reads up to 16,384 nucleotides at single-nucleotide resolution and was pre-trained on plant genome sequences with masked nucleotide modelling.

Architecture Transformer encoder (ESM-style), 36 layers ร— 12 heads, hidden size 768, rotary position embeddings
Parameters 256 M
Tokenisation one token per nucleotide (A, C, G, T, N)
Maximum input 16,384 nt (16,386 tokens with <cls> and <eos>)

Load the model

import torch
from transformers import AutoTokenizer, AutoModelForMaskedLM

tokenizer = AutoTokenizer.from_pretrained("ATLASBIOINFO/PlantDNA-FM")
model = AutoModelForMaskedLM.from_pretrained("ATLASBIOINFO/PlantDNA-FM").eval()

sequence = "ATGGCGTACGTTAGCCTAGGATCCGATTACA"   # upper-case DNA, up to 16,384 nt
inputs = tokenizer(sequence, return_tensors="pt")

with torch.no_grad():
    out = model(**inputs, output_hidden_states=True)

embeddings = out.hidden_states[-1][0, 1:-1]   # one 768-dimensional vector per nucleotide
logits = out.logits[0, 1:-1]                  # nucleotide predictions per position

Requires transformers โ‰ฅ 5.1 and PyTorch. For long sequences, use a GPU and model.half() or torch.autocast to save memory.

Fine-mapping prior

PlantDNA-FM attention can be used as a prior for association mapping and fine-mapping (weighted Bonferroni detection and SuSiE). Code: https://github.com/atlasbioinfo/PlantDNA-Finemapping

Citation

Yu H. et al. DNA foundation model priors improve fine-mapping in plant populations. (in preparation)

Downloads last month
25
Safetensors
Model size
0.3B params
Tensor type
F32
ยท
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support