Instructions to use ATLASBIOINFO/PlantDNA-FM with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use ATLASBIOINFO/PlantDNA-FM with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("fill-mask", model="ATLASBIOINFO/PlantDNA-FM")# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForMaskedLM tokenizer = AutoTokenizer.from_pretrained("ATLASBIOINFO/PlantDNA-FM") model = AutoModelForMaskedLM.from_pretrained("ATLASBIOINFO/PlantDNA-FM", device_map="auto") - Notebooks
- Google Colab
- Kaggle
# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForMaskedLM
tokenizer = AutoTokenizer.from_pretrained("ATLASBIOINFO/PlantDNA-FM")
model = AutoModelForMaskedLM.from_pretrained("ATLASBIOINFO/PlantDNA-FM", device_map="auto")Quick Links
PlantDNA-FM
PlantDNA-FM is a DNA foundation model for plant genomes. It reads up to 16,384 nucleotides at single-nucleotide resolution and was pre-trained on plant genome sequences with masked nucleotide modelling.
| Architecture | Transformer encoder (ESM-style), 36 layers ร 12 heads, hidden size 768, rotary position embeddings |
| Parameters | 256 M |
| Tokenisation | one token per nucleotide (A, C, G, T, N) |
| Maximum input | 16,384 nt (16,386 tokens with <cls> and <eos>) |
Load the model
import torch
from transformers import AutoTokenizer, AutoModelForMaskedLM
tokenizer = AutoTokenizer.from_pretrained("ATLASBIOINFO/PlantDNA-FM")
model = AutoModelForMaskedLM.from_pretrained("ATLASBIOINFO/PlantDNA-FM").eval()
sequence = "ATGGCGTACGTTAGCCTAGGATCCGATTACA" # upper-case DNA, up to 16,384 nt
inputs = tokenizer(sequence, return_tensors="pt")
with torch.no_grad():
out = model(**inputs, output_hidden_states=True)
embeddings = out.hidden_states[-1][0, 1:-1] # one 768-dimensional vector per nucleotide
logits = out.logits[0, 1:-1] # nucleotide predictions per position
Requires transformers โฅ 5.1 and PyTorch. For long sequences, use a GPU and model.half() or torch.autocast to save memory.
Fine-mapping prior
PlantDNA-FM attention can be used as a prior for association mapping and fine-mapping (weighted Bonferroni detection and SuSiE). Code: https://github.com/atlasbioinfo/PlantDNA-Finemapping
Citation
Yu H. et al. DNA foundation model priors improve fine-mapping in plant populations. (in preparation)
- Downloads last month
- 25
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("fill-mask", model="ATLASBIOINFO/PlantDNA-FM")