Instructions to use aakashdandekar2006/Coli-ByT5-Small with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use aakashdandekar2006/Coli-ByT5-Small with Transformers:
# Use a pipeline as a high-level helper # Warning: Pipeline type "translation" is no longer supported in transformers v5. # You must load the model directly (see below) or downgrade to v4.x with: # pip install "transformers<5.0.0" from transformers import pipeline pipe = pipeline("translation", model="aakashdandekar2006/Coli-ByT5-Small")# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForSeq2SeqLM tokenizer = AutoTokenizer.from_pretrained("aakashdandekar2006/Coli-ByT5-Small") model = AutoModelForSeq2SeqLM.from_pretrained("aakashdandekar2006/Coli-ByT5-Small", device_map="auto") - Notebooks
- Google Colab
- Kaggle
Coli-ByT5-Small
Byte-level Sequence-to-Sequence Model for Synonymous Codon Optimization in Escherichia coli
Model Description
- Developed by: Aakash Dandekar
- Model Type: Byte-level Encoder-Decoder Transformer (
T5ForConditionalGeneration/ByT5) - Base Model:
google/byt5-small - Precision:
float16 - Primary Task: Protein-to-DNA synonymous codon translation and optimization for Escherichia coli expression
Overview & Biological Motivation
Coli-ByT5-Small is a specialized byte-level sequence-to-sequence language model fine-tuned to solve the synonymous codon optimization task for the model bacterium Escherichia coli.
Given an amino acid sequence, the model generates an optimized synonymous nucleotide coding sequence (CDS). Unlike classical heuristic tools that rely on naive frequency maximization tables (which often eliminate natural translational pauses, disrupt co-translational folding, or destroy the 5' translational ramp), Coli-Byt5-Small learns contextual codon harmonization from validated endogenous sequences.
Why Byte-Level (ByT5)?
By operating directly on UTF-8 bytes rather than subword tokens:
- No Tokenizer Fragmentation: Biological sequences (amino acid letters
ACDEFGHIKLMNPQRSTVWYand nucleotidesATGC) are represented at single-character granularity without out-of-vocabulary (OOV) tokens. - Strict In-Frame Modeling: Enables fine-grained contextual awareness across codon triplets and dicodon boundaries without subword boundary mismatch.
Quickstart & Inference
Installation
pip install torch transformers sentencepiece
Direct Generation with Transformers
import torch
from transformers import AutoModelForSeq2SeqLM, AutoTokenizer
model_id = "aakashdandekar2006/Coli-Byt5-Small" # or local directory path
# Load byte-level tokenizer and model
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForSeq2SeqLM.from_pretrained(
model_id,
torch_dtype=torch.float16,
device_map="auto"
)
# Input amino acid sequence (e.g., GFP fragment / peptide sequence)
protein_seq = "MSKGEELFTGVVPILVELDGDVNGHKFSVSGEGEG"
prompt = f"Optimize codon sequence for Escherichia coli: {protein_seq}"
# Tokenize byte input
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
# Autoregressive generation
with torch.no_grad():
outputs = model.generate(
**inputs,
max_length=len(protein_seq) * 3 + 64,
num_beams=4,
early_stopping=True
)
optimized_dna = tokenizer.decode(outputs[0], skip_special_tokens=True)
print("Input Protein:", protein_seq)
print("Optimized DNA:", optimized_dna)
Hugging Face Pipeline Usage
from transformers import pipeline
pipe = pipeline(
"text2text-generation",
model="aakashdandekar2006/Coli-ByT5-Small",
torch_dtype="float16",
device_map="auto"
)
result = pipe("Optimize codon sequence for Escherichia coli: MSKGEELFTGVVPILVELDGDVNGHKFSVSGEGEG")
print(result[0]["generated_text"])
Training Details
- Dataset: Endogenous and curated Escherichia coli coding sequence pairs (CDS) sourced from
adibvafa/CodonTransformerand NCBI reference genomes. Filtered through biological schema verification:- 100% in-frame translation sequence identity (zero missense or nonsense substitutions)
- Strict IUPAC amino acid compliance (
ACDEFGHIKLMNPQRSTVWY) and canonical base compliance (A, C, G, T) - Exact nucleotide-to-protein length correspondence ($\text{length}(\text{DNA}) = 3 \times \text{length}(\text{Protein})$)
- Strict absence of premature internal stop codons
- Architecture: 12 encoder blocks, 4 decoder blocks, 6 attention heads, hidden dimension $d_{\text{model}} = 1472$, feed-forward dimension $d_{\text{ff}} = 3584$ (~300.77M total parameters).
- Adaptation Methodology: Parameter-Efficient Fine-Tuning via LoRA (
r=16,lora_alpha=32,target_modules="all-linear") usingpeftandSeq2SeqTrainer, subsequently merged into standalonesafetensorsfloat16 weights. - Hardware: 1x NVIDIA RTX GPU (CUDA-accelerated)
- Epochs / Steps: 3 Epochs
- Learning Rate:
5e-4with cosine learning rate scheduler and AdamW optimizer - Mixed Precision: FP16
Biological Verification & Evaluation Standards
| Criterion | Target Standard | Description |
|---|---|---|
| Translation Fidelity | 100% Sequence Identity | Generated nucleotide sequence must translate precisely to the input protein. |
| Codon Adaptation Index (CAI) | $0.65 - 0.85$ (Balanced) | Reflects endogenous E. coli K-12 codon frequency without unphysiological 1.0 maximization. |
| GC Equilibrium | $45% - 55%$ | Maintains host physiological GC range for mRNA stability. |
| Synthesis Safety | 0 Forbidden Sites | Suppresses unwanted restriction enzyme recognition motifs (EcoRI, BamHI, HindIII, XhoI, NdeI, NcoI). |
Benchmark Results
Comparative evaluation of Coli-ByT5-Small against a traditional frequency-maximizing greedy baseline on a test sequence of 539 aa (1617 nt expected):
| Metric | Coli-ByT5-Small (LLM Model) | Traditional Greedy |
|---|---|---|
| Translation Check | PASS | PASS |
| Generated Length (nt) | 1617 | 1617 |
| Codon Adaptation (CAI) | 0.884 | 1.000 |
| GC Content (%) | 54.6% | 52.6% |
| Restriction Sites | 0 | 1 |
Sequence translated successfully with zero amino acid mismatches.
Sequence Comparison (First 60 Nucleotides)
- Model:
ATGGAAGATGCAAAAAATATTAAAAAAGGTCCTGCACCATTTTATCCGCTGGAAGATGGC... - Naive Greedy:
ATGGAAGATGCGAAAAACATTAAAAAAGGCCCGGCGCCGTTTTATCCGCTGGAAGATGGC...
Analysis & Insights
- Translation Fidelity: The model achieved 100% translation accuracy across all 539 amino acids with zero missense or premature termination mutations.
- Balanced Codon Adaptation: Traditional greedy substitution achieves an artificial CAI of 1.000, which can exhaust specific tRNA pools and disrupt natural translational pause sites. Coli-ByT5-Small achieves a harmonized CAI of 0.884, matching natural high-expression transcripts.
- Synthesis Safety: Coli-ByT5-Small produced 0 forbidden restriction enzyme recognition sites, avoiding the cloning and synthesis motif conflict introduced by the naive algorithm (1 site).
Limitations & Ethical Considerations
- Host Specificity: This model is tuned specifically for Escherichia coli. Codon bias, tRNA pools, and translation kinetics differ substantially in other hosts (e.g., yeast, human, plant, insect expression systems).
- Secondary Structure & Experimental Verification: While the model learns contextual codon harmonization, synthetic constructs should be experimentally validated and analyzed for extreme RNA secondary structure (e.g., strong 5' hairpins) prior to wet-lab synthesis.
- Sequence Length: Optimized for standard protein domain lengths. For exceptionally large multi-kilobase proteins, generation length boundaries should be monitored.
- Biosafety & Dual-Use Research: This model must only be utilized for benign research and industrial protein expression. Users must comply with international biosecurity standards (e.g., IGSC gene synthesis screening guidelines) and refrain from optimizing sequences derived from dangerous pathogens or controlled toxins.
Citation
@misc{coli_byt5_small_2026,
author = {Dandekar, Aakash},
title = {Coli-ByT5-Small: Byte-Level Sequence-to-Sequence Model for Escherichia coli Codon Optimization},
year = {2026},
publisher = {Hugging Face},
howpublished = {\url{https://huggingface.co/aakashdandekar2006/Coli-ByT5-Small}}
}
- Downloads last month
- 59
Model tree for aakashdandekar2006/Coli-ByT5-Small
Base model
google/byt5-small