oteam/esmfold2-fast

ESMFold2

All-atom biomolecular structure prediction and protein binder design driven by ESMC representations, with optional protein MSA evidence for supported variants.

Disclaimer

This is an unofficial OFoldX implementation of ESMFold2, described in Language Modeling Materializes a World Model of Protein Biology. The upstream authors did not write, review, or endorse this implementation or model card.

Model Details

ESMFold2 predicts all-atom biomolecular structures from ESMC representations and optional protein MSA evidence. A frozen ESMC language model supplies sequence representations, a recurrent folding trunk updates token and pair features, and a diffusion head generates coordinates before distogram and confidence heads score the result. Experimental variants expose the differentiable inversion model used for protein binder sequence design.

Variants

• oteam/esmfold2: Full 48-block folding model with optional MSA conditioning.
• oteam/esmfold2-fast: Faster 24-block single-sequence folding model.
• oteam/esmfold2-experimental: Complete binder-design artifact containing the fixed 19 Model--Processor pairs used by the released protocol.
• oteam/esmfold2-experimental-fast: Experimental fast model used for binder sequence optimization.
• ESMFold2 experimental critics: Experimental-Fast-Cutoff2025, Experimental, Experimental-Cutoff2025, and the 15 released Fast-base{300M,600M,6B}-step{250,500,750,1000,1500}k scaling checkpoints.

Model Specification

Variant	Folding Blocks	MSA	ESMC	Single Size	Pair Size	Atom Size	Token Size	Recurrent Updates	Sampling Steps	Num Parameters (M)
ESMFold2	48	yes	6B	384	256	128	768	4	14	6586.242
ESMFold2 Fast	24	no	6B	384	256	128	768	4	14	6548.432
Experimental	48	yes	6B	384	256	128	768	4	14	6568.980
Experimental Fast	24	no	6B	384	256	128	768	4	14	6531.170

Links

• Code: ofoldx.models.esmfold2
• Paper: Language Modeling Materializes a World Model of Protein Biology
• Developed by: Salvatore Candido, Thomas Hayes, Alexander Derry, Roshan Rao, Zeming Lin, Robert Verkuil, Bryan Wu, Jin Sub Lee, Elise S. Bruguera, Jehan A. Keval, Mykhailo Kopylov, John E. Pak, Wesley Wu, Neil Thomas, Samson Mataraso, Alvin Hsu, Ashton C. Trotman-Grant, Kilian Fatras, Allan dos Santos Costa, Rohil Badkundri, Halil Akın, Deniz Oktay, Jonathan Deaton, Elizabeth Montabana, Hrishita Sitwala, Yue Yu, Marius Wiggert, Dylan Alexander Carlin, Anthony W. Goering, Tomasz Blazejewski, McCullen Sandora, Michael Hla, Tina Z. Jia, Leon H. Kloker, Nicholas J. Sofroniew, Masatoshi Uehara, Jassi Pannu, Sharrol Bachas, Daniel S. Liu, Tom Sercu, and Alexander Rives
• Model type: ESMC-conditioned recurrent all-atom folding model with diffusion coordinate generation
• Original Repository: Biohub/esm

Feature and geometry contract

All public model coordinate boundaries consume an explicit finite numerical projection. Unavailable values are selected out under the declared mask before arithmetic or dtype conversion and have zero coordinate derivatives. Active nonfinite values are invalid requests; finite values outside the model dtype range are an unsupported numeric range. Normalization returns the same input carrier type without changing canonical membership, source masks, or source precision. Decode uses prepared atom correspondences. APIs without an availability mask require complete finite model fields.

The raw reference atom axis is unpadded and complete. A component whose slot atoms lack a usable reference conformer is embedded from its determined graph with the processor seed and gets one reference_embedded record. Generated model references do not alter canonical coordinate knownness.

A representative is a declared chosen-atom convention: protein CB, then CA, then the first admitted atom; nucleic C4 for purine classes or C2 for remaining known classes, then C1′, then the first atom; unknown nucleic classes start at C1′. Atom availability does not switch this choice. Prepared actual atom maps drive all later gathers and decoding, including components whose biological label disagrees with their actual atoms. The relation bit is symmetric positive-edge presence and discards source relation typing.

Binder design takes the target as one protein one-letter sequence; each target–binder complex is built from two fresh linear protein chains. A structure or a file path is not a design target.

Usage

The model depends on the ofoldx library. Install it with pip:

pip install ofoldx

Structure Prediction

Load the concrete folding Pipeline:

The standard and Fast folding repositories reference the same frozen MultiMolecule ESMC-6B checkpoint. Its exact revision is recorded in the model configuration and its weights are shared through the Hugging Face cache.

import ofoldx

pipeline = ofoldx.pipeline("fold", model="oteam/esmfold2")
output = pipeline.fold("MKTAYIAKQRQISFVKSHFSRQ")

Binder Design

The repository marker selects the experimental ESMFold2 design Pipeline. Its Experimental-Fast child lives at fast/. The other hero roles live at fast_cutoff2025/, experimental/, and experimental_cutoff2025/; the 15 scaling roles live at scaling/base{300m,600m,6b}_step{250k,500k,750k,1000k,1500k}/. Every slot is required even when a call skips scaling-critic computation. Each saved Model owns its complete frozen ESMC trunk and masked-language-model head, so loading performs no hidden checkpoint download.

import ofoldx

pipeline = ofoldx.pipeline("design", model="oteam/esmfold2-experimental")
output = pipeline.design("MKTAYIAKQRQISFVKSHFSRQ", num_binder_residues=80)

The default call performs 150 inversion steps, retains each candidate's best actual low-temperature iPTM prediction, evaluates four hero and 15 scaling critics, applies the released non-antibody pI filter, and stably selects at most 84 designs. use_scaling_critics=False skips only the 15 scaling evaluations; their score component becomes zero. The scaling checkpoints publish no confidence head, so their evaluations contain structures and distogram proxies; the four hero evaluations also contain confidence. output.candidates preserves both evaluation groups, while output.selected uses each selected candidate's coherent Experimental-Cutoff2025 structure and confidence values. When is_antibody is not specified, the binder prompt is detected using the released antibody numbering rule.

Execution Stages

The complete callable and named task verb execute the same operation. Folding accepts a structure, polymer sequences, MSA evidence and the sampling arguments:

fold_pipeline = ofoldx.pipeline("fold", model="oteam/esmfold2")
result = fold_pipeline("MKTAYIAKQRQISFVKSHFSRQ", seed=7)
prepared = fold_pipeline.prepare("MKTAYIAKQRQISFVKSHFSRQ")
output = fold_pipeline.forward(prepared.input, seed=7)
result = fold_pipeline.postprocess(prepared, output)

Folding preparation returns Prepared[EsmFold2Input]; the sampler output retains its native tensor axes, and postprocessing decodes every sample onto the prepared target (every atom regenerated in a new frame) and binds confidence to the prepared token, slot and chain axes. Experimental checkpoints without confidence return structure predictions with empty confidence mappings.

Binder design preparation returns EsmFold2DesignPipelineInput, containing the resolved protein target, binder prompt and optimization dimensions. Its forward stage performs differentiable inversion and the selected critic ensemble, returning EsmFold2DesignPipelineOutput. Postprocessing applies the released confidence combination, pI filter and stable selection. Folding uses inference mode; binder optimization preserves gradients through the soft sequence.

Inference critics execute their child's forward stage, so a compatible sampling override also applies inside the design composition. Hero roles return confidence-enabled outputs; scaling roles return sampling outputs without a confidence head. Their batched decoding remains separate from the single-system folding postprocess. Differentiable inversion uses the Model experts directly to preserve gradients through the soft sequence.

Each child in the design repository has its own Pipeline marker and can be loaded independently by its declared role. All children belong to the same resolved repository revision.

Training Details

Training Data

Biohub reports that ESMFold2 was trained on sequences from the Protein Data Bank (PDB) and AlphaFold DB (AFDB). The release model cards give a September 2021 data cutoff for ESMFold2 and ESMFold2-Fast. ESMC was pretrained separately on large protein-sequence corpora; consult the selected ESMC artifact's model card for its training data. OFoldX does not redistribute the upstream training corpora.

Training Procedure

The paper describes diffusion MSE, smooth-LDDT, polymer-ligand bond, distogram, and confidence losses for the folding system. The released source and model cards do not publish every target construction, mask, mini-rollout, batching rule, and reduction needed to reproduce head training exactly. OFoldX exposes the differentiable single-step denoising path but does not present it as a complete reproduction of the unpublished training pipeline.

Binder design is inference-time optimization rather than model training. It performs gradient descent over soft binder-sequence logits under the released distogram contact objective and ESMC pseudo-perplexity regularizer. Low-temperature coordinate samples are scored by actual iPTM, and each candidate retains its best observed sequence and structure.

Citation

@misc{candido2026language,
  title = {Language Modeling Materializes a World Model of Protein Biology},
  author = {Candido, Salvatore and Hayes, Thomas and Derry, Alexander and others},
  year = {2026},
  url = {https://biohub.ai/papers/esm_protein.pdf},
  note = {Preprint}
}

Contact

Please use OFoldX GitHub issues for questions about this implementation or model card. Questions about the original model should be directed to the Biohub authors.

License

This model implementation is licensed under the GNU Affero General Public License.

SPDX-License-Identifier: AGPL-3.0-or-later