README.md

metadata

library_name: transformers
license: mit
tags:
  - protein-language-model
  - fastplms

Synthyra/FastESMFold

This checkpoint packages the FastPLMs ESMFold implementation.

Accepted inputs are raw amino-acid sequences through folding helpers, or prepared residue tensors. Supported Transformers entry points are AutoConfig, AutoModel.

Capabilities

Feature	Status
Sequence classification	Unavailable: no advertised AutoClass
Token classification	Unavailable: no advertised AutoClass
PEFT fine-tuning	Supported pattern: attach LoRA to the pretrained model
Embeddings	Unavailable for this structure-only checkpoint
Test-time training	Unavailable: the checkpoint has no trained MLM head
Attention variants	Supported: eager, sdpa, flex_attention
Compliance	Declared: exact release evidence is required

A supported interface is not a pretrained downstream predictor. Classification heads start untrained, and declared compliance metadata is not a claim that an arbitrary local build passed its release gate.

Install and platform requirements

Install the direct dependencies published with this model:

python -m pip install -r \
  "https://huggingface.co/Synthyra/FastESMFold/resolve/main/requirements.txt"

The FastPLMs implementation itself is embedded in the model repository and loaded by Transformers through trust_remote_code=True.

Python 3.11-3.14, PyTorch 2.13, and Transformers 5.13 are required. The artifact requirements include the direct structure dependencies. The published execution contract requires a CUDA device. The current validated release target is the exact NVIDIA GH200 on Linux aarch64; Linux x86-64, CPU-only, Windows, and macOS structure runs are not current release evidence. The Hub quick start below requires network access on first download. For an air-gapped run, first build the manifest-pinned local artifact and use the offline form shown in the example.

Quick start

from transformers import AutoModel

model_id = "Synthyra/FastESMFold"
model = AutoModel.from_pretrained(
    model_id,
    trust_remote_code=True,
    attn_implementation="sdpa",
).eval()

For offline validation, replace model_id with the manifest-built dist/hub/FastESMFold path and pass local_files_only=True.

Attention and compliance

The quick start selects sdpa explicitly. Declared variants are eager, sdpa, flex_attention. An unavailable requested backend raises instead of silently switching implementations. output_attentions=True may use the documented, one-call eager fallback solely to materialize attention tensors; the configured backend remains unchanged.

This family declares the compliance tier. Release evidence binds the exact checkpoint, backend, dtype, hardware, inputs, and reference revision.

PEFT fine-tuning

Install the direct training dependencies, then attach LoRA to the loaded checkpoint:

python -m pip install "datasets>=4.8,<5" "peft>=0.19,<0.20"

from peft import LoraConfig, get_peft_model

peft_model = get_peft_model(
    model,
    LoraConfig(
        r=8,
        lora_alpha=16,
        target_modules="all-linear",
    ),
)

This checkpoint has no advertised classifier. Supply the task-specific objective and preserve any new head through modules_to_save. All FastPLMs checkpoints follow the Transformers PreTrainedModel contract and can be adapted with PEFT. The ESM2-specific shipped CLI is an example, not a support boundary. Record the target modules, base revision, data identity, and trainable parameter scope.

Protein structure prediction

ESMFold accepts a raw sequence and returns structure tensors and confidence:

import torch

model = model.cuda().eval()
with torch.inference_mode():
    output = model.infer(
        "MKTLLILAVVAAALA",
        num_recycles=4,
    )

print(output["mean_plddt"])

summary = model.fold_protein(
    "MKTLLILAVVAAALA",
    return_pdb_string=True,
)
with open("prediction.pdb", "w", encoding="utf-8") as handle:
    handle.write(summary["pdb_string"])
print(summary["plddt"], summary["ptm"])

FastPLMs does not expose ProteinTTT for ESMFold. The pinned folding checkpoint does not contain a trained masked-language-model head for that objective, so ttt() and TTT folding requests raise explicitly.

Runtime contract

• Public input: Raw amino-acid sequences through folding helpers, or prepared residue tensors
• Advertised AutoClasses: AutoConfig, AutoModel
• AutoClass weight status: AutoConfig = FastPLMs extension, AutoModel = pretrained
• Attention implementations: eager, sdpa, flex_attention
• Precision policies: default
• BF16 execution: fp32_parameters_autocast
• Generation contract: not_applicable
• Artifact dependency set: core + structure
• Weight publication allowed: true
• Weight license status: resolved
• Redistributable: true
• Complete weight publication required: false

Release record

• FastPLMs weights: Synthyra/FastESMFold
• Runtime revision: recorded separately in the built artifact and published commit
• Source-tree and runtime-bundle SHA-256: recorded in provenance.json
• Official checkpoint: facebook/esmfold_v1
• Artifact source: fast
• State transform: esmfold_meta_to_fastplms_v1
• Pinned upstreams: fair-esm, openfold
• Release tiers: check, compliance, structure, feature, artifact, benchmark
• Unresolved required file identities: 0

provenance.json records exact file identities, conversion, source revisions, legal texts, schema, and attestations. A nonzero unresolved count blocks release.

Validation boundary

Declared tiers compare applicable configuration, tokenizer behavior, state, and representative inference with the pinned reference. Metadata alone does not claim a build passed, a backend is faster, or an output is biologically valid.

License

Checkpoint terms: MIT. The Hub model-card identifier is mit. Applicable source licenses, notices, attribution, and conversion records are distributed with the local artifact. Review them before use.