Update FastPLMs files from Synthyra/FastPLMs 32d9951 (PR 52: ESMFold2 noise cap, DPLM2 parity)

38fa0be verified 12 days ago

14.2 kB

---
library_name: transformers
license: "mit"
tags:
  - protein-language-model
  - fastplms
---
<!-- Generated from src/fastplms/models.toml. Do not edit. -->

# ESMFold2-600

## Quick start

Load the published model, fold two protein chains together, and write an mmCIF
file. This example uses 15 diffusion steps, matching the experimental config.

```python
import torch
from pathlib import Path
from transformers import AutoModel
model = AutoModel.from_pretrained(
    "Synthyra/ESMFold2-600",
    trust_remote_code=True,
    dtype=torch.float32,
    device_map="cuda",
    esmc_precision="bf16",
    attn_implementation="sdpa",
).eval()
model.set_chunk_size(32)
types = model.input_types
complex_input = types.StructurePredictionInput(
    sequences=[
        types.ProteinInput(id="A", sequence="MSTNPKPQRKTKRNT"),
        types.ProteinInput(id="B", sequence="MKTIIALSYIFCLVFA"),
    ]
)
with torch.inference_mode():
    result = model.fold(
        complex_input,
        num_loops=3,
        num_sampling_steps=15,
        num_diffusion_samples=1,
        seed=17,
        verbose=True,
    )
Path("complex.cif").write_text(model.result_to_cif(result), encoding="utf-8")
```

Set `verbose=False` to silence the folding progress display. This variant has a Synthyra-adapted native confidence head and returns pLDDT,
PAE, pTM, and iPTM fields. Confidence calculation is optional.

## Confidence training and evaluation

The packaged model includes the trained confidence head and enables it by default.

The confidence head completed 780 training updates in
18.1 hours on
[AtlasFold-Data](https://huggingface.co/datasets/lhallee/AtlasFold-Data). The backbone and folding
model stayed frozen, and evaluation used the final exponential moving-average
checkpoint. Training sampled from 475,969 eligible
structures, including monomers, dimers, and larger complexes.

The results below use 512 targets from the existing,
already-used test split, with 5 predictions per target,
3 recycling loops, and 50 diffusion steps.
Intervals are 95% bootstrap intervals over targets. Longer sequences were evaluated
separately and are not included in these tables.

| Model | Standard targets evaluated | Long targets evaluated |
| --- | ---: | ---: |
| ESMFold2-600 | 512 | 64 |
| Production ESMFold2 | 512 | 46 |

| Measurement | ESMFold2-600 | 95% interval |
| --- | ---: | ---: |
| pLDDT against all-atom lDDT, Spearman | 0.85185 | 0.81670 to 0.87953 |
| pTM against TM-score, Spearman | 0.86219 | 0.83103 to 0.88565 |
| ipTM against DockQ, Spearman | 0.84507 | 0.81196 to 0.87120 |
| Atom pLDDT mean absolute error | 0.07950 | 0.07625 to 0.08277 |
| Calibration error, 10 bins | 0.00567 | 0.00293 to 0.01027 |
| pLDDT cross-entropy | 2.68398 | 2.62732 to 2.73986 |
| PAE cross-entropy | 2.75040 | 2.69624 to 2.79981 |
| Within-target lDDT selection accuracy | 0.50962 | 0.41981 to 0.59490 |
| Within-target ipTM against DockQ selection accuracy | 0.50993 | 0.43450 to 0.58957 |
| Top-1 selection regret | 0.01946 | 0.01503 to 0.02496 |
| Random-choice regret | 0.02029 |  |
| Unresolved against resolved residue AUROC | 0.87270 | 0.85106 to 0.89298 |
| Resolved residue mean pLDDT | 0.79139 | 0.77773 to 0.80532 |
| Unresolved residue mean pLDDT | 0.50637 | 0.48717 to 0.52610 |
| Resolved residues below pLDDT 0.5 | 0.09616 | 0.07473 to 0.11890 |
| Unresolved residues below pLDDT 0.5 | 0.55747 | 0.50788 to 0.60834 |

Confidence scores, errors, accuracies, and fractions use a 0–1 scale.
Lower error, cross-entropy, and regret are better; higher correlation, ranking
accuracy, and AUROC are better. Ranking accuracy compares predictions of the
same target, with 0.5 representing chance. Regret is the quality lost by selecting
a prediction instead of the best available one. Unresolved residues are a proxy
for disorder, not definitive disorder labels.

| Agreement with production ESMFold2 | Spearman correlation | Mean difference |
| --- | ---: | ---: |
| Mean pLDDT | 0.71212 | -0.05294 |
| pTM | 0.78433 | -0.01344 |
| ipTM | 0.68299 | +0.02579 |

Production agreement compares each model's average confidence per target:
512 targets for pLDDT and pTM, and
320 multichain targets for ipTM. Differences are
ESMFold2-600 minus production. Each model predicts its own structures, so these
correlations do not measure folding accuracy or scores of identical structures.

See the [FastPLMs confidence training guide](https://github.com/Synthyra/FastPLMs/blob/main/docs/confidence_training.md)
for the training recipe, evaluation methods, and detailed records.

## Model overview

`Synthyra/ESMFold2-600` packages the
`biohub/ESMFold2-Experimental-Fast-base600M-step1500k` checkpoint with the
FastPLMs runtime and a Synthyra-adapted native confidence head for Hugging Face
Transformers. It accepts raw amino-acid sequences or typed molecular-complex
specifications; low-level forward accepts prepared feature tensors.

The repository uses the standard Transformers loading interface with
`trust_remote_code=True`. See Technical details for each registered class and
whether its weights come from the checkpoint.

The sequence- and token-classification classes reuse the pretrained backbone,
but their task heads are newly initialized. Fine-tune those heads before
interpreting their logits as predictions.

## Install and platform requirements

Install the direct dependencies published with this model:

```bash
python -m pip install -r \
  "https://huggingface.co/Synthyra/ESMFold2-600/resolve/main/requirements.txt"
```

The FastPLMs implementation itself is embedded in the model repository.
Transformers loads it through `trust_remote_code=True`.

This model requires Python 3.12-3.14, PyTorch 2.14, and Transformers 5.17.

The artifact requirements include the structure dependencies.

Validation runs in Docker on any compatible CUDA device. Record the container,
hardware, precision, and inputs; no GPU product or workstation is required.

The Hub quick start needs network access for the first download. For an
air-gapped run, build the manifest-pinned local artifact first and use the
offline example.

## Attention backends

The quick start uses `sdpa`.

Available backends are `eager`, `sdpa`, `flex_attention`. Requesting an
unavailable backend raises instead of silently changing implementation.

`output_attentions=True` can use the documented one-call eager fallback to
materialize attention tensors. The configured backend does not change.

## Downstream prediction

The sequence and token prediction AutoClasses use the checkpoint backbone and
create a new, untrained `classifier`. Sequence labels have shape `(b,)`.
Residue labels have shape `(b, l)` and use `-100` outside biological positions.
The folding trunk is skipped. The classifier uses the checkpoint's learned pLM
state mixture and projection, followed by one trainable transformer probe.

```python
import torch
from transformers import (
    AutoModelForSequenceClassification,
    AutoModelForTokenClassification,
)
model_id = "Synthyra/ESMFold2-600"
sequence_model = AutoModelForSequenceClassification.from_pretrained(
    model_id, num_labels=2, trust_remote_code=True
).eval()
token_model = AutoModelForTokenClassification.from_pretrained(
    model_id, num_labels=3, trust_remote_code=True
).eval()
sequences = ["MSTNPKPQRKTKRNT", "MKTIIALSYIFCLVFA"]
batch = sequence_model.prepare_classifier_inputs(sequences)
biological = batch["attention_mask"].bool()  # (b, l)
sequence_labels = torch.zeros(len(sequences), dtype=torch.long)  # (b,)
token_labels = torch.full_like(batch["input_ids"], -100)  # (b, l)
token_labels[biological] = 0  # selected biological positions; labels stay (b, l)
with torch.inference_mode():
    sequence_output = sequence_model(**batch, labels=sequence_labels)
    token_output = token_model(**batch, labels=token_labels)
print(sequence_output.logits.shape)  # (b, 2)
print(token_output.logits.shape)     # (b, l, 3)
```

## PEFT fine-tuning

Install the training dependencies. Then attach LoRA to the loaded checkpoint:

```bash
python -m pip install "datasets>=5.0" "peft>=0.21"
```

```python
from peft import LoraConfig, TaskType, get_peft_model
peft_model = get_peft_model(
    sequence_model,
    LoraConfig(
        task_type=TaskType.SEQ_CLS,
        r=8,
        lora_alpha=16,
        target_modules="all-linear",
        modules_to_save=["classifier"],
    ),
)
```

This checkpoint advertises a classification head. Save the separately trained
`classifier` with the adapter.
All FastPLMs checkpoints follow the Transformers `PreTrainedModel` contract and
can use PEFT. The ESM2-specific shipped CLI is an example, not a
support boundary. Record the target modules, base revision, data identity, and
trainable parameter scope.

## Protein folding

This experimental Fast checkpoint has 24 folding blocks and uses the frozen
`Synthyra/ESMplusplus_large` backbone. The config-declared step-1500000 backbone and the
pinned ESM++ weights are tensor-exact in BF16 after layout conversion.

```python
import torch
model = model.cuda().eval()
with torch.inference_mode():
    output = model.infer_protein(
        "MQYKLILNGKTLKGETTTEAVDAATAEKVFKQYANDNGVDGEWTYDDATKTFTVTE",
        seed=17,
        num_diffusion_samples=1,
    )
print(output.sample_atom_coords.shape)
```

Folding parameters remain FP32 with CUDA BF16 autocast. The backbone uses
BF16; FP8 requests fail. The 15-step sampler and three folding loops remain
the checkpoint defaults. Protein inputs require `msa=None`.
This checkpoint was trained without MSA conditioning. It rejects
`ProteinInput.msa` and MSA-derived features. Typed multichain and multimolecule
inputs remain supported without MSA conditioning.

The Synthyra-adapted native confidence head returns pLDDT, PAE, pTM, and iPTM. Confidence calculation is optional.
The 300 and 600 suffixes describe backbone scale, not total model parameters.

## Folding speed settings

Two runtime settings trade memory or exactness for speed on long proteins. They
need no extra package and no compilation, and neither is stored in the
configuration.

```python
model.set_chunk_size(None)            # unchunked pair updates
model.set_atom_attention("windowed")  # the official flash-attn atom window, through PyTorch
```

`set_chunk_size(None)` removes the row chunking of the pair-update blocks, which
costs most of a long fold's time on a data-center GPU and saves little peak
memory; pass a chunk such as 512 when the unchunked fold does not fit.
`set_atom_attention("windowed")` restricts each atom to 64 real neighbors on
each side, as the official model does when flash-attn is installed. It needs
CUDA and changes numerical output, within sampling spread on the measured
panel. The
[ESMFold2 guide](https://github.com/Synthyra/FastPLMs/blob/main/docs/esmfold2.md#measured-folding-cost)
records the conditions, the dense-versus-windowed comparison, and the figure.

| Residues | FastPLMs defaults (s) | Optimized (s) |
| ---: | ---: | ---: |
| 256 | 1.1 | 1.0 |
| 1,024 | 35 | 10 |
| 2,048 | not measured | 40 |

Measured on one NVIDIA H100 80GB HBM3 with PyTorch 2.13.0+cu130: one
fixed pseudo-random protein per length, 3 trunk loops,
50 requested sampling steps under the official noise cap,
1 diffusion sample, BF16 autocast over FP32 folding
parameters, median of end-to-end folds. "Defaults" changes no setting.

## Learned representation and ESMC precision

The learned projection maps `H: (b, l, 37, 1152) -> Z: (b, l, 256)`.
`embed_dataset` returns one `(l, 256)` residue representation per sequence.
The experimental architecture does not expose folding TTT.

## Notes and limitations

Experimental Fast model with a frozen 600M ESM++ backbone, 24 folding blocks,
no MSA conditioning, and a Synthyra-trained confidence head enabled by default.
BF16 execution uses FP32 folding parameters with CUDA autocast; FP8 is
unsupported. Confidence evaluation does not establish full structure-model
equivalence to production ESMFold2.

## Technical details

- Inputs: Raw amino-acid sequences or typed molecular-complex specifications; low-level forward accepts prepared feature tensors
- Transformers classes: `AutoConfig`, `AutoModel`, `AutoModelForSequenceClassification`, `AutoModelForTokenClassification`
- Checkpoint weights: `AutoConfig` = `FastPLMs extension`, `AutoModel` = `pretrained`, `AutoModelForSequenceClassification` = `base weights + untrained task head`, `AutoModelForTokenClassification` = `base weights + untrained task head`
- Attention backends: `eager`, `sdpa`, `flex_attention`
- Precision: `auto`, `fp32`, `bf16`
- BF16 execution: `fp32_parameters_autocast`
- Generation contract: `not_applicable`
- Dependencies: `core + structure`
- Weight publication allowed: `true`
- Weight license status: `resolved`
- Redistributable: `true`
- Complete weight publication required: `false`

## Validation and sources

FastPLMs pins the checkpoint, upstream source revisions, state transformation,
and required files in `models.toml`. Built artifacts record exact source
identities and conversion details in `source-record.json`.

- FastPLMs checkpoint: `Synthyra/ESMFold2-600`
- Runtime revision: recorded separately in the built artifact and published commit
- Runtime source identities: recorded in `source-record.json`
- Official checkpoint: `biohub/ESMFold2-Experimental-Fast-base600M-step1500k`
- Artifact source: `fast`
- State transform: `identity`
- Pinned upstreams: `biohub-esm`, `biohub-transformers`, `protein-ttt`
- Release tiers: `check`, `compliance`, `structure`, `feature`, `artifact`, `benchmark`
- Unresolved required file identities: `0`

The confidence evaluation above uses the existing test split. It does not
establish full structure-model equivalence.

Declared tiers compare configuration, tokenizer behavior, state, and
representative inference with the pinned reference. A nonzero unresolved count
blocks release. Metadata alone does not show that a build passed, that a backend
is faster, or that an output is biologically valid.

## License

Checkpoint terms: MIT. The Hub model-card identifier is
`mit`. The local artifact contains applicable source
licenses, notices, attribution, and conversion records. Review them before use.

	---
	library_name: transformers
	license: "mit"
	tags:
	- protein-language-model
	- fastplms
	---

	<!-- Generated from src/fastplms/models.toml. Do not edit. -->

	# ESMFold2-600

	## Quick start

	Load the published model, fold two protein chains together, and write an mmCIF
	file. This example uses 15 diffusion steps, matching the experimental config.

	```python
	import torch

	from pathlib import Path
	from transformers import AutoModel


	model = AutoModel.from_pretrained(
	"Synthyra/ESMFold2-600",
	trust_remote_code=True,
	dtype=torch.float32,
	device_map="cuda",
	esmc_precision="bf16",
	attn_implementation="sdpa",
	).eval()
	model.set_chunk_size(32)

	types = model.input_types
	complex_input = types.StructurePredictionInput(
	sequences=[
	types.ProteinInput(id="A", sequence="MSTNPKPQRKTKRNT"),
	types.ProteinInput(id="B", sequence="MKTIIALSYIFCLVFA"),
	]
	)
	with torch.inference_mode():
	result = model.fold(
	complex_input,
	num_loops=3,
	num_sampling_steps=15,
	num_diffusion_samples=1,
	seed=17,
	verbose=True,
	)
	Path("complex.cif").write_text(model.result_to_cif(result), encoding="utf-8")
	```

	Set `verbose=False` to silence the folding progress display. This variant has a Synthyra-adapted native confidence head and returns pLDDT,
	PAE, pTM, and iPTM fields. Confidence calculation is optional.

	## Confidence training and evaluation

	The packaged model includes the trained confidence head and enables it by default.

	The confidence head completed 780 training updates in
	18.1 hours on
	[AtlasFold-Data](https://huggingface.co/datasets/lhallee/AtlasFold-Data). The backbone and folding
	model stayed frozen, and evaluation used the final exponential moving-average
	checkpoint. Training sampled from 475,969 eligible
	structures, including monomers, dimers, and larger complexes.

	The results below use 512 targets from the existing,
	already-used test split, with 5 predictions per target,
	3 recycling loops, and 50 diffusion steps.
	Intervals are 95% bootstrap intervals over targets. Longer sequences were evaluated
	separately and are not included in these tables.

	\| Model \| Standard targets evaluated \| Long targets evaluated \|
	\| --- \| ---: \| ---: \|
	\| ESMFold2-600 \| 512 \| 64 \|
	\| Production ESMFold2 \| 512 \| 46 \|

	\| Measurement \| ESMFold2-600 \| 95% interval \|
	\| --- \| ---: \| ---: \|
	\| pLDDT against all-atom lDDT, Spearman \| 0.85185 \| 0.81670 to 0.87953 \|
	\| pTM against TM-score, Spearman \| 0.86219 \| 0.83103 to 0.88565 \|
	\| ipTM against DockQ, Spearman \| 0.84507 \| 0.81196 to 0.87120 \|
	\| Atom pLDDT mean absolute error \| 0.07950 \| 0.07625 to 0.08277 \|
	\| Calibration error, 10 bins \| 0.00567 \| 0.00293 to 0.01027 \|
	\| pLDDT cross-entropy \| 2.68398 \| 2.62732 to 2.73986 \|
	\| PAE cross-entropy \| 2.75040 \| 2.69624 to 2.79981 \|
	\| Within-target lDDT selection accuracy \| 0.50962 \| 0.41981 to 0.59490 \|
	\| Within-target ipTM against DockQ selection accuracy \| 0.50993 \| 0.43450 to 0.58957 \|
	\| Top-1 selection regret \| 0.01946 \| 0.01503 to 0.02496 \|
	\| Random-choice regret \| 0.02029 \| \|
	\| Unresolved against resolved residue AUROC \| 0.87270 \| 0.85106 to 0.89298 \|
	\| Resolved residue mean pLDDT \| 0.79139 \| 0.77773 to 0.80532 \|
	\| Unresolved residue mean pLDDT \| 0.50637 \| 0.48717 to 0.52610 \|
	\| Resolved residues below pLDDT 0.5 \| 0.09616 \| 0.07473 to 0.11890 \|
	\| Unresolved residues below pLDDT 0.5 \| 0.55747 \| 0.50788 to 0.60834 \|

	Confidence scores, errors, accuracies, and fractions use a 0–1 scale.
	Lower error, cross-entropy, and regret are better; higher correlation, ranking
	accuracy, and AUROC are better. Ranking accuracy compares predictions of the
	same target, with 0.5 representing chance. Regret is the quality lost by selecting
	a prediction instead of the best available one. Unresolved residues are a proxy
	for disorder, not definitive disorder labels.

	\| Agreement with production ESMFold2 \| Spearman correlation \| Mean difference \|
	\| --- \| ---: \| ---: \|
	\| Mean pLDDT \| 0.71212 \| -0.05294 \|
	\| pTM \| 0.78433 \| -0.01344 \|
	\| ipTM \| 0.68299 \| +0.02579 \|

	Production agreement compares each model's average confidence per target:
	512 targets for pLDDT and pTM, and
	320 multichain targets for ipTM. Differences are
	ESMFold2-600 minus production. Each model predicts its own structures, so these
	correlations do not measure folding accuracy or scores of identical structures.

	See the [FastPLMs confidence training guide](https://github.com/Synthyra/FastPLMs/blob/main/docs/confidence_training.md)
	for the training recipe, evaluation methods, and detailed records.

	## Model overview

	`Synthyra/ESMFold2-600` packages the
	`biohub/ESMFold2-Experimental-Fast-base600M-step1500k` checkpoint with the
	FastPLMs runtime and a Synthyra-adapted native confidence head for Hugging Face
	Transformers. It accepts raw amino-acid sequences or typed molecular-complex
	specifications; low-level forward accepts prepared feature tensors.

	The repository uses the standard Transformers loading interface with
	`trust_remote_code=True`. See Technical details for each registered class and
	whether its weights come from the checkpoint.

	The sequence- and token-classification classes reuse the pretrained backbone,
	but their task heads are newly initialized. Fine-tune those heads before
	interpreting their logits as predictions.

	## Install and platform requirements

	Install the direct dependencies published with this model:

	```bash
	python -m pip install -r \
	"https://huggingface.co/Synthyra/ESMFold2-600/resolve/main/requirements.txt"
	```

	The FastPLMs implementation itself is embedded in the model repository.
	Transformers loads it through `trust_remote_code=True`.

	This model requires Python 3.12-3.14, PyTorch 2.14, and Transformers 5.17.

	The artifact requirements include the structure dependencies.

	Validation runs in Docker on any compatible CUDA device. Record the container,
	hardware, precision, and inputs; no GPU product or workstation is required.

	The Hub quick start needs network access for the first download. For an
	air-gapped run, build the manifest-pinned local artifact first and use the
	offline example.

	## Attention backends

	The quick start uses `sdpa`.

	Available backends are `eager`, `sdpa`, `flex_attention`. Requesting an
	unavailable backend raises instead of silently changing implementation.

	`output_attentions=True` can use the documented one-call eager fallback to
	materialize attention tensors. The configured backend does not change.

	## Downstream prediction

	The sequence and token prediction AutoClasses use the checkpoint backbone and
	create a new, untrained `classifier`. Sequence labels have shape `(b,)`.
	Residue labels have shape `(b, l)` and use `-100` outside biological positions.
	The folding trunk is skipped. The classifier uses the checkpoint's learned pLM
	state mixture and projection, followed by one trainable transformer probe.

	```python
	import torch

	from transformers import (
	AutoModelForSequenceClassification,
	AutoModelForTokenClassification,
	)


	model_id = "Synthyra/ESMFold2-600"
	sequence_model = AutoModelForSequenceClassification.from_pretrained(
	model_id, num_labels=2, trust_remote_code=True
	).eval()
	token_model = AutoModelForTokenClassification.from_pretrained(
	model_id, num_labels=3, trust_remote_code=True
	).eval()
	sequences = ["MSTNPKPQRKTKRNT", "MKTIIALSYIFCLVFA"]
	batch = sequence_model.prepare_classifier_inputs(sequences)
	biological = batch["attention_mask"].bool() # (b, l)

	sequence_labels = torch.zeros(len(sequences), dtype=torch.long) # (b,)
	token_labels = torch.full_like(batch["input_ids"], -100) # (b, l)
	token_labels[biological] = 0 # selected biological positions; labels stay (b, l)

	with torch.inference_mode():
	sequence_output = sequence_model(**batch, labels=sequence_labels)
	token_output = token_model(**batch, labels=token_labels)
	print(sequence_output.logits.shape) # (b, 2)
	print(token_output.logits.shape) # (b, l, 3)
	```

	## PEFT fine-tuning

	Install the training dependencies. Then attach LoRA to the loaded checkpoint:

	```bash
	python -m pip install "datasets>=5.0" "peft>=0.21"
	```

	```python
	from peft import LoraConfig, TaskType, get_peft_model


	peft_model = get_peft_model(
	sequence_model,
	LoraConfig(
	task_type=TaskType.SEQ_CLS,
	r=8,
	lora_alpha=16,
	target_modules="all-linear",
	modules_to_save=["classifier"],
	),
	)
	```

	This checkpoint advertises a classification head. Save the separately trained
	`classifier` with the adapter.
	All FastPLMs checkpoints follow the Transformers `PreTrainedModel` contract and
	can use PEFT. The ESM2-specific shipped CLI is an example, not a
	support boundary. Record the target modules, base revision, data identity, and
	trainable parameter scope.

	## Protein folding

	This experimental Fast checkpoint has 24 folding blocks and uses the frozen
	`Synthyra/ESMplusplus_large` backbone. The config-declared step-1500000 backbone and the
	pinned ESM++ weights are tensor-exact in BF16 after layout conversion.

	```python
	import torch


	model = model.cuda().eval()
	with torch.inference_mode():
	output = model.infer_protein(
	"MQYKLILNGKTLKGETTTEAVDAATAEKVFKQYANDNGVDGEWTYDDATKTFTVTE",
	seed=17,
	num_diffusion_samples=1,
	)
	print(output.sample_atom_coords.shape)
	```

	Folding parameters remain FP32 with CUDA BF16 autocast. The backbone uses
	BF16; FP8 requests fail. The 15-step sampler and three folding loops remain
	the checkpoint defaults. Protein inputs require `msa=None`.
	This checkpoint was trained without MSA conditioning. It rejects
	`ProteinInput.msa` and MSA-derived features. Typed multichain and multimolecule
	inputs remain supported without MSA conditioning.

	The Synthyra-adapted native confidence head returns pLDDT, PAE, pTM, and iPTM. Confidence calculation is optional.
	The 300 and 600 suffixes describe backbone scale, not total model parameters.

	## Folding speed settings

	Two runtime settings trade memory or exactness for speed on long proteins. They
	need no extra package and no compilation, and neither is stored in the
	configuration.

	```python
	model.set_chunk_size(None) # unchunked pair updates
	model.set_atom_attention("windowed") # the official flash-attn atom window, through PyTorch
	```

	`set_chunk_size(None)` removes the row chunking of the pair-update blocks, which
	costs most of a long fold's time on a data-center GPU and saves little peak
	memory; pass a chunk such as 512 when the unchunked fold does not fit.
	`set_atom_attention("windowed")` restricts each atom to 64 real neighbors on
	each side, as the official model does when flash-attn is installed. It needs
	CUDA and changes numerical output, within sampling spread on the measured
	panel. The
	[ESMFold2 guide](https://github.com/Synthyra/FastPLMs/blob/main/docs/esmfold2.md#measured-folding-cost)
	records the conditions, the dense-versus-windowed comparison, and the figure.

	\| Residues \| FastPLMs defaults (s) \| Optimized (s) \|
	\| ---: \| ---: \| ---: \|
	\| 256 \| 1.1 \| 1.0 \|
	\| 1,024 \| 35 \| 10 \|
	\| 2,048 \| not measured \| 40 \|

	Measured on one NVIDIA H100 80GB HBM3 with PyTorch 2.13.0+cu130: one
	fixed pseudo-random protein per length, 3 trunk loops,
	50 requested sampling steps under the official noise cap,
	1 diffusion sample, BF16 autocast over FP32 folding
	parameters, median of end-to-end folds. "Defaults" changes no setting.

	## Learned representation and ESMC precision

	The learned projection maps `H: (b, l, 37, 1152) -> Z: (b, l, 256)`.
	`embed_dataset` returns one `(l, 256)` residue representation per sequence.
	The experimental architecture does not expose folding TTT.

	## Notes and limitations

	Experimental Fast model with a frozen 600M ESM++ backbone, 24 folding blocks,
	no MSA conditioning, and a Synthyra-trained confidence head enabled by default.
	BF16 execution uses FP32 folding parameters with CUDA autocast; FP8 is
	unsupported. Confidence evaluation does not establish full structure-model
	equivalence to production ESMFold2.

	## Technical details

	- Inputs: Raw amino-acid sequences or typed molecular-complex specifications; low-level forward accepts prepared feature tensors
	- Transformers classes: `AutoConfig`, `AutoModel`, `AutoModelForSequenceClassification`, `AutoModelForTokenClassification`
	- Checkpoint weights: `AutoConfig` = `FastPLMs extension`, `AutoModel` = `pretrained`, `AutoModelForSequenceClassification` = `base weights + untrained task head`, `AutoModelForTokenClassification` = `base weights + untrained task head`
	- Attention backends: `eager`, `sdpa`, `flex_attention`
	- Precision: `auto`, `fp32`, `bf16`
	- BF16 execution: `fp32_parameters_autocast`
	- Generation contract: `not_applicable`
	- Dependencies: `core + structure`
	- Weight publication allowed: `true`
	- Weight license status: `resolved`
	- Redistributable: `true`
	- Complete weight publication required: `false`

	## Validation and sources

	FastPLMs pins the checkpoint, upstream source revisions, state transformation,
	and required files in `models.toml`. Built artifacts record exact source
	identities and conversion details in `source-record.json`.

	- FastPLMs checkpoint: `Synthyra/ESMFold2-600`
	- Runtime revision: recorded separately in the built artifact and published commit
	- Runtime source identities: recorded in `source-record.json`
	- Official checkpoint: `biohub/ESMFold2-Experimental-Fast-base600M-step1500k`
	- Artifact source: `fast`
	- State transform: `identity`
	- Pinned upstreams: `biohub-esm`, `biohub-transformers`, `protein-ttt`
	- Release tiers: `check`, `compliance`, `structure`, `feature`, `artifact`, `benchmark`
	- Unresolved required file identities: `0`

	The confidence evaluation above uses the existing test split. It does not
	establish full structure-model equivalence.

	Declared tiers compare configuration, tokenizer behavior, state, and
	representative inference with the pinned reference. A nonzero unresolved count
	blocks release. Metadata alone does not show that a build passed, that a backend
	is faster, or that an output is biologically valid.

	## License

	Checkpoint terms: MIT. The Hub model-card identifier is
	`mit`. The local artifact contains applicable source
	licenses, notices, attribution, and conversion records. Review them before use.