Calpha
Toward native biomolecular intelligence in a general-purpose language model. Technical Report I: Protein Folding.
Homepage Β· Code Β· Technical report (coming soon)
Calpha is a chat model that folds proteins. A protein sequence enters Qwen3.5-9B inside an
ordinary chat turn, together with per-residue features from frozen ESM2-3B. Asked for a
structure, the model answers with a <structure> token, and its hidden states from nine layers
at the residue positions pass through a small conditioning front-end into the frozen
SimpleFold-3B flow-matching head, which generates all-atom coordinates.
This repository holds the weights that reproduce the numbers of the report, the raw benchmark outputs, and the training data.
Files
| Path | Content | Size |
|---|---|---|
llm/ |
Qwen3.5-9B after stage-1 training (bf16), tokenizer with amino-acid tokens and <structure> |
18 GB |
esm_adapter.safetensors |
adapter that adds ESM2-3B features to the amino-acid token embeddings (27M parameters) | 109 MB |
frontend.safetensors |
conditioning front-end of the structure head (8.4M parameters; stage-2 EMA, step 30,000) | 34 MB |
calpha_config.json |
layers read, canonical request, sampling settings | |
benchmark_outputs/ |
alpha-carbon predictions of every system in the report on the three test sets, raw text of the chat models | 54 MB |
training_data/ |
structure data and language data used for training (see its README) | 77 GB |
Not included, downloaded automatically from their original sources: the ESM2-3B encoder (fair-esm) and the SimpleFold-3B head (apple/ml-simplefold). Every weight of the head other than the front-end is the public SimpleFold-3B checkpoint, unchanged.
Use
git clone https://github.com/yliust/Calpha.git && cd Calpha
pip install -e .
pip install git+https://github.com/apple/ml-simplefold.git@94e2cd704cf7b86ef8e7cb27a1e77154c4e35b34
calpha fold MKTAYIAKQRQISFVKSHFSRQLEERLGLIEVQAPILSRVGDGTQDNLSGAEKAVQVKVKALPDAQFEVV
from calpha import Calpha
model = Calpha.from_pretrained("liuyong9975/Calpha")
out = model.fold("MKTAYIAKQRQISFVKSHFSRQLEERLGLIEVQ", out_dir="predictions")
One GPU with about 48 GB of memory. llm/ is a standard transformers checkpoint and can be
loaded on its own as a chat model; without the adapter and the code in the GitHub repository it
does not see ESM2 features and cannot produce structures.
Results
TM-score on alpha carbons, one sample per target, failures counted as zero, all systems scored by the same program.
| Model | FoldBench-334 | CASP16 | CAMEO-2025 |
|---|---|---|---|
| Calpha | 0.807 | 0.635 | 0.900 |
| Head-only control (ESM2 β SimpleFold-3B, no language model) | 0.822 | 0.682 | 0.917 |
| ESMFold | 0.869 | 0.762 | 0.905 |
| SimpleFold-3B | 0.842 | 0.704 | 0.938 |
| ESM3-open | 0.787 | 0.643 | 0.852 |
| DPLM-2 (650M) | 0.656 | 0.515 | 0.757 |
| Random walk (floor) | 0.140 | 0.139 | 0.136 |
Against the head-only control, no loss of accuracy is detected for chains of 100β300 residues on any set, and the loss is significant for chains of 300β600 residues on all three.
Training
- Stage 1. All of Qwen3.5-9B, the ESM2 adapter and a SimpleFold-100M head (initialised from its public weights) trained jointly for 2B tokens: language cross-entropy, trigger cross-entropy and the head's flow-matching loss. Language data: 75% protein tasks, 25% general English text and reasoning. Structure data: 84,658 PDB chains cropped to 256 residues.
- Stage 2. Language model frozen, SimpleFold-3B frozen, only the front-end trained: 20,000 steps at a crop of 256 residues, then 10,000 steps at 512 (effective batch 64, learning rate 1e-4, EMA 0.999), about 11 hours on eight H200 GPUs.
Limitations
- The language model does not make folding more accurate: Calpha is below the same head reading ESM2 directly on two of three sets and below ESMFold on FoldBench-334 and CASP16.
- Above 300 residues the language model costs accuracy on every set.
- The system depends on ESM2 and on the SimpleFold weights.
- MMLU-Pro accuracy fell from 0.774 to 0.703 during protein training.
- Single chains, single sequence, no ligands or complexes. Accuracy was measured on alpha carbons only, with one request phrasing; side chains, clashes and other phrasings were not evaluated.
- Not for clinical or diagnostic use.
License
llm/is a fine-tuned derivative of Qwen3.5-9B and is released under the Apache License 2.0.esm_adapter.safetensors,frontend.safetensors, the benchmark outputs and the code are released under the Apache License 2.0.- Producing a structure requires the SimpleFold-3B weights, which Apple releases under the Apple Machine Learning Research Model License (research use only). Calpha as a whole can therefore only be used under those terms.
- ESM2 is MIT-licensed. The training data has its own notes in
training_data/README.md.
Citation
@techreport{liu2026calpha,
title = {Calpha: Toward Native Biomolecular Intelligence in a General-Purpose Language Model.
Technical Report I: Protein Folding},
author = {Liu, Yong},
year = {2026},
url = {https://calpha-ai.com}
}