Calpha

Toward native biomolecular intelligence in a general-purpose language model. Technical Report I: Protein Folding.

Homepage Β· Code Β· Technical report (coming soon)

Calpha is a chat model that folds proteins. A protein sequence enters Qwen3.5-9B inside an ordinary chat turn, together with per-residue features from frozen ESM2-3B. Asked for a structure, the model answers with a <structure> token, and its hidden states from nine layers at the residue positions pass through a small conditioning front-end into the frozen SimpleFold-3B flow-matching head, which generates all-atom coordinates.

This repository holds the weights that reproduce the numbers of the report, the raw benchmark outputs, and the training data.

Files

Path Content Size
llm/ Qwen3.5-9B after stage-1 training (bf16), tokenizer with amino-acid tokens and <structure> 18 GB
esm_adapter.safetensors adapter that adds ESM2-3B features to the amino-acid token embeddings (27M parameters) 109 MB
frontend.safetensors conditioning front-end of the structure head (8.4M parameters; stage-2 EMA, step 30,000) 34 MB
calpha_config.json layers read, canonical request, sampling settings
benchmark_outputs/ alpha-carbon predictions of every system in the report on the three test sets, raw text of the chat models 54 MB
training_data/ structure data and language data used for training (see its README) 77 GB

Not included, downloaded automatically from their original sources: the ESM2-3B encoder (fair-esm) and the SimpleFold-3B head (apple/ml-simplefold). Every weight of the head other than the front-end is the public SimpleFold-3B checkpoint, unchanged.

Use

git clone https://github.com/yliust/Calpha.git && cd Calpha
pip install -e .
pip install git+https://github.com/apple/ml-simplefold.git@94e2cd704cf7b86ef8e7cb27a1e77154c4e35b34
calpha fold MKTAYIAKQRQISFVKSHFSRQLEERLGLIEVQAPILSRVGDGTQDNLSGAEKAVQVKVKALPDAQFEVV
from calpha import Calpha
model = Calpha.from_pretrained("liuyong9975/Calpha")
out = model.fold("MKTAYIAKQRQISFVKSHFSRQLEERLGLIEVQ", out_dir="predictions")

One GPU with about 48 GB of memory. llm/ is a standard transformers checkpoint and can be loaded on its own as a chat model; without the adapter and the code in the GitHub repository it does not see ESM2 features and cannot produce structures.

Results

TM-score on alpha carbons, one sample per target, failures counted as zero, all systems scored by the same program.

Model FoldBench-334 CASP16 CAMEO-2025
Calpha 0.807 0.635 0.900
Head-only control (ESM2 β†’ SimpleFold-3B, no language model) 0.822 0.682 0.917
ESMFold 0.869 0.762 0.905
SimpleFold-3B 0.842 0.704 0.938
ESM3-open 0.787 0.643 0.852
DPLM-2 (650M) 0.656 0.515 0.757
Random walk (floor) 0.140 0.139 0.136

Against the head-only control, no loss of accuracy is detected for chains of 100–300 residues on any set, and the loss is significant for chains of 300–600 residues on all three.

Training

  • Stage 1. All of Qwen3.5-9B, the ESM2 adapter and a SimpleFold-100M head (initialised from its public weights) trained jointly for 2B tokens: language cross-entropy, trigger cross-entropy and the head's flow-matching loss. Language data: 75% protein tasks, 25% general English text and reasoning. Structure data: 84,658 PDB chains cropped to 256 residues.
  • Stage 2. Language model frozen, SimpleFold-3B frozen, only the front-end trained: 20,000 steps at a crop of 256 residues, then 10,000 steps at 512 (effective batch 64, learning rate 1e-4, EMA 0.999), about 11 hours on eight H200 GPUs.

Limitations

  • The language model does not make folding more accurate: Calpha is below the same head reading ESM2 directly on two of three sets and below ESMFold on FoldBench-334 and CASP16.
  • Above 300 residues the language model costs accuracy on every set.
  • The system depends on ESM2 and on the SimpleFold weights.
  • MMLU-Pro accuracy fell from 0.774 to 0.703 during protein training.
  • Single chains, single sequence, no ligands or complexes. Accuracy was measured on alpha carbons only, with one request phrasing; side chains, clashes and other phrasings were not evaluated.
  • Not for clinical or diagnostic use.

License

  • llm/ is a fine-tuned derivative of Qwen3.5-9B and is released under the Apache License 2.0.
  • esm_adapter.safetensors, frontend.safetensors, the benchmark outputs and the code are released under the Apache License 2.0.
  • Producing a structure requires the SimpleFold-3B weights, which Apple releases under the Apple Machine Learning Research Model License (research use only). Calpha as a whole can therefore only be used under those terms.
  • ESM2 is MIT-licensed. The training data has its own notes in training_data/README.md.

Citation

@techreport{liu2026calpha,
  title  = {Calpha: Toward Native Biomolecular Intelligence in a General-Purpose Language Model.
            Technical Report I: Protein Folding},
  author = {Liu, Yong},
  year   = {2026},
  url    = {https://calpha-ai.com}
}
Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
This model isn't deployed by any Inference Provider. πŸ™‹ Ask for provider support

Model tree for liuyong9975/Calpha

Finetuned
Qwen/Qwen3.5-9B
Finetuned
(1,410)
this model

Space using liuyong9975/Calpha 1