|
Download README.md from liuyong9975/Calpha: direct link, hf CLI and curl.
- Browser
- Download file 5.63 kB
-
https://huggingface.co/liuyong9975/Calpha/resolve/main/README.md
- Command line
-
hf download hf://liuyong9975/Calpha/README.md
-
curl -L -o README.md https://huggingface.co/liuyong9975/Calpha/resolve/main/README.md
5.63 kB
| license: other | |
| license_name: mixed-see-model-card | |
| license_link: https://github.com/yliust/Calpha/blob/main/NOTICE | |
| base_model: Qwen/Qwen3.5-9B | |
| language: | |
| - en | |
| tags: | |
| - biology | |
| - protein | |
| - protein-structure-prediction | |
| - protein-folding | |
| library_name: calpha | |
| # Calpha | |
| **Toward native biomolecular intelligence in a general-purpose language model. Technical Report I: Protein Folding.** | |
| [Homepage](https://calpha-ai.com) · [Code](https://github.com/yliust/Calpha) · Technical report (coming soon) | |
| Calpha is a chat model that folds proteins. A protein sequence enters Qwen3.5-9B inside an | |
| ordinary chat turn, together with per-residue features from frozen ESM2-3B. Asked for a | |
| structure, the model answers with a `<structure>` token, and its hidden states from nine layers | |
| at the residue positions pass through a small conditioning front-end into the frozen | |
| SimpleFold-3B flow-matching head, which generates all-atom coordinates. | |
| This repository holds the weights that reproduce the numbers of the report, the raw benchmark | |
| outputs, and the training data. | |
| ## Files | |
| | Path | Content | Size | | |
| |---|---|---| | |
| | `llm/` | Qwen3.5-9B after stage-1 training (bf16), tokenizer with amino-acid tokens and `<structure>` | 18 GB | | |
| | `esm_adapter.safetensors` | adapter that adds ESM2-3B features to the amino-acid token embeddings (27M parameters) | 109 MB | | |
| | `frontend.safetensors` | conditioning front-end of the structure head (8.4M parameters; stage-2 EMA, step 30,000) | 34 MB | | |
| | `calpha_config.json` | layers read, canonical request, sampling settings | | | |
| | `benchmark_outputs/` | alpha-carbon predictions of every system in the report on the three test sets, raw text of the chat models | 54 MB | | |
| | `training_data/` | structure data and language data used for training (see its README) | 77 GB | | |
| **Not included, downloaded automatically from their original sources:** the ESM2-3B encoder | |
| (fair-esm) and the SimpleFold-3B head ([apple/ml-simplefold](https://github.com/apple/ml-simplefold)). | |
| Every weight of the head other than the front-end is the public SimpleFold-3B checkpoint, | |
| unchanged. | |
| ## Use | |
| ```bash | |
| git clone https://github.com/yliust/Calpha.git && cd Calpha | |
| pip install -e . | |
| pip install git+https://github.com/apple/ml-simplefold.git@94e2cd704cf7b86ef8e7cb27a1e77154c4e35b34 | |
| calpha fold MKTAYIAKQRQISFVKSHFSRQLEERLGLIEVQAPILSRVGDGTQDNLSGAEKAVQVKVKALPDAQFEVV | |
| ``` | |
| ```python | |
| from calpha import Calpha | |
| model = Calpha.from_pretrained("liuyong9975/Calpha") | |
| out = model.fold("MKTAYIAKQRQISFVKSHFSRQLEERLGLIEVQ", out_dir="predictions") | |
| ``` | |
| One GPU with about 48 GB of memory. `llm/` is a standard `transformers` checkpoint and can be | |
| loaded on its own as a chat model; without the adapter and the code in the GitHub repository it | |
| does not see ESM2 features and cannot produce structures. | |
| ## Results | |
| TM-score on alpha carbons, one sample per target, failures counted as zero, all systems scored | |
| by the same program. | |
| | Model | FoldBench-334 | CASP16 | CAMEO-2025 | | |
| |---|---|---|---| | |
| | **Calpha** | **0.807** | **0.635** | **0.900** | | |
| | Head-only control (ESM2 → SimpleFold-3B, no language model) | 0.822 | 0.682 | 0.917 | | |
| | ESMFold | 0.869 | 0.762 | 0.905 | | |
| | SimpleFold-3B | 0.842 | 0.704 | 0.938 | | |
| | ESM3-open | 0.787 | 0.643 | 0.852 | | |
| | DPLM-2 (650M) | 0.656 | 0.515 | 0.757 | | |
| | Random walk (floor) | 0.140 | 0.139 | 0.136 | | |
| Against the head-only control, no loss of accuracy is detected for chains of 100–300 residues | |
| on any set, and the loss is significant for chains of 300–600 residues on all three. | |
| ## Training | |
| - **Stage 1.** All of Qwen3.5-9B, the ESM2 adapter and a SimpleFold-100M head (initialised from | |
| its public weights) trained jointly for 2B tokens: language cross-entropy, trigger | |
| cross-entropy and the head's flow-matching loss. Language data: 75% protein tasks, 25% general | |
| English text and reasoning. Structure data: 84,658 PDB chains cropped to 256 residues. | |
| - **Stage 2.** Language model frozen, SimpleFold-3B frozen, only the front-end trained: 20,000 | |
| steps at a crop of 256 residues, then 10,000 steps at 512 (effective batch 64, learning rate | |
| 1e-4, EMA 0.999), about 11 hours on eight H200 GPUs. | |
| ## Limitations | |
| - The language model does not make folding more accurate: Calpha is below the same head reading | |
| ESM2 directly on two of three sets and below ESMFold on FoldBench-334 and CASP16. | |
| - Above 300 residues the language model costs accuracy on every set. | |
| - The system depends on ESM2 and on the SimpleFold weights. | |
| - MMLU-Pro accuracy fell from 0.774 to 0.703 during protein training. | |
| - Single chains, single sequence, no ligands or complexes. Accuracy was measured on alpha | |
| carbons only, with one request phrasing; side chains, clashes and other phrasings were not | |
| evaluated. | |
| - Not for clinical or diagnostic use. | |
| ## License | |
| - `llm/` is a fine-tuned derivative of Qwen3.5-9B and is released under the Apache License 2.0. | |
| - `esm_adapter.safetensors`, `frontend.safetensors`, the benchmark outputs and the code are | |
| released under the Apache License 2.0. | |
| - Producing a structure requires the SimpleFold-3B weights, which Apple releases under the | |
| **Apple Machine Learning Research Model License (research use only)**. Calpha as a whole can | |
| therefore only be used under those terms. | |
| - ESM2 is MIT-licensed. The training data has its own notes in `training_data/README.md`. | |
| ## Citation | |
| ```bibtex | |
| @techreport{liu2026calpha, | |
| title = {Calpha: Toward Native Biomolecular Intelligence in a General-Purpose Language Model. | |
| Technical Report I: Protein Folding}, | |
| author = {Liu, Yong}, | |
| year = {2026}, | |
| url = {https://calpha-ai.com} | |
| } | |
| ``` | |