File size: 2,227 Bytes
b661793 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 | # Calpha training data
Two data sets, packed as split tar archives. Join the parts before unpacking:
```bash
cat structure/targets_pdb.tar.part-* | tar -xf -
cat structure/tokens_pdb.tar.part-* | tar -xf -
cat language/v5_vlm.tar.part-* | tar -xf -
sha256sum -c SHA256SUMS
```
## `structure/` — 84,658 PDB chains (stage 1 and stage 2)
Protein chains from the Protein Data Bank in the input format of
[SimpleFold](https://github.com/apple/ml-simplefold) (derived from Boltz).
- `targets_pdb` (12 GB): per-chain structure records (`structures/*.npz`, `records/*.json`) and
`manifest.json`. Built from 98,254 PDB chains.
- `tokens_pdb` (32 GB): the same chains tokenised and featurised for training, filtered to the
84,658 protein-only chains actually used (`manifest.json`; `manifest.all.json` lists all).
Both stages crop chains during training (256 residues; 512 in the last phase of stage 2).
Built with `training/data/simplefold_targets.py` of the code repository.
## `language/` — stage-1 language data (`v5_vlm`, 33 GB)
Pre-tokenised chat-format samples, five epochs of about 547M tokens each (2.74B tokens in
total; stage 1 consumed 2B). Per epoch, by tokens:
| Source | Share | Content |
|---|---|---|
| protein tasks | 75% | inverse folding, secondary structure, infilling, contact prediction, written with amino-acid and structure-related tokens; sequences and labels derived from PDB and AlphaFold DB structures |
| `fineweb_edu` | 17% | general English text from the FineWeb-Edu corpus |
| `mot_reasoning` | 8% | general reasoning traces |
Files: `train/epoch<k>-<shard>.{jsonl,tok.npy,idx.npy,meta.parquet}` (text, token ids, offsets,
per-sample metadata), `val/`, `tokenizer/` (Qwen3.5 tokenizer with the added tokens),
`summary.json`. Homologs of the FoldBench-334 targets were removed from the protein samples.
## Sources and terms
The structure data and the protein tasks derive from the Protein Data Bank (CC0) and the
AlphaFold Protein Structure Database (CC BY 4.0). The general-text portion is redistributed
from third-party corpora in tokenised and text form and remains under the terms of those
corpora; if you reuse it, follow their licenses. The data is provided for research.
|