File size: 2,227 Bytes
b661793
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
# Calpha training data

Two data sets, packed as split tar archives. Join the parts before unpacking:

```bash
cat structure/targets_pdb.tar.part-* | tar -xf -
cat structure/tokens_pdb.tar.part-*  | tar -xf -
cat language/v5_vlm.tar.part-*       | tar -xf -
sha256sum -c SHA256SUMS
```

## `structure/` — 84,658 PDB chains (stage 1 and stage 2)

Protein chains from the Protein Data Bank in the input format of
[SimpleFold](https://github.com/apple/ml-simplefold) (derived from Boltz).

- `targets_pdb` (12 GB): per-chain structure records (`structures/*.npz`, `records/*.json`) and
  `manifest.json`. Built from 98,254 PDB chains.
- `tokens_pdb` (32 GB): the same chains tokenised and featurised for training, filtered to the
  84,658 protein-only chains actually used (`manifest.json`; `manifest.all.json` lists all).

Both stages crop chains during training (256 residues; 512 in the last phase of stage 2).
Built with `training/data/simplefold_targets.py` of the code repository.

## `language/` — stage-1 language data (`v5_vlm`, 33 GB)

Pre-tokenised chat-format samples, five epochs of about 547M tokens each (2.74B tokens in
total; stage 1 consumed 2B). Per epoch, by tokens:

| Source | Share | Content |
|---|---|---|
| protein tasks | 75% | inverse folding, secondary structure, infilling, contact prediction, written with amino-acid and structure-related tokens; sequences and labels derived from PDB and AlphaFold DB structures |
| `fineweb_edu` | 17% | general English text from the FineWeb-Edu corpus |
| `mot_reasoning` | 8% | general reasoning traces |

Files: `train/epoch<k>-<shard>.{jsonl,tok.npy,idx.npy,meta.parquet}` (text, token ids, offsets,
per-sample metadata), `val/`, `tokenizer/` (Qwen3.5 tokenizer with the added tokens),
`summary.json`. Homologs of the FoldBench-334 targets were removed from the protein samples.

## Sources and terms

The structure data and the protein tasks derive from the Protein Data Bank (CC0) and the
AlphaFold Protein Structure Database (CC BY 4.0). The general-text portion is redistributed
from third-party corpora in tokenised and text form and remains under the terms of those
corpora; if you reuse it, follow their licenses. The data is provided for research.