open-athena/MarinFold/data/document_structures/contacts_v1_think - Storage Bucket

contacts-v1-think protein-document dataset

Training documents in the contacts-v1 format with <think> (pause) tokens in the structure section — a think-augmented twin of contacts_v1 over the same afdb-24M entries, clusters, pLDDT rounds, and train/val/test splits. Generated by calling MarinFold's contacts_v1 generator with GenerationConfig(think=True) (no re-implementation). Format spec + think details: marinfold/.../document_structures/contacts_v1/SPEC.md (§ Think (pause) tokens).

Produced by experiment exp126 (marinfold @ 5526bb2; think path from #123 / PR #125). Intended as the training corpus for #124.

What <think> means here

<think> runs are inserted between <contact> statements in the structure section (never inside one), sampled per #123's spec: an initial run right after <begin_statements> (P=0.75, Geometric(0.13)), plus max(int(U(-4,4)), 0) extra runs each Geometric(0.25), with the total subtracted from the 8192-token budget before contacts are selected. No vocab/tokenizer change — <think> was already reserved in the contacts-v1 vocab. At training time, mask the loss on <think> tokens (they carry no target). The think_tokens column gives the per-document count.

Structure (rounds)

Identical selection to contacts_v1: up to 5 examples per structural cluster by pLDDT (round = 0 highest), clusters with < 3 usable members dropped, shards laid out round-descending (round-4 first … round-0 last) so a sequential reader trains on the highest-quality data last. The round column is authoritative regardless of file order. split is inherited from afdb-24M and is cluster-consistent (no train/test leakage).

Layout

<split>/contacts_v1-{shard:05d}-of-{total:05d}.parquet   # round-descending, matches contacts_v1 shard-for-shard
tokenizer/                                                   # contacts-v1 tokenizer (unchanged from contacts_v1)

Counts

4,213,203 documents over the same 960,054 structural clusters (0 generation drops).

split	round 0	round 1	round 2	round 3	round 4	total
train	941,028	941,028	941,028	719,519	587,079	4,129,682
val	9,558	9,558	9,558	7,316	5,964	41,954
test	9,468	9,468	9,468	7,248	5,915	41,567

~4.80 B tokens total (mean 1,140 num_tokens/doc, max 8,192). 36,918,016 <think> tokens across the corpus (84% of docs carry a think run; mean ~8.8 think tokens/doc).

Columns

Same schema as contacts_v1 plus think_tokens: document (token string) · structure ("contacts-v1") · entry_id · round · struct_cluster_id · seq_cluster_id · split · global_plddt · seq_len · start_index / n_term_index / c_term_index · contacts_pre_filter / contacts_passing_min_degree / contacts_emitted / contacts_excluded · truncated · highest_contact_degree / lowest_nonzero_contact_degree / lowest_included_contact_degree · num_tokens · think_tokens · sha1 · uniprot_accession / tax_id / organism_name.

sha1 = sha1 of document, so byte-equality with the MarinFold generator is a single column compare.