Carbon-A training data

This repository contains the Parquet training shards and the evaluation files used by Carbon-A.

Bucket layout

Carbon-A-training-data/
├── train/
│   ├── vertebrate_mammalian/
│   ├── vertebrate_other/
│   ├── invertebrate/
│   ├── plant/
│   ├── fungi/
│   └── protozoa/
└── eval/
    ├── seen/
    ├── unseen/
    └── nonstandard_code/

The bucket contains 4,561 training shards. These are already the sliding-window-augmented training examples: each Parquet row is a genomic window of 98,304 bp. The evaluation design contains 28 seen accessions, 14 unseen accessions, and one nonstandard_code accession. The domain breakdown is:

Training-data construction

The training corpus uses paired RefSeq GenBank Flat Files and FASTA records from RefSeq assemblies. A record is eligible when its genomic sequence is at least 98,304 bp. The six domains are mammals, other vertebrates, invertebrates, plants, fungi, and protozoa.

Each example contains a genomic window and two nucleotide-resolution binary target tracks, one per strand. CDS intervals from annotated isoforms are unioned on each strand; opposite-strand overlaps remain separate. Windows without CDS labels are retained, so the data include both coding and genomic background sequence. The model uses non-overlapping 6-mer tokens while the targets remain nucleotide aligned.

Domain-specific overlapping strides increase sampling of the rarer domains. A random 0--5 bp start offset changes both the crop boundary and the phase of the 6-mer grid.

The raw, pre-augmentation source contains 3.033 Tbp. Sliding-window augmentation expands the represented sequence to 4.35 Tbp across 44.27 million windows, which are the examples stored under train/. The shorter fungi and protozoa strides compensate for their small share of the raw base count.

License

Apache 2.0.