Clarify model documentation and explicitly disable Parquet page indexes

a74a209 3 days ago

5.56 kB

library_name: transformers
license: apache-2.0
pipeline_tag: token-classification
tags:
  - dna
  - genomics
  - biology
  - genome-annotation

Carbon-A-1.2B

A DNA annotation model from the Carbon family.

Carbon-A predicts protein-coding sequence (CDS) at single-base resolution in eukaryotic genomes. It produces separate forward- and reverse-strand probabilities, preserving coding regions that overlap on opposite strands. The annotation pipeline in this repository turns these probabilities into gene models with a confidence score per gene.

Facts

1.2B-parameter DNA annotation model with two strand-specific classification heads.
Tokenizer: non-overlapping 6-mers; each DNA token represents six bases.
Sequence length: 16,384 tokens (98,304 bp). The annotation pipeline reads longer records in windows that overlap by 1,024 tokens (6,144 bp).
Inputs: genomic DNA. The annotation pipeline reads FASTA and GenBank files, optionally gzip-compressed.
Outputs: per-base probabilities for non-coding and CDS on each strand. The annotation pipeline turns them into gene models with a confidence score per gene.
Training data: 2,055 RefSeq GCF assemblies (2,047 taxa) from mammals, other vertebrates, invertebrates, plants, fungi, and protozoa.
Transformers interface: AutoModelForTokenClassification with trust_remote_code=True; Transformers 4.56 or later, including 5.x. FlashAttention-2 needs Transformers 4.x.
Inference: BF16 with FlashAttention-2 for all reported results. PyTorch SDPA (--attn-implementation sdpa) needs no FlashAttention installation, but its probabilities are not bit-identical.

How to use

Install

The annotation pipeline runs on Linux. It needs Python 3.10 or later, PyTorch (a CUDA build for GPU inference), and a C++ compiler with OpenMP. If you do not have a suitable compiler, we recommend GCC, for example on Debian or Ubuntu:

sudo apt install build-essential

Then download this repository and install its requirements:

pip install -U huggingface_hub
hf download HuggingFaceBio/Carbon-A-1.2B --exclude model.safetensors --local-dir Carbon-A-1.2B
pip install -r Carbon-A-1.2B/requirements.txt
pip install --no-build-isolation flash-attn

The model weights (4.6 GB) are downloaded to the Hugging Face cache on first use.

Annotate a genome

bash Carbon-A-1.2B/annotation_pipeline/run_annotation.sh --input genome.fna.gz

The input is a FASTA or GenBank file, optionally gzip-compressed. Carbon-A reads it in 98,304-bp windows that overlap by 6,144 bp and averages the overlapping predictions. The pipeline then decodes the gene structures and writes the results to results/<accession>/:

prediction.gff: genes, mRNAs, and CDS. Every gene carries a model-derived confidence score for its predicted structure; higher values indicate greater model confidence.
prediction.fna and prediction.faa: the CDS and protein sequences.
*.raw_probs.parquet: the per-base CDS probabilities.

All genes are written. The reported results count the genes with a confidence of at least 0.1.

The standard genetic code is used by default. For another code, pass --codon-table. Tetrahymena thermophila (GCF_000189635.1) uses translation table 6 automatically. A likely codon-table mismatch is detected automatically: the pipeline adds a warning to results/WARNINGS.txt and renames the result directory to results/<accession>_WARNING.

The pipeline README lists all options. It also describes reference-based evaluation and BUSCO evaluation.

Test on a benchmark genome

The benchmark genomes and their NCBI annotations are in the data bucket under eval/. To test the pipeline on yeast, download its genome and annotation:

url=https://huggingface.co/buckets/HuggingFaceBio/Carbon-A-training-data/resolve/eval/seen/GCF_000146045.2
mkdir -p GCF_000146045.2
for file in GCF_000146045.2_R64_genomic.fna genomic.gff cds_from_genomic.fna sequence_report.jsonl; do
    curl -fsSL -o GCF_000146045.2/$file $url/$file
done

Annotate the genome and compare the result with the NCBI annotation:

bash Carbon-A-1.2B/annotation_pipeline/run_annotation.sh \
    --input GCF_000146045.2/GCF_000146045.2_R64_genomic.fna
bash Carbon-A-1.2B/annotation_pipeline/run_evaluation.sh \
    --prediction-prefix results/GCF_000146045.2/prediction \
    --genome-dir GCF_000146045.2

results/GCF_000146045.2/evaluation_metrics.json should then report a nucleotide F1 of 0.998, an exon F1 of 0.943, and a gene F1 of 0.953.

License

Apache 2.0.

Acknowledgements

Carbon-A is a collaboration between research teams at Hugging Face and Zhongguancun Academy, with contributions from collaborators at Tsinghua University, the Institute of Bio-Architecture and Bio-Interactions, the Affiliated High School of Peking University, the Telethon Institute of Genetics and Medicine, and the University of Naples “Federico II”. For technical details, see the Carbon-A technical report.

We also gratefully acknowledge Alex Kleinman and the rest of the ActiveSite team for their contributions, and thank the Stem Cell Genomics Core at the Sanford Stem Cell Institute, particularly Trevor Biddle, for providing sequencing services.