Carbon-X-GGUF

Carbon-X is a 14.8B-parameter decoder-only language model for DNA understanding and generation, the first model of the Carbon family, built by Hugging Face with Zhongguancun Academy and the Telethon Institute of Genetics and Medicine / University of Naples. It starts from Qwen3-14B, and DNA and English text share a single token stream with no separate DNA encoder. A hybrid tokenizer keeps Qwen's BPE vocabulary for text and encodes DNA as non-overlapping 6-mers (5,515 DNA tokens in all), detecting and tagging DNA in a prompt automatically. Users paste raw sequences into an ordinary chat and get English answers. It can classify mutations as benign or pathogenic, identify regulatory regions, classify sequences by organism group, predict the next bases of a sequence, and explain its reasoning. Training had four stages. First, about 10B tokens with only the new DNA embeddings trainable. Then 302B tokens of mid-training on DNA (55%), English, biology text and instruction data. Then supervised fine-tuning, and finally asynchronous GRPO with verifiable rewards. It was post-trained and evaluated with thinking disabled, and the card recommends keeping it that way. Across five DNA benchmarks it averages 66.7%, which is 27 points above Qwen3-14B, behind GPT-6.1 Sol (67.5%) by 0.8 points, and ahead of Opus 5.5 (60.9%) and the other models compared. It leads on BioReason Coding (74.9) and NextBase (39.4) but trails frontier models on BioReason Non-SNV (85.1 vs 94.8 for Opus 5.5) and FrontierVEP (70.4 vs 76.2), and two of the five benchmarks (FrontierVEP and NextBase) were built by the authors. The card says the cost per problem is roughly 1/1,800 of GPT-6.1 Sol's, and that analysing ClinVar's 2.4 million variants of uncertain significance would cost about $300. Its weaknesses are splice-site and histone-mark questions (29.8% and 34.6% on GenomeQA), exact sequence operations such as counting and coordinates, and long DNA generation, since it was trained to read sequences, not design them. It has not been clinically validated and should not be used for medical decisions. The weights fit on one 80 GB GPU in bfloat16, and it runs with Transformers v5 or later (with trust_remote_code=True for the tokenizer) or vLLM, but not with llama.cpp or other GGUF runtimes, which cannot load the custom tokenizer. The card includes examples for using it as a tool for agents, including a Claude Code and Codex CLI skill. It also warns that genetic data is sensitive and the user should understand how their provider handles it. It is released under Apache 2.0.

Model Files

File Name	Quant Type	File Size	File Link	Description
Carbon-X.BF16.gguf	BF16	29.7 GB	Link	Full BF16 weights. Highest quality, largest file size.
Carbon-X.Q3_K_L.gguf	Q3_K_L	7.93 GB	Link	Lower quality but usable, good for low RAM availability.
Carbon-X.Q3_K_M.gguf	Q3_K_M	7.36 GB	Link	Low quality.
Carbon-X.Q4_K_M.gguf	Q4_K_M	9.04 GB	Link	Good quality, default size for most use cases, recommended.
Carbon-X.Q4_K_S.gguf	Q4_K_S	8.61 GB	Link	Slightly lower quality with more space savings, recommended.
Carbon-X.Q5_K_M.gguf	Q5_K_M	10.6 GB	Link	High quality, recommended.
Carbon-X.Q5_K_S.gguf	Q5_K_S	10.3 GB	Link	High quality, recommended.
Carbon-X.Q6_K.gguf	Q6_K	12.2 GB	Link	Very high quality, near perfect, recommended.
Carbon-X.Q8_0.gguf	Q8_0	15.8 GB	Link	Extremely high quality, generally unneeded but max available quant.

BF16 Test Run

Tested Carbon-X.BF16.gguf on an NVIDIA RTX PRO 6000 Blackwell Server Edition GPU with 96 GB of VRAM, using llama-cli with full GPU offloading.

Performance Metrics

Metric	Value
Prompt Processing Speed	519.1 tokens/s
Generation Speed	49.8 tokens/s
Total Wall Time	7.461 seconds
Context Length	8,192 tokens
KV Cache (K)	q8_0
GPU	NVIDIA RTX PRO 6000 Blackwell Server Edition
GPU VRAM Capacity	96 GB
Peak GPU VRAM Usage	27.94 GiB
Peak GPU Utilization	98%
GPU Layers Offloaded	99 (-ngl 99)
Maximum Generation Tokens	16,384
Random Seed	42

Run Inference

./build/bin/llama-cli \
  -m /path/to/models/Carbon-X.BF16.gguf \
  -ngl 99 \
  -c 8192 \
  --cache-type-k q8_0 \
  --seed 42 \
  -n 16384 \
  -st \
  -p "Determine whether the following genetic variant is likely benign or pathogenic, and explain your reasoning based on the sequence difference and its potential biological significance: Reference: ATGGTGCATCTGACTCCTGAGGAGAAGTCTGCCGTTACTGCCCTGTGGGGCAAGG; Variant: ATGGTGCATCTGACTCCTGTGGAGAAGTCTGCCGTTACTGCCCTGTGGGGCAAGG."

---

Sample Inference

Given the reference sequence ATGGTGCATCTGACTCCTGAGGAGAAGTCTGCCGTTACTGCCCTGTGGGGCAAGG and the variant sequence ATGGTGCATCTGACTCCTGTGGAGAAGTCTGCCGTTACTGCCCTGTGGGGCAAGG, is the variant likely to be benign or pathogenic, and why?

The variant is a G>A transition at position 20. The local context is moderately GC-rich. Compared with what the surrounding genomic sequence predicts, an A at this position is somewhat less likely than the reference G, a modest drop relative to other variants, which suggests the position tolerates change. Although the alternate base is not strongly disfavored here, the overall evidence still points to a pathogenic variant.

Answer: Pathogenic

Prompt: 411.8 t/s | Generation: 49.7 t/s | Inference speed measured on NVIDIA RTX PRO 6000 96 GB (-ngl 99).

---

Notes

• Full GPU offloading: -ngl 99 requests offloading up to 99 model layers to the GPU. Actual offloading depends on the model architecture and the number of layers available.
• KV cache: --cache-type-k q8_0 configures the K cache to use the q8_0 quantization format.
• Context length: -c 8192 sets the context size to 8,192 tokens.
• Generation length: -n 16384 sets the maximum number of tokens to generate.
• Random seed: --seed 42 sets the random seed for reproducibility.

Performance results reflect this specific hardware and inference configuration. Actual throughput may vary depending on the prompt, model, software build, and runtime settings.

llama.cpp

LLM inference in C/C++ — https://github.com/ggml-org/llama.cpp