Move 'The importance of openness' after the cost section; drop the five-benchmark TMP refusal figure
Browse files- assets/refusal-vs-score-5.png +0 -3
- index.html +28 -32
assets/refusal-vs-score-5.png
DELETED
Git LFS Details
|
index.html
CHANGED
|
@@ -330,29 +330,6 @@
|
|
| 330 |
</figure>
|
| 331 |
</section>
|
| 332 |
|
| 333 |
-
<section>
|
| 334 |
-
<h2>The importance of openness</h2>
|
| 335 |
-
<p>As frontier labs move into biology, researchers should not have to depend on a handful of closed APIs. They need strong models they can run on their own hardware, keep sensitive data private, and afford to use at scale.</p>
|
| 336 |
-
<p><strong>Routine requests can trigger refusals.</strong> Safeguards matter in biology, but many closed models also refuse to answer legitimate biology questions. This has a measurable impact on benchmark performance. For example, on FrontierVEP, which tests whether mutations are benign or harmful, Gemini 4 Argon’s accuracy drops from 58% to 38% as reasoning effort increases, alongside a rise in refusals. We see a similar pattern with Claude Opus 5.5 on NextBase, which tests a model’s ability to generate DNA sequences: its score drops from 17.7% at low reasoning effort to just 1.1% at maximum effort.</p>
|
| 337 |
-
<p>Performance also depends on whether a model attempts the task. Several frontier models refused a large share of legitimate genomic requests. For example, on the NextBase task, the refusal rates for Opus 5.5 and Gemini 4 Argon increased significantly with reasoning effort, leading to a corresponding drop in performance.</p>
|
| 338 |
-
|
| 339 |
-
<figure>
|
| 340 |
-
<div class="legend">
|
| 341 |
-
<span><i style="background:#2b6cd4"></i>GPT-6.1 Sol</span><span><i style="background:#bf7039"></i>Opus 5.5</span><span><i style="background:#8460a5"></i>Gemini 4 Argon</span>
|
| 342 |
-
<span class="sep"><b class="ring"></b>Low reasoning</span><span><b class="dot"></b>Max reasoning</span>
|
| 343 |
-
</div>
|
| 344 |
-
<div class="chart" id="fig-refusal"></div>
|
| 345 |
-
<figcaption><b>Figure 4.</b> NextBase score against refusal rate at low and max reasoning effort. Refusals are provider-reported content filter stops.</figcaption>
|
| 346 |
-
</figure>
|
| 347 |
-
<figure>
|
| 348 |
-
<div class="tmp"><img src="assets/refusal-vs-score-5.png" alt="Five small charts, one per benchmark, of benchmark score against refusal rate for Gemini 4 Argon, Opus 5.5 and GPT-6.1 at low and high reasoning effort." loading="lazy"></div>
|
| 349 |
-
<figcaption><b>Figure 5.</b> Closed models often refuse to answer legitimate biology questions. Across five DNA benchmarks, higher reasoning effort increases refusal rates for both Gemini 4 Argon and Opus 5.5, alongside lower scores. Curiously, GPT 6.1 Sol has very few refusals on the same benchmarks.</figcaption>
|
| 350 |
-
</figure>
|
| 351 |
-
<p><strong>Sensitive data needs to stay under researchers’ control.</strong> Human DNA can identify individuals, and research sequences may be confidential or unpublished. Running a model locally lets researchers work within their own infrastructure and data-governance requirements.</p>
|
| 352 |
-
<p><strong>Inference costs add up.</strong> A DNA mutation isn’t necessarily harmful. For a variant of uncertain significance (VUS), scientists don’t yet have enough evidence to tell whether it contributes to disease. ClinVar collects these unresolved findings alongside other genetic variants. What would it cost to ask an AI model to assess 2.4 million of these variants? Extrapolating from our FrontierVEP benchmark, we estimate $32k–$4M for frontier models, depending on the model and reasoning effort, compared with about $500 for Carbon-X. At this scale, the cost of each answer adds up quickly.</p>
|
| 353 |
-
<p><span class="tbd">draft FIGURE 3 here is the ClinVar cost chart again (older numbers), same as Figure 14: keep one</span></p>
|
| 354 |
-
</section>
|
| 355 |
-
|
| 356 |
<section>
|
| 357 |
<h2>Carbon’s DNA capabilities</h2>
|
| 358 |
<p>We evaluate Carbon on five benchmarks that test whether models can use DNA knowledge to predict mutation effects, identify biological features, and recover DNA bases:</p>
|
|
@@ -360,7 +337,7 @@
|
|
| 360 |
<p>For example, given an 8,192-base window of human DNA with a single A→G change, Carbon-X correctly identifies the variant as pathogenic and explains why: the change disrupts the AG at a splice site. Most frontier models answered benign, some after reasoning about the wrong gene, while others were blocked or ran out of tokens.</p>
|
| 361 |
<figure>
|
| 362 |
<iframe class="fig" src="figures/vep-comparison.html" title="One variant, every model: NF1 c.6007-2A>G" loading="lazy"></iframe>
|
| 363 |
-
<figcaption><b>Figure
|
| 364 |
</figure>
|
| 365 |
</section>
|
| 366 |
|
|
@@ -373,12 +350,12 @@
|
|
| 373 |
<p><strong>Hybrid tokenizer.</strong> Qwen3’s tokenizer was built for text: it splits a DNA sequence into irregular fragments that change from one sequence to the next. Following <a class="inline" href="https://arxiv.org/abs/2502.07527">NatureLM</a>, we built a hybrid tokenizer. DNA is wrapped in <code><dna></code> tags and cut into 6-mers, while everything else still goes through Qwen3’s original tokenizer. The model keeps its English vocabulary intact and gains a new one for DNA.</p>
|
| 374 |
<figure>
|
| 375 |
<iframe class="fig" src="figures/tokenizer.html" title="Qwen3 tokenizer vs Carbon-X tokenizer" loading="lazy"></iframe>
|
| 376 |
-
<figcaption><b>Figure
|
| 377 |
</figure>
|
| 378 |
<p><strong>DNA and English favored different learning rates.</strong> During the midtraining ablations, we found that higher learning rates improved DNA performance but caused substantial forgetting. Lower learning rates preserved English better, but slowed progress on DNA. We struggled to find a setting that worked well for both. Training the new DNA embeddings after random initialization first, with the rest of the model frozen, helped a little but did not resolve the conflict.</p>
|
| 379 |
<figure>
|
| 380 |
<iframe class="fig" src="figures/lr-sweep-4b.html" title="Qwen3-4B midtraining learning-rate sweep" loading="lazy"></iframe>
|
| 381 |
-
<figcaption><b>Figure
|
| 382 |
</figure>
|
| 383 |
<details class="box">
|
| 384 |
<summary>The usual continual-pretraining fixes did not close the gap.</summary>
|
|
@@ -386,7 +363,7 @@
|
|
| 386 |
<p>We tried different DNA embeddings initialization techniques, separate learning rates for the embeddings and transformer body, protecting layers with conflicting DNA and English gradients, merging DNA and English specialists, and increasing English replay. None recovered strong performance in both domains.</p>
|
| 387 |
<figure>
|
| 388 |
<iframe class="fig" src="figures/cpt-fixes.html" title="Continual-pretraining fixes we tried" loading="lazy"></iframe>
|
| 389 |
-
<figcaption><b>Figure
|
| 390 |
</figure>
|
| 391 |
</div>
|
| 392 |
</details>
|
|
@@ -394,11 +371,11 @@
|
|
| 394 |
<p><strong>The training data.</strong> The final run used a 300B-token mixture made mostly of raw DNA from Carbon-3B’s data mixture, English text from web datasets and PDFs, and instruction data to help retain chat and instruction following. The rest paired DNA with text: documents mined and filtered from web pages, papers and forums, plus synthetic examples in which Gemini analyzes annotated RefSeq sequences.</p>
|
| 395 |
<figure>
|
| 396 |
<iframe class="fig" src="figures/data-mixture.html" title="Midtraining data mixture" loading="lazy"></iframe>
|
| 397 |
-
<figcaption><b>Figure
|
| 398 |
</figure>
|
| 399 |
<figure>
|
| 400 |
<iframe class="fig" src="figures/progression-300b.html" title="Carbon-X midtraining on 300B tokens" loading="lazy"></iframe>
|
| 401 |
-
<figcaption><b>Figure
|
| 402 |
</figure>
|
| 403 |
|
| 404 |
<h3>Post-training</h3>
|
|
@@ -412,7 +389,7 @@
|
|
| 412 |
<p>Instead, we initially adopted <em>self-distillation</em> where we apply RLVR directly to the mid-trained model and use the resulting rollouts as the source for on-policy SFT data. As shown below, on FrontierVEP this approach produced significant gains over the base model: a 10 point improvement and signal beyond the random baseline of 50%.</p>
|
| 413 |
<figure>
|
| 414 |
<iframe class="fig" src="figures/frontiervep-base-mid.html" title="FrontierVEP: Base + RL vs Mid-training + RL" loading="lazy"></iframe>
|
| 415 |
-
<figcaption><b>Figure
|
| 416 |
</figure>
|
| 417 |
<p>However, this self-distillation was not sufficient: the resulting model was prone to hallucinations (e.g. quoting databases like ClinVar or referring to incorrect genes). Instead, we took a different approach: work with domain experts to develop a set of deterministic verifiers that can be used to ground a response given the context DNA sequences. On FrontierVEP, the result is a model that generates far fewer hallucinations while obtaining higher performance:</p>
|
| 418 |
<figure>
|
|
@@ -420,7 +397,7 @@
|
|
| 420 |
</figure>
|
| 421 |
<figure>
|
| 422 |
<iframe class="fig" src="figures/training-grid-v26.html" title="Carbon-X RL training rewards and evaluations (stage 1)" loading="lazy"></iframe>
|
| 423 |
-
<figcaption><b>Figure
|
| 424 |
</figure>
|
| 425 |
|
| 426 |
<h3>Post-training, from latent DNA knowledge to a usable assistant <span class="tbd">older outline in the draft, merge or drop?</span></h3>
|
|
@@ -453,8 +430,27 @@
|
|
| 453 |
<p>Large-scale genomic studies can require millions of predictions. We estimate that analysing 2.4 million genetic variants whose effects on health remain uncertain would cost about $500 with Carbon-X, compared with $32,000 to $4 million using the frontier AI models evaluated, depending on the model and reasoning setting. <span class="tbd">figure says $271 for Carbon-X, align text</span> Researchers can also run Carbon-X on their own infrastructure, keeping sensitive or unpublished DNA sequences under their control.</p>
|
| 454 |
<figure>
|
| 455 |
<iframe class="fig" src="figures/clinvar-cost.html" title="Projected cost to process 2.4M ClinVar VUS" loading="lazy"></iframe>
|
| 456 |
-
<figcaption><b>Figure
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 457 |
</figure>
|
|
|
|
|
|
|
|
|
|
| 458 |
</section>
|
| 459 |
|
| 460 |
<section>
|
|
|
|
| 330 |
</figure>
|
| 331 |
</section>
|
| 332 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 333 |
<section>
|
| 334 |
<h2>Carbon’s DNA capabilities</h2>
|
| 335 |
<p>We evaluate Carbon on five benchmarks that test whether models can use DNA knowledge to predict mutation effects, identify biological features, and recover DNA bases:</p>
|
|
|
|
| 337 |
<p>For example, given an 8,192-base window of human DNA with a single A→G change, Carbon-X correctly identifies the variant as pathogenic and explains why: the change disrupts the AG at a splice site. Most frontier models answered benign, some after reasoning about the wrong gene, while others were blocked or ran out of tokens.</p>
|
| 338 |
<figure>
|
| 339 |
<iframe class="fig" src="figures/vep-comparison.html" title="One variant, every model: NF1 c.6007-2A>G" loading="lazy"></iframe>
|
| 340 |
+
<figcaption><b>Figure 4.</b> One variant, every model: NF1 c.6007-2A>G.</figcaption>
|
| 341 |
</figure>
|
| 342 |
</section>
|
| 343 |
|
|
|
|
| 350 |
<p><strong>Hybrid tokenizer.</strong> Qwen3’s tokenizer was built for text: it splits a DNA sequence into irregular fragments that change from one sequence to the next. Following <a class="inline" href="https://arxiv.org/abs/2502.07527">NatureLM</a>, we built a hybrid tokenizer. DNA is wrapped in <code><dna></code> tags and cut into 6-mers, while everything else still goes through Qwen3’s original tokenizer. The model keeps its English vocabulary intact and gains a new one for DNA.</p>
|
| 351 |
<figure>
|
| 352 |
<iframe class="fig" src="figures/tokenizer.html" title="Qwen3 tokenizer vs Carbon-X tokenizer" loading="lazy"></iframe>
|
| 353 |
+
<figcaption><b>Figure 5.</b> Qwen3 tokenizer vs Carbon-X tokenizer.</figcaption>
|
| 354 |
</figure>
|
| 355 |
<p><strong>DNA and English favored different learning rates.</strong> During the midtraining ablations, we found that higher learning rates improved DNA performance but caused substantial forgetting. Lower learning rates preserved English better, but slowed progress on DNA. We struggled to find a setting that worked well for both. Training the new DNA embeddings after random initialization first, with the rest of the model frozen, helped a little but did not resolve the conflict.</p>
|
| 356 |
<figure>
|
| 357 |
<iframe class="fig" src="figures/lr-sweep-4b.html" title="Qwen3-4B midtraining learning-rate sweep" loading="lazy"></iframe>
|
| 358 |
+
<figcaption><b>Figure 6.</b> Qwen3-4B midtraining learning-rate sweep.</figcaption>
|
| 359 |
</figure>
|
| 360 |
<details class="box">
|
| 361 |
<summary>The usual continual-pretraining fixes did not close the gap.</summary>
|
|
|
|
| 363 |
<p>We tried different DNA embeddings initialization techniques, separate learning rates for the embeddings and transformer body, protecting layers with conflicting DNA and English gradients, merging DNA and English specialists, and increasing English replay. None recovered strong performance in both domains.</p>
|
| 364 |
<figure>
|
| 365 |
<iframe class="fig" src="figures/cpt-fixes.html" title="Continual-pretraining fixes we tried" loading="lazy"></iframe>
|
| 366 |
+
<figcaption><b>Figure 7.</b> Continual-pretraining fixes we tried.</figcaption>
|
| 367 |
</figure>
|
| 368 |
</div>
|
| 369 |
</details>
|
|
|
|
| 371 |
<p><strong>The training data.</strong> The final run used a 300B-token mixture made mostly of raw DNA from Carbon-3B’s data mixture, English text from web datasets and PDFs, and instruction data to help retain chat and instruction following. The rest paired DNA with text: documents mined and filtered from web pages, papers and forums, plus synthetic examples in which Gemini analyzes annotated RefSeq sequences.</p>
|
| 372 |
<figure>
|
| 373 |
<iframe class="fig" src="figures/data-mixture.html" title="Midtraining data mixture" loading="lazy"></iframe>
|
| 374 |
+
<figcaption><b>Figure 8.</b> Midtraining data mixture.</figcaption>
|
| 375 |
</figure>
|
| 376 |
<figure>
|
| 377 |
<iframe class="fig" src="figures/progression-300b.html" title="Carbon-X midtraining on 300B tokens" loading="lazy"></iframe>
|
| 378 |
+
<figcaption><b>Figure 9.</b> Carbon-X midtraining on 300B tokens.</figcaption>
|
| 379 |
</figure>
|
| 380 |
|
| 381 |
<h3>Post-training</h3>
|
|
|
|
| 389 |
<p>Instead, we initially adopted <em>self-distillation</em> where we apply RLVR directly to the mid-trained model and use the resulting rollouts as the source for on-policy SFT data. As shown below, on FrontierVEP this approach produced significant gains over the base model: a 10 point improvement and signal beyond the random baseline of 50%.</p>
|
| 390 |
<figure>
|
| 391 |
<iframe class="fig" src="figures/frontiervep-base-mid.html" title="FrontierVEP: Base + RL vs Mid-training + RL" loading="lazy"></iframe>
|
| 392 |
+
<figcaption><b>Figure 10.</b> FrontierVEP: base + RL vs mid-training + RL.</figcaption>
|
| 393 |
</figure>
|
| 394 |
<p>However, this self-distillation was not sufficient: the resulting model was prone to hallucinations (e.g. quoting databases like ClinVar or referring to incorrect genes). Instead, we took a different approach: work with domain experts to develop a set of deterministic verifiers that can be used to ground a response given the context DNA sequences. On FrontierVEP, the result is a model that generates far fewer hallucinations while obtaining higher performance:</p>
|
| 395 |
<figure>
|
|
|
|
| 397 |
</figure>
|
| 398 |
<figure>
|
| 399 |
<iframe class="fig" src="figures/training-grid-v26.html" title="Carbon-X RL training rewards and evaluations (stage 1)" loading="lazy"></iframe>
|
| 400 |
+
<figcaption><b>Figure 11.</b> Carbon-X RL training rewards and evaluations (stage 1).</figcaption>
|
| 401 |
</figure>
|
| 402 |
|
| 403 |
<h3>Post-training, from latent DNA knowledge to a usable assistant <span class="tbd">older outline in the draft, merge or drop?</span></h3>
|
|
|
|
| 430 |
<p>Large-scale genomic studies can require millions of predictions. We estimate that analysing 2.4 million genetic variants whose effects on health remain uncertain would cost about $500 with Carbon-X, compared with $32,000 to $4 million using the frontier AI models evaluated, depending on the model and reasoning setting. <span class="tbd">figure says $271 for Carbon-X, align text</span> Researchers can also run Carbon-X on their own infrastructure, keeping sensitive or unpublished DNA sequences under their control.</p>
|
| 431 |
<figure>
|
| 432 |
<iframe class="fig" src="figures/clinvar-cost.html" title="Projected cost to process 2.4M ClinVar VUS" loading="lazy"></iframe>
|
| 433 |
+
<figcaption><b>Figure 12.</b> Projected cost to process 2.4 million ClinVar variants of uncertain significance.</figcaption>
|
| 434 |
+
</figure>
|
| 435 |
+
</section>
|
| 436 |
+
|
| 437 |
+
<section>
|
| 438 |
+
<h2>The importance of openness</h2>
|
| 439 |
+
<p>As frontier labs move into biology, researchers should not have to depend on a handful of closed APIs. They need strong models they can run on their own hardware, keep sensitive data private, and afford to use at scale.</p>
|
| 440 |
+
<p><strong>Routine requests can trigger refusals.</strong> Safeguards matter in biology, but many closed models also refuse to answer legitimate biology questions. This has a measurable impact on benchmark performance. For example, on FrontierVEP, which tests whether mutations are benign or harmful, Gemini 4 Argon’s accuracy drops from 58% to 38% as reasoning effort increases, alongside a rise in refusals. We see a similar pattern with Claude Opus 5.5 on NextBase, which tests a model’s ability to generate DNA sequences: its score drops from 17.7% at low reasoning effort to just 1.1% at maximum effort.</p>
|
| 441 |
+
<p>Performance also depends on whether a model attempts the task. Several frontier models refused a large share of legitimate genomic requests. For example, on the NextBase task, the refusal rates for Opus 5.5 and Gemini 4 Argon increased significantly with reasoning effort, leading to a corresponding drop in performance.</p>
|
| 442 |
+
|
| 443 |
+
<figure>
|
| 444 |
+
<div class="legend">
|
| 445 |
+
<span><i style="background:#2b6cd4"></i>GPT-6.1 Sol</span><span><i style="background:#bf7039"></i>Opus 5.5</span><span><i style="background:#8460a5"></i>Gemini 4 Argon</span>
|
| 446 |
+
<span class="sep"><b class="ring"></b>Low reasoning</span><span><b class="dot"></b>Max reasoning</span>
|
| 447 |
+
</div>
|
| 448 |
+
<div class="chart" id="fig-refusal"></div>
|
| 449 |
+
<figcaption><b>Figure 13.</b> NextBase score against refusal rate at low and max reasoning effort. Refusals are provider-reported content filter stops.</figcaption>
|
| 450 |
</figure>
|
| 451 |
+
<p><strong>Sensitive data needs to stay under researchers’ control.</strong> Human DNA can identify individuals, and research sequences may be confidential or unpublished. Running a model locally lets researchers work within their own infrastructure and data-governance requirements.</p>
|
| 452 |
+
<p><strong>Inference costs add up.</strong> A DNA mutation isn’t necessarily harmful. For a variant of uncertain significance (VUS), scientists don’t yet have enough evidence to tell whether it contributes to disease. ClinVar collects these unresolved findings alongside other genetic variants. What would it cost to ask an AI model to assess 2.4 million of these variants? Extrapolating from our FrontierVEP benchmark, we estimate $32k–$4M for frontier models, depending on the model and reasoning effort, compared with about $500 for Carbon-X. At this scale, the cost of each answer adds up quickly.</p>
|
| 453 |
+
<p><span class="tbd">draft FIGURE 3 here is the ClinVar cost chart again (older numbers), same as Figure 12: keep one</span></p>
|
| 454 |
</section>
|
| 455 |
|
| 456 |
<section>
|