---
license: mit
base_model: biomni/Biomni-R0-32B-Preview
language:
- en
tags:
- agent
- biology
- code
- fp8
---

# Biomni-R0-32B-Preview FP8 Dynamic

FP8_DYNAMIC compression of [biomni/Biomni-R0-32B-Preview](https://huggingface.co/biomni/Biomni-R0-32B-Preview) revision `71432eb3d5e583bee757e0f9437a17e711e8e3d1`. This is not an official Biomni release. The original weights stay on that repo.

- Scheme: compressed-tensors FP8, per-channel weights, dynamic per-token activations
- `lm_head` left dense
- No calibration dataset
- Weights about 35GB

vLLM reads the scheme from `config.json`. Do not pass `--quantization`.

```bash
vllm serve vonkaiser/Biomni-R0-32B-Preview-FP8-Dynamic \
  --gpu-memory-utilization 0.90 \
  --kv-cache-dtype fp8 \
  --max-model-len 32768
```

The chat template asks for a JSON object inside `<tool_call>` tags. Use a tool parser that reads that format (`qwen` or `hermes`).

Base architecture is Qwen/Qwen3-32B. License MIT, same as the source model.

## Comparison with the original

The original checkpoint and this FP8 checkpoint were scored on the same prompts. Both ran in vLLM at temperature 0 and seed 1234. Thinking was off, except one GSM8K slice. The original ran in BF16 with a context of 8192. FP8 used the compressed-tensors config, an FP8 KV cache, and a context of 32768. No Eval1 item hit a context limit on either side.

A gap of about one standard error is treated as the same result. The differences below are a few questions, and they do not all favor the same checkpoint.

### GSM8K

Full set, 1319 questions, 5-shot, thinking off. One standard error is about 11 questions.

| filter | original | FP8 |
| --- | --- | --- |
| strict-match | 1179/1319 (0.8939) | 1181/1319 (0.8954) |
| flexible-extract | 1185/1319 (0.8984) | 1189/1319 (0.9014) |

Thinking on, first 200 questions. The comparison uses flexible-extract. One standard error on this slice is about 4 questions.

| filter | original | FP8 |
| --- | --- | --- |
| flexible-extract | 180/200 (0.900) | 178/200 (0.890) |

### MMLU generative

5-shot, 5 questions from each subject, 285 questions. One standard error is about 6 questions.

| | correct | rate |
| --- | --- | --- |
| original | 248/285 | 0.8702 |
| FP8 | 250/285 | 0.8772 |

### HumanEval

Chat pass@1 on 164 problems. Generated Python was executed. One standard error is about 4 problems.

| | correct | rate |
| --- | --- | --- |
| original | 150/164 | 0.9146 |
| FP8 | 148/164 | 0.9024 |

### Biomni Eval1

All 433 rows were scored on both sides.

| task | n | original | FP8 |
| --- | --- | --- | --- |
| crispr_delivery | 10 | 3 | 2 |
| gwas_causal_gene_gwas_catalog | 50 | 21 | 18 |
| gwas_causal_gene_opentargets | 50 | 38 | 38 |
| gwas_causal_gene_pharmaprojects | 50 | 28 | 32 |
| gwas_variant_prioritization | 43 | 2 | 3 |
| lab_bench_dbqa | 50 | 5 | 4 |
| lab_bench_seqqa | 50 | 7 | 8 |
| patient_gene_detection | 50 | 0 | 2 |
| rare_disease_diagnosis | 30 | 0 | 0 |
| screen_gene_retrieval | 50 | 19 | 19 |

Catalog is down 3 and pharma is up 4, each about one standard error on 50 questions. The other tasks move by one or two questions or stay tied. `rare_disease_diagnosis` is 0/30 on both sides.

### Logprobs

160 prompts, compared on the shared top-1 prefix.

- Identical greedy completions: 110/160
- Aligned positions: 6812
- Mean absolute logprob difference: 0.008
- Largest absolute difference: 0.581
- Positions with a gap over 1.0 nat: 0
- First-token mismatches: 7/160

### Tool calls

20 prompts: 16 that should call a tool, and 4 that should not. A call counts as correct when the tool name matches. Arguments are compared only when both sides called a tool.

Both checkpoints named the right tool on 16/16 and stayed quiet on 4/4. Arguments matched on 15 of those 16 calls. The one mismatch was "Calculate 15 percent of 80": the original passed `15% of 80` and FP8 passed `0.15 * 80`.

### Capability Deviation Score

**+0.3 percentage points.** FP8 scored slightly higher than the original.

The score is the gap between the two accuracy rates on the graded items counted once: GSM8K flexible-extract, MMLU, HumanEval, and Eval1 (2,201 items). The original was correct on 1,706 (77.5%). FP8 was correct on 1,713 (77.8%). A negative score would mean FP8 was less capable.

Each task gap in the tables above sits inside about one standard error. On the biomedical set the original scored 123 of 433 and FP8 scored 126 of 433. The largest moves there are the GWAS catalog task, down 3 of 50, and the pharma task, up 4 of 50.

The thinking-on GSM8K slice is left out of the 2,201 because those questions are already in the full GSM8K set. On that slice FP8 was 2 questions lower, 178 versus 180, which is inside the wider error on 200 questions.