README.md

---
license: apache-2.0
tags:
- biology
- biomedical
- post-training
- deepseek-v4
pipeline_tag: text-generation
---

# Fx-Bio

## Introduction

Fx-Bio-0913 is a biomedical reasoning large language model post-trained on **DeepSeek-V4-Flash**, developed by The Endless Frontier lab. It is specialized for biological and biomedical research tasks — including gene-function puzzles, experimental reasoning, and multi-step evidence integration — through post-training on curated biomedical reasoning data (Biomni / BioMystery-related tasks). The model retains the 1M-token context length of its base model, and its weights are openly released under Apache-2.0.

## Main Results

Fx-Bio-0913 on two biomedical reasoning benchmarks, **BioMysteryBench** and **BiominiBench**, compared with the DeepSeek-V4-Flash base model and Opus-5. Post-training yields consistent gains over the base model across all metrics, with the largest improvements on harder problems (BioMysteryBench Human-Difficult, +5.8) and on pass@3 of BiominiBench (+6.6).

<p align="center">
  <img src="./assets/fx-bio-eval.png" alt="Fx-Bio-0913 evaluation results on BioMysteryBench and BiominiBench" width="100%">
</p>

## Benchmark Results

Comparison of Fx-Bio-0913 with representative frontier models on BioMysteryBench (avg@5 and pass@5) and BiominiBench (avg@3 and pass@3). All scores are Accuracy (%) from our internal evaluation pipeline; higher is better.

| Benchmark | Opus<br>5 | GPT<br>5.6 | Gemini 3.8<br>Flash | GLM<br>5.3 | Qwen3.8 Flash<br>Next (1M) | DeepSeek V4.1<br>Flash | DeepSeek V4<br>Flash | Fx-Bio<br>0913 |
|:--|--:|--:|--:|--:|--:|--:|--:|--:|
| **BioMysteryBench** |  |  |  |  |  |  |  |  |
| Human-Solvable (avg@5) | 89.3 | 85.5 | 88.8 | 84.7 | 86.9 | 89.0 | 85.2 | **85.5** |
| Human-Difficult (avg@5) | 45.9 | 34.1 | 42.4 | 47.1 | 49.4 | 36.5 | 31.8 | **37.6** |
| pass@5 | 86.7 | 85.6 | 84.4 | 88.9 | 86.7 | 85.6 | 82.2 | **86.7** |
| **BiominiBench** |  |  |  |  |  |  |  |  |
| avg@3 | 79.1 | 68.2 | 69.3 | 81.1 | 80.6 | 80.9 | 71.0 | **77.9** |
| pass@3 | 86.7 | 78.6 | 78.5 | 87.2 | 88.9 | 85.9 | 78.7 | **85.3** |

## Model Overview

| Item | Details |
|---|---|
| Base model | DeepSeek-V4-Flash |
| Architecture | MoE causal LM (`DeepseekV4ForCausalLM`) |
| Weights | bf16 safetensors (114 shards) |
| Context length | 1M tokens |
| Checkpoint | checkpoint-265 |
| License | Apache-2.0 |

## Quickstart

```python
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "endless-frontier/Fx-Bio"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype="bfloat16",
    device_map="auto",
)

inputs = tokenizer("<your prompt>", return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=1024)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
```

> [!Important]
> Requires a `transformers` version that supports the `DeepseekV4ForCausalLM` architecture. For production workloads or high-throughput scenarios, a dedicated serving framework (e.g. SGLang or vLLM) with DeepSeek-V4 support is recommended.

## Training

Fx-Bio is post-trained on biomedical reasoning data (Biomni / BioMystery-related tasks) with Megatron-LM. The detailed training recipe and data composition will be described in an upcoming technical report.

`args.json` contains the original Megatron-LM launch arguments for reproducibility.

## Disclaimer

For research use only. Model outputs may contain errors or inaccuracies and must not be used directly for clinical diagnosis or medical decision-making.

## License

[Apache-2.0](LICENSE)