| --- |
| license: apache-2.0 |
| base_model: Qwen/Qwen3.5-4B |
| library_name: transformers |
| pipeline_tag: text-generation |
| language: |
| - en |
| tags: |
| - omics |
| - bioinformatics |
| - single-cell |
| - spatial-transcriptomics |
| - omicverse |
| - agent |
| - tool-calling |
| - scientific-coding |
| --- |
| |
| # OmicAI-4B |
|
|
| **OmicAI** is a family of language models optimized for **omics data analysis and |
| tool interaction** — writing and running real bioinformatics pipelines over the |
| [omicverse](https://github.com/omicverse/omicverse) toolkit. This card is the |
| **4B** member; 9B and 27B variants are in progress. |
|
|
| OmicAI is built on **Qwen3.5** and fine-tuned with **ms-swift** using **LoRA |
| adapters merged into the final weights**. |
|
|
| ## The OmicAI family |
|
|
| | Model | Base | Access | |
| |---|---|---| |
| | [OmicAI-0.8B](https://huggingface.co/omicverse/OmicAI-0.8B) | Qwen3.5-0.8B | Open — Apache-2.0 | |
| | **OmicAI-4B** *(this model)* | Qwen3.5-4B | Open — Apache-2.0 | |
| | [OmicAI-9B](https://huggingface.co/omicverse/OmicAI-9B) | MiMo-V2.6-Distill-Qwen-9B | Open — Apache-2.0 | |
| | [OmicAI-27B](https://huggingface.co/omicverse/OmicAI-27B) | Qwen3.8-27B | By application | |
|
|
| ## Training Data |
|
|
| The training trajectories are produced by **OmicOS** and **OmicVerse** through a |
| **paper-reproduction process over the last three years of publications**: the |
| systems retrieve recent omics papers, then attempt to **reproduce their analyses |
| end-to-end** from the raw data. Each attempt yields a multi-turn agent trajectory — |
| literature/context retrieval, code generation, **omicverse** tool calls, and |
| recovery from execution feedback. |
|
|
| A trajectory is kept only when an automatic **judge** confirms the deliverable is |
| legitimate (not a placeholder or answer-leak) **and** a **scoreboard** confirms |
| the reproduced result **matches or surpasses** the paper's reported (SOTA) numbers. |
| Runs that were rejected, contaminated, or degenerate are discarded. This yields a |
| high-signal corpus grounded in real, verifiable scientific reproduction. |
|
|
| ## Key Details |
|
|
| | | | |
| |---|---| |
| | **Base model** | Qwen/Qwen3.5-4B | |
| | **Method** | ms-swift 4.5.3 · LoRA (`target_modules=all-linear`, rank 16, α 32) → merged | |
| | **Precision** | bfloat16 (merged weights) | |
| | **Chat template** | `qwen3_5` — *inference template must match training* | |
| | **Training data** | agent trajectories from OmicOS/OmicVerse reproducing 3 years of omics papers; judge + scoreboard (MATCH / SURPASS-SOTA) validated | |
| | **Epochs / seq len** | 3 epochs · max length 16384 · packing + padding-free | |
|
|
| ## Evaluation |
|
|
| OmicAI is evaluated on two axes (results reported separately as the suite lands): |
|
|
| - **Domain / tool-use (the training target):** an omics-analysis benchmark with a |
| deterministic execution verifier, plus function-calling (BFCL) and multi-turn |
| agentic (τ-bench) suites. |
| - **General-capability regression (catastrophic forgetting check):** MMLU, GSM8K, |
| IFEval — to ensure domain gains do not come at the cost of general ability. |
|
|
| > Benchmark numbers are omitted here rather than estimated; they will be added |
| > once the evaluation harness is finalized. |
|
|
| ## Usage |
|
|
| ```python |
| from transformers import AutoModelForCausalLM, AutoTokenizer |
| |
| tok = AutoTokenizer.from_pretrained("omicverse/OmicAI-4B") |
| model = AutoModelForCausalLM.from_pretrained( |
| "omicverse/OmicAI-4B", torch_dtype="bfloat16", device_map="auto") |
| |
| msgs = [ |
| {"role": "system", "content": "You are an omics bioinformatics assistant."}, |
| {"role": "user", "content": "Load pbmc3k, run QC, normalize, and cluster with omicverse."}, |
| ] |
| prompt = tok.apply_chat_template(msgs, tokenize=False, add_generation_prompt=True) |
| out = model.generate(**tok(prompt, return_tensors="pt").to(model.device), max_new_tokens=512) |
| print(tok.decode(out[0], skip_special_tokens=True)) |
| ``` |
|
|
| Also loadable with **vLLM** (use the `qwen3_5` chat template). |
|
|
| ## Intended Use & Limitations |
|
|
| - **Intended:** research assistant for single-cell / bulk / spatial omics analysis |
| and omicverse pipeline generation. |
| - **Limitations:** a research model — outputs (especially generated analysis code) |
| should be **executed and verified** before use; it does not replace expert |
| judgement, and may hallucinate APIs outside the omicverse/scanpy surface it was |
| trained on. |
|
|
| ## Ecosystem |
|
|
| Part of the **omicOS / omicAI** effort and the omicverse ecosystem. OmicAI is also |
| the student model of **omicOS-RSI**, an infrastructure-level recursive |
| self-improvement pipeline in which the model, the omicverse toolkit, and the task |
| benchmark co-evolve under an execution-grounded verifier. |
|
|
| ## Citation |
|
|
| If omicverse is useful in your work, please cite the omicverse paper. A dedicated |
| OmicAI report is forthcoming. |
|
|
