HoloModel-Research / README.md
HololabTHU's picture
Card: drop reasoning-low label
40557e1 verified
|
Raw History Blame Contribute Delete
7.41 kB
---
license: cc-by-nc-4.0
base_model: Qwen/Qwen3.8-27B
library_name: transformers
pipeline_tag: image-text-to-text
language:
- en
tags:
- optics
- photonics
- physics
- scientific-computing
- code
- reinforcement-learning
---
<div align="center">
<img src="assets/banner.webp" width="85%" alt="HoloModel"/>
</div>
<div align="center" style="line-height: 1;">
<a href="https://huggingface.co/HololabTHU/HoloModel-Research"><img alt="Model" src="https://img.shields.io/badge/%F0%9F%A4%97%20Model-HoloModel--Research-5799D1"/></a>
<a href="https://huggingface.co/datasets/HololabTHU/HoloModel-Bench"><img alt="Benchmark" src="https://img.shields.io/badge/%F0%9F%93%8A%20Benchmark-HoloModel--Bench-5799D1"/></a>
<a href="https://huggingface.co/HololabTHU"><img alt="Org" src="https://img.shields.io/badge/Hugging%20Face-HololabTHU-FFD21E"/></a>
<a href="LICENSE"><img alt="License" src="https://img.shields.io/badge/License-CC%20BY--NC%204.0-F6DC8A"/></a>
</div>
## 1. Model Introduction
**HoloModel-Research** is an open-weight language model for autonomous research in optical physics. It writes correct numerical code for optics and physics problems, and it is the reinforcement-learning branch of **HoloModel**, the model family that drives the *Optics Scientists* agent team. The family has two models: **HoloModel-Knowledge**, enhanced with optics knowledge, and **HoloModel-Research**. The team consists of agents that can be deployed locally and carries open optics questions from the literature to verified discoveries in holography, metasurfaces, spectral sensing and diffractive optics.
### Background
Frontier AI scientists depend on proprietary models that are costly, closed and hard to reproduce. HoloModel takes the opposite route: HoloModel-Research is a 27-billion-parameter open model (Qwen3.8-27B) post-trained with reinforcement learning, and it runs on a single GPU inside the laboratory.
### Key Features
- 🔬 **Built for optical physics.** On **Optics-Phy-Bench**, 217 multi-step computations reproduced from 217 published papers, HoloModel-Research solves **69.1 %** of problems against **46.5 %** for its base model: **+22.6 points, a 49 % relative gain** (95 % CI +15.7 to +29.5).
- 💻 **Stronger coder, not only in optics.** VCP code dev **+8.8** (P = 0.0015) and optics coding proxy **+5.4** over the base model.
- 🥊 **Punches above its size.** With thinking off, HoloModel-Research is ahead of **DeepSeek V4.1 Flash** (552B total parameters) on Optics-Phy-Bench (**69.1 %** against 62.7 % of problems; +2.9 points on steps, P = 0.025) and by **11.3 points** on VCP code dev; it leads **GLM-5.3** and **Kimi-K3.1-Pro** on all five optics and coding sets.
- 🔒 **Private and reproducible.** Fixed weights, temperature-0 evaluation and fully local deployment: data and unpublished designs never leave the lab.
## 2. Model Summary
| | HoloModel-Research |
|---|---|
| **Developers** | Jinwen Wei, Liangcai Cao (Department of Precision Instrument, Tsinghua University) |
| **Architecture** | Causal language model with vision encoder; 64 layers, hybrid linear and full attention |
| **Parameters** | 27B (bfloat16, 18 safetensors shards, 52 GB) |
| **Context length** | 262,144 tokens |
| **Base model** | Qwen3.8-27B |
| **Post-training** | Reinforcement learning |
| **License** | CC BY-NC 4.0 (base model: Apache-2.0) |
## 3. Evaluation Results
<div align="center">
<img src="assets/bench_bars.png" width="95%" alt="Optics and coding benchmarks"/>
</div>
### Optics, physics and coding
<div align="center">
| Benchmark | Items | Qwen3.8-27B | Kimi-K3.1-Pro | GLM-5.3 | DeepSeek V4.1 Flash | **HoloModel-Research** |
|:---|---:|---:|---:|---:|---:|---:|
| Optics-Phy-Bench, problems | 217 | 46.5 | 41.9 | 39.2 | 62.7 | **69.1** |
| Optics-Phy-Bench, steps | 833 | 82.0 | 79.5 | 77.8 | 88.1 | **91.0** |
| VCP code dev | 194 | 77.8 | 42.3 | 72.2 | 75.3 | **86.6** |
| Optics coding proxy | 300 | 81.3 | 71.0 | 69.7 | 83.7 | **86.7** |
| Physics dev | 198 | 85.4 | 72.7 | 65.7 | 84.3 | **87.9** |
</div>
<details>
<summary><b>Evaluation protocol</b></summary>
- All models: thinking disabled (GLM-5.3 at its lowest reasoning setting), temperature 0 (Kimi-K3.1-Pro: 0.6, the lowest its API accepts with thinking off), 14,000-token limit, one run per benchmark.
- HoloModel-Research and the base model are compared item by item; confidence intervals are paired bootstrap 95 % intervals (10,000 resamples) and P values exact McNemar tests.
- Optics-Phy-Bench: test inputs are passed to each step function as numpy arrays; a problem counts as solved only when every step function reproduces the reference outputs within tolerance.
- **Decontamination:** no training prompt shares a 13-word sequence with Optics-Phy-Bench, physics dev or the coding proxy. VCP code dev is a split of the public `open-r1/verifiable-coding-problems-python` dataset; six items with more than half of their text inside one training prompt were removed, leaving 194.
- The checkpoint (iteration 30) was chosen by a rule fixed before training, not by any benchmark.
- Frontier models with reasoning enabled still score higher on Optics-Phy-Bench.
</details>
## 4. Quickstart
### Transformers
```python
from transformers import AutoProcessor, AutoModelForImageTextToText
model_id = "HololabTHU/HoloModel-Research"
processor = AutoProcessor.from_pretrained(model_id)
model = AutoModelForImageTextToText.from_pretrained(model_id, torch_dtype="auto", device_map="auto")
messages = [{"role": "user", "content": [{"type": "text", "text":
"Write a Python function that propagates a 2D complex field by distance z with the angular spectrum method."}]}]
inputs = processor.apply_chat_template(messages, add_generation_prompt=True, tokenize=True,
return_dict=True, return_tensors="pt",
enable_thinking=False).to(model.device)
out = model.generate(**inputs, max_new_tokens=4096, do_sample=False)
print(processor.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))
```
### vLLM
```bash
vllm serve HololabTHU/HoloModel-Research --max-model-len 32768
```
Send requests with `"chat_template_kwargs": {"enable_thinking": false}` and `"temperature": 0` to match the evaluation setting. One 80–96 GB GPU holds the bfloat16 weights.
## 5. Intended Use
- Writing and checking numerical code for optics and physics: diffraction and propagation, holography and phase retrieval, metasurfaces and gratings, spectroscopy, thin films, resonators and coupled-mode models.
- The language-model core of local research agents.
- Generated code should be run and checked against physics (solvers, conservation laws, known limits) before its results are trusted.
## 6. License
The HoloModel-Research weights and this card are released under **CC BY-NC 4.0**: free to use, share and adapt for **non-commercial** purposes, with attribution. The base model Qwen3.8-27B is licensed by the Qwen team under Apache-2.0; its license is included as `LICENSE-Qwen` and its original card as `README-Qwen3.8-27B.md`.
## 7. Citation
```bibtex
@misc{holomodel2026,
title = {HoloModel: language models for optics and physics research and knowledge},
author = {Wei, Jinwen and Cao, Liangcai},
year = {2026},
howpublished = {\url{https://huggingface.co/HololabTHU/HoloModel-Research}}
}
```