Image-Text-to-Text
Transformers
Safetensors
English
qwen3_5
optics
photonics
physics
scientific-computing
code
reinforcement-learning
conversational
Instructions to use HololabTHU/HoloModel-Research with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use HololabTHU/HoloModel-Research with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("image-text-to-text", model="HololabTHU/HoloModel-Research") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] pipe(text=messages)# pip install -U transformers accelerate # Load model directly from transformers import AutoProcessor, AutoModelForMultimodalLM processor = AutoProcessor.from_pretrained("HololabTHU/HoloModel-Research") model = AutoModelForMultimodalLM.from_pretrained("HololabTHU/HoloModel-Research", device_map="auto") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] inputs = processor.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=256) print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use HololabTHU/HoloModel-Research with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "HololabTHU/HoloModel-Research" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "HololabTHU/HoloModel-Research", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker
docker model run hf.co/HololabTHU/HoloModel-Research
- SGLang
How to use HololabTHU/HoloModel-Research with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "HololabTHU/HoloModel-Research" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "HololabTHU/HoloModel-Research", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "HololabTHU/HoloModel-Research" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "HololabTHU/HoloModel-Research", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }' - Docker Model Runner
How to use HololabTHU/HoloModel-Research with Docker Model Runner:
docker model run hf.co/HololabTHU/HoloModel-Research
OPB scores under the array-input protocol
Browse files- .gitattributes +1 -0
- README.md +5 -5
- assets/bench_bars.png +2 -2
.gitattributes
CHANGED
|
@@ -49,3 +49,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
|
| 49 |
"/tokenizer.json" filter=lfs diff=lfs merge=lfs -text
|
| 50 |
"/vocab.json" filter=lfs diff=lfs merge=lfs -textassets/logo_hm.png filter=lfs diff=lfs merge=lfs -text
|
| 51 |
tokenizer.json filter=lfs diff=lfs merge=lfs -text
|
|
|
|
|
|
| 49 |
"/tokenizer.json" filter=lfs diff=lfs merge=lfs -text
|
| 50 |
"/vocab.json" filter=lfs diff=lfs merge=lfs -textassets/logo_hm.png filter=lfs diff=lfs merge=lfs -text
|
| 51 |
tokenizer.json filter=lfs diff=lfs merge=lfs -text
|
| 52 |
+
assets/bench_bars.png filter=lfs diff=lfs merge=lfs -text
|
README.md
CHANGED
|
@@ -35,9 +35,9 @@ Frontier AI scientists depend on proprietary models that are costly, closed and
|
|
| 35 |
|
| 36 |
### Key Features
|
| 37 |
|
| 38 |
-
- 🔬 **Built for optical physics.** On **Optics-Phy-Bench**, 217 multi-step computations reproduced from 217 published papers, HoloModel-Research solves **
|
| 39 |
- 💻 **Stronger coder, not only in optics.** VCP code dev **+8.8** (P = 0.0015) and optics coding proxy **+5.4** over the base model.
|
| 40 |
-
- 🥊 **Punches above its size.** With thinking off, HoloModel-Research
|
| 41 |
- 🔒 **Private and reproducible.** Fixed weights, temperature-0 evaluation and fully local deployment: data and unpublished designs never leave the lab.
|
| 42 |
|
| 43 |
## 2. Model Summary
|
|
@@ -64,8 +64,8 @@ Frontier AI scientists depend on proprietary models that are costly, closed and
|
|
| 64 |
|
| 65 |
| Benchmark | Items | Qwen3.8-27B | Kimi-K3.1-Pro | GLM-5.3<br/><sub>reasoning low</sub> | DeepSeek V4.1 Flash | **HoloModel-Research** |
|
| 66 |
|:---|---:|---:|---:|---:|---:|---:|
|
| 67 |
-
| Optics-Phy-Bench, problems | 217 |
|
| 68 |
-
| Optics-Phy-Bench, steps | 833 |
|
| 69 |
| VCP code dev | 194 | 77.8 | 42.3 | 72.2 | 75.3 | **86.6** |
|
| 70 |
| Optics coding proxy | 300 | 81.3 | 71.0 | 69.7 | 83.7 | **86.7** |
|
| 71 |
| Physics dev | 198 | 85.4 | 72.7 | 65.7 | 84.3 | **87.9** |
|
|
@@ -77,7 +77,7 @@ Frontier AI scientists depend on proprietary models that are costly, closed and
|
|
| 77 |
|
| 78 |
- All models: thinking disabled (GLM-5.3 at its lowest reasoning setting), temperature 0 (Kimi-K3.1-Pro: 0.6, the lowest its API accepts with thinking off), 14,000-token limit, one run per benchmark.
|
| 79 |
- HoloModel-Research and the base model are compared item by item; confidence intervals are paired bootstrap 95 % intervals (10,000 resamples) and P values exact McNemar tests.
|
| 80 |
-
- Optics-Phy-Bench: a problem counts as solved only when every step function reproduces the reference outputs within tolerance.
|
| 81 |
- **Decontamination:** no training prompt shares a 13-word sequence with Optics-Phy-Bench, physics dev or the coding proxy. VCP code dev is a split of the public `open-r1/verifiable-coding-problems-python` dataset; six items with more than half of their text inside one training prompt were removed, leaving 194.
|
| 82 |
- The checkpoint (iteration 30) was chosen by a rule fixed before training, not by any benchmark.
|
| 83 |
- Frontier models with reasoning enabled still score higher on Optics-Phy-Bench.
|
|
|
|
| 35 |
|
| 36 |
### Key Features
|
| 37 |
|
| 38 |
+
- 🔬 **Built for optical physics.** On **Optics-Phy-Bench**, 217 multi-step computations reproduced from 217 published papers, HoloModel-Research solves **69.1 %** of problems against **46.5 %** for its base model: **+22.6 points, a 49 % relative gain** (95 % CI +15.7 to +29.5).
|
| 39 |
- 💻 **Stronger coder, not only in optics.** VCP code dev **+8.8** (P = 0.0015) and optics coding proxy **+5.4** over the base model.
|
| 40 |
+
- 🥊 **Punches above its size.** With thinking off, HoloModel-Research is ahead of **DeepSeek V4.1 Flash** (552B total parameters) on Optics-Phy-Bench (**69.1 %** against 62.7 % of problems; +2.9 points on steps, P = 0.025) and by **11.3 points** on VCP code dev; it leads **GLM-5.3** and **Kimi-K3.1-Pro** on all five optics and coding sets.
|
| 41 |
- 🔒 **Private and reproducible.** Fixed weights, temperature-0 evaluation and fully local deployment: data and unpublished designs never leave the lab.
|
| 42 |
|
| 43 |
## 2. Model Summary
|
|
|
|
| 64 |
|
| 65 |
| Benchmark | Items | Qwen3.8-27B | Kimi-K3.1-Pro | GLM-5.3<br/><sub>reasoning low</sub> | DeepSeek V4.1 Flash | **HoloModel-Research** |
|
| 66 |
|:---|---:|---:|---:|---:|---:|---:|
|
| 67 |
+
| Optics-Phy-Bench, problems | 217 | 46.5 | 41.9 | 39.2 | 62.7 | **69.1** |
|
| 68 |
+
| Optics-Phy-Bench, steps | 833 | 82.0 | 79.5 | 77.8 | 88.1 | **91.0** |
|
| 69 |
| VCP code dev | 194 | 77.8 | 42.3 | 72.2 | 75.3 | **86.6** |
|
| 70 |
| Optics coding proxy | 300 | 81.3 | 71.0 | 69.7 | 83.7 | **86.7** |
|
| 71 |
| Physics dev | 198 | 85.4 | 72.7 | 65.7 | 84.3 | **87.9** |
|
|
|
|
| 77 |
|
| 78 |
- All models: thinking disabled (GLM-5.3 at its lowest reasoning setting), temperature 0 (Kimi-K3.1-Pro: 0.6, the lowest its API accepts with thinking off), 14,000-token limit, one run per benchmark.
|
| 79 |
- HoloModel-Research and the base model are compared item by item; confidence intervals are paired bootstrap 95 % intervals (10,000 resamples) and P values exact McNemar tests.
|
| 80 |
+
- Optics-Phy-Bench: test inputs are passed to each step function as numpy arrays; a problem counts as solved only when every step function reproduces the reference outputs within tolerance.
|
| 81 |
- **Decontamination:** no training prompt shares a 13-word sequence with Optics-Phy-Bench, physics dev or the coding proxy. VCP code dev is a split of the public `open-r1/verifiable-coding-problems-python` dataset; six items with more than half of their text inside one training prompt were removed, leaving 194.
|
| 82 |
- The checkpoint (iteration 30) was chosen by a rule fixed before training, not by any benchmark.
|
| 83 |
- Frontier models with reasoning enabled still score higher on Optics-Phy-Bench.
|
assets/bench_bars.png
CHANGED
|
Git LFS Details
|
|
Git LFS Details
|