Instructions to use InnovatorLab/InnovatorVL-v2 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use InnovatorLab/InnovatorVL-v2 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="InnovatorLab/InnovatorVL-v2")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("InnovatorLab/InnovatorVL-v2")
model = AutoModelForMultimodalLM.from_pretrained("InnovatorLab/InnovatorVL-v2", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use InnovatorLab/InnovatorVL-v2 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "InnovatorLab/InnovatorVL-v2"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "InnovatorLab/InnovatorVL-v2",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/InnovatorLab/InnovatorVL-v2

SGLang

How to use InnovatorLab/InnovatorVL-v2 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "InnovatorLab/InnovatorVL-v2" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "InnovatorLab/InnovatorVL-v2",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "InnovatorLab/InnovatorVL-v2" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "InnovatorLab/InnovatorVL-v2",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use InnovatorLab/InnovatorVL-v2 with Docker Model Runner:
```
docker model run hf.co/InnovatorLab/InnovatorVL-v2
```

turturtur250 commited on about 9 hours ago

Commit

9a7ffec

verified ·

1 Parent(s): 8220b8d

Upload README.md with huggingface_hub

Browse files

Files changed (1) hide show

README.md +241 -27

README.md CHANGED Viewed

@@ -69,33 +69,247 @@ Local non-thinking evaluation across all 22 benchmarks. The
 **InnovatorVL-v2 (Ours)** column is placed first and emphasised throughout;
 bold marks the best score in each row, including ties.
-| Benchmark | **InnovatorVL-v2 (Ours)** | Qwen3-VL-8B | Gemma4-12B-IT | InternVL3.5-8B | Step3-VL-10B | Qwen3.5-9B | Qwen3.5-9B-Base |
-| --- | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
-| *General* | | | | | | | |
-| AI2D | **88.34** | 83.74 | 77.49 | 82.22 | 83.55 | 85.46 | 63.28 |
-| ChartQA | **89.24** | 85.24 | 52.12 | 87.20 | 59.68 | 85.28 | 61.04 |
-| InfographicVQA (val) | **87.69** | 83.72 | 53.86 | 74.23 | 71.27 | 83.34 | 76.70 |
-| DocVQA (val) | **96.76** | 93.66 | 83.10 | 92.37 | 92.33 | 94.98 | 92.60 |
-| MMBench (EN dev) | **87.97** | 83.67 | 81.62 | 83.51 | 85.47 | 85.48 | 86.34 |
-| MME-RealWorld | **71.33** | 63.29 | 35.02 | 62.66 | 30.32 | 60.08 | 46.79 |
-| MMStar | **73.15** | 59.94 | 67.11 | 66.38 | 63.24 | 70.52 | 71.92 |
-| OCRBench | **88.20** | 87.00 | 77.80 | 83.20 | 85.30 | 83.10 | 86.30 |
-| RealWorldQA | **76.73** | 68.76 | 65.23 | 63.53 | 63.66 | 74.51 | 72.81 |
-| SEED-Bench-2-Plus | **72.38** | 71.28 | 68.25 | 70.58 | 69.35 | 70.84 | 64.08 |
-| V\*Bench | **90.58** | 84.82 | 52.36 | 70.68 | 67.02 | 84.29 | 82.20 |
-| SimpleVQA | 39.80 | 39.31 | 40.27 | 35.47 | **46.24** | 44.40 | 45.73 |
-| *Science* | | | | | | | |
-| EMVista | **49.62** | 44.27 | 39.30 | 46.81 | 43.03 | 43.35 | 37.84 |
-| ScienceQA | 93.89 | 91.91 | 89.39 | **95.94** | 85.76 | 93.35 | 89.53 |
-| MolParse | **75.40** | 4.35 | 4.95 | 19.05 | 4.25 | 7.00 | 5.20 |
-| OpenRxn | **56.50** | 9.25 | 12.60 | 14.05 | 12.70 | 14.40 | 15.65 |
-| RxnBench (EN) | **89.05** | 82.75 | 85.31 | 80.46 | 73.70 | 88.52 | 87.67 |
-| SmolInstruct_v2 | 27.08 | 20.15 | 24.18 | **40.27** | 6.87 | 14.38 | 27.44 |
-| ProteinLMBench | 61.97 | 59.32 | 56.67 | 61.76 | 56.25 | 54.77 | **65.89** |
-| SFE (EN) | 24.13 | **28.98** | 22.41 | 19.63 | 14.44 | 20.24 | 20.12 |
-| *Spatial* | | | | | | | |
-| EmbSpatial-Bench | **77.36** | 74.90 | 73.54 | 73.24 | 68.10 | 75.77 | 75.00 |
-| ERQA | **45.75** | 38.80 | 42.25 | 41.25 | 24.00 | 45.00 | **45.75** |
 ## Usage

 **InnovatorVL-v2 (Ours)** column is placed first and emphasised throughout;
 bold marks the best score in each row, including ties.
+<div class="max-w-full overflow-auto">
+<table>
+<thead>
+<tr>
+<th>Benchmark</th>
+<th align="center" style="background-color:#6d28d9;color:#ffffff">InnovatorVL-v2 (Ours)</th>
+<th align="center">Qwen3-VL-8B</th>
+<th align="center">Gemma4-12B-IT</th>
+<th align="center">InternVL3.5-8B</th>
+<th align="center">Step3-VL-10B</th>
+<th align="center">Qwen3.5-9B</th>
+<th align="center">Qwen3.5-9B-Base</th>
+</tr>
+</thead>
+<tbody>
+<tr><td colspan="8"><i>General</i></td></tr>
+<tr>
+<td>AI2D</td>
+<td align="center" style="background-color:#7c3aed;color:#ffffff"><b>88.34</b></td>
+<td align="center">83.74</td>
+<td align="center">77.49</td>
+<td align="center">82.22</td>
+<td align="center">83.55</td>
+<td align="center">85.46</td>
+<td align="center">63.28</td>
+</tr>
+<tr>
+<td>ChartQA</td>
+<td align="center" style="background-color:#7c3aed;color:#ffffff"><b>89.24</b></td>
+<td align="center">85.24</td>
+<td align="center">52.12</td>
+<td align="center">87.20</td>
+<td align="center">59.68</td>
+<td align="center">85.28</td>
+<td align="center">61.04</td>
+</tr>
+<tr>
+<td>InfographicVQA (val)</td>
+<td align="center" style="background-color:#7c3aed;color:#ffffff"><b>87.69</b></td>
+<td align="center">83.72</td>
+<td align="center">53.86</td>
+<td align="center">74.23</td>
+<td align="center">71.27</td>
+<td align="center">83.34</td>
+<td align="center">76.70</td>
+</tr>
+<tr>
+<td>DocVQA (val)</td>
+<td align="center" style="background-color:#7c3aed;color:#ffffff"><b>96.76</b></td>
+<td align="center">93.66</td>
+<td align="center">83.10</td>
+<td align="center">92.37</td>
+<td align="center">92.33</td>
+<td align="center">94.98</td>
+<td align="center">92.60</td>
+</tr>
+<tr>
+<td>MMBench (EN dev)</td>
+<td align="center" style="background-color:#7c3aed;color:#ffffff"><b>87.97</b></td>
+<td align="center">83.67</td>
+<td align="center">81.62</td>
+<td align="center">83.51</td>
+<td align="center">85.47</td>
+<td align="center">85.48</td>
+<td align="center">86.34</td>
+</tr>
+<tr>
+<td>MME-RealWorld</td>
+<td align="center" style="background-color:#7c3aed;color:#ffffff"><b>71.33</b></td>
+<td align="center">63.29</td>
+<td align="center">35.02</td>
+<td align="center">62.66</td>
+<td align="center">30.32</td>
+<td align="center">60.08</td>
+<td align="center">46.79</td>
+</tr>
+<tr>
+<td>MMStar</td>
+<td align="center" style="background-color:#7c3aed;color:#ffffff"><b>73.15</b></td>
+<td align="center">59.94</td>
+<td align="center">67.11</td>
+<td align="center">66.38</td>
+<td align="center">63.24</td>
+<td align="center">70.52</td>
+<td align="center">71.92</td>
+</tr>
+<tr>
+<td>OCRBench</td>
+<td align="center" style="background-color:#7c3aed;color:#ffffff"><b>88.20</b></td>
+<td align="center">87.00</td>
+<td align="center">77.80</td>
+<td align="center">83.20</td>
+<td align="center">85.30</td>
+<td align="center">83.10</td>
+<td align="center">86.30</td>
+</tr>
+<tr>
+<td>RealWorldQA</td>
+<td align="center" style="background-color:#7c3aed;color:#ffffff"><b>76.73</b></td>
+<td align="center">68.76</td>
+<td align="center">65.23</td>
+<td align="center">63.53</td>
+<td align="center">63.66</td>
+<td align="center">74.51</td>
+<td align="center">72.81</td>
+</tr>
+<tr>
+<td>SEED-Bench-2-Plus</td>
+<td align="center" style="background-color:#7c3aed;color:#ffffff"><b>72.38</b></td>
+<td align="center">71.28</td>
+<td align="center">68.25</td>
+<td align="center">70.58</td>
+<td align="center">69.35</td>
+<td align="center">70.84</td>
+<td align="center">64.08</td>
+</tr>
+<tr>
+<td>V*Bench</td>
+<td align="center" style="background-color:#7c3aed;color:#ffffff"><b>90.58</b></td>
+<td align="center">84.82</td>
+<td align="center">52.36</td>
+<td align="center">70.68</td>
+<td align="center">67.02</td>
+<td align="center">84.29</td>
+<td align="center">82.20</td>
+</tr>
+<tr>
+<td>SimpleVQA</td>
+<td align="center" style="background-color:#7c3aed;color:#ffffff">39.80</td>
+<td align="center">39.31</td>
+<td align="center">40.27</td>
+<td align="center">35.47</td>
+<td align="center"><b>46.24</b></td>
+<td align="center">44.40</td>
+<td align="center">45.73</td>
+</tr>
+<tr><td colspan="8"><i>Science</i></td></tr>
+<tr>
+<td>EMVista</td>
+<td align="center" style="background-color:#7c3aed;color:#ffffff"><b>49.62</b></td>
+<td align="center">44.27</td>
+<td align="center">39.30</td>
+<td align="center">46.81</td>
+<td align="center">43.03</td>
+<td align="center">43.35</td>
+<td align="center">37.84</td>
+</tr>
+<tr>
+<td>ScienceQA</td>
+<td align="center" style="background-color:#7c3aed;color:#ffffff">93.89</td>
+<td align="center">91.91</td>
+<td align="center">89.39</td>
+<td align="center"><b>95.94</b></td>
+<td align="center">85.76</td>
+<td align="center">93.35</td>
+<td align="center">89.53</td>
+</tr>
+<tr>
+<td>MolParse</td>
+<td align="center" style="background-color:#7c3aed;color:#ffffff"><b>75.40</b></td>
+<td align="center">4.35</td>
+<td align="center">4.95</td>
+<td align="center">19.05</td>
+<td align="center">4.25</td>
+<td align="center">7.00</td>
+<td align="center">5.20</td>
+</tr>
+<tr>
+<td>OpenRxn</td>
+<td align="center" style="background-color:#7c3aed;color:#ffffff"><b>56.50</b></td>
+<td align="center">9.25</td>
+<td align="center">12.60</td>
+<td align="center">14.05</td>
+<td align="center">12.70</td>
+<td align="center">14.40</td>
+<td align="center">15.65</td>
+</tr>
+<tr>
+<td>RxnBench (EN)</td>
+<td align="center" style="background-color:#7c3aed;color:#ffffff"><b>89.05</b></td>
+<td align="center">82.75</td>
+<td align="center">85.31</td>
+<td align="center">80.46</td>
+<td align="center">73.70</td>
+<td align="center">88.52</td>
+<td align="center">87.67</td>
+</tr>
+<tr>
+<td>SmolInstruct_v2</td>
+<td align="center" style="background-color:#7c3aed;color:#ffffff">27.08</td>
+<td align="center">20.15</td>
+<td align="center">24.18</td>
+<td align="center"><b>40.27</b></td>
+<td align="center">6.87</td>
+<td align="center">14.38</td>
+<td align="center">27.44</td>
+</tr>
+<tr>
+<td>ProteinLMBench</td>
+<td align="center" style="background-color:#7c3aed;color:#ffffff">61.97</td>
+<td align="center">59.32</td>
+<td align="center">56.67</td>
+<td align="center">61.76</td>
+<td align="center">56.25</td>
+<td align="center">54.77</td>
+<td align="center"><b>65.89</b></td>
+</tr>
+<tr>
+<td>SFE (EN)</td>
+<td align="center" style="background-color:#7c3aed;color:#ffffff">24.13</td>
+<td align="center"><b>28.98</b></td>
+<td align="center">22.41</td>
+<td align="center">19.63</td>
+<td align="center">14.44</td>
+<td align="center">20.24</td>
+<td align="center">20.12</td>
+</tr>
+<tr><td colspan="8"><i>Spatial</i></td></tr>
+<tr>
+<td>EmbSpatial-Bench</td>
+<td align="center" style="background-color:#7c3aed;color:#ffffff"><b>77.36</b></td>
+<td align="center">74.90</td>
+<td align="center">73.54</td>
+<td align="center">73.24</td>
+<td align="center">68.10</td>
+<td align="center">75.77</td>
+<td align="center">75.00</td>
+</tr>
+<tr>
+<td>ERQA</td>
+<td align="center" style="background-color:#7c3aed;color:#ffffff"><b>45.75</b></td>
+<td align="center">38.80</td>
+<td align="center">42.25</td>
+<td align="center">41.25</td>
+<td align="center">24.00</td>
+<td align="center">45.00</td>
+<td align="center"><b>45.75</b></td>
+</tr>
+</tbody>
+</table>
+</div>
 ## Usage