| --- |
| license: other |
| library_name: transformers |
| tags: |
| - multimodal |
| - vision-language |
| - reasoning |
| - science |
| - rl |
| - qwen3.5 |
| --- |
| |
| # InnovatorVL-v2 |
|
|
| <video src="https://huggingface.co/InnovatorLab/InnovatorVL-v2/resolve/main/InnovatorVL-v2-Main-Promo-v7-28s.mp4" controls width="100%" preload="metadata" playsinline></video> |
|
|
| **InnovatorVL-v2 is a dense 9B-parameter multimodal model that delivers |
| state-of-the-art scientific visual understanding at its scale, without giving up |
| general multimodal ability.** Built on Qwen3.5-9B-Base, it is trained through a |
| pipeline centred on systematic data governance — recursive self-improvement for |
| data scoring and filtering, rank-based mixture optimisation, failure-guided data |
| synthesis, and Visual Evidence-Calibrated Reinforcement Learning (VEC-RL). |
|
|
| Evaluated on 22 benchmarks with thinking disabled for every compared model, |
| InnovatorVL-v2 attains **the highest average in all three categories**: |
|
|
| | Category | **InnovatorVL-v2 (Ours)** | Margin over strongest baseline | |
| | --- | :---: | :---: | |
| | General (12 benchmarks) | **80.18** | +3.32 | |
| | Science (8 benchmarks) | **59.71** | **+12.46** | |
| | Spatial (2 benchmarks) | **61.56** | +1.17 | |
|
|
| Headline results: |
|
|
| - **First on 18 of the 22 benchmarks** among the models displayed, and |
| **beats Qwen3.5-9B on 21 of the 22** — the single exception is SimpleVQA. |
| - **Scientific understanding is the standout gain**: +17.70 points over |
| Qwen3.5-9B on the Science average. |
| - **MolParse 75.40** and **OpenRxn 56.50** — ahead of the best compared baseline |
| by **+56.35** and **+40.85** points respectively. Molecular structure parsing |
| and reaction understanding are near-solved relative to the field. |
| - **Best general score on 11 of the 12 general benchmarks**, including |
| DocVQA 96.76, V\*Bench 90.58, ChartQA 89.24, AI2D 88.34, |
| OCRBench 88.20 and InfographicVQA 87.69. |
| - **Highest Spatial average**, with EmbSpatial-Bench 77.36 and ERQA 45.75. |
| |
| ## Performance |
| |
|  |
| |
| **Performance overview across 22 benchmarks.** Blue bars indicate our model; the |
| remaining bars show the compared models available in each panel. Our model ranks |
| first on 18 of the 22 benchmarks among the models displayed and outperforms |
| Qwen3.5-9B on 21 of the 22. The signed value in the upper-right corner of each |
| panel is the score-point difference against Qwen3.5-9B. All results use our |
| local evaluation setting with thinking disabled. Higher scores are better; |
| vertical scales vary across panels. |
| |
|  |
| |
| **Category-level performance with thinking disabled.** Panels (a), (b) and (c) |
| report unweighted averages over 12 general, eight scientific and two spatial |
| benchmarks respectively. All seven models have complete coverage. Blue bars |
| denote InnovatorVL-v2 (Ours); the remaining bars are the six baselines, including |
| Qwen3.5-9B-Base. Models appear in the same order across panels, sorted by their |
| General scores. All panels use the same 0–100 scale. |
| |
| ### Full results |
| |
| Local non-thinking evaluation across all 22 benchmarks. The |
| **InnovatorVL-v2 (Ours)** column is placed first and emphasised throughout; |
| bold marks the best score in each row, including ties. |
| |
| <div class="max-w-full overflow-auto"> |
| <table> |
| <thead> |
| <tr> |
| <th>Benchmark</th> |
| <th align="center" style="background-color:#a78bfa;color:#1f2937">InnovatorVL-v2 (Ours)</th> |
| <th align="center">Qwen3-VL-8B</th> |
| <th align="center">Gemma4-12B-IT</th> |
| <th align="center">InternVL3.5-8B</th> |
| <th align="center">Step3-VL-10B</th> |
| <th align="center">Qwen3.5-9B</th> |
| <th align="center">Qwen3.5-9B-Base</th> |
| </tr> |
| </thead> |
| <tbody> |
| <tr><td colspan="8"><i>General</i></td></tr> |
| <tr> |
| <td>AI2D</td> |
| <td align="center" style="background-color:#be9df6;color:#1f2937"><b>88.34</b></td> |
| <td align="center">83.74</td> |
| <td align="center">77.49</td> |
| <td align="center">82.22</td> |
| <td align="center">83.55</td> |
| <td align="center">85.46</td> |
| <td align="center">63.28</td> |
| </tr> |
| <tr> |
| <td>ChartQA</td> |
| <td align="center" style="background-color:#be9df6;color:#1f2937"><b>89.24</b></td> |
| <td align="center">85.24</td> |
| <td align="center">52.12</td> |
| <td align="center">87.20</td> |
| <td align="center">59.68</td> |
| <td align="center">85.28</td> |
| <td align="center">61.04</td> |
| </tr> |
| <tr> |
| <td>InfographicVQA (val)</td> |
| <td align="center" style="background-color:#be9df6;color:#1f2937"><b>87.69</b></td> |
| <td align="center">83.72</td> |
| <td align="center">53.86</td> |
| <td align="center">74.23</td> |
| <td align="center">71.27</td> |
| <td align="center">83.34</td> |
| <td align="center">76.70</td> |
| </tr> |
| <tr> |
| <td>DocVQA (val)</td> |
| <td align="center" style="background-color:#be9df6;color:#1f2937"><b>96.76</b></td> |
| <td align="center">93.66</td> |
| <td align="center">83.10</td> |
| <td align="center">92.37</td> |
| <td align="center">92.33</td> |
| <td align="center">94.98</td> |
| <td align="center">92.60</td> |
| </tr> |
| <tr> |
| <td>MMBench (EN dev)</td> |
| <td align="center" style="background-color:#be9df6;color:#1f2937"><b>87.97</b></td> |
| <td align="center">83.67</td> |
| <td align="center">81.62</td> |
| <td align="center">83.51</td> |
| <td align="center">85.47</td> |
| <td align="center">85.48</td> |
| <td align="center">86.34</td> |
| </tr> |
| <tr> |
| <td>MME-RealWorld</td> |
| <td align="center" style="background-color:#be9df6;color:#1f2937"><b>71.33</b></td> |
| <td align="center">63.29</td> |
| <td align="center">35.02</td> |
| <td align="center">62.66</td> |
| <td align="center">30.32</td> |
| <td align="center">60.08</td> |
| <td align="center">46.79</td> |
| </tr> |
| <tr> |
| <td>MMStar</td> |
| <td align="center" style="background-color:#be9df6;color:#1f2937"><b>73.15</b></td> |
| <td align="center">59.94</td> |
| <td align="center">67.11</td> |
| <td align="center">66.38</td> |
| <td align="center">63.24</td> |
| <td align="center">70.52</td> |
| <td align="center">71.92</td> |
| </tr> |
| <tr> |
| <td>OCRBench</td> |
| <td align="center" style="background-color:#be9df6;color:#1f2937"><b>88.20</b></td> |
| <td align="center">87.00</td> |
| <td align="center">77.80</td> |
| <td align="center">83.20</td> |
| <td align="center">85.30</td> |
| <td align="center">83.10</td> |
| <td align="center">86.30</td> |
| </tr> |
| <tr> |
| <td>RealWorldQA</td> |
| <td align="center" style="background-color:#be9df6;color:#1f2937"><b>76.73</b></td> |
| <td align="center">68.76</td> |
| <td align="center">65.23</td> |
| <td align="center">63.53</td> |
| <td align="center">63.66</td> |
| <td align="center">74.51</td> |
| <td align="center">72.81</td> |
| </tr> |
| <tr> |
| <td>SEED-Bench-2-Plus</td> |
| <td align="center" style="background-color:#be9df6;color:#1f2937"><b>72.38</b></td> |
| <td align="center">71.28</td> |
| <td align="center">68.25</td> |
| <td align="center">70.58</td> |
| <td align="center">69.35</td> |
| <td align="center">70.84</td> |
| <td align="center">64.08</td> |
| </tr> |
| <tr> |
| <td>V*Bench</td> |
| <td align="center" style="background-color:#be9df6;color:#1f2937"><b>90.58</b></td> |
| <td align="center">84.82</td> |
| <td align="center">52.36</td> |
| <td align="center">70.68</td> |
| <td align="center">67.02</td> |
| <td align="center">84.29</td> |
| <td align="center">82.20</td> |
| </tr> |
| <tr> |
| <td>SimpleVQA</td> |
| <td align="center" style="background-color:#be9df6;color:#1f2937">39.80</td> |
| <td align="center">39.31</td> |
| <td align="center">40.27</td> |
| <td align="center">35.47</td> |
| <td align="center"><b>46.24</b></td> |
| <td align="center">44.40</td> |
| <td align="center">45.73</td> |
| </tr> |
| <tr><td colspan="8"><i>Science</i></td></tr> |
| <tr> |
| <td>EMVista</td> |
| <td align="center" style="background-color:#be9df6;color:#1f2937"><b>49.62</b></td> |
| <td align="center">44.27</td> |
| <td align="center">39.30</td> |
| <td align="center">46.81</td> |
| <td align="center">43.03</td> |
| <td align="center">43.35</td> |
| <td align="center">37.84</td> |
| </tr> |
| <tr> |
| <td>ScienceQA</td> |
| <td align="center" style="background-color:#be9df6;color:#1f2937">93.89</td> |
| <td align="center">91.91</td> |
| <td align="center">89.39</td> |
| <td align="center"><b>95.94</b></td> |
| <td align="center">85.76</td> |
| <td align="center">93.35</td> |
| <td align="center">89.53</td> |
| </tr> |
| <tr> |
| <td>MolParse</td> |
| <td align="center" style="background-color:#be9df6;color:#1f2937"><b>75.40</b></td> |
| <td align="center">4.35</td> |
| <td align="center">4.95</td> |
| <td align="center">19.05</td> |
| <td align="center">4.25</td> |
| <td align="center">7.00</td> |
| <td align="center">5.20</td> |
| </tr> |
| <tr> |
| <td>OpenRxn</td> |
| <td align="center" style="background-color:#be9df6;color:#1f2937"><b>56.50</b></td> |
| <td align="center">9.25</td> |
| <td align="center">12.60</td> |
| <td align="center">14.05</td> |
| <td align="center">12.70</td> |
| <td align="center">14.40</td> |
| <td align="center">15.65</td> |
| </tr> |
| <tr> |
| <td>RxnBench (EN)</td> |
| <td align="center" style="background-color:#be9df6;color:#1f2937"><b>89.05</b></td> |
| <td align="center">82.75</td> |
| <td align="center">85.31</td> |
| <td align="center">80.46</td> |
| <td align="center">73.70</td> |
| <td align="center">88.52</td> |
| <td align="center">87.67</td> |
| </tr> |
| <tr> |
| <td>SmolInstruct_v2</td> |
| <td align="center" style="background-color:#be9df6;color:#1f2937">27.08</td> |
| <td align="center">20.15</td> |
| <td align="center">24.18</td> |
| <td align="center"><b>40.27</b></td> |
| <td align="center">6.87</td> |
| <td align="center">14.38</td> |
| <td align="center">27.44</td> |
| </tr> |
| <tr> |
| <td>ProteinLMBench</td> |
| <td align="center" style="background-color:#be9df6;color:#1f2937">61.97</td> |
| <td align="center">59.32</td> |
| <td align="center">56.67</td> |
| <td align="center">61.76</td> |
| <td align="center">56.25</td> |
| <td align="center">54.77</td> |
| <td align="center"><b>65.89</b></td> |
| </tr> |
| <tr> |
| <td>SFE (EN)</td> |
| <td align="center" style="background-color:#be9df6;color:#1f2937">24.13</td> |
| <td align="center"><b>28.98</b></td> |
| <td align="center">22.41</td> |
| <td align="center">19.63</td> |
| <td align="center">14.44</td> |
| <td align="center">20.24</td> |
| <td align="center">20.12</td> |
| </tr> |
| <tr><td colspan="8"><i>Spatial</i></td></tr> |
| <tr> |
| <td>EmbSpatial-Bench</td> |
| <td align="center" style="background-color:#be9df6;color:#1f2937"><b>77.36</b></td> |
| <td align="center">74.90</td> |
| <td align="center">73.54</td> |
| <td align="center">73.24</td> |
| <td align="center">68.10</td> |
| <td align="center">75.77</td> |
| <td align="center">75.00</td> |
| </tr> |
| <tr> |
| <td>ERQA</td> |
| <td align="center" style="background-color:#be9df6;color:#1f2937"><b>45.75</b></td> |
| <td align="center">38.80</td> |
| <td align="center">42.25</td> |
| <td align="center">41.25</td> |
| <td align="center">24.00</td> |
| <td align="center">45.00</td> |
| <td align="center"><b>45.75</b></td> |
| </tr> |
| </tbody> |
| </table> |
| </div> |
| |
| ## Usage |
| |
| Verified with `transformers` 5.3.0. All 760 tensors in `model.safetensors` map |
| cleanly onto the model (no missing or unexpected keys). |
| |
| ```python |
| import torch |
| from transformers import AutoModelForImageTextToText, AutoProcessor |
| |
| path = "InnovatorLab/InnovatorVL-v2" |
| processor = AutoProcessor.from_pretrained(path) # Qwen3VLProcessor |
| model = AutoModelForImageTextToText.from_pretrained( # Qwen3_5ForConditionalGeneration |
| path, dtype=torch.bfloat16, device_map="auto" |
| ) |
| ``` |
| |
| ### Inference dependencies |
|
|
| The text stack uses hybrid attention with `linear_attention` layers. Without |
| [`flash-linear-attention`](https://github.com/fla-org/flash-linear-attention) |
| and [`causal-conv1d`](https://github.com/Dao-AILab/causal-conv1d) installed, |
| `transformers` logs *"The fast path is not available"* and falls back to a |
| slower pure-PyTorch implementation. Results are unaffected, but throughput is |
| lower — install both for production inference. |
|
|
| ## License |
|
|
| The license field above is a placeholder. It must be set by the model owner |
| before relying on this model commercially. |
|
|
