yiyexy commited on
Commit
3948ec8
·
verified ·
1 Parent(s): b0d235b

Model card: average VideoMME splits, drop video-spatial rows, benchmark figure and results highlight

Browse files
Files changed (3) hide show
  1. README.md +3 -16
  2. README_CN.md +3 -16
  3. assets/video_benchmarks.png +0 -3
README.md CHANGED
@@ -31,19 +31,16 @@ LLaVA-OneVision-2.5-4B-Instruct is a 4B vision-language model for single-image,
31
  - **Offline and streaming in one model.** `apply_chat_template(..., streaming=True)` switches mode; eight protocol tokens carry the per-frame decision to stay silent, respond or caption. Nothing else about the input changes.
32
  - **Codec-aligned video.** Besides uniform frame sampling, *codec mode* uses the encoder's bit-cost to pack the informative regions of a video into a handful of canvases. It beats frame sampling on **17 of 21** video benchmarks.
33
  - **9× spatial compression.** A patch-16 vision tower with 3×3 merging gives one visual token per 48×48 pixels, so long videos fit comfortably in the 262k context.
34
- - **Strong video, grounding and spatial results.** Against Qwen3.5-4B, frame sampling leads on **17 of 21** video benchmarks and codec on **16 of 18**, with the largest margins on temporal grounding (TimeLens Charades +12.5) and spatial video (VSI-Bench +9.4). On still images it is strongest on grounding and spatial tasks; OCR and document reading are not its focus.
35
 
36
  ## Performance
37
 
38
- <p align="center">
39
- <img src="assets/video_benchmarks.png" width="92%" alt="Video benchmarks: LLaVA-OneVision-2.5-4B vs Qwen3.5-4B">
40
- </p>
41
-
42
  ### Video
43
 
44
  | Benchmark | Qwen3.5-4B | OV2.5-4B (frames) | OV2.5-4B (codec) |
45
  |:---|:---:|:---:|:---:|
46
  | **General video understanding** | | | |
 
 
47
  | MotionBench | 60.1 <sup>128f×153,664</sup> | 62.4 <sup>128f×153,664</sup> | **64.2** <sup>TC128×313,600</sup> |
48
  | TOMATO \* | 34.9 <sup>16f×153,664, fps=2</sup> | 33.6 <sup>model default</sup> | **38.6** <sup>TC128×313,600</sup> |
49
  | MVBench | 64.0 <sup>64f×153,664</sup> | 72.0 <sup>64f×153,664</sup> | **75.5** <sup>TC64×313,600</sup> |
@@ -54,16 +51,6 @@ LLaVA-OneVision-2.5-4B-Instruct is a 4B vision-language model for single-image,
54
  | EgoSchema subset | 68.2 <sup>64f×153,664</sup> | **80.0** <sup>64f×153,664</sup> | -- |
55
  | Video-Index 512f | 19.6 <sup>task frames, ≤262,144/f</sup> | **20.0** <sup>task frames, ≤262,144/f</sup> | 19.8 <sup>TC≤256×313,600</sup> |
56
  | MMOU test-mini | -- | 46.0 <sup>model default</sup> | -- |
57
- | **VideoMME** | | | |
58
- | short, no subtitles | **81.7** <sup>128f×321,489</sup> | 79.7 <sup>128f×321,489</sup> | 81.1 <sup>TC256×313,600</sup> |
59
- | medium, no subtitles | 70.3 <sup>256f×136,900</sup> | 70.6 <sup>256f×136,900</sup> | **74.1** <sup>TC512×313,600</sup> |
60
- | long, no subtitles | **63.0** <sup>1450f×120,000</sup> | 59.0 <sup>750f×250,000</sup> | 60.6 <sup>TC512×313,600</sup> |
61
- | short, subtitles | -- | 82.4 <sup>128f×321,489</sup> | **82.7** <sup>TC256×313,600</sup> |
62
- | medium, subtitles | -- | 75.4 <sup>256f×136,900</sup> | **75.6** <sup>TC256×313,600</sup> |
63
- | long, subtitles | -- | 65.8 <sup>750f×250,000</sup> | **65.9** <sup>TC256×313,600</sup> |
64
- | **Video spatial** | | | |
65
- | VSI-Bench | 59.2 <sup>128f×153,664</sup> | **68.6** <sup>128f×153,664</sup> | 67.7 <sup>TC128×313,600</sup> |
66
- | ReVSI 64f | 47.1 <sup>64f×153,664</sup> | **55.9** <sup>64f×153,664</sup> | 55.5 <sup>TC64×313,600</sup> |
67
  | **Long video** | | | |
68
  | LVBench | 56.5 <sup>2048f×120,000</sup> | **62.0** <sup>950f×200,000</sup> | 61.6 <sup>TC1024×156,800</sup> |
69
  | MLVU dev | 73.8 <sup>512f×72,900</sup> | **75.3** <sup>512f×72,900</sup> | -- |
@@ -87,7 +74,7 @@ LLaVA-OneVision-2.5-4B-Instruct is a 4B vision-language model for single-image,
87
  | OCRBench | **85.2** | 79.8 | EmbSpatial | 74.7 | **83.0** |
88
  | DocVQA val | **95.2** | 92.9 | MMSI-Bench | 31.4 | **33.3** |
89
 
90
- <sub>Compared with Qwen3.5-4B, thinking disabled. **Bold** is the best score in each row; `--` means not available. The superscript on each video score is the setting that produced it: frames × pixels per frame, or for codec, canvases (TC) × pixels per canvas. Each side reports its best score over the settings tried, from single runs. `*` On TOMATO the two sides use different frame settings. The image table shows selected benchmarks; full results, settings and protocol will be given in the technical report.</sub>
91
 
92
  ## Model Overview
93
 
 
31
  - **Offline and streaming in one model.** `apply_chat_template(..., streaming=True)` switches mode; eight protocol tokens carry the per-frame decision to stay silent, respond or caption. Nothing else about the input changes.
32
  - **Codec-aligned video.** Besides uniform frame sampling, *codec mode* uses the encoder's bit-cost to pack the informative regions of a video into a handful of canvases. It beats frame sampling on **17 of 21** video benchmarks.
33
  - **9× spatial compression.** A patch-16 vision tower with 3×3 merging gives one visual token per 48×48 pixels, so long videos fit comfortably in the 262k context.
 
34
 
35
  ## Performance
36
 
 
 
 
 
37
  ### Video
38
 
39
  | Benchmark | Qwen3.5-4B | OV2.5-4B (frames) | OV2.5-4B (codec) |
40
  |:---|:---:|:---:|:---:|
41
  | **General video understanding** | | | |
42
+ | VideoMME (no subtitles) | 71.7 | 69.7 | **71.9** |
43
+ | VideoMME (subtitles) | -- | 74.6 | **74.7** |
44
  | MotionBench | 60.1 <sup>128f×153,664</sup> | 62.4 <sup>128f×153,664</sup> | **64.2** <sup>TC128×313,600</sup> |
45
  | TOMATO \* | 34.9 <sup>16f×153,664, fps=2</sup> | 33.6 <sup>model default</sup> | **38.6** <sup>TC128×313,600</sup> |
46
  | MVBench | 64.0 <sup>64f×153,664</sup> | 72.0 <sup>64f×153,664</sup> | **75.5** <sup>TC64×313,600</sup> |
 
51
  | EgoSchema subset | 68.2 <sup>64f×153,664</sup> | **80.0** <sup>64f×153,664</sup> | -- |
52
  | Video-Index 512f | 19.6 <sup>task frames, ≤262,144/f</sup> | **20.0** <sup>task frames, ≤262,144/f</sup> | 19.8 <sup>TC≤256×313,600</sup> |
53
  | MMOU test-mini | -- | 46.0 <sup>model default</sup> | -- |
 
 
 
 
 
 
 
 
 
 
54
  | **Long video** | | | |
55
  | LVBench | 56.5 <sup>2048f×120,000</sup> | **62.0** <sup>950f×200,000</sup> | 61.6 <sup>TC1024×156,800</sup> |
56
  | MLVU dev | 73.8 <sup>512f×72,900</sup> | **75.3** <sup>512f×72,900</sup> | -- |
 
74
  | OCRBench | **85.2** | 79.8 | EmbSpatial | 74.7 | **83.0** |
75
  | DocVQA val | **95.2** | 92.9 | MMSI-Bench | 31.4 | **33.3** |
76
 
77
+ <sub>Compared with Qwen3.5-4B, thinking disabled. **Bold** is the best score in each row; `--` means not available. The superscript on each video score is the setting that produced it: frames × pixels per frame, or for codec, canvases (TC) × pixels per canvas. VideoMME is the mean over the short, medium and long splits. Each side reports its best score over the settings tried, from single runs. `*` On TOMATO the two sides use different frame settings. The image table shows selected benchmarks; full results, settings and protocol will be given in the technical report.</sub>
78
 
79
  ## Model Overview
80
 
README_CN.md CHANGED
@@ -31,19 +31,16 @@ LLaVA-OneVision-2.5-4B-Instruct 是一个 4B 视觉语言模型,支持单图
31
  - **离线与流式合一。** `apply_chat_template(..., streaming=True)` 即可切换模式;八个协议 token 承载每一帧"沉默 / 回答 / caption"的决策,输入格式的其余部分完全相同。
32
  - **Codec 对齐的视频输入。** 除均匀抽帧外,**codec 模式**利用编码器的 bit-cost,把视频中信息量大的区域打包进少量画布。在 21 个视频 bench 中,它有 **17 个**优于抽帧。
33
  - **9 倍空间压缩。** patch 16 的视觉塔加 3×3 合并,每 48×48 像素对应一个视觉 token,长视频可以从容放进 262k 上下文。
34
- - **视频、定位与空间能力突出。** 对比 Qwen3.5-4B,抽帧在 21 个视频 bench 中领先 **17 个**,codec 在 18 个中领先 **16 个**,优势最大的是时间定位(TimeLens Charades +12.5)和视频空间(VSI-Bench +9.4)。静态图像上的优势集中在定位与空间任务;OCR 与文档阅读不是本模型的重点。
35
 
36
  ## 评测结果
37
 
38
- <p align="center">
39
- <img src="assets/video_benchmarks.png" width="92%" alt="视频评测:LLaVA-OneVision-2.5-4B 对比 Qwen3.5-4B">
40
- </p>
41
-
42
  ### 视频
43
 
44
  | Benchmark | Qwen3.5-4B | OV2.5-4B(抽帧) | OV2.5-4B(codec) |
45
  |:---|:---:|:---:|:---:|
46
  | **通用视频理解** | | | |
 
 
47
  | MotionBench | 60.1 <sup>128f×153,664</sup> | 62.4 <sup>128f×153,664</sup> | **64.2** <sup>TC128×313,600</sup> |
48
  | TOMATO \* | 34.9 <sup>16f×153,664, fps=2</sup> | 33.6 <sup>model default</sup> | **38.6** <sup>TC128×313,600</sup> |
49
  | MVBench | 64.0 <sup>64f×153,664</sup> | 72.0 <sup>64f×153,664</sup> | **75.5** <sup>TC64×313,600</sup> |
@@ -54,16 +51,6 @@ LLaVA-OneVision-2.5-4B-Instruct 是一个 4B 视觉语言模型,支持单图
54
  | EgoSchema subset | 68.2 <sup>64f×153,664</sup> | **80.0** <sup>64f×153,664</sup> | -- |
55
  | Video-Index 512f | 19.6 <sup>task frames, ≤262,144/f</sup> | **20.0** <sup>task frames, ≤262,144/f</sup> | 19.8 <sup>TC≤256×313,600</sup> |
56
  | MMOU test-mini | -- | 46.0 <sup>model default</sup> | -- |
57
- | **VideoMME** | | | |
58
- | short,无字幕 | **81.7** <sup>128f×321,489</sup> | 79.7 <sup>128f×321,489</sup> | 81.1 <sup>TC256×313,600</sup> |
59
- | medium,无字幕 | 70.3 <sup>256f×136,900</sup> | 70.6 <sup>256f×136,900</sup> | **74.1** <sup>TC512×313,600</sup> |
60
- | long,无字幕 | **63.0** <sup>1450f×120,000</sup> | 59.0 <sup>750f×250,000</sup> | 60.6 <sup>TC512×313,600</sup> |
61
- | short,有字幕 | -- | 82.4 <sup>128f×321,489</sup> | **82.7** <sup>TC256×313,600</sup> |
62
- | medium,有字幕 | -- | 75.4 <sup>256f×136,900</sup> | **75.6** <sup>TC256×313,600</sup> |
63
- | long,有字幕 | -- | 65.8 <sup>750f×250,000</sup> | **65.9** <sup>TC256×313,600</sup> |
64
- | **视频空间** | | | |
65
- | VSI-Bench | 59.2 <sup>128f×153,664</sup> | **68.6** <sup>128f×153,664</sup> | 67.7 <sup>TC128×313,600</sup> |
66
- | ReVSI 64f | 47.1 <sup>64f×153,664</sup> | **55.9** <sup>64f×153,664</sup> | 55.5 <sup>TC64×313,600</sup> |
67
  | **长视频** | | | |
68
  | LVBench | 56.5 <sup>2048f×120,000</sup> | **62.0** <sup>950f×200,000</sup> | 61.6 <sup>TC1024×156,800</sup> |
69
  | MLVU dev | 73.8 <sup>512f×72,900</sup> | **75.3** <sup>512f×72,900</sup> | -- |
@@ -87,7 +74,7 @@ LLaVA-OneVision-2.5-4B-Instruct 是一个 4B 视觉语言模型,支持单图
87
  | OCRBench | **85.2** | 79.8 | EmbSpatial | 74.7 | **83.0** |
88
  | DocVQA val | **95.2** | 92.9 | MMSI-Bench | 31.4 | **33.3** |
89
 
90
- <sub>对比对象为关闭 thinking 的 Qwen3.5-4B。**加粗**为该行最高分,`--` 表示暂无结果。视频分数的上标是取得该分数时的设置:帧数 × 每帧像素;codec 为画布数(TC)× 每画布像素。两边各取已测设置中的最高分,均为单次运行。`*` TOMATO 两边帧数设置不同。图像表为代表性 bench 的节选,完整结果、设置与评测口径见后续技术报告。</sub>
91
 
92
  ## 模型概览
93
 
 
31
  - **离线与流式合一。** `apply_chat_template(..., streaming=True)` 即可切换模式;八个协议 token 承载每一帧"沉默 / 回答 / caption"的决策,输入格式的其余部分完全相同。
32
  - **Codec 对齐的视频输入。** 除均匀抽帧外,**codec 模式**利用编码器的 bit-cost,把视频中信息量大的区域打包进少量画布。在 21 个视频 bench 中,它有 **17 个**优于抽帧。
33
  - **9 倍空间压缩。** patch 16 的视觉塔加 3×3 合并,每 48×48 像素对应一个视觉 token,长视频可以从容放进 262k 上下文。
 
34
 
35
  ## 评测结果
36
 
 
 
 
 
37
  ### 视频
38
 
39
  | Benchmark | Qwen3.5-4B | OV2.5-4B(抽帧) | OV2.5-4B(codec) |
40
  |:---|:---:|:---:|:---:|
41
  | **通用视频理解** | | | |
42
+ | VideoMME(无字幕) | 71.7 | 69.7 | **71.9** |
43
+ | VideoMME(有字幕) | -- | 74.6 | **74.7** |
44
  | MotionBench | 60.1 <sup>128f×153,664</sup> | 62.4 <sup>128f×153,664</sup> | **64.2** <sup>TC128×313,600</sup> |
45
  | TOMATO \* | 34.9 <sup>16f×153,664, fps=2</sup> | 33.6 <sup>model default</sup> | **38.6** <sup>TC128×313,600</sup> |
46
  | MVBench | 64.0 <sup>64f×153,664</sup> | 72.0 <sup>64f×153,664</sup> | **75.5** <sup>TC64×313,600</sup> |
 
51
  | EgoSchema subset | 68.2 <sup>64f×153,664</sup> | **80.0** <sup>64f×153,664</sup> | -- |
52
  | Video-Index 512f | 19.6 <sup>task frames, ≤262,144/f</sup> | **20.0** <sup>task frames, ≤262,144/f</sup> | 19.8 <sup>TC≤256×313,600</sup> |
53
  | MMOU test-mini | -- | 46.0 <sup>model default</sup> | -- |
 
 
 
 
 
 
 
 
 
 
54
  | **长视频** | | | |
55
  | LVBench | 56.5 <sup>2048f×120,000</sup> | **62.0** <sup>950f×200,000</sup> | 61.6 <sup>TC1024×156,800</sup> |
56
  | MLVU dev | 73.8 <sup>512f×72,900</sup> | **75.3** <sup>512f×72,900</sup> | -- |
 
74
  | OCRBench | **85.2** | 79.8 | EmbSpatial | 74.7 | **83.0** |
75
  | DocVQA val | **95.2** | 92.9 | MMSI-Bench | 31.4 | **33.3** |
76
 
77
+ <sub>对比对象为关闭 thinking 的 Qwen3.5-4B。**加粗**为该行最高分,`--` 表示暂无结果。视频分数的上标是取得该分数时的设置:帧数 × 每帧像素;codec 为画布数(TC)× 每画布像素。VideoMME 为 short、medium、long 三个子集的平均。两边各取已测设置中的最高分,均为单次运行。`*` TOMATO 两边帧数设置不同。图像表为代表性 bench 的节选,完整结果、设置与评测口径见后续技术报告。</sub>
78
 
79
  ## 模型概览
80
 
assets/video_benchmarks.png DELETED

Git LFS Details

  • SHA256: 58a514fbc94a524df77c369e8c65990cbe50de944baf586bca1ed9b3826c35f6
  • Pointer size: 131 Bytes
  • Size of remote file: 219 kB