hongzhuyi commited on
Commit
18bdc28
·
verified ·
1 Parent(s): 3bc3f2a

Add VDEBench evaluation results

Browse files

Hi 👋 this PR adds this model's [VDEBench](https://huggingface.co/datasets/hongzhuyi/VDEBench) scores to `.eval_results/`, so they show up on the model page and the VDEBench leaderboard.

VDEBench evaluates instruction-guided editing of dense English/Chinese document pages (3,506 examples). Each score is the equally weighted mean of IoU, CDM, BLEU-4 and TEDS (0–1, higher is better):

| Task | Score ↑ |
|---|---:|
| `local` (edited region, headline) | 0.638 |
| `global` (full page) | 0.750 |

These results come from the VDEBench team's new full evaluation of **FireRed-Image-Edit-1.1**. [Detailed results](https://huggingface.co/datasets/hongzhuyi/VDEBench/blob/18ee0824f34d41c6b91b302a38b5cf7404b53e17/results/2026-11-01/firered_image_edit_1_1/results.json).

Thanks!

Files changed (1) hide show
  1. .eval_results/vdebench.yaml +20 -0
.eval_results/vdebench.yaml ADDED
@@ -0,0 +1,20 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ - dataset:
2
+ id: hongzhuyi/VDEBench
3
+ task_id: local
4
+ value: 0.638
5
+ date: '2026-10-31T23:19:38.399323+00:00'
6
+ source:
7
+ url: https://huggingface.co/datasets/hongzhuyi/VDEBench/blob/18ee0824f34d41c6b91b302a38b5cf7404b53e17/results/2026-11-01/firered_image_edit_1_1/results.json
8
+ name: VDEBench full evaluation (2026-11-01)
9
+ user: hongzhuyi
10
+ notes: "local setting, n=3,506, PaddleOCR-VL; IoU 0.697, CDM 0.753, BLEU-4 0.430, TEDS 0.670"
11
+ - dataset:
12
+ id: hongzhuyi/VDEBench
13
+ task_id: global
14
+ value: 0.750
15
+ date: '2026-10-31T23:19:38.399323+00:00'
16
+ source:
17
+ url: https://huggingface.co/datasets/hongzhuyi/VDEBench/blob/18ee0824f34d41c6b91b302a38b5cf7404b53e17/results/2026-11-01/firered_image_edit_1_1/results.json
18
+ name: VDEBench full evaluation (2026-11-01)
19
+ user: hongzhuyi
20
+ notes: "global setting, n=3,506, PaddleOCR-VL; IoU 0.861, CDM 0.879, BLEU-4 0.518, TEDS 0.741"