begumcig Claude Opus 5.5 commited on
Commit
a2b189c
·
1 Parent(s): b4c3954

Address review: Arena/AA terms links, credit video-editing prompts, Pruna data all rights reserved

Browse files
Files changed (3) hide show
  1. README.md +12 -5
  2. app.py +5 -0
  3. ui.py +12 -3
README.md CHANGED
@@ -35,16 +35,18 @@ Rapidata Elo comes directly from these exports.
35
  The P-Bench code (the Gradio app, `app.py`, `ui.py`, `model_display.py`) is
36
  released under the [Apache License 2.0](https://www.apache.org/licenses/LICENSE-2.0).
37
 
38
- The benchmark prompts and third-party data in `data/` are **not** covered by
39
- that license. They keep the license or terms of their original source:
 
40
 
41
  | Data in this Space | Source | License / terms |
42
  | --- | --- | --- |
43
  | `data/oneig_combined/prompts.jsonl`: 299 OneIG-Bench prompts (anime, human and object alignment) | [OneIG-Bench](https://huggingface.co/datasets/OneIG-Bench/OneIG-Bench) (Chang et al., 2025) | [CC BY-NC 4.0](https://creativecommons.org/licenses/by-nc/4.0/) |
44
  | `data/qwen_image_bench_combined/prompts.jsonl`: 100 Qwen-Image-Bench prompts | [Qwen-Image-Bench](https://huggingface.co/datasets/Qwen/Qwen-Image-Bench) (Li et al., 2026) | [Apache 2.0](https://www.apache.org/licenses/LICENSE-2.0) |
45
  | `data/video_generation_combined/prompts.jsonl`: 90 VBench-2.0 prompts | [VBench-2.0](https://github.com/Vchitect/VBench/tree/master/VBench-2.0) (Zheng et al., 2025) | [Apache 2.0](https://github.com/Vchitect/VBench/blob/master/LICENSE) |
46
- | `data/arena_ai_text_to_image_leaderboard.csv` | Arena text-to-image leaderboard | the source's terms of use |
47
- | `data/artificial_analysis_text_to_image_leaderboard.csv` | [Artificial Analysis](https://artificialanalysis.ai/) text-to-image arena | the source's terms of use |
 
48
 
49
  The OneIG-Bench, Qwen-Image-Bench and VBench-2.0 prompts are copied verbatim.
50
  P-Bench uses a selection of each benchmark and adds its own prompt IDs. The
@@ -54,13 +56,18 @@ scores (Elo) were collected by Pruna AI through Datapoint and Rapidata.
54
  Because the OneIG-Bench prompts are licensed for non-commercial use only
55
  (CC BY-NC 4.0), any reuse of `data/oneig_combined/` must follow that license.
56
 
 
 
 
 
 
57
  ## Citation
58
 
59
  If you use P-Bench, please cite it together with the benchmarks whose prompts it uses:
60
 
61
  ```bibtex
62
  @misc{P-Bench,
63
- title={P-Bench: A Leaderboard for Text-to-Image Models},
64
  author={PrunaAI},
65
  year={2026},
66
  howpublished={\url{https://huggingface.co/spaces/PrunaAI/P-Bench}}
 
35
  The P-Bench code (the Gradio app, `app.py`, `ui.py`, `model_display.py`) is
36
  released under the [Apache License 2.0](https://www.apache.org/licenses/LICENSE-2.0).
37
 
38
+ The data in `data/` is **not** covered by that license. Third-party benchmark
39
+ prompts and leaderboard data keep the license or terms of their original
40
+ source, and Pruna AI's own data is all rights reserved:
41
 
42
  | Data in this Space | Source | License / terms |
43
  | --- | --- | --- |
44
  | `data/oneig_combined/prompts.jsonl`: 299 OneIG-Bench prompts (anime, human and object alignment) | [OneIG-Bench](https://huggingface.co/datasets/OneIG-Bench/OneIG-Bench) (Chang et al., 2025) | [CC BY-NC 4.0](https://creativecommons.org/licenses/by-nc/4.0/) |
45
  | `data/qwen_image_bench_combined/prompts.jsonl`: 100 Qwen-Image-Bench prompts | [Qwen-Image-Bench](https://huggingface.co/datasets/Qwen/Qwen-Image-Bench) (Li et al., 2026) | [Apache 2.0](https://www.apache.org/licenses/LICENSE-2.0) |
46
  | `data/video_generation_combined/prompts.jsonl`: 90 VBench-2.0 prompts | [VBench-2.0](https://github.com/Vchitect/VBench/tree/master/VBench-2.0) (Zheng et al., 2025) | [Apache 2.0](https://github.com/Vchitect/VBench/blob/master/LICENSE) |
47
+ | `data/video_editing_combined/prompts.jsonl`: 78 video-editing prompts and their source videos | Pruna AI internal video-editing set (not publicly released) | © Pruna AI, all rights reserved |
48
+ | `data/arena_ai_text_to_image_leaderboard.csv` | [Arena](https://arena.ai/leaderboard/text-to-image/overall) text-to-image leaderboard (formerly LMArena) | [Arena terms of use](https://arena.ai/terms) |
49
+ | `data/artificial_analysis_text_to_image_leaderboard.csv` | [Artificial Analysis](https://artificialanalysis.ai/) text-to-image arena | [Artificial Analysis terms of use](https://artificialanalysis.ai/docs/legal/Terms-of-Use.pdf) |
50
 
51
  The OneIG-Bench, Qwen-Image-Bench and VBench-2.0 prompts are copied verbatim.
52
  P-Bench uses a selection of each benchmark and adds its own prompt IDs. The
 
56
  Because the OneIG-Bench prompts are licensed for non-commercial use only
57
  (CC BY-NC 4.0), any reuse of `data/oneig_combined/` must follow that license.
58
 
59
+ All other data in `data/` was created by Pruna AI and is © Pruna AI, all
60
+ rights reserved. This covers the video-editing prompts and source videos, the
61
+ Elo exports (for example `t2v-pro.csv`, `i2v-pro.csv`,
62
+ `video-editing-leaderboard.csv`) and the price and generation-time CSVs.
63
+
64
  ## Citation
65
 
66
  If you use P-Bench, please cite it together with the benchmarks whose prompts it uses:
67
 
68
  ```bibtex
69
  @misc{P-Bench,
70
+ title={P-Bench: A Leaderboard for Image and Video Generation Models},
71
  author={PrunaAI},
72
  year={2026},
73
  howpublished={\url{https://huggingface.co/spaces/PrunaAI/P-Bench}}
app.py CHANGED
@@ -2641,11 +2641,16 @@ SAMPLE_PROMPT_CREDITS = {
2641
  'target="_blank" rel="noopener noreferrer">VBench-2.0</a> (Zheng et al., 2025), '
2642
  "Apache 2.0. See About for references."
2643
  ),
 
 
 
 
2644
  }
2645
  for _samples, _credit_id in (
2646
  (oneig_samples, "oneig"),
2647
  (qwen_samples, "qwen"),
2648
  (text_to_video_samples, "text_to_video"),
 
2649
  ):
2650
  if _samples:
2651
  _samples["credit"] = SAMPLE_PROMPT_CREDITS[_credit_id]
 
2641
  'target="_blank" rel="noopener noreferrer">VBench-2.0</a> (Zheng et al., 2025), '
2642
  "Apache 2.0. See About for references."
2643
  ),
2644
+ "video_editing": (
2645
+ "Prompts and source videos: Pruna AI's internal video-editing set "
2646
+ "(not publicly released). See About for details."
2647
+ ),
2648
  }
2649
  for _samples, _credit_id in (
2650
  (oneig_samples, "oneig"),
2651
  (qwen_samples, "qwen"),
2652
  (text_to_video_samples, "text_to_video"),
2653
+ (video_samples, "video_editing"),
2654
  ):
2655
  if _samples:
2656
  _samples["credit"] = SAMPLE_PROMPT_CREDITS[_credit_id]
ui.py CHANGED
@@ -188,9 +188,18 @@ own prompt IDs.
188
  Zhang, Jingwen He, Wei-Shi Zheng, Yu Qiao and Ziwei Liu. *VBench-2.0:
189
  Advancing Video Generation Benchmark Suite for Intrinsic Faithfulness.*
190
  arXiv:2503.21755, 2025.
191
- - **Artificial Analysis Dataset** and **Arena AI Dataset**: Elo (and
192
- Artificial Analysis price) as published on their leaderboards, under
193
- their terms of use.
 
 
 
 
 
 
 
 
 
194
 
195
  BibTeX for all of these is in the
196
  [Space README](https://huggingface.co/spaces/PrunaAI/P-Bench/blob/main/README.md#citation).
 
188
  Zhang, Jingwen He, Wei-Shi Zheng, Yu Qiao and Ziwei Liu. *VBench-2.0:
189
  Advancing Video Generation Benchmark Suite for Intrinsic Faithfulness.*
190
  arXiv:2503.21755, 2025.
191
+ - **Pruna Internal Video-Edit Benchmark**: prompts and source videos from
192
+ Pruna AI's internal video-editing set, created by Pruna AI and not
193
+ publicly released. © Pruna AI, all rights reserved.
194
+ - **Artificial Analysis Dataset**: Elo and price as published on the
195
+ [Artificial Analysis](https://artificialanalysis.ai/) text-to-image arena,
196
+ under their [terms of use](https://artificialanalysis.ai/docs/legal/Terms-of-Use.pdf).
197
+ - **Arena AI Dataset**: Elo as published on the
198
+ [Arena](https://arena.ai/leaderboard/text-to-image/overall) text-to-image
199
+ leaderboard (formerly LMArena), under their [terms of use](https://arena.ai/terms).
200
+
201
+ Pruna AI's own data (the video-editing prompts and source videos, Elo
202
+ scores, prices and generation times) is © Pruna AI, all rights reserved.
203
 
204
  BibTeX for all of these is in the
205
  [Space README](https://huggingface.co/spaces/PrunaAI/P-Bench/blob/main/README.md#citation).