Spaces:
Running
Running
Address review: Arena/AA terms links, credit video-editing prompts, Pruna data all rights reserved
Browse files
README.md
CHANGED
|
@@ -35,16 +35,18 @@ Rapidata Elo comes directly from these exports.
|
|
| 35 |
The P-Bench code (the Gradio app, `app.py`, `ui.py`, `model_display.py`) is
|
| 36 |
released under the [Apache License 2.0](https://www.apache.org/licenses/LICENSE-2.0).
|
| 37 |
|
| 38 |
-
The
|
| 39 |
-
|
|
|
|
| 40 |
|
| 41 |
| Data in this Space | Source | License / terms |
|
| 42 |
| --- | --- | --- |
|
| 43 |
| `data/oneig_combined/prompts.jsonl`: 299 OneIG-Bench prompts (anime, human and object alignment) | [OneIG-Bench](https://huggingface.co/datasets/OneIG-Bench/OneIG-Bench) (Chang et al., 2025) | [CC BY-NC 4.0](https://creativecommons.org/licenses/by-nc/4.0/) |
|
| 44 |
| `data/qwen_image_bench_combined/prompts.jsonl`: 100 Qwen-Image-Bench prompts | [Qwen-Image-Bench](https://huggingface.co/datasets/Qwen/Qwen-Image-Bench) (Li et al., 2026) | [Apache 2.0](https://www.apache.org/licenses/LICENSE-2.0) |
|
| 45 |
| `data/video_generation_combined/prompts.jsonl`: 90 VBench-2.0 prompts | [VBench-2.0](https://github.com/Vchitect/VBench/tree/master/VBench-2.0) (Zheng et al., 2025) | [Apache 2.0](https://github.com/Vchitect/VBench/blob/master/LICENSE) |
|
| 46 |
-
| `data/
|
| 47 |
-
| `data/
|
|
|
|
| 48 |
|
| 49 |
The OneIG-Bench, Qwen-Image-Bench and VBench-2.0 prompts are copied verbatim.
|
| 50 |
P-Bench uses a selection of each benchmark and adds its own prompt IDs. The
|
|
@@ -54,13 +56,18 @@ scores (Elo) were collected by Pruna AI through Datapoint and Rapidata.
|
|
| 54 |
Because the OneIG-Bench prompts are licensed for non-commercial use only
|
| 55 |
(CC BY-NC 4.0), any reuse of `data/oneig_combined/` must follow that license.
|
| 56 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 57 |
## Citation
|
| 58 |
|
| 59 |
If you use P-Bench, please cite it together with the benchmarks whose prompts it uses:
|
| 60 |
|
| 61 |
```bibtex
|
| 62 |
@misc{P-Bench,
|
| 63 |
-
title={P-Bench: A Leaderboard for
|
| 64 |
author={PrunaAI},
|
| 65 |
year={2026},
|
| 66 |
howpublished={\url{https://huggingface.co/spaces/PrunaAI/P-Bench}}
|
|
|
|
| 35 |
The P-Bench code (the Gradio app, `app.py`, `ui.py`, `model_display.py`) is
|
| 36 |
released under the [Apache License 2.0](https://www.apache.org/licenses/LICENSE-2.0).
|
| 37 |
|
| 38 |
+
The data in `data/` is **not** covered by that license. Third-party benchmark
|
| 39 |
+
prompts and leaderboard data keep the license or terms of their original
|
| 40 |
+
source, and Pruna AI's own data is all rights reserved:
|
| 41 |
|
| 42 |
| Data in this Space | Source | License / terms |
|
| 43 |
| --- | --- | --- |
|
| 44 |
| `data/oneig_combined/prompts.jsonl`: 299 OneIG-Bench prompts (anime, human and object alignment) | [OneIG-Bench](https://huggingface.co/datasets/OneIG-Bench/OneIG-Bench) (Chang et al., 2025) | [CC BY-NC 4.0](https://creativecommons.org/licenses/by-nc/4.0/) |
|
| 45 |
| `data/qwen_image_bench_combined/prompts.jsonl`: 100 Qwen-Image-Bench prompts | [Qwen-Image-Bench](https://huggingface.co/datasets/Qwen/Qwen-Image-Bench) (Li et al., 2026) | [Apache 2.0](https://www.apache.org/licenses/LICENSE-2.0) |
|
| 46 |
| `data/video_generation_combined/prompts.jsonl`: 90 VBench-2.0 prompts | [VBench-2.0](https://github.com/Vchitect/VBench/tree/master/VBench-2.0) (Zheng et al., 2025) | [Apache 2.0](https://github.com/Vchitect/VBench/blob/master/LICENSE) |
|
| 47 |
+
| `data/video_editing_combined/prompts.jsonl`: 78 video-editing prompts and their source videos | Pruna AI internal video-editing set (not publicly released) | © Pruna AI, all rights reserved |
|
| 48 |
+
| `data/arena_ai_text_to_image_leaderboard.csv` | [Arena](https://arena.ai/leaderboard/text-to-image/overall) text-to-image leaderboard (formerly LMArena) | [Arena terms of use](https://arena.ai/terms) |
|
| 49 |
+
| `data/artificial_analysis_text_to_image_leaderboard.csv` | [Artificial Analysis](https://artificialanalysis.ai/) text-to-image arena | [Artificial Analysis terms of use](https://artificialanalysis.ai/docs/legal/Terms-of-Use.pdf) |
|
| 50 |
|
| 51 |
The OneIG-Bench, Qwen-Image-Bench and VBench-2.0 prompts are copied verbatim.
|
| 52 |
P-Bench uses a selection of each benchmark and adds its own prompt IDs. The
|
|
|
|
| 56 |
Because the OneIG-Bench prompts are licensed for non-commercial use only
|
| 57 |
(CC BY-NC 4.0), any reuse of `data/oneig_combined/` must follow that license.
|
| 58 |
|
| 59 |
+
All other data in `data/` was created by Pruna AI and is © Pruna AI, all
|
| 60 |
+
rights reserved. This covers the video-editing prompts and source videos, the
|
| 61 |
+
Elo exports (for example `t2v-pro.csv`, `i2v-pro.csv`,
|
| 62 |
+
`video-editing-leaderboard.csv`) and the price and generation-time CSVs.
|
| 63 |
+
|
| 64 |
## Citation
|
| 65 |
|
| 66 |
If you use P-Bench, please cite it together with the benchmarks whose prompts it uses:
|
| 67 |
|
| 68 |
```bibtex
|
| 69 |
@misc{P-Bench,
|
| 70 |
+
title={P-Bench: A Leaderboard for Image and Video Generation Models},
|
| 71 |
author={PrunaAI},
|
| 72 |
year={2026},
|
| 73 |
howpublished={\url{https://huggingface.co/spaces/PrunaAI/P-Bench}}
|
app.py
CHANGED
|
@@ -2641,11 +2641,16 @@ SAMPLE_PROMPT_CREDITS = {
|
|
| 2641 |
'target="_blank" rel="noopener noreferrer">VBench-2.0</a> (Zheng et al., 2025), '
|
| 2642 |
"Apache 2.0. See About for references."
|
| 2643 |
),
|
|
|
|
|
|
|
|
|
|
|
|
|
| 2644 |
}
|
| 2645 |
for _samples, _credit_id in (
|
| 2646 |
(oneig_samples, "oneig"),
|
| 2647 |
(qwen_samples, "qwen"),
|
| 2648 |
(text_to_video_samples, "text_to_video"),
|
|
|
|
| 2649 |
):
|
| 2650 |
if _samples:
|
| 2651 |
_samples["credit"] = SAMPLE_PROMPT_CREDITS[_credit_id]
|
|
|
|
| 2641 |
'target="_blank" rel="noopener noreferrer">VBench-2.0</a> (Zheng et al., 2025), '
|
| 2642 |
"Apache 2.0. See About for references."
|
| 2643 |
),
|
| 2644 |
+
"video_editing": (
|
| 2645 |
+
"Prompts and source videos: Pruna AI's internal video-editing set "
|
| 2646 |
+
"(not publicly released). See About for details."
|
| 2647 |
+
),
|
| 2648 |
}
|
| 2649 |
for _samples, _credit_id in (
|
| 2650 |
(oneig_samples, "oneig"),
|
| 2651 |
(qwen_samples, "qwen"),
|
| 2652 |
(text_to_video_samples, "text_to_video"),
|
| 2653 |
+
(video_samples, "video_editing"),
|
| 2654 |
):
|
| 2655 |
if _samples:
|
| 2656 |
_samples["credit"] = SAMPLE_PROMPT_CREDITS[_credit_id]
|
ui.py
CHANGED
|
@@ -188,9 +188,18 @@ own prompt IDs.
|
|
| 188 |
Zhang, Jingwen He, Wei-Shi Zheng, Yu Qiao and Ziwei Liu. *VBench-2.0:
|
| 189 |
Advancing Video Generation Benchmark Suite for Intrinsic Faithfulness.*
|
| 190 |
arXiv:2503.21755, 2025.
|
| 191 |
-
- **
|
| 192 |
-
|
| 193 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 194 |
|
| 195 |
BibTeX for all of these is in the
|
| 196 |
[Space README](https://huggingface.co/spaces/PrunaAI/P-Bench/blob/main/README.md#citation).
|
|
|
|
| 188 |
Zhang, Jingwen He, Wei-Shi Zheng, Yu Qiao and Ziwei Liu. *VBench-2.0:
|
| 189 |
Advancing Video Generation Benchmark Suite for Intrinsic Faithfulness.*
|
| 190 |
arXiv:2503.21755, 2025.
|
| 191 |
+
- **Pruna Internal Video-Edit Benchmark**: prompts and source videos from
|
| 192 |
+
Pruna AI's internal video-editing set, created by Pruna AI and not
|
| 193 |
+
publicly released. © Pruna AI, all rights reserved.
|
| 194 |
+
- **Artificial Analysis Dataset**: Elo and price as published on the
|
| 195 |
+
[Artificial Analysis](https://artificialanalysis.ai/) text-to-image arena,
|
| 196 |
+
under their [terms of use](https://artificialanalysis.ai/docs/legal/Terms-of-Use.pdf).
|
| 197 |
+
- **Arena AI Dataset**: Elo as published on the
|
| 198 |
+
[Arena](https://arena.ai/leaderboard/text-to-image/overall) text-to-image
|
| 199 |
+
leaderboard (formerly LMArena), under their [terms of use](https://arena.ai/terms).
|
| 200 |
+
|
| 201 |
+
Pruna AI's own data (the video-editing prompts and source videos, Elo
|
| 202 |
+
scores, prices and generation times) is © Pruna AI, all rights reserved.
|
| 203 |
|
| 204 |
BibTeX for all of these is in the
|
| 205 |
[Space README](https://huggingface.co/spaces/PrunaAI/P-Bench/blob/main/README.md#citation).
|