Spaces:
Running on Zero
Running on Zero
| title: PadCaptioner | |
| emoji: 🎬 | |
| colorFrom: indigo | |
| colorTo: purple | |
| sdk: gradio | |
| sdk_version: 6.31.0 | |
| app_file: app.py | |
| short_description: Omni-modal dense video captioning with parallel decoding | |
| python_version: "3.12" | |
| startup_duration_timeout: 1h | |
| # PadCaptioner — Omni-Modal Dense Video Captioning | |
| Demo of **PadCaptioner-3B** ("Parallelized Autoregressive Decoding for Omni-Modal Dense | |
| Video Captioning", ECCV 2026). Upload a video with sound and the model produces | |
| **temporally grounded event captions**: one `start - end caption` record per event. | |
| The model watches the video frames *and* listens to the audio track. It first plans | |
| event boundaries with a grounding head (the `<G>` tokens), then decodes the captions of | |
| all planned events **in parallel** in a single forward pass — instead of captioning | |
| events one-by-one. | |
| - Paper: [huggingface.co/papers/2607.02963](https://huggingface.co/papers/2607.02963) | |
| - Code: [github.com/showlab/PadCaptioner](https://github.com/showlab/PadCaptioner) | |
| - Model: [wenzhengzeng/PadCaptioner-3B](https://huggingface.co/wenzhengzeng/PadCaptioner-3B) | |
| This Space bundles the authors' `qwenvl` package (modeling, preprocessing) from the | |
| official repository and runs the released-model evaluation configuration from | |
| `scripts_run/test.sh` (greedy decoding, 0.5 s event grid, grounding threshold 0.7 with | |
| boundary snapping, coverage masking, parallel branch interval attention, record NMS at | |
| IoU 0.75). | |
| Example videos are from the | |
| [video-SALMONN_2 test set](https://huggingface.co/datasets/tsinghua-ee/video-SALMONN_2_testset) | |
| (Apache-2.0 license). | |