README.md

metadata

title: PadCaptioner
emoji: 🎬
colorFrom: indigo
colorTo: purple
sdk: gradio
sdk_version: 6.31.0
app_file: app.py
short_description: Omni-modal dense video captioning with parallel decoding
python_version: '3.12'
startup_duration_timeout: 1h

PadCaptioner — Omni-Modal Dense Video Captioning

Demo of PadCaptioner-3B ("Parallelized Autoregressive Decoding for Omni-Modal Dense Video Captioning", ECCV 2026). Upload a video with sound and the model produces temporally grounded event captions: one start - end caption record per event.

The model watches the video frames and listens to the audio track. It first plans event boundaries with a grounding head (the <G> tokens), then decodes the captions of all planned events in parallel in a single forward pass — instead of captioning events one-by-one.

• Paper: huggingface.co/papers/2607.02963
• Code: github.com/showlab/PadCaptioner
• Model: wenzhengzeng/PadCaptioner-3B

This Space bundles the authors' qwenvl package (modeling, preprocessing) from the official repository and runs the released-model evaluation configuration from scripts_run/test.sh (greedy decoding, 0.5 s event grid, grounding threshold 0.7 with boundary snapping, coverage masking, parallel branch interval attention, record NMS at IoU 0.75).

Example videos are from the video-SALMONN_2 test set (Apache-2.0 license).