---
title: PadCaptioner
emoji: 🎬
colorFrom: indigo
colorTo: purple
sdk: gradio
sdk_version: 6.31.0
app_file: app.py
short_description: Omni-modal dense video captioning with parallel decoding
python_version: "3.12"
startup_duration_timeout: 1h
---

# PadCaptioner — Omni-Modal Dense Video Captioning

Demo of **PadCaptioner-3B** ("Parallelized Autoregressive Decoding for Omni-Modal Dense
Video Captioning", ECCV 2026). Upload a video with sound and the model produces
**temporally grounded event captions**: one `start - end caption` record per event.

The model watches the video frames *and* listens to the audio track. It first plans
event boundaries with a grounding head (the `<G>` tokens), then decodes the captions of
all planned events **in parallel** in a single forward pass — instead of captioning
events one-by-one.

- Paper: [huggingface.co/papers/2607.02963](https://huggingface.co/papers/2607.02963)
- Code: [github.com/showlab/PadCaptioner](https://github.com/showlab/PadCaptioner)
- Model: [wenzhengzeng/PadCaptioner-3B](https://huggingface.co/wenzhengzeng/PadCaptioner-3B)

This Space bundles the authors' `qwenvl` package (modeling, preprocessing) from the
official repository and runs the released-model evaluation configuration from
`scripts_run/test.sh` (greedy decoding, 0.5 s event grid, grounding threshold 0.7 with
boundary snapping, coverage masking, parallel branch interval attention, record NMS at
IoU 0.75).

Example videos are from the
[video-SALMONN_2 test set](https://huggingface.co/datasets/tsinghua-ee/video-SALMONN_2_testset)
(Apache-2.0 license).