Spaces:
Sleeping
Sleeping
| title: ProcObject-Qwen3-VL-4B Video QA | |
| emoji: 🍳 | |
| colorFrom: red | |
| colorTo: indigo | |
| sdk: gradio | |
| sdk_version: 6.31.0 | |
| app_file: app.py | |
| short_description: Object-centric video QA with temporal grounding | |
| python_version: "3.12" | |
| startup_duration_timeout: 30m | |
| # ProcObject-Qwen3-VL-4B — Object-Centric Video QA & Temporal Grounding | |
| Demo of **[BrightGuo/ProcObject-Qwen3-VL-4B](https://huggingface.co/BrightGuo/ProcObject-Qwen3-VL-4B)**, the fine-tuned | |
| model of *ProcObject-10K: Benchmarking and Learning Object-Centric Procedural Understanding in Instructional Videos* | |
| (NeurIPS 2026, Evaluations & Datasets Track). | |
| Upload an instructional video clip and ask a question about an **object** — how its state evolves, what happened to it | |
| before a step, what went wrong with it, whether it is ready for the next step. The model answers **and** localizes the | |
| supporting moments in the clip, returning `[[start_second, end_second], ...]` evidence intervals. | |
| ## How it works | |
| - **Frames**: 48 frames sampled uniformly over the clip (the model's training frame budget), resized to 512×512. | |
| - **Prompt**: each frame is sent as an image preceded by `[Timestamp: <t>s]` (seconds from the clip start), followed by | |
| the question — the exact protocol of the [released benchmark harness](https://github.com/WenliangGuo/ProcObject-10K). | |
| - **Decoding**: greedy (temperature 0). | |
| - **Output**: a JSON object `{"answer": "...", "evidence": [[start, end], ...]}`, parsed into an answer plus a | |
| highlighted timeline of the grounded evidence spans. | |
| ## The five object-centric reasoning types | |
| | Type | Asks | | |
| |---|---| | |
| | Precondition Grounding | what state an object had to reach before a step | | |
| | Object State Evolution | how an object's state changes across steps | | |
| | Counterfactual Reasoning | what would happen to an object if a step were skipped or done differently | | |
| | Mistake Recognition | what went wrong with an object, and why it is a mistake | | |
| | Readiness Assessment | whether an object is ready for the next step, and which visible cues show it | | |
| ## Example clips | |
| The bundled example clips come from the [CaptainCook4D](https://captaincook4d.github.io/captain-cook/) dataset | |
| (Apache-2.0 code license; videos distributed by its authors via the [public downloader](https://github.com/CaptainCook4D/downloader)), | |
| one of the four sources of ProcObject-10K. They were cut to the exact benchmark clip windows used by the test split. | |
| ## Paper | |
| [Benchmarking and Learning Object-Centric Procedural Understanding in Instructional Videos](https://arxiv.org/abs/2512.03479) — | |
| Wenliang Guo, Yu Kong (Michigan State University). Code: [github.com/WenliangGuo/ProcObject-10K](https://github.com/WenliangGuo/ProcObject-10K); | |
| dataset: [BrightGuo/ProcObject-10K](https://huggingface.co/datasets/BrightGuo/ProcObject-10K). | |