---
title: ProcObject-Qwen3-VL-4B Video QA
emoji: 🍳
colorFrom: red
colorTo: indigo
sdk: gradio
sdk_version: 6.31.0
app_file: app.py
short_description: Object-centric video QA with temporal grounding
python_version: "3.12"
startup_duration_timeout: 30m
---

# ProcObject-Qwen3-VL-4B — Object-Centric Video QA & Temporal Grounding

Demo of **[BrightGuo/ProcObject-Qwen3-VL-4B](https://huggingface.co/BrightGuo/ProcObject-Qwen3-VL-4B)**, the fine-tuned
model of *ProcObject-10K: Benchmarking and Learning Object-Centric Procedural Understanding in Instructional Videos*
(NeurIPS 2026, Evaluations & Datasets Track).

Upload an instructional video clip and ask a question about an **object** — how its state evolves, what happened to it
before a step, what went wrong with it, whether it is ready for the next step. The model answers **and** localizes the
supporting moments in the clip, returning `[[start_second, end_second], ...]` evidence intervals.

## How it works

- **Frames**: 48 frames sampled uniformly over the clip (the model's training frame budget), resized to 512×512.
- **Prompt**: each frame is sent as an image preceded by `[Timestamp: <t>s]` (seconds from the clip start), followed by
  the question — the exact protocol of the [released benchmark harness](https://github.com/WenliangGuo/ProcObject-10K).
- **Decoding**: greedy (temperature 0).
- **Output**: a JSON object `{"answer": "...", "evidence": [[start, end], ...]}`, parsed into an answer plus a
  highlighted timeline of the grounded evidence spans.

## The five object-centric reasoning types

| Type | Asks |
|---|---|
| Precondition Grounding | what state an object had to reach before a step |
| Object State Evolution | how an object's state changes across steps |
| Counterfactual Reasoning | what would happen to an object if a step were skipped or done differently |
| Mistake Recognition | what went wrong with an object, and why it is a mistake |
| Readiness Assessment | whether an object is ready for the next step, and which visible cues show it |

## Example clips

The bundled example clips come from the [CaptainCook4D](https://captaincook4d.github.io/captain-cook/) dataset
(Apache-2.0 code license; videos distributed by its authors via the [public downloader](https://github.com/CaptainCook4D/downloader)),
one of the four sources of ProcObject-10K. They were cut to the exact benchmark clip windows used by the test split.

## Paper

[Benchmarking and Learning Object-Centric Procedural Understanding in Instructional Videos](https://arxiv.org/abs/2512.03479) —
Wenliang Guo, Yu Kong (Michigan State University). Code: [github.com/WenliangGuo/ProcObject-10K](https://github.com/WenliangGuo/ProcObject-10K);
dataset: [BrightGuo/ProcObject-10K](https://huggingface.co/datasets/BrightGuo/ProcObject-10K).