DeltaStreamer: Towards Human-Like Agentic Streaming Video Understanding
Abstract
DeltaStreamer is an agentic framework for streaming video understanding that uses change-aware memory and query-driven replay to preserve event dynamics and recover missing visual evidence for accurate answers.
We propose DeltaStreamer, a novel agentic framework for streaming video understanding inspired by how humans observe, remember, and investigate. Streaming video understanding requires a system to retain evidence before knowing which details future queries will need. Existing approaches often obscure event transitions and omit relevant details when compressing video history, and cannot recover missing visual evidence. DeltaStreamer addresses these limitations by combining change-aware memory with query-driven planning and targeted evidence gathering with replay. Before a query arrives, it continuously observes the stream and explicitly records visual changes in structured memory, preserving how events unfold rather than only what is present. When a query arrives, a query-driven planner identifies the required evidence and guides memory retrieval and targeted replay. By revisiting historical frames, the agent can recover visual details omitted during observation and ground its answer in evidence beyond the retained descriptions. This connects continuous observation with active investigation tailored to each question. Experiments on OVO-Bench and StreamingBench demonstrate improvements over representative streaming video understanding methods, with overall scores of 66.07% and 59.53%, respectively. Project page: https://bbbbchan.github.io/DeltaStreamer/
Community
Meet DeltaStreamer, an open-source agent that continuously watches video, remembers what happened, and revisits visual evidence when memory falls short. 🎬
Video keeps arriving, while questions can come at any time. Before a question appears, the agent cannot know which details will matter. By the time it does, the crucial moment may already have passed.
DeltaStreamer connects three capabilities:
- Observe changes: compare consecutive sampled frames and record both scene descriptions and visual changes.
- Organize memory: build structured textual memory with pointers back to the original visual evidence.
- Investigate on demand: formulate a query-driven plan, retrieve relevant memories, and revisit past frames when important details are missing.
Evaluations on OVO-Bench and StreamingBench show stronger overall performance than multiple existing approaches.
💻 The code and locally deployable interactive demo are available! Upload a video, ask questions, and explore the agent’s observations, memory, and planning process.
🌐 Project page: https://bbbbchan.github.io/DeltaStreamer/
⭐ Code & demo: https://github.com/BBBBchan/DeltaStreamer
We’d love to hear your feedback! If you find this work interesting, a GitHub star or an upvote here would be greatly appreciated.
👀 A new application built on DeltaStreamer is coming soon. Stay tuned!
Get this paper in your agent:
hf papers read 2610.19849 Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash Models citing this paper 1
Datasets citing this paper 0
No dataset linking this paper
Spaces citing this paper 0
No Space linking this paper
Collections including this paper 0
No Collection including this paper