Papers
arxiv:2610.19063

WorldMemBench: A Comprehensive Benchmark for Memory of Video World Models

Published on Oct 13
Authors:
,
,
,
,
,
,
,
,
,
,
,

Abstract

WorldMemBench evaluates world memory in video models by testing static and dynamic retention across repeated revisits and state transitions.

As video models enable longer and more realistic interactive experiences, maintaining a world consistent with its history becomes essential. This requires both preserving previously observed content and ensuring coherent world evolution over time. World memory is a critical capability to evaluate alongside visual fidelity and video duration. We introduce WorldMemBench, a benchmark for systematically evaluating world memory in video world models. Across 260 test cases, it assesses static memory through the retention of human identity and appearance, object appearance and geometry, and environmental content and geometry, and dynamic memory through event progression and state persistence. Using controlled camera departures and multi-revisit trajectories, the benchmark evaluates static content against fixed initial references and dynamic events against expected state transitions. It combines appearance and geometry metrics with event-specific assessments, and reports video quality and camera-following accuracy as complementary diagnostics. Evaluation of nine representative models shows that strong static retention does not imply reliable dynamic state evolution, while repeated revisits reveal memory degradation beyond the first return. Human studies support the alignment between our automatic evaluations and human judgments. WorldMemBench provides a systematic testbed for diagnosing memory failures and guiding the development of persistent, temporally coherent video worlds.

Community

Hi everyone! We are the authors of WorldMemBench. Here are the official resources:

Questions and feedback are welcome!

Sign up or log in to comment

Get this paper in your agent:

hf papers read 2610.19063
Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash

Models citing this paper 1

Datasets citing this paper 2

Spaces citing this paper 0

No Space linking this paper

Cite arxiv.org/abs/2610.19063 in a Space README.md to link it from this page.

Collections including this paper 1