Papers
arxiv:2604.07279

Mem3R: Streaming 3D Reconstruction with Hybrid Memory

Published on Oct 26
Authors:
,
,
,
,
,
,

Abstract

Mem3R improves long-sequence streaming 3D reconstruction by using a hybrid memory that separates camera tracking from geometric mapping, reducing model size and trajectory error.

Streaming 3D perception is well suited to robotics and augmented reality, where long visual streams must be processed efficiently and consistently. Recent recurrent models offer a promising solution by maintaining fixed-size states and enabling linear-time inference, but they often suffer from temporal forgetting over long sequences due to the limited capacity of compressed memories. We propose Mem3R, a streaming 3D reconstruction model with a hybrid memory design that decouples camera tracking from geometric mapping to improve temporal consistency over long sequences. For camera tracking, Mem3R employs an implicit fast-weight memory implemented as a lightweight Multi-Layer Perceptron updated via Test-Time Training. For geometric mapping, Mem3R maintains an explicit token-based fixed-size state. Compared with CUT3R, this design not only significantly improves long-sequence performance but also reduces the model size from 793M to 644M parameters. Mem3R supports existing improved plug-and-play state update strategies developed for CUT3R. Specifically, Mem3R combined with TTT3R reduces absolute trajectory error by up to 39% relative to CUT3R combined with TTT3R on 500- to 1000-frame sequences from the TUM Dynamics dataset. The resulting improvements also extend to other downstream tasks, including video depth estimation and 3D reconstruction. Project page: https://lck666666.github.io/Mem3R/.

Community

Sign up or log in to comment

Models citing this paper 0

No model linking this paper

Cite arxiv.org/abs/2604.07279 in a model README.md to link it from this page.

Datasets citing this paper 0

No dataset linking this paper

Cite arxiv.org/abs/2604.07279 in a dataset README.md to link it from this page.

Spaces citing this paper 0

No Space linking this paper

Cite arxiv.org/abs/2604.07279 in a Space README.md to link it from this page.

Collections including this paper 0

No Collection including this paper

Add this paper to a collection to link it from this page.