| --- |
| license: apache-2.0 |
| language: |
| - en |
| tags: |
| - computer-vision |
| - video |
| - 3d-reconstruction |
| - 4d-reconstruction |
| - streaming |
| - instance-segmentation |
| - spatial-intelligence |
| --- |
| |
| # IGGT4D: Streaming 4D Instance-Grounded Geometry Transformer |
|
|
| [**Project Page**](https://iggt4d.github.io/) | [**Paper**](https://arxiv.org/abs/2607.19228) | [**Code**](https://github.com/HorizonRobotics/IGGT4D) |
|
|
| <p align="center"> |
| <img src="https://iggt4d.github.io/assets/images/teaser.png" alt="IGGT4D overview" width="100%"> |
| </p> |
|
|
| IGGT4D is a streaming instance-grounded geometry Transformer for online 4D scene understanding. It processes video frames sequentially, reuses historical context through causal spatial-temporal modeling, and incrementally maintains a unified representation of camera motion, scene geometry, and persistent object identity. This enables scalable feed-forward reconstruction of long, dynamic sequences with consistent geometry and instance-level understanding. |
|
|
| ## Highlights |
|
|
| - **Unified geometry-instance reconstruction:** jointly models camera motion, scene geometry, and temporally consistent instance features. |
| - **Streaming inference:** processes frames causally and updates the scene representation online for long video sequences. |
| - **Persistent object identities:** uses lightweight streaming instance clustering to associate local masks with global instance IDs. |
| - **InsScene4D-147K:** introduces a large-scale collection of real/synthetic and static/dynamic sequences with RGB, depth, poses, and temporally consistent instance masks. |
| - **Downstream applications:** supports 3D/4D reconstruction, pose estimation, instance spatial tracking, open-vocabulary segmentation, and LMM-based 4D scene grounding. |
|
|
| ## Release Status |
|
|
| The model checkpoint is hosted in this repository. Please follow the [GitHub repository](https://github.com/HorizonRobotics/IGGT4D) and [project page](https://iggt4d.github.io/) for code, dataset, benchmark, and usage updates. |
|
|
| ## Citation |
|
|
| If you find IGGT4D useful in your research, please cite: |
|
|
| ```bibtex |
| @article{zou2026iggt4d, |
| title = {IGGT4D: Streaming 4D Instance-Grounded Geometry Transformer}, |
| author = {Zou, Zhengyu and Li, Hao and Jiao, Kuixuan and Liu, Liu and Xiao, Tingyang and Zhou, Xiaolin and Hong, Fangzhou and Su, Zhizhong and Zhang, Dingwen and Liu, Ziwei}, |
| journal = {arXiv preprint arXiv:2607.19228}, |
| year = {2026} |
| } |
| ``` |
|
|
| ## License |
|
|
| The checkpoint is released under the [Apache License 2.0](https://www.apache.org/licenses/LICENSE-2.0). |
|
|
