| --- |
| license: mit |
| tags: |
| - 3d |
| - scene-completion |
| - scene-generation |
| - flow-matching |
| - tsdf |
| - sparse-transformer |
| - controlnet |
| --- |
| |
| # Seen2Scene: Completing Realistic 3D Scenes with Visibility-Guided Flow |
|
|
| [](https://quan-meng.github.io/projects/seen2scene/) |
| [](https://arxiv.org/abs/2603.28548) |
| [](https://github.com/quan-meng/seen2scene) |
| [](https://www.youtube.com/watch?v=5qJYLjMsJe8) |
|
|
| Pretrained checkpoints for **Seen2Scene**, the first flow matching-based approach that trains directly on |
| incomplete, real-world 3D scans for scene completion and generation. Seen2Scene introduces |
| **visibility-guided flow matching**, which explicitly masks out unknown regions in real scans, enabling |
| effective learning from real-world, partial observations. |
|
|
| Code, installation instructions, and inference commands live in the |
| [GitHub repository](https://github.com/quan-meng/seen2scene). |
|
|
| ## Checkpoints |
|
|
| | Component | Role | Size | Path in this repo | |
| |---|---|---|---| |
| | VAE (`AE_LOG`) | Encodes TSDF scenes into sparse latents | 925 MB | `experiments/auto_encoder/2025-12-19_01-23-28-525/checkpoint/vxl_0_011_last.ckpt` | |
| | Flow Matching generator (`GEN_LOG`) | Layout-conditioned scene generation | 4.5 GB | `experiments/auto_encoder/2025-12-19_01-23-28-525/generator/2026-02-23_16-22-25-152/checkpoint/vxl_0_011_last.ckpt` | |
| | ControlNet (`CONTROL_LOG`) | Partial-scan completion | 1.2 GB | `experiments/auto_encoder/2025-12-19_01-23-28-525/generator/2026-02-23_16-22-25-152/control/2026-02-26_14-01-23-930/checkpoint/vxl_0_011_last.ckpt` | |
|
|
| The three run folders correspond to the `AE_LOG`, `GEN_LOG`, and `CONTROL_LOG` arguments of the |
| inference commands: |
|
|
| ```text |
| AE_LOG = 2025-12-19_01-23-28-525 |
| GEN_LOG = 2026-02-23_16-22-25-152 |
| CONTROL_LOG = 2026-02-26_14-01-23-930 |
| ``` |
|
|
| - **Generation** requires the VAE and Flow Matching generator checkpoints. |
| - **Completion** requires the VAE, Flow Matching generator, and ControlNet checkpoints. |
|
|
| ## Download |
|
|
| The repository mirrors the training log hierarchy, so downloading into the repository root places |
| every checkpoint where the code expects it — no renaming or moving required. |
|
|
| ```bash |
| pip install -U "huggingface_hub[cli]" |
| |
| # From the root of your Seen2Scene checkout: |
| hf download MQ66/seen2scene --local-dir . |
| ``` |
|
|
| To fetch a single checkpoint, for example the VAE only: |
|
|
| ```bash |
| hf download MQ66/seen2scene \ |
| experiments/auto_encoder/2025-12-19_01-23-28-525/checkpoint/vxl_0_011_last.ckpt \ |
| --local-dir . |
| ``` |
|
|
| Or from Python: |
|
|
| ```python |
| from huggingface_hub import snapshot_download |
| |
| snapshot_download(repo_id="MQ66/seen2scene", local_dir=".") |
| ``` |
|
|
| ## Usage |
|
|
| After downloading, run inference from the repository root (see the |
| [GitHub README](https://github.com/quan-meng/seen2scene) for the full option list): |
|
|
| ```bash |
| # Partial-scan completion with the ControlNet. |
| python -m seen2scene.main control task:completion \ |
| --ae-log 2025-12-19_01-23-28-525 \ |
| --gen-log 2026-02-23_16-22-25-152 \ |
| --ckpt-path 2026-02-26_14-01-23-930 \ |
| --src-key tsdf_p_0.1 \ |
| --latent-key tsdf_p_1.0 \ |
| --task.num-samples 10 \ |
| --task.export-as bbox mesh volume \ |
| --slurm.cluster local |
| |
| # Layout-conditioned patch generation with the Flow Matching generator. |
| python -m seen2scene.main generator task:generation \ |
| --ae-log 2025-12-19_01-23-28-525 \ |
| --ckpt-path 2026-02-23_16-22-25-152 \ |
| --latent-key tsdf_p_1.0 \ |
| --task.num-samples 10 \ |
| --task.export-as bbox mesh \ |
| --slurm.cluster local |
| ``` |
|
|
| ## Training data |
|
|
| The models were trained on [3D-FRONT](https://tianchi.aliyun.com/specials/promotion/alibaba-3d-scene-dataset), |
| [ScanNet++](https://kaldir.vc.in.tum.de/scannetpp/), and [ARKitScenes](https://github.com/apple/ARKitScenes). |
| These datasets carry their own licenses and terms of use, which apply independently of the license of |
| these weights. |
|
|
| ## Citation |
|
|
| ```bibtex |
| @misc{meng2026seen2scene, |
| title={Seen2Scene: Completing Realistic 3D Scenes with Visibility-Guided Flow}, |
| author={Quan Meng and Yujin Chen and Lei Li and Matthias Nießner and Angela Dai}, |
| year={2026}, |
| eprint={2603.28548}, |
| archivePrefix={arXiv}, |
| primaryClass={cs.CV}, |
| url={https://arxiv.org/abs/2603.28548}, |
| } |
| ``` |
|
|
| ## License |
|
|
| Released under the [MIT License](https://github.com/quan-meng/seen2scene/blob/main/LICENSE). |
| Third-party components retain their respective licenses; see the |
| [Third-Party Notices](https://github.com/quan-meng/seen2scene/blob/main/THIRD_PARTY_NOTICES.md). |
|
|
