---
license: mit
pipeline_tag: image-to-3d
library_name: trellis2
language:
- en
tags:
- image-to-3d
- 3d-scene-generation
- indoor-scene
- 360-image
- panorama
- equirectangular
- inspace
- eccv2026
---
# InSpace: Structure-Aware 3D Indoor Scene Generation from a Single 360° Image
**Venue:** ECCV 2026 · Malmö, Sweden
## Introduction
**InSpace** generates a complete, **asset-aware** 3D indoor scene from a **single 360°
(equirectangular) panorama**, producing a full-room mesh along with individual, separable, textured
furniture meshes. It is built on the [TRELLIS.2](https://huggingface.co/microsoft/TRELLIS.2-4B)
O-Voxel representation and extends it with a panorama-native, **structure-aware** generation
pipeline: view-selective cross-attention driven by the camera center, layout-guided structure
inversion from monocular depth, a 3D bounding-box estimator, and asset-aware shape and texture
generation with global-local hybrid attention.
Concretely, the scene is produced in three cascaded flow-matching stages: (1) estimating **Partial
Scene Geometry (PSG)** as a spatial prior, (2) generating **coarse scene structure (CSG)** with
view-selective cross-attention, and (3) producing **detailed layout and asset geometry with
textures** through a global-local hybrid attention.
## Model Details
* **Model Type:** Three-stage flow-matching framework on O-Voxel structured latents, with a
CenterPoint-based 3D bounding-box estimator
* **Input:** Single 360° equirectangular (ERP) panorama
* **Output:** Complete 3D indoor scene (structural layout + separable, textured asset meshes with
PBR materials)
* **Base Model:** TRELLIS.2-4B (sparse-structure / shape / texture VAEs and decoders)
## Key Features
* **Structure-aware scene generation:** Recovers a coherent global layout from a single 360° image,
not just isolated assets.
* **Asset-aware output:** The scene is decomposed into a *layout* (floor and walls) and *individual
objects*, each exported as its own mesh rather than a single fused blob.
* **View-selective cross-attention:** The panorama is unwrapped into 6 cubemap faces (FOV 120°), and
each voxel attends only to the faces visible from its 3D position.
* **Layout-Guided Structure Inversion (optional):** A monocular-depth (Depth-Anything-2) point
cloud, the Partial Scene Geometry, seeds coarse generation via SDEdit-style noise inversion for
better room-scale fidelity.
* **PBR materials:** Base color, roughness, metallic, and opacity, inherited from the TRELLIS.2
texture decoder.
## Checkpoints
| Folder | Component | Role | Size |
|--------|-----------|------|------|
| `erp_ss_flow_img_dit_L_16l8_bf16_spatial/` | Coarse geometry | Coarse scene structure (sparse-structure flow, view-selective cross-attention) | ~4.9 GB |
| `bbox_centerpoint/` | 3D BBox | Per-asset oriented bounding-box estimator (CenterPoint) | ~48 MB |
| `erp_slat_flow_img2shape_asset_aware_bf16/` | Asset shape | Asset-aware shape generation | ~4.9 GB |
| `erp_slat_flow_imgshape2tex_asset_aware_bf16/` | Asset texture | Asset-aware texture generation (PBR) | ~4.9 GB |
Each folder holds the EMA weight under `ckpts/`. Model configs ship with the code repository
(under `configs/`), so no `config.json` is needed here.
## Requirements
- **System:** Tested on **Linux**.
- **Hardware:** An NVIDIA GPU with at least 24 GB of memory (verified on NVIDIA A100 and H100).
- **Software:**
- The [CUDA Toolkit](https://developer.nvidia.com/cuda-toolkit-archive) (recommended 12.4).
- [Conda](https://docs.anaconda.com/miniconda/install/#quick-command-line-install) for managing dependencies.
- Python 3.8 or higher.
## Usage
Please refer to the [official GitHub repository](https://github.com/kookie12/InSpace) for
installation. InSpace is run through the repository's scripts (`demo/app_inspace_*.py`,
`eval/pipeline/eval_pipeline.py`), which load these checkpoints and chain the multi-stage pipeline
together.
```sh
# 1. Get the code and set up the environment (same env as TRELLIS.2)
git clone https://github.com/kookie12/InSpace --recursive && cd InSpace
. ./setup.sh --new-env --basic --flash-attn --nvdiffrast --nvdiffrec --cumesh --o-voxel --flexgemm
# 2. Download the checkpoints into ckpts/ (repo mirrors the local layout)
pip install -U "huggingface_hub[cli]"
hf download GwanHyeong/InSpace --include "ckpts/*" --local-dir .
# 3a. Interactive demo (pick a scene, run the pipeline stage by stage)
python demo/app_inspace_erp_front.py --port 7860
# 3b. Batch inference over the test set
python eval/pipeline/eval_pipeline.py \
--data_dir datasets/ERP_3D_FRONT_test \
--noise_mode sdedit --sdedit_alpha 0.5 --bbox_mode predicted
```
The inference code loads each checkpoint from `ckpts//ckpts/*.pt`; the matching model
config is read from the code repository's `configs/` directory.
## Dataset
InSpace is trained on **[ERP-FRONT-30K](https://huggingface.co/datasets/GwanHyeong/ERP-FRONT-30K)**,
a paired ERP-Image-to-3D indoor scene dataset built on 3D-FRONT, with **26.5K training** and **2.5K
test** ERP-image-mesh pairs (~30K total). Each room is paired with 360° ERP observations rendered
from inside the scene and covers a wide range of room sizes.
```sh
hf download GwanHyeong/ERP-FRONT-30K --repo-type dataset --local-dir datasets/
```
## Known Limitations
* **Domain of training data:** InSpace is trained on ERP-FRONT (synthetic 3D-FRONT scenes). Results
on real captured panoramas may vary; for real images the pipeline relies on monocular depth to build the Partial Scene Geometry.
* **Raw mesh artifacts:** As with TRELLIS.2, generated raw meshes may occasionally contain small
holes or minor topological discontinuities; mesh post-processing (hole-filling, remeshing) is
provided.
## Citation
InSpace has been accepted to ECCV 2026. If you find our work useful, please cite:
```bibtex
@article{koo2026inspace,
title = {InSpace: Structure-Aware 3D Indoor Scene Generation from a Single 360{\deg} Image},
author = {Koo, Gwanhyeong and Kim, Hyunsu and Kim, Youngji and Lee, Taejae and
Lim, Siwoo and Yoon, Sunjae and Yeon, Suyong and Yoo, Chang D.},
journal = {arXiv preprint arXiv:2607.03990},
year = {2026}
}
```
## License
Released under the [MIT License](https://github.com/kookie12/InSpace/blob/main/LICENSE). This work
builds on TRELLIS.2 (MIT, Microsoft). Some dependencies (e.g. nvdiffrast, nvdiffrec) carry their own
licenses.