|
Download README.md from haoz19/scene-model-6layer: direct link, hf CLI and curl.
- Browser
- Download file 5.55 kB
-
https://huggingface.co/haoz19/scene-model-6layer/resolve/main/README.md
- Command line
-
hf download hf://haoz19/scene-model-6layer/README.md
-
curl -L -H "Authorization: Bearer $HF_TOKEN" -o README.md https://huggingface.co/haoz19/scene-model-6layer/resolve/main/README.md
5.55 kB
| license: cc-by-nc-nd-4.0 | |
| tags: | |
| - 3d | |
| - depth-estimation | |
| - multilayer-depth | |
| - point-cloud | |
| - diffusion | |
| - scene-understanding | |
| library_name: torch | |
| pipeline_tag: image-to-3d | |
| extra_gated_heading: "Request access to the World Tracing scene model" | |
| extra_gated_description: > | |
| These checkpoints are released for non-commercial research use under the | |
| **CC BY-NC-ND 4.0 license** (Attribution-NonCommercial-NoDerivatives). Please share a few | |
| details below so we can keep a light audit trail of how the | |
| weights are used in the wild. | |
| extra_gated_button_content: "Submit access request" | |
| extra_gated_fields: | |
| Full name: text | |
| Affiliation (university / company): text | |
| Country: country | |
| Primary intended use: | |
| type: select | |
| options: | |
| - Academic research | |
| - Personal / hobbyist project | |
| - Industrial research | |
| - Commercial product | |
| - Other | |
| Brief description of your intended use: text | |
| I agree to cite the World Tracing paper in any publication or release that uses these weights: checkbox | |
| # World Tracing — Scene Model (6-layer, 504 × 504, r69e) | |
| > **Weights last updated: 2026-10-30** — `r69e`, iter 428 000. | |
| > This is the **WT-S** scene model of Table 2 in the paper. | |
| ## Access | |
| The checkpoints in this repo are released under the **CC BY-NC-ND 4.0 license**. | |
| Downloads are gated behind a short form (basic contact info + intended | |
| use) so we can keep a light audit trail; access is granted right after | |
| you submit it. Then log in with `huggingface-cli login` (or set | |
| `HF_TOKEN`) and run any of the examples from the | |
| [GitHub repo](https://github.com/haoz19/world-tracing) — the `wt` | |
| package picks the token up automatically and `--ckpt r69e` triggers a | |
| normal `hf_hub_download`. | |
| EMA-only release weights for the **r69e** scene model from | |
| [*World Tracing: Generative Pixel-Aligned Geometry Beyond the Visible*](https://haoz19.github.io/world-tracing-page/). | |
| * **Repo**: <https://github.com/haoz19/world-tracing> | |
| * **Project page**: <https://haoz19.github.io/world-tracing-page/> | |
| * **Config name**: `r69e` | |
| * **Architecture**: `MultilayerXYZModel`, 1.5 B params | |
| * **Input**: 504 × 504 full-frame RGB (no alpha) | |
| * **Output**: per-layer XYZ in camera space, 6 stacked depth maps | |
| * **Training data**: synthetic indoor scene renders only (including | |
| 3D-FRONT), with 10 3D-FRONT rooms held out for evaluation. No | |
| real-world or sky data — pre-mask the sky externally for outdoor inputs. | |
| For sharper geometry on in-the-wild images, see the 840 × 840 variant | |
| [`scene-model-6layer-840`](https://huggingface.co/haoz19/scene-model-6layer-840) | |
| (`r69l`). | |
| ## 3D-FRONT benchmark (Table 2) | |
| `eval/3dfront_test50/` holds the ground truth of the 50 held-out | |
| 3D-FRONT frames of Table 2 (input RGB, the first 6 depth-peeled layers as | |
| camera-space XYZ normalised by the median depth, per-layer validity masks, | |
| intrinsics). It is downloaded automatically by | |
| [`benchmarks/3dfront/eval_3dfront.py`](https://github.com/haoz19/world-tracing/tree/main/benchmarks/3dfront), | |
| which evaluates this checkpoint by default and can also score your own | |
| predictions with the same protocol: | |
| ```bash | |
| python benchmarks/3dfront/eval_3dfront.py # this model, mean of 8 seeds | |
| python benchmarks/3dfront/eval_3dfront.py --pred-dir preds/ # your method | |
| ``` | |
| | | MAE | RMSE | AbsRel | L0 CD-L1 | L0 CD-L2 | L0 F@0.05 | All CD-L1 | All CD-L2 | All F@0.05 | | |
| |---|---:|---:|---:|---:|---:|---:|---:|---:|---:| | |
| | Paper (WT-S, 1 seed) | 0.0102 | 0.0215 | 0.0114 | 0.0093 | 0.000204 | 0.9867 | 0.0216 | 0.001255 | 0.8951 | | |
| | This checkpoint (mean of 8 seeds) | 0.0085 | 0.0194 | 0.0094 | 0.0081 | 0.000146 | 0.9925 | 0.0208 | 0.001240 | 0.8998 | | |
| The paper numbers come from an earlier checkpoint of the same run | |
| (~100k iterations before this one). The ground-truth frames are renders | |
| of [3D-FRONT](https://tianchi.aliyun.com/specials/promotion/alibaba-3d-scene-dataset) | |
| scenes and are provided for evaluation only, subject to the 3D-FRONT | |
| terms of use. | |
| ## Files | |
| | File | Size | Format | | |
| |---|---|---| | |
| | `model.pt` | 5.59 GB | bare `state_dict`, float32 | | |
| | `eval/3dfront_test50/` | 272 MB | 50 × `.npz` + `samples_3dfront_test50.json` | | |
| EMA weights only — ~26 % of the original training checkpoint. | |
| ## Usage | |
| ```bash | |
| git clone https://github.com/haoz19/world-tracing | |
| cd world-tracing | |
| pip install -e ".[viz]" | |
| python examples/infer_scene.py \ | |
| --image examples/test_images/scene/scene_indoor_01_modern_living_room__seed42.png \ | |
| --ckpt r69e \ | |
| --config r69e \ | |
| --out /tmp/wt_scene.rrd | |
| ``` | |
| Bare `--ckpt r69e` triggers `huggingface_hub.hf_hub_download` against | |
| this repo. | |
| ## Citation | |
| ```bibtex | |
| @inproceedings{zhang2026worldtracinggenerativepixelaligned, | |
| title = {World Tracing: Generative Pixel-Aligned Geometry Beyond the Visible}, | |
| author = {Hao Zhang and Mohamed El Banani and Jen-Hao Cheng and Paul Zhang | |
| and Yi Hua and Ben Mildenhall and Christoph Lassner | |
| and Narendra Ahuja and Gengshan Yang}, | |
| booktitle = {Advances in Neural Information Processing Systems (NeurIPS)}, | |
| year = {2026}, | |
| note = {Spotlight}, | |
| eprint = {2606.13652}, | |
| archivePrefix = {arXiv}, | |
| primaryClass = {cs.CV}, | |
| url = {https://arxiv.org/abs/2606.13652} | |
| } | |
| ``` | |
| ## License | |
| [CC BY-NC-ND 4.0](https://creativecommons.org/licenses/by-nc-nd/4.0/) (Attribution-NonCommercial-NoDerivatives) — see the [GitHub repo](https://github.com/haoz19/world-tracing/blob/main/LICENSE). Non-commercial research use only. | |