Spaces:
Running on Zero
Running on Zero
multimodalart HF Staff
Size ZeroGPU duration from measured runs; keep /optimize api_name; document results
b8afc4c verified | title: Learn2Splat | |
| emoji: 🫧 | |
| colorFrom: indigo | |
| colorTo: yellow | |
| sdk: gradio | |
| sdk_version: 6.28.0 | |
| app_file: app.py | |
| short_description: Watch a meta-learned optimizer refine 3D Gaussians | |
| python_version: "3.12" | |
| startup_duration_timeout: 1h | |
| license: mit | |
| tags: | |
| - gaussian-splatting | |
| - 3d-reconstruction | |
| - learned-optimizer | |
| - novel-view-synthesis | |
| models: | |
| - autonomousvision/learn2splat | |
| # 🫧 Learn2Splat — learned 3DGS optimization | |
| Interactive demo of **[Learn2Splat: Extending the Horizon of Learned 3DGS | |
| Optimization](https://huggingface.co/papers/2605.15760)** (Pearl, Esposito, Xu, | |
| Peleg, Gschoßmann, Porzi, Kontschieder, Pons-Moll, Geiger). | |
| Learn2Splat replaces Adam with a **meta-learned update rule** for 3D Gaussian | |
| Splatting: a network that reads the current Gaussians and their rendering error | |
| and directly predicts structured parameter updates. It converges faster early on | |
| and stays stable over long horizons, with no learning-rate schedule. | |
| ## What this Space does | |
| The Mip-NeRF 360 *garden* scene — shipped by the authors inside | |
| [`autonomousvision/learn2splat`](https://huggingface.co/autonomousvision/learn2splat) — | |
| is SfM-initialized from its COLMAP point cloud once at startup. Each run then: | |
| 1. refines those Gaussians with the optimizer you pick, | |
| 2. streams the decoder's render of a training view as it converges, | |
| 3. loads the finished splats into an interactive 3D viewer, and | |
| 4. scores them by PSNR on the held-out views (every 8th image). | |
| The **Learn2Splat** and **Adam (3DGS)** options run through the *same* | |
| `OptGS.optimize_iter` loop from an identical initialization, with the same view | |
| minibatches and step budget — only the update rule differs. That is the | |
| like-for-like comparison the paper makes. | |
| ## What it shows (measured on this Space) | |
| From the same SfM initialization (13.87 dB on the held-out views), 100 | |
| refinement steps over 161 training views, 8 views per step, no densification: | |
| | optimizer | held-out PSNR | time | | |
| |---|---|---| | |
| | Learn2Splat (learned) | **23.91 dB** | 12.2 s | | |
| | Adam (3DGS baseline) | 20.31 dB | 9.6 s | | |
| 200 learned steps reach 24.34 dB in 21.8 s. Numbers move a little run to run — | |
| the per-step view minibatch is sampled at random. | |
| ## Implementation notes | |
| - Port of `demo.py --with-gui gradio` from the official | |
| [Learn2Splat Space](https://huggingface.co/spaces/autonomousvision/Learn2Splat) | |
| onto Gradio + ZeroGPU. The `optgs` package is vendored unchanged (MIT). | |
| - Checkpoint: `joint/checkpoints/epoch_7-step_150000.ckpt` — the released model | |
| trained with the COLMAP/SfM scene initializer, so it is in-distribution for an | |
| SfM-initialized scene. (The repo's other checkpoint, `resplat_init`, is the | |
| ReSplat feed-forward-initializer baseline and needs a different pipeline.) | |
| - The CUDA extensions (`gsplat`, `pointops`, `fused_ssim`, `nerfacc`, | |
| `fused_knn_attn`) are prebuilt in `wheels/` for the ZeroGPU runtime | |
| (Python 3.12 · torch 2.11 · CUDA 13.0 · sm_120 Blackwell), because upstream | |
| ships no wheel for that cell and compiling them at build time is not viable. | |
| ## Credits | |
| Code and weights: © the Learn2Splat authors, MIT / CC-BY-4.0. Demo scene: | |
| *garden* from the Mip-NeRF 360 dataset, redistributed by the authors in their | |
| model repository. This Space is an unofficial community port. | |