README.md

metadata

title: Learn2Splat
emoji: 🫧
colorFrom: indigo
colorTo: yellow
sdk: gradio
sdk_version: 6.28.0
app_file: app.py
short_description: Watch a meta-learned optimizer refine 3D Gaussians
python_version: '3.12'
startup_duration_timeout: 1h
license: mit
tags:
  - gaussian-splatting
  - 3d-reconstruction
  - learned-optimizer
  - novel-view-synthesis
models:
  - autonomousvision/learn2splat

🫧 Learn2Splat — learned 3DGS optimization

Interactive demo of Learn2Splat: Extending the Horizon of Learned 3DGS Optimization (Pearl, Esposito, Xu, Peleg, Gschoßmann, Porzi, Kontschieder, Pons-Moll, Geiger).

Learn2Splat replaces Adam with a meta-learned update rule for 3D Gaussian Splatting: a network that reads the current Gaussians and their rendering error and directly predicts structured parameter updates. It converges faster early on and stays stable over long horizons, with no learning-rate schedule.

What this Space does

The Mip-NeRF 360 garden scene — shipped by the authors inside autonomousvision/learn2splat — is SfM-initialized from its COLMAP point cloud once at startup. Each run then:

1. refines those Gaussians with the optimizer you pick,
2. streams the decoder's render of a training view as it converges,
3. loads the finished splats into an interactive 3D viewer, and
4. scores them by PSNR on the held-out views (every 8th image).

The Learn2Splat and Adam (3DGS) options run through the same OptGS.optimize_iter loop from an identical initialization, with the same view minibatches and step budget — only the update rule differs. That is the like-for-like comparison the paper makes.

What it shows (measured on this Space)

From the same SfM initialization (13.87 dB on the held-out views), 100 refinement steps over 161 training views, 8 views per step, no densification:

optimizer	held-out PSNR	time
Learn2Splat (learned)	23.91 dB	12.2 s
Adam (3DGS baseline)	20.31 dB	9.6 s

200 learned steps reach 24.34 dB in 21.8 s. Numbers move a little run to run — the per-step view minibatch is sampled at random.

Implementation notes

• Port of demo.py --with-gui gradio from the official Learn2Splat Space onto Gradio + ZeroGPU. The optgs package is vendored unchanged (MIT).
• Checkpoint: joint/checkpoints/epoch_7-step_150000.ckpt — the released model trained with the COLMAP/SfM scene initializer, so it is in-distribution for an SfM-initialized scene. (The repo's other checkpoint, resplat_init, is the ReSplat feed-forward-initializer baseline and needs a different pipeline.)
• The CUDA extensions (gsplat, pointops, fused_ssim, nerfacc, fused_knn_attn) are prebuilt in wheels/ for the ZeroGPU runtime (Python 3.12 · torch 2.11 · CUDA 13.0 · sm_120 Blackwell), because upstream ships no wheel for that cell and compiling them at build time is not viable.

Credits

Code and weights: © the Learn2Splat authors, MIT / CC-BY-4.0. Demo scene: garden from the Mip-NeRF 360 dataset, redistributed by the authors in their model repository. This Space is an unofficial community port.