PanoGS / docs /optimization.md
SiddhuG01's picture
Deploy PanoGS Gradio proof of concept
42bed50
|
Raw History Blame Contribute Delete
3.18 kB

A newer version of the Gradio SDK is available: 6.32.0

Upgrade

3D Gaussian Splatting Optimization & Adaptive Densification

The PanoGS optimization engine refines 3D Gaussian positions, anisotropic scales, 3D rotations, opacities, and spherical harmonics colors against ground-truth multi-view perspective supervisory images sampled from panoramas.


1. Differentiable Photometric Loss

At each training step, a perspective camera view $\mathbf{I}{\text{pred}}$ is rendered via the differentiable PyTorch rasterizer and evaluated against the ground-truth perspective image $\mathbf{I}{\text{gt}}$:

L=(1βˆ’Ξ»)L1(Ipred,Igt)+Ξ»LSSIM(Ipred,Igt)\mathcal{L} = (1 - \lambda) \mathcal{L}_1(\mathbf{I}_{\text{pred}}, \mathbf{I}_{\text{gt}}) + \lambda \mathcal{L}_{\text{SSIM}}(\mathbf{I}_{\text{pred}}, \mathbf{I}_{\text{gt}})

Where:

  • $\lambda = 0.2$ balances pixel-wise accuracy and structural detail.
  • $\mathcal{L}1 = \frac{1}{HW} \sum{\mathbf{p}} |\mathbf{I}{\text{pred}}(\mathbf{p}) - \mathbf{I}{\text{gt}}(\mathbf{p})|$.
  • $\mathcal{L}{\text{SSIM}} = 1.0 - \text{SSIM}(\mathbf{I}{\text{pred}}, \mathbf{I}_{\text{gt}})$.

2. Multi-Parameter Adam Optimization

Each attribute group in TorchGaussianModel is updated with independent Adam learning rates:

Parameter Tensor Shape Learning Rate Representation
Position $\mathbf{\mu}$ $(N, 3)$ $1.6 \times 10^{-4}$ Unconstrained world coordinates
Scale $\mathbf{s}_{\log}$ $(N, 3)$ $5.0 \times 10^{-3}$ Log-space $\mathbf{s} = \exp(\mathbf{s}_{\log})$
Rotation $\mathbf{q}$ $(N, 4)$ $1.0 \times 10^{-3}$ Unit quaternion $\mathbf{q} / |\mathbf{q}|_2$
Opacity $o_{\text{logit}}$ $(N, 1)$ $5.0 \times 10^{-2}$ Logit $\alpha = \sigma(o_{\text{logit}})$
Color $SH_0$ $(N, 3)$ $2.5 \times 10^{-3}$ Zeroth-order spherical harmonics

3. Adaptive Densification & Pruning

Positional gradients are accumulated across training views: βˆ‡Λ‰ΞΌ=1Kβˆ‘k=1Kβˆ₯βˆ‡ΞΌLkβˆ₯\bar{\nabla}_{\mathbf{\mu}} = \frac{1}{K} \sum_{k=1}^K \|\nabla_{\mathbf{\mu}} \mathcal{L}_k\|

When $\bar{\nabla}{\mathbf{\mu}} \ge \tau{\text{grad}}$ (default: $2 \times 10^{-4}$):

  1. Cloning (Under-reconstruction): If $\max(\mathbf{s}) < \text{scale_threshold}$: Duplicate the small Gaussian to provide additional spatial capacity.
  2. Splitting (Over-reconstruction): If $\max(\mathbf{s}) \ge \text{scale_threshold}$: Replace the large Gaussian with two child Gaussians with scales divided by 1.6: $$\mathbf{s}{\text{child}} = \frac{\mathbf{s}{\text{parent}}}{1.6}, \quad \mathbf{\mu}{\text{child}} = \mathbf{\mu}{\text{parent}} + \mathcal{N}(0, \mathbf{\Sigma})$$
  3. Pruning: Remove any Gaussian where $\alpha < \alpha_{\min}$ (default: $0.05$) or $\max(\mathbf{s}) > s_{\max}$ (default: $0.5\text{m}$).
  4. Opacity Reset: Every $N_{\text{reset}}$ steps, reset $\alpha = \min(\alpha, 0.1)$ to eliminate floaters and transparent artifacts.

4. CLI Training Command

# Optimize 3D Gaussians from an initial point cloud / PLY against panorama perspective views
panogs train examples/data/abcd.png --init-ply output/abcd_gaussians.ply -o output/abcd_optimized.ply --iterations 100 --views 8 --res 128