|
Download docs/optimization.md from SiddhuG01/PanoGS: direct link, hf CLI and curl.
- Browser
- Download file 3.18 kB
-
https://huggingface.co/spaces/SiddhuG01/PanoGS/resolve/main/docs/optimization.md
- Command line
-
hf download hf://spaces/SiddhuG01/PanoGS/docs/optimization.md
-
curl -L -o optimization.md https://huggingface.co/spaces/SiddhuG01/PanoGS/resolve/main/docs/optimization.md
3.18 kB
A newer version of the Gradio SDK is available: 6.32.0
3D Gaussian Splatting Optimization & Adaptive Densification
The PanoGS optimization engine refines 3D Gaussian positions, anisotropic scales, 3D rotations, opacities, and spherical harmonics colors against ground-truth multi-view perspective supervisory images sampled from panoramas.
1. Differentiable Photometric Loss
At each training step, a perspective camera view $\mathbf{I}{\text{pred}}$ is rendered via the differentiable PyTorch rasterizer and evaluated against the ground-truth perspective image $\mathbf{I}{\text{gt}}$:
Where:
- $\lambda = 0.2$ balances pixel-wise accuracy and structural detail.
- $\mathcal{L}1 = \frac{1}{HW} \sum{\mathbf{p}} |\mathbf{I}{\text{pred}}(\mathbf{p}) - \mathbf{I}{\text{gt}}(\mathbf{p})|$.
- $\mathcal{L}{\text{SSIM}} = 1.0 - \text{SSIM}(\mathbf{I}{\text{pred}}, \mathbf{I}_{\text{gt}})$.
2. Multi-Parameter Adam Optimization
Each attribute group in TorchGaussianModel is updated with independent Adam learning rates:
| Parameter | Tensor Shape | Learning Rate | Representation |
|---|---|---|---|
| Position $\mathbf{\mu}$ | $(N, 3)$ | $1.6 \times 10^{-4}$ | Unconstrained world coordinates |
| Scale $\mathbf{s}_{\log}$ | $(N, 3)$ | $5.0 \times 10^{-3}$ | Log-space $\mathbf{s} = \exp(\mathbf{s}_{\log})$ |
| Rotation $\mathbf{q}$ | $(N, 4)$ | $1.0 \times 10^{-3}$ | Unit quaternion $\mathbf{q} / |\mathbf{q}|_2$ |
| Opacity $o_{\text{logit}}$ | $(N, 1)$ | $5.0 \times 10^{-2}$ | Logit $\alpha = \sigma(o_{\text{logit}})$ |
| Color $SH_0$ | $(N, 3)$ | $2.5 \times 10^{-3}$ | Zeroth-order spherical harmonics |
3. Adaptive Densification & Pruning
Positional gradients are accumulated across training views:
When $\bar{\nabla}{\mathbf{\mu}} \ge \tau{\text{grad}}$ (default: $2 \times 10^{-4}$):
- Cloning (Under-reconstruction): If $\max(\mathbf{s}) < \text{scale_threshold}$: Duplicate the small Gaussian to provide additional spatial capacity.
- Splitting (Over-reconstruction): If $\max(\mathbf{s}) \ge \text{scale_threshold}$: Replace the large Gaussian with two child Gaussians with scales divided by 1.6: $$\mathbf{s}{\text{child}} = \frac{\mathbf{s}{\text{parent}}}{1.6}, \quad \mathbf{\mu}{\text{child}} = \mathbf{\mu}{\text{parent}} + \mathcal{N}(0, \mathbf{\Sigma})$$
- Pruning: Remove any Gaussian where $\alpha < \alpha_{\min}$ (default: $0.05$) or $\max(\mathbf{s}) > s_{\max}$ (default: $0.5\text{m}$).
- Opacity Reset: Every $N_{\text{reset}}$ steps, reset $\alpha = \min(\alpha, 0.1)$ to eliminate floaters and transparent artifacts.
4. CLI Training Command
# Optimize 3D Gaussians from an initial point cloud / PLY against panorama perspective views
panogs train examples/data/abcd.png --init-ply output/abcd_gaussians.ply -o output/abcd_optimized.ply --iterations 100 --views 8 --res 128