|
Download docs/optimization.md from SiddhuG01/PanoGS: direct link, hf CLI and curl.
- Browser
- Download file 3.18 kB
-
https://huggingface.co/spaces/SiddhuG01/PanoGS/resolve/main/docs/optimization.md
- Command line
-
hf download hf://spaces/SiddhuG01/PanoGS/docs/optimization.md
-
curl -L -o optimization.md https://huggingface.co/spaces/SiddhuG01/PanoGS/resolve/main/docs/optimization.md
3.18 kB
| # 3D Gaussian Splatting Optimization & Adaptive Densification | |
| The PanoGS optimization engine refines 3D Gaussian positions, anisotropic scales, 3D rotations, opacities, and spherical harmonics colors against ground-truth multi-view perspective supervisory images sampled from panoramas. | |
| --- | |
| ## 1. Differentiable Photometric Loss | |
| At each training step, a perspective camera view $\mathbf{I}_{\text{pred}}$ is rendered via the differentiable PyTorch rasterizer and evaluated against the ground-truth perspective image $\mathbf{I}_{\text{gt}}$: | |
| $$\mathcal{L} = (1 - \lambda) \mathcal{L}_1(\mathbf{I}_{\text{pred}}, \mathbf{I}_{\text{gt}}) + \lambda \mathcal{L}_{\text{SSIM}}(\mathbf{I}_{\text{pred}}, \mathbf{I}_{\text{gt}})$$ | |
| Where: | |
| - $\lambda = 0.2$ balances pixel-wise accuracy and structural detail. | |
| - $\mathcal{L}_1 = \frac{1}{HW} \sum_{\mathbf{p}} |\mathbf{I}_{\text{pred}}(\mathbf{p}) - \mathbf{I}_{\text{gt}}(\mathbf{p})|$. | |
| - $\mathcal{L}_{\text{SSIM}} = 1.0 - \text{SSIM}(\mathbf{I}_{\text{pred}}, \mathbf{I}_{\text{gt}})$. | |
| --- | |
| ## 2. Multi-Parameter Adam Optimization | |
| Each attribute group in `TorchGaussianModel` is updated with independent Adam learning rates: | |
| | Parameter | Tensor Shape | Learning Rate | Representation | | |
| | :--- | :--- | :--- | :--- | | |
| | **Position $\mathbf{\mu}$** | $(N, 3)$ | $1.6 \times 10^{-4}$ | Unconstrained world coordinates | | |
| | **Scale $\mathbf{s}_{\log}$** | $(N, 3)$ | $5.0 \times 10^{-3}$ | Log-space $\mathbf{s} = \exp(\mathbf{s}_{\log})$ | | |
| | **Rotation $\mathbf{q}$** | $(N, 4)$ | $1.0 \times 10^{-3}$ | Unit quaternion $\mathbf{q} / \|\mathbf{q}\|_2$ | | |
| | **Opacity $o_{\text{logit}}$** | $(N, 1)$ | $5.0 \times 10^{-2}$ | Logit $\alpha = \sigma(o_{\text{logit}})$ | | |
| | **Color $SH_0$** | $(N, 3)$ | $2.5 \times 10^{-3}$ | Zeroth-order spherical harmonics | | |
| --- | |
| ## 3. Adaptive Densification & Pruning | |
| Positional gradients are accumulated across training views: | |
| $$\bar{\nabla}_{\mathbf{\mu}} = \frac{1}{K} \sum_{k=1}^K \|\nabla_{\mathbf{\mu}} \mathcal{L}_k\|$$ | |
| When $\bar{\nabla}_{\mathbf{\mu}} \ge \tau_{\text{grad}}$ (default: $2 \times 10^{-4}$): | |
| 1. **Cloning (Under-reconstruction)**: | |
| If $\max(\mathbf{s}) < \text{scale\_threshold}$: Duplicate the small Gaussian to provide additional spatial capacity. | |
| 2. **Splitting (Over-reconstruction)**: | |
| If $\max(\mathbf{s}) \ge \text{scale\_threshold}$: Replace the large Gaussian with two child Gaussians with scales divided by 1.6: | |
| $$\mathbf{s}_{\text{child}} = \frac{\mathbf{s}_{\text{parent}}}{1.6}, \quad \mathbf{\mu}_{\text{child}} = \mathbf{\mu}_{\text{parent}} + \mathcal{N}(0, \mathbf{\Sigma})$$ | |
| 3. **Pruning**: | |
| Remove any Gaussian where $\alpha < \alpha_{\min}$ (default: $0.05$) or $\max(\mathbf{s}) > s_{\max}$ (default: $0.5\text{m}$). | |
| 4. **Opacity Reset**: | |
| Every $N_{\text{reset}}$ steps, reset $\alpha = \min(\alpha, 0.1)$ to eliminate floaters and transparent artifacts. | |
| --- | |
| ## 4. CLI Training Command | |
| ```bash | |
| # Optimize 3D Gaussians from an initial point cloud / PLY against panorama perspective views | |
| panogs train examples/data/abcd.png --init-ply output/abcd_gaussians.ply -o output/abcd_optimized.ply --iterations 100 --views 8 --res 128 | |
| ``` | |