File size: 3,175 Bytes
42bed50
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
# 3D Gaussian Splatting Optimization & Adaptive Densification

The PanoGS optimization engine refines 3D Gaussian positions, anisotropic scales, 3D rotations, opacities, and spherical harmonics colors against ground-truth multi-view perspective supervisory images sampled from panoramas.

---

## 1. Differentiable Photometric Loss

At each training step, a perspective camera view $\mathbf{I}_{\text{pred}}$ is rendered via the differentiable PyTorch rasterizer and evaluated against the ground-truth perspective image $\mathbf{I}_{\text{gt}}$:

$$\mathcal{L} = (1 - \lambda) \mathcal{L}_1(\mathbf{I}_{\text{pred}}, \mathbf{I}_{\text{gt}}) + \lambda \mathcal{L}_{\text{SSIM}}(\mathbf{I}_{\text{pred}}, \mathbf{I}_{\text{gt}})$$

Where:
- $\lambda = 0.2$ balances pixel-wise accuracy and structural detail.
- $\mathcal{L}_1 = \frac{1}{HW} \sum_{\mathbf{p}} |\mathbf{I}_{\text{pred}}(\mathbf{p}) - \mathbf{I}_{\text{gt}}(\mathbf{p})|$.
- $\mathcal{L}_{\text{SSIM}} = 1.0 - \text{SSIM}(\mathbf{I}_{\text{pred}}, \mathbf{I}_{\text{gt}})$.

---

## 2. Multi-Parameter Adam Optimization

Each attribute group in `TorchGaussianModel` is updated with independent Adam learning rates:

| Parameter | Tensor Shape | Learning Rate | Representation |
| :--- | :--- | :--- | :--- |
| **Position $\mathbf{\mu}$** | $(N, 3)$ | $1.6 \times 10^{-4}$ | Unconstrained world coordinates |
| **Scale $\mathbf{s}_{\log}$** | $(N, 3)$ | $5.0 \times 10^{-3}$ | Log-space $\mathbf{s} = \exp(\mathbf{s}_{\log})$ |
| **Rotation $\mathbf{q}$** | $(N, 4)$ | $1.0 \times 10^{-3}$ | Unit quaternion $\mathbf{q} / \|\mathbf{q}\|_2$ |
| **Opacity $o_{\text{logit}}$** | $(N, 1)$ | $5.0 \times 10^{-2}$ | Logit $\alpha = \sigma(o_{\text{logit}})$ |
| **Color $SH_0$** | $(N, 3)$ | $2.5 \times 10^{-3}$ | Zeroth-order spherical harmonics |

---

## 3. Adaptive Densification & Pruning

Positional gradients are accumulated across training views:
$$\bar{\nabla}_{\mathbf{\mu}} = \frac{1}{K} \sum_{k=1}^K \|\nabla_{\mathbf{\mu}} \mathcal{L}_k\|$$

When $\bar{\nabla}_{\mathbf{\mu}} \ge \tau_{\text{grad}}$ (default: $2 \times 10^{-4}$):
1. **Cloning (Under-reconstruction)**:
   If $\max(\mathbf{s}) < \text{scale\_threshold}$: Duplicate the small Gaussian to provide additional spatial capacity.
2. **Splitting (Over-reconstruction)**:
   If $\max(\mathbf{s}) \ge \text{scale\_threshold}$: Replace the large Gaussian with two child Gaussians with scales divided by 1.6:
   $$\mathbf{s}_{\text{child}} = \frac{\mathbf{s}_{\text{parent}}}{1.6}, \quad \mathbf{\mu}_{\text{child}} = \mathbf{\mu}_{\text{parent}} + \mathcal{N}(0, \mathbf{\Sigma})$$
3. **Pruning**:
   Remove any Gaussian where $\alpha < \alpha_{\min}$ (default: $0.05$) or $\max(\mathbf{s}) > s_{\max}$ (default: $0.5\text{m}$).
4. **Opacity Reset**:
   Every $N_{\text{reset}}$ steps, reset $\alpha = \min(\alpha, 0.1)$ to eliminate floaters and transparent artifacts.

---

## 4. CLI Training Command

```bash
# Optimize 3D Gaussians from an initial point cloud / PLY against panorama perspective views
panogs train examples/data/abcd.png --init-ply output/abcd_gaussians.ply -o output/abcd_optimized.ply --iterations 100 --views 8 --res 128
```