PanoGS / docs /gaussian_math.md
SiddhuG01's picture
Deploy PanoGS Gradio proof of concept
42bed50
|
Raw History Blame Contribute Delete
3.48 kB

A newer version of the Gradio SDK is available: 6.32.0

Upgrade

3D Gaussian Mathematics & Representation

This document details the mathematical formulation, parameterization spaces, 3D covariance computation, and spherical harmonics representation used in PanoGS.


1. 3D Gaussian Definition

A 3D Gaussian in world space is defined by its center mean $\mathbf{\mu} \in \mathbb{R}^3$ and a $3 \times 3$ symmetric positive semi-definite covariance matrix $\mathbf{\Sigma} \in \mathbb{R}^{3 \times 3}$:

G(x)=exp⁑(βˆ’12(xβˆ’ΞΌ)TΞ£βˆ’1(xβˆ’ΞΌ))G(\mathbf{x}) = \exp\left( -\frac{1}{2} (\mathbf{x} - \mathbf{\mu})^T \mathbf{\Sigma}^{-1} (\mathbf{x} - \mathbf{\mu}) \right)


2. Numerically Stable Parameterization

To ensure gradient updates remain in valid physical domains, Gaussians are represented using unconstrained internal parameters:

A. Scale $\mathbf{s} \in \mathbb{R}^3$ (Log-Space)

  • Internal parameter: $\mathbf{s}_{\log} = (\ln s_x, \ln s_y, \ln s_z)$
  • Physical scale: $\mathbf{s} = \exp(\mathbf{s}_{\log}) > 0$

B. Rotation $\mathbf{q} \in \mathbb{H}$ (Unit Quaternions)

  • Internal quaternion: $\mathbf{q} = (q_w, q_x, q_y, q_z)$
  • Normalized unit quaternion: $\tilde{\mathbf{q}} = \frac{\mathbf{q}}{|\mathbf{q}|_2}$
  • Rotation matrix $\mathbf{R} \in \text{SO}(3)$: $$\mathbf{R} = \begin{bmatrix} 1 - 2(y^2 + z^2) & 2(xy - rz) & 2(xz + ry) \ 2(xy + rz) & 1 - 2(x^2 + z^2) & 2(yz - rx) \ 2(xz - ry) & 2(yz + rx) & 1 - 2(x^2 + y^2) \end{bmatrix}$$ where $(r, x, y, z) = (\tilde{q}_w, \tilde{q}_x, \tilde{q}_y, \tilde{q}_z)$.

C. Opacity $\alpha \in (0, 1)$ (Logit-Space)

  • Internal parameter: $o_{\text{logit}} = \ln \frac{\alpha}{1 - \alpha}$
  • Physical opacity: $\alpha = \sigma(o_{\text{logit}}) = \frac{1}{1 + \exp(-o_{\text{logit}})}$

D. Color Representation (Linear-Radiance Spherical Harmonics $SH_0$)

  • Zeroth-order constant: $C_0 = \frac{1}{2\sqrt{\pi}} \approx 0.28209479177387814$
  • RGB to $SH_0$: $\mathbf{f}_{\text{dc}} = \frac{\text{RGB} - 0.5}{C_0}$, where RGB is linear-light radiance.
  • $SH_0$ to RGB: $\text{RGB} = \max(\mathbf{f}_{\text{dc}} \cdot C_0 + 0.5, 0.0)$. Values above 1.0 are retained by the float PLY representation for HDR.
  • The legacy 32-byte .splat format stores 8-bit color and clips radiance to the display range. PanoGS .hdrsplat and Gaussian PLY preserve float radiance; the viewer converts HDR color to display sRGB and its exposure control sets the visible highlight range.
  • .hdrsplat is PanoGS-specific (44 bytes per splat: XYZ, scale, float RGB radiance, float opacity, packed rotation); use .splat when interoperability with other viewers is required.

3. 3D Covariance Matrix Formulation

The 3D covariance matrix $\mathbf{\Sigma}$ is parameterized via scale $\mathbf{S} = \text{diag}(s_x, s_y, s_z)$ and rotation $\mathbf{R}$:

M=RS\mathbf{M} = \mathbf{R} \mathbf{S} Ξ£=MMT=RSSTRT\mathbf{\Sigma} = \mathbf{M} \mathbf{M}^T = \mathbf{R} \mathbf{S} \mathbf{S}^T \mathbf{R}^T

This matrix decomposition guarantees that $\mathbf{\Sigma}$ is symmetric and positive semi-definite ($\forall \mathbf{v} \neq 0, \mathbf{v}^T \mathbf{\Sigma} \mathbf{v} \ge 0$).


4. Adaptive Initialization from Point Clouds

Rather than assigning an arbitrary constant scale to all Gaussians, PanoGS initializes each Gaussian's scale $s_i$ dynamically based on the average Euclidean distance to its $k=3$ nearest spatial neighbors via cKDTree:

si=clamp(1kβˆ‘j∈Nk(i)βˆ₯piβˆ’pjβˆ₯2,β€…β€Šsmin⁑,β€…β€Šsmax⁑)s_i = \text{clamp}\left( \frac{1}{k} \sum_{j \in \mathcal{N}_k(i)} \|\mathbf{p}_i - \mathbf{p}_j\|_2, \; s_{\min}, \; s_{\max} \right)