File size: 3,483 Bytes
42bed50
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
# 3D Gaussian Splatting CPU Renderer & Camera Geometry

The PanoGS rendering pipeline implements perspective camera geometry, affine Jacobian projection, 2D covariance computation, depth sorting, and front-to-back Over-operator alpha compositing.

---

## 1. Camera Projection & Viewport Geometry

### Pinhole Perspective Model
Given image dimensions $(W, H)$ and horizontal field of view $\text{FoV}_x$, the focal lengths and principal point are computed as:
$$f_x = \frac{W}{2 \tan(\text{FoV}_x / 2)}, \quad f_y = \frac{H}{2 \tan(\text{FoV}_y / 2)}$$
$$c_x = \frac{W}{2}, \quad c_y = \frac{H}{2}$$

World coordinates $\mathbf{P}_w \in \mathbb{R}^3$ are mapped to camera coordinate space $\mathbf{P}_c = (t_x, t_y, t_z)^T$:
$$\mathbf{P}_c = \mathbf{R}_{view} \mathbf{P}_w + \mathbf{t}_{view}$$

The 2D screen coordinate $(x_s, y_s)$ is obtained via central projection:
$$x_s = f_x \frac{t_x}{t_z} + c_x, \quad y_s = f_y \frac{t_y}{t_z} + c_y$$

---

## 2. Jacobian Matrix & 2D Covariance

Following the formulation in *3D Gaussian Splatting for Real-Time Radiance Field Rendering* (Kerbl et al., SIGGRAPH 2023), the 3D Gaussian covariance $\mathbf{\Sigma} \in \mathbb{R}^{3 \times 3}$ is projected into 2D image plane covariance $\mathbf{\Sigma}' \in \mathbb{R}^{2 \times 2}$ using the local affine approximation:

$$\mathbf{\Sigma}' = \mathbf{J} \, \mathbf{W} \, \mathbf{\Sigma} \, \mathbf{W}^T \mathbf{J}^T + s \mathbf{I}_{2 \times 2}$$

Where:
- $\mathbf{W} = \mathbf{R}_{view}$ is the camera rotation transformation.
- $\mathbf{J} \in \mathbb{R}^{2 \times 3}$ is the Jacobian matrix of the perspective projection evaluated at $\mathbf{P}_c = (t_x, t_y, t_z)$:
$$\mathbf{J} = \begin{bmatrix} \frac{f_x}{t_z} & 0 & -\frac{f_x t_x}{t_z^2} \\ 0 & \frac{f_y}{t_z} & -\frac{f_y t_y}{t_z^2} \end{bmatrix}$$
- $s = 0.3$ is a low-pass Gaussian antialiasing filter applied to the diagonal of $\mathbf{\Sigma}'$ to prevent sub-pixel sampling artifacts.

The 2D Gaussian bounding box radius is bounded by:
$$r = 3.0 \times \sqrt{\max(\mathbf{\Sigma}'_{00}, \mathbf{\Sigma}'_{11})}$$

---

## 3. Front-to-Back Alpha Compositing (The Over Operator)

Gaussians in front of the camera ($t_z > \text{min\_depth}$) are sorted in strictly ascending order of depth $t_z$:
$$\mathcal{G}_{(1)}, \mathcal{G}_{(2)}, \dots, \mathcal{G}_{(M)} \quad \text{where} \quad t_{z,(1)} \le t_{z,(2)} \le \dots \le t_{z,(M)}$$

For each pixel $\mathbf{p} = (u, v)^T$ within the bounding box of Gaussian $i$:
1. **Quadratic Form Evaluation**:
   $$\Delta = \mathbf{p} - \mathbf{x}_{s,i}$$
   $$G_i(\mathbf{p}) = \exp\left( -\frac{1}{2} \Delta^T (\mathbf{\Sigma}'_i)^{-1} \Delta \right)$$
2. **Alpha Density**:
   $$\alpha_i(\mathbf{p}) = \sigma(o_i) \cdot G_i(\mathbf{p})$$
3. **Color Accumulation & Transmittance**:
   $$C(\mathbf{p}) \leftarrow C(\mathbf{p}) + c_i \cdot \alpha_i(\mathbf{p}) \cdot T(\mathbf{p})$$
   $$T(\mathbf{p}) \leftarrow T(\mathbf{p}) \cdot (1 - \alpha_i(\mathbf{p}))$$
   where $T(\mathbf{p})$ initializes to $1.0$.

When $T(\mathbf{p}) < 10^{-4}$, the pixel is fully saturated and early-ray-termination skips subsequent Gaussians.

---

## 4. CLI Usage

Render a perspective view from an orbital camera:
```bash
# Render to default 512x512 with white background
panogs render output/gaussians.ply -o output/rendered.png --distance 2.0 --elevation 10.0 --azimuth 30.0

# Render with black background
panogs render output/gaussians.ply -o output/rendered.png --bg black --distance 1.5
```