| # LIPE V2 Student Model: Refinement & Contingency Plan |
|
|
| ## 1. CNN Bottleneck: Valid Padding Strategy |
| * **Primary Plan:** Use `padding=0` for all Conv layers. |
| * Input: (1, 8, 8) |
| * Conv1 (k3, p0): -> (16, 6, 6) |
| * Conv2 (k3, p0): -> (32, 4, 4) |
| * Conv3 (k3, p0): -> (64, 2, 2) |
| * GAP: -> (64,) |
| * **Contingency:** If FLOPs are still high, implement **Depthwise Separable Convolutions** for layers 2 and 3. |
|
|
| ## 2. Fusion Logic: Asymmetric Routing |
| * **Primary Plan (Residual Addition):** |
| * Geometric MLP output dim = 256 (matches total Appearance tokens). |
| * `if State A: Combined = Appearance + Geometry` |
| * `if State B: Combined = Geometry` |
| * *Note:* This requires `nn.Linear` in Geometry branch to output 256. |
| * **Contingency (EMA Caching):** |
| * Maintain a 384-dim Concatenated vector. |
| * `if State B: Use Appearance_tokens from t-1 (cached/EMA)` to avoid shape mismatch and zero-multiplication overhead. |
|
|
| ## 3. Geometric Stability: LayerNorm Integration |
| * **Primary Plan:** Replace `Dropout(0.1)` with `nn.LayerNorm(256)` after the first hidden layer of the Geometric branch. |
| * **Contingency:** If CPU latency increases, switch to `nn.utils.weight_norm` on Linear layers to stabilize gradients without explicit normalization steps. |
|
|
| ## 4. Summary of Architecture Changes |
| | Component | From (Old Spec) | To (Refined Spec) | |
| | :--- | :--- | :--- | |
| | **CNN Padding** | `p=1` (Same) | `p=0` (Valid) | |
| | **Fusion Mode** | `Concatenation(384)` | `Residual Addition(256)` | |
| | **Regularization** | `Dropout(0.1)` | `LayerNorm + Dropout(0.05)` | |
|
|
