Update README.md with 16-way Classical Geometric, Symplectic & Toroidal optimizer ablation

README.md

@@ -59,37 +59,48 @@ Starting from the pretrained [`MatterWaveAI/grt-30m`](https://huggingface.co/Mat
 
 ---
 
-## Evaluation Results & 9-Way Cyclide & Toroidal Optimizer Ablation (STS-B)
+## Evaluation Results & 16-Way Classical Geometric, Symplectic & Toroidal Optimizer Ablation (STS-B)
 
-Evaluated on the **STS Benchmark (`sentence-transformers/stsb`)** validation and test splits using `circular` pooling and cosine similarity across a 9-way controlled optimizer ablation on the identical `508`-step schedule (`150` steps AllNLI + `358` steps STS-B, `seed = 42`):
+Evaluated on the **STS Benchmark (`sentence-transformers/stsb`)** validation and test splits using `circular` pooling and cosine similarity across a 16-way controlled optimizer ablation on the identical `508`-step schedule (`150` steps AllNLI + `358` steps STS-B, `seed = 42`):
 
 | Rank (Test) | Optimizer Variant | Coupling LR | Stage 1 NLI Val Spearman (`×100`) | STS-B Val Spearman (`×100`) | STS-B Val Pearson (`×100`) | STS-B Test Spearman (`×100`) | STS-B Test Pearson (`×100`) | Optimizer State Memory (`FP32`) | Step Time (`508` Steps) | Delta vs. `PairAdam` (Val / Test) |
 |:---:|---|:---:|---:|---:|---:|---:|---:|---:|---:|---:|
-| **1** | **`PolarNaturalAdam` (Polar Metric + Hessian)** | `1.0×` | `62.31` | `72.62` | `70.20` | **`64.71` (BEST TEST)** | **`64.94` (BEST TEST)** | `80.7 MB` (`2.04×`) | `4,344s` XPU (`2.4×`) | `+0.83` / **`+2.29`** |
-| **2** | **`HessianPairAdam` (`Sophia-Torus`, `1×`)** | `1.0×` | `62.29` | `72.42` | `68.76` | **`63.37`** | `63.69` | `80.7 MB` (`2.04×`) | `2,219s` XPU (`1.2×`) | `+0.63` / **`+0.95`** |
-| **3** | **`RotorAdam` (Cayley Rational Retraction)** | `1.0×` | `62.67` | **`73.22` (#3 VAL)** | `70.99` | **`63.29`** | `63.09` | `80.7 MB` (`2.04×`) | `1,926s` XPU (`1.1×`) | **`+1.43`** / **`+0.87`** |
-| **4** | **`ToroidalLion` (Sign-Momentum on Torus)** | `1.0×` | `60.99` | `72.89` | `69.70` | **`63.19`** | `63.41` | **`39.6 MB` (`1.00×`, LOWEST)** | **`1,791s` XPU (`1.0×`, FASTEST)** | **`+1.10`** / **`+0.77`** |
-| 5 | **`AdamW` (Euclidean Diagonal Baseline)** | `1.0×` | — | `72.89` | — | `63.17` | `63.72` | `79.1 MB` (`2.00×`) | `~1,850s` XPU (`1.0×`) | `+1.10` / `+0.75` |
-| 6 | **`HessianPairAdam` (`Sophia-Torus`, `3×` Factored)** | `3.0×` | `62.72` | `72.78` | `69.17` | `62.88` | `63.16` | `44.9 MB` (`1.13×`) | `3,698s` XPU (`2.1×`) | `+0.99` / `+0.46` |
-| **7** | **`CyclideAdam` (Dupin Cyclide Conic + Deviatoric Twist)** | `1.0×` | `62.55` | **`73.68` (BEST VAL)** | **`71.79` (BEST VAL)** | `62.87` | **`64.76` (#2 TEST PEARSON)** | `80.7 MB` (`2.04×`) | **`380s` ROCm** (`~2,480s` XPU, `1.4×`) | **`+1.89`** / **`+0.45`** (`+1.46` Pearson) |
-| 8 | **`PairAdam` (Published Checkpoint, Factored)** | `3.0×` | — | `71.79` | — | `62.42` | `63.30` | `43.4 MB` (`1.10×`) | `~1,820s` XPU (`1.0×`) | `0.00` / `0.00` |
-| 9 | **`ToroidalMuon` (Newton–Schulz Cascade)** | `1.0×` | **`63.76` (BEST S1)** | **`73.61` (#2 VAL)** | **`71.05` (#2 VAL)** | `61.97` | `63.04` | `80.7 MB` (`2.04×`) | `5,775s` XPU (`3.2×`) | **`+1.82`** / `-0.45` |
+| **1** | **`SymplecticAdam` (Non-Squeezing Floor + Casimir Cavity)** | `1.0×` | `61.98` | **`73.60` (#4 VAL)** | `70.87` | **`65.16` (BEST TEST)** | **`65.64` (BEST TEST)** | `80.7 MB` (`2.04×`) | **`243s` ROCm** | **`+1.81`** / **`+2.74`** |
+| **2** | **`PolarNaturalAdam` (Polar Metric + Hessian)** | `1.0×` | `62.31` | `72.62` | `70.20` | **`64.71` (#2 TEST)** | `64.94` (#3 TEST) | `80.7 MB` (`2.04×`) | `4,344s` XPU | `+0.83` / **`+2.29`** |
+| **3** | **`GyricAdam` (Thomas–Landé Gyroscopic Projection)** | `1.0×` | `62.21` | `73.04` | `70.23` | **`64.55` (#3 TEST)** | **`65.03` (#2 TEST)** | `80.7 MB` (`2.04×`) | `420s` ROCm | **`+1.25`** / **`+2.13`** |
+| **4** | **`HuygensAdam` (Circular Order-Parameter Splay State)** | `1.0×` | **`63.83` (BEST S1)** | `72.95` | `70.35` | **`64.20` (#4 TEST)** | `64.69` | `80.7 MB` (`2.04×`) | `332s` ROCm | **`+1.16`** / **`+1.78`** |
+| **5** | **`HessianPairAdam` (`Sophia-Torus`, `1×`)** | `1.0×` | `62.29` | `72.42` | `68.76` | **`63.37`** | `63.69` | `80.7 MB` (`2.04×`) | `2,219s` XPU | `+0.63` / **`+0.95`** |
+| **6** | **`BeltramiAdam` (Force-Free Helicity Vortex Cascade)** | `1.0×` | `63.37` (#3 S1) | **`73.63` (#2 VAL)** | `70.68` | **`63.33`** | `63.28` | `80.7 MB` (`2.04×`) | `321s` ROCm | **`+1.84`** / **`+0.91`** |
+| **7** | **`HannayAdam` (Adiabatic Holonomy + Polar Metric)** | `1.0×` | `61.15` | `72.57` | `69.90` | **`63.33`** | `63.30` | `80.7 MB` (`2.04×`) | **`239s` ROCm** | `+0.78` / **`+0.91`** |
+| **8** | **`RotorAdam` (Cayley Rational Retraction)** | `1.0×` | `62.67` | `73.22` | `70.99` | **`63.29`** | `63.09` | `80.7 MB` (`2.04×`) | `1,926s` XPU | **`+1.43`** / **`+0.87`** |
+| **9** | **`ToroidalLion` (Sign-Momentum on Torus)** | `1.0×` | `60.99` | `72.89` | `69.70` | **`63.19`** | `63.41` | **`39.6 MB` (`1.00×`, LOWEST)** | `1,791s` XPU | **`+1.10`** / **`+0.77`** |
+| 10 | **`AdamW` (Euclidean Diagonal Baseline)** | `1.0×` | — | `72.89` | — | `63.17` | `63.72` | `79.1 MB` (`2.00×`) | `~1,850s` XPU | `+1.10` / `+0.75` |
+| **11** | **`PonceletAdam` (Confocal Elliptic Closure Metric)** | `1.0×` | `62.62` | `72.75` | `69.87` | `63.15` | `63.74` | `80.7 MB` (`2.04×`) | `267s` ROCm | `+0.96` / `+0.73` |
+| 12 | **`HessianPairAdam` (`Sophia-Torus`, `3×` Factored)** | `3.0×` | `62.72` | `72.78` | `69.17` | `62.88` | `63.16` | `44.9 MB` (`1.13×`) | `3,698s` XPU | `+0.99` / `+0.46` |
+| **13** | **`CyclideAdam` (Dupin Cyclide Conic + Deviatoric Twist)** | `1.0×` | `62.55` | **`73.68` (BEST VAL)** | **`71.79` (BEST VAL)** | `62.87` | **`64.76` (#4 TEST PEARSON)** | `80.7 MB` (`2.04×`) | `380s` ROCm | **`+1.89`** / `+0.45` (`+1.46` Pearson) |
+| **14** | **`RindlerAdam` (Rindler Hyperbolic Wedge Horizon)** | `1.0×` | `62.08` | `73.18` | `70.69` | `62.64` | `63.53` | `80.7 MB` (`2.04×`) | `272s` ROCm | **`+1.39`** / `+0.22` |
+| 15 | **`PairAdam` (Published Checkpoint, Factored)** | `3.0×` | — | `71.79` | — | `62.42` | `63.30` | `43.4 MB` (`1.10×`) | `~1,820s` XPU | `0.00` / `0.00` |
+| 16 | **`ToroidalMuon` (Newton–Schulz Cascade)** | `1.0×` | `63.76` (#2 S1) | **`73.61` (#3 VAL)** | **`71.05` (#2 VAL)** | `61.97` | `63.04` | `80.7 MB` (`2.04×`) | `5,775s` XPU | **`+1.82`** / `-0.45` |
 | — | **Pretrained `GRT-30M` (Zero-Shot Base)** | — | `45.51` | `45.51` | `44.14` | `28.49` | `29.81` | `0 MB` | — | — |
 
 ### Memory, Speed & Regime Tradeoffs Across GRT Optimizers (`optimization_grt.py`)
 
 For the `9,891,865` trainable parameters (`39.57 MB` in `FP32`: `190,464` rotation angles across `84` cascade tensors, `9,175,040` `Cl(2,0)` coupling parameters, and `526,361` radial scale / norm / `log_magnitude` parameters):
 
-| Optimizer | State Buffers per Param Group | Fine-Tune State Memory (`9.89M` trainable) | Full Pretrain State Memory (`29.31M` all params) | Step Overhead vs. `ToroidalLion` | Best Operating Regime & Tradeoff Summary |
+| Optimizer | State Buffers per Param Group | Fine-Tune State Memory (`9.89M` trainable) | Full Pretrain State Memory (`29.31M` all params) | Step Speed (`508` Steps) | Best Operating Regime & Tradeoff Summary |
 |---|---|:---:|:---:|:---:|---|
-| **`PolarNaturalAdam`** | Angles: `4×` (`m, v, h, c`) · Non-angles: `2×` (`m, v`) + live polar radius `r_p^2` | `80.7 MB` (`2.04×`) | `388.3 MB` (`3.31×`) | `2.42×` | **Best Out-of-Distribution Generalization (`64.71` Test Spearman / `64.94` Test Pearson, `+2.29` over `PairAdam`, `+1.54` over `AdamW`).** Equalizes arc-length displacement `r_p * d_theta` across high- and low-energy rotation planes via the polar metric `ds^2 = dr^2 + r^2 d_theta^2`. Extra time comes from per-step softplus pair-radius evaluation across 85 angle-scale pairs. |
-| **`CyclideAdam`** | Angles: `4×` (`m, v, h, c`) · Non-angles: `2×` (`m, v`) + conic metric & deviatoric projection | `80.7 MB` (`2.04×`) | `388.3 MB` (`3.31×`) | `1.38×` | **Best Validation Rank & Linear Calibration (`73.68` Val Spearman / `71.79` Val Pearson [BEST] / `64.76` Test Pearson [#2]).** Replaces flat-torus radii with a **Dupin Cyclide** manifold (`ds^2 = dr^2 + r_eff(theta)^2 d_theta^2`) derived from **Brianchon's conic tangent theorem** on the coupled pair ellipse, and applies an $O(K \cdot P)$ closed-form **deviatoric (trace-free) cascade twist** that removes redundant stage-mean drift (`+0.90` Test Spearman and `+1.72` Test Pearson over `ToroidalMuon` at less than half the step cost). |
-| **`RotorAdam`** | Angles: `4×` (`m, v, h, c`) · Non-angles: `2×` (`m, v`) | `80.7 MB` (`2.04×`) | `388.3 MB` (`3.31×`) | **`1.08×`** | **Best Speed / Accuracy Pareto Frontier (`73.22` Val / `63.29` Test Spearman in `1,926s`).** Smooth rational Cayley retraction `Delta_theta = -2 * r_c * arctan(lr * u / (2 * r_c))` on `SO(2)^P` eliminates hard clipping discontinuities with virtually zero compute overhead (`+8%` vs `ToroidalLion`). |
-| **`ToroidalLion`** | All params: **`1×` (`m` only)** + zero-state Theorem 2.1 saddle kick | **`39.6 MB` (`1.00×`)** | **`117.2 MB` (`1.00×`)** | **`1.00×` (Fastest)** | **Best Memory-Constrained & High-Throughput Optimizer (`72.89` Val / `63.19` Test Spearman in `1,791s`).** Cuts optimizer VRAM by **`50%` vs `AdamW` and `70%` vs full-pretrain `HessianPairAdam`** while still beating `AdamW` (`63.17`) and `PairAdam` (`62.42`) on Test because all angles share uniform radian units on `(S^1)^P`. |
-| **`HessianPairAdam` (`1×`)** | Angles: `4×` (`m, v, h, c`) · Non-angles: `2×` (`m, v`) | `80.7 MB` (`2.04×`) | `388.3 MB` (`3.31×`) | `1.24×` | **Best Pure Second-Order Test Generalization (`63.37` Test Spearman in `2,219s`).** Uses Theorem 2.1 Cauchy–Riemann curvature `c_theta` at zero extra backward passes; setting `coupling_lr_mult = 1.0` prevents coupling overshoot and beats `AdamW` by `+0.20` Test Spearman. |
-| **`HessianPairAdam` (`3×` Factored)** | Angles: `4×` · Coupling (`9.18M`): **`1.03×` (Adafactor row/col)** | **`44.9 MB` (`1.13×`)** | `312.9 MB` (`2.67×`) | `2.06×` | **Low-Memory Second-Order Pretraining (`72.78` Val / `62.88` Test Spearman).** Factoring the second moment `v_t` on the `9.18M` coupling parameters saves `35.8 MB` (`-44%` state memory vs diagonal `HessianPairAdam 1×`). |
-| **`PairAdam` (`3×` Factored)** | Angles: `2×` (`m, v`) · Coupling (`9.18M`): **`1.03×` (row/col)** | **`43.4 MB` (`1.10×`)** | **`198.1 MB` (`1.69×`)** | `1.02×` | **Canonical Large-Batch Pretraining Baseline (`71.79` Val / `62.42` Test Spearman).** Lowest memory among second-moment methods; designed for `524k`-token/step pretraining from scratch. |
-| **`ToroidalMuon`** | Angles: `4×` + in-place `5`-step Newton–Schulz `[K, P]` orthogonalization | `80.7 MB` (`2.04×`) | `388.3 MB` (`3.31×`) | `3.22×` | **Best Stage 1 NLI Alignment (`63.76` Stage 1 AllNLI / `73.61` STS-B Val Spearman).** Orthogonalizing updates across the `11` QKV and `21` FFN butterfly stages prevents stage collinearity and unlocks high cascade rank; best used in Stage 1 pretraining/alignment before annealing `muon_mix -> 0` for small-split calibration. |
+| **`SymplecticAdam`** | Angles: `4×` (`m, v, h, c`) · Non-angles: `2×` (`m, v`) + symplectic floor & cavity cascade | `80.7 MB` (`2.04×`) | `388.3 MB` (`3.31×`) | **`243s` ROCm (`1.02×`)** | **Best Overall Generalization & Calibration (`65.16` Test Spearman [BEST] / `65.64` Test Pearson [BEST] / `73.60` Val Spearman, `+2.74` over `PairAdam`).** Enforces Gromov–Liouville symplectic non-squeezing (`c_symp = sqrt(c_base^2 + c_rms^2 * zpe_floor^2 / (u_norm^2 + zpe_floor))`) so vanishing late-stage momentum cannot freeze into sharp training basins, coupled with Helmholtz–Casimir wave-cavity boundary smoothing and Cayley retraction. |
+| **`PolarNaturalAdam`** | Angles: `4×` (`m, v, h, c`) · Non-angles: `2×` (`m, v`) + live polar radius `r_p^2` | `80.7 MB` (`2.04×`) | `388.3 MB` (`3.31×`) | `4,344s` XPU | **Best Pure Riemannian Metric Generalization (`64.71` Test Spearman [#2] / `64.94` Test Pearson [#3], `+2.29` over `PairAdam`).** Equalizes arc-length displacement `r_p * d_theta` across high- and low-energy rotation planes via the polar metric `ds^2 = dr^2 + r^2 d_theta^2`. |
+| **`GyricAdam`** | Angles: `4×` (`m, v, h, c`) · Non-angles: `2×` (`m, v`) + Thomas–Landé projection | `80.7 MB` (`2.04×`) | `388.3 MB` (`3.31×`) | `420s` ROCm | **Best Mean (Val + Test) Balance (`73.04` Val / `64.55` Test Spearman [#3] / `65.03` Test Pearson [#2]).** Decomposes cascade velocities into collective orbital (`L`) and stage-differentiating gyric (`S`) components via Landé vector projection (`c_lande = (L^2 - S^2) / J^2`), boosting coherent rotation while retrograde Thomas precession damps incoherent inter-stage fighting. |
+| **`HuygensAdam`** | Angles: `4×` (`m, v, h, c`) · Non-angles: `2×` (`m, v`) + circular order parameter `R_p` | `80.7 MB` (`2.04×`) | `388.3 MB` (`3.31×`) | `332s` ROCm | **Best Stage 1 NLI Zero-Shot Alignment (`63.83` Stage 1 AllNLI [BEST] / `64.20` Test Spearman [#4]).** Tracks the circular synchronization order parameter `R_p = |1/K sum_k exp(i * theta_{k,p})|` and applies a repulsive splay-state restoring torque when stages bunch up, outperforming 5-step Newton–Schulz (`ToroidalMuon`) at a fraction of the compute cost. |
+| **`CyclideAdam`** | Angles: `4×` (`m, v, h, c`) · Non-angles: `2×` (`m, v`) + conic metric & deviatoric projection | `80.7 MB` (`2.04×`) | `388.3 MB` (`3.31×`) | `380s` ROCm | **Best Validation Rank & Pearson Correlation (`73.68` Val Spearman [BEST] / `71.79` Val Pearson [BEST] / `64.76` Test Pearson).** Replaces flat-torus radii with a **Dupin Cyclide** manifold derived from **Brianchon's conic tangent theorem** on the coupled pair ellipse plus a trace-free deviatoric cascade twist. |
+| **`BeltramiAdam`** | Angles: `4×` (`m, v, h, c`) · Non-angles: `2×` (`m, v`) + discrete stage vorticity | `80.7 MB` (`2.04×`) | `388.3 MB` (`3.31×`) | `321s` ROCm | **High-Helicity Cascade Alignment (`63.37` Stage 1 NLI / `73.63` Val Spearman [#2] / `63.33` Test Spearman).** Aligns deviatoric stage updates with discrete stage vorticity (`curl(u) = lambda * u`) to suppress inter-stage shear turbulence. |
+| **`HannayAdam`** | Angles: `4×` (`m, v, h, c`) · Non-angles: `2×` (`m, v`) + adiabatic scale-angle connection | `80.7 MB` (`2.04×`) | `388.3 MB` (`3.31×`) | **`239s` ROCm (`1.00×`)** | **Fastest Holonomy-Corrected Polar Optimizer (`72.57` Val / `63.33` Test Spearman in `239s`).** Subtracts the classical Hannay adiabatic holonomy drift `omega_hannay = (r1 * dr2 - r2 * dr1) / (r1^2 + r2^2)` induced by simultaneous scale updates. |
+| **`PonceletAdam` & `RindlerAdam`** | Angles: `4×` (`m, v, h, c`) · Non-angles: `2×` (`m, v`) | `80.7 MB` (`2.04×`) | `388.3 MB` (`3.31×`) | `267s–272s` ROCm | **Confocal Closure (`PonceletAdam`: `72.75` Val / `63.74` Test Pearson) & Hyperbolic Horizon Contraction (`RindlerAdam`: `73.18` Val / `63.53` Test Pearson).** Interpolates polar/conic metrics by confocal eccentricity (`PonceletAdam`) or contracts the Cayley horizon under proper angular acceleration (`RindlerAdam`). |
+| **`RotorAdam`** | Angles: `4×` (`m, v, h, c`) · Non-angles: `2×` (`m, v`) | `80.7 MB` (`2.04×`) | `388.3 MB` (`3.31×`) | `1,926s` XPU | **Smooth Cayley Retraction Baseline (`73.22` Val / `63.29` Test Spearman).** Smooth rational Cayley retraction `Delta_theta = -2 * r_c * arctan(lr * u / (2 * r_c))` on `SO(2)^P` eliminates hard clipping discontinuities. |
+| **`ToroidalLion`** | All params: **`1×` (`m` only)** + zero-state Theorem 2.1 saddle kick | **`39.6 MB` (`1.00×`)** | **`117.2 MB` (`1.00×`)** | `1,791s` XPU | **Best Memory-Constrained Optimizer (`72.89` Val / `63.19` Test Spearman).** Cuts optimizer VRAM by **`50%` vs `AdamW` and `70%` vs full-pretrain `HessianPairAdam`** while beating `AdamW` (`63.17`) and `PairAdam` (`62.42`). |
+| **`HessianPairAdam` (`1×` / `3×`) & `PairAdam`** | Angles: `2×–4×` · Coupling (`9.18M`): `1.03×` (factored) or `2×` (diag) | `43.4–80.7 MB` | `198.1–388.3 MB` | `1,820–3,698s` XPU | **Foundational Toroidal Baselines (`63.37` Test Spearman for `HessianPairAdam 1×`, `62.88` for `3×` factored, `62.42` for `PairAdam`).** |
+| **`ToroidalMuon`** | Angles: `4×` + in-place `5`-step Newton–Schulz `[K, P]` orthogonalization | `80.7 MB` (`2.04×`) | `388.3 MB` (`3.31×`) | `5,775s` XPU | **Stage-Orthogonalized Reference (`63.76` Stage 1 AllNLI / `73.61` STS-B Val Spearman).** |
 
 ---