Commit Β·
3cb52d4
1
Parent(s): 2d47f4f
docs: record P1 parameterization deep-dive findings
Browse filesDocuments the structural blocker (3-knob rotating-ellipse cannot reach
P1 feasibility), the original winning session's 4-knob approach with
tri_scale injection, live sweep validation (10/256 feasible), crash rate
analysis, and the agreed repair plan.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
docs/P1_PARAMETERIZATION_DEEPDIVE.md
ADDED
|
@@ -0,0 +1,439 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# P1 Parameterization Deep-Dive
|
| 2 |
+
|
| 3 |
+
**Date:** 2026-03-07
|
| 4 |
+
**Status:** Findings complete. Implementation pending.
|
| 5 |
+
|
| 6 |
+
This document records the investigation into why the current 3-knob rotating-ellipse
|
| 7 |
+
environment cannot produce P1-feasible designs, what the original winning session
|
| 8 |
+
actually did, and the validated plan for fixing it.
|
| 9 |
+
|
| 10 |
+
---
|
| 11 |
+
|
| 12 |
+
## 1. The Structural Blocker
|
| 13 |
+
|
| 14 |
+
### Symptom
|
| 15 |
+
|
| 16 |
+
The environment's 3-parameter action space (`aspect_ratio`, `elongation`,
|
| 17 |
+
`rotational_transform`) cannot satisfy the P1 constraints regardless of parameter
|
| 18 |
+
values.
|
| 19 |
+
|
| 20 |
+
### Evidence
|
| 21 |
+
|
| 22 |
+
A 125-point grid sweep over the full 3-knob range with the real `constellaration`
|
| 23 |
+
verifier:
|
| 24 |
+
|
| 25 |
+
```
|
| 26 |
+
aspect_ratio β [2.0, 8.0] (5 values)
|
| 27 |
+
elongation β [1.0, 5.0] (5 values)
|
| 28 |
+
rot_transform β [0.1, 1.0] (5 values)
|
| 29 |
+
n_field_periods = 3
|
| 30 |
+
```
|
| 31 |
+
|
| 32 |
+
**Result: 0/125 feasible.** Every configuration produced:
|
| 33 |
+
|
| 34 |
+
- `average_triangularity β +0.005` (constraint requires `β€ -0.5`, gap of ~0.505)
|
| 35 |
+
- `edge_iota_over_nfp β 0.05-0.22` (constraint requires `β₯ 0.3`)
|
| 36 |
+
|
| 37 |
+
Varying `n_field_periods` (3, 4, 5) did not change the result. The
|
| 38 |
+
`generate_rotating_ellipse` function structurally produces near-zero triangularity
|
| 39 |
+
regardless of its input parameters.
|
| 40 |
+
|
| 41 |
+
### Root cause
|
| 42 |
+
|
| 43 |
+
`generate_rotating_ellipse(aspect_ratio, elongation, rotational_transform,
|
| 44 |
+
n_field_periods)` sets Fourier coefficients that define the boundary shape. The
|
| 45 |
+
`m=2, n=0` mode (which controls triangularity) is not meaningfully set by any of the
|
| 46 |
+
three input parameters. Triangularity is structurally fixed near zero.
|
| 47 |
+
|
| 48 |
+
The `rotational_transform` range `[0.1, 1.0]` is also too low. Even with injected
|
| 49 |
+
triangularity, `edge_iota_over_nfp` doesn't reach 0.3 until `rotational_transform β 1.5+`.
|
| 50 |
+
|
| 51 |
+
---
|
| 52 |
+
|
| 53 |
+
## 2. The Original Winning Session
|
| 54 |
+
|
| 55 |
+
### Source
|
| 56 |
+
|
| 57 |
+
The original session that found P1-feasible designs is documented in:
|
| 58 |
+
|
| 59 |
+
```
|
| 60 |
+
ai-sci-feasible-designs/docs/harness/raw-session.md
|
| 61 |
+
```
|
| 62 |
+
|
| 63 |
+
Session: `rollout-2026-01-05T10-43-45-019b8bd3-14d6-7253-8235-f732ee43d683.jsonl`
|
| 64 |
+
(25,012 lines, 200 agent messages, Jan 5-9 2026)
|
| 65 |
+
|
| 66 |
+
### What the agent actually did for P1
|
| 67 |
+
|
| 68 |
+
1. Built `scripts/search_p1_lowdim.py` β a rotating-ellipse sweep with a **4th knob**
|
| 69 |
+
2. Found 3 feasible designs (feasibility 0.0) within ~20 minutes
|
| 70 |
+
3. Refined with trust-region local optimizer around those seeds
|
| 71 |
+
4. Downloaded scadena-pf leaderboard seed from HuggingFace as anchor
|
| 72 |
+
5. Ran `scripts/p1_alm_ngopt_multifidelity.py` (ALM + NGOpt multi-fidelity)
|
| 73 |
+
6. Final result: score 0.970141, beating leaderboard 0.969457
|
| 74 |
+
|
| 75 |
+
### The 4th knob: `tri_scale`
|
| 76 |
+
|
| 77 |
+
The script `search_p1_lowdim.py` (recovered from git at commit `300c191`) does NOT
|
| 78 |
+
use the raw `generate_rotating_ellipse` output. After generating the base shape, it:
|
| 79 |
+
|
| 80 |
+
1. Expands Fourier resolution: `set_max_mode_numbers(surface, max_poloidal_mode=3, max_toroidal_mode=3)`
|
| 81 |
+
2. Injects triangularity: `r_cos[2, center] = -tri_scale * minor_radius`
|
| 82 |
+
3. Cleans auxiliary modes: `r_cos[0, :center] = 0.0`, `z_sin[0, :center+1] = 0.0`
|
| 83 |
+
4. Returns the modified `SurfaceRZFourier`
|
| 84 |
+
|
| 85 |
+
The `tri_scale` knob directly controls the `m=2, n=0` Fourier mode, which is what
|
| 86 |
+
drives `average_triangularity` in the physics evaluation. This is the missing piece.
|
| 87 |
+
|
| 88 |
+
### Parameter ranges from the original script
|
| 89 |
+
|
| 90 |
+
```python
|
| 91 |
+
aspect_ratio: [3.0, 3.6]
|
| 92 |
+
elongation: [1.4, 2.2]
|
| 93 |
+
rotational_transform: [1.5, 2.2] # NOTE: much higher than our [0.1, 1.0]
|
| 94 |
+
tri_scale: [0.55, 0.8]
|
| 95 |
+
```
|
| 96 |
+
|
| 97 |
+
---
|
| 98 |
+
|
| 99 |
+
## 3. The Harness Campaign Results
|
| 100 |
+
|
| 101 |
+
### All campaigns found zero feasible designs
|
| 102 |
+
|
| 103 |
+
Queried SQLite databases across all P1 runs in `ai-sci-feasible-designs/runs/`:
|
| 104 |
+
|
| 105 |
+
| Run | Candidates | Feasible | Best feasibility |
|
| 106 |
+
|-----|-----------|----------|-----------------|
|
| 107 |
+
| p1_campaign | 58 | 0 | 0.615 |
|
| 108 |
+
| p1_campaign_v2 | 50 | 0 | 0.416 |
|
| 109 |
+
| p1_e2e_validate | 7 | 0 | 0.639 |
|
| 110 |
+
| p1_live | 23 | 0 | 0.569 |
|
| 111 |
+
|
| 112 |
+
The campaign candidates used full Fourier boundaries (`5x9` arrays, `n_field_periods=5`),
|
| 113 |
+
not the low-dimensional rotating-ellipse family.
|
| 114 |
+
|
| 115 |
+
### Postmortem diagnosis (from P1_CAMPAIGN_POSTMORTEM.md)
|
| 116 |
+
|
| 117 |
+
The campaign's `_COLD_START_GUIDANCE` told the LLM "Do NOT generate from scratch" and
|
| 118 |
+
"Start with small perturbations (0.01-0.05)." The winning raw-session agent did the
|
| 119 |
+
exact opposite: broad sweeps, large parameter variations, rotating-ellipse seeds. The
|
| 120 |
+
guidance actively prohibited the winning strategy.
|
| 121 |
+
|
| 122 |
+
---
|
| 123 |
+
|
| 124 |
+
## 4. Live Sweep Validation
|
| 125 |
+
|
| 126 |
+
### 4-knob sweep with `tri_scale` injection
|
| 127 |
+
|
| 128 |
+
A 256-point grid sweep using the same boundary construction as `search_p1_lowdim.py`:
|
| 129 |
+
|
| 130 |
+
```
|
| 131 |
+
aspect_ratio β [3.2, 3.8] step 0.2 (4 values)
|
| 132 |
+
elongation β [1.2, 1.8] step 0.2 (4 values)
|
| 133 |
+
rot_transform β [1.2, 1.8] step 0.2 (4 values)
|
| 134 |
+
tri_scale β [0.4, 0.7] step 0.1 (4 values)
|
| 135 |
+
n_field_periods = 3
|
| 136 |
+
mpol = 3, ntor = 3
|
| 137 |
+
```
|
| 138 |
+
|
| 139 |
+
**Result (reproduced twice):**
|
| 140 |
+
|
| 141 |
+
```
|
| 142 |
+
Total configs: 256
|
| 143 |
+
Evaluated: 228 (VMEC succeeded)
|
| 144 |
+
Crashed: 28 (VMEC solver failed)
|
| 145 |
+
Feasible: 10
|
| 146 |
+
Crash rate: 10.9%
|
| 147 |
+
Feasibility rate (of evaluated): 4.4%
|
| 148 |
+
```
|
| 149 |
+
|
| 150 |
+
### Top feasible results
|
| 151 |
+
|
| 152 |
+
| AR | elong | rot_t | tri_s | AR_out | tri | iota/nfp | elong_out | feas | ok |
|
| 153 |
+
|----|-------|-------|-------|--------|-----|----------|-----------|------|----|
|
| 154 |
+
| 3.6 | 1.4 | 1.6 | 0.60 | 3.287 | -0.5003 | 0.3005 | 7.3751 | 0.0000 | YES |
|
| 155 |
+
| 3.6 | 1.4 | 1.8 | 0.60 | 3.287 | -0.5003 | 0.3481 | 8.9318 | 0.0000 | YES |
|
| 156 |
+
| 3.8 | 1.4 | 1.6 | 0.60 | 3.487 | -0.5003 | 0.3256 | 7.9202 | 0.0000 | YES |
|
| 157 |
+
| 3.8 | 1.6 | 1.6 | 0.60 | 3.474 | -0.5037 | 0.3168 | 8.0626 | 0.0000 | YES |
|
| 158 |
+
| 3.8 | 1.8 | 1.6 | 0.60 | 3.459 | -0.5075 | 0.3097 | 8.2033 | 0.0000 | YES |
|
| 159 |
+
| 3.4 | 1.2 | 1.8 | 0.60 | 3.096 | -0.4977 | 0.3276 | 8.0849 | 0.0046 | YES |
|
| 160 |
+
| 3.8 | 1.2 | 1.6 | 0.60 | 3.496 | -0.4977 | 0.3345 | 7.7908 | 0.0046 | YES |
|
| 161 |
+
| 3.6 | 1.2 | 1.8 | 0.60 | 3.296 | -0.4977 | 0.3535 | 8.8140 | 0.0046 | YES |
|
| 162 |
+
| 3.2 | 1.2 | 1.8 | 0.60 | 2.896 | -0.4977 | 0.2995 | 7.4314 | 0.0046 | YES |
|
| 163 |
+
| 3.6 | 1.2 | 1.6 | 0.60 | 3.296 | -0.4977 | 0.3105 | 7.2363 | 0.0046 | YES |
|
| 164 |
+
|
| 165 |
+
**Key observations:**
|
| 166 |
+
|
| 167 |
+
- All feasible designs have `tri_scale = 0.60`
|
| 168 |
+
- `rot_transform β {1.6, 1.8}` only β lower values never reach feasibility
|
| 169 |
+
- `average_triangularity` clusters at `-0.500` to `-0.508` (right at the constraint)
|
| 170 |
+
- Triangularity is always the binding constraint (within the 1% tolerance)
|
| 171 |
+
- `max_elongation` ranges from 7.2 to 8.9 (score ~0.31 to ~0.12). Significant room
|
| 172 |
+
for optimization compared to the winning score of 0.970 (elongation 1.27)
|
| 173 |
+
|
| 174 |
+
### Crash rate by `rotational_transform`
|
| 175 |
+
|
| 176 |
+
| rot_transform | crash rate | feasible |
|
| 177 |
+
|---------------|-----------|----------|
|
| 178 |
+
| 1.2 | 0% (0/64) | 0 |
|
| 179 |
+
| 1.4 | 0% (0/64) | 0 |
|
| 180 |
+
| 1.6 | 0% (0/64) | 6 |
|
| 181 |
+
| 1.8 | 44% (28/64) | 4 |
|
| 182 |
+
|
| 183 |
+
**`rot_transform = 1.6` is the sweet spot:** zero crashes, highest feasible count.
|
| 184 |
+
|
| 185 |
+
### VMEC crash zone
|
| 186 |
+
|
| 187 |
+
Tested the extreme region (`rot_transform β [2.0, 2.4]`, `tri_scale β [0.7, 0.9]`):
|
| 188 |
+
|
| 189 |
+
```
|
| 190 |
+
600/600 crashed (100%)
|
| 191 |
+
```
|
| 192 |
+
|
| 193 |
+
VMEC solver fails universally when the boundary is too distorted. The crash boundary
|
| 194 |
+
is approximately `rot_transform β₯ 2.0` combined with `tri_scale β₯ 0.7`.
|
| 195 |
+
|
| 196 |
+
---
|
| 197 |
+
|
| 198 |
+
## 5. Verifier Analysis
|
| 199 |
+
|
| 200 |
+
### What's correct in the current verifier (`server/physics.py`)
|
| 201 |
+
|
| 202 |
+
1. **`_to_evaluation_metrics`** uses `GeometricalProblem` public API:
|
| 203 |
+
- `problem.is_feasible(metrics)` β applies the 1% tolerance internally
|
| 204 |
+
- `problem.compute_feasibility(metrics)` β infinity-norm of normalized violations
|
| 205 |
+
- `problem.get_objective(metrics)` β returns `(max_elongation, minimize=True)`
|
| 206 |
+
|
| 207 |
+
2. **`_score_from_objective`** matches the official formula:
|
| 208 |
+
`score = 1 - clip((max_elongation - 1) / 9, 0, 1)`
|
| 209 |
+
|
| 210 |
+
3. **Multi-fidelity split** is correct:
|
| 211 |
+
- `run` actions use low-fidelity VMEC (~0.6s per eval)
|
| 212 |
+
- `submit` uses high-fidelity VMEC (~24s per eval)
|
| 213 |
+
|
| 214 |
+
4. **Constraint constants** match the official P1 definition:
|
| 215 |
+
- `aspect_ratio β€ 4.0`
|
| 216 |
+
- `average_triangularity β€ -0.5`
|
| 217 |
+
- `edge_rotational_transform / n_field_periods β₯ 0.3`
|
| 218 |
+
|
| 219 |
+
### What needs to change
|
| 220 |
+
|
| 221 |
+
- `evaluate_params` currently takes `RotatingEllipseParams` and calls
|
| 222 |
+
`generate_rotating_ellipse` directly. It should be split into:
|
| 223 |
+
- `build_boundary_from_params(...)` β `SurfaceRZFourier` (handles mode expansion + tri_scale injection)
|
| 224 |
+
- `evaluate_boundary(boundary, fidelity)` β `EvaluationMetrics` (pure evaluation, no parameterization knowledge)
|
| 225 |
+
|
| 226 |
+
---
|
| 227 |
+
|
| 228 |
+
## 6. Reward Analysis
|
| 229 |
+
|
| 230 |
+
### Reward V0 structure (current, in `server/environment.py`)
|
| 231 |
+
|
| 232 |
+
```
|
| 233 |
+
Feasibility transition: Β±3.0 on crossing the feasible/infeasible boundary
|
| 234 |
+
Dual-track step shaping:
|
| 235 |
+
feasible + feasible β (prev_elongation - curr_elongation) * 10.0
|
| 236 |
+
otherwise β (prev_feasibility - curr_feasibility) * 5.0
|
| 237 |
+
Per-step cost: -0.1 for non-submit actions
|
| 238 |
+
Terminal bonus (submit): 5.0 * improvement_ratio + budget_efficiency
|
| 239 |
+
Terminal bonus (exhaust): 2.0 * improvement_ratio
|
| 240 |
+
Not improved penalty: -1.0 (submit) / -0.5 (exhaust)
|
| 241 |
+
```
|
| 242 |
+
|
| 243 |
+
### Assessment
|
| 244 |
+
|
| 245 |
+
**The reward is well-designed and should stay unchanged.** It only uses two scalars
|
| 246 |
+
from the verifier: `feasibility` and `objective (max_elongation)`. These are
|
| 247 |
+
problem-agnostic quantities that `GeometricalProblem` provides for any problem variant.
|
| 248 |
+
|
| 249 |
+
Things the reward correctly avoids:
|
| 250 |
+
- Per-constraint shaping (would overfit to P1's specific constraint structure)
|
| 251 |
+
- Tolerance-exploit bonus (would overfit to the 1% evaluator quirk)
|
| 252 |
+
- Mode-specific or parameter-specific weighting
|
| 253 |
+
- Any knowledge of which knob controls which metric
|
| 254 |
+
|
| 255 |
+
**One thing to monitor during playtesting:** the `5.0` multiplier on feasibility shaping
|
| 256 |
+
may need tuning once the action space changes. Mode perturbations produce different
|
| 257 |
+
feasibility deltas per step than the old 3-knob steps. But tune from playtest data,
|
| 258 |
+
not from theory.
|
| 259 |
+
|
| 260 |
+
---
|
| 261 |
+
|
| 262 |
+
## 7. Findings from the P1 Score Chase Notes
|
| 263 |
+
|
| 264 |
+
From `ai-sci-feasible-designs/docs/P1_SCORE_CHASE_NOTES.md`:
|
| 265 |
+
|
| 266 |
+
### Best submission metrics (high-fidelity)
|
| 267 |
+
|
| 268 |
+
```
|
| 269 |
+
max_elongation = 1.266744 β score = 0.970362
|
| 270 |
+
aspect_ratio = 3.999377 (tight, but feasible)
|
| 271 |
+
average_triangularity = -0.495236 (normalized violation β 0.009529)
|
| 272 |
+
iota/nfp = 0.298946 (normalized violation β 0.003515)
|
| 273 |
+
feasibility = 0.009529 (feasible due to 1% tolerance)
|
| 274 |
+
```
|
| 275 |
+
|
| 276 |
+
### Key patterns
|
| 277 |
+
|
| 278 |
+
- The winning region is "thin": improving elongation pushes triangularity and/or iota
|
| 279 |
+
over the constraint cliff
|
| 280 |
+
- The **1% feasibility tolerance** is a first-order effect: the best scores come from
|
| 281 |
+
intentionally pushing constraints to the edge of tolerance to squeeze more elongation
|
| 282 |
+
reduction
|
| 283 |
+
- Triangularity is usually the binding constraint near the best scores; iota is second
|
| 284 |
+
- The best submission used a full Fourier boundary (`mpol=8, ntor=8`, `n_field_periods=3`)
|
| 285 |
+
refined through multiple optimization stages, not the low-dimensional parameterization
|
| 286 |
+
|
| 287 |
+
### What this means for our environment
|
| 288 |
+
|
| 289 |
+
Our feasible designs from the 4-knob sweep have `max_elongation β 7.2-8.9`
|
| 290 |
+
(score β 0.12-0.31). The winning submission has `max_elongation = 1.27` (score 0.97).
|
| 291 |
+
The gap is large β the 4-knob family can reach feasibility but cannot reach competitive
|
| 292 |
+
scores. This is expected: the environment is a stepping stone for learning
|
| 293 |
+
constraint-satisfaction and basic optimization, not a path to the leaderboard.
|
| 294 |
+
|
| 295 |
+
---
|
| 296 |
+
|
| 297 |
+
## 8. Anti-Overfitting Design
|
| 298 |
+
|
| 299 |
+
### What constitutes overfitting in this context
|
| 300 |
+
|
| 301 |
+
- Per-constraint reward weighting (e.g., "triangularity progress is worth 2x")
|
| 302 |
+
- Reward bonuses for exploiting the 1% tolerance
|
| 303 |
+
- Action-space design that hardcodes which modes matter
|
| 304 |
+
- Single fixed starting state that agents memorize
|
| 305 |
+
|
| 306 |
+
### Agreed anti-overfitting levers
|
| 307 |
+
|
| 308 |
+
1. **Multiple reset seeds** (3-10 starting configurations with bounded jitter)
|
| 309 |
+
2. **Held-out evaluation seeds** (test generalization, not memorization)
|
| 310 |
+
3. **Reward based on official scalars only** (feasibility + objective, not per-constraint)
|
| 311 |
+
4. **Domain knowledge in initial state, not reward** (good baseline params in `reset()`,
|
| 312 |
+
not constraint-specific shaping in `_compute_reward`)
|
| 313 |
+
5. **Known winners as calibration fixtures only**, not optimization targets
|
| 314 |
+
|
| 315 |
+
---
|
| 316 |
+
|
| 317 |
+
## 9. Agreed Plan
|
| 318 |
+
|
| 319 |
+
### What everyone agrees on
|
| 320 |
+
|
| 321 |
+
1. **4-knob low-dimensional action space**: `aspect_ratio`, `elongation`,
|
| 322 |
+
`rotational_transform`, `triangularity_scale`
|
| 323 |
+
2. **Boundary-based verifier**: `build_boundary_from_params(...)` + `evaluate_boundary(...)`
|
| 324 |
+
3. **Explicit VMEC crash handling**: treat solver failures as bad-but-not-fatal
|
| 325 |
+
4. **Reward V0 unchanged in spirit**: feasibility-first, scalar-only, no P1-specific shaping
|
| 326 |
+
5. **Fidelity labeling in observation**: distinguish low-fi vs high-fi metrics
|
| 327 |
+
|
| 328 |
+
### What is deferred until after build + playtest
|
| 329 |
+
|
| 330 |
+
- Exact `rotational_transform` range bounds (data suggests ~[1.2, 1.9] is useful)
|
| 331 |
+
- Exact `triangularity_scale` delta values
|
| 332 |
+
- Seed pool construction (need empirical sweep in the repaired parameterization)
|
| 333 |
+
- Whether budget should be 6 or 8
|
| 334 |
+
- Any Reward V1 changes
|
| 335 |
+
|
| 336 |
+
### Implementation order
|
| 337 |
+
|
| 338 |
+
1. `server/physics.py` β boundary-based verifier interface
|
| 339 |
+
2. `fusion_lab/models.py` β action/observation/state for 4 knobs
|
| 340 |
+
3. `server/environment.py` β reset with seed pool, discrete knob perturbations, VMEC crash handling
|
| 341 |
+
4. `server/app.py` β expose new action schema
|
| 342 |
+
5. `baselines/` β random + heuristic (repair feasibility first, then reduce elongation)
|
| 343 |
+
6. Manual playtest β verify budget is sufficient, tune ranges/deltas/seeds empirically
|
| 344 |
+
|
| 345 |
+
---
|
| 346 |
+
|
| 347 |
+
## 10. Cross-Validation Record
|
| 348 |
+
|
| 349 |
+
This plan was cross-validated with an independent agent that:
|
| 350 |
+
|
| 351 |
+
- Independently confirmed the 3-knob blocker
|
| 352 |
+
- Independently confirmed the historical `tri_scale` implementation detail
|
| 353 |
+
- Reproduced VMEC crash failures
|
| 354 |
+
- Validated the layer decomposition (verifier / environment / parameterization)
|
| 355 |
+
|
| 356 |
+
### Pushbacks from the cross-validation agent and resolution
|
| 357 |
+
|
| 358 |
+
| Pushback | Verdict | Resolution |
|
| 359 |
+
|----------|---------|------------|
|
| 360 |
+
| "10/228 feasible is unverified" | **Refuted.** Reproduced exactly twice in live execution. | 256 total, 228 evaluated, 10 feasible. |
|
| 361 |
+
| "rt=1.8 comfortably feasible is too strong" | **Partially valid.** 44% crash rate at 1.8. | rt=1.6 is the true sweet spot: 0% crashes, 6 feasible. |
|
| 362 |
+
| "Delta values are design proposals, not facts" | **Valid.** | Defer to post-build playtesting. |
|
| 363 |
+
| "Seed pool not empirically validated" | **Valid.** | Methodology sound, execution pending. |
|
| 364 |
+
| "Budget change to 8-10 is speculative" | **Valid.** | Keep 6 until playtest proves otherwise. |
|
| 365 |
+
|
| 366 |
+
---
|
| 367 |
+
|
| 368 |
+
## Appendix A: Key File Locations
|
| 369 |
+
|
| 370 |
+
### Fusion Design Lab (this repo)
|
| 371 |
+
|
| 372 |
+
```
|
| 373 |
+
server/physics.py β verifier (needs boundary-based refactor)
|
| 374 |
+
server/environment.py β environment loop + reward V0
|
| 375 |
+
fusion_lab/models.py β action/observation/state schemas
|
| 376 |
+
server/app.py β FastAPI endpoints
|
| 377 |
+
baselines/ β random + heuristic agents
|
| 378 |
+
```
|
| 379 |
+
|
| 380 |
+
### ai-sci-feasible-designs (reference repo)
|
| 381 |
+
|
| 382 |
+
```
|
| 383 |
+
docs/harness/raw-session.md β original winning session narrative
|
| 384 |
+
docs/harness/P1_CAMPAIGN_POSTMORTEM.md β why campaigns found 0 feasible
|
| 385 |
+
docs/P1_SCORE_CHASE_NOTES.md β best P1 score details + approach
|
| 386 |
+
scripts/search_p1_lowdim.py β the 4-knob sweep script (git: 300c191)
|
| 387 |
+
scripts/p1_alm_ngopt_multifidelity.py β ALM+NGOpt optimizer (git: aba75b7)
|
| 388 |
+
runs/p1_campaign*/world.sqlite β campaign evaluation databases
|
| 389 |
+
```
|
| 390 |
+
|
| 391 |
+
## Appendix B: Boundary Construction Reference
|
| 392 |
+
|
| 393 |
+
The 4-knob boundary construction, as implemented in the original
|
| 394 |
+
`search_p1_lowdim.py`:
|
| 395 |
+
|
| 396 |
+
```python
|
| 397 |
+
from constellaration.initial_guess import generate_rotating_ellipse
|
| 398 |
+
from constellaration.geometry import surface_rz_fourier
|
| 399 |
+
import numpy as np
|
| 400 |
+
|
| 401 |
+
def build_boundary(aspect_ratio, elongation, rotational_transform, tri_scale, nfp=3):
|
| 402 |
+
# 1. Generate base rotating-ellipse shape
|
| 403 |
+
surface = generate_rotating_ellipse(
|
| 404 |
+
aspect_ratio=aspect_ratio,
|
| 405 |
+
elongation=elongation,
|
| 406 |
+
rotational_transform=rotational_transform,
|
| 407 |
+
n_field_periods=nfp,
|
| 408 |
+
)
|
| 409 |
+
|
| 410 |
+
# 2. Expand to higher Fourier modes
|
| 411 |
+
surface = surface_rz_fourier.set_max_mode_numbers(
|
| 412 |
+
surface, max_poloidal_mode=3, max_toroidal_mode=3,
|
| 413 |
+
)
|
| 414 |
+
|
| 415 |
+
# 3. Inject triangularity via the m=2, n=0 Fourier mode
|
| 416 |
+
r_cos = np.asarray(surface.r_cos, dtype=float).copy()
|
| 417 |
+
z_sin = np.asarray(surface.z_sin, dtype=float).copy()
|
| 418 |
+
center = r_cos.shape[1] // 2
|
| 419 |
+
minor = float(r_cos[1, center])
|
| 420 |
+
|
| 421 |
+
r_cos[2, center] = -tri_scale * minor
|
| 422 |
+
|
| 423 |
+
# 4. Clean auxiliary modes
|
| 424 |
+
r_cos[0, :center] = 0.0
|
| 425 |
+
z_sin[0, :center + 1] = 0.0
|
| 426 |
+
|
| 427 |
+
return SurfaceRZFourier(
|
| 428 |
+
r_cos=r_cos.tolist(),
|
| 429 |
+
z_sin=z_sin.tolist(),
|
| 430 |
+
n_field_periods=nfp,
|
| 431 |
+
is_stellarator_symmetric=True,
|
| 432 |
+
)
|
| 433 |
+
```
|
| 434 |
+
|
| 435 |
+
Key details:
|
| 436 |
+
- `r_cos[1, center]` is the minor radius of the base shape
|
| 437 |
+
- `r_cos[2, center]` is the `m=2, n=0` Fourier coefficient (controls triangularity)
|
| 438 |
+
- Setting it to `-tri_scale * minor` produces negative triangularity proportional to `tri_scale`
|
| 439 |
+
- The auxiliary mode cleanup ensures the boundary is well-conditioned for VMEC
|