matCercola18 commited on
Commit
7b27e61
Β·
verified Β·
1 Parent(s): cf35098

Add files using upload-large-folder tool

Browse files
Files changed (50) hide show
  1. data/histograms.png +0 -0
  2. data/system_specs_distribution.png +0 -0
  3. models/arm_prov/arm_prov.done +1 -0
  4. models/arm_prov2b/arm2b.done +1 -0
  5. models/armprov/armprov.done +1 -0
  6. models/f3/f3_s0.done +1 -0
  7. models/f3/f3_s1.done +1 -0
  8. models/f3/f3_s2.done +1 -0
  9. models/rig1/rig1.done +1 -0
  10. models/rig1w/rig1w.done +1 -0
  11. models/rig1wg/rig1wg.done +1 -0
  12. results/audit/cem_prior_summary.txt +11 -0
  13. results/audit/furuta_clip_mechanism.txt +4 -0
  14. results/audit/furuta_failure_modes.txt +7 -0
  15. results/audit/furuta_hybrid_union.txt +8 -0
  16. results/audit/furuta_modes.txt +9 -0
  17. results/audit/guidance_B_heldout.txt +14 -0
  18. results/audit/mimo_agnostic.txt +11 -0
  19. results/audit/nn_distance.txt +52 -0
  20. results/audit/pendulum_ema.txt +18 -0
  21. results/audit/pendulum_eval_ckpts_relabelM4_e150.txt +16 -0
  22. results/audit/pendulum_ksweep.txt +76 -0
  23. results/audit/pendulum_labelquality.txt +4 -0
  24. results/audit/pendulum_ood.txt +24 -0
  25. results/audit/pendulum_relabel_retrieval.txt +27 -0
  26. results/audit/rederive.txt +28 -0
  27. results/audit/siso_prereg_pooled_seeds.txt +5 -0
  28. results/audit/siso_retrieval_metric.txt +10 -0
  29. results/audit/target_range_clip.txt +9 -0
  30. results/audit/traj_gen_eval_v2_clip5.0.txt +94 -0
  31. results/audit/traj_gen_ksweep_clip5.0.txt +30 -0
  32. results/audit/w4_model_arm_v2.txt +3 -0
  33. results/furuta_v3/gate_sidebyside.txt +9 -0
  34. results/furuta_v3/traj_gen_eval_v2ctl2400_raw_clip5.txt +94 -0
  35. results/furuta_v3/traj_gen_eval_v3_ema_clip1.txt +94 -0
  36. results/furuta_v3/traj_gen_eval_v3e2400_raw_clip1.txt +94 -0
  37. results/furuta_v3/traj_gen_eval_v3s1_raw_clip1.txt +94 -0
  38. results/furuta_v3/traj_gen_v3_raw_clip1.txt +30 -0
  39. results/furuta_v3/traj_gen_v3e2400s1_raw_clip1.txt +30 -0
  40. results/furuta_v3/traj_gen_v3s1_raw_clip1.txt +30 -0
  41. results/furuta_v3/traj_gen_warmstart_v2clip5_s0.txt +17 -0
  42. results/furuta_v3/traj_gen_warmstart_v3_s0.txt +17 -0
  43. results/furuta_v3/traj_gen_warmstart_v3_s1.txt +17 -0
  44. results/furuta_v3/traj_gen_warmstart_v3e2400_n200.txt +17 -0
  45. results/furuta_v3/traj_gen_warmstart_v3e2400_s0.txt +17 -0
  46. results/furuta_v3/traj_gen_warmstart_v3e2400_s1.txt +17 -0
  47. results/furuta_v3/traj_gen_warmstart_v3e2400_s2.txt +17 -0
  48. results/furuta_v3/valloss.txt +7 -0
  49. results/k_sweep_summary.md +81 -0
  50. results/regressor_siso_train.log +11 -0
data/histograms.png ADDED
data/system_specs_distribution.png ADDED
models/arm_prov/arm_prov.done ADDED
@@ -0,0 +1 @@
 
 
1
+ 2500
models/arm_prov2b/arm2b.done ADDED
@@ -0,0 +1 @@
 
 
1
+ 2500
models/armprov/armprov.done ADDED
@@ -0,0 +1 @@
 
 
1
+ 6000
models/f3/f3_s0.done ADDED
@@ -0,0 +1 @@
 
 
1
+ 24000
models/f3/f3_s1.done ADDED
@@ -0,0 +1 @@
 
 
1
+ 24000
models/f3/f3_s2.done ADDED
@@ -0,0 +1 @@
 
 
1
+ 24000
models/rig1/rig1.done ADDED
@@ -0,0 +1 @@
 
 
1
+ 6000
models/rig1w/rig1w.done ADDED
@@ -0,0 +1 @@
 
 
1
+ 6000
models/rig1wg/rig1wg.done ADDED
@@ -0,0 +1 @@
 
 
1
+ 8000
results/audit/cem_prior_summary.txt ADDED
@@ -0,0 +1,11 @@
 
 
 
 
 
 
 
 
 
 
 
 
1
+ units 240
2
+ u_max=0.6 mgl coverage: {'e_retr': 0.5371, 'e_prior': 0.5363, 'a0': 0.5237, 'b0': 0.5236} diversity: {'e_retr': 0.201, 'e_prior': 0.162, 'a0': 0.068, 'b0': 0.019}
3
+ e_prior - e_retr: mean diff -0.0008 wins/ties/losses 9/12/19 Wilcoxon p=0.127
4
+ e_prior - b0: mean diff +0.0127 wins/ties/losses 13/22/5 Wilcoxon p=0.00568
5
+ e_retr - a0: mean diff +0.0135 wins/ties/losses 21/16/3 Wilcoxon p=6.66e-05
6
+ e_retr - b0: mean diff +0.0135 wins/ties/losses 21/16/3 Wilcoxon p=5.93e-05
7
+ u_max=0.9 mgl coverage: {'e_retr': 0.8833, 'e_prior': 0.8863, 'a0': 0.8749, 'b0': 0.8749} diversity: {'e_retr': 0.183, 'e_prior': 0.143, 'a0': 0.068, 'b0': 0.017}
8
+ e_prior - e_retr: mean diff +0.0029 wins/ties/losses 10/27/3 Wilcoxon p=0.0747
9
+ e_prior - b0: mean diff +0.0114 wins/ties/losses 11/29/0 Wilcoxon p=0.00335
10
+ e_retr - a0: mean diff +0.0085 wins/ties/losses 7/32/1 Wilcoxon p=0.0173
11
+ e_retr - b0: mean diff +0.0085 wins/ties/losses 7/32/1 Wilcoxon p=0.0173
results/audit/furuta_clip_mechanism.txt ADDED
@@ -0,0 +1,4 @@
 
 
 
 
 
1
+ T cap (z=+1) = 0.643 s; fraction of bank plants with T above cap: 0.112; max bank T 1.530
2
+ fraction of bank plants whose trajectory has any knot with |z|>1: 0.793
3
+ some of 8 nearest bank plants need T above the cap: n=33 generated+P 0.152 nearest+P 0.515
4
+ none do: n=87 generated+P 0.793 nearest+P 0.943
results/audit/furuta_failure_modes.txt ADDED
@@ -0,0 +1,7 @@
 
 
 
 
 
 
 
 
1
+ bank minority-mode (>1 pump) fraction: 0.087
2
+ N plants 120 mean gen+P success 0.617 nearest+P 0.825 (K=4)
3
+ local neighbourhood all-majority-mode: n=75 gen+P success 0.756 nearest+P 0.947
4
+ locally mixed (some minority-mode neighbours): n=45 gen+P success 0.385 nearest+P 0.622
5
+ Spearman(local latch-time CV, gen+P success): rho -0.502 p=5.28e-09
6
+ Spearman(local latch-time CV, nearest+P success): rho -0.361 p=5.22e-05
7
+ high-dispersion half: gen+P 0.439 nearest+P 0.700 | low half: gen+P 0.794 nearest+P 0.950
results/audit/furuta_hybrid_union.txt ADDED
@@ -0,0 +1,8 @@
 
 
 
 
 
 
 
 
 
1
+ seed4242 K=4: nearest 0.783 generated 0.600 union(K nearest + K generated) 0.833 model-only plants 3 ceiling 0.933; nearest K/2 + gen K/2 not computable from stored prefixes
2
+ seed4242 K=8: nearest 0.883 generated 0.650 union(K nearest + K generated) 0.883 model-only plants 0 ceiling 0.933; nearest K/2 + gen K/2 not computable from stored prefixes
3
+ seed4242 K=16: nearest 0.933 generated 0.667 union(K nearest + K generated) 0.933 model-only plants 0 ceiling 0.933; nearest K/2 + gen K/2 not computable from stored prefixes
4
+ plants the model solves that nearest K=16 does not: 0; plants nearest K=4 misses that generated K=4 solves: 3
5
+ seed9001 K=4: nearest 0.867 generated 0.550 union(K nearest + K generated) 0.900 model-only plants 2 ceiling 0.933; nearest K/2 + gen K/2 not computable from stored prefixes
6
+ seed9001 K=8: nearest 0.917 generated 0.650 union(K nearest + K generated) 0.917 model-only plants 0 ceiling 0.933; nearest K/2 + gen K/2 not computable from stored prefixes
7
+ seed9001 K=16: nearest 0.933 generated 0.683 union(K nearest + K generated) 0.933 model-only plants 0 ceiling 0.933; nearest K/2 + gen K/2 not computable from stored prefixes
8
+ plants the model solves that nearest K=16 does not: 0; plants nearest K=4 misses that generated K=4 solves: 2
results/audit/furuta_modes.txt ADDED
@@ -0,0 +1,9 @@
 
 
 
 
 
 
 
 
 
 
1
+ N 2801 T range 0.41 1.53 T quantiles [0.44 0.48 0.52 0.57 0.73]
2
+ u(tau) sign changes (swing count proxy): histogram [ 0 0 2435 88 196 62 6 6 6 1 1]
3
+ smoothed sign-change hist [ 0 0 2556 117 81 36 9 0 1 1]
4
+ NN (in plant space) mean dist 0.308
5
+ neighbour |dT|/std(T): median 0.081 p75 0.161 p90 0.323 p99 2.903
6
+ frac of plants whose NN has smoothed-sign-change count differing by >=1: 0.091 ; by >=2: 0.062
7
+ ||u_i-u_nn||: median 13.2 ; ||u_i-u_random||: median 40.4 ; ratio 0.33
8
+ kNN(8)-mean regressor of u: R^2 = 0.640
9
+ best-of-8 single neighbour trajectory distance / kNN-mean distance (median): 7.09 / 10.40
results/audit/guidance_B_heldout.txt ADDED
@@ -0,0 +1,14 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ held-out J_bb (16 fresh ICs), means over 3 seeds, n=60 plants: {'r0': 16.75, 'rc': 16.85, 'rm': 17.19, 'p0': 18.83, 'pm': 15.49, 'ps': 17.45}
2
+ search J_bb: {'r0': 16.05, 'rc': 15.18, 'rm': 16.42, 'p0': 17.02, 'pm': 11.31, 'ps': 13.97}
3
+ search
4
+ pm-rm: mean -5.11 med -5.70 p=1.1e-09 (a better on 54/60)
5
+ pm-rc: mean -3.87 med -3.38 p=2.7e-08 (a better on 50/60)
6
+ pm-r0: mean -4.74 med -4.78 p=2.7e-09 (a better on 52/60)
7
+ ps-pm: mean +2.66 med +2.68 p=3.3e-11 (a better on 2/60)
8
+ p0-r0: mean +0.97 med +0.55 p=0.14 (a better on 26/60)
9
+ held-out
10
+ pm-rm: mean -1.70 med -0.96 p=0.018 (a better on 37/60)
11
+ pm-rc: mean -1.36 med -0.62 p=0.074 (a better on 36/60)
12
+ pm-r0: mean -1.26 med -0.58 p=0.087 (a better on 36/60)
13
+ ps-pm: mean +1.96 med +1.82 p=8.7e-06 (a better on 13/60)
14
+ p0-r0: mean +2.08 med +1.54 p=0.0021 (a better on 18/60)
results/audit/mimo_agnostic.txt ADDED
@@ -0,0 +1,11 @@
 
 
 
 
 
 
 
 
 
 
 
 
1
+ random bank rows K=1 cov 0.833 per-sample 0.833 median regret 13.96% mean 45.66% p90 132.94%
2
+ random bank rows K=8 cov 0.983 per-sample 0.869 median regret 2.66% mean 4.83% p90 9.57%
3
+ bank-mean controller K=1 cov 0.892 per-sample 0.892 median regret 15.50% mean 25.15% p90 56.70%
4
+ bank-median controller K=1 cov 0.958 per-sample 0.958 median regret 5.99% mean 10.40% p90 23.42%
5
+ phi=0 (no controller) K=1 cov 1.000 per-sample 1.000 median regret 1.26% mean 5.48% p90 17.72%
6
+ bank-mean clipped to plant cap K=1 cov 0.983 per-sample 0.983 median regret 15.22% mean 22.16% p90 53.87%
7
+ paired rand8 vs retrieval K=8: n=117 median diff +0.11pp mean diff +0.06pp Wilcoxon p=0.772
8
+ paired phi0 vs retrieval K=8: n=119 median diff +0.00pp mean diff +1.21pp Wilcoxon p=0.513
9
+ paired rand8 vs model K=8 (seed0): n=118 median diff +0.02pp mean diff +0.84pp Wilcoxon p=0.639
10
+ paired phi0 vs model K=8 (seed0): n=120 median diff -0.34pp mean diff +1.54pp Wilcoxon p=0.602
11
+ oracle beaten (regret<0) -- retrieval K=8 npz: 0.03333333333333333 model: 0.03333333333333333
results/audit/nn_distance.txt ADDED
@@ -0,0 +1,52 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ == PENDULUM (standardized (l,m,b)); box l[0.5,1.5] m[0.3,1.2] b[0.02,0.3]; bank raw min/max: [0.5 0.3 0.02] [1.5 1.2 0.3]
2
+ unique bank rows: 2600 of 2600
3
+ [pend seed123] bank N=2600 dim=3 test->bank NN: mean 0.1570 med 0.1691 min 0.07331 max 0.2201 | bank LOO NN: mean 0.1424 med 0.1416 min 0.00943 ratio test/LOO(mean) 1.10
4
+ frac test with NN < 1e-6 (exact dup): 0.000; frac below bank-LOO 10th pct: 0.025
5
+ [pend seed321] bank N=2600 dim=3 test->bank NN: mean 0.1398 med 0.1312 min 0.04187 max 0.2631 | bank LOO NN: mean 0.1424 med 0.1416 min 0.00943 ratio test/LOO(mean) 0.98
6
+ frac test with NN < 1e-6 (exact dup): 0.000; frac below bank-LOO 10th pct: 0.050
7
+ [pend seed777] bank N=2600 dim=3 test->bank NN: mean 0.1333 med 0.1376 min 0.04364 max 0.2281 | bank LOO NN: mean 0.1424 med 0.1416 min 0.00943 ratio test/LOO(mean) 0.94
8
+ frac test with NN < 1e-6 (exact dup): 0.000; frac below bank-LOO 10th pct: 0.150
9
+ [pend] NN dist vs bank size: N=130:0.3998, N=260:0.3148, N=650:0.2300, N=1300:0.1780, N=2600:0.1434 | slope -0.345 -> d_eff ~ 2.9 (ambient dim 3)
10
+ [pend iid3000 (null of 'dense bank')] bank N=2600 dim=3 test->bank NN: mean 0.1426 med 0.1408 min 0.01501 max 0.3298 | bank LOO NN: mean 0.1424 med 0.1416 min 0.00943 ratio test/LOO(mean) 1.00
11
+ frac test with NN < 1e-6 (exact dup): 0.000; frac below bank-LOO 10th pct: 0.098
12
+ pend: NN-dist percentiles over iid test (5,50,95,99): [0.058 0.141 0.236 0.272]
13
+ == PENDULUM objective-cond bank (l,m,b,rho) std; rho_range (0.7, 1.0) rho min/max 0.7000994682312012 0.9999604821205139
14
+ cbar-cond seed321 rho0.7: test->bank NN mean 0.371
15
+ cbar-cond seed321 rho0.8: test->bank NN mean 0.307
16
+ cbar-cond seed321 rho0.9: test->bank NN mean 0.288
17
+ cbar-cond seed777 rho0.7: test->bank NN mean 0.372
18
+ cbar-cond seed777 rho0.8: test->bank NN mean 0.305
19
+ cbar-cond seed777 rho0.9: test->bank NN mean 0.304
20
+ cbar-cond bank LOO NN mean 0.31
21
+ == SISO bank rows 180160 unique plants 5635; G_scale 321.9
22
+ [siso seed0] bank N=5635 dim=10 test->bank NN: mean 0.0042 med 0.0026 min 0.00040 max 0.0267 | bank LOO NN: mean 0.0039 med 0.0023 min 0.00015 ratio test/LOO(mean) 1.08
23
+ frac test with NN < 1e-6 (exact dup): 0.000; frac below bank-LOO 10th pct: 0.067
24
+ [siso seed100] bank N=5635 dim=10 test->bank NN: mean 0.0030 med 0.0023 min 0.00027 max 0.0092 | bank LOO NN: mean 0.0039 med 0.0023 min 0.00015 ratio test/LOO(mean) 0.77
25
+ frac test with NN < 1e-6 (exact dup): 0.000; frac below bank-LOO 10th pct: 0.100
26
+ [siso seed200] bank N=5635 dim=10 test->bank NN: mean 0.0051 med 0.0024 min 0.00021 max 0.0566 | bank LOO NN: mean 0.0039 med 0.0023 min 0.00015 ratio test/LOO(mean) 1.33
27
+ frac test with NN < 1e-6 (exact dup): 0.000; frac below bank-LOO 10th pct: 0.117
28
+ [siso seed300] bank N=5635 dim=10 test->bank NN: mean 0.0077 med 0.0020 min 0.00044 max 0.1921 | bank LOO NN: mean 0.0039 med 0.0023 min 0.00015 ratio test/LOO(mean) 2.00
29
+ frac test with NN < 1e-6 (exact dup): 0.000; frac below bank-LOO 10th pct: 0.083
30
+ [siso seed400] bank N=5635 dim=10 test->bank NN: mean 0.0089 med 0.0028 min 0.00037 max 0.2771 | bank LOO NN: mean 0.0039 med 0.0023 min 0.00015 ratio test/LOO(mean) 2.29
31
+ frac test with NN < 1e-6 (exact dup): 0.000; frac below bank-LOO 10th pct: 0.050
32
+ [siso seed500] bank N=5635 dim=10 test->bank NN: mean 0.0037 med 0.0019 min 0.00020 max 0.0406 | bank LOO NN: mean 0.0039 med 0.0023 min 0.00015 ratio test/LOO(mean) 0.96
33
+ frac test with NN < 1e-6 (exact dup): 0.000; frac below bank-LOO 10th pct: 0.217
34
+ [siso seed4321] bank N=5635 dim=10 test->bank NN: mean 0.0028 med 0.0017 min 0.00023 max 0.0151 | bank LOO NN: mean 0.0039 med 0.0023 min 0.00015 ratio test/LOO(mean) 0.72
35
+ frac test with NN < 1e-6 (exact dup): 0.000; frac below bank-LOO 10th pct: 0.100
36
+ [siso] NN dist vs bank size: N=281:0.0124, N=563:0.0105, N=1408:0.0074, N=2817:0.0062, N=5635:0.0051 | slope -0.308 -> d_eff ~ 3.2 (ambient dim 10)
37
+ per-dim std of bank (normalized): [0. 0. 0.0015 0.0016 0.0004 0. 0.0022 0.099 0.1279 0.047 ]
38
+ per-dim std of held-out (normalized): [0. 0. 0.0016 0.0016 0.0005 0. 0.0022 0.1119 0.1402 0.0447]
39
+ frac held-out coords outside bank [min,max] per-dim: [0. 1. 0.002 0.002 0.002 1. 0. 0.005 0.002 0. ]
40
+ == MIMO bank 5000 dim 20, unique 5000
41
+ [mimo seed9999] bank N=5000 dim=20 test->bank NN: mean 0.1411 med 0.1388 min 0.08064 max 0.2122 | bank LOO NN: mean 0.1382 med 0.1359 min 0.07568 ratio test/LOO(mean) 1.02
42
+ frac test with NN < 1e-6 (exact dup): 0.000; frac below bank-LOO 10th pct: 0.075
43
+ [mimo seed4242] bank N=5000 dim=20 test->bank NN: mean 0.1342 med 0.1316 min 0.00000 max 0.2308 | bank LOO NN: mean 0.1382 med 0.1359 min 0.07568 ratio test/LOO(mean) 0.97
44
+ frac test with NN < 1e-6 (exact dup): 0.008; frac below bank-LOO 10th pct: 0.133
45
+ [mimo seed7777] bank N=5000 dim=20 test->bank NN: mean 0.1402 med 0.1391 min 0.08869 max 0.1996 | bank LOO NN: mean 0.1382 med 0.1359 min 0.07568 ratio test/LOO(mean) 1.01
46
+ frac test with NN < 1e-6 (exact dup): 0.000; frac below bank-LOO 10th pct: 0.083
47
+ [mimo] NN dist vs bank size: N=250:0.1764, N=500:0.1669, N=1250:0.1546, N=2500:0.1458, N=5000:0.1385 | slope -0.082 -> d_eff ~ 12.3 (ambient dim 20)
48
+ [furuta seed4242] bank N=2801 dim=4 test->bank NN: mean 0.3167 med 0.3222 min 0.07650 max 0.7878 | bank LOO NN: mean 0.3082 med 0.3104 min 0.02732 ratio test/LOO(mean) 1.03
49
+ frac test with NN < 1e-6 (exact dup): 0.000; frac below bank-LOO 10th pct: 0.150
50
+ [furuta seed9001] bank N=2801 dim=4 test->bank NN: mean 0.3149 med 0.3076 min 0.14310 max 0.5247 | bank LOO NN: mean 0.3082 med 0.3104 min 0.02732 ratio test/LOO(mean) 1.02
51
+ frac test with NN < 1e-6 (exact dup): 0.000; frac below bank-LOO 10th pct: 0.083
52
+ [furuta] NN dist vs bank size: N=140:0.6783, N=280:0.5651, N=700:0.4410, N=1400:0.3802, N=2801:0.3158 | slope -0.253 -> d_eff ~ 3.9 (ambient dim 4)
results/audit/pendulum_ema.txt ADDED
@@ -0,0 +1,18 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ deployed raw K= 1 in-sample regret med 1.56% fresh-IC J vs retrieval: median -0.21% (p=1.0e-01)
2
+ deployed raw K= 8 in-sample regret med 1.12% fresh-IC J vs retrieval: median +1.94% (p=1.5e-14)
3
+ deployed raw K=32 in-sample regret med 0.89% fresh-IC J vs retrieval: median +1.88% (p=9.8e-15)
4
+ deployed EMA(avg) K= 1 in-sample regret med 1.20% fresh-IC J vs retrieval: median -0.87% (p=1.2e-03)
5
+ deployed EMA(avg) K= 8 in-sample regret med 0.33% fresh-IC J vs retrieval: median +0.80% (p=1.9e-09)
6
+ deployed EMA(avg) K=32 in-sample regret med 0.22% fresh-IC J vs retrieval: median +1.56% (p=3.0e-12)
7
+ size_ref_s42 raw K= 1 in-sample regret med 1.46% fresh-IC J vs retrieval: median -0.47% (p=2.3e-02)
8
+ size_ref_s42 raw K= 8 in-sample regret med 1.10% fresh-IC J vs retrieval: median +1.50% (p=4.4e-14)
9
+ size_ref_s42 raw K=32 in-sample regret med 0.89% fresh-IC J vs retrieval: median +1.60% (p=3.1e-14)
10
+ size_ref_s42 EMA(avg) K= 1 in-sample regret med 1.06% fresh-IC J vs retrieval: median -0.24% (p=9.3e-01)
11
+ size_ref_s42 EMA(avg) K= 8 in-sample regret med 0.26% fresh-IC J vs retrieval: median +1.74% (p=3.9e-14)
12
+ size_ref_s42 EMA(avg) K=32 in-sample regret med 0.14% fresh-IC J vs retrieval: median +2.47% (p=8.2e-15)
13
+ e200_s42 raw K= 1 in-sample regret med 1.33% fresh-IC J vs retrieval: median +0.11% (p=8.6e-01)
14
+ e200_s42 raw K= 8 in-sample regret med 0.72% fresh-IC J vs retrieval: median +1.52% (p=4.0e-12)
15
+ e200_s42 raw K=32 in-sample regret med 0.44% fresh-IC J vs retrieval: median +1.76% (p=1.3e-13)
16
+ e200_s42 EMA(avg) K= 1 in-sample regret med 8.69% fresh-IC J vs retrieval: median +6.55% (p=1.0e-10)
17
+ e200_s42 EMA(avg) K= 8 in-sample regret med 0.42% fresh-IC J vs retrieval: median +2.53% (p=5.7e-11)
18
+ e200_s42 EMA(avg) K=32 in-sample regret med 0.09% fresh-IC J vs retrieval: median +2.16% (p=3.5e-12)
results/audit/pendulum_eval_ckpts_relabelM4_e150.txt ADDED
@@ -0,0 +1,16 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ retrieval(full bank) K= 1 cov 1.000 regret_sel med 1.64 Jfresh(selpick) med 1.1526 Jcv med 1.0808
2
+ retrieval(full bank) K= 2 cov 1.000 regret_sel med 1.12 Jfresh(selpick) med 1.1526 Jcv med 1.0808
3
+ retrieval(full bank) K= 4 cov 1.000 regret_sel med 0.56 Jfresh(selpick) med 1.1465 Jcv med 1.0758
4
+ retrieval(full bank) K= 8 cov 1.000 regret_sel med 0.32 Jfresh(selpick) med 1.1374 Jcv med 1.0681
5
+ retrieval(full bank) K=16 cov 1.000 regret_sel med 0.14 Jfresh(selpick) med 1.1374 Jcv med 1.0627
6
+ retrieval(full bank) K=32 cov 1.000 regret_sel med 0.10 Jfresh(selpick) med 1.1330 Jcv med 1.0620
7
+ relabelM4_e150 K= 1 cov 1.000 regret_sel med 0.23 Jfresh(selpick) med 1.1375 Jcv med 1.0680
8
+ relabelM4_e150 K= 2 cov 1.000 regret_sel med 0.13 Jfresh(selpick) med 1.1383 Jcv med 1.0671
9
+ relabelM4_e150 K= 4 cov 1.000 regret_sel med 0.09 Jfresh(selpick) med 1.1355 Jcv med 1.0640
10
+ relabelM4_e150 K= 8 cov 1.000 regret_sel med 0.07 Jfresh(selpick) med 1.1369 Jcv med 1.0640
11
+ relabelM4_e150 K=16 cov 1.000 regret_sel med 0.04 Jfresh(selpick) med 1.1369 Jcv med 1.0634
12
+ relabelM4_e150 K=32 cov 1.000 regret_sel med 0.04 Jfresh(selpick) med 1.1386 Jcv med 1.0634
13
+ (22s)
14
+ K=1 relabelM4_e150 - retrieval: in-sample regret diff med -1.44pp p=1.7e-14 | fresh-IC J rel diff med -2.81% p=1.1e-14
15
+ K=8 relabelM4_e150 - retrieval: in-sample regret diff med -0.14pp p=5.3e-14 | fresh-IC J rel diff med -0.17% p=6.5e-04
16
+ K=32 relabelM4_e150 - retrieval: in-sample regret diff med -0.04pp p=3.2e-12 | fresh-IC J rel diff med +0.06% p=2.3e-02
results/audit/pendulum_ksweep.txt ADDED
@@ -0,0 +1,76 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ retrieval(full bank) K= 1 cov 1.000 regret_sel med 1.64 Jfresh(selpick) med 1.1526 Jcv med 1.0808
2
+ retrieval(full bank) K= 2 cov 1.000 regret_sel med 1.12 Jfresh(selpick) med 1.1526 Jcv med 1.0808
3
+ retrieval(full bank) K= 4 cov 1.000 regret_sel med 0.56 Jfresh(selpick) med 1.1465 Jcv med 1.0758
4
+ retrieval(full bank) K= 8 cov 1.000 regret_sel med 0.32 Jfresh(selpick) med 1.1374 Jcv med 1.0681
5
+ retrieval(full bank) K=16 cov 1.000 regret_sel med 0.14 Jfresh(selpick) med 1.1374 Jcv med 1.0627
6
+ retrieval(full bank) K=32 cov 1.000 regret_sel med 0.10 Jfresh(selpick) med 1.1330 Jcv med 1.0620
7
+ retrieval(full bank) K=64 cov 1.000 regret_sel med 0.05 Jfresh(selpick) med 1.1341 Jcv med 1.0621
8
+ retrieval(25% bank) K= 1 cov 1.000 regret_sel med 1.36 Jfresh(selpick) med 1.1662 Jcv med 1.0925
9
+ retrieval(25% bank) K= 2 cov 1.000 regret_sel med 0.67 Jfresh(selpick) med 1.1512 Jcv med 1.0797
10
+ retrieval(25% bank) K= 4 cov 1.000 regret_sel med 0.41 Jfresh(selpick) med 1.1462 Jcv med 1.0756
11
+ retrieval(25% bank) K= 8 cov 1.000 regret_sel med 0.22 Jfresh(selpick) med 1.1347 Jcv med 1.0660
12
+ retrieval(25% bank) K=16 cov 1.000 regret_sel med 0.12 Jfresh(selpick) med 1.1347 Jcv med 1.0658
13
+ retrieval(25% bank) K=32 cov 1.000 regret_sel med 0.05 Jfresh(selpick) med 1.1347 Jcv med 1.0633
14
+ retrieval(25% bank) K=64 cov 1.000 regret_sel med 0.05 Jfresh(selpick) med 1.1359 Jcv med 1.0628
15
+ deployed K= 1 cov 1.000 regret_sel med 1.56 Jfresh(selpick) med 1.1614 Jcv med 1.0878
16
+ deployed K= 2 cov 1.000 regret_sel med 1.30 Jfresh(selpick) med 1.1609 Jcv med 1.0873
17
+ deployed K= 4 cov 1.000 regret_sel med 1.18 Jfresh(selpick) med 1.1609 Jcv med 1.0874
18
+ deployed K= 8 cov 1.000 regret_sel med 1.04 Jfresh(selpick) med 1.1592 Jcv med 1.0852
19
+ deployed K=16 cov 1.000 regret_sel med 0.98 Jfresh(selpick) med 1.1570 Jcv med 1.0838
20
+ deployed K=32 cov 1.000 regret_sel med 0.91 Jfresh(selpick) med 1.1533 Jcv med 1.0807
21
+ deployed K=64 cov 1.000 regret_sel med 0.86 Jfresh(selpick) med 1.1533 Jcv med 1.0807
22
+ (49s)
23
+ size_ref_s42 K= 1 cov 1.000 regret_sel med 1.51 Jfresh(selpick) med 1.1546 Jcv med 1.0830
24
+ size_ref_s42 K= 2 cov 1.000 regret_sel med 1.23 Jfresh(selpick) med 1.1544 Jcv med 1.0827
25
+ size_ref_s42 K= 4 cov 1.000 regret_sel med 1.15 Jfresh(selpick) med 1.1544 Jcv med 1.0827
26
+ size_ref_s42 K= 8 cov 1.000 regret_sel med 1.04 Jfresh(selpick) med 1.1526 Jcv med 1.0806
27
+ size_ref_s42 K=16 cov 1.000 regret_sel med 0.94 Jfresh(selpick) med 1.1510 Jcv med 1.0796
28
+ size_ref_s42 K=32 cov 1.000 regret_sel med 0.86 Jfresh(selpick) med 1.1481 Jcv med 1.0767
29
+ size_ref_s42 K=64 cov 1.000 regret_sel med 0.81 Jfresh(selpick) med 1.1481 Jcv med 1.0767
30
+ (170s)
31
+ size_ref_s43 K= 1 cov 1.000 regret_sel med 0.97 Jfresh(selpick) med 1.1571 Jcv med 1.0835
32
+ size_ref_s43 K= 2 cov 1.000 regret_sel med 0.84 Jfresh(selpick) med 1.1550 Jcv med 1.0818
33
+ size_ref_s43 K= 4 cov 1.000 regret_sel med 0.76 Jfresh(selpick) med 1.1550 Jcv med 1.0818
34
+ size_ref_s43 K= 8 cov 1.000 regret_sel med 0.58 Jfresh(selpick) med 1.1531 Jcv med 1.0804
35
+ size_ref_s43 K=16 cov 1.000 regret_sel med 0.49 Jfresh(selpick) med 1.1521 Jcv med 1.0795
36
+ size_ref_s43 K=32 cov 1.000 regret_sel med 0.42 Jfresh(selpick) med 1.1493 Jcv med 1.0756
37
+ size_ref_s43 K=64 cov 1.000 regret_sel med 0.37 Jfresh(selpick) med 1.1549 Jcv med 1.0756
38
+ (217s)
39
+ e200_s42 K= 1 cov 1.000 regret_sel med 1.45 Jfresh(selpick) med 1.1624 Jcv med 1.0882
40
+ e200_s42 K= 2 cov 1.000 regret_sel med 1.13 Jfresh(selpick) med 1.1591 Jcv med 1.0850
41
+ e200_s42 K= 4 cov 1.000 regret_sel med 0.95 Jfresh(selpick) med 1.1611 Jcv med 1.0850
42
+ e200_s42 K= 8 cov 1.000 regret_sel med 0.69 Jfresh(selpick) med 1.1650 Jcv med 1.0825
43
+ e200_s42 K=16 cov 1.000 regret_sel med 0.57 Jfresh(selpick) med 1.1728 Jcv med 1.0819
44
+ e200_s42 K=32 cov 1.000 regret_sel med 0.49 Jfresh(selpick) med 1.1571 Jcv med 1.0751
45
+ e200_s42 K=64 cov 1.000 regret_sel med 0.36 Jfresh(selpick) med 1.1555 Jcv med 1.0751
46
+ (258s)
47
+ e200_s43 K= 1 cov 1.000 regret_sel med 1.01 Jfresh(selpick) med 1.1675 Jcv med 1.0891
48
+ e200_s43 K= 2 cov 1.000 regret_sel med 0.56 Jfresh(selpick) med 1.1536 Jcv med 1.0775
49
+ e200_s43 K= 4 cov 1.000 regret_sel med 0.35 Jfresh(selpick) med 1.1536 Jcv med 1.0753
50
+ e200_s43 K= 8 cov 1.000 regret_sel med 0.21 Jfresh(selpick) med 1.1758 Jcv med 1.0756
51
+ e200_s43 K=16 cov 1.000 regret_sel med 0.12 Jfresh(selpick) med 1.1758 Jcv med 1.0749
52
+ e200_s43 K=32 cov 1.000 regret_sel med 0.07 Jfresh(selpick) med 1.1736 Jcv med 1.0677
53
+ e200_s43 K=64 cov 1.000 regret_sel med 0.04 Jfresh(selpick) med 1.1856 Jcv med 1.0710
54
+ (299s)
55
+ p25_s42 K= 1 cov 1.000 regret_sel med 1.19 Jfresh(selpick) med 1.1526 Jcv med 1.0810
56
+ p25_s42 K= 2 cov 1.000 regret_sel med 0.96 Jfresh(selpick) med 1.1500 Jcv med 1.0775
57
+ p25_s42 K= 4 cov 1.000 regret_sel med 0.75 Jfresh(selpick) med 1.1485 Jcv med 1.0763
58
+ p25_s42 K= 8 cov 1.000 regret_sel med 0.47 Jfresh(selpick) med 1.1520 Jcv med 1.0751
59
+ p25_s42 K=16 cov 1.000 regret_sel med 0.36 Jfresh(selpick) med 1.1495 Jcv med 1.0749
60
+ p25_s42 K=32 cov 1.000 regret_sel med 0.22 Jfresh(selpick) med 1.1451 Jcv med 1.0699
61
+ p25_s42 K=64 cov 1.000 regret_sel med 0.18 Jfresh(selpick) med 1.1482 Jcv med 1.0696
62
+ (346s)
63
+ p25_s43 K= 1 cov 1.000 regret_sel med 0.77 Jfresh(selpick) med 1.1716 Jcv med 1.0915
64
+ p25_s43 K= 2 cov 1.000 regret_sel med 0.36 Jfresh(selpick) med 1.1616 Jcv med 1.0826
65
+ p25_s43 K= 4 cov 1.000 regret_sel med 0.24 Jfresh(selpick) med 1.1616 Jcv med 1.0826
66
+ p25_s43 K= 8 cov 1.000 regret_sel med 0.13 Jfresh(selpick) med 1.1684 Jcv med 1.0822
67
+ p25_s43 K=16 cov 1.000 regret_sel med 0.10 Jfresh(selpick) med 1.1615 Jcv med 1.0846
68
+ p25_s43 K=32 cov 1.000 regret_sel med 0.07 Jfresh(selpick) med 1.1640 Jcv med 1.0742
69
+ p25_s43 K=64 cov 1.000 regret_sel med 0.05 Jfresh(selpick) med 1.1640 Jcv med 1.0742
70
+ (498s)
71
+ K=8 deployed - retrieval: in-sample regret diff med +0.59pp p=1.1e-11 | fresh-IC J rel diff med +1.68% p=4.1e-14 | CV-selected rel diff med +1.46% p=8.8e-14
72
+ K=8 e200_s42 - retrieval: in-sample regret diff med +0.26pp p=3.2e-06 | fresh-IC J rel diff med +1.64% p=1.2e-11 | CV-selected rel diff med +1.05% p=2.1e-10
73
+ K=8 p25_s42 - retrieval: in-sample regret diff med +0.17pp p=3.9e-04 | fresh-IC J rel diff med +1.43% p=2.1e-10 | CV-selected rel diff med +0.82% p=9.4e-10
74
+ K=32 deployed - retrieval: in-sample regret diff med +0.52pp p=3.4e-11 | fresh-IC J rel diff med +2.00% p=1.7e-14 | CV-selected rel diff med +1.92% p=2.6e-14
75
+ K=32 e200_s42 - retrieval: in-sample regret diff med +0.27pp p=4.6e-08 | fresh-IC J rel diff med +2.14% p=8.0e-13 | CV-selected rel diff med +1.28% p=3.7e-13
76
+ K=32 p25_s42 - retrieval: in-sample regret diff med +0.05pp p=1.6e-03 | fresh-IC J rel diff med +1.53% p=6.7e-13 | CV-selected rel diff med +0.81% p=1.8e-12
results/audit/pendulum_labelquality.txt ADDED
@@ -0,0 +1,4 @@
 
 
 
 
 
1
+ own label stable: 1.0
2
+ own-label excess over best of 33 (own + 32 neighbours): median 1.14% mean 1.66% p90 4.12%
3
+ own-label excess over best of own+8 neighbours: median 1.00%
4
+ rank of own label among 33 (0=best): median 15.0 ; fraction own is the best: 0.003
results/audit/pendulum_ood.txt ADDED
@@ -0,0 +1,24 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ == near-OOD (edge x1.25): N=24 NN-dist test->bank mean 1.429 (in-box iid null 0.143)
2
+ oracle finite: 24 / 24
3
+ K=1 model s0 : stable 1.000 cert 0.833 median regret -3.27% p90 -0.54%
4
+ K=1 model s1 : stable 1.000 cert 0.833 median regret -4.24% p90 -2.01%
5
+ K=1 model s2 : stable 1.000 cert 0.833 median regret -4.03% p90 -1.41%
6
+ K=1 retrieval: stable 1.000 cert 1.000 median regret -4.95% p90 -1.71%
7
+ K=1 paired Wilcoxon model(s0) vs retrieval on common: n=24 p=0.0491; median diff +0.58pp
8
+ K=8 model s0 : stable 1.000 cert 1.000 median regret -5.09% p90 -2.94%
9
+ K=8 model s1 : stable 1.000 cert 0.958 median regret -5.35% p90 -2.60%
10
+ K=8 model s2 : stable 1.000 cert 1.000 median regret -5.42% p90 -3.14%
11
+ K=8 retrieval: stable 1.000 cert 1.000 median regret -5.85% p90 -4.72%
12
+ K=8 paired Wilcoxon model(s0) vs retrieval on common: n=24 p=0.0059; median diff +0.86pp
13
+ == far-OOD (edge x1.67): N=24 NN-dist test->bank mean 3.940 (in-box iid null 0.143)
14
+ oracle finite: 24 / 24
15
+ K=1 model s0 : stable 0.958 cert 0.000 median regret 0.38% p90 44.23%
16
+ K=1 model s1 : stable 0.917 cert 0.000 median regret -0.67% p90 15.96%
17
+ K=1 model s2 : stable 0.833 cert 0.000 median regret 1.51% p90 27.89%
18
+ K=1 retrieval: stable 0.750 cert 0.625 median regret 18.48% p90 111.50%
19
+ K=1 paired Wilcoxon model(s0) vs retrieval on common: n=18 p=7.63e-06; median diff -21.03pp
20
+ K=8 model s0 : stable 1.000 cert 0.042 median regret -6.71% p90 25.09%
21
+ K=8 model s1 : stable 1.000 cert 0.000 median regret -6.14% p90 12.72%
22
+ K=8 model s2 : stable 1.000 cert 0.042 median regret -6.90% p90 6.89%
23
+ K=8 retrieval: stable 0.750 cert 0.625 median regret 18.48% p90 111.50%
24
+ K=8 paired Wilcoxon model(s0) vs retrieval on common: n=18 p=7.63e-06; median diff -26.63pp
results/audit/pendulum_relabel_retrieval.txt ADDED
@@ -0,0 +1,27 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ retrieval(orig bank) K= 1 cov 1.000 regret_sel med 1.64 Jfresh(selpick) med 1.1526 Jcv med 1.0808
2
+ retrieval(orig bank) K= 2 cov 1.000 regret_sel med 1.12 Jfresh(selpick) med 1.1526 Jcv med 1.0808
3
+ retrieval(orig bank) K= 4 cov 1.000 regret_sel med 0.56 Jfresh(selpick) med 1.1465 Jcv med 1.0758
4
+ retrieval(orig bank) K= 8 cov 1.000 regret_sel med 0.32 Jfresh(selpick) med 1.1374 Jcv med 1.0681
5
+ retrieval(orig bank) K=16 cov 1.000 regret_sel med 0.14 Jfresh(selpick) med 1.1374 Jcv med 1.0627
6
+ retrieval(orig bank) K=32 cov 1.000 regret_sel med 0.10 Jfresh(selpick) med 1.1330 Jcv med 1.0620
7
+ retrieval(relabelled bank) K= 1 cov 1.000 regret_sel med 0.25 Jfresh(selpick) med 1.1302 Jcv med 1.0627
8
+ retrieval(relabelled bank) K= 2 cov 1.000 regret_sel med 0.20 Jfresh(selpick) med 1.1306 Jcv med 1.0627
9
+ retrieval(relabelled bank) K= 4 cov 1.000 regret_sel med 0.17 Jfresh(selpick) med 1.1317 Jcv med 1.0623
10
+ retrieval(relabelled bank) K= 8 cov 1.000 regret_sel med 0.10 Jfresh(selpick) med 1.1356 Jcv med 1.0623
11
+ retrieval(relabelled bank) K=16 cov 1.000 regret_sel med 0.10 Jfresh(selpick) med 1.1381 Jcv med 1.0621
12
+ retrieval(relabelled bank) K=32 cov 1.000 regret_sel med 0.05 Jfresh(selpick) med 1.1365 Jcv med 1.0615
13
+ K=1 relabelled-bank retrieval - original-bank retrieval: fresh-IC J rel diff median -2.38% p=8.5e-15
14
+ K=8 relabelled-bank retrieval - original-bank retrieval: fresh-IC J rel diff median -0.23% p=4.1e-06
15
+ K=32 relabelled-bank retrieval - original-bank retrieval: fresh-IC J rel diff median +0.00% p=2.0e-01
16
+ model trained on relabelled targets K= 1 cov 1.000 regret_sel med 0.23 Jfresh(selpick) med 1.1375 Jcv med 1.0680
17
+ model trained on relabelled targets K= 2 cov 1.000 regret_sel med 0.13 Jfresh(selpick) med 1.1383 Jcv med 1.0671
18
+ model trained on relabelled targets K= 4 cov 1.000 regret_sel med 0.09 Jfresh(selpick) med 1.1355 Jcv med 1.0640
19
+ model trained on relabelled targets K= 8 cov 1.000 regret_sel med 0.07 Jfresh(selpick) med 1.1369 Jcv med 1.0640
20
+ model trained on relabelled targets K=16 cov 1.000 regret_sel med 0.04 Jfresh(selpick) med 1.1369 Jcv med 1.0634
21
+ model trained on relabelled targets K=32 cov 1.000 regret_sel med 0.04 Jfresh(selpick) med 1.1386 Jcv med 1.0634
22
+ K=1 relabel-trained model - original-bank retrieval: fresh-IC J rel diff median -2.81% p=1.1e-14; in-sample regret diff median -1.44pp
23
+ K=1 relabel-trained model - relabelled-bank retrieval: fresh-IC J rel diff median -0.20% p=2.7e-02; in-sample regret diff median -0.08pp
24
+ K=8 relabel-trained model - original-bank retrieval: fresh-IC J rel diff median -0.17% p=6.5e-04; in-sample regret diff median -0.14pp
25
+ K=8 relabel-trained model - relabelled-bank retrieval: fresh-IC J rel diff median +0.08% p=2.3e-02; in-sample regret diff median -0.03pp
26
+ K=32 relabel-trained model - original-bank retrieval: fresh-IC J rel diff median +0.06% p=2.3e-02; in-sample regret diff median -0.04pp
27
+ K=32 relabel-trained model - relabelled-bank retrieval: fresh-IC J rel diff median +0.06% p=2.1e-04; in-sample regret diff median -0.02pp
results/audit/rederive.txt ADDED
@@ -0,0 +1,28 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ [1] pendulum K=8 seed123: model median regret per seed [1.172 1.085 1.008] mean 1.089 | retrieval 0.396 | Wilcoxon(seed0) n,p (40, np.float64(7.880189514253289e-07))
2
+ cov model [1. 1. 1.] retr 1.0
3
+ [2] SISO prereg K=8: model 0.7733 distinct 0.75 McNemar(m-only,r-only,p) (9, 2, np.float64(0.0654296875))
4
+ seed 1 (12, 3, np.float64(0.03515625)) 0.78
5
+ seed 2 (13, 2, np.float64(0.00738525390625)) 0.787
6
+ K=1: 0.617 0.493 (60, 23, np.float64(5.973595638605142e-05))
7
+ [3] MIMO K=8: model cov [1. 0.992 1. ] retr 0.992 McNemar (1, 0, np.float64(1.0))
8
+ median regret model per seed [2.32 2.35 2.28] retr 2.62 Wilcoxon (119, np.float32(0.90291226))
9
+ MEAN regret model [3.94 3.44 3.48] retr 4.31 p90 model [9.6 8.29 8.78] retr 11.04
10
+ frac of plants with negative regret (model beats 'oracle') model s0: 0.033 retr: 0.033 | min regret model -29.58
11
+ [4] Furuta seed4242 K=1: nearest+P 0.517 generated+P 0.333 McNemar(near-only? ) (17, 6, np.float64(0.03468966484069824))
12
+ [4] Furuta seed4242 K=4: nearest+P 0.783 generated+P 0.600 McNemar(near-only? ) (14, 3, np.float64(0.012725830078125))
13
+ [4] Furuta seed4242 K=8: nearest+P 0.883 generated+P 0.650 McNemar(near-only? ) (14, 0, np.float64(0.0001220703125))
14
+ [4] Furuta seed4242 K=16: nearest+P 0.933 generated+P 0.667 McNemar(near-only? ) (16, 0, np.float64(3.0517578125e-05))
15
+ [4] Furuta seed9001 K=1: nearest+P 0.567 generated+P 0.300 McNemar(near-only? ) (21, 5, np.float64(0.002493917942047119))
16
+ [4] Furuta seed9001 K=4: nearest+P 0.867 generated+P 0.550 McNemar(near-only? ) (21, 2, np.float64(6.604194641113281e-05))
17
+ [4] Furuta seed9001 K=8: nearest+P 0.917 generated+P 0.650 McNemar(near-only? ) (16, 0, np.float64(3.0517578125e-05))
18
+ [4] Furuta seed9001 K=16: nearest+P 0.933 generated+P 0.683 McNemar(near-only? ) (15, 0, np.float64(6.103515625e-05))
19
+ [5] Furuta deploy nominal K=8 retr 0.9 K=16 0.933 NMPC 0.958 gen seeds [np.float64(0.65), np.float64(0.658), np.float64(0.65)]
20
+ retr vs NMPC nominal McNemar (0, 7, np.float64(0.015625))
21
+ fric0.6: retr 0.867 nmpc 0.958 trueID 0.992
22
+ fric1.6: retr 0.092 nmpc 0.642 trueID 0.583
23
+ fric2.5: retr 0.000 nmpc 0.292 trueID 0.250
24
+ [6] guidance A mu=0.6: a0=0.5237, a=0.5236, b0=0.5236, b=0.5236, c=0.5229, d=0.5236, e=0.5378, f=0.5237
25
+ [6] guidance A mu=0.9: a0=0.8749, a=0.8749, b0=0.8749, b=0.8749, c=0.8738, d=0.8749, e=0.8809, f=0.8749
26
+ [7] Witsenhausen S20 nearest: median regret -0.017% within1 90.0% (n=60; 54) gate gap 10.0
27
+ nearest strictly below ceiling: 58.3 pct within 1.5pct of ceiling: 91.7 pct
28
+ within-1pct relative to min(ceiling,nearest): 90.0
results/audit/siso_prereg_pooled_seeds.txt ADDED
@@ -0,0 +1,5 @@
 
 
 
 
 
 
1
+ K= 1: model 3-seed mean 0.622 distinct 0.493 diff +0.129 CI[+0.078,+0.181] sign-flip permutation p=0.0000
2
+ K= 2: model 3-seed mean 0.706 distinct 0.660 diff +0.046 CI[+0.011,+0.080] sign-flip permutation p=0.0123
3
+ K= 4: model 3-seed mean 0.756 distinct 0.743 diff +0.012 CI[-0.008,+0.032] sign-flip permutation p=0.2675
4
+ K= 8: model 3-seed mean 0.780 distinct 0.750 diff +0.030 CI[+0.009,+0.051] sign-flip permutation p=0.0064
5
+ K=16: model 3-seed mean 0.793 distinct 0.753 diff +0.040 CI[+0.018,+0.064] sign-flip permutation p=0.0009
results/audit/siso_retrieval_metric.txt ADDED
@@ -0,0 +1,10 @@
 
 
 
 
 
 
 
 
 
 
 
1
+ SISO N=300 coverage vs K (any of K candidates verified stable)
2
+ K model(DDIM20) retr global retr zscore retr den3 McNemar model vs zscore (m-only,r-only,p)
3
+ 1 0.630 0.493 0.580 0.490 (41,26,p=0.0864)
4
+ 2 0.717 0.660 0.667 0.660 (22,7,p=0.00813)
5
+ 4 0.757 0.743 0.733 0.737 (10,3,p=0.0923)
6
+ 8 0.767 0.750 0.747 0.747 (9,3,p=0.146)
7
+ 16 0.780 0.753 0.760 0.750 (8,2,p=0.109)
8
+ 32 0.797 0.757 0.767 0.753 (10,1,p=0.0117)
9
+ 64 0.807 0.763 0.773 0.770 (11,1,p=0.00635)
10
+ 128 - 0.790 0.780 0.780
results/audit/target_range_clip.txt ADDED
@@ -0,0 +1,9 @@
 
 
 
 
 
 
 
 
 
 
1
+ pendulum_dataset.spline: shape (2600, 11) frac|x|>1 0.0000 max|x| 1.00 frac rows with any >1: 0.000
2
+ pendulum_cbar_cond.spline: shape (2533, 11) frac|x|>1 0.0000 max|x| 1.00 frac rows with any >1: 0.000
3
+ popov_dataset_normalized_L.spline: shape (180160, 11) frac|x|>1 0.0000 max|x| 1.00 frac rows with any >1: 0.000
4
+ popov_dataset_normalized_L.F_vec: shape (180160, 10) frac|x|>1 0.0000 max|x| 1.00 frac rows with any >1: 0.000
5
+ mimo_dataset_2x2_opt.spline: shape (5000, 2, 11) frac|x|>1 0.0000 max|x| 1.00 frac rows with any >1: 0.000
6
+ furuta_traj_dataset_v2.spline: shape (2801, 116) frac|x|>1 0.1922 max|x| 37.41 frac rows with any >1: 0.954
7
+ furuta_dataset.spline: shape (1678, 4) frac|x|>1 0.2934 max|x| 2.76 frac rows with any >1: 0.672
8
+ furuta_handoff_dataset.spline: shape (1414, 5) frac|x|>1 0.3151 max|x| 4.81 frac rows with any >1: 0.798
9
+ furuta_full_dataset.spline: shape (1500, 7) frac|x|>1 0.1936 max|x| 3.85 frac rows with any >1: 0.663
results/audit/traj_gen_eval_v2_clip5.0.txt ADDED
@@ -0,0 +1,94 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ bank data/furuta_nmpc_bank3000.npz: 2801 entries
2
+ model models/Best_furuta_traj_cond_v2.pth: cond_dim=4 out_dim=116 hidden=256 layers=5 device=cpu
3
+ [seed4242] margin=0.3: resolving P for 227 unique bank source plants
4
+ [seed4242] sampler seed 0: 8.9s for 60x4 candidates (0 decode-invalid of 240)
5
+ [seed4242] sampler seed 1: 8.9s for 60x4 candidates (0 decode-invalid of 240)
6
+ [seed4242] sampler seed 2: 8.9s for 60x4 candidates (0 decode-invalid of 240)
7
+ [seed4242] 10/60 (27s)
8
+ [seed4242] 20/60 (46s)
9
+ [seed4242] 30/60 (69s)
10
+ [seed4242] 40/60 (87s)
11
+ [seed4242] 50/60 (111s)
12
+ [seed4242] 60/60 (138s)
13
+ [seed9001] margin=0.3: resolving P for 232 unique bank source plants
14
+ [seed9001] sampler seed 0: 8.8s for 60x4 candidates (0 decode-invalid of 240)
15
+ [seed9001] sampler seed 1: 8.8s for 60x4 candidates (0 decode-invalid of 240)
16
+ [seed9001] sampler seed 2: 8.9s for 60x4 candidates (0 decode-invalid of 240)
17
+ [seed9001] 10/60 (23s)
18
+ [seed9001] 20/60 (46s)
19
+ [seed9001] 30/60 (68s)
20
+ [seed9001] 40/60 (91s)
21
+ [seed9001] 50/60 (108s)
22
+ [seed9001] 60/60 (135s)
23
+
24
+ === traj_gen_eval (v2_clip5.0): generated vs retrieval vs NMPC ===
25
+
26
+ --- plant set seed4242 (seed 4242, n=60) ---
27
+ arm succ 95% CI medJ med regret p90 synth
28
+ per-plant 93% [ 87, 98] 4.08 0.00% 0.00% 3.3s
29
+ nearest 62% [ 50, 73] 4.15 0.36% 1.51% 272ms
30
+ nearest+P 78% [ 67, 88] 4.09 0.69% 2.52% 551ms
31
+ nearest-margin 62% [ 50, 73] 4.15 0.36% 1.51% 275ms
32
+ NMPC 95% [ 88, 100] 4.08 0.14% 0.24% 21ms
33
+ generated 50.0%(sd6.2) [ 45, 72]* 4.02 0.97%(sd0.2) 2.58%(sd0.5) 301ms
34
+ generated+P 68.9%(sd3.4) [ 62, 83]* 4.08 1.02%(sd0.2) 3.43%(sd0.7) 979ms
35
+ (* generated rows: mean (sd) over sampler seeds [0, 1, 2]; CI shown is for seed 0 only)
36
+ (certified hand-off rate == success rate for every arm except NMPC (a certificate is a precondition for success here); NMPC's own hand-off is n/a, it never switches to a static gain)
37
+
38
+ --- failure breakdown, primary seed 0 ---
39
+ per-plant NMPC never reached Omega:3 replay failed:1
40
+ nearest Lyapunov decrease fails at a vertex:13 rollout failed:9 true dynamics: 0.02% of the boundary has V-dot >= 0, 0.00% saturates:1
41
+ nearest+P rollout failed:13
42
+ nearest-margin Lyapunov decrease fails at a vertex:13 rollout failed:9 true dynamics: 0.02% of the boundary has V-dot >= 0, 0.00% saturates:1
43
+ NMPC :3
44
+ generated Lyapunov decrease fails at a vertex:17 rollout failed:6 peak |Kx| on Omega is 10.26 V > 10.0 V:1 peak |Kx| on Omega is 10.23 V > 10.0 V:1
45
+ generated+P rollout failed:16
46
+
47
+ --- paired McNemar on success (exact binomial on discordant pairs) ---
48
+ per-plant vs nearest beats on 20, loses on 1 p=0.0000 * cost ratio 1.004x p=0.0000
49
+ per-plant vs nearest+P beats on 10, loses on 1 p=0.0117 * cost ratio 1.007x p=0.0000
50
+ per-plant vs nearest-margin beats on 20, loses on 1 p=0.0000 * cost ratio 1.004x p=0.0000
51
+ per-plant vs NMPC beats on 0, loses on 1 p=1.0000 cost ratio 1.001x p=0.0002
52
+ per-plant vs generated beats on 21, loses on 0 p=0.0000 * cost ratio 1.006x p=0.0000
53
+ per-plant vs generated+P beats on 12, loses on 0 p=0.0005 * cost ratio 1.008x p=0.0000
54
+ nearest+P vs per-plant beats on 1, loses on 10 p=0.0117 * cost ratio 0.993x p=0.0000
55
+ nearest+P vs nearest beats on 10, loses on 0 p=0.0020 * cost ratio 1.000x p=0.8291
56
+ nearest+P vs nearest-margin beats on 10, loses on 0 p=0.0020 * cost ratio 1.000x p=0.8291
57
+ nearest+P vs NMPC beats on 0, loses on 10 p=0.0020 * cost ratio 0.995x p=0.0000
58
+ nearest+P vs generated beats on 15, loses on 3 p=0.0075 * cost ratio 1.002x p=0.7891
59
+ nearest+P vs generated+P beats on 9, loses on 6 p=0.6072 cost ratio 1.001x p=0.5183
60
+
61
+ --- plant set seed9001 (seed 9001, n=60) ---
62
+ arm succ 95% CI medJ med regret p90 synth
63
+ per-plant 93% [ 87, 98] 4.16 0.00% 0.00% 5.2s
64
+ nearest 65% [ 52, 77] 4.10 0.71% 2.11% 154ms
65
+ nearest+P 87% [ 78, 95] 4.14 0.59% 2.34% 276ms
66
+ nearest-margin 65% [ 52, 77] 4.10 0.71% 2.11% 148ms
67
+ NMPC 97% [ 92, 100] 4.17 0.07% 0.22% 21ms
68
+ generated 50.6%(sd1.6) [ 40, 63]* 4.14 0.69%(sd0.1) 2.46%(sd0.2) 310ms
69
+ generated+P 75.6%(sd3.9) [ 68, 88]* 4.17 0.92%(sd0.3) 3.06%(sd0.6) 719ms
70
+ (* generated rows: mean (sd) over sampler seeds [0, 1, 2]; CI shown is for seed 0 only)
71
+ (certified hand-off rate == success rate for every arm except NMPC (a certificate is a precondition for success here); NMPC's own hand-off is n/a, it never switches to a static gain)
72
+
73
+ --- failure breakdown, primary seed 0 ---
74
+ per-plant replay failed:2 NMPC never reached Omega:2
75
+ nearest Lyapunov decrease fails at a vertex:16 rollout failed:5
76
+ nearest+P rollout failed:8
77
+ nearest-margin Lyapunov decrease fails at a vertex:16 rollout failed:5
78
+ NMPC :2
79
+ generated Lyapunov decrease fails at a vertex:20 rollout failed:6 peak |Kx| on Omega is 10.02 V > 10.0 V:1 true dynamics: 0.37% of the boundary has V-dot >= 0, 0.00% saturates:1 peak |Kx| on Omega is 10.24 V > 10.0 V:1
80
+ generated+P rollout failed:13
81
+
82
+ --- paired McNemar on success (exact binomial on discordant pairs) ---
83
+ per-plant vs nearest beats on 18, loses on 1 p=0.0001 * cost ratio 1.007x p=0.0000
84
+ per-plant vs nearest+P beats on 5, loses on 1 p=0.2188 cost ratio 1.006x p=0.0000
85
+ per-plant vs nearest-margin beats on 18, loses on 1 p=0.0001 * cost ratio 1.007x p=0.0000
86
+ per-plant vs NMPC beats on 0, loses on 2 p=0.5000 cost ratio 1.001x p=0.0064
87
+ per-plant vs generated beats on 25, loses on 0 p=0.0000 * cost ratio 1.006x p=0.0000
88
+ per-plant vs generated+P beats on 10, loses on 1 p=0.0117 * cost ratio 1.007x p=0.0000
89
+ nearest+P vs per-plant beats on 1, loses on 5 p=0.2188 cost ratio 0.994x p=0.0000
90
+ nearest+P vs nearest beats on 13, loses on 0 p=0.0002 * cost ratio 1.000x p=0.1154
91
+ nearest+P vs nearest-margin beats on 13, loses on 0 p=0.0002 * cost ratio 1.000x p=0.1154
92
+ nearest+P vs NMPC beats on 0, loses on 6 p=0.0312 * cost ratio 0.995x p=0.0000
93
+ nearest+P vs generated beats on 23, loses on 2 p=0.0000 * cost ratio 1.003x p=0.1207
94
+ nearest+P vs generated+P beats on 10, loses on 5 p=0.3018 cost ratio 1.002x p=0.1870
results/audit/traj_gen_ksweep_clip5.0.txt ADDED
@@ -0,0 +1,30 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ bank data/furuta_nmpc_bank3000.npz: 2801 entries
2
+ model models/Best_furuta_traj_cond_v2.pth: cond_dim=4 out_dim=116
3
+ [seed4242] per-plant ceiling: 93%
4
+ [seed4242] nearest+P sequences done
5
+ [seed4242] generated+P seed 0 sequences done (33.9s sampling for 60x16)
6
+ [seed4242] generated+P seed 1 sequences done (33.8s sampling for 60x16)
7
+ [seed4242] generated+P seed 2 sequences done (33.7s sampling for 60x16)
8
+ [seed9001] per-plant ceiling: 93%
9
+ [seed9001] nearest+P sequences done
10
+ [seed9001] generated+P seed 0 sequences done (33.5s sampling for 60x16)
11
+ [seed9001] generated+P seed 1 sequences done (33.8s sampling for 60x16)
12
+ [seed9001] generated+P seed 2 sequences done (34.4s sampling for 60x16)
13
+
14
+ === candidate-budget sweep: generated+P vs nearest+P vs per-plant ceiling ===
15
+
16
+ --- plant set seed4242 (seed 4242, n=60) ---
17
+ K nearest+P generated+P per-plant
18
+ 1 51.7% t= 0.29s 33.9%(sd1.6) t= 0.86s 93.3% [McNemar nearest+P vs generated+P(seed0): 18-8, p=0.076]
19
+ 4 78.3% t= 0.54s 73.9%(sd2.8) t= 1.40s 93.3% [McNemar nearest+P vs generated+P(seed0): 9-4, p=0.267]
20
+ 8 88.3% t= 0.54s 83.3%(sd1.4) t= 1.40s 93.3% [McNemar nearest+P vs generated+P(seed0): 5-3, p=0.727]
21
+ 16 93.3% t= 0.54s 86.7%(sd0.0) t= 1.40s 93.3% [McNemar nearest+P vs generated+P(seed0): 5-1, p=0.219]
22
+ crossover K (generated+P mean >= nearest+P): never within tested range
23
+
24
+ --- plant set seed9001 (seed 9001, n=60) ---
25
+ K nearest+P generated+P per-plant
26
+ 1 56.7% t= 0.29s 34.4%(sd9.8) t= 0.86s 93.3% [McNemar nearest+P vs generated+P(seed0): 20-2, p=0.000]
27
+ 4 86.7% t= 0.45s 76.1%(sd2.8) t= 1.16s 93.3% [McNemar nearest+P vs generated+P(seed0): 11-3, p=0.057]
28
+ 8 91.7% t= 0.45s 88.3%(sd2.4) t= 1.16s 93.3% [McNemar nearest+P vs generated+P(seed0): 5-1, p=0.219]
29
+ 16 93.3% t= 0.45s 93.3%(sd1.4) t= 1.16s 93.3% [McNemar nearest+P vs generated+P(seed0): 1-0, p=1.000]
30
+ crossover K (generated+P mean >= nearest+P): never within tested range
results/audit/w4_model_arm_v2.txt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ W4 (v2, cell class): conditional diffusion model as arm (f), 3 seeded sampling draws, identical gate_v2 protocol (K=4, held-out 60 + benchmark point)
2
+ S=0: median_regret = 51.00 +- 6.91 % | within1% = 3.3 +- 1.4 % | median J = 0.56195 +- 0.02642 | bench_J = 0.454890 +- 0.056261 | p vs nearest = [0.0, 0.0, 0.0] [nearest: regret 0.11%, within1 71.7%]
3
+ S=20: median_regret = 2.90 +- 0.81 % | within1% = 34.4 +- 3.9 % | median J = 0.32561 +- 0.00326 | bench_J = 0.191351 +- 0.003120 | p vs nearest = [0.0, 0.0, 0.0] [nearest: regret -0.02%, within1 90.0%]
results/furuta_v3/gate_sidebyside.txt ADDED
@@ -0,0 +1,9 @@
 
 
 
 
 
 
 
 
 
 
1
+ config gen+P d4242 gen+P d9001 pooled gen raw pooled nearest+P diff pts [95% CI] McNemar s0 (n+P-only,gen-only,p)
2
+ v2 clipped (shipped) 63.3% 60.0% 61.7% 52.2% 82.5% -20.8 [-29.4,-12.5] 30-5 p=0.000
3
+ v2 clip +-5 (audit) 68.9% 75.6% 72.2% 50.3% 82.5% -10.3 [-16.9, -3.6] 19-11 p=0.200
4
+ v3 raw s0 (800ep) 76.7% 79.4% 78.1% 56.4% 82.5% -4.4 [-10.8, +1.9] 13-9 p=0.523
5
+ v3 raw s1 (800ep) 72.8% 77.2% 75.0% 54.7% 82.5% -7.5 [-13.6, -1.1] 13-7 p=0.263
6
+ v3 raw s0 (2400ep) 80.0% 88.3% 84.2% 59.7% 82.5% +1.7 [ -4.4, +8.6] 8-10 p=0.815
7
+ v3 raw s1 (2400ep) 81.7% 90.6% 86.1% 61.7% 82.5% +3.6 [ -2.5,+10.0] 7-12 p=0.359
8
+ v2-data clip5 raw (2400ep control) 82.8% 88.9% 85.8% 70.6% 82.5% +3.3 [ -2.5, +9.2] 5-9 p=0.424
9
+ v3 averaged-wts s0 (800ep) 38.3% 35.6% 36.9% 13.9% 82.5% -45.6 [-52.8,-38.3] 58-5 p=0.000
results/furuta_v3/traj_gen_eval_v2ctl2400_raw_clip5.txt ADDED
@@ -0,0 +1,94 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ bank data/furuta_nmpc_bank3000.npz: 2801 entries
2
+ model models/Best_furuta_traj_cond_v2ctl_e2400.pth: cond_dim=4 out_dim=116 hidden=256 layers=5 device=cpu
3
+ [seed4242] margin=0.3: resolving P for 227 unique bank source plants
4
+ [seed4242] sampler seed 0: 9.3s for 60x4 candidates (0 decode-invalid of 240)
5
+ [seed4242] sampler seed 1: 9.4s for 60x4 candidates (0 decode-invalid of 240)
6
+ [seed4242] sampler seed 2: 9.5s for 60x4 candidates (0 decode-invalid of 240)
7
+ [seed4242] 10/60 (24s)
8
+ [seed4242] 20/60 (37s)
9
+ [seed4242] 30/60 (58s)
10
+ [seed4242] 40/60 (71s)
11
+ [seed4242] 50/60 (92s)
12
+ [seed4242] 60/60 (111s)
13
+ [seed9001] margin=0.3: resolving P for 232 unique bank source plants
14
+ [seed9001] sampler seed 0: 10.1s for 60x4 candidates (0 decode-invalid of 240)
15
+ [seed9001] sampler seed 1: 9.7s for 60x4 candidates (0 decode-invalid of 240)
16
+ [seed9001] sampler seed 2: 9.9s for 60x4 candidates (0 decode-invalid of 240)
17
+ [seed9001] 10/60 (29s)
18
+ [seed9001] 20/60 (47s)
19
+ [seed9001] 30/60 (76s)
20
+ [seed9001] 40/60 (94s)
21
+ [seed9001] 50/60 (118s)
22
+ [seed9001] 60/60 (143s)
23
+
24
+ === traj_gen_eval (v2ctl2400_raw_clip5): generated vs retrieval vs NMPC ===
25
+
26
+ --- plant set seed4242 (seed 4242, n=60) ---
27
+ arm succ 95% CI medJ med regret p90 synth
28
+ per-plant 93% [ 87, 98] 4.08 0.00% 0.00% 4.8s
29
+ nearest 62% [ 50, 73] 4.15 0.36% 1.51% 285ms
30
+ nearest+P 78% [ 67, 88] 4.09 0.69% 2.52% 585ms
31
+ nearest-margin 62% [ 50, 73] 4.15 0.36% 1.51% 288ms
32
+ NMPC 95% [ 88, 100] 4.08 0.14% 0.24% 23ms
33
+ generated 68.9%(sd1.6) [ 58, 80]* 4.05 0.62%(sd0.2) 2.54%(sd0.2) 299ms
34
+ generated+P 82.8%(sd0.8) [ 72, 90]* 4.08 0.66%(sd0.2) 2.89%(sd0.3) 617ms
35
+ (* generated rows: mean (sd) over sampler seeds [0, 1, 2]; CI shown is for seed 0 only)
36
+ (certified hand-off rate == success rate for every arm except NMPC (a certificate is a precondition for success here); NMPC's own hand-off is n/a, it never switches to a static gain)
37
+
38
+ --- failure breakdown, primary seed 0 ---
39
+ per-plant NMPC never reached Omega:3 replay failed:1
40
+ nearest Lyapunov decrease fails at a vertex:13 rollout failed:9 true dynamics: 0.02% of the boundary has V-dot >= 0, 0.00% saturates:1
41
+ nearest+P rollout failed:13
42
+ nearest-margin Lyapunov decrease fails at a vertex:13 rollout failed:9 true dynamics: 0.02% of the boundary has V-dot >= 0, 0.00% saturates:1
43
+ NMPC :3
44
+ generated Lyapunov decrease fails at a vertex:7 rollout failed:5 peak |Kx| on Omega is 10.13 V > 10.0 V:1 peak |Kx| on Omega is 10.12 V > 10.0 V:1 peak |Kx| on Omega is 10.53 V > 10.0 V:1 peak |Kx| on Omega is 10.36 V > 10.0 V:1 peak |Kx| on Omega is 10.19 V > 10.0 V:1 peak |Kx| on Omega is 10.00 V > 10.0 V:1
45
+ generated+P rollout failed:11
46
+
47
+ --- paired McNemar on success (exact binomial on discordant pairs) ---
48
+ per-plant vs nearest beats on 20, loses on 1 p=0.0000 * cost ratio 1.004x p=0.0000
49
+ per-plant vs nearest+P beats on 10, loses on 1 p=0.0117 * cost ratio 1.007x p=0.0000
50
+ per-plant vs nearest-margin beats on 20, loses on 1 p=0.0000 * cost ratio 1.004x p=0.0000
51
+ per-plant vs NMPC beats on 0, loses on 1 p=1.0000 cost ratio 1.001x p=0.0002
52
+ per-plant vs generated beats on 14, loses on 0 p=0.0001 * cost ratio 1.008x p=0.0000
53
+ per-plant vs generated+P beats on 7, loses on 0 p=0.0156 * cost ratio 1.009x p=0.0000
54
+ nearest+P vs per-plant beats on 1, loses on 10 p=0.0117 * cost ratio 0.993x p=0.0000
55
+ nearest+P vs nearest beats on 10, loses on 0 p=0.0020 * cost ratio 1.000x p=0.8291
56
+ nearest+P vs nearest-margin beats on 10, loses on 0 p=0.0020 * cost ratio 1.000x p=0.8291
57
+ nearest+P vs NMPC beats on 0, loses on 10 p=0.0020 * cost ratio 0.995x p=0.0000
58
+ nearest+P vs generated beats on 9, loses on 4 p=0.2668 cost ratio 1.001x p=0.2675
59
+ nearest+P vs generated+P beats on 4, loses on 6 p=0.7539 cost ratio 1.001x p=0.2467
60
+
61
+ --- plant set seed9001 (seed 9001, n=60) ---
62
+ arm succ 95% CI medJ med regret p90 synth
63
+ per-plant 93% [ 87, 98] 4.16 0.00% 0.00% 7.9s
64
+ nearest 65% [ 52, 77] 4.10 0.71% 2.11% 305ms
65
+ nearest+P 87% [ 78, 95] 4.14 0.59% 2.34% 566ms
66
+ nearest-margin 65% [ 52, 77] 4.10 0.71% 2.11% 269ms
67
+ NMPC 97% [ 92, 100] 4.17 0.07% 0.22% 37ms
68
+ generated 72.2%(sd2.8) [ 57, 80]* 4.13 0.47%(sd0.1) 2.46%(sd0.7) 404ms
69
+ generated+P 88.9%(sd1.6) [ 82, 97]* 4.15 0.65%(sd0.2) 2.46%(sd0.2) 660ms
70
+ (* generated rows: mean (sd) over sampler seeds [0, 1, 2]; CI shown is for seed 0 only)
71
+ (certified hand-off rate == success rate for every arm except NMPC (a certificate is a precondition for success here); NMPC's own hand-off is n/a, it never switches to a static gain)
72
+
73
+ --- failure breakdown, primary seed 0 ---
74
+ per-plant replay failed:2 NMPC never reached Omega:2
75
+ nearest Lyapunov decrease fails at a vertex:16 rollout failed:5
76
+ nearest+P rollout failed:8
77
+ nearest-margin Lyapunov decrease fails at a vertex:16 rollout failed:5
78
+ NMPC :2
79
+ generated Lyapunov decrease fails at a vertex:10 rollout failed:6 peak |Kx| on Omega is 10.52 V > 10.0 V:1 peak |Kx| on Omega is 10.17 V > 10.0 V:1 peak |Kx| on Omega is 10.71 V > 10.0 V:1
80
+ generated+P rollout failed:6
81
+
82
+ --- paired McNemar on success (exact binomial on discordant pairs) ---
83
+ per-plant vs nearest beats on 18, loses on 1 p=0.0001 * cost ratio 1.007x p=0.0000
84
+ per-plant vs nearest+P beats on 5, loses on 1 p=0.2188 cost ratio 1.006x p=0.0000
85
+ per-plant vs nearest-margin beats on 18, loses on 1 p=0.0001 * cost ratio 1.007x p=0.0000
86
+ per-plant vs NMPC beats on 0, loses on 2 p=0.5000 cost ratio 1.001x p=0.0064
87
+ per-plant vs generated beats on 15, loses on 0 p=0.0001 * cost ratio 1.005x p=0.0000
88
+ per-plant vs generated+P beats on 2, loses on 0 p=0.5000 cost ratio 1.009x p=0.0000
89
+ nearest+P vs per-plant beats on 1, loses on 5 p=0.2188 cost ratio 0.994x p=0.0000
90
+ nearest+P vs nearest beats on 13, loses on 0 p=0.0002 * cost ratio 1.000x p=0.1154
91
+ nearest+P vs nearest-margin beats on 13, loses on 0 p=0.0002 * cost ratio 1.000x p=0.1154
92
+ nearest+P vs NMPC beats on 0, loses on 6 p=0.0312 * cost ratio 0.995x p=0.0000
93
+ nearest+P vs generated beats on 13, loses on 2 p=0.0074 * cost ratio 1.001x p=0.5116
94
+ nearest+P vs generated+P beats on 1, loses on 3 p=0.6250 cost ratio 1.002x p=0.1682
results/furuta_v3/traj_gen_eval_v3_ema_clip1.txt ADDED
@@ -0,0 +1,94 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ bank data/furuta_nmpc_bank3000.npz: 2801 entries
2
+ model models/Best_furuta_traj_cond_v3.pth: cond_dim=4 out_dim=116 hidden=256 layers=5 device=cpu
3
+ [seed4242] margin=0.3: resolving P for 227 unique bank source plants
4
+ [seed4242] sampler seed 0: 9.4s for 60x4 candidates (0 decode-invalid of 240)
5
+ [seed4242] sampler seed 1: 9.5s for 60x4 candidates (0 decode-invalid of 240)
6
+ [seed4242] sampler seed 2: 9.5s for 60x4 candidates (0 decode-invalid of 240)
7
+ [seed4242] 10/60 (32s)
8
+ [seed4242] 20/60 (53s)
9
+ [seed4242] 30/60 (77s)
10
+ [seed4242] 40/60 (96s)
11
+ [seed4242] 50/60 (118s)
12
+ [seed4242] 60/60 (141s)
13
+ [seed9001] margin=0.3: resolving P for 232 unique bank source plants
14
+ [seed9001] sampler seed 0: 10.6s for 60x4 candidates (0 decode-invalid of 240)
15
+ [seed9001] sampler seed 1: 10.8s for 60x4 candidates (0 decode-invalid of 240)
16
+ [seed9001] sampler seed 2: 11.3s for 60x4 candidates (0 decode-invalid of 240)
17
+ [seed9001] 10/60 (39s)
18
+ [seed9001] 20/60 (62s)
19
+ [seed9001] 30/60 (89s)
20
+ [seed9001] 40/60 (115s)
21
+ [seed9001] 50/60 (139s)
22
+ [seed9001] 60/60 (167s)
23
+
24
+ === traj_gen_eval (v3_ema_clip1): generated vs retrieval vs NMPC ===
25
+
26
+ --- plant set seed4242 (seed 4242, n=60) ---
27
+ arm succ 95% CI medJ med regret p90 synth
28
+ per-plant 93% [ 87, 98] 4.08 0.00% 0.00% 4.5s
29
+ nearest 62% [ 50, 73] 4.15 0.36% 1.51% 310ms
30
+ nearest+P 78% [ 67, 88] 4.09 0.69% 2.52% 637ms
31
+ nearest-margin 62% [ 50, 73] 4.15 0.36% 1.51% 310ms
32
+ NMPC 95% [ 88, 100] 4.08 0.14% 0.24% 25ms
33
+ generated 18.9%(sd3.4) [ 8, 28]* 4.00 2.04%(sd0.6) 28.63%(sd22.1) 422ms
34
+ generated+P 38.3%(sd2.7) [ 23, 47]* 4.15 1.97%(sd0.6) 45.55%(sd28.3) 1.7s
35
+ (* generated rows: mean (sd) over sampler seeds [0, 1, 2]; CI shown is for seed 0 only)
36
+ (certified hand-off rate == success rate for every arm except NMPC (a certificate is a precondition for success here); NMPC's own hand-off is n/a, it never switches to a static gain)
37
+
38
+ --- failure breakdown, primary seed 0 ---
39
+ per-plant NMPC never reached Omega:3 replay failed:1
40
+ nearest Lyapunov decrease fails at a vertex:13 rollout failed:9 true dynamics: 0.02% of the boundary has V-dot >= 0, 0.00% saturates:1
41
+ nearest+P rollout failed:13
42
+ nearest-margin Lyapunov decrease fails at a vertex:13 rollout failed:9 true dynamics: 0.02% of the boundary has V-dot >= 0, 0.00% saturates:1
43
+ NMPC :3
44
+ generated Lyapunov decrease fails at a vertex:29 rollout failed:19 peak |Kx| on Omega is 10.47 V > 10.0 V:1
45
+ generated+P rollout failed:39
46
+
47
+ --- paired McNemar on success (exact binomial on discordant pairs) ---
48
+ per-plant vs nearest beats on 20, loses on 1 p=0.0000 * cost ratio 1.004x p=0.0000
49
+ per-plant vs nearest+P beats on 10, loses on 1 p=0.0117 * cost ratio 1.007x p=0.0000
50
+ per-plant vs nearest-margin beats on 20, loses on 1 p=0.0000 * cost ratio 1.004x p=0.0000
51
+ per-plant vs NMPC beats on 0, loses on 1 p=1.0000 cost ratio 1.001x p=0.0002
52
+ per-plant vs generated beats on 45, loses on 0 p=0.0000 * cost ratio 1.028x p=0.0010
53
+ per-plant vs generated+P beats on 35, loses on 0 p=0.0000 * cost ratio 1.029x p=0.0000
54
+ nearest+P vs per-plant beats on 1, loses on 10 p=0.0117 * cost ratio 0.993x p=0.0000
55
+ nearest+P vs nearest beats on 10, loses on 0 p=0.0020 * cost ratio 1.000x p=0.8291
56
+ nearest+P vs nearest-margin beats on 10, loses on 0 p=0.0020 * cost ratio 1.000x p=0.8291
57
+ nearest+P vs NMPC beats on 0, loses on 10 p=0.0020 * cost ratio 0.995x p=0.0000
58
+ nearest+P vs generated beats on 38, loses on 2 p=0.0000 * cost ratio 1.008x p=0.0547
59
+ nearest+P vs generated+P beats on 30, loses on 4 p=0.0000 * cost ratio 1.013x p=0.0005
60
+
61
+ --- plant set seed9001 (seed 9001, n=60) ---
62
+ arm succ 95% CI medJ med regret p90 synth
63
+ per-plant 93% [ 87, 98] 4.16 0.00% 0.00% 7.6s
64
+ nearest 65% [ 52, 77] 4.10 0.71% 2.11% 226ms
65
+ nearest+P 87% [ 78, 95] 4.14 0.59% 2.34% 524ms
66
+ nearest-margin 65% [ 52, 77] 4.10 0.71% 2.11% 244ms
67
+ NMPC 97% [ 92, 100] 4.17 0.07% 0.22% 30ms
68
+ generated 8.9%(sd0.8) [ 2, 15]* 4.15 1.40%(sd0.6) 2.81%(sd1.2) 562ms
69
+ generated+P 35.6%(sd4.4) [ 30, 55]* 4.18 1.98%(sd0.2) 4.47%(sd0.8) 2.0s
70
+ (* generated rows: mean (sd) over sampler seeds [0, 1, 2]; CI shown is for seed 0 only)
71
+ (certified hand-off rate == success rate for every arm except NMPC (a certificate is a precondition for success here); NMPC's own hand-off is n/a, it never switches to a static gain)
72
+
73
+ --- failure breakdown, primary seed 0 ---
74
+ per-plant replay failed:2 NMPC never reached Omega:2
75
+ nearest Lyapunov decrease fails at a vertex:16 rollout failed:5
76
+ nearest+P rollout failed:8
77
+ nearest-margin Lyapunov decrease fails at a vertex:16 rollout failed:5
78
+ NMPC :2
79
+ generated Lyapunov decrease fails at a vertex:31 rollout failed:23 peak |Kx| on Omega is 10.04 V > 10.0 V:1
80
+ generated+P rollout failed:35
81
+
82
+ --- paired McNemar on success (exact binomial on discordant pairs) ---
83
+ per-plant vs nearest beats on 18, loses on 1 p=0.0001 * cost ratio 1.007x p=0.0000
84
+ per-plant vs nearest+P beats on 5, loses on 1 p=0.2188 cost ratio 1.006x p=0.0000
85
+ per-plant vs nearest-margin beats on 18, loses on 1 p=0.0001 * cost ratio 1.007x p=0.0000
86
+ per-plant vs NMPC beats on 0, loses on 2 p=0.5000 cost ratio 1.001x p=0.0064
87
+ per-plant vs generated beats on 51, loses on 0 p=0.0000 * cost ratio nanx p=nan
88
+ per-plant vs generated+P beats on 31, loses on 0 p=0.0000 * cost ratio 1.022x p=0.0000
89
+ nearest+P vs per-plant beats on 1, loses on 5 p=0.2188 cost ratio 0.994x p=0.0000
90
+ nearest+P vs nearest beats on 13, loses on 0 p=0.0002 * cost ratio 1.000x p=0.1154
91
+ nearest+P vs nearest-margin beats on 13, loses on 0 p=0.0002 * cost ratio 1.000x p=0.1154
92
+ nearest+P vs NMPC beats on 0, loses on 6 p=0.0312 * cost ratio 0.995x p=0.0000
93
+ nearest+P vs generated beats on 48, loses on 1 p=0.0000 * cost ratio nanx p=nan
94
+ nearest+P vs generated+P beats on 28, loses on 1 p=0.0000 * cost ratio 1.018x p=0.0001
results/furuta_v3/traj_gen_eval_v3e2400_raw_clip1.txt ADDED
@@ -0,0 +1,94 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ bank data/furuta_nmpc_bank3000.npz: 2801 entries
2
+ model models/Best_furuta_traj_cond_v3_e2400.pth: cond_dim=4 out_dim=116 hidden=256 layers=5 device=cpu
3
+ [seed4242] margin=0.3: resolving P for 227 unique bank source plants
4
+ [seed4242] sampler seed 0: 11.5s for 60x4 candidates (0 decode-invalid of 240)
5
+ [seed4242] sampler seed 1: 10.2s for 60x4 candidates (0 decode-invalid of 240)
6
+ [seed4242] sampler seed 2: 10.2s for 60x4 candidates (0 decode-invalid of 240)
7
+ [seed4242] 10/60 (31s)
8
+ [seed4242] 20/60 (47s)
9
+ [seed4242] 30/60 (66s)
10
+ [seed4242] 40/60 (83s)
11
+ [seed4242] 50/60 (105s)
12
+ [seed4242] 60/60 (128s)
13
+ [seed9001] margin=0.3: resolving P for 232 unique bank source plants
14
+ [seed9001] sampler seed 0: 9.4s for 60x4 candidates (0 decode-invalid of 240)
15
+ [seed9001] sampler seed 1: 9.6s for 60x4 candidates (0 decode-invalid of 240)
16
+ [seed9001] sampler seed 2: 9.4s for 60x4 candidates (0 decode-invalid of 240)
17
+ [seed9001] 10/60 (22s)
18
+ [seed9001] 20/60 (35s)
19
+ [seed9001] 30/60 (55s)
20
+ [seed9001] 40/60 (67s)
21
+ [seed9001] 50/60 (85s)
22
+ [seed9001] 60/60 (110s)
23
+
24
+ === traj_gen_eval (v3e2400_raw_clip1): generated vs retrieval vs NMPC ===
25
+
26
+ --- plant set seed4242 (seed 4242, n=60) ---
27
+ arm succ 95% CI medJ med regret p90 synth
28
+ per-plant 93% [ 87, 98] 4.08 0.00% 0.00% 5.1s
29
+ nearest 62% [ 50, 73] 4.15 0.36% 1.51% 317ms
30
+ nearest+P 78% [ 67, 88] 4.09 0.69% 2.52% 729ms
31
+ nearest-margin 62% [ 50, 73] 4.15 0.36% 1.51% 291ms
32
+ NMPC 95% [ 88, 100] 4.08 0.14% 0.24% 25ms
33
+ generated 55.6%(sd4.8) [ 37, 63]* 4.07 0.88%(sd0.2) 2.70%(sd0.4) 340ms
34
+ generated+P 80.0%(sd1.4) [ 68, 88]* 4.09 0.85%(sd0.1) 2.52%(sd0.6) 671ms
35
+ (* generated rows: mean (sd) over sampler seeds [0, 1, 2]; CI shown is for seed 0 only)
36
+ (certified hand-off rate == success rate for every arm except NMPC (a certificate is a precondition for success here); NMPC's own hand-off is n/a, it never switches to a static gain)
37
+
38
+ --- failure breakdown, primary seed 0 ---
39
+ per-plant NMPC never reached Omega:3 replay failed:1
40
+ nearest Lyapunov decrease fails at a vertex:13 rollout failed:9 true dynamics: 0.02% of the boundary has V-dot >= 0, 0.00% saturates:1
41
+ nearest+P rollout failed:13
42
+ nearest-margin Lyapunov decrease fails at a vertex:13 rollout failed:9 true dynamics: 0.02% of the boundary has V-dot >= 0, 0.00% saturates:1
43
+ NMPC :3
44
+ generated Lyapunov decrease fails at a vertex:18 rollout failed:6 peak |Kx| on Omega is 10.26 V > 10.0 V:1 peak |Kx| on Omega is 10.18 V > 10.0 V:1 peak |Kx| on Omega is 10.04 V > 10.0 V:1 peak |Kx| on Omega is 10.25 V > 10.0 V:1 peak |Kx| on Omega is 10.10 V > 10.0 V:1 peak |Kx| on Omega is 10.02 V > 10.0 V:1
45
+ generated+P rollout failed:13
46
+
47
+ --- paired McNemar on success (exact binomial on discordant pairs) ---
48
+ per-plant vs nearest beats on 20, loses on 1 p=0.0000 * cost ratio 1.004x p=0.0000
49
+ per-plant vs nearest+P beats on 10, loses on 1 p=0.0117 * cost ratio 1.007x p=0.0000
50
+ per-plant vs nearest-margin beats on 20, loses on 1 p=0.0000 * cost ratio 1.004x p=0.0000
51
+ per-plant vs NMPC beats on 0, loses on 1 p=1.0000 cost ratio 1.001x p=0.0002
52
+ per-plant vs generated beats on 26, loses on 0 p=0.0000 * cost ratio 1.007x p=0.0000
53
+ per-plant vs generated+P beats on 9, loses on 0 p=0.0039 * cost ratio 1.007x p=0.0000
54
+ nearest+P vs per-plant beats on 1, loses on 10 p=0.0117 * cost ratio 0.993x p=0.0000
55
+ nearest+P vs nearest beats on 10, loses on 0 p=0.0020 * cost ratio 1.000x p=0.8291
56
+ nearest+P vs nearest-margin beats on 10, loses on 0 p=0.0020 * cost ratio 1.000x p=0.8291
57
+ nearest+P vs NMPC beats on 0, loses on 10 p=0.0020 * cost ratio 0.995x p=0.0000
58
+ nearest+P vs generated beats on 19, loses on 2 p=0.0002 * cost ratio 1.002x p=0.4515
59
+ nearest+P vs generated+P beats on 6, loses on 6 p=1.0000 cost ratio 1.002x p=0.2411
60
+
61
+ --- plant set seed9001 (seed 9001, n=60) ---
62
+ arm succ 95% CI medJ med regret p90 synth
63
+ per-plant 93% [ 87, 98] 4.16 0.00% 0.00% 5.6s
64
+ nearest 65% [ 52, 77] 4.10 0.71% 2.11% 163ms
65
+ nearest+P 87% [ 78, 95] 4.14 0.59% 2.34% 423ms
66
+ nearest-margin 65% [ 52, 77] 4.10 0.71% 2.11% 163ms
67
+ NMPC 97% [ 92, 100] 4.17 0.07% 0.22% 22ms
68
+ generated 63.9%(sd2.8) [ 48, 72]* 4.21 0.82%(sd0.0) 3.52%(sd0.1) 307ms
69
+ generated+P 88.3%(sd2.4) [ 82, 97]* 4.20 0.83%(sd0.1) 3.50%(sd0.4) 615ms
70
+ (* generated rows: mean (sd) over sampler seeds [0, 1, 2]; CI shown is for seed 0 only)
71
+ (certified hand-off rate == success rate for every arm except NMPC (a certificate is a precondition for success here); NMPC's own hand-off is n/a, it never switches to a static gain)
72
+
73
+ --- failure breakdown, primary seed 0 ---
74
+ per-plant replay failed:2 NMPC never reached Omega:2
75
+ nearest Lyapunov decrease fails at a vertex:16 rollout failed:5
76
+ nearest+P rollout failed:8
77
+ nearest-margin Lyapunov decrease fails at a vertex:16 rollout failed:5
78
+ NMPC :2
79
+ generated Lyapunov decrease fails at a vertex:12 rollout failed:4 peak |Kx| on Omega is 10.10 V > 10.0 V:1 true dynamics: 0.38% of the boundary has V-dot >= 0, 0.00% saturates:1 peak |Kx| on Omega is 10.35 V > 10.0 V:1 peak |Kx| on Omega is 10.11 V > 10.0 V:1 peak |Kx| on Omega is 10.04 V > 10.0 V:1 true dynamics: 0.20% of the boundary has V-dot >= 0, 0.00% saturates:1 true dynamics: 0.03% of the boundary has V-dot >= 0, 0.00% saturates:1 peak |Kx| on Omega is 10.76 V > 10.0 V:1
80
+ generated+P rollout failed:6
81
+
82
+ --- paired McNemar on success (exact binomial on discordant pairs) ---
83
+ per-plant vs nearest beats on 18, loses on 1 p=0.0001 * cost ratio 1.007x p=0.0000
84
+ per-plant vs nearest+P beats on 5, loses on 1 p=0.2188 cost ratio 1.006x p=0.0000
85
+ per-plant vs nearest-margin beats on 18, loses on 1 p=0.0001 * cost ratio 1.007x p=0.0000
86
+ per-plant vs NMPC beats on 0, loses on 2 p=0.5000 cost ratio 1.001x p=0.0064
87
+ per-plant vs generated beats on 20, loses on 0 p=0.0000 * cost ratio 1.009x p=0.0000
88
+ per-plant vs generated+P beats on 2, loses on 0 p=0.5000 cost ratio 1.009x p=0.0000
89
+ nearest+P vs per-plant beats on 1, loses on 5 p=0.2188 cost ratio 0.994x p=0.0000
90
+ nearest+P vs nearest beats on 13, loses on 0 p=0.0002 * cost ratio 1.000x p=0.1154
91
+ nearest+P vs nearest-margin beats on 13, loses on 0 p=0.0002 * cost ratio 1.000x p=0.1154
92
+ nearest+P vs NMPC beats on 0, loses on 6 p=0.0312 * cost ratio 0.995x p=0.0000
93
+ nearest+P vs generated beats on 18, loses on 2 p=0.0004 * cost ratio 1.005x p=0.0036
94
+ nearest+P vs generated+P beats on 2, loses on 4 p=0.6875 cost ratio 1.003x p=0.0005
results/furuta_v3/traj_gen_eval_v3s1_raw_clip1.txt ADDED
@@ -0,0 +1,94 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ bank data/furuta_nmpc_bank3000.npz: 2801 entries
2
+ model models/Best_furuta_traj_cond_v3_s1.pth: cond_dim=4 out_dim=116 hidden=256 layers=5 device=cpu
3
+ [seed4242] margin=0.3: resolving P for 227 unique bank source plants
4
+ [seed4242] sampler seed 0: 9.7s for 60x4 candidates (0 decode-invalid of 240)
5
+ [seed4242] sampler seed 1: 9.7s for 60x4 candidates (0 decode-invalid of 240)
6
+ [seed4242] sampler seed 2: 9.5s for 60x4 candidates (0 decode-invalid of 240)
7
+ [seed4242] 10/60 (29s)
8
+ [seed4242] 20/60 (47s)
9
+ [seed4242] 30/60 (67s)
10
+ [seed4242] 40/60 (84s)
11
+ [seed4242] 50/60 (102s)
12
+ [seed4242] 60/60 (122s)
13
+ [seed9001] margin=0.3: resolving P for 232 unique bank source plants
14
+ [seed9001] sampler seed 0: 9.4s for 60x4 candidates (0 decode-invalid of 240)
15
+ [seed9001] sampler seed 1: 9.3s for 60x4 candidates (0 decode-invalid of 240)
16
+ [seed9001] sampler seed 2: 9.4s for 60x4 candidates (0 decode-invalid of 240)
17
+ [seed9001] 10/60 (31s)
18
+ [seed9001] 20/60 (49s)
19
+ [seed9001] 30/60 (70s)
20
+ [seed9001] 40/60 (88s)
21
+ [seed9001] 50/60 (110s)
22
+ [seed9001] 60/60 (148s)
23
+
24
+ === traj_gen_eval (v3s1_raw_clip1): generated vs retrieval vs NMPC ===
25
+
26
+ --- plant set seed4242 (seed 4242, n=60) ---
27
+ arm succ 95% CI medJ med regret p90 synth
28
+ per-plant 93% [ 87, 98] 4.08 0.00% 0.00% 3.9s
29
+ nearest 62% [ 50, 73] 4.15 0.36% 1.51% 321ms
30
+ nearest+P 78% [ 67, 88] 4.09 0.69% 2.52% 671ms
31
+ nearest-margin 62% [ 50, 73] 4.15 0.36% 1.51% 316ms
32
+ NMPC 95% [ 88, 100] 4.08 0.14% 0.24% 25ms
33
+ generated 52.8%(sd0.8) [ 40, 67]* 4.07 0.94%(sd0.1) 3.18%(sd0.2) 354ms
34
+ generated+P 72.8%(sd2.1) [ 63, 85]* 4.07 0.96%(sd0.1) 3.13%(sd0.2) 1.0s
35
+ (* generated rows: mean (sd) over sampler seeds [0, 1, 2]; CI shown is for seed 0 only)
36
+ (certified hand-off rate == success rate for every arm except NMPC (a certificate is a precondition for success here); NMPC's own hand-off is n/a, it never switches to a static gain)
37
+
38
+ --- failure breakdown, primary seed 0 ---
39
+ per-plant NMPC never reached Omega:3 replay failed:1
40
+ nearest Lyapunov decrease fails at a vertex:13 rollout failed:9 true dynamics: 0.02% of the boundary has V-dot >= 0, 0.00% saturates:1
41
+ nearest+P rollout failed:13
42
+ nearest-margin Lyapunov decrease fails at a vertex:13 rollout failed:9 true dynamics: 0.02% of the boundary has V-dot >= 0, 0.00% saturates:1
43
+ NMPC :3
44
+ generated rollout failed:12 Lyapunov decrease fails at a vertex:12 peak |Kx| on Omega is 10.15 V > 10.0 V:1 peak |Kx| on Omega is 10.43 V > 10.0 V:1 peak |Kx| on Omega is 10.07 V > 10.0 V:1 peak |Kx| on Omega is 10.11 V > 10.0 V:1
45
+ generated+P rollout failed:15
46
+
47
+ --- paired McNemar on success (exact binomial on discordant pairs) ---
48
+ per-plant vs nearest beats on 20, loses on 1 p=0.0000 * cost ratio 1.004x p=0.0000
49
+ per-plant vs nearest+P beats on 10, loses on 1 p=0.0117 * cost ratio 1.007x p=0.0000
50
+ per-plant vs nearest-margin beats on 20, loses on 1 p=0.0000 * cost ratio 1.004x p=0.0000
51
+ per-plant vs NMPC beats on 0, loses on 1 p=1.0000 cost ratio 1.001x p=0.0002
52
+ per-plant vs generated beats on 24, loses on 0 p=0.0000 * cost ratio 1.010x p=0.0000
53
+ per-plant vs generated+P beats on 11, loses on 0 p=0.0010 * cost ratio 1.010x p=0.0000
54
+ nearest+P vs per-plant beats on 1, loses on 10 p=0.0117 * cost ratio 0.993x p=0.0000
55
+ nearest+P vs nearest beats on 10, loses on 0 p=0.0020 * cost ratio 1.000x p=0.8291
56
+ nearest+P vs nearest-margin beats on 10, loses on 0 p=0.0020 * cost ratio 1.000x p=0.8291
57
+ nearest+P vs NMPC beats on 0, loses on 10 p=0.0020 * cost ratio 0.995x p=0.0000
58
+ nearest+P vs generated beats on 18, loses on 3 p=0.0015 * cost ratio 1.005x p=0.1316
59
+ nearest+P vs generated+P beats on 6, loses on 4 p=0.7539 cost ratio 1.004x p=0.0562
60
+
61
+ --- plant set seed9001 (seed 9001, n=60) ---
62
+ arm succ 95% CI medJ med regret p90 synth
63
+ per-plant 93% [ 87, 98] 4.16 0.00% 0.00% 7.0s
64
+ nearest 65% [ 52, 77] 4.10 0.71% 2.11% 229ms
65
+ nearest+P 87% [ 78, 95] 4.14 0.59% 2.34% 519ms
66
+ nearest-margin 65% [ 52, 77] 4.10 0.71% 2.11% 223ms
67
+ NMPC 97% [ 92, 100] 4.17 0.07% 0.22% 23ms
68
+ generated 56.7%(sd5.9) [ 47, 73]* 4.15 0.93%(sd0.1) 3.01%(sd0.1) 458ms
69
+ generated+P 77.2%(sd2.1) [ 70, 90]* 4.19 0.92%(sd0.1) 2.93%(sd0.1) 1.2s
70
+ (* generated rows: mean (sd) over sampler seeds [0, 1, 2]; CI shown is for seed 0 only)
71
+ (certified hand-off rate == success rate for every arm except NMPC (a certificate is a precondition for success here); NMPC's own hand-off is n/a, it never switches to a static gain)
72
+
73
+ --- failure breakdown, primary seed 0 ---
74
+ per-plant replay failed:2 NMPC never reached Omega:2
75
+ nearest Lyapunov decrease fails at a vertex:16 rollout failed:5
76
+ nearest+P rollout failed:8
77
+ nearest-margin Lyapunov decrease fails at a vertex:16 rollout failed:5
78
+ NMPC :2
79
+ generated Lyapunov decrease fails at a vertex:14 rollout failed:7 peak |Kx| on Omega is 10.18 V > 10.0 V:1 peak |Kx| on Omega is 10.06 V > 10.0 V:1 true dynamics: 0.57% of the boundary has V-dot >= 0, 0.00% saturates:1
80
+ generated+P rollout failed:12
81
+
82
+ --- paired McNemar on success (exact binomial on discordant pairs) ---
83
+ per-plant vs nearest beats on 18, loses on 1 p=0.0001 * cost ratio 1.007x p=0.0000
84
+ per-plant vs nearest+P beats on 5, loses on 1 p=0.2188 cost ratio 1.006x p=0.0000
85
+ per-plant vs nearest-margin beats on 18, loses on 1 p=0.0001 * cost ratio 1.007x p=0.0000
86
+ per-plant vs NMPC beats on 0, loses on 2 p=0.5000 cost ratio 1.001x p=0.0064
87
+ per-plant vs generated beats on 20, loses on 0 p=0.0000 * cost ratio 1.009x p=0.0000
88
+ per-plant vs generated+P beats on 8, loses on 0 p=0.0078 * cost ratio 1.010x p=0.0000
89
+ nearest+P vs per-plant beats on 1, loses on 5 p=0.2188 cost ratio 0.994x p=0.0000
90
+ nearest+P vs nearest beats on 13, loses on 0 p=0.0002 * cost ratio 1.000x p=0.1154
91
+ nearest+P vs nearest-margin beats on 13, loses on 0 p=0.0002 * cost ratio 1.000x p=0.1154
92
+ nearest+P vs NMPC beats on 0, loses on 6 p=0.0312 * cost ratio 0.995x p=0.0000
93
+ nearest+P vs generated beats on 18, loses on 2 p=0.0004 * cost ratio 1.004x p=0.0081
94
+ nearest+P vs generated+P beats on 7, loses on 3 p=0.3438 cost ratio 1.005x p=0.0008
results/furuta_v3/traj_gen_v3_raw_clip1.txt ADDED
@@ -0,0 +1,30 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ bank data/furuta_nmpc_bank3000.npz: 2801 entries
2
+ model models/Best_furuta_traj_cond_v3.pth: cond_dim=4 out_dim=116
3
+ [seed4242] per-plant ceiling: 93%
4
+ [seed4242] nearest+P sequences done
5
+ [seed4242] generated+P seed 0 sequences done (36.5s sampling for 60x16)
6
+ [seed4242] generated+P seed 1 sequences done (36.6s sampling for 60x16)
7
+ [seed4242] generated+P seed 2 sequences done (36.3s sampling for 60x16)
8
+ [seed9001] per-plant ceiling: 93%
9
+ [seed9001] nearest+P sequences done
10
+ [seed9001] generated+P seed 0 sequences done (38.4s sampling for 60x16)
11
+ [seed9001] generated+P seed 1 sequences done (36.7s sampling for 60x16)
12
+ [seed9001] generated+P seed 2 sequences done (35.7s sampling for 60x16)
13
+
14
+ === candidate-budget sweep: generated+P vs nearest+P vs per-plant ceiling ===
15
+
16
+ --- plant set seed4242 (seed 4242, n=60) ---
17
+ K nearest+P generated+P per-plant
18
+ 1 51.7% t= 0.44s 38.3%(sd1.4) t= 1.00s 93.3% [McNemar nearest+P vs generated+P(seed0): 16-7, p=0.093]
19
+ 4 78.3% t= 0.70s 73.9%(sd2.1) t= 1.61s 93.3% [McNemar nearest+P vs generated+P(seed0): 9-8, p=1.000]
20
+ 8 88.3% t= 0.70s 83.3%(sd3.6) t= 1.61s 93.3% [McNemar nearest+P vs generated+P(seed0): 3-3, p=1.000]
21
+ 16 93.3% t= 0.70s 85.0%(sd2.7) t= 1.61s 93.3% [McNemar nearest+P vs generated+P(seed0): 4-1, p=0.375]
22
+ crossover K (generated+P mean >= nearest+P): never within tested range
23
+
24
+ --- plant set seed9001 (seed 9001, n=60) ---
25
+ K nearest+P generated+P per-plant
26
+ 1 56.7% t= 0.33s 38.3%(sd6.2) t= 0.99s 93.3% [McNemar nearest+P vs generated+P(seed0): 22-6, p=0.004]
27
+ 4 86.7% t= 0.49s 77.8%(sd0.8) t= 1.50s 93.3% [McNemar nearest+P vs generated+P(seed0): 8-3, p=0.227]
28
+ 8 91.7% t= 0.49s 88.9%(sd0.8) t= 1.50s 93.3% [McNemar nearest+P vs generated+P(seed0): 3-1, p=0.625]
29
+ 16 93.3% t= 0.49s 92.2%(sd1.6) t= 1.50s 93.3% [McNemar nearest+P vs generated+P(seed0): 1-1, p=1.000]
30
+ crossover K (generated+P mean >= nearest+P): never within tested range
results/furuta_v3/traj_gen_v3e2400s1_raw_clip1.txt ADDED
@@ -0,0 +1,30 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ bank data/furuta_nmpc_bank3000.npz: 2801 entries
2
+ model models/Best_furuta_traj_cond_v3_e2400_s1.pth: cond_dim=4 out_dim=116
3
+ [seed4242] per-plant ceiling: 93%
4
+ [seed4242] nearest+P sequences done
5
+ [seed4242] generated+P seed 0 sequences done (36.6s sampling for 60x16)
6
+ [seed4242] generated+P seed 1 sequences done (37.9s sampling for 60x16)
7
+ [seed4242] generated+P seed 2 sequences done (37.1s sampling for 60x16)
8
+ [seed9001] per-plant ceiling: 93%
9
+ [seed9001] nearest+P sequences done
10
+ [seed9001] generated+P seed 0 sequences done (36.5s sampling for 60x16)
11
+ [seed9001] generated+P seed 1 sequences done (36.2s sampling for 60x16)
12
+ [seed9001] generated+P seed 2 sequences done (36.8s sampling for 60x16)
13
+
14
+ === candidate-budget sweep: generated+P vs nearest+P vs per-plant ceiling ===
15
+
16
+ --- plant set seed4242 (seed 4242, n=60) ---
17
+ K nearest+P generated+P per-plant
18
+ 1 51.7% t= 0.32s 55.6%(sd9.1) t= 1.10s 93.3% [McNemar nearest+P vs generated+P(seed0): 14-9, p=0.405]
19
+ 4 78.3% t= 0.61s 85.0%(sd2.7) t= 1.22s 93.3% [McNemar nearest+P vs generated+P(seed0): 3-7, p=0.344]
20
+ 8 88.3% t= 0.61s 90.0%(sd1.4) t= 1.22s 93.3% [McNemar nearest+P vs generated+P(seed0): 3-4, p=1.000]
21
+ 16 93.3% t= 0.61s 91.1%(sd1.6) t= 1.22s 93.3% [McNemar nearest+P vs generated+P(seed0): 3-1, p=0.625]
22
+ crossover K (generated+P mean >= nearest+P): 1
23
+
24
+ --- plant set seed9001 (seed 9001, n=60) ---
25
+ K nearest+P generated+P per-plant
26
+ 1 56.7% t= 0.48s 61.1%(sd5.7) t= 0.99s 93.3% [McNemar nearest+P vs generated+P(seed0): 15-13, p=0.851]
27
+ 4 86.7% t= 0.68s 90.0%(sd0.0) t= 1.17s 93.3% [McNemar nearest+P vs generated+P(seed0): 2-4, p=0.688]
28
+ 8 91.7% t= 0.68s 95.0%(sd1.4) t= 1.17s 93.3% [McNemar nearest+P vs generated+P(seed0): 1-2, p=1.000]
29
+ 16 93.3% t= 0.68s 95.6%(sd0.8) t= 1.17s 93.3% [McNemar nearest+P vs generated+P(seed0): 0-1, p=1.000]
30
+ crossover K (generated+P mean >= nearest+P): 1
results/furuta_v3/traj_gen_v3s1_raw_clip1.txt ADDED
@@ -0,0 +1,30 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ bank data/furuta_nmpc_bank3000.npz: 2801 entries
2
+ model models/Best_furuta_traj_cond_v3_s1.pth: cond_dim=4 out_dim=116
3
+ [seed4242] per-plant ceiling: 93%
4
+ [seed4242] nearest+P sequences done
5
+ [seed4242] generated+P seed 0 sequences done (39.8s sampling for 60x16)
6
+ [seed4242] generated+P seed 1 sequences done (35.2s sampling for 60x16)
7
+ [seed4242] generated+P seed 2 sequences done (35.2s sampling for 60x16)
8
+ [seed9001] per-plant ceiling: 93%
9
+ [seed9001] nearest+P sequences done
10
+ [seed9001] generated+P seed 0 sequences done (35.6s sampling for 60x16)
11
+ [seed9001] generated+P seed 1 sequences done (35.4s sampling for 60x16)
12
+ [seed9001] generated+P seed 2 sequences done (35.2s sampling for 60x16)
13
+
14
+ === candidate-budget sweep: generated+P vs nearest+P vs per-plant ceiling ===
15
+
16
+ --- plant set seed4242 (seed 4242, n=60) ---
17
+ K nearest+P generated+P per-plant
18
+ 1 51.7% t= 0.42s 29.4%(sd4.2) t= 0.91s 93.3% [McNemar nearest+P vs generated+P(seed0): 20-4, p=0.002]
19
+ 4 78.3% t= 0.76s 70.6%(sd5.5) t= 1.48s 93.3% [McNemar nearest+P vs generated+P(seed0): 9-5, p=0.424]
20
+ 8 88.3% t= 0.76s 81.1%(sd1.6) t= 1.48s 93.3% [McNemar nearest+P vs generated+P(seed0): 5-2, p=0.453]
21
+ 16 93.3% t= 0.76s 82.2%(sd0.8) t= 1.48s 93.3% [McNemar nearest+P vs generated+P(seed0): 7-1, p=0.070]
22
+ crossover K (generated+P mean >= nearest+P): never within tested range
23
+
24
+ --- plant set seed9001 (seed 9001, n=60) ---
25
+ K nearest+P generated+P per-plant
26
+ 1 56.7% t= 0.30s 33.9%(sd9.6) t= 0.93s 93.3% [McNemar nearest+P vs generated+P(seed0): 25-4, p=0.000]
27
+ 4 86.7% t= 0.47s 72.2%(sd2.1) t= 1.70s 93.3% [McNemar nearest+P vs generated+P(seed0): 10-1, p=0.012]
28
+ 8 91.7% t= 0.47s 82.8%(sd1.6) t= 1.70s 93.3% [McNemar nearest+P vs generated+P(seed0): 7-1, p=0.070]
29
+ 16 93.3% t= 0.47s 90.6%(sd2.1) t= 1.70s 93.3% [McNemar nearest+P vs generated+P(seed0): 2-0, p=0.500]
30
+ crossover K (generated+P mean >= nearest+P): never within tested range
results/furuta_v3/traj_gen_warmstart_v2clip5_s0.txt ADDED
@@ -0,0 +1,17 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ === warm-start test (n=60, seed 4242, max_iter=700) ===
2
+ target certificate: largest_certified_region per plant (unmargined); verifier: furuta_traj_bank.score_solution (identical to every other arm in this doc)
3
+
4
+ condition success med iter med t med J
5
+ cold 53.3% 363 2.03s 4.06
6
+ nearest-warm 76.7% 45 0.27s 4.15
7
+ generated-warm 75.0% 50 0.30s 4.14
8
+
9
+ --- paired tests ---
10
+ generated-warm vs cold success: beats 13, loses 0, McNemar p=0.0002 * time ratio 0.175x (n=32), Wilcoxon p=0.0000 *
11
+ generated-warm vs nearest-warm success: beats 0, loses 1, McNemar p=1.0000 time ratio 1.127x (n=45), Wilcoxon p=0.0707
12
+ nearest-warm vs cold success: beats 14, loses 0, McNemar p=0.0001 * time ratio 0.147x (n=32), Wilcoxon p=0.0000 *
13
+
14
+ --- failure breakdown ---
15
+ cold ipopt: Maximum_Iterations_Exceeded:13 ipopt: Infeasible_Problem_Detected:11 rollout failed:4
16
+ nearest-warm ipopt: Infeasible_Problem_Detected:6 rollout failed:5 ipopt: Maximum_Iterations_Exceeded:3
17
+ generated-warm ipopt: Maximum_Iterations_Exceeded:7 ipopt: Infeasible_Problem_Detected:4 rollout failed:4
results/furuta_v3/traj_gen_warmstart_v3_s0.txt ADDED
@@ -0,0 +1,17 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ === warm-start test (n=60, seed 4242, max_iter=700) ===
2
+ target certificate: largest_certified_region per plant (unmargined); verifier: furuta_traj_bank.score_solution (identical to every other arm in this doc)
3
+
4
+ condition success med iter med t med J
5
+ cold 53.3% 363 2.03s 4.06
6
+ nearest-warm 76.7% 45 0.31s 4.15
7
+ generated-warm 76.7% 50 0.32s 4.15
8
+
9
+ --- paired tests ---
10
+ generated-warm vs cold success: beats 14, loses 0, McNemar p=0.0001 * time ratio 0.181x (n=32), Wilcoxon p=0.0000 *
11
+ generated-warm vs nearest-warm success: beats 0, loses 0, McNemar p=1.0000 time ratio 1.061x (n=46), Wilcoxon p=0.1474
12
+ nearest-warm vs cold success: beats 14, loses 0, McNemar p=0.0001 * time ratio 0.161x (n=32), Wilcoxon p=0.0000 *
13
+
14
+ --- failure breakdown ---
15
+ cold ipopt: Maximum_Iterations_Exceeded:13 ipopt: Infeasible_Problem_Detected:11 rollout failed:4
16
+ nearest-warm ipopt: Infeasible_Problem_Detected:6 rollout failed:5 ipopt: Maximum_Iterations_Exceeded:3
17
+ generated-warm ipopt: Infeasible_Problem_Detected:6 rollout failed:5 ipopt: Maximum_Iterations_Exceeded:3
results/furuta_v3/traj_gen_warmstart_v3_s1.txt ADDED
@@ -0,0 +1,17 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ === warm-start test (n=60, seed 4242, max_iter=700) ===
2
+ target certificate: largest_certified_region per plant (unmargined); verifier: furuta_traj_bank.score_solution (identical to every other arm in this doc)
3
+
4
+ condition success med iter med t med J
5
+ cold 53.3% 363 2.04s 4.06
6
+ nearest-warm 76.7% 45 0.29s 4.15
7
+ generated-warm 75.0% 48 0.30s 4.14
8
+
9
+ --- paired tests ---
10
+ generated-warm vs cold success: beats 13, loses 0, McNemar p=0.0002 * time ratio 0.155x (n=32), Wilcoxon p=0.0000 *
11
+ generated-warm vs nearest-warm success: beats 0, loses 1, McNemar p=1.0000 time ratio 1.107x (n=45), Wilcoxon p=0.3238
12
+ nearest-warm vs cold success: beats 14, loses 0, McNemar p=0.0001 * time ratio 0.149x (n=32), Wilcoxon p=0.0000 *
13
+
14
+ --- failure breakdown ---
15
+ cold ipopt: Maximum_Iterations_Exceeded:13 ipopt: Infeasible_Problem_Detected:11 rollout failed:4
16
+ nearest-warm ipopt: Infeasible_Problem_Detected:6 rollout failed:5 ipopt: Maximum_Iterations_Exceeded:3
17
+ generated-warm ipopt: Infeasible_Problem_Detected:8 ipopt: Maximum_Iterations_Exceeded:4 rollout failed:3
results/furuta_v3/traj_gen_warmstart_v3e2400_n200.txt ADDED
@@ -0,0 +1,17 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ === warm-start test (n=200, seed 4242, max_iter=700) ===
2
+ target certificate: largest_certified_region per plant (unmargined); verifier: furuta_traj_bank.score_solution (identical to every other arm in this doc)
3
+
4
+ condition success med iter med t med J
5
+ cold 58.0% 290 1.54s 4.06
6
+ nearest-warm 78.0% 48 0.28s 4.10
7
+ generated-warm 78.0% 48 0.28s 4.10
8
+
9
+ --- paired tests ---
10
+ generated-warm vs cold success: beats 41, loses 1, McNemar p=0.0000 * time ratio 0.213x (n=115), Wilcoxon p=0.0000 *
11
+ generated-warm vs nearest-warm success: beats 1, loses 1, McNemar p=1.0000 time ratio 1.020x (n=155), Wilcoxon p=0.5271
12
+ nearest-warm vs cold success: beats 41, loses 1, McNemar p=0.0000 * time ratio 0.205x (n=115), Wilcoxon p=0.0000 *
13
+
14
+ --- failure breakdown ---
15
+ cold ipopt: Maximum_Iterations_Exceeded:36 ipopt: Infeasible_Problem_Detected:31 rollout failed:17
16
+ nearest-warm rollout failed:26 ipopt: Maximum_Iterations_Exceeded:10 ipopt: Infeasible_Problem_Detected:8
17
+ generated-warm rollout failed:24 ipopt: Infeasible_Problem_Detected:11 ipopt: Maximum_Iterations_Exceeded:9
results/furuta_v3/traj_gen_warmstart_v3e2400_s0.txt ADDED
@@ -0,0 +1,17 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ === warm-start test (n=60, seed 4242, max_iter=700) ===
2
+ target certificate: largest_certified_region per plant (unmargined); verifier: furuta_traj_bank.score_solution (identical to every other arm in this doc)
3
+
4
+ condition success med iter med t med J
5
+ cold 53.3% 363 1.97s 4.06
6
+ nearest-warm 76.7% 45 0.26s 4.15
7
+ generated-warm 75.0% 43 0.25s 4.14
8
+
9
+ --- paired tests ---
10
+ generated-warm vs cold success: beats 13, loses 0, McNemar p=0.0002 * time ratio 0.152x (n=32), Wilcoxon p=0.0000 *
11
+ generated-warm vs nearest-warm success: beats 0, loses 1, McNemar p=1.0000 time ratio 1.056x (n=45), Wilcoxon p=0.7713
12
+ nearest-warm vs cold success: beats 14, loses 0, McNemar p=0.0001 * time ratio 0.156x (n=32), Wilcoxon p=0.0000 *
13
+
14
+ --- failure breakdown ---
15
+ cold ipopt: Maximum_Iterations_Exceeded:13 ipopt: Infeasible_Problem_Detected:11 rollout failed:4
16
+ nearest-warm ipopt: Infeasible_Problem_Detected:6 rollout failed:5 ipopt: Maximum_Iterations_Exceeded:3
17
+ generated-warm ipopt: Infeasible_Problem_Detected:6 rollout failed:5 ipopt: Maximum_Iterations_Exceeded:4
results/furuta_v3/traj_gen_warmstart_v3e2400_s1.txt ADDED
@@ -0,0 +1,17 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ === warm-start test (n=60, seed 4242, max_iter=700) ===
2
+ target certificate: largest_certified_region per plant (unmargined); verifier: furuta_traj_bank.score_solution (identical to every other arm in this doc)
3
+
4
+ condition success med iter med t med J
5
+ cold 53.3% 363 2.03s 4.06
6
+ nearest-warm 76.7% 45 0.27s 4.15
7
+ generated-warm 75.0% 44 0.26s 4.14
8
+
9
+ --- paired tests ---
10
+ generated-warm vs cold success: beats 13, loses 0, McNemar p=0.0002 * time ratio 0.157x (n=32), Wilcoxon p=0.0000 *
11
+ generated-warm vs nearest-warm success: beats 0, loses 1, McNemar p=1.0000 time ratio 1.008x (n=45), Wilcoxon p=0.9643
12
+ nearest-warm vs cold success: beats 14, loses 0, McNemar p=0.0001 * time ratio 0.160x (n=32), Wilcoxon p=0.0000 *
13
+
14
+ --- failure breakdown ---
15
+ cold ipopt: Maximum_Iterations_Exceeded:13 ipopt: Infeasible_Problem_Detected:11 rollout failed:4
16
+ nearest-warm ipopt: Infeasible_Problem_Detected:6 rollout failed:5 ipopt: Maximum_Iterations_Exceeded:3
17
+ generated-warm ipopt: Infeasible_Problem_Detected:6 rollout failed:5 ipopt: Maximum_Iterations_Exceeded:4
results/furuta_v3/traj_gen_warmstart_v3e2400_s2.txt ADDED
@@ -0,0 +1,17 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ === warm-start test (n=60, seed 4242, max_iter=700) ===
2
+ target certificate: largest_certified_region per plant (unmargined); verifier: furuta_traj_bank.score_solution (identical to every other arm in this doc)
3
+
4
+ condition success med iter med t med J
5
+ cold 53.3% 363 2.04s 4.06
6
+ nearest-warm 76.7% 45 0.27s 4.15
7
+ generated-warm 75.0% 47 0.28s 4.14
8
+
9
+ --- paired tests ---
10
+ generated-warm vs cold success: beats 13, loses 0, McNemar p=0.0002 * time ratio 0.160x (n=32), Wilcoxon p=0.0000 *
11
+ generated-warm vs nearest-warm success: beats 0, loses 1, McNemar p=1.0000 time ratio 1.038x (n=45), Wilcoxon p=0.7799
12
+ nearest-warm vs cold success: beats 14, loses 0, McNemar p=0.0001 * time ratio 0.151x (n=32), Wilcoxon p=0.0000 *
13
+
14
+ --- failure breakdown ---
15
+ cold ipopt: Maximum_Iterations_Exceeded:13 ipopt: Infeasible_Problem_Detected:11 rollout failed:4
16
+ nearest-warm ipopt: Infeasible_Problem_Detected:6 rollout failed:5 ipopt: Maximum_Iterations_Exceeded:3
17
+ generated-warm ipopt: Infeasible_Problem_Detected:7 ipopt: Maximum_Iterations_Exceeded:4 rollout failed:4
results/furuta_v3/valloss.txt ADDED
@@ -0,0 +1,7 @@
 
 
 
 
 
 
 
 
1
+ models/Best_furuta_traj_cond_v3.pth: val eps-MSE raw 0.0560 averaged 0.0777 (trainer best_val 0.0704, ckpt time 14.9 min)
2
+ models/Last_furuta_traj_cond_v3.pth: val eps-MSE raw 0.0560 averaged 0.0777 (trainer best_val 0.0704, ckpt time 14.9 min)
3
+ models/Best_furuta_traj_cond_v3_s1.pth: val eps-MSE raw 0.0566 averaged 0.0808 (trainer best_val 0.0642, ckpt time 13.3 min)
4
+ models/Best_furuta_traj_cond_v2.pth: val eps-MSE raw 0.0997 averaged 0.1348 (trainer best_val 0.1208, ckpt time 8.6 min)
5
+ models/Best_furuta_traj_cond_v3_e2400.pth: val eps-MSE raw 0.0366 averaged 0.0665 (trainer best_val 0.0566, ckpt time 42.5 min)
6
+ models/Best_furuta_traj_cond_v3_e2400_s1.pth: val eps-MSE raw 0.0367 averaged 0.0632 (trainer best_val 0.0538, ckpt time 39.2 min)
7
+ models/Best_furuta_traj_cond_v2ctl_e2400.pth: val eps-MSE raw 0.0776 averaged 0.0993 (trainer best_val 0.0816, ckpt time 37.6 min)
results/k_sweep_summary.md ADDED
@@ -0,0 +1,81 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # K-sweep summary β€” coverage vs sampling budget K (SISO / MIMO / pendulum)
2
+
3
+ "Does drawing more controllers from the prior close the stabilizable gap?" β€” measured per family.
4
+ Coverage = fraction of held-out plants with β‰₯1 stabilizing controller among K draws. Scripts:
5
+ `k_sweep.py` (SISO), `k_sweep_mimo.py` (MIMO), `k_sweep_pendulum.py` (pendulum).
6
+
7
+ ## Per-plant oracle (what "stabilizing" means, per family)
8
+
9
+ Coverage counts a plant as covered if **β‰₯1 of its K drawn controllers stabilizes it**. The decision
10
+ procedure that labels a single (plant, controller) pair β€” the *per-plant oracle* β€” is **not the same
11
+ across families**, and only the pendulum uses a genuine certificate:
12
+
13
+ | family | script | oracle used in the sweep | parameters | certificate? |
14
+ |---|---|---|---|---|
15
+ | SISO | `k_sweep.py` | **rollout only** β€” `simulate_closed_loop_batch`; stable iff mean `final_energy` finite and `< 0.3` | n_ic=4 (`x0=randn`), dt=0.03, T=12 s | none β€” rel-deg-3 loop, not LMI-certifiable |
16
+ | MIMO | `k_sweep_mimo.py` | **rollout only** β€” `simulate_closed_loop_mimo_batched`; stable iff `final_energy < 0.3` and `J_integral` finite | n_ic=4, dt=0.03, T=12 s | exists (`certified_beta`) but **not called** in the sweep |
17
+ | Pendulum | `k_sweep_pendulum.py` | **rollout AND certificate** β€” `simulate_pendulum_batch` (stable iff `final_energy < 0.02`) **plus** `certify_controller` (regional LMI), reported as a separate curve | rollout: n_ic=3, dt=0.02, T=6 s | **yes** β€” regional LMI per plant |
18
+
19
+ Reading this precisely:
20
+
21
+ - **SISO** β€” the oracle is a **finite-horizon energy-decay rollout**, no certificate. SISO loops are
22
+ relative degree 3, outside what the quadratic Popov/circle criterion can certify, so there is no
23
+ per-plant certificate to call; "stable" means the state energy has decayed below 0.3 by T=12 s from
24
+ 4 random initial conditions.
25
+ - **MIMO** β€” the sweep oracle is **also rollout only** (energy + finite cost). A valid MIMO certificate
26
+ (`certified_beta`, the circle/diagonal-multiplier LMI) exists and certifies the *dataset targets*,
27
+ but it is **not invoked inside the K-sweep** β€” so the MIMO coverage numbers here are rollout-validated,
28
+ not per-draw-certified. (The "100% certified" MIMO claim elsewhere refers to the dataset targets and to
29
+ the eigenvalue-certified repair, not to this coverage sweep.)
30
+ - **Pendulum** β€” the **only** family whose sweep runs a true per-plant certificate oracle. Each draw is
31
+ both simulated (`final_energy < 0.02`) and passed through the regional LMI `certify_controller`; both
32
+ the "β‰₯1 stable" and "β‰₯1 certified" curves are reported, and both sit at 1.000 for all K.
33
+
34
+ So: SISO/MIMO coverage = empirical (rollout) oracle; pendulum coverage = empirical **and** certificate
35
+ oracle. This is the honest scope β€” the certified-of-K story is pendulum-only.
36
+
37
+ ## Cross-task comparison
38
+
39
+ The needed sampling budget is a direct read-out of dataset quality: cost-optimal + certified targets
40
+ (MIMO, pendulum) need essentially no sampling; SISO's unfiltered random dataset needs Kβ‰ˆ32.
41
+
42
+ | task | dataset targets | per-sample stability | K to reach β‰₯1-stable = 1.0 |
43
+ |---|---|---|---|
44
+ | SISO | random-in-sector (not filtered, ~52% stable) | 0.66 | ~32 |
45
+ | MIMO | cost-optimal, certified | 0.96 | 4 |
46
+ | Pendulum | cost-optimal, certified | 1.00 | 1 (and certified at K=1) |
47
+
48
+ ## Per-task curves
49
+
50
+ **SISO** (N=24, seed 0, per-sample stability 0.66):
51
+
52
+ | K | 1 | 2 | 4 | 8 | 16 | 32 | 64 |
53
+ |---|---|---|---|---|---|---|---|
54
+ | plants β‰₯1 stable | 0.583 | 0.792 | 0.792 | 0.875 | 0.917 | 1.000 | 1.000 |
55
+
56
+ **MIMO** (N=60, seed 9999, per-sample stability 0.96):
57
+
58
+ | K | 1 | 2 | 4 | 8 | 16 | 32 |
59
+ |---|---|---|---|---|---|---|
60
+ | plants β‰₯1 stable | 0.950 | 0.983 | 1.000 | 1.000 | 1.000 | 1.000 |
61
+
62
+ **Pendulum** (N=24, seed 123, per-sample stability 1.00; has a valid certificate, so also reported):
63
+
64
+ | K | 1 | 2 | 4 | 8 | 16 |
65
+ |---|---|---|---|---|---|
66
+ | plants β‰₯1 stable | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 |
67
+ | plants β‰₯1 **certified** | 1.000 | 1.000 | 1.000 | 1.000 | 1.000 |
68
+
69
+ ## Takeaways
70
+
71
+ - The SISO "gap" is specifically an artifact of its unfiltered random dataset (only ~52% of stored
72
+ controllers are actually stable). The prior imitates that mixed distribution, so more draws are
73
+ needed to land a stable one.
74
+ - MIMO and the pendulum have no stabilizable gap: one to four draws already stabilize every plant,
75
+ and the pendulum is regionally **certified** from a single draw.
76
+ - So the sampling budget K to stabilize scales inversely with dataset quality β€” the cleanest single
77
+ read-out of "the prior imitates its targets."
78
+
79
+ Figures: `figures/fig1b_ksweep.png` (cross-task coverage vs K, + SISO coverage vs inference time).
80
+ Also reproduced in `RESULTS.md` Β§4.2 ("Cross-task K-sweep"). Raw per-task logs:
81
+ `results/siso_coherent/k_sweep.txt`, `results/mimo/k_sweep_mimo.txt`, `results/pendulum/k_sweep_pendulum.txt`.
results/regressor_siso_train.log ADDED
@@ -0,0 +1,11 @@
 
 
 
 
 
 
 
 
 
 
 
 
1
+ device cuda | N=180160 in=10 out=21
2
+ epoch 0 | val MSE 0.00175 (best 0.00175)
3
+ epoch 5 | val MSE 0.00175 (best 0.00174)
4
+ epoch 10 | val MSE 0.00174 (best 0.00174)
5
+ epoch 15 | val MSE 0.00174 (best 0.00174)
6
+ epoch 20 | val MSE 0.00174 (best 0.00174)
7
+ epoch 25 | val MSE 0.00174 (best 0.00174)
8
+ epoch 30 | val MSE 0.00174 (best 0.00174)
9
+ epoch 35 | val MSE 0.00174 (best 0.00174)
10
+ epoch 39 | val MSE 0.00174 (best 0.00174)
11
+ saved -> models/regressor_siso.pt (best val MSE 0.00174)