DiffusionControl / results /ablations /pendulum_data_anomaly.txt
matCercola18's picture
Add files using upload-large-folder tool
929e74d verified
Raw History Blame Contribute Delete
2.31 kB
=== PENDULUM data-size anomaly diagnostics ===
held-out: seed123 (N=40) + seed321 (N=40), pooled N=80 plants; sampler draws (0, 1, 2); K=1 = first sample of each draw; K=8 = best of 8.
regret % vs strong per-plant oracle; failures (unstable) counted as +inf in the median, never dropped.
CI = 95% bootstrap over plants (5000) of the median pooled over (training seed x draw).
TARGET native regret (random 40 training targets, seed 2026, on own plants, K=1): median +1.41 [+1.27,+1.51] p90 +3.59 max +4.09 fail 0.000
config seeds K1 median [CI] K1 p90 K1 fail K8 median [CI] sd_seed K1 sd_seed K8 d_min d_nnplant
f0.25_e800 3 +1.16 [+0.99,+1.40] 4.18 0.000 +0.28 [+0.21,+0.37] 0.25 0.22 0.015 0.063
f0.50_e800 3 +1.48 [+1.33,+1.63] 3.06 0.000 +0.90 [+0.78,+1.04] 0.27 0.22 0.005 0.044
f1.00_e800 3 +1.32 [+1.15,+1.47] 3.09 0.000 +0.91 [+0.79,+1.03] 0.23 0.22 0.004 0.035
f0.25_e3200 2 +1.43 [+1.33,+1.58] 3.14 0.000 +1.07 [+0.95,+1.18] 0.03 0.08 0.004 0.049
f1.00_e200 2 +1.43 [+1.28,+1.63] 3.42 0.000 +0.52 [+0.45,+0.61] 0.28 0.31 0.012 0.046
per-plant paired Wilcoxon (plant = mean over seed x draw of regret; same 80 plants), two-sided:
f0.25_e800 vs f1.00_e800: K1 med diff +0.05pp p=0.146 (n=80) | K8 med diff -0.49pp p=8.46e-14 (n=80)
f0.25_e3200 vs f0.25_e800: K1 med diff +0.12pp p=0.77 (n=80) | K8 med diff +0.71pp p=3.38e-14 (n=80)
f0.25_e3200 vs f1.00_e800: K1 med diff +0.12pp p=1.59e-07 (n=80) | K8 med diff +0.12pp p=5.96e-09 (n=80)
f1.00_e200 vs f1.00_e800: K1 med diff +0.15pp p=0.00533 (n=80) | K8 med diff -0.34pp p=3.5e-11 (n=80)
f1.00_e200 vs f0.25_e800: K1 med diff +0.15pp p=0.181 (n=80) | K8 med diff +0.18pp p=1.7e-06 (n=80)
f0.25_e800 vs f0.50_e800: K1 med diff -0.11pp p=0.996 (n=80) | K8 med diff -0.48pp p=1.57e-13 (n=80)
Model K=1 vs native target regret (unpaired, different plants): compare K1 median column to the TARGET line above.
d_min = distance from a generated (clipped, normalized) controller to the nearest vector in that model's own training subset; d_nnplant = distance to the target of the nearest-condition training plant.