File size: 3,216 Bytes
68a167f
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45

==============================================================================
PENDULUM second held-out draw (seed=321)
==============================================================================
[pendulum plants (2nd draw)] seeds: 321
[pendulum diffusion] seeds: (0, 1, 2)
  [oracle] 10/40 plants done (24.6 s/plant avg)
  [oracle] 20/40 plants done (27.2 s/plant avg)
  [oracle] 30/40 plants done (27.0 s/plant avg)
  [oracle] 40/40 plants done (27.4 s/plant avg)
[oracle] 40 plants in 1095s (27.4 s/plant)
[oracle cached] -> results/baselines/pendulum_oracle_seed321.npz
oracle_J finite for 40/40 plants
diffusion generation: 35.43 ms/controller (mean over 3 seeds)
=== PENDULUM retrieval/interpolation baselines (N=40, seed=321) ===
verifier: certify_controller (regional LMI) + simulate_pendulum_batch rollout (n_ic=3, dt=0.02, duration=6.0, energy_thresh=0.02)
oracle: FRESH (not previously cached): eval_pendulum.strong_oracle (24 restarts x 100 steps), computed this run and cached to results/baselines/pendulum_oracle_seed321.npz
K_std=8, diffusion seeds=(0, 1, 2)

--- K=8 ---
arm             cert_any  stable_any   persamp  reg_med%  reg_p90%  reg_max%  ms/plant   McNemar(n10,n01,p)    Wilcoxon(n,p)
diffusion           1.00        1.00      1.00      0.97      2.21      3.17    283.48          (reference)      (reference)
  (sd across 3 seeds: cert_any 0.000, stable_any 0.000, per-samp 0.000)
  (seed[0] 95% CI: stable_any [1.00,1.00] regret_med [+0.75,+1.31]%)
retrieval           1.00        1.00      1.00      0.29      0.64      0.86     0.373              (0,0,1)       (40,1e-07)
  (Wilcoxon n=40 = plants where BOTH this arm and diffusion have a finite cost, i.e. the COMMON success set, not either arm's own coverage count)
  (95% CI: stable_any [1.00,1.00] regret_med [+0.16,+0.36]%)
interpolation       1.00        1.00      1.00      0.30      0.67      2.01     0.373              (0,0,1)       (40,1e-07)
  (Wilcoxon n=40 = plants where BOTH this arm and diffusion have a finite cost, i.e. the COMMON success set, not either arm's own coverage count)
  (95% CI: stable_any [1.00,1.00] regret_med [+0.16,+0.42]%)

--- K=1 ---
arm             cert_any  stable_any   persamp  reg_med%  reg_p90%  reg_max%  ms/plant   McNemar(n10,n01,p)    Wilcoxon(n,p)
diffusion           1.00        1.00      1.00      1.55      3.37      5.05    283.48          (reference)      (reference)
  (sd across 3 seeds: cert_any 0.000, stable_any 0.000, per-samp 0.000)
  (seed[0] 95% CI: stable_any [1.00,1.00] regret_med [+1.44,+1.94]%)
retrieval           1.00        1.00      1.00      1.36      2.68      5.20     0.128              (0,0,1)       (40,0.055)
  (Wilcoxon n=40 = plants where BOTH this arm and diffusion have a finite cost, i.e. the COMMON success set, not either arm's own coverage count)
  (95% CI: stable_any [1.00,1.00] regret_med [+1.14,+1.81]%)
interpolation       1.00        1.00      1.00      1.36      2.68      5.20     0.128              (0,0,1)       (40,0.055)
  (Wilcoxon n=40 = plants where BOTH this arm and diffusion have a finite cost, i.e. the COMMON success set, not either arm's own coverage count)
  (95% CI: stable_any [1.00,1.00] regret_med [+1.14,+1.81]%)