File size: 3,065 Bytes
20ec019
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
device=cuda
/home/cercola/matte_server/DiffusionControl/nonlinear/data_generation/pendulum_lmi.py:149: UserWarning: Solution may be inaccurate. Try another solver, adjusting the solver settings, or solve with verbose=True for more information.
  prob.solve(solver=solver, verbose=False)
[gen-baseline/pendulum] draw seed=0 done
[gen-baseline/pendulum] draw seed=1 done
[gen-baseline/pendulum] draw seed=2 done

=== A3 GENERATIVE BASELINE — PENDULUM (N=40, shared cached oracle, 3 draws/K, seeds=[0, 1, 2]) ===

budget: diffusion params=3,823,883 epochs=800 time_to_best=232.0s; cvae params=277,019 epochs=800 time_to_best=278.9s

-- K=1 --
  diffusion : stable>=1 1.000+-0.000  cert>=1 0.992+-0.014  persample 1.000+-0.000  median_regret +1.52%+-0.06pp  diversity 0.0000+-0.0000
  cvae      : stable>=1 1.000+-0.000  cert>=1 1.000+-0.000  persample 1.000+-0.000  median_regret +1.37%+-0.05pp  diversity 0.0000+-0.0000
  paired McNemar (diffusion vs cvae, pooled 120 plant x draw trials): cvae wins 0/0 discordant, p=1

-- K=8 --
  diffusion : stable>=1 1.000+-0.000  cert>=1 1.000+-0.000  persample 1.000+-0.000  median_regret +1.11%+-0.09pp  diversity 0.0142+-0.0007
  cvae      : stable>=1 1.000+-0.000  cert>=1 1.000+-0.000  persample 1.000+-0.000  median_regret +1.20%+-0.02pp  diversity 0.0027+-0.0001
  paired McNemar (diffusion vs cvae, pooled 120 plant x draw trials): cvae wins 0/0 discordant, p=1

[timing] pendulum arm: 131.6s
[gen-baseline/siso] draw seed=0 done
[gen-baseline/siso] draw seed=1 done
[gen-baseline/siso] draw seed=2 done

=== A3 GENERATIVE BASELINE — SISO (N=60 plants, plant_seed=100, 3 draws/K, seeds=[0, 1, 2]) ===

budget: diffusion params=21,445,653 epochs=1200 time_to_best=7940.6s; cvae params=1,108,021 epochs=60 time_to_best=33.9s; regressor params=804,373 epochs=40 time=not recorded

-- K=1 --
  diffusion : plant_stabilisable 0.667+-0.033  persample 0.667+-0.033  median_best_J 6.713+-3.633  diversity 0.0000+-0.0000
  cvae      : plant_stabilisable 0.750+-0.050  persample 0.750+-0.050  median_best_J 5.167+-2.512  diversity 0.0000+-0.0000
  regressor : plant_stabilisable 0.750+-0.044  persample 0.750+-0.044  median_best_J 2.403+-0.141  diversity 0.0000+-0.0000
  paired McNemar (diffusion vs cvae, pooled 180 plant x draw trials): cvae wins 18/21 discordant, p=0.00149
  paired McNemar (diffusion vs regressor, pooled 180 plant x draw trials): regressor wins 19/23 discordant, p=0.0026

-- K=8 --
  diffusion : plant_stabilisable 0.800+-0.050  persample 0.640+-0.024  median_best_J 1.363+-0.120  diversity 0.1844+-0.0076
  cvae      : plant_stabilisable 0.767+-0.033  persample 0.751+-0.045  median_best_J 0.748+-0.113  diversity 0.0034+-0.0001
  regressor : plant_stabilisable 0.750+-0.044  persample 0.750+-0.044  median_best_J 2.403+-0.141  diversity 0.0000+-0.0000
  paired McNemar (diffusion vs cvae, pooled 180 plant x draw trials): cvae wins 0/6 discordant, p=0.0312
  paired McNemar (diffusion vs regressor, pooled 180 plant x draw trials): regressor wins 0/9 discordant, p=0.00391

[timing] siso arm: 488.6s