File size: 5,074 Bytes
20ec019
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
device=cuda  training seeds=[42, 43, 44]  eval noise seed=0
[size] hidden=64 (~1/4 width) (base_tag=size_quarter, seeds=[42, 43, 44])
  evaluating size_quarter seed=42 <- models/Best_pendulum_abl_size_quarter.pth (eval noise seed=0)
/home/cercola/matte_server/DiffusionControl/nonlinear/data_generation/pendulum_lmi.py:149: UserWarning: Solution may be inaccurate. Try another solver, adjusting the solver settings, or solve with verbose=True for more information.
  prob.solve(solver=solver, verbose=False)
  evaluating size_quarter seed=43 <- models/Best_pendulum_abl_size_quarter_s43.pth (eval noise seed=0)
  evaluating size_quarter seed=44 <- models/Best_pendulum_abl_size_quarter_s44.pth (eval noise seed=0)
[size] hidden=256 (fair reference, ablation trainer) (base_tag=size_ref, seeds=[42, 43, 44])
  evaluating size_ref seed=42 <- models/Best_pendulum_abl_size_ref.pth (eval noise seed=0)
  evaluating size_ref seed=43 <- models/Best_pendulum_abl_size_ref_s43.pth (eval noise seed=0)
  evaluating size_ref seed=44 <- models/Best_pendulum_abl_size_ref_s44.pth (eval noise seed=0)
[size] hidden=512 (~2x width) (base_tag=size_2x, seeds=[42, 43, 44])
  evaluating size_2x seed=42 <- models/Best_pendulum_abl_size_2x.pth (eval noise seed=0)
  evaluating size_2x seed=43 <- models/Best_pendulum_abl_size_2x_s43.pth (eval noise seed=0)
  evaluating size_2x seed=44 <- models/Best_pendulum_abl_size_2x_s44.pth (eval noise seed=0)
[size] hidden=256 (DEPLOYED checkpoint -- pre-existing train_pendulum.py run, not part of the 3-seed sweep) <- models/Best_pendulum_cond.pth

=== SIZE ABLATION (pendulum, N=40, K=8, shared cached oracle, eval noise seed=0, 3 training seeds=[42, 43, 44]) ===

- hidden=64 (~1/4 width)
    params=255,563  time_to_best=405.9s
    median regret vs oracle: +0.68% +/- 0.16pp over seeds  [s42=+0.58%, s43=+0.87%, s44=+0.60%]
    coverage (mean over seeds): certified>=1/plant 0.992  stable>=1/plant 1.000  per-sample stable 0.999
- hidden=256 (fair reference, ablation trainer)
    params=3,823,883  time_to_best=421.4s
    median regret vs oracle: +0.92% +/- 0.22pp over seeds  [s42=+1.06%, s43=+0.67%, s44=+1.03%]
    coverage (mean over seeds): certified>=1/plant 1.000  stable>=1/plant 1.000  per-sample stable 1.000
- hidden=512 (~2x width)
    params=15,118,859  time_to_best=441.1s
    median regret vs oracle: +0.86% +/- 0.38pp over seeds  [s42=+0.98%, s43=+0.43%, s44=+1.16%]
    coverage (mean over seeds): certified>=1/plant 1.000  stable>=1/plant 1.000  per-sample stable 1.000
- hidden=256 (DEPLOYED checkpoint -- pre-existing train_pendulum.py run, not part of the 3-seed sweep)
    params=3,823,883  time_to_best=232.0s
    median regret vs oracle: +1.15% +/- 0.00pp over seeds  [ssingle=+1.15%]
    coverage (mean over seeds): certified>=1/plant 1.000  stable>=1/plant 1.000  per-sample stable 1.000
[data] 25% data (650/2600 examples) (base_tag=data_p25, seeds=[42, 43, 44])
  evaluating data_p25 seed=42 <- models/Best_pendulum_abl_data_p25.pth (eval noise seed=0)
  evaluating data_p25 seed=43 <- models/Best_pendulum_abl_data_p25_s43.pth (eval noise seed=0)
  evaluating data_p25 seed=44 <- models/Best_pendulum_abl_data_p25_s44.pth (eval noise seed=0)
[data] 50% data (1300/2600 examples) (base_tag=data_p50, seeds=[42, 43, 44])
  evaluating data_p50 seed=42 <- models/Best_pendulum_abl_data_p50.pth (eval noise seed=0)
  evaluating data_p50 seed=43 <- models/Best_pendulum_abl_data_p50_s43.pth (eval noise seed=0)
  evaluating data_p50 seed=44 <- models/Best_pendulum_abl_data_p50_s44.pth (eval noise seed=0)
[data] 100% data (= size_ref, same 3 seeds, reused) (base_tag=size_ref, seeds=[42, 43, 44])
[data] hidden=256 (DEPLOYED checkpoint -- pre-existing train_pendulum.py run, not part of the 3-seed sweep) <- models/Best_pendulum_cond.pth

=== DATA ABLATION (pendulum, N=40, K=8, shared cached oracle, eval noise seed=0, 3 training seeds=[42, 43, 44]) ===

- 25% data (650/2600 examples)
    params=3,823,883  time_to_best=261.8s
    median regret vs oracle: +0.29% +/- 0.27pp over seeds  [s42=+0.61%, s43=+0.16%, s44=+0.12%]
    coverage (mean over seeds): certified>=1/plant 0.992  stable>=1/plant 1.000  per-sample stable 1.000
- 50% data (1300/2600 examples)
    params=3,823,883  time_to_best=355.6s
    median regret vs oracle: +1.00% +/- 0.23pp over seeds  [s42=+1.22%, s43=+0.76%, s44=+1.02%]
    coverage (mean over seeds): certified>=1/plant 1.000  stable>=1/plant 1.000  per-sample stable 1.000
- 100% data (= size_ref, same 3 seeds, reused)
    params=3,823,883  time_to_best=421.4s
    median regret vs oracle: +0.92% +/- 0.22pp over seeds  [s42=+1.06%, s43=+0.67%, s44=+1.03%]
    coverage (mean over seeds): certified>=1/plant 1.000  stable>=1/plant 1.000  per-sample stable 1.000
- hidden=256 (DEPLOYED checkpoint -- pre-existing train_pendulum.py run, not part of the 3-seed sweep)
    params=3,823,883  time_to_best=232.0s
    median regret vs oracle: +1.15% +/- 0.00pp over seeds  [ssingle=+1.15%]
    coverage (mean over seeds): certified>=1/plant 1.000  stable>=1/plant 1.000  per-sample stable 1.000