diff --git a/results/icml_r3/X28/PREREG.md b/results/icml_r3/X28/PREREG.md index 3ebbfcf9fc4ecaf29b4c45be47c51a825efb3972..35912caa9b36d025bdbdcbe090e4ab9684f3012d 100644 --- a/results/icml_r3/X28/PREREG.md +++ b/results/icml_r3/X28/PREREG.md @@ -181,3 +181,46 @@ Written after the S3 validation runs (`eval_val_report.txt`) and before any test - **Paired test (section 1.5).** Seed j of clS against seed j of clL, plant-paired, at b = 16, regret against J* with uncertified = 1e3 (`x21_paired.py` conventions): median per-plant difference clL - clS with bootstrap CI and Wilcoxon signed-rank p, Holm over the 3 seed pairs; certified rate difference with exact McNemar p. A verdict ("spline better" or "spline worse", section 1.5 thresholds) holds iff it holds on at least 2 of the 3 seed pairs. All b = 1, 4, 16 (64 where present) are reported. - **Fresh-direction diagnostic** (addendum 2026-11-06): each arm's best certified candidate at b = 16 per test plant, scored on 64 unit directions at s (seed (7474, i)), against clL seed 0's; non-convergent rollouts counted per arm. - Compute: the CPU node is down, so this runs on idle `gpu-light` node 02 (Matteo's 2026-11-06 fallback). + +## Addendum 2026-11-13: MIMO spline-class study (Matteo: "choose the best one ... let's do a study on which one to use") + +Written after the MIMO held-out gate (`gate_heldout_m{2,3}.txt`, `fresh_heldout_m{2,3}.txt`) and before any number below exists. Held-out reading so far: free piecewise-linear knots pay at the optimum (median d 5.19% at m = 3, 6.36% at m = 2) but lose on fresh directions (-4.55%, -1.58%); odd piecewise-linear knots pay (4.36%, 5.52%) and keep a fresh-direction gain (+0.54% [0.20, 1.27], +2.71% [1.88, 3.39]). Matteo asked for the spline class to be chosen by a study. The study runs on validation plants only; the held-out plants then confirm the chosen class. + +**Classes** (all with knot ratios r = psi(x)/x in [0.1, 10] at x in {-5, ..., -1, 1, ..., 5}, ray extension past +-5, sector exactly [min r, max r] per channel, circle LMI of section 2.1): +- `sp` piecewise linear, free (10 per channel), the July class; +- `so` piecewise linear, odd (5 per channel); +- `so3` piecewise linear, odd, knots at |x| = 1, 3, 5 only (3 per channel; psi linear in between); +- `cf` cubic, free: psi(v) = v rho(v), rho the Fritsch-Carlson PCHIP interpolant of the 10 ratios (end slopes = end secants). PCHIP stays between its two end values on every interval, so the sector is exact (checked on a grid of 4001 points); +- `co` cubic, odd (5 per channel). +Baseline: `id` (identity knots). + +**Run.** The first 64 bank plants for m = 3 (`D2_scale/m3_bank.npz`) and m = 2 (`C_mimo/sof/bank.npz`), the K learning rate of the gate (0.02), knot lr 0.02, 400 steps, gauge starts c = 1, 2, 5 for every class, all starts of all classes in one Adam run, cheapest certified snapshot (`x28_mimo.py`, tag `study`). Fresh directions: 64 random unit directions in R^(2m) per plant, both signs, seed (7676, plant id) (distinct from the held-out seed 7575, because validation and held-out ids overlap). + +**Selection rule.** For class c and size m, f_c(m) = median over plants of the per-plant median over the 64 fresh directions of (J_id - J_c) / J_c (`stage_fresh`, both rollouts convergent). Eligible: the 95% bootstrap CI lower bound of f_c(m) is > 0 at both m, and the non-convergent fresh rollouts of c exceed those of `id` by at most 1% of the rollouts (41 of 4,096) at both m. Winner: the eligible class with the highest mean of f_c over m = 2, 3. Parsimony: among eligible classes within 0.25 points of the winner's mean, the one with the fewest parameters per channel (ties: piecewise linear). The gate number d at the optimum is reported for every class but does not select. If no class is eligible, the MIMO generator arm (b) waits for Matteo. + +**Confirmation.** The winner on the 120 held-out plants per m with the gate rule of 2.3 and the fresh-direction reading rule of addendum 2026-11-06 (seed 7575). If the winner is `so`, that held-out result already exists and was seen before this study (above). The winner becomes the spline class of MIMO arm (b) (sections 2.2 and 2.4); labels and generators follow in a later addendum. + +SISO S4 continues as registered (free piecewise-linear knots). The SISO odd-knot gate gave 0.00%, so the class study is run on MIMO, where a gain exists. + +Compute: inside Matteo's GPU hold job on node 05 (he allowed its use on 2026-11-12), with the SISO S4 job on node 02: 2 GPU nodes. + +## Addendum 2026-11-13: MIMO study result, labels, generators and evaluation (sections 2.2 to 2.4) + +Written after the study (`study_choice.txt`) and before the held-out so3 confirmation, any MIMO label, model or generator result. + +**Study result** (`study_choice.txt`, 64 validation plants per m). Fresh-direction gain f over identity: `so3` +3.99% [2.86, 5.13] at m = 2 and +1.82% [0.74, 2.57] at m = 3; `so` +3.11%, +1.32%; `co` +2.75%, +0.91%; free classes negative at both m (`sp` -2.11%, -5.94%; `cf` -3.09%, -6.06%) with 180 to 188 (`sp`) and 13 to 43 (`cf`) more non-convergent rollouts than identity. Choice by the rule: **so3**, odd piecewise linear with knots at |x| = 1, 3, 5. The held-out confirmation runs `so3` (starts c = 1, 2, 5) in its own Adam call next to the stored `id`, `sp`, `so` runs; starts never interact, so this equals one joint call. + +**Labels (S2-M).** Bank plants without the 64 validation plants: m = 3 `D2_scale/m3_bank.npz` [64, 8000) (7,936 plants), m = 2 `C_mimo/sof/bank.npz` [64, 4000) (3,936 plants). Optimizer of the gate: `id` at c = 1 and `so3` at c = 1, 2, 5, K lr 0.02, knot lr 0.02, 400 steps, snapshot every 25, cheapest certified snapshot (`x28_mimo.py` optimize and certify, tag `lab`), with gradient checkpointing of the rollout (`X28M_SEG=100`, identical values and gradients, checked). Label (a): the `id` arm's controller K. Label (b): the cheapest certified of the three `so3` starts and `id` (identity is in the so3 class, all ratios 1). Snapshot 0 is the bank gain, certified for identity, so every plant has both labels. + +**Generators (S3-M).** One row per plant per arm. Conditioning: plant parameters scaled to [-1, 1] by the family box (as X27). Target (a): K (m^2 numbers). Target (b): K and the knots in a fixed gauge: per channel i, K row i times r_i(1) and ratios divided by r_i(1), so r_i(1) = 1 and the controller is unchanged; target = K (m^2) and log r_i(3), log r_i(5) (2m). Targets min-max scaled to [-0.96, 0.96] (`make_dataset_sofm` convention). Trainer: X27's `train.py` (DDPM-500, MLP 256 x 5, 96,000 steps, batch 64, lr 2e-4, EMA 0.999), seeds 0, 1, 2 per arm and m. No checkpoint selection: the final EMA 0.999 weights. + +**Evaluation (S4-M).** The 120 held-out plants per m (`m{m}_heldout_{4242,9001}`). 16 samples per plant, DDPM with clip range 1 (as X27 `sample.py`), sampler seed = model seed. Decode, then for (b) project the ratios to [0.1, 10]. A sample counts iff certified: (a) the Lyapunov LMI of A - B K C at alpha = beta = 1 (as the `id` arm); (b) the circle LMI of section 2.1 at its sector. Cost J: 2m positive unit directions, 20 s, RK4 dt 0.01 (the label cost). J* = min over all held-out labels (`id`, `sp`, `so`, `so3`). Regret at b = 1, 4, 16 with uncertified = 1e3. +Primary, per m: (b) seed j against (a) seed j at b = 16, plant-paired; "spline better" iff median paired regret difference (a) - (b) >= 1 point, Holm (over the 3 seed pairs) Wilcoxon p < 0.05, and certified rate of (b) not lower by more than 2 points; "spline worse" with the arms swapped; verdict iff on >= 2 of 3 pairs, otherwise "no measurable difference" (the SISO rule of 1.5). +Diagnostic: each arm's best certified b = 16 sample per plant on the 64 fresh directions of seed (7575, plant id), against (a) seed 0, as in the gate. + +Compute: inside GPL_move's job 17025 on node 03 (Matteo 2026-11-13: share that session's node), within its 12 GB VRAM and 3 CPU budget. + +## Addendum 2026-11-13: correction to the arm (b) target + +Written before any MIMO label run, dataset or model exists, after a round-trip check of the code on the study's validation labels. The gauge of the previous addendum (K row i times r_i(1), ratios divided by r_i(1)) is not a symmetry of the so3 class: the knots sit at fixed positions, and a row scaling c is only matched by psi_i(w / c), whose knots move to c x_k. On the 64 validation labels the round trip changed J by up to 72% (m = 2) and 225% (m = 3); without it the round trip is exact (3e-16). The gauge starts c = 1, 2, 5 of the optimizer stay valid, because they start from a linear psi, for which the scaling is exact. +Target of arm (b), replacing the previous one: K (m^2) and log r_i(1), log r_i(3), log r_i(5) per channel (3m), each ratio projected to [0.1, 10] after decoding. Nothing else changes. diff --git a/results/icml_r3/X28/eval_test/x21_fgen_clL_best_tr_K16_clip1.5.npz b/results/icml_r3/X28/eval_test/x21_fgen_clL_best_tr_K16_clip1.5.npz new file mode 100644 index 0000000000000000000000000000000000000000..2b7d416936f9ca1b23415ad7e634ffac6fc06b2f --- /dev/null +++ b/results/icml_r3/X28/eval_test/x21_fgen_clL_best_tr_K16_clip1.5.npz @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:8c08bb2cb874310cf82830ac8d387a8e15f509f82f81f3f969fbadc1017540c9 +size 283098 diff --git a/results/icml_r3/X28/eval_test/x21_fgen_clL_best_tr_K64_clip1.5.npz b/results/icml_r3/X28/eval_test/x21_fgen_clL_best_tr_K64_clip1.5.npz new file mode 100644 index 0000000000000000000000000000000000000000..221da5655b6ad76fcb75b50a61ffb8216a030fff --- /dev/null +++ b/results/icml_r3/X28/eval_test/x21_fgen_clL_best_tr_K64_clip1.5.npz @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:801b6a9a8313ed566bd74b1a3fab6710789b72a9008f2fabdeb04e7a8b700df2 +size 1000130 diff --git a/results/icml_r3/X28/eval_test/x21_fgen_clL_s1_best_tr_K16_clip1.5.npz b/results/icml_r3/X28/eval_test/x21_fgen_clL_s1_best_tr_K16_clip1.5.npz new file mode 100644 index 0000000000000000000000000000000000000000..9701b998d626346f58c5bcb88979794e1899d477 --- /dev/null +++ b/results/icml_r3/X28/eval_test/x21_fgen_clL_s1_best_tr_K16_clip1.5.npz @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:144a92e411113600c40b95c7dbf6aa9b1f13c3d8874a41dab783dc4cc6d50d10 +size 282911 diff --git a/results/icml_r3/X28/eval_test/x21_fgen_clL_s2_best_tr_K16_clip1.5.npz b/results/icml_r3/X28/eval_test/x21_fgen_clL_s2_best_tr_K16_clip1.5.npz new file mode 100644 index 0000000000000000000000000000000000000000..349428a47a805bfcbaf7c6fe156584a7dd401f0a --- /dev/null +++ b/results/icml_r3/X28/eval_test/x21_fgen_clL_s2_best_tr_K16_clip1.5.npz @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:b1ec5ec6cc977ff1960ba830e4f70a9867faf493ef8300babca9d4ea023d3eee +size 283059 diff --git a/results/icml_r3/X28/eval_test/x28_clS_s0_best_K16_clip1.npz b/results/icml_r3/X28/eval_test/x28_clS_s0_best_K16_clip1.npz new file mode 100644 index 0000000000000000000000000000000000000000..a6c47fc252b7c6f8f11c27f37c5dfd091b829995 --- /dev/null +++ b/results/icml_r3/X28/eval_test/x28_clS_s0_best_K16_clip1.npz @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c7ae2df412a230999495c6874507edcbd7f90d31337d411a19d26135ee89936d +size 406038 diff --git a/results/icml_r3/X28/eval_test/x28_clS_s0_best_tr_K16_clip1.5.npz b/results/icml_r3/X28/eval_test/x28_clS_s0_best_tr_K16_clip1.5.npz new file mode 100644 index 0000000000000000000000000000000000000000..5d068bec6d03a5ad5a0709ef1d7401688e777d41 --- /dev/null +++ b/results/icml_r3/X28/eval_test/x28_clS_s0_best_tr_K16_clip1.5.npz @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:6a8ee07d59e0ae3d5ad03767c388c221312b88e53af75407c64e92dc3e833f6e +size 408708 diff --git a/results/icml_r3/X28/eval_test/x28_clS_s0_best_tr_K64_clip1.5.npz b/results/icml_r3/X28/eval_test/x28_clS_s0_best_tr_K64_clip1.5.npz new file mode 100644 index 0000000000000000000000000000000000000000..a832557ad1b05657e67c04c58b5ae6f516fef233 --- /dev/null +++ b/results/icml_r3/X28/eval_test/x28_clS_s0_best_tr_K64_clip1.5.npz @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d7010fc5e83dcd362d8b7528a4573d66aa4186c2b7fe0199153968228a6f1b99 +size 1501865 diff --git a/results/icml_r3/X28/eval_test/x28_clS_s1_best_K16_clip1.5.npz b/results/icml_r3/X28/eval_test/x28_clS_s1_best_K16_clip1.5.npz new file mode 100644 index 0000000000000000000000000000000000000000..4dcc7db380f3fdcbffa1c99882c01f374d95eeeb --- /dev/null +++ b/results/icml_r3/X28/eval_test/x28_clS_s1_best_K16_clip1.5.npz @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:9ecb26bbfc89abae03cc98bd1a99199e26f6f8f2cd47ab572e4a9032ad6cb9aa +size 406750 diff --git a/results/icml_r3/X28/eval_test/x28_clS_s1_best_tr_K16_clip1.5.npz b/results/icml_r3/X28/eval_test/x28_clS_s1_best_tr_K16_clip1.5.npz new file mode 100644 index 0000000000000000000000000000000000000000..739727f1cf47b6b55536078ad1da507e641d50d4 --- /dev/null +++ b/results/icml_r3/X28/eval_test/x28_clS_s1_best_tr_K16_clip1.5.npz @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:dcb17bdf400b35ede7c9c6224305a566ed9663d53dcaad680e1575954cd20071 +size 410425 diff --git a/results/icml_r3/X28/eval_test/x28_clS_s2_best_K16_clip1.npz b/results/icml_r3/X28/eval_test/x28_clS_s2_best_K16_clip1.npz new file mode 100644 index 0000000000000000000000000000000000000000..a1175742d7f4efe767ce9648c59a95367e2eb3f3 --- /dev/null +++ b/results/icml_r3/X28/eval_test/x28_clS_s2_best_K16_clip1.npz @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:92be1ce6a89259cc00acea1e30544976146a05de70c9578bb9fefb9992966679 +size 406592 diff --git a/results/icml_r3/X28/eval_test/x28_clS_s2_best_tr_K16_clip1.5.npz b/results/icml_r3/X28/eval_test/x28_clS_s2_best_tr_K16_clip1.5.npz new file mode 100644 index 0000000000000000000000000000000000000000..d788ce3b4fb0ae1cf526fa475023e4a846366d1d --- /dev/null +++ b/results/icml_r3/X28/eval_test/x28_clS_s2_best_tr_K16_clip1.5.npz @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:0c51c3ccdafc56f271d46e306c92435995c1f05e27575c47e89f14568dc03212 +size 409386 diff --git a/results/icml_r3/X28/eval_test/x28_coefS_s0_best_tr_K16_clip3.npz b/results/icml_r3/X28/eval_test/x28_coefS_s0_best_tr_K16_clip3.npz new file mode 100644 index 0000000000000000000000000000000000000000..89f489f31b3aa09d755a7cc5492af0e0ae5466a0 --- /dev/null +++ b/results/icml_r3/X28/eval_test/x28_coefS_s0_best_tr_K16_clip3.npz @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:66bd5cfa12c56f42cb22b277db1e7f2919340ea6f3fea65b6a25ec049b64afe6 +size 408282 diff --git a/results/icml_r3/X28/eval_test_fresh.npz b/results/icml_r3/X28/eval_test_fresh.npz new file mode 100644 index 0000000000000000000000000000000000000000..3d1cdea2007013a9ec7372d224c290b210861baf --- /dev/null +++ b/results/icml_r3/X28/eval_test_fresh.npz @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f57f48c9a3f3be96ead682ee9a999054c1083a76cb0df103db7cdbd598013e5d +size 1319384 diff --git a/results/icml_r3/X28/eval_test_fresh.txt b/results/icml_r3/X28/eval_test_fresh.txt new file mode 100644 index 0000000000000000000000000000000000000000..0a74e5f18bbbf67de14895e2830e05f7bc7a6bb6 --- /dev/null +++ b/results/icml_r3/X28/eval_test_fresh.txt @@ -0,0 +1,8 @@ +== S4 fresh directions: best certified b=16 candidate per test plant on 64 unit directions at s (seed (7474, i)), vs x21_fgen_clL_best_tr_K16_clip1.5 + x21_fgen_clL_best_tr_K16_clip1.5 plants 196 | median over plants of per-plant median (J_clL0 - J_arm) / J_arm +0.00% [+0.00, +0.00], > 0 on 0 | non-convergent rollouts 626 of 12544 + x28_clS_s0_best_tr_K16_clip1.5 plants 188 | median over plants of per-plant median (J_clL0 - J_arm) / J_arm -1.41% [-2.40, -0.30], > 0 on 74 | non-convergent rollouts 1160 of 12096 + x28_clS_s1_best_tr_K16_clip1.5 plants 190 | median over plants of per-plant median (J_clL0 - J_arm) / J_arm -1.50% [-2.68, -0.30], > 0 on 77 | non-convergent rollouts 1775 of 12288 + x28_clS_s2_best_tr_K16_clip1.5 plants 190 | median over plants of per-plant median (J_clL0 - J_arm) / J_arm -1.31% [-2.14, -0.30], > 0 on 80 | non-convergent rollouts 1189 of 12160 + x28_coefS_s0_best_tr_K16_clip3 plants 186 | median over plants of per-plant median (J_clL0 - J_arm) / J_arm -4.44% [-5.77, -2.89], > 0 on 47 | non-convergent rollouts 2089 of 12096 + x21_fgen_clL_s1_best_tr_K16_clip1.5 plants 193 | median over plants of per-plant median (J_clL0 - J_arm) / J_arm +0.22% [-0.09, +0.53], > 0 on 107 | non-convergent rollouts 568 of 12480 + x21_fgen_clL_s2_best_tr_K16_clip1.5 plants 193 | median over plants of per-plant median (J_clL0 - J_arm) / J_arm +0.23% [-0.08, +0.57], > 0 on 107 | non-convergent rollouts 1154 of 12416 diff --git a/results/icml_r3/X28/eval_test_paired.txt b/results/icml_r3/X28/eval_test_paired.txt new file mode 100644 index 0000000000000000000000000000000000000000..ca23890c8e0a2ce2e096f71f05f507454fd00a5e --- /dev/null +++ b/results/icml_r3/X28/eval_test_paired.txt @@ -0,0 +1,33 @@ +== PRIMARY: clS (t-repair) vs clL (t-repair), seed j vs seed j; Holm over the seed pairs of each b + x28_clS_s0_best_tr_K16_clip1.5 vs x21_fgen_clL_best_tr_K16_clip1.5 b= 1 | regret 100000.00% vs 54.90% | clL - spline median +0.00 pts [-10.29, +0.00] p_holm 0.001 | certified 49.2% vs 65.3%, diff -16.1 [-24.6, -7.5] McNemar p 0.00045 | spline cheaper on 29.1%, dearer on 48.2% + x28_clS_s1_best_tr_K16_clip1.5 vs x21_fgen_clL_s1_best_tr_K16_clip1.5 b= 1 | regret 135.35% vs 49.63% | clL - spline median +0.00 pts [-1.69, +0.00] p_holm 0.0063 | certified 52.8% vs 65.3%, diff -12.6 [-21.1, -4.5] McNemar p 0.0059 | spline cheaper on 32.7%, dearer on 45.7% + x28_clS_s2_best_tr_K16_clip1.5 vs x21_fgen_clL_s2_best_tr_K16_clip1.5 b= 1 | regret 100000.00% vs 42.49% | clL - spline median +0.00 pts [-3.61, +0.00] p_holm 0.0029 | certified 49.7% vs 61.3%, diff -11.6 [-19.6, -3.0] McNemar p 0.0095 | spline cheaper on 28.1%, dearer on 45.7% + x28_clS_s0_best_tr_K16_clip1.5 vs x21_fgen_clL_best_tr_K16_clip1.5 b= 4 | regret 16.47% vs 12.85% | clL - spline median -1.25 pts [-3.69, +0.00] p_holm 0.0055 | certified 85.9% vs 92.0%, diff -6.0 [-10.6, -1.0] McNemar p 0.023 | spline cheaper on 41.7%, dearer on 53.3% + x28_clS_s1_best_tr_K16_clip1.5 vs x21_fgen_clL_s1_best_tr_K16_clip1.5 b= 4 | regret 15.09% vs 13.25% | clL - spline median -1.12 pts [-2.08, +0.00] p_holm 0.0055 | certified 85.4% vs 94.0%, diff -8.5 [-14.1, -3.5] McNemar p 0.0023 | spline cheaper on 41.2%, dearer on 55.8% + x28_clS_s2_best_tr_K16_clip1.5 vs x21_fgen_clL_s2_best_tr_K16_clip1.5 b= 4 | regret 19.78% vs 11.93% | clL - spline median -1.27 pts [-4.65, +0.00] p_holm 0.0027 | certified 83.4% vs 89.9%, diff -6.5 [-12.1, -1.0] McNemar p 0.035 | spline cheaper on 39.7%, dearer on 55.3% + x28_clS_s0_best_tr_K16_clip1.5 vs x21_fgen_clL_best_tr_K16_clip1.5 b=16 | regret 7.14% vs 6.79% | clL - spline median -0.04 pts [-0.61, +0.48] p_holm 1 | certified 95.0% vs 98.5%, diff -3.5 [-6.5, -1.0] McNemar p 0.039 | spline cheaper on 48.7%, dearer on 50.3% + x28_clS_s1_best_tr_K16_clip1.5 vs x21_fgen_clL_s1_best_tr_K16_clip1.5 b=16 | regret 6.33% vs 6.62% | clL - spline median +0.19 pts [-0.42, +0.91] p_holm 1 | certified 96.5% vs 98.0%, diff -1.5 [-4.5, +1.5] McNemar p 0.51 | spline cheaper on 50.8%, dearer on 48.7% + x28_clS_s2_best_tr_K16_clip1.5 vs x21_fgen_clL_s2_best_tr_K16_clip1.5 b=16 | regret 6.30% vs 6.81% | clL - spline median -0.32 pts [-1.19, +0.19] p_holm 1 | certified 95.5% vs 97.5%, diff -2.0 [-4.5, +0.0] McNemar p 0.22 | spline cheaper on 45.2%, dearer on 52.8% + VERDICT b=16 (3 seed pairs): better on 0, worse on 0 -> no measurable difference +== secondary: clS without t-repair vs clL + x28_clS_s0_best_K16_clip1 vs x21_fgen_clL_best_tr_K16_clip1.5 b= 1 | regret 100000.00% vs 54.90% | clL - spline median -4.71 pts [-60.06, +0.00] p_holm 5.4e-06 | certified 43.7% vs 65.3%, diff -21.6 [-30.7, -12.1] McNemar p 1.5e-05 | spline cheaper on 25.6%, dearer on 53.3% + x28_clS_s1_best_K16_clip1.5 vs x21_fgen_clL_s1_best_tr_K16_clip1.5 b= 1 | regret 100000.00% vs 49.63% | clL - spline median -1.81 pts [-35.18, +0.00] p_holm 6.1e-08 | certified 35.7% vs 65.3%, diff -29.6 [-38.2, -21.1] McNemar p 4.4e-10 | spline cheaper on 21.6%, dearer on 52.3% + x28_clS_s2_best_K16_clip1 vs x21_fgen_clL_s2_best_tr_K16_clip1.5 b= 1 | regret 100000.00% vs 42.49% | clL - spline median +0.00 pts [-17.29, +0.00] p_holm 0.00015 | certified 44.2% vs 61.3%, diff -17.1 [-26.1, -8.0] McNemar p 0.00037 | spline cheaper on 27.6%, dearer on 47.2% + x28_clS_s0_best_K16_clip1 vs x21_fgen_clL_best_tr_K16_clip1.5 b= 4 | regret 26.65% vs 12.85% | clL - spline median -7.51 pts [-12.91, -3.98] p_holm 1.5e-10 | certified 82.4% vs 92.0%, diff -9.5 [-15.1, -4.0] McNemar p 0.0013 | spline cheaper on 26.1%, dearer on 69.3% + x28_clS_s1_best_K16_clip1.5 vs x21_fgen_clL_s1_best_tr_K16_clip1.5 b= 4 | regret 24.18% vs 13.25% | clL - spline median -5.23 pts [-10.55, -1.82] p_holm 4.6e-09 | certified 74.4% vs 94.0%, diff -19.6 [-26.1, -13.1] McNemar p 1.8e-08 | spline cheaper on 30.2%, dearer on 66.8% + x28_clS_s2_best_K16_clip1 vs x21_fgen_clL_s2_best_tr_K16_clip1.5 b= 4 | regret 33.60% vs 11.93% | clL - spline median -5.95 pts [-13.14, -2.90] p_holm 6.2e-10 | certified 79.9% vs 89.9%, diff -10.1 [-16.1, -4.0] McNemar p 0.0017 | spline cheaper on 26.1%, dearer on 68.3% + x28_clS_s0_best_K16_clip1 vs x21_fgen_clL_best_tr_K16_clip1.5 b=16 | regret 10.57% vs 6.79% | clL - spline median -2.92 pts [-4.67, -1.72] p_holm 3.8e-09 | certified 93.5% vs 98.5%, diff -5.0 [-8.5, -2.0] McNemar p 0.0063 | spline cheaper on 27.6%, dearer on 71.4% + x28_clS_s1_best_K16_clip1.5 vs x21_fgen_clL_s1_best_tr_K16_clip1.5 b=16 | regret 7.37% vs 6.62% | clL - spline median -0.75 pts [-1.67, -0.00] p_holm 0.0014 | certified 91.0% vs 98.0%, diff -7.0 [-11.1, -3.5] McNemar p 0.00052 | spline cheaper on 41.2%, dearer on 57.3% + x28_clS_s2_best_K16_clip1 vs x21_fgen_clL_s2_best_tr_K16_clip1.5 b=16 | regret 10.28% vs 6.81% | clL - spline median -2.85 pts [-4.08, -1.62] p_holm 1.5e-08 | certified 94.0% vs 97.5%, diff -3.5 [-6.5, -1.0] McNemar p 0.039 | spline cheaper on 29.6%, dearer on 68.3% +== secondary: coefS seed 0 (t-repair) vs clL seed 0 + x28_coefS_s0_best_tr_K16_clip3 vs x21_fgen_clL_best_tr_K16_clip1.5 b= 1 | regret 100000.00% vs 54.90% | clL - spline median -6.54 pts [-18.65, +0.00] p_holm 0.0012 | certified 48.7% vs 65.3%, diff -16.6 [-26.6, -6.5] McNemar p 0.002 | spline cheaper on 29.1%, dearer on 55.3% + x28_coefS_s0_best_tr_K16_clip3 vs x21_fgen_clL_best_tr_K16_clip1.5 b= 4 | regret 21.51% vs 12.85% | clL - spline median -8.35 pts [-12.66, -5.04] p_holm 2.4e-08 | certified 83.4% vs 92.0%, diff -8.5 [-14.6, -2.0] McNemar p 0.012 | spline cheaper on 30.7%, dearer on 67.3% + x28_coefS_s0_best_tr_K16_clip3 vs x21_fgen_clL_best_tr_K16_clip1.5 b=16 | regret 10.58% vs 6.79% | clL - spline median -3.23 pts [-5.46, -2.24] p_holm 4.2e-10 | certified 95.0% vs 98.5%, diff -3.5 [-7.0, +0.0] McNemar p 0.092 | spline cheaper on 28.1%, dearer on 71.9% +== secondary: K = 64, seed 0 + x28_clS_s0_best_tr_K64_clip1.5 vs x21_fgen_clL_best_tr_K64_clip1.5 b= 1 | regret 228.92% vs 53.11% | clL - spline median -0.24 pts [-8.79, +0.00] p_holm 0.0068 | certified 51.8% vs 67.8%, diff -16.1 [-25.6, -7.0] McNemar p 0.0013 | spline cheaper on 33.2%, dearer on 50.3% + x28_clS_s0_best_tr_K64_clip1.5 vs x21_fgen_clL_best_tr_K64_clip1.5 b= 4 | regret 14.50% vs 12.85% | clL - spline median -0.81 pts [-2.51, +0.00] p_holm 0.066 | certified 88.4% vs 91.0%, diff -2.5 [-6.5, +1.5] McNemar p 0.33 | spline cheaper on 41.7%, dearer on 52.3% + x28_clS_s0_best_tr_K64_clip1.5 vs x21_fgen_clL_best_tr_K64_clip1.5 b=16 | regret 6.61% vs 6.60% | clL - spline median -0.03 pts [-0.80, +1.00] p_holm 0.95 | certified 96.0% vs 97.5%, diff -1.5 [-3.5, +0.5] McNemar p 0.38 | spline cheaper on 47.7%, dearer on 50.3% + x28_clS_s0_best_tr_K64_clip1.5 vs x21_fgen_clL_best_tr_K64_clip1.5 b=64 | regret 2.96% vs 4.07% | clL - spline median +0.44 pts [+0.16, +0.91] p_holm 7.5e-05 | certified 99.0% vs 98.5%, diff +0.5 [+0.0, +1.5] McNemar p 1 | spline cheaper on 62.3%, dearer on 36.7% + knots of the best b=16 candidate, x28_clS_s0_best_tr_K16_clip1.5: max |phi - sat_1| median 0.086, > 0.05 on 66.8% of plants, max |phi(x) + phi(-x)| median 0.066 + knots of the best b=16 candidate, x28_clS_s1_best_tr_K16_clip1.5: max |phi - sat_1| median 0.083, > 0.05 on 62.8% of plants, max |phi(x) + phi(-x)| median 0.064 + knots of the best b=16 candidate, x28_clS_s2_best_tr_K16_clip1.5: max |phi - sat_1| median 0.077, > 0.05 on 60.3% of plants, max |phi(x) + phi(-x)| median 0.055 diff --git a/results/icml_r3/X28/eval_test_report.txt b/results/icml_r3/X28/eval_test_report.txt new file mode 100644 index 0000000000000000000000000000000000000000..749e3e1e2717510e9ba4a4e992bfc2d5b208224e --- /dev/null +++ b/results/icml_r3/X28/eval_test_report.txt @@ -0,0 +1,62 @@ +== x28_clS_s0_best_tr_K16_clip1.5 (step 24000), 199 plants; sampling 5 s, scoring 944 s + certified per candidate 52.2%; candidates with knots off sat_1 100.0% + b= 1: certified 49.2% | regret vs J* median 100000.00% [91.98, 100000.00] | vs J_Fonly 100000.00% [79.81, 100000.00] | beats tuned linear 23.1% + b= 4: certified 85.9% | regret vs J* median 16.47% [12.15, 20.50] | vs J_Fonly 9.81% [6.84, 13.43] | beats tuned linear 60.3% + b=16: certified 95.0% | regret vs J* median 7.14% [5.60, 8.64] | vs J_Fonly 2.31% [1.60, 3.33] | beats tuned linear 86.9% +== x28_clS_s1_best_tr_K16_clip1.5 (step 26000), 199 plants; sampling 5 s, scoring 939 s + certified per candidate 52.1%; candidates with knots off sat_1 100.0% + b= 1: certified 52.8% | regret vs J* median 135.35% [62.33, 100000.00] | vs J_Fonly 123.46% [49.75, 100000.00] | beats tuned linear 31.2% + b= 4: certified 85.4% | regret vs J* median 15.09% [11.08, 20.07] | vs J_Fonly 9.06% [6.10, 12.85] | beats tuned linear 60.8% + b=16: certified 96.5% | regret vs J* median 6.33% [5.00, 7.42] | vs J_Fonly 2.07% [1.38, 2.84] | beats tuned linear 83.4% +== x28_clS_s2_best_tr_K16_clip1.5 (step 24000), 199 plants; sampling 5 s, scoring 898 s + certified per candidate 52.7%; candidates with knots off sat_1 100.0% + b= 1: certified 49.7% | regret vs J* median 100000.00% [92.40, 100000.00] | vs J_Fonly 100000.00% [82.90, 100000.00] | beats tuned linear 26.6% + b= 4: certified 83.4% | regret vs J* median 19.78% [14.31, 24.44] | vs J_Fonly 11.97% [8.56, 15.73] | beats tuned linear 57.3% + b=16: certified 95.5% | regret vs J* median 6.30% [5.67, 7.26] | vs J_Fonly 2.34% [1.70, 3.23] | beats tuned linear 84.4% +== x21_fgen_clL_best_tr_K16_clip1.5 (step 26000), 199 plants; sampling 6 s, scoring 407 s + certified per candidate 65.9%; candidates with knots off sat_1 0.0% + b= 1: certified 65.3% | regret vs J* median 54.90% [35.80, 97.49] | vs J_Fonly 40.21% [24.37, 69.08] | beats tuned linear 39.7% + b= 4: certified 92.0% | regret vs J* median 12.85% [10.18, 15.87] | vs J_Fonly 6.32% [4.91, 7.28] | beats tuned linear 73.4% + b=16: certified 98.5% | regret vs J* median 6.79% [5.57, 8.07] | vs J_Fonly 2.16% [1.83, 2.65] | beats tuned linear 87.4% +== x21_fgen_clL_s1_best_tr_K16_clip1.5 (step 26000), 199 plants; sampling 5 s, scoring 466 s + certified per candidate 65.1%; candidates with knots off sat_1 0.0% + b= 1: certified 65.3% | regret vs J* median 49.63% [33.95, 92.94] | vs J_Fonly 36.62% [19.29, 53.54] | beats tuned linear 41.7% + b= 4: certified 94.0% | regret vs J* median 13.25% [11.42, 16.88] | vs J_Fonly 6.56% [4.79, 8.06] | beats tuned linear 73.4% + b=16: certified 98.0% | regret vs J* median 6.62% [5.45, 8.06] | vs J_Fonly 2.07% [1.65, 2.48] | beats tuned linear 88.4% +== x21_fgen_clL_s2_best_tr_K16_clip1.5 (step 24000), 199 plants; sampling 5 s, scoring 319 s + certified per candidate 65.5%; candidates with knots off sat_1 0.0% + b= 1: certified 61.3% | regret vs J* median 42.49% [31.99, 115.60] | vs J_Fonly 32.65% [22.81, 78.49] | beats tuned linear 37.2% + b= 4: certified 89.9% | regret vs J* median 11.93% [10.03, 15.00] | vs J_Fonly 6.62% [5.47, 7.84] | beats tuned linear 69.3% + b=16: certified 97.5% | regret vs J* median 6.81% [5.59, 7.99] | vs J_Fonly 2.02% [1.67, 2.51] | beats tuned linear 87.4% +== x28_coefS_s0_best_tr_K16_clip3 (step 28000), 199 plants; sampling 5 s, scoring 1111 s + certified per candidate 49.7%; candidates with knots off sat_1 100.0% + b= 1: certified 48.7% | regret vs J* median 100000.00% [103.27, 100000.00] | vs J_Fonly 100000.00% [73.09, 100000.00] | beats tuned linear 21.6% + b= 4: certified 83.4% | regret vs J* median 21.51% [16.58, 30.32] | vs J_Fonly 19.05% [13.32, 23.13] | beats tuned linear 54.3% + b=16: certified 95.0% | regret vs J* median 10.58% [8.29, 12.93] | vs J_Fonly 6.04% [4.70, 8.22] | beats tuned linear 78.4% +== x28_clS_s0_best_K16_clip1 (step 24000), 199 plants; sampling 5 s, scoring 759 s + certified per candidate 47.2%; candidates with knots off sat_1 99.9% + b= 1: certified 43.7% | regret vs J* median 100000.00% [509.71, 100000.00] | vs J_Fonly 100000.00% [351.64, 100000.00] | beats tuned linear 25.1% + b= 4: certified 82.4% | regret vs J* median 26.65% [19.34, 33.41] | vs J_Fonly 17.62% [13.49, 23.73] | beats tuned linear 55.3% + b=16: certified 93.5% | regret vs J* median 10.57% [8.47, 12.13] | vs J_Fonly 5.56% [4.26, 7.62] | beats tuned linear 79.4% +== x28_clS_s1_best_K16_clip1.5 (step 26000), 199 plants; sampling 5 s, scoring 678 s + certified per candidate 36.8%; candidates with knots off sat_1 100.0% + b= 1: certified 35.7% | regret vs J* median 100000.00% [100000.00, 100000.00] | vs J_Fonly 100000.00% [100000.00, 100000.00] | beats tuned linear 22.1% + b= 4: certified 74.4% | regret vs J* median 24.18% [18.72, 30.83] | vs J_Fonly 15.82% [9.80, 21.77] | beats tuned linear 51.8% + b=16: certified 91.0% | regret vs J* median 7.37% [6.21, 9.00] | vs J_Fonly 3.23% [2.42, 4.50] | beats tuned linear 76.9% +== x28_clS_s2_best_K16_clip1 (step 24000), 199 plants; sampling 5 s, scoring 718 s + certified per candidate 46.9%; candidates with knots off sat_1 100.0% + b= 1: certified 44.2% | regret vs J* median 100000.00% [295.81, 100000.00] | vs J_Fonly 100000.00% [295.46, 100000.00] | beats tuned linear 26.6% + b= 4: certified 79.9% | regret vs J* median 33.60% [25.08, 42.93] | vs J_Fonly 22.60% [14.87, 32.28] | beats tuned linear 53.8% + b=16: certified 94.0% | regret vs J* median 10.28% [8.61, 12.43] | vs J_Fonly 5.42% [4.21, 7.19] | beats tuned linear 77.4% +== x28_clS_s0_best_tr_K64_clip1.5 (step 24000), 199 plants; sampling 18 s, scoring 3915 s + certified per candidate 51.2%; candidates with knots off sat_1 100.0% + b= 1: certified 51.8% | regret vs J* median 228.92% [93.75, 100000.00] | vs J_Fonly 227.68% [68.18, 100000.00] | beats tuned linear 29.6% + b= 4: certified 88.4% | regret vs J* median 14.50% [12.28, 17.42] | vs J_Fonly 9.04% [7.54, 11.41] | beats tuned linear 67.3% + b=16: certified 96.0% | regret vs J* median 6.61% [5.82, 7.35] | vs J_Fonly 2.28% [1.60, 3.18] | beats tuned linear 84.4% + b=64: certified 99.0% | regret vs J* median 2.96% [2.61, 3.51] | vs J_Fonly 0.45% [0.00, 0.75] | beats tuned linear 94.0% +== x21_fgen_clL_best_tr_K64_clip1.5 (step 26000), 199 plants; sampling 14 s, scoring 1299 s + certified per candidate 66.6%; candidates with knots off sat_1 0.0% + b= 1: certified 67.8% | regret vs J* median 53.11% [37.09, 83.62] | vs J_Fonly 35.18% [27.02, 74.13] | beats tuned linear 37.7% + b= 4: certified 91.0% | regret vs J* median 12.85% [11.10, 16.72] | vs J_Fonly 6.55% [4.76, 8.68] | beats tuned linear 70.4% + b=16: certified 97.5% | regret vs J* median 6.60% [5.88, 8.24] | vs J_Fonly 2.44% [1.86, 2.86] | beats tuned linear 86.9% + b=64: certified 98.5% | regret vs J* median 4.07% [3.54, 4.77] | vs J_Fonly 0.81% [0.57, 1.03] | beats tuned linear 95.0% diff --git a/results/icml_r3/X28/eval_val/x28_clS_s0_best_tr_K16_clip1.5.npz b/results/icml_r3/X28/eval_val/x28_clS_s0_best_tr_K16_clip1.5.npz new file mode 100644 index 0000000000000000000000000000000000000000..79547d189e0ac3989932e179ebd3e0f0a189cbd6 --- /dev/null +++ b/results/icml_r3/X28/eval_val/x28_clS_s0_best_tr_K16_clip1.5.npz @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c3c8481b1318f690bf8ec523bc6bf123b8e2934d5f35cec0efc988d6e0c35f84 +size 84662 diff --git a/results/icml_r3/X28/eval_val/x28_clS_s0_best_tr_K16_clip1.npz b/results/icml_r3/X28/eval_val/x28_clS_s0_best_tr_K16_clip1.npz new file mode 100644 index 0000000000000000000000000000000000000000..9d0ba2bb87aa634b848a1279009a5ac4127dfff6 --- /dev/null +++ b/results/icml_r3/X28/eval_val/x28_clS_s0_best_tr_K16_clip1.npz @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:b97bd547f57e43e87f7ee929db6f050552e9a4104362ebdc945fb78fcf76cc3c +size 84773 diff --git a/results/icml_r3/X28/eval_val/x28_clS_s0_best_tr_K16_clip2.npz b/results/icml_r3/X28/eval_val/x28_clS_s0_best_tr_K16_clip2.npz new file mode 100644 index 0000000000000000000000000000000000000000..7d0181e6d57f6010aa92f1d122ab9bc9324e659a --- /dev/null +++ b/results/icml_r3/X28/eval_val/x28_clS_s0_best_tr_K16_clip2.npz @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:bad6516313bb689e0859b570b385c498eee08954cfb12b4b4378c0438d1d63a3 +size 84597 diff --git a/results/icml_r3/X28/eval_val/x28_clS_s0_best_tr_K16_clip3.npz b/results/icml_r3/X28/eval_val/x28_clS_s0_best_tr_K16_clip3.npz new file mode 100644 index 0000000000000000000000000000000000000000..39148626e9f7e1c27e64686390dca6f3649026b6 --- /dev/null +++ b/results/icml_r3/X28/eval_val/x28_clS_s0_best_tr_K16_clip3.npz @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f4dbff1fd7632c4f45506493142a7c20eba19d4aa13271cce6cd8b70d9290328 +size 84667 diff --git a/results/icml_r3/X28/eval_val/x28_clS_s1_best_tr_K16_clip1.5.npz b/results/icml_r3/X28/eval_val/x28_clS_s1_best_tr_K16_clip1.5.npz new file mode 100644 index 0000000000000000000000000000000000000000..ba9543f93b438a19411c6f3cfe38e458b9964fa2 --- /dev/null +++ b/results/icml_r3/X28/eval_val/x28_clS_s1_best_tr_K16_clip1.5.npz @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:2790e2334528a7e1fd347e2fe49f123dfcb908aa18e44743ac846fa4a7077cd6 +size 85300 diff --git a/results/icml_r3/X28/eval_val/x28_clS_s1_best_tr_K16_clip1.npz b/results/icml_r3/X28/eval_val/x28_clS_s1_best_tr_K16_clip1.npz new file mode 100644 index 0000000000000000000000000000000000000000..ab017ffb08024d0fb7f79ca1754c055da661481c --- /dev/null +++ b/results/icml_r3/X28/eval_val/x28_clS_s1_best_tr_K16_clip1.npz @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:28ec6d8eba839bb68503be32b9a9dbd8e16e594d568852f05564533617993774 +size 85525 diff --git a/results/icml_r3/X28/eval_val/x28_clS_s1_best_tr_K16_clip2.npz b/results/icml_r3/X28/eval_val/x28_clS_s1_best_tr_K16_clip2.npz new file mode 100644 index 0000000000000000000000000000000000000000..4d6c064bbae2b15b466aaec327428ed2a0202958 --- /dev/null +++ b/results/icml_r3/X28/eval_val/x28_clS_s1_best_tr_K16_clip2.npz @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:7275f33679eb9e86820e78cdb69a4f2668348544365bfc678c9378c86f8848ba +size 85138 diff --git a/results/icml_r3/X28/eval_val/x28_clS_s1_best_tr_K16_clip3.npz b/results/icml_r3/X28/eval_val/x28_clS_s1_best_tr_K16_clip3.npz new file mode 100644 index 0000000000000000000000000000000000000000..f913d1ede845d3369fabdba492faa5693956645e --- /dev/null +++ b/results/icml_r3/X28/eval_val/x28_clS_s1_best_tr_K16_clip3.npz @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:dbc31268e9a0cebf1d175202689a9f85b55260ff1ff6ea1d39324ce74253d8b3 +size 85259 diff --git a/results/icml_r3/X28/eval_val/x28_clS_s2_best_tr_K16_clip1.5.npz b/results/icml_r3/X28/eval_val/x28_clS_s2_best_tr_K16_clip1.5.npz new file mode 100644 index 0000000000000000000000000000000000000000..b9ed1b749a80614b346da93c6d6b6069bf56b1cc --- /dev/null +++ b/results/icml_r3/X28/eval_val/x28_clS_s2_best_tr_K16_clip1.5.npz @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:fc4a4d78bd89c1c1745680d58c41db5988cf6b16cfa6fbd15918d817a0e35bb7 +size 84783 diff --git a/results/icml_r3/X28/eval_val/x28_clS_s2_best_tr_K16_clip1.npz b/results/icml_r3/X28/eval_val/x28_clS_s2_best_tr_K16_clip1.npz new file mode 100644 index 0000000000000000000000000000000000000000..1756959720a80d59ff602ad3885a0057c3bde987 --- /dev/null +++ b/results/icml_r3/X28/eval_val/x28_clS_s2_best_tr_K16_clip1.npz @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:269e46e92e307bcff82fdb5b5e3f48884ba8bc19749efaad7049ecbf43f4022f +size 84976 diff --git a/results/icml_r3/X28/eval_val/x28_clS_s2_best_tr_K16_clip2.npz b/results/icml_r3/X28/eval_val/x28_clS_s2_best_tr_K16_clip2.npz new file mode 100644 index 0000000000000000000000000000000000000000..5439a43cc503196a07e9f2db284c36cd3468680b --- /dev/null +++ b/results/icml_r3/X28/eval_val/x28_clS_s2_best_tr_K16_clip2.npz @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:6b06a42dddc57a77247692e35fa94ca969557729d261de36f7f4c7a6d30a5750 +size 84587 diff --git a/results/icml_r3/X28/eval_val/x28_clS_s2_best_tr_K16_clip3.npz b/results/icml_r3/X28/eval_val/x28_clS_s2_best_tr_K16_clip3.npz new file mode 100644 index 0000000000000000000000000000000000000000..2b421344722e9ce3fb9f956cf195cb882a571e38 --- /dev/null +++ b/results/icml_r3/X28/eval_val/x28_clS_s2_best_tr_K16_clip3.npz @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:44b1f4182338f9ca63c1c02a6c93f27aad6af8ae0a755dbfb742f6f4740ba990 +size 84777 diff --git a/results/icml_r3/X28/eval_val/x28_coefS_s0_best_tr_K16_clip1.5.npz b/results/icml_r3/X28/eval_val/x28_coefS_s0_best_tr_K16_clip1.5.npz new file mode 100644 index 0000000000000000000000000000000000000000..ba6064644f35876746307b15d98c8f9fc547c2c4 --- /dev/null +++ b/results/icml_r3/X28/eval_val/x28_coefS_s0_best_tr_K16_clip1.5.npz @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:5029c66fb3d58987bfa2e801ec5545b7ea63d106374f21a05b37758314872974 +size 82275 diff --git a/results/icml_r3/X28/eval_val/x28_coefS_s0_best_tr_K16_clip1.npz b/results/icml_r3/X28/eval_val/x28_coefS_s0_best_tr_K16_clip1.npz new file mode 100644 index 0000000000000000000000000000000000000000..d105882f2bbfb793cddad331d9719a4fc21b49ef --- /dev/null +++ b/results/icml_r3/X28/eval_val/x28_coefS_s0_best_tr_K16_clip1.npz @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:9f6bbe9969d0b036e8511b98b6525a3f6a6c84e7619792f5933ac326fcf729ac +size 78666 diff --git a/results/icml_r3/X28/eval_val/x28_coefS_s0_best_tr_K16_clip2.npz b/results/icml_r3/X28/eval_val/x28_coefS_s0_best_tr_K16_clip2.npz new file mode 100644 index 0000000000000000000000000000000000000000..2361a5c019457d8ce653c663cb5ca94c9890c887 --- /dev/null +++ b/results/icml_r3/X28/eval_val/x28_coefS_s0_best_tr_K16_clip2.npz @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:328715e2ab7cd958ee883f01c1ead8834dc8d9f97874f811b291d02b0d21d8fe +size 83609 diff --git a/results/icml_r3/X28/eval_val/x28_coefS_s0_best_tr_K16_clip3.npz b/results/icml_r3/X28/eval_val/x28_coefS_s0_best_tr_K16_clip3.npz new file mode 100644 index 0000000000000000000000000000000000000000..e86d464ff91ba80ba520eba12d08de46b8187eed --- /dev/null +++ b/results/icml_r3/X28/eval_val/x28_coefS_s0_best_tr_K16_clip3.npz @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:587f8e35fce888e03f4436dbff11b9a3d8c6d3efb1afd0db83437ee363077bb4 +size 84638 diff --git a/results/icml_r3/X28/eval_val_report.txt b/results/icml_r3/X28/eval_val_report.txt index cad68f3a7a7951eb47282a1d210153204890ba43..33587f7824b5e4aab68ef979d2ab514809f92a45 100644 --- a/results/icml_r3/X28/eval_val_report.txt +++ b/results/icml_r3/X28/eval_val_report.txt @@ -138,3 +138,83 @@ b= 1: certified 62.5% | regret vs J* median 51.01% [18.44, 100000.00] | vs J_Fonly 35.53% [13.49, 100000.00] | beats tuned linear 47.5% b= 4: certified 92.5% | regret vs J* median 13.84% [7.81, 22.33] | vs J_Fonly 7.84% [4.72, 15.62] | beats tuned linear 67.5% b=16: certified 100.0% | regret vs J* median 7.12% [4.39, 10.44] | vs J_Fonly 2.19% [1.44, 4.58] | beats tuned linear 95.0% +== x28_clS_s0_best_tr_K16_clip1 (step 24000), 40 plants; sampling 2 s, scoring 293 s + certified per candidate 60.5%; candidates with knots off sat_1 100.0% + b= 1: certified 55.0% | regret vs J* median 382.55% [92.19, 100000.00] | vs J_Fonly 362.53% [90.27, 100000.00] | beats tuned linear 27.5% + b= 4: certified 92.5% | regret vs J* median 25.29% [11.90, 33.24] | vs J_Fonly 17.79% [5.93, 28.58] | beats tuned linear 70.0% + b=16: certified 100.0% | regret vs J* median 10.83% [6.51, 16.31] | vs J_Fonly 4.84% [2.77, 9.43] | beats tuned linear 87.5% +== x28_clS_s0_best_tr_K16_clip1.5 (step 24000), 40 plants; sampling 2 s, scoring 353 s + certified per candidate 49.8%; candidates with knots off sat_1 100.0% + b= 1: certified 42.5% | regret vs J* median 100000.00% [119.31, 100000.00] | vs J_Fonly 100000.00% [113.17, 100000.00] | beats tuned linear 25.0% + b= 4: certified 87.5% | regret vs J* median 19.14% [13.44, 28.47] | vs J_Fonly 17.02% [8.91, 23.23] | beats tuned linear 62.5% + b=16: certified 100.0% | regret vs J* median 9.79% [5.46, 13.59] | vs J_Fonly 4.88% [3.38, 8.01] | beats tuned linear 80.0% +== x28_clS_s0_best_tr_K16_clip2 (step 24000), 40 plants; sampling 2 s, scoring 279 s + certified per candidate 46.1%; candidates with knots off sat_1 100.0% + b= 1: certified 40.0% | regret vs J* median 100000.00% [213.65, 100000.00] | vs J_Fonly 100000.00% [207.37, 100000.00] | beats tuned linear 17.5% + b= 4: certified 82.5% | regret vs J* median 28.09% [14.87, 55.97] | vs J_Fonly 25.39% [9.51, 47.75] | beats tuned linear 50.0% + b=16: certified 97.5% | regret vs J* median 9.70% [5.87, 13.47] | vs J_Fonly 4.64% [3.47, 7.66] | beats tuned linear 77.5% +== x28_clS_s0_best_tr_K16_clip3 (step 24000), 40 plants; sampling 2 s, scoring 294 s + certified per candidate 42.5%; candidates with knots off sat_1 100.0% + b= 1: certified 37.5% | regret vs J* median 100000.00% [399.86, 100000.00] | vs J_Fonly 100000.00% [389.72, 100000.00] | beats tuned linear 15.0% + b= 4: certified 77.5% | regret vs J* median 24.52% [14.29, 35.88] | vs J_Fonly 20.89% [12.63, 29.21] | beats tuned linear 50.0% + b=16: certified 97.5% | regret vs J* median 9.84% [6.39, 15.65] | vs J_Fonly 5.05% [3.76, 8.45] | beats tuned linear 77.5% +== x28_clS_s1_best_tr_K16_clip1 (step 26000), 40 plants; sampling 3 s, scoring 314 s + certified per candidate 60.3%; candidates with knots off sat_1 100.0% + b= 1: certified 65.0% | regret vs J* median 129.01% [47.40, 50243.95] | vs J_Fonly 126.61% [41.23, 50229.98] | beats tuned linear 32.5% + b= 4: certified 92.5% | regret vs J* median 24.19% [12.61, 36.32] | vs J_Fonly 15.04% [6.37, 29.74] | beats tuned linear 70.0% + b=16: certified 100.0% | regret vs J* median 12.00% [7.00, 15.72] | vs J_Fonly 5.20% [2.50, 10.85] | beats tuned linear 87.5% +== x28_clS_s1_best_tr_K16_clip1.5 (step 26000), 40 plants; sampling 2 s, scoring 337 s + certified per candidate 51.2%; candidates with knots off sat_1 100.0% + b= 1: certified 45.0% | regret vs J* median 100000.00% [69.67, 100000.00] | vs J_Fonly 100000.00% [65.51, 100000.00] | beats tuned linear 22.5% + b= 4: certified 95.0% | regret vs J* median 17.88% [10.15, 41.27] | vs J_Fonly 13.22% [7.53, 39.24] | beats tuned linear 60.0% + b=16: certified 100.0% | regret vs J* median 9.57% [5.74, 12.88] | vs J_Fonly 4.33% [2.53, 8.85] | beats tuned linear 90.0% +== x28_clS_s1_best_tr_K16_clip2 (step 26000), 40 plants; sampling 2 s, scoring 293 s + certified per candidate 46.4%; candidates with knots off sat_1 100.0% + b= 1: certified 37.5% | regret vs J* median 100000.00% [195.37, 100000.00] | vs J_Fonly 100000.00% [191.09, 100000.00] | beats tuned linear 15.0% + b= 4: certified 85.0% | regret vs J* median 17.56% [11.54, 42.77] | vs J_Fonly 15.80% [7.97, 40.06] | beats tuned linear 57.5% + b=16: certified 95.0% | regret vs J* median 9.08% [5.87, 14.64] | vs J_Fonly 5.15% [3.80, 8.98] | beats tuned linear 80.0% +== x28_clS_s1_best_tr_K16_clip3 (step 26000), 40 plants; sampling 2 s, scoring 304 s + certified per candidate 42.5%; candidates with knots off sat_1 100.0% + b= 1: certified 35.0% | regret vs J* median 100000.00% [50092.27, 100000.00] | vs J_Fonly 100000.00% [50085.86, 100000.00] | beats tuned linear 17.5% + b= 4: certified 90.0% | regret vs J* median 19.00% [10.67, 39.55] | vs J_Fonly 13.53% [8.30, 33.41] | beats tuned linear 62.5% + b=16: certified 97.5% | regret vs J* median 9.38% [5.51, 13.31] | vs J_Fonly 6.38% [3.35, 8.98] | beats tuned linear 80.0% +== x28_clS_s2_best_tr_K16_clip1 (step 24000), 40 plants; sampling 4 s, scoring 295 s + certified per candidate 61.3%; candidates with knots off sat_1 100.0% + b= 1: certified 62.5% | regret vs J* median 160.16% [73.22, 100000.00] | vs J_Fonly 157.45% [71.40, 100000.00] | beats tuned linear 32.5% + b= 4: certified 92.5% | regret vs J* median 26.81% [13.57, 38.50] | vs J_Fonly 16.55% [6.75, 30.69] | beats tuned linear 67.5% + b=16: certified 100.0% | regret vs J* median 12.62% [5.37, 19.79] | vs J_Fonly 3.88% [2.58, 13.60] | beats tuned linear 87.5% +== x28_clS_s2_best_tr_K16_clip1.5 (step 24000), 40 plants; sampling 2 s, scoring 322 s + certified per candidate 48.9%; candidates with knots off sat_1 100.0% + b= 1: certified 52.5% | regret vs J* median 368.79% [62.28, 100000.00] | vs J_Fonly 353.62% [59.85, 100000.00] | beats tuned linear 27.5% + b= 4: certified 90.0% | regret vs J* median 17.08% [9.03, 36.35] | vs J_Fonly 13.24% [6.10, 28.48] | beats tuned linear 62.5% + b=16: certified 100.0% | regret vs J* median 8.56% [3.82, 12.98] | vs J_Fonly 4.33% [2.00, 8.47] | beats tuned linear 77.5% +== x28_clS_s2_best_tr_K16_clip2 (step 24000), 40 plants; sampling 2 s, scoring 325 s + certified per candidate 43.6%; candidates with knots off sat_1 100.0% + b= 1: certified 37.5% | regret vs J* median 100000.00% [415.67, 100000.00] | vs J_Fonly 100000.00% [405.35, 100000.00] | beats tuned linear 17.5% + b= 4: certified 77.5% | regret vs J* median 26.27% [18.38, 51.95] | vs J_Fonly 23.15% [10.00, 46.89] | beats tuned linear 52.5% + b=16: certified 100.0% | regret vs J* median 12.54% [3.76, 16.87] | vs J_Fonly 6.44% [2.50, 11.09] | beats tuned linear 77.5% +== x28_clS_s2_best_tr_K16_clip3 (step 24000), 40 plants; sampling 2 s, scoring 280 s + certified per candidate 40.5%; candidates with knots off sat_1 100.0% + b= 1: certified 37.5% | regret vs J* median 100000.00% [343.82, 100000.00] | vs J_Fonly 100000.00% [334.93, 100000.00] | beats tuned linear 15.0% + b= 4: certified 80.0% | regret vs J* median 29.34% [16.08, 83.13] | vs J_Fonly 26.24% [12.19, 71.00] | beats tuned linear 47.5% + b=16: certified 95.0% | regret vs J* median 12.05% [4.23, 16.16] | vs J_Fonly 7.26% [2.51, 11.54] | beats tuned linear 72.5% +== x28_coefS_s0_best_tr_K16_clip1 (step 28000), 40 plants; sampling 2 s, scoring 187 s + certified per candidate 38.1%; candidates with knots off sat_1 100.0% + b= 1: certified 37.5% | regret vs J* median 100000.00% [180.00, 100000.00] | vs J_Fonly 100000.00% [161.88, 100000.00] | beats tuned linear 22.5% + b= 4: certified 50.0% | regret vs J* median 50126.73% [51.91, 100000.00] | vs J_Fonly 50113.26% [47.20, 100000.00] | beats tuned linear 32.5% + b=16: certified 67.5% | regret vs J* median 137.13% [22.05, 662.38] | vs J_Fonly 97.82% [14.90, 647.99] | beats tuned linear 37.5% +== x28_coefS_s0_best_tr_K16_clip1.5 (step 28000), 40 plants; sampling 2 s, scoring 186 s + certified per candidate 44.4%; candidates with knots off sat_1 100.0% + b= 1: certified 52.5% | regret vs J* median 148.36% [40.57, 100000.00] | vs J_Fonly 140.26% [39.68, 100000.00] | beats tuned linear 25.0% + b= 4: certified 65.0% | regret vs J* median 46.44% [21.01, 50229.23] | vs J_Fonly 39.68% [14.63, 50219.54] | beats tuned linear 45.0% + b=16: certified 80.0% | regret vs J* median 22.97% [9.87, 57.48] | vs J_Fonly 14.10% [4.40, 54.46] | beats tuned linear 60.0% +== x28_coefS_s0_best_tr_K16_clip2 (step 28000), 40 plants; sampling 2 s, scoring 280 s + certified per candidate 46.2%; candidates with knots off sat_1 100.0% + b= 1: certified 50.0% | regret vs J* median 50081.59% [49.10, 100000.00] | vs J_Fonly 50076.27% [44.56, 100000.00] | beats tuned linear 32.5% + b= 4: certified 75.0% | regret vs J* median 41.63% [17.19, 71.88] | vs J_Fonly 33.16% [9.69, 59.76] | beats tuned linear 60.0% + b=16: certified 87.5% | regret vs J* median 10.47% [6.23, 16.43] | vs J_Fonly 4.99% [3.15, 6.38] | beats tuned linear 80.0% +== x28_coefS_s0_best_tr_K16_clip3 (step 28000), 40 plants; sampling 2 s, scoring 258 s + certified per candidate 44.5%; candidates with knots off sat_1 100.0% + b= 1: certified 42.5% | regret vs J* median 100000.00% [66.37, 100000.00] | vs J_Fonly 100000.00% [63.51, 100000.00] | beats tuned linear 25.0% + b= 4: certified 75.0% | regret vs J* median 34.01% [13.29, 55.61] | vs J_Fonly 21.54% [8.30, 47.26] | beats tuned linear 62.5% + b=16: certified 90.0% | regret vs J* median 9.07% [6.62, 14.19] | vs J_Fonly 4.63% [2.86, 8.85] | beats tuned linear 85.0% diff --git a/results/icml_r3/X28/mimo/fresh_study_m2.npz b/results/icml_r3/X28/mimo/fresh_study_m2.npz new file mode 100644 index 0000000000000000000000000000000000000000..0449d7c95ab468ec709ddc4de9c1c102c6d5c7b5 --- /dev/null +++ b/results/icml_r3/X28/mimo/fresh_study_m2.npz @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:4fd7aad4decf538d206e38e0fe4fc8a23979801da5da7467651363b5da635bf0 +size 504450 diff --git a/results/icml_r3/X28/mimo/fresh_study_m2.txt b/results/icml_r3/X28/mimo/fresh_study_m2.txt new file mode 100644 index 0000000000000000000000000000000000000000..e060df32b23ef43301fe3d90fa02b90f6dd8aa3c --- /dev/null +++ b/results/icml_r3/X28/mimo/fresh_study_m2.txt @@ -0,0 +1,6 @@ +X28-M fresh directions, set study, m = 2: 64 random unit directions per plant, seed (7676, plant id) + sp: per plant median over directions of (J_id - J_sp) / J_sp; median over plants -2.11% [-4.80, -1.32], > 0 on 19 of 64; non-convergent rollouts id 0, sp 180 + so: per plant median over directions of (J_id - J_so) / J_so; median over plants 3.11% [2.48, 4.11], > 0 on 55 of 64; non-convergent rollouts id 0, so 0 + so3: per plant median over directions of (J_id - J_so3) / J_so3; median over plants 3.99% [2.86, 5.13], > 0 on 56 of 64; non-convergent rollouts id 0, so3 0 + cf: per plant median over directions of (J_id - J_cf) / J_cf; median over plants -3.09% [-4.50, -2.32], > 0 on 15 of 64; non-convergent rollouts id 0, cf 43 + co: per plant median over directions of (J_id - J_co) / J_co; median over plants 2.75% [2.13, 3.45], > 0 on 54 of 64; non-convergent rollouts id 0, co 1 diff --git a/results/icml_r3/X28/mimo/fresh_study_m3.npz b/results/icml_r3/X28/mimo/fresh_study_m3.npz new file mode 100644 index 0000000000000000000000000000000000000000..1a462d539153ee49bf8f296f33a3500a99c96a09 --- /dev/null +++ b/results/icml_r3/X28/mimo/fresh_study_m3.npz @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:97614f56288c2dddc9881663b7cd23e66f709f32c27b88163baf9f694b2c0cbc +size 565124 diff --git a/results/icml_r3/X28/mimo/fresh_study_m3.txt b/results/icml_r3/X28/mimo/fresh_study_m3.txt new file mode 100644 index 0000000000000000000000000000000000000000..561ea81555f0c27181bc965b5eb1a4c804412f55 --- /dev/null +++ b/results/icml_r3/X28/mimo/fresh_study_m3.txt @@ -0,0 +1,6 @@ +X28-M fresh directions, set study, m = 3: 64 random unit directions per plant, seed (7676, plant id) + sp: per plant median over directions of (J_id - J_sp) / J_sp; median over plants -5.94% [-8.80, -3.77], > 0 on 8 of 64; non-convergent rollouts id 0, sp 188 + so: per plant median over directions of (J_id - J_so) / J_so; median over plants 1.32% [0.64, 1.94], > 0 on 47 of 64; non-convergent rollouts id 0, so 0 + so3: per plant median over directions of (J_id - J_so3) / J_so3; median over plants 1.82% [0.74, 2.57], > 0 on 50 of 64; non-convergent rollouts id 0, so3 0 + cf: per plant median over directions of (J_id - J_cf) / J_cf; median over plants -6.06% [-8.40, -4.46], > 0 on 8 of 64; non-convergent rollouts id 0, cf 13 + co: per plant median over directions of (J_id - J_co) / J_co; median over plants 0.91% [0.45, 1.51], > 0 on 47 of 64; non-convergent rollouts id 0, co 0 diff --git a/results/icml_r3/X28/mimo/gate_study_m2.txt b/results/icml_r3/X28/mimo/gate_study_m2.txt new file mode 100644 index 0000000000000000000000000000000000000000..e000352b6cd6f63875c369fd6e9b5e981986f4a3 --- /dev/null +++ b/results/icml_r3/X28/mimo/gate_study_m2.txt @@ -0,0 +1,27 @@ +X28-M gate, set study, m = 2, n = 64, id certified on 64, median J_id 11.8 + id: snapshot 0 kept on 0, last snapshot on 15 +piecewise linear, free (sp, 10 per channel): d = (J_id - J_spline) / J_spline median 6.61% [5.46, 7.52], >= 2% on 60 of 64, mean 7.57% -> PAYS + start sp_c1: certified 64, snapshot 0 kept on 0, median d 5.48% + start sp_c2: certified 64, snapshot 0 kept on 1, median d 6.13% + start sp_c5: certified 64, snapshot 0 kept on 0, median d 4.85% + winning splines 64: max beta / alpha of a channel median 5.411; max |log r(-x) / r(x)| median 1.346; rollouts with final |x| > 1e-2: 0 +piecewise linear, odd (so, 5 per channel): d = (J_id - J_spline) / J_spline median 5.40% [4.48, 6.41], >= 2% on 53 of 64, mean 6.02% -> PAYS + start so_c1: certified 64, snapshot 0 kept on 0, median d 4.02% + start so_c2: certified 64, snapshot 0 kept on 0, median d 5.08% + start so_c5: certified 64, snapshot 0 kept on 0, median d 3.71% + winning splines 64: max beta / alpha of a channel median 3.728; max |log r(-x) / r(x)| median 0.000; rollouts with final |x| > 1e-2: 0 +piecewise linear, odd, knots at |x| = 1, 3, 5 (so3, 3 per channel): d = (J_id - J_spline) / J_spline median 5.22% [4.37, 6.29], >= 2% on 53 of 64, mean 5.57% -> PAYS + start so3_c1: certified 64, snapshot 0 kept on 0, median d 4.36% + start so3_c2: certified 64, snapshot 0 kept on 0, median d 4.86% + start so3_c5: certified 64, snapshot 0 kept on 0, median d 3.99% + winning splines 64: max beta / alpha of a channel median 3.437; max |log r(-x) / r(x)| median 0.000; rollouts with final |x| > 1e-2: 0 +cubic (PCHIP ratio), free (cf, 10 per channel): d = (J_id - J_spline) / J_spline median 6.39% [5.07, 7.52], >= 2% on 58 of 64, mean 7.47% -> PAYS + start cf_c1: certified 64, snapshot 0 kept on 1, median d 5.21% + start cf_c2: certified 64, snapshot 0 kept on 1, median d 5.87% + start cf_c5: certified 64, snapshot 0 kept on 0, median d 4.90% + winning splines 64: max beta / alpha of a channel median 5.454; max |log r(-x) / r(x)| median 1.198; rollouts with final |x| > 1e-2: 0 +cubic (PCHIP ratio), odd (co, 5 per channel): d = (J_id - J_spline) / J_spline median 5.37% [4.41, 6.24], >= 2% on 53 of 64, mean 5.67% -> PAYS + start co_c1: certified 64, snapshot 0 kept on 0, median d 4.09% + start co_c2: certified 64, snapshot 0 kept on 0, median d 4.70% + start co_c5: certified 64, snapshot 0 kept on 0, median d 3.67% + winning splines 64: max beta / alpha of a channel median 3.671; max |log r(-x) / r(x)| median 0.000; rollouts with final |x| > 1e-2: 0 diff --git a/results/icml_r3/X28/mimo/gate_study_m3.txt b/results/icml_r3/X28/mimo/gate_study_m3.txt new file mode 100644 index 0000000000000000000000000000000000000000..d5ee122bc1289df47d94b711f1c1025e0d2dfc02 --- /dev/null +++ b/results/icml_r3/X28/mimo/gate_study_m3.txt @@ -0,0 +1,27 @@ +X28-M gate, set study, m = 3, n = 64, id certified on 64, median J_id 13.19 + id: snapshot 0 kept on 0, last snapshot on 15 +piecewise linear, free (sp, 10 per channel): d = (J_id - J_spline) / J_spline median 6.31% [4.87, 7.24], >= 2% on 55 of 64, mean 6.41% -> PAYS + start sp_c1: certified 64, snapshot 0 kept on 0, median d 4.62% + start sp_c2: certified 64, snapshot 0 kept on 0, median d 5.46% + start sp_c5: certified 64, snapshot 0 kept on 0, median d 4.83% + winning splines 62: max beta / alpha of a channel median 6.247; max |log r(-x) / r(x)| median 1.389; rollouts with final |x| > 1e-2: 0 +piecewise linear, odd (so, 5 per channel): d = (J_id - J_spline) / J_spline median 4.80% [4.05, 5.97], >= 2% on 51 of 64, mean 5.47% -> PAYS + start so_c1: certified 64, snapshot 0 kept on 0, median d 4.01% + start so_c2: certified 64, snapshot 0 kept on 0, median d 4.44% + start so_c5: certified 64, snapshot 0 kept on 0, median d 3.35% + winning splines 64: max beta / alpha of a channel median 4.856; max |log r(-x) / r(x)| median 0.000; rollouts with final |x| > 1e-2: 0 +piecewise linear, odd, knots at |x| = 1, 3, 5 (so3, 3 per channel): d = (J_id - J_spline) / J_spline median 4.54% [4.12, 5.59], >= 2% on 51 of 64, mean 5.24% -> PAYS + start so3_c1: certified 64, snapshot 0 kept on 0, median d 4.06% + start so3_c2: certified 64, snapshot 0 kept on 0, median d 4.20% + start so3_c5: certified 64, snapshot 0 kept on 0, median d 3.41% + winning splines 64: max beta / alpha of a channel median 4.361; max |log r(-x) / r(x)| median 0.000; rollouts with final |x| > 1e-2: 0 +cubic (PCHIP ratio), free (cf, 10 per channel): d = (J_id - J_spline) / J_spline median 5.39% [4.77, 6.42], >= 2% on 53 of 64, mean 5.90% -> PAYS + start cf_c1: certified 64, snapshot 0 kept on 0, median d 4.50% + start cf_c2: certified 64, snapshot 0 kept on 1, median d 5.12% + start cf_c5: certified 64, snapshot 0 kept on 0, median d 4.10% + winning splines 61: max beta / alpha of a channel median 5.582; max |log r(-x) / r(x)| median 1.225; rollouts with final |x| > 1e-2: 0 +cubic (PCHIP ratio), odd (co, 5 per channel): d = (J_id - J_spline) / J_spline median 5.02% [4.30, 5.81], >= 2% on 50 of 64, mean 5.59% -> PAYS + start co_c1: certified 64, snapshot 0 kept on 0, median d 3.79% + start co_c2: certified 64, snapshot 0 kept on 0, median d 4.53% + start co_c5: certified 64, snapshot 0 kept on 0, median d 3.14% + winning splines 62: max beta / alpha of a channel median 5.394; max |log r(-x) / r(x)| median 0.000; rollouts with final |x| > 1e-2: 0 diff --git a/results/icml_r3/X28/mimo/study_choice.txt b/results/icml_r3/X28/mimo/study_choice.txt new file mode 100644 index 0000000000000000000000000000000000000000..801352ad790f52584b0898228e5a6e140c0a74ec --- /dev/null +++ b/results/icml_r3/X28/mimo/study_choice.txt @@ -0,0 +1,8 @@ +X28-M spline-class study, selection rule of PREREG addendum 2026-11-13 (64 validation plants per m, fresh seed (7676, id)) + sp piecewise linear, free params/ch 10 | m=2: f -2.11% [-4.80, -1.32], non-conv excess 180 | m=3: f -5.94% [-8.80, -3.77], non-conv excess 188 | mean f -4.02% | not eligible + so piecewise linear, odd params/ch 5 | m=2: f +3.11% [+2.48, +4.11], non-conv excess 0 | m=3: f +1.32% [+0.64, +1.94], non-conv excess 0 | mean f +2.22% | eligible + so3 piecewise linear, odd, knots at |x| = 1, 3, 5 params/ch 3 | m=2: f +3.99% [+2.86, +5.13], non-conv excess 0 | m=3: f +1.82% [+0.74, +2.57], non-conv excess 0 | mean f +2.90% | eligible + cf cubic (PCHIP ratio), free params/ch 10 | m=2: f -3.09% [-4.50, -2.32], non-conv excess 43 | m=3: f -6.06% [-8.40, -4.46], non-conv excess 13 | mean f -4.58% | not eligible + co cubic (PCHIP ratio), odd params/ch 5 | m=2: f +2.75% [+2.13, +3.45], non-conv excess 1 | m=3: f +0.91% [+0.45, +1.51], non-conv excess 0 | mean f +1.83% | eligible +highest mean f: so3 (+2.90%); within 0.25 points: so3 +CHOICE: so3 (piecewise linear, odd, knots at |x| = 1, 3, 5, 3 parameters per channel) diff --git a/results/icml_r3/X28/mimo/study_m2/cert_cf_c1.npz b/results/icml_r3/X28/mimo/study_m2/cert_cf_c1.npz new file mode 100644 index 0000000000000000000000000000000000000000..997b8e31b403386c4c13e1354cff9755d5039fa3 --- /dev/null +++ b/results/icml_r3/X28/mimo/study_m2/cert_cf_c1.npz @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:dae6840d2c16bab58f6068905e2845ccc19e5874f8d7a003891f231c882464f0 +size 13157 diff --git a/results/icml_r3/X28/mimo/study_m2/cert_cf_c2.npz b/results/icml_r3/X28/mimo/study_m2/cert_cf_c2.npz new file mode 100644 index 0000000000000000000000000000000000000000..a44493d8f8ea32527b23e004ac7fedd6bd3925bb --- /dev/null +++ b/results/icml_r3/X28/mimo/study_m2/cert_cf_c2.npz @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:193b78e354d2b3b8df683c0309b4b691a73de2b0d093eb54e52eb5b02e9ab343 +size 12506 diff --git a/results/icml_r3/X28/mimo/study_m2/cert_cf_c5.npz b/results/icml_r3/X28/mimo/study_m2/cert_cf_c5.npz new file mode 100644 index 0000000000000000000000000000000000000000..16a63e39c4f7847e168d9547a51213d7d0d918d3 --- /dev/null +++ b/results/icml_r3/X28/mimo/study_m2/cert_cf_c5.npz @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:7c81b65b2bda71cab5d8aa859d8fb246b657066333be57304a8adeafbeb80040 +size 10662 diff --git a/results/icml_r3/X28/mimo/study_m2/cert_co_c1.npz b/results/icml_r3/X28/mimo/study_m2/cert_co_c1.npz new file mode 100644 index 0000000000000000000000000000000000000000..57d3f6d6c2c34f4113e6f394a5851af73728ace8 --- /dev/null +++ b/results/icml_r3/X28/mimo/study_m2/cert_co_c1.npz @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:9f5aed427832a7e28df22bbe0e437889d73f7a7224fd7f7fe5b743ee563a3e91 +size 10428 diff --git a/results/icml_r3/X28/mimo/study_m2/cert_co_c2.npz b/results/icml_r3/X28/mimo/study_m2/cert_co_c2.npz new file mode 100644 index 0000000000000000000000000000000000000000..07795e41470b38137bfd4c1e48177442b7833f85 --- /dev/null +++ b/results/icml_r3/X28/mimo/study_m2/cert_co_c2.npz @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f660b2e230916a4b26409df0ec51f17fff36b2b494a90a52e5dd8c2a0f833531 +size 10313 diff --git a/results/icml_r3/X28/mimo/study_m2/cert_co_c5.npz b/results/icml_r3/X28/mimo/study_m2/cert_co_c5.npz new file mode 100644 index 0000000000000000000000000000000000000000..ad48349d30b729b2290a578c25e1c4c3d215fc5e --- /dev/null +++ b/results/icml_r3/X28/mimo/study_m2/cert_co_c5.npz @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:a475533fb570be544e66ebd696b6a6d8219fae7c368609527e7ff56b4ccb37a5 +size 9431 diff --git a/results/icml_r3/X28/mimo/study_m2/cert_id_c1.npz b/results/icml_r3/X28/mimo/study_m2/cert_id_c1.npz new file mode 100644 index 0000000000000000000000000000000000000000..a5b675200b657eeae154cb71cceca7dc2b919304 --- /dev/null +++ b/results/icml_r3/X28/mimo/study_m2/cert_id_c1.npz @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:a65eaa8b6036193a293eade0fc030012c984d9323253634c7561897c59c308d0 +size 4640 diff --git a/results/icml_r3/X28/mimo/study_m2/cert_so3_c1.npz b/results/icml_r3/X28/mimo/study_m2/cert_so3_c1.npz new file mode 100644 index 0000000000000000000000000000000000000000..1824a982c526a9838d5dd259512fae059c81ef51 --- /dev/null +++ b/results/icml_r3/X28/mimo/study_m2/cert_so3_c1.npz @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:1486b8be5691efc688b16d9a1c4738f33baef7da5d907bf2c2f5fd0a03574255 +size 10442 diff --git a/results/icml_r3/X28/mimo/study_m2/cert_so3_c2.npz b/results/icml_r3/X28/mimo/study_m2/cert_so3_c2.npz new file mode 100644 index 0000000000000000000000000000000000000000..6e05f968f77e27dbf66f158494e63d393bce0ed4 --- /dev/null +++ b/results/icml_r3/X28/mimo/study_m2/cert_so3_c2.npz @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c266bd9aceb424566dd1815186ac9949a2f5a66f0389a8cf7fd16a899c83c1a0 +size 10363 diff --git a/results/icml_r3/X28/mimo/study_m2/cert_so3_c5.npz b/results/icml_r3/X28/mimo/study_m2/cert_so3_c5.npz new file mode 100644 index 0000000000000000000000000000000000000000..977c8714de5d4ef5276f126384543a63f330b93a --- /dev/null +++ b/results/icml_r3/X28/mimo/study_m2/cert_so3_c5.npz @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:803f648c059f56e36c0e9c33975ee3aa263b1cd43bee80626549e9d21c4fe4c8 +size 9902 diff --git a/results/icml_r3/X28/mimo/study_m2/cert_so_c1.npz b/results/icml_r3/X28/mimo/study_m2/cert_so_c1.npz new file mode 100644 index 0000000000000000000000000000000000000000..e65bb9d42f26a4c432059a4486d601f7c8de1767 --- /dev/null +++ b/results/icml_r3/X28/mimo/study_m2/cert_so_c1.npz @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:880b584f4aa8dda3cb60384f7304d32f3cab7d398431355960234b628e3056d5 +size 10401 diff --git a/results/icml_r3/X28/mimo/study_m2/cert_so_c2.npz b/results/icml_r3/X28/mimo/study_m2/cert_so_c2.npz new file mode 100644 index 0000000000000000000000000000000000000000..0fd334ea058d2768e717111ff37ff839b7812822 --- /dev/null +++ b/results/icml_r3/X28/mimo/study_m2/cert_so_c2.npz @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:9393b5802fe738f360758698354f1fbdbd2aa9f2d2090440f59ba9a767b75102 +size 10070 diff --git a/results/icml_r3/X28/mimo/study_m2/cert_so_c5.npz b/results/icml_r3/X28/mimo/study_m2/cert_so_c5.npz new file mode 100644 index 0000000000000000000000000000000000000000..21ada6180bd0b7e156895271cbe2d96da6187c4e --- /dev/null +++ b/results/icml_r3/X28/mimo/study_m2/cert_so_c5.npz @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:6a49de53c669227b34d12aad86b8e187e4d780718b6fe5da076ff77cce16d2e8 +size 8783 diff --git a/results/icml_r3/X28/mimo/study_m2/cert_sp_c1.npz b/results/icml_r3/X28/mimo/study_m2/cert_sp_c1.npz new file mode 100644 index 0000000000000000000000000000000000000000..8c0e71ec7bcddd895c3b2e4d6936bb1ecf32e283 --- /dev/null +++ b/results/icml_r3/X28/mimo/study_m2/cert_sp_c1.npz @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:1e25867cacf2cddd30613bc82c741b684d026966aede035084b6e2f844b0135c +size 12882 diff --git a/results/icml_r3/X28/mimo/study_m2/cert_sp_c2.npz b/results/icml_r3/X28/mimo/study_m2/cert_sp_c2.npz new file mode 100644 index 0000000000000000000000000000000000000000..e717dc0b5dcbb03a4db5337f81189d542ff6752b --- /dev/null +++ b/results/icml_r3/X28/mimo/study_m2/cert_sp_c2.npz @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:e32dfc5ed21868248d65ebb50199ad32bfd8a56ae759235057e4f2efc15dd2a6 +size 11672 diff --git a/results/icml_r3/X28/mimo/study_m2/cert_sp_c5.npz b/results/icml_r3/X28/mimo/study_m2/cert_sp_c5.npz new file mode 100644 index 0000000000000000000000000000000000000000..3139b255ce6d464a1e457fe1a7382e9a05f98759 --- /dev/null +++ b/results/icml_r3/X28/mimo/study_m2/cert_sp_c5.npz @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:b410cc1aafc942be8f16ea6cea1ad73c2cba6b3572240e3a38e5d4454f4d4e4e +size 9543 diff --git a/results/icml_r3/X28/mimo/study_m2/cf_c1.npz b/results/icml_r3/X28/mimo/study_m2/cf_c1.npz new file mode 100644 index 0000000000000000000000000000000000000000..b10b89a95977bf03986b8db58af98477616b50bf --- /dev/null +++ b/results/icml_r3/X28/mimo/study_m2/cf_c1.npz @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:cdc2d97d88c692bd6ab23efbc3ab3a8f495426f9625db7de096149c093ccb6fb +size 190274 diff --git a/results/icml_r3/X28/mimo/study_m2/cf_c2.npz b/results/icml_r3/X28/mimo/study_m2/cf_c2.npz new file mode 100644 index 0000000000000000000000000000000000000000..7d14c44f344a7635f4a2a0d27ffe50d9edd28180 --- /dev/null +++ b/results/icml_r3/X28/mimo/study_m2/cf_c2.npz @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c845d5c17501df0b735e3d5350b032db981d00baf1bded43ed5f8a39d88737ec +size 178307 diff --git a/results/icml_r3/X28/mimo/study_m2/cf_c5.npz b/results/icml_r3/X28/mimo/study_m2/cf_c5.npz new file mode 100644 index 0000000000000000000000000000000000000000..fb0e206e162548aaaf603d359679b05ebef9272b --- /dev/null +++ b/results/icml_r3/X28/mimo/study_m2/cf_c5.npz @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:882b1f0fc5d7c2a70caff25a6971ca695972bddc7f974d8ade77efdef264369c +size 147138 diff --git a/results/icml_r3/X28/mimo/study_m2/co_c1.npz b/results/icml_r3/X28/mimo/study_m2/co_c1.npz new file mode 100644 index 0000000000000000000000000000000000000000..4cbc975366621eb56cb570cfc761de0ef007cc5a --- /dev/null +++ b/results/icml_r3/X28/mimo/study_m2/co_c1.npz @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:8b31b0aa24574e307cf97d322b30170486faf13bd10ce0625c9b8dbf015ff1da +size 149657 diff --git a/results/icml_r3/X28/mimo/study_m2/co_c2.npz b/results/icml_r3/X28/mimo/study_m2/co_c2.npz new file mode 100644 index 0000000000000000000000000000000000000000..1a686621531236a38ff28f38d73a2d78107f6a03 --- /dev/null +++ b/results/icml_r3/X28/mimo/study_m2/co_c2.npz @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:7b2218a486f1ad06a7b22a2fc2f4e6844dd29ec340a8ee9dc36d4c537626fa6a +size 146320 diff --git a/results/icml_r3/X28/mimo/study_m2/co_c5.npz b/results/icml_r3/X28/mimo/study_m2/co_c5.npz new file mode 100644 index 0000000000000000000000000000000000000000..60e454f1c76cf8f7480c4bb6fad04b00e4b03ade --- /dev/null +++ b/results/icml_r3/X28/mimo/study_m2/co_c5.npz @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ad2ee7823780940c902e71c739249e7d5bcf84ec7a1f4b0e1d9bf8595ed0bcb1 +size 131528 diff --git a/results/icml_r3/X28/mimo/study_m2/id_c1.npz b/results/icml_r3/X28/mimo/study_m2/id_c1.npz new file mode 100644 index 0000000000000000000000000000000000000000..8992fd15a61710ed1c211d5add9b6f2e68d0db24 --- /dev/null +++ b/results/icml_r3/X28/mimo/study_m2/id_c1.npz @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:814c3e01ba6b7e9b50a1f1dbc83df307c74cb6f3cf52ce999331c54df24caa3b +size 52123 diff --git a/results/icml_r3/X28/mimo/study_m2/so3_c1.npz b/results/icml_r3/X28/mimo/study_m2/so3_c1.npz new file mode 100644 index 0000000000000000000000000000000000000000..5cc33cf7bc0f599531b722b05785dff64c0f6229 --- /dev/null +++ b/results/icml_r3/X28/mimo/study_m2/so3_c1.npz @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c705aef90e3c36336fed5a1cb975282ec441b2f84b6ad30544d96ded7e7dfe53 +size 149947 diff --git a/results/icml_r3/X28/mimo/study_m2/so3_c2.npz b/results/icml_r3/X28/mimo/study_m2/so3_c2.npz new file mode 100644 index 0000000000000000000000000000000000000000..25a87a2886109249bbbaf3827658d5294f3e4fad --- /dev/null +++ b/results/icml_r3/X28/mimo/study_m2/so3_c2.npz @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:8761afaf6833cb651cc93d1a6af6d19411ef4c6fc1e7493958c0df706cc51a9c +size 148359 diff --git a/results/icml_r3/X28/mimo/study_m2/so3_c5.npz b/results/icml_r3/X28/mimo/study_m2/so3_c5.npz new file mode 100644 index 0000000000000000000000000000000000000000..b256e707cc5204cacf2d6fecb1c4278f5a3e94d7 --- /dev/null +++ b/results/icml_r3/X28/mimo/study_m2/so3_c5.npz @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:862c8f416f0677742a8135df96dd1031bf8ae46f821cdf0554163e8bc4b1ab4e +size 139918 diff --git a/results/icml_r3/X28/mimo/study_m2/so_c1.npz b/results/icml_r3/X28/mimo/study_m2/so_c1.npz new file mode 100644 index 0000000000000000000000000000000000000000..5582e36a9a982d43703e186f4ae2a6e655ee1f69 --- /dev/null +++ b/results/icml_r3/X28/mimo/study_m2/so_c1.npz @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:52deb33f8b39f3139bbe1d7baa6b9fc93c4fad09b8c4045423987121684e29c1 +size 148666 diff --git a/results/icml_r3/X28/mimo/study_m2/so_c2.npz b/results/icml_r3/X28/mimo/study_m2/so_c2.npz new file mode 100644 index 0000000000000000000000000000000000000000..bcfeabae6876e8c632a0c327c193610294276052 --- /dev/null +++ b/results/icml_r3/X28/mimo/study_m2/so_c2.npz @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:3548fb5f03c2f9e85ab4bb1439fbd23051125cbee7cdcb2a23014126519cb146 +size 143106 diff --git a/results/icml_r3/X28/mimo/study_m2/so_c5.npz b/results/icml_r3/X28/mimo/study_m2/so_c5.npz new file mode 100644 index 0000000000000000000000000000000000000000..e64526321e1514458eb1f55f3ede83506eecab10 --- /dev/null +++ b/results/icml_r3/X28/mimo/study_m2/so_c5.npz @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:c7a176cc73a515295e0998491f277081614dec0375f2470c5c7510c65a7cb2a9 +size 122159 diff --git a/results/icml_r3/X28/mimo/study_m2/sp_c1.npz b/results/icml_r3/X28/mimo/study_m2/sp_c1.npz new file mode 100644 index 0000000000000000000000000000000000000000..09130a4be69d33a90df21a8cf717fdd549f93a8a --- /dev/null +++ b/results/icml_r3/X28/mimo/study_m2/sp_c1.npz @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:3be9290e2f43d81999ea73bf394bbaf48dbc141221d4181bbfe3f502e3df8e22 +size 185738 diff --git a/results/icml_r3/X28/mimo/study_m2/sp_c2.npz b/results/icml_r3/X28/mimo/study_m2/sp_c2.npz new file mode 100644 index 0000000000000000000000000000000000000000..bfa40ad6c2f15b9e3f19904a7a5794a4ddf5dd50 --- /dev/null +++ b/results/icml_r3/X28/mimo/study_m2/sp_c2.npz @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:2ec15b16c9e853f26989e91f1e998ce2b5b4c047d03be8ae8b0fc7091d6806f6 +size 168717 diff --git a/results/icml_r3/X28/mimo/study_m2/sp_c5.npz b/results/icml_r3/X28/mimo/study_m2/sp_c5.npz new file mode 100644 index 0000000000000000000000000000000000000000..ee8a0051b7a002fd11cfc3a2a7e110bd90289647 --- /dev/null +++ b/results/icml_r3/X28/mimo/study_m2/sp_c5.npz @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:5b2dbee4ab4f0e8c9bfd80bac6c16958fa283fb82495ed900f1c9dc0cba2926a +size 134719 diff --git a/results/icml_r3/X28/mimo/study_m3/cert_cf_c1.npz b/results/icml_r3/X28/mimo/study_m3/cert_cf_c1.npz new file mode 100644 index 0000000000000000000000000000000000000000..a921095eb6abaae274487f2b8d92c0f4c7fee7a1 --- /dev/null +++ b/results/icml_r3/X28/mimo/study_m3/cert_cf_c1.npz @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:eefa4f1ab1e26bfb9eb36d307bad5b056b469a0760fdaee0694570af129e920f +size 20254 diff --git a/results/icml_r3/X28/mimo/study_m3/cert_cf_c2.npz b/results/icml_r3/X28/mimo/study_m3/cert_cf_c2.npz new file mode 100644 index 0000000000000000000000000000000000000000..17cf11fb1ebb6cb5a245dc6ad2f835a68fb9dd27 --- /dev/null +++ b/results/icml_r3/X28/mimo/study_m3/cert_cf_c2.npz @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:5391ecdc7205981c788541ce7e1b61cd1a8dc92d7350f92ded83e9aed7b32b23 +size 18877 diff --git a/results/icml_r3/X28/mimo/study_m3/cert_cf_c5.npz b/results/icml_r3/X28/mimo/study_m3/cert_cf_c5.npz new file mode 100644 index 0000000000000000000000000000000000000000..03cfff86ec842a34e758bdc2f9707c943bb198a4 --- /dev/null +++ b/results/icml_r3/X28/mimo/study_m3/cert_cf_c5.npz @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:ad9e0a57376abc7ae803392cb8fd28247cc1ddd3f8f8b538e96e628ef43fa6bb +size 15892 diff --git a/results/icml_r3/X28/mimo/study_m3/cert_co_c1.npz b/results/icml_r3/X28/mimo/study_m3/cert_co_c1.npz new file mode 100644 index 0000000000000000000000000000000000000000..a4dd0acb0cf9af7acb9ae982d2b7bbc26f8ced23 --- /dev/null +++ b/results/icml_r3/X28/mimo/study_m3/cert_co_c1.npz @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:96061cb3f7b4f0535505784bbb9aa21667f1fa58cd09c85a26278a391ef11b52 +size 15750 diff --git a/results/icml_r3/X28/mimo/study_m3/cert_co_c2.npz b/results/icml_r3/X28/mimo/study_m3/cert_co_c2.npz new file mode 100644 index 0000000000000000000000000000000000000000..6e0d3037eb44ca63708bee848604faa22f385037 --- /dev/null +++ b/results/icml_r3/X28/mimo/study_m3/cert_co_c2.npz @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:fa45113107c5bbc6ed23b4b44413900ad7a088dad1303bd685c13d0bc675f4f6 +size 15544 diff --git a/results/icml_r3/X28/mimo/study_m3/cert_co_c5.npz b/results/icml_r3/X28/mimo/study_m3/cert_co_c5.npz new file mode 100644 index 0000000000000000000000000000000000000000..9b1cf090da441a0562f040d4f8ebe7ada0a8656e --- /dev/null +++ b/results/icml_r3/X28/mimo/study_m3/cert_co_c5.npz @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:112ca6a92348fb6bfce62c24e6152de4d6af526ad64d87f581e13c9a3f4bcaeb +size 14286 diff --git a/results/icml_r3/X28/mimo/study_m3/cert_id_c1.npz b/results/icml_r3/X28/mimo/study_m3/cert_id_c1.npz new file mode 100644 index 0000000000000000000000000000000000000000..40d4c5a9915c66bdd5b1320457f241562b4e268e --- /dev/null +++ b/results/icml_r3/X28/mimo/study_m3/cert_id_c1.npz @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:7db0e45e90346e76502b74634beab1fdbc8098b622ef32efa0c9a0d7e38a473d +size 7130 diff --git a/results/icml_r3/X28/mimo/study_m3/cert_so3_c1.npz b/results/icml_r3/X28/mimo/study_m3/cert_so3_c1.npz new file mode 100644 index 0000000000000000000000000000000000000000..ba7bf6e36b1d5c4f2b8a41fdd17b3cf121b1221f --- /dev/null +++ b/results/icml_r3/X28/mimo/study_m3/cert_so3_c1.npz @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:346705e7bee0eba6d82af49554a27e4cf6992a43e1fe56172bdee2a1cfeb954e +size 15783 diff --git a/results/icml_r3/X28/mimo/study_m3/cert_so3_c2.npz b/results/icml_r3/X28/mimo/study_m3/cert_so3_c2.npz new file mode 100644 index 0000000000000000000000000000000000000000..f30ab4e21df4a0ea4478825058ae33b5d77a2f6d --- /dev/null +++ b/results/icml_r3/X28/mimo/study_m3/cert_so3_c2.npz @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:56d70af828f16f199b1c319453f21c45ba4c09a15b6643b1f8f97d8471ddc57d +size 15688 diff --git a/results/icml_r3/X28/mimo/study_m3/cert_so3_c5.npz b/results/icml_r3/X28/mimo/study_m3/cert_so3_c5.npz new file mode 100644 index 0000000000000000000000000000000000000000..925c6efbf1cabc7a1c378fb6bb82e003a493915d --- /dev/null +++ b/results/icml_r3/X28/mimo/study_m3/cert_so3_c5.npz @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d1b62b39e0ff516b4fb23f18a6ec81918e0738b69ddec1fa6dfcb82b828514cc +size 14997 diff --git a/results/icml_r3/X28/mimo/study_m3/cert_so_c1.npz b/results/icml_r3/X28/mimo/study_m3/cert_so_c1.npz new file mode 100644 index 0000000000000000000000000000000000000000..2719e816ff887c84539a7ba0b5383d0eebfd18cf --- /dev/null +++ b/results/icml_r3/X28/mimo/study_m3/cert_so_c1.npz @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:261e270505cae69d894850a6674057fce9187846c2094244a1eed87a8b85b8ef +size 15726 diff --git a/results/icml_r3/X28/mimo/study_m3/cert_so_c2.npz b/results/icml_r3/X28/mimo/study_m3/cert_so_c2.npz new file mode 100644 index 0000000000000000000000000000000000000000..7a2923adf57288cd14b1586e499d6e5a9cf5d60e --- /dev/null +++ b/results/icml_r3/X28/mimo/study_m3/cert_so_c2.npz @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:2cca5be247babfcfdc8f8fe470ef6845376c17cfb222c02dc374967b2d617149 +size 15394 diff --git a/results/icml_r3/X28/mimo/study_m3/cert_so_c5.npz b/results/icml_r3/X28/mimo/study_m3/cert_so_c5.npz new file mode 100644 index 0000000000000000000000000000000000000000..0ad526ceb9f3ebf352f4f51d0d940edf6d9e9672 --- /dev/null +++ b/results/icml_r3/X28/mimo/study_m3/cert_so_c5.npz @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:92ed7639eda8cfce78b79a1199f6f4f233ad505d576f624ce20e68617b2fdcaa +size 13339 diff --git a/results/icml_r3/X28/mimo/study_m3/cert_sp_c1.npz b/results/icml_r3/X28/mimo/study_m3/cert_sp_c1.npz new file mode 100644 index 0000000000000000000000000000000000000000..74d43d287c0b51173cf3b96016b750a28af7c192 --- /dev/null +++ b/results/icml_r3/X28/mimo/study_m3/cert_sp_c1.npz @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:6ff19de135741f7b190aa3441830835d42249a1597fbdc37b9d07a1ddb1e7c06 +size 19789 diff --git a/results/icml_r3/X28/mimo/study_m3/cert_sp_c2.npz b/results/icml_r3/X28/mimo/study_m3/cert_sp_c2.npz new file mode 100644 index 0000000000000000000000000000000000000000..93cde9da78ae88b1eda08ae340593140abb663c6 --- /dev/null +++ b/results/icml_r3/X28/mimo/study_m3/cert_sp_c2.npz @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:65dc6bcc1a70e27966ff2c1347efd9735193e92f4e7cce657401eb954596fe98 +size 17957 diff --git a/results/icml_r3/X28/mimo/study_m3/cert_sp_c5.npz b/results/icml_r3/X28/mimo/study_m3/cert_sp_c5.npz new file mode 100644 index 0000000000000000000000000000000000000000..1937035787344456334ea2301dde323165edcab2 --- /dev/null +++ b/results/icml_r3/X28/mimo/study_m3/cert_sp_c5.npz @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:874d30429970c43cff3f7d212284e8f13182a171ba884a0af4842e87661e943b +size 14292 diff --git a/results/icml_r3/X28/mimo/study_m3/cf_c1.npz b/results/icml_r3/X28/mimo/study_m3/cf_c1.npz new file mode 100644 index 0000000000000000000000000000000000000000..e42c78ba32c288eaed72e45d7d99cd67e4178832 --- /dev/null +++ b/results/icml_r3/X28/mimo/study_m3/cf_c1.npz @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:0000dda48123f5029f6ad9640394a56f11bb5595f911abfa14cb10eb1a40a2b0 +size 311005 diff --git a/results/icml_r3/X28/mimo/study_m3/cf_c2.npz b/results/icml_r3/X28/mimo/study_m3/cf_c2.npz new file mode 100644 index 0000000000000000000000000000000000000000..58cafed1343f7fc9c8ee26049237a74d8632c033 --- /dev/null +++ b/results/icml_r3/X28/mimo/study_m3/cf_c2.npz @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:a6f430ee78d0e470ae0e5120867c6cbcb485061532ac250fcf6b179bbc66fd71 +size 293551 diff --git a/results/icml_r3/X28/mimo/study_m3/cf_c5.npz b/results/icml_r3/X28/mimo/study_m3/cf_c5.npz new file mode 100644 index 0000000000000000000000000000000000000000..0719b4aed8057e4707b588e9c34169304aee67c1 --- /dev/null +++ b/results/icml_r3/X28/mimo/study_m3/cf_c5.npz @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d4270a2529136ec883e2478a5da4d34512b560f88ca4bfacb6bb77db11bcbd0d +size 241535 diff --git a/results/icml_r3/X28/mimo/study_m3/co_c1.npz b/results/icml_r3/X28/mimo/study_m3/co_c1.npz new file mode 100644 index 0000000000000000000000000000000000000000..d8ae0867d06afdac11c81c117e25b8533f7a9578 --- /dev/null +++ b/results/icml_r3/X28/mimo/study_m3/co_c1.npz @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:99a549200113c198f1274d99a577a1d02e40e891bf937c7552b9c69ad0d3222b +size 243464 diff --git a/results/icml_r3/X28/mimo/study_m3/co_c2.npz b/results/icml_r3/X28/mimo/study_m3/co_c2.npz new file mode 100644 index 0000000000000000000000000000000000000000..8bbbc6a77f64755d6e5c4d55b4f8ff5928035865 --- /dev/null +++ b/results/icml_r3/X28/mimo/study_m3/co_c2.npz @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:e3192ffd7501c23e5f131255973b33f8133a10e51e4afe749a30c7a8c583e467 +size 239709 diff --git a/results/icml_r3/X28/mimo/study_m3/co_c5.npz b/results/icml_r3/X28/mimo/study_m3/co_c5.npz new file mode 100644 index 0000000000000000000000000000000000000000..20c6e59a832d428ac03021e8e07c2436105285f1 --- /dev/null +++ b/results/icml_r3/X28/mimo/study_m3/co_c5.npz @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:286d4c43fd3ce3f0e4cb2b104a9a6f3d63ce840b0c98f5b9dbfe75053ce9dd42 +size 216727 diff --git a/results/icml_r3/X28/mimo/study_m3/id_c1.npz b/results/icml_r3/X28/mimo/study_m3/id_c1.npz new file mode 100644 index 0000000000000000000000000000000000000000..14fe659706770d3671bab74ee1097271f44f3281 --- /dev/null +++ b/results/icml_r3/X28/mimo/study_m3/id_c1.npz @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:441d29e6f368a349ed2e2e0b7ff2dfb938f74372427ecbfe641bb8eb6ed8c351 +size 98991 diff --git a/results/icml_r3/X28/mimo/study_m3/so3_c1.npz b/results/icml_r3/X28/mimo/study_m3/so3_c1.npz new file mode 100644 index 0000000000000000000000000000000000000000..32c441738e49e78bbb00de412b70a73d3926f588 --- /dev/null +++ b/results/icml_r3/X28/mimo/study_m3/so3_c1.npz @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:25f57b1f246b90bf02a0a7d32a022d4cffa6160c9e0af25a51d45b2e095830c5 +size 243579 diff --git a/results/icml_r3/X28/mimo/study_m3/so3_c2.npz b/results/icml_r3/X28/mimo/study_m3/so3_c2.npz new file mode 100644 index 0000000000000000000000000000000000000000..8f7165614d3e412d0983e783c3f30d7fb95e5aba --- /dev/null +++ b/results/icml_r3/X28/mimo/study_m3/so3_c2.npz @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:89e283527e7d1894f8c2fdc27ff10c2ea283291673ee0af50899db32f7ae5fa8 +size 241949 diff --git a/results/icml_r3/X28/mimo/study_m3/so3_c5.npz b/results/icml_r3/X28/mimo/study_m3/so3_c5.npz new file mode 100644 index 0000000000000000000000000000000000000000..1d31d034c3f724a90ea3b7aca046240b981704ef --- /dev/null +++ b/results/icml_r3/X28/mimo/study_m3/so3_c5.npz @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:4b2e921d0d24689ccc2d3a5c0713a06912ab6a8f4596a4b670c15b1adee3bb53 +size 230484 diff --git a/results/icml_r3/X28/mimo/study_m3/so_c1.npz b/results/icml_r3/X28/mimo/study_m3/so_c1.npz new file mode 100644 index 0000000000000000000000000000000000000000..09e3bea81c4c5b516132e3e3349c1cf1c4e42111 --- /dev/null +++ b/results/icml_r3/X28/mimo/study_m3/so_c1.npz @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:04d4186e33401aa707e8dd4c7274a7a97e769535fa252b9965f42465371453f6 +size 243168 diff --git a/results/icml_r3/X28/mimo/study_m3/so_c2.npz b/results/icml_r3/X28/mimo/study_m3/so_c2.npz new file mode 100644 index 0000000000000000000000000000000000000000..67e74619b52c87100d043d916c7dfd572169c139 --- /dev/null +++ b/results/icml_r3/X28/mimo/study_m3/so_c2.npz @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:234f13ba16010aa5398b0b7c2d844bf14b5ee551ced2b34587109b560ccdd77c +size 236213 diff --git a/results/icml_r3/X28/mimo/study_m3/so_c5.npz b/results/icml_r3/X28/mimo/study_m3/so_c5.npz new file mode 100644 index 0000000000000000000000000000000000000000..53752a7d20caf395bbed5b330ba4f3ff37fb14bd --- /dev/null +++ b/results/icml_r3/X28/mimo/study_m3/so_c5.npz @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:3c8ebd32bb778fc557060a138c5a0c8292b4f3021f5aaf242c9869fa4a19f035 +size 204654 diff --git a/results/icml_r3/X28/mimo/study_m3/sp_c1.npz b/results/icml_r3/X28/mimo/study_m3/sp_c1.npz new file mode 100644 index 0000000000000000000000000000000000000000..c8475783cae158753b0645702d534cc0644d0279 --- /dev/null +++ b/results/icml_r3/X28/mimo/study_m3/sp_c1.npz @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:681ee4db4b187b5cdce0b3bd646a85c67cade900ae9849fed0d09a2fe2f00928 +size 305707 diff --git a/results/icml_r3/X28/mimo/study_m3/sp_c2.npz b/results/icml_r3/X28/mimo/study_m3/sp_c2.npz new file mode 100644 index 0000000000000000000000000000000000000000..983d545c06eefd1c56f0dce56eee0eca34105ae7 --- /dev/null +++ b/results/icml_r3/X28/mimo/study_m3/sp_c2.npz @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:9baa0862dd0259378e28490c11d0bd21c3549d931a38ad2df6607bb438ca7b1c +size 280406 diff --git a/results/icml_r3/X28/mimo/study_m3/sp_c5.npz b/results/icml_r3/X28/mimo/study_m3/sp_c5.npz new file mode 100644 index 0000000000000000000000000000000000000000..cfd6f0535a148875d91efc0ebedf9d2bcc624a0c --- /dev/null +++ b/results/icml_r3/X28/mimo/study_m3/sp_c5.npz @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:1192fbe57088198b0e2cfdea3ec1dae4a3e446c935268d536383af6ad72de318 +size 224684 diff --git a/results/icml_r3/X28/val_clip_choice.json b/results/icml_r3/X28/val_clip_choice.json index bf52c2b780cb780e4b33e6935b9625b43b7006bb..5f0a4557122235e33e86b4b30b2e0fd5fd818df8 100644 --- a/results/icml_r3/X28/val_clip_choice.json +++ b/results/icml_r3/X28/val_clip_choice.json @@ -1,37 +1,255 @@ { - "x28_clS_s0": { - "clip": "1.5", - "reg": 10.91, - "cert": 92.5 - }, - "x28_clS_s1": { - "clip": "2", - "reg": 9.4, - "cert": 95.0 + "x28_clS_s0_best": { + "clip": "1", + "cert": 100.0, + "reg": 11.06, + "grid": { + "1": [ + 100.0, + 11.06 + ], + "1.5": [ + 92.5, + 10.91 + ], + "2": [ + 90.0, + 11.37 + ], + "3": [ + 90.0, + 11.94 + ] + } }, - "x28_clS_s2": { + "x28_clS_s1_best": { "clip": "1.5", - "reg": 11.6, - "cert": 95.0 + "cert": 100.0, + "reg": 9.8, + "grid": { + "1": [ + 100.0, + 12.0 + ], + "1.5": [ + 100.0, + 9.8 + ], + "2": [ + 95.0, + 9.4 + ], + "3": [ + 97.5, + 9.89 + ] + } + }, + "x28_clS_s2_best": { + "clip": "1", + "cert": 100.0, + "reg": 12.62, + "grid": { + "1": [ + 100.0, + 12.62 + ], + "1.5": [ + 95.0, + 11.6 + ], + "2": [ + 97.5, + 12.88 + ], + "3": [ + 90.0, + 12.65 + ] + } }, - "x28_coefS_s0": { + "x28_coefS_s0_best": { "clip": "3", + "cert": 90.0, "reg": 9.07, - "cert": 90.0 + "grid": { + "1": [ + 67.5, + 151.65 + ], + "1.5": [ + 80.0, + 22.97 + ], + "2": [ + 87.5, + 10.47 + ], + "3": [ + 90.0, + 9.07 + ] + } }, - "x21_fgen_clL": { + "x21_fgen_clL_best_tr": { "clip": "1.5", + "cert": 100.0, "reg": 6.87, - "cert": 100.0 + "grid": { + "1": [ + 100.0, + 8.67 + ], + "1.5": [ + 100.0, + 6.87 + ], + "2": [ + 100.0, + 8.12 + ], + "3": [ + 100.0, + 8.3 + ] + } }, - "x21_fgen_clL_s1": { + "x21_fgen_clL_s1_best_tr": { "clip": "1.5", + "cert": 100.0, "reg": 7.15, - "cert": 100.0 + "grid": { + "1": [ + 100.0, + 11.12 + ], + "1.5": [ + 100.0, + 7.15 + ], + "2": [ + 97.5, + 8.35 + ], + "3": [ + 100.0, + 8.55 + ] + } }, - "x21_fgen_clL_s2": { + "x21_fgen_clL_s2_best_tr": { "clip": "1.5", + "cert": 100.0, "reg": 6.27, - "cert": 100.0 + "grid": { + "1": [ + 100.0, + 9.4 + ], + "1.5": [ + 100.0, + 6.27 + ], + "2": [ + 100.0, + 7.08 + ], + "3": [ + 100.0, + 7.12 + ] + } + }, + "x28_clS_s0_best_tr": { + "clip": "1.5", + "cert": 100.0, + "reg": 9.79, + "grid": { + "1": [ + 100.0, + 10.83 + ], + "1.5": [ + 100.0, + 9.79 + ], + "2": [ + 97.5, + 9.7 + ], + "3": [ + 97.5, + 9.84 + ] + } + }, + "x28_clS_s1_best_tr": { + "clip": "1.5", + "cert": 100.0, + "reg": 9.57, + "grid": { + "1": [ + 100.0, + 12.0 + ], + "1.5": [ + 100.0, + 9.57 + ], + "2": [ + 95.0, + 9.08 + ], + "3": [ + 97.5, + 9.38 + ] + } + }, + "x28_clS_s2_best_tr": { + "clip": "1.5", + "cert": 100.0, + "reg": 8.56, + "grid": { + "1": [ + 100.0, + 12.62 + ], + "1.5": [ + 100.0, + 8.56 + ], + "2": [ + 100.0, + 12.54 + ], + "3": [ + 95.0, + 12.05 + ] + } + }, + "x28_coefS_s0_best_tr": { + "clip": "3", + "cert": 90.0, + "reg": 9.07, + "grid": { + "1": [ + 67.5, + 137.13 + ], + "1.5": [ + 80.0, + 22.97 + ], + "2": [ + 87.5, + 10.47 + ], + "3": [ + 90.0, + 9.07 + ] + } } } \ No newline at end of file