Add files using upload-large-folder tool
Browse filesThis view is limited to 50 files because it contains too many changes. See raw diff
- results/baselines/bank_density_mimo.txt +59 -0
- results/baselines/pendulum_bank_density_diagnosis_summary.txt +59 -0
- results/baselines/retrieval_mimo.txt +36 -0
- results/baselines/siso_baselines.txt +12 -0
- results/furuta_v4/ablation_unetAseed1_ema999_unetAseed2_ema999_unetA_ema999_unetS_ema99.txt +8 -0
- results/furuta_v4/budget_curves_ddim10.txt +27 -0
- results/furuta_v4/budget_unetA_ema999_ddim10_n200.txt +24 -0
- results/furuta_v4/budget_unetAseed1_ema999_n200.txt +24 -0
- results/furuta_v4/ddim_unetA_ema999_20.txt +2 -0
- results/furuta_v4/ddim_unetAseed1_ema999_10.txt +2 -0
- results/furuta_v4/ddim_unetAseed1_ema999_20.txt +2 -0
- results/furuta_v4/ddim_unetAseed1_ema999_5.txt +2 -0
- results/furuta_v4/gate_mlp384k_ema999.txt +2 -0
- results/furuta_v4/gate_mlpB256_ema999.txt +2 -0
- results/furuta_v4/gate_unetA_ema9995.txt +2 -0
- results/furuta_v4/gate_unetAseed2_ema999.txt +2 -0
- results/furuta_v4/gate_unetS_ema999.txt +2 -0
- results/furuta_v4/memo_mlp384k_ema999.txt +7 -0
- results/furuta_v4/perseed.txt +13 -0
- results/furuta_v4/rank_mlp384k_ema999.txt +32 -0
- results/furuta_v4/refine_mlpA_ema999_summary.txt +12 -0
- results/furuta_v4/reg_attribution_partial.txt +36 -0
- results/furuta_v4/run_gates.sh +6 -0
- results/furuta_v4/scan_mlp_a.txt +19 -0
- results/furuta_v4/scan_unet_a.txt +9 -0
- results/furuta_v4/traj_gen_warmstart_v4unetA_s0.txt +17 -0
- results/furuta_v4/traj_gen_warmstart_v4unetA_s1.txt +17 -0
- results/guidance/A16x_results.txt +13 -0
- results/guidance/A_ddim_check.txt +1 -0
- results/guidance/A_results.txt +47 -0
- results/guidance/B_ddim_check.txt +5 -0
- results/guidance/B_results.txt +16 -0
- results/guidance/pilot_choice.json +17 -0
- results/icml_r3/RESUME.md +35 -0
- results/icml_r3/RULES.md +44 -0
- results/ood/furuta_ood.txt +165 -0
- results/ood/pendulum_ood.txt +804 -0
- results/ood/siso_ood.txt +165 -0
- results/ood/train_holes.sh +6 -0
- results/pendulum_full/dp_bank_rank.txt +23 -0
- results/pendulum_full/dp_baseline.txt +25 -0
- results/pendulum_full/dp_interp.txt +28 -0
- results/pendulum_full/dp_interp4.txt +19 -0
- results/pendulum_full/dp_interp_kappa.txt +57 -0
- results/pendulum_full/dp_target_eval.txt +24 -0
- results/pendulum_full/family_sweep.txt +60 -0
- results/refine_reference/refine_reference.png +0 -0
- results/timing/classical_mimo_cpu1.json +253 -0
- results/timing/classical_mimo_cpu1q.json +253 -0
- results/timing/classical_mimo_cpu8.json +260 -0
results/baselines/bank_density_mimo.txt
ADDED
|
@@ -0,0 +1,59 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
=== BANK-DENSITY SWEEP, MIMO (held-out draws: seeds [9999, 4242], N=[120, 120], pooled N=240) ===
|
| 2 |
+
MODEL SIDE IS FIXED: full-data checkpoint, NOT retrained per bank fraction (the pendulum sweep retrained; this one does not).
|
| 3 |
+
bank = seeded random subset of training PLANTS (all stored controllers of the chosen plants); fractions (0.05, 0.1, 0.25, 0.5, 1.0); subset seeds (0, 1, 2) (f=1.0: single bank);
|
| 4 |
+
model draws: sampler seeds (0, 1, 2); McNemar/regret reference = model seed 0. Verifier/x0/N as retrieval_baselines.py mimo; K=8 and K=1 (K=1 = first candidate).
|
| 5 |
+
oracle: retrieval_baselines.mimo_oracle (150-step batched Adam, arm-independent)
|
| 6 |
+
regret % = (J_best - J_oracle)/J_oracle on the COMMON success set (plants where BOTH the arm and the model seed-0 draw are covered at that K); label 'cond.'.
|
| 7 |
+
coverage 95% CI: bootstrap (5000) over pooled plants of subset-seed 0. McNemar: exact, paired on pooled plants vs model seed 0, reported for each subset seed (min..max p).
|
| 8 |
+
|
| 9 |
+
----- K=8 ----- MODEL coverage 0.996 +- 0.000 (3 draws), seed-0 CI [0.988,1.000], per-sample 0.961, cond. regret med 2.27%, gen 54.28 ms/ctrl
|
| 10 |
+
per-draw model coverage (seed 0/1/2): seed 9999: 1.00/1.00/1.00; seed 4242: 0.99/0.99/0.99
|
| 11 |
+
arm=retrieval
|
| 12 |
+
f cov mean+-sd CI(ss0) per-samp dCov(ret-model) McNemar p(ss0..) reg_med% cond model reg% same set n_common NN dist
|
| 13 |
+
0.05 0.996+-0.003 [1.000,1.000] 0.914 +0.000 1..1 2.68 2.27 238 0.1747
|
| 14 |
+
0.10 0.994+-0.002 [0.988,1.000] 0.915 -0.001 1..1 2.76 2.27 238 0.1657
|
| 15 |
+
0.25 0.993+-0.002 [0.988,1.000] 0.927 -0.003 1..1 2.55 2.27 238 0.1540
|
| 16 |
+
0.50 0.993+-0.002 [0.988,1.000] 0.933 -0.003 1..1 2.59 2.27 238 0.1454
|
| 17 |
+
1.00 0.992+-0.000 [0.979,1.000] 0.934 -0.004 1..1 2.79 2.28 238 0.1376
|
| 18 |
+
arm=blend
|
| 19 |
+
f cov mean+-sd CI(ss0) per-samp dCov(ret-model) McNemar p(ss0..) reg_med% cond model reg% same set n_common NN dist
|
| 20 |
+
0.05 0.993+-0.004 [0.988,1.000] 0.916 -0.003 0.5..1 2.85 2.27 237 0.1747
|
| 21 |
+
0.10 0.993+-0.004 [0.988,1.000] 0.918 -0.003 0.5..1 2.85 2.27 238 0.1657
|
| 22 |
+
0.25 0.992+-0.000 [0.979,1.000] 0.930 -0.004 1..1 2.59 2.27 238 0.1540
|
| 23 |
+
0.50 0.993+-0.002 [0.988,1.000] 0.936 -0.003 1..1 2.75 2.27 238 0.1454
|
| 24 |
+
1.00 0.992+-0.000 [0.979,1.000] 0.940 -0.004 1..1 2.87 2.28 238 0.1376
|
| 25 |
+
|
| 26 |
+
----- K=1 ----- MODEL coverage 0.963 +- 0.000 (3 draws), seed-0 CI [0.938,0.983], per-sample 0.963, cond. regret med 11.62%, gen 54.28 ms/ctrl
|
| 27 |
+
per-draw model coverage (seed 0/1/2): seed 9999: 0.94/0.94/0.94; seed 4242: 0.98/0.98/0.98
|
| 28 |
+
arm=retrieval (blend K=1 == retrieval K=1 by construction)
|
| 29 |
+
f cov mean+-sd CI(ss0) per-samp dCov(ret-model) McNemar p(ss0..) reg_med% cond model reg% same set n_common NN dist
|
| 30 |
+
0.05 0.932+-0.002 [0.900,0.963] 0.932 -0.031 0.12..0.17 13.62 11.68 218 0.1747
|
| 31 |
+
0.10 0.922+-0.019 [0.896,0.958] 0.922 -0.040 0.00086..0.23 15.73 11.52 217 0.1657
|
| 32 |
+
0.25 0.925+-0.014 [0.892,0.958] 0.925 -0.037 0.015..0.23 15.29 11.61 217 0.1540
|
| 33 |
+
0.50 0.933+-0.009 [0.912,0.971] 0.933 -0.029 0.031..0.27 14.49 11.70 220 0.1454
|
| 34 |
+
1.00 0.908+-0.000 [0.871,0.942] 0.908 -0.054 0.0044..0.0044 13.81 11.48 215 0.1376
|
| 35 |
+
|
| 36 |
+
----- retrieval coverage K=8 vs K=1 per f (mean over subset seeds; pooled draws) -----
|
| 37 |
+
f K=1 K=8
|
| 38 |
+
0.05 0.932 0.996
|
| 39 |
+
0.10 0.922 0.994
|
| 40 |
+
0.25 0.925 0.993
|
| 41 |
+
0.50 0.933 0.993
|
| 42 |
+
1.00 0.908 0.992
|
| 43 |
+
|
| 44 |
+
----- smallest f at which retrieval MATCHES the model -----
|
| 45 |
+
rule (fixed before running): mean retrieval coverage >= model coverage - 0.05 AND pooled McNemar p >= 0.05 for >= 2 of 3 subset seeds (f=1.0: its single bank); 'never' if no f qualifies.
|
| 46 |
+
mimo arm=retrieval K=8: f=0.05
|
| 47 |
+
mimo arm=blend K=8: f=0.05
|
| 48 |
+
mimo arm=retrieval K=1: f=0.05
|
| 49 |
+
|
| 50 |
+
----- NN distance vs f (normalized conditioning space, mean over queries; mean over subset seeds) -----
|
| 51 |
+
f=0.05: draw9999: 0.1782, draw4242: 0.1713
|
| 52 |
+
f=0.1: draw9999: 0.1683, draw4242: 0.1631
|
| 53 |
+
f=0.25: draw9999: 0.1561, draw4242: 0.1519
|
| 54 |
+
f=0.5: draw9999: 0.1483, draw4242: 0.1424
|
| 55 |
+
f=1.0: draw9999: 0.1411, draw4242: 0.1342
|
| 56 |
+
|
| 57 |
+
ADDENDUM: per-held-out-draw K=8 coverage (mean over subset seeds; model = seed-0 sampler / 3-seed mean)
|
| 58 |
+
draw0: model 1.000; retrieval f=0.05: 0.994 f=0.1: 0.994 f=0.25: 0.994 f=0.5: 0.994 f=1.0: 0.992
|
| 59 |
+
draw1: model 0.992; retrieval f=0.05: 0.997 f=0.1: 0.994 f=0.25: 0.992 f=0.5: 0.992 f=1.0: 0.992
|
results/baselines/pendulum_bank_density_diagnosis_summary.txt
ADDED
|
@@ -0,0 +1,59 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
=== PENDULUM bank-density anomaly: diagnosis summary ===
|
| 2 |
+
Anomaly: in pendulum_bank_density.txt, BOTH retrieval and the model get WORSE (higher median
|
| 3 |
+
regret) as f increases (0.25 -> 0.5 -> 1.0), on both held-out draws. Since every f<1.0 bank is a
|
| 4 |
+
subset of the f=1.0 pool, retrieval getting worse with more candidates available is not possible
|
| 5 |
+
via "closer neighbour = better transfer" alone. Three checks, in order:
|
| 6 |
+
|
| 7 |
+
1) Does nearest-neighbour DISTANCE shrink with f, as the geometry demands?
|
| 8 |
+
YES, monotonically and cleanly (median NN-distance in standardized (l,m,b) space):
|
| 9 |
+
f=0.25 (n=650): 0.2496
|
| 10 |
+
f=0.50 (n=1300): 0.1914
|
| 11 |
+
f=1.00 (n=2600): 0.1691
|
| 12 |
+
So the geometric premise holds: denser banks give closer matches, exactly as expected.
|
| 13 |
+
|
| 14 |
+
2) Does retrieval regret track that shrinking distance? (diagnostic a: K=1, no best-of-K selection)
|
| 15 |
+
NO. K=1 median regret vs f (mean +/- sd over 3 seeds):
|
| 16 |
+
seed123: f=0.25: 1.71+-0.16% f=0.50: 1.63+-0.20% f=1.00: 1.84+-0.00%
|
| 17 |
+
seed321: f=0.25: 1.19+-0.22% f=0.50: 1.42+-0.16% f=1.00: 1.36+-0.00%
|
| 18 |
+
Flat/noisy, not decreasing -- so the coordinator's hypothesized mechanism ("K=1 improves with
|
| 19 |
+
f, best-of-8 does not, because a dense bank's 8 nearest are near-duplicate") does NOT apply:
|
| 20 |
+
its premise (K=1 improving with f) is not observed. Distance and regret are decoupled here.
|
| 21 |
+
|
| 22 |
+
3) Is the oracle itself introducing noise (stochastic local solver, different luck per call)?
|
| 23 |
+
NO. Re-ran eval_pendulum.strong_oracle TWICE, independent RNG streams, on 3 plants:
|
| 24 |
+
plant 0: 0.57009 vs 0.57009 (0.00% spread)
|
| 25 |
+
plant 5: 1.88592 vs 1.88590 (0.00% spread)
|
| 26 |
+
plant 10: 0.57613 vs 0.57613 (0.00% spread)
|
| 27 |
+
The oracle is reproducible to 5 significant figures regardless of restart-RNG seed -- ruled out
|
| 28 |
+
as a confound. Regret numbers are not noisy because of oracle non-determinism.
|
| 29 |
+
|
| 30 |
+
4) So what IS retrieval's K=8 regret so much lower than K=1's, if not distance or oracle luck?
|
| 31 |
+
SELECTION, not native target excellence. Diagnostic (b): the SINGLE nearest training plant's
|
| 32 |
+
OWN stored controller, evaluated on ITS OWN plant against a fresh oracle solve of that same
|
| 33 |
+
plant ("native quality"), has median regret +1.40% (p90 +2.67%, max +3.58%, n=40) -- NOT the
|
| 34 |
+
~0.2-0.4% the coordinator's hypothesis anticipated. Compared LIKE-FOR-LIKE (native K=1 quality
|
| 35 |
+
vs the SAME candidate transferred to the nearby held-out query, i.e. diagnostic-a's K=1 column):
|
| 36 |
+
native 1.40% vs transferred 1.71-1.84% (seed123) -- a modest, sensible ~0.3-0.4 point TRANSFER
|
| 37 |
+
PENALTY, not the dramatic gap the original (mismatched) comparison against K=8 suggested.
|
| 38 |
+
Best-of-8 retrieval's much lower regret (0.17-0.40%) is therefore explained by SELECTION: among
|
| 39 |
+
8 candidates whose INDIVIDUAL native quality is unremarkable (~1.4% typical), the one that
|
| 40 |
+
happens to transfer best to THIS query (verified directly on it) is kept -- an order-statistic
|
| 41 |
+
effect of trying 8 non-identical candidates and keeping the minimum-cost one, not evidence that
|
| 42 |
+
the stored targets are individually near-oracle. This is consistent with, but a more precise
|
| 43 |
+
version of, the "Iteration 3" reading (retrieval's edge is in the SELECTION machinery applied to
|
| 44 |
+
an ordinary bank, not in the bank being unusually good).
|
| 45 |
+
|
| 46 |
+
5) Remaining, NOT resolved by (1)-(4): the MODEL's own regret rise (0.24-0.26% at f=0.25 to
|
| 47 |
+
0.87-0.98% at f=0.5/1.0, reproducible across both held-out draws, tight sd across training
|
| 48 |
+
seeds) is a real, separate effect concerning model TRAINING, not retrieval geometry. A plausible
|
| 49 |
+
but UNTESTED factor: at a FIXED 800 epochs, f=0.25 (650 examples) receives ~4x fewer gradient
|
| 50 |
+
steps than f=1.0 (2600 examples) at the same batch size, an implicit early-stopping-like effect
|
| 51 |
+
that could make the f=0.25 model's outputs smoother / less fit to per-plant idiosyncrasies that
|
| 52 |
+
do not generalize to nearby held-out points. Flagged as an open question, not verified here.
|
| 53 |
+
|
| 54 |
+
6) The residual small ordering wobble in retrieval's own regret across f (e.g. seed123 K=8:
|
| 55 |
+
0.17% -> 0.30% -> 0.40%) is within noise of averaging only 3 INDEPENDENT (non-nested) random
|
| 56 |
+
bank draws per f on N=40 query plants -- given (1)-(3) rule out the distance and oracle
|
| 57 |
+
explanations, this residual is most plausibly small-sample composition noise across 3 seeds,
|
| 58 |
+
not a systematic density effect. It does not change the headline: the model never overtakes
|
| 59 |
+
retrieval's median regret at any swept f, on either held-out draw (pendulum_bank_density.txt).
|
results/baselines/retrieval_mimo.txt
ADDED
|
@@ -0,0 +1,36 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
=== MIMO retrieval/interpolation baselines (N=120, seed=9999) ===
|
| 2 |
+
verifier: simulate_closed_loop_mimo_batched rollout (n_ic=4, dt=0.03, duration=12.0, energy_thresh=0.3)
|
| 3 |
+
oracle: eval_mimo_regret.py's oracle protocol (fixed generic init 0.3*cap, 150-step batched Adam lr=0.08 through the differentiable MIMO sim, sector-projected) -- arm-independent, computed ONCE, shared by every arm (1900.0 ms/plant)
|
| 4 |
+
K_std=8, diffusion seeds=(0, 1, 2)
|
| 5 |
+
|
| 6 |
+
--- K=8 ---
|
| 7 |
+
arm coverage persamp reg_med% reg_p90% reg_max% ms/plant McNemar(n10,n01,p) Wilcoxon(n,p)
|
| 8 |
+
diffusion 1.00 0.95 2.32 8.89 31.76 305.42 (reference) (reference)
|
| 9 |
+
(sd across 3 seeds: coverage 0.004, per-samp 0.004)
|
| 10 |
+
retrieval 0.99 0.92 2.62 11.04 33.89 1.165 (0,1,1) (119,0.87)
|
| 11 |
+
interpolation 0.99 0.93 2.86 12.50 36.31 1.165 (0,1,1) (119,0.89)
|
| 12 |
+
|
| 13 |
+
--- K=1 ---
|
| 14 |
+
arm coverage persamp reg_med% reg_p90% reg_max% ms/plant McNemar(n10,n01,p) Wilcoxon(n,p)
|
| 15 |
+
diffusion 0.94 0.94 9.83 70.78 819.17 305.42 (reference) (reference)
|
| 16 |
+
(sd across 3 seeds: coverage 0.004, per-samp 0.004)
|
| 17 |
+
retrieval 0.86 0.86 13.66 96.49 496.91 0.338 (0,10,0.002) (103,0.17)
|
| 18 |
+
interpolation 0.86 0.86 13.66 96.49 496.91 0.338 (0,10,0.002) (103,0.17)
|
| 19 |
+
|
| 20 |
+
|
| 21 |
+
--- K=8 bootstrap 95% CIs (addendum, computed post-hoc from saved npz; not re-simulated) ---
|
| 22 |
+
diffusion (seed[0]): coverage 95% CI [1.000,1.000] regret_med 95% CI [+1.63,+2.99]%
|
| 23 |
+
retrieval: coverage 95% CI [0.975,1.000] regret_med 95% CI [+1.45,+3.45]%
|
| 24 |
+
interpolation: coverage 95% CI [0.975,1.000] regret_med 95% CI [+1.78,+3.58]%
|
| 25 |
+
--- K=1 bootstrap 95% CIs (addendum, computed post-hoc from saved npz; not re-simulated) ---
|
| 26 |
+
diffusion (seed[0]): coverage 95% CI [0.892,0.983] regret_med 95% CI [+6.14,+12.50]%
|
| 27 |
+
retrieval: coverage 95% CI [0.792,0.917] regret_med 95% CI [+10.22,+19.86]%
|
| 28 |
+
interpolation: coverage 95% CI [0.792,0.917] regret_med 95% CI [+10.22,+19.86]%
|
| 29 |
+
|
| 30 |
+
--- addendum: confirming the success set used by the Wilcoxon cost test above (coordinator follow-up #3) ---
|
| 31 |
+
K=8: diffusion(seed0) covers 120/120 plants
|
| 32 |
+
retrieval: covers 119/120; common-with-diffusion=119 (= a strict subset of diffusion coverage)
|
| 33 |
+
interpolation: covers 119/120; common-with-diffusion=119 (= a strict subset of diffusion coverage)
|
| 34 |
+
K=1: diffusion(seed0) covers 113/120 plants
|
| 35 |
+
retrieval: covers 103/120; common-with-diffusion=103 (= a strict subset of diffusion coverage)
|
| 36 |
+
interpolation: covers 103/120; common-with-diffusion=103 (= a strict subset of diffusion coverage)
|
results/baselines/siso_baselines.txt
ADDED
|
@@ -0,0 +1,12 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
SISO why-generative baseline — pooled over 5 seeds × 60 plants (N=300), K=8
|
| 2 |
+
|
| 3 |
+
regressor : plant-stabilisable 0.72 [0.67,0.77] per-sample-stable 0.72
|
| 4 |
+
diff_1shot : plant-stabilisable 0.61 [0.56,0.67] per-sample-stable 0.61
|
| 5 |
+
diff_Kshot : plant-stabilisable 0.75 [0.70,0.80] per-sample-stable 0.61
|
| 6 |
+
|
| 7 |
+
diffusion K-shot vs regressor (paired McNemar-ish sign test): p = 3.09e-02
|
| 8 |
+
controller diversity (per-point std across K diffusion samples): 3.4 (regressor = 0, a single point)
|
| 9 |
+
|
| 10 |
+
TAKEAWAY: the regressor (conditional mean) stabilises fewer plants than diffusion sampled
|
| 11 |
+
K times; the generative DISTRIBUTION is what lets test-time selection/repair find a
|
| 12 |
+
stabilising controller a point estimate misses.
|
results/furuta_v4/ablation_unetAseed1_ema999_unetAseed2_ema999_unetA_ema999_unetS_ema99.txt
ADDED
|
@@ -0,0 +1,8 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
model K=1 (draw4242/draw9001, pooled, paired diff to nearest+P) // K=4
|
| 2 |
+
unetAseed1_ema999 K=1: 92.8/93.9 pooled 93.3 vs retr 54.2 | diff +39.2 [+30.0,+47.8] p=0.0000 * K=4: 93.3/94.4 pooled 93.9 vs retr 82.5 | diff +11.4 [+5.6,+17.5] p=0.0006 *
|
| 3 |
+
unetAseed2_ema999 K=1: 90.6/93.3 pooled 91.9 vs retr 54.2 | diff +37.8 [+29.2,+46.7] p=0.0000 * K=4: 91.7/93.3 pooled 92.5 vs retr 82.5 | diff +10.0 [+4.2,+15.8] p=0.0022 *
|
| 4 |
+
unetA_ema999 K=1: 90.6/93.3 pooled 91.9 vs retr 54.2 | diff +37.8 [+28.6,+46.9] p=0.0000 * K=4: 92.8/93.9 pooled 93.3 vs retr 82.5 | diff +10.8 [+5.0,+17.2] p=0.0006 *
|
| 5 |
+
unetS_ema999 K=1: 88.3/92.2 pooled 90.3 vs retr 54.2 | diff +36.1 [+27.2,+44.7] p=0.0000 * K=4: 95.0/95.0 pooled 95.0 vs retr 82.5 | diff +12.5 [+6.7,+18.3] p=0.0001 *
|
| 6 |
+
unetSXD_ema999 K=1: 90.0/91.1 pooled 90.6 vs retr 67.5 | diff +23.1 [+15.0,+31.4] p=0.0000 * K=4: 95.0/95.6 pooled 95.3 vs retr 90.0 | diff +5.3 [+0.8,+10.0] p=0.0380 *
|
| 7 |
+
mlpB256_ema999 K=1: 74.4/75.0 pooled 74.7 vs retr 54.2 | diff +20.6 [+10.3,+30.3] p=0.0002 * K=4: 88.3/91.7 pooled 90.0 vs retr 82.5 | diff +7.5 [+1.9,+13.3] p=0.0113 *
|
| 8 |
+
mlpXD_ema999 K=1: 75.6/77.8 pooled 76.7 vs retr 67.5 | diff +9.2 [+0.3,+18.1] p=0.0500 K=4: 89.4/93.3 pooled 91.4 vs retr 90.0 | diff +1.4 [-2.2,+5.6] p=0.5806
|
results/furuta_v4/budget_curves_ddim10.txt
ADDED
|
@@ -0,0 +1,27 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# success (%) vs per-plant single-thread wall-clock budget; verify times as measured, costs from budget_timing_cpu.txt; refine n=200
|
| 2 |
+
# cost model: sample1 0.03s sample4 0.07s regfwd1 3.2ms refine(40 steps) 0.70s (flat in batch<=4)
|
| 3 |
+
method 0.1s 0.25s 0.5s 1s 2s 3s 5s 10s 60s best config at each budget
|
| 4 |
+
retrieval alone 0.0 0.0 48.3 69.2 77.5 83.3 90.0 93.3 93.3 - | - | S=16,m=1 | S=16,m=3 | S=16,m=6 | S=16,m=6 | S=16,m=7 | S=16,m=11 | S=16,m=11
|
| 5 |
+
retrieval + refine 0.0 0.0 48.3 83.3 87.5 93.3 97.5 97.5 97.5 - | - | S=4,m=1 | S=4,m=1,I=20 | S=4,m=2,I=40 | S=4,m=2,I=40 | S=4,m=3,I=40 | S=4,m=3,I=40 | S=4,m=3,I=40
|
| 6 |
+
generated (U-Net, DDIM-10) alone 0.0 0.0 80.1 81.4 81.8 93.1 93.8 94.4 94.4 - | - | S=1,m=1 | S=4,m=3 | S=4,m=4 | S=8,m=8 | S=16,m=11 | S=16,m=15 | S=16,m=15
|
| 7 |
+
generated (U-Net, DDIM-10) + refine 0.0 0.0 80.1 84.4 86.0 97.3 98.1 98.3 98.3 - | - | S=1,m=1 | S=1,m=1,I=20 | S=4,m=2,I=40 | S=4,m=3,I=40 | S=4,m=4,I=40 | S=4,m=4,I=40 | S=4,m=4,I=40
|
| 8 |
+
regression twin (deterministic) alone 0.0 0.8 76.9 76.9 76.9 85.6 85.6 85.6 85.6 - | S=1,m=1 | S=1,m=1 | S=1,m=1 | S=1,m=1 | S=1,m=1 | S=1,m=1 | S=1,m=1 | S=1,m=1
|
| 9 |
+
regression twin (deterministic) + refine 0.0 0.8 76.9 83.3 84.7 93.3 95.6 95.6 95.6 - | S=1,m=1 | S=1,m=1 | S=1,m=1,I=20 | S=1,m=1,I=40 | S=1,m=1,I=40 | S=1,m=1,I=40 | S=1,m=1,I=40 | S=1,m=1,I=40
|
| 10 |
+
|
| 11 |
+
# paired difference A - B in pp at each budget [95% CI] p (best config per method per budget; 120 plants)
|
| 12 |
+
generated (U-Net, DDIM-10) alone - retrieval alone
|
| 13 |
+
+0.0 [ +0.0, +0.0] p=1.000 +0.0 [ +0.0, +0.0] p=1.000 +31.8 [+22.9,+40.9] p=0.000 +12.2 [ +4.7,+19.6] p=0.002 +4.3 [ -3.1,+11.8] p=0.276 +9.7 [ +4.4,+15.6] p=0.001 +3.8 [ +0.3, +8.1] p=0.086 +1.0 [ -2.1, +4.6] p=0.627 +1.1 [ -2.2, +4.8] p=0.622
|
| 14 |
+
generated (U-Net, DDIM-10) + refine - retrieval + refine
|
| 15 |
+
+0.0 [ +0.0, +0.0] p=1.000 +0.0 [ +0.0, +0.0] p=1.000 +31.8 [+22.6,+40.9] p=0.000 +1.1 [ -5.7, +8.1] p=0.776 -1.5 [ -8.3, +5.6] p=0.708 +4.0 [ +0.5, +8.1] p=0.039 +0.6 [ -0.6, +2.6] p=0.877 +0.8 [ -0.2, +2.7] p=0.750 +0.8 [ -0.2, +2.6] p=0.747
|
| 16 |
+
generated (U-Net, DDIM-10) alone - retrieval + refine
|
| 17 |
+
+0.0 [ +0.0, +0.0] p=1.000 +0.0 [ +0.0, +0.0] p=1.000 +31.8 [+22.9,+40.5] p=0.000 -1.9 [ -8.7, +5.0] p=0.605 -5.7 [-13.4, +1.4] p=0.142 -0.3 [ -4.8, +4.1] p=0.934 -3.7 [ -7.9, +0.0] p=0.065 -3.1 [ -7.1, +0.4] p=0.099 -3.1 [ -6.9, +0.4] p=0.141
|
| 18 |
+
generated (U-Net, DDIM-10) + refine - generated (U-Net, DDIM-10) alone
|
| 19 |
+
+0.0 [ +0.0, +0.0] p=1.000 +0.0 [ +0.0, +0.0] p=1.000 +0.0 [ +0.0, +0.0] p=1.000 +3.1 [ +1.0, +5.6] p=0.004 +4.3 [ +1.4, +7.7] p=0.003 +4.3 [ +1.5, +7.9] p=0.002 +4.4 [ +1.4, +7.7] p=0.005 +4.0 [ +1.0, +7.6] p=0.008 +3.9 [ +1.0, +7.4] p=0.015
|
| 20 |
+
regression twin (deterministic) alone - retrieval alone
|
| 21 |
+
+0.0 [ +0.0, +0.0] p=1.000 +0.8 [ +0.0, +2.5] p=1.000 +28.6 [+19.4,+38.1] p=0.000 +7.8 [ -0.0,+15.3] p=0.058 -0.6 [ -8.3, +7.2] p=0.943 +2.2 [ -3.6, +8.1] p=0.523 -4.4 [ -9.2, -0.0] p=0.085 -7.8 [-12.8, -3.1] p=0.002 -7.8 [-13.1, -3.1] p=0.002
|
| 22 |
+
regression twin (deterministic) alone - generated (U-Net, DDIM-10) alone
|
| 23 |
+
+0.0 [ +0.0, +0.0] p=1.000 +0.8 [ +0.0, +2.5] p=1.000 -3.1 [ -7.9, +1.1] p=0.189 -4.4 [ -9.5, +0.4] p=0.080 -4.8 [ -9.9, +0.0] p=0.066 -7.5 [-11.9, -3.5] p=0.000 -8.2 [-13.1, -4.1] p=0.000 -8.8 [-13.5, -4.5] p=0.000 -8.9 [-13.6, -4.6] p=0.000
|
| 24 |
+
regression twin (deterministic) + refine - retrieval + refine
|
| 25 |
+
+0.0 [ +0.0, +0.0] p=1.000 +0.8 [ +0.0, +2.5] p=1.000 +28.6 [+19.2,+38.1] p=0.000 -0.0 [ -7.2, +7.2] p=1.000 -2.8 [ -9.4, +4.2] p=0.479 -0.0 [ -3.9, +3.9] p=1.000 -1.9 [ -5.3, +1.1] p=0.381 -1.9 [ -5.6, +1.1] p=0.366 -1.9 [ -5.3, +1.1] p=0.371
|
| 26 |
+
regression twin (deterministic) + refine - generated (U-Net, DDIM-10) + refine
|
| 27 |
+
+0.0 [ +0.0, +0.0] p=1.000 +0.8 [ +0.0, +2.5] p=1.000 -3.1 [ -7.7, +1.1] p=0.186 -1.1 [ -5.5, +3.2] p=0.650 -1.3 [ -5.8, +3.1] p=0.606 -4.0 [ -7.5, -1.0] p=0.016 -2.6 [ -5.7, -0.2] p=0.111 -2.8 [ -5.8, -0.5] p=0.096 -2.8 [ -5.9, -0.5] p=0.093
|
results/furuta_v4/budget_unetA_ema999_ddim10_n200.txt
ADDED
|
@@ -0,0 +1,24 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
seed4242 gen_s0 n_sub=200
|
| 2 |
+
[gen_s0] SDP+verify 66s plain ok 0.900; 24 failed-certified in 8 plants
|
| 3 |
+
[gen_s0] refine done 86s
|
| 4 |
+
[gen_s0] refined ok at (5, 10, 20, 40): [0.917, 0.917, 0.95, 0.983] (K=1) (91s)
|
| 5 |
+
seed4242 gen_s1 n_sub=200
|
| 6 |
+
[gen_s1] SDP+verify 65s plain ok 0.903; 25 failed-certified in 9 plants
|
| 7 |
+
[gen_s1] refine done 86s
|
| 8 |
+
[gen_s1] refined ok at (5, 10, 20, 40): [0.95, 0.95, 0.983, 0.983] (K=1) (90s)
|
| 9 |
+
seed4242 gen_s2 n_sub=200
|
| 10 |
+
[gen_s2] SDP+verify 65s plain ok 0.901; 27 failed-certified in 8 plants
|
| 11 |
+
[gen_s2] refine done 85s
|
| 12 |
+
[gen_s2] refined ok at (5, 10, 20, 40): [0.933, 0.95, 0.967, 0.983] (K=1) (89s)
|
| 13 |
+
seed9001 gen_s0 n_sub=200
|
| 14 |
+
[gen_s0] SDP+verify 57s plain ok 0.934; 16 failed-certified in 4 plants
|
| 15 |
+
[gen_s0] refine done 72s
|
| 16 |
+
[gen_s0] refined ok at (5, 10, 20, 40): [0.967, 0.967, 0.95, 0.983] (K=1) (76s)
|
| 17 |
+
seed9001 gen_s1 n_sub=200
|
| 18 |
+
[gen_s1] SDP+verify 58s plain ok 0.934; 15 failed-certified in 4 plants
|
| 19 |
+
[gen_s1] refine done 72s
|
| 20 |
+
[gen_s1] refined ok at (5, 10, 20, 40): [0.967, 0.967, 0.95, 0.983] (K=1) (76s)
|
| 21 |
+
seed9001 gen_s2 n_sub=200
|
| 22 |
+
[gen_s2] SDP+verify 59s plain ok 0.934; 16 failed-certified in 4 plants
|
| 23 |
+
[gen_s2] refine done 73s
|
| 24 |
+
[gen_s2] refined ok at (5, 10, 20, 40): [0.967, 0.967, 0.95, 0.983] (K=1) (77s)
|
results/furuta_v4/budget_unetAseed1_ema999_n200.txt
ADDED
|
@@ -0,0 +1,24 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
seed4242 gen_s0 n_sub=200
|
| 2 |
+
[gen_s0] SDP+verify 100s plain ok 0.924; 17 failed-certified in 5 plants
|
| 3 |
+
[gen_s0] refine done 122s
|
| 4 |
+
[gen_s0] refined ok at (5, 10, 20, 40): [0.933, 0.933, 0.967, 0.95] (K=1) (129s)
|
| 5 |
+
seed4242 gen_s1 n_sub=200
|
| 6 |
+
[gen_s1] SDP+verify 106s plain ok 0.915; 21 failed-certified in 7 plants
|
| 7 |
+
[gen_s1] refine done 133s
|
| 8 |
+
[gen_s1] refined ok at (5, 10, 20, 40): [0.933, 0.95, 0.967, 0.95] (K=1) (139s)
|
| 9 |
+
seed4242 gen_s2 n_sub=200
|
| 10 |
+
[gen_s2] SDP+verify 107s plain ok 0.911; 21 failed-certified in 8 plants
|
| 11 |
+
[gen_s2] refine done 137s
|
| 12 |
+
[gen_s2] refined ok at (5, 10, 20, 40): [0.933, 0.95, 0.967, 0.967] (K=1) (143s)
|
| 13 |
+
seed9001 gen_s0 n_sub=200
|
| 14 |
+
[gen_s0] SDP+verify 100s plain ok 0.938; 15 failed-certified in 4 plants
|
| 15 |
+
[gen_s0] refine done 120s
|
| 16 |
+
[gen_s0] refined ok at (5, 10, 20, 40): [0.967, 0.967, 0.95, 0.983] (K=1) (126s)
|
| 17 |
+
seed9001 gen_s1 n_sub=200
|
| 18 |
+
[gen_s1] SDP+verify 101s plain ok 0.934; 15 failed-certified in 4 plants
|
| 19 |
+
[gen_s1] refine done 121s
|
| 20 |
+
[gen_s1] refined ok at (5, 10, 20, 40): [0.967, 0.967, 0.95, 0.983] (K=1) (126s)
|
| 21 |
+
seed9001 gen_s2 n_sub=200
|
| 22 |
+
[gen_s2] SDP+verify 95s plain ok 0.936; 16 failed-certified in 4 plants
|
| 23 |
+
[gen_s2] refine done 115s
|
| 24 |
+
[gen_s2] refined ok at (5, 10, 20, 40): [0.95, 0.967, 0.95, 1.0] (K=1) (121s)
|
results/furuta_v4/ddim_unetA_ema999_20.txt
ADDED
|
@@ -0,0 +1,2 @@
|
|
|
|
|
|
|
|
|
|
| 1 |
+
unetA_ema999 DDIM 20
|
| 2 |
+
K=1: 91.9% K=4: 93.3% K=8: 93.3% K=16: 94.7%
|
results/furuta_v4/ddim_unetAseed1_ema999_10.txt
ADDED
|
@@ -0,0 +1,2 @@
|
|
|
|
|
|
|
|
|
|
| 1 |
+
unetAseed1_ema999 DDIM 10
|
| 2 |
+
K=1: 91.7% K=4: 93.6% K=8: 93.9% K=16: 94.2%
|
results/furuta_v4/ddim_unetAseed1_ema999_20.txt
ADDED
|
@@ -0,0 +1,2 @@
|
|
|
|
|
|
|
|
|
|
| 1 |
+
unetAseed1_ema999 DDIM 20
|
| 2 |
+
K=1: 91.9% K=4: 93.6% K=8: 93.9% K=16: 93.9%
|
results/furuta_v4/ddim_unetAseed1_ema999_5.txt
ADDED
|
@@ -0,0 +1,2 @@
|
|
|
|
|
|
|
|
|
|
| 1 |
+
unetAseed1_ema999 DDIM 5
|
| 2 |
+
K=1: 90.3% K=4: 92.5% K=8: 93.3% K=16: 93.9%
|
results/furuta_v4/gate_mlp384k_ema999.txt
ADDED
|
@@ -0,0 +1,2 @@
|
|
|
|
|
|
|
|
|
|
| 1 |
+
mlp384k_ema999 ema999 models/v4/mlp_384k_s384000.pth
|
| 2 |
+
K=1: generated+P 73.3% nearest+P 54.2% K=4: generated+P 89.4% nearest+P 82.5% K=8: generated+P 92.8% nearest+P 90.0% K=16: generated+P 95.3% nearest+P 93.3%
|
results/furuta_v4/gate_mlpB256_ema999.txt
ADDED
|
@@ -0,0 +1,2 @@
|
|
|
|
|
|
|
|
|
|
| 1 |
+
mlpB256_ema999 ema999 models/v4/mlp_b256_s80000.pth
|
| 2 |
+
K=1: generated+P 74.7% nearest+P 54.2% K=4: generated+P 90.0% nearest+P 82.5% K=8: generated+P 92.2% nearest+P 90.0% K=16: generated+P 94.4% nearest+P 93.3%
|
results/furuta_v4/gate_unetA_ema9995.txt
ADDED
|
@@ -0,0 +1,2 @@
|
|
|
|
|
|
|
|
|
|
| 1 |
+
unetA_ema9995 ema9995 models/v4/unet_a_s96000.pth
|
| 2 |
+
K=1: generated+P 91.9% nearest+P 54.2% K=4: generated+P 93.3% nearest+P 82.5% K=8: generated+P 93.6% nearest+P 90.0% K=16: generated+P 93.9% nearest+P 93.3%
|
results/furuta_v4/gate_unetAseed2_ema999.txt
ADDED
|
@@ -0,0 +1,2 @@
|
|
|
|
|
|
|
|
|
|
| 1 |
+
unetAseed2_ema999 ema999 models/v4/unet_a_seed2_s96000.pth
|
| 2 |
+
K=1: generated+P 91.9% nearest+P 54.2% K=4: generated+P 92.5% nearest+P 82.5% K=8: generated+P 93.6% nearest+P 90.0% K=16: generated+P 94.4% nearest+P 93.3%
|
results/furuta_v4/gate_unetS_ema999.txt
ADDED
|
@@ -0,0 +1,2 @@
|
|
|
|
|
|
|
|
|
|
| 1 |
+
unetS_ema999 ema999 models/v4/unet_s_s40000.pth
|
| 2 |
+
K=1: generated+P 90.3% nearest+P 54.2% K=4: generated+P 95.0% nearest+P 82.5% K=8: generated+P 95.3% nearest+P 90.0% K=16: generated+P 96.1% nearest+P 93.3%
|
results/furuta_v4/memo_mlp384k_ema999.txt
ADDED
|
@@ -0,0 +1,7 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
bank rows 2801; bank-to-bank NN (leave-one-out) RMS: median 0.0444 p10 0.0201 p90 0.1061
|
| 2 |
+
sample diversity: mean pairwise RMS between the 16 samples of one plant (seed 0): median over plants 0.0867 (bank LOO NN median 0.0444)
|
| 3 |
+
generated (candidate 1, 360 = 2 draws x 60 plants x seeds): dist to nearest training row: median 0.0550 p10 0.0336 p90 0.1153
|
| 4 |
+
ratio median(d_gen)/median(d_loo) = 1.24; frac of candidates closer than the 10th pct of bank LOO: 0.006; frac closer than 0.25*median LOO: 0.000
|
| 5 |
+
distance to the same-plant retrieved row: median 0.1270
|
| 6 |
+
succeeded K=1: median d_gen 0.0519 (n=264); failed: 0.0719 (n=96)
|
| 7 |
+
copy test (threshold 0.0111 = 0.25 x median bank LOO): successes with d_gen<thr: 0.0% (0/264); successes with d to same-plant retrieved row<thr: 0.0% (0/264)
|
results/furuta_v4/perseed.txt
ADDED
|
@@ -0,0 +1,13 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
name | draw | retrieval K1/K4 | sampler seed 0 K1/K4 | seed 1 | seed 2
|
| 2 |
+
unetA_ema999 | 4242 | 51.7/78.3 | 91.7/93.3 | 91.7/93.3 | 88.3/91.7
|
| 3 |
+
unetA_ema999 | 9001 | 56.7/86.7 | 93.3/95.0 | 93.3/93.3 | 93.3/93.3
|
| 4 |
+
unetAseed1_ema999 | 4242 | 51.7/78.3 | 93.3/93.3 | 93.3/93.3 | 91.7/93.3
|
| 5 |
+
unetAseed1_ema999 | 9001 | 56.7/86.7 | 95.0/95.0 | 93.3/95.0 | 93.3/93.3
|
| 6 |
+
unetAseed2_ema999 | 4242 | 51.7/78.3 | 91.7/91.7 | 88.3/91.7 | 91.7/91.7
|
| 7 |
+
unetAseed2_ema999 | 9001 | 56.7/86.7 | 93.3/93.3 | 93.3/93.3 | 93.3/93.3
|
| 8 |
+
unetS_ema999 | 4242 | 51.7/78.3 | 85.0/95.0 | 90.0/95.0 | 90.0/95.0
|
| 9 |
+
unetS_ema999 | 9001 | 56.7/86.7 | 90.0/95.0 | 93.3/95.0 | 93.3/95.0
|
| 10 |
+
unetSXD_ema999 | 4242 | 68.3/91.7 | 88.3/93.3 | 88.3/96.7 | 93.3/95.0
|
| 11 |
+
unetSXD_ema999 | 9001 | 66.7/88.3 | 88.3/96.7 | 91.7/95.0 | 93.3/95.0
|
| 12 |
+
mlp384k_ema999 | 4242 | 51.7/78.3 | 70.0/88.3 | 65.0/88.3 | 63.3/86.7
|
| 13 |
+
mlp384k_ema999 | 9001 | 56.7/86.7 | 80.0/91.7 | 80.0/90.0 | 81.7/91.7
|
results/furuta_v4/rank_mlp384k_ema999.txt
ADDED
|
@@ -0,0 +1,32 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
seed4242 gen_s0: ranking 16 candidates
|
| 2 |
+
[gen_s0] SDP+verify 54s; plain ok 0.900; 24 failed with a certified P -> refine
|
| 3 |
+
[gen_s0] refine done 54s
|
| 4 |
+
[gen_s0] refined ok 0.900 (54s)
|
| 5 |
+
seed4242 gen_s1: ranking 16 candidates
|
| 6 |
+
[gen_s1] SDP+verify 53s; plain ok 0.900; 24 failed with a certified P -> refine
|
| 7 |
+
[gen_s1] refine done 53s
|
| 8 |
+
[gen_s1] refined ok 0.900 (53s)
|
| 9 |
+
seed4242 gen_s2: ranking 16 candidates
|
| 10 |
+
[gen_s2] SDP+verify 48s; plain ok 0.875; 30 failed with a certified P -> refine
|
| 11 |
+
[gen_s2] refine done 48s
|
| 12 |
+
[gen_s2] refined ok 0.875 (48s)
|
| 13 |
+
seed4242 retr: ranking 16 candidates
|
| 14 |
+
[retr] SDP+verify 62s; plain ok 0.767; 56 failed with a certified P -> refine
|
| 15 |
+
[retr] refine done 62s
|
| 16 |
+
[retr] refined ok 0.767 (62s)
|
| 17 |
+
seed9001 gen_s0: ranking 16 candidates
|
| 18 |
+
[gen_s0] SDP+verify 55s; plain ok 0.942; 14 failed with a certified P -> refine
|
| 19 |
+
[gen_s0] refine done 55s
|
| 20 |
+
[gen_s0] refined ok 0.942 (55s)
|
| 21 |
+
seed9001 gen_s1: ranking 16 candidates
|
| 22 |
+
[gen_s1] SDP+verify 61s; plain ok 0.929; 17 failed with a certified P -> refine
|
| 23 |
+
[gen_s1] refine done 61s
|
| 24 |
+
[gen_s1] refined ok 0.929 (61s)
|
| 25 |
+
seed9001 gen_s2: ranking 16 candidates
|
| 26 |
+
[gen_s2] SDP+verify 64s; plain ok 0.925; 18 failed with a certified P -> refine
|
| 27 |
+
[gen_s2] refine done 64s
|
| 28 |
+
[gen_s2] refined ok 0.925 (64s)
|
| 29 |
+
seed9001 retr: ranking 16 candidates
|
| 30 |
+
[retr] SDP+verify 62s; plain ok 0.783; 52 failed with a certified P -> refine
|
| 31 |
+
[retr] refine done 62s
|
| 32 |
+
[retr] refined ok 0.783 (62s)
|
results/furuta_v4/refine_mlpA_ema999_summary.txt
ADDED
|
@@ -0,0 +1,12 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
results/furuta_v4/refine_mlpA_ema999.npz
|
| 2 |
+
retrieval arm taken from results/furuta_v4/refine_mlpA_ema999.npz
|
| 3 |
+
K=1: generated plain 60.8 -> refined 95.0 | retrieval plain 54.2 -> refined 95.0
|
| 4 |
+
gen-refined minus retr-refined: -0.0 [-3.9,+4.2] p=1.0000
|
| 5 |
+
gen-plain minus retr-plain: +6.7 [-2.8,+16.7] p=0.2131
|
| 6 |
+
retr refined minus plain: +40.8 [+32.5,+50.0] p=0.0000
|
| 7 |
+
gen refined minus plain: +34.2 [+28.6,+40.0] p=0.0000
|
| 8 |
+
K=4: generated plain 87.8 -> refined 97.8 | retrieval plain 82.5 -> refined 97.5
|
| 9 |
+
gen-refined minus retr-refined: +0.3 [-1.7,+2.5] p=1.0000
|
| 10 |
+
gen-plain minus retr-plain: +5.3 [-0.6,+11.7] p=0.1138
|
| 11 |
+
retr refined minus plain: +15.0 [+9.2,+21.7] p=0.0000
|
| 12 |
+
gen refined minus plain: +10.0 [+6.1,+14.4] p=0.0000
|
results/furuta_v4/reg_attribution_partial.txt
ADDED
|
@@ -0,0 +1,36 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
== pooled success % (120 plants); per-training-seed values after the slash ==
|
| 2 |
+
method K=1 K=2 K=3 K=4 K=8
|
| 3 |
+
nearest+P retrieval 54.2 82.5 90.0
|
| 4 |
+
diffusion unet_a (9 runs; per train seed) 92.4 / 91.9/93.3/91.9 93.2 / 93.3/93.9/92.593.8 / 93.6/94.2/93.6
|
| 5 |
+
regression twin, deterministic (3 seeds) 83.6 / 85.0/82.5/83.3
|
| 6 |
+
regression ens3 (members as K candidates) 83.6 86.9 88.3
|
| 7 |
+
regression MC-dropout p=.1 (9 runs) 78.6 / 81.1/78.6/76.1 86.9 / 87.2/87.8/85.888.1 / 88.3/88.3/87.8
|
| 8 |
+
same net, dropout OFF (K=1) 83.9 / 84.2/85.8/81.7
|
| 9 |
+
|
| 10 |
+
== paired tests, per-plant seed-averaged difference A-B (pp), bootstrap 95% CI, sign-flip p ==
|
| 11 |
+
K=1: diffusion - retrieval +38.2 [+29.2,+46.9] p=0.0000
|
| 12 |
+
K=4: diffusion - retrieval +10.7 [ +5.0,+16.7] p=0.0006
|
| 13 |
+
K=8: diffusion - retrieval +3.8 [ +0.0, +8.0] p=0.0858
|
| 14 |
+
K=1: regression det - retrieval +29.4 [+20.8,+38.3] p=0.0000
|
| 15 |
+
K=1: diffusion - regression det +8.8 [ +4.9,+13.1] p=0.0001
|
| 16 |
+
K=1: diffusion - MC-dropout net +13.8 [ +9.5,+18.3] p=0.0000
|
| 17 |
+
K=1: regression det - MC-dropout net (K=1) +5.0 [ +2.6, +7.5] p=0.0000
|
| 18 |
+
K=4: diffusion - MC-dropout +6.3 [ +2.9,+10.5] p=0.0001
|
| 19 |
+
K=4: MC-dropout - retrieval +4.4 [ -0.6, +9.8] p=0.1119
|
| 20 |
+
K=8: diffusion - MC-dropout +5.6 [ +2.3, +9.8] p=0.0008
|
| 21 |
+
K=8: MC-dropout - retrieval -1.9 [ -6.0, +2.1] p=0.4133
|
| 22 |
+
ens3 K=3 - diffusion K=4 (diffusion gets MORE candidates) -4.9 [ -9.1, -1.6] p=0.0155
|
| 23 |
+
ens3 K=3 - regression det K=1 +4.7 [ +2.2, +7.5] p=0.0008
|
| 24 |
+
MC-dropout K=4 - regression det K=1 +3.3 [ +1.5, +5.6] p=0.0011
|
| 25 |
+
diffusion K=4 - diffusion K=1 +0.8 [ +0.2, +1.8] p=0.0626
|
| 26 |
+
|
| 27 |
+
== exact McNemar, single runs on 120 plants (b = diffusion only, c = regression only) ==
|
| 28 |
+
train seed 0: diffusion(samp0) 92.5 vs det 85.0 b=9 c=0 p=0.004
|
| 29 |
+
train seed 1: diffusion(samp0) 94.2 vs det 82.5 b=14 c=0 p=0.000
|
| 30 |
+
train seed 2: diffusion(samp0) 92.5 vs det 83.3 b=11 c=0 p=0.001
|
| 31 |
+
majority over runs: diffusion 92.5 vs det 84.2 b=10 c=0 p=0.002
|
| 32 |
+
K=1 majority diffusion vs retrieval b=47 c=1 p=3.5e-13
|
| 33 |
+
K=4 majority diffusion vs retrieval b=14 c=1 p=0.00098
|
| 34 |
+
K=1 majority regression det vs retrieval b=40 c=4 p=1.7e-08
|
| 35 |
+
|
| 36 |
+
diversity (pairwise RMS of 8 MC-dropout samples, normalised space, mean over plants): 0.037 (diffusion unet_a 0.029, bank NN 0.044)
|
results/furuta_v4/run_gates.sh
ADDED
|
@@ -0,0 +1,6 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
#!/bin/bash
|
| 2 |
+
# usage: run_gates.sh "name ckpt weights" ... ; sequential gate evaluations (CPU)
|
| 3 |
+
cd /home/cercola/matte_server/DiffusionControl; export PYTHONPATH=. OMP_NUM_THREADS=1 CUDA_VISIBLE_DEVICES=""
|
| 4 |
+
for spec in "$@"; do set -- $spec
|
| 5 |
+
[ -f results/furuta_v4/gate_$1.npz ] || myenv/bin/python -u -m nonlinear.experiments.furuta_v4_eval --ckpt $2 --weights $3 --name $1 --plants gate --workers 6 ${4:+--bank $4}
|
| 6 |
+
done
|
results/furuta_v4/scan_mlp_a.txt
ADDED
|
@@ -0,0 +1,19 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
mlp_a_s32000_ema999 K1=42.5 K4=81.2 K8=85.0 nearK1=57.5 nearK4=85.0 score=61.9
|
| 2 |
+
mlp_a_s32000_ema9995 K1=42.5 K4=80.0 K8=85.0 nearK1=57.5 nearK4=85.0 score=61.2
|
| 3 |
+
mlp_a_s40000_ema999 K1=46.2 K4=85.0 K8=88.8 nearK1=57.5 nearK4=85.0 score=65.6
|
| 4 |
+
mlp_a_s40000_ema9995 K1=43.8 K4=83.7 K8=88.8 nearK1=57.5 nearK4=85.0 score=63.7
|
| 5 |
+
mlp_a_s48000_ema999 K1=51.2 K4=85.0 K8=88.8 nearK1=57.5 nearK4=85.0 score=68.1
|
| 6 |
+
mlp_a_s48000_ema9995 K1=51.2 K4=86.2 K8=88.8 nearK1=57.5 nearK4=85.0 score=68.8
|
| 7 |
+
mlp_a_s56000_ema999 K1=60.0 K4=90.0 K8=91.3 nearK1=57.5 nearK4=85.0 score=75.0
|
| 8 |
+
mlp_a_s56000_ema9995 K1=57.5 K4=87.5 K8=91.3 nearK1=57.5 nearK4=85.0 score=72.5
|
| 9 |
+
mlp_a_s64000_ema999 K1=55.0 K4=88.8 K8=90.0 nearK1=57.5 nearK4=85.0 score=71.9
|
| 10 |
+
mlp_a_s64000_ema9995 K1=57.5 K4=88.8 K8=90.0 nearK1=57.5 nearK4=85.0 score=73.1
|
| 11 |
+
mlp_a_s72000_ema999 K1=57.5 K4=87.5 K8=90.0 nearK1=57.5 nearK4=85.0 score=72.5
|
| 12 |
+
mlp_a_s72000_ema9995 K1=58.8 K4=86.2 K8=88.8 nearK1=57.5 nearK4=85.0 score=72.5
|
| 13 |
+
mlp_a_s80000_ema999 K1=61.3 K4=88.8 K8=90.0 nearK1=57.5 nearK4=85.0 score=75.0
|
| 14 |
+
mlp_a_s80000_ema9995 K1=61.3 K4=90.0 K8=91.3 nearK1=57.5 nearK4=85.0 score=75.6
|
| 15 |
+
mlp_a_s88000_ema999 K1=62.5 K4=86.2 K8=88.8 nearK1=57.5 nearK4=85.0 score=74.4
|
| 16 |
+
mlp_a_s88000_ema9995 K1=61.3 K4=87.5 K8=88.8 nearK1=57.5 nearK4=85.0 score=74.4
|
| 17 |
+
mlp_a_s96000_ema999 K1=63.7 K4=90.0 K8=90.0 nearK1=57.5 nearK4=85.0 score=76.9
|
| 18 |
+
mlp_a_s96000_ema9995 K1=62.5 K4=90.0 K8=90.0 nearK1=57.5 nearK4=85.0 score=76.2
|
| 19 |
+
BEST mlp_a_s96000_ema999 K1=63.7 K4=90.0 K8=90.0 nearK1=57.5 nearK4=85.0 score=76.9
|
results/furuta_v4/scan_unet_a.txt
ADDED
|
@@ -0,0 +1,9 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
unet_a_s40000_ema999 K1=87.5 K4=100.0 K8=100.0 nearK1=57.5 nearK4=85.0 score=93.8
|
| 2 |
+
unet_a_s48000_ema999 K1=86.2 K4=100.0 K8=100.0 nearK1=57.5 nearK4=85.0 score=93.1
|
| 3 |
+
unet_a_s56000_ema999 K1=95.0 K4=100.0 K8=100.0 nearK1=57.5 nearK4=85.0 score=97.5
|
| 4 |
+
unet_a_s64000_ema999 K1=97.5 K4=100.0 K8=100.0 nearK1=57.5 nearK4=85.0 score=98.8
|
| 5 |
+
unet_a_s72000_ema999 K1=98.8 K4=100.0 K8=100.0 nearK1=57.5 nearK4=85.0 score=99.4
|
| 6 |
+
unet_a_s80000_ema999 K1=95.0 K4=100.0 K8=100.0 nearK1=57.5 nearK4=85.0 score=97.5
|
| 7 |
+
unet_a_s88000_ema999 K1=96.2 K4=100.0 K8=100.0 nearK1=57.5 nearK4=85.0 score=98.1
|
| 8 |
+
unet_a_s96000_ema999 K1=97.5 K4=100.0 K8=100.0 nearK1=57.5 nearK4=85.0 score=98.8
|
| 9 |
+
BEST unet_a_s72000_ema999 K1=98.8 K4=100.0 K8=100.0 nearK1=57.5 nearK4=85.0 score=99.4
|
results/furuta_v4/traj_gen_warmstart_v4unetA_s0.txt
ADDED
|
@@ -0,0 +1,17 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
=== warm-start test (n=60, seed 4242, max_iter=700) ===
|
| 2 |
+
target certificate: largest_certified_region per plant (unmargined); verifier: furuta_traj_bank.score_solution (identical to every other arm in this doc)
|
| 3 |
+
|
| 4 |
+
condition success med iter med t med J
|
| 5 |
+
cold 53.3% 363 1.95s 4.06
|
| 6 |
+
nearest-warm 76.7% 45 0.25s 4.15
|
| 7 |
+
generated-warm 76.7% 42 0.25s 4.15
|
| 8 |
+
|
| 9 |
+
--- paired tests ---
|
| 10 |
+
generated-warm vs cold success: beats 14, loses 0, McNemar p=0.0001 * time ratio 0.152x (n=32), Wilcoxon p=0.0000 *
|
| 11 |
+
generated-warm vs nearest-warm success: beats 0, loses 0, McNemar p=1.0000 time ratio 0.959x (n=46), Wilcoxon p=0.3024
|
| 12 |
+
nearest-warm vs cold success: beats 14, loses 0, McNemar p=0.0001 * time ratio 0.157x (n=32), Wilcoxon p=0.0000 *
|
| 13 |
+
|
| 14 |
+
--- failure breakdown ---
|
| 15 |
+
cold ipopt: Maximum_Iterations_Exceeded:13 ipopt: Infeasible_Problem_Detected:11 rollout failed:4
|
| 16 |
+
nearest-warm ipopt: Infeasible_Problem_Detected:6 rollout failed:5 ipopt: Maximum_Iterations_Exceeded:3
|
| 17 |
+
generated-warm ipopt: Infeasible_Problem_Detected:6 ipopt: Maximum_Iterations_Exceeded:5 rollout failed:3
|
results/furuta_v4/traj_gen_warmstart_v4unetA_s1.txt
ADDED
|
@@ -0,0 +1,17 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
=== warm-start test (n=60, seed 4242, max_iter=700) ===
|
| 2 |
+
target certificate: largest_certified_region per plant (unmargined); verifier: furuta_traj_bank.score_solution (identical to every other arm in this doc)
|
| 3 |
+
|
| 4 |
+
condition success med iter med t med J
|
| 5 |
+
cold 53.3% 363 1.98s 4.06
|
| 6 |
+
nearest-warm 76.7% 45 0.26s 4.15
|
| 7 |
+
generated-warm 76.7% 46 0.26s 4.15
|
| 8 |
+
|
| 9 |
+
--- paired tests ---
|
| 10 |
+
generated-warm vs cold success: beats 14, loses 0, McNemar p=0.0001 * time ratio 0.159x (n=32), Wilcoxon p=0.0000 *
|
| 11 |
+
generated-warm vs nearest-warm success: beats 0, loses 0, McNemar p=1.0000 time ratio 1.023x (n=46), Wilcoxon p=0.5957
|
| 12 |
+
nearest-warm vs cold success: beats 14, loses 0, McNemar p=0.0001 * time ratio 0.161x (n=32), Wilcoxon p=0.0000 *
|
| 13 |
+
|
| 14 |
+
--- failure breakdown ---
|
| 15 |
+
cold ipopt: Maximum_Iterations_Exceeded:13 ipopt: Infeasible_Problem_Detected:11 rollout failed:4
|
| 16 |
+
nearest-warm ipopt: Infeasible_Problem_Detected:6 rollout failed:5 ipopt: Maximum_Iterations_Exceeded:3
|
| 17 |
+
generated-warm ipopt: Maximum_Iterations_Exceeded:6 ipopt: Infeasible_Problem_Detected:5 rollout failed:3
|
results/guidance/A16x_results.txt
ADDED
|
@@ -0,0 +1,13 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
Task A budget-scaled sensitivity: 16x budget (evals/arm = 7688), arms a,d,e,f, 240 units
|
| 2 |
+
u_max=0.6mgl: a: cov 0.308 div 0.120 union 0.312 cost 16.68 wall 92s d: cov 0.524 div 0.011 union 0.524 cost 4.81 wall 21s e: cov 0.562 div 0.183 union 0.565 cost 4.94 wall 21s f: cov 0.539 div 0.165 union 0.541 cost 4.92 wall 106s
|
| 3 |
+
d vs e: diff -0.0387 p=1.53e-05 wins/ties/losses 3/13/24
|
| 4 |
+
d vs a: diff +0.2161 p=1.14e-07 wins/ties/losses 37/3/0
|
| 5 |
+
d vs f: diff -0.0154 p=4.89e-05 wins/ties/losses 2/17/21
|
| 6 |
+
e vs a: diff +0.2547 p=5.26e-08 wins/ties/losses 39/1/0
|
| 7 |
+
e vs f: diff +0.0233 p=0.0251 wins/ties/losses 22/6/12
|
| 8 |
+
u_max=0.9mgl: a: cov 0.822 div 0.218 union 0.845 cost 5.88 wall 103s d: cov 0.875 div 0.010 union 0.875 cost 3.83 wall 17s e: cov 0.911 div 0.192 union 0.916 cost 3.97 wall 15s f: cov 0.877 div 0.100 union 0.878 cost 3.93 wall 53s
|
| 9 |
+
d vs e: diff -0.0364 p=2.7e-05 wins/ties/losses 0/17/23
|
| 10 |
+
d vs a: diff +0.0527 p=2.01e-05 wins/ties/losses 28/8/4
|
| 11 |
+
d vs f: diff -0.0026 p=0.0408 wins/ties/losses 4/29/7
|
| 12 |
+
e vs a: diff +0.0891 p=1.15e-07 wins/ties/losses 37/2/1
|
| 13 |
+
e vs f: diff +0.0338 p=1.18e-05 wins/ties/losses 25/14/1
|
results/guidance/A_ddim_check.txt
ADDED
|
@@ -0,0 +1 @@
|
|
|
|
|
|
|
| 1 |
+
DDIM20 vs DDPM500 (pendulum, 10 plants, u_max=0.9mgl, 41x41 grid coverage): best-of-8 DDPM 0.887 DDIM 0.887; mean-of-8 DDPM 0.887 DDIM 0.881
|
results/guidance/A_results.txt
ADDED
|
@@ -0,0 +1,47 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
Task A: torque-bounded pendulum, N=40 plants (random_pendulum_params seed 321), seeds (0, 1, 2), K=8, 240 units
|
| 2 |
+
budget: evals/arm = 3*K*S+K = 488 (S=20); backward=2 forward-equivalents; 1 eval = 1 controller x 80 ICs
|
| 3 |
+
|
| 4 |
+
===== u_max = 0.6 * m g l (units 120) =====
|
| 5 |
+
arm coverage union(K) in-reg cost cert diversity wall s evals
|
| 6 |
+
retrieval K=8 (no search) 0.524+-0.000[0.48,0.57] 0.524 4.81 1.00 0.068 0.1 8
|
| 7 |
+
(a) retrieval + steering 0.524+-0.000[0.48,0.57] 0.524 4.92 0.06 0.068 5.0 488
|
| 8 |
+
prior K=8 (no search) 0.524+-0.000[0.48,0.57] 0.524 4.80 0.98 0.018 0.1 8
|
| 9 |
+
(b) prior + steering 0.524+-0.000[0.48,0.57] 0.524 4.95 0.11 0.017 4.9 488
|
| 10 |
+
(c) in-denoising gradient guidance 0.523+-0.000[0.47,0.57] 0.523 4.81 0.21 0.032 6.0 488
|
| 11 |
+
(d) SMC gradient-free guidance 0.524+-0.000[0.48,0.57] 0.524 4.80 1.00 0.008 1.2 488
|
| 12 |
+
(e) CEM on retrieval 0.538+-0.006[0.49,0.58] 0.539 4.91 0.20 0.198 1.2 488
|
| 13 |
+
prior best-of-488 0.524+-0.000[0.48,0.57] 0.524 4.82 0.72 0.046 1.5 488
|
| 14 |
+
paired Wilcoxon on per-plant coverage (seed-mean over 3 seeds), 40 plants; diff = arm - ref, two-sided p
|
| 15 |
+
b vs a: diff +0.0000 [+0.0000,+0.0000] p=1 wins/ties/losses 0/40/0
|
| 16 |
+
b vs e: diff -0.0141 [-0.0250,-0.0057] p=0.000155 wins/ties/losses 4/18/18
|
| 17 |
+
c vs a: diff -0.0007 [-0.0018,+0.0000] p=0.109 wins/ties/losses 0/37/3
|
| 18 |
+
c vs e: diff -0.0148 [-0.0259,-0.0062] p=6.33e-05 wins/ties/losses 3/17/20
|
| 19 |
+
d vs a: diff +0.0000 [+0.0000,+0.0000] p=1 wins/ties/losses 0/40/0
|
| 20 |
+
d vs e: diff -0.0141 [-0.0250,-0.0057] p=0.000155 wins/ties/losses 4/18/18
|
| 21 |
+
d vs f: diff -0.0000 p=0.317
|
| 22 |
+
e vs a: diff +0.0141 p=0.000155
|
| 23 |
+
e vs f: diff +0.0141 p=0.000166
|
| 24 |
+
PRE-REGISTERED (c or d beats BOTH a and e, p<0.05, paired): c=False d=False
|
| 25 |
+
|
| 26 |
+
===== u_max = 0.9 * m g l (units 120) =====
|
| 27 |
+
arm coverage union(K) in-reg cost cert diversity wall s evals
|
| 28 |
+
retrieval K=8 (no search) 0.875+-0.000[0.85,0.90] 0.875 3.84 1.00 0.068 0.1 8
|
| 29 |
+
(a) retrieval + steering 0.875+-0.000[0.85,0.90] 0.875 3.99 0.00 0.068 4.9 488
|
| 30 |
+
prior K=8 (no search) 0.875+-0.000[0.85,0.90] 0.875 3.84 0.97 0.019 0.1 8
|
| 31 |
+
(b) prior + steering 0.875+-0.000[0.85,0.90] 0.875 4.02 0.05 0.018 4.9 488
|
| 32 |
+
(c) in-denoising gradient guidance 0.874+-0.001[0.85,0.90] 0.874 4.01 0.00 0.045 6.1 488
|
| 33 |
+
(d) SMC gradient-free guidance 0.875+-0.000[0.85,0.90] 0.875 3.83 1.00 0.008 1.2 488
|
| 34 |
+
(e) CEM on retrieval 0.881+-0.001[0.86,0.90] 0.882 3.96 0.19 0.168 1.2 488
|
| 35 |
+
prior best-of-488 0.875+-0.000[0.85,0.90] 0.875 3.93 0.77 0.039 1.5 488
|
| 36 |
+
paired Wilcoxon on per-plant coverage (seed-mean over 3 seeds), 40 plants; diff = arm - ref, two-sided p
|
| 37 |
+
b vs a: diff +0.0000 [+0.0000,+0.0000] p=1 wins/ties/losses 0/40/0
|
| 38 |
+
b vs e: diff -0.0061 [-0.0107,-0.0024] p=0.00444 wins/ties/losses 1/29/10
|
| 39 |
+
c vs a: diff -0.0011 [-0.0032,+0.0000] p=0.18 wins/ties/losses 0/38/2
|
| 40 |
+
c vs e: diff -0.0072 [-0.0130,-0.0026] p=0.0037 wins/ties/losses 1/28/11
|
| 41 |
+
d vs a: diff +0.0000 [+0.0000,+0.0000] p=1 wins/ties/losses 0/40/0
|
| 42 |
+
d vs e: diff -0.0061 [-0.0107,-0.0024] p=0.00444 wins/ties/losses 1/29/10
|
| 43 |
+
d vs f: diff +0.0000 p=1
|
| 44 |
+
e vs a: diff +0.0061 p=0.00444
|
| 45 |
+
e vs f: diff +0.0061 p=0.00444
|
| 46 |
+
PRE-REGISTERED (c or d beats BOTH a and e, p<0.05, paired): c=False d=False
|
| 47 |
+
|
results/guidance/B_ddim_check.txt
ADDED
|
@@ -0,0 +1,5 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
SISO 30 plants K=8 (search ICs): DDPM500 best J_bb 15.67 stable-any 0.70 per-sample stable 0.542
|
| 2 |
+
DDIM20 eta=0.0: best J_bb 15.62 stable-any 0.67 per-sample stable 0.542
|
| 3 |
+
DDIM20 eta=1.0: best J_bb 17.19 stable-any 0.70 per-sample stable 0.567
|
| 4 |
+
DDIM50 eta=0.0: best J_bb 16.64 stable-any 0.70 per-sample stable 0.512
|
| 5 |
+
DDIM50 eta=1.0: best J_bb 16.10 stable-any 0.67 per-sample stable 0.533
|
results/guidance/B_results.txt
ADDED
|
@@ -0,0 +1,16 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
Task B: SISO black-box J_bb, N=60 plants (gen_siso_plants seed 100), seeds 3, K=8, budget 200 evals/plant (spec: settle 0.05*peak, opposite-side excursion>0.5*peak -> +12.0s, fail=36.0)
|
| 2 |
+
arm J_bb (search ICs) J_bb held-out 16 ICs stable(held-out) evals wall s
|
| 3 |
+
distinct retrieval K=8 (no search) 16.046+-0.000[13.79,18.32] 16.753+-0.000[14.75,18.82] 0.483 8 0.6
|
| 4 |
+
distinct retrieval + CEM 15.179+-0.414[13.02,17.36] 16.855+-0.412[14.94,18.87] 0.428 200 15.4
|
| 5 |
+
retrieval best-of-200 (bank rows) 16.422+-0.000[14.12,18.70] 17.189+-0.000[15.21,19.20] 0.483 200 13.4
|
| 6 |
+
diffusion K=8 (no search) 17.020+-0.302[15.08,19.02] 18.835+-0.412[17.05,20.64] 0.533 8 0.6
|
| 7 |
+
diffusion best-of-200 (no guidance) 11.311+-0.079[9.35,13.37] 15.494+-0.442[13.38,17.67] 0.522 200 179.2
|
| 8 |
+
diffusion + SMC guidance 13.970+-0.096[12.06,16.00] 17.452+-0.123[15.43,19.54] 0.472 200 35.8
|
| 9 |
+
paired Wilcoxon on per-plant J_bb (seed-mean), negative diff = first arm better (lower J)
|
| 10 |
+
[search] ps vs rc: diff -1.209 [-2.232,-0.179] p=0.0407 wins/ties/losses 33/0/27
|
| 11 |
+
[search] ps vs rm: diff -2.452 [-3.508,-1.384] p=5.83e-05 wins/ties/losses 41/0/19
|
| 12 |
+
[search] ps vs pm: diff +2.660 [+2.270,+3.050] p=3.29e-11 wins/ties/losses 2/0/58
|
| 13 |
+
[held-out] ps vs rc: diff +0.597 [-0.657,+1.848] p=0.484 wins/ties/losses 29/0/31
|
| 14 |
+
[held-out] ps vs rm: diff +0.263 [-0.859,+1.413] p=0.546 wins/ties/losses 30/0/30
|
| 15 |
+
[held-out] ps vs pm: diff +1.958 [+1.245,+2.691] p=8.73e-06 wins/ties/losses 13/0/47
|
| 16 |
+
PRE-REGISTERED (ps beats rc, rm AND pm on search J_bb, p<0.05 paired): False
|
results/guidance/pilot_choice.json
ADDED
|
@@ -0,0 +1,17 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"lr": 0.5,
|
| 3 |
+
"lam": 0.03,
|
| 4 |
+
"means": {
|
| 5 |
+
"ret0": -0.5157668590545654,
|
| 6 |
+
"pri0": -0.5157651901245117,
|
| 7 |
+
"steer_ret_lr0.1": -0.5157667398452759,
|
| 8 |
+
"steer_pri_lr0.1": -0.5157657861709595,
|
| 9 |
+
"steer_ret_lr0.5": -0.518862247467041,
|
| 10 |
+
"steer_pri_lr0.5": -0.51885986328125,
|
| 11 |
+
"steer_ret_lr2.0": -0.40492570400238037,
|
| 12 |
+
"steer_pri_lr2.0": -0.37729185819625854,
|
| 13 |
+
"guide_lam0.03": -0.5188661813735962,
|
| 14 |
+
"guide_lam0.1": -0.5141855478286743,
|
| 15 |
+
"guide_lam0.3": -0.3907387852668762
|
| 16 |
+
}
|
| 17 |
+
}
|
results/icml_r3/RESUME.md
ADDED
|
@@ -0,0 +1,35 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Round 3 resume note (written 2026-10-13 02:45 PT, in case the orchestrator session ends)
|
| 2 |
+
|
| 3 |
+
State at writing:
|
| 4 |
+
- DONE on virgo (CPU): X4 verifier audit, X9 multimodality, X7 + X10 SOF baselines. Numbers are in
|
| 5 |
+
`paper_icml/plan/inventory/r3.md`; claim-hierarchy, experiments-needed and terminology are updated.
|
| 6 |
+
- RUNNING detached on andromeda (setsid/nohup, PPID 1, `timeout` ending 08:20 PT; watchdog
|
| 7 |
+
`~/icml_r3_pids/watchdog.sh` TERMs recorded PIDs at 08:25 PT):
|
| 8 |
+
- X2 (Furuta, corrected arm): 3 U-Net seeds + MC-dropout twin training. Post-training steps
|
| 9 |
+
are scripted as a detached driver; see `X2/RESUME.md`.
|
| 10 |
+
- X1S + X3 (SOF multi-hypothesis arms + seeds): `X1S/chain_grid.sh` chains; see `X1S/RESUME.md`.
|
| 11 |
+
- X1F (Furuta multi-hypothesis arms): `X1F/x1f_queue.py`; see `X1F/RESUME.md`.
|
| 12 |
+
- PIDs: `andromeda:~/icml_r3_pids/{X2,X1S,X1F}.txt`.
|
| 13 |
+
- andromeda use (GPU and CPU) ends 08:30 PT 2026-10-13. Never touch processes not in those PID files
|
| 14 |
+
(another user, and the user's own `claude` session there).
|
| 15 |
+
|
| 16 |
+
Next session, in order:
|
| 17 |
+
1. After 08:30 PT: `ssh andromeda 'ps -u cercola -o pid,etime,args'`. Nothing from the PID files may be
|
| 18 |
+
left; check `~/icml_r3_pids/watchdog.log`.
|
| 19 |
+
2. Copy results back (file reads only, no compute):
|
| 20 |
+
`rsync -a andromeda:~/DiffusionControl/results/icml_r3/{X2,X1S,X1F}/ results/icml_r3/<same>/`, plus
|
| 21 |
+
any `models/` checkpoints the per-experiment RESUME.md names.
|
| 22 |
+
3. Any step a RESUME.md lists as "not done" runs on virgo CPU only (the virgo GPU is NOT authorized).
|
| 23 |
+
4. Write docs/icml_r3_x2.md, icml_r3_x1_sof.md, icml_r3_x1_furuta.md (research-doc skill) if the agents
|
| 24 |
+
did not finish them; add the numbers to `paper_icml/plan/inventory/r3.md`.
|
| 25 |
+
5. Then continue the paper plan: rebuild Fig 2/3 and Table 1 (retrieval = learned ARD metric, triage
|
| 26 |
+
rows 1-3), build Fig 4 (X9 index vs margins), then plan and draft Experiments via the zeroshot loop.
|
| 27 |
+
- X11 (idle-GPU timing) waits for the user's go. Until then, no time claims.
|
| 28 |
+
|
| 29 |
+
## Update 2026-10-13 07:20 PT: ALL ROUND-3 COMPUTE DONE
|
| 30 |
+
- X2, X1S/X3 and X1F drivers finished by 06:30 PT. The orchestrator stopped the idle X1S queue runner and the
|
| 31 |
+
watchdog at ~07:05; andromeda has no processes from this work. Results and checkpoints (models/mhx,
|
| 32 |
+
models/x1f, models/v4/x2_*) were copied back to virgo.
|
| 33 |
+
- Numbers: paper_icml/plan/inventory/r3.md (X2a-k, X1Sa-c, X1Fa-c, X3a-b). claim-hierarchy C1/C2 updated.
|
| 34 |
+
- In progress (Sonnet agents): docs/icml_r3_{x2,x1_sof,x1_furuta}.md; Fig 2/3 + Table 1 rebuild + appendix X1 figures.
|
| 35 |
+
- Optional cleanup (ask the user first): andromeda models/x1f/*_resume.pt and models/mhx scratch.
|
results/icml_r3/RULES.md
ADDED
|
@@ -0,0 +1,44 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Shared rules for every agent in the ICML round-3 runs (read fully before doing anything)
|
| 2 |
+
|
| 3 |
+
Repo: /home/cercola/matte_server/DiffusionControl (branch icml-experiments; do NOT commit or push).
|
| 4 |
+
Paper: ICML 2027. Plan: paper_icml/plan/{claim-hierarchy,experiments-needed}.md; per-area evidence in
|
| 5 |
+
paper_icml/plan/inventory/*.md (read the parts relevant to you). Round-2 rules and lessons:
|
| 6 |
+
results/icml_r2/RULES.md (the clip bug, recipes, statistics) - read it too, but THIS file overrides it.
|
| 7 |
+
Make the method as strong as possible: baselines are targets, not verdicts; diagnose before concluding;
|
| 8 |
+
report honestly. Same backbone, data, steps, EMA and verifier for every arm you compare.
|
| 9 |
+
|
| 10 |
+
## Compute (authorized by the user for this window only)
|
| 11 |
+
- GPU: ONLY andromeda's RTX 5090 (ssh andromeda). The user authorized it until 08:30 PT 2026-10-13.
|
| 12 |
+
Launch EVERY GPU process under a hard timeout that ends by 08:20 PT:
|
| 13 |
+
T=$(( $(TZ=America/Los_Angeles date -d 'today 08:20' +%s) - $(date +%s) )); setsid nohup timeout --signal=TERM $T <cmd> > <log> 2>&1 < /dev/null & disown
|
| 14 |
+
Checkpoint at least every ~30 min so runs can resume in a later window. No GPU launches after 08:00 PT.
|
| 15 |
+
- The GPU is shared with other agents of this session and possibly other users. Your VRAM budget is stated
|
| 16 |
+
in your task; check `nvidia-smi --query-gpu=memory.free --format=csv,noheader,nounits` before each launch
|
| 17 |
+
and keep >= 3000 MiB free. NEVER kill/stop/pause a process you did not start. Record your PIDs in
|
| 18 |
+
~/icml_r3_pids/<your agent id>.txt on the machine where they run; kill only by those PIDs.
|
| 19 |
+
- virgo (local): CPU ONLY. Never use the virgo GPU (export CUDA_VISIBLE_DEVICES= for every virgo job).
|
| 20 |
+
Another user is running jobs on virgo; respect your worker cap.
|
| 21 |
+
- andromeda CPU: 32 threads; respect your stated thread cap. The user's window covers ALL of andromeda (CPU too): every andromeda process of yours must finish or be stopped by 08:30 PT (use the same timeout for CPU jobs there); anything still needed after that moves to virgo CPU. andromeda disk is 99% full (~54 GB free):
|
| 22 |
+
keep your andromeda outputs under 8 GB, never duplicate big datasets, delete your own scratch when done.
|
| 23 |
+
|
| 24 |
+
## Code and sync
|
| 25 |
+
- Local python: myenv/bin/python (PYTHONPATH=. OMP_NUM_THREADS=1). andromeda: ~/DiffusionControl (rsync copy,
|
| 26 |
+
not git; code synced 2026-10-13 00:15), venv ~/DiffusionControl/.venv (torch cu128, numpy 1.26: use the
|
| 27 |
+
np.trapezoid = np.trapz shim when a module needs it), run with WANDB_MODE=offline.
|
| 28 |
+
- Other agents edit code concurrently: write NEW files (or opt-in flags in files nobody else touches); never
|
| 29 |
+
change defaults of shared modules. Sync to andromeda ONLY the specific files/dirs you created or need
|
| 30 |
+
(rsync -az -e ssh <path> andromeda:~/DiffusionControl/<path>); never a whole-repo rsync, never --delete.
|
| 31 |
+
|
| 32 |
+
## Protocol
|
| 33 |
+
- Before scoring any test set, write results/icml_r3/<ID>/PREREG.md: arms, n, plants (seeds), metric,
|
| 34 |
+
test, multiplicity correction. Choose configs on validation plants only. Held-out plants never in a bank.
|
| 35 |
+
- Paired tests (exact McNemar / sign-flip permutation), bootstrap CIs over plants; when several training
|
| 36 |
+
seeds exist, report per-seed results and treat the SEED as the resampling unit for the across-seed
|
| 37 |
+
claim (no pooled-over-seeds McNemar). Name the bank (size, file) next to every retrieval number.
|
| 38 |
+
- Outputs: results/icml_r3/<ID>/ (copy results back from andromeda), figures under figures/icml_r3/ with
|
| 39 |
+
NO titles or in-axes text (explanations go in captions), and a doc docs/icml_r3_<id>.md written with the
|
| 40 |
+
research-doc skill (~/.claude/skills/research-doc/SKILL.md).
|
| 41 |
+
- Every job > 2 min must survive session death (setsid nohup ... & disown) and be resumable; poll with
|
| 42 |
+
sleep <= 300 loops.
|
| 43 |
+
- Return a SHORT report (<= 25 lines): headline numbers with n/CI/p, paths, what is still running
|
| 44 |
+
(pid, log, expected end), what is blocked and why.
|
results/ood/furuta_ood.txt
ADDED
|
@@ -0,0 +1,165 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
=== Furuta lvl0: N=60, NN ratio 1.0x; per-plant fresh-solve success 0.950 ===
|
| 2 |
+
arm K=1 K=4 K=8
|
| 3 |
+
nn 0.567 0.850 0.983
|
| 4 |
+
idw 0.567 0.800 0.967
|
| 5 |
+
gen_v3_s0 0.333 0.800 0.867
|
| 6 |
+
gen_v3_s1 0.417 0.800 0.850
|
| 7 |
+
gen_v3_s2 0.533 0.817 0.917
|
| 8 |
+
gen_v3e_s0 0.433 0.850 0.883
|
| 9 |
+
gen_v3e_s1 0.550 0.867 0.917
|
| 10 |
+
gen_v3e_s2 0.483 0.833 0.900
|
| 11 |
+
gen_v3e_t1_s0 0.583 0.867 0.900
|
| 12 |
+
K=1 gen_v3_s0 vs best retrieval (nn): model-only 8 retr-only 22 p=0.0161 diff -0.233
|
| 13 |
+
K=1 gen_v3_s1 vs best retrieval (nn): model-only 8 retr-only 17 p=0.108 diff -0.150
|
| 14 |
+
K=1 gen_v3_s2 vs best retrieval (nn): model-only 9 retr-only 11 p=0.824 diff -0.033
|
| 15 |
+
K=1 gen_v3e_s0 vs best retrieval (nn): model-only 10 retr-only 18 p=0.185 diff -0.133
|
| 16 |
+
K=1 gen_v3e_s1 vs best retrieval (nn): model-only 12 retr-only 13 p=1 diff -0.017
|
| 17 |
+
K=1 gen_v3e_s2 vs best retrieval (nn): model-only 8 retr-only 13 p=0.383 diff -0.083
|
| 18 |
+
K=1 gen_v3e_t1_s0 vs best retrieval (nn): model-only 12 retr-only 11 p=1 diff +0.017
|
| 19 |
+
K=4 gen_v3_s0 vs best retrieval (nn): model-only 4 retr-only 7 p=0.549 diff -0.050
|
| 20 |
+
K=4 gen_v3_s1 vs best retrieval (nn): model-only 3 retr-only 6 p=0.508 diff -0.050
|
| 21 |
+
K=4 gen_v3_s2 vs best retrieval (nn): model-only 4 retr-only 6 p=0.754 diff -0.033
|
| 22 |
+
K=4 gen_v3e_s0 vs best retrieval (nn): model-only 5 retr-only 5 p=1 diff +0.000
|
| 23 |
+
K=4 gen_v3e_s1 vs best retrieval (nn): model-only 4 retr-only 3 p=1 diff +0.017
|
| 24 |
+
K=4 gen_v3e_s2 vs best retrieval (nn): model-only 5 retr-only 6 p=1 diff -0.017
|
| 25 |
+
K=4 gen_v3e_t1_s0 vs best retrieval (nn): model-only 5 retr-only 4 p=1 diff +0.017
|
| 26 |
+
K=8 gen_v3_s0 vs best retrieval (nn): model-only 0 retr-only 7 p=0.0156 diff -0.117
|
| 27 |
+
K=8 gen_v3_s1 vs best retrieval (nn): model-only 0 retr-only 8 p=0.00781 diff -0.133
|
| 28 |
+
K=8 gen_v3_s2 vs best retrieval (nn): model-only 0 retr-only 4 p=0.125 diff -0.067
|
| 29 |
+
K=8 gen_v3e_s0 vs best retrieval (nn): model-only 0 retr-only 6 p=0.0312 diff -0.100
|
| 30 |
+
K=8 gen_v3e_s1 vs best retrieval (nn): model-only 0 retr-only 4 p=0.125 diff -0.067
|
| 31 |
+
K=8 gen_v3e_s2 vs best retrieval (nn): model-only 0 retr-only 5 p=0.0625 diff -0.083
|
| 32 |
+
K=8 gen_v3e_t1_s0 vs best retrieval (nn): model-only 0 retr-only 5 p=0.0625 diff -0.083
|
| 33 |
+
|
| 34 |
+
=== Furuta lvl1.1: N=60, NN ratio 1.3x; per-plant fresh-solve success 0.833 ===
|
| 35 |
+
arm K=1 K=4 K=8
|
| 36 |
+
nn 0.400 0.550 0.650
|
| 37 |
+
idw 0.400 0.533 0.617
|
| 38 |
+
gen_v3_s0 0.267 0.633 0.667
|
| 39 |
+
gen_v3_s1 0.217 0.550 0.700
|
| 40 |
+
gen_v3_s2 0.300 0.583 0.633
|
| 41 |
+
gen_v3e_s0 0.350 0.733 0.750
|
| 42 |
+
gen_v3e_s1 0.400 0.733 0.767
|
| 43 |
+
gen_v3e_s2 0.433 0.683 0.733
|
| 44 |
+
gen_v3e_t1_s0 0.400 0.733 0.783
|
| 45 |
+
K=1 gen_v3_s0 vs best retrieval (nn): model-only 8 retr-only 16 p=0.152 diff -0.133
|
| 46 |
+
K=1 gen_v3_s1 vs best retrieval (nn): model-only 4 retr-only 15 p=0.0192 diff -0.183
|
| 47 |
+
K=1 gen_v3_s2 vs best retrieval (nn): model-only 6 retr-only 12 p=0.238 diff -0.100
|
| 48 |
+
K=1 gen_v3e_s0 vs best retrieval (nn): model-only 10 retr-only 13 p=0.678 diff -0.050
|
| 49 |
+
K=1 gen_v3e_s1 vs best retrieval (nn): model-only 12 retr-only 12 p=1 diff +0.000
|
| 50 |
+
K=1 gen_v3e_s2 vs best retrieval (nn): model-only 10 retr-only 8 p=0.815 diff +0.033
|
| 51 |
+
K=1 gen_v3e_t1_s0 vs best retrieval (nn): model-only 10 retr-only 10 p=1 diff +0.000
|
| 52 |
+
K=4 gen_v3_s0 vs best retrieval (nn): model-only 9 retr-only 4 p=0.267 diff +0.083
|
| 53 |
+
K=4 gen_v3_s1 vs best retrieval (nn): model-only 9 retr-only 9 p=1 diff +0.000
|
| 54 |
+
K=4 gen_v3_s2 vs best retrieval (nn): model-only 6 retr-only 4 p=0.754 diff +0.033
|
| 55 |
+
K=4 gen_v3e_s0 vs best retrieval (nn): model-only 12 retr-only 1 p=0.00342 diff +0.183
|
| 56 |
+
K=4 gen_v3e_s1 vs best retrieval (nn): model-only 12 retr-only 1 p=0.00342 diff +0.183
|
| 57 |
+
K=4 gen_v3e_s2 vs best retrieval (nn): model-only 11 retr-only 3 p=0.0574 diff +0.133
|
| 58 |
+
K=4 gen_v3e_t1_s0 vs best retrieval (nn): model-only 12 retr-only 1 p=0.00342 diff +0.183
|
| 59 |
+
K=8 gen_v3_s0 vs best retrieval (nn): model-only 5 retr-only 4 p=1 diff +0.017
|
| 60 |
+
K=8 gen_v3_s1 vs best retrieval (nn): model-only 6 retr-only 3 p=0.508 diff +0.050
|
| 61 |
+
K=8 gen_v3_s2 vs best retrieval (nn): model-only 2 retr-only 3 p=1 diff -0.017
|
| 62 |
+
K=8 gen_v3e_s0 vs best retrieval (nn): model-only 7 retr-only 1 p=0.0703 diff +0.100
|
| 63 |
+
K=8 gen_v3e_s1 vs best retrieval (nn): model-only 8 retr-only 1 p=0.0391 diff +0.117
|
| 64 |
+
K=8 gen_v3e_s2 vs best retrieval (nn): model-only 8 retr-only 3 p=0.227 diff +0.083
|
| 65 |
+
K=8 gen_v3e_t1_s0 vs best retrieval (nn): model-only 9 retr-only 1 p=0.0215 diff +0.133
|
| 66 |
+
|
| 67 |
+
=== Furuta lvl1.25: N=60, NN ratio 2.1x; per-plant fresh-solve success 0.750 ===
|
| 68 |
+
arm K=1 K=4 K=8
|
| 69 |
+
nn 0.217 0.533 0.633
|
| 70 |
+
idw 0.217 0.483 0.600
|
| 71 |
+
gen_v3_s0 0.233 0.567 0.600
|
| 72 |
+
gen_v3_s1 0.267 0.550 0.583
|
| 73 |
+
gen_v3_s2 0.317 0.483 0.617
|
| 74 |
+
gen_v3e_s0 0.317 0.567 0.617
|
| 75 |
+
gen_v3e_s1 0.217 0.500 0.567
|
| 76 |
+
gen_v3e_s2 0.300 0.517 0.617
|
| 77 |
+
gen_v3e_t1_s0 0.333 0.633 0.650
|
| 78 |
+
K=1 gen_v3_s0 vs best retrieval (nn): model-only 8 retr-only 7 p=1 diff +0.017
|
| 79 |
+
K=1 gen_v3_s1 vs best retrieval (nn): model-only 12 retr-only 9 p=0.664 diff +0.050
|
| 80 |
+
K=1 gen_v3_s2 vs best retrieval (nn): model-only 14 retr-only 8 p=0.286 diff +0.100
|
| 81 |
+
K=1 gen_v3e_s0 vs best retrieval (nn): model-only 13 retr-only 7 p=0.263 diff +0.100
|
| 82 |
+
K=1 gen_v3e_s1 vs best retrieval (nn): model-only 10 retr-only 10 p=1 diff +0.000
|
| 83 |
+
K=1 gen_v3e_s2 vs best retrieval (nn): model-only 13 retr-only 8 p=0.383 diff +0.083
|
| 84 |
+
K=1 gen_v3e_t1_s0 vs best retrieval (nn): model-only 14 retr-only 7 p=0.189 diff +0.117
|
| 85 |
+
K=4 gen_v3_s0 vs best retrieval (nn): model-only 8 retr-only 6 p=0.791 diff +0.033
|
| 86 |
+
K=4 gen_v3_s1 vs best retrieval (nn): model-only 7 retr-only 6 p=1 diff +0.017
|
| 87 |
+
K=4 gen_v3_s2 vs best retrieval (nn): model-only 6 retr-only 9 p=0.607 diff -0.050
|
| 88 |
+
K=4 gen_v3e_s0 vs best retrieval (nn): model-only 8 retr-only 6 p=0.791 diff +0.033
|
| 89 |
+
K=4 gen_v3e_s1 vs best retrieval (nn): model-only 5 retr-only 7 p=0.774 diff -0.033
|
| 90 |
+
K=4 gen_v3e_s2 vs best retrieval (nn): model-only 7 retr-only 8 p=1 diff -0.017
|
| 91 |
+
K=4 gen_v3e_t1_s0 vs best retrieval (nn): model-only 9 retr-only 3 p=0.146 diff +0.100
|
| 92 |
+
K=8 gen_v3_s0 vs best retrieval (nn): model-only 2 retr-only 4 p=0.688 diff -0.033
|
| 93 |
+
K=8 gen_v3_s1 vs best retrieval (nn): model-only 2 retr-only 5 p=0.453 diff -0.050
|
| 94 |
+
K=8 gen_v3_s2 vs best retrieval (nn): model-only 3 retr-only 4 p=1 diff -0.017
|
| 95 |
+
K=8 gen_v3e_s0 vs best retrieval (nn): model-only 3 retr-only 4 p=1 diff -0.017
|
| 96 |
+
K=8 gen_v3e_s1 vs best retrieval (nn): model-only 3 retr-only 7 p=0.344 diff -0.067
|
| 97 |
+
K=8 gen_v3e_s2 vs best retrieval (nn): model-only 3 retr-only 4 p=1 diff -0.017
|
| 98 |
+
K=8 gen_v3e_t1_s0 vs best retrieval (nn): model-only 4 retr-only 3 p=1 diff +0.017
|
| 99 |
+
|
| 100 |
+
=== Furuta lvl1.5: N=60, NN ratio 3.1x; per-plant fresh-solve success 0.667 ===
|
| 101 |
+
arm K=1 K=4 K=8
|
| 102 |
+
nn 0.233 0.433 0.533
|
| 103 |
+
idw 0.233 0.383 0.500
|
| 104 |
+
gen_v3_s0 0.167 0.417 0.483
|
| 105 |
+
gen_v3_s1 0.117 0.367 0.483
|
| 106 |
+
gen_v3_s2 0.183 0.417 0.517
|
| 107 |
+
gen_v3e_s0 0.267 0.400 0.517
|
| 108 |
+
gen_v3e_s1 0.167 0.450 0.500
|
| 109 |
+
gen_v3e_s2 0.183 0.450 0.550
|
| 110 |
+
gen_v3e_t1_s0 0.300 0.483 0.517
|
| 111 |
+
K=1 gen_v3_s0 vs best retrieval (nn): model-only 7 retr-only 11 p=0.481 diff -0.067
|
| 112 |
+
K=1 gen_v3_s1 vs best retrieval (nn): model-only 6 retr-only 13 p=0.167 diff -0.117
|
| 113 |
+
K=1 gen_v3_s2 vs best retrieval (nn): model-only 5 retr-only 8 p=0.581 diff -0.050
|
| 114 |
+
K=1 gen_v3e_s0 vs best retrieval (nn): model-only 9 retr-only 7 p=0.804 diff +0.033
|
| 115 |
+
K=1 gen_v3e_s1 vs best retrieval (nn): model-only 8 retr-only 12 p=0.503 diff -0.067
|
| 116 |
+
K=1 gen_v3e_s2 vs best retrieval (nn): model-only 5 retr-only 8 p=0.581 diff -0.050
|
| 117 |
+
K=1 gen_v3e_t1_s0 vs best retrieval (nn): model-only 10 retr-only 6 p=0.454 diff +0.067
|
| 118 |
+
K=4 gen_v3_s0 vs best retrieval (nn): model-only 5 retr-only 6 p=1 diff -0.017
|
| 119 |
+
K=4 gen_v3_s1 vs best retrieval (nn): model-only 4 retr-only 8 p=0.388 diff -0.067
|
| 120 |
+
K=4 gen_v3_s2 vs best retrieval (nn): model-only 6 retr-only 7 p=1 diff -0.017
|
| 121 |
+
K=4 gen_v3e_s0 vs best retrieval (nn): model-only 5 retr-only 7 p=0.774 diff -0.033
|
| 122 |
+
K=4 gen_v3e_s1 vs best retrieval (nn): model-only 7 retr-only 6 p=1 diff +0.017
|
| 123 |
+
K=4 gen_v3e_s2 vs best retrieval (nn): model-only 7 retr-only 6 p=1 diff +0.017
|
| 124 |
+
K=4 gen_v3e_t1_s0 vs best retrieval (nn): model-only 8 retr-only 5 p=0.581 diff +0.050
|
| 125 |
+
K=8 gen_v3_s0 vs best retrieval (nn): model-only 3 retr-only 6 p=0.508 diff -0.050
|
| 126 |
+
K=8 gen_v3_s1 vs best retrieval (nn): model-only 4 retr-only 7 p=0.549 diff -0.050
|
| 127 |
+
K=8 gen_v3_s2 vs best retrieval (nn): model-only 2 retr-only 3 p=1 diff -0.017
|
| 128 |
+
K=8 gen_v3e_s0 vs best retrieval (nn): model-only 4 retr-only 5 p=1 diff -0.017
|
| 129 |
+
K=8 gen_v3e_s1 vs best retrieval (nn): model-only 4 retr-only 6 p=0.754 diff -0.033
|
| 130 |
+
K=8 gen_v3e_s2 vs best retrieval (nn): model-only 4 retr-only 3 p=1 diff +0.017
|
| 131 |
+
K=8 gen_v3e_t1_s0 vs best retrieval (nn): model-only 4 retr-only 5 p=1 diff -0.017
|
| 132 |
+
|
| 133 |
+
=== Furuta lvl2.0: N=60, NN ratio 6.2x; per-plant fresh-solve success 0.533 ===
|
| 134 |
+
arm K=1 K=4 K=8
|
| 135 |
+
nn 0.083 0.217 0.267
|
| 136 |
+
idw 0.083 0.167 0.233
|
| 137 |
+
gen_v3_s0 0.067 0.183 0.283
|
| 138 |
+
gen_v3_s1 0.083 0.150 0.267
|
| 139 |
+
gen_v3_s2 0.100 0.217 0.317
|
| 140 |
+
gen_v3e_s0 0.067 0.200 0.267
|
| 141 |
+
gen_v3e_s1 0.067 0.133 0.183
|
| 142 |
+
gen_v3e_s2 0.067 0.217 0.250
|
| 143 |
+
gen_v3e_t1_s0 0.050 0.183 0.250
|
| 144 |
+
K=1 gen_v3_s0 vs best retrieval (nn): model-only 2 retr-only 3 p=1 diff -0.017
|
| 145 |
+
K=1 gen_v3_s1 vs best retrieval (nn): model-only 4 retr-only 4 p=1 diff +0.000
|
| 146 |
+
K=1 gen_v3_s2 vs best retrieval (nn): model-only 3 retr-only 2 p=1 diff +0.017
|
| 147 |
+
K=1 gen_v3e_s0 vs best retrieval (nn): model-only 1 retr-only 2 p=1 diff -0.017
|
| 148 |
+
K=1 gen_v3e_s1 vs best retrieval (nn): model-only 2 retr-only 3 p=1 diff -0.017
|
| 149 |
+
K=1 gen_v3e_s2 vs best retrieval (nn): model-only 1 retr-only 2 p=1 diff -0.017
|
| 150 |
+
K=1 gen_v3e_t1_s0 vs best retrieval (nn): model-only 2 retr-only 4 p=0.688 diff -0.033
|
| 151 |
+
K=4 gen_v3_s0 vs best retrieval (nn): model-only 3 retr-only 5 p=0.727 diff -0.033
|
| 152 |
+
K=4 gen_v3_s1 vs best retrieval (nn): model-only 3 retr-only 7 p=0.344 diff -0.067
|
| 153 |
+
K=4 gen_v3_s2 vs best retrieval (nn): model-only 3 retr-only 3 p=1 diff +0.000
|
| 154 |
+
K=4 gen_v3e_s0 vs best retrieval (nn): model-only 4 retr-only 5 p=1 diff -0.017
|
| 155 |
+
K=4 gen_v3e_s1 vs best retrieval (nn): model-only 1 retr-only 6 p=0.125 diff -0.083
|
| 156 |
+
K=4 gen_v3e_s2 vs best retrieval (nn): model-only 4 retr-only 4 p=1 diff +0.000
|
| 157 |
+
K=4 gen_v3e_t1_s0 vs best retrieval (nn): model-only 4 retr-only 6 p=0.754 diff -0.033
|
| 158 |
+
K=8 gen_v3_s0 vs best retrieval (nn): model-only 6 retr-only 5 p=1 diff +0.017
|
| 159 |
+
K=8 gen_v3_s1 vs best retrieval (nn): model-only 5 retr-only 5 p=1 diff +0.000
|
| 160 |
+
K=8 gen_v3_s2 vs best retrieval (nn): model-only 4 retr-only 1 p=0.375 diff +0.050
|
| 161 |
+
K=8 gen_v3e_s0 vs best retrieval (nn): model-only 4 retr-only 4 p=1 diff +0.000
|
| 162 |
+
K=8 gen_v3e_s1 vs best retrieval (nn): model-only 2 retr-only 7 p=0.18 diff -0.083
|
| 163 |
+
K=8 gen_v3e_s2 vs best retrieval (nn): model-only 4 retr-only 5 p=1 diff -0.017
|
| 164 |
+
K=8 gen_v3e_t1_s0 vs best retrieval (nn): model-only 5 retr-only 6 p=1 diff -0.017
|
| 165 |
+
|
results/ood/pendulum_ood.txt
ADDED
|
@@ -0,0 +1,804 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
=== lvl0: N=60 NN-dist ratio test/bank 1.0x (mean NN 0.148); oracle finite 60 ===
|
| 2 |
+
-- no steering: cov / certified-stable cov / median regret % (inf = unstable counts as failure)
|
| 3 |
+
arm K=1: cov cert regret | K=4: cov cert regret | K=8: cov cert regret
|
| 4 |
+
diff_s0 | 1.000 1.000 1.44 | 1.000 1.000 1.23 | 1.000 1.000 1.09
|
| 5 |
+
diff_s1 | 1.000 1.000 1.49 | 1.000 1.000 1.21 | 1.000 1.000 1.15
|
| 6 |
+
diff_s2 | 1.000 1.000 1.39 | 1.000 1.000 1.20 | 1.000 1.000 1.07
|
| 7 |
+
x_s42 | 1.000 1.000 1.51 | 1.000 1.000 1.28 | 1.000 1.000 1.08
|
| 8 |
+
x_s43 | 1.000 1.000 1.09 | 1.000 1.000 0.73 | 1.000 1.000 0.65
|
| 9 |
+
x_s44 | 1.000 1.000 1.35 | 1.000 1.000 1.17 | 1.000 1.000 1.02
|
| 10 |
+
nn | 1.000 1.000 1.78 | 1.000 1.000 0.68 | 1.000 1.000 0.29
|
| 11 |
+
idw | 1.000 1.000 1.78 | 1.000 1.000 0.71 | 1.000 1.000 0.29
|
| 12 |
+
nns | 1.000 1.000 1.40 | 1.000 1.000 1.21 | 1.000 1.000 1.11
|
| 13 |
+
dls | 1.000 1.000 1.41 | 1.000 1.000 1.23 | 1.000 1.000 1.14
|
| 14 |
+
|
| 15 |
+
-- 50-step steering on every candidate: cov / certified-stable cov / median regret % (inf = unstable counts as failure)
|
| 16 |
+
arm K=1: cov cert regret | K=4: cov cert regret | K=8: cov cert regret
|
| 17 |
+
diff_s0 | 1.000 0.983 0.02 | 1.000 1.000 0.00 | 1.000 1.000 0.00
|
| 18 |
+
diff_s1 | 1.000 0.983 0.01 | 1.000 1.000 0.00 | 1.000 1.000 0.00
|
| 19 |
+
diff_s2 | 1.000 0.983 0.01 | 1.000 1.000 0.00 | 1.000 1.000 0.00
|
| 20 |
+
x_s42 | 1.000 0.967 0.03 | 1.000 1.000 0.00 | 1.000 1.000 0.00
|
| 21 |
+
x_s43 | 1.000 0.983 0.01 | 1.000 1.000 0.00 | 1.000 1.000 -0.00
|
| 22 |
+
x_s44 | 1.000 0.983 0.02 | 1.000 1.000 0.00 | 1.000 1.000 0.00
|
| 23 |
+
nn | 1.000 1.000 0.01 | 1.000 1.000 -0.00 | 1.000 1.000 -0.00
|
| 24 |
+
idw | 1.000 1.000 0.01 | 1.000 1.000 -0.00 | 1.000 1.000 -0.00
|
| 25 |
+
nns | 1.000 1.000 0.01 | 1.000 1.000 0.00 | 1.000 1.000 0.00
|
| 26 |
+
dls | 1.000 1.000 0.01 | 1.000 1.000 0.00 | 1.000 1.000 0.00
|
| 27 |
+
|
| 28 |
+
-- paired tests, diffusion (model - retrieval): [cov McNemar model-only/retr-only p | certified-cov | paired regret diff median [95% CI], Wilcoxon p]; * = best retrieval arm by median regret
|
| 29 |
+
K=1 st=0 diff_s0 vs nn cov 0-0 p=1 | cert 0-0 p=1 | regret -0.21 [-0.48,+0.09] p=0.112
|
| 30 |
+
K=1 st=0 diff_s0 vs idw cov 0-0 p=1 | cert 0-0 p=1 | regret -0.21 [-0.48,+0.09] p=0.112
|
| 31 |
+
K=1 st=0 diff_s0 vs nns * cov 0-0 p=1 | cert 0-0 p=1 | regret +0.07 [-0.01,+0.22] p=0.11
|
| 32 |
+
K=1 st=0 diff_s0 vs dls cov 0-0 p=1 | cert 0-0 p=1 | regret -0.14 [-0.27,+0.04] p=0.178
|
| 33 |
+
K=1 st=0 diff_s1 vs nn cov 0-0 p=1 | cert 0-0 p=1 | regret -0.18 [-0.44,+0.14] p=0.236
|
| 34 |
+
K=1 st=0 diff_s1 vs idw cov 0-0 p=1 | cert 0-0 p=1 | regret -0.18 [-0.44,+0.14] p=0.236
|
| 35 |
+
K=1 st=0 diff_s1 vs nns * cov 0-0 p=1 | cert 0-0 p=1 | regret +0.09 [-0.12,+0.25] p=0.342
|
| 36 |
+
K=1 st=0 diff_s1 vs dls cov 0-0 p=1 | cert 0-0 p=1 | regret -0.09 [-0.21,+0.05] p=0.216
|
| 37 |
+
K=1 st=0 diff_s2 vs nn cov 0-0 p=1 | cert 0-0 p=1 | regret -0.21 [-0.48,-0.01] p=0.153
|
| 38 |
+
K=1 st=0 diff_s2 vs idw cov 0-0 p=1 | cert 0-0 p=1 | regret -0.21 [-0.48,-0.01] p=0.153
|
| 39 |
+
K=1 st=0 diff_s2 vs nns * cov 0-0 p=1 | cert 0-0 p=1 | regret +0.04 [-0.09,+0.14] p=0.422
|
| 40 |
+
K=1 st=0 diff_s2 vs dls cov 0-0 p=1 | cert 0-0 p=1 | regret -0.13 [-0.29,-0.04] p=0.216
|
| 41 |
+
K=1 st=1 diff_s0 vs nn cov 0-0 p=1 | cert 0-1 p=1 | regret -0.00 [-0.03,+0.00] p=0.484
|
| 42 |
+
K=1 st=1 diff_s0 vs idw cov 0-0 p=1 | cert 0-1 p=1 | regret -0.00 [-0.03,+0.00] p=0.484
|
| 43 |
+
K=1 st=1 diff_s0 vs nns cov 0-0 p=1 | cert 0-1 p=1 | regret +0.01 [+0.00,+0.02] p=4.26e-05
|
| 44 |
+
K=1 st=1 diff_s0 vs dls * cov 0-0 p=1 | cert 0-1 p=1 | regret +0.00 [-0.00,+0.02] p=0.0393
|
| 45 |
+
K=1 st=1 diff_s1 vs nn cov 0-0 p=1 | cert 0-1 p=1 | regret -0.00 [-0.02,+0.00] p=0.357
|
| 46 |
+
K=1 st=1 diff_s1 vs idw cov 0-0 p=1 | cert 0-1 p=1 | regret -0.00 [-0.02,+0.00] p=0.357
|
| 47 |
+
K=1 st=1 diff_s1 vs nns cov 0-0 p=1 | cert 0-1 p=1 | regret +0.00 [+0.00,+0.01] p=0.00551
|
| 48 |
+
K=1 st=1 diff_s1 vs dls * cov 0-0 p=1 | cert 0-1 p=1 | regret +0.00 [-0.00,+0.00] p=0.414
|
| 49 |
+
K=1 st=1 diff_s2 vs nn cov 0-0 p=1 | cert 0-1 p=1 | regret -0.00 [-0.03,-0.00] p=0.328
|
| 50 |
+
K=1 st=1 diff_s2 vs idw cov 0-0 p=1 | cert 0-1 p=1 | regret -0.00 [-0.03,-0.00] p=0.328
|
| 51 |
+
K=1 st=1 diff_s2 vs nns cov 0-0 p=1 | cert 0-1 p=1 | regret +0.00 [+0.00,+0.01] p=0.004
|
| 52 |
+
K=1 st=1 diff_s2 vs dls * cov 0-0 p=1 | cert 0-1 p=1 | regret +0.00 [-0.00,+0.01] p=0.357
|
| 53 |
+
K=4 st=0 diff_s0 vs nn * cov 0-0 p=1 | cert 0-0 p=1 | regret +0.53 [+0.41,+0.84] p=2.3e-08
|
| 54 |
+
K=4 st=0 diff_s0 vs idw cov 0-0 p=1 | cert 0-0 p=1 | regret +0.53 [+0.38,+0.84] p=5.35e-07
|
| 55 |
+
K=4 st=0 diff_s0 vs nns cov 0-0 p=1 | cert 0-0 p=1 | regret +0.04 [-0.13,+0.16] p=0.702
|
| 56 |
+
K=4 st=0 diff_s0 vs dls cov 0-0 p=1 | cert 0-0 p=1 | regret -0.11 [-0.26,-0.03] p=0.263
|
| 57 |
+
K=4 st=0 diff_s1 vs nn * cov 0-0 p=1 | cert 0-0 p=1 | regret +0.55 [+0.42,+0.87] p=3.1e-08
|
| 58 |
+
K=4 st=0 diff_s1 vs idw cov 0-0 p=1 | cert 0-0 p=1 | regret +0.55 [+0.36,+0.87] p=6e-07
|
| 59 |
+
K=4 st=0 diff_s1 vs nns cov 0-0 p=1 | cert 0-0 p=1 | regret +0.01 [-0.15,+0.11] p=0.906
|
| 60 |
+
K=4 st=0 diff_s1 vs dls cov 0-0 p=1 | cert 0-0 p=1 | regret -0.13 [-0.31,+0.01] p=0.276
|
| 61 |
+
K=4 st=0 diff_s2 vs nn * cov 0-0 p=1 | cert 0-0 p=1 | regret +0.59 [+0.41,+0.80] p=6.89e-09
|
| 62 |
+
K=4 st=0 diff_s2 vs idw cov 0-0 p=1 | cert 0-0 p=1 | regret +0.57 [+0.39,+0.79] p=2.1e-07
|
| 63 |
+
K=4 st=0 diff_s2 vs nns cov 0-0 p=1 | cert 0-0 p=1 | regret +0.02 [-0.12,+0.13] p=1
|
| 64 |
+
K=4 st=0 diff_s2 vs dls cov 0-0 p=1 | cert 0-0 p=1 | regret -0.12 [-0.19,-0.06] p=0.198
|
| 65 |
+
K=4 st=1 diff_s0 vs nn * cov 0-0 p=1 | cert 0-0 p=1 | regret +0.01 [+0.00,+0.03] p=5.48e-05
|
| 66 |
+
K=4 st=1 diff_s0 vs idw cov 0-0 p=1 | cert 0-0 p=1 | regret +0.01 [+0.00,+0.02] p=0.000118
|
| 67 |
+
K=4 st=1 diff_s0 vs nns cov 0-0 p=1 | cert 0-0 p=1 | regret +0.00 [-0.00,+0.01] p=0.0105
|
| 68 |
+
K=4 st=1 diff_s0 vs dls cov 0-0 p=1 | cert 0-0 p=1 | regret +0.00 [-0.00,+0.00] p=0.178
|
| 69 |
+
K=4 st=1 diff_s1 vs nn * cov 0-0 p=1 | cert 0-0 p=1 | regret +0.00 [+0.00,+0.03] p=5.83e-05
|
| 70 |
+
K=4 st=1 diff_s1 vs idw cov 0-0 p=1 | cert 0-0 p=1 | regret +0.00 [+0.00,+0.03] p=0.000125
|
| 71 |
+
K=4 st=1 diff_s1 vs nns cov 0-0 p=1 | cert 0-0 p=1 | regret +0.00 [-0.00,+0.00] p=0.0158
|
| 72 |
+
K=4 st=1 diff_s1 vs dls cov 0-0 p=1 | cert 0-0 p=1 | regret +0.00 [-0.00,+0.00] p=0.0625
|
| 73 |
+
K=4 st=1 diff_s2 vs nn * cov 0-0 p=1 | cert 0-0 p=1 | regret +0.00 [+0.00,+0.03] p=3.4e-05
|
| 74 |
+
K=4 st=1 diff_s2 vs idw cov 0-0 p=1 | cert 0-0 p=1 | regret +0.00 [+0.00,+0.02] p=1.77e-05
|
| 75 |
+
K=4 st=1 diff_s2 vs nns cov 0-0 p=1 | cert 0-0 p=1 | regret +0.00 [+0.00,+0.01] p=0.000747
|
| 76 |
+
K=4 st=1 diff_s2 vs dls cov 0-0 p=1 | cert 0-0 p=1 | regret +0.00 [-0.00,+0.00] p=0.12
|
| 77 |
+
K=8 st=0 diff_s0 vs nn * cov 0-0 p=1 | cert 0-0 p=1 | regret +0.68 [+0.46,+0.84] p=2.21e-10
|
| 78 |
+
K=8 st=0 diff_s0 vs idw cov 0-0 p=1 | cert 0-0 p=1 | regret +0.68 [+0.45,+0.84] p=2.32e-10
|
| 79 |
+
K=8 st=0 diff_s0 vs nns cov 0-0 p=1 | cert 0-0 p=1 | regret -0.02 [-0.09,+0.11] p=0.648
|
| 80 |
+
K=8 st=0 diff_s0 vs dls cov 0-0 p=1 | cert 0-0 p=1 | regret -0.18 [-0.28,-0.11] p=0.385
|
| 81 |
+
K=8 st=0 diff_s1 vs nn * cov 0-0 p=1 | cert 0-0 p=1 | regret +0.76 [+0.56,+0.96] p=1.74e-10
|
| 82 |
+
K=8 st=0 diff_s1 vs idw cov 0-0 p=1 | cert 0-0 p=1 | regret +0.76 [+0.56,+0.96] p=1.74e-10
|
| 83 |
+
K=8 st=0 diff_s1 vs nns cov 0-0 p=1 | cert 0-0 p=1 | regret +0.03 [-0.06,+0.19] p=0.151
|
| 84 |
+
K=8 st=0 diff_s1 vs dls cov 0-0 p=1 | cert 0-0 p=1 | regret -0.12 [-0.23,-0.03] p=0.675
|
| 85 |
+
K=8 st=0 diff_s2 vs nn * cov 0-0 p=1 | cert 0-0 p=1 | regret +0.68 [+0.57,+0.80] p=1.58e-10
|
| 86 |
+
K=8 st=0 diff_s2 vs idw cov 0-0 p=1 | cert 0-0 p=1 | regret +0.68 [+0.57,+0.80] p=1.74e-10
|
| 87 |
+
K=8 st=0 diff_s2 vs nns cov 0-0 p=1 | cert 0-0 p=1 | regret +0.00 [-0.08,+0.07] p=1
|
| 88 |
+
K=8 st=0 diff_s2 vs dls cov 0-0 p=1 | cert 0-0 p=1 | regret -0.21 [-0.28,-0.03] p=0.16
|
| 89 |
+
K=8 st=1 diff_s0 vs nn * cov 0-0 p=1 | cert 0-0 p=1 | regret +0.01 [+0.00,+0.03] p=9.45e-07
|
| 90 |
+
K=8 st=1 diff_s0 vs idw cov 0-0 p=1 | cert 0-0 p=1 | regret +0.01 [+0.00,+0.03] p=1.36e-07
|
| 91 |
+
K=8 st=1 diff_s0 vs nns cov 0-0 p=1 | cert 0-0 p=1 | regret +0.00 [-0.00,+0.01] p=0.00806
|
| 92 |
+
K=8 st=1 diff_s0 vs dls cov 0-0 p=1 | cert 0-0 p=1 | regret +0.00 [-0.00,+0.01] p=0.0636
|
| 93 |
+
K=8 st=1 diff_s1 vs nn * cov 0-0 p=1 | cert 0-0 p=1 | regret +0.01 [+0.00,+0.03] p=2.46e-06
|
| 94 |
+
K=8 st=1 diff_s1 vs idw cov 0-0 p=1 | cert 0-0 p=1 | regret +0.01 [+0.00,+0.03] p=3.12e-07
|
| 95 |
+
K=8 st=1 diff_s1 vs nns cov 0-0 p=1 | cert 0-0 p=1 | regret +0.00 [-0.00,+0.00] p=0.00721
|
| 96 |
+
K=8 st=1 diff_s1 vs dls cov 0-0 p=1 | cert 0-0 p=1 | regret +0.00 [-0.00,+0.00] p=0.0515
|
| 97 |
+
K=8 st=1 diff_s2 vs nn * cov 0-0 p=1 | cert 0-0 p=1 | regret +0.01 [+0.00,+0.03] p=2.46e-07
|
| 98 |
+
K=8 st=1 diff_s2 vs idw cov 0-0 p=1 | cert 0-0 p=1 | regret +0.01 [+0.00,+0.03] p=2.23e-08
|
| 99 |
+
K=8 st=1 diff_s2 vs nns cov 0-0 p=1 | cert 0-0 p=1 | regret +0.00 [-0.00,+0.00] p=0.0477
|
| 100 |
+
K=8 st=1 diff_s2 vs dls cov 0-0 p=1 | cert 0-0 p=1 | regret +0.00 [-0.00,+0.00] p=0.0812
|
| 101 |
+
|
| 102 |
+
=== lvl1.1: N=60 NN-dist ratio test/bank 1.9x (mean NN 0.268); oracle finite 60 ===
|
| 103 |
+
-- no steering: cov / certified-stable cov / median regret % (inf = unstable counts as failure)
|
| 104 |
+
arm K=1: cov cert regret | K=4: cov cert regret | K=8: cov cert regret
|
| 105 |
+
diff_s0 | 1.000 1.000 1.29 | 1.000 1.000 1.00 | 1.000 1.000 0.92
|
| 106 |
+
diff_s1 | 1.000 1.000 1.28 | 1.000 1.000 1.03 | 1.000 1.000 0.92
|
| 107 |
+
diff_s2 | 1.000 1.000 1.14 | 1.000 1.000 0.94 | 1.000 1.000 0.84
|
| 108 |
+
x_s42 | 1.000 1.000 1.20 | 1.000 1.000 0.96 | 1.000 1.000 0.88
|
| 109 |
+
x_s43 | 1.000 0.983 1.15 | 1.000 1.000 0.85 | 1.000 1.000 0.74
|
| 110 |
+
x_s44 | 1.000 1.000 1.17 | 1.000 1.000 0.93 | 1.000 1.000 0.88
|
| 111 |
+
nn | 1.000 1.000 1.41 | 1.000 1.000 0.68 | 1.000 1.000 0.53
|
| 112 |
+
idw | 1.000 1.000 1.41 | 1.000 1.000 0.75 | 1.000 1.000 0.55
|
| 113 |
+
nns | 1.000 1.000 1.27 | 1.000 1.000 1.06 | 1.000 1.000 0.95
|
| 114 |
+
dls | 1.000 1.000 1.17 | 1.000 1.000 1.05 | 1.000 1.000 0.95
|
| 115 |
+
|
| 116 |
+
-- 50-step steering on every candidate: cov / certified-stable cov / median regret % (inf = unstable counts as failure)
|
| 117 |
+
arm K=1: cov cert regret | K=4: cov cert regret | K=8: cov cert regret
|
| 118 |
+
diff_s0 | 1.000 0.917 0.03 | 1.000 0.967 0.00 | 1.000 0.967 0.00
|
| 119 |
+
diff_s1 | 1.000 0.933 0.02 | 1.000 0.983 0.00 | 1.000 1.000 0.00
|
| 120 |
+
diff_s2 | 1.000 0.917 0.02 | 1.000 0.967 0.00 | 1.000 0.967 0.00
|
| 121 |
+
x_s42 | 1.000 0.900 0.03 | 1.000 0.917 0.00 | 1.000 0.933 0.00
|
| 122 |
+
x_s43 | 1.000 0.867 0.04 | 1.000 0.933 0.01 | 1.000 0.933 0.00
|
| 123 |
+
x_s44 | 1.000 0.917 0.02 | 1.000 0.950 0.00 | 1.000 0.950 0.00
|
| 124 |
+
nn | 1.000 0.967 0.01 | 1.000 1.000 -0.01 | 1.000 1.000 -0.01
|
| 125 |
+
idw | 1.000 0.967 0.01 | 1.000 1.000 -0.00 | 1.000 1.000 -0.01
|
| 126 |
+
nns | 1.000 0.950 0.01 | 1.000 0.950 0.01 | 1.000 0.950 0.00
|
| 127 |
+
dls | 1.000 0.950 0.01 | 1.000 0.950 0.00 | 1.000 0.950 0.00
|
| 128 |
+
|
| 129 |
+
-- paired tests, diffusion (model - retrieval): [cov McNemar model-only/retr-only p | certified-cov | paired regret diff median [95% CI], Wilcoxon p]; * = best retrieval arm by median regret
|
| 130 |
+
K=1 st=0 diff_s0 vs nn cov 0-0 p=1 | cert 0-0 p=1 | regret +0.04 [-0.33,+0.69] p=0.617
|
| 131 |
+
K=1 st=0 diff_s0 vs idw cov 0-0 p=1 | cert 0-0 p=1 | regret +0.04 [-0.33,+0.69] p=0.617
|
| 132 |
+
K=1 st=0 diff_s0 vs nns cov 0-0 p=1 | cert 0-0 p=1 | regret +0.17 [-0.03,+0.29] p=0.046
|
| 133 |
+
K=1 st=0 diff_s0 vs dls * cov 0-0 p=1 | cert 0-0 p=1 | regret +0.04 [-0.13,+0.22] p=0.245
|
| 134 |
+
K=1 st=0 diff_s1 vs nn cov 0-0 p=1 | cert 0-0 p=1 | regret +0.00 [-0.76,+0.47] p=0.814
|
| 135 |
+
K=1 st=0 diff_s1 vs idw cov 0-0 p=1 | cert 0-0 p=1 | regret +0.00 [-0.76,+0.47] p=0.814
|
| 136 |
+
K=1 st=0 diff_s1 vs nns cov 0-0 p=1 | cert 0-0 p=1 | regret +0.05 [-0.08,+0.21] p=0.373
|
| 137 |
+
K=1 st=0 diff_s1 vs dls * cov 0-0 p=1 | cert 0-0 p=1 | regret -0.05 [-0.22,+0.15] p=0.802
|
| 138 |
+
K=1 st=0 diff_s2 vs nn cov 0-0 p=1 | cert 0-0 p=1 | regret -0.07 [-0.55,+0.48] p=0.883
|
| 139 |
+
K=1 st=0 diff_s2 vs idw cov 0-0 p=1 | cert 0-0 p=1 | regret -0.07 [-0.55,+0.48] p=0.883
|
| 140 |
+
K=1 st=0 diff_s2 vs nns cov 0-0 p=1 | cert 0-0 p=1 | regret +0.00 [-0.22,+0.17] p=0.48
|
| 141 |
+
K=1 st=0 diff_s2 vs dls * cov 0-0 p=1 | cert 0-0 p=1 | regret -0.04 [-0.19,+0.17] p=0.912
|
| 142 |
+
K=1 st=1 diff_s0 vs nn * cov 0-0 p=1 | cert 0-3 p=0.25 | regret +0.00 [-0.00,+0.05] p=0.0126
|
| 143 |
+
K=1 st=1 diff_s0 vs idw cov 0-0 p=1 | cert 0-3 p=0.25 | regret +0.00 [-0.00,+0.05] p=0.0126
|
| 144 |
+
K=1 st=1 diff_s0 vs nns cov 0-0 p=1 | cert 0-2 p=0.5 | regret +0.00 [-0.00,+0.03] p=0.00373
|
| 145 |
+
K=1 st=1 diff_s0 vs dls cov 0-0 p=1 | cert 0-2 p=0.5 | regret +0.00 [-0.00,+0.02] p=0.00705
|
| 146 |
+
K=1 st=1 diff_s1 vs nn * cov 0-0 p=1 | cert 0-2 p=0.5 | regret -0.00 [-0.01,+0.01] p=0.401
|
| 147 |
+
K=1 st=1 diff_s1 vs idw cov 0-0 p=1 | cert 0-2 p=0.5 | regret -0.00 [-0.01,+0.01] p=0.401
|
| 148 |
+
K=1 st=1 diff_s1 vs nns cov 0-0 p=1 | cert 0-1 p=1 | regret -0.00 [-0.01,+0.00] p=0.982
|
| 149 |
+
K=1 st=1 diff_s1 vs dls cov 0-0 p=1 | cert 0-1 p=1 | regret -0.00 [-0.00,+0.00] p=0.571
|
| 150 |
+
K=1 st=1 diff_s2 vs nn * cov 0-0 p=1 | cert 1-4 p=0.375 | regret +0.00 [-0.00,+0.02] p=0.052
|
| 151 |
+
K=1 st=1 diff_s2 vs idw cov 0-0 p=1 | cert 1-4 p=0.375 | regret +0.00 [-0.00,+0.02] p=0.052
|
| 152 |
+
K=1 st=1 diff_s2 vs nns cov 0-0 p=1 | cert 1-3 p=0.625 | regret +0.00 [-0.00,+0.03] p=0.00737
|
| 153 |
+
K=1 st=1 diff_s2 vs dls cov 0-0 p=1 | cert 1-3 p=0.625 | regret +0.00 [-0.00,+0.01] p=0.105
|
| 154 |
+
K=4 st=0 diff_s0 vs nn * cov 0-0 p=1 | cert 0-0 p=1 | regret +0.31 [+0.00,+0.67] p=0.0084
|
| 155 |
+
K=4 st=0 diff_s0 vs idw cov 0-0 p=1 | cert 0-0 p=1 | regret +0.20 [-0.06,+0.58] p=0.052
|
| 156 |
+
K=4 st=0 diff_s0 vs nns cov 0-0 p=1 | cert 0-0 p=1 | regret -0.05 [-0.14,+0.05] p=0.906
|
| 157 |
+
K=4 st=0 diff_s0 vs dls cov 0-0 p=1 | cert 0-0 p=1 | regret -0.14 [-0.24,-0.03] p=0.178
|
| 158 |
+
K=4 st=0 diff_s1 vs nn * cov 0-0 p=1 | cert 0-0 p=1 | regret +0.28 [-0.03,+0.65] p=0.0252
|
| 159 |
+
K=4 st=0 diff_s1 vs idw cov 0-0 p=1 | cert 0-0 p=1 | regret +0.17 [-0.11,+0.42] p=0.113
|
| 160 |
+
K=4 st=0 diff_s1 vs nns cov 0-0 p=1 | cert 0-0 p=1 | regret -0.08 [-0.17,-0.01] p=0.306
|
| 161 |
+
K=4 st=0 diff_s1 vs dls cov 0-0 p=1 | cert 0-0 p=1 | regret -0.17 [-0.29,-0.08] p=0.0668
|
| 162 |
+
K=4 st=0 diff_s2 vs nn * cov 0-0 p=1 | cert 0-0 p=1 | regret +0.34 [+0.04,+0.54] p=0.0192
|
| 163 |
+
K=4 st=0 diff_s2 vs idw cov 0-0 p=1 | cert 0-0 p=1 | regret +0.24 [-0.03,+0.53] p=0.0947
|
| 164 |
+
K=4 st=0 diff_s2 vs nns cov 0-0 p=1 | cert 0-0 p=1 | regret -0.05 [-0.23,+0.02] p=0.393
|
| 165 |
+
K=4 st=0 diff_s2 vs dls cov 0-0 p=1 | cert 0-0 p=1 | regret -0.13 [-0.22,-0.04] p=0.0595
|
| 166 |
+
K=4 st=1 diff_s0 vs nn * cov 0-0 p=1 | cert 0-2 p=0.5 | regret +0.02 [+0.00,+0.05] p=2.16e-05
|
| 167 |
+
K=4 st=1 diff_s0 vs idw cov 0-0 p=1 | cert 0-2 p=0.5 | regret +0.01 [+0.00,+0.04] p=6.02e-05
|
| 168 |
+
K=4 st=1 diff_s0 vs nns cov 0-0 p=1 | cert 1-0 p=1 | regret -0.00 [-0.00,+0.00] p=0.424
|
| 169 |
+
K=4 st=1 diff_s0 vs dls cov 0-0 p=1 | cert 1-0 p=1 | regret -0.00 [-0.00,+0.00] p=0.686
|
| 170 |
+
K=4 st=1 diff_s1 vs nn * cov 0-0 p=1 | cert 0-1 p=1 | regret +0.01 [+0.00,+0.03] p=0.000141
|
| 171 |
+
K=4 st=1 diff_s1 vs idw cov 0-0 p=1 | cert 0-1 p=1 | regret +0.01 [+0.00,+0.03] p=0.000268
|
| 172 |
+
K=4 st=1 diff_s1 vs nns cov 0-0 p=1 | cert 2-0 p=0.5 | regret -0.00 [-0.01,+0.00] p=0.478
|
| 173 |
+
K=4 st=1 diff_s1 vs dls cov 0-0 p=1 | cert 2-0 p=0.5 | regret -0.00 [-0.00,+0.00] p=0.695
|
| 174 |
+
K=4 st=1 diff_s2 vs nn * cov 0-0 p=1 | cert 0-2 p=0.5 | regret +0.01 [+0.00,+0.04] p=1.11e-05
|
| 175 |
+
K=4 st=1 diff_s2 vs idw cov 0-0 p=1 | cert 0-2 p=0.5 | regret +0.01 [+0.00,+0.04] p=8.46e-05
|
| 176 |
+
K=4 st=1 diff_s2 vs nns cov 0-0 p=1 | cert 1-0 p=1 | regret -0.00 [-0.00,+0.00] p=0.422
|
| 177 |
+
K=4 st=1 diff_s2 vs dls cov 0-0 p=1 | cert 1-0 p=1 | regret -0.00 [-0.00,+0.00] p=0.466
|
| 178 |
+
K=8 st=0 diff_s0 vs nn * cov 0-0 p=1 | cert 0-0 p=1 | regret +0.28 [+0.02,+0.68] p=0.00419
|
| 179 |
+
K=8 st=0 diff_s0 vs idw cov 0-0 p=1 | cert 0-0 p=1 | regret +0.26 [-0.07,+0.56] p=0.0111
|
| 180 |
+
K=8 st=0 diff_s0 vs nns cov 0-0 p=1 | cert 0-0 p=1 | regret -0.06 [-0.15,+0.07] p=0.825
|
| 181 |
+
K=8 st=0 diff_s0 vs dls cov 0-0 p=1 | cert 0-0 p=1 | regret -0.18 [-0.29,-0.03] p=0.296
|
| 182 |
+
K=8 st=0 diff_s1 vs nn * cov 0-0 p=1 | cert 0-0 p=1 | regret +0.21 [+0.02,+0.69] p=0.00492
|
| 183 |
+
K=8 st=0 diff_s1 vs idw cov 0-0 p=1 | cert 0-0 p=1 | regret +0.19 [+0.01,+0.62] p=0.0113
|
| 184 |
+
K=8 st=0 diff_s1 vs nns cov 0-0 p=1 | cert 0-0 p=1 | regret -0.02 [-0.20,+0.12] p=0.883
|
| 185 |
+
K=8 st=0 diff_s1 vs dls cov 0-0 p=1 | cert 0-0 p=1 | regret -0.18 [-0.29,-0.05] p=0.339
|
| 186 |
+
K=8 st=0 diff_s2 vs nn * cov 0-0 p=1 | cert 0-0 p=1 | regret +0.31 [+0.01,+0.58] p=0.00753
|
| 187 |
+
K=8 st=0 diff_s2 vs idw cov 0-0 p=1 | cert 0-0 p=1 | regret +0.23 [+0.01,+0.56] p=0.0154
|
| 188 |
+
K=8 st=0 diff_s2 vs nns cov 0-0 p=1 | cert 0-0 p=1 | regret -0.06 [-0.17,+0.01] p=0.393
|
| 189 |
+
K=8 st=0 diff_s2 vs dls cov 0-0 p=1 | cert 0-0 p=1 | regret -0.15 [-0.30,-0.08] p=0.18
|
| 190 |
+
K=8 st=1 diff_s0 vs nn * cov 0-0 p=1 | cert 0-2 p=0.5 | regret +0.01 [+0.00,+0.05] p=2.09e-05
|
| 191 |
+
K=8 st=1 diff_s0 vs idw cov 0-0 p=1 | cert 0-2 p=0.5 | regret +0.01 [+0.00,+0.04] p=6.61e-05
|
| 192 |
+
K=8 st=1 diff_s0 vs nns cov 0-0 p=1 | cert 1-0 p=1 | regret +0.00 [-0.00,+0.00] p=0.397
|
| 193 |
+
K=8 st=1 diff_s0 vs dls cov 0-0 p=1 | cert 1-0 p=1 | regret -0.00 [-0.00,+0.00] p=0.437
|
| 194 |
+
K=8 st=1 diff_s1 vs nn * cov 0-0 p=1 | cert 0-0 p=1 | regret +0.01 [+0.00,+0.03] p=3.16e-06
|
| 195 |
+
K=8 st=1 diff_s1 vs idw cov 0-0 p=1 | cert 0-0 p=1 | regret +0.01 [+0.00,+0.03] p=4.84e-06
|
| 196 |
+
K=8 st=1 diff_s1 vs nns cov 0-0 p=1 | cert 3-0 p=0.25 | regret -0.00 [-0.00,+0.00] p=0.551
|
| 197 |
+
K=8 st=1 diff_s1 vs dls cov 0-0 p=1 | cert 3-0 p=0.25 | regret -0.00 [-0.00,+0.00] p=0.478
|
| 198 |
+
K=8 st=1 diff_s2 vs nn * cov 0-0 p=1 | cert 0-2 p=0.5 | regret +0.01 [+0.00,+0.05] p=6.55e-07
|
| 199 |
+
K=8 st=1 diff_s2 vs idw cov 0-0 p=1 | cert 0-2 p=0.5 | regret +0.01 [+0.00,+0.04] p=4.43e-07
|
| 200 |
+
K=8 st=1 diff_s2 vs nns cov 0-0 p=1 | cert 1-0 p=1 | regret -0.00 [-0.00,+0.00] p=0.592
|
| 201 |
+
K=8 st=1 diff_s2 vs dls cov 0-0 p=1 | cert 1-0 p=1 | regret -0.00 [-0.00,+0.00] p=0.299
|
| 202 |
+
|
| 203 |
+
=== lvl1.25: N=60 NN-dist ratio test/bank 3.9x (mean NN 0.565); oracle finite 60 ===
|
| 204 |
+
-- no steering: cov / certified-stable cov / median regret % (inf = unstable counts as failure)
|
| 205 |
+
arm K=1: cov cert regret | K=4: cov cert regret | K=8: cov cert regret
|
| 206 |
+
diff_s0 | 1.000 0.983 1.21 | 1.000 1.000 0.83 | 1.000 1.000 0.70
|
| 207 |
+
diff_s1 | 1.000 0.983 1.11 | 1.000 1.000 0.81 | 1.000 1.000 0.67
|
| 208 |
+
diff_s2 | 1.000 1.000 1.26 | 1.000 1.000 0.81 | 1.000 1.000 0.72
|
| 209 |
+
x_s42 | 1.000 0.983 1.28 | 1.000 1.000 0.85 | 1.000 1.000 0.69
|
| 210 |
+
x_s43 | 1.000 1.000 1.37 | 1.000 1.000 0.88 | 1.000 1.000 0.65
|
| 211 |
+
x_s44 | 1.000 0.983 1.19 | 1.000 1.000 0.79 | 1.000 1.000 0.71
|
| 212 |
+
nn | 1.000 1.000 1.80 | 1.000 1.000 1.22 | 1.000 1.000 1.06
|
| 213 |
+
idw | 1.000 1.000 1.80 | 1.000 1.000 1.27 | 1.000 1.000 1.00
|
| 214 |
+
nns | 1.000 1.000 1.11 | 1.000 1.000 0.97 | 1.000 1.000 0.95
|
| 215 |
+
dls | 1.000 1.000 1.06 | 1.000 1.000 1.00 | 1.000 1.000 0.97
|
| 216 |
+
|
| 217 |
+
-- 50-step steering on every candidate: cov / certified-stable cov / median regret % (inf = unstable counts as failure)
|
| 218 |
+
arm K=1: cov cert regret | K=4: cov cert regret | K=8: cov cert regret
|
| 219 |
+
diff_s0 | 1.000 0.817 0.01 | 1.000 0.833 0.00 | 1.000 0.883 -0.00
|
| 220 |
+
diff_s1 | 1.000 0.783 0.01 | 1.000 0.917 0.00 | 1.000 0.950 -0.00
|
| 221 |
+
diff_s2 | 1.000 0.767 0.01 | 1.000 0.883 -0.00 | 1.000 0.967 -0.00
|
| 222 |
+
x_s42 | 1.000 0.750 0.01 | 1.000 0.817 -0.00 | 1.000 0.833 -0.01
|
| 223 |
+
x_s43 | 1.000 0.800 0.03 | 1.000 0.833 0.00 | 1.000 0.917 0.00
|
| 224 |
+
x_s44 | 1.000 0.817 0.02 | 1.000 0.867 0.00 | 1.000 0.933 0.00
|
| 225 |
+
nn | 1.000 0.950 -0.00 | 1.000 1.000 -0.01 | 1.000 1.000 -0.01
|
| 226 |
+
idw | 1.000 0.950 -0.00 | 1.000 1.000 -0.00 | 1.000 1.000 -0.01
|
| 227 |
+
nns | 1.000 0.850 0.01 | 1.000 0.883 0.00 | 1.000 0.883 0.00
|
| 228 |
+
dls | 1.000 0.833 0.01 | 1.000 0.867 0.00 | 1.000 0.883 0.00
|
| 229 |
+
|
| 230 |
+
-- paired tests, diffusion (model - retrieval): [cov McNemar model-only/retr-only p | certified-cov | paired regret diff median [95% CI], Wilcoxon p]; * = best retrieval arm by median regret
|
| 231 |
+
K=1 st=0 diff_s0 vs nn cov 0-0 p=1 | cert 0-1 p=1 | regret -0.86 [-1.73,+0.08] p=0.178
|
| 232 |
+
K=1 st=0 diff_s0 vs idw cov 0-0 p=1 | cert 0-1 p=1 | regret -0.86 [-1.73,+0.08] p=0.178
|
| 233 |
+
K=1 st=0 diff_s0 vs nns cov 0-0 p=1 | cert 0-1 p=1 | regret +0.12 [-0.14,+0.36] p=0.107
|
| 234 |
+
K=1 st=0 diff_s0 vs dls * cov 0-0 p=1 | cert 0-1 p=1 | regret -0.03 [-0.25,+0.16] p=0.837
|
| 235 |
+
K=1 st=0 diff_s1 vs nn cov 0-0 p=1 | cert 0-1 p=1 | regret -0.65 [-1.80,+0.29] p=0.248
|
| 236 |
+
K=1 st=0 diff_s1 vs idw cov 0-0 p=1 | cert 0-1 p=1 | regret -0.65 [-1.80,+0.29] p=0.248
|
| 237 |
+
K=1 st=0 diff_s1 vs nns cov 0-0 p=1 | cert 0-1 p=1 | regret -0.02 [-0.09,+0.29] p=0.242
|
| 238 |
+
K=1 st=0 diff_s1 vs dls * cov 0-0 p=1 | cert 0-1 p=1 | regret -0.04 [-0.30,+0.23] p=0.848
|
| 239 |
+
K=1 st=0 diff_s2 vs nn cov 0-0 p=1 | cert 0-0 p=1 | regret -0.55 [-1.94,+0.30] p=0.279
|
| 240 |
+
K=1 st=0 diff_s2 vs idw cov 0-0 p=1 | cert 0-0 p=1 | regret -0.55 [-1.94,+0.30] p=0.279
|
| 241 |
+
K=1 st=0 diff_s2 vs nns cov 0-0 p=1 | cert 0-0 p=1 | regret +0.10 [-0.14,+0.38] p=0.0933
|
| 242 |
+
K=1 st=0 diff_s2 vs dls * cov 0-0 p=1 | cert 0-0 p=1 | regret +0.10 [-0.14,+0.31] p=0.494
|
| 243 |
+
K=1 st=1 diff_s0 vs nn * cov 0-0 p=1 | cert 0-8 p=0.00781 | regret +0.01 [+0.00,+0.08] p=0.0069
|
| 244 |
+
K=1 st=1 diff_s0 vs idw cov 0-0 p=1 | cert 0-8 p=0.00781 | regret +0.01 [+0.00,+0.08] p=0.0069
|
| 245 |
+
K=1 st=1 diff_s0 vs nns cov 0-0 p=1 | cert 1-3 p=0.625 | regret +0.00 [-0.00,+0.01] p=0.381
|
| 246 |
+
K=1 st=1 diff_s0 vs dls cov 0-0 p=1 | cert 1-2 p=1 | regret -0.00 [-0.02,+0.00] p=0.248
|
| 247 |
+
K=1 st=1 diff_s1 vs nn * cov 0-0 p=1 | cert 0-10 p=0.00195 | regret +0.01 [-0.01,+0.05] p=0.085
|
| 248 |
+
K=1 st=1 diff_s1 vs idw cov 0-0 p=1 | cert 0-10 p=0.00195 | regret +0.01 [-0.01,+0.05] p=0.085
|
| 249 |
+
K=1 st=1 diff_s1 vs nns cov 0-0 p=1 | cert 2-6 p=0.289 | regret +0.00 [-0.00,+0.00] p=0.527
|
| 250 |
+
K=1 st=1 diff_s1 vs dls cov 0-0 p=1 | cert 2-5 p=0.453 | regret -0.00 [-0.02,-0.00] p=0.155
|
| 251 |
+
K=1 st=1 diff_s2 vs nn * cov 0-0 p=1 | cert 0-11 p=0.000977 | regret +0.01 [-0.00,+0.08] p=0.0113
|
| 252 |
+
K=1 st=1 diff_s2 vs idw cov 0-0 p=1 | cert 0-11 p=0.000977 | regret +0.01 [-0.00,+0.08] p=0.0113
|
| 253 |
+
K=1 st=1 diff_s2 vs nns cov 0-0 p=1 | cert 0-5 p=0.0625 | regret +0.00 [-0.00,+0.02] p=0.0836
|
| 254 |
+
K=1 st=1 diff_s2 vs dls cov 0-0 p=1 | cert 1-5 p=0.219 | regret +0.00 [-0.00,+0.02] p=0.489
|
| 255 |
+
K=4 st=0 diff_s0 vs nn cov 0-0 p=1 | cert 0-0 p=1 | regret -0.24 [-0.81,+0.24] p=0.466
|
| 256 |
+
K=4 st=0 diff_s0 vs idw cov 0-0 p=1 | cert 0-0 p=1 | regret -0.27 [-0.98,+0.13] p=0.23
|
| 257 |
+
K=4 st=0 diff_s0 vs nns * cov 0-0 p=1 | cert 0-0 p=1 | regret -0.09 [-0.16,+0.06] p=0.269
|
| 258 |
+
K=4 st=0 diff_s0 vs dls cov 0-0 p=1 | cert 0-0 p=1 | regret -0.18 [-0.27,-0.03] p=0.0469
|
| 259 |
+
K=4 st=0 diff_s1 vs nn cov 0-0 p=1 | cert 0-0 p=1 | regret -0.27 [-0.71,+0.35] p=0.729
|
| 260 |
+
K=4 st=0 diff_s1 vs idw cov 0-0 p=1 | cert 0-0 p=1 | regret -0.36 [-1.14,+0.38] p=0.354
|
| 261 |
+
K=4 st=0 diff_s1 vs nns * cov 0-0 p=1 | cert 0-0 p=1 | regret -0.09 [-0.19,-0.05] p=0.0346
|
| 262 |
+
K=4 st=0 diff_s1 vs dls cov 0-0 p=1 | cert 0-0 p=1 | regret -0.18 [-0.27,-0.05] p=0.0174
|
| 263 |
+
K=4 st=0 diff_s2 vs nn cov 0-0 p=1 | cert 0-0 p=1 | regret -0.30 [-0.68,+0.12] p=0.556
|
| 264 |
+
K=4 st=0 diff_s2 vs idw cov 0-0 p=1 | cert 0-0 p=1 | regret -0.37 [-0.96,+0.07] p=0.282
|
| 265 |
+
K=4 st=0 diff_s2 vs nns * cov 0-0 p=1 | cert 0-0 p=1 | regret -0.13 [-0.19,-0.02] p=0.0234
|
| 266 |
+
K=4 st=0 diff_s2 vs dls cov 0-0 p=1 | cert 0-0 p=1 | regret -0.18 [-0.26,-0.09] p=0.0252
|
| 267 |
+
K=4 st=1 diff_s0 vs nn * cov 0-0 p=1 | cert 0-10 p=0.00195 | regret +0.00 [+0.00,+0.07] p=0.000446
|
| 268 |
+
K=4 st=1 diff_s0 vs idw cov 0-0 p=1 | cert 0-10 p=0.00195 | regret +0.00 [-0.00,+0.07] p=0.00355
|
| 269 |
+
K=4 st=1 diff_s0 vs nns cov 0-0 p=1 | cert 1-4 p=0.375 | regret -0.00 [-0.01,+0.00] p=0.124
|
| 270 |
+
K=4 st=1 diff_s0 vs dls cov 0-0 p=1 | cert 1-3 p=0.625 | regret -0.00 [-0.02,-0.00] p=0.0566
|
| 271 |
+
K=4 st=1 diff_s1 vs nn * cov 0-0 p=1 | cert 0-5 p=0.0625 | regret +0.01 [+0.00,+0.03] p=0.00148
|
| 272 |
+
K=4 st=1 diff_s1 vs idw cov 0-0 p=1 | cert 0-5 p=0.0625 | regret +0.00 [-0.00,+0.03] p=0.0077
|
| 273 |
+
K=4 st=1 diff_s1 vs nns cov 0-0 p=1 | cert 3-1 p=0.625 | regret -0.00 [-0.02,-0.00] p=0.0174
|
| 274 |
+
K=4 st=1 diff_s1 vs dls cov 0-0 p=1 | cert 4-1 p=0.375 | regret -0.00 [-0.02,-0.00] p=0.052
|
| 275 |
+
K=4 st=1 diff_s2 vs nn * cov 0-0 p=1 | cert 0-7 p=0.0156 | regret +0.01 [+0.00,+0.05] p=0.000571
|
| 276 |
+
K=4 st=1 diff_s2 vs idw cov 0-0 p=1 | cert 0-7 p=0.0156 | regret +0.01 [+0.00,+0.05] p=0.00152
|
| 277 |
+
K=4 st=1 diff_s2 vs nns cov 0-0 p=1 | cert 2-2 p=1 | regret -0.00 [-0.01,-0.00] p=0.0309
|
| 278 |
+
K=4 st=1 diff_s2 vs dls cov 0-0 p=1 | cert 3-2 p=1 | regret -0.01 [-0.02,-0.00] p=0.0511
|
| 279 |
+
K=8 st=0 diff_s0 vs nn cov 0-0 p=1 | cert 0-0 p=1 | regret -0.28 [-0.66,+0.24] p=0.763
|
| 280 |
+
K=8 st=0 diff_s0 vs idw cov 0-0 p=1 | cert 0-0 p=1 | regret -0.19 [-0.64,+0.27] p=0.752
|
| 281 |
+
K=8 st=0 diff_s0 vs nns * cov 0-0 p=1 | cert 0-0 p=1 | regret -0.17 [-0.24,-0.10] p=0.0026
|
| 282 |
+
K=8 st=0 diff_s0 vs dls cov 0-0 p=1 | cert 0-0 p=1 | regret -0.24 [-0.30,-0.13] p=0.0104
|
| 283 |
+
K=8 st=0 diff_s1 vs nn cov 0-0 p=1 | cert 0-0 p=1 | regret -0.21 [-0.53,+0.22] p=0.848
|
| 284 |
+
K=8 st=0 diff_s1 vs idw cov 0-0 p=1 | cert 0-0 p=1 | regret -0.14 [-0.56,+0.22] p=0.854
|
| 285 |
+
K=8 st=0 diff_s1 vs nns * cov 0-0 p=1 | cert 0-0 p=1 | regret -0.17 [-0.21,-0.07] p=0.00117
|
| 286 |
+
K=8 st=0 diff_s1 vs dls cov 0-0 p=1 | cert 0-0 p=1 | regret -0.22 [-0.30,-0.15] p=0.0262
|
| 287 |
+
K=8 st=0 diff_s2 vs nn cov 0-0 p=1 | cert 0-0 p=1 | regret -0.21 [-0.54,+0.16] p=0.814
|
| 288 |
+
K=8 st=0 diff_s2 vs idw cov 0-0 p=1 | cert 0-0 p=1 | regret -0.19 [-0.46,+0.16] p=0.735
|
| 289 |
+
K=8 st=0 diff_s2 vs nns * cov 0-0 p=1 | cert 0-0 p=1 | regret -0.12 [-0.19,-0.03] p=0.00111
|
| 290 |
+
K=8 st=0 diff_s2 vs dls cov 0-0 p=1 | cert 0-0 p=1 | regret -0.20 [-0.26,-0.09] p=0.0657
|
| 291 |
+
K=8 st=1 diff_s0 vs nn * cov 0-0 p=1 | cert 0-7 p=0.0156 | regret +0.01 [+0.00,+0.06] p=0.000159
|
| 292 |
+
K=8 st=1 diff_s0 vs idw cov 0-0 p=1 | cert 0-7 p=0.0156 | regret +0.01 [-0.00,+0.06] p=0.000356
|
| 293 |
+
K=8 st=1 diff_s0 vs nns cov 0-0 p=1 | cert 1-1 p=1 | regret -0.00 [-0.01,-0.00] p=0.0059
|
| 294 |
+
K=8 st=1 diff_s0 vs dls cov 0-0 p=1 | cert 2-2 p=1 | regret -0.00 [-0.01,-0.00] p=0.245
|
| 295 |
+
K=8 st=1 diff_s1 vs nn * cov 0-0 p=1 | cert 0-3 p=0.25 | regret +0.00 [+0.00,+0.03] p=0.000516
|
| 296 |
+
K=8 st=1 diff_s1 vs idw cov 0-0 p=1 | cert 0-3 p=0.25 | regret +0.00 [+0.00,+0.03] p=0.0014
|
| 297 |
+
K=8 st=1 diff_s1 vs nns cov 0-0 p=1 | cert 5-1 p=0.219 | regret -0.00 [-0.04,-0.00] p=0.00515
|
| 298 |
+
K=8 st=1 diff_s1 vs dls cov 0-0 p=1 | cert 4-0 p=0.125 | regret -0.00 [-0.02,-0.00] p=0.143
|
| 299 |
+
K=8 st=1 diff_s2 vs nn * cov 0-0 p=1 | cert 0-2 p=0.5 | regret +0.01 [+0.00,+0.03] p=7.48e-05
|
| 300 |
+
K=8 st=1 diff_s2 vs idw cov 0-0 p=1 | cert 0-2 p=0.5 | regret +0.01 [+0.00,+0.03] p=0.000118
|
| 301 |
+
K=8 st=1 diff_s2 vs nns cov 0-0 p=1 | cert 5-0 p=0.0625 | regret -0.01 [-0.03,-0.00] p=0.000708
|
| 302 |
+
K=8 st=1 diff_s2 vs dls cov 0-0 p=1 | cert 5-0 p=0.0625 | regret -0.00 [-0.01,-0.00] p=0.19
|
| 303 |
+
|
| 304 |
+
=== lvl1.5: N=60 NN-dist ratio test/bank 7.5x (mean NN 1.075); oracle finite 60 ===
|
| 305 |
+
-- no steering: cov / certified-stable cov / median regret % (inf = unstable counts as failure)
|
| 306 |
+
arm K=1: cov cert regret | K=4: cov cert regret | K=8: cov cert regret
|
| 307 |
+
diff_s0 | 1.000 0.967 1.70 | 1.000 0.983 1.07 | 1.000 0.983 0.83
|
| 308 |
+
diff_s1 | 1.000 0.967 1.58 | 1.000 0.983 1.02 | 1.000 0.983 0.92
|
| 309 |
+
diff_s2 | 1.000 0.983 1.48 | 1.000 0.983 0.98 | 1.000 0.983 0.86
|
| 310 |
+
x_s42 | 1.000 0.950 1.70 | 1.000 0.983 1.04 | 1.000 0.983 0.97
|
| 311 |
+
x_s43 | 1.000 0.900 2.13 | 1.000 0.967 1.02 | 1.000 0.983 0.96
|
| 312 |
+
x_s44 | 1.000 0.967 2.04 | 1.000 0.983 1.05 | 1.000 0.983 0.99
|
| 313 |
+
nn | 1.000 1.000 3.77 | 1.000 1.000 1.53 | 1.000 1.000 1.08
|
| 314 |
+
idw | 1.000 1.000 3.77 | 1.000 1.000 2.38 | 1.000 1.000 1.16
|
| 315 |
+
nns | 1.000 1.000 1.19 | 1.000 1.000 1.07 | 1.000 1.000 0.98
|
| 316 |
+
dls | 1.000 1.000 1.37 | 1.000 1.000 1.11 | 1.000 1.000 1.05
|
| 317 |
+
|
| 318 |
+
-- 50-step steering on every candidate: cov / certified-stable cov / median regret % (inf = unstable counts as failure)
|
| 319 |
+
arm K=1: cov cert regret | K=4: cov cert regret | K=8: cov cert regret
|
| 320 |
+
diff_s0 | 1.000 0.817 0.05 | 1.000 0.900 0.01 | 1.000 0.917 0.00
|
| 321 |
+
diff_s1 | 1.000 0.783 0.03 | 1.000 0.850 0.00 | 1.000 0.917 0.00
|
| 322 |
+
diff_s2 | 1.000 0.850 0.02 | 1.000 0.883 0.00 | 1.000 0.933 -0.00
|
| 323 |
+
x_s42 | 1.000 0.783 0.05 | 1.000 0.883 0.00 | 1.000 0.900 0.00
|
| 324 |
+
x_s43 | 1.000 0.767 0.07 | 1.000 0.883 0.01 | 1.000 0.917 0.00
|
| 325 |
+
x_s44 | 1.000 0.800 0.09 | 1.000 0.917 0.01 | 1.000 0.917 0.00
|
| 326 |
+
nn | 1.000 0.950 0.02 | 1.000 1.000 -0.01 | 1.000 1.000 -0.02
|
| 327 |
+
idw | 1.000 0.950 0.02 | 1.000 1.000 -0.01 | 1.000 1.000 -0.01
|
| 328 |
+
nns | 1.000 0.833 0.04 | 1.000 0.850 0.03 | 1.000 0.850 0.03
|
| 329 |
+
dls | 1.000 0.833 0.05 | 1.000 0.833 0.02 | 1.000 0.833 0.01
|
| 330 |
+
|
| 331 |
+
-- paired tests, diffusion (model - retrieval): [cov McNemar model-only/retr-only p | certified-cov | paired regret diff median [95% CI], Wilcoxon p]; * = best retrieval arm by median regret
|
| 332 |
+
K=1 st=0 diff_s0 vs nn cov 0-0 p=1 | cert 0-2 p=0.5 | regret -1.50 [-3.13,+0.45] p=0.617
|
| 333 |
+
K=1 st=0 diff_s0 vs idw cov 0-0 p=1 | cert 0-2 p=0.5 | regret -1.50 [-3.13,+0.45] p=0.617
|
| 334 |
+
K=1 st=0 diff_s0 vs nns * cov 0-0 p=1 | cert 0-2 p=0.5 | regret +0.57 [+0.26,+1.44] p=3.65e-06
|
| 335 |
+
K=1 st=0 diff_s0 vs dls cov 0-0 p=1 | cert 0-2 p=0.5 | regret +0.45 [+0.09,+0.93] p=0.000602
|
| 336 |
+
K=1 st=0 diff_s1 vs nn cov 0-0 p=1 | cert 0-2 p=0.5 | regret -2.30 [-3.30,-0.05] p=0.317
|
| 337 |
+
K=1 st=0 diff_s1 vs idw cov 0-0 p=1 | cert 0-2 p=0.5 | regret -2.30 [-3.30,-0.05] p=0.317
|
| 338 |
+
K=1 st=0 diff_s1 vs nns * cov 0-0 p=1 | cert 0-2 p=0.5 | regret +0.50 [+0.19,+1.18] p=0.000195
|
| 339 |
+
K=1 st=0 diff_s1 vs dls cov 0-0 p=1 | cert 0-2 p=0.5 | regret +0.22 [-0.06,+0.51] p=0.0668
|
| 340 |
+
K=1 st=0 diff_s2 vs nn cov 0-0 p=1 | cert 0-1 p=1 | regret -1.89 [-3.17,-0.26] p=0.248
|
| 341 |
+
K=1 st=0 diff_s2 vs idw cov 0-0 p=1 | cert 0-1 p=1 | regret -1.89 [-3.17,-0.26] p=0.248
|
| 342 |
+
K=1 st=0 diff_s2 vs nns * cov 0-0 p=1 | cert 0-1 p=1 | regret +0.14 [-0.01,+0.97] p=0.00515
|
| 343 |
+
K=1 st=0 diff_s2 vs dls cov 0-0 p=1 | cert 0-1 p=1 | regret +0.05 [-0.13,+0.46] p=0.0962
|
| 344 |
+
K=1 st=1 diff_s0 vs nn * cov 0-0 p=1 | cert 0-8 p=0.00781 | regret +0.01 [-0.01,+0.12] p=0.129
|
| 345 |
+
K=1 st=1 diff_s0 vs idw cov 0-0 p=1 | cert 0-8 p=0.00781 | regret +0.01 [-0.01,+0.12] p=0.129
|
| 346 |
+
K=1 st=1 diff_s0 vs nns cov 0-0 p=1 | cert 1-2 p=1 | regret +0.00 [-0.00,+0.08] p=0.0109
|
| 347 |
+
K=1 st=1 diff_s0 vs dls cov 0-0 p=1 | cert 1-2 p=1 | regret +0.01 [-0.00,+0.11] p=0.0142
|
| 348 |
+
K=1 st=1 diff_s1 vs nn * cov 0-0 p=1 | cert 0-10 p=0.00195 | regret +0.01 [-0.03,+0.07] p=0.317
|
| 349 |
+
K=1 st=1 diff_s1 vs idw cov 0-0 p=1 | cert 0-10 p=0.00195 | regret +0.01 [-0.03,+0.07] p=0.317
|
| 350 |
+
K=1 st=1 diff_s1 vs nns cov 0-0 p=1 | cert 0-3 p=0.25 | regret +0.00 [-0.01,+0.04] p=0.171
|
| 351 |
+
K=1 st=1 diff_s1 vs dls cov 0-0 p=1 | cert 0-3 p=0.25 | regret -0.00 [-0.02,+0.01] p=0.988
|
| 352 |
+
K=1 st=1 diff_s2 vs nn * cov 0-0 p=1 | cert 1-7 p=0.0703 | regret +0.01 [-0.03,+0.03] p=0.414
|
| 353 |
+
K=1 st=1 diff_s2 vs idw cov 0-0 p=1 | cert 1-7 p=0.0703 | regret +0.01 [-0.03,+0.03] p=0.414
|
| 354 |
+
K=1 st=1 diff_s2 vs nns cov 0-0 p=1 | cert 2-1 p=1 | regret -0.00 [-0.01,+0.01] p=0.44
|
| 355 |
+
K=1 st=1 diff_s2 vs dls cov 0-0 p=1 | cert 2-1 p=1 | regret -0.00 [-0.04,+0.00] p=0.935
|
| 356 |
+
K=4 st=0 diff_s0 vs nn cov 0-0 p=1 | cert 0-1 p=1 | regret -0.19 [-0.91,+0.36] p=0.977
|
| 357 |
+
K=4 st=0 diff_s0 vs idw cov 0-0 p=1 | cert 0-1 p=1 | regret -0.13 [-1.40,+0.18] p=0.752
|
| 358 |
+
K=4 st=0 diff_s0 vs nns * cov 0-0 p=1 | cert 0-1 p=1 | regret +0.14 [-0.02,+0.52] p=0.0151
|
| 359 |
+
K=4 st=0 diff_s0 vs dls cov 0-0 p=1 | cert 0-1 p=1 | regret +0.02 [-0.27,+0.24] p=0.317
|
| 360 |
+
K=4 st=0 diff_s1 vs nn cov 0-0 p=1 | cert 0-1 p=1 | regret -0.33 [-1.03,+0.18] p=0.527
|
| 361 |
+
K=4 st=0 diff_s1 vs idw cov 0-0 p=1 | cert 0-1 p=1 | regret -0.19 [-1.49,+0.08] p=0.414
|
| 362 |
+
K=4 st=0 diff_s1 vs nns * cov 0-0 p=1 | cert 0-1 p=1 | regret +0.16 [-0.08,+0.42] p=0.0452
|
| 363 |
+
K=4 st=0 diff_s1 vs dls cov 0-0 p=1 | cert 0-1 p=1 | regret +0.06 [-0.28,+0.20] p=0.686
|
| 364 |
+
K=4 st=0 diff_s2 vs nn cov 0-0 p=1 | cert 0-1 p=1 | regret -0.29 [-0.91,+0.19] p=0.576
|
| 365 |
+
K=4 st=0 diff_s2 vs idw cov 0-0 p=1 | cert 0-1 p=1 | regret -0.30 [-1.47,+0.19] p=0.457
|
| 366 |
+
K=4 st=0 diff_s2 vs nns * cov 0-0 p=1 | cert 0-1 p=1 | regret +0.03 [-0.19,+0.16] p=0.44
|
| 367 |
+
K=4 st=0 diff_s2 vs dls cov 0-0 p=1 | cert 0-1 p=1 | regret -0.13 [-0.36,+0.04] p=0.263
|
| 368 |
+
K=4 st=1 diff_s0 vs nn * cov 0-0 p=1 | cert 0-6 p=0.0312 | regret +0.01 [-0.00,+0.09] p=0.0106
|
| 369 |
+
K=4 st=1 diff_s0 vs idw cov 0-0 p=1 | cert 0-6 p=0.0312 | regret +0.01 [-0.00,+0.10] p=0.0196
|
| 370 |
+
K=4 st=1 diff_s0 vs nns cov 0-0 p=1 | cert 4-1 p=0.375 | regret +0.00 [-0.01,+0.01] p=0.431
|
| 371 |
+
K=4 st=1 diff_s0 vs dls cov 0-0 p=1 | cert 5-1 p=0.219 | regret -0.00 [-0.03,+0.00] p=0.959
|
| 372 |
+
K=4 st=1 diff_s1 vs nn * cov 0-0 p=1 | cert 0-9 p=0.00391 | regret +0.01 [-0.00,+0.05] p=0.0353
|
| 373 |
+
K=4 st=1 diff_s1 vs idw cov 0-0 p=1 | cert 0-9 p=0.00391 | regret +0.01 [-0.00,+0.05] p=0.0529
|
| 374 |
+
K=4 st=1 diff_s1 vs nns cov 0-0 p=1 | cert 2-2 p=1 | regret +0.00 [-0.02,+0.01] p=0.527
|
| 375 |
+
K=4 st=1 diff_s1 vs dls cov 0-0 p=1 | cert 2-1 p=1 | regret -0.01 [-0.04,+0.00] p=0.982
|
| 376 |
+
K=4 st=1 diff_s2 vs nn * cov 0-0 p=1 | cert 0-7 p=0.0156 | regret +0.01 [+0.00,+0.06] p=0.0077
|
| 377 |
+
K=4 st=1 diff_s2 vs idw cov 0-0 p=1 | cert 0-7 p=0.0156 | regret +0.01 [+0.00,+0.06] p=0.0167
|
| 378 |
+
K=4 st=1 diff_s2 vs nns cov 0-0 p=1 | cert 3-1 p=0.625 | regret -0.00 [-0.02,+0.00] p=0.69
|
| 379 |
+
K=4 st=1 diff_s2 vs dls cov 0-0 p=1 | cert 4-1 p=0.375 | regret -0.01 [-0.05,-0.00] p=0.178
|
| 380 |
+
K=8 st=0 diff_s0 vs nn cov 0-0 p=1 | cert 0-1 p=1 | regret -0.09 [-0.45,+0.15] p=0.941
|
| 381 |
+
K=8 st=0 diff_s0 vs idw cov 0-0 p=1 | cert 0-1 p=1 | regret -0.09 [-0.51,+0.15] p=0.837
|
| 382 |
+
K=8 st=0 diff_s0 vs nns * cov 0-0 p=1 | cert 0-1 p=1 | regret -0.03 [-0.18,+0.10] p=0.475
|
| 383 |
+
K=8 st=0 diff_s0 vs dls cov 0-0 p=1 | cert 0-1 p=1 | regret -0.15 [-0.35,+0.11] p=0.686
|
| 384 |
+
K=8 st=0 diff_s1 vs nn cov 0-0 p=1 | cert 0-1 p=1 | regret -0.10 [-0.42,+0.30] p=0.906
|
| 385 |
+
K=8 st=0 diff_s1 vs idw cov 0-0 p=1 | cert 0-1 p=1 | regret -0.10 [-0.52,+0.30] p=0.774
|
| 386 |
+
K=8 st=0 diff_s1 vs nns * cov 0-0 p=1 | cert 0-1 p=1 | regret +0.08 [-0.11,+0.37] p=0.076
|
| 387 |
+
K=8 st=0 diff_s1 vs dls cov 0-0 p=1 | cert 0-1 p=1 | regret +0.00 [-0.27,+0.23] p=0.422
|
| 388 |
+
K=8 st=0 diff_s2 vs nn cov 0-0 p=1 | cert 0-1 p=1 | regret -0.05 [-0.28,+0.21] p=0.877
|
| 389 |
+
K=8 st=0 diff_s2 vs idw cov 0-0 p=1 | cert 0-1 p=1 | regret -0.05 [-0.50,+0.21] p=0.718
|
| 390 |
+
K=8 st=0 diff_s2 vs nns * cov 0-0 p=1 | cert 0-1 p=1 | regret -0.03 [-0.14,+0.09] p=0.691
|
| 391 |
+
K=8 st=0 diff_s2 vs dls cov 0-0 p=1 | cert 0-1 p=1 | regret -0.17 [-0.31,+0.03] p=0.643
|
| 392 |
+
K=8 st=1 diff_s0 vs nn * cov 0-0 p=1 | cert 0-5 p=0.0625 | regret +0.01 [-0.00,+0.05] p=0.00936
|
| 393 |
+
K=8 st=1 diff_s0 vs idw cov 0-0 p=1 | cert 0-5 p=0.0625 | regret +0.01 [-0.00,+0.05] p=0.0128
|
| 394 |
+
K=8 st=1 diff_s0 vs nns cov 0-0 p=1 | cert 5-1 p=0.219 | regret -0.00 [-0.02,+0.00] p=0.982
|
| 395 |
+
K=8 st=1 diff_s0 vs dls cov 0-0 p=1 | cert 6-1 p=0.125 | regret -0.00 [-0.02,+0.00] p=0.752
|
| 396 |
+
K=8 st=1 diff_s1 vs nn * cov 0-0 p=1 | cert 0-5 p=0.0625 | regret +0.00 [-0.00,+0.02] p=0.0293
|
| 397 |
+
K=8 st=1 diff_s1 vs idw cov 0-0 p=1 | cert 0-5 p=0.0625 | regret +0.00 [-0.00,+0.02] p=0.052
|
| 398 |
+
K=8 st=1 diff_s1 vs nns cov 0-0 p=1 | cert 5-1 p=0.219 | regret -0.00 [-0.02,+0.00] p=0.68
|
| 399 |
+
K=8 st=1 diff_s1 vs dls cov 0-0 p=1 | cert 6-1 p=0.125 | regret -0.01 [-0.04,+0.00] p=0.994
|
| 400 |
+
K=8 st=1 diff_s2 vs nn * cov 0-0 p=1 | cert 0-4 p=0.125 | regret +0.00 [-0.00,+0.06] p=0.0154
|
| 401 |
+
K=8 st=1 diff_s2 vs idw cov 0-0 p=1 | cert 0-4 p=0.125 | regret +0.00 [-0.00,+0.06] p=0.0221
|
| 402 |
+
K=8 st=1 diff_s2 vs nns cov 0-0 p=1 | cert 6-1 p=0.125 | regret -0.00 [-0.02,+0.00] p=0.774
|
| 403 |
+
K=8 st=1 diff_s2 vs dls cov 0-0 p=1 | cert 7-1 p=0.0703 | regret -0.02 [-0.04,-0.00] p=0.289
|
| 404 |
+
|
| 405 |
+
=== lvl2.0: N=60 NN-dist ratio test/bank 23.5x (mean NN 3.376); oracle finite 60 ===
|
| 406 |
+
-- no steering: cov / certified-stable cov / median regret % (inf = unstable counts as failure)
|
| 407 |
+
arm K=1: cov cert regret | K=4: cov cert regret | K=8: cov cert regret
|
| 408 |
+
diff_s0 | 1.000 0.867 3.21 | 1.000 0.933 0.88 | 1.000 0.933 0.73
|
| 409 |
+
diff_s1 | 1.000 0.817 1.92 | 1.000 0.950 0.73 | 1.000 0.950 0.66
|
| 410 |
+
diff_s2 | 1.000 0.800 2.10 | 1.000 0.917 0.81 | 1.000 0.950 0.65
|
| 411 |
+
x_s42 | 0.967 0.767 4.09 | 1.000 0.900 0.85 | 1.000 0.933 0.62
|
| 412 |
+
x_s43 | 0.983 0.817 4.11 | 1.000 0.867 1.49 | 1.000 0.917 0.89
|
| 413 |
+
x_s44 | 0.983 0.817 4.48 | 1.000 0.917 1.13 | 1.000 0.933 0.78
|
| 414 |
+
nn | 0.967 0.950 8.97 | 0.967 0.950 5.67 | 0.967 0.950 4.00
|
| 415 |
+
idw | 0.967 0.950 8.97 | 0.950 0.933 5.92 | 0.950 0.933 4.31
|
| 416 |
+
nns | 1.000 1.000 1.01 | 1.000 1.000 0.87 | 1.000 1.000 0.78
|
| 417 |
+
dls | 1.000 1.000 1.02 | 1.000 1.000 0.85 | 1.000 1.000 0.81
|
| 418 |
+
|
| 419 |
+
-- 50-step steering on every candidate: cov / certified-stable cov / median regret % (inf = unstable counts as failure)
|
| 420 |
+
arm K=1: cov cert regret | K=4: cov cert regret | K=8: cov cert regret
|
| 421 |
+
diff_s0 | 1.000 0.767 0.02 | 1.000 0.850 -0.00 | 1.000 0.883 -0.00
|
| 422 |
+
diff_s1 | 1.000 0.700 0.00 | 1.000 0.900 -0.00 | 1.000 0.917 -0.01
|
| 423 |
+
diff_s2 | 1.000 0.650 0.02 | 1.000 0.817 -0.00 | 1.000 0.867 -0.00
|
| 424 |
+
x_s42 | 1.000 0.700 0.03 | 1.000 0.817 -0.00 | 1.000 0.900 -0.01
|
| 425 |
+
x_s43 | 1.000 0.783 0.31 | 1.000 0.850 0.00 | 1.000 0.917 -0.00
|
| 426 |
+
x_s44 | 1.000 0.733 0.26 | 1.000 0.817 0.00 | 1.000 0.883 -0.00
|
| 427 |
+
nn | 0.967 0.900 0.15 | 0.983 0.950 0.02 | 0.983 0.950 0.00
|
| 428 |
+
idw | 0.967 0.900 0.15 | 0.983 0.950 0.03 | 0.983 0.950 0.00
|
| 429 |
+
nns | 1.000 0.750 0.02 | 1.000 0.750 0.01 | 1.000 0.750 0.00
|
| 430 |
+
dls | 1.000 0.700 0.01 | 1.000 0.700 0.00 | 1.000 0.733 0.00
|
| 431 |
+
|
| 432 |
+
-- paired tests, diffusion (model - retrieval): [cov McNemar model-only/retr-only p | certified-cov | paired regret diff median [95% CI], Wilcoxon p]; * = best retrieval arm by median regret
|
| 433 |
+
K=1 st=0 diff_s0 vs nn cov 2-0 p=0.5 | cert 0-5 p=0.0625 | regret -2.61 [-8.11,+0.41] p=0.135
|
| 434 |
+
K=1 st=0 diff_s0 vs idw cov 2-0 p=0.5 | cert 0-5 p=0.0625 | regret -2.61 [-8.11,+0.41] p=0.135
|
| 435 |
+
K=1 st=0 diff_s0 vs nns * cov 0-0 p=1 | cert 0-8 p=0.00781 | regret +2.56 [+0.98,+3.75] p=9.44e-08
|
| 436 |
+
K=1 st=0 diff_s0 vs dls cov 0-0 p=1 | cert 0-8 p=0.00781 | regret +1.68 [+0.94,+3.37] p=6.23e-07
|
| 437 |
+
K=1 st=0 diff_s1 vs nn cov 2-0 p=0.5 | cert 0-8 p=0.00781 | regret -4.63 [-9.49,+0.51] p=0.0485
|
| 438 |
+
K=1 st=0 diff_s1 vs idw cov 2-0 p=0.5 | cert 0-8 p=0.00781 | regret -4.63 [-9.49,+0.51] p=0.0485
|
| 439 |
+
K=1 st=0 diff_s1 vs nns * cov 0-0 p=1 | cert 0-11 p=0.000977 | regret +0.87 [+0.30,+2.79] p=0.000498
|
| 440 |
+
K=1 st=0 diff_s1 vs dls cov 0-0 p=1 | cert 0-11 p=0.000977 | regret +0.90 [+0.30,+2.94] p=0.000949
|
| 441 |
+
K=1 st=0 diff_s2 vs nn cov 2-0 p=0.5 | cert 0-9 p=0.00391 | regret -3.68 [-9.38,+0.47] p=0.12
|
| 442 |
+
K=1 st=0 diff_s2 vs idw cov 2-0 p=0.5 | cert 0-9 p=0.00391 | regret -3.68 [-9.38,+0.47] p=0.12
|
| 443 |
+
K=1 st=0 diff_s2 vs nns * cov 0-0 p=1 | cert 0-12 p=0.000488 | regret +1.33 [+0.26,+2.69] p=4.68e-05
|
| 444 |
+
K=1 st=0 diff_s2 vs dls cov 0-0 p=1 | cert 0-12 p=0.000488 | regret +1.28 [-0.02,+2.69] p=0.000268
|
| 445 |
+
K=1 st=1 diff_s0 vs nn cov 2-0 p=0.5 | cert 3-11 p=0.0574 | regret -0.01 [-0.16,+0.07] p=0.757
|
| 446 |
+
K=1 st=1 diff_s0 vs idw cov 2-0 p=0.5 | cert 3-11 p=0.0574 | regret -0.01 [-0.16,+0.07] p=0.757
|
| 447 |
+
K=1 st=1 diff_s0 vs nns cov 0-0 p=1 | cert 8-7 p=1 | regret +0.01 [+0.00,+0.19] p=0.00539
|
| 448 |
+
K=1 st=1 diff_s0 vs dls * cov 0-0 p=1 | cert 11-7 p=0.481 | regret +0.01 [-0.00,+0.22] p=0.0109
|
| 449 |
+
K=1 st=1 diff_s1 vs nn cov 2-0 p=0.5 | cert 3-15 p=0.00754 | regret -0.03 [-0.54,+0.00] p=0.233
|
| 450 |
+
K=1 st=1 diff_s1 vs idw cov 2-0 p=0.5 | cert 3-15 p=0.00754 | regret -0.03 [-0.54,+0.00] p=0.233
|
| 451 |
+
K=1 st=1 diff_s1 vs nns cov 0-0 p=1 | cert 5-8 p=0.581 | regret -0.00 [-0.03,+0.01] p=0.659
|
| 452 |
+
K=1 st=1 diff_s1 vs dls * cov 0-0 p=1 | cert 8-8 p=1 | regret -0.01 [-0.03,+0.01] p=0.669
|
| 453 |
+
K=1 st=1 diff_s2 vs nn cov 2-0 p=0.5 | cert 3-18 p=0.00149 | regret -0.01 [-0.36,+0.02] p=0.527
|
| 454 |
+
K=1 st=1 diff_s2 vs idw cov 2-0 p=0.5 | cert 3-18 p=0.00149 | regret -0.01 [-0.36,+0.02] p=0.527
|
| 455 |
+
K=1 st=1 diff_s2 vs nns cov 0-0 p=1 | cert 3-9 p=0.146 | regret +0.01 [-0.00,+0.14] p=0.0204
|
| 456 |
+
K=1 st=1 diff_s2 vs dls * cov 0-0 p=1 | cert 6-9 p=0.607 | regret +0.00 [-0.01,+0.13] p=0.0605
|
| 457 |
+
K=4 st=0 diff_s0 vs nn cov 2-0 p=0.5 | cert 0-1 p=1 | regret -1.85 [-7.30,-0.06] p=0.00753
|
| 458 |
+
K=4 st=0 diff_s0 vs idw cov 3-0 p=0.25 | cert 1-1 p=1 | regret -3.53 [-8.30,+0.01] p=0.00409
|
| 459 |
+
K=4 st=0 diff_s0 vs nns cov 0-0 p=1 | cert 0-4 p=0.125 | regret +0.06 [-0.13,+0.46] p=0.155
|
| 460 |
+
K=4 st=0 diff_s0 vs dls * cov 0-0 p=1 | cert 0-4 p=0.125 | regret -0.09 [-0.31,+0.46] p=0.484
|
| 461 |
+
K=4 st=0 diff_s1 vs nn cov 2-0 p=0.5 | cert 0-0 p=1 | regret -3.33 [-7.54,-0.35] p=0.0039
|
| 462 |
+
K=4 st=0 diff_s1 vs idw cov 3-0 p=0.25 | cert 1-0 p=1 | regret -3.49 [-8.54,-0.35] p=0.00194
|
| 463 |
+
K=4 st=0 diff_s1 vs nns cov 0-0 p=1 | cert 0-3 p=0.25 | regret -0.00 [-0.18,+0.22] p=0.536
|
| 464 |
+
K=4 st=0 diff_s1 vs dls * cov 0-0 p=1 | cert 0-3 p=0.25 | regret +0.06 [-0.41,+0.23] p=0.669
|
| 465 |
+
K=4 st=0 diff_s2 vs nn cov 2-0 p=0.5 | cert 0-2 p=0.5 | regret -2.14 [-7.53,-0.21] p=0.00355
|
| 466 |
+
K=4 st=0 diff_s2 vs idw cov 3-0 p=0.25 | cert 1-2 p=1 | regret -3.55 [-8.53,-0.37] p=0.00117
|
| 467 |
+
K=4 st=0 diff_s2 vs nns cov 0-0 p=1 | cert 0-5 p=0.0625 | regret -0.11 [-0.22,+0.45] p=0.303
|
| 468 |
+
K=4 st=0 diff_s2 vs dls * cov 0-0 p=1 | cert 0-5 p=0.0625 | regret -0.17 [-0.38,+0.37] p=0.627
|
| 469 |
+
K=4 st=1 diff_s0 vs nn cov 1-0 p=1 | cert 0-6 p=0.0312 | regret -0.02 [-0.22,+0.00] p=0.0798
|
| 470 |
+
K=4 st=1 diff_s0 vs idw cov 1-0 p=1 | cert 0-6 p=0.0312 | regret -0.02 [-0.35,+0.00] p=0.0657
|
| 471 |
+
K=4 st=1 diff_s0 vs nns cov 0-0 p=1 | cert 10-4 p=0.18 | regret -0.00 [-0.02,+0.00] p=0.517
|
| 472 |
+
K=4 st=1 diff_s0 vs dls * cov 0-0 p=1 | cert 13-4 p=0.049 | regret -0.01 [-0.03,+0.00] p=0.32
|
| 473 |
+
K=4 st=1 diff_s1 vs nn cov 1-0 p=1 | cert 0-3 p=0.25 | regret -0.04 [-0.29,-0.00] p=0.046
|
| 474 |
+
K=4 st=1 diff_s1 vs idw cov 1-0 p=1 | cert 0-3 p=0.25 | regret -0.01 [-0.49,+0.00] p=0.0836
|
| 475 |
+
K=4 st=1 diff_s1 vs nns cov 0-0 p=1 | cert 13-4 p=0.049 | regret -0.02 [-0.04,-0.00] p=0.0529
|
| 476 |
+
K=4 st=1 diff_s1 vs dls * cov 0-0 p=1 | cert 16-4 p=0.0118 | regret -0.01 [-0.06,-0.00] p=0.0386
|
| 477 |
+
K=4 st=1 diff_s2 vs nn cov 1-0 p=1 | cert 0-8 p=0.00781 | regret -0.05 [-0.25,+0.00] p=0.0556
|
| 478 |
+
K=4 st=1 diff_s2 vs idw cov 1-0 p=1 | cert 0-8 p=0.00781 | regret -0.04 [-0.38,+0.00] p=0.0372
|
| 479 |
+
K=4 st=1 diff_s2 vs nns cov 0-0 p=1 | cert 8-4 p=0.388 | regret -0.00 [-0.03,+0.00] p=0.41
|
| 480 |
+
K=4 st=1 diff_s2 vs dls * cov 0-0 p=1 | cert 11-4 p=0.118 | regret -0.01 [-0.03,+0.00] p=0.263
|
| 481 |
+
K=8 st=0 diff_s0 vs nn cov 2-0 p=0.5 | cert 0-1 p=1 | regret -1.33 [-4.89,-0.04] p=0.00449
|
| 482 |
+
K=8 st=0 diff_s0 vs idw cov 3-0 p=0.25 | cert 1-1 p=1 | regret -1.33 [-5.16,-0.04] p=0.00316
|
| 483 |
+
K=8 st=0 diff_s0 vs nns * cov 0-0 p=1 | cert 0-4 p=0.125 | regret -0.08 [-0.20,+0.29] p=0.606
|
| 484 |
+
K=8 st=0 diff_s0 vs dls cov 0-0 p=1 | cert 0-4 p=0.125 | regret -0.11 [-0.41,+0.08] p=0.912
|
| 485 |
+
K=8 st=0 diff_s1 vs nn cov 2-0 p=0.5 | cert 0-0 p=1 | regret -1.67 [-4.70,-0.50] p=0.00287
|
| 486 |
+
K=8 st=0 diff_s1 vs idw cov 3-0 p=0.25 | cert 1-0 p=1 | regret -1.67 [-4.94,-0.41] p=0.00171
|
| 487 |
+
K=8 st=0 diff_s1 vs nns * cov 0-0 p=1 | cert 0-3 p=0.25 | regret -0.10 [-0.31,+0.13] p=0.819
|
| 488 |
+
K=8 st=0 diff_s1 vs dls cov 0-0 p=1 | cert 0-3 p=0.25 | regret -0.26 [-0.59,+0.15] p=0.696
|
| 489 |
+
K=8 st=0 diff_s2 vs nn cov 2-0 p=0.5 | cert 0-0 p=1 | regret -1.45 [-5.03,-0.05] p=0.00787
|
| 490 |
+
K=8 st=0 diff_s2 vs idw cov 3-0 p=0.25 | cert 1-0 p=1 | regret -1.45 [-5.60,-0.05] p=0.0028
|
| 491 |
+
K=8 st=0 diff_s2 vs nns * cov 0-0 p=1 | cert 0-3 p=0.25 | regret -0.12 [-0.30,+0.07] p=0.947
|
| 492 |
+
K=8 st=0 diff_s2 vs dls cov 0-0 p=1 | cert 0-3 p=0.25 | regret -0.21 [-0.40,+0.03] p=0.74
|
| 493 |
+
K=8 st=1 diff_s0 vs nn cov 1-0 p=1 | cert 0-4 p=0.125 | regret -0.00 [-0.05,+0.00] p=0.0983
|
| 494 |
+
K=8 st=1 diff_s0 vs idw * cov 1-0 p=1 | cert 0-4 p=0.125 | regret -0.00 [-0.08,+0.00] p=0.122
|
| 495 |
+
K=8 st=1 diff_s0 vs nns cov 0-0 p=1 | cert 12-4 p=0.0768 | regret -0.00 [-0.04,-0.00] p=0.0221
|
| 496 |
+
K=8 st=1 diff_s0 vs dls cov 0-0 p=1 | cert 13-4 p=0.049 | regret -0.01 [-0.08,-0.00] p=0.04
|
| 497 |
+
K=8 st=1 diff_s1 vs nn cov 1-0 p=1 | cert 0-2 p=0.5 | regret -0.02 [-0.16,+0.00] p=0.052
|
| 498 |
+
K=8 st=1 diff_s1 vs idw * cov 1-0 p=1 | cert 0-2 p=0.5 | regret -0.01 [-0.14,+0.00] p=0.0494
|
| 499 |
+
K=8 st=1 diff_s1 vs nns cov 0-0 p=1 | cert 14-4 p=0.0309 | regret -0.02 [-0.11,-0.00] p=0.0154
|
| 500 |
+
K=8 st=1 diff_s1 vs dls cov 0-0 p=1 | cert 15-4 p=0.0192 | regret -0.02 [-0.08,-0.01] p=0.0106
|
| 501 |
+
K=8 st=1 diff_s2 vs nn cov 1-0 p=1 | cert 0-5 p=0.0625 | regret -0.01 [-0.10,+0.00] p=0.12
|
| 502 |
+
K=8 st=1 diff_s2 vs idw * cov 1-0 p=1 | cert 0-5 p=0.0625 | regret -0.01 [-0.12,+0.00] p=0.085
|
| 503 |
+
K=8 st=1 diff_s2 vs nns cov 0-0 p=1 | cert 10-3 p=0.0923 | regret -0.01 [-0.03,+0.00] p=0.0773
|
| 504 |
+
K=8 st=1 diff_s2 vs dls cov 0-0 p=1 | cert 11-3 p=0.0574 | regret -0.01 [-0.03,-0.00] p=0.0918
|
| 505 |
+
|
| 506 |
+
=== hole_in: N=60 NN-dist ratio test/bank 2.5x (mean NN 0.368); oracle finite 60 ===
|
| 507 |
+
-- no steering: cov / certified-stable cov / median regret % (inf = unstable counts as failure)
|
| 508 |
+
arm K=1: cov cert regret | K=4: cov cert regret | K=8: cov cert regret
|
| 509 |
+
diff_s0 | 1.000 1.000 1.62 | 1.000 1.000 0.91 | 1.000 1.000 0.68
|
| 510 |
+
diff_s1 | 1.000 1.000 1.49 | 1.000 1.000 0.82 | 1.000 1.000 0.68
|
| 511 |
+
diff_s2 | 1.000 1.000 1.58 | 1.000 1.000 0.95 | 1.000 1.000 0.74
|
| 512 |
+
x_s43 | 1.000 1.000 1.55 | 1.000 1.000 0.61 | 1.000 1.000 0.42
|
| 513 |
+
x_s44 | 1.000 1.000 1.30 | 1.000 1.000 0.64 | 1.000 1.000 0.41
|
| 514 |
+
nn | 1.000 1.000 1.95 | 1.000 1.000 0.56 | 1.000 1.000 0.45
|
| 515 |
+
idw | 1.000 1.000 1.95 | 1.000 1.000 0.64 | 1.000 1.000 0.41
|
| 516 |
+
nns | 1.000 1.000 1.46 | 1.000 1.000 1.30 | 1.000 1.000 1.17
|
| 517 |
+
dls | 1.000 1.000 1.39 | 1.000 1.000 1.23 | 1.000 1.000 1.16
|
| 518 |
+
|
| 519 |
+
-- 50-step steering on every candidate: cov / certified-stable cov / median regret % (inf = unstable counts as failure)
|
| 520 |
+
arm K=1: cov cert regret | K=4: cov cert regret | K=8: cov cert regret
|
| 521 |
+
diff_s0 | 1.000 1.000 0.01 | 1.000 1.000 -0.00 | 1.000 1.000 -0.00
|
| 522 |
+
diff_s1 | 1.000 1.000 0.00 | 1.000 1.000 -0.00 | 1.000 1.000 -0.00
|
| 523 |
+
diff_s2 | 1.000 1.000 0.01 | 1.000 1.000 -0.00 | 1.000 1.000 -0.00
|
| 524 |
+
x_s43 | 1.000 0.983 0.01 | 1.000 1.000 -0.00 | 1.000 1.000 -0.00
|
| 525 |
+
x_s44 | 1.000 0.983 0.00 | 1.000 1.000 -0.00 | 1.000 1.000 -0.00
|
| 526 |
+
nn | 1.000 1.000 0.04 | 1.000 1.000 0.00 | 1.000 1.000 -0.00
|
| 527 |
+
idw | 1.000 1.000 0.04 | 1.000 1.000 0.01 | 1.000 1.000 -0.00
|
| 528 |
+
nns | 1.000 1.000 0.00 | 1.000 1.000 -0.00 | 1.000 1.000 -0.00
|
| 529 |
+
dls | 1.000 1.000 0.00 | 1.000 1.000 -0.00 | 1.000 1.000 -0.00
|
| 530 |
+
|
| 531 |
+
-- paired tests, diffusion (model - retrieval): [cov McNemar model-only/retr-only p | certified-cov | paired regret diff median [95% CI], Wilcoxon p]; * = best retrieval arm by median regret
|
| 532 |
+
K=1 st=0 diff_s0 vs nn cov 0-0 p=1 | cert 0-0 p=1 | regret -0.14 [-0.76,+0.27] p=0.178
|
| 533 |
+
K=1 st=0 diff_s0 vs idw cov 0-0 p=1 | cert 0-0 p=1 | regret -0.14 [-0.76,+0.27] p=0.178
|
| 534 |
+
K=1 st=0 diff_s0 vs nns cov 0-0 p=1 | cert 0-0 p=1 | regret +0.09 [-0.16,+0.25] p=0.627
|
| 535 |
+
K=1 st=0 diff_s0 vs dls * cov 0-0 p=1 | cert 0-0 p=1 | regret +0.12 [-0.23,+0.33] p=0.317
|
| 536 |
+
K=1 st=0 diff_s1 vs nn cov 0-0 p=1 | cert 0-0 p=1 | regret -0.21 [-0.59,+0.33] p=0.292
|
| 537 |
+
K=1 st=0 diff_s1 vs idw cov 0-0 p=1 | cert 0-0 p=1 | regret -0.21 [-0.59,+0.33] p=0.292
|
| 538 |
+
K=1 st=0 diff_s1 vs nns cov 0-0 p=1 | cert 0-0 p=1 | regret +0.00 [-0.38,+0.30] p=0.494
|
| 539 |
+
K=1 st=0 diff_s1 vs dls * cov 0-0 p=1 | cert 0-0 p=1 | regret +0.05 [-0.25,+0.57] p=0.183
|
| 540 |
+
K=1 st=0 diff_s2 vs nn cov 0-0 p=1 | cert 0-0 p=1 | regret -0.09 [-0.73,+0.36] p=0.541
|
| 541 |
+
K=1 st=0 diff_s2 vs idw cov 0-0 p=1 | cert 0-0 p=1 | regret -0.09 [-0.73,+0.36] p=0.541
|
| 542 |
+
K=1 st=0 diff_s2 vs nns cov 0-0 p=1 | cert 0-0 p=1 | regret +0.15 [+0.06,+0.22] p=0.169
|
| 543 |
+
K=1 st=0 diff_s2 vs dls * cov 0-0 p=1 | cert 0-0 p=1 | regret +0.06 [-0.09,+0.31] p=0.149
|
| 544 |
+
K=1 st=1 diff_s0 vs nn cov 0-0 p=1 | cert 0-0 p=1 | regret -0.02 [-0.06,+0.00] p=0.00419
|
| 545 |
+
K=1 st=1 diff_s0 vs idw cov 0-0 p=1 | cert 0-0 p=1 | regret -0.02 [-0.06,+0.00] p=0.00419
|
| 546 |
+
K=1 st=1 diff_s0 vs nns cov 0-0 p=1 | cert 0-0 p=1 | regret +0.00 [+0.00,+0.03] p=0.004
|
| 547 |
+
K=1 st=1 diff_s0 vs dls * cov 0-0 p=1 | cert 0-0 p=1 | regret +0.00 [-0.00,+0.05] p=0.0316
|
| 548 |
+
K=1 st=1 diff_s1 vs nn cov 0-0 p=1 | cert 0-0 p=1 | regret -0.03 [-0.05,-0.00] p=0.00897
|
| 549 |
+
K=1 st=1 diff_s1 vs idw cov 0-0 p=1 | cert 0-0 p=1 | regret -0.03 [-0.05,-0.00] p=0.00897
|
| 550 |
+
K=1 st=1 diff_s1 vs nns cov 0-0 p=1 | cert 0-0 p=1 | regret +0.00 [-0.00,+0.01] p=0.076
|
| 551 |
+
K=1 st=1 diff_s1 vs dls * cov 0-0 p=1 | cert 0-0 p=1 | regret +0.00 [-0.00,+0.01] p=0.109
|
| 552 |
+
K=1 st=1 diff_s2 vs nn cov 0-0 p=1 | cert 0-0 p=1 | regret -0.01 [-0.04,+0.00] p=0.052
|
| 553 |
+
K=1 st=1 diff_s2 vs idw cov 0-0 p=1 | cert 0-0 p=1 | regret -0.01 [-0.04,+0.00] p=0.052
|
| 554 |
+
K=1 st=1 diff_s2 vs nns cov 0-0 p=1 | cert 0-0 p=1 | regret +0.02 [+0.00,+0.04] p=3.19e-05
|
| 555 |
+
K=1 st=1 diff_s2 vs dls * cov 0-0 p=1 | cert 0-0 p=1 | regret +0.01 [-0.00,+0.02] p=0.00381
|
| 556 |
+
K=4 st=0 diff_s0 vs nn * cov 0-0 p=1 | cert 0-0 p=1 | regret +0.25 [-0.07,+0.43] p=0.576
|
| 557 |
+
K=4 st=0 diff_s0 vs idw cov 0-0 p=1 | cert 0-0 p=1 | regret +0.16 [-0.19,+0.41] p=0.707
|
| 558 |
+
K=4 st=0 diff_s0 vs nns cov 0-0 p=1 | cert 0-0 p=1 | regret -0.37 [-0.58,-0.19] p=1.07e-05
|
| 559 |
+
K=4 st=0 diff_s0 vs dls cov 0-0 p=1 | cert 0-0 p=1 | regret -0.25 [-0.62,-0.19] p=0.0012
|
| 560 |
+
K=4 st=0 diff_s1 vs nn * cov 0-0 p=1 | cert 0-0 p=1 | regret +0.22 [+0.05,+0.48] p=0.195
|
| 561 |
+
K=4 st=0 diff_s1 vs idw cov 0-0 p=1 | cert 0-0 p=1 | regret +0.18 [-0.04,+0.41] p=0.842
|
| 562 |
+
K=4 st=0 diff_s1 vs nns cov 0-0 p=1 | cert 0-0 p=1 | regret -0.35 [-0.63,-0.09] p=0.000154
|
| 563 |
+
K=4 st=0 diff_s1 vs dls cov 0-0 p=1 | cert 0-0 p=1 | regret -0.36 [-0.65,-0.11] p=0.0102
|
| 564 |
+
K=4 st=0 diff_s2 vs nn * cov 0-0 p=1 | cert 0-0 p=1 | regret +0.22 [+0.04,+0.39] p=0.248
|
| 565 |
+
K=4 st=0 diff_s2 vs idw cov 0-0 p=1 | cert 0-0 p=1 | regret +0.14 [-0.14,+0.35] p=0.994
|
| 566 |
+
K=4 st=0 diff_s2 vs nns cov 0-0 p=1 | cert 0-0 p=1 | regret -0.26 [-0.43,-0.07] p=0.000831
|
| 567 |
+
K=4 st=0 diff_s2 vs dls cov 0-0 p=1 | cert 0-0 p=1 | regret -0.31 [-0.47,-0.12] p=0.0123
|
| 568 |
+
K=4 st=1 diff_s0 vs nn cov 0-0 p=1 | cert 0-0 p=1 | regret -0.00 [-0.01,+0.00] p=0.0181
|
| 569 |
+
K=4 st=1 diff_s0 vs idw cov 0-0 p=1 | cert 0-0 p=1 | regret -0.00 [-0.02,-0.00] p=0.000949
|
| 570 |
+
K=4 st=1 diff_s0 vs nns * cov 0-0 p=1 | cert 0-0 p=1 | regret +0.00 [-0.00,+0.00] p=0.78
|
| 571 |
+
K=4 st=1 diff_s0 vs dls cov 0-0 p=1 | cert 0-0 p=1 | regret -0.00 [-0.00,+0.00] p=0.494
|
| 572 |
+
K=4 st=1 diff_s1 vs nn cov 0-0 p=1 | cert 0-0 p=1 | regret -0.00 [-0.00,+0.00] p=0.219
|
| 573 |
+
K=4 st=1 diff_s1 vs idw cov 0-0 p=1 | cert 0-0 p=1 | regret -0.00 [-0.02,-0.00] p=0.0238
|
| 574 |
+
K=4 st=1 diff_s1 vs nns * cov 0-0 p=1 | cert 0-0 p=1 | regret +0.00 [-0.00,+0.00] p=0.844
|
| 575 |
+
K=4 st=1 diff_s1 vs dls cov 0-0 p=1 | cert 0-0 p=1 | regret -0.00 [-0.00,+0.00] p=0.556
|
| 576 |
+
K=4 st=1 diff_s2 vs nn cov 0-0 p=1 | cert 0-0 p=1 | regret -0.00 [-0.01,+0.00] p=0.04
|
| 577 |
+
K=4 st=1 diff_s2 vs idw cov 0-0 p=1 | cert 0-0 p=1 | regret -0.01 [-0.02,-0.00] p=0.00504
|
| 578 |
+
K=4 st=1 diff_s2 vs nns * cov 0-0 p=1 | cert 0-0 p=1 | regret -0.00 [-0.00,+0.00] p=0.953
|
| 579 |
+
K=4 st=1 diff_s2 vs dls cov 0-0 p=1 | cert 0-0 p=1 | regret -0.00 [-0.00,+0.00] p=0.587
|
| 580 |
+
K=8 st=0 diff_s0 vs nn cov 0-0 p=1 | cert 0-0 p=1 | regret +0.15 [+0.01,+0.35] p=0.0257
|
| 581 |
+
K=8 st=0 diff_s0 vs idw * cov 0-0 p=1 | cert 0-0 p=1 | regret +0.18 [+0.02,+0.35] p=0.0366
|
| 582 |
+
K=8 st=0 diff_s0 vs nns cov 0-0 p=1 | cert 0-0 p=1 | regret -0.37 [-0.56,-0.20] p=1.8e-07
|
| 583 |
+
K=8 st=0 diff_s0 vs dls cov 0-0 p=1 | cert 0-0 p=1 | regret -0.51 [-0.71,-0.34] p=0.00054
|
| 584 |
+
K=8 st=0 diff_s1 vs nn cov 0-0 p=1 | cert 0-0 p=1 | regret +0.20 [+0.04,+0.29] p=0.0422
|
| 585 |
+
K=8 st=0 diff_s1 vs idw * cov 0-0 p=1 | cert 0-0 p=1 | regret +0.16 [+0.04,+0.29] p=0.0407
|
| 586 |
+
K=8 st=0 diff_s1 vs nns cov 0-0 p=1 | cert 0-0 p=1 | regret -0.41 [-0.59,-0.25] p=6.28e-08
|
| 587 |
+
K=8 st=0 diff_s1 vs dls cov 0-0 p=1 | cert 0-0 p=1 | regret -0.55 [-0.70,-0.24] p=0.000347
|
| 588 |
+
K=8 st=0 diff_s2 vs nn cov 0-0 p=1 | cert 0-0 p=1 | regret +0.14 [+0.05,+0.30] p=0.0257
|
| 589 |
+
K=8 st=0 diff_s2 vs idw * cov 0-0 p=1 | cert 0-0 p=1 | regret +0.14 [+0.05,+0.30] p=0.0267
|
| 590 |
+
K=8 st=0 diff_s2 vs nns cov 0-0 p=1 | cert 0-0 p=1 | regret -0.34 [-0.57,-0.23] p=9.1e-07
|
| 591 |
+
K=8 st=0 diff_s2 vs dls cov 0-0 p=1 | cert 0-0 p=1 | regret -0.41 [-0.62,-0.14] p=0.000586
|
| 592 |
+
K=8 st=1 diff_s0 vs nn cov 0-0 p=1 | cert 0-0 p=1 | regret -0.00 [-0.00,+0.00] p=0.328
|
| 593 |
+
K=8 st=1 diff_s0 vs idw cov 0-0 p=1 | cert 0-0 p=1 | regret -0.00 [-0.00,+0.00] p=0.185
|
| 594 |
+
K=8 st=1 diff_s0 vs nns * cov 0-0 p=1 | cert 0-0 p=1 | regret -0.00 [-0.00,+0.00] p=0.0904
|
| 595 |
+
K=8 st=1 diff_s0 vs dls cov 0-0 p=1 | cert 0-0 p=1 | regret -0.00 [-0.00,+0.00] p=0.405
|
| 596 |
+
K=8 st=1 diff_s1 vs nn cov 0-0 p=1 | cert 0-0 p=1 | regret -0.00 [-0.00,+0.00] p=0.785
|
| 597 |
+
K=8 st=1 diff_s1 vs idw cov 0-0 p=1 | cert 0-0 p=1 | regret +0.00 [-0.00,+0.00] p=0.664
|
| 598 |
+
K=8 st=1 diff_s1 vs nns * cov 0-0 p=1 | cert 0-0 p=1 | regret +0.00 [-0.00,+0.00] p=0.723
|
| 599 |
+
K=8 st=1 diff_s1 vs dls cov 0-0 p=1 | cert 0-0 p=1 | regret +0.00 [-0.00,+0.00] p=0.591
|
| 600 |
+
K=8 st=1 diff_s2 vs nn cov 0-0 p=1 | cert 0-0 p=1 | regret -0.00 [-0.00,+0.00] p=0.402
|
| 601 |
+
K=8 st=1 diff_s2 vs idw cov 0-0 p=1 | cert 0-0 p=1 | regret -0.00 [-0.00,+0.00] p=0.31
|
| 602 |
+
K=8 st=1 diff_s2 vs nns * cov 0-0 p=1 | cert 0-0 p=1 | regret -0.00 [-0.00,+0.00] p=0.0393
|
| 603 |
+
K=8 st=1 diff_s2 vs dls cov 0-0 p=1 | cert 0-0 p=1 | regret -0.00 [-0.00,+0.00] p=0.994
|
| 604 |
+
|
| 605 |
+
=== hole_out: N=60 NN-dist ratio test/bank 1.0x (mean NN 0.149); oracle finite 60 ===
|
| 606 |
+
-- no steering: cov / certified-stable cov / median regret % (inf = unstable counts as failure)
|
| 607 |
+
arm K=1: cov cert regret | K=4: cov cert regret | K=8: cov cert regret
|
| 608 |
+
diff_s0 | 1.000 1.000 1.59 | 1.000 1.000 1.29 | 1.000 1.000 1.22
|
| 609 |
+
diff_s1 | 1.000 1.000 1.58 | 1.000 1.000 1.24 | 1.000 1.000 1.16
|
| 610 |
+
diff_s2 | 1.000 1.000 1.47 | 1.000 1.000 1.18 | 1.000 1.000 1.12
|
| 611 |
+
x_s43 | 1.000 0.983 1.43 | 1.000 1.000 0.95 | 1.000 1.000 0.81
|
| 612 |
+
x_s44 | 1.000 1.000 1.31 | 1.000 1.000 0.97 | 1.000 1.000 0.94
|
| 613 |
+
nn | 1.000 1.000 1.40 | 1.000 1.000 0.76 | 1.000 1.000 0.36
|
| 614 |
+
idw | 1.000 1.000 1.40 | 1.000 1.000 0.78 | 1.000 1.000 0.36
|
| 615 |
+
nns | 1.000 1.000 1.35 | 1.000 1.000 1.15 | 1.000 1.000 1.04
|
| 616 |
+
dls | 1.000 1.000 1.38 | 1.000 1.000 1.10 | 1.000 1.000 1.01
|
| 617 |
+
|
| 618 |
+
-- 50-step steering on every candidate: cov / certified-stable cov / median regret % (inf = unstable counts as failure)
|
| 619 |
+
arm K=1: cov cert regret | K=4: cov cert regret | K=8: cov cert regret
|
| 620 |
+
diff_s0 | 1.000 0.950 0.01 | 1.000 0.967 0.00 | 1.000 0.967 0.00
|
| 621 |
+
diff_s1 | 1.000 0.950 0.02 | 1.000 0.950 0.00 | 1.000 0.983 0.00
|
| 622 |
+
diff_s2 | 1.000 0.950 0.01 | 1.000 0.967 0.00 | 1.000 0.967 0.00
|
| 623 |
+
x_s43 | 1.000 0.933 0.03 | 1.000 0.933 0.00 | 1.000 0.933 -0.00
|
| 624 |
+
x_s44 | 1.000 0.933 0.01 | 1.000 0.933 0.00 | 1.000 0.933 -0.00
|
| 625 |
+
nn | 1.000 0.967 0.01 | 1.000 0.983 -0.00 | 1.000 1.000 -0.01
|
| 626 |
+
idw | 1.000 0.967 0.01 | 1.000 0.983 -0.00 | 1.000 1.000 -0.01
|
| 627 |
+
nns | 1.000 0.950 0.01 | 1.000 0.950 0.00 | 1.000 0.950 0.00
|
| 628 |
+
dls | 1.000 0.950 0.01 | 1.000 0.950 0.00 | 1.000 0.950 0.00
|
| 629 |
+
|
| 630 |
+
-- paired tests, diffusion (model - retrieval): [cov McNemar model-only/retr-only p | certified-cov | paired regret diff median [95% CI], Wilcoxon p]; * = best retrieval arm by median regret
|
| 631 |
+
K=1 st=0 diff_s0 vs nn cov 0-0 p=1 | cert 0-0 p=1 | regret +0.17 [-0.04,+0.49] p=0.151
|
| 632 |
+
K=1 st=0 diff_s0 vs idw cov 0-0 p=1 | cert 0-0 p=1 | regret +0.17 [-0.04,+0.49] p=0.151
|
| 633 |
+
K=1 st=0 diff_s0 vs nns * cov 0-0 p=1 | cert 0-0 p=1 | regret +0.20 [+0.10,+0.40] p=0.000129
|
| 634 |
+
K=1 st=0 diff_s0 vs dls cov 0-0 p=1 | cert 0-0 p=1 | regret +0.14 [+0.01,+0.39] p=0.0161
|
| 635 |
+
K=1 st=0 diff_s1 vs nn cov 0-0 p=1 | cert 0-0 p=1 | regret +0.20 [-0.14,+0.32] p=0.339
|
| 636 |
+
K=1 st=0 diff_s1 vs idw cov 0-0 p=1 | cert 0-0 p=1 | regret +0.20 [-0.14,+0.32] p=0.339
|
| 637 |
+
K=1 st=0 diff_s1 vs nns * cov 0-0 p=1 | cert 0-0 p=1 | regret +0.21 [+0.04,+0.47] p=0.00136
|
| 638 |
+
K=1 st=0 diff_s1 vs dls cov 0-0 p=1 | cert 0-0 p=1 | regret +0.15 [+0.01,+0.42] p=0.0167
|
| 639 |
+
K=1 st=0 diff_s2 vs nn cov 0-0 p=1 | cert 0-0 p=1 | regret +0.12 [-0.05,+0.38] p=0.401
|
| 640 |
+
K=1 st=0 diff_s2 vs idw cov 0-0 p=1 | cert 0-0 p=1 | regret +0.12 [-0.05,+0.38] p=0.401
|
| 641 |
+
K=1 st=0 diff_s2 vs nns * cov 0-0 p=1 | cert 0-0 p=1 | regret +0.22 [+0.06,+0.33] p=0.000292
|
| 642 |
+
K=1 st=0 diff_s2 vs dls cov 0-0 p=1 | cert 0-0 p=1 | regret +0.02 [-0.05,+0.28] p=0.0299
|
| 643 |
+
K=1 st=1 diff_s0 vs nn cov 0-0 p=1 | cert 0-1 p=1 | regret +0.00 [-0.01,+0.02] p=0.361
|
| 644 |
+
K=1 st=1 diff_s0 vs idw cov 0-0 p=1 | cert 0-1 p=1 | regret +0.00 [-0.01,+0.02] p=0.361
|
| 645 |
+
K=1 st=1 diff_s0 vs nns cov 0-0 p=1 | cert 1-1 p=1 | regret +0.00 [+0.00,+0.03] p=0.000946
|
| 646 |
+
K=1 st=1 diff_s0 vs dls * cov 0-0 p=1 | cert 1-1 p=1 | regret +0.01 [+0.00,+0.03] p=0.0014
|
| 647 |
+
K=1 st=1 diff_s1 vs nn cov 0-0 p=1 | cert 0-1 p=1 | regret +0.00 [-0.00,+0.01] p=0.435
|
| 648 |
+
K=1 st=1 diff_s1 vs idw cov 0-0 p=1 | cert 0-1 p=1 | regret +0.00 [-0.00,+0.01] p=0.435
|
| 649 |
+
K=1 st=1 diff_s1 vs nns cov 0-0 p=1 | cert 0-0 p=1 | regret +0.01 [+0.00,+0.03] p=0.000102
|
| 650 |
+
K=1 st=1 diff_s1 vs dls * cov 0-0 p=1 | cert 0-0 p=1 | regret +0.01 [+0.00,+0.02] p=0.000526
|
| 651 |
+
K=1 st=1 diff_s2 vs nn cov 0-0 p=1 | cert 0-1 p=1 | regret +0.00 [-0.01,+0.01] p=0.531
|
| 652 |
+
K=1 st=1 diff_s2 vs idw cov 0-0 p=1 | cert 0-1 p=1 | regret +0.00 [-0.01,+0.01] p=0.531
|
| 653 |
+
K=1 st=1 diff_s2 vs nns cov 0-0 p=1 | cert 0-0 p=1 | regret +0.01 [+0.00,+0.03] p=0.000337
|
| 654 |
+
K=1 st=1 diff_s2 vs dls * cov 0-0 p=1 | cert 0-0 p=1 | regret +0.00 [+0.00,+0.02] p=0.00189
|
| 655 |
+
K=4 st=0 diff_s0 vs nn * cov 0-0 p=1 | cert 0-0 p=1 | regret +0.45 [+0.31,+0.80] p=3e-07
|
| 656 |
+
K=4 st=0 diff_s0 vs idw cov 0-0 p=1 | cert 0-0 p=1 | regret +0.43 [+0.28,+0.64] p=1.27e-05
|
| 657 |
+
K=4 st=0 diff_s0 vs nns cov 0-0 p=1 | cert 0-0 p=1 | regret +0.17 [+0.08,+0.22] p=0.00155
|
| 658 |
+
K=4 st=0 diff_s0 vs dls cov 0-0 p=1 | cert 0-0 p=1 | regret +0.03 [-0.09,+0.15] p=0.122
|
| 659 |
+
K=4 st=0 diff_s1 vs nn * cov 0-0 p=1 | cert 0-0 p=1 | regret +0.45 [+0.34,+0.75] p=6.02e-08
|
| 660 |
+
K=4 st=0 diff_s1 vs idw cov 0-0 p=1 | cert 0-0 p=1 | regret +0.39 [+0.29,+0.71] p=5.97e-06
|
| 661 |
+
K=4 st=0 diff_s1 vs nns cov 0-0 p=1 | cert 0-0 p=1 | regret +0.17 [+0.08,+0.26] p=0.00176
|
| 662 |
+
K=4 st=0 diff_s1 vs dls cov 0-0 p=1 | cert 0-0 p=1 | regret +0.05 [-0.03,+0.26] p=0.0359
|
| 663 |
+
K=4 st=0 diff_s2 vs nn * cov 0-0 p=1 | cert 0-0 p=1 | regret +0.45 [+0.31,+0.69] p=2.77e-07
|
| 664 |
+
K=4 st=0 diff_s2 vs idw cov 0-0 p=1 | cert 0-0 p=1 | regret +0.40 [+0.24,+0.66] p=1.36e-05
|
| 665 |
+
K=4 st=0 diff_s2 vs nns cov 0-0 p=1 | cert 0-0 p=1 | regret +0.09 [+0.02,+0.26] p=0.0084
|
| 666 |
+
K=4 st=0 diff_s2 vs dls cov 0-0 p=1 | cert 0-0 p=1 | regret +0.02 [-0.06,+0.16] p=0.105
|
| 667 |
+
K=4 st=1 diff_s0 vs nn * cov 0-0 p=1 | cert 0-1 p=1 | regret +0.01 [+0.00,+0.03] p=0.0003
|
| 668 |
+
K=4 st=1 diff_s0 vs idw cov 0-0 p=1 | cert 0-1 p=1 | regret +0.00 [+0.00,+0.03] p=0.00178
|
| 669 |
+
K=4 st=1 diff_s0 vs nns cov 0-0 p=1 | cert 1-0 p=1 | regret +0.00 [-0.00,+0.01] p=0.00574
|
| 670 |
+
K=4 st=1 diff_s0 vs dls cov 0-0 p=1 | cert 1-0 p=1 | regret +0.00 [-0.00,+0.02] p=0.0113
|
| 671 |
+
K=4 st=1 diff_s1 vs nn * cov 0-0 p=1 | cert 0-2 p=0.5 | regret +0.01 [+0.00,+0.03] p=0.00105
|
| 672 |
+
K=4 st=1 diff_s1 vs idw cov 0-0 p=1 | cert 0-2 p=0.5 | regret +0.01 [-0.00,+0.02] p=0.00214
|
| 673 |
+
K=4 st=1 diff_s1 vs nns cov 0-0 p=1 | cert 0-0 p=1 | regret +0.00 [-0.00,+0.01] p=0.0386
|
| 674 |
+
K=4 st=1 diff_s1 vs dls cov 0-0 p=1 | cert 0-0 p=1 | regret +0.00 [-0.00,+0.01] p=0.0216
|
| 675 |
+
K=4 st=1 diff_s2 vs nn * cov 0-0 p=1 | cert 0-1 p=1 | regret +0.01 [+0.00,+0.04] p=0.000213
|
| 676 |
+
K=4 st=1 diff_s2 vs idw cov 0-0 p=1 | cert 0-1 p=1 | regret +0.00 [+0.00,+0.02] p=0.00316
|
| 677 |
+
K=4 st=1 diff_s2 vs nns cov 0-0 p=1 | cert 1-0 p=1 | regret +0.00 [+0.00,+0.01] p=0.0547
|
| 678 |
+
K=4 st=1 diff_s2 vs dls cov 0-0 p=1 | cert 1-0 p=1 | regret +0.00 [-0.00,+0.01] p=0.00459
|
| 679 |
+
K=8 st=0 diff_s0 vs nn * cov 0-0 p=1 | cert 0-0 p=1 | regret +0.68 [+0.48,+0.92] p=5.19e-10
|
| 680 |
+
K=8 st=0 diff_s0 vs idw cov 0-0 p=1 | cert 0-0 p=1 | regret +0.68 [+0.48,+0.92] p=6.26e-10
|
| 681 |
+
K=8 st=0 diff_s0 vs nns cov 0-0 p=1 | cert 0-0 p=1 | regret +0.16 [+0.05,+0.19] p=0.000239
|
| 682 |
+
K=8 st=0 diff_s0 vs dls cov 0-0 p=1 | cert 0-0 p=1 | regret +0.02 [-0.04,+0.16] p=0.0798
|
| 683 |
+
K=8 st=0 diff_s1 vs nn * cov 0-0 p=1 | cert 0-0 p=1 | regret +0.69 [+0.48,+0.92] p=4.3e-10
|
| 684 |
+
K=8 st=0 diff_s1 vs idw cov 0-0 p=1 | cert 0-0 p=1 | regret +0.69 [+0.48,+0.92] p=6.26e-10
|
| 685 |
+
K=8 st=0 diff_s1 vs nns cov 0-0 p=1 | cert 0-0 p=1 | regret +0.13 [+0.08,+0.27] p=0.000399
|
| 686 |
+
K=8 st=0 diff_s1 vs dls cov 0-0 p=1 | cert 0-0 p=1 | regret +0.07 [-0.05,+0.18] p=0.0657
|
| 687 |
+
K=8 st=0 diff_s2 vs nn * cov 0-0 p=1 | cert 0-0 p=1 | regret +0.66 [+0.43,+0.97] p=4.3e-10
|
| 688 |
+
K=8 st=0 diff_s2 vs idw cov 0-0 p=1 | cert 0-0 p=1 | regret +0.66 [+0.43,+0.97] p=5.19e-10
|
| 689 |
+
K=8 st=0 diff_s2 vs nns cov 0-0 p=1 | cert 0-0 p=1 | regret +0.08 [+0.02,+0.18] p=0.00631
|
| 690 |
+
K=8 st=0 diff_s2 vs dls cov 0-0 p=1 | cert 0-0 p=1 | regret +0.04 [-0.08,+0.12] p=0.185
|
| 691 |
+
K=8 st=1 diff_s0 vs nn * cov 0-0 p=1 | cert 0-2 p=0.5 | regret +0.02 [+0.00,+0.03] p=1.08e-05
|
| 692 |
+
K=8 st=1 diff_s0 vs idw cov 0-0 p=1 | cert 0-2 p=0.5 | regret +0.02 [+0.00,+0.03] p=8.16e-06
|
| 693 |
+
K=8 st=1 diff_s0 vs nns cov 0-0 p=1 | cert 1-0 p=1 | regret +0.00 [-0.00,+0.01] p=0.0106
|
| 694 |
+
K=8 st=1 diff_s0 vs dls cov 0-0 p=1 | cert 1-0 p=1 | regret +0.00 [-0.00,+0.02] p=0.00645
|
| 695 |
+
K=8 st=1 diff_s1 vs nn * cov 0-0 p=1 | cert 0-1 p=1 | regret +0.01 [+0.00,+0.03] p=0.000169
|
| 696 |
+
K=8 st=1 diff_s1 vs idw cov 0-0 p=1 | cert 0-1 p=1 | regret +0.01 [+0.00,+0.03] p=0.000169
|
| 697 |
+
K=8 st=1 diff_s1 vs nns cov 0-0 p=1 | cert 2-0 p=0.5 | regret +0.00 [-0.00,+0.01] p=0.0701
|
| 698 |
+
K=8 st=1 diff_s1 vs dls cov 0-0 p=1 | cert 2-0 p=0.5 | regret +0.00 [-0.00,+0.02] p=0.00419
|
| 699 |
+
K=8 st=1 diff_s2 vs nn * cov 0-0 p=1 | cert 0-2 p=0.5 | regret +0.01 [+0.00,+0.03] p=8.15e-06
|
| 700 |
+
K=8 st=1 diff_s2 vs idw cov 0-0 p=1 | cert 0-2 p=0.5 | regret +0.01 [+0.00,+0.02] p=7.61e-06
|
| 701 |
+
K=8 st=1 diff_s2 vs nns cov 0-0 p=1 | cert 1-0 p=1 | regret +0.00 [-0.00,+0.00] p=0.115
|
| 702 |
+
K=8 st=1 diff_s2 vs dls cov 0-0 p=1 | cert 1-0 p=1 | regret +0.00 [-0.00,+0.02] p=0.00617
|
| 703 |
+
|
| 704 |
+
=== pilotfar: N=60 NN-dist ratio test/bank 27.2x (mean NN 3.897); oracle finite 60 ===
|
| 705 |
+
-- no steering: cov / certified-stable cov / median regret % (inf = unstable counts as failure)
|
| 706 |
+
arm K=1: cov cert regret | K=4: cov cert regret | K=8: cov cert regret
|
| 707 |
+
diff_s0 | 0.967 0.000 1.20 | 1.000 0.033 -5.36 | 1.000 0.033 -6.49
|
| 708 |
+
diff_s1 | 0.950 0.000 1.37 | 1.000 0.000 -4.15 | 1.000 0.000 -6.74
|
| 709 |
+
diff_s2 | 0.933 0.000 0.80 | 1.000 0.000 -3.90 | 1.000 0.017 -6.68
|
| 710 |
+
x_s42 | 0.950 0.017 1.03 | 1.000 0.067 -6.11 | 1.000 0.117 -7.49
|
| 711 |
+
x_s43 | 0.967 0.017 4.81 | 1.000 0.050 -3.51 | 1.000 0.083 -5.40
|
| 712 |
+
x_s44 | 0.950 0.033 3.56 | 1.000 0.083 -4.13 | 1.000 0.117 -5.92
|
| 713 |
+
nn | 0.717 0.517 69.64 | 0.717 0.517 69.64 | 0.717 0.517 69.64
|
| 714 |
+
idw | 0.717 0.517 69.64 | 0.383 0.250 1000.00 | 0.383 0.233 1000.00
|
| 715 |
+
nns | 1.000 1.000 2.25 | 1.000 1.000 0.25 | 1.000 1.000 0.04
|
| 716 |
+
dls | 1.000 1.000 1.05 | 1.000 1.000 0.18 | 1.000 1.000 -1.73
|
| 717 |
+
|
| 718 |
+
-- 50-step steering on every candidate: cov / certified-stable cov / median regret % (inf = unstable counts as failure)
|
| 719 |
+
arm K=1: cov cert regret | K=4: cov cert regret | K=8: cov cert regret
|
| 720 |
+
diff_s0 | 1.000 0.000 -3.21 | 1.000 0.033 -7.11 | 1.000 0.033 -8.32
|
| 721 |
+
diff_s1 | 0.983 0.000 -3.35 | 1.000 0.000 -7.20 | 1.000 0.000 -8.70
|
| 722 |
+
diff_s2 | 0.983 0.000 -3.61 | 1.000 0.000 -6.68 | 1.000 0.017 -8.44
|
| 723 |
+
x_s42 | 0.983 0.017 -3.16 | 1.000 0.067 -7.46 | 1.000 0.117 -8.68
|
| 724 |
+
x_s43 | 0.983 0.017 -0.83 | 1.000 0.050 -6.11 | 1.000 0.083 -7.52
|
| 725 |
+
x_s44 | 0.983 0.033 -1.83 | 1.000 0.083 -6.67 | 1.000 0.117 -7.78
|
| 726 |
+
nn | 0.867 0.550 32.16 | 0.867 0.550 32.16 | 0.867 0.550 32.16
|
| 727 |
+
idw | 0.867 0.550 32.16 | 0.617 0.233 107.65 | 0.617 0.283 99.44
|
| 728 |
+
nns | 1.000 1.000 0.61 | 1.000 1.000 -1.34 | 1.000 1.000 -1.48
|
| 729 |
+
dls | 1.000 1.000 -0.64 | 1.000 1.000 -1.44 | 1.000 1.000 -3.25
|
| 730 |
+
|
| 731 |
+
-- paired tests, diffusion (model - retrieval): [cov McNemar model-only/retr-only p | certified-cov | paired regret diff median [95% CI], Wilcoxon p]; * = best retrieval arm by median regret
|
| 732 |
+
K=1 st=0 diff_s0 vs nn cov 15-0 p=6.1e-05 | cert 0-31 p=9.31e-10 | regret -58.08 [-92.63,-29.15] p=3.51e-11
|
| 733 |
+
K=1 st=0 diff_s0 vs idw cov 15-0 p=6.1e-05 | cert 0-31 p=9.31e-10 | regret -58.08 [-92.63,-29.15] p=3.51e-11
|
| 734 |
+
K=1 st=0 diff_s0 vs nns cov 0-2 p=0.5 | cert 0-60 p=1.73e-18 | regret -1.27 [-4.91,+2.54] p=0.842
|
| 735 |
+
K=1 st=0 diff_s0 vs dls * cov 0-2 p=0.5 | cert 0-60 p=1.73e-18 | regret -0.44 [-3.61,+3.92] p=0.365
|
| 736 |
+
K=1 st=0 diff_s1 vs nn cov 14-0 p=0.000122 | cert 0-31 p=9.31e-10 | regret -58.51 [-88.98,-24.34] p=1.08e-10
|
| 737 |
+
K=1 st=0 diff_s1 vs idw cov 14-0 p=0.000122 | cert 0-31 p=9.31e-10 | regret -58.51 [-88.98,-24.34] p=1.08e-10
|
| 738 |
+
K=1 st=0 diff_s1 vs nns cov 0-3 p=0.25 | cert 0-60 p=1.73e-18 | regret -0.38 [-2.89,+9.62] p=0.0247
|
| 739 |
+
K=1 st=0 diff_s1 vs dls * cov 0-3 p=0.25 | cert 0-60 p=1.73e-18 | regret +0.61 [-1.87,+10.49] p=0.00897
|
| 740 |
+
K=1 st=0 diff_s2 vs nn cov 15-2 p=0.00235 | cert 0-31 p=9.31e-10 | regret -46.96 [-99.93,-28.23] p=3.56e-09
|
| 741 |
+
K=1 st=0 diff_s2 vs idw cov 15-2 p=0.00235 | cert 0-31 p=9.31e-10 | regret -46.96 [-99.93,-28.23] p=3.56e-09
|
| 742 |
+
K=1 st=0 diff_s2 vs nns cov 0-4 p=0.125 | cert 0-60 p=1.73e-18 | regret -1.12 [-4.05,+3.92] p=0.263
|
| 743 |
+
K=1 st=0 diff_s2 vs dls * cov 0-4 p=0.125 | cert 0-60 p=1.73e-18 | regret -0.08 [-2.69,+4.63] p=0.0976
|
| 744 |
+
K=1 st=1 diff_s0 vs nn cov 8-0 p=0.00781 | cert 0-33 p=2.33e-10 | regret -32.32 [-52.19,-18.43] p=1.63e-11
|
| 745 |
+
K=1 st=1 diff_s0 vs idw cov 8-0 p=0.00781 | cert 0-33 p=2.33e-10 | regret -32.32 [-52.19,-18.43] p=1.63e-11
|
| 746 |
+
K=1 st=1 diff_s0 vs nns cov 0-0 p=1 | cert 0-60 p=1.73e-18 | regret -4.67 [-6.66,-2.56] p=0.0437
|
| 747 |
+
K=1 st=1 diff_s0 vs dls * cov 0-0 p=1 | cert 0-60 p=1.73e-18 | regret -3.91 [-5.21,-1.34] p=0.0773
|
| 748 |
+
K=1 st=1 diff_s1 vs nn cov 8-1 p=0.0391 | cert 0-33 p=2.33e-10 | regret -34.71 [-45.72,-15.72] p=3.92e-10
|
| 749 |
+
K=1 st=1 diff_s1 vs idw cov 8-1 p=0.0391 | cert 0-33 p=2.33e-10 | regret -34.71 [-45.72,-15.72] p=3.92e-10
|
| 750 |
+
K=1 st=1 diff_s1 vs nns cov 0-1 p=1 | cert 0-60 p=1.73e-18 | regret -3.70 [-5.51,+1.78] p=0.883
|
| 751 |
+
K=1 st=1 diff_s1 vs dls * cov 0-1 p=1 | cert 0-60 p=1.73e-18 | regret -2.71 [-4.72,+2.44] p=0.611
|
| 752 |
+
K=1 st=1 diff_s2 vs nn cov 8-1 p=0.0391 | cert 0-33 p=2.33e-10 | regret -30.56 [-53.61,-16.60] p=1.57e-09
|
| 753 |
+
K=1 st=1 diff_s2 vs idw cov 8-1 p=0.0391 | cert 0-33 p=2.33e-10 | regret -30.56 [-53.61,-16.60] p=1.57e-09
|
| 754 |
+
K=1 st=1 diff_s2 vs nns cov 0-1 p=1 | cert 0-60 p=1.73e-18 | regret -4.43 [-6.23,-2.05] p=0.11
|
| 755 |
+
K=1 st=1 diff_s2 vs dls * cov 0-1 p=1 | cert 0-60 p=1.73e-18 | regret -3.21 [-4.98,-0.61] p=0.227
|
| 756 |
+
K=4 st=0 diff_s0 vs nn cov 17-0 p=1.53e-05 | cert 0-29 p=3.73e-09 | regret -75.78 [-111.25,-48.62] p=1.63e-11
|
| 757 |
+
K=4 st=0 diff_s0 vs idw cov 37-0 p=1.46e-11 | cert 0-13 p=0.000244 | regret -992.50 [-1001.55,-149.64] p=1.63e-11
|
| 758 |
+
K=4 st=0 diff_s0 vs nns cov 0-0 p=1 | cert 0-58 p=6.94e-18 | regret -7.13 [-7.78,-5.43] p=0.000446
|
| 759 |
+
K=4 st=0 diff_s0 vs dls * cov 0-0 p=1 | cert 0-58 p=6.94e-18 | regret -6.43 [-7.10,-4.69] p=0.000876
|
| 760 |
+
K=4 st=0 diff_s1 vs nn cov 17-0 p=1.53e-05 | cert 0-31 p=9.31e-10 | regret -70.24 [-103.53,-48.90] p=1.63e-11
|
| 761 |
+
K=4 st=0 diff_s1 vs idw cov 37-0 p=1.46e-11 | cert 0-15 p=6.1e-05 | regret -992.34 [-1000.23,-150.59] p=1.63e-11
|
| 762 |
+
K=4 st=0 diff_s1 vs nns cov 0-0 p=1 | cert 0-60 p=1.73e-18 | regret -4.70 [-6.46,-3.24] p=0.000164
|
| 763 |
+
K=4 st=0 diff_s1 vs dls * cov 0-0 p=1 | cert 0-60 p=1.73e-18 | regret -5.18 [-6.07,-3.05] p=0.000484
|
| 764 |
+
K=4 st=0 diff_s2 vs nn cov 17-0 p=1.53e-05 | cert 0-31 p=9.31e-10 | regret -71.89 [-114.87,-45.49] p=1.63e-11
|
| 765 |
+
K=4 st=0 diff_s2 vs idw cov 37-0 p=1.46e-11 | cert 0-15 p=6.1e-05 | regret -997.48 [-999.36,-147.56] p=1.63e-11
|
| 766 |
+
K=4 st=0 diff_s2 vs nns cov 0-0 p=1 | cert 0-60 p=1.73e-18 | regret -5.38 [-6.72,-3.39] p=4.84e-06
|
| 767 |
+
K=4 st=0 diff_s2 vs dls * cov 0-0 p=1 | cert 0-60 p=1.73e-18 | regret -4.67 [-6.32,-2.59] p=1.71e-05
|
| 768 |
+
K=4 st=1 diff_s0 vs nn cov 8-0 p=0.00781 | cert 0-31 p=9.31e-10 | regret -39.27 [-52.67,-25.66] p=1.63e-11
|
| 769 |
+
K=4 st=1 diff_s0 vs idw cov 23-0 p=2.38e-07 | cert 0-12 p=0.000488 | regret -115.13 [-564.70,-68.45] p=1.63e-11
|
| 770 |
+
K=4 st=1 diff_s0 vs nns cov 0-0 p=1 | cert 0-58 p=6.94e-18 | regret -7.11 [-7.58,-6.27] p=3.12e-07
|
| 771 |
+
K=4 st=1 diff_s0 vs dls * cov 0-0 p=1 | cert 0-58 p=6.94e-18 | regret -6.28 [-7.70,-5.03] p=2.29e-06
|
| 772 |
+
K=4 st=1 diff_s1 vs nn cov 8-0 p=0.00781 | cert 0-33 p=2.33e-10 | regret -36.37 [-51.65,-26.14] p=1.63e-11
|
| 773 |
+
K=4 st=1 diff_s1 vs idw cov 23-0 p=2.38e-07 | cert 0-14 p=0.000122 | regret -111.16 [-571.43,-69.65] p=1.63e-11
|
| 774 |
+
K=4 st=1 diff_s1 vs nns cov 0-0 p=1 | cert 0-60 p=1.73e-18 | regret -5.95 [-7.10,-5.00] p=3.24e-09
|
| 775 |
+
K=4 st=1 diff_s1 vs dls * cov 0-0 p=1 | cert 0-60 p=1.73e-18 | regret -5.47 [-6.58,-4.63] p=4.9e-08
|
| 776 |
+
K=4 st=1 diff_s2 vs nn cov 8-0 p=0.00781 | cert 0-33 p=2.33e-10 | regret -37.30 [-56.56,-24.37] p=1.63e-11
|
| 777 |
+
K=4 st=1 diff_s2 vs idw cov 23-0 p=2.38e-07 | cert 0-14 p=0.000122 | regret -113.67 [-572.30,-67.88] p=1.63e-11
|
| 778 |
+
K=4 st=1 diff_s2 vs nns cov 0-0 p=1 | cert 0-60 p=1.73e-18 | regret -6.32 [-7.28,-5.21] p=9.35e-09
|
| 779 |
+
K=4 st=1 diff_s2 vs dls * cov 0-0 p=1 | cert 0-60 p=1.73e-18 | regret -5.13 [-7.07,-4.54] p=1.21e-08
|
| 780 |
+
K=8 st=0 diff_s0 vs nn cov 17-0 p=1.53e-05 | cert 0-29 p=3.73e-09 | regret -75.78 [-115.18,-48.62] p=1.63e-11
|
| 781 |
+
K=8 st=0 diff_s0 vs idw cov 37-0 p=1.46e-11 | cert 0-12 p=0.000488 | regret -1001.19 [-1003.63,-149.64] p=1.63e-11
|
| 782 |
+
K=8 st=0 diff_s0 vs nns cov 0-0 p=1 | cert 0-58 p=6.94e-18 | regret -7.87 [-8.53,-6.59] p=7.7e-08
|
| 783 |
+
K=8 st=0 diff_s0 vs dls * cov 0-0 p=1 | cert 0-58 p=6.94e-18 | regret -5.28 [-6.10,-4.50] p=2.95e-06
|
| 784 |
+
K=8 st=0 diff_s1 vs nn cov 17-0 p=1.53e-05 | cert 0-31 p=9.31e-10 | regret -76.92 [-114.56,-48.90] p=1.63e-11
|
| 785 |
+
K=8 st=0 diff_s1 vs idw cov 37-0 p=1.46e-11 | cert 0-14 p=0.000122 | regret -999.54 [-1003.53,-150.96] p=1.63e-11
|
| 786 |
+
K=8 st=0 diff_s1 vs nns cov 0-0 p=1 | cert 0-60 p=1.73e-18 | regret -7.19 [-7.89,-6.11] p=1.86e-08
|
| 787 |
+
K=8 st=0 diff_s1 vs dls * cov 0-0 p=1 | cert 0-60 p=1.73e-18 | regret -5.14 [-5.94,-3.69] p=2.95e-06
|
| 788 |
+
K=8 st=0 diff_s2 vs nn cov 17-0 p=1.53e-05 | cert 0-30 p=1.86e-09 | regret -75.14 [-116.95,-47.11] p=1.63e-11
|
| 789 |
+
K=8 st=0 diff_s2 vs idw cov 37-0 p=1.46e-11 | cert 0-13 p=0.000244 | regret -1001.80 [-1003.53,-148.13] p=1.63e-11
|
| 790 |
+
K=8 st=0 diff_s2 vs nns cov 0-0 p=1 | cert 0-59 p=3.47e-18 | regret -7.13 [-8.37,-6.37] p=3.56e-10
|
| 791 |
+
K=8 st=0 diff_s2 vs dls * cov 0-0 p=1 | cert 0-59 p=3.47e-18 | regret -5.25 [-5.94,-3.91] p=1.38e-08
|
| 792 |
+
K=8 st=1 diff_s0 vs nn cov 8-0 p=0.00781 | cert 0-31 p=9.31e-10 | regret -39.55 [-57.32,-25.66] p=1.63e-11
|
| 793 |
+
K=8 st=1 diff_s0 vs idw cov 23-0 p=2.38e-07 | cert 0-15 p=6.1e-05 | regret -107.57 [-576.87,-68.61] p=1.63e-11
|
| 794 |
+
K=8 st=1 diff_s0 vs nns cov 0-0 p=1 | cert 0-58 p=6.94e-18 | regret -7.42 [-7.96,-7.02] p=5.44e-10
|
| 795 |
+
K=8 st=1 diff_s0 vs dls * cov 0-0 p=1 | cert 0-58 p=6.94e-18 | regret -5.01 [-5.26,-4.53] p=8.57e-09
|
| 796 |
+
K=8 st=1 diff_s1 vs nn cov 8-0 p=0.00781 | cert 0-33 p=2.33e-10 | regret -40.52 [-57.25,-26.39] p=1.63e-11
|
| 797 |
+
K=8 st=1 diff_s1 vs idw cov 23-0 p=2.38e-07 | cert 0-17 p=1.53e-05 | regret -106.24 [-576.80,-69.90] p=1.63e-11
|
| 798 |
+
K=8 st=1 diff_s1 vs nns cov 0-0 p=1 | cert 0-60 p=1.73e-18 | regret -7.06 [-7.42,-6.61] p=4.1e-10
|
| 799 |
+
K=8 st=1 diff_s1 vs dls * cov 0-0 p=1 | cert 0-60 p=1.73e-18 | regret -4.88 [-5.15,-4.32] p=3.88e-09
|
| 800 |
+
K=8 st=1 diff_s2 vs nn cov 8-0 p=0.00781 | cert 0-32 p=4.66e-10 | regret -39.27 [-58.19,-25.03] p=1.63e-11
|
| 801 |
+
K=8 st=1 diff_s2 vs idw cov 23-0 p=2.38e-07 | cert 0-16 p=3.05e-05 | regret -106.29 [-578.28,-68.13] p=1.63e-11
|
| 802 |
+
K=8 st=1 diff_s2 vs nns cov 0-0 p=1 | cert 0-59 p=3.47e-18 | regret -7.15 [-7.37,-6.66] p=2.01e-10
|
| 803 |
+
K=8 st=1 diff_s2 vs dls * cov 0-0 p=1 | cert 0-59 p=3.47e-18 | regret -5.01 [-5.26,-4.22] p=2.21e-10
|
| 804 |
+
|
results/ood/siso_ood.txt
ADDED
|
@@ -0,0 +1,165 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
=== SISO lvl0: N=120, z-NN ratio test/bank 1.1x (median-based 0.242) ===
|
| 2 |
+
-- no steering: coverage (any of K stable) | per-candidate stable rate
|
| 3 |
+
diff_s0 K=1: 0.550 0.550 | K=4: 0.650 0.515 | K=8: 0.667 0.518
|
| 4 |
+
diff_s1 K=1: 0.483 0.483 | K=4: 0.667 0.525 | K=8: 0.692 0.512
|
| 5 |
+
diff_s2 K=1: 0.517 0.517 | K=4: 0.642 0.521 | K=8: 0.667 0.532
|
| 6 |
+
nn K=1: 0.375 0.375 | K=4: 0.608 0.394 | K=8: 0.642 0.401
|
| 7 |
+
idw K=1: 0.375 0.375 | K=4: 0.608 0.423 | K=8: 0.650 0.424
|
| 8 |
+
-- 50-step steering: coverage (any of K stable) | per-candidate stable rate
|
| 9 |
+
diff_s0 K=1: 0.550 0.550 | K=4: 0.617 0.531 | K=8: 0.642 0.531
|
| 10 |
+
diff_s1 K=1: 0.492 0.492 | K=4: 0.617 0.537 | K=8: 0.625 0.531
|
| 11 |
+
diff_s2 K=1: 0.542 0.542 | K=4: 0.650 0.544 | K=8: 0.650 0.545
|
| 12 |
+
nn K=1: 0.375 0.375 | K=4: 0.600 0.398 | K=8: 0.642 0.401
|
| 13 |
+
idw K=1: 0.375 0.375 | K=4: 0.600 0.429 | K=8: 0.642 0.425
|
| 14 |
+
-- paired McNemar, model vs retrieval (model-only / retrieval-only, p); * = best retrieval arm
|
| 15 |
+
K=1 st=0 diff_s0: vs nn* 27-6 p=0.000324 diff +0.175 [+0.083,+0.267]; vs idw 27-6 p=0.000324 diff +0.175 [+0.083,+0.267];
|
| 16 |
+
K=1 st=0 diff_s1: vs nn* 25-12 p=0.047 diff +0.108 [+0.008,+0.200]; vs idw 25-12 p=0.047 diff +0.108 [+0.008,+0.200];
|
| 17 |
+
K=1 st=0 diff_s2: vs nn* 26-9 p=0.00599 diff +0.142 [+0.050,+0.233]; vs idw 26-9 p=0.00599 diff +0.142 [+0.050,+0.233];
|
| 18 |
+
K=1 st=1 diff_s0: vs nn* 24-3 p=4.92e-05 diff +0.175 [+0.100,+0.250]; vs idw 24-3 p=4.92e-05 diff +0.175 [+0.100,+0.250];
|
| 19 |
+
K=1 st=1 diff_s1: vs nn* 24-10 p=0.0243 diff +0.117 [+0.025,+0.208]; vs idw 24-10 p=0.0243 diff +0.117 [+0.025,+0.208];
|
| 20 |
+
K=1 st=1 diff_s2: vs nn* 27-7 p=0.000821 diff +0.167 [+0.075,+0.258]; vs idw 27-7 p=0.000821 diff +0.167 [+0.075,+0.258];
|
| 21 |
+
K=4 st=0 diff_s0: vs nn* 7-2 p=0.18 diff +0.042 [-0.008,+0.092]; vs idw 8-3 p=0.227 diff +0.042 [-0.008,+0.100];
|
| 22 |
+
K=4 st=0 diff_s1: vs nn* 8-1 p=0.0391 diff +0.058 [+0.017,+0.108]; vs idw 8-1 p=0.0391 diff +0.058 [+0.008,+0.108];
|
| 23 |
+
K=4 st=0 diff_s2: vs nn* 5-1 p=0.219 diff +0.033 [-0.008,+0.075]; vs idw 5-1 p=0.219 diff +0.033 [+0.000,+0.075];
|
| 24 |
+
K=4 st=1 diff_s0: vs nn* 3-1 p=0.625 diff +0.017 [-0.017,+0.050]; vs idw 3-1 p=0.625 diff +0.017 [-0.017,+0.050];
|
| 25 |
+
K=4 st=1 diff_s1: vs nn* 4-2 p=0.688 diff +0.017 [-0.025,+0.058]; vs idw 3-1 p=0.625 diff +0.017 [-0.017,+0.050];
|
| 26 |
+
K=4 st=1 diff_s2: vs nn* 7-1 p=0.0703 diff +0.050 [+0.008,+0.092]; vs idw 6-0 p=0.0312 diff +0.050 [+0.017,+0.092];
|
| 27 |
+
K=8 st=0 diff_s0: vs nn 4-1 p=0.375 diff +0.025 [-0.008,+0.067]; vs idw* 4-2 p=0.688 diff +0.017 [-0.025,+0.058];
|
| 28 |
+
K=8 st=0 diff_s1: vs nn 6-0 p=0.0312 diff +0.050 [+0.017,+0.092]; vs idw* 5-0 p=0.0625 diff +0.042 [+0.008,+0.075];
|
| 29 |
+
K=8 st=0 diff_s2: vs nn 4-1 p=0.375 diff +0.025 [-0.008,+0.067]; vs idw* 4-2 p=0.688 diff +0.017 [-0.025,+0.058];
|
| 30 |
+
K=8 st=1 diff_s0: vs nn* 1-1 p=1 diff +0.000 [-0.025,+0.025]; vs idw 1-1 p=1 diff +0.000 [-0.025,+0.025];
|
| 31 |
+
K=8 st=1 diff_s1: vs nn* 1-3 p=0.625 diff -0.017 [-0.050,+0.017]; vs idw 1-3 p=0.625 diff -0.017 [-0.050,+0.017];
|
| 32 |
+
K=8 st=1 diff_s2: vs nn* 2-1 p=1 diff +0.008 [-0.017,+0.042]; vs idw 2-1 p=1 diff +0.008 [-0.017,+0.042];
|
| 33 |
+
|
| 34 |
+
=== SISO lvl1.1: N=120, z-NN ratio test/bank 2.0x (median-based 0.499) ===
|
| 35 |
+
-- no steering: coverage (any of K stable) | per-candidate stable rate
|
| 36 |
+
diff_s0 K=1: 0.567 0.567 | K=4: 0.692 0.577 | K=8: 0.717 0.589
|
| 37 |
+
diff_s1 K=1: 0.550 0.550 | K=4: 0.675 0.550 | K=8: 0.692 0.572
|
| 38 |
+
diff_s2 K=1: 0.600 0.600 | K=4: 0.692 0.579 | K=8: 0.708 0.589
|
| 39 |
+
nn K=1: 0.542 0.542 | K=4: 0.675 0.521 | K=8: 0.683 0.502
|
| 40 |
+
idw K=1: 0.542 0.542 | K=4: 0.667 0.544 | K=8: 0.675 0.517
|
| 41 |
+
-- 50-step steering: coverage (any of K stable) | per-candidate stable rate
|
| 42 |
+
diff_s0 K=1: 0.592 0.592 | K=4: 0.683 0.596 | K=8: 0.700 0.601
|
| 43 |
+
diff_s1 K=1: 0.567 0.567 | K=4: 0.675 0.575 | K=8: 0.683 0.592
|
| 44 |
+
diff_s2 K=1: 0.600 0.600 | K=4: 0.708 0.590 | K=8: 0.717 0.594
|
| 45 |
+
nn K=1: 0.550 0.550 | K=4: 0.683 0.529 | K=8: 0.683 0.508
|
| 46 |
+
idw K=1: 0.550 0.550 | K=4: 0.675 0.550 | K=8: 0.675 0.523
|
| 47 |
+
-- paired McNemar, model vs retrieval (model-only / retrieval-only, p); * = best retrieval arm
|
| 48 |
+
K=1 st=0 diff_s0: vs nn* 15-12 p=0.701 diff +0.025 [-0.058,+0.108]; vs idw 15-12 p=0.701 diff +0.025 [-0.058,+0.108];
|
| 49 |
+
K=1 st=0 diff_s1: vs nn* 12-11 p=1 diff +0.008 [-0.067,+0.092]; vs idw 12-11 p=1 diff +0.008 [-0.067,+0.092];
|
| 50 |
+
K=1 st=0 diff_s2: vs nn* 16-9 p=0.23 diff +0.058 [-0.025,+0.142]; vs idw 16-9 p=0.23 diff +0.058 [-0.025,+0.142];
|
| 51 |
+
K=1 st=1 diff_s0: vs nn* 14-9 p=0.405 diff +0.042 [-0.042,+0.117]; vs idw 14-9 p=0.405 diff +0.042 [-0.042,+0.117];
|
| 52 |
+
K=1 st=1 diff_s1: vs nn* 11-9 p=0.824 diff +0.017 [-0.058,+0.092]; vs idw 11-9 p=0.824 diff +0.017 [-0.058,+0.092];
|
| 53 |
+
K=1 st=1 diff_s2: vs nn* 16-10 p=0.327 diff +0.050 [-0.033,+0.133]; vs idw 16-10 p=0.327 diff +0.050 [-0.033,+0.133];
|
| 54 |
+
K=4 st=0 diff_s0: vs nn* 2-0 p=0.5 diff +0.017 [+0.000,+0.042]; vs idw 3-0 p=0.25 diff +0.025 [+0.000,+0.058];
|
| 55 |
+
K=4 st=0 diff_s1: vs nn* 2-2 p=1 diff +0.000 [-0.033,+0.033]; vs idw 3-2 p=1 diff +0.008 [-0.025,+0.042];
|
| 56 |
+
K=4 st=0 diff_s2: vs nn* 3-1 p=0.625 diff +0.017 [-0.017,+0.050]; vs idw 4-1 p=0.375 diff +0.025 [-0.008,+0.067];
|
| 57 |
+
K=4 st=1 diff_s0: vs nn* 1-1 p=1 diff +0.000 [-0.025,+0.025]; vs idw 2-1 p=1 diff +0.008 [-0.017,+0.042];
|
| 58 |
+
K=4 st=1 diff_s1: vs nn* 0-1 p=1 diff -0.008 [-0.025,+0.000]; vs idw 1-1 p=1 diff +0.000 [-0.025,+0.025];
|
| 59 |
+
K=4 st=1 diff_s2: vs nn* 4-1 p=0.375 diff +0.025 [-0.008,+0.067]; vs idw 5-1 p=0.219 diff +0.033 [-0.008,+0.075];
|
| 60 |
+
K=8 st=0 diff_s0: vs nn* 4-0 p=0.125 diff +0.033 [+0.008,+0.067]; vs idw 5-0 p=0.0625 diff +0.042 [+0.008,+0.083];
|
| 61 |
+
K=8 st=0 diff_s1: vs nn* 3-2 p=1 diff +0.008 [-0.025,+0.042]; vs idw 3-1 p=0.625 diff +0.017 [-0.017,+0.050];
|
| 62 |
+
K=8 st=0 diff_s2: vs nn* 3-0 p=0.25 diff +0.025 [+0.000,+0.058]; vs idw 4-0 p=0.125 diff +0.033 [+0.008,+0.067];
|
| 63 |
+
K=8 st=1 diff_s0: vs nn* 3-1 p=0.625 diff +0.017 [-0.017,+0.050]; vs idw 3-0 p=0.25 diff +0.025 [+0.000,+0.058];
|
| 64 |
+
K=8 st=1 diff_s1: vs nn* 1-1 p=1 diff +0.000 [-0.025,+0.025]; vs idw 2-1 p=1 diff +0.008 [-0.017,+0.042];
|
| 65 |
+
K=8 st=1 diff_s2: vs nn* 4-0 p=0.125 diff +0.033 [+0.008,+0.067]; vs idw 5-0 p=0.0625 diff +0.042 [+0.008,+0.083];
|
| 66 |
+
|
| 67 |
+
=== SISO lvl1.25: N=120, z-NN ratio test/bank 3.3x (median-based 0.592) ===
|
| 68 |
+
-- no steering: coverage (any of K stable) | per-candidate stable rate
|
| 69 |
+
diff_s0 K=1: 0.492 0.492 | K=4: 0.592 0.492 | K=8: 0.608 0.489
|
| 70 |
+
diff_s1 K=1: 0.508 0.508 | K=4: 0.583 0.481 | K=8: 0.583 0.482
|
| 71 |
+
diff_s2 K=1: 0.475 0.475 | K=4: 0.583 0.490 | K=8: 0.600 0.491
|
| 72 |
+
nn K=1: 0.458 0.458 | K=4: 0.575 0.429 | K=8: 0.575 0.431
|
| 73 |
+
idw K=1: 0.458 0.458 | K=4: 0.575 0.450 | K=8: 0.575 0.444
|
| 74 |
+
-- 50-step steering: coverage (any of K stable) | per-candidate stable rate
|
| 75 |
+
diff_s0 K=1: 0.517 0.517 | K=4: 0.583 0.510 | K=8: 0.583 0.508
|
| 76 |
+
diff_s1 K=1: 0.517 0.517 | K=4: 0.583 0.494 | K=8: 0.592 0.495
|
| 77 |
+
diff_s2 K=1: 0.492 0.492 | K=4: 0.583 0.502 | K=8: 0.600 0.505
|
| 78 |
+
nn K=1: 0.467 0.467 | K=4: 0.575 0.435 | K=8: 0.575 0.435
|
| 79 |
+
idw K=1: 0.467 0.467 | K=4: 0.575 0.454 | K=8: 0.575 0.447
|
| 80 |
+
-- paired McNemar, model vs retrieval (model-only / retrieval-only, p); * = best retrieval arm
|
| 81 |
+
K=1 st=0 diff_s0: vs nn* 12-8 p=0.503 diff +0.033 [-0.042,+0.108]; vs idw 12-8 p=0.503 diff +0.033 [-0.042,+0.108];
|
| 82 |
+
K=1 st=0 diff_s1: vs nn* 11-5 p=0.21 diff +0.050 [-0.017,+0.117]; vs idw 11-5 p=0.21 diff +0.050 [-0.017,+0.117];
|
| 83 |
+
K=1 st=0 diff_s2: vs nn* 12-10 p=0.832 diff +0.017 [-0.067,+0.092]; vs idw 12-10 p=0.832 diff +0.017 [-0.067,+0.092];
|
| 84 |
+
K=1 st=1 diff_s0: vs nn* 12-6 p=0.238 diff +0.050 [-0.017,+0.117]; vs idw 12-6 p=0.238 diff +0.050 [-0.017,+0.117];
|
| 85 |
+
K=1 st=1 diff_s1: vs nn* 11-5 p=0.21 diff +0.050 [-0.017,+0.117]; vs idw 11-5 p=0.21 diff +0.050 [-0.017,+0.117];
|
| 86 |
+
K=1 st=1 diff_s2: vs nn* 12-9 p=0.664 diff +0.025 [-0.050,+0.100]; vs idw 12-9 p=0.664 diff +0.025 [-0.050,+0.100];
|
| 87 |
+
K=4 st=0 diff_s0: vs nn* 2-0 p=0.5 diff +0.017 [+0.000,+0.042]; vs idw 2-0 p=0.5 diff +0.017 [+0.000,+0.042];
|
| 88 |
+
K=4 st=0 diff_s1: vs nn* 1-0 p=1 diff +0.008 [+0.000,+0.025]; vs idw 1-0 p=1 diff +0.008 [+0.000,+0.025];
|
| 89 |
+
K=4 st=0 diff_s2: vs nn* 1-0 p=1 diff +0.008 [+0.000,+0.025]; vs idw 1-0 p=1 diff +0.008 [+0.000,+0.025];
|
| 90 |
+
K=4 st=1 diff_s0: vs nn* 1-0 p=1 diff +0.008 [+0.000,+0.025]; vs idw 1-0 p=1 diff +0.008 [+0.000,+0.025];
|
| 91 |
+
K=4 st=1 diff_s1: vs nn* 1-0 p=1 diff +0.008 [+0.000,+0.025]; vs idw 1-0 p=1 diff +0.008 [+0.000,+0.025];
|
| 92 |
+
K=4 st=1 diff_s2: vs nn* 1-0 p=1 diff +0.008 [+0.000,+0.025]; vs idw 1-0 p=1 diff +0.008 [+0.000,+0.025];
|
| 93 |
+
K=8 st=0 diff_s0: vs nn* 4-0 p=0.125 diff +0.033 [+0.008,+0.067]; vs idw 4-0 p=0.125 diff +0.033 [+0.008,+0.067];
|
| 94 |
+
K=8 st=0 diff_s1: vs nn* 1-0 p=1 diff +0.008 [+0.000,+0.025]; vs idw 1-0 p=1 diff +0.008 [+0.000,+0.025];
|
| 95 |
+
K=8 st=0 diff_s2: vs nn* 3-0 p=0.25 diff +0.025 [+0.000,+0.058]; vs idw 3-0 p=0.25 diff +0.025 [+0.000,+0.058];
|
| 96 |
+
K=8 st=1 diff_s0: vs nn* 1-0 p=1 diff +0.008 [+0.000,+0.025]; vs idw 1-0 p=1 diff +0.008 [+0.000,+0.025];
|
| 97 |
+
K=8 st=1 diff_s1: vs nn* 2-0 p=0.5 diff +0.017 [+0.000,+0.042]; vs idw 2-0 p=0.5 diff +0.017 [+0.000,+0.042];
|
| 98 |
+
K=8 st=1 diff_s2: vs nn* 3-0 p=0.25 diff +0.025 [+0.000,+0.058]; vs idw 3-0 p=0.25 diff +0.025 [+0.000,+0.058];
|
| 99 |
+
|
| 100 |
+
=== SISO lvl1.5: N=120, z-NN ratio test/bank 8.3x (median-based 1.519) ===
|
| 101 |
+
-- no steering: coverage (any of K stable) | per-candidate stable rate
|
| 102 |
+
diff_s0 K=1: 0.533 0.533 | K=4: 0.675 0.517 | K=8: 0.700 0.518
|
| 103 |
+
diff_s1 K=1: 0.508 0.508 | K=4: 0.642 0.481 | K=8: 0.683 0.508
|
| 104 |
+
diff_s2 K=1: 0.517 0.517 | K=4: 0.675 0.502 | K=8: 0.692 0.506
|
| 105 |
+
nn K=1: 0.583 0.583 | K=4: 0.700 0.550 | K=8: 0.700 0.549
|
| 106 |
+
idw K=1: 0.583 0.583 | K=4: 0.700 0.575 | K=8: 0.700 0.559
|
| 107 |
+
-- 50-step steering: coverage (any of K stable) | per-candidate stable rate
|
| 108 |
+
diff_s0 K=1: 0.542 0.542 | K=4: 0.675 0.535 | K=8: 0.692 0.537
|
| 109 |
+
diff_s1 K=1: 0.508 0.508 | K=4: 0.642 0.506 | K=8: 0.675 0.528
|
| 110 |
+
diff_s2 K=1: 0.525 0.525 | K=4: 0.683 0.515 | K=8: 0.708 0.527
|
| 111 |
+
nn K=1: 0.592 0.592 | K=4: 0.692 0.550 | K=8: 0.692 0.550
|
| 112 |
+
idw K=1: 0.592 0.592 | K=4: 0.692 0.573 | K=8: 0.692 0.559
|
| 113 |
+
-- paired McNemar, model vs retrieval (model-only / retrieval-only, p); * = best retrieval arm
|
| 114 |
+
K=1 st=0 diff_s0: vs nn* 10-16 p=0.327 diff -0.050 [-0.133,+0.033]; vs idw 10-16 p=0.327 diff -0.050 [-0.133,+0.033];
|
| 115 |
+
K=1 st=0 diff_s1: vs nn* 7-16 p=0.0931 diff -0.075 [-0.150,+0.000]; vs idw 7-16 p=0.0931 diff -0.075 [-0.150,+0.000];
|
| 116 |
+
K=1 st=0 diff_s2: vs nn* 8-16 p=0.152 diff -0.067 [-0.142,+0.008]; vs idw 8-16 p=0.152 diff -0.067 [-0.142,+0.008];
|
| 117 |
+
K=1 st=1 diff_s0: vs nn* 7-13 p=0.263 diff -0.050 [-0.125,+0.017]; vs idw 7-13 p=0.263 diff -0.050 [-0.125,+0.017];
|
| 118 |
+
K=1 st=1 diff_s1: vs nn* 5-15 p=0.0414 diff -0.083 [-0.158,-0.017]; vs idw 5-15 p=0.0414 diff -0.083 [-0.158,-0.017];
|
| 119 |
+
K=1 st=1 diff_s2: vs nn* 8-16 p=0.152 diff -0.067 [-0.142,+0.008]; vs idw 8-16 p=0.152 diff -0.067 [-0.142,+0.008];
|
| 120 |
+
K=4 st=0 diff_s0: vs nn* 3-6 p=0.508 diff -0.025 [-0.075,+0.025]; vs idw 3-6 p=0.508 diff -0.025 [-0.075,+0.025];
|
| 121 |
+
K=4 st=0 diff_s1: vs nn* 1-8 p=0.0391 diff -0.058 [-0.108,-0.017]; vs idw 1-8 p=0.0391 diff -0.058 [-0.108,-0.017];
|
| 122 |
+
K=4 st=0 diff_s2: vs nn* 2-5 p=0.453 diff -0.025 [-0.067,+0.017]; vs idw 2-5 p=0.453 diff -0.025 [-0.067,+0.017];
|
| 123 |
+
K=4 st=1 diff_s0: vs nn* 2-4 p=0.688 diff -0.017 [-0.058,+0.025]; vs idw 2-4 p=0.688 diff -0.017 [-0.058,+0.025];
|
| 124 |
+
K=4 st=1 diff_s1: vs nn* 1-7 p=0.0703 diff -0.050 [-0.100,-0.008]; vs idw 1-7 p=0.0703 diff -0.050 [-0.100,-0.008];
|
| 125 |
+
K=4 st=1 diff_s2: vs nn* 2-3 p=1 diff -0.008 [-0.042,+0.025]; vs idw 2-3 p=1 diff -0.008 [-0.042,+0.025];
|
| 126 |
+
K=8 st=0 diff_s0: vs nn* 3-3 p=1 diff +0.000 [-0.042,+0.042]; vs idw 3-3 p=1 diff +0.000 [-0.042,+0.042];
|
| 127 |
+
K=8 st=0 diff_s1: vs nn* 3-5 p=0.727 diff -0.017 [-0.067,+0.025]; vs idw 3-5 p=0.727 diff -0.017 [-0.067,+0.025];
|
| 128 |
+
K=8 st=0 diff_s2: vs nn* 2-3 p=1 diff -0.008 [-0.050,+0.025]; vs idw 2-3 p=1 diff -0.008 [-0.050,+0.025];
|
| 129 |
+
K=8 st=1 diff_s0: vs nn* 2-2 p=1 diff +0.000 [-0.033,+0.033]; vs idw 2-2 p=1 diff +0.000 [-0.033,+0.033];
|
| 130 |
+
K=8 st=1 diff_s1: vs nn* 3-5 p=0.727 diff -0.017 [-0.067,+0.025]; vs idw 3-5 p=0.727 diff -0.017 [-0.067,+0.025];
|
| 131 |
+
K=8 st=1 diff_s2: vs nn* 4-2 p=0.688 diff +0.017 [-0.025,+0.058]; vs idw 4-2 p=0.688 diff +0.017 [-0.025,+0.058];
|
| 132 |
+
|
| 133 |
+
=== SISO lvl2.0: N=120, z-NN ratio test/bank 36.3x (median-based 6.683) ===
|
| 134 |
+
-- no steering: coverage (any of K stable) | per-candidate stable rate
|
| 135 |
+
diff_s0 K=1: 0.408 0.408 | K=4: 0.608 0.433 | K=8: 0.667 0.438
|
| 136 |
+
diff_s1 K=1: 0.433 0.433 | K=4: 0.592 0.435 | K=8: 0.642 0.453
|
| 137 |
+
diff_s2 K=1: 0.450 0.450 | K=4: 0.642 0.458 | K=8: 0.692 0.448
|
| 138 |
+
nn K=1: 0.792 0.792 | K=4: 0.850 0.744 | K=8: 0.850 0.717
|
| 139 |
+
idw K=1: 0.792 0.792 | K=4: 0.850 0.758 | K=8: 0.850 0.724
|
| 140 |
+
-- 50-step steering: coverage (any of K stable) | per-candidate stable rate
|
| 141 |
+
diff_s0 K=1: 0.442 0.442 | K=4: 0.608 0.446 | K=8: 0.658 0.456
|
| 142 |
+
diff_s1 K=1: 0.458 0.458 | K=4: 0.625 0.465 | K=8: 0.658 0.474
|
| 143 |
+
diff_s2 K=1: 0.458 0.458 | K=4: 0.650 0.471 | K=8: 0.708 0.463
|
| 144 |
+
nn K=1: 0.783 0.783 | K=4: 0.850 0.740 | K=8: 0.850 0.715
|
| 145 |
+
idw K=1: 0.783 0.783 | K=4: 0.850 0.756 | K=8: 0.850 0.723
|
| 146 |
+
-- paired McNemar, model vs retrieval (model-only / retrieval-only, p); * = best retrieval arm
|
| 147 |
+
K=1 st=0 diff_s0: vs nn* 3-49 p=1.04e-11 diff -0.383 [-0.483,-0.283]; vs idw 3-49 p=1.04e-11 diff -0.383 [-0.483,-0.283];
|
| 148 |
+
K=1 st=0 diff_s1: vs nn* 4-47 p=2.42e-10 diff -0.358 [-0.458,-0.258]; vs idw 4-47 p=2.42e-10 diff -0.358 [-0.458,-0.258];
|
| 149 |
+
K=1 st=0 diff_s2: vs nn* 4-45 p=8.23e-10 diff -0.342 [-0.442,-0.242]; vs idw 4-45 p=8.23e-10 diff -0.342 [-0.442,-0.242];
|
| 150 |
+
K=1 st=1 diff_s0: vs nn* 3-44 p=2.46e-10 diff -0.342 [-0.442,-0.250]; vs idw 3-44 p=2.46e-10 diff -0.342 [-0.442,-0.250];
|
| 151 |
+
K=1 st=1 diff_s1: vs nn* 5-44 p=7.6e-09 diff -0.325 [-0.425,-0.225]; vs idw 5-44 p=7.6e-09 diff -0.325 [-0.425,-0.225];
|
| 152 |
+
K=1 st=1 diff_s2: vs nn* 4-43 p=2.78e-09 diff -0.325 [-0.425,-0.225]; vs idw 4-43 p=2.78e-09 diff -0.325 [-0.425,-0.225];
|
| 153 |
+
K=4 st=0 diff_s0: vs nn* 1-30 p=2.98e-08 diff -0.242 [-0.325,-0.158]; vs idw 1-30 p=2.98e-08 diff -0.242 [-0.325,-0.158];
|
| 154 |
+
K=4 st=0 diff_s1: vs nn* 2-33 p=3.67e-08 diff -0.258 [-0.342,-0.175]; vs idw 2-33 p=3.67e-08 diff -0.258 [-0.342,-0.175];
|
| 155 |
+
K=4 st=0 diff_s2: vs nn* 1-26 p=4.17e-07 diff -0.208 [-0.283,-0.133]; vs idw 1-26 p=4.17e-07 diff -0.208 [-0.283,-0.133];
|
| 156 |
+
K=4 st=1 diff_s0: vs nn* 1-30 p=2.98e-08 diff -0.242 [-0.325,-0.158]; vs idw 1-30 p=2.98e-08 diff -0.242 [-0.325,-0.158];
|
| 157 |
+
K=4 st=1 diff_s1: vs nn* 3-30 p=1.4e-06 diff -0.225 [-0.308,-0.142]; vs idw 3-30 p=1.4e-06 diff -0.225 [-0.308,-0.142];
|
| 158 |
+
K=4 st=1 diff_s2: vs nn* 1-25 p=8.05e-07 diff -0.200 [-0.275,-0.125]; vs idw 1-25 p=8.05e-07 diff -0.200 [-0.275,-0.125];
|
| 159 |
+
K=8 st=0 diff_s0: vs nn* 1-23 p=2.98e-06 diff -0.183 [-0.258,-0.117]; vs idw 1-23 p=2.98e-06 diff -0.183 [-0.258,-0.117];
|
| 160 |
+
K=8 st=0 diff_s1: vs nn* 2-27 p=1.62e-06 diff -0.208 [-0.292,-0.125]; vs idw 2-27 p=1.62e-06 diff -0.208 [-0.292,-0.125];
|
| 161 |
+
K=8 st=0 diff_s2: vs nn* 1-20 p=2.1e-05 diff -0.158 [-0.233,-0.092]; vs idw 1-20 p=2.1e-05 diff -0.158 [-0.233,-0.092];
|
| 162 |
+
K=8 st=1 diff_s0: vs nn* 1-24 p=1.55e-06 diff -0.192 [-0.267,-0.117]; vs idw 1-24 p=1.55e-06 diff -0.192 [-0.267,-0.117];
|
| 163 |
+
K=8 st=1 diff_s1: vs nn* 3-26 p=1.52e-05 diff -0.192 [-0.275,-0.117]; vs idw 3-26 p=1.52e-05 diff -0.192 [-0.275,-0.117];
|
| 164 |
+
K=8 st=1 diff_s2: vs nn* 1-18 p=7.63e-05 diff -0.142 [-0.217,-0.075]; vs idw 1-18 p=7.63e-05 diff -0.142 [-0.217,-0.075];
|
| 165 |
+
|
results/ood/train_holes.sh
ADDED
|
@@ -0,0 +1,6 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
#!/bin/bash
|
| 2 |
+
cd /home/cercola/matte_server/DiffusionControl
|
| 3 |
+
export PEND_DATA=data/pendulum_dataset_hole20.pt OMP_NUM_THREADS=1 WANDB_MODE=disabled PYTHONPATH=. CUDA_VISIBLE_DEVICES=0
|
| 4 |
+
for s in 42 43 44; do sfx=""; [ $s != 42 ] && sfx="_s$s"
|
| 5 |
+
myenv/bin/python -u -m nonlinear.training.train_pendulum_ablation --tag ood_hole20$sfx --data_fraction 1.0 --seed $s --epochs 1000
|
| 6 |
+
done
|
results/pendulum_full/dp_bank_rank.txt
ADDED
|
@@ -0,0 +1,23 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
=== DP policy bank + rank study (torque-bounded pendulum) ===
|
| 2 |
+
300 plants, policy grid 60x61 = 3660 values
|
| 3 |
+
family: l[0.5,1.5] m[0.3,1.2] zeta~logU(0.01, 0.15) rho_u~U(0.35, 0.65)
|
| 4 |
+
|
| 5 |
+
variance explained: k=1:42.43% k=2:66.55% k=4:78.23% k=8:85.51% k=16:91.72% k=32:95.43% k=64:97.74%
|
| 6 |
+
|
| 7 |
+
rank scored by ROLLOUT COST vs each plant's own DP (MSE is not the metric that matters):
|
| 8 |
+
k var% med regret p90 max fell over
|
| 9 |
+
(reference: 25/25 held-out DP policies stabilise at this grid)
|
| 10 |
+
1 42.43% +9.52% +40.59% +71.57% 1/25
|
| 11 |
+
2 66.55% +9.69% +22.06% +36.00% 0/25
|
| 12 |
+
4 78.23% +4.33% +20.12% +32.76% 0/25
|
| 13 |
+
8 85.51% +3.43% +17.22% +39.13% 0/25
|
| 14 |
+
16 91.72% +2.07% +8.99% +15.06% 0/25
|
| 15 |
+
32 95.43% +0.46% +1.54% +7.44% 0/25
|
| 16 |
+
64 97.74% +0.16% +0.69% +0.82% 0/25
|
| 17 |
+
|
| 18 |
+
This picks the generation dimension only. It is NOT the test of whether a model is
|
| 19 |
+
worth training: a 256-solve interpolation table already reaches +0.1% median regret
|
| 20 |
+
against per-plant value iteration, so the median is settled. The open question is the
|
| 21 |
+
~8-13% tail where that table loses 10-25%, and it is answered by the matched-budget
|
| 22 |
+
comparison (same K candidates, same rollout verifier, both scored vs per-plant VI),
|
| 23 |
+
not by anything measurable here.
|
results/pendulum_full/dp_baseline.txt
ADDED
|
@@ -0,0 +1,25 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
=== DP (value iteration) vs the hand-designed two-stage architecture ===
|
| 2 |
+
DP assumes no architecture: it returns the globally optimal u*(theta,omega) under the
|
| 3 |
+
torque bound. The architecture restricts u to -phi(c1*theta + c2*omega) plus an energy
|
| 4 |
+
swing-up stage. Same cost, same RK4 loop, same 12 s horizon, same hanging IC.
|
| 5 |
+
|
| 6 |
+
plant rho_u J_DP J_arch J_fixed arch/DP fixed/DP t_DP
|
| 7 |
+
0 0.416 17.13 19.80 19.80 1.16x 1.16x 2.1s
|
| 8 |
+
48 0.521 13.02 14.02 14.05 1.08x 1.08x 2.1s
|
| 9 |
+
95 0.448 69.02 80.70 80.75 1.17x 1.17x 2.2s
|
| 10 |
+
143 0.625 73.23 107.91 109.09 1.47x 1.49x 2.2s
|
| 11 |
+
190 0.555 70.50 90.75 90.77 1.29x 1.29x 2.2s
|
| 12 |
+
239 0.556 14.64 15.73 15.76 1.07x 1.08x 2.2s
|
| 13 |
+
|
| 14 |
+
median J_arch / J_DP = 1.16x (how much the hand-designed architecture gives up vs optimal)
|
| 15 |
+
median J_fixed / J_DP = 1.16x
|
| 16 |
+
median DP solve time = 2.2 s per plant
|
| 17 |
+
|
| 18 |
+
Reading: if J_arch/J_DP is close to 1 the architecture is near-optimal and there is
|
| 19 |
+
nothing for a richer generated controller to win. If it is well above 1, the earlier
|
| 20 |
+
'+0.3% headroom' was a property of the restricted controller class, not of the family,
|
| 21 |
+
and the amortization target should be DP itself.
|
| 22 |
+
|
| 23 |
+
NOTE the controller classes differ: DP gives u(theta,omega) on the full state; the
|
| 24 |
+
spline gives u(c1*theta + c2*omega), a 1-D projection. Part of any gap is that
|
| 25 |
+
projection, which a generated 2-D feedback map could remove.
|
results/pendulum_full/dp_interp.txt
ADDED
|
@@ -0,0 +1,28 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
=== is the pendulum plant->controller map just INTERPOLABLE? ===
|
| 2 |
+
|
| 3 |
+
--- A. same (zeta, rho_u), different (l, m): do the policies coincide? ---
|
| 4 |
+
l=1.00 m=0.75: reference, J=42.284
|
| 5 |
+
l=0.60 m=1.10: max|du_hat|=1.6000 rms=0.2239 J=30.267 J/J_ref=0.716
|
| 6 |
+
l=1.40 m=0.35: max|du_hat|=1.7000 rms=0.2459 J=33.782 J/J_ref=0.799
|
| 7 |
+
(small max|du_hat| => the family really is 2-D in (zeta, rho_u))
|
| 8 |
+
|
| 9 |
+
--- B. 5x5 table of DP solves over (log zeta, rho_u), then interpolate ---
|
| 10 |
+
table built: 25 DP solves in 57 s (2.3 s each), reused for every plant
|
| 11 |
+
plant zeta rho_u J_ownDP J_interp ratio
|
| 12 |
+
55 0.0132 0.623 57.13 57.27 1.002x
|
| 13 |
+
207 0.0115 0.503 57.30 57.49 1.003x
|
| 14 |
+
190 0.0991 0.555 70.06 70.40 1.005x
|
| 15 |
+
236 0.1094 0.623 22.66 27.03 1.193x
|
| 16 |
+
42 0.0400 0.372 33.87 36.79 1.086x
|
| 17 |
+
20 0.0196 0.387 101.09 109.08 1.079x
|
| 18 |
+
139 0.0173 0.633 29.52 32.71 1.108x
|
| 19 |
+
189 0.0215 0.572 40.48 40.41 0.998x
|
| 20 |
+
|
| 21 |
+
interpolation regret vs per-plant DP: median +4.2% p95 +16.3% max +19.3%
|
| 22 |
+
(for scale: the hand-designed two-stage architecture is +16% median, +47% worst)
|
| 23 |
+
|
| 24 |
+
Reading: if interpolation is within a few percent, a lookup table amortizes this family
|
| 25 |
+
and a generative model adds nothing HERE — the map is smooth. The Furuta certified
|
| 26 |
+
hand-off map is not smooth (a gain weighting fails between two that work), so the same
|
| 27 |
+
test there should behave very differently. That contrast, not the size of the plant
|
| 28 |
+
family, is the argument for a learned model.
|
results/pendulum_full/dp_interp4.txt
ADDED
|
@@ -0,0 +1,19 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
=== FAIR interpolation baseline: full 4-D table over (l, m, zeta, rho_u) ===
|
| 2 |
+
4^4 = 256 DP solves in 620 s, reused for every plant
|
| 3 |
+
plant J_ownDP J_interp4 ratio
|
| 4 |
+
55 57.13 56.95 0.997x
|
| 5 |
+
207 57.30 57.15 0.997x
|
| 6 |
+
190 70.06 75.64 1.080x
|
| 7 |
+
236 22.66 27.02 1.192x
|
| 8 |
+
42 33.87 37.54 1.108x
|
| 9 |
+
20 101.09 100.40 0.993x
|
| 10 |
+
139 29.52 29.61 1.003x
|
| 11 |
+
189 40.48 40.46 0.999x
|
| 12 |
+
|
| 13 |
+
4-D interpolation regret vs per-plant DP: median +0.1% p95 +16.3% max +19.2%
|
| 14 |
+
for scale: 2-D table (wrong parameterisation) +4.2% med / +19.3% max
|
| 15 |
+
hand-designed two-stage architecture +16% med / +47% max
|
| 16 |
+
|
| 17 |
+
This is the number a generated feedback map has to beat. Note the table cost grows as
|
| 18 |
+
npts^4 (256 solves here); a learned model needs a comparable number of training
|
| 19 |
+
solves but no grid, which is the real argument for amortisation over tabulation.
|
results/pendulum_full/dp_interp_kappa.txt
ADDED
|
@@ -0,0 +1,57 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
=== does interpolation fail specifically at LOW kappa (multi-swing)? ===
|
| 2 |
+
kappa-stratified sample, n=40, 4-D table (256 DP solves) as the baseline
|
| 3 |
+
kappa zeta rho_u J_ownDP J_interp ratio
|
| 4 |
+
1.65 0.1420 0.468 69.42 66.64 0.960x
|
| 5 |
+
1.96 0.0991 0.388 64.14 79.69 1.242x
|
| 6 |
+
1.72 0.1060 0.364 109.63 115.87 1.057x (FELL OVER)
|
| 7 |
+
1.92 0.1429 0.549 34.12 34.49 1.011x
|
| 8 |
+
2.56 0.1060 0.542 23.65 24.68 1.043x
|
| 9 |
+
2.52 0.1281 0.646 49.02 51.28 1.046x
|
| 10 |
+
2.80 0.0931 0.521 16.13 17.05 1.057x
|
| 11 |
+
2.14 0.1428 0.611 25.81 26.43 1.024x
|
| 12 |
+
2.82 0.0688 0.388 51.33 52.02 1.013x
|
| 13 |
+
4.11 0.0559 0.460 34.43 34.17 0.992x
|
| 14 |
+
3.42 0.0647 0.443 26.22 26.32 1.004x
|
| 15 |
+
3.19 0.0829 0.529 85.73 86.46 1.008x
|
| 16 |
+
3.54 0.0652 0.462 31.02 31.06 1.001x
|
| 17 |
+
3.68 0.0715 0.527 35.10 35.51 1.012x
|
| 18 |
+
4.73 0.0389 0.368 20.77 24.88 1.198x
|
| 19 |
+
5.60 0.0562 0.629 67.07 70.33 1.049x
|
| 20 |
+
5.59 0.0331 0.371 47.45 47.61 1.003x
|
| 21 |
+
5.71 0.0442 0.505 41.56 41.58 1.001x
|
| 22 |
+
4.79 0.0381 0.365 54.08 54.58 1.009x
|
| 23 |
+
7.16 0.0248 0.356 19.87 20.15 1.014x
|
| 24 |
+
7.90 0.0301 0.476 45.58 45.57 1.000x
|
| 25 |
+
7.72 0.0342 0.528 51.97 51.90 0.999x
|
| 26 |
+
8.16 0.0337 0.550 37.03 37.90 1.024x
|
| 27 |
+
7.14 0.0411 0.587 31.26 31.66 1.013x
|
| 28 |
+
12.30 0.0264 0.649 83.20 82.87 0.996x
|
| 29 |
+
11.82 0.0255 0.603 36.63 36.60 0.999x
|
| 30 |
+
12.89 0.0235 0.606 41.91 42.25 1.008x
|
| 31 |
+
9.53 0.0214 0.408 58.48 58.38 0.998x
|
| 32 |
+
10.73 0.0185 0.397 25.14 25.48 1.014x
|
| 33 |
+
17.52 0.0105 0.367 46.72 46.70 1.000x
|
| 34 |
+
14.18 0.0222 0.629 104.14 103.26 0.992x
|
| 35 |
+
18.52 0.0139 0.516 31.70 34.99 1.104x
|
| 36 |
+
15.75 0.0200 0.630 20.79 20.67 0.994x
|
| 37 |
+
13.58 0.0217 0.589 40.05 40.05 1.000x
|
| 38 |
+
19.78 0.0164 0.647 75.17 75.86 1.009x
|
| 39 |
+
26.83 0.0109 0.585 16.95 16.98 1.001x
|
| 40 |
+
26.33 0.0102 0.537 106.99 105.82 0.989x
|
| 41 |
+
29.19 0.0100 0.585 20.81 20.91 1.005x
|
| 42 |
+
21.07 0.0118 0.495 32.34 32.36 1.001x
|
| 43 |
+
|
| 44 |
+
kappa in [ 1.3, 2.0): n= 4 median regret +3.4% max +24.2%
|
| 45 |
+
kappa in [ 2.0, 3.0): n= 5 median regret +4.3% max +5.7%
|
| 46 |
+
kappa in [ 3.0, 4.5): n= 5 median regret +0.4% max +1.2%
|
| 47 |
+
kappa in [ 4.5, 6.5): n= 5 median regret +0.9% max +19.8%
|
| 48 |
+
kappa in [ 6.5, 9.0): n= 5 median regret +1.3% max +2.4%
|
| 49 |
+
kappa in [ 9.0, 13.0): n= 5 median regret -0.1% max +1.4%
|
| 50 |
+
kappa in [ 13.0, 19.0): n= 5 median regret -0.0% max +10.4%
|
| 51 |
+
kappa in [ 19.0, 30.0): n= 5 median regret +0.1% max +0.9%
|
| 52 |
+
|
| 53 |
+
kappa < 5 : n=16 median +1.3% max +24.2%
|
| 54 |
+
kappa >= 9 : n=15 median -0.0% max +10.4%
|
| 55 |
+
|
| 56 |
+
If the low-kappa band really is where tabulation breaks down, THAT is the target for a
|
| 57 |
+
generated policy — and it is the same non-smooth-map failure as the Furuta gain holes.
|
results/pendulum_full/dp_target_eval.txt
ADDED
|
@@ -0,0 +1,24 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
=== DP-target eval, matched budget (N=40, 40 with a valid VI ref) ===
|
| 2 |
+
ceiling: value iteration, 4.20 s/plant
|
| 3 |
+
|
| 4 |
+
--- K = 1 candidate(s) per plant, identical for every arm ---
|
| 5 |
+
generated (K=1) med +2.03% p90 +11.78% max +24.83% >10%: 15.4% fail 2.5% [33 ms]
|
| 6 |
+
grid table +verifier (K=1) med +8.25% p90 +24.57% max +49.17% >10%: 40.0% fail 25.0%
|
| 7 |
+
nearest neighbour (K=1) med +0.20% p90 +2.55% max +24.53% >10%: 2.6% fail 2.5%
|
| 8 |
+
paired vs grid table generated better on 22, worse on 7 -> p = 0.0081 *
|
| 9 |
+
paired vs nearest nb generated better on 7, worse on 32 -> p = 0.0001 *
|
| 10 |
+
|
| 11 |
+
--- K = 4 candidate(s) per plant, identical for every arm ---
|
| 12 |
+
generated (K=4) med +0.87% p90 +9.97% max +15.43% >10%: 10.3% fail 2.5% [64 ms]
|
| 13 |
+
grid table +verifier (K=4) med +2.67% p90 +13.67% max +29.92% >10%: 14.3% fail 12.5%
|
| 14 |
+
nearest neighbour (K=4) med -0.20% p90 +0.46% max +19.46% >10%: 2.5% fail 0.0%
|
| 15 |
+
paired vs grid table generated better on 20, worse on 14 -> p = 0.3915
|
| 16 |
+
paired vs nearest nb generated better on 9, worse on 30 -> p = 0.0011 *
|
| 17 |
+
|
| 18 |
+
--- K = 8 candidate(s) per plant, identical for every arm ---
|
| 19 |
+
generated (K=8) med +0.34% p90 +2.52% max +23.42% >10%: 5.1% fail 2.5% [94 ms]
|
| 20 |
+
grid table +verifier (K=8) med +0.56% p90 +8.33% max +56.93% >10%: 10.0% fail 0.0%
|
| 21 |
+
nearest neighbour (K=8) med -1.17% p90 +0.26% max +0.90% >10%: 0.0% fail 0.0%
|
| 22 |
+
paired vs grid table generated better on 22, worse on 17 -> p = 0.5224
|
| 23 |
+
paired vs nearest nb generated better on 7, worse on 32 -> p = 0.0001 *
|
| 24 |
+
|
results/pendulum_full/family_sweep.txt
ADDED
|
@@ -0,0 +1,60 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
--- 1. analytic swing-up feasibility (kappa = rho_u/(2 zeta) > 1.3614) ---
|
| 2 |
+
zeta~logU[0.01,0.25], rho~U[.35,1.2] feasible 96.4% kappa med 7.31 zeta med 0.0501 frac kappa<3 (multi-swing) 22.3%
|
| 3 |
+
zeta~logU[0.02,0.30], rho~U[.35,0.9] feasible 88.6% kappa med 3.89 zeta med 0.0778 frac kappa<3 (multi-swing) 40.3%
|
| 4 |
+
zeta~logU[0.01,0.20], rho~U[.40,1.2] feasible 99.1% kappa med 8.52 zeta med 0.0449 frac kappa<3 (multi-swing) 15.1%
|
| 5 |
+
LEGACY b~U[0.02,0.3], rho=0.4 feasible 96.3% kappa med 5.83 zeta med 0.0343 frac kappa<3 (multi-swing) 22.0%
|
| 6 |
+
LEGACY b~U[0.02,0.3], rho=0.6 feasible 99.1% kappa med 8.75 zeta med 0.0343 frac kappa<3 (multi-swing) 10.0%
|
| 7 |
+
|
| 8 |
+
--- 2. chosen family: zeta~logU(0.01, 0.15), rho_u~U(0.35, 0.65), N=240 ---
|
| 9 |
+
analytically feasible: 99.6%
|
| 10 |
+
kappa min 1.31 p5 1.78 med 6.46 p95 20.97 max 28.09
|
| 11 |
+
rho_u min 0.352 p5 0.365 med 0.489 p95 0.629 max 0.649
|
| 12 |
+
zeta min 0.0101 p5 0.0117 med 0.0368 p95 0.1329 max 0.1458
|
| 13 |
+
|
| 14 |
+
--- 3. certified saturation-free region (max-gamma SDP) ---
|
| 15 |
+
plants with a non-empty certified region: 100.0%
|
| 16 |
+
gamma (best over alpha_hat) min 0.125 p5 0.138 med 0.187 p95 0.254 max 0.278
|
| 17 |
+
gamma / rho_u min 0.329 p5 0.335 med 0.394 p95 0.435 max 0.440
|
| 18 |
+
gamma by alpha_hat (median): a=1.2:0.187 a=1.8:0.124 a=2.5:0.090 a=3.5:0.064 a=5.0:0.045 a=7.0:0.032 a=10.0:0.022
|
| 19 |
+
|
| 20 |
+
--- 4. end-to-end keep rate (certified AND full-cycle AND never saturating) ---
|
| 21 |
+
kept: 98.8% (237/240)
|
| 22 |
+
of the analytically-feasible ones: 99.2%
|
| 23 |
+
J = int(q)+int(u^2) (kept) min 12.31 p5 17.60 med 42.40 p95 149.83 max 350.17
|
| 24 |
+
t_settle (kept) min 2.320 p5 2.569 med 3.905 p95 6.704 max 10.385
|
| 25 |
+
|
| 26 |
+
selected by ABSOLUTE-WEIGHT COST J (the §4.7 convention):
|
| 27 |
+
alpha_hat: 1.2:24 1.8:144 2.5:54 3.5:14 5:1 7:0 10:0
|
| 28 |
+
ke_mult : 5:0 10:35 20:73 30:56 60:55 120:18 300:0 1000:0
|
| 29 |
+
catch box: 0.03:0 0.05:8 0.08:33 0.12:81 0.18:97 0.25:18
|
| 30 |
+
selected by SCALE-INVARIANT t_settle (contrast):
|
| 31 |
+
alpha_hat: 1.2:105 1.8:16 2.5:14 3.5:25 5:57 7:20 10:0
|
| 32 |
+
ke_mult : 5:0 10:0 20:0 30:0 60:0 120:0 300:9 1000:228
|
| 33 |
+
catch box: 0.03:194 0.05:11 0.08:0 0.12:5 0.18:17 0.25:10
|
| 34 |
+
the two rules pick the SAME config on 0.0% of kept plants
|
| 35 |
+
|
| 36 |
+
--- 5. NUISANCE BASELINE A: the BEST single fixed controller, no per-plant solve ---
|
| 37 |
+
tuned over the same grid -> alpha_hat=1.8, ke_mult=120, catch box (0.08, 0.08*wn)
|
| 38 |
+
full-cycle success without saturating: 98.8%
|
| 39 |
+
AND handing off inside the certified set: 98.8% (vs 98.8% for the per-plant pipeline)
|
| 40 |
+
COST regret vs per-plant % min +0.0 p5 +0.0 med +0.3 p95 +1.1 max +2.2
|
| 41 |
+
fraction of kept plants where the fixed config IS the per-plant argmin: 0.0%
|
| 42 |
+
best 5 fixed configs by certified success:
|
| 43 |
+
a=1.8 ke=120 box=0.08 success 98.8% certified 98.8%
|
| 44 |
+
a=1.2 ke=120 box=0.08 success 98.8% certified 98.8%
|
| 45 |
+
a=1.2 ke=120 box=0.12 success 98.8% certified 98.8%
|
| 46 |
+
a=2.5 ke=300 box=0.03 success 98.8% certified 98.8%
|
| 47 |
+
a=1.8 ke=300 box=0.03 success 98.8% certified 98.8%
|
| 48 |
+
-> a tie on success AND certification AND cost means nothing is being amortized.
|
| 49 |
+
|
| 50 |
+
--- 6. NUISANCE BASELINE B: static sector spline alone, launched from hanging ---
|
| 51 |
+
(no swing-up stage; predicted to succeed once rho_u > 2/pi = 0.637)
|
| 52 |
+
rho_u in [0.35,0.50): static-only success 0.0% (n=126)
|
| 53 |
+
rho_u in [0.50,0.65): static-only success 0.0% (n=114)
|
| 54 |
+
|
| 55 |
+
--- 7. spline grid width: how far does the loop actually travel in y? ---
|
| 56 |
+
max|y| during balance min 0.0001 p5 0.0014 med 0.0100 p95 0.0528 max 0.2296
|
| 57 |
+
certified radius r min 0.0352 p5 0.0450 med 0.1396 p95 0.4406 max 0.5412
|
| 58 |
+
grid spans [-0.60,0.60], spacing 0.120
|
| 59 |
+
a grid whose spacing exceeds the travelled |y| leaves most control points
|
| 60 |
+
unexercised on the certified set (they then matter only OUTSIDE it).
|
results/refine_reference/refine_reference.png
ADDED
|
results/timing/classical_mimo_cpu1.json
ADDED
|
@@ -0,0 +1,253 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"machine": {
|
| 3 |
+
"host": "virgo-asl",
|
| 4 |
+
"python": "3.13.13",
|
| 5 |
+
"torch": "2.10.0+cu128",
|
| 6 |
+
"numpy": "2.2.6",
|
| 7 |
+
"scipy": "1.18.0",
|
| 8 |
+
"cvxpy": "1.9.2",
|
| 9 |
+
"diffusers": "0.38.0",
|
| 10 |
+
"threads": 1,
|
| 11 |
+
"tag": "cpu1",
|
| 12 |
+
"load1": 17.33642578125,
|
| 13 |
+
"date": "2026-09-30 07:20:14 PDT",
|
| 14 |
+
"cpu": "AMD Ryzen Threadripper 7960X 24-Cores",
|
| 15 |
+
"affinity": [
|
| 16 |
+
21
|
| 17 |
+
]
|
| 18 |
+
},
|
| 19 |
+
"family": "mimo",
|
| 20 |
+
"cells": {
|
| 21 |
+
"lmi/circle_beta_iters5/b1": {
|
| 22 |
+
"n": 200,
|
| 23 |
+
"median_ms": 68.87386087328196,
|
| 24 |
+
"p99_ms": 3816.0280430503167,
|
| 25 |
+
"mean_ms": 546.3023953093216,
|
| 26 |
+
"min_ms": 36.024462431669235,
|
| 27 |
+
"max_ms": 4755.798971280456,
|
| 28 |
+
"note": "certified_beta(bmax=8, iters=5): bisection of the decentralised circle-criterion LMI; controller = linear gain 0.3*(0.7*beta) (the oracle's own init) => synthesis+certificate, per plant",
|
| 29 |
+
"samples_ms": [
|
| 30 |
+
1539.46105,
|
| 31 |
+
84.89908,
|
| 32 |
+
58.99341,
|
| 33 |
+
158.97685,
|
| 34 |
+
57.06773,
|
| 35 |
+
46.85211,
|
| 36 |
+
52.05384,
|
| 37 |
+
2038.34223,
|
| 38 |
+
317.71592,
|
| 39 |
+
46.14167,
|
| 40 |
+
47.39384,
|
| 41 |
+
46.71565,
|
| 42 |
+
86.24514,
|
| 43 |
+
2685.7059,
|
| 44 |
+
3813.63335,
|
| 45 |
+
372.45745,
|
| 46 |
+
55.60401,
|
| 47 |
+
89.29427,
|
| 48 |
+
53.32176,
|
| 49 |
+
711.88774,
|
| 50 |
+
3314.75986,
|
| 51 |
+
55.28606,
|
| 52 |
+
53.7078,
|
| 53 |
+
46.19153,
|
| 54 |
+
66.45013,
|
| 55 |
+
1330.51316,
|
| 56 |
+
56.31458,
|
| 57 |
+
61.06931,
|
| 58 |
+
2636.02433,
|
| 59 |
+
90.99745,
|
| 60 |
+
42.93542,
|
| 61 |
+
44.30927,
|
| 62 |
+
1233.37236,
|
| 63 |
+
49.36498,
|
| 64 |
+
852.99159,
|
| 65 |
+
102.28822,
|
| 66 |
+
2655.41108,
|
| 67 |
+
54.04861,
|
| 68 |
+
82.58464,
|
| 69 |
+
61.15766,
|
| 70 |
+
1476.64616,
|
| 71 |
+
2740.09727,
|
| 72 |
+
238.39509,
|
| 73 |
+
49.03152,
|
| 74 |
+
100.22615,
|
| 75 |
+
52.01367,
|
| 76 |
+
51.79844,
|
| 77 |
+
2763.47313,
|
| 78 |
+
273.39254,
|
| 79 |
+
1506.32314,
|
| 80 |
+
52.71514,
|
| 81 |
+
84.49058,
|
| 82 |
+
54.64163,
|
| 83 |
+
195.18373,
|
| 84 |
+
1010.33022,
|
| 85 |
+
107.77339,
|
| 86 |
+
162.07648,
|
| 87 |
+
2714.82362,
|
| 88 |
+
789.56922,
|
| 89 |
+
70.2019,
|
| 90 |
+
71.74916,
|
| 91 |
+
50.32908,
|
| 92 |
+
63.23676,
|
| 93 |
+
54.94314,
|
| 94 |
+
48.97369,
|
| 95 |
+
63.71007,
|
| 96 |
+
57.70152,
|
| 97 |
+
46.84208,
|
| 98 |
+
1641.44114,
|
| 99 |
+
65.3701,
|
| 100 |
+
1499.00568,
|
| 101 |
+
1583.04308,
|
| 102 |
+
155.7974,
|
| 103 |
+
48.37973,
|
| 104 |
+
57.47,
|
| 105 |
+
2099.01559,
|
| 106 |
+
44.31368,
|
| 107 |
+
409.94296,
|
| 108 |
+
61.88901,
|
| 109 |
+
51.12013,
|
| 110 |
+
46.062,
|
| 111 |
+
70.53595,
|
| 112 |
+
516.58963,
|
| 113 |
+
45.07162,
|
| 114 |
+
1932.52514,
|
| 115 |
+
72.99564,
|
| 116 |
+
980.11014,
|
| 117 |
+
57.26145,
|
| 118 |
+
63.32942,
|
| 119 |
+
45.80935,
|
| 120 |
+
70.64325,
|
| 121 |
+
2655.0548,
|
| 122 |
+
60.55124,
|
| 123 |
+
1398.9475,
|
| 124 |
+
54.07696,
|
| 125 |
+
50.63864,
|
| 126 |
+
1385.55271,
|
| 127 |
+
4053.10305,
|
| 128 |
+
62.20255,
|
| 129 |
+
49.0091,
|
| 130 |
+
47.3842,
|
| 131 |
+
467.22038,
|
| 132 |
+
50.84759,
|
| 133 |
+
1313.331,
|
| 134 |
+
172.51838,
|
| 135 |
+
68.47264,
|
| 136 |
+
55.15518,
|
| 137 |
+
52.60684,
|
| 138 |
+
41.62144,
|
| 139 |
+
89.66991,
|
| 140 |
+
47.79533,
|
| 141 |
+
47.35295,
|
| 142 |
+
58.36662,
|
| 143 |
+
42.34562,
|
| 144 |
+
3054.16087,
|
| 145 |
+
1367.51997,
|
| 146 |
+
57.07521,
|
| 147 |
+
110.66511,
|
| 148 |
+
60.31887,
|
| 149 |
+
47.43358,
|
| 150 |
+
56.15808,
|
| 151 |
+
2782.96257,
|
| 152 |
+
73.97002,
|
| 153 |
+
572.89503,
|
| 154 |
+
68.09734,
|
| 155 |
+
168.34665,
|
| 156 |
+
143.64995,
|
| 157 |
+
91.76317,
|
| 158 |
+
46.70815,
|
| 159 |
+
2656.82335,
|
| 160 |
+
44.81606,
|
| 161 |
+
113.42168,
|
| 162 |
+
59.43455,
|
| 163 |
+
53.72118,
|
| 164 |
+
533.67817,
|
| 165 |
+
44.59412,
|
| 166 |
+
44.90975,
|
| 167 |
+
2548.15145,
|
| 168 |
+
60.62354,
|
| 169 |
+
2613.68835,
|
| 170 |
+
41.80415,
|
| 171 |
+
58.57468,
|
| 172 |
+
3097.16299,
|
| 173 |
+
47.00665,
|
| 174 |
+
102.20869,
|
| 175 |
+
55.85084,
|
| 176 |
+
71.33744,
|
| 177 |
+
47.38481,
|
| 178 |
+
409.22454,
|
| 179 |
+
1329.93732,
|
| 180 |
+
1342.6898,
|
| 181 |
+
65.32284,
|
| 182 |
+
107.73836,
|
| 183 |
+
59.92452,
|
| 184 |
+
180.91818,
|
| 185 |
+
2805.83807,
|
| 186 |
+
66.97288,
|
| 187 |
+
1850.27357,
|
| 188 |
+
48.42656,
|
| 189 |
+
62.97545,
|
| 190 |
+
56.19333,
|
| 191 |
+
52.17917,
|
| 192 |
+
4755.79897,
|
| 193 |
+
36.02446,
|
| 194 |
+
40.4797,
|
| 195 |
+
45.0965,
|
| 196 |
+
43.91727,
|
| 197 |
+
139.24127,
|
| 198 |
+
42.6585,
|
| 199 |
+
259.45502,
|
| 200 |
+
588.53512,
|
| 201 |
+
43.94901,
|
| 202 |
+
77.34191,
|
| 203 |
+
152.85974,
|
| 204 |
+
38.38812,
|
| 205 |
+
94.10778,
|
| 206 |
+
56.41848,
|
| 207 |
+
1435.67258,
|
| 208 |
+
58.60595,
|
| 209 |
+
182.33116,
|
| 210 |
+
48.6597,
|
| 211 |
+
50.30897,
|
| 212 |
+
43.92796,
|
| 213 |
+
570.07636,
|
| 214 |
+
69.27508,
|
| 215 |
+
47.14419,
|
| 216 |
+
43.0231,
|
| 217 |
+
2290.25402,
|
| 218 |
+
55.12404,
|
| 219 |
+
50.66497,
|
| 220 |
+
62.87157,
|
| 221 |
+
1245.61535,
|
| 222 |
+
158.08528,
|
| 223 |
+
69.43793,
|
| 224 |
+
454.00061,
|
| 225 |
+
69.28398,
|
| 226 |
+
682.18592,
|
| 227 |
+
1371.39834,
|
| 228 |
+
49.64081,
|
| 229 |
+
158.31147
|
| 230 |
+
]
|
| 231 |
+
},
|
| 232 |
+
"lmi/circle_beta_iters5/batch120": {
|
| 233 |
+
"n": 2,
|
| 234 |
+
"median_ms": 540.0699076397965,
|
| 235 |
+
"p99_ms": 570.1727172235647,
|
| 236 |
+
"mean_ms": 540.0699076397965,
|
| 237 |
+
"min_ms": 509.35275500329834,
|
| 238 |
+
"max_ms": 570.7870602762947,
|
| 239 |
+
"note": "looped over batch=120 (cvxpy solves are per plant); ms per plant; n = batches",
|
| 240 |
+
"samples_ms": [
|
| 241 |
+
570.78706,
|
| 242 |
+
509.35276
|
| 243 |
+
]
|
| 244 |
+
},
|
| 245 |
+
"lmi/quality_heldout120": {
|
| 246 |
+
"coverage": 1.0,
|
| 247 |
+
"regret_median_pct": 13.914860725402832,
|
| 248 |
+
"regret_p90_pct": 43.86730766296388,
|
| 249 |
+
"note": "linear decentralised gain at 0.3*cap under the circle-criterion-certified sector (sound certificate), vs stored eval_mimo_regret oracle"
|
| 250 |
+
}
|
| 251 |
+
},
|
| 252 |
+
"done": true
|
| 253 |
+
}
|
results/timing/classical_mimo_cpu1q.json
ADDED
|
@@ -0,0 +1,253 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"machine": {
|
| 3 |
+
"host": "virgo-asl",
|
| 4 |
+
"python": "3.13.13",
|
| 5 |
+
"torch": "2.10.0+cu128",
|
| 6 |
+
"numpy": "2.2.6",
|
| 7 |
+
"scipy": "1.18.0",
|
| 8 |
+
"cvxpy": "1.9.2",
|
| 9 |
+
"diffusers": "0.38.0",
|
| 10 |
+
"threads": 1,
|
| 11 |
+
"tag": "cpu1q",
|
| 12 |
+
"load1": 8.916015625,
|
| 13 |
+
"date": "2026-09-30 09:11:48 PDT",
|
| 14 |
+
"cpu": "AMD Ryzen Threadripper 7960X 24-Cores",
|
| 15 |
+
"affinity": [
|
| 16 |
+
20
|
| 17 |
+
]
|
| 18 |
+
},
|
| 19 |
+
"family": "mimo",
|
| 20 |
+
"cells": {
|
| 21 |
+
"lmi/circle_beta_iters5/b1": {
|
| 22 |
+
"n": 200,
|
| 23 |
+
"median_ms": 66.19771895930171,
|
| 24 |
+
"p99_ms": 3756.5721647534497,
|
| 25 |
+
"mean_ms": 536.2993241753429,
|
| 26 |
+
"min_ms": 36.544194445014,
|
| 27 |
+
"max_ms": 4701.850468292832,
|
| 28 |
+
"note": "certified_beta(bmax=8, iters=5): bisection of the decentralised circle-criterion LMI; controller = linear gain 0.3*(0.7*beta) (the oracle's own init) => synthesis+certificate, per plant",
|
| 29 |
+
"samples_ms": [
|
| 30 |
+
1513.84927,
|
| 31 |
+
79.18051,
|
| 32 |
+
52.53494,
|
| 33 |
+
154.0505,
|
| 34 |
+
50.48537,
|
| 35 |
+
43.06555,
|
| 36 |
+
47.73189,
|
| 37 |
+
1990.77106,
|
| 38 |
+
308.27518,
|
| 39 |
+
39.86873,
|
| 40 |
+
39.18953,
|
| 41 |
+
39.90704,
|
| 42 |
+
80.71017,
|
| 43 |
+
2640.85822,
|
| 44 |
+
3754.09782,
|
| 45 |
+
362.3173,
|
| 46 |
+
50.45921,
|
| 47 |
+
85.18535,
|
| 48 |
+
48.75933,
|
| 49 |
+
703.31981,
|
| 50 |
+
3278.91677,
|
| 51 |
+
48.70152,
|
| 52 |
+
48.8602,
|
| 53 |
+
40.56209,
|
| 54 |
+
60.90773,
|
| 55 |
+
1314.93117,
|
| 56 |
+
53.80343,
|
| 57 |
+
56.41611,
|
| 58 |
+
2593.9615,
|
| 59 |
+
85.1521,
|
| 60 |
+
37.23508,
|
| 61 |
+
38.32452,
|
| 62 |
+
1214.554,
|
| 63 |
+
45.87818,
|
| 64 |
+
841.45585,
|
| 65 |
+
100.99499,
|
| 66 |
+
2616.62442,
|
| 67 |
+
48.40769,
|
| 68 |
+
76.61527,
|
| 69 |
+
54.58448,
|
| 70 |
+
1444.48981,
|
| 71 |
+
2700.64529,
|
| 72 |
+
230.40055,
|
| 73 |
+
43.38491,
|
| 74 |
+
97.00003,
|
| 75 |
+
47.4023,
|
| 76 |
+
47.15563,
|
| 77 |
+
2698.40246,
|
| 78 |
+
266.67435,
|
| 79 |
+
1473.26395,
|
| 80 |
+
47.04946,
|
| 81 |
+
73.90279,
|
| 82 |
+
51.1841,
|
| 83 |
+
189.5483,
|
| 84 |
+
989.51642,
|
| 85 |
+
102.48008,
|
| 86 |
+
159.59952,
|
| 87 |
+
2656.26725,
|
| 88 |
+
773.19128,
|
| 89 |
+
64.42876,
|
| 90 |
+
67.6078,
|
| 91 |
+
45.53384,
|
| 92 |
+
56.61694,
|
| 93 |
+
50.95768,
|
| 94 |
+
44.0698,
|
| 95 |
+
57.52367,
|
| 96 |
+
53.23725,
|
| 97 |
+
41.41171,
|
| 98 |
+
1607.05642,
|
| 99 |
+
61.50824,
|
| 100 |
+
1490.06605,
|
| 101 |
+
1558.93327,
|
| 102 |
+
151.64866,
|
| 103 |
+
44.4635,
|
| 104 |
+
53.02526,
|
| 105 |
+
2068.91261,
|
| 106 |
+
40.47347,
|
| 107 |
+
397.91506,
|
| 108 |
+
58.21347,
|
| 109 |
+
46.31337,
|
| 110 |
+
40.89776,
|
| 111 |
+
66.87334,
|
| 112 |
+
510.13955,
|
| 113 |
+
43.84304,
|
| 114 |
+
1900.14085,
|
| 115 |
+
68.75502,
|
| 116 |
+
955.51208,
|
| 117 |
+
53.99163,
|
| 118 |
+
58.21375,
|
| 119 |
+
42.01659,
|
| 120 |
+
66.53192,
|
| 121 |
+
2616.54124,
|
| 122 |
+
60.78525,
|
| 123 |
+
1365.12419,
|
| 124 |
+
53.3804,
|
| 125 |
+
49.62038,
|
| 126 |
+
1364.8067,
|
| 127 |
+
4001.53255,
|
| 128 |
+
61.37351,
|
| 129 |
+
47.13935,
|
| 130 |
+
44.06032,
|
| 131 |
+
458.43294,
|
| 132 |
+
47.19989,
|
| 133 |
+
1296.0189,
|
| 134 |
+
176.05968,
|
| 135 |
+
66.27581,
|
| 136 |
+
55.15212,
|
| 137 |
+
47.63964,
|
| 138 |
+
37.87151,
|
| 139 |
+
86.97425,
|
| 140 |
+
45.56593,
|
| 141 |
+
42.92863,
|
| 142 |
+
55.09171,
|
| 143 |
+
39.88255,
|
| 144 |
+
3009.76388,
|
| 145 |
+
1341.09018,
|
| 146 |
+
55.08479,
|
| 147 |
+
107.95607,
|
| 148 |
+
57.45166,
|
| 149 |
+
40.81644,
|
| 150 |
+
54.03973,
|
| 151 |
+
2751.96685,
|
| 152 |
+
71.72596,
|
| 153 |
+
561.86074,
|
| 154 |
+
66.16003,
|
| 155 |
+
164.70586,
|
| 156 |
+
140.85416,
|
| 157 |
+
87.79745,
|
| 158 |
+
40.03635,
|
| 159 |
+
2612.99797,
|
| 160 |
+
42.57246,
|
| 161 |
+
110.65331,
|
| 162 |
+
57.25475,
|
| 163 |
+
51.48026,
|
| 164 |
+
506.16988,
|
| 165 |
+
41.54455,
|
| 166 |
+
41.59874,
|
| 167 |
+
2522.02333,
|
| 168 |
+
59.40518,
|
| 169 |
+
2614.56268,
|
| 170 |
+
38.08357,
|
| 171 |
+
50.6474,
|
| 172 |
+
3063.74876,
|
| 173 |
+
48.43063,
|
| 174 |
+
96.17522,
|
| 175 |
+
57.26778,
|
| 176 |
+
70.33154,
|
| 177 |
+
46.36715,
|
| 178 |
+
404.33629,
|
| 179 |
+
1317.2864,
|
| 180 |
+
1322.3159,
|
| 181 |
+
63.02168,
|
| 182 |
+
102.9791,
|
| 183 |
+
57.52541,
|
| 184 |
+
176.35879,
|
| 185 |
+
2769.55748,
|
| 186 |
+
64.22492,
|
| 187 |
+
1817.0326,
|
| 188 |
+
48.23126,
|
| 189 |
+
60.74989,
|
| 190 |
+
53.38415,
|
| 191 |
+
50.49312,
|
| 192 |
+
4701.85047,
|
| 193 |
+
36.54419,
|
| 194 |
+
40.09773,
|
| 195 |
+
44.22369,
|
| 196 |
+
42.31991,
|
| 197 |
+
136.46588,
|
| 198 |
+
40.87392,
|
| 199 |
+
255.31355,
|
| 200 |
+
584.29782,
|
| 201 |
+
44.94602,
|
| 202 |
+
75.77784,
|
| 203 |
+
148.22984,
|
| 204 |
+
38.3939,
|
| 205 |
+
91.66273,
|
| 206 |
+
56.50123,
|
| 207 |
+
1425.14511,
|
| 208 |
+
58.22189,
|
| 209 |
+
181.3864,
|
| 210 |
+
46.81972,
|
| 211 |
+
48.84703,
|
| 212 |
+
41.58017,
|
| 213 |
+
558.48009,
|
| 214 |
+
69.34392,
|
| 215 |
+
44.08439,
|
| 216 |
+
43.00825,
|
| 217 |
+
2259.23838,
|
| 218 |
+
53.32712,
|
| 219 |
+
47.9518,
|
| 220 |
+
59.84782,
|
| 221 |
+
1227.37103,
|
| 222 |
+
154.69733,
|
| 223 |
+
67.36043,
|
| 224 |
+
444.89144,
|
| 225 |
+
66.23541,
|
| 226 |
+
668.46963,
|
| 227 |
+
1351.32746,
|
| 228 |
+
47.17544,
|
| 229 |
+
148.08138
|
| 230 |
+
]
|
| 231 |
+
},
|
| 232 |
+
"lmi/circle_beta_iters5/batch120": {
|
| 233 |
+
"n": 2,
|
| 234 |
+
"median_ms": 479.01488344650716,
|
| 235 |
+
"p99_ms": 557.0329894960548,
|
| 236 |
+
"mean_ms": 479.01488344650716,
|
| 237 |
+
"min_ms": 399.40457115105045,
|
| 238 |
+
"max_ms": 558.6251957419639,
|
| 239 |
+
"note": "looped over batch=120 (cvxpy solves are per plant); ms per plant; n = batches",
|
| 240 |
+
"samples_ms": [
|
| 241 |
+
558.6252,
|
| 242 |
+
399.40457
|
| 243 |
+
]
|
| 244 |
+
},
|
| 245 |
+
"lmi/quality_heldout120": {
|
| 246 |
+
"coverage": 1.0,
|
| 247 |
+
"regret_median_pct": 13.914860725402832,
|
| 248 |
+
"regret_p90_pct": 43.86730766296388,
|
| 249 |
+
"note": "linear decentralised gain at 0.3*cap under the circle-criterion-certified sector (sound certificate), vs stored eval_mimo_regret oracle"
|
| 250 |
+
}
|
| 251 |
+
},
|
| 252 |
+
"done": true
|
| 253 |
+
}
|
results/timing/classical_mimo_cpu8.json
ADDED
|
@@ -0,0 +1,260 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"machine": {
|
| 3 |
+
"host": "virgo-asl",
|
| 4 |
+
"python": "3.13.13",
|
| 5 |
+
"torch": "2.10.0+cu128",
|
| 6 |
+
"numpy": "2.2.6",
|
| 7 |
+
"scipy": "1.18.0",
|
| 8 |
+
"cvxpy": "1.9.2",
|
| 9 |
+
"diffusers": "0.38.0",
|
| 10 |
+
"threads": 8,
|
| 11 |
+
"tag": "cpu8",
|
| 12 |
+
"load1": 23.9970703125,
|
| 13 |
+
"date": "2026-09-30 07:54:54 PDT",
|
| 14 |
+
"cpu": "AMD Ryzen Threadripper 7960X 24-Cores",
|
| 15 |
+
"affinity": [
|
| 16 |
+
14,
|
| 17 |
+
15,
|
| 18 |
+
17,
|
| 19 |
+
18,
|
| 20 |
+
19,
|
| 21 |
+
20,
|
| 22 |
+
22,
|
| 23 |
+
23
|
| 24 |
+
]
|
| 25 |
+
},
|
| 26 |
+
"family": "mimo",
|
| 27 |
+
"cells": {
|
| 28 |
+
"lmi/circle_beta_iters5/b1": {
|
| 29 |
+
"n": 200,
|
| 30 |
+
"median_ms": 72.32201099395752,
|
| 31 |
+
"p99_ms": 3823.6898711882513,
|
| 32 |
+
"mean_ms": 549.2923274170607,
|
| 33 |
+
"min_ms": 36.78356297314167,
|
| 34 |
+
"max_ms": 4809.1598181054,
|
| 35 |
+
"note": "certified_beta(bmax=8, iters=5): bisection of the decentralised circle-criterion LMI; controller = linear gain 0.3*(0.7*beta) (the oracle's own init) => synthesis+certificate, per plant",
|
| 36 |
+
"samples_ms": [
|
| 37 |
+
1527.77675,
|
| 38 |
+
78.22582,
|
| 39 |
+
51.39571,
|
| 40 |
+
155.94155,
|
| 41 |
+
51.53694,
|
| 42 |
+
48.72184,
|
| 43 |
+
53.77331,
|
| 44 |
+
2011.14939,
|
| 45 |
+
312.37963,
|
| 46 |
+
41.20297,
|
| 47 |
+
44.20665,
|
| 48 |
+
46.50073,
|
| 49 |
+
89.90167,
|
| 50 |
+
2670.06131,
|
| 51 |
+
3821.51445,
|
| 52 |
+
378.58834,
|
| 53 |
+
60.86336,
|
| 54 |
+
99.62023,
|
| 55 |
+
58.56525,
|
| 56 |
+
727.18937,
|
| 57 |
+
3344.48929,
|
| 58 |
+
63.00118,
|
| 59 |
+
60.56186,
|
| 60 |
+
51.17881,
|
| 61 |
+
73.37691,
|
| 62 |
+
1338.93066,
|
| 63 |
+
62.08986,
|
| 64 |
+
66.98864,
|
| 65 |
+
2647.59406,
|
| 66 |
+
88.45621,
|
| 67 |
+
37.99611,
|
| 68 |
+
36.96601,
|
| 69 |
+
1232.83921,
|
| 70 |
+
57.47533,
|
| 71 |
+
854.23513,
|
| 72 |
+
110.16709,
|
| 73 |
+
2649.42927,
|
| 74 |
+
51.12484,
|
| 75 |
+
72.32689,
|
| 76 |
+
54.27222,
|
| 77 |
+
1462.28117,
|
| 78 |
+
2739.18881,
|
| 79 |
+
251.10868,
|
| 80 |
+
63.34201,
|
| 81 |
+
115.7227,
|
| 82 |
+
65.93608,
|
| 83 |
+
65.77918,
|
| 84 |
+
2737.63095,
|
| 85 |
+
290.35823,
|
| 86 |
+
1514.26963,
|
| 87 |
+
67.85548,
|
| 88 |
+
86.52914,
|
| 89 |
+
62.82481,
|
| 90 |
+
204.84217,
|
| 91 |
+
1016.16613,
|
| 92 |
+
117.729,
|
| 93 |
+
176.0141,
|
| 94 |
+
2726.83456,
|
| 95 |
+
818.55577,
|
| 96 |
+
84.46952,
|
| 97 |
+
87.13758,
|
| 98 |
+
65.15015,
|
| 99 |
+
83.34302,
|
| 100 |
+
62.24988,
|
| 101 |
+
54.73903,
|
| 102 |
+
67.388,
|
| 103 |
+
62.95666,
|
| 104 |
+
52.93454,
|
| 105 |
+
1631.87829,
|
| 106 |
+
69.88871,
|
| 107 |
+
1507.96469,
|
| 108 |
+
1608.04162,
|
| 109 |
+
165.65432,
|
| 110 |
+
57.12839,
|
| 111 |
+
64.94526,
|
| 112 |
+
2155.20359,
|
| 113 |
+
60.23292,
|
| 114 |
+
417.62554,
|
| 115 |
+
72.31713,
|
| 116 |
+
57.88626,
|
| 117 |
+
50.91124,
|
| 118 |
+
82.93688,
|
| 119 |
+
535.6281,
|
| 120 |
+
47.53878,
|
| 121 |
+
1968.79024,
|
| 122 |
+
95.56295,
|
| 123 |
+
1013.76554,
|
| 124 |
+
53.16884,
|
| 125 |
+
54.09356,
|
| 126 |
+
38.89047,
|
| 127 |
+
63.42843,
|
| 128 |
+
2660.12211,
|
| 129 |
+
86.45663,
|
| 130 |
+
1406.20906,
|
| 131 |
+
75.55625,
|
| 132 |
+
71.74694,
|
| 133 |
+
1388.89967,
|
| 134 |
+
4039.0562,
|
| 135 |
+
71.4912,
|
| 136 |
+
56.84959,
|
| 137 |
+
55.11437,
|
| 138 |
+
479.19634,
|
| 139 |
+
62.62968,
|
| 140 |
+
1373.05836,
|
| 141 |
+
188.31666,
|
| 142 |
+
104.92228,
|
| 143 |
+
90.90466,
|
| 144 |
+
58.57755,
|
| 145 |
+
48.79924,
|
| 146 |
+
97.79338,
|
| 147 |
+
44.27852,
|
| 148 |
+
41.38853,
|
| 149 |
+
54.24409,
|
| 150 |
+
37.54529,
|
| 151 |
+
3093.92548,
|
| 152 |
+
1354.13561,
|
| 153 |
+
53.30585,
|
| 154 |
+
106.70779,
|
| 155 |
+
55.76267,
|
| 156 |
+
37.69584,
|
| 157 |
+
50.46429,
|
| 158 |
+
2773.99922,
|
| 159 |
+
70.60852,
|
| 160 |
+
565.89937,
|
| 161 |
+
63.99923,
|
| 162 |
+
163.93369,
|
| 163 |
+
143.35188,
|
| 164 |
+
87.26837,
|
| 165 |
+
37.56383,
|
| 166 |
+
2628.61575,
|
| 167 |
+
41.91269,
|
| 168 |
+
108.21044,
|
| 169 |
+
55.18098,
|
| 170 |
+
50.32528,
|
| 171 |
+
513.04181,
|
| 172 |
+
39.77673,
|
| 173 |
+
40.20145,
|
| 174 |
+
2545.9515,
|
| 175 |
+
57.029,
|
| 176 |
+
2617.88251,
|
| 177 |
+
37.43142,
|
| 178 |
+
48.70009,
|
| 179 |
+
3095.50921,
|
| 180 |
+
46.63382,
|
| 181 |
+
97.19735,
|
| 182 |
+
55.36159,
|
| 183 |
+
68.30094,
|
| 184 |
+
44.91931,
|
| 185 |
+
405.59209,
|
| 186 |
+
1329.90252,
|
| 187 |
+
1335.46317,
|
| 188 |
+
60.15871,
|
| 189 |
+
100.17693,
|
| 190 |
+
56.598,
|
| 191 |
+
174.14835,
|
| 192 |
+
2820.7193,
|
| 193 |
+
62.5164,
|
| 194 |
+
1833.50239,
|
| 195 |
+
46.7516,
|
| 196 |
+
62.33442,
|
| 197 |
+
53.0722,
|
| 198 |
+
50.74952,
|
| 199 |
+
4809.15982,
|
| 200 |
+
37.87016,
|
| 201 |
+
40.12395,
|
| 202 |
+
43.84106,
|
| 203 |
+
45.63039,
|
| 204 |
+
141.18765,
|
| 205 |
+
39.07686,
|
| 206 |
+
256.61544,
|
| 207 |
+
596.76008,
|
| 208 |
+
47.17568,
|
| 209 |
+
76.84058,
|
| 210 |
+
150.91957,
|
| 211 |
+
36.78356,
|
| 212 |
+
89.76167,
|
| 213 |
+
54.36926,
|
| 214 |
+
1448.62717,
|
| 215 |
+
59.61437,
|
| 216 |
+
181.54364,
|
| 217 |
+
44.4173,
|
| 218 |
+
46.29237,
|
| 219 |
+
38.50612,
|
| 220 |
+
559.32421,
|
| 221 |
+
66.76084,
|
| 222 |
+
40.65791,
|
| 223 |
+
39.96966,
|
| 224 |
+
2289.57182,
|
| 225 |
+
51.63352,
|
| 226 |
+
44.79614,
|
| 227 |
+
57.01578,
|
| 228 |
+
1249.04927,
|
| 229 |
+
152.37869,
|
| 230 |
+
64.89577,
|
| 231 |
+
453.71979,
|
| 232 |
+
65.1298,
|
| 233 |
+
679.60313,
|
| 234 |
+
1370.48742,
|
| 235 |
+
43.84292,
|
| 236 |
+
149.43281
|
| 237 |
+
]
|
| 238 |
+
},
|
| 239 |
+
"lmi/circle_beta_iters5/batch120": {
|
| 240 |
+
"n": 2,
|
| 241 |
+
"median_ms": 492.2846951871179,
|
| 242 |
+
"p99_ms": 569.9142948704928,
|
| 243 |
+
"mean_ms": 492.2846951871179,
|
| 244 |
+
"min_ms": 413.07081795918447,
|
| 245 |
+
"max_ms": 571.4985724150514,
|
| 246 |
+
"note": "looped over batch=120 (cvxpy solves are per plant); ms per plant; n = batches",
|
| 247 |
+
"samples_ms": [
|
| 248 |
+
571.49857,
|
| 249 |
+
413.07082
|
| 250 |
+
]
|
| 251 |
+
},
|
| 252 |
+
"lmi/quality_heldout120": {
|
| 253 |
+
"coverage": 1.0,
|
| 254 |
+
"regret_median_pct": 13.914860725402832,
|
| 255 |
+
"regret_p90_pct": 43.86730766296388,
|
| 256 |
+
"note": "linear decentralised gain at 0.3*cap under the circle-criterion-certified sector (sound certificate), vs stored eval_mimo_regret oracle"
|
| 257 |
+
}
|
| 258 |
+
},
|
| 259 |
+
"done": true
|
| 260 |
+
}
|