matCercola18's picture
Add files using upload-large-folder tool
929e74d verified
Raw History Blame Contribute Delete
20.4 kB
=== K=1 (N=80 plants; model rows = mean over runs; regret on the 24 fresh scoring ICs vs the fresh-IC oracle) ===
arm stable% cert% anyCert% reg_med [95% CI] reg_p90 reg_mean in-sample reg_med
retrieval, original bank 100.0 100.0 100.0 3.60 [+2.92,+4.03] 5.57 3.70 1.64
retrieval, multi bank (K nearest rows) 100.0 100.0 100.0 0.36 [+0.13,+0.53] 1.61 0.58 0.66
retrieval, multi bank best-per-plant 100.0 100.0 100.0 0.36 [+0.13,+0.53] 1.61 0.58 0.66
inverse-distance blend, original bank 100.0 100.0 100.0 3.29 [+3.00,+3.63] 5.01 3.42 1.54
retrieval, multi-B bank (K nearest rows) 100.0 100.0 100.0 0.15 [+0.10,+0.32] 0.97 0.33 0.47
retrieval, multi-B bank best-per-plant 100.0 100.0 100.0 0.15 [+0.10,+0.32] 0.97 0.33 0.47
tunedB 100.0 88.2 88.2 0.37 [+0.29,+0.51] 1.18 0.57 0.51
tunedA 100.0 98.8 98.8 0.62 [+0.47,+0.81] 1.73 0.91 0.77
shipped 100.0 100.0 100.0 3.27 [+3.12,+3.50] 5.22 3.50 1.54
=== K=2 (N=80 plants; model rows = mean over runs; regret on the 24 fresh scoring ICs vs the fresh-IC oracle) ===
arm stable% cert% anyCert% reg_med [95% CI] reg_p90 reg_mean in-sample reg_med
retrieval, original bank 100.0 100.0 100.0 2.68 [+2.46,+2.99] 4.53 2.80 1.12
retrieval, multi bank (K nearest rows) 100.0 100.0 100.0 0.18 [+0.11,+0.31] 0.86 0.34 0.41
retrieval, multi bank best-per-plant 100.0 100.0 100.0 0.18 [+0.11,+0.31] 0.86 0.34 0.41
inverse-distance blend, original bank 100.0 100.0 100.0 2.94 [+2.68,+3.21] 4.38 2.86 1.30
retrieval, multi-B bank (K nearest rows) 100.0 98.8 100.0 0.19 [+0.11,+0.38] 0.97 0.37 0.40
retrieval, multi-B bank best-per-plant 100.0 98.8 100.0 0.13 [+0.07,+0.18] 0.72 0.29 0.37
tunedB 100.0 91.5 98.8 0.31 [+0.20,+0.35] 0.85 0.35 0.34
tunedA 100.0 98.9 99.6 0.52 [+0.39,+0.69] 1.74 0.72 0.47
shipped 100.0 100.0 100.0 3.09 [+2.87,+3.26] 5.02 3.19 1.27
=== K=4 (N=80 plants; model rows = mean over runs; regret on the 24 fresh scoring ICs vs the fresh-IC oracle) ===
arm stable% cert% anyCert% reg_med [95% CI] reg_p90 reg_mean in-sample reg_med
retrieval, original bank 100.0 100.0 100.0 1.52 [+1.31,+1.85] 2.98 1.69 0.56
retrieval, multi bank (K nearest rows) 100.0 98.8 100.0 0.13 [+0.09,+0.20] 0.54 0.15 0.26
retrieval, multi bank best-per-plant 100.0 98.8 100.0 0.13 [+0.09,+0.20] 0.54 0.15 0.26
inverse-distance blend, original bank 100.0 100.0 100.0 1.57 [+1.34,+1.89] 3.39 1.83 0.59
retrieval, multi-B bank (K nearest rows) 100.0 97.5 100.0 0.24 [+0.12,+0.40] 1.57 0.54 0.26
retrieval, multi-B bank best-per-plant 100.0 98.8 100.0 0.14 [+0.08,+0.21] 0.74 0.36 0.23
tunedB 100.0 90.3 99.7 0.30 [+0.25,+0.42] 0.73 0.36 0.20
tunedA 100.0 99.0 99.9 0.49 [+0.36,+0.56] 1.53 0.66 0.27
shipped 100.0 100.0 100.0 2.93 [+2.57,+3.12] 4.75 3.00 1.11
=== K=8 (N=80 plants; model rows = mean over runs; regret on the 24 fresh scoring ICs vs the fresh-IC oracle) ===
arm stable% cert% anyCert% reg_med [95% CI] reg_p90 reg_mean in-sample reg_med
retrieval, original bank 100.0 100.0 100.0 0.92 [+0.79,+1.15] 1.92 1.09 0.32
retrieval, multi bank (K nearest rows) 100.0 98.8 100.0 0.11 [+0.06,+0.17] 0.42 0.10 0.18
retrieval, multi bank best-per-plant 100.0 98.8 100.0 0.12 [+0.06,+0.17] 0.42 0.10 0.18
inverse-distance blend, original bank 100.0 100.0 100.0 0.92 [+0.80,+1.15] 1.93 1.12 0.32
retrieval, multi-B bank (K nearest rows) 100.0 96.2 100.0 0.16 [+0.11,+0.29] 1.40 0.47 0.09
retrieval, multi-B bank best-per-plant 100.0 98.8 100.0 0.15 [+0.10,+0.23] 0.77 0.36 0.12
tunedB 100.0 88.6 100.0 0.32 [+0.21,+0.41] 0.88 0.35 0.10
tunedA 100.0 98.6 100.0 0.52 [+0.45,+0.63] 1.29 0.66 0.17
shipped 100.0 100.0 100.0 2.82 [+2.46,+3.00] 4.55 2.86 1.02
--- paired tests: tunedB minus other arm (negative = tunedB better); median paired diff in regret points [bootstrap 95% CI], Wilcoxon p, McNemar on majority-stable / picked-certified (n10 = tunedB only, n01 = other only) ---
K=1 vs retrieval, original bank -2.96 [-3.60,-2.44] p=1.3e-14 n=80 | stable McNemar 0-0 p=1 | cert 0-3 p=0.25 [PRIMARY]
K=1 vs retrieval, multi bank (K nearest rows) +0.07 [-0.03,+0.17] p=0.53 n=80 | stable McNemar 0-0 p=1 | cert 0-3 p=0.25 [PRIMARY]
K=1 vs retrieval, multi bank best-per-plant +0.07 [-0.03,+0.17] p=0.53 n=80 | stable McNemar 0-0 p=1 | cert 0-3 p=0.25 [exploratory]
K=1 vs inverse-distance blend, original bank -2.88 [-3.03,-2.59] p=8.2e-15 n=80 | stable McNemar 0-0 p=1 | cert 0-3 p=0.25 [exploratory]
K=1 vs retrieval, multi-B bank (K nearest rows) +0.18 [+0.08,+0.28] p=0.001 n=80 | stable McNemar 0-0 p=1 | cert 0-3 p=0.25 [exploratory]
K=1 vs retrieval, multi-B bank best-per-plant +0.18 [+0.08,+0.28] p=0.001 n=80 | stable McNemar 0-0 p=1 | cert 0-3 p=0.25 [exploratory]
K=1 vs tunedA -0.15 [-0.29,-0.09] p=1.8e-08 n=80 | stable McNemar 0-0 p=1 | cert 0-3 p=0.25 [exploratory]
K=1 vs shipped -2.88 [-3.11,-2.61] p=8.8e-15 n=80 | stable McNemar 0-0 p=1 | cert 0-3 p=0.25 [exploratory]
K=2 vs retrieval, original bank -2.33 [-2.60,-1.94] p=1.3e-14 n=80 | stable McNemar 0-0 p=1 | cert 0-1 p=1 [exploratory]
K=2 vs retrieval, multi bank (K nearest rows) +0.08 [+0.01,+0.18] p=0.14 n=80 | stable McNemar 0-0 p=1 | cert 0-1 p=1 [exploratory]
K=2 vs retrieval, multi bank best-per-plant +0.09 [+0.02,+0.18] p=0.16 n=80 | stable McNemar 0-0 p=1 | cert 0-1 p=1 [exploratory]
K=2 vs inverse-distance blend, original bank -2.50 [-2.89,-2.30] p=1.1e-14 n=80 | stable McNemar 0-0 p=1 | cert 0-1 p=1 [exploratory]
K=2 vs retrieval, multi-B bank (K nearest rows) +0.06 [-0.09,+0.16] p=0.69 n=80 | stable McNemar 0-0 p=1 | cert 1-1 p=1 [exploratory]
K=2 vs retrieval, multi-B bank best-per-plant +0.15 [+0.08,+0.23] p=0.0015 n=80 | stable McNemar 0-0 p=1 | cert 1-1 p=1 [exploratory]
K=2 vs tunedA -0.18 [-0.35,-0.10] p=3.8e-09 n=80 | stable McNemar 0-0 p=1 | cert 1-1 p=1 [exploratory]
K=2 vs shipped -2.73 [-2.93,-2.43] p=8.2e-15 n=80 | stable McNemar 0-0 p=1 | cert 0-1 p=1 [exploratory]
K=4 vs retrieval, original bank -1.20 [-1.65,-0.89] p=2.4e-13 n=80 | stable McNemar 0-0 p=1 | cert 0-2 p=0.5 [exploratory]
K=4 vs retrieval, multi bank (K nearest rows) +0.17 [+0.08,+0.24] p=2.6e-05 n=80 | stable McNemar 0-0 p=1 | cert 1-2 p=1 [exploratory]
K=4 vs retrieval, multi bank best-per-plant +0.17 [+0.08,+0.24] p=2.6e-05 n=80 | stable McNemar 0-0 p=1 | cert 1-2 p=1 [exploratory]
K=4 vs inverse-distance blend, original bank -1.28 [-1.71,-1.00] p=1.6e-13 n=80 | stable McNemar 0-0 p=1 | cert 0-2 p=0.5 [exploratory]
K=4 vs retrieval, multi-B bank (K nearest rows) +0.07 [-0.07,+0.17] p=0.88 n=80 | stable McNemar 0-0 p=1 | cert 2-2 p=1 [exploratory]
K=4 vs retrieval, multi-B bank best-per-plant +0.18 [+0.08,+0.23] p=0.00053 n=80 | stable McNemar 0-0 p=1 | cert 1-2 p=1 [exploratory]
K=4 vs tunedA -0.14 [-0.24,-0.10] p=1e-06 n=80 | stable McNemar 0-0 p=1 | cert 1-2 p=1 [exploratory]
K=4 vs shipped -2.59 [-2.82,-2.22] p=8.2e-15 n=80 | stable McNemar 0-0 p=1 | cert 0-2 p=0.5 [exploratory]
K=8 vs retrieval, original bank -0.51 [-0.72,-0.28] p=1.4e-11 n=80 | stable McNemar 0-0 p=1 | cert 0-2 p=0.5 [PRIMARY]
K=8 vs retrieval, multi bank (K nearest rows) +0.19 [+0.15,+0.34] p=3e-09 n=80 | stable McNemar 0-0 p=1 | cert 1-2 p=1 [PRIMARY]
K=8 vs retrieval, multi bank best-per-plant +0.19 [+0.15,+0.33] p=3.3e-09 n=80 | stable McNemar 0-0 p=1 | cert 1-2 p=1 [exploratory]
K=8 vs inverse-distance blend, original bank -0.51 [-0.76,-0.29] p=1e-11 n=80 | stable McNemar 0-0 p=1 | cert 0-2 p=0.5 [exploratory]
K=8 vs retrieval, multi-B bank (K nearest rows) +0.07 [-0.00,+0.16] p=0.75 n=80 | stable McNemar 0-0 p=1 | cert 3-2 p=1 [exploratory]
K=8 vs retrieval, multi-B bank best-per-plant +0.16 [+0.09,+0.25] p=0.00072 n=80 | stable McNemar 0-0 p=1 | cert 1-2 p=1 [exploratory]
K=8 vs tunedA -0.14 [-0.27,-0.06] p=1.3e-07 n=80 | stable McNemar 0-0 p=1 | cert 1-2 p=1 [exploratory]
K=8 vs shipped -2.25 [-2.75,-2.08] p=7.8e-15 n=80 | stable McNemar 0-0 p=1 | cert 0-2 p=0.5 [exploratory]
--- paired tests: tunedA minus other arm (negative = tunedA better); median paired diff in regret points [bootstrap 95% CI], Wilcoxon p, McNemar on majority-stable / picked-certified (n10 = tunedA only, n01 = other only) ---
K=1 vs retrieval, original bank -3.01 [-3.51,-2.36] p=1.5e-13 n=80 | stable McNemar 0-0 p=1 | cert 0-0 p=1 [exploratory]
K=1 vs retrieval, multi bank (K nearest rows) +0.19 [+0.11,+0.33] p=0.0037 n=80 | stable McNemar 0-0 p=1 | cert 0-0 p=1 [exploratory]
K=1 vs retrieval, multi bank best-per-plant +0.19 [+0.11,+0.33] p=0.0037 n=80 | stable McNemar 0-0 p=1 | cert 0-0 p=1 [exploratory]
K=1 vs inverse-distance blend, original bank -2.66 [-2.86,-2.44] p=2e-14 n=80 | stable McNemar 0-0 p=1 | cert 0-0 p=1 [exploratory]
K=1 vs retrieval, multi-B bank (K nearest rows) +0.34 [+0.22,+0.52] p=2.9e-06 n=80 | stable McNemar 0-0 p=1 | cert 0-0 p=1 [exploratory]
K=1 vs retrieval, multi-B bank best-per-plant +0.34 [+0.22,+0.52] p=2.9e-06 n=80 | stable McNemar 0-0 p=1 | cert 0-0 p=1 [exploratory]
K=1 vs tunedB +0.15 [+0.09,+0.29] p=1.8e-08 n=80 | stable McNemar 0-0 p=1 | cert 3-0 p=0.25 [exploratory]
K=1 vs shipped -2.76 [-3.03,-2.44] p=1.6e-14 n=80 | stable McNemar 0-0 p=1 | cert 0-0 p=1 [exploratory]
K=2 vs retrieval, original bank -2.05 [-2.36,-1.72] p=1.2e-13 n=80 | stable McNemar 0-0 p=1 | cert 0-1 p=1 [exploratory]
K=2 vs retrieval, multi bank (K nearest rows) +0.22 [+0.16,+0.42] p=9.3e-07 n=80 | stable McNemar 0-0 p=1 | cert 0-1 p=1 [exploratory]
K=2 vs retrieval, multi bank best-per-plant +0.22 [+0.16,+0.43] p=1.7e-06 n=80 | stable McNemar 0-0 p=1 | cert 0-1 p=1 [exploratory]
K=2 vs inverse-distance blend, original bank -2.16 [-2.62,-1.94] p=2.7e-14 n=80 | stable McNemar 0-0 p=1 | cert 0-1 p=1 [exploratory]
K=2 vs retrieval, multi-B bank (K nearest rows) +0.24 [+0.10,+0.42] p=4.8e-05 n=80 | stable McNemar 0-0 p=1 | cert 1-1 p=1 [exploratory]
K=2 vs retrieval, multi-B bank best-per-plant +0.38 [+0.20,+0.53] p=2.1e-08 n=80 | stable McNemar 0-0 p=1 | cert 1-1 p=1 [exploratory]
K=2 vs tunedB +0.18 [+0.10,+0.35] p=3.8e-09 n=80 | stable McNemar 0-0 p=1 | cert 1-1 p=1 [exploratory]
K=2 vs shipped -2.62 [-2.88,-2.25] p=1.2e-14 n=80 | stable McNemar 0-0 p=1 | cert 0-1 p=1 [exploratory]
K=4 vs retrieval, original bank -0.97 [-1.25,-0.58] p=7.8e-11 n=80 | stable McNemar 0-0 p=1 | cert 0-1 p=1 [exploratory]
K=4 vs retrieval, multi bank (K nearest rows) +0.33 [+0.23,+0.51] p=5.3e-10 n=80 | stable McNemar 0-0 p=1 | cert 1-1 p=1 [exploratory]
K=4 vs retrieval, multi bank best-per-plant +0.33 [+0.23,+0.51] p=5.3e-10 n=80 | stable McNemar 0-0 p=1 | cert 1-1 p=1 [exploratory]
K=4 vs inverse-distance blend, original bank -1.04 [-1.30,-0.61] p=1.2e-11 n=80 | stable McNemar 0-0 p=1 | cert 0-1 p=1 [exploratory]
K=4 vs retrieval, multi-B bank (K nearest rows) +0.18 [+0.04,+0.34] p=0.031 n=80 | stable McNemar 0-0 p=1 | cert 2-1 p=1 [exploratory]
K=4 vs retrieval, multi-B bank best-per-plant +0.32 [+0.20,+0.42] p=5.3e-07 n=80 | stable McNemar 0-0 p=1 | cert 1-1 p=1 [exploratory]
K=4 vs tunedB +0.14 [+0.10,+0.24] p=1e-06 n=80 | stable McNemar 0-0 p=1 | cert 2-1 p=1 [exploratory]
K=4 vs shipped -2.57 [-2.81,-2.10] p=1.5e-14 n=80 | stable McNemar 0-0 p=1 | cert 0-1 p=1 [exploratory]
K=8 vs retrieval, original bank -0.25 [-0.48,-0.19] p=9.8e-06 n=80 | stable McNemar 0-0 p=1 | cert 0-1 p=1 [exploratory]
K=8 vs retrieval, multi bank (K nearest rows) +0.44 [+0.34,+0.53] p=4.9e-13 n=80 | stable McNemar 0-0 p=1 | cert 1-1 p=1 [exploratory]
K=8 vs retrieval, multi bank best-per-plant +0.43 [+0.33,+0.53] p=5.3e-13 n=80 | stable McNemar 0-0 p=1 | cert 1-1 p=1 [exploratory]
K=8 vs inverse-distance blend, original bank -0.25 [-0.48,-0.19] p=7.1e-06 n=80 | stable McNemar 0-0 p=1 | cert 0-1 p=1 [exploratory]
K=8 vs retrieval, multi-B bank (K nearest rows) +0.34 [+0.11,+0.42] p=0.0038 n=80 | stable McNemar 0-0 p=1 | cert 3-1 p=0.62 [exploratory]
K=8 vs retrieval, multi-B bank best-per-plant +0.37 [+0.20,+0.44] p=6.5e-09 n=80 | stable McNemar 0-0 p=1 | cert 1-1 p=1 [exploratory]
K=8 vs tunedB +0.14 [+0.06,+0.27] p=1.3e-07 n=80 | stable McNemar 0-0 p=1 | cert 2-1 p=1 [exploratory]
K=8 vs shipped -2.25 [-2.62,-1.99] p=1.9e-14 n=80 | stable McNemar 0-0 p=1 | cert 0-1 p=1 [exploratory]
--- paired tests: shipped minus other arm (negative = shipped better); median paired diff in regret points [bootstrap 95% CI], Wilcoxon p, McNemar on majority-stable / picked-certified (n10 = shipped only, n01 = other only) ---
K=1 vs retrieval, original bank -0.19 [-0.65,+0.42] p=0.28 n=80 | stable McNemar 0-0 p=1 | cert 0-0 p=1 [exploratory]
K=1 vs retrieval, multi bank (K nearest rows) +2.89 [+2.56,+3.13] p=9.1e-15 n=80 | stable McNemar 0-0 p=1 | cert 0-0 p=1 [exploratory]
K=1 vs retrieval, multi bank best-per-plant +2.89 [+2.56,+3.13] p=9.1e-15 n=80 | stable McNemar 0-0 p=1 | cert 0-0 p=1 [exploratory]
K=1 vs inverse-distance blend, original bank +0.24 [-0.28,+0.37] p=0.4 n=80 | stable McNemar 0-0 p=1 | cert 0-0 p=1 [exploratory]
K=1 vs retrieval, multi-B bank (K nearest rows) +3.10 [+2.81,+3.21] p=9.1e-15 n=80 | stable McNemar 0-0 p=1 | cert 0-0 p=1 [exploratory]
K=1 vs retrieval, multi-B bank best-per-plant +3.10 [+2.81,+3.21] p=9.1e-15 n=80 | stable McNemar 0-0 p=1 | cert 0-0 p=1 [exploratory]
K=1 vs tunedB +2.88 [+2.61,+3.11] p=8.8e-15 n=80 | stable McNemar 0-0 p=1 | cert 3-0 p=0.25 [exploratory]
K=1 vs tunedA +2.76 [+2.44,+3.03] p=1.6e-14 n=80 | stable McNemar 0-0 p=1 | cert 0-0 p=1 [exploratory]
K=2 vs retrieval, original bank +0.68 [+0.15,+0.93] p=0.015 n=80 | stable McNemar 0-0 p=1 | cert 0-0 p=1 [exploratory]
K=2 vs retrieval, multi bank (K nearest rows) +2.92 [+2.59,+3.12] p=7.8e-15 n=80 | stable McNemar 0-0 p=1 | cert 0-0 p=1 [exploratory]
K=2 vs retrieval, multi bank best-per-plant +2.88 [+2.53,+3.12] p=7.8e-15 n=80 | stable McNemar 0-0 p=1 | cert 0-0 p=1 [exploratory]
K=2 vs inverse-distance blend, original bank +0.34 [+0.06,+0.76] p=0.016 n=80 | stable McNemar 0-0 p=1 | cert 0-0 p=1 [exploratory]
K=2 vs retrieval, multi-B bank (K nearest rows) +2.85 [+2.50,+3.04] p=9.5e-15 n=80 | stable McNemar 0-0 p=1 | cert 1-0 p=1 [exploratory]
K=2 vs retrieval, multi-B bank best-per-plant +2.92 [+2.53,+3.12] p=9.8e-15 n=80 | stable McNemar 0-0 p=1 | cert 1-0 p=1 [exploratory]
K=2 vs tunedB +2.73 [+2.43,+2.93] p=8.2e-15 n=80 | stable McNemar 0-0 p=1 | cert 1-0 p=1 [exploratory]
K=2 vs tunedA +2.62 [+2.25,+2.88] p=1.2e-14 n=80 | stable McNemar 0-0 p=1 | cert 1-0 p=1 [exploratory]
K=4 vs retrieval, original bank +1.29 [+0.96,+1.62] p=1.4e-11 n=80 | stable McNemar 0-0 p=1 | cert 0-0 p=1 [exploratory]
K=4 vs retrieval, multi bank (K nearest rows) +2.70 [+2.32,+2.98] p=7.8e-15 n=80 | stable McNemar 0-0 p=1 | cert 1-0 p=1 [exploratory]
K=4 vs retrieval, multi bank best-per-plant +2.70 [+2.32,+2.98] p=7.8e-15 n=80 | stable McNemar 0-0 p=1 | cert 1-0 p=1 [exploratory]
K=4 vs inverse-distance blend, original bank +1.18 [+0.87,+1.58] p=2e-09 n=80 | stable McNemar 0-0 p=1 | cert 0-0 p=1 [exploratory]
K=4 vs retrieval, multi-B bank (K nearest rows) +2.47 [+1.98,+2.68] p=2.5e-14 n=80 | stable McNemar 0-0 p=1 | cert 2-0 p=0.5 [exploratory]
K=4 vs retrieval, multi-B bank best-per-plant +2.55 [+2.23,+2.85] p=9.8e-15 n=80 | stable McNemar 0-0 p=1 | cert 1-0 p=1 [exploratory]
K=4 vs tunedB +2.59 [+2.22,+2.82] p=8.2e-15 n=80 | stable McNemar 0-0 p=1 | cert 2-0 p=0.5 [exploratory]
K=4 vs tunedA +2.57 [+2.10,+2.81] p=1.5e-14 n=80 | stable McNemar 0-0 p=1 | cert 1-0 p=1 [exploratory]
K=8 vs retrieval, original bank +1.81 [+1.46,+1.95] p=1.8e-14 n=80 | stable McNemar 0-0 p=1 | cert 0-0 p=1 [exploratory]
K=8 vs retrieval, multi bank (K nearest rows) +2.59 [+2.25,+2.97] p=7.8e-15 n=80 | stable McNemar 0-0 p=1 | cert 1-0 p=1 [exploratory]
K=8 vs retrieval, multi bank best-per-plant +2.59 [+2.20,+2.97] p=7.8e-15 n=80 | stable McNemar 0-0 p=1 | cert 1-0 p=1 [exploratory]
K=8 vs inverse-distance blend, original bank +1.78 [+1.42,+1.94] p=1.9e-14 n=80 | stable McNemar 0-0 p=1 | cert 0-0 p=1 [exploratory]
K=8 vs retrieval, multi-B bank (K nearest rows) +2.17 [+1.92,+2.58] p=3.4e-14 n=80 | stable McNemar 0-0 p=1 | cert 3-0 p=0.25 [exploratory]
K=8 vs retrieval, multi-B bank best-per-plant +2.55 [+2.13,+2.78] p=1.5e-14 n=80 | stable McNemar 0-0 p=1 | cert 1-0 p=1 [exploratory]
K=8 vs tunedB +2.25 [+2.08,+2.75] p=7.8e-15 n=80 | stable McNemar 0-0 p=1 | cert 2-0 p=0.5 [exploratory]
K=8 vs tunedA +2.25 [+1.99,+2.62] p=1.9e-14 n=80 | stable McNemar 0-0 p=1 | cert 1-0 p=1 [exploratory]
Holm-corrected primary tests (alpha=0.05):
tunedB vs retr_orig K=1: p=1.28e-14, Holm threshold 0.0125 -> reject
tunedB vs retr_orig K=8: p=1.35e-11, Holm threshold 0.0167 -> reject
tunedB vs retr_multi_rows K=8: p=2.97e-09, Holm threshold 0.0250 -> reject
tunedB vs retr_multi_rows K=1: p=0.53, Holm threshold 0.0500 -> not significant