DiffusionControl / results /baselines /logs /bd_prereg_eval.log
matCercola18's picture
Add files using upload-large-folder tool
68a167f verified
Raw History Blame Contribute Delete
2.22 kB
bank 180160 rows / 5635 plants; plant seeds (100, 200, 300, 400, 500); sampler (0, 1, 2); random-stored seeds 0,1,2; x0 seed 777
/home/cercola/matte_server/DiffusionControl/myenv/lib/python3.13/site-packages/scipy/signal/_lti_conversion.py:77: BadCoefficients: Badly conditioned filter coefficients (numerator): the results may be meaningless
num, den = normalize(num, den) # Strips zeros, checks arrays
plant-seed 100 done
plant-seed 200 done
plant-seed 300 done
plant-seed 400 done
plant-seed 500 done
=== PRE-REGISTERED distinct-plant retrieval vs diffusion, SISO, N=300 (plant seeds (100, 200, 300, 400, 500) x 60), full bank f=1.0, coverage only ===
model: full-data checkpoint, K<=16 samples/plant, sampler seeds 0/1/2 (seed 0 = paired reference; K-prefix of the 16 samples). verifier/x0 as bank_density sweep.
PRIMARY K=8: model(seed0) 0.773 distinct-first 0.750 diff +0.023 95% CI [+0.003,+0.047] McNemar model-only 9, retrieval-only 2, p=0.0654 => VERDICT: TIE
sampler seed 1: model 0.780 McNemar 12-3 p=0.0352
sampler seed 2: model 0.787 McNemar 13-2 p=0.00739
model 3-seed mean K=8: 0.780 +- 0.005
per plant-seed K=8 coverage (model seed0 / distinct-first):
100: 0.800 / 0.783
200: 0.800 / 0.783
300: 0.833 / 0.800
400: 0.733 / 0.717
500: 0.700 / 0.667
SECONDARY coverage vs K (model seed0 [3-seed mean]; distinct-first; distinct-random mean of 3 seeds; McNemar model vs distinct-first)
K model s0 model mean dist-first dist-random McNemar (m-only,r-only,p)
1 0.617 0.622 0.493 0.493+-0.000 (60,23,5.97e-05)
2 0.707 0.706 0.660 0.660+-0.000 (24,10,0.0243)
4 0.747 0.756 0.743 0.743+-0.000 (8,7,1)
8 0.773 0.780 0.750 0.750+-0.000 (9,2,0.0654)
16 0.787 0.793 0.753 0.753+-0.000 (12,2,0.0129)
random-stored K=8 (seed 0 draw) vs model: model-only 9, retrieval-only 2, p=0.0654
MIMO (distinct == plain retrieval, one controller per plant), f=1.0, K=8, from the sweep npz:
draw 9999: model 1.000 retrieval 0.992 McNemar 1-0 p=1
draw 4242: model 0.992 retrieval 0.992 McNemar 0-0 p=1