agent_reports_bgkf/t5_fleetcomp.md

t5_fleetcomp — FLEET-COMPOSITION probe of the A-vs-B planner-dial trade

Question. Config A (timing_weight=2.0, split_days=3; lam2sp3 head) vs Config B (timing_weight=1.0, split_days=7; old champion 4c93883). Same model, same counts, same ordering — only the two packing dials differ. CV says A wins −4.1±1.4; public says A loses +7.8 (swaps identical, late/early −1.6 better, ops +9.45 worse). Is the CV win composition-robust?

Verdict: A's win is composition-fragile — the public +7.8 is unsurprising. A's CV edge is earned in a low-pressure regime (five ~5-building fold fleets where daily/weekly limits almost never bind). Under public-like fleet pressure the A−B trade becomes an ops-limit lottery with fleet-level swings of ±10–25, and 3 of 7 high-pressure synthetic fleets — including the monolithic 24-building universe itself — reproduce the public loss with the same component signature (timing slightly better, ops ~+7..+11 worse).

Method: reused cv.run_bgkf machinery verbatim (same table build, complement fit with the deployed submit.Ft2RUL, deployment-exact planner state, real evaluator, floor gate as in run_bgkf); one fit per fleet shared across arms, paired per fleet×origin. Runner and raw CSVs in /tmp/c35/. Module hashes (cv.py 6b3460e9…, models_cv.py ea340538…) verified unchanged before and after.

Stage 1 — the existing CV comparison, split by fold fleet (48 origins each)

Canonical pair: /tmp/cv_bgkfcand_ft2a10.csv (A) vs /tmp/cv_bgkfcand_ft2a10_sp7.csv (B); reproduces the known ALL delta −4.11±1.41, 25W/4L exactly. (Note: cv_bgkfcand_head.csv at 19:25 is a stale model state — 42 cells differ from ft2a10.csv; do not pair against it.)

fold	dTotal	W/L/T (48)	dLate	dEarly	dOps
0	−0.60	8/2/38	−0.42	−0.45	+0.14
1	−0.74	12/3/33	−0.62	−0.36	+0.11
2	−0.95	3/0/45	−0.83	+0.04	−0.16
3	−0.02	1/1/46	0.00	−0.07	+0.04
4	−1.80	12/8/28	−1.67	−0.36	+0.13

No fold-level sign flip, but only 50/240 cells are nonzero, and A already pays a small ops tax in 4 of 5 fold fleets (ops-worse on 46% of origins) — the entire CV win is late/early. The public signature (ops-dominated) is the same trade at ~40x the ops magnitude.

Stage 2 — 32 random building subsets as standalone fleets (sizes 3/5/8/12, all 48 origins, paired)

1536 paired cells; 432 nonzero; 157 A-loss cells; swap counts identical in every single cell (the dials are count-flat everywhere). Fleet-level (mean over 48 origins): A wins 20 / loses 8 / ties 4; mean −2.10, median −0.14; all 8 losses tiny (max +0.95).

Pressure activates the trade — P(cell differs at all) and mean delta by fleet device count:

fleet devices	cells	P(nonzero)	P(A-loss)	mean dt
0–50	617	5.5%	1.3%	−0.04
50–100	180	19%	6.7%	−0.35
100–200	376	24%	9.3%	−0.62
200–300	310	74%	28%	−7.01
300–400	53	85%	30%	−13.7

Composition predictors of the fleet-level delta: corr(dt, n_dev) = −0.71, n_bldg −0.64, max building size −0.59, n_remote −0.51. So on AVERAGE bigger fleets favor A — but the variance explodes with size: cell deltas are late-swap quanta of ±10..±80 (a timing lottery on which deaths each packing catches) plus rare ±100/±200 daily/weekly-limit quanta (e.g. fleet n12_4 saw dOps −214 at two origins: B tripped daily+weekly limits A avoided).

Stage 3 — pressure stress: 18-building fleets and the 24-building monolith (48 origins)

Six random 18-building fleets (250–384 devices; ≥15/24 buildings shared pairwise, yet the sign flips fleet-to-fleet):

fleet	n_dev	dt	dOps	dLate	per-origin W/L
n18_1	352	−24.24	−21.89	−1.46	20/23
n18_2	251	−5.25	−4.23	−0.83	15/9
n18_4	353	−3.73	−2.94	−0.21	23/16
n18_5	322	−1.58	−5.43	+5.00	19/19
n18_0	384	+7.90	+10.82	−3.54	20/28
n18_3	349	+9.31	+9.35	−0.42	21/26

3 of 6 flip to A-loses, and the two big losses are carbon copies of the public measurement (+7.9 with ops +10.8; +9.3 with ops +9.4; timing slightly better in both — public: +7.8, ops +9.45, late/early −1.6).

n24_all — the exact CV 24-building universe planned as ONE fleet instead of five fold sub-fleets: mean +0.97 (24W/20L), dLate −4.38, dEarly −1.62, dOps +6.47 (daily +2.08, weekly +4.17, travel +0.36). The same buildings that give A a 2.9σ win when planned as five small fleets give a coin-flip-to-loss with the public's ops signature when planned as one fleet. Worst origins: s21 +257 (ops +311), s16 +192 (ops +214) — pure daily/weekly-limit quanta. (Caveat: complement is empty for this fleet, so the fit is self-fit — leaky, but identical in both arms; the paired planner delta stands.)

Across all 39 synthetic fleets: A loses in 11/39 (28%); among the 15 fleets with ≥200 devices, 3 losses, all with the ops signature.

Which composition features predict the sign

1. Constraint pressure (total fleet devices) is the master variable — it predicts whether the dials matter at all (5% → 85% of origins) and blows up the variance. Below ~100 devices A≈B almost everywhere.
2. At high pressure the sign is a daily/weekly-limit collision lottery (±100/±200 quanta), quasi-random in composition; weakly suggestive (n=6, anecdotal): including the two death-densest mid-size buildings (b_9a5a45d6e498 24dev/15deaths, b_a0e6376410cb 28dev/11deaths) correlates with A-worse ops (membership-dOps corr +0.65/+0.74).
3. Remote buildings and the 117-device giant increase activation but do not fix the sign (fleets containing the giant span dt −19.9 to +8.8 at n≤12, and the giant is absent from both n18 losers).

Implications

• The CV structurally cannot price this trade: its test fleets are ~5-building sub-fleets where limits don't bind. The public fleet is a single large fleet where they do. The n24 monolith is the cheapest local proxy for that regime, and it sides with the public (A ≈ +1, ops-worse).
• The dial choice (split3/tw2 vs split7/tw1) is a ~±10–25 ops lottery on any large unseen fleet; the −4.1 CV delta should not be treated as transferable evidence for A on the private split. On pressure-matched local evidence the two configs are statistically indistinguishable (n24: +0.97±8.64).

Raw data: /tmp/c35/stage1_merged.csv, stage2_all48.csv (+ per-fleet stage2_perfleet_48orig.csv, cells stage2_cells_48orig.csv), stage3_stress.csv (+ cells), stage3_n24.csv (+ cells), building_meta.csv, fleet defs, and the runner fleetprobe.py. Stage-2 6-origin pilot (stage2_results.csv) superseded by the all-48 rerun. Origins for stress/all48: all 48; fits per fleet on the complement buildings' closed pairs (deployment-faithful), shared across arms.