Thibaut Forest
Claude Opus 5 (1M context)
Phase 56: NEW CHAMPION logbag_rank + gbm rank-blend w=0.30 (purge 1630.3, was 1972.0)
698401a
|
Raw History Blame Contribute Delete
6.71 kB

e2 — end-to-end plan optimization & fleet-level forecasting (radical pipeline)

STATUS: DONE — 2026-08-22 ~17:20. Top candidate: e2_hgatez (-245.8 +/- 88.3, clears 2*SE bar).

Baseline: champion logbag/tsroute, SCORE 1972.0 (warmup 3560.3 / panel 1839.6 / nearend 604.8), swaps 9.21. Champion PREDICTIONS frozen & cached (48 temporal-purge logbag fits reproduce baseline EXACTLY: 1972.01, max|diff|=0.000000). Every delta below is a pure PLANNER/BUDGET change on identical predictions. True eol only ever enters evaluate_plan. Harness /tmp/e2/harness.py, cache /tmp/e2/preds.pkl, candidate CSVs /tmp/cv_oofcand_e2_*.csv.

Results table (full-48, paired vs /tmp/cv_all48oof.csv; d/dsw = delta per extra swap)

label ALL48 delta SE W/L swaps d/dsw blocks (wu/pn/ne) verdict
e2_hgatez 1726.3 -245.8 88.3 17/8 11.96 -89 -853 / -146 / 0 SURVIVOR — top pick
e2_hgate 1773.8 -198.3 84.0 15/8 11.77 -77 -568 / -146 / 0 survivor; superseded by hgatez
e2_hgate_b 1785.2 -186.8 84.9 14/9 12.04 -66 -568 / -130 / 0 budget floor on top adds nothing
e2_nbud 1875.5 -96.5 62.2 11/10 10.67 -66 -211 / -87 / 0 appetite-equivalent, not adopted
e2_a093 1908.9 -63.1 33.8 16/23 10.23 -62 -192 / -45 / +3 control (blanket alpha 0.93)
e2_nbud075 1933.2 -38.9 39.2 8/7 9.75 -72 -42 / -45 / 0 muted beta loses the big wins
e2_lsadd 1938.6 -33.4 29.1 14/19 11.21 -17 -40 / -46 / +46 surrogate-EV adds < simple p_be rule
e2_opt 1970.3 -1.7 1.6 21/18 9.21 — +1 / -3 / +2 day placement is CONVERGED
triage-only: e2_ls -4.7+/-76.8 (9.71 sw), e2_cvar -4.4+/-67.6 (8.47 sw), e2_ls_pmax +97.6+/-57.9 (7.71 sw) washes / harmful

Top candidate: e2_hgatez — hazard-gated break-even top-up

Rule (all plan-time observable): let r42 = fleet deaths RECORDED in (T-42d, T], zone42 = census of devices currently below the 2.45V corrected threshold that crossed within the last 42d (telemetry <= T). If max(r42, zone42) > sum(p42): open the break-even top-up p_be=0.2 (include every device with p42 >= 0.2). Else: plan EXACTLY as champion. Fires at ~22/48 origins (warmup s_0-2 via zone42 — the ~34d recording lag blinds r42 there; high-hazard panel eras s_4-16, s_18-23 via r42; NEVER at s_24-47 calm/near-end).

  • -245.8 +/- 88.3, 17W/8L, swaps 11.96 (+2.75). Clears the survivor bar (2*SE = 177) despite the swap rise.
  • Head-to-head vs pl_pbe20 (same p_be mechanism, always-on, -254.0 +/- 98.5, swaps 14.69): means statistically identical (+8.3 +/- 44.6), hgatez better on 14 of 20 differing origins, lower per-origin std (612 vs 683), 2.7 fewer swaps/scenario, nearend swaps unchanged (16.7 = base) vs pbe20's 18.5.
  • Where the gate closes, pbe20 is bimodal: it forfeits 2 big wins the gate misses (s_3 -979, s_17 -1374 — origins where the COUNT was adequate but the RANKING was broken; not plan-time detectable) but avoids 14 losses averaging +220 (calm origins where extra swaps caught nothing).
  • Deployment asymmetry: at near-end (most deployment-like) origins the gate never fires — adopting it cannot change the submission plan unless recorded arrivals demonstrably exceed the model's expected count at runtime. It is a safety net, not a recalibration.

COORDINATION: e2_hgatez and pl_pbe20 are ONE mechanism family (gated vs always-on p_be) — adjudicate together, never stack. If appetite-driven wins are distrusted (static appetite was publicly refuted historically), the gated form is the defensible variant: swaps stay at champion level except under an observable hazard signal.

MULTIPLICITY: 10 full-48 planner variants were run; the gate signal was widened once (r42 -> max(r42, zone42)) after seeing full-48 results. p_be=0.2 itself was found independently by the pl agent. Selection bias is real; the survivor margin (-245.8 vs bar -177) should be read with that in mind.

Direction verdicts

A) SURROGATE-COST PLAN SEARCH: DEAD, with strong evidence. (i) Day placement/routing: 4x search effort (restarts 8, iters 100, constructive start, NN routing) = -1.7 +/- 1.6 — the champion's optimize_days output is ~converged; no recoverable slack in WHEN/ORDER. (ii) Membership local search (drop/piggyback-add/new-group vs surrogate expected cost with per-device emergency mobility, exact mobility deltas): wash (e2_ls -4.7 +/- 76.8); add-only full-48 -33.4 +/- 29.1 with +2.0 swaps — strictly dominated by the simple p_be>=0.2 rule at the same appetite; the drop pass is actively harmful (+97.6). EV membership decisions inherit p42-scale noise, exactly the historical tspig failure, reconfirmed under the purge instrument. The champion planner is ~optimal given its beliefs; ALL remaining planner upside is in the beliefs.

B) FLEET-LEVEL DEATH FORECAST: THE WINNER, but only as a GATE, not a budget. As a budget floor (n = 0.8*max(sum_p42, r42)): -96.5 +/- 62.2 — real, but per-swap efficiency (-66) matches blanket appetite (-58 to -62): targeting the count adds ~nothing because conversion of extra slots is governed by per-origin ranking luck (s_8/s_9: late 2620->660, 2310->300; s_13/14/15: late UNCHANGED, pure waste). As a gate on the p_be top-up it is the most efficient appetite form measured (-89/extra swap). Diagnostic that motivated it: sum(p42) undercounts the arrival process at warmup (s_0: 2.0 vs 9 deaths) and mid-panel (s_15: 5.3, late 3210).

C) PORTFOLIO/CVaR SELECTION: DEAD. (i) Building-correlation premise falsified: marginal-zone (rank 8-40) devices die at 0.129 with NO recent building deaths vs 0.105 WITH; std logit coef of building recent-deaths 0.05 vs p42's 0.57. Deaths do not cluster by building beyond p42. (ii) Count-matched reordering by expected parked loss (p42 x early-death mass x 10/day + emergency-trip ops incl. travel/overtime): -4.4 +/- 67.6 wash — the p42 order already carries the value signal; day-quantile noise on marginal devices eats the rest.

Conclusion

The radical-pipeline premise "the planner leaks cost" is 3/4 refuted: WHEN, ORDER, and WHO-at-fixed-count are all at (or within noise of) their optimum given the champion's predictions. The one real leak is WHO-at-what- count in eras where the model's belief about fleet hazard is wrong, and the fleet's own death-arrival record (r42 + zone census) detects exactly those eras at plan time. e2_hgatez converts that detection into -245.8 +/- 88.3 (12.5% of SCORE) at +2.75 swaps. Recommend adjudicating e2_hgatez vs pl_pbe20 as the two forms of the same finding; prefer hgatez on variance, appetite, and deployment-asymmetry grounds.