Thibaut Forest
Claude Opus 5 (1M context)
Phase 56: NEW CHAMPION logbag_rank + gbm rank-blend w=0.30 (purge 1630.3, was 1972.0)
698401a
|
Raw History Blame Contribute Delete
6.71 kB
# e2 — end-to-end plan optimization & fleet-level forecasting (radical pipeline)
STATUS: DONE — 2026-08-22 ~17:20. Top candidate: e2_hgatez (-245.8 +/- 88.3, clears 2*SE bar).
Baseline: champion logbag/tsroute, SCORE 1972.0 (warmup 3560.3 / panel 1839.6 / nearend 604.8), swaps 9.21.
Champion PREDICTIONS frozen & cached (48 temporal-purge logbag fits reproduce baseline EXACTLY: 1972.01, max|diff|=0.000000).
Every delta below is a pure PLANNER/BUDGET change on identical predictions. True eol only ever enters evaluate_plan.
Harness /tmp/e2/harness.py, cache /tmp/e2/preds.pkl, candidate CSVs /tmp/cv_oofcand_e2_*.csv.
## Results table (full-48, paired vs /tmp/cv_all48oof.csv; d/dsw = delta per extra swap)
| label | ALL48 | delta | SE | W/L | swaps | d/dsw | blocks (wu/pn/ne) | verdict |
|---|---|---|---|---|---|---|---|---|
| e2_hgatez | 1726.3 | -245.8 | 88.3 | 17/8 | 11.96 | -89 | -853 / -146 / 0 | SURVIVOR — top pick |
| e2_hgate | 1773.8 | -198.3 | 84.0 | 15/8 | 11.77 | -77 | -568 / -146 / 0 | survivor; superseded by hgatez |
| e2_hgate_b | 1785.2 | -186.8 | 84.9 | 14/9 | 12.04 | -66 | -568 / -130 / 0 | budget floor on top adds nothing |
| e2_nbud | 1875.5 | -96.5 | 62.2 | 11/10 | 10.67 | -66 | -211 / -87 / 0 | appetite-equivalent, not adopted |
| e2_a093 | 1908.9 | -63.1 | 33.8 | 16/23 | 10.23 | -62 | -192 / -45 / +3 | control (blanket alpha 0.93) |
| e2_nbud075 | 1933.2 | -38.9 | 39.2 | 8/7 | 9.75 | -72 | -42 / -45 / 0 | muted beta loses the big wins |
| e2_lsadd | 1938.6 | -33.4 | 29.1 | 14/19 | 11.21 | -17 | -40 / -46 / +46 | surrogate-EV adds < simple p_be rule |
| e2_opt | 1970.3 | -1.7 | 1.6 | 21/18 | 9.21 | — | +1 / -3 / +2 | day placement is CONVERGED |
| triage-only: e2_ls -4.7+/-76.8 (9.71 sw), e2_cvar -4.4+/-67.6 (8.47 sw), e2_ls_pmax +97.6+/-57.9 (7.71 sw) | | | | | | | | washes / harmful |
## Top candidate: e2_hgatez — hazard-gated break-even top-up
Rule (all plan-time observable): let r42 = fleet deaths RECORDED in (T-42d, T], zone42 = census of devices
currently below the 2.45V corrected threshold that crossed within the last 42d (telemetry <= T).
If max(r42, zone42) > sum(p42): open the break-even top-up p_be=0.2 (include every device with p42 >= 0.2).
Else: plan EXACTLY as champion. Fires at ~22/48 origins (warmup s_0-2 via zone42 — the ~34d recording lag
blinds r42 there; high-hazard panel eras s_4-16, s_18-23 via r42; NEVER at s_24-47 calm/near-end).
- -245.8 +/- 88.3, 17W/8L, swaps 11.96 (+2.75). Clears the survivor bar (2*SE = 177) despite the swap rise.
- Head-to-head vs pl_pbe20 (same p_be mechanism, always-on, -254.0 +/- 98.5, swaps 14.69): means statistically
identical (+8.3 +/- 44.6), hgatez better on 14 of 20 differing origins, lower per-origin std (612 vs 683),
2.7 fewer swaps/scenario, nearend swaps unchanged (16.7 = base) vs pbe20's 18.5.
- Where the gate closes, pbe20 is bimodal: it forfeits 2 big wins the gate misses (s_3 -979, s_17 -1374 — origins
where the COUNT was adequate but the RANKING was broken; not plan-time detectable) but avoids 14 losses
averaging +220 (calm origins where extra swaps caught nothing).
- Deployment asymmetry: at near-end (most deployment-like) origins the gate never fires — adopting it cannot
change the submission plan unless recorded arrivals demonstrably exceed the model's expected count at runtime.
It is a safety net, not a recalibration.
COORDINATION: e2_hgatez and pl_pbe20 are ONE mechanism family (gated vs always-on p_be) — adjudicate together,
never stack. If appetite-driven wins are distrusted (static appetite was publicly refuted historically), the
gated form is the defensible variant: swaps stay at champion level except under an observable hazard signal.
MULTIPLICITY: 10 full-48 planner variants were run; the gate signal was widened once (r42 -> max(r42, zone42))
after seeing full-48 results. p_be=0.2 itself was found independently by the pl agent. Selection bias is real;
the survivor margin (-245.8 vs bar -177) should be read with that in mind.
## Direction verdicts
A) SURROGATE-COST PLAN SEARCH: DEAD, with strong evidence. (i) Day placement/routing: 4x search effort
(restarts 8, iters 100, constructive start, NN routing) = -1.7 +/- 1.6 — the champion's optimize_days output is
~converged; no recoverable slack in WHEN/ORDER. (ii) Membership local search (drop/piggyback-add/new-group vs
surrogate expected cost with per-device emergency mobility, exact mobility deltas): wash (e2_ls -4.7 +/- 76.8);
add-only full-48 -33.4 +/- 29.1 with +2.0 swaps — strictly dominated by the simple p_be>=0.2 rule at the same
appetite; the drop pass is actively harmful (+97.6). EV membership decisions inherit p42-scale noise, exactly
the historical tspig failure, reconfirmed under the purge instrument. The champion planner is ~optimal given
its beliefs; ALL remaining planner upside is in the beliefs.
B) FLEET-LEVEL DEATH FORECAST: THE WINNER, but only as a GATE, not a budget. As a budget floor
(n = 0.8*max(sum_p42, r42)): -96.5 +/- 62.2 — real, but per-swap efficiency (-66) matches blanket appetite
(-58 to -62): targeting the count adds ~nothing because conversion of extra slots is governed by per-origin
ranking luck (s_8/s_9: late 2620->660, 2310->300; s_13/14/15: late UNCHANGED, pure waste). As a gate on the
p_be top-up it is the most efficient appetite form measured (-89/extra swap). Diagnostic that motivated it:
sum(p42) undercounts the arrival process at warmup (s_0: 2.0 vs 9 deaths) and mid-panel (s_15: 5.3, late 3210).
C) PORTFOLIO/CVaR SELECTION: DEAD. (i) Building-correlation premise falsified: marginal-zone (rank 8-40)
devices die at 0.129 with NO recent building deaths vs 0.105 WITH; std logit coef of building recent-deaths
0.05 vs p42's 0.57. Deaths do not cluster by building beyond p42. (ii) Count-matched reordering by expected
parked loss (p42 x early-death mass x 10/day + emergency-trip ops incl. travel/overtime): -4.4 +/- 67.6 wash —
the p42 order already carries the value signal; day-quantile noise on marginal devices eats the rest.
## Conclusion
The radical-pipeline premise "the planner leaks cost" is 3/4 refuted: WHEN, ORDER, and WHO-at-fixed-count are
all at (or within noise of) their optimum given the champion's predictions. The one real leak is WHO-at-what-
count in eras where the model's belief about fleet hazard is wrong, and the fleet's own death-arrival record
(r42 + zone census) detects exactly those eras at plan time. e2_hgatez converts that detection into -245.8
+/- 88.3 (12.5% of SCORE) at +2.75 swaps. Recommend adjudicating e2_hgatez vs pl_pbe20 as the two forms of the
same finding; prefer hgatez on variance, appetite, and deployment-asymmetry grounds.