Phase 11: price-adaptive break-even top-up (fresh-only, m0.8/pm25) + two-sided cost-curve fix

Panel 1702.0 vs 1715.8, 63% random-split selection, never beaten in a
decided blocked split. All-48 Docker metric 2175.4 (was 2202). Includes
user's phase-10 CohortGate + merged-block protocol write-up.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

EXPERIMENTS.md

@@ -278,9 +278,129 @@ block), deep sequence RUL (82 deaths), RL scheduling (sample complexity),
 MNAR state-space voltage models (gap features were already dev-neutral;
 highest-effort/lowest-evidence of the batch).
 
+## Phase 10 - late_swap gap: cohort-gated rescue (rejected; bucket is empty)
+
+Motivation: public-LB late/early = 1296/156 vs leader 660/658; misses→emergency
+is ~718/origin of test cost. Hypothesis: the phase-6 cohort clock (AUC 0.906 on
+silent pairs) was unfairly rejected because dev-era coverage was 35%, vs 72%
+test / 84% deployment-era - so deploy it *gated to covered devices only* and
+validate on the late-dev block s_18-23 where coverage is representative.
+
+Quantifying first where the cohort signal lives (covered pairs, s_18-41):
+
+| subgroup | n | deaths | cohort_overdue_m AUC |
+|---|---|---|---|
+| all covered | 6134 | 131 | 0.584 |
+| dark >14d / >30d (the actual miss bucket) | 1282/1144 | 26/23 | **0.426 / 0.410 - worse than chance** |
+| looks healthy (v_corr > 2.55) | 4734 | **4** | 0.902 |
+
+The 0.906 was real but rests on 4 positives: fast plungers that were still
+healthy at origin. Even perfect recall there is ~4 catches per 24 origins, and
+reaching them means including high-overdue healthy devices at hopeless
+precision (~60:1) x ~100 early-cost each.
+
+CV confirmation (`CohortGate`: logbag + per-device lift to a cohort-hazard
+logistic on covered devices; 'silent' mode lifts only p42 < 0.2), late-dev
+s_18-23: logbag 1943 · cg_all 2421 · cg_silent 2421 (identical plans - all
+lifts land in the p<0.2 bucket) · cg_silent α0.9 2588. **Rejected.**
+
+Conclusion on the late_swap gap: the remaining misses' plunge onset is *after*
+the planning origin. No origin-time feature in this dataset sees them: voltage
+is healthy, usage stress is flat (phase 8), FHT drift is flat (phase 9), and
+the batch clock is uninformative exactly where the misses live. The late/early
+imbalance vs the leader is not recoverable by rebalancing (phase 6 α-sweep) or
+by any tested signal; parity would require either mid-window replanning (not
+allowed by the submission format) or a signal source we do not have.
+
+## Protocol change (2026-08-18): dev and test merged into one 34-origin block
+
+After 8 counted looks the test block no longer provides an unbiased holdout,
+and dev s_8-23 is structurally unrepresentative of deployment (smaller refits,
+35% cohort coverage vs 84% at deployment-era origins). Before merging, every
+near-miss candidate without a test number was scored on s_24-41 (look 9):
+
+| model | dev16 | test18 | combined34 |
+|---|---|---|---|
+| **logbag (incumbent)** | 2073.9 | 1398.0 | **1716.1** |
+| logbag_rank | 2055.3 | 1446.0 | 1732.7 |
+| logbag_aux | 2035.5 | 1510.2 | 1757.4 |
+| dthaz | 2089.3 | 1502.6 | 1778.7 |
+| aux_proj | 2036.5 | 1596.7 | 1803.7 |
+
+The incumbent wins the combined block against every dev-winner - merging the
+blocks retroactively changes no decision ever made, which is the strongest
+available evidence the dev-only selection was sound. Going forward: tune on all
+34 origins (s_8-41) with a pre-registered acceptance rule (mean improvement
+AND per-origin win count AND no era-block regression); the public leaderboard
+is the only remaining holdout and is spent deliberately.
+
+**Repeated-split selection (adopted with the merge)**: training stays
+walk-forward (time-locked), but champion selection is resampled - choose the
+best config on a random half of the 34 origins, measure it on the other half,
+repeat (2000 iid splits + all 20 blocked 3-of-6-chunk splits, which respect
+window-overlap correlation). On the 5-finalist panel: logbag is the modal
+champion (63% iid / 50% blocked selection frequency, and just as often the
+true best on the held-out half); logbag_rank is the only live challenger
+(27%/35%); aux/dthaz variants essentially never win. Mean selection regret
+~38-64 cost units. The historical dev->test split turns out to have been an
+unlucky draw: on the current feature set, dev-16 alone would select
+logbag_rank while the test half's best is logbag. Acceptance rule for future
+candidates: selected in >=60% of random splits AND modal under blocked splits.
+
+## Phase 11 - price-adaptive inclusion (ADOPTED: break-even top-up, fresh-only, m0.8/pm25)
+
+Trigger: new public leader (sackriti 1477.48, late/early 804/343 + 179 in
+worker-limit penalties). Their +187 early for -492 late vs us is a 2.6:1
+winning trade; our own dev frontier re-measured that same trade LOSING at
+0.5:1 (α0.8→1.2: -213 late, +406 early). Not a modelling gap - a *price* gap.
+
+**The price structure**: a false swap costs 0.5/day against the imputed EOL
+(device end_time + 30d), so it depends on distance to the split's data end:
+dev origins 81-133 per false swap (day-42 placement), test 18-77 falling
+monotonically, s_41 17.7 - and ~0.5 for scenarios adjacent to a split's data
+end, which the runtime splits contain. α=0.8 is the CV-optimal response to
+dev's prices, not a universal constant. Crucially the runtime planner and the
+evaluator read the same /tmp/data metadata, so a price-based rule cannot
+disagree with the scorer about prices.
+
+**Counterfactual harness** (`/tmp/counterfactual.py`): same dev origins, same
+real deaths, same evaluator - end_time/EOL observability truncated at
+origin+50d to reproduce cheap pricing (~19/false swap) from local data only.
+There, break-even inclusion (ecroute m1.0) wins 1678 vs champion 1754; α
+widening still fails (+10 at best) - a fixed count cannot express "include
+everything break-even at today's prices".
+
+**Iterations under the merged-block protocol** (panel = all 34 origins +
+repeated-split selection):
+
+| candidate | panel mean | random-split sel% | verdict |
+|---|---|---|---|
+| champion (fixed α0.8) | 1715.8 | - | baseline |
+| ungated break-even top-up m1.0 | +24 vs champ on real dev, worst +461 | - | rejected: EV errors are punished where wrong costs ~100 |
+| + price gate pm25 (v1 e_far clip) | 1760.5 | 5.5% | rejected: clip made long-stale devices (imputed EOL in the past) look FREE; evaluator charges those swaps LATE 10/day (s_34 +411, s_36 +850, s_37 +668) |
+| + two-sided unclipped pricing | 1706.6 | 42.5% | closer; price-0 "stale, imputed EOL in-window" fiction still costs mobility (s_23/s_41 flips) |
+| + fresh-only (e_far >= 42), m1.0 | 1703.8 | 25.6% | s_39 -78 / s_40 -406 / s_41 +76 |
+| **+ fresh-only, m0.8** | **1702.0** | **63.0%** | **ADOPTED**: s_40 -438.7, s_41 -102.2, s_39 +72.5; 31/34 exact ties |
+
+Blocked splits: 50/50 with champion, but every champion "win" is an
+alphabetical tie-break on byte-identical costs (splits excluding the last
+chunk are exact ties); the hybrid wins every decided split. Mean selection
+regret drops 36 -> 8. The cost_curves fix (two-sided, unclipped survivor
+pricing) is byte-inert for the old champion on all 34 origins and is adopted
+as a pure correction.
+
+New champion: **logbag / tsroute α=0.8 + break-even top-up (ec_margin=0.8,
+ec_pmax=25, fresh devices only) + fixed cost curves**. The top-up is
+structurally empty at dev-like prices (all 16 dev origins byte-identical) and
+widens exactly where a wrong inclusion is provably cheap - which is where the
+runtime splits live. Expected effect is largest on public/private scenarios
+near their data end; the panel contains only ~3 such origins, so the local
+delta (-14) understates the deployable one.
+
 ## Test-look ledger
 
-8 looks total (threshold stack, topsum, ecroute+ablations, tsroute, aug-tsroute,
+8 looks total (now historical - see protocol change above); look 9 = the
+near-miss backfill table. Original ledger: 8 looks (threshold stack, topsum, ecroute+ablations, tsroute, aug-tsroute,
 +tau_projected, logbag+α-challenger, aux-features). Dev/test disagreed
 persistently by ~0.7σ (dev prefers richer stacks, test the simpler v1);
 selection followed dev + the combined mean, with the public leaderboard as

batteryswap_example/models_cv.py

@@ -736,6 +736,49 @@ class ConformalCut:
         return out
 
 
+
+class CohortGate:
+    """
+    logbag everywhere; on devices whose install batch has a calibrated monthly
+    clock (cohort_rel_age_m present), lift p42 to a cohort-hazard logistic's
+    probability when it is higher. mode 'silent' lifts only devices the voltage
+    model scores low (<0.2) - the miss bucket - leaving confident scores alone.
+    """
+
+    CFEATS = ['cohort_rel_age_m', 'cohort_overdue_m', 'age_days']
+
+    def __init__(self, mode='all'):
+        self.mode = mode
+
+    def fit(self, train):
+        self.base = SklearnRUL('logbag').fit(train)
+        cov = train[train['cohort_rel_age_m'].notna()]
+        self.cmodel = None
+        if len(cov) >= 300 and cov['dies42'].sum() >= 10:
+            self.cmodel = make_pipeline(SimpleImputer(strategy='median'),
+                                        StandardScaler(),
+                                        LogisticRegression(max_iter=2000, C=1.0))
+            self.cmodel.fit(cov[self.CFEATS], cov['dies42'])
+        return self
+
+    def predict(self, rows):
+        out = self.base.predict(rows)
+        if self.cmodel is None:
+            return out
+        covered = rows['cohort_rel_age_m'].notna().values
+        if not covered.any():
+            return out
+        pc = np.zeros(len(rows))
+        pc[covered] = self.cmodel.predict_proba(
+            rows.loc[covered, self.CFEATS])[:, 1]
+        p = out['p42'].values
+        lift = covered & (pc > p)
+        if self.mode == 'silent':
+            lift &= p < 0.2
+        out['p42'] = np.where(lift, pc, p)
+        return out
+
+
 MODELS = {
     'rank-v': RankVoltage,
     'logistic': lambda: SklearnRUL('logistic'),
@@ -787,6 +830,8 @@ MODELS = {
     'crt_in50': lambda: ConformalCut(0.50, 'in'),
     'logbag_fht': lambda: LogBagFeats(['fht_days', 'wnr_m', 'wnr_s2']),
     'logbag_fht2': lambda: LogBagFeats(['fht_days']),
+    'cg_all': lambda: CohortGate('all'),
+    'cg_silent': lambda: CohortGate('silent'),
 }
 
 
@@ -953,13 +998,19 @@ def cost_curves(pred, locations, start_time, blend=0.0):
     t = np.arange(1, WINDOW_DAYS + 1, dtype=float)
 
     end = locations.set_index('battery')['end_time'].dt.tz_localize(None)
+    # unclipped: for long-stale devices the imputed EOL (last data + 30d) can
+    # precede the window, and the evaluator then charges a swap as LATE at
+    # 10/day - the old clip(lower=43) made exactly those devices look free
     e_far = ((end.reindex(pred.index) - start_time).dt.total_seconds() / 86400
-             + 30.0).clip(lower=WINDOW_DAYS + 1).values
+             + 30.0).values
 
     delta = grid[None, :, None] - t[None, None, :]        # 1 x 43 x 42
     pen = np.where(delta >= 0, LATE * delta, EARLY * (-delta))
     curves = p42[:, None] * (pmf[:, None, :] * pen).sum(axis=2)
-    curves += (1 - p42)[:, None] * EARLY * np.maximum(e_far[:, None] - grid[None, :], 0)
+    surv_delta = e_far[:, None] - grid[None, :]
+    curves += (1 - p42)[:, None] * np.where(surv_delta >= 0,
+                                            EARLY * surv_delta,
+                                            LATE * -surv_delta)
 
     park = p42 * (pmf * (LATE * (EMERGENCY_DAY - t))[None, :]).sum(axis=1)
     return curves, park
@@ -1147,10 +1198,34 @@ def plan_tsroute(pred, included, locations, start_time, day_q,
     if plan_tsroute.p_be > 0:
         topups = pred.index[pred['p42'] >= plan_tsroute.p_be]
         chosen = chosen.union(topups)
+    curves, park = cost_curves(pred, locations, start_time)
+    if plan_tsroute.ec_margin > 0:
+        # break-even top-up: include every battery whose best swap day is
+        # cheaper than parking it. The false-swap price is 0.5/day against the
+        # imputed EOL (end_time+30), so this set is tiny when data extends far
+        # past the window (train dev block) and widens automatically when the
+        # scenario sits near the split's data end (runtime), where false swaps
+        # cost almost nothing but misses still cost 10/day.
+        # ec_pmax guards the p42/curve estimation error: the top-up only fires
+        # for batteries whose worst-case cost of a wrong inclusion (false swap
+        # at day 42) is at most ec_pmax - EV says include, price caps the risk.
+        # Real-dev CV rejected the ungated rule (+24, worst origin +461); the
+        # counterfactual cheap-price CV shows -76 where wrong costs ~19.
+        end = locations.set_index('battery')['end_time'].dt.tz_localize(None)
+        e_far = ((end.reindex(pred.index) - start_time).dt.total_seconds() / 86400
+                 + 30.0).values
+        # top-up only fresh devices (imputed EOL beyond the window): for stale
+        # ones the survivor branch is a bookkeeping fiction - swapping "on the
+        # imputed day" scores ~0 but buys nothing and costs real mobility
+        # (panel: admitting them turned s_23/s_41 from ties into losses)
+        fp_price = EARLY * (e_far - WINDOW_DAYS)
+        be = pred.index[(e_far >= WINDOW_DAYS)
+                        & (curves.min(axis=1) < plan_tsroute.ec_margin * park)
+                        & (fp_price <= plan_tsroute.ec_pmax)]
+        chosen = chosen.union(be)
     if len(chosen) == 0:
         return assemble(pd.Series(dtype='datetime64[ns]'), locations, start_time)
 
-    curves, _ = cost_curves(pred, locations, start_time)
     if plan_tsroute.floor_day > 0:
         uncertain = (pred['p42'] < plan_tsroute.p_conf).values
         curves[np.ix_(uncertain, np.arange(plan_tsroute.floor_day))] = np.inf
@@ -1173,6 +1248,8 @@ plan_tsroute.alpha = 0.8
 plan_tsroute.p_be = 0.0
 plan_tsroute.floor_day = 0
 plan_tsroute.p_conf = 0.5
+plan_tsroute.ec_margin = 0.0
+plan_tsroute.ec_pmax = 25.0
 
 def plan_tspig(pred, included, locations, start_time, day_q,
                travel_costs=None, settings=None):
@@ -1407,6 +1484,8 @@ def walk_forward(table, scenario_ctx, origins, configs):
                 PLANNERS['tsroute'].p_be = cfg.get('p_be', 0.0)
                 PLANNERS['tsroute'].floor_day = cfg.get('floor_day', 0)
                 PLANNERS['tsroute'].p_conf = cfg.get('p_conf', 0.5)
+                PLANNERS['tsroute'].ec_margin = cfg.get('ec_margin', 0.0)
+                PLANNERS['tsroute'].ec_pmax = cfg.get('ec_pmax', 25.0)
             global EMERGENCY_DAY
             EMERGENCY_DAY = cfg.get('emergency_day', WINDOW_DAYS + 3)
             optimize_days.restarts = int(cfg.get('restarts', 2))
@@ -1447,7 +1526,7 @@ def summarize(df, label):
                   df['p_star'].astype(str) + '/q' + df['day_q'].astype(str))
     for key, tag in [('blend', 'b'), ('p_be', 'be'), ('floor_day', 'f'), ('p_conf', 'c'),
                      ('alpha2', 'a2'), ('emergency_day', 'e'), ('restarts', 'r'),
-                     ('iters', 'i')]:
+                     ('iters', 'i'), ('ec_margin', 'ec'), ('ec_pmax', 'pm')]:
         if key in df.columns and df[key].nunique(dropna=False) > 1:
             df['name'] += '/' + tag + df[key].fillna(0).astype(str)
     mean = df.groupby('name')['total_cost'].agg(['mean', 'std']).sort_values('mean')
@@ -1456,6 +1535,67 @@ def summarize(df, label):
     return mean
 
 
+ALL_ORIGINS = range(8, 42)  # merged dev+test block (protocol 2026-08-18)
+
+
+def selection_stability(df, n_random=2000, n_blocks=6, seed=0):
+    """
+    Repeated-split champion selection over the merged origin block.
+
+    Training is walk-forward and time-locked, so only the *selection* step is
+    resampled: choose the best config on half the origins, measure it on the
+    other half. Random iid splits estimate selection frequency; blocked splits
+    (contiguous chunks) respect the correlation between origins whose 42-day
+    windows overlap. Acceptance rule (EXPERIMENTS.md): a challenger should be
+    selected in >= 60% of random splits AND be modal under blocked splits.
+    """
+    from itertools import combinations
+
+    df = df.copy()
+    df['name'] = (df['model'] + '/' + df['planner'] + '/p' +
+                  df['p_star'].astype(str))
+    # distinguish configs that differ only in planner knobs (same tags as summarize)
+    for key, tag in [('blend', 'b'), ('p_be', 'be'), ('floor_day', 'f'), ('p_conf', 'c'),
+                     ('alpha2', 'a2'), ('emergency_day', 'e'), ('restarts', 'r'),
+                     ('iters', 'i'), ('ec_margin', 'ec'), ('ec_pmax', 'pm')]:
+        if key in df.columns and df[key].nunique(dropna=False) > 1:
+            df['name'] += '/' + tag + df[key].fillna(0).astype(str)
+    pv = df.pivot_table(index='origin', columns='name', values='total_cost')
+    pv = pv.sort_index()
+    names = pv.columns.tolist()
+    X = pv.values
+    n = len(pv)
+    rng = np.random.RandomState(seed)
+
+    def run(splits, label):
+        sel = {m: 0 for m in names}
+        best = {m: 0 for m in names}
+        regrets = []
+        for A in splits:
+            B = np.setdiff1d(np.arange(n), A)
+            i = X[A].mean(0).argmin()
+            j = X[B].mean(0).argmin()
+            sel[names[i]] += 1
+            best[names[j]] += 1
+            regrets.append(X[B].mean(0)[i] - X[B].mean(0)[j])
+        k = len(splits)
+        print(f'\n--- selection stability: {label} ({k} splits) ---')
+        print(f"{'config':40s} {'selected%':>9s} {'best-holdout%':>13s} {'mean':>8s}")
+        for m in sorted(names, key=lambda m: -sel[m]):
+            print(f'{m:40s} {100*sel[m]/k:9.1f} {100*best[m]/k:13.1f} '
+                  f'{X[:, names.index(m)].mean():8.1f}')
+        print(f'mean selection regret {np.mean(regrets):.1f} '
+              f'(90th pct {np.percentile(regrets, 90):.1f})')
+
+    half = n // 2
+    run([rng.choice(n, half, replace=False) for _ in range(n_random)],
+        f'random {half}/{n - half}')
+    blocks = np.array_split(np.arange(n), n_blocks)
+    run([np.concatenate([blocks[i] for i in c])
+         for c in combinations(range(n_blocks), n_blocks // 2)],
+        f'blocked ({n_blocks} chunks choose {n_blocks // 2})')
+
+
 def main():
     dataset_path = Path(os.environ.get('BATTERYSWAP_DATASET_PATH', 'dataset'))
     table = load_features(dataset_path)
@@ -1498,6 +1638,15 @@ def main():
         df = walk_forward(table, ctx, DEV_ORIGINS, configs)
         df.to_csv('/tmp/dev_results.csv', index=False)
         summarize(df, f'DEV s_8..s_23 ({len(configs)} configs)')
+    elif phase == 'panel':
+        # merged-block evaluation (protocol of 2026-08-18): score configs on all
+        # 34 usable origins, then repeated-split selection stability
+        import json
+        configs = json.loads(Path('/tmp/panel_configs.json').read_text())
+        df = walk_forward(table, ctx, ALL_ORIGINS, configs)
+        df.to_csv('/tmp/panel_results.csv', index=False)
+        summarize(df, f'PANEL s_{ALL_ORIGINS[0]}..s_{ALL_ORIGINS[-1]}')
+        selection_stability(df)
     else:
         # phase 'dev2' or 'test': configs from /tmp/<phase>_configs.json
         import json

batteryswap_example/planner.py

@@ -101,6 +101,8 @@ class ECRoutePlanner(Planner):
         feats = feats.join(phys).reindex(locations['battery'])
         pred = self.model.predict(feats)
         plan_tsroute.alpha = 0.8
+        plan_tsroute.ec_margin = 0.8   # phase-11 break-even top-up
+        plan_tsroute.ec_pmax = 25.0
         return plan_tsroute(pred, None, locations, origin, 0.05,
                             travel_costs=travel_costs, settings=settings)
 

batteryswap_example/planners/best.pickle

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:d768c5ac63b5308c6d0c446fa7e468fdb78fc94df843d8f9d6a8dc9ac71e288c
-size 318867
+oid sha256:1d263e1282560dc608868cb9ce3360007c2230b360bd4ea272b260bd3b08b0d0
+size 318953

batteryswap_example/submit.py

@@ -43,6 +43,14 @@ MODEL_KIND = 'logbag'
 # late saturates ~1100 regardless - the remaining misses are voltage-silent
 # deaths that no inclusion appetite can catch.
 ALPHA = float(os.environ.get('BATTERYSWAP_ALPHA', '0.8'))
+# Break-even top-up (phase 11): additionally include every FRESH battery
+# (imputed EOL beyond the window) whose best-day expected cost beats parking,
+# gated to fire only where a wrong inclusion costs <= EC_PMAX. Empty at
+# dev-like prices; widens near a split's data end, where false swaps are
+# nearly free but misses still cost 10/day. Panel 1702.0 vs 1715.8, selected
+# in 63% of random half-splits, never beaten in a decided blocked split.
+EC_MARGIN = float(os.environ.get('BATTERYSWAP_EC_MARGIN', '0.8'))
+EC_PMAX = float(os.environ.get('BATTERYSWAP_EC_PMAX', '25'))
 
 
 def build_split_table(locations, timeseries, eol_times, scen_starts, log=print):
@@ -118,6 +126,8 @@ def adaptive_split_plans(split, split_path, log=print, deadline=None):
 
         pred = model.predict(feats)
         plan_tsroute.alpha = ALPHA
+        plan_tsroute.ec_margin = EC_MARGIN
+        plan_tsroute.ec_pmax = EC_PMAX
         plan = plan_tsroute(pred, None, locs, T, 0.05,
                             travel_costs=scenario['travel_costs'],
                             settings=scenario['settings'])