|
Download agent_reports_bgkf/t5_base.md from thibautforest/BatterySwapAI2026-Example: direct link, hf CLI and curl.
- Browser
- Download file 6.01 kB
-
https://huggingface.co/thibautforest/BatterySwapAI2026-Example/resolve/main/agent_reports_bgkf/t5_base.md
- Command line
-
hf download hf://thibautforest/BatterySwapAI2026-Example/agent_reports_bgkf/t5_base.md
-
curl -L -o t5_base.md https://huggingface.co/thibautforest/BatterySwapAI2026-Example/resolve/main/agent_reports_bgkf/t5_base.md
6.01 kB
| # T5 BASE-REBUILD lane: wide-feature stacked nonlinear base vs the linear order-primary — REJECTED (wash, not capacity-limited) | |
| Date: 2026-08-23. Lane: BASE-REBUILD (T5 global-optimum probe). | |
| Baseline: head config 60d65da, CV 1541.3 on bgkf (`/tmp/ref_head_1541.csv`; | |
| parity re-verified through this lane's own driver: 1541.3, +0.0, swaps 2.27). | |
| Champion order = .50*rank(logbag vm7) + .30*rank(WienerFHT) + .20*rank(gapHGB | |
| on FEATURES+vmin7_rank+GAP4); values/counts always champion logbag (order-only | |
| via blend w2=1.0). | |
| ## Hypothesis | |
| The champion's order primary is a bagged LOGISTIC on ~14 hand-picked snapshot | |
| features. The rank-4 public competitor's base is a stacked nonlinear ensemble | |
| over a wide multi-scale feature space. Grafts onto our pipeline all failed | |
| (saturated local optimum) — this lane REBUILT the base as a unit: wide table + | |
| stacked ensemble, and tested it as order primary / full order / diversity | |
| partner. | |
| ## Build | |
| 1. WIDE TABLE (`/tmp/c29/build_wide.py` -> `/tmp/c29/wide.parquet`): 78 | |
| feature cols per (device_id, origin_idx), all 168 origins (real+synthetic), | |
| strictly as-of-origin from the daily corrected-voltage series (beta 0.005 | |
| fixed): rolling {3,7,14,30,60,90,180,365}d x {mean,min,max,std,coverage, | |
| masked-OLS slope}; window contrasts (mean/min/slope 7-30, 14-90, 30-180, | |
| 60-365); days-since-min {30,90,180,365}; temp means {7,30,90}; | |
| v_last_c/data_gap_days; within-snapshot pct-ranks of v_last_c and 7d-min | |
| globally/per-building/per-room; building/room device counts. Leak-freedom | |
| verified by recomputing 25 random (device,origin) rows from the raw series | |
| truncated at origin — 0 mismatches beyond float32 precision (2e-5). | |
| Coverage >= 86% on every column. | |
| 2. STACKED BASE (`/tmp/c29/run.py` StackBase): learners = HGB(depth3, lr.06, | |
| 300it), HGB(depth2, lr.03, 600it), RF(350, min_leaf 5) on wide+FEATURES+ | |
| vmin7_rank+GAP4 (~96 cols), LogisticRegression on the champion's scaled | |
| feature set; building-GroupKFold(4) OOF probabilities (deaths-sorted snake, | |
| deterministic — the established contamination rule) -> LogisticRegression | |
| (C=0.03) calibrator; learners refit on full fold-train. Fold-safe inside | |
| .fit(train); memoized per outer fold. cv.py/models_cv.py md5-pinned per arm | |
| (race guard); no repo files touched. | |
| ## Results (paired vs head 1541.3, 48 scenarios; bgkf, p_star=1.0, split 3, tw 2.0) | |
| | arm | order | SCORE | delta±SE | sig | W/L | wilcoxon | swaps | | |
| |---|---|---|---|---|---|---|---| | |
| | parity | champion .50/.30/.20 rebuilt | 1541.3 | +0.0±0.0 | 0.4σ | 1W/1L | – | 2.27 | | |
| | arm_a | stack replaces logbag at .50 (+.30 wn +.20 gap) | 1549.4 | +8.1±48.0 | 0.2σ | 25W/19L | p0.441 | 2.28 | | |
| | arm_b | stack as FULL order (w 1.0) | 1533.1 | -8.2±55.1 | 0.1σ | 30W/17L | p0.295 | 2.26 | | |
| | arm_c | .50*rank(champ composite) + .50*rank(stack) | 1532.2 | -9.2±36.6 | 0.3σ | 24W/19L | p0.583 | 2.28 | | |
| No arm reaches 1σ (adoption bar for a new mechanism: 2σ). Weight mini-sweep | |
| not triggered (rule: only if a/c < 0 at >= 1σ). Swaps flat 2.26-2.28 | |
| everywhere — count-flat as designed. | |
| Block structure (delta means): every arm WINS warmup s0-7 (arm_a -78, arm_b | |
| -262, arm_c -104) and LOSES the panel s8-41 (+27 / +48 / +9); nearend ~+13. | |
| The tiny negative means are entirely the 8 warmup scenarios; the 34-scenario | |
| panel prefers the champion. Deltas are heavy-tailed (arm_b: s21 +1545, s0 | |
| -810, s12 +794, s28 -757) — hence SE 37-55 and no resolvable effect. | |
| ## AUC comparison (dies42, building-OOF, closed rows at real origins) | |
| | fold | n_test | deaths | logbag | STACK | wiener | gapHGB | | |
| |---|---|---|---|---|---|---| | |
| | 0 | 3056 | 123 | 0.962 | **0.974** | 0.929 | 0.969 | | |
| | 1 | 1987 | 91 | 0.963 | **0.979** | 0.930 | 0.970 | | |
| | 2 | 3661 | 80 | **0.968** | 0.962 | 0.939 | 0.974 | | |
| | 3 | 967 | 71 | 0.837 | **0.930** | 0.833 | 0.893 | | |
| | 4 | 7917 | 61 | 0.958 | **0.978** | 0.965 | 0.963 | | |
| Stack beats logbag in 4/5 folds (mean per-fold 0.965 vs 0.938; +9.3 pts on | |
| the hard fold 3, exactly where logbag collapses). Pooled-across-folds AUC is | |
| lower for the stack (0.948 vs 0.957) only because its probability scale varies | |
| more across fold fits — per-fold is the fair capacity read. Inner-OOF AUCs per | |
| learner: h3 .947-.977, h2 .955-.977, rf .960-.977, lr .956-.972, meta .972-.981 | |
| — the linear learner is competitive with the trees even inside the wide space. | |
| ## Verdict | |
| REJECTED — hypothesis answered, both halves: | |
| 1. **The capacity is real.** The rebuilt wide-space stacked base is a genuinely | |
| better building-OOF dies42 classifier than the champion logbag (4/5 folds, | |
| large margin where training deaths are scarce). The competitor-style base | |
| CAN be built on this data and it does classify better. This is not a | |
| feature-engineering or model-capacity failure. | |
| 2. **The capacity does not price.** At every mixing weight (0.5 primary, 1.0 | |
| full, 0.5 diversity ensemble) plan cost is a wash (|delta| <= 9.2, <= 0.3σ, | |
| wilcoxon p >= 0.29). Its classification edge concentrates in warmup/sparse- | |
| label regimes (fold 3, s0-7) where scenario costs are small; in the panel | |
| block that dominates THE SCORE, logbag's calibrated top ranks near the | |
| inclusion cutoff are already sufficient, and the stack's reorderings there | |
| are net-slightly-negative. Same mechanism as the t4 stacking lane | |
| (`t4_stack.md`): dies42-AUC gains live in separating obvious cases, cost | |
| lives in the fine ordering near the cutoff. | |
| The linear order-primary is genuinely sufficient at this data size (461 | |
| devices, ~1130 death-rows): base-model capacity is NOT the binding constraint | |
| on bgkf cost. Do not revisit base-model family swaps as a cost lever; any | |
| future gain must come from a signal that reorders the marginal band | |
| specifically (or from count/timing machinery, which is closed/tiered | |
| separately). | |
| Artifacts: /tmp/c29/build_wide.py, /tmp/c29/wide.parquet, /tmp/c29/run.py, | |
| /tmp/c29/arms.out (incl. full AUC diag), /tmp/cv_bgkfcand_c29_{parity,arm_a, | |
| arm_b,arm_c}.csv. | |