Thibaut Forest
Claude Opus 5 (1M context)
Phase 56: NEW CHAMPION logbag_rank + gbm rank-blend w=0.30 (purge 1630.3, was 1972.0)
698401a |
Download agent_reports/ft_findings.md from thibautforest/BatterySwapAI2026-Example: direct link, hf CLI and curl.
- Browser
- Download file 4.59 kB
-
https://huggingface.co/thibautforest/BatterySwapAI2026-Example/resolve/main/agent_reports/ft_findings.md
- Command line
-
hf download hf://thibautforest/BatterySwapAI2026-Example/agent_reports/ft_findings.md
-
curl -L -o ft_findings.md https://huggingface.co/thibautforest/BatterySwapAI2026-Example/resolve/main/agent_reports/ft_findings.md
4.59 kB
| # ft — new feature engineering (temporal-purge CV) | |
| STATUS: DONE (last updated 2026-08-22 ~20:1x) | |
| Baseline: champion logbag/base SCORE 1972.0, swaps 9.21. All rows paired per-scenario vs /tmp/cv_all48oof.csv. | |
| Reference points from siblings: logbag_rank 1746.4 (-225.6), frontier logbag_rank+gbm-0.30 blend 1630.3 (-341.7). | |
| ## Results table (full-48 runs, sorted by delta vs champion 1972.0) | |
| | label | ALL48 | delta | SE | W/L | swaps | | |
| |---|---|---|---|---|---| | |
| | ft_fr_ddown | 1675.0 | -297.0 | 101.6 | 34/13 | 9.29 | | |
| | ft_fr_whit | 1748.9 | -223.1 | 84.9 | 32/15 | 9.15 | | |
| | ft_rkw | 1758.3 | -213.7 | 68.8 | 34/13 | 8.92 | | |
| | ft_fr_rwhit | 1782.3 | -189.7 | 81.9 | 31/16 | 8.96 | | |
| | ft_rkwd | 1794.2 | -177.8 | 68.1 | 32/15 | 8.94 | | |
| | ft_rwhit | 1922.7 | -49.3 | 41.0 | 29/19 | 8.96 | | |
| | ft_rkw2 | 1931.5 | -40.6 | 37.3 | 26/19 | 8.96 | | |
| | ft_rdip7 | 1961.9 | -10.1 | 28.4 | 24/17 | 9.27 | | |
| | ft_dip7 | 1968.0 | -4.0 | 33.2 | 20/23 | 9.31 | | |
| | ft_rddown | 1977.8 | +5.8 | 25.6 | 22/15 | 9.00 | | |
| | ft_whit | 2015.1 | +43.1 | 39.2 | 18/30 | 9.44 | | |
| | ft_wd | 2035.1 | +63.1 | 42.4 | 20/26 | 9.31 | | |
| CRITICAL READING: the negative deltas in the top half are NOT my features — they are the r_vlast/r_tau rank pair and the gbm blend those configs sit on. The correct paired comparisons: | |
| - ft_rkw vs logbag_rank alone: +11.9 SE 33.1 (30W/18L) — r_whit adds nothing significant. | |
| - ft_fr_ddown vs frontier: +44.8 SE 25.7, 17W/28L — DEGRADES. | |
| - ft_fr_whit vs frontier: +118.6 SE 48.9, 16W/32L — DEGRADES. | |
| - ft_fr_rwhit vs frontier: +152.1 SE 42.3, 16W/31L — DEGRADES. | |
| - ft_rkwd vs ft_rkw: r_dip7 costs ~36. | |
| - ft_rkw2 (r_whit REPLACING r_tau): -40.6 vs champion — collapses from rkw's -213.7. r_tau is load-bearing; whit is no substitute. | |
| ## What each feature was | |
| - whit: analytic Wiener 42d first-hit probability (staleness-conditioned inverse-Gaussian CDF from wnr_m/wnr_s2/v_corr_last, per-origin fleet-median shrinkage), fed as logit. r_whit = within-origin pct-rank. | |
| - dip7: v_last - v_min_7 (recent dip). newlow: v_min_30 - v_min_life. ddown: drawdown from running max of temp-corrected V (raw rebuild). | |
| - b250_60/b242_60: days below 2.50/2.42 V in last 60d (raw rebuild). readslope60: 60d slope of daily read count. macd: EWMA(10d)-EWMA(30d) momentum. divmm: min-vs-median slope divergence. All from a causal per-device rebuild of battery_metrics.parquet (/tmp/ft/rawfeat.parquet). | |
| - cohort_overdue/rel_age: install-batch calibrated clock (already in table, never in FEATURES). | |
| - ghost: (days_below_245>42 & v_corr_last<2.55) label-process ghost indicator; ft_gz = hard GHOST_ZERO. | |
| ## Triage graveyard (17 origins; cut, not confirmed) | |
| cohort +20/+43, b250_60 +66 (5W/11L), obsr +75, stsl +52, b242_60 +7, readsl +2, fiterr +22, ghost +44, ghostv -22 (noise), GHOST_ZERO +533 (swaps 8.5->5.6, count rule gutted — phase-15 non-adoption emphatically confirmed under the purge; historical: never adopted). | |
| ## Structural findings worth keeping | |
| 1. Slot squatters vs wall misses (purged fits, 9 origins): the alive devices occupying top-12 ranks are largely label-process ghosts (median days_below_245=43, b250_60=24.5); the missed deaths at ranks 13-30 sit HIGHER in voltage (v_corr 2.479 vs 2.441, tau_projected 23 vs 9). BUT ghost-flagged devices still die at 19% (vs 29% non-ghost at same depth), so both demoting them (feature) and zeroing them (GHOST_ZERO) lose more late-swap than they free in slots. The squatter problem is real but not solvable by a ghost flag. | |
| 2. Rank transforms are the transfer device on this instrument: raw whit +43 -> r_whit -49 (same information, rank form). Confirms the sibling logbag_rank finding from the feature side. | |
| 3. Offline marginal-signal screens (AUC beyond v_last) DO NOT predict cost: b250_60 screened best (0.643) and hurt worst; the "extra signal" pointed at ghosts. | |
| 4. 17-origin even-panel triage can cut losers but cannot confirm winners: ft_whit read -71 on the subset and +43 on the full 48 (the full panel block alone flipped to +57). | |
| ## Conclusion / recommendation | |
| NO FEATURE SURVIVOR. Recommend the frontier config UNCHANGED (logbag_rank + gbm-0.30, 1630.3) — every tested addition (whit raw/ranked, dip7, ddown, and by triage all others) is neutral-to-harmful on top of it. The champion's BASE_FEATURES remain fully distilled; the r_vlast/r_tau rank pair subsumes what my candidates offered. | |
| Scripts: /tmp/ft/run.py (all variants), /tmp/ft/rawbuild.py (+ /tmp/ft/rawfeat.parquet). | |
| CSVs: /tmp/cv_oofcand_ft_{whit,dip7,wd,rwhit,rdip7,rddown,rkw,rkwd,rkw2,fr_whit,fr_rwhit,fr_ddown}.csv; triage /tmp/ft/tri_ft_*.csv. | |