Karar-git's picture
Add FINAL-2 pipeline, overnight search scripts and results (11 Nov 2026)
2c2dd6c verified
|
Raw History Blame Contribute Delete
2.96 kB

Final picks, fixed before confirmation (2026-11-11 01:36)

Written before any score on half B alone was tabulated and before draws 5-11 existed for the ensemble members.

What was used to choose

  • Half A of Amsterdam (checkerboard of 2 km blocks, lab.half_A), draws 0-4, python tools/nnlab/lab.py board _A.
  • Honest caveat: full-city scores (A + B together) of the single models and of ensembles e1-e3 were printed earlier in the night, so B was not perfectly blind. B on its own was never looked at.

The picks

name pipeline decision
FINAL-AVG ens_e4__lp: fixed-rule ensemble (zero-shot XGBoost+DANN with weight 10/(n+10); the rest shared equally by retrained LightGBM, met_protoda_maha, met_cos_maha, tab_dann_rz_ft), then label propagation with the shots clamped rule + neighbour smoothing
FINAL-PEAK ens_e2__lapshot: zero-shot XGBoost+DANN (weight 10/(n+10)) + retrained LightGBM, then LaplacianShot-style graph refinement rule + neighbour smoothing
FINAL-SWITCH n <= 50: FINAL-AVG; n >= 100: FINAL-PEAK (n is an input of modelCreate, so this is one method) as above

Half A, draws 0-4 (the evidence): ens_e4__lp 0.7033 / 0.7109 / 0.7290 / 0.7355 / 0.7392; ens_e2__lapshot 0.6856 / 0.6952 / 0.7172 / 0.7376 / 0.7524.

Why e4 and not e6 / e7 (which add a self-trained LightGBM or LDA): they differ from e4 by under 0.001 on half A, which is noise, and e4 is one stage simpler.

Confirmation to be reported, whatever it shows

  1. Half B, draws 0-11.
  2. All pixels, draws 5-11 only (draws never used for choosing).
  3. The same for the reference: retrained LightGBM with the rule (0.6563 average).

Round 2 pick, fixed before its confirmation (2026-11-11 03:26)

A LightGBM variant from round 2, lgbm_feda (Daume feature augmentation: shared, Madrid-only and Amsterdam-only copies of the top 120 features; everything else as the retrained LightGBM), scored well above the others on draws 0-4.

name pipeline decision
FINAL-2 ens_e12: zero-shot XGBoost+DANN with weight 10/(n+10) + lgbm_feda with the rest rule + neighbour smoothing
FINAL-2 without coordinates the same rule only
single model lgbm_feda rule + smoothing, and rule only

Evidence used: half A, draws 0-4. ens_e12 rule_smooth 0.7052 / 0.7161 / 0.7410 / 0.7481 / 0.7591 (avg 0.734) against ens_e4 0.7139 avg; lgbm_feda alone 0.7236 avg. Ensembles that keep the metric-learning and neural members next to lgbm_feda (e10, e11) were lower on half A (0.721, 0.724) and are not picked.

Sanity check done before confirmation: training lgbm_feda with the shot labels shuffled collapses macro F1 to 0.32 (n=5) and 0.23 (n=200), so the gain comes from the shot labels.

Confirmation to be reported whatever it shows: all pixels on draws 5-11, on which lgbm_feda has never been run, and half B on draws 0-11. lgbm_fedaw100 (feda + shot weight 100) is exploratory only, draws 0-4.