Add FINAL-2 pipeline, overnight search scripts and results (11 Nov 2026)
fe9543f verified 4 days ago
26.7 kB
---
tags:
- tabular-classification
- lightgbm
- remote-sensing
- landsat
- domain-adaptation
- few-shot
library_name: lightgbm
---
# Building age from Landsat: Madrid → Amsterdam (Hack4Dev 2026)

This repo holds the models from our Iraq Hackathon 2026 / Hack4Dev entry
"Building Age Classification from Satellite Imagery", together with the code,
data and notebooks needed to reproduce them or compare against them. It has
our best pipeline (FINAL-2, added on 11 November 2026) and the two LightGBM
models of the first release.

**The task.** Predict a building-age class (1–4) for each 30 m Landsat pixel.
Train on Madrid and transfer to Amsterdam. Scoring is macro F1 on Amsterdam
after you are given **n = 5, 25, 50, 100 or 200 labelled Amsterdam pixels per
class** (the few-shot protocol).

| class | Amsterdam | Madrid |
|---|---|---|
| 1 | before 1945 | before 1960 |
| 2 | 1945–1984 | 1960–1984 |
| 3 | 1984–2004 | 1984–2004 |
| 4 | 2004–2024 | 2004–2024 |

## Update, 11 November 2026: the best pipeline (FINAL-2)

After the first release we scored about 130 more model variants in one night
on a laptop CPU: neural networks with a frozen backbone, metric learning,
SVMs and other classical models, and ideas from the domain-adaptation
literature. The best result combines two models with a fixed weighting and a
fixed decision rule:

1. **Zero-shot anchor.** A teammate's XGBoost + DANN design on the organisers'
   60 features, re-implemented by us on CPU. It is trained on Madrid labels
   and unlabelled Amsterdam features, and never sees an Amsterdam label.
2. **LightGBM with domain copies.** The retrained LightGBM from the first
   release with one change, Daumé's "frustratingly easy" domain adaptation.
   The 120 most important features each get two extra copies: one filled only
   on Madrid rows and one filled only on Amsterdam rows.
3. **Combine.** Average the two models' log-probabilities. The zero-shot model
   gets weight 10/(n+10) and the LightGBM gets the rest. No weight is fitted.
4. **Decide.** Smooth each pixel's probabilities with its neighbours (radius 6
   pixels, the teammate's setting, which they report choosing on Madrid).
   Then apply our decision rule: tau = 0.5 logit adjustment with the Madrid
   class prior, then the macro-F1 plug-in. The rule was chosen on Madrid.

The XGBoost + DANN anchor and the neighbour smoothing are a teammate's
design. Their functions are in `tools/compare_teammate.py`, published with
their agreement.

Macro F1 on all of Amsterdam, on draws 5–11 (seven per n). These draws were
never used to choose anything.

| pipeline | n=5 | n=25 | n=50 | n=100 | n=200 | average |
|---|---|---|---|---|---|---|
| **FINAL-2, with neighbour smoothing** | 0.672 | 0.705 | 0.716 | 0.732 | 0.744 | **0.714** |
| FINAL-2 without coordinates (no smoothing) | 0.652 | 0.694 | 0.702 | 0.720 | 0.734 | 0.700 |
| LightGBM with domain copies alone, with smoothing | 0.623 | 0.699 | 0.712 | 0.730 | 0.744 | 0.702 |
| LightGBM with domain copies alone, no smoothing | 0.607 | 0.687 | 0.698 | 0.718 | 0.734 | 0.689 |
| Round-1 five-model ensemble + label propagation, with smoothing | 0.677 | 0.705 | 0.714 | 0.725 | 0.728 | 0.710 |
| Retrained LightGBM (first release), with smoothing | 0.587 | 0.652 | 0.680 | 0.712 | 0.732 | 0.673 |
| Retrained LightGBM (first release), no smoothing | 0.559 | 0.629 | 0.660 | 0.697 | 0.721 | 0.653 |
| Teammate's zero-shot XGBoost + DANN (our re-implementation), with smoothing and our decision rule | 0.654 | 0.653 | 0.653 | 0.652 | 0.651 | 0.653 |

- **Compare like with like.** With smoothing on both sides FINAL-2 averages
  0.714 against 0.673 for the retrained LightGBM. Without coordinates on both
  sides it averages 0.700 against 0.653.
- **No pixel coordinates?** Quote the second row, 0.700.
- **What the anchor adds.** Against the LightGBM with domain copies alone,
  both smoothed, FINAL-2 gains about 0.05 at n=5 and nothing at n=200 (0.744
  for both).
- **The margin over the round-1 ensemble is small.** FINAL-2 is 0.004 ahead
  on average. At n=5 the ensemble is 0.004 ahead, which is inside the draw
  noise. We prefer FINAL-2 because it is simpler: two models and no network
  fitted per draw.
- **The draws differ from the first-release table.** The retrained LightGBM
  averages 0.653 here and 0.656 in the table further down, which uses draws
  0–4.
- **Standard deviation over the seven draws** for FINAL-2: 0.007 / 0.012 /
  0.015 / 0.004 / 0.003 at n = 5 / 25 / 50 / 100 / 200.

![macro F1 against labelling budget, overnight search](reports/nnlab/f1_vs_n_overnight.png)

Run it for one draw. It takes one to two minutes. Set `NNLAB_THREADS` to your
core count, because the default is 3.

```bash
python tools/nnlab/model_create2.py --n 25 --rep 6
python tools/nnlab/model_create2.py --n 200 --rep 3 --no-spatial
```

Expected output, from a run inside a clean copy of this repo:

```
FINAL-2 n=25 rep=6 spatial=True: macro F1 0.696921
FINAL-2 n=200 rep=3 spatial=False: macro F1 0.735816
```

From Python: `model_create(shot_idx, shot_labels).predict()` in
`tools/nnlab/model_create2.py`. The full write-up is
`reports/nnlab/OVERNIGHT_RESULTS.md`.

### Caveats for FINAL-2

- **The choice of models was made on Amsterdam.** About 130 variants were
  scored on all of Amsterdam on draws 0–4. The LightGBM with domain copies
  was picked from those full-city scores, and the two-model combination was
  picked on half of the city. FINAL-2 is the second of two rounds of picks. A
  round-1 rule (switch pipelines at n=100) failed its confirmation and was
  dropped. Only draws 5–11 were never used for choosing. FINAL-2 reads 0.005
  lower on them than on draws 0–4, and the review estimated 0.005 to 0.01 of
  optimism for the top picks.
- **It uses more of the unlabelled target city.** The DANN inside the anchor
  is trained against unlabelled Amsterdam features, each city is scaled with
  its own statistics, and the smoothing needs pixel coordinates.
- **Batch-level.** The decision rule and the smoothing label the whole city at
  once.
- **The anchor weight 10/(n+10) was fixed in advance** and was not validated
  on Madrid.
- **The anchor's settings are the teammate's.** Their XGBoost and DANN
  hyper-parameters and the smoothing radius are copied, not re-tuned here, and
  this repo cannot show how they were chosen.
- **The anchor ships as cached Amsterdam probabilities**
  (`data/processed/zeroshot_xgb_dann_ams.npz`). `python
  tools/full_score_combo.py zeroshot` rebuilds it in six to thirteen minutes and
  needs xgboost and torch. On our CPU the rebuild is identical to the shipped
  file.
- **Fresh draws are not a fresh city.** They are new labelled pixels in the
  same Amsterdam.
- **The neural networks use one seed.**
- **The checks were automated.** `verification*.json` and
  `script_reviews.json` come from separate AI review agents, not from human or
  external review.

### What else was tried overnight

Selected variants of each kind. Draws 0–4, all of Amsterdam, our decision
rule, no smoothing.

| model | script | n=5 | n=25 | n=50 | n=100 | n=200 | average |
|---|---|---|---|---|---|---|---|
| **LightGBM with domain copies** | `lgbm_var.py` | 0.634 | 0.678 | 0.712 | 0.719 | 0.736 | **0.696** |
| DANN network, last layer fine-tuned, then the whole network | `lpft.py` | 0.601 | 0.642 | 0.674 | 0.690 | 0.703 | 0.662 |
| Retrained LightGBM (first release) | `baselines.py` | 0.566 | 0.628 | 0.670 | 0.697 | 0.720 | 0.656 |
| Metric learning: cosine encoder, Mahalanobis head | `metric.py` | 0.583 | 0.638 | 0.678 | 0.688 | 0.691 | 0.656 |
| Metric learning: prototypical net + CORAL, Mahalanobis head | `metric.py` | 0.605 | 0.640 | 0.671 | 0.679 | 0.682 | 0.655 |
| Shrinkage LDA on 64 PCA components | `classical.py` | 0.600 | 0.625 | 0.648 | 0.673 | 0.688 | 0.647 |
| Metric learning: prototypical net + CORAL, projection layer fine-tuned on the shots | `metric.py` | 0.597 | 0.627 | 0.656 | 0.673 | 0.677 | 0.646 |
| Logistic regression | `classical.py` | 0.564 | 0.608 | 0.658 | 0.679 | 0.694 | 0.640 |
| DANN network, backbone frozen, last layer fine-tuned | `tab_nn.py` | 0.605 | 0.628 | 0.643 | 0.650 | 0.660 | 0.637 |
| RBF SVM, domain-augmented features | `classical.py` | 0.556 | 0.611 | 0.651 | 0.661 | 0.674 | 0.631 |
| ResNet, backbone frozen, last layer fine-tuned | `tab_nn.py` | 0.552 | 0.603 | 0.642 | 0.662 | 0.673 | 0.626 |
| 1D CNN + DANN on the raw yearly series, last layer fine-tuned | `ts_nn.py` | 0.516 | 0.601 | 0.620 | 0.649 | 0.658 | 0.609 |
| DANN network, same backbone, last layer refit on Madrid + shots | `tab_nn.py` | 0.567 | 0.572 | 0.578 | 0.587 | 0.603 | 0.581 |

- **Frozen backbone, last layer fine-tuned.** Starting the last layer from its
  Madrid weights and fine-tuning it on the shots beats refitting it on Madrid
  plus shots: 0.637 against 0.581 for the same DANN backbone.
- **Unfreezing afterwards helps.** Fine-tuning the whole network for 60 small
  steps after the last layer lifts the DANN to 0.662. That is level with the
  retrained LightGBM. A variant with an entropy term scores 0.663.
- **Metric learning matches the retrained LightGBM on average.** It is ahead
  at n=5 and n=25 and about 0.03 behind at n=200. The Mahalanobis head does
  better than fine-tuning the projection layer on three of the four encoders.
- **None of them beats the LightGBM with domain copies**, and adding them to
  the combination lowered it.
- **Did not help:** prior re-estimation by EM, neural embeddings as LightGBM
  inputs, networks on the raw time series.
- **Not recommended:** self-training (+0.006 on one ensemble, but below its
  teacher on its own) and label propagation (+0.006 on Amsterdam, but worse on
  a Madrid stand-in).

All 134 scored methods are in `reports/nnlab/leaderboard.csv`.

## The two winners of the first release

| | **Zero-shot LightGBM** | **Retrained LightGBM** |
|---|---|---|
| What it is | LightGBM trained on Madrid only. It never sees an Amsterdam label. | A fresh LightGBM trained on all of Madrid plus the n labelled Amsterdam pixels per class, with those pixels weighted 30×. |
| When to use it | You may not use any Amsterdam labels | You are given a few labelled target pixels, which is the official protocol |
| Amsterdam macro F1 | **0.590–0.593** at every n (it ignores the labels; it drifts only because each draw's labelled pixels leave the scored set) | **0.656** average over n, **0.720** at n=200 |
| Saved model | `models/zero_shot_lgbm.txt` | Refit for every draw, so there is no single model. `models/retrained_n{25,200}_rep0_lgbm.txt` are two example draws. |
| Training cost | A few minutes on a 12-thread CPU | ~1–4 min per draw (84 s mean on a 12-thread CPU) |

Both models use the same 393 features and the same decision rule. All numbers
are on **all of Amsterdam** (25,992 pixels), minus the labelled pixels of each
draw.

### Results on all of Amsterdam

| model | n=5 | n=25 | n=50 | n=100 | n=200 | average | peak |
|---|---|---|---|---|---|---|---|
| **Retrained LightGBM** (5 draws per n) | 0.5658 ± 0.0108 | 0.6284 ± 0.0120 | 0.6697 ± 0.0087 | 0.6973 ± 0.0038 | **0.7204 ± 0.0060** | **0.6563** | **0.7204** |
| Prototype, within-class Mahalanobis (12 draws) | 0.5040 ± 0.0402 | **0.6361 ± 0.0170** | 0.6733 ± 0.0135 | 0.7023 ± 0.0067 | 0.7141 ± 0.0031 | 0.6460 | 0.7141 |
| **Zero-shot LightGBM** (20 draws) | **0.5928** | 0.5925 | 0.5922 | 0.5918 | 0.5903 | 0.5919 | 0.5928 |
| Organisers' prototype method, re-run by us on our 393 features (20 draws) | 0.5066 | 0.5821 | 0.5965 | 0.6034 | 0.5972 | 0.5772 | 0.6034 |
| *Organisers' prototype, as reported in their `4-Modelling.ipynb` (their 60 features, their draws)* | *0.5437* | *0.5986* | *0.6079* | *0.6129* | *0.6150* | *0.5956* | *0.6150* |
| *Organisers' reported zero-shot Random Forest (60 features)* | *0.3427* | | | | | | |

Mean ± std over draws. Each draw is a fresh random sample of labelled pixels,
so the ± is the spread you get from which pixels you happen to be handed.

The two italic rows are the organisers' own figures, copied from their
notebook. Their draws and evaluation set are not ours, so compare them
loosely. Their prototype scores higher on their 60 features than on our 393.
Both of our winners still beat it at every n.

**Best by prize category, within the first release.** FINAL-2 above is higher
than each of these: 0.714 average, 0.744 peak, 0.705 at n=25, 0.672 at n=5.
Those figures use smoothing, which needs pixel coordinates, and draws 5–11.
Without coordinates they are 0.700 / 0.734 / 0.694 / 0.652. The draws and the
amount of unlabelled Amsterdam information differ from the rows below, so
this is not a like-for-like ranking.
- **Average:** retrained LightGBM, 0.6563.
- **Peak:** retrained LightGBM at n=200, 0.7204.
- **n=25:** the prototype, 0.6361. It is implemented in `src/gbdt.py`
  (`mc_prototype(metric="within")`) but not packaged as a model here.
- **n=5:** zero-shot LightGBM, 0.5928.

![macro F1 against labelling budget](results/f1_vs_n_full_city.png)

### The fine-tuned LLM we also tried, for comparison

We also fine-tuned Qwen3-4B-Base with LoRA on text prompts that describe each
Madrid pixel. It took about 2 h on an A100: 1,513 steps at ~0.2 it/s.
- **Held-out Madrid areas:** it went from 0.305 untrained to 0.537 macro F1.
  These come from the fine-tuning notebook's validation output, which is not
  in this repo.
- **Amsterdam:** 0.554 with no labels, on a 6,000-pixel random subset. 0.622
  at best (n=200), on a fixed 2,000-pixel subset with 6 draws. These are not
  full-city numbers like the table above.

It does not beat the zero-shot LightGBM below n=200, so it is not one of the
winners. Details are in `results/llm_protocol_results_colab.md`.

## Quick start

```bash
pip install numpy pandas pyarrow scikit-learn scipy lightgbm matplotlib
python winners.py zero-shot                  # ~15-40 s
python winners.py retrained --n 25 --rep 0   # ~2-4 min
python tools/nnlab/model_create2.py --n 25 --rep 6   # FINAL-2, 1-2 min
```

The neural and metric-learning scripts in `tools/nnlab/` also need `torch`.
Rebuilding the zero-shot anchor needs `torch` and `xgboost`.

Expected output, from the verification run on our machine:

```
zero-shot, all 25,992 Amsterdam pixels: macro F1 0.5928
retrained, n=25 rep=0: macro F1 0.6303
FINAL-2 n=25 rep=6 spatial=True: macro F1 0.696921
```

To re-run the whole protocol: `python winners.py protocol --model retrained
--reps 5` (25 refits: ~35 min on our machine, up to ~1.5 h on a slower
one). Or open **`notebooks/winners_demo.ipynb`**, which is
already executed, for a walk-through of both models with outputs and the
chart.
LightGBM is multi-threaded, so on a different machine the numbers can differ
in the 4th decimal.
## What we did, exactly
### 1. Features (`notebooks/01_build_features.ipynb`, `src/features.py`)
We started from the organisers' raw per-pixel Landsat time series and built
one row per pixel with 393 usable features:
| group | count | what |
|---|---|---|
| official | 59 | The organisers' 60 features, reproduced exactly. One has zero variance in a city and was dropped. |
| reflectance | 276 | Surface reflectance per band; correct spectral indices; tasseled cap; chromaticity; per-decade and early/late-period means |
| events | 15 | Construction-event detection: when and how sharply the pixel's spectrum stepped |
| neighbourhood | 15 | The same signals averaged over surrounding pixels |
| other | 28 | Disturbance spikes and post-construction trends (12), Landsat sensor shares (9), observation coverage (4), position in the city: radial distance and distance to water (3) |
Two features with zero variance in one city were dropped, leaving 393 of 395.
The pixel coordinates and the label are never features. The resulting tables
are in `data/processed/` (Madrid 76,263 pixels, Amsterdam 25,992). Notebook 1
needs the raw hackathon data, which is not in this repo; the processed tables
are.
### 2. Model selection, on Madrid only (`notebooks/03_gbdt_models.ipynb`)
The rule we followed was that **Amsterdam labels are used only to draw the
permitted n-per-class sample and to score**. Every design choice was made on
Madrid.
**Spatial cross-validation.** Madrid was split into 25 geographic blocks, with
5-fold cross-validation over whole blocks. Random-pixel folds leak, because
neighbouring pixels share a construction era. That is why the published 0.618
Madrid baseline looks better than it transfers.
| model | 59 official | 393 all features |
|---|---|---|
| LightGBM, class-balanced | 0.556 | **0.570** |
| LightGBM, unweighted | 0.550 | 0.569 |
| LightGBM, ordinal | 0.546 | 0.562 |
| CatBoost, class-balanced | 0.545 | 0.562 * |
This comparison used cheap settings (400 rounds, learning rate 0.06) and
robust-z scaling per block. We were reading the ranking here, not the final
scores. The normalisation test below then dropped the scaling.
\* From an earlier run of the same notebook
(`results/logs/03_gbdt_models_fullscore_run.log`).
**Per-city normalisation, decided on Madrid.** Each held-out block was scaled
with its own statistics, the same way a new city would be.
- No normalisation: **0.599**.
- Robust z (median/IQR): 0.569.
So the zero-shot model uses raw features. This went against our starting
intuition, and Amsterdam agreed: robust z scored 0.527 there against 0.593
with no normalisation.
**Decision rule.** Plain argmax maximises accuracy, not macro F1. A simulated
shift harness makes held-out Madrid blocks look like a new city, by resampling
the class mix and mapping the features onto Amsterdam's unlabelled
distribution. In it, the most robust rule was:
1. Divide the probabilities by the Madrid class prior raised to 0.5 (tau-logit
   adjustment).
2. Apply the macro-F1 plug-in rule (Ramaswamy et al.).
It never lost to argmax in any simulated regime: +0.012 on average. EM-based
prior estimation was the worst option tested (−0.099) and is not used.
### 3. The two winners
**Zero-shot LightGBM** (`winners.train_zero_shot`)
- 1,200 rounds, learning rate 0.03, 31 leaves, `min_child_samples` 80,
  subsample 0.8, `colsample_bytree` 0.6, L1 0.5, L2 5.0, seed 42.
- Class-balanced weights, raw features.
- Prediction is the decision rule above, applied to all of Amsterdam's
  probabilities at once. The rule uses the batch's predicted class shares,
  never its labels.
**Retrained LightGBM** (`winners.train_retrained`). This follows the
organisers' `modelCreate` spec. The deck
`Building-Age-from-Space-A-Transferable-Classifier.pptx` is in their
`hackathon.zip` download, and its slide 4 says:

> "We are NOT building a classifier. We are building a METHOD that creates
> classifiers."

Slide 5 says that `modelCreate` takes N labelled pixels per class from the new
city and outputs "a fully functional classifier for that city". There is one
liberty beyond the tightest reading of slides 5 and 7, which give
`modelCreate` only the labelled sample: step 1 below also reads Amsterdam's
**unlabelled** feature table to compute scaling statistics (see Caveats). For
each draw of n labelled Amsterdam pixels per class:
1. Robust-z scale Madrid with Madrid's median/IQR. Scale Amsterdam with
   Amsterdam's median/IQR, computed from its whole unlabelled feature table.
2. Stack the 76,263 Madrid rows and the 4n Amsterdam rows. The Amsterdam rows
   get weight 30, then everything is multiplied by class-balanced weights.
3. Fit LightGBM with the same parameters as the zero-shot model, but 400 rounds
   at learning rate 0.06. This is the cheaper setting the protocol was run
   with, because it refits on every draw.
4. Predict every other Amsterdam pixel and apply the same decision rule.

### 4. The protocol, i.e. how every number was measured

- **Labelled sample:** for each n and each repeat r, n pixels per class are
  drawn with `numpy.random.default_rng(42 + 1000·n + r)`. Use
  `winners.shots_for(y_ams, n, r)` to get the identical pixels.
- **Scoring:** those pixels are removed and macro F1 is computed on every
  remaining Amsterdam pixel.
- **Repeats:** 5 for the retrained model, since each draw is a full refit. 12
  for the prototype, 20 for the zero-shot and the organisers' baseline.
- **Exploration subsample:** notebook 3 explored 14 variants on a fixed
  8,000-pixel subsample.
- **Full-city re-scoring:** `tools/full_score.py` re-scored the retrained
  LightGBM and the within-class prototype on all of Amsterdam. In the same run
  it re-scored the 8,000-pixel subsample, and that reproduced notebook 3's
  table to the 4th decimal, so the full-city numbers come from identical
  models. The full city reads 0.002–0.006 higher per n, and 0.003–0.004 on the
  average.
- **The other full-city rows:** the zero-shot and organisers' baseline rows
  come from an earlier full-city run of notebook 3
  (`results/logs/03_gbdt_models_fullscore_run.log`, 20 draws).

### 5. The other 12 Amsterdam variants (8,000-pixel subsample, average over n)

| variant | average |
|---|---|
| refit leaf values, tree structure frozen † | 0.612 |
| prototype, total-covariance Mahalanobis | 0.612 |
| prototype, rank + Euclidean | 0.591 |
| pure k-NN on the labelled pixels only | 0.589 |
| zero-shot, no normalisation (the zero-shot winner, before full-city scoring) | 0.586 |
| organisers' Euclidean prototype (re-run on our 393 features) | 0.575 |
| zero-shot, per-city rank / robust z / robust z + CORAL | 0.553 / 0.524 / 0.537 |
| recalibrated LightGBM / CatBoost | 0.544 / 0.550 |
| prototype, robust z + Euclidean | 0.401 |

† Measured with a normalisation bug: it applied robust z when its label says
none.

The models that actually learn from the labelled Amsterdam pixels win. The
ones that only rescale or recalibrate Madrid's output score below the
organisers' baseline (0.524–0.553 against 0.575). The exception is the plain
zero-shot model with no scaling at all.

## Caveats

- **Transductive scaling.** The retrained model scales Amsterdam with the
  median/IQR of Amsterdam's whole unlabelled feature table. That uses features
  only, never labels, and the protocol passes the table in. The organisers'
  baseline does **not** do this: it scales Amsterdam with Madrid's scaler. On a
  strict reading, where `modelCreate` gets only the labelled sample, you would
  compute those statistics from the shots alone. On a new test set, compute
  them on that set's features. The zero-shot model is unaffected, because it
  uses no scaling.
- **Batch-level decision rule.** The rule looks at the predicted class shares
  of the batch it labels. With plain argmax the zero-shot model scores 0.5680
  on all of Amsterdam instead of 0.5928.
- **Few repeats.** The retrained model's ± comes from 5 draws per n.
- **Hidden test sets.** The organisers score on held-out Amsterdam and Madrid
  sets we never saw. Expect those numbers to be lower.

## Repo layout

```
winners.py                     self-contained: both models, the decision rule, the protocol
models/
  zero_shot_lgbm.txt           the zero-shot model (LightGBM text format)
  retrained_n25_rep0_lgbm.txt  example retrained models, draw rep 0
  retrained_n200_rep0_lgbm.txt
  amsterdam_robustz_stats.npz  Amsterdam median/IQR the retrained models expect
  retrained_example_shot_indices.json   which Amsterdam rows they were trained on
  meta.json                    feature order, Madrid prior, verification results
data/processed/                madrid_features.parquet, amsterdam_features.parquet
  *_official60_first.parquet   the organisers' 60 features on the same pixels
  zeroshot_xgb_dann_ams.npz    cached Amsterdam probabilities of the zero-shot anchor
notebooks/
  winners_demo.ipynb           executed walk-through of both winners (start here)
  03_gbdt_models.ipynb         full model study: CV, normalisation, decision rule, all 14 variants
  01_build_features.ipynb      raw data -> features (needs the hackathon raw data)
  src/                         the same notebooks as plain .py
src/                           project code the notebooks import (gbdt.py, evaluation.py, ...)
tools/full_score.py            full-city re-scoring of the two winners
tools/nnlab/                   the overnight search (11 November)
  model_create2.py             FINAL-2 from a set of shots (start here)
  lab.py                       shared harness: draws, prediction store, scoring
  lgbm_var.py                  LightGBM variants, including the domain copies
  tab_nn.py, lpft.py           tabular networks: frozen backbone, last-layer and whole-network fine-tuning
  metric.py                    metric-learning encoders and heads
  classical.py                 SVM, logistic regression, kNN, LDA
  ts_nn.py                     networks on the raw yearly series (needs the hackathon raw data)
  transductive.py, ensemble.py, baselines.py, confirm.py, plot_confirm.py, model_create.py, lpft_steps.py
tools/compare_teammate.py      the teammate's XGBoost, DANN and smoothing functions
tools/full_score_combo.py      `zeroshot` mode rebuilds the anchor cache
tools/build_official60_first.py   builds the 60-feature tables (needs the hackathon raw data)
results/                       per-draw JSON, logs of every run, the LLM results (first release)
reports/nnlab/                 results of the overnight search
  OVERNIGHT_RESULTS.md         the full write-up
  confirmation.csv             the fresh-draw tables
  leaderboard.csv              every scored method (leaderboard_A.csv: half A of the city only)
  f1_vs_n_overnight.png        the chart above
  research_shortlist.json      the literature sweep behind the ideas tried
  scores/                      per-draw scores of every method
  PREREGISTRATION.md           the picks, written before their confirmation
  verification*.json, script_reviews.json, verify/   checks by separate AI review agents
  cache/                       the knobs chosen on Madrid, with their evidence
  logs/                        logs of every run
```

The per-draw prediction files and the trained network weights of the overnight
search are not in this repo (about 3 GB). The scripts regenerate them into
`reports/nnlab/preds/` and `reports/nnlab/cache/`. The scripts in
`reports/nnlab/verify/` are the record of the checks as they were run; they
read those prediction files.

`models/meta.json` → `verification` records the checks run when these files
were exported:
- the ported code in `winners.py` gives the same probabilities as the project
  code (max abs diff 0);
- the saved models reload identically;
- the two example draws reproduce the reported per-draw scores exactly.