Add FINAL-2 pipeline, overnight search scripts and results (11 Nov 2026)

README.md

@@ -51,6 +51,10 @@ fixed decision rule:
    Then apply our decision rule: tau = 0.5 logit adjustment with the Madrid
    class prior, then the macro-F1 plug-in. The rule was chosen on Madrid.
 
+The XGBoost + DANN anchor and the neighbour smoothing are a teammate's
+design. Their functions are in `tools/compare_teammate.py`, published with
+their agreement.
+
 Macro F1 on all of Amsterdam, on draws 5–11 (seven per n). These draws were
 never used to choose anything.
 
@@ -124,8 +128,10 @@ From Python: `model_create(shot_idx, shot_labels).predict()` in
   hyper-parameters and the smoothing radius are copied, not re-tuned here, and
   this repo cannot show how they were chosen.
 - **The anchor ships as cached Amsterdam probabilities**
-  (`data/processed/zeroshot_xgb_dann_ams.npz`). The code that builds it
-  follows the teammate's code and is not in this repo.
+  (`data/processed/zeroshot_xgb_dann_ams.npz`). `python
+  tools/full_score_combo.py zeroshot` rebuilds it in six to thirteen minutes and
+  needs xgboost and torch. On our CPU the rebuild is identical to the shipped
+  file.
 - **Fresh draws are not a fresh city.** They are new labelled pixels in the
   same Amsterdam.
 - **The neural networks use one seed.**
@@ -244,6 +250,7 @@ python tools/nnlab/model_create2.py --n 25 --rep 6   # FINAL-2, 1-2 min
 ```
 
 The neural and metric-learning scripts in `tools/nnlab/` also need `torch`.
+Rebuilding the zero-shot anchor needs `torch` and `xgboost`.
 
 Expected output, from the verification run on our machine:
 
@@ -451,6 +458,8 @@ tools/nnlab/                   the overnight search (11 November)
   classical.py                 SVM, logistic regression, kNN, LDA
   ts_nn.py                     networks on the raw yearly series (needs the hackathon raw data)
   transductive.py, ensemble.py, baselines.py, confirm.py, plot_confirm.py, model_create.py, lpft_steps.py
+tools/compare_teammate.py      the teammate's XGBoost, DANN and smoothing functions
+tools/full_score_combo.py      `zeroshot` mode rebuilds the anchor cache
 tools/build_official60_first.py   builds the 60-feature tables (needs the hackathon raw data)
 results/                       per-draw JSON, logs of every run, the LLM results (first release)
 reports/nnlab/                 results of the overnight search

reports/nnlab/OVERNIGHT_RESULTS.md

@@ -199,7 +199,7 @@ Caveats that belong next to any headline number:
 ## Reproducing
 
 ```
-python -I tools/full_score_combo.py zeroshot        # teammate's zero-shot cache (once; this script is not in the published repo, the cache is)
+python -I tools/full_score_combo.py zeroshot        # teammate's zero-shot cache (once)
 python tools/nnlab/baselines.py all                 # references + retrained LightGBM per draw
 python tools/nnlab/lgbm_var.py                      # LightGBM variants, incl. lgbm_feda
 python tools/nnlab/ensemble.py e12 --zs zs_xgb_dann --shot lgbm_feda      # FINAL-2

tools/compare_teammate.py

@@ -0,0 +1,279 @@
+"""
+Head-to-head: a teammate's pipeline against ours, on two common yardsticks.
+
+NOTE FOR THIS BUNDLE. This file is here because tools/full_score_combo.py imports
+it to rebuild the zero-shot anchor of FINAL-2 (its XGBoost, DANN, region
+normalisation and smoothing functions). The head-to-head in main() needs the
+teammate's lockbox block list, which is not in this repo, so main() cannot be run
+from the bundle.
+
+  lockbox    their own Madrid test: train on their 80% dev blocks, score once on
+             their 29 locked 2 km blocks (15,157 px). Rebuilt from our pixel keys
+             with their block rule. Their own lockbox scores, which are not in
+             this repo, were the fidelity check.
+  amsterdam  train on ALL of Madrid, score ALL 25,992 Amsterdam pixels with no
+             Amsterdam labels used except to score (zero-shot: the setting their
+             pipeline was built for).
+
+The functions here are adapted from the teammate's code and published with their
+agreement. One difference: XGBoost and the DANN run
+on CPU because this PC has no CUDA (different random streams; their DANN seeds
+vary by 0.002-0.004). Their features are the official 60 on the same pixels:
+pass `first` as the third argument to use the organisers' exact construction
+(tools/build_official60_first.py); the default is our median-per-year rebuild.
+
+Every model is scored under two decision rules:
+  argmax        what their pipeline uses
+  tau+plugin    ours: tau=0.5 logit adjustment then the macro-F1 plug-in, Madrid prior
+and with/without their neighbour smoothing (r = 6 px = 180 m, alpha 0.5).
+
+    python -I tools/compare_teammate.py <lockbox_blocks.json> reports/teammate_compare.csv
+"""
+from __future__ import annotations
+
+import json
+import os
+import sys
+import time
+from pathlib import Path
+
+ROOT = Path(__file__).resolve().parent.parent
+sys.path.insert(0, str(ROOT / "src"))
+os.chdir(ROOT)
+
+import numpy as np
+import pandas as pd
+import torch
+import torch.nn as nn
+from lightgbm import LGBMClassifier
+from scipy.spatial import cKDTree
+from sklearn.ensemble import RandomForestClassifier
+from sklearn.utils.class_weight import compute_sample_weight
+from xgboost import XGBClassifier
+
+import config as C
+import evaluation as E
+import gbdt as G
+
+torch.set_num_threads(max(1, (os.cpu_count() or 4) - 2))
+DEV = torch.device("cpu")
+BLOCK = 67
+SMOOTH_ALPHA = 0.5
+R_PX = 6
+XGB_PARAMS = {"n_estimators": 500, "learning_rate": 0.08, "max_depth": 8,
+              "min_child_weight": 3.0, "subsample": 0.8, "colsample_bytree": 1.0,
+              "reg_lambda": 10.0, "gamma": 1.0}            # their best_config.json
+DANN_EPOCHS, DANN_SEEDS = 40, (0, 1, 2)
+
+
+# ---- copied from their compare_models.py ---------------------------------- #
+def block_ids(pid):
+    return (pid[:, 0] // BLOCK) * 100000 + (pid[:, 1] // BLOCK)
+
+
+def region_norm(X):
+    mu, sd = X.mean(0), X.std(0)
+    sd[sd == 0] = 1
+    return (X - mu) / sd
+
+
+def smooth(P, coords, r):
+    tree = cKDTree(coords)
+    nbrs = tree.query_ball_point(coords, r=r)
+    out = np.empty_like(P)
+    for i, nb in enumerate(nbrs):
+        out[i] = SMOOTH_ALPHA * P[i] + (1 - SMOOTH_ALPHA) * P[nb].mean(0)
+    return out
+
+
+# ---- copied from their dann.py --------------------------------------------- #
+class GradReverse(torch.autograd.Function):
+    @staticmethod
+    def forward(ctx, x, lam):
+        ctx.lam = lam
+        return x.view_as(x)
+
+    @staticmethod
+    def backward(ctx, g):
+        return -ctx.lam * g, None
+
+
+class DANN(nn.Module):
+    def __init__(self, d_in=60, h=256, z=64):
+        super().__init__()
+        self.feat = nn.Sequential(nn.Linear(d_in, h), nn.BatchNorm1d(h), nn.ReLU(), nn.Dropout(0.2),
+                                  nn.Linear(h, h), nn.BatchNorm1d(h), nn.ReLU(), nn.Dropout(0.2),
+                                  nn.Linear(h, z), nn.ReLU())
+        self.age = nn.Sequential(nn.Linear(z, 64), nn.ReLU(), nn.Linear(64, 4))
+        self.city = nn.Sequential(nn.Linear(z, 64), nn.ReLU(), nn.Linear(64, 1))
+
+    def forward(self, x, lam=0.0):
+        f = self.feat(x)
+        return self.age(f), self.city(GradReverse.apply(f, lam)), f
+
+
+def train_dann(Xs, ys, Xt, epochs, adversarial=True, seed=0):
+    torch.manual_seed(seed)
+    np.random.seed(seed)
+    net = DANN(Xs.shape[1]).to(DEV)
+    opt = torch.optim.AdamW(net.parameters(), lr=1e-3, weight_decay=1e-4)
+    counts = np.bincount(ys, minlength=5)[1:]
+    w = torch.tensor(len(ys) / (4 * counts), dtype=torch.float32, device=DEV)
+    ce, bce = nn.CrossEntropyLoss(weight=w), nn.BCEWithLogitsLoss()
+    S, Y = torch.tensor(Xs, dtype=torch.float32, device=DEV), torch.tensor(ys - 1, device=DEV)
+    T = torch.tensor(Xt, dtype=torch.float32, device=DEV)
+    bs, steps = 512, int(np.ceil(len(S) / 512))
+    for ep in range(epochs):
+        net.train()
+        perm_s = torch.randperm(len(S), device=DEV)
+        for k in range(steps):
+            p = (ep * steps + k) / (epochs * steps)
+            lam = (2 / (1 + np.exp(-10 * p)) - 1) if adversarial else 0.0
+            i = perm_s[k * bs:(k + 1) * bs]
+            j = torch.randint(0, len(T), (len(i),), device=DEV)
+            ya, cs, _ = net(S[i], lam)
+            _, ct, _ = net(T[j], lam)
+            loss = ce(ya, Y[i])
+            if adversarial:
+                loss = loss + bce(cs.squeeze(1), torch.zeros(len(i), device=DEV)) \
+                            + bce(ct.squeeze(1), torch.ones(len(j), device=DEV))
+            opt.zero_grad()
+            loss.backward()
+            opt.step()
+    net.eval()
+    return net
+
+
+@torch.no_grad()
+def dann_proba(net, X):
+    a, _, _ = net(torch.tensor(X, dtype=torch.float32, device=DEV))
+    return torch.softmax(a, 1).cpu().numpy()
+
+
+# ---- their tree models (final_lockbox.py / tune_boosting.py) --------------- #
+def their_trees(Xtr, ytr, Xte):
+    sw = compute_sample_weight("balanced", ytr)
+    xgb = XGBClassifier(objective="multi:softprob", tree_method="hist", device="cpu",
+                        eval_metric="mlogloss", random_state=42, **XGB_PARAMS)
+    xgb.fit(Xtr, ytr - 1, sample_weight=sw)
+    lgb = LGBMClassifier(n_estimators=600, learning_rate=0.05, num_leaves=63, subsample=0.8,
+                         subsample_freq=1, colsample_bytree=0.8, class_weight="balanced",
+                         n_jobs=6, random_state=42, verbose=-1).fit(Xtr, ytr)
+    rf = RandomForestClassifier(n_estimators=300, class_weight="balanced", n_jobs=6,
+                                random_state=42).fit(Xtr, ytr)
+    return {"T: XGBoost (tuned)": xgb.predict_proba(Xte),
+            "T: LightGBM": lgb.predict_proba(Xte),
+            "T: Random Forest": rf.predict_proba(Xte)}
+
+
+# ---- ours ------------------------------------------------------------------ #
+def our_lgbm(Xtr, ytr, Xte):
+    """notebook 3's 'zero-shot, no normalisation': full LGBM_PARAMS, balanced."""
+    cw = G.balanced_class_weights(ytr)
+    m = G.fit_lgbm(Xtr, ytr, class_weight=dict(zip(range(4), cw)))
+    return m.predict_proba(Xte)
+
+
+def load(official: str = "median"):
+    mad = pd.read_parquet(C.PROC_DIR / "madrid_features.parquet")
+    ams = pd.read_parquet(C.PROC_DIR / "amsterdam_features.parquet")
+    drop = {"px_key", "py_key", "age_class", "weighted_mean_year", "city"}
+    feats = [c for c in mad.columns if c not in drop and mad[c].dtype.kind in "fiu"]
+    feats = [c for c in feats if not (mad[c].std() == 0 or ams[c].std() == 0)]
+    assert len(feats) == 393, len(feats)
+    f60 = list(C.FEATURE_COLS_60)                      # all 60, as their pkl has
+    if official == "first":
+        # The organisers' exact construction (first QA-valid observation per
+        # year), from tools/build_official60_first.py, aligned row by row.
+        def aligned(city, ref):
+            o = pd.read_parquet(C.PROC_DIR / f"{city}_official60_first.parquet")
+            has_label = "age_class" in o.columns       # the published tables carry no label
+            o = ref[C.PID + ["age_class"]].merge(o, on=C.PID, how="left",
+                                                 suffixes=("", "_o"))
+            assert o[f60].notna().all().all() and len(o) == len(ref)
+            assert not has_label or (o["age_class"] == o["age_class_o"]).all()
+            return o[f60].to_numpy(float)
+        Xm60, Xa60 = aligned("madrid", mad), aligned("amsterdam", ams)
+    else:
+        Xm60, Xa60 = mad[f60].to_numpy(float), ams[f60].to_numpy(float)
+    print(f"official 60 built with agg={official}", flush=True)
+    return dict(
+        Xm60=Xm60, Xa60=Xa60,
+        Xm=mad[feats].to_numpy(float), Xa=ams[feats].to_numpy(float),
+        ym=mad["age_class"].to_numpy().astype(int), ya=ams["age_class"].to_numpy().astype(int),
+        pm=mad[C.PID].to_numpy().astype(np.int64), pa=ams[C.PID].to_numpy().astype(np.int64))
+
+
+def run_setting(name, d, tr, te_X60, te_X, te_y, te_xy, Xadv60):
+    """Fit every model on Madrid rows `tr`, score on the given test set."""
+    t0 = time.time()
+    Xtr60, Xtr, ytr = d["Xm60"][tr], d["Xm"][tr], d["ym"][tr]
+    prior = np.array([(ytr == c).mean() for c in E.CLASSES])
+    P = their_trees(Xtr60, ytr, te_X60)
+    print(f"[{name}] their trees done {time.time() - t0:.0f}s", flush=True)
+    P["O: LightGBM, 393 features (our zero-shot)"] = our_lgbm(Xtr, ytr, te_X)
+    P["O: LightGBM, official 60 (our params)"] = our_lgbm(Xtr60, ytr, te_X60)
+    print(f"[{name}] our LightGBM done {time.time() - t0:.0f}s", flush=True)
+    seeds = {}
+    Xtr_n, Xte_n, Xadv_n = region_norm(Xtr60), region_norm(te_X60), region_norm(Xadv60)
+    for s in DANN_SEEDS:
+        net = train_dann(Xtr_n, ytr, Xadv_n, DANN_EPOCHS, adversarial=True, seed=s)
+        Pd = dann_proba(net, Xte_n)
+        seeds[s] = {"T: DANN": Pd,
+                    "T: XGBoost + DANN (their FINAL)": (P["T: XGBoost (tuned)"] + Pd) / 2}
+        print(f"[{name}] DANN seed {s} done {time.time() - t0:.0f}s", flush=True)
+
+    coords = te_xy.astype(float)
+    rows = []
+
+    def score(model, seed, PP):
+        for sm in (False, True):
+            Q = smooth(PP, coords, R_PX) if sm else PP
+            for rule in ("argmax", "tau+plugin"):
+                lab = (np.array(E.CLASSES)[Q.argmax(1)] if rule == "argmax"
+                       else E.adapt_pipeline(Q, prior, tau=0.5)[0])
+                pc = E.per_class_f1(te_y, lab)
+                rows.append(dict(setting=name, model=model, seed=seed, smooth=sm, rule=rule,
+                                 macro_f1=E.macro_f1(te_y, lab),
+                                 **{f"f1_c{c}": float(v) for c, v in zip(E.CLASSES, pc)},
+                                 n_test=len(te_y)))
+
+    for k, PP in P.items():
+        score(k, 0, PP)
+    for s, dd in seeds.items():
+        for k, PP in dd.items():
+            score(k, s, PP)
+    print(f"[{name}] scored, {time.time() - t0:.0f}s total", flush=True)
+    return rows
+
+
+def main():
+    lock_json, out_csv = Path(sys.argv[1]), Path(sys.argv[2])
+    d = load(sys.argv[3] if len(sys.argv) > 3 else "median")
+    g = block_ids(d["pm"])
+    lock = np.isin(g, json.loads(lock_json.read_text())["blocks"])
+    print(f"lockbox pixels {lock.sum():,} (theirs: 15,157); dev {(~lock).sum():,} "
+          f"(theirs: 61,106)", flush=True)
+    assert lock.sum() == 15157 and (~lock).sum() == 61106
+
+    rows = []
+    dev, lk = np.flatnonzero(~lock), np.flatnonzero(lock)
+    rows += run_setting("lockbox", d, dev, d["Xm60"][lk], d["Xm"][lk], d["ym"][lk],
+                        d["pm"][lk], d["Xa60"])
+    pd.DataFrame(rows).to_csv(out_csv, index=False)
+    allm = np.arange(len(d["ym"]))
+    rows += run_setting("amsterdam", d, allm, d["Xa60"], d["Xa"], d["ya"], d["pa"], d["Xa60"])
+    df = pd.DataFrame(rows)
+    df.to_csv(out_csv, index=False)
+
+    summ = (df.groupby(["setting", "model", "smooth", "rule"])["macro_f1"]
+              .agg(["mean", "std", "size"]).round(4).reset_index())
+    for st in ("lockbox", "amsterdam"):
+        t = summ[summ.setting == st].pivot_table(index="model", columns=["rule", "smooth"],
+                                                 values="mean")
+        print(f"\n=== {st} (macro F1; DANN rows = mean of 3 seeds) ===")
+        print(t.sort_values(("argmax", True), ascending=False).to_string())
+
+
+if __name__ == "__main__":
+    main()

tools/full_score_combo.py

@@ -0,0 +1,167 @@
+"""
+NOTE FOR THIS BUNDLE. Only one mode of this script is used here:
+
+    python tools/full_score_combo.py zeroshot
+
+It rebuilds data/processed/zeroshot_xgb_dann_ams.npz, the zero-shot anchor of the
+FINAL-2 pipeline (tools/nnlab/model_create2.py): a teammate's XGBoost and DANN on
+the official 60 features, from tools/compare_teammate.py. It needs xgboost and
+torch and takes six to thirteen minutes on a 12-thread CPU. The combination described
+below is NOT FINAL-2, and its numbers are not quoted anywhere in this bundle.
+
+Our few-shot winners combined with the teammate's DANN and neighbour smoothing,
+scored on ALL of Amsterdam under the official protocol.
+
+Intended as the primary model when this script was written (2026-11-10). This
+bundle holds no time-stamped record of that, and Amsterdam scores of its
+ingredients (the upweighted LightGBM, the teammate's zero-shot pipeline) had
+already been seen. The primary model is
+    "upweighted + DANN + smooth":
+        p = (P_upweighted_lgbm + P_dann) / 2  ->  neighbour smoothing (6 px)  ->  our rule
+Every ingredient was chosen on Madrid, not Amsterdam:
+  * upweighted retrained LightGBM - notebook 3's design, 393 features;
+  * DANN (60 official features, adversary on UNLABELLED Amsterdam) and the
+    50/50 probability average with a tree model - teammate's design, +0.014 on
+    their Madrid lockbox;
+  * neighbour smoothing r = 6 px, alpha 0.5 - teammate's choice on Madrid
+    (+0.025 on their CV); ours (r = 2) also gained on Madrid;
+  * decision rule tau 0.5 then the macro-F1 plug-in with the Madrid prior -
+    chosen on Madrid by the shift harness.
+The other rows are ablations that show what each ingredient contributes. Some
+were added after Amsterdam results had been seen.
+
+One deliberate change from the teammate: the 3 DANN seeds are averaged into one
+probability (their lockbox reported seeds separately), so a protocol draw does not
+depend on which seed was used.
+
+Protocol: identical to tools/full_score.py. Seed 42, shots drawn with
+rng(42 + 1000 n + rep), shots removed from the scored set, and every draw scored on
+all remaining Amsterdam pixels. Smoothing uses only the scored pixels as
+neighbours (no shot, no label), as in the teammate's code. Amsterdam labels are
+read only to draw the shots and to score.
+
+    python -I tools/full_score_combo.py zeroshot                 # cache XGB + DANN
+    python -I tools/full_score_combo.py upweighted reports/combo_upweighted.json
+    python -I tools/full_score_combo.py prototype  reports/combo_prototype.json
+"""
+from __future__ import annotations
+
+import json
+import sys
+import time
+from pathlib import Path
+
+sys.path.insert(0, str(Path(__file__).resolve().parent))
+import compare_teammate as CT   # noqa: E402  (also puts src/ on the path, chdirs to root)
+import full_score as FS         # noqa: E402
+
+import numpy as np              # noqa: E402
+import pandas as pd             # noqa: E402
+
+import config as C              # noqa: E402
+import evaluation as E          # noqa: E402
+
+SEED, N_LIST, R_PX = 42, (5, 25, 50, 100, 200), 6
+CACHE = C.PROC_DIR / "zeroshot_xgb_dann_ams.npz"
+
+
+def build_cache(d):
+    """Zero-shot parts: they never see a shot, so they are fitted once."""
+    t0 = time.time()
+    sw = CT.compute_sample_weight("balanced", d["ym"])
+    xgb = CT.XGBClassifier(objective="multi:softprob", tree_method="hist", device="cpu",
+                           eval_metric="mlogloss", random_state=42, **CT.XGB_PARAMS)
+    xgb.fit(d["Xm60"], d["ym"] - 1, sample_weight=sw)
+    P_xgb = xgb.predict_proba(d["Xa60"])
+    Xm_n, Xa_n = CT.region_norm(d["Xm60"]), CT.region_norm(d["Xa60"])
+    P_d = []
+    for s in CT.DANN_SEEDS:
+        net = CT.train_dann(Xm_n, d["ym"], Xa_n, CT.DANN_EPOCHS, adversarial=True, seed=s)
+        P_d.append(CT.dann_proba(net, Xa_n))
+        print(f"DANN seed {s} done {time.time() - t0:.0f}s", flush=True)
+    np.savez(CACHE, P_xgb=P_xgb, P_dann_seeds=np.stack(P_d), pa=d["pa"])
+    print(f"cached -> {CACHE}", flush=True)
+
+
+def main():
+    mode = sys.argv[1]
+    d = CT.load("first")
+    if mode == "zeroshot":
+        build_cache(d)
+        return
+    out_path = Path(sys.argv[2])
+    z = np.load(CACHE)
+    assert (z["pa"] == d["pa"]).all()
+    P_dann = z["P_dann_seeds"].mean(0)                 # seed-averaged
+    P_final = (z["P_xgb"] + P_dann) / 2                # teammate's FINAL, zero-shot
+
+    name, make, n_repeats = FS.VARIANTS[mode]
+    X_mad, y_mad, X_ams, y_ams = d["Xm"], d["ym"], d["Xa"], d["ya"]
+    src_prior = np.array([(y_mad == c).mean() for c in E.CLASSES])
+    coords = d["pa"].astype(float)
+    base = "upweighted" if mode == "upweighted" else "prototype"
+
+    def rules(P, y):
+        out = {}
+        out["argmax"] = E.macro_f1(y, np.array(E.CLASSES)[P.argmax(1)])
+        out["tau+plugin"] = E.macro_f1(y, E.adapt_pipeline(P, src_prior, tau=0.5)[0])
+        return out
+
+    draws = []
+    if out_path.exists():
+        prev = json.loads(out_path.read_text())
+        if prev.get("key") == mode:
+            draws = prev["draws"]
+            print(f"resuming: {len(draws)} draws already saved", flush=True)
+    done = {(r["n"], r["rep"]) for r in draws}
+    mc = make()
+    t_all = time.time()
+    for n in N_LIST:
+        for r in range(n_repeats):
+            if (n, r) in done:
+                continue
+            t0 = time.time()
+            rng = np.random.default_rng(SEED + 1000 * n + r)
+            shot_idx = E.draw_shots(y_ams, n, rng)
+            keep = np.setdiff1d(np.arange(len(y_ams)), shot_idx)
+            fn = mc(X_mad, y_mad, X_ams, X_ams[shot_idx], y_ams[shot_idx])
+            Pb = np.asarray(fn(X_ams[keep]), dtype=float)
+            ck, yk = coords[keep], y_ams[keep]
+            combos = {
+                f"{base}": Pb,
+                f"{base} + smooth": CT.smooth(Pb, ck, R_PX),
+                f"{base} + DANN": (Pb + P_dann[keep]) / 2,
+                f"{base} + DANN + smooth": CT.smooth((Pb + P_dann[keep]) / 2, ck, R_PX),
+            }
+            if mode == "upweighted":     # zero-shot references, cheap, scored per draw
+                combos["their FINAL (zero-shot)"] = P_final[keep]
+                combos["their FINAL + smooth (zero-shot)"] = CT.smooth(P_final[keep], ck, R_PX)
+            rec = dict(n=n, rep=r, n_scored=int(len(keep)))
+            for k, P in combos.items():
+                for rule, v in rules(P, yk).items():
+                    rec[f"{k} | {rule}"] = float(v)
+            rec["seconds"] = round(time.time() - t0, 1)
+            draws.append(rec)
+            print(f"n={n:<4d} rep={r:<2d} " + "  ".join(
+                f"{k}={rec[k + ' | tau+plugin']:.4f}" for k in combos) +
+                f" ({rec['seconds']}s)", flush=True)
+            out_path.write_text(json.dumps(dict(key=mode, variant=name, seed=SEED,
+                                                complete=False, draws=draws), indent=1))
+
+    df = pd.DataFrame(draws)
+    cols = [c for c in df.columns if " | " in c]
+    g = df.groupby("n")[cols]
+    summ = {c: {**{int(n): dict(mean=float(g.mean().loc[n, c]), std=float(g.std(ddof=1).loc[n, c]))
+                   for n in N_LIST},
+                "average": float(g.mean()[c].mean()), "peak": float(g.mean()[c].max())}
+            for c in cols}
+    out_path.write_text(json.dumps(dict(key=mode, variant=name, seed=SEED, complete=True,
+                                        minutes=round((time.time() - t_all) / 60, 1),
+                                        summary=summ, draws=draws), indent=1))
+    tbl = pd.DataFrame({c: {**{n: summ[c][n]["mean"] for n in N_LIST},
+                            "avg": summ[c]["average"]} for c in cols}).T
+    print("\n" + tbl.round(4).to_string(), flush=True)
+
+
+if __name__ == "__main__":
+    main()

tools/nnlab/tab_nn.py

@@ -10,7 +10,7 @@ wd 1e-4, batch 512, seeds 0,1,2):
         BN-ReLU -> Linear(64)-ReLU -> Linear(64,4)
   dann  mlp + domain head (Lin(64,64)-ReLU-Lin(64,1)) on the first 64-d features through
         gradient reversal, lambda = 2/(1+exp(-10p))-1, adversary vs UNLABELLED Amsterdam rows
-        (as in the teammate's DANN; Madrid and Amsterdam batches go through
+        (as tools/compare_teammate.py train_dann; Madrid and Amsterdam batches go through
         the net separately)
   coral mlp + deep-CORAL loss (weight 1.0) between the penultimate features of a Madrid
         batch and an unlabelled Amsterdam batch