Add FINAL-2 pipeline, overnight search scripts and results (11 Nov 2026)

.gitattributes

@@ -35,3 +35,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
 models/zero_shot_lgbm.txt filter=lfs diff=lfs merge=lfs -text
 results/f1_vs_n_full_city.png filter=lfs diff=lfs merge=lfs -text
+reports/nnlab/f1_vs_n_overnight.png filter=lfs diff=lfs merge=lfs -text

README.md

@@ -11,9 +11,11 @@ library_name: lightgbm
 
 # Building age from Landsat: Madrid → Amsterdam (Hack4Dev 2026)
 
-This repo holds the two winning models from our Iraq Hackathon 2026 / Hack4Dev
-entry "Building Age Classification from Satellite Imagery", together with
-the code, data and notebooks needed to reproduce them or compare against them.
+This repo holds the models from our Iraq Hackathon 2026 / Hack4Dev entry
+"Building Age Classification from Satellite Imagery", together with the code,
+data and notebooks needed to reproduce them or compare against them. It has
+our best pipeline (FINAL-2, added on 11 November 2026) and the two LightGBM
+models of the first release.
 
 **The task.** Predict a building-age class (1–4) for each 30 m Landsat pixel.
 Train on Madrid and transfer to Amsterdam. Scoring is macro F1 on Amsterdam
@@ -27,7 +29,151 @@ class** (the few-shot protocol).
 | 3 | 1984–2004 | 1984–2004 |
 | 4 | 2004–2024 | 2004–2024 |
 
-## The two winners
+## Update, 11 November 2026: the best pipeline (FINAL-2)
+
+After the first release we scored about 130 more model variants in one night
+on a laptop CPU: neural networks with a frozen backbone, metric learning,
+SVMs and other classical models, and ideas from the domain-adaptation
+literature. The best result combines two models with a fixed weighting and a
+fixed decision rule:
+
+1. **Zero-shot anchor.** A teammate's XGBoost + DANN design on the organisers'
+   60 features, re-implemented by us on CPU. It is trained on Madrid labels
+   and unlabelled Amsterdam features, and never sees an Amsterdam label.
+2. **LightGBM with domain copies.** The retrained LightGBM from the first
+   release with one change, Daumé's "frustratingly easy" domain adaptation.
+   The 120 most important features each get two extra copies: one filled only
+   on Madrid rows and one filled only on Amsterdam rows.
+3. **Combine.** Average the two models' log-probabilities. The zero-shot model
+   gets weight 10/(n+10) and the LightGBM gets the rest. No weight is fitted.
+4. **Decide.** Smooth each pixel's probabilities with its neighbours (radius 6
+   pixels, the teammate's setting, which they report choosing on Madrid).
+   Then apply our decision rule: tau = 0.5 logit adjustment with the Madrid
+   class prior, then the macro-F1 plug-in. The rule was chosen on Madrid.
+
+Macro F1 on all of Amsterdam, on draws 5–11 (seven per n). These draws were
+never used to choose anything.
+
+| pipeline | n=5 | n=25 | n=50 | n=100 | n=200 | average |
+|---|---|---|---|---|---|---|
+| **FINAL-2, with neighbour smoothing** | 0.672 | 0.705 | 0.716 | 0.732 | 0.744 | **0.714** |
+| FINAL-2 without coordinates (no smoothing) | 0.652 | 0.694 | 0.702 | 0.720 | 0.734 | 0.700 |
+| LightGBM with domain copies alone, with smoothing | 0.623 | 0.699 | 0.712 | 0.730 | 0.744 | 0.702 |
+| LightGBM with domain copies alone, no smoothing | 0.607 | 0.687 | 0.698 | 0.718 | 0.734 | 0.689 |
+| Round-1 five-model ensemble + label propagation, with smoothing | 0.677 | 0.705 | 0.714 | 0.725 | 0.728 | 0.710 |
+| Retrained LightGBM (first release), with smoothing | 0.587 | 0.652 | 0.680 | 0.712 | 0.732 | 0.673 |
+| Retrained LightGBM (first release), no smoothing | 0.559 | 0.629 | 0.660 | 0.697 | 0.721 | 0.653 |
+| Teammate's zero-shot XGBoost + DANN (our re-implementation), with smoothing and our decision rule | 0.654 | 0.653 | 0.653 | 0.652 | 0.651 | 0.653 |
+
+- **Compare like with like.** With smoothing on both sides FINAL-2 averages
+  0.714 against 0.673 for the retrained LightGBM. Without coordinates on both
+  sides it averages 0.700 against 0.653.
+- **No pixel coordinates?** Quote the second row, 0.700.
+- **What the anchor adds.** Against the LightGBM with domain copies alone,
+  both smoothed, FINAL-2 gains about 0.05 at n=5 and nothing at n=200 (0.744
+  for both).
+- **The margin over the round-1 ensemble is small.** FINAL-2 is 0.004 ahead
+  on average. At n=5 the ensemble is 0.004 ahead, which is inside the draw
+  noise. We prefer FINAL-2 because it is simpler: two models and no network
+  fitted per draw.
+- **The draws differ from the first-release table.** The retrained LightGBM
+  averages 0.653 here and 0.656 in the table further down, which uses draws
+  0–4.
+- **Standard deviation over the seven draws** for FINAL-2: 0.007 / 0.012 /
+  0.015 / 0.004 / 0.003 at n = 5 / 25 / 50 / 100 / 200.
+
+![macro F1 against labelling budget, overnight search](reports/nnlab/f1_vs_n_overnight.png)
+
+Run it for one draw. It takes one to two minutes. Set `NNLAB_THREADS` to your
+core count, because the default is 3.
+
+```bash
+python tools/nnlab/model_create2.py --n 25 --rep 6
+python tools/nnlab/model_create2.py --n 200 --rep 3 --no-spatial
+```
+
+Expected output, from a run inside a clean copy of this repo:
+
+```
+FINAL-2 n=25 rep=6 spatial=True: macro F1 0.696921
+FINAL-2 n=200 rep=3 spatial=False: macro F1 0.735816
+```
+
+From Python: `model_create(shot_idx, shot_labels).predict()` in
+`tools/nnlab/model_create2.py`. The full write-up is
+`reports/nnlab/OVERNIGHT_RESULTS.md`.
+
+### Caveats for FINAL-2
+
+- **The choice of models was made on Amsterdam.** About 130 variants were
+  scored on all of Amsterdam on draws 0–4. The LightGBM with domain copies
+  was picked from those full-city scores, and the two-model combination was
+  picked on half of the city. FINAL-2 is the second of two rounds of picks. A
+  round-1 rule (switch pipelines at n=100) failed its confirmation and was
+  dropped. Only draws 5–11 were never used for choosing. FINAL-2 reads 0.005
+  lower on them than on draws 0–4, and the review estimated 0.005 to 0.01 of
+  optimism for the top picks.
+- **It uses more of the unlabelled target city.** The DANN inside the anchor
+  is trained against unlabelled Amsterdam features, each city is scaled with
+  its own statistics, and the smoothing needs pixel coordinates.
+- **Batch-level.** The decision rule and the smoothing label the whole city at
+  once.
+- **The anchor weight 10/(n+10) was fixed in advance** and was not validated
+  on Madrid.
+- **The anchor's settings are the teammate's.** Their XGBoost and DANN
+  hyper-parameters and the smoothing radius are copied, not re-tuned here, and
+  this repo cannot show how they were chosen.
+- **The anchor ships as cached Amsterdam probabilities**
+  (`data/processed/zeroshot_xgb_dann_ams.npz`). The code that builds it
+  follows the teammate's code and is not in this repo.
+- **Fresh draws are not a fresh city.** They are new labelled pixels in the
+  same Amsterdam.
+- **The neural networks use one seed.**
+- **The checks were automated.** `verification*.json` and
+  `script_reviews.json` come from separate AI review agents, not from human or
+  external review.
+
+### What else was tried overnight
+
+Selected variants of each kind. Draws 0–4, all of Amsterdam, our decision
+rule, no smoothing.
+
+| model | script | n=5 | n=25 | n=50 | n=100 | n=200 | average |
+|---|---|---|---|---|---|---|---|
+| **LightGBM with domain copies** | `lgbm_var.py` | 0.634 | 0.678 | 0.712 | 0.719 | 0.736 | **0.696** |
+| DANN network, last layer fine-tuned, then the whole network | `lpft.py` | 0.601 | 0.642 | 0.674 | 0.690 | 0.703 | 0.662 |
+| Retrained LightGBM (first release) | `baselines.py` | 0.566 | 0.628 | 0.670 | 0.697 | 0.720 | 0.656 |
+| Metric learning: cosine encoder, Mahalanobis head | `metric.py` | 0.583 | 0.638 | 0.678 | 0.688 | 0.691 | 0.656 |
+| Metric learning: prototypical net + CORAL, Mahalanobis head | `metric.py` | 0.605 | 0.640 | 0.671 | 0.679 | 0.682 | 0.655 |
+| Shrinkage LDA on 64 PCA components | `classical.py` | 0.600 | 0.625 | 0.648 | 0.673 | 0.688 | 0.647 |
+| Metric learning: prototypical net + CORAL, projection layer fine-tuned on the shots | `metric.py` | 0.597 | 0.627 | 0.656 | 0.673 | 0.677 | 0.646 |
+| Logistic regression | `classical.py` | 0.564 | 0.608 | 0.658 | 0.679 | 0.694 | 0.640 |
+| DANN network, backbone frozen, last layer fine-tuned | `tab_nn.py` | 0.605 | 0.628 | 0.643 | 0.650 | 0.660 | 0.637 |
+| RBF SVM, domain-augmented features | `classical.py` | 0.556 | 0.611 | 0.651 | 0.661 | 0.674 | 0.631 |
+| ResNet, backbone frozen, last layer fine-tuned | `tab_nn.py` | 0.552 | 0.603 | 0.642 | 0.662 | 0.673 | 0.626 |
+| 1D CNN + DANN on the raw yearly series, last layer fine-tuned | `ts_nn.py` | 0.516 | 0.601 | 0.620 | 0.649 | 0.658 | 0.609 |
+| DANN network, same backbone, last layer refit on Madrid + shots | `tab_nn.py` | 0.567 | 0.572 | 0.578 | 0.587 | 0.603 | 0.581 |
+
+- **Frozen backbone, last layer fine-tuned.** Starting the last layer from its
+  Madrid weights and fine-tuning it on the shots beats refitting it on Madrid
+  plus shots: 0.637 against 0.581 for the same DANN backbone.
+- **Unfreezing afterwards helps.** Fine-tuning the whole network for 60 small
+  steps after the last layer lifts the DANN to 0.662. That is level with the
+  retrained LightGBM. A variant with an entropy term scores 0.663.
+- **Metric learning matches the retrained LightGBM on average.** It is ahead
+  at n=5 and n=25 and about 0.03 behind at n=200. The Mahalanobis head does
+  better than fine-tuning the projection layer on three of the four encoders.
+- **None of them beats the LightGBM with domain copies**, and adding them to
+  the combination lowered it.
+- **Did not help:** prior re-estimation by EM, neural embeddings as LightGBM
+  inputs, networks on the raw time series.
+- **Not recommended:** self-training (+0.006 on one ensemble, but below its
+  teacher on its own) and label propagation (+0.006 on Amsterdam, but worse on
+  a Madrid stand-in).
+
+All 134 scored methods are in `reports/nnlab/leaderboard.csv`.
+
+## The two winners of the first release
 
 | | **Zero-shot LightGBM** | **Retrained LightGBM** |
 |---|---|---|
@@ -60,7 +206,12 @@ notebook. Their draws and evaluation set are not ours, so compare them
 loosely. Their prototype scores higher on their 60 features than on our 393.
 Both of our winners still beat it at every n.
 
-**Best by prize category:**
+**Best by prize category, within the first release.** FINAL-2 above is higher
+than each of these: 0.714 average, 0.744 peak, 0.705 at n=25, 0.672 at n=5.
+Those figures use smoothing, which needs pixel coordinates, and draws 5–11.
+Without coordinates they are 0.700 / 0.734 / 0.694 / 0.652. The draws and the
+amount of unlabelled Amsterdam information differ from the rows below, so
+this is not a like-for-like ranking.
 - **Average:** retrained LightGBM, 0.6563.
 - **Peak:** retrained LightGBM at n=200, 0.7204.
 - **n=25:** the prototype, 0.6361. It is implemented in `src/gbdt.py`
@@ -86,16 +237,20 @@ winners. Details are in `results/llm_protocol_results_colab.md`.
 ## Quick start
 
 ```bash
-pip install numpy pandas pyarrow scikit-learn lightgbm matplotlib
+pip install numpy pandas pyarrow scikit-learn scipy lightgbm matplotlib
 python winners.py zero-shot                  # ~15-40 s
 python winners.py retrained --n 25 --rep 0   # ~2-4 min
+python tools/nnlab/model_create2.py --n 25 --rep 6   # FINAL-2, 1-2 min
 ```
 
+The neural and metric-learning scripts in `tools/nnlab/` also need `torch`.
+
 Expected output, from the verification run on our machine:
 
 ```
 zero-shot, all 25,992 Amsterdam pixels: macro F1 0.5928
 retrained, n=25 rep=0: macro F1 0.6303
+FINAL-2 n=25 rep=6 spatial=True: macro F1 0.696921
 ```
 
 To re-run the whole protocol: `python winners.py protocol --model retrained
@@ -278,6 +433,8 @@ models/
   retrained_example_shot_indices.json   which Amsterdam rows they were trained on
   meta.json                    feature order, Madrid prior, verification results
 data/processed/                madrid_features.parquet, amsterdam_features.parquet
+  *_official60_first.parquet   the organisers' 60 features on the same pixels
+  zeroshot_xgb_dann_ams.npz    cached Amsterdam probabilities of the zero-shot anchor
 notebooks/
   winners_demo.ipynb           executed walk-through of both winners (start here)
   03_gbdt_models.ipynb         full model study: CV, normalisation, decision rule, all 14 variants
@@ -285,9 +442,36 @@ notebooks/
   src/                         the same notebooks as plain .py
 src/                           project code the notebooks import (gbdt.py, evaluation.py, ...)
 tools/full_score.py            full-city re-scoring of the two winners
-results/                       per-draw JSON, logs of every run, the LLM results
+tools/nnlab/                   the overnight search (11 November)
+  model_create2.py             FINAL-2 from a set of shots (start here)
+  lab.py                       shared harness: draws, prediction store, scoring
+  lgbm_var.py                  LightGBM variants, including the domain copies
+  tab_nn.py, lpft.py           tabular networks: frozen backbone, last-layer and whole-network fine-tuning
+  metric.py                    metric-learning encoders and heads
+  classical.py                 SVM, logistic regression, kNN, LDA
+  ts_nn.py                     networks on the raw yearly series (needs the hackathon raw data)
+  transductive.py, ensemble.py, baselines.py, confirm.py, plot_confirm.py, model_create.py, lpft_steps.py
+tools/build_official60_first.py   builds the 60-feature tables (needs the hackathon raw data)
+results/                       per-draw JSON, logs of every run, the LLM results (first release)
+reports/nnlab/                 results of the overnight search
+  OVERNIGHT_RESULTS.md         the full write-up
+  confirmation.csv             the fresh-draw tables
+  leaderboard.csv              every scored method (leaderboard_A.csv: half A of the city only)
+  f1_vs_n_overnight.png        the chart above
+  research_shortlist.json      the literature sweep behind the ideas tried
+  scores/                      per-draw scores of every method
+  PREREGISTRATION.md           the picks, written before their confirmation
+  verification*.json, script_reviews.json, verify/   checks by separate AI review agents
+  cache/                       the knobs chosen on Madrid, with their evidence
+  logs/                        logs of every run
 ```
 
+The per-draw prediction files and the trained network weights of the overnight
+search are not in this repo (about 3 GB). The scripts regenerate them into
+`reports/nnlab/preds/` and `reports/nnlab/cache/`. The scripts in
+`reports/nnlab/verify/` are the record of the checks as they were run; they
+read those prediction files.
+
 `models/meta.json` → `verification` records the checks run when these files
 were exported:
 - the ported code in `winners.py` gives the same probabilities as the project

data/processed/amsterdam_official60_first.parquet

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4b89183fe348067ad37fdc85c2194049695c0829c02d1cb9fae0eb4131c89ef4
+size 8858304

data/processed/madrid_official60_first.parquet

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4dc24ffd0dfcb05590919900a7ab1951b9aa46954f844fcc1ee179be455e5bd7
+size 26294722

data/processed/zeroshot_xgb_dann_ams.npz

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:847cd046aceedc89352ad7eb4394031b3ef854dd7d5c799e1f7ca80ef158ef06
+size 2080116

reports/nnlab/OVERNIGHT_RESULTS.md

@@ -0,0 +1,223 @@
+# Overnight model search, 2026-11-11
+
+About 130 model variants were scored on the Amsterdam protocol in one night, on
+the laptop CPU. The best result is a two-model pipeline: the teammate's
+zero-shot XGBoost+DANN combined with a LightGBM that is retrained on Madrid plus
+the shots **with domain-specific feature copies**. On draws never used for
+choosing it averages **0.714** macro F1 with neighbour smoothing (0.700 without
+coordinates), against 0.673 (0.653) for the retrained LightGBM that was the
+previous best.
+
+![Macro F1 against labelling budget](f1_vs_n_overnight.png)
+
+## The result
+
+Macro F1 on all of Amsterdam except each draw's labelled pixels. These are the
+"fresh" draws 5-11, seven per n, which were never used to choose anything.
+
+| pipeline | n=5 | n=25 | n=50 | n=100 | n=200 | average |
+|---|---|---|---|---|---|---|
+| **FINAL-2: zero-shot + LightGBM with domain copies, smoothing** | **0.672** | **0.705** | **0.716** | **0.732** | **0.744** | **0.714** |
+| FINAL-2 without coordinates (no smoothing) | 0.652 | 0.694 | 0.702 | 0.720 | 0.734 | 0.700 |
+| LightGBM with domain copies alone, smoothing | 0.623 | 0.699 | 0.712 | 0.730 | 0.744 | 0.702 |
+| LightGBM with domain copies alone, no smoothing | 0.607 | 0.687 | 0.698 | 0.718 | 0.734 | 0.689 |
+| Five-model ensemble (round 1), smoothing | 0.669 | 0.700 | 0.706 | 0.721 | 0.725 | 0.704 |
+| Five-model ensemble + label propagation, smoothing | 0.677 | 0.705 | 0.714 | 0.725 | 0.728 | 0.710 |
+| Five-model ensemble, no coordinates | 0.647 | 0.683 | 0.690 | 0.706 | 0.713 | 0.688 |
+| Retrained LightGBM, smoothing | 0.587 | 0.652 | 0.680 | 0.712 | 0.732 | 0.673 |
+| Retrained LightGBM, no smoothing (previous best, as published) | 0.559 | 0.629 | 0.660 | 0.697 | 0.721 | 0.653 |
+| Teammate's zero-shot XGBoost+DANN, smoothing, our rule | 0.654 | 0.653 | 0.653 | 0.652 | 0.651 | 0.653 |
+
+Standard deviation over the seven draws for FINAL-2: 0.007 / 0.012 / 0.015 /
+0.004 / 0.003. All 12 draws and the half-B view are in `confirmation.csv`.
+
+What the comparison says:
+
+- **FINAL-2 has the best average on the fresh draws (0.714).** Its peak, 0.744
+  at n=200, is the same as the LightGBM with domain copies alone. At n=25 it
+  scores 0.705, level with the five-model ensemble plus label propagation. At
+  n=5 that ensemble is 0.004 ahead, which is inside the draw noise.
+- **Like for like against the previous best.** With smoothing on both, FINAL-2
+  gains +0.085 / +0.053 / +0.035 / +0.020 / +0.012 at n = 5 / 25 / 50 / 100 /
+  200. Without coordinates on both, +0.047 on the average (0.700 against 0.653).
+- **Where the gain comes from.** At n=5 most of the level is the zero-shot model
+  (0.654 with no labels at all); from n=25 up it is the LightGBM with domain
+  copies, which alone beats the previous LightGBM by +0.036 on average.
+- **Against the round-1 five-model ensemble** the margin is small on average
+  (+0.010; +0.004 if label propagation is added to the ensemble) and widens
+  with n. FINAL-2 needs two models and no neural network fitted per draw.
+- **Selection optimism is visible and small**: the LightGBM with domain copies
+  scored 0.696 on the first five draws, where it was picked, and 0.689 on the
+  fresh ones. The retrained LightGBM, which was never selected, also reads
+  0.003 lower on the fresh draws, so the net optimism is about 0.004.
+- **The gain does not depend on the decision rule.** The domain copies beat the
+  plain retrained LightGBM on 35 of 35 fresh draws under plain argmax (+0.039),
+  our rule (+0.036) and rule plus smoothing (+0.029).
+
+### FINAL-2, step by step
+
+1. **Zero-shot anchor**: the teammate's XGBoost + DANN on the official 60
+   features. Madrid labels, unlabelled Amsterdam features, no Amsterdam labels.
+2. **LightGBM with domain copies** (`lgbm_feda`): our retrained LightGBM
+   (Madrid + shots weighted 30x, classes balanced, each city robust-z scaled
+   with its own median and IQR) with one change, Daume's "frustratingly easy"
+   domain adaptation. The 120 most important features each get two extra
+   copies, one filled only for Madrid rows and one filled only for Amsterdam
+   rows. Trees can then learn Madrid-specific splits that Amsterdam rows
+   bypass through the missing-value branch.
+3. **Combine**: average of log-probabilities, the zero-shot model weighted
+   10/(n+10) and the LightGBM the rest. No weight is fitted.
+4. **Decide**: neighbour smoothing (radius 6 pixels, the teammate's setting,
+   which they report choosing on Madrid), then our rule (tau = 0.5 logit adjustment and the
+   macro-F1 plug-in), both chosen on Madrid.
+
+`tools/nnlab/model_create2.py` runs it from a set of shots:
+`model_create(shot_idx, shot_labels).predict()`. It reproduces the stored
+scores to six decimals on the two draws tested. One call is one LightGBM fit,
+about one to two minutes.
+
+### Evidence for the domain copies beyond the Amsterdam score
+
+- **Madrid only.** With a spatial quarter of Madrid standing in as the target
+  city, the domain copies beat the plain retrained LightGBM in 8 of 9 draws:
+  +0.005 at n=5, +0.018 at n=50, +0.020 at n=200
+  (`verify/feda_madrid_check.json`). Caveat: the list of 120 features came from
+  a model trained on all of Madrid, the stand-in part included.
+- **Shuffled labels.** Trained with the shot labels shuffled, the model
+  collapses to 0.32 at n=5 and 0.23 at n=200. The gain comes from the shot
+  labels, not from a leak.
+- **Fixed knobs.** 120 features and shot weight 30 were set before any score;
+  shot weight 100 on top of the copies changed nothing (0.694 against 0.696).
+- **What does the work** (one draw, n=50, argmax, by an independent verifier):
+  plain retrained LightGBM 0.641; one Madrid/Amsterdam indicator column 0.647;
+  Madrid-only copies 0.662; Amsterdam-only copies 0.659; both 0.664. So most of
+  the gain comes from letting the trees tell the cities apart, and about 29% of
+  the split gain sits on the Madrid-only copies against 5% on the
+  Amsterdam-only ones.
+
+## Every family, single models
+
+Draws 0-4, all pixels, our decision rule, no smoothing.
+
+| family | best variant | n=5 | n=25 | n=50 | n=100 | n=200 | average |
+|---|---|---|---|---|---|---|---|
+| **LightGBM variants** | **domain copies (Daume)** | 0.634 | 0.678 | 0.712 | 0.719 | 0.736 | **0.696** |
+| LightGBM variants | shot weight 100 instead of 30 | 0.571 | 0.636 | 0.680 | 0.706 | 0.731 | 0.665 |
+| Tabular network | DANN, last layer then whole network fine-tuned | 0.601 | 0.642 | 0.674 | 0.690 | 0.703 | 0.662 |
+| Retrained LightGBM | Madrid + shots x30 (previous best) | 0.566 | 0.628 | 0.670 | 0.697 | 0.720 | 0.656 |
+| Metric learning | cosine encoder, Mahalanobis head | 0.583 | 0.638 | 0.678 | 0.688 | 0.691 | 0.656 |
+| Metric learning | prototypical net + CORAL, Mahalanobis head | 0.605 | 0.640 | 0.671 | 0.679 | 0.682 | 0.655 |
+| Classical | shrinkage LDA on 64 PCA components | 0.600 | 0.625 | 0.648 | 0.673 | 0.688 | 0.647 |
+| Classical | logistic regression | 0.564 | 0.608 | 0.658 | 0.679 | 0.695 | 0.640 |
+| Tabular network | DANN, frozen, last layer fine-tuned | 0.605 | 0.628 | 0.643 | 0.650 | 0.660 | 0.637 |
+| Classical | RBF SVM, domain-augmented features | 0.556 | 0.611 | 0.651 | 0.661 | 0.674 | 0.631 |
+| LightGBM variants | neural embeddings as extra columns | 0.568 | 0.598 | 0.626 | 0.657 | 0.689 | 0.628 |
+| Tabular network | ResNet, frozen, last layer fine-tuned | 0.552 | 0.603 | 0.642 | 0.662 | 0.673 | 0.626 |
+| Sequence network | 1D CNN + DANN on raw yearly series, last layer fine-tuned | 0.516 | 0.601 | 0.620 | 0.649 | 0.658 | 0.609 |
+| Tabular network | CORAL, frozen, last layer fine-tuned | 0.544 | 0.562 | 0.576 | 0.595 | 0.614 | 0.578 |
+
+Findings:
+
+- **Frozen backbone, fine-tune the last layer.** On every backbone, starting the
+  last layer from the Madrid weights and fine-tuning it on the shots (with a
+  pull back toward the Madrid weights) beats re-fitting the last layer on
+  Madrid + shots. For the DANN: 0.637 against 0.582.
+- **Then unfreeze, gently.** Fine-tuning the last layer first and then the
+  whole network for 60 small steps (Madrid examples mixed in, a penalty for
+  drifting from the start) lifts the DANN from 0.637 to 0.662, more at large n.
+  On the fresh draws it scores 0.658, the best neural model of the night and
+  level with the old LightGBM. The 60 steps were fixed in advance. A Madrid
+  stand-in preferred 300 steps, which was worse on Amsterdam (0.644 on draws
+  0-4, 0.640 fresh), so 60 was kept.
+- **Domain-adversarial training is the best backbone.** DANN 0.637, ResNet
+  0.626, plain MLP 0.603, CORAL 0.578.
+- **Metric learning: the head matters more than the encoder.** A within-class
+  Mahalanobis head is best on three of the four encoders (on the supervised
+  contrastive one, fine-tuning the projection wins, 0.641 against 0.630). Plain prototypes reach 0.56 to
+  0.60, soft k-means refinement with unlabelled pixels 0.54 to 0.60, and
+  fine-tuning the projection layer on the shots about 0.64.
+- **Raw time series transfer worst.** The sequence networks see the 42-year
+  series directly and are the weakest family. Their zero-shot scores (0.32 to
+  0.53) show how city-specific the raw signal is.
+- **Feeding neural embeddings to LightGBM hurt** (0.628 against 0.656).
+- **Prior re-estimation by EM hurt on every base on Amsterdam** (for example
+  0.656 to 0.646 on the retrained LightGBM), although it helped slightly on
+  the Madrid stand-in run tonight (+0.010).
+- **Self-training** (retrain LightGBM on the ensemble's confident Amsterdam
+  predictions) added +0.006 to the five-model ensemble without label
+  propagation and +0.001 with it; on its own it scored below its teacher.
+- **Label propagation** added about +0.006 on top of the five-model ensemble on
+  Amsterdam but lowered the score on a Madrid stand-in (0.475 against 0.498),
+  so it is reported and not recommended.
+
+The full list is in `leaderboard.csv`.
+
+## How far to trust it
+
+Round 1 was checked by three separate AI review agents (`verification.json`): a
+leakage audit (none found; Amsterdam labels are read only for the shots of the
+draw being built and by the scorer), an independent recomputation (exact to
+float32 precision; the harness reproduces the published retrained-LightGBM
+numbers 0.5658 / 0.6284 / 0.6697 / 0.6973 / 0.7204 with zero difference), and a
+methodology critique. Round 2 was checked by two more
+(`verification_round2.json`): an independent refit of the LightGBM with domain
+copies (identical to 3e-8, no leakage found) and a cell-by-cell fact check of
+this report, whose corrections are applied. Every model script was also
+reviewed by a second agent before its full run (`script_reviews.json`).
+
+Caveats that belong next to any headline number:
+
+1. **Chosen on Amsterdam, not Madrid:** which models go into the combination
+   (FINAL-2 was picked on half A of the first five draws, where adding the
+   metric-learning and neural models back lowered it). Fixed in advance but
+   not validated on Madrid: the anchor weight 10/(n+10), a heuristic from the
+   literature shortlist, and the label-propagation settings. Fixed on Madrid:
+   the smoothing radius, the decision rule, the teammate's XGBoost and DANN
+   settings (their own Madrid tuning), and the training lengths and
+   fine-tuning strength of the networks.
+2. **Selection optimism.** About 130 variants were scored. The reviewers
+   estimated 0.005 to 0.01 of optimism for the top picks; the picks read 0.004
+   to 0.007 lower on the fresh draws, of which 0.003 is shared by the
+   unselected reference. The fresh-draw table is the one to quote.
+3. **Fresh draws are not a fresh city.** They use new labelled pixels but the
+   same Amsterdam. Half A / half B (69 interleaved blocks) is only a sanity
+   split; on half B FINAL-2 averages 0.700 against 0.696 for the five-model
+   ensemble (0.699 with label propagation) and 0.656 for the retrained
+   LightGBM, all with smoothing.
+4. **Madrid is a weak guide for adaptation strength.** Three times a Madrid
+   stand-in and Amsterdam disagreed (label propagation, fine-tuning length, EM
+   prior correction). They agree on the domain copies and on unfreezing the
+   network after the last-layer fine-tune.
+5. **What the organisers' harness must provide.** All unlabelled Amsterdam
+   features when the model is built (per-city scaling, the DANN in the anchor),
+   pixel coordinates for smoothing, and batch prediction of the whole city.
+   Without coordinates use the "no coordinates" row: 0.700.
+6. **A rule that failed.** "Switch pipelines from n=100", chosen on half A in
+   round 1, did not hold on half B. It is dropped.
+7. **The neural networks use one seed** (three were estimated at four hours).
+
+## Reproducing
+
+```
+python -I tools/full_score_combo.py zeroshot        # teammate's zero-shot cache (once; this script is not in the published repo, the cache is)
+python tools/nnlab/baselines.py all                 # references + retrained LightGBM per draw
+python tools/nnlab/lgbm_var.py                      # LightGBM variants, incl. lgbm_feda
+python tools/nnlab/ensemble.py e12 --zs zs_xgb_dann --shot lgbm_feda      # FINAL-2
+python tools/nnlab/metric.py                        # metric-learning encoders and heads
+python tools/nnlab/tab_nn.py --seeds 1              # tabular networks, frozen backbone
+python tools/nnlab/lpft.py                          # whole-network fine-tuning
+python tools/nnlab/ts_nn.py                         # sequence networks
+python tools/nnlab/classical.py                     # SVM, logistic regression, kNN, LDA
+python tools/nnlab/lab.py score                     # score everything in the store
+python tools/nnlab/confirm.py                       # the tables above
+python tools/nnlab/model_create2.py --n 25 --rep 0  # FINAL-2 for one draw, from the shots
+```
+
+Set `NNLAB_REPS=12` for all twelve draws. `model_create.py` is the same kind of
+entry point for the round-1 five-model ensemble.
+
+Files: `PREREGISTRATION.md` (both picks, written before their confirmation),
+`research_shortlist.json` (the literature sweep; the domain copies are its item
+10), `verification.json`, `verification_round2.json`, `script_reviews.json`,
+`verify/` (shuffle test, Madrid checks), `logs/`,
+`cache/*/selection*.json` (every knob chosen on Madrid, with its evidence).

reports/nnlab/PREREGISTRATION.md

@@ -0,0 +1,54 @@
+# Final picks, fixed before confirmation (2026-11-11 01:36)
+
+Written before any score on half B alone was tabulated and before draws 5-11
+existed for the ensemble members.
+
+## What was used to choose
+- Half A of Amsterdam (checkerboard of 2 km blocks, `lab.half_A`), draws 0-4,
+  `python tools/nnlab/lab.py board _A`.
+- Honest caveat: full-city scores (A + B together) of the single models and of
+  ensembles e1-e3 were printed earlier in the night, so B was not perfectly
+  blind. B on its own was never looked at.
+
+## The picks
+| name | pipeline | decision |
+|---|---|---|
+| FINAL-AVG | `ens_e4__lp`: fixed-rule ensemble (zero-shot XGBoost+DANN with weight 10/(n+10); the rest shared equally by retrained LightGBM, met_protoda_maha, met_cos_maha, tab_dann_rz_ft), then label propagation with the shots clamped | rule + neighbour smoothing |
+| FINAL-PEAK | `ens_e2__lapshot`: zero-shot XGBoost+DANN (weight 10/(n+10)) + retrained LightGBM, then LaplacianShot-style graph refinement | rule + neighbour smoothing |
+| FINAL-SWITCH | n <= 50: FINAL-AVG; n >= 100: FINAL-PEAK (n is an input of modelCreate, so this is one method) | as above |
+
+Half A, draws 0-4 (the evidence): ens_e4__lp 0.7033 / 0.7109 / 0.7290 / 0.7355 / 0.7392;
+ens_e2__lapshot 0.6856 / 0.6952 / 0.7172 / 0.7376 / 0.7524.
+
+Why e4 and not e6 / e7 (which add a self-trained LightGBM or LDA): they differ
+from e4 by under 0.001 on half A, which is noise, and e4 is one stage simpler.
+
+## Confirmation to be reported, whatever it shows
+1. Half B, draws 0-11.
+2. All pixels, draws 5-11 only (draws never used for choosing).
+3. The same for the reference: retrained LightGBM with the rule (0.6563 average).
+
+## Round 2 pick, fixed before its confirmation (2026-11-11 03:26)
+
+A LightGBM variant from round 2, `lgbm_feda` (Daume feature augmentation: shared,
+Madrid-only and Amsterdam-only copies of the top 120 features; everything else as
+the retrained LightGBM), scored well above the others on draws 0-4.
+
+| name | pipeline | decision |
+|---|---|---|
+| FINAL-2 | `ens_e12`: zero-shot XGBoost+DANN with weight 10/(n+10) + `lgbm_feda` with the rest | rule + neighbour smoothing |
+| FINAL-2 without coordinates | the same | rule only |
+| single model | `lgbm_feda` | rule + smoothing, and rule only |
+
+Evidence used: half A, draws 0-4. ens_e12 rule_smooth 0.7052 / 0.7161 / 0.7410 /
+0.7481 / 0.7591 (avg 0.734) against ens_e4 0.7139 avg; lgbm_feda alone 0.7236 avg.
+Ensembles that keep the metric-learning and neural members next to lgbm_feda
+(e10, e11) were lower on half A (0.721, 0.724) and are not picked.
+
+Sanity check done before confirmation: training lgbm_feda with the shot labels
+shuffled collapses macro F1 to 0.32 (n=5) and 0.23 (n=200), so the gain comes
+from the shot labels.
+
+Confirmation to be reported whatever it shows: all pixels on draws 5-11, on which
+lgbm_feda has never been run, and half B on draws 0-11.
+`lgbm_fedaw100` (feda + shot weight 100) is exploratory only, draws 0-4.

reports/nnlab/cache/lpft/base_check.json

@@ -0,0 +1,7 @@
+{
+ "compared_with": "reports\\nnlab\\cache\\tab_nn\\emb_dann_rz_s0.npz",
+ "base_threads": 3,
+ "Za_maxabs": 1.430511474609375e-06,
+ "W0_maxabs": 0.0,
+ "match_1e-3": true
+}

reports/nnlab/cache/lpft/madrid_check.json

@@ -0,0 +1,94 @@
+{
+ "ns": [
+  5,
+  50,
+  200
+ ],
+ "reps": 3,
+ "shot_rng": "default_rng(9000 + 100 n + rep)",
+ "epochs": 15,
+ "lam": 10.0,
+ "draws": [
+  {
+   "n": 5,
+   "rep": 0,
+   "lp": 0.4570928762811606,
+   "lpft": 0.4752690893967487,
+   "lpft_ent": 0.48058559187233046
+  },
+  {
+   "n": 5,
+   "rep": 1,
+   "lp": 0.4962300677451825,
+   "lpft": 0.47896254829584733,
+   "lpft_ent": 0.48623655425208534
+  },
+  {
+   "n": 5,
+   "rep": 2,
+   "lp": 0.49912649201630965,
+   "lpft": 0.5194485521018687,
+   "lpft_ent": 0.5218634141305216
+  },
+  {
+   "n": 50,
+   "rep": 0,
+   "lp": 0.5469353536909174,
+   "lpft": 0.5490712016882752,
+   "lpft_ent": 0.5508626235873457
+  },
+  {
+   "n": 50,
+   "rep": 1,
+   "lp": 0.5190261997622546,
+   "lpft": 0.5527083307285695,
+   "lpft_ent": 0.5516332670404107
+  },
+  {
+   "n": 50,
+   "rep": 2,
+   "lp": 0.5177433426465476,
+   "lpft": 0.5471363494676702,
+   "lpft_ent": 0.5486211076489819
+  },
+  {
+   "n": 200,
+   "rep": 0,
+   "lp": 0.5293177774783984,
+   "lpft": 0.569542112151458,
+   "lpft_ent": 0.5710629594921057
+  },
+  {
+   "n": 200,
+   "rep": 1,
+   "lp": 0.5289184955036026,
+   "lpft": 0.5796882061538661,
+   "lpft_ent": 0.5783839304518672
+  },
+  {
+   "n": 200,
+   "rep": 2,
+   "lp": 0.5228766050047738,
+   "lpft": 0.5713353577497671,
+   "lpft_ent": 0.5703748385220877
+  }
+ ],
+ "src_base_vs_tab_nn_sel_Zt_maxabs": 0.0,
+ "mean": {
+  "lp": {
+   "5": 0.4841498120142176,
+   "50": 0.5279016320332399,
+   "200": 0.5270376259955917
+  },
+  "lpft": {
+   "5": 0.49122672993148825,
+   "50": 0.5496386272948383,
+   "200": 0.5735218920183637
+  },
+  "lpft_ent": {
+   "5": 0.49622852008497914,
+   "50": 0.5503723327589127,
+   "200": 0.5732739094886868
+  }
+ }
+}

reports/nnlab/cache/lpft/steps_selection.json

@@ -0,0 +1,256 @@
+{
+ "grid": [
+  60,
+  150,
+  300,
+  600
+ ],
+ "rows": [
+  {
+   "steps": 60,
+   "n": 5,
+   "rep": 0,
+   "f1": 0.4752690893967487
+  },
+  {
+   "steps": 60,
+   "n": 5,
+   "rep": 1,
+   "f1": 0.47896254829584733
+  },
+  {
+   "steps": 60,
+   "n": 5,
+   "rep": 2,
+   "f1": 0.5194485521018687
+  },
+  {
+   "steps": 60,
+   "n": 50,
+   "rep": 0,
+   "f1": 0.5490712016882752
+  },
+  {
+   "steps": 60,
+   "n": 50,
+   "rep": 1,
+   "f1": 0.5527083307285695
+  },
+  {
+   "steps": 60,
+   "n": 50,
+   "rep": 2,
+   "f1": 0.5471363494676702
+  },
+  {
+   "steps": 60,
+   "n": 200,
+   "rep": 0,
+   "f1": 0.569542112151458
+  },
+  {
+   "steps": 60,
+   "n": 200,
+   "rep": 1,
+   "f1": 0.5796882061538661
+  },
+  {
+   "steps": 60,
+   "n": 200,
+   "rep": 2,
+   "f1": 0.5713353577497671
+  },
+  {
+   "steps": 150,
+   "n": 5,
+   "rep": 0,
+   "f1": 0.4852931580202636
+  },
+  {
+   "steps": 150,
+   "n": 5,
+   "rep": 1,
+   "f1": 0.48677743627345416
+  },
+  {
+   "steps": 150,
+   "n": 5,
+   "rep": 2,
+   "f1": 0.5172077383688378
+  },
+  {
+   "steps": 150,
+   "n": 50,
+   "rep": 0,
+   "f1": 0.5500076016916534
+  },
+  {
+   "steps": 150,
+   "n": 50,
+   "rep": 1,
+   "f1": 0.5535162303676714
+  },
+  {
+   "steps": 150,
+   "n": 50,
+   "rep": 2,
+   "f1": 0.5492764425118571
+  },
+  {
+   "steps": 150,
+   "n": 200,
+   "rep": 0,
+   "f1": 0.5817608944835705
+  },
+  {
+   "steps": 150,
+   "n": 200,
+   "rep": 1,
+   "f1": 0.5931042791351125
+  },
+  {
+   "steps": 150,
+   "n": 200,
+   "rep": 2,
+   "f1": 0.5911666068678278
+  },
+  {
+   "steps": 300,
+   "n": 5,
+   "rep": 0,
+   "f1": 0.4897960121180538
+  },
+  {
+   "steps": 300,
+   "n": 5,
+   "rep": 1,
+   "f1": 0.48523236779901824
+  },
+  {
+   "steps": 300,
+   "n": 5,
+   "rep": 2,
+   "f1": 0.5119732403602113
+  },
+  {
+   "steps": 300,
+   "n": 50,
+   "rep": 0,
+   "f1": 0.5511885140219575
+  },
+  {
+   "steps": 300,
+   "n": 50,
+   "rep": 1,
+   "f1": 0.5555369949864464
+  },
+  {
+   "steps": 300,
+   "n": 50,
+   "rep": 2,
+   "f1": 0.5495949162897769
+  },
+  {
+   "steps": 300,
+   "n": 200,
+   "rep": 0,
+   "f1": 0.5823060301505336
+  },
+  {
+   "steps": 300,
+   "n": 200,
+   "rep": 1,
+   "f1": 0.5952846799978556
+  },
+  {
+   "steps": 300,
+   "n": 200,
+   "rep": 2,
+   "f1": 0.5921326359044334
+  },
+  {
+   "steps": 600,
+   "n": 5,
+   "rep": 0,
+   "f1": 0.49339758915215653
+  },
+  {
+   "steps": 600,
+   "n": 5,
+   "rep": 1,
+   "f1": 0.486544192320482
+  },
+  {
+   "steps": 600,
+   "n": 5,
+   "rep": 2,
+   "f1": 0.5095993040970357
+  },
+  {
+   "steps": 600,
+   "n": 50,
+   "rep": 0,
+   "f1": 0.5494823439061667
+  },
+  {
+   "steps": 600,
+   "n": 50,
+   "rep": 1,
+   "f1": 0.55489372104404
+  },
+  {
+   "steps": 600,
+   "n": 50,
+   "rep": 2,
+   "f1": 0.550785704695016
+  },
+  {
+   "steps": 600,
+   "n": 200,
+   "rep": 0,
+   "f1": 0.5804620115742651
+  },
+  {
+   "steps": 600,
+   "n": 200,
+   "rep": 1,
+   "f1": 0.5912347874606855
+  },
+  {
+   "steps": 600,
+   "n": 200,
+   "rep": 2,
+   "f1": 0.5928397211154817
+  }
+ ],
+ "mean_by_steps": {
+  "60": 0.5381290830815634,
+  "150": 0.5453455986355832,
+  "300": 0.545893932403143,
+  "600": 0.5454710417072588
+ },
+ "by_n": {
+  "60": {
+   "5": 0.49122672993148825,
+   "50": 0.5496386272948383,
+   "200": 0.5735218920183637
+  },
+  "150": {
+   "5": 0.49642611088751853,
+   "50": 0.5509334248570606,
+   "200": 0.5886772601621703
+  },
+  "300": {
+   "5": 0.4956672067590944,
+   "50": 0.5521068084327269,
+   "200": 0.5899077820176076
+  },
+  "600": {
+   "5": 0.4965136951898914,
+   "50": 0.5517205898817409,
+   "200": 0.5881788400501441
+  }
+ },
+ "chosen": 300,
+ "rule": "highest mean macro F1 over n in (5, 50, 200) x 3 draws on the Madrid pseudo-target"
+}

reports/nnlab/cache/metric/cos_select.json

@@ -0,0 +1,16 @@
+{
+ "candidates": [
+  300,
+  700,
+  1500
+ ],
+ "pseudo_target_f1": {
+  "300": 0.5166631760428458,
+  "700": 0.5191297935628119,
+  "1500": 0.5267551538906341
+ },
+ "chosen": 1500,
+ "seed": 100,
+ "split": "lab.madrid_split(pm) default",
+ "da_target_in_selection": null
+}

reports/nnlab/cache/metric/proto_select.json

@@ -0,0 +1,16 @@
+{
+ "candidates": [
+  400,
+  800,
+  1600
+ ],
+ "pseudo_target_f1": {
+  "400": 0.4932358821647188,
+  "800": 0.5287072535546748,
+  "1600": 0.5097482730733088
+ },
+ "chosen": 800,
+ "seed": 100,
+ "split": "lab.madrid_split(pm) default",
+ "da_target_in_selection": null
+}

reports/nnlab/cache/metric/protoda_select.json

@@ -0,0 +1,16 @@
+{
+ "candidates": [
+  400,
+  800,
+  1600
+ ],
+ "pseudo_target_f1": {
+  "400": 0.5346068260062208,
+  "800": 0.5117476445589391,
+  "1600": 0.5269532337806007
+ },
+ "chosen": 400,
+ "seed": 100,
+ "split": "lab.madrid_split(pm) default",
+ "da_target_in_selection": "pseudo-target (unlabelled)"
+}

reports/nnlab/cache/metric/supcon_select.json

@@ -0,0 +1,16 @@
+{
+ "candidates": [
+  300,
+  700,
+  1500
+ ],
+ "pseudo_target_f1": {
+  "300": 0.5113092490498672,
+  "700": 0.5227456085770579,
+  "1500": 0.5148359504081166
+ },
+ "chosen": 700,
+ "seed": 100,
+ "split": "lab.madrid_split(pm) default",
+ "da_target_in_selection": null
+}

reports/nnlab/cache/tab_nn/selection.json

@@ -0,0 +1,71 @@
+{
+ "epochs": {
+  "mlp": 30,
+  "res": 15,
+  "dann": 15,
+  "coral": 45
+ },
+ "epoch_scores": {
+  "mlp": {
+   "15": 0.4999554636261916,
+   "30": 0.5049793102098873,
+   "45": 0.5036779222509986
+  },
+  "res": {
+   "15": 0.5075904273140687,
+   "30": 0.49876532366138193,
+   "45": 0.48974432628349995
+  },
+  "dann": {
+   "15": 0.47348192372436493,
+   "30": 0.4588599874942516,
+   "45": 0.4520852473361714
+  },
+  "coral": {
+   "15": 0.5034106048326475,
+   "30": 0.49598761101812194,
+   "45": 0.5052691962026917
+  }
+ },
+ "lam": 10.0,
+ "lam_scores": {
+  "0.1": 0.47686438636808737,
+  "1.0": 0.5023598505755028,
+  "10.0": 0.5164665414687039,
+  "100.0": 0.5110354354278966
+ },
+ "lam_scores_detail": {
+  "mlp_n5_lam0.1": 0.45843394893840683,
+  "mlp_n5_lam1.0": 0.4902173111525717,
+  "mlp_n5_lam10.0": 0.5159979937689978,
+  "mlp_n5_lam100.0": 0.5110197940367305,
+  "mlp_n50_lam0.1": 0.5188497753529593,
+  "mlp_n50_lam1.0": 0.5315092766217162,
+  "mlp_n50_lam10.0": 0.5211942721123203,
+  "mlp_n50_lam100.0": 0.5217126350704279,
+  "res_n5_lam0.1": 0.44164850765149855,
+  "res_n5_lam1.0": 0.48026217346247213,
+  "res_n5_lam10.0": 0.5157151590135133,
+  "res_n5_lam100.0": 0.5115771641086307,
+  "res_n50_lam0.1": 0.5039531195788676,
+  "res_n50_lam1.0": 0.531478393529792,
+  "res_n50_lam10.0": 0.5354666708732717,
+  "res_n50_lam100.0": 0.5252084667522385,
+  "dann_n5_lam0.1": 0.4237513819893938,
+  "dann_n5_lam1.0": 0.4558276064004225,
+  "dann_n5_lam10.0": 0.48826092860771464,
+  "dann_n5_lam100.0": 0.48228629574435083,
+  "dann_n50_lam0.1": 0.5117978780121744,
+  "dann_n50_lam1.0": 0.5280537862420951,
+  "dann_n50_lam10.0": 0.5259261419367528,
+  "dann_n50_lam100.0": 0.5167561712218101,
+  "coral_n5_lam0.1": 0.44543159288615286,
+  "coral_n5_lam1.0": 0.47716034693851395,
+  "coral_n5_lam10.0": 0.5038241213303521,
+  "coral_n5_lam100.0": 0.5058353781970332,
+  "coral_n50_lam0.1": 0.5110488865352449,
+  "coral_n50_lam1.0": 0.5243699102564388,
+  "coral_n50_lam10.0": 0.5253470441067087,
+  "coral_n50_lam100.0": 0.5138875782919512
+ }
+}

reports/nnlab/cache/ts_nn/select_tcnn_dann_st.json

@@ -0,0 +1,34 @@
+{
+ "backbone": "tcnn_dann",
+ "variant": "st",
+ "seed_used": 0,
+ "split": "lab.madrid_split(pm) defaults (frac .25, axis 0, high)",
+ "epoch_grid": [
+  5,
+  10,
+  15
+ ],
+ "pseudo_target_f1_by_epoch": {
+  "5": 0.5044430346339247,
+  "10": 0.5093685357526514,
+  "15": 0.514016324587161
+ },
+ "epochs": 15,
+ "lam_grid": [
+  0.1,
+  1.0,
+  10.0,
+  100.0
+ ],
+ "pseudo_target_ft_f1_by_lam": {
+  "0.1": 0.4312605201838526,
+  "1.0": 0.4417334872748878,
+  "10.0": 0.4598041504441417,
+  "100.0": 0.48013005722863794
+ },
+ "lam": 100.0,
+ "lam_sim": "n per class in (5, 50) x 5 reps on the pseudo-target, argmax macro F1 on the rest",
+ "sec_per_full_epoch": 67.5705314318339,
+ "n_src": 61932,
+ "n_pseudo_target": 14331
+}

reports/nnlab/cache/ts_nn/select_tcnn_ry.json

@@ -0,0 +1,34 @@
+{
+ "backbone": "tcnn",
+ "variant": "ry",
+ "seed_used": 0,
+ "split": "lab.madrid_split(pm) defaults (frac .25, axis 0, high)",
+ "epoch_grid": [
+  5,
+  10,
+  15
+ ],
+ "pseudo_target_f1_by_epoch": {
+  "5": 0.5340027718176347,
+  "10": 0.5362399360770098,
+  "15": 0.5545017383586993
+ },
+ "epochs": 15,
+ "lam_grid": [
+  0.1,
+  1.0,
+  10.0,
+  100.0
+ ],
+ "pseudo_target_ft_f1_by_lam": {
+  "0.1": 0.48627533687387514,
+  "1.0": 0.5195767800933085,
+  "10.0": 0.5493702887227292,
+  "100.0": 0.5540445428257714
+ },
+ "lam": 100.0,
+ "lam_sim": "n per class in (5, 50) x 5 reps on the pseudo-target, argmax macro F1 on the rest",
+ "sec_per_full_epoch": 33.42512311935425,
+ "n_src": 61932,
+ "n_pseudo_target": 14331
+}

reports/nnlab/cache/ts_nn/select_tcnn_st.json

@@ -0,0 +1,34 @@
+{
+ "backbone": "tcnn",
+ "variant": "st",
+ "seed_used": 0,
+ "split": "lab.madrid_split(pm) defaults (frac .25, axis 0, high)",
+ "epoch_grid": [
+  5,
+  10,
+  15
+ ],
+ "pseudo_target_f1_by_epoch": {
+  "5": 0.526023912531838,
+  "10": 0.5385065163765071,
+  "15": 0.5474806666222004
+ },
+ "epochs": 15,
+ "lam_grid": [
+  0.1,
+  1.0,
+  10.0,
+  100.0
+ ],
+ "pseudo_target_ft_f1_by_lam": {
+  "0.1": 0.47930721991287895,
+  "1.0": 0.5090665027050114,
+  "10.0": 0.5454646105613945,
+  "100.0": 0.554988512762798
+ },
+ "lam": 100.0,
+ "lam_sim": "n per class in (5, 50) x 5 reps on the pseudo-target, argmax macro F1 on the rest",
+ "sec_per_full_epoch": 35.32085949579875,
+ "n_src": 61932,
+ "n_pseudo_target": 14331
+}

reports/nnlab/confirmation.csv

@@ -0,0 +1,65 @@
+method,decision,view,desc,draws_per_n,n5,n25,n50,n100,n200,sd5,sd25,sd50,sd100,sd200,avg,peak
+ens_e12,rule_smooth,fresh,"FINAL-2: zero-shot anchor + LightGBM with domain copies, smoothing",7,0.672499216993028,0.7049940677798013,0.7156098725090241,0.7316009925751713,0.7442810489806773,0.00738342158795785,0.01158221309657147,0.014639118643277261,0.0036558892779421336,0.0033358534108253635,0.7137970397675404,0.7442810489806773
+ens_e12,rule_smooth,B,"FINAL-2: zero-shot anchor + LightGBM with domain copies, smoothing",12,0.6618963189230493,0.686588267133969,0.7061669326017977,0.7169059063747305,0.7308672265727516,0.00977124277542788,0.01088337217866623,0.01506383648735865,0.004865555505143384,0.007348442321925852,0.7004849303212597,0.7308672265727516
+ens_e12,rule_smooth,all,"FINAL-2: zero-shot anchor + LightGBM with domain copies, smoothing",12,0.6788992991872043,0.7031459999822108,0.720872838064683,0.7322218410198911,0.7443948262586132,0.0111451684532862,0.010204675886384367,0.013728418074680249,0.0033115318819225566,0.004448169264457225,0.7159069609025205,0.7443948262586132
+ens_e12,rule_smooth,first5,"FINAL-2: zero-shot anchor + LightGBM with domain copies, smoothing",5,0.6878594142590512,0.700558705065584,0.7282409898426059,0.7330910288424988,0.7445541144477235,0.009369197099419522,0.008414653752560104,0.0089721983797269,0.0029137566853388915,0.006137249373050134,0.7188608504914926,0.7445541144477235
+ens_e12,rule,fresh,FINAL-2 without coordinates,7,0.6516885123651744,0.693797220653644,0.7021475444659664,0.7195121117797006,0.7343699773574539,0.012212803354992453,0.013891042304498263,0.01581522936874291,0.004241910523961266,0.002770748028318377,0.7003030733243879,0.7343699773574539
+ens_e12,rule,B,FINAL-2 without coordinates,12,0.6392308173266427,0.6751414668474016,0.6926853048553282,0.7032561079631837,0.7208019501214507,0.010789283170353445,0.011787671993686173,0.014337748317707606,0.004760529058473119,0.006033071617100818,0.6862231294228014,0.7208019501214507
+ens_e12,rule,all,FINAL-2 without coordinates,12,0.6581184151032142,0.6916508913430035,0.7076119674665423,0.7195498421304539,0.735108629399027,0.013005091638930509,0.012009882553406733,0.014470594175165122,0.0044122055691860375,0.004074468051105237,0.7024079490884482,0.735108629399027
+ens_e12,rule,first5,FINAL-2 without coordinates,5,0.6671202789364699,0.6886460303081068,0.7152621596673485,0.7196026646215085,0.7361427422572293,0.00823054968203162,0.009373508878982978,0.008674726783001862,0.0051516112125037285,0.00564326426345011,0.7053547751581327,0.7361427422572293
+lgbm_feda,rule_smooth,fresh,single model: LightGBM with domain copies + smoothing,7,0.6225215988073963,0.6987180472657218,0.7117810874254776,0.7302946783914661,0.7442753480620923,0.0286171373738811,0.014437439688972821,0.015818057636161422,0.003977961611314101,0.0034275753829592676,0.7015181519904308,0.7442753480620923
+lgbm_feda,rule_smooth,B,single model: LightGBM with domain copies + smoothing,12,0.6181771043949493,0.6787718645922062,0.7030722589903492,0.7153855785703288,0.7306202195898068,0.029386425041193903,0.012936623659237619,0.015967345918769565,0.004846691346137096,0.00731989164743793,0.689205405227528,0.7306202195898068
+lgbm_feda,rule_smooth,all,single model: LightGBM with domain copies + smoothing,12,0.6344174583550974,0.6956395257372289,0.7176313726745821,0.7309265706578091,0.7441859203220972,0.030987330440029034,0.013163777578211404,0.014991614797866972,0.003542693927368331,0.004359711473597244,0.704560169549363,0.7441859203220972
+lgbm_feda,rule_smooth,first5,single model: LightGBM with domain copies + smoothing,5,0.6510716617218788,0.6913295955973388,0.7258217720233281,0.7318112198306893,0.7440607214861039,0.028597749244392248,0.011138659031892529,0.009949546503577466,0.003016801939031133,0.005883322455053052,0.7088189941318678,0.7440607214861039
+lgbm_feda,rule,fresh,single model: LightGBM with domain copies,7,0.6068064907339805,0.6865433879814257,0.6978570759264401,0.7182891453372162,0.7340105004959818,0.030806604830152108,0.0172002023875822,0.01813868075010362,0.004443684453089558,0.0027368647529825694,0.688701320095009,0.7340105004959818
+lgbm_feda,rule,B,single model: LightGBM with domain copies,12,0.602663850027922,0.6666432437898443,0.6890515635985178,0.7022168476683185,0.7205734858550638,0.029499212847983382,0.01440777885566807,0.015885497702506506,0.004896987522407284,0.005983854154559139,0.6762297981879333,0.7205734858550638
+lgbm_feda,rule,all,single model: LightGBM with domain copies,12,0.6179690339594646,0.6830595979741189,0.7038838195807409,0.7184562345920446,0.7347887799261619,0.0307051778830208,0.015388864703454018,0.016459283438953774,0.00458140603355678,0.00419473500196923,0.6916314932065062,0.7347887799261619
+lgbm_feda,rule,first5,single model: LightGBM with domain copies,5,0.6335965944751423,0.6781822919638897,0.7123212606967619,0.7186901595488039,0.735878371128414,0.02541301926650535,0.012510084263689629,0.009946282621948807,0.005289968535235071,0.0058829238055425935,0.6957337355626023,0.735878371128414
+tab_dann_rz_lpft,rule,fresh,"single model: DANN, last layer then whole network fine-tuned",7,0.5932221557189944,0.6503028904656575,0.6584299522248689,0.6864550465487932,0.6993375177971751,0.011100498482265965,0.018116311824433993,0.016537925888728613,0.009169442937689311,0.001820490918382736,0.6575495125510978,0.6993375177971751
+tab_dann_rz_lpft,rule,B,"single model: DANN, last layer then whole network fine-tuned",12,0.5898668678096975,0.6397950263758413,0.6583727546169323,0.6825387949941787,0.6953526153663443,0.01491814667234172,0.016688917197257812,0.014978510681864291,0.008237294415316855,0.00366544871443404,0.6531852118325988,0.6953526153663443
+tab_dann_rz_lpft,rule,all,"single model: DANN, last layer then whole network fine-tuned",12,0.5963733968985947,0.6468589190108454,0.6647318290794766,0.6878619475633201,0.7007834713174289,0.01570019936095448,0.01690866624695203,0.01511421575059053,0.007991903031818825,0.003003142089115108,0.6593219127739331,0.7007834713174289
+tab_dann_rz_lpft,rule,first5,"single model: DANN, last layer then whole network fine-tuned",5,0.600785134550035,0.6420373589741085,0.6735544566759273,0.6898316089836578,0.7028078062457841,0.021244429081834586,0.01562421702380619,0.007152401473284613,0.00641969875146283,0.0033240499690942106,0.6618032730859026,0.7028078062457841
+ens_e4__lp,rule_smooth,fresh,FINAL-AVG as pre-registered: ensemble + label propagation + smoothing,7,0.6768457570277929,0.7046501196233966,0.7137423584365991,0.7246133955872265,0.7284902150148829,0.008721904733148569,0.008428447379451663,0.009542978640932986,0.004621388020941063,0.00445863903919215,0.7096683691379797,0.7284902150148829
+ens_e4__lp,rule_smooth,B,FINAL-AVG as pre-registered: ensemble + label propagation + smoothing,12,0.6719915853181985,0.6924151637836582,0.7051837526854823,0.7123632875785991,0.7146859985388013,0.013048519814050715,0.008997019302678605,0.009674094468344845,0.005915144940895477,0.0072342302081197865,0.6993279575809479,0.7146859985388013
+ens_e4__lp,rule_smooth,all,FINAL-AVG as pre-registered: ensemble + label propagation + smoothing,12,0.6840473093405883,0.7038279491367313,0.71639443300221,0.7247136175582963,0.7278244561748326,0.01185205913059215,0.0075418024463774,0.008068664511018252,0.0041484573243592985,0.005434747568186179,0.7113615530425317,0.7278244561748326
+ens_e4__lp,rule_smooth,first5,FINAL-AVG as pre-registered: ensemble + label propagation + smoothing,5,0.6941294825785019,0.7026769104553997,0.7201073373940654,0.724853928317794,0.7268923937987621,0.0073733820758668436,0.006857077981533023,0.0035903821952188494,0.0039049510176568858,0.007038788509785335,0.7137320105089048,0.7268923937987621
+ens_e4,rule_smooth,fresh,ensemble + smoothing (no label propagation),7,0.6687499694029553,0.6995427983568853,0.7059857993622197,0.7205085631245615,0.725464165138276,0.009325201751519418,0.012436499355543026,0.006058458327959933,0.005256988142917495,0.0033962739202174717,0.7040502590769795,0.725464165138276
+ens_e4,rule_smooth,B,ensemble + smoothing (no label propagation),12,0.6634531641304235,0.6874755161468822,0.702065889113855,0.712409439874463,0.7169275491105283,0.011182794394758727,0.012027901088742686,0.007002809370420917,0.004994466241767745,0.004416079699906983,0.6964663116752304,0.7169275491105283
+ens_e4,rule_smooth,all,ensemble + smoothing (no label propagation),12,0.674401995605212,0.6961771577557442,0.7099124427641058,0.7206348121086751,0.7254093646317085,0.010994352861178856,0.011324747409215897,0.0067869540519492145,0.00431061282428305,0.0041041019538490394,0.7053071545730891,0.7254093646317085
+ens_e4,rule_smooth,first5,ensemble + smoothing (no label propagation),5,0.6823148322883712,0.6914652609141468,0.7154097435267461,0.7208115606864343,0.7253326439225136,0.008233994986476242,0.008550544274544908,0.0026186550580667525,0.003094830131862326,0.005385667787559618,0.7070668082676423,0.7253326439225136
+ens_e4,rule,fresh,"ensemble, no coordinates used",7,0.6465751493319589,0.6833292312537262,0.6898034210079995,0.706245986902281,0.7131636536261885,0.009345035915703424,0.014295504265282696,0.006119337575231637,0.005408978326102121,0.002232989025031671,0.6878234884244309,0.7131636536261885
+ens_e4,rule,B,"ensemble, no coordinates used",12,0.6394250094414752,0.668878059365579,0.6835747677001226,0.69645382913262,0.7026027487751983,0.01098563328102197,0.012422269042199053,0.005982028867070602,0.004059093182501103,0.003268755122546141,0.6781868828829991,0.7026027487751983
+ens_e4,rule,all,"ensemble, no coordinates used",12,0.6519397192963646,0.6792924772161718,0.6935719636267647,0.7064385647158286,0.7133183128535028,0.010776702476115356,0.013051802719372597,0.006660278590003988,0.004725770104129584,0.0033307703589248254,0.6889122075417264,0.7133183128535028
+ens_e4,rule,first5,"ensemble, no coordinates used",5,0.6594501172465327,0.6736410215635955,0.6988479232930359,0.7067081736547951,0.7135348357717427,0.008216623973394226,0.009668428080667852,0.0024840752513013765,0.004168231227233318,0.004788398894739578,0.6904364143059404,0.7135348357717427
+ens_e2__lapshot,rule_smooth,fresh,"FINAL-PEAK as pre-registered: zero-shot + LightGBM, graph refinement + smoothing",7,0.6686536646547763,0.6816532046885747,0.6951857214146546,0.7206055971997382,0.7374120599540205,0.002463763631428892,0.0083120821554866,0.005765376516195115,0.006160247970887419,0.004463529749615921,0.7007020495823528,0.7374120599540205
+ens_e2__lapshot,rule_smooth,B,"FINAL-PEAK as pre-registered: zero-shot + LightGBM, graph refinement + smoothing",12,0.6561908371726143,0.6663640486027115,0.6819525278258015,0.7009470046597029,0.7209545044945992,0.005166844883804179,0.0074962076602055035,0.008248233884150661,0.006141985125033626,0.006633220933080744,0.6852817845510859,0.7209545044945992
+ens_e2__lapshot,rule_smooth,all,"FINAL-PEAK as pre-registered: zero-shot + LightGBM, graph refinement + smoothing",12,0.670967575803796,0.6821005482838619,0.6985253375166253,0.7200842603212064,0.7370038220653504,0.004130158027865544,0.0072038990328673735,0.006912698465907201,0.0047486961785936236,0.0045939852843930795,0.7017363087981681,0.7370038220653504
+ens_e2__lapshot,rule_smooth,first5,"FINAL-PEAK as pre-registered: zero-shot + LightGBM, graph refinement + smoothing",5,0.6742070514124234,0.6827268293172637,0.7032008000593839,0.7193543886912621,0.7364322890212123,0.00391373116616002,0.006183679361152368,0.005891328431297651,0.00198703491527846,0.00523959121621748,0.7031842717003091,0.7364322890212123
+lgbm_upw__lp,rule_smooth,fresh,retrained LightGBM + label propagation + smoothing,7,0.5332826718559118,0.6860908842993512,0.7020830674695988,0.7241251629789701,0.7366129454097383,0.046613172483015004,0.00846274247260879,0.005667813623528082,0.005745510725552485,0.004148989844138986,0.676438946402714,0.7366129454097383
+lgbm_upw__lp,rule_smooth,B,retrained LightGBM + label propagation + smoothing,12,0.5223299976669146,0.6695038836392128,0.6882573601504917,0.705891179278817,0.7195916410498672,0.030230393775252423,0.008852061742802543,0.009081452146300116,0.0064992529978443025,0.006123455532624046,0.6611148123570606,0.7195916410498672
+lgbm_upw__lp,rule_smooth,all,retrained LightGBM + label propagation + smoothing,12,0.5346177958775603,0.6857636253509423,0.7046945969273692,0.723997052596177,0.7351489112880073,0.036544447895441264,0.00845979808095178,0.0065195661426676,0.004441789348001175,0.00483738104022122,0.6768443964080112,0.7351489112880073
+lgbm_upw__lp,rule_smooth,first5,retrained LightGBM + label propagation + smoothing,5,0.536486969507868,0.6853054628231698,0.7083507381682477,0.7238176980602666,0.7330992635175837,0.02014813486930934,0.009430571362390776,0.006329100554593296,0.002161210980477082,0.005433899542902769,0.6774120264154272,0.7330992635175837
+lgbm_upw,rule_smooth,fresh,reference: retrained LightGBM + smoothing,7,0.5873160414221517,0.6522176400810775,0.6802419456929016,0.7117838755731898,0.7318983525013045,0.010654587813852371,0.014183367085171355,0.006999748234154283,0.005859491344115335,0.00403604058709312,0.672691571054125,0.7318983525013045
+lgbm_upw,rule_smooth,B,reference: retrained LightGBM + smoothing,12,0.5736752918817979,0.6328102428094997,0.6663115816000734,0.6915085720312771,0.7155085541619539,0.008518342851765332,0.010774366769898725,0.007924964674703551,0.0066747538027266615,0.005991438264944419,0.6559628484969204,0.7155085541619539
+lgbm_upw,rule_smooth,all,reference: retrained LightGBM + smoothing,12,0.5905330955325642,0.652969029060798,0.6837711439979994,0.7117136521731874,0.7315300661782537,0.010462037818516226,0.012068981411196921,0.008189844656681324,0.004835159425841999,0.003974037174302396,0.6741033973885606,0.7315300661782537
+lgbm_upw,rule_smooth,first5,reference: retrained LightGBM + smoothing,5,0.5950369712871415,0.6540209736324069,0.6887120216251362,0.7116153394131841,0.7310144653259822,0.009340832343393684,0.00982052683674668,0.007658038506830065,0.0035735750398084905,0.004292618066564809,0.6760799542567701,0.7310144653259822
+lgbm_upw,rule,fresh,reference: retrained LightGBM (published 0.656),7,0.5587162873028325,0.6290793019791455,0.6602826649106962,0.6970530639683284,0.7210263907222026,0.010457145961885562,0.015981643113231046,0.01047899688378953,0.004960545342808152,0.00369711592882089,0.653231541776641,0.7210263907222026
+lgbm_upw,rule,B,reference: retrained LightGBM (published 0.656),12,0.5470265227304633,0.6097048483399865,0.6474342753259147,0.6779914585815363,0.7047731065711577,0.010148481612631072,0.011986885424747393,0.008664565337145729,0.0053619421317610124,0.005934952070844711,0.6373860423098117,0.7047731065711577
+lgbm_upw,rule,all,reference: retrained LightGBM (published 0.656),12,0.5616558634936385,0.6287830043018005,0.6641905213606436,0.6971584267926021,0.720760272830069,0.010722353202077647,0.013840857119647339,0.01051548818692131,0.004334363717749738,0.00456689021045552,0.6545096177557508,0.720760272830069
+lgbm_upw,rule,first5,reference: retrained LightGBM (published 0.656),5,0.5657712701607671,0.6283681875535173,0.66966152039057,0.6973059347465853,0.7203877077810819,0.010763078238031729,0.011971948185350032,0.008673459655728825,0.003834880518714044,0.0060460647590979385,0.6562989241265044,0.7203877077810819
+zs_xgb_dann,rule_smooth,fresh,reference: teammate zero-shot + smoothing + our rule,7,0.65363215794613,0.6532586859668588,0.6530865433456813,0.6520272602263627,0.650977205347756,0.0001174352118254124,0.00022328487192891828,0.00037411780796178746,0.0004587168303349501,0.0004554582321847033,0.6525963705665578,0.65363215794613
+zs_xgb_dann,rule_smooth,B,reference: teammate zero-shot + smoothing + our rule,12,0.6354078791919407,0.6353133392594531,0.6349340700548592,0.6342129540788476,0.6330576970096694,0.0002094259082653045,0.000320338562006621,0.0005901311973584275,0.0007515405870228785,0.0010520206077295922,0.634585187918954,0.6354078791919407
+zs_xgb_dann,rule_smooth,all,reference: teammate zero-shot + smoothing + our rule,12,0.6536055914878547,0.6533087776462186,0.6530446126889283,0.6521444204207306,0.6510544616218926,0.00011917257407918898,0.00022062826702112413,0.0003826700084946487,0.0004965265933157694,0.000759724253637244,0.652631572773125,0.6536055914878547
+zs_xgb_dann,rule_smooth,first5,reference: teammate zero-shot + smoothing + our rule,5,0.6535683984462691,0.6533789059973224,0.6529859097694743,0.6523084446928457,0.6511626204056836,0.00012411678374760522,0.00022031465428563876,0.0004305454274571706,0.0005519948067571508,0.0011184884681806721,0.652680855862319,0.6535683984462691
+met_protoda_maha,rule,fresh,"member: metric learning (protoda, Mahalanobis)",7,0.5833704351012031,0.6528758680438218,0.6602302192290609,0.6785200533124248,0.6814483907530346,0.01670259290442601,0.01742972048901337,0.010366372859478267,0.00549330240125798,0.0019256071767115434,0.6512889932879091,0.6814483907530346
+met_protoda_maha,rule,B,"member: metric learning (protoda, Mahalanobis)",12,0.5871303284675647,0.6393251173355056,0.656614654258726,0.6689159504489116,0.6716191442623094,0.02699440493575894,0.016514850402680043,0.009897001535316516,0.0062893366574944525,0.0038546275694179674,0.6447210389546034,0.6716191442623094
+met_protoda_maha,rule,all,"member: metric learning (protoda, Mahalanobis)",12,0.592402334432983,0.6473261377867038,0.6648906748633155,0.6785510462472226,0.6818108874867211,0.027979468356922103,0.01704110300254631,0.010147042144437766,0.006928089904001761,0.0036557442526554606,0.6529962161633892,0.6818108874867211
+met_protoda_maha,rule,first5,"member: metric learning (protoda, Mahalanobis)",5,0.6050469934974748,0.6395565154267386,0.6714153127512723,0.6785944363559396,0.6823183829138824,0.037306279744327736,0.014613251430266759,0.005543544999451586,0.00931273807515169,0.005535200159853275,0.6553863281890615,0.6823183829138824
+met_cos_maha,rule,fresh,"member: metric learning (cosine, Mahalanobis)",7,0.5700533977719742,0.6551110163539057,0.6640934945019076,0.687023900498173,0.6894433813264372,0.024801545651275504,0.018100606591864576,0.015193295993671033,0.008454609310388933,0.0033235438941739363,0.6531450380904795,0.6894433813264372
+met_cos_maha,rule,B,"member: metric learning (cosine, Mahalanobis)",12,0.5697170150489529,0.6372636632579837,0.6592399730945279,0.6770127760900965,0.677967323463002,0.025090043628907273,0.019164213012153188,0.012181700038200733,0.006264196403055141,0.005044835969091504,0.6442401501909126,0.677967323463002
+met_cos_maha,rule,all,"member: metric learning (cosine, Mahalanobis)",12,0.5755893233491208,0.6480781951863225,0.6699615891765048,0.6876211062826534,0.6900448774923755,0.02744083610942304,0.01871623886374535,0.013693737551877219,0.006703987842617818,0.0038189889247863032,0.6542590182973954,0.6900448774923755
+met_cos_maha,rule,first5,"member: metric learning (cosine, Mahalanobis)",5,0.583339619157126,0.6382322455517061,0.678176921720941,0.6884571943809261,0.690886972124689,0.031927392708083355,0.01625226275959985,0.0049795903641816335,0.0038569311506127324,0.004692482713679311,0.6558185905870777,0.690886972124689
+tab_dann_rz_ft,rule,fresh,"member: DANN MLP, last layer fine-tuned",7,0.5871333868982462,0.6290693988106211,0.63836675665432,0.6490971519746042,0.6578265983649486,0.013559630581688964,0.013254474598541758,0.011100502590838847,0.0054902171129630785,0.003628651433373751,0.6322986585405481,0.6578265983649486
+tab_dann_rz_ft,rule,B,"member: DANN MLP, last layer fine-tuned",12,0.5896764505190034,0.6226657532710625,0.6361443589442797,0.6467032320577332,0.6540751682995946,0.01623228642536196,0.012510796481692522,0.007777074530588764,0.005589452349877741,0.0028081404597116917,0.6298529926183346,0.6540751682995946
+tab_dann_rz_ft,rule,all,"member: DANN MLP, last layer fine-tuned",12,0.59459600863892,0.6287141867665803,0.6402791739424749,0.6496058418860532,0.6586499165001437,0.017129754849396422,0.011872176030203204,0.008858102582867585,0.005203083509861856,0.003622612964030036,0.6343690255468345,0.6586499165001437
+tab_dann_rz_ft,rule,first5,"member: DANN MLP, last layer fine-tuned",5,0.6050436790758632,0.6282168899049231,0.6429565581458918,0.6503180077620816,0.659802561889417,0.01724041671210301,0.011115618171028857,0.003948276155095075,0.0053054381363549594,0.003673076979180324,0.6372675393556353,0.659802561889417

reports/nnlab/leaderboard.csv

@@ -0,0 +1,403 @@
+method,decision,n5,n25,n50,n100,n200,avg,peak,draws
+ens_e12,rule_smooth,0.6788992991872043,0.7031459999822108,0.720872838064683,0.7322218410198911,0.7443948262586132,0.7159069609025205,0.7443948262586132,60
+ens_e11,rule_smooth,0.6842954020444802,0.6975747938537019,0.7237695537762203,0.730258736011159,0.7360978029200862,0.7143992577211294,0.7360978029200862,25
+ens_e10,rule_smooth,0.683175486555451,0.6969789186006878,0.7230823758598024,0.7289044181700924,0.7334709363938623,0.7131224271159793,0.7334709363938623,25
+ens_e6__lapshot,rule_smooth,0.684660852807642,0.7001035446137475,0.7223524798021297,0.7253904591089992,0.7327536796555214,0.713052203197608,0.7327536796555214,25
+ens_e7__lp,rule_smooth,0.6899195731804131,0.7050044149678546,0.7199217932566309,0.7246812946067405,0.725671587674035,0.7130397327371348,0.725671587674035,25
+ens_e6__lp,rule_smooth,0.688223225154486,0.7035375631218013,0.7189207150888388,0.7241918270535026,0.7274471883927498,0.7124641037622756,0.7274471883927498,25
+ens_e5__lp,rule_smooth,0.6874494823188417,0.7035349492230926,0.7185790110716536,0.7229046191629571,0.7252284024971469,0.7115392928547383,0.7252284024971469,25
+ens_e5__lapshot,rule_smooth,0.6844372397521665,0.6997819837478436,0.7215792119983583,0.7228622935584907,0.7282558798552665,0.7113833217824252,0.7282558798552665,25
+ens_e4__lp,rule_smooth,0.6840473093405883,0.7038279491367313,0.71639443300221,0.7247136175582963,0.7278244561748326,0.7113615530425317,0.7278244561748326,60
+ens_e1__lp,rule_smooth,0.6888083000118221,0.6986280508448665,0.7200456849846978,0.7231922889884103,0.725583427544083,0.711251550474776,0.725583427544083,25
+ens_e4__lapshot,rule_smooth,0.686879180873759,0.6959792696102169,0.7200239601361694,0.7230378280963274,0.728932610622016,0.7109705698676978,0.728932610622016,25
+ens_e6,rule_smooth,0.6828073572803083,0.6968775761232975,0.7185214393317755,0.7235218797015998,0.7291556002408152,0.7101767705355593,0.7291556002408152,25
+ens_e7__lp,rule,0.6865727626047263,0.7006176919078652,0.7162218853769106,0.7201824843481935,0.7240102518456265,0.7095210152166643,0.7240102518456265,25
+ens_e5,rule_smooth,0.6827970547659374,0.6966957208140533,0.7179832154635605,0.7209060198695303,0.725384383486056,0.7087532788798276,0.725384383486056,25
+ens_e1__lapshot,rule_smooth,0.6818306366312885,0.6920326767430793,0.7187386278863788,0.7223075014029182,0.7281752504885486,0.7086169386304426,0.7281752504885486,25
+ens_e7__lp,argmax,0.6767875533065513,0.6978345951616433,0.7188288258525167,0.7207476158393813,0.728799381823614,0.7085995943967414,0.728799381823614,25
+ens_e6__lp,rule,0.6818555668549866,0.6996118992625291,0.7151202190823999,0.7192241460831713,0.7258069297740228,0.7083237522114219,0.7258069297740228,25
+ens_e7,rule_smooth,0.6841296812091141,0.6950024021141654,0.7156012162408263,0.7209922505134886,0.7246465098111662,0.708074411977752,0.7246465098111662,25
+lgbm_fedaw100,rule_smooth,0.6500842616561742,0.687390531243028,0.7254599697744972,0.7322793224455494,0.7437737018259962,0.7077975573890491,0.7437737018259962,25
+ens_e5__lp,rule,0.6807755387079839,0.6991788426705094,0.7139162294108341,0.717502936758577,0.7226925141979343,0.7068132123491677,0.7226925141979343,25
+ens_e6__lp,argmax,0.6731980196095082,0.6926228045955141,0.7172722378483695,0.7188959127993446,0.72979119941179,0.7063560348529052,0.72979119941179,25
+ens_e8,rule_smooth,0.6728847738561173,0.6948278316516164,0.7107131284081779,0.723745057237481,0.7285388807742893,0.7061419343855364,0.7285388807742893,60
+ens_e1__lp,rule,0.6822047044399846,0.6941384583575335,0.7141815523047264,0.7170300041459481,0.7227853981201029,0.706068023473659,0.7227853981201029,25
+ens_e4__lp,rule,0.6763772432758866,0.6987979360146405,0.7110754437734649,0.7200364073353337,0.7240321382533627,0.7060638337305376,0.7240321382533627,60
+ens_e1__clamp,argmax,0.6738183210708576,0.6840081680070381,0.7180044156361696,0.7223339472158569,0.7305200297380277,0.7057369763335899,0.7305200297380277,25
+ens_e1__clamp,rule,0.6779918176079663,0.6883731202158894,0.7149685376254891,0.7209287346103229,0.7262652818088705,0.7057054983737077,0.7262652818088705,25
+ens_e4,rule_smooth,0.674401995605212,0.6961771577557442,0.7099124427641058,0.7206348121086751,0.7254093646317085,0.7053071545730891,0.7254093646317085,60
+ens_e9,rule_smooth,0.6731362263976588,0.6919253508271642,0.7077315408728251,0.7224112192129142,0.7306226982211478,0.705165407106342,0.7306226982211478,60
+ens_e1,rule_smooth,0.6779931622315245,0.6880808371075484,0.714570396227594,0.7201875519351136,0.7249945649002891,0.7051653024804139,0.7249945649002891,25
+ens_e5__lp,argmax,0.6724519352037773,0.6933209932959403,0.7160024813056867,0.7172726662479348,0.7267247700923808,0.705154569229144,0.7267247700923808,25
+ens_e3__lp,rule_smooth,0.6638932454179811,0.6924879158659347,0.7196363203931748,0.7232360248832708,0.7257187637653211,0.7049944540651364,0.7257187637653211,25
+lgbm_feda,rule_smooth,0.6344174583550974,0.6956395257372289,0.7176313726745821,0.7309265706578091,0.7441859203220972,0.704560169549363,0.7441859203220972,60
+ens_e1__lp,argmax,0.6739622128558759,0.6853940842679482,0.7164328864751373,0.7167866346400545,0.7275705076810837,0.7040292651840199,0.7275705076810837,25
+ens_e6__lapshot,argmax,0.6685439037719079,0.6876305747728289,0.7149094390302729,0.7179888501689298,0.7283961764990362,0.7034937888485951,0.7283961764990362,25
+ens_e6__lapshot,rule,0.6710436735076469,0.6897962582093984,0.7130037829222156,0.7169031312522514,0.725642898705997,0.7032779489195018,0.725642898705997,25
+ens_e4__lp,argmax,0.6641485814321423,0.6925100878430208,0.7103725757441786,0.7200994975669213,0.7269158044159045,0.7028093094004335,0.7269158044159045,60
+ens_e12,rule,0.6581184151032142,0.6916508913430035,0.7076119674665423,0.7195498421304539,0.735108629399027,0.7024079490884482,0.735108629399027,60
+ens_e5__lapshot,argmax,0.6698208611881155,0.687739450273839,0.7139908875492168,0.71520807353071,0.7230475272305904,0.7019613599544943,0.7230475272305904,25
+ens_e2__lapshot,rule_smooth,0.670967575803796,0.6821005482838619,0.6985253375166253,0.7200842603212064,0.7370038220653504,0.7017363087981681,0.7370038220653504,60
+ens_e12,argmax,0.6496094288329871,0.6885067387896454,0.7085193738649255,0.7221050023975951,0.738766135175447,0.70150133581212,0.738766135175447,60
+ens_e5__lapshot,rule,0.6710284512652981,0.689330140274898,0.711330730946502,0.7138991274718375,0.7206914673314889,0.7012559834580049,0.7206914673314889,25
+ens_e11,argmax,0.6594475506734996,0.6790179148707506,0.7136241093216416,0.7214062335032108,0.7320691431192242,0.7011129902976652,0.7320691431192242,25
+ens_e11,rule,0.662852976774265,0.6818004198269607,0.7109109949627515,0.7195889681210266,0.728094193209878,0.7006495105789764,0.728094193209878,25
+ens_e1__clamp,rule_smooth,0.6763894779691869,0.6855208262377365,0.7095274742761302,0.7145767830075294,0.7166774781748722,0.700538407933091,0.7166774781748722,25
+ens_e10,argmax,0.6590726936800924,0.6780807125640536,0.7117005888442345,0.7192208464908223,0.7280180595870787,0.6992185802332562,0.7280180595870787,25
+ens_e3__lapshot,rule_smooth,0.6463367097725268,0.6826195210700937,0.715820449391489,0.7216371275640785,0.7279134741966947,0.6988654563989767,0.7279134741966947,25
+ens_e3__lp,rule,0.6547102241932757,0.6870903827435778,0.712813170151048,0.7167755678531197,0.7228679304004153,0.6988514550682873,0.7228679304004153,25
+ens_e4__lapshot,rule,0.6699664552386826,0.682930236344943,0.7075511759329997,0.7133291670337969,0.7202788581688091,0.6988111785438462,0.7202788581688091,25
+ens_e2__lp,argmax,0.6567738552972365,0.6751965576637119,0.7059156551892906,0.7189337506413449,0.7369719909920555,0.6987583619567279,0.7369719909920555,25
+ens_e10,rule,0.6611961507270271,0.6806064605801465,0.7095458288443016,0.7170775706647083,0.7244663301178421,0.698578468186805,0.7244663301178421,25
+ens_e4__lapshot,argmax,0.6675112440380424,0.6794751262451613,0.7093839381472614,0.7137100039017785,0.7220233522818085,0.6984207329228104,0.7220233522818085,25
+ens_e1__lp__st,rule_smooth,0.657999534932628,0.6812848467128154,0.7054084936765098,0.7158803414804977,0.7308323348963768,0.6982811103397655,0.7308323348963768,25
+ens_e2__clamp,rule,0.6693259566310157,0.6757454828016316,0.6965301003577913,0.7142301112125666,0.732721360743505,0.697710602349302,0.732721360743505,25
+ens_e6,argmax,0.6613055686295564,0.6800172325175586,0.7093151004543584,0.7140637923614612,0.7237523174898568,0.6976908022905584,0.7237523174898568,25
+ens_e2__lp,rule_smooth,0.6226929796465261,0.6967757155563706,0.71110279282024,0.723834255967514,0.7329847596658243,0.697478100731295,0.7329847596658243,25
+ens_e3__lp,argmax,0.6502573417821178,0.6770456952730507,0.7151790723993547,0.7166787386628103,0.7278983387021959,0.6974118373639059,0.7278983387021959,25
+ens_e6,rule,0.6645475503266836,0.6822577636416034,0.7062967187067954,0.7123113759950035,0.7202493242908878,0.6971325465921947,0.7202493242908878,25
+ens_e1__lapshot,argmax,0.6632153450405317,0.6762581280770071,0.7087064712674956,0.7126031221327315,0.7222770372114,0.6966120207458332,0.7222770372114,25
+ens_e2__lp,rule,0.6261546422087345,0.6919580003112824,0.7102541793303935,0.7213064844801049,0.7326904734249415,0.6964727559510914,0.7326904734249415,25
+ens_e5,argmax,0.6626943386568603,0.6810658046616269,0.7082582755142466,0.7113297530191065,0.7185212373884589,0.69637388184806,0.7185212373884589,25
+ens_e1__lapshot,rule,0.6653430385362213,0.6788368607809968,0.7065068872978642,0.7117135259831138,0.7192228327484689,0.6963246290693329,0.7192228327484689,25
+ens_e3__clamp,argmax,0.6382915526817348,0.6746128808383689,0.71548420648657,0.7213216496216139,0.730184137288635,0.6959788853833845,0.730184137288635,25
+ens_e5,rule,0.6648027354142452,0.6830019267113232,0.7051682391496723,0.7094029700494859,0.7151383577785366,0.6955028458206527,0.7151383577785366,25
+ens_e2,rule_smooth,0.666914306649892,0.6740643820950228,0.6908207084606484,0.7138587969715933,0.7317242621612211,0.6954764912676754,0.7317242621612211,60
+ens_e3__clamp,rule,0.639900379235732,0.6788332067218603,0.7119571266527366,0.7199312391523097,0.7259630619424184,0.6953170027410114,0.7259630619424184,25
+ens_e3,rule_smooth,0.6398729222243562,0.6786513653499651,0.7114196219151803,0.7192209966377394,0.7246719346782782,0.6947673681611038,0.7246719346782782,25
+lgbm_fedaw100,argmax,0.6279771015655384,0.6690488889561583,0.7149832239887846,0.721267750502745,0.7389757194171381,0.6944505368860729,0.7389757194171381,25
+lgbm_fedaw100,rule,0.6324609855303651,0.6733886990545495,0.7121306616655836,0.7184222949834074,0.7352095103148821,0.6943224303097575,0.7352095103148821,25
+ens_e7,argmax,0.6573757447487659,0.6772060308542658,0.7038777237958398,0.7121242074963867,0.7187927045700879,0.6938752822930692,0.7187927045700879,25
+ens_e2__clamp,argmax,0.6549429138564002,0.664580673332074,0.6946157517949004,0.7141580260709476,0.7383612205858847,0.6933317171280414,0.7383612205858847,25
+ens_e7,rule,0.6614162365991414,0.67803259544767,0.7011058845432995,0.7096468206365754,0.7156370267010563,0.6931677127855485,0.7156370267010563,25
+ens_e3__clamp,rule_smooth,0.6455146680860786,0.6776802494067866,0.7075111429492462,0.7143668300507424,0.7166420553376598,0.6923429891661026,0.7166420553376598,25
+lgbm_feda,rule,0.6179690339594646,0.6830595979741189,0.7038838195807409,0.7184562345920446,0.7347887799261619,0.6916314932065062,0.7347887799261619,60
+ens_e1__lp__st,argmax,0.6462557197889276,0.6714572955426237,0.6997574733477883,0.7096022661582985,0.7296862415896399,0.6913517992854555,0.7296862415896399,25
+ens_e2__clamp,rule_smooth,0.6677866224047998,0.6725393619092234,0.6897325226426657,0.7059861528100584,0.720465354374403,0.6913020028282302,0.720465354374403,25
+ens_e8,argmax,0.6462938517931496,0.6776377681565112,0.6964289321860405,0.7132202560244624,0.7204572677507803,0.6908076151821888,0.7204572677507803,60
+lgbm_feda,argmax,0.6073573893526882,0.6800330419332279,0.7048122297280783,0.7210286292812292,0.7382809100690788,0.6903024400728606,0.7382809100690788,60
+ens_e8,rule,0.6506775561238369,0.6782110045785,0.6945395336152954,0.7104507714997602,0.7174487140664875,0.690265515976776,0.7174487140664875,60
+ens_e1__lp__st,rule,0.6478260863987348,0.6728955473184891,0.6971915066201448,0.7069272487574709,0.7249943697373681,0.6899669517664415,0.7249943697373681,25
+ens_e2__lapshot,rule,0.6513293206475943,0.6679992615544461,0.6870985237418217,0.710908674293131,0.7297232411848992,0.6894118042843784,0.7297232411848992,60
+ens_e9,argmax,0.6456701370135759,0.6739620953150252,0.6934097318318792,0.7113823068697732,0.722335931696553,0.6893520405453613,0.722335931696553,60
+ens_e1,argmax,0.6521243156839601,0.667930180478006,0.7012087147581714,0.707450988283721,0.7172979373873212,0.689202427318236,0.7172979373873212,25
+ens_e4,rule,0.6519397192963646,0.6792924772161718,0.6935719636267647,0.7064385647158286,0.7133183128535028,0.6889122075417264,0.7133183128535028,60
+ens_e4,argmax,0.6464409048152091,0.6777014241969569,0.6952631047892059,0.708653247131639,0.7159883188460837,0.6888093999558189,0.7159883188460837,60
+ens_e9,rule,0.6508483871096729,0.6741403445679629,0.6908598727594679,0.7085027337449178,0.7190119568065207,0.6886726589977085,0.7190119568065207,60
+ens_e1,rule,0.6545856261750449,0.6696019652879277,0.6985096395119862,0.7059959224191272,0.7138560643425625,0.6885098435473297,0.7138560643425625,25
+ens_e3__lapshot,argmax,0.6272620377369909,0.6659944615196254,0.706139437119879,0.711919606700311,0.7220333758011248,0.6866697837755862,0.7220333758011248,25
+ens_e2__lapshot,argmax,0.6402635091255183,0.6610127393657268,0.6847695138836176,0.7118506008680452,0.7323258799568748,0.6860444486399565,0.7323258799568748,60
+ens_e3__lapshot,rule,0.6282645973899544,0.6680554975817083,0.703496883605778,0.7109260864874556,0.7187054524705079,0.6858897035070809,0.7187054524705079,25
+lgbm_upw__lapshot,rule_smooth,0.604772217426235,0.6629222780502004,0.6966458784956412,0.7176847602463369,0.7363847871654352,0.6836819842767697,0.7363847871654352,25
+lgbm_w100,rule_smooth,0.6000146875795004,0.6583273814949774,0.6968499703105209,0.7204836404399736,0.7396834638050606,0.6830718287260066,0.7396834638050606,25
+tab_dann_rz_lpft_ent,rule_smooth,0.6246573380905047,0.658532944634719,0.6929202108973769,0.709195489242237,0.7175097319885173,0.6805631429706709,0.7175097319885173,25
+tab_dann_rz_lpft,rule_smooth,0.6184475788811031,0.6654070433796238,0.6864262758533548,0.7074603120734615,0.7180036855847995,0.6791489791544685,0.7180036855847995,60
+ens_e3,argmax,0.6156519229322155,0.6577155507600295,0.6982410914341861,0.7064325546241762,0.7169359327547455,0.6789954105010706,0.7169359327547455,25
+ens_e3,rule,0.6166089942568499,0.6590703946363548,0.6952057189383869,0.7047831030817577,0.713506342420837,0.6778349106668372,0.713506342420837,25
+ens_e2,rule,0.6381503449806335,0.6527651382027831,0.6733863114367059,0.6994618858304888,0.721517378847548,0.6770562118596318,0.721517378847548,60
+lgbm_upw__lp,rule_smooth,0.5346177958775603,0.6857636253509423,0.7046945969273692,0.723997052596177,0.7351489112880073,0.6768443964080112,0.7351489112880073,60
+lgbm_upw__clamp,rule,0.5952116427473666,0.6544069267199623,0.6890583502913022,0.7124126851942238,0.7324423385634137,0.6767063887032536,0.7324423385634137,25
+lgbm_upw__lp,argmax,0.5638141063169904,0.6635430167704945,0.6961642516576109,0.7213008031119048,0.7373521240758697,0.6764348603865741,0.7373521240758697,60
+lgbm_upw__lp,rule,0.5416126834039883,0.681415557517043,0.7029489087178898,0.721884725247548,0.7341045608917419,0.6763932871556422,0.7341045608917419,60
+met_cos_maha,rule_smooth,0.6017035139850071,0.6714034110984493,0.69159637865643,0.7056233770701725,0.705481130025968,0.6751615621672054,0.7056233770701725,60
+met_protoda_maha,rule_smooth,0.6172205377709873,0.6691546462604298,0.6870972711839983,0.6981598372362078,0.7003542632295291,0.6743973111362305,0.7003542632295291,60
+lgbm_upw,rule_smooth,0.5905330955325642,0.652969029060798,0.6837711439979994,0.7117136521731874,0.7315300661782537,0.6741033973885606,0.7315300661782537,60
+ens_e2,argmax,0.6256039400382424,0.6459616954004502,0.6714552318692059,0.7008937935835334,0.7241321391592567,0.6736093600101377,0.7241321391592567,60
+lgbm_upw__clamp,rule_smooth,0.6020422615771632,0.6547646952303333,0.684472006171013,0.7046856040262577,0.7205535314428515,0.6733036196895237,0.7205535314428515,25
+zs_xgb_dann__lp,rule,0.6463859416937227,0.6601926111647844,0.670794985969557,0.685365447348994,0.6975685431119831,0.6720615058578081,0.6975685431119831,25
+lgbm_upw__clamp,argmax,0.5712339602939884,0.6412896097557623,0.6884074617865447,0.7130583885142818,0.7381325318379082,0.6704243904376971,0.7381325318379082,25
+lgbm_upw__lapshot,rule,0.5822294542824318,0.6450207944282533,0.684738893654426,0.708441536998834,0.729801011020183,0.6700463380768256,0.729801011020183,25
+met_proto_maha,rule_smooth,0.6261144998955801,0.6550407551222655,0.6842692004190614,0.68876473825635,0.6931792529360562,0.6694736893258626,0.6931792529360562,25
+lgbm_upw__lapshot,argmax,0.5652553482507335,0.6364681978068394,0.6840264163059728,0.7094816309766688,0.7327713063799138,0.6656005799440257,0.7327713063799138,25
+tab_dann_o60_ft,rule_smooth,0.6435835404436034,0.6583574994243668,0.6698122574955846,0.6741293287989487,0.6819852733988002,0.6655735799122607,0.6819852733988002,25
+lgbm_upw__em,rule_smooth,0.5882490788485105,0.6324610191287515,0.6766429809729717,0.7001612441801864,0.728508289243497,0.6652045224747833,0.728508289243497,25
+zs_dann__lp,rule_smooth,0.6523517397428568,0.656917238027465,0.6606796931315558,0.6709954522803638,0.682176047458991,0.6646240341282466,0.682176047458991,25
+lgbm_w100,rule,0.5707143778241539,0.6360140269333134,0.6799941512224404,0.7057380697315042,0.7306413614229645,0.6646203974268753,0.7306413614229645,25
+tab_dann_rz_lpft_ent,argmax,0.6051632206036209,0.6409495204351893,0.6769676094436888,0.6931843417220256,0.7049226810593636,0.6642374746527777,0.7049226810593636,25
+zs_xgb_dann__lp,argmax,0.6455511669123342,0.6522458080957273,0.6606160810339047,0.671944439250554,0.690732082160099,0.6642179154905238,0.690732082160099,25
+zs_xgb_dann__clamp,rule_smooth,0.6617223516059658,0.6622464378445522,0.6628863729549723,0.6638135554727566,0.6656241008169246,0.6632585637390344,0.6656241008169246,25
+tab_dann_rz_lpft_s300,rule_smooth,0.608627240427375,0.6451900391780931,0.6666532288395696,0.6903720614787452,0.7053376268628838,0.6632360393573334,0.7053376268628838,60
+lgbm_w100,argmax,0.5614270206719791,0.6306240280307852,0.6807684997647232,0.7074661918289153,0.7350500526861372,0.663067158596508,0.7350500526861372,25
+tab_dann_rz_lpft_ent,rule,0.6054737510955908,0.642210707251061,0.6751498444609497,0.6902938449505983,0.7020879446577174,0.6630432184831835,0.7020879446577174,25
+tab_dann_rz_ft,rule_smooth,0.6246415561042958,0.6578377596755574,0.670313885003445,0.6767047345653056,0.6835657929220577,0.6626127456541322,0.6835657929220577,60
+zs_xgb_dann__lp,rule_smooth,0.6410298408303241,0.6444206953179489,0.6477281362667228,0.6717775962443542,0.7077301038503764,0.6625372745019452,0.7077301038503764,25
+cls_lda,rule_smooth,0.6226039952979872,0.6459760123318098,0.6661360833648843,0.684399540332975,0.6932842913601724,0.6624799845375657,0.6932842913601724,25
+tab_res_o60_ft,rule_smooth,0.6213176579170143,0.6502698185542323,0.6665372566262117,0.681559171408216,0.6927104079004562,0.6624788624812261,0.6927104079004562,25
+cls_lr,rule_smooth,0.5873404367304655,0.6287322036647722,0.6809839756236095,0.7009323324326207,0.7133034579776811,0.6622584812858298,0.7133034579776811,25
+met_cos_knn,rule_smooth,0.6116142042262857,0.6505229343850311,0.6710092487200954,0.6792569679173712,0.6943513465678587,0.6613509403633284,0.6943513465678587,25
+tab_dann_rz_lpft,argmax,0.5949773582206593,0.6467586458370027,0.6665399004355198,0.6909004349324376,0.7032911646980855,0.660493500824741,0.7032911646980855,60
+zs_xgb_dann__lapshot,rule_smooth,0.659708096956059,0.6598797012323525,0.6603402416912814,0.660056703959029,0.6606847094150047,0.6601338906507455,0.6606847094150047,25
+tab_dann_rz_lpft,rule,0.5963733968985947,0.6468589190108454,0.6647318290794766,0.6878619475633201,0.7007834713174289,0.6593219127739331,0.7007834713174289,60
+met_cos_maha,argmax,0.5729525070421407,0.6499201998267592,0.6739181917434399,0.6910626298265385,0.6940095508636382,0.6563726158605033,0.6940095508636382,60
+zs_xgb_dann__clamp,rule,0.6537382679782234,0.6544556766589665,0.6549671470803532,0.655945500902946,0.6579719987889503,0.6554157182818878,0.6579719987889503,25
+zs_dann__lp,rule,0.6414154163447613,0.6464567281391727,0.6513763895883009,0.660461429534484,0.675246934186299,0.6549913795586035,0.675246934186299,25
+tab_dann_rg_ft,rule_smooth,0.6297830287445263,0.636612348189814,0.658941184222574,0.6724186349965553,0.6753833530371988,0.6546277098381337,0.6753833530371988,25
+lgbm_upw,rule,0.5616558634936385,0.6287830043018005,0.6641905213606436,0.6971584267926021,0.720760272830069,0.6545096177557508,0.720760272830069,60
+met_cos_maha,rule,0.5755893233491208,0.6480781951863225,0.6699615891765048,0.6876211062826534,0.6900448774923755,0.6542590182973954,0.6900448774923755,60
+met_protoda_maha,rule,0.592402334432983,0.6473261377867038,0.6648906748633155,0.6785510462472226,0.6818108874867211,0.6529962161633892,0.6818108874867211,60
+met_protoda_maha,argmax,0.5882802794153847,0.6467313316301272,0.6655468070358569,0.6804094047081422,0.6826870599093432,0.6527309765397709,0.6826870599093432,60
+zs_xgb_dann,rule_smooth,0.6536055914878547,0.6533087776462186,0.6530446126889283,0.6521444204207306,0.6510544616218926,0.652631572773125,0.6536055914878547,60
+tab_res_rz_ft,rule_smooth,0.5820817603443296,0.6312593647406763,0.6712801434767427,0.6856718553249211,0.6921712784130433,0.6524928804599426,0.6921712784130433,25
+lgbm_emb,rule_smooth,0.5964484854192444,0.6236127320450932,0.6515362186721095,0.6801135506327072,0.7068204959467888,0.6517062965431887,0.7068204959467888,25
+met_supcon_maha,rule_smooth,0.6034206174388984,0.6350683189406736,0.6665994906967955,0.6762438779142483,0.6770604808474706,0.6516785571676172,0.6770604808474706,25
+met_protoda_knn,rule_smooth,0.6053617532925761,0.6342255535433925,0.6559192970362341,0.6727246664635462,0.6888570791204071,0.6514176698912312,0.6888570791204071,25
+met_protoda_ftproj,rule_smooth,0.6021266115982477,0.6330579166303931,0.6620792384113372,0.6780097460166544,0.6814598006770245,0.6513466626667314,0.6814598006770245,25
+lgbm_upw,argmax,0.5427817786896311,0.6221033152878102,0.6621061334270542,0.698581058112028,0.7237250812339558,0.649859473350096,0.7237250812339558,60
+met_proto_maha,rule,0.6010112711968566,0.6335149726630092,0.6633058484443392,0.6690048319525733,0.6748249159000876,0.6483323680313731,0.6748249159000876,25
+met_proto_maha,argmax,0.5990448345957267,0.6311124502470206,0.6644091578438084,0.6690521104901659,0.6761324989296773,0.6479502104212796,0.6761324989296773,25
+cls_nys_lr,rule_smooth,0.5879345486393156,0.6174831235226079,0.6596114851547485,0.6793107055627071,0.6941452265383619,0.6476970178835483,0.6941452265383619,25
+met_cos_ftproj,rule_smooth,0.5723137201491953,0.6230496970635507,0.6711192700424424,0.685047261206038,0.686693625775624,0.6476447148473701,0.686693625775624,25
+cls_lda,rule,0.6000087018577746,0.6250480441007118,0.6481226692393137,0.6730233049709022,0.6875297247292447,0.6467464889795894,0.6875297247292447,25
+met_supcon_ftproj,rule_smooth,0.5858531238827218,0.627012270914474,0.6617461579289461,0.6781916764378011,0.6799936260148659,0.6465593710357618,0.6799936260148659,25
+met_protoda_ftproj,argmax,0.5970444634576766,0.6275521939986552,0.6564945454675494,0.6730018827586682,0.6776152441478533,0.6463416659660806,0.6776152441478533,25
+lgbm_upw__em,rule,0.5582952679709171,0.6111768662045096,0.6577448652883376,0.6867667626861927,0.7166803805511262,0.6461328285402166,0.7166803805511262,25
+met_protoda_ftproj,rule,0.5966061823577448,0.6272612513004321,0.6559664511193896,0.6729622158880775,0.6768438591284826,0.6459279919588252,0.6768438591284826,25
+zs_dann__lp,argmax,0.6281727987873016,0.6337914914596482,0.6397695742225237,0.6518590882861682,0.6693556105106481,0.6445897126532579,0.6693556105106481,25
+zs_xgb_dann__clamp,argmax,0.6421957116303091,0.6428869038483136,0.6435020663213444,0.6446716160582098,0.6476178909657715,0.6441748377647898,0.6476178909657715,25
+tab_dann_rz_lpft_s300,argmax,0.5825342353875445,0.6255922996650667,0.6472878776529972,0.6730477233523176,0.6881043404158653,0.6433132952947582,0.6881043404158653,60
+met_proto_ftproj,rule_smooth,0.5791086099611946,0.6243929940111582,0.6580848059017806,0.6736550358834943,0.6746505658610397,0.6419784023237334,0.6746505658610397,25
+tab_dann_rz_lpft_s300,rule,0.5846028271820413,0.6247899400513209,0.6447463733483348,0.6700647588399935,0.6850364006975113,0.6418480600238403,0.6850364006975113,60
+met_cos_ftproj,argmax,0.566109527922886,0.6164073141283293,0.6635277892190763,0.6781849494415383,0.6816386566024253,0.641173647462851,0.6816386566024253,25
+ts_tcnn_dann_st_ft,rule_smooth,0.5409583541726188,0.634256307875598,0.6565290860787003,0.6831426277211345,0.6905808979540893,0.6410934547604281,0.6905808979540893,25
+met_proto_knn,rule_smooth,0.5888568255886154,0.6285504719362178,0.6464024156551439,0.6595937792617994,0.6817175247873839,0.6410242034458321,0.6817175247873839,25
+met_supcon_ftproj,argmax,0.5800812178104284,0.6215033884293171,0.6556051160520224,0.6720749533738855,0.675789444388287,0.6410108240107881,0.675789444388287,25
+met_cos_ftproj,rule,0.5657316362075362,0.616299610948104,0.6629808104058189,0.6777005394539428,0.6807680825790187,0.6406961359188841,0.6807680825790187,25
+met_supcon_ftproj,rule,0.5799919485659061,0.621178241871625,0.6551349091169365,0.6717284258754499,0.6753002677210823,0.6406667586301998,0.6753002677210823,25
+cls_lr,rule,0.5639279659381311,0.6075924201880035,0.6575884781873389,0.6785931439726444,0.6944947696918324,0.6404393555955901,0.6944947696918324,25
+cls_lr_feda,rule_smooth,0.5625816799997898,0.6214452947416567,0.6598371704454574,0.6680990806411511,0.6864376851209963,0.6396801821898102,0.6864376851209963,25
+zs_xgb_dann__lapshot,rule,0.637447031781476,0.6377015405828058,0.6379455666624111,0.6378184946228587,0.6383243891479753,0.6378474045595054,0.6383243891479753,25
+cls_svm_feda,rule_smooth,0.5631787583437604,0.6166482517476826,0.6561149497241245,0.6675614121966221,0.682922325879719,0.6372851395783817,0.682922325879719,25
+met_cos_knn,rule,0.5835191958070264,0.6245056575237996,0.6443252944737526,0.6560185274703746,0.6751618889741668,0.636706112849824,0.6751618889741668,25
+tab_mlp_rg_ft,rule_smooth,0.5931805686269611,0.6165794572911545,0.6410053516207107,0.6627296111234026,0.6688100720413788,0.6364610121407216,0.6688100720413788,25
+met_proto_ftproj,argmax,0.5728439213173242,0.6182782298926286,0.652106889054429,0.6677534390392827,0.6702171229458085,0.6362399204498945,0.6702171229458085,25
+met_proto_ftproj,rule,0.5726884262825205,0.618060570980723,0.6517324223408889,0.6675948799720604,0.6697184452838172,0.6359589489720021,0.6697184452838172,25
+tab_dann_rz_ft,rule,0.59459600863892,0.6287141867665803,0.6402791739424749,0.6496058418860532,0.6586499165001437,0.6343690255468345,0.6586499165001437,60
+zs_lgbm__clamp,rule_smooth,0.63086457063295,0.6314042385649954,0.6319995887687145,0.6326623842707786,0.6346433177736772,0.6323148200022232,0.6346433177736772,25
+zs_lgbm__lapshot,rule_smooth,0.631893262295766,0.6318868040727723,0.6321449857964444,0.6325201874060223,0.6329055061623543,0.6322701491466718,0.6329055061623543,25
+cls_svm_feda,argmax,0.5549017094736209,0.61084949801365,0.6529376799303478,0.6642603690201011,0.6783622470966435,0.6322623007068727,0.6783622470966435,25
+zs_xgb_dann__lapshot,argmax,0.6306460692777874,0.6310066977780832,0.6312058286190736,0.6312805128008316,0.6327963399599631,0.6313870896871479,0.6327963399599631,25
+cls_svm_feda,rule,0.55595449544196,0.6112017203254153,0.6511503425065026,0.6609450423443807,0.6741484618287912,0.63068001248941,0.6741484618287912,25
+met_supcon_maha,rule,0.5780696430285361,0.6146849626027192,0.6428274978513409,0.6568149883487642,0.6596049305072962,0.6304004044677314,0.6596049305072962,25
+tab_dann_o60_ft,rule,0.6045609255427655,0.6202387118261482,0.6339433979735206,0.6408794758427296,0.649843830310075,0.6298932682990478,0.649843830310075,25
+tab_dann_rz_ft,argmax,0.5809365168290851,0.621006646711826,0.6373844108297434,0.6491579134667799,0.6593232013542124,0.6295617378383294,0.6593232013542124,60
+tab_res_rz_ft,argmax,0.5506508902363919,0.6020611267281859,0.6483314175059022,0.6665508588320017,0.678743396347549,0.6292675379300061,0.678743396347549,25
+tab_dann_rg_ft,rule,0.6029964567373064,0.612586807192321,0.6315823692626794,0.6454007500100436,0.6505801563761647,0.628629307915703,0.6505801563761647,25
+tab_dann_rz_proto,rule_smooth,0.6111115013362006,0.6165580082623805,0.6370622189044953,0.6368819851333982,0.6394884530601286,0.6282204333393206,0.6394884530601286,25
+met_cos_tpn,rule_smooth,0.6226290588213003,0.6127375253580916,0.6351239591819133,0.6349239113914308,0.633713112392377,0.6278255134290226,0.6351239591819133,25
+lgbm_emb,rule,0.5675054991951634,0.597980604944661,0.6260911689921352,0.6565853678899195,0.689316752985693,0.6274958788015145,0.689316752985693,25
+met_protoda_knn,rule,0.5876820631717972,0.6064968395070534,0.6298245657039975,0.646852432119253,0.6658056133875421,0.6273323027779286,0.6658056133875421,25
+tab_dann_o60_ft,argmax,0.6046724864479478,0.6124590215322814,0.628886436847229,0.6395751522886983,0.6510565569868584,0.6273299308206031,0.6510565569868584,25
+tab_res_o60_proto,rule_smooth,0.6116466202106505,0.6201980921229191,0.6343658703497561,0.6347368709194118,0.6355902945571721,0.627307549631982,0.6355902945571721,25
+met_supcon_maha,argmax,0.5694914915353163,0.610887642423887,0.6397202750418349,0.6549758955876974,0.6600971958925632,0.6270345000962598,0.6600971958925632,25
+met_cos_knn,argmax,0.5870864719156316,0.5919426607163396,0.6313529530992903,0.6483284818279266,0.6740211981317012,0.6265463531381779,0.6740211981317012,25
+tab_res_rz_ft,rule,0.5518861568455385,0.6027904889928344,0.6423528764450834,0.6617376220515138,0.6732546782490794,0.6264043645168099,0.6732546782490794,25
+cls_svm,rule_smooth,0.5440844300144887,0.5975861438305677,0.6428842690486455,0.6634291680503578,0.6840361893044762,0.6264040400497072,0.6840361893044762,25
+tab_res_o60_ft,rule,0.5796331929659543,0.6116897421425318,0.629959559492182,0.648158087197898,0.6620720770593618,0.6263025317715856,0.6620720770593618,25
+zs_dann__clamp,rule_smooth,0.6252920544451652,0.6257414630041693,0.6259638325354435,0.6261082712595881,0.6269609038144774,0.6260133050117688,0.6269609038144774,25
+met_cos_pn,rule_smooth,0.6116142042262857,0.6160189660586777,0.6318211246786125,0.632503815200886,0.6347480706296653,0.6253412361588254,0.6347480706296653,25
+zs_lgbm__clamp,rule,0.6236617201058224,0.6239004506013195,0.6247108716617,0.6250001203525788,0.6267348184237893,0.6248015962290421,0.6267348184237893,25
+zs_xgb_dann__em,rule_smooth,0.6254317160665442,0.62536833237458,0.6251030197927742,0.6242597591152683,0.6234029540847137,0.624713156286776,0.6254317160665442,25
+lgbm_emb,argmax,0.5624491050851728,0.591064936852244,0.6228070138487987,0.6546593323465096,0.6905673338280597,0.6243095443921569,0.6905673338280597,25
+tab_mlp_rz_ft,rule_smooth,0.5724295858458623,0.6045728551772724,0.6304746459740314,0.647717563955113,0.6632892506839286,0.6236967803272415,0.6632892506839286,25
+tab_res_o60_ft,argmax,0.5751925063945661,0.6042256749428284,0.6269861958817913,0.6477702895827118,0.6638078403958527,0.62359650143955,0.6638078403958527,25
+cls_nys_lr,rule,0.5583239387727257,0.5934761327586908,0.6342845662876714,0.6558617450902677,0.6743795679281447,0.6232651901675,0.6743795679281447,25
+tab_mlp_o60_ft,rule_smooth,0.5713460540949904,0.6092436546173401,0.6275148709264272,0.6466728018391896,0.6603789998191278,0.6230312762594149,0.6603789998191278,25
+tab_dann_o60_joint,rule_smooth,0.606619005747442,0.6142876131438577,0.6208076298828681,0.6286320883318641,0.6440447869960242,0.6228782248204112,0.6440447869960242,25
+met_supcon_knn,rule_smooth,0.5751351218299324,0.6040848950649026,0.6248004880984935,0.6421107794970276,0.6674320298384983,0.6227126628657709,0.6674320298384983,25
+zs_lgbm,rule_smooth,0.6235170770946137,0.6230831095022396,0.6230366470664969,0.6218829751015775,0.6203861532204591,0.6223811923970773,0.6235170770946137,25
+zs_dann__lapshot,rule_smooth,0.6224201169932595,0.6226332870674307,0.622515559258895,0.6219923240970996,0.6221420731308546,0.6223406721095078,0.6226332870674307,25
+cls_lr,argmax,0.5522019632869162,0.5876454951612403,0.6339729020211727,0.6597507137647999,0.677565483549008,0.6222273115566275,0.677565483549008,25
+tab_dann_o60_proto,rule_smooth,0.6171413454857165,0.6123892246112701,0.6233286261402075,0.627050014812415,0.6284711281443638,0.6216760678387946,0.6284711281443638,25
+tab_dann_rg_ft,argmax,0.5817609904905376,0.6007449079449106,0.6271979797729024,0.6437648132707443,0.6518180577199624,0.6210573498398114,0.6518180577199624,25
+tab_coral_o60_ft,rule_smooth,0.5834643091047192,0.6031194308310317,0.61852216606739,0.6395738499472847,0.6571051010521802,0.6203569714005213,0.6571051010521802,25
+cls_knn,rule_smooth,0.5027451051532876,0.6150002861593297,0.6368362237264386,0.6622333020131468,0.6838173208237013,0.6201264475751808,0.6838173208237013,25
+cls_svm,argmax,0.5341342310094406,0.5910798776276179,0.6373395128099677,0.658025419140586,0.6793237434594563,0.6199805568094138,0.6793237434594563,25
+cls_svm,rule,0.5352697964156211,0.5901015291172208,0.635499079679698,0.6552457270895002,0.6756635787395504,0.6183559422083181,0.6756635787395504,25
+zs_xgb_dann,rule,0.6187345372467739,0.6185528011676743,0.6182348036282918,0.6171753567928356,0.6160739638091858,0.6177542925289523,0.6187345372467739,60
+cls_lr_feda,argmax,0.5378184878318053,0.5990576773513396,0.639977588090531,0.647022385978521,0.6610330125401946,0.6169818303584783,0.6610330125401946,25
+cls_lr_feda,rule,0.539018302804282,0.5998468686347997,0.6389157988584578,0.6455058909561162,0.6609345930664379,0.6168442908640188,0.6609345930664379,25
+cls_lda,argmax,0.5392477282061222,0.5845180180046403,0.6153972044350884,0.6587738271237397,0.6861667261218826,0.6168207007782947,0.6861667261218826,25
+tab_dann_rz_proto,argmax,0.5998376350178567,0.6046203226664104,0.6227771417988548,0.6254250402729256,0.6262148209845776,0.615774992148125,0.6262148209845776,25
+tab_dann_rz_proto,rule,0.599856830468019,0.6047839771438727,0.6229770541140438,0.6251266870386301,0.626023541438647,0.6157536180406424,0.626023541438647,25
+lgbm_upw__em,argmax,0.5279825108276301,0.548576868284955,0.6270469232335731,0.6659870221929667,0.7076007705593588,0.6154388190196968,0.7076007705593588,25
+met_proto_knn,rule,0.5700295722492321,0.6002517285685531,0.6190254679258567,0.6317219507626373,0.6561449284794771,0.6154347295971513,0.6561449284794771,25
+tab_dann_rg_proto,rule_smooth,0.6068111970954835,0.6096624883425805,0.6200924347300921,0.6192273698025603,0.6207672398965414,0.6153121459734516,0.6207672398965414,25
+zs_dann__clamp,rule,0.6134921126112918,0.6141470760051663,0.6142975096406302,0.614651566583664,0.6161191257766758,0.6145414781234855,0.6161191257766758,25
+met_protoda_knn,argmax,0.5841162593078026,0.5791569106586627,0.6104381108879334,0.6351635691099855,0.6573283161621933,0.6132406332253155,0.6573283161621933,25
+tab_res_rz_proto,rule_smooth,0.5713943065265323,0.6025915894763487,0.6246360387257632,0.6278430987801618,0.6361984847182475,0.6125327036454107,0.6361984847182475,25
+zs_dann,rule_smooth,0.6133820367295184,0.613263841146175,0.6129979142280992,0.611886194698046,0.610433121325868,0.6123926216255413,0.6133820367295184,25
+tab_dann_rg_joint,rule_smooth,0.6018592439481585,0.6032615980537669,0.6090461980107883,0.6167523687942227,0.629546713782519,0.6120932245178912,0.629546713782519,25
+zs_lgbm__em,rule_smooth,0.6130629242140546,0.6126547909449833,0.6125534026708453,0.6115313598116232,0.6104936271465948,0.6120592209576203,0.6130629242140546,25
+zs_xgb_dann,argmax,0.6126772665684909,0.612495247028393,0.612390864553025,0.611587366176605,0.6108617624293143,0.6120025013511656,0.6126772665684909,60
+tab_res_o60_proto,argmax,0.5934241622927126,0.6020841831968051,0.6162712879381365,0.617845214755092,0.6182051931242488,0.609566008261399,0.6182051931242488,25
+tab_res_o60_proto,rule,0.5934718085530715,0.6027564615186262,0.6161781982972304,0.6174772435946397,0.6178864829341983,0.6095540389795532,0.6178864829341983,25
+tab_dann_rz_joint,rule_smooth,0.593912807096568,0.5983312311374174,0.6067314150064457,0.6153845138171266,0.6296449075548629,0.6088009749224842,0.6296449075548629,25
+ts_tcnn_dann_st_ft,rule,0.5161819850479267,0.6006834216019348,0.6195388451384579,0.6489866649904362,0.658156935807537,0.6087095705172585,0.658156935807537,25
+zs_lgbm__lapshot,rule,0.6081412001796405,0.6080771439861392,0.6083989344466243,0.6085022217756197,0.6098046506597268,0.6085848302095501,0.6098046506597268,25
+tab_dann_o60_proto,rule,0.6023072860749605,0.598336175639935,0.609226803376747,0.614820556089483,0.6145900358986718,0.6078561714159595,0.614820556089483,25
+tab_dann_o60_proto,argmax,0.6025729927102959,0.5975677063211338,0.6087134482548444,0.6145884728676935,0.6146611186689145,0.6076207477645764,0.6146611186689145,25
+met_protoda_pn,rule_smooth,0.6053617532925761,0.6022946984478463,0.6109606292791374,0.6109019343482857,0.6059603128731721,0.6070958656482036,0.6109606292791374,25
+tab_dann_rz_zs,rule_smooth,0.6079605714198733,0.6076148940781131,0.6074572235843277,0.6064323962177565,0.6052242715218027,0.6069378713643746,0.6079605714198733,25
+cls_nys_lr,argmax,0.5355641807257931,0.5760498298004966,0.6169564939838502,0.6401436242019036,0.661097258346149,0.6059622774116384,0.661097258346149,25
+met_cos_tpn,argmax,0.5987435818073902,0.5892164472242674,0.6120806182178018,0.6121257886951281,0.6132107646939043,0.6050754401276983,0.6132107646939043,25
+tab_dann_rg_proto,rule,0.5974221584461843,0.5992648039060646,0.6096441511422528,0.6086603572392215,0.6087715538006132,0.6047526049068673,0.6096441511422528,25
+tab_dann_rg_proto,argmax,0.5979682956416528,0.5987430685140884,0.6094083253077397,0.6081489290277907,0.6084736660063254,0.6045484568995193,0.6094083253077397,25
+ts_tcnn_dann_st_ft,argmax,0.5022059868760326,0.5947551791064752,0.6185569872326964,0.6479409345338729,0.6572066920391103,0.6041331559576375,0.6572066920391103,25
+tab_mlp_rg_ft,rule,0.5631777272583085,0.5838437630614275,0.6034544212438113,0.6265284033461122,0.637644893125023,0.6029298416069365,0.637644893125023,25
+zs_dann__em,rule_smooth,0.6038842845185292,0.6037234506802858,0.6033698883484535,0.6020894531575689,0.6012078269050123,0.6028549807219699,0.6038842845185292,25
+zs_dann__clamp,argmax,0.6011081407423383,0.6016019733627948,0.6019351309295622,0.6025387166614262,0.6043783670558545,0.6023124657503953,0.6043783670558545,25
+tab_dann_o60_zs,rule_smooth,0.6031780912782284,0.6030910502631108,0.6028843220171034,0.6014138220639913,0.6002735646717479,0.6021681700588364,0.6031780912782284,25
+met_proto_knn,argmax,0.5655529598471377,0.5730439990859383,0.602702472251931,0.6203396714157695,0.6475632200709152,0.6018404645343384,0.6475632200709152,25
+met_cos_pn,argmax,0.5870864719156316,0.5916611000257808,0.6089422692426105,0.6087843709519399,0.6116472562424141,0.6016242936756754,0.6116472562424141,25
+tab_dann_rg_zs,rule_smooth,0.6024469748338761,0.6020973464126783,0.6019153795048817,0.6009140295918523,0.5991010987295091,0.6012949658145595,0.6024469748338761,25
+tab_mlp_rg_ft,argmax,0.5582925212642229,0.5769445154268074,0.6026558294263918,0.6283884237911334,0.6397878733063967,0.6012138326429904,0.6397878733063967,25
+met_cos_pn,rule,0.5835191958070264,0.5929934606076949,0.6076769026822746,0.6089620038119242,0.6104868338033034,0.6007276793424448,0.6104868338033034,25
+met_cos_tpn,rule,0.595264652375984,0.5849070040518571,0.6067770519763143,0.6071868028724117,0.6074359611764357,0.6003142944906006,0.6074359611764357,25
+zs_xgb_dann__em,rule,0.6006889846444727,0.6007150845873612,0.6004707025788687,0.5995817661095187,0.5990746174293742,0.6001062310699191,0.6007150845873612,25
+met_proto_pn,rule_smooth,0.5888568255886154,0.5930701070703274,0.6056067545542195,0.6058659635843646,0.6025973871453606,0.5991994075885774,0.6058659635843646,25
+tab_res_o60_joint,rule_smooth,0.5444567321508188,0.5755797675104619,0.6017603539166194,0.626851939973967,0.6470632688147508,0.5991424124733236,0.6470632688147508,25
+met_supcon_knn,rule,0.55679307615682,0.5787676560459115,0.5998020627466405,0.6171061915026541,0.6418000596491146,0.5988538092202281,0.6418000596491146,25
+tab_res_rz_proto,rule,0.5538823055851785,0.5870469483262513,0.6064709921769719,0.6117113140827688,0.6193967502064883,0.5957016620755317,0.6193967502064883,25
+tab_coral_rz_ft,rule_smooth,0.5528714459203703,0.5737076281054423,0.5938599110451023,0.6165126740807128,0.6379223268425129,0.5949747971988281,0.6379223268425129,25
+tab_res_rz_proto,argmax,0.5538043201574406,0.5858827366365781,0.6055937587072823,0.6092005784411918,0.6169741927102199,0.5942911173305425,0.6169741927102199,25
+zs_dann__lapshot,rule,0.593783389768639,0.593836409421785,0.5941008562313598,0.5934408501259403,0.5932937796401813,0.5936910570375811,0.5941008562313598,25
+cls_knn,rule,0.5471251694797491,0.5662120407689927,0.5896155615827786,0.618146742892972,0.6449378729158669,0.5932074775280718,0.6449378729158669,25
+tab_mlp_rz_ft,rule,0.5447497676562894,0.5754603136095823,0.5959544150115986,0.6142356695999359,0.6352636990938207,0.5931327729942454,0.6352636990938207,25
+met_protoda_pn,rule,0.5876820631717972,0.5878099285872038,0.5962038922000937,0.5975583286178435,0.5934849366051497,0.5925478298364175,0.5975583286178435,25
+zs_lgbm,rule,0.5927544843144255,0.5924973098097549,0.5923412806740466,0.5916320787733866,0.5902670380787424,0.5918984383300712,0.5927544843144255,25
+zs_lgbm__lp,argmax,0.5530516408474849,0.5630541953599373,0.577093885710591,0.608076911659886,0.6530676788319661,0.5908688624819731,0.6530676788319661,25
+tab_mlp_rz_ft,argmax,0.5355919966052267,0.5703135759908096,0.5935676169984566,0.6144387472144673,0.6352462754570443,0.5898316424532009,0.6352462754570443,25
+met_protoda_tpn,rule_smooth,0.5929333899108137,0.5888944458513636,0.5894537079923751,0.5891664248399038,0.5872966443528773,0.5895489225894668,0.5929333899108137,25
+zs_lgbm__clamp,argmax,0.5871967673723406,0.5878112700478667,0.5885307288331811,0.5900988043647702,0.5923017826095387,0.5891878706455396,0.5923017826095387,25
+cls_knn,argmax,0.5192734643914119,0.5546818395213632,0.5896660227718478,0.6236737762646021,0.6519229120445301,0.587843602998751,0.6519229120445301,25
+zs_dann__lapshot,argmax,0.58731406995048,0.5876005599652203,0.5876741112986498,0.5873013656565691,0.5879277407996861,0.587563569534121,0.5879277407996861,25
+tab_dann_o60_joint,rule,0.5733845263960993,0.5784740583657035,0.5855435942297205,0.5909141339445434,0.6093649541657183,0.587536253420357,0.6093649541657183,25
+tab_mlp_rg_proto,rule_smooth,0.5627005407118553,0.5816222900854449,0.5956925206312654,0.5982400571400894,0.5989268379891461,0.5874364493115601,0.5989268379891461,25
+tab_coral_o60_proto,rule_smooth,0.5714904138553228,0.5804102178785622,0.5898644718456275,0.5944943419235638,0.592320013494876,0.5857158917995904,0.5944943419235638,25
+tab_dann_rg_joint,rule,0.5772484337616662,0.5779496020981318,0.5831169103219855,0.5884319356101597,0.600139635022376,0.5853773033628639,0.600139635022376,25
+met_protoda_pn,argmax,0.5841162593078026,0.5794604483030312,0.5867570078546047,0.5874222262457265,0.5822222588467902,0.5839956401115911,0.5874222262457265,25
+tab_dann_rz_zs,rule,0.5834128867492175,0.5830601888668777,0.5830466386481545,0.5823093249088485,0.5808567944297411,0.582537166720568,0.5834128867492175,25
+zs_lgbm__em,rule,0.5832047297791046,0.5830370463652392,0.5831548591690394,0.5822464755184835,0.5807596559238689,0.5824805533511472,0.5832047297791046,25
+tab_dann_rz_joint,rule,0.5674934367707609,0.5719346094803035,0.5779467883757463,0.5874635248680595,0.6025915633844202,0.5814859845758582,0.6025915633844202,25
+met_proto_pn,rule,0.5700295722492321,0.5771087273464721,0.5864962286327037,0.5871489631427738,0.584920481440496,0.5811407945623355,0.5871489631427738,25
+tab_mlp_o60_ft,rule,0.5450353872605389,0.5676904573007179,0.5787222494483585,0.5982451591412481,0.6154892586539159,0.5810365023609558,0.6154892586539159,25
+met_supcon_pn,rule_smooth,0.5751351218299324,0.5765597093927779,0.5838749873367006,0.5832311345561846,0.583718593832158,0.5805039093895507,0.5838749873367006,25
+tab_res_rz_joint,rule_smooth,0.5219308814829811,0.5514444525565653,0.5804985523785142,0.6070980630008863,0.637479801188271,0.5796903501214435,0.637479801188271,25
+zs_lgbm__lapshot,argmax,0.5784926757241469,0.5787226292404946,0.5793701326562017,0.5797006730683598,0.5810267988392532,0.5794625819056913,0.5810267988392532,25
+ts_tcnn_dann_st_joint,rule_smooth,0.4848044264888176,0.5530549889757703,0.5866895767776246,0.6188381059934105,0.6526049942988271,0.57919841850689,0.6526049942988271,25
+tab_dann_rg_zs,rule,0.5799811695798142,0.5797413307285667,0.5795582510212203,0.5787441821261698,0.5772982358577574,0.5790646338627056,0.5799811695798142,25
+tab_dann_rg_joint,argmax,0.5714232090026939,0.5715706095021312,0.5772529944499354,0.5820526252005273,0.5923961545105809,0.5789391185331738,0.5923961545105809,25
+tab_coral_o60_ft,rule,0.5443352234910026,0.5618623037227073,0.5759606154100534,0.595264047391343,0.6141463613040148,0.5783137102638243,0.6141463613040148,25
+tab_mlp_rg_joint,rule_smooth,0.5502835020498569,0.5574213969403127,0.5738283086165046,0.5881775380616622,0.6181780725624489,0.577577763646157,0.6181780725624489,25
+tab_dann_o60_joint,argmax,0.5589531977307018,0.5672679706481055,0.5749417228331597,0.5826877485864154,0.5995068201907465,0.5766714919978257,0.5995068201907465,25
+zs_dann,rule,0.5772414174359534,0.5770952912539341,0.5770014087738893,0.575701700906843,0.574661674975559,0.5763402986692358,0.5772414174359534,25
+tab_coral_o60_ft,argmax,0.5426363268503565,0.5578521291327657,0.5715555601168371,0.5926243786247819,0.612509165889267,0.5754355121228016,0.612509165889267,25
+zs_lgbm__lp,rule,0.5311144343893515,0.5387410494427572,0.5474484983671206,0.5664821997304226,0.6924636766464473,0.5752499717152199,0.6924636766464473,25
+met_protoda_tpn,rule,0.5768851623032276,0.5738281720939239,0.5748837484952065,0.5746162555686756,0.5725573801609267,0.5745541437243921,0.5768851623032276,25
+tab_dann_rz_zs,argmax,0.5750758160946872,0.5747712238302485,0.5747072948415174,0.5740104402566124,0.5727835823648345,0.57426967147758,0.5750758160946872,25
+tab_mlp_o60_ft,argmax,0.5407439203117596,0.5602314774146987,0.567202814800748,0.5918780985394198,0.6104430573885462,0.5740998736910343,0.6104430573885462,25
+met_proto_pn,argmax,0.5655529598471377,0.5693757231372741,0.5797885309206198,0.5795644286600254,0.5757965889602227,0.574015646305056,0.5797885309206198,25
+met_supcon_knn,argmax,0.5488830956800322,0.5399016614085281,0.5659819099632962,0.5936622141427432,0.6215671541300087,0.5739992070649217,0.6215671541300087,25
+tab_dann_rz_joint,argmax,0.5587817469970251,0.5630902786132371,0.5697020668953321,0.5800810694332439,0.5931507901256976,0.5729611904129073,0.5931507901256976,25
+met_proto_tpn,rule_smooth,0.5769408052084485,0.5712848883568689,0.5729033510470807,0.5715214218301254,0.569462616372501,0.5724226165630049,0.5769408052084485,25
+tab_mlp_rz_proto,rule_smooth,0.5475409003151649,0.5659887694473275,0.5795834644545492,0.5826227617891212,0.5857809781199176,0.5723033748252161,0.5857809781199176,25
+met_protoda_tpn,argmax,0.574412594339919,0.5704428787003212,0.571353885695318,0.5711860730578572,0.5689844028991231,0.5712759669385077,0.574412594339919,25
+tab_coral_o60_joint,rule_smooth,0.5236994697383535,0.5469120404604663,0.5686578253675814,0.5958609418238794,0.6203430575120985,0.5710946669804757,0.6203430575120985,25
+zs_dann__em,rule,0.5719746391719362,0.5716837193012638,0.5714269410028854,0.5700837442259933,0.5693594645424965,0.570905701648915,0.5719746391719362,25
+tab_mlp_rg_proto,rule,0.5467224008715947,0.5660328417649233,0.578712911149073,0.5808860486899048,0.5804203434798662,0.5705549091910724,0.5808860486899048,25
+tab_dann_o60_zs,rule,0.5708839134647432,0.5707606033648647,0.5705312563053254,0.5693632922678329,0.5685498221083083,0.5700177775022149,0.5708839134647432,25
+ts_tcnn_dann_st_joint,rule,0.5264393433629886,0.5453422272702749,0.5619286386920788,0.5912453893380073,0.6222346250365499,0.5694380447399798,0.6222346250365499,25
+tab_dann_rg_zs,argmax,0.5699528911025601,0.5697447413423229,0.5696295063681018,0.5688922969317168,0.5676082872439444,0.5691655445977293,0.5699528911025601,25
+tab_coral_rz_ft,rule,0.5326914730625901,0.5499112106042874,0.5665537671683473,0.5876991360503246,0.60832435506571,0.5690359883902518,0.60832435506571,25
+tab_coral_o60_proto,rule,0.5569803432226912,0.5621623776143243,0.5705350387313015,0.5776682737008534,0.5741934155344172,0.5683078897607176,0.5776682737008534,25
+tab_coral_o60_proto,argmax,0.5571244380006573,0.5617555964269121,0.5692809573161033,0.5773819619994831,0.5734440607446223,0.5677974028975555,0.5773819619994831,25
+tab_coral_rz_ft,argmax,0.5221299399073571,0.5420595000989397,0.564940460534167,0.5944502629406709,0.6136158445284499,0.567439201601917,0.6136158445284499,25
+zs_lgbm,argmax,0.5679436964284867,0.5677794234496167,0.5677835065763327,0.5671181790537674,0.5660976909097271,0.5673444992835861,0.5679436964284867,25
+tab_mlp_rg_proto,argmax,0.5462350051096003,0.5637688662140825,0.5752588947667171,0.5763811308396396,0.5741778849325325,0.5671643563725144,0.5763811308396396,25
+tab_mlp_o60_proto,rule_smooth,0.5491147930176937,0.560254374275551,0.5703369160174396,0.5794590136824201,0.5756357159373097,0.5669601625860828,0.5794590136824201,25
+zs_dann,argmax,0.566722069374948,0.5666082246448072,0.5664636421915936,0.5652966469348886,0.5645088744931168,0.5659198915278708,0.566722069374948,25
+tab_mlp_rz_joint,rule_smooth,0.5294311901216555,0.5429719711084974,0.5597423123141698,0.5827506454992064,0.6103359646767108,0.5650464167440481,0.6103359646767108,25
+tab_mlp_o60_joint,rule_smooth,0.5169648327933505,0.5299122860805145,0.5632579015402589,0.5924502086132722,0.6166991040926106,0.5638568666240014,0.6166991040926106,25
+met_supcon_pn,rule,0.55679307615682,0.5603240249799432,0.5662962800238136,0.566729606781982,0.5658745929639891,0.5632035161813096,0.566729606781982,25
+tab_res_o60_joint,rule,0.5111218143648875,0.5358292512847191,0.5579662134169576,0.5833374826879325,0.6079586582169035,0.55924268399428,0.6079586582169035,25
+met_proto_tpn,rule,0.5628071336409615,0.5580310188450215,0.5583592452863433,0.5582141029734795,0.5564166146691062,0.5587656230829824,0.5628071336409615,25
+tab_coral_rz_proto,rule_smooth,0.5382800096500783,0.5509722714790428,0.5646463188946019,0.5662910140322388,0.5665757836029717,0.5573530795317867,0.5665757836029717,25
+met_proto_tpn,argmax,0.5585660092443077,0.5547320760686313,0.5558475013428447,0.5552606753538245,0.5532046324274862,0.5555221788874188,0.5585660092443077,25
+met_supcon_tpn,rule_smooth,0.5583574595287688,0.5548235273771296,0.5561863757813499,0.555566545762051,0.5524599314468897,0.5554787679792378,0.5583574595287688,25
+tab_mlp_rz_proto,rule,0.5323985721597686,0.5490950589310682,0.5609736565380972,0.565796371697697,0.5684792968622286,0.5553485912377719,0.5684792968622286,25
+tab_dann_o60_zs,argmax,0.5556328096329569,0.5555260655570041,0.5552991536168961,0.5543106860029221,0.553423218072671,0.55483838657649,0.5556328096329569,25
+tab_mlp_rg_zs,rule_smooth,0.5559559129227728,0.5554685324757327,0.5550848681540426,0.5541027385240839,0.5530316133098361,0.5547287330772936,0.5559559129227728,25
+tab_res_rz_joint,rule,0.4988133738325269,0.5232840673655004,0.5481908773800557,0.573230369307185,0.6058378760853541,0.5498713127941245,0.6058378760853541,25
+tab_mlp_rz_proto,argmax,0.5315439627284985,0.5449402424718898,0.554061867995469,0.5586110473359288,0.559877990230229,0.5498070221524031,0.559877990230229,25
+met_supcon_pn,argmax,0.5488830956800322,0.5471130297503939,0.5533004739226074,0.550189888821264,0.5492017367415137,0.5497376449831622,0.5533004739226074,25
+tab_res_o60_joint,argmax,0.4956655673336675,0.5240879978014897,0.5497077661811588,0.5773974397648878,0.5989314017268759,0.549158034561616,0.5989314017268759,25
+tab_coral_o60_zs,rule_smooth,0.5495050737028935,0.5492218582048753,0.548831675643418,0.547911266648992,0.5465142077912466,0.5483968163982851,0.5495050737028935,25
+ts_tcnn_dann_st_joint,argmax,0.5092056602910952,0.5323241589650083,0.5466818981117693,0.5687828792963486,0.5824412600824604,0.5478871713493364,0.5824412600824604,25
+tab_coral_rz_joint,rule_smooth,0.5205156200920916,0.5319585677172556,0.5451581770640479,0.5585012207260573,0.580874262702112,0.5474015696603128,0.580874262702112,25
+tab_mlp_o60_proto,rule,0.5334488419818925,0.5400638321655126,0.548388951214133,0.557902913522624,0.5528676226683181,0.546534432310496,0.557902913522624,25
+cls_lr_zs,rule_smooth,0.5474472819149646,0.5469981102827524,0.546786421071882,0.5461021064482454,0.5448364906271241,0.5464340820689937,0.5474472819149646,25
+tab_mlp_rg_joint,rule,0.520540429599438,0.5282051973516193,0.5416816290511044,0.5543575025287055,0.5851062931464057,0.5459782103354545,0.5851062931464057,25
+tab_coral_rz_proto,rule,0.5281192779026073,0.5396022736187904,0.5523146714357405,0.5552828473271918,0.5545003116970249,0.545963876396271,0.5552828473271918,25
+tab_coral_rz_proto,argmax,0.5285499660880294,0.539027284181836,0.5514071448933869,0.5552287266017261,0.5534857478856309,0.5455397739301218,0.5552287266017261,25
+tab_mlp_o60_proto,argmax,0.5322903766483013,0.5377352571155345,0.5448389649774104,0.5555409519822005,0.5483054711072429,0.543742204366138,0.5555409519822005,25
+met_supcon_tpn,rule,0.5462233932398396,0.5426642141606933,0.5436925451588783,0.5431251301407116,0.5412032790811205,0.5433817123562485,0.5462233932398396,25
+tab_res_rz_joint,argmax,0.4861579648173965,0.5166319115664285,0.5426007856516646,0.5697875899167719,0.6004360892024284,0.5431228682309379,0.6004360892024284,25
+met_supcon_tpn,argmax,0.5403879457850796,0.5381217006347313,0.5392582689351537,0.5385064788165839,0.5364323757751971,0.5385413539893491,0.5403879457850796,25
+tab_mlp_rz_zs,rule_smooth,0.5388428942755187,0.5384709189553418,0.5384362280734533,0.5371067937899257,0.5359712901405822,0.5377656250469645,0.5388428942755187,25
+ts_tcnn_ry_ft,rule_smooth,0.47162837832447807,0.5106269192366648,0.5267165852693309,0.5730836560749852,0.5967883691407837,0.5357687816092487,0.5967883691407837,25
+tab_mlp_rg_joint,argmax,0.5067151685023175,0.5161621044738409,0.5312205844906784,0.5474394829732593,0.5771794419685858,0.5357433564817364,0.5771794419685858,25
+tab_coral_o60_joint,rule,0.49508452975294504,0.5136483681355013,0.5323950541800981,0.5532957198473764,0.5764671638750509,0.5341781671581944,0.5764671638750509,25
+tab_mlp_rz_joint,rule,0.503205095507103,0.514243560870226,0.5282684266597892,0.5468026915500357,0.5739411511550039,0.5332921851484315,0.5739411511550039,25
+ts_tcnn_dann_st_zs,rule,0.5341278288668868,0.5339063379515137,0.5334976409609616,0.5324053614972661,0.5319991468122897,0.5331872632177836,0.5341278288668868,25
+tab_mlp_o60_joint,rule,0.4902674932934241,0.5098776089064814,0.5295339426914595,0.5511490551619537,0.5763666970954867,0.5314389594297612,0.5763666970954867,25
+zs_lgbm__lp,rule_smooth,0.49915255962928073,0.5099491889230108,0.5217585098213633,0.5451599383259558,0.5782372425994384,0.5308514878598098,0.5782372425994384,25
+tab_res_o60_zs,rule,0.529631994113192,0.5294054274362814,0.5292617205268112,0.5283042534992906,0.527219864111317,0.5287646519373785,0.529631994113192,25
+tab_mlp_rg_zs,rule,0.5299748209109412,0.5296489809855074,0.5291575801573627,0.5282253908762592,0.5268061880732302,0.5287625922006601,0.5299748209109412,25
+cls_lr_zs,rule,0.5266717967797998,0.5263908043092713,0.5260185022474081,0.5254744262187382,0.5244422746142077,0.525799560833885,0.5266717967797998,25
+zs_lgbm__em,argmax,0.5247933450229629,0.5246600190704902,0.5246270698344577,0.5243709921262111,0.523566243551469,0.5244035339211182,0.5247933450229629,25
+tab_coral_rz_zs,rule_smooth,0.5249967835421663,0.5246230795299296,0.5245000728616029,0.5233036234585065,0.5225684810862559,0.5239984080956923,0.5249967835421663,25
+tab_coral_rz_joint,rule,0.5003181702846595,0.5105538818313671,0.521939954668604,0.5323501220800371,0.5530574381732934,0.5236439134075922,0.5530574381732934,25
+tab_coral_o60_joint,argmax,0.4670548570738733,0.491819886333547,0.5211805981262968,0.5467139438454676,0.5696914542236582,0.5192921479205685,0.5696914542236582,25
+tab_coral_rz_joint,argmax,0.4934534170597583,0.5050984199034361,0.5173579759290384,0.5288424850408566,0.5485317898242925,0.5186568175514765,0.5485317898242925,25
+tab_res_o60_zs,rule_smooth,0.5187261599141272,0.5185043055184616,0.5183073769389297,0.5172298789264224,0.5160718325381829,0.5177679107672247,0.5187261599141272,25
+tab_mlp_rz_joint,argmax,0.47970334182327556,0.49506154645855316,0.5136634988129016,0.5346732648877266,0.563718845232612,0.5173640994430138,0.563718845232612,25
+tab_mlp_o60_joint,argmax,0.4679997943257167,0.4877475953884291,0.5148744813554915,0.5446953383143016,0.5660557449915756,0.516274590875103,0.5660557449915756,25
+tab_res_o60_zs,argmax,0.5159476841737358,0.5158533675220461,0.5157014631481197,0.5146639037984804,0.513815894704045,0.5151964626692855,0.5159476841737358,25
+tab_mlp_rz_zs,rule,0.5148745717103704,0.5145607567086312,0.5143998199383164,0.5134102574107628,0.5126831001536069,0.5139857011843375,0.5148745717103704,25
+tab_coral_o60_zs,rule,0.5136815565227376,0.5135590445096421,0.5134338404237737,0.5123183389310078,0.5112880506422541,0.512856166205883,0.5136815565227376,25
+tab_mlp_rg_zs,argmax,0.5122514797178079,0.5119661550303107,0.5117126109808365,0.5110974747163374,0.5099626628117769,0.5113980766514139,0.5122514797178079,25
+cls_lr_zs,argmax,0.5111039657018718,0.5108900651907904,0.5106031185298956,0.5101660120340945,0.509112719040667,0.5103751760994639,0.5111039657018718,25
+ts_tcnn_dann_st_zs,argmax,0.5088958398296077,0.5087644926645984,0.5086065134804272,0.5080872860899512,0.5077741213272731,0.5084256506783715,0.5088958398296077,25
+tab_mlp_o60_zs,rule,0.5091830469461989,0.5089131456188029,0.508743940728937,0.5074602166200842,0.5067744230848479,0.5082149545997741,0.5091830469461989,25
+cls_svm_zs,rule_smooth,0.508284907800179,0.5079316082451029,0.5077715206005968,0.506806551468145,0.5056295509899906,0.5072848278208028,0.508284907800179,25
+zs_dann__em,argmax,0.5070107925129248,0.5068117123761609,0.5065329539641011,0.505488564641201,0.5047289350882662,0.5061145917165308,0.5070107925129248,25
+tab_coral_rz_zs,rule,0.506282808974186,0.5060194677531407,0.5058793225822269,0.5048098073743503,0.5040450428768123,0.5054072899121432,0.506282808974186,25
+ts_tcnn_ry_ft,rule,0.45109368517522974,0.4827485194985564,0.49666850445840593,0.5325409655247131,0.5543814428987398,0.5034866235111289,0.5543814428987398,25
+tab_res_rz_zs,rule_smooth,0.5042078073948891,0.5039716657597555,0.5035782067006824,0.5024417527406394,0.5015007236143464,0.5031400312420625,0.5042078073948891,25
+ts_tcnn_st_ft,rule_smooth,0.4009400075594233,0.4567308957963272,0.4875181070168611,0.5539147281909779,0.602984426142091,0.5004176329411362,0.602984426142091,25
+cls_svm_zs,rule,0.5007028866429847,0.5003258729051041,0.5000462860714805,0.49905857007332416,0.4981042673718674,0.49964757661295217,0.5007028866429847,25
+ts_tcnn_ry_joint,rule_smooth,0.45075762526515495,0.47373284331024196,0.4898699776518837,0.5224427235101696,0.5552877845474784,0.4984181908569857,0.5552877845474784,25
+cls_svm_zs,argmax,0.49906718876179,0.4987765894793393,0.49851001414355844,0.4976020616379852,0.49675503879474087,0.49814217856348275,0.49906718876179,25
+tab_coral_o60_zs,argmax,0.49824642182609297,0.49815194454554534,0.4980680852714916,0.4969986992354358,0.49610249221759395,0.4975135286192319,0.49824642182609297,25
+tab_coral_rz_zs,argmax,0.49482824745533777,0.4946360366322306,0.49451125836478094,0.49347561921816363,0.4926840477379188,0.4940270418816863,0.49482824745533777,25
+ts_tcnn_ry_ft,argmax,0.4304845664498778,0.4739375273839184,0.48794186113327376,0.5279159586316367,0.5490486798188001,0.4938657186835013,0.5490486798188001,25
+tab_mlp_rz_zs,argmax,0.4893823975340846,0.48916378576890773,0.48898100295414587,0.4881063424102381,0.4873144918968163,0.48858960411283847,0.4893823975340846,25
+tab_res_rz_zs,rule,0.4842587857756369,0.48404106702926786,0.48380037423480904,0.48285421992162886,0.48170274432384863,0.4833314382570382,0.4842587857756369,25
+ts_tcnn_dann_st_zs,rule_smooth,0.48335677164733387,0.48316047662668227,0.4827819010199895,0.4820818801213863,0.4806768725063855,0.4824115803843555,0.48335677164733387,25
+tab_mlp_o60_zs,argmax,0.48201499904201056,0.4818040650005083,0.4816773189534544,0.48070147600289,0.4800258595055551,0.48124474370088366,0.48201499904201056,25
+tab_mlp_o60_zs,rule_smooth,0.47816320857987415,0.47792328145434143,0.4772455354091244,0.47640882863081746,0.47524245418765076,0.47699666165236165,0.47816320857987415,25
+zs_xgb_dann__em,argmax,0.47713042748703505,0.47696560034273816,0.4768811448083833,0.4759571126424798,0.4757579529739068,0.47653844765090864,0.47713042748703505,25
+ts_tcnn_st_ft,rule,0.39041465054114805,0.43559365107199977,0.4624041505616785,0.5167883794946397,0.5636451130255788,0.47376918893900904,0.5636451130255788,25
+ts_tcnn_st_joint,rule_smooth,0.3679101954423639,0.42464320708244613,0.4676669905931565,0.5236320962686489,0.569461029616172,0.47066270380055747,0.569461029616172,25
+ts_tcnn_ry_joint,rule,0.4331847122586133,0.45014110720556844,0.462972635715514,0.48624120056602677,0.5121447563686372,0.4689368824228719,0.5121447563686372,25
+tab_res_rz_zs,argmax,0.46929397506440784,0.46917323016602247,0.4689399104792689,0.4681900392211407,0.4673131965937249,0.46858207030491295,0.46929397506440784,25
+ts_tcnn_st_ft,argmax,0.3418516064947565,0.41996842727653644,0.4485440445807245,0.5121252556821351,0.559662437898931,0.4564303543866167,0.559662437898931,25
+ts_tcnn_ry_joint,argmax,0.41062709115762364,0.42760553599107887,0.4449483234945541,0.4713527110865977,0.498418421406347,0.4505904166272403,0.498418421406347,25
+ts_tcnn_st_joint,rule,0.35588269519387744,0.40661209657631836,0.4419268202572173,0.4883599557012694,0.5307269596804935,0.44470170548183524,0.5307269596804935,25
+ts_tcnn_ry_zs,rule_smooth,0.4356709987706367,0.43541535327114067,0.43491208631609046,0.43457886468553114,0.4332604325875319,0.43476754712618615,0.4356709987706367,25
+ts_tcnn_ry_zs,rule,0.4224831085879578,0.4221502039790731,0.4217075655327361,0.4211294633787168,0.4194749256558641,0.4213890534268696,0.4224831085879578,25
+ts_tcnn_ry_zs,argmax,0.3954616854154896,0.395222238646323,0.39479795962014225,0.39446712691515406,0.3933017560040373,0.39465015332022924,0.3954616854154896,25
+ts_tcnn_st_joint,argmax,0.28879417527833806,0.350327674638756,0.3877930910159043,0.4404836593936182,0.4799127842032114,0.3894622769059656,0.4799127842032114,25
+ts_tcnn_st_zs,rule_smooth,0.3275516289410064,0.3274124695453508,0.3269001928719971,0.32639749891184106,0.32524822820945043,0.32670200369592917,0.3275516289410064,25
+ts_tcnn_st_zs,rule,0.31799012531408366,0.3177342112382657,0.31738635463963977,0.31652906430204986,0.3153308551701609,0.31699412213283995,0.31799012531408366,25
+ts_tcnn_st_zs,argmax,0.2481830113228612,0.24798912112431828,0.24770624231570718,0.24703456849781072,0.24603382476259877,0.24738935360465925,0.2481830113228612,25

reports/nnlab/leaderboard_A.csv

@@ -0,0 +1,343 @@
+method,decision,n5,n25,n50,n100,n200,avg,peak,draws
+ens_e6__lp,rule_smooth,0.7000094213160887,0.714257419564768,0.7295050471911657,0.7364591120131324,0.7414540785445525,0.7243370157259414,0.7414540785445525,25
+ens_e7__lp,rule_smooth,0.7005585510637877,0.7145889274677815,0.7293565014154655,0.7364653591205584,0.738468868671182,0.7238876415477551,0.738468868671182,25
+ens_e4__lp,rule_smooth,0.7033039816064857,0.7108593600007864,0.7290487145521857,0.7355249493849749,0.7392250465649489,0.7235924104218763,0.7392250465649489,25
+ens_e5__lp,rule_smooth,0.7004430967334736,0.7141452680498364,0.7289806631463603,0.7342003142096859,0.7378891605971858,0.7231317005473084,0.7378891605971858,25
+ens_e6__lapshot,rule_smooth,0.6959012747147905,0.7106633441264207,0.7304166688481676,0.7351367065047734,0.7421487287887574,0.7228533445965819,0.7421487287887574,25
+ens_e1__lp,rule_smooth,0.6969895772166317,0.7082793679553152,0.7301167056863663,0.7353938689921561,0.7388468246219259,0.721925268894479,0.7388468246219259,25
+ens_e7__lp,argmax,0.6896548069854656,0.7081321118278481,0.7298899064705013,0.7331448039058373,0.7429955923065629,0.720763444299243,0.7429955923065629,25
+ens_e7__lp,rule,0.6982159621301054,0.7104683402363965,0.7265574420832392,0.7310984491865018,0.7362527392705768,0.7205185865813639,0.7362527392705768,25
+ens_e6__lp,rule,0.6949265924504137,0.7106349480783065,0.7264818251978721,0.731206811266408,0.739045086156176,0.7204590526298352,0.739045086156176,25
+ens_e5__lapshot,rule_smooth,0.695556442435904,0.7101427532749163,0.7289852704186038,0.7310398129396766,0.7359010539107642,0.7203250665959728,0.7359010539107642,25
+ens_e6,rule_smooth,0.6946740391443391,0.7072540560334086,0.7266137394421659,0.7324070175579998,0.7383949695017785,0.7198687643359384,0.7383949695017785,25
+ens_e6__lp,argmax,0.6880495862931459,0.7047351871443502,0.7302228441945481,0.7321531960117572,0.744104866683043,0.7198531360653689,0.744104866683043,25
+ens_e5__lp,rule,0.6947131059757111,0.7100662958258839,0.7249220802166435,0.7290823675962386,0.7349609502651424,0.7187489599759239,0.7349609502651424,25
+ens_e4__lp,rule,0.6971862664320629,0.7063810132259426,0.7247250785028888,0.7292681335134542,0.7355113890994557,0.7186143761547609,0.7355113890994557,25
+ens_e5__lp,argmax,0.6877375794234745,0.7057612488740043,0.7289829403502331,0.7298355458171468,0.7404095409912057,0.7185453710912129,0.7404095409912057,25
+ens_e4__lapshot,rule_smooth,0.6946340377420817,0.7023216590205981,0.7262732436097366,0.7298351905226979,0.7361838260674002,0.7178495913925029,0.7361838260674002,25
+ens_e2__lapshot,rule_smooth,0.6856176011342144,0.6951505780707102,0.7172384852409893,0.7375674808366128,0.7524349982701544,0.7176018287105362,0.7524349982701544,25
+ens_e4__lp,argmax,0.6896553193247508,0.6985427139443438,0.7276855100667698,0.7302212823078198,0.7410628308704599,0.7174335313028288,0.7410628308704599,25
+ens_e5,rule_smooth,0.6941080595118108,0.7061068022233439,0.7250495128851561,0.7282443256875395,0.7328381407261938,0.7172693682068089,0.7328381407261938,25
+ens_e1__lp,rule,0.6919662539534132,0.7039513632158775,0.7251202250000386,0.7294683820649096,0.7355749404997034,0.7172162329467884,0.7355749404997034,25
+ens_e1__lp__st,rule_smooth,0.6774928084946754,0.7025764550070596,0.7220258803745111,0.7350713323034261,0.7471841467894881,0.716870124593832,0.7471841467894881,25
+ens_e1__lapshot,rule_smooth,0.6890483507558056,0.6995581982675407,0.7260318216506766,0.73219876559073,0.737167653356434,0.7168009579242375,0.737167653356434,25
+ens_e1__lp,argmax,0.685754087718647,0.6949847483238182,0.7291393165917988,0.7303884572925966,0.7422362949770669,0.7165005809807854,0.7422362949770669,25
+ens_e7,rule_smooth,0.6949486228228434,0.7038110672078963,0.7224477796717566,0.7284176337103031,0.7319903921949196,0.7163230991215439,0.7319903921949196,25
+ens_e6__lapshot,argmax,0.6830169021044034,0.6998621795016537,0.7269663587458934,0.7301853200411059,0.7406886863302522,0.7161438893446618,0.7406886863302522,25
+ens_e1__clamp,argmax,0.6845679345171878,0.6934624288716336,0.7271136792721298,0.733482114181205,0.7411269918412751,0.7159506297366862,0.7411269918412751,25
+ens_e3__lp,rule_smooth,0.6705987846873092,0.701548097675427,0.7299353536252109,0.735556088052572,0.7390081996796019,0.7153293047440241,0.7390081996796019,25
+ens_e6__lapshot,rule,0.684650989861695,0.7007833947628672,0.7243097485170588,0.7279746339580904,0.7372624360200366,0.7149962406239496,0.7372624360200366,25
+ens_e5__lapshot,argmax,0.6839770293422103,0.699917178089191,0.7255690304213316,0.7259097020338908,0.7341312335466583,0.7139008346866564,0.7341312335466583,25
+ens_e4,rule_smooth,0.6911219174473178,0.6977924714236657,0.7208839879466173,0.7272259684658224,0.732342401662512,0.713873349389187,0.732342401662512,25
+ens_e1__clamp,rule,0.6860155564018738,0.6957731148596664,0.7221212458494485,0.7305420901768324,0.7347963996724848,0.7138496813920612,0.7347963996724848,25
+ens_e1,rule_smooth,0.6859606695910152,0.6956996256553384,0.7218758831496864,0.7298306752888519,0.7335486475505109,0.7133831002470806,0.7335486475505109,25
+ens_e2__clamp,rule,0.6835888434226478,0.6894937347924492,0.7115649929094892,0.7333813983493023,0.747898586696747,0.7131855112341271,0.747898586696747,25
+ens_e2,rule_smooth,0.6835090995660785,0.6890729266906662,0.711002971948167,0.7326760668326259,0.7466581247636983,0.7125838379602472,0.7466581247636983,25
+ens_e2__lp,argmax,0.6666236390160135,0.6869005574081994,0.7201005726245205,0.735727635702108,0.752291716322748,0.7123288242147179,0.752291716322748,25
+ens_e5__lapshot,rule,0.6845272043534703,0.7004235750600912,0.7219978579576398,0.7234844785221959,0.7311458287642995,0.7123157889315394,0.7311458287642995,25
+ens_e2__lp,rule_smooth,0.6358088762446739,0.7082983514228764,0.7250126549111033,0.739686607043359,0.7487888294660844,0.7115190638176194,0.7487888294660844,25
+ens_e1__lp__st,argmax,0.6689884661782967,0.6935540766848695,0.7181516095600063,0.728586628304781,0.7470128008361303,0.7112587163128168,0.7470128008361303,25
+ens_e6,argmax,0.6762636804239971,0.6925263602914249,0.7216268230469597,0.726631250441156,0.7360070242542537,0.7106110276915583,0.7360070242542537,25
+ens_e2__clamp,argmax,0.6686396628103888,0.6801541432949572,0.7111252931464439,0.7351934857333117,0.7556077678182865,0.7101440705606776,0.7556077678182865,25
+ens_e2__lp,rule,0.6365990576258271,0.7029991257363399,0.7240000152721336,0.7365906622725407,0.748396490142973,0.7097170702099629,0.748396490142973,25
+ens_e3__lp,rule,0.6620391295539727,0.6965282939920802,0.7238689549781215,0.7288175859040019,0.7358421122974318,0.7094192153451215,0.7358421122974318,25
+ens_e3__lp,argmax,0.6588465741571301,0.6862890548048355,0.7277001868886128,0.7304145623492079,0.7424771401433257,0.7091455036686224,0.7424771401433257,25
+ens_e6,rule,0.6785156732096473,0.6942167288345201,0.717277114168683,0.7238082684795228,0.7310139660078874,0.7089663501400522,0.7310139660078874,25
+ens_e1__lp__st,rule,0.669075514005492,0.6941509379386792,0.714443283433247,0.7259710325420281,0.7409484819079172,0.7089178499654727,0.7409484819079172,25
+ens_e5,argmax,0.6774040848472367,0.6929676152471284,0.7203963513984583,0.7223053898056279,0.7296868274665164,0.7085520537529935,0.7296868274665164,25
+ens_e4__lapshot,argmax,0.6792046260193118,0.6880649462303665,0.7193485565234715,0.7230530794782888,0.7323893374406265,0.7084121091384129,0.7323893374406265,25
+ens_e4__lapshot,rule,0.6813216140300155,0.6906035693212066,0.7168531716519159,0.7215329628656766,0.7296088837881683,0.7079840403313965,0.7296088837881683,25
+ens_e1__clamp,rule_smooth,0.6825280967782803,0.6921550580095917,0.7162740646634091,0.7236663445366374,0.7249187260186561,0.707908458001315,0.7249187260186561,25
+ens_e1__lapshot,argmax,0.6743000817115121,0.6859793615367111,0.7193917109966501,0.7244101434395169,0.734294192970377,0.7076750981309534,0.734294192970377,25
+ens_e1__lapshot,rule,0.6761596645168357,0.6881130411163883,0.7165588145771822,0.7231768244346494,0.7302622881807577,0.7068541265651627,0.7302622881807577,25
+ens_e3__lapshot,rule_smooth,0.6512239741940777,0.6908200853934217,0.723119593665859,0.7316721516351311,0.7371842779776826,0.7068040165732344,0.7371842779776826,25
+ens_e5,rule,0.6787567654892881,0.6943066283572801,0.7156608349668983,0.7189014699707994,0.7248042244238067,0.7064859846416145,0.7248042244238067,25
+ens_e2__lapshot,rule,0.6669826207292273,0.6824162533559069,0.7083908404825079,0.7275676864535433,0.7459520502341105,0.7062618902510592,0.7459520502341105,25
+ens_e2__clamp,rule_smooth,0.6790880463212131,0.6852321719499253,0.7039911895325457,0.7247842168349502,0.736899110652211,0.705998947058169,0.736899110652211,25
+ens_e7,argmax,0.6710403867741735,0.6882122509854762,0.7150249137633014,0.7222206875461274,0.7294076928506719,0.7051811863839502,0.7294076928506719,25
+ens_e3__clamp,argmax,0.6429163762150873,0.6834815783879662,0.7237649421395529,0.7323753472874286,0.7408810037180504,0.7046838495496172,0.7408810037180504,25
+ens_e2__lapshot,argmax,0.6586719249611513,0.6741497292272004,0.7076702667561557,0.7288294683425801,0.7501682566666245,0.7038979291907423,0.7501682566666245,25
+ens_e7,rule,0.6743963033218162,0.688130926317844,0.7114379551300624,0.7185337740086801,0.7250141298617724,0.703502617728035,0.7250141298617724,25
+ens_e3__clamp,rule,0.6446917305926224,0.6868488080111018,0.7187537763281533,0.7296415151575806,0.7345615940991236,0.7028994848377164,0.7345615940991236,25
+ens_e3,rule_smooth,0.6447139795551848,0.6865624411255119,0.7182853302196559,0.7290189856689452,0.7332766578937615,0.7023714788926119,0.7332766578937615,25
+ens_e4,argmax,0.6681755656513744,0.6806595191410065,0.712173173156087,0.7179717139783257,0.7264564004557629,0.7010872744765113,0.7264564004557629,25
+lgbm_upw__lapshot,rule_smooth,0.6238198794057216,0.679443317599238,0.71173993712909,0.7366283022251583,0.752378929728982,0.700802073217638,0.752378929728982,25
+ens_e1,argmax,0.6632348497913279,0.6783557873898037,0.7118022970410399,0.7192778651031582,0.7290091017793421,0.7003359802209342,0.7290091017793421,25
+ens_e4,rule,0.670049384756816,0.6821630088548558,0.7081244936014546,0.7147969383010454,0.7226088676425217,0.6995485386313387,0.7226088676425217,25
+ens_e3__clamp,rule_smooth,0.6500560498970873,0.6838510534041372,0.7130722394617283,0.7237675339802168,0.725266047994643,0.6992025849475625,0.725266047994643,25
+ens_e1,rule,0.6655892596196459,0.6794837518601687,0.7088825110362388,0.7174570458427156,0.7244322273406596,0.6991689591398856,0.7244322273406596,25
+ens_e3__lapshot,argmax,0.6324001131979846,0.6747558876183957,0.7162051157770979,0.7239269393034327,0.7342373101881211,0.6963050732170065,0.7342373101881211,25
+ens_e3__lapshot,rule,0.633470297053441,0.6765212802941085,0.7127558827425522,0.7221474361714536,0.7297549063945571,0.6949299605312225,0.7297549063945571,25
+lgbm_upw__lp,argmax,0.5952888198196117,0.6718568698357403,0.7182240536695205,0.7358440086715881,0.7527163067820243,0.694786011755697,0.7527163067820243,25
+ens_e2,rule,0.6551770837907271,0.6680989015378825,0.6944728051330801,0.7175153639846046,0.736393936068781,0.694331618103015,0.736393936068781,25
+lgbm_upw__clamp,rule,0.6139281470322056,0.67235399143661,0.7048364650938714,0.731805696839426,0.7477744904054024,0.694139758161503,0.7477744904054024,25
+lgbm_upw,rule_smooth,0.6137788538826262,0.6721584965633471,0.7045784998108477,0.7312431742689887,0.746780415864728,0.6937078880781075,0.746780415864728,25
+ens_e2,argmax,0.6440798670584078,0.6612458089983257,0.6947773276199495,0.7188730355775739,0.7415049628694093,0.6920962004247333,0.7415049628694093,25
+lgbm_upw__lp,rule_smooth,0.5490333658415227,0.6979082743344268,0.7221792800274099,0.7402318344615166,0.7488583335518053,0.6916422176433363,0.7488583335518053,25
+lgbm_upw__lp,rule,0.5538677396408792,0.6942491012468898,0.7217859414549581,0.7366946851322478,0.7488380469274493,0.6910871028804848,0.7488380469274493,25
+lgbm_upw__clamp,rule_smooth,0.6210508849941293,0.6714731201302746,0.6994974693615281,0.7246095204281792,0.7368567383735092,0.690697546657524,0.7368567383735092,25
+lgbm_upw__clamp,argmax,0.5888556812854271,0.6604151745148658,0.7049674610178274,0.7340710258139037,0.7554140920684875,0.6887446869401023,0.7554140920684875,25
+ens_e3,argmax,0.6207395046777547,0.665880703383615,0.708592343081843,0.7177461540735097,0.728511558676085,0.6882940527785616,0.728511558676085,25
+lgbm_upw__lapshot,rule,0.6000545083025298,0.6625836532492244,0.7011557222571861,0.7258332251138174,0.7457720943404681,0.6870798406526452,0.7457720943404681,25
+ens_e3,rule,0.6218469189426604,0.6671297468559787,0.705023957816542,0.715564071238535,0.7240292391879428,0.6867187868083319,0.7240292391879428,25
+zs_xgb_dann__lp,rule,0.6606104392929166,0.6736422200745483,0.6829481967674988,0.6979332256134055,0.7095284029281979,0.6849324969353134,0.7095284029281979,25
+met_cos_maha,rule_smooth,0.6169669252453689,0.6693325085324647,0.7061729247262489,0.71261984147787,0.7159744592101864,0.6842133318384278,0.7159744592101864,25
+lgbm_upw__lapshot,argmax,0.5813676363443454,0.6545591431484432,0.7002573569518991,0.7275453195665053,0.7501718962068691,0.6827802704436123,0.7501718962068691,25
+lgbm_upw__em,rule_smooth,0.6037417546084711,0.6493619272650986,0.6927125850549611,0.7211872971911528,0.7461484912714933,0.6826304110782354,0.7461484912714933,25
+met_protoda_maha,rule_smooth,0.6311943532085137,0.6686880039379184,0.6997017788605275,0.7056683702976707,0.7072226529357245,0.6824950318480709,0.7072226529357245,25
+tab_res_o60_ft,rule_smooth,0.6420647709633138,0.671693553365275,0.685828984110598,0.6986878952682265,0.7111470096408637,0.6818844426696554,0.7111470096408637,25
+tab_dann_o60_ft,rule_smooth,0.6598074203388509,0.6734499025764158,0.683247015019727,0.687561842193181,0.6971029301598999,0.680233822057615,0.6971029301598999,25
+cls_lda,rule_smooth,0.642938004332449,0.6646365390121373,0.681745633913861,0.6970715820578829,0.705872490393778,0.6784528499420216,0.705872490393778,25
+zs_xgb_dann__lp,argmax,0.6614227532799436,0.6656570058943816,0.6732762730262449,0.6854795032845086,0.7029216422938683,0.6777514355557894,0.7029216422938683,25
+zs_xgb_dann__clamp,rule_smooth,0.6748682666310118,0.6750977978434388,0.6751496929332277,0.6768934610760198,0.6772704406477971,0.675855931826299,0.6772704406477971,25
+zs_dann__lp,rule_smooth,0.6623736564897871,0.6666503648812138,0.6709076023521992,0.6826127314340206,0.6940248000320421,0.6753138310378526,0.6940248000320421,25
+zs_xgb_dann__lapshot,rule_smooth,0.6747377523851137,0.6745026325310827,0.6746892823068297,0.6749008705710483,0.6747927382801976,0.6747246552148544,0.6749008705710483,25
+met_proto_maha,rule_smooth,0.6288985508037701,0.6615252465467882,0.6880729275757017,0.6939017166705934,0.6998493331038529,0.6744495549401412,0.6998493331038529,25
+zs_xgb_dann__lp,rule_smooth,0.6528762139069049,0.6546909840051445,0.6581119540457137,0.6847767632243382,0.7180860859306767,0.6737084002225556,0.7180860859306767,25
+lgbm_upw,rule,0.5809007580747197,0.6460083091721153,0.6865307991350221,0.7154981458353632,0.7363567350001791,0.6730589494434799,0.7363567350001791,25
+zs_xgb_dann__clamp,rule,0.6702281259633962,0.6704582460514315,0.6707220378429627,0.6726903987516188,0.6741075434845104,0.671641270418784,0.6741075434845104,25
+cls_lr,rule_smooth,0.5943768799597489,0.6366344048924286,0.6885054815248385,0.7095984243462986,0.719659011294566,0.669754840403576,0.719659011294566,25
+zs_xgb_dann,rule_smooth,0.6700842908171334,0.6695736232717798,0.6691391753583119,0.6689795927842144,0.6674866273912742,0.6690526619245427,0.6700842908171334,25
+lgbm_upw,argmax,0.5627676084190295,0.6380123172795462,0.6855188576839635,0.7167639011369908,0.7412595451848811,0.6688644459408823,0.7412595451848811,25
+zs_dann__lp,rule,0.654849444746455,0.6587796188534147,0.6642571451603188,0.6742865925186947,0.6873888187610726,0.6679123240079912,0.6873888187610726,25
+tab_dann_rz_ft,rule_smooth,0.6386172776466095,0.6614166735942764,0.674421558368827,0.6773022669342972,0.6861097452227899,0.66757350435336,0.6861097452227899,25
+met_cos_maha,argmax,0.5895730373670105,0.6471818874618271,0.6933873614992885,0.7011524160452024,0.7063730084558667,0.667533542165839,0.7063730084558667,25
+met_cos_maha,rule,0.5885174477621913,0.6466266352942087,0.6883783919851865,0.6978800154301507,0.7015934114097547,0.6645991803762984,0.7015934114097547,25
+tab_res_rz_ft,rule_smooth,0.5833190664816568,0.6425869873051633,0.6840941502772041,0.7018446453311908,0.7098451024474925,0.6643379903685416,0.7098451024474925,25
+lgbm_upw__em,rule,0.573182939852915,0.6301760984448883,0.6751486285180142,0.7054220500014358,0.7339348487223075,0.6635729131079122,0.7339348487223075,25
+met_protoda_maha,argmax,0.605511414329188,0.6475364087007616,0.6799004517537639,0.6903069974769711,0.6933715583569204,0.663325366123521,0.6933715583569204,25
+met_cos_knn,rule_smooth,0.6046700941424457,0.6490498200472579,0.6749396120494467,0.6831125617678152,0.6999697115325446,0.6623483599079021,0.6999697115325446,25
+met_protoda_maha,rule,0.6064277632851909,0.6469596472361632,0.6774964714573547,0.6877284344079507,0.6913430110447742,0.6619910654862868,0.6913430110447742,25
+zs_xgb_dann__clamp,argmax,0.6592597633957767,0.6597715896232861,0.6601671630287013,0.6625629880157646,0.6647659236383808,0.6613054855403819,0.6647659236383808,25
+cls_lda,rule,0.6161335733587663,0.6395359099633409,0.6606917922380602,0.6868779850155213,0.7015667611300481,0.6609612043411472,0.7015667611300481,25
+met_protoda_ftproj,rule_smooth,0.6069896619612571,0.6393764389412445,0.6702364493408034,0.6866200978486555,0.6904248897633533,0.6587295075710627,0.6904248897633533,25
+tab_dann_rg_ft,rule_smooth,0.6333168802092154,0.6410816512999651,0.6617733287916723,0.676847319737346,0.6774776225968786,0.6580993605270155,0.6774776225968786,25
+zs_xgb_dann__lapshot,rule,0.6564792850547823,0.6566341544412856,0.6567700439423974,0.6569153632624491,0.6569621333264635,0.6567521960054756,0.6569621333264635,25
+cls_nys_lr,rule_smooth,0.5931346366111283,0.6249405225162723,0.6660520384964668,0.6886312533610792,0.707269403539611,0.6560055709049115,0.707269403539611,25
+zs_dann__lp,argmax,0.639031747540614,0.6440983768704049,0.6504705317749251,0.6650160505309133,0.6813053332981769,0.6559844080030068,0.6813053332981769,25
+met_supcon_maha,rule_smooth,0.6031022019762211,0.638622568675486,0.6706555415673279,0.6820888894119188,0.6836778136289021,0.6556294030519713,0.6836778136289021,25
+met_proto_maha,rule,0.6049162194697181,0.6399776359364207,0.6685185985519368,0.6754689303301843,0.6835291656163683,0.6544821099809256,0.6835291656163683,25
+met_proto_maha,argmax,0.6030393171897808,0.6373648582453143,0.6696244395028439,0.6763313003805305,0.6854412861663125,0.6543602402969564,0.6854412861663125,25
+met_protoda_ftproj,argmax,0.6020787520993526,0.634435346727796,0.6652762691918479,0.6820015786816056,0.6871479980124517,0.6541879889426108,0.6871479980124517,25
+met_protoda_ftproj,rule,0.6016621057727708,0.6343711884372045,0.6646716100371493,0.6818970575548432,0.686031258917436,0.6537266441438808,0.686031258917436,25
+met_cos_ftproj,rule_smooth,0.5736816298274501,0.6287457049418876,0.679247382699028,0.6902274615924351,0.6933461529703886,0.6530496664062377,0.6933461529703886,25
+met_protoda_knn,rule_smooth,0.604734280933839,0.6336915320859653,0.657735582769737,0.6766012772880173,0.6924657828182169,0.6530456911791551,0.6924657828182169,25
+met_supcon_ftproj,rule_smooth,0.5861799233885033,0.6326470028268016,0.66703450527654,0.6853624479214518,0.6896931102696152,0.6521833979365824,0.6896931102696152,25
+ts_tcnn_dann_st_ft,rule_smooth,0.5499121267631775,0.643239537021209,0.6656048936429139,0.6943837444878479,0.7020106591061446,0.6510301922042586,0.7020106591061446,25
+cls_lr_feda,rule_smooth,0.5720530999671608,0.6341701460330134,0.6708353163438238,0.6814234953847154,0.6964057874198387,0.6509775690297104,0.6964057874198387,25
+zs_xgb_dann__lapshot,argmax,0.6499378568792997,0.6501929888541859,0.6504648488574813,0.6505749640898771,0.6515738573695067,0.6505489032100702,0.6515738573695067,25
+cls_svm_feda,rule_smooth,0.5744870245381217,0.6280327017509892,0.6705660812522007,0.6797965893453842,0.6983966258880286,0.6502558045549449,0.6983966258880286,25
+met_proto_ftproj,rule_smooth,0.5850985277804367,0.6300306170645826,0.664438431237395,0.6810732326768796,0.6843261339858755,0.6489933885490338,0.6843261339858755,25
+tab_res_o60_proto,rule_smooth,0.6285311304677953,0.6412293541576155,0.657274412217468,0.6585721309991813,0.6587583410320395,0.6488730737748198,0.6587583410320395,25
+cls_lr,rule,0.569089647776282,0.616115858619621,0.6651324407930003,0.688766749638446,0.7013388919455286,0.6480887177545757,0.7013388919455286,25
+met_cos_ftproj,argmax,0.5675376863409142,0.6225220744857451,0.6723188879401564,0.6842808501129036,0.6891044405781815,0.64715278789158,0.6891044405781815,25
+met_supcon_ftproj,argmax,0.5802002324179086,0.6270347016358322,0.6613927560059072,0.6794949801732287,0.685608540160025,0.6467462420785803,0.685608540160025,25
+met_cos_ftproj,rule,0.5670413598474505,0.6223963637177992,0.6717244725022923,0.6835139128325901,0.6881212552742031,0.6465594728348669,0.6881212552742031,25
+met_supcon_ftproj,rule,0.5799888866413246,0.6270122049076439,0.6608466460947617,0.6789666368906422,0.6853288109787183,0.6464286371026182,0.6853288109787183,25
+cls_svm_feda,argmax,0.5659897131218354,0.6223569696709796,0.6677060858317752,0.6766640587887361,0.6936526038396468,0.6452738862505947,0.6936526038396468,25
+tab_res_o60_ft,rule,0.5979553322243606,0.6334086288004325,0.6484744781234506,0.6655296975410664,0.6791896577306407,0.6449115588839901,0.6791896577306407,25
+tab_mlp_rg_ft,rule_smooth,0.6004990169602425,0.6237686610885673,0.649888738612938,0.6706705682796988,0.677671946985937,0.6444997863854767,0.677671946985937,25
+tab_dann_o60_ft,rule,0.6189483534276465,0.6336847035601284,0.6475530711341492,0.6552592839775807,0.6656112786147592,0.6442113381428527,0.6656112786147592,25
+met_proto_ftproj,argmax,0.5789295086204714,0.6244268436468223,0.6586647539676417,0.6757435568834497,0.6800488945155744,0.6435627115267919,0.6800488945155744,25
+cls_svm_feda,rule,0.5670907398612159,0.622656526745969,0.6654417197186968,0.6732458517131131,0.6892797615990822,0.6435429199276154,0.6892797615990822,25
+tab_res_o60_ft,argmax,0.5946418995708085,0.6256477430667211,0.646760749035517,0.6669584374608543,0.6833649023797511,0.6434747463027304,0.6833649023797511,25
+met_proto_ftproj,rule,0.5787844559738138,0.6242884219953337,0.6580139833004297,0.67544652596437,0.6797031359182133,0.6432473046304322,0.6797031359182133,25
+tab_dann_o60_ft,argmax,0.6184930037446118,0.6254819893748584,0.6432909379949321,0.6552844878760262,0.6676095043285455,0.6420319846637949,0.6676095043285455,25
+met_proto_knn,rule_smooth,0.5889611764114875,0.6293300502070625,0.646894856016229,0.6619679237408673,0.6827787576133275,0.6419865527977947,0.6827787576133275,25
+tab_dann_rz_ft,rule,0.6098840925567837,0.6329414329327718,0.6456946006187089,0.6505957800896212,0.6621296002236059,0.6402491012842983,0.6621296002236059,25
+met_cos_knn,rule,0.5772920162577007,0.6261181142388168,0.6496431624567005,0.6619387002997006,0.6827954240784002,0.6395574834662637,0.6827954240784002,25
+tab_res_rz_ft,argmax,0.5500174337106085,0.6103131328629232,0.660103081313863,0.680123382358734,0.6947250582936588,0.6390564177079575,0.6947250582936588,25
+tab_dann_rz_ft,argmax,0.6048713400848463,0.6248629190361876,0.6452398754193185,0.6524828050717184,0.6629368426190497,0.6380787564462241,0.6629368426190497,25
+zs_xgb_dann__em,rule_smooth,0.6387458562467911,0.6386382343809178,0.6380973736519416,0.6380888708439887,0.6366323046002027,0.6380405279447684,0.6387458562467911,25
+cls_svm,rule_smooth,0.5499451503763111,0.6066298386767208,0.6531324760021862,0.677309024664319,0.6984191669127001,0.6370871313264475,0.6984191669127001,25
+zs_lgbm__clamp,rule_smooth,0.6348981593893566,0.6354471252144946,0.6359794712602223,0.6373997536135512,0.6398482613664052,0.636714554168806,0.6398482613664052,25
+tab_res_rz_ft,rule,0.5515182305986819,0.6126322283485511,0.6531033838100411,0.6763983699146807,0.689060845850069,0.6365426117044047,0.689060845850069,25
+zs_xgb_dann,rule,0.6372881287744528,0.6370903543935894,0.6364364887609785,0.63581280631347,0.6343608811664969,0.6361977318817976,0.6372881287744528,25
+tab_dann_rz_proto,rule_smooth,0.616690092359247,0.6245779174452254,0.6439576833897345,0.6455468697725995,0.6472058599895526,0.6355956845912718,0.6472058599895526,25
+tab_dann_o60_joint,rule_smooth,0.6199492513690281,0.6272710778522603,0.6332078392883383,0.6402239860296544,0.6564556626356791,0.635421563434992,0.6564556626356791,25
+met_supcon_maha,rule,0.5786920752553291,0.6196531325834196,0.6471410981601304,0.6636546114598357,0.667337743076333,0.6352957321070095,0.667337743076333,25
+zs_lgbm__lapshot,rule_smooth,0.6340987142092878,0.6342308950083766,0.6343866841112231,0.6353333011207413,0.6358406989979845,0.6347780586895226,0.6358406989979845,25
+zs_dann__clamp,rule_smooth,0.6333358675518412,0.6336706275260138,0.6335736196482438,0.6349132097657596,0.6355975068308852,0.6342181662645487,0.6355975068308852,25
+tab_dann_rg_ft,rule,0.607149336411237,0.619951309714633,0.6362319841065716,0.6500754901807911,0.6554341103693768,0.633768446156522,0.6554341103693768,25
+tab_dann_o60_proto,rule_smooth,0.626375872530979,0.6228261734913919,0.6351805520733245,0.639999769448038,0.641701426467498,0.6332167588022463,0.641701426467498,25
+tab_mlp_o60_ft,rule_smooth,0.5807778819772584,0.6189516931454375,0.6358434561473345,0.6547798138403575,0.6715804799461103,0.6323866650112997,0.6715804799461103,25
+cls_nys_lr,rule,0.5629641496743514,0.6000512242757271,0.6427949211829074,0.6666095001310284,0.6892814108482125,0.6323402412224454,0.6892814108482125,25
+cls_knn,rule_smooth,0.5103981879598429,0.625538229365174,0.6504844114023417,0.6731646295957605,0.6996265761892226,0.6318424069024683,0.6996265761892226,25
+met_supcon_maha,argmax,0.5699865724037098,0.6149892381074101,0.6440593706305184,0.6618304591621532,0.6683142097137185,0.631835970003502,0.6683142097137185,25
+lgbm_upw__em,argmax,0.5408656915583954,0.5618706228091063,0.6441638417633568,0.6841572037236329,0.7270082211851381,0.6316131162079259,0.7270082211851381,25
+zs_dann__lapshot,rule_smooth,0.6316481135700778,0.6317817283362748,0.6313873959809804,0.631993152483674,0.6312357847592179,0.6316092350260449,0.631993152483674,25
+zs_xgb_dann,argmax,0.6318693615791402,0.6317939578806993,0.6313364185359351,0.6309173227708127,0.6298273451219714,0.6311488811777117,0.6318693615791402,25
+cls_svm,argmax,0.5402176208268246,0.6005903247938226,0.648160547075947,0.6714910772800874,0.6933521165379397,0.6307623373029243,0.6933521165379397,25
+tab_res_o60_proto,argmax,0.6104940650082286,0.6230170764787084,0.6382590832071589,0.6403610266713444,0.6404898185704702,0.6305242139871821,0.6404898185704702,25
+met_protoda_knn,rule,0.5880696931104629,0.6079397716744285,0.6327699841406242,0.651867119325589,0.6716833664721322,0.6304659869446473,0.6716833664721322,25
+tab_res_o60_proto,rule,0.6102328067920855,0.6235008685495353,0.6374636487364705,0.6395879633972036,0.6396725035793595,0.6300915582109309,0.6396725035793595,25
+met_cos_knn,argmax,0.5806295783879261,0.5928493729175524,0.6378290215942686,0.6547152450565278,0.6817203362560469,0.6295487108424643,0.6817203362560469,25
+cls_lda,argmax,0.5524302421323284,0.594381624056762,0.6255389665191086,0.6720854106479097,0.7010519093969831,0.6290976305506183,0.7010519093969831,25
+cls_svm,rule,0.5415921960051819,0.5994115124787969,0.6456640227099641,0.6683995327925656,0.6895079156737183,0.6289150359320453,0.6895079156737183,25
+cls_lr,argmax,0.5548740299541388,0.5930701443404341,0.6402395772374844,0.6692148305390213,0.6840505781587457,0.6282898320459649,0.6840505781587457,25
+cls_lr_feda,argmax,0.5450781021509263,0.6105239953549156,0.6511175780228233,0.661486608998833,0.6707959174034837,0.6278004403861963,0.6707959174034837,25
+cls_lr_feda,rule,0.5469065856979803,0.611755249402741,0.6497241348087859,0.6596421539012585,0.6706062512044563,0.6277268750030444,0.6706062512044563,25
+zs_lgbm__clamp,rule,0.6256535210618893,0.6259814480362068,0.6265579118996183,0.6274419448771258,0.6297675340182995,0.6270804719786279,0.6297675340182995,25
+tab_mlp_rz_ft,rule_smooth,0.57424221270024,0.6047473363345125,0.6300241436460364,0.6517961914298825,0.6709366876501194,0.6263493143521581,0.6709366876501194,25
+tab_dann_rg_ft,argmax,0.5835783880907092,0.6075161999292602,0.6327379851798274,0.6497150223596964,0.6578886897412296,0.6262872570601445,0.6578886897412296,25
+zs_dann__clamp,rule,0.6243985737334673,0.6248711186373404,0.624518570974345,0.6261842037307142,0.6265328462191919,0.6253010626590119,0.6265328462191919,25
+zs_lgbm,rule_smooth,0.6256321524484935,0.6253067363876985,0.624992731217214,0.6243357446205673,0.6226050546358206,0.6245744838619588,0.6256321524484935,25
+tab_dann_rz_proto,argmax,0.6055235900834772,0.6133618320965551,0.6311038228275123,0.6347582371282562,0.6346529185469761,0.6238800801365555,0.6347582371282562,25
+tab_dann_rz_proto,rule,0.6055752176677001,0.6136286629188578,0.6308147348682316,0.6343789082922364,0.63425493224471,0.6237304911983472,0.6343789082922364,25
+zs_dann,rule_smooth,0.6242284580953879,0.6240578514985973,0.623479961495913,0.6231335165356315,0.6207186764553326,0.6231236928161724,0.6242284580953879,25
+tab_res_rz_proto,rule_smooth,0.5795347163159368,0.6127729245853833,0.6347614052414555,0.6394223455281316,0.6485323445082573,0.6230047472358329,0.6485323445082573,25
+tab_dann_rg_proto,rule_smooth,0.614853299857469,0.6172016656983276,0.6254184485437257,0.625172285404374,0.6262931501988376,0.6217877699405469,0.6262931501988376,25
+met_cos_tpn,rule_smooth,0.614135711175145,0.6045525413369799,0.6279313939274337,0.6270063197184537,0.6259888223034584,0.619922957692294,0.6279313939274337,25
+tab_dann_o60_proto,rule,0.612510925984829,0.6095395665071346,0.6214052669582869,0.6274297460548783,0.6283287824159973,0.6198428575842252,0.6283287824159973,25
+tab_dann_o60_proto,argmax,0.6131047222658881,0.6085454838074112,0.6209245288187146,0.6272680077917691,0.6282113918829529,0.6196108269133471,0.6282113918829529,25
+met_supcon_knn,rule_smooth,0.5696636230659001,0.5977371899710651,0.6207767050205539,0.6408861734352338,0.6681638965648344,0.6194455176115176,0.6681638965648344,25
+met_proto_knn,rule,0.5702412224741353,0.6021731411047162,0.6207706474359036,0.6354918908199302,0.659125586448942,0.6175604976567255,0.659125586448942,25
+met_protoda_knn,argmax,0.5838093256756676,0.5806566787871104,0.6144623550234922,0.6408225602553441,0.6647729256432077,0.6169047690769643,0.6647729256432077,25
+zs_xgb_dann__em,rule,0.6175618937288878,0.6174058304770794,0.6170917025002158,0.6166187989449347,0.6156158037479404,0.6168588058798117,0.6175618937288878,25
+ts_tcnn_dann_st_ft,rule,0.522323882814272,0.6072734353135426,0.6259534398061299,0.6585318299991024,0.6683394715340523,0.6164844118934198,0.6683394715340523,25
+met_cos_pn,rule_smooth,0.6046700941424457,0.6072274017730608,0.6213648171576761,0.6228720176989417,0.6240222288304038,0.6160313119205055,0.6240222288304038,25
+tab_dann_rg_joint,rule_smooth,0.6059106258194898,0.606661782322705,0.6121368213367364,0.6219134639818995,0.6328965588970691,0.6159038504715799,0.6328965588970691,25
+cls_nys_lr,argmax,0.5383136316214812,0.581827551991323,0.6252347731905077,0.6516312797404554,0.6771210546444689,0.6148256582376472,0.6771210546444689,25
+tab_res_o60_joint,rule_smooth,0.5565974249903688,0.5891457135959743,0.6173068944353342,0.644222484879893,0.6651452761803662,0.6144835588163874,0.6651452761803662,25
+zs_dann__em,rule_smooth,0.6153456531664834,0.6149525644910041,0.6144217354682597,0.6136522883385286,0.6123228016569712,0.6141390086242494,0.6153456531664834,25
+tab_dann_o60_zs,rule_smooth,0.6143116000752282,0.6143084465378761,0.6137998810379257,0.612573073306362,0.6110159546949518,0.6132017911304688,0.6143116000752282,25
+zs_lgbm__em,rule_smooth,0.613887282486391,0.613371173257228,0.6133387843593108,0.6126194262717105,0.6112757740202176,0.6128984880789716,0.613887282486391,25
+zs_dann__clamp,argmax,0.6114370738816194,0.6119988556616023,0.6116650220150135,0.6133468464687979,0.6137923071155041,0.6124480210285075,0.6137923071155041,25
+tab_mlp_rg_ft,rule,0.5704630269970015,0.5925520071974797,0.6125164206210008,0.6373402368414971,0.6470764851235492,0.6119896353561056,0.6470764851235492,25
+ts_tcnn_dann_st_ft,argmax,0.5078143130151954,0.6017296911534731,0.6247926500484847,0.6573699595071577,0.6676455250672853,0.6118704277583193,0.6676455250672853,25
+tab_dann_rg_proto,rule,0.6050447874774167,0.607182860853898,0.6155940561951276,0.6154355786176806,0.615165131067686,0.6116844828423618,0.6155940561951276,25
+tab_dann_rg_proto,argmax,0.6054719293991666,0.6063864667897996,0.6155197500670934,0.6147299155780237,0.6150734404452178,0.6114363004558603,0.6155197500670934,25
+zs_lgbm__lapshot,rule,0.6104344268085816,0.6101108443282598,0.6105303407955678,0.6109094845412338,0.6125418781701116,0.610905394928751,0.6125418781701116,25
+tab_dann_rz_joint,rule_smooth,0.5954506495375894,0.5996147006212171,0.6083306862865706,0.6159605712790921,0.630433683832217,0.6099580583113373,0.630433683832217,25
+tab_mlp_rg_ft,argmax,0.5638767861002116,0.5849246011931835,0.6121195414562447,0.6394379166803701,0.6491822781411377,0.6099082247142296,0.6491822781411377,25
+tab_dann_rz_zs,rule_smooth,0.6102894020968754,0.610059341481583,0.6100030335611597,0.6086423613260866,0.6074472650659765,0.6092882807063362,0.6102894020968754,25
+met_protoda_pn,rule_smooth,0.604734280933839,0.6013288659334171,0.6099024556409838,0.6108131227594185,0.6051430081920394,0.6063843466919395,0.6108131227594185,25
+tab_res_rz_proto,rule,0.561688406981476,0.5970442402875269,0.616462798932096,0.6221450721951098,0.6310332814470583,0.6056747599686533,0.6310332814470583,25
+met_proto_knn,argmax,0.5650146578478555,0.5750222835499504,0.6049719848213836,0.6247015113126266,0.6519264089709828,0.6043273693005599,0.6519264089709828,25
+tab_res_rz_proto,argmax,0.5618309026988583,0.5956115155691676,0.6156411200315193,0.6196942550709981,0.6287831643052884,0.6043121915351664,0.6287831643052884,25
+zs_dann__lapshot,rule,0.6038718327366265,0.6037692383749723,0.6039604390277862,0.6038035948602722,0.6032361406344757,0.6037282491268267,0.6039604390277862,25
+tab_dann_rg_zs,rule_smooth,0.6046740485232609,0.6044909142180496,0.6042953646321527,0.6036911014123303,0.6011821952024253,0.6036667247976438,0.6046740485232609,25
+cls_knn,rule,0.5534667304377219,0.571891759628534,0.6005195114235706,0.6279643380646405,0.6586095856589421,0.6024903850426818,0.6586095856589421,25
+tab_dann_o60_joint,rule,0.5839215001786149,0.589452228371087,0.5979337259104874,0.6028717084135339,0.6225249706339521,0.5993408267015351,0.6225249706339521,25
+tab_mlp_rg_proto,rule_smooth,0.5753759847165272,0.5931533011167782,0.6073335473156602,0.609618658821031,0.6093793449083414,0.5989721673756676,0.609618658821031,25
+cls_knn,argmax,0.5277194954227529,0.5615263288920308,0.6017620075070169,0.6348486489566869,0.6670723445571328,0.598585765067124,0.6670723445571328,25
+met_proto_pn,rule_smooth,0.5889611764114875,0.5920183499784588,0.604367374237906,0.6048224111244725,0.6005793089020403,0.5981497241308731,0.6048224111244725,25
+met_cos_tpn,argmax,0.5906639775305353,0.5812729856750651,0.6053147719786189,0.60649190600537,0.6065086123887272,0.5980504507156633,0.6065086123887272,25
+zs_dann__lapshot,argmax,0.5970405505148098,0.5973595276064101,0.5972311131226512,0.597393760162178,0.5975859124184721,0.5973221727649042,0.5975859124184721,25
+met_supcon_knn,rule,0.5521200955309793,0.5752221747276022,0.5969719838904732,0.6171318634739145,0.6446591477238369,0.5972210530693612,0.6446591477238369,25
+tab_mlp_rz_ft,rule,0.5454875555986013,0.5761105776475558,0.5979582818918523,0.6197135338846486,0.6449947262629945,0.5968529350571306,0.6449947262629945,25
+zs_lgbm,rule,0.5973085605223679,0.5972030723617756,0.5969227109417037,0.5961210688389498,0.5946409742293857,0.5964392773788366,0.5973085605223679,25
+met_cos_tpn,rule,0.5888734217948302,0.5788114288057811,0.6025446943804514,0.6033215170138888,0.603446109101419,0.5953994342192741,0.603446109101419,25
+met_cos_pn,argmax,0.5806295783879261,0.5844109689977118,0.6011794807016299,0.6010619906206351,0.6033244019556345,0.5941212841327076,0.6033244019556345,25
+met_cos_pn,rule,0.5772920162577007,0.5859591305447094,0.6001885002238062,0.6020897745863847,0.6025132776058271,0.5936085398436856,0.6025132776058271,25
+zs_lgbm__lp,argmax,0.5533538144417126,0.5636659629393156,0.5790384177537936,0.6110809338439875,0.6604690523394235,0.5935216362636465,0.6604690523394235,25
+met_protoda_pn,rule,0.5880696931104629,0.5876777107374849,0.5958970335051967,0.5979695675865411,0.5935334296134193,0.592629486910621,0.5979695675865411,25
+tab_mlp_rz_ft,argmax,0.5346257274083804,0.5697043155102878,0.5942609860505057,0.6185290355781596,0.6441967819478011,0.592263369299027,0.6441967819478011,25
+zs_lgbm__clamp,argmax,0.5885711019716845,0.5888564480223752,0.5896806103696736,0.5916145084855402,0.5943082992474703,0.5906061936193487,0.5943082992474703,25
+tab_mlp_o60_ft,rule,0.5536267426825706,0.5765902141911841,0.5855401233076458,0.606598637117764,0.6271749975239815,0.5899061429646292,0.6271749975239815,25
+met_protoda_tpn,rule_smooth,0.5931118519026827,0.5885401978717283,0.5888605465465824,0.5888203608899328,0.5864296893973737,0.58915252932166,0.5931118519026827,25
+tab_dann_rg_joint,rule,0.5805234478260315,0.5811576870052948,0.5859099505475329,0.5914342287728545,0.6027737984106324,0.5883598225124692,0.6027737984106324,25
+tab_dann_o60_joint,argmax,0.5682784538543465,0.5766284579250847,0.5866781238820697,0.5937846082669992,0.6119873836371925,0.5874714055131385,0.6119873836371925,25
+tab_mlp_rg_joint,rule_smooth,0.5601872593402332,0.5666760444998946,0.5831342080861855,0.5981120111921095,0.6288393454402234,0.5873897737117292,0.6288393454402234,25
+zs_dann,rule,0.5880856361144101,0.5879166406614093,0.5876955627847341,0.5868806746739715,0.5850353476566885,0.5871227723782427,0.5880856361144101,25
+tab_mlp_rz_proto,rule_smooth,0.5626704784714024,0.5793118531557744,0.5918244041573912,0.5957082559551391,0.5979986333354776,0.585502725015037,0.5979986333354776,25
+tab_dann_rz_zs,rule,0.5857880274157412,0.5855575381347656,0.5855138130995517,0.5849107010659356,0.5830945564062955,0.5849729272244579,0.5857880274157412,25
+met_protoda_pn,argmax,0.5838093256756676,0.5788872305044193,0.5871402951337262,0.588834958134848,0.5833098762445323,0.5843963371386386,0.588834958134848,25
+tab_dann_rg_zs,rule,0.5847513629891836,0.5845114987848442,0.5842860165915628,0.583632639813914,0.5818860525094249,0.583813514137786,0.5847513629891836,25
+tab_dann_rz_joint,rule,0.56953623498549,0.5734648710964123,0.5790638421459741,0.589614274372312,0.6025634639488455,0.5828485373098068,0.6025634639488455,25
+tab_mlp_o60_ft,argmax,0.5485626321272941,0.5687554487444432,0.5735059810135649,0.6003932419543452,0.6226608814412273,0.582775637056175,0.6226608814412273,25
+tab_mlp_rg_proto,rule,0.5598968929889878,0.5788345365780772,0.5903804634626522,0.5929527741664102,0.5914523590150407,0.5827034052422336,0.5929527741664102,25
+tab_dann_rg_joint,argmax,0.5756976309343637,0.5757806642603294,0.5807986591364228,0.5853197959852032,0.5948927302683032,0.5824978961169245,0.5948927302683032,25
+zs_lgbm__em,rule,0.5830671067542008,0.5830208962149912,0.5830658600947285,0.5820852009837381,0.5806455326142006,0.5823769193323718,0.5830671067542008,25
+zs_dann__em,rule,0.5828290648136438,0.5825396952994949,0.5820983712327903,0.5812230725491806,0.5800419891240325,0.5817464386038285,0.5828290648136438,25
+tab_res_rz_joint,rule_smooth,0.5158198220573014,0.5481611394743535,0.5828479074510942,0.6122960977416863,0.6472320236362814,0.5812713980721433,0.6472320236362814,25
+tab_dann_o60_zs,rule,0.5814475775949661,0.5814880523993795,0.5811046970510552,0.5804151685533304,0.5790301923330505,0.5806971375863563,0.5814880523993795,25
+met_proto_pn,rule,0.5702412224741353,0.5764313690401484,0.5860044505574991,0.5861050849891835,0.5834175938525311,0.5804399441826995,0.5861050849891835,25
+zs_lgbm__lapshot,argmax,0.5786245358323873,0.5789034111633649,0.5794357658825532,0.5801509390106169,0.58160035342155,0.5797430010620944,0.58160035342155,25
+tab_mlp_rg_proto,argmax,0.5595183833516801,0.5762499507500622,0.5866814966090136,0.5882495785717092,0.585003971804514,0.5791406762173958,0.5882495785717092,25
+tab_mlp_o60_proto,rule_smooth,0.5592998389860192,0.5710540481907089,0.5801413312362629,0.5903436855917942,0.5858240301180719,0.5773325868245713,0.5903436855917942,25
+zs_lgbm__lp,rule,0.5323247657810504,0.5384620174519049,0.5471961809964965,0.5677195441570803,0.7005066390272358,0.5772418294827536,0.7005066390272358,25
+tab_dann_rz_zs,argmax,0.5770675540542065,0.5768459323361841,0.5767464099716595,0.5762147489987852,0.5746719313717125,0.5763093153465096,0.5770675540542065,25
+met_protoda_tpn,rule,0.5774936662771812,0.5742502327775258,0.5751354826155516,0.5747556518266571,0.5728068525655583,0.5748883772124949,0.5774936662771812,25
+tab_dann_rz_joint,argmax,0.5599926976609015,0.5652207400901251,0.5720656927305441,0.5828553707296311,0.593674605153337,0.5747618212729078,0.593674605153337,25
+zs_dann,argmax,0.5756061113305703,0.5755268294831193,0.5752230914065912,0.5744417496821618,0.5729071536789802,0.5747409871162845,0.5756061113305703,25
+ts_tcnn_dann_st_joint,rule_smooth,0.46965540861797184,0.5489531990046661,0.5810883735797608,0.6169148707179779,0.6542126236218996,0.5741648951084553,0.6542126236218996,25
+met_proto_pn,argmax,0.5650146578478555,0.5680249447238527,0.5802737916162073,0.5798054389731176,0.5749501231386172,0.5736137912599301,0.5802737916162073,25
+met_proto_tpn,rule_smooth,0.5779413112678037,0.5728257600966801,0.5738896934736654,0.5722894646378678,0.5695398632010262,0.5732972185354086,0.5779413112678037,25
+tab_dann_rg_zs,argmax,0.5733713834905095,0.573227034323157,0.5729553995882275,0.5724153287138787,0.5708267223611839,0.5725591736953913,0.5733713834905095,25
+met_supcon_pn,rule_smooth,0.5696636230659001,0.5687995846224985,0.5745265951617815,0.573743707559889,0.5747522052986918,0.5722971431417522,0.5747522052986918,25
+tab_res_o60_joint,rule,0.5192390934062984,0.547259844010157,0.5712465524053384,0.5981520955046609,0.6230253961436668,0.5717845962940242,0.6230253961436668,25
+met_supcon_knn,argmax,0.5428132128371351,0.5338778817757355,0.5624673306292894,0.5917467294419106,0.6235051471324534,0.5708820603633049,0.6235051471324534,25
+met_protoda_tpn,argmax,0.5738877323871036,0.569517803437287,0.5700466951355339,0.5701840403792342,0.5676585358431223,0.5702589614364563,0.5738877323871036,25
+zs_lgbm,argmax,0.5692060173545372,0.569109392287499,0.568995379615763,0.5684720981806066,0.5673211899547748,0.5686208154786361,0.5692060173545372,25
+tab_mlp_o60_joint,rule_smooth,0.5210463894406905,0.5281348737741303,0.5678326030364339,0.5993872093764768,0.6257578306421994,0.5684317812539861,0.6257578306421994,25
+tab_mlp_rz_proto,rule,0.5470586928885941,0.5617533185987452,0.5729267138499796,0.5793982376349951,0.5804754032456743,0.5683224732435976,0.5804754032456743,25
+tab_mlp_rz_joint,rule_smooth,0.5323086976167589,0.5454885610455917,0.5621151559214368,0.5854448527400045,0.6139038262413699,0.5678522187130323,0.6139038262413699,25
+ts_tcnn_dann_st_joint,rule,0.5218515817684327,0.5423637194660051,0.5592088958533535,0.5892997341845675,0.6222352669122564,0.566991839636923,0.6222352669122564,25
+tab_dann_o60_zs,argmax,0.5653865594403267,0.5653162494609157,0.5649232163886467,0.5644248107851998,0.5629085018016109,0.56459186757534,0.5653865594403267,25
+tab_mlp_rz_proto,argmax,0.5461115384686256,0.5577879061851883,0.565714248868871,0.5718618646610337,0.5721395391585242,0.5627230194684486,0.5721395391585242,25
+tab_res_o60_joint,argmax,0.5036272309501284,0.534602934398826,0.5619375152682056,0.5923438775584918,0.6144992803979302,0.5614021677147164,0.6144992803979302,25
+tab_mlp_rg_zs,rule_smooth,0.5599126666144938,0.5595436623550404,0.5593087458462377,0.5583808530577462,0.5571861146547228,0.5588664085056483,0.5599126666144938,25
+met_proto_tpn,rule,0.5632152217618598,0.5580495021662062,0.5580087701881824,0.5579359516977271,0.5557936257901114,0.5586006143208173,0.5632152217618598,25
+tab_mlp_o60_proto,rule,0.5422639472012103,0.5506111103129528,0.5579973693416636,0.5695357169194754,0.5653466928325697,0.5571509673215743,0.5695357169194754,25
+met_supcon_pn,rule,0.5521200955309793,0.5544242275726556,0.5581750067178585,0.559308960422677,0.5583655071605931,0.5564787594809527,0.559308960422677,25
+met_proto_tpn,argmax,0.5583030540279919,0.5547483388541095,0.5558563632364465,0.5549108787787597,0.5526344984933691,0.5552906266781352,0.5583030540279919,25
+tab_mlp_rg_joint,rule,0.5287036113904253,0.5360696706948045,0.5501609667068988,0.5643856979084068,0.5948174126552018,0.5548274718711474,0.5948174126552018,25
+tab_mlp_o60_proto,argmax,0.5411002594908372,0.5485599761513381,0.5547268351752774,0.5673624873128309,0.5615867309962759,0.5546672578253119,0.5673624873128309,25
+met_supcon_tpn,rule_smooth,0.5549531225070026,0.5513759545696357,0.5528521269562754,0.552146774003367,0.5489707145419469,0.5520597385156455,0.5549531225070026,25
+tab_res_rz_joint,rule,0.4924442061009417,0.5192598625302612,0.5468633958230047,0.5751682939753798,0.611852841603574,0.5491177200066323,0.611852841603574,25
+ts_tcnn_dann_st_joint,argmax,0.5028941848359018,0.5289839232363402,0.5426662680678058,0.5654788109103228,0.5841383855302602,0.5448323145161262,0.5841383855302602,25
+tab_mlp_rg_joint,argmax,0.5141665788498795,0.5233734071155653,0.5393392467090236,0.5563277285024317,0.5860767169876085,0.5438567356329018,0.5860767169876085,25
+tab_mlp_rz_zs,rule_smooth,0.5445714769142747,0.5443464105452743,0.5442813757939376,0.542926863532046,0.5416060834036113,0.5435464420378288,0.5445714769142747,25
+cls_lr_zs,rule_smooth,0.5445485876645517,0.5442793085691509,0.5439996233174316,0.5432101320884932,0.5416752372494053,0.5435425777778066,0.5445485876645517,25
+tab_res_o60_zs,rule,0.5431467695281389,0.5430252161906917,0.5425681033548847,0.542198541770804,0.540416284000875,0.5422709829690788,0.5431467695281389,25
+tab_res_rz_joint,argmax,0.47872218040949815,0.5111888488605171,0.5409002825291489,0.5709464919479189,0.6058170856636768,0.541514977882152,0.6058170856636768,25
+met_supcon_pn,argmax,0.5428132128371351,0.539317176530844,0.5439244909764815,0.5405017904603583,0.5395151204533274,0.5412143582516292,0.5439244909764815,25
+met_supcon_tpn,rule,0.5423193140005884,0.5384736308378371,0.5395363545957231,0.5389118038787307,0.5369522756215227,0.5392386757868805,0.5423193140005884,25
+tab_mlp_o60_joint,rule,0.494952703421505,0.5158870976082757,0.536093137317441,0.5594559771069869,0.5869008087893082,0.5386579448487033,0.5869008087893082,25
+ts_tcnn_ry_ft,rule_smooth,0.46485081790558064,0.5087991787641932,0.5264767493850201,0.5788717640338676,0.6076062125318787,0.5373209445241081,0.6076062125318787,25
+zs_lgbm__lp,rule_smooth,0.5033070316420145,0.5130309258351171,0.5245124192352439,0.5503235218543303,0.5862979298626599,0.5354943656858732,0.5862979298626599,25
+tab_mlp_rz_joint,rule,0.503921751474847,0.5139613210927939,0.5282687736120961,0.548179930266462,0.5774549978166499,0.5343573548525697,0.5774549978166499,25
+met_supcon_tpn,argmax,0.5358138867686761,0.5330467825816138,0.5342786530218058,0.5336943755898285,0.5317910398087691,0.5337249475541386,0.5358138867686761,25
+tab_mlp_rg_zs,rule,0.534717008201463,0.5344676078764695,0.5339176629374895,0.5328374182325445,0.5311071793345707,0.5334093753165075,0.534717008201463,25
+ts_tcnn_dann_st_zs,rule,0.5297860478646774,0.5296293755599334,0.5291740147329642,0.5275355626920503,0.5273441037014119,0.5286938209102076,0.5297860478646774,25
+tab_res_o60_zs,rule_smooth,0.5292395787911406,0.5290653930261697,0.5286086827356464,0.5277522475270755,0.5261598122196292,0.5281651428599323,0.5292395787911406,25
+tab_res_o60_zs,argmax,0.5257933572946315,0.5256595543002501,0.5252289974550344,0.524704699358295,0.5235105420725237,0.524979430096147,0.5257933572946315,25
+tab_mlp_o60_joint,argmax,0.473057823304248,0.49344967967922104,0.520676611016921,0.5521754387621189,0.5762052427624631,0.5231129591049944,0.5762052427624631,25
+cls_lr_zs,rule,0.5235372265050621,0.5234453131254976,0.5228386859966806,0.5222686541501756,0.5210245623723537,0.522622888429954,0.5235372265050621,25
+zs_lgbm__em,argmax,0.520495613115607,0.5203915360432558,0.5202813863250955,0.5200655748178984,0.5190796573150406,0.5200627535233795,0.520495613115607,25
+tab_mlp_rz_joint,argmax,0.479708186091169,0.4951142582252224,0.513723423741525,0.535862379370695,0.5671177975413197,0.5183052089939862,0.5671177975413197,25
+tab_mlp_rz_zs,rule,0.5184626741266707,0.5182577146669474,0.5179587130009444,0.5169627275162081,0.5162473622888384,0.5175778383199219,0.5184626741266707,25
+tab_mlp_o60_zs,rule,0.5178427405139988,0.5174745008650438,0.5170566038310154,0.5158964015920963,0.5152315080600651,0.5167003509724439,0.5178427405139988,25
+tab_mlp_rg_zs,argmax,0.5143432686501652,0.5142128595614801,0.5139514875691689,0.5133820043792188,0.5119351171466573,0.5135649474613381,0.5143432686501652,25
+cls_svm_zs,rule_smooth,0.5144808447650704,0.514122870693057,0.51393658444761,0.5129051832255159,0.5114944345341883,0.5133879835330883,0.5144808447650704,25
+zs_dann__em,argmax,0.5095529920630784,0.5093244145203577,0.5088643968362854,0.5081930227220106,0.5070934064788868,0.5086056465241238,0.5095529920630784,25
+cls_lr_zs,argmax,0.5071928969398255,0.5070409909633208,0.5066202022279211,0.5061268181128608,0.5050185347984114,0.506399888608468,0.5071928969398255,25
+cls_svm_zs,rule,0.5068830067503021,0.5065160506953371,0.5060681319151491,0.5049895375181583,0.5036542103446898,0.5056221874447272,0.5068830067503021,25
+ts_tcnn_dann_st_zs,argmax,0.5042768321985243,0.5042599168936553,0.5041291165292223,0.5034090469564967,0.503141651817059,0.5038433128789915,0.5042768321985243,25
+cls_svm_zs,argmax,0.5043305697290678,0.5041058954506041,0.5036848442670435,0.502722384970548,0.501559188905013,0.5032805766644552,0.5043305697290678,25
+ts_tcnn_ry_ft,rule,0.4442134015613249,0.4800798853987934,0.49500239200494967,0.5344938549411132,0.5602795788101694,0.5028138225432702,0.5602795788101694,25
+ts_tcnn_st_ft,rule_smooth,0.3849735700301594,0.45049642325350464,0.48692744625958523,0.5666975946346359,0.6213227617869286,0.5020835591929627,0.6213227617869286,25
+tab_res_rz_zs,rule_smooth,0.500277997552148,0.5001815443406576,0.49994486229488455,0.4985094400822395,0.4974395052654684,0.4992706699070796,0.500277997552148,25
+ts_tcnn_ry_joint,rule_smooth,0.4476530218723226,0.4720736271159609,0.48734719427106077,0.5237621413183058,0.5577322858163569,0.4977136540788013,0.5577322858163569,25
+tab_mlp_rz_zs,argmax,0.49298959051400687,0.4929866441070624,0.49258034217169194,0.4917165077208045,0.49109837174166404,0.49227429125104594,0.49298959051400687,25
+ts_tcnn_ry_ft,argmax,0.4227104818353789,0.4693237048197593,0.4848932490907078,0.5284136789160943,0.553434860758324,0.49175519508405285,0.553434860758324,25
+tab_mlp_o60_zs,argmax,0.4901116330789149,0.48993380075539594,0.489558429695607,0.4889279217518842,0.48813356713023703,0.4893330704824078,0.4901116330789149,25
+zs_xgb_dann__em,argmax,0.485539836947286,0.4852956300582393,0.48517782249803154,0.4845205789836843,0.48400772521411256,0.4849083187402707,0.485539836947286,25
+tab_res_rz_zs,rule,0.47676984023270635,0.4767097638644103,0.47648146518072343,0.4755389174319225,0.4746046212232817,0.4760209215866088,0.47676984023270635,25
+ts_tcnn_st_ft,rule,0.37606410961365716,0.42676587186436626,0.4568425529006495,0.5224998927122958,0.5742682028115271,0.4712881259804992,0.5742682028115271,25
+ts_tcnn_dann_st_zs,rule_smooth,0.4702146305262637,0.4701171751582107,0.4699968147002703,0.46907004775835015,0.46710686306975147,0.46930110624256927,0.4702146305262637,25
+tab_mlp_o60_zs,rule_smooth,0.4684821166599134,0.46841079247601974,0.4673568221763336,0.4668206109902687,0.4658934822718649,0.46739276491488013,0.4684821166599134,25
+ts_tcnn_ry_joint,rule,0.43018034541593486,0.44748111667618795,0.459926385356016,0.4855397018715715,0.5127931803730279,0.4671841459385476,0.5127931803730279,25
+ts_tcnn_st_joint,rule_smooth,0.3472525368684599,0.41316957690357176,0.46025492978814403,0.525519255365836,0.5750076138472174,0.46424078255464585,0.5750076138472174,25
+tab_res_rz_zs,argmax,0.46022420540715075,0.46015806157587474,0.45990035456699274,0.4591981500781886,0.45837213662257714,0.45957058165015685,0.46022420540715075,25
+ts_tcnn_st_ft,argmax,0.3254537664591867,0.41036924151193893,0.4420264127542253,0.5177915233693039,0.5695884649299956,0.45304588180493005,0.5695884649299956,25
+ts_tcnn_ry_joint,argmax,0.40346407545916063,0.4218738372905843,0.440118264083979,0.4682224495869568,0.4963012566563168,0.4459959766153996,0.4963012566563168,25
+ts_tcnn_st_joint,rule,0.3395608421763471,0.3972383580294001,0.4351096221085064,0.48856009315452276,0.5314350957109835,0.4383808022359519,0.5314350957109835,25
+ts_tcnn_ry_zs,rule_smooth,0.4290246965719352,0.42865235687002057,0.4282164795580082,0.42809009383090213,0.42639720732426356,0.42807616683102595,0.4290246965719352,25
+ts_tcnn_ry_zs,rule,0.41615335184216623,0.41586389810393587,0.41549566760663065,0.4148243157987472,0.41242544362442457,0.4149525353951809,0.41615335184216623,25
+ts_tcnn_ry_zs,argmax,0.38629812800964947,0.3860265203322294,0.38566732080214844,0.3854704044510031,0.38360976863291946,0.38541442844558993,0.38629812800964947,25
+ts_tcnn_st_joint,argmax,0.27087947317958694,0.33929121857491473,0.3779464505506491,0.43857362861245874,0.4801821347871006,0.381374581140942,0.4801821347871006,25
+ts_tcnn_st_zs,rule_smooth,0.30250545928879924,0.3023094829382745,0.3019170541441609,0.3015930478588448,0.30036258800756854,0.3017375264475296,0.30250545928879924,25
+ts_tcnn_st_zs,rule,0.2983428984282512,0.2981478861054498,0.2977132054260889,0.29717192363657996,0.29587490547380496,0.297450163814035,0.2983428984282512,25
+ts_tcnn_st_zs,argmax,0.22717336812791186,0.22700384115189814,0.22670153269422197,0.22617943600989948,0.22511777559997367,0.22643519071678103,0.22717336812791186,25

reports/nnlab/logs/classical.log

@@ -0,0 +1,179 @@
+[    6.7s] loaded Madrid (76263, 393) Amsterdam (25992, 393)
+[   17.4s] cls_lr_zs n=0 r=0 done in 10.7s
+[   38.8s] cls_svm_zs n=0 r=0 done in 21.4s
+[   46.8s] cls_lr n=5 r=0 done in 8.0s
+[   54.6s] cls_lr n=5 r=1 done in 7.8s
+[   62.8s] cls_lr n=5 r=2 done in 8.2s
+[   72.1s] cls_lr n=5 r=3 done in 9.3s
+[   81.0s] cls_lr n=5 r=4 done in 8.9s
+[   89.4s] cls_lr n=25 r=0 done in 8.4s
+[   98.2s] cls_lr n=25 r=1 done in 8.7s
+[  106.3s] cls_lr n=25 r=2 done in 8.1s
+[  114.9s] cls_lr n=25 r=3 done in 8.6s
+[  123.7s] cls_lr n=25 r=4 done in 8.8s
+[  132.6s] cls_lr n=50 r=0 done in 8.9s
+[  141.6s] cls_lr n=50 r=1 done in 9.1s
+[  150.0s] cls_lr n=50 r=2 done in 8.4s
+[  159.0s] cls_lr n=50 r=3 done in 8.9s
+[  168.3s] cls_lr n=50 r=4 done in 9.3s
+[  177.8s] cls_lr n=100 r=0 done in 9.5s
+[  187.2s] cls_lr n=100 r=1 done in 9.4s
+[  196.2s] cls_lr n=100 r=2 done in 8.9s
+[  204.5s] cls_lr n=100 r=3 done in 8.4s
+[  212.6s] cls_lr n=100 r=4 done in 8.1s
+[  220.9s] cls_lr n=200 r=0 done in 8.3s
+[  229.5s] cls_lr n=200 r=1 done in 8.6s
+[  237.2s] cls_lr n=200 r=2 done in 7.6s
+[  244.8s] cls_lr n=200 r=3 done in 7.6s
+[  252.4s] cls_lr n=200 r=4 done in 7.6s
+[  273.7s] cls_lr_feda n=5 r=0 done in 21.3s
+[  294.6s] cls_lr_feda n=5 r=1 done in 20.9s
+[  317.3s] cls_lr_feda n=5 r=2 done in 22.7s
+[  338.6s] cls_lr_feda n=5 r=3 done in 21.3s
+[  358.2s] cls_lr_feda n=5 r=4 done in 19.6s
+[  378.7s] cls_lr_feda n=25 r=0 done in 20.5s
+[  399.2s] cls_lr_feda n=25 r=1 done in 20.5s
+[  419.1s] cls_lr_feda n=25 r=2 done in 19.9s
+[  437.9s] cls_lr_feda n=25 r=3 done in 18.9s
+[  457.3s] cls_lr_feda n=25 r=4 done in 19.4s
+[  475.9s] cls_lr_feda n=50 r=0 done in 18.6s
+[  494.7s] cls_lr_feda n=50 r=1 done in 18.9s
+[  513.4s] cls_lr_feda n=50 r=2 done in 18.7s
+[  531.6s] cls_lr_feda n=50 r=3 done in 18.2s
+[  549.8s] cls_lr_feda n=50 r=4 done in 18.3s
+[  568.9s] cls_lr_feda n=100 r=0 done in 19.1s
+[  589.1s] cls_lr_feda n=100 r=1 done in 20.2s
+[  608.8s] cls_lr_feda n=100 r=2 done in 19.8s
+[  628.3s] cls_lr_feda n=100 r=3 done in 19.4s
+[  647.7s] cls_lr_feda n=100 r=4 done in 19.4s
+[  666.4s] cls_lr_feda n=200 r=0 done in 18.7s
+[  683.8s] cls_lr_feda n=200 r=1 done in 17.4s
+[  704.1s] cls_lr_feda n=200 r=2 done in 20.3s
+[  726.2s] cls_lr_feda n=200 r=3 done in 22.1s
+[  748.6s] cls_lr_feda n=200 r=4 done in 22.3s
+[  773.2s] cls_svm n=5 r=0 done in 24.7s
+[  796.0s] cls_svm n=5 r=1 done in 22.8s
+[  820.8s] cls_svm n=5 r=2 done in 24.7s
+[  845.7s] cls_svm n=5 r=3 done in 24.9s
+[  870.1s] cls_svm n=5 r=4 done in 24.4s
+[  895.0s] cls_svm n=25 r=0 done in 24.9s
+[  918.4s] cls_svm n=25 r=1 done in 23.4s
+[  943.2s] cls_svm n=25 r=2 done in 24.9s
+[  968.3s] cls_svm n=25 r=3 done in 25.1s
+[  992.7s] cls_svm n=25 r=4 done in 24.4s
+[ 1018.6s] cls_svm n=50 r=0 done in 25.9s
+[ 1045.4s] cls_svm n=50 r=1 done in 26.8s
+[ 1070.9s] cls_svm n=50 r=2 done in 25.5s
+[ 1097.6s] cls_svm n=50 r=3 done in 26.8s
+[ 1123.8s] cls_svm n=50 r=4 done in 26.2s
+[ 1149.6s] cls_svm n=100 r=0 done in 25.7s
+[ 1176.5s] cls_svm n=100 r=1 done in 27.0s
+[ 1203.2s] cls_svm n=100 r=2 done in 26.7s
+[ 1229.2s] cls_svm n=100 r=3 done in 26.0s
+[ 1255.8s] cls_svm n=100 r=4 done in 26.5s
+[ 1281.1s] cls_svm n=200 r=0 done in 25.3s
+[ 1309.2s] cls_svm n=200 r=1 done in 28.1s
+[ 1339.1s] cls_svm n=200 r=2 done in 29.9s
+[ 1366.6s] cls_svm n=200 r=3 done in 27.5s
+[ 1394.7s] cls_svm n=200 r=4 done in 28.1s
+[ 1447.7s] cls_svm_feda n=5 r=0 done in 53.1s
+[ 1501.8s] cls_svm_feda n=5 r=1 done in 54.1s
+[ 1551.7s] cls_svm_feda n=5 r=2 done in 49.9s
+[ 1600.2s] cls_svm_feda n=5 r=3 done in 48.5s
+[ 1652.6s] cls_svm_feda n=5 r=4 done in 52.4s
+[ 1710.2s] cls_svm_feda n=25 r=0 done in 57.6s
+[ 1762.1s] cls_svm_feda n=25 r=1 done in 51.9s
+[ 1810.4s] cls_svm_feda n=25 r=2 done in 48.3s
+[ 1864.6s] cls_svm_feda n=25 r=3 done in 54.2s
+[ 1920.8s] cls_svm_feda n=25 r=4 done in 56.3s
+[ 1970.7s] cls_svm_feda n=50 r=0 done in 49.8s
+[ 2021.9s] cls_svm_feda n=50 r=1 done in 51.3s
+[ 2079.2s] cls_svm_feda n=50 r=2 done in 57.3s
+[ 2136.3s] cls_svm_feda n=50 r=3 done in 57.1s
+[ 2186.3s] cls_svm_feda n=50 r=4 done in 49.9s
+[ 2240.0s] cls_svm_feda n=100 r=0 done in 53.7s
+[ 2298.0s] cls_svm_feda n=100 r=1 done in 58.1s
+[ 2354.1s] cls_svm_feda n=100 r=2 done in 56.0s
+[ 2411.0s] cls_svm_feda n=100 r=3 done in 56.9s
+[ 2467.6s] cls_svm_feda n=100 r=4 done in 56.6s
+[ 2512.3s] cls_svm_feda n=200 r=0 done in 44.7s
+[ 2542.7s] cls_svm_feda n=200 r=1 done in 30.5s
+[ 2577.2s] cls_svm_feda n=200 r=2 done in 34.4s
+[ 2608.2s] cls_svm_feda n=200 r=3 done in 31.0s
+[ 2639.8s] cls_svm_feda n=200 r=4 done in 31.6s
+[ 2644.7s] cls_nys_lr n=5 r=0 done in 4.9s
+[ 2648.8s] cls_nys_lr n=5 r=1 done in 4.1s
+[ 2652.6s] cls_nys_lr n=5 r=2 done in 3.8s
+[ 2657.5s] cls_nys_lr n=5 r=3 done in 4.9s
+[ 2664.1s] cls_nys_lr n=5 r=4 done in 6.5s
+[ 2668.8s] cls_nys_lr n=25 r=0 done in 4.8s
+[ 2674.7s] cls_nys_lr n=25 r=1 done in 5.9s
+[ 2680.5s] cls_nys_lr n=25 r=2 done in 5.8s
+[ 2684.6s] cls_nys_lr n=25 r=3 done in 4.1s
+[ 2687.8s] cls_nys_lr n=25 r=4 done in 3.2s
+[ 2692.5s] cls_nys_lr n=50 r=0 done in 4.7s
+[ 2697.4s] cls_nys_lr n=50 r=1 done in 4.9s
+[ 2701.6s] cls_nys_lr n=50 r=2 done in 4.2s
+[ 2705.0s] cls_nys_lr n=50 r=3 done in 3.4s
+[ 2709.4s] cls_nys_lr n=50 r=4 done in 4.4s
+[ 2713.2s] cls_nys_lr n=100 r=0 done in 3.8s
+[ 2717.9s] cls_nys_lr n=100 r=1 done in 4.7s
+[ 2720.8s] cls_nys_lr n=100 r=2 done in 2.9s
+[ 2723.3s] cls_nys_lr n=100 r=3 done in 2.5s
+[ 2727.8s] cls_nys_lr n=100 r=4 done in 4.5s
+[ 2733.9s] cls_nys_lr n=200 r=0 done in 6.0s
+[ 2738.1s] cls_nys_lr n=200 r=1 done in 4.2s
+[ 2742.2s] cls_nys_lr n=200 r=2 done in 4.1s
+[ 2744.6s] cls_nys_lr n=200 r=3 done in 2.4s
+[ 2750.3s] cls_nys_lr n=200 r=4 done in 5.7s
+[ 2750.4s] cls_knn n=5 r=0 done in 0.1s
+[ 2750.5s] cls_knn n=5 r=1 done in 0.1s
+[ 2750.6s] cls_knn n=5 r=2 done in 0.1s
+[ 2750.7s] cls_knn n=5 r=3 done in 0.1s
+[ 2750.8s] cls_knn n=5 r=4 done in 0.1s
+[ 2751.0s] cls_knn n=25 r=0 done in 0.2s
+[ 2751.2s] cls_knn n=25 r=1 done in 0.2s
+[ 2751.4s] cls_knn n=25 r=2 done in 0.2s
+[ 2751.5s] cls_knn n=25 r=3 done in 0.2s
+[ 2751.7s] cls_knn n=25 r=4 done in 0.2s
+[ 2752.0s] cls_knn n=50 r=0 done in 0.3s
+[ 2752.3s] cls_knn n=50 r=1 done in 0.3s
+[ 2752.6s] cls_knn n=50 r=2 done in 0.3s
+[ 2752.9s] cls_knn n=50 r=3 done in 0.3s
+[ 2753.2s] cls_knn n=50 r=4 done in 0.3s
+[ 2753.5s] cls_knn n=100 r=0 done in 0.4s
+[ 2753.9s] cls_knn n=100 r=1 done in 0.4s
+[ 2754.4s] cls_knn n=100 r=2 done in 0.4s
+[ 2754.8s] cls_knn n=100 r=3 done in 0.4s
+[ 2755.1s] cls_knn n=100 r=4 done in 0.3s
+[ 2755.7s] cls_knn n=200 r=0 done in 0.6s
+[ 2756.3s] cls_knn n=200 r=1 done in 0.6s
+[ 2756.9s] cls_knn n=200 r=2 done in 0.6s
+[ 2757.6s] cls_knn n=200 r=3 done in 0.7s
+[ 2758.2s] cls_knn n=200 r=4 done in 0.6s
+[ 2758.5s] cls_lda n=5 r=0 done in 0.3s
+[ 2758.7s] cls_lda n=5 r=1 done in 0.3s
+[ 2759.0s] cls_lda n=5 r=2 done in 0.2s
+[ 2759.2s] cls_lda n=5 r=3 done in 0.2s
+[ 2759.4s] cls_lda n=5 r=4 done in 0.2s
+[ 2759.6s] cls_lda n=25 r=0 done in 0.2s
+[ 2759.9s] cls_lda n=25 r=1 done in 0.2s
+[ 2760.1s] cls_lda n=25 r=2 done in 0.3s
+[ 2760.4s] cls_lda n=25 r=3 done in 0.2s
+[ 2760.6s] cls_lda n=25 r=4 done in 0.2s
+[ 2760.9s] cls_lda n=50 r=0 done in 0.2s
+[ 2761.1s] cls_lda n=50 r=1 done in 0.2s
+[ 2761.3s] cls_lda n=50 r=2 done in 0.3s
+[ 2761.6s] cls_lda n=50 r=3 done in 0.2s
+[ 2761.8s] cls_lda n=50 r=4 done in 0.2s
+[ 2762.0s] cls_lda n=100 r=0 done in 0.2s
+[ 2762.3s] cls_lda n=100 r=1 done in 0.3s
+[ 2762.5s] cls_lda n=100 r=2 done in 0.2s
+[ 2762.7s] cls_lda n=100 r=3 done in 0.2s
+[ 2763.0s] cls_lda n=100 r=4 done in 0.2s
+[ 2763.2s] cls_lda n=200 r=0 done in 0.3s
+[ 2763.5s] cls_lda n=200 r=1 done in 0.2s
+[ 2763.7s] cls_lda n=200 r=2 done in 0.3s
+[ 2764.0s] cls_lda n=200 r=3 done in 0.3s
+[ 2764.3s] cls_lda n=200 r=4 done in 0.3s
+[ 2764.3s] all done

reports/nnlab/logs/feda_madrid_check.log

@@ -0,0 +1,14 @@
+[  244s] n=5 rep=0 base=0.5113 feda=0.5166  (argmax 0.5212 / 0.5165)
+[  502s] n=5 rep=1 base=0.5072 feda=0.5274  (argmax 0.5159 / 0.5312)
+[  748s] n=5 rep=2 base=0.5110 feda=0.5003  (argmax 0.5224 / 0.5039)
+[  998s] n=50 rep=0 base=0.5426 feda=0.5629  (argmax 0.5585 / 0.5663)
+[ 1250s] n=50 rep=1 base=0.5449 feda=0.5559  (argmax 0.5528 / 0.5581)
+[ 1502s] n=50 rep=2 base=0.5445 feda=0.5674  (argmax 0.5509 / 0.5685)
+[ 1754s] n=200 rep=0 base=0.5867 feda=0.5985  (argmax 0.6000 / 0.6082)
+[ 2009s] n=200 rep=1 base=0.5915 feda=0.6173  (argmax 0.6082 / 0.6243)
+[ 2263s] n=200 rep=2 base=0.5894 feda=0.6131  (argmax 0.6071 / 0.6258)
+       base  lgbm_feda  base_argmax  lgbm_feda_argmax
+n                                                    
+5    0.5098     0.5147       0.5199            0.5172
+50   0.5440     0.5621       0.5541            0.5643
+200  0.5892     0.6096       0.6051            0.6195

reports/nnlab/logs/lgbm_emb.log

@@ -0,0 +1,26 @@
+lgbm_emb n=5 rep=0 115s (total 115s)
+lgbm_emb n=5 rep=1 131s (total 246s)
+lgbm_emb n=5 rep=2 128s (total 374s)
+lgbm_emb n=5 rep=3 129s (total 503s)
+lgbm_emb n=5 rep=4 129s (total 632s)
+lgbm_emb n=25 rep=0 124s (total 756s)
+lgbm_emb n=25 rep=1 129s (total 885s)
+lgbm_emb n=25 rep=2 129s (total 1014s)
+lgbm_emb n=25 rep=3 129s (total 1143s)
+lgbm_emb n=25 rep=4 136s (total 1278s)
+lgbm_emb n=50 rep=0 139s (total 1417s)
+lgbm_emb n=50 rep=1 130s (total 1547s)
+lgbm_emb n=50 rep=2 131s (total 1678s)
+lgbm_emb n=50 rep=3 126s (total 1804s)
+lgbm_emb n=50 rep=4 114s (total 1918s)
+lgbm_emb n=100 rep=0 113s (total 2031s)
+lgbm_emb n=100 rep=1 113s (total 2144s)
+lgbm_emb n=100 rep=2 113s (total 2257s)
+lgbm_emb n=100 rep=3 113s (total 2371s)
+lgbm_emb n=100 rep=4 113s (total 2484s)
+lgbm_emb n=200 rep=0 97s (total 2581s)
+lgbm_emb n=200 rep=1 89s (total 2670s)
+lgbm_emb n=200 rep=2 88s (total 2758s)
+lgbm_emb n=200 rep=3 91s (total 2849s)
+lgbm_emb n=200 rep=4 90s (total 2939s)
+done

reports/nnlab/logs/lgbm_feda.log

@@ -0,0 +1,26 @@
+lgbm_feda n=5 rep=0 125s (total 125s)
+lgbm_feda n=5 rep=1 141s (total 267s)
+lgbm_feda n=5 rep=2 139s (total 405s)
+lgbm_feda n=5 rep=3 140s (total 545s)
+lgbm_feda n=5 rep=4 136s (total 681s)
+lgbm_feda n=25 rep=0 143s (total 824s)
+lgbm_feda n=25 rep=1 146s (total 970s)
+lgbm_feda n=25 rep=2 146s (total 1116s)
+lgbm_feda n=25 rep=3 150s (total 1266s)
+lgbm_feda n=25 rep=4 154s (total 1420s)
+lgbm_feda n=50 rep=0 150s (total 1571s)
+lgbm_feda n=50 rep=1 150s (total 1720s)
+lgbm_feda n=50 rep=2 140s (total 1860s)
+lgbm_feda n=50 rep=3 131s (total 1991s)
+lgbm_feda n=50 rep=4 131s (total 2121s)
+lgbm_feda n=100 rep=0 133s (total 2255s)
+lgbm_feda n=100 rep=1 132s (total 2387s)
+lgbm_feda n=100 rep=2 131s (total 2518s)
+lgbm_feda n=100 rep=3 104s (total 2622s)
+lgbm_feda n=100 rep=4 103s (total 2725s)
+lgbm_feda n=200 rep=0 105s (total 2830s)
+lgbm_feda n=200 rep=1 105s (total 2935s)
+lgbm_feda n=200 rep=2 65s (total 2999s)
+lgbm_feda n=200 rep=3 63s (total 3063s)
+lgbm_feda n=200 rep=4 64s (total 3127s)
+done

reports/nnlab/logs/lgbm_feda_r12.log

@@ -0,0 +1,36 @@
+lgbm_feda n=5 rep=5 60s (total 60s)
+lgbm_feda n=5 rep=6 66s (total 127s)
+lgbm_feda n=5 rep=7 71s (total 198s)
+lgbm_feda n=5 rep=8 71s (total 269s)
+lgbm_feda n=5 rep=9 70s (total 339s)
+lgbm_feda n=5 rep=10 67s (total 406s)
+lgbm_feda n=5 rep=11 58s (total 464s)
+lgbm_feda n=25 rep=5 92s (total 557s)
+lgbm_feda n=25 rep=6 102s (total 659s)
+lgbm_feda n=25 rep=7 107s (total 766s)
+lgbm_feda n=25 rep=8 116s (total 882s)
+lgbm_feda n=25 rep=9 111s (total 993s)
+lgbm_feda n=25 rep=10 99s (total 1092s)
+lgbm_feda n=25 rep=11 104s (total 1196s)
+lgbm_feda n=50 rep=5 105s (total 1301s)
+lgbm_feda n=50 rep=6 103s (total 1403s)
+lgbm_feda n=50 rep=7 101s (total 1504s)
+lgbm_feda n=50 rep=8 99s (total 1603s)
+lgbm_feda n=50 rep=9 102s (total 1705s)
+lgbm_feda n=50 rep=10 105s (total 1811s)
+lgbm_feda n=50 rep=11 102s (total 1913s)
+lgbm_feda n=100 rep=5 109s (total 2021s)
+lgbm_feda n=100 rep=6 103s (total 2125s)
+lgbm_feda n=100 rep=7 104s (total 2229s)
+lgbm_feda n=100 rep=8 102s (total 2331s)
+lgbm_feda n=100 rep=9 107s (total 2439s)
+lgbm_feda n=100 rep=10 104s (total 2543s)
+lgbm_feda n=100 rep=11 105s (total 2648s)
+lgbm_feda n=200 rep=5 105s (total 2753s)
+lgbm_feda n=200 rep=6 83s (total 2836s)
+lgbm_feda n=200 rep=7 77s (total 2914s)
+lgbm_feda n=200 rep=8 78s (total 2991s)
+lgbm_feda n=200 rep=9 76s (total 3068s)
+lgbm_feda n=200 rep=10 80s (total 3147s)
+lgbm_feda n=200 rep=11 75s (total 3222s)
+done

reports/nnlab/logs/lgbm_fedaw100.log

@@ -0,0 +1,26 @@
+lgbm_fedaw100 n=5 rep=0 108s (total 108s)
+lgbm_fedaw100 n=5 rep=1 111s (total 219s)
+lgbm_fedaw100 n=5 rep=2 118s (total 337s)
+lgbm_fedaw100 n=5 rep=3 123s (total 460s)
+lgbm_fedaw100 n=5 rep=4 112s (total 571s)
+lgbm_fedaw100 n=25 rep=0 112s (total 683s)
+lgbm_fedaw100 n=25 rep=1 117s (total 801s)
+lgbm_fedaw100 n=25 rep=2 115s (total 916s)
+lgbm_fedaw100 n=25 rep=3 112s (total 1028s)
+lgbm_fedaw100 n=25 rep=4 111s (total 1139s)
+lgbm_fedaw100 n=50 rep=0 114s (total 1253s)
+lgbm_fedaw100 n=50 rep=1 117s (total 1370s)
+lgbm_fedaw100 n=50 rep=2 118s (total 1488s)
+lgbm_fedaw100 n=50 rep=3 115s (total 1603s)
+lgbm_fedaw100 n=50 rep=4 115s (total 1718s)
+lgbm_fedaw100 n=100 rep=0 115s (total 1833s)
+lgbm_fedaw100 n=100 rep=1 117s (total 1950s)
+lgbm_fedaw100 n=100 rep=2 114s (total 2063s)
+lgbm_fedaw100 n=100 rep=3 114s (total 2177s)
+lgbm_fedaw100 n=100 rep=4 114s (total 2291s)
+lgbm_fedaw100 n=200 rep=0 91s (total 2382s)
+lgbm_fedaw100 n=200 rep=1 88s (total 2470s)
+lgbm_fedaw100 n=200 rep=2 88s (total 2559s)
+lgbm_fedaw100 n=200 rep=3 88s (total 2647s)
+lgbm_fedaw100 n=200 rep=4 90s (total 2737s)
+done

reports/nnlab/logs/lgbm_upw.log

@@ -0,0 +1,26 @@
+lgbm_upw n=5 rep=0 122s
+lgbm_upw n=5 rep=1 120s
+lgbm_upw n=5 rep=2 408s
+lgbm_upw n=5 rep=3 340s
+lgbm_upw n=5 rep=4 161s
+lgbm_upw n=25 rep=0 174s
+lgbm_upw n=25 rep=1 145s
+lgbm_upw n=25 rep=2 73s
+lgbm_upw n=25 rep=3 67s
+lgbm_upw n=25 rep=4 100s
+lgbm_upw n=50 rep=0 62s
+lgbm_upw n=50 rep=1 66s
+lgbm_upw n=50 rep=2 82s
+lgbm_upw n=50 rep=3 63s
+lgbm_upw n=50 rep=4 58s
+lgbm_upw n=100 rep=0 53s
+lgbm_upw n=100 rep=1 55s
+lgbm_upw n=100 rep=2 61s
+lgbm_upw n=100 rep=3 61s
+lgbm_upw n=100 rep=4 59s
+lgbm_upw n=200 rep=0 52s
+lgbm_upw n=200 rep=1 96s
+lgbm_upw n=200 rep=2 100s
+lgbm_upw n=200 rep=3 87s
+lgbm_upw n=200 rep=4 82s
+done

reports/nnlab/logs/lgbm_upw_r12.log

@@ -0,0 +1,36 @@
+lgbm_upw n=5 rep=5 64s
+lgbm_upw n=5 rep=6 61s
+lgbm_upw n=5 rep=7 66s
+lgbm_upw n=5 rep=8 73s
+lgbm_upw n=5 rep=9 98s
+lgbm_upw n=5 rep=10 68s
+lgbm_upw n=5 rep=11 63s
+lgbm_upw n=25 rep=5 61s
+lgbm_upw n=25 rep=6 82s
+lgbm_upw n=25 rep=7 91s
+lgbm_upw n=25 rep=8 73s
+lgbm_upw n=25 rep=9 58s
+lgbm_upw n=25 rep=10 49s
+lgbm_upw n=25 rep=11 47s
+lgbm_upw n=50 rep=5 49s
+lgbm_upw n=50 rep=6 49s
+lgbm_upw n=50 rep=7 50s
+lgbm_upw n=50 rep=8 49s
+lgbm_upw n=50 rep=9 49s
+lgbm_upw n=50 rep=10 49s
+lgbm_upw n=50 rep=11 43s
+lgbm_upw n=100 rep=5 37s
+lgbm_upw n=100 rep=6 49s
+lgbm_upw n=100 rep=7 49s
+lgbm_upw n=100 rep=8 48s
+lgbm_upw n=100 rep=9 47s
+lgbm_upw n=100 rep=10 48s
+lgbm_upw n=100 rep=11 48s
+lgbm_upw n=200 rep=5 49s
+lgbm_upw n=200 rep=6 47s
+lgbm_upw n=200 rep=7 51s
+lgbm_upw n=200 rep=8 49s
+lgbm_upw n=200 rep=9 48s
+lgbm_upw n=200 rep=10 39s
+lgbm_upw n=200 rep=11 48s
+done

reports/nnlab/logs/lgbm_w100.log

@@ -0,0 +1,26 @@
+lgbm_w100 n=5 rep=0 92s (total 92s)
+lgbm_w100 n=5 rep=1 105s (total 197s)
+lgbm_w100 n=5 rep=2 106s (total 303s)
+lgbm_w100 n=5 rep=3 104s (total 406s)
+lgbm_w100 n=5 rep=4 105s (total 512s)
+lgbm_w100 n=25 rep=0 104s (total 616s)
+lgbm_w100 n=25 rep=1 100s (total 716s)
+lgbm_w100 n=25 rep=2 105s (total 821s)
+lgbm_w100 n=25 rep=3 103s (total 924s)
+lgbm_w100 n=25 rep=4 105s (total 1029s)
+lgbm_w100 n=50 rep=0 105s (total 1133s)
+lgbm_w100 n=50 rep=1 110s (total 1244s)
+lgbm_w100 n=50 rep=2 114s (total 1358s)
+lgbm_w100 n=50 rep=3 108s (total 1466s)
+lgbm_w100 n=50 rep=4 105s (total 1571s)
+lgbm_w100 n=100 rep=0 106s (total 1677s)
+lgbm_w100 n=100 rep=1 103s (total 1780s)
+lgbm_w100 n=100 rep=2 94s (total 1874s)
+lgbm_w100 n=100 rep=3 92s (total 1966s)
+lgbm_w100 n=100 rep=4 91s (total 2057s)
+lgbm_w100 n=200 rep=0 92s (total 2148s)
+lgbm_w100 n=200 rep=1 94s (total 2242s)
+lgbm_w100 n=200 rep=2 92s (total 2334s)
+lgbm_w100 n=200 rep=3 92s (total 2426s)
+lgbm_w100 n=200 rep=4 92s (total 2518s)
+done

reports/nnlab/logs/lp_madrid_check.log

@@ -0,0 +1,10 @@
+[    1.0s] loaded: Madrid (76263, 393), Amsterdam (25992, 393)
+[  169.2s] madrid-check: base model fitted on 61932 src rows
+[  169.3s] madrid-check: pseudo-target 14331 rows, mean degree 23.4
+[  177.6s] madrid-check: rule macro F1 on pseudo-target (non-shot rows), mean over reps
+   base         n5=0.5009  n25=0.4996  n50=0.4991  n100=0.4974  n200=0.4936  avg=0.4981
+   clamp        n5=0.5033  n25=0.5041  n50=0.5046  n100=0.5072  n200=0.5094  avg=0.5057
+   lp           n5=0.4621  n25=0.4660  n50=0.4739  n100=0.4803  n200=0.4910  avg=0.4747
+   lapshot_row  n5=0.5044  n25=0.5044  n50=0.5051  n100=0.5060  n200=0.5055  avg=0.5051
+   lapshot_raw  n5=0.4308  n25=0.4316  n50=0.4335  n100=0.4369  n200=0.4394  avg=0.4344
+   em           n5=0.5101  n25=0.5091  n50=0.5088  n100=0.5078  n200=0.5042  avg=0.5080

reports/nnlab/logs/lpft.log

@@ -0,0 +1,39 @@
+[    4.5s] data loaded
+[  100.2s] trained base dann_rz_src_s0.pt (3 threads): 15 epochs on 61932 rows  88s
+[  100.3s] madrid-check base vs tab_nn sel_dann.npz Zt: max abs diff 0.00e+00
+[  109.1s] madrid-check n=5 rep=0: lp=0.4571  lpft=0.4753  lpft_ent=0.4806  8.7s
+[  113.6s] madrid-check n=5 rep=1: lp=0.4962  lpft=0.4790  lpft_ent=0.4862  4.5s
+[  117.8s] madrid-check n=5 rep=2: lp=0.4991  lpft=0.5194  lpft_ent=0.5219  4.1s
+[  123.1s] madrid-check n=50 rep=0: lp=0.5469  lpft=0.5491  lpft_ent=0.5509  5.3s
+[  127.7s] madrid-check n=50 rep=1: lp=0.5190  lpft=0.5527  lpft_ent=0.5516  4.6s
+[  133.5s] madrid-check n=50 rep=2: lp=0.5177  lpft=0.5471  lpft_ent=0.5486  5.8s
+[  139.6s] madrid-check n=200 rep=0: lp=0.5293  lpft=0.5695  lpft_ent=0.5711  6.2s
+[  146.0s] madrid-check n=200 rep=1: lp=0.5289  lpft=0.5797  lpft_ent=0.5784  6.3s
+[  152.5s] madrid-check n=200 rep=2: lp=0.5229  lpft=0.5713  lpft_ent=0.5704  6.6s
+[  152.6s] madrid-check means: {'lp': {5: 0.4841498120142176, 50: 0.5279016320332399, 200: 0.5270376259955917}, 'lpft': {5: 0.49122672993148825, 50: 0.5496386272948383, 200: 0.5735218920183637}, 'lpft_ent': {5: 0.49622852008497914, 50: 0.5503723327589127, 200: 0.5732739094886868}}
+[  157.9s] tab_dann_rz_lpft+tab_dann_rz_lpft_ent n=5 rep=0 5.3s
+[  163.8s] tab_dann_rz_lpft+tab_dann_rz_lpft_ent n=5 rep=1 5.9s
+[  168.9s] tab_dann_rz_lpft+tab_dann_rz_lpft_ent n=5 rep=2 5.1s
+[  174.1s] tab_dann_rz_lpft+tab_dann_rz_lpft_ent n=5 rep=3 5.2s
+[  179.7s] tab_dann_rz_lpft+tab_dann_rz_lpft_ent n=5 rep=4 5.6s
+[  185.6s] tab_dann_rz_lpft+tab_dann_rz_lpft_ent n=25 rep=0 5.9s
+[  191.9s] tab_dann_rz_lpft+tab_dann_rz_lpft_ent n=25 rep=1 6.3s
+[  197.2s] tab_dann_rz_lpft+tab_dann_rz_lpft_ent n=25 rep=2 5.3s
+[  202.8s] tab_dann_rz_lpft+tab_dann_rz_lpft_ent n=25 rep=3 5.6s
+[  207.9s] tab_dann_rz_lpft+tab_dann_rz_lpft_ent n=25 rep=4 5.1s
+[  213.7s] tab_dann_rz_lpft+tab_dann_rz_lpft_ent n=50 rep=0 5.8s
+[  220.3s] tab_dann_rz_lpft+tab_dann_rz_lpft_ent n=50 rep=1 6.6s
+[  226.1s] tab_dann_rz_lpft+tab_dann_rz_lpft_ent n=50 rep=2 5.8s
+[  231.9s] tab_dann_rz_lpft+tab_dann_rz_lpft_ent n=50 rep=3 5.8s
+[  237.6s] tab_dann_rz_lpft+tab_dann_rz_lpft_ent n=50 rep=4 5.8s
+[  243.8s] tab_dann_rz_lpft+tab_dann_rz_lpft_ent n=100 rep=0 6.2s
+[  251.1s] tab_dann_rz_lpft+tab_dann_rz_lpft_ent n=100 rep=1 7.3s
+[  256.9s] tab_dann_rz_lpft+tab_dann_rz_lpft_ent n=100 rep=2 5.8s
+[  262.8s] tab_dann_rz_lpft+tab_dann_rz_lpft_ent n=100 rep=3 5.9s
+[  269.0s] tab_dann_rz_lpft+tab_dann_rz_lpft_ent n=100 rep=4 6.2s
+[  275.8s] tab_dann_rz_lpft+tab_dann_rz_lpft_ent n=200 rep=0 6.8s
+[  283.5s] tab_dann_rz_lpft+tab_dann_rz_lpft_ent n=200 rep=1 7.7s
+[  290.7s] tab_dann_rz_lpft+tab_dann_rz_lpft_ent n=200 rep=2 7.2s
+[  297.9s] tab_dann_rz_lpft+tab_dann_rz_lpft_ent n=200 rep=3 7.2s
+[  305.4s] tab_dann_rz_lpft+tab_dann_rz_lpft_ent n=200 rep=4 7.4s
+[  305.4s] done

reports/nnlab/logs/lpft_r12.log

@@ -0,0 +1,37 @@
+[    5.1s] data loaded
+[   24.3s] tab_dann_rz_lpft+tab_dann_rz_lpft_ent n=5 rep=5 12.0s
+[   29.8s] tab_dann_rz_lpft+tab_dann_rz_lpft_ent n=5 rep=6 5.5s
+[   35.1s] tab_dann_rz_lpft+tab_dann_rz_lpft_ent n=5 rep=7 5.3s
+[   39.5s] tab_dann_rz_lpft+tab_dann_rz_lpft_ent n=5 rep=8 4.4s
+[   45.0s] tab_dann_rz_lpft+tab_dann_rz_lpft_ent n=5 rep=9 5.5s
+[   50.4s] tab_dann_rz_lpft+tab_dann_rz_lpft_ent n=5 rep=10 5.4s
+[   56.7s] tab_dann_rz_lpft+tab_dann_rz_lpft_ent n=5 rep=11 6.2s
+[   62.4s] tab_dann_rz_lpft+tab_dann_rz_lpft_ent n=25 rep=5 5.7s
+[   68.9s] tab_dann_rz_lpft+tab_dann_rz_lpft_ent n=25 rep=6 6.5s
+[   74.7s] tab_dann_rz_lpft+tab_dann_rz_lpft_ent n=25 rep=7 5.8s
+[   81.0s] tab_dann_rz_lpft+tab_dann_rz_lpft_ent n=25 rep=8 6.3s
+[   87.1s] tab_dann_rz_lpft+tab_dann_rz_lpft_ent n=25 rep=9 6.1s
+[   92.5s] tab_dann_rz_lpft+tab_dann_rz_lpft_ent n=25 rep=10 5.5s
+[   98.3s] tab_dann_rz_lpft+tab_dann_rz_lpft_ent n=25 rep=11 5.7s
+[  104.4s] tab_dann_rz_lpft+tab_dann_rz_lpft_ent n=50 rep=5 6.1s
+[  110.2s] tab_dann_rz_lpft+tab_dann_rz_lpft_ent n=50 rep=6 5.8s
+[  116.5s] tab_dann_rz_lpft+tab_dann_rz_lpft_ent n=50 rep=7 6.3s
+[  122.9s] tab_dann_rz_lpft+tab_dann_rz_lpft_ent n=50 rep=8 6.3s
+[  128.9s] tab_dann_rz_lpft+tab_dann_rz_lpft_ent n=50 rep=9 6.1s
+[  134.7s] tab_dann_rz_lpft+tab_dann_rz_lpft_ent n=50 rep=10 5.8s
+[  140.6s] tab_dann_rz_lpft+tab_dann_rz_lpft_ent n=50 rep=11 5.9s
+[  147.2s] tab_dann_rz_lpft+tab_dann_rz_lpft_ent n=100 rep=5 6.6s
+[  154.2s] tab_dann_rz_lpft+tab_dann_rz_lpft_ent n=100 rep=6 7.0s
+[  160.6s] tab_dann_rz_lpft+tab_dann_rz_lpft_ent n=100 rep=7 6.4s
+[  167.1s] tab_dann_rz_lpft+tab_dann_rz_lpft_ent n=100 rep=8 6.5s
+[  174.0s] tab_dann_rz_lpft+tab_dann_rz_lpft_ent n=100 rep=9 6.9s
+[  179.6s] tab_dann_rz_lpft+tab_dann_rz_lpft_ent n=100 rep=10 5.6s
+[  186.0s] tab_dann_rz_lpft+tab_dann_rz_lpft_ent n=100 rep=11 6.4s
+[  193.7s] tab_dann_rz_lpft+tab_dann_rz_lpft_ent n=200 rep=5 7.7s
+[  201.6s] tab_dann_rz_lpft+tab_dann_rz_lpft_ent n=200 rep=6 7.8s
+[  209.9s] tab_dann_rz_lpft+tab_dann_rz_lpft_ent n=200 rep=7 8.3s
+[  217.7s] tab_dann_rz_lpft+tab_dann_rz_lpft_ent n=200 rep=8 7.9s
+[  225.3s] tab_dann_rz_lpft+tab_dann_rz_lpft_ent n=200 rep=9 7.6s
+[  233.6s] tab_dann_rz_lpft+tab_dann_rz_lpft_ent n=200 rep=10 8.2s
+[  241.3s] tab_dann_rz_lpft+tab_dann_rz_lpft_ent n=200 rep=11 7.7s
+[  241.3s] done

reports/nnlab/logs/lpft_steps.log

@@ -0,0 +1,98 @@
+[    23s] madrid steps=60 n=5 rep=0 f1=0.4753
+[    26s] madrid steps=60 n=5 rep=1 f1=0.4790
+[    28s] madrid steps=60 n=5 rep=2 f1=0.5194
+[    31s] madrid steps=60 n=50 rep=0 f1=0.5491
+[    33s] madrid steps=60 n=50 rep=1 f1=0.5527
+[    35s] madrid steps=60 n=50 rep=2 f1=0.5471
+[    38s] madrid steps=60 n=200 rep=0 f1=0.5695
+[    41s] madrid steps=60 n=200 rep=1 f1=0.5797
+[    44s] madrid steps=60 n=200 rep=2 f1=0.5713
+[    48s] madrid steps=150 n=5 rep=0 f1=0.4853
+[    53s] madrid steps=150 n=5 rep=1 f1=0.4868
+[    56s] madrid steps=150 n=5 rep=2 f1=0.5172
+[    61s] madrid steps=150 n=50 rep=0 f1=0.5500
+[    65s] madrid steps=150 n=50 rep=1 f1=0.5535
+[    70s] madrid steps=150 n=50 rep=2 f1=0.5493
+[    76s] madrid steps=150 n=200 rep=0 f1=0.5818
+[    84s] madrid steps=150 n=200 rep=1 f1=0.5931
+[    90s] madrid steps=150 n=200 rep=2 f1=0.5912
+[    98s] madrid steps=300 n=5 rep=0 f1=0.4898
+[   105s] madrid steps=300 n=5 rep=1 f1=0.4852
+[   114s] madrid steps=300 n=5 rep=2 f1=0.5120
+[   122s] madrid steps=300 n=50 rep=0 f1=0.5512
+[   130s] madrid steps=300 n=50 rep=1 f1=0.5555
+[   139s] madrid steps=300 n=50 rep=2 f1=0.5496
+[   152s] madrid steps=300 n=200 rep=0 f1=0.5823
+[   165s] madrid steps=300 n=200 rep=1 f1=0.5953
+[   177s] madrid steps=300 n=200 rep=2 f1=0.5921
+[   191s] madrid steps=600 n=5 rep=0 f1=0.4934
+[   207s] madrid steps=600 n=5 rep=1 f1=0.4865
+[   221s] madrid steps=600 n=5 rep=2 f1=0.5096
+[   239s] madrid steps=600 n=50 rep=0 f1=0.5495
+[   256s] madrid steps=600 n=50 rep=1 f1=0.5549
+[   278s] madrid steps=600 n=50 rep=2 f1=0.5508
+[   303s] madrid steps=600 n=200 rep=0 f1=0.5805
+[   327s] madrid steps=600 n=200 rep=1 f1=0.5912
+[   352s] madrid steps=600 n=200 rep=2 f1=0.5928
+madrid selection: {60: 0.5381290830815634, 150: 0.5453455986355832, 300: 0.545893932403143, 600: 0.5454710417072588} by n: {60: {5: 0.49122672993148825, 50: 0.5496386272948383, 200: 0.5735218920183637}, 150: {5: 0.49642611088751853, 50: 0.5509334248570606, 200: 0.5886772601621703}, 300: {5: 0.4956672067590944, 50: 0.5521068084327269, 200: 0.5899077820176076}, 600: {5: 0.4965136951898914, 50: 0.5517205898817409, 200: 0.5881788400501441}} -> chosen 300
+[   365s] tab_dann_rz_lpft_s300 n=5 rep=0 8.4s
+[   374s] tab_dann_rz_lpft_s300 n=5 rep=1 8.9s
+[   384s] tab_dann_rz_lpft_s300 n=5 rep=2 10.1s
+[   393s] tab_dann_rz_lpft_s300 n=5 rep=3 8.4s
+[   401s] tab_dann_rz_lpft_s300 n=5 rep=4 8.3s
+[   410s] tab_dann_rz_lpft_s300 n=5 rep=5 9.4s
+[   418s] tab_dann_rz_lpft_s300 n=5 rep=6 7.4s
+[   426s] tab_dann_rz_lpft_s300 n=5 rep=7 8.4s
+[   433s] tab_dann_rz_lpft_s300 n=5 rep=8 7.2s
+[   442s] tab_dann_rz_lpft_s300 n=5 rep=9 8.6s
+[   449s] tab_dann_rz_lpft_s300 n=5 rep=10 7.4s
+[   457s] tab_dann_rz_lpft_s300 n=5 rep=11 8.0s
+[   466s] tab_dann_rz_lpft_s300 n=25 rep=0 8.6s
+[   479s] tab_dann_rz_lpft_s300 n=25 rep=1 13.3s
+[   490s] tab_dann_rz_lpft_s300 n=25 rep=2 11.3s
+[   501s] tab_dann_rz_lpft_s300 n=25 rep=3 10.4s
+[   513s] tab_dann_rz_lpft_s300 n=25 rep=4 12.3s
+[   526s] tab_dann_rz_lpft_s300 n=25 rep=5 12.6s
+[   539s] tab_dann_rz_lpft_s300 n=25 rep=6 13.6s
+[   551s] tab_dann_rz_lpft_s300 n=25 rep=7 12.0s
+[   562s] tab_dann_rz_lpft_s300 n=25 rep=8 11.0s
+[   573s] tab_dann_rz_lpft_s300 n=25 rep=9 10.8s
+[   584s] tab_dann_rz_lpft_s300 n=25 rep=10 10.9s
+[   596s] tab_dann_rz_lpft_s300 n=25 rep=11 11.4s
+[   609s] tab_dann_rz_lpft_s300 n=50 rep=0 13.4s
+[   622s] tab_dann_rz_lpft_s300 n=50 rep=1 13.1s
+[   636s] tab_dann_rz_lpft_s300 n=50 rep=2 13.4s
+[   651s] tab_dann_rz_lpft_s300 n=50 rep=3 15.3s
+[   664s] tab_dann_rz_lpft_s300 n=50 rep=4 12.8s
+[   678s] tab_dann_rz_lpft_s300 n=50 rep=5 14.0s
+[   690s] tab_dann_rz_lpft_s300 n=50 rep=6 12.2s
+[   702s] tab_dann_rz_lpft_s300 n=50 rep=7 12.6s
+[   712s] tab_dann_rz_lpft_s300 n=50 rep=8 9.9s
+[   720s] tab_dann_rz_lpft_s300 n=50 rep=9 7.6s
+[   729s] tab_dann_rz_lpft_s300 n=50 rep=10 9.1s
+[   740s] tab_dann_rz_lpft_s300 n=50 rep=11 10.5s
+[   753s] tab_dann_rz_lpft_s300 n=100 rep=0 13.4s
+[   767s] tab_dann_rz_lpft_s300 n=100 rep=1 13.7s
+[   777s] tab_dann_rz_lpft_s300 n=100 rep=2 10.6s
+[   788s] tab_dann_rz_lpft_s300 n=100 rep=3 10.8s
+[   799s] tab_dann_rz_lpft_s300 n=100 rep=4 11.3s
+[   809s] tab_dann_rz_lpft_s300 n=100 rep=5 10.0s
+[   820s] tab_dann_rz_lpft_s300 n=100 rep=6 10.4s
+[   832s] tab_dann_rz_lpft_s300 n=100 rep=7 11.8s
+[   842s] tab_dann_rz_lpft_s300 n=100 rep=8 10.5s
+[   852s] tab_dann_rz_lpft_s300 n=100 rep=9 9.8s
+[   868s] tab_dann_rz_lpft_s300 n=100 rep=10 16.0s
+[   879s] tab_dann_rz_lpft_s300 n=100 rep=11 10.9s
+[   892s] tab_dann_rz_lpft_s300 n=200 rep=0 13.3s
+[   905s] tab_dann_rz_lpft_s300 n=200 rep=1 13.2s
+[   919s] tab_dann_rz_lpft_s300 n=200 rep=2 13.4s
+[   934s] tab_dann_rz_lpft_s300 n=200 rep=3 15.5s
+[   948s] tab_dann_rz_lpft_s300 n=200 rep=4 13.6s
+[   961s] tab_dann_rz_lpft_s300 n=200 rep=5 12.8s
+[   973s] tab_dann_rz_lpft_s300 n=200 rep=6 12.9s
+[   985s] tab_dann_rz_lpft_s300 n=200 rep=7 12.0s
+[   999s] tab_dann_rz_lpft_s300 n=200 rep=8 13.0s
+[  1012s] tab_dann_rz_lpft_s300 n=200 rep=9 13.4s
+[  1024s] tab_dann_rz_lpft_s300 n=200 rep=10 11.9s
+[  1037s] tab_dann_rz_lpft_s300 n=200 rep=11 13.5s
+done

reports/nnlab/logs/metric.log

@@ -0,0 +1,138 @@
+[    5.2s] loaded Madrid (76263, 393) Amsterdam (25992, 393); encoders ['supcon', 'proto', 'protoda', 'cos']
+[   18.0s] supcon: selecting length on Madrid split (src 61932, pseudo-target 14331) from [300, 700, 1500]
+[   45.0s]   supcon seed100 step 300: loss 5.9978 pseudo-target F1 0.5113 (20s)
+[   60.5s]   supcon seed100 step 700: loss 5.9759 pseudo-target F1 0.5227 (36s)
+[   99.1s]   supcon seed100 step 1500: loss 5.9317 pseudo-target F1 0.5148 (75s)
+[   99.1s] supcon: chose 700 steps
+[   99.1s] supcon: training seed 0 on all Madrid for 700 steps
+[  131.0s]   supcon seed0 step 700: loss 5.9621 (32s)
+[  131.8s] supcon: training seed 1 on all Madrid for 700 steps
+[  180.3s]   supcon seed1 step 700: loss 6.0183 (48s)
+[  186.0s] supcon n=5 rep=0: saved 5 heads (3.5s)
+[  189.5s] supcon n=5 rep=1: saved 5 heads (3.5s)
+[  192.9s] supcon n=5 rep=2: saved 5 heads (3.4s)
+[  196.3s] supcon n=5 rep=3: saved 5 heads (3.4s)
+[  199.6s] supcon n=5 rep=4: saved 5 heads (3.2s)
+[  203.3s] supcon n=25 rep=0: saved 5 heads (3.8s)
+[  207.1s] supcon n=25 rep=1: saved 5 heads (3.7s)
+[  210.8s] supcon n=25 rep=2: saved 5 heads (3.7s)
+[  214.1s] supcon n=25 rep=3: saved 5 heads (3.3s)
+[  216.2s] supcon n=25 rep=4: saved 5 heads (2.1s)
+[  220.1s] supcon n=50 rep=0: saved 5 heads (3.9s)
+[  224.4s] supcon n=50 rep=1: saved 5 heads (4.3s)
+[  228.8s] supcon n=50 rep=2: saved 5 heads (4.4s)
+[  232.9s] supcon n=50 rep=3: saved 5 heads (4.0s)
+[  237.1s] supcon n=50 rep=4: saved 5 heads (4.2s)
+[  242.2s] supcon n=100 rep=0: saved 5 heads (5.0s)
+[  246.7s] supcon n=100 rep=1: saved 5 heads (4.5s)
+[  251.6s] supcon n=100 rep=2: saved 5 heads (4.9s)
+[  256.3s] supcon n=100 rep=3: saved 5 heads (4.7s)
+[  260.7s] supcon n=100 rep=4: saved 5 heads (4.4s)
+[  265.4s] supcon n=200 rep=0: saved 5 heads (4.8s)
+[  271.0s] supcon n=200 rep=1: saved 5 heads (5.6s)
+[  276.5s] supcon n=200 rep=2: saved 5 heads (5.5s)
+[  279.4s] supcon n=200 rep=3: saved 5 heads (2.8s)
+[  284.6s] supcon n=200 rep=4: saved 5 heads (5.2s)
+[  288.8s] proto: selecting length on Madrid split (src 61932, pseudo-target 14331) from [400, 800, 1600]
+[  301.0s]   proto seed100 step 400: loss 0.8636 pseudo-target F1 0.4932 (12s)
+[  311.7s]   proto seed100 step 800: loss 0.9729 pseudo-target F1 0.5287 (23s)
+[  335.6s]   proto seed100 step 1600: loss 0.8058 pseudo-target F1 0.5097 (47s)
+[  335.6s] proto: chose 800 steps
+[  335.6s] proto: training seed 0 on all Madrid for 800 steps
+[  356.8s]   proto seed0 step 800: loss 0.9539 (21s)
+[  358.0s] proto: training seed 1 on all Madrid for 800 steps
+[  386.6s]   proto seed1 step 800: loss 1.0048 (29s)
+[  393.0s] proto n=5 rep=0: saved 5 heads (4.1s)
+[  396.3s] proto n=5 rep=1: saved 5 heads (3.3s)
+[  399.5s] proto n=5 rep=2: saved 5 heads (3.2s)
+[  402.8s] proto n=5 rep=3: saved 5 heads (3.3s)
+[  406.4s] proto n=5 rep=4: saved 5 heads (3.6s)
+[  410.5s] proto n=25 rep=0: saved 5 heads (4.1s)
+[  414.3s] proto n=25 rep=1: saved 5 heads (3.8s)
+[  418.1s] proto n=25 rep=2: saved 5 heads (3.8s)
+[  421.8s] proto n=25 rep=3: saved 5 heads (3.7s)
+[  424.7s] proto n=25 rep=4: saved 5 heads (2.9s)
+[  426.9s] proto n=50 rep=0: saved 5 heads (2.2s)
+[  430.1s] proto n=50 rep=1: saved 5 heads (3.2s)
+[  434.6s] proto n=50 rep=2: saved 5 heads (4.4s)
+[  439.2s] proto n=50 rep=3: saved 5 heads (4.7s)
+[  443.5s] proto n=50 rep=4: saved 5 heads (4.2s)
+[  448.3s] proto n=100 rep=0: saved 5 heads (4.8s)
+[  453.2s] proto n=100 rep=1: saved 5 heads (4.9s)
+[  458.0s] proto n=100 rep=2: saved 5 heads (4.8s)
+[  462.8s] proto n=100 rep=3: saved 5 heads (4.8s)
+[  467.5s] proto n=100 rep=4: saved 5 heads (4.6s)
+[  472.8s] proto n=200 rep=0: saved 5 heads (5.3s)
+[  478.2s] proto n=200 rep=1: saved 5 heads (5.4s)
+[  483.6s] proto n=200 rep=2: saved 5 heads (5.4s)
+[  487.9s] proto n=200 rep=3: saved 5 heads (4.3s)
+[  490.6s] proto n=200 rep=4: saved 5 heads (2.7s)
+[  495.0s] protoda: selecting length on Madrid split (src 61932, pseudo-target 14331) from [400, 800, 1600]
+[  513.7s]   protoda seed100 step 400: loss 0.9048 pseudo-target F1 0.5346 (19s)
+[  533.2s]   protoda seed100 step 800: loss 0.9346 pseudo-target F1 0.5117 (38s)
+[  567.8s]   protoda seed100 step 1600: loss 0.8521 pseudo-target F1 0.5270 (73s)
+[  567.8s] protoda: chose 400 steps
+[  567.8s] protoda: training seed 0 on all Madrid for 400 steps
+[  587.5s]   protoda seed0 step 400: loss 1.0030 (20s)
+[  588.5s] protoda: training seed 1 on all Madrid for 400 steps
+[  603.6s]   protoda seed1 step 400: loss 1.0488 (15s)
+[  609.0s] protoda n=5 rep=0: saved 5 heads (3.4s)
+[  612.2s] protoda n=5 rep=1: saved 5 heads (3.1s)
+[  615.3s] protoda n=5 rep=2: saved 5 heads (3.1s)
+[  618.7s] protoda n=5 rep=3: saved 5 heads (3.4s)
+[  621.8s] protoda n=5 rep=4: saved 5 heads (3.2s)
+[  625.6s] protoda n=25 rep=0: saved 5 heads (3.8s)
+[  629.3s] protoda n=25 rep=1: saved 5 heads (3.7s)
+[  633.1s] protoda n=25 rep=2: saved 5 heads (3.8s)
+[  636.5s] protoda n=25 rep=3: saved 5 heads (3.4s)
+[  639.9s] protoda n=25 rep=4: saved 5 heads (3.4s)
+[  644.4s] protoda n=50 rep=0: saved 5 heads (4.5s)
+[  648.7s] protoda n=50 rep=1: saved 5 heads (4.3s)
+[  652.6s] protoda n=50 rep=2: saved 5 heads (3.9s)
+[  654.9s] protoda n=50 rep=3: saved 5 heads (2.4s)
+[  657.6s] protoda n=50 rep=4: saved 5 heads (2.6s)
+[  661.9s] protoda n=100 rep=0: saved 5 heads (4.3s)
+[  666.7s] protoda n=100 rep=1: saved 5 heads (4.8s)
+[  671.6s] protoda n=100 rep=2: saved 5 heads (4.9s)
+[  676.2s] protoda n=100 rep=3: saved 5 heads (4.6s)
+[  680.8s] protoda n=100 rep=4: saved 5 heads (4.7s)
+[  685.9s] protoda n=200 rep=0: saved 5 heads (5.0s)
+[  691.2s] protoda n=200 rep=1: saved 5 heads (5.3s)
+[  696.8s] protoda n=200 rep=2: saved 5 heads (5.5s)
+[  701.6s] protoda n=200 rep=3: saved 5 heads (4.8s)
+[  706.9s] protoda n=200 rep=4: saved 5 heads (5.3s)
+[  711.3s] cos: selecting length on Madrid split (src 61932, pseudo-target 14331) from [300, 700, 1500]
+[  717.6s]   cos seed100 step 300: loss 0.8239 pseudo-target F1 0.5167 (6s)
+[  731.6s]   cos seed100 step 700: loss 0.8426 pseudo-target F1 0.5191 (20s)
+[  760.8s]   cos seed100 step 1500: loss 0.7345 pseudo-target F1 0.5268 (50s)
+[  760.8s] cos: chose 1500 steps
+[  760.8s] cos: training seed 0 on all Madrid for 1500 steps
+[  814.0s]   cos seed0 step 1500: loss 0.7407 (53s)
+[  815.2s] cos: training seed 1 on all Madrid for 1500 steps
+[  865.5s]   cos seed1 step 1500: loss 0.7244 (50s)
+[  870.7s] cos n=5 rep=0: saved 5 heads (3.0s)
+[  873.8s] cos n=5 rep=1: saved 5 heads (3.1s)
+[  876.9s] cos n=5 rep=2: saved 5 heads (3.0s)
+[  879.9s] cos n=5 rep=3: saved 5 heads (3.0s)
+[  882.9s] cos n=5 rep=4: saved 5 heads (3.0s)
+[  886.5s] cos n=25 rep=0: saved 5 heads (3.6s)
+[  888.2s] cos n=25 rep=1: saved 5 heads (1.7s)
+[  890.3s] cos n=25 rep=2: saved 5 heads (2.1s)
+[  893.6s] cos n=25 rep=3: saved 5 heads (3.3s)
+[  897.0s] cos n=25 rep=4: saved 5 heads (3.5s)
+[  901.1s] cos n=50 rep=0: saved 5 heads (4.1s)
+[  905.3s] cos n=50 rep=1: saved 5 heads (4.1s)
+[  909.0s] cos n=50 rep=2: saved 5 heads (3.8s)
+[  913.3s] cos n=50 rep=3: saved 5 heads (4.2s)
+[  918.7s] cos n=50 rep=4: saved 5 heads (5.4s)
+[  925.9s] cos n=100 rep=0: saved 5 heads (7.1s)
+[  933.4s] cos n=100 rep=1: saved 5 heads (7.6s)
+[  938.9s] cos n=100 rep=2: saved 5 heads (5.4s)
+[  944.6s] cos n=100 rep=3: saved 5 heads (5.8s)
+[  950.2s] cos n=100 rep=4: saved 5 heads (5.5s)
+[  956.7s] cos n=200 rep=0: saved 5 heads (6.5s)
+[  964.2s] cos n=200 rep=1: saved 5 heads (7.5s)
+[  971.7s] cos n=200 rep=2: saved 5 heads (7.5s)
+[  978.4s] cos n=200 rep=3: saved 5 heads (6.7s)
+[  984.5s] cos n=200 rep=4: saved 5 heads (6.1s)
+[  984.5s] done

reports/nnlab/logs/metric_r12.log

@@ -0,0 +1,74 @@
+[    5.5s] loaded Madrid (76263, 393) Amsterdam (25992, 393); encoders ['protoda']
+[   27.2s] protoda n=5 rep=5: saved 5 heads (11.8s)
+[   30.7s] protoda n=5 rep=6: saved 5 heads (3.5s)
+[   34.1s] protoda n=5 rep=7: saved 5 heads (3.4s)
+[   37.4s] protoda n=5 rep=8: saved 5 heads (3.4s)
+[   40.9s] protoda n=5 rep=9: saved 5 heads (3.4s)
+[   44.2s] protoda n=5 rep=10: saved 5 heads (3.3s)
+[   47.5s] protoda n=5 rep=11: saved 5 heads (3.3s)
+[   51.5s] protoda n=25 rep=5: saved 5 heads (3.9s)
+[   55.5s] protoda n=25 rep=6: saved 5 heads (4.0s)
+[   59.6s] protoda n=25 rep=7: saved 5 heads (4.1s)
+[   63.3s] protoda n=25 rep=8: saved 5 heads (3.7s)
+[   67.5s] protoda n=25 rep=9: saved 5 heads (4.2s)
+[   71.3s] protoda n=25 rep=10: saved 5 heads (3.9s)
+[   75.3s] protoda n=25 rep=11: saved 5 heads (4.0s)
+[   79.7s] protoda n=50 rep=5: saved 5 heads (4.4s)
+[   84.8s] protoda n=50 rep=6: saved 5 heads (5.1s)
+[   87.4s] protoda n=50 rep=7: saved 5 heads (2.6s)
+[   90.9s] protoda n=50 rep=8: saved 5 heads (3.5s)
+[   95.2s] protoda n=50 rep=9: saved 5 heads (4.3s)
+[   99.6s] protoda n=50 rep=10: saved 5 heads (4.4s)
+[  105.8s] protoda n=50 rep=11: saved 5 heads (6.1s)
+[  111.8s] protoda n=100 rep=5: saved 5 heads (6.0s)
+[  118.1s] protoda n=100 rep=6: saved 5 heads (6.3s)
+[  123.2s] protoda n=100 rep=7: saved 5 heads (5.2s)
+[  129.6s] protoda n=100 rep=8: saved 5 heads (6.4s)
+[  136.5s] protoda n=100 rep=9: saved 5 heads (6.9s)
+[  143.3s] protoda n=100 rep=10: saved 5 heads (6.9s)
+[  148.7s] protoda n=100 rep=11: saved 5 heads (5.4s)
+[  154.8s] protoda n=200 rep=5: saved 5 heads (6.1s)
+[  161.0s] protoda n=200 rep=6: saved 5 heads (6.2s)
+[  166.6s] protoda n=200 rep=7: saved 5 heads (5.6s)
+[  172.4s] protoda n=200 rep=8: saved 5 heads (5.8s)
+[  178.0s] protoda n=200 rep=9: saved 5 heads (5.6s)
+[  183.7s] protoda n=200 rep=10: saved 5 heads (5.6s)
+[  186.3s] protoda n=200 rep=11: saved 5 heads (2.6s)
+[  186.3s] done
+[    4.6s] loaded Madrid (76263, 393) Amsterdam (25992, 393); encoders ['cos']
+[   25.9s] cos n=5 rep=5: saved 5 heads (10.9s)
+[   29.4s] cos n=5 rep=6: saved 5 heads (3.6s)
+[   32.9s] cos n=5 rep=7: saved 5 heads (3.5s)
+[   35.9s] cos n=5 rep=8: saved 5 heads (3.0s)
+[   39.1s] cos n=5 rep=9: saved 5 heads (3.2s)
+[   42.2s] cos n=5 rep=10: saved 5 heads (3.0s)
+[   45.1s] cos n=5 rep=11: saved 5 heads (3.0s)
+[   49.1s] cos n=25 rep=5: saved 5 heads (3.9s)
+[   52.9s] cos n=25 rep=6: saved 5 heads (3.8s)
+[   56.6s] cos n=25 rep=7: saved 5 heads (3.7s)
+[   60.2s] cos n=25 rep=8: saved 5 heads (3.6s)
+[   62.8s] cos n=25 rep=9: saved 5 heads (2.6s)
+[   64.7s] cos n=25 rep=10: saved 5 heads (1.9s)
+[   67.7s] cos n=25 rep=11: saved 5 heads (3.0s)
+[   72.0s] cos n=50 rep=5: saved 5 heads (4.3s)
+[   76.4s] cos n=50 rep=6: saved 5 heads (4.4s)
+[   80.7s] cos n=50 rep=7: saved 5 heads (4.3s)
+[   85.1s] cos n=50 rep=8: saved 5 heads (4.4s)
+[   89.7s] cos n=50 rep=9: saved 5 heads (4.6s)
+[   94.2s] cos n=50 rep=10: saved 5 heads (4.5s)
+[   98.4s] cos n=50 rep=11: saved 5 heads (4.2s)
+[  104.9s] cos n=100 rep=5: saved 5 heads (6.5s)
+[  109.6s] cos n=100 rep=6: saved 5 heads (4.7s)
+[  114.1s] cos n=100 rep=7: saved 5 heads (4.5s)
+[  118.7s] cos n=100 rep=8: saved 5 heads (4.6s)
+[  123.4s] cos n=100 rep=9: saved 5 heads (4.7s)
+[  126.3s] cos n=100 rep=10: saved 5 heads (2.9s)
+[  128.7s] cos n=100 rep=11: saved 5 heads (2.3s)
+[  134.0s] cos n=200 rep=5: saved 5 heads (5.3s)
+[  139.3s] cos n=200 rep=6: saved 5 heads (5.3s)
+[  144.9s] cos n=200 rep=7: saved 5 heads (5.6s)
+[  150.3s] cos n=200 rep=8: saved 5 heads (5.4s)
+[  155.6s] cos n=200 rep=9: saved 5 heads (5.3s)
+[  160.7s] cos n=200 rep=10: saved 5 heads (5.1s)
+[  165.9s] cos n=200 rep=11: saved 5 heads (5.2s)
+[  165.9s] done

reports/nnlab/logs/st_e1lp.log

@@ -0,0 +1,28 @@
+[    0.7s] loaded: Madrid (76263, 393), Amsterdam (25992, 393)
+[    0.8s] graph: 583310 edges, mean degree 22.4, min 10
+[   82.5s] ens_e1__lp__st n=5 rep=0 base=ens_e1__lp pseudo=[1199, 1230, 724, 742] done in 81.5s
+[  152.1s] ens_e1__lp__st n=5 rep=1 base=ens_e1__lp pseudo=[1182, 1356, 664, 694] done in 69.7s
+[  222.9s] ens_e1__lp__st n=5 rep=2 base=ens_e1__lp pseudo=[1206, 1288, 835, 566] done in 70.8s
+[  295.3s] ens_e1__lp__st n=5 rep=3 base=ens_e1__lp pseudo=[1059, 1467, 662, 708] done in 72.4s
+[  371.9s] ens_e1__lp__st n=5 rep=4 base=ens_e1__lp pseudo=[1274, 1299, 676, 648] done in 76.6s
+[  447.5s] ens_e1__lp__st n=25 rep=0 base=ens_e1__lp pseudo=[1297, 1168, 832, 587] done in 75.6s
+[  517.2s] ens_e1__lp__st n=25 rep=1 base=ens_e1__lp pseudo=[1338, 997, 858, 691] done in 69.7s
+[  585.8s] ens_e1__lp__st n=25 rep=2 base=ens_e1__lp pseudo=[1526, 875, 756, 727] done in 68.6s
+[  655.0s] ens_e1__lp__st n=25 rep=3 base=ens_e1__lp pseudo=[1301, 1060, 807, 715] done in 69.2s
+[  724.6s] ens_e1__lp__st n=25 rep=4 base=ens_e1__lp pseudo=[1261, 1272, 755, 596] done in 69.6s
+[  799.3s] ens_e1__lp__st n=50 rep=0 base=ens_e1__lp pseudo=[1164, 1236, 906, 562] done in 74.7s
+[  871.5s] ens_e1__lp__st n=50 rep=1 base=ens_e1__lp pseudo=[1232, 1152, 908, 578] done in 72.2s
+[  942.0s] ens_e1__lp__st n=50 rep=2 base=ens_e1__lp pseudo=[1292, 1166, 749, 662] done in 70.5s
+[ 1016.6s] ens_e1__lp__st n=50 rep=3 base=ens_e1__lp pseudo=[1311, 1163, 810, 584] done in 74.6s
+[ 1087.1s] ens_e1__lp__st n=50 rep=4 base=ens_e1__lp pseudo=[1188, 1253, 795, 633] done in 70.5s
+[ 1157.9s] ens_e1__lp__st n=100 rep=0 base=ens_e1__lp pseudo=[1331, 1121, 788, 598] done in 70.8s
+[ 1230.3s] ens_e1__lp__st n=100 rep=1 base=ens_e1__lp pseudo=[1326, 1093, 809, 611] done in 72.4s
+[ 1300.3s] ens_e1__lp__st n=100 rep=2 base=ens_e1__lp pseudo=[1285, 1093, 895, 565] done in 70.0s
+[ 1374.4s] ens_e1__lp__st n=100 rep=3 base=ens_e1__lp pseudo=[1372, 1030, 830, 606] done in 74.2s
+[ 1448.0s] ens_e1__lp__st n=100 rep=4 base=ens_e1__lp pseudo=[1124, 1267, 907, 541] done in 73.5s
+[ 1518.4s] ens_e1__lp__st n=200 rep=0 base=ens_e1__lp pseudo=[1210, 1174, 873, 522] done in 70.4s
+[ 1590.9s] ens_e1__lp__st n=200 rep=1 base=ens_e1__lp pseudo=[1300, 1031, 876, 572] done in 72.6s
+[ 1659.8s] ens_e1__lp__st n=200 rep=2 base=ens_e1__lp pseudo=[1143, 1210, 863, 563] done in 68.8s
+[ 1728.9s] ens_e1__lp__st n=200 rep=3 base=ens_e1__lp pseudo=[1247, 1148, 811, 573] done in 69.1s
+[ 1800.7s] ens_e1__lp__st n=200 rep=4 base=ens_e1__lp pseudo=[1240, 1154, 868, 517] done in 71.9s
+[ 1800.7s] done

reports/nnlab/logs/tab_nn.log

@@ -0,0 +1,777 @@
+[    1.2s] data loaded
+[  257.4s] select epochs mlp: pseudo-target rule F1 {15: 0.4999554636261916, 30: 0.5049793102098873, 45: 0.5036779222509986} -> 30  (251s)
+[  522.5s] select epochs res: pseudo-target rule F1 {15: 0.5075904273140687, 30: 0.49876532366138193, 45: 0.48974432628349995} -> 15  (265s)
+[  995.5s] select epochs dann: pseudo-target rule F1 {15: 0.47348192372436493, 30: 0.4588599874942516, 45: 0.4520852473361714} -> 15  (473s)
+[ 1254.3s] select epochs coral: pseudo-target rule F1 {15: 0.5034106048326475, 30: 0.49598761101812194, 45: 0.5052691962026917} -> 45  (259s)
+[ 1288.5s] select lam: {'0.1': 0.47686438636808737, '1.0': 0.5023598505755028, '10.0': 0.5164665414687039, '100.0': 0.5110354354278966} -> 10.0  (34s)
+[ 1398.7s] train mlp/rz seed 0: 30 epochs  106s
+[ 1398.8s] tab_mlp_rz_zs n=0 saved
+[ 1398.9s] tab_mlp_rz_ft n=5 rep=0 0.1s
+[ 1407.0s] tab_mlp_rz_joint n=5 rep=0 8.2s
+[ 1407.2s] tab_mlp_rz_proto n=5 rep=0 0.2s
+[ 1407.3s] tab_mlp_rz_ft n=5 rep=1 0.1s
+[ 1410.0s] tab_mlp_rz_joint n=5 rep=1 2.7s
+[ 1410.1s] tab_mlp_rz_proto n=5 rep=1 0.1s
+[ 1410.2s] tab_mlp_rz_ft n=5 rep=2 0.1s
+[ 1411.6s] tab_mlp_rz_joint n=5 rep=2 1.4s
+[ 1411.7s] tab_mlp_rz_proto n=5 rep=2 0.1s
+[ 1411.8s] tab_mlp_rz_ft n=5 rep=3 0.1s
+[ 1413.8s] tab_mlp_rz_joint n=5 rep=3 2.1s
+[ 1413.9s] tab_mlp_rz_proto n=5 rep=3 0.1s
+[ 1414.0s] tab_mlp_rz_ft n=5 rep=4 0.1s
+[ 1416.6s] tab_mlp_rz_joint n=5 rep=4 2.6s
+[ 1416.7s] tab_mlp_rz_proto n=5 rep=4 0.1s
+[ 1416.7s] tab_mlp_rz_ft n=25 rep=0 0.1s
+[ 1418.9s] tab_mlp_rz_joint n=25 rep=0 2.2s
+[ 1419.0s] tab_mlp_rz_proto n=25 rep=0 0.1s
+[ 1419.1s] tab_mlp_rz_ft n=25 rep=1 0.1s
+[ 1421.4s] tab_mlp_rz_joint n=25 rep=1 2.4s
+[ 1421.5s] tab_mlp_rz_proto n=25 rep=1 0.1s
+[ 1421.6s] tab_mlp_rz_ft n=25 rep=2 0.1s
+[ 1424.4s] tab_mlp_rz_joint n=25 rep=2 2.7s
+[ 1424.4s] tab_mlp_rz_proto n=25 rep=2 0.1s
+[ 1424.5s] tab_mlp_rz_ft n=25 rep=3 0.1s
+[ 1428.1s] tab_mlp_rz_joint n=25 rep=3 3.6s
+[ 1428.2s] tab_mlp_rz_proto n=25 rep=3 0.1s
+[ 1428.3s] tab_mlp_rz_ft n=25 rep=4 0.1s
+[ 1431.1s] tab_mlp_rz_joint n=25 rep=4 2.8s
+[ 1431.2s] tab_mlp_rz_proto n=25 rep=4 0.1s
+[ 1431.3s] tab_mlp_rz_ft n=50 rep=0 0.1s
+[ 1434.9s] tab_mlp_rz_joint n=50 rep=0 3.5s
+[ 1435.0s] tab_mlp_rz_proto n=50 rep=0 0.1s
+[ 1435.1s] tab_mlp_rz_ft n=50 rep=1 0.1s
+[ 1437.2s] tab_mlp_rz_joint n=50 rep=1 2.1s
+[ 1437.3s] tab_mlp_rz_proto n=50 rep=1 0.1s
+[ 1437.4s] tab_mlp_rz_ft n=50 rep=2 0.1s
+[ 1440.9s] tab_mlp_rz_joint n=50 rep=2 3.5s
+[ 1441.0s] tab_mlp_rz_proto n=50 rep=2 0.1s
+[ 1441.1s] tab_mlp_rz_ft n=50 rep=3 0.2s
+[ 1444.3s] tab_mlp_rz_joint n=50 rep=3 3.2s
+[ 1444.5s] tab_mlp_rz_proto n=50 rep=3 0.2s
+[ 1446.0s] tab_mlp_rz_ft n=50 rep=4 1.5s
+[ 1451.6s] tab_mlp_rz_joint n=50 rep=4 5.6s
+[ 1451.7s] tab_mlp_rz_proto n=50 rep=4 0.1s
+[ 1451.9s] tab_mlp_rz_ft n=100 rep=0 0.3s
+[ 1456.8s] tab_mlp_rz_joint n=100 rep=0 4.8s
+[ 1456.9s] tab_mlp_rz_proto n=100 rep=0 0.1s
+[ 1457.1s] tab_mlp_rz_ft n=100 rep=1 0.2s
+[ 1461.2s] tab_mlp_rz_joint n=100 rep=1 4.1s
+[ 1461.3s] tab_mlp_rz_proto n=100 rep=1 0.1s
+[ 1461.5s] tab_mlp_rz_ft n=100 rep=2 0.2s
+[ 1467.0s] tab_mlp_rz_joint n=100 rep=2 5.5s
+[ 1467.1s] tab_mlp_rz_proto n=100 rep=2 0.1s
+[ 1467.3s] tab_mlp_rz_ft n=100 rep=3 0.3s
+[ 1471.1s] tab_mlp_rz_joint n=100 rep=3 3.8s
+[ 1471.2s] tab_mlp_rz_proto n=100 rep=3 0.1s
+[ 1471.4s] tab_mlp_rz_ft n=100 rep=4 0.2s
+[ 1475.7s] tab_mlp_rz_joint n=100 rep=4 4.3s
+[ 1475.8s] tab_mlp_rz_proto n=100 rep=4 0.1s
+[ 1475.9s] tab_mlp_rz_ft n=200 rep=0 0.2s
+[ 1479.9s] tab_mlp_rz_joint n=200 rep=0 3.9s
+[ 1479.9s] tab_mlp_rz_proto n=200 rep=0 0.1s
+[ 1480.2s] tab_mlp_rz_ft n=200 rep=1 0.2s
+[ 1484.5s] tab_mlp_rz_joint n=200 rep=1 4.3s
+[ 1484.6s] tab_mlp_rz_proto n=200 rep=1 0.1s
+[ 1484.8s] tab_mlp_rz_ft n=200 rep=2 0.2s
+[ 1488.5s] tab_mlp_rz_joint n=200 rep=2 3.7s
+[ 1488.6s] tab_mlp_rz_proto n=200 rep=2 0.1s
+[ 1488.8s] tab_mlp_rz_ft n=200 rep=3 0.2s
+[ 1492.7s] tab_mlp_rz_joint n=200 rep=3 4.0s
+[ 1492.8s] tab_mlp_rz_proto n=200 rep=3 0.1s
+[ 1492.9s] tab_mlp_rz_ft n=200 rep=4 0.1s
+[ 1496.5s] tab_mlp_rz_joint n=200 rep=4 3.5s
+[ 1496.5s] tab_mlp_rz_proto n=200 rep=4 0.1s
+[ 1619.4s] train mlp/rg seed 0: 30 epochs  112s
+[ 1619.5s] tab_mlp_rg_zs n=0 saved
+[ 1619.6s] tab_mlp_rg_ft n=5 rep=0 0.1s
+[ 1626.5s] tab_mlp_rg_joint n=5 rep=0 6.9s
+[ 1626.7s] tab_mlp_rg_proto n=5 rep=0 0.2s
+[ 1626.8s] tab_mlp_rg_ft n=5 rep=1 0.1s
+[ 1629.8s] tab_mlp_rg_joint n=5 rep=1 3.0s
+[ 1629.9s] tab_mlp_rg_proto n=5 rep=1 0.1s
+[ 1630.0s] tab_mlp_rg_ft n=5 rep=2 0.1s
+[ 1631.6s] tab_mlp_rg_joint n=5 rep=2 1.6s
+[ 1631.7s] tab_mlp_rg_proto n=5 rep=2 0.1s
+[ 1631.8s] tab_mlp_rg_ft n=5 rep=3 0.1s
+[ 1633.2s] tab_mlp_rg_joint n=5 rep=3 1.4s
+[ 1633.2s] tab_mlp_rg_proto n=5 rep=3 0.1s
+[ 1633.3s] tab_mlp_rg_ft n=5 rep=4 0.1s
+[ 1634.6s] tab_mlp_rg_joint n=5 rep=4 1.2s
+[ 1634.6s] tab_mlp_rg_proto n=5 rep=4 0.1s
+[ 1634.7s] tab_mlp_rg_ft n=25 rep=0 0.1s
+[ 1636.5s] tab_mlp_rg_joint n=25 rep=0 1.7s
+[ 1636.5s] tab_mlp_rg_proto n=25 rep=0 0.1s
+[ 1636.6s] tab_mlp_rg_ft n=25 rep=1 0.1s
+[ 1639.9s] tab_mlp_rg_joint n=25 rep=1 3.2s
+[ 1639.9s] tab_mlp_rg_proto n=25 rep=1 0.1s
+[ 1640.1s] tab_mlp_rg_ft n=25 rep=2 0.1s
+[ 1642.3s] tab_mlp_rg_joint n=25 rep=2 2.2s
+[ 1642.4s] tab_mlp_rg_proto n=25 rep=2 0.1s
+[ 1642.6s] tab_mlp_rg_ft n=25 rep=3 0.2s
+[ 1650.3s] tab_mlp_rg_joint n=25 rep=3 7.8s
+[ 1650.4s] tab_mlp_rg_proto n=25 rep=3 0.1s
+[ 1650.5s] tab_mlp_rg_ft n=25 rep=4 0.1s
+[ 1652.7s] tab_mlp_rg_joint n=25 rep=4 2.2s
+[ 1652.8s] tab_mlp_rg_proto n=25 rep=4 0.1s
+[ 1652.9s] tab_mlp_rg_ft n=50 rep=0 0.1s
+[ 1655.7s] tab_mlp_rg_joint n=50 rep=0 2.8s
+[ 1655.8s] tab_mlp_rg_proto n=50 rep=0 0.1s
+[ 1655.9s] tab_mlp_rg_ft n=50 rep=1 0.1s
+[ 1662.7s] tab_mlp_rg_joint n=50 rep=1 6.8s
+[ 1662.8s] tab_mlp_rg_proto n=50 rep=1 0.1s
+[ 1662.9s] tab_mlp_rg_ft n=50 rep=2 0.1s
+[ 1667.1s] tab_mlp_rg_joint n=50 rep=2 4.1s
+[ 1667.2s] tab_mlp_rg_proto n=50 rep=2 0.1s
+[ 1667.3s] tab_mlp_rg_ft n=50 rep=3 0.1s
+[ 1670.9s] tab_mlp_rg_joint n=50 rep=3 3.6s
+[ 1671.0s] tab_mlp_rg_proto n=50 rep=3 0.1s
+[ 1671.1s] tab_mlp_rg_ft n=50 rep=4 0.1s
+[ 1674.0s] tab_mlp_rg_joint n=50 rep=4 2.9s
+[ 1674.1s] tab_mlp_rg_proto n=50 rep=4 0.1s
+[ 1674.2s] tab_mlp_rg_ft n=100 rep=0 0.1s
+[ 1678.6s] tab_mlp_rg_joint n=100 rep=0 4.4s
+[ 1678.7s] tab_mlp_rg_proto n=100 rep=0 0.1s
+[ 1678.9s] tab_mlp_rg_ft n=100 rep=1 0.1s
+[ 1682.4s] tab_mlp_rg_joint n=100 rep=1 3.5s
+[ 1682.5s] tab_mlp_rg_proto n=100 rep=1 0.1s
+[ 1682.6s] tab_mlp_rg_ft n=100 rep=2 0.1s
+[ 1686.1s] tab_mlp_rg_joint n=100 rep=2 3.5s
+[ 1686.2s] tab_mlp_rg_proto n=100 rep=2 0.1s
+[ 1686.4s] tab_mlp_rg_ft n=100 rep=3 0.2s
+[ 1690.6s] tab_mlp_rg_joint n=100 rep=3 4.2s
+[ 1690.7s] tab_mlp_rg_proto n=100 rep=3 0.1s
+[ 1690.8s] tab_mlp_rg_ft n=100 rep=4 0.1s
+[ 1694.3s] tab_mlp_rg_joint n=100 rep=4 3.5s
+[ 1694.4s] tab_mlp_rg_proto n=100 rep=4 0.1s
+[ 1694.6s] tab_mlp_rg_ft n=200 rep=0 0.2s
+[ 1698.9s] tab_mlp_rg_joint n=200 rep=0 4.3s
+[ 1699.0s] tab_mlp_rg_proto n=200 rep=0 0.1s
+[ 1699.2s] tab_mlp_rg_ft n=200 rep=1 0.2s
+[ 1703.6s] tab_mlp_rg_joint n=200 rep=1 4.3s
+[ 1703.7s] tab_mlp_rg_proto n=200 rep=1 0.1s
+[ 1703.9s] tab_mlp_rg_ft n=200 rep=2 0.2s
+[ 1707.4s] tab_mlp_rg_joint n=200 rep=2 3.5s
+[ 1707.4s] tab_mlp_rg_proto n=200 rep=2 0.1s
+[ 1707.6s] tab_mlp_rg_ft n=200 rep=3 0.2s
+[ 1720.3s] tab_mlp_rg_joint n=200 rep=3 12.7s
+[ 1720.4s] tab_mlp_rg_proto n=200 rep=3 0.1s
+[ 1720.6s] tab_mlp_rg_ft n=200 rep=4 0.2s
+[ 1734.0s] tab_mlp_rg_joint n=200 rep=4 13.4s
+[ 1734.1s] tab_mlp_rg_proto n=200 rep=4 0.1s
+[ 1824.2s] train mlp/o60 seed 0: 30 epochs  90s
+[ 1824.3s] tab_mlp_o60_zs n=0 saved
+[ 1824.4s] tab_mlp_o60_ft n=5 rep=0 0.1s
+[ 1830.9s] tab_mlp_o60_joint n=5 rep=0 6.6s
+[ 1831.2s] tab_mlp_o60_proto n=5 rep=0 0.3s
+[ 1831.3s] tab_mlp_o60_ft n=5 rep=1 0.1s
+[ 1832.9s] tab_mlp_o60_joint n=5 rep=1 1.6s
+[ 1833.0s] tab_mlp_o60_proto n=5 rep=1 0.1s
+[ 1833.1s] tab_mlp_o60_ft n=5 rep=2 0.1s
+[ 1834.6s] tab_mlp_o60_joint n=5 rep=2 1.5s
+[ 1834.7s] tab_mlp_o60_proto n=5 rep=2 0.1s
+[ 1834.8s] tab_mlp_o60_ft n=5 rep=3 0.1s
+[ 1836.3s] tab_mlp_o60_joint n=5 rep=3 1.6s
+[ 1836.4s] tab_mlp_o60_proto n=5 rep=3 0.1s
+[ 1836.5s] tab_mlp_o60_ft n=5 rep=4 0.1s
+[ 1838.1s] tab_mlp_o60_joint n=5 rep=4 1.6s
+[ 1838.2s] tab_mlp_o60_proto n=5 rep=4 0.1s
+[ 1838.3s] tab_mlp_o60_ft n=25 rep=0 0.1s
+[ 1841.4s] tab_mlp_o60_joint n=25 rep=0 3.1s
+[ 1841.4s] tab_mlp_o60_proto n=25 rep=0 0.1s
+[ 1841.5s] tab_mlp_o60_ft n=25 rep=1 0.1s
+[ 1844.4s] tab_mlp_o60_joint n=25 rep=1 2.9s
+[ 1844.5s] tab_mlp_o60_proto n=25 rep=1 0.1s
+[ 1844.6s] tab_mlp_o60_ft n=25 rep=2 0.1s
+[ 1848.1s] tab_mlp_o60_joint n=25 rep=2 3.5s
+[ 1848.2s] tab_mlp_o60_proto n=25 rep=2 0.1s
+[ 1848.3s] tab_mlp_o60_ft n=25 rep=3 0.1s
+[ 1850.9s] tab_mlp_o60_joint n=25 rep=3 2.6s
+[ 1850.9s] tab_mlp_o60_proto n=25 rep=3 0.1s
+[ 1851.0s] tab_mlp_o60_ft n=25 rep=4 0.1s
+[ 1852.8s] tab_mlp_o60_joint n=25 rep=4 1.7s
+[ 1852.8s] tab_mlp_o60_proto n=25 rep=4 0.1s
+[ 1852.9s] tab_mlp_o60_ft n=50 rep=0 0.1s
+[ 1856.1s] tab_mlp_o60_joint n=50 rep=0 3.1s
+[ 1856.2s] tab_mlp_o60_proto n=50 rep=0 0.1s
+[ 1856.3s] tab_mlp_o60_ft n=50 rep=1 0.1s
+[ 1858.4s] tab_mlp_o60_joint n=50 rep=1 2.1s
+[ 1858.5s] tab_mlp_o60_proto n=50 rep=1 0.1s
+[ 1858.6s] tab_mlp_o60_ft n=50 rep=2 0.1s
+[ 1862.0s] tab_mlp_o60_joint n=50 rep=2 3.4s
+[ 1862.1s] tab_mlp_o60_proto n=50 rep=2 0.1s
+[ 1862.2s] tab_mlp_o60_ft n=50 rep=3 0.1s
+[ 1864.3s] tab_mlp_o60_joint n=50 rep=3 2.1s
+[ 1864.4s] tab_mlp_o60_proto n=50 rep=3 0.1s
+[ 1864.5s] tab_mlp_o60_ft n=50 rep=4 0.1s
+[ 1868.0s] tab_mlp_o60_joint n=50 rep=4 3.5s
+[ 1868.0s] tab_mlp_o60_proto n=50 rep=4 0.1s
+[ 1868.2s] tab_mlp_o60_ft n=100 rep=0 0.1s
+[ 1873.8s] tab_mlp_o60_joint n=100 rep=0 5.7s
+[ 1873.9s] tab_mlp_o60_proto n=100 rep=0 0.1s
+[ 1874.1s] tab_mlp_o60_ft n=100 rep=1 0.1s
+[ 1877.6s] tab_mlp_o60_joint n=100 rep=1 3.6s
+[ 1877.7s] tab_mlp_o60_proto n=100 rep=1 0.1s
+[ 1877.9s] tab_mlp_o60_ft n=100 rep=2 0.2s
+[ 1881.5s] tab_mlp_o60_joint n=100 rep=2 3.5s
+[ 1881.6s] tab_mlp_o60_proto n=100 rep=2 0.1s
+[ 1881.7s] tab_mlp_o60_ft n=100 rep=3 0.2s
+[ 1884.6s] tab_mlp_o60_joint n=100 rep=3 2.9s
+[ 1884.7s] tab_mlp_o60_proto n=100 rep=3 0.1s
+[ 1884.8s] tab_mlp_o60_ft n=100 rep=4 0.1s
+[ 1888.2s] tab_mlp_o60_joint n=100 rep=4 3.4s
+[ 1888.3s] tab_mlp_o60_proto n=100 rep=4 0.1s
+[ 1888.5s] tab_mlp_o60_ft n=200 rep=0 0.2s
+[ 1891.3s] tab_mlp_o60_joint n=200 rep=0 2.9s
+[ 1891.4s] tab_mlp_o60_proto n=200 rep=0 0.1s
+[ 1891.5s] tab_mlp_o60_ft n=200 rep=1 0.1s
+[ 1894.4s] tab_mlp_o60_joint n=200 rep=1 2.9s
+[ 1894.5s] tab_mlp_o60_proto n=200 rep=1 0.1s
+[ 1894.7s] tab_mlp_o60_ft n=200 rep=2 0.2s
+[ 1898.2s] tab_mlp_o60_joint n=200 rep=2 3.5s
+[ 1898.3s] tab_mlp_o60_proto n=200 rep=2 0.1s
+[ 1898.5s] tab_mlp_o60_ft n=200 rep=3 0.2s
+[ 1904.0s] tab_mlp_o60_joint n=200 rep=3 5.6s
+[ 1904.1s] tab_mlp_o60_proto n=200 rep=3 0.1s
+[ 1904.4s] tab_mlp_o60_ft n=200 rep=4 0.2s
+[ 1907.2s] tab_mlp_o60_joint n=200 rep=4 2.9s
+[ 1907.3s] tab_mlp_o60_proto n=200 rep=4 0.1s
+[ 2021.0s] train res/rz seed 0: 15 epochs  114s
+[ 2021.1s] tab_res_rz_zs n=0 saved
+[ 2021.2s] tab_res_rz_ft n=5 rep=0 0.1s
+[ 2027.5s] tab_res_rz_joint n=5 rep=0 6.3s
+[ 2027.7s] tab_res_rz_proto n=5 rep=0 0.2s
+[ 2027.8s] tab_res_rz_ft n=5 rep=1 0.1s
+[ 2030.5s] tab_res_rz_joint n=5 rep=1 2.7s
+[ 2030.6s] tab_res_rz_proto n=5 rep=1 0.1s
+[ 2030.7s] tab_res_rz_ft n=5 rep=2 0.1s
+[ 2032.1s] tab_res_rz_joint n=5 rep=2 1.4s
+[ 2032.2s] tab_res_rz_proto n=5 rep=2 0.1s
+[ 2032.3s] tab_res_rz_ft n=5 rep=3 0.1s
+[ 2033.7s] tab_res_rz_joint n=5 rep=3 1.4s
+[ 2033.8s] tab_res_rz_proto n=5 rep=3 0.1s
+[ 2033.9s] tab_res_rz_ft n=5 rep=4 0.1s
+[ 2036.1s] tab_res_rz_joint n=5 rep=4 2.2s
+[ 2036.2s] tab_res_rz_proto n=5 rep=4 0.1s
+[ 2036.3s] tab_res_rz_ft n=25 rep=0 0.1s
+[ 2039.2s] tab_res_rz_joint n=25 rep=0 2.9s
+[ 2039.3s] tab_res_rz_proto n=25 rep=0 0.1s
+[ 2039.4s] tab_res_rz_ft n=25 rep=1 0.1s
+[ 2041.6s] tab_res_rz_joint n=25 rep=1 2.2s
+[ 2041.7s] tab_res_rz_proto n=25 rep=1 0.1s
+[ 2041.8s] tab_res_rz_ft n=25 rep=2 0.1s
+[ 2045.9s] tab_res_rz_joint n=25 rep=2 4.2s
+[ 2046.1s] tab_res_rz_proto n=25 rep=2 0.1s
+[ 2046.5s] tab_res_rz_ft n=25 rep=3 0.4s
+[ 2051.0s] tab_res_rz_joint n=25 rep=3 4.5s
+[ 2051.1s] tab_res_rz_proto n=25 rep=3 0.1s
+[ 2051.2s] tab_res_rz_ft n=25 rep=4 0.1s
+[ 2054.1s] tab_res_rz_joint n=25 rep=4 2.9s
+[ 2054.2s] tab_res_rz_proto n=25 rep=4 0.1s
+[ 2054.3s] tab_res_rz_ft n=50 rep=0 0.2s
+[ 2057.4s] tab_res_rz_joint n=50 rep=0 3.1s
+[ 2057.5s] tab_res_rz_proto n=50 rep=0 0.1s
+[ 2057.6s] tab_res_rz_ft n=50 rep=1 0.1s
+[ 2061.5s] tab_res_rz_joint n=50 rep=1 3.9s
+[ 2061.6s] tab_res_rz_proto n=50 rep=1 0.1s
+[ 2061.7s] tab_res_rz_ft n=50 rep=2 0.2s
+[ 2064.2s] tab_res_rz_joint n=50 rep=2 2.4s
+[ 2064.2s] tab_res_rz_proto n=50 rep=2 0.1s
+[ 2064.4s] tab_res_rz_ft n=50 rep=3 0.2s
+[ 2066.5s] tab_res_rz_joint n=50 rep=3 2.1s
+[ 2066.5s] tab_res_rz_proto n=50 rep=3 0.1s
+[ 2066.6s] tab_res_rz_ft n=50 rep=4 0.1s
+[ 2068.5s] tab_res_rz_joint n=50 rep=4 1.8s
+[ 2068.5s] tab_res_rz_proto n=50 rep=4 0.1s
+[ 2068.6s] tab_res_rz_ft n=100 rep=0 0.1s
+[ 2073.1s] tab_res_rz_joint n=100 rep=0 4.5s
+[ 2073.2s] tab_res_rz_proto n=100 rep=0 0.1s
+[ 2073.4s] tab_res_rz_ft n=100 rep=1 0.2s
+[ 2079.6s] tab_res_rz_joint n=100 rep=1 6.2s
+[ 2079.7s] tab_res_rz_proto n=100 rep=1 0.1s
+[ 2079.9s] tab_res_rz_ft n=100 rep=2 0.2s
+[ 2084.2s] tab_res_rz_joint n=100 rep=2 4.3s
+[ 2084.2s] tab_res_rz_proto n=100 rep=2 0.1s
+[ 2084.4s] tab_res_rz_ft n=100 rep=3 0.2s
+[ 2088.0s] tab_res_rz_joint n=100 rep=3 3.5s
+[ 2088.0s] tab_res_rz_proto n=100 rep=3 0.1s
+[ 2088.2s] tab_res_rz_ft n=100 rep=4 0.2s
+[ 2092.4s] tab_res_rz_joint n=100 rep=4 4.2s
+[ 2092.5s] tab_res_rz_proto n=100 rep=4 0.1s
+[ 2092.8s] tab_res_rz_ft n=200 rep=0 0.3s
+[ 2096.3s] tab_res_rz_joint n=200 rep=0 3.6s
+[ 2096.4s] tab_res_rz_proto n=200 rep=0 0.1s
+[ 2096.7s] tab_res_rz_ft n=200 rep=1 0.3s
+[ 2100.3s] tab_res_rz_joint n=200 rep=1 3.6s
+[ 2100.4s] tab_res_rz_proto n=200 rep=1 0.1s
+[ 2100.6s] tab_res_rz_ft n=200 rep=2 0.2s
+[ 2104.1s] tab_res_rz_joint n=200 rep=2 3.5s
+[ 2104.2s] tab_res_rz_proto n=200 rep=2 0.1s
+[ 2104.3s] tab_res_rz_ft n=200 rep=3 0.2s
+[ 2108.0s] tab_res_rz_joint n=200 rep=3 3.7s
+[ 2108.1s] tab_res_rz_proto n=200 rep=3 0.1s
+[ 2108.3s] tab_res_rz_ft n=200 rep=4 0.2s
+[ 2111.2s] tab_res_rz_joint n=200 rep=4 2.9s
+[ 2111.3s] tab_res_rz_proto n=200 rep=4 0.1s
+[ 2216.9s] train res/o60 seed 0: 15 epochs  106s
+[ 2217.0s] tab_res_o60_zs n=0 saved
+[ 2217.1s] tab_res_o60_ft n=5 rep=0 0.1s
+[ 2223.3s] tab_res_o60_joint n=5 rep=0 6.2s
+[ 2223.4s] tab_res_o60_proto n=5 rep=0 0.2s
+[ 2223.5s] tab_res_o60_ft n=5 rep=1 0.1s
+[ 2226.2s] tab_res_o60_joint n=5 rep=1 2.7s
+[ 2226.3s] tab_res_o60_proto n=5 rep=1 0.1s
+[ 2226.4s] tab_res_o60_ft n=5 rep=2 0.1s
+[ 2227.8s] tab_res_o60_joint n=5 rep=2 1.4s
+[ 2227.8s] tab_res_o60_proto n=5 rep=2 0.1s
+[ 2227.9s] tab_res_o60_ft n=5 rep=3 0.1s
+[ 2229.2s] tab_res_o60_joint n=5 rep=3 1.3s
+[ 2229.3s] tab_res_o60_proto n=5 rep=3 0.1s
+[ 2229.4s] tab_res_o60_ft n=5 rep=4 0.1s
+[ 2230.7s] tab_res_o60_joint n=5 rep=4 1.3s
+[ 2230.7s] tab_res_o60_proto n=5 rep=4 0.1s
+[ 2230.9s] tab_res_o60_ft n=25 rep=0 0.1s
+[ 2232.9s] tab_res_o60_joint n=25 rep=0 2.0s
+[ 2232.9s] tab_res_o60_proto n=25 rep=0 0.1s
+[ 2233.1s] tab_res_o60_ft n=25 rep=1 0.1s
+[ 2235.0s] tab_res_o60_joint n=25 rep=1 2.0s
+[ 2235.1s] tab_res_o60_proto n=25 rep=1 0.1s
+[ 2235.2s] tab_res_o60_ft n=25 rep=2 0.1s
+[ 2237.2s] tab_res_o60_joint n=25 rep=2 2.0s
+[ 2237.3s] tab_res_o60_proto n=25 rep=2 0.1s
+[ 2237.4s] tab_res_o60_ft n=25 rep=3 0.1s
+[ 2240.1s] tab_res_o60_joint n=25 rep=3 2.7s
+[ 2240.1s] tab_res_o60_proto n=25 rep=3 0.1s
+[ 2240.2s] tab_res_o60_ft n=25 rep=4 0.1s
+[ 2243.5s] tab_res_o60_joint n=25 rep=4 3.2s
+[ 2243.5s] tab_res_o60_proto n=25 rep=4 0.1s
+[ 2243.7s] tab_res_o60_ft n=50 rep=0 0.1s
+[ 2247.7s] tab_res_o60_joint n=50 rep=0 4.0s
+[ 2247.8s] tab_res_o60_proto n=50 rep=0 0.1s
+[ 2247.9s] tab_res_o60_ft n=50 rep=1 0.1s
+[ 2250.0s] tab_res_o60_joint n=50 rep=1 2.1s
+[ 2250.1s] tab_res_o60_proto n=50 rep=1 0.1s
+[ 2250.2s] tab_res_o60_ft n=50 rep=2 0.1s
+[ 2253.3s] tab_res_o60_joint n=50 rep=2 3.1s
+[ 2253.4s] tab_res_o60_proto n=50 rep=2 0.1s
+[ 2253.5s] tab_res_o60_ft n=50 rep=3 0.1s
+[ 2256.1s] tab_res_o60_joint n=50 rep=3 2.5s
+[ 2256.2s] tab_res_o60_proto n=50 rep=3 0.1s
+[ 2256.3s] tab_res_o60_ft n=50 rep=4 0.2s
+[ 2259.7s] tab_res_o60_joint n=50 rep=4 3.4s
+[ 2259.8s] tab_res_o60_proto n=50 rep=4 0.1s
+[ 2259.9s] tab_res_o60_ft n=100 rep=0 0.2s
+[ 2266.2s] tab_res_o60_joint n=100 rep=0 6.2s
+[ 2266.2s] tab_res_o60_proto n=100 rep=0 0.1s
+[ 2266.4s] tab_res_o60_ft n=100 rep=1 0.1s
+[ 2270.5s] tab_res_o60_joint n=100 rep=1 4.1s
+[ 2270.6s] tab_res_o60_proto n=100 rep=1 0.1s
+[ 2270.8s] tab_res_o60_ft n=100 rep=2 0.2s
+[ 2274.9s] tab_res_o60_joint n=100 rep=2 4.1s
+[ 2274.9s] tab_res_o60_proto n=100 rep=2 0.1s
+[ 2275.1s] tab_res_o60_ft n=100 rep=3 0.1s
+[ 2277.2s] tab_res_o60_joint n=100 rep=3 2.1s
+[ 2277.2s] tab_res_o60_proto n=100 rep=3 0.1s
+[ 2277.4s] tab_res_o60_ft n=100 rep=4 0.1s
+[ 2280.1s] tab_res_o60_joint n=100 rep=4 2.8s
+[ 2280.2s] tab_res_o60_proto n=100 rep=4 0.1s
+[ 2280.4s] tab_res_o60_ft n=200 rep=0 0.2s
+[ 2284.2s] tab_res_o60_joint n=200 rep=0 3.8s
+[ 2284.3s] tab_res_o60_proto n=200 rep=0 0.1s
+[ 2284.4s] tab_res_o60_ft n=200 rep=1 0.1s
+[ 2287.3s] tab_res_o60_joint n=200 rep=1 2.9s
+[ 2287.4s] tab_res_o60_proto n=200 rep=1 0.1s
+[ 2287.5s] tab_res_o60_ft n=200 rep=2 0.1s
+[ 2294.2s] tab_res_o60_joint n=200 rep=2 6.7s
+[ 2294.3s] tab_res_o60_proto n=200 rep=2 0.1s
+[ 2294.4s] tab_res_o60_ft n=200 rep=3 0.2s
+[ 2298.5s] tab_res_o60_joint n=200 rep=3 4.1s
+[ 2298.6s] tab_res_o60_proto n=200 rep=3 0.1s
+[ 2298.8s] tab_res_o60_ft n=200 rep=4 0.2s
+[ 2302.9s] tab_res_o60_joint n=200 rep=4 4.1s
+[ 2303.0s] tab_res_o60_proto n=200 rep=4 0.1s
+[ 2396.6s] train dann/rz seed 0: 15 epochs  94s
+[ 2396.7s] tab_dann_rz_zs n=0 saved
+[ 2396.8s] tab_dann_rz_ft n=5 rep=0 0.1s
+[ 2403.0s] tab_dann_rz_joint n=5 rep=0 6.2s
+[ 2403.2s] tab_dann_rz_proto n=5 rep=0 0.2s
+[ 2403.3s] tab_dann_rz_ft n=5 rep=1 0.1s
+[ 2405.3s] tab_dann_rz_joint n=5 rep=1 2.1s
+[ 2405.4s] tab_dann_rz_proto n=5 rep=1 0.1s
+[ 2405.5s] tab_dann_rz_ft n=5 rep=2 0.1s
+[ 2407.6s] tab_dann_rz_joint n=5 rep=2 2.1s
+[ 2407.7s] tab_dann_rz_proto n=5 rep=2 0.1s
+[ 2407.7s] tab_dann_rz_ft n=5 rep=3 0.1s
+[ 2409.1s] tab_dann_rz_joint n=5 rep=3 1.4s
+[ 2409.2s] tab_dann_rz_proto n=5 rep=3 0.1s
+[ 2409.3s] tab_dann_rz_ft n=5 rep=4 0.1s
+[ 2411.3s] tab_dann_rz_joint n=5 rep=4 2.0s
+[ 2411.4s] tab_dann_rz_proto n=5 rep=4 0.1s
+[ 2411.6s] tab_dann_rz_ft n=25 rep=0 0.2s
+[ 2415.1s] tab_dann_rz_joint n=25 rep=0 3.5s
+[ 2415.2s] tab_dann_rz_proto n=25 rep=0 0.1s
+[ 2415.3s] tab_dann_rz_ft n=25 rep=1 0.1s
+[ 2417.4s] tab_dann_rz_joint n=25 rep=1 2.1s
+[ 2417.4s] tab_dann_rz_proto n=25 rep=1 0.1s
+[ 2417.5s] tab_dann_rz_ft n=25 rep=2 0.1s
+[ 2420.8s] tab_dann_rz_joint n=25 rep=2 3.2s
+[ 2420.9s] tab_dann_rz_proto n=25 rep=2 0.1s
+[ 2420.9s] tab_dann_rz_ft n=25 rep=3 0.1s
+[ 2422.6s] tab_dann_rz_joint n=25 rep=3 1.7s
+[ 2422.7s] tab_dann_rz_proto n=25 rep=3 0.1s
+[ 2422.8s] tab_dann_rz_ft n=25 rep=4 0.1s
+[ 2425.1s] tab_dann_rz_joint n=25 rep=4 2.3s
+[ 2425.1s] tab_dann_rz_proto n=25 rep=4 0.1s
+[ 2425.3s] tab_dann_rz_ft n=50 rep=0 0.1s
+[ 2427.9s] tab_dann_rz_joint n=50 rep=0 2.6s
+[ 2428.0s] tab_dann_rz_proto n=50 rep=0 0.1s
+[ 2428.1s] tab_dann_rz_ft n=50 rep=1 0.1s
+[ 2430.8s] tab_dann_rz_joint n=50 rep=1 2.7s
+[ 2430.9s] tab_dann_rz_proto n=50 rep=1 0.1s
+[ 2431.1s] tab_dann_rz_ft n=50 rep=2 0.2s
+[ 2433.8s] tab_dann_rz_joint n=50 rep=2 2.8s
+[ 2433.9s] tab_dann_rz_proto n=50 rep=2 0.1s
+[ 2434.0s] tab_dann_rz_ft n=50 rep=3 0.1s
+[ 2436.8s] tab_dann_rz_joint n=50 rep=3 2.8s
+[ 2436.9s] tab_dann_rz_proto n=50 rep=3 0.1s
+[ 2437.0s] tab_dann_rz_ft n=50 rep=4 0.2s
+[ 2440.4s] tab_dann_rz_joint n=50 rep=4 3.4s
+[ 2440.5s] tab_dann_rz_proto n=50 rep=4 0.1s
+[ 2440.7s] tab_dann_rz_ft n=100 rep=0 0.2s
+[ 2444.1s] tab_dann_rz_joint n=100 rep=0 3.5s
+[ 2444.2s] tab_dann_rz_proto n=100 rep=0 0.1s
+[ 2444.4s] tab_dann_rz_ft n=100 rep=1 0.2s
+[ 2447.2s] tab_dann_rz_joint n=100 rep=1 2.9s
+[ 2447.3s] tab_dann_rz_proto n=100 rep=1 0.1s
+[ 2447.5s] tab_dann_rz_ft n=100 rep=2 0.2s
+[ 2451.1s] tab_dann_rz_joint n=100 rep=2 3.6s
+[ 2451.2s] tab_dann_rz_proto n=100 rep=2 0.1s
+[ 2451.4s] tab_dann_rz_ft n=100 rep=3 0.2s
+[ 2455.6s] tab_dann_rz_joint n=100 rep=3 4.2s
+[ 2455.6s] tab_dann_rz_proto n=100 rep=3 0.1s
+[ 2455.8s] tab_dann_rz_ft n=100 rep=4 0.2s
+[ 2459.3s] tab_dann_rz_joint n=100 rep=4 3.5s
+[ 2459.4s] tab_dann_rz_proto n=100 rep=4 0.1s
+[ 2459.6s] tab_dann_rz_ft n=200 rep=0 0.2s
+[ 2463.9s] tab_dann_rz_joint n=200 rep=0 4.4s
+[ 2464.0s] tab_dann_rz_proto n=200 rep=0 0.1s
+[ 2464.3s] tab_dann_rz_ft n=200 rep=1 0.2s
+[ 2467.8s] tab_dann_rz_joint n=200 rep=1 3.5s
+[ 2467.9s] tab_dann_rz_proto n=200 rep=1 0.1s
+[ 2468.0s] tab_dann_rz_ft n=200 rep=2 0.2s
+[ 2471.3s] tab_dann_rz_joint n=200 rep=2 3.3s
+[ 2471.4s] tab_dann_rz_proto n=200 rep=2 0.1s
+[ 2471.5s] tab_dann_rz_ft n=200 rep=3 0.2s
+[ 2474.9s] tab_dann_rz_joint n=200 rep=3 3.4s
+[ 2475.0s] tab_dann_rz_proto n=200 rep=3 0.1s
+[ 2475.2s] tab_dann_rz_ft n=200 rep=4 0.2s
+[ 2485.7s] tab_dann_rz_joint n=200 rep=4 10.5s
+[ 2485.8s] tab_dann_rz_proto n=200 rep=4 0.1s
+[ 2541.7s] train dann/rg seed 0: 15 epochs  56s
+[ 2541.7s] tab_dann_rg_zs n=0 saved
+[ 2541.8s] tab_dann_rg_ft n=5 rep=0 0.1s
+[ 2546.2s] tab_dann_rg_joint n=5 rep=0 4.4s
+[ 2546.5s] tab_dann_rg_proto n=5 rep=0 0.2s
+[ 2546.6s] tab_dann_rg_ft n=5 rep=1 0.1s
+[ 2548.1s] tab_dann_rg_joint n=5 rep=1 1.6s
+[ 2548.2s] tab_dann_rg_proto n=5 rep=1 0.1s
+[ 2548.3s] tab_dann_rg_ft n=5 rep=2 0.1s
+[ 2549.4s] tab_dann_rg_joint n=5 rep=2 1.1s
+[ 2549.4s] tab_dann_rg_proto n=5 rep=2 0.1s
+[ 2549.5s] tab_dann_rg_ft n=5 rep=3 0.1s
+[ 2550.4s] tab_dann_rg_joint n=5 rep=3 0.9s
+[ 2550.4s] tab_dann_rg_proto n=5 rep=3 0.1s
+[ 2550.5s] tab_dann_rg_ft n=5 rep=4 0.1s
+[ 2551.4s] tab_dann_rg_joint n=5 rep=4 0.8s
+[ 2551.4s] tab_dann_rg_proto n=5 rep=4 0.1s
+[ 2551.5s] tab_dann_rg_ft n=25 rep=0 0.1s
+[ 2552.8s] tab_dann_rg_joint n=25 rep=0 1.3s
+[ 2552.8s] tab_dann_rg_proto n=25 rep=0 0.0s
+[ 2552.9s] tab_dann_rg_ft n=25 rep=1 0.1s
+[ 2554.1s] tab_dann_rg_joint n=25 rep=1 1.2s
+[ 2554.1s] tab_dann_rg_proto n=25 rep=1 0.0s
+[ 2554.2s] tab_dann_rg_ft n=25 rep=2 0.1s
+[ 2555.4s] tab_dann_rg_joint n=25 rep=2 1.2s
+[ 2555.4s] tab_dann_rg_proto n=25 rep=2 0.0s
+[ 2555.5s] tab_dann_rg_ft n=25 rep=3 0.1s
+[ 2556.7s] tab_dann_rg_joint n=25 rep=3 1.2s
+[ 2556.7s] tab_dann_rg_proto n=25 rep=3 0.0s
+[ 2556.8s] tab_dann_rg_ft n=25 rep=4 0.1s
+[ 2558.0s] tab_dann_rg_joint n=25 rep=4 1.2s
+[ 2558.1s] tab_dann_rg_proto n=25 rep=4 0.0s
+[ 2558.1s] tab_dann_rg_ft n=50 rep=0 0.1s
+[ 2559.3s] tab_dann_rg_joint n=50 rep=0 1.2s
+[ 2559.4s] tab_dann_rg_proto n=50 rep=0 0.0s
+[ 2559.5s] tab_dann_rg_ft n=50 rep=1 0.1s
+[ 2560.7s] tab_dann_rg_joint n=50 rep=1 1.2s
+[ 2560.7s] tab_dann_rg_proto n=50 rep=1 0.1s
+[ 2560.8s] tab_dann_rg_ft n=50 rep=2 0.1s
+[ 2562.4s] tab_dann_rg_joint n=50 rep=2 1.6s
+[ 2562.5s] tab_dann_rg_proto n=50 rep=2 0.0s
+[ 2562.6s] tab_dann_rg_ft n=50 rep=3 0.1s
+[ 2563.8s] tab_dann_rg_joint n=50 rep=3 1.2s
+[ 2563.8s] tab_dann_rg_proto n=50 rep=3 0.0s
+[ 2563.9s] tab_dann_rg_ft n=50 rep=4 0.1s
+[ 2566.0s] tab_dann_rg_joint n=50 rep=4 2.0s
+[ 2566.0s] tab_dann_rg_proto n=50 rep=4 0.1s
+[ 2566.1s] tab_dann_rg_ft n=100 rep=0 0.1s
+[ 2568.2s] tab_dann_rg_joint n=100 rep=0 2.1s
+[ 2568.2s] tab_dann_rg_proto n=100 rep=0 0.0s
+[ 2568.3s] tab_dann_rg_ft n=100 rep=1 0.1s
+[ 2570.0s] tab_dann_rg_joint n=100 rep=1 1.7s
+[ 2570.1s] tab_dann_rg_proto n=100 rep=1 0.1s
+[ 2570.2s] tab_dann_rg_ft n=100 rep=2 0.1s
+[ 2571.8s] tab_dann_rg_joint n=100 rep=2 1.6s
+[ 2571.8s] tab_dann_rg_proto n=100 rep=2 0.1s
+[ 2571.9s] tab_dann_rg_ft n=100 rep=3 0.1s
+[ 2573.2s] tab_dann_rg_joint n=100 rep=3 1.3s
+[ 2573.3s] tab_dann_rg_proto n=100 rep=3 0.0s
+[ 2573.4s] tab_dann_rg_ft n=100 rep=4 0.1s
+[ 2574.7s] tab_dann_rg_joint n=100 rep=4 1.3s
+[ 2574.7s] tab_dann_rg_proto n=100 rep=4 0.1s
+[ 2574.9s] tab_dann_rg_ft n=200 rep=0 0.1s
+[ 2576.6s] tab_dann_rg_joint n=200 rep=0 1.7s
+[ 2576.6s] tab_dann_rg_proto n=200 rep=0 0.1s
+[ 2576.7s] tab_dann_rg_ft n=200 rep=1 0.1s
+[ 2578.5s] tab_dann_rg_joint n=200 rep=1 1.8s
+[ 2578.6s] tab_dann_rg_proto n=200 rep=1 0.1s
+[ 2578.7s] tab_dann_rg_ft n=200 rep=2 0.2s
+[ 2580.9s] tab_dann_rg_joint n=200 rep=2 2.2s
+[ 2581.0s] tab_dann_rg_proto n=200 rep=2 0.1s
+[ 2581.1s] tab_dann_rg_ft n=200 rep=3 0.1s
+[ 2582.9s] tab_dann_rg_joint n=200 rep=3 1.7s
+[ 2582.9s] tab_dann_rg_proto n=200 rep=3 0.1s
+[ 2583.0s] tab_dann_rg_ft n=200 rep=4 0.1s
+[ 2584.7s] tab_dann_rg_joint n=200 rep=4 1.7s
+[ 2584.8s] tab_dann_rg_proto n=200 rep=4 0.0s
+[ 2628.3s] train dann/o60 seed 0: 15 epochs  44s
+[ 2628.4s] tab_dann_o60_zs n=0 saved
+[ 2628.4s] tab_dann_o60_ft n=5 rep=0 0.0s
+[ 2631.8s] tab_dann_o60_joint n=5 rep=0 3.4s
+[ 2632.0s] tab_dann_o60_proto n=5 rep=0 0.1s
+[ 2632.0s] tab_dann_o60_ft n=5 rep=1 0.0s
+[ 2632.9s] tab_dann_o60_joint n=5 rep=1 0.9s
+[ 2633.0s] tab_dann_o60_proto n=5 rep=1 0.1s
+[ 2633.0s] tab_dann_o60_ft n=5 rep=2 0.1s
+[ 2633.9s] tab_dann_o60_joint n=5 rep=2 0.8s
+[ 2633.9s] tab_dann_o60_proto n=5 rep=2 0.1s
+[ 2634.0s] tab_dann_o60_ft n=5 rep=3 0.0s
+[ 2634.9s] tab_dann_o60_joint n=5 rep=3 0.9s
+[ 2634.9s] tab_dann_o60_proto n=5 rep=3 0.0s
+[ 2635.0s] tab_dann_o60_ft n=5 rep=4 0.1s
+[ 2635.8s] tab_dann_o60_joint n=5 rep=4 0.9s
+[ 2635.9s] tab_dann_o60_proto n=5 rep=4 0.1s
+[ 2636.0s] tab_dann_o60_ft n=25 rep=0 0.1s
+[ 2637.6s] tab_dann_o60_joint n=25 rep=0 1.7s
+[ 2637.7s] tab_dann_o60_proto n=25 rep=0 0.1s
+[ 2637.7s] tab_dann_o60_ft n=25 rep=1 0.1s
+[ 2639.0s] tab_dann_o60_joint n=25 rep=1 1.3s
+[ 2639.1s] tab_dann_o60_proto n=25 rep=1 0.1s
+[ 2639.2s] tab_dann_o60_ft n=25 rep=2 0.1s
+[ 2640.5s] tab_dann_o60_joint n=25 rep=2 1.3s
+[ 2640.5s] tab_dann_o60_proto n=25 rep=2 0.1s
+[ 2640.6s] tab_dann_o60_ft n=25 rep=3 0.1s
+[ 2642.4s] tab_dann_o60_joint n=25 rep=3 1.8s
+[ 2642.5s] tab_dann_o60_proto n=25 rep=3 0.1s
+[ 2642.6s] tab_dann_o60_ft n=25 rep=4 0.1s
+[ 2645.0s] tab_dann_o60_joint n=25 rep=4 2.4s
+[ 2645.1s] tab_dann_o60_proto n=25 rep=4 0.1s
+[ 2645.3s] tab_dann_o60_ft n=50 rep=0 0.1s
+[ 2647.1s] tab_dann_o60_joint n=50 rep=0 1.9s
+[ 2647.2s] tab_dann_o60_proto n=50 rep=0 0.1s
+[ 2647.3s] tab_dann_o60_ft n=50 rep=1 0.1s
+[ 2651.8s] tab_dann_o60_joint n=50 rep=1 4.5s
+[ 2651.9s] tab_dann_o60_proto n=50 rep=1 0.1s
+[ 2652.0s] tab_dann_o60_ft n=50 rep=2 0.1s
+[ 2653.1s] tab_dann_o60_joint n=50 rep=2 1.1s
+[ 2653.1s] tab_dann_o60_proto n=50 rep=2 0.1s
+[ 2653.2s] tab_dann_o60_ft n=50 rep=3 0.1s
+[ 2654.5s] tab_dann_o60_joint n=50 rep=3 1.3s
+[ 2654.6s] tab_dann_o60_proto n=50 rep=3 0.1s
+[ 2654.8s] tab_dann_o60_ft n=50 rep=4 0.1s
+[ 2657.1s] tab_dann_o60_joint n=50 rep=4 2.3s
+[ 2657.1s] tab_dann_o60_proto n=50 rep=4 0.1s
+[ 2657.3s] tab_dann_o60_ft n=100 rep=0 0.1s
+[ 2659.9s] tab_dann_o60_joint n=100 rep=0 2.7s
+[ 2660.0s] tab_dann_o60_proto n=100 rep=0 0.1s
+[ 2660.1s] tab_dann_o60_ft n=100 rep=1 0.1s
+[ 2662.5s] tab_dann_o60_joint n=100 rep=1 2.4s
+[ 2662.6s] tab_dann_o60_proto n=100 rep=1 0.1s
+[ 2662.7s] tab_dann_o60_ft n=100 rep=2 0.1s
+[ 2665.6s] tab_dann_o60_joint n=100 rep=2 3.0s
+[ 2665.7s] tab_dann_o60_proto n=100 rep=2 0.1s
+[ 2665.8s] tab_dann_o60_ft n=100 rep=3 0.1s
+[ 2668.4s] tab_dann_o60_joint n=100 rep=3 2.6s
+[ 2668.5s] tab_dann_o60_proto n=100 rep=3 0.1s
+[ 2668.6s] tab_dann_o60_ft n=100 rep=4 0.2s
+[ 2670.9s] tab_dann_o60_joint n=100 rep=4 2.3s
+[ 2671.0s] tab_dann_o60_proto n=100 rep=4 0.1s
+[ 2671.2s] tab_dann_o60_ft n=200 rep=0 0.2s
+[ 2674.2s] tab_dann_o60_joint n=200 rep=0 3.0s
+[ 2674.3s] tab_dann_o60_proto n=200 rep=0 0.1s
+[ 2674.4s] tab_dann_o60_ft n=200 rep=1 0.1s
+[ 2676.9s] tab_dann_o60_joint n=200 rep=1 2.4s
+[ 2676.9s] tab_dann_o60_proto n=200 rep=1 0.1s
+[ 2677.1s] tab_dann_o60_ft n=200 rep=2 0.1s
+[ 2680.9s] tab_dann_o60_joint n=200 rep=2 3.8s
+[ 2681.0s] tab_dann_o60_proto n=200 rep=2 0.1s
+[ 2681.2s] tab_dann_o60_ft n=200 rep=3 0.1s
+[ 2684.4s] tab_dann_o60_joint n=200 rep=3 3.2s
+[ 2684.5s] tab_dann_o60_proto n=200 rep=3 0.1s
+[ 2684.6s] tab_dann_o60_ft n=200 rep=4 0.2s
+[ 2692.3s] tab_dann_o60_joint n=200 rep=4 7.7s
+[ 2692.5s] tab_dann_o60_proto n=200 rep=4 0.1s
+[ 2966.5s] train coral/rz seed 0: 45 epochs  274s
+[ 2966.6s] tab_coral_rz_zs n=0 saved
+[ 2966.6s] tab_coral_rz_ft n=5 rep=0 0.0s
+[ 2973.9s] tab_coral_rz_joint n=5 rep=0 7.3s
+[ 2974.1s] tab_coral_rz_proto n=5 rep=0 0.2s
+[ 2974.2s] tab_coral_rz_ft n=5 rep=1 0.1s
+[ 2977.1s] tab_coral_rz_joint n=5 rep=1 2.9s
+[ 2977.2s] tab_coral_rz_proto n=5 rep=1 0.1s
+[ 2977.3s] tab_coral_rz_ft n=5 rep=2 0.1s
+[ 2978.8s] tab_coral_rz_joint n=5 rep=2 1.5s
+[ 2978.9s] tab_coral_rz_proto n=5 rep=2 0.1s
+[ 2979.0s] tab_coral_rz_ft n=5 rep=3 0.1s
+[ 2982.1s] tab_coral_rz_joint n=5 rep=3 3.1s
+[ 2982.1s] tab_coral_rz_proto n=5 rep=3 0.1s
+[ 2982.3s] tab_coral_rz_ft n=5 rep=4 0.1s
+[ 2985.4s] tab_coral_rz_joint n=5 rep=4 3.1s
+[ 2985.5s] tab_coral_rz_proto n=5 rep=4 0.1s
+[ 2985.6s] tab_coral_rz_ft n=25 rep=0 0.1s
+[ 2989.4s] tab_coral_rz_joint n=25 rep=0 3.8s
+[ 2989.5s] tab_coral_rz_proto n=25 rep=0 0.1s
+[ 2989.6s] tab_coral_rz_ft n=25 rep=1 0.1s
+[ 2991.9s] tab_coral_rz_joint n=25 rep=1 2.3s
+[ 2992.0s] tab_coral_rz_proto n=25 rep=1 0.1s
+[ 2992.1s] tab_coral_rz_ft n=25 rep=2 0.1s
+[ 2995.8s] tab_coral_rz_joint n=25 rep=2 3.7s
+[ 2995.9s] tab_coral_rz_proto n=25 rep=2 0.1s
+[ 2996.0s] tab_coral_rz_ft n=25 rep=3 0.1s
+[ 2999.0s] tab_coral_rz_joint n=25 rep=3 3.1s
+[ 2999.1s] tab_coral_rz_proto n=25 rep=3 0.1s
+[ 2999.3s] tab_coral_rz_ft n=25 rep=4 0.1s
+[ 3003.1s] tab_coral_rz_joint n=25 rep=4 3.9s
+[ 3003.2s] tab_coral_rz_proto n=25 rep=4 0.1s
+[ 3003.4s] tab_coral_rz_ft n=50 rep=0 0.1s
+[ 3020.1s] tab_coral_rz_joint n=50 rep=0 16.7s
+[ 3020.2s] tab_coral_rz_proto n=50 rep=0 0.1s
+[ 3020.4s] tab_coral_rz_ft n=50 rep=1 0.2s
+[ 3024.0s] tab_coral_rz_joint n=50 rep=1 3.6s
+[ 3024.1s] tab_coral_rz_proto n=50 rep=1 0.1s
+[ 3024.2s] tab_coral_rz_ft n=50 rep=2 0.2s
+[ 3028.0s] tab_coral_rz_joint n=50 rep=2 3.7s
+[ 3028.1s] tab_coral_rz_proto n=50 rep=2 0.1s
+[ 3028.3s] tab_coral_rz_ft n=50 rep=3 0.2s
+[ 3032.1s] tab_coral_rz_joint n=50 rep=3 3.9s
+[ 3032.2s] tab_coral_rz_proto n=50 rep=3 0.1s
+[ 3032.4s] tab_coral_rz_ft n=50 rep=4 0.2s
+[ 3036.2s] tab_coral_rz_joint n=50 rep=4 3.8s
+[ 3036.3s] tab_coral_rz_proto n=50 rep=4 0.1s
+[ 3036.5s] tab_coral_rz_ft n=100 rep=0 0.3s
+[ 3040.1s] tab_coral_rz_joint n=100 rep=0 3.6s
+[ 3040.2s] tab_coral_rz_proto n=100 rep=0 0.1s
+[ 3040.3s] tab_coral_rz_ft n=100 rep=1 0.1s
+[ 3045.1s] tab_coral_rz_joint n=100 rep=1 4.8s
+[ 3045.2s] tab_coral_rz_proto n=100 rep=1 0.1s
+[ 3045.4s] tab_coral_rz_ft n=100 rep=2 0.2s
+[ 3050.9s] tab_coral_rz_joint n=100 rep=2 5.5s
+[ 3051.0s] tab_coral_rz_proto n=100 rep=2 0.1s
+[ 3051.2s] tab_coral_rz_ft n=100 rep=3 0.2s
+[ 3056.4s] tab_coral_rz_joint n=100 rep=3 5.2s
+[ 3056.5s] tab_coral_rz_proto n=100 rep=3 0.1s
+[ 3056.9s] tab_coral_rz_ft n=100 rep=4 0.4s
+[ 3060.9s] tab_coral_rz_joint n=100 rep=4 4.0s
+[ 3061.0s] tab_coral_rz_proto n=100 rep=4 0.1s
+[ 3061.3s] tab_coral_rz_ft n=200 rep=0 0.3s
+[ 3065.3s] tab_coral_rz_joint n=200 rep=0 4.0s
+[ 3065.4s] tab_coral_rz_proto n=200 rep=0 0.1s
+[ 3066.1s] tab_coral_rz_ft n=200 rep=1 0.6s
+[ 3073.0s] tab_coral_rz_joint n=200 rep=1 7.0s
+[ 3073.2s] tab_coral_rz_proto n=200 rep=1 0.1s
+[ 3073.5s] tab_coral_rz_ft n=200 rep=2 0.4s
+[ 3077.5s] tab_coral_rz_joint n=200 rep=2 4.0s
+[ 3077.6s] tab_coral_rz_proto n=200 rep=2 0.1s
+[ 3078.0s] tab_coral_rz_ft n=200 rep=3 0.4s
+[ 3082.6s] tab_coral_rz_joint n=200 rep=3 4.6s
+[ 3082.7s] tab_coral_rz_proto n=200 rep=3 0.1s
+[ 3083.3s] tab_coral_rz_ft n=200 rep=4 0.7s
+[ 3104.3s] tab_coral_rz_joint n=200 rep=4 21.0s
+[ 3104.4s] tab_coral_rz_proto n=200 rep=4 0.1s
+[ 3468.9s] train coral/o60 seed 0: 45 epochs  364s
+[ 3469.0s] tab_coral_o60_zs n=0 saved
+[ 3469.1s] tab_coral_o60_ft n=5 rep=0 0.1s
+[ 3476.6s] tab_coral_o60_joint n=5 rep=0 7.5s
+[ 3476.9s] tab_coral_o60_proto n=5 rep=0 0.3s
+[ 3477.0s] tab_coral_o60_ft n=5 rep=1 0.1s
+[ 3480.3s] tab_coral_o60_joint n=5 rep=1 3.4s
+[ 3480.4s] tab_coral_o60_proto n=5 rep=1 0.1s
+[ 3480.5s] tab_coral_o60_ft n=5 rep=2 0.1s
+[ 3482.2s] tab_coral_o60_joint n=5 rep=2 1.7s
+[ 3482.3s] tab_coral_o60_proto n=5 rep=2 0.1s
+[ 3482.4s] tab_coral_o60_ft n=5 rep=3 0.1s
+[ 3484.1s] tab_coral_o60_joint n=5 rep=3 1.7s
+[ 3484.2s] tab_coral_o60_proto n=5 rep=3 0.1s
+[ 3484.2s] tab_coral_o60_ft n=5 rep=4 0.1s
+[ 3486.0s] tab_coral_o60_joint n=5 rep=4 1.8s
+[ 3486.1s] tab_coral_o60_proto n=5 rep=4 0.1s
+[ 3486.3s] tab_coral_o60_ft n=25 rep=0 0.1s
+[ 3490.3s] tab_coral_o60_joint n=25 rep=0 4.0s
+[ 3490.4s] tab_coral_o60_proto n=25 rep=0 0.1s
+[ 3490.5s] tab_coral_o60_ft n=25 rep=1 0.1s
+[ 3495.3s] tab_coral_o60_joint n=25 rep=1 4.8s
+[ 3495.4s] tab_coral_o60_proto n=25 rep=1 0.1s
+[ 3495.5s] tab_coral_o60_ft n=25 rep=2 0.1s
+[ 3498.1s] tab_coral_o60_joint n=25 rep=2 2.6s
+[ 3498.2s] tab_coral_o60_proto n=25 rep=2 0.1s
+[ 3498.3s] tab_coral_o60_ft n=25 rep=3 0.1s
+[ 3503.3s] tab_coral_o60_joint n=25 rep=3 5.0s
+[ 3503.4s] tab_coral_o60_proto n=25 rep=3 0.1s
+[ 3503.5s] tab_coral_o60_ft n=25 rep=4 0.1s
+[ 3505.3s] tab_coral_o60_joint n=25 rep=4 1.8s
+[ 3505.4s] tab_coral_o60_proto n=25 rep=4 0.1s
+[ 3505.5s] tab_coral_o60_ft n=50 rep=0 0.1s
+[ 3509.1s] tab_coral_o60_joint n=50 rep=0 3.6s
+[ 3509.2s] tab_coral_o60_proto n=50 rep=0 0.1s
+[ 3509.4s] tab_coral_o60_ft n=50 rep=1 0.2s
+[ 3512.5s] tab_coral_o60_joint n=50 rep=1 3.1s
+[ 3512.6s] tab_coral_o60_proto n=50 rep=1 0.1s
+[ 3512.7s] tab_coral_o60_ft n=50 rep=2 0.1s
+[ 3518.4s] tab_coral_o60_joint n=50 rep=2 5.7s
+[ 3518.5s] tab_coral_o60_proto n=50 rep=2 0.1s
+[ 3518.7s] tab_coral_o60_ft n=50 rep=3 0.1s
+[ 3521.1s] tab_coral_o60_joint n=50 rep=3 2.4s
+[ 3521.2s] tab_coral_o60_proto n=50 rep=3 0.1s
+[ 3521.3s] tab_coral_o60_ft n=50 rep=4 0.1s
+[ 3523.8s] tab_coral_o60_joint n=50 rep=4 2.5s
+[ 3523.9s] tab_coral_o60_proto n=50 rep=4 0.1s
+[ 3524.1s] tab_coral_o60_ft n=100 rep=0 0.2s
+[ 3529.0s] tab_coral_o60_joint n=100 rep=0 4.9s
+[ 3529.1s] tab_coral_o60_proto n=100 rep=0 0.1s
+[ 3529.3s] tab_coral_o60_ft n=100 rep=1 0.2s
+[ 3536.3s] tab_coral_o60_joint n=100 rep=1 7.0s
+[ 3536.4s] tab_coral_o60_proto n=100 rep=1 0.1s
+[ 3536.5s] tab_coral_o60_ft n=100 rep=2 0.2s
+[ 3540.5s] tab_coral_o60_joint n=100 rep=2 4.0s
+[ 3540.6s] tab_coral_o60_proto n=100 rep=2 0.1s
+[ 3540.8s] tab_coral_o60_ft n=100 rep=3 0.2s
+[ 3543.2s] tab_coral_o60_joint n=100 rep=3 2.4s
+[ 3543.3s] tab_coral_o60_proto n=100 rep=3 0.1s
+[ 3543.5s] tab_coral_o60_ft n=100 rep=4 0.2s
+[ 3547.4s] tab_coral_o60_joint n=100 rep=4 3.9s
+[ 3547.5s] tab_coral_o60_proto n=100 rep=4 0.1s
+[ 3547.7s] tab_coral_o60_ft n=200 rep=0 0.2s
+[ 3555.5s] tab_coral_o60_joint n=200 rep=0 7.8s
+[ 3555.6s] tab_coral_o60_proto n=200 rep=0 0.1s
+[ 3555.8s] tab_coral_o60_ft n=200 rep=1 0.2s
+[ 3560.5s] tab_coral_o60_joint n=200 rep=1 4.6s
+[ 3560.6s] tab_coral_o60_proto n=200 rep=1 0.1s
+[ 3560.8s] tab_coral_o60_ft n=200 rep=2 0.2s
+[ 3569.0s] tab_coral_o60_joint n=200 rep=2 8.2s
+[ 3569.1s] tab_coral_o60_proto n=200 rep=2 0.1s
+[ 3569.3s] tab_coral_o60_ft n=200 rep=3 0.2s
+[ 3575.0s] tab_coral_o60_joint n=200 rep=3 5.7s
+[ 3575.1s] tab_coral_o60_proto n=200 rep=3 0.1s
+[ 3575.2s] tab_coral_o60_ft n=200 rep=4 0.1s
+[ 3577.3s] tab_coral_o60_joint n=200 rep=4 2.1s
+[ 3577.4s] tab_coral_o60_proto n=200 rep=4 0.1s
+[ 3577.4s] done

reports/nnlab/logs/tab_nn_r12.log

@@ -0,0 +1,107 @@
+[    1.1s] data loaded
+[   17.5s] tab_dann_rz_ft n=5 rep=5 12.2s
+[   28.7s] tab_dann_rz_joint n=5 rep=5 11.3s
+[   29.0s] tab_dann_rz_proto n=5 rep=5 0.3s
+[   29.1s] tab_dann_rz_ft n=5 rep=6 0.1s
+[   30.8s] tab_dann_rz_joint n=5 rep=6 1.7s
+[   30.8s] tab_dann_rz_proto n=5 rep=6 0.1s
+[   30.9s] tab_dann_rz_ft n=5 rep=7 0.1s
+[   32.6s] tab_dann_rz_joint n=5 rep=7 1.7s
+[   32.7s] tab_dann_rz_proto n=5 rep=7 0.1s
+[   32.8s] tab_dann_rz_ft n=5 rep=8 0.1s
+[   34.5s] tab_dann_rz_joint n=5 rep=8 1.7s
+[   34.6s] tab_dann_rz_proto n=5 rep=8 0.1s
+[   34.7s] tab_dann_rz_ft n=5 rep=9 0.1s
+[   36.3s] tab_dann_rz_joint n=5 rep=9 1.7s
+[   36.4s] tab_dann_rz_proto n=5 rep=9 0.1s
+[   36.5s] tab_dann_rz_ft n=5 rep=10 0.1s
+[   38.2s] tab_dann_rz_joint n=5 rep=10 1.7s
+[   38.3s] tab_dann_rz_proto n=5 rep=10 0.1s
+[   38.4s] tab_dann_rz_ft n=5 rep=11 0.1s
+[   40.0s] tab_dann_rz_joint n=5 rep=11 1.6s
+[   40.1s] tab_dann_rz_proto n=5 rep=11 0.1s
+[   40.3s] tab_dann_rz_ft n=25 rep=5 0.2s
+[   43.4s] tab_dann_rz_joint n=25 rep=5 3.2s
+[   43.5s] tab_dann_rz_proto n=25 rep=5 0.1s
+[   43.7s] tab_dann_rz_ft n=25 rep=6 0.1s
+[   46.2s] tab_dann_rz_joint n=25 rep=6 2.5s
+[   46.2s] tab_dann_rz_proto n=25 rep=6 0.1s
+[   46.4s] tab_dann_rz_ft n=25 rep=7 0.1s
+[   50.5s] tab_dann_rz_joint n=25 rep=7 4.2s
+[   50.6s] tab_dann_rz_proto n=25 rep=7 0.1s
+[   50.8s] tab_dann_rz_ft n=25 rep=8 0.2s
+[   54.3s] tab_dann_rz_joint n=25 rep=8 3.5s
+[   54.4s] tab_dann_rz_proto n=25 rep=8 0.1s
+[   54.6s] tab_dann_rz_ft n=25 rep=9 0.1s
+[   57.7s] tab_dann_rz_joint n=25 rep=9 3.1s
+[   57.8s] tab_dann_rz_proto n=25 rep=9 0.1s
+[   57.9s] tab_dann_rz_ft n=25 rep=10 0.2s
+[   63.4s] tab_dann_rz_joint n=25 rep=10 5.4s
+[   63.4s] tab_dann_rz_proto n=25 rep=10 0.1s
+[   63.7s] tab_dann_rz_ft n=25 rep=11 0.3s
+[   67.0s] tab_dann_rz_joint n=25 rep=11 3.3s
+[   67.0s] tab_dann_rz_proto n=25 rep=11 0.1s
+[   67.3s] tab_dann_rz_ft n=50 rep=5 0.2s
+[   71.4s] tab_dann_rz_joint n=50 rep=5 4.1s
+[   71.5s] tab_dann_rz_proto n=50 rep=5 0.1s
+[   71.7s] tab_dann_rz_ft n=50 rep=6 0.2s
+[   76.9s] tab_dann_rz_joint n=50 rep=6 5.2s
+[   77.0s] tab_dann_rz_proto n=50 rep=6 0.1s
+[   77.2s] tab_dann_rz_ft n=50 rep=7 0.2s
+[   81.8s] tab_dann_rz_joint n=50 rep=7 4.6s
+[   81.9s] tab_dann_rz_proto n=50 rep=7 0.1s
+[   82.1s] tab_dann_rz_ft n=50 rep=8 0.2s
+[   85.0s] tab_dann_rz_joint n=50 rep=8 2.8s
+[   85.0s] tab_dann_rz_proto n=50 rep=8 0.1s
+[   85.2s] tab_dann_rz_ft n=50 rep=9 0.2s
+[   88.1s] tab_dann_rz_joint n=50 rep=9 2.9s
+[   88.2s] tab_dann_rz_proto n=50 rep=9 0.1s
+[   88.4s] tab_dann_rz_ft n=50 rep=10 0.2s
+[   93.1s] tab_dann_rz_joint n=50 rep=10 4.6s
+[   93.1s] tab_dann_rz_proto n=50 rep=10 0.1s
+[   93.3s] tab_dann_rz_ft n=50 rep=11 0.2s
+[   98.5s] tab_dann_rz_joint n=50 rep=11 5.2s
+[   98.6s] tab_dann_rz_proto n=50 rep=11 0.1s
+[   98.7s] tab_dann_rz_ft n=100 rep=5 0.2s
+[  109.8s] tab_dann_rz_joint n=100 rep=5 11.1s
+[  109.9s] tab_dann_rz_proto n=100 rep=5 0.1s
+[  110.2s] tab_dann_rz_ft n=100 rep=6 0.3s
+[  114.2s] tab_dann_rz_joint n=100 rep=6 4.0s
+[  114.3s] tab_dann_rz_proto n=100 rep=6 0.1s
+[  114.6s] tab_dann_rz_ft n=100 rep=7 0.3s
+[  119.5s] tab_dann_rz_joint n=100 rep=7 4.9s
+[  119.6s] tab_dann_rz_proto n=100 rep=7 0.1s
+[  119.9s] tab_dann_rz_ft n=100 rep=8 0.3s
+[  125.4s] tab_dann_rz_joint n=100 rep=8 5.6s
+[  125.5s] tab_dann_rz_proto n=100 rep=8 0.1s
+[  125.8s] tab_dann_rz_ft n=100 rep=9 0.3s
+[  131.1s] tab_dann_rz_joint n=100 rep=9 5.3s
+[  131.2s] tab_dann_rz_proto n=100 rep=9 0.1s
+[  131.4s] tab_dann_rz_ft n=100 rep=10 0.2s
+[  135.8s] tab_dann_rz_joint n=100 rep=10 4.4s
+[  135.9s] tab_dann_rz_proto n=100 rep=10 0.1s
+[  136.2s] tab_dann_rz_ft n=100 rep=11 0.3s
+[  140.1s] tab_dann_rz_joint n=100 rep=11 3.9s
+[  140.2s] tab_dann_rz_proto n=100 rep=11 0.1s
+[  140.5s] tab_dann_rz_ft n=200 rep=5 0.4s
+[  145.1s] tab_dann_rz_joint n=200 rep=5 4.6s
+[  145.2s] tab_dann_rz_proto n=200 rep=5 0.1s
+[  145.5s] tab_dann_rz_ft n=200 rep=6 0.3s
+[  149.1s] tab_dann_rz_joint n=200 rep=6 3.5s
+[  149.2s] tab_dann_rz_proto n=200 rep=6 0.1s
+[  149.5s] tab_dann_rz_ft n=200 rep=7 0.4s
+[  153.8s] tab_dann_rz_joint n=200 rep=7 4.3s
+[  153.9s] tab_dann_rz_proto n=200 rep=7 0.1s
+[  154.2s] tab_dann_rz_ft n=200 rep=8 0.3s
+[  159.0s] tab_dann_rz_joint n=200 rep=8 4.8s
+[  159.1s] tab_dann_rz_proto n=200 rep=8 0.1s
+[  159.5s] tab_dann_rz_ft n=200 rep=9 0.4s
+[  163.8s] tab_dann_rz_joint n=200 rep=9 4.3s
+[  163.9s] tab_dann_rz_proto n=200 rep=9 0.1s
+[  164.3s] tab_dann_rz_ft n=200 rep=10 0.4s
+[  169.7s] tab_dann_rz_joint n=200 rep=10 5.4s
+[  169.8s] tab_dann_rz_proto n=200 rep=10 0.1s
+[  170.2s] tab_dann_rz_ft n=200 rep=11 0.4s
+[  175.7s] tab_dann_rz_joint n=200 rep=11 5.5s
+[  175.8s] tab_dann_rz_proto n=200 rep=11 0.1s
+[  175.8s] done

reports/nnlab/logs/transductive.log

@@ -0,0 +1,758 @@
+[    0.8s] loaded: Madrid (76263, 393), Amsterdam (25992, 393)
+[    0.9s] graph: 583310 edges, mean degree 22.4, min 10
+[    1.4s] lgbm_upw__clamp n=5 rep=0 base=lgbm_upw done in 0.5s
+[    1.5s] lgbm_upw__clamp n=5 rep=1 base=lgbm_upw done in 0.0s
+[    1.5s] lgbm_upw__clamp n=5 rep=2 base=lgbm_upw done in 0.1s
+[    1.6s] lgbm_upw__clamp n=5 rep=3 base=lgbm_upw done in 0.0s
+[    1.6s] lgbm_upw__clamp n=5 rep=4 base=lgbm_upw done in 0.0s
+[    1.6s] lgbm_upw__clamp n=25 rep=0 base=lgbm_upw done in 0.0s
+[    1.7s] lgbm_upw__clamp n=25 rep=1 base=lgbm_upw done in 0.0s
+[    1.7s] lgbm_upw__clamp n=25 rep=2 base=lgbm_upw done in 0.0s
+[    1.8s] lgbm_upw__clamp n=25 rep=3 base=lgbm_upw done in 0.0s
+[    1.8s] lgbm_upw__clamp n=25 rep=4 base=lgbm_upw done in 0.0s
+[    1.9s] lgbm_upw__clamp n=50 rep=0 base=lgbm_upw done in 0.0s
+[    1.9s] lgbm_upw__clamp n=50 rep=1 base=lgbm_upw done in 0.1s
+[    2.0s] lgbm_upw__clamp n=50 rep=2 base=lgbm_upw done in 0.0s
+[    2.0s] lgbm_upw__clamp n=50 rep=3 base=lgbm_upw done in 0.0s
+[    2.0s] lgbm_upw__clamp n=50 rep=4 base=lgbm_upw done in 0.0s
+[    2.1s] lgbm_upw__clamp n=100 rep=0 base=lgbm_upw done in 0.1s
+[    2.1s] lgbm_upw__clamp n=100 rep=1 base=lgbm_upw done in 0.0s
+[    2.2s] lgbm_upw__clamp n=100 rep=2 base=lgbm_upw done in 0.0s
+[    2.2s] lgbm_upw__clamp n=100 rep=3 base=lgbm_upw done in 0.0s
+[    2.2s] lgbm_upw__clamp n=100 rep=4 base=lgbm_upw done in 0.0s
+[    2.3s] lgbm_upw__clamp n=200 rep=0 base=lgbm_upw done in 0.0s
+[    2.3s] lgbm_upw__clamp n=200 rep=1 base=lgbm_upw done in 0.0s
+[    2.3s] lgbm_upw__clamp n=200 rep=2 base=lgbm_upw done in 0.0s
+[    2.4s] lgbm_upw__clamp n=200 rep=3 base=lgbm_upw done in 0.0s
+[    2.4s] lgbm_upw__clamp n=200 rep=4 base=lgbm_upw done in 0.0s
+[    2.5s] zs_xgb_dann__clamp n=5 rep=0 base=zs_xgb_dann done in 0.1s
+[    2.5s] zs_xgb_dann__clamp n=5 rep=1 base=zs_xgb_dann done in 0.0s
+[    2.6s] zs_xgb_dann__clamp n=5 rep=2 base=zs_xgb_dann done in 0.0s
+[    2.6s] zs_xgb_dann__clamp n=5 rep=3 base=zs_xgb_dann done in 0.1s
+[    2.7s] zs_xgb_dann__clamp n=5 rep=4 base=zs_xgb_dann done in 0.0s
+[    2.7s] zs_xgb_dann__clamp n=25 rep=0 base=zs_xgb_dann done in 0.0s
+[    2.8s] zs_xgb_dann__clamp n=25 rep=1 base=zs_xgb_dann done in 0.1s
+[    2.8s] zs_xgb_dann__clamp n=25 rep=2 base=zs_xgb_dann done in 0.1s
+[    2.9s] zs_xgb_dann__clamp n=25 rep=3 base=zs_xgb_dann done in 0.1s
+[    2.9s] zs_xgb_dann__clamp n=25 rep=4 base=zs_xgb_dann done in 0.1s
+[    3.0s] zs_xgb_dann__clamp n=50 rep=0 base=zs_xgb_dann done in 0.1s
+[    3.1s] zs_xgb_dann__clamp n=50 rep=1 base=zs_xgb_dann done in 0.1s
+[    3.1s] zs_xgb_dann__clamp n=50 rep=2 base=zs_xgb_dann done in 0.1s
+[    3.2s] zs_xgb_dann__clamp n=50 rep=3 base=zs_xgb_dann done in 0.1s
+[    3.2s] zs_xgb_dann__clamp n=50 rep=4 base=zs_xgb_dann done in 0.0s
+[    3.3s] zs_xgb_dann__clamp n=100 rep=0 base=zs_xgb_dann done in 0.1s
+[    3.3s] zs_xgb_dann__clamp n=100 rep=1 base=zs_xgb_dann done in 0.0s
+[    3.4s] zs_xgb_dann__clamp n=100 rep=2 base=zs_xgb_dann done in 0.1s
+[    3.4s] zs_xgb_dann__clamp n=100 rep=3 base=zs_xgb_dann done in 0.0s
+[    3.5s] zs_xgb_dann__clamp n=100 rep=4 base=zs_xgb_dann done in 0.0s
+[    3.5s] zs_xgb_dann__clamp n=200 rep=0 base=zs_xgb_dann done in 0.0s
+[    3.6s] zs_xgb_dann__clamp n=200 rep=1 base=zs_xgb_dann done in 0.1s
+[    3.6s] zs_xgb_dann__clamp n=200 rep=2 base=zs_xgb_dann done in 0.1s
+[    3.7s] zs_xgb_dann__clamp n=200 rep=3 base=zs_xgb_dann done in 0.1s
+[    3.8s] zs_xgb_dann__clamp n=200 rep=4 base=zs_xgb_dann done in 0.0s
+[    3.8s] zs_lgbm__clamp n=5 rep=0 base=zs_lgbm done in 0.1s
+[    3.9s] zs_lgbm__clamp n=5 rep=1 base=zs_lgbm done in 0.1s
+[    3.9s] zs_lgbm__clamp n=5 rep=2 base=zs_lgbm done in 0.0s
+[    4.0s] zs_lgbm__clamp n=5 rep=3 base=zs_lgbm done in 0.1s
+[    4.0s] zs_lgbm__clamp n=5 rep=4 base=zs_lgbm done in 0.1s
+[    4.1s] zs_lgbm__clamp n=25 rep=0 base=zs_lgbm done in 0.1s
+[    4.2s] zs_lgbm__clamp n=25 rep=1 base=zs_lgbm done in 0.1s
+[    4.2s] zs_lgbm__clamp n=25 rep=2 base=zs_lgbm done in 0.1s
+[    4.3s] zs_lgbm__clamp n=25 rep=3 base=zs_lgbm done in 0.1s
+[    4.3s] zs_lgbm__clamp n=25 rep=4 base=zs_lgbm done in 0.0s
+[    4.4s] zs_lgbm__clamp n=50 rep=0 base=zs_lgbm done in 0.1s
+[    4.4s] zs_lgbm__clamp n=50 rep=1 base=zs_lgbm done in 0.1s
+[    4.5s] zs_lgbm__clamp n=50 rep=2 base=zs_lgbm done in 0.0s
+[    4.5s] zs_lgbm__clamp n=50 rep=3 base=zs_lgbm done in 0.0s
+[    4.6s] zs_lgbm__clamp n=50 rep=4 base=zs_lgbm done in 0.1s
+[    4.6s] zs_lgbm__clamp n=100 rep=0 base=zs_lgbm done in 0.1s
+[    4.6s] zs_lgbm__clamp n=100 rep=1 base=zs_lgbm done in 0.0s
+[    4.7s] zs_lgbm__clamp n=100 rep=2 base=zs_lgbm done in 0.0s
+[    4.7s] zs_lgbm__clamp n=100 rep=3 base=zs_lgbm done in 0.0s
+[    4.8s] zs_lgbm__clamp n=100 rep=4 base=zs_lgbm done in 0.0s
+[    4.8s] zs_lgbm__clamp n=200 rep=0 base=zs_lgbm done in 0.0s
+[    4.9s] zs_lgbm__clamp n=200 rep=1 base=zs_lgbm done in 0.0s
+[    4.9s] zs_lgbm__clamp n=200 rep=2 base=zs_lgbm done in 0.1s
+[    5.0s] zs_lgbm__clamp n=200 rep=3 base=zs_lgbm done in 0.0s
+[    5.0s] zs_lgbm__clamp n=200 rep=4 base=zs_lgbm done in 0.0s
+[    5.0s] zs_dann__clamp n=5 rep=0 base=zs_dann done in 0.0s
+[    5.1s] zs_dann__clamp n=5 rep=1 base=zs_dann done in 0.0s
+[    5.1s] zs_dann__clamp n=5 rep=2 base=zs_dann done in 0.0s
+[    5.2s] zs_dann__clamp n=5 rep=3 base=zs_dann done in 0.0s
+[    5.2s] zs_dann__clamp n=5 rep=4 base=zs_dann done in 0.0s
+[    5.2s] zs_dann__clamp n=25 rep=0 base=zs_dann done in 0.0s
+[    5.3s] zs_dann__clamp n=25 rep=1 base=zs_dann done in 0.0s
+[    5.3s] zs_dann__clamp n=25 rep=2 base=zs_dann done in 0.0s
+[    5.3s] zs_dann__clamp n=25 rep=3 base=zs_dann done in 0.0s
+[    5.4s] zs_dann__clamp n=25 rep=4 base=zs_dann done in 0.0s
+[    5.4s] zs_dann__clamp n=50 rep=0 base=zs_dann done in 0.0s
+[    5.4s] zs_dann__clamp n=50 rep=1 base=zs_dann done in 0.0s
+[    5.4s] zs_dann__clamp n=50 rep=2 base=zs_dann done in 0.0s
+[    5.5s] zs_dann__clamp n=50 rep=3 base=zs_dann done in 0.0s
+[    5.5s] zs_dann__clamp n=50 rep=4 base=zs_dann done in 0.0s
+[    5.6s] zs_dann__clamp n=100 rep=0 base=zs_dann done in 0.0s
+[    5.6s] zs_dann__clamp n=100 rep=1 base=zs_dann done in 0.0s
+[    5.6s] zs_dann__clamp n=100 rep=2 base=zs_dann done in 0.0s
+[    5.6s] zs_dann__clamp n=100 rep=3 base=zs_dann done in 0.0s
+[    5.7s] zs_dann__clamp n=100 rep=4 base=zs_dann done in 0.0s
+[    5.7s] zs_dann__clamp n=200 rep=0 base=zs_dann done in 0.0s
+[    5.7s] zs_dann__clamp n=200 rep=1 base=zs_dann done in 0.0s
+[    5.8s] zs_dann__clamp n=200 rep=2 base=zs_dann done in 0.0s
+[    5.8s] zs_dann__clamp n=200 rep=3 base=zs_dann done in 0.0s
+[    5.8s] zs_dann__clamp n=200 rep=4 base=zs_dann done in 0.0s
+[    5.8s] done
+[    0.8s] loaded: Madrid (76263, 393), Amsterdam (25992, 393)
+[    0.9s] graph: 583310 edges, mean degree 22.4, min 10
+[    1.0s] lgbm_upw__lp n=5 rep=0 base=lgbm_upw done in 0.1s
+[    1.2s] lgbm_upw__lp n=5 rep=1 base=lgbm_upw done in 0.1s
+[    1.3s] lgbm_upw__lp n=5 rep=2 base=lgbm_upw done in 0.1s
+[    1.5s] lgbm_upw__lp n=5 rep=3 base=lgbm_upw done in 0.2s
+[    1.6s] lgbm_upw__lp n=5 rep=4 base=lgbm_upw done in 0.1s
+[    1.8s] lgbm_upw__lp n=25 rep=0 base=lgbm_upw done in 0.2s
+[    1.9s] lgbm_upw__lp n=25 rep=1 base=lgbm_upw done in 0.2s
+[    2.1s] lgbm_upw__lp n=25 rep=2 base=lgbm_upw done in 0.2s
+[    2.2s] lgbm_upw__lp n=25 rep=3 base=lgbm_upw done in 0.1s
+[    2.4s] lgbm_upw__lp n=25 rep=4 base=lgbm_upw done in 0.2s
+[    2.5s] lgbm_upw__lp n=50 rep=0 base=lgbm_upw done in 0.2s
+[    2.7s] lgbm_upw__lp n=50 rep=1 base=lgbm_upw done in 0.2s
+[    2.8s] lgbm_upw__lp n=50 rep=2 base=lgbm_upw done in 0.2s
+[    3.0s] lgbm_upw__lp n=50 rep=3 base=lgbm_upw done in 0.1s
+[    3.1s] lgbm_upw__lp n=50 rep=4 base=lgbm_upw done in 0.2s
+[    3.3s] lgbm_upw__lp n=100 rep=0 base=lgbm_upw done in 0.1s
+[    3.4s] lgbm_upw__lp n=100 rep=1 base=lgbm_upw done in 0.1s
+[    3.6s] lgbm_upw__lp n=100 rep=2 base=lgbm_upw done in 0.1s
+[    3.7s] lgbm_upw__lp n=100 rep=3 base=lgbm_upw done in 0.1s
+[    3.8s] lgbm_upw__lp n=100 rep=4 base=lgbm_upw done in 0.1s
+[    4.0s] lgbm_upw__lp n=200 rep=0 base=lgbm_upw done in 0.1s
+[    4.1s] lgbm_upw__lp n=200 rep=1 base=lgbm_upw done in 0.1s
+[    4.2s] lgbm_upw__lp n=200 rep=2 base=lgbm_upw done in 0.1s
+[    4.4s] lgbm_upw__lp n=200 rep=3 base=lgbm_upw done in 0.1s
+[    4.5s] lgbm_upw__lp n=200 rep=4 base=lgbm_upw done in 0.1s
+[    4.7s] zs_xgb_dann__lp n=5 rep=0 base=zs_xgb_dann done in 0.1s
+[    4.8s] zs_xgb_dann__lp n=5 rep=1 base=zs_xgb_dann done in 0.1s
+[    5.0s] zs_xgb_dann__lp n=5 rep=2 base=zs_xgb_dann done in 0.1s
+[    5.1s] zs_xgb_dann__lp n=5 rep=3 base=zs_xgb_dann done in 0.2s
+[    5.3s] zs_xgb_dann__lp n=5 rep=4 base=zs_xgb_dann done in 0.2s
+[    5.4s] zs_xgb_dann__lp n=25 rep=0 base=zs_xgb_dann done in 0.1s
+[    5.6s] zs_xgb_dann__lp n=25 rep=1 base=zs_xgb_dann done in 0.2s
+[    5.7s] zs_xgb_dann__lp n=25 rep=2 base=zs_xgb_dann done in 0.1s
+[    5.9s] zs_xgb_dann__lp n=25 rep=3 base=zs_xgb_dann done in 0.2s
+[    6.0s] zs_xgb_dann__lp n=25 rep=4 base=zs_xgb_dann done in 0.1s
+[    6.2s] zs_xgb_dann__lp n=50 rep=0 base=zs_xgb_dann done in 0.1s
+[    6.3s] zs_xgb_dann__lp n=50 rep=1 base=zs_xgb_dann done in 0.2s
+[    6.5s] zs_xgb_dann__lp n=50 rep=2 base=zs_xgb_dann done in 0.2s
+[    6.6s] zs_xgb_dann__lp n=50 rep=3 base=zs_xgb_dann done in 0.1s
+[    6.7s] zs_xgb_dann__lp n=50 rep=4 base=zs_xgb_dann done in 0.1s
+[    6.9s] zs_xgb_dann__lp n=100 rep=0 base=zs_xgb_dann done in 0.1s
+[    7.0s] zs_xgb_dann__lp n=100 rep=1 base=zs_xgb_dann done in 0.1s
+[    7.2s] zs_xgb_dann__lp n=100 rep=2 base=zs_xgb_dann done in 0.1s
+[    7.3s] zs_xgb_dann__lp n=100 rep=3 base=zs_xgb_dann done in 0.1s
+[    7.4s] zs_xgb_dann__lp n=100 rep=4 base=zs_xgb_dann done in 0.2s
+[    7.6s] zs_xgb_dann__lp n=200 rep=0 base=zs_xgb_dann done in 0.1s
+[    7.7s] zs_xgb_dann__lp n=200 rep=1 base=zs_xgb_dann done in 0.1s
+[    7.9s] zs_xgb_dann__lp n=200 rep=2 base=zs_xgb_dann done in 0.1s
+[    8.0s] zs_xgb_dann__lp n=200 rep=3 base=zs_xgb_dann done in 0.1s
+[    8.1s] zs_xgb_dann__lp n=200 rep=4 base=zs_xgb_dann done in 0.1s
+[    8.3s] zs_lgbm__lp n=5 rep=0 base=zs_lgbm done in 0.1s
+[    8.4s] zs_lgbm__lp n=5 rep=1 base=zs_lgbm done in 0.1s
+[    8.5s] zs_lgbm__lp n=5 rep=2 base=zs_lgbm done in 0.1s
+[    8.7s] zs_lgbm__lp n=5 rep=3 base=zs_lgbm done in 0.1s
+[    8.8s] zs_lgbm__lp n=5 rep=4 base=zs_lgbm done in 0.1s
+[    9.0s] zs_lgbm__lp n=25 rep=0 base=zs_lgbm done in 0.1s
+[    9.1s] zs_lgbm__lp n=25 rep=1 base=zs_lgbm done in 0.2s
+[    9.3s] zs_lgbm__lp n=25 rep=2 base=zs_lgbm done in 0.1s
+[    9.4s] zs_lgbm__lp n=25 rep=3 base=zs_lgbm done in 0.2s
+[    9.6s] zs_lgbm__lp n=25 rep=4 base=zs_lgbm done in 0.2s
+[    9.7s] zs_lgbm__lp n=50 rep=0 base=zs_lgbm done in 0.1s
+[    9.9s] zs_lgbm__lp n=50 rep=1 base=zs_lgbm done in 0.2s
+[   10.1s] zs_lgbm__lp n=50 rep=2 base=zs_lgbm done in 0.2s
+[   10.2s] zs_lgbm__lp n=50 rep=3 base=zs_lgbm done in 0.1s
+[   10.3s] zs_lgbm__lp n=50 rep=4 base=zs_lgbm done in 0.1s
+[   10.5s] zs_lgbm__lp n=100 rep=0 base=zs_lgbm done in 0.1s
+[   10.6s] zs_lgbm__lp n=100 rep=1 base=zs_lgbm done in 0.1s
+[   10.7s] zs_lgbm__lp n=100 rep=2 base=zs_lgbm done in 0.1s
+[   10.9s] zs_lgbm__lp n=100 rep=3 base=zs_lgbm done in 0.1s
+[   11.0s] zs_lgbm__lp n=100 rep=4 base=zs_lgbm done in 0.1s
+[   11.1s] zs_lgbm__lp n=200 rep=0 base=zs_lgbm done in 0.1s
+[   11.2s] zs_lgbm__lp n=200 rep=1 base=zs_lgbm done in 0.1s
+[   11.4s] zs_lgbm__lp n=200 rep=2 base=zs_lgbm done in 0.1s
+[   11.5s] zs_lgbm__lp n=200 rep=3 base=zs_lgbm done in 0.1s
+[   11.7s] zs_lgbm__lp n=200 rep=4 base=zs_lgbm done in 0.1s
+[   11.8s] zs_dann__lp n=5 rep=0 base=zs_dann done in 0.1s
+[   11.9s] zs_dann__lp n=5 rep=1 base=zs_dann done in 0.1s
+[   12.1s] zs_dann__lp n=5 rep=2 base=zs_dann done in 0.1s
+[   12.2s] zs_dann__lp n=5 rep=3 base=zs_dann done in 0.1s
+[   12.4s] zs_dann__lp n=5 rep=4 base=zs_dann done in 0.2s
+[   12.6s] zs_dann__lp n=25 rep=0 base=zs_dann done in 0.2s
+[   12.7s] zs_dann__lp n=25 rep=1 base=zs_dann done in 0.1s
+[   12.8s] zs_dann__lp n=25 rep=2 base=zs_dann done in 0.2s
+[   13.0s] zs_dann__lp n=25 rep=3 base=zs_dann done in 0.2s
+[   13.2s] zs_dann__lp n=25 rep=4 base=zs_dann done in 0.2s
+[   13.3s] zs_dann__lp n=50 rep=0 base=zs_dann done in 0.2s
+[   13.4s] zs_dann__lp n=50 rep=1 base=zs_dann done in 0.1s
+[   13.6s] zs_dann__lp n=50 rep=2 base=zs_dann done in 0.1s
+[   13.7s] zs_dann__lp n=50 rep=3 base=zs_dann done in 0.1s
+[   13.9s] zs_dann__lp n=50 rep=4 base=zs_dann done in 0.1s
+[   14.0s] zs_dann__lp n=100 rep=0 base=zs_dann done in 0.1s
+[   14.1s] zs_dann__lp n=100 rep=1 base=zs_dann done in 0.1s
+[   14.3s] zs_dann__lp n=100 rep=2 base=zs_dann done in 0.1s
+[   14.4s] zs_dann__lp n=100 rep=3 base=zs_dann done in 0.1s
+[   14.5s] zs_dann__lp n=100 rep=4 base=zs_dann done in 0.1s
+[   14.6s] zs_dann__lp n=200 rep=0 base=zs_dann done in 0.1s
+[   14.8s] zs_dann__lp n=200 rep=1 base=zs_dann done in 0.1s
+[   14.9s] zs_dann__lp n=200 rep=2 base=zs_dann done in 0.1s
+[   15.0s] zs_dann__lp n=200 rep=3 base=zs_dann done in 0.1s
+[   15.2s] zs_dann__lp n=200 rep=4 base=zs_dann done in 0.1s
+[   15.2s] done
+[    0.9s] loaded: Madrid (76263, 393), Amsterdam (25992, 393)
+[    0.9s] graph: 583310 edges, mean degree 22.4, min 10
+[    1.0s] lgbm_upw__lapshot n=5 rep=0 base=lgbm_upw done in 0.1s
+[    1.2s] lgbm_upw__lapshot n=5 rep=1 base=lgbm_upw done in 0.1s
+[    1.3s] lgbm_upw__lapshot n=5 rep=2 base=lgbm_upw done in 0.1s
+[    1.5s] lgbm_upw__lapshot n=5 rep=3 base=lgbm_upw done in 0.2s
+[    1.6s] lgbm_upw__lapshot n=5 rep=4 base=lgbm_upw done in 0.1s
+[    1.8s] lgbm_upw__lapshot n=25 rep=0 base=lgbm_upw done in 0.2s
+[    1.9s] lgbm_upw__lapshot n=25 rep=1 base=lgbm_upw done in 0.1s
+[    2.1s] lgbm_upw__lapshot n=25 rep=2 base=lgbm_upw done in 0.2s
+[    2.2s] lgbm_upw__lapshot n=25 rep=3 base=lgbm_upw done in 0.1s
+[    2.4s] lgbm_upw__lapshot n=25 rep=4 base=lgbm_upw done in 0.1s
+[    2.5s] lgbm_upw__lapshot n=50 rep=0 base=lgbm_upw done in 0.1s
+[    2.6s] lgbm_upw__lapshot n=50 rep=1 base=lgbm_upw done in 0.1s
+[    2.8s] lgbm_upw__lapshot n=50 rep=2 base=lgbm_upw done in 0.2s
+[    2.9s] lgbm_upw__lapshot n=50 rep=3 base=lgbm_upw done in 0.1s
+[    3.1s] lgbm_upw__lapshot n=50 rep=4 base=lgbm_upw done in 0.2s
+[    3.2s] lgbm_upw__lapshot n=100 rep=0 base=lgbm_upw done in 0.1s
+[    3.4s] lgbm_upw__lapshot n=100 rep=1 base=lgbm_upw done in 0.1s
+[    3.5s] lgbm_upw__lapshot n=100 rep=2 base=lgbm_upw done in 0.2s
+[    3.6s] lgbm_upw__lapshot n=100 rep=3 base=lgbm_upw done in 0.1s
+[    3.8s] lgbm_upw__lapshot n=100 rep=4 base=lgbm_upw done in 0.1s
+[    3.9s] lgbm_upw__lapshot n=200 rep=0 base=lgbm_upw done in 0.1s
+[    4.0s] lgbm_upw__lapshot n=200 rep=1 base=lgbm_upw done in 0.1s
+[    4.2s] lgbm_upw__lapshot n=200 rep=2 base=lgbm_upw done in 0.1s
+[    4.3s] lgbm_upw__lapshot n=200 rep=3 base=lgbm_upw done in 0.1s
+[    4.5s] lgbm_upw__lapshot n=200 rep=4 base=lgbm_upw done in 0.1s
+[    4.6s] zs_xgb_dann__lapshot n=5 rep=0 base=zs_xgb_dann done in 0.2s
+[    4.8s] zs_xgb_dann__lapshot n=5 rep=1 base=zs_xgb_dann done in 0.1s
+[    4.9s] zs_xgb_dann__lapshot n=5 rep=2 base=zs_xgb_dann done in 0.1s
+[    5.0s] zs_xgb_dann__lapshot n=5 rep=3 base=zs_xgb_dann done in 0.1s
+[    5.2s] zs_xgb_dann__lapshot n=5 rep=4 base=zs_xgb_dann done in 0.1s
+[    5.3s] zs_xgb_dann__lapshot n=25 rep=0 base=zs_xgb_dann done in 0.1s
+[    5.4s] zs_xgb_dann__lapshot n=25 rep=1 base=zs_xgb_dann done in 0.1s
+[    5.6s] zs_xgb_dann__lapshot n=25 rep=2 base=zs_xgb_dann done in 0.2s
+[    5.8s] zs_xgb_dann__lapshot n=25 rep=3 base=zs_xgb_dann done in 0.2s
+[    5.9s] zs_xgb_dann__lapshot n=25 rep=4 base=zs_xgb_dann done in 0.2s
+[    6.0s] zs_xgb_dann__lapshot n=50 rep=0 base=zs_xgb_dann done in 0.1s
+[    6.2s] zs_xgb_dann__lapshot n=50 rep=1 base=zs_xgb_dann done in 0.1s
+[    6.3s] zs_xgb_dann__lapshot n=50 rep=2 base=zs_xgb_dann done in 0.1s
+[    6.5s] zs_xgb_dann__lapshot n=50 rep=3 base=zs_xgb_dann done in 0.2s
+[    6.6s] zs_xgb_dann__lapshot n=50 rep=4 base=zs_xgb_dann done in 0.1s
+[    6.8s] zs_xgb_dann__lapshot n=100 rep=0 base=zs_xgb_dann done in 0.2s
+[    6.9s] zs_xgb_dann__lapshot n=100 rep=1 base=zs_xgb_dann done in 0.1s
+[    7.1s] zs_xgb_dann__lapshot n=100 rep=2 base=zs_xgb_dann done in 0.1s
+[    7.2s] zs_xgb_dann__lapshot n=100 rep=3 base=zs_xgb_dann done in 0.1s
+[    7.4s] zs_xgb_dann__lapshot n=100 rep=4 base=zs_xgb_dann done in 0.1s
+[    7.5s] zs_xgb_dann__lapshot n=200 rep=0 base=zs_xgb_dann done in 0.1s
+[    7.6s] zs_xgb_dann__lapshot n=200 rep=1 base=zs_xgb_dann done in 0.1s
+[    7.8s] zs_xgb_dann__lapshot n=200 rep=2 base=zs_xgb_dann done in 0.2s
+[    7.9s] zs_xgb_dann__lapshot n=200 rep=3 base=zs_xgb_dann done in 0.1s
+[    8.0s] zs_xgb_dann__lapshot n=200 rep=4 base=zs_xgb_dann done in 0.1s
+[    8.2s] zs_lgbm__lapshot n=5 rep=0 base=zs_lgbm done in 0.1s
+[    8.3s] zs_lgbm__lapshot n=5 rep=1 base=zs_lgbm done in 0.1s
+[    8.5s] zs_lgbm__lapshot n=5 rep=2 base=zs_lgbm done in 0.2s
+[    8.6s] zs_lgbm__lapshot n=5 rep=3 base=zs_lgbm done in 0.1s
+[    8.7s] zs_lgbm__lapshot n=5 rep=4 base=zs_lgbm done in 0.1s
+[    8.9s] zs_lgbm__lapshot n=25 rep=0 base=zs_lgbm done in 0.2s
+[    9.0s] zs_lgbm__lapshot n=25 rep=1 base=zs_lgbm done in 0.2s
+[    9.2s] zs_lgbm__lapshot n=25 rep=2 base=zs_lgbm done in 0.2s
+[    9.4s] zs_lgbm__lapshot n=25 rep=3 base=zs_lgbm done in 0.2s
+[    9.5s] zs_lgbm__lapshot n=25 rep=4 base=zs_lgbm done in 0.2s
+[    9.7s] zs_lgbm__lapshot n=50 rep=0 base=zs_lgbm done in 0.2s
+[    9.8s] zs_lgbm__lapshot n=50 rep=1 base=zs_lgbm done in 0.1s
+[   10.0s] zs_lgbm__lapshot n=50 rep=2 base=zs_lgbm done in 0.2s
+[   10.2s] zs_lgbm__lapshot n=50 rep=3 base=zs_lgbm done in 0.2s
+[   10.3s] zs_lgbm__lapshot n=50 rep=4 base=zs_lgbm done in 0.2s
+[   10.5s] zs_lgbm__lapshot n=100 rep=0 base=zs_lgbm done in 0.2s
+[   10.6s] zs_lgbm__lapshot n=100 rep=1 base=zs_lgbm done in 0.1s
+[   10.8s] zs_lgbm__lapshot n=100 rep=2 base=zs_lgbm done in 0.1s
+[   10.9s] zs_lgbm__lapshot n=100 rep=3 base=zs_lgbm done in 0.1s
+[   11.0s] zs_lgbm__lapshot n=100 rep=4 base=zs_lgbm done in 0.1s
+[   11.2s] zs_lgbm__lapshot n=200 rep=0 base=zs_lgbm done in 0.1s
+[   11.3s] zs_lgbm__lapshot n=200 rep=1 base=zs_lgbm done in 0.1s
+[   11.4s] zs_lgbm__lapshot n=200 rep=2 base=zs_lgbm done in 0.1s
+[   11.6s] zs_lgbm__lapshot n=200 rep=3 base=zs_lgbm done in 0.1s
+[   11.7s] zs_lgbm__lapshot n=200 rep=4 base=zs_lgbm done in 0.1s
+[   11.9s] zs_dann__lapshot n=5 rep=0 base=zs_dann done in 0.1s
+[   12.0s] zs_dann__lapshot n=5 rep=1 base=zs_dann done in 0.1s
+[   12.1s] zs_dann__lapshot n=5 rep=2 base=zs_dann done in 0.1s
+[   12.3s] zs_dann__lapshot n=5 rep=3 base=zs_dann done in 0.1s
+[   12.4s] zs_dann__lapshot n=5 rep=4 base=zs_dann done in 0.1s
+[   12.6s] zs_dann__lapshot n=25 rep=0 base=zs_dann done in 0.2s
+[   12.7s] zs_dann__lapshot n=25 rep=1 base=zs_dann done in 0.1s
+[   12.8s] zs_dann__lapshot n=25 rep=2 base=zs_dann done in 0.1s
+[   13.0s] zs_dann__lapshot n=25 rep=3 base=zs_dann done in 0.1s
+[   13.1s] zs_dann__lapshot n=25 rep=4 base=zs_dann done in 0.1s
+[   13.2s] zs_dann__lapshot n=50 rep=0 base=zs_dann done in 0.1s
+[   13.4s] zs_dann__lapshot n=50 rep=1 base=zs_dann done in 0.1s
+[   13.5s] zs_dann__lapshot n=50 rep=2 base=zs_dann done in 0.1s
+[   13.6s] zs_dann__lapshot n=50 rep=3 base=zs_dann done in 0.1s
+[   13.7s] zs_dann__lapshot n=50 rep=4 base=zs_dann done in 0.1s
+[   13.9s] zs_dann__lapshot n=100 rep=0 base=zs_dann done in 0.1s
+[   14.0s] zs_dann__lapshot n=100 rep=1 base=zs_dann done in 0.1s
+[   14.1s] zs_dann__lapshot n=100 rep=2 base=zs_dann done in 0.1s
+[   14.2s] zs_dann__lapshot n=100 rep=3 base=zs_dann done in 0.1s
+[   14.4s] zs_dann__lapshot n=100 rep=4 base=zs_dann done in 0.1s
+[   14.5s] zs_dann__lapshot n=200 rep=0 base=zs_dann done in 0.1s
+[   14.6s] zs_dann__lapshot n=200 rep=1 base=zs_dann done in 0.1s
+[   14.7s] zs_dann__lapshot n=200 rep=2 base=zs_dann done in 0.1s
+[   14.9s] zs_dann__lapshot n=200 rep=3 base=zs_dann done in 0.1s
+[   15.0s] zs_dann__lapshot n=200 rep=4 base=zs_dann done in 0.1s
+[   15.0s] done
+[    0.9s] loaded: Madrid (76263, 393), Amsterdam (25992, 393)
+[    0.9s] graph: 583310 edges, mean degree 22.4, min 10
+[    1.1s] lgbm_upw__em n=5 rep=0 base=lgbm_upw done in 0.2s
+[    1.3s] lgbm_upw__em n=5 rep=1 base=lgbm_upw done in 0.2s
+[    1.4s] lgbm_upw__em n=5 rep=2 base=lgbm_upw done in 0.2s
+[    1.6s] lgbm_upw__em n=5 rep=3 base=lgbm_upw done in 0.2s
+[    1.8s] lgbm_upw__em n=5 rep=4 base=lgbm_upw done in 0.1s
+[    1.9s] lgbm_upw__em n=25 rep=0 base=lgbm_upw done in 0.2s
+[    2.1s] lgbm_upw__em n=25 rep=1 base=lgbm_upw done in 0.2s
+[    2.3s] lgbm_upw__em n=25 rep=2 base=lgbm_upw done in 0.2s
+[    2.5s] lgbm_upw__em n=25 rep=3 base=lgbm_upw done in 0.2s
+[    2.6s] lgbm_upw__em n=25 rep=4 base=lgbm_upw done in 0.1s
+[    2.8s] lgbm_upw__em n=50 rep=0 base=lgbm_upw done in 0.2s
+[    3.0s] lgbm_upw__em n=50 rep=1 base=lgbm_upw done in 0.2s
+[    3.1s] lgbm_upw__em n=50 rep=2 base=lgbm_upw done in 0.2s
+[    3.3s] lgbm_upw__em n=50 rep=3 base=lgbm_upw done in 0.2s
+[    3.4s] lgbm_upw__em n=50 rep=4 base=lgbm_upw done in 0.1s
+[    3.6s] lgbm_upw__em n=100 rep=0 base=lgbm_upw done in 0.2s
+[    3.7s] lgbm_upw__em n=100 rep=1 base=lgbm_upw done in 0.1s
+[    3.9s] lgbm_upw__em n=100 rep=2 base=lgbm_upw done in 0.1s
+[    4.0s] lgbm_upw__em n=100 rep=3 base=lgbm_upw done in 0.1s
+[    4.2s] lgbm_upw__em n=100 rep=4 base=lgbm_upw done in 0.2s
+[    4.4s] lgbm_upw__em n=200 rep=0 base=lgbm_upw done in 0.2s
+[    4.5s] lgbm_upw__em n=200 rep=1 base=lgbm_upw done in 0.1s
+[    4.7s] lgbm_upw__em n=200 rep=2 base=lgbm_upw done in 0.2s
+[    4.9s] lgbm_upw__em n=200 rep=3 base=lgbm_upw done in 0.1s
+[    5.0s] lgbm_upw__em n=200 rep=4 base=lgbm_upw done in 0.2s
+[    5.2s] zs_xgb_dann__em n=0 rep=0 base=zs_xgb_dann done in 0.1s
+[    5.4s] zs_lgbm__em n=0 rep=0 base=zs_lgbm done in 0.2s
+[    5.5s] zs_dann__em n=0 rep=0 base=zs_dann done in 0.2s
+[    5.5s] done
+[    0.9s] loaded: Madrid (76263, 393), Amsterdam (25992, 393)
+[    1.0s] graph: 583310 edges, mean degree 22.4, min 10
+[    1.6s] ens_e1__clamp n=5 rep=0 base=ens_e1 done in 0.6s
+[    1.6s] ens_e1__clamp n=5 rep=1 base=ens_e1 done in 0.0s
+[    1.7s] ens_e1__clamp n=5 rep=2 base=ens_e1 done in 0.0s
+[    1.7s] ens_e1__clamp n=5 rep=3 base=ens_e1 done in 0.0s
+[    1.7s] ens_e1__clamp n=5 rep=4 base=ens_e1 done in 0.0s
+[    1.8s] ens_e1__clamp n=25 rep=0 base=ens_e1 done in 0.0s
+[    1.8s] ens_e1__clamp n=25 rep=1 base=ens_e1 done in 0.0s
+[    1.9s] ens_e1__clamp n=25 rep=2 base=ens_e1 done in 0.0s
+[    1.9s] ens_e1__clamp n=25 rep=3 base=ens_e1 done in 0.0s
+[    1.9s] ens_e1__clamp n=25 rep=4 base=ens_e1 done in 0.0s
+[    2.0s] ens_e1__clamp n=50 rep=0 base=ens_e1 done in 0.0s
+[    2.0s] ens_e1__clamp n=50 rep=1 base=ens_e1 done in 0.0s
+[    2.0s] ens_e1__clamp n=50 rep=2 base=ens_e1 done in 0.0s
+[    2.1s] ens_e1__clamp n=50 rep=3 base=ens_e1 done in 0.0s
+[    2.1s] ens_e1__clamp n=50 rep=4 base=ens_e1 done in 0.0s
+[    2.2s] ens_e1__clamp n=100 rep=0 base=ens_e1 done in 0.1s
+[    2.2s] ens_e1__clamp n=100 rep=1 base=ens_e1 done in 0.0s
+[    2.2s] ens_e1__clamp n=100 rep=2 base=ens_e1 done in 0.0s
+[    2.3s] ens_e1__clamp n=100 rep=3 base=ens_e1 done in 0.0s
+[    2.3s] ens_e1__clamp n=100 rep=4 base=ens_e1 done in 0.0s
+[    2.4s] ens_e1__clamp n=200 rep=0 base=ens_e1 done in 0.0s
+[    2.4s] ens_e1__clamp n=200 rep=1 base=ens_e1 done in 0.0s
+[    2.4s] ens_e1__clamp n=200 rep=2 base=ens_e1 done in 0.0s
+[    2.4s] ens_e1__clamp n=200 rep=3 base=ens_e1 done in 0.0s
+[    2.5s] ens_e1__clamp n=200 rep=4 base=ens_e1 done in 0.0s
+[    2.5s] ens_e2__clamp n=5 rep=0 base=ens_e2 done in 0.0s
+[    2.6s] ens_e2__clamp n=5 rep=1 base=ens_e2 done in 0.0s
+[    2.6s] ens_e2__clamp n=5 rep=2 base=ens_e2 done in 0.0s
+[    2.6s] ens_e2__clamp n=5 rep=3 base=ens_e2 done in 0.0s
+[    2.7s] ens_e2__clamp n=5 rep=4 base=ens_e2 done in 0.0s
+[    2.7s] ens_e2__clamp n=25 rep=0 base=ens_e2 done in 0.0s
+[    2.8s] ens_e2__clamp n=25 rep=1 base=ens_e2 done in 0.0s
+[    2.8s] ens_e2__clamp n=25 rep=2 base=ens_e2 done in 0.0s
+[    2.8s] ens_e2__clamp n=25 rep=3 base=ens_e2 done in 0.0s
+[    2.9s] ens_e2__clamp n=25 rep=4 base=ens_e2 done in 0.0s
+[    2.9s] ens_e2__clamp n=50 rep=0 base=ens_e2 done in 0.0s
+[    3.0s] ens_e2__clamp n=50 rep=1 base=ens_e2 done in 0.0s
+[    3.0s] ens_e2__clamp n=50 rep=2 base=ens_e2 done in 0.1s
+[    3.1s] ens_e2__clamp n=50 rep=3 base=ens_e2 done in 0.1s
+[    3.1s] ens_e2__clamp n=50 rep=4 base=ens_e2 done in 0.0s
+[    3.2s] ens_e2__clamp n=100 rep=0 base=ens_e2 done in 0.0s
+[    3.2s] ens_e2__clamp n=100 rep=1 base=ens_e2 done in 0.0s
+[    3.2s] ens_e2__clamp n=100 rep=2 base=ens_e2 done in 0.0s
+[    3.3s] ens_e2__clamp n=100 rep=3 base=ens_e2 done in 0.0s
+[    3.3s] ens_e2__clamp n=100 rep=4 base=ens_e2 done in 0.0s
+[    3.3s] ens_e2__clamp n=200 rep=0 base=ens_e2 done in 0.0s
+[    3.3s] ens_e2__clamp n=200 rep=1 base=ens_e2 done in 0.0s
+[    3.4s] ens_e2__clamp n=200 rep=2 base=ens_e2 done in 0.0s
+[    3.4s] ens_e2__clamp n=200 rep=3 base=ens_e2 done in 0.0s
+[    3.4s] ens_e2__clamp n=200 rep=4 base=ens_e2 done in 0.0s
+[    3.4s] ens_e3__clamp n=5 rep=0 base=ens_e3 done in 0.0s
+[    3.5s] ens_e3__clamp n=5 rep=1 base=ens_e3 done in 0.0s
+[    3.5s] ens_e3__clamp n=5 rep=2 base=ens_e3 done in 0.0s
+[    3.5s] ens_e3__clamp n=5 rep=3 base=ens_e3 done in 0.0s
+[    3.5s] ens_e3__clamp n=5 rep=4 base=ens_e3 done in 0.0s
+[    3.5s] ens_e3__clamp n=25 rep=0 base=ens_e3 done in 0.0s
+[    3.6s] ens_e3__clamp n=25 rep=1 base=ens_e3 done in 0.0s
+[    3.6s] ens_e3__clamp n=25 rep=2 base=ens_e3 done in 0.0s
+[    3.6s] ens_e3__clamp n=25 rep=3 base=ens_e3 done in 0.0s
+[    3.7s] ens_e3__clamp n=25 rep=4 base=ens_e3 done in 0.0s
+[    3.7s] ens_e3__clamp n=50 rep=0 base=ens_e3 done in 0.0s
+[    3.7s] ens_e3__clamp n=50 rep=1 base=ens_e3 done in 0.0s
+[    3.7s] ens_e3__clamp n=50 rep=2 base=ens_e3 done in 0.0s
+[    3.7s] ens_e3__clamp n=50 rep=3 base=ens_e3 done in 0.0s
+[    3.8s] ens_e3__clamp n=50 rep=4 base=ens_e3 done in 0.0s
+[    3.8s] ens_e3__clamp n=100 rep=0 base=ens_e3 done in 0.0s
+[    3.8s] ens_e3__clamp n=100 rep=1 base=ens_e3 done in 0.0s
+[    3.8s] ens_e3__clamp n=100 rep=2 base=ens_e3 done in 0.0s
+[    3.8s] ens_e3__clamp n=100 rep=3 base=ens_e3 done in 0.0s
+[    3.9s] ens_e3__clamp n=100 rep=4 base=ens_e3 done in 0.0s
+[    3.9s] ens_e3__clamp n=200 rep=0 base=ens_e3 done in 0.0s
+[    3.9s] ens_e3__clamp n=200 rep=1 base=ens_e3 done in 0.0s
+[    3.9s] ens_e3__clamp n=200 rep=2 base=ens_e3 done in 0.0s
+[    4.0s] ens_e3__clamp n=200 rep=3 base=ens_e3 done in 0.0s
+[    4.0s] ens_e3__clamp n=200 rep=4 base=ens_e3 done in 0.0s
+[    4.0s] done
+[    0.9s] loaded: Madrid (76263, 393), Amsterdam (25992, 393)
+[    0.9s] graph: 583310 edges, mean degree 22.4, min 10
+[    1.1s] ens_e1__lp n=5 rep=0 base=ens_e1 done in 0.1s
+[    1.2s] ens_e1__lp n=5 rep=1 base=ens_e1 done in 0.1s
+[    1.3s] ens_e1__lp n=5 rep=2 base=ens_e1 done in 0.1s
+[    1.4s] ens_e1__lp n=5 rep=3 base=ens_e1 done in 0.1s
+[    1.6s] ens_e1__lp n=5 rep=4 base=ens_e1 done in 0.1s
+[    1.7s] ens_e1__lp n=25 rep=0 base=ens_e1 done in 0.1s
+[    1.8s] ens_e1__lp n=25 rep=1 base=ens_e1 done in 0.1s
+[    2.0s] ens_e1__lp n=25 rep=2 base=ens_e1 done in 0.1s
+[    2.1s] ens_e1__lp n=25 rep=3 base=ens_e1 done in 0.1s
+[    2.3s] ens_e1__lp n=25 rep=4 base=ens_e1 done in 0.1s
+[    2.4s] ens_e1__lp n=50 rep=0 base=ens_e1 done in 0.1s
+[    2.5s] ens_e1__lp n=50 rep=1 base=ens_e1 done in 0.1s
+[    2.7s] ens_e1__lp n=50 rep=2 base=ens_e1 done in 0.1s
+[    2.8s] ens_e1__lp n=50 rep=3 base=ens_e1 done in 0.1s
+[    2.9s] ens_e1__lp n=50 rep=4 base=ens_e1 done in 0.2s
+[    3.1s] ens_e1__lp n=100 rep=0 base=ens_e1 done in 0.1s
+[    3.2s] ens_e1__lp n=100 rep=1 base=ens_e1 done in 0.2s
+[    3.3s] ens_e1__lp n=100 rep=2 base=ens_e1 done in 0.1s
+[    3.5s] ens_e1__lp n=100 rep=3 base=ens_e1 done in 0.1s
+[    3.6s] ens_e1__lp n=100 rep=4 base=ens_e1 done in 0.1s
+[    3.7s] ens_e1__lp n=200 rep=0 base=ens_e1 done in 0.1s
+[    3.9s] ens_e1__lp n=200 rep=1 base=ens_e1 done in 0.1s
+[    4.0s] ens_e1__lp n=200 rep=2 base=ens_e1 done in 0.1s
+[    4.1s] ens_e1__lp n=200 rep=3 base=ens_e1 done in 0.1s
+[    4.2s] ens_e1__lp n=200 rep=4 base=ens_e1 done in 0.1s
+[    4.3s] ens_e2__lp n=5 rep=0 base=ens_e2 done in 0.1s
+[    4.5s] ens_e2__lp n=5 rep=1 base=ens_e2 done in 0.1s
+[    4.6s] ens_e2__lp n=5 rep=2 base=ens_e2 done in 0.1s
+[    4.7s] ens_e2__lp n=5 rep=3 base=ens_e2 done in 0.1s
+[    4.9s] ens_e2__lp n=5 rep=4 base=ens_e2 done in 0.1s
+[    5.0s] ens_e2__lp n=25 rep=0 base=ens_e2 done in 0.1s
+[    5.1s] ens_e2__lp n=25 rep=1 base=ens_e2 done in 0.1s
+[    5.3s] ens_e2__lp n=25 rep=2 base=ens_e2 done in 0.1s
+[    5.4s] ens_e2__lp n=25 rep=3 base=ens_e2 done in 0.1s
+[    5.5s] ens_e2__lp n=25 rep=4 base=ens_e2 done in 0.1s
+[    5.6s] ens_e2__lp n=50 rep=0 base=ens_e2 done in 0.1s
+[    5.8s] ens_e2__lp n=50 rep=1 base=ens_e2 done in 0.1s
+[    5.9s] ens_e2__lp n=50 rep=2 base=ens_e2 done in 0.1s
+[    6.1s] ens_e2__lp n=50 rep=3 base=ens_e2 done in 0.1s
+[    6.2s] ens_e2__lp n=50 rep=4 base=ens_e2 done in 0.2s
+[    6.4s] ens_e2__lp n=100 rep=0 base=ens_e2 done in 0.2s
+[    6.5s] ens_e2__lp n=100 rep=1 base=ens_e2 done in 0.1s
+[    6.7s] ens_e2__lp n=100 rep=2 base=ens_e2 done in 0.1s
+[    6.8s] ens_e2__lp n=100 rep=3 base=ens_e2 done in 0.2s
+[    7.0s] ens_e2__lp n=100 rep=4 base=ens_e2 done in 0.2s
+[    7.1s] ens_e2__lp n=200 rep=0 base=ens_e2 done in 0.2s
+[    7.3s] ens_e2__lp n=200 rep=1 base=ens_e2 done in 0.2s
+[    7.4s] ens_e2__lp n=200 rep=2 base=ens_e2 done in 0.2s
+[    7.6s] ens_e2__lp n=200 rep=3 base=ens_e2 done in 0.1s
+[    7.7s] ens_e2__lp n=200 rep=4 base=ens_e2 done in 0.1s
+[    7.9s] ens_e3__lp n=5 rep=0 base=ens_e3 done in 0.1s
+[    8.0s] ens_e3__lp n=5 rep=1 base=ens_e3 done in 0.2s
+[    8.2s] ens_e3__lp n=5 rep=2 base=ens_e3 done in 0.2s
+[    8.3s] ens_e3__lp n=5 rep=3 base=ens_e3 done in 0.1s
+[    8.4s] ens_e3__lp n=5 rep=4 base=ens_e3 done in 0.1s
+[    8.6s] ens_e3__lp n=25 rep=0 base=ens_e3 done in 0.2s
+[    8.7s] ens_e3__lp n=25 rep=1 base=ens_e3 done in 0.2s
+[    8.9s] ens_e3__lp n=25 rep=2 base=ens_e3 done in 0.2s
+[    9.1s] ens_e3__lp n=25 rep=3 base=ens_e3 done in 0.2s
+[    9.2s] ens_e3__lp n=25 rep=4 base=ens_e3 done in 0.2s
+[    9.4s] ens_e3__lp n=50 rep=0 base=ens_e3 done in 0.2s
+[    9.6s] ens_e3__lp n=50 rep=1 base=ens_e3 done in 0.2s
+[    9.7s] ens_e3__lp n=50 rep=2 base=ens_e3 done in 0.2s
+[    9.8s] ens_e3__lp n=50 rep=3 base=ens_e3 done in 0.1s
+[   10.0s] ens_e3__lp n=50 rep=4 base=ens_e3 done in 0.1s
+[   10.1s] ens_e3__lp n=100 rep=0 base=ens_e3 done in 0.1s
+[   10.3s] ens_e3__lp n=100 rep=1 base=ens_e3 done in 0.2s
+[   10.5s] ens_e3__lp n=100 rep=2 base=ens_e3 done in 0.1s
+[   10.6s] ens_e3__lp n=100 rep=3 base=ens_e3 done in 0.2s
+[   10.8s] ens_e3__lp n=100 rep=4 base=ens_e3 done in 0.1s
+[   10.9s] ens_e3__lp n=200 rep=0 base=ens_e3 done in 0.1s
+[   11.0s] ens_e3__lp n=200 rep=1 base=ens_e3 done in 0.1s
+[   11.2s] ens_e3__lp n=200 rep=2 base=ens_e3 done in 0.1s
+[   11.3s] ens_e3__lp n=200 rep=3 base=ens_e3 done in 0.1s
+[   11.4s] ens_e3__lp n=200 rep=4 base=ens_e3 done in 0.1s
+[   11.4s] done
+[    0.9s] loaded: Madrid (76263, 393), Amsterdam (25992, 393)
+[    1.0s] graph: 583310 edges, mean degree 22.4, min 10
+[    1.1s] ens_e1__lapshot n=5 rep=0 base=ens_e1 done in 0.1s
+[    1.2s] ens_e1__lapshot n=5 rep=1 base=ens_e1 done in 0.1s
+[    1.3s] ens_e1__lapshot n=5 rep=2 base=ens_e1 done in 0.1s
+[    1.5s] ens_e1__lapshot n=5 rep=3 base=ens_e1 done in 0.1s
+[    1.6s] ens_e1__lapshot n=5 rep=4 base=ens_e1 done in 0.1s
+[    1.7s] ens_e1__lapshot n=25 rep=0 base=ens_e1 done in 0.2s
+[    1.9s] ens_e1__lapshot n=25 rep=1 base=ens_e1 done in 0.1s
+[    2.0s] ens_e1__lapshot n=25 rep=2 base=ens_e1 done in 0.1s
+[    2.1s] ens_e1__lapshot n=25 rep=3 base=ens_e1 done in 0.1s
+[    2.3s] ens_e1__lapshot n=25 rep=4 base=ens_e1 done in 0.1s
+[    2.4s] ens_e1__lapshot n=50 rep=0 base=ens_e1 done in 0.1s
+[    2.5s] ens_e1__lapshot n=50 rep=1 base=ens_e1 done in 0.1s
+[    2.7s] ens_e1__lapshot n=50 rep=2 base=ens_e1 done in 0.1s
+[    2.8s] ens_e1__lapshot n=50 rep=3 base=ens_e1 done in 0.1s
+[    2.9s] ens_e1__lapshot n=50 rep=4 base=ens_e1 done in 0.1s
+[    3.0s] ens_e1__lapshot n=100 rep=0 base=ens_e1 done in 0.1s
+[    3.2s] ens_e1__lapshot n=100 rep=1 base=ens_e1 done in 0.1s
+[    3.3s] ens_e1__lapshot n=100 rep=2 base=ens_e1 done in 0.1s
+[    3.4s] ens_e1__lapshot n=100 rep=3 base=ens_e1 done in 0.1s
+[    3.5s] ens_e1__lapshot n=100 rep=4 base=ens_e1 done in 0.1s
+[    3.7s] ens_e1__lapshot n=200 rep=0 base=ens_e1 done in 0.1s
+[    3.8s] ens_e1__lapshot n=200 rep=1 base=ens_e1 done in 0.1s
+[    3.9s] ens_e1__lapshot n=200 rep=2 base=ens_e1 done in 0.1s
+[    4.1s] ens_e1__lapshot n=200 rep=3 base=ens_e1 done in 0.1s
+[    4.2s] ens_e1__lapshot n=200 rep=4 base=ens_e1 done in 0.1s
+[    4.3s] ens_e2__lapshot n=5 rep=0 base=ens_e2 done in 0.1s
+[    4.5s] ens_e2__lapshot n=5 rep=1 base=ens_e2 done in 0.1s
+[    4.6s] ens_e2__lapshot n=5 rep=2 base=ens_e2 done in 0.1s
+[    4.7s] ens_e2__lapshot n=5 rep=3 base=ens_e2 done in 0.1s
+[    4.9s] ens_e2__lapshot n=5 rep=4 base=ens_e2 done in 0.1s
+[    5.0s] ens_e2__lapshot n=25 rep=0 base=ens_e2 done in 0.1s
+[    5.1s] ens_e2__lapshot n=25 rep=1 base=ens_e2 done in 0.1s
+[    5.2s] ens_e2__lapshot n=25 rep=2 base=ens_e2 done in 0.1s
+[    5.3s] ens_e2__lapshot n=25 rep=3 base=ens_e2 done in 0.1s
+[    5.5s] ens_e2__lapshot n=25 rep=4 base=ens_e2 done in 0.1s
+[    5.6s] ens_e2__lapshot n=50 rep=0 base=ens_e2 done in 0.1s
+[    5.7s] ens_e2__lapshot n=50 rep=1 base=ens_e2 done in 0.1s
+[    5.9s] ens_e2__lapshot n=50 rep=2 base=ens_e2 done in 0.1s
+[    6.0s] ens_e2__lapshot n=50 rep=3 base=ens_e2 done in 0.1s
+[    6.1s] ens_e2__lapshot n=50 rep=4 base=ens_e2 done in 0.1s
+[    6.3s] ens_e2__lapshot n=100 rep=0 base=ens_e2 done in 0.1s
+[    6.4s] ens_e2__lapshot n=100 rep=1 base=ens_e2 done in 0.1s
+[    6.5s] ens_e2__lapshot n=100 rep=2 base=ens_e2 done in 0.1s
+[    6.6s] ens_e2__lapshot n=100 rep=3 base=ens_e2 done in 0.1s
+[    6.8s] ens_e2__lapshot n=100 rep=4 base=ens_e2 done in 0.1s
+[    6.9s] ens_e2__lapshot n=200 rep=0 base=ens_e2 done in 0.1s
+[    7.0s] ens_e2__lapshot n=200 rep=1 base=ens_e2 done in 0.1s
+[    7.2s] ens_e2__lapshot n=200 rep=2 base=ens_e2 done in 0.2s
+[    7.3s] ens_e2__lapshot n=200 rep=3 base=ens_e2 done in 0.1s
+[    7.4s] ens_e2__lapshot n=200 rep=4 base=ens_e2 done in 0.1s
+[    7.6s] ens_e3__lapshot n=5 rep=0 base=ens_e3 done in 0.1s
+[    7.7s] ens_e3__lapshot n=5 rep=1 base=ens_e3 done in 0.1s
+[    7.8s] ens_e3__lapshot n=5 rep=2 base=ens_e3 done in 0.1s
+[    8.0s] ens_e3__lapshot n=5 rep=3 base=ens_e3 done in 0.1s
+[    8.1s] ens_e3__lapshot n=5 rep=4 base=ens_e3 done in 0.1s
+[    8.2s] ens_e3__lapshot n=25 rep=0 base=ens_e3 done in 0.1s
+[    8.4s] ens_e3__lapshot n=25 rep=1 base=ens_e3 done in 0.1s
+[    8.5s] ens_e3__lapshot n=25 rep=2 base=ens_e3 done in 0.1s
+[    8.6s] ens_e3__lapshot n=25 rep=3 base=ens_e3 done in 0.1s
+[    8.8s] ens_e3__lapshot n=25 rep=4 base=ens_e3 done in 0.1s
+[    8.9s] ens_e3__lapshot n=50 rep=0 base=ens_e3 done in 0.1s
+[    9.0s] ens_e3__lapshot n=50 rep=1 base=ens_e3 done in 0.1s
+[    9.1s] ens_e3__lapshot n=50 rep=2 base=ens_e3 done in 0.1s
+[    9.2s] ens_e3__lapshot n=50 rep=3 base=ens_e3 done in 0.1s
+[    9.3s] ens_e3__lapshot n=50 rep=4 base=ens_e3 done in 0.1s
+[    9.4s] ens_e3__lapshot n=100 rep=0 base=ens_e3 done in 0.1s
+[    9.5s] ens_e3__lapshot n=100 rep=1 base=ens_e3 done in 0.1s
+[    9.7s] ens_e3__lapshot n=100 rep=2 base=ens_e3 done in 0.1s
+[    9.8s] ens_e3__lapshot n=100 rep=3 base=ens_e3 done in 0.2s
+[   10.0s] ens_e3__lapshot n=100 rep=4 base=ens_e3 done in 0.2s
+[   10.1s] ens_e3__lapshot n=200 rep=0 base=ens_e3 done in 0.1s
+[   10.3s] ens_e3__lapshot n=200 rep=1 base=ens_e3 done in 0.1s
+[   10.4s] ens_e3__lapshot n=200 rep=2 base=ens_e3 done in 0.2s
+[   10.6s] ens_e3__lapshot n=200 rep=3 base=ens_e3 done in 0.1s
+[   10.7s] ens_e3__lapshot n=200 rep=4 base=ens_e3 done in 0.1s
+[   10.7s] done
+[    0.7s] loaded: Madrid (76263, 393), Amsterdam (25992, 393)
+[    0.7s] graph: 583310 edges, mean degree 22.4, min 10
+[    0.9s] ens_e4__lp n=5 rep=0 base=ens_e4 done in 0.1s
+[    1.0s] ens_e4__lp n=5 rep=1 base=ens_e4 done in 0.1s
+[    1.1s] ens_e4__lp n=5 rep=2 base=ens_e4 done in 0.1s
+[    1.2s] ens_e4__lp n=5 rep=3 base=ens_e4 done in 0.1s
+[    1.4s] ens_e4__lp n=5 rep=4 base=ens_e4 done in 0.1s
+[    1.5s] ens_e4__lp n=25 rep=0 base=ens_e4 done in 0.1s
+[    1.6s] ens_e4__lp n=25 rep=1 base=ens_e4 done in 0.2s
+[    1.8s] ens_e4__lp n=25 rep=2 base=ens_e4 done in 0.2s
+[    1.9s] ens_e4__lp n=25 rep=3 base=ens_e4 done in 0.1s
+[    2.1s] ens_e4__lp n=25 rep=4 base=ens_e4 done in 0.1s
+[    2.2s] ens_e4__lp n=50 rep=0 base=ens_e4 done in 0.1s
+[    2.3s] ens_e4__lp n=50 rep=1 base=ens_e4 done in 0.1s
+[    2.5s] ens_e4__lp n=50 rep=2 base=ens_e4 done in 0.1s
+[    2.6s] ens_e4__lp n=50 rep=3 base=ens_e4 done in 0.1s
+[    2.7s] ens_e4__lp n=50 rep=4 base=ens_e4 done in 0.1s
+[    2.9s] ens_e4__lp n=100 rep=0 base=ens_e4 done in 0.1s
+[    2.9s] ens_e4__lp n=100 rep=1 base=ens_e4 done in 0.1s
+[    3.1s] ens_e4__lp n=100 rep=2 base=ens_e4 done in 0.1s
+[    3.2s] ens_e4__lp n=100 rep=3 base=ens_e4 done in 0.1s
+[    3.4s] ens_e4__lp n=100 rep=4 base=ens_e4 done in 0.2s
+[    3.6s] ens_e4__lp n=200 rep=0 base=ens_e4 done in 0.2s
+[    3.7s] ens_e4__lp n=200 rep=1 base=ens_e4 done in 0.2s
+[    3.9s] ens_e4__lp n=200 rep=2 base=ens_e4 done in 0.1s
+[    4.0s] ens_e4__lp n=200 rep=3 base=ens_e4 done in 0.1s
+[    4.2s] ens_e4__lp n=200 rep=4 base=ens_e4 done in 0.2s
+[    4.3s] ens_e5__lp n=5 rep=0 base=ens_e5 done in 0.1s
+[    4.4s] ens_e5__lp n=5 rep=1 base=ens_e5 done in 0.1s
+[    4.6s] ens_e5__lp n=5 rep=2 base=ens_e5 done in 0.2s
+[    4.7s] ens_e5__lp n=5 rep=3 base=ens_e5 done in 0.1s
+[    4.9s] ens_e5__lp n=5 rep=4 base=ens_e5 done in 0.1s
+[    5.0s] ens_e5__lp n=25 rep=0 base=ens_e5 done in 0.2s
+[    5.2s] ens_e5__lp n=25 rep=1 base=ens_e5 done in 0.1s
+[    5.3s] ens_e5__lp n=25 rep=2 base=ens_e5 done in 0.1s
+[    5.4s] ens_e5__lp n=25 rep=3 base=ens_e5 done in 0.1s
+[    5.5s] ens_e5__lp n=25 rep=4 base=ens_e5 done in 0.1s
+[    5.6s] ens_e5__lp n=50 rep=0 base=ens_e5 done in 0.1s
+[    5.7s] ens_e5__lp n=50 rep=1 base=ens_e5 done in 0.1s
+[    5.8s] ens_e5__lp n=50 rep=2 base=ens_e5 done in 0.1s
+[    5.9s] ens_e5__lp n=50 rep=3 base=ens_e5 done in 0.1s
+[    6.0s] ens_e5__lp n=50 rep=4 base=ens_e5 done in 0.1s
+[    6.1s] ens_e5__lp n=100 rep=0 base=ens_e5 done in 0.1s
+[    6.3s] ens_e5__lp n=100 rep=1 base=ens_e5 done in 0.1s
+[    6.4s] ens_e5__lp n=100 rep=2 base=ens_e5 done in 0.1s
+[    6.4s] ens_e5__lp n=100 rep=3 base=ens_e5 done in 0.1s
+[    6.6s] ens_e5__lp n=100 rep=4 base=ens_e5 done in 0.1s
+[    6.7s] ens_e5__lp n=200 rep=0 base=ens_e5 done in 0.1s
+[    6.8s] ens_e5__lp n=200 rep=1 base=ens_e5 done in 0.1s
+[    6.9s] ens_e5__lp n=200 rep=2 base=ens_e5 done in 0.1s
+[    7.0s] ens_e5__lp n=200 rep=3 base=ens_e5 done in 0.1s
+[    7.1s] ens_e5__lp n=200 rep=4 base=ens_e5 done in 0.1s
+[    7.2s] ens_e6__lp n=5 rep=0 base=ens_e6 done in 0.1s
+[    7.3s] ens_e6__lp n=5 rep=1 base=ens_e6 done in 0.1s
+[    7.4s] ens_e6__lp n=5 rep=2 base=ens_e6 done in 0.1s
+[    7.5s] ens_e6__lp n=5 rep=3 base=ens_e6 done in 0.1s
+[    7.6s] ens_e6__lp n=5 rep=4 base=ens_e6 done in 0.1s
+[    7.7s] ens_e6__lp n=25 rep=0 base=ens_e6 done in 0.1s
+[    7.8s] ens_e6__lp n=25 rep=1 base=ens_e6 done in 0.1s
+[    7.9s] ens_e6__lp n=25 rep=2 base=ens_e6 done in 0.1s
+[    8.0s] ens_e6__lp n=25 rep=3 base=ens_e6 done in 0.1s
+[    8.1s] ens_e6__lp n=25 rep=4 base=ens_e6 done in 0.1s
+[    8.2s] ens_e6__lp n=50 rep=0 base=ens_e6 done in 0.1s
+[    8.3s] ens_e6__lp n=50 rep=1 base=ens_e6 done in 0.1s
+[    8.4s] ens_e6__lp n=50 rep=2 base=ens_e6 done in 0.1s
+[    8.5s] ens_e6__lp n=50 rep=3 base=ens_e6 done in 0.1s
+[    8.7s] ens_e6__lp n=50 rep=4 base=ens_e6 done in 0.1s
+[    8.8s] ens_e6__lp n=100 rep=0 base=ens_e6 done in 0.1s
+[    8.9s] ens_e6__lp n=100 rep=1 base=ens_e6 done in 0.1s
+[    9.0s] ens_e6__lp n=100 rep=2 base=ens_e6 done in 0.1s
+[    9.1s] ens_e6__lp n=100 rep=3 base=ens_e6 done in 0.1s
+[    9.2s] ens_e6__lp n=100 rep=4 base=ens_e6 done in 0.1s
+[    9.3s] ens_e6__lp n=200 rep=0 base=ens_e6 done in 0.1s
+[    9.4s] ens_e6__lp n=200 rep=1 base=ens_e6 done in 0.1s
+[    9.5s] ens_e6__lp n=200 rep=2 base=ens_e6 done in 0.1s
+[    9.6s] ens_e6__lp n=200 rep=3 base=ens_e6 done in 0.1s
+[    9.6s] ens_e6__lp n=200 rep=4 base=ens_e6 done in 0.1s
+[    9.6s] done
+[    0.8s] loaded: Madrid (76263, 393), Amsterdam (25992, 393)
+[    0.8s] graph: 583310 edges, mean degree 22.4, min 10
+[    0.9s] ens_e4__lapshot n=5 rep=0 base=ens_e4 done in 0.1s
+[    1.0s] ens_e4__lapshot n=5 rep=1 base=ens_e4 done in 0.1s
+[    1.1s] ens_e4__lapshot n=5 rep=2 base=ens_e4 done in 0.1s
+[    1.2s] ens_e4__lapshot n=5 rep=3 base=ens_e4 done in 0.1s
+[    1.3s] ens_e4__lapshot n=5 rep=4 base=ens_e4 done in 0.1s
+[    1.4s] ens_e4__lapshot n=25 rep=0 base=ens_e4 done in 0.1s
+[    1.5s] ens_e4__lapshot n=25 rep=1 base=ens_e4 done in 0.1s
+[    1.6s] ens_e4__lapshot n=25 rep=2 base=ens_e4 done in 0.1s
+[    1.7s] ens_e4__lapshot n=25 rep=3 base=ens_e4 done in 0.1s
+[    1.8s] ens_e4__lapshot n=25 rep=4 base=ens_e4 done in 0.1s
+[    1.9s] ens_e4__lapshot n=50 rep=0 base=ens_e4 done in 0.1s
+[    2.0s] ens_e4__lapshot n=50 rep=1 base=ens_e4 done in 0.1s
+[    2.1s] ens_e4__lapshot n=50 rep=2 base=ens_e4 done in 0.1s
+[    2.2s] ens_e4__lapshot n=50 rep=3 base=ens_e4 done in 0.1s
+[    2.3s] ens_e4__lapshot n=50 rep=4 base=ens_e4 done in 0.1s
+[    2.4s] ens_e4__lapshot n=100 rep=0 base=ens_e4 done in 0.1s
+[    2.5s] ens_e4__lapshot n=100 rep=1 base=ens_e4 done in 0.1s
+[    2.6s] ens_e4__lapshot n=100 rep=2 base=ens_e4 done in 0.1s
+[    2.7s] ens_e4__lapshot n=100 rep=3 base=ens_e4 done in 0.1s
+[    2.8s] ens_e4__lapshot n=100 rep=4 base=ens_e4 done in 0.1s
+[    2.8s] ens_e4__lapshot n=200 rep=0 base=ens_e4 done in 0.1s
+[    3.0s] ens_e4__lapshot n=200 rep=1 base=ens_e4 done in 0.1s
+[    3.1s] ens_e4__lapshot n=200 rep=2 base=ens_e4 done in 0.1s
+[    3.1s] ens_e4__lapshot n=200 rep=3 base=ens_e4 done in 0.1s
+[    3.2s] ens_e4__lapshot n=200 rep=4 base=ens_e4 done in 0.1s
+[    3.3s] ens_e5__lapshot n=5 rep=0 base=ens_e5 done in 0.1s
+[    3.4s] ens_e5__lapshot n=5 rep=1 base=ens_e5 done in 0.1s
+[    3.5s] ens_e5__lapshot n=5 rep=2 base=ens_e5 done in 0.1s
+[    3.6s] ens_e5__lapshot n=5 rep=3 base=ens_e5 done in 0.1s
+[    3.7s] ens_e5__lapshot n=5 rep=4 base=ens_e5 done in 0.1s
+[    3.8s] ens_e5__lapshot n=25 rep=0 base=ens_e5 done in 0.1s
+[    3.9s] ens_e5__lapshot n=25 rep=1 base=ens_e5 done in 0.1s
+[    4.0s] ens_e5__lapshot n=25 rep=2 base=ens_e5 done in 0.1s
+[    4.1s] ens_e5__lapshot n=25 rep=3 base=ens_e5 done in 0.1s
+[    4.2s] ens_e5__lapshot n=25 rep=4 base=ens_e5 done in 0.1s
+[    4.2s] ens_e5__lapshot n=50 rep=0 base=ens_e5 done in 0.1s
+[    4.3s] ens_e5__lapshot n=50 rep=1 base=ens_e5 done in 0.1s
+[    4.4s] ens_e5__lapshot n=50 rep=2 base=ens_e5 done in 0.1s
+[    4.5s] ens_e5__lapshot n=50 rep=3 base=ens_e5 done in 0.1s
+[    4.6s] ens_e5__lapshot n=50 rep=4 base=ens_e5 done in 0.1s
+[    4.7s] ens_e5__lapshot n=100 rep=0 base=ens_e5 done in 0.1s
+[    4.8s] ens_e5__lapshot n=100 rep=1 base=ens_e5 done in 0.1s
+[    4.9s] ens_e5__lapshot n=100 rep=2 base=ens_e5 done in 0.1s
+[    4.9s] ens_e5__lapshot n=100 rep=3 base=ens_e5 done in 0.1s
+[    5.0s] ens_e5__lapshot n=100 rep=4 base=ens_e5 done in 0.1s
+[    5.1s] ens_e5__lapshot n=200 rep=0 base=ens_e5 done in 0.1s
+[    5.2s] ens_e5__lapshot n=200 rep=1 base=ens_e5 done in 0.1s
+[    5.3s] ens_e5__lapshot n=200 rep=2 base=ens_e5 done in 0.1s
+[    5.4s] ens_e5__lapshot n=200 rep=3 base=ens_e5 done in 0.1s
+[    5.5s] ens_e5__lapshot n=200 rep=4 base=ens_e5 done in 0.1s
+[    5.6s] ens_e6__lapshot n=5 rep=0 base=ens_e6 done in 0.1s
+[    5.7s] ens_e6__lapshot n=5 rep=1 base=ens_e6 done in 0.1s
+[    5.8s] ens_e6__lapshot n=5 rep=2 base=ens_e6 done in 0.1s
+[    5.9s] ens_e6__lapshot n=5 rep=3 base=ens_e6 done in 0.1s
+[    5.9s] ens_e6__lapshot n=5 rep=4 base=ens_e6 done in 0.1s
+[    6.0s] ens_e6__lapshot n=25 rep=0 base=ens_e6 done in 0.1s
+[    6.1s] ens_e6__lapshot n=25 rep=1 base=ens_e6 done in 0.1s
+[    6.2s] ens_e6__lapshot n=25 rep=2 base=ens_e6 done in 0.1s
+[    6.3s] ens_e6__lapshot n=25 rep=3 base=ens_e6 done in 0.1s
+[    6.4s] ens_e6__lapshot n=25 rep=4 base=ens_e6 done in 0.1s
+[    6.5s] ens_e6__lapshot n=50 rep=0 base=ens_e6 done in 0.1s
+[    6.6s] ens_e6__lapshot n=50 rep=1 base=ens_e6 done in 0.1s
+[    6.7s] ens_e6__lapshot n=50 rep=2 base=ens_e6 done in 0.1s
+[    6.8s] ens_e6__lapshot n=50 rep=3 base=ens_e6 done in 0.1s
+[    6.9s] ens_e6__lapshot n=50 rep=4 base=ens_e6 done in 0.1s
+[    7.0s] ens_e6__lapshot n=100 rep=0 base=ens_e6 done in 0.1s
+[    7.1s] ens_e6__lapshot n=100 rep=1 base=ens_e6 done in 0.1s
+[    7.2s] ens_e6__lapshot n=100 rep=2 base=ens_e6 done in 0.1s
+[    7.3s] ens_e6__lapshot n=100 rep=3 base=ens_e6 done in 0.1s
+[    7.4s] ens_e6__lapshot n=100 rep=4 base=ens_e6 done in 0.1s
+[    7.5s] ens_e6__lapshot n=200 rep=0 base=ens_e6 done in 0.1s
+[    7.7s] ens_e6__lapshot n=200 rep=1 base=ens_e6 done in 0.1s
+[    7.8s] ens_e6__lapshot n=200 rep=2 base=ens_e6 done in 0.1s
+[    7.9s] ens_e6__lapshot n=200 rep=3 base=ens_e6 done in 0.1s
+[    8.0s] ens_e6__lapshot n=200 rep=4 base=ens_e6 done in 0.1s
+[    8.0s] done
+[    1.0s] loaded: Madrid (76263, 393), Amsterdam (25992, 393)
+[    1.0s] graph: 583310 edges, mean degree 22.4, min 10
+[    1.2s] ens_e7__lp n=5 rep=0 base=ens_e7 done in 0.2s
+[    1.4s] ens_e7__lp n=5 rep=1 base=ens_e7 done in 0.2s
+[    1.6s] ens_e7__lp n=5 rep=2 base=ens_e7 done in 0.1s
+[    1.7s] ens_e7__lp n=5 rep=3 base=ens_e7 done in 0.1s
+[    1.9s] ens_e7__lp n=5 rep=4 base=ens_e7 done in 0.2s
+[    2.1s] ens_e7__lp n=25 rep=0 base=ens_e7 done in 0.2s
+[    2.2s] ens_e7__lp n=25 rep=1 base=ens_e7 done in 0.1s
+[    2.4s] ens_e7__lp n=25 rep=2 base=ens_e7 done in 0.2s
+[    2.5s] ens_e7__lp n=25 rep=3 base=ens_e7 done in 0.2s
+[    2.7s] ens_e7__lp n=25 rep=4 base=ens_e7 done in 0.2s
+[    2.8s] ens_e7__lp n=50 rep=0 base=ens_e7 done in 0.2s
+[    3.0s] ens_e7__lp n=50 rep=1 base=ens_e7 done in 0.1s
+[    3.1s] ens_e7__lp n=50 rep=2 base=ens_e7 done in 0.2s
+[    3.3s] ens_e7__lp n=50 rep=3 base=ens_e7 done in 0.2s
+[    3.4s] ens_e7__lp n=50 rep=4 base=ens_e7 done in 0.2s
+[    3.6s] ens_e7__lp n=100 rep=0 base=ens_e7 done in 0.2s
+[    3.8s] ens_e7__lp n=100 rep=1 base=ens_e7 done in 0.2s
+[    3.9s] ens_e7__lp n=100 rep=2 base=ens_e7 done in 0.2s
+[    4.1s] ens_e7__lp n=100 rep=3 base=ens_e7 done in 0.1s
+[    4.2s] ens_e7__lp n=100 rep=4 base=ens_e7 done in 0.1s
+[    4.4s] ens_e7__lp n=200 rep=0 base=ens_e7 done in 0.2s
+[    4.5s] ens_e7__lp n=200 rep=1 base=ens_e7 done in 0.1s
+[    4.7s] ens_e7__lp n=200 rep=2 base=ens_e7 done in 0.1s
+[    4.8s] ens_e7__lp n=200 rep=3 base=ens_e7 done in 0.2s
+[    5.0s] ens_e7__lp n=200 rep=4 base=ens_e7 done in 0.1s
+[    5.0s] done

reports/nnlab/logs/transductive_r12.log

@@ -0,0 +1,111 @@
+[    0.5s] loaded: Madrid (76263, 393), Amsterdam (25992, 393)
+[    0.5s] graph: 583310 edges, mean degree 22.4, min 10
+[    0.6s] ens_e4__lp n=5 rep=5 base=ens_e4 done in 0.1s
+[    0.7s] ens_e4__lp n=5 rep=6 base=ens_e4 done in 0.1s
+[    0.7s] ens_e4__lp n=5 rep=7 base=ens_e4 done in 0.1s
+[    0.8s] ens_e4__lp n=5 rep=8 base=ens_e4 done in 0.1s
+[    0.9s] ens_e4__lp n=5 rep=9 base=ens_e4 done in 0.1s
+[    1.0s] ens_e4__lp n=5 rep=10 base=ens_e4 done in 0.1s
+[    1.1s] ens_e4__lp n=5 rep=11 base=ens_e4 done in 0.1s
+[    1.1s] ens_e4__lp n=25 rep=5 base=ens_e4 done in 0.1s
+[    1.2s] ens_e4__lp n=25 rep=6 base=ens_e4 done in 0.1s
+[    1.3s] ens_e4__lp n=25 rep=7 base=ens_e4 done in 0.1s
+[    1.4s] ens_e4__lp n=25 rep=8 base=ens_e4 done in 0.1s
+[    1.4s] ens_e4__lp n=25 rep=9 base=ens_e4 done in 0.1s
+[    1.5s] ens_e4__lp n=25 rep=10 base=ens_e4 done in 0.1s
+[    1.6s] ens_e4__lp n=25 rep=11 base=ens_e4 done in 0.1s
+[    1.7s] ens_e4__lp n=50 rep=5 base=ens_e4 done in 0.1s
+[    1.7s] ens_e4__lp n=50 rep=6 base=ens_e4 done in 0.1s
+[    1.8s] ens_e4__lp n=50 rep=7 base=ens_e4 done in 0.1s
+[    1.9s] ens_e4__lp n=50 rep=8 base=ens_e4 done in 0.1s
+[    2.0s] ens_e4__lp n=50 rep=9 base=ens_e4 done in 0.1s
+[    2.1s] ens_e4__lp n=50 rep=10 base=ens_e4 done in 0.1s
+[    2.2s] ens_e4__lp n=50 rep=11 base=ens_e4 done in 0.1s
+[    2.2s] ens_e4__lp n=100 rep=5 base=ens_e4 done in 0.1s
+[    2.3s] ens_e4__lp n=100 rep=6 base=ens_e4 done in 0.1s
+[    2.4s] ens_e4__lp n=100 rep=7 base=ens_e4 done in 0.1s
+[    2.5s] ens_e4__lp n=100 rep=8 base=ens_e4 done in 0.1s
+[    2.5s] ens_e4__lp n=100 rep=9 base=ens_e4 done in 0.1s
+[    2.6s] ens_e4__lp n=100 rep=10 base=ens_e4 done in 0.1s
+[    2.7s] ens_e4__lp n=100 rep=11 base=ens_e4 done in 0.1s
+[    2.8s] ens_e4__lp n=200 rep=5 base=ens_e4 done in 0.1s
+[    2.9s] ens_e4__lp n=200 rep=6 base=ens_e4 done in 0.1s
+[    3.0s] ens_e4__lp n=200 rep=7 base=ens_e4 done in 0.1s
+[    3.0s] ens_e4__lp n=200 rep=8 base=ens_e4 done in 0.1s
+[    3.1s] ens_e4__lp n=200 rep=9 base=ens_e4 done in 0.1s
+[    3.2s] ens_e4__lp n=200 rep=10 base=ens_e4 done in 0.1s
+[    3.3s] ens_e4__lp n=200 rep=11 base=ens_e4 done in 0.1s
+[    3.4s] lgbm_upw__lp n=5 rep=5 base=lgbm_upw done in 0.1s
+[    3.5s] lgbm_upw__lp n=5 rep=6 base=lgbm_upw done in 0.1s
+[    3.5s] lgbm_upw__lp n=5 rep=7 base=lgbm_upw done in 0.1s
+[    3.6s] lgbm_upw__lp n=5 rep=8 base=lgbm_upw done in 0.1s
+[    3.7s] lgbm_upw__lp n=5 rep=9 base=lgbm_upw done in 0.1s
+[    3.8s] lgbm_upw__lp n=5 rep=10 base=lgbm_upw done in 0.1s
+[    3.9s] lgbm_upw__lp n=5 rep=11 base=lgbm_upw done in 0.1s
+[    3.9s] lgbm_upw__lp n=25 rep=5 base=lgbm_upw done in 0.1s
+[    4.0s] lgbm_upw__lp n=25 rep=6 base=lgbm_upw done in 0.1s
+[    4.1s] lgbm_upw__lp n=25 rep=7 base=lgbm_upw done in 0.1s
+[    4.2s] lgbm_upw__lp n=25 rep=8 base=lgbm_upw done in 0.1s
+[    4.2s] lgbm_upw__lp n=25 rep=9 base=lgbm_upw done in 0.1s
+[    4.3s] lgbm_upw__lp n=25 rep=10 base=lgbm_upw done in 0.1s
+[    4.4s] lgbm_upw__lp n=25 rep=11 base=lgbm_upw done in 0.1s
+[    4.5s] lgbm_upw__lp n=50 rep=5 base=lgbm_upw done in 0.1s
+[    4.5s] lgbm_upw__lp n=50 rep=6 base=lgbm_upw done in 0.1s
+[    4.6s] lgbm_upw__lp n=50 rep=7 base=lgbm_upw done in 0.1s
+[    4.7s] lgbm_upw__lp n=50 rep=8 base=lgbm_upw done in 0.1s
+[    4.8s] lgbm_upw__lp n=50 rep=9 base=lgbm_upw done in 0.1s
+[    4.9s] lgbm_upw__lp n=50 rep=10 base=lgbm_upw done in 0.1s
+[    4.9s] lgbm_upw__lp n=50 rep=11 base=lgbm_upw done in 0.1s
+[    5.0s] lgbm_upw__lp n=100 rep=5 base=lgbm_upw done in 0.1s
+[    5.1s] lgbm_upw__lp n=100 rep=6 base=lgbm_upw done in 0.1s
+[    5.2s] lgbm_upw__lp n=100 rep=7 base=lgbm_upw done in 0.1s
+[    5.2s] lgbm_upw__lp n=100 rep=8 base=lgbm_upw done in 0.1s
+[    5.3s] lgbm_upw__lp n=100 rep=9 base=lgbm_upw done in 0.1s
+[    5.4s] lgbm_upw__lp n=100 rep=10 base=lgbm_upw done in 0.1s
+[    5.5s] lgbm_upw__lp n=100 rep=11 base=lgbm_upw done in 0.1s
+[    5.5s] lgbm_upw__lp n=200 rep=5 base=lgbm_upw done in 0.1s
+[    5.6s] lgbm_upw__lp n=200 rep=6 base=lgbm_upw done in 0.1s
+[    5.7s] lgbm_upw__lp n=200 rep=7 base=lgbm_upw done in 0.1s
+[    5.8s] lgbm_upw__lp n=200 rep=8 base=lgbm_upw done in 0.1s
+[    5.8s] lgbm_upw__lp n=200 rep=9 base=lgbm_upw done in 0.1s
+[    5.9s] lgbm_upw__lp n=200 rep=10 base=lgbm_upw done in 0.1s
+[    6.0s] lgbm_upw__lp n=200 rep=11 base=lgbm_upw done in 0.1s
+[    6.0s] done
+[    0.5s] loaded: Madrid (76263, 393), Amsterdam (25992, 393)
+[    0.5s] graph: 583310 edges, mean degree 22.4, min 10
+[    0.6s] ens_e2__lapshot n=5 rep=5 base=ens_e2 done in 0.1s
+[    0.7s] ens_e2__lapshot n=5 rep=6 base=ens_e2 done in 0.1s
+[    0.8s] ens_e2__lapshot n=5 rep=7 base=ens_e2 done in 0.1s
+[    0.8s] ens_e2__lapshot n=5 rep=8 base=ens_e2 done in 0.1s
+[    0.9s] ens_e2__lapshot n=5 rep=9 base=ens_e2 done in 0.1s
+[    1.0s] ens_e2__lapshot n=5 rep=10 base=ens_e2 done in 0.1s
+[    1.1s] ens_e2__lapshot n=5 rep=11 base=ens_e2 done in 0.1s
+[    1.1s] ens_e2__lapshot n=25 rep=5 base=ens_e2 done in 0.1s
+[    1.2s] ens_e2__lapshot n=25 rep=6 base=ens_e2 done in 0.1s
+[    1.3s] ens_e2__lapshot n=25 rep=7 base=ens_e2 done in 0.1s
+[    1.4s] ens_e2__lapshot n=25 rep=8 base=ens_e2 done in 0.1s
+[    1.5s] ens_e2__lapshot n=25 rep=9 base=ens_e2 done in 0.1s
+[    1.5s] ens_e2__lapshot n=25 rep=10 base=ens_e2 done in 0.1s
+[    1.6s] ens_e2__lapshot n=25 rep=11 base=ens_e2 done in 0.1s
+[    1.7s] ens_e2__lapshot n=50 rep=5 base=ens_e2 done in 0.1s
+[    1.8s] ens_e2__lapshot n=50 rep=6 base=ens_e2 done in 0.1s
+[    1.8s] ens_e2__lapshot n=50 rep=7 base=ens_e2 done in 0.1s
+[    1.9s] ens_e2__lapshot n=50 rep=8 base=ens_e2 done in 0.1s
+[    2.0s] ens_e2__lapshot n=50 rep=9 base=ens_e2 done in 0.1s
+[    2.1s] ens_e2__lapshot n=50 rep=10 base=ens_e2 done in 0.1s
+[    2.2s] ens_e2__lapshot n=50 rep=11 base=ens_e2 done in 0.1s
+[    2.2s] ens_e2__lapshot n=100 rep=5 base=ens_e2 done in 0.1s
+[    2.3s] ens_e2__lapshot n=100 rep=6 base=ens_e2 done in 0.1s
+[    2.4s] ens_e2__lapshot n=100 rep=7 base=ens_e2 done in 0.1s
+[    2.5s] ens_e2__lapshot n=100 rep=8 base=ens_e2 done in 0.1s
+[    2.5s] ens_e2__lapshot n=100 rep=9 base=ens_e2 done in 0.1s
+[    2.6s] ens_e2__lapshot n=100 rep=10 base=ens_e2 done in 0.1s
+[    2.7s] ens_e2__lapshot n=100 rep=11 base=ens_e2 done in 0.1s
+[    2.7s] ens_e2__lapshot n=200 rep=5 base=ens_e2 done in 0.1s
+[    2.8s] ens_e2__lapshot n=200 rep=6 base=ens_e2 done in 0.1s
+[    2.9s] ens_e2__lapshot n=200 rep=7 base=ens_e2 done in 0.1s
+[    3.0s] ens_e2__lapshot n=200 rep=8 base=ens_e2 done in 0.1s
+[    3.0s] ens_e2__lapshot n=200 rep=9 base=ens_e2 done in 0.1s
+[    3.1s] ens_e2__lapshot n=200 rep=10 base=ens_e2 done in 0.1s
+[    3.2s] ens_e2__lapshot n=200 rep=11 base=ens_e2 done in 0.1s
+[    3.2s] done

reports/nnlab/logs/ts_nn.log

@@ -0,0 +1,98 @@
+[    0.0s] methods: ['ts_tcnn_st_zs', 'ts_tcnn_st_ft', 'ts_tcnn_st_joint', 'ts_tcnn_ry_zs', 'ts_tcnn_ry_ft', 'ts_tcnn_ry_joint', 'ts_tcnn_dann_st_zs', 'ts_tcnn_dann_st_ft', 'ts_tcnn_dann_st_joint'] rss=0.09G peak=0.09G
+[    9.0s] inputs st: Madrid (76263, 10, 42) Amsterdam (25992, 10, 42) rss=0.83G peak=1.15G
+[  234.5s]   tcnn seed0 epoch 5/15 40.7s/full-epoch eval=0.5260 rss=1.33G peak=1.33G
+[  416.0s]   tcnn seed0 epoch 10/15 29.2s/full-epoch eval=0.5385 rss=1.33G peak=1.38G
+[  561.8s]   tcnn seed0 epoch 15/15 28.7s/full-epoch eval=0.5475 rss=1.34G peak=1.38G
+[  570.7s] select tcnn_st: epochs=15 {'5': 0.526023912531838, '10': 0.5385065163765071, '15': 0.5474806666222004} lam=100.0 {0.1: 0.4793, 1.0: 0.5091, 10.0: 0.5455, 100.0: 0.555} rss=1.17G peak=1.38G
+[  578.5s] backbone tcnn_st seed0: 35.3s/full-epoch, 569s rss=1.22G peak=1.38G
+[  578.5s] ts_tcnn_st_zs saved rss=1.04G peak=1.38G
+[  580.9s] tcnn_st n=5 rep=0 heads=['ft', 'joint'] 2.3s rss=1.08G peak=1.38G
+[  582.7s] tcnn_st n=5 rep=1 heads=['ft', 'joint'] 1.8s rss=1.08G peak=1.38G
+[  584.6s] tcnn_st n=5 rep=2 heads=['ft', 'joint'] 1.9s rss=1.07G peak=1.38G
+[  586.4s] tcnn_st n=5 rep=3 heads=['ft', 'joint'] 1.8s rss=1.07G peak=1.38G
+[  588.0s] tcnn_st n=5 rep=4 heads=['ft', 'joint'] 1.6s rss=1.07G peak=1.38G
+[  589.5s] tcnn_st n=25 rep=0 heads=['ft', 'joint'] 1.5s rss=1.07G peak=1.38G
+[  591.1s] tcnn_st n=25 rep=1 heads=['ft', 'joint'] 1.5s rss=1.07G peak=1.38G
+[  592.9s] tcnn_st n=25 rep=2 heads=['ft', 'joint'] 1.8s rss=1.07G peak=1.38G
+[  594.4s] tcnn_st n=25 rep=3 heads=['ft', 'joint'] 1.6s rss=1.07G peak=1.38G
+[  596.2s] tcnn_st n=25 rep=4 heads=['ft', 'joint'] 1.8s rss=1.07G peak=1.38G
+[  597.8s] tcnn_st n=50 rep=0 heads=['ft', 'joint'] 1.5s rss=1.07G peak=1.38G
+[  599.4s] tcnn_st n=50 rep=1 heads=['ft', 'joint'] 1.6s rss=1.07G peak=1.38G
+[  601.1s] tcnn_st n=50 rep=2 heads=['ft', 'joint'] 1.7s rss=1.07G peak=1.38G
+[  602.7s] tcnn_st n=50 rep=3 heads=['ft', 'joint'] 1.5s rss=1.07G peak=1.38G
+[  603.9s] tcnn_st n=50 rep=4 heads=['ft', 'joint'] 1.2s rss=1.07G peak=1.38G
+[  605.5s] tcnn_st n=100 rep=0 heads=['ft', 'joint'] 1.6s rss=1.07G peak=1.38G
+[  607.0s] tcnn_st n=100 rep=1 heads=['ft', 'joint'] 1.5s rss=1.07G peak=1.38G
+[  608.3s] tcnn_st n=100 rep=2 heads=['ft', 'joint'] 1.3s rss=1.07G peak=1.38G
+[  609.9s] tcnn_st n=100 rep=3 heads=['ft', 'joint'] 1.6s rss=1.07G peak=1.38G
+[  611.5s] tcnn_st n=100 rep=4 heads=['ft', 'joint'] 1.5s rss=1.07G peak=1.38G
+[  612.9s] tcnn_st n=200 rep=0 heads=['ft', 'joint'] 1.4s rss=1.08G peak=1.38G
+[  614.4s] tcnn_st n=200 rep=1 heads=['ft', 'joint'] 1.5s rss=1.08G peak=1.38G
+[  615.9s] tcnn_st n=200 rep=2 heads=['ft', 'joint'] 1.5s rss=1.08G peak=1.38G
+[  617.5s] tcnn_st n=200 rep=3 heads=['ft', 'joint'] 1.6s rss=1.08G peak=1.38G
+[  619.0s] tcnn_st n=200 rep=4 heads=['ft', 'joint'] 1.5s rss=1.08G peak=1.38G
+[  626.0s] inputs ry: Madrid (76263, 10, 42) Amsterdam (25992, 10, 42) rss=1.24G peak=1.55G
+[  784.7s]   tcnn seed0 epoch 5/15 32.9s/full-epoch eval=0.5340 rss=1.42G peak=1.55G
+[  953.3s]   tcnn seed0 epoch 10/15 33.9s/full-epoch eval=0.5362 rss=1.42G peak=1.55G
+[ 1131.4s]   tcnn seed0 epoch 15/15 36.1s/full-epoch eval=0.5545 rss=1.22G peak=1.55G
+[ 1140.4s] select tcnn_ry: epochs=15 {'5': 0.5340027718176347, '10': 0.5362399360770098, '15': 0.5545017383586993} lam=100.0 {0.1: 0.4863, 1.0: 0.5196, 10.0: 0.5494, 100.0: 0.554} rss=1.03G peak=1.55G
+[ 1148.4s] backbone tcnn_ry seed0: 33.4s/full-epoch, 522s rss=1.06G peak=1.55G
+[ 1148.5s] ts_tcnn_ry_zs saved rss=0.89G peak=1.55G
+[ 1150.7s] tcnn_ry n=5 rep=0 heads=['ft', 'joint'] 2.1s rss=0.87G peak=1.55G
+[ 1152.5s] tcnn_ry n=5 rep=1 heads=['ft', 'joint'] 1.8s rss=0.87G peak=1.55G
+[ 1154.6s] tcnn_ry n=5 rep=2 heads=['ft', 'joint'] 2.1s rss=0.87G peak=1.55G
+[ 1156.6s] tcnn_ry n=5 rep=3 heads=['ft', 'joint'] 2.0s rss=0.87G peak=1.55G
+[ 1158.4s] tcnn_ry n=5 rep=4 heads=['ft', 'joint'] 1.8s rss=0.87G peak=1.55G
+[ 1160.2s] tcnn_ry n=25 rep=0 heads=['ft', 'joint'] 1.8s rss=0.86G peak=1.55G
+[ 1162.1s] tcnn_ry n=25 rep=1 heads=['ft', 'joint'] 1.9s rss=0.86G peak=1.55G
+[ 1164.5s] tcnn_ry n=25 rep=2 heads=['ft', 'joint'] 2.4s rss=0.86G peak=1.55G
+[ 1166.5s] tcnn_ry n=25 rep=3 heads=['ft', 'joint'] 2.0s rss=0.86G peak=1.55G
+[ 1168.2s] tcnn_ry n=25 rep=4 heads=['ft', 'joint'] 1.7s rss=0.86G peak=1.55G
+[ 1169.8s] tcnn_ry n=50 rep=0 heads=['ft', 'joint'] 1.7s rss=0.86G peak=1.55G
+[ 1171.9s] tcnn_ry n=50 rep=1 heads=['ft', 'joint'] 2.0s rss=0.86G peak=1.55G
+[ 1174.0s] tcnn_ry n=50 rep=2 heads=['ft', 'joint'] 2.2s rss=0.86G peak=1.55G
+[ 1175.8s] tcnn_ry n=50 rep=3 heads=['ft', 'joint'] 1.7s rss=0.86G peak=1.55G
+[ 1177.4s] tcnn_ry n=50 rep=4 heads=['ft', 'joint'] 1.7s rss=0.86G peak=1.55G
+[ 1179.1s] tcnn_ry n=100 rep=0 heads=['ft', 'joint'] 1.7s rss=0.86G peak=1.55G
+[ 1180.9s] tcnn_ry n=100 rep=1 heads=['ft', 'joint'] 1.8s rss=0.86G peak=1.55G
+[ 1182.6s] tcnn_ry n=100 rep=2 heads=['ft', 'joint'] 1.6s rss=0.86G peak=1.55G
+[ 1184.3s] tcnn_ry n=100 rep=3 heads=['ft', 'joint'] 1.7s rss=0.86G peak=1.55G
+[ 1186.0s] tcnn_ry n=100 rep=4 heads=['ft', 'joint'] 1.7s rss=0.86G peak=1.55G
+[ 1187.7s] tcnn_ry n=200 rep=0 heads=['ft', 'joint'] 1.7s rss=0.86G peak=1.55G
+[ 1189.3s] tcnn_ry n=200 rep=1 heads=['ft', 'joint'] 1.6s rss=0.86G peak=1.55G
+[ 1191.2s] tcnn_ry n=200 rep=2 heads=['ft', 'joint'] 1.9s rss=0.86G peak=1.55G
+[ 1192.7s] tcnn_ry n=200 rep=3 heads=['ft', 'joint'] 1.6s rss=0.86G peak=1.55G
+[ 1194.2s] tcnn_ry n=200 rep=4 heads=['ft', 'joint'] 1.5s rss=0.86G peak=1.55G
+[ 1199.5s] inputs st: Madrid (76263, 10, 42) Amsterdam (25992, 10, 42) rss=1.02G peak=1.55G
+[ 1530.9s]   tcnn_dann seed0 epoch 5/15 66.4s/full-epoch eval=0.5044 rss=1.20G peak=1.55G
+[ 1875.5s]   tcnn_dann seed0 epoch 10/15 68.2s/full-epoch eval=0.5094 rss=1.20G peak=1.55G
+[ 2217.2s]   tcnn_dann seed0 epoch 15/15 68.0s/full-epoch eval=0.5140 rss=1.22G peak=1.55G
+[ 2251.4s] select tcnn_dann_st: epochs=15 {'5': 0.5044430346339247, '10': 0.5093685357526514, '15': 0.514016324587161} lam=100.0 {0.1: 0.4313, 1.0: 0.4417, 10.0: 0.4598, 100.0: 0.4801} rss=1.02G peak=1.55G
+[ 2260.2s] backbone tcnn_dann_st seed0: 67.6s/full-epoch, 1061s rss=1.06G peak=1.55G
+[ 2260.2s] ts_tcnn_dann_st_zs saved rss=0.89G peak=1.55G
+[ 2267.7s] tcnn_dann_st n=5 rep=0 heads=['ft', 'joint'] 7.4s rss=0.85G peak=1.55G
+[ 2275.3s] tcnn_dann_st n=5 rep=1 heads=['ft', 'joint'] 7.6s rss=0.85G peak=1.55G
+[ 2282.6s] tcnn_dann_st n=5 rep=2 heads=['ft', 'joint'] 7.3s rss=0.85G peak=1.55G
+[ 2288.9s] tcnn_dann_st n=5 rep=3 heads=['ft', 'joint'] 6.4s rss=0.85G peak=1.55G
+[ 2296.1s] tcnn_dann_st n=5 rep=4 heads=['ft', 'joint'] 7.2s rss=0.85G peak=1.55G
+[ 2303.5s] tcnn_dann_st n=25 rep=0 heads=['ft', 'joint'] 7.4s rss=0.85G peak=1.55G
+[ 2311.6s] tcnn_dann_st n=25 rep=1 heads=['ft', 'joint'] 8.1s rss=0.85G peak=1.55G
+[ 2319.1s] tcnn_dann_st n=25 rep=2 heads=['ft', 'joint'] 7.5s rss=0.85G peak=1.55G
+[ 2326.5s] tcnn_dann_st n=25 rep=3 heads=['ft', 'joint'] 7.4s rss=0.85G peak=1.55G
+[ 2334.5s] tcnn_dann_st n=25 rep=4 heads=['ft', 'joint'] 8.0s rss=0.85G peak=1.55G
+[ 2342.9s] tcnn_dann_st n=50 rep=0 heads=['ft', 'joint'] 8.3s rss=0.85G peak=1.55G
+[ 2351.2s] tcnn_dann_st n=50 rep=1 heads=['ft', 'joint'] 8.4s rss=0.85G peak=1.55G
+[ 2357.9s] tcnn_dann_st n=50 rep=2 heads=['ft', 'joint'] 6.7s rss=0.85G peak=1.55G
+[ 2366.2s] tcnn_dann_st n=50 rep=3 heads=['ft', 'joint'] 8.3s rss=0.85G peak=1.55G
+[ 2374.6s] tcnn_dann_st n=50 rep=4 heads=['ft', 'joint'] 8.4s rss=0.85G peak=1.55G
+[ 2383.5s] tcnn_dann_st n=100 rep=0 heads=['ft', 'joint'] 8.9s rss=0.85G peak=1.55G
+[ 2392.6s] tcnn_dann_st n=100 rep=1 heads=['ft', 'joint'] 9.0s rss=0.85G peak=1.55G
+[ 2401.1s] tcnn_dann_st n=100 rep=2 heads=['ft', 'joint'] 8.5s rss=0.85G peak=1.55G
+[ 2409.9s] tcnn_dann_st n=100 rep=3 heads=['ft', 'joint'] 8.8s rss=0.85G peak=1.55G
+[ 2418.1s] tcnn_dann_st n=100 rep=4 heads=['ft', 'joint'] 8.1s rss=0.85G peak=1.55G
+[ 2426.3s] tcnn_dann_st n=200 rep=0 heads=['ft', 'joint'] 8.3s rss=0.85G peak=1.55G
+[ 2436.8s] tcnn_dann_st n=200 rep=1 heads=['ft', 'joint'] 10.5s rss=0.85G peak=1.55G
+[ 2446.7s] tcnn_dann_st n=200 rep=2 heads=['ft', 'joint'] 9.9s rss=0.85G peak=1.55G
+[ 2455.6s] tcnn_dann_st n=200 rep=3 heads=['ft', 'joint'] 8.9s rss=0.85G peak=1.55G
+[ 2464.3s] tcnn_dann_st n=200 rep=4 heads=['ft', 'joint'] 8.6s rss=0.85G peak=1.55G
+[ 2464.3s] done rss=0.85G peak=1.55G

reports/nnlab/logs/ts_nn_smoke.log

@@ -0,0 +1,28 @@
+[    0.0s] methods: ['ts_tcnn_st_zs_SMOKE', 'ts_tcnn_st_ft_SMOKE', 'ts_tcnn_st_joint_SMOKE', 'ts_tcnn_ry_zs_SMOKE', 'ts_tcnn_ry_ft_SMOKE', 'ts_tcnn_ry_joint_SMOKE', 'ts_tcnn_dann_st_zs_SMOKE', 'ts_tcnn_dann_st_ft_SMOKE', 'ts_tcnn_dann_st_joint_SMOKE'] rss=0.09G peak=0.09G
+[   17.9s]   gru seed0 epoch 1/1 3.4s/full-epoch rss=0.95G peak=1.02G
+[   18.2s] gru check ok: ~204s per Madrid-source epoch (extrapolated) rss=0.97G peak=1.02G
+[   25.1s] inputs st: Madrid (76263, 10, 42) Amsterdam (25992, 10, 42) rss=1.12G peak=1.44G
+[   34.9s]   tcnn seed0 epoch 1/2 47.8s/full-epoch eval=0.4092 rss=1.33G peak=1.44G
+[   39.9s]   tcnn seed0 epoch 2/2 48.0s/full-epoch eval=0.4755 rss=1.34G peak=1.44G
+[   45.0s] select tcnn_st_SMOKE: epochs=2 {'1': 0.40919231367742104, '2': 0.4754845381892462} lam=1.0 {0.1: 0.5026, 1.0: 0.5153, 10.0: 0.509, 100.0: 0.4977} rss=1.17G peak=1.44G
+[   59.0s] backbone tcnn_st_SMOKE seed0: 47.9s/full-epoch, 34s rss=1.20G peak=1.44G
+[   59.1s] ts_tcnn_st_zs_SMOKE saved rss=1.04G peak=1.44G
+[   67.0s] tcnn_st n=5 rep=0 heads=['ft', 'joint'] 7.9s rss=1.08G peak=1.44G
+[   76.8s] tcnn_st n=200 rep=0 heads=['ft', 'joint'] 9.8s rss=1.07G peak=1.44G
+[   85.5s] inputs ry: Madrid (76263, 10, 42) Amsterdam (25992, 10, 42) rss=1.23G peak=1.55G
+[   90.8s]   tcnn seed0 epoch 1/2 51.9s/full-epoch eval=0.3879 rss=1.41G peak=1.55G
+[   96.2s]   tcnn seed0 epoch 2/2 46.3s/full-epoch eval=0.4781 rss=1.41G peak=1.55G
+[  102.1s] select tcnn_ry_SMOKE: epochs=2 {'1': 0.38785458289169794, '2': 0.4780757221716626} lam=1.0 {0.1: 0.4986, 1.0: 0.518, 10.0: 0.5106, 100.0: 0.5018} rss=1.24G peak=1.55G
+[  116.2s] backbone tcnn_ry_SMOKE seed0: 49.1s/full-epoch, 31s rss=1.28G peak=1.55G
+[  116.2s] ts_tcnn_ry_zs_SMOKE saved rss=1.10G peak=1.55G
+[  123.3s] tcnn_ry n=5 rep=0 heads=['ft', 'joint'] 7.0s rss=1.08G peak=1.55G
+[  131.2s] tcnn_ry n=200 rep=0 heads=['ft', 'joint'] 7.9s rss=1.07G peak=1.55G
+[  138.4s] inputs st: Madrid (76263, 10, 42) Amsterdam (25992, 10, 42) rss=1.23G peak=1.55G
+[  147.2s]   tcnn_dann seed0 epoch 1/2 105.1s/full-epoch eval=0.4122 rss=1.41G peak=1.55G
+[  156.8s]   tcnn_dann seed0 epoch 2/2 108.4s/full-epoch eval=0.4255 rss=1.17G peak=1.55G
+[  166.1s] select tcnn_dann_st_SMOKE: epochs=2 {'1': 0.41219350565994745, '2': 0.42554601012067195} lam=100.0 {0.1: 0.4255, 1.0: 0.4625, 10.0: 0.4735, 100.0: 0.4767} rss=1.06G peak=1.55G
+[  179.7s] backbone tcnn_dann_st_SMOKE seed0: 106.7s/full-epoch, 41s rss=1.12G peak=1.55G
+[  180.1s] ts_tcnn_dann_st_zs_SMOKE saved rss=0.94G peak=1.55G
+[  195.0s] tcnn_dann_st n=5 rep=0 heads=['ft', 'joint'] 14.9s rss=0.91G peak=1.55G
+[  208.8s] tcnn_dann_st n=200 rep=0 heads=['ft', 'joint'] 13.8s rss=0.91G peak=1.55G
+[  208.8s] done rss=0.91G peak=1.55G

reports/nnlab/logs/ts_nn_smoke_review.log

@@ -0,0 +1,28 @@
+[    0.0s] methods: ['ts_tcnn_st_zs_SMOKE', 'ts_tcnn_st_ft_SMOKE', 'ts_tcnn_st_joint_SMOKE', 'ts_tcnn_ry_zs_SMOKE', 'ts_tcnn_ry_ft_SMOKE', 'ts_tcnn_ry_joint_SMOKE', 'ts_tcnn_dann_st_zs_SMOKE', 'ts_tcnn_dann_st_ft_SMOKE', 'ts_tcnn_dann_st_joint_SMOKE'] rss=0.09G peak=0.09G
+[   18.9s]   gru seed0 epoch 1/1 3.6s/full-epoch rss=0.96G peak=1.02G
+[   19.2s] gru check ok: ~216s per Madrid-source epoch (extrapolated) rss=0.97G peak=1.02G
+[   25.7s] inputs st: Madrid (76263, 10, 42) Amsterdam (25992, 10, 42) rss=1.12G peak=1.44G
+[   35.2s]   tcnn seed0 epoch 1/2 50.3s/full-epoch eval=0.4092 rss=1.33G peak=1.44G
+[   40.5s]   tcnn seed0 epoch 2/2 50.8s/full-epoch eval=0.4755 rss=1.34G peak=1.44G
+[   44.7s] select tcnn_st_SMOKE: epochs=2 {'1': 0.40919231367742104, '2': 0.4754845381892462} lam=1.0 {0.1: 0.5026, 1.0: 0.5153, 10.0: 0.509, 100.0: 0.4977} rss=1.17G peak=1.44G
+[   57.0s] backbone tcnn_st_SMOKE seed0: 50.5s/full-epoch, 31s rss=1.20G peak=1.44G
+[   57.1s] ts_tcnn_st_zs_SMOKE saved rss=1.04G peak=1.44G
+[   65.1s] tcnn_st n=5 rep=0 heads=['ft', 'joint'] 8.0s rss=1.08G peak=1.44G
+[   75.3s] tcnn_st n=200 rep=0 heads=['ft', 'joint'] 10.2s rss=1.07G peak=1.44G
+[   83.3s] inputs ry: Madrid (76263, 10, 42) Amsterdam (25992, 10, 42) rss=1.23G peak=1.55G
+[   88.1s]   tcnn seed0 epoch 1/2 47.3s/full-epoch eval=0.3879 rss=1.41G peak=1.55G
+[   92.9s]   tcnn seed0 epoch 2/2 44.7s/full-epoch eval=0.4781 rss=1.42G peak=1.55G
+[   97.6s] select tcnn_ry_SMOKE: epochs=2 {'1': 0.38785458289169794, '2': 0.4780757221716626} lam=1.0 {0.1: 0.4986, 1.0: 0.518, 10.0: 0.5106, 100.0: 0.5018} rss=1.24G peak=1.55G
+[  106.5s] backbone tcnn_ry_SMOKE seed0: 46.0s/full-epoch, 23s rss=1.29G peak=1.55G
+[  106.6s] ts_tcnn_ry_zs_SMOKE saved rss=1.10G peak=1.55G
+[  113.6s] tcnn_ry n=5 rep=0 heads=['ft', 'joint'] 7.0s rss=1.08G peak=1.55G
+[  121.3s] tcnn_ry n=200 rep=0 heads=['ft', 'joint'] 7.7s rss=1.07G peak=1.55G
+[  126.9s] inputs st: Madrid (76263, 10, 42) Amsterdam (25992, 10, 42) rss=1.23G peak=1.55G
+[  134.7s]   tcnn_dann seed0 epoch 1/2 89.6s/full-epoch eval=0.4122 rss=1.41G peak=1.55G
+[  142.9s]   tcnn_dann seed0 epoch 2/2 96.6s/full-epoch eval=0.4255 rss=1.41G peak=1.55G
+[  150.7s] select tcnn_dann_st_SMOKE: epochs=2 {'1': 0.41219350565994745, '2': 0.42554601012067195} lam=100.0 {0.1: 0.4255, 1.0: 0.4625, 10.0: 0.4735, 100.0: 0.4767} rss=1.24G peak=1.55G
+[  161.9s] backbone tcnn_dann_st_SMOKE seed0: 93.1s/full-epoch, 35s rss=1.29G peak=1.55G
+[  161.9s] ts_tcnn_dann_st_zs_SMOKE saved rss=1.11G peak=1.55G
+[  174.2s] tcnn_dann_st n=5 rep=0 heads=['ft', 'joint'] 12.2s rss=1.07G peak=1.55G
+[  187.6s] tcnn_dann_st n=200 rep=0 heads=['ft', 'joint'] 13.4s rss=1.08G peak=1.55G
+[  187.6s] done rss=1.08G peak=1.55G

reports/nnlab/research_shortlist.json

@@ -0,0 +1,227 @@
+{
+ "shortlist": [
+  {
+   "rank": 1,
+   "name": "Per-city input normalisation (robust z-score or quantile transform fitted separately per city, with an optional prior-reweighted Madrid pass)",
+   "family": "input-transform",
+   "source": "Merged: per-city standardisation (idea 'Per-city input normalisation', TimeMatch Nyborg et al. 2022 arXiv 2111.02682 for the motivation); surgical fine-tuning init (Lee et al. ICLR 2023, arXiv 2210.11466); AdaBN / CL2N re-centring (SimpleShot, arXiv 1911.04623)",
+   "why": "It costs seconds, every other item here builds on it, and it goes straight at the input-level shift (the city detector reaches AUC 1.0). With 20 labels at n=5 the shots cannot absorb a city-wide offset themselves. Expected +0.01 to +0.04, mostly at n<=25.",
+   "recipe": "Default A: for each of the 393 features, x' = (x - median_city) / IQR_city, computed on ALL pixels of that city (76,263 Madrid, 25,992 Amsterdam unlabelled). Clip to [-5,5]. Variant B: sklearn QuantileTransformer(output_distribution='normal', n_quantiles=200), fitted separately per city. Label-shift fix (one pass, fixed in advance): after you have an EM prior estimate pi_A (item 2), recompute the Madrid median/IQR with each Madrid row weighted by pi_A[y]/pi_M[y] (weighted quantiles), so both cities' statistics refer to the same class mix, then refit downstream models once. Keep a raw-feature copy, because trees may still want the raw scale. Choosing A vs B vs raw: run the Madrid spatial-block pseudo-target harness (see notes) on the 3 options and keep the winner for every n.",
+   "rule_compliance": "Uses no target labels. It uses unlabelled Amsterdam features only, which is allowed. The choice between variants is made on the Madrid harness, not on Amsterdam labels.",
+   "est_cpu_minutes": 2,
+   "uses_shots": "No",
+   "uses_unlabelled_target": "Yes (per-city statistics)"
+  },
+  {
+   "name": "EM (MLLS) prior re-estimation on unlabelled Amsterdam, with the correct source prior and a partial-exponent logit adjustment",
+   "rank": 2,
+   "family": "head",
+   "source": "Merged from 4 near-duplicates: Saerens et al. 2002; Lipton et al. ICML 2018 (arXiv 1802.03916); Garg et al. NeurIPS 2020 (arXiv 2003.07554: MLLS beats BBSE); Sipka et al. WACV 2022 (MAP/Dirichlet variant)",
+   "why": "The label shift is stated outright and macro F1 is sensitive to per-class thresholds. This is essentially free and applies to every model and every n. The balanced shots carry no prior information, so the prior has to come from the unlabelled pool. Expected +0.01 to +0.03, with the most room at n=5/25.",
+   "recipe": "1) Get the probabilities P (25,992 x 4) from any model. 2) Set pi_S to the model's EFFECTIVE training prior: for Madrid + weighted shots, pi_S[c] = (Madrid count_c*w_M + n*w_shot) / total weight; for a head fitted on balanced shots, pi_S = uniform; for a class-balanced loss, pi_S = uniform. 3) Calibrate with one scalar temperature T fitted on held-out Madrid folds (not on the shots at n=5). 4) EM: start pi = pi_S; repeat 200 times or until change < 1e-6: q \u221d P*pi/pi_S; pi = (sum_i q_i + alpha_dir) / (N + 4*alpha_dir), with Dirichlet floor alpha_dir = 0.01*N; clip pi/pi_S to [0.25, 4]. 5) Final logits = log P + gamma*log(pi/pi_S). Fix gamma ahead of time from {0, 0.5, 1} on the Madrid harness with the held-out block resampled to the EM-estimated Amsterdam prior. If you skip the harness, default gamma = 0.5 (the macro-F1 compromise). 6) Apply BEFORE graph smoothing (item 4). Sanity check: the predicted class histogram should move toward pi. If EM puts <5% on any class, fall back to gamma=0.",
+   "rule_compliance": "Allowed. The prior comes from unlabelled pixels; gamma is chosen on Madrid. Do NOT use idea 3's suggestion to tune gamma on 'Amsterdam shots simulated via n=200 splits' scored on other Amsterdam pixels (that uses Amsterdam labels beyond n). Also do not estimate the prior from the shots.",
+   "est_cpu_minutes": 1,
+   "uses_shots": "Only to define pi_S when shots are in the training set",
+   "uses_unlabelled_target": "Yes (EM prior)"
+  },
+  {
+   "name": "n-dependent anchoring to the zero-shot model: shrink the shot-fitted head toward the Madrid/DANN head (tau = c/n) and schedule the LightGBM shot weight with n",
+   "rank": 3,
+   "family": "head",
+   "source": "Source-regularised head (L2-SP applied to the head: Li et al. ICML 2018, arXiv 1802.01483; DFR, Kirichenko et al. ICLR 2023, arXiv 2204.02937); prototype mixing alpha = n/(n+k) from SimpleShot-style ideas",
+   "why": "The current n=5 result (0.566) is BELOW zero-shot LightGBM (0.593) and far below zero-shot XGB+DANN+smoothing (0.639). Weighting 20 shots 30x overrides the source model, so a rule where shot influence grows with n should recover at least the zero-shot level at n=5 and keep the gains at large n. This is probably the biggest single gain at n=5 for the least work.",
+   "recipe": "(a) LightGBM/XGB: shot weight w(n) = r * N_madrid / (4n) * n/(n+k0), so the shots' total mass is a fraction r*n/(n+k0) of Madrid's mass. Defaults r=1, k0=25 (n=5 gives about 17% of r, n=200 gives about 89%). Warm-start from the zero-shot DANN+XGB model where possible (init_model / base_margin = zero-shot logits). (b) Convex head on frozen embeddings (item 5): minimise sum_shots CE + (tau/2)(||W-W_0||^2+||b-b_0||^2), L-BFGS, tau = 50/n. W_0 is the Madrid head, or (better under shift) the Madrid class prototypes after per-city centring. (c) Choose r, k0 and c on the Madrid harness over 20 simulated shot draws per n, from the grids r in {0.3,1,3}, k0 in {10,25,50}, c in {5,50,500}, then freeze them.",
+   "rule_compliance": "Allowed if c/r/k0 come from the Madrid harness. Idea 6's suggestion to tune tau 'using simulated n-shot draws from the labelled Amsterdam pool' BREAKS the rule and must not be used.",
+   "est_cpu_minutes": 20,
+   "uses_shots": "Yes",
+   "uses_unlabelled_target": "Indirectly (through the DANN warm start)"
+  },
+  {
+   "name": "Graph label spreading on a mixed spatial-kNN + feature-kNN Amsterdam graph, with shots clamped as seeds (replaces plain spatial smoothing)",
+   "rank": 4,
+   "family": "transductive-refinement",
+   "source": "Zhou et al. NIPS 2003 (Learning with Local and Global Consistency); BD-CSPN/Prototype Rectification (Liu et al., arXiv 1911.10713); merged with the label-propagation parts of the self-training ideas",
+   "why": "Smoothing already gives +0.02. Adding the shots as seeds and a feature graph lets labels spread along both space and spectrum, which helps most at n=5-50. It runs in seconds with scipy.sparse, and its hyperparameters can be tuned on Madrid, which also has coordinates.",
+   "recipe": "Graph features: 30-d PCA of the per-city-standardised features (item 1), or the 128-d frozen embedding (item 5). W_feat: kNN with k=15, Gaussian weights with sigma = mean kNN distance. W_spat: kNN on coordinates, k=8, weight 1. W = a*W_feat + (1-a)*W_spat; symmetrise; S = D^-1/2 W D^-1/2. Y0 = P_model after item 2's prior correction, with each shot row replaced by its one-hot label times 5. Iterate F <- alpha*S F + (1-alpha)*Y0 30 times. Divide each column of F by the class mass sum_i F_ic, multiply by pi_EM, then argmax. Defaults a=0.3, alpha=0.9. Pick (a, alpha) from {0.0,0.3,0.6} x {0.8,0.9,0.99} on the Madrid harness, with the graph built on the held-out Madrid block only.",
+   "rule_compliance": "Allowed. Shot pixels are excluded from scoring anyway; the hyperparameters come from Madrid. Do not use leave-shots-out on Amsterdam to tune a at n=5; it is legal but too noisy.",
+   "est_cpu_minutes": 3,
+   "uses_shots": "Yes (clamped seeds)",
+   "uses_unlabelled_target": "Yes (graph over all 25,992 pixels)"
+  },
+  {
+   "name": "Frozen DANN-MLP backbone (importance-weighted DANN, per-city inputs, AdaBN) + CL2N embeddings + shrunk prototype / logistic head",
+   "rank": 5,
+   "family": "backbone",
+   "source": "Merged from 9 ideas: SimpleShot CL2N (arXiv 1911.04623); DFR last-layer retraining (arXiv 2204.02937); IWDAN (Tachet des Combes et al. NeurIPS 2020, arXiv 2003.04475); AdaBN/SHOT-lite (arXiv 2002.08546); MME cosine-head init (arXiv 1904.06487); TabM/RealMLP training tricks; frozen-head plan (a)",
+   "why": "This is plan (a), built so it can work. The backbone is trained with domain-invariance pressure, because Madrid-only CE embeddings may throw away cues Amsterdam needs. The head fits few parameters and is shrunk toward Madrid as n gets small. It gives a strong n=5/25 member that differs from the tree models. Expected +0.02 to +0.05 at n=5/25 over the current results, about parity at n>=100.",
+   "recipe": "Input: the 60 DANN features plus the top 100 LightGBM-importance features, per-city standardised (item 1). Network: 160 -> 256 -> 128 (BatchNorm, SiLU, dropout 0.2) -> 4. Loss: class-balanced CE on Madrid (label smoothing 0.1) + DANN with a GRL on the 128-d layer, lambda ramp 2/(1+exp(-10p))-1 scaled by 0.3, discriminator 128-64-1. Discriminator loss on Madrid rows weighted by w_y = pi_EM/pi_M (IWDAN; pi_EM from item 2 run on the zero-shot model). AdamW lr 1e-3, wd 1e-4, batch 512 per domain, a FIXED 30 epochs, 5 seeds. AdaBN: recompute BN running statistics on all Amsterdam pixels (train mode, no gradients). Embed both cities; CL2N: z = (e - mu_city)/||e - mu_city||, where mu_M is the prior-reweighted Madrid mean. Head (default, closed form): cosine prototypes p_c = alpha*mean(shots_c) + (1-alpha)*mean(Madrid_c), alpha = n/(n+k), k=10, logits = 15*cos(z, p_c). Alternative head: item 3(b) shrunk logistic regression. Report the average over the 5 seeds. Pipeline: item 2 prior correction, then item 4 graph spreading.",
+   "rule_compliance": "Allowed. k, temperature and the epoch count are fixed or chosen on the Madrid harness. Do NOT run idea 'Warning' as stated: its 20-seed diagnostic scores methods on the remaining Amsterdam pixels, which is selection on test labels. Run that diagnostic on the Madrid harness instead.",
+   "est_cpu_minutes": 20,
+   "uses_shots": "Yes (head only)",
+   "uses_unlabelled_target": "Yes (DANN, AdaBN, centring)"
+  },
+  {
+   "name": "Probability-level ensemble: zero-shot XGB+DANN, LightGBM with the item-3 schedule, frozen-MLP head, Mahalanobis prototype; uniform log-prob average, then EM, then graph spreading",
+   "rank": 6,
+   "family": "ensemble",
+   "source": "Merged ensemble ideas (cheap ensembling/calibration layer; stacking idea without TrAdaBoost); TableShift (arXiv 2312.07577: no single method dominates under shift)",
+   "why": "Trees, MLPs and prototypes make errors in different places, and averaging is the most reliable +0.01 to +0.03 at every n. The base models already exist, so it costs nothing more.",
+   "recipe": "Members: (1) zero-shot XGB+DANN, (2) LightGBM with the item-3 schedule, (3) item-5 frozen MLP head (5-seed mean), (4) Mahalanobis prototype with shrinkage to the Madrid class means (alpha = n/(n+10)), (5) item 7 if it is ready. Calibrate each member with one temperature fitted on held-out Madrid. Combine with an unweighted mean of log-probabilities. Fixed n-dependent weight rule: member 1 gets weight 10/(n+10), and the rest share the remainder equally. Then run item 2 EM on the ensemble output with pi_S = mean of the members' effective priors (or uniform after each member's own correction; simpler: prior-correct each member to uniform first, then use pi_S = uniform), then item 4 graph spreading. No learned stacking weights at n<=25.",
+   "rule_compliance": "Allowed. The weights are a fixed rule; optional stacking by CV log-loss on the shots is legal only for n>=50 and is not recommended.",
+   "est_cpu_minutes": 3,
+   "uses_shots": "Through the members",
+   "uses_unlabelled_target": "Through EM and the graph"
+  },
+  {
+   "name": "Cross-fitted self-training of LightGBM on class-quota pseudo-labels from the graph-refined ensemble",
+   "rank": 7,
+   "family": "transductive-refinement",
+   "source": "Merged from 3 ideas: self-training with class-balanced pseudo-labels (FixMatch/CDAC-inspired, arXiv 2104.09415); LightGBM pseudo-label retraining; TimeMatch's semi-supervised stage (arXiv 2111.02682)",
+   "why": "It turns the 26k unlabelled pixels into Amsterdam-distributed training data, so the trees learn Amsterdam-specific splits that the graph cannot. Expected +0.02 to +0.05 when the starting point is the ensemble; riskiest at n=5.",
+   "recipe": "Start: Q = item 6 output (prior-corrected, graph-spread). Round r = 1, 2 (FIXED at 2 rounds): for each class c, take the top fraction f_r of pixels by Q_c among pixels whose argmax is c, with f_1=0.2 and f_2=0.35. Cap the per-class counts proportional to pi_EM. Train LightGBM on Madrid (weight 0.3) + shots (item-3 schedule weight) + pseudo-labels (weight 1, hard labels), num_leaves 15, lr 0.05, 300 rounds, feature_fraction 0.5, min_data_in_leaf 50. Cross-fit: split Amsterdam into 5 SPATIAL blocks; predict block b with a model that excludes block b's pseudo-labels. Then repeat the EM -> graph step for the next round. Final output = mean of the round-2 cross-fitted model and Q (log-prob average).",
+   "rule_compliance": "Allowed (pseudo-labels are not ground-truth labels). The number of rounds and the fractions are fixed in advance, not chosen on Amsterdam; if you validate, do it on the Madrid harness.",
+   "est_cpu_minutes": 40,
+   "uses_shots": "Yes (heavily weighted rows)",
+   "uses_unlabelled_target": "Yes (pseudo-labels)"
+  },
+  {
+   "name": "Transductive prototype rectification with marginal constraints (soft k-means / Sinkhorn toward EM priors, TIM-lite) on the frozen embeddings",
+   "rank": 8,
+   "family": "head",
+   "source": "Merged: TIM (Boudiaf et al. NeurIPS 2020, arXiv 2008.11297); LaplacianShot (arXiv 2006.15486); BD-CSPN (arXiv 1911.10713); PT-MAP (arXiv 2006.03806); T3A (NeurIPS 2021); the entropy part of MME",
+   "why": "Prototypes from 5 shots are noisy, and 26k unlabelled embeddings can rectify them. Sinkhorn with the EM prior as the column marginal avoids TIM's uniform-marginal mistake under label shift. Closed form, seconds, few hyperparameters. Expected +0.01 to +0.04 at n=5/25.",
+   "recipe": "On the item-5 CL2N embeddings: initialise p_c from item 5's shrunk prototypes. Iterate 10 times: logits = 15*cos(z, p_c); P = Sinkhorn(softmax(logits)) with 5 iterations, row sums 1, column sums N*pi_EM; mask pixels with max P < 0.6; p_c <- 0.5*p_c_init + 0.5*normalise(sum_i P_ic z_i). Shots stay hard-assigned in every iteration. Output P, then item 4. A TIM variant is optional: CE(shots) + 0.3*mean H(p_i) + 1.0*KL(mean p || pi_EM), 100 Adam steps on W only. Use one variant, not both. Default to Sinkhorn.",
+   "rule_compliance": "Allowed. Constants are fixed in advance or set on the Madrid harness; nothing is tuned on the shots.",
+   "est_cpu_minutes": 3,
+   "uses_shots": "Yes",
+   "uses_unlabelled_target": "Yes"
+  },
+  {
+   "name": "Ordinal head (CORN / cumulative logits) on the frozen embedding",
+   "rank": 9,
+   "family": "head",
+   "source": "Ordinal / change-point idea (BATSCCD, IJDE 2024; LandTrendr building-age, IEEE JSTARS 2024); CORN/CORAL ordinal regression (general)",
+   "why": "Age classes are ordered. A shared direction plus 3 thresholds has far fewer free parameters than 4 class vectors, which matters at n=5. It also reduces errors between non-adjacent classes, and it is a cheap extra head on the same embedding.",
+   "recipe": "On the item-5 embedding (128-d CL2N): fit 3 binary logistic regressions for y>1, y>2, y>3, sharing one weight vector w (cumulative link, mord.LogisticAT or a small torch model) on shots + Madrid rows at weight 1/w(n) from the item-3 rule. L2 C=0.1. p(y=k) = P(>k-1) - P(>k), clipped at 1e-4 and renormalised. Then EM (item 2) with pi_S matching the training mix, then use it as an extra member in item 6.",
+   "rule_compliance": "Allowed. C is fixed in advance or chosen on the Madrid harness.",
+   "est_cpu_minutes": 3,
+   "uses_shots": "Yes",
+   "uses_unlabelled_target": "Via the embedding and EM"
+  },
+  {
+   "name": "Daume feature augmentation (shared / Madrid-only / Amsterdam-only copies) for LightGBM",
+   "rank": 10,
+   "family": "classical",
+   "source": "Daume III, Frustratingly Easy Domain Adaptation, ACL 2007 (arXiv 0907.1815)",
+   "why": "Nearly free: the trees can split on city-specific copies instead of having 30x weights fight 76k Madrid rows. Likely +0.01 to +0.02 at n=25-100; small at n=5. It adds a diverse tree member to the ensemble.",
+   "recipe": "Use the 60 DANN features + top 60 by importance = 120 features. Madrid rows -> [X, X, NaN]; Amsterdam shots and pseudo-labels -> [X, NaN, X]; predict Amsterdam with [X, NaN, X]. LightGBM num_leaves 15, lr 0.05, 400 rounds, feature_fraction 0.5, min_child_samples 5, shot weight from the item-3 rule. Then item 2, and use it as a member in item 6.",
+   "rule_compliance": "Allowed. Shot weight comes from the Madrid-tuned rule, not from CV on Amsterdam.",
+   "est_cpu_minutes": 10,
+   "uses_shots": "Yes",
+   "uses_unlabelled_target": "No (optionally pseudo-labels from item 7)"
+  },
+  {
+   "name": "Second backbone: small TempCNN on the 41-year x 6-band series (per-city normalised, AdaBN), used frozen with the item-5 head",
+   "rank": 11,
+   "family": "backbone",
+   "source": "Pelletier et al. 2019 TempCNN (arXiv 1811.10166); Russwurm & Korner 2020 (arXiv 1910.10536); 'Frozen sequence backbone + target-BN' idea (Modular Adaptation, arXiv 2104.00619)",
+   "why": "It sees the raw temporal trajectory rather than hand-built summaries, so it adds ensemble diversity beyond the MLP. Only worth doing after items 1-8 work.",
+   "recipe": "Input: 41 x (6 bands + NDVI, NDBI, MNDWI + missing mask), yearly mean of up to 3 observations, per-city per-band robust z-score. Three Conv1d layers (64 filters, k=5, BN, ReLU, dropout 0.3, no pooling) -> flatten -> 128-d -> 4. Training: class-balanced CE + label smoothing 0.1, CORAL loss (weight 0.5) between Madrid and Amsterdam 128-d batch covariances, augmentations: random year masking 15% and band gain U(0.9,1.1). AdamW 1e-3, 20 epochs fixed, 3 seeds. AdaBN on Amsterdam. Then the same CL2N + shrunk prototype head as item 5, and add it as a member in item 6.",
+   "rule_compliance": "Allowed. The epoch count and CORAL weight are fixed in advance.",
+   "est_cpu_minutes": 35,
+   "uses_shots": "Head only",
+   "uses_unlabelled_target": "Yes (CORAL, AdaBN, centring)"
+  },
+  {
+   "name": "LP-FT with L2-SP for n>=50 only (unfreeze the item-5 MLP gently from the probed head)",
+   "rank": 12,
+   "family": "backbone",
+   "source": "Kumar et al. ICLR 2022 (arXiv 2202.10054); L2-SP Li et al. ICML 2018 (arXiv 1802.01483)",
+   "why": "Optional, for the n=100/200 regime where frozen features saturate: a controlled fine-tune can add +0.01 to +0.02. Skip it at n<=25.",
+   "recipe": "Start from item 5 with the head fitted (LP). Unfreeze everything; AdamW lr 1e-4 for the backbone and 1e-3 for the head, 60 full-batch steps FIXED. Loss = class-balanced CE on the shots + 0.1*CE on a 5k Madrid replay subsample (resampled each step) + 0.01*sum||theta - theta_0||^2. No early stopping. Re-run AdaBN afterwards. Apply only for n in {50,100,200}, and add it as an ensemble member, not a replacement.",
+   "rule_compliance": "Allowed. Step count and lambda are fixed in advance or set on the Madrid harness; there is no shot-based early stopping at small n.",
+   "est_cpu_minutes": 10,
+   "uses_shots": "Yes",
+   "uses_unlabelled_target": "Via AdaBN"
+  }
+ ],
+ "dropped": [
+  {
+   "name": "Duplicate EM/BBSE prior-correction entries (4 copies across scouts)",
+   "reason": "Merged into shortlist item 2."
+  },
+  {
+   "name": "Duplicate TIM / LaplacianShot entries (2 copies)",
+   "reason": "Merged into item 8 (Sinkhorn-constrained rectification, TIM-lite variant) and item 4 (the Laplacian/graph part)."
+  },
+  {
+   "name": "Duplicate self-training entries (3 copies, incl. Tent-style entropy head tuning)",
+   "reason": "Merged into item 7. The Tent entropy-only update was dropped as unstable at about 55% initial accuracy."
+  },
+  {
+   "name": "Duplicate frozen-MLP last-layer entries (SimpleShot/DFR, MME frozen variant, LP head of LP-FT, TabM frozen backbone, sequence BN-head)",
+   "reason": "Merged into item 5 (MLP) and item 11 (TempCNN)."
+  },
+  {
+   "name": "BBSE confusion-matrix inversion",
+   "reason": "A 4x4 confusion matrix from 20-100 shots is too noisy, and Garg et al. show MLLS/EM is better; EM is kept in item 2."
+  },
+  {
+   "name": "Warning / negative-result protocol",
+   "reason": "Not a method; kept as the Madrid harness in notes. As written, its diagnostic scores methods on the remaining Amsterdam pixels, which is selection on test labels and breaks the rule."
+  },
+  {
+   "name": "Surgical first-layer FiLM tuning",
+   "reason": "Its init (per-city standardisation) is item 1. Training 786 affine parameters on 20 shots adds overfitting risk for a small, unverified extra gain."
+  },
+  {
+   "name": "MME minimax entropy (full adversarial version)",
+   "reason": "Known to be unstable, and it collapses to the majority class under label shift; its useful part (prototype-init cosine head + entropy) lives in items 5 and 8."
+  },
+  {
+   "name": "Full SHOT target adaptation of the feature extractor",
+   "reason": "Entropy-driven backbone updates are a gamble at about 60% pseudo-label accuracy; the cheap AdaBN part is kept in item 5."
+  },
+  {
+   "name": "Class-conditional alignment with d-SNE / FADA contrastive training",
+   "reason": "Metric learning overfits with 5 positives per class. The closed-form shrinkage (prototype mixing alpha = n/(n+k)) is already in item 5."
+  },
+  {
+   "name": "Separate IW-DANN / MCD backbone idea",
+   "reason": "IW-DANN is folded into item 5's backbone. MCD adds another unstable adversarial method with no unsupervised model-selection criterion."
+  },
+  {
+   "name": "Episodic meta-training on Madrid spatial blocks (ProtoNet / ANIL)",
+   "reason": "Costs 20-40 min. Madrid blocks are a weak proxy for the Madrid-to-Amsterdam shift, and supervised pretraining + probe is usually as good."
+  },
+  {
+   "name": "Multi-detector change-point features (LandTrendr/CCDC/BIC breaks)",
+   "reason": "Moderate engineering cost, and you already have a step detector; the gain (+0.01 to +0.03) is uncertain for a single night. The cheap ordinal-head part is kept as item 9."
+  },
+  {
+   "name": "TimeMatch-style year shift of the break-year feature chosen by max confidence",
+   "reason": "Break year IS the age signal, so shifting it moves the target directly, and confidence-maximisation can pick a wrong shift. Risky for small expected gain."
+  },
+  {
+   "name": "Classifier-based density-ratio / KMM source reweighting",
+   "reason": "With domain AUC 1.0 the weights are degenerate; expected gain about 0 to +0.015."
+  },
+  {
+   "name": "TrAdaBoost",
+   "reason": "Old and unstable (source weights collapse), the multiclass extension is unverified, and the shot error signal is too thin at n=5."
+  },
+  {
+   "name": "TabPFN v2 in-context on the shots",
+   "reason": "Needs a weight download (possibly with a terms prompt). CPU inference over 26k queries x 5 n takes about an hour, a large share of the 5-hour budget, and it is weak at n=5/25, which are the priority settings."
+  },
+  {
+   "name": "Separate 'architecture choice: GRU/transformer' idea",
+   "reason": "The GRU is about 3x slower and the transformer is riskier with no evidence for 41 yearly points. Only the TempCNN is kept (item 11)."
+  },
+  {
+   "name": "Separate 'cheap ensembling/calibration' and 'stacking' ideas",
+   "reason": "Merged into item 6. Learned stacking weights at n<=25 would overfit and are replaced by a fixed rule."
+  }
+ ],
+ "notes": "BIGGEST RED FLAG IN THE CURRENT RESULTS: at n=5, LightGBM+30x shots (0.566) is worse than zero-shot LightGBM (0.593) and far worse than zero-shot XGB+DANN+smoothing (0.639). Twenty shots weighted 30x override the source model (and also push its effective prior toward uniform). So at n=5 every pipeline should START from the 0.639 zero-shot model and let shots in gently (item 3). That alone should lift n=5 by about +0.07, and the other items then build on that level, not on 0.566.\n\nSUGGESTED ORDER TONIGHT (about 5 h): items 1+2+3+4 in the first hour on the existing XGB-DANN/LightGBM. Then item 5 (backbone, about 20 min of CPU) and items 6 and 8. Then item 7 (self-training, the largest CPU block). Items 9-12 only if time remains.\n\nCOMPOSITION: per-city inputs (1) -> backbones (5, 11, trees with 3/10) -> heads (3, 8, 9) -> ensemble (6) -> EM prior (2) -> graph spreading (4) -> optional self-training loop (7) -> EM + graph again. Always smooth AFTER prior correction, and compare everything with smoothing on.\n\nHYPERPARAMETER RULE: tune ONLY on a Madrid pseudo-target harness. Build it like this:\n1. Cluster the Madrid coordinates into 5 spatial blocks with KMeans.\n2. Hold out one block and re-standardise it with its own statistics, mimicking the per-city transform.\n3. Resample its labels to the EM-estimated Amsterdam prior to imitate label shift.\n4. Draw n shots per class from it over 20 seeds, train on the other 4 blocks (plus unlabelled = the held-out block), and score macro F1 on the rest of the block.\n\nThis underestimates the real city shift, so prefer conservative defaults when the harness is flat. After that, freeze one pipeline per n and evaluate on Amsterdam once.\n\nRULE FLAGS IN THE SCOUTS' RECIPES:\n(a) Idea 3 (label-shift head) says to tune gamma 'on Amsterdam shots simulated via your n=200 splits'. This uses Amsterdam labels beyond n for n<200, so it breaks the rule.\n(b) Idea 6 (source-regularised head) says to tune tau 'using simulated n-shot draws from the labelled Amsterdam pool'. This breaks the rule.\n(c) The Warning idea's 20-seed diagnostic scores on the remaining Amsterdam pixels, which is test-set model selection.\n(d) Choosing among existing pipelines by their reported Amsterdam test F1 is already mild test leakage; pre-register choices with the harness.\n\nLeave-shots-out CV on the n shots is legal but too noisy at n=5 (std about 0.03-0.05), so it is used nowhere for tuning here. EM priors, AdaBN, centring, graphs and pseudo-labels use only unlabelled Amsterdam features and coordinates, which is allowed.\n\nLABEL-SHIFT TRAPS:\n- Centring or standardising on the unlabelled Amsterdam mean bakes in the prior difference. Item 1 handles it with a prior-reweighted Madrid pass.\n- When EM is used, pi_S must be the model's effective training prior (Madrid + weighted shots, or uniform for balanced heads).\n\nGains quoted are guesses from the literature, not measured; expect them to overlap rather than add, especially items 4/7/8, which all exploit the same unlabelled structure."
+}

reports/nnlab/scores/cls_knn.json

@@ -0,0 +1,461 @@
+{
+ "method": "cls_knn",
+ "draws": [
+  {
+   "n": 5,
+   "rep": 0,
+   "argmax": 0.5221903557289336,
+   "argmax_A": 0.5302732879082349,
+   "argmax_B": 0.5137141488882171,
+   "rule": 0.5454851293814139,
+   "rule_A": 0.5515532967625634,
+   "rule_B": 0.5389193256438298,
+   "rule_smooth": 0.4988736472287242,
+   "rule_smooth_A": 0.5009881027559154,
+   "rule_smooth_B": 0.4958580103200862
+  },
+  {
+   "n": 5,
+   "rep": 1,
+   "argmax": 0.5119907561381871,
+   "argmax_A": 0.5216378510358448,
+   "argmax_B": 0.5017868353933344,
+   "rule": 0.5486684112823532,
+   "rule_A": 0.5567088430941621,
+   "rule_B": 0.5400570590243093,
+   "rule_smooth": 0.4974621711285271,
+   "rule_smooth_A": 0.5015480669622733,
+   "rule_smooth_B": 0.4921200292601477
+  },
+  {
+   "n": 5,
+   "rep": 2,
+   "argmax": 0.4955448493912643,
+   "argmax_A": 0.49981843281848437,
+   "argmax_B": 0.49046218924647844,
+   "rule": 0.5339155884633616,
+   "rule_A": 0.5377915654653465,
+   "rule_B": 0.5294515523848216,
+   "rule_smooth": 0.4764034671843657,
+   "rule_smooth_A": 0.4763656053838464,
+   "rule_smooth_B": 0.47547060359737325
+  },
+  {
+   "n": 5,
+   "rep": 3,
+   "argmax": 0.5322824841553842,
+   "argmax_A": 0.5443454407106949,
+   "argmax_B": 0.5198616434356386,
+   "rule": 0.5494137520343005,
+   "rule_A": 0.5571559413577993,
+   "rule_B": 0.5412073549562813,
+   "rule_smooth": 0.5193325034378558,
+   "rule_smooth_A": 0.5388496531318825,
+   "rule_smooth_B": 0.49947994656039496
+  },
+  {
+   "n": 5,
+   "rep": 4,
+   "argmax": 0.5343588765432901,
+   "argmax_A": 0.5425224646405052,
+   "argmax_B": 0.5255996824938862,
+   "rule": 0.5581429662373161,
+   "rule_A": 0.564124005508738,
+   "rule_B": 0.5516385961570824,
+   "rule_smooth": 0.5216537367869656,
+   "rule_smooth_A": 0.5342395115652971,
+   "rule_smooth_B": 0.5085522770630639
+  },
+  {
+   "n": 25,
+   "rep": 0,
+   "argmax": 0.544530016242307,
+   "argmax_A": 0.5471698353756039,
+   "argmax_B": 0.5402462130136624,
+   "rule": 0.559623558999716,
+   "rule_A": 0.5642833752360714,
+   "rule_B": 0.553493280496952,
+   "rule_smooth": 0.6074027851960493,
+   "rule_smooth_A": 0.6163202934529866,
+   "rule_smooth_B": 0.5968257402012866
+  },
+  {
+   "n": 25,
+   "rep": 1,
+   "argmax": 0.558450208191207,
+   "argmax_A": 0.5646878830194714,
+   "argmax_B": 0.5509845721333538,
+   "rule": 0.5632929456796664,
+   "rule_A": 0.5671973635078328,
+   "rule_B": 0.5580549886408409,
+   "rule_smooth": 0.604686208423393,
+   "rule_smooth_A": 0.613197313589069,
+   "rule_smooth_B": 0.5946568002741754
+  },
+  {
+   "n": 25,
+   "rep": 2,
+   "argmax": 0.541774006522576,
+   "argmax_A": 0.5489758253640541,
+   "argmax_B": 0.5338765396538827,
+   "rule": 0.5557954740367835,
+   "rule_A": 0.5618169186879821,
+   "rule_B": 0.5490849941117011,
+   "rule_smooth": 0.6078555078108219,
+   "rule_smooth_A": 0.6168670376829177,
+   "rule_smooth_B": 0.5976643479793631
+  },
+  {
+   "n": 25,
+   "rep": 3,
+   "argmax": 0.5617255945826463,
+   "argmax_A": 0.570500997259324,
+   "argmax_B": 0.5523374207428853,
+   "rule": 0.5735561719595167,
+   "rule_A": 0.5801337610455867,
+   "rule_B": 0.566332744860759,
+   "rule_smooth": 0.6258791561467171,
+   "rule_smooth_A": 0.6397088259301007,
+   "rule_smooth_B": 0.6116354228874683
+  },
+  {
+   "n": 25,
+   "rep": 4,
+   "argmax": 0.5669293720680795,
+   "argmax_A": 0.5762971034417004,
+   "argmax_B": 0.5568006196170951,
+   "rule": 0.5787920531692807,
+   "rule_A": 0.5860273796651972,
+   "rule_B": 0.5706697796028596,
+   "rule_smooth": 0.6291777732196673,
+   "rule_smooth_A": 0.6415976761707962,
+   "rule_smooth_B": 0.6159451861927173
+  },
+  {
+   "n": 50,
+   "rep": 0,
+   "argmax": 0.603386377404125,
+   "argmax_A": 0.6117882340923513,
+   "argmax_B": 0.5939551995304603,
+   "rule": 0.602221285968398,
+   "rule_A": 0.6097035429056695,
+   "rule_B": 0.593746959674469,
+   "rule_smooth": 0.6544238156601488,
+   "rule_smooth_A": 0.6631652217525043,
+   "rule_smooth_B": 0.6442062341214111
+  },
+  {
+   "n": 50,
+   "rep": 1,
+   "argmax": 0.5867541505778833,
+   "argmax_A": 0.5990635727855224,
+   "argmax_B": 0.5743881408080364,
+   "rule": 0.5889920796011013,
+   "rule_A": 0.5992784828229465,
+   "rule_B": 0.578372418536182,
+   "rule_smooth": 0.6370535967493842,
+   "rule_smooth_A": 0.6525581574202475,
+   "rule_smooth_B": 0.6213405839357088
+  },
+  {
+   "n": 50,
+   "rep": 2,
+   "argmax": 0.5803315928957401,
+   "argmax_A": 0.5897071836070953,
+   "argmax_B": 0.5700945710489879,
+   "rule": 0.5803905428097911,
+   "rule_A": 0.5900060315642504,
+   "rule_B": 0.5699317799685054,
+   "rule_smooth": 0.6263129312093206,
+   "rule_smooth_A": 0.6357785751085238,
+   "rule_smooth_B": 0.6154357027528483
+  },
+  {
+   "n": 50,
+   "rep": 3,
+   "argmax": 0.5932178894904423,
+   "argmax_A": 0.6119446463523107,
+   "argmax_B": 0.5743588058131927,
+   "rule": 0.5899236440382118,
+   "rule_A": 0.6059913107845565,
+   "rule_B": 0.573583837831281,
+   "rule_smooth": 0.6306866073871119,
+   "rule_smooth_A": 0.6499497929396838,
+   "rule_smooth_B": 0.6110396012391656
+  },
+  {
+   "n": 50,
+   "rep": 4,
+   "argmax": 0.584640103491048,
+   "argmax_A": 0.5963064006978047,
+   "argmax_B": 0.5725211920433917,
+   "rule": 0.586550255496391,
+   "rule_A": 0.5976181890404303,
+   "rule_B": 0.5749251132005265,
+   "rule_smooth": 0.635704167626227,
+   "rule_smooth_A": 0.650970309790749,
+   "rule_smooth_B": 0.6200737890484778
+  },
+  {
+   "n": 100,
+   "rep": 0,
+   "argmax": 0.6189458337707394,
+   "argmax_A": 0.6295102300664865,
+   "argmax_B": 0.607580884805924,
+   "rule": 0.6127645342104602,
+   "rule_A": 0.620948837041724,
+   "rule_B": 0.6036032548733399,
+   "rule_smooth": 0.6637363166315352,
+   "rule_smooth_A": 0.6724124163940371,
+   "rule_smooth_B": 0.6537407968776158
+  },
+  {
+   "n": 100,
+   "rep": 1,
+   "argmax": 0.6320445907544573,
+   "argmax_A": 0.6461112166151115,
+   "argmax_B": 0.6172143255268754,
+   "rule": 0.6264960572188512,
+   "rule_A": 0.6399680588672318,
+   "rule_B": 0.6122020742824107,
+   "rule_smooth": 0.6676434316984261,
+   "rule_smooth_A": 0.6831404389938729,
+   "rule_smooth_B": 0.6512097393544525
+  },
+  {
+   "n": 100,
+   "rep": 2,
+   "argmax": 0.6038537634556386,
+   "argmax_A": 0.6146148306699405,
+   "argmax_B": 0.5919478709617256,
+   "rule": 0.6013588082843965,
+   "rule_A": 0.611935508322315,
+   "rule_B": 0.5896418059501088,
+   "rule_smooth": 0.6447903063152287,
+   "rule_smooth_A": 0.6532219424710841,
+   "rule_smooth_B": 0.6345440837470699
+  },
+  {
+   "n": 100,
+   "rep": 3,
+   "argmax": 0.6323714605965217,
+   "argmax_A": 0.645612677252045,
+   "argmax_B": 0.6178149454293874,
+   "rule": 0.6243849486542664,
+   "rule_A": 0.6356778963786193,
+   "rule_B": 0.6115045399208938,
+   "rule_smooth": 0.6660971087725769,
+   "rule_smooth_A": 0.6785921504414572,
+   "rule_smooth_B": 0.6515944695264607
+  },
+  {
+   "n": 100,
+   "rep": 4,
+   "argmax": 0.6311532327456532,
+   "argmax_A": 0.6383942901798509,
+   "argmax_B": 0.6223189631412858,
+   "rule": 0.6257293660968859,
+   "rule_A": 0.6312913897133126,
+   "rule_B": 0.6184668641257095,
+   "rule_smooth": 0.6688993466479674,
+   "rule_smooth_A": 0.6784561996783511,
+   "rule_smooth_B": 0.6577820713709417
+  },
+  {
+   "n": 200,
+   "rep": 0,
+   "argmax": 0.6537334954916916,
+   "argmax_A": 0.6710772227200532,
+   "argmax_B": 0.6350410740675069,
+   "rule": 0.6480983956363349,
+   "rule_A": 0.6619239570350949,
+   "rule_B": 0.6326581896478654,
+   "rule_smooth": 0.686174976385777,
+   "rule_smooth_A": 0.7052496280229952,
+   "rule_smooth_B": 0.6658619106931702
+  },
+  {
+   "n": 200,
+   "rep": 1,
+   "argmax": 0.6454152375668575,
+   "argmax_A": 0.6579287762035422,
+   "argmax_B": 0.6320075801340217,
+   "rule": 0.6390228838533115,
+   "rule_A": 0.6510045637139471,
+   "rule_B": 0.6260224087686114,
+   "rule_smooth": 0.6798270063838376,
+   "rule_smooth_A": 0.6910667143125047,
+   "rule_smooth_B": 0.6672500270435481
+  },
+  {
+   "n": 200,
+   "rep": 2,
+   "argmax": 0.6496980832386541,
+   "argmax_A": 0.6631652720465674,
+   "argmax_B": 0.6352843352399209,
+   "rule": 0.6423985693430536,
+   "rule_A": 0.6563207949033433,
+   "rule_B": 0.6274672222203319,
+   "rule_smooth": 0.6801225589583662,
+   "rule_smooth_A": 0.6939003903442055,
+   "rule_smooth_B": 0.6650113270419143
+  },
+  {
+   "n": 200,
+   "rep": 3,
+   "argmax": 0.6548203343906911,
+   "argmax_A": 0.6690725566067011,
+   "argmax_B": 0.6393782617753303,
+   "rule": 0.646962826468885,
+   "rule_A": 0.6604522437735978,
+   "rule_B": 0.6323445529612446,
+   "rule_smooth": 0.6854995559353292,
+   "rule_smooth_A": 0.7009115788190119,
+   "rule_smooth_B": 0.6688407921706513
+  },
+  {
+   "n": 200,
+   "rep": 4,
+   "argmax": 0.655947409534756,
+   "argmax_A": 0.6741178952088,
+   "argmax_B": 0.6369700066645203,
+   "rule": 0.6482066892777492,
+   "rule_A": 0.6633463688687273,
+   "rule_B": 0.6322394663610543,
+   "rule_smooth": 0.6874625064551962,
+   "rule_smooth_A": 0.7070045694473963,
+   "rule_smooth_B": 0.6670577418417787
+  }
+ ],
+ "n_draws": 25,
+ "mean": {
+  "argmax": {
+   "5": 0.5192734643914119,
+   "25": 0.5546818395213632,
+   "50": 0.5896660227718478,
+   "100": 0.6236737762646021,
+   "200": 0.6519229120445301
+  },
+  "argmax_A": {
+   "5": 0.5277194954227529,
+   "25": 0.5615263288920308,
+   "50": 0.6017620075070169,
+   "100": 0.6348486489566869,
+   "200": 0.6670723445571328
+  },
+  "argmax_B": {
+   "5": 0.5102848998915109,
+   "25": 0.5468490730321759,
+   "50": 0.5770635818488138,
+   "100": 0.6113753979730396,
+   "200": 0.63573625157626
+  },
+  "rule": {
+   "5": 0.5471251694797491,
+   "25": 0.5662120407689927,
+   "50": 0.5896155615827786,
+   "100": 0.618146742892972,
+   "200": 0.6449378729158669
+  },
+  "rule_A": {
+   "5": 0.5534667304377219,
+   "25": 0.571891759628534,
+   "50": 0.6005195114235706,
+   "100": 0.6279643380646405,
+   "200": 0.6586095856589421
+  },
+  "rule_B": {
+   "5": 0.5402547776332649,
+   "25": 0.5595271575426225,
+   "50": 0.5781120218421928,
+   "100": 0.6070837078304925,
+   "200": 0.6301463679918216
+  },
+  "rule_smooth": {
+   "5": 0.5027451051532876,
+   "25": 0.6150002861593297,
+   "50": 0.6368362237264386,
+   "100": 0.6622333020131468,
+   "200": 0.6838173208237013
+  },
+  "rule_smooth_A": {
+   "5": 0.5103981879598429,
+   "25": 0.625538229365174,
+   "50": 0.6504844114023417,
+   "100": 0.6731646295957605,
+   "200": 0.6996265761892226
+  },
+  "rule_smooth_B": {
+   "5": 0.49429617336021325,
+   "25": 0.6033454995070022,
+   "50": 0.6224191822195223,
+   "100": 0.6497742321753082,
+   "200": 0.6668043597582125
+  }
+ },
+ "std": {
+  "argmax": {
+   "5": 0.01597965518568202,
+   "25": 0.010994189486859093,
+   "50": 0.008968477209442832,
+   "100": 0.012418221140585878,
+   "200": 0.004335018426800814
+  },
+  "argmax_A": {
+   "5": 0.018148880842454106,
+   "25": 0.01296475918108159,
+   "50": 0.00983075948560391,
+   "100": 0.01316422639006545,
+   "200": 0.006491335362771758
+  },
+  "argmax_B": {
+   "5": 0.014170156044317858,
+   "25": 0.009462319688061754,
+   "50": 0.009604698885714108,
+   "100": 0.012115585014392555,
+   "200": 0.0027100221759162155
+  },
+  "rule": {
+   "5": 0.008749197290523404,
+   "25": 0.009651775038642054,
+   "50": 0.007966884658262622,
+   "100": 0.01091931035149454,
+   "200": 0.004068465909013989
+  },
+  "rule_A": {
+   "5": 0.009837061412962523,
+   "25": 0.010596858368858203,
+   "50": 0.007661235292397016,
+   "100": 0.011410501798273891,
+   "200": 0.004997311367871731
+  },
+  "rule_B": {
+   "5": 0.00789035690213935,
+   "25": 0.008917507719617883,
+   "50": 0.009247922492327906,
+   "100": 0.011086604222448651,
+   "200": 0.0031506819783359524
+  },
+  "rule_smooth": {
+   "5": 0.018503091642793148,
+   "25": 0.01155963915556857,
+   "50": 0.010714588869779254,
+   "100": 0.009939212856406894,
+   "200": 0.003579445054011733
+  },
+  "rule_smooth_A": {
+   "5": 0.025995122637259217,
+   "25": 0.013884971629184787,
+   "50": 0.009770963475121224,
+   "100": 0.011781893566724402,
+   "200": 0.00695997787140304
+  },
+  "rule_smooth_B": {
+   "5": 0.012162288149299733,
+   "25": 0.009717910946741171,
+   "50": 0.01284137573149952,
+   "100": 0.008904921253761747,
+   "200": 0.001458857030506801
+  }
+ }
+}

reports/nnlab/scores/cls_lda.json

@@ -0,0 +1,461 @@
+{
+ "method": "cls_lda",
+ "draws": [
+  {
+   "n": 5,
+   "rep": 0,
+   "argmax": 0.5376312190255781,
+   "argmax_A": 0.5511587212443828,
+   "argmax_B": 0.5244410970022535,
+   "rule": 0.597109970624901,
+   "rule_A": 0.6133373167877392,
+   "rule_B": 0.5804332526817574,
+   "rule_smooth": 0.6204846931717696,
+   "rule_smooth_A": 0.641590791030382,
+   "rule_smooth_B": 0.5988694475696076
+  },
+  {
+   "n": 5,
+   "rep": 1,
+   "argmax": 0.5432043683353756,
+   "argmax_A": 0.555882723304179,
+   "argmax_B": 0.5305616258305014,
+   "rule": 0.6052209577522911,
+   "rule_A": 0.6207476568961575,
+   "rule_B": 0.5891507589456564,
+   "rule_smooth": 0.6273616250270102,
+   "rule_smooth_A": 0.647934095634735,
+   "rule_smooth_B": 0.6059847511241502
+  },
+  {
+   "n": 5,
+   "rep": 2,
+   "argmax": 0.5409019352230329,
+   "argmax_A": 0.5528653795823546,
+   "argmax_B": 0.5291946042872189,
+   "rule": 0.5994202488680378,
+   "rule_A": 0.6156571176762564,
+   "rule_B": 0.5827821767312183,
+   "rule_smooth": 0.6209493871264053,
+   "rule_smooth_A": 0.642309907101086,
+   "rule_smooth_B": 0.5990755086519977
+  },
+  {
+   "n": 5,
+   "rep": 3,
+   "argmax": 0.540563179070726,
+   "argmax_A": 0.5539029533781534,
+   "argmax_B": 0.5274617926474963,
+   "rule": 0.5999969337839951,
+   "rule_A": 0.6173426655854667,
+   "rule_B": 0.5824472395914366,
+   "rule_smooth": 0.6227565175203399,
+   "rule_smooth_A": 0.6427843641989021,
+   "rule_smooth_B": 0.6020225913136547
+  },
+  {
+   "n": 5,
+   "rep": 4,
+   "argmax": 0.5339379393758984,
+   "argmax_A": 0.5483414331525718,
+   "argmax_B": 0.5199264880333394,
+   "rule": 0.5982953982596477,
+   "rule_A": 0.6135831098482114,
+   "rule_B": 0.5825385584940495,
+   "rule_smooth": 0.6214677536444109,
+   "rule_smooth_A": 0.6400708636971397,
+   "rule_smooth_B": 0.6021166887804097
+  },
+  {
+   "n": 25,
+   "rep": 0,
+   "argmax": 0.5924303022919786,
+   "argmax_A": 0.6018566697766148,
+   "argmax_B": 0.5822616285260945,
+   "rule": 0.6296930417854193,
+   "rule_A": 0.6435412647376821,
+   "rule_B": 0.614709942984425,
+   "rule_smooth": 0.6505928983692424,
+   "rule_smooth_A": 0.6685650790803244,
+   "rule_smooth_B": 0.6312804076525025
+  },
+  {
+   "n": 25,
+   "rep": 1,
+   "argmax": 0.5685924262811393,
+   "argmax_A": 0.5768334571646407,
+   "argmax_B": 0.5597185244929823,
+   "rule": 0.6167038566106902,
+   "rule_A": 0.6317391633537222,
+   "rule_B": 0.6006220518263845,
+   "rule_smooth": 0.6387596199864828,
+   "rule_smooth_A": 0.6588996505033098,
+   "rule_smooth_B": 0.6174762423610116
+  },
+  {
+   "n": 25,
+   "rep": 2,
+   "argmax": 0.5791907569749017,
+   "argmax_A": 0.5908938635429828,
+   "argmax_B": 0.567101855072258,
+   "rule": 0.6241921398268802,
+   "rule_A": 0.6390998285901409,
+   "rule_B": 0.6083444198783375,
+   "rule_smooth": 0.6451633951190602,
+   "rule_smooth_A": 0.6669285196642853,
+   "rule_smooth_B": 0.6225897355233294
+  },
+  {
+   "n": 25,
+   "rep": 3,
+   "argmax": 0.5987953379792242,
+   "argmax_A": 0.6073206807987017,
+   "argmax_B": 0.5894081502324778,
+   "rule": 0.6304748644665593,
+   "rule_A": 0.6451356290721328,
+   "rule_B": 0.6150534821589685,
+   "rule_smooth": 0.647941571444465,
+   "rule_smooth_A": 0.6638833578658001,
+   "rule_smooth_B": 0.6308059903964075
+  },
+  {
+   "n": 25,
+   "rep": 4,
+   "argmax": 0.5835812664959577,
+   "argmax_A": 0.59500344900087,
+   "argmax_B": 0.571943805178859,
+   "rule": 0.62417631781401,
+   "rule_A": 0.6381636640630262,
+   "rule_B": 0.6095104444090118,
+   "rule_smooth": 0.6474225767397987,
+   "rule_smooth_A": 0.6649060879469669,
+   "rule_smooth_B": 0.6290442601995351
+  },
+  {
+   "n": 50,
+   "rep": 0,
+   "argmax": 0.6165964472543028,
+   "argmax_A": 0.6279470662172173,
+   "argmax_B": 0.6045408442125846,
+   "rule": 0.6497961842518227,
+   "rule_A": 0.664720616241929,
+   "rule_B": 0.6338733668200732,
+   "rule_smooth": 0.6696708904179026,
+   "rule_smooth_A": 0.6872742220805987,
+   "rule_smooth_B": 0.650911859231482
+  },
+  {
+   "n": 50,
+   "rep": 1,
+   "argmax": 0.6190953793280723,
+   "argmax_A": 0.6240964715301771,
+   "argmax_B": 0.6125930497189052,
+   "rule": 0.6471250542318957,
+   "rule_A": 0.6580672413581583,
+   "rule_B": 0.63480258067383,
+   "rule_smooth": 0.6640034006574473,
+   "rule_smooth_A": 0.6781780786291205,
+   "rule_smooth_B": 0.6484363279897325
+  },
+  {
+   "n": 50,
+   "rep": 2,
+   "argmax": 0.621897300498004,
+   "argmax_A": 0.6328792231020584,
+   "argmax_B": 0.6102086028394144,
+   "rule": 0.6527743976540556,
+   "rule_A": 0.66532315766734,
+   "rule_B": 0.6392555668949318,
+   "rule_smooth": 0.6695851787481367,
+   "rule_smooth_A": 0.6836622766391084,
+   "rule_smooth_B": 0.6542694776348812
+  },
+  {
+   "n": 50,
+   "rep": 3,
+   "argmax": 0.6107088149848042,
+   "argmax_A": 0.625748760187766,
+   "argmax_B": 0.5950326158384989,
+   "rule": 0.6485882183270834,
+   "rule_A": 0.6641668520073105,
+   "rule_B": 0.6320351744447166,
+   "rule_smooth": 0.6645013161637355,
+   "rule_smooth_A": 0.6830963395158274,
+   "rule_smooth_B": 0.644604632887883
+  },
+  {
+   "n": 50,
+   "rep": 4,
+   "argmax": 0.6086880801102584,
+   "argmax_A": 0.6170233115583239,
+   "argmax_B": 0.599176265358904,
+   "rule": 0.642329491731711,
+   "rule_A": 0.651181093915563,
+   "rule_B": 0.6320753619393069,
+   "rule_smooth": 0.6629196308371988,
+   "rule_smooth_A": 0.6765172527046505,
+   "rule_smooth_B": 0.6479746342633872
+  },
+  {
+   "n": 100,
+   "rep": 0,
+   "argmax": 0.6518276443926183,
+   "argmax_A": 0.6643957878794547,
+   "argmax_B": 0.638235027733708,
+   "rule": 0.6723243933550449,
+   "rule_A": 0.6874574534326093,
+   "rule_B": 0.656323904789339,
+   "rule_smooth": 0.6849869753927855,
+   "rule_smooth_A": 0.6981834592270311,
+   "rule_smooth_B": 0.6704417901494407
+  },
+  {
+   "n": 100,
+   "rep": 1,
+   "argmax": 0.6555507781174014,
+   "argmax_A": 0.668728455895058,
+   "argmax_B": 0.6412028648187148,
+   "rule": 0.6696650646643971,
+   "rule_A": 0.6828876253946771,
+   "rule_B": 0.6553226500270429,
+   "rule_smooth": 0.6817498799946087,
+   "rule_smooth_A": 0.6929405844292991,
+   "rule_smooth_B": 0.668812907783929
+  },
+  {
+   "n": 100,
+   "rep": 2,
+   "argmax": 0.6513514466091217,
+   "argmax_A": 0.6635362289642015,
+   "argmax_B": 0.6382601277966944,
+   "rule": 0.670755992314365,
+   "rule_A": 0.6827269977544999,
+   "rule_B": 0.657605641538398,
+   "rule_smooth": 0.6836076694562915,
+   "rule_smooth_A": 0.6969623673448957,
+   "rule_smooth_B": 0.6690626592629947
+  },
+  {
+   "n": 100,
+   "rep": 3,
+   "argmax": 0.6680423379036571,
+   "argmax_A": 0.6803463827162416,
+   "argmax_B": 0.6545336869653126,
+   "rule": 0.6755487835704502,
+   "rule_A": 0.6898024337221698,
+   "rule_B": 0.6601438712065204,
+   "rule_smooth": 0.6864213376981829,
+   "rule_smooth_A": 0.6983126979364068,
+   "rule_smooth_B": 0.6727719066742097
+  },
+  {
+   "n": 100,
+   "rep": 4,
+   "argmax": 0.6670969285959001,
+   "argmax_A": 0.6834201977845926,
+   "argmax_B": 0.6497917001282397,
+   "rule": 0.6768222909502543,
+   "rule_A": 0.6915154147736503,
+   "rule_B": 0.6609970927635027,
+   "rule_smooth": 0.6852318391230063,
+   "rule_smooth_A": 0.6989588013517818,
+   "rule_smooth_B": 0.6700405721546938
+  },
+  {
+   "n": 200,
+   "rep": 0,
+   "argmax": 0.6838539601041201,
+   "argmax_A": 0.7017586454225216,
+   "argmax_B": 0.6652068616453237,
+   "rule": 0.6863533938025597,
+   "rule_A": 0.7021566541609952,
+   "rule_B": 0.6695451515050952,
+   "rule_smooth": 0.6913150130001513,
+   "rule_smooth_A": 0.7050692564565098,
+   "rule_smooth_B": 0.6761625419281433
+  },
+  {
+   "n": 200,
+   "rep": 1,
+   "argmax": 0.6892602171063519,
+   "argmax_A": 0.7029387899128811,
+   "argmax_B": 0.6746036738188707,
+   "rule": 0.69003550267791,
+   "rule_A": 0.7019773132096314,
+   "rule_B": 0.6769171007488275,
+   "rule_smooth": 0.6977275009537828,
+   "rule_smooth_A": 0.7082657350179621,
+   "rule_smooth_B": 0.685803551226173
+  },
+  {
+   "n": 200,
+   "rep": 2,
+   "argmax": 0.680666271641737,
+   "argmax_A": 0.694032424653578,
+   "argmax_B": 0.6658477441634941,
+   "rule": 0.6821713802730565,
+   "rule_A": 0.6952433599866797,
+   "rule_B": 0.6675238313828464,
+   "rule_smooth": 0.6861005571387531,
+   "rule_smooth_A": 0.6970726729991594,
+   "rule_smooth_B": 0.6729837854407852
+  },
+  {
+   "n": 200,
+   "rep": 3,
+   "argmax": 0.6890089104744515,
+   "argmax_A": 0.704763774902148,
+   "argmax_B": 0.6725860369583316,
+   "rule": 0.688995324579926,
+   "rule_A": 0.7053656147639216,
+   "rule_B": 0.6717530845252586,
+   "rule_smooth": 0.6962278128750664,
+   "rule_smooth_A": 0.7114057640426944,
+   "rule_smooth_B": 0.6799599533852225
+  },
+  {
+   "n": 200,
+   "rep": 4,
+   "argmax": 0.6880442712827526,
+   "argmax_A": 0.7017659120937871,
+   "argmax_B": 0.6732847523528462,
+   "rule": 0.6900930223127711,
+   "rule_A": 0.7030908635290128,
+   "rule_B": 0.6759990489987522,
+   "rule_smooth": 0.6950505728331087,
+   "rule_smooth_A": 0.7075490234525641,
+   "rule_smooth_B": 0.6810479993145169
+  }
+ ],
+ "n_draws": 25,
+ "mean": {
+  "argmax": {
+   "5": 0.5392477282061222,
+   "25": 0.5845180180046403,
+   "50": 0.6153972044350884,
+   "100": 0.6587738271237397,
+   "200": 0.6861667261218826
+  },
+  "argmax_A": {
+   "5": 0.5524302421323284,
+   "25": 0.594381624056762,
+   "50": 0.6255389665191086,
+   "100": 0.6720854106479097,
+   "200": 0.7010519093969831
+  },
+  "argmax_B": {
+   "5": 0.5263171215601619,
+   "25": 0.5740867927005343,
+   "50": 0.6043102755936614,
+   "100": 0.6444046814885339,
+   "200": 0.6703058137877733
+  },
+  "rule": {
+   "5": 0.6000087018577746,
+   "25": 0.6250480441007118,
+   "50": 0.6481226692393137,
+   "100": 0.6730233049709022,
+   "200": 0.6875297247292447
+  },
+  "rule_A": {
+   "5": 0.6161335733587663,
+   "25": 0.6395359099633409,
+   "50": 0.6606917922380602,
+   "100": 0.6868779850155213,
+   "200": 0.7015667611300481
+  },
+  "rule_B": {
+   "5": 0.5834703972888237,
+   "25": 0.6096480682514255,
+   "50": 0.6344084101545717,
+   "100": 0.6580786320649606,
+   "200": 0.672347643432156
+  },
+  "rule_smooth": {
+   "5": 0.6226039952979872,
+   "25": 0.6459760123318098,
+   "50": 0.6661360833648843,
+   "100": 0.684399540332975,
+   "200": 0.6932842913601724
+  },
+  "rule_smooth_A": {
+   "5": 0.642938004332449,
+   "25": 0.6646365390121373,
+   "50": 0.681745633913861,
+   "100": 0.6970715820578829,
+   "200": 0.705872490393778
+  },
+  "rule_smooth_B": {
+   "5": 0.601613797487964,
+   "25": 0.6262393272265572,
+   "50": 0.6492393864014732,
+   "100": 0.6702259672050536,
+   "200": 0.6791915662589683
+  }
+ },
+ "std": {
+  "argmax": {
+   "5": 0.003568219874697839,
+   "25": 0.011718871463497996,
+   "50": 0.005575824480496659,
+   "100": 0.008199215543704294,
+   "200": 0.0037675177616421913
+  },
+  "argmax_A": {
+   "5": 0.002855388480461047,
+   "25": 0.011659596892917854,
+   "50": 0.005794814394076988,
+   "100": 0.0092223778108273,
+   "200": 0.004111590487069576
+  },
+  "argmax_B": {
+   "5": 0.004241628822152028,
+   "25": 0.011846949162860682,
+   "50": 0.007346441400756388,
+   "100": 0.007377135610903094,
+   "200": 0.00442771925642443
+  },
+  "rule": {
+   "5": 0.0031165717903814095,
+   "25": 0.005525946219599845,
+   "50": 0.003847470176374117,
+   "100": 0.0030707205471028174,
+   "200": 0.003357566466664728
+  },
+  "rule_A": {
+   "5": 0.003053792553930771,
+   "25": 0.0052502905737514104,
+   "50": 0.006064211797428514,
+   "100": 0.003985837546571218,
+   "200": 0.003783388225918875
+  },
+  "rule_B": {
+   "5": 0.003312083130966324,
+   "25": 0.005874276279513161,
+   "50": 0.002958544065200969,
+   "100": 0.0024331487009695897,
+   "200": 0.004052447617265657
+  },
+  "rule_smooth": {
+   "5": 0.002791886390318063,
+   "25": 0.004472367189492407,
+   "50": 0.003238729923143022,
+   "100": 0.001786982456922483,
+   "200": 0.004663964924457301
+  },
+  "rule_smooth_A": {
+   "5": 0.0029755049498937265,
+   "25": 0.003682337253951456,
+   "50": 0.004362528878280476,
+   "100": 0.0024194681950461504,
+   "200": 0.005413812684670389
+  },
+  "rule_smooth_B": {
+   "5": 0.0028939425712513713,
+   "25": 0.006005117178532746,
+   "50": 0.0035994591294638747,
+   "100": 0.0015742284573970603,
+   "200": 0.004885351541992635
+  }
+ }
+}

reports/nnlab/scores/cls_lr.json

@@ -0,0 +1,461 @@
+{
+ "method": "cls_lr",
+ "draws": [
+  {
+   "n": 5,
+   "rep": 0,
+   "argmax": 0.5548656186828875,
+   "argmax_A": 0.5634249028851491,
+   "argmax_B": 0.5455699494001431,
+   "rule": 0.5739622800094296,
+   "rule_A": 0.5853346078675083,
+   "rule_B": 0.5622513093159801,
+   "rule_smooth": 0.5985767939969655,
+   "rule_smooth_A": 0.6090592200568539,
+   "rule_smooth_B": 0.5872021154790553
+  },
+  {
+   "n": 5,
+   "rep": 1,
+   "argmax": 0.5539956298371425,
+   "argmax_A": 0.5649656488145789,
+   "argmax_B": 0.5424173301242894,
+   "rule": 0.5615243554328875,
+   "rule_A": 0.5734292081024772,
+   "rule_B": 0.5492400986036872,
+   "rule_smooth": 0.58583297718226,
+   "rule_smooth_A": 0.6019161493333052,
+   "rule_smooth_B": 0.5695643933562766
+  },
+  {
+   "n": 5,
+   "rep": 2,
+   "argmax": 0.516624902153029,
+   "argmax_A": 0.5112241453049694,
+   "argmax_B": 0.5206448554713254,
+   "rule": 0.5324000911038262,
+   "rule_A": 0.5301857126717627,
+   "rule_B": 0.5336967531051505,
+   "rule_smooth": 0.5518588511324238,
+   "rule_smooth_A": 0.5514958349245604,
+   "rule_smooth_B": 0.5514953052708527
+  },
+  {
+   "n": 5,
+   "rep": 3,
+   "argmax": 0.5412386964610093,
+   "argmax_A": 0.5333414582372202,
+   "argmax_B": 0.5474815005912389,
+   "rule": 0.5576841778722788,
+   "rule_A": 0.5532582365434697,
+   "rule_B": 0.5608085910048913,
+   "rule_smooth": 0.5803190230213608,
+   "rule_smooth_A": 0.5800550600656775,
+   "rule_smooth_B": 0.5795023466876599
+  },
+  {
+   "n": 5,
+   "rep": 4,
+   "argmax": 0.5942849693005126,
+   "argmax_A": 0.6014139945287763,
+   "argmax_B": 0.5860322848879799,
+   "rule": 0.5940689252722333,
+   "rule_A": 0.6032404736961919,
+   "rule_B": 0.583939767646517,
+   "rule_smooth": 0.6201145383193176,
+   "rule_smooth_A": 0.6293581354183474,
+   "rule_smooth_B": 0.6095304732532667
+  },
+  {
+   "n": 25,
+   "rep": 0,
+   "argmax": 0.5831884712015479,
+   "argmax_A": 0.5845871734831471,
+   "argmax_B": 0.5801844232068519,
+   "rule": 0.612923261971633,
+   "rule_A": 0.6171742408371519,
+   "rule_B": 0.6072576141288245,
+   "rule_smooth": 0.6311399121420458,
+   "rule_smooth_A": 0.6371820206508436,
+   "rule_smooth_B": 0.623566353473806
+  },
+  {
+   "n": 25,
+   "rep": 1,
+   "argmax": 0.578409368840873,
+   "argmax_A": 0.5805832357313799,
+   "argmax_B": 0.5743008557483109,
+   "rule": 0.6096346005315425,
+   "rule_A": 0.6164085539314712,
+   "rule_B": 0.6011977797247989,
+   "rule_smooth": 0.633470938398173,
+   "rule_smooth_A": 0.6393831225422246,
+   "rule_smooth_B": 0.6253607914180956
+  },
+  {
+   "n": 25,
+   "rep": 2,
+   "argmax": 0.5912465304688213,
+   "argmax_A": 0.5994464274042361,
+   "argmax_B": 0.5822971747508907,
+   "rule": 0.6084508745234708,
+   "rule_A": 0.620900354013268,
+   "rule_B": 0.5954054688818808,
+   "rule_smooth": 0.6299409931199086,
+   "rule_smooth_A": 0.641659818882109,
+   "rule_smooth_B": 0.6173624096762249
+  },
+  {
+   "n": 25,
+   "rep": 3,
+   "argmax": 0.6253036451735685,
+   "argmax_A": 0.6283794161245265,
+   "argmax_B": 0.620478912496694,
+   "rule": 0.6372383444011859,
+   "rule_A": 0.6432540686835589,
+   "rule_B": 0.6295495265073567,
+   "rule_smooth": 0.6562796273521703,
+   "rule_smooth_A": 0.6586172727535156,
+   "rule_smooth_B": 0.6518386150113097
+  },
+  {
+   "n": 25,
+   "rep": 4,
+   "argmax": 0.5600794601213908,
+   "argmax_A": 0.5723544689588811,
+   "argmax_B": 0.5486749045555436,
+   "rule": 0.5697150195121852,
+   "rule_A": 0.582842075632655,
+   "rule_B": 0.5574862783951486,
+   "rule_smooth": 0.5928295473115635,
+   "rule_smooth_A": 0.6063297896334501,
+   "rule_smooth_B": 0.5799512360745733
+  },
+  {
+   "n": 50,
+   "rep": 0,
+   "argmax": 0.6372925348390761,
+   "argmax_A": 0.6463507106889425,
+   "argmax_B": 0.6275612646431169,
+   "rule": 0.65548055237134,
+   "rule_A": 0.6643144818456029,
+   "rule_B": 0.6461741322451078,
+   "rule_smooth": 0.6802260516837282,
+   "rule_smooth_A": 0.6888796284796427,
+   "rule_smooth_B": 0.670660641952796
+  },
+  {
+   "n": 50,
+   "rep": 1,
+   "argmax": 0.644122749632134,
+   "argmax_A": 0.6486388153520081,
+   "argmax_B": 0.6385350505056465,
+   "rule": 0.6635762381921503,
+   "rule_A": 0.6692344993142946,
+   "rule_B": 0.6569429412944168,
+   "rule_smooth": 0.6905313206418042,
+   "rule_smooth_A": 0.6970992590808128,
+   "rule_smooth_B": 0.6830122285279523
+  },
+  {
+   "n": 50,
+   "rep": 2,
+   "argmax": 0.6319382123168779,
+   "argmax_A": 0.6381773268693284,
+   "argmax_B": 0.6248582055523514,
+   "rule": 0.6628672715259226,
+   "rule_A": 0.6700965397415284,
+   "rule_B": 0.654943655618291,
+   "rule_smooth": 0.6867169577477583,
+   "rule_smooth_A": 0.6940127776741578,
+   "rule_smooth_B": 0.6782053743042744
+  },
+  {
+   "n": 50,
+   "rep": 3,
+   "argmax": 0.6371326747166447,
+   "argmax_A": 0.6467964642959251,
+   "argmax_B": 0.6268528015574905,
+   "rule": 0.6588885436694032,
+   "rule_A": 0.6714900089044978,
+   "rule_B": 0.6457879363028907,
+   "rule_smooth": 0.677909467415388,
+   "rule_smooth_A": 0.6895755824107908,
+   "rule_smooth_B": 0.6654502012907014
+  },
+  {
+   "n": 50,
+   "rep": 4,
+   "argmax": 0.6193783386011311,
+   "argmax_A": 0.6212345689812175,
+   "argmax_B": 0.6163025124582051,
+   "rule": 0.6471297851778782,
+   "rule_A": 0.6505266741590776,
+   "rule_B": 0.6428222768018685,
+   "rule_smooth": 0.6695360806293686,
+   "rule_smooth_A": 0.6729601599787884,
+   "rule_smooth_B": 0.6648938824496708
+  },
+  {
+   "n": 100,
+   "rep": 0,
+   "argmax": 0.6578990545296379,
+   "argmax_A": 0.669325205686045,
+   "argmax_B": 0.6459568919160734,
+   "rule": 0.677475086613163,
+   "rule_A": 0.6893076923310808,
+   "rule_B": 0.6651931707605897,
+   "rule_smooth": 0.6971974597551633,
+   "rule_smooth_A": 0.7108585536640983,
+   "rule_smooth_B": 0.6828855261104456
+  },
+  {
+   "n": 100,
+   "rep": 1,
+   "argmax": 0.6586062689210499,
+   "argmax_A": 0.6670682756387134,
+   "argmax_B": 0.6489471079241501,
+   "rule": 0.6764587721091635,
+   "rule_A": 0.6851089164655914,
+   "rule_B": 0.6667776130990601,
+   "rule_smooth": 0.6978458938519413,
+   "rule_smooth_A": 0.7043191434508486,
+   "rule_smooth_B": 0.6899615440168321
+  },
+  {
+   "n": 100,
+   "rep": 2,
+   "argmax": 0.6700752235047815,
+   "argmax_A": 0.6764323989562697,
+   "argmax_B": 0.6631654149997821,
+   "rule": 0.683472596543626,
+   "rule_A": 0.6921631749719601,
+   "rule_B": 0.6744696439905771,
+   "rule_smooth": 0.7055714537979028,
+   "rule_smooth_A": 0.7110103335402839,
+   "rule_smooth_B": 0.6993938402792548
+  },
+  {
+   "n": 100,
+   "rep": 3,
+   "argmax": 0.6616571681458806,
+   "argmax_A": 0.6709632131665214,
+   "argmax_B": 0.6512014730722008,
+   "rule": 0.6809949592537503,
+   "rule_A": 0.689649059070933,
+   "rule_B": 0.6715282844921435,
+   "rule_smooth": 0.7064679273846705,
+   "rule_smooth_A": 0.7123476230987695,
+   "rule_smooth_B": 0.6993118496860938
+  },
+  {
+   "n": 100,
+   "rep": 4,
+   "argmax": 0.6505158537226495,
+   "argmax_A": 0.6622850592475571,
+   "argmax_B": 0.6381829431003364,
+   "rule": 0.6745643053435189,
+   "rule_A": 0.6876049053526646,
+   "rule_B": 0.661204283542899,
+   "rule_smooth": 0.6975789273734255,
+   "rule_smooth_A": 0.7094564679774931,
+   "rule_smooth_B": 0.68518924399675
+  },
+  {
+   "n": 200,
+   "rep": 0,
+   "argmax": 0.6874705957253613,
+   "argmax_A": 0.6952692030164485,
+   "argmax_B": 0.678477001640029,
+   "rule": 0.700314491893222,
+   "rule_A": 0.7069740934977541,
+   "rule_B": 0.692428504509652,
+   "rule_smooth": 0.7182306216783116,
+   "rule_smooth_A": 0.721852644338021,
+   "rule_smooth_B": 0.713075992015587
+  },
+  {
+   "n": 200,
+   "rep": 1,
+   "argmax": 0.6708163929054928,
+   "argmax_A": 0.6761830154815136,
+   "argmax_B": 0.6643237632796017,
+   "rule": 0.6926737103739056,
+   "rule_A": 0.700229860538487,
+   "rule_B": 0.6841925574569627,
+   "rule_smooth": 0.7137431751951573,
+   "rule_smooth_A": 0.7213892257800305,
+   "rule_smooth_B": 0.7050933321535093
+  },
+  {
+   "n": 200,
+   "rep": 2,
+   "argmax": 0.6830746017014666,
+   "argmax_A": 0.6892760935454384,
+   "argmax_B": 0.6758234480785745,
+   "rule": 0.6930544921464066,
+   "rule_A": 0.6986559675322612,
+   "rule_B": 0.6863539336602484,
+   "rule_smooth": 0.7105746218670094,
+   "rule_smooth_A": 0.7148004150525684,
+   "rule_smooth_B": 0.705057482202303
+  },
+  {
+   "n": 200,
+   "rep": 3,
+   "argmax": 0.6726842885133946,
+   "argmax_A": 0.6791018993980443,
+   "argmax_B": 0.6650456032460721,
+   "rule": 0.6933965815538178,
+   "rule_A": 0.7010058123627396,
+   "rule_B": 0.6851871831953917,
+   "rule_smooth": 0.7102407225919019,
+   "rule_smooth_A": 0.7180422706056011,
+   "rule_smooth_B": 0.7013812594652382
+  },
+  {
+   "n": 200,
+   "rep": 4,
+   "argmax": 0.673781538899325,
+   "argmax_A": 0.6804226793522838,
+   "argmax_B": 0.6660142098229029,
+   "rule": 0.6930345724918103,
+   "rule_A": 0.6998287257964013,
+   "rule_B": 0.6854306846423124,
+   "rule_smooth": 0.7137281485560256,
+   "rule_smooth_A": 0.7222105006966087,
+   "rule_smooth_B": 0.7042953299585348
+  }
+ ],
+ "n_draws": 25,
+ "mean": {
+  "argmax": {
+   "5": 0.5522019632869162,
+   "25": 0.5876454951612403,
+   "50": 0.6339729020211727,
+   "100": 0.6597507137647999,
+   "200": 0.677565483549008
+  },
+  "argmax_A": {
+   "5": 0.5548740299541388,
+   "25": 0.5930701443404341,
+   "50": 0.6402395772374844,
+   "100": 0.6692148305390213,
+   "200": 0.6840505781587457
+  },
+  "argmax_B": {
+   "5": 0.5484291840949954,
+   "25": 0.5811872541516582,
+   "50": 0.6268219669433621,
+   "100": 0.6494907662025086,
+   "200": 0.669936805213436
+  },
+  "rule": {
+   "5": 0.5639279659381311,
+   "25": 0.6075924201880035,
+   "50": 0.6575884781873389,
+   "100": 0.6785931439726444,
+   "200": 0.6944947696918324
+  },
+  "rule_A": {
+   "5": 0.569089647776282,
+   "25": 0.616115858619621,
+   "50": 0.6651324407930003,
+   "100": 0.688766749638446,
+   "200": 0.7013388919455286
+  },
+  "rule_B": {
+   "5": 0.5579873039352452,
+   "25": 0.5981793335276019,
+   "50": 0.6493341884525149,
+   "100": 0.6678345991770539,
+   "200": 0.6867185726929135
+  },
+  "rule_smooth": {
+   "5": 0.5873404367304655,
+   "25": 0.6287322036647722,
+   "50": 0.6809839756236095,
+   "100": 0.7009323324326207,
+   "200": 0.7133034579776811
+  },
+  "rule_smooth_A": {
+   "5": 0.5943768799597489,
+   "25": 0.6366344048924286,
+   "50": 0.6885054815248385,
+   "100": 0.7095984243462986,
+   "200": 0.719659011294566
+  },
+  "rule_smooth_B": {
+   "5": 0.5794589268094222,
+   "25": 0.6196158811308019,
+   "50": 0.672444465705079,
+   "100": 0.6913484008178752,
+   "200": 0.7057806791590344
+  }
+ },
+ "std": {
+  "argmax": {
+   "5": 0.028138995154956454,
+   "25": 0.023959137596325984,
+   "50": 0.009235323162630347,
+   "100": 0.007075505381239022,
+   "200": 0.007282790616340668
+  },
+  "argmax_A": {
+   "5": 0.034314321268040654,
+   "25": 0.022047814785840857,
+   "50": 0.011361438468179027,
+   "100": 0.005191140203147171,
+   "200": 0.007948477345711377
+  },
+  "argmax_B": {
+   "5": 0.023617672522626313,
+   "25": 0.025741733471024625,
+   "50": 0.007941333639028467,
+   "100": 0.009090836496560126,
+   "200": 0.006678404941691481
+  },
+  "rule": {
+   "5": 0.022627052464369145,
+   "25": 0.024222425678149726,
+   "50": 0.006693965752008388,
+   "100": 0.0035921758812472093,
+   "200": 0.003263357779881097
+  },
+  "rule_A": {
+   "5": 0.028342367453343058,
+   "25": 0.021608588103887806,
+   "50": 0.008600658558788792,
+   "100": 0.00261421210130059,
+   "200": 0.0032624789045201594
+  },
+  "rule_B": {
+   "5": 0.01848118121254038,
+   "25": 0.02616884724626128,
+   "50": 0.006211402711577744,
+   "100": 0.005237427325063028,
+   "200": 0.003283323216354152
+  },
+  "rule_smooth": {
+   "5": 0.025054505886625767,
+   "25": 0.022790212394586982,
+   "50": 0.00814094942485961,
+   "100": 0.004660605138689975,
+   "200": 0.0032201533353902905
+  },
+  "rule_smooth_A": {
+   "5": 0.029748943154597553,
+   "25": 0.01893889056213141,
+   "50": 0.009315651444985855,
+   "100": 0.0031237183625422325,
+   "200": 0.0031833776936570153
+  },
+  "rule_smooth_B": {
+   "5": 0.021471624783603527,
+   "25": 0.025813160430820502,
+   "50": 0.00796835535639377,
+   "100": 0.007739903856814391,
+   "200": 0.00435253389730617
+  }
+ }
+}

reports/nnlab/scores/cls_lr_feda.json

@@ -0,0 +1,461 @@
+{
+ "method": "cls_lr_feda",
+ "draws": [
+  {
+   "n": 5,
+   "rep": 0,
+   "argmax": 0.5145038577960739,
+   "argmax_A": 0.5302408733589656,
+   "argmax_B": 0.49912819057146296,
+   "rule": 0.5277217270028166,
+   "rule_A": 0.5449585660028473,
+   "rule_B": 0.5109259294371515,
+   "rule_smooth": 0.5513895343089443,
+   "rule_smooth_A": 0.5708410580930072,
+   "rule_smooth_B": 0.533000352199926
+  },
+  {
+   "n": 5,
+   "rep": 1,
+   "argmax": 0.5979128529196888,
+   "argmax_A": 0.61036216050473,
+   "argmax_B": 0.5855505155072638,
+   "rule": 0.5940425189736874,
+   "rule_A": 0.6057286440758128,
+   "rule_B": 0.5824096030076114,
+   "rule_smooth": 0.6189180625772526,
+   "rule_smooth_A": 0.6288850190464694,
+   "rule_smooth_B": 0.6087880314515528
+  },
+  {
+   "n": 5,
+   "rep": 2,
+   "argmax": 0.4955319676651565,
+   "argmax_A": 0.4945047272819032,
+   "argmax_B": 0.4953565001482236,
+   "rule": 0.4934290283186852,
+   "rule_A": 0.49037963638425147,
+   "rule_B": 0.49517952574498086,
+   "rule_smooth": 0.5119325323213797,
+   "rule_smooth_A": 0.5115980270682831,
+   "rule_smooth_B": 0.5113204328446409
+  },
+  {
+   "n": 5,
+   "rep": 3,
+   "argmax": 0.5253569127113353,
+   "argmax_A": 0.5295082512981527,
+   "argmax_B": 0.5206559802295758,
+   "rule": 0.5263234525948937,
+   "rule_A": 0.5335866095889413,
+   "rule_B": 0.5187912183807873,
+   "rule_smooth": 0.5479114190264307,
+   "rule_smooth_A": 0.5574663276146073,
+   "rule_smooth_B": 0.5381017695018128
+  },
+  {
+   "n": 5,
+   "rep": 4,
+   "argmax": 0.555786848066772,
+   "argmax_A": 0.56077449831088,
+   "argmax_B": 0.5490813979239071,
+   "rule": 0.5535747871313272,
+   "rule_A": 0.5598794724380488,
+   "rule_B": 0.5456263885930133,
+   "rule_smooth": 0.582756851764942,
+   "rule_smooth_A": 0.5914750680134366,
+   "rule_smooth_B": 0.5722389300727813
+  },
+  {
+   "n": 25,
+   "rep": 0,
+   "argmax": 0.5872555096084833,
+   "argmax_A": 0.5957876185743578,
+   "argmax_B": 0.5773241048812977,
+   "rule": 0.5878717050861932,
+   "rule_A": 0.5964495112132427,
+   "rule_B": 0.5778076710400573,
+   "rule_smooth": 0.607863219362236,
+   "rule_smooth_A": 0.6184657304209408,
+   "rule_smooth_B": 0.5956636636179656
+  },
+  {
+   "n": 25,
+   "rep": 1,
+   "argmax": 0.5821211142948611,
+   "argmax_A": 0.5874198151627479,
+   "argmax_B": 0.5750880321321824,
+   "rule": 0.5857284207873188,
+   "rule_A": 0.592299682100855,
+   "rule_B": 0.577573857621064,
+   "rule_smooth": 0.6099657775795859,
+   "rule_smooth_A": 0.619379175330373,
+   "rule_smooth_B": 0.5987067626544823
+  },
+  {
+   "n": 25,
+   "rep": 2,
+   "argmax": 0.5887242433021331,
+   "argmax_A": 0.6092295385767765,
+   "argmax_B": 0.5689615376486392,
+   "rule": 0.5882272122853967,
+   "rule_A": 0.6093184412358688,
+   "rule_B": 0.567947715420027,
+   "rule_smooth": 0.6087764736592991,
+   "rule_smooth_A": 0.6308453649689152,
+   "rule_smooth_B": 0.5875855425017751
+  },
+  {
+   "n": 25,
+   "rep": 3,
+   "argmax": 0.6408101385286561,
+   "argmax_A": 0.6513210971042436,
+   "argmax_B": 0.6289129850186452,
+   "rule": 0.6418441781222247,
+   "rule_A": 0.6522637856339402,
+   "rule_B": 0.6301339272724791,
+   "rule_smooth": 0.6592783286465385,
+   "rule_smooth_A": 0.6694419120800847,
+   "rule_smooth_B": 0.6476308076883872
+  },
+  {
+   "n": 25,
+   "rep": 4,
+   "argmax": 0.5963773810225643,
+   "argmax_A": 0.6088619073564523,
+   "argmax_B": 0.5848529770350098,
+   "rule": 0.5955628268928652,
+   "rule_A": 0.6084448268297984,
+   "rule_B": 0.5837015645172857,
+   "rule_smooth": 0.6213426744606242,
+   "rule_smooth_A": 0.6327185473647529,
+   "rule_smooth_B": 0.6109252367970788
+  },
+  {
+   "n": 50,
+   "rep": 0,
+   "argmax": 0.6535362468537785,
+   "argmax_A": 0.6665818642117705,
+   "argmax_B": 0.6397803796426766,
+   "rule": 0.6506782824615732,
+   "rule_A": 0.6637224947009204,
+   "rule_B": 0.6369949769568585,
+   "rule_smooth": 0.6725904652533936,
+   "rule_smooth_A": 0.685594718448566,
+   "rule_smooth_B": 0.6587709476985932
+  },
+  {
+   "n": 50,
+   "rep": 1,
+   "argmax": 0.6362610891439757,
+   "argmax_A": 0.6437012281055667,
+   "argmax_B": 0.6277302493348889,
+   "rule": 0.6354944192828049,
+   "rule_A": 0.6432729319074433,
+   "rule_B": 0.6265859051669088,
+   "rule_smooth": 0.6599309454017108,
+   "rule_smooth_A": 0.6693582162874185,
+   "rule_smooth_B": 0.6495301395200126
+  },
+  {
+   "n": 50,
+   "rep": 2,
+   "argmax": 0.6361799262828902,
+   "argmax_A": 0.6419899659938835,
+   "argmax_B": 0.6291460453646046,
+   "rule": 0.6364944704335819,
+   "rule_A": 0.641784906696576,
+   "rule_B": 0.6299479174792358,
+   "rule_smooth": 0.6570527821030432,
+   "rule_smooth_A": 0.6615389316039932,
+   "rule_smooth_B": 0.651021429770176
+  },
+  {
+   "n": 50,
+   "rep": 3,
+   "argmax": 0.6457670005919303,
+   "argmax_A": 0.6663005813806925,
+   "argmax_B": 0.625481169889025,
+   "rule": 0.6450673087484309,
+   "rule_A": 0.6648947980685819,
+   "rule_B": 0.6254394518777662,
+   "rule_smooth": 0.6617721597738118,
+   "rule_smooth_A": 0.6818675266073972,
+   "rule_smooth_B": 0.6420745912282821
+  },
+  {
+   "n": 50,
+   "rep": 4,
+   "argmax": 0.6281436775800802,
+   "argmax_A": 0.6370142504222032,
+   "argmax_B": 0.6186923038383624,
+   "rule": 0.6268445133658986,
+   "rule_A": 0.6349455426704078,
+   "rule_B": 0.6181511327706818,
+   "rule_smooth": 0.6478394996953276,
+   "rule_smooth_A": 0.655817188771744,
+   "rule_smooth_B": 0.6393771026207976
+  },
+  {
+   "n": 100,
+   "rep": 0,
+   "argmax": 0.646982710544322,
+   "argmax_A": 0.6675386889650547,
+   "argmax_B": 0.6271486939009124,
+   "rule": 0.6457176642346906,
+   "rule_A": 0.6658248450866742,
+   "rule_B": 0.6263261668211777,
+   "rule_smooth": 0.6671516440663906,
+   "rule_smooth_A": 0.6863033410407259,
+   "rule_smooth_B": 0.6485897942950761
+  },
+  {
+   "n": 100,
+   "rep": 1,
+   "argmax": 0.6621137073036597,
+   "argmax_A": 0.6701028371153472,
+   "argmax_B": 0.6528273723355686,
+   "rule": 0.6600054792879323,
+   "rule_A": 0.6675694261665159,
+   "rule_B": 0.6510477695856409,
+   "rule_smooth": 0.6806684769013661,
+   "rule_smooth_A": 0.6859891166892582,
+   "rule_smooth_B": 0.6737313727638409
+  },
+  {
+   "n": 100,
+   "rep": 2,
+   "argmax": 0.6316185166695627,
+   "argmax_A": 0.6530145539484408,
+   "argmax_B": 0.6106966729600245,
+   "rule": 0.6298213447221523,
+   "rule_A": 0.6505543036642532,
+   "rule_B": 0.6094847753621256,
+   "rule_smooth": 0.6539145026016212,
+   "rule_smooth_A": 0.6736756643175837,
+   "rule_smooth_B": 0.6346606581745455
+  },
+  {
+   "n": 100,
+   "rep": 3,
+   "argmax": 0.6491832474245705,
+   "argmax_A": 0.6604707690352074,
+   "argmax_B": 0.6367206385745094,
+   "rule": 0.6471083203410289,
+   "rule_A": 0.6588099670154377,
+   "rule_B": 0.6342565078573374,
+   "rule_smooth": 0.6680478268235135,
+   "rule_smooth_A": 0.6799691940970933,
+   "rule_smooth_B": 0.6551015524128057
+  },
+  {
+   "n": 100,
+   "rep": 4,
+   "argmax": 0.6452137479504902,
+   "argmax_A": 0.656306195930115,
+   "argmax_B": 0.6333515765559375,
+   "rule": 0.6448766461947766,
+   "rule_A": 0.6554522275734113,
+   "rule_B": 0.6335085032422371,
+   "rule_smooth": 0.670712952812864,
+   "rule_smooth_A": 0.6811801607789157,
+   "rule_smooth_B": 0.6594261191868775
+  },
+  {
+   "n": 200,
+   "rep": 0,
+   "argmax": 0.6686050407272106,
+   "argmax_A": 0.6771559669687843,
+   "argmax_B": 0.6589660775509681,
+   "rule": 0.6652004577543693,
+   "rule_A": 0.6737770788972108,
+   "rule_B": 0.6554905375614811,
+   "rule_smooth": 0.6881476466345521,
+   "rule_smooth_A": 0.6972831812896176,
+   "rule_smooth_B": 0.6779682978309558
+  },
+  {
+   "n": 200,
+   "rep": 1,
+   "argmax": 0.6410038865383495,
+   "argmax_A": 0.6527921762335811,
+   "argmax_B": 0.6282427545394503,
+   "rule": 0.6438601207774043,
+   "rule_A": 0.656165779877525,
+   "rule_B": 0.6306289437282911,
+   "rule_smooth": 0.6726383730521055,
+   "rule_smooth_A": 0.685752847366896,
+   "rule_smooth_B": 0.6585484719868016
+  },
+  {
+   "n": 200,
+   "rep": 2,
+   "argmax": 0.6630382575036267,
+   "argmax_A": 0.671662484397811,
+   "argmax_B": 0.6529301426884337,
+   "rule": 0.6626144974838902,
+   "rule_A": 0.6714513951812946,
+   "rule_B": 0.6523619894443797,
+   "rule_smooth": 0.6895117596708605,
+   "rule_smooth_A": 0.6991687449462152,
+   "rule_smooth_B": 0.6781394756564174
+  },
+  {
+   "n": 200,
+   "rep": 3,
+   "argmax": 0.6611598491845653,
+   "argmax_A": 0.6718293940827454,
+   "argmax_B": 0.6491091594009364,
+   "rule": 0.6636885493776585,
+   "rule_A": 0.6756899949499215,
+   "rule_B": 0.6503423549612679,
+   "rule_smooth": 0.6888259444300848,
+   "rule_smooth_A": 0.6981322818185987,
+   "rule_smooth_B": 0.6779062217008505
+  },
+  {
+   "n": 200,
+   "rep": 4,
+   "argmax": 0.671358028747221,
+   "argmax_A": 0.6805395653344961,
+   "argmax_B": 0.6611179661246304,
+   "rule": 0.669309339938867,
+   "rule_A": 0.6759470071163292,
+   "rule_B": 0.6615885258126528,
+   "rule_smooth": 0.6930647018173787,
+   "rule_smooth_A": 0.7016918816778661,
+   "rule_smooth_B": 0.6833072249993696
+  }
+ ],
+ "n_draws": 25,
+ "mean": {
+  "argmax": {
+   "5": 0.5378184878318053,
+   "25": 0.5990576773513396,
+   "50": 0.639977588090531,
+   "100": 0.647022385978521,
+   "200": 0.6610330125401946
+  },
+  "argmax_A": {
+   "5": 0.5450781021509263,
+   "25": 0.6105239953549156,
+   "50": 0.6511175780228233,
+   "100": 0.661486608998833,
+   "200": 0.6707959174034837
+  },
+  "argmax_B": {
+   "5": 0.5299545168760866,
+   "25": 0.5870279273431549,
+   "50": 0.6281660296139115,
+   "100": 0.6321489908653906,
+   "200": 0.6500732200608839
+  },
+  "rule": {
+   "5": 0.539018302804282,
+   "25": 0.5998468686347997,
+   "50": 0.6389157988584578,
+   "100": 0.6455058909561162,
+   "200": 0.6609345930664379
+  },
+  "rule_A": {
+   "5": 0.5469065856979803,
+   "25": 0.611755249402741,
+   "50": 0.6497241348087859,
+   "100": 0.6596421539012585,
+   "200": 0.6706062512044563
+  },
+  "rule_B": {
+   "5": 0.5305865330327089,
+   "25": 0.5874329471741826,
+   "50": 0.6274238768502902,
+   "100": 0.6309247445737037,
+   "200": 0.6500824703016146
+  },
+  "rule_smooth": {
+   "5": 0.5625816799997898,
+   "25": 0.6214452947416567,
+   "50": 0.6598371704454574,
+   "100": 0.6680990806411511,
+   "200": 0.6864376851209963
+  },
+  "rule_smooth_A": {
+   "5": 0.5720530999671608,
+   "25": 0.6341701460330134,
+   "50": 0.6708353163438238,
+   "100": 0.6814234953847154,
+   "200": 0.6964057874198387
+  },
+  "rule_smooth_B": {
+   "5": 0.5526899032141428,
+   "25": 0.6081024026519378,
+   "50": 0.6481548421675722,
+   "100": 0.6543018993666292,
+   "200": 0.6751739384348789
+  }
+ },
+ "std": {
+  "argmax": {
+   "5": 0.04006621800482266,
+   "25": 0.02389231149145717,
+   "50": 0.009818739254148976,
+   "100": 0.01087392795717806,
+   "200": 0.01192837534867843
+  },
+  "argmax_A": {
+   "5": 0.043383795900653964,
+   "25": 0.024595759842810528,
+   "50": 0.01420287941893458,
+   "100": 0.0072545052561231085,
+   "200": 0.010739443816973093
+  },
+  "argmax_B": {
+   "5": 0.03771770334048208,
+   "25": 0.024094647589954495,
+   "50": 0.007632224671196612,
+   "100": 0.015290082967331007,
+   "200": 0.013102766641915325
+  },
+  "rule": {
+   "5": 0.03743898167625931,
+   "25": 0.023769089923392772,
+   "50": 0.009212514551205248,
+   "100": 0.010715078501605923,
+   "200": 0.009877726773882059
+  },
+  "rule_A": {
+   "5": 0.04184746971195107,
+   "25": 0.023825528114879434,
+   "50": 0.013685386831364775,
+   "100": 0.007104652620601777,
+   "200": 0.008271392762443494
+  },
+  "rule_B": {
+   "5": 0.034245995325917884,
+   "25": 0.024529669595588877,
+   "50": 0.0068665977770319,
+   "100": 0.01502940275099018,
+   "200": 0.011676257412029434
+  },
+  "rule_smooth": {
+   "5": 0.040269847281833,
+   "25": 0.0218409025618983,
+   "50": 0.0089176289978235,
+   "100": 0.009577637590409397,
+   "200": 0.00794380338868258
+  },
+  "rule_smooth_A": {
+   "5": 0.04323233362139438,
+   "25": 0.02075247137053974,
+   "50": 0.012783762781043909,
+   "100": 0.005168676855186149,
+   "200": 0.006180866878532831
+  },
+  "rule_smooth_B": {
+   "5": 0.0382135106535111,
+   "25": 0.023634746812029504,
+   "50": 0.007694618687312529,
+   "100": 0.014342315103980841,
+   "200": 0.009573723312386751
+  }
+ }
+}