Publish selected full checkpoint and concise inference model card

NOTICE

@@ -1,28 +1,8 @@
-SPECTRA merged weights -- OpenMidnight
-=====================================================================
+OpenMidnight-1.0-SPECTRA
 
-This repository contains FULL MODEL WEIGHTS that are a derivative work of:
+Base model: https://huggingface.co/SophontAI/OpenMidnight
+Base authors: MedARC and Sophont. Original authors retain their rights in the base weights.
 
-    SophontAI/OpenMidnight
-    pinned revision: 87189e66
-    upstream licence: Apache License, Version 2.0
+MedARC and Sophont adapted the base encoder with SPECTRA on registered PLISM tiles and merged the learned rank-32 LoRA updates into the full weights in float32.
 
-Original authors / copyright holders of the base model retain their rights in the
-base weights. The SPECTRA contribution is a rank-32 LoRA delta (alpha 64, scaling
-2.0) trained on registered PLISM tiles and folded into those base weights; it is a
-modification of the original work within the meaning of Apache-2.0 Section 4.
-
-Modifications made:
-  - attn.qkv, attn.proj, mlp.fc1, mlp.fc2 (4 per block x 40 blocks = 160 modules)
-    had W replaced by W + 2.0 * (B @ A) from the SPECTRA adapter for the given seed.
-  - No other tensor was changed. All other weights are byte-identical to the base
-    checkpoint at the pinned revision.
-  - Weights are stored in fp32 as `model.safetensors`; `config.json` is a verbatim
-    copy of the base checkpoint's config.
-
-The `LICENSE` file in this repository is the Apache License, Version 2.0, carried
-forward from the base model.
-
-NOTE: the base repository is GATED on the Hugging Face Hub. Redistribution of
-derived full weights has not yet been cleared with the upstream authors. See
-PENDING_LICENCE_DECISION.md.
+The full model weights are distributed under the Apache License, Version 2.0. See LICENSE.

README.md

@@ -1,242 +1,60 @@
 ---
+language: en
+license: apache-2.0
 library_name: timm
+pipeline_tag: image-feature-extraction
 base_model: SophontAI/OpenMidnight
-license: apache-2.0
-pipeline_tag: feature-extraction
+base_model_relation: finetune
 tags:
 - pathology
 - histopathology
 - feature-extraction
-- computational-pathology
-- vision-transformer
+- spectra
 ---
+# OpenMidnight-1.0-SPECTRA
 
-# SPECTRA (merged) - OpenMidnight
-
-Full-weight release of `SophontAI/OpenMidnight` made robust to changes in slide acquisition --
-scanner, stain, centre. The SPECTRA rank-32 LoRA delta has already been **folded into
-the base weights**, so this repository is a drop-in replacement for the base model: you
-build the architecture and `load_state_dict`, and you do not need `peft` installed.
-
-Training was contrastive on registered PLISM tiles -- the same physical tissue location
-imaged under many scanner/stain conditions -- with an objective that pulls matched
-conditions of one tile together while pushing different tiles apart.
-
-If you would rather keep the base weights untouched and apply a small delta at load
-time, use the adapter-only release instead.
-
-> **Base model.** These are `SophontAI/OpenMidnight` weights (Apache-2.0) with a SPECTRA
-> LoRA adapter merged in, released here by the same organisation. Apache-2.0 terms and the
-> attribution in `NOTICE` apply.
-
-## Base model
-
-| | |
-|---|---|
-| Base repository | [`SophontAI/OpenMidnight`](https://huggingface.co/SophontAI/OpenMidnight) |
-| Pinned revision | `87189e66` |
-| Loader | `timm` |
-| Merged modules | attn.qkv, attn.proj, mlp.fc1, mlp.fc2 (4 per block x 40 blocks = 160 modules) |
-| LoRA rank / alpha / scaling | 32 / 64 / 2.0 |
-| Weight dtype | fp32 |
-| Embedding dimension (this readout) | **1536** |
-
-The delta was trained and evaluated against **that exact base revision**. The merge is
-`W <- W + (alpha/r) * (B @ A)`, computed in fp32 before any device move or dtype cast.
-
-## Seeds
-
-Three independent training seeds are shipped as subfolders. They are **not** an
-ensemble -- pick one, and report spread across all three.
-
-| Folder | Original training seed | Selected step | Training run |
-|---|---|---|---|
-| `seed0/` | s0 | 150 | `genMASK-c50-ms500-openmidnight-s0-t900-438670` |
-| `seed1/` | s1 | 150 | `genMASK-c50-ms500-openmidnight-s1-t900-438671` |
-| `seed2/` | s2 | 150 | `genMASK-c50-ms500-openmidnight-s2-t900-438672` |
-
-Folder names are positional (`seed0/1/2`); the "original training seed" column records
-the seed label the run actually used, so provenance is not lost.
+[MedARC](https://www.medarc.ai/) · [Sophont](https://sophont.med/) · [Paper (coming soon)](#paper)
 
-## Usage
+OpenMidnight-1.0-SPECTRA is [OpenMidnight](https://huggingface.co/SophontAI/OpenMidnight) adapted with **SPECTRA** by MedARC and Sophont. We trained on registered PLISM image tiles, aligning representations of the same tissue across scanners and stains with a contrastive objective. The learned updates are merged into the full model weights.
 
-```python
-import json, timm, torch
-import safetensors.torch as st
-from huggingface_hub import snapshot_download
+**Legacy OpenMidnight 1.0 release.** For the current model, use [OpenMidnight-1.1-SPECTRA](https://huggingface.co/medarc/OpenMidnight-1.1-SPECTRA).
 
-d = snapshot_download("medarc/spectra-openmidnight-merged", allow_patterns=["seed0/*"])
-cfg = json.load(open(f"{d}/seed0/config.json"))
+The model maps RGB histology tiles to **1,536-dimensional embeddings** for downstream analysis. All inference weights are included; no separate base checkpoint, adapter, or cluster access is required.
 
-model = timm.create_model(
-    cfg["architecture"],            # "vit_giant_patch14_reg4_dinov2"
-    pretrained=False,
-    num_classes=0, global_pool="token",
-    img_size=224, init_values=1e-5, dynamic_img_size=False,
-)
-model.load_state_dict(st.load_file(f"{d}/seed0/model.safetensors"), strict=True)
-model = model.float().eval().cuda()
-```
+## Inference
 
-Preprocessing -- this must match exactly:
+In a Python environment:
 
-```python
-import torchvision.transforms as T
-
-# DEFAULT transform -- used for HEST, PathoROB and CPTAC in the paper.
-tf = T.Compose([
-    T.Resize(256, interpolation=T.InterpolationMode.BICUBIC),   # crop_pct 0.875
-    T.CenterCrop(224),
-    T.ToTensor(),
-    T.Normalize(mean=(0.485, 0.456, 0.406), std=(0.229, 0.224, 0.225)),
-])
-
-# THUNDER-ONLY transform -- squash resize, no crop. See warning 1.
-tf_thunder = T.Compose([
-    T.Resize((224, 224)),
-    T.ToTensor(),
-    T.Normalize(mean=(0.485, 0.456, 0.406), std=(0.229, 0.224, 0.225)),
-])
+```bash
+uv pip install torch torchvision "timm==1.0.30" huggingface_hub safetensors pillow
 ```
 
-Forward pass and readout:
-
 ```python
+import torch
+import timm
 from PIL import Image
+from timm.data import create_transform, resolve_model_data_config
 
-img = Image.open("tile.png").convert("RGB")
-x = tf(img).unsqueeze(0).cuda()
-with torch.inference_mode():
-    h = model.forward_features(x)     # (B, 5 + 256, 1536)
-feat = h[:, 0]                        # (B, 1536) -- CLS token alone
-```
-
-**Readout:** CLS token alone. `h[:, 0]`
-
-**Embedding dimension:** 1536
-
-**Token layout:** `num_prefix_tokens = 5` (1 CLS + 4 register tokens), then 256 spatial tokens. Patch tokens begin at index **5**, not 1.
-
-## WARNINGS
-
-### 1. There are TWO transforms, and which one you want depends on the benchmark
-
-This is the one genuinely unusual inference detail in this release.
-
-- **Default (HEST, PathoROB, CPTAC and general use):** `Resize(256, bicubic)` ->
-  `CenterCrop(224)`, from the checkpoint's own `pretrained_cfg` under timm's defaults
-  (it carries no `crop_pct` and no `interpolation`).
-- **THUNDER only:** `Resize((224, 224))` -- a **squash** resize with no crop. THUNDER's
-  `get_openmidnight` hand-builds this transform.
-
-Using the generic resize+crop path on THUNDER puts the base model roughly **6.2 F1
-below** the published leaderboard on segmentation, and about a point below on kNN and
-linear probing. Match the transform to the benchmark you are reproducing.
-
-### 2. Normalisation is ImageNet, unlike `kaiko-ai/midnight`
-
-OpenMidnight replicates Midnight's recipe but uses **ImageNet** statistics
-(`mean = (0.485, 0.456, 0.406)`, `std = (0.229, 0.224, 0.225)`), whereas
-`kaiko-ai/midnight` demands `(0.5, 0.5, 0.5)` / `(0.5, 0.5, 0.5)`. The two are not
-interchangeable. The original training checkpoint stores no normalisation at all; these
-statistics come from SophontAI's own model card.
-
-### 3. Readout is CLS alone, and four register tokens sit in front of the patches
-
-`num_prefix_tokens = 5`. Patch/dense tokens begin at index **5**.
-
-### Un-annealed checkpoints
-
-The 1-SE selection rule picked steps inside the 200-step warmup, so these weights come
-from a **mid-warmup, un-annealed** point of the schedule -- not from the end of a
-completed LR decay. That is what the selection rule chose and what the paper reports,
-but it is unusual enough to state plainly.
-
-## Results
-
-Base model versus these merged weights. Values are read from the SPECTRA paper's
-tables. n = 3 seeds; the interval is **mean +/- 2SD** across the three seeds.
-
-| Metric | Base model | SPECTRA (merged, n=3 seeds, mean +/- 2SD) |
-|---|---|---|
-| PathoROB mean robustness index (cross-centre) | 0.618 | **0.879** +/- 0.007 |
-| PLISM top-1 retrieval across scanners | 0.703 | **0.948** +/- 0.015 |
-| PLISM top-1 retrieval across stains | 0.486 | **0.782** +/- 0.029 |
-| HEST mean Pearson r | 0.3902 | **0.4048** +/- 0.0014 |
-| CPTAC AUC | 0.6561 | **0.6844** +/- 0.0022 |
-
-## Training
-
-| | |
-|---|---|
-| Base-model weights | **modified** -- the LoRA delta has been folded in |
-| Method | LoRA (PEFT), r = 32, lora_alpha = 64 (scaling alpha/r = 2.0), lora_dropout = 0.0, bias = none, no DoRA, no rsLoRA |
-| Optimiser | AdamW, lr 1e-4, weight decay 0.05, grad clip 1.0 |
-| Schedule | 500 steps total, 200 warmup |
-| Objective | InfoNCE over registered PLISM tiles, split CLS / mean heads (weights 0.5 / 0.5), temperature 0.07 |
-| Batch geometry | 4 groups x 64 tiles, 2 acquisition conditions per batch, 900-tile grid sampler |
-| Same-core masking | on (`same_core_logit_bias_cls = +3.0`, `same_core_logit_bias_mean = -inf`) |
-| Checkpoint selection | 1-SE rule on the PathoROB robustness-index curve, applied per seed |
-| Precision | bf16 autocast during training; the released weights are **fp32** |
-
-Checkpoints were written every 50 steps; only the 1-SE-selected step per seed is
-released. Because selection is per seed, the three seeds of a backbone need not sit at
-the same step -- and because the selected steps fall **inside** the 200-step warmup,
-the released checkpoints are **un-annealed**. This is deliberate (the 1-SE rule picked
-them) but it means they are not the end of a completed LR schedule.
-
-Full run hyper-parameters are in each seed folder's `training_config.json`.
-
-> **`training_config.json` encoding note.** The training code writes
-> `same_core_logit_bias_mean` as negative infinity, and Python's `json` module emits the
-> bare token `-Infinity`. That token is **not valid JSON** and is rejected by
-> `JSON.parse` and most non-Python parsers. In the released files it is encoded as the
-> **string `"-Infinity"`** so the file parses everywhere. Read it back with
-> `float("-inf")`. That is the only edit made to the training configuration.
-
-## What is NOT in this repository
-
-Nothing else from the training run is needed to reproduce the published numbers. The
-contrastive projector heads (`projector*.pt`, `projector_heads.json`) and the GeM
-pooling head (`pool_head.pt`) are training-only machinery and are deliberately not
-released. In particular the GeM pooling head is vacuous at inference: with it applied
-the features are bit-identical to the plain readout below.
-
-## Citation
+device = "cuda"  # Use "cpu" to run without a GPU.
+model = timm.create_model(
+    "hf-hub:medarc/OpenMidnight-1.0-SPECTRA", pretrained=True,
+).eval().to(device)
+transform = create_transform(**resolve_model_data_config(model), is_training=False)
 
-```bibtex
-@inproceedings{spectra2026,
-  title     = {SPECTRA: cross-acquisition robustness for pathology foundation models},
-  author    = {TODO: author list},
-  year      = {2026},
-  note      = {TODO: confirm venue and year before citing -- submitted to NeurIPS 2026},
-  url       = {https://github.com/TODO-org/spectra}
-}
+image = Image.open("tile.png").convert("RGB")
+batch = transform(image).unsqueeze(0).to(device)
+with torch.inference_mode():
+    tokens = model.forward_features(batch)
+    embedding = tokens[:, 0]
+print(embedding.shape)  # torch.Size([1, 1536])
 ```
 
-Code: **SPECTRA** on GitHub (TODO: fill in the canonical repository URL before publishing).
-
-## Merge verification
-
-Every seed in this repository was verified at staging time, on CPU in fp32:
-
-| Folder | Modules merged | merged-vs-base feature diff (mean / max abs) | Reload bit-identical |
-|---|---|---|---|
-| `seed0/` | 160 | 0.1175 / 25.8296 | yes |
-| `seed1/` | 160 | 0.1322 / 26.8647 | yes |
-| `seed2/` | 160 | 0.1186 / 23.0432 | yes |
+The configuration supplies bicubic resizing to 256 pixels, a 224 × 224 center crop, and ImageNet normalization stored in `config.json`. The embedding uses the CLS token.
 
-"Reload bit-identical" means: the merged model was serialised to
-`model.safetensors`, re-instantiated from the architecture, re-loaded with
-`strict=True`, and its features on a fixed input compared with `torch.equal` against
-the in-memory merged model. All differences were exactly zero.
+These are tile embeddings. Slide-level predictions require a downstream model.
 
-The merged-vs-base column proves the merge is not a no-op: every one of the merged
-modules had a non-zero delta norm, and the resulting features differ substantially from
-the base model's on the same input.
+Released checkpoint: training seed **0**, step **150**. The full weights are stored in float32; provenance is recorded in [`provenance.json`](provenance.json).
 
-## Licence
+## Paper
 
-See `LICENSE` (and `NOTICE`, where present) in this repository. These are **merged full
-weights**, i.e. a derivative of the base model, so the base model's licence governs this
-artifact -- it is not merely the SPECTRA delta's licence.
+Coming soon. We will add the paper link and citation here.

seed1/config.json → config.json

@@ -4,7 +4,7 @@
   "num_features": 1536,
   "global_pool": "token",
   "pretrained_cfg": {
-    "custom_load": true,
+    "custom_load": false,
     "input_size": [
       3,
       224,
@@ -22,6 +22,16 @@
       0.225
     ],
     "num_classes": 0,
-    "license": "Apache 2.0"
+    "license": "Apache 2.0",
+    "interpolation": "bicubic",
+    "crop_pct": 0.875,
+    "crop_mode": "center"
+  },
+  "model_args": {
+    "img_size": 224,
+    "init_values": 1e-05,
+    "dynamic_img_size": false,
+    "num_classes": 0,
+    "global_pool": "token"
   }
-}
+}

provenance.json

@@ -0,0 +1,15 @@
+{
+  "model": "medarc/OpenMidnight-1.0-SPECTRA",
+  "weights_sha256": "b0153c417d9ccce0674a39486c1935f7a7cf637f7edc0a4c390de3a4feed631d",
+  "dtype": "float32",
+  "embedding_dimension": 1536,
+  "training_seed": 0,
+  "training_step": 150,
+  "base_model": "SophontAI/OpenMidnight",
+  "base_revision": "87189e6674d397a14a5cd342c97b1a1615a185aa",
+  "method": "SPECTRA",
+  "training_data": "registered PLISM tiles",
+  "lora_rank": 32,
+  "lora_alpha": 64,
+  "merge": "W <- W + 2 * (B @ A), float32"
+}

seed0/_verify.json

@@ -1,28 +0,0 @@
-{
-  "backbone": "openmidnightsq",
-  "hub": "SophontAI/OpenMidnight",
-  "revision": "87189e66",
-  "seed_label": "s0",
-  "run": "genMASK-c50-ms500-openmidnight-s0-t900-438670",
-  "step": "step_0000150",
-  "adapter_dir": "/admin/home/ryan.kim/waiv/runs/genMASK-c50-ms500-openmidnight-s0-t900-438670/step_0000150/adapter",
-  "out": "/admin/home/ryan.kim/spectra-release/merged/spectra-openmidnight-merged/seed0",
-  "merge_info": {
-    "merged": 160,
-    "r": 32,
-    "alpha": 64,
-    "scaling": 2.0,
-    "delta_norm_min": 0.37314584851264954,
-    "delta_norm_max": 1.7552255392074585,
-    "delta_norm_mean": 0.9496017875149846
-  },
-  "merged_vs_base": {
-    "mean_abs_diff": 0.11748708039522171,
-    "max_abs_diff": 25.829559326171875,
-    "base_feat_absmean": 0.5962615609169006,
-    "feat_dim": 1536
-  },
-  "reload_bit_identical": true,
-  "reload_max_abs_diff": 0.0,
-  "size_bytes": 4539151496
-}

seed0/config.json

@@ -1,27 +0,0 @@
-{
-  "architecture": "vit_giant_patch14_reg4_dinov2",
-  "num_classes": 0,
-  "num_features": 1536,
-  "global_pool": "token",
-  "pretrained_cfg": {
-    "custom_load": true,
-    "input_size": [
-      3,
-      224,
-      224
-    ],
-    "fixed_input_size": true,
-    "mean": [
-      0.485,
-      0.456,
-      0.406
-    ],
-    "std": [
-      0.229,
-      0.224,
-      0.225
-    ],
-    "num_classes": 0,
-    "license": "Apache 2.0"
-  }
-}

seed0/ri_curve.json

@@ -1,892 +0,0 @@
-{
-  "run_dir": "/data/ryan.kim/waiv_runs/genMASK-c50-ms500-openmidnight-s0-t900-438670",
-  "datasets": [
-    "camelyon",
-    "tolkach_esca",
-    "tcga"
-  ],
-  "targets": {
-    "phikon_v2_base": {
-      "tcga": 0.619,
-      "camelyon": 0.019,
-      "tolkach_esca": 0.768,
-      "avg": 0.469
-    },
-    "phaet_target": {
-      "tcga": 0.785,
-      "camelyon": 0.702,
-      "tolkach_esca": 0.932,
-      "avg": 0.806
-    },
-    "midnight_base": {
-      "tcga": 0.858,
-      "camelyon": 0.478,
-      "tolkach_esca": 0.941,
-      "avg": 0.759
-    },
-    "mascaret_target": {
-      "tcga": 0.893,
-      "camelyon": 0.907,
-      "tolkach_esca": 0.972,
-      "avg": 0.924
-    },
-    "virchow2_base": {
-      "avg": 0.858
-    },
-    "virchow2_target": {
-      "avg": 0.918
-    }
-  },
-  "points": [
-    {
-      "model": "waiv_genMASK_c50_ms500_openmidnight_s0_t900_438670_s0000050",
-      "adapter_rel_l2_delta": {
-        "camelyon": 0.06374021,
-        "tolkach_esca": 0.0206254,
-        "tcga": 0.03763534
-      },
-      "datasets": {
-        "camelyon": {
-          "robustness_index": 0.29756242568370983,
-          "balanced_accuracy": 0.9660294117647059,
-          "k_opt": 11,
-          "generalization_index": 0.9913733496391026,
-          "confounder_insensitivity": 0.01916613679716595,
-          "prediction_performance": 0.9569162210338681,
-          "ID_performance": 0.9570714869652103,
-          "OOD_performance": 0.9488151658767773
-        },
-        "tolkach_esca": {
-          "robustness_index": 0.8845985248432408,
-          "balanced_accuracy": 0.9554444444444444,
-          "k_opt": 46,
-          "generalization_index": 1.00991924821455,
-          "confounder_insensitivity": 0.5562681141714376,
-          "prediction_performance": 0.9350193236714976,
-          "ID_performance": 0.9317159171644345,
-          "OOD_performance": 0.9409578386122355
-        },
-        "tcga": {
-          "robustness_index": 0.8085340762578971,
-          "balanced_accuracy": 0.9244592198581559,
-          "k_opt": 61,
-          "generalization_index": 1.006949167173255,
-          "confounder_insensitivity": 0.4979636159075689,
-          "prediction_performance": 0.8959273630973144,
-          "ID_performance": 0.8938624593877003,
-          "OOD_performance": 0.9000740590478822
-        }
-      },
-      "avg_robustness_index": 0.6635650089282826,
-      "avg_balanced_accuracy": 0.9486443586891021,
-      "step": 50,
-      "checkpoint": "/data/ryan.kim/waiv_runs/genMASK-c50-ms500-openmidnight-s0-t900-438670/step_0000050",
-      "probe": {
-        "train.cross_scanner.separation": 0.0005232888460159302,
-        "train.cross_scanner.matched": 0.9998923575878144,
-        "train.cross_scanner.random": 0.9993690687417984,
-        "train.cross_scanner.top1": 0.7503125,
-        "train.cross_stain.separation": 0.0004410478803846571,
-        "train.cross_stain.matched": 0.999780216217041,
-        "train.cross_stain.random": 0.9993391683366564,
-        "train.cross_stain.top1": 0.4851215277777778,
-        "train.within_condition_random": 0.9993936657905579,
-        "heldout.cross_scanner.separation": 0.0005500659550706002,
-        "heldout.cross_scanner.matched": 0.9998943421938647,
-        "heldout.cross_scanner.random": 0.9993442762387942,
-        "heldout.cross_scanner.top1": 0.7379066780821918,
-        "heldout.cross_stain.separation": 0.0005110564984773335,
-        "heldout.cross_stain.matched": 0.9998160154498809,
-        "heldout.cross_stain.random": 0.9993049589514035,
-        "heldout.cross_stain.top1": 0.5181834795321637,
-        "heldout.within_condition_random": 0.9993553655903514
-      },
-      "eval_seconds": 3570.8,
-      "train_metrics": {
-        "step": 50,
-        "loss_cls": 4.892350196838379,
-        "top1_cls": 0.028888888657093048,
-        "weight_cls": 0.5,
-        "negatives_per_anchor": 899.0,
-        "n_rows": 1800.0,
-        "n_cond": 2.0,
-        "n_tiles": 900.0,
-        "loss_mean": 0.6314055323600769,
-        "top1_mean": 0.9111111164093018,
-        "weight_mean": 0.5,
-        "loss": 2.7618777751922607,
-        "top1": 0.4700000025331974,
-        "n_heads": 2.0,
-        "split_input_rel_distance": 1.9106868505477905,
-        "split_input_cosine": -0.12762205302715302,
-        "heldout_loss": 2.427219945192337,
-        "heldout_top1": 0.49636112414300443
-      }
-    },
-    {
-      "model": "waiv_genMASK_c50_ms500_openmidnight_s0_t900_438670_s0000100",
-      "adapter_rel_l2_delta": {
-        "camelyon": 0.1126843,
-        "tolkach_esca": 0.04074381,
-        "tcga": 0.06952679
-      },
-      "datasets": {
-        "camelyon": {
-          "robustness_index": 0.7134971842016933,
-          "balanced_accuracy": 0.9709313725490196,
-          "k_opt": 11,
-          "generalization_index": 0.9971324075061127,
-          "confounder_insensitivity": 0.09749834935074463,
-          "prediction_performance": 0.9621836007130126,
-          "ID_performance": 0.9624287775413886,
-          "OOD_performance": 0.9596689240030097
-        },
-        "tolkach_esca": {
-          "robustness_index": 0.951786159268093,
-          "balanced_accuracy": 0.9672222222222223,
-          "k_opt": 46,
-          "generalization_index": 1.009492712801384,
-          "confounder_insensitivity": 1.187571003587828,
-          "prediction_performance": 0.9476062801932368,
-          "ID_performance": 0.9427479907635891,
-          "OOD_performance": 0.9516972266839896
-        },
-        "tcga": {
-          "robustness_index": 0.8463655111487451,
-          "balanced_accuracy": 0.9303812056737591,
-          "k_opt": 61,
-          "generalization_index": 1.0132432010185413,
-          "confounder_insensitivity": 0.6466215398046015,
-          "prediction_performance": 0.8983580397628183,
-          "ID_performance": 0.8937102541485012,
-          "OOD_performance": 0.9055458386965214
-        }
-      },
-      "avg_robustness_index": 0.8372162848728438,
-      "avg_balanced_accuracy": 0.9561782668150004,
-      "step": 100,
-      "checkpoint": "/data/ryan.kim/waiv_runs/genMASK-c50-ms500-openmidnight-s0-t900-438670/step_0000100",
-      "probe": {
-        "train.cross_scanner.separation": 0.0004898476600646973,
-        "train.cross_scanner.matched": 0.9999630546569824,
-        "train.cross_scanner.random": 0.9994732069969178,
-        "train.cross_scanner.top1": 0.9084375,
-        "train.cross_stain.separation": 0.0004304936197068956,
-        "train.cross_stain.matched": 0.9998944293128119,
-        "train.cross_stain.random": 0.9994639356931051,
-        "train.cross_stain.top1": 0.6517881944444445,
-        "train.within_condition_random": 0.9994787991046905,
-        "heldout.cross_scanner.separation": 0.0005155744617932463,
-        "heldout.cross_scanner.matched": 0.9999579893399592,
-        "heldout.cross_scanner.random": 0.9994424148781659,
-        "heldout.cross_scanner.top1": 0.8894477739726028,
-        "heldout.cross_stain.separation": 0.0004799390396876642,
-        "heldout.cross_stain.matched": 0.9999059027398539,
-        "heldout.cross_stain.random": 0.9994259637001662,
-        "heldout.cross_stain.top1": 0.6676489400584795,
-        "heldout.within_condition_random": 0.9994440107810788
-      },
-      "eval_seconds": 2810.1,
-      "train_metrics": {
-        "step": 100,
-        "loss_cls": 3.64172625541687,
-        "top1_cls": 0.11166667193174362,
-        "weight_cls": 0.5,
-        "negatives_per_anchor": 899.0,
-        "n_rows": 1800.0,
-        "n_cond": 2.0,
-        "n_tiles": 900.0,
-        "loss_mean": 0.3925608992576599,
-        "top1_mean": 0.9377778172492981,
-        "weight_mean": 0.5,
-        "loss": 2.017143487930298,
-        "top1": 0.5247222445905209,
-        "n_heads": 2.0,
-        "split_input_rel_distance": 1.9103822708129883,
-        "split_input_cosine": -0.09801985323429108,
-        "heldout_loss": 1.6705534160137177,
-        "heldout_top1": 0.708847238868475
-      }
-    },
-    {
-      "model": "waiv_genMASK_c50_ms500_openmidnight_s0_t900_438670_s0000150",
-      "adapter_rel_l2_delta": {
-        "camelyon": 0.1060357,
-        "tolkach_esca": 0.03727667,
-        "tcga": 0.0692298
-      },
-      "datasets": {
-        "camelyon": {
-          "robustness_index": 0.8415839370360434,
-          "balanced_accuracy": 0.972843137254902,
-          "k_opt": 11,
-          "generalization_index": 1.0055968843391836,
-          "confounder_insensitivity": 0.20980785404671018,
-          "prediction_performance": 0.9627361853832442,
-          "ID_performance": 0.9618026352677319,
-          "OOD_performance": 0.9671857333744476
-        },
-        "tolkach_esca": {
-          "robustness_index": 0.9604878776034846,
-          "balanced_accuracy": 0.9667777777777777,
-          "k_opt": 46,
-          "generalization_index": 1.0099744602023812,
-          "confounder_insensitivity": 1.5420294482445263,
-          "prediction_performance": 0.945475845410628,
-          "ID_performance": 0.9397895150495511,
-          "OOD_performance": 0.9491634081660282
-        },
-        "tcga": {
-          "robustness_index": 0.8464677207107342,
-          "balanced_accuracy": 0.9233421985815604,
-          "k_opt": 61,
-          "generalization_index": 1.0121879701648233,
-          "confounder_insensitivity": 0.74241536023921,
-          "prediction_performance": 0.8846190849901174,
-          "ID_performance": 0.8800489035074948,
-          "OOD_performance": 0.8907749132870295
-        }
-      },
-      "avg_robustness_index": 0.8828465117834208,
-      "avg_balanced_accuracy": 0.9543210378714134,
-      "step": 150,
-      "checkpoint": "/data/ryan.kim/waiv_runs/genMASK-c50-ms500-openmidnight-s0-t900-438670/step_0000150",
-      "probe": {
-        "train.cross_scanner.separation": 0.0005586463212966919,
-        "train.cross_scanner.matched": 0.9999723851680755,
-        "train.cross_scanner.random": 0.9994137388467789,
-        "train.cross_scanner.top1": 0.9619921875,
-        "train.cross_stain.separation": 0.0004952155219184028,
-        "train.cross_stain.matched": 0.9999027323722839,
-        "train.cross_stain.random": 0.9994075168503656,
-        "train.cross_stain.top1": 0.7573784722222222,
-        "train.within_condition_random": 0.9994162178039551,
-        "heldout.cross_scanner.separation": 0.0005905391418770568,
-        "heldout.cross_scanner.matched": 0.9999654978921969,
-        "heldout.cross_scanner.random": 0.9993749587503198,
-        "heldout.cross_scanner.top1": 0.9398544520547946,
-        "heldout.cross_stain.separation": 0.0005396799734461378,
-        "heldout.cross_stain.matched": 0.9999090108955115,
-        "heldout.cross_stain.random": 0.9993693309220654,
-        "heldout.cross_stain.top1": 0.7663788377192983,
-        "heldout.within_condition_random": 0.9993788919797758
-      },
-      "eval_seconds": 2298.1,
-      "train_metrics": {
-        "step": 150,
-        "loss_cls": 1.8985670804977417,
-        "top1_cls": 0.40666666626930237,
-        "weight_cls": 0.5,
-        "negatives_per_anchor": 899.0,
-        "n_rows": 1800.0,
-        "n_cond": 2.0,
-        "n_tiles": 900.0,
-        "loss_mean": 0.10505829006433487,
-        "top1_mean": 0.9938889145851135,
-        "weight_mean": 0.5,
-        "loss": 1.0018126964569092,
-        "top1": 0.700277790427208,
-        "n_heads": 2.0,
-        "split_input_rel_distance": 1.9079663753509521,
-        "split_input_cosine": -0.0761035680770874,
-        "heldout_loss": 1.5808512270450592,
-        "heldout_top1": 0.7832222409546375
-      }
-    },
-    {
-      "model": "waiv_genMASK_c50_ms500_openmidnight_s0_t900_438670_s0000200",
-      "adapter_rel_l2_delta": {
-        "camelyon": 0.08966193,
-        "tolkach_esca": 0.02933603,
-        "tcga": 0.05624159
-      },
-      "datasets": {
-        "camelyon": {
-          "robustness_index": 0.8695477454247108,
-          "balanced_accuracy": 0.9697058823529412,
-          "k_opt": 11,
-          "generalization_index": 1.007689422740446,
-          "confounder_insensitivity": 0.31694005105783624,
-          "prediction_performance": 0.9560427807486631,
-          "ID_performance": 0.9542768273716953,
-          "OOD_performance": 0.9616146653087677
-        },
-        "tolkach_esca": {
-          "robustness_index": 0.9615664394109408,
-          "balanced_accuracy": 0.9634444444444444,
-          "k_opt": 46,
-          "generalization_index": 1.008042917084278,
-          "confounder_insensitivity": 1.7602760276027603,
-          "prediction_performance": 0.9385628019323672,
-          "ID_performance": 0.9337733773377337,
-          "OOD_performance": 0.9412836391871673
-        },
-        "tcga": {
-          "robustness_index": 0.8377346428524514,
-          "balanced_accuracy": 0.9184308510638297,
-          "k_opt": 61,
-          "generalization_index": 1.0098778458475681,
-          "confounder_insensitivity": 0.8057638634654825,
-          "prediction_performance": 0.8676530345308685,
-          "ID_performance": 0.8638454886739545,
-          "OOD_performance": 0.8723784212471929
-        }
-      },
-      "avg_robustness_index": 0.8896162758960343,
-      "avg_balanced_accuracy": 0.9505270592870718,
-      "step": 200,
-      "checkpoint": "/data/ryan.kim/waiv_runs/genMASK-c50-ms500-openmidnight-s0-t900-438670/step_0000200",
-      "probe": {
-        "train.cross_scanner.separation": 0.0005888128280639648,
-        "train.cross_scanner.matched": 0.9999768853187561,
-        "train.cross_scanner.random": 0.9993880724906922,
-        "train.cross_scanner.top1": 0.989453125,
-        "train.cross_stain.separation": 0.0005249057875739203,
-        "train.cross_stain.matched": 0.9999101400375366,
-        "train.cross_stain.random": 0.9993852342499627,
-        "train.cross_stain.top1": 0.8552604166666666,
-        "train.within_condition_random": 0.9993896424770355,
-        "heldout.cross_scanner.separation": 0.0006078375528936517,
-        "heldout.cross_scanner.matched": 0.9999700474412474,
-        "heldout.cross_scanner.random": 0.9993622098883538,
-        "heldout.cross_scanner.top1": 0.979933647260274,
-        "heldout.cross_stain.separation": 0.0005537609607852691,
-        "heldout.cross_stain.matched": 0.9999124833017762,
-        "heldout.cross_stain.random": 0.9993587223409909,
-        "heldout.cross_stain.top1": 0.8576845760233918,
-        "heldout.within_condition_random": 0.9993644429416191
-      },
-      "eval_seconds": 2351.9,
-      "train_metrics": {
-        "step": 200,
-        "loss_cls": 2.2011358737945557,
-        "top1_cls": 0.4327777922153473,
-        "weight_cls": 0.5,
-        "negatives_per_anchor": 899.0,
-        "n_rows": 1800.0,
-        "n_cond": 2.0,
-        "n_tiles": 900.0,
-        "loss_mean": 0.08813183009624481,
-        "top1_mean": 0.991111159324646,
-        "weight_mean": 0.5,
-        "loss": 1.1446338891983032,
-        "top1": 0.7119444757699966,
-        "n_heads": 2.0,
-        "split_input_rel_distance": 1.9042781591415405,
-        "split_input_cosine": -0.05710367113351822,
-        "heldout_loss": 1.2109025865793228,
-        "heldout_top1": 0.8121944665908813
-      }
-    },
-    {
-      "model": "waiv_genMASK_c50_ms500_openmidnight_s0_t900_438670_s0000250",
-      "adapter_rel_l2_delta": {
-        "camelyon": 0.08658062,
-        "tolkach_esca": 0.02974765,
-        "tcga": 0.05555091
-      },
-      "datasets": {
-        "camelyon": {
-          "robustness_index": 0.8655360387643852,
-          "balanced_accuracy": 0.9691176470588235,
-          "k_opt": 11,
-          "generalization_index": 1.0075849399214611,
-          "confounder_insensitivity": 0.3014806952830024,
-          "prediction_performance": 0.9566131907308378,
-          "ID_performance": 0.9549353609520992,
-          "OOD_performance": 0.9621784882937997
-        },
-        "tolkach_esca": {
-          "robustness_index": 0.9614416298706329,
-          "balanced_accuracy": 0.9633333333333334,
-          "k_opt": 46,
-          "generalization_index": 1.0075070664975965,
-          "confounder_insensitivity": 1.7576651768169615,
-          "prediction_performance": 0.9381570048309178,
-          "ID_performance": 0.9336894762434472,
-          "OOD_performance": 0.9406987452297128
-        },
-        "tcga": {
-          "robustness_index": 0.8391377455382278,
-          "balanced_accuracy": 0.9204609929078015,
-          "k_opt": 61,
-          "generalization_index": 1.0110562716115323,
-          "confounder_insensitivity": 0.7884517914928457,
-          "prediction_performance": 0.8716685559818625,
-          "ID_performance": 0.8674404460212874,
-          "OOD_performance": 0.8770311031993275
-        }
-      },
-      "avg_robustness_index": 0.8887051380577486,
-      "avg_balanced_accuracy": 0.9509706577666529,
-      "step": 250,
-      "checkpoint": "/data/ryan.kim/waiv_runs/genMASK-c50-ms500-openmidnight-s0-t900-438670/step_0000250",
-      "probe": {
-        "train.cross_scanner.separation": 0.0006528121232986451,
-        "train.cross_scanner.matched": 0.9999767744541168,
-        "train.cross_scanner.random": 0.9993239623308182,
-        "train.cross_scanner.top1": 0.9903515625,
-        "train.cross_stain.separation": 0.0005891905890570747,
-        "train.cross_stain.matched": 0.9999101919598049,
-        "train.cross_stain.random": 0.9993210013707479,
-        "train.cross_stain.top1": 0.86625,
-        "train.within_condition_random": 0.9993254554271698,
-        "heldout.cross_scanner.separation": 0.0006717247505710549,
-        "heldout.cross_scanner.matched": 0.9999692831953911,
-        "heldout.cross_scanner.random": 0.99929755844482,
-        "heldout.cross_scanner.top1": 0.981538955479452,
-        "heldout.cross_stain.separation": 0.0006194233197217797,
-        "heldout.cross_stain.matched": 0.999912492713036,
-        "heldout.cross_stain.random": 0.9992930693933141,
-        "heldout.cross_stain.top1": 0.8686723318713451,
-        "heldout.within_condition_random": 0.9992995465674052
-      },
-      "eval_seconds": 2471.4,
-      "train_metrics": {
-        "step": 250,
-        "loss_cls": 1.1153584718704224,
-        "top1_cls": 0.6611111164093018,
-        "weight_cls": 0.5,
-        "negatives_per_anchor": 899.0,
-        "n_rows": 1800.0,
-        "n_cond": 2.0,
-        "n_tiles": 900.0,
-        "loss_mean": 0.01960674487054348,
-        "top1_mean": 0.9994444847106934,
-        "weight_mean": 0.5,
-        "loss": 0.567482590675354,
-        "top1": 0.8302778005599976,
-        "n_heads": 2.0,
-        "split_input_rel_distance": 1.9031012058258057,
-        "split_input_cosine": -0.04557694122195244,
-        "heldout_loss": 1.0882921874523164,
-        "heldout_top1": 0.8291389077901841
-      }
-    },
-    {
-      "model": "waiv_genMASK_c50_ms500_openmidnight_s0_t900_438670_s0000300",
-      "adapter_rel_l2_delta": {
-        "camelyon": 0.07430177,
-        "tolkach_esca": 0.03844914,
-        "tcga": 0.04813877
-      },
-      "datasets": {
-        "camelyon": {
-          "robustness_index": 0.8623951001624256,
-          "balanced_accuracy": 0.9678431372549019,
-          "k_opt": 11,
-          "generalization_index": 1.007191760520451,
-          "confounder_insensitivity": 0.3102119460500963,
-          "prediction_performance": 0.9541354723707666,
-          "ID_performance": 0.9525135750569277,
-          "OOD_performance": 0.9593638245812159
-        },
-        "tolkach_esca": {
-          "robustness_index": 0.9614466537771131,
-          "balanced_accuracy": 0.9634444444444444,
-          "k_opt": 46,
-          "generalization_index": 1.007525769183128,
-          "confounder_insensitivity": 1.804023163669613,
-          "prediction_performance": 0.9365797101449276,
-          "ID_performance": 0.9320667818077145,
-          "OOD_performance": 0.9390813012708603
-        },
-        "tcga": {
-          "robustness_index": 0.835332495349946,
-          "balanced_accuracy": 0.9208687943262412,
-          "k_opt": 61,
-          "generalization_index": 1.0112616636580545,
-          "confounder_insensitivity": 0.7945378324399068,
-          "prediction_performance": 0.8675717939774445,
-          "ID_performance": 0.863267421293367,
-          "OOD_performance": 0.872989248638929
-        }
-      },
-      "avg_robustness_index": 0.8863914164298282,
-      "avg_balanced_accuracy": 0.9507187920085292,
-      "step": 300,
-      "checkpoint": "/data/ryan.kim/waiv_runs/genMASK-c50-ms500-openmidnight-s0-t900-438670/step_0000300",
-      "probe": {
-        "train.cross_scanner.separation": 0.0007215380668640137,
-        "train.cross_scanner.matched": 0.9999754893779754,
-        "train.cross_scanner.random": 0.9992539513111115,
-        "train.cross_scanner.top1": 0.99484375,
-        "train.cross_stain.separation": 0.0006525437037150065,
-        "train.cross_stain.matched": 0.9999038802252875,
-        "train.cross_stain.random": 0.9992513365215725,
-        "train.cross_stain.top1": 0.8904861111111111,
-        "train.within_condition_random": 0.9992554938793182,
-        "heldout.cross_scanner.separation": 0.0007306337356567383,
-        "heldout.cross_scanner.matched": 0.9999680004707755,
-        "heldout.cross_scanner.random": 0.9992373667351188,
-        "heldout.cross_scanner.top1": 0.9875856164383562,
-        "heldout.cross_stain.separation": 0.0006734481331897758,
-        "heldout.cross_stain.matched": 0.9999066044015494,
-        "heldout.cross_stain.random": 0.9992331562683596,
-        "heldout.cross_stain.top1": 0.8902595029239766,
-        "heldout.within_condition_random": 0.9992389286436686
-      },
-      "eval_seconds": 2429.1,
-      "train_metrics": {
-        "step": 300,
-        "loss_cls": 1.7683112621307373,
-        "top1_cls": 0.48500001430511475,
-        "weight_cls": 0.5,
-        "negatives_per_anchor": 899.0,
-        "n_rows": 1800.0,
-        "n_cond": 2.0,
-        "n_tiles": 900.0,
-        "loss_mean": 0.06796896457672119,
-        "top1_mean": 0.9927778244018555,
-        "weight_mean": 0.5,
-        "loss": 0.9181401133537292,
-        "top1": 0.7388889193534851,
-        "n_heads": 2.0,
-        "split_input_rel_distance": 1.9006847143173218,
-        "split_input_cosine": -0.041205547749996185,
-        "heldout_loss": 1.0133501410484314,
-        "heldout_top1": 0.8375555753707886
-      }
-    },
-    {
-      "model": "waiv_genMASK_c50_ms500_openmidnight_s0_t900_438670_s0000350",
-      "adapter_rel_l2_delta": {
-        "camelyon": 0.07401788,
-        "tolkach_esca": 0.03976233,
-        "tcga": 0.04607474
-      },
-      "datasets": {
-        "camelyon": {
-          "robustness_index": 0.8625927234192865,
-          "balanced_accuracy": 0.9672549019607843,
-          "k_opt": 11,
-          "generalization_index": 1.0076057464400203,
-          "confounder_insensitivity": 0.31265684318898396,
-          "prediction_performance": 0.953939393939394,
-          "ID_performance": 0.9522143772192032,
-          "OOD_performance": 0.9594566783288743
-        },
-        "tolkach_esca": {
-          "robustness_index": 0.9621462910232964,
-          "balanced_accuracy": 0.963777777777778,
-          "k_opt": 46,
-          "generalization_index": 1.0072227604027881,
-          "confounder_insensitivity": 1.779250944878189,
-          "prediction_performance": 0.9384565217391304,
-          "ID_performance": 0.9341371231396137,
-          "OOD_performance": 0.9408841717634009
-        },
-        "tcga": {
-          "robustness_index": 0.8366918027878243,
-          "balanced_accuracy": 0.9210638297872342,
-          "k_opt": 61,
-          "generalization_index": 1.0119224227122665,
-          "confounder_insensitivity": 0.7874659174037374,
-          "prediction_performance": 0.8699472445064528,
-          "ID_performance": 0.865401800558622,
-          "OOD_performance": 0.8757194866408385
-        }
-      },
-      "avg_robustness_index": 0.8871436057434691,
-      "avg_balanced_accuracy": 0.9506988365085989,
-      "step": 350,
-      "checkpoint": "/data/ryan.kim/waiv_runs/genMASK-c50-ms500-openmidnight-s0-t900-438670/step_0000350",
-      "probe": {
-        "train.cross_scanner.separation": 0.0008098453283309937,
-        "train.cross_scanner.matched": 0.9999728953838348,
-        "train.cross_scanner.random": 0.9991630500555039,
-        "train.cross_scanner.top1": 0.994140625,
-        "train.cross_stain.separation": 0.0007334121068318685,
-        "train.cross_stain.matched": 0.999893672466278,
-        "train.cross_stain.random": 0.9991602603594462,
-        "train.cross_stain.top1": 0.8898784722222223,
-        "train.within_condition_random": 0.9991646647453308,
-        "heldout.cross_scanner.separation": 0.0008189229116047898,
-        "heldout.cross_scanner.matched": 0.9999644944112595,
-        "heldout.cross_scanner.random": 0.9991455714996547,
-        "heldout.cross_scanner.top1": 0.987478595890411,
-        "heldout.cross_stain.separation": 0.0007573645017300433,
-        "heldout.cross_stain.matched": 0.9998967319901226,
-        "heldout.cross_stain.random": 0.9991393674883926,
-        "heldout.cross_stain.top1": 0.8921783625730995,
-        "heldout.within_condition_random": 0.9991464469490982
-      },
-      "eval_seconds": 2588.6,
-      "train_metrics": {
-        "step": 350,
-        "loss_cls": 0.5551905035972595,
-        "top1_cls": 0.8388888835906982,
-        "weight_cls": 0.5,
-        "negatives_per_anchor": 899.0,
-        "n_rows": 1800.0,
-        "n_cond": 2.0,
-        "n_tiles": 900.0,
-        "loss_mean": 0.011262433603405952,
-        "top1_mean": 0.9983333349227905,
-        "weight_mean": 0.5,
-        "loss": 0.28322646021842957,
-        "top1": 0.9186111092567444,
-        "n_heads": 2.0,
-        "split_input_rel_distance": 1.9005026817321777,
-        "split_input_cosine": -0.04086735472083092,
-        "heldout_loss": 0.9614806115627289,
-        "heldout_top1": 0.8447916835546494
-      }
-    },
-    {
-      "model": "waiv_genMASK_c50_ms500_openmidnight_s0_t900_438670_s0000400",
-      "adapter_rel_l2_delta": {
-        "camelyon": 0.07261994,
-        "tolkach_esca": 0.03964668,
-        "tcga": 0.04430719
-      },
-      "datasets": {
-        "camelyon": {
-          "robustness_index": 0.8651184455682726,
-          "balanced_accuracy": 0.9678431372549019,
-          "k_opt": 11,
-          "generalization_index": 1.0083015206770733,
-          "confounder_insensitivity": 0.3181003847396399,
-          "prediction_performance": 0.954282531194296,
-          "ID_performance": 0.9523745190754501,
-          "OOD_performance": 0.9602806758378728
-        },
-        "tolkach_esca": {
-          "robustness_index": 0.9625429500315866,
-          "balanced_accuracy": 0.9646666666666667,
-          "k_opt": 46,
-          "generalization_index": 1.0067634533411522,
-          "confounder_insensitivity": 1.7898702103858646,
-          "prediction_performance": 0.9385265700483092,
-          "ID_performance": 0.9344717441405986,
-          "OOD_performance": 0.9407920001807187
-        },
-        "tcga": {
-          "robustness_index": 0.8338010717520378,
-          "balanced_accuracy": 0.9184397163120568,
-          "k_opt": 61,
-          "generalization_index": 1.0118268964021844,
-          "confounder_insensitivity": 0.7906732903465565,
-          "prediction_performance": 0.8669696837577026,
-          "ID_performance": 0.8624657358850585,
-          "OOD_performance": 0.8726660287938048
-        }
-      },
-      "avg_robustness_index": 0.8871541557839656,
-      "avg_balanced_accuracy": 0.9503165067445418,
-      "step": 400,
-      "checkpoint": "/data/ryan.kim/waiv_runs/genMASK-c50-ms500-openmidnight-s0-t900-438670/step_0000400",
-      "probe": {
-        "train.cross_scanner.separation": 0.0008532857894897461,
-        "train.cross_scanner.matched": 0.9999725717306137,
-        "train.cross_scanner.random": 0.999119285941124,
-        "train.cross_scanner.top1": 0.99515625,
-        "train.cross_stain.separation": 0.0007745387819078234,
-        "train.cross_stain.matched": 0.9998913372887506,
-        "train.cross_stain.random": 0.9991167985068428,
-        "train.cross_stain.top1": 0.8952777777777777,
-        "train.within_condition_random": 0.9991208338737487,
-        "heldout.cross_scanner.separation": 0.0008624326692868586,
-        "heldout.cross_scanner.matched": 0.9999637587429726,
-        "heldout.cross_scanner.random": 0.9991013260736857,
-        "heldout.cross_scanner.top1": 0.9892444349315068,
-        "heldout.cross_stain.separation": 0.0007984906609295405,
-        "heldout.cross_stain.matched": 0.9998938646930003,
-        "heldout.cross_stain.random": 0.9990953740320707,
-        "heldout.cross_stain.top1": 0.897546600877193,
-        "heldout.within_condition_random": 0.999102218848903
-      },
-      "eval_seconds": 2661.9,
-      "train_metrics": {
-        "step": 400,
-        "loss_cls": 0.4332435727119446,
-        "top1_cls": 0.8516666889190674,
-        "weight_cls": 0.5,
-        "negatives_per_anchor": 899.0,
-        "n_rows": 1800.0,
-        "n_cond": 2.0,
-        "n_tiles": 900.0,
-        "loss_mean": 0.006968224886804819,
-        "top1_mean": 1.0,
-        "weight_mean": 0.5,
-        "loss": 0.22010590136051178,
-        "top1": 0.9258333444595337,
-        "n_heads": 2.0,
-        "split_input_rel_distance": 1.9003044366836548,
-        "split_input_cosine": -0.041082028299570084,
-        "heldout_loss": 0.9407137572765351,
-        "heldout_top1": 0.8465416833758355
-      }
-    },
-    {
-      "model": "waiv_genMASK_c50_ms500_openmidnight_s0_t900_438670_s0000450",
-      "adapter_rel_l2_delta": {
-        "camelyon": 0.07137169,
-        "tolkach_esca": 0.04297168,
-        "tcga": 0.04275866
-      },
-      "datasets": {
-        "camelyon": {
-          "robustness_index": 0.8652830372669799,
-          "balanced_accuracy": 0.9669607843137256,
-          "k_opt": 11,
-          "generalization_index": 1.008411797054637,
-          "confounder_insensitivity": 0.3287620130389213,
-          "prediction_performance": 0.9528966131907308,
-          "ID_performance": 0.9509175208285221,
-          "OOD_performance": 0.9589164460294303
-        },
-        "tolkach_esca": {
-          "robustness_index": 0.9615984427865844,
-          "balanced_accuracy": 0.9633333333333334,
-          "k_opt": 46,
-          "generalization_index": 1.0070129119890157,
-          "confounder_insensitivity": 1.82615076899972,
-          "prediction_performance": 0.9358067632850241,
-          "ID_performance": 0.9315853067465816,
-          "OOD_performance": 0.9381184325130555
-        },
-        "tcga": {
-          "robustness_index": 0.8312702113635062,
-          "balanced_accuracy": 0.9178546099290781,
-          "k_opt": 61,
-          "generalization_index": 1.0117985336222193,
-          "confounder_insensitivity": 0.799711242702601,
-          "prediction_performance": 0.8634331763748402,
-          "ID_performance": 0.8589300286664854,
-          "OOD_performance": 0.8690641434888406
-        }
-      },
-      "avg_robustness_index": 0.8860505638056901,
-      "avg_balanced_accuracy": 0.9493829091920457,
-      "step": 450,
-      "checkpoint": "/data/ryan.kim/waiv_runs/genMASK-c50-ms500-openmidnight-s0-t900-438670/step_0000450",
-      "probe": {
-        "train.cross_scanner.separation": 0.0008522337675094605,
-        "train.cross_scanner.matched": 0.9999733287096023,
-        "train.cross_scanner.random": 0.9991210949420929,
-        "train.cross_scanner.top1": 0.9957421875,
-        "train.cross_stain.separation": 0.0007747122976515028,
-        "train.cross_stain.matched": 0.9998934732543098,
-        "train.cross_stain.random": 0.9991187609566583,
-        "train.cross_stain.top1": 0.9002430555555555,
-        "train.within_condition_random": 0.9991225719451904,
-        "heldout.cross_scanner.separation": 0.0008638395022039545,
-        "heldout.cross_scanner.matched": 0.9999647630404119,
-        "heldout.cross_scanner.random": 0.999100923538208,
-        "heldout.cross_scanner.top1": 0.9910102739726028,
-        "heldout.cross_stain.separation": 0.0007991285351981893,
-        "heldout.cross_stain.matched": 0.99989542556785,
-        "heldout.cross_stain.random": 0.9990962970326518,
-        "heldout.cross_stain.top1": 0.9013157894736842,
-        "heldout.within_condition_random": 0.9991024470910793
-      },
-      "eval_seconds": 5885.5,
-      "train_metrics": {
-        "step": 450,
-        "loss_cls": 1.4154455661773682,
-        "top1_cls": 0.6016666889190674,
-        "weight_cls": 0.5,
-        "negatives_per_anchor": 899.0,
-        "n_rows": 1800.0,
-        "n_cond": 2.0,
-        "n_tiles": 900.0,
-        "loss_mean": 0.03302536532282829,
-        "top1_mean": 0.9988889098167419,
-        "weight_mean": 0.5,
-        "loss": 0.724235475063324,
-        "top1": 0.8002777993679047,
-        "n_heads": 2.0,
-        "split_input_rel_distance": 1.899962306022644,
-        "split_input_cosine": -0.039675090461969376,
-        "heldout_loss": 0.9294995337724685,
-        "heldout_top1": 0.8493611305952072
-      }
-    },
-    {
-      "model": "waiv_genMASK_c50_ms500_openmidnight_s0_t900_438670_s0000500",
-      "adapter_rel_l2_delta": {
-        "camelyon": 0.07128169,
-        "tolkach_esca": 0.04185599,
-        "tcga": 0.0431932
-      },
-      "datasets": {
-        "camelyon": {
-          "robustness_index": 0.8666294917422466,
-          "balanced_accuracy": 0.9674019607843137,
-          "k_opt": 11,
-          "generalization_index": 1.0095163885592746,
-          "confounder_insensitivity": 0.3245345831021497,
-          "prediction_performance": 0.9542201426024955,
-          "ID_performance": 0.9520003777638739,
-          "OOD_performance": 0.9610599832672511
-        },
-        "tolkach_esca": {
-          "robustness_index": 0.9627737226277372,
-          "balanced_accuracy": 0.9642222222222222,
-          "k_opt": 46,
-          "generalization_index": 1.0063881863918485,
-          "confounder_insensitivity": 1.8074077590240525,
-          "prediction_performance": 0.9381328502415459,
-          "ID_performance": 0.9342903836112486,
-          "OOD_performance": 0.9402588047258689
-        },
-        "tcga": {
-          "robustness_index": 0.8345735773214562,
-          "balanced_accuracy": 0.9192641843971632,
-          "k_opt": 61,
-          "generalization_index": 1.0118627481626299,
-          "confounder_insensitivity": 0.7930082372374415,
-          "prediction_performance": 0.8673006045808627,
-          "ID_performance": 0.8627739459288462,
-          "OOD_performance": 0.8730088159706785
-        }
-      },
-      "avg_robustness_index": 0.8879922638971466,
-      "avg_balanced_accuracy": 0.9502961224678996,
-      "step": 500,
-      "checkpoint": "/data/ryan.kim/waiv_runs/genMASK-c50-ms500-openmidnight-s0-t900-438670/step_0000500",
-      "probe": {
-        "train.cross_scanner.separation": 0.0008488714694976807,
-        "train.cross_scanner.matched": 0.9999734890460968,
-        "train.cross_scanner.random": 0.9991246175765991,
-        "train.cross_scanner.top1": 0.995234375,
-        "train.cross_stain.separation": 0.0007730041609870063,
-        "train.cross_stain.matched": 0.9998952264255948,
-        "train.cross_stain.random": 0.9991222222646078,
-        "train.cross_stain.top1": 0.8969270833333334,
-        "train.within_condition_random": 0.9991261005401612,
-        "heldout.cross_scanner.separation": 0.0008602093343865381,
-        "heldout.cross_scanner.matched": 0.9999649671659078,
-        "heldout.cross_scanner.random": 0.9991047578315212,
-        "heldout.cross_scanner.top1": 0.9894049657534246,
-        "heldout.cross_stain.separation": 0.0007979082085235774,
-        "heldout.cross_stain.matched": 0.9998973974010401,
-        "heldout.cross_stain.random": 0.9990994891925165,
-        "heldout.cross_stain.top1": 0.8986659356725146,
-        "heldout.within_condition_random": 0.9991059681264366
-      },
-      "eval_seconds": 2878.3,
-      "train_metrics": {
-        "step": 500,
-        "loss_cls": 1.6445374488830566,
-        "top1_cls": 0.5477777719497681,
-        "weight_cls": 0.5,
-        "negatives_per_anchor": 899.0,
-        "n_rows": 1800.0,
-        "n_cond": 2.0,
-        "n_tiles": 900.0,
-        "loss_mean": 0.031973697245121,
-        "top1_mean": 0.9972222447395325,
-        "weight_mean": 0.5,
-        "loss": 0.8382555842399597,
-        "top1": 0.7725000083446503,
-        "n_heads": 2.0,
-        "split_input_rel_distance": 1.8998554944992065,
-        "split_input_cosine": -0.0377693697810173,
-        "heldout_loss": 0.9279552236199379,
-        "heldout_top1": 0.8495833516120911
-      }
-    }
-  ]
-}

seed0/training_config.json

@@ -1,61 +0,0 @@
-{
-  "packed_dir": "/data/plism/repacked",
-  "out_dir": "/admin/home/ryan.kim/waiv/runs/genMASK-c50-ms500-openmidnight-s0-t900-438670",
-  "lr": 0.0001,
-  "weight_decay": 0.05,
-  "warmup_steps": 200,
-  "max_steps": 500,
-  "temperature": 0.07,
-  "symmetric": false,
-  "grad_accum": 1,
-  "grad_clip": 1.0,
-  "n_groups": 4,
-  "group_size": 64,
-  "grid": true,
-  "grid_conditions": 2,
-  "grid_tiles": 900,
-  "grid_forward_chunk": 0,
-  "activation_offload": true,
-  "num_workers": 10,
-  "seed": 0,
-  "amp_dtype": "bfloat16",
-  "ckpt_every": 50,
-  "log_every": 20,
-  "eval_every": 50,
-  "eval_batches": 20,
-  "ckpt_schedule": null,
-  "resume_from": null,
-  "grid_blocked_loss": false,
-  "grid_pair_block": 8,
-  "eval_heldout": true,
-  "retention_kl_weight": 0.0,
-  "retention_kl_temperature": 0.07,
-  "split_heads": true,
-  "cls_weight": 0.5,
-  "mean_weight": 0.5,
-  "pool_head": "gem",
-  "min_tissue_frac": 0.0,
-  "cores_per_batch": 0,
-  "same_core_logit_bias": null,
-  "center_embeddings": false,
-  "same_core_logit_bias_cls": 3.0,
-  "same_core_logit_bias_mean": "-Infinity",
-  "mask_same_core": true,
-  "core_labels_path": "/admin/home/ryan.kim/waiv/runs/.plism_core_labels.npy",
-  "mask_sim_thresh": 0.0,
-  "sim_emb_path": "/admin/home/ryan.kim/waiv/runs/.plism_ref_emb.npy",
-  "encoder": {
-    "backbone": "SophontAI/OpenMidnight",
-    "use_lora": true,
-    "lora_rank": 32,
-    "lora_alpha": 64,
-    "pooling": "clsmean",
-    "proj_out_dim": 512,
-    "grad_checkpointing": true,
-    "split_heads": [
-      "cls",
-      "mean"
-    ],
-    "pool_head": "gem"
-  }
-}

seed1/_verify.json

@@ -1,28 +0,0 @@
-{
-  "backbone": "openmidnightsq",
-  "hub": "SophontAI/OpenMidnight",
-  "revision": "87189e66",
-  "seed_label": "s1",
-  "run": "genMASK-c50-ms500-openmidnight-s1-t900-438671",
-  "step": "step_0000150",
-  "adapter_dir": "/admin/home/ryan.kim/waiv/runs/genMASK-c50-ms500-openmidnight-s1-t900-438671/step_0000150/adapter",
-  "out": "/admin/home/ryan.kim/spectra-release/merged/spectra-openmidnight-merged/seed1",
-  "merge_info": {
-    "merged": 160,
-    "r": 32,
-    "alpha": 64,
-    "scaling": 2.0,
-    "delta_norm_min": 0.384583979845047,
-    "delta_norm_max": 1.860943078994751,
-    "delta_norm_mean": 0.9923485158011317
-  },
-  "merged_vs_base": {
-    "mean_abs_diff": 0.13222017884254456,
-    "max_abs_diff": 26.86468505859375,
-    "base_feat_absmean": 0.5962615609169006,
-    "feat_dim": 1536
-  },
-  "reload_bit_identical": true,
-  "reload_max_abs_diff": 0.0,
-  "size_bytes": 4539151496
-}

seed1/model.safetensors

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:e15932b775d2ec82628005e2eafeea3bb56178d8eebf37544a59b6cef961f5dd
-size 4539151496

seed1/ri_curve.json

@@ -1,892 +0,0 @@
-{
-  "run_dir": "/data/ryan.kim/waiv_runs/genMASK-c50-ms500-openmidnight-s1-t900-438671",
-  "datasets": [
-    "camelyon",
-    "tolkach_esca",
-    "tcga"
-  ],
-  "targets": {
-    "phikon_v2_base": {
-      "tcga": 0.619,
-      "camelyon": 0.019,
-      "tolkach_esca": 0.768,
-      "avg": 0.469
-    },
-    "phaet_target": {
-      "tcga": 0.785,
-      "camelyon": 0.702,
-      "tolkach_esca": 0.932,
-      "avg": 0.806
-    },
-    "midnight_base": {
-      "tcga": 0.858,
-      "camelyon": 0.478,
-      "tolkach_esca": 0.941,
-      "avg": 0.759
-    },
-    "mascaret_target": {
-      "tcga": 0.893,
-      "camelyon": 0.907,
-      "tolkach_esca": 0.972,
-      "avg": 0.924
-    },
-    "virchow2_base": {
-      "avg": 0.858
-    },
-    "virchow2_target": {
-      "avg": 0.918
-    }
-  },
-  "points": [
-    {
-      "model": "waiv_genMASK_c50_ms500_openmidnight_s1_t900_438671_s0000050",
-      "adapter_rel_l2_delta": {
-        "camelyon": 0.06136422,
-        "tolkach_esca": 0.01991326,
-        "tcga": 0.03652676
-      },
-      "datasets": {
-        "camelyon": {
-          "robustness_index": 0.3044762889643965,
-          "balanced_accuracy": 0.9660294117647059,
-          "k_opt": 11,
-          "generalization_index": 0.991768613888262,
-          "confounder_insensitivity": 0.019731161784620414,
-          "prediction_performance": 0.9570588235294117,
-          "ID_performance": 0.9572112806623708,
-          "OOD_performance": 0.9493321050207277
-        },
-        "tolkach_esca": {
-          "robustness_index": 0.8849231316771212,
-          "balanced_accuracy": 0.955888888888889,
-          "k_opt": 46,
-          "generalization_index": 1.009563751050557,
-          "confounder_insensitivity": 0.5563617225240127,
-          "prediction_performance": 0.9351159420289855,
-          "ID_performance": 0.9319298509426515,
-          "OOD_performance": 0.9408425960336498
-        },
-        "tcga": {
-          "robustness_index": 0.8084059065558935,
-          "balanced_accuracy": 0.9247517730496452,
-          "k_opt": 61,
-          "generalization_index": 1.0070766601767718,
-          "confounder_insensitivity": 0.49730223165664406,
-          "prediction_performance": 0.8959990117428206,
-          "ID_performance": 0.8938980059689281,
-          "OOD_performance": 0.9002238183898641
-        }
-      },
-      "avg_robustness_index": 0.6659351090658038,
-      "avg_balanced_accuracy": 0.9488900245677466,
-      "step": 50,
-      "checkpoint": "/data/ryan.kim/waiv_runs/genMASK-c50-ms500-openmidnight-s1-t900-438671/step_0000050",
-      "probe": {
-        "train.cross_scanner.separation": 0.000524788498878479,
-        "train.cross_scanner.matched": 0.9998936450481415,
-        "train.cross_scanner.random": 0.999368856549263,
-        "train.cross_scanner.top1": 0.7526171875,
-        "train.cross_stain.separation": 0.0004425819714864095,
-        "train.cross_stain.matched": 0.9997820056809319,
-        "train.cross_stain.random": 0.9993394237094455,
-        "train.cross_stain.top1": 0.48694444444444446,
-        "train.within_condition_random": 0.9993930041790009,
-        "heldout.cross_scanner.separation": 0.0005519610561736643,
-        "heldout.cross_scanner.matched": 0.99989515951235,
-        "heldout.cross_scanner.random": 0.9993431984561764,
-        "heldout.cross_scanner.top1": 0.7398330479452054,
-        "heldout.cross_stain.separation": 0.0005128359236912421,
-        "heldout.cross_stain.matched": 0.9998170465056659,
-        "heldout.cross_stain.random": 0.9993042105819747,
-        "heldout.cross_stain.top1": 0.5186631944444444,
-        "heldout.within_condition_random": 0.9993541037164083
-      },
-      "eval_seconds": 7305.3,
-      "train_metrics": {
-        "step": 50,
-        "loss_cls": 5.47167444229126,
-        "top1_cls": 0.012222222983837128,
-        "weight_cls": 0.5,
-        "negatives_per_anchor": 899.0,
-        "n_rows": 1800.0,
-        "n_cond": 2.0,
-        "n_tiles": 900.0,
-        "loss_mean": 0.9203278422355652,
-        "top1_mean": 0.8600000143051147,
-        "weight_mean": 0.5,
-        "loss": 3.1960010528564453,
-        "top1": 0.43611111864447594,
-        "n_heads": 2.0,
-        "split_input_rel_distance": 1.910172939300537,
-        "split_input_cosine": -0.12282799929380417,
-        "heldout_loss": 2.6758341789245605,
-        "heldout_top1": 0.40261112041771413
-      }
-    },
-    {
-      "model": "waiv_genMASK_c50_ms500_openmidnight_s1_t900_438671_s0000100",
-      "adapter_rel_l2_delta": {
-        "camelyon": 0.1100272,
-        "tolkach_esca": 0.03856258,
-        "tcga": 0.06697544
-      },
-      "datasets": {
-        "camelyon": {
-          "robustness_index": 0.7221724965092549,
-          "balanced_accuracy": 0.9707843137254901,
-          "k_opt": 11,
-          "generalization_index": 0.9961661210001546,
-          "confounder_insensitivity": 0.10352695871117494,
-          "prediction_performance": 0.9614928698752229,
-          "ID_performance": 0.9618388181836065,
-          "OOD_performance": 0.9581512445373361
-        },
-        "tolkach_esca": {
-          "robustness_index": 0.9518410338112941,
-          "balanced_accuracy": 0.9667777777777777,
-          "k_opt": 46,
-          "generalization_index": 1.0098529619460113,
-          "confounder_insensitivity": 1.202947906135263,
-          "prediction_performance": 0.9471642512077295,
-          "ID_performance": 0.9420954610759324,
-          "OOD_performance": 0.9513778918034237
-        },
-        "tcga": {
-          "robustness_index": 0.8477243466555902,
-          "balanced_accuracy": 0.9310195035460994,
-          "k_opt": 61,
-          "generalization_index": 1.0131529900729055,
-          "confounder_insensitivity": 0.6518604676286196,
-          "prediction_performance": 0.8985882455528428,
-          "ID_performance": 0.8939482437219196,
-          "OOD_performance": 0.9057063360972852
-        }
-      },
-      "avg_robustness_index": 0.8405792923253798,
-      "avg_balanced_accuracy": 0.9561938650164558,
-      "step": 100,
-      "checkpoint": "/data/ryan.kim/waiv_runs/genMASK-c50-ms500-openmidnight-s1-t900-438671/step_0000100",
-      "probe": {
-        "train.cross_scanner.separation": 0.0004965239763259888,
-        "train.cross_scanner.matched": 0.9999625027179718,
-        "train.cross_scanner.random": 0.9994659787416458,
-        "train.cross_scanner.top1": 0.9090234375,
-        "train.cross_stain.separation": 0.00043502224816216364,
-        "train.cross_stain.matched": 0.9998919036653307,
-        "train.cross_stain.random": 0.9994568814171685,
-        "train.cross_stain.top1": 0.6513541666666667,
-        "train.within_condition_random": 0.9994713950157166,
-        "heldout.cross_scanner.separation": 0.0005213275347670463,
-        "heldout.cross_scanner.matched": 0.9999570152530931,
-        "heldout.cross_scanner.random": 0.9994356877183261,
-        "heldout.cross_scanner.top1": 0.8900363869863014,
-        "heldout.cross_stain.separation": 0.0004827850743343956,
-        "heldout.cross_stain.matched": 0.9999033470599972,
-        "heldout.cross_stain.random": 0.9994205619856628,
-        "heldout.cross_stain.top1": 0.6680601242690059,
-        "heldout.within_condition_random": 0.9994376636132961
-      },
-      "eval_seconds": 5675.3,
-      "train_metrics": {
-        "step": 100,
-        "loss_cls": 2.1127030849456787,
-        "top1_cls": 0.3216666877269745,
-        "weight_cls": 0.5,
-        "negatives_per_anchor": 899.0,
-        "n_rows": 1800.0,
-        "n_cond": 2.0,
-        "n_tiles": 900.0,
-        "loss_mean": 0.03901568055152893,
-        "top1_mean": 0.9977778196334839,
-        "weight_mean": 0.5,
-        "loss": 1.0758594274520874,
-        "top1": 0.6597222536802292,
-        "n_heads": 2.0,
-        "split_input_rel_distance": 1.910251259803772,
-        "split_input_cosine": -0.09836520254611969,
-        "heldout_loss": 2.0313774585723876,
-        "heldout_top1": 0.5919444590806962
-      }
-    },
-    {
-      "model": "waiv_genMASK_c50_ms500_openmidnight_s1_t900_438671_s0000150",
-      "adapter_rel_l2_delta": {
-        "camelyon": 0.1047443,
-        "tolkach_esca": 0.03587291,
-        "tcga": 0.06865262
-      },
-      "datasets": {
-        "camelyon": {
-          "robustness_index": 0.8574703557312253,
-          "balanced_accuracy": 0.9713235294117647,
-          "k_opt": 11,
-          "generalization_index": 1.0073306807248654,
-          "confounder_insensitivity": 0.24996518590725528,
-          "prediction_performance": 0.96123440285205,
-          "ID_performance": 0.9598273220999861,
-          "OOD_performance": 0.9668635097493037
-        },
-        "tolkach_esca": {
-          "robustness_index": 0.9612054076588649,
-          "balanced_accuracy": 0.9668888888888888,
-          "k_opt": 46,
-          "generalization_index": 1.0115899564469417,
-          "confounder_insensitivity": 1.6035934620875283,
-          "prediction_performance": 0.9452512077294686,
-          "ID_performance": 0.9385514209708762,
-          "OOD_performance": 0.9494291910631439
-        },
-        "tcga": {
-          "robustness_index": 0.848878838723779,
-          "balanced_accuracy": 0.9243794326241135,
-          "k_opt": 61,
-          "generalization_index": 1.010758210192123,
-          "confounder_insensitivity": 0.7593499344155827,
-          "prediction_performance": 0.8838733868154867,
-          "ID_performance": 0.8797882377607716,
-          "OOD_performance": 0.8892531845471596
-        }
-      },
-      "avg_robustness_index": 0.8891848673712897,
-      "avg_balanced_accuracy": 0.9541972836415891,
-      "step": 150,
-      "checkpoint": "/data/ryan.kim/waiv_runs/genMASK-c50-ms500-openmidnight-s1-t900-438671/step_0000150",
-      "probe": {
-        "train.cross_scanner.separation": 0.0005439198017120361,
-        "train.cross_scanner.matched": 0.9999751263856888,
-        "train.cross_scanner.random": 0.9994312065839768,
-        "train.cross_scanner.top1": 0.969140625,
-        "train.cross_stain.separation": 0.00048600117365519207,
-        "train.cross_stain.matched": 0.9999127350913154,
-        "train.cross_stain.random": 0.9994267339176602,
-        "train.cross_stain.top1": 0.7814236111111111,
-        "train.within_condition_random": 0.99943323969841,
-        "heldout.cross_scanner.separation": 0.000571564452288902,
-        "heldout.cross_scanner.matched": 0.9999684030062532,
-        "heldout.cross_scanner.random": 0.9993968385539643,
-        "heldout.cross_scanner.top1": 0.950556506849315,
-        "heldout.cross_stain.separation": 0.0005242015883239389,
-        "heldout.cross_stain.matched": 0.9999169243706597,
-        "heldout.cross_stain.random": 0.9993927227823358,
-        "heldout.cross_stain.top1": 0.7856816520467836,
-        "heldout.within_condition_random": 0.9994002842321629
-      },
-      "eval_seconds": 3770.6,
-      "train_metrics": {
-        "step": 150,
-        "loss_cls": 1.6272633075714111,
-        "top1_cls": 0.4766666889190674,
-        "weight_cls": 0.5,
-        "negatives_per_anchor": 899.0,
-        "n_rows": 1800.0,
-        "n_cond": 2.0,
-        "n_tiles": 900.0,
-        "loss_mean": 0.04308953136205673,
-        "top1_mean": 0.996666669845581,
-        "weight_mean": 0.5,
-        "loss": 0.835176408290863,
-        "top1": 0.7366666793823242,
-        "n_heads": 2.0,
-        "split_input_rel_distance": 1.9065666198730469,
-        "split_input_cosine": -0.0632912665605545,
-        "heldout_loss": 1.69220871925354,
-        "heldout_top1": 0.6917500168085098
-      }
-    },
-    {
-      "model": "waiv_genMASK_c50_ms500_openmidnight_s1_t900_438671_s0000200",
-      "adapter_rel_l2_delta": {
-        "camelyon": 0.08966768,
-        "tolkach_esca": 0.03021276,
-        "tcga": 0.05946582
-      },
-      "datasets": {
-        "camelyon": {
-          "robustness_index": 0.8728087986463621,
-          "balanced_accuracy": 0.9699509803921569,
-          "k_opt": 11,
-          "generalization_index": 1.0092718562742162,
-          "confounder_insensitivity": 0.3127413127413127,
-          "prediction_performance": 0.9581372549019608,
-          "ID_performance": 0.9560255060255061,
-          "OOD_performance": 0.9648896371118594
-        },
-        "tolkach_esca": {
-          "robustness_index": 0.9615246202960239,
-          "balanced_accuracy": 0.964111111111111,
-          "k_opt": 46,
-          "generalization_index": 1.0100094707560654,
-          "confounder_insensitivity": 1.749223045661009,
-          "prediction_performance": 0.9399202898550725,
-          "ID_performance": 0.9339721624565039,
-          "OOD_performance": 0.9433207295035914
-        },
-        "tcga": {
-          "robustness_index": 0.8427904100903469,
-          "balanced_accuracy": 0.921108156028369,
-          "k_opt": 61,
-          "generalization_index": 1.0101032635588179,
-          "confounder_insensitivity": 0.7948720624125494,
-          "prediction_performance": 0.8736312638065341,
-          "ID_performance": 0.869739784243899,
-          "OOD_performance": 0.8785269945117045
-        }
-      },
-      "avg_robustness_index": 0.8923746096775776,
-      "avg_balanced_accuracy": 0.9517234158438791,
-      "step": 200,
-      "checkpoint": "/data/ryan.kim/waiv_runs/genMASK-c50-ms500-openmidnight-s1-t900-438671/step_0000200",
-      "probe": {
-        "train.cross_scanner.separation": 0.0005558806657791137,
-        "train.cross_scanner.matched": 0.9999790400266647,
-        "train.cross_scanner.random": 0.9994231593608857,
-        "train.cross_scanner.top1": 0.9857421875,
-        "train.cross_stain.separation": 0.0005010872417026096,
-        "train.cross_stain.matched": 0.9999212633238899,
-        "train.cross_stain.random": 0.9994201760821873,
-        "train.cross_stain.top1": 0.8493402777777778,
-        "train.within_condition_random": 0.9994246196746827,
-        "heldout.cross_scanner.separation": 0.000574913743424089,
-        "heldout.cross_scanner.matched": 0.999972889684651,
-        "heldout.cross_scanner.random": 0.9993979759412269,
-        "heldout.cross_scanner.top1": 0.9743150684931506,
-        "heldout.cross_stain.separation": 0.000529170036315918,
-        "heldout.cross_stain.matched": 0.9999240567112526,
-        "heldout.cross_stain.random": 0.9993948866749367,
-        "heldout.cross_stain.top1": 0.8535270467836257,
-        "heldout.within_condition_random": 0.999400167930417
-      },
-      "eval_seconds": 2158.7,
-      "train_metrics": {
-        "step": 200,
-        "loss_cls": 2.260517120361328,
-        "top1_cls": 0.40666666626930237,
-        "weight_cls": 0.5,
-        "negatives_per_anchor": 899.0,
-        "n_rows": 1800.0,
-        "n_cond": 2.0,
-        "n_tiles": 900.0,
-        "loss_mean": 0.08051653206348419,
-        "top1_mean": 0.987777829170227,
-        "weight_mean": 0.5,
-        "loss": 1.170516848564148,
-        "top1": 0.6972222477197647,
-        "n_heads": 2.0,
-        "split_input_rel_distance": 1.9042853116989136,
-        "split_input_cosine": -0.05271483585238457,
-        "heldout_loss": 1.5069379478693008,
-        "heldout_top1": 0.7278055712580681
-      }
-    },
-    {
-      "model": "waiv_genMASK_c50_ms500_openmidnight_s1_t900_438671_s0000250",
-      "adapter_rel_l2_delta": {
-        "camelyon": 0.07895029,
-        "tolkach_esca": 0.0334071,
-        "tcga": 0.05336334
-      },
-      "datasets": {
-        "camelyon": {
-          "robustness_index": 0.8699556043298249,
-          "balanced_accuracy": 0.9684313725490197,
-          "k_opt": 11,
-          "generalization_index": 1.0058516529929036,
-          "confounder_insensitivity": 0.32097907861120595,
-          "prediction_performance": 0.9553163992869875,
-          "ID_performance": 0.9539599938778153,
-          "OOD_performance": 0.9595422367311007
-        },
-        "tolkach_esca": {
-          "robustness_index": 0.961832560718301,
-          "balanced_accuracy": 0.9634444444444444,
-          "k_opt": 46,
-          "generalization_index": 1.009500678882938,
-          "confounder_insensitivity": 1.7805949398545224,
-          "prediction_performance": 0.9390990338164251,
-          "ID_performance": 0.93342019894015,
-          "OOD_performance": 0.9422883245131285
-        },
-        "tcga": {
-          "robustness_index": 0.836673059812409,
-          "balanced_accuracy": 0.9200531914893617,
-          "k_opt": 61,
-          "generalization_index": 1.0102742338538446,
-          "confounder_insensitivity": 0.793427435623962,
-          "prediction_performance": 0.8686260318567609,
-          "ID_performance": 0.8646956375779451,
-          "OOD_performance": 0.8735797227708201
-        }
-      },
-      "avg_robustness_index": 0.8894870749535116,
-      "avg_balanced_accuracy": 0.9506430028276086,
-      "step": 250,
-      "checkpoint": "/data/ryan.kim/waiv_runs/genMASK-c50-ms500-openmidnight-s1-t900-438671/step_0000250",
-      "probe": {
-        "train.cross_scanner.separation": 0.0006345057487487793,
-        "train.cross_scanner.matched": 0.9999766367673874,
-        "train.cross_scanner.random": 0.9993421310186386,
-        "train.cross_scanner.top1": 0.9923828125,
-        "train.cross_stain.separation": 0.0005705189704895019,
-        "train.cross_stain.matched": 0.9999101000361973,
-        "train.cross_stain.random": 0.9993395810657077,
-        "train.cross_stain.top1": 0.8784895833333334,
-        "train.within_condition_random": 0.9993436658382415,
-        "heldout.cross_scanner.separation": 0.0006515440875536775,
-        "heldout.cross_scanner.matched": 0.9999696636853153,
-        "heldout.cross_scanner.random": 0.9993181195977616,
-        "heldout.cross_scanner.top1": 0.9840539383561644,
-        "heldout.cross_stain.separation": 0.00059856483113696,
-        "heldout.cross_stain.matched": 0.9999122480202837,
-        "heldout.cross_stain.random": 0.9993136831891467,
-        "heldout.cross_stain.top1": 0.8813733552631579,
-        "heldout.within_condition_random": 0.9993195693667342
-      },
-      "eval_seconds": 2117.8,
-      "train_metrics": {
-        "step": 250,
-        "loss_cls": 2.9609413146972656,
-        "top1_cls": 0.3266666829586029,
-        "weight_cls": 0.5,
-        "negatives_per_anchor": 899.0,
-        "n_rows": 1800.0,
-        "n_cond": 2.0,
-        "n_tiles": 900.0,
-        "loss_mean": 0.12087370455265045,
-        "top1_mean": 0.9855555891990662,
-        "weight_mean": 0.5,
-        "loss": 1.5409075021743774,
-        "top1": 0.6561111360788345,
-        "n_heads": 2.0,
-        "split_input_rel_distance": 1.9015880823135376,
-        "split_input_cosine": -0.03394003584980965,
-        "heldout_loss": 1.3572636425495148,
-        "heldout_top1": 0.7470000118017197
-      }
-    },
-    {
-      "model": "waiv_genMASK_c50_ms500_openmidnight_s1_t900_438671_s0000300",
-      "adapter_rel_l2_delta": {
-        "camelyon": 0.07525133,
-        "tolkach_esca": 0.03591809,
-        "tcga": 0.04877162
-      },
-      "datasets": {
-        "camelyon": {
-          "robustness_index": 0.8662398978443134,
-          "balanced_accuracy": 0.967156862745098,
-          "k_opt": 11,
-          "generalization_index": 1.0090940244182038,
-          "confounder_insensitivity": 0.3137327221314786,
-          "prediction_performance": 0.9554634581105169,
-          "ID_performance": 0.9533929313685886,
-          "OOD_performance": 0.9620631099665976
-        },
-        "tolkach_esca": {
-          "robustness_index": 0.9616502998544171,
-          "balanced_accuracy": 0.9641111111111113,
-          "k_opt": 46,
-          "generalization_index": 1.010164367142275,
-          "confounder_insensitivity": 1.7600184001226677,
-          "prediction_performance": 0.9398454106280194,
-          "ID_performance": 0.933792891952613,
-          "OOD_performance": 0.9432843057412661
-        },
-        "tcga": {
-          "robustness_index": 0.8402202896971682,
-          "balanced_accuracy": 0.9218351063829787,
-          "k_opt": 61,
-          "generalization_index": 1.0115006809123899,
-          "confounder_insensitivity": 0.780581292199787,
-          "prediction_performance": 0.8738386524822696,
-          "ID_performance": 0.8694550916116424,
-          "OOD_performance": 0.8794544171879206
-        }
-      },
-      "avg_robustness_index": 0.8893701624652995,
-      "avg_balanced_accuracy": 0.9510343600797294,
-      "step": 300,
-      "checkpoint": "/data/ryan.kim/waiv_runs/genMASK-c50-ms500-openmidnight-s1-t900-438671/step_0000300",
-      "probe": {
-        "train.cross_scanner.separation": 0.0007030421495437622,
-        "train.cross_scanner.matched": 0.9999754470586777,
-        "train.cross_scanner.random": 0.9992724049091339,
-        "train.cross_scanner.top1": 0.991953125,
-        "train.cross_stain.separation": 0.000636943711174859,
-        "train.cross_stain.matched": 0.9999064768685235,
-        "train.cross_stain.random": 0.9992695331573487,
-        "train.cross_stain.top1": 0.8796527777777777,
-        "train.within_condition_random": 0.9992739570140838,
-        "heldout.cross_scanner.separation": 0.0007180440915773992,
-        "heldout.cross_scanner.matched": 0.9999682086787812,
-        "heldout.cross_scanner.random": 0.9992501645872037,
-        "heldout.cross_scanner.top1": 0.9856057363013698,
-        "heldout.cross_stain.separation": 0.0006636754114028306,
-        "heldout.cross_stain.matched": 0.9999092162003991,
-        "heldout.cross_stain.random": 0.9992455407889963,
-        "heldout.cross_stain.top1": 0.8839546783625731,
-        "heldout.within_condition_random": 0.9992518802968468
-      },
-      "eval_seconds": 2087.0,
-      "train_metrics": {
-        "step": 300,
-        "loss_cls": 2.3239221572875977,
-        "top1_cls": 0.38277778029441833,
-        "weight_cls": 0.5,
-        "negatives_per_anchor": 899.0,
-        "n_rows": 1800.0,
-        "n_cond": 2.0,
-        "n_tiles": 900.0,
-        "loss_mean": 0.04672984778881073,
-        "top1_mean": 0.9972222447395325,
-        "weight_mean": 0.5,
-        "loss": 1.1853259801864624,
-        "top1": 0.6900000125169754,
-        "n_heads": 2.0,
-        "split_input_rel_distance": 1.901183009147644,
-        "split_input_cosine": -0.036429513245821,
-        "heldout_loss": 1.2729288652539252,
-        "heldout_top1": 0.7586944624781609
-      }
-    },
-    {
-      "model": "waiv_genMASK_c50_ms500_openmidnight_s1_t900_438671_s0000350",
-      "adapter_rel_l2_delta": {
-        "camelyon": 0.07301231,
-        "tolkach_esca": 0.03730401,
-        "tcga": 0.04727442
-      },
-      "datasets": {
-        "camelyon": {
-          "robustness_index": 0.8662895815018162,
-          "balanced_accuracy": 0.9666666666666667,
-          "k_opt": 11,
-          "generalization_index": 1.008263218075922,
-          "confounder_insensitivity": 0.3198136733637604,
-          "prediction_performance": 0.9544964349376115,
-          "ID_performance": 0.9525890462522938,
-          "OOD_performance": 0.9604604972782109
-        },
-        "tolkach_esca": {
-          "robustness_index": 0.9624874739844292,
-          "balanced_accuracy": 0.9652222222222222,
-          "k_opt": 46,
-          "generalization_index": 1.009079460603085,
-          "confounder_insensitivity": 1.773757838880849,
-          "prediction_performance": 0.9402173913043479,
-          "ID_performance": 0.934789891193654,
-          "OOD_performance": 0.9432772791829088
-        },
-        "tcga": {
-          "robustness_index": 0.8380770589332686,
-          "balanced_accuracy": 0.9207535460992907,
-          "k_opt": 61,
-          "generalization_index": 1.011741011721634,
-          "confounder_insensitivity": 0.7855053028272316,
-          "prediction_performance": 0.8713687361934659,
-          "ID_performance": 0.8668910052393817,
-          "OOD_performance": 0.8770691826932765
-        }
-      },
-      "avg_robustness_index": 0.8889513714731714,
-      "avg_balanced_accuracy": 0.9508808116627265,
-      "step": 350,
-      "checkpoint": "/data/ryan.kim/waiv_runs/genMASK-c50-ms500-openmidnight-s1-t900-438671/step_0000350",
-      "probe": {
-        "train.cross_scanner.separation": 0.0007235342264175415,
-        "train.cross_scanner.matched": 0.9999768221378327,
-        "train.cross_scanner.random": 0.9992532879114151,
-        "train.cross_scanner.top1": 0.993515625,
-        "train.cross_stain.separation": 0.0006596321529812283,
-        "train.cross_stain.matched": 0.9999103175269233,
-        "train.cross_stain.random": 0.999250685373942,
-        "train.cross_stain.top1": 0.8869270833333334,
-        "train.within_condition_random": 0.999254720211029,
-        "heldout.cross_scanner.separation": 0.0007370857343281785,
-        "heldout.cross_scanner.matched": 0.9999695608060654,
-        "heldout.cross_scanner.random": 0.9992324750717372,
-        "heldout.cross_scanner.top1": 0.9864619006849316,
-        "heldout.cross_stain.separation": 0.0006846450922781962,
-        "heldout.cross_stain.matched": 0.9999129085512887,
-        "heldout.cross_stain.random": 0.9992282634590104,
-        "heldout.cross_stain.top1": 0.8902595029239766,
-        "heldout.within_condition_random": 0.9992342562210269
-      },
-      "eval_seconds": 2106.1,
-      "train_metrics": {
-        "step": 350,
-        "loss_cls": 0.8300271034240723,
-        "top1_cls": 0.7233333587646484,
-        "weight_cls": 0.5,
-        "negatives_per_anchor": 899.0,
-        "n_rows": 1800.0,
-        "n_cond": 2.0,
-        "n_tiles": 900.0,
-        "loss_mean": 0.008635702542960644,
-        "top1_mean": 0.9994444847106934,
-        "weight_mean": 0.5,
-        "loss": 0.4193314015865326,
-        "top1": 0.8613889217376709,
-        "n_heads": 2.0,
-        "split_input_rel_distance": 1.900965929031372,
-        "split_input_cosine": -0.03565763682126999,
-        "heldout_loss": 1.1916852489113807,
-        "heldout_top1": 0.7697916850447655
-      }
-    },
-    {
-      "model": "waiv_genMASK_c50_ms500_openmidnight_s1_t900_438671_s0000400",
-      "adapter_rel_l2_delta": {
-        "camelyon": 0.07072918,
-        "tolkach_esca": 0.04523943,
-        "tcga": 0.04370555
-      },
-      "datasets": {
-        "camelyon": {
-          "robustness_index": 0.8638497652582159,
-          "balanced_accuracy": 0.965343137254902,
-          "k_opt": 11,
-          "generalization_index": 1.0081303998845148,
-          "confounder_insensitivity": 0.3237766569330147,
-          "prediction_performance": 0.952963458110517,
-          "ID_performance": 0.9510721765035542,
-          "OOD_performance": 0.958804773617564
-        },
-        "tolkach_esca": {
-          "robustness_index": 0.9622235436287956,
-          "balanced_accuracy": 0.9656666666666666,
-          "k_opt": 46,
-          "generalization_index": 1.0095604389967847,
-          "confounder_insensitivity": 1.800570937650091,
-          "prediction_performance": 0.9391256038647343,
-          "ID_performance": 0.9333883526013516,
-          "OOD_performance": 0.9423119550067062
-        },
-        "tcga": {
-          "robustness_index": 0.8323548123867612,
-          "balanced_accuracy": 0.9184042553191489,
-          "k_opt": 61,
-          "generalization_index": 1.0112666259457297,
-          "confounder_insensitivity": 0.7918936655590163,
-          "prediction_performance": 0.8653967561911406,
-          "ID_performance": 0.8611092303044823,
-          "OOD_performance": 0.8708110259007379
-        }
-      },
-      "avg_robustness_index": 0.8861427070912575,
-      "avg_balanced_accuracy": 0.9498046864135725,
-      "step": 400,
-      "checkpoint": "/data/ryan.kim/waiv_runs/genMASK-c50-ms500-openmidnight-s1-t900-438671/step_0000400",
-      "probe": {
-        "train.cross_scanner.separation": 0.0007612556219100952,
-        "train.cross_scanner.matched": 0.9999756723642349,
-        "train.cross_scanner.random": 0.9992144167423248,
-        "train.cross_scanner.top1": 0.9950390625,
-        "train.cross_stain.separation": 0.0006937800513373481,
-        "train.cross_stain.matched": 0.999905706246694,
-        "train.cross_stain.random": 0.9992119261953566,
-        "train.cross_stain.top1": 0.8957986111111111,
-        "train.within_condition_random": 0.9992157924175262,
-        "heldout.cross_scanner.separation": 0.0007743802789139421,
-        "heldout.cross_scanner.matched": 0.9999682584854022,
-        "heldout.cross_scanner.random": 0.9991938782064882,
-        "heldout.cross_scanner.top1": 0.9894049657534246,
-        "heldout.cross_stain.separation": 0.000717160297416107,
-        "heldout.cross_stain.matched": 0.9999082566004748,
-        "heldout.cross_stain.random": 0.9991910963030587,
-        "heldout.cross_stain.top1": 0.8984603435672515,
-        "heldout.within_condition_random": 0.9991961921133646
-      },
-      "eval_seconds": 2189.8,
-      "train_metrics": {
-        "step": 400,
-        "loss_cls": 0.766818642616272,
-        "top1_cls": 0.7566666603088379,
-        "weight_cls": 0.5,
-        "negatives_per_anchor": 899.0,
-        "n_rows": 1800.0,
-        "n_cond": 2.0,
-        "n_tiles": 900.0,
-        "loss_mean": 0.009244825690984726,
-        "top1_mean": 1.0,
-        "weight_mean": 0.5,
-        "loss": 0.3880317211151123,
-        "top1": 0.878333330154419,
-        "n_heads": 2.0,
-        "split_input_rel_distance": 1.899885892868042,
-        "split_input_cosine": -0.030453987419605255,
-        "heldout_loss": 1.150944358110428,
-        "heldout_top1": 0.7765555739402771
-      }
-    },
-    {
-      "model": "waiv_genMASK_c50_ms500_openmidnight_s1_t900_438671_s0000450",
-      "adapter_rel_l2_delta": {
-        "camelyon": 0.06967939,
-        "tolkach_esca": 0.05029909,
-        "tcga": 0.04127648
-      },
-      "datasets": {
-        "camelyon": {
-          "robustness_index": 0.8661296263672907,
-          "balanced_accuracy": 0.9661274509803921,
-          "k_opt": 11,
-          "generalization_index": 1.0087585103690795,
-          "confounder_insensitivity": 0.33587331825217287,
-          "prediction_performance": 0.952330659536542,
-          "ID_performance": 0.9502381235861412,
-          "OOD_performance": 0.9585607940446651
-        },
-        "tolkach_esca": {
-          "robustness_index": 0.9620122338674486,
-          "balanced_accuracy": 0.965,
-          "k_opt": 46,
-          "generalization_index": 1.0090931506038932,
-          "confounder_insensitivity": 1.8181861500445873,
-          "prediction_performance": 0.9379154589371981,
-          "ID_performance": 0.9324452189540037,
-          "OOD_performance": 0.9409240837598326
-        },
-        "tcga": {
-          "robustness_index": 0.8299693383131104,
-          "balanced_accuracy": 0.9167464539007091,
-          "k_opt": 61,
-          "generalization_index": 1.0110463808269818,
-          "confounder_insensitivity": 0.8028059193280507,
-          "prediction_performance": 0.8616428322288106,
-          "ID_performance": 0.8574251116863,
-          "OOD_performance": 0.8668965560006044
-        }
-      },
-      "avg_robustness_index": 0.8860370661826166,
-      "avg_balanced_accuracy": 0.9492913016270337,
-      "step": 450,
-      "checkpoint": "/data/ryan.kim/waiv_runs/genMASK-c50-ms500-openmidnight-s1-t900-438671/step_0000450",
-      "probe": {
-        "train.cross_scanner.separation": 0.0007996958494186401,
-        "train.cross_scanner.matched": 0.9999754446744918,
-        "train.cross_scanner.random": 0.9991757488250732,
-        "train.cross_scanner.top1": 0.99578125,
-        "train.cross_stain.separation": 0.0007282469007703993,
-        "train.cross_stain.matched": 0.9999017741945054,
-        "train.cross_stain.random": 0.9991735272937351,
-        "train.cross_stain.top1": 0.8998784722222222,
-        "train.within_condition_random": 0.9991770792007446,
-        "heldout.cross_scanner.separation": 0.0008127656701492937,
-        "heldout.cross_scanner.matched": 0.9999674697444864,
-        "heldout.cross_scanner.random": 0.999154704074337,
-        "heldout.cross_scanner.top1": 0.991117294520548,
-        "heldout.cross_stain.separation": 0.0007515012869360851,
-        "heldout.cross_stain.matched": 0.99990364299183,
-        "heldout.cross_stain.random": 0.999152141704894,
-        "heldout.cross_stain.top1": 0.9014300073099415,
-        "heldout.within_condition_random": 0.9991570638447274
-      },
-      "eval_seconds": 2201.5,
-      "train_metrics": {
-        "step": 450,
-        "loss_cls": 0.8002045750617981,
-        "top1_cls": 0.7311111092567444,
-        "weight_cls": 0.5,
-        "negatives_per_anchor": 899.0,
-        "n_rows": 1800.0,
-        "n_cond": 2.0,
-        "n_tiles": 900.0,
-        "loss_mean": 0.011907201260328293,
-        "top1_mean": 0.9994444847106934,
-        "weight_mean": 0.5,
-        "loss": 0.40605589747428894,
-        "top1": 0.8652777969837189,
-        "n_heads": 2.0,
-        "split_input_rel_distance": 1.8990858793258667,
-        "split_input_cosine": -0.030092019587755203,
-        "heldout_loss": 1.1497170731425286,
-        "heldout_top1": 0.7777083501219749
-      }
-    },
-    {
-      "model": "waiv_genMASK_c50_ms500_openmidnight_s1_t900_438671_s0000500",
-      "adapter_rel_l2_delta": {
-        "camelyon": 0.06940158,
-        "tolkach_esca": 0.05033597,
-        "tcga": 0.04104985
-      },
-      "datasets": {
-        "camelyon": {
-          "robustness_index": 0.8652814906116438,
-          "balanced_accuracy": 0.9657352941176471,
-          "k_opt": 11,
-          "generalization_index": 1.0085097181324945,
-          "confounder_insensitivity": 0.33592103539833545,
-          "prediction_performance": 0.9519295900178253,
-          "ID_performance": 0.9498970078702657,
-          "OOD_performance": 0.9579803636621415
-        },
-        "tolkach_esca": {
-          "robustness_index": 0.9619455413720268,
-          "balanced_accuracy": 0.9646666666666667,
-          "k_opt": 46,
-          "generalization_index": 1.0088396975531004,
-          "confounder_insensitivity": 1.8222975137876731,
-          "prediction_performance": 0.937524154589372,
-          "ID_performance": 0.9322035053753179,
-          "OOD_performance": 0.9404439024207759
-        },
-        "tcga": {
-          "robustness_index": 0.8292147328004599,
-          "balanced_accuracy": 0.9163563829787232,
-          "k_opt": 61,
-          "generalization_index": 1.0111507409134082,
-          "confounder_insensitivity": 0.8026358912853998,
-          "prediction_performance": 0.8610385420299965,
-          "ID_performance": 0.8567846548550108,
-          "OOD_performance": 0.8663384385598829
-        }
-      },
-      "avg_robustness_index": 0.8854805882613768,
-      "avg_balanced_accuracy": 0.9489194479210123,
-      "step": 500,
-      "checkpoint": "/data/ryan.kim/waiv_runs/genMASK-c50-ms500-openmidnight-s1-t900-438671/step_0000500",
-      "probe": {
-        "train.cross_scanner.separation": 0.0008003395795822143,
-        "train.cross_scanner.matched": 0.9999753910303116,
-        "train.cross_scanner.random": 0.9991750514507294,
-        "train.cross_scanner.top1": 0.9958203125,
-        "train.cross_stain.separation": 0.0007288002967834473,
-        "train.cross_stain.matched": 0.999901660018497,
-        "train.cross_stain.random": 0.9991728597217135,
-        "train.cross_stain.top1": 0.9006770833333333,
-        "train.within_condition_random": 0.9991763818264008,
-        "heldout.cross_scanner.separation": 0.0008131592241052078,
-        "heldout.cross_scanner.matched": 0.9999674093233396,
-        "heldout.cross_scanner.random": 0.9991542500992344,
-        "heldout.cross_scanner.top1": 0.9914383561643836,
-        "heldout.cross_stain.separation": 0.0007518979541042395,
-        "heldout.cross_stain.matched": 0.9999035359823216,
-        "heldout.cross_stain.random": 0.9991516380282174,
-        "heldout.cross_stain.top1": 0.9017955043859649,
-        "heldout.within_condition_random": 0.9991565681085354
-      },
-      "eval_seconds": 2199.0,
-      "train_metrics": {
-        "step": 500,
-        "loss_cls": 0.38542670011520386,
-        "top1_cls": 0.8655555844306946,
-        "weight_cls": 0.5,
-        "negatives_per_anchor": 899.0,
-        "n_rows": 1800.0,
-        "n_cond": 2.0,
-        "n_tiles": 900.0,
-        "loss_mean": 0.004598951432853937,
-        "top1_mean": 1.0,
-        "weight_mean": 0.5,
-        "loss": 0.19501282274723053,
-        "top1": 0.9327777922153473,
-        "n_heads": 2.0,
-        "split_input_rel_distance": 1.8991509675979614,
-        "split_input_cosine": -0.02834084816277027,
-        "heldout_loss": 1.1481799900531768,
-        "heldout_top1": 0.7784583494067192
-      }
-    }
-  ]
-}

seed1/training_config.json

@@ -1,61 +0,0 @@
-{
-  "packed_dir": "/data/plism/repacked",
-  "out_dir": "/admin/home/ryan.kim/waiv/runs/genMASK-c50-ms500-openmidnight-s1-t900-438671",
-  "lr": 0.0001,
-  "weight_decay": 0.05,
-  "warmup_steps": 200,
-  "max_steps": 500,
-  "temperature": 0.07,
-  "symmetric": false,
-  "grad_accum": 1,
-  "grad_clip": 1.0,
-  "n_groups": 4,
-  "group_size": 64,
-  "grid": true,
-  "grid_conditions": 2,
-  "grid_tiles": 900,
-  "grid_forward_chunk": 0,
-  "activation_offload": true,
-  "num_workers": 10,
-  "seed": 1,
-  "amp_dtype": "bfloat16",
-  "ckpt_every": 50,
-  "log_every": 20,
-  "eval_every": 50,
-  "eval_batches": 20,
-  "ckpt_schedule": null,
-  "resume_from": null,
-  "grid_blocked_loss": false,
-  "grid_pair_block": 8,
-  "eval_heldout": true,
-  "retention_kl_weight": 0.0,
-  "retention_kl_temperature": 0.07,
-  "split_heads": true,
-  "cls_weight": 0.5,
-  "mean_weight": 0.5,
-  "pool_head": "gem",
-  "min_tissue_frac": 0.0,
-  "cores_per_batch": 0,
-  "same_core_logit_bias": null,
-  "center_embeddings": false,
-  "same_core_logit_bias_cls": 3.0,
-  "same_core_logit_bias_mean": "-Infinity",
-  "mask_same_core": true,
-  "core_labels_path": "/admin/home/ryan.kim/waiv/runs/.plism_core_labels.npy",
-  "mask_sim_thresh": 0.0,
-  "sim_emb_path": "/admin/home/ryan.kim/waiv/runs/.plism_ref_emb.npy",
-  "encoder": {
-    "backbone": "SophontAI/OpenMidnight",
-    "use_lora": true,
-    "lora_rank": 32,
-    "lora_alpha": 64,
-    "pooling": "clsmean",
-    "proj_out_dim": 512,
-    "grad_checkpointing": true,
-    "split_heads": [
-      "cls",
-      "mean"
-    ],
-    "pool_head": "gem"
-  }
-}

seed2/_verify.json

@@ -1,28 +0,0 @@
-{
-  "backbone": "openmidnightsq",
-  "hub": "SophontAI/OpenMidnight",
-  "revision": "87189e66",
-  "seed_label": "s2",
-  "run": "genMASK-c50-ms500-openmidnight-s2-t900-438672",
-  "step": "step_0000150",
-  "adapter_dir": "/admin/home/ryan.kim/waiv/runs/genMASK-c50-ms500-openmidnight-s2-t900-438672/step_0000150/adapter",
-  "out": "/admin/home/ryan.kim/spectra-release/merged/spectra-openmidnight-merged/seed2",
-  "merge_info": {
-    "merged": 160,
-    "r": 32,
-    "alpha": 64,
-    "scaling": 2.0,
-    "delta_norm_min": 0.3815061151981354,
-    "delta_norm_max": 1.9115675687789917,
-    "delta_norm_mean": 0.9905468866229057
-  },
-  "merged_vs_base": {
-    "mean_abs_diff": 0.11856531351804733,
-    "max_abs_diff": 23.04315185546875,
-    "base_feat_absmean": 0.5962615609169006,
-    "feat_dim": 1536
-  },
-  "reload_bit_identical": true,
-  "reload_max_abs_diff": 0.0,
-  "size_bytes": 4539151496
-}

seed2/config.json

@@ -1,27 +0,0 @@
-{
-  "architecture": "vit_giant_patch14_reg4_dinov2",
-  "num_classes": 0,
-  "num_features": 1536,
-  "global_pool": "token",
-  "pretrained_cfg": {
-    "custom_load": true,
-    "input_size": [
-      3,
-      224,
-      224
-    ],
-    "fixed_input_size": true,
-    "mean": [
-      0.485,
-      0.456,
-      0.406
-    ],
-    "std": [
-      0.229,
-      0.224,
-      0.225
-    ],
-    "num_classes": 0,
-    "license": "Apache 2.0"
-  }
-}

seed2/model.safetensors

@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:3ef8c5cabf08f0e8c4e776cdfb826c86ef9b52d66f0c509f83da62531e463efd
-size 4539151496

seed2/ri_curve.json

@@ -1,637 +0,0 @@
-{
-  "run_dir": "/data/ryan.kim/waiv_runs/genMASK-c50-ms500-openmidnight-s2-t900-438672",
-  "datasets": [
-    "camelyon",
-    "tolkach_esca",
-    "tcga"
-  ],
-  "targets": {
-    "phikon_v2_base": {
-      "tcga": 0.619,
-      "camelyon": 0.019,
-      "tolkach_esca": 0.768,
-      "avg": 0.469
-    },
-    "phaet_target": {
-      "tcga": 0.785,
-      "camelyon": 0.702,
-      "tolkach_esca": 0.932,
-      "avg": 0.806
-    },
-    "midnight_base": {
-      "tcga": 0.858,
-      "camelyon": 0.478,
-      "tolkach_esca": 0.941,
-      "avg": 0.759
-    },
-    "mascaret_target": {
-      "tcga": 0.893,
-      "camelyon": 0.907,
-      "tolkach_esca": 0.972,
-      "avg": 0.924
-    },
-    "virchow2_base": {
-      "avg": 0.858
-    },
-    "virchow2_target": {
-      "avg": 0.918
-    }
-  },
-  "points": [
-    {
-      "model": "waiv_genMASK_c50_ms500_openmidnight_s2_t900_438672_s0000050",
-      "adapter_rel_l2_delta": {
-        "camelyon": 0.06135381,
-        "tolkach_esca": 0.01984267,
-        "tcga": 0.03784909
-      },
-      "datasets": {
-        "camelyon": {
-          "robustness_index": 0.29402527480744783,
-          "balanced_accuracy": 0.9658333333333333,
-          "k_opt": 11,
-          "generalization_index": 0.9906174588987253,
-          "confounder_insensitivity": 0.018828347400489436,
-          "prediction_performance": 0.956969696969697,
-          "ID_performance": 0.957135657630089,
-          "OOD_performance": 0.9481552929828793
-        },
-        "tolkach_esca": {
-          "robustness_index": 0.8844883648419056,
-          "balanced_accuracy": 0.9557777777777777,
-          "k_opt": 46,
-          "generalization_index": 1.0098829811425731,
-          "confounder_insensitivity": 0.554690169776673,
-          "prediction_performance": 0.9351159420289855,
-          "ID_performance": 0.9318302158165316,
-          "OOD_performance": 0.941039476267526
-        },
-        "tcga": {
-          "robustness_index": 0.8087960069956179,
-          "balanced_accuracy": 0.9245124113475179,
-          "k_opt": 61,
-          "generalization_index": 1.0069926440169015,
-          "confounder_insensitivity": 0.4974019397370145,
-          "prediction_performance": 0.8962026217881642,
-          "ID_performance": 0.8941257527759104,
-          "OOD_performance": 0.9003780558714164
-        }
-      },
-      "avg_robustness_index": 0.6624365488816572,
-      "avg_balanced_accuracy": 0.948707840819543,
-      "step": 50,
-      "checkpoint": "/data/ryan.kim/waiv_runs/genMASK-c50-ms500-openmidnight-s2-t900-438672/step_0000050",
-      "probe": {
-        "train.cross_scanner.separation": 0.000521281361579895,
-        "train.cross_scanner.matched": 0.9998932844400406,
-        "train.cross_scanner.random": 0.9993720030784607,
-        "train.cross_scanner.top1": 0.752109375,
-        "train.cross_stain.separation": 0.0004398277070787218,
-        "train.cross_stain.matched": 0.9997829243871901,
-        "train.cross_stain.random": 0.9993430966801113,
-        "train.cross_stain.top1": 0.48635416666666664,
-        "train.within_condition_random": 0.9993962526321412,
-        "heldout.cross_scanner.separation": 0.0005480606261998006,
-        "heldout.cross_scanner.matched": 0.9998940417211349,
-        "heldout.cross_scanner.random": 0.9993459810949352,
-        "heldout.cross_scanner.top1": 0.7392444349315068,
-        "heldout.cross_stain.separation": 0.0005094837724116811,
-        "heldout.cross_stain.matched": 0.9998166250903704,
-        "heldout.cross_stain.random": 0.9993071413179587,
-        "heldout.cross_stain.top1": 0.5176809210526315,
-        "heldout.within_condition_random": 0.9993571987966212
-      },
-      "eval_seconds": 18959.9,
-      "train_metrics": {
-        "step": 50,
-        "loss_cls": 4.723287105560303,
-        "top1_cls": 0.03333333507180214,
-        "weight_cls": 0.5,
-        "negatives_per_anchor": 899.0,
-        "n_rows": 1800.0,
-        "n_cond": 2.0,
-        "n_tiles": 900.0,
-        "loss_mean": 0.31233638525009155,
-        "top1_mean": 0.9733333587646484,
-        "weight_mean": 0.5,
-        "loss": 2.5178117752075195,
-        "top1": 0.5033333469182253,
-        "n_heads": 2.0,
-        "split_input_rel_distance": 1.9103232622146606,
-        "split_input_cosine": -0.12291879206895828,
-        "heldout_loss": 2.873713743686676,
-        "heldout_top1": 0.41397223249077797
-      }
-    },
-    {
-      "model": "waiv_genMASK_c50_ms500_openmidnight_s2_t900_438672_s0000100",
-      "adapter_rel_l2_delta": {
-        "camelyon": 0.1116684,
-        "tolkach_esca": 0.03930933,
-        "tcga": 0.06849402
-      },
-      "datasets": {
-        "camelyon": {
-          "robustness_index": 0.7362077922077922,
-          "balanced_accuracy": 0.9715686274509804,
-          "k_opt": 11,
-          "generalization_index": 0.9992648104753599,
-          "confounder_insensitivity": 0.1092765974275065,
-          "prediction_performance": 0.962277183600713,
-          "ID_performance": 0.9623468812718123,
-          "OOD_performance": 0.961639373925631
-        },
-        "tolkach_esca": {
-          "robustness_index": 0.9542504442973133,
-          "balanced_accuracy": 0.9665555555555555,
-          "k_opt": 46,
-          "generalization_index": 1.0086772238627593,
-          "confounder_insensitivity": 1.2544599098216036,
-          "prediction_performance": 0.9473574879227054,
-          "ID_performance": 0.9428053322877867,
-          "OOD_performance": 0.950986265215051
-        },
-        "tcga": {
-          "robustness_index": 0.8497310220339728,
-          "balanced_accuracy": 0.9312056737588654,
-          "k_opt": 61,
-          "generalization_index": 1.013121271335831,
-          "confounder_insensitivity": 0.6597069992898823,
-          "prediction_performance": 0.8989623299616324,
-          "ID_performance": 0.8942981220476088,
-          "OOD_performance": 0.9060324503621195
-        }
-      },
-      "avg_robustness_index": 0.8467297528463594,
-      "avg_balanced_accuracy": 0.9564432855884671,
-      "step": 100,
-      "checkpoint": "/data/ryan.kim/waiv_runs/genMASK-c50-ms500-openmidnight-s2-t900-438672/step_0000100",
-      "probe": {
-        "train.cross_scanner.separation": 0.0004972052574157714,
-        "train.cross_scanner.matched": 0.999962602853775,
-        "train.cross_scanner.random": 0.9994653975963592,
-        "train.cross_scanner.top1": 0.91453125,
-        "train.cross_stain.separation": 0.0004331326484680176,
-        "train.cross_stain.matched": 0.999888620906406,
-        "train.cross_stain.random": 0.999455488257938,
-        "train.cross_stain.top1": 0.6504861111111111,
-        "train.within_condition_random": 0.9994706416130066,
-        "heldout.cross_scanner.separation": 0.0005227670277634713,
-        "heldout.cross_scanner.matched": 0.9999576415101142,
-        "heldout.cross_scanner.random": 0.9994348744823508,
-        "heldout.cross_scanner.top1": 0.8951198630136986,
-        "heldout.cross_stain.separation": 0.000481627140825952,
-        "heldout.cross_stain.matched": 0.9999014682937086,
-        "heldout.cross_stain.random": 0.9994198411528827,
-        "heldout.cross_stain.top1": 0.66890533625731,
-        "heldout.within_condition_random": 0.9994370036008882
-      },
-      "eval_seconds": 5102.8,
-      "train_metrics": {
-        "step": 100,
-        "loss_cls": 4.2960381507873535,
-        "top1_cls": 0.07333333790302277,
-        "weight_cls": 0.5,
-        "negatives_per_anchor": 899.0,
-        "n_rows": 1800.0,
-        "n_cond": 2.0,
-        "n_tiles": 900.0,
-        "loss_mean": 0.8268017768859863,
-        "top1_mean": 0.8866667151451111,
-        "weight_mean": 0.5,
-        "loss": 2.56141996383667,
-        "top1": 0.4800000265240669,
-        "n_heads": 2.0,
-        "split_input_rel_distance": 1.909881830215454,
-        "split_input_cosine": -0.09640105068683624,
-        "heldout_loss": 1.8407508969306945,
-        "heldout_top1": 0.659666682779789
-      }
-    },
-    {
-      "model": "waiv_genMASK_c50_ms500_openmidnight_s2_t900_438672_s0000150",
-      "adapter_rel_l2_delta": {
-        "camelyon": 0.1003916,
-        "tolkach_esca": 0.0329724,
-        "tcga": 0.06768148
-      },
-      "datasets": {
-        "camelyon": {
-          "robustness_index": 0.8590614569016297,
-          "balanced_accuracy": 0.9716666666666667,
-          "k_opt": 11,
-          "generalization_index": 1.0044632696203488,
-          "confounder_insensitivity": 0.2534702244962938,
-          "prediction_performance": 0.960770944741533,
-          "ID_performance": 0.9599045932645526,
-          "OOD_performance": 0.9641889062741037
-        },
-        "tolkach_esca": {
-          "robustness_index": 0.961135959008207,
-          "balanced_accuracy": 0.9660000000000001,
-          "k_opt": 46,
-          "generalization_index": 1.0107858304832258,
-          "confounder_insensitivity": 1.6367747277243487,
-          "prediction_performance": 0.943572463768116,
-          "ID_performance": 0.9372969874530285,
-          "OOD_performance": 0.947406513872135
-        },
-        "tcga": {
-          "robustness_index": 0.8473497201017163,
-          "balanced_accuracy": 0.9227659574468086,
-          "k_opt": 61,
-          "generalization_index": 1.0103340710417517,
-          "confounder_insensitivity": 0.773014100780525,
-          "prediction_performance": 0.88060617951401,
-          "ID_performance": 0.8766563708095831,
-          "OOD_performance": 0.8857158000247337
-        }
-      },
-      "avg_robustness_index": 0.8891823786705176,
-      "avg_balanced_accuracy": 0.9534775413711585,
-      "step": 150,
-      "checkpoint": "/data/ryan.kim/waiv_runs/genMASK-c50-ms500-openmidnight-s2-t900-438672/step_0000150",
-      "probe": {
-        "train.cross_scanner.separation": 0.0005248391628265381,
-        "train.cross_scanner.matched": 0.9999757087230683,
-        "train.cross_scanner.random": 0.9994508695602417,
-        "train.cross_scanner.top1": 0.97203125,
-        "train.cross_stain.separation": 0.00046601136525472003,
-        "train.cross_stain.matched": 0.9999127949608697,
-        "train.cross_stain.random": 0.999446783595615,
-        "train.cross_stain.top1": 0.7904513888888889,
-        "train.within_condition_random": 0.9994528937339783,
-        "heldout.cross_scanner.separation": 0.0005484950052548762,
-        "heldout.cross_scanner.matched": 0.9999696081631804,
-        "heldout.cross_scanner.random": 0.9994211131579256,
-        "heldout.cross_scanner.top1": 0.9548908390410958,
-        "heldout.cross_stain.separation": 0.0004999975711978667,
-        "heldout.cross_stain.matched": 0.9999169313419632,
-        "heldout.cross_stain.random": 0.9994169337707653,
-        "heldout.cross_stain.top1": 0.7951160453216374,
-        "heldout.within_condition_random": 0.9994239719902597
-      },
-      "eval_seconds": 4516.4,
-      "train_metrics": {
-        "step": 150,
-        "loss_cls": 1.152949333190918,
-        "top1_cls": 0.6322222352027893,
-        "weight_cls": 0.5,
-        "negatives_per_anchor": 899.0,
-        "n_rows": 1800.0,
-        "n_cond": 2.0,
-        "n_tiles": 900.0,
-        "loss_mean": 0.02266651578247547,
-        "top1_mean": 0.9988889098167419,
-        "weight_mean": 0.5,
-        "loss": 0.5878079533576965,
-        "top1": 0.8155555725097656,
-        "n_heads": 2.0,
-        "split_input_rel_distance": 1.90652596950531,
-        "split_input_cosine": -0.06583566218614578,
-        "heldout_loss": 1.441209152340889,
-        "heldout_top1": 0.7740555718541146
-      }
-    },
-    {
-      "model": "waiv_genMASK_c50_ms500_openmidnight_s2_t900_438672_s0000200",
-      "adapter_rel_l2_delta": {
-        "camelyon": 0.0830723,
-        "tolkach_esca": 0.03013019,
-        "tcga": 0.05842631
-      },
-      "datasets": {
-        "camelyon": {
-          "robustness_index": 0.8603496360791782,
-          "balanced_accuracy": 0.9691666666666666,
-          "k_opt": 11,
-          "generalization_index": 1.0085976580778504,
-          "confounder_insensitivity": 0.30677847658979734,
-          "prediction_performance": 0.9540998217468806,
-          "ID_performance": 0.9521779641276497,
-          "OOD_performance": 0.9603644646924829
-        },
-        "tolkach_esca": {
-          "robustness_index": 0.9621059224719883,
-          "balanced_accuracy": 0.9653333333333335,
-          "k_opt": 46,
-          "generalization_index": 1.0077102227850199,
-          "confounder_insensitivity": 1.7642015864113452,
-          "prediction_performance": 0.9391594202898551,
-          "ID_performance": 0.9345605320086532,
-          "OOD_performance": 0.9417662019165266
-        },
-        "tcga": {
-          "robustness_index": 0.8355797485518814,
-          "balanced_accuracy": 0.9186790780141842,
-          "k_opt": 61,
-          "generalization_index": 1.0098498166386747,
-          "confounder_insensitivity": 0.796373887415436,
-          "prediction_performance": 0.8671421927682827,
-          "ID_performance": 0.8633721484248441,
-          "OOD_performance": 0.8718762057777674
-        }
-      },
-      "avg_robustness_index": 0.8860117690343493,
-      "avg_balanced_accuracy": 0.9510596926713948,
-      "step": 200,
-      "checkpoint": "/data/ryan.kim/waiv_runs/genMASK-c50-ms500-openmidnight-s2-t900-438672/step_0000200",
-      "probe": {
-        "train.cross_scanner.separation": 0.000561721920967102,
-        "train.cross_scanner.matched": 0.9999778664112091,
-        "train.cross_scanner.random": 0.999416144490242,
-        "train.cross_scanner.top1": 0.98859375,
-        "train.cross_stain.separation": 0.0005025479528639052,
-        "train.cross_stain.matched": 0.9999160668585035,
-        "train.cross_stain.random": 0.9994135189056397,
-        "train.cross_stain.top1": 0.8607291666666667,
-        "train.within_condition_random": 0.999417861700058,
-        "heldout.cross_scanner.separation": 0.0005812253037544146,
-        "heldout.cross_scanner.matched": 0.9999714012015356,
-        "heldout.cross_scanner.random": 0.9993901758977811,
-        "heldout.cross_scanner.top1": 0.9780072773972602,
-        "heldout.cross_stain.separation": 0.0005306183246144077,
-        "heldout.cross_stain.matched": 0.9999188669243751,
-        "heldout.cross_stain.random": 0.9993882485997607,
-        "heldout.cross_stain.top1": 0.8625274122807017,
-        "heldout.within_condition_random": 0.9993932029096092
-      },
-      "eval_seconds": 2198.1,
-      "train_metrics": {
-        "step": 200,
-        "loss_cls": 0.9190048575401306,
-        "top1_cls": 0.695555567741394,
-        "weight_cls": 0.5,
-        "negatives_per_anchor": 899.0,
-        "n_rows": 1800.0,
-        "n_cond": 2.0,
-        "n_tiles": 900.0,
-        "loss_mean": 0.014111900702118874,
-        "top1_mean": 0.9994444847106934,
-        "weight_mean": 0.5,
-        "loss": 0.4665583670139313,
-        "top1": 0.8475000262260437,
-        "n_heads": 2.0,
-        "split_input_rel_distance": 1.902852177619934,
-        "split_input_cosine": -0.044305868446826935,
-        "heldout_loss": 1.2631883502006531,
-        "heldout_top1": 0.8060833513736725
-      }
-    },
-    {
-      "model": "waiv_genMASK_c50_ms500_openmidnight_s2_t900_438672_s0000250",
-      "adapter_rel_l2_delta": {
-        "camelyon": 0.08021808,
-        "tolkach_esca": 0.03456734,
-        "tcga": 0.05260206
-      },
-      "datasets": {
-        "camelyon": {
-          "robustness_index": 0.8684781367736043,
-          "balanced_accuracy": 0.9674019607843137,
-          "k_opt": 11,
-          "generalization_index": 1.00733173196086,
-          "confounder_insensitivity": 0.3286912036379139,
-          "prediction_performance": 0.9539795008912656,
-          "ID_performance": 0.9522523802756857,
-          "OOD_performance": 0.9592340394869578
-        },
-        "tolkach_esca": {
-          "robustness_index": 0.9631153207964601,
-          "balanced_accuracy": 0.9634444444444444,
-          "k_opt": 46,
-          "generalization_index": 1.0064429705396893,
-          "confounder_insensitivity": 1.8066464642355957,
-          "prediction_performance": 0.9387753623188406,
-          "ID_performance": 0.9348980048404482,
-          "OOD_performance": 0.9409215251432494
-        },
-        "tcga": {
-          "robustness_index": 0.8372437935206859,
-          "balanced_accuracy": 0.9204166666666664,
-          "k_opt": 61,
-          "generalization_index": 1.0102595304806377,
-          "confounder_insensitivity": 0.8021165598137541,
-          "prediction_performance": 0.8678554819207069,
-          "ID_performance": 0.8639104473155644,
-          "OOD_performance": 0.8727737628823398
-        }
-      },
-      "avg_robustness_index": 0.8896124170302501,
-      "avg_balanced_accuracy": 0.9504210239651415,
-      "step": 250,
-      "checkpoint": "/data/ryan.kim/waiv_runs/genMASK-c50-ms500-openmidnight-s2-t900-438672/step_0000250",
-      "probe": {
-        "train.cross_scanner.separation": 0.000628265142440796,
-        "train.cross_scanner.matched": 0.9999768042564392,
-        "train.cross_scanner.random": 0.9993485391139985,
-        "train.cross_scanner.top1": 0.99265625,
-        "train.cross_stain.separation": 0.0005653095245361328,
-        "train.cross_stain.matched": 0.9999114685588413,
-        "train.cross_stain.random": 0.9993461590343051,
-        "train.cross_stain.top1": 0.8754166666666666,
-        "train.within_condition_random": 0.9993501269817352,
-        "heldout.cross_scanner.separation": 0.0006491329571972154,
-        "heldout.cross_scanner.matched": 0.999969627759228,
-        "heldout.cross_scanner.random": 0.9993204948020308,
-        "heldout.cross_scanner.top1": 0.9841609589041096,
-        "heldout.cross_stain.separation": 0.000592950143312153,
-        "heldout.cross_stain.matched": 0.9999126506130598,
-        "heldout.cross_stain.random": 0.9993197004697476,
-        "heldout.cross_stain.top1": 0.8781524122807017,
-        "heldout.within_condition_random": 0.9993243508222627
-      },
-      "eval_seconds": 2211.5,
-      "train_metrics": {
-        "step": 250,
-        "loss_cls": 0.23604129254817963,
-        "top1_cls": 0.9316666722297668,
-        "weight_cls": 0.5,
-        "negatives_per_anchor": 899.0,
-        "n_rows": 1800.0,
-        "n_cond": 2.0,
-        "n_tiles": 900.0,
-        "loss_mean": 0.005885029677301645,
-        "top1_mean": 1.0,
-        "weight_mean": 0.5,
-        "loss": 0.12096316367387772,
-        "top1": 0.9658333361148834,
-        "n_heads": 2.0,
-        "split_input_rel_distance": 1.9015450477600098,
-        "split_input_cosine": -0.03455965220928192,
-        "heldout_loss": 1.1432920694351196,
-        "heldout_top1": 0.822000016272068
-      }
-    },
-    {
-      "model": "waiv_genMASK_c50_ms500_openmidnight_s2_t900_438672_s0000300",
-      "adapter_rel_l2_delta": {
-        "camelyon": 0.07439523,
-        "tolkach_esca": 0.03673597,
-        "tcga": 0.04736481
-      },
-      "datasets": {
-        "camelyon": {
-          "robustness_index": 0.8698659047371669,
-          "balanced_accuracy": 0.9682352941176471,
-          "k_opt": 11,
-          "generalization_index": 1.005579320505482,
-          "confounder_insensitivity": 0.3321223130487435,
-          "prediction_performance": 0.9537388591800356,
-          "ID_performance": 0.952414026464355,
-          "OOD_performance": 0.9577278495719164
-        },
-        "tolkach_esca": {
-          "robustness_index": 0.9639718771215858,
-          "balanced_accuracy": 0.9645555555555555,
-          "k_opt": 46,
-          "generalization_index": 1.0064427285687858,
-          "confounder_insensitivity": 1.8110104699955187,
-          "prediction_performance": 0.9401086956521739,
-          "ID_performance": 0.9362226537568408,
-          "OOD_performance": 0.9422544821949445
-        },
-        "tcga": {
-          "robustness_index": 0.8352211433041281,
-          "balanced_accuracy": 0.9199822695035461,
-          "k_opt": 61,
-          "generalization_index": 1.010830803863689,
-          "confounder_insensitivity": 0.7948317984262616,
-          "prediction_performance": 0.8673186257411929,
-          "ID_performance": 0.8631785070926741,
-          "OOD_performance": 0.8725274242023467
-        }
-      },
-      "avg_robustness_index": 0.8896863083876269,
-      "avg_balanced_accuracy": 0.9509243730589162,
-      "step": 300,
-      "checkpoint": "/data/ryan.kim/waiv_runs/genMASK-c50-ms500-openmidnight-s2-t900-438672/step_0000300",
-      "probe": {
-        "train.cross_scanner.separation": 0.0006830227375030517,
-        "train.cross_scanner.matched": 0.9999776357412338,
-        "train.cross_scanner.random": 0.9992946130037308,
-        "train.cross_scanner.top1": 0.9953125,
-        "train.cross_stain.separation": 0.0006196782324049207,
-        "train.cross_stain.matched": 0.9999123795827229,
-        "train.cross_stain.random": 0.9992927013503181,
-        "train.cross_stain.top1": 0.8923958333333334,
-        "train.within_condition_random": 0.9992960214614868,
-        "heldout.cross_scanner.separation": 0.0006946268146985197,
-        "heldout.cross_scanner.matched": 0.9999706573682289,
-        "heldout.cross_scanner.random": 0.9992760305535303,
-        "heldout.cross_scanner.top1": 0.9886023116438356,
-        "heldout.cross_stain.separation": 0.0006400761548538655,
-        "heldout.cross_stain.matched": 0.9999137973924827,
-        "heldout.cross_stain.random": 0.9992737212376288,
-        "heldout.cross_stain.top1": 0.8931149488304093,
-        "heldout.within_condition_random": 0.9992785003127121
-      },
-      "eval_seconds": 2490.6,
-      "train_metrics": {
-        "step": 300,
-        "loss_cls": 1.4541085958480835,
-        "top1_cls": 0.554444432258606,
-        "weight_cls": 0.5,
-        "negatives_per_anchor": 899.0,
-        "n_rows": 1800.0,
-        "n_cond": 2.0,
-        "n_tiles": 900.0,
-        "loss_mean": 0.022152811288833618,
-        "top1_mean": 0.9983333349227905,
-        "weight_mean": 0.5,
-        "loss": 0.7381306886672974,
-        "top1": 0.7763888835906982,
-        "n_heads": 2.0,
-        "split_input_rel_distance": 1.9002515077590942,
-        "split_input_cosine": -0.02878390997648239,
-        "heldout_loss": 1.0337130695581436,
-        "heldout_top1": 0.8357361286878586
-      }
-    },
-    {
-      "model": "waiv_genMASK_c50_ms500_openmidnight_s2_t900_438672_s0000350",
-      "adapter_rel_l2_delta": {
-        "camelyon": 0.073182,
-        "tolkach_esca": 0.04284,
-        "tcga": 0.04353784
-      },
-      "datasets": {
-        "camelyon": {
-          "robustness_index": 0.8651014918072879,
-          "balanced_accuracy": 0.9663725490196078,
-          "k_opt": 11,
-          "generalization_index": 1.0063352088816737,
-          "confounder_insensitivity": 0.3281407213627055,
-          "prediction_performance": 0.952517825311943,
-          "ID_performance": 0.9510292498727494,
-          "OOD_performance": 0.9570542188232748
-        },
-        "tolkach_esca": {
-          "robustness_index": 0.962552298049072,
-          "balanced_accuracy": 0.963111111111111,
-          "k_opt": 46,
-          "generalization_index": 1.0068131937015685,
-          "confounder_insensitivity": 1.8230095736846412,
-          "prediction_performance": 0.9374589371980676,
-          "ID_performance": 0.933352426151706,
-          "OOD_performance": 0.9397115370229064
-        },
-        "tcga": {
-          "robustness_index": 0.8314431861604742,
-          "balanced_accuracy": 0.9186702127659576,
-          "k_opt": 61,
-          "generalization_index": 1.0117116206622394,
-          "confounder_insensitivity": 0.7929143300256868,
-          "prediction_performance": 0.8645749040809209,
-          "ID_performance": 0.8601199549530535,
-          "OOD_performance": 0.8701933535894862
-        }
-      },
-      "avg_robustness_index": 0.8863656586722781,
-      "avg_balanced_accuracy": 0.9493846242988923,
-      "step": 350,
-      "checkpoint": "/data/ryan.kim/waiv_runs/genMASK-c50-ms500-openmidnight-s2-t900-438672/step_0000350",
-      "probe": {
-        "train.cross_scanner.separation": 0.0007353496551513672,
-        "train.cross_scanner.matched": 0.9999756443500519,
-        "train.cross_scanner.random": 0.9992402946949005,
-        "train.cross_scanner.top1": 0.9950390625,
-        "train.cross_stain.separation": 0.0006662252214219835,
-        "train.cross_stain.matched": 0.9999043149418301,
-        "train.cross_stain.random": 0.9992380897204082,
-        "train.cross_stain.top1": 0.8957118055555555,
-        "train.within_condition_random": 0.9992416834831238,
-        "heldout.cross_scanner.separation": 0.0007469172347081851,
-        "heldout.cross_scanner.matched": 0.9999676624389544,
-        "heldout.cross_scanner.random": 0.9992207452042462,
-        "heldout.cross_scanner.top1": 0.9891374143835616,
-        "heldout.cross_stain.separation": 0.0006879756325169614,
-        "heldout.cross_stain.matched": 0.9999062025059037,
-        "heldout.cross_stain.random": 0.9992182268733867,
-        "heldout.cross_stain.top1": 0.8955135233918129,
-        "heldout.within_condition_random": 0.9992232860588446
-      },
-      "eval_seconds": 2452.3,
-      "train_metrics": {
-        "step": 350,
-        "loss_cls": 1.4318788051605225,
-        "top1_cls": 0.5766666531562805,
-        "weight_cls": 0.5,
-        "negatives_per_anchor": 899.0,
-        "n_rows": 1800.0,
-        "n_cond": 2.0,
-        "n_tiles": 900.0,
-        "loss_mean": 0.028840575367212296,
-        "top1_mean": 1.0,
-        "weight_mean": 0.5,
-        "loss": 0.730359673500061,
-        "top1": 0.7883333265781403,
-        "n_heads": 2.0,
-        "split_input_rel_distance": 1.8998087644577026,
-        "split_input_cosine": -0.030021917074918747,
-        "heldout_loss": 0.9633975774049759,
-        "heldout_top1": 0.8410833522677421
-      }
-    }
-  ]
-}

seed2/training_config.json

@@ -1,61 +0,0 @@
-{
-  "packed_dir": "/data/plism/repacked",
-  "out_dir": "/admin/home/ryan.kim/waiv/runs/genMASK-c50-ms500-openmidnight-s2-t900-438672",
-  "lr": 0.0001,
-  "weight_decay": 0.05,
-  "warmup_steps": 200,
-  "max_steps": 500,
-  "temperature": 0.07,
-  "symmetric": false,
-  "grad_accum": 1,
-  "grad_clip": 1.0,
-  "n_groups": 4,
-  "group_size": 64,
-  "grid": true,
-  "grid_conditions": 2,
-  "grid_tiles": 900,
-  "grid_forward_chunk": 0,
-  "activation_offload": true,
-  "num_workers": 10,
-  "seed": 2,
-  "amp_dtype": "bfloat16",
-  "ckpt_every": 50,
-  "log_every": 20,
-  "eval_every": 50,
-  "eval_batches": 20,
-  "ckpt_schedule": null,
-  "resume_from": null,
-  "grid_blocked_loss": false,
-  "grid_pair_block": 8,
-  "eval_heldout": true,
-  "retention_kl_weight": 0.0,
-  "retention_kl_temperature": 0.07,
-  "split_heads": true,
-  "cls_weight": 0.5,
-  "mean_weight": 0.5,
-  "pool_head": "gem",
-  "min_tissue_frac": 0.0,
-  "cores_per_batch": 0,
-  "same_core_logit_bias": null,
-  "center_embeddings": false,
-  "same_core_logit_bias_cls": 3.0,
-  "same_core_logit_bias_mean": "-Infinity",
-  "mask_same_core": true,
-  "core_labels_path": "/admin/home/ryan.kim/waiv/runs/.plism_core_labels.npy",
-  "mask_sim_thresh": 0.0,
-  "sim_emb_path": "/admin/home/ryan.kim/waiv/runs/.plism_ref_emb.npy",
-  "encoder": {
-    "backbone": "SophontAI/OpenMidnight",
-    "use_lora": true,
-    "lora_rank": 32,
-    "lora_alpha": 64,
-    "pooling": "clsmean",
-    "proj_out_dim": 512,
-    "grad_checkpointing": true,
-    "split_heads": [
-      "cls",
-      "mean"
-    ],
-    "pool_head": "gem"
-  }
-}