Release development-selected OA-CXR weights and attributed reproduction masks

LICENSE

@@ -0,0 +1,22 @@
+OA-CXR author contributions and upstream terms
+
+Copyright (c) 2026 hekaiwang.
+The author's original contributions to the released model are made available
+under the MIT terms in LICENSE-CODE. This grant does not relicense third-party
+pretrained material, datasets or annotations.
+
+The image model was initialized from the official TorchXRayVision CheXpert-only
+DenseNet121 checkpoint. TorchXRayVision identifies its main xrv.models library
+as Apache-2.0. No separate weight-specific license accompanied that checkpoint;
+this provenance limitation is retained. Consult the upstream rights and
+THIRD_PARTY_NOTICES.md rather than treating this bundle as wholly original MIT
+material. Original initialization weights are not mirrored here.
+
+reproduction/kermany-v7-masks.tar.gz is separately CC BY 4.0, attributed to V7
+and CloudFactory. It is an adaptation into rasterized two-lung PNG masks of
+human-assisted and reviewed lung polygons. It contains no radiographs.
+https://github.com/v7labs/covid-19-xray-dataset
+https://creativecommons.org/licenses/by/4.0/
+
+The model is supplied for research, without clinical validation or warranty.
+MAIRA-2 weights and raw medical images are not included.

LICENSE-CODE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 hekaiwang
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

README.md

@@ -0,0 +1,61 @@
+---
+language:
+- en
+license: other
+license_name: oa-cxr-contributions-and-upstream-terms
+license_link: LICENSE
+library_name: pytorch
+tags:
+- chest-xray
+- anatomical-retention
+- reproducibility
+- medical-imaging
+---
+# OA-CXR — default paper model
+
+Code, installation, training and evaluation: [hekaiwang/OA-CXR](https://github.com/hekaiwang/OA-CXR).
+
+This is the original development-selected OA-CXR model: seed 17, image epoch 12,
+direct linear L1 readout epoch 27. These are not newly selected release-test weights.
+
+## Contents
+
+- `main/image_state.pt`: 29,573,522 bytes; pure tensor image-model state.
+- `main/readout_state.pt`: 2,358,434 bytes; selected readout with fit-only normalization.
+- `main/config.json`: architecture, tensor SHA256, implementation bindings and provenance.
+- `main/training_history.json`: original aggregate epoch histories and development metrics.
+- `reproduction/kermany-v7-masks.tar.gz`: optional CC BY 4.0 derived annotation PNGs
+  required to reconstruct the exact Kermany training/evaluation labels. No radiographs.
+
+Install the linked GitHub project and run `python scripts/public/download.py --output weights`.
+Then `python scripts/public/predict.py --weights weights/main --image /path/image.png --output runs/example`.
+The repository pins an immutable revision and checks hashes; no Hugging Face login is
+required to use this public repository.
+
+## Training and intended interpretation
+
+The image model was initialized from XRV's CheXpert-only DenseNet121 and trained
+on 19,086 COVIDQU/Kermany source images for 13 epochs (975 updates). Three readout
+candidates were independently trained for 30 epochs (10,920 updates each). Selection
+used development data, not release test results. The encoder is frozen in readout
+training. Full reconstruction/training commands and frozen split manifests are in
+GitHub. Training archives and source images are obtained from their providers.
+
+Scores estimate relative retention of basal, peripheral and whole-lung annotation
+regions for effusion, pneumothorax and consolidation queries. They do not diagnose
+those conditions. An uncropped source is defined to have retention one even if its
+clinical acquisition was incomplete. The model is not clinically validated.
+
+All original experiments used seed 17; reported bootstrap intervals do not measure
+training-seed variation. Some sources were historically observed; patient independence
+and foundation-pretraining non-overlap are not fully established. NIH evaluation uses
+pseudo masks. CUDA BF16 and CPU FP32 may produce small numerical differences.
+
+## Attribution and terms
+
+See [LICENSE](LICENSE) and [THIRD_PARTY_NOTICES.md](THIRD_PARTY_NOTICES.md).
+Author-owned contributions and third-party pretrained material have distinct scope.
+XRV's main library declares Apache-2.0; a separate checkpoint-specific license was
+not supplied upstream. No blanket MIT claim for all upstream weights/data is made.
+V7/CloudFactory annotation derivatives retain CC BY 4.0 and are not relicensed as MIT.
+MAIRA-2 and other restricted third-party weights are not redistributed.

THIRD_PARTY_NOTICES.md

@@ -0,0 +1,65 @@
+# Code, weights and data have separate provenance
+
+## Project code
+
+Original OA-CXR code is distributed under the repository's existing MIT license,
+copyright 2026 hekaiwang. Dependencies retain their own licenses; installing them
+does not relicense them as MIT. No MAIRA-2 implementation or checkpoint is copied
+into this release.
+
+## OA-CXR weights and TorchXRayVision initialization
+
+The public model consists of the author's fine-tuned image model and trained
+readout. It starts from TorchXRayVision's official CheXpert-only DenseNet121
+checkpoint, then trains on COVID-QU-Ex and Kermany/V7. The released tensor files
+are the exact development-selected OA artifacts, not a renamed baseline.
+
+Upstream: [TorchXRayVision](https://github.com/mlmed/torchxrayvision).
+The upstream [license declaration](https://github.com/mlmed/torchxrayvision/blob/main/LICENSE)
+identifies its main `xrv.models` library as Apache-2.0 and distinguishes separately
+licensed `baseline_models`. OA uses the main `xrv.models.DenseNet`.
+The official release checkpoint did not include a separate weight-specific
+license document; we preserve that provenance limitation and do not represent
+the entire pretrained model or training data as independently authored MIT work.
+
+The Hugging Face model card and weight-license notice distinguish author-owned
+contributions, upstream rights and intended research use. The original XRV
+checkpoint is fetched directly from upstream by `download_initial.py`, after
+verification against a fixed SHA, rather than mirrored here.
+
+Reference: Cohen et al., *TorchXRayVision: A library of chest X-ray datasets and
+models*, [arXiv:2111.00595](https://arxiv.org/abs/2111.00595).
+
+## V7-derived annotation masks
+
+The optional `reproduction/kermany-v7-masks.tar.gz` contains rasterized masks
+derived from V7/CloudFactory human-assisted and human-reviewed lung polygons.
+The [upstream attribution/license section](https://github.com/v7labs/covid-19-xray-dataset#data-sources-and-licenses)
+links [Creative Commons Attribution 4.0 International](https://creativecommons.org/licenses/by/4.0/).
+We retain that license, credit V7 and CloudFactory, and identify the adaptation:
+polygons rendered as two image-coordinate lung PNG masks and associated with
+the frozen Kermany cohort. The mask archive does not contain radiographs or
+the original metadata's image-access URLs.
+
+Source radiographs: Kermany, Daniel; Zhang, Kang; Goldbaum, Michael (2018),
+*Labeled Optical Coherence Tomography (OCT) and Chest X-Ray Images for
+Classification*, Mendeley Data v2, [doi:10.17632/rscbjbr9sj.2](https://doi.org/10.17632/rscbjbr9sj.2).
+
+## Other datasets
+
+COVID-QU-Ex images and annotations are obtained directly from the version-7
+provider. NIH images, CheXmask annotations and Shenzhen images/masks are also
+obtained under their individual provider terms. The repository supplies fixed
+membership and reconstruction code, not a blanket redistribution license for
+those datasets. See [DATA.md](docs/DATA.md) for exact sources and versions.
+
+## MAIRA-2 and report experiments
+
+The historical report-generation experiments used frozen MAIRA-2 revision
+`795a2b1cd4a310624b4e3d14b5a23e41fd273deb`, greedy BF16 generation with a 512-token
+limit. MAIRA-2 is governed by the Microsoft Research License Terms (MSRLA),
+which prohibit redistribution of the supplied model. Users seeking to repeat
+the report-generation stage must obtain it through
+[Microsoft's official model page](https://huggingface.co/microsoft/maira-2)
+and comply with those terms. The OA image training and inference commands do
+not require MAIRA-2.

main/config.json

@@ -0,0 +1,65 @@
+{
+  "schema": "oa-cxr-public-model-v1",
+  "findings": [
+    "pleural_effusion",
+    "pneumothorax",
+    "consolidation"
+  ],
+  "readout_config": {
+    "input_dim": 1253,
+    "width": 256,
+    "inner_width": 256,
+    "blocks": 2,
+    "dropout": 0.0,
+    "activation": "linear_clamp",
+    "residual_mode": "none",
+    "minimum_scale": 1e-06,
+    "retained_head": null
+  },
+  "selection": {
+    "candidate": "direct_linear_l1",
+    "candidate_order": 0,
+    "best_epoch": 27,
+    "dev_mae": 0.028647494974946227,
+    "original_fp32_control_dev_mae": 0.0503299951180114,
+    "improved_vs_original_fp32_control": true,
+    "recommendation": "direct_linear_l1",
+    "selection_used_heldout": false,
+    "selection_used_original512_secondary": false,
+    "candidate_status_sha256": {
+      "direct_linear_l1": "e0152bd60f17da8cc47ed10993d24a087def65c910e9e66f9d8848bbff9b9e00",
+      "direct_linear_huber002": "d2c47fc1505a112e29ea54f64eace122e27526ad37492283911b90810c5dd3b0",
+      "old_logit_residual_l1": "a1d1a9a2a96a84339417dfd3055e6250d8cb50364144abc6f84894841cbf7d3f"
+    }
+  },
+  "provenance": {
+    "seed": 17,
+    "image_epoch": 12,
+    "readout_epoch": 27,
+    "image_optimizer_steps": 975,
+    "fit_source_images": 19086,
+    "dev_source_images": 2609,
+    "image_original_sha256": "0e65a604fb32428c74b61add88bd92f445b08966fe64d1c38d751a714b53f895",
+    "readout_original_sha256": "03b6e4c094d68e764a0340f1c4ff13c7b75bfd5ed84372767396c77f89fe195b",
+    "selection": "dev only; default paper model; not Robust"
+  },
+  "preprocessing": "grayscale; aspect-preserving bilinear letterbox256; (gray/255*2-1)*1024",
+  "clinical_support_model": false,
+  "files": {
+    "image_state.pt": {
+      "bytes": 29573522,
+      "sha256": "0e65a604fb32428c74b61add88bd92f445b08966fe64d1c38d751a714b53f895"
+    },
+    "readout_state.pt": {
+      "bytes": 2358434,
+      "sha256": "03b6e4c094d68e764a0340f1c4ff13c7b75bfd5ed84372767396c77f89fe195b"
+    }
+  },
+  "implementation_sha256": {
+    "src/oa_cxr/rebuild/vision.py": "da3ed05c2ad2f208578450767c591774b2bb20ab2db5f68f7f7a8495a26be0e4",
+    "src/oa_cxr/rebuild/neural_readout.py": "fd74005eea32a3200fe6efce7fbf591b51b4464356b886609f87a3620f091842",
+    "src/oa_cxr/rebuild/data.py": "9b448e32218b96e0ea3e3e9fda703e82a2f906292ef3265d6c07e46506d19e04",
+    "src/oa_cxr/anatomy_geometry.py": "d83204b13aacc001418dc9717feedd85bf36c7a2979aabfb661802eb3d5bdd48",
+    "scripts/rebuild/predict_image.py": "7c9f098f7dd3541aea492857128330a41bcc167776da7e8e1f3341bc53653e4c"
+  }
+}

main/image_state.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0e65a604fb32428c74b61add88bd92f445b08966fe64d1c38d751a714b53f895
+size 29573522

main/readout_state.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:03b6e4c094d68e764a0340f1c4ff13c7b75bfd5ed84372767396c77f89fe195b
+size 2358434

main/training_history.json

@@ -0,0 +1,2717 @@
+{
+  "image_epochs": [
+    {
+      "checkpoint": "epoch_01.pt",
+      "checkpoint_sha256": "9be173e2fb003af981b15014039da67fa9957dfc5566167e0938f5e7db22a75d",
+      "dev": {
+        "empty_mask_rate": 0.009615384615384616,
+        "empty_masks": 128,
+        "finite_score_rate": 1.0,
+        "inputs": 6656,
+        "inputs_with_any_empty_lung": 103,
+        "inputs_with_both_empty_lungs": 25,
+        "mae": 0.1227096968736404,
+        "mask_denominator": 13312,
+        "mask_dice": 0.7868532171616187,
+        "mean_loss": 0.20446334034204483,
+        "retention_targets": 19968,
+        "unique_sources": 512
+      },
+      "epoch": 1,
+      "fit": {
+        "empty_mask_rate": 0.18104893639316777,
+        "empty_masks": 6911,
+        "finite_score_rate": 1.0,
+        "inputs": 19086,
+        "inputs_with_any_empty_lung": 5492,
+        "inputs_with_both_empty_lungs": 1419,
+        "mae": 0.16587682019994238,
+        "mask_denominator": 38172,
+        "mask_dice": 0.48709572119794275,
+        "mean_loss": 0.29659602698241216,
+        "retention_targets": 57258,
+        "unique_sources": 19086
+      },
+      "optimizer_steps": 75,
+      "optimizer_steps_in_epoch": 75
+    },
+    {
+      "checkpoint": "epoch_02.pt",
+      "checkpoint_sha256": "a9a6de0914c5ecbfda4ce462b3d5042c73380d77bafcdf26424bfe5e3c19b8ec",
+      "dev": {
+        "empty_mask_rate": 0.0015024038461538462,
+        "empty_masks": 20,
+        "finite_score_rate": 1.0,
+        "inputs": 6656,
+        "inputs_with_any_empty_lung": 20,
+        "inputs_with_both_empty_lungs": 0,
+        "mae": 0.11647739356909043,
+        "mask_denominator": 13312,
+        "mask_dice": 0.8515955759928777,
+        "mean_loss": 0.1748739515359585,
+        "retention_targets": 19968,
+        "unique_sources": 512
+      },
+      "epoch": 2,
+      "fit": {
+        "empty_mask_rate": 0.002672115686890915,
+        "empty_masks": 102,
+        "finite_score_rate": 1.0,
+        "inputs": 19086,
+        "inputs_with_any_empty_lung": 92,
+        "inputs_with_both_empty_lungs": 10,
+        "mae": 0.11818958801996521,
+        "mask_denominator": 38172,
+        "mask_dice": 0.8247444570957708,
+        "mean_loss": 0.18730735444652827,
+        "retention_targets": 57258,
+        "unique_sources": 19086
+      },
+      "optimizer_steps": 150,
+      "optimizer_steps_in_epoch": 75
+    },
+    {
+      "checkpoint": "epoch_03.pt",
+      "checkpoint_sha256": "4ae5514308913cecf3eadfc46792a9d92f0ba9a18f55629e2807b45c1cd36c11",
+      "dev": {
+        "empty_mask_rate": 0.0008263221153846154,
+        "empty_masks": 11,
+        "finite_score_rate": 1.0,
+        "inputs": 6656,
+        "inputs_with_any_empty_lung": 11,
+        "inputs_with_both_empty_lungs": 0,
+        "mae": 0.0945519676957375,
+        "mask_denominator": 13312,
+        "mask_dice": 0.8744298769877508,
+        "mean_loss": 0.14625132083892822,
+        "retention_targets": 19968,
+        "unique_sources": 512
+      },
+      "epoch": 3,
+      "fit": {
+        "empty_mask_rate": 0.0011002829298962591,
+        "empty_masks": 42,
+        "finite_score_rate": 1.0,
+        "inputs": 19086,
+        "inputs_with_any_empty_lung": 36,
+        "inputs_with_both_empty_lungs": 6,
+        "mae": 0.10536944994846005,
+        "mask_denominator": 38172,
+        "mask_dice": 0.8617484768169251,
+        "mean_loss": 0.16004001111319938,
+        "retention_targets": 57258,
+        "unique_sources": 19086
+      },
+      "optimizer_steps": 225,
+      "optimizer_steps_in_epoch": 75
+    },
+    {
+      "checkpoint": "epoch_04.pt",
+      "checkpoint_sha256": "539c305f35409ecb2e8c37d0557b3a1fbe2137a277001f028a1a4baccc515fd9",
+      "dev": {
+        "empty_mask_rate": 0.00037560096153846156,
+        "empty_masks": 5,
+        "finite_score_rate": 1.0,
+        "inputs": 6656,
+        "inputs_with_any_empty_lung": 5,
+        "inputs_with_both_empty_lungs": 0,
+        "mae": 0.0830361069394992,
+        "mask_denominator": 13312,
+        "mask_dice": 0.8853865174146799,
+        "mean_loss": 0.12520248270951784,
+        "retention_targets": 19968,
+        "unique_sources": 512
+      },
+      "epoch": 4,
+      "fit": {
+        "empty_mask_rate": 0.0006025358901812847,
+        "empty_masks": 23,
+        "finite_score_rate": 1.0,
+        "inputs": 19086,
+        "inputs_with_any_empty_lung": 19,
+        "inputs_with_both_empty_lungs": 4,
+        "mae": 0.08604315175100395,
+        "mask_denominator": 38172,
+        "mask_dice": 0.8785503710839004,
+        "mean_loss": 0.13423762584861865,
+        "retention_targets": 57258,
+        "unique_sources": 19086
+      },
+      "optimizer_steps": 300,
+      "optimizer_steps_in_epoch": 75
+    },
+    {
+      "checkpoint": "epoch_05.pt",
+      "checkpoint_sha256": "e6666ec36d4eacaeabd4f2d705626213c8762ef7e23ffc87519ba59e2522c23d",
+      "dev": {
+        "empty_mask_rate": 0.00015024038461538462,
+        "empty_masks": 2,
+        "finite_score_rate": 1.0,
+        "inputs": 6656,
+        "inputs_with_any_empty_lung": 2,
+        "inputs_with_both_empty_lungs": 0,
+        "mae": 0.07220350874540134,
+        "mask_denominator": 13312,
+        "mask_dice": 0.8924208535597875,
+        "mean_loss": 0.10896660129611309,
+        "retention_targets": 19968,
+        "unique_sources": 512
+      },
+      "epoch": 5,
+      "fit": {
+        "empty_mask_rate": 0.00039295818924866394,
+        "empty_masks": 15,
+        "finite_score_rate": 1.0,
+        "inputs": 19086,
+        "inputs_with_any_empty_lung": 14,
+        "inputs_with_both_empty_lungs": 1,
+        "mae": 0.07479647190339567,
+        "mask_denominator": 38172,
+        "mask_dice": 0.8883345131352927,
+        "mean_loss": 0.1159912211632911,
+        "retention_targets": 57258,
+        "unique_sources": 19086
+      },
+      "optimizer_steps": 375,
+      "optimizer_steps_in_epoch": 75
+    },
+    {
+      "checkpoint": "epoch_06.pt",
+      "checkpoint_sha256": "9b654faa7716b187147db0e7d08cc80e162364106e69a126be17cc662ab6391b",
+      "dev": {
+        "empty_mask_rate": 0.00015024038461538462,
+        "empty_masks": 2,
+        "finite_score_rate": 1.0,
+        "inputs": 6656,
+        "inputs_with_any_empty_lung": 2,
+        "inputs_with_both_empty_lungs": 0,
+        "mae": 0.06615302826349552,
+        "mask_denominator": 13312,
+        "mask_dice": 0.898768656528913,
+        "mean_loss": 0.09883683375441112,
+        "retention_targets": 19968,
+        "unique_sources": 512
+      },
+      "epoch": 6,
+      "fit": {
+        "empty_mask_rate": 0.0004977470397149744,
+        "empty_masks": 19,
+        "finite_score_rate": 1.0,
+        "inputs": 19086,
+        "inputs_with_any_empty_lung": 15,
+        "inputs_with_both_empty_lungs": 4,
+        "mae": 0.06858779310684607,
+        "mask_denominator": 38172,
+        "mask_dice": 0.8956410970550956,
+        "mean_loss": 0.10332147007585384,
+        "retention_targets": 57258,
+        "unique_sources": 19086
+      },
+      "optimizer_steps": 450,
+      "optimizer_steps_in_epoch": 75
+    },
+    {
+      "checkpoint": "epoch_07.pt",
+      "checkpoint_sha256": "3902a6a57bb872ccc302306654c7640779c5c084aa4d14185259393f7122c113",
+      "dev": {
+        "empty_mask_rate": 0.00015024038461538462,
+        "empty_masks": 2,
+        "finite_score_rate": 1.0,
+        "inputs": 6656,
+        "inputs_with_any_empty_lung": 2,
+        "inputs_with_both_empty_lungs": 0,
+        "mae": 0.06451035634829448,
+        "mask_denominator": 13312,
+        "mask_dice": 0.9034108038132007,
+        "mean_loss": 0.09044287038537172,
+        "retention_targets": 19968,
+        "unique_sources": 512
+      },
+      "epoch": 7,
+      "fit": {
+        "empty_mask_rate": 0.0004977470397149744,
+        "empty_masks": 19,
+        "finite_score_rate": 1.0,
+        "inputs": 19086,
+        "inputs_with_any_empty_lung": 18,
+        "inputs_with_both_empty_lungs": 1,
+        "mae": 0.06669137207135714,
+        "mask_denominator": 38172,
+        "mask_dice": 0.9006698474086556,
+        "mean_loss": 0.09575640312670712,
+        "retention_targets": 57258,
+        "unique_sources": 19086
+      },
+      "optimizer_steps": 525,
+      "optimizer_steps_in_epoch": 75
+    },
+    {
+      "checkpoint": "epoch_08.pt",
+      "checkpoint_sha256": "5605e1a3f10d9f6455ddef6e41a3b6007743b8a3306eb5e40f095e0bc0144cd2",
+      "dev": {
+        "empty_mask_rate": 7.512019230769231e-05,
+        "empty_masks": 1,
+        "finite_score_rate": 1.0,
+        "inputs": 6656,
+        "inputs_with_any_empty_lung": 1,
+        "inputs_with_both_empty_lungs": 0,
+        "mae": 0.06200080288526339,
+        "mask_denominator": 13312,
+        "mask_dice": 0.9076958711330707,
+        "mean_loss": 0.08590875365413152,
+        "retention_targets": 19968,
+        "unique_sources": 512
+      },
+      "epoch": 8,
+      "fit": {
+        "empty_mask_rate": 0.00047154982709839675,
+        "empty_masks": 18,
+        "finite_score_rate": 1.0,
+        "inputs": 19086,
+        "inputs_with_any_empty_lung": 16,
+        "inputs_with_both_empty_lungs": 2,
+        "mae": 0.06330932095662993,
+        "mask_denominator": 38172,
+        "mask_dice": 0.9054412090289922,
+        "mean_loss": 0.08775575796828684,
+        "retention_targets": 57258,
+        "unique_sources": 19086
+      },
+      "optimizer_steps": 600,
+      "optimizer_steps_in_epoch": 75
+    },
+    {
+      "checkpoint": "epoch_09.pt",
+      "checkpoint_sha256": "52f6ace4ec8e9736b7b6bc58df2e2200e52e9e7fd7faa2dc4621b6d4e484bf1e",
+      "dev": {
+        "empty_mask_rate": 0.00022536057692307694,
+        "empty_masks": 3,
+        "finite_score_rate": 1.0,
+        "inputs": 6656,
+        "inputs_with_any_empty_lung": 3,
+        "inputs_with_both_empty_lungs": 0,
+        "mae": 0.057530529414996125,
+        "mask_denominator": 13312,
+        "mask_dice": 0.9110865203233866,
+        "mean_loss": 0.07776057089750583,
+        "retention_targets": 19968,
+        "unique_sources": 512
+      },
+      "epoch": 9,
+      "fit": {
+        "empty_mask_rate": 0.00044535261448181915,
+        "empty_masks": 17,
+        "finite_score_rate": 1.0,
+        "inputs": 19086,
+        "inputs_with_any_empty_lung": 15,
+        "inputs_with_both_empty_lungs": 2,
+        "mae": 0.06157164573835991,
+        "mask_denominator": 38172,
+        "mask_dice": 0.9092764488171446,
+        "mean_loss": 0.08204685590195479,
+        "retention_targets": 57258,
+        "unique_sources": 19086
+      },
+      "optimizer_steps": 675,
+      "optimizer_steps_in_epoch": 75
+    },
+    {
+      "checkpoint": "epoch_10.pt",
+      "checkpoint_sha256": "7598673318ee2c690528ddf009d85348eff8a5ede1ff06a36e23c2aafefb73b7",
+      "dev": {
+        "empty_mask_rate": 7.512019230769231e-05,
+        "empty_masks": 1,
+        "finite_score_rate": 1.0,
+        "inputs": 6656,
+        "inputs_with_any_empty_lung": 1,
+        "inputs_with_both_empty_lungs": 0,
+        "mae": 0.05608832740630859,
+        "mask_denominator": 13312,
+        "mask_dice": 0.9137806846545293,
+        "mean_loss": 0.0727225080705606,
+        "retention_targets": 19968,
+        "unique_sources": 512
+      },
+      "epoch": 10,
+      "fit": {
+        "empty_mask_rate": 0.00036676097663208634,
+        "empty_masks": 14,
+        "finite_score_rate": 1.0,
+        "inputs": 19086,
+        "inputs_with_any_empty_lung": 13,
+        "inputs_with_both_empty_lungs": 1,
+        "mae": 0.060027012336499375,
+        "mask_denominator": 38172,
+        "mask_dice": 0.9121508038195126,
+        "mean_loss": 0.07720384848051622,
+        "retention_targets": 57258,
+        "unique_sources": 19086
+      },
+      "optimizer_steps": 750,
+      "optimizer_steps_in_epoch": 75
+    },
+    {
+      "checkpoint": "epoch_11.pt",
+      "checkpoint_sha256": "18029f135dafb667bf128756efaa0b3f70bd5dafe6bc7f05dee8021bb74f999f",
+      "dev": {
+        "empty_mask_rate": 0.0,
+        "empty_masks": 0,
+        "finite_score_rate": 1.0,
+        "inputs": 6656,
+        "inputs_with_any_empty_lung": 0,
+        "inputs_with_both_empty_lungs": 0,
+        "mae": 0.060585747162501015,
+        "mask_denominator": 13312,
+        "mask_dice": 0.9164397326799539,
+        "mean_loss": 0.07166450356061642,
+        "retention_targets": 19968,
+        "unique_sources": 512
+      },
+      "epoch": 11,
+      "fit": {
+        "empty_mask_rate": 0.00031436655139893113,
+        "empty_masks": 12,
+        "finite_score_rate": 1.0,
+        "inputs": 19086,
+        "inputs_with_any_empty_lung": 12,
+        "inputs_with_both_empty_lungs": 0,
+        "mae": 0.0584834917045763,
+        "mask_denominator": 38172,
+        "mask_dice": 0.9149107546797841,
+        "mean_loss": 0.07303050692090446,
+        "retention_targets": 57258,
+        "unique_sources": 19086
+      },
+      "optimizer_steps": 825,
+      "optimizer_steps_in_epoch": 75
+    },
+    {
+      "checkpoint": "epoch_12.pt",
+      "checkpoint_sha256": "1a864d165c02826c4a338c922f36150387fd13bd39f2419b48f885f76e3ef7c9",
+      "dev": {
+        "empty_mask_rate": 7.512019230769231e-05,
+        "empty_masks": 1,
+        "finite_score_rate": 1.0,
+        "inputs": 6656,
+        "inputs_with_any_empty_lung": 1,
+        "inputs_with_both_empty_lungs": 0,
+        "mae": 0.05139062830652946,
+        "mask_denominator": 13312,
+        "mask_dice": 0.9179969842617328,
+        "mean_loss": 0.0643508335432181,
+        "retention_targets": 19968,
+        "unique_sources": 512
+      },
+      "epoch": 12,
+      "fit": {
+        "empty_mask_rate": 0.00023577491354919838,
+        "empty_masks": 9,
+        "finite_score_rate": 1.0,
+        "inputs": 19086,
+        "inputs_with_any_empty_lung": 8,
+        "inputs_with_both_empty_lungs": 1,
+        "mae": 0.054722813464988224,
+        "mask_denominator": 38172,
+        "mask_dice": 0.9175331701738185,
+        "mean_loss": 0.06751939949430123,
+        "retention_targets": 57258,
+        "unique_sources": 19086
+      },
+      "optimizer_steps": 900,
+      "optimizer_steps_in_epoch": 75
+    },
+    {
+      "checkpoint": "epoch_13.pt",
+      "checkpoint_sha256": "1d3b96211c5258d898ab9eb4381dd3a891be762759d8f2e72840e990762e8398",
+      "dev": {
+        "empty_mask_rate": 0.0,
+        "empty_masks": 0,
+        "finite_score_rate": 1.0,
+        "inputs": 6656,
+        "inputs_with_any_empty_lung": 0,
+        "inputs_with_both_empty_lungs": 0,
+        "mae": 0.05219766669548475,
+        "mask_denominator": 13312,
+        "mask_dice": 0.9196936121353736,
+        "mean_loss": 0.06384247082930344,
+        "retention_targets": 19968,
+        "unique_sources": 512
+      },
+      "epoch": 13,
+      "fit": {
+        "empty_mask_rate": 0.00036676097663208634,
+        "empty_masks": 14,
+        "finite_score_rate": 1.0,
+        "inputs": 19086,
+        "inputs_with_any_empty_lung": 13,
+        "inputs_with_both_empty_lungs": 1,
+        "mae": 0.052771906509780496,
+        "mask_denominator": 38172,
+        "mask_dice": 0.9193242023191799,
+        "mean_loss": 0.06380555448880126,
+        "retention_targets": 57258,
+        "unique_sources": 19086
+      },
+      "optimizer_steps": 975,
+      "optimizer_steps_in_epoch": 75
+    }
+  ],
+  "readout_candidates": {
+    "direct_linear_l1": [
+      {
+        "epoch": 1,
+        "optimizer_steps": 364,
+        "optimizer_steps_in_epoch": 364,
+        "fit_online": {
+          "rows": 744354,
+          "loss": 0.06058220328840595,
+          "mae": 0.05091046968392321,
+          "rmse": 0.10031725100500272
+        },
+        "dev": {
+          "rows": 101751,
+          "mae": 0.04094255438561131,
+          "rmse": 0.07446232722023048,
+          "loss": 0.042449258700887595
+        },
+        "original512dev_secondary": {
+          "rows": 19968,
+          "mae": 0.04230055904892465,
+          "rmse": 0.07750042750383625
+        },
+        "learning_rate_first_step": 0.001,
+        "learning_rate_after_epoch": 0.0009972883382072955,
+        "elapsed_seconds": 7.30779504776001
+      },
+      {
+        "epoch": 2,
+        "optimizer_steps": 728,
+        "optimizer_steps_in_epoch": 364,
+        "fit_online": {
+          "rows": 744354,
+          "loss": 0.04147709251894019,
+          "mae": 0.037474345893086625,
+          "rmse": 0.07018590861366263
+        },
+        "dev": {
+          "rows": 101751,
+          "mae": 0.03487036950043545,
+          "rmse": 0.0705200557321309,
+          "loss": 0.03916088080805639
+        },
+        "original512dev_secondary": {
+          "rows": 19968,
+          "mae": 0.03622851871559527,
+          "rmse": 0.07325892123875513
+        },
+        "learning_rate_first_step": 0.0009972883382072955,
+        "learning_rate_after_epoch": 0.0009891830623632333,
+        "elapsed_seconds": 6.807323217391968
+      },
+      {
+        "epoch": 3,
+        "optimizer_steps": 1092,
+        "optimizer_steps_in_epoch": 364,
+        "fit_online": {
+          "rows": 744354,
+          "loss": 0.03835659186823945,
+          "mae": 0.03512048021936125,
+          "rmse": 0.0673968452330957
+        },
+        "dev": {
+          "rows": 101751,
+          "mae": 0.03308241309988131,
+          "rmse": 0.07036046837329007,
+          "loss": 0.0415421234095111
+        },
+        "original512dev_secondary": {
+          "rows": 19968,
+          "mae": 0.03454097929643934,
+          "rmse": 0.07362228222933895
+        },
+        "learning_rate_first_step": 0.0009891830623632333,
+        "learning_rate_after_epoch": 0.0009757729755660874,
+        "elapsed_seconds": 6.626096963882446
+      },
+      {
+        "epoch": 4,
+        "optimizer_steps": 1456,
+        "optimizer_steps_in_epoch": 364,
+        "fit_online": {
+          "rows": 744354,
+          "loss": 0.036504661086595745,
+          "mae": 0.03365991767376169,
+          "rmse": 0.06548440742141944
+        },
+        "dev": {
+          "rows": 101751,
+          "mae": 0.0323335764990457,
+          "rmse": 0.06848000238312762,
+          "loss": 0.03761528485741958
+        },
+        "original512dev_secondary": {
+          "rows": 19968,
+          "mae": 0.03386198177234361,
+          "rmse": 0.07212458337004116
+        },
+        "learning_rate_first_step": 0.0009757729755660874,
+        "learning_rate_after_epoch": 0.0009572050015330751,
+        "elapsed_seconds": 6.544482231140137
+      },
+      {
+        "epoch": 5,
+        "optimizer_steps": 1820,
+        "optimizer_steps_in_epoch": 364,
+        "fit_online": {
+          "rows": 744354,
+          "loss": 0.03451652270318648,
+          "mae": 0.03225052952254655,
+          "rmse": 0.06402622786339915
+        },
+        "dev": {
+          "rows": 101751,
+          "mae": 0.03631617115075516,
+          "rmse": 0.06876409938565552,
+          "loss": 0.037365560927709346
+        },
+        "original512dev_secondary": {
+          "rows": 19968,
+          "mae": 0.037732945167380634,
+          "rmse": 0.0723052532324048
+        },
+        "learning_rate_first_step": 0.0009572050015330751,
+        "learning_rate_after_epoch": 0.0009336825748732852,
+        "elapsed_seconds": 7.047106742858887
+      },
+      {
+        "epoch": 6,
+        "optimizer_steps": 2184,
+        "optimizer_steps_in_epoch": 364,
+        "fit_online": {
+          "rows": 744354,
+          "loss": 0.03368364261328352,
+          "mae": 0.03153022695597802,
+          "rmse": 0.06296290181300622
+        },
+        "dev": {
+          "rows": 101751,
+          "mae": 0.03200254639351097,
+          "rmse": 0.06726191579502339,
+          "loss": 0.035621887280235284
+        },
+        "original512dev_secondary": {
+          "rows": 19968,
+          "mae": 0.033763617147017685,
+          "rmse": 0.0709374572065831
+        },
+        "learning_rate_first_step": 0.0009336825748732852,
+        "learning_rate_after_epoch": 0.0009054634122155876,
+        "elapsed_seconds": 6.681865930557251
+      },
+      {
+        "epoch": 7,
+        "optimizer_steps": 2548,
+        "optimizer_steps_in_epoch": 364,
+        "fit_online": {
+          "rows": 744354,
+          "loss": 0.0328715886893635,
+          "mae": 0.030847544139012247,
+          "rmse": 0.06190260665204419
+        },
+        "dev": {
+          "rows": 101751,
+          "mae": 0.0327118679838962,
+          "rmse": 0.06657324516781472,
+          "loss": 0.03309010461813686
+        },
+        "original512dev_secondary": {
+          "rows": 19968,
+          "mae": 0.034002047428680855,
+          "rmse": 0.06935273326140395
+        },
+        "learning_rate_first_step": 0.0009054634122155876,
+        "learning_rate_after_epoch": 0.0008728566886112981,
+        "elapsed_seconds": 6.713434934616089
+      },
+      {
+        "epoch": 8,
+        "optimizer_steps": 2912,
+        "optimizer_steps_in_epoch": 364,
+        "fit_online": {
+          "rows": 744354,
+          "loss": 0.03251594360128511,
+          "mae": 0.030462478328990086,
+          "rmse": 0.06113859715257605
+        },
+        "dev": {
+          "rows": 101751,
+          "mae": 0.03113375807660508,
+          "rmse": 0.06634480810318082,
+          "loss": 0.035080138701550904
+        },
+        "original512dev_secondary": {
+          "rows": 19968,
+          "mae": 0.03210634992748195,
+          "rmse": 0.06859976178787415
+        },
+        "learning_rate_first_step": 0.0008728566886112981,
+        "learning_rate_after_epoch": 0.0008362196501476234,
+        "elapsed_seconds": 6.780497789382935
+      },
+      {
+        "epoch": 9,
+        "optimizer_steps": 3276,
+        "optimizer_steps_in_epoch": 364,
+        "fit_online": {
+          "rows": 744354,
+          "loss": 0.03062308308570109,
+          "mae": 0.029195708972220357,
+          "rmse": 0.06006962400030129
+        },
+        "dev": {
+          "rows": 101751,
+          "mae": 0.03019921024477784,
+          "rmse": 0.06575808518854956,
+          "loss": 0.033124916422057914
+        },
+        "original512dev_secondary": {
+          "rows": 19968,
+          "mae": 0.03169540487329542,
+          "rmse": 0.06974010562077977
+        },
+        "learning_rate_first_step": 0.0008362196501476234,
+        "learning_rate_after_epoch": 0.0007959536998847633,
+        "elapsed_seconds": 7.007699489593506
+      },
+      {
+        "epoch": 10,
+        "optimizer_steps": 3640,
+        "optimizer_steps_in_epoch": 364,
+        "fit_online": {
+          "rows": 744354,
+          "loss": 0.0309133592615224,
+          "mae": 0.02925924421918499,
+          "rmse": 0.05956520873959362
+        },
+        "dev": {
+          "rows": 101751,
+          "mae": 0.031106386554098916,
+          "rmse": 0.06615741478187104,
+          "loss": 0.03420988305332044
+        },
+        "original512dev_secondary": {
+          "rows": 19968,
+          "mae": 0.03277212518708275,
+          "rmse": 0.07027952657332451
+        },
+        "learning_rate_first_step": 0.0007959536998847633,
+        "learning_rate_after_epoch": 0.0007524999999999889,
+        "elapsed_seconds": 6.67080283164978
+      },
+      {
+        "epoch": 11,
+        "optimizer_steps": 4004,
+        "optimizer_steps_in_epoch": 364,
+        "fit_online": {
+          "rows": 744354,
+          "loss": 0.0302926722812925,
+          "mae": 0.028685745296399313,
+          "rmse": 0.05874912901305199
+        },
+        "dev": {
+          "rows": 101751,
+          "mae": 0.031600128899487906,
+          "rmse": 0.06488314102854835,
+          "loss": 0.03210968502316409
+        },
+        "original512dev_secondary": {
+          "rows": 19968,
+          "mae": 0.03305021849141317,
+          "rmse": 0.068267171959538
+        },
+        "learning_rate_first_step": 0.0007524999999999889,
+        "learning_rate_after_epoch": 0.0007063346383225108,
+        "elapsed_seconds": 6.706934213638306
+      },
+      {
+        "epoch": 12,
+        "optimizer_steps": 4368,
+        "optimizer_steps_in_epoch": 364,
+        "fit_online": {
+          "rows": 744354,
+          "loss": 0.029442142222206567,
+          "mae": 0.028091330909729876,
+          "rmse": 0.05806229995045065
+        },
+        "dev": {
+          "rows": 101751,
+          "mae": 0.030871191382079777,
+          "rmse": 0.06522664711822819,
+          "loss": 0.032730800780952966
+        },
+        "original512dev_secondary": {
+          "rows": 19968,
+          "mae": 0.032027735486525884,
+          "rmse": 0.06773762084544913
+        },
+        "learning_rate_first_step": 0.0007063346383225108,
+        "learning_rate_after_epoch": 0.00065796341221559,
+        "elapsed_seconds": 6.76559042930603
+      },
+      {
+        "epoch": 13,
+        "optimizer_steps": 4732,
+        "optimizer_steps_in_epoch": 364,
+        "fit_online": {
+          "rows": 744354,
+          "loss": 0.02828365176879668,
+          "mae": 0.027225919030976885,
+          "rmse": 0.05693161443246114
+        },
+        "dev": {
+          "rows": 101751,
+          "mae": 0.031574329285243306,
+          "rmse": 0.06555494550413722,
+          "loss": 0.03237986339806644
+        },
+        "original512dev_secondary": {
+          "rows": 19968,
+          "mae": 0.033230618248942335,
+          "rmse": 0.06957409628157213
+        },
+        "learning_rate_first_step": 0.00065796341221559,
+        "learning_rate_after_epoch": 0.0006079162869547829,
+        "elapsed_seconds": 6.494726181030273
+      },
+      {
+        "epoch": 14,
+        "optimizer_steps": 5096,
+        "optimizer_steps_in_epoch": 364,
+        "fit_online": {
+          "rows": 744354,
+          "loss": 0.028207708243997077,
+          "mae": 0.0270052325982638,
+          "rmse": 0.05625659114067197
+        },
+        "dev": {
+          "rows": 101751,
+          "mae": 0.02998567952835894,
+          "rmse": 0.06464089876589953,
+          "loss": 0.031098736212483633
+        },
+        "original512dev_secondary": {
+          "rows": 19968,
+          "mae": 0.03154111559883742,
+          "rmse": 0.0686455408851101
+        },
+        "learning_rate_first_step": 0.0006079162869547829,
+        "learning_rate_after_epoch": 0.0005567415893174813,
+        "elapsed_seconds": 6.534796953201294
+      },
+      {
+        "epoch": 15,
+        "optimizer_steps": 5460,
+        "optimizer_steps_in_epoch": 364,
+        "fit_online": {
+          "rows": 744354,
+          "loss": 0.027491302859587004,
+          "mae": 0.026400591839777318,
+          "rmse": 0.05538358624134277
+        },
+        "dev": {
+          "rows": 101751,
+          "mae": 0.030261084956018917,
+          "rmse": 0.06458833751654644,
+          "loss": 0.031043904132992323
+        },
+        "original512dev_secondary": {
+          "rows": 19968,
+          "mae": 0.03183068711160349,
+          "rmse": 0.06815882902993789
+        },
+        "learning_rate_first_step": 0.0005567415893174813,
+        "learning_rate_after_epoch": 0.0005049999999999935,
+        "elapsed_seconds": 6.57889461517334
+      },
+      {
+        "epoch": 16,
+        "optimizer_steps": 5824,
+        "optimizer_steps_in_epoch": 364,
+        "fit_online": {
+          "rows": 744354,
+          "loss": 0.026675204599734034,
+          "mae": 0.025799767152434587,
+          "rmse": 0.05462575658139202
+        },
+        "dev": {
+          "rows": 101751,
+          "mae": 0.030058256266742144,
+          "rmse": 0.06362809370451253,
+          "loss": 0.030738392211027102
+        },
+        "original512dev_secondary": {
+          "rows": 19968,
+          "mae": 0.031580004041526805,
+          "rmse": 0.06719660093195959
+        },
+        "learning_rate_first_step": 0.0005049999999999935,
+        "learning_rate_after_epoch": 0.0004532584106825036,
+        "elapsed_seconds": 6.493250370025635
+      },
+      {
+        "epoch": 17,
+        "optimizer_steps": 6188,
+        "optimizer_steps_in_epoch": 364,
+        "fit_online": {
+          "rows": 744354,
+          "loss": 0.026308463338122107,
+          "mae": 0.025374927060411185,
+          "rmse": 0.053704852303090446
+        },
+        "dev": {
+          "rows": 101751,
+          "mae": 0.029631323119471312,
+          "rmse": 0.06444099020814983,
+          "loss": 0.0307903477647658
+        },
+        "original512dev_secondary": {
+          "rows": 19968,
+          "mae": 0.03154317564992827,
+          "rmse": 0.06872505296314751
+        },
+        "learning_rate_first_step": 0.0004532584106825036,
+        "learning_rate_after_epoch": 0.0004020837130452019,
+        "elapsed_seconds": 7.000838756561279
+      },
+      {
+        "epoch": 18,
+        "optimizer_steps": 6552,
+        "optimizer_steps_in_epoch": 364,
+        "fit_online": {
+          "rows": 744354,
+          "loss": 0.025616433877499983,
+          "mae": 0.02481770623691889,
+          "rmse": 0.052856837265773345
+        },
+        "dev": {
+          "rows": 101751,
+          "mae": 0.029918858391230617,
+          "rmse": 0.06401756685015632,
+          "loss": 0.030529506138929942
+        },
+        "original512dev_secondary": {
+          "rows": 19968,
+          "mae": 0.03159752009529863,
+          "rmse": 0.067738737603663
+        },
+        "learning_rate_first_step": 0.0004020837130452019,
+        "learning_rate_after_epoch": 0.0003520365877843945,
+        "elapsed_seconds": 6.529137849807739
+      },
+      {
+        "epoch": 19,
+        "optimizer_steps": 6916,
+        "optimizer_steps_in_epoch": 364,
+        "fit_online": {
+          "rows": 744354,
+          "loss": 0.024908378293164894,
+          "mae": 0.024211338811415354,
+          "rmse": 0.051977200613113325
+        },
+        "dev": {
+          "rows": 101751,
+          "mae": 0.029201405788266143,
+          "rmse": 0.06329769284939732,
+          "loss": 0.029964010075624062
+        },
+        "original512dev_secondary": {
+          "rows": 19968,
+          "mae": 0.030975230067852886,
+          "rmse": 0.06743248941656216
+        },
+        "learning_rate_first_step": 0.0003520365877843945,
+        "learning_rate_after_epoch": 0.0003036653616774734,
+        "elapsed_seconds": 6.625686407089233
+      },
+      {
+        "epoch": 20,
+        "optimizer_steps": 7280,
+        "optimizer_steps_in_epoch": 364,
+        "fit_online": {
+          "rows": 744354,
+          "loss": 0.02438169519042887,
+          "mae": 0.02374144191428653,
+          "rmse": 0.051113580382246775
+        },
+        "dev": {
+          "rows": 101751,
+          "mae": 0.029543249290725635,
+          "rmse": 0.06358945282846644,
+          "loss": 0.030116087247771286
+        },
+        "original512dev_secondary": {
+          "rows": 19968,
+          "mae": 0.030758793906198453,
+          "rmse": 0.06636259999751981
+        },
+        "learning_rate_first_step": 0.0003036653616774734,
+        "learning_rate_after_epoch": 0.00025749999999999574,
+        "elapsed_seconds": 6.683897018432617
+      },
+      {
+        "epoch": 21,
+        "optimizer_steps": 7644,
+        "optimizer_steps_in_epoch": 364,
+        "fit_online": {
+          "rows": 744354,
+          "loss": 0.02381758011081825,
+          "mae": 0.02322390503651869,
+          "rmse": 0.0502464771445294
+        },
+        "dev": {
+          "rows": 101751,
+          "mae": 0.029440362574935208,
+          "rmse": 0.06400964295845371,
+          "loss": 0.03013190550132695
+        },
+        "original512dev_secondary": {
+          "rows": 19968,
+          "mae": 0.03100051673810902,
+          "rmse": 0.06765519397623931
+        },
+        "learning_rate_first_step": 0.00025749999999999574,
+        "learning_rate_after_epoch": 0.00021404630011522352,
+        "elapsed_seconds": 6.457980155944824
+      },
+      {
+        "epoch": 22,
+        "optimizer_steps": 8008,
+        "optimizer_steps_in_epoch": 364,
+        "fit_online": {
+          "rows": 744354,
+          "loss": 0.023243114632731328,
+          "mae": 0.02270645267174862,
+          "rmse": 0.04936532560428204
+        },
+        "dev": {
+          "rows": 101751,
+          "mae": 0.029485488784940728,
+          "rmse": 0.06326844660456636,
+          "loss": 0.02984920664818494
+        },
+        "original512dev_secondary": {
+          "rows": 19968,
+          "mae": 0.030840384475712385,
+          "rmse": 0.06650411290333826
+        },
+        "learning_rate_first_step": 0.00021404630011522352,
+        "learning_rate_after_epoch": 0.0001737803498523636,
+        "elapsed_seconds": 6.626300573348999
+      },
+      {
+        "epoch": 23,
+        "optimizer_steps": 8372,
+        "optimizer_steps_in_epoch": 364,
+        "fit_online": {
+          "rows": 744354,
+          "loss": 0.022744763166059244,
+          "mae": 0.022224517993621764,
+          "rmse": 0.048459180988542926
+        },
+        "dev": {
+          "rows": 101751,
+          "mae": 0.02892236697920604,
+          "rmse": 0.06318939747080124,
+          "loss": 0.029543335650865758
+        },
+        "original512dev_secondary": {
+          "rows": 19968,
+          "mae": 0.030464202442952882,
+          "rmse": 0.06664260307260796
+        },
+        "learning_rate_first_step": 0.0001737803498523636,
+        "learning_rate_after_epoch": 0.00013714331138868892,
+        "elapsed_seconds": 6.474420070648193
+      },
+      {
+        "epoch": 24,
+        "optimizer_steps": 8736,
+        "optimizer_steps_in_epoch": 364,
+        "fit_online": {
+          "rows": 744354,
+          "loss": 0.022228786125627207,
+          "mae": 0.021765739612196057,
+          "rmse": 0.047690749633964455
+        },
+        "dev": {
+          "rows": 101751,
+          "mae": 0.02881881360167337,
+          "rmse": 0.0630085297399997,
+          "loss": 0.029368910750755893
+        },
+        "original512dev_secondary": {
+          "rows": 19968,
+          "mae": 0.03023305935804297,
+          "rmse": 0.06632915731115335
+        },
+        "learning_rate_first_step": 0.00013714331138868892,
+        "learning_rate_after_epoch": 0.0001045365877844009,
+        "elapsed_seconds": 6.4998228549957275
+      },
+      {
+        "epoch": 25,
+        "optimizer_steps": 9100,
+        "optimizer_steps_in_epoch": 364,
+        "fit_online": {
+          "rows": 744354,
+          "loss": 0.021728201556093386,
+          "mae": 0.021306427203852615,
+          "rmse": 0.04697806510070597
+        },
+        "dev": {
+          "rows": 101751,
+          "mae": 0.028995420477154397,
+          "rmse": 0.06327068668034523,
+          "loss": 0.02934709768646949
+        },
+        "original512dev_secondary": {
+          "rows": 19968,
+          "mae": 0.03070051783328008,
+          "rmse": 0.06721116849085028
+        },
+        "learning_rate_first_step": 0.0001045365877844009,
+        "learning_rate_after_epoch": 7.631742512670258e-05,
+        "elapsed_seconds": 6.36391282081604
+      },
+      {
+        "epoch": 26,
+        "optimizer_steps": 9464,
+        "optimizer_steps_in_epoch": 364,
+        "fit_online": {
+          "rows": 744354,
+          "loss": 0.021352519107416788,
+          "mae": 0.020952917934951375,
+          "rmse": 0.04635970126689831
+        },
+        "dev": {
+          "rows": 101751,
+          "mae": 0.02885526050311611,
+          "rmse": 0.06282146456826385,
+          "loss": 0.029229418828520645
+        },
+        "original512dev_secondary": {
+          "rows": 19968,
+          "mae": 0.03033251428200273,
+          "rmse": 0.06619906558569182
+        },
+        "learning_rate_first_step": 7.631742512670258e-05,
+        "learning_rate_after_epoch": 5.2794998466912824e-05,
+        "elapsed_seconds": 6.567789793014526
+      },
+      {
+        "epoch": 27,
+        "optimizer_steps": 9828,
+        "optimizer_steps_in_epoch": 364,
+        "fit_online": {
+          "rows": 744354,
+          "loss": 0.021012643755046644,
+          "mae": 0.020633238350969578,
+          "rmse": 0.04585088079719173
+        },
+        "dev": {
+          "rows": 101751,
+          "mae": 0.028647494974946227,
+          "rmse": 0.06296550996127663,
+          "loss": 0.02913083199735023
+        },
+        "original512dev_secondary": {
+          "rows": 19968,
+          "mae": 0.030173709561518665,
+          "rmse": 0.06642364578800998
+        },
+        "learning_rate_first_step": 5.2794998466912824e-05,
+        "learning_rate_after_epoch": 3.4227024433899344e-05,
+        "elapsed_seconds": 6.570519924163818
+      },
+      {
+        "epoch": 28,
+        "optimizer_steps": 10192,
+        "optimizer_steps_in_epoch": 364,
+        "fit_online": {
+          "rows": 744354,
+          "loss": 0.020758838305494135,
+          "mae": 0.020392714093210498,
+          "rmse": 0.045482779222410244
+        },
+        "dev": {
+          "rows": 101751,
+          "mae": 0.028667092577853548,
+          "rmse": 0.0628530052642868,
+          "loss": 0.02909969400914241
+        },
+        "original512dev_secondary": {
+          "rows": 19968,
+          "mae": 0.030135565899236567,
+          "rmse": 0.06617831773623223
+        },
+        "learning_rate_first_step": 3.4227024433899344e-05,
+        "learning_rate_after_epoch": 2.0816937636766568e-05,
+        "elapsed_seconds": 6.565901041030884
+      },
+      {
+        "epoch": 29,
+        "optimizer_steps": 10556,
+        "optimizer_steps_in_epoch": 364,
+        "fit_online": {
+          "rows": 744354,
+          "loss": 0.020574413233970246,
+          "mae": 0.02021760333100663,
+          "rmse": 0.04519503268328141
+        },
+        "dev": {
+          "rows": 101751,
+          "mae": 0.028734173133948684,
+          "rmse": 0.06297860538568872,
+          "loss": 0.02912594999158165
+        },
+        "original512dev_secondary": {
+          "rows": 19968,
+          "mae": 0.030253989823974665,
+          "rmse": 0.0663602681194177
+        },
+        "learning_rate_first_step": 2.0816937636766568e-05,
+        "learning_rate_after_epoch": 1.2711661792704754e-05,
+        "elapsed_seconds": 6.50202751159668
+      },
+      {
+        "epoch": 30,
+        "optimizer_steps": 10920,
+        "optimizer_steps_in_epoch": 364,
+        "fit_online": {
+          "rows": 744354,
+          "loss": 0.02046529108239672,
+          "mae": 0.020111129582613574,
+          "rmse": 0.04504105601223995
+        },
+        "dev": {
+          "rows": 101751,
+          "mae": 0.028726131914638315,
+          "rmse": 0.06295859321453623,
+          "loss": 0.029134791254082835
+        },
+        "original512dev_secondary": {
+          "rows": 19968,
+          "mae": 0.030276363099521683,
+          "rmse": 0.06636366785773566
+        },
+        "learning_rate_first_step": 1.2711661792704754e-05,
+        "learning_rate_after_epoch": 1e-05,
+        "elapsed_seconds": 6.664385795593262
+      }
+    ],
+    "direct_linear_huber002": [
+      {
+        "epoch": 1,
+        "optimizer_steps": 364,
+        "optimizer_steps_in_epoch": 364,
+        "fit_online": {
+          "rows": 744354,
+          "loss": 0.04984432069008648,
+          "mae": 0.049429969395444376,
+          "rmse": 0.099194989957091
+        },
+        "dev": {
+          "rows": 101751,
+          "mae": 0.035827672188343096,
+          "rmse": 0.07312504036847618,
+          "loss": 0.03509623854741716
+        },
+        "original512dev_secondary": {
+          "rows": 19968,
+          "mae": 0.03743612162690205,
+          "rmse": 0.0762776500599208
+        },
+        "learning_rate_first_step": 0.001,
+        "learning_rate_after_epoch": 0.0009972883382072955,
+        "elapsed_seconds": 6.693447589874268
+      },
+      {
+        "epoch": 2,
+        "optimizer_steps": 728,
+        "optimizer_steps_in_epoch": 364,
+        "fit_online": {
+          "rows": 744354,
+          "loss": 0.03139053445025022,
+          "mae": 0.03620392026555886,
+          "rmse": 0.06904588992299891
+        },
+        "dev": {
+          "rows": 101751,
+          "mae": 0.034123940445588336,
+          "rmse": 0.0703385678038954,
+          "loss": 0.03182720056051354
+        },
+        "original512dev_secondary": {
+          "rows": 19968,
+          "mae": 0.03518747945574246,
+          "rmse": 0.07279465977076613
+        },
+        "learning_rate_first_step": 0.0009972883382072955,
+        "learning_rate_after_epoch": 0.0009891830623632333,
+        "elapsed_seconds": 6.890185832977295
+      },
+      {
+        "epoch": 3,
+        "optimizer_steps": 1092,
+        "optimizer_steps_in_epoch": 364,
+        "fit_online": {
+          "rows": 744354,
+          "loss": 0.028129160545248603,
+          "mae": 0.033551130167183196,
+          "rmse": 0.06611485059521838
+        },
+        "dev": {
+          "rows": 101751,
+          "mae": 0.03353400695458655,
+          "rmse": 0.06906471687123385,
+          "loss": 0.028168519010955143
+        },
+        "original512dev_secondary": {
+          "rows": 19968,
+          "mae": 0.0348008897886897,
+          "rmse": 0.07218366090860931
+        },
+        "learning_rate_first_step": 0.0009891830623632333,
+        "learning_rate_after_epoch": 0.0009757729755660874,
+        "elapsed_seconds": 6.684747934341431
+      },
+      {
+        "epoch": 4,
+        "optimizer_steps": 1456,
+        "optimizer_steps_in_epoch": 364,
+        "fit_online": {
+          "rows": 744354,
+          "loss": 0.026502383577197232,
+          "mae": 0.0320608447859219,
+          "rmse": 0.06408607810424559
+        },
+        "dev": {
+          "rows": 101751,
+          "mae": 0.03240945103169726,
+          "rmse": 0.06750224008948673,
+          "loss": 0.02856281117793811
+        },
+        "original512dev_secondary": {
+          "rows": 19968,
+          "mae": 0.03394312622880628,
+          "rmse": 0.07103672968123644
+        },
+        "learning_rate_first_step": 0.0009757729755660874,
+        "learning_rate_after_epoch": 0.0009572050015330751,
+        "elapsed_seconds": 6.7033984661102295
+      },
+      {
+        "epoch": 5,
+        "optimizer_steps": 1820,
+        "optimizer_steps_in_epoch": 364,
+        "fit_online": {
+          "rows": 744354,
+          "loss": 0.025661551979369583,
+          "mae": 0.031244058372205136,
+          "rmse": 0.06278084961431092
+        },
+        "dev": {
+          "rows": 101751,
+          "mae": 0.031586418981493065,
+          "rmse": 0.06680494512397446,
+          "loss": 0.02687006156292139
+        },
+        "original512dev_secondary": {
+          "rows": 19968,
+          "mae": 0.03319146174719488,
+          "rmse": 0.0705755658931673
+        },
+        "learning_rate_first_step": 0.0009572050015330751,
+        "learning_rate_after_epoch": 0.0009336825748732852,
+        "elapsed_seconds": 6.602811336517334
+      },
+      {
+        "epoch": 6,
+        "optimizer_steps": 2184,
+        "optimizer_steps_in_epoch": 364,
+        "fit_online": {
+          "rows": 744354,
+          "loss": 0.024938349317211086,
+          "mae": 0.030543004727533785,
+          "rmse": 0.061669788081319364
+        },
+        "dev": {
+          "rows": 101751,
+          "mae": 0.03171195382978924,
+          "rmse": 0.06729413352059711,
+          "loss": 0.027672946199228055
+        },
+        "original512dev_secondary": {
+          "rows": 19968,
+          "mae": 0.033330549073980364,
+          "rmse": 0.07076705091309617
+        },
+        "learning_rate_first_step": 0.0009336825748732852,
+        "learning_rate_after_epoch": 0.0009054634122155876,
+        "elapsed_seconds": 6.732528924942017
+      },
+      {
+        "epoch": 7,
+        "optimizer_steps": 2548,
+        "optimizer_steps_in_epoch": 364,
+        "fit_online": {
+          "rows": 744354,
+          "loss": 0.024509088706286906,
+          "mae": 0.0301049552784409,
+          "rmse": 0.06085016525970353
+        },
+        "dev": {
+          "rows": 101751,
+          "mae": 0.03214564776785915,
+          "rmse": 0.06676459093860473,
+          "loss": 0.026549576049480984
+        },
+        "original512dev_secondary": {
+          "rows": 19968,
+          "mae": 0.03355657159404356,
+          "rmse": 0.06972440865705071
+        },
+        "learning_rate_first_step": 0.0009054634122155876,
+        "learning_rate_after_epoch": 0.0008728566886112981,
+        "elapsed_seconds": 6.771991729736328
+      },
+      {
+        "epoch": 8,
+        "optimizer_steps": 2912,
+        "optimizer_steps_in_epoch": 364,
+        "fit_online": {
+          "rows": 744354,
+          "loss": 0.023768851128284193,
+          "mae": 0.0293813518543087,
+          "rmse": 0.059757863152635336
+        },
+        "dev": {
+          "rows": 101751,
+          "mae": 0.036509859650867636,
+          "rmse": 0.06572634034030703,
+          "loss": 0.028184297610347735
+        },
+        "original512dev_secondary": {
+          "rows": 19968,
+          "mae": 0.03759202991027567,
+          "rmse": 0.06887892261553111
+        },
+        "learning_rate_first_step": 0.0008728566886112981,
+        "learning_rate_after_epoch": 0.0008362196501476234,
+        "elapsed_seconds": 6.68094539642334
+      },
+      {
+        "epoch": 9,
+        "optimizer_steps": 3276,
+        "optimizer_steps_in_epoch": 364,
+        "fit_online": {
+          "rows": 744354,
+          "loss": 0.02337226400067204,
+          "mae": 0.02898374962942019,
+          "rmse": 0.05906340205531044
+        },
+        "dev": {
+          "rows": 101751,
+          "mae": 0.03030742292118074,
+          "rmse": 0.06599711020092029,
+          "loss": 0.027187716006941823
+        },
+        "original512dev_secondary": {
+          "rows": 19968,
+          "mae": 0.03168679501987544,
+          "rmse": 0.06932175963748116
+        },
+        "learning_rate_first_step": 0.0008362196501476234,
+        "learning_rate_after_epoch": 0.0007959536998847633,
+        "elapsed_seconds": 6.4947099685668945
+      },
+      {
+        "epoch": 10,
+        "optimizer_steps": 3640,
+        "optimizer_steps_in_epoch": 364,
+        "fit_online": {
+          "rows": 744354,
+          "loss": 0.022886235494999428,
+          "mae": 0.02848987103672732,
+          "rmse": 0.05815389503282315
+        },
+        "dev": {
+          "rows": 101751,
+          "mae": 0.03146657454769741,
+          "rmse": 0.06500343344169653,
+          "loss": 0.02605054139139181
+        },
+        "original512dev_secondary": {
+          "rows": 19968,
+          "mae": 0.03339396470700871,
+          "rmse": 0.06960028156442657
+        },
+        "learning_rate_first_step": 0.0007959536998847633,
+        "learning_rate_after_epoch": 0.0007524999999999889,
+        "elapsed_seconds": 6.598133563995361
+      },
+      {
+        "epoch": 11,
+        "optimizer_steps": 4004,
+        "optimizer_steps_in_epoch": 364,
+        "fit_online": {
+          "rows": 744354,
+          "loss": 0.022327483575615006,
+          "mae": 0.027911835137585623,
+          "rmse": 0.05724636855828063
+        },
+        "dev": {
+          "rows": 101751,
+          "mae": 0.03106796976225289,
+          "rmse": 0.06479682163809641,
+          "loss": 0.025748833508878712
+        },
+        "original512dev_secondary": {
+          "rows": 19968,
+          "mae": 0.032512807792395154,
+          "rmse": 0.06804500003137418
+        },
+        "learning_rate_first_step": 0.0007524999999999889,
+        "learning_rate_after_epoch": 0.0007063346383225108,
+        "elapsed_seconds": 6.592005252838135
+      },
+      {
+        "epoch": 12,
+        "optimizer_steps": 4368,
+        "optimizer_steps_in_epoch": 364,
+        "fit_online": {
+          "rows": 744354,
+          "loss": 0.02181470372487065,
+          "mae": 0.027375004324197413,
+          "rmse": 0.05649558509329766
+        },
+        "dev": {
+          "rows": 101751,
+          "mae": 0.03133940748079588,
+          "rmse": 0.06517046637069757,
+          "loss": 0.02588013147187859
+        },
+        "original512dev_secondary": {
+          "rows": 19968,
+          "mae": 0.032584269121961815,
+          "rmse": 0.06834765249004657
+        },
+        "learning_rate_first_step": 0.0007063346383225108,
+        "learning_rate_after_epoch": 0.00065796341221559,
+        "elapsed_seconds": 6.59993839263916
+      },
+      {
+        "epoch": 13,
+        "optimizer_steps": 4732,
+        "optimizer_steps_in_epoch": 364,
+        "fit_online": {
+          "rows": 744354,
+          "loss": 0.021467018790697165,
+          "mae": 0.0270260020140972,
+          "rmse": 0.05561983900797279
+        },
+        "dev": {
+          "rows": 101751,
+          "mae": 0.03142293147030038,
+          "rmse": 0.0647440209614246,
+          "loss": 0.025664451637938127
+        },
+        "original512dev_secondary": {
+          "rows": 19968,
+          "mae": 0.033400322976889676,
+          "rmse": 0.06917499693251958
+        },
+        "learning_rate_first_step": 0.00065796341221559,
+        "learning_rate_after_epoch": 0.0006079162869547829,
+        "elapsed_seconds": 6.6793787479400635
+      },
+      {
+        "epoch": 14,
+        "optimizer_steps": 5096,
+        "optimizer_steps_in_epoch": 364,
+        "fit_online": {
+          "rows": 744354,
+          "loss": 0.020844115403417615,
+          "mae": 0.026362105641648433,
+          "rmse": 0.05461912835667318
+        },
+        "dev": {
+          "rows": 101751,
+          "mae": 0.03032765749269126,
+          "rmse": 0.06421384036755157,
+          "loss": 0.02509134706806522
+        },
+        "original512dev_secondary": {
+          "rows": 19968,
+          "mae": 0.03186334526930505,
+          "rmse": 0.0684256694772552
+        },
+        "learning_rate_first_step": 0.0006079162869547829,
+        "learning_rate_after_epoch": 0.0005567415893174813,
+        "elapsed_seconds": 6.6162354946136475
+      },
+      {
+        "epoch": 15,
+        "optimizer_steps": 5460,
+        "optimizer_steps_in_epoch": 364,
+        "fit_online": {
+          "rows": 744354,
+          "loss": 0.020332033233212483,
+          "mae": 0.025824738409226652,
+          "rmse": 0.05372458221563485
+        },
+        "dev": {
+          "rows": 101751,
+          "mae": 0.0300815250716968,
+          "rmse": 0.06381505077127242,
+          "loss": 0.024923165345687885
+        },
+        "original512dev_secondary": {
+          "rows": 19968,
+          "mae": 0.03157166410248161,
+          "rmse": 0.0671033868675765
+        },
+        "learning_rate_first_step": 0.0005567415893174813,
+        "learning_rate_after_epoch": 0.0005049999999999935,
+        "elapsed_seconds": 6.54558801651001
+      },
+      {
+        "epoch": 16,
+        "optimizer_steps": 5824,
+        "optimizer_steps_in_epoch": 364,
+        "fit_online": {
+          "rows": 744354,
+          "loss": 0.019937418661292044,
+          "mae": 0.025408796234736932,
+          "rmse": 0.05291997550941795
+        },
+        "dev": {
+          "rows": 101751,
+          "mae": 0.031230793997639148,
+          "rmse": 0.06340936459247495,
+          "loss": 0.024821342086203214
+        },
+        "original512dev_secondary": {
+          "rows": 19968,
+          "mae": 0.032729066208001964,
+          "rmse": 0.06756287162520054
+        },
+        "learning_rate_first_step": 0.0005049999999999935,
+        "learning_rate_after_epoch": 0.0004532584106825036,
+        "elapsed_seconds": 6.589112043380737
+      },
+      {
+        "epoch": 17,
+        "optimizer_steps": 6188,
+        "optimizer_steps_in_epoch": 364,
+        "fit_online": {
+          "rows": 744354,
+          "loss": 0.019449896201723455,
+          "mae": 0.024888971409670786,
+          "rmse": 0.052002239329828844
+        },
+        "dev": {
+          "rows": 101751,
+          "mae": 0.029768303497000284,
+          "rmse": 0.06390655315300403,
+          "loss": 0.0245869603443017
+        },
+        "original512dev_secondary": {
+          "rows": 19968,
+          "mae": 0.03152461490744594,
+          "rmse": 0.06771385585483099
+        },
+        "learning_rate_first_step": 0.0004532584106825036,
+        "learning_rate_after_epoch": 0.0004020837130452019,
+        "elapsed_seconds": 6.548985242843628
+      },
+      {
+        "epoch": 18,
+        "optimizer_steps": 6552,
+        "optimizer_steps_in_epoch": 364,
+        "fit_online": {
+          "rows": 744354,
+          "loss": 0.019004923527650035,
+          "mae": 0.02442282673620076,
+          "rmse": 0.05105046640160359
+        },
+        "dev": {
+          "rows": 101751,
+          "mae": 0.029900278278168345,
+          "rmse": 0.06373138527578812,
+          "loss": 0.0247895180185548
+        },
+        "original512dev_secondary": {
+          "rows": 19968,
+          "mae": 0.03181753766925911,
+          "rmse": 0.0681987686667001
+        },
+        "learning_rate_first_step": 0.0004020837130452019,
+        "learning_rate_after_epoch": 0.0003520365877843945,
+        "elapsed_seconds": 6.503614187240601
+      },
+      {
+        "epoch": 19,
+        "optimizer_steps": 6916,
+        "optimizer_steps_in_epoch": 364,
+        "fit_online": {
+          "rows": 744354,
+          "loss": 0.018380038365151563,
+          "mae": 0.02373136941920571,
+          "rmse": 0.049993458554334534
+        },
+        "dev": {
+          "rows": 101751,
+          "mae": 0.03042028630546969,
+          "rmse": 0.0635257517279025,
+          "loss": 0.02446135498920219
+        },
+        "original512dev_secondary": {
+          "rows": 19968,
+          "mae": 0.03205142774578025,
+          "rmse": 0.06739493518057424
+        },
+        "learning_rate_first_step": 0.0003520365877843945,
+        "learning_rate_after_epoch": 0.0003036653616774734,
+        "elapsed_seconds": 6.635026931762695
+      },
+      {
+        "epoch": 20,
+        "optimizer_steps": 7280,
+        "optimizer_steps_in_epoch": 364,
+        "fit_online": {
+          "rows": 744354,
+          "loss": 0.017978554177169957,
+          "mae": 0.023316665459856122,
+          "rmse": 0.04908208206746422
+        },
+        "dev": {
+          "rows": 101751,
+          "mae": 0.031746316184994745,
+          "rmse": 0.06318924637348002,
+          "loss": 0.02491274783707859
+        },
+        "original512dev_secondary": {
+          "rows": 19968,
+          "mae": 0.03329090759360462,
+          "rmse": 0.06697555629260309
+        },
+        "learning_rate_first_step": 0.0003036653616774734,
+        "learning_rate_after_epoch": 0.00025749999999999574,
+        "elapsed_seconds": 6.6318652629852295
+      },
+      {
+        "epoch": 21,
+        "optimizer_steps": 7644,
+        "optimizer_steps_in_epoch": 364,
+        "fit_online": {
+          "rows": 744354,
+          "loss": 0.017396829781327307,
+          "mae": 0.022675323442288186,
+          "rmse": 0.048046840389243284
+        },
+        "dev": {
+          "rows": 101751,
+          "mae": 0.02939830306211538,
+          "rmse": 0.06345438944333362,
+          "loss": 0.024406630289095805
+        },
+        "original512dev_secondary": {
+          "rows": 19968,
+          "mae": 0.0309763493053255,
+          "rmse": 0.0674574181144763
+        },
+        "learning_rate_first_step": 0.00025749999999999574,
+        "learning_rate_after_epoch": 0.00021404630011522352,
+        "elapsed_seconds": 6.815945625305176
+      },
+      {
+        "epoch": 22,
+        "optimizer_steps": 8008,
+        "optimizer_steps_in_epoch": 364,
+        "fit_online": {
+          "rows": 744354,
+          "loss": 0.016972144893477533,
+          "mae": 0.022239596963642597,
+          "rmse": 0.04714710992821975
+        },
+        "dev": {
+          "rows": 101751,
+          "mae": 0.02975942849346404,
+          "rmse": 0.06295266432211538,
+          "loss": 0.024242054627334593
+        },
+        "original512dev_secondary": {
+          "rows": 19968,
+          "mae": 0.03125629304408962,
+          "rmse": 0.06674786337141295
+        },
+        "learning_rate_first_step": 0.00021404630011522352,
+        "learning_rate_after_epoch": 0.0001737803498523636,
+        "elapsed_seconds": 6.825618267059326
+      },
+      {
+        "epoch": 23,
+        "optimizer_steps": 8372,
+        "optimizer_steps_in_epoch": 364,
+        "fit_online": {
+          "rows": 744354,
+          "loss": 0.0164416371247407,
+          "mae": 0.021642145908876732,
+          "rmse": 0.04618773272936043
+        },
+        "dev": {
+          "rows": 101751,
+          "mae": 0.029994438424103,
+          "rmse": 0.06318560666431945,
+          "loss": 0.024248712056828426
+        },
+        "original512dev_secondary": {
+          "rows": 19968,
+          "mae": 0.031761655196133595,
+          "rmse": 0.06743455491437689
+        },
+        "learning_rate_first_step": 0.0001737803498523636,
+        "learning_rate_after_epoch": 0.00013714331138868892,
+        "elapsed_seconds": 6.648803949356079
+      },
+      {
+        "epoch": 24,
+        "optimizer_steps": 8736,
+        "optimizer_steps_in_epoch": 364,
+        "fit_online": {
+          "rows": 744354,
+          "loss": 0.01601903194422304,
+          "mae": 0.021178164970956006,
+          "rmse": 0.04538008375633202
+        },
+        "dev": {
+          "rows": 101751,
+          "mae": 0.02962571152068476,
+          "rmse": 0.06293438605005593,
+          "loss": 0.02410268729061644
+        },
+        "original512dev_secondary": {
+          "rows": 19968,
+          "mae": 0.03129297122585484,
+          "rmse": 0.06709166419974966
+        },
+        "learning_rate_first_step": 0.00013714331138868892,
+        "learning_rate_after_epoch": 0.0001045365877844009,
+        "elapsed_seconds": 6.518649339675903
+      },
+      {
+        "epoch": 25,
+        "optimizer_steps": 9100,
+        "optimizer_steps_in_epoch": 364,
+        "fit_online": {
+          "rows": 744354,
+          "loss": 0.01558180933259378,
+          "mae": 0.020689866971655247,
+          "rmse": 0.044555455546414206
+        },
+        "dev": {
+          "rows": 101751,
+          "mae": 0.029759433082169405,
+          "rmse": 0.06300754374431557,
+          "loss": 0.02404300468490347
+        },
+        "original512dev_secondary": {
+          "rows": 19968,
+          "mae": 0.0314143664954477,
+          "rmse": 0.06720902758169559
+        },
+        "learning_rate_first_step": 0.0001045365877844009,
+        "learning_rate_after_epoch": 7.631742512670258e-05,
+        "elapsed_seconds": 6.610796928405762
+      },
+      {
+        "epoch": 26,
+        "optimizer_steps": 9464,
+        "optimizer_steps_in_epoch": 364,
+        "fit_online": {
+          "rows": 744354,
+          "loss": 0.015234813825988177,
+          "mae": 0.02031605235873638,
+          "rmse": 0.04390374914812054
+        },
+        "dev": {
+          "rows": 101751,
+          "mae": 0.029315911042029078,
+          "rmse": 0.06289572256856334,
+          "loss": 0.02404446452596852
+        },
+        "original512dev_secondary": {
+          "rows": 19968,
+          "mae": 0.030892824772265088,
+          "rmse": 0.06685482913544216
+        },
+        "learning_rate_first_step": 7.631742512670258e-05,
+        "learning_rate_after_epoch": 5.2794998466912824e-05,
+        "elapsed_seconds": 6.651164531707764
+      },
+      {
+        "epoch": 27,
+        "optimizer_steps": 9828,
+        "optimizer_steps_in_epoch": 364,
+        "fit_online": {
+          "rows": 744354,
+          "loss": 0.014934488369213473,
+          "mae": 0.019977984324470913,
+          "rmse": 0.043346524504916585
+        },
+        "dev": {
+          "rows": 101751,
+          "mae": 0.029293083427212596,
+          "rmse": 0.06297385708578285,
+          "loss": 0.02397536895194945
+        },
+        "original512dev_secondary": {
+          "rows": 19968,
+          "mae": 0.030876006297533492,
+          "rmse": 0.06700469181220634
+        },
+        "learning_rate_first_step": 5.2794998466912824e-05,
+        "learning_rate_after_epoch": 3.4227024433899344e-05,
+        "elapsed_seconds": 6.609819412231445
+      },
+      {
+        "epoch": 28,
+        "optimizer_steps": 10192,
+        "optimizer_steps_in_epoch": 364,
+        "fit_online": {
+          "rows": 744354,
+          "loss": 0.014699707810430104,
+          "mae": 0.019716238948416565,
+          "rmse": 0.04294533034208814
+        },
+        "dev": {
+          "rows": 101751,
+          "mae": 0.029206813929558404,
+          "rmse": 0.06283869817800834,
+          "loss": 0.023940476112429353
+        },
+        "original512dev_secondary": {
+          "rows": 19968,
+          "mae": 0.03081663845405157,
+          "rmse": 0.06698770541350459
+        },
+        "learning_rate_first_step": 3.4227024433899344e-05,
+        "learning_rate_after_epoch": 2.0816937636766568e-05,
+        "elapsed_seconds": 6.760056018829346
+      },
+      {
+        "epoch": 29,
+        "optimizer_steps": 10556,
+        "optimizer_steps_in_epoch": 364,
+        "fit_online": {
+          "rows": 744354,
+          "loss": 0.014536236776051441,
+          "mae": 0.019533188947923816,
+          "rmse": 0.04264248388363534
+        },
+        "dev": {
+          "rows": 101751,
+          "mae": 0.029332996838254174,
+          "rmse": 0.06301263575268187,
+          "loss": 0.02397848307031828
+        },
+        "original512dev_secondary": {
+          "rows": 19968,
+          "mae": 0.030904083734173745,
+          "rmse": 0.06708371948361569
+        },
+        "learning_rate_first_step": 2.0816937636766568e-05,
+        "learning_rate_after_epoch": 1.2711661792704754e-05,
+        "elapsed_seconds": 6.730714321136475
+      },
+      {
+        "epoch": 30,
+        "optimizer_steps": 10920,
+        "optimizer_steps_in_epoch": 364,
+        "fit_online": {
+          "rows": 744354,
+          "loss": 0.01444127694407146,
+          "mae": 0.019427839926005337,
+          "rmse": 0.04248720846249908
+        },
+        "dev": {
+          "rows": 101751,
+          "mae": 0.02936776782157642,
+          "rmse": 0.06302819715282469,
+          "loss": 0.024012600424272692
+        },
+        "original512dev_secondary": {
+          "rows": 19968,
+          "mae": 0.030989496129502817,
+          "rmse": 0.06719197308785259
+        },
+        "learning_rate_first_step": 1.2711661792704754e-05,
+        "learning_rate_after_epoch": 1e-05,
+        "elapsed_seconds": 6.60991907119751
+      }
+    ],
+    "old_logit_residual_l1": [
+      {
+        "epoch": 1,
+        "optimizer_steps": 364,
+        "optimizer_steps_in_epoch": 364,
+        "fit_online": {
+          "rows": 744354,
+          "loss": 0.03481446064339268,
+          "mae": 0.03481446061292883,
+          "rmse": 0.0730040017759882
+        },
+        "dev": {
+          "rows": 101751,
+          "mae": 0.03299481767693684,
+          "rmse": 0.07059542516164026,
+          "loss": 0.03299481755349005
+        },
+        "original512dev_secondary": {
+          "rows": 19968,
+          "mae": 0.03469992961051433,
+          "rmse": 0.07408208077490051
+        },
+        "learning_rate_first_step": 0.001,
+        "learning_rate_after_epoch": 0.0009972883382072955,
+        "elapsed_seconds": 7.757522106170654
+      },
+      {
+        "epoch": 2,
+        "optimizer_steps": 728,
+        "optimizer_steps_in_epoch": 364,
+        "fit_online": {
+          "rows": 744354,
+          "loss": 0.031060462883822332,
+          "mae": 0.031060462938416987,
+          "rmse": 0.06685905118190176
+        },
+        "dev": {
+          "rows": 101751,
+          "mae": 0.03156582953558207,
+          "rmse": 0.06833865898385816,
+          "loss": 0.03156582966648681
+        },
+        "original512dev_secondary": {
+          "rows": 19968,
+          "mae": 0.032986725736429016,
+          "rmse": 0.07102488140843116
+        },
+        "learning_rate_first_step": 0.0009972883382072955,
+        "learning_rate_after_epoch": 0.0009891830623632333,
+        "elapsed_seconds": 7.819254398345947
+      },
+      {
+        "epoch": 3,
+        "optimizer_steps": 1092,
+        "optimizer_steps_in_epoch": 364,
+        "fit_online": {
+          "rows": 744354,
+          "loss": 0.029707936506463247,
+          "mae": 0.029707936531007943,
+          "rmse": 0.06445756471898705
+        },
+        "dev": {
+          "rows": 101751,
+          "mae": 0.03106128174225605,
+          "rmse": 0.06851477891025416,
+          "loss": 0.031061281799671697
+        },
+        "original512dev_secondary": {
+          "rows": 19968,
+          "mae": 0.032734806643300265,
+          "rmse": 0.07232909051695112
+        },
+        "learning_rate_first_step": 0.0009891830623632333,
+        "learning_rate_after_epoch": 0.0009757729755660874,
+        "elapsed_seconds": 7.789494276046753
+      },
+      {
+        "epoch": 4,
+        "optimizer_steps": 1456,
+        "optimizer_steps_in_epoch": 364,
+        "fit_online": {
+          "rows": 744354,
+          "loss": 0.0286489580970104,
+          "mae": 0.028648957996681157,
+          "rmse": 0.06247450215353599
+        },
+        "dev": {
+          "rows": 101751,
+          "mae": 0.030123195480900672,
+          "rmse": 0.06559044494160804,
+          "loss": 0.030123195548727805
+        },
+        "original512dev_secondary": {
+          "rows": 19968,
+          "mae": 0.03183509249526814,
+          "rmse": 0.06891011237844373
+        },
+        "learning_rate_first_step": 0.0009757729755660874,
+        "learning_rate_after_epoch": 0.0009572050015330751,
+        "elapsed_seconds": 7.765609502792358
+      },
+      {
+        "epoch": 5,
+        "optimizer_steps": 1820,
+        "optimizer_steps_in_epoch": 364,
+        "fit_online": {
+          "rows": 744354,
+          "loss": 0.027931789214890663,
+          "mae": 0.027931789240682164,
+          "rmse": 0.06111932540333608
+        },
+        "dev": {
+          "rows": 101751,
+          "mae": 0.02976028240919939,
+          "rmse": 0.06599237980128285,
+          "loss": 0.02976028210573108
+        },
+        "original512dev_secondary": {
+          "rows": 19968,
+          "mae": 0.031365831178554335,
+          "rmse": 0.06928439647689201
+        },
+        "learning_rate_first_step": 0.0009572050015330751,
+        "learning_rate_after_epoch": 0.0009336825748732852,
+        "elapsed_seconds": 7.791011333465576
+      },
+      {
+        "epoch": 6,
+        "optimizer_steps": 2184,
+        "optimizer_steps_in_epoch": 364,
+        "fit_online": {
+          "rows": 744354,
+          "loss": 0.02718501476554153,
+          "mae": 0.027185014664494578,
+          "rmse": 0.05970387750019782
+        },
+        "dev": {
+          "rows": 101751,
+          "mae": 0.03005690405853144,
+          "rmse": 0.06612017628452022,
+          "loss": 0.03005690412359667
+        },
+        "original512dev_secondary": {
+          "rows": 19968,
+          "mae": 0.031731648258056126,
+          "rmse": 0.0696032650185473
+        },
+        "learning_rate_first_step": 0.0009336825748732852,
+        "learning_rate_after_epoch": 0.0009054634122155876,
+        "elapsed_seconds": 7.673263788223267
+      },
+      {
+        "epoch": 7,
+        "optimizer_steps": 2548,
+        "optimizer_steps_in_epoch": 364,
+        "fit_online": {
+          "rows": 744354,
+          "loss": 0.026625435780442398,
+          "mae": 0.026625435814524312,
+          "rmse": 0.05865112478860799
+        },
+        "dev": {
+          "rows": 101751,
+          "mae": 0.02974400511569165,
+          "rmse": 0.06611777242134909,
+          "loss": 0.02974400507193508
+        },
+        "original512dev_secondary": {
+          "rows": 19968,
+          "mae": 0.031140867924186282,
+          "rmse": 0.06892496845064283
+        },
+        "learning_rate_first_step": 0.0009054634122155876,
+        "learning_rate_after_epoch": 0.0008728566886112981,
+        "elapsed_seconds": 7.59534478187561
+      },
+      {
+        "epoch": 8,
+        "optimizer_steps": 2912,
+        "optimizer_steps_in_epoch": 364,
+        "fit_online": {
+          "rows": 744354,
+          "loss": 0.026082294680990114,
+          "mae": 0.026082294580648164,
+          "rmse": 0.05743067348337614
+        },
+        "dev": {
+          "rows": 101751,
+          "mae": 0.029678800829418334,
+          "rmse": 0.06548951715053522,
+          "loss": 0.029678800815027884
+        },
+        "original512dev_secondary": {
+          "rows": 19968,
+          "mae": 0.030991354288295414,
+          "rmse": 0.06818187465847103
+        },
+        "learning_rate_first_step": 0.0008728566886112981,
+        "learning_rate_after_epoch": 0.0008362196501476234,
+        "elapsed_seconds": 7.62297797203064
+      },
+      {
+        "epoch": 9,
+        "optimizer_steps": 3276,
+        "optimizer_steps_in_epoch": 364,
+        "fit_online": {
+          "rows": 744354,
+          "loss": 0.025459979443731406,
+          "mae": 0.02545997951021908,
+          "rmse": 0.056300124642473824
+        },
+        "dev": {
+          "rows": 101751,
+          "mae": 0.029693151416301935,
+          "rmse": 0.06551920005568991,
+          "loss": 0.02969315180578802
+        },
+        "original512dev_secondary": {
+          "rows": 19968,
+          "mae": 0.031028917735791486,
+          "rmse": 0.06797774348932002
+        },
+        "learning_rate_first_step": 0.0008362196501476234,
+        "learning_rate_after_epoch": 0.0007959536998847633,
+        "elapsed_seconds": 7.619675397872925
+      },
+      {
+        "epoch": 10,
+        "optimizer_steps": 3640,
+        "optimizer_steps_in_epoch": 364,
+        "fit_online": {
+          "rows": 744354,
+          "loss": 0.024876128794296685,
+          "mae": 0.024876128676543714,
+          "rmse": 0.05513206253376371
+        },
+        "dev": {
+          "rows": 101751,
+          "mae": 0.02935671928629687,
+          "rmse": 0.06500197130692403,
+          "loss": 0.02935671953485716
+        },
+        "original512dev_secondary": {
+          "rows": 19968,
+          "mae": 0.030810788450416553,
+          "rmse": 0.06778502062396369
+        },
+        "learning_rate_first_step": 0.0007959536998847633,
+        "learning_rate_after_epoch": 0.0007524999999999889,
+        "elapsed_seconds": 7.62933611869812
+      },
+      {
+        "epoch": 11,
+        "optimizer_steps": 4004,
+        "optimizer_steps_in_epoch": 364,
+        "fit_online": {
+          "rows": 744354,
+          "loss": 0.024322330474094357,
+          "mae": 0.024322330506510302,
+          "rmse": 0.05384919733564249
+        },
+        "dev": {
+          "rows": 101751,
+          "mae": 0.02916658760889948,
+          "rmse": 0.06500479510271906,
+          "loss": 0.029166587656294922
+        },
+        "original512dev_secondary": {
+          "rows": 19968,
+          "mae": 0.030709498327263274,
+          "rmse": 0.06853684029076144
+        },
+        "learning_rate_first_step": 0.0007524999999999889,
+        "learning_rate_after_epoch": 0.0007063346383225108,
+        "elapsed_seconds": 7.684103012084961
+      },
+      {
+        "epoch": 12,
+        "optimizer_steps": 4368,
+        "optimizer_steps_in_epoch": 364,
+        "fit_online": {
+          "rows": 744354,
+          "loss": 0.02373968785114195,
+          "mae": 0.02373968784567448,
+          "rmse": 0.052731430173156645
+        },
+        "dev": {
+          "rows": 101751,
+          "mae": 0.029196837861408226,
+          "rmse": 0.06494523188043477,
+          "loss": 0.029196838050951246
+        },
+        "original512dev_secondary": {
+          "rows": 19968,
+          "mae": 0.030645916094225612,
+          "rmse": 0.06855661945255195
+        },
+        "learning_rate_first_step": 0.0007063346383225108,
+        "learning_rate_after_epoch": 0.00065796341221559,
+        "elapsed_seconds": 7.642890691757202
+      },
+      {
+        "epoch": 13,
+        "optimizer_steps": 4732,
+        "optimizer_steps_in_epoch": 364,
+        "fit_online": {
+          "rows": 744354,
+          "loss": 0.0232419811471031,
+          "mae": 0.023241981102973555,
+          "rmse": 0.05170778556391345
+        },
+        "dev": {
+          "rows": 101751,
+          "mae": 0.02914001260818683,
+          "rmse": 0.06476283975864496,
+          "loss": 0.029140012961140316
+        },
+        "original512dev_secondary": {
+          "rows": 19968,
+          "mae": 0.030671346487475278,
+          "rmse": 0.068352993585362
+        },
+        "learning_rate_first_step": 0.00065796341221559,
+        "learning_rate_after_epoch": 0.0006079162869547829,
+        "elapsed_seconds": 7.669292449951172
+      },
+      {
+        "epoch": 14,
+        "optimizer_steps": 5096,
+        "optimizer_steps_in_epoch": 364,
+        "fit_online": {
+          "rows": 744354,
+          "loss": 0.02266253257992191,
+          "mae": 0.02266253264523388,
+          "rmse": 0.05046753437399855
+        },
+        "dev": {
+          "rows": 101751,
+          "mae": 0.029047732672827266,
+          "rmse": 0.06538319999127858,
+          "loss": 0.029047732687502815
+        },
+        "original512dev_secondary": {
+          "rows": 19968,
+          "mae": 0.030766814015950016,
+          "rmse": 0.06928770702969553
+        },
+        "learning_rate_first_step": 0.0006079162869547829,
+        "learning_rate_after_epoch": 0.0005567415893174813,
+        "elapsed_seconds": 7.552573919296265
+      },
+      {
+        "epoch": 15,
+        "optimizer_steps": 5460,
+        "optimizer_steps_in_epoch": 364,
+        "fit_online": {
+          "rows": 744354,
+          "loss": 0.022143988336947987,
+          "mae": 0.022143988410477544,
+          "rmse": 0.049423678298396626
+        },
+        "dev": {
+          "rows": 101751,
+          "mae": 0.028972709301376416,
+          "rmse": 0.06500418999183508,
+          "loss": 0.028972709311408128
+        },
+        "original512dev_secondary": {
+          "rows": 19968,
+          "mae": 0.030453159349990645,
+          "rmse": 0.06846674350454049
+        },
+        "learning_rate_first_step": 0.0005567415893174813,
+        "learning_rate_after_epoch": 0.0005049999999999935,
+        "elapsed_seconds": 7.754409551620483
+      },
+      {
+        "epoch": 16,
+        "optimizer_steps": 5824,
+        "optimizer_steps_in_epoch": 364,
+        "fit_online": {
+          "rows": 744354,
+          "loss": 0.021562574119115156,
+          "mae": 0.021562574063877862,
+          "rmse": 0.0482056973534981
+        },
+        "dev": {
+          "rows": 101751,
+          "mae": 0.029189051260162822,
+          "rmse": 0.06515704770316376,
+          "loss": 0.02918905100837029
+        },
+        "original512dev_secondary": {
+          "rows": 19968,
+          "mae": 0.03074945419456153,
+          "rmse": 0.06940001506691877
+        },
+        "learning_rate_first_step": 0.0005049999999999935,
+        "learning_rate_after_epoch": 0.0004532584106825036,
+        "elapsed_seconds": 7.691293716430664
+      },
+      {
+        "epoch": 17,
+        "optimizer_steps": 6188,
+        "optimizer_steps_in_epoch": 364,
+        "fit_online": {
+          "rows": 744354,
+          "loss": 0.0210544942690692,
+          "mae": 0.021054494244301593,
+          "rmse": 0.0471572618076133
+        },
+        "dev": {
+          "rows": 101751,
+          "mae": 0.028990073991517387,
+          "rmse": 0.06478903465378548,
+          "loss": 0.028990073841360223
+        },
+        "original512dev_secondary": {
+          "rows": 19968,
+          "mae": 0.03076192641202187,
+          "rmse": 0.06868439465954321
+        },
+        "learning_rate_first_step": 0.0004532584106825036,
+        "learning_rate_after_epoch": 0.0004020837130452019,
+        "elapsed_seconds": 7.645296812057495
+      },
+      {
+        "epoch": 18,
+        "optimizer_steps": 6552,
+        "optimizer_steps_in_epoch": 364,
+        "fit_online": {
+          "rows": 744354,
+          "loss": 0.02053506853127654,
+          "mae": 0.020535068530930473,
+          "rmse": 0.046072735966283335
+        },
+        "dev": {
+          "rows": 101751,
+          "mae": 0.029060250222782528,
+          "rmse": 0.06470247315214002,
+          "loss": 0.02906025032858194
+        },
+        "original512dev_secondary": {
+          "rows": 19968,
+          "mae": 0.03083196240122969,
+          "rmse": 0.06900320634156627
+        },
+        "learning_rate_first_step": 0.0004020837130452019,
+        "learning_rate_after_epoch": 0.0003520365877843945,
+        "elapsed_seconds": 7.6619203090667725
+      },
+      {
+        "epoch": 19,
+        "optimizer_steps": 6916,
+        "optimizer_steps_in_epoch": 364,
+        "fit_online": {
+          "rows": 744354,
+          "loss": 0.01997941350861066,
+          "mae": 0.019979413379486912,
+          "rmse": 0.04505860982077838
+        },
+        "dev": {
+          "rows": 101751,
+          "mae": 0.02908882497592276,
+          "rmse": 0.06488960056421834,
+          "loss": 0.029088824967384828
+        },
+        "original512dev_secondary": {
+          "rows": 19968,
+          "mae": 0.030701644224141324,
+          "rmse": 0.06859208741340594
+        },
+        "learning_rate_first_step": 0.0003520365877843945,
+        "learning_rate_after_epoch": 0.0003036653616774734,
+        "elapsed_seconds": 7.637484312057495
+      },
+      {
+        "epoch": 20,
+        "optimizer_steps": 7280,
+        "optimizer_steps_in_epoch": 364,
+        "fit_online": {
+          "rows": 744354,
+          "loss": 0.01952760834880823,
+          "mae": 0.019527608308918663,
+          "rmse": 0.0441231873645619
+        },
+        "dev": {
+          "rows": 101751,
+          "mae": 0.029121246316653762,
+          "rmse": 0.06517525870498869,
+          "loss": 0.029121245969793822
+        },
+        "original512dev_secondary": {
+          "rows": 19968,
+          "mae": 0.03060992861995624,
+          "rmse": 0.06895463831655804
+        },
+        "learning_rate_first_step": 0.0003036653616774734,
+        "learning_rate_after_epoch": 0.00025749999999999574,
+        "elapsed_seconds": 7.6443846225738525
+      },
+      {
+        "epoch": 21,
+        "optimizer_steps": 7644,
+        "optimizer_steps_in_epoch": 364,
+        "fit_online": {
+          "rows": 744354,
+          "loss": 0.0190419192889227,
+          "mae": 0.019041919303283562,
+          "rmse": 0.04319554731251824
+        },
+        "dev": {
+          "rows": 101751,
+          "mae": 0.02903850899664967,
+          "rmse": 0.065012294230067,
+          "loss": 0.029038509146278108
+        },
+        "original512dev_secondary": {
+          "rows": 19968,
+          "mae": 0.03058071618365159,
+          "rmse": 0.06884989986713619
+        },
+        "learning_rate_first_step": 0.00025749999999999574,
+        "learning_rate_after_epoch": 0.00021404630011522352,
+        "elapsed_seconds": 7.524526357650757
+      },
+      {
+        "epoch": 22,
+        "optimizer_steps": 8008,
+        "optimizer_steps_in_epoch": 364,
+        "fit_online": {
+          "rows": 744354,
+          "loss": 0.018618349864931067,
+          "mae": 0.018618350003019757,
+          "rmse": 0.04243386904658221
+        },
+        "dev": {
+          "rows": 101751,
+          "mae": 0.029186370482013387,
+          "rmse": 0.06504120222727702,
+          "loss": 0.029186370622295017
+        },
+        "original512dev_secondary": {
+          "rows": 19968,
+          "mae": 0.03069438986266394,
+          "rmse": 0.06862637660820091
+        },
+        "learning_rate_first_step": 0.00021404630011522352,
+        "learning_rate_after_epoch": 0.0001737803498523636,
+        "elapsed_seconds": 7.729598045349121
+      },
+      {
+        "epoch": 23,
+        "optimizer_steps": 8372,
+        "optimizer_steps_in_epoch": 364,
+        "fit_online": {
+          "rows": 744354,
+          "loss": 0.01821716064288428,
+          "mae": 0.01821716060081166,
+          "rmse": 0.0416880601203598
+        },
+        "dev": {
+          "rows": 101751,
+          "mae": 0.02925581866464478,
+          "rmse": 0.06538072141430383,
+          "loss": 0.029255818921822115
+        },
+        "original512dev_secondary": {
+          "rows": 19968,
+          "mae": 0.030964746983582286,
+          "rmse": 0.06948671875551604
+        },
+        "learning_rate_first_step": 0.0001737803498523636,
+        "learning_rate_after_epoch": 0.00013714331138868892,
+        "elapsed_seconds": 7.804940938949585
+      },
+      {
+        "epoch": 24,
+        "optimizer_steps": 8736,
+        "optimizer_steps_in_epoch": 364,
+        "fit_online": {
+          "rows": 744354,
+          "loss": 0.017871890911682203,
+          "mae": 0.017871890964279533,
+          "rmse": 0.041017160853699546
+        },
+        "dev": {
+          "rows": 101751,
+          "mae": 0.029401524923901325,
+          "rmse": 0.06562146012549949,
+          "loss": 0.02940152470856494
+        },
+        "original512dev_secondary": {
+          "rows": 19968,
+          "mae": 0.031078706623975886,
+          "rmse": 0.06973338530252332
+        },
+        "learning_rate_first_step": 0.00013714331138868892,
+        "learning_rate_after_epoch": 0.0001045365877844009,
+        "elapsed_seconds": 7.790158987045288
+      },
+      {
+        "epoch": 25,
+        "optimizer_steps": 9100,
+        "optimizer_steps_in_epoch": 364,
+        "fit_online": {
+          "rows": 744354,
+          "loss": 0.01755823062772188,
+          "mae": 0.017558230623266516,
+          "rmse": 0.04049282252793988
+        },
+        "dev": {
+          "rows": 101751,
+          "mae": 0.029272923090530274,
+          "rmse": 0.06541096760378806,
+          "loss": 0.02927292316348467
+        },
+        "original512dev_secondary": {
+          "rows": 19968,
+          "mae": 0.030785133642134557,
+          "rmse": 0.06901292402677386
+        },
+        "learning_rate_first_step": 0.0001045365877844009,
+        "learning_rate_after_epoch": 7.631742512670258e-05,
+        "elapsed_seconds": 7.525925636291504
+      },
+      {
+        "epoch": 26,
+        "optimizer_steps": 9464,
+        "optimizer_steps_in_epoch": 364,
+        "fit_online": {
+          "rows": 744354,
+          "loss": 0.017305670650632127,
+          "mae": 0.017305670705870975,
+          "rmse": 0.040060089430110045
+        },
+        "dev": {
+          "rows": 101751,
+          "mae": 0.029269261772952814,
+          "rmse": 0.06547611127228035,
+          "loss": 0.02926926211947877
+        },
+        "original512dev_secondary": {
+          "rows": 19968,
+          "mae": 0.030958537875298984,
+          "rmse": 0.06970337380407728
+        },
+        "learning_rate_first_step": 7.631742512670258e-05,
+        "learning_rate_after_epoch": 5.2794998466912824e-05,
+        "elapsed_seconds": 7.559709548950195
+      },
+      {
+        "epoch": 27,
+        "optimizer_steps": 9828,
+        "optimizer_steps_in_epoch": 364,
+        "fit_online": {
+          "rows": 744354,
+          "loss": 0.017082581930652346,
+          "mae": 0.0170825819079003,
+          "rmse": 0.03971982099104555
+        },
+        "dev": {
+          "rows": 101751,
+          "mae": 0.0292648801470158,
+          "rmse": 0.06556179721336927,
+          "loss": 0.02926487998161318
+        },
+        "original512dev_secondary": {
+          "rows": 19968,
+          "mae": 0.030859260240357747,
+          "rmse": 0.06940307820811487
+        },
+        "learning_rate_first_step": 5.2794998466912824e-05,
+        "learning_rate_after_epoch": 3.4227024433899344e-05,
+        "elapsed_seconds": 7.667202949523926
+      },
+      {
+        "epoch": 28,
+        "optimizer_steps": 10192,
+        "optimizer_steps_in_epoch": 364,
+        "fit_online": {
+          "rows": 744354,
+          "loss": 0.016923388635754162,
+          "mae": 0.016923388641118874,
+          "rmse": 0.039470112717279515
+        },
+        "dev": {
+          "rows": 101751,
+          "mae": 0.02930065795798157,
+          "rmse": 0.06554800066110622,
+          "loss": 0.029300657722397254
+        },
+        "original512dev_secondary": {
+          "rows": 19968,
+          "mae": 0.030910359731462788,
+          "rmse": 0.06947266565299481
+        },
+        "learning_rate_first_step": 3.4227024433899344e-05,
+        "learning_rate_after_epoch": 2.0816937636766568e-05,
+        "elapsed_seconds": 7.7496254444122314
+      },
+      {
+        "epoch": 29,
+        "optimizer_steps": 10556,
+        "optimizer_steps_in_epoch": 364,
+        "fit_online": {
+          "rows": 744354,
+          "loss": 0.016809408890105768,
+          "mae": 0.016809408877141933,
+          "rmse": 0.039319504346044885
+        },
+        "dev": {
+          "rows": 101751,
+          "mae": 0.029335801114054524,
+          "rmse": 0.06564822581344662,
+          "loss": 0.029335801168977337
+        },
+        "original512dev_secondary": {
+          "rows": 19968,
+          "mae": 0.030945132475546318,
+          "rmse": 0.0695938068576658
+        },
+        "learning_rate_first_step": 2.0816937636766568e-05,
+        "learning_rate_after_epoch": 1.2711661792704754e-05,
+        "elapsed_seconds": 7.627472400665283
+      },
+      {
+        "epoch": 30,
+        "optimizer_steps": 10920,
+        "optimizer_steps_in_epoch": 364,
+        "fit_online": {
+          "rows": 744354,
+          "loss": 0.016741896141221124,
+          "mae": 0.01674189615757177,
+          "rmse": 0.039221067196786336
+        },
+        "dev": {
+          "rows": 101751,
+          "mae": 0.029316811733385552,
+          "rmse": 0.06556778362492445,
+          "loss": 0.0293168119494253
+        },
+        "original512dev_secondary": {
+          "rows": 19968,
+          "mae": 0.030923395721146107,
+          "rmse": 0.06945634961756057
+        },
+        "learning_rate_first_step": 1.2711661792704754e-05,
+        "learning_rate_after_epoch": 1e-05,
+        "elapsed_seconds": 7.530182838439941
+      }
+    ]
+  }
+}

reproduction/kermany-v7-masks.tar.gz

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:579c8c0902b8c94e5688dffcad3811cc767f7529fdbcb98e9cb45b0a80814d0a
+size 37182535