Add reviewer-requested controlled and sensor mechanism tests
Browse filesThis view is limited to 50 files because it contains too many changes. See raw diff
- README.md +38 -0
- SHA256SUMS.json +117 -2
- revision_r1/controlled/compute.json +33 -0
- revision_r1/controlled/family_sample_frontier.csv +25 -0
- revision_r1/controlled/final_audit.json +34 -0
- revision_r1/controlled/frontier_all_conditions.csv +0 -0
- revision_r1/controlled/indices.json +466 -0
- revision_r1/controlled/native_worlds.csv.gz +3 -0
- revision_r1/controlled/numeric_compute.json +9 -0
- revision_r1/controlled/numeric_diagnostic.json +154 -0
- revision_r1/controlled/numeric_push_sensitivity.csv.gz +3 -0
- revision_r1/controlled/paired_by_condition.csv +0 -0
- revision_r1/controlled/privileged_geometry.csv +0 -0
- revision_r1/controlled/reference_replay_discrepancies.csv.gz +3 -0
- revision_r1/controlled/replay_discrepancies.csv.gz +3 -0
- revision_r1/controlled/run.py +572 -0
- revision_r1/controlled/sample_frontier.csv +5 -0
- revision_r1/controlled/specification.json +108 -0
- revision_r1/controlled/specification_01.json +88 -0
- revision_r1/controlled/specification_02.json +96 -0
- revision_r1/controlled/summary.json +949 -0
- revision_r1/controlled/timing.json +0 -0
- revision_r1/external/article_table.csv +20 -0
- revision_r1/external/execution.json +24 -0
- revision_r1/external/implementation_correction.json +14 -0
- revision_r1/external/metrics.csv.gz +3 -0
- revision_r1/external/overall.json +0 -0
- revision_r1/external/paired_contrasts.json +1338 -0
- revision_r1/external/run.py +515 -0
- revision_r1/external/shift_by_task.json +743 -0
- revision_r1/external/shift_overall.json +102 -0
- revision_r1/external/specification.json +123 -0
- revision_r1/external/table_provenance.json +10 -0
- revision_r1/sensor/analysis_spec.json +79 -0
- revision_r1/sensor/export_safe.py +39 -0
- revision_r1/sensor/findings.json +123 -0
- revision_r1/sensor/full_replay.json +39 -0
- revision_r1/sensor/objective_extension.py +63 -0
- revision_r1/sensor/objective_extension_01/cell_compute.json +146 -0
- revision_r1/sensor/objective_extension_01/compute.json +17 -0
- revision_r1/sensor/objective_extension_01/hydraulic_n128_s20751.json +1 -0
- revision_r1/sensor/objective_extension_01/hydraulic_n128_s20751.safetensors +3 -0
- revision_r1/sensor/objective_extension_01/hydraulic_n128_s20752.json +1 -0
- revision_r1/sensor/objective_extension_01/hydraulic_n128_s20752.safetensors +3 -0
- revision_r1/sensor/objective_extension_01/hydraulic_n128_s20753.json +1 -0
- revision_r1/sensor/objective_extension_01/hydraulic_n128_s20753.safetensors +3 -0
- revision_r1/sensor/objective_extension_01/hydraulic_n64_s20751.json +1 -0
- revision_r1/sensor/objective_extension_01/hydraulic_n64_s20751.safetensors +3 -0
- revision_r1/sensor/objective_extension_01/hydraulic_n64_s20752.json +1 -0
- revision_r1/sensor/objective_extension_01/hydraulic_n64_s20752.safetensors +3 -0
README.md
CHANGED
|
@@ -63,3 +63,41 @@ An action-value compression control, a complete 1,212-episode partition and norm
|
|
| 63 |
## Data and license
|
| 64 |
|
| 65 |
Code is MIT; original models, generated data and research outputs are CC BY 4.0. Hydraulic features derive from Helwig, Pignanelli and Schuetze (UCI DOI 10.24432/C5CW21); temperature-modulated sensor features derive from Burgues (10.24432/C5S302). Gas-drift recordings (Vergara, 10.24432/C5RP6W) are retrieved separately and are not redistributed; preserve the source's legacy research-only notice. Full attribution and terms are in LICENSE.md.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 63 |
## Data and license
|
| 64 |
|
| 65 |
Code is MIT; original models, generated data and research outputs are CC BY 4.0. Hydraulic features derive from Helwig, Pignanelli and Schuetze (UCI DOI 10.24432/C5CW21); temperature-modulated sensor features derive from Burgues (10.24432/C5S302). Gas-drift recordings (Vergara, 10.24432/C5RP6W) are retrieved separately and are not redistributed; preserve the source's legacy research-only notice. Full attribution and terms are in LICENSE.md.
|
| 66 |
+
|
| 67 |
+
## Reviewer-revision experiments
|
| 68 |
+
|
| 69 |
+
`revision_r1/` adds exploratory mechanism tests on the already opened data:
|
| 70 |
+
full controlled action-value/loss comparisons, external sample-support and
|
| 71 |
+
fixed-policy transfer comparisons, and 318 sensor fits with all seeds and
|
| 72 |
+
budgets retained. Original results, protocols and promotion decisions are unchanged.
|
| 73 |
+
The 27 sensor state bundles use only Safetensors and JSON. Specifications,
|
| 74 |
+
all comparative scores, cost records and numerical tie-sensitivity diagnostics
|
| 75 |
+
are included; these are additional analyses, not new confirmation.
|
| 76 |
+
|
| 77 |
+
Replay every added sensor state without fitting:
|
| 78 |
+
|
| 79 |
+
```bash
|
| 80 |
+
python revision_r1/sensor/verify_all.py --release . --base revision_r1/sensor/run_01 --extension revision_r1/sensor/objective_extension_01 --output revision_sensor_replay.json --device cuda:0
|
| 81 |
+
```
|
| 82 |
+
|
| 83 |
+
Recompute the controlled interventions in a new directory:
|
| 84 |
+
|
| 85 |
+
```bash
|
| 86 |
+
python revision_r1/controlled/run.py --prepare --release . --output new_controlled --device cuda:0
|
| 87 |
+
python revision_r1/controlled/run.py --release . --output new_controlled --device cuda:0
|
| 88 |
+
```
|
| 89 |
+
|
| 90 |
+
The external analysis requires separately acquired gas-drift recordings. Retain
|
| 91 |
+
the provider's research-use notice; no transformed gas records are redistributed.
|
| 92 |
+
|
| 93 |
+
```bash
|
| 94 |
+
python rebuild_external.py --output new_external_source --device cuda:0
|
| 95 |
+
python revision_r1/external/run.py specify --release . --input new_external_source --output new_external_analysis --device cuda:0
|
| 96 |
+
python revision_r1/external/run.py run --release . --input new_external_source --output new_external_analysis --device cuda:0
|
| 97 |
+
```
|
| 98 |
+
|
| 99 |
+
Sensor refitting is optional through `revision_r1/sensor/review_r1_sensor.py`
|
| 100 |
+
and `objective_extension.py`, each accepting `--release .`, `--output` and
|
| 101 |
+
`--device`. Use a new output directory. Rank gains do not establish calibrated
|
| 102 |
+
acceptance, and sparse fitted-law ties can affect ancillary realized utilities;
|
| 103 |
+
the unchanged primary mixture frontiers and all disclosed discrepancies remain available.
|
SHA256SUMS.json
CHANGED
|
@@ -2,7 +2,7 @@
|
|
| 2 |
".gitattributes": "0aad2046ff3535fdb3750ad77100b4f6aed80009c097ffc8a9e264a5d1b0c4e7",
|
| 3 |
"ENVIRONMENT.json": "786e73bb84b1300d1b5062961af74aacbab2bbfe1e12675ba6ceebd12d26b0eb",
|
| 4 |
"LICENSE.md": "4a8b21f5c8ee1383fdf12d8622df51066334d446513d7318ab8a31763f435813",
|
| 5 |
-
"README.md": "
|
| 6 |
"analyze.py": "3e2ee19da7cdec5b735db55a368132977d8aa6cfe4cfd8e49dbe86019a56a444",
|
| 7 |
"data/controlled/manifest.json": "2b50155e7f4d198449e2499dad21ba82d51e94423a1711eca8de4a01ab9ca156",
|
| 8 |
"data/controlled/worlds.json": "860e154c4ffd0690080ab8c263756d4340cc119fffa66e1cc41d1b42ac3848cb",
|
|
@@ -356,5 +356,120 @@
|
|
| 356 |
"unionmind/predictive_transfer/__init__.py": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
|
| 357 |
"unionmind/predictive_transfer/data.py": "259202796bf921db2beaa75166809e455e44aaddef73da96bdec970067abfbd6",
|
| 358 |
"unionmind/supported_observations/__init__.py": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
|
| 359 |
-
"unionmind/supported_observations/data.py": "d359f27c4cd23482ed4ef8326d3026c10b19caf9ba75b3da677249f5d9c9e43f"
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 360 |
}
|
|
|
|
| 2 |
".gitattributes": "0aad2046ff3535fdb3750ad77100b4f6aed80009c097ffc8a9e264a5d1b0c4e7",
|
| 3 |
"ENVIRONMENT.json": "786e73bb84b1300d1b5062961af74aacbab2bbfe1e12675ba6ceebd12d26b0eb",
|
| 4 |
"LICENSE.md": "4a8b21f5c8ee1383fdf12d8622df51066334d446513d7318ab8a31763f435813",
|
| 5 |
+
"README.md": "ba4c2be9b705ec3b49377003e9b9d6819efd8b8c654dd9a4eb125ac1fb7fb389",
|
| 6 |
"analyze.py": "3e2ee19da7cdec5b735db55a368132977d8aa6cfe4cfd8e49dbe86019a56a444",
|
| 7 |
"data/controlled/manifest.json": "2b50155e7f4d198449e2499dad21ba82d51e94423a1711eca8de4a01ab9ca156",
|
| 8 |
"data/controlled/worlds.json": "860e154c4ffd0690080ab8c263756d4340cc119fffa66e1cc41d1b42ac3848cb",
|
|
|
|
| 356 |
"unionmind/predictive_transfer/__init__.py": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
|
| 357 |
"unionmind/predictive_transfer/data.py": "259202796bf921db2beaa75166809e455e44aaddef73da96bdec970067abfbd6",
|
| 358 |
"unionmind/supported_observations/__init__.py": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
|
| 359 |
+
"unionmind/supported_observations/data.py": "d359f27c4cd23482ed4ef8326d3026c10b19caf9ba75b3da677249f5d9c9e43f",
|
| 360 |
+
"revision_r1/sensor/primary_table.csv": "4503de189e284a9c5a31bc144c3bdff69e75fdbf2bfd8a5d434bf74f7128e55b",
|
| 361 |
+
"revision_r1/sensor/safe_replay.json": "32c12d1bbd1e0161d272b9632c3a6e5d55953c92c9e53bdc3c9919325c8731c2",
|
| 362 |
+
"revision_r1/sensor/review_r1_sensor.py": "22e61a23b610a1399e478499e932bea5b79e3aa30603d8aa19e660562d8ec037",
|
| 363 |
+
"revision_r1/sensor/findings.json": "4aff5cf1f2f76094fd170e097e8ed6aa8b40d9199c35eb6527d2787cd69f2c58",
|
| 364 |
+
"revision_r1/sensor/full_replay.json": "eb948f80c7b116a7c174ebd63f64a1c4c16eb1f69a60cee61a8fd76f307e6286",
|
| 365 |
+
"revision_r1/sensor/analysis_spec.json": "4644bcb2ed1a9f152d6b5e74be0dafc339a3ae7ccbfae7aa0d79d9f4caa4ad02",
|
| 366 |
+
"revision_r1/sensor/verify_all.py": "5316a84ede79929187ffb31483e681c5860be267720e6920f15d029f5ccfef61",
|
| 367 |
+
"revision_r1/sensor/safe_tree.py": "4e89c1560a55fc91cd800fe88d3a8de8b6f1951effb4ce068b23ad808355853e",
|
| 368 |
+
"revision_r1/sensor/objective_extension_spec.json": "ea8ccb96ecf1b94e7c9cf68211428b23f6adf3de13653e38e9b61ce7c83cd18f",
|
| 369 |
+
"revision_r1/sensor/export_safe.py": "39a447b022689fd6d4ac8441d5c83dafe5d66136a632e80d2c7e244d2bd36a64",
|
| 370 |
+
"revision_r1/sensor/objective_extension.py": "bd24384d8512572d6cf880748a98714482f90c8da4dc0b14c75b4cfe53b527f3",
|
| 371 |
+
"revision_r1/controlled/numeric_push_sensitivity.csv.gz": "655e73bc4111c4c5dab7507e4fd029cd5fba7efdde6375757a6b281c94027358",
|
| 372 |
+
"revision_r1/controlled/summary.json": "903a81f9440772657a9590911e772ce2beba514a2d87f0b4789a8a0915742b0d",
|
| 373 |
+
"revision_r1/controlled/replay_discrepancies.csv.gz": "6ef693532219b9ba0abab9aaafcc07558a72a20f0ffe961b4bc42a138e873d25",
|
| 374 |
+
"revision_r1/controlled/reference_replay_discrepancies.csv.gz": "59fad6d56023fedd16c50f0faab3b4f85a8cb322c3ecbcb4f7d5f9c7e5076051",
|
| 375 |
+
"revision_r1/controlled/native_worlds.csv.gz": "1390af820583c90646ceb5edd7da2647ea0b9d8aeedfd516c921cdeea1b58581",
|
| 376 |
+
"revision_r1/controlled/specification.json": "d361b91b341dd596b246932f9113a0245adafcf5dcdce7b4e0f340d364b7b9da",
|
| 377 |
+
"revision_r1/controlled/privileged_geometry.csv": "8acd6348b334b91b9f9b0486a9e07b9ea4cd1eee71d4eb4124cbc8cfe7bca233",
|
| 378 |
+
"revision_r1/controlled/paired_by_condition.csv": "e432889ff30f13909602467258b211fa3f0248a11c7cd875b957d2942a3704af",
|
| 379 |
+
"revision_r1/controlled/frontier_all_conditions.csv": "5b5b9e761163a7f1800047503cc0eae9d9d3450e221161daad14f6a44ffa704d",
|
| 380 |
+
"revision_r1/controlled/run.py": "4a300dfc769473f5afb08e15e877f6c53dfff5f6d3a0abfe17e0043798eecbab",
|
| 381 |
+
"revision_r1/controlled/final_audit.json": "f3c2daa7eba8ddd2a364ee3cc363de6510c5fb0171d88db5b71480530c9d44a1",
|
| 382 |
+
"revision_r1/controlled/compute.json": "b07dc17453b5f7dd00512913def5dd5fb00d3244d8188ed159ff09f556364823",
|
| 383 |
+
"revision_r1/controlled/specification_02.json": "d0a57d8c485cb9ca380095c8ed018834d0df79382d9cffcc0709bc747129f9eb",
|
| 384 |
+
"revision_r1/controlled/timing.json": "852f0d26180eaebe616841f03894646a4252f57214a81856df984eccb8f99947",
|
| 385 |
+
"revision_r1/controlled/numeric_diagnostic.json": "cdd2528c9f74b817a1b006b75d16c80be2b8f8ed236cd22927f971e14afcfa89",
|
| 386 |
+
"revision_r1/controlled/family_sample_frontier.csv": "793bd7f0fe0da1a09dc6d89572d36f2afc8aeca149094942959ad4c13f441fb8",
|
| 387 |
+
"revision_r1/controlled/indices.json": "dae400dfe60ac990d206d0bf943e276e5ead6a454d317e34ae6ed57e0ad14e45",
|
| 388 |
+
"revision_r1/controlled/specification_01.json": "d820d635c001d28e9a884eaa1a5040dfacf9829ddb2387b301a13d07d33bbbc2",
|
| 389 |
+
"revision_r1/controlled/sample_frontier.csv": "f51b0fd71c1ffd9e14caaaa30a1677705190c08f02227ad72e5548ce33c4e12c",
|
| 390 |
+
"revision_r1/controlled/numeric_compute.json": "d6eda6bae1d8a6d023ccdb7d2e0677b3e409c0fc49d5e56f51101effcee39d08",
|
| 391 |
+
"revision_r1/external/shift_overall.json": "52aff394a2116b9b507482d7fdc96168036c0275db4f30f40f2d0671a7cb9965",
|
| 392 |
+
"revision_r1/external/execution.json": "0fae50882a58a2a589c673a9c6ae2e2ec83d464571bb48dfa2b0c8ada1924c6d",
|
| 393 |
+
"revision_r1/external/specification.json": "541dd0c1d4d7e242ee20c797bd9ef0928f7afdfffa4aacfeca5c3205cb212fab",
|
| 394 |
+
"revision_r1/external/shift_by_task.json": "92034faf5fe6f94508cb47cf4f4f6e207c9143a49ebe8f3ed312d2e29cb01c46",
|
| 395 |
+
"revision_r1/external/implementation_correction.json": "beb720147e0083ef857ebcb745c1403ccdf083c44ba70f6ed82d8a6d3e9e325e",
|
| 396 |
+
"revision_r1/external/metrics.csv.gz": "9813ff30348ad76cb8b672e022a5f3fe852b8f0ee13625e7a5799400cfc01120",
|
| 397 |
+
"revision_r1/external/run.py": "33e4d7ff42763ccd48db10013dd91831af13aa368a115c39987f65636a06c63f",
|
| 398 |
+
"revision_r1/external/paired_contrasts.json": "b1afae2e0ee103a5f9c65afb49781037e7b3d24e6427e890821931fe640cf3e5",
|
| 399 |
+
"revision_r1/external/overall.json": "af8fefc215e51f30684cdfefade17eac81f99a18302a864c5a9b48cf62c927a7",
|
| 400 |
+
"revision_r1/external/article_table.csv": "09b7110cc185574d01f5aeb4681aa20910048a50429b274a5422969c2654fad2",
|
| 401 |
+
"revision_r1/external/table_provenance.json": "d538e18c7367c0b27778d8a5cdf8a1cfdb1ecdc73a6906faa1980d20bb46b406",
|
| 402 |
+
"revision_r1/sensor/run_01/hydraulic_reversal_s20752.json": "3bf241d1167e33c19f17027ccdd39c8c96082c2977a67e9e8092928ba4dab2f0",
|
| 403 |
+
"revision_r1/sensor/run_01/hydraulic_n64_s20752.json": "ee4c987ec56a2d57909f89febb948f30ad1a13608c01dcebf9f304a4f7ba14e8",
|
| 404 |
+
"revision_r1/sensor/run_01/hydraulic_n64_s20753.json": "5105b5fcc456ddc4b1a6e046d9c63c60a4f080f1823da476870d2020fb6ee954",
|
| 405 |
+
"revision_r1/sensor/run_01/temperature_n128_s20853.json": "c67145613fcb17cf16ada5ec71cd3d127576e590759dacd6a4a5d36f35a05e93",
|
| 406 |
+
"revision_r1/sensor/run_01/hydraulic_n64_s20751.json": "461cded6ade8630bc2652e2e7bab15471f07d124eee50a44a1246611b01a8f46",
|
| 407 |
+
"revision_r1/sensor/run_01/cell_compute.json": "fd139f0c1ff8a284591455fdb2a61043ccc966c631df0d74c1360a5aa2e8c444",
|
| 408 |
+
"revision_r1/sensor/run_01/hydraulic_n128_s20752.safetensors": "7a2759068a289b615624b001dcd0115cd79cf6c9609b30fcc7529846379f18db",
|
| 409 |
+
"revision_r1/sensor/run_01/hydraulic_n128_s20753.json": "6783f3aa43c05882b5c46f42bbbf035945db49e4834a3f65e91ba63f858e4433",
|
| 410 |
+
"revision_r1/sensor/run_01/reversal_diagnostics.json": "7ec82dc059717d2f382c924c8b29682c465c26508475a0955ea3576bfb3c8cd6",
|
| 411 |
+
"revision_r1/sensor/run_01/hydraulic_reversal_s20753.safetensors": "6fd698aa04579d0beb6ca3cdc08d5b8fd1f6edcb2991d062d65d74e86c0933e7",
|
| 412 |
+
"revision_r1/sensor/run_01/metrics.csv": "5aa8f4b6980113a976f79c9ee1d2acd2af9c545b06ce15416fad37f4337c5473",
|
| 413 |
+
"revision_r1/sensor/run_01/temperature_n128_s20853.safetensors": "8fce6292bf5d5740855525d26ac91e3363e0d2740f1271c9d56309ea0fed1d98",
|
| 414 |
+
"revision_r1/sensor/run_01/hydraulic_n128_s20752.json": "d0c21c696f9ee7e4eec30c19b1a797b401b185d9507a25922ff4d07e23014506",
|
| 415 |
+
"revision_r1/sensor/run_01/hydraulic_n128_s20751.safetensors": "2fda58bcf415701c8a2231e46fb0253e7e4aeb72b5bf296ad8b3ae4326c9eef4",
|
| 416 |
+
"revision_r1/sensor/run_01/hydraulic_reversal_s20753.json": "a2156b5048ee86e6022495b2e464c5c89d98db03757d9f70cd9d43a1777bf9ad",
|
| 417 |
+
"revision_r1/sensor/run_01/temperature_n64_s20852.safetensors": "77c7972c39b6f405d53686da88fafa680d6aec2303b175e1f9c858faf392f126",
|
| 418 |
+
"revision_r1/sensor/run_01/reversal_metrics.csv": "a8ebb206fc609e78b3ad8e9c327224dcc693d37b264bfdb0c1edd83e92750b7d",
|
| 419 |
+
"revision_r1/sensor/run_01/calibration_choices.csv": "7b822a9e32b6445168c22255b2e6e5789ecf2e8dfc857f2027dbce3e188ab6da",
|
| 420 |
+
"revision_r1/sensor/run_01/temperature_n128_s20851.safetensors": "00973079adcb876833d0af0f9399ca70a165639f99a225f44abb3292f0b914d2",
|
| 421 |
+
"revision_r1/sensor/run_01/hydraulic_n128_s20751.json": "8e91c7b827846c153095b14b8fed42028443f816f56856c06e26ff734be3c7fd",
|
| 422 |
+
"revision_r1/sensor/run_01/temperature_n64_s20851.safetensors": "6c64fb224e027bb5cab760fd37725567c4b470caff93a0384f6905aa02b4bdc1",
|
| 423 |
+
"revision_r1/sensor/run_01/temperature_n64_s20853.safetensors": "51819ee75d951c15106f03eb08132c305886b9f3fe44155e71efc5b7e0cd50a0",
|
| 424 |
+
"revision_r1/sensor/run_01/compute.json": "8f0adcf124145f1b91191b00ee452df2fb7679ba57f597f75cbd12511b74b11d",
|
| 425 |
+
"revision_r1/sensor/run_01/hydraulic_reversal_s20751.json": "6451cea208f09bd9e541ffaf8b3d1d112e5e6c153e5581ca354b3a98aa612f74",
|
| 426 |
+
"revision_r1/sensor/run_01/temperature_n64_s20851.json": "5c3cbc5181a479e94c9a3fa37f55d171619ae4f72fb3e3ce7d7b1660a3cc70cf",
|
| 427 |
+
"revision_r1/sensor/run_01/numerical_tests.json": "61759ebaf9ea643356170c39e30dfe666c82aa5cdf3146cb1b946e161f960edb",
|
| 428 |
+
"revision_r1/sensor/run_01/temperature_n128_s20852.json": "ee2fb82904fa151f1cc6b7b402700cbb5ca3e9257c377bd57d60e5658b59ef07",
|
| 429 |
+
"revision_r1/sensor/run_01/temperature_n64_s20852.json": "6251d047615ae184d78e974bb6d7ed67f148523fccd9970e7e9f9e3de1556530",
|
| 430 |
+
"revision_r1/sensor/run_01/summary.csv": "6a87bbb63231c3d2680e5f6329dea91bd4819016450cd6264633d3c9be9eda0b",
|
| 431 |
+
"revision_r1/sensor/run_01/reversal_summary.csv": "d9b2e591d90c92e202ef4278cf23c9dc01ed5829abde75174a826eddac18c91f",
|
| 432 |
+
"revision_r1/sensor/run_01/temperature_n128_s20852.safetensors": "dbf86d345e884c3ebe5729873f30cadedea12db9103c1106e08e8cb7098bcaec",
|
| 433 |
+
"revision_r1/sensor/run_01/hydraulic_n64_s20752.safetensors": "bffa83502f802cc8cb4475ebddef49479380fbf90114b744f734a8866b979813",
|
| 434 |
+
"revision_r1/sensor/run_01/replay_verification.json": "5cc75dafd0322febcee4ab1e4f2722423d24b5c8d4dfd8b9788639080e682c41",
|
| 435 |
+
"revision_r1/sensor/run_01/hydraulic_n64_s20753.safetensors": "a0bee153eeabb6ad7346aea4a0872c5b829b7e76b309b643d52c2412373ff3a9",
|
| 436 |
+
"revision_r1/sensor/run_01/hydraulic_reversal_s20751.safetensors": "b360e8dc3a93fe93ceba9e7bdbeb23fd5a1489934c1657bf9747a7152c0c6bc6",
|
| 437 |
+
"revision_r1/sensor/run_01/mean_diagnostics.csv": "66918f9e476716271f022e8a09ca25baaa40a33f63211ddaef2cc18584ca8d0f",
|
| 438 |
+
"revision_r1/sensor/run_01/run_manifest.json": "e1c73045c497d48a4765b6bdd6e99f29080d81dd585be03259877d20c35aa81a",
|
| 439 |
+
"revision_r1/sensor/run_01/temperature_n128_s20851.json": "95ce9f0bd3833eb99aec7ea044e033d6cbcd1846e13d76ed7c4bf545514dcaa2",
|
| 440 |
+
"revision_r1/sensor/run_01/hydraulic_n128_s20753.safetensors": "a0dedab19716b992c8a8fa5d6cc87de17ba472f88c27a3e8e2f24e98cb7b39fb",
|
| 441 |
+
"revision_r1/sensor/run_01/temperature_n64_s20853.json": "82e29ee85f769acfca702b26404d4b887e706e01a10d1c6b2642d71641481d42",
|
| 442 |
+
"revision_r1/sensor/run_01/hydraulic_n64_s20751.safetensors": "cbebd95806651b704991c11d0903046cb174d076b6eae52fb9fe72e946534d93",
|
| 443 |
+
"revision_r1/sensor/run_01/hydraulic_reversal_s20752.safetensors": "ee84572929b830064ff9e92e01ba503ddc2bbb359e6052c62b1f8ce1ec956303",
|
| 444 |
+
"revision_r1/sensor/objective_extension_01/hydraulic_n64_s20752.json": "e86e470016ba09d6f7da725a0f352118cb6012652caa4ea06e5b2921338b863e",
|
| 445 |
+
"revision_r1/sensor/objective_extension_01/hydraulic_n64_s20753.json": "8292642abe36e112830007a9a68d987bb09ab0fb738e50f99fb2e8d455593618",
|
| 446 |
+
"revision_r1/sensor/objective_extension_01/temperature_n128_s20853.json": "12a70593382be57116c35034ae0b9d22646a9acbfead91bf75015f85d6129ec4",
|
| 447 |
+
"revision_r1/sensor/objective_extension_01/hydraulic_n64_s20751.json": "026c7b03639d791649caaca18f9da894b114ae82905af552a10a7c89a49a2283",
|
| 448 |
+
"revision_r1/sensor/objective_extension_01/cell_compute.json": "041df92042872985572d039dc831ac4e31a89b963ef298a05ae559681f0e7dea",
|
| 449 |
+
"revision_r1/sensor/objective_extension_01/hydraulic_n128_s20752.safetensors": "84e37c6d7a7ec02217682fedad56211ef67be4757a0376682fc6428f70f92f61",
|
| 450 |
+
"revision_r1/sensor/objective_extension_01/summary.json": "39e95464252c3e3e948a6418ca33a77eedc236c672cbf28087ecec249650d6b8",
|
| 451 |
+
"revision_r1/sensor/objective_extension_01/hydraulic_n128_s20753.json": "51bbf95853904a4fcc7d51b9de5b1695a0c107df4a6ea6c18e818afa8c2630f2",
|
| 452 |
+
"revision_r1/sensor/objective_extension_01/metrics.csv": "714c629729eec808ced1d3b281989182d7a28b1299fb0d4f51f13b76dedce22d",
|
| 453 |
+
"revision_r1/sensor/objective_extension_01/temperature_n128_s20853.safetensors": "0e056e512d4ee1f5a3efe4344290d4668a8b3aa71f2e840ad1ab9276b76f1851",
|
| 454 |
+
"revision_r1/sensor/objective_extension_01/hydraulic_n128_s20752.json": "5891441524bac20f98801f92155f8f4a4fd55d01ee4135a3a452cd712a8761d0",
|
| 455 |
+
"revision_r1/sensor/objective_extension_01/hydraulic_n128_s20751.safetensors": "d7a21511d36785300dd873f79aadcec87bf8de59a73b621328e26cc45a137bda",
|
| 456 |
+
"revision_r1/sensor/objective_extension_01/temperature_n64_s20852.safetensors": "5ef676925efe8be2d36c8d26d21c874409db2547074ef99010628f0de46d0ae5",
|
| 457 |
+
"revision_r1/sensor/objective_extension_01/manifest.json": "74aa43959c81bf2cfc5b8f7e10653e05365250216451ee320e2964dc0a6b1733",
|
| 458 |
+
"revision_r1/sensor/objective_extension_01/temperature_n128_s20851.safetensors": "dfc1891afa7d41b2e26e19348598c522070cd20223d1a70a9e72030ed801e2a9",
|
| 459 |
+
"revision_r1/sensor/objective_extension_01/hydraulic_n128_s20751.json": "a4bfa9d2dfa4c95b8eca35b1edd0ec4d21988f8bb9d2b088b9553daafb72c3f4",
|
| 460 |
+
"revision_r1/sensor/objective_extension_01/temperature_n64_s20851.safetensors": "8e93164e9f58e98154ccd2bf69a3fd8323f9615b63ceae62a74dc5997b4dc3db",
|
| 461 |
+
"revision_r1/sensor/objective_extension_01/temperature_n64_s20853.safetensors": "3967dd4e5ba220d02fdc5808f3f6b08dcf643e43f39d4b70d9eaad9eecef3d88",
|
| 462 |
+
"revision_r1/sensor/objective_extension_01/compute.json": "c8daa706c9c4222737008ec8efd5ae98ecd921bc11642580d5821cf0da0960a5",
|
| 463 |
+
"revision_r1/sensor/objective_extension_01/temperature_n64_s20851.json": "95e4dedaa248d4d30912e2633f4e5d3f8a2846f9dc68682bec94a2206da61d4d",
|
| 464 |
+
"revision_r1/sensor/objective_extension_01/temperature_n128_s20852.json": "8d84d08589d246c2e413820e74572b96b16fdbac3dd810902319cd2b4e9611f6",
|
| 465 |
+
"revision_r1/sensor/objective_extension_01/temperature_n64_s20852.json": "b259c61869934fa7d19b37e154c669ccc39d19318ce7061dd53eea62a94e127f",
|
| 466 |
+
"revision_r1/sensor/objective_extension_01/summary.csv": "db94a3a67be24e9a3d7fae4fb8d475f329b78df7bfb528e9ce7600f32d220092",
|
| 467 |
+
"revision_r1/sensor/objective_extension_01/temperature_n128_s20852.safetensors": "d5fad93647f548d3385f5c946987d64d7aa78fd7fa87f03a884b0a5258cfb8cd",
|
| 468 |
+
"revision_r1/sensor/objective_extension_01/hydraulic_n64_s20752.safetensors": "e4ff49f92f19eeb70adf1a6c200eb238839153310e435c938951a6658adfbc7c",
|
| 469 |
+
"revision_r1/sensor/objective_extension_01/hydraulic_n64_s20753.safetensors": "e33317df751bf702ad4cc096ffe209c450ec2783c5f6c0e318b10f57d29f5879",
|
| 470 |
+
"revision_r1/sensor/objective_extension_01/summary_per_seed.csv": "8399b7bec3e4ea9c62ed2786c937d7a44495fed7982cdf0dfebc4f10112c0104",
|
| 471 |
+
"revision_r1/sensor/objective_extension_01/temperature_n128_s20851.json": "2f2e1c580bce42b35e93c8537b2fd219cc3b6de071df323560479270ae7bd108",
|
| 472 |
+
"revision_r1/sensor/objective_extension_01/hydraulic_n128_s20753.safetensors": "fb1eb8085bbd37b334f4a7033d7b0ca0863981f0263528ccced9ada34167dace",
|
| 473 |
+
"revision_r1/sensor/objective_extension_01/temperature_n64_s20853.json": "398edf09511ff818a3bb2e77829b121e5b1af9b1646eb29507556f6d756a5cba",
|
| 474 |
+
"revision_r1/sensor/objective_extension_01/hydraulic_n64_s20751.safetensors": "3e5a56962a870d83d2cd4037f8ec1c0aa39fe0b20111579bda7d435a98683f28"
|
| 475 |
}
|
revision_r1/controlled/compute.json
ADDED
|
@@ -0,0 +1,33 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"created_utc": "2026-11-09T14:43:53.549254+00:00",
|
| 3 |
+
"status": "COMPLETED",
|
| 4 |
+
"source_release_files_hash_verified": 358,
|
| 5 |
+
"code_sha256": "aa82ccdc6a1bb3c7d1e92b9ab0c4c88478c7d26e0442fe91523036a63260b2aa",
|
| 6 |
+
"specification_sha256": "d0a57d8c485cb9ca380095c8ed018834d0df79382d9cffcc0709bc747129f9eb",
|
| 7 |
+
"command": [
|
| 8 |
+
"articles/make_1/reproduce/review_r1_controlled.py",
|
| 9 |
+
"--resume-aggregation",
|
| 10 |
+
"--device",
|
| 11 |
+
"cuda:4",
|
| 12 |
+
"--output",
|
| 13 |
+
"articles/make_1/metadata/review_r1/controlled"
|
| 14 |
+
],
|
| 15 |
+
"device": "cuda:4",
|
| 16 |
+
"gpu": "NVIDIA H200",
|
| 17 |
+
"python": "3.13.11 | packaged by Anaconda, Inc. | (main, Dec 10 2025, 21:28:48) [GCC 14.3.0]",
|
| 18 |
+
"pytorch": "2.6.0+cu124",
|
| 19 |
+
"cuda": "12.4",
|
| 20 |
+
"elapsed_analysis_seconds": 10.689740672009066,
|
| 21 |
+
"elapsed_since_standard_library_import_seconds": 37.757182730943896,
|
| 22 |
+
"first_cuda_noop_seconds": 1.7166021279990673e-05,
|
| 23 |
+
"peak_cuda_allocated_bytes": 168696832,
|
| 24 |
+
"opened_worlds": true,
|
| 25 |
+
"new_backbone_executions": 0,
|
| 26 |
+
"new_trainable_parameters": 0,
|
| 27 |
+
"refitted_neural_models": 0,
|
| 28 |
+
"fitted_statistical_tables": "Recomputed original estimators from archived counts, unchanged formulas",
|
| 29 |
+
"original_files_modified": false,
|
| 30 |
+
"all_selected_units_retained": true,
|
| 31 |
+
"all_statistics_descriptive": true,
|
| 32 |
+
"promotion_decisions_changed": false
|
| 33 |
+
}
|
revision_r1/controlled/family_sample_frontier.csv
ADDED
|
@@ -0,0 +1,25 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
family,records,worlds,identity,query_joint,posterior_clusters,posterior_bins,future_marginal,raw_information,raw_entropy,hash_code,action_values,true_oracle,joint_minus_action,paired_95_low,paired_95_high,action_encoding_loss,joint_encoding_loss,action_estimation_loss,joint_estimation_loss,encoding_contribution,estimation_contribution,action_remaining_headroom,fraction_action_headroom_closed,true_code_joint_oracle,true_code_action_oracle,true_code_joint_fitted,true_code_action_fitted,joint_root_choice_mismatch,action_root_choice_mismatch,strongest_uncompressed_mean
|
| 2 |
+
context,64,32,0.09339283465187889,0.0691595408487013,0.05884192092177272,0.06287654697819856,0.07187382385734353,0.07728978551777194,0.09455913414529662,0.10673495166633204,0.06409177954684019,0.7880888584118835,0.005067761301861108,0.0015311813058195526,0.008840865316316502,0.3813222379944849,0.3669534072130477,0.34267484087055833,0.3519759103501344,0.014368830781437176,-0.00930106947957607,0.7239970788650432,0.00699969854823926,0.7880888584118835,0.7880888584118835,0.12334932494026522,0.12491782030181064,0.6849965145249336,0.6875732689753824,0.2821009589784481
|
| 3 |
+
context,256,32,0.34655353787902665,0.3367188419663062,0.16416722984358104,0.1463272094176618,0.1871258673613156,0.26397328263687364,0.25243683714316034,0.15529673741167777,0.31618507415861496,0.7880888584118835,0.0205337678076913,0.01188513381910629,0.03021118308609348,0.11119897518661376,0.07603887002395061,0.3607048090666547,0.37533114642162657,0.035160105162663144,-0.014626337354971848,0.47190378425326845,0.043512615267927005,0.7880888584118835,0.7880888584118835,0.3856346205277096,0.38320113863439054,0.629779568015901,0.6496391850613675,0.5904656904258859
|
| 4 |
+
context,1024,32,0.6650919319515614,0.6814637624980395,0.23779038974868094,0.17124022355036714,0.36268052770898085,0.5145775504161055,0.5794729711288229,0.2170459709045936,0.667719627246173,0.7880888584118835,0.013744135251866517,0.009521684888968644,0.01821770096658289,0.01430655071271671,0.006186382463298001,0.10606268045299369,0.10043871345054588,0.008120168249418708,0.005623967002447808,0.12036923116571041,0.11418312735540517,0.7880888584118835,0.7880888584118835,0.6988418074434672,0.6964377598607214,0.3473903728720782,0.3575560574608555,0.7340548744395632
|
| 5 |
+
context,4096,32,0.766972716874273,0.7766349438098745,0.26807855851736373,0.1733365486789426,0.5632767718308346,0.5928278598851834,0.6885210614964501,0.24707180576088097,0.7768669245376707,0.7880888584118835,-0.00023198072779632983,-0.001376293766611737,0.0008712872445568171,0.0006891208483935686,0.00018064349661515988,0.010532813025819078,0.011273271105393817,0.0005084773517784087,-0.0007404580795747386,0.011221933874212647,-0.02067208115790168,0.7880888584118835,0.7880888584118835,0.7795383990383913,0.7790645144622885,0.1620783588267283,0.1554586985921155,0.7708361142534652
|
| 6 |
+
independent,64,32,0.909510548381619,0.9071402048294446,0.9072370207266524,0.9093082065269007,0.9074681824272653,0.8896605460004141,0.688211917829685,0.44151472051268514,0.9068080779301189,0.928933733504975,0.00033212689932572356,-0.0004962724151037125,0.0012075628763129809,0.009646569741400077,0.00982195667832023,0.012479085833456041,0.011971571997210165,-0.00017538693692015217,0.0005075138362458757,0.02212565557485612,0.015010940498556656,0.927818092070147,0.9281063650817161,0.9185297572467013,0.9192315821057243,0.23523002477266935,0.23312046998815447,0.9136191430312471
|
| 7 |
+
independent,256,32,0.9116024194164163,0.9098670177155274,0.9100779075741663,0.9126143882919777,0.905503332137763,0.896961421482459,0.6703717514265709,0.4619766730060043,0.9090004625042527,0.928933733504975,0.0008665552112746971,-0.0004301648384077622,0.0021276027149003647,0.006369755441728155,0.0065092495744983395,0.01356351555899413,0.01255746621494925,-0.00013949413277018416,0.0010060493440448813,0.019933271000722286,0.04347280540375422,0.927818092070147,0.9281063650817161,0.9170722759812633,0.9171807569702725,0.23936604222246965,0.23868371954799117,0.9251088162712999
|
| 8 |
+
independent,1024,32,0.9100079337524132,0.913013489675506,0.915318662687657,0.9168213117935321,0.8954121957914773,0.9031660787545921,0.6489306322062101,0.470594937948292,0.9133526580360904,0.928933733504975,-0.00033916836058426117,-0.0011080639841925125,0.00048298344501220073,0.006376698269395811,0.00708733789545923,0.009204377199488828,0.00883290593400967,-0.0007106396260634189,0.0003714712654791577,0.01558107546888464,-0.021767968537318196,0.927818092070147,0.9281063650817161,0.9209817567018171,0.9206480724749362,0.22556813929192912,0.21870599191031026,0.9280646584591856
|
| 9 |
+
independent,4096,32,0.9190137606668483,0.9212120370356884,0.9233281672038094,0.9234185604697762,0.9120224134643348,0.9084610246510227,0.6461753611274292,0.47480680272329256,0.9224258752130949,0.928933733504975,-0.0012138381774065114,-0.0017147112652704927,-0.000726757585522743,0.0038865080122648353,0.004790920676205797,0.002621350279615183,0.002930775793080733,-0.0009044126639409615,-0.0003094255134655499,0.0065078582918800185,-0.18651883968048918,0.927818092070147,0.9281063650817161,0.9258990135155555,0.9259546622363166,0.21008579345280287,0.19455413681065953,0.9287125775280298
|
| 10 |
+
orientation_alias,64,32,0.1635502787070287,0.12885644874690116,0.11989069408297903,0.12129218066799444,0.11565627135211173,0.1194667363872123,0.08451253981799893,0.062005774703194766,0.12837855808157933,0.828928467957698,0.00047789066532179854,-0.005027820336695715,0.005779686114251358,0.3951517175523762,0.4017012049487419,0.3053981923237425,0.29837081426205503,-0.0065494873963657056,0.007027378061687502,0.7005499098761186,0.0006821650514612244,0.828928467957698,0.828928467957698,0.23582020614293137,0.23582020614293134,0.6245539383534132,0.6283048497052103,0.3744516848229764
|
| 11 |
+
orientation_alias,256,32,0.48105690410695323,0.45413249913205544,0.3267375095162406,0.3044234970537981,0.17946386454389426,0.3385633917456905,0.26942870751962467,0.14858964279293768,0.4108957134778612,0.828928467957698,0.04323678565419422,0.03091808741028508,0.056253384606153346,0.22593360992407183,0.15025128394784587,0.19209914455576496,0.22454468487779672,0.07568232597622598,-0.03244554032203176,0.4180327544798368,0.1034291815434278,0.828928467957698,0.828928467957698,0.5438758040593306,0.5438758040593306,0.5760678352632569,0.6058617660589638,0.6522145924562783
|
| 12 |
+
orientation_alias,1024,32,0.7375432253689826,0.749695480009097,0.5557157182324963,0.5477364417189148,0.18531714088359752,0.6134052546318305,0.6043586411911206,0.2567112571360594,0.7156206511171621,0.828928467957698,0.03407482889193493,0.026490100551048997,0.0415008522799025,0.040254364563248254,0.01143065598560973,0.07305345227728763,0.06780233196299122,0.028823708577638524,0.00525112031429641,0.1133078168405359,0.30072796248373795,0.828928467957698,0.828928467957698,0.766961318774352,0.7669613187743521,0.3803245810966434,0.4051527363935008,0.767114986178982
|
| 13 |
+
orientation_alias,4096,32,0.804304431565352,0.8195916562097663,0.6530579291374521,0.648285517967734,0.18829678111200815,0.6964865610322947,0.706006590523646,0.3003141222527754,0.819480859898903,0.828928467957698,0.00011079631086339387,-2.228212253385705e-05,0.0002634376906898875,0.0002445290260795034,0.00019810226223412192,0.009203079032715649,0.009138709485697637,4.64267638453815e-05,6.436954701801237e-05,0.009447608058795152,0.011727445737998115,0.828928467957698,0.828928467957698,0.8198637628550727,0.8198637628550727,0.21189316146075493,0.21099546987915718,0.8096451880020169
|
| 14 |
+
parity,64,32,0.06273197484460194,0.02431126538233747,0.011591763903878482,0.011981140143721973,0.015523777582997956,0.011150427588187746,0.020184840684643574,0.024015757122275827,0.02252062020077706,0.8347957761429843,0.0017906451815604108,-0.0038315227161669556,0.006652954501555841,0.46294985993286397,0.4524966576959601,0.34932529600934337,0.35798785306468683,0.010453202236903802,-0.008662557055343396,0.8122751559422072,0.0022044810412591444,0.8347957761429844,0.8347957761429844,0.12286533950243893,0.12288347183177817,0.5209659718033554,0.5285378271397623,0.385086063569183
|
| 15 |
+
parity,256,32,0.4475888678222731,0.41757388132172324,0.2750026478906139,0.23783397003296353,0.1298958512570368,0.20886862239855064,0.18373006939535583,0.11691960704302928,0.38718031379931517,0.8347957761429843,0.030393567522408023,0.019497351412313457,0.041409246567013835,0.22704928735222724,0.15105877295815492,0.220566174991442,0.2661631218631063,0.07599051439407231,-0.04559694687166428,0.4476154623436692,0.06790106705266699,0.8347957761429844,0.8347957761429844,0.4773303993370155,0.4773303993370155,0.5556145869981621,0.5840915047008,0.6622060304284867
|
| 16 |
+
parity,1024,32,0.7505863638432653,0.7643539645704318,0.5785860682616655,0.5650067039347257,0.17619358394784873,0.491345522934861,0.5428645204900009,0.20833025114024675,0.749467269935558,0.8347957761429843,0.01488669463487384,0.010441901192385099,0.019552134986879535,0.021393038552985848,0.0075410150709803,0.06393546765444057,0.06290079650157226,0.013852023482005548,0.0010346711528682918,0.08532850620742641,0.17446332177297866,0.8347957761429844,0.8347957761429844,0.7761508394516206,0.7761508394516206,0.3591226824418755,0.35639638530427453,0.7761873106567354
|
| 17 |
+
parity,4096,32,0.811457808557315,0.824131626938427,0.694169383413467,0.6900830814062695,0.18472462130093503,0.6242484323345312,0.6210217172753493,0.2392919195668825,0.824102588730724,0.8347957761429843,2.9038207703040858e-05,-1.3477962034352808e-06,7.886248154405023e-05,9.112519989473711e-05,6.378990475829419e-05,0.010602062212365704,0.010600359299799106,2.7335295136442922e-05,1.7029125665979361e-06,0.010693187412260441,0.0027155801711421096,0.8347957761429844,0.8347957761429844,0.8242244796942859,0.8242244796942859,0.199476969506594,0.1989397435570667,0.8150734693187245
|
| 18 |
+
redundant,64,32,0.5363358739730864,0.5113855171174998,0.5036569357456289,0.5085041397628068,0.5000048354948977,0.5772794551090056,0.4124257595149454,-0.04050167310864608,0.5053617140765039,0.8364283751534423,0.006023803040995961,0.0009731619737643064,0.011502247775086136,0.14576415220704214,0.13364129983182016,0.1853025088698963,0.1914015582041223,0.012122852375221975,-0.006099049334226014,0.3310666610769384,0.018195136355321676,0.8364283751534423,0.8364283751534423,0.5890955182088521,0.5890955182088521,0.8539832767484425,0.8572854915108054,0.5662414160281566
|
| 19 |
+
redundant,256,32,0.6453415693659241,0.6198034615018286,0.5877161849929605,0.591621214981205,0.42260265019739973,0.6312017727896639,0.5514639211048289,0.019364542901451464,0.6050694499969314,0.8364283751534423,0.014734011504897247,0.009603019224582635,0.01995868486298478,0.0932629369589013,0.07785092527231657,0.13809598819760954,0.13877398837929703,0.015412011686584716,-0.0006780001816874687,0.23135892515651085,0.06368464711240299,0.8364283751534423,0.8364283751534423,0.653629840527671,0.653629840527671,0.763869962732947,0.7826760570524698,0.7105456685441063
|
| 20 |
+
redundant,1024,32,0.7653668596783447,0.7545480306631399,0.6989231534206255,0.6916977366851904,0.29795754465378854,0.7244842637951285,0.6854194139838374,0.04484946990041256,0.7412029739417946,0.8364283751534423,0.013345056721345335,0.010143917478369981,0.016472364541550773,0.039402157560219295,0.02814080307390973,0.05582324365142841,0.05373954141639264,0.01126135448630956,0.0020837022350357753,0.0952254012116477,0.14014177468976635,0.8364283751534423,0.8364283751534423,0.7721932591100672,0.7721932591100673,0.4023677500023982,0.43296693574774525,0.7863078402596235
|
| 21 |
+
redundant,4096,32,0.8155857149751549,0.8194726488422938,0.7193415226940727,0.6960627204154617,0.18000890013383308,0.7543788456935281,0.752804281570126,0.05292377176364181,0.8165418628657128,0.8364283751534423,0.002930785976580884,0.0019304463049228831,0.004100380433294095,0.00730164446795117,0.0038787169871150502,0.012584867819778103,0.013077009324033339,0.00342292748083612,-0.0004921415042552363,0.019886512287729273,0.14737556461266912,0.8364283751534423,0.8364283751534423,0.8226174136373886,0.8226174136373885,0.1746054019842278,0.18535970463320198,0.8207655371219653
|
| 22 |
+
uninformative,64,32,-0.04890966142219218,-0.04575341745644379,-0.047049768625332646,-0.04950046362247272,-0.026933077678025653,-0.041441113900889705,-0.010995531167365718,0.02058205608137745,-0.043723918882921195,0.0667348907999358,-0.0020294985735225856,-0.0039889178805482635,-0.000114955541232761,-2.0599841277224584e-18,-9.75781955236954e-19,0.11045880968285701,0.11248830825637958,-1.0842021724855044e-18,-0.002029498573522584,0.11045880968285701,-0.018373351834494372,0.0667348907999358,0.0667348907999358,-0.008709004374880524,-0.01331114610045102,0.5172644451120586,0.5137298310222932,-0.04890966142219218
|
| 23 |
+
uninformative,256,32,0.015731854753485836,0.03111698106830645,0.02924553921598229,0.03234940005689452,0.046069027949175986,0.03744741765448152,0.04500280677191939,0.04957765804565338,0.024803729988287868,0.0667348907999358,0.006313251080018578,0.0035633612177853723,0.009296090920171755,-1.1926223897340549e-18,5.421010862427522e-19,0.04193116081164793,0.03561790973162935,-1.734723475976807e-18,0.006313251080018579,0.04193116081164793,0.15056227773844122,0.0667348907999358,0.0667348907999358,0.046611496345385736,0.04479898126383822,0.318326143510767,0.3350521771578457,0.026880985378230898
|
| 24 |
+
uninformative,1024,32,0.02450926400985953,0.046334453063967655,0.041247390040046704,0.048595464204311184,0.055973307164560476,0.05435897532129334,0.05452489439127397,0.05498349310416952,0.038204503983425536,0.0667348907999358,0.008129949080542126,0.0048984844703365985,0.011408492793951204,4.553649124439119e-18,-3.3610267347050637e-18,0.028530386816510264,0.020400437735968148,7.914675859144182e-18,0.008129949080542116,0.028530386816510267,0.2849575483441185,0.0667348907999358,0.0667348907999358,0.05766653710546027,0.0569982079524269,0.27031338447698905,0.29409977769850704,0.05871121561510939
|
| 25 |
+
uninformative,4096,32,0.044186530387099784,0.05829518844278467,0.05390322882895525,0.05970059877167674,0.06250539980375937,0.06113657669308175,0.061513705438783525,0.0609891079973558,0.05473918863077931,0.0667348907999358,0.0035559998120053464,0.002404317887698644,0.004959589260649769,4.7704895589362195e-18,5.421010862427522e-19,0.011995702169156477,0.008439702357151135,4.228388472693467e-18,0.003555999812005342,0.011995702169156483,0.2964394882317588,0.0667348907999358,0.0667348907999358,0.06340955513774679,0.0624701246085254,0.21447414713172985,0.22634002044987156,0.06369397045613248
|
revision_r1/controlled/final_audit.json
ADDED
|
@@ -0,0 +1,34 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"created_utc": "2026-11-09T14:50:38.476227+00:00",
|
| 3 |
+
"status": "COMPLETED_EXPLORATORY_WITH_DISCLOSED_ARCHIVAL_TIE_SENSITIVITY",
|
| 4 |
+
"current_code_sha256": "4a300dfc769473f5afb08e15e877f6c53dfff5f6d3a0abfe17e0043798eecbab",
|
| 5 |
+
"specification_sha256": "d361b91b341dd596b246932f9113a0245adafcf5dcdce7b4e0f340d364b7b9da",
|
| 6 |
+
"primary_frontier_exact_replay_maximum_error": 3.3306690738754696e-16,
|
| 7 |
+
"original_reference_replay_changed_rows": 968,
|
| 8 |
+
"original_reference_replay_maximum_error": 0.1456998408107582,
|
| 9 |
+
"decomposition_identity_maximum_error": 1.6653345369377348e-16,
|
| 10 |
+
"all_192_worlds_6_families_3_streams_4_sizes_3_estimators_2_availability_9_methods_9_budget_cost_cells_retained": true,
|
| 11 |
+
"new_decomposition_grid_rows": 124416,
|
| 12 |
+
"new_decomposition_cell_rows": 1119744,
|
| 13 |
+
"no_source_results_protocols_models_or_manuscripts_modified": true,
|
| 14 |
+
"compute_accounting": "compute.json records only resumed aggregation/benchmark process. numeric_compute.json records the additional accumulation-order experiment. Initial full-grid collection completed but validation failed before its process ledger; its exact elapsed time and CUDA peak were not captured. Do not treat resumed process totals as total revision compute. No backbones or neural fits executed.",
|
| 15 |
+
"alias_caution": "Root-action mismatch can reflect equal-value zero-cost query ties; it is not itself decision regret. The utility loss decomposition and true-law encoders support the mechanism explanation.",
|
| 16 |
+
"paired_intervals": "All descriptive on opened worlds, uncorrected; world is paired unit, streams/costs/budgets averaged before primary resampling. No new confirmation or promotion.",
|
| 17 |
+
"statistical_fallacy_scan": {
|
| 18 |
+
"checked": 11,
|
| 19 |
+
"total": 11,
|
| 20 |
+
"items": {
|
| 21 |
+
"Simpson_paradox": "All six family contrasts and each cost/budget retained; heterogeneous signs disclosed.",
|
| 22 |
+
"Ecological_fallacy": "World-level intervals describe designed world population only; no sensor/person extrapolation.",
|
| 23 |
+
"Berkson_paradox": "No selection on results; every archived unit retained; constructed family selection limits generalization.",
|
| 24 |
+
"Collider_bias": "No covariate conditioning or exclusions based on outcomes.",
|
| 25 |
+
"Base_rate_neglect": "World-specific generating priors and true-law utility retain base rates; no sensitivity-only inference.",
|
| 26 |
+
"Regression_to_mean": "No extreme-world selection or pre/post effectiveness claim.",
|
| 27 |
+
"Survivorship_bias": "All 192 worlds retained; numerical discrepancies preserved.",
|
| 28 |
+
"Look_elsewhere": "Full condition grid and negative family results retained; intervals descriptive and not multiplicity adjusted.",
|
| 29 |
+
"Forking_paths": "Original specification and both post-failure amendments timestamped; no tuning or confirmatory relabeling.",
|
| 30 |
+
"Correlation_causation": "Fixed-code true-law replanning and true-code interventions isolate finite-grid losses; no unique general theoretical cause claimed.",
|
| 31 |
+
"Reverse_causality": "Encoders/policies constructed before separate true-law scoring; no target outcomes enter native fitted encoders."
|
| 32 |
+
}
|
| 33 |
+
}
|
| 34 |
+
}
|
revision_r1/controlled/frontier_all_conditions.csv
ADDED
|
The diff for this file is too large to render.
See raw diff
|
|
|
revision_r1/controlled/indices.json
ADDED
|
@@ -0,0 +1,466 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"axes": [
|
| 3 |
+
"availability",
|
| 4 |
+
"labels",
|
| 5 |
+
"estimator",
|
| 6 |
+
"stream",
|
| 7 |
+
"method",
|
| 8 |
+
"world",
|
| 9 |
+
"budget",
|
| 10 |
+
"cost",
|
| 11 |
+
"metric"
|
| 12 |
+
],
|
| 13 |
+
"availability": [
|
| 14 |
+
6,
|
| 15 |
+
3
|
| 16 |
+
],
|
| 17 |
+
"labels": [
|
| 18 |
+
64,
|
| 19 |
+
256,
|
| 20 |
+
1024,
|
| 21 |
+
4096
|
| 22 |
+
],
|
| 23 |
+
"estimator": [
|
| 24 |
+
"joint_jeffreys",
|
| 25 |
+
"naive_bayes",
|
| 26 |
+
"support_mixture"
|
| 27 |
+
],
|
| 28 |
+
"stream": [
|
| 29 |
+
0,
|
| 30 |
+
1,
|
| 31 |
+
2
|
| 32 |
+
],
|
| 33 |
+
"method": [
|
| 34 |
+
"identity",
|
| 35 |
+
"query_joint",
|
| 36 |
+
"posterior_clusters",
|
| 37 |
+
"posterior_bins",
|
| 38 |
+
"future_marginal",
|
| 39 |
+
"raw_information",
|
| 40 |
+
"raw_entropy",
|
| 41 |
+
"hash_code",
|
| 42 |
+
"action_values"
|
| 43 |
+
],
|
| 44 |
+
"world": [
|
| 45 |
+
"confirmation:independent:000",
|
| 46 |
+
"confirmation:independent:001",
|
| 47 |
+
"confirmation:independent:002",
|
| 48 |
+
"confirmation:independent:003",
|
| 49 |
+
"confirmation:independent:004",
|
| 50 |
+
"confirmation:independent:005",
|
| 51 |
+
"confirmation:independent:006",
|
| 52 |
+
"confirmation:independent:007",
|
| 53 |
+
"confirmation:independent:008",
|
| 54 |
+
"confirmation:independent:009",
|
| 55 |
+
"confirmation:independent:010",
|
| 56 |
+
"confirmation:independent:011",
|
| 57 |
+
"confirmation:independent:012",
|
| 58 |
+
"confirmation:independent:013",
|
| 59 |
+
"confirmation:independent:014",
|
| 60 |
+
"confirmation:independent:015",
|
| 61 |
+
"confirmation:independent:016",
|
| 62 |
+
"confirmation:independent:017",
|
| 63 |
+
"confirmation:independent:018",
|
| 64 |
+
"confirmation:independent:019",
|
| 65 |
+
"confirmation:independent:020",
|
| 66 |
+
"confirmation:independent:021",
|
| 67 |
+
"confirmation:independent:022",
|
| 68 |
+
"confirmation:independent:023",
|
| 69 |
+
"confirmation:independent:024",
|
| 70 |
+
"confirmation:independent:025",
|
| 71 |
+
"confirmation:independent:026",
|
| 72 |
+
"confirmation:independent:027",
|
| 73 |
+
"confirmation:independent:028",
|
| 74 |
+
"confirmation:independent:029",
|
| 75 |
+
"confirmation:independent:030",
|
| 76 |
+
"confirmation:independent:031",
|
| 77 |
+
"confirmation:context:000",
|
| 78 |
+
"confirmation:context:001",
|
| 79 |
+
"confirmation:context:002",
|
| 80 |
+
"confirmation:context:003",
|
| 81 |
+
"confirmation:context:004",
|
| 82 |
+
"confirmation:context:005",
|
| 83 |
+
"confirmation:context:006",
|
| 84 |
+
"confirmation:context:007",
|
| 85 |
+
"confirmation:context:008",
|
| 86 |
+
"confirmation:context:009",
|
| 87 |
+
"confirmation:context:010",
|
| 88 |
+
"confirmation:context:011",
|
| 89 |
+
"confirmation:context:012",
|
| 90 |
+
"confirmation:context:013",
|
| 91 |
+
"confirmation:context:014",
|
| 92 |
+
"confirmation:context:015",
|
| 93 |
+
"confirmation:context:016",
|
| 94 |
+
"confirmation:context:017",
|
| 95 |
+
"confirmation:context:018",
|
| 96 |
+
"confirmation:context:019",
|
| 97 |
+
"confirmation:context:020",
|
| 98 |
+
"confirmation:context:021",
|
| 99 |
+
"confirmation:context:022",
|
| 100 |
+
"confirmation:context:023",
|
| 101 |
+
"confirmation:context:024",
|
| 102 |
+
"confirmation:context:025",
|
| 103 |
+
"confirmation:context:026",
|
| 104 |
+
"confirmation:context:027",
|
| 105 |
+
"confirmation:context:028",
|
| 106 |
+
"confirmation:context:029",
|
| 107 |
+
"confirmation:context:030",
|
| 108 |
+
"confirmation:context:031",
|
| 109 |
+
"confirmation:parity:000",
|
| 110 |
+
"confirmation:parity:001",
|
| 111 |
+
"confirmation:parity:002",
|
| 112 |
+
"confirmation:parity:003",
|
| 113 |
+
"confirmation:parity:004",
|
| 114 |
+
"confirmation:parity:005",
|
| 115 |
+
"confirmation:parity:006",
|
| 116 |
+
"confirmation:parity:007",
|
| 117 |
+
"confirmation:parity:008",
|
| 118 |
+
"confirmation:parity:009",
|
| 119 |
+
"confirmation:parity:010",
|
| 120 |
+
"confirmation:parity:011",
|
| 121 |
+
"confirmation:parity:012",
|
| 122 |
+
"confirmation:parity:013",
|
| 123 |
+
"confirmation:parity:014",
|
| 124 |
+
"confirmation:parity:015",
|
| 125 |
+
"confirmation:parity:016",
|
| 126 |
+
"confirmation:parity:017",
|
| 127 |
+
"confirmation:parity:018",
|
| 128 |
+
"confirmation:parity:019",
|
| 129 |
+
"confirmation:parity:020",
|
| 130 |
+
"confirmation:parity:021",
|
| 131 |
+
"confirmation:parity:022",
|
| 132 |
+
"confirmation:parity:023",
|
| 133 |
+
"confirmation:parity:024",
|
| 134 |
+
"confirmation:parity:025",
|
| 135 |
+
"confirmation:parity:026",
|
| 136 |
+
"confirmation:parity:027",
|
| 137 |
+
"confirmation:parity:028",
|
| 138 |
+
"confirmation:parity:029",
|
| 139 |
+
"confirmation:parity:030",
|
| 140 |
+
"confirmation:parity:031",
|
| 141 |
+
"confirmation:uninformative:000",
|
| 142 |
+
"confirmation:uninformative:001",
|
| 143 |
+
"confirmation:uninformative:002",
|
| 144 |
+
"confirmation:uninformative:003",
|
| 145 |
+
"confirmation:uninformative:004",
|
| 146 |
+
"confirmation:uninformative:005",
|
| 147 |
+
"confirmation:uninformative:006",
|
| 148 |
+
"confirmation:uninformative:007",
|
| 149 |
+
"confirmation:uninformative:008",
|
| 150 |
+
"confirmation:uninformative:009",
|
| 151 |
+
"confirmation:uninformative:010",
|
| 152 |
+
"confirmation:uninformative:011",
|
| 153 |
+
"confirmation:uninformative:012",
|
| 154 |
+
"confirmation:uninformative:013",
|
| 155 |
+
"confirmation:uninformative:014",
|
| 156 |
+
"confirmation:uninformative:015",
|
| 157 |
+
"confirmation:uninformative:016",
|
| 158 |
+
"confirmation:uninformative:017",
|
| 159 |
+
"confirmation:uninformative:018",
|
| 160 |
+
"confirmation:uninformative:019",
|
| 161 |
+
"confirmation:uninformative:020",
|
| 162 |
+
"confirmation:uninformative:021",
|
| 163 |
+
"confirmation:uninformative:022",
|
| 164 |
+
"confirmation:uninformative:023",
|
| 165 |
+
"confirmation:uninformative:024",
|
| 166 |
+
"confirmation:uninformative:025",
|
| 167 |
+
"confirmation:uninformative:026",
|
| 168 |
+
"confirmation:uninformative:027",
|
| 169 |
+
"confirmation:uninformative:028",
|
| 170 |
+
"confirmation:uninformative:029",
|
| 171 |
+
"confirmation:uninformative:030",
|
| 172 |
+
"confirmation:uninformative:031",
|
| 173 |
+
"confirmation:redundant:000",
|
| 174 |
+
"confirmation:redundant:001",
|
| 175 |
+
"confirmation:redundant:002",
|
| 176 |
+
"confirmation:redundant:003",
|
| 177 |
+
"confirmation:redundant:004",
|
| 178 |
+
"confirmation:redundant:005",
|
| 179 |
+
"confirmation:redundant:006",
|
| 180 |
+
"confirmation:redundant:007",
|
| 181 |
+
"confirmation:redundant:008",
|
| 182 |
+
"confirmation:redundant:009",
|
| 183 |
+
"confirmation:redundant:010",
|
| 184 |
+
"confirmation:redundant:011",
|
| 185 |
+
"confirmation:redundant:012",
|
| 186 |
+
"confirmation:redundant:013",
|
| 187 |
+
"confirmation:redundant:014",
|
| 188 |
+
"confirmation:redundant:015",
|
| 189 |
+
"confirmation:redundant:016",
|
| 190 |
+
"confirmation:redundant:017",
|
| 191 |
+
"confirmation:redundant:018",
|
| 192 |
+
"confirmation:redundant:019",
|
| 193 |
+
"confirmation:redundant:020",
|
| 194 |
+
"confirmation:redundant:021",
|
| 195 |
+
"confirmation:redundant:022",
|
| 196 |
+
"confirmation:redundant:023",
|
| 197 |
+
"confirmation:redundant:024",
|
| 198 |
+
"confirmation:redundant:025",
|
| 199 |
+
"confirmation:redundant:026",
|
| 200 |
+
"confirmation:redundant:027",
|
| 201 |
+
"confirmation:redundant:028",
|
| 202 |
+
"confirmation:redundant:029",
|
| 203 |
+
"confirmation:redundant:030",
|
| 204 |
+
"confirmation:redundant:031",
|
| 205 |
+
"confirmation:orientation_alias:000",
|
| 206 |
+
"confirmation:orientation_alias:001",
|
| 207 |
+
"confirmation:orientation_alias:002",
|
| 208 |
+
"confirmation:orientation_alias:003",
|
| 209 |
+
"confirmation:orientation_alias:004",
|
| 210 |
+
"confirmation:orientation_alias:005",
|
| 211 |
+
"confirmation:orientation_alias:006",
|
| 212 |
+
"confirmation:orientation_alias:007",
|
| 213 |
+
"confirmation:orientation_alias:008",
|
| 214 |
+
"confirmation:orientation_alias:009",
|
| 215 |
+
"confirmation:orientation_alias:010",
|
| 216 |
+
"confirmation:orientation_alias:011",
|
| 217 |
+
"confirmation:orientation_alias:012",
|
| 218 |
+
"confirmation:orientation_alias:013",
|
| 219 |
+
"confirmation:orientation_alias:014",
|
| 220 |
+
"confirmation:orientation_alias:015",
|
| 221 |
+
"confirmation:orientation_alias:016",
|
| 222 |
+
"confirmation:orientation_alias:017",
|
| 223 |
+
"confirmation:orientation_alias:018",
|
| 224 |
+
"confirmation:orientation_alias:019",
|
| 225 |
+
"confirmation:orientation_alias:020",
|
| 226 |
+
"confirmation:orientation_alias:021",
|
| 227 |
+
"confirmation:orientation_alias:022",
|
| 228 |
+
"confirmation:orientation_alias:023",
|
| 229 |
+
"confirmation:orientation_alias:024",
|
| 230 |
+
"confirmation:orientation_alias:025",
|
| 231 |
+
"confirmation:orientation_alias:026",
|
| 232 |
+
"confirmation:orientation_alias:027",
|
| 233 |
+
"confirmation:orientation_alias:028",
|
| 234 |
+
"confirmation:orientation_alias:029",
|
| 235 |
+
"confirmation:orientation_alias:030",
|
| 236 |
+
"confirmation:orientation_alias:031"
|
| 237 |
+
],
|
| 238 |
+
"family": [
|
| 239 |
+
"independent",
|
| 240 |
+
"independent",
|
| 241 |
+
"independent",
|
| 242 |
+
"independent",
|
| 243 |
+
"independent",
|
| 244 |
+
"independent",
|
| 245 |
+
"independent",
|
| 246 |
+
"independent",
|
| 247 |
+
"independent",
|
| 248 |
+
"independent",
|
| 249 |
+
"independent",
|
| 250 |
+
"independent",
|
| 251 |
+
"independent",
|
| 252 |
+
"independent",
|
| 253 |
+
"independent",
|
| 254 |
+
"independent",
|
| 255 |
+
"independent",
|
| 256 |
+
"independent",
|
| 257 |
+
"independent",
|
| 258 |
+
"independent",
|
| 259 |
+
"independent",
|
| 260 |
+
"independent",
|
| 261 |
+
"independent",
|
| 262 |
+
"independent",
|
| 263 |
+
"independent",
|
| 264 |
+
"independent",
|
| 265 |
+
"independent",
|
| 266 |
+
"independent",
|
| 267 |
+
"independent",
|
| 268 |
+
"independent",
|
| 269 |
+
"independent",
|
| 270 |
+
"independent",
|
| 271 |
+
"context",
|
| 272 |
+
"context",
|
| 273 |
+
"context",
|
| 274 |
+
"context",
|
| 275 |
+
"context",
|
| 276 |
+
"context",
|
| 277 |
+
"context",
|
| 278 |
+
"context",
|
| 279 |
+
"context",
|
| 280 |
+
"context",
|
| 281 |
+
"context",
|
| 282 |
+
"context",
|
| 283 |
+
"context",
|
| 284 |
+
"context",
|
| 285 |
+
"context",
|
| 286 |
+
"context",
|
| 287 |
+
"context",
|
| 288 |
+
"context",
|
| 289 |
+
"context",
|
| 290 |
+
"context",
|
| 291 |
+
"context",
|
| 292 |
+
"context",
|
| 293 |
+
"context",
|
| 294 |
+
"context",
|
| 295 |
+
"context",
|
| 296 |
+
"context",
|
| 297 |
+
"context",
|
| 298 |
+
"context",
|
| 299 |
+
"context",
|
| 300 |
+
"context",
|
| 301 |
+
"context",
|
| 302 |
+
"context",
|
| 303 |
+
"parity",
|
| 304 |
+
"parity",
|
| 305 |
+
"parity",
|
| 306 |
+
"parity",
|
| 307 |
+
"parity",
|
| 308 |
+
"parity",
|
| 309 |
+
"parity",
|
| 310 |
+
"parity",
|
| 311 |
+
"parity",
|
| 312 |
+
"parity",
|
| 313 |
+
"parity",
|
| 314 |
+
"parity",
|
| 315 |
+
"parity",
|
| 316 |
+
"parity",
|
| 317 |
+
"parity",
|
| 318 |
+
"parity",
|
| 319 |
+
"parity",
|
| 320 |
+
"parity",
|
| 321 |
+
"parity",
|
| 322 |
+
"parity",
|
| 323 |
+
"parity",
|
| 324 |
+
"parity",
|
| 325 |
+
"parity",
|
| 326 |
+
"parity",
|
| 327 |
+
"parity",
|
| 328 |
+
"parity",
|
| 329 |
+
"parity",
|
| 330 |
+
"parity",
|
| 331 |
+
"parity",
|
| 332 |
+
"parity",
|
| 333 |
+
"parity",
|
| 334 |
+
"parity",
|
| 335 |
+
"uninformative",
|
| 336 |
+
"uninformative",
|
| 337 |
+
"uninformative",
|
| 338 |
+
"uninformative",
|
| 339 |
+
"uninformative",
|
| 340 |
+
"uninformative",
|
| 341 |
+
"uninformative",
|
| 342 |
+
"uninformative",
|
| 343 |
+
"uninformative",
|
| 344 |
+
"uninformative",
|
| 345 |
+
"uninformative",
|
| 346 |
+
"uninformative",
|
| 347 |
+
"uninformative",
|
| 348 |
+
"uninformative",
|
| 349 |
+
"uninformative",
|
| 350 |
+
"uninformative",
|
| 351 |
+
"uninformative",
|
| 352 |
+
"uninformative",
|
| 353 |
+
"uninformative",
|
| 354 |
+
"uninformative",
|
| 355 |
+
"uninformative",
|
| 356 |
+
"uninformative",
|
| 357 |
+
"uninformative",
|
| 358 |
+
"uninformative",
|
| 359 |
+
"uninformative",
|
| 360 |
+
"uninformative",
|
| 361 |
+
"uninformative",
|
| 362 |
+
"uninformative",
|
| 363 |
+
"uninformative",
|
| 364 |
+
"uninformative",
|
| 365 |
+
"uninformative",
|
| 366 |
+
"uninformative",
|
| 367 |
+
"redundant",
|
| 368 |
+
"redundant",
|
| 369 |
+
"redundant",
|
| 370 |
+
"redundant",
|
| 371 |
+
"redundant",
|
| 372 |
+
"redundant",
|
| 373 |
+
"redundant",
|
| 374 |
+
"redundant",
|
| 375 |
+
"redundant",
|
| 376 |
+
"redundant",
|
| 377 |
+
"redundant",
|
| 378 |
+
"redundant",
|
| 379 |
+
"redundant",
|
| 380 |
+
"redundant",
|
| 381 |
+
"redundant",
|
| 382 |
+
"redundant",
|
| 383 |
+
"redundant",
|
| 384 |
+
"redundant",
|
| 385 |
+
"redundant",
|
| 386 |
+
"redundant",
|
| 387 |
+
"redundant",
|
| 388 |
+
"redundant",
|
| 389 |
+
"redundant",
|
| 390 |
+
"redundant",
|
| 391 |
+
"redundant",
|
| 392 |
+
"redundant",
|
| 393 |
+
"redundant",
|
| 394 |
+
"redundant",
|
| 395 |
+
"redundant",
|
| 396 |
+
"redundant",
|
| 397 |
+
"redundant",
|
| 398 |
+
"redundant",
|
| 399 |
+
"orientation_alias",
|
| 400 |
+
"orientation_alias",
|
| 401 |
+
"orientation_alias",
|
| 402 |
+
"orientation_alias",
|
| 403 |
+
"orientation_alias",
|
| 404 |
+
"orientation_alias",
|
| 405 |
+
"orientation_alias",
|
| 406 |
+
"orientation_alias",
|
| 407 |
+
"orientation_alias",
|
| 408 |
+
"orientation_alias",
|
| 409 |
+
"orientation_alias",
|
| 410 |
+
"orientation_alias",
|
| 411 |
+
"orientation_alias",
|
| 412 |
+
"orientation_alias",
|
| 413 |
+
"orientation_alias",
|
| 414 |
+
"orientation_alias",
|
| 415 |
+
"orientation_alias",
|
| 416 |
+
"orientation_alias",
|
| 417 |
+
"orientation_alias",
|
| 418 |
+
"orientation_alias",
|
| 419 |
+
"orientation_alias",
|
| 420 |
+
"orientation_alias",
|
| 421 |
+
"orientation_alias",
|
| 422 |
+
"orientation_alias",
|
| 423 |
+
"orientation_alias",
|
| 424 |
+
"orientation_alias",
|
| 425 |
+
"orientation_alias",
|
| 426 |
+
"orientation_alias",
|
| 427 |
+
"orientation_alias",
|
| 428 |
+
"orientation_alias",
|
| 429 |
+
"orientation_alias",
|
| 430 |
+
"orientation_alias"
|
| 431 |
+
],
|
| 432 |
+
"budget": [
|
| 433 |
+
0,
|
| 434 |
+
1,
|
| 435 |
+
2
|
| 436 |
+
],
|
| 437 |
+
"cost": [
|
| 438 |
+
0.0,
|
| 439 |
+
0.01,
|
| 440 |
+
0.05
|
| 441 |
+
],
|
| 442 |
+
"metric": [
|
| 443 |
+
"utility",
|
| 444 |
+
"predicted_utility",
|
| 445 |
+
"queries",
|
| 446 |
+
"abstention",
|
| 447 |
+
"available_oracle",
|
| 448 |
+
"encoded_oracle",
|
| 449 |
+
"true_encoder_oracle",
|
| 450 |
+
"true_encoder_fitted_utility",
|
| 451 |
+
"true_conditional_dispersion",
|
| 452 |
+
"root_choice_mismatch"
|
| 453 |
+
],
|
| 454 |
+
"score_shape": [
|
| 455 |
+
2,
|
| 456 |
+
4,
|
| 457 |
+
3,
|
| 458 |
+
3,
|
| 459 |
+
9,
|
| 460 |
+
192,
|
| 461 |
+
3,
|
| 462 |
+
3,
|
| 463 |
+
10
|
| 464 |
+
],
|
| 465 |
+
"all_units_retained": true
|
| 466 |
+
}
|
revision_r1/controlled/native_worlds.csv.gz
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:1390af820583c90646ceb5edd7da2647ea0b9d8aeedfd516c921cdeea1b58581
|
| 3 |
+
size 3779770
|
revision_r1/controlled/numeric_compute.json
ADDED
|
@@ -0,0 +1,9 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"code_sha256": "4a300dfc769473f5afb08e15e877f6c53dfff5f6d3a0abfe17e0043798eecbab",
|
| 3 |
+
"elapsed_analysis_seconds": 19.021096944925375,
|
| 4 |
+
"elapsed_process_seconds": 37.11383769393433,
|
| 5 |
+
"peak_cuda_allocated_bytes": 167947264,
|
| 6 |
+
"device": "cuda:4",
|
| 7 |
+
"original_files_modified": false,
|
| 8 |
+
"new_trainable_parameters": 0
|
| 9 |
+
}
|
revision_r1/controlled/numeric_diagnostic.json
ADDED
|
@@ -0,0 +1,154 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"status": "POST_REVIEW_NUMERICAL_DIAGNOSTIC_NOT_A_COMPETITOR",
|
| 3 |
+
"code_sha256": "4a300dfc769473f5afb08e15e877f6c53dfff5f6d3a0abfe17e0043798eecbab",
|
| 4 |
+
"operation": "Hold each encoder fixed. Compare ordinary pushforward to reversal of raw-state accumulation order, which is the same probability law in exact arithmetic.",
|
| 5 |
+
"metric_axis": [
|
| 6 |
+
"ordinary_realized_utility",
|
| 7 |
+
"reversed_sum_realized_utility",
|
| 8 |
+
"absolute_fitted_optimal_value_difference",
|
| 9 |
+
"maximum_law_cell_difference"
|
| 10 |
+
],
|
| 11 |
+
"all_world_stream_method_rows": 124416,
|
| 12 |
+
"changed_world_stream_method_rows": 2637,
|
| 13 |
+
"maximum_law_cell_difference": 2.220446049250313e-16,
|
| 14 |
+
"maximum_fitted_optimal_value_difference": 6.106226635438361e-16,
|
| 15 |
+
"maximum_realized_utility_difference": 0.27986264044260756,
|
| 16 |
+
"changed_by_estimator": {
|
| 17 |
+
"joint_jeffreys": 2609,
|
| 18 |
+
"naive_bayes": 26,
|
| 19 |
+
"support_mixture": 2
|
| 20 |
+
},
|
| 21 |
+
"interpretation": "Tiny law perturbations can select different fitted-optimal tied or nearly tied actions whose true-law utilities differ. This demonstrates numerical tie sensitivity; it does not uniquely reconstruct every archived discrepancy or justify replacing frozen scores.",
|
| 22 |
+
"strongest_witnesses": [
|
| 23 |
+
{
|
| 24 |
+
"available": 6,
|
| 25 |
+
"labels": 64,
|
| 26 |
+
"estimator": "joint_jeffreys",
|
| 27 |
+
"stream": 1,
|
| 28 |
+
"method": "raw_entropy",
|
| 29 |
+
"world": "confirmation:redundant:009",
|
| 30 |
+
"family": "redundant",
|
| 31 |
+
"maximum_law_cell_difference": 2.7755575615628914e-17,
|
| 32 |
+
"maximum_fitted_optimal_value_difference": 2.7755575615628914e-17,
|
| 33 |
+
"maximum_realized_utility_difference": 0.27986264044260756,
|
| 34 |
+
"positive_budget_average_utility_difference": 0.0
|
| 35 |
+
},
|
| 36 |
+
{
|
| 37 |
+
"available": 6,
|
| 38 |
+
"labels": 64,
|
| 39 |
+
"estimator": "joint_jeffreys",
|
| 40 |
+
"stream": 0,
|
| 41 |
+
"method": "raw_information",
|
| 42 |
+
"world": "confirmation:redundant:030",
|
| 43 |
+
"family": "redundant",
|
| 44 |
+
"maximum_law_cell_difference": 2.7755575615628914e-17,
|
| 45 |
+
"maximum_fitted_optimal_value_difference": 4.163336342344337e-17,
|
| 46 |
+
"maximum_realized_utility_difference": 0.2505645108394803,
|
| 47 |
+
"positive_budget_average_utility_difference": 0.0
|
| 48 |
+
},
|
| 49 |
+
{
|
| 50 |
+
"available": 3,
|
| 51 |
+
"labels": 64,
|
| 52 |
+
"estimator": "joint_jeffreys",
|
| 53 |
+
"stream": 2,
|
| 54 |
+
"method": "posterior_bins",
|
| 55 |
+
"world": "confirmation:orientation_alias:007",
|
| 56 |
+
"family": "orientation_alias",
|
| 57 |
+
"maximum_law_cell_difference": 1.3877787807814457e-17,
|
| 58 |
+
"maximum_fitted_optimal_value_difference": 1.3877787807814457e-17,
|
| 59 |
+
"maximum_realized_utility_difference": 0.24603762586320843,
|
| 60 |
+
"positive_budget_average_utility_difference": 0.0410062709772014
|
| 61 |
+
},
|
| 62 |
+
{
|
| 63 |
+
"available": 6,
|
| 64 |
+
"labels": 64,
|
| 65 |
+
"estimator": "joint_jeffreys",
|
| 66 |
+
"stream": 0,
|
| 67 |
+
"method": "raw_information",
|
| 68 |
+
"world": "confirmation:redundant:027",
|
| 69 |
+
"family": "redundant",
|
| 70 |
+
"maximum_law_cell_difference": 1.3877787807814457e-17,
|
| 71 |
+
"maximum_fitted_optimal_value_difference": 2.7755575615628914e-17,
|
| 72 |
+
"maximum_realized_utility_difference": 0.22993269695902768,
|
| 73 |
+
"positive_budget_average_utility_difference": 0.0
|
| 74 |
+
},
|
| 75 |
+
{
|
| 76 |
+
"available": 3,
|
| 77 |
+
"labels": 64,
|
| 78 |
+
"estimator": "joint_jeffreys",
|
| 79 |
+
"stream": 0,
|
| 80 |
+
"method": "posterior_bins",
|
| 81 |
+
"world": "confirmation:orientation_alias:005",
|
| 82 |
+
"family": "orientation_alias",
|
| 83 |
+
"maximum_law_cell_difference": 2.7755575615628914e-17,
|
| 84 |
+
"maximum_fitted_optimal_value_difference": 5.551115123125783e-17,
|
| 85 |
+
"maximum_realized_utility_difference": 0.21863751129021827,
|
| 86 |
+
"positive_budget_average_utility_difference": 0.10931875564510912
|
| 87 |
+
},
|
| 88 |
+
{
|
| 89 |
+
"available": 6,
|
| 90 |
+
"labels": 64,
|
| 91 |
+
"estimator": "naive_bayes",
|
| 92 |
+
"stream": 1,
|
| 93 |
+
"method": "raw_entropy",
|
| 94 |
+
"world": "confirmation:parity:009",
|
| 95 |
+
"family": "parity",
|
| 96 |
+
"maximum_law_cell_difference": 6.938893903907228e-18,
|
| 97 |
+
"maximum_fitted_optimal_value_difference": 2.7755575615628914e-17,
|
| 98 |
+
"maximum_realized_utility_difference": 0.20848183477792573,
|
| 99 |
+
"positive_budget_average_utility_difference": 0.1036287884194994
|
| 100 |
+
},
|
| 101 |
+
{
|
| 102 |
+
"available": 6,
|
| 103 |
+
"labels": 64,
|
| 104 |
+
"estimator": "joint_jeffreys",
|
| 105 |
+
"stream": 1,
|
| 106 |
+
"method": "raw_information",
|
| 107 |
+
"world": "confirmation:orientation_alias:012",
|
| 108 |
+
"family": "orientation_alias",
|
| 109 |
+
"maximum_law_cell_difference": 1.3877787807814457e-17,
|
| 110 |
+
"maximum_fitted_optimal_value_difference": 1.3877787807814457e-17,
|
| 111 |
+
"maximum_realized_utility_difference": 0.19696465415136627,
|
| 112 |
+
"positive_budget_average_utility_difference": -0.03282744235856104
|
| 113 |
+
},
|
| 114 |
+
{
|
| 115 |
+
"available": 6,
|
| 116 |
+
"labels": 64,
|
| 117 |
+
"estimator": "joint_jeffreys",
|
| 118 |
+
"stream": 0,
|
| 119 |
+
"method": "raw_information",
|
| 120 |
+
"world": "confirmation:orientation_alias:003",
|
| 121 |
+
"family": "orientation_alias",
|
| 122 |
+
"maximum_law_cell_difference": 1.3877787807814457e-17,
|
| 123 |
+
"maximum_fitted_optimal_value_difference": 1.3877787807814457e-17,
|
| 124 |
+
"maximum_realized_utility_difference": 0.15956330285126807,
|
| 125 |
+
"positive_budget_average_utility_difference": 0.026593883808544678
|
| 126 |
+
},
|
| 127 |
+
{
|
| 128 |
+
"available": 6,
|
| 129 |
+
"labels": 64,
|
| 130 |
+
"estimator": "joint_jeffreys",
|
| 131 |
+
"stream": 1,
|
| 132 |
+
"method": "posterior_clusters",
|
| 133 |
+
"world": "confirmation:redundant:014",
|
| 134 |
+
"family": "redundant",
|
| 135 |
+
"maximum_law_cell_difference": 3.469446951953614e-18,
|
| 136 |
+
"maximum_fitted_optimal_value_difference": 1.3877787807814457e-17,
|
| 137 |
+
"maximum_realized_utility_difference": 0.15947780468816652,
|
| 138 |
+
"positive_budget_average_utility_difference": -0.02657963411469442
|
| 139 |
+
},
|
| 140 |
+
{
|
| 141 |
+
"available": 6,
|
| 142 |
+
"labels": 64,
|
| 143 |
+
"estimator": "joint_jeffreys",
|
| 144 |
+
"stream": 1,
|
| 145 |
+
"method": "posterior_bins",
|
| 146 |
+
"world": "confirmation:redundant:014",
|
| 147 |
+
"family": "redundant",
|
| 148 |
+
"maximum_law_cell_difference": 1.3877787807814457e-17,
|
| 149 |
+
"maximum_fitted_optimal_value_difference": 1.3877787807814457e-17,
|
| 150 |
+
"maximum_realized_utility_difference": 0.15947780468816652,
|
| 151 |
+
"positive_budget_average_utility_difference": -0.02657963411469442
|
| 152 |
+
}
|
| 153 |
+
]
|
| 154 |
+
}
|
revision_r1/controlled/numeric_push_sensitivity.csv.gz
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:655e73bc4111c4c5dab7507e4fd029cd5fba7efdde6375757a6b281c94027358
|
| 3 |
+
size 597542
|
revision_r1/controlled/paired_by_condition.csv
ADDED
|
The diff for this file is too large to render.
See raw diff
|
|
|
revision_r1/controlled/privileged_geometry.csv
ADDED
|
The diff for this file is too large to render.
See raw diff
|
|
|
revision_r1/controlled/reference_replay_discrepancies.csv.gz
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:59fad6d56023fedd16c50f0faab3b4f85a8cb322c3ecbcb4f7d5f9c7e5076051
|
| 3 |
+
size 34992
|
revision_r1/controlled/replay_discrepancies.csv.gz
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:6ef693532219b9ba0abab9aaafcc07558a72a20f0ffe961b4bc42a138e873d25
|
| 3 |
+
size 34118
|
revision_r1/controlled/run.py
ADDED
|
@@ -0,0 +1,572 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Post-review opened-world diagnostics; frozen confirmation results are immutable.
|
| 2 |
+
|
| 3 |
+
First run --prepare, then run without --prepare using the same output directory.
|
| 4 |
+
All statistical tensors and decision calculations use CUDA float64. CPU work is
|
| 5 |
+
limited to provenance, output, and descriptive paired-bootstrap aggregation.
|
| 6 |
+
"""
|
| 7 |
+
from __future__ import annotations
|
| 8 |
+
|
| 9 |
+
import argparse
|
| 10 |
+
import csv
|
| 11 |
+
import datetime as dt
|
| 12 |
+
import gzip
|
| 13 |
+
import hashlib
|
| 14 |
+
import json
|
| 15 |
+
import os
|
| 16 |
+
from pathlib import Path
|
| 17 |
+
import sys
|
| 18 |
+
import time
|
| 19 |
+
|
| 20 |
+
PROCESS_START = time.perf_counter()
|
| 21 |
+
HERE = Path(__file__).resolve()
|
| 22 |
+
ARTICLE = HERE.parents[1]
|
| 23 |
+
DEFAULT_RELEASE = next((p for p in [HERE.parent, HERE.parent.parent, ARTICLE / "release/huggingface-revised"]
|
| 24 |
+
if (p / "data/controlled/manifest.json").is_file()), ARTICLE / "release/huggingface-revised")
|
| 25 |
+
METHODS = ["identity", "query_joint", "posterior_clusters", "posterior_bins",
|
| 26 |
+
"future_marginal", "raw_information", "raw_entropy", "hash_code", "action_values"]
|
| 27 |
+
AVAILABILITY = [6, 3]
|
| 28 |
+
SIZES = [64, 256, 1024, 4096]
|
| 29 |
+
ESTIMATORS = ["joint_jeffreys", "naive_bayes", "support_mixture"]
|
| 30 |
+
COSTS = [0.0, 0.01, 0.05]
|
| 31 |
+
METRICS = ["utility", "predicted_utility", "queries", "abstention", "available_oracle",
|
| 32 |
+
"encoded_oracle", "true_encoder_oracle", "true_encoder_fitted_utility",
|
| 33 |
+
"true_conditional_dispersion", "root_choice_mismatch"]
|
| 34 |
+
|
| 35 |
+
|
| 36 |
+
def sha(path):
|
| 37 |
+
return hashlib.sha256(Path(path).read_bytes()).hexdigest()
|
| 38 |
+
|
| 39 |
+
|
| 40 |
+
def dump(path, value):
|
| 41 |
+
Path(path).write_text(json.dumps(value, indent=2, allow_nan=False) + "\n")
|
| 42 |
+
|
| 43 |
+
|
| 44 |
+
def csv_write(path, rows):
|
| 45 |
+
rows = list(rows)
|
| 46 |
+
opener = gzip.open if str(path).endswith(".gz") else open
|
| 47 |
+
with opener(path, "wt", newline="") as f:
|
| 48 |
+
writer = csv.DictWriter(f, fieldnames=list(rows[0]))
|
| 49 |
+
writer.writeheader()
|
| 50 |
+
writer.writerows(rows)
|
| 51 |
+
|
| 52 |
+
|
| 53 |
+
def prepare(out, release, device):
|
| 54 |
+
assert not out.exists(), "The specification must use a fresh output directory."
|
| 55 |
+
out.mkdir(parents=True)
|
| 56 |
+
inputs = [release / "data/controlled/worlds.safetensors",
|
| 57 |
+
release / "data/controlled/worlds.json",
|
| 58 |
+
release / "data/controlled/manifest.json",
|
| 59 |
+
release / "postconfirmation.py",
|
| 60 |
+
release / "unionmind/decision_sufficiency/core.py",
|
| 61 |
+
release / "results/postconfirmation/controlled_worlds.csv.gz",
|
| 62 |
+
release / "results/postconfirmation/controlled_frontiers.csv.gz",
|
| 63 |
+
release / "results/postconfirmation/controlled_timing.json",
|
| 64 |
+
release / "provenance/controlled/confirmation_benchmark.json"]
|
| 65 |
+
spec = dict(created_utc=dt.datetime.now(dt.timezone.utc).isoformat(),
|
| 66 |
+
status="Post-review exploratory analysis of already opened worlds; not confirmation",
|
| 67 |
+
scientific_question="Why does joint minus action-value utility shrink with sample size?",
|
| 68 |
+
authorization="User authorized additional reviewer-response experiments; no new approval requested.",
|
| 69 |
+
source_release=str(release), code_sha256=sha(HERE), device=device,
|
| 70 |
+
input_sha256={str(p.relative_to(release)): sha(p) for p in inputs},
|
| 71 |
+
units="All 192 archived worlds, all six families, all three original nested sample streams",
|
| 72 |
+
grid=dict(availability=AVAILABILITY, records=SIZES, estimators=ESTIMATORS,
|
| 73 |
+
methods=METHODS, budgets=[0, 1, 2], costs=COSTS),
|
| 74 |
+
primary_aggregation="Average positive budgets, all costs and three streams within each world; equal worlds and six families",
|
| 75 |
+
interventions=[
|
| 76 |
+
"Native: estimated-law encoder, estimated encoded-law exact planner, true-law evaluation",
|
| 77 |
+
"Replan the identical native code under the pushed true law to separate encoding and model-estimation loss",
|
| 78 |
+
"Privileged diagnostic: construct each unchanged encoder under the true law, then plan under either true or fitted pushed law; never a deployable competitor"],
|
| 79 |
+
exact_decomposition="J-A = (encoding_loss_A-encoding_loss_J) + (estimation_loss_A-estimation_loss_J)",
|
| 80 |
+
encoding_loss="V(available true raw law)-V(true law pushed through native code)",
|
| 81 |
+
estimation_loss="V(true law pushed through native code)-realized utility of fitted policy",
|
| 82 |
+
alias_diagnostic="Evaluate all unordered raw-state pairs with true current-posterior difference <=1e-12 (exact) and <=0.01 (fixed near-alias threshold), weighted by 2 P(e)P(e'); compare true optimal root choices at each cost and budget and whether each native code merges disagreeing aliases",
|
| 83 |
+
intervals="Descriptive paired 95% percentile bootstrap, 10,000 draws, seed 18701, worlds resampled within fixed families after averaging streams; intervals are not corrected and are not new confirmatory tests",
|
| 84 |
+
practical_scale="Joint-action gain divided by remaining true available-oracle headroom of action-values; report signed gain, not just a percentage",
|
| 85 |
+
timing="Retain published complete-cost records. Fresh warm alternating-order 11-repeat 192-world primary-batch timings include count/estimate, profile, clustering/other encoding, push, exact planning, materialization and 1/32/1024/65536 recorded-event replays. One warmup charged and separately reported. Timing is descriptive on shared hardware.",
|
| 86 |
+
prohibition="No tuning, no model refits, no changed codes/iterations/costs, no dropped units, no original results or manuscript edits, no promotion",
|
| 87 |
+
planned_outputs=["scores.npz", "indices.json", "frontier_all_conditions.csv", "paired_by_condition.csv",
|
| 88 |
+
"sample_frontier.csv", "family_sample_frontier.csv", "native_worlds.csv.gz",
|
| 89 |
+
"alias_pairs.csv.gz", "alias_summary.csv", "privileged_geometry.csv",
|
| 90 |
+
"timing.json", "summary.json", "compute.json"])
|
| 91 |
+
dump(out / "specification.json", spec)
|
| 92 |
+
print(json.dumps(dict(status="SPECIFICATION_RECORDED_BEFORE_NEW_COMPUTATIONS", output=str(out))))
|
| 93 |
+
|
| 94 |
+
|
| 95 |
+
def sync():
|
| 96 |
+
torch.cuda.synchronize()
|
| 97 |
+
|
| 98 |
+
|
| 99 |
+
def timed(fn):
|
| 100 |
+
sync()
|
| 101 |
+
start = time.perf_counter()
|
| 102 |
+
result = fn()
|
| 103 |
+
sync()
|
| 104 |
+
return result, time.perf_counter() - start
|
| 105 |
+
|
| 106 |
+
|
| 107 |
+
def mismatch(policy, code, raw_policy, mass):
|
| 108 |
+
w, r = code.shape
|
| 109 |
+
table = policy["choices"].reshape(3, w, policy["shape"][1], 3, 9)[..., 0]
|
| 110 |
+
selected = table.gather(2, code[None, :, :, None].expand(3, w, r, 3))
|
| 111 |
+
reference = raw_policy["choices"].reshape(3, w, r, 3, 9)[..., 0]
|
| 112 |
+
return ((selected != reference).to(mass.dtype) * mass[None, :, :, None]).sum(2).permute(1, 0, 2)
|
| 113 |
+
|
| 114 |
+
|
| 115 |
+
def collect(out, release, device):
|
| 116 |
+
archive = load("data/controlled/worlds", device)
|
| 117 |
+
manifest = json.loads((release / "data/controlled/manifest.json").read_text())["worlds"]
|
| 118 |
+
world_ids = [r["id"] for r in manifest]
|
| 119 |
+
families = np.array([r["family"] for r in manifest])
|
| 120 |
+
assert len(world_ids) == 192 and len(set(families)) == 6
|
| 121 |
+
scores = np.empty((2, 4, 3, 3, 9, 192, 3, 3, len(METRICS)), dtype=np.float64)
|
| 122 |
+
alias_rows = []
|
| 123 |
+
alias_pair_rows = []
|
| 124 |
+
geometry_rows = []
|
| 125 |
+
progress = []
|
| 126 |
+
for ai, available in enumerate(AVAILABILITY):
|
| 127 |
+
truth = archive["joint"] if available == 6 else core.project(archive["joint"], 3)
|
| 128 |
+
oracle_policy = core.plan(truth)
|
| 129 |
+
oracle = oracle_policy["expected"]
|
| 130 |
+
true_mass = truth.sum((2, 3))
|
| 131 |
+
posterior = truth.sum(-1)[..., 1] / true_mass.clamp_min(1e-300)
|
| 132 |
+
r = truth.shape[1]
|
| 133 |
+
pi, pj = torch.triu_indices(r, r, 1, device=device)
|
| 134 |
+
pair_weight = 2 * true_mass[:, pi] * true_mass[:, pj]
|
| 135 |
+
gap = (posterior[:, pi] - posterior[:, pj]).abs()
|
| 136 |
+
raw_choices = oracle_policy["choices"].reshape(3, 192, r, 3, 9)[..., 0].permute(1, 2, 0, 3)
|
| 137 |
+
disagree = raw_choices[:, pi] != raw_choices[:, pj]
|
| 138 |
+
true_codes = {}
|
| 139 |
+
for method in METHODS:
|
| 140 |
+
code, k = encode(truth, method)
|
| 141 |
+
pushed = core.push(truth, code, k)
|
| 142 |
+
privileged = core.plan(pushed)["expected"]
|
| 143 |
+
true_codes[method] = (code, k, privileged)
|
| 144 |
+
for wi, world in enumerate(world_ids):
|
| 145 |
+
geometry_rows.append(dict(available=available, world=world, family=str(families[wi]), method=method,
|
| 146 |
+
utility=float(privileged[wi, 1:].mean()),
|
| 147 |
+
encoding_loss=float((oracle[wi, 1:] - privileged[wi, 1:]).mean())))
|
| 148 |
+
# Store every eligible pair and its exact law-derived statistics; no sampled pair subset.
|
| 149 |
+
for label, threshold in [("exact", 1e-12), ("near", 0.01)]:
|
| 150 |
+
mask = gap <= threshold
|
| 151 |
+
for wi in range(192):
|
| 152 |
+
inds = mask[wi].nonzero().flatten()
|
| 153 |
+
for ix in inds.tolist():
|
| 154 |
+
alias_pair_rows.append(dict(available=available, world=world_ids[wi], family=str(families[wi]),
|
| 155 |
+
threshold=label, raw_i=int(pi[ix]), raw_j=int(pj[ix]),
|
| 156 |
+
posterior_difference=float(gap[wi, ix]), pair_weight=float(pair_weight[wi, ix]),
|
| 157 |
+
disagreeing_positive_budget_cost_cells=int(disagree[wi, ix, 1:].sum())))
|
| 158 |
+
for ni, n in enumerate(SIZES):
|
| 159 |
+
for ei, estimator in enumerate(ESTIMATORS):
|
| 160 |
+
for stream in range(3):
|
| 161 |
+
fitted = core.estimate(core.count_samples(archive["samples"][:, stream, :n], available), estimator)
|
| 162 |
+
for mi, method in enumerate(METHODS):
|
| 163 |
+
code, k = encode(fitted, method)
|
| 164 |
+
compressed = core.push(fitted, code, k)
|
| 165 |
+
policy = core.plan(compressed)
|
| 166 |
+
pushed_truth = core.push(truth, code, k)
|
| 167 |
+
realized = core.realized(pushed_truth, policy)
|
| 168 |
+
encoded_oracle = core.plan(pushed_truth)["expected"]
|
| 169 |
+
true_code, true_k, privileged_oracle = true_codes[method]
|
| 170 |
+
privileged_fitted = core.plan(core.push(fitted, true_code, true_k))
|
| 171 |
+
privileged_actual = core.realized(core.push(truth, true_code, true_k), privileged_fitted)["utility"]
|
| 172 |
+
dispersion = core.dispersion(truth, code, pushed_truth)[:, None, None].expand_as(oracle)
|
| 173 |
+
root_mismatch = mismatch(policy, code, oracle_policy, true_mass)
|
| 174 |
+
metrics = [realized["utility"], policy["expected"], realized["queries"], realized["abstention"],
|
| 175 |
+
oracle, encoded_oracle, privileged_oracle, privileged_actual, dispersion, root_mismatch]
|
| 176 |
+
assert min(float((oracle - encoded_oracle).min()),
|
| 177 |
+
float((encoded_oracle - realized["utility"]).min())) > -1e-10
|
| 178 |
+
scores[ai, ni, ei, stream, mi] = torch.stack(metrics, -1).cpu().numpy()
|
| 179 |
+
# Exact expected-mass summaries across every eligible pair, all worlds and all cells.
|
| 180 |
+
same_code = code[:, pi] == code[:, pj]
|
| 181 |
+
for label, threshold in [("exact", 1e-12), ("near", 0.01)]:
|
| 182 |
+
weighted = pair_weight * (gap <= threshold)
|
| 183 |
+
denominator = weighted.sum(1)
|
| 184 |
+
den_disagree = (weighted[..., None, None] * disagree).sum(1)
|
| 185 |
+
merged_disagree = (weighted[..., None, None] * disagree * same_code[..., None, None]).sum(1)
|
| 186 |
+
for b in range(3):
|
| 187 |
+
for ci, cost in enumerate(COSTS):
|
| 188 |
+
alias_rows.append(dict(available=available, labels=n, estimator=estimator,
|
| 189 |
+
stream=stream, method=method, threshold=label, budget=b, cost=cost,
|
| 190 |
+
eligible_pair_mass=float(denominator.mean()),
|
| 191 |
+
disagreeing_pair_mass=float(den_disagree[:, b, ci].mean()),
|
| 192 |
+
merged_disagreeing_pair_mass=float(merged_disagree[:, b, ci].mean())))
|
| 193 |
+
progress.append(dict(available=available, labels=n, estimator=estimator))
|
| 194 |
+
print(json.dumps(dict(completed=progress[-1])), flush=True)
|
| 195 |
+
indices = dict(axes=["availability", "labels", "estimator", "stream", "method", "world", "budget", "cost", "metric"],
|
| 196 |
+
availability=AVAILABILITY, labels=SIZES, estimator=ESTIMATORS, stream=[0, 1, 2],
|
| 197 |
+
method=METHODS, world=world_ids, family=families.tolist(), budget=[0, 1, 2], cost=COSTS, metric=METRICS,
|
| 198 |
+
score_shape=list(scores.shape), all_units_retained=True)
|
| 199 |
+
np.savez_compressed(out / "scores.npz", scores=scores)
|
| 200 |
+
dump(out / "indices.json", indices)
|
| 201 |
+
csv_write(out / "alias_pairs.csv.gz", alias_pair_rows)
|
| 202 |
+
csv_write(out / "alias_summary.csv", alias_rows)
|
| 203 |
+
csv_write(out / "privileged_geometry.csv", geometry_rows)
|
| 204 |
+
return scores, indices, archive
|
| 205 |
+
|
| 206 |
+
|
| 207 |
+
def bootstrap_weights(families):
|
| 208 |
+
rng = np.random.default_rng(18701)
|
| 209 |
+
by_family = {}
|
| 210 |
+
combined = np.zeros((10000, len(families)))
|
| 211 |
+
groups = sorted(set(families))
|
| 212 |
+
for family in groups:
|
| 213 |
+
selected = np.flatnonzero(families == family)
|
| 214 |
+
draw = rng.integers(len(selected), size=(10000, len(selected)))
|
| 215 |
+
weights = np.stack([np.bincount(d, minlength=len(selected)) for d in draw]).astype(float) / len(selected)
|
| 216 |
+
combined[:, selected] = weights / len(groups)
|
| 217 |
+
by_family[family] = (selected, weights)
|
| 218 |
+
return combined, by_family
|
| 219 |
+
|
| 220 |
+
|
| 221 |
+
def summarize(out, release, scores, indices):
|
| 222 |
+
families = np.asarray(indices["family"])
|
| 223 |
+
# Average streams before any resampling. Cells and streams are not independent worlds.
|
| 224 |
+
worlds = scores.mean(3)
|
| 225 |
+
positive = worlds[..., 1:, :, :].mean((-3, -2))
|
| 226 |
+
metric = {name: i for i, name in enumerate(METRICS)}
|
| 227 |
+
ji, av = METHODS.index("query_joint"), METHODS.index("action_values")
|
| 228 |
+
utility = worlds[..., metric["utility"]]
|
| 229 |
+
encoding = worlds[..., metric["available_oracle"]] - worlds[..., metric["encoded_oracle"]]
|
| 230 |
+
estimation = worlds[..., metric["encoded_oracle"]] - utility
|
| 231 |
+
delta = utility[:, :, :, ji] - utility[:, :, :, av]
|
| 232 |
+
encoding_contribution = encoding[:, :, :, av] - encoding[:, :, :, ji]
|
| 233 |
+
estimation_contribution = estimation[:, :, :, av] - estimation[:, :, :, ji]
|
| 234 |
+
assert np.max(np.abs(delta - encoding_contribution - estimation_contribution)) < 1e-12
|
| 235 |
+
# Reproduce every saved native world score, not only a headline slice.
|
| 236 |
+
maximum_error = 0.0
|
| 237 |
+
primary_replay_error = 0.0
|
| 238 |
+
checked = 0
|
| 239 |
+
discrepancies = []
|
| 240 |
+
with gzip.open(release / "results/postconfirmation/controlled_worlds.csv.gz", "rt") as f:
|
| 241 |
+
world_index = {w: i for i, w in enumerate(indices["world"])}
|
| 242 |
+
for row in csv.DictReader(f):
|
| 243 |
+
key = (AVAILABILITY.index(int(row["available"])), SIZES.index(int(row["labels"])),
|
| 244 |
+
ESTIMATORS.index(row["estimator"]), int(row["stream"]), METHODS.index(row["method"]), world_index[row["world"]])
|
| 245 |
+
replayed = scores[key][1:, :, metric["utility"]].mean()
|
| 246 |
+
error = abs(float(row["utility"]) - replayed)
|
| 247 |
+
maximum_error = max(maximum_error, error)
|
| 248 |
+
if int(row["available"]) == 6 and row["estimator"] == "support_mixture" and row["method"] in ("identity", "query_joint", "action_values"):
|
| 249 |
+
primary_replay_error = max(primary_replay_error, error)
|
| 250 |
+
if error > 1e-10:
|
| 251 |
+
discrepancies.append(dict(**row, replayed_utility=float(replayed), difference=float(replayed)-float(row["utility"])))
|
| 252 |
+
checked += 1
|
| 253 |
+
# Preserve finite-precision/tie discrepancies outside the exact headline frontier.
|
| 254 |
+
# Never mix archived realized values with freshly recomputed code oracles.
|
| 255 |
+
assert checked == 124416 and primary_replay_error < 1e-10
|
| 256 |
+
if discrepancies:
|
| 257 |
+
csv_write(out / "replay_discrepancies.csv.gz", discrepancies)
|
| 258 |
+
native_rows = []
|
| 259 |
+
for ai, available in enumerate(AVAILABILITY):
|
| 260 |
+
for ni, n in enumerate(SIZES):
|
| 261 |
+
for ei, estimator in enumerate(ESTIMATORS):
|
| 262 |
+
for mi, method in enumerate(METHODS):
|
| 263 |
+
for wi, world in enumerate(indices["world"]):
|
| 264 |
+
values = positive[ai, ni, ei, mi, wi]
|
| 265 |
+
native_rows.append(dict(available=available, labels=n, estimator=estimator, method=method,
|
| 266 |
+
world=world, family=indices["family"][wi], **dict(zip(METRICS, values.tolist())),
|
| 267 |
+
encoding_loss=float(values[metric["available_oracle"]] - values[metric["encoded_oracle"]]),
|
| 268 |
+
estimation_loss=float(values[metric["encoded_oracle"]] - values[metric["utility"]])))
|
| 269 |
+
csv_write(out / "native_worlds.csv.gz", native_rows)
|
| 270 |
+
frontiers = []
|
| 271 |
+
for ai, available in enumerate(AVAILABILITY):
|
| 272 |
+
for ni, n in enumerate(SIZES):
|
| 273 |
+
for ei, estimator in enumerate(ESTIMATORS):
|
| 274 |
+
for mi, method in enumerate(METHODS):
|
| 275 |
+
for b in range(3):
|
| 276 |
+
for ci, cost in enumerate(COSTS):
|
| 277 |
+
v = worlds[ai, ni, ei, mi, :, b, ci].mean(0)
|
| 278 |
+
frontiers.append(dict(available=available, labels=n, estimator=estimator, method=method,
|
| 279 |
+
budget=b, cost=cost, **dict(zip(METRICS, v.tolist())),
|
| 280 |
+
encoding_loss=float(v[4] - v[5]), estimation_loss=float(v[5] - v[0])))
|
| 281 |
+
csv_write(out / "frontier_all_conditions.csv", frontiers)
|
| 282 |
+
weights, family_weights = bootstrap_weights(families)
|
| 283 |
+
# All conditions form one descriptive matrix, retaining every family/cost/budget.
|
| 284 |
+
flat_delta = delta.transpose(3, 0, 1, 2, 4, 5).reshape(192, -1)
|
| 285 |
+
paired_rows = []
|
| 286 |
+
for group in ["ALL"] + sorted(set(families)):
|
| 287 |
+
selected, w = (np.arange(192), weights) if group == "ALL" else family_weights[group]
|
| 288 |
+
boot = w @ flat_delta[selected]
|
| 289 |
+
interval = np.quantile(boot, [.025, .975], axis=0).reshape(2, 2, 4, 3, 3, 3)
|
| 290 |
+
for ai, available in enumerate(AVAILABILITY):
|
| 291 |
+
for ni, n in enumerate(SIZES):
|
| 292 |
+
for ei, estimator in enumerate(ESTIMATORS):
|
| 293 |
+
for b in range(3):
|
| 294 |
+
for ci, cost in enumerate(COSTS):
|
| 295 |
+
key = (ai, ni, ei)
|
| 296 |
+
oracle = worlds[key][ji, selected, b, ci, metric["available_oracle"]].mean()
|
| 297 |
+
a = utility[key][av, selected, b, ci].mean()
|
| 298 |
+
d = delta[key][selected, b, ci].mean()
|
| 299 |
+
paired_rows.append(dict(family=group, available=available, labels=n, estimator=estimator,
|
| 300 |
+
budget=b, cost=cost, worlds=len(selected), joint_minus_action=float(d),
|
| 301 |
+
paired_95_low=float(interval[0, ai, ni, ei, b, ci]),
|
| 302 |
+
paired_95_high=float(interval[1, ai, ni, ei, b, ci]),
|
| 303 |
+
encoding_contribution=float(encoding_contribution[key][selected, b, ci].mean()),
|
| 304 |
+
estimation_contribution=float(estimation_contribution[key][selected, b, ci].mean()),
|
| 305 |
+
action_remaining_headroom=float(oracle-a),
|
| 306 |
+
fraction_action_headroom_closed=float(d/(oracle-a)) if oracle-a > 1e-12 else None))
|
| 307 |
+
csv_write(out / "paired_by_condition.csv", paired_rows)
|
| 308 |
+
samples, family_samples = [], []
|
| 309 |
+
for group in ["ALL"] + sorted(set(families)):
|
| 310 |
+
selected, w = (np.arange(192), weights) if group == "ALL" else family_weights[group]
|
| 311 |
+
for ni, n in enumerate(SIZES):
|
| 312 |
+
v = positive[0, ni, 2]
|
| 313 |
+
d = v[ji, :, 0] - v[av, :, 0]
|
| 314 |
+
ci = np.quantile(w @ d[selected], [.025, .975])
|
| 315 |
+
econt = (v[av, :, 4]-v[av, :, 5]) - (v[ji, :, 4]-v[ji, :, 5])
|
| 316 |
+
mcont = (v[av, :, 5]-v[av, :, 0]) - (v[ji, :, 5]-v[ji, :, 0])
|
| 317 |
+
headroom = (v[av, selected, 4]-v[av, selected, 0]).mean()
|
| 318 |
+
row = dict(family=group, records=n, worlds=len(selected),
|
| 319 |
+
**{method: float(v[mi, selected, 0].mean()) for mi, method in enumerate(METHODS)},
|
| 320 |
+
true_oracle=float(v[0, selected, 4].mean()), joint_minus_action=float(d[selected].mean()),
|
| 321 |
+
paired_95_low=float(ci[0]), paired_95_high=float(ci[1]),
|
| 322 |
+
action_encoding_loss=float((v[av, selected, 4]-v[av, selected, 5]).mean()),
|
| 323 |
+
joint_encoding_loss=float((v[ji, selected, 4]-v[ji, selected, 5]).mean()),
|
| 324 |
+
action_estimation_loss=float((v[av, selected, 5]-v[av, selected, 0]).mean()),
|
| 325 |
+
joint_estimation_loss=float((v[ji, selected, 5]-v[ji, selected, 0]).mean()),
|
| 326 |
+
encoding_contribution=float(econt[selected].mean()), estimation_contribution=float(mcont[selected].mean()),
|
| 327 |
+
action_remaining_headroom=float(headroom),
|
| 328 |
+
fraction_action_headroom_closed=float(d[selected].mean()/headroom) if headroom > 1e-12 else None,
|
| 329 |
+
true_code_joint_oracle=float(v[ji, selected, 6].mean()),
|
| 330 |
+
true_code_action_oracle=float(v[av, selected, 6].mean()),
|
| 331 |
+
true_code_joint_fitted=float(v[ji, selected, 7].mean()),
|
| 332 |
+
true_code_action_fitted=float(v[av, selected, 7].mean()),
|
| 333 |
+
joint_root_choice_mismatch=float(v[ji, selected, 9].mean()),
|
| 334 |
+
action_root_choice_mismatch=float(v[av, selected, 9].mean()),
|
| 335 |
+
strongest_uncompressed_mean=max(float(positive[0, ni, ei, 0, selected, 0].mean()) for ei in range(3)))
|
| 336 |
+
(samples if group == "ALL" else family_samples).append(row)
|
| 337 |
+
csv_write(out / "sample_frontier.csv", samples)
|
| 338 |
+
csv_write(out / "family_sample_frontier.csv", family_samples)
|
| 339 |
+
summary = dict(status="POST_REVIEW_EXPLORATORY_OPENED_DATA_NO_NEW_CONFIRMATION", all_worlds=192,
|
| 340 |
+
native_saved_world_scores_checked=checked, native_saved_world_scores_within_1e_10=checked-len(discrepancies),
|
| 341 |
+
replay_discrepancy_count=len(discrepancies), maximum_absolute_replay_error=maximum_error,
|
| 342 |
+
exact_primary_mixture_joint_action_identity_frontier_maximum_replay_error=primary_replay_error,
|
| 343 |
+
exact_decomposition_max_error=float(np.max(np.abs(delta-encoding_contribution-estimation_contribution))),
|
| 344 |
+
primary_sample_frontier=samples, family_sample_frontier=family_samples,
|
| 345 |
+
all_conditions_reported=True, intervals="Descriptive paired within-family bootstrap, 10000 draws, seed 18701; no multiplicity correction or confirmatory interpretation",
|
| 346 |
+
limitations=["Constructed worlds share favorable finite-law structure; no external validity follows.",
|
| 347 |
+
"True-law encoders and true-law replanning have privileged access and are explanatory diagnostics only.",
|
| 348 |
+
"Exact additive loss separation identifies where the observed finite-grid gap arises; it does not identify a unique learning-theoretic cause or guarantee asymptotic dominance.",
|
| 349 |
+
"All algorithms retain the released deterministic initialization and 32 iterations; no attempt to optimize either clustering objective.",
|
| 350 |
+
"Headroom fractions are ratios of averages and not operational money or sensing savings.",
|
| 351 |
+
"Warm timing is a local batch observation on shared hardware; hardware startup and source data access are reported separately."])
|
| 352 |
+
if discrepancies:
|
| 353 |
+
summary["limitations"].append(f"{len(discrepancies)} archived native world means do not exactly replay in the expanded execution trace; all such rows are retained separately. Primary-mixture joint/action-value/identity world means at every sample budget replay with maximum error {primary_replay_error:.3g}. New full-grid diagnostics are recomputations, not replacements for frozen archival scores.")
|
| 354 |
+
dump(out / "summary.json", summary)
|
| 355 |
+
return summary
|
| 356 |
+
|
| 357 |
+
|
| 358 |
+
def benchmark(out, release, device, archive):
|
| 359 |
+
samples = archive["samples"][:, 0, :1024]
|
| 360 |
+
g = torch.Generator(device=device).manual_seed(18751)
|
| 361 |
+
ns = [1, 32, 1024, 65536]
|
| 362 |
+
def fixture():
|
| 363 |
+
selected = torch.randint(1024, (max(ns),), device=device, generator=g)
|
| 364 |
+
w = torch.arange(max(ns), device=device) % 192
|
| 365 |
+
events = samples[w, selected]
|
| 366 |
+
return (events//4) % 64, events % 4
|
| 367 |
+
(requests, future), fixture_seconds = timed(fixture)
|
| 368 |
+
records, warmups = [], []
|
| 369 |
+
for repetition in range(-1, 11):
|
| 370 |
+
order = METHODS if repetition % 2 == 0 else list(reversed(METHODS))
|
| 371 |
+
for method in order:
|
| 372 |
+
(counts, fitted), estimation_seconds = timed(lambda: (lambda counts: (counts, core.estimate(counts, "support_mixture")))(core.count_samples(samples, 6)))
|
| 373 |
+
profile_seconds = 0.0
|
| 374 |
+
if method in ("query_joint", "action_values"):
|
| 375 |
+
def profile():
|
| 376 |
+
mass = fitted.sum((2, 3))
|
| 377 |
+
features = action_profiles(fitted) if method == "action_values" else (fitted/mass[..., None, None]).flatten(2).sqrt()
|
| 378 |
+
return features, mass
|
| 379 |
+
(features, mass), profile_seconds = timed(profile)
|
| 380 |
+
code, encoding_seconds = timed(lambda: core.weighted_clusters(features, mass))
|
| 381 |
+
k = 8
|
| 382 |
+
else:
|
| 383 |
+
(code, k), encoding_seconds = timed(lambda: encode(fitted, method))
|
| 384 |
+
compressed, push_seconds = timed(lambda: core.push(fitted, code, k))
|
| 385 |
+
policy, planning_seconds = timed(lambda: core.plan(compressed))
|
| 386 |
+
def materialize():
|
| 387 |
+
deployed = dict(policy)
|
| 388 |
+
deployed["choices"] = policy["choices"].contiguous()
|
| 389 |
+
return (None if method == "identity" else code.to(torch.uint8).contiguous()), deployed
|
| 390 |
+
(deployed_code, deployed_policy), materialization_seconds = timed(materialize)
|
| 391 |
+
replay_times = {}
|
| 392 |
+
checksum = {}
|
| 393 |
+
for n in ns:
|
| 394 |
+
(actions, queries), elapsed = timed(lambda: replay(deployed_policy, deployed_code, requests[:n], future[:n]))
|
| 395 |
+
replay_times[str(n)] = elapsed
|
| 396 |
+
checksum[str(n)] = dict(actions=int(actions.sum()), queries=int(queries.sum()))
|
| 397 |
+
total = estimation_seconds+profile_seconds+encoding_seconds+push_seconds+planning_seconds+materialization_seconds
|
| 398 |
+
row = dict(repetition=repetition, method=method, estimation_seconds=estimation_seconds,
|
| 399 |
+
profile_seconds=profile_seconds, clustering_or_other_encoder_seconds=encoding_seconds,
|
| 400 |
+
push_seconds=push_seconds, exact_planning_seconds=planning_seconds,
|
| 401 |
+
materialization_seconds=materialization_seconds, complete_warm_construction_seconds=total,
|
| 402 |
+
replay_seconds=replay_times, checksums=checksum,
|
| 403 |
+
resident_law_bytes_per_world=compressed[0].numel()*compressed.element_size(),
|
| 404 |
+
resident_encoder_bytes_per_world=0 if deployed_code is None else deployed_code[0].numel()*deployed_code.element_size(),
|
| 405 |
+
deployed_choices_bytes_per_world=policy["choices"].numel()*policy["choices"].element_size()//192,
|
| 406 |
+
planner_values_bytes_per_world=policy["values"].numel()*policy["values"].element_size()//192,
|
| 407 |
+
count_state_bytes_per_world=counts[0].numel()*counts.element_size(),
|
| 408 |
+
source_events_bytes_per_world=samples[0].numel()*samples.element_size(),
|
| 409 |
+
temporary_profile_bytes_per_world=features[0].numel()*features.element_size() if method in ("query_joint", "action_values") else 0)
|
| 410 |
+
(warmups if repetition == -1 else records).append(row)
|
| 411 |
+
medians = {}
|
| 412 |
+
phases = ["estimation_seconds", "profile_seconds", "clustering_or_other_encoder_seconds", "push_seconds",
|
| 413 |
+
"exact_planning_seconds", "materialization_seconds", "complete_warm_construction_seconds"]
|
| 414 |
+
for method in METHODS:
|
| 415 |
+
rows = [r for r in records if r["method"] == method]
|
| 416 |
+
median = {p: float(np.median([r[p] for r in rows])) for p in phases}
|
| 417 |
+
warm = next(r for r in warmups if r["method"] == method)
|
| 418 |
+
median["replay_seconds"] = {str(n): float(np.median([r["replay_seconds"][str(n)] for r in rows])) for n in ns}
|
| 419 |
+
median["charged_warmup_seconds"] = warm["complete_warm_construction_seconds"] + sum(warm["replay_seconds"].values())
|
| 420 |
+
median["construction_plus_replay_seconds"] = {str(n): float(np.median([r["complete_warm_construction_seconds"]+r["replay_seconds"][str(n)] for r in rows])) for n in ns}
|
| 421 |
+
for k in warm:
|
| 422 |
+
if k.endswith("bytes_per_world"):
|
| 423 |
+
median[k] = warm[k]
|
| 424 |
+
medians[method] = median
|
| 425 |
+
original = json.loads((release / "provenance/controlled/confirmation_benchmark.json").read_text())
|
| 426 |
+
published = json.loads((release / "results/postconfirmation/controlled_timing.json").read_text())
|
| 427 |
+
old_medians = {m: float(np.median([r["complete_construction_seconds"] for r in published["repetitions"] if r["method"] == m])) for m in METHODS}
|
| 428 |
+
timing = dict(scope="192 worlds, 1024 records, stream 0, support mixture, float64 CUDA; 11 alternating-order warm repetitions plus one charged warmup",
|
| 429 |
+
shared_hardware=True, phase_synchronization_overhead_included=True, request_fixture_seconds=fixture_seconds,
|
| 430 |
+
fresh_medians=medians, fresh_records=records, fresh_warmups=warmups,
|
| 431 |
+
source_postconfirmation_medians_seconds=old_medians,
|
| 432 |
+
source_complete_benchmark={k: original[k] for k in ["scope", "cuda_startup_seconds", "cuda_warmup_seconds", "medians"]},
|
| 433 |
+
notes=["Phase synchronization can increase fresh construction latency relative to a fused production call.",
|
| 434 |
+
"The replay has fixed two-query budget and 0.01 query cost, matching the original transport benchmark; scientific utilities retain all nine cost/budget cells.",
|
| 435 |
+
"Retaining choices alone suffices for decisions; law/value-table storage is stated separately, not silently omitted.",
|
| 436 |
+
"Request fixtures are already recorded events, not sensor/backend access; operational sensing and data movement are not measured."])
|
| 437 |
+
dump(out / "timing.json", timing)
|
| 438 |
+
return timing
|
| 439 |
+
|
| 440 |
+
|
| 441 |
+
def numeric_diagnostic(out, release, device, archive):
|
| 442 |
+
"""Keep encoder fixed; change only the mathematically irrelevant summation order.
|
| 443 |
+
|
| 444 |
+
This diagnoses finite-precision argmax sensitivity, without changing or
|
| 445 |
+
repairing the archived algorithms. It is never a utility competitor.
|
| 446 |
+
"""
|
| 447 |
+
rows = []
|
| 448 |
+
values = np.empty((2, 4, 3, 3, 9, 192, 3, 3, 4))
|
| 449 |
+
manifest = json.loads((release / "data/controlled/manifest.json").read_text())["worlds"]
|
| 450 |
+
for ai, available in enumerate(AVAILABILITY):
|
| 451 |
+
truth = archive["joint"] if available == 6 else core.project(archive["joint"], 3)
|
| 452 |
+
for ni, n in enumerate(SIZES):
|
| 453 |
+
for ei, estimator in enumerate(ESTIMATORS):
|
| 454 |
+
for stream in range(3):
|
| 455 |
+
fitted = core.estimate(core.count_samples(archive["samples"][:, stream, :n], available), estimator)
|
| 456 |
+
for mi, method in enumerate(METHODS):
|
| 457 |
+
code, k = encode(fitted, method)
|
| 458 |
+
normal = core.push(fitted, code, k)
|
| 459 |
+
reversed_sum = core.push(fitted.flip(1), code.flip(1), k)
|
| 460 |
+
p, q = core.plan(normal), core.plan(reversed_sum)
|
| 461 |
+
gold = core.push(truth, code, k)
|
| 462 |
+
a, b = core.realized(gold, p)["utility"], core.realized(gold, q)["utility"]
|
| 463 |
+
law_difference = (normal-reversed_sum).abs().flatten(1).max(1).values
|
| 464 |
+
fitted_difference = (p["expected"]-q["expected"]).abs()
|
| 465 |
+
values[ai, ni, ei, stream, mi] = torch.stack([a, b, fitted_difference,
|
| 466 |
+
law_difference[:, None, None].expand_as(a)], -1).cpu().numpy()
|
| 467 |
+
for wi, world in enumerate(manifest):
|
| 468 |
+
rows.append(dict(available=available, labels=n, estimator=estimator, stream=stream,
|
| 469 |
+
method=method, world=world["id"], family=world["family"],
|
| 470 |
+
maximum_law_cell_difference=float(law_difference[wi]),
|
| 471 |
+
maximum_fitted_optimal_value_difference=float(fitted_difference[wi].max()),
|
| 472 |
+
maximum_realized_utility_difference=float((a[wi]-b[wi]).abs().max()),
|
| 473 |
+
positive_budget_average_utility_difference=float((b[wi, 1:]-a[wi, 1:]).mean())))
|
| 474 |
+
np.savez_compressed(out / "numeric_push_sensitivity.npz", scores=values)
|
| 475 |
+
csv_write(out / "numeric_push_sensitivity.csv.gz", rows)
|
| 476 |
+
changed = [r for r in rows if r["maximum_realized_utility_difference"] > 1e-10]
|
| 477 |
+
report = dict(status="POST_REVIEW_NUMERICAL_DIAGNOSTIC_NOT_A_COMPETITOR", code_sha256=sha(HERE),
|
| 478 |
+
operation="Hold each encoder fixed. Compare ordinary pushforward to reversal of raw-state accumulation order, which is the same probability law in exact arithmetic.",
|
| 479 |
+
metric_axis=["ordinary_realized_utility", "reversed_sum_realized_utility", "absolute_fitted_optimal_value_difference", "maximum_law_cell_difference"],
|
| 480 |
+
all_world_stream_method_rows=len(rows), changed_world_stream_method_rows=len(changed),
|
| 481 |
+
maximum_law_cell_difference=max(r["maximum_law_cell_difference"] for r in rows),
|
| 482 |
+
maximum_fitted_optimal_value_difference=max(r["maximum_fitted_optimal_value_difference"] for r in rows),
|
| 483 |
+
maximum_realized_utility_difference=max(r["maximum_realized_utility_difference"] for r in rows),
|
| 484 |
+
changed_by_estimator={e: sum(r["estimator"] == e for r in changed) for e in ESTIMATORS},
|
| 485 |
+
interpretation="Tiny law perturbations can select different fitted-optimal tied or nearly tied actions whose true-law utilities differ. This demonstrates numerical tie sensitivity; it does not uniquely reconstruct every archived discrepancy or justify replacing frozen scores.",
|
| 486 |
+
strongest_witnesses=sorted(changed, key=lambda r: r["maximum_realized_utility_difference"], reverse=True)[:10])
|
| 487 |
+
dump(out / "numeric_diagnostic.json", report)
|
| 488 |
+
print(json.dumps(report), flush=True)
|
| 489 |
+
return report
|
| 490 |
+
|
| 491 |
+
|
| 492 |
+
def main():
|
| 493 |
+
parser = argparse.ArgumentParser(description=__doc__)
|
| 494 |
+
parser.add_argument("--release", type=Path, default=DEFAULT_RELEASE)
|
| 495 |
+
parser.add_argument("--output", required=True, type=Path)
|
| 496 |
+
parser.add_argument("--device", default="cuda:4")
|
| 497 |
+
parser.add_argument("--prepare", action="store_true")
|
| 498 |
+
parser.add_argument("--resume-aggregation", action="store_true", help="Continue an explicitly recorded completed grid after a validation/reporting repair.")
|
| 499 |
+
parser.add_argument("--reference-only", action="store_true", help="Run the unmodified released postconfirmation.controlled function in a fresh reference_replay directory.")
|
| 500 |
+
parser.add_argument("--numeric-only", action="store_true", help="Only run the recorded fixed-code accumulation-order sensitivity diagnostic.")
|
| 501 |
+
args = parser.parse_args()
|
| 502 |
+
out, release = args.output.resolve(), args.release.resolve()
|
| 503 |
+
if args.prepare:
|
| 504 |
+
prepare(out, release, args.device)
|
| 505 |
+
return
|
| 506 |
+
spec = json.loads((out / "specification.json").read_text())
|
| 507 |
+
assert spec["code_sha256"] == sha(HERE), "Code changed after the recorded specification."
|
| 508 |
+
if not (args.resume_aggregation or args.reference_only or args.numeric_only):
|
| 509 |
+
assert not (out / "scores.npz").exists(), "Never overwrite a result."
|
| 510 |
+
for relative, expected in spec["input_sha256"].items():
|
| 511 |
+
assert sha(release / relative) == expected, relative
|
| 512 |
+
global np, torch, core, load, encode, action_profiles, replay
|
| 513 |
+
import numpy as np
|
| 514 |
+
import torch
|
| 515 |
+
sys.path.insert(0, str(release))
|
| 516 |
+
from reproduce import load, verify
|
| 517 |
+
from postconfirmation import encode, action_profiles
|
| 518 |
+
from unionmind.decision_sufficiency import core
|
| 519 |
+
from unionmind.decision_sufficiency.benchmark import replay
|
| 520 |
+
assert args.device.startswith("cuda") and torch.cuda.is_available()
|
| 521 |
+
torch.cuda.set_device(args.device)
|
| 522 |
+
torch.backends.cuda.matmul.allow_tf32 = False
|
| 523 |
+
torch.set_num_threads(4)
|
| 524 |
+
_, startup_seconds = timed(lambda: torch.cuda.init())
|
| 525 |
+
files_verified = verify()
|
| 526 |
+
analysis_start = time.perf_counter()
|
| 527 |
+
with torch.no_grad():
|
| 528 |
+
if args.reference_only:
|
| 529 |
+
from postconfirmation import controlled
|
| 530 |
+
reference = out / "reference_replay"
|
| 531 |
+
reference.mkdir()
|
| 532 |
+
controlled(reference, args.device)
|
| 533 |
+
print(json.dumps(dict(status="REFERENCE_REPLAY_COMPLETED", output=str(reference))))
|
| 534 |
+
return
|
| 535 |
+
if args.numeric_only:
|
| 536 |
+
assert not (out / "numeric_push_sensitivity.npz").exists()
|
| 537 |
+
numeric_diagnostic(out, release, args.device, load("data/controlled/worlds", args.device))
|
| 538 |
+
dump(out / "numeric_compute.json", dict(code_sha256=sha(HERE),
|
| 539 |
+
elapsed_analysis_seconds=time.perf_counter()-analysis_start,
|
| 540 |
+
elapsed_process_seconds=time.perf_counter()-PROCESS_START,
|
| 541 |
+
peak_cuda_allocated_bytes=torch.cuda.max_memory_allocated(), device=args.device,
|
| 542 |
+
original_files_modified=False, new_trainable_parameters=0))
|
| 543 |
+
return
|
| 544 |
+
if args.resume_aggregation:
|
| 545 |
+
scores = np.load(out / "scores.npz")["scores"]
|
| 546 |
+
indices = json.loads((out / "indices.json").read_text())
|
| 547 |
+
archive = load("data/controlled/worlds", args.device)
|
| 548 |
+
else:
|
| 549 |
+
scores, indices, archive = collect(out, release, args.device)
|
| 550 |
+
summary = summarize(out, release, scores, indices)
|
| 551 |
+
timing = benchmark(out, release, args.device, archive)
|
| 552 |
+
numeric_diagnostic(out, release, args.device, archive)
|
| 553 |
+
sync()
|
| 554 |
+
ledger = dict(created_utc=dt.datetime.now(dt.timezone.utc).isoformat(), status="COMPLETED",
|
| 555 |
+
source_release_files_hash_verified=files_verified, code_sha256=sha(HERE), specification_sha256=sha(out / "specification.json"),
|
| 556 |
+
command=sys.argv, device=args.device, gpu=torch.cuda.get_device_name(args.device),
|
| 557 |
+
python=sys.version, pytorch=torch.__version__, cuda=torch.version.cuda,
|
| 558 |
+
elapsed_analysis_seconds=time.perf_counter()-analysis_start,
|
| 559 |
+
elapsed_since_standard_library_import_seconds=time.perf_counter()-PROCESS_START,
|
| 560 |
+
first_cuda_noop_seconds=startup_seconds, peak_cuda_allocated_bytes=torch.cuda.max_memory_allocated(),
|
| 561 |
+
opened_worlds=True, new_backbone_executions=0, new_trainable_parameters=0, refitted_neural_models=0,
|
| 562 |
+
fitted_statistical_tables="Recomputed original estimators from archived counts, unchanged formulas",
|
| 563 |
+
original_files_modified=False, all_selected_units_retained=True,
|
| 564 |
+
all_statistics_descriptive=True, promotion_decisions_changed=False)
|
| 565 |
+
dump(out / "compute.json", ledger)
|
| 566 |
+
(out / "replay_code.py").write_bytes(HERE.read_bytes())
|
| 567 |
+
dump(out / "output_sha256.json", {p.name: sha(p) for p in sorted(out.iterdir()) if p.is_file() and p.name != "output_sha256.json"})
|
| 568 |
+
print(json.dumps(dict(status="COMPLETED", sample_frontier=summary["primary_sample_frontier"], compute=ledger), indent=2))
|
| 569 |
+
|
| 570 |
+
|
| 571 |
+
if __name__ == "__main__":
|
| 572 |
+
main()
|
revision_r1/controlled/sample_frontier.csv
ADDED
|
@@ -0,0 +1,5 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
family,records,worlds,identity,query_joint,posterior_clusters,posterior_bins,future_marginal,raw_information,raw_entropy,hash_code,action_values,true_oracle,joint_minus_action,paired_95_low,paired_95_high,action_encoding_loss,joint_encoding_loss,action_estimation_loss,joint_estimation_loss,encoding_contribution,estimation_contribution,action_remaining_headroom,fraction_action_headroom_closed,true_code_joint_oracle,true_code_action_oracle,true_code_joint_fitted,true_code_action_fitted,joint_root_choice_mismatch,action_root_choice_mismatch,strongest_uncompressed_mean
|
| 2 |
+
ALL,64,192,0.2861019748560038,0.26584992657807344,0.2590280944592631,0.26074362507619164,0.2639323021727651,0.2722343061169504,0.21481644347086729,0.10239193116286986,0.26390613849214967,0.7139850169951533,0.001943788085923736,0.00021308171557470748,0.003647181976886664,0.23247242290469453,0.22743575439464836,0.21760645559830893,0.22069933602243133,0.005036668510046183,-0.0030928804241224478,0.4500788785030035,0.004318772061441592,0.7137990767560151,0.7138471222579433,0.33015852361105136,0.32977290874844095,0.5728323618858121,0.5747586230569347,0.33512680859485605
|
| 3 |
+
ALL,256,192,0.47464585889067984,0.46153544711762456,0.3821578365055907,0.37086161330575007,0.3117767655744309,0.39616931811795314,0.3287390155602434,0.15862081020012567,0.4421891239875439,0.7139850169951533,0.019346323130080677,0.016004547809026087,0.02272642638711617,0.11063576081059036,0.07695151696279438,0.16116013219701888,0.1754980529147342,0.033684243847795996,-0.014337920717715315,0.27179589300760926,0.07117960067755348,0.7137990767560151,0.7138471222579433,0.5040257394630626,0.5033361534654197,0.5138373564572506,0.532667401596573,0.5503906781894238
|
| 4 |
+
ALL,1024,192,0.6421842631007378,0.651568196746697,0.5045968970651954,0.4901829803145068,0.3289223833583756,0.5502229409756352,0.5192618455652109,0.20875256335562897,0.6375946140433673,0.7139850169951533,0.013973582703329749,0.012156191041507863,0.01579707133440269,0.020288801609760987,0.010064365748209498,0.05610160134202489,0.052352454500246635,0.010224435861551489,0.00374914684177826,0.07639040295178588,0.18292327522017698,0.7137990767560151,0.7138471222579433,0.6654659197644641,0.6648982429373541,0.33084781836365224,0.3441463140858656,0.6556393678743125
|
| 5 |
+
ALL,4096,192,0.6935868271710071,0.7032230168798058,0.5519797982991866,0.5318145046183101,0.34847248127428415,0.6062565500482736,0.5793404529052973,0.22923292167747147,0.7023595499794807,0.7139850169951533,0.0008634669003249706,0.0005238745996855586,0.001229058529941919,0.00203548792576397,0.0015186955544880705,0.009589979089908366,0.009243304560859296,0.0005167923712758993,0.00034667452904907126,0.011625467015672337,0.07427373878063802,0.7137990767560151,0.7138471222579433,0.7059254373130734,0.7056991595823129,0.19543563872713965,0.19527462898701206,0.6946993832338948
|
revision_r1/controlled/specification.json
ADDED
|
@@ -0,0 +1,108 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"created_utc": "2026-11-09T14:38:24.151548+00:00",
|
| 3 |
+
"status": "Post-review exploratory analysis of already opened worlds; not confirmation",
|
| 4 |
+
"scientific_question": "Why does joint minus action-value utility shrink with sample size?",
|
| 5 |
+
"authorization": "User authorized additional reviewer-response experiments; no new approval requested.",
|
| 6 |
+
"source_release": "/home/nurgaly/experiment/unionmind/articles/make_1/release/huggingface-revised",
|
| 7 |
+
"code_sha256": "4a300dfc769473f5afb08e15e877f6c53dfff5f6d3a0abfe17e0043798eecbab",
|
| 8 |
+
"device": "cuda:4",
|
| 9 |
+
"input_sha256": {
|
| 10 |
+
"data/controlled/worlds.safetensors": "e8649aeaed970fa78c3f402bbe9e775c486f3cf94ce2494cfe98a8099dbf3c7c",
|
| 11 |
+
"data/controlled/worlds.json": "860e154c4ffd0690080ab8c263756d4340cc119fffa66e1cc41d1b42ac3848cb",
|
| 12 |
+
"data/controlled/manifest.json": "2b50155e7f4d198449e2499dad21ba82d51e94423a1711eca8de4a01ab9ca156",
|
| 13 |
+
"postconfirmation.py": "3365341c056cee9b153cc864bad864965c918e11e0cec1819d49c8875c68dcbf",
|
| 14 |
+
"unionmind/decision_sufficiency/core.py": "fa2f7405a8d0dc6ec96ebae1bb2a2b810d03ef1fd2877f5ebef7df1722992cee",
|
| 15 |
+
"results/postconfirmation/controlled_worlds.csv.gz": "bdb15095eadf96dff2aa316619a838b74024eece87c57143bcd44b4710e9547f",
|
| 16 |
+
"results/postconfirmation/controlled_frontiers.csv.gz": "946ebf8764ee3dfed5a741fa3a45e0cb5f8476960453716d9abebed0aa617e57",
|
| 17 |
+
"results/postconfirmation/controlled_timing.json": "082e8b981d9e6b13a07bfd3136d6afe08681bb8c702ba2c0c2b9a2d4966b6eea",
|
| 18 |
+
"provenance/controlled/confirmation_benchmark.json": "133420e45f42ff5dbacc8d01700aeba56bf65c02f5bc0b30bb7aff45a88bc249"
|
| 19 |
+
},
|
| 20 |
+
"units": "All 192 archived worlds, all six families, all three original nested sample streams",
|
| 21 |
+
"grid": {
|
| 22 |
+
"availability": [
|
| 23 |
+
6,
|
| 24 |
+
3
|
| 25 |
+
],
|
| 26 |
+
"records": [
|
| 27 |
+
64,
|
| 28 |
+
256,
|
| 29 |
+
1024,
|
| 30 |
+
4096
|
| 31 |
+
],
|
| 32 |
+
"estimators": [
|
| 33 |
+
"joint_jeffreys",
|
| 34 |
+
"naive_bayes",
|
| 35 |
+
"support_mixture"
|
| 36 |
+
],
|
| 37 |
+
"methods": [
|
| 38 |
+
"identity",
|
| 39 |
+
"query_joint",
|
| 40 |
+
"posterior_clusters",
|
| 41 |
+
"posterior_bins",
|
| 42 |
+
"future_marginal",
|
| 43 |
+
"raw_information",
|
| 44 |
+
"raw_entropy",
|
| 45 |
+
"hash_code",
|
| 46 |
+
"action_values"
|
| 47 |
+
],
|
| 48 |
+
"budgets": [
|
| 49 |
+
0,
|
| 50 |
+
1,
|
| 51 |
+
2
|
| 52 |
+
],
|
| 53 |
+
"costs": [
|
| 54 |
+
0.0,
|
| 55 |
+
0.01,
|
| 56 |
+
0.05
|
| 57 |
+
]
|
| 58 |
+
},
|
| 59 |
+
"primary_aggregation": "Average positive budgets, all costs and three streams within each world; equal worlds and six families",
|
| 60 |
+
"interventions": [
|
| 61 |
+
"Native: estimated-law encoder, estimated encoded-law exact planner, true-law evaluation",
|
| 62 |
+
"Replan the identical native code under the pushed true law to separate encoding and model-estimation loss",
|
| 63 |
+
"Privileged diagnostic: construct each unchanged encoder under the true law, then plan under either true or fitted pushed law; never a deployable competitor"
|
| 64 |
+
],
|
| 65 |
+
"exact_decomposition": "J-A = (encoding_loss_A-encoding_loss_J) + (estimation_loss_A-estimation_loss_J)",
|
| 66 |
+
"encoding_loss": "V(available true raw law)-V(true law pushed through native code)",
|
| 67 |
+
"estimation_loss": "V(true law pushed through native code)-realized utility of fitted policy",
|
| 68 |
+
"alias_diagnostic": "Evaluate all unordered raw-state pairs with true current-posterior difference <=1e-12 (exact) and <=0.01 (fixed near-alias threshold), weighted by 2 P(e)P(e'); compare true optimal root choices at each cost and budget and whether each native code merges disagreeing aliases",
|
| 69 |
+
"intervals": "Descriptive paired 95% percentile bootstrap, 10,000 draws, seed 18701, worlds resampled within fixed families after averaging streams; intervals are not corrected and are not new confirmatory tests",
|
| 70 |
+
"practical_scale": "Joint-action gain divided by remaining true available-oracle headroom of action-values; report signed gain, not just a percentage",
|
| 71 |
+
"timing": "Retain published complete-cost records. Fresh warm alternating-order 11-repeat 192-world primary-batch timings include count/estimate, profile, clustering/other encoding, push, exact planning, materialization and 1/32/1024/65536 recorded-event replays. One warmup charged and separately reported. Timing is descriptive on shared hardware.",
|
| 72 |
+
"prohibition": "No tuning, no model refits, no changed codes/iterations/costs, no dropped units, no original results or manuscript edits, no promotion",
|
| 73 |
+
"planned_outputs": [
|
| 74 |
+
"scores.npz",
|
| 75 |
+
"indices.json",
|
| 76 |
+
"frontier_all_conditions.csv",
|
| 77 |
+
"paired_by_condition.csv",
|
| 78 |
+
"sample_frontier.csv",
|
| 79 |
+
"family_sample_frontier.csv",
|
| 80 |
+
"native_worlds.csv.gz",
|
| 81 |
+
"alias_pairs.csv.gz",
|
| 82 |
+
"alias_summary.csv",
|
| 83 |
+
"privileged_geometry.csv",
|
| 84 |
+
"timing.json",
|
| 85 |
+
"summary.json",
|
| 86 |
+
"compute.json"
|
| 87 |
+
],
|
| 88 |
+
"amendment_01": {
|
| 89 |
+
"created_utc": "2026-11-09T14:43:15.749761+00:00",
|
| 90 |
+
"reason": "The expanded execution trace completed and saved all grid cells, then strict whole-grid native replay validation failed. 946/124416 archived world means differ; primary-mixture joint/action-value/identity at all sample sizes match within 3.34e-16. Preserve all discrepancies and original code/spec/log; aggregate fresh diagnostics without altering source results. Run the released original full grid as a separate trace comparison to investigate tie/numerical sensitivity.",
|
| 91 |
+
"previous_code_sha256": "db7360ee28143a5b9f1e16f373d887354f4824826d8d9b5ed64425e6b732a5dc",
|
| 92 |
+
"completed_grid_code": "collection_code_01.py",
|
| 93 |
+
"completed_grid_specification": "specification_01.json",
|
| 94 |
+
"analysis_changes": "Validation/reporting repair only; no new encoders, thresholds, unit exclusions, tuning, or changed data. Add --resume-aggregation and --reference-only. All native values/oracles in each decomposition come from the same execution trace."
|
| 95 |
+
},
|
| 96 |
+
"amendment_02": {
|
| 97 |
+
"created_utc": "2026-11-09T14:47:08.492706+00:00",
|
| 98 |
+
"reason": "The unmodified released full-grid controlled replay also differs from archive in 968/124416 native world means while exactly reproducing the headline frontier. Test finite-precision argmax sensitivity directly.",
|
| 99 |
+
"operation": "For all grid cells and fixed freshly constructed encoders, compare normal pushforward to accumulation over reversed raw-state order. This is mathematically the same law; measure maximum cell-law perturbation, fitted-optimal expected-value difference, and true-law realized-utility difference.",
|
| 100 |
+
"no_scientific_comparator_change": true,
|
| 101 |
+
"new_outputs": [
|
| 102 |
+
"numeric_push_sensitivity.npz",
|
| 103 |
+
"numeric_push_sensitivity.csv.gz",
|
| 104 |
+
"numeric_diagnostic.json",
|
| 105 |
+
"numeric_compute.json"
|
| 106 |
+
]
|
| 107 |
+
}
|
| 108 |
+
}
|
revision_r1/controlled/specification_01.json
ADDED
|
@@ -0,0 +1,88 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"created_utc": "2026-11-09T14:38:24.151548+00:00",
|
| 3 |
+
"status": "Post-review exploratory analysis of already opened worlds; not confirmation",
|
| 4 |
+
"scientific_question": "Why does joint minus action-value utility shrink with sample size?",
|
| 5 |
+
"authorization": "User authorized additional reviewer-response experiments; no new approval requested.",
|
| 6 |
+
"source_release": "/home/nurgaly/experiment/unionmind/articles/make_1/release/huggingface-revised",
|
| 7 |
+
"code_sha256": "db7360ee28143a5b9f1e16f373d887354f4824826d8d9b5ed64425e6b732a5dc",
|
| 8 |
+
"device": "cuda:4",
|
| 9 |
+
"input_sha256": {
|
| 10 |
+
"data/controlled/worlds.safetensors": "e8649aeaed970fa78c3f402bbe9e775c486f3cf94ce2494cfe98a8099dbf3c7c",
|
| 11 |
+
"data/controlled/worlds.json": "860e154c4ffd0690080ab8c263756d4340cc119fffa66e1cc41d1b42ac3848cb",
|
| 12 |
+
"data/controlled/manifest.json": "2b50155e7f4d198449e2499dad21ba82d51e94423a1711eca8de4a01ab9ca156",
|
| 13 |
+
"postconfirmation.py": "3365341c056cee9b153cc864bad864965c918e11e0cec1819d49c8875c68dcbf",
|
| 14 |
+
"unionmind/decision_sufficiency/core.py": "fa2f7405a8d0dc6ec96ebae1bb2a2b810d03ef1fd2877f5ebef7df1722992cee",
|
| 15 |
+
"results/postconfirmation/controlled_worlds.csv.gz": "bdb15095eadf96dff2aa316619a838b74024eece87c57143bcd44b4710e9547f",
|
| 16 |
+
"results/postconfirmation/controlled_frontiers.csv.gz": "946ebf8764ee3dfed5a741fa3a45e0cb5f8476960453716d9abebed0aa617e57",
|
| 17 |
+
"results/postconfirmation/controlled_timing.json": "082e8b981d9e6b13a07bfd3136d6afe08681bb8c702ba2c0c2b9a2d4966b6eea",
|
| 18 |
+
"provenance/controlled/confirmation_benchmark.json": "133420e45f42ff5dbacc8d01700aeba56bf65c02f5bc0b30bb7aff45a88bc249"
|
| 19 |
+
},
|
| 20 |
+
"units": "All 192 archived worlds, all six families, all three original nested sample streams",
|
| 21 |
+
"grid": {
|
| 22 |
+
"availability": [
|
| 23 |
+
6,
|
| 24 |
+
3
|
| 25 |
+
],
|
| 26 |
+
"records": [
|
| 27 |
+
64,
|
| 28 |
+
256,
|
| 29 |
+
1024,
|
| 30 |
+
4096
|
| 31 |
+
],
|
| 32 |
+
"estimators": [
|
| 33 |
+
"joint_jeffreys",
|
| 34 |
+
"naive_bayes",
|
| 35 |
+
"support_mixture"
|
| 36 |
+
],
|
| 37 |
+
"methods": [
|
| 38 |
+
"identity",
|
| 39 |
+
"query_joint",
|
| 40 |
+
"posterior_clusters",
|
| 41 |
+
"posterior_bins",
|
| 42 |
+
"future_marginal",
|
| 43 |
+
"raw_information",
|
| 44 |
+
"raw_entropy",
|
| 45 |
+
"hash_code",
|
| 46 |
+
"action_values"
|
| 47 |
+
],
|
| 48 |
+
"budgets": [
|
| 49 |
+
0,
|
| 50 |
+
1,
|
| 51 |
+
2
|
| 52 |
+
],
|
| 53 |
+
"costs": [
|
| 54 |
+
0.0,
|
| 55 |
+
0.01,
|
| 56 |
+
0.05
|
| 57 |
+
]
|
| 58 |
+
},
|
| 59 |
+
"primary_aggregation": "Average positive budgets, all costs and three streams within each world; equal worlds and six families",
|
| 60 |
+
"interventions": [
|
| 61 |
+
"Native: estimated-law encoder, estimated encoded-law exact planner, true-law evaluation",
|
| 62 |
+
"Replan the identical native code under the pushed true law to separate encoding and model-estimation loss",
|
| 63 |
+
"Privileged diagnostic: construct each unchanged encoder under the true law, then plan under either true or fitted pushed law; never a deployable competitor"
|
| 64 |
+
],
|
| 65 |
+
"exact_decomposition": "J-A = (encoding_loss_A-encoding_loss_J) + (estimation_loss_A-estimation_loss_J)",
|
| 66 |
+
"encoding_loss": "V(available true raw law)-V(true law pushed through native code)",
|
| 67 |
+
"estimation_loss": "V(true law pushed through native code)-realized utility of fitted policy",
|
| 68 |
+
"alias_diagnostic": "Evaluate all unordered raw-state pairs with true current-posterior difference <=1e-12 (exact) and <=0.01 (fixed near-alias threshold), weighted by 2 P(e)P(e'); compare true optimal root choices at each cost and budget and whether each native code merges disagreeing aliases",
|
| 69 |
+
"intervals": "Descriptive paired 95% percentile bootstrap, 10,000 draws, seed 18701, worlds resampled within fixed families after averaging streams; intervals are not corrected and are not new confirmatory tests",
|
| 70 |
+
"practical_scale": "Joint-action gain divided by remaining true available-oracle headroom of action-values; report signed gain, not just a percentage",
|
| 71 |
+
"timing": "Retain published complete-cost records. Fresh warm alternating-order 11-repeat 192-world primary-batch timings include count/estimate, profile, clustering/other encoding, push, exact planning, materialization and 1/32/1024/65536 recorded-event replays. One warmup charged and separately reported. Timing is descriptive on shared hardware.",
|
| 72 |
+
"prohibition": "No tuning, no model refits, no changed codes/iterations/costs, no dropped units, no original results or manuscript edits, no promotion",
|
| 73 |
+
"planned_outputs": [
|
| 74 |
+
"scores.npz",
|
| 75 |
+
"indices.json",
|
| 76 |
+
"frontier_all_conditions.csv",
|
| 77 |
+
"paired_by_condition.csv",
|
| 78 |
+
"sample_frontier.csv",
|
| 79 |
+
"family_sample_frontier.csv",
|
| 80 |
+
"native_worlds.csv.gz",
|
| 81 |
+
"alias_pairs.csv.gz",
|
| 82 |
+
"alias_summary.csv",
|
| 83 |
+
"privileged_geometry.csv",
|
| 84 |
+
"timing.json",
|
| 85 |
+
"summary.json",
|
| 86 |
+
"compute.json"
|
| 87 |
+
]
|
| 88 |
+
}
|
revision_r1/controlled/specification_02.json
ADDED
|
@@ -0,0 +1,96 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"created_utc": "2026-11-09T14:38:24.151548+00:00",
|
| 3 |
+
"status": "Post-review exploratory analysis of already opened worlds; not confirmation",
|
| 4 |
+
"scientific_question": "Why does joint minus action-value utility shrink with sample size?",
|
| 5 |
+
"authorization": "User authorized additional reviewer-response experiments; no new approval requested.",
|
| 6 |
+
"source_release": "/home/nurgaly/experiment/unionmind/articles/make_1/release/huggingface-revised",
|
| 7 |
+
"code_sha256": "aa82ccdc6a1bb3c7d1e92b9ab0c4c88478c7d26e0442fe91523036a63260b2aa",
|
| 8 |
+
"device": "cuda:4",
|
| 9 |
+
"input_sha256": {
|
| 10 |
+
"data/controlled/worlds.safetensors": "e8649aeaed970fa78c3f402bbe9e775c486f3cf94ce2494cfe98a8099dbf3c7c",
|
| 11 |
+
"data/controlled/worlds.json": "860e154c4ffd0690080ab8c263756d4340cc119fffa66e1cc41d1b42ac3848cb",
|
| 12 |
+
"data/controlled/manifest.json": "2b50155e7f4d198449e2499dad21ba82d51e94423a1711eca8de4a01ab9ca156",
|
| 13 |
+
"postconfirmation.py": "3365341c056cee9b153cc864bad864965c918e11e0cec1819d49c8875c68dcbf",
|
| 14 |
+
"unionmind/decision_sufficiency/core.py": "fa2f7405a8d0dc6ec96ebae1bb2a2b810d03ef1fd2877f5ebef7df1722992cee",
|
| 15 |
+
"results/postconfirmation/controlled_worlds.csv.gz": "bdb15095eadf96dff2aa316619a838b74024eece87c57143bcd44b4710e9547f",
|
| 16 |
+
"results/postconfirmation/controlled_frontiers.csv.gz": "946ebf8764ee3dfed5a741fa3a45e0cb5f8476960453716d9abebed0aa617e57",
|
| 17 |
+
"results/postconfirmation/controlled_timing.json": "082e8b981d9e6b13a07bfd3136d6afe08681bb8c702ba2c0c2b9a2d4966b6eea",
|
| 18 |
+
"provenance/controlled/confirmation_benchmark.json": "133420e45f42ff5dbacc8d01700aeba56bf65c02f5bc0b30bb7aff45a88bc249"
|
| 19 |
+
},
|
| 20 |
+
"units": "All 192 archived worlds, all six families, all three original nested sample streams",
|
| 21 |
+
"grid": {
|
| 22 |
+
"availability": [
|
| 23 |
+
6,
|
| 24 |
+
3
|
| 25 |
+
],
|
| 26 |
+
"records": [
|
| 27 |
+
64,
|
| 28 |
+
256,
|
| 29 |
+
1024,
|
| 30 |
+
4096
|
| 31 |
+
],
|
| 32 |
+
"estimators": [
|
| 33 |
+
"joint_jeffreys",
|
| 34 |
+
"naive_bayes",
|
| 35 |
+
"support_mixture"
|
| 36 |
+
],
|
| 37 |
+
"methods": [
|
| 38 |
+
"identity",
|
| 39 |
+
"query_joint",
|
| 40 |
+
"posterior_clusters",
|
| 41 |
+
"posterior_bins",
|
| 42 |
+
"future_marginal",
|
| 43 |
+
"raw_information",
|
| 44 |
+
"raw_entropy",
|
| 45 |
+
"hash_code",
|
| 46 |
+
"action_values"
|
| 47 |
+
],
|
| 48 |
+
"budgets": [
|
| 49 |
+
0,
|
| 50 |
+
1,
|
| 51 |
+
2
|
| 52 |
+
],
|
| 53 |
+
"costs": [
|
| 54 |
+
0.0,
|
| 55 |
+
0.01,
|
| 56 |
+
0.05
|
| 57 |
+
]
|
| 58 |
+
},
|
| 59 |
+
"primary_aggregation": "Average positive budgets, all costs and three streams within each world; equal worlds and six families",
|
| 60 |
+
"interventions": [
|
| 61 |
+
"Native: estimated-law encoder, estimated encoded-law exact planner, true-law evaluation",
|
| 62 |
+
"Replan the identical native code under the pushed true law to separate encoding and model-estimation loss",
|
| 63 |
+
"Privileged diagnostic: construct each unchanged encoder under the true law, then plan under either true or fitted pushed law; never a deployable competitor"
|
| 64 |
+
],
|
| 65 |
+
"exact_decomposition": "J-A = (encoding_loss_A-encoding_loss_J) + (estimation_loss_A-estimation_loss_J)",
|
| 66 |
+
"encoding_loss": "V(available true raw law)-V(true law pushed through native code)",
|
| 67 |
+
"estimation_loss": "V(true law pushed through native code)-realized utility of fitted policy",
|
| 68 |
+
"alias_diagnostic": "Evaluate all unordered raw-state pairs with true current-posterior difference <=1e-12 (exact) and <=0.01 (fixed near-alias threshold), weighted by 2 P(e)P(e'); compare true optimal root choices at each cost and budget and whether each native code merges disagreeing aliases",
|
| 69 |
+
"intervals": "Descriptive paired 95% percentile bootstrap, 10,000 draws, seed 18701, worlds resampled within fixed families after averaging streams; intervals are not corrected and are not new confirmatory tests",
|
| 70 |
+
"practical_scale": "Joint-action gain divided by remaining true available-oracle headroom of action-values; report signed gain, not just a percentage",
|
| 71 |
+
"timing": "Retain published complete-cost records. Fresh warm alternating-order 11-repeat 192-world primary-batch timings include count/estimate, profile, clustering/other encoding, push, exact planning, materialization and 1/32/1024/65536 recorded-event replays. One warmup charged and separately reported. Timing is descriptive on shared hardware.",
|
| 72 |
+
"prohibition": "No tuning, no model refits, no changed codes/iterations/costs, no dropped units, no original results or manuscript edits, no promotion",
|
| 73 |
+
"planned_outputs": [
|
| 74 |
+
"scores.npz",
|
| 75 |
+
"indices.json",
|
| 76 |
+
"frontier_all_conditions.csv",
|
| 77 |
+
"paired_by_condition.csv",
|
| 78 |
+
"sample_frontier.csv",
|
| 79 |
+
"family_sample_frontier.csv",
|
| 80 |
+
"native_worlds.csv.gz",
|
| 81 |
+
"alias_pairs.csv.gz",
|
| 82 |
+
"alias_summary.csv",
|
| 83 |
+
"privileged_geometry.csv",
|
| 84 |
+
"timing.json",
|
| 85 |
+
"summary.json",
|
| 86 |
+
"compute.json"
|
| 87 |
+
],
|
| 88 |
+
"amendment_01": {
|
| 89 |
+
"created_utc": "2026-11-09T14:43:15.749761+00:00",
|
| 90 |
+
"reason": "The expanded execution trace completed and saved all grid cells, then strict whole-grid native replay validation failed. 946/124416 archived world means differ; primary-mixture joint/action-value/identity at all sample sizes match within 3.34e-16. Preserve all discrepancies and original code/spec/log; aggregate fresh diagnostics without altering source results. Run the released original full grid as a separate trace comparison to investigate tie/numerical sensitivity.",
|
| 91 |
+
"previous_code_sha256": "db7360ee28143a5b9f1e16f373d887354f4824826d8d9b5ed64425e6b732a5dc",
|
| 92 |
+
"completed_grid_code": "collection_code_01.py",
|
| 93 |
+
"completed_grid_specification": "specification_01.json",
|
| 94 |
+
"analysis_changes": "Validation/reporting repair only; no new encoders, thresholds, unit exclusions, tuning, or changed data. Add --resume-aggregation and --reference-only. All native values/oracles in each decomposition come from the same execution trace."
|
| 95 |
+
}
|
| 96 |
+
}
|
revision_r1/controlled/summary.json
ADDED
|
@@ -0,0 +1,949 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"status": "POST_REVIEW_EXPLORATORY_OPENED_DATA_NO_NEW_CONFIRMATION",
|
| 3 |
+
"all_worlds": 192,
|
| 4 |
+
"native_saved_world_scores_checked": 124416,
|
| 5 |
+
"native_saved_world_scores_within_1e_10": 123470,
|
| 6 |
+
"replay_discrepancy_count": 946,
|
| 7 |
+
"maximum_absolute_replay_error": 0.1456998408107582,
|
| 8 |
+
"exact_primary_mixture_joint_action_identity_frontier_maximum_replay_error": 3.3306690738754696e-16,
|
| 9 |
+
"exact_decomposition_max_error": 1.6653345369377348e-16,
|
| 10 |
+
"primary_sample_frontier": [
|
| 11 |
+
{
|
| 12 |
+
"family": "ALL",
|
| 13 |
+
"records": 64,
|
| 14 |
+
"worlds": 192,
|
| 15 |
+
"identity": 0.2861019748560038,
|
| 16 |
+
"query_joint": 0.26584992657807344,
|
| 17 |
+
"posterior_clusters": 0.2590280944592631,
|
| 18 |
+
"posterior_bins": 0.26074362507619164,
|
| 19 |
+
"future_marginal": 0.2639323021727651,
|
| 20 |
+
"raw_information": 0.2722343061169504,
|
| 21 |
+
"raw_entropy": 0.21481644347086729,
|
| 22 |
+
"hash_code": 0.10239193116286986,
|
| 23 |
+
"action_values": 0.26390613849214967,
|
| 24 |
+
"true_oracle": 0.7139850169951533,
|
| 25 |
+
"joint_minus_action": 0.001943788085923736,
|
| 26 |
+
"paired_95_low": 0.00021308171557470748,
|
| 27 |
+
"paired_95_high": 0.003647181976886664,
|
| 28 |
+
"action_encoding_loss": 0.23247242290469453,
|
| 29 |
+
"joint_encoding_loss": 0.22743575439464836,
|
| 30 |
+
"action_estimation_loss": 0.21760645559830893,
|
| 31 |
+
"joint_estimation_loss": 0.22069933602243133,
|
| 32 |
+
"encoding_contribution": 0.005036668510046183,
|
| 33 |
+
"estimation_contribution": -0.0030928804241224478,
|
| 34 |
+
"action_remaining_headroom": 0.4500788785030035,
|
| 35 |
+
"fraction_action_headroom_closed": 0.004318772061441592,
|
| 36 |
+
"true_code_joint_oracle": 0.7137990767560151,
|
| 37 |
+
"true_code_action_oracle": 0.7138471222579433,
|
| 38 |
+
"true_code_joint_fitted": 0.33015852361105136,
|
| 39 |
+
"true_code_action_fitted": 0.32977290874844095,
|
| 40 |
+
"joint_root_choice_mismatch": 0.5728323618858121,
|
| 41 |
+
"action_root_choice_mismatch": 0.5747586230569347,
|
| 42 |
+
"strongest_uncompressed_mean": 0.33512680859485605
|
| 43 |
+
},
|
| 44 |
+
{
|
| 45 |
+
"family": "ALL",
|
| 46 |
+
"records": 256,
|
| 47 |
+
"worlds": 192,
|
| 48 |
+
"identity": 0.47464585889067984,
|
| 49 |
+
"query_joint": 0.46153544711762456,
|
| 50 |
+
"posterior_clusters": 0.3821578365055907,
|
| 51 |
+
"posterior_bins": 0.37086161330575007,
|
| 52 |
+
"future_marginal": 0.3117767655744309,
|
| 53 |
+
"raw_information": 0.39616931811795314,
|
| 54 |
+
"raw_entropy": 0.3287390155602434,
|
| 55 |
+
"hash_code": 0.15862081020012567,
|
| 56 |
+
"action_values": 0.4421891239875439,
|
| 57 |
+
"true_oracle": 0.7139850169951533,
|
| 58 |
+
"joint_minus_action": 0.019346323130080677,
|
| 59 |
+
"paired_95_low": 0.016004547809026087,
|
| 60 |
+
"paired_95_high": 0.02272642638711617,
|
| 61 |
+
"action_encoding_loss": 0.11063576081059036,
|
| 62 |
+
"joint_encoding_loss": 0.07695151696279438,
|
| 63 |
+
"action_estimation_loss": 0.16116013219701888,
|
| 64 |
+
"joint_estimation_loss": 0.1754980529147342,
|
| 65 |
+
"encoding_contribution": 0.033684243847795996,
|
| 66 |
+
"estimation_contribution": -0.014337920717715315,
|
| 67 |
+
"action_remaining_headroom": 0.27179589300760926,
|
| 68 |
+
"fraction_action_headroom_closed": 0.07117960067755348,
|
| 69 |
+
"true_code_joint_oracle": 0.7137990767560151,
|
| 70 |
+
"true_code_action_oracle": 0.7138471222579433,
|
| 71 |
+
"true_code_joint_fitted": 0.5040257394630626,
|
| 72 |
+
"true_code_action_fitted": 0.5033361534654197,
|
| 73 |
+
"joint_root_choice_mismatch": 0.5138373564572506,
|
| 74 |
+
"action_root_choice_mismatch": 0.532667401596573,
|
| 75 |
+
"strongest_uncompressed_mean": 0.5503906781894238
|
| 76 |
+
},
|
| 77 |
+
{
|
| 78 |
+
"family": "ALL",
|
| 79 |
+
"records": 1024,
|
| 80 |
+
"worlds": 192,
|
| 81 |
+
"identity": 0.6421842631007378,
|
| 82 |
+
"query_joint": 0.651568196746697,
|
| 83 |
+
"posterior_clusters": 0.5045968970651954,
|
| 84 |
+
"posterior_bins": 0.4901829803145068,
|
| 85 |
+
"future_marginal": 0.3289223833583756,
|
| 86 |
+
"raw_information": 0.5502229409756352,
|
| 87 |
+
"raw_entropy": 0.5192618455652109,
|
| 88 |
+
"hash_code": 0.20875256335562897,
|
| 89 |
+
"action_values": 0.6375946140433673,
|
| 90 |
+
"true_oracle": 0.7139850169951533,
|
| 91 |
+
"joint_minus_action": 0.013973582703329749,
|
| 92 |
+
"paired_95_low": 0.012156191041507863,
|
| 93 |
+
"paired_95_high": 0.01579707133440269,
|
| 94 |
+
"action_encoding_loss": 0.020288801609760987,
|
| 95 |
+
"joint_encoding_loss": 0.010064365748209498,
|
| 96 |
+
"action_estimation_loss": 0.05610160134202489,
|
| 97 |
+
"joint_estimation_loss": 0.052352454500246635,
|
| 98 |
+
"encoding_contribution": 0.010224435861551489,
|
| 99 |
+
"estimation_contribution": 0.00374914684177826,
|
| 100 |
+
"action_remaining_headroom": 0.07639040295178588,
|
| 101 |
+
"fraction_action_headroom_closed": 0.18292327522017698,
|
| 102 |
+
"true_code_joint_oracle": 0.7137990767560151,
|
| 103 |
+
"true_code_action_oracle": 0.7138471222579433,
|
| 104 |
+
"true_code_joint_fitted": 0.6654659197644641,
|
| 105 |
+
"true_code_action_fitted": 0.6648982429373541,
|
| 106 |
+
"joint_root_choice_mismatch": 0.33084781836365224,
|
| 107 |
+
"action_root_choice_mismatch": 0.3441463140858656,
|
| 108 |
+
"strongest_uncompressed_mean": 0.6556393678743125
|
| 109 |
+
},
|
| 110 |
+
{
|
| 111 |
+
"family": "ALL",
|
| 112 |
+
"records": 4096,
|
| 113 |
+
"worlds": 192,
|
| 114 |
+
"identity": 0.6935868271710071,
|
| 115 |
+
"query_joint": 0.7032230168798058,
|
| 116 |
+
"posterior_clusters": 0.5519797982991866,
|
| 117 |
+
"posterior_bins": 0.5318145046183101,
|
| 118 |
+
"future_marginal": 0.34847248127428415,
|
| 119 |
+
"raw_information": 0.6062565500482736,
|
| 120 |
+
"raw_entropy": 0.5793404529052973,
|
| 121 |
+
"hash_code": 0.22923292167747147,
|
| 122 |
+
"action_values": 0.7023595499794807,
|
| 123 |
+
"true_oracle": 0.7139850169951533,
|
| 124 |
+
"joint_minus_action": 0.0008634669003249706,
|
| 125 |
+
"paired_95_low": 0.0005238745996855586,
|
| 126 |
+
"paired_95_high": 0.001229058529941919,
|
| 127 |
+
"action_encoding_loss": 0.00203548792576397,
|
| 128 |
+
"joint_encoding_loss": 0.0015186955544880705,
|
| 129 |
+
"action_estimation_loss": 0.009589979089908366,
|
| 130 |
+
"joint_estimation_loss": 0.009243304560859296,
|
| 131 |
+
"encoding_contribution": 0.0005167923712758993,
|
| 132 |
+
"estimation_contribution": 0.00034667452904907126,
|
| 133 |
+
"action_remaining_headroom": 0.011625467015672337,
|
| 134 |
+
"fraction_action_headroom_closed": 0.07427373878063802,
|
| 135 |
+
"true_code_joint_oracle": 0.7137990767560151,
|
| 136 |
+
"true_code_action_oracle": 0.7138471222579433,
|
| 137 |
+
"true_code_joint_fitted": 0.7059254373130734,
|
| 138 |
+
"true_code_action_fitted": 0.7056991595823129,
|
| 139 |
+
"joint_root_choice_mismatch": 0.19543563872713965,
|
| 140 |
+
"action_root_choice_mismatch": 0.19527462898701206,
|
| 141 |
+
"strongest_uncompressed_mean": 0.6946993832338948
|
| 142 |
+
}
|
| 143 |
+
],
|
| 144 |
+
"family_sample_frontier": [
|
| 145 |
+
{
|
| 146 |
+
"family": "context",
|
| 147 |
+
"records": 64,
|
| 148 |
+
"worlds": 32,
|
| 149 |
+
"identity": 0.09339283465187889,
|
| 150 |
+
"query_joint": 0.0691595408487013,
|
| 151 |
+
"posterior_clusters": 0.05884192092177272,
|
| 152 |
+
"posterior_bins": 0.06287654697819856,
|
| 153 |
+
"future_marginal": 0.07187382385734353,
|
| 154 |
+
"raw_information": 0.07728978551777194,
|
| 155 |
+
"raw_entropy": 0.09455913414529662,
|
| 156 |
+
"hash_code": 0.10673495166633204,
|
| 157 |
+
"action_values": 0.06409177954684019,
|
| 158 |
+
"true_oracle": 0.7880888584118835,
|
| 159 |
+
"joint_minus_action": 0.005067761301861108,
|
| 160 |
+
"paired_95_low": 0.0015311813058195526,
|
| 161 |
+
"paired_95_high": 0.008840865316316502,
|
| 162 |
+
"action_encoding_loss": 0.3813222379944849,
|
| 163 |
+
"joint_encoding_loss": 0.3669534072130477,
|
| 164 |
+
"action_estimation_loss": 0.34267484087055833,
|
| 165 |
+
"joint_estimation_loss": 0.3519759103501344,
|
| 166 |
+
"encoding_contribution": 0.014368830781437176,
|
| 167 |
+
"estimation_contribution": -0.00930106947957607,
|
| 168 |
+
"action_remaining_headroom": 0.7239970788650432,
|
| 169 |
+
"fraction_action_headroom_closed": 0.00699969854823926,
|
| 170 |
+
"true_code_joint_oracle": 0.7880888584118835,
|
| 171 |
+
"true_code_action_oracle": 0.7880888584118835,
|
| 172 |
+
"true_code_joint_fitted": 0.12334932494026522,
|
| 173 |
+
"true_code_action_fitted": 0.12491782030181064,
|
| 174 |
+
"joint_root_choice_mismatch": 0.6849965145249336,
|
| 175 |
+
"action_root_choice_mismatch": 0.6875732689753824,
|
| 176 |
+
"strongest_uncompressed_mean": 0.2821009589784481
|
| 177 |
+
},
|
| 178 |
+
{
|
| 179 |
+
"family": "context",
|
| 180 |
+
"records": 256,
|
| 181 |
+
"worlds": 32,
|
| 182 |
+
"identity": 0.34655353787902665,
|
| 183 |
+
"query_joint": 0.3367188419663062,
|
| 184 |
+
"posterior_clusters": 0.16416722984358104,
|
| 185 |
+
"posterior_bins": 0.1463272094176618,
|
| 186 |
+
"future_marginal": 0.1871258673613156,
|
| 187 |
+
"raw_information": 0.26397328263687364,
|
| 188 |
+
"raw_entropy": 0.25243683714316034,
|
| 189 |
+
"hash_code": 0.15529673741167777,
|
| 190 |
+
"action_values": 0.31618507415861496,
|
| 191 |
+
"true_oracle": 0.7880888584118835,
|
| 192 |
+
"joint_minus_action": 0.0205337678076913,
|
| 193 |
+
"paired_95_low": 0.01188513381910629,
|
| 194 |
+
"paired_95_high": 0.03021118308609348,
|
| 195 |
+
"action_encoding_loss": 0.11119897518661376,
|
| 196 |
+
"joint_encoding_loss": 0.07603887002395061,
|
| 197 |
+
"action_estimation_loss": 0.3607048090666547,
|
| 198 |
+
"joint_estimation_loss": 0.37533114642162657,
|
| 199 |
+
"encoding_contribution": 0.035160105162663144,
|
| 200 |
+
"estimation_contribution": -0.014626337354971848,
|
| 201 |
+
"action_remaining_headroom": 0.47190378425326845,
|
| 202 |
+
"fraction_action_headroom_closed": 0.043512615267927005,
|
| 203 |
+
"true_code_joint_oracle": 0.7880888584118835,
|
| 204 |
+
"true_code_action_oracle": 0.7880888584118835,
|
| 205 |
+
"true_code_joint_fitted": 0.3856346205277096,
|
| 206 |
+
"true_code_action_fitted": 0.38320113863439054,
|
| 207 |
+
"joint_root_choice_mismatch": 0.629779568015901,
|
| 208 |
+
"action_root_choice_mismatch": 0.6496391850613675,
|
| 209 |
+
"strongest_uncompressed_mean": 0.5904656904258859
|
| 210 |
+
},
|
| 211 |
+
{
|
| 212 |
+
"family": "context",
|
| 213 |
+
"records": 1024,
|
| 214 |
+
"worlds": 32,
|
| 215 |
+
"identity": 0.6650919319515614,
|
| 216 |
+
"query_joint": 0.6814637624980395,
|
| 217 |
+
"posterior_clusters": 0.23779038974868094,
|
| 218 |
+
"posterior_bins": 0.17124022355036714,
|
| 219 |
+
"future_marginal": 0.36268052770898085,
|
| 220 |
+
"raw_information": 0.5145775504161055,
|
| 221 |
+
"raw_entropy": 0.5794729711288229,
|
| 222 |
+
"hash_code": 0.2170459709045936,
|
| 223 |
+
"action_values": 0.667719627246173,
|
| 224 |
+
"true_oracle": 0.7880888584118835,
|
| 225 |
+
"joint_minus_action": 0.013744135251866517,
|
| 226 |
+
"paired_95_low": 0.009521684888968644,
|
| 227 |
+
"paired_95_high": 0.01821770096658289,
|
| 228 |
+
"action_encoding_loss": 0.01430655071271671,
|
| 229 |
+
"joint_encoding_loss": 0.006186382463298001,
|
| 230 |
+
"action_estimation_loss": 0.10606268045299369,
|
| 231 |
+
"joint_estimation_loss": 0.10043871345054588,
|
| 232 |
+
"encoding_contribution": 0.008120168249418708,
|
| 233 |
+
"estimation_contribution": 0.005623967002447808,
|
| 234 |
+
"action_remaining_headroom": 0.12036923116571041,
|
| 235 |
+
"fraction_action_headroom_closed": 0.11418312735540517,
|
| 236 |
+
"true_code_joint_oracle": 0.7880888584118835,
|
| 237 |
+
"true_code_action_oracle": 0.7880888584118835,
|
| 238 |
+
"true_code_joint_fitted": 0.6988418074434672,
|
| 239 |
+
"true_code_action_fitted": 0.6964377598607214,
|
| 240 |
+
"joint_root_choice_mismatch": 0.3473903728720782,
|
| 241 |
+
"action_root_choice_mismatch": 0.3575560574608555,
|
| 242 |
+
"strongest_uncompressed_mean": 0.7340548744395632
|
| 243 |
+
},
|
| 244 |
+
{
|
| 245 |
+
"family": "context",
|
| 246 |
+
"records": 4096,
|
| 247 |
+
"worlds": 32,
|
| 248 |
+
"identity": 0.766972716874273,
|
| 249 |
+
"query_joint": 0.7766349438098745,
|
| 250 |
+
"posterior_clusters": 0.26807855851736373,
|
| 251 |
+
"posterior_bins": 0.1733365486789426,
|
| 252 |
+
"future_marginal": 0.5632767718308346,
|
| 253 |
+
"raw_information": 0.5928278598851834,
|
| 254 |
+
"raw_entropy": 0.6885210614964501,
|
| 255 |
+
"hash_code": 0.24707180576088097,
|
| 256 |
+
"action_values": 0.7768669245376707,
|
| 257 |
+
"true_oracle": 0.7880888584118835,
|
| 258 |
+
"joint_minus_action": -0.00023198072779632983,
|
| 259 |
+
"paired_95_low": -0.001376293766611737,
|
| 260 |
+
"paired_95_high": 0.0008712872445568171,
|
| 261 |
+
"action_encoding_loss": 0.0006891208483935686,
|
| 262 |
+
"joint_encoding_loss": 0.00018064349661515988,
|
| 263 |
+
"action_estimation_loss": 0.010532813025819078,
|
| 264 |
+
"joint_estimation_loss": 0.011273271105393817,
|
| 265 |
+
"encoding_contribution": 0.0005084773517784087,
|
| 266 |
+
"estimation_contribution": -0.0007404580795747386,
|
| 267 |
+
"action_remaining_headroom": 0.011221933874212647,
|
| 268 |
+
"fraction_action_headroom_closed": -0.02067208115790168,
|
| 269 |
+
"true_code_joint_oracle": 0.7880888584118835,
|
| 270 |
+
"true_code_action_oracle": 0.7880888584118835,
|
| 271 |
+
"true_code_joint_fitted": 0.7795383990383913,
|
| 272 |
+
"true_code_action_fitted": 0.7790645144622885,
|
| 273 |
+
"joint_root_choice_mismatch": 0.1620783588267283,
|
| 274 |
+
"action_root_choice_mismatch": 0.1554586985921155,
|
| 275 |
+
"strongest_uncompressed_mean": 0.7708361142534652
|
| 276 |
+
},
|
| 277 |
+
{
|
| 278 |
+
"family": "independent",
|
| 279 |
+
"records": 64,
|
| 280 |
+
"worlds": 32,
|
| 281 |
+
"identity": 0.909510548381619,
|
| 282 |
+
"query_joint": 0.9071402048294446,
|
| 283 |
+
"posterior_clusters": 0.9072370207266524,
|
| 284 |
+
"posterior_bins": 0.9093082065269007,
|
| 285 |
+
"future_marginal": 0.9074681824272653,
|
| 286 |
+
"raw_information": 0.8896605460004141,
|
| 287 |
+
"raw_entropy": 0.688211917829685,
|
| 288 |
+
"hash_code": 0.44151472051268514,
|
| 289 |
+
"action_values": 0.9068080779301189,
|
| 290 |
+
"true_oracle": 0.928933733504975,
|
| 291 |
+
"joint_minus_action": 0.00033212689932572356,
|
| 292 |
+
"paired_95_low": -0.0004962724151037125,
|
| 293 |
+
"paired_95_high": 0.0012075628763129809,
|
| 294 |
+
"action_encoding_loss": 0.009646569741400077,
|
| 295 |
+
"joint_encoding_loss": 0.00982195667832023,
|
| 296 |
+
"action_estimation_loss": 0.012479085833456041,
|
| 297 |
+
"joint_estimation_loss": 0.011971571997210165,
|
| 298 |
+
"encoding_contribution": -0.00017538693692015217,
|
| 299 |
+
"estimation_contribution": 0.0005075138362458757,
|
| 300 |
+
"action_remaining_headroom": 0.02212565557485612,
|
| 301 |
+
"fraction_action_headroom_closed": 0.015010940498556656,
|
| 302 |
+
"true_code_joint_oracle": 0.927818092070147,
|
| 303 |
+
"true_code_action_oracle": 0.9281063650817161,
|
| 304 |
+
"true_code_joint_fitted": 0.9185297572467013,
|
| 305 |
+
"true_code_action_fitted": 0.9192315821057243,
|
| 306 |
+
"joint_root_choice_mismatch": 0.23523002477266935,
|
| 307 |
+
"action_root_choice_mismatch": 0.23312046998815447,
|
| 308 |
+
"strongest_uncompressed_mean": 0.9136191430312471
|
| 309 |
+
},
|
| 310 |
+
{
|
| 311 |
+
"family": "independent",
|
| 312 |
+
"records": 256,
|
| 313 |
+
"worlds": 32,
|
| 314 |
+
"identity": 0.9116024194164163,
|
| 315 |
+
"query_joint": 0.9098670177155274,
|
| 316 |
+
"posterior_clusters": 0.9100779075741663,
|
| 317 |
+
"posterior_bins": 0.9126143882919777,
|
| 318 |
+
"future_marginal": 0.905503332137763,
|
| 319 |
+
"raw_information": 0.896961421482459,
|
| 320 |
+
"raw_entropy": 0.6703717514265709,
|
| 321 |
+
"hash_code": 0.4619766730060043,
|
| 322 |
+
"action_values": 0.9090004625042527,
|
| 323 |
+
"true_oracle": 0.928933733504975,
|
| 324 |
+
"joint_minus_action": 0.0008665552112746971,
|
| 325 |
+
"paired_95_low": -0.0004301648384077622,
|
| 326 |
+
"paired_95_high": 0.0021276027149003647,
|
| 327 |
+
"action_encoding_loss": 0.006369755441728155,
|
| 328 |
+
"joint_encoding_loss": 0.0065092495744983395,
|
| 329 |
+
"action_estimation_loss": 0.01356351555899413,
|
| 330 |
+
"joint_estimation_loss": 0.01255746621494925,
|
| 331 |
+
"encoding_contribution": -0.00013949413277018416,
|
| 332 |
+
"estimation_contribution": 0.0010060493440448813,
|
| 333 |
+
"action_remaining_headroom": 0.019933271000722286,
|
| 334 |
+
"fraction_action_headroom_closed": 0.04347280540375422,
|
| 335 |
+
"true_code_joint_oracle": 0.927818092070147,
|
| 336 |
+
"true_code_action_oracle": 0.9281063650817161,
|
| 337 |
+
"true_code_joint_fitted": 0.9170722759812633,
|
| 338 |
+
"true_code_action_fitted": 0.9171807569702725,
|
| 339 |
+
"joint_root_choice_mismatch": 0.23936604222246965,
|
| 340 |
+
"action_root_choice_mismatch": 0.23868371954799117,
|
| 341 |
+
"strongest_uncompressed_mean": 0.9251088162712999
|
| 342 |
+
},
|
| 343 |
+
{
|
| 344 |
+
"family": "independent",
|
| 345 |
+
"records": 1024,
|
| 346 |
+
"worlds": 32,
|
| 347 |
+
"identity": 0.9100079337524132,
|
| 348 |
+
"query_joint": 0.913013489675506,
|
| 349 |
+
"posterior_clusters": 0.915318662687657,
|
| 350 |
+
"posterior_bins": 0.9168213117935321,
|
| 351 |
+
"future_marginal": 0.8954121957914773,
|
| 352 |
+
"raw_information": 0.9031660787545921,
|
| 353 |
+
"raw_entropy": 0.6489306322062101,
|
| 354 |
+
"hash_code": 0.470594937948292,
|
| 355 |
+
"action_values": 0.9133526580360904,
|
| 356 |
+
"true_oracle": 0.928933733504975,
|
| 357 |
+
"joint_minus_action": -0.00033916836058426117,
|
| 358 |
+
"paired_95_low": -0.0011080639841925125,
|
| 359 |
+
"paired_95_high": 0.00048298344501220073,
|
| 360 |
+
"action_encoding_loss": 0.006376698269395811,
|
| 361 |
+
"joint_encoding_loss": 0.00708733789545923,
|
| 362 |
+
"action_estimation_loss": 0.009204377199488828,
|
| 363 |
+
"joint_estimation_loss": 0.00883290593400967,
|
| 364 |
+
"encoding_contribution": -0.0007106396260634189,
|
| 365 |
+
"estimation_contribution": 0.0003714712654791577,
|
| 366 |
+
"action_remaining_headroom": 0.01558107546888464,
|
| 367 |
+
"fraction_action_headroom_closed": -0.021767968537318196,
|
| 368 |
+
"true_code_joint_oracle": 0.927818092070147,
|
| 369 |
+
"true_code_action_oracle": 0.9281063650817161,
|
| 370 |
+
"true_code_joint_fitted": 0.9209817567018171,
|
| 371 |
+
"true_code_action_fitted": 0.9206480724749362,
|
| 372 |
+
"joint_root_choice_mismatch": 0.22556813929192912,
|
| 373 |
+
"action_root_choice_mismatch": 0.21870599191031026,
|
| 374 |
+
"strongest_uncompressed_mean": 0.9280646584591856
|
| 375 |
+
},
|
| 376 |
+
{
|
| 377 |
+
"family": "independent",
|
| 378 |
+
"records": 4096,
|
| 379 |
+
"worlds": 32,
|
| 380 |
+
"identity": 0.9190137606668483,
|
| 381 |
+
"query_joint": 0.9212120370356884,
|
| 382 |
+
"posterior_clusters": 0.9233281672038094,
|
| 383 |
+
"posterior_bins": 0.9234185604697762,
|
| 384 |
+
"future_marginal": 0.9120224134643348,
|
| 385 |
+
"raw_information": 0.9084610246510227,
|
| 386 |
+
"raw_entropy": 0.6461753611274292,
|
| 387 |
+
"hash_code": 0.47480680272329256,
|
| 388 |
+
"action_values": 0.9224258752130949,
|
| 389 |
+
"true_oracle": 0.928933733504975,
|
| 390 |
+
"joint_minus_action": -0.0012138381774065114,
|
| 391 |
+
"paired_95_low": -0.0017147112652704927,
|
| 392 |
+
"paired_95_high": -0.000726757585522743,
|
| 393 |
+
"action_encoding_loss": 0.0038865080122648353,
|
| 394 |
+
"joint_encoding_loss": 0.004790920676205797,
|
| 395 |
+
"action_estimation_loss": 0.002621350279615183,
|
| 396 |
+
"joint_estimation_loss": 0.002930775793080733,
|
| 397 |
+
"encoding_contribution": -0.0009044126639409615,
|
| 398 |
+
"estimation_contribution": -0.0003094255134655499,
|
| 399 |
+
"action_remaining_headroom": 0.0065078582918800185,
|
| 400 |
+
"fraction_action_headroom_closed": -0.18651883968048918,
|
| 401 |
+
"true_code_joint_oracle": 0.927818092070147,
|
| 402 |
+
"true_code_action_oracle": 0.9281063650817161,
|
| 403 |
+
"true_code_joint_fitted": 0.9258990135155555,
|
| 404 |
+
"true_code_action_fitted": 0.9259546622363166,
|
| 405 |
+
"joint_root_choice_mismatch": 0.21008579345280287,
|
| 406 |
+
"action_root_choice_mismatch": 0.19455413681065953,
|
| 407 |
+
"strongest_uncompressed_mean": 0.9287125775280298
|
| 408 |
+
},
|
| 409 |
+
{
|
| 410 |
+
"family": "orientation_alias",
|
| 411 |
+
"records": 64,
|
| 412 |
+
"worlds": 32,
|
| 413 |
+
"identity": 0.1635502787070287,
|
| 414 |
+
"query_joint": 0.12885644874690116,
|
| 415 |
+
"posterior_clusters": 0.11989069408297903,
|
| 416 |
+
"posterior_bins": 0.12129218066799444,
|
| 417 |
+
"future_marginal": 0.11565627135211173,
|
| 418 |
+
"raw_information": 0.1194667363872123,
|
| 419 |
+
"raw_entropy": 0.08451253981799893,
|
| 420 |
+
"hash_code": 0.062005774703194766,
|
| 421 |
+
"action_values": 0.12837855808157933,
|
| 422 |
+
"true_oracle": 0.828928467957698,
|
| 423 |
+
"joint_minus_action": 0.00047789066532179854,
|
| 424 |
+
"paired_95_low": -0.005027820336695715,
|
| 425 |
+
"paired_95_high": 0.005779686114251358,
|
| 426 |
+
"action_encoding_loss": 0.3951517175523762,
|
| 427 |
+
"joint_encoding_loss": 0.4017012049487419,
|
| 428 |
+
"action_estimation_loss": 0.3053981923237425,
|
| 429 |
+
"joint_estimation_loss": 0.29837081426205503,
|
| 430 |
+
"encoding_contribution": -0.0065494873963657056,
|
| 431 |
+
"estimation_contribution": 0.007027378061687502,
|
| 432 |
+
"action_remaining_headroom": 0.7005499098761186,
|
| 433 |
+
"fraction_action_headroom_closed": 0.0006821650514612244,
|
| 434 |
+
"true_code_joint_oracle": 0.828928467957698,
|
| 435 |
+
"true_code_action_oracle": 0.828928467957698,
|
| 436 |
+
"true_code_joint_fitted": 0.23582020614293137,
|
| 437 |
+
"true_code_action_fitted": 0.23582020614293134,
|
| 438 |
+
"joint_root_choice_mismatch": 0.6245539383534132,
|
| 439 |
+
"action_root_choice_mismatch": 0.6283048497052103,
|
| 440 |
+
"strongest_uncompressed_mean": 0.3744516848229764
|
| 441 |
+
},
|
| 442 |
+
{
|
| 443 |
+
"family": "orientation_alias",
|
| 444 |
+
"records": 256,
|
| 445 |
+
"worlds": 32,
|
| 446 |
+
"identity": 0.48105690410695323,
|
| 447 |
+
"query_joint": 0.45413249913205544,
|
| 448 |
+
"posterior_clusters": 0.3267375095162406,
|
| 449 |
+
"posterior_bins": 0.3044234970537981,
|
| 450 |
+
"future_marginal": 0.17946386454389426,
|
| 451 |
+
"raw_information": 0.3385633917456905,
|
| 452 |
+
"raw_entropy": 0.26942870751962467,
|
| 453 |
+
"hash_code": 0.14858964279293768,
|
| 454 |
+
"action_values": 0.4108957134778612,
|
| 455 |
+
"true_oracle": 0.828928467957698,
|
| 456 |
+
"joint_minus_action": 0.04323678565419422,
|
| 457 |
+
"paired_95_low": 0.03091808741028508,
|
| 458 |
+
"paired_95_high": 0.056253384606153346,
|
| 459 |
+
"action_encoding_loss": 0.22593360992407183,
|
| 460 |
+
"joint_encoding_loss": 0.15025128394784587,
|
| 461 |
+
"action_estimation_loss": 0.19209914455576496,
|
| 462 |
+
"joint_estimation_loss": 0.22454468487779672,
|
| 463 |
+
"encoding_contribution": 0.07568232597622598,
|
| 464 |
+
"estimation_contribution": -0.03244554032203176,
|
| 465 |
+
"action_remaining_headroom": 0.4180327544798368,
|
| 466 |
+
"fraction_action_headroom_closed": 0.1034291815434278,
|
| 467 |
+
"true_code_joint_oracle": 0.828928467957698,
|
| 468 |
+
"true_code_action_oracle": 0.828928467957698,
|
| 469 |
+
"true_code_joint_fitted": 0.5438758040593306,
|
| 470 |
+
"true_code_action_fitted": 0.5438758040593306,
|
| 471 |
+
"joint_root_choice_mismatch": 0.5760678352632569,
|
| 472 |
+
"action_root_choice_mismatch": 0.6058617660589638,
|
| 473 |
+
"strongest_uncompressed_mean": 0.6522145924562783
|
| 474 |
+
},
|
| 475 |
+
{
|
| 476 |
+
"family": "orientation_alias",
|
| 477 |
+
"records": 1024,
|
| 478 |
+
"worlds": 32,
|
| 479 |
+
"identity": 0.7375432253689826,
|
| 480 |
+
"query_joint": 0.749695480009097,
|
| 481 |
+
"posterior_clusters": 0.5557157182324963,
|
| 482 |
+
"posterior_bins": 0.5477364417189148,
|
| 483 |
+
"future_marginal": 0.18531714088359752,
|
| 484 |
+
"raw_information": 0.6134052546318305,
|
| 485 |
+
"raw_entropy": 0.6043586411911206,
|
| 486 |
+
"hash_code": 0.2567112571360594,
|
| 487 |
+
"action_values": 0.7156206511171621,
|
| 488 |
+
"true_oracle": 0.828928467957698,
|
| 489 |
+
"joint_minus_action": 0.03407482889193493,
|
| 490 |
+
"paired_95_low": 0.026490100551048997,
|
| 491 |
+
"paired_95_high": 0.0415008522799025,
|
| 492 |
+
"action_encoding_loss": 0.040254364563248254,
|
| 493 |
+
"joint_encoding_loss": 0.01143065598560973,
|
| 494 |
+
"action_estimation_loss": 0.07305345227728763,
|
| 495 |
+
"joint_estimation_loss": 0.06780233196299122,
|
| 496 |
+
"encoding_contribution": 0.028823708577638524,
|
| 497 |
+
"estimation_contribution": 0.00525112031429641,
|
| 498 |
+
"action_remaining_headroom": 0.1133078168405359,
|
| 499 |
+
"fraction_action_headroom_closed": 0.30072796248373795,
|
| 500 |
+
"true_code_joint_oracle": 0.828928467957698,
|
| 501 |
+
"true_code_action_oracle": 0.828928467957698,
|
| 502 |
+
"true_code_joint_fitted": 0.766961318774352,
|
| 503 |
+
"true_code_action_fitted": 0.7669613187743521,
|
| 504 |
+
"joint_root_choice_mismatch": 0.3803245810966434,
|
| 505 |
+
"action_root_choice_mismatch": 0.4051527363935008,
|
| 506 |
+
"strongest_uncompressed_mean": 0.767114986178982
|
| 507 |
+
},
|
| 508 |
+
{
|
| 509 |
+
"family": "orientation_alias",
|
| 510 |
+
"records": 4096,
|
| 511 |
+
"worlds": 32,
|
| 512 |
+
"identity": 0.804304431565352,
|
| 513 |
+
"query_joint": 0.8195916562097663,
|
| 514 |
+
"posterior_clusters": 0.6530579291374521,
|
| 515 |
+
"posterior_bins": 0.648285517967734,
|
| 516 |
+
"future_marginal": 0.18829678111200815,
|
| 517 |
+
"raw_information": 0.6964865610322947,
|
| 518 |
+
"raw_entropy": 0.706006590523646,
|
| 519 |
+
"hash_code": 0.3003141222527754,
|
| 520 |
+
"action_values": 0.819480859898903,
|
| 521 |
+
"true_oracle": 0.828928467957698,
|
| 522 |
+
"joint_minus_action": 0.00011079631086339387,
|
| 523 |
+
"paired_95_low": -2.228212253385705e-05,
|
| 524 |
+
"paired_95_high": 0.0002634376906898875,
|
| 525 |
+
"action_encoding_loss": 0.0002445290260795034,
|
| 526 |
+
"joint_encoding_loss": 0.00019810226223412192,
|
| 527 |
+
"action_estimation_loss": 0.009203079032715649,
|
| 528 |
+
"joint_estimation_loss": 0.009138709485697637,
|
| 529 |
+
"encoding_contribution": 4.64267638453815e-05,
|
| 530 |
+
"estimation_contribution": 6.436954701801237e-05,
|
| 531 |
+
"action_remaining_headroom": 0.009447608058795152,
|
| 532 |
+
"fraction_action_headroom_closed": 0.011727445737998115,
|
| 533 |
+
"true_code_joint_oracle": 0.828928467957698,
|
| 534 |
+
"true_code_action_oracle": 0.828928467957698,
|
| 535 |
+
"true_code_joint_fitted": 0.8198637628550727,
|
| 536 |
+
"true_code_action_fitted": 0.8198637628550727,
|
| 537 |
+
"joint_root_choice_mismatch": 0.21189316146075493,
|
| 538 |
+
"action_root_choice_mismatch": 0.21099546987915718,
|
| 539 |
+
"strongest_uncompressed_mean": 0.8096451880020169
|
| 540 |
+
},
|
| 541 |
+
{
|
| 542 |
+
"family": "parity",
|
| 543 |
+
"records": 64,
|
| 544 |
+
"worlds": 32,
|
| 545 |
+
"identity": 0.06273197484460194,
|
| 546 |
+
"query_joint": 0.02431126538233747,
|
| 547 |
+
"posterior_clusters": 0.011591763903878482,
|
| 548 |
+
"posterior_bins": 0.011981140143721973,
|
| 549 |
+
"future_marginal": 0.015523777582997956,
|
| 550 |
+
"raw_information": 0.011150427588187746,
|
| 551 |
+
"raw_entropy": 0.020184840684643574,
|
| 552 |
+
"hash_code": 0.024015757122275827,
|
| 553 |
+
"action_values": 0.02252062020077706,
|
| 554 |
+
"true_oracle": 0.8347957761429843,
|
| 555 |
+
"joint_minus_action": 0.0017906451815604108,
|
| 556 |
+
"paired_95_low": -0.0038315227161669556,
|
| 557 |
+
"paired_95_high": 0.006652954501555841,
|
| 558 |
+
"action_encoding_loss": 0.46294985993286397,
|
| 559 |
+
"joint_encoding_loss": 0.4524966576959601,
|
| 560 |
+
"action_estimation_loss": 0.34932529600934337,
|
| 561 |
+
"joint_estimation_loss": 0.35798785306468683,
|
| 562 |
+
"encoding_contribution": 0.010453202236903802,
|
| 563 |
+
"estimation_contribution": -0.008662557055343396,
|
| 564 |
+
"action_remaining_headroom": 0.8122751559422072,
|
| 565 |
+
"fraction_action_headroom_closed": 0.0022044810412591444,
|
| 566 |
+
"true_code_joint_oracle": 0.8347957761429844,
|
| 567 |
+
"true_code_action_oracle": 0.8347957761429844,
|
| 568 |
+
"true_code_joint_fitted": 0.12286533950243893,
|
| 569 |
+
"true_code_action_fitted": 0.12288347183177817,
|
| 570 |
+
"joint_root_choice_mismatch": 0.5209659718033554,
|
| 571 |
+
"action_root_choice_mismatch": 0.5285378271397623,
|
| 572 |
+
"strongest_uncompressed_mean": 0.385086063569183
|
| 573 |
+
},
|
| 574 |
+
{
|
| 575 |
+
"family": "parity",
|
| 576 |
+
"records": 256,
|
| 577 |
+
"worlds": 32,
|
| 578 |
+
"identity": 0.4475888678222731,
|
| 579 |
+
"query_joint": 0.41757388132172324,
|
| 580 |
+
"posterior_clusters": 0.2750026478906139,
|
| 581 |
+
"posterior_bins": 0.23783397003296353,
|
| 582 |
+
"future_marginal": 0.1298958512570368,
|
| 583 |
+
"raw_information": 0.20886862239855064,
|
| 584 |
+
"raw_entropy": 0.18373006939535583,
|
| 585 |
+
"hash_code": 0.11691960704302928,
|
| 586 |
+
"action_values": 0.38718031379931517,
|
| 587 |
+
"true_oracle": 0.8347957761429843,
|
| 588 |
+
"joint_minus_action": 0.030393567522408023,
|
| 589 |
+
"paired_95_low": 0.019497351412313457,
|
| 590 |
+
"paired_95_high": 0.041409246567013835,
|
| 591 |
+
"action_encoding_loss": 0.22704928735222724,
|
| 592 |
+
"joint_encoding_loss": 0.15105877295815492,
|
| 593 |
+
"action_estimation_loss": 0.220566174991442,
|
| 594 |
+
"joint_estimation_loss": 0.2661631218631063,
|
| 595 |
+
"encoding_contribution": 0.07599051439407231,
|
| 596 |
+
"estimation_contribution": -0.04559694687166428,
|
| 597 |
+
"action_remaining_headroom": 0.4476154623436692,
|
| 598 |
+
"fraction_action_headroom_closed": 0.06790106705266699,
|
| 599 |
+
"true_code_joint_oracle": 0.8347957761429844,
|
| 600 |
+
"true_code_action_oracle": 0.8347957761429844,
|
| 601 |
+
"true_code_joint_fitted": 0.4773303993370155,
|
| 602 |
+
"true_code_action_fitted": 0.4773303993370155,
|
| 603 |
+
"joint_root_choice_mismatch": 0.5556145869981621,
|
| 604 |
+
"action_root_choice_mismatch": 0.5840915047008,
|
| 605 |
+
"strongest_uncompressed_mean": 0.6622060304284867
|
| 606 |
+
},
|
| 607 |
+
{
|
| 608 |
+
"family": "parity",
|
| 609 |
+
"records": 1024,
|
| 610 |
+
"worlds": 32,
|
| 611 |
+
"identity": 0.7505863638432653,
|
| 612 |
+
"query_joint": 0.7643539645704318,
|
| 613 |
+
"posterior_clusters": 0.5785860682616655,
|
| 614 |
+
"posterior_bins": 0.5650067039347257,
|
| 615 |
+
"future_marginal": 0.17619358394784873,
|
| 616 |
+
"raw_information": 0.491345522934861,
|
| 617 |
+
"raw_entropy": 0.5428645204900009,
|
| 618 |
+
"hash_code": 0.20833025114024675,
|
| 619 |
+
"action_values": 0.749467269935558,
|
| 620 |
+
"true_oracle": 0.8347957761429843,
|
| 621 |
+
"joint_minus_action": 0.01488669463487384,
|
| 622 |
+
"paired_95_low": 0.010441901192385099,
|
| 623 |
+
"paired_95_high": 0.019552134986879535,
|
| 624 |
+
"action_encoding_loss": 0.021393038552985848,
|
| 625 |
+
"joint_encoding_loss": 0.0075410150709803,
|
| 626 |
+
"action_estimation_loss": 0.06393546765444057,
|
| 627 |
+
"joint_estimation_loss": 0.06290079650157226,
|
| 628 |
+
"encoding_contribution": 0.013852023482005548,
|
| 629 |
+
"estimation_contribution": 0.0010346711528682918,
|
| 630 |
+
"action_remaining_headroom": 0.08532850620742641,
|
| 631 |
+
"fraction_action_headroom_closed": 0.17446332177297866,
|
| 632 |
+
"true_code_joint_oracle": 0.8347957761429844,
|
| 633 |
+
"true_code_action_oracle": 0.8347957761429844,
|
| 634 |
+
"true_code_joint_fitted": 0.7761508394516206,
|
| 635 |
+
"true_code_action_fitted": 0.7761508394516206,
|
| 636 |
+
"joint_root_choice_mismatch": 0.3591226824418755,
|
| 637 |
+
"action_root_choice_mismatch": 0.35639638530427453,
|
| 638 |
+
"strongest_uncompressed_mean": 0.7761873106567354
|
| 639 |
+
},
|
| 640 |
+
{
|
| 641 |
+
"family": "parity",
|
| 642 |
+
"records": 4096,
|
| 643 |
+
"worlds": 32,
|
| 644 |
+
"identity": 0.811457808557315,
|
| 645 |
+
"query_joint": 0.824131626938427,
|
| 646 |
+
"posterior_clusters": 0.694169383413467,
|
| 647 |
+
"posterior_bins": 0.6900830814062695,
|
| 648 |
+
"future_marginal": 0.18472462130093503,
|
| 649 |
+
"raw_information": 0.6242484323345312,
|
| 650 |
+
"raw_entropy": 0.6210217172753493,
|
| 651 |
+
"hash_code": 0.2392919195668825,
|
| 652 |
+
"action_values": 0.824102588730724,
|
| 653 |
+
"true_oracle": 0.8347957761429843,
|
| 654 |
+
"joint_minus_action": 2.9038207703040858e-05,
|
| 655 |
+
"paired_95_low": -1.3477962034352808e-06,
|
| 656 |
+
"paired_95_high": 7.886248154405023e-05,
|
| 657 |
+
"action_encoding_loss": 9.112519989473711e-05,
|
| 658 |
+
"joint_encoding_loss": 6.378990475829419e-05,
|
| 659 |
+
"action_estimation_loss": 0.010602062212365704,
|
| 660 |
+
"joint_estimation_loss": 0.010600359299799106,
|
| 661 |
+
"encoding_contribution": 2.7335295136442922e-05,
|
| 662 |
+
"estimation_contribution": 1.7029125665979361e-06,
|
| 663 |
+
"action_remaining_headroom": 0.010693187412260441,
|
| 664 |
+
"fraction_action_headroom_closed": 0.0027155801711421096,
|
| 665 |
+
"true_code_joint_oracle": 0.8347957761429844,
|
| 666 |
+
"true_code_action_oracle": 0.8347957761429844,
|
| 667 |
+
"true_code_joint_fitted": 0.8242244796942859,
|
| 668 |
+
"true_code_action_fitted": 0.8242244796942859,
|
| 669 |
+
"joint_root_choice_mismatch": 0.199476969506594,
|
| 670 |
+
"action_root_choice_mismatch": 0.1989397435570667,
|
| 671 |
+
"strongest_uncompressed_mean": 0.8150734693187245
|
| 672 |
+
},
|
| 673 |
+
{
|
| 674 |
+
"family": "redundant",
|
| 675 |
+
"records": 64,
|
| 676 |
+
"worlds": 32,
|
| 677 |
+
"identity": 0.5363358739730864,
|
| 678 |
+
"query_joint": 0.5113855171174998,
|
| 679 |
+
"posterior_clusters": 0.5036569357456289,
|
| 680 |
+
"posterior_bins": 0.5085041397628068,
|
| 681 |
+
"future_marginal": 0.5000048354948977,
|
| 682 |
+
"raw_information": 0.5772794551090056,
|
| 683 |
+
"raw_entropy": 0.4124257595149454,
|
| 684 |
+
"hash_code": -0.04050167310864608,
|
| 685 |
+
"action_values": 0.5053617140765039,
|
| 686 |
+
"true_oracle": 0.8364283751534423,
|
| 687 |
+
"joint_minus_action": 0.006023803040995961,
|
| 688 |
+
"paired_95_low": 0.0009731619737643064,
|
| 689 |
+
"paired_95_high": 0.011502247775086136,
|
| 690 |
+
"action_encoding_loss": 0.14576415220704214,
|
| 691 |
+
"joint_encoding_loss": 0.13364129983182016,
|
| 692 |
+
"action_estimation_loss": 0.1853025088698963,
|
| 693 |
+
"joint_estimation_loss": 0.1914015582041223,
|
| 694 |
+
"encoding_contribution": 0.012122852375221975,
|
| 695 |
+
"estimation_contribution": -0.006099049334226014,
|
| 696 |
+
"action_remaining_headroom": 0.3310666610769384,
|
| 697 |
+
"fraction_action_headroom_closed": 0.018195136355321676,
|
| 698 |
+
"true_code_joint_oracle": 0.8364283751534423,
|
| 699 |
+
"true_code_action_oracle": 0.8364283751534423,
|
| 700 |
+
"true_code_joint_fitted": 0.5890955182088521,
|
| 701 |
+
"true_code_action_fitted": 0.5890955182088521,
|
| 702 |
+
"joint_root_choice_mismatch": 0.8539832767484425,
|
| 703 |
+
"action_root_choice_mismatch": 0.8572854915108054,
|
| 704 |
+
"strongest_uncompressed_mean": 0.5662414160281566
|
| 705 |
+
},
|
| 706 |
+
{
|
| 707 |
+
"family": "redundant",
|
| 708 |
+
"records": 256,
|
| 709 |
+
"worlds": 32,
|
| 710 |
+
"identity": 0.6453415693659241,
|
| 711 |
+
"query_joint": 0.6198034615018286,
|
| 712 |
+
"posterior_clusters": 0.5877161849929605,
|
| 713 |
+
"posterior_bins": 0.591621214981205,
|
| 714 |
+
"future_marginal": 0.42260265019739973,
|
| 715 |
+
"raw_information": 0.6312017727896639,
|
| 716 |
+
"raw_entropy": 0.5514639211048289,
|
| 717 |
+
"hash_code": 0.019364542901451464,
|
| 718 |
+
"action_values": 0.6050694499969314,
|
| 719 |
+
"true_oracle": 0.8364283751534423,
|
| 720 |
+
"joint_minus_action": 0.014734011504897247,
|
| 721 |
+
"paired_95_low": 0.009603019224582635,
|
| 722 |
+
"paired_95_high": 0.01995868486298478,
|
| 723 |
+
"action_encoding_loss": 0.0932629369589013,
|
| 724 |
+
"joint_encoding_loss": 0.07785092527231657,
|
| 725 |
+
"action_estimation_loss": 0.13809598819760954,
|
| 726 |
+
"joint_estimation_loss": 0.13877398837929703,
|
| 727 |
+
"encoding_contribution": 0.015412011686584716,
|
| 728 |
+
"estimation_contribution": -0.0006780001816874687,
|
| 729 |
+
"action_remaining_headroom": 0.23135892515651085,
|
| 730 |
+
"fraction_action_headroom_closed": 0.06368464711240299,
|
| 731 |
+
"true_code_joint_oracle": 0.8364283751534423,
|
| 732 |
+
"true_code_action_oracle": 0.8364283751534423,
|
| 733 |
+
"true_code_joint_fitted": 0.653629840527671,
|
| 734 |
+
"true_code_action_fitted": 0.653629840527671,
|
| 735 |
+
"joint_root_choice_mismatch": 0.763869962732947,
|
| 736 |
+
"action_root_choice_mismatch": 0.7826760570524698,
|
| 737 |
+
"strongest_uncompressed_mean": 0.7105456685441063
|
| 738 |
+
},
|
| 739 |
+
{
|
| 740 |
+
"family": "redundant",
|
| 741 |
+
"records": 1024,
|
| 742 |
+
"worlds": 32,
|
| 743 |
+
"identity": 0.7653668596783447,
|
| 744 |
+
"query_joint": 0.7545480306631399,
|
| 745 |
+
"posterior_clusters": 0.6989231534206255,
|
| 746 |
+
"posterior_bins": 0.6916977366851904,
|
| 747 |
+
"future_marginal": 0.29795754465378854,
|
| 748 |
+
"raw_information": 0.7244842637951285,
|
| 749 |
+
"raw_entropy": 0.6854194139838374,
|
| 750 |
+
"hash_code": 0.04484946990041256,
|
| 751 |
+
"action_values": 0.7412029739417946,
|
| 752 |
+
"true_oracle": 0.8364283751534423,
|
| 753 |
+
"joint_minus_action": 0.013345056721345335,
|
| 754 |
+
"paired_95_low": 0.010143917478369981,
|
| 755 |
+
"paired_95_high": 0.016472364541550773,
|
| 756 |
+
"action_encoding_loss": 0.039402157560219295,
|
| 757 |
+
"joint_encoding_loss": 0.02814080307390973,
|
| 758 |
+
"action_estimation_loss": 0.05582324365142841,
|
| 759 |
+
"joint_estimation_loss": 0.05373954141639264,
|
| 760 |
+
"encoding_contribution": 0.01126135448630956,
|
| 761 |
+
"estimation_contribution": 0.0020837022350357753,
|
| 762 |
+
"action_remaining_headroom": 0.0952254012116477,
|
| 763 |
+
"fraction_action_headroom_closed": 0.14014177468976635,
|
| 764 |
+
"true_code_joint_oracle": 0.8364283751534423,
|
| 765 |
+
"true_code_action_oracle": 0.8364283751534423,
|
| 766 |
+
"true_code_joint_fitted": 0.7721932591100672,
|
| 767 |
+
"true_code_action_fitted": 0.7721932591100673,
|
| 768 |
+
"joint_root_choice_mismatch": 0.4023677500023982,
|
| 769 |
+
"action_root_choice_mismatch": 0.43296693574774525,
|
| 770 |
+
"strongest_uncompressed_mean": 0.7863078402596235
|
| 771 |
+
},
|
| 772 |
+
{
|
| 773 |
+
"family": "redundant",
|
| 774 |
+
"records": 4096,
|
| 775 |
+
"worlds": 32,
|
| 776 |
+
"identity": 0.8155857149751549,
|
| 777 |
+
"query_joint": 0.8194726488422938,
|
| 778 |
+
"posterior_clusters": 0.7193415226940727,
|
| 779 |
+
"posterior_bins": 0.6960627204154617,
|
| 780 |
+
"future_marginal": 0.18000890013383308,
|
| 781 |
+
"raw_information": 0.7543788456935281,
|
| 782 |
+
"raw_entropy": 0.752804281570126,
|
| 783 |
+
"hash_code": 0.05292377176364181,
|
| 784 |
+
"action_values": 0.8165418628657128,
|
| 785 |
+
"true_oracle": 0.8364283751534423,
|
| 786 |
+
"joint_minus_action": 0.002930785976580884,
|
| 787 |
+
"paired_95_low": 0.0019304463049228831,
|
| 788 |
+
"paired_95_high": 0.004100380433294095,
|
| 789 |
+
"action_encoding_loss": 0.00730164446795117,
|
| 790 |
+
"joint_encoding_loss": 0.0038787169871150502,
|
| 791 |
+
"action_estimation_loss": 0.012584867819778103,
|
| 792 |
+
"joint_estimation_loss": 0.013077009324033339,
|
| 793 |
+
"encoding_contribution": 0.00342292748083612,
|
| 794 |
+
"estimation_contribution": -0.0004921415042552363,
|
| 795 |
+
"action_remaining_headroom": 0.019886512287729273,
|
| 796 |
+
"fraction_action_headroom_closed": 0.14737556461266912,
|
| 797 |
+
"true_code_joint_oracle": 0.8364283751534423,
|
| 798 |
+
"true_code_action_oracle": 0.8364283751534423,
|
| 799 |
+
"true_code_joint_fitted": 0.8226174136373886,
|
| 800 |
+
"true_code_action_fitted": 0.8226174136373885,
|
| 801 |
+
"joint_root_choice_mismatch": 0.1746054019842278,
|
| 802 |
+
"action_root_choice_mismatch": 0.18535970463320198,
|
| 803 |
+
"strongest_uncompressed_mean": 0.8207655371219653
|
| 804 |
+
},
|
| 805 |
+
{
|
| 806 |
+
"family": "uninformative",
|
| 807 |
+
"records": 64,
|
| 808 |
+
"worlds": 32,
|
| 809 |
+
"identity": -0.04890966142219218,
|
| 810 |
+
"query_joint": -0.04575341745644379,
|
| 811 |
+
"posterior_clusters": -0.047049768625332646,
|
| 812 |
+
"posterior_bins": -0.04950046362247272,
|
| 813 |
+
"future_marginal": -0.026933077678025653,
|
| 814 |
+
"raw_information": -0.041441113900889705,
|
| 815 |
+
"raw_entropy": -0.010995531167365718,
|
| 816 |
+
"hash_code": 0.02058205608137745,
|
| 817 |
+
"action_values": -0.043723918882921195,
|
| 818 |
+
"true_oracle": 0.0667348907999358,
|
| 819 |
+
"joint_minus_action": -0.0020294985735225856,
|
| 820 |
+
"paired_95_low": -0.0039889178805482635,
|
| 821 |
+
"paired_95_high": -0.000114955541232761,
|
| 822 |
+
"action_encoding_loss": -2.0599841277224584e-18,
|
| 823 |
+
"joint_encoding_loss": -9.75781955236954e-19,
|
| 824 |
+
"action_estimation_loss": 0.11045880968285701,
|
| 825 |
+
"joint_estimation_loss": 0.11248830825637958,
|
| 826 |
+
"encoding_contribution": -1.0842021724855044e-18,
|
| 827 |
+
"estimation_contribution": -0.002029498573522584,
|
| 828 |
+
"action_remaining_headroom": 0.11045880968285701,
|
| 829 |
+
"fraction_action_headroom_closed": -0.018373351834494372,
|
| 830 |
+
"true_code_joint_oracle": 0.0667348907999358,
|
| 831 |
+
"true_code_action_oracle": 0.0667348907999358,
|
| 832 |
+
"true_code_joint_fitted": -0.008709004374880524,
|
| 833 |
+
"true_code_action_fitted": -0.01331114610045102,
|
| 834 |
+
"joint_root_choice_mismatch": 0.5172644451120586,
|
| 835 |
+
"action_root_choice_mismatch": 0.5137298310222932,
|
| 836 |
+
"strongest_uncompressed_mean": -0.04890966142219218
|
| 837 |
+
},
|
| 838 |
+
{
|
| 839 |
+
"family": "uninformative",
|
| 840 |
+
"records": 256,
|
| 841 |
+
"worlds": 32,
|
| 842 |
+
"identity": 0.015731854753485836,
|
| 843 |
+
"query_joint": 0.03111698106830645,
|
| 844 |
+
"posterior_clusters": 0.02924553921598229,
|
| 845 |
+
"posterior_bins": 0.03234940005689452,
|
| 846 |
+
"future_marginal": 0.046069027949175986,
|
| 847 |
+
"raw_information": 0.03744741765448152,
|
| 848 |
+
"raw_entropy": 0.04500280677191939,
|
| 849 |
+
"hash_code": 0.04957765804565338,
|
| 850 |
+
"action_values": 0.024803729988287868,
|
| 851 |
+
"true_oracle": 0.0667348907999358,
|
| 852 |
+
"joint_minus_action": 0.006313251080018578,
|
| 853 |
+
"paired_95_low": 0.0035633612177853723,
|
| 854 |
+
"paired_95_high": 0.009296090920171755,
|
| 855 |
+
"action_encoding_loss": -1.1926223897340549e-18,
|
| 856 |
+
"joint_encoding_loss": 5.421010862427522e-19,
|
| 857 |
+
"action_estimation_loss": 0.04193116081164793,
|
| 858 |
+
"joint_estimation_loss": 0.03561790973162935,
|
| 859 |
+
"encoding_contribution": -1.734723475976807e-18,
|
| 860 |
+
"estimation_contribution": 0.006313251080018579,
|
| 861 |
+
"action_remaining_headroom": 0.04193116081164793,
|
| 862 |
+
"fraction_action_headroom_closed": 0.15056227773844122,
|
| 863 |
+
"true_code_joint_oracle": 0.0667348907999358,
|
| 864 |
+
"true_code_action_oracle": 0.0667348907999358,
|
| 865 |
+
"true_code_joint_fitted": 0.046611496345385736,
|
| 866 |
+
"true_code_action_fitted": 0.04479898126383822,
|
| 867 |
+
"joint_root_choice_mismatch": 0.318326143510767,
|
| 868 |
+
"action_root_choice_mismatch": 0.3350521771578457,
|
| 869 |
+
"strongest_uncompressed_mean": 0.026880985378230898
|
| 870 |
+
},
|
| 871 |
+
{
|
| 872 |
+
"family": "uninformative",
|
| 873 |
+
"records": 1024,
|
| 874 |
+
"worlds": 32,
|
| 875 |
+
"identity": 0.02450926400985953,
|
| 876 |
+
"query_joint": 0.046334453063967655,
|
| 877 |
+
"posterior_clusters": 0.041247390040046704,
|
| 878 |
+
"posterior_bins": 0.048595464204311184,
|
| 879 |
+
"future_marginal": 0.055973307164560476,
|
| 880 |
+
"raw_information": 0.05435897532129334,
|
| 881 |
+
"raw_entropy": 0.05452489439127397,
|
| 882 |
+
"hash_code": 0.05498349310416952,
|
| 883 |
+
"action_values": 0.038204503983425536,
|
| 884 |
+
"true_oracle": 0.0667348907999358,
|
| 885 |
+
"joint_minus_action": 0.008129949080542126,
|
| 886 |
+
"paired_95_low": 0.0048984844703365985,
|
| 887 |
+
"paired_95_high": 0.011408492793951204,
|
| 888 |
+
"action_encoding_loss": 4.553649124439119e-18,
|
| 889 |
+
"joint_encoding_loss": -3.3610267347050637e-18,
|
| 890 |
+
"action_estimation_loss": 0.028530386816510264,
|
| 891 |
+
"joint_estimation_loss": 0.020400437735968148,
|
| 892 |
+
"encoding_contribution": 7.914675859144182e-18,
|
| 893 |
+
"estimation_contribution": 0.008129949080542116,
|
| 894 |
+
"action_remaining_headroom": 0.028530386816510267,
|
| 895 |
+
"fraction_action_headroom_closed": 0.2849575483441185,
|
| 896 |
+
"true_code_joint_oracle": 0.0667348907999358,
|
| 897 |
+
"true_code_action_oracle": 0.0667348907999358,
|
| 898 |
+
"true_code_joint_fitted": 0.05766653710546027,
|
| 899 |
+
"true_code_action_fitted": 0.0569982079524269,
|
| 900 |
+
"joint_root_choice_mismatch": 0.27031338447698905,
|
| 901 |
+
"action_root_choice_mismatch": 0.29409977769850704,
|
| 902 |
+
"strongest_uncompressed_mean": 0.05871121561510939
|
| 903 |
+
},
|
| 904 |
+
{
|
| 905 |
+
"family": "uninformative",
|
| 906 |
+
"records": 4096,
|
| 907 |
+
"worlds": 32,
|
| 908 |
+
"identity": 0.044186530387099784,
|
| 909 |
+
"query_joint": 0.05829518844278467,
|
| 910 |
+
"posterior_clusters": 0.05390322882895525,
|
| 911 |
+
"posterior_bins": 0.05970059877167674,
|
| 912 |
+
"future_marginal": 0.06250539980375937,
|
| 913 |
+
"raw_information": 0.06113657669308175,
|
| 914 |
+
"raw_entropy": 0.061513705438783525,
|
| 915 |
+
"hash_code": 0.0609891079973558,
|
| 916 |
+
"action_values": 0.05473918863077931,
|
| 917 |
+
"true_oracle": 0.0667348907999358,
|
| 918 |
+
"joint_minus_action": 0.0035559998120053464,
|
| 919 |
+
"paired_95_low": 0.002404317887698644,
|
| 920 |
+
"paired_95_high": 0.004959589260649769,
|
| 921 |
+
"action_encoding_loss": 4.7704895589362195e-18,
|
| 922 |
+
"joint_encoding_loss": 5.421010862427522e-19,
|
| 923 |
+
"action_estimation_loss": 0.011995702169156477,
|
| 924 |
+
"joint_estimation_loss": 0.008439702357151135,
|
| 925 |
+
"encoding_contribution": 4.228388472693467e-18,
|
| 926 |
+
"estimation_contribution": 0.003555999812005342,
|
| 927 |
+
"action_remaining_headroom": 0.011995702169156483,
|
| 928 |
+
"fraction_action_headroom_closed": 0.2964394882317588,
|
| 929 |
+
"true_code_joint_oracle": 0.0667348907999358,
|
| 930 |
+
"true_code_action_oracle": 0.0667348907999358,
|
| 931 |
+
"true_code_joint_fitted": 0.06340955513774679,
|
| 932 |
+
"true_code_action_fitted": 0.0624701246085254,
|
| 933 |
+
"joint_root_choice_mismatch": 0.21447414713172985,
|
| 934 |
+
"action_root_choice_mismatch": 0.22634002044987156,
|
| 935 |
+
"strongest_uncompressed_mean": 0.06369397045613248
|
| 936 |
+
}
|
| 937 |
+
],
|
| 938 |
+
"all_conditions_reported": true,
|
| 939 |
+
"intervals": "Descriptive paired within-family bootstrap, 10000 draws, seed 18701; no multiplicity correction or confirmatory interpretation",
|
| 940 |
+
"limitations": [
|
| 941 |
+
"Constructed worlds share favorable finite-law structure; no external validity follows.",
|
| 942 |
+
"True-law encoders and true-law replanning have privileged access and are explanatory diagnostics only.",
|
| 943 |
+
"Exact additive loss separation identifies where the observed finite-grid gap arises; it does not identify a unique learning-theoretic cause or guarantee asymptotic dominance.",
|
| 944 |
+
"All algorithms retain the released deterministic initialization and 32 iterations; no attempt to optimize either clustering objective.",
|
| 945 |
+
"Headroom fractions are ratios of averages and not operational money or sensing savings.",
|
| 946 |
+
"Warm timing is a local batch observation on shared hardware; hardware startup and source data access are reported separately.",
|
| 947 |
+
"946 archived native world means do not exactly replay in the expanded execution trace; all such rows are retained separately. Primary-mixture joint/action-value/identity world means at every sample budget replay with maximum error 3.33e-16. New full-grid diagnostics are recomputations, not replacements for frozen archival scores."
|
| 948 |
+
]
|
| 949 |
+
}
|
revision_r1/controlled/timing.json
ADDED
|
The diff for this file is too large to render.
See raw diff
|
|
|
revision_r1/external/article_table.csv
ADDED
|
@@ -0,0 +1,20 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
analysis,common_current_joint_conditional_log_loss,fitted_compression_loss,gas_environments,gas_tasks,hydraulic_environments,hydraulic_tasks,joint_conditional_log_loss,labels_max,labels_min,noquery_utility,observation_given_h_log_loss,posterior_brier,posterior_log_loss,predicted_noquery_utility,predicted_query_increment,predicted_utility,queries,query_increment_optimism,realized_query_increment,tasks,utility
|
| 2 |
+
target_64_query_joint_on_target_evaluation_all57,0.9021537524971011,0.0014791213985141378,5,48,5,9,0.9202235047375056,64,64,0.4341858311122304,0.4713431260533678,0.14207717814233886,0.4488803786841378,0.5663140395782351,0.022600137737574737,0.5889141773158098,0.43706698485258466,0.028167567457767426,-0.005567429720192687,57,0.42861840139203766
|
| 3 |
+
target_64_posterior_clusters_on_target_evaluation_all57,0.9021537524971011,0.00206425583399923,5,48,5,9,0.9558448652595863,64,64,0.44368506699317145,0.5090721054033512,0.14116722448105434,0.4467727598562351,0.5668075955170042,0.021521447363320578,0.5883290428803247,0.4238130422310427,0.03794864053802975,-0.01642719317470917,57,0.42725787381846236
|
| 4 |
+
target_64_identity_on_target_evaluation_all57,0.9021537524971011,0.0,5,48,5,9,0.895850072130834,64,64,0.44420340770454436,0.4495203347543967,0.1413264442896859,0.4463297373764372,0.5674811876349388,0.02291211107938515,0.5903932987143239,0.42518703941463853,0.03604549979708882,-0.013133388717703665,57,0.4310700189868407
|
| 5 |
+
target_all_query_joint_on_target_evaluation_all57,0.6960366903855149,0.004799697896160156,5,48,5,9,0.7233527889368968,823,67,0.47670002693270863,0.32887130510623214,0.12778584484692165,0.3944814838306647,0.4952529448146883,0.01462258567675154,0.5098755304914399,0.4392621835902733,0.024546899602511985,-0.009924313925760445,57,0.4667757130069482
|
| 6 |
+
target_all_posterior_clusters_on_target_evaluation_all57,0.6960366903855149,0.003328144327637535,5,48,5,9,0.7876505767188415,823,67,0.47802205359215116,0.39681467189397057,0.12695844593296693,0.390835904824871,0.4996075681671639,0.011739515892798504,0.5113470840599624,0.43890637105567376,0.020765217143797324,-0.009025701250998821,57,0.4689963523411524
|
| 7 |
+
target_all_identity_on_target_evaluation_all57,0.6960366903855149,0.0,5,48,5,9,0.7078394813768716,823,67,0.4781249221284385,0.3163960529068561,0.1271812908421664,0.39144342847001556,0.5004346730166809,0.014240555370919127,0.5146752283876,0.44147345706354346,0.025157372572011803,-0.01091681720109267,57,0.46720810492734577
|
| 8 |
+
source_64_identity_on_source_held_all57,1.2938847482373521,0.0,5,48,5,9,1.3487109515234905,64,64,0.33440662650937636,0.8708436382289805,0.16025896218096947,0.47786731329451004,0.722022254088492,0.03006104296153668,0.7520832970500287,0.43304080035036646,-0.10170635547664639,0.13176739843818305,57,0.46617402494755944
|
| 9 |
+
source_64_identity_on_target_evaluation_all57,1.8532995603728606,0.0,5,48,5,9,1.9108555023828004,64,64,0.06673920740483243,1.052876482477108,0.2517142747070905,0.8579790199056924,0.722022254088492,0.03006104296153668,0.7520832970500287,0.483762359836609,-0.0035392117736777126,0.03360025473521439,57,0.10033946214004681
|
| 10 |
+
source_all_identity_on_source_held_all57,0.6404937484025411,0.0,5,48,5,9,0.7595273468083056,1430,400,0.731124166423397,0.5233765353700208,0.0719639418595227,0.23615081143828473,0.621897176545058,0.021625598675926755,0.6435227752209849,0.3886282868792795,0.05550853631963279,-0.033882937643706035,57,0.697241228779691
|
| 11 |
+
source_all_identity_on_target_evaluation_all57,1.3572171839606018,0.0,5,48,5,9,1.4448199908242385,1430,400,0.15937529013976298,0.7240851456651568,0.22750526451333272,0.7207348451590816,0.621897176545058,0.021625598675926755,0.6435227752209849,0.5231779414396178,-0.0766275842845273,0.09825318296045407,57,0.2576284731002171
|
| 12 |
+
target_64_query_joint_on_matched256eligible43,0.962944367808101,0.0016629665986308453,3,40,1,3,0.994952042783602,64,64,0.2933455394412311,0.4616198060448181,0.17695561042215174,0.533332236738784,0.4253933455778002,0.03476522133637247,0.46015856691417256,0.59982019504992,0.06686856540866529,-0.03210334407229281,43,0.2612421953689383
|
| 13 |
+
target_64_posterior_clusters_on_matched256eligible43,0.962944367808101,0.0017653999194657526,3,40,1,3,1.0543124169919997,64,64,0.29194760701280736,0.5212075436434427,0.17684661293020898,0.533104873348557,0.4261052889389456,0.03395084465439209,0.46005613359333775,0.5858980488484417,0.065767218097636,-0.031816373443243895,43,0.26013123356956347
|
| 14 |
+
target_64_identity_on_matched256eligible43,0.962944367808101,0.0,3,40,1,3,0.9812372654895479,64,64,0.2928115081984287,0.44856675165797866,0.17670638476417255,0.5326705138315693,0.4265949278730705,0.03522660563973295,0.4618215335128035,0.5999636913467485,0.06528806868106468,-0.030061463041331723,43,0.26275004515709705
|
| 15 |
+
target_256_query_joint_on_matched256eligible43,0.776763839151255,0.003964107645207928,3,40,1,3,0.7996329029836975,256,256,0.35585914250674133,0.2936103196675086,0.17152407443861453,0.5060225833161888,0.3512115814277825,0.023362338109154078,0.37457391953693664,0.564424019838484,0.027867767449796077,-0.0045054293406419955,43,0.3513537131660993
|
| 16 |
+
target_256_posterior_clusters_on_matched256eligible43,0.776763839151255,0.0028598850269441057,3,40,1,3,0.876625249191228,256,256,0.35662972148521666,0.371948609277301,0.1710897885778891,0.504676639913927,0.35368450489557457,0.021993637259625866,0.37567814215520046,0.548167026891073,0.02739402083024205,-0.00540038357061618,43,0.3512293379146005
|
| 17 |
+
target_256_identity_on_matched256eligible43,0.776763839151255,0.0,3,40,1,3,0.790112388208412,256,256,0.3565941836628872,0.28571420648711593,0.171004567859407,0.504398181721296,0.35440576895751386,0.024132258224630665,0.3785380271821446,0.5638333748702054,0.02748811340439971,-0.0033558551797690404,43,0.35323832848311815
|
| 18 |
+
empirical_alladaptation_query_joint_on_target_evaluation_all57,,6.807968017642793e-05,5,48,5,9,,823,67,0.46112566095566565,,,,0.6108515238336867,0.0013898851426896106,0.6122414089763762,0.02542517831387839,0.0013517400318673917,3.814511082221898e-05,57,0.4611638060664879
|
| 19 |
+
empirical_alladaptation_posterior_clusters_on_target_evaluation_all57,,2.4444444444429724e-05,5,48,5,9,,823,67,0.45795827338402395,,,,0.6109052872745468,0.0013797569375614061,0.6122850442121082,0.026571915606292424,0.001533767101958253,-0.00015401016439684685,57,0.4578042632196271
|
| 20 |
+
empirical_alladaptation_identity_on_target_evaluation_all57,,0.0,5,48,5,9,,823,67,0.4221120882269902,,,,0.6109052872745468,0.0014042013820058526,0.6123094886565527,0.025668404150831797,0.0015028618267391783,-9.866044473332557e-05,57,0.4220134277822568
|
revision_r1/external/execution.json
ADDED
|
@@ -0,0 +1,24 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"material_passport": {
|
| 3 |
+
"material_type": "experiment execution result",
|
| 4 |
+
"scientific_status": "post-review exploratory",
|
| 5 |
+
"created_utc": "2026-11-09T14:40:43.317363+00:00"
|
| 6 |
+
},
|
| 7 |
+
"status": "COMPLETED_FROM_SAVED_ARTIFACTS_AFTER_DIAGNOSTIC_DTYPE_CORRECTION",
|
| 8 |
+
"metric_rows": 13644,
|
| 9 |
+
"task_count": 57,
|
| 10 |
+
"original_fit_and_trace_seconds": 11.562392784981057,
|
| 11 |
+
"recovery_seconds": 9.258156107040122,
|
| 12 |
+
"maximum_replay_error": 6.661338147750939e-16,
|
| 13 |
+
"maximum_original_primary_expected_error": 2.220446049250313e-16,
|
| 14 |
+
"maximum_original_float32_query_charge_error": 1.4901161138336505e-09,
|
| 15 |
+
"gpu": "NVIDIA H200",
|
| 16 |
+
"device": "cuda:4",
|
| 17 |
+
"torch_version": "2.6.0+cu124",
|
| 18 |
+
"python_version": "3.13.11",
|
| 19 |
+
"peak_gpu_allocated_bytes": 63030784,
|
| 20 |
+
"peak_gpu_reserved_bytes": 81788928,
|
| 21 |
+
"script_sha256": "9739bca6e347271b91e4d7f9ffd4635b1bc7d58abe67d148f423b517fdea9d13",
|
| 22 |
+
"specification_sha256": "541dd0c1d4d7e242ee20c797bd9ef0928f7afdfffa4aacfeca5c3205cb212fab",
|
| 23 |
+
"original_results_and_gates_unchanged": true
|
| 24 |
+
}
|
revision_r1/external/implementation_correction.json
ADDED
|
@@ -0,0 +1,14 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"created_utc": "2026-11-09T14:40:20.960831+00:00",
|
| 3 |
+
"authorization": "Parent authorized routine numerical correction and artifact-only recovery after receiving failure report",
|
| 4 |
+
"failed_log_sha256": "9397cb2b52c3b201edd1b225736cef2f94060dcd026a2423e4765a7499b9ee49",
|
| 5 |
+
"original_specification_script_sha256": "18a499610df5927f78ae97af5bd4249b3c18beaef98ea39caecc5ad1303b7956",
|
| 6 |
+
"corrected_script_sha256": "9739bca6e347271b91e4d7f9ffd4635b1bc7d58abe67d148f423b517fdea9d13",
|
| 7 |
+
"failure": "Final equality assertion rejected float32 intermediate cost*integer_query_count in diagnostic batched replay utility; policy tables and traces unaffected",
|
| 8 |
+
"correction": "Multiply cost by queries.double(), matching original independent replay utility convention",
|
| 9 |
+
"estimators_rerun": 0,
|
| 10 |
+
"encoders_rerun": 0,
|
| 11 |
+
"planners_rerun": 0,
|
| 12 |
+
"policies_changed": 0,
|
| 13 |
+
"recovery": "Compute verified metrics and summaries from all 57 already saved model/policy/record-trace artifacts"
|
| 14 |
+
}
|
revision_r1/external/metrics.csv.gz
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:9813ff30348ad76cb8b672e022a5f3fe852b8f0ee13625e7a5799400cfc01120
|
| 3 |
+
size 1003440
|
revision_r1/external/overall.json
ADDED
|
The diff for this file is too large to render.
See raw diff
|
|
|
revision_r1/external/paired_contrasts.json
ADDED
|
@@ -0,0 +1,1338 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"scientific_status": "post-review exploratory; unadjusted descriptive intervals; no new confirmation",
|
| 3 |
+
"bootstrap_unit": "Target environment stratified by fixed dataset; tasks and seeds collapsed first",
|
| 4 |
+
"contrasts": [
|
| 5 |
+
{
|
| 6 |
+
"label": "joint_minus_posterior_target_64",
|
| 7 |
+
"field": "utility",
|
| 8 |
+
"overall": 0.0013605275735753635,
|
| 9 |
+
"by_dataset": {
|
| 10 |
+
"gas": -0.0026355068094112443,
|
| 11 |
+
"hydraulic": 0.005356561956561971
|
| 12 |
+
},
|
| 13 |
+
"by_environment": {
|
| 14 |
+
"gas:batch10": -0.006561130874027278,
|
| 15 |
+
"gas:batch6": 0.002410469681347993,
|
| 16 |
+
"gas:batch7": 0.010816431988927917,
|
| 17 |
+
"gas:batch8": -0.019843304843304852,
|
| 18 |
+
"gas:batch9": 0.0,
|
| 19 |
+
"hydraulic:context0=20": 0.0,
|
| 20 |
+
"hydraulic:context0=3": 0.0,
|
| 21 |
+
"hydraulic:context1=73": -0.0012910052910051577,
|
| 22 |
+
"hydraulic:context1=80": -0.0014615384615384401,
|
| 23 |
+
"hydraulic:context1=90": 0.029535353535353453
|
| 24 |
+
},
|
| 25 |
+
"tasks": 57,
|
| 26 |
+
"descriptive_paired_95_interval": [
|
| 27 |
+
-0.005097829104425849,
|
| 28 |
+
0.008910461444960618
|
| 29 |
+
],
|
| 30 |
+
"paired_fit_rows": 171,
|
| 31 |
+
"environment_resampling_only": true
|
| 32 |
+
},
|
| 33 |
+
{
|
| 34 |
+
"label": "target_minus_source_64_query_joint",
|
| 35 |
+
"field": "utility",
|
| 36 |
+
"overall": 0.35509999172132917,
|
| 37 |
+
"by_dataset": {
|
| 38 |
+
"gas": 0.473884680377134,
|
| 39 |
+
"hydraulic": 0.23631530306552437
|
| 40 |
+
},
|
| 41 |
+
"by_environment": {
|
| 42 |
+
"gas:batch10": 0.6827465125509808,
|
| 43 |
+
"gas:batch6": 0.5016533358279275,
|
| 44 |
+
"gas:batch7": 0.36589534226978987,
|
| 45 |
+
"gas:batch8": 0.5727454577218728,
|
| 46 |
+
"gas:batch9": 0.24638275351509886,
|
| 47 |
+
"hydraulic:context0=20": 0.019041394335511972,
|
| 48 |
+
"hydraulic:context0=3": 0.9375619275189165,
|
| 49 |
+
"hydraulic:context1=73": 0.06611111111111119,
|
| 50 |
+
"hydraulic:context1=80": 0.0742307692307691,
|
| 51 |
+
"hydraulic:context1=90": 0.084631313131313
|
| 52 |
+
},
|
| 53 |
+
"tasks": 57,
|
| 54 |
+
"descriptive_paired_95_interval": [
|
| 55 |
+
0.21639674731918707,
|
| 56 |
+
0.5385383430980829
|
| 57 |
+
],
|
| 58 |
+
"paired_fit_rows": 171,
|
| 59 |
+
"environment_resampling_only": true
|
| 60 |
+
},
|
| 61 |
+
{
|
| 62 |
+
"label": "target_minus_source_64_posterior_clusters",
|
| 63 |
+
"field": "utility",
|
| 64 |
+
"overall": 0.374339618686491,
|
| 65 |
+
"by_dataset": {
|
| 66 |
+
"gas": 0.49935518424102254,
|
| 67 |
+
"hydraulic": 0.24932405313195946
|
| 68 |
+
},
|
| 69 |
+
"by_environment": {
|
| 70 |
+
"gas:batch10": 0.6940641487563329,
|
| 71 |
+
"gas:batch6": 0.5654931501290714,
|
| 72 |
+
"gas:batch7": 0.36169786854927394,
|
| 73 |
+
"gas:batch8": 0.6189817502553353,
|
| 74 |
+
"gas:batch9": 0.25653900351509884,
|
| 75 |
+
"hydraulic:context0=20": -0.07336819172113289,
|
| 76 |
+
"hydraulic:context0=3": 1.074090003982477,
|
| 77 |
+
"hydraulic:context1=73": 0.07578306878306873,
|
| 78 |
+
"hydraulic:context1=80": 0.08100427350427342,
|
| 79 |
+
"hydraulic:context1=90": 0.08911111111111107
|
| 80 |
+
},
|
| 81 |
+
"tasks": 57,
|
| 82 |
+
"descriptive_paired_95_interval": [
|
| 83 |
+
0.21297235433542372,
|
| 84 |
+
0.5883940113046724
|
| 85 |
+
],
|
| 86 |
+
"paired_fit_rows": 171,
|
| 87 |
+
"environment_resampling_only": true
|
| 88 |
+
},
|
| 89 |
+
{
|
| 90 |
+
"label": "target_minus_source_64_identity",
|
| 91 |
+
"field": "utility",
|
| 92 |
+
"overall": 0.33073055684679387,
|
| 93 |
+
"by_dataset": {
|
| 94 |
+
"gas": 0.45630625630836985,
|
| 95 |
+
"hydraulic": 0.20515485738521785
|
| 96 |
+
},
|
| 97 |
+
"by_environment": {
|
| 98 |
+
"gas:batch10": 0.6827996391750814,
|
| 99 |
+
"gas:batch6": 0.4698706071131909,
|
| 100 |
+
"gas:batch7": 0.3612545191061068,
|
| 101 |
+
"gas:batch8": 0.521282790410149,
|
| 102 |
+
"gas:batch9": 0.24632372573732098,
|
| 103 |
+
"hydraulic:context0=20": -0.01192447349310094,
|
| 104 |
+
"hydraulic:context0=3": 0.9138932696136995,
|
| 105 |
+
"hydraulic:context1=73": 0.03314814814814826,
|
| 106 |
+
"hydraulic:context1=80": 0.026829059829059736,
|
| 107 |
+
"hydraulic:context1=90": 0.06382828282828272
|
| 108 |
+
},
|
| 109 |
+
"tasks": 57,
|
| 110 |
+
"descriptive_paired_95_interval": [
|
| 111 |
+
0.19386722406937337,
|
| 112 |
+
0.516573172190588
|
| 113 |
+
],
|
| 114 |
+
"paired_fit_rows": 171,
|
| 115 |
+
"environment_resampling_only": true
|
| 116 |
+
},
|
| 117 |
+
{
|
| 118 |
+
"label": "joint_minus_posterior_target_256",
|
| 119 |
+
"field": "utility",
|
| 120 |
+
"overall": 0.0001243752514988102,
|
| 121 |
+
"by_dataset": {
|
| 122 |
+
"gas": 0.0002487505029976204,
|
| 123 |
+
"hydraulic": 0.0
|
| 124 |
+
},
|
| 125 |
+
"by_environment": {
|
| 126 |
+
"gas:batch10": -0.001310365636289264,
|
| 127 |
+
"gas:batch6": 2.2926454800525085e-06,
|
| 128 |
+
"gas:batch7": 0.0020543244998020727,
|
| 129 |
+
"hydraulic:context0=3": 0.0
|
| 130 |
+
},
|
| 131 |
+
"tasks": 43,
|
| 132 |
+
"descriptive_paired_95_interval": [
|
| 133 |
+
-0.0006551828181446319,
|
| 134 |
+
0.0010271622499010364
|
| 135 |
+
],
|
| 136 |
+
"paired_fit_rows": 129,
|
| 137 |
+
"environment_resampling_only": true
|
| 138 |
+
},
|
| 139 |
+
{
|
| 140 |
+
"label": "target_minus_source_256_query_joint",
|
| 141 |
+
"field": "utility",
|
| 142 |
+
"overall": 0.6972687563149934,
|
| 143 |
+
"by_dataset": {
|
| 144 |
+
"gas": 0.41733799052723897,
|
| 145 |
+
"hydraulic": 0.9771995221027479
|
| 146 |
+
},
|
| 147 |
+
"by_environment": {
|
| 148 |
+
"gas:batch10": 0.6106930751014984,
|
| 149 |
+
"gas:batch6": 0.26386387194882227,
|
| 150 |
+
"gas:batch7": 0.37745702453139623,
|
| 151 |
+
"hydraulic:context0=3": 0.9771995221027479
|
| 152 |
+
},
|
| 153 |
+
"tasks": 43,
|
| 154 |
+
"descriptive_paired_95_interval": [
|
| 155 |
+
0.620531697025785,
|
| 156 |
+
0.7939462986021231
|
| 157 |
+
],
|
| 158 |
+
"paired_fit_rows": 129,
|
| 159 |
+
"environment_resampling_only": true
|
| 160 |
+
},
|
| 161 |
+
{
|
| 162 |
+
"label": "target_minus_source_256_posterior_clusters",
|
| 163 |
+
"field": "utility",
|
| 164 |
+
"overall": 0.7542922085489637,
|
| 165 |
+
"by_dataset": {
|
| 166 |
+
"gas": 0.45509855489163514,
|
| 167 |
+
"hydraulic": 1.0534858622062924
|
| 168 |
+
},
|
| 169 |
+
"by_environment": {
|
| 170 |
+
"gas:batch10": 0.6495552048920008,
|
| 171 |
+
"gas:batch6": 0.34364683092011294,
|
| 172 |
+
"gas:batch7": 0.37209362886279174,
|
| 173 |
+
"hydraulic:context0=3": 1.0534858622062924
|
| 174 |
+
},
|
| 175 |
+
"tasks": 43,
|
| 176 |
+
"descriptive_paired_95_interval": [
|
| 177 |
+
0.6985663465632026,
|
| 178 |
+
0.8515205335491466
|
| 179 |
+
],
|
| 180 |
+
"paired_fit_rows": 129,
|
| 181 |
+
"environment_resampling_only": true
|
| 182 |
+
},
|
| 183 |
+
{
|
| 184 |
+
"label": "target_minus_source_256_identity",
|
| 185 |
+
"field": "utility",
|
| 186 |
+
"overall": 0.6795324016704808,
|
| 187 |
+
"by_dataset": {
|
| 188 |
+
"gas": 0.41381390728361395,
|
| 189 |
+
"hydraulic": 0.9452508960573477
|
| 190 |
+
},
|
| 191 |
+
"by_environment": {
|
| 192 |
+
"gas:batch10": 0.5945331914895884,
|
| 193 |
+
"gas:batch6": 0.2842302213368665,
|
| 194 |
+
"gas:batch7": 0.362678309024387,
|
| 195 |
+
"hydraulic:context0=3": 0.9452508960573477
|
| 196 |
+
},
|
| 197 |
+
"tasks": 43,
|
| 198 |
+
"descriptive_paired_95_interval": [
|
| 199 |
+
0.6147405586971071,
|
| 200 |
+
0.769892043773468
|
| 201 |
+
],
|
| 202 |
+
"paired_fit_rows": 129,
|
| 203 |
+
"environment_resampling_only": true
|
| 204 |
+
},
|
| 205 |
+
{
|
| 206 |
+
"label": "joint_minus_posterior_target_all",
|
| 207 |
+
"field": "utility",
|
| 208 |
+
"overall": -0.0022206393342042033,
|
| 209 |
+
"by_dataset": {
|
| 210 |
+
"gas": -0.0042490486761783755,
|
| 211 |
+
"hydraulic": -0.00019222999223003078
|
| 212 |
+
},
|
| 213 |
+
"by_environment": {
|
| 214 |
+
"gas:batch10": 0.00128551606023862,
|
| 215 |
+
"gas:batch6": -0.00404940473759051,
|
| 216 |
+
"gas:batch7": 0.004146850424665106,
|
| 217 |
+
"gas:batch8": -0.022628205128205094,
|
| 218 |
+
"gas:batch9": 0.0,
|
| 219 |
+
"hydraulic:context0=20": 0.0,
|
| 220 |
+
"hydraulic:context0=3": 0.0,
|
| 221 |
+
"hydraulic:context1=73": -0.012555555555555542,
|
| 222 |
+
"hydraulic:context1=80": 0.011230769230769155,
|
| 223 |
+
"hydraulic:context1=90": 0.00036363636363623275
|
| 224 |
+
},
|
| 225 |
+
"tasks": 57,
|
| 226 |
+
"descriptive_paired_95_interval": [
|
| 227 |
+
-0.007971387996951206,
|
| 228 |
+
0.0026320522784431004
|
| 229 |
+
],
|
| 230 |
+
"paired_fit_rows": 171,
|
| 231 |
+
"environment_resampling_only": true
|
| 232 |
+
},
|
| 233 |
+
{
|
| 234 |
+
"label": "target_minus_source_all_query_joint",
|
| 235 |
+
"field": "utility",
|
| 236 |
+
"overall": 0.25648045155959887,
|
| 237 |
+
"by_dataset": {
|
| 238 |
+
"gas": 0.4477259480328195,
|
| 239 |
+
"hydraulic": 0.06523495508637825
|
| 240 |
+
},
|
| 241 |
+
"by_environment": {
|
| 242 |
+
"gas:batch10": 0.6156761617886918,
|
| 243 |
+
"gas:batch6": 0.23057048031017344,
|
| 244 |
+
"gas:batch7": 0.40794457305418264,
|
| 245 |
+
"gas:batch8": 0.5168382115787776,
|
| 246 |
+
"gas:batch9": 0.46760031343227215,
|
| 247 |
+
"hydraulic:context0=20": -0.5321677559912854,
|
| 248 |
+
"hydraulic:context0=3": 0.8422102747909199,
|
| 249 |
+
"hydraulic:context1=73": 0.01523015873015876,
|
| 250 |
+
"hydraulic:context1=80": 0.03553846153846152,
|
| 251 |
+
"hydraulic:context1=90": -0.034636363636363576
|
| 252 |
+
},
|
| 253 |
+
"tasks": 57,
|
| 254 |
+
"descriptive_paired_95_interval": [
|
| 255 |
+
0.07132033952671643,
|
| 256 |
+
0.4667324712229042
|
| 257 |
+
],
|
| 258 |
+
"paired_fit_rows": 171,
|
| 259 |
+
"environment_resampling_only": true
|
| 260 |
+
},
|
| 261 |
+
{
|
| 262 |
+
"label": "target_minus_source_all_posterior_clusters",
|
| 263 |
+
"field": "utility",
|
| 264 |
+
"overall": 0.26156087109101617,
|
| 265 |
+
"by_dataset": {
|
| 266 |
+
"gas": 0.4580375849238713,
|
| 267 |
+
"hydraulic": 0.06508415725816101
|
| 268 |
+
},
|
| 269 |
+
"by_environment": {
|
| 270 |
+
"gas:batch10": 0.6322578156867729,
|
| 271 |
+
"gas:batch6": 0.2463184991774343,
|
| 272 |
+
"gas:batch7": 0.40254274286375746,
|
| 273 |
+
"gas:batch8": 0.5360102201257863,
|
| 274 |
+
"gas:batch9": 0.4730586467656055,
|
| 275 |
+
"hydraulic:context0=20": -0.5141677559912854,
|
| 276 |
+
"hydraulic:context0=3": 0.7874157706093189,
|
| 277 |
+
"hydraulic:context1=73": 0.06619841269841265,
|
| 278 |
+
"hydraulic:context1=80": 0.04314102564102562,
|
| 279 |
+
"hydraulic:context1=90": -0.0571666666666667
|
| 280 |
+
},
|
| 281 |
+
"tasks": 57,
|
| 282 |
+
"descriptive_paired_95_interval": [
|
| 283 |
+
0.08312789207608386,
|
| 284 |
+
0.45910042894694186
|
| 285 |
+
],
|
| 286 |
+
"paired_fit_rows": 171,
|
| 287 |
+
"environment_resampling_only": true
|
| 288 |
+
},
|
| 289 |
+
{
|
| 290 |
+
"label": "target_minus_source_all_identity",
|
| 291 |
+
"field": "utility",
|
| 292 |
+
"overall": 0.20957963182712874,
|
| 293 |
+
"by_dataset": {
|
| 294 |
+
"gas": 0.4421509491140644,
|
| 295 |
+
"hydraulic": -0.02299168545980692
|
| 296 |
+
},
|
| 297 |
+
"by_environment": {
|
| 298 |
+
"gas:batch10": 0.6093241353610174,
|
| 299 |
+
"gas:batch6": 0.23886951796836567,
|
| 300 |
+
"gas:batch7": 0.38617590540115165,
|
| 301 |
+
"gas:batch8": 0.508784873407515,
|
| 302 |
+
"gas:batch9": 0.46760031343227215,
|
| 303 |
+
"hydraulic:context0=20": -0.6987799564270153,
|
| 304 |
+
"hydraulic:context0=3": 0.5693249701314217,
|
| 305 |
+
"hydraulic:context1=73": 0.036039682539682705,
|
| 306 |
+
"hydraulic:context1=80": 0.03548717948717939,
|
| 307 |
+
"hydraulic:context1=90": -0.0570303030303031
|
| 308 |
+
},
|
| 309 |
+
"tasks": 57,
|
| 310 |
+
"descriptive_paired_95_interval": [
|
| 311 |
+
0.02266356074770446,
|
| 312 |
+
0.3862194467643001
|
| 313 |
+
],
|
| 314 |
+
"paired_fit_rows": 171,
|
| 315 |
+
"environment_resampling_only": true
|
| 316 |
+
},
|
| 317 |
+
{
|
| 318 |
+
"label": "target_256_minus_64_query_joint",
|
| 319 |
+
"field": "utility",
|
| 320 |
+
"overall": 0.09011151779716105,
|
| 321 |
+
"by_dataset": {
|
| 322 |
+
"gas": 0.06971407502084541,
|
| 323 |
+
"hydraulic": 0.11050896057347669
|
| 324 |
+
},
|
| 325 |
+
"by_environment": {
|
| 326 |
+
"gas:batch10": 0.0756806253273861,
|
| 327 |
+
"gas:batch6": 0.09056258626038896,
|
| 328 |
+
"gas:batch7": 0.04289901347476115,
|
| 329 |
+
"hydraulic:context0=3": 0.11050896057347669
|
| 330 |
+
},
|
| 331 |
+
"tasks": 43,
|
| 332 |
+
"descriptive_paired_95_interval": [
|
| 333 |
+
0.07670398702411892,
|
| 334 |
+
0.10053577341693282
|
| 335 |
+
],
|
| 336 |
+
"paired_fit_rows": 129,
|
| 337 |
+
"environment_resampling_only": true
|
| 338 |
+
},
|
| 339 |
+
{
|
| 340 |
+
"label": "target_256_minus_64_query_joint",
|
| 341 |
+
"field": "noquery_utility",
|
| 342 |
+
"overall": 0.06251360306551022,
|
| 343 |
+
"by_dataset": {
|
| 344 |
+
"gas": 0.05334261831740041,
|
| 345 |
+
"hydraulic": 0.07168458781362004
|
| 346 |
+
},
|
| 347 |
+
"by_environment": {
|
| 348 |
+
"gas:batch10": 0.08115054963730205,
|
| 349 |
+
"gas:batch6": 0.0496728948353739,
|
| 350 |
+
"gas:batch7": 0.029204410479525288,
|
| 351 |
+
"hydraulic:context0=3": 0.07168458781362004
|
| 352 |
+
},
|
| 353 |
+
"tasks": 43,
|
| 354 |
+
"descriptive_paired_95_interval": [
|
| 355 |
+
0.05044449914657266,
|
| 356 |
+
0.07641756872546104
|
| 357 |
+
],
|
| 358 |
+
"paired_fit_rows": 129,
|
| 359 |
+
"environment_resampling_only": true
|
| 360 |
+
},
|
| 361 |
+
{
|
| 362 |
+
"label": "target_256_minus_64_query_joint",
|
| 363 |
+
"field": "realized_query_increment",
|
| 364 |
+
"overall": 0.02759791473165081,
|
| 365 |
+
"by_dataset": {
|
| 366 |
+
"gas": 0.016371456703444996,
|
| 367 |
+
"hydraulic": 0.038824372759856625
|
| 368 |
+
},
|
| 369 |
+
"by_environment": {
|
| 370 |
+
"gas:batch10": -0.00546992430991594,
|
| 371 |
+
"gas:batch6": 0.040889691425015064,
|
| 372 |
+
"gas:batch7": 0.013694602995235868,
|
| 373 |
+
"hydraulic:context0=3": 0.038824372759856625
|
| 374 |
+
},
|
| 375 |
+
"tasks": 43,
|
| 376 |
+
"descriptive_paired_95_interval": [
|
| 377 |
+
0.016677224224970342,
|
| 378 |
+
0.039857032092435844
|
| 379 |
+
],
|
| 380 |
+
"paired_fit_rows": 129,
|
| 381 |
+
"environment_resampling_only": true
|
| 382 |
+
},
|
| 383 |
+
{
|
| 384 |
+
"label": "target_256_minus_64_query_joint",
|
| 385 |
+
"field": "joint_conditional_log_loss",
|
| 386 |
+
"overall": -0.1953191397999047,
|
| 387 |
+
"by_dataset": {
|
| 388 |
+
"gas": -0.2657428747277448,
|
| 389 |
+
"hydraulic": -0.12489540487206459
|
| 390 |
+
},
|
| 391 |
+
"by_environment": {
|
| 392 |
+
"gas:batch10": -0.3102340868093479,
|
| 393 |
+
"gas:batch6": -0.21311841464661282,
|
| 394 |
+
"gas:batch7": -0.27387612272727374,
|
| 395 |
+
"hydraulic:context0=3": -0.12489540487206459
|
| 396 |
+
},
|
| 397 |
+
"tasks": 43,
|
| 398 |
+
"descriptive_paired_95_interval": [
|
| 399 |
+
-0.21756474584070623,
|
| 400 |
+
-0.16900690975933869
|
| 401 |
+
],
|
| 402 |
+
"paired_fit_rows": 129,
|
| 403 |
+
"environment_resampling_only": true
|
| 404 |
+
},
|
| 405 |
+
{
|
| 406 |
+
"label": "target_all_minus_64_query_joint",
|
| 407 |
+
"field": "utility",
|
| 408 |
+
"overall": 0.038157311614910504,
|
| 409 |
+
"by_dataset": {
|
| 410 |
+
"gas": 0.05676291617893625,
|
| 411 |
+
"hydraulic": 0.019551707050884753
|
| 412 |
+
},
|
| 413 |
+
"by_environment": {
|
| 414 |
+
"gas:batch10": 0.11750666905912108,
|
| 415 |
+
"gas:batch6": 0.10113160947997442,
|
| 416 |
+
"gas:batch7": 0.07350602580668636,
|
| 417 |
+
"gas:batch8": -0.008274740633231198,
|
| 418 |
+
"gas:batch9": -5.498281786941268e-05,
|
| 419 |
+
"hydraulic:context0=20": -0.14266521423384168,
|
| 420 |
+
"hydraulic:context0=3": 0.11050896057347669,
|
| 421 |
+
"hydraulic:context1=73": -0.0011481481481481943,
|
| 422 |
+
"hydraulic:context1=80": 0.0032649572649571614,
|
| 423 |
+
"hydraulic:context1=90": 0.1277979797979798
|
| 424 |
+
},
|
| 425 |
+
"tasks": 57,
|
| 426 |
+
"descriptive_paired_95_interval": [
|
| 427 |
+
-0.01084189535725685,
|
| 428 |
+
0.0850130341338505
|
| 429 |
+
],
|
| 430 |
+
"paired_fit_rows": 171,
|
| 431 |
+
"environment_resampling_only": true
|
| 432 |
+
},
|
| 433 |
+
{
|
| 434 |
+
"label": "target_all_minus_64_query_joint",
|
| 435 |
+
"field": "noquery_utility",
|
| 436 |
+
"overall": 0.04251419582047826,
|
| 437 |
+
"by_dataset": {
|
| 438 |
+
"gas": 0.04728120243266674,
|
| 439 |
+
"hydraulic": 0.03774718920828978
|
| 440 |
+
},
|
| 441 |
+
"by_environment": {
|
| 442 |
+
"gas:batch10": 0.11361625735686477,
|
| 443 |
+
"gas:batch6": 0.056744070821711144,
|
| 444 |
+
"gas:batch7": 0.05963542757450136,
|
| 445 |
+
"gas:batch8": 0.0064102564102564005,
|
| 446 |
+
"gas:batch9": 0.0,
|
| 447 |
+
"hydraulic:context0=20": -0.1305010893246187,
|
| 448 |
+
"hydraulic:context0=3": 0.07168458781362004,
|
| 449 |
+
"hydraulic:context1=73": -1.4802973661668753e-16,
|
| 450 |
+
"hydraulic:context1=80": 0.005128205128205295,
|
| 451 |
+
"hydraulic:context1=90": 0.24242424242424243
|
| 452 |
+
},
|
| 453 |
+
"tasks": 57,
|
| 454 |
+
"descriptive_paired_95_interval": [
|
| 455 |
+
-0.009599849722433367,
|
| 456 |
+
0.10225396410031547
|
| 457 |
+
],
|
| 458 |
+
"paired_fit_rows": 171,
|
| 459 |
+
"environment_resampling_only": true
|
| 460 |
+
},
|
| 461 |
+
{
|
| 462 |
+
"label": "target_all_minus_64_query_joint",
|
| 463 |
+
"field": "realized_query_increment",
|
| 464 |
+
"overall": -0.004356884205567757,
|
| 465 |
+
"by_dataset": {
|
| 466 |
+
"gas": 0.00948171374626951,
|
| 467 |
+
"hydraulic": -0.018195482157405025
|
| 468 |
+
},
|
| 469 |
+
"by_environment": {
|
| 470 |
+
"gas:batch10": 0.0038904117022563103,
|
| 471 |
+
"gas:batch6": 0.04438753865826327,
|
| 472 |
+
"gas:batch7": 0.013870598232184988,
|
| 473 |
+
"gas:batch8": -0.014684997043487599,
|
| 474 |
+
"gas:batch9": -5.498281786941268e-05,
|
| 475 |
+
"hydraulic:context0=20": -0.012164124909222947,
|
| 476 |
+
"hydraulic:context0=3": 0.038824372759856625,
|
| 477 |
+
"hydraulic:context1=73": -0.0011481481481480464,
|
| 478 |
+
"hydraulic:context1=80": -0.0018632478632481335,
|
| 479 |
+
"hydraulic:context1=90": -0.11462626262626262
|
| 480 |
+
},
|
| 481 |
+
"tasks": 57,
|
| 482 |
+
"descriptive_paired_95_interval": [
|
| 483 |
+
-0.030791613659059625,
|
| 484 |
+
0.01679233011427237
|
| 485 |
+
],
|
| 486 |
+
"paired_fit_rows": 171,
|
| 487 |
+
"environment_resampling_only": true
|
| 488 |
+
},
|
| 489 |
+
{
|
| 490 |
+
"label": "target_all_minus_64_query_joint",
|
| 491 |
+
"field": "joint_conditional_log_loss",
|
| 492 |
+
"overall": -0.19687071580060886,
|
| 493 |
+
"by_dataset": {
|
| 494 |
+
"gas": -0.23411345652960858,
|
| 495 |
+
"hydraulic": -0.15962797507160914
|
| 496 |
+
},
|
| 497 |
+
"by_environment": {
|
| 498 |
+
"gas:batch10": -0.44018328456321953,
|
| 499 |
+
"gas:batch6": -0.2833204085585301,
|
| 500 |
+
"gas:batch7": -0.4036770926283303,
|
| 501 |
+
"gas:batch8": -0.02071081915566557,
|
| 502 |
+
"gas:batch9": -0.022675677742297333,
|
| 503 |
+
"hydraulic:context0=20": -0.1815063570268768,
|
| 504 |
+
"hydraulic:context0=3": -0.1292211545357918,
|
| 505 |
+
"hydraulic:context1=73": -0.08545253886246955,
|
| 506 |
+
"hydraulic:context1=80": -0.13971554729893834,
|
| 507 |
+
"hydraulic:context1=90": -0.2622442776339691
|
| 508 |
+
},
|
| 509 |
+
"tasks": 57,
|
| 510 |
+
"descriptive_paired_95_interval": [
|
| 511 |
+
-0.2781864450355927,
|
| 512 |
+
-0.1120529765173951
|
| 513 |
+
],
|
| 514 |
+
"paired_fit_rows": 171,
|
| 515 |
+
"environment_resampling_only": true
|
| 516 |
+
},
|
| 517 |
+
{
|
| 518 |
+
"label": "target_256_minus_64_posterior_clusters",
|
| 519 |
+
"field": "utility",
|
| 520 |
+
"overall": 0.09109810434503701,
|
| 521 |
+
"by_dataset": {
|
| 522 |
+
"gas": 0.07168724811659732,
|
| 523 |
+
"hydraulic": 0.11050896057347669
|
| 524 |
+
},
|
| 525 |
+
"by_environment": {
|
| 526 |
+
"gas:batch10": 0.07042986008964809,
|
| 527 |
+
"gas:batch6": 0.09297076329625689,
|
| 528 |
+
"gas:batch7": 0.051661120963886996,
|
| 529 |
+
"hydraulic:context0=3": 0.11050896057347669
|
| 530 |
+
},
|
| 531 |
+
"tasks": 43,
|
| 532 |
+
"descriptive_paired_95_interval": [
|
| 533 |
+
0.08108504076868184,
|
| 534 |
+
0.10173986193486678
|
| 535 |
+
],
|
| 536 |
+
"paired_fit_rows": 129,
|
| 537 |
+
"environment_resampling_only": true
|
| 538 |
+
},
|
| 539 |
+
{
|
| 540 |
+
"label": "target_256_minus_64_posterior_clusters",
|
| 541 |
+
"field": "noquery_utility",
|
| 542 |
+
"overall": 0.06468211447240929,
|
| 543 |
+
"by_dataset": {
|
| 544 |
+
"gas": 0.05767964113119853,
|
| 545 |
+
"hydraulic": 0.07168458781362004
|
| 546 |
+
},
|
| 547 |
+
"by_environment": {
|
| 548 |
+
"gas:batch10": 0.08343377230673357,
|
| 549 |
+
"gas:batch6": 0.05454737389719194,
|
| 550 |
+
"gas:batch7": 0.03505777718967007,
|
| 551 |
+
"hydraulic:context0=3": 0.07168458781362004
|
| 552 |
+
},
|
| 553 |
+
"tasks": 43,
|
| 554 |
+
"descriptive_paired_95_interval": [
|
| 555 |
+
0.053371182501645055,
|
| 556 |
+
0.07755918006017681
|
| 557 |
+
],
|
| 558 |
+
"paired_fit_rows": 129,
|
| 559 |
+
"environment_resampling_only": true
|
| 560 |
+
},
|
| 561 |
+
{
|
| 562 |
+
"label": "target_256_minus_64_posterior_clusters",
|
| 563 |
+
"field": "realized_query_increment",
|
| 564 |
+
"overall": 0.026415989872627714,
|
| 565 |
+
"by_dataset": {
|
| 566 |
+
"gas": 0.014007606985398803,
|
| 567 |
+
"hydraulic": 0.038824372759856625
|
| 568 |
+
},
|
| 569 |
+
"by_environment": {
|
| 570 |
+
"gas:batch10": -0.01300391221708548,
|
| 571 |
+
"gas:batch6": 0.038423389399064964,
|
| 572 |
+
"gas:batch7": 0.016603343774216926,
|
| 573 |
+
"hydraulic:context0=3": 0.038824372759856625
|
| 574 |
+
},
|
| 575 |
+
"tasks": 43,
|
| 576 |
+
"descriptive_paired_95_interval": [
|
| 577 |
+
0.012910230271385574,
|
| 578 |
+
0.0386238810794608
|
| 579 |
+
],
|
| 580 |
+
"paired_fit_rows": 129,
|
| 581 |
+
"environment_resampling_only": true
|
| 582 |
+
},
|
| 583 |
+
{
|
| 584 |
+
"label": "target_256_minus_64_posterior_clusters",
|
| 585 |
+
"field": "joint_conditional_log_loss",
|
| 586 |
+
"overall": -0.1776871678007718,
|
| 587 |
+
"by_dataset": {
|
| 588 |
+
"gas": -0.23048284772806546,
|
| 589 |
+
"hydraulic": -0.1248914878734781
|
| 590 |
+
},
|
| 591 |
+
"by_environment": {
|
| 592 |
+
"gas:batch10": -0.2824511247837857,
|
| 593 |
+
"gas:batch6": -0.1741531575627679,
|
| 594 |
+
"gas:batch7": -0.23484426083764282,
|
| 595 |
+
"hydraulic:context0=3": -0.1248914878734781
|
| 596 |
+
},
|
| 597 |
+
"tasks": 43,
|
| 598 |
+
"descriptive_paired_95_interval": [
|
| 599 |
+
-0.20367130632863187,
|
| 600 |
+
-0.149522322718123
|
| 601 |
+
],
|
| 602 |
+
"paired_fit_rows": 129,
|
| 603 |
+
"environment_resampling_only": true
|
| 604 |
+
},
|
| 605 |
+
{
|
| 606 |
+
"label": "target_all_minus_64_posterior_clusters",
|
| 607 |
+
"field": "utility",
|
| 608 |
+
"overall": 0.04173847852269007,
|
| 609 |
+
"by_dataset": {
|
| 610 |
+
"gas": 0.05837645804570337,
|
| 611 |
+
"hydraulic": 0.025100498999676757
|
| 612 |
+
},
|
| 613 |
+
"by_environment": {
|
| 614 |
+
"gas:batch10": 0.10966002212485516,
|
| 615 |
+
"gas:batch6": 0.1075914838989129,
|
| 616 |
+
"gas:batch7": 0.08017560737094916,
|
| 617 |
+
"gas:batch8": -0.0054898403483309545,
|
| 618 |
+
"gas:batch9": -5.498281786941268e-05,
|
| 619 |
+
"hydraulic:context0=20": -0.14266521423384168,
|
| 620 |
+
"hydraulic:context0=3": 0.11050896057347669,
|
| 621 |
+
"hydraulic:context1=73": 0.01011640211640219,
|
| 622 |
+
"hydraulic:context1=80": -0.009427350427350434,
|
| 623 |
+
"hydraulic:context1=90": 0.156969696969697
|
| 624 |
+
},
|
| 625 |
+
"tasks": 57,
|
| 626 |
+
"descriptive_paired_95_interval": [
|
| 627 |
+
-0.009828601087338618,
|
| 628 |
+
0.09135077312892757
|
| 629 |
+
],
|
| 630 |
+
"paired_fit_rows": 171,
|
| 631 |
+
"environment_resampling_only": true
|
| 632 |
+
},
|
| 633 |
+
{
|
| 634 |
+
"label": "target_all_minus_64_posterior_clusters",
|
| 635 |
+
"field": "noquery_utility",
|
| 636 |
+
"overall": 0.03433698659897971,
|
| 637 |
+
"by_dataset": {
|
| 638 |
+
"gas": 0.05032072338360901,
|
| 639 |
+
"hydraulic": 0.018353249814350416
|
| 640 |
+
},
|
| 641 |
+
"by_environment": {
|
| 642 |
+
"gas:batch10": 0.11928082949136254,
|
| 643 |
+
"gas:batch6": 0.061912046117019635,
|
| 644 |
+
"gas:batch7": 0.07041074130966289,
|
| 645 |
+
"gas:batch8": -1.2335811384723961e-17,
|
| 646 |
+
"gas:batch9": 0.0,
|
| 647 |
+
"hydraulic:context0=20": -0.1305010893246187,
|
| 648 |
+
"hydraulic:context0=3": 0.07168458781362004,
|
| 649 |
+
"hydraulic:context1=73": -7.401486830834377e-17,
|
| 650 |
+
"hydraulic:context1=80": 0.005128205128205184,
|
| 651 |
+
"hydraulic:context1=90": 0.14545454545454564
|
| 652 |
+
},
|
| 653 |
+
"tasks": 57,
|
| 654 |
+
"descriptive_paired_95_interval": [
|
| 655 |
+
-0.011377864439774938,
|
| 656 |
+
0.07841280648218549
|
| 657 |
+
],
|
| 658 |
+
"paired_fit_rows": 171,
|
| 659 |
+
"environment_resampling_only": true
|
| 660 |
+
},
|
| 661 |
+
{
|
| 662 |
+
"label": "target_all_minus_64_posterior_clusters",
|
| 663 |
+
"field": "realized_query_increment",
|
| 664 |
+
"overall": 0.007401491923710351,
|
| 665 |
+
"by_dataset": {
|
| 666 |
+
"gas": 0.008055734662094364,
|
| 667 |
+
"hydraulic": 0.006747249185326339
|
| 668 |
+
},
|
| 669 |
+
"by_environment": {
|
| 670 |
+
"gas:batch10": -0.009620807366507366,
|
| 671 |
+
"gas:batch6": 0.045679437781893276,
|
| 672 |
+
"gas:batch7": 0.009764866061286272,
|
| 673 |
+
"gas:batch8": -0.005489840348330943,
|
| 674 |
+
"gas:batch9": -5.498281786941268e-05,
|
| 675 |
+
"hydraulic:context0=20": -0.012164124909222947,
|
| 676 |
+
"hydraulic:context0=3": 0.038824372759856625,
|
| 677 |
+
"hydraulic:context1=73": 0.010116402116402265,
|
| 678 |
+
"hydraulic:context1=80": -0.014555555555555618,
|
| 679 |
+
"hydraulic:context1=90": 0.011515151515151367
|
| 680 |
+
},
|
| 681 |
+
"tasks": 57,
|
| 682 |
+
"descriptive_paired_95_interval": [
|
| 683 |
+
-0.0038314389064033304,
|
| 684 |
+
0.020036524552761063
|
| 685 |
+
],
|
| 686 |
+
"paired_fit_rows": 171,
|
| 687 |
+
"environment_resampling_only": true
|
| 688 |
+
},
|
| 689 |
+
{
|
| 690 |
+
"label": "target_all_minus_64_posterior_clusters",
|
| 691 |
+
"field": "joint_conditional_log_loss",
|
| 692 |
+
"overall": -0.1681942885407447,
|
| 693 |
+
"by_dataset": {
|
| 694 |
+
"gas": -0.2319239590935684,
|
| 695 |
+
"hydraulic": -0.10446461798792102
|
| 696 |
+
},
|
| 697 |
+
"by_environment": {
|
| 698 |
+
"gas:batch10": -0.4418217882384215,
|
| 699 |
+
"gas:batch6": -0.29143101306951796,
|
| 700 |
+
"gas:batch7": -0.3907604992191855,
|
| 701 |
+
"gas:batch8": -0.012722694648025131,
|
| 702 |
+
"gas:batch9": -0.02288380029269186,
|
| 703 |
+
"hydraulic:context0=20": -0.1815063570268768,
|
| 704 |
+
"hydraulic:context0=3": -0.12923307115226043,
|
| 705 |
+
"hydraulic:context1=73": -0.04543127935882071,
|
| 706 |
+
"hydraulic:context1=80": 0.0002010261398282346,
|
| 707 |
+
"hydraulic:context1=90": -0.1663534085414754
|
| 708 |
+
},
|
| 709 |
+
"tasks": 57,
|
| 710 |
+
"descriptive_paired_95_interval": [
|
| 711 |
+
-0.25084335229608745,
|
| 712 |
+
-0.08113264067417841
|
| 713 |
+
],
|
| 714 |
+
"paired_fit_rows": 171,
|
| 715 |
+
"environment_resampling_only": true
|
| 716 |
+
},
|
| 717 |
+
{
|
| 718 |
+
"label": "target_256_minus_64_identity",
|
| 719 |
+
"field": "utility",
|
| 720 |
+
"overall": 0.09048828332602109,
|
| 721 |
+
"by_dataset": {
|
| 722 |
+
"gas": 0.07046760607856549,
|
| 723 |
+
"hydraulic": 0.11050896057347669
|
| 724 |
+
},
|
| 725 |
+
"by_environment": {
|
| 726 |
+
"gas:batch10": 0.07419248136066033,
|
| 727 |
+
"gas:batch6": 0.0904448025664073,
|
| 728 |
+
"gas:batch7": 0.04676553430862883,
|
| 729 |
+
"hydraulic:context0=3": 0.11050896057347669
|
| 730 |
+
},
|
| 731 |
+
"tasks": 43,
|
| 732 |
+
"descriptive_paired_95_interval": [
|
| 733 |
+
0.07863724744105276,
|
| 734 |
+
0.100476881569942
|
| 735 |
+
],
|
| 736 |
+
"paired_fit_rows": 129,
|
| 737 |
+
"environment_resampling_only": true
|
| 738 |
+
},
|
| 739 |
+
{
|
| 740 |
+
"label": "target_256_minus_64_identity",
|
| 741 |
+
"field": "noquery_utility",
|
| 742 |
+
"overall": 0.0637826754644584,
|
| 743 |
+
"by_dataset": {
|
| 744 |
+
"gas": 0.055880763115296744,
|
| 745 |
+
"hydraulic": 0.07168458781362005
|
| 746 |
+
},
|
| 747 |
+
"by_environment": {
|
| 748 |
+
"gas:batch10": 0.08491540788816963,
|
| 749 |
+
"gas:batch6": 0.0473874535281433,
|
| 750 |
+
"gas:batch7": 0.0353394279295773,
|
| 751 |
+
"hydraulic:context0=3": 0.07168458781362005
|
| 752 |
+
},
|
| 753 |
+
"tasks": 43,
|
| 754 |
+
"descriptive_paired_95_interval": [
|
| 755 |
+
0.053512007871598676,
|
| 756 |
+
0.07829999785089484
|
| 757 |
+
],
|
| 758 |
+
"paired_fit_rows": 129,
|
| 759 |
+
"environment_resampling_only": true
|
| 760 |
+
},
|
| 761 |
+
{
|
| 762 |
+
"label": "target_256_minus_64_identity",
|
| 763 |
+
"field": "realized_query_increment",
|
| 764 |
+
"overall": 0.026705607861562685,
|
| 765 |
+
"by_dataset": {
|
| 766 |
+
"gas": 0.01458684296326875,
|
| 767 |
+
"hydraulic": 0.03882437275985662
|
| 768 |
+
},
|
| 769 |
+
"by_environment": {
|
| 770 |
+
"gas:batch10": -0.010722926527509303,
|
| 771 |
+
"gas:batch6": 0.04305734903826401,
|
| 772 |
+
"gas:batch7": 0.011426106379051539,
|
| 773 |
+
"hydraulic:context0=3": 0.03882437275985662
|
| 774 |
+
},
|
| 775 |
+
"tasks": 43,
|
| 776 |
+
"descriptive_paired_95_interval": [
|
| 777 |
+
0.014050723116173659,
|
| 778 |
+
0.04094086089906031
|
| 779 |
+
],
|
| 780 |
+
"paired_fit_rows": 129,
|
| 781 |
+
"environment_resampling_only": true
|
| 782 |
+
},
|
| 783 |
+
{
|
| 784 |
+
"label": "target_256_minus_64_identity",
|
| 785 |
+
"field": "joint_conditional_log_loss",
|
| 786 |
+
"overall": -0.19112487728113603,
|
| 787 |
+
"by_dataset": {
|
| 788 |
+
"gas": -0.2573433341697351,
|
| 789 |
+
"hydraulic": -0.12490642039253695
|
| 790 |
+
},
|
| 791 |
+
"by_environment": {
|
| 792 |
+
"gas:batch10": -0.30373009969425085,
|
| 793 |
+
"gas:batch6": -0.20457846206949787,
|
| 794 |
+
"gas:batch7": -0.26372144074545656,
|
| 795 |
+
"hydraulic:context0=3": -0.12490642039253695
|
| 796 |
+
},
|
| 797 |
+
"tasks": 43,
|
| 798 |
+
"descriptive_paired_95_interval": [
|
| 799 |
+
-0.21431826004339388,
|
| 800 |
+
-0.1647424412310174
|
| 801 |
+
],
|
| 802 |
+
"paired_fit_rows": 129,
|
| 803 |
+
"environment_resampling_only": true
|
| 804 |
+
},
|
| 805 |
+
{
|
| 806 |
+
"label": "target_all_minus_64_identity",
|
| 807 |
+
"field": "utility",
|
| 808 |
+
"overall": 0.03613808594050513,
|
| 809 |
+
"by_dataset": {
|
| 810 |
+
"gas": 0.05583580164146228,
|
| 811 |
+
"hydraulic": 0.01644037023954799
|
| 812 |
+
},
|
| 813 |
+
"by_environment": {
|
| 814 |
+
"gas:batch10": 0.12010778243557424,
|
| 815 |
+
"gas:batch6": 0.10662597263758453,
|
| 816 |
+
"gas:batch7": 0.0705773270126037,
|
| 817 |
+
"gas:batch8": -0.018077091060581647,
|
| 818 |
+
"gas:batch9": -5.498281786941268e-05,
|
| 819 |
+
"hydraulic:context0=20": -0.14266521423384168,
|
| 820 |
+
"hydraulic:context0=3": 0.11050896057347669,
|
| 821 |
+
"hydraulic:context1=73": -0.002232804232804253,
|
| 822 |
+
"hydraulic:context1=80": -0.009388888888888891,
|
| 823 |
+
"hydraulic:context1=90": 0.12597979797979808
|
| 824 |
+
},
|
| 825 |
+
"tasks": 57,
|
| 826 |
+
"descriptive_paired_95_interval": [
|
| 827 |
+
-0.013789338141264412,
|
| 828 |
+
0.08366247811554169
|
| 829 |
+
],
|
| 830 |
+
"paired_fit_rows": 171,
|
| 831 |
+
"environment_resampling_only": true
|
| 832 |
+
},
|
| 833 |
+
{
|
| 834 |
+
"label": "target_all_minus_64_identity",
|
| 835 |
+
"field": "noquery_utility",
|
| 836 |
+
"overall": 0.03392151442389414,
|
| 837 |
+
"by_dataset": {
|
| 838 |
+
"gas": 0.04948977903343789,
|
| 839 |
+
"hydraulic": 0.01835324981435039
|
| 840 |
+
},
|
| 841 |
+
"by_environment": {
|
| 842 |
+
"gas:batch10": 0.11999143590250941,
|
| 843 |
+
"gas:batch6": 0.060364321921130296,
|
| 844 |
+
"gas:batch7": 0.06709313734354976,
|
| 845 |
+
"gas:batch8": 0.0,
|
| 846 |
+
"gas:batch9": 0.0,
|
| 847 |
+
"hydraulic:context0=20": -0.1305010893246187,
|
| 848 |
+
"hydraulic:context0=3": 0.07168458781362005,
|
| 849 |
+
"hydraulic:context1=73": 0.0,
|
| 850 |
+
"hydraulic:context1=80": 0.00512820512820511,
|
| 851 |
+
"hydraulic:context1=90": 0.1454545454545455
|
| 852 |
+
},
|
| 853 |
+
"tasks": 57,
|
| 854 |
+
"descriptive_paired_95_interval": [
|
| 855 |
+
-0.011763300302199731,
|
| 856 |
+
0.07763983304671877
|
| 857 |
+
],
|
| 858 |
+
"paired_fit_rows": 171,
|
| 859 |
+
"environment_resampling_only": true
|
| 860 |
+
},
|
| 861 |
+
{
|
| 862 |
+
"label": "target_all_minus_64_identity",
|
| 863 |
+
"field": "realized_query_increment",
|
| 864 |
+
"overall": 0.002216571516610994,
|
| 865 |
+
"by_dataset": {
|
| 866 |
+
"gas": 0.006346022608024389,
|
| 867 |
+
"hydraulic": -0.0019128795748024011
|
| 868 |
+
},
|
| 869 |
+
"by_environment": {
|
| 870 |
+
"gas:batch10": 0.00011634653306482027,
|
| 871 |
+
"gas:batch6": 0.04626165071645424,
|
| 872 |
+
"gas:batch7": 0.003484189669053949,
|
| 873 |
+
"gas:batch8": -0.018077091060581647,
|
| 874 |
+
"gas:batch9": -5.498281786941268e-05,
|
| 875 |
+
"hydraulic:context0=20": -0.012164124909222947,
|
| 876 |
+
"hydraulic:context0=3": 0.03882437275985662,
|
| 877 |
+
"hydraulic:context1=73": -0.002232804232804253,
|
| 878 |
+
"hydraulic:context1=80": -0.014517094017094001,
|
| 879 |
+
"hydraulic:context1=90": -0.019474747474747423
|
| 880 |
+
},
|
| 881 |
+
"tasks": 57,
|
| 882 |
+
"descriptive_paired_95_interval": [
|
| 883 |
+
-0.009652936741599463,
|
| 884 |
+
0.016161801817197174
|
| 885 |
+
],
|
| 886 |
+
"paired_fit_rows": 171,
|
| 887 |
+
"environment_resampling_only": true
|
| 888 |
+
},
|
| 889 |
+
{
|
| 890 |
+
"label": "target_all_minus_64_identity",
|
| 891 |
+
"field": "joint_conditional_log_loss",
|
| 892 |
+
"overall": -0.18801059075396234,
|
| 893 |
+
"by_dataset": {
|
| 894 |
+
"gas": -0.22465903697780676,
|
| 895 |
+
"hydraulic": -0.15136214453011793
|
| 896 |
+
},
|
| 897 |
+
"by_environment": {
|
| 898 |
+
"gas:batch10": -0.42647144343500426,
|
| 899 |
+
"gas:batch6": -0.27447906953799683,
|
| 900 |
+
"gas:batch7": -0.38825936559980373,
|
| 901 |
+
"gas:batch8": -0.011590157952489915,
|
| 902 |
+
"gas:batch9": -0.022495148363738955,
|
| 903 |
+
"hydraulic:context0=20": -0.1815063570268768,
|
| 904 |
+
"hydraulic:context0=3": -0.1292322133249814,
|
| 905 |
+
"hydraulic:context1=73": -0.09410779468813202,
|
| 906 |
+
"hydraulic:context1=80": -0.10823786750704219,
|
| 907 |
+
"hydraulic:context1=90": -0.24372649010355726
|
| 908 |
+
},
|
| 909 |
+
"tasks": 57,
|
| 910 |
+
"descriptive_paired_95_interval": [
|
| 911 |
+
-0.26639647181970033,
|
| 912 |
+
-0.10573772708136289
|
| 913 |
+
],
|
| 914 |
+
"paired_fit_rows": 171,
|
| 915 |
+
"environment_resampling_only": true
|
| 916 |
+
},
|
| 917 |
+
{
|
| 918 |
+
"label": "source_fixed_policy_target_minus_sourceheld_64",
|
| 919 |
+
"field": "utility",
|
| 920 |
+
"overall": -0.3658345628075126,
|
| 921 |
+
"by_dataset": {
|
| 922 |
+
"gas": -0.5610563133386619,
|
| 923 |
+
"hydraulic": -0.1706128122763632
|
| 924 |
+
},
|
| 925 |
+
"by_environment": {
|
| 926 |
+
"gas:batch10": -0.9396158433078271,
|
| 927 |
+
"gas:batch6": -0.5596432280059105,
|
| 928 |
+
"gas:batch7": -0.6502933684535855,
|
| 929 |
+
"gas:batch8": -0.19518422367138946,
|
| 930 |
+
"gas:batch9": -0.46054490325459685,
|
| 931 |
+
"hydraulic:context0=20": -0.5833520872736558,
|
| 932 |
+
"hydraulic:context0=3": -0.8878212699287965,
|
| 933 |
+
"hydraulic:context1=73": 0.2936963399334534,
|
| 934 |
+
"hydraulic:context1=80": 0.2983467265838401,
|
| 935 |
+
"hydraulic:context1=90": 0.02606622930334268
|
| 936 |
+
},
|
| 937 |
+
"tasks": 57,
|
| 938 |
+
"descriptive_paired_95_interval": [
|
| 939 |
+
-0.6094731109065757,
|
| 940 |
+
-0.14076501702032948
|
| 941 |
+
],
|
| 942 |
+
"paired_fit_rows": 171,
|
| 943 |
+
"environment_resampling_only": true
|
| 944 |
+
},
|
| 945 |
+
{
|
| 946 |
+
"label": "source_fixed_policy_target_minus_sourceheld_64",
|
| 947 |
+
"field": "noquery_utility",
|
| 948 |
+
"overall": -0.2676674191045439,
|
| 949 |
+
"by_dataset": {
|
| 950 |
+
"gas": -0.5513721366349306,
|
| 951 |
+
"hydraulic": 0.016037298425842818
|
| 952 |
+
},
|
| 953 |
+
"by_environment": {
|
| 954 |
+
"gas:batch10": -0.9166768695714907,
|
| 955 |
+
"gas:batch6": -0.5431060684074404,
|
| 956 |
+
"gas:batch7": -0.630188728970858,
|
| 957 |
+
"gas:batch8": -0.19767658909014943,
|
| 958 |
+
"gas:batch9": -0.4692124271347146,
|
| 959 |
+
"hydraulic:context0=20": -0.7049630108453638,
|
| 960 |
+
"hydraulic:context0=3": -0.951136318878254,
|
| 961 |
+
"hydraulic:context1=73": 0.705187367043037,
|
| 962 |
+
"hydraulic:context1=80": 0.7126354744911446,
|
| 963 |
+
"hydraulic:context1=90": 0.31846298031865033
|
| 964 |
+
},
|
| 965 |
+
"tasks": 57,
|
| 966 |
+
"descriptive_paired_95_interval": [
|
| 967 |
+
-0.6011976930988733,
|
| 968 |
+
0.041075856067274444
|
| 969 |
+
],
|
| 970 |
+
"paired_fit_rows": 171,
|
| 971 |
+
"environment_resampling_only": true
|
| 972 |
+
},
|
| 973 |
+
{
|
| 974 |
+
"label": "source_fixed_policy_target_minus_sourceheld_64",
|
| 975 |
+
"field": "realized_query_increment",
|
| 976 |
+
"overall": -0.09816714370296867,
|
| 977 |
+
"by_dataset": {
|
| 978 |
+
"gas": -0.009684176703731236,
|
| 979 |
+
"hydraulic": -0.1866501107022061
|
| 980 |
+
},
|
| 981 |
+
"by_environment": {
|
| 982 |
+
"gas:batch10": -0.02293897373633634,
|
| 983 |
+
"gas:batch6": -0.0165371595984701,
|
| 984 |
+
"gas:batch7": -0.020104639482727473,
|
| 985 |
+
"gas:batch8": 0.0024923654187599905,
|
| 986 |
+
"gas:batch9": 0.00866752388011774,
|
| 987 |
+
"hydraulic:context0=20": 0.12161092357170783,
|
| 988 |
+
"hydraulic:context0=3": 0.06331504894945743,
|
| 989 |
+
"hydraulic:context1=73": -0.4114910271095837,
|
| 990 |
+
"hydraulic:context1=80": -0.4142887479073045,
|
| 991 |
+
"hydraulic:context1=90": -0.2923967510153076
|
| 992 |
+
},
|
| 993 |
+
"tasks": 57,
|
| 994 |
+
"descriptive_paired_95_interval": [
|
| 995 |
+
-0.197981945594568,
|
| 996 |
+
0.003237513471611564
|
| 997 |
+
],
|
| 998 |
+
"paired_fit_rows": 171,
|
| 999 |
+
"environment_resampling_only": true
|
| 1000 |
+
},
|
| 1001 |
+
{
|
| 1002 |
+
"label": "source_fixed_policy_target_minus_sourceheld_64",
|
| 1003 |
+
"field": "joint_conditional_log_loss",
|
| 1004 |
+
"overall": 0.5621445508593099,
|
| 1005 |
+
"by_dataset": {
|
| 1006 |
+
"gas": 0.5800953291658606,
|
| 1007 |
+
"hydraulic": 0.5441937725527592
|
| 1008 |
+
},
|
| 1009 |
+
"by_environment": {
|
| 1010 |
+
"gas:batch10": 0.9486170445415449,
|
| 1011 |
+
"gas:batch6": 0.4437875424241321,
|
| 1012 |
+
"gas:batch7": 0.7164416200807845,
|
| 1013 |
+
"gas:batch8": 0.426021559021634,
|
| 1014 |
+
"gas:batch9": 0.36560887976120765,
|
| 1015 |
+
"hydraulic:context0=20": 1.6799077884463471,
|
| 1016 |
+
"hydraulic:context0=3": 1.682189335498656,
|
| 1017 |
+
"hydraulic:context1=73": -0.4180981671810516,
|
| 1018 |
+
"hydraulic:context1=80": -0.19134701166286475,
|
| 1019 |
+
"hydraulic:context1=90": -0.03168308233729015
|
| 1020 |
+
},
|
| 1021 |
+
"tasks": 57,
|
| 1022 |
+
"descriptive_paired_95_interval": [
|
| 1023 |
+
0.15401369963728354,
|
| 1024 |
+
0.9865531222258157
|
| 1025 |
+
],
|
| 1026 |
+
"paired_fit_rows": 171,
|
| 1027 |
+
"environment_resampling_only": true
|
| 1028 |
+
},
|
| 1029 |
+
{
|
| 1030 |
+
"label": "source_fixed_policy_target_minus_sourceheld_64",
|
| 1031 |
+
"field": "common_current_joint_conditional_log_loss",
|
| 1032 |
+
"overall": 0.5594148121355085,
|
| 1033 |
+
"by_dataset": {
|
| 1034 |
+
"gas": 0.20223821826777272,
|
| 1035 |
+
"hydraulic": 0.9165914060032442
|
| 1036 |
+
},
|
| 1037 |
+
"by_environment": {
|
| 1038 |
+
"gas:batch10": 0.6142603038782647,
|
| 1039 |
+
"gas:batch6": 0.02758507900012181,
|
| 1040 |
+
"gas:batch7": 0.5621359751087247,
|
| 1041 |
+
"gas:batch8": 0.2504521509982081,
|
| 1042 |
+
"gas:batch9": -0.4432424176464558,
|
| 1043 |
+
"hydraulic:context0=20": 2.0467262627943366,
|
| 1044 |
+
"hydraulic:context0=3": 2.4213909252713104,
|
| 1045 |
+
"hydraulic:context1=73": 0.017684862182114403,
|
| 1046 |
+
"hydraulic:context1=80": 0.12330671842806,
|
| 1047 |
+
"hydraulic:context1=90": -0.02615173865960056
|
| 1048 |
+
},
|
| 1049 |
+
"tasks": 57,
|
| 1050 |
+
"descriptive_paired_95_interval": [
|
| 1051 |
+
0.08425091300955856,
|
| 1052 |
+
1.0753570244945965
|
| 1053 |
+
],
|
| 1054 |
+
"paired_fit_rows": 171,
|
| 1055 |
+
"environment_resampling_only": true
|
| 1056 |
+
},
|
| 1057 |
+
{
|
| 1058 |
+
"label": "source_fixed_policy_target_minus_sourceheld_256",
|
| 1059 |
+
"field": "utility",
|
| 1060 |
+
"overall": -0.45219716166405916,
|
| 1061 |
+
"by_dataset": {
|
| 1062 |
+
"gas": -0.6613710559765582,
|
| 1063 |
+
"hydraulic": -0.24302326735156013
|
| 1064 |
+
},
|
| 1065 |
+
"by_environment": {
|
| 1066 |
+
"gas:batch10": -0.9162104276816436,
|
| 1067 |
+
"gas:batch6": -0.45642588117381144,
|
| 1068 |
+
"gas:batch7": -0.7440051374832067,
|
| 1069 |
+
"gas:batch8": -0.4961445700097846,
|
| 1070 |
+
"gas:batch9": -0.6940692635343448,
|
| 1071 |
+
"hydraulic:context0=20": -0.3936751897340132,
|
| 1072 |
+
"hydraulic:context0=3": -0.9409959431249754,
|
| 1073 |
+
"hydraulic:context1=73": 0.05824164075710466,
|
| 1074 |
+
"hydraulic:context1=80": 0.044280509295973235,
|
| 1075 |
+
"hydraulic:context1=90": 0.017032646048109983
|
| 1076 |
+
},
|
| 1077 |
+
"tasks": 57,
|
| 1078 |
+
"descriptive_paired_95_interval": [
|
| 1079 |
+
-0.6480843443520771,
|
| 1080 |
+
-0.286414985093672
|
| 1081 |
+
],
|
| 1082 |
+
"paired_fit_rows": 171,
|
| 1083 |
+
"environment_resampling_only": true
|
| 1084 |
+
},
|
| 1085 |
+
{
|
| 1086 |
+
"label": "source_fixed_policy_target_minus_sourceheld_256",
|
| 1087 |
+
"field": "noquery_utility",
|
| 1088 |
+
"overall": -0.49481529547920394,
|
| 1089 |
+
"by_dataset": {
|
| 1090 |
+
"gas": -0.6612623846504923,
|
| 1091 |
+
"hydraulic": -0.32836820630791563
|
| 1092 |
+
},
|
| 1093 |
+
"by_environment": {
|
| 1094 |
+
"gas:batch10": -0.9064315822975246,
|
| 1095 |
+
"gas:batch6": -0.4365394381842272,
|
| 1096 |
+
"gas:batch7": -0.7362261658674275,
|
| 1097 |
+
"gas:batch8": -0.551893963728601,
|
| 1098 |
+
"gas:batch9": -0.6752207731746808,
|
| 1099 |
+
"hydraulic:context0=20": -0.6481936364289306,
|
| 1100 |
+
"hydraulic:context0=3": -1.2564443919282626,
|
| 1101 |
+
"hydraulic:context1=73": 0.11600392734413338,
|
| 1102 |
+
"hydraulic:context1=80": 0.09659000793021395,
|
| 1103 |
+
"hydraulic:context1=90": 0.05020306154326767
|
| 1104 |
+
},
|
| 1105 |
+
"tasks": 57,
|
| 1106 |
+
"descriptive_paired_95_interval": [
|
| 1107 |
+
-0.755367243841711,
|
| 1108 |
+
-0.270041361311064
|
| 1109 |
+
],
|
| 1110 |
+
"paired_fit_rows": 171,
|
| 1111 |
+
"environment_resampling_only": true
|
| 1112 |
+
},
|
| 1113 |
+
{
|
| 1114 |
+
"label": "source_fixed_policy_target_minus_sourceheld_256",
|
| 1115 |
+
"field": "realized_query_increment",
|
| 1116 |
+
"overall": 0.042618133815144746,
|
| 1117 |
+
"by_dataset": {
|
| 1118 |
+
"gas": -0.00010867132606601146,
|
| 1119 |
+
"hydraulic": 0.0853449389563555
|
| 1120 |
+
},
|
| 1121 |
+
"by_environment": {
|
| 1122 |
+
"gas:batch10": -0.00977884538411901,
|
| 1123 |
+
"gas:batch6": -0.019886442989584283,
|
| 1124 |
+
"gas:batch7": -0.007778971615779189,
|
| 1125 |
+
"gas:batch8": 0.055749393718816466,
|
| 1126 |
+
"gas:batch9": -0.01884849035966404,
|
| 1127 |
+
"hydraulic:context0=20": 0.25451844669491724,
|
| 1128 |
+
"hydraulic:context0=3": 0.3154484488032874,
|
| 1129 |
+
"hydraulic:context1=73": -0.05776228658702872,
|
| 1130 |
+
"hydraulic:context1=80": -0.052309498634240725,
|
| 1131 |
+
"hydraulic:context1=90": -0.03317041549515768
|
| 1132 |
+
},
|
| 1133 |
+
"tasks": 57,
|
| 1134 |
+
"descriptive_paired_95_interval": [
|
| 1135 |
+
-0.027765450120834784,
|
| 1136 |
+
0.11627422347851524
|
| 1137 |
+
],
|
| 1138 |
+
"paired_fit_rows": 171,
|
| 1139 |
+
"environment_resampling_only": true
|
| 1140 |
+
},
|
| 1141 |
+
{
|
| 1142 |
+
"label": "source_fixed_policy_target_minus_sourceheld_256",
|
| 1143 |
+
"field": "joint_conditional_log_loss",
|
| 1144 |
+
"overall": 0.6138085845900283,
|
| 1145 |
+
"by_dataset": {
|
| 1146 |
+
"gas": 0.6632822137991161,
|
| 1147 |
+
"hydraulic": 0.5643349553809406
|
| 1148 |
+
},
|
| 1149 |
+
"by_environment": {
|
| 1150 |
+
"gas:batch10": 0.8991049325233872,
|
| 1151 |
+
"gas:batch6": 0.37054835953278614,
|
| 1152 |
+
"gas:batch7": 0.8245430070724563,
|
| 1153 |
+
"gas:batch8": 0.6995219189865814,
|
| 1154 |
+
"gas:batch9": 0.5226928508803698,
|
| 1155 |
+
"hydraulic:context0=20": 1.7382918974288823,
|
| 1156 |
+
"hydraulic:context0=3": 1.7819212294649285,
|
| 1157 |
+
"hydraulic:context1=73": -0.38166563484429133,
|
| 1158 |
+
"hydraulic:context1=80": -0.21327311004018704,
|
| 1159 |
+
"hydraulic:context1=90": -0.10359960510462929
|
| 1160 |
+
},
|
| 1161 |
+
"tasks": 57,
|
| 1162 |
+
"descriptive_paired_95_interval": [
|
| 1163 |
+
0.19329377403326248,
|
| 1164 |
+
1.053042905163758
|
| 1165 |
+
],
|
| 1166 |
+
"paired_fit_rows": 171,
|
| 1167 |
+
"environment_resampling_only": true
|
| 1168 |
+
},
|
| 1169 |
+
{
|
| 1170 |
+
"label": "source_fixed_policy_target_minus_sourceheld_256",
|
| 1171 |
+
"field": "common_current_joint_conditional_log_loss",
|
| 1172 |
+
"overall": 0.6519535658720754,
|
| 1173 |
+
"by_dataset": {
|
| 1174 |
+
"gas": 0.38500890988983894,
|
| 1175 |
+
"hydraulic": 0.9188982218543119
|
| 1176 |
+
},
|
| 1177 |
+
"by_environment": {
|
| 1178 |
+
"gas:batch10": 0.7156680549909323,
|
| 1179 |
+
"gas:batch6": 0.16467203783818304,
|
| 1180 |
+
"gas:batch7": 0.7492018890832239,
|
| 1181 |
+
"gas:batch8": 0.36610723064655865,
|
| 1182 |
+
"gas:batch9": -0.07060466310970315,
|
| 1183 |
+
"hydraulic:context0=20": 2.128697318307584,
|
| 1184 |
+
"hydraulic:context0=3": 2.451537531866698,
|
| 1185 |
+
"hydraulic:context1=73": 0.011443334798441415,
|
| 1186 |
+
"hydraulic:context1=80": 0.07249372345870368,
|
| 1187 |
+
"hydraulic:context1=90": -0.06968079915986725
|
| 1188 |
+
},
|
| 1189 |
+
"tasks": 57,
|
| 1190 |
+
"descriptive_paired_95_interval": [
|
| 1191 |
+
0.16244540247836492,
|
| 1192 |
+
1.1709003990409212
|
| 1193 |
+
],
|
| 1194 |
+
"paired_fit_rows": 171,
|
| 1195 |
+
"environment_resampling_only": true
|
| 1196 |
+
},
|
| 1197 |
+
{
|
| 1198 |
+
"label": "source_fixed_policy_target_minus_sourceheld_all",
|
| 1199 |
+
"field": "utility",
|
| 1200 |
+
"overall": -0.4396127556794738,
|
| 1201 |
+
"by_dataset": {
|
| 1202 |
+
"gas": -0.7350152367869054,
|
| 1203 |
+
"hydraulic": -0.14421027457204222
|
| 1204 |
+
},
|
| 1205 |
+
"by_environment": {
|
| 1206 |
+
"gas:batch10": -0.9367731818344003,
|
| 1207 |
+
"gas:batch6": -0.460957579508446,
|
| 1208 |
+
"gas:batch7": -0.795378052512238,
|
| 1209 |
+
"gas:batch8": -0.5596553210223444,
|
| 1210 |
+
"gas:batch9": -0.9223120490570984,
|
| 1211 |
+
"hydraulic:context0=20": -0.21553340994517464,
|
| 1212 |
+
"hydraulic:context0=3": -0.6091156012446335,
|
| 1213 |
+
"hydraulic:context1=73": 0.04251243658975612,
|
| 1214 |
+
"hydraulic:context1=80": 0.03424015331747299,
|
| 1215 |
+
"hydraulic:context1=90": 0.026845048422368012
|
| 1216 |
+
},
|
| 1217 |
+
"tasks": 57,
|
| 1218 |
+
"descriptive_paired_95_interval": [
|
| 1219 |
+
-0.5862730773581387,
|
| 1220 |
+
-0.30774058981316016
|
| 1221 |
+
],
|
| 1222 |
+
"paired_fit_rows": 171,
|
| 1223 |
+
"environment_resampling_only": true
|
| 1224 |
+
},
|
| 1225 |
+
{
|
| 1226 |
+
"label": "source_fixed_policy_target_minus_sourceheld_all",
|
| 1227 |
+
"field": "noquery_utility",
|
| 1228 |
+
"overall": -0.5717488762836339,
|
| 1229 |
+
"by_dataset": {
|
| 1230 |
+
"gas": -0.7452046483002596,
|
| 1231 |
+
"hydraulic": -0.39829310426700826
|
| 1232 |
+
},
|
| 1233 |
+
"by_environment": {
|
| 1234 |
+
"gas:batch10": -0.8891339260517567,
|
| 1235 |
+
"gas:batch6": -0.4653948925826942,
|
| 1236 |
+
"gas:batch7": -0.8336366769520721,
|
| 1237 |
+
"gas:batch8": -0.5831796742059052,
|
| 1238 |
+
"gas:batch9": -0.9546780717088698,
|
| 1239 |
+
"hydraulic:context0=20": -0.7205774617539324,
|
| 1240 |
+
"hydraulic:context0=3": -1.2731812989877502,
|
| 1241 |
+
"hydraulic:context1=73": 0.01183112420225818,
|
| 1242 |
+
"hydraulic:context1=80": 0.015860428231562196,
|
| 1243 |
+
"hydraulic:context1=90": -0.025398313027178987
|
| 1244 |
+
},
|
| 1245 |
+
"tasks": 57,
|
| 1246 |
+
"descriptive_paired_95_interval": [
|
| 1247 |
+
-0.8251105760935765,
|
| 1248 |
+
-0.3498471014539569
|
| 1249 |
+
],
|
| 1250 |
+
"paired_fit_rows": 171,
|
| 1251 |
+
"environment_resampling_only": true
|
| 1252 |
+
},
|
| 1253 |
+
{
|
| 1254 |
+
"label": "source_fixed_policy_target_minus_sourceheld_all",
|
| 1255 |
+
"field": "realized_query_increment",
|
| 1256 |
+
"overall": 0.13213612060416013,
|
| 1257 |
+
"by_dataset": {
|
| 1258 |
+
"gas": 0.010189411513354157,
|
| 1259 |
+
"hydraulic": 0.2540828296949661
|
| 1260 |
+
},
|
| 1261 |
+
"by_environment": {
|
| 1262 |
+
"gas:batch10": -0.0476392557826436,
|
| 1263 |
+
"gas:batch6": 0.004437313074248212,
|
| 1264 |
+
"gas:batch7": 0.038258624439834056,
|
| 1265 |
+
"gas:batch8": 0.02352435318356087,
|
| 1266 |
+
"gas:batch9": 0.03236602265177125,
|
| 1267 |
+
"hydraulic:context0=20": 0.5050440518087577,
|
| 1268 |
+
"hydraulic:context0=3": 0.664065697743117,
|
| 1269 |
+
"hydraulic:context1=73": 0.03068131238749794,
|
| 1270 |
+
"hydraulic:context1=80": 0.018379725085910792,
|
| 1271 |
+
"hydraulic:context1=90": 0.052243361449547
|
| 1272 |
+
},
|
| 1273 |
+
"tasks": 57,
|
| 1274 |
+
"descriptive_paired_95_interval": [
|
| 1275 |
+
0.01925016215607177,
|
| 1276 |
+
0.2549383828614076
|
| 1277 |
+
],
|
| 1278 |
+
"paired_fit_rows": 171,
|
| 1279 |
+
"environment_resampling_only": true
|
| 1280 |
+
},
|
| 1281 |
+
{
|
| 1282 |
+
"label": "source_fixed_policy_target_minus_sourceheld_all",
|
| 1283 |
+
"field": "joint_conditional_log_loss",
|
| 1284 |
+
"overall": 0.685292644015933,
|
| 1285 |
+
"by_dataset": {
|
| 1286 |
+
"gas": 0.7914155648485723,
|
| 1287 |
+
"hydraulic": 0.5791697231832938
|
| 1288 |
+
},
|
| 1289 |
+
"by_environment": {
|
| 1290 |
+
"gas:batch10": 0.9767006328121373,
|
| 1291 |
+
"gas:batch6": 0.3787905554315623,
|
| 1292 |
+
"gas:batch7": 0.932105359957967,
|
| 1293 |
+
"gas:batch8": 0.9789245055964885,
|
| 1294 |
+
"gas:batch9": 0.6905567704447064,
|
| 1295 |
+
"hydraulic:context0=20": 1.7437744440617207,
|
| 1296 |
+
"hydraulic:context0=3": 1.8285083029868723,
|
| 1297 |
+
"hydraulic:context1=73": -0.39084329451424943,
|
| 1298 |
+
"hydraulic:context1=80": -0.19394298606297666,
|
| 1299 |
+
"hydraulic:context1=90": -0.09164785055489755
|
| 1300 |
+
},
|
| 1301 |
+
"tasks": 57,
|
| 1302 |
+
"descriptive_paired_95_interval": [
|
| 1303 |
+
0.25785834074167685,
|
| 1304 |
+
1.1380925405286935
|
| 1305 |
+
],
|
| 1306 |
+
"paired_fit_rows": 171,
|
| 1307 |
+
"environment_resampling_only": true
|
| 1308 |
+
},
|
| 1309 |
+
{
|
| 1310 |
+
"label": "source_fixed_policy_target_minus_sourceheld_all",
|
| 1311 |
+
"field": "common_current_joint_conditional_log_loss",
|
| 1312 |
+
"overall": 0.7167234355580608,
|
| 1313 |
+
"by_dataset": {
|
| 1314 |
+
"gas": 0.531140981710618,
|
| 1315 |
+
"hydraulic": 0.9023058894055037
|
| 1316 |
+
},
|
| 1317 |
+
"by_environment": {
|
| 1318 |
+
"gas:batch10": 0.8650012114531468,
|
| 1319 |
+
"gas:batch6": 0.23007087447055435,
|
| 1320 |
+
"gas:batch7": 0.8912368902514587,
|
| 1321 |
+
"gas:batch8": 0.541883922758298,
|
| 1322 |
+
"gas:batch9": 0.12751200961963197,
|
| 1323 |
+
"hydraulic:context0=20": 2.0541632239252148,
|
| 1324 |
+
"hydraulic:context0=3": 2.3804864627490447,
|
| 1325 |
+
"hydraulic:context1=73": 0.008144612778454824,
|
| 1326 |
+
"hydraulic:context1=80": 0.05183122892122599,
|
| 1327 |
+
"hydraulic:context1=90": 0.016903918653577987
|
| 1328 |
+
},
|
| 1329 |
+
"tasks": 57,
|
| 1330 |
+
"descriptive_paired_95_interval": [
|
| 1331 |
+
0.2463337707830208,
|
| 1332 |
+
1.221176584780543
|
| 1333 |
+
],
|
| 1334 |
+
"paired_fit_rows": 171,
|
| 1335 |
+
"environment_resampling_only": true
|
| 1336 |
+
}
|
| 1337 |
+
]
|
| 1338 |
+
}
|
revision_r1/external/run.py
ADDED
|
@@ -0,0 +1,515 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Post-review external failure analysis; original data, policies and gates stay frozen.
|
| 2 |
+
|
| 3 |
+
All scientific tensor operations run on CUDA. JSON/CSV, hashes, split membership,
|
| 4 |
+
and archive serialization are CPU bookkeeping. No neural model is introduced.
|
| 5 |
+
"""
|
| 6 |
+
from __future__ import annotations
|
| 7 |
+
|
| 8 |
+
import argparse
|
| 9 |
+
import csv
|
| 10 |
+
from datetime import datetime, timezone
|
| 11 |
+
import gzip
|
| 12 |
+
import hashlib
|
| 13 |
+
import json
|
| 14 |
+
from pathlib import Path
|
| 15 |
+
import platform
|
| 16 |
+
import sys
|
| 17 |
+
import time
|
| 18 |
+
from collections import defaultdict
|
| 19 |
+
|
| 20 |
+
ROOT = Path(__file__).resolve().parents[3]
|
| 21 |
+
_bootstrap = argparse.ArgumentParser(add_help=False)
|
| 22 |
+
_bootstrap.add_argument("--release", type=Path, default=ROOT / "articles/make_1/release/huggingface-revised")
|
| 23 |
+
_boot_args, _ = _bootstrap.parse_known_args()
|
| 24 |
+
RELEASE = _boot_args.release.resolve()
|
| 25 |
+
sys.path.insert(0, str(RELEASE))
|
| 26 |
+
import torch
|
| 27 |
+
from unionmind.decision_sufficiency.core import (
|
| 28 |
+
CONTEXTS, calibration, count_samples, encode, estimate, plan, push, realized,
|
| 29 |
+
)
|
| 30 |
+
from unionmind.external_sufficiency.runtime import replay, select_samples
|
| 31 |
+
|
| 32 |
+
INPUT = ROOT / "artifacts/external_decision_sufficiency_01"
|
| 33 |
+
DEFAULT_OUTPUT = ROOT / "articles/make_1/metadata/review_r1/external"
|
| 34 |
+
SEEDS = [20111, 20112, 20113]
|
| 35 |
+
METHODS = ["query_joint", "posterior_clusters", "identity"]
|
| 36 |
+
MODELS = ["support_mixture", "joint_jeffreys", "naive_bayes"]
|
| 37 |
+
|
| 38 |
+
|
| 39 |
+
def sha(path):
|
| 40 |
+
return hashlib.sha256(Path(path).read_bytes()).hexdigest()
|
| 41 |
+
|
| 42 |
+
|
| 43 |
+
def write(path, obj):
|
| 44 |
+
path.parent.mkdir(parents=True, exist_ok=True)
|
| 45 |
+
path.write_text(json.dumps(obj, indent=2, allow_nan=False) + "\n")
|
| 46 |
+
|
| 47 |
+
|
| 48 |
+
def utc():
|
| 49 |
+
return datetime.now(timezone.utc).isoformat()
|
| 50 |
+
|
| 51 |
+
|
| 52 |
+
def specification(out, device):
|
| 53 |
+
manifest = json.loads((INPUT / "prepared/manifest.json").read_text())
|
| 54 |
+
spec = dict(
|
| 55 |
+
material_passport=dict(material_type="post-review exploratory experiment specification",
|
| 56 |
+
provenance="Previously opened external sensor archives and original eligible task manifest",
|
| 57 |
+
license="Hydraulic UCI CC BY 4.0; gas legacy research-only restriction retained; local research use only",
|
| 58 |
+
created_utc=utc(), scientific_status="exploratory; does not replace original confirmation"),
|
| 59 |
+
purpose="Distinguish small within-fitted-law encoding headroom, finite labeled support, and observed source-target transfer failure without claiming an identifiable population decomposition",
|
| 60 |
+
device=device, dtype="float64", backbone_executions=0, trainable_parameters=0,
|
| 61 |
+
task_indices=[t["index"] for t in manifest["tasks"]], seeds=SEEDS,
|
| 62 |
+
methods=METHODS, estimators=MODELS,
|
| 63 |
+
interventions=[
|
| 64 |
+
"Original target64 exact seed selection, nested target256 when available, and all original target adaptation records",
|
| 65 |
+
"Source group-disjoint train/held split: SHA256('review_r1_source_held:' + original_group) mod 5 == 0 held; source64/256/all train only",
|
| 66 |
+
"Each fixed policy replayed on original target evaluation and, for source-trained policies, source held records",
|
| 67 |
+
"All-target-adaptation unsmoothed empirical reference law: encoders and exact policy fitted without target evaluation access, evaluated on original independent-group target evaluation; not a population oracle",
|
| 68 |
+
],
|
| 69 |
+
source_preprocessing="Original source-only binary thresholds kept; source-held records contributed unlabeled preprocessing before this post-review split, never labeled model fitting",
|
| 70 |
+
metrics=["paired realized utility and no-query utility", "model-predicted versus held-realized query increment", "fitted encoding loss", "posterior/joint/observation conditional log losses", "same-policy source-held versus shifted-target scores", "finite-sample empirical conditional TV on overlapping current support", "target mass outside source-held current support"],
|
| 71 |
+
aggregation="Original weighting: seeds, then tasks within environment, then environments within dataset, then the two datasets equally. Sample-size contrasts restricted to identical tasks. Costs 0,.01,.05 and budgets 1,2 averaged; no-query average from budget0.",
|
| 72 |
+
inference="Descriptive 95% paired bootstrap by target environment within fixed dataset; collapse tasks/seeds first. Few clusters; not population transfer inference; no confirmatory gate rescues.",
|
| 73 |
+
limitations=["True external law is unknown; histogram reference and TV have finite-sample error", "Source-target contrast also includes differences in physical groups and sample composition; no causal attribution percentages", "Labels-all support varies by task; 256 contrasts require available support and use matched task subsets", "Source group split introduces a new held partition; original target adaptation/evaluation split unchanged", "Three seeds are repeated fitting samples, not independent environmental replications", "Empirical-reference policies are exploratory benchmarks and their NLL is not reported because zero fitted cells yield uninformative infinite log scores"],
|
| 74 |
+
input_hashes={str(p.relative_to(ROOT) if p.is_relative_to(ROOT) else p): sha(p) for p in [INPUT / "prepared/manifest.json", RELEASE / "protocols/external_sufficiency_protocol_01.json", RELEASE / "unionmind/decision_sufficiency/core.py", RELEASE / "unionmind/external_sufficiency/runtime.py"]},
|
| 75 |
+
script_sha256=sha(__file__), no_recipe_selection=True, original_results_and_gates_unchanged=True,
|
| 76 |
+
)
|
| 77 |
+
assert not (out / "specification.json").exists(), "Specification already exists; never overwrite it."
|
| 78 |
+
write(out / "specification.json", spec)
|
| 79 |
+
return spec
|
| 80 |
+
|
| 81 |
+
|
| 82 |
+
def normalized_counts(events):
|
| 83 |
+
c = count_samples(events[None], 6)
|
| 84 |
+
return c / c.sum()
|
| 85 |
+
|
| 86 |
+
|
| 87 |
+
def scores(model, truth):
|
| 88 |
+
result = calibration(model, truth)
|
| 89 |
+
cond = model / model.sum((2, 3), keepdim=True).clamp_min(1e-300)
|
| 90 |
+
givenh = model / model.sum(3, keepdim=True).clamp_min(1e-300)
|
| 91 |
+
result["joint_conditional_log_loss"] = -(truth * cond.clamp_min(1e-300).log()).sum((1, 2, 3))
|
| 92 |
+
result["observation_given_h_log_loss"] = -(truth * givenh.clamp_min(1e-300).log()).sum((1, 2, 3))
|
| 93 |
+
return result
|
| 94 |
+
|
| 95 |
+
|
| 96 |
+
def batched_replay(policy, code, events, budget, ci):
|
| 97 |
+
"""Same paid-bit access convention as original replay; batch independent policies."""
|
| 98 |
+
b, n = len(code), len(events)
|
| 99 |
+
r = policy["shape"][1]
|
| 100 |
+
table = policy["choices"].reshape(3, b, r, 3, 9)
|
| 101 |
+
z = code[:, (events // 4) % 64].long()
|
| 102 |
+
row = torch.arange(b, device=events.device)[:, None].expand(b, n)
|
| 103 |
+
context = torch.zeros((b, n), device=events.device, dtype=torch.long)
|
| 104 |
+
action, queries, seen = context.clone(), context.clone(), context.clone()
|
| 105 |
+
live = torch.ones_like(context, dtype=torch.bool)
|
| 106 |
+
child = torch.zeros((9, 2, 2), device=events.device, dtype=torch.long)
|
| 107 |
+
for t, c in enumerate(CONTEXTS):
|
| 108 |
+
for q in (0, 1):
|
| 109 |
+
if c[q] == -1:
|
| 110 |
+
for o in (0, 1):
|
| 111 |
+
child[t, q, o] = CONTEXTS.index(tuple(o if i == q else v for i, v in enumerate(c)))
|
| 112 |
+
for left in range(budget, -1, -1):
|
| 113 |
+
choice = table[left, row, z, ci, context]
|
| 114 |
+
querying = live & (choice >= 3)
|
| 115 |
+
q = (choice - 3).clamp(0, 1)
|
| 116 |
+
selected = querying.nonzero()
|
| 117 |
+
revealed = torch.zeros_like(context)
|
| 118 |
+
rr, nn = selected[:, 0], selected[:, 1]
|
| 119 |
+
revealed[rr, nn] = (events[nn] >> (1 - q[rr, nn])) & 1
|
| 120 |
+
context = torch.where(querying, child[context, q, revealed], context)
|
| 121 |
+
action = torch.where(live & ~querying, choice, action)
|
| 122 |
+
queries += querying.long()
|
| 123 |
+
seen |= torch.where(querying, 1 << q, 0)
|
| 124 |
+
live = querying
|
| 125 |
+
assert not bool(live.any())
|
| 126 |
+
utility = torch.where(action == 2, 0., torch.where(action == events[None] // 256, 1., -2.)).double()
|
| 127 |
+
utility -= policy["costs"][ci] * queries.double()
|
| 128 |
+
return torch.stack((action, queries, seen), -1).to(torch.uint8), utility.mean(1)
|
| 129 |
+
|
| 130 |
+
|
| 131 |
+
def get_samples(archive, provenance, seed):
|
| 132 |
+
device = archive["source"].device
|
| 133 |
+
# Exact original selection consumes source permutation before adaptation.
|
| 134 |
+
g = torch.Generator(device=device).manual_seed(seed)
|
| 135 |
+
torch.randperm(len(archive["source"]), device=device, generator=g)
|
| 136 |
+
target_order = torch.randperm(len(archive["adaptation"]), device=device, generator=g)
|
| 137 |
+
held_mask = torch.tensor([int(hashlib.sha256(("review_r1_source_held:" + x["group"]).encode()).hexdigest(), 16) % 5 == 0 for x in provenance["source"]], device=device)
|
| 138 |
+
source_held = held_mask.nonzero().flatten()
|
| 139 |
+
source_train = (~held_mask).nonzero().flatten()
|
| 140 |
+
sg = torch.Generator(device=device).manual_seed(seed)
|
| 141 |
+
source_order = source_train[torch.randperm(len(source_train), device=device, generator=sg)]
|
| 142 |
+
assert len(source_order) >= 64 and len(source_held) > 0
|
| 143 |
+
values = []
|
| 144 |
+
indices = {}
|
| 145 |
+
for domain, order in [("target", target_order), ("source", source_order)]:
|
| 146 |
+
source_key = "adaptation" if domain == "target" else "source"
|
| 147 |
+
for size in [64, 256, "all"]:
|
| 148 |
+
if size != "all" and len(order) < size:
|
| 149 |
+
continue
|
| 150 |
+
ix = order if size == "all" else order[:size]
|
| 151 |
+
events = archive[source_key][ix]
|
| 152 |
+
values.append((domain, str(size), events))
|
| 153 |
+
indices[f"{domain}:{size}:{seed}"] = ix
|
| 154 |
+
original = select_samples(archive, "target_64", seed)
|
| 155 |
+
assert torch.equal(archive["adaptation"][target_order[:64]], original)
|
| 156 |
+
return values, indices, source_held, source_train
|
| 157 |
+
|
| 158 |
+
|
| 159 |
+
def evaluate(out, device):
|
| 160 |
+
spec = json.loads((out / "specification.json").read_text())
|
| 161 |
+
assert spec["script_sha256"] == sha(__file__)
|
| 162 |
+
for path, digest in spec["input_hashes"].items():
|
| 163 |
+
assert sha(ROOT / path) == digest
|
| 164 |
+
assert not (out / "metrics.csv.gz").exists(), "Never overwrite an analysis run."
|
| 165 |
+
manifest = json.loads((INPUT / "prepared/manifest.json").read_text())
|
| 166 |
+
torch.cuda.set_device(device)
|
| 167 |
+
torch.backends.cuda.matmul.allow_tf32 = False
|
| 168 |
+
torch.backends.cudnn.allow_tf32 = False
|
| 169 |
+
torch.cuda.reset_peak_memory_stats(device)
|
| 170 |
+
start = time.perf_counter()
|
| 171 |
+
rows, shifts = [], []
|
| 172 |
+
maximum_replay_error, maximum_original_error = 0., 0.
|
| 173 |
+
for task in manifest["tasks"]:
|
| 174 |
+
ti = task["index"]
|
| 175 |
+
archive = torch.load(INPUT / "prepared" / task["file"], map_location=device, weights_only=True)
|
| 176 |
+
prov = json.loads((INPUT / "prepared" / f"task_{ti:03}_provenance.json").read_text())
|
| 177 |
+
meta, counts, indices = [], [], {}
|
| 178 |
+
for seed in SEEDS:
|
| 179 |
+
samples, ix, source_held, source_train = get_samples(archive, prov, seed)
|
| 180 |
+
indices.update(ix)
|
| 181 |
+
for domain, size, events in samples:
|
| 182 |
+
c = count_samples(events[None], 6)[0]
|
| 183 |
+
for modelname in MODELS:
|
| 184 |
+
meta.append(dict(training=domain, size=size, sample_seed=seed, model=modelname, sample_count=len(events)))
|
| 185 |
+
counts.append(c)
|
| 186 |
+
# Nondeployable population-oracle claims are avoided: only independent
|
| 187 |
+
# adaptation labels define this empirical finite-sample reference.
|
| 188 |
+
meta.append(dict(training="target", size="all", sample_seed=-1, model="empirical_reference", sample_count=len(archive["adaptation"])))
|
| 189 |
+
counts.append(count_samples(archive["adaptation"][None], 6)[0])
|
| 190 |
+
allcounts = torch.stack(counts)
|
| 191 |
+
model = torch.empty_like(allcounts)
|
| 192 |
+
for name in MODELS:
|
| 193 |
+
ix = torch.tensor([i for i, m in enumerate(meta) if m["model"] == name], device=device)
|
| 194 |
+
model[ix] = estimate(allcounts[ix], name)
|
| 195 |
+
model[-1] = allcounts[-1] / allcounts[-1].sum()
|
| 196 |
+
full_policy = plan(model)
|
| 197 |
+
source_events = archive["source"][source_held]
|
| 198 |
+
laws = {"target_evaluation": normalized_counts(archive["evaluation"]), "source_held": normalized_counts(source_events)}
|
| 199 |
+
events_by_domain = {"target_evaluation": archive["evaluation"], "source_held": source_events}
|
| 200 |
+
saved = dict(metadata=meta, indices=indices, source_held_indices=source_held,
|
| 201 |
+
source_train_indices=source_train, source_held_events=source_events,
|
| 202 |
+
target_evaluation_events=archive["evaluation"], fitted_models=model,
|
| 203 |
+
input_sha256=sha(INPUT / "prepared" / task["file"]), methods={})
|
| 204 |
+
# Descriptive empirical shift; never use these laws to build a policy.
|
| 205 |
+
ps, pt = laws["source_held"], laws["target_evaluation"]
|
| 206 |
+
ms, mt = ps.sum((2, 3)), pt.sum((2, 3))
|
| 207 |
+
overlap = torch.minimum(ms, mt)
|
| 208 |
+
cs, ct = ps / ms[..., None, None].clamp_min(1e-300), pt / mt[..., None, None].clamp_min(1e-300)
|
| 209 |
+
conditional_tv = (.5 * (cs - ct).abs().sum((2, 3)) * overlap).sum() / overlap.sum().clamp_min(1e-300)
|
| 210 |
+
posterior_tv = (.5 * (cs.sum(3) - ct.sum(3)).abs().sum(2) * overlap).sum() / overlap.sum().clamp_min(1e-300)
|
| 211 |
+
shift = dict(task_index=ti, dataset=task["dataset"], environment=task["environment"], task=task["task"],
|
| 212 |
+
empirical_joint_tv=float(.5 * (ps-pt).abs().sum()), overlap_current_mass=float(overlap.sum()),
|
| 213 |
+
empirical_conditional_tv_overlap=float(conditional_tv), empirical_posterior_tv_overlap=float(posterior_tv),
|
| 214 |
+
target_mass_absent_source_held=float((mt * (ms == 0)).sum()), source_held_records=len(source_events), target_evaluation_records=len(archive["evaluation"]))
|
| 215 |
+
shifts.append(shift)
|
| 216 |
+
for method in METHODS:
|
| 217 |
+
code, k = encode(model, method)
|
| 218 |
+
compact = push(model, code, k)
|
| 219 |
+
policy = plan(compact)
|
| 220 |
+
fitted_loss = full_policy["expected"][:, 1:].mean((1, 2)) - policy["expected"][:, 1:].mean((1, 2))
|
| 221 |
+
state = dict(code=code.to(torch.uint8), compact=compact, choices=policy["choices"], values=policy["values"], expected=policy["expected"], traces={})
|
| 222 |
+
for domain, law in laws.items():
|
| 223 |
+
truth = push(law.expand(len(model), -1, -1, -1), code, k)
|
| 224 |
+
actual = realized(truth, policy)
|
| 225 |
+
sc = scores(compact, truth)
|
| 226 |
+
traces = []
|
| 227 |
+
for budget in [1, 2]:
|
| 228 |
+
for ci in range(3):
|
| 229 |
+
trace, utility = batched_replay(policy, code, events_by_domain[domain], budget, ci)
|
| 230 |
+
maximum_replay_error = max(maximum_replay_error, float((utility-actual["utility"][:, budget, ci]).abs().max()))
|
| 231 |
+
traces.append(trace)
|
| 232 |
+
state["traces"][domain] = torch.stack(traces, 1)
|
| 233 |
+
# Original independent replay API confirms batched access and aggregation.
|
| 234 |
+
check_i = next(i for i, m in enumerate(meta) if m["training"] == "target" and m["size"] == "64" and m["model"] == "support_mixture")
|
| 235 |
+
single = {**policy, "shape": (1, k), "choices": policy["choices"].reshape(3, len(meta), k, 3, 9)[:, check_i:check_i+1].reshape(3, k, 3, 9)}
|
| 236 |
+
a, q, seen = replay(single, code[check_i:check_i+1].to(torch.uint8), (events_by_domain[domain]//4)%64, events_by_domain[domain]%4, 2, 1)
|
| 237 |
+
assert torch.equal(torch.stack((a, q, seen), -1).to(torch.uint8), state["traces"][domain][check_i, 4])
|
| 238 |
+
for i, m in enumerate(meta):
|
| 239 |
+
if domain == "source_held" and m["training"] != "source":
|
| 240 |
+
continue
|
| 241 |
+
pred0 = float(policy["expected"][i, 0].mean())
|
| 242 |
+
predq = float(policy["expected"][i, 1:].mean())
|
| 243 |
+
act0 = float(actual["utility"][i, 0].mean())
|
| 244 |
+
actq = float(actual["utility"][i, 1:].mean())
|
| 245 |
+
row = dict(task_index=ti, dataset=task["dataset"], environment=task["environment"], task=task["task"], **m,
|
| 246 |
+
representation=method, evaluation=domain, evaluation_records=len(events_by_domain[domain]),
|
| 247 |
+
utility=actq, noquery_utility=act0, realized_query_increment=actq-act0,
|
| 248 |
+
predicted_utility=predq, predicted_noquery_utility=pred0, predicted_query_increment=predq-pred0,
|
| 249 |
+
query_increment_optimism=(predq-pred0)-(actq-act0), queries=float(actual["queries"][i, 1:].mean()),
|
| 250 |
+
fitted_compression_loss=float(fitted_loss[i]))
|
| 251 |
+
if m["model"] != "empirical_reference":
|
| 252 |
+
row.update({name:float(v[i]) for name, v in sc.items()})
|
| 253 |
+
raw_cond = model[i] / model[i].sum((1, 2), keepdim=True).clamp_min(1e-300)
|
| 254 |
+
loss_by_raw = -(law[0] * raw_cond.clamp_min(1e-300).log()).sum((1, 2))
|
| 255 |
+
# Same current-pattern weighting in both domains isolates
|
| 256 |
+
# conditional predictive mismatch from current-mass shifts.
|
| 257 |
+
row["common_current_joint_conditional_log_loss"] = float((loss_by_raw/law[0].sum((1,2)).clamp_min(1e-300)*overlap[0]).sum()/overlap.sum().clamp_min(1e-300))
|
| 258 |
+
rows.append(row)
|
| 259 |
+
# Exact same trained primary models/encoders/predictions as frozen study.
|
| 260 |
+
old = torch.load(INPUT / "evaluation" / f"primary_task_{ti:03}.pt", map_location=device, weights_only=True)
|
| 261 |
+
for i, m in enumerate(meta):
|
| 262 |
+
if m["training"] == "target" and m["size"] == "64" and (m["model"] == "support_mixture" or method == "identity"):
|
| 263 |
+
original = old[f'{m["sample_seed"]}:{m["model"]}:{method}']
|
| 264 |
+
assert torch.equal(code[i], original["code"][0].long())
|
| 265 |
+
maximum_original_error = max(maximum_original_error, float((policy["expected"][i]-original["expected"][0]).abs().max()))
|
| 266 |
+
saved["methods"][method] = state
|
| 267 |
+
torch.save(saved, out / "predictions" / f"task_{ti:03}.pt")
|
| 268 |
+
print(json.dumps(dict(task=ti, finished=len(shifts), tasks=len(manifest["tasks"]), elapsed_seconds=time.perf_counter()-start)), flush=True)
|
| 269 |
+
assert maximum_replay_error < 1e-12 and maximum_original_error < 1e-12
|
| 270 |
+
fields = sorted(set().union(*(r.keys() for r in rows)))
|
| 271 |
+
with gzip.open(out / "metrics.csv.gz", "wt", newline="") as f:
|
| 272 |
+
writer = csv.DictWriter(f, fieldnames=fields)
|
| 273 |
+
writer.writeheader(); writer.writerows(rows)
|
| 274 |
+
write(out / "shift_by_task.json", shifts)
|
| 275 |
+
torch.cuda.synchronize(device)
|
| 276 |
+
write(out / "execution.json", dict(material_passport=dict(material_type="experiment execution result", scientific_status="post-review exploratory", created_utc=utc()),
|
| 277 |
+
status="COMPLETED", metric_rows=len(rows), task_count=len(shifts), elapsed_seconds=time.perf_counter()-start,
|
| 278 |
+
maximum_replay_error=maximum_replay_error, maximum_original_primary_expected_error=maximum_original_error,
|
| 279 |
+
gpu=torch.cuda.get_device_name(device), device=device, torch_version=torch.__version__, python_version=platform.python_version(),
|
| 280 |
+
peak_gpu_allocated_bytes=torch.cuda.max_memory_allocated(device), peak_gpu_reserved_bytes=torch.cuda.max_memory_reserved(device),
|
| 281 |
+
script_sha256=sha(__file__), specification_sha256=sha(out / "specification.json")))
|
| 282 |
+
|
| 283 |
+
|
| 284 |
+
def aggregate(rows, field):
|
| 285 |
+
tasks = defaultdict(list)
|
| 286 |
+
for r in rows:
|
| 287 |
+
if field in r and r[field] != "":
|
| 288 |
+
tasks[(r["dataset"], r["environment"], r["task"])].append(float(r[field]))
|
| 289 |
+
envs = defaultdict(list)
|
| 290 |
+
for key, vals in tasks.items():
|
| 291 |
+
envs[key[:2]].append(sum(vals)/len(vals))
|
| 292 |
+
datasets = defaultdict(list)
|
| 293 |
+
for (d, e), vals in envs.items():
|
| 294 |
+
datasets[d].append(sum(vals)/len(vals))
|
| 295 |
+
ds = {d:sum(vals)/len(vals) for d, vals in datasets.items()}
|
| 296 |
+
return dict(overall=sum(ds.values())/len(ds), by_dataset=ds,
|
| 297 |
+
by_environment={d+":"+e:sum(v)/len(v) for (d,e),v in envs.items()}, tasks=len(tasks))
|
| 298 |
+
|
| 299 |
+
|
| 300 |
+
def recover_saved(out, device):
|
| 301 |
+
"""Export saved fits/traces after correcting only diagnostic query-charge dtype."""
|
| 302 |
+
assert not (out / "metrics.csv.gz").exists()
|
| 303 |
+
spec = json.loads((out / "specification.json").read_text())
|
| 304 |
+
correction = dict(created_utc=utc(), authorization="Parent authorized routine numerical correction and artifact-only recovery after receiving failure report", failed_log_sha256=sha(out / "run.log"),
|
| 305 |
+
original_specification_script_sha256=spec["script_sha256"], corrected_script_sha256=sha(__file__),
|
| 306 |
+
failure="Final equality assertion rejected float32 intermediate cost*integer_query_count in diagnostic batched replay utility; policy tables and traces unaffected",
|
| 307 |
+
correction="Multiply cost by queries.double(), matching original independent replay utility convention",
|
| 308 |
+
estimators_rerun=0, encoders_rerun=0, planners_rerun=0, policies_changed=0,
|
| 309 |
+
recovery="Compute verified metrics and summaries from all 57 already saved model/policy/record-trace artifacts")
|
| 310 |
+
write(out / "implementation_correction.json", correction)
|
| 311 |
+
for path, digest in spec["input_hashes"].items():
|
| 312 |
+
assert sha(ROOT / path) == digest
|
| 313 |
+
torch.cuda.set_device(device)
|
| 314 |
+
torch.backends.cuda.matmul.allow_tf32 = False
|
| 315 |
+
torch.cuda.reset_peak_memory_stats(device)
|
| 316 |
+
start = time.perf_counter()
|
| 317 |
+
rows, shifts = [], []
|
| 318 |
+
maxerr, maxoriginal, maxfloat32 = 0., 0., 0.
|
| 319 |
+
manifest = json.loads((INPUT / "prepared/manifest.json").read_text())
|
| 320 |
+
for task in manifest["tasks"]:
|
| 321 |
+
ti = task["index"]
|
| 322 |
+
saved = torch.load(out / "predictions" / f"task_{ti:03}.pt", map_location=device, weights_only=True)
|
| 323 |
+
assert saved["input_sha256"] == sha(INPUT / "prepared" / task["file"])
|
| 324 |
+
meta, model = saved["metadata"], saved["fitted_models"]
|
| 325 |
+
full_expected = saved["methods"]["identity"]["expected"]
|
| 326 |
+
events_by_domain = {"target_evaluation":saved["target_evaluation_events"],"source_held":saved["source_held_events"]}
|
| 327 |
+
laws = {d:normalized_counts(e) for d,e in events_by_domain.items()}
|
| 328 |
+
ps, pt = laws["source_held"], laws["target_evaluation"]
|
| 329 |
+
ms, mt = ps.sum((2,3)),pt.sum((2,3))
|
| 330 |
+
overlap = torch.minimum(ms,mt)
|
| 331 |
+
cs,ct = ps/ms[...,None,None].clamp_min(1e-300),pt/mt[...,None,None].clamp_min(1e-300)
|
| 332 |
+
shifts.append(dict(task_index=ti,dataset=task["dataset"],environment=task["environment"],task=task["task"],
|
| 333 |
+
empirical_joint_tv=float(.5*(ps-pt).abs().sum()),overlap_current_mass=float(overlap.sum()),
|
| 334 |
+
empirical_conditional_tv_overlap=float((.5*(cs-ct).abs().sum((2,3))*overlap).sum()/overlap.sum().clamp_min(1e-300)),
|
| 335 |
+
empirical_posterior_tv_overlap=float((.5*(cs.sum(3)-ct.sum(3)).abs().sum(2)*overlap).sum()/overlap.sum().clamp_min(1e-300)),
|
| 336 |
+
target_mass_absent_source_held=float((mt*(ms==0)).sum()),source_held_records=len(events_by_domain["source_held"]),target_evaluation_records=len(events_by_domain["target_evaluation"])))
|
| 337 |
+
original = torch.load(INPUT / "evaluation" / f"primary_task_{ti:03}.pt", map_location=device, weights_only=True)
|
| 338 |
+
for method,state in saved["methods"].items():
|
| 339 |
+
code,compact = state["code"].long(),state["compact"]
|
| 340 |
+
k=compact.shape[1]
|
| 341 |
+
policy=dict(choices=state["choices"],values=state["values"],expected=state["expected"],costs=(0.,.01,.05),shape=(len(meta),k))
|
| 342 |
+
fitted_loss=full_expected[:,1:].mean((1,2))-policy["expected"][:,1:].mean((1,2))
|
| 343 |
+
for domain,law in laws.items():
|
| 344 |
+
events=events_by_domain[domain]
|
| 345 |
+
truth=push(law.expand(len(model),-1,-1,-1),code,k)
|
| 346 |
+
actual=realized(truth,policy)
|
| 347 |
+
sc=scores(compact,truth)
|
| 348 |
+
# Check every saved action/count/mask trace against newly evaluated
|
| 349 |
+
# fixed tables, and every double-precision charge against exact law replay.
|
| 350 |
+
for bi,budget in enumerate([1,2]):
|
| 351 |
+
for ci,cost in enumerate(policy["costs"]):
|
| 352 |
+
recorded=state["traces"][domain][:,bi*3+ci]
|
| 353 |
+
recomputed,utility=batched_replay(policy,code,events,budget,ci)
|
| 354 |
+
assert torch.equal(recorded,recomputed)
|
| 355 |
+
maxerr=max(maxerr,float((utility-actual["utility"][:,budget,ci]).abs().max()))
|
| 356 |
+
# Retain quantitative evidence for original failure.
|
| 357 |
+
q=recorded[:,:,1].long()
|
| 358 |
+
maxfloat32=max(maxfloat32,float((cost*q-cost*q.double()).abs().max()))
|
| 359 |
+
check_i=next(i for i,m in enumerate(meta) if m["training"]=="target" and m["size"]=="64" and m["model"]=="support_mixture")
|
| 360 |
+
single={**policy,"shape":(1,k),"choices":policy["choices"].reshape(3,len(meta),k,3,9)[:,check_i:check_i+1].reshape(3,k,3,9)}
|
| 361 |
+
a,q,seen=replay(single,code[check_i:check_i+1].to(torch.uint8),(events//4)%64,events%4,2,1)
|
| 362 |
+
assert torch.equal(torch.stack((a,q,seen),-1).to(torch.uint8),state["traces"][domain][check_i,4])
|
| 363 |
+
independent_u=torch.where(a==2,0.,torch.where(a==events//256,1.,-2.)).double()-.01*q.double()
|
| 364 |
+
maxerr=max(maxerr,abs(float(independent_u.mean())-float(actual["utility"][check_i,2,1])))
|
| 365 |
+
for i,m in enumerate(meta):
|
| 366 |
+
if domain=="source_held" and m["training"]!="source":continue
|
| 367 |
+
p0,pq=float(policy["expected"][i,0].mean()),float(policy["expected"][i,1:].mean())
|
| 368 |
+
a0,aq=float(actual["utility"][i,0].mean()),float(actual["utility"][i,1:].mean())
|
| 369 |
+
row=dict(task_index=ti,dataset=task["dataset"],environment=task["environment"],task=task["task"],**m,
|
| 370 |
+
representation=method,evaluation=domain,evaluation_records=len(events),utility=aq,noquery_utility=a0,
|
| 371 |
+
realized_query_increment=aq-a0,predicted_utility=pq,predicted_noquery_utility=p0,predicted_query_increment=pq-p0,
|
| 372 |
+
query_increment_optimism=(pq-p0)-(aq-a0),queries=float(actual["queries"][i,1:].mean()),fitted_compression_loss=float(fitted_loss[i]))
|
| 373 |
+
if m["model"]!="empirical_reference":
|
| 374 |
+
row.update({name:float(v[i]) for name,v in sc.items()})
|
| 375 |
+
rawcond=model[i]/model[i].sum((1,2),keepdim=True).clamp_min(1e-300)
|
| 376 |
+
loss=-(law[0]*rawcond.clamp_min(1e-300).log()).sum((1,2))
|
| 377 |
+
row["common_current_joint_conditional_log_loss"]=float((loss/law[0].sum((1,2)).clamp_min(1e-300)*overlap[0]).sum()/overlap.sum().clamp_min(1e-300))
|
| 378 |
+
rows.append(row)
|
| 379 |
+
for i,m in enumerate(meta):
|
| 380 |
+
if m["training"]=="target" and m["size"]=="64" and (m["model"]=="support_mixture" or method=="identity"):
|
| 381 |
+
old=original[f'{m["sample_seed"]}:{m["model"]}:{method}']
|
| 382 |
+
assert torch.equal(code[i],old["code"][0].long())
|
| 383 |
+
maxoriginal=max(maxoriginal,float((policy["expected"][i]-old["expected"][0]).abs().max()))
|
| 384 |
+
print(json.dumps(dict(recovered_task=ti,finished=len(shifts),elapsed_seconds=time.perf_counter()-start)),flush=True)
|
| 385 |
+
assert maxerr<1e-12 and maxoriginal<1e-12
|
| 386 |
+
fields=sorted(set().union(*(r.keys() for r in rows)))
|
| 387 |
+
with gzip.open(out/"metrics.csv.gz","wt",newline="") as f:
|
| 388 |
+
writer=csv.DictWriter(f,fieldnames=fields);writer.writeheader();writer.writerows(rows)
|
| 389 |
+
write(out/"shift_by_task.json",shifts)
|
| 390 |
+
write(out/"execution.json",dict(material_passport=dict(material_type="experiment execution result",scientific_status="post-review exploratory",created_utc=utc()),
|
| 391 |
+
status="COMPLETED_FROM_SAVED_ARTIFACTS_AFTER_DIAGNOSTIC_DTYPE_CORRECTION",metric_rows=len(rows),task_count=len(shifts),
|
| 392 |
+
original_fit_and_trace_seconds=11.562392784981057,recovery_seconds=time.perf_counter()-start,
|
| 393 |
+
maximum_replay_error=maxerr,maximum_original_primary_expected_error=maxoriginal,maximum_original_float32_query_charge_error=maxfloat32,
|
| 394 |
+
gpu=torch.cuda.get_device_name(device),device=device,torch_version=torch.__version__,python_version=platform.python_version(),
|
| 395 |
+
peak_gpu_allocated_bytes=torch.cuda.max_memory_allocated(device),peak_gpu_reserved_bytes=torch.cuda.max_memory_reserved(device),
|
| 396 |
+
script_sha256=sha(__file__),specification_sha256=sha(out/"specification.json"),original_results_and_gates_unchanged=True))
|
| 397 |
+
|
| 398 |
+
|
| 399 |
+
def summarize(out, device):
|
| 400 |
+
with gzip.open(out / "metrics.csv.gz", "rt") as f:
|
| 401 |
+
rows = list(csv.DictReader(f))
|
| 402 |
+
keys = ["training", "size", "model", "representation", "evaluation"]
|
| 403 |
+
fields = ["utility", "noquery_utility", "realized_query_increment", "predicted_utility", "predicted_noquery_utility", "predicted_query_increment", "query_increment_optimism", "queries", "fitted_compression_loss", "posterior_brier", "posterior_log_loss", "joint_conditional_log_loss", "observation_given_h_log_loss", "common_current_joint_conditional_log_loss"]
|
| 404 |
+
groups = defaultdict(list)
|
| 405 |
+
for r in rows:
|
| 406 |
+
groups[tuple(r[k] for k in keys)].append(r)
|
| 407 |
+
overall = []
|
| 408 |
+
for key, rs in groups.items():
|
| 409 |
+
obj = dict(zip(keys, key))
|
| 410 |
+
for f in fields:
|
| 411 |
+
if any(r.get(f, "") != "" for r in rs):
|
| 412 |
+
obj[f] = aggregate(rs, f)
|
| 413 |
+
overall.append(obj)
|
| 414 |
+
write(out / "overall.json", overall)
|
| 415 |
+
# Contrasts pair at task and seed, then resample target environments only.
|
| 416 |
+
contrasts = []
|
| 417 |
+
def contrast(label, left, right, field="utility"):
|
| 418 |
+
a = { (r["dataset"],r["environment"],r["task"],r["sample_seed"]):r for r in rows if left(r) }
|
| 419 |
+
b = { (r["dataset"],r["environment"],r["task"],r["sample_seed"]):r for r in rows if right(r) }
|
| 420 |
+
pairs = []
|
| 421 |
+
for key in sorted(a.keys() & b.keys()):
|
| 422 |
+
pairs.append(dict(dataset=key[0],environment=key[1],task=key[2],difference=float(a[key][field])-float(b[key][field])))
|
| 423 |
+
if not pairs:
|
| 424 |
+
return
|
| 425 |
+
agg = aggregate(pairs, "difference")
|
| 426 |
+
g = torch.Generator(device=device).manual_seed(20261109)
|
| 427 |
+
boots = []
|
| 428 |
+
for d in sorted(agg["by_dataset"]):
|
| 429 |
+
vals = torch.tensor([v for name,v in sorted(agg["by_environment"].items()) if name.startswith(d+":")], device=device, dtype=torch.float64)
|
| 430 |
+
ix = torch.randint(len(vals), (10000,len(vals)), generator=g,device=device)
|
| 431 |
+
boots.append(vals[ix].mean(1))
|
| 432 |
+
low, high = torch.quantile(torch.stack(boots).mean(0),torch.tensor([.025,.975],device=device,dtype=torch.float64)).tolist()
|
| 433 |
+
contrasts.append(dict(label=label, field=field, **agg, descriptive_paired_95_interval=[low,high], paired_fit_rows=len(pairs), environment_resampling_only=True))
|
| 434 |
+
base = lambda r:r["model"] == "support_mixture" and r["evaluation"] == "target_evaluation"
|
| 435 |
+
for size in ["64","256","all"]:
|
| 436 |
+
contrast("joint_minus_posterior_target_"+size,
|
| 437 |
+
lambda r:base(r) and r["training"]=="target" and r["size"]==size and r["representation"]=="query_joint",
|
| 438 |
+
lambda r:base(r) and r["training"]=="target" and r["size"]==size and r["representation"]=="posterior_clusters")
|
| 439 |
+
for method in METHODS:
|
| 440 |
+
contrast("target_minus_source_"+size+"_"+method,
|
| 441 |
+
lambda r:base(r) and r["training"]=="target" and r["size"]==size and r["representation"]==method,
|
| 442 |
+
lambda r:base(r) and r["training"]=="source" and r["size"]==size and r["representation"]==method)
|
| 443 |
+
for method in METHODS:
|
| 444 |
+
for size in ["256","all"]:
|
| 445 |
+
for field in ["utility","noquery_utility","realized_query_increment","joint_conditional_log_loss"]:
|
| 446 |
+
contrast("target_"+size+"_minus_64_"+method,
|
| 447 |
+
lambda r:base(r) and r["training"]=="target" and r["size"]==size and r["representation"]==method,
|
| 448 |
+
lambda r:base(r) and r["training"]=="target" and r["size"]=="64" and r["representation"]==method, field)
|
| 449 |
+
for size in ["64","256","all"]:
|
| 450 |
+
for field in ["utility","noquery_utility","realized_query_increment","joint_conditional_log_loss","common_current_joint_conditional_log_loss"]:
|
| 451 |
+
contrast("source_fixed_policy_target_minus_sourceheld_"+size,
|
| 452 |
+
lambda r:r["model"]=="support_mixture" and r["training"]=="source" and r["size"]==size and r["representation"]=="identity" and r["evaluation"]=="target_evaluation",
|
| 453 |
+
lambda r:r["model"]=="support_mixture" and r["training"]=="source" and r["size"]==size and r["representation"]=="identity" and r["evaluation"]=="source_held", field)
|
| 454 |
+
write(out / "paired_contrasts.json",dict(scientific_status="post-review exploratory; unadjusted descriptive intervals; no new confirmation", bootstrap_unit="Target environment stratified by fixed dataset; tasks and seeds collapsed first", contrasts=contrasts))
|
| 455 |
+
shifts = json.loads((out / "shift_by_task.json").read_text())
|
| 456 |
+
write(out / "shift_overall.json", {f:aggregate(shifts,f) for f in ["empirical_joint_tv","overlap_current_mass","empirical_conditional_tv_overlap","empirical_posterior_tv_overlap","target_mass_absent_source_held"]})
|
| 457 |
+
# Compact tables retain the original weighting and explicitly flag subsets.
|
| 458 |
+
table=[]
|
| 459 |
+
def table_row(label,selected):
|
| 460 |
+
taskset={(r["dataset"],r["environment"],r["task"]) for r in selected}
|
| 461 |
+
envset={(r["dataset"],r["environment"]) for r in selected}
|
| 462 |
+
samples=[int(r["sample_count"]) for r in selected]
|
| 463 |
+
table.append(dict(analysis=label,tasks=len(taskset),
|
| 464 |
+
hydraulic_tasks=sum(x[0]=="hydraulic" for x in taskset),gas_tasks=sum(x[0]=="gas" for x in taskset),
|
| 465 |
+
hydraulic_environments=sum(x[0]=="hydraulic" for x in envset),gas_environments=sum(x[0]=="gas" for x in envset),
|
| 466 |
+
labels_min=min(samples),labels_max=max(samples),
|
| 467 |
+
**{f:aggregate(selected,f)["overall"] for f in fields if any(r.get(f,"")!="" for r in selected)}))
|
| 468 |
+
for training in ["target","source"]:
|
| 469 |
+
for size in ["64","all"]:
|
| 470 |
+
for method in (METHODS if training=="target" else ["identity"]):
|
| 471 |
+
for evaluation in (["target_evaluation"] if training=="target" else ["source_held","target_evaluation"]):
|
| 472 |
+
table_row(f"{training}_{size}_{method}_on_{evaluation}_all57",[r for r in rows if r["training"]==training and r["size"]==size and r["model"]=="support_mixture" and r["representation"]==method and r["evaluation"]==evaluation])
|
| 473 |
+
eligible256={r["task_index"] for r in rows if r["training"]=="target" and r["size"]=="256"}
|
| 474 |
+
for size in ["64","256"]:
|
| 475 |
+
for method in METHODS:
|
| 476 |
+
table_row(f"target_{size}_{method}_on_matched256eligible43",[r for r in rows if r["task_index"] in eligible256 and r["training"]=="target" and r["size"]==size and r["model"]=="support_mixture" and r["representation"]==method and r["evaluation"]=="target_evaluation"])
|
| 477 |
+
for method in METHODS:
|
| 478 |
+
table_row(f"empirical_alladaptation_{method}_on_target_evaluation_all57",[r for r in rows if r["model"]=="empirical_reference" and r["representation"]==method])
|
| 479 |
+
tablefields=sorted(set().union(*(r.keys() for r in table)))
|
| 480 |
+
with (out/"article_table.csv").open("w",newline="") as f:
|
| 481 |
+
writer=csv.DictWriter(f,fieldnames=tablefields);writer.writeheader();writer.writerows(table)
|
| 482 |
+
write(out/"table_provenance.json",dict(created_utc=utc(),script_sha256=sha(__file__),
|
| 483 |
+
input_metrics_sha256=sha(out/"metrics.csv.gz"),table_sha256=sha(out/"article_table.csv"),
|
| 484 |
+
design_changed=False,scientific_status="post-review exploratory",weighting="As specification; each table row lists support and independent environment counts",
|
| 485 |
+
caution="The 256-label subset has only one hydraulic environment; empirical TV contains finite-count noise and is not a causal decomposition. Gas record artifacts remain research-only local data."))
|
| 486 |
+
print(json.dumps(dict(summary_groups=len(overall), contrasts=len(contrasts))), flush=True)
|
| 487 |
+
|
| 488 |
+
|
| 489 |
+
def main():
|
| 490 |
+
global INPUT
|
| 491 |
+
parser = argparse.ArgumentParser(description=__doc__)
|
| 492 |
+
parser.add_argument("stage", choices=["specify","run","recover","summarize"])
|
| 493 |
+
parser.add_argument("--output", type=Path, default=DEFAULT_OUTPUT)
|
| 494 |
+
parser.add_argument("--device", default="cuda:4")
|
| 495 |
+
parser.add_argument("--release", type=Path, default=RELEASE, help="Unmodified release containing unionmind/ and protocols/")
|
| 496 |
+
parser.add_argument("--input", type=Path, default=INPUT, help="Previously opened external study artifacts: prepared/ and evaluation/")
|
| 497 |
+
args = parser.parse_args()
|
| 498 |
+
INPUT = args.input.resolve()
|
| 499 |
+
out = args.output.resolve()
|
| 500 |
+
if args.stage == "specify":
|
| 501 |
+
out.mkdir(parents=True, exist_ok=True)
|
| 502 |
+
(out / "predictions").mkdir(exist_ok=True)
|
| 503 |
+
specification(out,args.device)
|
| 504 |
+
elif args.stage == "run":
|
| 505 |
+
evaluate(out,args.device)
|
| 506 |
+
summarize(out,args.device)
|
| 507 |
+
elif args.stage == "recover":
|
| 508 |
+
recover_saved(out,args.device)
|
| 509 |
+
summarize(out,args.device)
|
| 510 |
+
else:
|
| 511 |
+
summarize(out,args.device)
|
| 512 |
+
|
| 513 |
+
|
| 514 |
+
if __name__ == "__main__":
|
| 515 |
+
main()
|
revision_r1/external/shift_by_task.json
ADDED
|
@@ -0,0 +1,743 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
[
|
| 2 |
+
{
|
| 3 |
+
"task_index": 0,
|
| 4 |
+
"dataset": "hydraulic",
|
| 5 |
+
"environment": "context1=73",
|
| 6 |
+
"task": "cooler",
|
| 7 |
+
"empirical_joint_tv": 0.5891016200294551,
|
| 8 |
+
"overlap_current_mass": 0.4204221894943544,
|
| 9 |
+
"empirical_conditional_tv_overlap": 0.022652965903783285,
|
| 10 |
+
"empirical_posterior_tv_overlap": 0.0,
|
| 11 |
+
"target_mass_absent_source_held": 0.41904761904761906,
|
| 12 |
+
"source_held_records": 97,
|
| 13 |
+
"target_evaluation_records": 210
|
| 14 |
+
},
|
| 15 |
+
{
|
| 16 |
+
"task_index": 1,
|
| 17 |
+
"dataset": "hydraulic",
|
| 18 |
+
"environment": "context1=80",
|
| 19 |
+
"task": "cooler",
|
| 20 |
+
"empirical_joint_tv": 0.6455194290245836,
|
| 21 |
+
"overlap_current_mass": 0.4006344171292625,
|
| 22 |
+
"empirical_conditional_tv_overlap": 0.11520190023752969,
|
| 23 |
+
"empirical_posterior_tv_overlap": 0.0,
|
| 24 |
+
"target_mass_absent_source_held": 0.4,
|
| 25 |
+
"source_held_records": 97,
|
| 26 |
+
"target_evaluation_records": 130
|
| 27 |
+
},
|
| 28 |
+
{
|
| 29 |
+
"task_index": 2,
|
| 30 |
+
"dataset": "hydraulic",
|
| 31 |
+
"environment": "context1=90",
|
| 32 |
+
"task": "cooler",
|
| 33 |
+
"empirical_joint_tv": 0.4949390815370197,
|
| 34 |
+
"overlap_current_mass": 0.5638238050609184,
|
| 35 |
+
"empirical_conditional_tv_overlap": 0.13436391843971632,
|
| 36 |
+
"empirical_posterior_tv_overlap": 2.86413918586809e-17,
|
| 37 |
+
"target_mass_absent_source_held": 0.3090909090909091,
|
| 38 |
+
"source_held_records": 97,
|
| 39 |
+
"target_evaluation_records": 110
|
| 40 |
+
},
|
| 41 |
+
{
|
| 42 |
+
"task_index": 3,
|
| 43 |
+
"dataset": "hydraulic",
|
| 44 |
+
"environment": "context0=3",
|
| 45 |
+
"task": "valve",
|
| 46 |
+
"empirical_joint_tv": 0.9999999999999999,
|
| 47 |
+
"overlap_current_mass": 0.3076923076923077,
|
| 48 |
+
"empirical_conditional_tv_overlap": 1.0,
|
| 49 |
+
"empirical_posterior_tv_overlap": 0.34560439560439554,
|
| 50 |
+
"target_mass_absent_source_held": 0.30107526881720426,
|
| 51 |
+
"source_held_records": 91,
|
| 52 |
+
"target_evaluation_records": 465
|
| 53 |
+
},
|
| 54 |
+
{
|
| 55 |
+
"task_index": 4,
|
| 56 |
+
"dataset": "hydraulic",
|
| 57 |
+
"environment": "context0=20",
|
| 58 |
+
"task": "valve",
|
| 59 |
+
"empirical_joint_tv": 1.0,
|
| 60 |
+
"overlap_current_mass": 0.3076923076923077,
|
| 61 |
+
"empirical_conditional_tv_overlap": 1.0,
|
| 62 |
+
"empirical_posterior_tv_overlap": 0.3014660012476606,
|
| 63 |
+
"target_mass_absent_source_held": 0.10196078431372549,
|
| 64 |
+
"source_held_records": 91,
|
| 65 |
+
"target_evaluation_records": 510
|
| 66 |
+
},
|
| 67 |
+
{
|
| 68 |
+
"task_index": 5,
|
| 69 |
+
"dataset": "hydraulic",
|
| 70 |
+
"environment": "context0=3",
|
| 71 |
+
"task": "pump",
|
| 72 |
+
"empirical_joint_tv": 0.9999999999999999,
|
| 73 |
+
"overlap_current_mass": 0.30769230769230765,
|
| 74 |
+
"empirical_conditional_tv_overlap": 1.0,
|
| 75 |
+
"empirical_posterior_tv_overlap": 0.8646153846153845,
|
| 76 |
+
"target_mass_absent_source_held": 0.30107526881720426,
|
| 77 |
+
"source_held_records": 91,
|
| 78 |
+
"target_evaluation_records": 465
|
| 79 |
+
},
|
| 80 |
+
{
|
| 81 |
+
"task_index": 6,
|
| 82 |
+
"dataset": "hydraulic",
|
| 83 |
+
"environment": "context0=20",
|
| 84 |
+
"task": "pump",
|
| 85 |
+
"empirical_joint_tv": 1.0,
|
| 86 |
+
"overlap_current_mass": 0.30769230769230765,
|
| 87 |
+
"empirical_conditional_tv_overlap": 1.0,
|
| 88 |
+
"empirical_posterior_tv_overlap": 0.6157205240174672,
|
| 89 |
+
"target_mass_absent_source_held": 0.10196078431372549,
|
| 90 |
+
"source_held_records": 91,
|
| 91 |
+
"target_evaluation_records": 510
|
| 92 |
+
},
|
| 93 |
+
{
|
| 94 |
+
"task_index": 7,
|
| 95 |
+
"dataset": "hydraulic",
|
| 96 |
+
"environment": "context0=3",
|
| 97 |
+
"task": "accumulator",
|
| 98 |
+
"empirical_joint_tv": 0.9999999999999999,
|
| 99 |
+
"overlap_current_mass": 0.30769230769230765,
|
| 100 |
+
"empirical_conditional_tv_overlap": 1.0,
|
| 101 |
+
"empirical_posterior_tv_overlap": 0.7753846153846153,
|
| 102 |
+
"target_mass_absent_source_held": 0.30107526881720426,
|
| 103 |
+
"source_held_records": 91,
|
| 104 |
+
"target_evaluation_records": 465
|
| 105 |
+
},
|
| 106 |
+
{
|
| 107 |
+
"task_index": 8,
|
| 108 |
+
"dataset": "hydraulic",
|
| 109 |
+
"environment": "context0=20",
|
| 110 |
+
"task": "accumulator",
|
| 111 |
+
"empirical_joint_tv": 1.0,
|
| 112 |
+
"overlap_current_mass": 0.30769230769230765,
|
| 113 |
+
"empirical_conditional_tv_overlap": 0.9999999999999998,
|
| 114 |
+
"empirical_posterior_tv_overlap": 0.08951965065502185,
|
| 115 |
+
"target_mass_absent_source_held": 0.10196078431372549,
|
| 116 |
+
"source_held_records": 91,
|
| 117 |
+
"target_evaluation_records": 510
|
| 118 |
+
},
|
| 119 |
+
{
|
| 120 |
+
"task_index": 9,
|
| 121 |
+
"dataset": "gas",
|
| 122 |
+
"environment": "batch6",
|
| 123 |
+
"task": "gas1_vs_2",
|
| 124 |
+
"empirical_joint_tv": 0.6525147928994083,
|
| 125 |
+
"overlap_current_mass": 0.37140039447731754,
|
| 126 |
+
"empirical_conditional_tv_overlap": 0.0967938892854072,
|
| 127 |
+
"empirical_posterior_tv_overlap": 0.04700992883347823,
|
| 128 |
+
"target_mass_absent_source_held": 0.022916666666666665,
|
| 129 |
+
"source_held_records": 338,
|
| 130 |
+
"target_evaluation_records": 480
|
| 131 |
+
},
|
| 132 |
+
{
|
| 133 |
+
"task_index": 10,
|
| 134 |
+
"dataset": "gas",
|
| 135 |
+
"environment": "batch7",
|
| 136 |
+
"task": "gas1_vs_2",
|
| 137 |
+
"empirical_joint_tv": 0.49545848565936756,
|
| 138 |
+
"overlap_current_mass": 0.5531602155805978,
|
| 139 |
+
"empirical_conditional_tv_overlap": 0.17988007041133974,
|
| 140 |
+
"empirical_posterior_tv_overlap": 0.07768418217812881,
|
| 141 |
+
"target_mass_absent_source_held": 0.2372611464968153,
|
| 142 |
+
"source_held_records": 338,
|
| 143 |
+
"target_evaluation_records": 628
|
| 144 |
+
},
|
| 145 |
+
{
|
| 146 |
+
"task_index": 13,
|
| 147 |
+
"dataset": "gas",
|
| 148 |
+
"environment": "batch10",
|
| 149 |
+
"task": "gas1_vs_2",
|
| 150 |
+
"empirical_joint_tv": 0.7252295449908184,
|
| 151 |
+
"overlap_current_mass": 0.4283411548663538,
|
| 152 |
+
"empirical_conditional_tv_overlap": 0.5057037565792537,
|
| 153 |
+
"empirical_posterior_tv_overlap": 0.497653454097474,
|
| 154 |
+
"target_mass_absent_source_held": 0.22586206896551725,
|
| 155 |
+
"source_held_records": 338,
|
| 156 |
+
"target_evaluation_records": 580
|
| 157 |
+
},
|
| 158 |
+
{
|
| 159 |
+
"task_index": 14,
|
| 160 |
+
"dataset": "gas",
|
| 161 |
+
"environment": "batch6",
|
| 162 |
+
"task": "gas1_vs_3",
|
| 163 |
+
"empirical_joint_tv": 0.5750790638836178,
|
| 164 |
+
"overlap_current_mass": 0.4597780461158071,
|
| 165 |
+
"empirical_conditional_tv_overlap": 0.21295510392558914,
|
| 166 |
+
"empirical_posterior_tv_overlap": 0.07479508985261864,
|
| 167 |
+
"target_mass_absent_source_held": 0.07624633431085043,
|
| 168 |
+
"source_held_records": 255,
|
| 169 |
+
"target_evaluation_records": 341
|
| 170 |
+
},
|
| 171 |
+
{
|
| 172 |
+
"task_index": 15,
|
| 173 |
+
"dataset": "gas",
|
| 174 |
+
"environment": "batch7",
|
| 175 |
+
"task": "gas1_vs_3",
|
| 176 |
+
"empirical_joint_tv": 0.5147367608460707,
|
| 177 |
+
"overlap_current_mass": 0.5225412999845607,
|
| 178 |
+
"empirical_conditional_tv_overlap": 0.10063793699050405,
|
| 179 |
+
"empirical_posterior_tv_overlap": 0.029726850735940166,
|
| 180 |
+
"target_mass_absent_source_held": 0.12204724409448818,
|
| 181 |
+
"source_held_records": 255,
|
| 182 |
+
"target_evaluation_records": 508
|
| 183 |
+
},
|
| 184 |
+
{
|
| 185 |
+
"task_index": 17,
|
| 186 |
+
"dataset": "gas",
|
| 187 |
+
"environment": "batch9",
|
| 188 |
+
"task": "gas1_vs_3",
|
| 189 |
+
"empirical_joint_tv": 0.7607843137254902,
|
| 190 |
+
"overlap_current_mass": 0.2627450980392157,
|
| 191 |
+
"empirical_conditional_tv_overlap": 0.272636815920398,
|
| 192 |
+
"empirical_posterior_tv_overlap": 0.18308457711442785,
|
| 193 |
+
"target_mass_absent_source_held": 0.0,
|
| 194 |
+
"source_held_records": 255,
|
| 195 |
+
"target_evaluation_records": 90
|
| 196 |
+
},
|
| 197 |
+
{
|
| 198 |
+
"task_index": 18,
|
| 199 |
+
"dataset": "gas",
|
| 200 |
+
"environment": "batch10",
|
| 201 |
+
"task": "gas1_vs_3",
|
| 202 |
+
"empirical_joint_tv": 0.7049428104575164,
|
| 203 |
+
"overlap_current_mass": 0.4061887254901961,
|
| 204 |
+
"empirical_conditional_tv_overlap": 0.4271407873087596,
|
| 205 |
+
"empirical_posterior_tv_overlap": 0.34255774684355955,
|
| 206 |
+
"target_mass_absent_source_held": 0.06770833333333333,
|
| 207 |
+
"source_held_records": 255,
|
| 208 |
+
"target_evaluation_records": 576
|
| 209 |
+
},
|
| 210 |
+
{
|
| 211 |
+
"task_index": 20,
|
| 212 |
+
"dataset": "gas",
|
| 213 |
+
"environment": "batch7",
|
| 214 |
+
"task": "gas1_vs_4",
|
| 215 |
+
"empirical_joint_tv": 0.5835438073574398,
|
| 216 |
+
"overlap_current_mass": 0.41809553690485524,
|
| 217 |
+
"empirical_conditional_tv_overlap": 0.15349406350528697,
|
| 218 |
+
"empirical_posterior_tv_overlap": 0.14940959760816513,
|
| 219 |
+
"target_mass_absent_source_held": 0.03934426229508197,
|
| 220 |
+
"source_held_records": 209,
|
| 221 |
+
"target_evaluation_records": 610
|
| 222 |
+
},
|
| 223 |
+
{
|
| 224 |
+
"task_index": 23,
|
| 225 |
+
"dataset": "gas",
|
| 226 |
+
"environment": "batch10",
|
| 227 |
+
"task": "gas1_vs_4",
|
| 228 |
+
"empirical_joint_tv": 0.6279124193883919,
|
| 229 |
+
"overlap_current_mass": 0.4050689966021773,
|
| 230 |
+
"empirical_conditional_tv_overlap": 0.13612500940242103,
|
| 231 |
+
"empirical_posterior_tv_overlap": 0.07040986774360053,
|
| 232 |
+
"target_mass_absent_source_held": 0.04710144927536232,
|
| 233 |
+
"source_held_records": 209,
|
| 234 |
+
"target_evaluation_records": 552
|
| 235 |
+
},
|
| 236 |
+
{
|
| 237 |
+
"task_index": 24,
|
| 238 |
+
"dataset": "gas",
|
| 239 |
+
"environment": "batch6",
|
| 240 |
+
"task": "gas1_vs_5",
|
| 241 |
+
"empirical_joint_tv": 0.4761886761886762,
|
| 242 |
+
"overlap_current_mass": 0.6386964386964387,
|
| 243 |
+
"empirical_conditional_tv_overlap": 0.21147300294298005,
|
| 244 |
+
"empirical_posterior_tv_overlap": 0.19232550737466725,
|
| 245 |
+
"target_mass_absent_source_held": 0.018711018711018712,
|
| 246 |
+
"source_held_records": 385,
|
| 247 |
+
"target_evaluation_records": 481
|
| 248 |
+
},
|
| 249 |
+
{
|
| 250 |
+
"task_index": 25,
|
| 251 |
+
"dataset": "gas",
|
| 252 |
+
"environment": "batch7",
|
| 253 |
+
"task": "gas1_vs_5",
|
| 254 |
+
"empirical_joint_tv": 0.5707052441229656,
|
| 255 |
+
"overlap_current_mass": 0.5403254972875227,
|
| 256 |
+
"empirical_conditional_tv_overlap": 0.2699232523524929,
|
| 257 |
+
"empirical_posterior_tv_overlap": 0.25119377854739744,
|
| 258 |
+
"target_mass_absent_source_held": 0.09945750452079566,
|
| 259 |
+
"source_held_records": 385,
|
| 260 |
+
"target_evaluation_records": 553
|
| 261 |
+
},
|
| 262 |
+
{
|
| 263 |
+
"task_index": 26,
|
| 264 |
+
"dataset": "gas",
|
| 265 |
+
"environment": "batch8",
|
| 266 |
+
"task": "gas1_vs_5",
|
| 267 |
+
"empirical_joint_tv": 0.590566037735849,
|
| 268 |
+
"overlap_current_mass": 0.40943396226415096,
|
| 269 |
+
"empirical_conditional_tv_overlap": 0.07292505923842325,
|
| 270 |
+
"empirical_posterior_tv_overlap": 0.07292505923842325,
|
| 271 |
+
"target_mass_absent_source_held": 0.0,
|
| 272 |
+
"source_held_records": 385,
|
| 273 |
+
"target_evaluation_records": 106
|
| 274 |
+
},
|
| 275 |
+
{
|
| 276 |
+
"task_index": 28,
|
| 277 |
+
"dataset": "gas",
|
| 278 |
+
"environment": "batch10",
|
| 279 |
+
"task": "gas1_vs_5",
|
| 280 |
+
"empirical_joint_tv": 0.7159182366928847,
|
| 281 |
+
"overlap_current_mass": 0.7303228461679165,
|
| 282 |
+
"empirical_conditional_tv_overlap": 0.6639737282499998,
|
| 283 |
+
"empirical_posterior_tv_overlap": 0.652671781560124,
|
| 284 |
+
"target_mass_absent_source_held": 0.08274647887323942,
|
| 285 |
+
"source_held_records": 385,
|
| 286 |
+
"target_evaluation_records": 568
|
| 287 |
+
},
|
| 288 |
+
{
|
| 289 |
+
"task_index": 29,
|
| 290 |
+
"dataset": "gas",
|
| 291 |
+
"environment": "batch6",
|
| 292 |
+
"task": "gas1_vs_6",
|
| 293 |
+
"empirical_joint_tv": 0.6327683615819208,
|
| 294 |
+
"overlap_current_mass": 0.37160800603295874,
|
| 295 |
+
"empirical_conditional_tv_overlap": 0.40441907700569835,
|
| 296 |
+
"empirical_posterior_tv_overlap": 0.40441907700569835,
|
| 297 |
+
"target_mass_absent_source_held": 0.015317286652078774,
|
| 298 |
+
"source_held_records": 177,
|
| 299 |
+
"target_evaluation_records": 457
|
| 300 |
+
},
|
| 301 |
+
{
|
| 302 |
+
"task_index": 30,
|
| 303 |
+
"dataset": "gas",
|
| 304 |
+
"environment": "batch7",
|
| 305 |
+
"task": "gas1_vs_6",
|
| 306 |
+
"empirical_joint_tv": 0.6958751576984258,
|
| 307 |
+
"overlap_current_mass": 0.41407492732159507,
|
| 308 |
+
"empirical_conditional_tv_overlap": 0.6075821071209954,
|
| 309 |
+
"empirical_posterior_tv_overlap": 0.5880430953313543,
|
| 310 |
+
"target_mass_absent_source_held": 0.1197411003236246,
|
| 311 |
+
"source_held_records": 177,
|
| 312 |
+
"target_evaluation_records": 618
|
| 313 |
+
},
|
| 314 |
+
{
|
| 315 |
+
"task_index": 32,
|
| 316 |
+
"dataset": "gas",
|
| 317 |
+
"environment": "batch9",
|
| 318 |
+
"task": "gas1_vs_6",
|
| 319 |
+
"empirical_joint_tv": 0.6327683615819208,
|
| 320 |
+
"overlap_current_mass": 0.3672316384180791,
|
| 321 |
+
"empirical_conditional_tv_overlap": 0.5384615384615384,
|
| 322 |
+
"empirical_posterior_tv_overlap": 0.5384615384615384,
|
| 323 |
+
"target_mass_absent_source_held": 0.0,
|
| 324 |
+
"source_held_records": 177,
|
| 325 |
+
"target_evaluation_records": 91
|
| 326 |
+
},
|
| 327 |
+
{
|
| 328 |
+
"task_index": 33,
|
| 329 |
+
"dataset": "gas",
|
| 330 |
+
"environment": "batch10",
|
| 331 |
+
"task": "gas1_vs_6",
|
| 332 |
+
"empirical_joint_tv": 0.6091301566997123,
|
| 333 |
+
"overlap_current_mass": 0.43262978360515936,
|
| 334 |
+
"empirical_conditional_tv_overlap": 0.558964700310025,
|
| 335 |
+
"empirical_posterior_tv_overlap": 0.550913222405153,
|
| 336 |
+
"target_mass_absent_source_held": 0.01257861635220126,
|
| 337 |
+
"source_held_records": 177,
|
| 338 |
+
"target_evaluation_records": 636
|
| 339 |
+
},
|
| 340 |
+
{
|
| 341 |
+
"task_index": 34,
|
| 342 |
+
"dataset": "gas",
|
| 343 |
+
"environment": "batch6",
|
| 344 |
+
"task": "gas2_vs_3",
|
| 345 |
+
"empirical_joint_tv": 0.6544434519611991,
|
| 346 |
+
"overlap_current_mass": 0.35477097143215336,
|
| 347 |
+
"empirical_conditional_tv_overlap": 0.24161786483966885,
|
| 348 |
+
"empirical_posterior_tv_overlap": 0.1855972808916146,
|
| 349 |
+
"target_mass_absent_source_held": 0.16104868913857676,
|
| 350 |
+
"source_held_records": 239,
|
| 351 |
+
"target_evaluation_records": 267
|
| 352 |
+
},
|
| 353 |
+
{
|
| 354 |
+
"task_index": 35,
|
| 355 |
+
"dataset": "gas",
|
| 356 |
+
"environment": "batch7",
|
| 357 |
+
"task": "gas2_vs_3",
|
| 358 |
+
"empirical_joint_tv": 0.7481554824491361,
|
| 359 |
+
"overlap_current_mass": 0.39395221821201565,
|
| 360 |
+
"empirical_conditional_tv_overlap": 0.6199017107064613,
|
| 361 |
+
"empirical_posterior_tv_overlap": 0.5149738337243449,
|
| 362 |
+
"target_mass_absent_source_held": 0.25,
|
| 363 |
+
"source_held_records": 239,
|
| 364 |
+
"target_evaluation_records": 524
|
| 365 |
+
},
|
| 366 |
+
{
|
| 367 |
+
"task_index": 37,
|
| 368 |
+
"dataset": "gas",
|
| 369 |
+
"environment": "batch9",
|
| 370 |
+
"task": "gas2_vs_3",
|
| 371 |
+
"empirical_joint_tv": 0.8786610878661087,
|
| 372 |
+
"overlap_current_mass": 0.12133891213389121,
|
| 373 |
+
"empirical_conditional_tv_overlap": 0.3977011494252874,
|
| 374 |
+
"empirical_posterior_tv_overlap": 0.3977011494252874,
|
| 375 |
+
"target_mass_absent_source_held": 0.53125,
|
| 376 |
+
"source_held_records": 239,
|
| 377 |
+
"target_evaluation_records": 64
|
| 378 |
+
},
|
| 379 |
+
{
|
| 380 |
+
"task_index": 38,
|
| 381 |
+
"dataset": "gas",
|
| 382 |
+
"environment": "batch10",
|
| 383 |
+
"task": "gas2_vs_3",
|
| 384 |
+
"empirical_joint_tv": 0.7213929005263868,
|
| 385 |
+
"overlap_current_mass": 0.4157713591577811,
|
| 386 |
+
"empirical_conditional_tv_overlap": 0.49642875215628623,
|
| 387 |
+
"empirical_posterior_tv_overlap": 0.44338837322005364,
|
| 388 |
+
"target_mass_absent_source_held": 0.35322580645161294,
|
| 389 |
+
"source_held_records": 239,
|
| 390 |
+
"target_evaluation_records": 620
|
| 391 |
+
},
|
| 392 |
+
{
|
| 393 |
+
"task_index": 40,
|
| 394 |
+
"dataset": "gas",
|
| 395 |
+
"environment": "batch7",
|
| 396 |
+
"task": "gas2_vs_4",
|
| 397 |
+
"empirical_joint_tv": 0.6515750964260292,
|
| 398 |
+
"overlap_current_mass": 0.41268685129699223,
|
| 399 |
+
"empirical_conditional_tv_overlap": 0.4458951213954642,
|
| 400 |
+
"empirical_posterior_tv_overlap": 0.30977899624504307,
|
| 401 |
+
"target_mass_absent_source_held": 0.26837060702875404,
|
| 402 |
+
"source_held_records": 193,
|
| 403 |
+
"target_evaluation_records": 626
|
| 404 |
+
},
|
| 405 |
+
{
|
| 406 |
+
"task_index": 43,
|
| 407 |
+
"dataset": "gas",
|
| 408 |
+
"environment": "batch10",
|
| 409 |
+
"task": "gas2_vs_4",
|
| 410 |
+
"empirical_joint_tv": 0.7339691205619502,
|
| 411 |
+
"overlap_current_mass": 0.31912230065723124,
|
| 412 |
+
"empirical_conditional_tv_overlap": 0.39851565155399254,
|
| 413 |
+
"empirical_posterior_tv_overlap": 0.3672921077446364,
|
| 414 |
+
"target_mass_absent_source_held": 0.16610738255033558,
|
| 415 |
+
"source_held_records": 193,
|
| 416 |
+
"target_evaluation_records": 596
|
| 417 |
+
},
|
| 418 |
+
{
|
| 419 |
+
"task_index": 44,
|
| 420 |
+
"dataset": "gas",
|
| 421 |
+
"environment": "batch6",
|
| 422 |
+
"task": "gas2_vs_5",
|
| 423 |
+
"empirical_joint_tv": 0.5464200342249123,
|
| 424 |
+
"overlap_current_mass": 0.45603696823209017,
|
| 425 |
+
"empirical_conditional_tv_overlap": 0.1789962863774888,
|
| 426 |
+
"empirical_posterior_tv_overlap": 0.1596942727730253,
|
| 427 |
+
"target_mass_absent_source_held": 0.11302211302211303,
|
| 428 |
+
"source_held_records": 369,
|
| 429 |
+
"target_evaluation_records": 407
|
| 430 |
+
},
|
| 431 |
+
{
|
| 432 |
+
"task_index": 45,
|
| 433 |
+
"dataset": "gas",
|
| 434 |
+
"environment": "batch7",
|
| 435 |
+
"task": "gas2_vs_5",
|
| 436 |
+
"empirical_joint_tv": 0.6536690147217816,
|
| 437 |
+
"overlap_current_mass": 0.47187811069674845,
|
| 438 |
+
"empirical_conditional_tv_overlap": 0.2851729943473662,
|
| 439 |
+
"empirical_posterior_tv_overlap": 0.28068628505264265,
|
| 440 |
+
"target_mass_absent_source_held": 0.3479789103690685,
|
| 441 |
+
"source_held_records": 369,
|
| 442 |
+
"target_evaluation_records": 569
|
| 443 |
+
},
|
| 444 |
+
{
|
| 445 |
+
"task_index": 46,
|
| 446 |
+
"dataset": "gas",
|
| 447 |
+
"environment": "batch8",
|
| 448 |
+
"task": "gas2_vs_5",
|
| 449 |
+
"empirical_joint_tv": 0.8285780027611596,
|
| 450 |
+
"overlap_current_mass": 0.2196911591757427,
|
| 451 |
+
"empirical_conditional_tv_overlap": 0.3386942860467823,
|
| 452 |
+
"empirical_posterior_tv_overlap": 0.3386942860467823,
|
| 453 |
+
"target_mass_absent_source_held": 0.7547169811320754,
|
| 454 |
+
"source_held_records": 369,
|
| 455 |
+
"target_evaluation_records": 106
|
| 456 |
+
},
|
| 457 |
+
{
|
| 458 |
+
"task_index": 48,
|
| 459 |
+
"dataset": "gas",
|
| 460 |
+
"environment": "batch10",
|
| 461 |
+
"task": "gas2_vs_5",
|
| 462 |
+
"empirical_joint_tv": 0.7809660449545672,
|
| 463 |
+
"overlap_current_mass": 0.2971465008767734,
|
| 464 |
+
"empirical_conditional_tv_overlap": 0.30540410245849353,
|
| 465 |
+
"empirical_posterior_tv_overlap": 0.2818978940013974,
|
| 466 |
+
"target_mass_absent_source_held": 0.05392156862745098,
|
| 467 |
+
"source_held_records": 369,
|
| 468 |
+
"target_evaluation_records": 612
|
| 469 |
+
},
|
| 470 |
+
{
|
| 471 |
+
"task_index": 49,
|
| 472 |
+
"dataset": "gas",
|
| 473 |
+
"environment": "batch6",
|
| 474 |
+
"task": "gas2_vs_6",
|
| 475 |
+
"empirical_joint_tv": 0.629307688565266,
|
| 476 |
+
"overlap_current_mass": 0.3733032774921752,
|
| 477 |
+
"empirical_conditional_tv_overlap": 0.33795152248052684,
|
| 478 |
+
"empirical_posterior_tv_overlap": 0.32629448554002843,
|
| 479 |
+
"target_mass_absent_source_held": 0.057441253263707574,
|
| 480 |
+
"source_held_records": 161,
|
| 481 |
+
"target_evaluation_records": 383
|
| 482 |
+
},
|
| 483 |
+
{
|
| 484 |
+
"task_index": 50,
|
| 485 |
+
"dataset": "gas",
|
| 486 |
+
"environment": "batch7",
|
| 487 |
+
"task": "gas2_vs_6",
|
| 488 |
+
"empirical_joint_tv": 0.6953680663048376,
|
| 489 |
+
"overlap_current_mass": 0.4500558418402335,
|
| 490 |
+
"empirical_conditional_tv_overlap": 0.5535311076862758,
|
| 491 |
+
"empirical_posterior_tv_overlap": 0.4116158625907264,
|
| 492 |
+
"target_mass_absent_source_held": 0.26025236593059936,
|
| 493 |
+
"source_held_records": 161,
|
| 494 |
+
"target_evaluation_records": 634
|
| 495 |
+
},
|
| 496 |
+
{
|
| 497 |
+
"task_index": 52,
|
| 498 |
+
"dataset": "gas",
|
| 499 |
+
"environment": "batch9",
|
| 500 |
+
"task": "gas2_vs_6",
|
| 501 |
+
"empirical_joint_tv": 0.8136645962732918,
|
| 502 |
+
"overlap_current_mass": 0.3756330625895843,
|
| 503 |
+
"empirical_conditional_tv_overlap": 0.5039430170440092,
|
| 504 |
+
"empirical_posterior_tv_overlap": 0.5039430170440092,
|
| 505 |
+
"target_mass_absent_source_held": 0.0,
|
| 506 |
+
"source_held_records": 161,
|
| 507 |
+
"target_evaluation_records": 65
|
| 508 |
+
},
|
| 509 |
+
{
|
| 510 |
+
"task_index": 53,
|
| 511 |
+
"dataset": "gas",
|
| 512 |
+
"environment": "batch10",
|
| 513 |
+
"task": "gas2_vs_6",
|
| 514 |
+
"empirical_joint_tv": 0.8132352941176471,
|
| 515 |
+
"overlap_current_mass": 0.46251370113262696,
|
| 516 |
+
"empirical_conditional_tv_overlap": 0.6759810844684891,
|
| 517 |
+
"empirical_posterior_tv_overlap": 0.532736489055476,
|
| 518 |
+
"target_mass_absent_source_held": 0.34705882352941175,
|
| 519 |
+
"source_held_records": 161,
|
| 520 |
+
"target_evaluation_records": 680
|
| 521 |
+
},
|
| 522 |
+
{
|
| 523 |
+
"task_index": 55,
|
| 524 |
+
"dataset": "gas",
|
| 525 |
+
"environment": "batch7",
|
| 526 |
+
"task": "gas3_vs_4",
|
| 527 |
+
"empirical_joint_tv": 0.6470355731225297,
|
| 528 |
+
"overlap_current_mass": 0.3608695652173913,
|
| 529 |
+
"empirical_conditional_tv_overlap": 0.49843510391772583,
|
| 530 |
+
"empirical_posterior_tv_overlap": 0.43945793650682347,
|
| 531 |
+
"target_mass_absent_source_held": 0.21936758893280633,
|
| 532 |
+
"source_held_records": 110,
|
| 533 |
+
"target_evaluation_records": 506
|
| 534 |
+
},
|
| 535 |
+
{
|
| 536 |
+
"task_index": 58,
|
| 537 |
+
"dataset": "gas",
|
| 538 |
+
"environment": "batch10",
|
| 539 |
+
"task": "gas3_vs_4",
|
| 540 |
+
"empirical_joint_tv": 0.8231879606879606,
|
| 541 |
+
"overlap_current_mass": 0.42767199017199015,
|
| 542 |
+
"empirical_conditional_tv_overlap": 0.634041110907394,
|
| 543 |
+
"empirical_posterior_tv_overlap": 0.5045233454696252,
|
| 544 |
+
"target_mass_absent_source_held": 0.18581081081081083,
|
| 545 |
+
"source_held_records": 110,
|
| 546 |
+
"target_evaluation_records": 592
|
| 547 |
+
},
|
| 548 |
+
{
|
| 549 |
+
"task_index": 59,
|
| 550 |
+
"dataset": "gas",
|
| 551 |
+
"environment": "batch6",
|
| 552 |
+
"task": "gas3_vs_5",
|
| 553 |
+
"empirical_joint_tv": 0.44115958668197475,
|
| 554 |
+
"overlap_current_mass": 0.6581515499425947,
|
| 555 |
+
"empirical_conditional_tv_overlap": 0.17422076547450627,
|
| 556 |
+
"empirical_posterior_tv_overlap": 0.0971143394103828,
|
| 557 |
+
"target_mass_absent_source_held": 0.0037313432835820895,
|
| 558 |
+
"source_held_records": 286,
|
| 559 |
+
"target_evaluation_records": 268
|
| 560 |
+
},
|
| 561 |
+
{
|
| 562 |
+
"task_index": 60,
|
| 563 |
+
"dataset": "gas",
|
| 564 |
+
"environment": "batch7",
|
| 565 |
+
"task": "gas3_vs_5",
|
| 566 |
+
"empirical_joint_tv": 0.6446804865513106,
|
| 567 |
+
"overlap_current_mass": 0.6366439796283894,
|
| 568 |
+
"empirical_conditional_tv_overlap": 0.5124004229911332,
|
| 569 |
+
"empirical_posterior_tv_overlap": 0.4630126614281086,
|
| 570 |
+
"target_mass_absent_source_held": 0.18930957683741648,
|
| 571 |
+
"source_held_records": 286,
|
| 572 |
+
"target_evaluation_records": 449
|
| 573 |
+
},
|
| 574 |
+
{
|
| 575 |
+
"task_index": 61,
|
| 576 |
+
"dataset": "gas",
|
| 577 |
+
"environment": "batch8",
|
| 578 |
+
"task": "gas3_vs_5",
|
| 579 |
+
"empirical_joint_tv": 0.9353146853146854,
|
| 580 |
+
"overlap_current_mass": 0.15034965034965037,
|
| 581 |
+
"empirical_conditional_tv_overlap": 0.6046511627906976,
|
| 582 |
+
"empirical_posterior_tv_overlap": 0.5813953488372093,
|
| 583 |
+
"target_mass_absent_source_held": 0.21153846153846156,
|
| 584 |
+
"source_held_records": 286,
|
| 585 |
+
"target_evaluation_records": 104
|
| 586 |
+
},
|
| 587 |
+
{
|
| 588 |
+
"task_index": 63,
|
| 589 |
+
"dataset": "gas",
|
| 590 |
+
"environment": "batch10",
|
| 591 |
+
"task": "gas3_vs_5",
|
| 592 |
+
"empirical_joint_tv": 0.587861152005889,
|
| 593 |
+
"overlap_current_mass": 0.5445574162679425,
|
| 594 |
+
"empirical_conditional_tv_overlap": 0.25630833252918106,
|
| 595 |
+
"empirical_posterior_tv_overlap": 0.18376488214615264,
|
| 596 |
+
"target_mass_absent_source_held": 0.06414473684210527,
|
| 597 |
+
"source_held_records": 286,
|
| 598 |
+
"target_evaluation_records": 608
|
| 599 |
+
},
|
| 600 |
+
{
|
| 601 |
+
"task_index": 64,
|
| 602 |
+
"dataset": "gas",
|
| 603 |
+
"environment": "batch6",
|
| 604 |
+
"task": "gas3_vs_6",
|
| 605 |
+
"empirical_joint_tv": 0.8314417822614544,
|
| 606 |
+
"overlap_current_mass": 0.36433375367801596,
|
| 607 |
+
"empirical_conditional_tv_overlap": 0.5713873665993654,
|
| 608 |
+
"empirical_posterior_tv_overlap": 0.3464089991346985,
|
| 609 |
+
"target_mass_absent_source_held": 0.06147540983606557,
|
| 610 |
+
"source_held_records": 78,
|
| 611 |
+
"target_evaluation_records": 244
|
| 612 |
+
},
|
| 613 |
+
{
|
| 614 |
+
"task_index": 65,
|
| 615 |
+
"dataset": "gas",
|
| 616 |
+
"environment": "batch7",
|
| 617 |
+
"task": "gas3_vs_6",
|
| 618 |
+
"empirical_joint_tv": 0.9218297914795968,
|
| 619 |
+
"overlap_current_mass": 0.2318667065748778,
|
| 620 |
+
"empirical_conditional_tv_overlap": 0.6910856228708551,
|
| 621 |
+
"empirical_posterior_tv_overlap": 0.6237305475607231,
|
| 622 |
+
"target_mass_absent_source_held": 0.2898832684824903,
|
| 623 |
+
"source_held_records": 78,
|
| 624 |
+
"target_evaluation_records": 514
|
| 625 |
+
},
|
| 626 |
+
{
|
| 627 |
+
"task_index": 67,
|
| 628 |
+
"dataset": "gas",
|
| 629 |
+
"environment": "batch9",
|
| 630 |
+
"task": "gas3_vs_6",
|
| 631 |
+
"empirical_joint_tv": 0.6025641025641026,
|
| 632 |
+
"overlap_current_mass": 0.4230769230769231,
|
| 633 |
+
"empirical_conditional_tv_overlap": 0.06060606060606061,
|
| 634 |
+
"empirical_posterior_tv_overlap": 0.030303030303030304,
|
| 635 |
+
"target_mass_absent_source_held": 0.030927835051546393,
|
| 636 |
+
"source_held_records": 78,
|
| 637 |
+
"target_evaluation_records": 97
|
| 638 |
+
},
|
| 639 |
+
{
|
| 640 |
+
"task_index": 68,
|
| 641 |
+
"dataset": "gas",
|
| 642 |
+
"environment": "batch10",
|
| 643 |
+
"task": "gas3_vs_6",
|
| 644 |
+
"empirical_joint_tv": 0.6158777120315583,
|
| 645 |
+
"overlap_current_mass": 0.5443786982248521,
|
| 646 |
+
"empirical_conditional_tv_overlap": 0.32674974000268386,
|
| 647 |
+
"empirical_posterior_tv_overlap": 0.31002079978529257,
|
| 648 |
+
"target_mass_absent_source_held": 0.2751479289940828,
|
| 649 |
+
"source_held_records": 78,
|
| 650 |
+
"target_evaluation_records": 676
|
| 651 |
+
},
|
| 652 |
+
{
|
| 653 |
+
"task_index": 70,
|
| 654 |
+
"dataset": "gas",
|
| 655 |
+
"environment": "batch7",
|
| 656 |
+
"task": "gas4_vs_5",
|
| 657 |
+
"empirical_joint_tv": 0.7660843920145191,
|
| 658 |
+
"overlap_current_mass": 0.5896324863883847,
|
| 659 |
+
"empirical_conditional_tv_overlap": 0.6704926684133363,
|
| 660 |
+
"empirical_posterior_tv_overlap": 0.6623088991152682,
|
| 661 |
+
"target_mass_absent_source_held": 0.21415607985480944,
|
| 662 |
+
"source_held_records": 240,
|
| 663 |
+
"target_evaluation_records": 551
|
| 664 |
+
},
|
| 665 |
+
{
|
| 666 |
+
"task_index": 73,
|
| 667 |
+
"dataset": "gas",
|
| 668 |
+
"environment": "batch10",
|
| 669 |
+
"task": "gas4_vs_5",
|
| 670 |
+
"empirical_joint_tv": 0.761986301369863,
|
| 671 |
+
"overlap_current_mass": 0.6600456621004567,
|
| 672 |
+
"empirical_conditional_tv_overlap": 0.6487041835499334,
|
| 673 |
+
"empirical_posterior_tv_overlap": 0.6380813145476109,
|
| 674 |
+
"target_mass_absent_source_held": 0.07705479452054795,
|
| 675 |
+
"source_held_records": 240,
|
| 676 |
+
"target_evaluation_records": 584
|
| 677 |
+
},
|
| 678 |
+
{
|
| 679 |
+
"task_index": 75,
|
| 680 |
+
"dataset": "gas",
|
| 681 |
+
"environment": "batch7",
|
| 682 |
+
"task": "gas4_vs_6",
|
| 683 |
+
"empirical_joint_tv": 0.96875,
|
| 684 |
+
"overlap_current_mass": 0.09780844155844155,
|
| 685 |
+
"empirical_conditional_tv_overlap": 0.8437989857076994,
|
| 686 |
+
"empirical_posterior_tv_overlap": 0.16330106039649608,
|
| 687 |
+
"target_mass_absent_source_held": 0.12987012987012989,
|
| 688 |
+
"source_held_records": 32,
|
| 689 |
+
"target_evaluation_records": 616
|
| 690 |
+
},
|
| 691 |
+
{
|
| 692 |
+
"task_index": 78,
|
| 693 |
+
"dataset": "gas",
|
| 694 |
+
"environment": "batch10",
|
| 695 |
+
"task": "gas4_vs_6",
|
| 696 |
+
"empirical_joint_tv": 0.9672162576687117,
|
| 697 |
+
"overlap_current_mass": 0.04505368098159509,
|
| 698 |
+
"empirical_conditional_tv_overlap": 0.6148246405263438,
|
| 699 |
+
"empirical_posterior_tv_overlap": 0.3424842149944288,
|
| 700 |
+
"target_mass_absent_source_held": 0.3711656441717791,
|
| 701 |
+
"source_held_records": 32,
|
| 702 |
+
"target_evaluation_records": 652
|
| 703 |
+
},
|
| 704 |
+
{
|
| 705 |
+
"task_index": 79,
|
| 706 |
+
"dataset": "gas",
|
| 707 |
+
"environment": "batch6",
|
| 708 |
+
"task": "gas5_vs_6",
|
| 709 |
+
"empirical_joint_tv": 0.578525641025641,
|
| 710 |
+
"overlap_current_mass": 0.4919871794871795,
|
| 711 |
+
"empirical_conditional_tv_overlap": 0.19297279172255224,
|
| 712 |
+
"empirical_posterior_tv_overlap": 0.18808679823721022,
|
| 713 |
+
"target_mass_absent_source_held": 0.11718750000000001,
|
| 714 |
+
"source_held_records": 208,
|
| 715 |
+
"target_evaluation_records": 384
|
| 716 |
+
},
|
| 717 |
+
{
|
| 718 |
+
"task_index": 80,
|
| 719 |
+
"dataset": "gas",
|
| 720 |
+
"environment": "batch7",
|
| 721 |
+
"task": "gas5_vs_6",
|
| 722 |
+
"empirical_joint_tv": 0.7978533094812165,
|
| 723 |
+
"overlap_current_mass": 0.44689177101967803,
|
| 724 |
+
"empirical_conditional_tv_overlap": 0.5812557616410505,
|
| 725 |
+
"empirical_posterior_tv_overlap": 0.574980221652726,
|
| 726 |
+
"target_mass_absent_source_held": 0.44186046511627913,
|
| 727 |
+
"source_held_records": 208,
|
| 728 |
+
"target_evaluation_records": 559
|
| 729 |
+
},
|
| 730 |
+
{
|
| 731 |
+
"task_index": 83,
|
| 732 |
+
"dataset": "gas",
|
| 733 |
+
"environment": "batch10",
|
| 734 |
+
"task": "gas5_vs_6",
|
| 735 |
+
"empirical_joint_tv": 0.6132830492860435,
|
| 736 |
+
"overlap_current_mass": 0.529335559649931,
|
| 737 |
+
"empirical_conditional_tv_overlap": 0.274517045008442,
|
| 738 |
+
"empirical_posterior_tv_overlap": 0.27072698878866275,
|
| 739 |
+
"target_mass_absent_source_held": 0.10628742514970059,
|
| 740 |
+
"source_held_records": 208,
|
| 741 |
+
"target_evaluation_records": 668
|
| 742 |
+
}
|
| 743 |
+
]
|
revision_r1/external/shift_overall.json
ADDED
|
@@ -0,0 +1,102 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"empirical_joint_tv": {
|
| 3 |
+
"overall": 0.7264348414836221,
|
| 4 |
+
"by_dataset": {
|
| 5 |
+
"hydraulic": 0.7459120261182116,
|
| 6 |
+
"gas": 0.7069576568490327
|
| 7 |
+
},
|
| 8 |
+
"by_environment": {
|
| 9 |
+
"hydraulic:context1=73": 0.5891016200294551,
|
| 10 |
+
"hydraulic:context1=80": 0.6455194290245836,
|
| 11 |
+
"hydraulic:context1=90": 0.4949390815370197,
|
| 12 |
+
"hydraulic:context0=3": 0.9999999999999999,
|
| 13 |
+
"hydraulic:context0=20": 1.0,
|
| 14 |
+
"gas:batch6": 0.6017849079274071,
|
| 15 |
+
"gas:batch7": 0.6903547112156817,
|
| 16 |
+
"gas:batch10": 0.7201405974293267,
|
| 17 |
+
"gas:batch9": 0.7376884924021828,
|
| 18 |
+
"gas:batch8": 0.7848195752705647
|
| 19 |
+
},
|
| 20 |
+
"tasks": 57
|
| 21 |
+
},
|
| 22 |
+
"overlap_current_mass": {
|
| 23 |
+
"overall": 0.39033438581068947,
|
| 24 |
+
"by_dataset": {
|
| 25 |
+
"hydraulic": 0.40005300541383015,
|
| 26 |
+
"gas": 0.38061576620754883
|
| 27 |
+
},
|
| 28 |
+
"by_environment": {
|
| 29 |
+
"hydraulic:context1=73": 0.4204221894943544,
|
| 30 |
+
"hydraulic:context1=80": 0.4006344171292625,
|
| 31 |
+
"hydraulic:context1=90": 0.5638238050609184,
|
| 32 |
+
"hydraulic:context0=3": 0.30769230769230765,
|
| 33 |
+
"hydraulic:context0=20": 0.30769230769230765,
|
| 34 |
+
"gas:batch6": 0.45400665855867306,
|
| 35 |
+
"gas:batch7": 0.43603222996748564,
|
| 36 |
+
"gas:batch10": 0.44320989173019887,
|
| 37 |
+
"gas:batch9": 0.3100051268515387,
|
| 38 |
+
"gas:batch8": 0.259824923929848
|
| 39 |
+
},
|
| 40 |
+
"tasks": 57
|
| 41 |
+
},
|
| 42 |
+
"empirical_conditional_tv_overlap": {
|
| 43 |
+
"overall": 0.41570487409678125,
|
| 44 |
+
"by_dataset": {
|
| 45 |
+
"hydraulic": 0.4544437569162058,
|
| 46 |
+
"gas": 0.37696599127735675
|
| 47 |
+
},
|
| 48 |
+
"by_environment": {
|
| 49 |
+
"hydraulic:context1=73": 0.022652965903783285,
|
| 50 |
+
"hydraulic:context1=80": 0.11520190023752969,
|
| 51 |
+
"hydraulic:context1=90": 0.13436391843971632,
|
| 52 |
+
"hydraulic:context0=3": 1.0,
|
| 53 |
+
"hydraulic:context0=20": 1.0,
|
| 54 |
+
"gas:batch6": 0.2622787670653783,
|
| 55 |
+
"gas:batch7": 0.4675657953371991,
|
| 56 |
+
"gas:batch10": 0.4615588416674466,
|
| 57 |
+
"gas:batch9": 0.35466971629145877,
|
| 58 |
+
"gas:batch8": 0.33875683602530104
|
| 59 |
+
},
|
| 60 |
+
"tasks": 57
|
| 61 |
+
},
|
| 62 |
+
"empirical_posterior_tv_overlap": {
|
| 63 |
+
"overall": 0.26299167483291297,
|
| 64 |
+
"by_dataset": {
|
| 65 |
+
"hydraulic": 0.19948737143496967,
|
| 66 |
+
"gas": 0.3264959782308563
|
| 67 |
+
},
|
| 68 |
+
"by_environment": {
|
| 69 |
+
"hydraulic:context1=73": 0.0,
|
| 70 |
+
"hydraulic:context1=80": 0.0,
|
| 71 |
+
"hydraulic:context1=90": 2.86413918586809e-17,
|
| 72 |
+
"hydraulic:context0=3": 0.6618681318681318,
|
| 73 |
+
"hydraulic:context0=20": 0.33556872530671655,
|
| 74 |
+
"gas:batch6": 0.20217457790534224,
|
| 75 |
+
"gas:batch7": 0.3693269205782592,
|
| 76 |
+
"gas:batch10": 0.3992748321602165,
|
| 77 |
+
"gas:batch9": 0.3306986624696586,
|
| 78 |
+
"gas:batch8": 0.33100489804080496
|
| 79 |
+
},
|
| 80 |
+
"tasks": 57
|
| 81 |
+
},
|
| 82 |
+
"target_mass_absent_source_held": {
|
| 83 |
+
"overall": 0.2408059865231789,
|
| 84 |
+
"by_dataset": {
|
| 85 |
+
"hydraulic": 0.3062349162538916,
|
| 86 |
+
"gas": 0.17537705679246618
|
| 87 |
+
},
|
| 88 |
+
"by_environment": {
|
| 89 |
+
"hydraulic:context1=73": 0.41904761904761906,
|
| 90 |
+
"hydraulic:context1=80": 0.4,
|
| 91 |
+
"hydraulic:context1=90": 0.3090909090909091,
|
| 92 |
+
"hydraulic:context0=3": 0.30107526881720426,
|
| 93 |
+
"hydraulic:context0=20": 0.1019607843137255,
|
| 94 |
+
"gas:batch6": 0.06470976148846597,
|
| 95 |
+
"gas:batch7": 0.21526001667687727,
|
| 96 |
+
"gas:batch10": 0.16239479122983275,
|
| 97 |
+
"gas:batch9": 0.11243556701030928,
|
| 98 |
+
"gas:batch8": 0.32208514755684564
|
| 99 |
+
},
|
| 100 |
+
"tasks": 57
|
| 101 |
+
}
|
| 102 |
+
}
|
revision_r1/external/specification.json
ADDED
|
@@ -0,0 +1,123 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"material_passport": {
|
| 3 |
+
"material_type": "post-review exploratory experiment specification",
|
| 4 |
+
"provenance": "Previously opened external sensor archives and original eligible task manifest",
|
| 5 |
+
"license": "Hydraulic UCI CC BY 4.0; gas legacy research-only restriction retained; local research use only",
|
| 6 |
+
"created_utc": "2026-11-09T14:37:09.140791+00:00",
|
| 7 |
+
"scientific_status": "exploratory; does not replace original confirmation"
|
| 8 |
+
},
|
| 9 |
+
"purpose": "Distinguish small within-fitted-law encoding headroom, finite labeled support, and observed source-target transfer failure without claiming an identifiable population decomposition",
|
| 10 |
+
"device": "cuda:4",
|
| 11 |
+
"dtype": "float64",
|
| 12 |
+
"backbone_executions": 0,
|
| 13 |
+
"trainable_parameters": 0,
|
| 14 |
+
"task_indices": [
|
| 15 |
+
0,
|
| 16 |
+
1,
|
| 17 |
+
2,
|
| 18 |
+
3,
|
| 19 |
+
4,
|
| 20 |
+
5,
|
| 21 |
+
6,
|
| 22 |
+
7,
|
| 23 |
+
8,
|
| 24 |
+
9,
|
| 25 |
+
10,
|
| 26 |
+
13,
|
| 27 |
+
14,
|
| 28 |
+
15,
|
| 29 |
+
17,
|
| 30 |
+
18,
|
| 31 |
+
20,
|
| 32 |
+
23,
|
| 33 |
+
24,
|
| 34 |
+
25,
|
| 35 |
+
26,
|
| 36 |
+
28,
|
| 37 |
+
29,
|
| 38 |
+
30,
|
| 39 |
+
32,
|
| 40 |
+
33,
|
| 41 |
+
34,
|
| 42 |
+
35,
|
| 43 |
+
37,
|
| 44 |
+
38,
|
| 45 |
+
40,
|
| 46 |
+
43,
|
| 47 |
+
44,
|
| 48 |
+
45,
|
| 49 |
+
46,
|
| 50 |
+
48,
|
| 51 |
+
49,
|
| 52 |
+
50,
|
| 53 |
+
52,
|
| 54 |
+
53,
|
| 55 |
+
55,
|
| 56 |
+
58,
|
| 57 |
+
59,
|
| 58 |
+
60,
|
| 59 |
+
61,
|
| 60 |
+
63,
|
| 61 |
+
64,
|
| 62 |
+
65,
|
| 63 |
+
67,
|
| 64 |
+
68,
|
| 65 |
+
70,
|
| 66 |
+
73,
|
| 67 |
+
75,
|
| 68 |
+
78,
|
| 69 |
+
79,
|
| 70 |
+
80,
|
| 71 |
+
83
|
| 72 |
+
],
|
| 73 |
+
"seeds": [
|
| 74 |
+
20111,
|
| 75 |
+
20112,
|
| 76 |
+
20113
|
| 77 |
+
],
|
| 78 |
+
"methods": [
|
| 79 |
+
"query_joint",
|
| 80 |
+
"posterior_clusters",
|
| 81 |
+
"identity"
|
| 82 |
+
],
|
| 83 |
+
"estimators": [
|
| 84 |
+
"support_mixture",
|
| 85 |
+
"joint_jeffreys",
|
| 86 |
+
"naive_bayes"
|
| 87 |
+
],
|
| 88 |
+
"interventions": [
|
| 89 |
+
"Original target64 exact seed selection, nested target256 when available, and all original target adaptation records",
|
| 90 |
+
"Source group-disjoint train/held split: SHA256('review_r1_source_held:' + original_group) mod 5 == 0 held; source64/256/all train only",
|
| 91 |
+
"Each fixed policy replayed on original target evaluation and, for source-trained policies, source held records",
|
| 92 |
+
"All-target-adaptation unsmoothed empirical reference law: encoders and exact policy fitted without target evaluation access, evaluated on original independent-group target evaluation; not a population oracle"
|
| 93 |
+
],
|
| 94 |
+
"source_preprocessing": "Original source-only binary thresholds kept; source-held records contributed unlabeled preprocessing before this post-review split, never labeled model fitting",
|
| 95 |
+
"metrics": [
|
| 96 |
+
"paired realized utility and no-query utility",
|
| 97 |
+
"model-predicted versus held-realized query increment",
|
| 98 |
+
"fitted encoding loss",
|
| 99 |
+
"posterior/joint/observation conditional log losses",
|
| 100 |
+
"same-policy source-held versus shifted-target scores",
|
| 101 |
+
"finite-sample empirical conditional TV on overlapping current support",
|
| 102 |
+
"target mass outside source-held current support"
|
| 103 |
+
],
|
| 104 |
+
"aggregation": "Original weighting: seeds, then tasks within environment, then environments within dataset, then the two datasets equally. Sample-size contrasts restricted to identical tasks. Costs 0,.01,.05 and budgets 1,2 averaged; no-query average from budget0.",
|
| 105 |
+
"inference": "Descriptive 95% paired bootstrap by target environment within fixed dataset; collapse tasks/seeds first. Few clusters; not population transfer inference; no confirmatory gate rescues.",
|
| 106 |
+
"limitations": [
|
| 107 |
+
"True external law is unknown; histogram reference and TV have finite-sample error",
|
| 108 |
+
"Source-target contrast also includes differences in physical groups and sample composition; no causal attribution percentages",
|
| 109 |
+
"Labels-all support varies by task; 256 contrasts require available support and use matched task subsets",
|
| 110 |
+
"Source group split introduces a new held partition; original target adaptation/evaluation split unchanged",
|
| 111 |
+
"Three seeds are repeated fitting samples, not independent environmental replications",
|
| 112 |
+
"Empirical-reference policies are exploratory benchmarks and their NLL is not reported because zero fitted cells yield uninformative infinite log scores"
|
| 113 |
+
],
|
| 114 |
+
"input_hashes": {
|
| 115 |
+
"artifacts/external_decision_sufficiency_01/prepared/manifest.json": "c97a59bbf5f04fe8dc3ccfe00986b4df288b8e7ef2f3fbd9eb12f3636dceef5d",
|
| 116 |
+
"articles/make_1/release/huggingface-revised/protocols/external_sufficiency_protocol_01.json": "e0229ef8a50270b7a590c6ed79a850ec8f902d17f2c8213111bf432669ef9d04",
|
| 117 |
+
"articles/make_1/release/huggingface-revised/unionmind/decision_sufficiency/core.py": "fa2f7405a8d0dc6ec96ebae1bb2a2b810d03ef1fd2877f5ebef7df1722992cee",
|
| 118 |
+
"articles/make_1/release/huggingface-revised/unionmind/external_sufficiency/runtime.py": "fa19ce1b7fba9e6953e9e4081d4b6001ed55c33edc48a05960d5a13ae6aad61d"
|
| 119 |
+
},
|
| 120 |
+
"script_sha256": "18a499610df5927f78ae97af5bd4249b3c18beaef98ea39caecc5ad1303b7956",
|
| 121 |
+
"no_recipe_selection": true,
|
| 122 |
+
"original_results_and_gates_unchanged": true
|
| 123 |
+
}
|
revision_r1/external/table_provenance.json
ADDED
|
@@ -0,0 +1,10 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"created_utc": "2026-11-09T14:43:13.718361+00:00",
|
| 3 |
+
"script_sha256": "33e4d7ff42763ccd48db10013dd91831af13aa368a115c39987f65636a06c63f",
|
| 4 |
+
"input_metrics_sha256": "9813ff30348ad76cb8b672e022a5f3fe852b8f0ee13625e7a5799400cfc01120",
|
| 5 |
+
"table_sha256": "09b7110cc185574d01f5aeb4681aa20910048a50429b274a5422969c2654fad2",
|
| 6 |
+
"design_changed": false,
|
| 7 |
+
"scientific_status": "post-review exploratory",
|
| 8 |
+
"weighting": "As specification; each table row lists support and independent environment counts",
|
| 9 |
+
"caution": "The 256-label subset has only one hydraulic environment; empirical TV contains finite-count noise and is not a causal decomposition. Gas record artifacts remain research-only local data."
|
| 10 |
+
}
|
revision_r1/sensor/analysis_spec.json
ADDED
|
@@ -0,0 +1,79 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"id": "MAKE-review-r1-sensor-01",
|
| 3 |
+
"timestamp_utc": "2026-11-09T14:35:00.586928+00:00",
|
| 4 |
+
"status": "Prospective specification for new post-review exploratory analyses on previously opened data; no new confirmation or model promotion",
|
| 5 |
+
"authorization": "Reviewer-driven additional experiments requested by author, delegated within academic-research-suite experiment workflow",
|
| 6 |
+
"cohorts": {
|
| 7 |
+
"hydraulic": {
|
| 8 |
+
"sizes": [
|
| 9 |
+
3,
|
| 10 |
+
4,
|
| 11 |
+
3,
|
| 12 |
+
4
|
| 13 |
+
],
|
| 14 |
+
"seeds": [
|
| 15 |
+
20751,
|
| 16 |
+
20752,
|
| 17 |
+
20753
|
| 18 |
+
]
|
| 19 |
+
},
|
| 20 |
+
"temperature": {
|
| 21 |
+
"sizes": [
|
| 22 |
+
10,
|
| 23 |
+
3
|
| 24 |
+
],
|
| 25 |
+
"seeds": [
|
| 26 |
+
20851,
|
| 27 |
+
20852,
|
| 28 |
+
20853
|
| 29 |
+
]
|
| 30 |
+
}
|
| 31 |
+
},
|
| 32 |
+
"budgets": [
|
| 33 |
+
64,
|
| 34 |
+
128
|
| 35 |
+
],
|
| 36 |
+
"prefixes": "Use exact saved fit_ids for every original seed; assert nested prefix, all source, no unseen binding fitting labels; normalize using exact saved prefix mean/std",
|
| 37 |
+
"optimizer": "400 full-batch AdamW updates, lr=.003, weight_decay=.01, float32 neural training, same original initialization seeds and architectures; float64 probability/statistical operations on cuda:3; TF32 disabled",
|
| 38 |
+
"questions": [
|
| 39 |
+
"At fixed pretrained-free forward mean, how do shared shrunken full, diagonal, and identity covariance affect ranking and probabilities?",
|
| 40 |
+
"At fixed joint forward posterior, what changes when replacing it by the product of its exact factor marginals?",
|
| 41 |
+
"With identical factor-one-hot -> SiLU -> sensor-mean decoder and fixed identity Gaussian likelihood, what changes when the decoder training objective is conditional joint cross-entropy instead of observation MSE?",
|
| 42 |
+
"At fixed inverse classifier architecture, does unweighted versus original class-balanced factor cross-entropy account for the gap?",
|
| 43 |
+
"Can a scalar temperature selected using only out-of-fold fitting-prefix predictions address held probability failure?",
|
| 44 |
+
"In hydraulic data, is reversal reproduced under fixed64 normalization/covariance, longer optimization, duplicate64 input, and equal-count additions of new versus already observed bindings?"
|
| 45 |
+
],
|
| 46 |
+
"main_grid": [
|
| 47 |
+
"MSE forward: full covariance, diagonal, identity; full-covariance posterior projected to product of marginals",
|
| 48 |
+
"CE forward decoder: fixed identity covariance; train conditional CE over full supplied ontology including unobserved bindings without held labels",
|
| 49 |
+
"Inverse MLP: original class-balanced and unweighted factor CE; compare raw posterior and original source per-factor calibration",
|
| 50 |
+
"For full-cov MSE, identity MSE, identity CE, balanced inverse, unweighted inverse: scalar temperature from fixed grid [1,2,4,8,16,32,64] minimizing prefix OOF joint log loss; apply without target tuning; preserve raw outputs"
|
| 51 |
+
],
|
| 52 |
+
"crossfit": "Four original deterministic folds i mod4, seed+101+fold; one full fit plus four folds for each of four neural objective/architecture choices, 400 steps each. Forward covariance from MSE source OOF residuals with original shrinkage .5/full + .5/diag + .05 I; calibrator OOF forward scores use this same pooled OOF covariance and are not nested. Disclose possible source calibration optimism. No held/source-outside-prefix tuning.",
|
| 53 |
+
"hydraulic_reversal": {
|
| 54 |
+
"normalization_and_inference_covariance": "fixed64 for all controlled refits; use saved64 source OOF covariance; unchanged uniform prior",
|
| 55 |
+
"scenarios": [
|
| 56 |
+
"saved64/400",
|
| 57 |
+
"duplicate64/400 (128 duplicated rows, no new information)",
|
| 58 |
+
"64/800",
|
| 59 |
+
"actual128/400 fixed64 normalization",
|
| 60 |
+
"actual128/800 fixed64 normalization",
|
| 61 |
+
"64 plus q old-binding extra rows/400",
|
| 62 |
+
"64 plus q new-binding extra rows/400"
|
| 63 |
+
],
|
| 64 |
+
"matched_additions": "Partition later64 rows by joint-binding membership in first64; q=min(partition sizes); preserve prefix order, use first q each. Same initial seed and number of updates. This controls added count, normalization, covariance and initialization; row values and binding identities still differ. All rows inside original128 prefix."
|
| 65 |
+
},
|
| 66 |
+
"outcomes": "Every seed retained. Joint Top1/Top3 with original tolerance1e-12 uniform tie credit, NLL stable log-softmax, multiclass Brier, confidence, ECE10, coverage and accuracy at .9; source OOF and held seen/unseen, temperature each day9-12 and equal-day mean, hydraulic record mean. Standardized sensor-mean MSE and Gaussian observation scores; original source outside128 diagnostic without fitting. No confirmatory p-values or favorable selection.",
|
| 67 |
+
"replay_and_cost": "Save fit_ids, normalization, all full/fold network states, derived mean/covariance, OOF/held log posterior and per-record scores, scalar temperatures, losses at updates0,100,200,400 (and800), raw/source test diagnostics; record all fit/prediction/calibration/serialization times, trainable counts, forward executions, approximate dense arithmetic counts separately from latency, peak GPU bytes and complete process time. No backbone or CPU scientific execution; no offload or DeepSpeed.",
|
| 68 |
+
"checks": [
|
| 69 |
+
"Gaussian CE decoder identity likelihood score matches frozen posterior scores up to additive normalizer",
|
| 70 |
+
"factor-marginal projection normalizes and preserves each factor marginal",
|
| 71 |
+
"duplicate-row MSE gradients agree within float32 tolerance",
|
| 72 |
+
"all original MSE and balanced inverse refit full predictions match saved checkpoints within1e-5 (report residuals)",
|
| 73 |
+
"all reported scalars finite except empty-stratum values explicitly null",
|
| 74 |
+
"all original seeds/prefix counts and original held targets retained"
|
| 75 |
+
],
|
| 76 |
+
"limitations": "Post-review, opened-data mechanism probes, no independent systems or new confirmation. Objective can be isolated within fixed decoder only; pure modeling direction remains entangled with input/output parameterization. Product projection removes posterior dependence without retraining and does not isolate all factorization assumptions. Temperature calibration cannot establish operational reliability or extrapolate to new platforms.",
|
| 77 |
+
"command": "PYTHONDONTWRITEBYTECODE=1 python articles/make_1/reproduce/review_r1_sensor.py --mode run --device cuda:3 --output articles/make_1/metadata/review_r1/sensor/run_01",
|
| 78 |
+
"hard_timeout_seconds": 1800
|
| 79 |
+
}
|
revision_r1/sensor/export_safe.py
ADDED
|
@@ -0,0 +1,39 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Export complete review artifacts to lossless non-executable tensor trees."""
|
| 2 |
+
from pathlib import Path
|
| 3 |
+
import argparse,hashlib,json,shutil,time
|
| 4 |
+
import torch
|
| 5 |
+
from safe_tree import save_tree,load_tree
|
| 6 |
+
|
| 7 |
+
|
| 8 |
+
def sha(path):return hashlib.sha256(path.read_bytes()).hexdigest()
|
| 9 |
+
|
| 10 |
+
|
| 11 |
+
def compare(a,b):
|
| 12 |
+
if torch.is_tensor(a):assert torch.is_tensor(b) and torch.equal(a,b);return 1
|
| 13 |
+
if isinstance(a,dict):assert list(a)==list(b);return sum(compare(a[k],b[k]) for k in a)
|
| 14 |
+
if isinstance(a,(tuple,list)):assert type(a)==type(b) and len(a)==len(b);return sum(compare(x,y) for x,y in zip(a,b))
|
| 15 |
+
assert a==b;return 0
|
| 16 |
+
|
| 17 |
+
|
| 18 |
+
def main():
|
| 19 |
+
parser=argparse.ArgumentParser();parser.add_argument('--source',default=str(Path(__file__).resolve().parent));parser.add_argument('--output',required=True);parser.add_argument('--device',default='cuda:3');args=parser.parse_args()
|
| 20 |
+
source=Path(args.source).resolve();out=Path(args.output).resolve();assert not out.exists();out.mkdir(parents=True);torch.cuda.set_device(args.device);start=time.perf_counter();trees=[];tensor_count=0
|
| 21 |
+
for name in ['run_01','objective_extension_01']:
|
| 22 |
+
directory=out/name;directory.mkdir()
|
| 23 |
+
for file in sorted((source/name).iterdir()):
|
| 24 |
+
if file.suffix=='.pt':
|
| 25 |
+
original=torch.load(file,map_location=args.device,weights_only=True);base=directory/file.stem;save_tree(original,base)
|
| 26 |
+
restored=load_tree(base,args.device);count=compare(original,restored);tensor_count+=count
|
| 27 |
+
trees.append(dict(file=str(base.relative_to(out)),source_sha256=sha(file),tensors=count,json_sha256=sha(base.with_suffix('.json')),safetensors_sha256=sha(base.with_suffix('.safetensors'))))
|
| 28 |
+
elif file.suffix in ('.json','.csv','.md','.py'):shutil.copyfile(file,directory/file.name)
|
| 29 |
+
for name in ['analysis_spec.json','objective_extension_spec.json','findings.json','findings.md','primary_table.csv','full_replay.json','verify_all.py','safe_tree.py','objective_extension.py','export_safe.py']:
|
| 30 |
+
shutil.copyfile(source/name,out/name)
|
| 31 |
+
trainer=source.parents[2]/'reproduce/review_r1_sensor.py';shutil.copyfile(trainer,out/'review_r1_sensor.py')
|
| 32 |
+
assert not list(out.rglob('*.pt'))
|
| 33 |
+
report=dict(format='Lossless Safetensors + JSON tensor trees; no executable pickle checkpoints',trees=trees,roundtrip_tensor_count=tensor_count,roundtrip_maximum_difference=0,seconds=time.perf_counter()-start,
|
| 34 |
+
source_artifacts_modified=False,device=args.device,export_uses_cpu_only_for_serialization=True,
|
| 35 |
+
replay_command='python verify_all.py --release /path/to/original-release --base run_01 --extension objective_extension_01 --output replay.json --device cuda:0')
|
| 36 |
+
(out/'EXPORT_MANIFEST.json').write_text(json.dumps(report,indent=2)+'\n');print(json.dumps({k:v for k,v in report.items() if k!='trees'},indent=2))
|
| 37 |
+
|
| 38 |
+
|
| 39 |
+
if __name__=='__main__':main()
|
revision_r1/sensor/findings.json
ADDED
|
@@ -0,0 +1,123 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"status": "Post-review exploratory, all data previously opened",
|
| 3 |
+
"aggregation": {
|
| 4 |
+
"hydraulic": "Mean across all304held unseen-binding rows, then3seeds",
|
| 5 |
+
"temperature_primary_top3": "Equal average across days9,10,11,12, then3seeds;65uniqueunseenepisodes",
|
| 6 |
+
"probability_metrics": "Pool unseen rows and seeds;304x3=912 hydraulic and65x3=195 gas prediction instances, not independent observations or systems"
|
| 7 |
+
},
|
| 8 |
+
"primary_table": [
|
| 9 |
+
{
|
| 10 |
+
"method": "forward_mse_full",
|
| 11 |
+
"description": "MSE decoder; full covariance",
|
| 12 |
+
"hydraulic_64": 0.4682017543859649,
|
| 13 |
+
"hydraulic_128": 0.3267543859649123,
|
| 14 |
+
"temperature_64": 0.6700367647058822,
|
| 15 |
+
"temperature_128": 0.7267156862745098
|
| 16 |
+
},
|
| 17 |
+
{
|
| 18 |
+
"method": "forward_mse_diag",
|
| 19 |
+
"description": "MSE decoder; diagonal covariance",
|
| 20 |
+
"hydraulic_64": 0.15901428538103854,
|
| 21 |
+
"hydraulic_128": 0.12720831799779167,
|
| 22 |
+
"temperature_64": 0.4818206377378537,
|
| 23 |
+
"temperature_128": 0.4679810501786921
|
| 24 |
+
},
|
| 25 |
+
{
|
| 26 |
+
"method": "forward_mse_identity",
|
| 27 |
+
"description": "MSE decoder; identity covariance",
|
| 28 |
+
"hydraulic_64": 0.10416666666666667,
|
| 29 |
+
"hydraulic_128": 0.06579719545342229,
|
| 30 |
+
"temperature_64": 0.3230540060851927,
|
| 31 |
+
"temperature_128": 0.3443985891770501
|
| 32 |
+
},
|
| 33 |
+
{
|
| 34 |
+
"method": "forward_mse_full_product",
|
| 35 |
+
"description": "MSE/full; product of factor marginals",
|
| 36 |
+
"hydraulic_64": 0.46381578947368424,
|
| 37 |
+
"hydraulic_128": 0.42214912280701755,
|
| 38 |
+
"temperature_64": 0.6804534313725489,
|
| 39 |
+
"temperature_128": 0.7058823529411765
|
| 40 |
+
},
|
| 41 |
+
{
|
| 42 |
+
"method": "forward_ce_identity",
|
| 43 |
+
"description": "Joint CE decoder; identity covariance",
|
| 44 |
+
"hydraulic_64": 0.0668859649122807,
|
| 45 |
+
"hydraulic_128": 0.041666666666666664,
|
| 46 |
+
"temperature_64": 0.015318627450980393,
|
| 47 |
+
"temperature_128": 0.05606617647058824
|
| 48 |
+
},
|
| 49 |
+
{
|
| 50 |
+
"method": "forward_factor_ce_identity",
|
| 51 |
+
"description": "Factor marginal CE decoder; identity covariance",
|
| 52 |
+
"hydraulic_64": 0.11622807017543858,
|
| 53 |
+
"hydraulic_128": 0.06578947368421052,
|
| 54 |
+
"temperature_64": 0.07659313725490197,
|
| 55 |
+
"temperature_128": 0.0974264705882353
|
| 56 |
+
},
|
| 57 |
+
{
|
| 58 |
+
"method": "inverse_balanced",
|
| 59 |
+
"description": "Direct balanced CE; raw",
|
| 60 |
+
"hydraulic_64": 0.24122807017543857,
|
| 61 |
+
"hydraulic_128": 0.5285087719298246,
|
| 62 |
+
"temperature_64": 0.3633578431372549,
|
| 63 |
+
"temperature_128": 0.45067401960784315
|
| 64 |
+
},
|
| 65 |
+
{
|
| 66 |
+
"method": "inverse_unweighted",
|
| 67 |
+
"description": "Direct unweighted CE; raw",
|
| 68 |
+
"hydraulic_64": 0.2434210526315789,
|
| 69 |
+
"hydraulic_128": 0.5131578947368421,
|
| 70 |
+
"temperature_64": 0.3026960784313726,
|
| 71 |
+
"temperature_128": 0.42524509803921573
|
| 72 |
+
},
|
| 73 |
+
{
|
| 74 |
+
"method": "inverse_balanced_factor_temperature",
|
| 75 |
+
"description": "Direct balanced CE; original source factor temperatures",
|
| 76 |
+
"hydraulic_64": 0.25548245614035087,
|
| 77 |
+
"hydraulic_128": 0.5219298245614036,
|
| 78 |
+
"temperature_64": 0.3633578431372549,
|
| 79 |
+
"temperature_128": 0.45067401960784315
|
| 80 |
+
},
|
| 81 |
+
{
|
| 82 |
+
"method": "inverse_unweighted_factor_temperature",
|
| 83 |
+
"description": "Direct unweighted CE; source factor temperatures",
|
| 84 |
+
"hydraulic_64": 0.24890350877192982,
|
| 85 |
+
"hydraulic_128": 0.5043859649122807,
|
| 86 |
+
"temperature_64": 0.3026960784313726,
|
| 87 |
+
"temperature_128": 0.42524509803921573
|
| 88 |
+
}
|
| 89 |
+
],
|
| 90 |
+
"compute": {
|
| 91 |
+
"new_fits": 318,
|
| 92 |
+
"optimizer_updates": 129600,
|
| 93 |
+
"measured_fit_inference_analysis_seconds": 246.56308321398683,
|
| 94 |
+
"training_seconds": 222.80916958197486,
|
| 95 |
+
"estimated_training_dense_matmul_flops": 1256574873600,
|
| 96 |
+
"peak_gpu_allocated_bytes": 288381440,
|
| 97 |
+
"main_process_seconds_including_startup": 171.79019222594798,
|
| 98 |
+
"startup_limitation": "Extension timer starts after imports/CUDA context initialization; extension framework startup is not measured. All fitting/inference/calibration/serialization/replay costs are inside measured analysis time. Summary/replay postprocessing recorded separately. FLOPs are partial dense matmul estimates, not total inference compute.",
|
| 99 |
+
"cpu_scientific_execution": false,
|
| 100 |
+
"backbone_executions": 0,
|
| 101 |
+
"gpu": "NVIDIA H200 cuda:3",
|
| 102 |
+
"frozen_release_changed": false
|
| 103 |
+
},
|
| 104 |
+
"files": {
|
| 105 |
+
"main_summary": "run_01/summary.csv",
|
| 106 |
+
"main_all_metrics": "run_01/metrics.csv",
|
| 107 |
+
"extension_summary": "objective_extension_01/summary.csv",
|
| 108 |
+
"extension_per_seed": "objective_extension_01/summary_per_seed.csv",
|
| 109 |
+
"hydraulic_reversal": "run_01/reversal_summary.csv",
|
| 110 |
+
"hydraulic_mean_diagnostics": "run_01/reversal_diagnostics.json",
|
| 111 |
+
"frozen_source_diagnostics": "run_01/mean_diagnostics.csv",
|
| 112 |
+
"source_calibration_choices": "run_01/calibration_choices.csv"
|
| 113 |
+
},
|
| 114 |
+
"safe_public_export": "public_safe/",
|
| 115 |
+
"verification": {
|
| 116 |
+
"new_saved_metric_score_error": 0,
|
| 117 |
+
"original_archived_top1_top3_vector_error": 0,
|
| 118 |
+
"original_archived_maximum_confidence_error": 1.6825558705191312e-06,
|
| 119 |
+
"original_archived_maximum_nll_error": 2.488220451368761e-05,
|
| 120 |
+
"evaluated_method_arrays": 192,
|
| 121 |
+
"reversal_scenarios": 21
|
| 122 |
+
}
|
| 123 |
+
}
|
revision_r1/sensor/full_replay.json
ADDED
|
@@ -0,0 +1,39 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"status": "passed",
|
| 3 |
+
"refitted_models": 0,
|
| 4 |
+
"cells": 24,
|
| 5 |
+
"held_methods": 192,
|
| 6 |
+
"reversal_scenarios": 21,
|
| 7 |
+
"maximum_saved_score_error": 0.0,
|
| 8 |
+
"maximum_metric_error": 0.0,
|
| 9 |
+
"maximum_reversal_score_error": 0.0,
|
| 10 |
+
"original_archived_metric_vector_max_errors": {
|
| 11 |
+
"hydraulic_factorized_neural_top1": 0.0,
|
| 12 |
+
"hydraulic_factorized_neural_top3": 0.0,
|
| 13 |
+
"hydraulic_factorized_neural_nll": 2.488220451368761e-05,
|
| 14 |
+
"hydraulic_factorized_neural_confidence": 1.6825558705191312e-06,
|
| 15 |
+
"hydraulic_factorized_classifier_top1": 0.0,
|
| 16 |
+
"hydraulic_factorized_classifier_top3": 0.0,
|
| 17 |
+
"hydraulic_factorized_classifier_nll": 1.6876770096985183e-06,
|
| 18 |
+
"hydraulic_factorized_classifier_confidence": 4.1392002436690944e-08,
|
| 19 |
+
"temperature_factorized_neural_top1": 0.0,
|
| 20 |
+
"temperature_factorized_neural_top3": 0.0,
|
| 21 |
+
"temperature_factorized_neural_nll": 1.0818260330225371e-05,
|
| 22 |
+
"temperature_factorized_neural_confidence": 1.6244467709869426e-06,
|
| 23 |
+
"temperature_factorized_classifier_top1": 0.0,
|
| 24 |
+
"temperature_factorized_classifier_top3": 0.0,
|
| 25 |
+
"temperature_factorized_classifier_nll": 9.632170758777647e-07,
|
| 26 |
+
"temperature_factorized_classifier_confidence": 2.6531173480393022e-08
|
| 27 |
+
},
|
| 28 |
+
"archive_tolerances": {
|
| 29 |
+
"top1": 1e-05,
|
| 30 |
+
"top3": 1e-05,
|
| 31 |
+
"confidence": 1e-05,
|
| 32 |
+
"nll": 0.001
|
| 33 |
+
},
|
| 34 |
+
"notes": "New refits use float32 training and explicit dense evaluation; original archived scores use original Linear kernels. NLL tolerance accommodates small probability changes in log space; all ranking vectors agree exactly when reported error0.",
|
| 35 |
+
"seconds": 1.074040636071004,
|
| 36 |
+
"device": "cuda:3",
|
| 37 |
+
"gpu": "NVIDIA H200",
|
| 38 |
+
"cpu_scientific_execution": false
|
| 39 |
+
}
|
revision_r1/sensor/objective_extension.py
ADDED
|
@@ -0,0 +1,63 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Prospectively amended factor-marginal CE decoder, all original cells."""
|
| 2 |
+
from pathlib import Path
|
| 3 |
+
import argparse, hashlib, importlib.util, json, sys, time
|
| 4 |
+
import torch
|
| 5 |
+
|
| 6 |
+
ROOT=Path(__file__).resolve().parents[3]
|
| 7 |
+
local_trainer=Path(__file__).with_name('review_r1_sensor.py')
|
| 8 |
+
spec=importlib.util.spec_from_file_location('review_sensor',local_trainer if local_trainer.exists() else ROOT/'reproduce/review_r1_sensor.py')
|
| 9 |
+
M=importlib.util.module_from_spec(spec);spec.loader.exec_module(M)
|
| 10 |
+
|
| 11 |
+
|
| 12 |
+
@torch.no_grad()
|
| 13 |
+
def main():
|
| 14 |
+
parser=argparse.ArgumentParser();parser.add_argument('--output',required=True);parser.add_argument('--device',default='cuda:3');parser.add_argument('--release');args=parser.parse_args()
|
| 15 |
+
torch.cuda.set_device(args.device);torch.backends.cuda.matmul.allow_tf32=False;torch.backends.cudnn.allow_tf32=False
|
| 16 |
+
out=Path(args.output);assert not out.exists();out.mkdir();start=time.perf_counter();rows=[];timings=[];summary=[]
|
| 17 |
+
specpath=Path(__file__).with_name('objective_extension_spec.json')
|
| 18 |
+
M.write(out/'manifest.json',dict(timestamp_utc=M.stamp(),command=sys.argv,status='Post-review exploratory objective extension',
|
| 19 |
+
spec_sha256=hashlib.sha256(specpath.read_bytes()).hexdigest(),helper_sha256=hashlib.sha256(Path(__file__).read_bytes()).hexdigest(),
|
| 20 |
+
trainer_sha256=hashlib.sha256((ROOT/'reproduce/review_r1_sensor.py').read_bytes()).hexdigest(),torch_version=torch.__version__,gpu=torch.cuda.get_device_name()))
|
| 21 |
+
for cohort,sizes,seeds in [('hydraulic',[3,4,3,4],[20751,20752,20753]),('temperature',[10,3],[20851,20852,20853])]:
|
| 22 |
+
p=M.load(f'data/{cohort}/prepared',args.device)
|
| 23 |
+
if cohort=='hydraulic':p.update(p['masks'])
|
| 24 |
+
original=M.load(f'results/{cohort}/predictions',args.device)['results'];grid=M.F.hypotheses(sizes,args.device);zg=M.F.design(grid,sizes,True)
|
| 25 |
+
for n in [64,128]:
|
| 26 |
+
for seed in seeds:
|
| 27 |
+
cell_start=time.perf_counter();c=M.load(f'models/{cohort}/n{n}_s{seed}/factorized_neural',args.device);ids=c['fit_ids'];h=p['h'][ids]
|
| 28 |
+
record=next(r for r in original if r['seed']==seed and r['samples']==n and r['model']=='factorized_neural');held=record['held_rows'];true=M.F.joint_index(p['h'][held],sizes)
|
| 29 |
+
x=(p['x'][ids]-c['mean'])/c['scale'];target=(p['x'][held]-c['mean'])/c['scale'];st=M.F.joint_index(h,sizes);cov=torch.eye(x.shape[1],device=args.device,dtype=x.dtype)
|
| 30 |
+
fits=M.fit_cross(x,h,sizes,seed,'forward_factor_ce');fold=torch.arange(n,device=args.device)%4;oof=x.new_empty((n,len(grid)))
|
| 31 |
+
for f in range(4):oof[fold==f]=M.gaussian_scores(x[fold==f],M.dense(fits[f+1]['state'],zg),cov).log_softmax(-1)
|
| 32 |
+
logp=M.gaussian_scores(target,M.dense(fits[0]['state'],zg),cov).log_softmax(-1);calibrated,cal=M.calibrate(oof,st,logp)
|
| 33 |
+
scores={'forward_factor_ce_identity':logp,'forward_factor_ce_identity_scalar_temperature':calibrated};vectors={}
|
| 34 |
+
for name,v in scores.items():
|
| 35 |
+
rr,vectors[name]=M.score_rows(cohort,n,seed,name,'held',v,true,record['unseen'],record.get('days'));rows+=rr
|
| 36 |
+
source=oof if name=='forward_factor_ce_identity' else (oof/cal['temperature']).log_softmax(-1)
|
| 37 |
+
rr,_=M.score_rows(cohort,n,seed,name,'source_oof',source,st,torch.zeros(n,device=args.device,dtype=torch.bool));rows+=rr
|
| 38 |
+
mask=record['unseen'];top=vectors[name]['top3']
|
| 39 |
+
primary=torch.stack([top[mask&(record['days']==d)].mean() for d in (9,10,11,12)]).mean() if cohort=='temperature' else top[mask].mean()
|
| 40 |
+
summary.append(dict(cohort=cohort,labels=n,seed=seed,method=name,top3_primary=float(primary),**M.aggregate(vectors[name],mask),temperature=cal['temperature']))
|
| 41 |
+
artifact=dict(cohort=cohort,labels=n,seed=seed,fit_ids=ids,mean=c['mean'],scale=c['scale'],held_rows=held,true=true,unseen=record['unseen'],units=record.get('days'),fits=fits,held_logpost=scores,source_oof_logpost=oof,calibration=cal,metrics=vectors)
|
| 42 |
+
torch.save(artifact,out/f'{cohort}_n{n}_s{seed}.pt');M.sync()
|
| 43 |
+
# Replay full decoder scores from saved tensors and all per-row metrics.
|
| 44 |
+
saved=torch.load(out/f'{cohort}_n{n}_s{seed}.pt',map_location=args.device,weights_only=True)
|
| 45 |
+
fresh=M.gaussian_scores(target,M.dense(saved['fits'][0]['state'],zg),cov).log_softmax(-1);error=float((fresh-saved['held_logpost']['forward_factor_ce_identity']).abs().max());assert error<1e-8
|
| 46 |
+
timings.append(dict(cohort=cohort,labels=n,seed=seed,training_seconds=sum(f['training_seconds'] for f in fits),cell_seconds=time.perf_counter()-cell_start,
|
| 47 |
+
new_fits=len(fits),trainable_parameters=fits[0]['trainable_parameters'],network_training_executions=sum(f['steps'] for f in fits),estimated_training_dense_matmul_flops=sum(f['estimated_training_dense_matmul_flops'] for f in fits),replay_error=error))
|
| 48 |
+
M.dump_csv(out/'metrics.csv',rows);M.dump_csv(out/'summary_per_seed.csv',summary);M.write(out/'cell_compute.json',timings)
|
| 49 |
+
print(json.dumps(dict(cohort=cohort,labels=n,seed=seed,top3=float(primary),temperature=cal['temperature'],seconds=timings[-1]['cell_seconds'])),flush=True)
|
| 50 |
+
aggregate=[]
|
| 51 |
+
for cohort in ['hydraulic','temperature']:
|
| 52 |
+
for n in [64,128]:
|
| 53 |
+
for name in scores:
|
| 54 |
+
rr=[r for r in summary if r['cohort']==cohort and r['labels']==n and r['method']==name]
|
| 55 |
+
aggregate.append(dict(cohort=cohort,labels=n,method=name,**{k:sum(r[k] for r in rr)/len(rr) if all(r[k] is not None for r in rr) else None for k in ['top3_primary','top1','top3','nll','brier','confidence','ece10','coverage90','accuracy90']},seed_top3=[r['top3_primary'] for r in rr],temperatures=[r['temperature'] for r in rr]))
|
| 56 |
+
M.write(out/'summary.json',aggregate);M.dump_csv(out/'summary.csv',aggregate)
|
| 57 |
+
M.write(out/'compute.json',dict(status='completed',timestamp_utc=M.stamp(),new_fits=60,optimizer_updates=24000,complete_seconds=time.perf_counter()-start,
|
| 58 |
+
training_seconds=sum(t['training_seconds'] for t in timings),estimated_training_dense_matmul_flops=sum(t['estimated_training_dense_matmul_flops'] for t in timings),
|
| 59 |
+
flop_scope='Partial dense matmuls, forward/backward; complete time includes all inference/calibration/serialization/replay',
|
| 60 |
+
peak_cuda_allocated_bytes=torch.cuda.max_memory_allocated(),peak_cuda_reserved_bytes=torch.cuda.max_memory_reserved(),cpu_scientific_execution=False,cpu_offload=False,backbone_executions=0,device=args.device,gpu=torch.cuda.get_device_name()))
|
| 61 |
+
|
| 62 |
+
|
| 63 |
+
if __name__=='__main__':main()
|
revision_r1/sensor/objective_extension_01/cell_compute.json
ADDED
|
@@ -0,0 +1,146 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
[
|
| 2 |
+
{
|
| 3 |
+
"cohort": "hydraulic",
|
| 4 |
+
"labels": 64,
|
| 5 |
+
"seed": 20751,
|
| 6 |
+
"training_seconds": 9.247948644100688,
|
| 7 |
+
"cell_seconds": 11.992396004963666,
|
| 8 |
+
"new_fits": 5,
|
| 9 |
+
"trainable_parameters": 19936,
|
| 10 |
+
"network_training_executions": 2000,
|
| 11 |
+
"estimated_training_dense_matmul_flops": 33868800000,
|
| 12 |
+
"replay_error": 0.0
|
| 13 |
+
},
|
| 14 |
+
{
|
| 15 |
+
"cohort": "hydraulic",
|
| 16 |
+
"labels": 64,
|
| 17 |
+
"seed": 20752,
|
| 18 |
+
"training_seconds": 7.901687476900406,
|
| 19 |
+
"cell_seconds": 7.933070569997653,
|
| 20 |
+
"new_fits": 5,
|
| 21 |
+
"trainable_parameters": 19936,
|
| 22 |
+
"network_training_executions": 2000,
|
| 23 |
+
"estimated_training_dense_matmul_flops": 33868800000,
|
| 24 |
+
"replay_error": 0.0
|
| 25 |
+
},
|
| 26 |
+
{
|
| 27 |
+
"cohort": "hydraulic",
|
| 28 |
+
"labels": 64,
|
| 29 |
+
"seed": 20753,
|
| 30 |
+
"training_seconds": 7.900175572023727,
|
| 31 |
+
"cell_seconds": 7.930946532986127,
|
| 32 |
+
"new_fits": 5,
|
| 33 |
+
"trainable_parameters": 19936,
|
| 34 |
+
"network_training_executions": 2000,
|
| 35 |
+
"estimated_training_dense_matmul_flops": 33868800000,
|
| 36 |
+
"replay_error": 0.0
|
| 37 |
+
},
|
| 38 |
+
{
|
| 39 |
+
"cohort": "hydraulic",
|
| 40 |
+
"labels": 128,
|
| 41 |
+
"seed": 20751,
|
| 42 |
+
"training_seconds": 8.145796927856281,
|
| 43 |
+
"cell_seconds": 8.176515604951419,
|
| 44 |
+
"new_fits": 5,
|
| 45 |
+
"trainable_parameters": 19936,
|
| 46 |
+
"network_training_executions": 2000,
|
| 47 |
+
"estimated_training_dense_matmul_flops": 33868800000,
|
| 48 |
+
"replay_error": 0.0
|
| 49 |
+
},
|
| 50 |
+
{
|
| 51 |
+
"cohort": "hydraulic",
|
| 52 |
+
"labels": 128,
|
| 53 |
+
"seed": 20752,
|
| 54 |
+
"training_seconds": 7.722086073132232,
|
| 55 |
+
"cell_seconds": 10.04799306706991,
|
| 56 |
+
"new_fits": 5,
|
| 57 |
+
"trainable_parameters": 19936,
|
| 58 |
+
"network_training_executions": 2000,
|
| 59 |
+
"estimated_training_dense_matmul_flops": 33868800000,
|
| 60 |
+
"replay_error": 0.0
|
| 61 |
+
},
|
| 62 |
+
{
|
| 63 |
+
"cohort": "hydraulic",
|
| 64 |
+
"labels": 128,
|
| 65 |
+
"seed": 20753,
|
| 66 |
+
"training_seconds": 8.524675879976712,
|
| 67 |
+
"cell_seconds": 8.556502330000512,
|
| 68 |
+
"new_fits": 5,
|
| 69 |
+
"trainable_parameters": 19936,
|
| 70 |
+
"network_training_executions": 2000,
|
| 71 |
+
"estimated_training_dense_matmul_flops": 33868800000,
|
| 72 |
+
"replay_error": 0.0
|
| 73 |
+
},
|
| 74 |
+
{
|
| 75 |
+
"cohort": "temperature",
|
| 76 |
+
"labels": 64,
|
| 77 |
+
"seed": 20851,
|
| 78 |
+
"training_seconds": 7.763742531067692,
|
| 79 |
+
"cell_seconds": 7.835612436989322,
|
| 80 |
+
"new_fits": 5,
|
| 81 |
+
"trainable_parameters": 19936,
|
| 82 |
+
"network_training_executions": 2000,
|
| 83 |
+
"estimated_training_dense_matmul_flops": 7054560000,
|
| 84 |
+
"replay_error": 0.0
|
| 85 |
+
},
|
| 86 |
+
{
|
| 87 |
+
"cohort": "temperature",
|
| 88 |
+
"labels": 64,
|
| 89 |
+
"seed": 20852,
|
| 90 |
+
"training_seconds": 10.738027861109003,
|
| 91 |
+
"cell_seconds": 10.862897809944116,
|
| 92 |
+
"new_fits": 5,
|
| 93 |
+
"trainable_parameters": 19936,
|
| 94 |
+
"network_training_executions": 2000,
|
| 95 |
+
"estimated_training_dense_matmul_flops": 7054560000,
|
| 96 |
+
"replay_error": 0.0
|
| 97 |
+
},
|
| 98 |
+
{
|
| 99 |
+
"cohort": "temperature",
|
| 100 |
+
"labels": 64,
|
| 101 |
+
"seed": 20853,
|
| 102 |
+
"training_seconds": 8.26926949096378,
|
| 103 |
+
"cell_seconds": 8.328956548939459,
|
| 104 |
+
"new_fits": 5,
|
| 105 |
+
"trainable_parameters": 19936,
|
| 106 |
+
"network_training_executions": 2000,
|
| 107 |
+
"estimated_training_dense_matmul_flops": 7054560000,
|
| 108 |
+
"replay_error": 0.0
|
| 109 |
+
},
|
| 110 |
+
{
|
| 111 |
+
"cohort": "temperature",
|
| 112 |
+
"labels": 128,
|
| 113 |
+
"seed": 20851,
|
| 114 |
+
"training_seconds": 7.83897131413687,
|
| 115 |
+
"cell_seconds": 7.901723493938334,
|
| 116 |
+
"new_fits": 5,
|
| 117 |
+
"trainable_parameters": 19936,
|
| 118 |
+
"network_training_executions": 2000,
|
| 119 |
+
"estimated_training_dense_matmul_flops": 7054560000,
|
| 120 |
+
"replay_error": 0.0
|
| 121 |
+
},
|
| 122 |
+
{
|
| 123 |
+
"cohort": "temperature",
|
| 124 |
+
"labels": 128,
|
| 125 |
+
"seed": 20852,
|
| 126 |
+
"training_seconds": 7.855879723909311,
|
| 127 |
+
"cell_seconds": 7.91638747905381,
|
| 128 |
+
"new_fits": 5,
|
| 129 |
+
"trainable_parameters": 19936,
|
| 130 |
+
"network_training_executions": 2000,
|
| 131 |
+
"estimated_training_dense_matmul_flops": 7054560000,
|
| 132 |
+
"replay_error": 0.0
|
| 133 |
+
},
|
| 134 |
+
{
|
| 135 |
+
"cohort": "temperature",
|
| 136 |
+
"labels": 128,
|
| 137 |
+
"seed": 20853,
|
| 138 |
+
"training_seconds": 7.856508697965182,
|
| 139 |
+
"cell_seconds": 7.917562803952023,
|
| 140 |
+
"new_fits": 5,
|
| 141 |
+
"trainable_parameters": 19936,
|
| 142 |
+
"network_training_executions": 2000,
|
| 143 |
+
"estimated_training_dense_matmul_flops": 7054560000,
|
| 144 |
+
"replay_error": 0.0
|
| 145 |
+
}
|
| 146 |
+
]
|
revision_r1/sensor/objective_extension_01/compute.json
ADDED
|
@@ -0,0 +1,17 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"status": "completed",
|
| 3 |
+
"timestamp_utc": "2026-11-09T14:45:55.244783+00:00",
|
| 4 |
+
"new_fits": 60,
|
| 5 |
+
"optimizer_updates": 24000,
|
| 6 |
+
"complete_seconds": 105.47859474900179,
|
| 7 |
+
"training_seconds": 99.76477019314189,
|
| 8 |
+
"estimated_training_dense_matmul_flops": 245540160000,
|
| 9 |
+
"flop_scope": "Partial dense matmuls, forward/backward; complete time includes all inference/calibration/serialization/replay",
|
| 10 |
+
"peak_cuda_allocated_bytes": 229255680,
|
| 11 |
+
"peak_cuda_reserved_bytes": 255852544,
|
| 12 |
+
"cpu_scientific_execution": false,
|
| 13 |
+
"cpu_offload": false,
|
| 14 |
+
"backbone_executions": 0,
|
| 15 |
+
"device": "cuda:3",
|
| 16 |
+
"gpu": "NVIDIA H200"
|
| 17 |
+
}
|
revision_r1/sensor/objective_extension_01/hydraulic_n128_s20751.json
ADDED
|
@@ -0,0 +1 @@
|
|
|
|
|
|
|
| 1 |
+
{"dict":[["cohort","hydraulic"],["labels",128],["seed",20751],["fit_ids",{"tensor":"t000000"}],["mean",{"tensor":"t000001"}],["scale",{"tensor":"t000002"}],["held_rows",{"tensor":"t000003"}],["true",{"tensor":"t000004"}],["unseen",{"tensor":"t000005"}],["units",null],["fits",{"list":[{"dict":[["state",{"dict":[["net.0.weight",{"tensor":"t000006"}],["net.0.bias",{"tensor":"t000007"}],["net.2.weight",{"tensor":"t000008"}],["net.2.bias",{"tensor":"t000009"}]]}],["trainable_parameters",19936],["steps",400],["kind","forward_factor_ce"],["seed",20751],["training_seconds",1.7130040070042014],["training_network_executions",400],["estimated_training_dense_matmul_flops",6773760000],["loss_trace",{"list":[{"dict":[["update",1],["preupdate_loss",1.3153672218322754]]},{"dict":[["update",100],["preupdate_loss",0.11518652737140656]]},{"dict":[["update",200],["preupdate_loss",0.07788994908332825]]},{"dict":[["update",400],["preupdate_loss",0.1391909271478653]]}]}]]},{"dict":[["state",{"dict":[["net.0.weight",{"tensor":"t000010"}],["net.0.bias",{"tensor":"t000011"}],["net.2.weight",{"tensor":"t000012"}],["net.2.bias",{"tensor":"t000013"}]]}],["trainable_parameters",19936],["steps",400],["kind","forward_factor_ce"],["seed",20852],["training_seconds",1.7215261249803007],["training_network_executions",400],["estimated_training_dense_matmul_flops",6773760000],["loss_trace",{"list":[{"dict":[["update",1],["preupdate_loss",1.26554274559021]]},{"dict":[["update",100],["preupdate_loss",0.10422307252883911]]},{"dict":[["update",200],["preupdate_loss",0.060535937547683716]]},{"dict":[["update",400],["preupdate_loss",0.21105968952178955]]}]}]]},{"dict":[["state",{"dict":[["net.0.weight",{"tensor":"t000014"}],["net.0.bias",{"tensor":"t000015"}],["net.2.weight",{"tensor":"t000016"}],["net.2.bias",{"tensor":"t000017"}]]}],["trainable_parameters",19936],["steps",400],["kind","forward_factor_ce"],["seed",20853],["training_seconds",1.632717469939962],["training_network_executions",400],["estimated_training_dense_matmul_flops",6773760000],["loss_trace",{"list":[{"dict":[["update",1],["preupdate_loss",1.3454340696334839]]},{"dict":[["update",100],["preupdate_loss",0.08547832071781158]]},{"dict":[["update",200],["preupdate_loss",0.04600851982831955]]},{"dict":[["update",400],["preupdate_loss",0.03054380789399147]]}]}]]},{"dict":[["state",{"dict":[["net.0.weight",{"tensor":"t000018"}],["net.0.bias",{"tensor":"t000019"}],["net.2.weight",{"tensor":"t000020"}],["net.2.bias",{"tensor":"t000021"}]]}],["trainable_parameters",19936],["steps",400],["kind","forward_factor_ce"],["seed",20854],["training_seconds",1.564134583924897],["training_network_executions",400],["estimated_training_dense_matmul_flops",6773760000],["loss_trace",{"list":[{"dict":[["update",1],["preupdate_loss",1.3180103302001953]]},{"dict":[["update",100],["preupdate_loss",0.10781454294919968]]},{"dict":[["update",200],["preupdate_loss",0.06580552458763123]]},{"dict":[["update",400],["preupdate_loss",0.04327245429158211]]}]}]]},{"dict":[["state",{"dict":[["net.0.weight",{"tensor":"t000022"}],["net.0.bias",{"tensor":"t000023"}],["net.2.weight",{"tensor":"t000024"}],["net.2.bias",{"tensor":"t000025"}]]}],["trainable_parameters",19936],["steps",400],["kind","forward_factor_ce"],["seed",20855],["training_seconds",1.5144147420069203],["training_network_executions",400],["estimated_training_dense_matmul_flops",6773760000],["loss_trace",{"list":[{"dict":[["update",1],["preupdate_loss",1.234365701675415]]},{"dict":[["update",100],["preupdate_loss",0.10966001451015472]]},{"dict":[["update",200],["preupdate_loss",0.058905404061079025]]},{"dict":[["update",400],["preupdate_loss",0.2836851179599762]]}]}]]}]}],["held_logpost",{"dict":[["forward_factor_ce_identity",{"tensor":"t000026"}],["forward_factor_ce_identity_scalar_temperature",{"tensor":"t000027"}]]}],["source_oof_logpost",{"tensor":"t000028"}],["calibration",{"dict":[["temperature",8.0],["grid",{"list":[1.0,2.0,4.0,8.0,16.0,32.0,64.0]}],["oof_nll",{"list":[11.849315593533733,6.273474467157997,3.9106504839236744,3.373548745521792,3.6546327409681982,4.099476012113334,4.459736913706311]}]]}],["metrics",{"dict":[["forward_factor_ce_identity",{"dict":[["top1",{"tensor":"t000029"}],["top3",{"tensor":"t000030"}],["nll",{"tensor":"t000031"}],["confidence",{"tensor":"t000032"}],["brier",{"tensor":"t000033"}]]}],["forward_factor_ce_identity_scalar_temperature",{"dict":[["top1",{"tensor":"t000034"}],["top3",{"tensor":"t000035"}],["nll",{"tensor":"t000036"}],["confidence",{"tensor":"t000037"}],["brier",{"tensor":"t000038"}]]}]]}]]}
|
revision_r1/sensor/objective_extension_01/hydraulic_n128_s20751.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:d7a21511d36785300dd873f79aadcec87bf8de59a73b621328e26cc45a137bda
|
| 3 |
+
size 2258151
|
revision_r1/sensor/objective_extension_01/hydraulic_n128_s20752.json
ADDED
|
@@ -0,0 +1 @@
|
|
|
|
|
|
|
| 1 |
+
{"dict":[["cohort","hydraulic"],["labels",128],["seed",20752],["fit_ids",{"tensor":"t000000"}],["mean",{"tensor":"t000001"}],["scale",{"tensor":"t000002"}],["held_rows",{"tensor":"t000003"}],["true",{"tensor":"t000004"}],["unseen",{"tensor":"t000005"}],["units",null],["fits",{"list":[{"dict":[["state",{"dict":[["net.0.weight",{"tensor":"t000006"}],["net.0.bias",{"tensor":"t000007"}],["net.2.weight",{"tensor":"t000008"}],["net.2.bias",{"tensor":"t000009"}]]}],["trainable_parameters",19936],["steps",400],["kind","forward_factor_ce"],["seed",20752],["training_seconds",1.521117574069649],["training_network_executions",400],["estimated_training_dense_matmul_flops",6773760000],["loss_trace",{"list":[{"dict":[["update",1],["preupdate_loss",1.2658435106277466]]},{"dict":[["update",100],["preupdate_loss",0.10903939604759216]]},{"dict":[["update",200],["preupdate_loss",0.16448232531547546]]},{"dict":[["update",400],["preupdate_loss",0.03899599239230156]]}]}]]},{"dict":[["state",{"dict":[["net.0.weight",{"tensor":"t000010"}],["net.0.bias",{"tensor":"t000011"}],["net.2.weight",{"tensor":"t000012"}],["net.2.bias",{"tensor":"t000013"}]]}],["trainable_parameters",19936],["steps",400],["kind","forward_factor_ce"],["seed",20853],["training_seconds",1.534898433019407],["training_network_executions",400],["estimated_training_dense_matmul_flops",6773760000],["loss_trace",{"list":[{"dict":[["update",1],["preupdate_loss",1.3638601303100586]]},{"dict":[["update",100],["preupdate_loss",0.09481599181890488]]},{"dict":[["update",200],["preupdate_loss",0.04286932945251465]]},{"dict":[["update",400],["preupdate_loss",0.023131797090172768]]}]}]]},{"dict":[["state",{"dict":[["net.0.weight",{"tensor":"t000014"}],["net.0.bias",{"tensor":"t000015"}],["net.2.weight",{"tensor":"t000016"}],["net.2.bias",{"tensor":"t000017"}]]}],["trainable_parameters",19936],["steps",400],["kind","forward_factor_ce"],["seed",20854],["training_seconds",1.5903978729620576],["training_network_executions",400],["estimated_training_dense_matmul_flops",6773760000],["loss_trace",{"list":[{"dict":[["update",1],["preupdate_loss",1.3067092895507812]]},{"dict":[["update",100],["preupdate_loss",0.1004253476858139]]},{"dict":[["update",200],["preupdate_loss",0.27468007802963257]]},{"dict":[["update",400],["preupdate_loss",0.04639362171292305]]}]}]]},{"dict":[["state",{"dict":[["net.0.weight",{"tensor":"t000018"}],["net.0.bias",{"tensor":"t000019"}],["net.2.weight",{"tensor":"t000020"}],["net.2.bias",{"tensor":"t000021"}]]}],["trainable_parameters",19936],["steps",400],["kind","forward_factor_ce"],["seed",20855],["training_seconds",1.5241462660487741],["training_network_executions",400],["estimated_training_dense_matmul_flops",6773760000],["loss_trace",{"list":[{"dict":[["update",1],["preupdate_loss",1.2226519584655762]]},{"dict":[["update",100],["preupdate_loss",0.061646513640880585]]},{"dict":[["update",200],["preupdate_loss",0.028353851288557053]]},{"dict":[["update",400],["preupdate_loss",0.02910551242530346]]}]}]]},{"dict":[["state",{"dict":[["net.0.weight",{"tensor":"t000022"}],["net.0.bias",{"tensor":"t000023"}],["net.2.weight",{"tensor":"t000024"}],["net.2.bias",{"tensor":"t000025"}]]}],["trainable_parameters",19936],["steps",400],["kind","forward_factor_ce"],["seed",20856],["training_seconds",1.551525927032344],["training_network_executions",400],["estimated_training_dense_matmul_flops",6773760000],["loss_trace",{"list":[{"dict":[["update",1],["preupdate_loss",1.2441983222961426]]},{"dict":[["update",100],["preupdate_loss",0.09223012626171112]]},{"dict":[["update",200],["preupdate_loss",0.05816187337040901]]},{"dict":[["update",400],["preupdate_loss",0.04458361864089966]]}]}]]}]}],["held_logpost",{"dict":[["forward_factor_ce_identity",{"tensor":"t000026"}],["forward_factor_ce_identity_scalar_temperature",{"tensor":"t000027"}]]}],["source_oof_logpost",{"tensor":"t000028"}],["calibration",{"dict":[["temperature",4.0],["grid",{"list":[1.0,2.0,4.0,8.0,16.0,32.0,64.0]}],["oof_nll",{"list":[6.900842189252238,3.717516208510413,2.527189305081876,2.615379022527092,3.2545442193886887,3.8928117248978285,4.353807781944336]}]]}],["metrics",{"dict":[["forward_factor_ce_identity",{"dict":[["top1",{"tensor":"t000029"}],["top3",{"tensor":"t000030"}],["nll",{"tensor":"t000031"}],["confidence",{"tensor":"t000032"}],["brier",{"tensor":"t000033"}]]}],["forward_factor_ce_identity_scalar_temperature",{"dict":[["top1",{"tensor":"t000034"}],["top3",{"tensor":"t000035"}],["nll",{"tensor":"t000036"}],["confidence",{"tensor":"t000037"}],["brier",{"tensor":"t000038"}]]}]]}]]}
|
revision_r1/sensor/objective_extension_01/hydraulic_n128_s20752.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:84e37c6d7a7ec02217682fedad56211ef67be4757a0376682fc6428f70f92f61
|
| 3 |
+
size 2258151
|
revision_r1/sensor/objective_extension_01/hydraulic_n128_s20753.json
ADDED
|
@@ -0,0 +1 @@
|
|
|
|
|
|
|
| 1 |
+
{"dict":[["cohort","hydraulic"],["labels",128],["seed",20753],["fit_ids",{"tensor":"t000000"}],["mean",{"tensor":"t000001"}],["scale",{"tensor":"t000002"}],["held_rows",{"tensor":"t000003"}],["true",{"tensor":"t000004"}],["unseen",{"tensor":"t000005"}],["units",null],["fits",{"list":[{"dict":[["state",{"dict":[["net.0.weight",{"tensor":"t000006"}],["net.0.bias",{"tensor":"t000007"}],["net.2.weight",{"tensor":"t000008"}],["net.2.bias",{"tensor":"t000009"}]]}],["trainable_parameters",19936],["steps",400],["kind","forward_factor_ce"],["seed",20753],["training_seconds",1.7182763000018895],["training_network_executions",400],["estimated_training_dense_matmul_flops",6773760000],["loss_trace",{"list":[{"dict":[["update",1],["preupdate_loss",1.3494446277618408]]},{"dict":[["update",100],["preupdate_loss",0.09609153866767883]]},{"dict":[["update",200],["preupdate_loss",0.04170093685388565]]},{"dict":[["update",400],["preupdate_loss",0.11369231343269348]]}]}]]},{"dict":[["state",{"dict":[["net.0.weight",{"tensor":"t000010"}],["net.0.bias",{"tensor":"t000011"}],["net.2.weight",{"tensor":"t000012"}],["net.2.bias",{"tensor":"t000013"}]]}],["trainable_parameters",19936],["steps",400],["kind","forward_factor_ce"],["seed",20854],["training_seconds",1.6955450871028006],["training_network_executions",400],["estimated_training_dense_matmul_flops",6773760000],["loss_trace",{"list":[{"dict":[["update",1],["preupdate_loss",1.3265430927276611]]},{"dict":[["update",100],["preupdate_loss",0.08156754076480865]]},{"dict":[["update",200],["preupdate_loss",0.025151949375867844]]},{"dict":[["update",400],["preupdate_loss",0.03886566683650017]]}]}]]},{"dict":[["state",{"dict":[["net.0.weight",{"tensor":"t000014"}],["net.0.bias",{"tensor":"t000015"}],["net.2.weight",{"tensor":"t000016"}],["net.2.bias",{"tensor":"t000017"}]]}],["trainable_parameters",19936],["steps",400],["kind","forward_factor_ce"],["seed",20855],["training_seconds",1.7018361849477515],["training_network_executions",400],["estimated_training_dense_matmul_flops",6773760000],["loss_trace",{"list":[{"dict":[["update",1],["preupdate_loss",1.2241082191467285]]},{"dict":[["update",100],["preupdate_loss",0.0709477961063385]]},{"dict":[["update",200],["preupdate_loss",0.03540191799402237]]},{"dict":[["update",400],["preupdate_loss",0.02366386540234089]]}]}]]},{"dict":[["state",{"dict":[["net.0.weight",{"tensor":"t000018"}],["net.0.bias",{"tensor":"t000019"}],["net.2.weight",{"tensor":"t000020"}],["net.2.bias",{"tensor":"t000021"}]]}],["trainable_parameters",19936],["steps",400],["kind","forward_factor_ce"],["seed",20856],["training_seconds",1.7114315149374306],["training_network_executions",400],["estimated_training_dense_matmul_flops",6773760000],["loss_trace",{"list":[{"dict":[["update",1],["preupdate_loss",1.2223014831542969]]},{"dict":[["update",100],["preupdate_loss",0.09891912341117859]]},{"dict":[["update",200],["preupdate_loss",0.05410601198673248]]},{"dict":[["update",400],["preupdate_loss",0.1466662436723709]]}]}]]},{"dict":[["state",{"dict":[["net.0.weight",{"tensor":"t000022"}],["net.0.bias",{"tensor":"t000023"}],["net.2.weight",{"tensor":"t000024"}],["net.2.bias",{"tensor":"t000025"}]]}],["trainable_parameters",19936],["steps",400],["kind","forward_factor_ce"],["seed",20857],["training_seconds",1.69758679298684],["training_network_executions",400],["estimated_training_dense_matmul_flops",6773760000],["loss_trace",{"list":[{"dict":[["update",1],["preupdate_loss",1.3295478820800781]]},{"dict":[["update",100],["preupdate_loss",0.08611084520816803]]},{"dict":[["update",200],["preupdate_loss",0.03314342349767685]]},{"dict":[["update",400],["preupdate_loss",0.027886219322681427]]}]}]]}]}],["held_logpost",{"dict":[["forward_factor_ce_identity",{"tensor":"t000026"}],["forward_factor_ce_identity_scalar_temperature",{"tensor":"t000027"}]]}],["source_oof_logpost",{"tensor":"t000028"}],["calibration",{"dict":[["temperature",4.0],["grid",{"list":[1.0,2.0,4.0,8.0,16.0,32.0,64.0]}],["oof_nll",{"list":[5.086095402329374,2.831339798198033,2.1274483841182823,2.397373893919174,3.089160951570917,3.771096803890446,4.2730538150441015]}]]}],["metrics",{"dict":[["forward_factor_ce_identity",{"dict":[["top1",{"tensor":"t000029"}],["top3",{"tensor":"t000030"}],["nll",{"tensor":"t000031"}],["confidence",{"tensor":"t000032"}],["brier",{"tensor":"t000033"}]]}],["forward_factor_ce_identity_scalar_temperature",{"dict":[["top1",{"tensor":"t000034"}],["top3",{"tensor":"t000035"}],["nll",{"tensor":"t000036"}],["confidence",{"tensor":"t000037"}],["brier",{"tensor":"t000038"}]]}]]}]]}
|
revision_r1/sensor/objective_extension_01/hydraulic_n128_s20753.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:fb1eb8085bbd37b334f4a7033d7b0ca0863981f0263528ccced9ada34167dace
|
| 3 |
+
size 2258151
|
revision_r1/sensor/objective_extension_01/hydraulic_n64_s20751.json
ADDED
|
@@ -0,0 +1 @@
|
|
|
|
|
|
|
| 1 |
+
{"dict":[["cohort","hydraulic"],["labels",64],["seed",20751],["fit_ids",{"tensor":"t000000"}],["mean",{"tensor":"t000001"}],["scale",{"tensor":"t000002"}],["held_rows",{"tensor":"t000003"}],["true",{"tensor":"t000004"}],["unseen",{"tensor":"t000005"}],["units",null],["fits",{"list":[{"dict":[["state",{"dict":[["net.0.weight",{"tensor":"t000006"}],["net.0.bias",{"tensor":"t000007"}],["net.2.weight",{"tensor":"t000008"}],["net.2.bias",{"tensor":"t000009"}]]}],["trainable_parameters",19936],["steps",400],["kind","forward_factor_ce"],["seed",20751],["training_seconds",2.5850065340055153],["training_network_executions",400],["estimated_training_dense_matmul_flops",6773760000],["loss_trace",{"list":[{"dict":[["update",1],["preupdate_loss",1.3451077938079834]]},{"dict":[["update",100],["preupdate_loss",0.055335015058517456]]},{"dict":[["update",200],["preupdate_loss",0.009100543335080147]]},{"dict":[["update",400],["preupdate_loss",0.0018230908317491412]]}]}]]},{"dict":[["state",{"dict":[["net.0.weight",{"tensor":"t000010"}],["net.0.bias",{"tensor":"t000011"}],["net.2.weight",{"tensor":"t000012"}],["net.2.bias",{"tensor":"t000013"}]]}],["trainable_parameters",19936],["steps",400],["kind","forward_factor_ce"],["seed",20852],["training_seconds",1.6536050019785762],["training_network_executions",400],["estimated_training_dense_matmul_flops",6773760000],["loss_trace",{"list":[{"dict":[["update",1],["preupdate_loss",1.2514609098434448]]},{"dict":[["update",100],["preupdate_loss",0.049963369965553284]]},{"dict":[["update",200],["preupdate_loss",0.007042030803859234]]},{"dict":[["update",400],["preupdate_loss",0.0014284837525337934]]}]}]]},{"dict":[["state",{"dict":[["net.0.weight",{"tensor":"t000014"}],["net.0.bias",{"tensor":"t000015"}],["net.2.weight",{"tensor":"t000016"}],["net.2.bias",{"tensor":"t000017"}]]}],["trainable_parameters",19936],["steps",400],["kind","forward_factor_ce"],["seed",20853],["training_seconds",1.6466316950973123],["training_network_executions",400],["estimated_training_dense_matmul_flops",6773760000],["loss_trace",{"list":[{"dict":[["update",1],["preupdate_loss",1.3599019050598145]]},{"dict":[["update",100],["preupdate_loss",0.03786250203847885]]},{"dict":[["update",200],["preupdate_loss",0.0057496195659041405]]},{"dict":[["update",400],["preupdate_loss",0.0012487568892538548]]}]}]]},{"dict":[["state",{"dict":[["net.0.weight",{"tensor":"t000018"}],["net.0.bias",{"tensor":"t000019"}],["net.2.weight",{"tensor":"t000020"}],["net.2.bias",{"tensor":"t000021"}]]}],["trainable_parameters",19936],["steps",400],["kind","forward_factor_ce"],["seed",20854],["training_seconds",1.6881663490785286],["training_network_executions",400],["estimated_training_dense_matmul_flops",6773760000],["loss_trace",{"list":[{"dict":[["update",1],["preupdate_loss",1.3245327472686768]]},{"dict":[["update",100],["preupdate_loss",0.04571075364947319]]},{"dict":[["update",200],["preupdate_loss",0.006450130604207516]]},{"dict":[["update",400],["preupdate_loss",0.0015182875795289874]]}]}]]},{"dict":[["state",{"dict":[["net.0.weight",{"tensor":"t000022"}],["net.0.bias",{"tensor":"t000023"}],["net.2.weight",{"tensor":"t000024"}],["net.2.bias",{"tensor":"t000025"}]]}],["trainable_parameters",19936],["steps",400],["kind","forward_factor_ce"],["seed",20855],["training_seconds",1.674539063940756],["training_network_executions",400],["estimated_training_dense_matmul_flops",6773760000],["loss_trace",{"list":[{"dict":[["update",1],["preupdate_loss",1.220789909362793]]},{"dict":[["update",100],["preupdate_loss",0.07876121997833252]]},{"dict":[["update",200],["preupdate_loss",0.04276982694864273]]},{"dict":[["update",400],["preupdate_loss",0.03329639509320259]]}]}]]}]}],["held_logpost",{"dict":[["forward_factor_ce_identity",{"tensor":"t000026"}],["forward_factor_ce_identity_scalar_temperature",{"tensor":"t000027"}]]}],["source_oof_logpost",{"tensor":"t000028"}],["calibration",{"dict":[["temperature",4.0],["grid",{"list":[1.0,2.0,4.0,8.0,16.0,32.0,64.0]}],["oof_nll",{"list":[6.792243070171102,3.5698963043271705,2.327733437548014,2.3977731484902653,3.0476395521039077,3.7180662668380786,4.225876077364923]}]]}],["metrics",{"dict":[["forward_factor_ce_identity",{"dict":[["top1",{"tensor":"t000029"}],["top3",{"tensor":"t000030"}],["nll",{"tensor":"t000031"}],["confidence",{"tensor":"t000032"}],["brier",{"tensor":"t000033"}]]}],["forward_factor_ce_identity_scalar_temperature",{"dict":[["top1",{"tensor":"t000034"}],["top3",{"tensor":"t000035"}],["nll",{"tensor":"t000036"}],["confidence",{"tensor":"t000037"}],["brier",{"tensor":"t000038"}]]}]]}]]}
|
revision_r1/sensor/objective_extension_01/hydraulic_n64_s20751.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:3e5a56962a870d83d2cd4037f8ec1c0aa39fe0b20111579bda7d435a98683f28
|
| 3 |
+
size 2183911
|
revision_r1/sensor/objective_extension_01/hydraulic_n64_s20752.json
ADDED
|
@@ -0,0 +1 @@
|
|
|
|
|
|
|
| 1 |
+
{"dict":[["cohort","hydraulic"],["labels",64],["seed",20752],["fit_ids",{"tensor":"t000000"}],["mean",{"tensor":"t000001"}],["scale",{"tensor":"t000002"}],["held_rows",{"tensor":"t000003"}],["true",{"tensor":"t000004"}],["unseen",{"tensor":"t000005"}],["units",null],["fits",{"list":[{"dict":[["state",{"dict":[["net.0.weight",{"tensor":"t000006"}],["net.0.bias",{"tensor":"t000007"}],["net.2.weight",{"tensor":"t000008"}],["net.2.bias",{"tensor":"t000009"}]]}],["trainable_parameters",19936],["steps",400],["kind","forward_factor_ce"],["seed",20752],["training_seconds",1.7232382470974699],["training_network_executions",400],["estimated_training_dense_matmul_flops",6773760000],["loss_trace",{"list":[{"dict":[["update",1],["preupdate_loss",1.2678219079971313]]},{"dict":[["update",100],["preupdate_loss",0.0787072479724884]]},{"dict":[["update",200],["preupdate_loss",0.056699153035879135]]},{"dict":[["update",400],["preupdate_loss",0.7450239658355713]]}]}]]},{"dict":[["state",{"dict":[["net.0.weight",{"tensor":"t000010"}],["net.0.bias",{"tensor":"t000011"}],["net.2.weight",{"tensor":"t000012"}],["net.2.bias",{"tensor":"t000013"}]]}],["trainable_parameters",19936],["steps",400],["kind","forward_factor_ce"],["seed",20853],["training_seconds",1.6219279859215021],["training_network_executions",400],["estimated_training_dense_matmul_flops",6773760000],["loss_trace",{"list":[{"dict":[["update",1],["preupdate_loss",1.383439302444458]]},{"dict":[["update",100],["preupdate_loss",0.05290248245000839]]},{"dict":[["update",200],["preupdate_loss",0.022894427180290222]]},{"dict":[["update",400],["preupdate_loss",0.017069250345230103]]}]}]]},{"dict":[["state",{"dict":[["net.0.weight",{"tensor":"t000014"}],["net.0.bias",{"tensor":"t000015"}],["net.2.weight",{"tensor":"t000016"}],["net.2.bias",{"tensor":"t000017"}]]}],["trainable_parameters",19936],["steps",400],["kind","forward_factor_ce"],["seed",20854],["training_seconds",1.518231512978673],["training_network_executions",400],["estimated_training_dense_matmul_flops",6773760000],["loss_trace",{"list":[{"dict":[["update",1],["preupdate_loss",1.2927300930023193]]},{"dict":[["update",100],["preupdate_loss",0.07159949839115143]]},{"dict":[["update",200],["preupdate_loss",0.026884913444519043]]},{"dict":[["update",400],["preupdate_loss",0.018253501504659653]]}]}]]},{"dict":[["state",{"dict":[["net.0.weight",{"tensor":"t000018"}],["net.0.bias",{"tensor":"t000019"}],["net.2.weight",{"tensor":"t000020"}],["net.2.bias",{"tensor":"t000021"}]]}],["trainable_parameters",19936],["steps",400],["kind","forward_factor_ce"],["seed",20855],["training_seconds",1.5201711799018085],["training_network_executions",400],["estimated_training_dense_matmul_flops",6773760000],["loss_trace",{"list":[{"dict":[["update",1],["preupdate_loss",1.2043473720550537]]},{"dict":[["update",100],["preupdate_loss",0.040777452290058136]]},{"dict":[["update",200],["preupdate_loss",0.02084321342408657]]},{"dict":[["update",400],["preupdate_loss",0.017022639513015747]]}]}]]},{"dict":[["state",{"dict":[["net.0.weight",{"tensor":"t000022"}],["net.0.bias",{"tensor":"t000023"}],["net.2.weight",{"tensor":"t000024"}],["net.2.bias",{"tensor":"t000025"}]]}],["trainable_parameters",19936],["steps",400],["kind","forward_factor_ce"],["seed",20856],["training_seconds",1.5181185510009527],["training_network_executions",400],["estimated_training_dense_matmul_flops",6773760000],["loss_trace",{"list":[{"dict":[["update",1],["preupdate_loss",1.2638598680496216]]},{"dict":[["update",100],["preupdate_loss",0.0664515271782875]]},{"dict":[["update",200],["preupdate_loss",0.038003109395504]]},{"dict":[["update",400],["preupdate_loss",0.03006787970662117]]}]}]]}]}],["held_logpost",{"dict":[["forward_factor_ce_identity",{"tensor":"t000026"}],["forward_factor_ce_identity_scalar_temperature",{"tensor":"t000027"}]]}],["source_oof_logpost",{"tensor":"t000028"}],["calibration",{"dict":[["temperature",8.0],["grid",{"list":[1.0,2.0,4.0,8.0,16.0,32.0,64.0]}],["oof_nll",{"list":[9.869375650516783,5.1392550287919665,3.160878618792568,2.883185872356636,3.3682674820768237,3.9486493509371563,4.385519147434494]}]]}],["metrics",{"dict":[["forward_factor_ce_identity",{"dict":[["top1",{"tensor":"t000029"}],["top3",{"tensor":"t000030"}],["nll",{"tensor":"t000031"}],["confidence",{"tensor":"t000032"}],["brier",{"tensor":"t000033"}]]}],["forward_factor_ce_identity_scalar_temperature",{"dict":[["top1",{"tensor":"t000034"}],["top3",{"tensor":"t000035"}],["nll",{"tensor":"t000036"}],["confidence",{"tensor":"t000037"}],["brier",{"tensor":"t000038"}]]}]]}]]}
|
revision_r1/sensor/objective_extension_01/hydraulic_n64_s20752.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:e4ff49f92f19eeb70adf1a6c200eb238839153310e435c938951a6658adfbc7c
|
| 3 |
+
size 2183911
|