Publish completed WeatherNext Cyclones Mini daily comparison and audited three-model chart

docs/daily_storm_benchmark.md

@@ -6,7 +6,9 @@ Mean track error is 798.4 km for 1.1 and 471.2 km for 1.2 (50-member mean); +120
 
 The completed [native pressure follow-up](intensity_benchmark.md) now provides a matched 1.1 comparison on 134 daily starts / 40 storms from the same frozen plan. JMA MAE is 13.53 / 12.84 hPa for 1.1 / 1.2; USA MAE is 12.84 / 12.55 hPa. Each uses exact common agency-specific labels. The 1,339 remaining issue inputs are unavailable to the frozen 1.1 intensity pipeline, never zero-scored or filled with future observations. Paired whole-storm intervals include zero. These are not all-270-storm pressure scores.
 
-On 1 October, `release_tools/build_release_benchmark.py` independently rechecked all forecast hashes, coordinate conversions and daily/storm route aggregates, reproduced the native pressure postprocessor and generated the [shared Site snapshot](../evaluation/released_daily/released_daily_benchmark.json) and [audit receipt](../evaluation/released_daily/released_daily_verification.json). No inference or weight changes were necessary. DeepMind has not been evaluated on this larger daily plan; an older cohort's score cannot be transferred into it.
+On 1 October, `release_tools/build_release_benchmark.py` independently rechecked all forecast hashes, coordinate conversions and daily/storm route aggregates, reproduced the native pressure postprocessor and generated the [shared snapshot](../evaluation/released_daily/released_daily_benchmark.json) and [original Trackformer audit receipt](../evaluation/released_daily/released_daily_verification.json). No inference or weight changes were necessary.
+
+The [completed WeatherNext Cyclones Mini comparison](deepmind_daily_benchmark.md) now covers exactly the same 1,473 starts / 270 storms. Mini's combined mean track error is 498.4 km, direction error 39.76°, and JMA pressure MAE 10.20 hPa on 134 shared intensity starts. Its recent-only mean track error is 288.5 km versus 477.2 km for 1.2. The [independent three-way publication audit](../evaluation/deepmind_daily/publication_audit.json) rechecks the CUDA archive, unchanged original Trackformer arrays and recomputed common-support scores; the older 1 October receipt remains the audit of the original two-model snapshot, not a hash receipt for the subsequently merged JSON. No older benchmark's DeepMind score has been transferred into this comparison.
 
 ## What counts as a case and a score
 

docs/deepmind_daily_benchmark.md

@@ -1,8 +1,10 @@
-# Matched WeatherNext Cyclones Mini daily benchmark
+# Completed WeatherNext Cyclones Mini daily benchmark
 
 ## Model and frozen plan
 
-The reference is Google's official [WeatherNext Cyclones Mini](https://github.com/google-deepmind/weathernext), the released `<2024` checkpoint trained through 2023—not a custom imitation or full-sized WeatherNext. It runs one seeded member using the existing JAX CPU runtime on the Mac. This is a new benchmark; scores remain pending until saved forecasts and the full verification receipt exist.
+**Complete and verified:** all **1,473 daily forecasts / 270 Western Pacific storms** finished on an **NVIDIA GeForce RTX 3070 using JAX CUDA in Windows/WSL**. The final worker audit is dated **4 October 2026, 12:55 UTC**. Publication independently rechecked all returned case hashes, physical grids, unchanged Trackformer reference arrays and exact common-support scores. No new inference was performed during this review.
+
+The reference is Google's official [WeatherNext Cyclones Mini](https://github.com/google-deepmind/weathernext#weathernext-cyclones-mini): **`WeatherNextCyclones_Mini_<2024`**, trained through 2023 at native **1° resolution**. The pinned [software package is v0.3.0](https://github.com/google-deepmind/weathernext/blob/89c4b2a77a1c57b328b909c575550fd2e5aadc9c/setup.py); the package version and checkpoint generation identify different things. This is not a custom imitation, full-sized WeatherNext 2/Cyclones, or a WeatherNext 3 checkpoint. Mini's scores do not stand in for those larger models.
 
 - Cohort: **1,473 daily starts / 270 Western Pacific storms**, identical to the frozen Trackformer 1.1/1.2 route comparison.
 - Forecast: twenty six-hour steps, **+6 to +120 h**, with the exact observed issue-time centre as the initialized-storm anchor.
@@ -10,6 +12,34 @@ The reference is Google's official [WeatherNext Cyclones Mini](https://github.co
 - Weights SHA-256: `a1bb151457077248d70a458b1a7b19deacd2926fbcb46eee4ff2691444e3596e`.
 - Cohort SHA-256: `965184f1e5ab3fcb52e304f39b38583a14f753870aa2295f2d01942a441dcfd6`.
 - Official code commit: `89c4b2a77a1c57b328b909c575550fd2e5aadc9c`.
+- CUDA protocol SHA-256: `91b31582f24ffdd4f2309eddb0cc68498a2c31cab7cbabaf06f43dfbff413606`.
+- Runtime: JAX/JAXlib 0.4.38, Python 3.12.3, highest float32 matmul precision, TF32 disabled.
+
+## Results
+
+| Matched development metric | Trackformer 1.1 | Trackformer 1.2 · mean of 50 | Mini · one member | Coverage |
+| --- | ---: | ---: | ---: | --- |
+| Mean track error · km, lower is better | 798.41 | 471.20 | 498.37 | 1,473 days / 270 storms |
+| +120 h track error · km, lower is better | 1,646.40 | 1,031.63 | 1,031.91 | 1,473 days / 270 storms |
+| Direction error · degrees, lower is better | 51.58 | 34.96 | 39.76 | 1,473 days / 270 storms |
+| Centred route-shape similarity · higher is better | 0.7544 | 0.8837 | 0.9010 | 1,473 days / 270 storms |
+| JMA central-pressure MAE · hPa, lower is better | 13.53 | 12.84 | 10.20 | 134 days / 40 storms / 2,637 leads |
+| JMA pressure-curve similarity · higher is better | 0.7074 | 0.7118 | 0.8088 | 134 days / 40 storms |
+| USA central-pressure MAE · hPa, lower is better | 12.84 | 12.55 | 11.55 | 134 days / 40 storms / 2,325 leads |
+| USA pressure-curve similarity · higher is better | 0.7237 | 0.6637 | 0.7445 | 131 days / 40 storms |
+
+Pressure measures central-pressure intensity, not map-grid error. The lower USA curve count excludes flat/short curves; it is not zero-filled. All three models' pressure-curve case IDs were verified equal for each agency. DeepMind geographic path/Fréchet scores and whole-storm confidence intervals are not supplied by this export and remain unavailable.
+
+### Historical and recent groups
+
+| Period | Daily starts / storms | Mean track error · 1.1 / 1.2 / Mini, km | Direction error · 1.1 / 1.2 / Mini, degrees |
+| --- | --- | --- | --- |
+| 1980–1999 | 1,336 / 230 | 780.50 / 470.17 / 534.86 | 51.36 / 35.01 / 41.26 |
+| Storms beginning in 2024+ | 137 / 40 | 901.45 / 477.16 / 288.51 | 52.90 / 34.68 / 31.16 |
+
+Mini has better recent-only track scores even though 1.2 has lower position/heading errors in the combined cohort. Mini also has lower JMA/USA pressure MAE and higher pressure-curve similarity. Historical fitting-year overlap, different weather inputs and different ensemble sizes prevent a general or equal-compute superiority claim. All pressure comparisons here belong to the recent group; no historical pressure score is inferred from missing native 1.1 inputs.
+
+[Full results, period breakdowns and per-lead track errors](../evaluation/deepmind_daily/benchmark.json) · [Shared snapshot and existing 1.1/1.2 uncertainty](../evaluation/released_daily/released_daily_benchmark.json) · [Completion receipt](../evaluation/deepmind_daily/verification.json) · [Independent review](../evaluation/deepmind_daily/publication_audit.json)
 
 ## Like-for-like scoring
 
@@ -23,21 +53,19 @@ The official six-hour direct tracker uses a frozen initialized-storm continuity
 
 ## Execution and verification
 
-The [resumable runner](../release_tools/deepmind_daily_benchmark.py) freezes source hashes, the cohort, original baseline forecast hashes and checkpoint identity. Workers save exact input-state hashes, twenty route points, central-pressure values/masks and twenty native WP grids. Future truth is read only **after** inference for scoring; Trackformer forecasts and weights are unchanged.
+The original [Mac CPU runner and scoring definitions](../release_tools/deepmind_daily_benchmark.py) provided the frozen plan. The completed run used the separately pinned, resumable RTX 3070 CUDA package; the old CPU launch instructions are not the execution record of these results. The [frozen CUDA protocol](../evaluation/deepmind_daily/protocol.json), [2,946-file case manifest](../evaluation/deepmind_daily/case-manifest.json) and [CPU/CUDA canary](../evaluation/deepmind_daily/canary.json) identify the completed run.
 
-Local output: `/Volumes/D/typhoon_predict/output/deepmind-daily-20261002`. `progress.json` records on-disk coverage; `logs/` holds diagnostics. `verification.json` is produced only after all 1,473 cases pass the final audit. These are local results, not automatically published Site API data. No score is uploaded before review.
+The strict same-runtime CPU/CUDA canary passed unchanged 0.01 hPa mean / 0.1 hPa maximum field tolerances. Its older cross-runtime Mac comparison failed the maximum-field gate and remains explicitly recorded as failed; that diagnostic is not numerical-equivalence certification. No CPU forecast was counted or relabelled as a CUDA case.
 
-Existing Mac runtime and weights are reused without package installation. Earlier CPU rollouts suggest **several days** for the full run; retrieval and compilation affect the total. All generated states, caches and outputs stay on `/Volumes/D`. Cloud backfill, training and Site deployments are untouched.
+The publication review checks all 1,473 IDs, original reference hashes, exact issue−6 h / issue input-time metadata and saved input hashes, all twenty output times, finite native pressure grids, and recomputed three-model route/pressure metrics. **Raw ERA5 input states are not in the returned result-only ZIP**: this review preserves the original workers' causal-input receipts and hashes, rather than claiming to independently reopen those raw states.
 
-```bash
-# Existing compatible WeatherNext environment and project assets required.
-python release_tools/deepmind_daily_benchmark.py prepare \
-  --project /Volumes/D/typhoon_predict \
-  --output /Volumes/D/typhoon_predict/output/deepmind-daily-20261002
+To repeat the read-only archive review, use the existing frozen project reference forecasts and intensity labels, NumPy, and the returned result archive. It does not run a model or download weather:
 
-python release_tools/deepmind_daily_benchmark.py launch \
+```bash
+python release_tools/import_deepmind_release_results.py \
+  --archive /path/to/DeepMind_RTX3070_results_20261004_125651_467960.zip \
   --project /Volumes/D/typhoon_predict \
-  --output /Volumes/D/typhoon_predict/output/deepmind-daily-20261002
+  --output /Volumes/D/typhoon_predict/remote_typhoon_predict_repo/evaluation/deepmind_daily
 ```
 
-`prepare` is only for a fresh directory. `launch` refuses an active run; `run` resumes completed cases without repetition. Source retrieval has bounded retries and failures remain visible. Final completion requires all planned cases, exact leads, valid routes/fields, unchanged forecast hashes and the verification receipt—not a process exit or count alone.
+Completed scores are also exposed by the existing [public benchmark API](https://trackformer-weatherlab.rudin-euler-8253.chatgpt.site/api/benchmarks/released). Documentation publication does not replace immutable forecasts, change either model's weights, restart the benchmark, modify active training, or redeploy the website. The original two-model verification receipt remains unchanged; the new publication audit identifies the merged snapshot's exact hash.

evaluation/deepmind_daily/benchmark.json

@@ -0,0 +1,1007 @@
+{
+  "schema": "deepmind-site-import-v1",
+  "status": "complete_verified",
+  "cohort": {
+    "storms": 270,
+    "daily_issues": 1473,
+    "sha256": "965184f1e5ab3fcb52e304f39b38583a14f753870aa2295f2d01942a441dcfd6",
+    "leads_hours": [
+      6,
+      12,
+      18,
+      24,
+      30,
+      36,
+      42,
+      48,
+      54,
+      60,
+      66,
+      72,
+      78,
+      84,
+      90,
+      96,
+      102,
+      108,
+      114,
+      120
+    ]
+  },
+  "models": [
+    {
+      "key": "v11",
+      "version": "1.1",
+      "member_count": null,
+      "member_policy": "Existing frozen weighted-route reference",
+      "checkpoint_sha256": null
+    },
+    {
+      "key": "v12",
+      "version": "1.2",
+      "member_count": 50,
+      "member_policy": "Existing 50-member mean",
+      "checkpoint_sha256": "f194a23d3f91ea76ad776dfad942fabd669eeae8b3fd665815463095367e9ee0"
+    },
+    {
+      "key": "deepmind",
+      "version": "deepmind",
+      "member_count": 1,
+      "member_policy": "One official Mini member, seed 0",
+      "checkpoint_sha256": "a1bb151457077248d70a458b1a7b19deacd2926fbcb46eee4ff2691444e3596e"
+    }
+  ],
+  "metrics": [
+    {
+      "key": "mean_track_error_km",
+      "title": "Track position error",
+      "unit": "km",
+      "higher_is_better": false,
+      "values": {
+        "1.1": 798.4143395733709,
+        "1.2": 471.2045276140213,
+        "deepmind": 498.3660837262175
+      },
+      "coverage": {
+        "1.1": {
+          "storms": 270
+        },
+        "1.2": {
+          "storms": 270
+        },
+        "deepmind": {
+          "storms": 270
+        }
+      }
+    },
+    {
+      "key": "direction_error_deg",
+      "title": "Track direction error",
+      "unit": "degrees",
+      "higher_is_better": false,
+      "values": {
+        "1.1": 51.58419207122625,
+        "1.2": 34.959947454592786,
+        "deepmind": 39.76380497312257
+      },
+      "coverage": {
+        "1.1": {
+          "storms": 270
+        },
+        "1.2": {
+          "storms": 270
+        },
+        "deepmind": {
+          "storms": 270
+        }
+      }
+    },
+    {
+      "key": "shape_similarity",
+      "title": "Track shape similarity",
+      "unit": "0-1",
+      "higher_is_better": true,
+      "values": {
+        "1.1": 0.7543824571538857,
+        "1.2": 0.8837028632459877,
+        "deepmind": 0.9010381186926554
+      },
+      "coverage": {
+        "1.1": {
+          "storms": 270
+        },
+        "1.2": {
+          "storms": 270
+        },
+        "deepmind": {
+          "storms": 270
+        }
+      }
+    },
+    {
+      "key": "pressure_JMA_hpa",
+      "title": "Central-pressure error (JMA)",
+      "unit": "hPa",
+      "higher_is_better": false,
+      "values": {
+        "1.1": 13.53233861138585,
+        "1.2": 12.837077366789085,
+        "deepmind": 10.197678417354012
+      },
+      "common_lead_points": 2637,
+      "coverage": {
+        "1.1": {
+          "storms": 40,
+          "daily_cases": 134
+        },
+        "1.2": {
+          "storms": 40,
+          "daily_cases": 134
+        },
+        "deepmind": {
+          "storms": 40,
+          "daily_cases": 134
+        }
+      }
+    },
+    {
+      "key": "pressure_JMA_curve_similarity",
+      "title": "Pressure-curve similarity (JMA)",
+      "unit": "0-1",
+      "higher_is_better": true,
+      "values": {
+        "1.1": 0.7074066420243609,
+        "1.2": 0.7118461112422476,
+        "deepmind": 0.8087969164941555
+      },
+      "common_lead_points": 2637,
+      "coverage": {
+        "1.1": {
+          "storms": 40,
+          "daily_cases": 134
+        },
+        "1.2": {
+          "storms": 40,
+          "daily_cases": 134
+        },
+        "deepmind": {
+          "storms": 40,
+          "daily_cases": 134
+        }
+      }
+    },
+    {
+      "key": "pressure_USA_hpa",
+      "title": "Central-pressure error (USA)",
+      "unit": "hPa",
+      "higher_is_better": false,
+      "values": {
+        "1.1": 12.835629459598886,
+        "1.2": 12.552942457392373,
+        "deepmind": 11.546903198440878
+      },
+      "common_lead_points": 2325,
+      "coverage": {
+        "1.1": {
+          "storms": 40,
+          "daily_cases": 134
+        },
+        "1.2": {
+          "storms": 40,
+          "daily_cases": 134
+        },
+        "deepmind": {
+          "storms": 40,
+          "daily_cases": 134
+        }
+      }
+    },
+    {
+      "key": "pressure_USA_curve_similarity",
+      "title": "Pressure-curve similarity (USA)",
+      "unit": "0-1",
+      "higher_is_better": true,
+      "values": {
+        "1.1": 0.7236794525870405,
+        "1.2": 0.6636769485468179,
+        "deepmind": 0.7445409788756472
+      },
+      "common_lead_points": 2325,
+      "coverage": {
+        "1.1": {
+          "storms": 40,
+          "daily_cases": 131
+        },
+        "1.2": {
+          "storms": 40,
+          "daily_cases": 131
+        },
+        "deepmind": {
+          "storms": 40,
+          "daily_cases": 131
+        }
+      }
+    }
+  ],
+  "aggregate_equal_complete_storm": {
+    "route": {
+      "1.1": {
+        "mean_track_error_km": {
+          "value": 798.4143395733709,
+          "storms": 270
+        },
+        "track_error_120h_km": {
+          "value": 1646.3960476448165,
+          "storms": 270
+        },
+        "direction_error_deg": {
+          "value": 51.58419207122625,
+          "storms": 270
+        },
+        "shape_similarity": {
+          "value": 0.7543824571538857,
+          "storms": 270
+        },
+        "track_error_by_lead_km": [
+          66.10969036943266,
+          133.4783672768527,
+          202.3585175411488,
+          273.2879621656292,
+          346.90051929279923,
+          419.8767857705207,
+          493.5704476997114,
+          568.0858703129328,
+          644.1251568785259,
+          722.6238658146501,
+          803.5946300337959,
+          886.616879788694,
+          973.2847405022445,
+          1060.5849499902793,
+          1151.5785924519794,
+          1244.9737736697598,
+          1343.5921960496646,
+          1443.1911582149028,
+          1544.0566399990746,
+          1646.396047644816
+        ]
+      },
+      "1.2": {
+        "mean_track_error_km": {
+          "value": 471.2045276140213,
+          "storms": 270
+        },
+        "track_error_120h_km": {
+          "value": 1031.633570368749,
+          "storms": 270
+        },
+        "direction_error_deg": {
+          "value": 34.959947454592786,
+          "storms": 270
+        },
+        "shape_similarity": {
+          "value": 0.8837028632459877,
+          "storms": 270
+        },
+        "track_error_by_lead_km": [
+          48.276258844589,
+          91.12774309041492,
+          129.1208135780606,
+          164.481223771445,
+          199.06958720909273,
+          234.93256251923398,
+          272.48049128499,
+          311.4789218238006,
+          353.46861840507904,
+          399.3840596538071,
+          449.2708765186405,
+          502.4908306263511,
+          558.3174531497493,
+          616.1753924353181,
+          677.6609275099131,
+          740.9340851986844,
+          808.8023210064223,
+          880.0772095202088,
+          954.9076057658763,
+          1031.633570368749
+        ]
+      },
+      "deepmind": {
+        "mean_track_error_km": {
+          "value": 498.3660837262175,
+          "storms": 270
+        },
+        "track_error_120h_km": {
+          "value": 1031.9091421404376,
+          "storms": 270
+        },
+        "direction_error_deg": {
+          "value": 39.76380497312257,
+          "storms": 270
+        },
+        "shape_similarity": {
+          "value": 0.9010381186926554,
+          "storms": 270
+        },
+        "track_error_by_lead_km": [
+          105.79431937309762,
+          136.30842390708193,
+          168.8925890316698,
+          208.95643055237963,
+          243.63979975302811,
+          278.11432944685924,
+          316.6521861156824,
+          357.1736913330865,
+          392.8310729414844,
+          436.5373191336134,
+          478.075813984495,
+          522.4673167277231,
+          576.9905349536721,
+          634.2907658367048,
+          690.6818730442626,
+          746.7391690084353,
+          807.6243133903178,
+          879.0423409543573,
+          954.6002428959632,
+          1031.909142140437
+        ]
+      }
+    },
+    "pressure": {
+      "JMA": {
+        "daily_cases": 134,
+        "storms": 40,
+        "common_lead_points": 2637,
+        "models": {
+          "1.1": {
+            "mae_hpa": {
+              "value": 13.53233861138585,
+              "storms": 40,
+              "daily_cases": 134
+            },
+            "curve_similarity": {
+              "value": 0.7074066420243609,
+              "storms": 40,
+              "daily_cases": 134
+            }
+          },
+          "1.2": {
+            "mae_hpa": {
+              "value": 12.837077366789085,
+              "storms": 40,
+              "daily_cases": 134
+            },
+            "curve_similarity": {
+              "value": 0.7118461112422476,
+              "storms": 40,
+              "daily_cases": 134
+            }
+          },
+          "deepmind": {
+            "mae_hpa": {
+              "value": 10.197678417354012,
+              "storms": 40,
+              "daily_cases": 134
+            },
+            "curve_similarity": {
+              "value": 0.8087969164941555,
+              "storms": 40,
+              "daily_cases": 134
+            }
+          }
+        }
+      },
+      "USA": {
+        "daily_cases": 134,
+        "storms": 40,
+        "common_lead_points": 2325,
+        "models": {
+          "1.1": {
+            "mae_hpa": {
+              "value": 12.835629459598886,
+              "storms": 40,
+              "daily_cases": 134
+            },
+            "curve_similarity": {
+              "value": 0.7236794525870405,
+              "storms": 40,
+              "daily_cases": 131
+            }
+          },
+          "1.2": {
+            "mae_hpa": {
+              "value": 12.552942457392373,
+              "storms": 40,
+              "daily_cases": 134
+            },
+            "curve_similarity": {
+              "value": 0.6636769485468179,
+              "storms": 40,
+              "daily_cases": 131
+            }
+          },
+          "deepmind": {
+            "mae_hpa": {
+              "value": 11.546903198440878,
+              "storms": 40,
+              "daily_cases": 134
+            },
+            "curve_similarity": {
+              "value": 0.7445409788756472,
+              "storms": 40,
+              "daily_cases": 131
+            }
+          }
+        }
+      }
+    }
+  },
+  "periods": {
+    "historical_1980_1999": {
+      "route": {
+        "1.1": {
+          "mean_track_error_km": {
+            "value": 780.4956251297209,
+            "storms": 230
+          },
+          "track_error_120h_km": {
+            "value": 1617.68537007603,
+            "storms": 230
+          },
+          "direction_error_deg": {
+            "value": 51.35513468466848,
+            "storms": 230
+          },
+          "shape_similarity": {
+            "value": 0.756462863476536,
+            "storms": 230
+          },
+          "track_error_by_lead_km": [
+            63.95070387461008,
+            129.56057152366327,
+            196.91757281750267,
+            266.29351479253165,
+            337.4313487866377,
+            408.5365624559959,
+            480.4125299649792,
+            554.0338033012117,
+            628.3701179860931,
+            704.556054381504,
+            783.1646921434294,
+            864.6586584087509,
+            949.2682475583533,
+            1035.1766032588005,
+            1125.2495149384633,
+            1218.047210654655,
+            1315.6802847719387,
+            1415.099441487216,
+            1515.8196994120542,
+            1617.6853700760307
+          ]
+        },
+        "1.2": {
+          "mean_track_error_km": {
+            "value": 470.1683227329181,
+            "storms": 230
+          },
+          "track_error_120h_km": {
+            "value": 1022.9686679865174,
+            "storms": 230
+          },
+          "direction_error_deg": {
+            "value": 35.008663500237326,
+            "storms": 230
+          },
+          "shape_similarity": {
+            "value": 0.880227238887145,
+            "storms": 230
+          },
+          "track_error_by_lead_km": [
+            48.00936024802012,
+            91.97411308450529,
+            131.2279581683132,
+            167.32238620520673,
+            201.3545994142221,
+            236.4041620555792,
+            272.8634618572267,
+            311.62044814780137,
+            353.54628740878155,
+            398.8030667186859,
+            448.4963800108917,
+            501.7638464578764,
+            557.7134662207559,
+            615.9415271471161,
+            677.1770146233011,
+            739.235914695639,
+            804.7389191190093,
+            874.8402182910024,
+            947.3646567979115,
+            1022.9686679865174
+          ]
+        },
+        "deepmind": {
+          "mean_track_error_km": {
+            "value": 534.8619522162429,
+            "storms": 230
+          },
+          "track_error_120h_km": {
+            "value": 1082.253921983746,
+            "storms": 230
+          },
+          "direction_error_deg": {
+            "value": 41.2608569444939,
+            "storms": 230
+          },
+          "shape_similarity": {
+            "value": 0.8933690501351934,
+            "storms": 230
+          },
+          "track_error_by_lead_km": [
+            114.73579161496248,
+            150.05176773716605,
+            186.62578636599312,
+            230.15185300349486,
+            269.44361640486176,
+            308.8927597927205,
+            350.43195179308117,
+            393.53364129433595,
+            432.49137917943165,
+            476.05487406828956,
+            517.4587499637993,
+            564.5613200677591,
+            620.1750658120939,
+            679.1636664565294,
+            737.6134951348805,
+            794.3707077948451,
+            855.8112853459712,
+            927.6904921578597,
+            1005.7269183530368,
+            1082.2539219837458
+          ]
+        }
+      },
+      "pressure": {
+        "JMA": {
+          "daily_cases": 0,
+          "storms": 0,
+          "common_lead_points": 0,
+          "models": {
+            "1.1": {
+              "mae_hpa": {
+                "value": null,
+                "storms": 0,
+                "daily_cases": 0
+              },
+              "curve_similarity": {
+                "value": null,
+                "storms": 0,
+                "daily_cases": 0
+              }
+            },
+            "1.2": {
+              "mae_hpa": {
+                "value": null,
+                "storms": 0,
+                "daily_cases": 0
+              },
+              "curve_similarity": {
+                "value": null,
+                "storms": 0,
+                "daily_cases": 0
+              }
+            },
+            "deepmind": {
+              "mae_hpa": {
+                "value": null,
+                "storms": 0,
+                "daily_cases": 0
+              },
+              "curve_similarity": {
+                "value": null,
+                "storms": 0,
+                "daily_cases": 0
+              }
+            }
+          }
+        },
+        "USA": {
+          "daily_cases": 0,
+          "storms": 0,
+          "common_lead_points": 0,
+          "models": {
+            "1.1": {
+              "mae_hpa": {
+                "value": null,
+                "storms": 0,
+                "daily_cases": 0
+              },
+              "curve_similarity": {
+                "value": null,
+                "storms": 0,
+                "daily_cases": 0
+              }
+            },
+            "1.2": {
+              "mae_hpa": {
+                "value": null,
+                "storms": 0,
+                "daily_cases": 0
+              },
+              "curve_similarity": {
+                "value": null,
+                "storms": 0,
+                "daily_cases": 0
+              }
+            },
+            "deepmind": {
+              "mae_hpa": {
+                "value": null,
+                "storms": 0,
+                "daily_cases": 0
+              },
+              "curve_similarity": {
+                "value": null,
+                "storms": 0,
+                "daily_cases": 0
+              }
+            }
+          }
+        }
+      }
+    },
+    "recent_2024_onward": {
+      "route": {
+        "1.1": {
+          "mean_track_error_km": {
+            "value": 901.4469476243576,
+            "storms": 40
+          },
+          "track_error_120h_km": {
+            "value": 1811.4824436653348,
+            "storms": 40
+          },
+          "direction_error_deg": {
+            "value": 52.90127204393345,
+            "storms": 40
+          },
+          "shape_similarity": {
+            "value": 0.7424201207986456,
+            "storms": 40
+          },
+          "track_error_by_lead_km": [
+            78.52386271466261,
+            156.0056928576924,
+            233.6439497021143,
+            313.5060345609392,
+            401.3482497032272,
+            485.08306982903804,
+            569.2284746744191,
+            648.8852556303291,
+            734.7166305100175,
+            826.5137815552425,
+            921.0667729034061,
+            1012.8766527233684,
+            1111.3795749296182,
+            1206.6829436962798,
+            1302.9707881546994,
+            1399.8015110066206,
+            1504.0856858965933,
+            1604.7185293991047,
+            1706.4190483744474,
+            1811.4824436653344
+          ]
+        },
+        "1.2": {
+          "mean_track_error_km": {
+            "value": 477.1627056803647,
+            "storms": 40
+          },
+          "track_error_120h_km": {
+            "value": 1081.4567590665797,
+            "storms": 40
+          },
+          "direction_error_deg": {
+            "value": 34.679830192136706,
+            "storms": 40
+          },
+          "shape_similarity": {
+            "value": 0.9036877033093333,
+            "storms": 40
+          },
+          "track_error_by_lead_km": [
+            49.81092577486012,
+            86.26111562439522,
+            117.00473218410767,
+            148.14453977731517,
+            185.93076702959854,
+            226.4708651852496,
+            270.27841049462984,
+            310.6651454607977,
+            353.022021633789,
+            402.7247690307546,
+            453.72423143819543,
+            506.6709895950815,
+            561.790377991463,
+            617.5201178424808,
+            680.4434266079353,
+            750.6985655911949,
+            832.166881859046,
+            910.1899090881452,
+            998.2795623316731,
+            1081.45675906658
+          ]
+        },
+        "deepmind": {
+          "mean_track_error_km": {
+            "value": 288.51483990857145,
+            "storms": 40
+          },
+          "track_error_120h_km": {
+            "value": 742.426658041413,
+            "storms": 40
+          },
+          "direction_error_deg": {
+            "value": 31.155756137737363,
+            "storms": 40
+          },
+          "shape_similarity": {
+            "value": 0.9451352628980612,
+            "storms": 40
+          },
+          "track_error_by_lead_km": [
+            54.38085398237462,
+            57.28419688409821,
+            66.92670435931014,
+            87.08275145846693,
+            95.26785400498491,
+            101.13835495815766,
+            122.41853347063868,
+            148.10397905590278,
+            164.7843120732874,
+            209.3113782592264,
+            251.62393210349515,
+            280.4267975225144,
+            328.6794825177443,
+            376.2715872727139,
+            420.82504602320796,
+            472.8578209865762,
+            530.5492246453126,
+            599.3154715342171,
+            660.6218590177866,
+            742.426658041413
+          ]
+        }
+      },
+      "pressure": {
+        "JMA": {
+          "daily_cases": 134,
+          "storms": 40,
+          "common_lead_points": 2637,
+          "models": {
+            "1.1": {
+              "mae_hpa": {
+                "value": 13.53233861138585,
+                "storms": 40,
+                "daily_cases": 134
+              },
+              "curve_similarity": {
+                "value": 0.7074066420243609,
+                "storms": 40,
+                "daily_cases": 134
+              }
+            },
+            "1.2": {
+              "mae_hpa": {
+                "value": 12.837077366789085,
+                "storms": 40,
+                "daily_cases": 134
+              },
+              "curve_similarity": {
+                "value": 0.7118461112422476,
+                "storms": 40,
+                "daily_cases": 134
+              }
+            },
+            "deepmind": {
+              "mae_hpa": {
+                "value": 10.197678417354012,
+                "storms": 40,
+                "daily_cases": 134
+              },
+              "curve_similarity": {
+                "value": 0.8087969164941555,
+                "storms": 40,
+                "daily_cases": 134
+              }
+            }
+          }
+        },
+        "USA": {
+          "daily_cases": 134,
+          "storms": 40,
+          "common_lead_points": 2325,
+          "models": {
+            "1.1": {
+              "mae_hpa": {
+                "value": 12.835629459598886,
+                "storms": 40,
+                "daily_cases": 134
+              },
+              "curve_similarity": {
+                "value": 0.7236794525870405,
+                "storms": 40,
+                "daily_cases": 131
+              }
+            },
+            "1.2": {
+              "mae_hpa": {
+                "value": 12.552942457392373,
+                "storms": 40,
+                "daily_cases": 134
+              },
+              "curve_similarity": {
+                "value": 0.6636769485468179,
+                "storms": 40,
+                "daily_cases": 131
+              }
+            },
+            "deepmind": {
+              "mae_hpa": {
+                "value": 11.546903198440878,
+                "storms": 40,
+                "daily_cases": 134
+              },
+              "curve_similarity": {
+                "value": 0.7445409788756472,
+                "storms": 40,
+                "daily_cases": 131
+              }
+            }
+          }
+        }
+      }
+    }
+  },
+  "interpretation": "Development comparison: 1.2 reference is 50-member input-perturbed mean; Mini is one member, different weather source. Historical fitting-year overlap; recent 2024+ reported separately.",
+  "aggregation": "Same frozen scorer: leads per day; days within storm; equal storms. Common three-way moving directions and pressure-valid leads; separate JMA/USA labels.",
+  "pressure_scope": "Native matched subset, common three-way leads. Missing is null, never zero.",
+  "verification_sha256": "8f29a767ab5db61cf6773f5806cc110789329b04db48361715e5b4c295545c89",
+  "case_manifest_sha256": "30d35cfb79e809227fd0d255f9b5173ad18e781c7450ed4bbf0211553f8e63bd",
+  "canary": {
+    "wp_pressure_mean_abs_difference_hpa": 0.00025739270264116574,
+    "wp_pressure_max_abs_difference_hpa": 0.04547119140625,
+    "wp_pressure_max_abs_difference_hpa_by_lead": [
+      0.00030517578125,
+      0.000244140625,
+      0.000244140625,
+      0.00018310546875,
+      0.00030517578125,
+      0.00054931640625,
+      0.00079345703125,
+      0.0009765625,
+      0.002685546875,
+      0.00341796875,
+      0.0032958984375,
+      0.0048828125,
+      0.004638671875,
+      0.0032958984375,
+      0.00543212890625,
+      0.00787353515625,
+      0.00836181640625,
+      0.023681640625,
+      0.04547119140625,
+      0.042724609375
+    ],
+    "field_gate_limits_hpa": {
+      "mean": 0.01,
+      "max": 0.1
+    },
+    "field_gate_passed": true,
+    "tracker_max_difference_km": 0.14586496919052705,
+    "central_pressure_max_abs_difference_hpa": 0.07400935954797205,
+    "pressure_masks_match": true,
+    "cpu_numerical_equivalence": true,
+    "status": "cuda_canary_passed",
+    "tested_at_utc": "2026-10-04T12:55:06.985459+00:00",
+    "validation_protocol": {
+      "id": "same-runtime-strict-fp32-cpu-cuda-v2",
+      "primary_reference": "One fresh CPU rollout in the same pinned WSL runtime and inference code path",
+      "legacy_reference": "Bundled Mac CPU output retained as a separate cross-runtime diagnostic, not the primary gate",
+      "field_gate_limits_hpa": {
+        "mean": 0.01,
+        "max": 0.1
+      },
+      "jax": "0.4.38",
+      "jaxlib": "0.4.38",
+      "case_index": 0,
+      "cpu_counts_as_benchmark": false,
+      "tolerances_changed": false
+    },
+    "runtime_match": true,
+    "input_bytes_match": true,
+    "protocol_match": true,
+    "case_index": 0,
+    "checkpoint_sha256": "a1bb151457077248d70a458b1a7b19deacd2926fbcb46eee4ff2691444e3596e",
+    "gpu_verified_steps": 20,
+    "protocol_sha256": "91b31582f24ffdd4f2309eddb0cc68498a2c31cab7cbabaf06f43dfbff413606",
+    "cpu_forecast_sha256": "a8700f10b93dcf76b20439ed0cf71c312274398813af4ef8c6df51cbc75b0964",
+    "cuda_forecast_sha256": "0338d2b907bf0f4a605883b4c778bf961fe7406b6916161317b25a39b66fa68c",
+    "cpu_metadata_identity": "a2b686e98309ebcda11482f521b68cc8657171d08606d24e0c303484c446cddb",
+    "cuda_metadata_identity": "5eb223c4c969f2df64b29289ef3e96a467b418721ee7071faa81e64c59fbb1a1",
+    "cpu_input_state_sha256": "f0e8ca0d826e52e76a2b090084a09845a58871d3e6e02b8513b7a4f9cbf8d6c2",
+    "cuda_input_state_sha256": "f0e8ca0d826e52e76a2b090084a09845a58871d3e6e02b8513b7a4f9cbf8d6c2",
+    "cpu_device": {
+      "backend": "cpu",
+      "device_kind": "cpu",
+      "platform": "cpu",
+      "platform_version": "cpu",
+      "jax": "0.4.38",
+      "jaxlib": "0.4.38",
+      "python": "3.12.3",
+      "gpu_matmul_verified": false,
+      "numerics": {
+        "matmul_precision": "highest",
+        "NVIDIA_TF32_OVERRIDE": "0",
+        "jax_enable_x64": false,
+        "jax_threefry_partitionable": true
+      }
+    },
+    "device": {
+      "backend": "cuda",
+      "device_kind": "NVIDIA GeForce RTX 3070",
+      "platform": "gpu",
+      "platform_version": "PJRT C API\ncuda 12030",
+      "jax": "0.4.38",
+      "jaxlib": "0.4.38",
+      "python": "3.12.3",
+      "gpu_matmul_verified": true,
+      "numerics": {
+        "matmul_precision": "highest",
+        "NVIDIA_TF32_OVERRIDE": "0",
+        "jax_enable_x64": false,
+        "jax_threefry_partitionable": true
+      }
+    },
+    "cpu_predictions_reused_as_cuda": false,
+    "legacy_cross_runtime_comparison": {
+      "wp_pressure_mean_abs_difference_hpa": 0.0074994690769043465,
+      "wp_pressure_max_abs_difference_hpa": 0.59765625,
+      "wp_pressure_max_abs_difference_hpa_by_lead": [
+        0.03729248046875,
+        0.00933837890625,
+        0.0706787109375,
+        0.03485107421875,
+        0.0855712890625,
+        0.05712890625,
+        0.0518798828125,
+        0.04205322265625,
+        0.0977783203125,
+        0.07293701171875,
+        0.05584716796875,
+        0.0611572265625,
+        0.145751953125,
+        0.1763916015625,
+        0.19189453125,
+        0.26953125,
+        0.20556640625,
+        0.29156494140625,
+        0.5030517578125,
+        0.59765625
+      ],
+      "field_gate_limits_hpa": {
+        "mean": 0.01,
+        "max": 0.1
+      },
+      "field_gate_passed": false,
+      "tracker_max_difference_km": 35.802407171684315,
+      "central_pressure_max_abs_difference_hpa": 1.597311381356235,
+      "pressure_masks_match": true,
+      "cpu_numerical_equivalence": false,
+      "gating": false,
+      "reference_sha256": "982501139e39ebf6e6308c23d7abf3131c221f0ce489000400cafcaa65b06eb4",
+      "input_bytes_match": true,
+      "note": "The older Mac implementation/runtime differs. A failed legacy comparison remains failed; it is not numerical-equivalence certification."
+    }
+  },
+  "published_to_site": false,
+  "integration_note": "Import after independent review into the EXISTING site. The current frontend explicitly suppresses DeepMind until its verified import adapter is updated."
+}

evaluation/deepmind_daily/canary.json

@@ -0,0 +1,136 @@
+{
+  "wp_pressure_mean_abs_difference_hpa": 0.00025739270264116574,
+  "wp_pressure_max_abs_difference_hpa": 0.04547119140625,
+  "wp_pressure_max_abs_difference_hpa_by_lead": [
+    0.00030517578125,
+    0.000244140625,
+    0.000244140625,
+    0.00018310546875,
+    0.00030517578125,
+    0.00054931640625,
+    0.00079345703125,
+    0.0009765625,
+    0.002685546875,
+    0.00341796875,
+    0.0032958984375,
+    0.0048828125,
+    0.004638671875,
+    0.0032958984375,
+    0.00543212890625,
+    0.00787353515625,
+    0.00836181640625,
+    0.023681640625,
+    0.04547119140625,
+    0.042724609375
+  ],
+  "field_gate_limits_hpa": {
+    "mean": 0.01,
+    "max": 0.1
+  },
+  "field_gate_passed": true,
+  "tracker_max_difference_km": 0.14586496919052705,
+  "central_pressure_max_abs_difference_hpa": 0.07400935954797205,
+  "pressure_masks_match": true,
+  "cpu_numerical_equivalence": true,
+  "status": "cuda_canary_passed",
+  "tested_at_utc": "2026-10-04T12:55:06.985459+00:00",
+  "validation_protocol": {
+    "id": "same-runtime-strict-fp32-cpu-cuda-v2",
+    "primary_reference": "One fresh CPU rollout in the same pinned WSL runtime and inference code path",
+    "legacy_reference": "Bundled Mac CPU output retained as a separate cross-runtime diagnostic, not the primary gate",
+    "field_gate_limits_hpa": {
+      "mean": 0.01,
+      "max": 0.1
+    },
+    "jax": "0.4.38",
+    "jaxlib": "0.4.38",
+    "case_index": 0,
+    "cpu_counts_as_benchmark": false,
+    "tolerances_changed": false
+  },
+  "runtime_match": true,
+  "input_bytes_match": true,
+  "protocol_match": true,
+  "case_index": 0,
+  "checkpoint_sha256": "a1bb151457077248d70a458b1a7b19deacd2926fbcb46eee4ff2691444e3596e",
+  "gpu_verified_steps": 20,
+  "protocol_sha256": "91b31582f24ffdd4f2309eddb0cc68498a2c31cab7cbabaf06f43dfbff413606",
+  "cpu_forecast_sha256": "a8700f10b93dcf76b20439ed0cf71c312274398813af4ef8c6df51cbc75b0964",
+  "cuda_forecast_sha256": "0338d2b907bf0f4a605883b4c778bf961fe7406b6916161317b25a39b66fa68c",
+  "cpu_metadata_identity": "a2b686e98309ebcda11482f521b68cc8657171d08606d24e0c303484c446cddb",
+  "cuda_metadata_identity": "5eb223c4c969f2df64b29289ef3e96a467b418721ee7071faa81e64c59fbb1a1",
+  "cpu_input_state_sha256": "f0e8ca0d826e52e76a2b090084a09845a58871d3e6e02b8513b7a4f9cbf8d6c2",
+  "cuda_input_state_sha256": "f0e8ca0d826e52e76a2b090084a09845a58871d3e6e02b8513b7a4f9cbf8d6c2",
+  "cpu_device": {
+    "backend": "cpu",
+    "device_kind": "cpu",
+    "platform": "cpu",
+    "platform_version": "cpu",
+    "jax": "0.4.38",
+    "jaxlib": "0.4.38",
+    "python": "3.12.3",
+    "gpu_matmul_verified": false,
+    "numerics": {
+      "matmul_precision": "highest",
+      "NVIDIA_TF32_OVERRIDE": "0",
+      "jax_enable_x64": false,
+      "jax_threefry_partitionable": true
+    }
+  },
+  "device": {
+    "backend": "cuda",
+    "device_kind": "NVIDIA GeForce RTX 3070",
+    "platform": "gpu",
+    "platform_version": "PJRT C API\ncuda 12030",
+    "jax": "0.4.38",
+    "jaxlib": "0.4.38",
+    "python": "3.12.3",
+    "gpu_matmul_verified": true,
+    "numerics": {
+      "matmul_precision": "highest",
+      "NVIDIA_TF32_OVERRIDE": "0",
+      "jax_enable_x64": false,
+      "jax_threefry_partitionable": true
+    }
+  },
+  "cpu_predictions_reused_as_cuda": false,
+  "legacy_cross_runtime_comparison": {
+    "wp_pressure_mean_abs_difference_hpa": 0.0074994690769043465,
+    "wp_pressure_max_abs_difference_hpa": 0.59765625,
+    "wp_pressure_max_abs_difference_hpa_by_lead": [
+      0.03729248046875,
+      0.00933837890625,
+      0.0706787109375,
+      0.03485107421875,
+      0.0855712890625,
+      0.05712890625,
+      0.0518798828125,
+      0.04205322265625,
+      0.0977783203125,
+      0.07293701171875,
+      0.05584716796875,
+      0.0611572265625,
+      0.145751953125,
+      0.1763916015625,
+      0.19189453125,
+      0.26953125,
+      0.20556640625,
+      0.29156494140625,
+      0.5030517578125,
+      0.59765625
+    ],
+    "field_gate_limits_hpa": {
+      "mean": 0.01,
+      "max": 0.1
+    },
+    "field_gate_passed": false,
+    "tracker_max_difference_km": 35.802407171684315,
+    "central_pressure_max_abs_difference_hpa": 1.597311381356235,
+    "pressure_masks_match": true,
+    "cpu_numerical_equivalence": false,
+    "gating": false,
+    "reference_sha256": "982501139e39ebf6e6308c23d7abf3131c221f0ce489000400cafcaa65b06eb4",
+    "input_bytes_match": true,
+    "note": "The older Mac implementation/runtime differs. A failed legacy comparison remains failed; it is not numerical-equivalence certification."
+  }
+}

evaluation/deepmind_daily/publication_audit.json

@@ -0,0 +1,45 @@
+{
+  "status": "complete_verified",
+  "reviewed_at_utc": "2026-10-05T05:18:43.939189+00:00",
+  "archive_sha256": "602425d3ff9284cce44fee4d3a75f50354b6e3f26f391f5402f72afabd762d45",
+  "verified_daily_cases": 1473,
+  "verified_storms": 270,
+  "hashed_case_files": 2946,
+  "verified_physical_pressure_grids": 29460,
+  "model": "Google DeepMind WeatherNext Cyclones Mini (<2024)",
+  "software_version": "0.3.0",
+  "software_commit": "89c4b2a77a1c57b328b909c575550fd2e5aadc9c",
+  "checkpoint_sha256": "a1bb151457077248d70a458b1a7b19deacd2926fbcb46eee4ff2691444e3596e",
+  "cohort_sha256": "965184f1e5ab3fcb52e304f39b38583a14f753870aa2295f2d01942a441dcfd6",
+  "members": 1,
+  "backend": "NVIDIA GeForce RTX 3070 / CUDA",
+  "all_route_and_pressure_scores_recomputed": true,
+  "exact_common_curve_case_ids": true,
+  "all_original_forecasts_sha256_verified": true,
+  "causal_input_metadata_and_saved_hashes_verified": true,
+  "raw_era5_states_in_return_zip": false,
+  "raw_input_audit_scope": "Original worker receipt and saved input hashes/times; raw ERA5 states were not returned in this result-only ZIP.",
+  "new_inference_performed": false,
+  "model_weights_changed": false,
+  "forecasts_modified": false,
+  "missing_results_scored_as_zero": 0,
+  "previous_snapshot_sha256": "79f67a5f4490150cf0ab24d73e5d3cdedb4884d37e86560cbadde1d4ecdafc43",
+  "snapshot_sha256": "adff1baa1dbc38a6b443cfdd436cd7d4893c04241355c9eaf5004f439ce469a4",
+  "imported_files_sha256": {
+    "benchmark.json": "85a0965b7a87d400ccad67b437d05b2ac651055b8f4a473ff30fdd85ac778645",
+    "verification.json": "8f29a767ab5db61cf6773f5806cc110789329b04db48361715e5b4c295545c89",
+    "protocol.json": "91b31582f24ffdd4f2309eddb0cc68498a2c31cab7cbabaf06f43dfbff413606",
+    "case-manifest.json": "30d35cfb79e809227fd0d255f9b5173ad18e781c7450ed4bbf0211553f8e63bd",
+    "canary.json": "c7c9724752aab3f2208b53aad31949580800c0c60e28b187bccbd5d0a3c5e075"
+  },
+  "period_coverage": {
+    "historical_1980_1999": {
+      "daily_issues": 1336,
+      "storms": 230
+    },
+    "recent_2024_onward": {
+      "daily_issues": 137,
+      "storms": 40
+    }
+  }
+}

evaluation/deepmind_daily/verification.json

@@ -0,0 +1,1501 @@
+{
+  "status": "complete_verified",
+  "verified_at_utc": "2026-10-04T12:55:16.370140+00:00",
+  "verified_daily_cases": 1473,
+  "verified_storms": 270,
+  "backend": "NVIDIA CUDA",
+  "members": 1,
+  "checkpoint_sha256": "a1bb151457077248d70a458b1a7b19deacd2926fbcb46eee4ff2691444e3596e",
+  "cohort_sha256": "965184f1e5ab3fcb52e304f39b38583a14f753870aa2295f2d01942a441dcfd6",
+  "bundle_manifest_sha256": "c9d43105989cb38622bb7b7549008c75aff49f089410312df1192a1102765bd9",
+  "protocol_sha256": "91b31582f24ffdd4f2309eddb0cc68498a2c31cab7cbabaf06f43dfbff413606",
+  "case_indices": [
+    0,
+    1,
+    2,
+    3,
+    4,
+    5,
+    6,
+    7,
+    8,
+    9,
+    10,
+    11,
+    12,
+    13,
+    14,
+    15,
+    16,
+    17,
+    18,
+    19,
+    20,
+    21,
+    22,
+    23,
+    24,
+    25,
+    26,
+    27,
+    28,
+    29,
+    30,
+    31,
+    32,
+    33,
+    34,
+    35,
+    36,
+    37,
+    38,
+    39,
+    40,
+    41,
+    42,
+    43,
+    44,
+    45,
+    46,
+    47,
+    48,
+    49,
+    50,
+    51,
+    52,
+    53,
+    54,
+    55,
+    56,
+    57,
+    58,
+    59,
+    60,
+    61,
+    62,
+    63,
+    64,
+    65,
+    66,
+    67,
+    68,
+    69,
+    70,
+    71,
+    72,
+    73,
+    74,
+    75,
+    76,
+    77,
+    78,
+    79,
+    80,
+    81,
+    82,
+    83,
+    84,
+    85,
+    86,
+    87,
+    88,
+    89,
+    90,
+    91,
+    92,
+    93,
+    94,
+    95,
+    96,
+    97,
+    98,
+    99,
+    100,
+    101,
+    102,
+    103,
+    104,
+    105,
+    106,
+    107,
+    108,
+    109,
+    110,
+    111,
+    112,
+    113,
+    114,
+    115,
+    116,
+    117,
+    118,
+    119,
+    120,
+    121,
+    122,
+    123,
+    124,
+    125,
+    126,
+    127,
+    128,
+    129,
+    130,
+    131,
+    132,
+    133,
+    134,
+    135,
+    136,
+    137,
+    138,
+    139,
+    140,
+    141,
+    142,
+    143,
+    144,
+    145,
+    146,
+    147,
+    148,
+    149,
+    150,
+    151,
+    152,
+    153,
+    154,
+    155,
+    156,
+    157,
+    158,
+    159,
+    160,
+    161,
+    162,
+    163,
+    164,
+    165,
+    166,
+    167,
+    168,
+    169,
+    170,
+    171,
+    172,
+    173,
+    174,
+    175,
+    176,
+    177,
+    178,
+    179,
+    180,
+    181,
+    182,
+    183,
+    184,
+    185,
+    186,
+    187,
+    188,
+    189,
+    190,
+    191,
+    192,
+    193,
+    194,
+    195,
+    196,
+    197,
+    198,
+    199,
+    200,
+    201,
+    202,
+    203,
+    204,
+    205,
+    206,
+    207,
+    208,
+    209,
+    210,
+    211,
+    212,
+    213,
+    214,
+    215,
+    216,
+    217,
+    218,
+    219,
+    220,
+    221,
+    222,
+    223,
+    224,
+    225,
+    226,
+    227,
+    228,
+    229,
+    230,
+    231,
+    232,
+    233,
+    234,
+    235,
+    236,
+    237,
+    238,
+    239,
+    240,
+    241,
+    242,
+    243,
+    244,
+    245,
+    246,
+    247,
+    248,
+    249,
+    250,
+    251,
+    252,
+    253,
+    254,
+    255,
+    256,
+    257,
+    258,
+    259,
+    260,
+    261,
+    262,
+    263,
+    264,
+    265,
+    266,
+    267,
+    268,
+    269,
+    270,
+    271,
+    272,
+    273,
+    274,
+    275,
+    276,
+    277,
+    278,
+    279,
+    280,
+    281,
+    282,
+    283,
+    284,
+    285,
+    286,
+    287,
+    288,
+    289,
+    290,
+    291,
+    292,
+    293,
+    294,
+    295,
+    296,
+    297,
+    298,
+    299,
+    300,
+    301,
+    302,
+    303,
+    304,
+    305,
+    306,
+    307,
+    308,
+    309,
+    310,
+    311,
+    312,
+    313,
+    314,
+    315,
+    316,
+    317,
+    318,
+    319,
+    320,
+    321,
+    322,
+    323,
+    324,
+    325,
+    326,
+    327,
+    328,
+    329,
+    330,
+    331,
+    332,
+    333,
+    334,
+    335,
+    336,
+    337,
+    338,
+    339,
+    340,
+    341,
+    342,
+    343,
+    344,
+    345,
+    346,
+    347,
+    348,
+    349,
+    350,
+    351,
+    352,
+    353,
+    354,
+    355,
+    356,
+    357,
+    358,
+    359,
+    360,
+    361,
+    362,
+    363,
+    364,
+    365,
+    366,
+    367,
+    368,
+    369,
+    370,
+    371,
+    372,
+    373,
+    374,
+    375,
+    376,
+    377,
+    378,
+    379,
+    380,
+    381,
+    382,
+    383,
+    384,
+    385,
+    386,
+    387,
+    388,
+    389,
+    390,
+    391,
+    392,
+    393,
+    394,
+    395,
+    396,
+    397,
+    398,
+    399,
+    400,
+    401,
+    402,
+    403,
+    404,
+    405,
+    406,
+    407,
+    408,
+    409,
+    410,
+    411,
+    412,
+    413,
+    414,
+    415,
+    416,
+    417,
+    418,
+    419,
+    420,
+    421,
+    422,
+    423,
+    424,
+    425,
+    426,
+    427,
+    428,
+    429,
+    430,
+    431,
+    432,
+    433,
+    434,
+    435,
+    436,
+    437,
+    438,
+    439,
+    440,
+    441,
+    442,
+    443,
+    444,
+    445,
+    446,
+    447,
+    448,
+    449,
+    450,
+    451,
+    452,
+    453,
+    454,
+    455,
+    456,
+    457,
+    458,
+    459,
+    460,
+    461,
+    462,
+    463,
+    464,
+    465,
+    466,
+    467,
+    468,
+    469,
+    470,
+    471,
+    472,
+    473,
+    474,
+    475,
+    476,
+    477,
+    478,
+    479,
+    480,
+    481,
+    482,
+    483,
+    484,
+    485,
+    486,
+    487,
+    488,
+    489,
+    490,
+    491,
+    492,
+    493,
+    494,
+    495,
+    496,
+    497,
+    498,
+    499,
+    500,
+    501,
+    502,
+    503,
+    504,
+    505,
+    506,
+    507,
+    508,
+    509,
+    510,
+    511,
+    512,
+    513,
+    514,
+    515,
+    516,
+    517,
+    518,
+    519,
+    520,
+    521,
+    522,
+    523,
+    524,
+    525,
+    526,
+    527,
+    528,
+    529,
+    530,
+    531,
+    532,
+    533,
+    534,
+    535,
+    536,
+    537,
+    538,
+    539,
+    540,
+    541,
+    542,
+    543,
+    544,
+    545,
+    546,
+    547,
+    548,
+    549,
+    550,
+    551,
+    552,
+    553,
+    554,
+    555,
+    556,
+    557,
+    558,
+    559,
+    560,
+    561,
+    562,
+    563,
+    564,
+    565,
+    566,
+    567,
+    568,
+    569,
+    570,
+    571,
+    572,
+    573,
+    574,
+    575,
+    576,
+    577,
+    578,
+    579,
+    580,
+    581,
+    582,
+    583,
+    584,
+    585,
+    586,
+    587,
+    588,
+    589,
+    590,
+    591,
+    592,
+    593,
+    594,
+    595,
+    596,
+    597,
+    598,
+    599,
+    600,
+    601,
+    602,
+    603,
+    604,
+    605,
+    606,
+    607,
+    608,
+    609,
+    610,
+    611,
+    612,
+    613,
+    614,
+    615,
+    616,
+    617,
+    618,
+    619,
+    620,
+    621,
+    622,
+    623,
+    624,
+    625,
+    626,
+    627,
+    628,
+    629,
+    630,
+    631,
+    632,
+    633,
+    634,
+    635,
+    636,
+    637,
+    638,
+    639,
+    640,
+    641,
+    642,
+    643,
+    644,
+    645,
+    646,
+    647,
+    648,
+    649,
+    650,
+    651,
+    652,
+    653,
+    654,
+    655,
+    656,
+    657,
+    658,
+    659,
+    660,
+    661,
+    662,
+    663,
+    664,
+    665,
+    666,
+    667,
+    668,
+    669,
+    670,
+    671,
+    672,
+    673,
+    674,
+    675,
+    676,
+    677,
+    678,
+    679,
+    680,
+    681,
+    682,
+    683,
+    684,
+    685,
+    686,
+    687,
+    688,
+    689,
+    690,
+    691,
+    692,
+    693,
+    694,
+    695,
+    696,
+    697,
+    698,
+    699,
+    700,
+    701,
+    702,
+    703,
+    704,
+    705,
+    706,
+    707,
+    708,
+    709,
+    710,
+    711,
+    712,
+    713,
+    714,
+    715,
+    716,
+    717,
+    718,
+    719,
+    720,
+    721,
+    722,
+    723,
+    724,
+    725,
+    726,
+    727,
+    728,
+    729,
+    730,
+    731,
+    732,
+    733,
+    734,
+    735,
+    736,
+    737,
+    738,
+    739,
+    740,
+    741,
+    742,
+    743,
+    744,
+    745,
+    746,
+    747,
+    748,
+    749,
+    750,
+    751,
+    752,
+    753,
+    754,
+    755,
+    756,
+    757,
+    758,
+    759,
+    760,
+    761,
+    762,
+    763,
+    764,
+    765,
+    766,
+    767,
+    768,
+    769,
+    770,
+    771,
+    772,
+    773,
+    774,
+    775,
+    776,
+    777,
+    778,
+    779,
+    780,
+    781,
+    782,
+    783,
+    784,
+    785,
+    786,
+    787,
+    788,
+    789,
+    790,
+    791,
+    792,
+    793,
+    794,
+    795,
+    796,
+    797,
+    798,
+    799,
+    800,
+    801,
+    802,
+    803,
+    804,
+    805,
+    806,
+    807,
+    808,
+    809,
+    810,
+    811,
+    812,
+    813,
+    814,
+    815,
+    816,
+    817,
+    818,
+    819,
+    820,
+    821,
+    822,
+    823,
+    824,
+    825,
+    826,
+    827,
+    828,
+    829,
+    830,
+    831,
+    832,
+    833,
+    834,
+    835,
+    836,
+    837,
+    838,
+    839,
+    840,
+    841,
+    842,
+    843,
+    844,
+    845,
+    846,
+    847,
+    848,
+    849,
+    850,
+    851,
+    852,
+    853,
+    854,
+    855,
+    856,
+    857,
+    858,
+    859,
+    860,
+    861,
+    862,
+    863,
+    864,
+    865,
+    866,
+    867,
+    868,
+    869,
+    870,
+    871,
+    872,
+    873,
+    874,
+    875,
+    876,
+    877,
+    878,
+    879,
+    880,
+    881,
+    882,
+    883,
+    884,
+    885,
+    886,
+    887,
+    888,
+    889,
+    890,
+    891,
+    892,
+    893,
+    894,
+    895,
+    896,
+    897,
+    898,
+    899,
+    900,
+    901,
+    902,
+    903,
+    904,
+    905,
+    906,
+    907,
+    908,
+    909,
+    910,
+    911,
+    912,
+    913,
+    914,
+    915,
+    916,
+    917,
+    918,
+    919,
+    920,
+    921,
+    922,
+    923,
+    924,
+    925,
+    926,
+    927,
+    928,
+    929,
+    930,
+    931,
+    932,
+    933,
+    934,
+    935,
+    936,
+    937,
+    938,
+    939,
+    940,
+    941,
+    942,
+    943,
+    944,
+    945,
+    946,
+    947,
+    948,
+    949,
+    950,
+    951,
+    952,
+    953,
+    954,
+    955,
+    956,
+    957,
+    958,
+    959,
+    960,
+    961,
+    962,
+    963,
+    964,
+    965,
+    966,
+    967,
+    968,
+    969,
+    970,
+    971,
+    972,
+    973,
+    974,
+    975,
+    976,
+    977,
+    978,
+    979,
+    980,
+    981,
+    982,
+    983,
+    984,
+    985,
+    986,
+    987,
+    988,
+    989,
+    990,
+    991,
+    992,
+    993,
+    994,
+    995,
+    996,
+    997,
+    998,
+    999,
+    1000,
+    1001,
+    1002,
+    1003,
+    1004,
+    1005,
+    1006,
+    1007,
+    1008,
+    1009,
+    1010,
+    1011,
+    1012,
+    1013,
+    1014,
+    1015,
+    1016,
+    1017,
+    1018,
+    1019,
+    1020,
+    1021,
+    1022,
+    1023,
+    1024,
+    1025,
+    1026,
+    1027,
+    1028,
+    1029,
+    1030,
+    1031,
+    1032,
+    1033,
+    1034,
+    1035,
+    1036,
+    1037,
+    1038,
+    1039,
+    1040,
+    1041,
+    1042,
+    1043,
+    1044,
+    1045,
+    1046,
+    1047,
+    1048,
+    1049,
+    1050,
+    1051,
+    1052,
+    1053,
+    1054,
+    1055,
+    1056,
+    1057,
+    1058,
+    1059,
+    1060,
+    1061,
+    1062,
+    1063,
+    1064,
+    1065,
+    1066,
+    1067,
+    1068,
+    1069,
+    1070,
+    1071,
+    1072,
+    1073,
+    1074,
+    1075,
+    1076,
+    1077,
+    1078,
+    1079,
+    1080,
+    1081,
+    1082,
+    1083,
+    1084,
+    1085,
+    1086,
+    1087,
+    1088,
+    1089,
+    1090,
+    1091,
+    1092,
+    1093,
+    1094,
+    1095,
+    1096,
+    1097,
+    1098,
+    1099,
+    1100,
+    1101,
+    1102,
+    1103,
+    1104,
+    1105,
+    1106,
+    1107,
+    1108,
+    1109,
+    1110,
+    1111,
+    1112,
+    1113,
+    1114,
+    1115,
+    1116,
+    1117,
+    1118,
+    1119,
+    1120,
+    1121,
+    1122,
+    1123,
+    1124,
+    1125,
+    1126,
+    1127,
+    1128,
+    1129,
+    1130,
+    1131,
+    1132,
+    1133,
+    1134,
+    1135,
+    1136,
+    1137,
+    1138,
+    1139,
+    1140,
+    1141,
+    1142,
+    1143,
+    1144,
+    1145,
+    1146,
+    1147,
+    1148,
+    1149,
+    1150,
+    1151,
+    1152,
+    1153,
+    1154,
+    1155,
+    1156,
+    1157,
+    1158,
+    1159,
+    1160,
+    1161,
+    1162,
+    1163,
+    1164,
+    1165,
+    1166,
+    1167,
+    1168,
+    1169,
+    1170,
+    1171,
+    1172,
+    1173,
+    1174,
+    1175,
+    1176,
+    1177,
+    1178,
+    1179,
+    1180,
+    1181,
+    1182,
+    1183,
+    1184,
+    1185,
+    1186,
+    1187,
+    1188,
+    1189,
+    1190,
+    1191,
+    1192,
+    1193,
+    1194,
+    1195,
+    1196,
+    1197,
+    1198,
+    1199,
+    1200,
+    1201,
+    1202,
+    1203,
+    1204,
+    1205,
+    1206,
+    1207,
+    1208,
+    1209,
+    1210,
+    1211,
+    1212,
+    1213,
+    1214,
+    1215,
+    1216,
+    1217,
+    1218,
+    1219,
+    1220,
+    1221,
+    1222,
+    1223,
+    1224,
+    1225,
+    1226,
+    1227,
+    1228,
+    1229,
+    1230,
+    1231,
+    1232,
+    1233,
+    1234,
+    1235,
+    1236,
+    1237,
+    1238,
+    1239,
+    1240,
+    1241,
+    1242,
+    1243,
+    1244,
+    1245,
+    1246,
+    1247,
+    1248,
+    1249,
+    1250,
+    1251,
+    1252,
+    1253,
+    1254,
+    1255,
+    1256,
+    1257,
+    1258,
+    1259,
+    1260,
+    1261,
+    1262,
+    1263,
+    1264,
+    1265,
+    1266,
+    1267,
+    1268,
+    1269,
+    1270,
+    1271,
+    1272,
+    1273,
+    1274,
+    1275,
+    1276,
+    1277,
+    1278,
+    1279,
+    1280,
+    1281,
+    1282,
+    1283,
+    1284,
+    1285,
+    1286,
+    1287,
+    1288,
+    1289,
+    1290,
+    1291,
+    1292,
+    1293,
+    1294,
+    1295,
+    1296,
+    1297,
+    1298,
+    1299,
+    1300,
+    1301,
+    1302,
+    1303,
+    1304,
+    1305,
+    1306,
+    1307,
+    1308,
+    1309,
+    1310,
+    1311,
+    1312,
+    1313,
+    1314,
+    1315,
+    1316,
+    1317,
+    1318,
+    1319,
+    1320,
+    1321,
+    1322,
+    1323,
+    1324,
+    1325,
+    1326,
+    1327,
+    1328,
+    1329,
+    1330,
+    1331,
+    1332,
+    1333,
+    1334,
+    1335,
+    1336,
+    1337,
+    1338,
+    1339,
+    1340,
+    1341,
+    1342,
+    1343,
+    1344,
+    1345,
+    1346,
+    1347,
+    1348,
+    1349,
+    1350,
+    1351,
+    1352,
+    1353,
+    1354,
+    1355,
+    1356,
+    1357,
+    1358,
+    1359,
+    1360,
+    1361,
+    1362,
+    1363,
+    1364,
+    1365,
+    1366,
+    1367,
+    1368,
+    1369,
+    1370,
+    1371,
+    1372,
+    1373,
+    1374,
+    1375,
+    1376,
+    1377,
+    1378,
+    1379,
+    1380,
+    1381,
+    1382,
+    1383,
+    1384,
+    1385,
+    1386,
+    1387,
+    1388,
+    1389,
+    1390,
+    1391,
+    1392,
+    1393,
+    1394,
+    1395,
+    1396,
+    1397,
+    1398,
+    1399,
+    1400,
+    1401,
+    1402,
+    1403,
+    1404,
+    1405,
+    1406,
+    1407,
+    1408,
+    1409,
+    1410,
+    1411,
+    1412,
+    1413,
+    1414,
+    1415,
+    1416,
+    1417,
+    1418,
+    1419,
+    1420,
+    1421,
+    1422,
+    1423,
+    1424,
+    1425,
+    1426,
+    1427,
+    1428,
+    1429,
+    1430,
+    1431,
+    1432,
+    1433,
+    1434,
+    1435,
+    1436,
+    1437,
+    1438,
+    1439,
+    1440,
+    1441,
+    1442,
+    1443,
+    1444,
+    1445,
+    1446,
+    1447,
+    1448,
+    1449,
+    1450,
+    1451,
+    1452,
+    1453,
+    1454,
+    1455,
+    1456,
+    1457,
+    1458,
+    1459,
+    1460,
+    1461,
+    1462,
+    1463,
+    1464,
+    1465,
+    1466,
+    1467,
+    1468,
+    1469,
+    1470,
+    1471,
+    1472
+  ],
+  "cpu_predictions_reused_as_cuda": false,
+  "cpu_numerical_equivalence": true,
+  "case_manifest_sha256": "30d35cfb79e809227fd0d255f9b5173ad18e781c7450ed4bbf0211553f8e63bd",
+  "canary_sha256": "c7c9724752aab3f2208b53aad31949580800c0c60e28b187bccbd5d0a3c5e075",
+  "checks": [
+    "all frozen IDs",
+    "original references and official weights unchanged",
+    "CUDA on every step",
+    "exact two causal analyses and retained input SHA256",
+    "20 exact +6..+120h leads",
+    "unshifted finite routes",
+    "20 physical native WP pressure fields",
+    "recomputed original three-way masks and metrics"
+  ]
+}

evaluation/released_daily/model_1_2_benchmark.json

@@ -1,8 +1,8 @@
 {
   "source": "evaluation/released_daily/released_daily_benchmark.json",
-  "source_sha256": "79f67a5f4490150cf0ab24d73e5d3cdedb4884d37e86560cbadde1d4ecdafc43",
+  "source_sha256": "adff1baa1dbc38a6b443cfdd436cd7d4893c04241355c9eaf5004f439ce469a4",
   "chart": "evaluation/released_daily/model_1_2_benchmark.png",
-  "chart_sha256": "3bd866eede33fc93d36a0a5373a3b1637bf0f56860938c3d5691b9b7dfe770b4",
+  "chart_sha256": "a1aec843491af2c81c587410de9d9a478a7aae75c4b79ee4b95d2fd18ef9b2de",
   "metrics": [
     "pressure_JMA_hpa",
     "mean_track_error_km",
@@ -11,18 +11,43 @@
   "values": {
     "pressure_JMA_hpa": {
       "1.1": 13.53233861138585,
-      "1.2": 12.837077366789085
+      "1.2": 12.837077366789085,
+      "deepmind": 10.197678417354012
     },
     "mean_track_error_km": {
       "1.1": 798.4143395733709,
-      "1.2": 471.2045276140213
+      "1.2": 471.2045276140213,
+      "deepmind": 498.3660837262175
     },
     "direction_error_deg": {
       "1.1": 51.58419207122625,
-      "1.2": 34.95994745459279
+      "1.2": 34.95994745459279,
+      "deepmind": 39.76380497312257
     }
   },
-  "deepmind_score": null,
+  "deepmind_status": "complete_verified",
+  "deepmind_model": {
+    "key": "deepmind",
+    "label": "Google DeepMind WeatherNext Cyclones Mini (<2024)",
+    "model_checkpoint": "WeatherNextCyclones_Mini_<2024",
+    "software_version": "0.3.0",
+    "software_commit": "89c4b2a77a1c57b328b909c575550fd2e5aadc9c",
+    "trained_through": 2023,
+    "resolution_degrees": 1,
+    "member_count": 1,
+    "member_policy": "One official Mini member, seed 0",
+    "checkpoint_sha256": "a1bb151457077248d70a458b1a7b19deacd2926fbcb46eee4ff2691444e3596e",
+    "method": "Official predictor and initialized-storm tracker; causal ERA5; RTX 3070 CUDA",
+    "status": "complete_verified"
+  },
+  "models": [
+    "1.1",
+    "1.2",
+    "deepmind"
+  ],
+  "bars_per_panel": 3,
+  "error_bars": false,
   "zero_fill": false,
-  "model_forecasts_modified": false
+  "model_forecasts_modified": false,
+  "publication_audit_sha256": "52c767b46452a29341c024f24a774328ed6fd4169f5f28e923a81e01c641b222"
 }

release_tools/import_deepmind_release_results.py

@@ -0,0 +1,276 @@
+"""Audit the returned CUDA ZIP and merge documentation data, without inference.
+
+Read immutable local reference forecasts and labels; never execute a model,
+modify a forecast, publish a Site, or extract paths supplied by the archive.
+"""
+from __future__ import annotations
+
+import argparse
+from collections import defaultdict
+import copy
+from datetime import datetime, timezone
+import hashlib
+import io
+import json
+from pathlib import Path, PurePosixPath
+import re
+import zipfile
+
+import numpy as np
+
+from deepmind_daily_benchmark import (
+    COHORT, WEIGHTS, CHECKPOINT, STEP_NS, aggregate, curve_similarity,
+    local, pressure_mask, route_metrics,
+)
+
+ROOT = Path(__file__).resolve().parents[1]
+CODE_COMMIT = '89c4b2a77a1c57b328b909c575550fd2e5aadc9c'
+MODEL_LABEL = 'Google DeepMind WeatherNext Cyclones Mini (<2024)'
+LEADS = np.arange(6, 121, 6)
+
+
+def require(condition, message):
+    if not condition:
+        raise ValueError(message)
+
+
+def digest(content):
+    return hashlib.sha256(content).hexdigest()
+
+
+def file_sha(path):
+    with Path(path).open('rb') as stream:
+        return hashlib.file_digest(stream, 'sha256').hexdigest()
+
+
+def same(actual, expected, context):
+    """Compare nested scores without treating missing values as zero."""
+    if isinstance(expected, dict):
+        require(isinstance(actual, dict) and actual.keys() == expected.keys(), context)
+        for key in expected:
+            same(actual[key], expected[key], context + '/' + key)
+    elif isinstance(expected, list):
+        require(isinstance(actual, list) and len(actual) == len(expected), context)
+        for index, (a, b) in enumerate(zip(actual, expected)):
+            same(a, b, context + '/' + str(index))
+    elif expected is None or isinstance(expected, (str, bool)):
+        require(actual == expected, context)
+    else:
+        require(actual is not None and np.isfinite(actual)
+                and np.isclose(actual, expected, rtol=0, atol=1e-8), context)
+
+
+def validate_identity(protocol, receipt, exported):
+    require(receipt['status'] == exported['status'] == 'complete_verified', 'Incomplete run')
+    require(receipt['verified_daily_cases'] == 1473 and receipt['verified_storms'] == 270,
+            'Incomplete coverage')
+    require(receipt['case_indices'] == list(range(1473)), 'Missing or duplicate case IDs')
+    require([c['case_index'] for c in protocol['cases']] == list(range(1473)), 'Wrong plan IDs')
+    require(protocol['target_daily_cases'] == 1473 and protocol['target_storms'] == 270,
+            'Wrong target scope')
+    require(protocol['leads_hours'] == LEADS.tolist(), 'Wrong lead schedule')
+    require(protocol['vendor_commit'] == CODE_COMMIT, 'Wrong official software commit')
+    require(protocol['resolution_degrees'] == 1 and protocol['trained_through'] == 2023,
+            'Wrong Mini checkpoint generation')
+    require(protocol['reference_v12_checkpoint_sha256'] == CHECKPOINT,
+            'Wrong Trackformer reference')
+    require(protocol['members'] == receipt['members'] == 1, 'Wrong member count')
+    require(protocol['checkpoint_sha256'] == receipt['checkpoint_sha256'] == WEIGHTS,
+            'Wrong official checkpoint')
+    require(protocol['cohort_sha256'] == receipt['cohort_sha256']
+            == exported['cohort']['sha256'] == COHORT, 'Wrong daily cohort')
+    require(not receipt['cpu_predictions_reused_as_cuda'], 'CPU results relabelled as CUDA')
+    require(receipt['cpu_numerical_equivalence'], 'Same-runtime numerical gate did not pass')
+
+
+def merge_snapshot(base, exported, receipt, pressures):
+    """Preserve every old 1.1/1.2 score and unsupported metric explicitly."""
+    result = copy.deepcopy(base)
+    require(all(base['cohort'][k] == v for k, v in exported['cohort'].items()),
+            'Published baseline cohort differs')
+    rows = {row['key']: row for row in result['metrics']}
+    for row in exported['metrics']:
+        target = rows[row['key']]
+        for model in ('1.1', '1.2'):
+            same(target['values'][model], row['values'][model], 'Original score changed')
+        target['values']['deepmind'] = row['values']['deepmind']
+        if 'common_lead_points' in row:
+            target['coverage']['three_way_valid_issue_leads'] = row['common_lead_points']
+    rows['track_error_120h_km']['values']['deepmind'] = (
+        exported['aggregate_equal_complete_storm']['route']['deepmind']['track_error_120h_km']['value'])
+    model = next(m for m in result['models'] if m['key'] == 'deepmind')
+    model.clear()
+    model.update(key='deepmind', label=MODEL_LABEL, model_checkpoint='WeatherNextCyclones_Mini_<2024',
+                 software_version='0.3.0', software_commit=CODE_COMMIT,
+                 trained_through=2023, resolution_degrees=1, member_count=1,
+                 member_policy='One official Mini member, seed 0', checkpoint_sha256=WEIGHTS,
+                 method='Official predictor and initialized-storm tracker; causal ERA5; RTX 3070 CUDA',
+                 status='complete_verified')
+    result['per_lead_track']['deepmind'] = (
+        exported['aggregate_equal_complete_storm']['route']['deepmind']['track_error_by_lead_km'])
+    for row in result['pressure_curves']:
+        row['forecasts']['deepmind'] = pressures[row['case_index']]
+    result.update(deepmind_verification=receipt, deepmind_periods=exported['periods'],
+                  deepmind_interpretation=exported['interpretation'])
+    result['notes'].append('WeatherNext Cyclones Mini <2024, software v0.3.0: one member; '
+                           'not full-sized WeatherNext. Historic fitting-year overlap; '
+                           'recent 2024+ results are separate. DeepMind path/Frechet scores '
+                           'and confidence intervals are not supplied by this export.')
+    return result
+
+
+def audit(archive, project, destination):
+    destination = destination.resolve()
+    require(destination.is_relative_to(Path('/Volumes/D')), 'Diagnostics must stay on D')
+    snapshot_path = ROOT / 'evaluation/released_daily/released_daily_benchmark.json'
+    base_bytes = snapshot_path.read_bytes()
+    base = json.loads(base_bytes)
+    groups = defaultdict(list)
+    pressures = {}
+    shapes = {agency: {m: set() for m in ('1.1', '1.2', 'deepmind')} for agency in ('JMA', 'USA')}
+    with zipfile.ZipFile(archive) as bundle:
+        require(len(bundle.namelist()) == len(set(bundle.namelist())), 'Duplicate ZIP entries')
+        require(bundle.testzip() is None, 'ZIP CRC failure')
+
+        def read(name):
+            require(not PurePosixPath(name).is_absolute() and '..' not in PurePosixPath(name).parts,
+                    'Unsafe ZIP path')
+            return bundle.read('results/' + name)
+
+        def load(name):
+            return json.loads(read(name))
+
+        receipt = load('verification.json')
+        exported = load('site_export/benchmark.json')
+        protocol = load('protocol.json')
+        validate_identity(protocol, receipt, exported)
+        require(digest(read('verification.json')) == exported['verification_sha256'], 'Receipt hash')
+        require(digest(read('protocol.json')) == receipt['protocol_sha256'], 'Protocol hash')
+        require(digest(read('case-manifest.json')) == receipt['case_manifest_sha256'], 'Case manifest hash')
+        canary = load('canary.json')
+        require(digest(read('canary.json')) == receipt['canary_sha256'], 'Canary hash')
+        require(canary['field_gate_passed'] and canary['runtime_match']
+                and canary['input_bytes_match'] and canary['protocol_match'], 'Canary did not pass')
+        require(canary['device']['device_kind'] == 'NVIDIA GeForce RTX 3070', 'Wrong GPU')
+        require(not canary['cpu_predictions_reused_as_cuda'], 'Wrong CPU/CUDA provenance')
+        case_manifest = load('case-manifest.json')
+        require({f['path'] for f in case_manifest['files']} == {
+            f'cases/{i:05d}.{ext}' for i in range(1473) for ext in ('npz', 'json')},
+            'Case manifest is not the exact full archive')
+        for manifest in (case_manifest, load('site_export/manifest.json')):
+            for item in manifest['files']:
+                content = read(item['path'])
+                require(len(content) == item['bytes'] and digest(content) == item['sha256'],
+                        'Manifest mismatch: ' + item['path'])
+        for case in protocol['cases']:
+            index = case['case_index']
+            stem = f'{index:05d}'
+            record = load('cases/' + stem + '.json')
+            require(all(record.get(k) == v for k, v in case.items()), 'Case identity: ' + stem)
+            require(record['status'] == 'complete' and record['members'] == 1
+                    and record['backend'] == 'cuda' and record['gpu_verified_steps'] == 20
+                    and record['verified_device_steps'] == 20, 'CUDA/coverage: ' + stem)
+            require(record['device']['gpu_matmul_verified']
+                    and record['device']['device_kind'] == 'NVIDIA GeForce RTX 3070', 'GPU: ' + stem)
+            require(record['checkpoint_sha256'] == WEIGHTS
+                    and record['protocol_sha256'] == receipt['protocol_sha256'], 'Model: ' + stem)
+            require(record['input_time_ns'] == [case['issue_ns'] - STEP_NS, case['issue_ns']]
+                    and not record['future_weather_used'] and not record['future_storm_labels_used'],
+                    'Input causality metadata: ' + stem)
+            require(re.fullmatch('[0-9a-f]{64}', record['input_state_sha256']) is not None,
+                    'Missing saved input hash: ' + stem)
+            content = read('cases/' + stem + '.npz')
+            require(digest(content) == record['forecast_sha256'], 'Forecast hash: ' + stem)
+            reference_content = (project / 'benchmark_daily_storms_1_2/cases' / (stem + '.npz')).read_bytes()
+            require(digest(reference_content) == case['original_forecast_sha256'], 'Baseline changed: ' + stem)
+            with np.load(io.BytesIO(content), allow_pickle=False) as fields, np.load(
+                    io.BytesIO(reference_content), allow_pickle=False) as reference:
+                require(np.array_equal(fields['lead_hours'], LEADS) and np.array_equal(
+                    fields['valid_time_ns'], case['issue_ns'] + LEADS * 3600 * 10**9), 'Lead times: ' + stem)
+                route = fields['route_lat_lon']
+                field = fields['wp_pressure_hpa']
+                require(route.shape == (20, 2) and np.isfinite(route).all(), 'Route missing: ' + stem)
+                require(field.shape == (20, 61, 81) and np.isfinite(field).all()
+                        and field.min() >= 800 and field.max() <= 1100, 'Physical fields: ' + stem)
+                require(np.array_equal(np.sort(fields['latitude']), np.arange(61))
+                        and np.array_equal(np.sort(fields['longitude']), np.arange(100, 181)),
+                        'Native one-degree grid: ' + stem)
+                p = fields['central_pressure_hpa'].copy()
+                valid = fields['pressure_valid'].astype(bool)
+                require(p.shape == valid.shape == (20,), 'Pressure schedule: ' + stem)
+                require(np.all(~valid | (np.isfinite(p) & (p >= 800) & (p <= 1100))),
+                        'Invalid pressure marked available: ' + stem)
+                require(np.all(valid | ~np.isfinite(p)), 'Unavailable pressure not masked: ' + stem)
+                routes = route_metrics({'1.1': reference['v11_local'], '1.2': reference['v12_local'],
+                                        'deepmind': local(route, case['base_lat'], case['base_lon'])},
+                                       reference['truth_local'])
+            same(routes, record['route_metrics'], 'Route scoring: ' + stem)
+            source = json.loads((project / 'output/intensity-v12-v11-20261001/cases' / (stem + '.json')).read_bytes())
+            pressure_scores = {}
+            if source['v11'] is not None:
+                curves = {'1.1': np.array([r['central_pressure_hpa'] for r in source['v11']], dtype=float),
+                          '1.2': np.array(source['v12_pressure_hpa'], dtype=float), 'deepmind': p}
+                for agency in ('JMA', 'USA'):
+                    truth = np.array(source['truth']['pressure_' + agency + '_hpa'], dtype=float)
+                    mask = pressure_mask(truth, curves)
+                    pressure_scores[agency] = {'valid_leads': int(mask.sum()), 'models': {}}
+                    for model, prediction in curves.items():
+                        similarity = curve_similarity(prediction[mask], truth[mask])
+                        if similarity is not None:
+                            shapes[agency][model].add(index)
+                        pressure_scores[agency]['models'][model] = {
+                            'mae_hpa': float(np.abs(prediction[mask] - truth[mask]).mean()) if mask.any() else None,
+                            'curve_similarity': similarity,
+                            'error_by_lead_hpa': [float(abs(prediction[i] - truth[i])) if mask[i] else None for i in range(20)]}
+            same(pressure_scores, record['pressure_metrics'], 'Pressure scoring: ' + stem)
+            pressures[index] = [float(value) if ok else None for value, ok in zip(p, valid)]
+            groups[case['storm_id']].append(dict(record, route_metrics=routes, pressure_metrics=pressure_scores))
+        require(len(groups) == 270 and sum(map(len, groups.values())) == 1473, 'Incomplete storms')
+        for agency, models in shapes.items():
+            require(models['1.1'] == models['1.2'] == models['deepmind'],
+                    'Pressure shape scores do not use the same cases: ' + agency)
+        same(aggregate(groups), exported['aggregate_equal_complete_storm'], 'Equal-storm aggregate')
+        for period, expected in exported['periods'].items():
+            same(aggregate({sid: days for sid, days in groups.items() if days[0]['period'] == period}),
+                 expected, 'Period scoring: ' + period)
+        result = merge_snapshot(base, exported, receipt, pressures)
+        destination.mkdir(parents=True, exist_ok=True)
+        saved = {}
+        for target, original in (
+                ('benchmark.json', 'site_export/benchmark.json'), ('verification.json', 'verification.json'),
+                ('protocol.json', 'protocol.json'), ('case-manifest.json', 'case-manifest.json'),
+                ('canary.json', 'canary.json')):
+            content = read(original)
+            (destination / target).write_bytes(content)
+            saved[target] = digest(content)
+    snapshot_content = (json.dumps(result, indent=2, allow_nan=False) + '\n').encode()
+    review = dict(status='complete_verified', reviewed_at_utc=datetime.now(timezone.utc).isoformat(),
+                  archive_sha256=file_sha(archive), verified_daily_cases=1473, verified_storms=270,
+                  hashed_case_files=2946, verified_physical_pressure_grids=1473 * 20,
+                  model=MODEL_LABEL, software_version='0.3.0', software_commit=CODE_COMMIT,
+                  checkpoint_sha256=WEIGHTS, cohort_sha256=COHORT, members=1,
+                  backend='NVIDIA GeForce RTX 3070 / CUDA',
+                  all_route_and_pressure_scores_recomputed=True,
+                  exact_common_curve_case_ids=True, all_original_forecasts_sha256_verified=True,
+                  causal_input_metadata_and_saved_hashes_verified=True,
+                  raw_era5_states_in_return_zip=False,
+                  raw_input_audit_scope='Original worker receipt and saved input hashes/times; '
+                                       'raw ERA5 states were not returned in this result-only ZIP.',
+                  new_inference_performed=False, model_weights_changed=False,
+                  forecasts_modified=False, missing_results_scored_as_zero=0,
+                  previous_snapshot_sha256=digest(base_bytes), snapshot_sha256=digest(snapshot_content),
+                  imported_files_sha256=saved,
+                  period_coverage={period: dict(daily_issues=sum(len(d) for d in groups.values() if d[0]['period'] == period),
+                      storms=sum(d[0]['period'] == period for d in groups.values())) for period in exported['periods']})
+    snapshot_path.write_bytes(snapshot_content)
+    (destination / 'publication_audit.json').write_text(json.dumps(review, indent=2) + '\n')
+    print(json.dumps(review, indent=2), flush=True)
+
+
+if __name__ == '__main__':
+    parser = argparse.ArgumentParser(description=__doc__)
+    parser.add_argument('--archive', type=Path, required=True)
+    parser.add_argument('--project', type=Path, default=ROOT.parent)
+    parser.add_argument('--output', type=Path, default=ROOT / 'evaluation/deepmind_daily')
+    args = parser.parse_args()
+    audit(args.archive, args.project, args.output)

release_tools/plot_model_announcement.py

@@ -1,10 +1,11 @@
-"""Three release metrics from frozen scores; pending models have no fake bars."""
+"""Three release metrics from the independently audited three-model cohort."""
 import hashlib
 import json
 from pathlib import Path
 import matplotlib
 matplotlib.use('Agg')
 import matplotlib.pyplot as plt
+import numpy as np
 
 ROOT = Path(__file__).resolve().parents[1]
 SOURCE = ROOT/'evaluation/released_daily/released_daily_benchmark.json'
@@ -15,50 +16,54 @@ def main():
     source = json.loads(SOURCE.read_text())
     if source['status'] != 'complete_verified' or source['cohort']['daily_issues'] != 1473:
         raise ValueError('Chart requires verified daily-cohort results')
+    reference = next(m for m in source['models'] if m['key'] == 'deepmind')
+    audit = json.loads((ROOT/'evaluation/deepmind_daily/publication_audit.json').read_text())
+    if reference['status'] != 'complete_verified' or audit['snapshot_sha256'] != hashlib.sha256(SOURCE.read_bytes()).hexdigest():
+        raise ValueError('DeepMind bars require the matching independent publication audit')
     metrics = {m['key']: m for m in source['metrics']}
+    models = ('1.1', '1.2', 'deepmind')
     plt.rcParams.update({'font.family': 'DejaVu Sans', 'font.size': 11,
                          'axes.spines.top': False, 'axes.spines.right': False,
                          'axes.spines.left': False, 'axes.spines.bottom': False})
-    fig, axes = plt.subplots(1, 3, figsize=(13.8, 5.7))
+    fig, axes = plt.subplots(1, 3, figsize=(15.2, 6.1))
     fig.set_facecolor('#f6f8fc')
     specs = [('pressure_JMA_hpa', 'Pressure intensity', 'Central-pressure MAE · hPa', 18, 2),
              ('mean_track_error_km', 'Track position', 'Mean position error · km', 1050, 1),
              ('direction_error_deg', 'Track direction', 'Six-hour heading error · degrees', 65, 2)]
     for ax, (key, title, label, maximum, digits) in zip(axes, specs):
         row = metrics[key]
-        values = [row['values'][k] for k in ('1.1', '1.2')]
+        values = [row['values'][k] for k in models]
+        if any(v is None or not np.isfinite(v) or v < 0 for v in values):
+            raise ValueError('Missing or invalid metric cannot become a zero bar')
         ax.set_facecolor('#f6f8fc')
         ax.set_axisbelow(True)
         ax.grid(axis='y', color='#dfe5ee', linewidth=.8)
-        ax.bar([0, 1], values, width=.53, color=['#9aa9bc', '#3c67d6'], zorder=3)
-        if key == 'pressure_JMA_hpa':
-            intervals = [row['intervals'][k] for k in ('1.1', '1.2')]
-            ax.errorbar([0, 1], values, yerr=[[v-low for v, (low, high) in zip(values, intervals)],
-                        [high-v for v, (low, high) in zip(values, intervals)]],
-                        fmt='none', color='#253853', capsize=5, linewidth=1.2, zorder=4)
+        ax.bar([0, 1, 2], values, width=.55, color=['#9aa9bc', '#3c67d6', '#9073cf'], zorder=3)
         for x, value in enumerate(values):
-            top = row['intervals'][('1.1', '1.2')[x]][1] if key == 'pressure_JMA_hpa' else value
-            ax.text(x, top+maximum*.038, f'{value:,.{digits}f}', ha='center',
+            ax.text(x, value+maximum*.038, f'{value:,.{digits}f}', ha='center',
                     fontsize=17, fontweight='bold', color='#21334c')
         ax.set_ylim(0, maximum)
-        ax.set_xticks([0, 1], ['1.1', '1.2 · mean of 50'])
+        ax.set_xticks([0, 1, 2], ['1.1', '1.2\nmean of 50', 'DeepMind Mini\none member'])
         ax.tick_params(axis='both', length=0, labelcolor='#53647c')
         ax.set_title(title, loc='left', fontsize=16, fontweight='bold', color='#21334c', pad=38)
         ax.text(0, 1.03, label, transform=ax.transAxes, color='#53647c', fontsize=10)
         coverage = row['coverage']
-        ax.text(.5, -.16, f"{coverage['daily_issues']:,} daily starts / {coverage['storms']} storms",
+        ax.text(.5, -.225, f"{coverage['daily_issues']:,} daily starts / {coverage['storms']} storms",
                 transform=ax.transAxes, ha='center', color='#53647c', fontsize=10)
-    fig.text(.055, .945, 'Trackformer 1.2', fontsize=23, fontweight='bold', color='#182b45')
+    fig.text(.055, .945, 'Trackformer 1.2 · matched model comparison', fontsize=23, fontweight='bold', color='#182b45')
     fig.text(.055, .895, 'Matched five-day forecasts · equal weight per storm · lower is better', fontsize=12, color='#53647c')
-    fig.text(.055, .095, 'Pressure: JMA reference; bars show 95% whole-storm intervals. Paired improvement is uncertain.', fontsize=10, color='#53647c')
-    fig.text(.055, .057, 'DeepMind Mini: matched run in progress — no score yet. Different pipelines/member policies; development evidence.', fontsize=10, color='#53647c')
-    fig.subplots_adjust(left=.055, right=.975, top=.735, bottom=.25, wspace=.36)
+    fig.text(.055, .095, 'DeepMind: WeatherNext Cyclones Mini <2024 · software v0.3.0 · 1° · RTX 3070 CUDA · all 1,473 forecasts verified.', fontsize=10, color='#53647c')
+    fig.text(.055, .057, 'Mean errors, not significance claims. Mixed historical/recent cohort; Mini has better recent-only track scores. Different pipelines/member policies.', fontsize=10, color='#53647c')
+    fig.subplots_adjust(left=.055, right=.975, top=.735, bottom=.29, wspace=.36)
     fig.savefig(DEST, dpi=180, facecolor=fig.get_facecolor())
     plt.close(fig)
     receipt = {'source': SOURCE.relative_to(ROOT).as_posix(), 'source_sha256': hashlib.sha256(SOURCE.read_bytes()).hexdigest(),
                'chart': DEST.relative_to(ROOT).as_posix(), 'chart_sha256': hashlib.sha256(DEST.read_bytes()).hexdigest(),
                'metrics': [s[0] for s in specs], 'values': {k: metrics[k]['values'] for k, *_ in specs},
-               'deepmind_score': None, 'zero_fill': False, 'model_forecasts_modified': False}
+               'deepmind_status': 'complete_verified', 'deepmind_model': reference,
+               'models': list(models), 'bars_per_panel': 3, 'error_bars': False,
+               'zero_fill': False, 'model_forecasts_modified': False,
+               'publication_audit_sha256': hashlib.sha256((ROOT/'evaluation/deepmind_daily/publication_audit.json').read_bytes()).hexdigest()}
     DEST.with_suffix('.json').write_text(json.dumps(receipt, indent=2)+'\n')
     print(json.dumps(receipt, indent=2))
 

release_tools/sync_public_model_cards.py

@@ -40,6 +40,8 @@ SYNC_FILES = (
     'release_tools/plot_release_pressure_benchmark.py',
     'release_tools/plot_daily_storm_final.py',
     'release_tools/plot_model_announcement.py',
+    'release_tools/import_deepmind_release_results.py',
+    'release_tools/test_deepmind_release_results.py',
     'release_tools/build_mangkhut_gif.py',
     'release_tools/deepmind_daily_benchmark.py',
     'release_tools/test_deepmind_daily_benchmark.py',
@@ -60,6 +62,12 @@ SYNC_FILES = (
     'evaluation/released_daily/pressure_comparison.png',
     'evaluation/released_daily/model_1_2_benchmark.png',
     'evaluation/released_daily/model_1_2_benchmark.json',
+    'evaluation/deepmind_daily/benchmark.json',
+    'evaluation/deepmind_daily/verification.json',
+    'evaluation/deepmind_daily/publication_audit.json',
+    'evaluation/deepmind_daily/protocol.json',
+    'evaluation/deepmind_daily/case-manifest.json',
+    'evaluation/deepmind_daily/canary.json',
 )
 
 
@@ -172,13 +180,13 @@ def run(output, publish=False):
         # immutable commit in the card. Never pin new bytes to the old movie
         # revision or let a missing future image render as a stale figure.
         assets = api.create_commit(repo_id=REPO, repo_type='model', parent_commit=parent,
-            commit_message='Publish Trackformer 1.2 announcement assets and matched DeepMind benchmark protocol',
+            commit_message='Publish completed WeatherNext Cyclones Mini daily comparison and audited three-model chart',
             operations=[CommitOperationAdd(path_in_repo=name, path_or_fileobj=str(ROOT/name)) for name in SYNC_FILES])
         card_path.write_text(render_card(original.read_text(), github, assets.oid))
         receipt['files']['README.md'] = sha(card_path)
         receipt['figure_revision'] = assets.oid
         committed = api.create_commit(repo_id=REPO, repo_type='model', parent_commit=assets.oid,
-            commit_message='Introduce Trackformer 1.2 with a focused showcase and verified daily metrics',
+            commit_message='Update Trackformer 1.2 model card with completed DeepMind CUDA benchmark',
             operations=[CommitOperationAdd(path_in_repo='README.md', path_or_fileobj=str(card_path))])
         for name, expected in receipt['files'].items():
             saved = hf_hub_download(REPO, name, revision=committed.oid, cache_dir=CACHE)

release_tools/test_deepmind_release_results.py

@@ -0,0 +1,84 @@
+"""Offline publication regressions: no inference, network, or model imports."""
+import copy
+import hashlib
+import json
+import unittest
+
+from import_deepmind_release_results import ROOT, merge_snapshot, same, validate_identity
+
+
+class DeepMindReleaseTest(unittest.TestCase):
+    @classmethod
+    def setUpClass(cls):
+        folder = ROOT/'evaluation/deepmind_daily'
+        cls.protocol = json.loads((folder/'protocol.json').read_text())
+        cls.receipt = json.loads((folder/'verification.json').read_text())
+        cls.export = json.loads((folder/'benchmark.json').read_text())
+        cls.base = json.loads((ROOT/'evaluation/released_daily/released_daily_benchmark.json').read_text())
+
+    def test_exact_completed_model_and_cohort(self):
+        validate_identity(self.protocol, self.receipt, self.export)
+
+    def test_partial_run_cannot_be_published(self):
+        receipt = copy.deepcopy(self.receipt)
+        receipt['verified_daily_cases'] -= 1
+        with self.assertRaisesRegex(ValueError, 'Incomplete coverage'):
+            validate_identity(self.protocol, receipt, self.export)
+
+    def test_missing_and_duplicate_ids_are_rejected(self):
+        receipt = copy.deepcopy(self.receipt)
+        receipt['case_indices'][-1] = receipt['case_indices'][-2]
+        with self.assertRaisesRegex(ValueError, 'duplicate case IDs'):
+            validate_identity(self.protocol, receipt, self.export)
+
+    def test_checkpoint_substitution_is_rejected(self):
+        protocol = copy.deepcopy(self.protocol)
+        protocol['checkpoint_sha256'] = '0' * 64
+        with self.assertRaisesRegex(ValueError, 'official checkpoint'):
+            validate_identity(protocol, self.receipt, self.export)
+
+    def test_cpu_results_cannot_be_relabelled_gpu(self):
+        receipt = copy.deepcopy(self.receipt)
+        receipt['cpu_predictions_reused_as_cuda'] = True
+        with self.assertRaisesRegex(ValueError, 'CPU results relabelled'):
+            validate_identity(self.protocol, receipt, self.export)
+
+    def test_unavailable_does_not_equal_zero(self):
+        with self.assertRaises(ValueError):
+            same(0, None, 'unavailable')
+        with self.assertRaises(ValueError):
+            same(None, 0, 'unavailable')
+
+    def test_original_scores_cannot_change_on_import(self):
+        base = copy.deepcopy(self.base)
+        next(m for m in base['metrics'] if m['key'] == 'mean_track_error_km')['values']['1.2'] = 0
+        with self.assertRaisesRegex(ValueError, 'Original score changed'):
+            merge_snapshot(base, self.export, self.receipt, {})
+
+    def test_mean_of_fifty_and_mini_identity_remain_separate(self):
+        models = {m['key']: m for m in self.base['models']}
+        self.assertEqual(models['v12']['member_count'], 50)
+        self.assertEqual(models['deepmind']['member_count'], 1)
+        self.assertEqual(models['deepmind']['software_version'], '0.3.0')
+        self.assertEqual(models['deepmind']['model_checkpoint'], 'WeatherNextCyclones_Mini_<2024')
+
+    def test_audit_matches_imported_bytes_and_chart_snapshot(self):
+        folder = ROOT/'evaluation/deepmind_daily'
+        audit = json.loads((folder/'publication_audit.json').read_text())
+        for name, expected in audit['imported_files_sha256'].items():
+            self.assertEqual(hashlib.sha256((folder/name).read_bytes()).hexdigest(), expected)
+        self.assertEqual(hashlib.sha256((ROOT/'evaluation/released_daily/released_daily_benchmark.json').read_bytes()).hexdigest(),
+                         audit['snapshot_sha256'])
+        self.assertFalse(audit['raw_era5_states_in_return_zip'])
+        self.assertEqual(audit['missing_results_scored_as_zero'], 0)
+
+    def test_all_published_values_are_actual_completed_export(self):
+        rows = {m['key']: m for m in self.base['metrics']}
+        for metric in self.export['metrics']:
+            self.assertEqual(rows[metric['key']]['values']['deepmind'], metric['values']['deepmind'])
+        self.assertNotIn('deepmind', rows['frechet_distance_km']['values'])
+        self.assertNotIn('deepmind', rows['path_similarity']['values'])
+
+
+if __name__ == '__main__':
+    unittest.main()

release_tools/test_public_model_cards.py

@@ -15,11 +15,15 @@ class PublicModelCardsTest(unittest.TestCase):
         self.assertTrue(rendered.startswith(self.header))
         for text in ('# Introducing Trackformer 1.2', '1,473 days / 270 storms',
                      '134 days / 40 storms', '13.53', '12.84',
-                     'scores are **pending**', 'no native wind-radius forecast head'):
+                     'WeatherNext software v0.3.0', '10.20 hPa', '288.5 km',
+                     'no native wind-radius forecast head'):
             self.assertIn(text, rendered)
         self.assertNotIn('README revision', rendered)
         self.assertNotIn('Corrected pressure forecast', rendered)
         self.assertNotIn('Old model card', rendered)
+        self.assertNotIn('scores are **pending**', rendered)
+        self.assertIn('Mini, not the full-sized', rendered)
+        self.assertIn('recent-only track results favour Mini', rendered)
 
     def test_one_pinned_featured_player(self):
         rendered = render_card(self.original, self.source)
@@ -65,7 +69,9 @@ class PublicModelCardsTest(unittest.TestCase):
             self.assertIn(path, SYNC_FILES)
         for path in ('docs/deepmind_daily_benchmark.md', 'docs/showcase_archive.md',
                      'evaluation/released_daily/released_daily_benchmark.json',
-                     'evaluation/released_daily/released_daily_verification.json'):
+                     'evaluation/released_daily/released_daily_verification.json',
+                     'evaluation/deepmind_daily/publication_audit.json',
+                     'evaluation/deepmind_daily/verification.json'):
             self.assertIn(path, SYNC_FILES)
         self.assertNotIn('models/trackformer_1_2_field/weights.pt', SYNC_FILES)
         self.assertNotIn('paper/trackformer.tex', SYNC_FILES)