Align Flash checkpoint layout with PhysicsNeMo and document MRMS evaluation

Update the six Flash .mdlus archives to the validated native layer-key layout and earth2studio.models.nn.stormscope_flash serialization path. Preserve trained tensors, expert numbering, selected weights and sampling recipes. Update deployment hashes and sizes only in the registry. Move the MRMS skill table into a standalone evaluation note linked from the model card. Baseline entries, checkpoints and shared assets are unchanged.

README.md

@@ -22,26 +22,8 @@ GOES and MRMS/GLM inference. It uses five model evaluations for GOES and seven f
 MRMS/GLM per forecast step, and supports full-CONUS and regional inference through
 Earth2Studio.
 
-### MRMS forecasting skill
-
-Comparison of the published StormScope teacher baseline and the distilled
-StormScope Flash model. Higher scores indicate better forecasting skill; bold
-marks the higher score for each metric and forecast lead.
-
-| Metric | Model | +30 min | +60 min | +120 min | +180 min |
-| --- | --- | ---: | ---: | ---: | ---: |
-| Object CSI | StormScope (published, 3 km / 10 min) | 0.685 | **0.533** | **0.335** | **0.220** |
-| Object CSI | StormScope Flash | **0.688** | 0.532 | 0.315 | 0.204 |
-| CSI 40 dBZ | StormScope (published, 3 km / 10 min) | **0.597** | **0.479** | **0.348** | **0.271** |
-| CSI 40 dBZ | StormScope Flash | 0.593 | 0.464 | 0.326 | 0.246 |
-| CSI 55 dBZ | StormScope (published, 3 km / 10 min) | 0.274 | **0.170** | **0.068** | **0.028** |
-| CSI 55 dBZ | StormScope Flash | **0.286** | 0.167 | 0.060 | 0.024 |
-| pFSS 40 dBZ | StormScope (published, 3 km / 10 min) | 0.907 | 0.788 | 0.588 | **0.449** |
-| pFSS 40 dBZ | StormScope Flash | **0.913** | **0.800** | **0.593** | 0.433 |
-| pFSS 55 dBZ | StormScope (published, 3 km / 10 min) | 0.719 | **0.502** | **0.201** | **0.064** |
-| pFSS 55 dBZ | StormScope Flash | **0.735** | 0.490 | 0.137 | 0.036 |
-
-CSI: critical success index. pFSS: probabilistic fractions skill score.
+See [MRMS forecasting skill](mrms_forecast_skill.md) for the comparison with
+StormScope, including the evaluation setup, metric definitions, and results.
 
 ### License/Terms of Use:
 

checkpoints/goes/3km_10min_flash/expert_0.mdlus

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:75aeb03cddd5596570a9e1feaa6444173abc24ae736b84c63daf74218cc07007
-size 723168328
+oid sha256:de2f5a2127f62d6dd868a8c4043c033604b77ca5492b9cd0de9aa5a6cd277e83
+size 723168495

checkpoints/goes/3km_10min_flash/expert_1.mdlus

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:4f53361f589ddb3ee4097e4d362d59c027b8551022ee42bb81460df1c3ec11b4
-size 704267546
+oid sha256:0c860c5ef5088ea7a2718231d50c5026438d5d99e774e28db64845c3ee29a31e
+size 704268145

checkpoints/goes/3km_10min_flash/expert_2.mdlus

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:6b8cb151b07c9b754f6422f3f7a763e5154a2165d2482e2b9db37c46f2e0c362
-size 697967288
+oid sha256:6e761285cc87e4d8088cb7fdbebd94c7e7e135b81306d4d0152c27649e40eafd
+size 697968031

checkpoints/mrms/3km_10min_flash/expert_0.mdlus

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:7c3112543db0e4cf299fea12caec180508251230822cc975891079ba3d19e29e
-size 703869930
+oid sha256:322228b92fbdbcca34c3ef50c2e15d37e8b0ebf22b4e0c15992e9aefd384fb40
+size 703869841

checkpoints/mrms/3km_10min_flash/expert_1.mdlus

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:078cfc51e040f156cba7a0db326e1a04fc30a6b899e13c0310662bb70009dbba
-size 696780988
+oid sha256:e572f356680b1a8a7811916aae38052a9c92c12ca2cd1370828224f334ac5dea
+size 696781331

checkpoints/mrms/3km_10min_flash/expert_2.mdlus

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:1aea9704cdf1372ef186574619e4671c6b1ed4e4157e571b6ef35f2501e61a9e
-size 694418010
+oid sha256:d690fa0e363abaff772978fd662ebe15dfd47f3647acfc196906a9a70e2aa2b8
+size 694418497

mrms_forecast_skill.csv

@@ -0,0 +1,11 @@
+Metric,Model,30 min,60 min,120 min,180 min
+Object CSI,StormScope,0.685,0.533,0.335,0.220
+Object CSI,StormScope Flash,0.688,0.532,0.315,0.204
+CSI 40 dBZ,StormScope,0.597,0.479,0.348,0.271
+CSI 40 dBZ,StormScope Flash,0.593,0.464,0.326,0.246
+CSI 55 dBZ,StormScope,0.274,0.170,0.068,0.028
+CSI 55 dBZ,StormScope Flash,0.286,0.167,0.060,0.024
+pFSS 40 dBZ,StormScope,0.907,0.788,0.588,0.449
+pFSS 40 dBZ,StormScope Flash,0.913,0.800,0.593,0.433
+pFSS 55 dBZ,StormScope,0.719,0.502,0.201,0.064
+pFSS 55 dBZ,StormScope Flash,0.735,0.490,0.137,0.036

mrms_forecast_skill.md

@@ -0,0 +1,57 @@
+# StormScope Flash: MRMS forecasting skill
+
+This note compares the 3 km, 10-minute StormScope baseline with its distilled
+StormScope Flash variant for composite radar reflectivity. Higher scores are better;
+1 is perfect. StormScope values are the published baseline scores; Flash values
+come from the archived full-domain evaluation.
+
+## Evaluation
+
+The evaluation uses 929 WoFS-catalog initializations from March 4 through August 9,
+2025. Verification is observed MRMS composite reflectivity on the 3 km model grid
+at matching valid times. Flash runs coupled GOES and MRMS/GLM forecasts over the
+full 1024 x 1792 domain, then selects each catalog region for scoring. Forecasts
+contain 18 ten-minute leads and 16 ensemble members; GOES and MRMS use five and
+seven network calls per lead, respectively.
+
+Object CSI and neighborhood CSI use member 0, not the ensemble mean. pFSS uses all
+16 members. Contingency counts and pFSS numerator/denominator terms are summed
+over cases at each lead before computing the score; these are not averages of
+individual case scores.
+
+## Metrics
+
+- **Object CSI** measures storm detection using connected regions at or above
+  40 dBZ, each containing a 45 dBZ core and at least 12 grid cells (108 km²).
+  One-to-one matching uses centroid and boundary distances with 40 km cutoffs,
+  area-ratio weighting, five matching passes, and total interest greater than 0.2.
+  CSI = hits / (hits + false alarms + misses). Redundant unmatched objects
+  classified as extras are excluded. The table pools all storm ages.
+- **CSI at 40/55 dBZ** is neighborhood-grid CSI, not object CSI or exact-pixel CSI.
+  Forecast and observation exceedances are each dilated with an 11 x 11 cell
+  (33 x 33 km) window before forming the same contingency ratio.
+- **pFSS at 40/55 dBZ** compares neighborhood exceedance fractions. Forecast
+  fractions are averaged across the 16 members and compared with the observed
+  fraction: pFSS = 1 - sum((forecast fraction - observed fraction)²) /
+  sum(forecast fraction² + observed fraction²). The window is also 11 x 11 cells.
+
+## Results
+
+| Metric | Model | +30 min | +60 min | +120 min | +180 min |
+|---|---|---:|---:|---:|---:|
+| Object CSI | StormScope | 0.685 | 0.533 | 0.335 | 0.220 |
+| Object CSI | StormScope Flash | 0.688 | 0.532 | 0.315 | 0.204 |
+| CSI 40 dBZ | StormScope | 0.597 | 0.479 | 0.348 | 0.271 |
+| CSI 40 dBZ | StormScope Flash | 0.593 | 0.464 | 0.326 | 0.246 |
+| CSI 55 dBZ | StormScope | 0.274 | 0.170 | 0.068 | 0.028 |
+| CSI 55 dBZ | StormScope Flash | 0.286 | 0.167 | 0.060 | 0.024 |
+| pFSS 40 dBZ | StormScope | 0.907 | 0.788 | 0.588 | 0.449 |
+| pFSS 40 dBZ | StormScope Flash | 0.913 | 0.800 | 0.593 | 0.433 |
+| pFSS 55 dBZ | StormScope | 0.719 | 0.502 | 0.201 | 0.064 |
+| pFSS 55 dBZ | StormScope Flash | 0.735 | 0.490 | 0.137 | 0.036 |
+
+These results describe the archived evaluation, not a new evaluation of the
+repackaged Earth2Studio checkpoints. Baseline and Flash scoring conventions were
+not fully harmonized, and statistical equivalence was not assessed.
+
+[Download table values](mrms_forecast_skill.csv).

registry.json

@@ -157,8 +157,8 @@
             "source_checkpoint": "checkpoint_92800.pt",
             "source_sha256": "3185d14dc34549b52e02d93bf31fdc89b2ae78b325e963786736894028bfda47",
             "source_size_bytes": 2093144509,
-            "deployment_sha256": "6b8cb151b07c9b754f6422f3f7a763e5154a2165d2482e2b9db37c46f2e0c362",
-            "deployment_size_bytes": 697967288,
+            "deployment_sha256": "6e761285cc87e4d8088cb7fdbebd94c7e7e135b81306d4d0152c27649e40eafd",
+            "deployment_size_bytes": 697968031,
             "deployment_format": "flash-mdlus-v1",
             "tensor_parity": "exact"
           },
@@ -168,8 +168,8 @@
             "source_checkpoint": "checkpoint_88800.pt",
             "source_sha256": "01d1aa63ce440602fd9a56c5692450ed90e943f8e9745760680054bdf66f22e2",
             "source_size_bytes": 2112043773,
-            "deployment_sha256": "4f53361f589ddb3ee4097e4d362d59c027b8551022ee42bb81460df1c3ec11b4",
-            "deployment_size_bytes": 704267546,
+            "deployment_sha256": "0c860c5ef5088ea7a2718231d50c5026438d5d99e774e28db64845c3ee29a31e",
+            "deployment_size_bytes": 704268145,
             "deployment_format": "flash-mdlus-v1",
             "tensor_parity": "exact"
           },
@@ -179,8 +179,8 @@
             "source_checkpoint": "checkpoint_82400.pt",
             "source_sha256": "a7a1eef05f21ebf64b2eeca6989cbab60da4e471c3c91e8b8cd753730ff37696",
             "source_size_bytes": 2168741437,
-            "deployment_sha256": "75aeb03cddd5596570a9e1feaa6444173abc24ae736b84c63daf74218cc07007",
-            "deployment_size_bytes": 723168328,
+            "deployment_sha256": "de2f5a2127f62d6dd868a8c4043c033604b77ca5492b9cd0de9aa5a6cd277e83",
+            "deployment_size_bytes": 723168495,
             "deployment_format": "flash-mdlus-v1",
             "tensor_parity": "exact"
           }
@@ -328,8 +328,8 @@
             "source_checkpoint": "checkpoint_76400.pt",
             "source_sha256": "813d47717b7be8325c36d59292a4895d61a1b152bb1739fba5143de546e4034e",
             "source_size_bytes": 2082983991,
-            "deployment_sha256": "1aea9704cdf1372ef186574619e4671c6b1ed4e4157e571b6ef35f2501e61a9e",
-            "deployment_size_bytes": 694418010,
+            "deployment_sha256": "d690fa0e363abaff772978fd662ebe15dfd47f3647acfc196906a9a70e2aa2b8",
+            "deployment_size_bytes": 694418497,
             "deployment_format": "flash-mdlus-v1",
             "tensor_parity": "exact"
           },
@@ -339,8 +339,8 @@
             "source_checkpoint": "checkpoint_84400.pt",
             "source_sha256": "f9767d0ac01ab61094559b9cc445c4c0e8a312a9ebe059bfc3af650182a9329d",
             "source_size_bytes": 2090071415,
-            "deployment_sha256": "078cfc51e040f156cba7a0db326e1a04fc30a6b899e13c0310662bb70009dbba",
-            "deployment_size_bytes": 696780988,
+            "deployment_sha256": "e572f356680b1a8a7811916aae38052a9c92c12ca2cd1370828224f334ac5dea",
+            "deployment_size_bytes": 696781331,
             "deployment_format": "flash-mdlus-v1",
             "tensor_parity": "exact"
           },
@@ -350,8 +350,8 @@
             "source_checkpoint": "checkpoint_80000.pt",
             "source_sha256": "0736cc36330ac87f7f71f0a5b1b7431a094f6f947aebf5063813bf01f4e2e8e1",
             "source_size_bytes": 2111333559,
-            "deployment_sha256": "7c3112543db0e4cf299fea12caec180508251230822cc975891079ba3d19e29e",
-            "deployment_size_bytes": 703869930,
+            "deployment_sha256": "322228b92fbdbcca34c3ef50c2e15d37e8b0ebf22b4e0c15992e9aefd384fb40",
+            "deployment_size_bytes": 703869841,
             "deployment_format": "flash-mdlus-v1",
             "tensor_parity": "exact"
           }
@@ -381,4 +381,4 @@
     "glm_preprocessing": "bilinear_raw_counts_then_log1p",
     "coordinate": "edm_linear_t"
   }
-}
+}