Show total, pre-2024 and post-2024 results with explicit 1.2 training years
Browse files
README.md
CHANGED
|
@@ -50,7 +50,7 @@ Trackformer 1.1 predicts track and scalar intensity/structure outputs. **1.2 add
|
|
| 50 |
|
| 51 |
### Total · all completed forecast starts
|
| 52 |
|
| 53 |
-
 · [Original completed results](https://huggingface.co/euler314/typhoon-predict/blob/main/evaluation/deepmind_daily/benchmark.json) · [CUDA completion receipt](https://huggingface.co/euler314/typhoon-predict/blob/main/evaluation/deepmind_daily/verification.json) · [Independent publication audit](https://huggingface.co/euler314/typhoon-predict/blob/main/evaluation/deepmind_daily/publication_audit.json) · [Model identity and reproducible protocol](https://huggingface.co/euler314/typhoon-predict/blob/main/docs/deepmind_daily_benchmark.md)
|
| 105 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 106 |
## How Trackformer 1.2 works
|
| 107 |
|
| 108 |
-
|
| 54 |
|
| 55 |
| Development metric | 1.1 | 1.2 · mean of 50 | DeepMind Mini <2024 · one member | Shared coverage |
|
| 56 |
| --- | ---: | ---: | ---: | --- |
|
|
|
|
| 85 |
|
| 86 |
Because the final score weights storms equally, the historical group contributes **85.2% of the total track score's storm weight**, not its 90.7% share of starts. These dates overlap Mini's fitting years; the mixed-year total is not an unused temporal test. Calendar 2024 is already later than Mini's training cutoff, but is intentionally excluded from the requested strict `>2024` chart.
|
| 87 |
|
| 88 |
+
### Before 2024 · historical forecast starts only
|
| 89 |
+
|
| 90 |
+
|
| 91 |
+
|
| 92 |
+
| Pre-2024 development metric | 1.1 | 1.2 · mean of 50 | DeepMind Mini <2024 · one member | Shared coverage |
|
| 93 |
+
| --- | ---: | ---: | ---: | --- |
|
| 94 |
+
| Historical track position MAE · lower is better | 780.5 km | **470.2 km** | 534.9 km | 1,336 days / 230 storms |
|
| 95 |
+
| Historical track-direction error · lower is better | 51.36° | **35.01°** | 41.26° | 1,336 days / 230 storms |
|
| 96 |
+
| Historical route-shape similarity · centred · higher is better | 0.7565 | 0.8802 | **0.8934** | 1,336 days / 230 storms |
|
| 97 |
+
| Historical JMA central-pressure MAE | Not scored | Not scored | Not scored | 0 shared valid pressure starts |
|
| 98 |
+
|
| 99 |
+
This is strict UTC issue year `<2024`, **all 1980–1999 in this frozen cohort**, not every storm before 2024. The historical group overlaps Mini's training years. **Pressure is not scored** because the frozen 1.1 historical issue-time intensity inputs are unavailable; the shared three-model pressure mask is empty. The chart deliberately draws no pressure bars—missing data are not zero error. Route-shape similarity is separate from geographic track accuracy.
|
| 100 |
+
|
| 101 |
### After 2024 · 2025–2026 forecast starts only
|
| 102 |
|
| 103 |
+
|
| 104 |
|
| 105 |
| Post-2024 development metric | 1.1 | 1.2 · mean of 50 | DeepMind Mini <2024 · one member | Shared coverage |
|
| 106 |
| --- | ---: | ---: | ---: | --- |
|
|
|
|
| 116 |
|
| 117 |
[Exact date proportions and recomputed subset scores](https://huggingface.co/euler314/typhoon-predict/blob/main/evaluation/deepmind_daily/period_comparison.json) · [Original completed results](https://huggingface.co/euler314/typhoon-predict/blob/main/evaluation/deepmind_daily/benchmark.json) · [CUDA completion receipt](https://huggingface.co/euler314/typhoon-predict/blob/main/evaluation/deepmind_daily/verification.json) · [Independent publication audit](https://huggingface.co/euler314/typhoon-predict/blob/main/evaluation/deepmind_daily/publication_audit.json) · [Model identity and reproducible protocol](https://huggingface.co/euler314/typhoon-predict/blob/main/docs/deepmind_daily_benchmark.md)
|
| 118 |
|
| 119 |
+
## Training data and year cutoffs
|
| 120 |
+
|
| 121 |
+
**Released Trackformer 1.2 was fitted on 2000–2021 weather and typhoon data.** Validation used 2022–2023 for checkpoint selection; the original test partition used 2024–2025. **2026 live or historical forecasts are inference with frozen 1.2 weights, not training through 2026.**
|
| 122 |
+
|
| 123 |
+
| Data used by 1.2 | Source and variables | Years used for fitting |
|
| 124 |
+
| --- | --- | --- |
|
| 125 |
+
| Typhoon observations and labels | [NOAA IBTrACS](https://www.ncei.noaa.gov/products/international-best-track-archive)-derived storm positions, recent motion, available maximum wind and central pressure, with validity masks | 2000–2021 · Western Pacific-domain windows |
|
| 126 |
+
| Large-scale weather | [NOAA PSL NCEP/NCAR Reanalysis 1](https://psl.noaa.gov/data/gridded/data.ncep.reanalysis.html): sea-level pressure, 500 hPa height, and east/north winds at 850, 500 and 200 hPa; six-hour, 2.5° basin grid | 2000–2021 |
|
| 127 |
+
| Native regional pressure detail | [ARCO-ERA5](https://github.com/google-research/arco-era5) mean sea-level pressure at 0.25°, fixed issue-centred patches when available | 2000–2021 · 800 fitting windows |
|
| 128 |
+
| Static geography | ERA5 land fraction and surface geopotential converted to elevation, plus latitude/longitude | Static fields; not extra typhoon training years |
|
| 129 |
+
|
| 130 |
+
| Original whole-storm partition | Weather / typhoon years | Eligible six-hour windows | Native-pressure windows |
|
| 131 |
+
| --- | --- | ---: | ---: |
|
| 132 |
+
| Fitting · gradient updates and weather normalization | 2000–2021 | 13,949 | 800 |
|
| 133 |
+
| Validation · checkpoint selection, no gradient updates | 2022–2023 | 1,041 | 100 |
|
| 134 |
+
| Original test · evaluation only | 2024–2025 | 1,195 | 100 |
|
| 135 |
+
|
| 136 |
+
Each input contains nine analyses from −48 to 0 h. Later fields and best-track values are **training targets or evaluation labels**, never positive-lead prediction inputs. Entire storms stay in one partition and windows crossing a year-partition boundary are excluded. Weather normalization is fitted only on 2000–2021.
|
| 137 |
+
|
| 138 |
+
The checksum-pinned source caches extend further: the derived typhoon archive reaches **13 July 2026**, and the NCEP basin archive reaches **17 March 2026, 18 UTC**. Those endpoints describe archive availability, **not the fitting cutoff**. Later live weather uses separately documented experimental GFS transfer; it does not update the released weights. This release has no satellite cloud-image or humidity input channel. Previously inspected test/showcase storms remain development evidence, not a newly untouched holdout.
|
| 139 |
+
|
| 140 |
+
[Training-source details and exact date ranges](https://huggingface.co/euler314/typhoon-predict/blob/main/docs/trackformer_1_2_training_data.md) · [Recomputed split counts and verified dataset hashes](https://huggingface.co/euler314/typhoon-predict/blob/main/evaluation/training_data/trackformer_1_2_provenance.json)
|
| 141 |
+
|
| 142 |
## How Trackformer 1.2 works
|
| 143 |
|
| 144 |
+
|
| 145 |
|
| 146 |
1. **Initialize from the past.** Nine six-hour analyses provide MSLP, 500 hPa height and winds at 850/500/200 hPa, ending at issue time. Static geography, the current observed centre, recent motion, current intensity and validity masks accompany the history. Native regional pressure detail is used when available; missing detail stays flagged. Current observations initialize the core, not future labels.
|
| 147 |
2. **Evolve the environment.** A convolutional recurrent network combines steering-based advection with learned pressure tendencies. Multiscale spatial tokens and attention condition the environmental memory.
|