Consolidate documentation into three entry pages; preserve full archive and training PDF
Browse files- CONTRIBUTING.md +0 -18
- OPENNESS.md +0 -29
- README.md +31 -155
- RELEASE_NOTES_TRACKFORMER_1_2.md +0 -45
- docs/REFERENCE.md +1068 -0
- docs/archive/documentation-before-consolidation-20261030.zip +0 -0
- docs/archive/documentation-manifest.json +228 -0
- docs/daily_storm_benchmark.md +0 -44
- docs/deepmind_daily_benchmark.md +0 -98
- docs/intensity_benchmark.md +0 -131
- docs/repository_layout.md +0 -23
- docs/research_report_zh.md +0 -128
- docs/showcase_archive.md +0 -36
- docs/trackformer_1_2_evaluation.md +0 -64
- docs/trackformer_1_2_training_data.md +0 -5
- evaluation/README.md +0 -24
- models/trackformer_1_2_field/README.md +6 -6
- models/trackformer_1_2_field/WIND_ESTIMATION.md +0 -46
- release_tools/STORM_STRUCTURE_DIAGNOSTICS.md +0 -118
- release_tools/sync_public_model_cards.py +90 -17
- release_tools/test_public_model_cards.py +136 -208
- training/trackformer_1_2/PUBLICATION.md +0 -19
- training/trackformer_1_2/README.md +0 -9
- training/trackformer_1_2/SOURCE_MANIFEST.json +46 -31
- training/trackformer_1_2/THIRD_PARTY_NOTICES.md +0 -9
- training/trackformer_1_2/source/data_preparation/v164_dataset_contract.md +0 -41
- training/trackformer_1_2/source/data_preparation/v164_structure_research.md +0 -142
- training/trackformer_1_2/source/v173/DESIGN.md +0 -52
CONTRIBUTING.md
DELETED
|
@@ -1,18 +0,0 @@
|
|
| 1 |
-
# Contributing to TrackFormer
|
| 2 |
-
|
| 3 |
-
Anyone may inspect, fork, modify and retrain the public release under its MIT license. No contributor agreement or account approval is required to download or use the code and weights. Submitting a GitHub pull request requires your own GitHub account; local use does not.
|
| 4 |
-
|
| 5 |
-
## Reproduce first
|
| 6 |
-
|
| 7 |
-
1. Follow `training/trackformer_1_2/README.md` and pin the asset revision from `PUBLICATION.md`.
|
| 8 |
-
2. Verify dataset/checkpoint SHA-256 values before changing anything. Keep original weights and source artifacts unchanged and write experiments into new output directories.
|
| 9 |
-
3. Use `--preflight` for the archive/architecture check and `test_training_cpu.py` for the portable CPU loss/backward check. These reconstruct a writable local mmap, so use your own data copy rather than an active trainer's directory.
|
| 10 |
-
4. For pressure export changes, use the existing `release_tools/test_pressure_field_export.py` checks as applicable. There is no requirement to run the full multi-hour training job for a documentation correction.
|
| 11 |
-
|
| 12 |
-
## Share a change
|
| 13 |
-
|
| 14 |
-
Open an issue or pull request at https://github.com/yu314-coder/typhoon-predict . Explain the problem, changed behavior, exact model/data revision and relevant checks. Document new dependencies and retain third-party notices. Never commit credentials, private data or unrelated large caches. New project contributions should use the existing MIT license; third-party material must keep its own license and attribution.
|
| 15 |
-
|
| 16 |
-
For scientific claims, preserve issue-time causal inputs and whole-storm partitions; keep future observations as labels only. Compare on identical cases, grids, times and masks. Report misses and false alarms as well as mean errors, and identify previously reused development data. Do not infer forecast skill from a smoother map or training loss. Label changed architectures and weights as new candidates rather than silently replacing released 1.2.
|
| 17 |
-
|
| 18 |
-
The MIT license permits broad reuse; scientific review criteria apply to what this repository accepts or claims, not to your permission to fork or experiment independently.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
OPENNESS.md
DELETED
|
@@ -1,29 +0,0 @@
|
|
| 1 |
-
# Open access and reuse
|
| 2 |
-
|
| 3 |
-
TrackFormer 1.2 is publicly accessible without account approval, a download gate, a paid API or Google Drive permissions. Project code and project model weights/checkpoints are distributed under the [MIT license](LICENSE). You may use, study, modify, retrain, redistribute, sublicense and sell copies, including commercial use, subject to retaining the required copyright and permission notice. No research-only or non-commercial restriction is added by this project. The research/safety cautions describe capability limits; they do not replace or narrow the MIT license.
|
| 4 |
-
|
| 5 |
-
## What is public
|
| 6 |
-
|
| 7 |
-
| Component | Public location |
|
| 8 |
-
| --- | --- |
|
| 9 |
-
| Architecture, inference and physical pressure exporter | `models/trackformer_1_2_field/` on both repositories |
|
| 10 |
-
| Original training source, losses, sampler, curriculum and resume logic | `training/trackformer_1_2/source/v173/` |
|
| 11 |
-
| Portable launcher, configuration, tests and exact saved training history | `training/trackformer_1_2/` |
|
| 12 |
-
| Exact six derived dataset files, masks and original dataset manifest | Hugging Face: `training/trackformer_1_2/data/` |
|
| 13 |
-
| Original warm start and selected full training checkpoint with optimizer/RNG state | Hugging Face: `training/trackformer_1_2/checkpoints/` |
|
| 14 |
-
| Hashes and anonymous checksum-verifying asset downloader | `training/trackformer_1_2/assets.json` and `download_assets.py` |
|
| 15 |
-
| Benchmark protocols, measured results and figure provenance | `evaluation/`, `docs/` and `release_tools/` |
|
| 16 |
-
|
| 17 |
-
[GitHub source](https://github.com/yu314-coder/typhoon-predict) · [Hugging Face source and binary assets](https://huggingface.co/euler314/typhoon-predict) · [Training instructions](training/trackformer_1_2/README.md) · [Immutable publication links](training/trackformer_1_2/PUBLICATION.md) · [Contributing](CONTRIBUTING.md)
|
| 18 |
-
|
| 19 |
-
Large arrays and optimizer checkpoints are hosted on Hugging Face and linked from GitHub, rather than duplicated in ordinary Git history. Downloads are standard public HTTPS; the asset downloader uses Python's standard library, verifies exact hashes and requires no token. Training/inference still require the documented open-source runtime dependencies. There is no hosted-service dependency in the published training launcher.
|
| 20 |
-
|
| 21 |
-
## Upstream data terms
|
| 22 |
-
|
| 23 |
-
NOAA/IBTrACS and Copernicus/ERA5 data retain their own provenance, attribution and applicable upstream terms. The project's MIT license cannot remove third-party conditions. See [the source notices](training/trackformer_1_2/THIRD_PARTY_NOTICES.md) and [data provenance](docs/trackformer_1_2_training_data.md). The root `license: mit` metadata describes project software and model artifacts, not a blanket relicensing of all weather data.
|
| 24 |
-
|
| 25 |
-
## What is not claimed
|
| 26 |
-
|
| 27 |
-
The frozen derived arrays and their exact hashes are public, but the original raw IBTrACS snapshot/download ID was not recorded. A complete original runtime lockfile, original process argv and earlier warm-start training history have not been recovered. The archived recipe and warm-start checkpoint make the released v173 path runnable; bitwise full retraining across devices has not been demonstrated. The report selects epoch 4, while the saved aggregate validation-loss minimum is epoch 2. Previously inspected tests and examples are development evidence, not an untouched holdout.
|
| 28 |
-
|
| 29 |
-
Public availability is not a claim that every earlier 1.2 experiment or later rainfall experiment is this released model. No unavailable artifact, qualification result or score is invented to claim complete reproducibility. These remaining gaps stay visible so others can audit and improve the release.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
README.md
CHANGED
|
@@ -16,186 +16,62 @@ tags:
|
|
| 16 |
- reproducible-research
|
| 17 |
---
|
| 18 |
|
| 19 |
-
# TrackFormer 1.2 — pressure maps and
|
| 20 |
|
| 21 |
-
An
|
| 22 |
|
| 23 |
-
|
| 24 |
-
|
| 25 |
-
[Get the model](https://github.com/yu314-coder/typhoon-predict/releases/tag/trackformer-1.2) · [Hugging Face](https://huggingface.co/euler314/typhoon-predict) · [中文研究說明(與報告對照)](docs/research_report_zh.md) · [Live forecast](https://trackformer-weatherlab.rudin-euler-8253.chatgpt.site/live) · [Technical paper](paper/trackformer.pdf)
|
| 26 |
-
|
| 27 |
-
This page follows the Chinese research report **「訓練AI模型探討西北太平洋氣壓系統影響臺灣附近颱風的預測評估」** and its approved figure/text updates. It describes the released **1.2** model; later rainfall research is not a capability of these weights. The existing technical paper and older evaluation records remain separate, versioned references.
|
| 28 |
-
|
| 29 |
-
## Forecasting the field, not just the route
|
| 30 |
-
|
| 31 |
-
The model connects large-scale weather with the typhoon's pressure core:
|
| 32 |
-
|
| 33 |
-
- **Pressure maps through five days.** Basin-wide conditions and the moving core evolve at every forecast step.
|
| 34 |
-
- **Coupled track and intensity.** The predicted pressure core determines the centre and central-pressure estimate.
|
| 35 |
-
- **Causal inputs.** Nine historical weather analyses end at the issue time; no future observations or official forecast routes are used as prediction inputs.
|
| 36 |
-
- **Inspectable outputs.** Physical pressure grids, route points, timestamps and provenance are available alongside the visual forecasts.
|
| 37 |
-
|
| 38 |
-
This is a **research release**, not an operational warning service. Follow official agencies for safety-critical decisions.
|
| 39 |
|
| 40 |
## See the forecast: Mangkhut (2018)
|
| 41 |
|
| 42 |
[](https://huggingface.co/euler314/typhoon-predict/resolve/87a6e366b42bb4cc0edc95d2c50c55fca21a2c93/docs/trackformer_1_2_mangkhut.mp4)
|
| 43 |
|
| 44 |
-
[Watch
|
| 45 |
-
|
| 46 |
-
GitHub plays the lightweight animated GIF inline; click it for the full-quality MP4. Hugging Face provides a native video player. The GIF preserves all twenty forecast states, one second each, and loops without adding a frozen ending.
|
| 47 |
-
|
| 48 |
-
The animation shows twenty actual six-hour forecasts through +120 h. The Western Pacific overview and close-up use the same geographic mean of the model's basin and moving-core pressure fields. Blue denotes low pressure and red high pressure, with 2 hPa isobars and 4 hPa labels. Forecast and observed routes share the same coordinates and valid times—neither is shifted to make them overlap.
|
| 49 |
-
|
| 50 |
-
This selected example illustrates the model, not typical skill. Observations are verification only. The 50 members are distinct seeded perturbations of **past and issue-time inputs**, not separately trained networks. Member fields are registered geographically before averaging; the mean centre need not locate the minimum of the mean map. Display interpolation does not add resolution to the approximately 20 km core.
|
| 51 |
-
|
| 52 |
-
[Pressure arrays](evaluation/release_data/pressure_core/mangkhut/common-pressure.npz) · [Field verification](evaluation/release_data/pressure_core/mangkhut/verification.json) · [Playback verification](evaluation/release_data/mangkhut_playback_verification.json) · [More examples and diagnostics](docs/showcase_archive.md)
|
| 53 |
|
| 54 |
## Results in the research report
|
| 55 |
|
| 56 |
-
|
| 57 |
-
|
| 58 |
-
**DeepMind checkpoint: `WeatherNextCyclones_Mini_<2024` · trained through 2023 · native 1° · one member · WeatherNext software v0.3.0.** This is the official **Mini, not the full-sized WeatherNext model**. All reference forecasts completed on an **NVIDIA GeForce RTX 3070 using CUDA**. [Model identity and protocol](docs/deepmind_daily_benchmark.md) · [Completion receipt](evaluation/deepmind_daily/verification.json).
|
| 59 |
-
|
| 60 |
-

|
| 61 |
-
|
| 62 |
-
The bars show exact **24, 48, 72, 96 and 120 h** leads, not averages over each day. Track uses 270 storms; pressure uses 39–40 storms per displayed lead, with the counts printed in the figure. The table below averages all available six-hour leads using the common scoring procedure.
|
| 63 |
-
|
| 64 |
-
| Development metric · lower is better | TrackFormer 1.2 · mean of 50 | DeepMind Mini <2024 · one member | Shared coverage |
|
| 65 |
-
| --- | ---: | ---: | --- |
|
| 66 |
-
| Mean track error, +6 to +120 h | **471.20 km** | 498.37 km | 1,473 starts / 270 storms |
|
| 67 |
-
| Day 5 track error, +120 h | **1031.63 km** | 1031.91 km | 1,473 starts / 270 storms |
|
| 68 |
-
| Six-hour track-direction error | **34.96°** | 39.76° | 1,473 starts / 270 storms |
|
| 69 |
-
| Central-pressure MAE · JMA | 12.84 hPa | **10.20 hPa** | 134 starts / 40 storms / 2,637 valid lead records |
|
| 70 |
-
|
| 71 |
-
**Pressure coverage:** Of the 270 storms used for track, 40 meet the frozen common pressure-scoring conditions, giving 134 starts and **2,637 shared valid forecast-time records**. Each model is compared separately with the Japan Meteorological Agency records at the same valid times. This does **not** mean the remaining 230 storms have no pressure observations: the common subset inherits the original pipeline's issue-time intensity eligibility.
|
| 72 |
-
|
| 73 |
-
**Equal-storm scoring:** For each start, average the valid forecast-time errors; average starts within each storm; then average storms with equal weight. Missing labels and unsupported outputs are excluded, never zero-scored. Central-pressure MAE is not whole-map pressure error.
|
| 74 |
-
|
| 75 |
-
On this mixed historical/recent development cohort, 1.2 has lower mean position and heading errors; Mini has lower central-pressure MAE. **The recent-only track results favour Mini**: for strict issue years after 2024 (2025–2026), mean error is **258.7 km** for Mini versus **469.4 km** for 1.2, on 61 starts / 18 storms. Calendar 2024 is excluded from that subset. The combined cohort contains 230 storms from 1980–1999 and 40 recent storms; historical dates overlap Mini's training years. Different inputs and member counts make this a development comparison, **not a controlled architecture ablation, equal-compute test or certified untouched holdout**.
|
| 76 |
-
|
| 77 |
-
[Full saved scores and common support](evaluation/deepmind_daily/benchmark.json) · [Period breakdowns](evaluation/deepmind_daily/period_comparison.json) · [Independent publication audit](evaluation/deepmind_daily/publication_audit.json) · [Daily scoring protocol](docs/daily_storm_benchmark.md) · [Evaluation details and older comparisons](docs/trackformer_1_2_evaluation.md)
|
| 78 |
-
|
| 79 |
-
## Three report examples
|
| 80 |
-
|
| 81 |
-
The report compares predicted and reference pressure at **+48 h**, and routes and central pressure at **all twenty +6 to +120 h leads**. All maps retain the same valid time, coordinates and pressure scale; no forecast low is moved onto an observed route. Reference pressure is **NCEP analysis**, not a station-observation map. The report panels use a common **2.5° comparison grid**; smoother drawing does not create finer information.
|
| 82 |
-
|
| 83 |
-
| Example | Forecast issue (UTC) | Report figures |
|
| 84 |
-
| --- | --- | --- |
|
| 85 |
-
| Fung-wong / 鳳凰 (2025) | 2025-11-07 00:00 | [Pressure](docs/report_figures/fung_wong_pressure_comparison.png) · [Track / central pressure](docs/report_figures/fung_wong_track_pressure.png) · [Observed R34](docs/report_figures/fung_wong_observed_r34.png) |
|
| 86 |
-
| Mangkhut / 山竹 (2018) | 2018-09-11 00:00 | [Pressure](docs/report_figures/mangkhut_pressure_comparison.png) · [Track / central pressure](docs/report_figures/mangkhut_track_pressure.png) · [Observed R34](docs/report_figures/mangkhut_observed_r34.png) |
|
| 87 |
-
| Jebi / 燕子 (2018) | 2018-08-31 00:00 | [Pressure](docs/report_figures/jebi_pressure_comparison.png) · [Track / central pressure](docs/report_figures/jebi_track_pressure.png) · [Observed R34](docs/report_figures/jebi_observed_r34.png) |
|
| 88 |
-
|
| 89 |
-
All three use the same released checkpoint and 50-input-member policy. Fung-wong and Mangkhut use saved Metal forecasts; Jebi uses CPU forecasts from the same weights. Mangkhut and Jebi lie within the fitting years and are retrospective demonstrations, not unseen-case accuracy evidence. **The R34 panels are IBTrACS observed 34-knot quadrant-radius records, not model radius predictions or full wind maps.** Missing radii are left missing. There is **no native wind-radius forecast head** in 1.2.
|
| 90 |
-
|
| 91 |
-
[All report panels with captions](docs/research_report_zh.md#三個颱風展示) · [Figure hashes and sources](docs/report_figures/provenance.json) · [Earlier films and diagnostic archive](docs/showcase_archive.md)
|
| 92 |
-
|
| 93 |
-
## Training data and year cutoffs
|
| 94 |
-
|
| 95 |
-
**Released Trackformer 1.2 was fitted on 2000–2021 weather and typhoon data.** Validation used 2022–2023 for checkpoint selection; the original test partition used 2024–2025. **2026 live or historical forecasts are inference with frozen 1.2 weights, not training through 2026.**
|
| 96 |
-
|
| 97 |
-
| Data used by 1.2 | Source and variables | Years used for fitting |
|
| 98 |
-
| --- | --- | --- |
|
| 99 |
-
| Typhoon observations and labels | [NOAA IBTrACS](https://www.ncei.noaa.gov/products/international-best-track-archive)-derived storm positions, recent motion, available maximum wind and central pressure, with validity masks | 2000–2021 · Western Pacific-domain windows |
|
| 100 |
-
| Large-scale weather | [NOAA PSL NCEP/NCAR Reanalysis 1](https://psl.noaa.gov/data/gridded/data.ncep.reanalysis.html): sea-level pressure, 500 hPa height, and east/north winds at 850, 500 and 200 hPa; six-hour, 2.5° basin grid | 2000–2021 |
|
| 101 |
-
| Native regional pressure detail | [ARCO-ERA5](https://github.com/google-research/arco-era5) mean sea-level pressure at 0.25°, fixed issue-centred patches when available | 2000–2021 · 800 fitting windows |
|
| 102 |
-
| Static geography | ERA5 land fraction and surface geopotential converted to elevation, plus latitude/longitude | Static fields; not extra typhoon training years |
|
| 103 |
-
|
| 104 |
-
| Original whole-storm partition | Weather / typhoon years | Eligible six-hour windows | Native-pressure windows |
|
| 105 |
-
| --- | --- | ---: | ---: |
|
| 106 |
-
| Fitting · gradient updates and weather normalization | 2000–2021 | 13,949 | 800 |
|
| 107 |
-
| Validation · checkpoint selection, no gradient updates | 2022–2023 | 1,041 | 100 |
|
| 108 |
-
| Original test · evaluation only | 2024–2025 | 1,195 | 100 |
|
| 109 |
-
|
| 110 |
-
Each input contains nine analyses from −48 to 0 h. Later fields and best-track values are **training targets or evaluation labels**, never positive-lead prediction inputs. Entire storms stay in one partition and windows crossing a year-partition boundary are excluded. Weather normalization is fitted only on 2000–2021.
|
| 111 |
|
| 112 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 113 |
|
| 114 |
-
|
| 115 |
|
| 116 |
-
|
| 117 |
-
|
| 118 |
-
## How Trackformer 1.2 works
|
| 119 |
-
|
| 120 |
-

|
| 121 |
-
|
| 122 |
-
1. **Initialize from the past.** Nine six-hour analyses provide MSLP, 500 hPa height and winds at 850/500/200 hPa, ending at issue time. Static geography, the current observed centre, recent motion, current intensity and validity masks accompany the history. Native regional pressure detail is used when available; missing detail stays flagged. Current observations initialize the core, not future labels.
|
| 123 |
-
2. **Update the surrounding weather.** The model estimates changes in pressure and horizontal winds and carries information forward to the next six-hour step.
|
| 124 |
-
3. **Move and evolve the core.** A storm-centred pressure anomaly is transported relative to the environment and updated by learned tendencies. Basin pressure plus the tapered core anomaly forms the spatial forecast.
|
| 125 |
-
4. **Read out the storm.** A local low-pressure association identifies the centre within 300 km, and central pressure is sampled there. An auxiliary head provides a maximum-wind scalar, not a resolved surface-wind map.
|
| 126 |
-
5. **Roll forward.** Twenty autoregressive transitions produce +6, +12, …, +120 h forecasts. Training uses field, core, track, central-pressure, auxiliary-wind and masked change/displacement losses; future truth is a target, never an inference input.
|
| 127 |
-
|
| 128 |
-
| Released architecture | Configuration |
|
| 129 |
-
| --- | --- |
|
| 130 |
-
| Trainable parameters | 21,452,595 |
|
| 131 |
-
| Basin history | 9 × 8 × 25 × 33; −48 to 0 h |
|
| 132 |
-
| Moving core | 65 × 65; approximately 20 km spacing |
|
| 133 |
-
| Forecast steps | 20 × 6 hours |
|
| 134 |
-
|
| 135 |
-
Training increases the prediction horizon from one to two, three and five days, checking the full five-day forecast each epoch. The report uses the selected **epoch-4** checkpoint. [Model source and input contract](models/trackformer_1_2_field/README.md) · [Exact architecture and provenance](models/trackformer_1_2_field/manifest.json).
|
| 136 |
-
|
| 137 |
-
### What “50 forecasts averaged” means
|
| 138 |
-
|
| 139 |
-
The **same model weights** run 50 times with distinct seeded, small, smoothed zero-centred noise added only to normalized historical weather inputs. In the saved benchmark policy, basin noise has amplitude 0.025 with a 5-cell smoothing kernel; regional noise has amplitude 0.015 with a 9-cell kernel. These amplitudes are in normalized input units, not hPa or wind units. Issue-time storm position, motion and intensity, geography and validity masks stay fixed. Original observations and weights are not edited.
|
| 140 |
-
|
| 141 |
-
Routes and central pressures are averaged separately. Physical member pressure fields are placed on **common geographic coordinates before averaging**, rather than averaging unaligned moving-core array indices. This is a 50-member input-perturbation mean, **not 50 trained networks**. It is not proven optimal or automatically a calibrated uncertainty estimate; a full 270-storm clean-single-member versus 50-member accuracy comparison has not been established.
|
| 142 |
-
|
| 143 |
-
### Computers and software
|
| 144 |
-
|
| 145 |
-
- **Apple M4 Mac, 24 GB unified memory:** local model execution, data handling and plotting; PyTorch MPS uses the built-in GPU through Metal. Household-computer training and inference are the design goal, not a claim that every historical run used only this one machine.
|
| 146 |
-
- **NVIDIA GeForce RTX 3070 / CUDA:** verified execution of the completed DeepMind reference benchmark. This is hardware, not a software package.
|
| 147 |
-
- **Python / PyTorch**, NumPy, SciPy, xarray, netCDF4 and Matplotlib support modelling, data handling and figures. The Chinese report uses XeLaTeX / xeCJK.
|
| 148 |
-
|
| 149 |
-
## Open access and reuse
|
| 150 |
-
|
| 151 |
-
Code and project weights are **MIT licensed**, including modification, redistribution and commercial use. Source, training arrays and checkpoints are public without account approval or a download gate. [Openness inventory and remaining provenance gaps](OPENNESS.md) · [Contributing](CONTRIBUTING.md) · [Software citation](CITATION.cff). Upstream weather data retain their own terms and attribution.
|
| 152 |
-
|
| 153 |
-
## Complete training manual (PDF)
|
| 154 |
-
|
| 155 |
-
**[Download the single 106-page training and reproducibility PDF](docs/TrackFormer_1.2_Training_and_Reproducibility.pdf).** It contains data sources and processing, whole-storm partitions, normalization, tensor construction, architecture, initialization, all active losses, sampling, batches, truncated backpropagation, optimizer settings, curriculum, validation, checkpoint/resume behavior, recorded history, reproduction commands and limitations. A complete 40-file source/provenance appendix and an embedded exact-source ZIP are included; no separate Markdown pages are needed to read the method.
|
| 156 |
-
|
| 157 |
-
The checkpoint audit distinguishes facts from the archived recipe: the selected epoch-4 optimizer records **4,384 updates**, compatible with effective batch **16** under the published four-epoch sampler; the archived notebook recipe uses batch **10** and would produce 7,016 updates. Original argv/microbatch/worker settings were not recorded. The PDF explains that discrepancy, the epoch-2 minimum validation loss versus selected epoch 4, and the remaining raw-provenance/runtime gaps.
|
| 158 |
-
|
| 159 |
-
[Training source and asset index](training/trackformer_1_2/) · [Public exact corpus and optimizer checkpoints](https://huggingface.co/euler314/typhoon-predict/tree/main/training/trackformer_1_2). Project code/weights remain MIT; weather data retain upstream terms. This documentation publication does not rerun or modify any trainer.
|
| 160 |
-
|
| 161 |
-
## Get started
|
| 162 |
-
|
| 163 |
-
[Download weights and the complete pressure-map exporter](https://github.com/yu314-coder/typhoon-predict/releases/download/trackformer-1.2/trackformer_1_2_field_pressure_export_v2.tar.gz), or use this repository with the [Hugging Face weights](https://huggingface.co/euler314/typhoon-predict/resolve/main/models/trackformer_1_2_field/weights.pt). Put `weights.pt` in `models/trackformer_1_2_field/`. The inference weights and learned model equations are unchanged; the package includes the moving-core output and map renderer.
|
| 164 |
-
|
| 165 |
-
Prepare a normalized causal issue packet following the [input schema](models/trackformer_1_2_field/README.md), then run with PyTorch and NumPy:
|
| 166 |
|
| 167 |
```bash
|
|
|
|
| 168 |
python models/trackformer_1_2_field/predict.py causal_issue_packet.npz forecast.npz --device mps
|
| 169 |
```
|
| 170 |
|
| 171 |
-
Use `--device cuda`
|
| 172 |
|
| 173 |
-
|
| 174 |
|
| 175 |
-
|
| 176 |
-
python models/trackformer_1_2_field/plot_pressure.py forecast.npz pressure_24h.png --lead 24 --interval 2
|
| 177 |
-
```
|
| 178 |
-
|
| 179 |
-
Blue denotes low pressure and red high pressure. The renderer uses the actual saved moving field, coordinates and masks; it never draws a replacement vortex around a scalar pressure or observed track. See the [pressure-output schema](models/trackformer_1_2_field/README.md#render-a-pressure-map-directly).
|
| 180 |
-
|
| 181 |
-
The inference weight SHA-256 is `db49f36e85a3766defc4c172746897a1f783705d1ce8e6f9dfb8e87ae1d902cb`.
|
| 182 |
|
| 183 |
-
|
| 184 |
|
| 185 |
-
The
|
| 186 |
|
| 187 |
-
|
| 188 |
|
| 189 |
-
|
| 190 |
|
| 191 |
-
|
| 192 |
-
|
| 193 |
-
|
| 194 |
-
|
| 195 |
-
|
|
|
|
|
|
|
| 196 |
|
| 197 |
-
##
|
| 198 |
|
| 199 |
-
|
| 200 |
|
| 201 |
-
|
|
|
|
| 16 |
- reproducible-research
|
| 17 |
---
|
| 18 |
|
| 19 |
+
# TrackFormer 1.2 — pressure maps and typhoon forecasts
|
| 20 |
|
| 21 |
+
An open-source weather-research model designed to run on an ordinary home computer. It evolves Western Pacific pressure and winds every six hours, then reads the typhoon track and central pressure from the same moving pressure core through +120 h.
|
| 22 |
|
| 23 |
+
**[Complete training manual · PDF](docs/TrackFormer_1.2_Training_and_Reproducibility.pdf)** · [Model input/output guide](models/trackformer_1_2_field/README.md) · [Reference and protocols](docs/REFERENCE.md) · [Live / History](https://trackformer-weatherlab.rudin-euler-8253.chatgpt.site/live)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 24 |
|
| 25 |
## See the forecast: Mangkhut (2018)
|
| 26 |
|
| 27 |
[](https://huggingface.co/euler314/typhoon-predict/resolve/87a6e366b42bb4cc0edc95d2c50c55fca21a2c93/docs/trackformer_1_2_mangkhut.mp4)
|
| 28 |
|
| 29 |
+
[Watch MP4](https://huggingface.co/euler314/typhoon-predict/resolve/87a6e366b42bb4cc0edc95d2c50c55fca21a2c93/docs/trackformer_1_2_mangkhut.mp4) · 11 September 2018, 00 UTC · 50-member input-perturbation mean. This selected retrospective example illustrates the model, not typical or unseen-case skill.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 30 |
|
| 31 |
## Results in the research report
|
| 32 |
|
| 33 |
+
Completed **1,473 daily starts / 270 storms**, all twenty +6 to +120 h leads. Comparator: **DeepMind WeatherNext Cyclones Mini <2024**, native 1°, one member, software v0.3.0. TrackFormer uses the same weights with 50 perturbed historical-input runs. This is a development comparison, not equal-compute or certified untouched-holdout evaluation.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 34 |
|
| 35 |
+
| Metric · lower is better | TrackFormer 1.2 | DeepMind Mini |
|
| 36 |
+
| --- | ---: | ---: |
|
| 37 |
+
| Mean track error, +6 to +120 h | 471.20 km | 498.37 km |
|
| 38 |
+
| Day 5 track error, +120 h | 1031.63 km | 1031.91 km |
|
| 39 |
+
| Six-hour track-direction error | 34.96° | 39.76° |
|
| 40 |
+
| Central-pressure MAE · JMA | 12.84 hPa | 10.20 hPa |
|
| 41 |
|
| 42 |
+
Pressure uses **134 starts / 40 storms / 2,637 common valid records**, a subset of the same 270 storms. This does not imply the other storms lack observations. Errors are averaged within starts, then within storms, then equally across storms. For the strict 2025–2026 subset, **Mini is better on track: 258.7 km versus 469.4 km**, 61 starts / 18 storms. Historical overlap and unequal inputs/member counts limit interpretation. [Full results and period breakdowns](docs/REFERENCE.md#docs-deepmind-daily-benchmark-md) · [Machine-readable scores](evaluation/deepmind_daily/benchmark.json).
|
| 43 |
|
| 44 |
+
## Run a forecast
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 45 |
|
| 46 |
```bash
|
| 47 |
+
python -m pip install -r requirements.txt
|
| 48 |
python models/trackformer_1_2_field/predict.py causal_issue_packet.npz forecast.npz --device mps
|
| 49 |
```
|
| 50 |
|
| 51 |
+
Download [weights.pt](https://huggingface.co/euler314/typhoon-predict/resolve/main/models/trackformer_1_2_field/weights.pt) into `models/trackformer_1_2_field/`. Use `--device cuda` or `--device cpu` as appropriate. The caller prepares the causal issue packet using the [input/output guide](models/trackformer_1_2_field/README.md). One clean run is distinct from a 50-member mean. Add `--pressure-map pressure_120h.png --map-lead 120` for an exported pressure image.
|
| 52 |
|
| 53 |
+
Inference-weight SHA-256: `db49f36e85a3766defc4c172746897a1f783705d1ce8e6f9dfb8e87ae1d902cb`.
|
| 54 |
|
| 55 |
+
## Train and inspect
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 56 |
|
| 57 |
+
The **[single 106-page PDF](docs/TrackFormer_1.2_Training_and_Reproducibility.pdf)** contains the complete available data process, architecture, active losses, sampler, optimizer, curriculum, validation, checkpoint/resume behavior, records and reproduction commands, plus 40 original text-source/provenance files and an embedded exact-source ZIP.
|
| 58 |
|
| 59 |
+
The original training code remains in `training/trackformer_1_2/`. Exact arrays, masks, warm start and full selected optimizer checkpoint are [public on Hugging Face](https://huggingface.co/euler314/typhoon-predict/tree/main/training/trackformer_1_2); `download_assets.py` verifies hashes without tokens or Drive permissions. Fitting years are **2000–2021**, validation **2022–2023**, original test **2024–2025**: 13,949 / 1,041 / 1,195 windows, with 800 / 100 / 100 native-pressure subsets. Nine analyses span −48 to 0 h. Future fields and observations are targets only.
|
| 60 |
|
| 61 |
+
The selected report checkpoint is epoch 4; epoch 2 has the minimum saved validation loss. Its recorded 4,384 optimizer updates are compatible with effective batch 16 under the published four-epoch sampler, while the archived notebook recipe uses batch 10. The PDF distinguishes that inference from the missing original argv/runtime and raw IBTrACS snapshot ID. No bitwise full retraining is claimed.
|
| 62 |
|
| 63 |
+
## Repository guide
|
| 64 |
|
| 65 |
+
| Location | Contents |
|
| 66 |
+
| --- | --- |
|
| 67 |
+
| `models/trackformer_1_2_field/` | Inference architecture, loader and pressure-map tools |
|
| 68 |
+
| `training/trackformer_1_2/` | Original trainer, data utilities, configuration, asset hashes and tests |
|
| 69 |
+
| `evaluation/` | Saved scores, support, audits and provenance |
|
| 70 |
+
| `release_tools/` | Export, benchmark, plotting and publication utilities |
|
| 71 |
+
| `docs/` | Training PDF, one reference document, figures and one archived documentation snapshot |
|
| 72 |
|
| 73 |
+
## Reuse, contribution and limits
|
| 74 |
|
| 75 |
+
Project code and weights use [MIT](LICENSE), including commercial use, modification and redistribution. Downloads are public and ungated. Upstream weather data retain their own attribution/terms. [Licensing and openness](docs/REFERENCE.md#openness-md) · [Contribution guide](docs/REFERENCE.md#contributing-md) · [Citation](CITATION.cff).
|
| 76 |
|
| 77 |
+
This is a research release, not an operational warning service. The approximately 20 km core is learned reconstruction, not new native observations. Released 1.2 has no rainfall head or humidity input; auxiliary wind and pressure-derived radius diagnostics are not validated agency-equivalent surface-wind forecasts. Follow official warnings for safety-critical decisions.
|
RELEASE_NOTES_TRACKFORMER_1_2.md
DELETED
|
@@ -1,45 +0,0 @@
|
|
| 1 |
-
# Trackformer 1.2 — pressure-field research model
|
| 2 |
-
|
| 3 |
-
**Documentation clarification (18 October 2026):** the current [English model page](README.md) and [Chinese research explanation](docs/research_report_zh.md) follow the research report's released-1.2 scope, household-computer goal, day-1–5 chart and three typhoon examples. Route uses 270 storms / 1,473 starts; the JMA pressure comparison uses 40 of those storms / 134 starts / 2,637 common valid records. The report's R34 panels are observations, not model radius forecasts, and Taiwan rainfall remains separate ongoing research. This updates documentation and existing figure copies only; weights, neural source and benchmark arrays are unchanged. Earlier release history below is retained as an archive.
|
| 4 |
-
|
| 5 |
-
**[Download weights + complete inference code](https://github.com/yu314-coder/typhoon-predict/releases/download/trackformer-1.2/trackformer_1_2_field_pressure_export_v2.tar.gz)** · **[Hugging Face](https://huggingface.co/euler314/typhoon-predict)** · **[Technical paper](https://github.com/yu314-coder/typhoon-predict/blob/main/paper/trackformer.pdf)**
|
| 6 |
-
|
| 7 |
-
Trackformer 1.2 forecasts Western Pacific storm tracks, central pressure and evolving sea-level-pressure fields through +120 hours. A multiscale environmental-attention network conditions the moving pressure core; track and central pressure are read from that evolving field.
|
| 8 |
-
|
| 9 |
-
## Direct detailed pressure-map output
|
| 10 |
-
|
| 11 |
-
The current package exports all twenty six-hour leads with:
|
| 12 |
-
|
| 13 |
-
- Whole-WP basin pressure, the original fixed regional composite, and the actual **65×65 moving-core pressure field in physical hPa**.
|
| 14 |
-
- Basin, regional and per-lead core latitude/longitude grids, coverage masks, issue time and exact valid times.
|
| 15 |
-
- Original track/central-pressure outputs and an explicit **one-member** count.
|
| 16 |
-
- Frozen weight/checkpoint identity and an optional PNG renderer: blue low pressure, red high pressure, labelled isobars.
|
| 17 |
-
|
| 18 |
-
```bash
|
| 19 |
-
python models/trackformer_1_2_field/predict.py causal_issue_packet.npz forecast.npz --device cpu --pressure-map pressure_120h.png --map-lead 120
|
| 20 |
-
python models/trackformer_1_2_field/plot_pressure.py forecast.npz pressure_24h.png --lead 24 --interval 2
|
| 21 |
-
```
|
| 22 |
-
|
| 23 |
-
NumPy and PyTorch are required for inference; Matplotlib is optional for images. Use `--device mps` or `--device cuda` on a compatible system. Prepare the nine-analysis causal input packet using the [published schema](https://github.com/yu314-coder/typhoon-predict/blob/main/models/trackformer_1_2_field/README.md); this package does not fetch live weather automatically.
|
| 24 |
-
|
| 25 |
-
The core's 20-km spacing is a learned computational reconstruction, not new native observations. Invalid coverage remains masked. Fields are not shifted onto a route, and central pressure is not inserted as a display vortex. Ensemble members require geographic registration before physical-field averaging; this command is not the separate 50-member benchmark policy.
|
| 26 |
-
|
| 27 |
-
**The learned modules, inference weights and forecast equations are unchanged.** The added export captures fields already produced by the released model. The original September 29 package remains available separately; use the pressure-export-v2 archive for the complete default output.
|
| 28 |
-
|
| 29 |
-
Inference weight SHA-256: `db49f36e85a3766defc4c172746897a1f783705d1ce8e6f9dfb8e87ae1d902cb`.
|
| 30 |
-
|
| 31 |
-
## Matched development results
|
| 32 |
-
|
| 33 |
-
| Measure | 1.1 | 1.2 mean of 50 | DeepMind Mini, one member | Matched coverage |
|
| 34 |
-
| --- | ---: | ---: | ---: | --- |
|
| 35 |
-
| Mean track error | 798.4 km | 471.2 km | 498.4 km | 1,473 daily starts / 270 storms |
|
| 36 |
-
| Direction error | 51.58° | 34.96° | 39.76° | Same daily starts |
|
| 37 |
-
| Central-pressure MAE against JMA | 13.53 hPa | 12.84 hPa | 10.20 hPa | 134 common starts / 40 storms |
|
| 38 |
-
|
| 39 |
-
Storms receive equal weight after valid leads and daily starts are averaged. The mean pressure reduction is small and its paired whole-storm uncertainty includes no improvement. These repeatedly inspected results are development evidence, not a certified untouched holdout. [Verified common-support metrics](https://github.com/yu314-coder/typhoon-predict/blob/main/evaluation/released_daily/released_daily_benchmark.json).
|
| 40 |
-
|
| 41 |
-
**The DeepMind comparison is complete and verified:** the official **WeatherNext Cyclones Mini `<2024`** checkpoint (1° resolution, trained through 2023), using **WeatherNext software v0.3.0**, completed all 1,473 daily starts / 270 storms on an RTX 3070 using CUDA. Mini's recent-only mean track error is **288.5 km**, better than 1.2's **477.2 km**. Historical fitting-year overlap, different weather inputs and unequal member counts prevent a general superiority claim. [Completed results and model identity](https://github.com/yu314-coder/typhoon-predict/blob/main/docs/deepmind_daily_benchmark.md) · [CUDA completion receipt](https://github.com/yu314-coder/typhoon-predict/blob/main/evaluation/deepmind_daily/verification.json) · [Independent publication audit](https://github.com/yu314-coder/typhoon-predict/blob/main/evaluation/deepmind_daily/publication_audit.json).
|
| 42 |
-
|
| 43 |
-
The [model announcement](https://github.com/yu314-coder/typhoon-predict) includes the selected Mangkhut pressure-map animation and architecture. Selected examples are not representative skill. Auxiliary wind and pressure-derived radius diagnostics remain unvalidated; there is no native wind-radius forecast head in 1.2.
|
| 44 |
-
|
| 45 |
-
This is a research model, not an operational warning service or a safety-critical forecast. Trackformer 1.1 remains a separate release. No historical archive, media, benchmark predictions or training run is modified by this exporter update.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
docs/REFERENCE.md
ADDED
|
@@ -0,0 +1,1068 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# TrackFormer reference
|
| 2 |
+
Detailed protocols, source notices and historical release explanations are consolidated here. For the complete training method, read the [single training PDF](TrackFormer_1.2_Training_and_Reproducibility.pdf). Executable code, arrays, masks and measured scores retain their existing paths.
|
| 3 |
+
The source documents below were preserved before consolidation. Their former file paths identify provenance; time-sensitive operational counts are snapshots, not fresh status checks. Source-level training/design notes are reproduced in the PDF. The exact pre-cleanup Markdown files are retained in [one ZIP](archive/documentation-before-consolidation-20261030.zip) and immutable Git history.
|
| 4 |
+
For the current comparison use the main README and saved benchmark JSON. Individual historical sections can describe different cohorts or revisions; consolidation does not relabel those values as a new matched result.
|
| 5 |
+
## Contents
|
| 6 |
+
- [CONTRIBUTING.md](#contributing-md)
|
| 7 |
+
- [OPENNESS.md](#openness-md)
|
| 8 |
+
- [RELEASE_NOTES_TRACKFORMER_1_2.md](#release-notes-trackformer-1-2-md)
|
| 9 |
+
- [docs/automatic_forecasts.md](#docs-automatic-forecasts-md)
|
| 10 |
+
- [docs/daily_storm_benchmark.md](#docs-daily-storm-benchmark-md)
|
| 11 |
+
- [docs/deepmind_daily_benchmark.md](#docs-deepmind-daily-benchmark-md)
|
| 12 |
+
- [docs/intensity_benchmark.md](#docs-intensity-benchmark-md)
|
| 13 |
+
- [docs/repository_layout.md](#docs-repository-layout-md)
|
| 14 |
+
- [docs/research_report_zh.md](#docs-research-report-zh-md)
|
| 15 |
+
- [docs/showcase_archive.md](#docs-showcase-archive-md)
|
| 16 |
+
- [docs/trackformer_1_2_evaluation.md](#docs-trackformer-1-2-evaluation-md)
|
| 17 |
+
- [evaluation/README.md](#evaluation-readme-md)
|
| 18 |
+
- [models/trackformer_1_2_field/WIND_ESTIMATION.md](#models-trackformer-1-2-field-wind-estimation-md)
|
| 19 |
+
- [release_tools/STORM_STRUCTURE_DIAGNOSTICS.md](#release-tools-storm-structure-diagnostics-md)
|
| 20 |
+
- [training/trackformer_1_2/THIRD_PARTY_NOTICES.md](#training-trackformer-1-2-third-party-notices-md)
|
| 21 |
+
|
| 22 |
+
|
| 23 |
+
<a id="contributing-md"></a>
|
| 24 |
+
|
| 25 |
+
## CONTRIBUTING.md
|
| 26 |
+
|
| 27 |
+
# Contributing to TrackFormer
|
| 28 |
+
|
| 29 |
+
Anyone may inspect, fork, modify and retrain the public release under its MIT license. No contributor agreement or account approval is required to download or use the code and weights. Submitting a GitHub pull request requires your own GitHub account; local use does not.
|
| 30 |
+
|
| 31 |
+
## Reproduce first
|
| 32 |
+
|
| 33 |
+
1. Follow `training/trackformer_1_2/README.md` and pin the asset revision from `PUBLICATION.md`.
|
| 34 |
+
2. Verify dataset/checkpoint SHA-256 values before changing anything. Keep original weights and source artifacts unchanged and write experiments into new output directories.
|
| 35 |
+
3. Use `--preflight` for the archive/architecture check and `test_training_cpu.py` for the portable CPU loss/backward check. These reconstruct a writable local mmap, so use your own data copy rather than an active trainer's directory.
|
| 36 |
+
4. For pressure export changes, use the existing `release_tools/test_pressure_field_export.py` checks as applicable. There is no requirement to run the full multi-hour training job for a documentation correction.
|
| 37 |
+
|
| 38 |
+
## Share a change
|
| 39 |
+
|
| 40 |
+
Open an issue or pull request at https://github.com/yu314-coder/typhoon-predict . Explain the problem, changed behavior, exact model/data revision and relevant checks. Document new dependencies and retain third-party notices. Never commit credentials, private data or unrelated large caches. New project contributions should use the existing MIT license; third-party material must keep its own license and attribution.
|
| 41 |
+
|
| 42 |
+
For scientific claims, preserve issue-time causal inputs and whole-storm partitions; keep future observations as labels only. Compare on identical cases, grids, times and masks. Report misses and false alarms as well as mean errors, and identify previously reused development data. Do not infer forecast skill from a smoother map or training loss. Label changed architectures and weights as new candidates rather than silently replacing released 1.2.
|
| 43 |
+
|
| 44 |
+
The MIT license permits broad reuse; scientific review criteria apply to what this repository accepts or claims, not to your permission to fork or experiment independently.
|
| 45 |
+
|
| 46 |
+
|
| 47 |
+
<a id="openness-md"></a>
|
| 48 |
+
|
| 49 |
+
## OPENNESS.md
|
| 50 |
+
|
| 51 |
+
# Open access and reuse
|
| 52 |
+
|
| 53 |
+
TrackFormer 1.2 is publicly accessible without account approval, a download gate, a paid API or Google Drive permissions. Project code and project model weights/checkpoints are distributed under the [MIT license](../LICENSE). You may use, study, modify, retrain, redistribute, sublicense and sell copies, including commercial use, subject to retaining the required copyright and permission notice. No research-only or non-commercial restriction is added by this project. The research/safety cautions describe capability limits; they do not replace or narrow the MIT license.
|
| 54 |
+
|
| 55 |
+
## What is public
|
| 56 |
+
|
| 57 |
+
| Component | Public location |
|
| 58 |
+
| --- | --- |
|
| 59 |
+
| Architecture, inference and physical pressure exporter | `models/trackformer_1_2_field/` on both repositories |
|
| 60 |
+
| Original training source, losses, sampler, curriculum and resume logic | `training/trackformer_1_2/source/v173/` |
|
| 61 |
+
| Portable launcher, configuration, tests and exact saved training history | `training/trackformer_1_2/` |
|
| 62 |
+
| Exact six derived dataset files, masks and original dataset manifest | Hugging Face: `training/trackformer_1_2/data/` |
|
| 63 |
+
| Original warm start and selected full training checkpoint with optimizer/RNG state | Hugging Face: `training/trackformer_1_2/checkpoints/` |
|
| 64 |
+
| Hashes and anonymous checksum-verifying asset downloader | `training/trackformer_1_2/assets.json` and `download_assets.py` |
|
| 65 |
+
| Benchmark protocols, measured results and figure provenance | `evaluation/`, `docs/` and `release_tools/` |
|
| 66 |
+
|
| 67 |
+
[GitHub source](https://github.com/yu314-coder/typhoon-predict) · [Hugging Face source and binary assets](https://huggingface.co/euler314/typhoon-predict) · [Training instructions](TrackFormer_1.2_Training_and_Reproducibility.pdf) · [Immutable publication links](TrackFormer_1.2_Training_and_Reproducibility.pdf) · [Contributing](#contributing-md)
|
| 68 |
+
|
| 69 |
+
Large arrays and optimizer checkpoints are hosted on Hugging Face and linked from GitHub, rather than duplicated in ordinary Git history. Downloads are standard public HTTPS; the asset downloader uses Python's standard library, verifies exact hashes and requires no token. Training/inference still require the documented open-source runtime dependencies. There is no hosted-service dependency in the published training launcher.
|
| 70 |
+
|
| 71 |
+
## Upstream data terms
|
| 72 |
+
|
| 73 |
+
NOAA/IBTrACS and Copernicus/ERA5 data retain their own provenance, attribution and applicable upstream terms. The project's MIT license cannot remove third-party conditions. See [the source notices](TrackFormer_1.2_Training_and_Reproducibility.pdf) and [data provenance](TrackFormer_1.2_Training_and_Reproducibility.pdf). The root `license: mit` metadata describes project software and model artifacts, not a blanket relicensing of all weather data.
|
| 74 |
+
|
| 75 |
+
## What is not claimed
|
| 76 |
+
|
| 77 |
+
The frozen derived arrays and their exact hashes are public, but the original raw IBTrACS snapshot/download ID was not recorded. A complete original runtime lockfile, original process argv and earlier warm-start training history have not been recovered. The archived recipe and warm-start checkpoint make the released v173 path runnable; bitwise full retraining across devices has not been demonstrated. The report selects epoch 4, while the saved aggregate validation-loss minimum is epoch 2. Previously inspected tests and examples are development evidence, not an untouched holdout.
|
| 78 |
+
|
| 79 |
+
Public availability is not a claim that every earlier 1.2 experiment or later rainfall experiment is this released model. No unavailable artifact, qualification result or score is invented to claim complete reproducibility. These remaining gaps stay visible so others can audit and improve the release.
|
| 80 |
+
|
| 81 |
+
|
| 82 |
+
<a id="release-notes-trackformer-1-2-md"></a>
|
| 83 |
+
|
| 84 |
+
## RELEASE_NOTES_TRACKFORMER_1_2.md
|
| 85 |
+
|
| 86 |
+
# Trackformer 1.2 — pressure-field research model
|
| 87 |
+
|
| 88 |
+
**Documentation clarification (18 October 2026):** the current [English model page](../README.md) and [Chinese research explanation](#docs-research-report-zh-md) follow the research report's released-1.2 scope, household-computer goal, day-1–5 chart and three typhoon examples. Route uses 270 storms / 1,473 starts; the JMA pressure comparison uses 40 of those storms / 134 starts / 2,637 common valid records. The report's R34 panels are observations, not model radius forecasts, and Taiwan rainfall remains separate ongoing research. This updates documentation and existing figure copies only; weights, neural source and benchmark arrays are unchanged. Earlier release history below is retained as an archive.
|
| 89 |
+
|
| 90 |
+
**[Download weights + complete inference code](https://github.com/yu314-coder/typhoon-predict/releases/download/trackformer-1.2/trackformer_1_2_field_pressure_export_v2.tar.gz)** · **[Hugging Face](https://huggingface.co/euler314/typhoon-predict)** · **[Technical paper](https://github.com/yu314-coder/typhoon-predict/blob/main/paper/trackformer.pdf)**
|
| 91 |
+
|
| 92 |
+
Trackformer 1.2 forecasts Western Pacific storm tracks, central pressure and evolving sea-level-pressure fields through +120 hours. A multiscale environmental-attention network conditions the moving pressure core; track and central pressure are read from that evolving field.
|
| 93 |
+
|
| 94 |
+
## Direct detailed pressure-map output
|
| 95 |
+
|
| 96 |
+
The current package exports all twenty six-hour leads with:
|
| 97 |
+
|
| 98 |
+
- Whole-WP basin pressure, the original fixed regional composite, and the actual **65×65 moving-core pressure field in physical hPa**.
|
| 99 |
+
- Basin, regional and per-lead core latitude/longitude grids, coverage masks, issue time and exact valid times.
|
| 100 |
+
- Original track/central-pressure outputs and an explicit **one-member** count.
|
| 101 |
+
- Frozen weight/checkpoint identity and an optional PNG renderer: blue low pressure, red high pressure, labelled isobars.
|
| 102 |
+
|
| 103 |
+
```bash
|
| 104 |
+
python models/trackformer_1_2_field/predict.py causal_issue_packet.npz forecast.npz --device cpu --pressure-map pressure_120h.png --map-lead 120
|
| 105 |
+
python models/trackformer_1_2_field/plot_pressure.py forecast.npz pressure_24h.png --lead 24 --interval 2
|
| 106 |
+
```
|
| 107 |
+
|
| 108 |
+
NumPy and PyTorch are required for inference; Matplotlib is optional for images. Use `--device mps` or `--device cuda` on a compatible system. Prepare the nine-analysis causal input packet using the [published schema](https://github.com/yu314-coder/typhoon-predict/blob/main/models/trackformer_1_2_field/README.md); this package does not fetch live weather automatically.
|
| 109 |
+
|
| 110 |
+
The core's 20-km spacing is a learned computational reconstruction, not new native observations. Invalid coverage remains masked. Fields are not shifted onto a route, and central pressure is not inserted as a display vortex. Ensemble members require geographic registration before physical-field averaging; this command is not the separate 50-member benchmark policy.
|
| 111 |
+
|
| 112 |
+
**The learned modules, inference weights and forecast equations are unchanged.** The added export captures fields already produced by the released model. The original September 29 package remains available separately; use the pressure-export-v2 archive for the complete default output.
|
| 113 |
+
|
| 114 |
+
Inference weight SHA-256: `db49f36e85a3766defc4c172746897a1f783705d1ce8e6f9dfb8e87ae1d902cb`.
|
| 115 |
+
|
| 116 |
+
## Matched development results
|
| 117 |
+
|
| 118 |
+
| Measure | 1.1 | 1.2 mean of 50 | DeepMind Mini, one member | Matched coverage |
|
| 119 |
+
| --- | ---: | ---: | ---: | --- |
|
| 120 |
+
| Mean track error | 798.4 km | 471.2 km | 498.4 km | 1,473 daily starts / 270 storms |
|
| 121 |
+
| Direction error | 51.58° | 34.96° | 39.76° | Same daily starts |
|
| 122 |
+
| Central-pressure MAE against JMA | 13.53 hPa | 12.84 hPa | 10.20 hPa | 134 common starts / 40 storms |
|
| 123 |
+
|
| 124 |
+
Storms receive equal weight after valid leads and daily starts are averaged. The mean pressure reduction is small and its paired whole-storm uncertainty includes no improvement. These repeatedly inspected results are development evidence, not a certified untouched holdout. [Verified common-support metrics](https://github.com/yu314-coder/typhoon-predict/blob/main/evaluation/released_daily/released_daily_benchmark.json).
|
| 125 |
+
|
| 126 |
+
**The DeepMind comparison is complete and verified:** the official **WeatherNext Cyclones Mini `<2024`** checkpoint (1° resolution, trained through 2023), using **WeatherNext software v0.3.0**, completed all 1,473 daily starts / 270 storms on an RTX 3070 using CUDA. Mini's recent-only mean track error is **288.5 km**, better than 1.2's **477.2 km**. Historical fitting-year overlap, different weather inputs and unequal member counts prevent a general superiority claim. [Completed results and model identity](#docs-deepmind-daily-benchmark-md) · [CUDA completion receipt](https://github.com/yu314-coder/typhoon-predict/blob/main/evaluation/deepmind_daily/verification.json) · [Independent publication audit](https://github.com/yu314-coder/typhoon-predict/blob/main/evaluation/deepmind_daily/publication_audit.json).
|
| 127 |
+
|
| 128 |
+
The [model announcement](https://github.com/yu314-coder/typhoon-predict) includes the selected Mangkhut pressure-map animation and architecture. Selected examples are not representative skill. Auxiliary wind and pressure-derived radius diagnostics remain unvalidated; there is no native wind-radius forecast head in 1.2.
|
| 129 |
+
|
| 130 |
+
This is a research model, not an operational warning service or a safety-critical forecast. Trackformer 1.1 remains a separate release. No historical archive, media, benchmark predictions or training run is modified by this exporter update.
|
| 131 |
+
|
| 132 |
+
|
| 133 |
+
<a id="docs-automatic-forecasts-md"></a>
|
| 134 |
+
|
| 135 |
+
## docs/automatic_forecasts.md
|
| 136 |
+
|
| 137 |
+
# Automatic 1.2 forecast archive
|
| 138 |
+
|
| 139 |
+
The existing repository runs the released Python checkpoint on **GitHub-hosted
|
| 140 |
+
CPU runners**, not on a visitor's browser or the owner's Mac. Every hour at
|
| 141 |
+
minute 17 (UTC and Asia/Taipei), the workflow checks all current in-domain
|
| 142 |
+
JMA storms. Lightweight backup checks at minutes 32, 47 and 57 start the same
|
| 143 |
+
writer only when the last published live check is at least one hour old.
|
| 144 |
+
The gate skips active or queued archive writers; the shared non-cancelling
|
| 145 |
+
publication lock remains on the inference job. Scheduled runs use
|
| 146 |
+
`historical_limit=0`; historical batches remain explicit or existing continuations.
|
| 147 |
+
GitHub may delay or drop schedules; this is not an exact wall-clock guarantee.
|
| 148 |
+
Public-repository schedules can be disabled after 60 days of inactivity.
|
| 149 |
+
|
| 150 |
+
The existing `automatic-forecasts.yml` schedule is the only live inference
|
| 151 |
+
scheduler; no second Sites or Mac automation is required. A new, validated JMA
|
| 152 |
+
analysis produces a new +120-hour issue and a separate **50-member live ensemble**
|
| 153 |
+
with genuine common-grid pressure means. Already completed issue IDs are skipped.
|
| 154 |
+
An hourly check does not imply that JMA or the six-hour GFS analysis grid supplies
|
| 155 |
+
new input every hour; unavailable sources remain explicit. The Live and History
|
| 156 |
+
views follow the latest completed
|
| 157 |
+
issue unless the viewer deliberately selects an archive. Updates continue with
|
| 158 |
+
the browser and Mac closed. The Live page polls every minute while visible and
|
| 159 |
+
again on focus or reconnect; History retains its five-minute poll. `status.json` exposes the last check, requested
|
| 160 |
+
3600-second interval, ensemble member count, source errors and cloud run URL.
|
| 161 |
+
|
| 162 |
+
Historical backfill now continues in **back-to-back serialized batches**: after
|
| 163 |
+
successfully publishing a productive batch, the workflow immediately dispatches
|
| 164 |
+
the next batch if uncomputed issues are ready. The same concurrency group permits
|
| 165 |
+
only one runner at a time. There can still be GitHub queue and environment setup
|
| 166 |
+
gaps; this is not a permanently running server. The hourly schedule remains
|
| 167 |
+
for live updates and delayed source retries. Continuation stops when all work is
|
| 168 |
+
complete, only cooling-down failures remain, a batch makes no successful progress,
|
| 169 |
+
or inference/publishing fails. A targeted single-issue retry does not start a chain.
|
| 170 |
+
No model, input boundary, membership, or forecast values are changed by this policy.
|
| 171 |
+
|
| 172 |
+
Historical outputs are **one deterministic member**, never labelled as the
|
| 173 |
+
separate 50-member live means. Recovery shares the same publication lock and
|
| 174 |
+
checks for overdue live input at an hourly threshold before starting a batch.
|
| 175 |
+
The historical target starts in **1970**. Recent playback takes priority:
|
| 176 |
+
**4,015 independent six-hour issues from all 135 eligible Western Pacific storms
|
| 177 |
+
in 2022–2026** in the current observed snapshot. Each tick initializes a new
|
| 178 |
+
+120-hour run, with that issue's centre, intensity, motion and nine causal analyses.
|
| 179 |
+
Fung-wong's 39 consecutive ticks are first for playback verification. These
|
| 180 |
+
are not shifted copies of an earlier route and are not selected by forecast error.
|
| 181 |
+
The completed first phase is preserved. The next phase adds **26,575 six-hour
|
| 182 |
+
issues for 1996–2021**; 2022 is already covered and is not rerun. The combined
|
| 183 |
+
queue now contains **32,143 issues**, including **30,590 six-hour ticks** and
|
| 184 |
+
1,553 preserved older first issues. All 5,568 original issue definitions and
|
| 185 |
+
completed forecasts remain unchanged. Older backfill retains the first
|
| 186 |
+
supported issue of each uncovered storm.
|
| 187 |
+
Out-of-domain global storms are not
|
| 188 |
+
silently extrapolated. A storm record is not a promise that usable weather exists.
|
| 189 |
+
|
| 190 |
+
Nine consecutive six-hour weather analyses end no later than issue time. Live
|
| 191 |
+
GFS is f000 only, with its final analysis at most twelve hours before the JMA
|
| 192 |
+
issue. Live sustained wind is read from JMA's analysis in knots when available;
|
| 193 |
+
missing wind remains missing. Motion remains explicitly missing; native detail
|
| 194 |
+
is masked. Initialization uses the analysis valid time, not bulletin publication
|
| 195 |
+
time, and rejects future or stale analyses. JMA's future forecast points are
|
| 196 |
+
comparison data only, never model inputs.
|
| 197 |
+
The resulting GFS transfer is experimental. Historical NOAA NCEP reanalyses are
|
| 198 |
+
retrospective, not operationally available inputs; fitting-year overlaps are
|
| 199 |
+
labelled and these forecasts are not a fresh independent evaluation.
|
| 200 |
+
|
| 201 |
+
NOAA **ended NCEP/NCAR Reanalysis 1 at March 17, 2026**
|
| 202 |
+
([source notice](https://psl.noaa.gov/data/reanalysis/)). Later historical issues
|
| 203 |
+
now use nine exact six-hour **archived GFS f000 analyses** from NOAA's public
|
| 204 |
+
archive. This is explicitly labelled an experimental GFS-input transfer, not
|
| 205 |
+
the original R1 evaluation or a claim of operational availability at the issue.
|
| 206 |
+
The complete history window uses one provider; R1 and GFS are not mixed within
|
| 207 |
+
an initialization. No later-valid analysis, forecast step, nearest-date fill,
|
| 208 |
+
or future observation can replace a missing analysis. Each new archive forecast
|
| 209 |
+
records the nine source URLs, valid times and GRIB-subset SHA-256 hashes.
|
| 210 |
+
Only the eight required GRIB messages are fetched with checked byte ranges;
|
| 211 |
+
timestamps, levels, units and missing values are checked again after decoding.
|
| 212 |
+
New analyses are cached across cloud batches to avoid re-downloading overlapping
|
| 213 |
+
history windows.
|
| 214 |
+
|
| 215 |
+
The input-only archive is checksum-pinned in `release_tools/history_inputs.json`.
|
| 216 |
+
Weights and source modules must match the release hashes before inference.
|
| 217 |
+
Complete finite +6…+120-hour outputs are saved on the `forecast-data` branch.
|
| 218 |
+
Old issues are immutable; new outputs are added, not substituted into old dates.
|
| 219 |
+
The live job is deduplicated by storm and exact analysis time. Historical errors
|
| 220 |
+
are retried after 24 hours while other storms can continue. A versioned input
|
| 221 |
+
reader correction releases old failures for one immediate retry; a new failure
|
| 222 |
+
then backs off normally. A manual exact-ID retry also bypasses the cooldown,
|
| 223 |
+
without starting an automatic batch chain. Status separately reports unresolved
|
| 224 |
+
planned-issue errors; obsolete issue IDs from earlier queues are preserved under
|
| 225 |
+
`retired_source_errors`, not presented as unresolved current work. Publication
|
| 226 |
+
and chaining require the independent saved-output audit to pass.
|
| 227 |
+
|
| 228 |
+
Public files beneath that branch's `data/` directory:
|
| 229 |
+
|
| 230 |
+
- `catalog.json`: available storms, exact issues, member counts and worker status.
|
| 231 |
+
- `status.json`: completion counts, live-issue availability, errors and run URL.
|
| 232 |
+
- `coverage.json`: 1970 boundary, domain and input-queue classifications.
|
| 233 |
+
- `verification.json`: an independent saved-output audit of every completed
|
| 234 |
+
planned issue: release/member identity, observed +0 alignment, causal history,
|
| 235 |
+
21 route points, 20 finite physical pressure grids, exact leads and catalogue
|
| 236 |
+
coverage. `complete` is true only when every pinned planned ID is present.
|
| 237 |
+
- `forecasts/{id}.json`: geographic route, central pressure, provenance.
|
| 238 |
+
- `fields/{id}.json.gz`: losslessly compressed genuine model basin-pressure
|
| 239 |
+
arrays in hPa. Existing uncompressed files remain readable. The public API
|
| 240 |
+
transparently returns ordinary JSON for either format.
|
| 241 |
+
|
| 242 |
+
The Weather Lab API merges these results with its existing verified archive.
|
| 243 |
+
The GitHub Pages website can use the same public History API. A cache miss or
|
| 244 |
+
source outage remains explicit; no future observation is substituted as forecast.
|
| 245 |
+
|
| 246 |
+
## Playback and scale
|
| 247 |
+
|
| 248 |
+
The History player defaults to Western Pacific and orders storms by formation
|
| 249 |
+
time (first observed record), newest first. Its issue-time slider advances in
|
| 250 |
+
six-hour ticks. Each tick requires an exact matching saved initialization; it
|
| 251 |
+
pauses on an uncomputed tick rather than showing a previous issue as a new one.
|
| 252 |
+
A separate preview-lead selector chooses the pressure field from that issue.
|
| 253 |
+
The original one-issue animation remains available.
|
| 254 |
+
|
| 255 |
+
The full 1970-onward snapshot contains approximately 65,146 eligible six-hour
|
| 256 |
+
ticks across 1,919 WP storm records. The requested 1996-onward subset is queued
|
| 257 |
+
at six-hour frequency; pre-1996 full-tick expansion is not launched. Runtime
|
| 258 |
+
and compressed output size should be measured on the cloud pilot before
|
| 259 |
+
estimating total storage and wall-clock completion. The queued combined plan
|
| 260 |
+
contains 32,143 issues (30,590 ticks and 1,553 older first issues).
|
| 261 |
+
Scheduled batches attempt up to 500 issues or 45 minutes, whichever comes first.
|
| 262 |
+
An unavailable archived analysis may delay an individual historical tick; the
|
| 263 |
+
source error is retained and retried after 24 hours, never filled with truth.
|
| 264 |
+
|
| 265 |
+
The read-only hosted integration run
|
| 266 |
+
[36554550793](https://github.com/yu314-coder/typhoon-predict/actions/runs/36554550793)
|
| 267 |
+
verified two consecutive Fung-wong initializations: six hours apart, distinct
|
| 268 |
+
causal input hashes, distinct routes and distinct pressure arrays, with 20
|
| 269 |
+
forecast leads each. The second forecast completed about 1.27 seconds after the
|
| 270 |
+
first; the two-run pipeline took 6.5 seconds excluding package setup. This tiny
|
| 271 |
+
warm-cache sample is a throughput lower bound, not an ETA. At that rate the
|
| 272 |
+
recent cohort needs about 1.4 compute-hours and the full archive about 23 hours,
|
| 273 |
+
before weather retrieval, initialization, failures, publishing and scheduler
|
| 274 |
+
gaps. The former six-hour pause between productive backfill batches is removed;
|
| 275 |
+
NOAA availability can still block completion regardless of compute throughput.
|
| 276 |
+
|
| 277 |
+
A measured basin-field sequence used 44,311 bytes with lossless gzip: about
|
| 278 |
+
170 MiB for 4,015 issues, or 2.7 GiB for 65,146, excluding routes, catalogues,
|
| 279 |
+
Git history and existing outputs. These are single-sample storage estimates,
|
| 280 |
+
not fixed quotas or guaranteed total sizes.
|
| 281 |
+
|
| 282 |
+
|
| 283 |
+
<a id="docs-daily-storm-benchmark-md"></a>
|
| 284 |
+
|
| 285 |
+
## docs/daily_storm_benchmark.md
|
| 286 |
+
|
| 287 |
+
# Daily forecasts, equal-storm benchmark
|
| 288 |
+
|
| 289 |
+
Status: **complete** at 2026-09-29T08:22:39Z. All **1,473 daily cases / 270 storms** finished; the SHA-256 of every saved forecast was verified. [Final equal-storm results and all 270 storm scores](../evaluation/daily_storm_final.json).
|
| 290 |
+
|
| 291 |
+
Mean track error is 798.4 km for 1.1 and 471.2 km for 1.2 (50-member mean); +120 h error is 1,646.4 versus 1,031.6 km. Direction error is 51.58° versus 34.96°, centred shape similarity 0.7544 versus 0.8837, and geographic path similarity 0.5345 versus 0.6560. The original track run reported 1.2-only central-pressure MAE of 12.62 hPa on 40 pressure-labelled storms and basin MSLP MAE of 2.72 hPa on 270 storms. It did not produce 1.1 intensity outputs.
|
| 292 |
+
|
| 293 |
+
The completed [native pressure follow-up](#docs-intensity-benchmark-md) now provides a matched 1.1 comparison on 134 daily starts / 40 storms from the same frozen plan. JMA MAE is 13.53 / 12.84 hPa for 1.1 / 1.2; USA MAE is 12.84 / 12.55 hPa. Each uses exact common agency-specific labels. The 1,339 remaining issue inputs are unavailable to the frozen 1.1 intensity pipeline, never zero-scored or filled with future observations. Paired whole-storm intervals include zero. These are not all-270-storm pressure scores.
|
| 294 |
+
|
| 295 |
+
On 1 October, `release_tools/build_release_benchmark.py` independently rechecked all forecast hashes, coordinate conversions and daily/storm route aggregates, reproduced the native pressure postprocessor and generated the [shared snapshot](../evaluation/released_daily/released_daily_benchmark.json) and [original Trackformer audit receipt](../evaluation/released_daily/released_daily_verification.json). No inference or weight changes were necessary.
|
| 296 |
+
|
| 297 |
+
The [completed WeatherNext Cyclones Mini comparison](#docs-deepmind-daily-benchmark-md) now covers exactly the same 1,473 starts / 270 storms. Mini's combined mean track error is 498.4 km, direction error 39.76°, and JMA pressure MAE 10.20 hPa on 134 shared intensity starts. Its recent-only mean track error is 288.5 km versus 477.2 km for 1.2. The [independent three-way publication audit](../evaluation/deepmind_daily/publication_audit.json) rechecks the CUDA archive, unchanged original Trackformer arrays and recomputed common-support scores; the older 1 October receipt remains the audit of the original two-model snapshot, not a hash receipt for the subsequently merged JSON. No older benchmark's DeepMind score has been transferred into this comparison.
|
| 298 |
+
|
| 299 |
+
## What counts as a case and a score
|
| 300 |
+
|
| 301 |
+
One case is one typhoon at one issue time on one UTC day. Choose the earliest eligible issue on that day, once per typhoon. Each forecast predicts all 20 six-hour leads, +6 through +120 h, from information at or before issue time.
|
| 302 |
+
|
| 303 |
+
For position error, daily score = mean of the 20 lead errors; storm score = mean of that storm's daily scores; benchmark score = mean of the 270 storm scores. A storm with ten available daily starts has exactly the same final weight as a storm with three. The per-lead comparison also averages daily values within storm before averaging storms.
|
| 304 |
+
|
| 305 |
+
Direction error, route-shape similarity, Fréchet distance and path similarity remain separate metrics. Their definitions match the [evaluation notes](#docs-trackformer-1-2-evaluation-md); direction uses only common non-stationary leads. Missing direction scores are reported, not replaced by zero. There is no invented combined weighted quality mark.
|
| 306 |
+
|
| 307 |
+
## Frozen cohort
|
| 308 |
+
|
| 309 |
+
- 270 distinct storms, 1,473 daily issue cases, selected from 523 locally eligible storms.
|
| 310 |
+
- All 40 eligible recent storms beginning in 2024 or later, plus 230 historical storms selected at evenly spaced positions in the chronological 1980–1999 storm list.
|
| 311 |
+
- The entire storm must lie outside the selected 1.2 checkpoint's 2000–2021 fitting and 2022–2023 validation years, with the history/target boundary also outside those years.
|
| 312 |
+
- Issue centre within 0–60°N / 100–180°E; exact six-hour weather history through issue; complete route labels through +120 h; matching pressure-atlas coverage for scoring.
|
| 313 |
+
- Use all eligible daily starts for each chosen storm. Never drop a case because its forecast looks poor.
|
| 314 |
+
- [Cohort manifest](../evaluation/release_data/daily_storm_cohort.json) freezes rows, dates, storm IDs, source hashes and checkpoint identity before inference. Forecast artifacts are new; some recent storms overlap earlier development comparisons.
|
| 315 |
+
|
| 316 |
+
Requiring complete five-day truth excludes short remaining lifetimes. This is a declared coverage limitation, not an all-storm operational sample. Historical cases are retrospective hindcasts from a model trained on later years; reanalysis and best-track issue states do not establish real-time data availability. This cohort has not been certified untouched across all earlier experiments.
|
| 317 |
+
|
| 318 |
+
## Models and input boundaries
|
| 319 |
+
|
| 320 |
+
Trackformer **1.2** uses the unchanged released checkpoint and 50 independently seeded, smooth input-perturbation members. Seeds are `2043 + source_row * 100 + member_id`, with member IDs 0–49. Normalized perturbation amplitudes remain 0.025 for basin history and 0.015 for regional history. These are not learned latent samples or 50 separate trained models. Member routes and central pressures are averaged; physical fields are averaged on fixed common geographic grids. Missing native regional inputs retain the model's missing-detail mask.
|
| 321 |
+
|
| 322 |
+
Trackformer **1.1** uses the released causal weighted route builder with issue, −12 h and −24 h analyses and historical motion. Its absolute route is converted into the same issue-centred local-km coordinate system as 1.2. The pipelines differ, so this is a released-system comparison, not an architecture-only ablation.
|
| 323 |
+
|
| 324 |
+
Future weather/track/pressure are used for eligibility coverage and scoring only, never inference. No retraining, coefficient tuning, new weather downloads or checkpoint substitution occurs in this run.
|
| 325 |
+
|
| 326 |
+
## Pressure and failures
|
| 327 |
+
|
| 328 |
+
For the original 1.2 run, central-pressure MAE uses valid labels and cosine-area-weighted basin MSLP MAE uses the native basin grid, daily then equally per storm. Keep pressure label coverage visible. The separate native-pressure follow-up uses a paired mask for both models at each agency's exact valid times; it does not substitute its smaller coverage into all-storm track results. Regional/core field scores need their own valid coverage and are not implied by basin-wide error. No same-cohort 1.1 basin-field score is claimed.
|
| 329 |
+
|
| 330 |
+
Completed daily forecasts are saved atomically with hashes. A nonfinite forecast or source mismatch stops the run visibly rather than silently deleting the case. Resume skips completed cases without rerunning them. Partial aggregate scores include only fully completed storms, identify their counts, and must not be presented as the final 270-storm result. Final uncertainty should resample whole storms, not overlapping daily issues independently.
|
| 331 |
+
|
| 332 |
+
|
| 333 |
+
<a id="docs-deepmind-daily-benchmark-md"></a>
|
| 334 |
+
|
| 335 |
+
## docs/deepmind_daily_benchmark.md
|
| 336 |
+
|
| 337 |
+
# Completed WeatherNext Cyclones Mini daily benchmark
|
| 338 |
+
|
| 339 |
+
## Model and frozen plan
|
| 340 |
+
|
| 341 |
+
**Complete and verified:** all **1,473 daily forecasts / 270 Western Pacific storms** finished on an **NVIDIA GeForce RTX 3070 using JAX CUDA in Windows/WSL**. The final worker audit is dated **4 October 2026, 12:55 UTC**. Publication independently rechecked all returned case hashes, physical grids, unchanged Trackformer reference arrays and exact common-support scores. No new inference was performed during this review.
|
| 342 |
+
|
| 343 |
+
The reference is Google's official [WeatherNext Cyclones Mini](https://github.com/google-deepmind/weathernext#weathernext-cyclones-mini): **`WeatherNextCyclones_Mini_<2024`**, trained through 2023 at native **1° resolution**. The pinned [software package is v0.3.0](https://github.com/google-deepmind/weathernext/blob/89c4b2a77a1c57b328b909c575550fd2e5aadc9c/setup.py); the package version and checkpoint generation identify different things. This is not a custom imitation, full-sized WeatherNext 2/Cyclones, or a WeatherNext 3 checkpoint. Mini's scores do not stand in for those larger models.
|
| 344 |
+
|
| 345 |
+
- Cohort: **1,473 daily starts / 270 Western Pacific storms**, identical to the frozen Trackformer 1.1/1.2 route comparison.
|
| 346 |
+
- Forecast: twenty six-hour steps, **+6 to +120 h**, with the exact observed issue-time centre as the initialized-storm anchor.
|
| 347 |
+
- Inputs: exact ERA5 analyses at **issue−6 h and issue**, calendar forcings only after issue. No future weather, storm labels or official forecast tracks enter the worker.
|
| 348 |
+
- Weights SHA-256: `a1bb151457077248d70a458b1a7b19deacd2926fbcb46eee4ff2691444e3596e`.
|
| 349 |
+
- Cohort SHA-256: `965184f1e5ab3fcb52e304f39b38583a14f753870aa2295f2d01942a441dcfd6`.
|
| 350 |
+
- Official code commit: `89c4b2a77a1c57b328b909c575550fd2e5aadc9c`.
|
| 351 |
+
- CUDA protocol SHA-256: `91b31582f24ffdd4f2309eddb0cc68498a2c31cab7cbabaf06f43dfbff413606`.
|
| 352 |
+
- Runtime: JAX/JAXlib 0.4.38, Python 3.12.3, highest float32 matmul precision, TF32 disabled.
|
| 353 |
+
|
| 354 |
+
## Results
|
| 355 |
+
|
| 356 |
+
| Matched development metric | Trackformer 1.1 | Trackformer 1.2 · mean of 50 | Mini · one member | Coverage |
|
| 357 |
+
| --- | ---: | ---: | ---: | --- |
|
| 358 |
+
| Mean track error · km, lower is better | 798.41 | 471.20 | 498.37 | 1,473 days / 270 storms |
|
| 359 |
+
| +120 h track error · km, lower is better | 1,646.40 | 1,031.63 | 1,031.91 | 1,473 days / 270 storms |
|
| 360 |
+
| Direction error · degrees, lower is better | 51.58 | 34.96 | 39.76 | 1,473 days / 270 storms |
|
| 361 |
+
| Centred route-shape similarity · higher is better | 0.7544 | 0.8837 | 0.9010 | 1,473 days / 270 storms |
|
| 362 |
+
| JMA central-pressure MAE · hPa, lower is better | 13.53 | 12.84 | 10.20 | 134 days / 40 storms / 2,637 leads |
|
| 363 |
+
| JMA pressure-curve similarity · higher is better | 0.7074 | 0.7118 | 0.8088 | 134 days / 40 storms |
|
| 364 |
+
| USA central-pressure MAE · hPa, lower is better | 12.84 | 12.55 | 11.55 | 134 days / 40 storms / 2,325 leads |
|
| 365 |
+
| USA pressure-curve similarity · higher is better | 0.7237 | 0.6637 | 0.7445 | 131 days / 40 storms |
|
| 366 |
+
|
| 367 |
+
Pressure measures central-pressure intensity, not map-grid error. The lower USA curve count excludes flat/short curves; it is not zero-filled. All three models' pressure-curve case IDs were verified equal for each agency. DeepMind geographic path/Fréchet scores and whole-storm confidence intervals are not supplied by this export and remain unavailable.
|
| 368 |
+
|
| 369 |
+
### Historical and recent groups
|
| 370 |
+
|
| 371 |
+
| Period | Daily starts / storms | Mean track error · 1.1 / 1.2 / Mini, km | Direction error · 1.1 / 1.2 / Mini, degrees |
|
| 372 |
+
| --- | --- | --- | --- |
|
| 373 |
+
| 1980–1999 | 1,336 / 230 | 780.50 / 470.17 / 534.86 | 51.36 / 35.01 / 41.26 |
|
| 374 |
+
| Storms beginning in 2024+ | 137 / 40 | 901.45 / 477.16 / 288.51 | 52.90 / 34.68 / 31.16 |
|
| 375 |
+
|
| 376 |
+
Mini has better recent-only track scores even though 1.2 has lower position/heading errors in the combined cohort. Mini also has lower JMA/USA pressure MAE and higher pressure-curve similarity. Historical fitting-year overlap, different weather inputs and different ensemble sizes prevent a general or equal-compute superiority claim. All pressure comparisons here belong to the recent group; no historical pressure score is inferred from missing native 1.1 inputs.
|
| 377 |
+
|
| 378 |
+
[Full results, period breakdowns and per-lead track errors](../evaluation/deepmind_daily/benchmark.json) · [Shared snapshot and existing 1.1/1.2 uncertainty](../evaluation/released_daily/released_daily_benchmark.json) · [Completion receipt](../evaluation/deepmind_daily/verification.json) · [Independent review](../evaluation/deepmind_daily/publication_audit.json)
|
| 379 |
+
|
| 380 |
+
### Strict forecast-date partition and separate charts
|
| 381 |
+
|
| 382 |
+
All three README charts—total, strict `<2024` and strict `>2024`—use **`WeatherNextCyclones_Mini_<2024`**, the same official checkpoint trained through 2023, software v0.3.0, native 1° and one CUDA member. A post-2024 forecast is not a post-2024-trained model. The following proportions refer to this evaluation cohort, not the model's training dataset.
|
| 383 |
+
|
| 384 |
+
| UTC issue year | Starts / share | Storms / share |
|
| 385 |
+
| --- | --- | --- |
|
| 386 |
+
| `<2024` (1980–1999 here) | 1,336 / 90.7% | 230 / 85.2% |
|
| 387 |
+
| Calendar 2024 | 76 / 5.2% | 22 / 8.1% |
|
| 388 |
+
| `>2024` (2025–2026 here) | 61 / 4.1% | 18 / 6.7% |
|
| 389 |
+
|
| 390 |
+
The original `recent_2024_onward` aggregate includes 2024 and is retained unchanged. The new **strict `after_2024`** derivative excludes 2024: track position MAE is **963.02 / 469.40 / 258.67 km**, direction error **57.88 / 33.81 / 28.73°**, and JMA pressure MAE **12.59 / 11.12 / 9.55 hPa** for 1.1 / 1.2 / Mini. Track uses 61 starts / 18 storms; pressure uses 59 starts / 18 storms / 1,174 exact common leads. The total chart still uses the original scores. Its historical 230 storms contribute 85.2% of equal-storm track weight, not 90.7%.
|
| 391 |
+
|
| 392 |
+
The strict **`before_2024`** chart uses the 1,336 historical starts / 230 storms (1980–1999 here): position MAE **780.50 / 470.17 / 534.86 km** and direction error **51.36 / 35.01 / 41.26°** for 1.1 / 1.2 / Mini. All three shared JMA pressure scores are null, with zero eligible common starts: frozen historical 1.1 intensity inputs are unavailable. That panel says **Not scored** and contains no bars, not zero-height bars. Track coverage must never be reused as pressure coverage.
|
| 393 |
+
|
| 394 |
+
[UTC partitions, exact case IDs, all metrics and source hashes](../evaluation/deepmind_daily/period_comparison.json) · [Total chart](../evaluation/released_daily/model_1_2_benchmark.png) · [Pre-2024 chart](../evaluation/released_daily/model_1_2_before_2024_benchmark.png) · [Post-2024 chart](../evaluation/released_daily/model_1_2_after_2024_benchmark.png)
|
| 395 |
+
|
| 396 |
+
Reproduce these derivative summaries without inference or raw-weather downloads:
|
| 397 |
+
|
| 398 |
+
```bash
|
| 399 |
+
python release_tools/build_deepmind_period_comparison.py --archive /path/to/DeepMind_RTX3070_results_20261004_125651_467960.zip
|
| 400 |
+
python release_tools/plot_model_announcement.py
|
| 401 |
+
```
|
| 402 |
+
|
| 403 |
+
The helper checks the exact archive hash against the existing publication audit and every one of the 1,473 case-score JSON hashes against the frozen manifest. It rechecks identities, causal-time metadata, CUDA completion and common pressure masks, reproduces the original total and period scores, then recomputes only the UTC subsets using the unchanged equal-storm aggregation. It never rewrites the original worker export, changes a forecast array or claims a new raw-input audit. Chart sidecars identify their derivative source hash, checkpoint, period, exact values and missing-data policy.
|
| 404 |
+
|
| 405 |
+
## Like-for-like scoring
|
| 406 |
+
|
| 407 |
+
All three models use the frozen starts, exact future label times and original issue-relative kilometre projection. Routes are unshifted. Direction is recomputed on **common moving steps across truth and all three models**, not compared using different masks.
|
| 408 |
+
|
| 409 |
+
Pressure uses the original 134 eligible native-intensity starts / 40 storms, restricted to common valid leads across three predictions and the selected reference. JMA and USA remain separate. MAE and centred time-curve similarity are separate; missing, nonphysical or failed outputs are not zeros. Flat or shorter-than-six-point curves have unavailable shape similarity.
|
| 410 |
+
|
| 411 |
+
Average valid leads within an issue, days within a storm, then storms equally. Partial aggregates contain only fully completed storms and explicit coverage. Recent 2024+ and historical 1980–1999 groups are separate because the latter overlap WeatherNext fitting years. Different input pipelines and the one-member Mini versus 50-input-member 1.2 policy make this an output comparison, not an equal-compute architecture ablation or a certified unused test.
|
| 412 |
+
|
| 413 |
+
The official six-hour direct tracker uses a frozen initialized-storm continuity policy: no cyclogenesis, no dissipation pruning and no nearby-cyclone pruning. This is disclosed rather than called the operational default. Native 1° WP MSLP grids are saved in physical hPa; cyclone-head central pressure is not the basin-grid minimum.
|
| 414 |
+
|
| 415 |
+
## Execution and verification
|
| 416 |
+
|
| 417 |
+
The original [Mac CPU runner and scoring definitions](../release_tools/deepmind_daily_benchmark.py) provided the frozen plan. The completed run used the separately pinned, resumable RTX 3070 CUDA package; the old CPU launch instructions are not the execution record of these results. The [frozen CUDA protocol](../evaluation/deepmind_daily/protocol.json), [2,946-file case manifest](../evaluation/deepmind_daily/case-manifest.json) and [CPU/CUDA canary](../evaluation/deepmind_daily/canary.json) identify the completed run.
|
| 418 |
+
|
| 419 |
+
The strict same-runtime CPU/CUDA canary passed unchanged 0.01 hPa mean / 0.1 hPa maximum field tolerances. Its older cross-runtime Mac comparison failed the maximum-field gate and remains explicitly recorded as failed; that diagnostic is not numerical-equivalence certification. No CPU forecast was counted or relabelled as a CUDA case.
|
| 420 |
+
|
| 421 |
+
The publication review checks all 1,473 IDs, original reference hashes, exact issue−6 h / issue input-time metadata and saved input hashes, all twenty output times, finite native pressure grids, and recomputed three-model route/pressure metrics. **Raw ERA5 input states are not in the returned result-only ZIP**: this review preserves the original workers' causal-input receipts and hashes, rather than claiming to independently reopen those raw states.
|
| 422 |
+
|
| 423 |
+
To repeat the read-only archive review, use the existing frozen project reference forecasts and intensity labels, NumPy, and the returned result archive. It does not run a model or download weather:
|
| 424 |
+
|
| 425 |
+
```bash
|
| 426 |
+
python release_tools/import_deepmind_release_results.py \
|
| 427 |
+
--archive /path/to/DeepMind_RTX3070_results_20261004_125651_467960.zip \
|
| 428 |
+
--project /Volumes/D/typhoon_predict \
|
| 429 |
+
--output /Volumes/D/typhoon_predict/remote_typhoon_predict_repo/evaluation/deepmind_daily
|
| 430 |
+
```
|
| 431 |
+
|
| 432 |
+
Completed scores are also exposed by the existing [public benchmark API](https://trackformer-weatherlab.rudin-euler-8253.chatgpt.site/api/benchmarks/released). Documentation publication does not replace immutable forecasts, change either model's weights, restart the benchmark, modify active training, or redeploy the website. The original two-model verification receipt remains unchanged; the new publication audit identifies the merged snapshot's exact hash.
|
| 433 |
+
|
| 434 |
+
The returned worker's `benchmark.json` is retained byte-for-byte for its audit hashes. Its `published_to_site: false` and `integration_note` describe the handoff before import, not the current benchmark status. The completed CUDA receipt, independent publication audit and shared public snapshot above are the current completion/publication evidence; those historical worker fields do not mean the comparison is unfinished.
|
| 435 |
+
|
| 436 |
+
|
| 437 |
+
<a id="docs-intensity-benchmark-md"></a>
|
| 438 |
+
|
| 439 |
+
## docs/intensity_benchmark.md
|
| 440 |
+
|
| 441 |
+
# Matched daily intensity benchmark
|
| 442 |
+
|
| 443 |
+
Completed GPU inference and verified aggregation on 1 October 2026 (Taipei).
|
| 444 |
+
The released weights, original forecasts and original calibration are unchanged.
|
| 445 |
+
|
| 446 |
+
## Same hierarchy as track
|
| 447 |
+
|
| 448 |
+
The plan is the frozen 270-storm / 1,473-daily-issue track cohort. One storm on
|
| 449 |
+
one UTC day is one case, at its earliest eligible issue. Forecast errors at
|
| 450 |
+
valid +6, +12, …, +120 h leads are averaged per daily issue, then across its
|
| 451 |
+
storm's daily issues, then across storms with equal weights. Long-lived storms
|
| 452 |
+
do not gain extra final weight. Per-lead errors use the same daily → storm
|
| 453 |
+
averaging. Both models use exactly the same native observation mask for each
|
| 454 |
+
wind/pressure comparison. Wind and each agency's pressure are separate metrics;
|
| 455 |
+
there is no arbitrary weighted combined rank.
|
| 456 |
+
|
| 457 |
+
Native 1.1 intensity needs valid issue-time current wind and pressure. Its
|
| 458 |
+
frozen input archive provides them for 134 daily issues / 40 storms, not the
|
| 459 |
+
other 1,339 daily starts. These missing starts are reported, never repaired
|
| 460 |
+
with future observations or relabeled as zero error. Thus the result shares
|
| 461 |
+
the **method and planned cohort** of track, but does not claim the same valid
|
| 462 |
+
intensity coverage. Missing future observations also remain unavailable.
|
| 463 |
+
|
| 464 |
+
## Model identity and inference
|
| 465 |
+
|
| 466 |
+
1.2 is the released checkpoint SHA-256
|
| 467 |
+
`f194a23d3f91ea76ad776dfad942fabd669eeae8b3fd665815463095367e9ee0`.
|
| 468 |
+
Every original pressure forecast file is rehashed. To recover previously
|
| 469 |
+
unsaved auxiliary wind, the MPS runner replays the identical 50 input members
|
| 470 |
+
with seeds `2043 + source_row * 100 + member_id`, nine exact causal analyses,
|
| 471 |
+
original detail masks and original issue inputs. Every replay has 50 distinct
|
| 472 |
+
input hashes, finite 50 × 20 wind and pressure outputs and verified means.
|
| 473 |
+
The maximum pressure difference from original forecasts is **0.0 hPa**.
|
| 474 |
+
No weights, preprocessing or coefficients are fitted here.
|
| 475 |
+
|
| 476 |
+
1.1 uses the original public intensity pipeline: frozen primary and structure
|
| 477 |
+
experts, validation-fit anchor/calibration coefficients, and the frozen
|
| 478 |
+
same-storm −12/−24 h temporal branch where its current-wind gate permits.
|
| 479 |
+
Expert count/activation is preserved in each case receipt; this is not a
|
| 480 |
+
50-member 1.1 ensemble. Issue states and causal weather only enter inference;
|
| 481 |
+
future labels are loaded separately for scoring. The two systems' input
|
| 482 |
+
pipelines differ: this is not an architecture-controlled ablation.
|
| 483 |
+
|
| 484 |
+
## Native observation definitions
|
| 485 |
+
|
| 486 |
+
- Central pressure: exact storm SID and valid UTC; native `USA_PRES` and
|
| 487 |
+
`TOKYO_PRES` are scored separately, in hPa, on a common mask for each agency.
|
| 488 |
+
- Wind: `USA_WIND` nominal one-minute reference. The 1.2 scalar is an auxiliary
|
| 489 |
+
output, not a certified ten-metre sustained surface-wind product. JMA
|
| 490 |
+
`TOKYO_WIND` is ten-minute and is never converted or pooled in this score.
|
| 491 |
+
- Native 1.1 radius: `USA_RMW` and `USA_R34/R50/R64_NE/SE/SW/NW`, nautical miles
|
| 492 |
+
multiplied by 1.852 to kilometres. They are radii, not diameters. RMW must be
|
| 493 |
+
positive. A reported zero wind-radius extent remains valid; blanks remain
|
| 494 |
+
unavailable. JMA R30 longest/shortest, eye size and ROCI are not substitutes.
|
| 495 |
+
- Native 1.2 radius: **not available**, because it has no radius head. The
|
| 496 |
+
experimental pressure-scaled candidate is separate; its strict definition
|
| 497 |
+
guard prevents scoring it as calibrated official surface-wind radii.
|
| 498 |
+
|
| 499 |
+
Radius errors are averaged within the daily issue over valid leads for each
|
| 500 |
+
component, then equally across available quadrant components, then across
|
| 501 |
+
days within storm and equally across storms. Coverage differs by threshold;
|
| 502 |
+
all counts are printed. Observational RMW has reporting/provisional limitations.
|
| 503 |
+
|
| 504 |
+
## Curve comparison, analogous to route direction and shape
|
| 505 |
+
|
| 506 |
+
The released Site defaults to **central-pressure MAE**, with slight descriptive mean improvements (JMA 13.53 / 12.84 hPa; USA 12.84 / 12.55 hPa, for 1.1 / 1.2) and paired uncertainty including zero. Its secondary **similarity** view is the exact complement of the shape-error metric below: `(1 + centred cosine) / 2 = 1 − shape_error`. This is a presentation transform, not a new fit or score. JMA similarity is 0.7074 / 0.7118 on 40 storms and 134 starts; USA similarity is 0.7237 / 0.6637 on 40 storms and 131 non-flat starts. Unshifted physical hPa timelines remain visible separately. The public snapshot includes every input-eligible pressure timeline.
|
| 507 |
+
|
| 508 |
+
Magnitude MAE is retained alongside time-curve diagnostics, without optimizing
|
| 509 |
+
time lag, warping time, shifting observations or adjusting forecasts to truth.
|
| 510 |
+
These are separate, versioned post-hoc descriptive metrics, not a preregistered
|
| 511 |
+
untouched-holdout hypothesis test.
|
| 512 |
+
|
| 513 |
+
1. Curve-shape error = `(1 - centred cosine similarity) / 2`, lower is better.
|
| 514 |
+
Subtract each curve's common-lead mean, normalize its scalar magnitude and
|
| 515 |
+
keep original time order and sign. At least six common valid points and
|
| 516 |
+
non-flat truth and both forecasts are required. A flat/unavailable curve
|
| 517 |
+
does not receive a perfect score. This measure ignores level and amplitude,
|
| 518 |
+
so it does not replace MAE or demonstrate matching absolute intensity.
|
| 519 |
+
2. Trend-direction mismatch = fraction of adjacent six-hour changes whose
|
| 520 |
+
signs differ (increasing / flat / decreasing). Deadband is exactly zero;
|
| 521 |
+
flat-versus-changing is a mismatch. Missing leads break adjacency, never
|
| 522 |
+
form twelve-hour changes disguised as six-hour steps.
|
| 523 |
+
3. Six-hour tendency MAE = mean absolute error of those adjacent changes in
|
| 524 |
+
kt/6 h, hPa/6 h or km/6 h. It remains separate from shape error.
|
| 525 |
+
|
| 526 |
+
All daily curve scores use the same valid model/reference masks before the
|
| 527 |
+
storm aggregation. Uncertainty uses 2,000 whole-storm bootstrap replicates
|
| 528 |
+
with seed 4712, not overlapping issue resampling. Paired 1.2-minus-1.1
|
| 529 |
+
differences and their intervals are included in the full JSON report.
|
| 530 |
+
|
| 531 |
+
## Results and honest boundaries
|
| 532 |
+
|
| 533 |
+
USA pressure MAE: 1.1 **12.84** vs 1.2 **12.55 hPa**. JMA pressure MAE:
|
| 534 |
+
**13.53** vs **12.84 hPa**. Auxiliary wind diagnostic MAE: **16.98** vs
|
| 535 |
+
**25.03 kt**. Wind curve-shape error: **0.271** vs **0.385**. Thus wind is
|
| 536 |
+
worse for 1.2 despite the slight mean-pressure improvement. At +120 h USA
|
| 537 |
+
pressure is worse for 1.2, whereas JMA pressure is better; no all-lead/agency
|
| 538 |
+
win is claimed. Radius 1.1 errors are reported, but native 1.2 is N/A.
|
| 539 |
+
|
| 540 |
+
The old 1980–1999 cohort overlaps original 1.1 fitting, and 1.2 hindcasts are
|
| 541 |
+
retrospective predictions by a model trained on later years. Some recent
|
| 542 |
+
storms have been inspected in development. Complete five-day route labels
|
| 543 |
+
were required by the inherited cohort. This is not untouched generalization,
|
| 544 |
+
an operational comparison or proof of no overfitting. Scientific safety and
|
| 545 |
+
wind/radius definition limitations apply to all illustrated examples.
|
| 546 |
+
|
| 547 |
+
## Files and reproduction
|
| 548 |
+
|
| 549 |
+
[Verified aggregate and per-storm metrics](../evaluation/intensity/intensity_final.json)
|
| 550 |
+
· [Chart verification receipt](../evaluation/intensity/verification.json)
|
| 551 |
+
· [GPU runner](../release_tools/benchmark_intensity_v12_v11.py)
|
| 552 |
+
· [Independent verifier and plotter](../release_tools/plot_intensity_benchmark.py).
|
| 553 |
+
|
| 554 |
+
```sh
|
| 555 |
+
MPLCONFIGDIR=/Volumes/D/typhoon_predict/.cache/matplotlib \
|
| 556 |
+
/Volumes/D/typhoon_predict/.venv/bin/python -B \
|
| 557 |
+
release_tools/benchmark_intensity_v12_v11.py \
|
| 558 |
+
--project /Volumes/D/typhoon_predict \
|
| 559 |
+
--output /Volumes/D/typhoon_predict/output/intensity-v12-v11-20261001 --chunk 10
|
| 560 |
+
|
| 561 |
+
MPLCONFIGDIR=/Volumes/D/typhoon_predict/.cache/matplotlib \
|
| 562 |
+
/Volumes/D/typhoon_predict/.venv/bin/python -B \
|
| 563 |
+
release_tools/plot_intensity_benchmark.py \
|
| 564 |
+
--source /Volumes/D/typhoon_predict/output/intensity-v12-v11-20261001 \
|
| 565 |
+
--project /Volumes/D/typhoon_predict \
|
| 566 |
+
--output /Volumes/D/typhoon_predict/remote_typhoon_predict_repo/evaluation/intensity
|
| 567 |
+
```
|
| 568 |
+
|
| 569 |
+
The runner locks one process and resumes immutable completed cases. It never
|
| 570 |
+
silently restarts training. Large original forecasts/raw members remain on D;
|
| 571 |
+
public reports include exact source hashes and complete verified aggregates.
|
| 572 |
+
|
| 573 |
+
|
| 574 |
+
<a id="docs-repository-layout-md"></a>
|
| 575 |
+
|
| 576 |
+
## docs/repository_layout.md
|
| 577 |
+
|
| 578 |
+
# Current repository layout
|
| 579 |
+
|
| 580 |
+
The current source tree is focused on the selected pressure-field **Trackformer 1.2** release. The implementation remains at `models/trackformer_1_2_field/`; module filenames such as `v165_base.py` are required architecture dependencies, not unused legacy files. They and their source hashes have not been changed.
|
| 581 |
+
|
| 582 |
+
## Cleanup on 29 September 2026
|
| 583 |
+
|
| 584 |
+
Removed from the current branch:
|
| 585 |
+
|
| 586 |
+
- The withdrawn route/scalar `trackformer_1_2.py` candidate and its incompatible `examples/predict_trackformer_1_2.py` entry point.
|
| 587 |
+
- Root-level 1.1 inference/route/intensity/temporal modules and their old analysis utility, plus `colab_train_v17.ipynb`.
|
| 588 |
+
- The old 1.1 paper and superseded 1.2.26/matched100 comparison assets.
|
| 589 |
+
|
| 590 |
+
Moved current benchmark figures, metrics and plot data from `paper/` into `evaluation/`, updating documentation and reproduction paths. `paper/trackformer.tex` and `paper/trackformer.pdf` stay at their existing locations. Published HTML and MP4 URLs under `docs/` stay unchanged.
|
| 591 |
+
|
| 592 |
+
The 1.1-versus-1.2 comparison arrays, TIP diagnostic, current paper, weights, model source hashes and release tags are retained. No local research checkpoints, running benchmark, Hugging Face files or historical release assets were deleted by this source-tree cleanup.
|
| 593 |
+
|
| 594 |
+
## Recovery
|
| 595 |
+
|
| 596 |
+
The complete pre-cleanup repository remains in [commit 459a3d014aa4794652ceba3ecdffcde0beada2cd](https://github.com/yu314-coder/typhoon-predict/tree/459a3d014aa4794652ceba3ecdffcde0beada2cd). The [Trackformer 1.1 release](https://github.com/yu314-coder/typhoon-predict/releases/tag/trackformer-1.1) remains available. This cleanup does not rewrite Git history.
|
| 597 |
+
|
| 598 |
+
## Released 1.2 training
|
| 599 |
+
|
| 600 |
+
`training/trackformer_1_2/` contains the original training source, launch wrapper, requirements, historical preparation tools, configuration, hashes and public asset downloader. Large arrays and optimizer checkpoints live at the identical Hugging Face path and are referenced from GitHub.
|
| 601 |
+
|
| 602 |
+
|
| 603 |
+
<a id="docs-research-report-zh-md"></a>
|
| 604 |
+
|
| 605 |
+
## docs/research_report_zh.md
|
| 606 |
+
|
| 607 |
+
# TrackFormer 1.2 中文研究說明
|
| 608 |
+
|
| 609 |
+
**作品名稱:訓練AI模型探討西北太平洋氣壓系統影響臺灣附近颱風的預測評估**
|
| 610 |
+
|
| 611 |
+
本頁對照中文研究報告及本次核定的文字、圖表修改。只介紹發布的 **TrackFormer 1.2**;原有技術論文與歷史比較紀錄保留於各自的檔案,不取代本頁的報告範圍。
|
| 612 |
+
|
| 613 |
+
[GitHub 程式與研究紀錄](https://github.com/yu314-coder/typhoon-predict) · [Hugging Face 模型](https://huggingface.co/euler314/typhoon-predict) · [即時預測頁](https://trackformer-weatherlab.rudin-euler-8253.chatgpt.site/live) · [English overview](../README.md)
|
| 614 |
+
|
| 615 |
+
## 研究動機與目標
|
| 616 |
+
|
| 617 |
+
- 以一般家用電腦可訓練、可執行為目標,讓個人及學生能參與天氣預測研究。
|
| 618 |
+
- 建立預測西北太平洋氣壓圖的 AI 模型,再從同一預測氣壓取得颱風路徑與中心氣壓。
|
| 619 |
+
- 結合不同高度的水平風,提供研究東北風、西南風與颱風周圍氣流變化的基礎。本次成果評分以路徑與氣壓為主,並非已證明季風預測改善。
|
| 620 |
+
|
| 621 |
+
## 研究設備與器材
|
| 622 |
+
|
| 623 |
+
| 設備或工具 | 實際用途與範圍 |
|
| 624 |
+
| --- | --- |
|
| 625 |
+
| Apple M4 Mac、24 GB 統一記憶體 | 本機模型執行、資料處理與製圖;透過 PyTorch MPS 使用內建 GPU 與 Metal。家用電腦訓練與執行是研究目標,不代表每一次歷史計算都只使用同一部電腦。 |
|
| 626 |
+
| NVIDIA GeForce RTX 3070、CUDA | 已完成的 DeepMind 輕量版比較預測使用此設備;RTX 3070 是硬體,不是軟體。 |
|
| 627 |
+
| 外接儲存裝置與網路 | 保存資料、模型、預測及圖表,取得公開資料並分享成果。 |
|
| 628 |
+
| Python、PyTorch | 建立模型、計算誤差、更新參數與執行預測。 |
|
| 629 |
+
| NumPy、SciPy、xarray、netCDF4 | 數值運算,依時間及經緯度整理天氣網格。 |
|
| 630 |
+
| Matplotlib | 繪製氣壓、路徑、曲線與長條圖。 |
|
| 631 |
+
| XeLaTeX、xeCJK | 中文報告排版。 |
|
| 632 |
+
|
| 633 |
+
## 資料來源與分配
|
| 634 |
+
|
| 635 |
+
| 資料 | 模型實際使用的內容 |
|
| 636 |
+
| --- | --- |
|
| 637 |
+
| NOAA NCEP/NCAR Reanalysis 1 | 海平面氣壓、500 百帕高度,以及 850、500、200 百帕的東西向風 u 和南北向風 v。u、v 共同提供風向與風速。 |
|
| 638 |
+
| ARCO-ERA5 氣壓 | 1,000 個具有原生 0.25 度氣壓資料的案例;以起報中心向四方各取 15 度,共 121×121 格點。 |
|
| 639 |
+
| NOAA IBTrACS 颱風紀錄 | 位置、先前移動、可用的中心氣壓及最大風,缺值保留有效性標記。中心氣壓比較採日本氣象廳紀錄。 |
|
| 640 |
+
| 固定地理資料 | 經緯度、陸地比例與地形高度。技術輸入同時保留環境及中心附近的地理資訊,並非未來天氣。 |
|
| 641 |
+
|
| 642 |
+
環境資料涵蓋北緯 0–60 度、東經 100–180 度,間距 2.5 度,共 25×33 格點。較細氣壓案例是下表各組的子集,不另加到案例總數。
|
| 643 |
+
|
| 644 |
+
| 用途 | 年份 | 歷史資料片段 | 其中有原生較細氣壓資料 |
|
| 645 |
+
| --- | --- | ---: | ---: |
|
| 646 |
+
| 訓練與正規化 | 2000–2021 | 13,949 | 800 |
|
| 647 |
+
| 驗證與模型選擇 | 2022–2023 | 1,041 | 100 |
|
| 648 |
+
| 原始測試分組 | 2024–2025 | 1,195 | 100 |
|
| 649 |
+
|
| 650 |
+
同一颱風保持在同一分組;跨越分組界線的資料片段不使用。資料的平均值與變動範圍只由訓練組計算。後來取得的 2026 年資料不表示發布模型重新訓練到 2026 年。[來源與檔案檢查](TrackFormer_1.2_Training_and_Reproducibility.pdf)。
|
| 651 |
+
|
| 652 |
+
## 模型如何預測與訓練
|
| 653 |
+
|
| 654 |
+
1. 輸入起報前兩天至起報時刻的資料,每六小時一筆,共 **9 筆**,搭配當時的颱風位置、強度、先前移動與固定地形。
|
| 655 |
+
2. 分別計算周圍氣壓、水平風與颱風中心附近氣壓的變化,再一起更新六小時後的狀態。
|
| 656 |
+
3. 從預測氣壓找出颱風中心,再讀取同一中心的氣壓,得到氣壓圖、路徑與中心氣壓。
|
| 657 |
+
4. 把預測狀態接著往下計算,共 **20 步、+6 至 +120 小時**。
|
| 658 |
+
5. 訓練時與之後的真實紀錄比較並更新參數;預測時不使用未來觀測。訓練預報長度由一天逐步增加到兩天、三天及五天;報告使用選定的第四輪模型。
|
| 659 |
+
|
| 660 |
+
模型有 21,452,595 個參數,中心氣壓計算網格為 65×65、約 20 公里。這是計算與重建間距,不是新取得的 20 公里原生觀測。兩天是本次模型固定的輸入設計,尚未以相同條件比較一天、兩天及三天,不能說兩天已被證明最佳。[完整技術輸入與輸出](../models/trackformer_1_2_field/README.md)。
|
| 661 |
+
|
| 662 |
+
### 為什麼有 50 組預測
|
| 663 |
+
|
| 664 |
+
報告使用**同一組模型參數**,只在正規化後的歷史天氣加入 50 組不同的小幅、平滑、平均為零的雜訊,再取平均。環境與區域資料的雜訊幅度分別為 0.025、0.015,平滑範圍分別為 5、9 格;幅度是正規化單位,並非直接加上相同數字的百帕。颱風起報位置、移動、強度、地形與有效性標記不變,原始觀測和模型參數也不修改。
|
| 665 |
+
|
| 666 |
+
路徑與中心氣壓分別平均;氣壓地圖則先把各組結果放到相同經緯度,再平均。這**不是訓練 50 個模型**,也未證明 50 組是最佳數量、一定優於單次預測,或其分散程度已是可靠的不確定性範圍。
|
| 667 |
+
|
| 668 |
+
## 多個颱風的比較
|
| 669 |
+
|
| 670 |
+
路徑使用 **270 個颱風、1,473 次起報**,比較每六小時、未來五天的預測。先平均一次起報內各時間的有效誤差,再平均同一颱風的起報,最後讓每個颱風具有相同權重。
|
| 671 |
+
|
| 672 |
+
**中心氣壓比較使用270個颱風中符合共同評分條件的40個颱風,共134次起報。將本研究模型與DeepMind輕量版模型在相同預報時間的中心氣壓預測,分別與日本氣象廳紀錄比較,共有2,637筆共同有效紀錄。**
|
| 673 |
+
|
| 674 |
+
這不代表其餘 230 個颱風都沒有氣壓觀測;本次共同子集沿用原評估流程的起報強度有效條件。缺值不當成零誤差。
|
| 675 |
+
|
| 676 |
+
比較模型是官方 **WeatherNext Cyclones Mini `<2024`**,訓練資料到 2023 年,原生間距 1 度,使用一個預測成員及軟體 v0.3.0,並非 DeepMind 完整大型模型。TrackFormer 使用 50 組輸入擾動平均,兩者保留各自資料流程,不是等算力的模型結構實驗。
|
| 677 |
+
|
| 678 |
+

|
| 679 |
+
|
| 680 |
+
第一至第五天分別指 **24、48、72、96、120 小時**,不是一天內所有時段的平均。路徑為 270 個颱風;氣壓依各時間的共同有效紀錄使用 39–40 個颱風,樣本數列於圖下。[逐日數據](report_figures/day_1_to_5.json)。
|
| 681 |
+
|
| 682 |
+
| 比較項目 | TrackFormer 1.2 | DeepMind 輕量版 |
|
| 683 |
+
| --- | ---: | ---: |
|
| 684 |
+
| 平均路徑誤差(公里) | 471.20 | 498.37 |
|
| 685 |
+
| 第五天路徑誤差(公里) | 1031.63 | 1031.91 |
|
| 686 |
+
| 六小時移動方向誤差(度) | 34.96 | 39.76 |
|
| 687 |
+
| 中心氣壓平均絕對誤差(百帕) | 12.84 | 10.20 |
|
| 688 |
+
|
| 689 |
+
這組開發評估的平均路徑及移動方向誤差較低,中心氣壓誤差則以比較模型較低。全組包含 230 個 1980–1999 年颱風及 40 個較近期颱風;歷史年份與比較模型的訓練期間重疊。只看 2025–2026 年起報的 18 個颱風、61 次起報,DeepMind 輕量版路徑誤差為 258.7 公里,TrackFormer 為 469.4 公里。已反覆查看的案例屬開發證據,不是全新、未使用的測試,不能由全組平均推論普遍優於 DeepMind。[原始完成數據與流程](#docs-deepmind-daily-benchmark-md)。
|
| 690 |
+
|
| 691 |
+
## 三個颱風展示
|
| 692 |
+
|
| 693 |
+
三案皆以相同發布模型的 50 組預測平均展示。氣壓圖比較 +48 小時、相同座標及色階下的模型與 **NCEP 參考氣壓**,採 2.5 度共同網格;NCEP 仍為分析資料,不因圖中文字簡化而變成測站觀測。路徑與中心氣壓保留全部 +6 至 +120 小時,以 IBTrACS/日本氣象廳紀錄對照,不搬移預測低壓來配合實況。
|
| 694 |
+
|
| 695 |
+
### 鳳凰(Fung-wong,2025)
|
| 696 |
+
|
| 697 |
+
起報:**2025-11-07 00 UTC**。使用保存的 Metal 結果。
|
| 698 |
+
|
| 699 |
+

|
| 700 |
+
|
| 701 |
+

|
| 702 |
+
|
| 703 |
+

|
| 704 |
+
|
| 705 |
+
### 山竹(Mangkhut,2018)
|
| 706 |
+
|
| 707 |
+
起報:**2018-09-11 00 UTC**。使用保存的 Metal 結果;位於訓練年份內,為回溯展示。
|
| 708 |
+
|
| 709 |
+

|
| 710 |
+
|
| 711 |
+

|
| 712 |
+
|
| 713 |
+

|
| 714 |
+
|
| 715 |
+
### 燕子(Jebi,2018)
|
| 716 |
+
|
| 717 |
+
起報:**2018-08-31 00 UTC**。相同發布權重在 CPU 執行的 50 組預測;未重新訓練,也未使用起報以後的天氣輸入。位於訓練年份內,為回溯展示。
|
| 718 |
+
|
| 719 |
+

|
| 720 |
+
|
| 721 |
+

|
| 722 |
+
|
| 723 |
+

|
| 724 |
+
|
| 725 |
+
三張風圈圖均是 **IBTrACS 的 34 節四方向觀測半徑,不是本模型半徑預測**。左為 +48 小時四方向最大範圍,右為五日紀錄;海里乘以 1.852 換成公里,缺值保留空白。扇形不是完整風場。發布模型沒有原生風圈半徑預測頭。[圖片雜湊與來源](report_figures/provenance.json)。
|
| 726 |
+
|
| 727 |
+
## 結論、成果特色與後續研究
|
| 728 |
+
|
| 729 |
+
- **研究結論:**以家用電腦可訓練、可執行為核心目標,完成五日氣壓、颱風路徑及中心氣壓的整合流程,並可在一般 Mac 電腦執行預測與評估。
|
| 730 |
+
- **同一套預測:**地圖、路徑與中心氣壓來自同一模型結果,可相互對照。
|
| 731 |
+
- **周圍天氣與颱風中心一起更新:**同時保留環境氣壓、風場及移動中的中心氣壓計算。
|
| 732 |
+
- **彩色圖表與清楚資料流程:**以相同時間、座標比較三個颱風,保留資料年份、案例數、缺值與來源紀錄。
|
| 733 |
+
- **應用:**提供個人及學生進行氣象學習、歷史颱風比較與地圖展示的研究工具。這不是取代官方警報的作業預報服務。
|
| 734 |
+
- **持續研究:**結合臺灣島外風向、水氣輸送與地形,研究一般降雨和強降雨的分布,並以降雨觀測另行評估。**發布的 1.2 不包含降雨預測頭或水氣輸入**,本段不是已完成的降雨準確度成果。
|
| 735 |
+
|
| 736 |
+
|
| 737 |
+
<a id="docs-showcase-archive-md"></a>
|
| 738 |
+
|
| 739 |
+
## docs/showcase_archive.md
|
| 740 |
+
|
| 741 |
+
# Forecast examples and diagnostics
|
| 742 |
+
|
| 743 |
+
The [model announcement](../README.md) features the geographically registered moving-core Mangkhut (2018) forecast. Its 50-member arrays and replay/playback receipts are linked there. These are development illustrations, not representative or untouched-test evidence.
|
| 744 |
+
|
| 745 |
+
## Current research-report figures
|
| 746 |
+
|
| 747 |
+
The [Chinese report explanation](#docs-research-report-zh-md) contains the current **Fung-wong (2025), Mangkhut (2018) and Jebi (2018)** pressure-map, five-day route/central-pressure and observed-radius panels. The day-1–5 bar chart compares released 1.2 with DeepMind Mini only, using unchanged saved scores. [Figure hashes and source identities](report_figures/provenance.json).
|
| 748 |
+
|
| 749 |
+
The report's radius figures are **IBTrACS observed R34 quadrant records, not model predictions**. The pressure reference remains NCEP analysis, with a common 2.5° comparison grid and no shifted forecasts. Mangkhut and Jebi are in the training years; the examples are retrospective demonstrations, not additional untouched validation. Older films and experimental radius diagnostics below keep their original provenance and should not be confused with these report figures.
|
| 750 |
+
|
| 751 |
+
## Additional films
|
| 752 |
+
|
| 753 |
+
These earlier 50-member films retain a fixed issue-relative regional pressure patch. When the centre leaves it, only the saved coarse basin field covers that position. They use 4 hPa isobars and 23-second playback with a held ending; do not confuse them with the current moving-core Mangkhut rendering. Their original model outputs remain preserved.
|
| 754 |
+
|
| 755 |
+
- [Fung-wong (2025), 7 November 00 UTC](https://huggingface.co/euler314/typhoon-predict/resolve/87a6e366b42bb4cc0edc95d2c50c55fca21a2c93/docs/trackformer_1_2_fung_wong.mp4) · [provenance](../evaluation/release_data/fung_wong_video.json)
|
| 756 |
+
- [Soudelor (2015), 5 August 00 UTC](https://huggingface.co/euler314/typhoon-predict/resolve/87a6e366b42bb4cc0edc95d2c50c55fca21a2c93/docs/trackformer_1_2_soudelor.mp4) · [provenance](../evaluation/release_data/soudelor_video.json)
|
| 757 |
+
- [Meranti (2016), 10 September 00 UTC](https://huggingface.co/euler314/typhoon-predict/resolve/87a6e366b42bb4cc0edc95d2c50c55fca21a2c93/docs/trackformer_1_2_meranti.mp4) · [provenance](../evaluation/release_data/meranti_video.json)
|
| 758 |
+
|
| 759 |
+
## Wind and structure
|
| 760 |
+
|
| 761 |
+
[Four-storm wind comparison](../evaluation/storm_structure/four_storm_wind_candidate.png) · [audit index](../evaluation/storm_structure/index.json) · [experimental structure method](#release-tools-storm-structure-diagnostics-md)
|
| 762 |
+
|
| 763 |
+
USA one-minute and JMA ten-minute winds are separate references. Native 1.1 RMW and quadrant R34/R50/R64 are scored against matching USA definitions. Trackformer 1.2 has an auxiliary maximum-wind scalar but no native radius head. Its pressure-derived radius candidate is not an official-equivalent prediction; masked/persistent components are not learned radius forecasts.
|
| 764 |
+
|
| 765 |
+
| Storm | Exact-time comparisons | Data and assumptions |
|
| 766 |
+
| --- | --- | --- |
|
| 767 |
+
| Fung-wong | [Plot](../evaluation/storm_structure/fung_wong_wind_radius.png) | [Record](../evaluation/storm_structure/fung_wong_structure.json) |
|
| 768 |
+
| Soudelor | [Plot](../evaluation/storm_structure/soudelor_wind_radius.png) | [Record](../evaluation/storm_structure/soudelor_structure.json) |
|
| 769 |
+
| Mangkhut | [Plot](../evaluation/storm_structure/mangkhut_wind_radius.png) | [Record](../evaluation/storm_structure/mangkhut_structure.json) |
|
| 770 |
+
| Meranti | [Plot](../evaluation/storm_structure/meranti_wind_radius.png) | [Record](../evaluation/storm_structure/meranti_structure.json) |
|
| 771 |
+
|
| 772 |
+
## Full evaluation
|
| 773 |
+
|
| 774 |
+
[Daily protocol](#docs-daily-storm-benchmark-md) · [all daily scores](../evaluation/daily_storm_final.json) · [intensity/radius protocol](#docs-intensity-benchmark-md) · [native intensity results](../evaluation/intensity/intensity_final.json) · [matched pressure curves](../evaluation/released_daily/pressure_comparison.png)
|
| 775 |
+
|
| 776 |
+
Route uses 1,473 starts / 270 storms; the matched native pressure comparison uses 134 starts / 40 storms. JMA pressure MAE falls from 13.53 to 12.84 hPa, with paired uncertainty including zero. JMA curve similarity is 0.7074 / 0.7118 (1.1 / 1.2), while USA curve similarity is 0.7237 / 0.6637 on 131 eligible days—a regression. A centred shape score discards level and amplitude and must be read with physical hPa timelines, not used to conceal pressure errors.
|
| 777 |
+
|
| 778 |
+
|
| 779 |
+
<a id="docs-trackformer-1-2-evaluation-md"></a>
|
| 780 |
+
|
| 781 |
+
## docs/trackformer_1_2_evaluation.md
|
| 782 |
+
|
| 783 |
+
# Trackformer 1.2 evaluation notes
|
| 784 |
+
|
| 785 |
+
## Completed equal-storm benchmark
|
| 786 |
+
|
| 787 |
+
The current comparison uses **1,473 daily forecasts across 270 Western Pacific storms**, with 20 six-hour leads from +6 through +120 h. Scores average leads within issues, daily issues within each storm, then storms equally. The completed results and all forecast SHA-256 checks are in [the final report](../evaluation/daily_storm_final.json); the [daily-storm protocol](#docs-daily-storm-benchmark-md) defines the frozen cohort and metrics.
|
| 788 |
+
|
| 789 |
+
The current [Chinese research-report view](#docs-research-report-zh-md) compares **released 1.2 with DeepMind Mini**, using unchanged saved results. TrackFormer uses the mean of 50 distinct causal input perturbations of one model, not 50 trained networks; Mini uses one seeded member. These are matched outputs with different inputs and member policies, not a controlled architecture ablation, equal-compute test or untouched generalization test.
|
| 790 |
+
|
| 791 |
+
| Report metric | TrackFormer 1.2 | DeepMind Mini |
|
| 792 |
+
| --- | ---: | ---: |
|
| 793 |
+
| Mean track error | 471.20 km | 498.37 km |
|
| 794 |
+
| +120 h track error | 1031.63 km | 1031.91 km |
|
| 795 |
+
| Six-hour direction error | 34.96° | 39.76° |
|
| 796 |
+
| Central-pressure MAE · JMA | 12.84 hPa | 10.20 hPa |
|
| 797 |
+
|
| 798 |
+
Direction, centred shape, geographic path and same-time position error measure different aspects of a forecast. A translated or mistimed route can have high shape similarity without overlapping the observed route. No combined weighted ranking is introduced.
|
| 799 |
+
|
| 800 |
+
**Central-pressure coverage is a subset of route coverage:** **40 of the 270 storms / 134 starts / 2,637 common valid lead records**. Each model's pressure is compared separately with the JMA record at the identical forecast time. This is not a separate 270-storm pressure evaluation. The original 1.2-only pressure mean of **12.62 hPa** uses a different mask and must not replace the report's common-support **12.84 hPa**. The original basin MSLP mean of **2.72 hPa on 270 storms** is a whole-map measure, not central-pressure error. [Common scores](../evaluation/released_daily/released_daily_benchmark.json) · [Pressure protocol](#docs-intensity-benchmark-md).
|
| 801 |
+
|
| 802 |
+
The inherited shared-case eligibility excludes **1,339 starts with unavailable frozen legacy issue-time intensity inputs**. That does **not** mean all excluded storms lack pressure observations. Original masks remain unchanged; unsupported scores are unavailable, not zero. The cohort contains 40 recent and 230 historical storms from 1980–1999. Retrospective analyses, complete-five-day eligibility and prior model selection limit interpretation.
|
| 803 |
+
|
| 804 |
+
The [report's day-1–5 chart](report_figures/day_1_to_5.png) shows **exact 24, 48, 72, 96 and 120 h** leads rather than daily-bin averages. Track covers 270 storms at each displayed lead. JMA pressure has 40, 40, 40, 40 and 39 storms, with 131, 133, 134, 133 and 127 common starts respectively. All-lead pressure totals and per-lead counts describe different aggregations. [Exact chart values](report_figures/day_1_to_5.json).
|
| 805 |
+
|
| 806 |
+
## Completed DeepMind comparison
|
| 807 |
+
|
| 808 |
+
Google's official **WeatherNext Cyclones Mini `<2024`**, using **WeatherNext software v0.3.0**, completed the same **1,473 daily starts / 270 storms** on an **NVIDIA GeForce RTX 3070 with CUDA**. It is a 1° model trained through 2023. Mini is one seeded member, not the full-sized WeatherNext model; 1.2 remains a 50-member mean. The common-support results are **498.37 km** mean track error, **1031.91 km** at +120 h, **39.76°** direction error and **10.20 hPa** JMA central-pressure MAE on **134 starts / 40 storms / 2,637 valid records**. Its JMA pressure-curve similarity is **0.8088**.
|
| 809 |
+
|
| 810 |
+
Mini's recent-only mean track error is **288.5 km** versus **477.2 km** for 1.2. Historical fitting-year overlap and different input/member policies limit the combined-cohort comparison; no general superiority claim is justified. The [completed protocol and period breakdowns](#docs-deepmind-daily-benchmark-md), [CUDA receipt](../evaluation/deepmind_daily/verification.json) and [independent three-model publication audit](../evaluation/deepmind_daily/publication_audit.json) establish completion. The older two-model receipt remains the original Trackformer audit, not the hash receipt for the subsequently merged snapshot. No missing or unsupported score is treated as zero.
|
| 811 |
+
|
| 812 |
+
That **recent-only** group includes 2024. The report-facing strict **after-2024** group is different: 2025–2026, 61 starts / 18 storms, with **258.7 km Mini versus 469.4 km 1.2**. Calendar 2024 is not part of that strict subset. Do not interchange these period scores.
|
| 813 |
+
|
| 814 |
+
## Report showcases and preserved earlier records
|
| 815 |
+
|
| 816 |
+
The [three report examples](#docs-research-report-zh-md) use unchanged released-1.2 weights: Fung-wong and Mangkhut are saved Metal forecasts; Jebi was run on CPU. All use 50 input-perturbed members and exact +6 to +120 h times. Mangkhut and Jebi are within the fitting years. Their R34 panels show **IBTrACS observations only**, not model radius skill. NCEP is still an analysis reference even where the figure label is simplified to “reference pressure”.
|
| 817 |
+
|
| 818 |
+
Earlier release comparisons and their pressure-curve diagnostics remain in the [archived shared snapshot](../evaluation/released_daily/released_daily_benchmark.json), [full original report](../evaluation/daily_storm_final.json) and [intensity protocol](#docs-intensity-benchmark-md). They are not silently substituted for the current report's two-model table. Centred curve similarity removes level and amplitude; read it with hPa MAE, masks and original timelines rather than treating it as accurate pressure levels.
|
| 819 |
+
|
| 820 |
+
## Recent pressure-line example
|
| 821 |
+
|
| 822 |
+
### Fung-wong MP4: selected route-and-pressure example
|
| 823 |
+
|
| 824 |
+
The [Fung-wong video](https://yu314-coder.github.io/typhoon-predict/trackformer_1_2_fung_wong.mp4) uses the unchanged saved 50-member pressure fields and mean route for case 262, issued 2025-11-07 00 UTC. It is a user-selected development example after inspection of full routes and pressure curves, not an untouched test or a representative sample. The geographic mean error is 130.9 km, the +120 h error is 142.3 km, and maximum error is 198.3 km over 20 leads. These are great-circle distances to exact observed coordinates, separate from the main benchmark's saved local-coordinate scores. The route follows the broad observed curve but is not perfectly overlapping.
|
| 825 |
+
|
| 826 |
+
The pressure MAE is **7.36 hPa** against IBTrACS TOKYO_PRES (JMA best track), with 20 valid labels. Intensification and weakening still differ from observations. Fields are physical model means on fixed common grids, without shifting them onto the observed track. Native-detail history was unavailable; 0.25° regional output sampling is not native resolution. From +66 h the forecast centre leaves the regional patch: only saved coarse basin pressure is shown there, while member-mean central pressure remains the moving-core readout. No display-only core is added. Observed future routes and pressures are verification overlays only. [Data and provenance](../evaluation/release_data/fung_wong_video.json) record hashes and member policy. Reproduce with `python release_tools/build_fung_wong_video.py --work /path/to/frames` using NumPy, Matplotlib and ffmpeg with VideoToolbox on macOS; this renders saved outputs without model inference. The former interactive showcase has been removed.
|
| 827 |
+
|
| 828 |
+
### Recent single-member example
|
| 829 |
+
|
| 830 |
+
The Surigae archive issued **2026-09-27 12 UTC** supplies real model regional MSLP fields at +6, +24 and +36 h. Isobars are drawn every 4 hPa, with selected labels every 12 hPa and one common colour scale. The magenta curve is the saved model route. Fields are neither relocated nor altered to force the map minimum onto the route. The regional map is a resampled composite and can differ slightly from central pressure sampled in the moving core.
|
| 831 |
+
|
| 832 |
+
This example is one deterministic member from the released checkpoint. Inputs contain nine GFS analyses ending at 06 UTC (six hours before issue), issue-time JMA centre/central pressure, and static geography. Issue-time wind, prior motion and native regional history were unavailable and masked/zeroed. The stored +0 field is an input analysis; it is excluded from these forecast panels. The GFS transfer and input lag remain limitations. The selected leads keep the centre within the fixed 114–144°E, 11–41°N map. Names, times, member count and checkpoint/source hashes are in `evaluation/release_data/surigae_provenance.json`.
|
| 833 |
+
|
| 834 |
+
## Reproduce the benchmark chart
|
| 835 |
+
|
| 836 |
+
```bash
|
| 837 |
+
python release_tools/plot_daily_storm_final.py
|
| 838 |
+
```
|
| 839 |
+
|
| 840 |
+
This plots the saved completed equal-storm results; it performs no new inference or weather retrieval. Original research arrays and reproduction tools remain preserved for provenance, but the retired smaller-cohort comparison is not presented as the current benchmark.
|
| 841 |
+
|
| 842 |
+
## Validation boundary
|
| 843 |
+
|
| 844 |
+
Training used whole-storm year partitions: 2000–2021 training, 2022–2023 validation, 2024–2025 test in the original pipeline, with 13,949 / 1,041 / 1,195 windows respectively; native-pressure windows total 800 / 100 / 100. Repeatedly viewed benchmarks, TIP and calibration cases have become development evidence. Freeze a genuinely unused storm-level holdout with matched issue-time data before claiming generalization. Evaluate route, central pressure and pressure fields at every lead on common cases and masks; a lower map average alone is not a model win.
|
| 845 |
+
|
| 846 |
+
The fitting sources are IBTrACS-derived storm records, NOAA PSL NCEP/NCAR basin analyses, ARCO-ERA5 regional MSLP and static ERA5 land fraction/elevation. Newer 2026 archive records are not part of the released model's gradient fitting. See [training-source documentation](TrackFormer_1.2_Training_and_Reproducibility.pdf) and [checksum-verified split receipt](../evaluation/training_data/trackformer_1_2_provenance.json).
|
| 847 |
+
|
| 848 |
+
|
| 849 |
+
<a id="evaluation-readme-md"></a>
|
| 850 |
+
|
| 851 |
+
## evaluation/README.md
|
| 852 |
+
|
| 853 |
+
# Evaluation assets
|
| 854 |
+
|
| 855 |
+
## Research-report view
|
| 856 |
+
|
| 857 |
+
The current [report-facing summary](../README.md#results-in-the-research-report) and [Chinese explanation](#docs-research-report-zh-md) compare **TrackFormer 1.2 with DeepMind Mini**. Their [day-1–5 chart](report_figures/day_1_to_5.png) uses exact 24/48/72/96/120 h leads. Route covers **270 storms / 1,473 starts**; central pressure uses **40 of those storms / 134 starts / 2,637 common valid lead records**. The pressure subset follows frozen scoring eligibility, not a claim that the remaining storms lack all pressure observations. Source arrays and the larger historical archive below are unchanged.
|
| 858 |
+
|
| 859 |
+
## Full historical archive
|
| 860 |
+
|
| 861 |
+
The current **completed three-model comparison** scores Trackformer 1.1, Trackformer 1.2 and Google's **WeatherNext Cyclones Mini `<2024` (software v0.3.0)** on **1,473 daily starts / 270 Western Pacific storms**, through +120 h. Mini's RTX 3070 CUDA run is complete and verified; Trackformer 1.2 is a mean of 50 members and Mini is one member.
|
| 862 |
+
|
| 863 |
+
- [Announcement chart](../evaluation/released_daily/model_1_2_benchmark.png) and [shared three-model scores](../evaluation/released_daily/released_daily_benchmark.json).
|
| 864 |
+
- [Completed DeepMind results](../evaluation/deepmind_daily/benchmark.json), [CUDA completion receipt](../evaluation/deepmind_daily/verification.json) and [independent publication audit](../evaluation/deepmind_daily/publication_audit.json).
|
| 865 |
+
- [Daily-case protocol](#docs-daily-storm-benchmark-md), [DeepMind model identity and period breakdowns](#docs-deepmind-daily-benchmark-md) and [Trackformer evaluation definitions](#docs-trackformer-1-2-evaluation-md).
|
| 866 |
+
- [Matched pressure, wind and radius evaluation](#docs-intensity-benchmark-md): pressure uses 134 shared daily starts / 40 storms, not all 270 storms. Unsupported scores stay unavailable, never zero.
|
| 867 |
+
|
| 868 |
+
Scores average valid leads within each day, days within each storm, then storms equally. The historical 1980–1999 group overlaps Mini's fitting years; recent 2024+ results are reported separately. This is development evidence, not a certified untouched holdout or an equal-compute architecture ablation.
|
| 869 |
+
|
| 870 |
+
For geographically aligned routes, compare forecasts and observations at the **same valid times on the same map**. Centred shape similarity alone cannot establish alignment. Never translate, rotate or rescale a forecast to make it match the observed route.
|
| 871 |
+
|
| 872 |
+
Reproduce the current three-model announcement chart from saved results with `python release_tools/plot_model_announcement.py`. This renders audited scores without new inference or weather retrieval; NumPy and Matplotlib are required. The paper is in [`../paper/`](../paper), and saved pressure arrays and preview provenance are under [`release_data/`](../evaluation/release_data).
|
| 873 |
+
|
| 874 |
+
## Archived diagnostics and selected examples
|
| 875 |
+
|
| 876 |
+
Older figures, the [TIP diagnostic](../evaluation/trackformer_1_2_vs_1_1_tip_metrics.json) and [showcase selection](../evaluation/trackformer_1_2_showcase_selection.json) are preserved for provenance, not presented as the current benchmark. See the [showcase archive](#docs-showcase-archive-md) for selected pressure-map films and their limitations.
|
| 877 |
+
|
| 878 |
+
|
| 879 |
+
<a id="models-trackformer-1-2-field-wind-estimation-md"></a>
|
| 880 |
+
|
| 881 |
+
## models/trackformer_1_2_field/WIND_ESTIMATION.md
|
| 882 |
+
|
| 883 |
+
# Experimental wind and circular-radius diagnostics
|
| 884 |
+
|
| 885 |
+
The frozen 1.2 weights and neural architecture are unchanged. Maximum wind is
|
| 886 |
+
the existing learned auxiliary `vmax` output, in knots, not a resolved wind map.
|
| 887 |
+
Its averaging period is not harmonized to JMA, and skill has not been established
|
| 888 |
+
by the route/pressure benchmark. Do not treat this product as operational advice.
|
| 889 |
+
|
| 890 |
+
`wind_estimation.py` optionally diagnoses each model-generated pressure member:
|
| 891 |
+
|
| 892 |
+
1. Sample complete circular rings about that member's predicted centre, no more
|
| 893 |
+
finely than the source/reconstruction grid, out to at most 480 km.
|
| 894 |
+
2. Compute the azimuthal-mean radial pressure gradient (hPa converted to Pa).
|
| 895 |
+
3. Solve `Vg^2/r + |f| Vg = (1/rho) dp/dr`, with air density 1.15 kg/m3, then
|
| 896 |
+
apply a fixed 0.8 ocean surface-reduction assumption. This is uncalibrated;
|
| 897 |
+
motion, land friction, asymmetry, boundary-layer jets and averaging periods
|
| 898 |
+
are not represented. No future observations or official routes are inputs.
|
| 899 |
+
4. Report a resolved gradient-wind peak/RMW and outward 34/50/64-kt crossings.
|
| 900 |
+
These are **circular equivalent radius estimates**, not quadrant radii and
|
| 901 |
+
not isobar radii. Coarse (>50 km) source grids, incomplete rings, unresolved
|
| 902 |
+
peaks and missing crossings produce nulls, not invented zeros or edge radii.
|
| 903 |
+
5. Diagnose members individually, then summarize. A mean is supplied only when
|
| 904 |
+
every member has that estimate. Valid counts and p10/p90 describe spread;
|
| 905 |
+
they are not calibrated confidence intervals. Never diagnose a mean map and
|
| 906 |
+
call it a mean of 50 radius predictions.
|
| 907 |
+
|
| 908 |
+
The 20-km moving core is a learned reconstruction. A resampled 0.25-degree
|
| 909 |
+
regional image does not add resolved observations. Even an available estimate
|
| 910 |
+
therefore remains experimental. A single gridded basin mean cannot reconstruct
|
| 911 |
+
the missing member diagnostics. Legacy forecasts without saved member/core
|
| 912 |
+
outputs retain unavailable estimates instead of being rerun or relabelled.
|
| 913 |
+
|
| 914 |
+
For the saved 50-member regional composite, core origins were not archived.
|
| 915 |
+
That release's core spans +/-640 km and centre association is restricted to
|
| 916 |
+
300 km about its origin. Consequently this read-only legacy exporter samples
|
| 917 |
+
only rings within 300 km of each predicted centre, conservatively inside the
|
| 918 |
+
core footprint; a missing crossing beyond that limit remains unavailable.
|
| 919 |
+
New forecasts use the actual moving-core coordinates and their valid masks.
|
| 920 |
+
|
| 921 |
+
Defaults are fixed physical assumptions, not fitted to showcase outcomes. Before
|
| 922 |
+
claiming skill, freeze eligible whole-storm held-out cases; score +6 to +120 h
|
| 923 |
+
maximum winds and agency-compatible wind radii separately, with coverage,
|
| 924 |
+
unresolved cases, land/ocean breakdowns and whole-storm uncertainty. No such
|
| 925 |
+
validation or calibration is claimed by this implementation.
|
| 926 |
+
|
| 927 |
+
References: [NOAA PaHM theory and limitations](https://noaa-ocs-modeling.github.io/PaHM/pahm_manual.pdf)
|
| 928 |
+
and [Chavas, Reed and Knaff (2017)](https://doi.org/10.1038/s41467-017-01546-9).
|
| 929 |
+
|
| 930 |
+
|
| 931 |
+
<a id="release-tools-storm-structure-diagnostics-md"></a>
|
| 932 |
+
|
| 933 |
+
## release_tools/STORM_STRUCTURE_DIAGNOSTICS.md
|
| 934 |
+
|
| 935 |
+
# Any-storm wind and radius plots
|
| 936 |
+
|
| 937 |
+
This is a local diagnostic/plotting extension. It does not change the released
|
| 938 |
+
Trackformer 1.2 weights, existing pressure maps, videos, cloud archive, or Site.
|
| 939 |
+
It is **not** a new validated wind/radius forecast model.
|
| 940 |
+
|
| 941 |
+
## What works
|
| 942 |
+
|
| 943 |
+
- The renderer accepts any exact SID in the local Western Pacific IBTrACS file.
|
| 944 |
+
The October 1 local snapshot contains 1,910 storms since 1970. This is an
|
| 945 |
+
observational coverage catalogue, **not** a claim that 1,910 forecasts ran.
|
| 946 |
+
- Every storm can produce a chart layout. Pressure, wind, RMW, and quadrant
|
| 947 |
+
plots are drawn where reports exist. An empty panel explains missing data.
|
| 948 |
+
- Forecast plotting uses genuine, hashed physical member archives. All four
|
| 949 |
+
examples now have 20 candidate wind and R34/R50/R64 quadrant outputs through
|
| 950 |
+
+120 h, diagnosed from all 50 members before taking the mean.
|
| 951 |
+
- The historical MPS runner additionally accepts arbitrary `--case SLUG SID
|
| 952 |
+
ISSUE_UTC`. It still requires nine exact six-hour analyses in the existing
|
| 953 |
+
local atlas and the released WP domain. It neither downloads missing weather
|
| 954 |
+
nor changes the independently authorized cloud backfill.
|
| 955 |
+
|
| 956 |
+
## Candidate assumptions and official definitions
|
| 957 |
+
|
| 958 |
+
IBTrACS `USA_WIND` is a 1-minute reference. USA R34/R50/R64 are maximum extents
|
| 959 |
+
in the NE/SE/SW/NW quadrants, in nautical miles; multiply by 1.852 for **radius**
|
| 960 |
+
in km, never by two. JMA wind is a separate 10-minute reference. JMA R30 cannot
|
| 961 |
+
be relabeled R34, and longest/shortest JMA radii are not USA quadrant radii.
|
| 962 |
+
Blank values remain missing; a native reported zero remains zero. USA RMW is
|
| 963 |
+
not universally reanalyzed best track, especially in WP. Fung-Wong's selected
|
| 964 |
+
2025 USA reports are provisional TCVitals, not final JTWC observations.
|
| 965 |
+
See the [NOAA IBTrACS column documentation](https://www.ncei.noaa.gov/sites/default/files/2025-09/IBTrACS_v04r01_column_documentation.pdf).
|
| 966 |
+
|
| 967 |
+
The new experimental extension starts with only the **exact issue-time** USA
|
| 968 |
+
wind, RMW and available quadrant isotachs. No future wind/radius is fitted.
|
| 969 |
+
The neural runner itself continues to use its original JMA issue-intensity
|
| 970 |
+
inputs; USA structure is a separate diagnostic input, not a silent neural
|
| 971 |
+
input substitution. Reference and model centers can differ, and both are
|
| 972 |
+
recorded in the initialization receipt.
|
| 973 |
+
|
| 974 |
+
For each forecast member, ambient pressure is the median actual basin cells
|
| 975 |
+
600–1000 km from that member's predicted center, with at least two available
|
| 976 |
+
cells in each quadrant. This uses coarse pressure for the **ambient value**,
|
| 977 |
+
not to claim a resolved eyewall. Let `D0 = Pambient0 - Pmodel_issue` and
|
| 978 |
+
`Dm = Pambient_member - Pcentral_member`. The candidate uses
|
| 979 |
+
|
| 980 |
+
```text
|
| 981 |
+
wind_member = USA_wind_issue * sqrt(Dm / D0)
|
| 982 |
+
```
|
| 983 |
+
|
| 984 |
+
The square-root deficit scaling is motivated by gradient/Holland balance,
|
| 985 |
+
but this extension is **not PaHM/GAHM**. It builds a decreasing outer envelope
|
| 986 |
+
from consistent issue-time quadrant isotachs, interpolates in log-radius and
|
| 987 |
+
log-wind, and extrapolates with a Holland-inspired `V ~ r^(-B/2)` tail. A fixed
|
| 988 |
+
literature-motivated B range 1–2.5 is used, without showcase tuning. Radius
|
| 989 |
+
solutions beyond 1,500 km remain unavailable rather than becoming a grid-edge
|
| 990 |
+
radius. Nonphysical pressure, unresolved backgrounds, out-of-domain tracks and
|
| 991 |
+
inconsistent zero isotachs remain explicit failures.
|
| 992 |
+
The [NOAA PaHM theory](https://noaa-ocs-modeling.github.io/PaHM/html/models.html)
|
| 993 |
+
also explains why gradient wind is not automatically a validated 10-m wind.
|
| 994 |
+
|
| 995 |
+
RMW is currently **issue-time persistence**. The flat dashed RMW curve is not
|
| 996 |
+
a model-measured future RMW, an eye diameter, or proof of radius skill.
|
| 997 |
+
The candidate's nominal threshold/units/quadrant geometry target the native
|
| 998 |
+
USA definition, but its 1-minute surface-wind averaging behavior has not been
|
| 999 |
+
validated. The strict scorer therefore rejects wind/radius skill claims.
|
| 1000 |
+
Plots are exploratory reference comparisons, not a fair validated benchmark.
|
| 1001 |
+
All four are development examples; no unused generalization holdout is claimed.
|
| 1002 |
+
|
| 1003 |
+
## Reusable commands
|
| 1004 |
+
|
| 1005 |
+
Use the existing Python environment; no installation is needed. All outputs,
|
| 1006 |
+
temporary files and Matplotlib caches must stay on `/Volumes/D`.
|
| 1007 |
+
|
| 1008 |
+
Draw a full observed storm, even without a model-member archive:
|
| 1009 |
+
|
| 1010 |
+
```sh
|
| 1011 |
+
/Volumes/D/typhoon_predict/.venv/bin/python -B \
|
| 1012 |
+
/Volumes/D/typhoon_predict/remote_typhoon_predict_repo/release_tools/plot_storm_wind_radius.py \
|
| 1013 |
+
--ibtracs /Volumes/D/typhoon_predict/data/ibtracs/ibtracs.WP.list.v04r01.csv \
|
| 1014 |
+
--output /Volumes/D/typhoon_predict/output/nari_observed_plots \
|
| 1015 |
+
--observed-storm 2001248N23125
|
| 1016 |
+
```
|
| 1017 |
+
|
| 1018 |
+
Generate the data-coverage catalogue with `--catalogue-only`. To draw a saved
|
| 1019 |
+
model ensemble, pass `--forecast-folder` (repeatable), pointing to one immutable
|
| 1020 |
+
folder containing `verification.json`, `*_video.json`, `*_video.npz`,
|
| 1021 |
+
`ensemble-members.npz`, `weather-history.npz` and the original input receipt.
|
| 1022 |
+
|
| 1023 |
+
Create an explicit additional historical ensemble on the Mac GPU only when
|
| 1024 |
+
the required nine local analyses and exact issue report exist:
|
| 1025 |
+
|
| 1026 |
+
```sh
|
| 1027 |
+
/Volumes/D/typhoon_predict/.venv/bin/python -B \
|
| 1028 |
+
/Volumes/D/typhoon_predict/remote_typhoon_predict_repo/release_tools/forecast_historical_video50_mac.py \
|
| 1029 |
+
--project /Volumes/D/typhoon_predict \
|
| 1030 |
+
--output /Volumes/D/typhoon_predict/output/nari_ensemble50 \
|
| 1031 |
+
--case nari 2001248N23125 2001-09-08T00:00:00Z --chunk 5
|
| 1032 |
+
```
|
| 1033 |
+
|
| 1034 |
+
SID/date in a command is an explicit requested issue, not a nearest-date fill.
|
| 1035 |
+
If unavailable, choose an actually reported issue after checking the catalogue.
|
| 1036 |
+
The runner refuses duplicate issues, unsafe slugs, three-hour dates, changed
|
| 1037 |
+
completed slugs and unsupported causal input histories.
|
| 1038 |
+
|
| 1039 |
+
## October 1 verification
|
| 1040 |
+
|
| 1041 |
+
Local results: `/Volumes/D/typhoon_predict/output/all-storm-wind-radius.uSvmWY/plots`.
|
| 1042 |
+
Fung-Wong needed a fresh 50-member MPS replay (14.33 seconds inference). Its
|
| 1043 |
+
inputs/output differ from the older benchmark video and it is clearly labeled;
|
| 1044 |
+
the original video/data were not overwritten. Soudelor, Mangkhut and Meranti
|
| 1045 |
+
reuse the exact saved original member archives. All mean routes and common-grid
|
| 1046 |
+
pressure fields are rechecked against their saved video arrays. No inference
|
| 1047 |
+
or fitting is required for the three archived issues or for observed-only plots.
|
| 1048 |
+
|
| 1049 |
+
Before promotion: develop a genuine evolving RMW/outer structure model, fit any
|
| 1050 |
+
calibration only on declared whole-storm training data, freeze a genuinely unused
|
| 1051 |
+
storm cohort, validate surface-wind period/height and quadrant definitions, and
|
| 1052 |
+
report errors plus failed/missing coverage for identical leads through +120 h.
|
| 1053 |
+
|
| 1054 |
+
|
| 1055 |
+
<a id="training-trackformer-1-2-third-party-notices-md"></a>
|
| 1056 |
+
|
| 1057 |
+
## training/trackformer_1_2/THIRD_PARTY_NOTICES.md
|
| 1058 |
+
|
| 1059 |
+
# Data sources and attribution
|
| 1060 |
+
|
| 1061 |
+
The MIT license covers project code and project model artifacts; it does not replace upstream data terms.
|
| 1062 |
+
|
| 1063 |
+
- **NOAA PSL NCEP/NCAR Reanalysis 1**: basin mean sea-level pressure, 500 hPa height and winds at 850/500/200 hPa. Data provided by NOAA PSL, Boulder, Colorado, USA. https://psl.noaa.gov/data/gridded/data.ncep.reanalysis.html
|
| 1064 |
+
- **NOAA/NCEI IBTrACS**: derived storm observations, labels and masks; credit the original contributing meteorological agencies and NOAA/NCEI. Knapp et al. (2010), doi:10.1175/2009BAMS2755.1. https://www.ncei.noaa.gov/products/international-best-track-archive . Exact derived archive SHA is in assets.json; its original raw snapshot ID was not recorded.
|
| 1065 |
+
- **Copernicus Climate Change Service (C3S), ERA5, accessed through Google's ARCO-ERA5 mirror**: native mean-sea-level-pressure patches and static land fraction/elevation. Contains modified Copernicus Climate Change Service information (2026). ERA5 hourly single-level data, doi:10.24381/cds.adbb2d47, https://cds.climate.copernicus.eu/datasets/reanalysis-era5-single-levels . The current dataset lists a CC-BY licence; attribution and upstream terms apply. ECMWF or the European Commission does not endorse this model. ARCO mirror and recipes: https://github.com/google-research/arco-era5 . The mirror's code licence is separate from the underlying meteorological data licence.
|
| 1066 |
+
|
| 1067 |
+
The published arrays are transformed subsets: quantized/coarse basin grids, float16 pressure, issue-centred patches, normalized derived track features and masks. These transformations do not create new native observational resolution. No claim of endorsement by NOAA, ECMWF, C3S or Google is made.
|
| 1068 |
+
|
docs/archive/documentation-before-consolidation-20261030.zip
ADDED
|
Binary file (83.4 kB). View file
|
|
|
docs/archive/documentation-manifest.json
ADDED
|
@@ -0,0 +1,228 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"schema": "documentation-consolidation-v1",
|
| 3 |
+
"github_before": "b16332c5feb0ca3ba87715ebde4c4e375ef4a7aa",
|
| 4 |
+
"huggingface_before": "73e0f8f150182993773d4c549850ae2b1bbdf72c",
|
| 5 |
+
"final_markdown_paths": [
|
| 6 |
+
"README.md",
|
| 7 |
+
"docs/REFERENCE.md",
|
| 8 |
+
"models/trackformer_1_2_field/README.md"
|
| 9 |
+
],
|
| 10 |
+
"archive_path": "docs/archive/documentation-before-consolidation-20261030.zip",
|
| 11 |
+
"archive_sha256": "4ee4408366e76505415a0b2a3805a97543f73979f041ceec2bcbff0fb83cbc96",
|
| 12 |
+
"original_docs": [
|
| 13 |
+
{
|
| 14 |
+
"path": "CONTRIBUTING.md",
|
| 15 |
+
"sha256": "86e953e9a804566cefb142340d373eb00669a30937002857de8f2b46665ea11b",
|
| 16 |
+
"bytes": 2108,
|
| 17 |
+
"current_location": "docs/REFERENCE.md#contributing-md"
|
| 18 |
+
},
|
| 19 |
+
{
|
| 20 |
+
"path": "OPENNESS.md",
|
| 21 |
+
"sha256": "114b3dfef38441d29cb3a9e335c337f36598a760c192e925597466436304d8d9",
|
| 22 |
+
"bytes": 3640,
|
| 23 |
+
"current_location": "docs/REFERENCE.md#openness-md"
|
| 24 |
+
},
|
| 25 |
+
{
|
| 26 |
+
"path": "README.md",
|
| 27 |
+
"sha256": "abbaa7bbcdca3e86aca01ca46b183da5c202d3816561699b763b4ac9455be1d7",
|
| 28 |
+
"bytes": 23048,
|
| 29 |
+
"current_location": "README.md"
|
| 30 |
+
},
|
| 31 |
+
{
|
| 32 |
+
"path": "RELEASE_NOTES_TRACKFORMER_1_2.md",
|
| 33 |
+
"sha256": "d67aa075e85911b6753714fdea15a425382d8c0ff75e3b4f18cbea42124ef63d",
|
| 34 |
+
"bytes": 5616,
|
| 35 |
+
"current_location": "docs/REFERENCE.md#release-notes-trackformer-1-2-md"
|
| 36 |
+
},
|
| 37 |
+
{
|
| 38 |
+
"path": "docs/automatic_forecasts.md",
|
| 39 |
+
"sha256": "20be63648e49d66817d9fdf8c3ea204bd9b787ed336fbca303b87b91bbf90a2e",
|
| 40 |
+
"bytes": 9661,
|
| 41 |
+
"current_location": "docs/REFERENCE.md#docs-automatic-forecasts-md"
|
| 42 |
+
},
|
| 43 |
+
{
|
| 44 |
+
"path": "docs/daily_storm_benchmark.md",
|
| 45 |
+
"sha256": "d5e6525c968f8e47c0adaa1956f4d74b37bc7f1da484011959dba553d68c12e7",
|
| 46 |
+
"bytes": 6923,
|
| 47 |
+
"current_location": "docs/REFERENCE.md#docs-daily-storm-benchmark-md"
|
| 48 |
+
},
|
| 49 |
+
{
|
| 50 |
+
"path": "docs/deepmind_daily_benchmark.md",
|
| 51 |
+
"sha256": "45e902884e45a5ed209a88ea41d8f323b9ba4822de63b55978d33146bd1f832b",
|
| 52 |
+
"bytes": 11593,
|
| 53 |
+
"current_location": "docs/REFERENCE.md#docs-deepmind-daily-benchmark-md"
|
| 54 |
+
},
|
| 55 |
+
{
|
| 56 |
+
"path": "docs/intensity_benchmark.md",
|
| 57 |
+
"sha256": "13dba33f4007526c0f31cbb497d9bd91bba64f2c85fee30113ef49c47fbc6b8a",
|
| 58 |
+
"bytes": 8048,
|
| 59 |
+
"current_location": "docs/REFERENCE.md#docs-intensity-benchmark-md"
|
| 60 |
+
},
|
| 61 |
+
{
|
| 62 |
+
"path": "docs/repository_layout.md",
|
| 63 |
+
"sha256": "97331372e230c17f208d853029873c34e820b79af4d09bbf3f237174915a5f8a",
|
| 64 |
+
"bytes": 1980,
|
| 65 |
+
"current_location": "docs/REFERENCE.md#docs-repository-layout-md"
|
| 66 |
+
},
|
| 67 |
+
{
|
| 68 |
+
"path": "docs/research_report_zh.md",
|
| 69 |
+
"sha256": "8775c6c3591ceda39fe93e0f33fd2e7416a6d6036d561b541194f251ec7b2104",
|
| 70 |
+
"bytes": 10385,
|
| 71 |
+
"current_location": "docs/REFERENCE.md#docs-research-report-zh-md"
|
| 72 |
+
},
|
| 73 |
+
{
|
| 74 |
+
"path": "docs/showcase_archive.md",
|
| 75 |
+
"sha256": "10789a07a277c14c0ab61be0c88d104908fcd5050a237bf35920d76f075d0897",
|
| 76 |
+
"bytes": 4380,
|
| 77 |
+
"current_location": "docs/REFERENCE.md#docs-showcase-archive-md"
|
| 78 |
+
},
|
| 79 |
+
{
|
| 80 |
+
"path": "docs/trackformer_1_2_evaluation.md",
|
| 81 |
+
"sha256": "9cb4bb25fe80ab012e9c21c4bca56d8b0075ca24e54ae4cc94cae43826bb9a5c",
|
| 82 |
+
"bytes": 9870,
|
| 83 |
+
"current_location": "docs/REFERENCE.md#docs-trackformer-1-2-evaluation-md"
|
| 84 |
+
},
|
| 85 |
+
{
|
| 86 |
+
"path": "docs/trackformer_1_2_training_data.md",
|
| 87 |
+
"sha256": "0d8b21a15d5ed69a8655fa519bb99a106d741c0cc58a4f21bee2941bbd8f0b41",
|
| 88 |
+
"bytes": 936,
|
| 89 |
+
"current_location": "docs/TrackFormer_1.2_Training_and_Reproducibility.pdf"
|
| 90 |
+
},
|
| 91 |
+
{
|
| 92 |
+
"path": "evaluation/README.md",
|
| 93 |
+
"sha256": "fab70734f14564b46f01ea51ba5d6f044a6b5ee986ecab919b3c81d44819606f",
|
| 94 |
+
"bytes": 3114,
|
| 95 |
+
"current_location": "docs/REFERENCE.md#evaluation-readme-md"
|
| 96 |
+
},
|
| 97 |
+
{
|
| 98 |
+
"path": "models/trackformer_1_2_field/README.md",
|
| 99 |
+
"sha256": "cb4f3f1182547bb1ca19f7dc1d3c9e5aa292cec51b739b94c418065d211f1423",
|
| 100 |
+
"bytes": 11984,
|
| 101 |
+
"current_location": "models/trackformer_1_2_field/README.md"
|
| 102 |
+
},
|
| 103 |
+
{
|
| 104 |
+
"path": "models/trackformer_1_2_field/WIND_ESTIMATION.md",
|
| 105 |
+
"sha256": "e3bceb94600872521ec983affff6c8a5ca5b0a53774bd91fcb4f342a5c53c64d",
|
| 106 |
+
"bytes": 2996,
|
| 107 |
+
"current_location": "docs/REFERENCE.md#models-trackformer-1-2-field-wind-estimation-md"
|
| 108 |
+
},
|
| 109 |
+
{
|
| 110 |
+
"path": "release_tools/STORM_STRUCTURE_DIAGNOSTICS.md",
|
| 111 |
+
"sha256": "6675f1dfd781802ab012bcd0dfaa6f344df16a4803d76bd12265af89a28c5dbd",
|
| 112 |
+
"bytes": 6587,
|
| 113 |
+
"current_location": "docs/REFERENCE.md#release-tools-storm-structure-diagnostics-md"
|
| 114 |
+
},
|
| 115 |
+
{
|
| 116 |
+
"path": "release_tools/V1312_CLOUD_SOURCE_SECURITY.md",
|
| 117 |
+
"sha256": "e4574a975645c000d9259303c44c616ce91df9130a764c6b3eca56e65f17965e",
|
| 118 |
+
"bytes": 2765,
|
| 119 |
+
"current_location": "docs/REFERENCE.md#release-tools-v1312-cloud-source-security-md"
|
| 120 |
+
},
|
| 121 |
+
{
|
| 122 |
+
"path": "training/trackformer_1_2/PUBLICATION.md",
|
| 123 |
+
"sha256": "1bcaa51a1999422925db2a100927985b927b454a2ff4250e318f7049ceec81c8",
|
| 124 |
+
"bytes": 1505,
|
| 125 |
+
"current_location": "docs/TrackFormer_1.2_Training_and_Reproducibility.pdf"
|
| 126 |
+
},
|
| 127 |
+
{
|
| 128 |
+
"path": "training/trackformer_1_2/README.md",
|
| 129 |
+
"sha256": "97ac00bff15891218956a4743ae0d12cddad56106683dbc3c14b4922a463023c",
|
| 130 |
+
"bytes": 1328,
|
| 131 |
+
"current_location": "docs/TrackFormer_1.2_Training_and_Reproducibility.pdf"
|
| 132 |
+
},
|
| 133 |
+
{
|
| 134 |
+
"path": "training/trackformer_1_2/THIRD_PARTY_NOTICES.md",
|
| 135 |
+
"sha256": "f764dd203eb124fa6192edd3b7b49a7a546cd5d69d3a8a250589a14ce43e2115",
|
| 136 |
+
"bytes": 1697,
|
| 137 |
+
"current_location": "docs/TrackFormer_1.2_Training_and_Reproducibility.pdf"
|
| 138 |
+
},
|
| 139 |
+
{
|
| 140 |
+
"path": "training/trackformer_1_2/source/data_preparation/v164_dataset_contract.md",
|
| 141 |
+
"sha256": "cfd838651e632aba0d2439ba202a59895bd4df22137185974df1d97f0e2525d4",
|
| 142 |
+
"bytes": 4664,
|
| 143 |
+
"current_location": "docs/TrackFormer_1.2_Training_and_Reproducibility.pdf"
|
| 144 |
+
},
|
| 145 |
+
{
|
| 146 |
+
"path": "training/trackformer_1_2/source/data_preparation/v164_structure_research.md",
|
| 147 |
+
"sha256": "a4b40e50b7d0d1433a162438f14ed110541702c4cec2a386959857293ea78141",
|
| 148 |
+
"bytes": 17119,
|
| 149 |
+
"current_location": "docs/TrackFormer_1.2_Training_and_Reproducibility.pdf"
|
| 150 |
+
},
|
| 151 |
+
{
|
| 152 |
+
"path": "training/trackformer_1_2/source/v173/DESIGN.md",
|
| 153 |
+
"sha256": "163ae39ff572f23c519d65e27ee6a8db23b47df9461670709229ccf95130049a",
|
| 154 |
+
"bytes": 3278,
|
| 155 |
+
"current_location": "docs/TrackFormer_1.2_Training_and_Reproducibility.pdf"
|
| 156 |
+
}
|
| 157 |
+
],
|
| 158 |
+
"removed_paths": [
|
| 159 |
+
"CONTRIBUTING.md",
|
| 160 |
+
"OPENNESS.md",
|
| 161 |
+
"RELEASE_NOTES_TRACKFORMER_1_2.md",
|
| 162 |
+
"docs/automatic_forecasts.md",
|
| 163 |
+
"docs/daily_storm_benchmark.md",
|
| 164 |
+
"docs/deepmind_daily_benchmark.md",
|
| 165 |
+
"docs/intensity_benchmark.md",
|
| 166 |
+
"docs/repository_layout.md",
|
| 167 |
+
"docs/research_report_zh.md",
|
| 168 |
+
"docs/showcase_archive.md",
|
| 169 |
+
"docs/trackformer_1_2_evaluation.md",
|
| 170 |
+
"docs/trackformer_1_2_training_data.md",
|
| 171 |
+
"evaluation/README.md",
|
| 172 |
+
"models/trackformer_1_2_field/WIND_ESTIMATION.md",
|
| 173 |
+
"release_tools/STORM_STRUCTURE_DIAGNOSTICS.md",
|
| 174 |
+
"release_tools/V1312_CLOUD_SOURCE_SECURITY.md",
|
| 175 |
+
"training/trackformer_1_2/PUBLICATION.md",
|
| 176 |
+
"training/trackformer_1_2/README.md",
|
| 177 |
+
"training/trackformer_1_2/THIRD_PARTY_NOTICES.md",
|
| 178 |
+
"training/trackformer_1_2/source/data_preparation/v164_dataset_contract.md",
|
| 179 |
+
"training/trackformer_1_2/source/data_preparation/v164_structure_research.md",
|
| 180 |
+
"training/trackformer_1_2/source/v173/DESIGN.md"
|
| 181 |
+
],
|
| 182 |
+
"github_removed": [
|
| 183 |
+
"CONTRIBUTING.md",
|
| 184 |
+
"OPENNESS.md",
|
| 185 |
+
"RELEASE_NOTES_TRACKFORMER_1_2.md",
|
| 186 |
+
"docs/automatic_forecasts.md",
|
| 187 |
+
"docs/daily_storm_benchmark.md",
|
| 188 |
+
"docs/deepmind_daily_benchmark.md",
|
| 189 |
+
"docs/intensity_benchmark.md",
|
| 190 |
+
"docs/repository_layout.md",
|
| 191 |
+
"docs/research_report_zh.md",
|
| 192 |
+
"docs/showcase_archive.md",
|
| 193 |
+
"docs/trackformer_1_2_evaluation.md",
|
| 194 |
+
"docs/trackformer_1_2_training_data.md",
|
| 195 |
+
"evaluation/README.md",
|
| 196 |
+
"models/trackformer_1_2_field/WIND_ESTIMATION.md",
|
| 197 |
+
"release_tools/STORM_STRUCTURE_DIAGNOSTICS.md",
|
| 198 |
+
"release_tools/V1312_CLOUD_SOURCE_SECURITY.md",
|
| 199 |
+
"training/trackformer_1_2/PUBLICATION.md",
|
| 200 |
+
"training/trackformer_1_2/README.md",
|
| 201 |
+
"training/trackformer_1_2/THIRD_PARTY_NOTICES.md",
|
| 202 |
+
"training/trackformer_1_2/source/data_preparation/v164_dataset_contract.md",
|
| 203 |
+
"training/trackformer_1_2/source/data_preparation/v164_structure_research.md",
|
| 204 |
+
"training/trackformer_1_2/source/v173/DESIGN.md"
|
| 205 |
+
],
|
| 206 |
+
"hf_removed": [
|
| 207 |
+
"CONTRIBUTING.md",
|
| 208 |
+
"OPENNESS.md",
|
| 209 |
+
"RELEASE_NOTES_TRACKFORMER_1_2.md",
|
| 210 |
+
"docs/daily_storm_benchmark.md",
|
| 211 |
+
"docs/deepmind_daily_benchmark.md",
|
| 212 |
+
"docs/intensity_benchmark.md",
|
| 213 |
+
"docs/repository_layout.md",
|
| 214 |
+
"docs/research_report_zh.md",
|
| 215 |
+
"docs/showcase_archive.md",
|
| 216 |
+
"docs/trackformer_1_2_evaluation.md",
|
| 217 |
+
"docs/trackformer_1_2_training_data.md",
|
| 218 |
+
"evaluation/README.md",
|
| 219 |
+
"models/trackformer_1_2_field/WIND_ESTIMATION.md",
|
| 220 |
+
"release_tools/STORM_STRUCTURE_DIAGNOSTICS.md",
|
| 221 |
+
"training/trackformer_1_2/PUBLICATION.md",
|
| 222 |
+
"training/trackformer_1_2/README.md",
|
| 223 |
+
"training/trackformer_1_2/THIRD_PARTY_NOTICES.md",
|
| 224 |
+
"training/trackformer_1_2/source/data_preparation/v164_dataset_contract.md",
|
| 225 |
+
"training/trackformer_1_2/source/data_preparation/v164_structure_research.md",
|
| 226 |
+
"training/trackformer_1_2/source/v173/DESIGN.md"
|
| 227 |
+
]
|
| 228 |
+
}
|
docs/daily_storm_benchmark.md
DELETED
|
@@ -1,44 +0,0 @@
|
|
| 1 |
-
# Daily forecasts, equal-storm benchmark
|
| 2 |
-
|
| 3 |
-
Status: **complete** at 2026-09-29T08:22:39Z. All **1,473 daily cases / 270 storms** finished; the SHA-256 of every saved forecast was verified. [Final equal-storm results and all 270 storm scores](../evaluation/daily_storm_final.json).
|
| 4 |
-
|
| 5 |
-
Mean track error is 798.4 km for 1.1 and 471.2 km for 1.2 (50-member mean); +120 h error is 1,646.4 versus 1,031.6 km. Direction error is 51.58° versus 34.96°, centred shape similarity 0.7544 versus 0.8837, and geographic path similarity 0.5345 versus 0.6560. The original track run reported 1.2-only central-pressure MAE of 12.62 hPa on 40 pressure-labelled storms and basin MSLP MAE of 2.72 hPa on 270 storms. It did not produce 1.1 intensity outputs.
|
| 6 |
-
|
| 7 |
-
The completed [native pressure follow-up](intensity_benchmark.md) now provides a matched 1.1 comparison on 134 daily starts / 40 storms from the same frozen plan. JMA MAE is 13.53 / 12.84 hPa for 1.1 / 1.2; USA MAE is 12.84 / 12.55 hPa. Each uses exact common agency-specific labels. The 1,339 remaining issue inputs are unavailable to the frozen 1.1 intensity pipeline, never zero-scored or filled with future observations. Paired whole-storm intervals include zero. These are not all-270-storm pressure scores.
|
| 8 |
-
|
| 9 |
-
On 1 October, `release_tools/build_release_benchmark.py` independently rechecked all forecast hashes, coordinate conversions and daily/storm route aggregates, reproduced the native pressure postprocessor and generated the [shared snapshot](../evaluation/released_daily/released_daily_benchmark.json) and [original Trackformer audit receipt](../evaluation/released_daily/released_daily_verification.json). No inference or weight changes were necessary.
|
| 10 |
-
|
| 11 |
-
The [completed WeatherNext Cyclones Mini comparison](deepmind_daily_benchmark.md) now covers exactly the same 1,473 starts / 270 storms. Mini's combined mean track error is 498.4 km, direction error 39.76°, and JMA pressure MAE 10.20 hPa on 134 shared intensity starts. Its recent-only mean track error is 288.5 km versus 477.2 km for 1.2. The [independent three-way publication audit](../evaluation/deepmind_daily/publication_audit.json) rechecks the CUDA archive, unchanged original Trackformer arrays and recomputed common-support scores; the older 1 October receipt remains the audit of the original two-model snapshot, not a hash receipt for the subsequently merged JSON. No older benchmark's DeepMind score has been transferred into this comparison.
|
| 12 |
-
|
| 13 |
-
## What counts as a case and a score
|
| 14 |
-
|
| 15 |
-
One case is one typhoon at one issue time on one UTC day. Choose the earliest eligible issue on that day, once per typhoon. Each forecast predicts all 20 six-hour leads, +6 through +120 h, from information at or before issue time.
|
| 16 |
-
|
| 17 |
-
For position error, daily score = mean of the 20 lead errors; storm score = mean of that storm's daily scores; benchmark score = mean of the 270 storm scores. A storm with ten available daily starts has exactly the same final weight as a storm with three. The per-lead comparison also averages daily values within storm before averaging storms.
|
| 18 |
-
|
| 19 |
-
Direction error, route-shape similarity, Fréchet distance and path similarity remain separate metrics. Their definitions match the [evaluation notes](trackformer_1_2_evaluation.md); direction uses only common non-stationary leads. Missing direction scores are reported, not replaced by zero. There is no invented combined weighted quality mark.
|
| 20 |
-
|
| 21 |
-
## Frozen cohort
|
| 22 |
-
|
| 23 |
-
- 270 distinct storms, 1,473 daily issue cases, selected from 523 locally eligible storms.
|
| 24 |
-
- All 40 eligible recent storms beginning in 2024 or later, plus 230 historical storms selected at evenly spaced positions in the chronological 1980–1999 storm list.
|
| 25 |
-
- The entire storm must lie outside the selected 1.2 checkpoint's 2000–2021 fitting and 2022–2023 validation years, with the history/target boundary also outside those years.
|
| 26 |
-
- Issue centre within 0–60°N / 100–180°E; exact six-hour weather history through issue; complete route labels through +120 h; matching pressure-atlas coverage for scoring.
|
| 27 |
-
- Use all eligible daily starts for each chosen storm. Never drop a case because its forecast looks poor.
|
| 28 |
-
- [Cohort manifest](../evaluation/release_data/daily_storm_cohort.json) freezes rows, dates, storm IDs, source hashes and checkpoint identity before inference. Forecast artifacts are new; some recent storms overlap earlier development comparisons.
|
| 29 |
-
|
| 30 |
-
Requiring complete five-day truth excludes short remaining lifetimes. This is a declared coverage limitation, not an all-storm operational sample. Historical cases are retrospective hindcasts from a model trained on later years; reanalysis and best-track issue states do not establish real-time data availability. This cohort has not been certified untouched across all earlier experiments.
|
| 31 |
-
|
| 32 |
-
## Models and input boundaries
|
| 33 |
-
|
| 34 |
-
Trackformer **1.2** uses the unchanged released checkpoint and 50 independently seeded, smooth input-perturbation members. Seeds are `2043 + source_row * 100 + member_id`, with member IDs 0–49. Normalized perturbation amplitudes remain 0.025 for basin history and 0.015 for regional history. These are not learned latent samples or 50 separate trained models. Member routes and central pressures are averaged; physical fields are averaged on fixed common geographic grids. Missing native regional inputs retain the model's missing-detail mask.
|
| 35 |
-
|
| 36 |
-
Trackformer **1.1** uses the released causal weighted route builder with issue, −12 h and −24 h analyses and historical motion. Its absolute route is converted into the same issue-centred local-km coordinate system as 1.2. The pipelines differ, so this is a released-system comparison, not an architecture-only ablation.
|
| 37 |
-
|
| 38 |
-
Future weather/track/pressure are used for eligibility coverage and scoring only, never inference. No retraining, coefficient tuning, new weather downloads or checkpoint substitution occurs in this run.
|
| 39 |
-
|
| 40 |
-
## Pressure and failures
|
| 41 |
-
|
| 42 |
-
For the original 1.2 run, central-pressure MAE uses valid labels and cosine-area-weighted basin MSLP MAE uses the native basin grid, daily then equally per storm. Keep pressure label coverage visible. The separate native-pressure follow-up uses a paired mask for both models at each agency's exact valid times; it does not substitute its smaller coverage into all-storm track results. Regional/core field scores need their own valid coverage and are not implied by basin-wide error. No same-cohort 1.1 basin-field score is claimed.
|
| 43 |
-
|
| 44 |
-
Completed daily forecasts are saved atomically with hashes. A nonfinite forecast or source mismatch stops the run visibly rather than silently deleting the case. Resume skips completed cases without rerunning them. Partial aggregate scores include only fully completed storms, identify their counts, and must not be presented as the final 270-storm result. Final uncertainty should resample whole storms, not overlapping daily issues independently.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
docs/deepmind_daily_benchmark.md
DELETED
|
@@ -1,98 +0,0 @@
|
|
| 1 |
-
# Completed WeatherNext Cyclones Mini daily benchmark
|
| 2 |
-
|
| 3 |
-
## Model and frozen plan
|
| 4 |
-
|
| 5 |
-
**Complete and verified:** all **1,473 daily forecasts / 270 Western Pacific storms** finished on an **NVIDIA GeForce RTX 3070 using JAX CUDA in Windows/WSL**. The final worker audit is dated **4 October 2026, 12:55 UTC**. Publication independently rechecked all returned case hashes, physical grids, unchanged Trackformer reference arrays and exact common-support scores. No new inference was performed during this review.
|
| 6 |
-
|
| 7 |
-
The reference is Google's official [WeatherNext Cyclones Mini](https://github.com/google-deepmind/weathernext#weathernext-cyclones-mini): **`WeatherNextCyclones_Mini_<2024`**, trained through 2023 at native **1° resolution**. The pinned [software package is v0.3.0](https://github.com/google-deepmind/weathernext/blob/89c4b2a77a1c57b328b909c575550fd2e5aadc9c/setup.py); the package version and checkpoint generation identify different things. This is not a custom imitation, full-sized WeatherNext 2/Cyclones, or a WeatherNext 3 checkpoint. Mini's scores do not stand in for those larger models.
|
| 8 |
-
|
| 9 |
-
- Cohort: **1,473 daily starts / 270 Western Pacific storms**, identical to the frozen Trackformer 1.1/1.2 route comparison.
|
| 10 |
-
- Forecast: twenty six-hour steps, **+6 to +120 h**, with the exact observed issue-time centre as the initialized-storm anchor.
|
| 11 |
-
- Inputs: exact ERA5 analyses at **issue−6 h and issue**, calendar forcings only after issue. No future weather, storm labels or official forecast tracks enter the worker.
|
| 12 |
-
- Weights SHA-256: `a1bb151457077248d70a458b1a7b19deacd2926fbcb46eee4ff2691444e3596e`.
|
| 13 |
-
- Cohort SHA-256: `965184f1e5ab3fcb52e304f39b38583a14f753870aa2295f2d01942a441dcfd6`.
|
| 14 |
-
- Official code commit: `89c4b2a77a1c57b328b909c575550fd2e5aadc9c`.
|
| 15 |
-
- CUDA protocol SHA-256: `91b31582f24ffdd4f2309eddb0cc68498a2c31cab7cbabaf06f43dfbff413606`.
|
| 16 |
-
- Runtime: JAX/JAXlib 0.4.38, Python 3.12.3, highest float32 matmul precision, TF32 disabled.
|
| 17 |
-
|
| 18 |
-
## Results
|
| 19 |
-
|
| 20 |
-
| Matched development metric | Trackformer 1.1 | Trackformer 1.2 · mean of 50 | Mini · one member | Coverage |
|
| 21 |
-
| --- | ---: | ---: | ---: | --- |
|
| 22 |
-
| Mean track error · km, lower is better | 798.41 | 471.20 | 498.37 | 1,473 days / 270 storms |
|
| 23 |
-
| +120 h track error · km, lower is better | 1,646.40 | 1,031.63 | 1,031.91 | 1,473 days / 270 storms |
|
| 24 |
-
| Direction error · degrees, lower is better | 51.58 | 34.96 | 39.76 | 1,473 days / 270 storms |
|
| 25 |
-
| Centred route-shape similarity · higher is better | 0.7544 | 0.8837 | 0.9010 | 1,473 days / 270 storms |
|
| 26 |
-
| JMA central-pressure MAE · hPa, lower is better | 13.53 | 12.84 | 10.20 | 134 days / 40 storms / 2,637 leads |
|
| 27 |
-
| JMA pressure-curve similarity · higher is better | 0.7074 | 0.7118 | 0.8088 | 134 days / 40 storms |
|
| 28 |
-
| USA central-pressure MAE · hPa, lower is better | 12.84 | 12.55 | 11.55 | 134 days / 40 storms / 2,325 leads |
|
| 29 |
-
| USA pressure-curve similarity · higher is better | 0.7237 | 0.6637 | 0.7445 | 131 days / 40 storms |
|
| 30 |
-
|
| 31 |
-
Pressure measures central-pressure intensity, not map-grid error. The lower USA curve count excludes flat/short curves; it is not zero-filled. All three models' pressure-curve case IDs were verified equal for each agency. DeepMind geographic path/Fréchet scores and whole-storm confidence intervals are not supplied by this export and remain unavailable.
|
| 32 |
-
|
| 33 |
-
### Historical and recent groups
|
| 34 |
-
|
| 35 |
-
| Period | Daily starts / storms | Mean track error · 1.1 / 1.2 / Mini, km | Direction error · 1.1 / 1.2 / Mini, degrees |
|
| 36 |
-
| --- | --- | --- | --- |
|
| 37 |
-
| 1980–1999 | 1,336 / 230 | 780.50 / 470.17 / 534.86 | 51.36 / 35.01 / 41.26 |
|
| 38 |
-
| Storms beginning in 2024+ | 137 / 40 | 901.45 / 477.16 / 288.51 | 52.90 / 34.68 / 31.16 |
|
| 39 |
-
|
| 40 |
-
Mini has better recent-only track scores even though 1.2 has lower position/heading errors in the combined cohort. Mini also has lower JMA/USA pressure MAE and higher pressure-curve similarity. Historical fitting-year overlap, different weather inputs and different ensemble sizes prevent a general or equal-compute superiority claim. All pressure comparisons here belong to the recent group; no historical pressure score is inferred from missing native 1.1 inputs.
|
| 41 |
-
|
| 42 |
-
[Full results, period breakdowns and per-lead track errors](../evaluation/deepmind_daily/benchmark.json) · [Shared snapshot and existing 1.1/1.2 uncertainty](../evaluation/released_daily/released_daily_benchmark.json) · [Completion receipt](../evaluation/deepmind_daily/verification.json) · [Independent review](../evaluation/deepmind_daily/publication_audit.json)
|
| 43 |
-
|
| 44 |
-
### Strict forecast-date partition and separate charts
|
| 45 |
-
|
| 46 |
-
All three README charts—total, strict `<2024` and strict `>2024`—use **`WeatherNextCyclones_Mini_<2024`**, the same official checkpoint trained through 2023, software v0.3.0, native 1° and one CUDA member. A post-2024 forecast is not a post-2024-trained model. The following proportions refer to this evaluation cohort, not the model's training dataset.
|
| 47 |
-
|
| 48 |
-
| UTC issue year | Starts / share | Storms / share |
|
| 49 |
-
| --- | --- | --- |
|
| 50 |
-
| `<2024` (1980–1999 here) | 1,336 / 90.7% | 230 / 85.2% |
|
| 51 |
-
| Calendar 2024 | 76 / 5.2% | 22 / 8.1% |
|
| 52 |
-
| `>2024` (2025–2026 here) | 61 / 4.1% | 18 / 6.7% |
|
| 53 |
-
|
| 54 |
-
The original `recent_2024_onward` aggregate includes 2024 and is retained unchanged. The new **strict `after_2024`** derivative excludes 2024: track position MAE is **963.02 / 469.40 / 258.67 km**, direction error **57.88 / 33.81 / 28.73°**, and JMA pressure MAE **12.59 / 11.12 / 9.55 hPa** for 1.1 / 1.2 / Mini. Track uses 61 starts / 18 storms; pressure uses 59 starts / 18 storms / 1,174 exact common leads. The total chart still uses the original scores. Its historical 230 storms contribute 85.2% of equal-storm track weight, not 90.7%.
|
| 55 |
-
|
| 56 |
-
The strict **`before_2024`** chart uses the 1,336 historical starts / 230 storms (1980–1999 here): position MAE **780.50 / 470.17 / 534.86 km** and direction error **51.36 / 35.01 / 41.26°** for 1.1 / 1.2 / Mini. All three shared JMA pressure scores are null, with zero eligible common starts: frozen historical 1.1 intensity inputs are unavailable. That panel says **Not scored** and contains no bars, not zero-height bars. Track coverage must never be reused as pressure coverage.
|
| 57 |
-
|
| 58 |
-
[UTC partitions, exact case IDs, all metrics and source hashes](../evaluation/deepmind_daily/period_comparison.json) · [Total chart](../evaluation/released_daily/model_1_2_benchmark.png) · [Pre-2024 chart](../evaluation/released_daily/model_1_2_before_2024_benchmark.png) · [Post-2024 chart](../evaluation/released_daily/model_1_2_after_2024_benchmark.png)
|
| 59 |
-
|
| 60 |
-
Reproduce these derivative summaries without inference or raw-weather downloads:
|
| 61 |
-
|
| 62 |
-
```bash
|
| 63 |
-
python release_tools/build_deepmind_period_comparison.py --archive /path/to/DeepMind_RTX3070_results_20261004_125651_467960.zip
|
| 64 |
-
python release_tools/plot_model_announcement.py
|
| 65 |
-
```
|
| 66 |
-
|
| 67 |
-
The helper checks the exact archive hash against the existing publication audit and every one of the 1,473 case-score JSON hashes against the frozen manifest. It rechecks identities, causal-time metadata, CUDA completion and common pressure masks, reproduces the original total and period scores, then recomputes only the UTC subsets using the unchanged equal-storm aggregation. It never rewrites the original worker export, changes a forecast array or claims a new raw-input audit. Chart sidecars identify their derivative source hash, checkpoint, period, exact values and missing-data policy.
|
| 68 |
-
|
| 69 |
-
## Like-for-like scoring
|
| 70 |
-
|
| 71 |
-
All three models use the frozen starts, exact future label times and original issue-relative kilometre projection. Routes are unshifted. Direction is recomputed on **common moving steps across truth and all three models**, not compared using different masks.
|
| 72 |
-
|
| 73 |
-
Pressure uses the original 134 eligible native-intensity starts / 40 storms, restricted to common valid leads across three predictions and the selected reference. JMA and USA remain separate. MAE and centred time-curve similarity are separate; missing, nonphysical or failed outputs are not zeros. Flat or shorter-than-six-point curves have unavailable shape similarity.
|
| 74 |
-
|
| 75 |
-
Average valid leads within an issue, days within a storm, then storms equally. Partial aggregates contain only fully completed storms and explicit coverage. Recent 2024+ and historical 1980–1999 groups are separate because the latter overlap WeatherNext fitting years. Different input pipelines and the one-member Mini versus 50-input-member 1.2 policy make this an output comparison, not an equal-compute architecture ablation or a certified unused test.
|
| 76 |
-
|
| 77 |
-
The official six-hour direct tracker uses a frozen initialized-storm continuity policy: no cyclogenesis, no dissipation pruning and no nearby-cyclone pruning. This is disclosed rather than called the operational default. Native 1° WP MSLP grids are saved in physical hPa; cyclone-head central pressure is not the basin-grid minimum.
|
| 78 |
-
|
| 79 |
-
## Execution and verification
|
| 80 |
-
|
| 81 |
-
The original [Mac CPU runner and scoring definitions](../release_tools/deepmind_daily_benchmark.py) provided the frozen plan. The completed run used the separately pinned, resumable RTX 3070 CUDA package; the old CPU launch instructions are not the execution record of these results. The [frozen CUDA protocol](../evaluation/deepmind_daily/protocol.json), [2,946-file case manifest](../evaluation/deepmind_daily/case-manifest.json) and [CPU/CUDA canary](../evaluation/deepmind_daily/canary.json) identify the completed run.
|
| 82 |
-
|
| 83 |
-
The strict same-runtime CPU/CUDA canary passed unchanged 0.01 hPa mean / 0.1 hPa maximum field tolerances. Its older cross-runtime Mac comparison failed the maximum-field gate and remains explicitly recorded as failed; that diagnostic is not numerical-equivalence certification. No CPU forecast was counted or relabelled as a CUDA case.
|
| 84 |
-
|
| 85 |
-
The publication review checks all 1,473 IDs, original reference hashes, exact issue−6 h / issue input-time metadata and saved input hashes, all twenty output times, finite native pressure grids, and recomputed three-model route/pressure metrics. **Raw ERA5 input states are not in the returned result-only ZIP**: this review preserves the original workers' causal-input receipts and hashes, rather than claiming to independently reopen those raw states.
|
| 86 |
-
|
| 87 |
-
To repeat the read-only archive review, use the existing frozen project reference forecasts and intensity labels, NumPy, and the returned result archive. It does not run a model or download weather:
|
| 88 |
-
|
| 89 |
-
```bash
|
| 90 |
-
python release_tools/import_deepmind_release_results.py \
|
| 91 |
-
--archive /path/to/DeepMind_RTX3070_results_20261004_125651_467960.zip \
|
| 92 |
-
--project /Volumes/D/typhoon_predict \
|
| 93 |
-
--output /Volumes/D/typhoon_predict/remote_typhoon_predict_repo/evaluation/deepmind_daily
|
| 94 |
-
```
|
| 95 |
-
|
| 96 |
-
Completed scores are also exposed by the existing [public benchmark API](https://trackformer-weatherlab.rudin-euler-8253.chatgpt.site/api/benchmarks/released). Documentation publication does not replace immutable forecasts, change either model's weights, restart the benchmark, modify active training, or redeploy the website. The original two-model verification receipt remains unchanged; the new publication audit identifies the merged snapshot's exact hash.
|
| 97 |
-
|
| 98 |
-
The returned worker's `benchmark.json` is retained byte-for-byte for its audit hashes. Its `published_to_site: false` and `integration_note` describe the handoff before import, not the current benchmark status. The completed CUDA receipt, independent publication audit and shared public snapshot above are the current completion/publication evidence; those historical worker fields do not mean the comparison is unfinished.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
docs/intensity_benchmark.md
DELETED
|
@@ -1,131 +0,0 @@
|
|
| 1 |
-
# Matched daily intensity benchmark
|
| 2 |
-
|
| 3 |
-
Completed GPU inference and verified aggregation on 1 October 2026 (Taipei).
|
| 4 |
-
The released weights, original forecasts and original calibration are unchanged.
|
| 5 |
-
|
| 6 |
-
## Same hierarchy as track
|
| 7 |
-
|
| 8 |
-
The plan is the frozen 270-storm / 1,473-daily-issue track cohort. One storm on
|
| 9 |
-
one UTC day is one case, at its earliest eligible issue. Forecast errors at
|
| 10 |
-
valid +6, +12, …, +120 h leads are averaged per daily issue, then across its
|
| 11 |
-
storm's daily issues, then across storms with equal weights. Long-lived storms
|
| 12 |
-
do not gain extra final weight. Per-lead errors use the same daily → storm
|
| 13 |
-
averaging. Both models use exactly the same native observation mask for each
|
| 14 |
-
wind/pressure comparison. Wind and each agency's pressure are separate metrics;
|
| 15 |
-
there is no arbitrary weighted combined rank.
|
| 16 |
-
|
| 17 |
-
Native 1.1 intensity needs valid issue-time current wind and pressure. Its
|
| 18 |
-
frozen input archive provides them for 134 daily issues / 40 storms, not the
|
| 19 |
-
other 1,339 daily starts. These missing starts are reported, never repaired
|
| 20 |
-
with future observations or relabeled as zero error. Thus the result shares
|
| 21 |
-
the **method and planned cohort** of track, but does not claim the same valid
|
| 22 |
-
intensity coverage. Missing future observations also remain unavailable.
|
| 23 |
-
|
| 24 |
-
## Model identity and inference
|
| 25 |
-
|
| 26 |
-
1.2 is the released checkpoint SHA-256
|
| 27 |
-
`f194a23d3f91ea76ad776dfad942fabd669eeae8b3fd665815463095367e9ee0`.
|
| 28 |
-
Every original pressure forecast file is rehashed. To recover previously
|
| 29 |
-
unsaved auxiliary wind, the MPS runner replays the identical 50 input members
|
| 30 |
-
with seeds `2043 + source_row * 100 + member_id`, nine exact causal analyses,
|
| 31 |
-
original detail masks and original issue inputs. Every replay has 50 distinct
|
| 32 |
-
input hashes, finite 50 × 20 wind and pressure outputs and verified means.
|
| 33 |
-
The maximum pressure difference from original forecasts is **0.0 hPa**.
|
| 34 |
-
No weights, preprocessing or coefficients are fitted here.
|
| 35 |
-
|
| 36 |
-
1.1 uses the original public intensity pipeline: frozen primary and structure
|
| 37 |
-
experts, validation-fit anchor/calibration coefficients, and the frozen
|
| 38 |
-
same-storm −12/−24 h temporal branch where its current-wind gate permits.
|
| 39 |
-
Expert count/activation is preserved in each case receipt; this is not a
|
| 40 |
-
50-member 1.1 ensemble. Issue states and causal weather only enter inference;
|
| 41 |
-
future labels are loaded separately for scoring. The two systems' input
|
| 42 |
-
pipelines differ: this is not an architecture-controlled ablation.
|
| 43 |
-
|
| 44 |
-
## Native observation definitions
|
| 45 |
-
|
| 46 |
-
- Central pressure: exact storm SID and valid UTC; native `USA_PRES` and
|
| 47 |
-
`TOKYO_PRES` are scored separately, in hPa, on a common mask for each agency.
|
| 48 |
-
- Wind: `USA_WIND` nominal one-minute reference. The 1.2 scalar is an auxiliary
|
| 49 |
-
output, not a certified ten-metre sustained surface-wind product. JMA
|
| 50 |
-
`TOKYO_WIND` is ten-minute and is never converted or pooled in this score.
|
| 51 |
-
- Native 1.1 radius: `USA_RMW` and `USA_R34/R50/R64_NE/SE/SW/NW`, nautical miles
|
| 52 |
-
multiplied by 1.852 to kilometres. They are radii, not diameters. RMW must be
|
| 53 |
-
positive. A reported zero wind-radius extent remains valid; blanks remain
|
| 54 |
-
unavailable. JMA R30 longest/shortest, eye size and ROCI are not substitutes.
|
| 55 |
-
- Native 1.2 radius: **not available**, because it has no radius head. The
|
| 56 |
-
experimental pressure-scaled candidate is separate; its strict definition
|
| 57 |
-
guard prevents scoring it as calibrated official surface-wind radii.
|
| 58 |
-
|
| 59 |
-
Radius errors are averaged within the daily issue over valid leads for each
|
| 60 |
-
component, then equally across available quadrant components, then across
|
| 61 |
-
days within storm and equally across storms. Coverage differs by threshold;
|
| 62 |
-
all counts are printed. Observational RMW has reporting/provisional limitations.
|
| 63 |
-
|
| 64 |
-
## Curve comparison, analogous to route direction and shape
|
| 65 |
-
|
| 66 |
-
The released Site defaults to **central-pressure MAE**, with slight descriptive mean improvements (JMA 13.53 / 12.84 hPa; USA 12.84 / 12.55 hPa, for 1.1 / 1.2) and paired uncertainty including zero. Its secondary **similarity** view is the exact complement of the shape-error metric below: `(1 + centred cosine) / 2 = 1 − shape_error`. This is a presentation transform, not a new fit or score. JMA similarity is 0.7074 / 0.7118 on 40 storms and 134 starts; USA similarity is 0.7237 / 0.6637 on 40 storms and 131 non-flat starts. Unshifted physical hPa timelines remain visible separately. The public snapshot includes every input-eligible pressure timeline.
|
| 67 |
-
|
| 68 |
-
Magnitude MAE is retained alongside time-curve diagnostics, without optimizing
|
| 69 |
-
time lag, warping time, shifting observations or adjusting forecasts to truth.
|
| 70 |
-
These are separate, versioned post-hoc descriptive metrics, not a preregistered
|
| 71 |
-
untouched-holdout hypothesis test.
|
| 72 |
-
|
| 73 |
-
1. Curve-shape error = `(1 - centred cosine similarity) / 2`, lower is better.
|
| 74 |
-
Subtract each curve's common-lead mean, normalize its scalar magnitude and
|
| 75 |
-
keep original time order and sign. At least six common valid points and
|
| 76 |
-
non-flat truth and both forecasts are required. A flat/unavailable curve
|
| 77 |
-
does not receive a perfect score. This measure ignores level and amplitude,
|
| 78 |
-
so it does not replace MAE or demonstrate matching absolute intensity.
|
| 79 |
-
2. Trend-direction mismatch = fraction of adjacent six-hour changes whose
|
| 80 |
-
signs differ (increasing / flat / decreasing). Deadband is exactly zero;
|
| 81 |
-
flat-versus-changing is a mismatch. Missing leads break adjacency, never
|
| 82 |
-
form twelve-hour changes disguised as six-hour steps.
|
| 83 |
-
3. Six-hour tendency MAE = mean absolute error of those adjacent changes in
|
| 84 |
-
kt/6 h, hPa/6 h or km/6 h. It remains separate from shape error.
|
| 85 |
-
|
| 86 |
-
All daily curve scores use the same valid model/reference masks before the
|
| 87 |
-
storm aggregation. Uncertainty uses 2,000 whole-storm bootstrap replicates
|
| 88 |
-
with seed 4712, not overlapping issue resampling. Paired 1.2-minus-1.1
|
| 89 |
-
differences and their intervals are included in the full JSON report.
|
| 90 |
-
|
| 91 |
-
## Results and honest boundaries
|
| 92 |
-
|
| 93 |
-
USA pressure MAE: 1.1 **12.84** vs 1.2 **12.55 hPa**. JMA pressure MAE:
|
| 94 |
-
**13.53** vs **12.84 hPa**. Auxiliary wind diagnostic MAE: **16.98** vs
|
| 95 |
-
**25.03 kt**. Wind curve-shape error: **0.271** vs **0.385**. Thus wind is
|
| 96 |
-
worse for 1.2 despite the slight mean-pressure improvement. At +120 h USA
|
| 97 |
-
pressure is worse for 1.2, whereas JMA pressure is better; no all-lead/agency
|
| 98 |
-
win is claimed. Radius 1.1 errors are reported, but native 1.2 is N/A.
|
| 99 |
-
|
| 100 |
-
The old 1980–1999 cohort overlaps original 1.1 fitting, and 1.2 hindcasts are
|
| 101 |
-
retrospective predictions by a model trained on later years. Some recent
|
| 102 |
-
storms have been inspected in development. Complete five-day route labels
|
| 103 |
-
were required by the inherited cohort. This is not untouched generalization,
|
| 104 |
-
an operational comparison or proof of no overfitting. Scientific safety and
|
| 105 |
-
wind/radius definition limitations apply to all illustrated examples.
|
| 106 |
-
|
| 107 |
-
## Files and reproduction
|
| 108 |
-
|
| 109 |
-
[Verified aggregate and per-storm metrics](../evaluation/intensity/intensity_final.json)
|
| 110 |
-
· [Chart verification receipt](../evaluation/intensity/verification.json)
|
| 111 |
-
· [GPU runner](../release_tools/benchmark_intensity_v12_v11.py)
|
| 112 |
-
· [Independent verifier and plotter](../release_tools/plot_intensity_benchmark.py).
|
| 113 |
-
|
| 114 |
-
```sh
|
| 115 |
-
MPLCONFIGDIR=/Volumes/D/typhoon_predict/.cache/matplotlib \
|
| 116 |
-
/Volumes/D/typhoon_predict/.venv/bin/python -B \
|
| 117 |
-
release_tools/benchmark_intensity_v12_v11.py \
|
| 118 |
-
--project /Volumes/D/typhoon_predict \
|
| 119 |
-
--output /Volumes/D/typhoon_predict/output/intensity-v12-v11-20261001 --chunk 10
|
| 120 |
-
|
| 121 |
-
MPLCONFIGDIR=/Volumes/D/typhoon_predict/.cache/matplotlib \
|
| 122 |
-
/Volumes/D/typhoon_predict/.venv/bin/python -B \
|
| 123 |
-
release_tools/plot_intensity_benchmark.py \
|
| 124 |
-
--source /Volumes/D/typhoon_predict/output/intensity-v12-v11-20261001 \
|
| 125 |
-
--project /Volumes/D/typhoon_predict \
|
| 126 |
-
--output /Volumes/D/typhoon_predict/remote_typhoon_predict_repo/evaluation/intensity
|
| 127 |
-
```
|
| 128 |
-
|
| 129 |
-
The runner locks one process and resumes immutable completed cases. It never
|
| 130 |
-
silently restarts training. Large original forecasts/raw members remain on D;
|
| 131 |
-
public reports include exact source hashes and complete verified aggregates.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
docs/repository_layout.md
DELETED
|
@@ -1,23 +0,0 @@
|
|
| 1 |
-
# Current repository layout
|
| 2 |
-
|
| 3 |
-
The current source tree is focused on the selected pressure-field **Trackformer 1.2** release. The implementation remains at `models/trackformer_1_2_field/`; module filenames such as `v165_base.py` are required architecture dependencies, not unused legacy files. They and their source hashes have not been changed.
|
| 4 |
-
|
| 5 |
-
## Cleanup on 29 September 2026
|
| 6 |
-
|
| 7 |
-
Removed from the current branch:
|
| 8 |
-
|
| 9 |
-
- The withdrawn route/scalar `trackformer_1_2.py` candidate and its incompatible `examples/predict_trackformer_1_2.py` entry point.
|
| 10 |
-
- Root-level 1.1 inference/route/intensity/temporal modules and their old analysis utility, plus `colab_train_v17.ipynb`.
|
| 11 |
-
- The old 1.1 paper and superseded 1.2.26/matched100 comparison assets.
|
| 12 |
-
|
| 13 |
-
Moved current benchmark figures, metrics and plot data from `paper/` into `evaluation/`, updating documentation and reproduction paths. `paper/trackformer.tex` and `paper/trackformer.pdf` stay at their existing locations. Published HTML and MP4 URLs under `docs/` stay unchanged.
|
| 14 |
-
|
| 15 |
-
The 1.1-versus-1.2 comparison arrays, TIP diagnostic, current paper, weights, model source hashes and release tags are retained. No local research checkpoints, running benchmark, Hugging Face files or historical release assets were deleted by this source-tree cleanup.
|
| 16 |
-
|
| 17 |
-
## Recovery
|
| 18 |
-
|
| 19 |
-
The complete pre-cleanup repository remains in [commit 459a3d014aa4794652ceba3ecdffcde0beada2cd](https://github.com/yu314-coder/typhoon-predict/tree/459a3d014aa4794652ceba3ecdffcde0beada2cd). The [Trackformer 1.1 release](https://github.com/yu314-coder/typhoon-predict/releases/tag/trackformer-1.1) remains available. This cleanup does not rewrite Git history.
|
| 20 |
-
|
| 21 |
-
## Released 1.2 training
|
| 22 |
-
|
| 23 |
-
`training/trackformer_1_2/` contains the original training source, launch wrapper, requirements, historical preparation tools, configuration, hashes and public asset downloader. Large arrays and optimizer checkpoints live at the identical Hugging Face path and are referenced from GitHub.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
docs/research_report_zh.md
DELETED
|
@@ -1,128 +0,0 @@
|
|
| 1 |
-
# TrackFormer 1.2 中文研究說明
|
| 2 |
-
|
| 3 |
-
**作品名稱:訓練AI模型探討西北太平洋氣壓系統影響臺灣附近颱風的預測評估**
|
| 4 |
-
|
| 5 |
-
本頁對照中文研究報告及本次核定的文字、圖表修改。只介紹發布的 **TrackFormer 1.2**;原有技術論文與歷史比較紀錄保留於各自的檔案,不取代本頁的報告範圍。
|
| 6 |
-
|
| 7 |
-
[GitHub 程式與研究紀錄](https://github.com/yu314-coder/typhoon-predict) · [Hugging Face 模型](https://huggingface.co/euler314/typhoon-predict) · [即時預測頁](https://trackformer-weatherlab.rudin-euler-8253.chatgpt.site/live) · [English overview](../README.md)
|
| 8 |
-
|
| 9 |
-
## 研究動機與目標
|
| 10 |
-
|
| 11 |
-
- 以一般家用電腦可訓練、可執行為目標,讓個人及學生能參與天氣預測研究。
|
| 12 |
-
- 建立預測西北太平洋氣壓圖的 AI 模型,再從同一預測氣壓取得颱風路徑與中心氣壓。
|
| 13 |
-
- 結合不同高度的水平風,提供研究東北風、西南風與颱風周圍氣流變化的基礎。本次成果評分以路徑與氣壓為主,並非已證明季風預測改善。
|
| 14 |
-
|
| 15 |
-
## 研究設備與器材
|
| 16 |
-
|
| 17 |
-
| 設備或工具 | 實際用途與範圍 |
|
| 18 |
-
| --- | --- |
|
| 19 |
-
| Apple M4 Mac、24 GB 統一記憶體 | 本機模型執行、資料處理與製圖;透過 PyTorch MPS 使用內建 GPU 與 Metal。家用電腦訓練與執行是研究目標,不代表每一次歷史計算都只使用同一部電腦。 |
|
| 20 |
-
| NVIDIA GeForce RTX 3070、CUDA | 已完成的 DeepMind 輕量版比較預測使用此設備;RTX 3070 是硬體,不是軟體。 |
|
| 21 |
-
| 外接儲存裝置與網路 | 保存資料、模型、預測及圖表,取得公開資料並分享成果。 |
|
| 22 |
-
| Python、PyTorch | 建立模型、計算誤差、更新參數與執行預測。 |
|
| 23 |
-
| NumPy、SciPy、xarray、netCDF4 | 數值運算,依時間及經緯度整理天氣網格。 |
|
| 24 |
-
| Matplotlib | 繪製氣壓、路徑、曲線與長條圖。 |
|
| 25 |
-
| XeLaTeX、xeCJK | 中文報告排版。 |
|
| 26 |
-
|
| 27 |
-
## 資料來源與分配
|
| 28 |
-
|
| 29 |
-
| 資料 | 模型實際使用的內容 |
|
| 30 |
-
| --- | --- |
|
| 31 |
-
| NOAA NCEP/NCAR Reanalysis 1 | 海平面氣壓、500 百帕高度,以及 850、500、200 百帕的東西向風 u 和南北向風 v。u、v 共同提供風向與風速。 |
|
| 32 |
-
| ARCO-ERA5 氣壓 | 1,000 個具有原生 0.25 度氣壓資料的案例;以起報中心向四方各取 15 度,共 121×121 格點。 |
|
| 33 |
-
| NOAA IBTrACS 颱風紀錄 | 位置、先前移動、可用的中心氣壓及最大風,缺值保留有效性標記。中心氣壓比較採日本氣象廳紀錄。 |
|
| 34 |
-
| 固定地理資料 | 經緯度、陸地比例與地形高度。技術輸入同時保留環境及中心附近的地理資訊,並非未來天氣。 |
|
| 35 |
-
|
| 36 |
-
環境資料涵蓋北緯 0–60 度、東經 100–180 度,間距 2.5 度,共 25×33 格點。較細氣壓案例是下表各組的子集,不另加到案例總數。
|
| 37 |
-
|
| 38 |
-
| 用途 | 年份 | 歷史資料片段 | 其中有原生較細氣壓資料 |
|
| 39 |
-
| --- | --- | ---: | ---: |
|
| 40 |
-
| 訓練與正規化 | 2000–2021 | 13,949 | 800 |
|
| 41 |
-
| 驗證與模型選擇 | 2022–2023 | 1,041 | 100 |
|
| 42 |
-
| 原始測試分組 | 2024–2025 | 1,195 | 100 |
|
| 43 |
-
|
| 44 |
-
同一颱風保持在同一分組;跨越分組界線的資料片段不使用。資料的平均值與變動範圍只由訓練組計算。後來取得的 2026 年資料不表示發布模型重新訓練到 2026 年。[來源與檔案檢查](trackformer_1_2_training_data.md)。
|
| 45 |
-
|
| 46 |
-
## 模型如何預測與訓練
|
| 47 |
-
|
| 48 |
-
1. 輸入起報前兩天至起報時刻的資料,每六小時一筆,共 **9 筆**,搭配當時的颱風位置、強度、先前移動與固定地形。
|
| 49 |
-
2. 分別計算周圍氣壓、水平風與颱風中心附近氣壓的變化,再一起更新六小時後的狀態。
|
| 50 |
-
3. 從預測氣壓找出颱風中心,再讀取同一中心的氣壓,得到氣壓圖、路徑與中心氣壓。
|
| 51 |
-
4. 把預測狀態接著往下計算,共 **20 步、+6 至 +120 小時**。
|
| 52 |
-
5. 訓練時與之後的真實紀錄比較並更新參數;預測時不使用未來觀測。訓練預報長度由一天逐步增加到兩天、三天及五天;報告使用選定的第四輪模型。
|
| 53 |
-
|
| 54 |
-
模型有 21,452,595 個參數,中心氣壓計算網格為 65×65、約 20 公里。這是計算與重建間距,不是新取得的 20 公里原生觀測。兩天是本次模型固定的輸入設計,尚未以相同條件比較一天、兩天及三天,不能說兩天已被證明最佳。[完整技術輸入與輸出](../models/trackformer_1_2_field/README.md)。
|
| 55 |
-
|
| 56 |
-
### 為什麼有 50 組預測
|
| 57 |
-
|
| 58 |
-
報告使用**同一組模型參數**,只在正規化後的歷史天氣加入 50 組不同的小幅、平滑、平均為零的雜訊,再取平均。環境與區域資料的雜訊幅度分別為 0.025、0.015,平滑範圍分別為 5、9 格;幅度是正規化單位,並非直接加上相同數字的百帕。颱風起報位置、移動、強度、地形與有效性標記不變,原始觀測和模型參數也不修改。
|
| 59 |
-
|
| 60 |
-
路徑與中心氣壓分別平均;氣壓地圖則先把各組結果放到相同經緯度,再平均。這**不是訓練 50 個模型**,也未證明 50 組是最佳數量、一定優於單次預測,或其分散程度已是可靠的不確定性範圍。
|
| 61 |
-
|
| 62 |
-
## 多個颱風的比較
|
| 63 |
-
|
| 64 |
-
路徑使用 **270 個颱風、1,473 次起報**,比較每六小時、未來五天的預測。先平均一次起報內各時間的有效誤差,再平均同一颱風的起報,最後讓每個颱風具有相同權重。
|
| 65 |
-
|
| 66 |
-
**中心氣壓比較使用270個颱風中符合共同評分條件的40個颱風,共134次起報。將本研究模型與DeepMind輕量版模型在相同預報時間的中心氣壓預測,分別與日本氣象廳紀錄比較,共有2,637筆共同有效紀錄。**
|
| 67 |
-
|
| 68 |
-
這不代表其餘 230 個颱風都沒有氣壓觀測;本次共同子集沿用原評估流程的起報強度有效條件。缺值不當成零誤差。
|
| 69 |
-
|
| 70 |
-
比較模型是官方 **WeatherNext Cyclones Mini `<2024`**,訓練資料到 2023 年,原生間距 1 度,使用一個預測成員及軟體 v0.3.0,並非 DeepMind 完整大型模型。TrackFormer 使用 50 組輸入擾動平均,兩者保留各自資料流程,不是等算力的模型結構實驗。
|
| 71 |
-
|
| 72 |
-

|
| 73 |
-
|
| 74 |
-
第一至第五天分別指 **24、48、72、96、120 小時**,不是一天內所有時段的平均。路徑為 270 個颱風;氣壓依各時間的共同有效紀錄使用 39–40 個颱風,樣本數列於圖下。[逐日數據](report_figures/day_1_to_5.json)。
|
| 75 |
-
|
| 76 |
-
| 比較項目 | TrackFormer 1.2 | DeepMind 輕量版 |
|
| 77 |
-
| --- | ---: | ---: |
|
| 78 |
-
| 平均路徑誤差(公里) | 471.20 | 498.37 |
|
| 79 |
-
| 第五天路徑誤差(公里) | 1031.63 | 1031.91 |
|
| 80 |
-
| 六小時移動方向誤差(度) | 34.96 | 39.76 |
|
| 81 |
-
| 中心氣壓平均絕對誤差(百帕) | 12.84 | 10.20 |
|
| 82 |
-
|
| 83 |
-
這組開發評估的平均路徑及移動方向誤差較低,中心氣壓誤差則以比較模型較低。全組包含 230 個 1980–1999 年颱風及 40 個較近期颱風;歷史年份與比較模型的訓練期間重疊。只看 2025–2026 年起報的 18 個颱風、61 次起報,DeepMind 輕量版路徑誤差為 258.7 公里,TrackFormer 為 469.4 公里。已反覆查看的案例屬開發證據,不是全新、未使用的測試,不能由全組平均推論普遍優於 DeepMind。[原始完成數據與流程](deepmind_daily_benchmark.md)。
|
| 84 |
-
|
| 85 |
-
## 三個颱風展示
|
| 86 |
-
|
| 87 |
-
三案皆以相同發布模型的 50 組預測平均展示。氣壓圖比較 +48 小時、相同座標及色階下的模型與 **NCEP 參考氣壓**,採 2.5 度共同網格;NCEP 仍為分析資料,不因圖中文字簡化而變成測站觀測。路徑與中心氣壓保留全部 +6 至 +120 小時,以 IBTrACS/日本氣象廳紀錄對照,不搬移預測低壓來配合實況。
|
| 88 |
-
|
| 89 |
-
### 鳳凰(Fung-wong,2025)
|
| 90 |
-
|
| 91 |
-
起報:**2025-11-07 00 UTC**。使用保存的 Metal 結果。
|
| 92 |
-
|
| 93 |
-

|
| 94 |
-
|
| 95 |
-

|
| 96 |
-
|
| 97 |
-

|
| 98 |
-
|
| 99 |
-
### 山竹(Mangkhut,2018)
|
| 100 |
-
|
| 101 |
-
起報:**2018-09-11 00 UTC**。使用保存的 Metal 結果;位於訓練年份內,為回溯展示。
|
| 102 |
-
|
| 103 |
-

|
| 104 |
-
|
| 105 |
-

|
| 106 |
-
|
| 107 |
-

|
| 108 |
-
|
| 109 |
-
### 燕子(Jebi,2018)
|
| 110 |
-
|
| 111 |
-
起報:**2018-08-31 00 UTC**。相同發布權重在 CPU 執行的 50 組預測;未重新訓練,也未使用起報以後的天氣輸入。位於訓練年份內,為回溯展示。
|
| 112 |
-
|
| 113 |
-

|
| 114 |
-
|
| 115 |
-

|
| 116 |
-
|
| 117 |
-

|
| 118 |
-
|
| 119 |
-
三張風圈圖均是 **IBTrACS 的 34 節四方向觀測半徑,不是本模型半徑預測**。左為 +48 小時四方向最大範圍,右為五日紀錄;海里乘以 1.852 換成公里,缺值保留空白。扇形不是完整風場。發布模型沒有原生風圈半徑預測頭。[圖片雜湊與來源](report_figures/provenance.json)。
|
| 120 |
-
|
| 121 |
-
## 結論、成果特色與後續研究
|
| 122 |
-
|
| 123 |
-
- **研究結論:**以家用電腦可訓練、可執行為核心目標,完成五日氣壓、颱風路徑及中心氣壓的整合流程,並可在一般 Mac 電腦執行預測與評估。
|
| 124 |
-
- **同一套預測:**地圖、路徑與中心氣壓來自同一模型結果,可相互對照。
|
| 125 |
-
- **周圍天氣與颱風中心一起更新:**同時保留環境氣壓、風場及移動中的中心氣壓計算。
|
| 126 |
-
- **彩色圖表與清楚資料流程:**以相同時間、座標比較三個颱風,保留資料年份、案例數、缺值與來源紀錄。
|
| 127 |
-
- **應用:**提供個人及學生進行氣象學習、歷史颱風比較與地圖展示的研究工具。這不是取代官方警報的作業預報服務。
|
| 128 |
-
- **持續研究:**結合臺灣島外風向、水氣輸送與地形,研究一般降雨和強降雨的分布,並以降雨觀測另行評估。**發布的 1.2 不包含降雨預測頭或水氣輸入**,本段不是已完成的降雨準確度成果。
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
docs/showcase_archive.md
DELETED
|
@@ -1,36 +0,0 @@
|
|
| 1 |
-
# Forecast examples and diagnostics
|
| 2 |
-
|
| 3 |
-
The [model announcement](../README.md) features the geographically registered moving-core Mangkhut (2018) forecast. Its 50-member arrays and replay/playback receipts are linked there. These are development illustrations, not representative or untouched-test evidence.
|
| 4 |
-
|
| 5 |
-
## Current research-report figures
|
| 6 |
-
|
| 7 |
-
The [Chinese report explanation](research_report_zh.md#三個颱風展示) contains the current **Fung-wong (2025), Mangkhut (2018) and Jebi (2018)** pressure-map, five-day route/central-pressure and observed-radius panels. The day-1–5 bar chart compares released 1.2 with DeepMind Mini only, using unchanged saved scores. [Figure hashes and source identities](report_figures/provenance.json).
|
| 8 |
-
|
| 9 |
-
The report's radius figures are **IBTrACS observed R34 quadrant records, not model predictions**. The pressure reference remains NCEP analysis, with a common 2.5° comparison grid and no shifted forecasts. Mangkhut and Jebi are in the training years; the examples are retrospective demonstrations, not additional untouched validation. Older films and experimental radius diagnostics below keep their original provenance and should not be confused with these report figures.
|
| 10 |
-
|
| 11 |
-
## Additional films
|
| 12 |
-
|
| 13 |
-
These earlier 50-member films retain a fixed issue-relative regional pressure patch. When the centre leaves it, only the saved coarse basin field covers that position. They use 4 hPa isobars and 23-second playback with a held ending; do not confuse them with the current moving-core Mangkhut rendering. Their original model outputs remain preserved.
|
| 14 |
-
|
| 15 |
-
- [Fung-wong (2025), 7 November 00 UTC](https://huggingface.co/euler314/typhoon-predict/resolve/87a6e366b42bb4cc0edc95d2c50c55fca21a2c93/docs/trackformer_1_2_fung_wong.mp4) · [provenance](../evaluation/release_data/fung_wong_video.json)
|
| 16 |
-
- [Soudelor (2015), 5 August 00 UTC](https://huggingface.co/euler314/typhoon-predict/resolve/87a6e366b42bb4cc0edc95d2c50c55fca21a2c93/docs/trackformer_1_2_soudelor.mp4) · [provenance](../evaluation/release_data/soudelor_video.json)
|
| 17 |
-
- [Meranti (2016), 10 September 00 UTC](https://huggingface.co/euler314/typhoon-predict/resolve/87a6e366b42bb4cc0edc95d2c50c55fca21a2c93/docs/trackformer_1_2_meranti.mp4) · [provenance](../evaluation/release_data/meranti_video.json)
|
| 18 |
-
|
| 19 |
-
## Wind and structure
|
| 20 |
-
|
| 21 |
-
[Four-storm wind comparison](../evaluation/storm_structure/four_storm_wind_candidate.png) · [audit index](../evaluation/storm_structure/index.json) · [experimental structure method](../release_tools/STORM_STRUCTURE_DIAGNOSTICS.md)
|
| 22 |
-
|
| 23 |
-
USA one-minute and JMA ten-minute winds are separate references. Native 1.1 RMW and quadrant R34/R50/R64 are scored against matching USA definitions. Trackformer 1.2 has an auxiliary maximum-wind scalar but no native radius head. Its pressure-derived radius candidate is not an official-equivalent prediction; masked/persistent components are not learned radius forecasts.
|
| 24 |
-
|
| 25 |
-
| Storm | Exact-time comparisons | Data and assumptions |
|
| 26 |
-
| --- | --- | --- |
|
| 27 |
-
| Fung-wong | [Plot](../evaluation/storm_structure/fung_wong_wind_radius.png) | [Record](../evaluation/storm_structure/fung_wong_structure.json) |
|
| 28 |
-
| Soudelor | [Plot](../evaluation/storm_structure/soudelor_wind_radius.png) | [Record](../evaluation/storm_structure/soudelor_structure.json) |
|
| 29 |
-
| Mangkhut | [Plot](../evaluation/storm_structure/mangkhut_wind_radius.png) | [Record](../evaluation/storm_structure/mangkhut_structure.json) |
|
| 30 |
-
| Meranti | [Plot](../evaluation/storm_structure/meranti_wind_radius.png) | [Record](../evaluation/storm_structure/meranti_structure.json) |
|
| 31 |
-
|
| 32 |
-
## Full evaluation
|
| 33 |
-
|
| 34 |
-
[Daily protocol](daily_storm_benchmark.md) · [all daily scores](../evaluation/daily_storm_final.json) · [intensity/radius protocol](intensity_benchmark.md) · [native intensity results](../evaluation/intensity/intensity_final.json) · [matched pressure curves](../evaluation/released_daily/pressure_comparison.png)
|
| 35 |
-
|
| 36 |
-
Route uses 1,473 starts / 270 storms; the matched native pressure comparison uses 134 starts / 40 storms. JMA pressure MAE falls from 13.53 to 12.84 hPa, with paired uncertainty including zero. JMA curve similarity is 0.7074 / 0.7118 (1.1 / 1.2), while USA curve similarity is 0.7237 / 0.6637 on 131 eligible days—a regression. A centred shape score discards level and amplitude and must be read with physical hPa timelines, not used to conceal pressure errors.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
docs/trackformer_1_2_evaluation.md
DELETED
|
@@ -1,64 +0,0 @@
|
|
| 1 |
-
# Trackformer 1.2 evaluation notes
|
| 2 |
-
|
| 3 |
-
## Completed equal-storm benchmark
|
| 4 |
-
|
| 5 |
-
The current comparison uses **1,473 daily forecasts across 270 Western Pacific storms**, with 20 six-hour leads from +6 through +120 h. Scores average leads within issues, daily issues within each storm, then storms equally. The completed results and all forecast SHA-256 checks are in [the final report](../evaluation/daily_storm_final.json); the [daily-storm protocol](daily_storm_benchmark.md) defines the frozen cohort and metrics.
|
| 6 |
-
|
| 7 |
-
The current [Chinese research-report view](research_report_zh.md) compares **released 1.2 with DeepMind Mini**, using unchanged saved results. TrackFormer uses the mean of 50 distinct causal input perturbations of one model, not 50 trained networks; Mini uses one seeded member. These are matched outputs with different inputs and member policies, not a controlled architecture ablation, equal-compute test or untouched generalization test.
|
| 8 |
-
|
| 9 |
-
| Report metric | TrackFormer 1.2 | DeepMind Mini |
|
| 10 |
-
| --- | ---: | ---: |
|
| 11 |
-
| Mean track error | 471.20 km | 498.37 km |
|
| 12 |
-
| +120 h track error | 1031.63 km | 1031.91 km |
|
| 13 |
-
| Six-hour direction error | 34.96° | 39.76° |
|
| 14 |
-
| Central-pressure MAE · JMA | 12.84 hPa | 10.20 hPa |
|
| 15 |
-
|
| 16 |
-
Direction, centred shape, geographic path and same-time position error measure different aspects of a forecast. A translated or mistimed route can have high shape similarity without overlapping the observed route. No combined weighted ranking is introduced.
|
| 17 |
-
|
| 18 |
-
**Central-pressure coverage is a subset of route coverage:** **40 of the 270 storms / 134 starts / 2,637 common valid lead records**. Each model's pressure is compared separately with the JMA record at the identical forecast time. This is not a separate 270-storm pressure evaluation. The original 1.2-only pressure mean of **12.62 hPa** uses a different mask and must not replace the report's common-support **12.84 hPa**. The original basin MSLP mean of **2.72 hPa on 270 storms** is a whole-map measure, not central-pressure error. [Common scores](../evaluation/released_daily/released_daily_benchmark.json) · [Pressure protocol](intensity_benchmark.md).
|
| 19 |
-
|
| 20 |
-
The inherited shared-case eligibility excludes **1,339 starts with unavailable frozen legacy issue-time intensity inputs**. That does **not** mean all excluded storms lack pressure observations. Original masks remain unchanged; unsupported scores are unavailable, not zero. The cohort contains 40 recent and 230 historical storms from 1980–1999. Retrospective analyses, complete-five-day eligibility and prior model selection limit interpretation.
|
| 21 |
-
|
| 22 |
-
The [report's day-1–5 chart](report_figures/day_1_to_5.png) shows **exact 24, 48, 72, 96 and 120 h** leads rather than daily-bin averages. Track covers 270 storms at each displayed lead. JMA pressure has 40, 40, 40, 40 and 39 storms, with 131, 133, 134, 133 and 127 common starts respectively. All-lead pressure totals and per-lead counts describe different aggregations. [Exact chart values](report_figures/day_1_to_5.json).
|
| 23 |
-
|
| 24 |
-
## Completed DeepMind comparison
|
| 25 |
-
|
| 26 |
-
Google's official **WeatherNext Cyclones Mini `<2024`**, using **WeatherNext software v0.3.0**, completed the same **1,473 daily starts / 270 storms** on an **NVIDIA GeForce RTX 3070 with CUDA**. It is a 1° model trained through 2023. Mini is one seeded member, not the full-sized WeatherNext model; 1.2 remains a 50-member mean. The common-support results are **498.37 km** mean track error, **1031.91 km** at +120 h, **39.76°** direction error and **10.20 hPa** JMA central-pressure MAE on **134 starts / 40 storms / 2,637 valid records**. Its JMA pressure-curve similarity is **0.8088**.
|
| 27 |
-
|
| 28 |
-
Mini's recent-only mean track error is **288.5 km** versus **477.2 km** for 1.2. Historical fitting-year overlap and different input/member policies limit the combined-cohort comparison; no general superiority claim is justified. The [completed protocol and period breakdowns](deepmind_daily_benchmark.md), [CUDA receipt](../evaluation/deepmind_daily/verification.json) and [independent three-model publication audit](../evaluation/deepmind_daily/publication_audit.json) establish completion. The older two-model receipt remains the original Trackformer audit, not the hash receipt for the subsequently merged snapshot. No missing or unsupported score is treated as zero.
|
| 29 |
-
|
| 30 |
-
That **recent-only** group includes 2024. The report-facing strict **after-2024** group is different: 2025–2026, 61 starts / 18 storms, with **258.7 km Mini versus 469.4 km 1.2**. Calendar 2024 is not part of that strict subset. Do not interchange these period scores.
|
| 31 |
-
|
| 32 |
-
## Report showcases and preserved earlier records
|
| 33 |
-
|
| 34 |
-
The [three report examples](research_report_zh.md#三個颱風展示) use unchanged released-1.2 weights: Fung-wong and Mangkhut are saved Metal forecasts; Jebi was run on CPU. All use 50 input-perturbed members and exact +6 to +120 h times. Mangkhut and Jebi are within the fitting years. Their R34 panels show **IBTrACS observations only**, not model radius skill. NCEP is still an analysis reference even where the figure label is simplified to “reference pressure”.
|
| 35 |
-
|
| 36 |
-
Earlier release comparisons and their pressure-curve diagnostics remain in the [archived shared snapshot](../evaluation/released_daily/released_daily_benchmark.json), [full original report](../evaluation/daily_storm_final.json) and [intensity protocol](intensity_benchmark.md). They are not silently substituted for the current report's two-model table. Centred curve similarity removes level and amplitude; read it with hPa MAE, masks and original timelines rather than treating it as accurate pressure levels.
|
| 37 |
-
|
| 38 |
-
## Recent pressure-line example
|
| 39 |
-
|
| 40 |
-
### Fung-wong MP4: selected route-and-pressure example
|
| 41 |
-
|
| 42 |
-
The [Fung-wong video](https://yu314-coder.github.io/typhoon-predict/trackformer_1_2_fung_wong.mp4) uses the unchanged saved 50-member pressure fields and mean route for case 262, issued 2025-11-07 00 UTC. It is a user-selected development example after inspection of full routes and pressure curves, not an untouched test or a representative sample. The geographic mean error is 130.9 km, the +120 h error is 142.3 km, and maximum error is 198.3 km over 20 leads. These are great-circle distances to exact observed coordinates, separate from the main benchmark's saved local-coordinate scores. The route follows the broad observed curve but is not perfectly overlapping.
|
| 43 |
-
|
| 44 |
-
The pressure MAE is **7.36 hPa** against IBTrACS TOKYO_PRES (JMA best track), with 20 valid labels. Intensification and weakening still differ from observations. Fields are physical model means on fixed common grids, without shifting them onto the observed track. Native-detail history was unavailable; 0.25° regional output sampling is not native resolution. From +66 h the forecast centre leaves the regional patch: only saved coarse basin pressure is shown there, while member-mean central pressure remains the moving-core readout. No display-only core is added. Observed future routes and pressures are verification overlays only. [Data and provenance](../evaluation/release_data/fung_wong_video.json) record hashes and member policy. Reproduce with `python release_tools/build_fung_wong_video.py --work /path/to/frames` using NumPy, Matplotlib and ffmpeg with VideoToolbox on macOS; this renders saved outputs without model inference. The former interactive showcase has been removed.
|
| 45 |
-
|
| 46 |
-
### Recent single-member example
|
| 47 |
-
|
| 48 |
-
The Surigae archive issued **2026-09-27 12 UTC** supplies real model regional MSLP fields at +6, +24 and +36 h. Isobars are drawn every 4 hPa, with selected labels every 12 hPa and one common colour scale. The magenta curve is the saved model route. Fields are neither relocated nor altered to force the map minimum onto the route. The regional map is a resampled composite and can differ slightly from central pressure sampled in the moving core.
|
| 49 |
-
|
| 50 |
-
This example is one deterministic member from the released checkpoint. Inputs contain nine GFS analyses ending at 06 UTC (six hours before issue), issue-time JMA centre/central pressure, and static geography. Issue-time wind, prior motion and native regional history were unavailable and masked/zeroed. The stored +0 field is an input analysis; it is excluded from these forecast panels. The GFS transfer and input lag remain limitations. The selected leads keep the centre within the fixed 114–144°E, 11–41°N map. Names, times, member count and checkpoint/source hashes are in `evaluation/release_data/surigae_provenance.json`.
|
| 51 |
-
|
| 52 |
-
## Reproduce the benchmark chart
|
| 53 |
-
|
| 54 |
-
```bash
|
| 55 |
-
python release_tools/plot_daily_storm_final.py
|
| 56 |
-
```
|
| 57 |
-
|
| 58 |
-
This plots the saved completed equal-storm results; it performs no new inference or weather retrieval. Original research arrays and reproduction tools remain preserved for provenance, but the retired smaller-cohort comparison is not presented as the current benchmark.
|
| 59 |
-
|
| 60 |
-
## Validation boundary
|
| 61 |
-
|
| 62 |
-
Training used whole-storm year partitions: 2000–2021 training, 2022–2023 validation, 2024–2025 test in the original pipeline, with 13,949 / 1,041 / 1,195 windows respectively; native-pressure windows total 800 / 100 / 100. Repeatedly viewed benchmarks, TIP and calibration cases have become development evidence. Freeze a genuinely unused storm-level holdout with matched issue-time data before claiming generalization. Evaluate route, central pressure and pressure fields at every lead on common cases and masks; a lower map average alone is not a model win.
|
| 63 |
-
|
| 64 |
-
The fitting sources are IBTrACS-derived storm records, NOAA PSL NCEP/NCAR basin analyses, ARCO-ERA5 regional MSLP and static ERA5 land fraction/elevation. Newer 2026 archive records are not part of the released model's gradient fitting. See [training-source documentation](trackformer_1_2_training_data.md) and [checksum-verified split receipt](../evaluation/training_data/trackformer_1_2_provenance.json).
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
docs/trackformer_1_2_training_data.md
DELETED
|
@@ -1,5 +0,0 @@
|
|
| 1 |
-
# TrackFormer 1.2 training data and process
|
| 2 |
-
|
| 3 |
-
The complete explanation is consolidated in **[one 106-page PDF](TrackFormer_1.2_Training_and_Reproducibility.pdf)**, rather than split across Markdown pages. It includes sources, exact corpus, train/validation/test partitions, eligibility, normalization, all input/target tensors and masks, training stages, active losses, optimizer and checkpoint details, reproduction commands, full original text-source appendices and provenance limits.
|
| 4 |
-
|
| 5 |
-
The original training-data audit remains machine-readable at [evaluation/training_data/trackformer_1_2_provenance.json](../evaluation/training_data/trackformer_1_2_provenance.json). Exact public assets and source are indexed at [training/trackformer_1_2/](../training/trackformer_1_2/). The PDF reproduces the full earlier data explanation and audit, including the unrecorded original raw IBTrACS snapshot ID; it does not invent missing provenance.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
evaluation/README.md
DELETED
|
@@ -1,24 +0,0 @@
|
|
| 1 |
-
# Evaluation assets
|
| 2 |
-
|
| 3 |
-
## Research-report view
|
| 4 |
-
|
| 5 |
-
The current [report-facing summary](../README.md#results-in-the-research-report) and [Chinese explanation](../docs/research_report_zh.md) compare **TrackFormer 1.2 with DeepMind Mini**. Their [day-1–5 chart](../docs/report_figures/day_1_to_5.png) uses exact 24/48/72/96/120 h leads. Route covers **270 storms / 1,473 starts**; central pressure uses **40 of those storms / 134 starts / 2,637 common valid lead records**. The pressure subset follows frozen scoring eligibility, not a claim that the remaining storms lack all pressure observations. Source arrays and the larger historical archive below are unchanged.
|
| 6 |
-
|
| 7 |
-
## Full historical archive
|
| 8 |
-
|
| 9 |
-
The current **completed three-model comparison** scores Trackformer 1.1, Trackformer 1.2 and Google's **WeatherNext Cyclones Mini `<2024` (software v0.3.0)** on **1,473 daily starts / 270 Western Pacific storms**, through +120 h. Mini's RTX 3070 CUDA run is complete and verified; Trackformer 1.2 is a mean of 50 members and Mini is one member.
|
| 10 |
-
|
| 11 |
-
- [Announcement chart](released_daily/model_1_2_benchmark.png) and [shared three-model scores](released_daily/released_daily_benchmark.json).
|
| 12 |
-
- [Completed DeepMind results](deepmind_daily/benchmark.json), [CUDA completion receipt](deepmind_daily/verification.json) and [independent publication audit](deepmind_daily/publication_audit.json).
|
| 13 |
-
- [Daily-case protocol](../docs/daily_storm_benchmark.md), [DeepMind model identity and period breakdowns](../docs/deepmind_daily_benchmark.md) and [Trackformer evaluation definitions](../docs/trackformer_1_2_evaluation.md).
|
| 14 |
-
- [Matched pressure, wind and radius evaluation](../docs/intensity_benchmark.md): pressure uses 134 shared daily starts / 40 storms, not all 270 storms. Unsupported scores stay unavailable, never zero.
|
| 15 |
-
|
| 16 |
-
Scores average valid leads within each day, days within each storm, then storms equally. The historical 1980–1999 group overlaps Mini's fitting years; recent 2024+ results are reported separately. This is development evidence, not a certified untouched holdout or an equal-compute architecture ablation.
|
| 17 |
-
|
| 18 |
-
For geographically aligned routes, compare forecasts and observations at the **same valid times on the same map**. Centred shape similarity alone cannot establish alignment. Never translate, rotate or rescale a forecast to make it match the observed route.
|
| 19 |
-
|
| 20 |
-
Reproduce the current three-model announcement chart from saved results with `python release_tools/plot_model_announcement.py`. This renders audited scores without new inference or weather retrieval; NumPy and Matplotlib are required. The paper is in [`../paper/`](../paper/), and saved pressure arrays and preview provenance are under [`release_data/`](release_data/).
|
| 21 |
-
|
| 22 |
-
## Archived diagnostics and selected examples
|
| 23 |
-
|
| 24 |
-
Older figures, the [TIP diagnostic](trackformer_1_2_vs_1_1_tip_metrics.json) and [showcase selection](trackformer_1_2_showcase_selection.json) are preserved for provenance, not presented as the current benchmark. See the [showcase archive](../docs/showcase_archive.md) for selected pressure-map films and their limitations.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
models/trackformer_1_2_field/README.md
CHANGED
|
@@ -4,7 +4,7 @@ The released pressure-field model predicts Western Pacific storm tracks, central
|
|
| 4 |
|
| 5 |
This folder contains the exact source, input contract and inference-only weights for the research model in the [main README](../../README.md). `model.py`, `baseline_model.py` and `v165_base.py` are source-identical to the selected training implementation. The weight file is distributed via the [GitHub release](https://github.com/yu314-coder/typhoon-predict/releases/tag/trackformer-1.2) and [Hugging Face](https://huggingface.co/euler314/typhoon-predict/tree/main/models/trackformer_1_2_field), not Git.
|
| 6 |
|
| 7 |
-
**Training-data cutoff:** fitting and weather normalization use **2000–2021**; validation/checkpoint selection uses **2022–2023**; the original test partition uses **2024–2025**. Inputs come from IBTrACS-derived storm records, NOAA PSL NCEP/NCAR Reanalysis basin fields, ARCO-ERA5 regional pressure and static land/elevation. Newer 2026 archive/live inputs are inference, not retraining. [Sources and exact split coverage](../../docs/
|
| 8 |
|
| 9 |
Use Python with PyTorch and NumPy. The export environment used PyTorch 2.13.0; the source was evaluated on macOS MPS. Example:
|
| 10 |
|
|
@@ -14,7 +14,7 @@ python models/trackformer_1_2_field/predict.py causal_issue_packet.npz forecast.
|
|
| 14 |
|
| 15 |
The `.npz` packet must contain exactly these arrays, with no object/pickle content:
|
| 16 |
|
| 17 |
-
The nine history timestamps span **−48 to 0 h**, including both endpoints. This is the release's fixed two-day input format, not evidence that two days outperforms one or three. The eight basin channels are MSLP, HGT500, U850, V850, U500, V500, U200 and V200: each u/v pair represents **both horizontal wind direction and speed**. Static coordinates, land fraction and elevation remain part of the actual input contract below. See the [plain-language Chinese report](../../docs/
|
| 18 |
|
| 19 |
| Key | Shape | Meaning |
|
| 20 |
| --- | --- | --- |
|
|
@@ -75,17 +75,17 @@ The auxiliary wind also has `maximum_wind_auxiliary_kt_valid`, a numerical/domai
|
|
| 75 |
validity mask, not a claim of calibrated wind skill.
|
| 76 |
These are uncalibrated circular-equivalent ocean estimates, not resolved
|
| 77 |
surface winds, agency-compatible sustained winds or quadrant radii. See
|
| 78 |
-
[the estimation method and validation requirements](
|
| 79 |
No model weights, learned architecture or existing benchmark results changed.
|
| 80 |
|
| 81 |
**Causality:** Never populate arrays from positive-lead analysis, later best track, agency forecast or future pressure field. The wrapper checks timestamp order and shapes, but cannot authenticate the upstream archive. Do not silently replace missing values with future data. This is not a raw-weather downloader or live warning service.
|
| 82 |
|
| 83 |
The published 50-member mean is a separate evaluation policy: 50 deterministic seeds perturb only normalized historical basin/regional fields with smooth zero-centred noise, then average routes, central pressures and common-grid fields. The weights are the same for every member; this is **not 50 separately trained models**. Issue-time centre, motion, intensity, masks and static geography are not perturbed. `predict.py` computes one clean forecast; do not label it a 50-member mean. Neither this member count nor the 48-hour history has been established as optimal by a matched ablation; spread is not automatically calibrated uncertainty.
|
| 84 |
|
| 85 |
-
**Report's matched development evaluation:** route uses the frozen **1,473 daily starts / 270 equal-weight storms**, +6 to +120 h. Central pressure uses **40 of those 270 storms**, **134 common starts / 2,637 valid lead records**, comparing each model separately with JMA at identical valid times. This is an inherited common-scoring subset, not evidence that all other storms lack pressure observations. Missing labels and unsupported outputs are not zero-scored. TrackFormer 1.2's mean route error is **471.20 km**, +120 h error **1031.63 km**, six-hour direction error **34.96°**, and JMA central-pressure MAE **12.84 hPa**. Central-pressure error is not whole-map error. See the [report figures](../../docs/
|
| 86 |
|
| 87 |
-
**Completed DeepMind comparison:** Google's official **WeatherNext Cyclones Mini `<2024`**, run with **WeatherNext software v0.3.0**, completed all **1,473 daily starts / 270 storms** on an **NVIDIA GeForce RTX 3070 using CUDA**. This is the 1° Mini checkpoint trained through 2023, not the full-sized WeatherNext model. Mini uses one seeded member; TrackFormer 1.2 uses a 50-member mean. Mini's report values are **498.37 km** mean track error, **1031.91 km** at +120 h, **39.76°** direction error and **10.20 hPa** JMA central-pressure MAE on the **134 starts / 40 storms / 2,637 common valid records**. The combined cohort favours 1.2 for position and heading, but Mini has lower pressure MAE. Mini also has lower recent-only track error: **258.7 versus 469.4 km** on the strict 2025–2026 subset (61 starts / 18 storms). Different input/member policies and prior case inspection prevent general superiority, equal-compute or untouched-holdout claims. [Completed results and model identity](../../docs/
|
| 88 |
|
| 89 |
**Rainfall and radius scope:** ongoing Taiwan rainfall work is separate from these weights. Released 1.2 has no rainfall head or humidity input. The report's three R34 diagrams are **observed IBTrACS quadrant radii**, not native 1.2 radius forecasts. Experimental pressure-derived radius outputs described above remain separately labelled diagnostics.
|
| 90 |
|
| 91 |
-
**Pressure-display coverage:** `regional_mslp_hpa` remains the original fixed issue-relative composite. Use the separately exported `core_mslp_hpa` and its per-lead coordinates/mask when the storm moves away from that fixed patch. Do not extend it by inventing a vortex or moving it onto a route. If the moving core leaves supported basin geography, the corresponding masked cells remain unavailable. For an ensemble, register each physical member field onto a common geographic grid **before** averaging; do not average moving-frame array indices. The Mangkhut film uses that separate 50-member policy. Older fixed-patch films retain their original data in [the showcase archive](../../docs/
|
|
|
|
| 4 |
|
| 5 |
This folder contains the exact source, input contract and inference-only weights for the research model in the [main README](../../README.md). `model.py`, `baseline_model.py` and `v165_base.py` are source-identical to the selected training implementation. The weight file is distributed via the [GitHub release](https://github.com/yu314-coder/typhoon-predict/releases/tag/trackformer-1.2) and [Hugging Face](https://huggingface.co/euler314/typhoon-predict/tree/main/models/trackformer_1_2_field), not Git.
|
| 6 |
|
| 7 |
+
**Training-data cutoff:** fitting and weather normalization use **2000–2021**; validation/checkpoint selection uses **2022–2023**; the original test partition uses **2024–2025**. Inputs come from IBTrACS-derived storm records, NOAA PSL NCEP/NCAR Reanalysis basin fields, ARCO-ERA5 regional pressure and static land/elevation. Newer 2026 archive/live inputs are inference, not retraining. [Sources and exact split coverage](../../docs/TrackFormer_1.2_Training_and_Reproducibility.pdf) · [Verified training-data receipt](../../evaluation/training_data/trackformer_1_2_provenance.json).
|
| 8 |
|
| 9 |
Use Python with PyTorch and NumPy. The export environment used PyTorch 2.13.0; the source was evaluated on macOS MPS. Example:
|
| 10 |
|
|
|
|
| 14 |
|
| 15 |
The `.npz` packet must contain exactly these arrays, with no object/pickle content:
|
| 16 |
|
| 17 |
+
The nine history timestamps span **−48 to 0 h**, including both endpoints. This is the release's fixed two-day input format, not evidence that two days outperforms one or three. The eight basin channels are MSLP, HGT500, U850, V850, U500, V500, U200 and V200: each u/v pair represents **both horizontal wind direction and speed**. Static coordinates, land fraction and elevation remain part of the actual input contract below. See the [plain-language Chinese report](../../docs/REFERENCE.md#docs-research-report-zh-md) for the research motivation and household-computer equipment.
|
| 18 |
|
| 19 |
| Key | Shape | Meaning |
|
| 20 |
| --- | --- | --- |
|
|
|
|
| 75 |
validity mask, not a claim of calibrated wind skill.
|
| 76 |
These are uncalibrated circular-equivalent ocean estimates, not resolved
|
| 77 |
surface winds, agency-compatible sustained winds or quadrant radii. See
|
| 78 |
+
[the estimation method and validation requirements](../../docs/REFERENCE.md#models-trackformer-1-2-field-wind-estimation-md).
|
| 79 |
No model weights, learned architecture or existing benchmark results changed.
|
| 80 |
|
| 81 |
**Causality:** Never populate arrays from positive-lead analysis, later best track, agency forecast or future pressure field. The wrapper checks timestamp order and shapes, but cannot authenticate the upstream archive. Do not silently replace missing values with future data. This is not a raw-weather downloader or live warning service.
|
| 82 |
|
| 83 |
The published 50-member mean is a separate evaluation policy: 50 deterministic seeds perturb only normalized historical basin/regional fields with smooth zero-centred noise, then average routes, central pressures and common-grid fields. The weights are the same for every member; this is **not 50 separately trained models**. Issue-time centre, motion, intensity, masks and static geography are not perturbed. `predict.py` computes one clean forecast; do not label it a 50-member mean. Neither this member count nor the 48-hour history has been established as optimal by a matched ablation; spread is not automatically calibrated uncertainty.
|
| 84 |
|
| 85 |
+
**Report's matched development evaluation:** route uses the frozen **1,473 daily starts / 270 equal-weight storms**, +6 to +120 h. Central pressure uses **40 of those 270 storms**, **134 common starts / 2,637 valid lead records**, comparing each model separately with JMA at identical valid times. This is an inherited common-scoring subset, not evidence that all other storms lack pressure observations. Missing labels and unsupported outputs are not zero-scored. TrackFormer 1.2's mean route error is **471.20 km**, +120 h error **1031.63 km**, six-hour direction error **34.96°**, and JMA central-pressure MAE **12.84 hPa**. Central-pressure error is not whole-map error. See the [report figures](../../docs/REFERENCE.md#docs-research-report-zh-md), [frozen common scores](../../evaluation/released_daily/released_daily_benchmark.json), [pressure protocol](../../docs/REFERENCE.md#docs-intensity-benchmark-md) and [detailed evaluation notes](../../docs/REFERENCE.md#docs-trackformer-1-2-evaluation-md).
|
| 86 |
|
| 87 |
+
**Completed DeepMind comparison:** Google's official **WeatherNext Cyclones Mini `<2024`**, run with **WeatherNext software v0.3.0**, completed all **1,473 daily starts / 270 storms** on an **NVIDIA GeForce RTX 3070 using CUDA**. This is the 1° Mini checkpoint trained through 2023, not the full-sized WeatherNext model. Mini uses one seeded member; TrackFormer 1.2 uses a 50-member mean. Mini's report values are **498.37 km** mean track error, **1031.91 km** at +120 h, **39.76°** direction error and **10.20 hPa** JMA central-pressure MAE on the **134 starts / 40 storms / 2,637 common valid records**. The combined cohort favours 1.2 for position and heading, but Mini has lower pressure MAE. Mini also has lower recent-only track error: **258.7 versus 469.4 km** on the strict 2025–2026 subset (61 starts / 18 storms). Different input/member policies and prior case inspection prevent general superiority, equal-compute or untouched-holdout claims. [Completed results and model identity](../../docs/REFERENCE.md#docs-deepmind-daily-benchmark-md) · [CUDA completion receipt](../../evaluation/deepmind_daily/verification.json) · [Independent publication audit](../../evaluation/deepmind_daily/publication_audit.json). No older benchmark's DeepMind scores were substituted.
|
| 88 |
|
| 89 |
**Rainfall and radius scope:** ongoing Taiwan rainfall work is separate from these weights. Released 1.2 has no rainfall head or humidity input. The report's three R34 diagrams are **observed IBTrACS quadrant radii**, not native 1.2 radius forecasts. Experimental pressure-derived radius outputs described above remain separately labelled diagnostics.
|
| 90 |
|
| 91 |
+
**Pressure-display coverage:** `regional_mslp_hpa` remains the original fixed issue-relative composite. Use the separately exported `core_mslp_hpa` and its per-lead coordinates/mask when the storm moves away from that fixed patch. Do not extend it by inventing a vortex or moving it onto a route. If the moving core leaves supported basin geography, the corresponding masked cells remain unavailable. For an ensemble, register each physical member field onto a common geographic grid **before** averaging; do not average moving-frame array indices. The Mangkhut film uses that separate 50-member policy. Older fixed-patch films retain their original data in [the showcase archive](../../docs/REFERENCE.md#docs-showcase-archive-md). The automatic History archive and its separate core recovery expose geography and coverage through [the public API](https://trackformer-weatherlab.rudin-euler-8253.chatgpt.site/history-api).
|
models/trackformer_1_2_field/WIND_ESTIMATION.md
DELETED
|
@@ -1,46 +0,0 @@
|
|
| 1 |
-
# Experimental wind and circular-radius diagnostics
|
| 2 |
-
|
| 3 |
-
The frozen 1.2 weights and neural architecture are unchanged. Maximum wind is
|
| 4 |
-
the existing learned auxiliary `vmax` output, in knots, not a resolved wind map.
|
| 5 |
-
Its averaging period is not harmonized to JMA, and skill has not been established
|
| 6 |
-
by the route/pressure benchmark. Do not treat this product as operational advice.
|
| 7 |
-
|
| 8 |
-
`wind_estimation.py` optionally diagnoses each model-generated pressure member:
|
| 9 |
-
|
| 10 |
-
1. Sample complete circular rings about that member's predicted centre, no more
|
| 11 |
-
finely than the source/reconstruction grid, out to at most 480 km.
|
| 12 |
-
2. Compute the azimuthal-mean radial pressure gradient (hPa converted to Pa).
|
| 13 |
-
3. Solve `Vg^2/r + |f| Vg = (1/rho) dp/dr`, with air density 1.15 kg/m3, then
|
| 14 |
-
apply a fixed 0.8 ocean surface-reduction assumption. This is uncalibrated;
|
| 15 |
-
motion, land friction, asymmetry, boundary-layer jets and averaging periods
|
| 16 |
-
are not represented. No future observations or official routes are inputs.
|
| 17 |
-
4. Report a resolved gradient-wind peak/RMW and outward 34/50/64-kt crossings.
|
| 18 |
-
These are **circular equivalent radius estimates**, not quadrant radii and
|
| 19 |
-
not isobar radii. Coarse (>50 km) source grids, incomplete rings, unresolved
|
| 20 |
-
peaks and missing crossings produce nulls, not invented zeros or edge radii.
|
| 21 |
-
5. Diagnose members individually, then summarize. A mean is supplied only when
|
| 22 |
-
every member has that estimate. Valid counts and p10/p90 describe spread;
|
| 23 |
-
they are not calibrated confidence intervals. Never diagnose a mean map and
|
| 24 |
-
call it a mean of 50 radius predictions.
|
| 25 |
-
|
| 26 |
-
The 20-km moving core is a learned reconstruction. A resampled 0.25-degree
|
| 27 |
-
regional image does not add resolved observations. Even an available estimate
|
| 28 |
-
therefore remains experimental. A single gridded basin mean cannot reconstruct
|
| 29 |
-
the missing member diagnostics. Legacy forecasts without saved member/core
|
| 30 |
-
outputs retain unavailable estimates instead of being rerun or relabelled.
|
| 31 |
-
|
| 32 |
-
For the saved 50-member regional composite, core origins were not archived.
|
| 33 |
-
That release's core spans +/-640 km and centre association is restricted to
|
| 34 |
-
300 km about its origin. Consequently this read-only legacy exporter samples
|
| 35 |
-
only rings within 300 km of each predicted centre, conservatively inside the
|
| 36 |
-
core footprint; a missing crossing beyond that limit remains unavailable.
|
| 37 |
-
New forecasts use the actual moving-core coordinates and their valid masks.
|
| 38 |
-
|
| 39 |
-
Defaults are fixed physical assumptions, not fitted to showcase outcomes. Before
|
| 40 |
-
claiming skill, freeze eligible whole-storm held-out cases; score +6 to +120 h
|
| 41 |
-
maximum winds and agency-compatible wind radii separately, with coverage,
|
| 42 |
-
unresolved cases, land/ocean breakdowns and whole-storm uncertainty. No such
|
| 43 |
-
validation or calibration is claimed by this implementation.
|
| 44 |
-
|
| 45 |
-
References: [NOAA PaHM theory and limitations](https://noaa-ocs-modeling.github.io/PaHM/pahm_manual.pdf)
|
| 46 |
-
and [Chavas, Reed and Knaff (2017)](https://doi.org/10.1038/s41467-017-01546-9).
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
release_tools/STORM_STRUCTURE_DIAGNOSTICS.md
DELETED
|
@@ -1,118 +0,0 @@
|
|
| 1 |
-
# Any-storm wind and radius plots
|
| 2 |
-
|
| 3 |
-
This is a local diagnostic/plotting extension. It does not change the released
|
| 4 |
-
Trackformer 1.2 weights, existing pressure maps, videos, cloud archive, or Site.
|
| 5 |
-
It is **not** a new validated wind/radius forecast model.
|
| 6 |
-
|
| 7 |
-
## What works
|
| 8 |
-
|
| 9 |
-
- The renderer accepts any exact SID in the local Western Pacific IBTrACS file.
|
| 10 |
-
The October 1 local snapshot contains 1,910 storms since 1970. This is an
|
| 11 |
-
observational coverage catalogue, **not** a claim that 1,910 forecasts ran.
|
| 12 |
-
- Every storm can produce a chart layout. Pressure, wind, RMW, and quadrant
|
| 13 |
-
plots are drawn where reports exist. An empty panel explains missing data.
|
| 14 |
-
- Forecast plotting uses genuine, hashed physical member archives. All four
|
| 15 |
-
examples now have 20 candidate wind and R34/R50/R64 quadrant outputs through
|
| 16 |
-
+120 h, diagnosed from all 50 members before taking the mean.
|
| 17 |
-
- The historical MPS runner additionally accepts arbitrary `--case SLUG SID
|
| 18 |
-
ISSUE_UTC`. It still requires nine exact six-hour analyses in the existing
|
| 19 |
-
local atlas and the released WP domain. It neither downloads missing weather
|
| 20 |
-
nor changes the independently authorized cloud backfill.
|
| 21 |
-
|
| 22 |
-
## Candidate assumptions and official definitions
|
| 23 |
-
|
| 24 |
-
IBTrACS `USA_WIND` is a 1-minute reference. USA R34/R50/R64 are maximum extents
|
| 25 |
-
in the NE/SE/SW/NW quadrants, in nautical miles; multiply by 1.852 for **radius**
|
| 26 |
-
in km, never by two. JMA wind is a separate 10-minute reference. JMA R30 cannot
|
| 27 |
-
be relabeled R34, and longest/shortest JMA radii are not USA quadrant radii.
|
| 28 |
-
Blank values remain missing; a native reported zero remains zero. USA RMW is
|
| 29 |
-
not universally reanalyzed best track, especially in WP. Fung-Wong's selected
|
| 30 |
-
2025 USA reports are provisional TCVitals, not final JTWC observations.
|
| 31 |
-
See the [NOAA IBTrACS column documentation](https://www.ncei.noaa.gov/sites/default/files/2025-09/IBTrACS_v04r01_column_documentation.pdf).
|
| 32 |
-
|
| 33 |
-
The new experimental extension starts with only the **exact issue-time** USA
|
| 34 |
-
wind, RMW and available quadrant isotachs. No future wind/radius is fitted.
|
| 35 |
-
The neural runner itself continues to use its original JMA issue-intensity
|
| 36 |
-
inputs; USA structure is a separate diagnostic input, not a silent neural
|
| 37 |
-
input substitution. Reference and model centers can differ, and both are
|
| 38 |
-
recorded in the initialization receipt.
|
| 39 |
-
|
| 40 |
-
For each forecast member, ambient pressure is the median actual basin cells
|
| 41 |
-
600–1000 km from that member's predicted center, with at least two available
|
| 42 |
-
cells in each quadrant. This uses coarse pressure for the **ambient value**,
|
| 43 |
-
not to claim a resolved eyewall. Let `D0 = Pambient0 - Pmodel_issue` and
|
| 44 |
-
`Dm = Pambient_member - Pcentral_member`. The candidate uses
|
| 45 |
-
|
| 46 |
-
```text
|
| 47 |
-
wind_member = USA_wind_issue * sqrt(Dm / D0)
|
| 48 |
-
```
|
| 49 |
-
|
| 50 |
-
The square-root deficit scaling is motivated by gradient/Holland balance,
|
| 51 |
-
but this extension is **not PaHM/GAHM**. It builds a decreasing outer envelope
|
| 52 |
-
from consistent issue-time quadrant isotachs, interpolates in log-radius and
|
| 53 |
-
log-wind, and extrapolates with a Holland-inspired `V ~ r^(-B/2)` tail. A fixed
|
| 54 |
-
literature-motivated B range 1–2.5 is used, without showcase tuning. Radius
|
| 55 |
-
solutions beyond 1,500 km remain unavailable rather than becoming a grid-edge
|
| 56 |
-
radius. Nonphysical pressure, unresolved backgrounds, out-of-domain tracks and
|
| 57 |
-
inconsistent zero isotachs remain explicit failures.
|
| 58 |
-
The [NOAA PaHM theory](https://noaa-ocs-modeling.github.io/PaHM/html/models.html)
|
| 59 |
-
also explains why gradient wind is not automatically a validated 10-m wind.
|
| 60 |
-
|
| 61 |
-
RMW is currently **issue-time persistence**. The flat dashed RMW curve is not
|
| 62 |
-
a model-measured future RMW, an eye diameter, or proof of radius skill.
|
| 63 |
-
The candidate's nominal threshold/units/quadrant geometry target the native
|
| 64 |
-
USA definition, but its 1-minute surface-wind averaging behavior has not been
|
| 65 |
-
validated. The strict scorer therefore rejects wind/radius skill claims.
|
| 66 |
-
Plots are exploratory reference comparisons, not a fair validated benchmark.
|
| 67 |
-
All four are development examples; no unused generalization holdout is claimed.
|
| 68 |
-
|
| 69 |
-
## Reusable commands
|
| 70 |
-
|
| 71 |
-
Use the existing Python environment; no installation is needed. All outputs,
|
| 72 |
-
temporary files and Matplotlib caches must stay on `/Volumes/D`.
|
| 73 |
-
|
| 74 |
-
Draw a full observed storm, even without a model-member archive:
|
| 75 |
-
|
| 76 |
-
```sh
|
| 77 |
-
/Volumes/D/typhoon_predict/.venv/bin/python -B \
|
| 78 |
-
/Volumes/D/typhoon_predict/remote_typhoon_predict_repo/release_tools/plot_storm_wind_radius.py \
|
| 79 |
-
--ibtracs /Volumes/D/typhoon_predict/data/ibtracs/ibtracs.WP.list.v04r01.csv \
|
| 80 |
-
--output /Volumes/D/typhoon_predict/output/nari_observed_plots \
|
| 81 |
-
--observed-storm 2001248N23125
|
| 82 |
-
```
|
| 83 |
-
|
| 84 |
-
Generate the data-coverage catalogue with `--catalogue-only`. To draw a saved
|
| 85 |
-
model ensemble, pass `--forecast-folder` (repeatable), pointing to one immutable
|
| 86 |
-
folder containing `verification.json`, `*_video.json`, `*_video.npz`,
|
| 87 |
-
`ensemble-members.npz`, `weather-history.npz` and the original input receipt.
|
| 88 |
-
|
| 89 |
-
Create an explicit additional historical ensemble on the Mac GPU only when
|
| 90 |
-
the required nine local analyses and exact issue report exist:
|
| 91 |
-
|
| 92 |
-
```sh
|
| 93 |
-
/Volumes/D/typhoon_predict/.venv/bin/python -B \
|
| 94 |
-
/Volumes/D/typhoon_predict/remote_typhoon_predict_repo/release_tools/forecast_historical_video50_mac.py \
|
| 95 |
-
--project /Volumes/D/typhoon_predict \
|
| 96 |
-
--output /Volumes/D/typhoon_predict/output/nari_ensemble50 \
|
| 97 |
-
--case nari 2001248N23125 2001-09-08T00:00:00Z --chunk 5
|
| 98 |
-
```
|
| 99 |
-
|
| 100 |
-
SID/date in a command is an explicit requested issue, not a nearest-date fill.
|
| 101 |
-
If unavailable, choose an actually reported issue after checking the catalogue.
|
| 102 |
-
The runner refuses duplicate issues, unsafe slugs, three-hour dates, changed
|
| 103 |
-
completed slugs and unsupported causal input histories.
|
| 104 |
-
|
| 105 |
-
## October 1 verification
|
| 106 |
-
|
| 107 |
-
Local results: `/Volumes/D/typhoon_predict/output/all-storm-wind-radius.uSvmWY/plots`.
|
| 108 |
-
Fung-Wong needed a fresh 50-member MPS replay (14.33 seconds inference). Its
|
| 109 |
-
inputs/output differ from the older benchmark video and it is clearly labeled;
|
| 110 |
-
the original video/data were not overwritten. Soudelor, Mangkhut and Meranti
|
| 111 |
-
reuse the exact saved original member archives. All mean routes and common-grid
|
| 112 |
-
pressure fields are rechecked against their saved video arrays. No inference
|
| 113 |
-
or fitting is required for the three archived issues or for observed-only plots.
|
| 114 |
-
|
| 115 |
-
Before promotion: develop a genuine evolving RMW/outer structure model, fit any
|
| 116 |
-
calibration only on declared whole-storm training data, freeze a genuinely unused
|
| 117 |
-
storm cohort, validate surface-wind period/height and quadrant definitions, and
|
| 118 |
-
report errors plus failed/missing coverage for identical leads through +120 h.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
release_tools/sync_public_model_cards.py
CHANGED
|
@@ -4,6 +4,7 @@ Preserve remote metadata, the featured film and frozen neural weights. Default
|
|
| 4 |
mode only prepares artifacts on D; --publish explicitly updates the HF card,
|
| 5 |
matching existing inference wrapper/docs and this small reproduction utility.
|
| 6 |
No forecasts are rerun, no media are regenerated and no neural modules change.
|
|
|
|
| 7 |
"""
|
| 8 |
from __future__ import annotations
|
| 9 |
|
|
@@ -21,13 +22,20 @@ MEDIA_REVISION = '87a6e366b42bb4cc0edc95d2c50c55fca21a2c93'
|
|
| 21 |
CACHE = '/Volumes/D/typhoon_predict/.cache/huggingface-intensity'
|
| 22 |
HF = 'https://huggingface.co/' + REPO
|
| 23 |
FILMS = ('mangkhut',)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 24 |
CURRENT_FIGURES = frozenset((
|
| 25 |
'evaluation/released_daily/model_1_2_benchmark.png',
|
| 26 |
'evaluation/released_daily/model_1_2_after_2024_benchmark.png',
|
| 27 |
'evaluation/released_daily/model_1_2_before_2024_benchmark.png',
|
| 28 |
'docs/trackformer_1_2_architecture.svg',
|
|
|
|
| 29 |
))
|
| 30 |
-
|
| 31 |
'models/trackformer_1_2_field/README.md',
|
| 32 |
'models/trackformer_1_2_field/WIND_ESTIMATION.md',
|
| 33 |
'models/trackformer_1_2_field/predict.py',
|
|
@@ -81,7 +89,37 @@ SYNC_FILES = (
|
|
| 81 |
'evaluation/deepmind_daily/case-manifest.json',
|
| 82 |
'evaluation/deepmind_daily/canary.json',
|
| 83 |
'evaluation/deepmind_daily/period_comparison.json',
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 84 |
)
|
|
|
|
|
|
|
|
|
|
|
|
|
| 85 |
|
| 86 |
|
| 87 |
def sha(path):
|
|
@@ -108,7 +146,9 @@ def render_card(original, github, figure_revision='main'):
|
|
| 108 |
raise ValueError('Preserve existing HF YAML metadata; missing header')
|
| 109 |
if github.count('## See the forecast: Mangkhut (2018)\n') != 1:
|
| 110 |
raise ValueError('Missing or ambiguous featured-film section')
|
| 111 |
-
|
|
|
|
|
|
|
| 112 |
for stem, alt in (('mangkhut', 'Trackformer 1.2 Mangkhut pressure and route forecast'),):
|
| 113 |
old = '[](' + media_url(stem) + ')'
|
| 114 |
if card.count(old) != 1:
|
|
@@ -125,7 +165,8 @@ def render_card(original, github, figure_revision='main'):
|
|
| 125 |
if not resolved.is_relative_to(ROOT) or not resolved.exists():
|
| 126 |
raise ValueError('Invalid or missing relative public link: ' + path)
|
| 127 |
revision = figure_revision if path in CURRENT_FIGURES else MEDIA_REVISION
|
| 128 |
-
kind = 'resolve/' + revision if image
|
|
|
|
| 129 |
target = HF + '/' + kind + '/' + path + ('#' + fragment if fragment else '')
|
| 130 |
return image + '[' + label + '](' + target + ')'
|
| 131 |
card = re.sub(r'(!?)\[([^\[\]\n]*)\]\(([^)\s]+)\)', link, card)
|
|
@@ -158,24 +199,48 @@ def frozen_identity(api, revision):
|
|
| 158 |
return actual
|
| 159 |
|
| 160 |
|
| 161 |
-
def
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 162 |
# Lazy imports keep portable CI document tests independent of HF packages.
|
| 163 |
from huggingface_hub import CommitOperationAdd, HfApi, hf_hub_download
|
| 164 |
output = output.resolve()
|
| 165 |
if not output.is_relative_to(Path('/Volumes/D')):
|
| 166 |
raise ValueError('All generated documentation/cache artifacts must stay on D')
|
| 167 |
output.mkdir(parents=True, exist_ok=True)
|
| 168 |
-
api = HfApi()
|
| 169 |
parent = api.model_info(REPO).sha
|
| 170 |
weights_before = frozen_identity(api, parent)
|
| 171 |
-
original = Path(hf_hub_download(REPO, 'README.md', revision=parent, cache_dir=CACHE))
|
| 172 |
github = (ROOT / 'README.md').read_text()
|
| 173 |
card_path = output / 'README.md'
|
| 174 |
card_path.write_text(render_card(original.read_text(), github))
|
| 175 |
-
|
|
|
|
| 176 |
source_commit = subprocess.check_output(['git', 'rev-parse', 'HEAD'], cwd=ROOT, text=True).strip()
|
| 177 |
receipt = {
|
| 178 |
'status': 'prepared', 'repo_id': REPO, 'parent_commit': parent,
|
|
|
|
| 179 |
'github_source_commit': source_commit,
|
| 180 |
'github_readme_sha256': sha(ROOT / 'README.md'),
|
| 181 |
'original_hf_readme_sha256': sha(original),
|
|
@@ -186,23 +251,29 @@ def run(output, publish=False):
|
|
| 186 |
'forecast_arrays_changed': False, 'videos_changed': False,
|
| 187 |
}
|
| 188 |
if publish:
|
| 189 |
-
dirty = subprocess.check_output(['git', 'status', '--porcelain', '--', 'README.md', *
|
| 190 |
if dirty:
|
| 191 |
raise ValueError('Commit canonical README and matching source files before publishing')
|
| 192 |
# Publish scientific assets first, then pin the new figures to that
|
| 193 |
# immutable commit in the card. Never pin new bytes to the old movie
|
| 194 |
# revision or let a missing future image render as a stale figure.
|
| 195 |
-
|
| 196 |
-
|
| 197 |
-
|
| 198 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 199 |
receipt['files']['README.md'] = sha(card_path)
|
| 200 |
-
receipt['figure_revision'] =
|
| 201 |
-
|
| 202 |
-
|
|
|
|
| 203 |
operations=[CommitOperationAdd(path_in_repo='README.md', path_or_fileobj=str(card_path))])
|
| 204 |
for name, expected in receipt['files'].items():
|
| 205 |
-
saved = hf_hub_download(REPO, name, revision=committed.oid, cache_dir=CACHE)
|
| 206 |
if sha(saved) != expected:
|
| 207 |
raise ValueError('Published documentation/source differs: ' + name)
|
| 208 |
if frozen_identity(api, committed.oid) != weights_before:
|
|
@@ -217,5 +288,7 @@ if __name__ == '__main__':
|
|
| 217 |
parser = argparse.ArgumentParser(description=__doc__)
|
| 218 |
parser.add_argument('--output', type=Path, required=True)
|
| 219 |
parser.add_argument('--publish', action='store_true')
|
|
|
|
|
|
|
| 220 |
args = parser.parse_args()
|
| 221 |
-
run(args.output, args.publish)
|
|
|
|
| 4 |
mode only prepares artifacts on D; --publish explicitly updates the HF card,
|
| 5 |
matching existing inference wrapper/docs and this small reproduction utility.
|
| 6 |
No forecasts are rerun, no media are regenerated and no neural modules change.
|
| 7 |
+
--docs-only limits uploads to an explicit documentation/figure allowlist.
|
| 8 |
"""
|
| 9 |
from __future__ import annotations
|
| 10 |
|
|
|
|
| 22 |
CACHE = '/Volumes/D/typhoon_predict/.cache/huggingface-intensity'
|
| 23 |
HF = 'https://huggingface.co/' + REPO
|
| 24 |
FILMS = ('mangkhut',)
|
| 25 |
+
REPORT_FIGURES = tuple('docs/report_figures/' + name for name in (
|
| 26 |
+
'day_1_to_5.png',
|
| 27 |
+
'fung_wong_pressure_comparison.png', 'fung_wong_track_pressure.png', 'fung_wong_observed_r34.png',
|
| 28 |
+
'mangkhut_pressure_comparison.png', 'mangkhut_track_pressure.png', 'mangkhut_observed_r34.png',
|
| 29 |
+
'jebi_pressure_comparison.png', 'jebi_track_pressure.png', 'jebi_observed_r34.png',
|
| 30 |
+
))
|
| 31 |
CURRENT_FIGURES = frozenset((
|
| 32 |
'evaluation/released_daily/model_1_2_benchmark.png',
|
| 33 |
'evaluation/released_daily/model_1_2_after_2024_benchmark.png',
|
| 34 |
'evaluation/released_daily/model_1_2_before_2024_benchmark.png',
|
| 35 |
'docs/trackformer_1_2_architecture.svg',
|
| 36 |
+
*REPORT_FIGURES,
|
| 37 |
))
|
| 38 |
+
LEGACY_SYNC_FILES = (
|
| 39 |
'models/trackformer_1_2_field/README.md',
|
| 40 |
'models/trackformer_1_2_field/WIND_ESTIMATION.md',
|
| 41 |
'models/trackformer_1_2_field/predict.py',
|
|
|
|
| 89 |
'evaluation/deepmind_daily/case-manifest.json',
|
| 90 |
'evaluation/deepmind_daily/canary.json',
|
| 91 |
'evaluation/deepmind_daily/period_comparison.json',
|
| 92 |
+
'docs/research_report_zh.md',
|
| 93 |
+
'docs/report_figures/day_1_to_5.json',
|
| 94 |
+
'docs/report_figures/provenance.json',
|
| 95 |
+
*REPORT_FIGURES,
|
| 96 |
+
)
|
| 97 |
+
LEGACY_DOC_SYNC_FILES = (
|
| 98 |
+
'models/trackformer_1_2_field/README.md',
|
| 99 |
+
'models/trackformer_1_2_field/WIND_ESTIMATION.md',
|
| 100 |
+
'RELEASE_NOTES_TRACKFORMER_1_2.md',
|
| 101 |
+
'docs/daily_storm_benchmark.md', 'docs/intensity_benchmark.md',
|
| 102 |
+
'docs/trackformer_1_2_evaluation.md', 'docs/trackformer_1_2_training_data.md',
|
| 103 |
+
'docs/deepmind_daily_benchmark.md', 'docs/showcase_archive.md',
|
| 104 |
+
'evaluation/README.md', 'docs/research_report_zh.md',
|
| 105 |
+
'docs/report_figures/day_1_to_5.json', 'docs/report_figures/provenance.json',
|
| 106 |
+
'docs/trackformer_1_2_architecture.svg', 'docs/trackformer_1_2_mangkhut.gif',
|
| 107 |
+
*REPORT_FIGURES,
|
| 108 |
+
)
|
| 109 |
+
|
| 110 |
+
|
| 111 |
+
_DOC_INDEX = json.loads((ROOT / 'docs/archive/documentation-manifest.json').read_text())
|
| 112 |
+
_RETIRED_DOCS = frozenset(_DOC_INDEX['removed_paths'])
|
| 113 |
+
_COMPACT_DOCS = (
|
| 114 |
+
'docs/REFERENCE.md',
|
| 115 |
+
'docs/TrackFormer_1.2_Training_and_Reproducibility.pdf',
|
| 116 |
+
'docs/archive/documentation-before-consolidation-20261030.zip',
|
| 117 |
+
'docs/archive/documentation-manifest.json',
|
| 118 |
)
|
| 119 |
+
SYNC_FILES = tuple(dict.fromkeys(name for name in (*LEGACY_SYNC_FILES, *_COMPACT_DOCS)
|
| 120 |
+
if name not in _RETIRED_DOCS))
|
| 121 |
+
DOC_SYNC_FILES = tuple(dict.fromkeys(name for name in (*LEGACY_DOC_SYNC_FILES, *_COMPACT_DOCS)
|
| 122 |
+
if name not in _RETIRED_DOCS))
|
| 123 |
|
| 124 |
|
| 125 |
def sha(path):
|
|
|
|
| 146 |
raise ValueError('Preserve existing HF YAML metadata; missing header')
|
| 147 |
if github.count('## See the forecast: Mangkhut (2018)\n') != 1:
|
| 148 |
raise ValueError('Missing or ambiguous featured-film section')
|
| 149 |
+
# The canonical GitHub README also carries valid YAML now. Retain exactly
|
| 150 |
+
# the existing remote header rather than nesting a second YAML block.
|
| 151 |
+
card = re.sub(r'\A---\n.*?\n---\n', '', github, count=1, flags=re.S).lstrip()
|
| 152 |
for stem, alt in (('mangkhut', 'Trackformer 1.2 Mangkhut pressure and route forecast'),):
|
| 153 |
old = '[](' + media_url(stem) + ')'
|
| 154 |
if card.count(old) != 1:
|
|
|
|
| 165 |
if not resolved.is_relative_to(ROOT) or not resolved.exists():
|
| 166 |
raise ValueError('Invalid or missing relative public link: ' + path)
|
| 167 |
revision = figure_revision if path in CURRENT_FIGURES else MEDIA_REVISION
|
| 168 |
+
kind = ('resolve/' + revision if image or path in CURRENT_FIGURES
|
| 169 |
+
else ('tree/main' if resolved.is_dir() else 'blob/main'))
|
| 170 |
target = HF + '/' + kind + '/' + path + ('#' + fragment if fragment else '')
|
| 171 |
return image + '[' + label + '](' + target + ')'
|
| 172 |
card = re.sub(r'(!?)\[([^\[\]\n]*)\]\(([^)\s]+)\)', link, card)
|
|
|
|
| 199 |
return actual
|
| 200 |
|
| 201 |
|
| 202 |
+
def git_hf_token():
|
| 203 |
+
"""Use the existing credential helper for this exact HF host, never log it."""
|
| 204 |
+
import os
|
| 205 |
+
result = subprocess.run(['git', 'credential', 'fill'], cwd=ROOT,
|
| 206 |
+
input='protocol=https\nhost=huggingface.co\n\n',
|
| 207 |
+
text=True, capture_output=True, env={**os.environ, 'GIT_TERMINAL_PROMPT': '0'})
|
| 208 |
+
fields = dict(line.split('=', 1) for line in result.stdout.splitlines() if '=' in line)
|
| 209 |
+
if result.returncode or not fields.get('password'):
|
| 210 |
+
raise ValueError('No existing Hugging Face git credential; authenticate before publishing')
|
| 211 |
+
return fields['password']
|
| 212 |
+
|
| 213 |
+
|
| 214 |
+
def remote_matches(entry, path):
|
| 215 |
+
if entry is None:
|
| 216 |
+
return False
|
| 217 |
+
lfs_sha = getattr(getattr(entry, 'lfs', None), 'sha256', None)
|
| 218 |
+
if lfs_sha:
|
| 219 |
+
return lfs_sha == sha(path)
|
| 220 |
+
data = path.read_bytes()
|
| 221 |
+
return entry.blob_id == hashlib.sha1(b'blob ' + str(len(data)).encode() + b'\0' + data).hexdigest()
|
| 222 |
+
|
| 223 |
+
|
| 224 |
+
def run(output, publish=False, docs_only=False, use_git_credential=False):
|
| 225 |
# Lazy imports keep portable CI document tests independent of HF packages.
|
| 226 |
from huggingface_hub import CommitOperationAdd, HfApi, hf_hub_download
|
| 227 |
output = output.resolve()
|
| 228 |
if not output.is_relative_to(Path('/Volumes/D')):
|
| 229 |
raise ValueError('All generated documentation/cache artifacts must stay on D')
|
| 230 |
output.mkdir(parents=True, exist_ok=True)
|
| 231 |
+
api = HfApi(token=git_hf_token() if use_git_credential else None)
|
| 232 |
parent = api.model_info(REPO).sha
|
| 233 |
weights_before = frozen_identity(api, parent)
|
| 234 |
+
original = Path(hf_hub_download(REPO, 'README.md', revision=parent, cache_dir=CACHE, token=api.token))
|
| 235 |
github = (ROOT / 'README.md').read_text()
|
| 236 |
card_path = output / 'README.md'
|
| 237 |
card_path.write_text(render_card(original.read_text(), github))
|
| 238 |
+
selected = DOC_SYNC_FILES if docs_only else SYNC_FILES
|
| 239 |
+
files = {'README.md': card_path, **{name: ROOT / name for name in selected}}
|
| 240 |
source_commit = subprocess.check_output(['git', 'rev-parse', 'HEAD'], cwd=ROOT, text=True).strip()
|
| 241 |
receipt = {
|
| 242 |
'status': 'prepared', 'repo_id': REPO, 'parent_commit': parent,
|
| 243 |
+
'sync_scope': 'documentation_only' if docs_only else 'full_existing_release_sync',
|
| 244 |
'github_source_commit': source_commit,
|
| 245 |
'github_readme_sha256': sha(ROOT / 'README.md'),
|
| 246 |
'original_hf_readme_sha256': sha(original),
|
|
|
|
| 251 |
'forecast_arrays_changed': False, 'videos_changed': False,
|
| 252 |
}
|
| 253 |
if publish:
|
| 254 |
+
dirty = subprocess.check_output(['git', 'status', '--porcelain', '--', 'README.md', *selected], cwd=ROOT, text=True)
|
| 255 |
if dirty:
|
| 256 |
raise ValueError('Commit canonical README and matching source files before publishing')
|
| 257 |
# Publish scientific assets first, then pin the new figures to that
|
| 258 |
# immutable commit in the card. Never pin new bytes to the old movie
|
| 259 |
# revision or let a missing future image render as a stale figure.
|
| 260 |
+
existing = {entry.path: entry for entry in api.get_paths_info(REPO, list(selected), revision=parent)}
|
| 261 |
+
changed = [name for name in selected if not remote_matches(existing.get(name), ROOT/name)]
|
| 262 |
+
figure_revision = parent
|
| 263 |
+
if changed:
|
| 264 |
+
assets = api.create_commit(repo_id=REPO, repo_type='model', parent_commit=parent,
|
| 265 |
+
commit_message='Align 1.2 research documentation and report figures',
|
| 266 |
+
operations=[CommitOperationAdd(path_in_repo=name, path_or_fileobj=str(ROOT/name)) for name in changed])
|
| 267 |
+
figure_revision = assets.oid
|
| 268 |
+
card_path.write_text(render_card(original.read_text(), github, figure_revision))
|
| 269 |
receipt['files']['README.md'] = sha(card_path)
|
| 270 |
+
receipt['figure_revision'] = figure_revision
|
| 271 |
+
receipt['uploaded_asset_paths'] = changed
|
| 272 |
+
committed = api.create_commit(repo_id=REPO, repo_type='model', parent_commit=figure_revision,
|
| 273 |
+
commit_message='Match released 1.2 model card to the Chinese research report',
|
| 274 |
operations=[CommitOperationAdd(path_in_repo='README.md', path_or_fileobj=str(card_path))])
|
| 275 |
for name, expected in receipt['files'].items():
|
| 276 |
+
saved = hf_hub_download(REPO, name, revision=committed.oid, cache_dir=CACHE, token=api.token)
|
| 277 |
if sha(saved) != expected:
|
| 278 |
raise ValueError('Published documentation/source differs: ' + name)
|
| 279 |
if frozen_identity(api, committed.oid) != weights_before:
|
|
|
|
| 288 |
parser = argparse.ArgumentParser(description=__doc__)
|
| 289 |
parser.add_argument('--output', type=Path, required=True)
|
| 290 |
parser.add_argument('--publish', action='store_true')
|
| 291 |
+
parser.add_argument('--docs-only', action='store_true', help='Upload only whitelisted Markdown and report assets')
|
| 292 |
+
parser.add_argument('--use-git-credential', action='store_true', help='Use the existing HF credential helper without saving a token')
|
| 293 |
args = parser.parse_args()
|
| 294 |
+
run(args.output, args.publish, args.docs_only, args.use_git_credential)
|
release_tools/test_public_model_cards.py
CHANGED
|
@@ -1,218 +1,146 @@
|
|
| 1 |
-
"""Portable
|
|
|
|
| 2 |
import json
|
|
|
|
| 3 |
import re
|
| 4 |
import unittest
|
| 5 |
-
|
|
|
|
|
|
|
|
|
|
| 6 |
|
| 7 |
|
| 8 |
class PublicModelCardsTest(unittest.TestCase):
|
| 9 |
def setUp(self):
|
| 10 |
-
self.source
|
| 11 |
-
self.
|
| 12 |
-
self.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 13 |
|
| 14 |
-
def
|
| 15 |
-
rendered
|
| 16 |
self.assertTrue(rendered.startswith(self.header))
|
| 17 |
-
|
| 18 |
-
|
| 19 |
-
|
| 20 |
-
|
| 21 |
-
|
| 22 |
-
self.assertNotIn('
|
| 23 |
-
|
| 24 |
-
|
| 25 |
-
|
| 26 |
-
|
| 27 |
-
self.
|
| 28 |
-
|
| 29 |
-
|
| 30 |
-
|
| 31 |
-
self.
|
| 32 |
-
|
| 33 |
-
|
| 34 |
-
|
| 35 |
-
self.
|
| 36 |
-
|
| 37 |
-
|
| 38 |
-
|
| 39 |
-
|
| 40 |
-
|
| 41 |
-
|
| 42 |
-
|
| 43 |
-
|
| 44 |
-
|
| 45 |
-
|
| 46 |
-
|
| 47 |
-
|
| 48 |
-
|
| 49 |
-
|
| 50 |
-
|
| 51 |
-
|
| 52 |
-
|
| 53 |
-
|
| 54 |
-
|
| 55 |
-
|
| 56 |
-
|
| 57 |
-
|
| 58 |
-
|
| 59 |
-
|
| 60 |
-
|
| 61 |
-
|
| 62 |
-
|
| 63 |
-
|
| 64 |
-
|
| 65 |
-
|
| 66 |
-
|
| 67 |
-
|
| 68 |
-
|
| 69 |
-
|
| 70 |
-
|
| 71 |
-
for
|
| 72 |
-
|
| 73 |
-
|
| 74 |
-
|
| 75 |
-
|
| 76 |
-
|
| 77 |
-
|
| 78 |
-
self.assertNotIn('paper/trackformer.tex', SYNC_FILES)
|
| 79 |
-
self.assertNotIn('paper/trackformer.pdf', SYNC_FILES)
|
| 80 |
-
for path in SYNC_FILES:
|
| 81 |
-
self.assertTrue((ROOT / path).is_file(), path)
|
| 82 |
-
|
| 83 |
-
def test_linked_current_docs_cannot_call_completed_run_pending(self):
|
| 84 |
-
for name in ('README.md', 'models/trackformer_1_2_field/README.md',
|
| 85 |
-
'evaluation/README.md', 'docs/deepmind_daily_benchmark.md',
|
| 86 |
-
'docs/daily_storm_benchmark.md', 'docs/trackformer_1_2_evaluation.md',
|
| 87 |
-
'RELEASE_NOTES_TRACKFORMER_1_2.md'):
|
| 88 |
-
with self.subTest(path=name):
|
| 89 |
-
source = (ROOT / name).read_text()
|
| 90 |
-
self.assertIn('Mini', source)
|
| 91 |
-
self.assertIn('1,473', source)
|
| 92 |
-
self.assertIn('270', source)
|
| 93 |
-
for stale in ('results remain pending', 'run is in progress',
|
| 94 |
-
'scores are **pending**', 'deepmind comparison has not been run'):
|
| 95 |
-
self.assertNotIn(stale, source.lower())
|
| 96 |
-
if name != 'README.md':
|
| 97 |
-
self.assertIn(name, SYNC_FILES)
|
| 98 |
-
|
| 99 |
-
def assert_metric_row(self, source, label, values, places):
|
| 100 |
-
rows = [line for line in source.splitlines() if line.startswith('| ')
|
| 101 |
-
and line.split('|')[1].strip().startswith(label)]
|
| 102 |
-
self.assertEqual(len(rows), 1, label)
|
| 103 |
-
cells = [cell.strip() for cell in rows[0].strip().strip('|').split('|')]
|
| 104 |
-
self.assertEqual(len(cells), 5, label)
|
| 105 |
-
for index, key in enumerate(('1.1', '1.2', 'deepmind'), 1):
|
| 106 |
-
match = re.search(r'\d+(?:\.\d+)?', cells[index].replace('**', '').replace(',', ''))
|
| 107 |
-
self.assertIsNotNone(match, (label, key))
|
| 108 |
-
self.assertEqual(match.group(), f'{values[key]:.{places}f}', (label, key))
|
| 109 |
-
|
| 110 |
-
def test_readme_and_release_tables_match_completed_metrics_not_estimates(self):
|
| 111 |
-
exported = json.loads((ROOT / 'evaluation/deepmind_daily/benchmark.json').read_text())
|
| 112 |
-
receipt = json.loads((ROOT / 'evaluation/deepmind_daily/verification.json').read_text())
|
| 113 |
-
metrics = {row['key']: row['values'] for row in exported['metrics']}
|
| 114 |
-
self.assertEqual(exported['status'], 'complete_verified')
|
| 115 |
-
self.assertEqual(receipt['verified_daily_cases'], 1473)
|
| 116 |
-
self.assertEqual(receipt['verified_storms'], 270)
|
| 117 |
-
specs = (
|
| 118 |
-
('Mean track error, +6 to +120 h', 'mean_track_error_km', 1),
|
| 119 |
-
('Six-hour track-direction error', 'direction_error_deg', 2),
|
| 120 |
-
('Central-pressure MAE · JMA', 'pressure_JMA_hpa', 2),
|
| 121 |
-
('Centred route-shape similarity', 'shape_similarity', 4),
|
| 122 |
-
('Pressure-curve similarity · JMA', 'pressure_JMA_curve_similarity', 4),
|
| 123 |
-
)
|
| 124 |
-
for source in (self.source, render_card(self.original, self.source)):
|
| 125 |
-
for label, key, places in specs:
|
| 126 |
-
with self.subTest(label=label):
|
| 127 |
-
self.assert_metric_row(source, label, metrics[key], places)
|
| 128 |
-
release = (ROOT / 'RELEASE_NOTES_TRACKFORMER_1_2.md').read_text()
|
| 129 |
-
for label, key, places in (
|
| 130 |
-
('Mean track error', 'mean_track_error_km', 1),
|
| 131 |
-
('Direction error', 'direction_error_deg', 2),
|
| 132 |
-
('Central-pressure MAE against JMA', 'pressure_JMA_hpa', 2),
|
| 133 |
-
):
|
| 134 |
-
self.assert_metric_row(release, label, metrics[key], places)
|
| 135 |
-
|
| 136 |
-
def test_recent_group_is_not_confused_with_combined_cohort(self):
|
| 137 |
-
report = json.loads((ROOT / 'evaluation/deepmind_daily/period_comparison.json').read_text())
|
| 138 |
-
post = report['periods']['after_2024']
|
| 139 |
-
self.assertEqual(post['daily_issues'], 61)
|
| 140 |
-
self.assertEqual(post['storms'], 18)
|
| 141 |
-
self.assertEqual(post['years'], [2025, 2026])
|
| 142 |
-
section = self.source.split('### After 2024', 1)[1]
|
| 143 |
-
for label, key, places in (('Track position MAE', 'mean_track_error_km', 1),
|
| 144 |
-
('Track-direction error', 'direction_error_deg', 2),
|
| 145 |
-
('Route-shape similarity', 'shape_similarity', 4)):
|
| 146 |
-
values = {model: post['scores']['route'][model][key]['value']
|
| 147 |
-
for model in ('1.1', '1.2', 'deepmind')}
|
| 148 |
-
self.assert_metric_row(section, label, values, places)
|
| 149 |
-
pressure = post['scores']['pressure']['JMA']['models']
|
| 150 |
-
for label, key, places in (('JMA central-pressure MAE', 'mae_hpa', 2),
|
| 151 |
-
('JMA pressure-curve similarity', 'curve_similarity', 4)):
|
| 152 |
-
self.assert_metric_row(section, label,
|
| 153 |
-
{m: pressure[m][key]['value'] for m in ('1.1', '1.2', 'deepmind')}, places)
|
| 154 |
-
|
| 155 |
-
def test_checkpoint_identity_and_proportions_precede_separate_charts(self):
|
| 156 |
-
for source in (self.source, render_card(self.original, self.source)):
|
| 157 |
-
self.assertLess(source.index('**DeepMind checkpoint:'), source.index('model_1_2_benchmark.png'))
|
| 158 |
-
self.assertEqual(source.count('model_1_2_benchmark.png'), 1)
|
| 159 |
-
self.assertEqual(source.count('model_1_2_before_2024_benchmark.png'), 1)
|
| 160 |
-
self.assertEqual(source.count('model_1_2_after_2024_benchmark.png'), 1)
|
| 161 |
-
for text in ('90.7%', '85.2%', '5.2%', '8.1%', '4.1%', '6.7%',
|
| 162 |
-
'Calendar 2024', '59 days / 18 storms / 1,174',
|
| 163 |
-
'not training-data composition', 'same Mini `<2024` checkpoint',
|
| 164 |
-
'100% Mini `<2024`; 0% other checkpoints'):
|
| 165 |
-
self.assertIn(text, source)
|
| 166 |
-
|
| 167 |
-
def test_historical_chart_and_missing_pressure_are_explicit(self):
|
| 168 |
-
report = json.loads((ROOT / 'evaluation/deepmind_daily/period_comparison.json').read_text())
|
| 169 |
-
before = report['periods']['before_2024']
|
| 170 |
-
for source in (self.source, render_card(self.original, self.source)):
|
| 171 |
-
section = source.split('### Before 2024', 1)[1].split('### After 2024', 1)[0]
|
| 172 |
-
for label, key, places in (('Historical track position MAE', 'mean_track_error_km', 1),
|
| 173 |
-
('Historical track-direction error', 'direction_error_deg', 2),
|
| 174 |
-
('Historical route-shape similarity', 'shape_similarity', 4)):
|
| 175 |
-
self.assert_metric_row(section, label,
|
| 176 |
-
{m: before['scores']['route'][m][key]['value'] for m in ('1.1', '1.2', 'deepmind')}, places)
|
| 177 |
-
for text in ('Pressure is not scored', '0 shared valid pressure starts', 'not zero error', '1980–1999'):
|
| 178 |
-
self.assertIn(text, section)
|
| 179 |
-
|
| 180 |
-
def test_training_years_sources_and_receipt_match_release(self):
|
| 181 |
-
manifest = json.loads((ROOT / 'models/trackformer_1_2_field/manifest.json').read_text())
|
| 182 |
-
audit = json.loads((ROOT / 'evaluation/training_data/trackformer_1_2_provenance.json').read_text())
|
| 183 |
-
self.assertEqual(audit['status'], 'verified')
|
| 184 |
-
for key in ('source_checkpoint_sha256', 'inference_weights_sha256'):
|
| 185 |
-
self.assertEqual(audit[key], manifest[key])
|
| 186 |
-
self.assertEqual(audit['dataset_manifest_sha256'], manifest['dataset_sha256'])
|
| 187 |
-
self.assertEqual(audit['normalization_fit_years'], [2000, 2021])
|
| 188 |
-
self.assertEqual(len(audit['verified_archive_files']), 6)
|
| 189 |
self.assertTrue(audit['whole_storm_splits'])
|
| 190 |
-
for
|
| 191 |
-
|
| 192 |
-
|
| 193 |
-
|
| 194 |
-
|
| 195 |
-
|
| 196 |
-
|
| 197 |
-
|
| 198 |
-
|
| 199 |
-
|
| 200 |
-
|
| 201 |
-
|
| 202 |
-
|
| 203 |
-
|
| 204 |
-
|
| 205 |
-
|
| 206 |
-
|
| 207 |
-
|
| 208 |
-
|
| 209 |
-
|
| 210 |
-
self.assertNotIn('results are not implied', index)
|
| 211 |
-
self.assertIn('evaluation/README.md', SYNC_FILES)
|
| 212 |
-
notes = (ROOT / 'docs/deepmind_daily_benchmark.md').read_text()
|
| 213 |
-
self.assertIn('retained byte-for-byte', notes)
|
| 214 |
-
self.assertIn('handoff before import, not the current benchmark status', notes)
|
| 215 |
-
|
| 216 |
-
|
| 217 |
-
if __name__ == '__main__':
|
| 218 |
-
unittest.main()
|
|
|
|
| 1 |
+
"""Portable integrity checks for the compact public documentation layout."""
|
| 2 |
+
import hashlib
|
| 3 |
import json
|
| 4 |
+
from pathlib import Path
|
| 5 |
import re
|
| 6 |
import unittest
|
| 7 |
+
import zipfile
|
| 8 |
+
|
| 9 |
+
from sync_public_model_cards import (ROOT, SYNC_FILES, DOC_SYNC_FILES, REPORT_FIGURES,
|
| 10 |
+
MEDIA_REVISION, media_url, render_card)
|
| 11 |
|
| 12 |
|
| 13 |
class PublicModelCardsTest(unittest.TestCase):
|
| 14 |
def setUp(self):
|
| 15 |
+
self.source=(ROOT/'README.md').read_text()
|
| 16 |
+
self.reference=(ROOT/'docs/REFERENCE.md').read_text()
|
| 17 |
+
self.header='---\nlicense: mit\ncustom_preserved: yes\n---\n'
|
| 18 |
+
self.index=json.loads((ROOT/'docs/archive/documentation-manifest.json').read_text())
|
| 19 |
+
|
| 20 |
+
def test_three_current_markdown_entry_points(self):
|
| 21 |
+
actual={str(p.relative_to(ROOT)) for p in ROOT.rglob('*.md') if '.git' not in p.parts}
|
| 22 |
+
self.assertEqual(actual,set(self.index['final_markdown_paths']))
|
| 23 |
+
self.assertLess(len(self.source.encode()),8000)
|
| 24 |
+
|
| 25 |
+
def test_all_removed_documents_survive_exactly_in_archive(self):
|
| 26 |
+
path=ROOT/self.index['archive_path']
|
| 27 |
+
self.assertEqual(hashlib.sha256(path.read_bytes()).hexdigest(),self.index['archive_sha256'])
|
| 28 |
+
with zipfile.ZipFile(path) as z:
|
| 29 |
+
for entry in self.index['original_docs']:
|
| 30 |
+
self.assertEqual(hashlib.sha256(z.read(entry['path'])).hexdigest(),entry['sha256'])
|
| 31 |
+
for name in self.index['removed_paths']:
|
| 32 |
+
self.assertFalse((ROOT/name).exists(),name)
|
| 33 |
+
|
| 34 |
+
def test_pdf_remains_canonical_and_exact(self):
|
| 35 |
+
source=json.loads((ROOT/'training/trackformer_1_2/SOURCE_MANIFEST.json').read_text())
|
| 36 |
+
pdf=source['documentation_pdf']
|
| 37 |
+
data=(ROOT/pdf['path']).read_bytes()
|
| 38 |
+
self.assertTrue(data.startswith(b'%PDF-'))
|
| 39 |
+
self.assertEqual(hashlib.sha256(data).hexdigest(),pdf['sha256'])
|
| 40 |
+
self.assertEqual(pdf['pages'],106)
|
| 41 |
+
self.assertIn(pdf['path'],self.source)
|
| 42 |
+
|
| 43 |
+
def test_training_source_ledger_and_archived_document_links(self):
|
| 44 |
+
m=json.loads((ROOT/'training/trackformer_1_2/SOURCE_MANIFEST.json').read_text())
|
| 45 |
+
for entry in m['files']:
|
| 46 |
+
self.assertEqual(hashlib.sha256((ROOT/entry['path']).read_bytes()).hexdigest(),entry['sha256'])
|
| 47 |
+
with zipfile.ZipFile(ROOT/self.index['archive_path']) as z:
|
| 48 |
+
for entry in m['archived_documentation']:
|
| 49 |
+
self.assertEqual(hashlib.sha256(z.read(entry['archive_member'])).hexdigest(),entry['sha256'])
|
| 50 |
+
|
| 51 |
+
def test_sync_does_not_recreate_retired_files(self):
|
| 52 |
+
retired=set(self.index['removed_paths'])
|
| 53 |
+
self.assertFalse(retired.intersection(SYNC_FILES))
|
| 54 |
+
self.assertFalse(retired.intersection(DOC_SYNC_FILES))
|
| 55 |
+
self.assertEqual(len(SYNC_FILES),len(set(SYNC_FILES)))
|
| 56 |
+
self.assertEqual(len(DOC_SYNC_FILES),len(set(DOC_SYNC_FILES)))
|
| 57 |
+
for path in SYNC_FILES:
|
| 58 |
+
self.assertTrue((ROOT/path).is_file(),path)
|
| 59 |
+
self.assertNotIn('models/trackformer_1_2_field/weights.pt',SYNC_FILES)
|
| 60 |
+
self.assertNotIn('paper/trackformer.pdf',SYNC_FILES)
|
| 61 |
|
| 62 |
+
def test_hf_keeps_one_metadata_header_and_one_native_player(self):
|
| 63 |
+
rendered=render_card(self.header+'\nOld card',self.source)
|
| 64 |
self.assertTrue(rendered.startswith(self.header))
|
| 65 |
+
self.assertEqual(rendered.count('license: mit'),1)
|
| 66 |
+
self.assertEqual(rendered.count('<video '),1)
|
| 67 |
+
self.assertEqual(rendered.count('src="'+media_url('mangkhut')+'"'),1)
|
| 68 |
+
self.assertIn(MEDIA_REVISION,rendered)
|
| 69 |
+
self.assertNotIn('Old card',rendered)
|
| 70 |
+
self.assertNotIn('/resolve/main/docs/trackformer_1_2_mangkhut.mp4',rendered)
|
| 71 |
+
for url in re.findall(r'\]\(([^)\s]+)\)',rendered):
|
| 72 |
+
self.assertTrue(url.startswith(('https:','http:','#','mailto:')),url)
|
| 73 |
+
|
| 74 |
+
def test_bad_metadata_or_missing_featured_section_fail_closed(self):
|
| 75 |
+
with self.assertRaisesRegex(ValueError,'YAML metadata'):
|
| 76 |
+
render_card('No metadata',self.source)
|
| 77 |
+
with self.assertRaisesRegex(ValueError,'featured-film'):
|
| 78 |
+
render_card(self.header,self.source.replace('## See the forecast:','## Removed:'))
|
| 79 |
+
with self.assertRaisesRegex(ValueError,'relative public link'):
|
| 80 |
+
render_card(self.header,self.source+'\n[Private](../../private.txt)\n')
|
| 81 |
+
|
| 82 |
+
def test_current_relative_links_and_fragments_resolve(self):
|
| 83 |
+
for name in self.index['final_markdown_paths']:
|
| 84 |
+
path=ROOT/name
|
| 85 |
+
for url in re.findall(r'\]\(([^)\s]+)\)',path.read_text()):
|
| 86 |
+
if url.startswith(('https:','http:','mailto:')):continue
|
| 87 |
+
target,_,fragment=url.partition('#')
|
| 88 |
+
dest=(path.parent/target).resolve() if target else path
|
| 89 |
+
self.assertTrue(dest.is_relative_to(ROOT) and dest.exists(),(name,url))
|
| 90 |
+
if fragment and dest.suffix=='.md':
|
| 91 |
+
text=dest.read_text()
|
| 92 |
+
anchors=set(re.findall(r'<a id="([^"]+)"',text))
|
| 93 |
+
for heading in re.findall(r'^#+\s+(.+)$',text,re.M):
|
| 94 |
+
anchors.add(re.sub(r'[^\w\- ]','',heading.lower()).replace(' ','-'))
|
| 95 |
+
self.assertIn(fragment,anchors,(name,url))
|
| 96 |
+
|
| 97 |
+
def test_summary_numbers_match_actual_common_support(self):
|
| 98 |
+
exported=json.loads((ROOT/'evaluation/deepmind_daily/benchmark.json').read_text())
|
| 99 |
+
receipt=json.loads((ROOT/'evaluation/deepmind_daily/verification.json').read_text())
|
| 100 |
+
self.assertEqual(exported['status'],'complete_verified')
|
| 101 |
+
self.assertEqual((receipt['verified_daily_cases'],receipt['verified_storms']),(1473,270))
|
| 102 |
+
metrics={row['key']:row['values'] for row in exported['metrics']}
|
| 103 |
+
for label,key in [('Mean track error','mean_track_error_km'),('Six-hour track-direction','direction_error_deg'),('Central-pressure MAE','pressure_JMA_hpa')]:
|
| 104 |
+
row=next(x for x in self.source.splitlines() if x.startswith('| '+label))
|
| 105 |
+
for model in ('1.2','deepmind'):self.assertIn(f'{metrics[key][model]:.2f}',row)
|
| 106 |
+
pressure=next(x for x in exported['metrics'] if x['key']=='pressure_JMA_hpa')
|
| 107 |
+
self.assertEqual(pressure['common_lead_points'],2637)
|
| 108 |
+
for coverage in pressure['coverage'].values():
|
| 109 |
+
self.assertEqual((coverage['storms'],coverage['daily_cases']),(40,134))
|
| 110 |
+
recent=json.loads((ROOT/'evaluation/deepmind_daily/period_comparison.json').read_text())['periods']['after_2024']
|
| 111 |
+
self.assertEqual((recent['daily_issues'],recent['storms']),(61,18))
|
| 112 |
+
for model in ('1.2','deepmind'):
|
| 113 |
+
self.assertIn(f"{recent['scores']['route'][model]['mean_track_error_km']['value']:.1f} km",self.source)
|
| 114 |
+
|
| 115 |
+
def test_scientific_identity_and_split_provenance_unchanged(self):
|
| 116 |
+
m=json.loads((ROOT/'models/trackformer_1_2_field/manifest.json').read_text())
|
| 117 |
+
audit=json.loads((ROOT/'evaluation/training_data/trackformer_1_2_provenance.json').read_text())
|
| 118 |
+
self.assertEqual(audit['dataset_manifest_sha256'],m['dataset_sha256'])
|
| 119 |
+
for key in ['source_checkpoint_sha256','inference_weights_sha256']:
|
| 120 |
+
self.assertEqual(audit[key],m[key])
|
| 121 |
+
for name,expected in m['source_module_sha256'].items():
|
| 122 |
+
self.assertEqual(hashlib.sha256((ROOT/'models/trackformer_1_2_field'/name).read_bytes()).hexdigest(),expected)
|
| 123 |
+
for name,count,native in [('train',13949,800),('validation',1041,100),('test',1195,100)]:
|
| 124 |
+
p=audit['partitions'][name]
|
| 125 |
+
self.assertEqual((p['windows'],p['native_pressure_windows']),(count,native))
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 126 |
self.assertTrue(audit['whole_storm_splits'])
|
| 127 |
+
for phrase in ['no rainfall head','humidity input','not an operational warning service','batch 16','batch 10','4,384']:
|
| 128 |
+
self.assertIn(phrase,self.source)
|
| 129 |
+
|
| 130 |
+
def test_figure_provenance_and_all_report_bytes_stay_unchanged(self):
|
| 131 |
+
folder=ROOT/'docs/report_figures'
|
| 132 |
+
receipt=json.loads((folder/'provenance.json').read_text())
|
| 133 |
+
for key in ['model_weights_changed','model_or_truth_arrays_changed','coordinates_or_times_shifted','model_radius_head_available']:
|
| 134 |
+
self.assertFalse(receipt[key])
|
| 135 |
+
self.assertEqual(receipt['forecast_members_per_case'],50)
|
| 136 |
+
self.assertEqual(len(REPORT_FIGURES),10)
|
| 137 |
+
for name,expected in receipt['files_sha256'].items():
|
| 138 |
+
self.assertEqual(hashlib.sha256((folder/name).read_bytes()).hexdigest(),expected)
|
| 139 |
+
days=json.loads((folder/'day_1_to_5.json').read_text())
|
| 140 |
+
self.assertEqual(days['lead_hours'],[24,48,72,96,120])
|
| 141 |
+
self.assertEqual((days['route_storms'],days['route_issues']),(270,1473))
|
| 142 |
+
row=next(x for x in self.source.splitlines() if x.startswith('| Day 5'))
|
| 143 |
+
for model in ['1.2','deepmind']:self.assertIn(f"{days['track_error_km'][model][-1]:.2f}",row)
|
| 144 |
+
|
| 145 |
+
|
| 146 |
+
if __name__=='__main__':unittest.main()
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
training/trackformer_1_2/PUBLICATION.md
DELETED
|
@@ -1,19 +0,0 @@
|
|
| 1 |
-
# Immutable publication and execution checks
|
| 2 |
-
|
| 3 |
-
- Source publication on GitHub: https://github.com/yu314-coder/typhoon-predict/commit/ac05e1974ac90ad7819bf0483da8e8637d880525
|
| 4 |
-
- Source and all nine assets on Hugging Face: https://huggingface.co/euler314/typhoon-predict/commit/2d1464e9a310e6a6e8840ef1c23017fdededd57b
|
| 5 |
-
- All asset hashes remain fixed in `assets.json`.
|
| 6 |
-
|
| 7 |
-
Pin the binary download to the completed publication:
|
| 8 |
-
|
| 9 |
-
```bash
|
| 10 |
-
python training/trackformer_1_2/download_assets.py --revision 2d1464e9a310e6a6e8840ef1c23017fdededd57b
|
| 11 |
-
```
|
| 12 |
-
|
| 13 |
-
The original trainer's CPU preflight passed, including all six input-file hashes, exact split counts, normalization and original warm-start loading (21,452,595 parameters). The portable CPU check passed on one coarse and one native TRAIN example: finite forward/loss/backward, nonzero attention gradient, pressure exactly read from the predicted field, rejection of an added future input, and full TRAIN sampler coverage. Zero optimizer updates were performed. See `EXECUTION_CHECKS.json`. This is an execution check, not a new end-to-end training reproduction or an accuracy claim.
|
| 14 |
-
|
| 15 |
-
```bash
|
| 16 |
-
python training/trackformer_1_2/test_training_cpu.py
|
| 17 |
-
```
|
| 18 |
-
|
| 19 |
-
This test also reconstructs the loader's local basin mmap; use a dedicated writable data copy, never another trainer's data directory. Original GPU-specific tests are archived without rewriting their historical relative-path assumptions in `source/v173/test_model.py`; they were not rerun on GPU for this publication.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
training/trackformer_1_2/README.md
DELETED
|
@@ -1,9 +0,0 @@
|
|
| 1 |
-
# TrackFormer 1.2 training
|
| 2 |
-
|
| 3 |
-
**[Read / download the complete training manual (106-page PDF)](../../docs/TrackFormer_1.2_Training_and_Reproducibility.pdf).**
|
| 4 |
-
|
| 5 |
-
The single PDF contains the entire training method and data process, active losses, settings, checkpoint history, reproduction commands, limitations and all 40 text-source/provenance files. Its embedded ZIP preserves their exact bytes. You do not need to open separate Markdown documents to read the method. The original source files remain executable and inspectable here.
|
| 6 |
-
|
| 7 |
-
Large assets are public at the same [Hugging Face training path](https://huggingface.co/euler314/typhoon-predict/tree/main/training/trackformer_1_2); `assets.json` and `download_assets.py` verify their hashes without tokens or Drive permissions. Code and weights use MIT; upstream data attribution remains in `THIRD_PARTY_NOTICES.md` and the PDF.
|
| 8 |
-
|
| 9 |
-
Important: the archived notebook/config recipe uses batch 10, while the selected full checkpoint records 4,384 updates, compatible with batch 16 for the published four-epoch sampler. The PDF documents the evidence and provides both commands without claiming recovery of the original argv. See `records/selected-checkpoint-training.json` for the machine-readable inspection. The report selects epoch 4; the saved validation-loss minimum is epoch 2.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
training/trackformer_1_2/SOURCE_MANIFEST.json
CHANGED
|
@@ -13,21 +13,6 @@
|
|
| 13 |
"sha256": "090a97ebbccef9de77cb3898c473f5c9ada7e140777957f1195fc6ab89b9a313",
|
| 14 |
"bytes": 1083
|
| 15 |
},
|
| 16 |
-
{
|
| 17 |
-
"path": "training/trackformer_1_2/PUBLICATION.md",
|
| 18 |
-
"sha256": "1bcaa51a1999422925db2a100927985b927b454a2ff4250e318f7049ceec81c8",
|
| 19 |
-
"bytes": 1505
|
| 20 |
-
},
|
| 21 |
-
{
|
| 22 |
-
"path": "training/trackformer_1_2/README.md",
|
| 23 |
-
"sha256": "97ac00bff15891218956a4743ae0d12cddad56106683dbc3c14b4922a463023c",
|
| 24 |
-
"bytes": 1328
|
| 25 |
-
},
|
| 26 |
-
{
|
| 27 |
-
"path": "training/trackformer_1_2/THIRD_PARTY_NOTICES.md",
|
| 28 |
-
"sha256": "f764dd203eb124fa6192edd3b7b49a7a546cd5d69d3a8a250589a14ce43e2115",
|
| 29 |
-
"bytes": 1697
|
| 30 |
-
},
|
| 31 |
{
|
| 32 |
"path": "training/trackformer_1_2/assets.json",
|
| 33 |
"sha256": "a59148e9ba2617599614784b47789c3654df1a7809ee039e78bcd00fe982b211",
|
|
@@ -58,11 +43,6 @@
|
|
| 58 |
"sha256": "54918b8f29330299e18b8924e9cd93d05807e7f8cd99b2efd4805e9be94003c1",
|
| 59 |
"bytes": 55
|
| 60 |
},
|
| 61 |
-
{
|
| 62 |
-
"path": "training/trackformer_1_2/source/data_preparation/v164_dataset_contract.md",
|
| 63 |
-
"sha256": "cfd838651e632aba0d2439ba202a59895bd4df22137185974df1d97f0e2525d4",
|
| 64 |
-
"bytes": 4664
|
| 65 |
-
},
|
| 66 |
{
|
| 67 |
"path": "training/trackformer_1_2/source/data_preparation/v164_download_pressure_geography.py",
|
| 68 |
"sha256": "b8602cd2fba131a287dc9e25781c8f037d3325fec8a23203576916c2f7d2189c",
|
|
@@ -88,16 +68,6 @@
|
|
| 88 |
"sha256": "2da0078d4a74faa93f8cf874e42707b01f286406f0ceaa4db1814aab0eceb7d1",
|
| 89 |
"bytes": 18414
|
| 90 |
},
|
| 91 |
-
{
|
| 92 |
-
"path": "training/trackformer_1_2/source/data_preparation/v164_structure_research.md",
|
| 93 |
-
"sha256": "a4b40e50b7d0d1433a162438f14ed110541702c4cec2a386959857293ea78141",
|
| 94 |
-
"bytes": 17119
|
| 95 |
-
},
|
| 96 |
-
{
|
| 97 |
-
"path": "training/trackformer_1_2/source/v173/DESIGN.md",
|
| 98 |
-
"sha256": "163ae39ff572f23c519d65e27ee6a8db23b47df9461670709229ccf95130049a",
|
| 99 |
-
"bytes": 3278
|
| 100 |
-
},
|
| 101 |
{
|
| 102 |
"path": "training/trackformer_1_2/source/v173/baseline_model.py",
|
| 103 |
"sha256": "6c05941fd93baaca04508877edc69f8f0394b70e0a64e734ef0d2c1e8832246a",
|
|
@@ -200,5 +170,50 @@
|
|
| 200 |
"sha256": "2ff4dfe29509fa62e5cffd91e4046d5ca2d436e606cd8ea0c14f15df33adf7b1",
|
| 201 |
"pages": 106,
|
| 202 |
"source_snapshot_huggingface_revision": "1f4ed985a05589076d37f170cc8053e26cec9fba"
|
| 203 |
-
}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 204 |
}
|
|
|
|
| 13 |
"sha256": "090a97ebbccef9de77cb3898c473f5c9ada7e140777957f1195fc6ab89b9a313",
|
| 14 |
"bytes": 1083
|
| 15 |
},
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 16 |
{
|
| 17 |
"path": "training/trackformer_1_2/assets.json",
|
| 18 |
"sha256": "a59148e9ba2617599614784b47789c3654df1a7809ee039e78bcd00fe982b211",
|
|
|
|
| 43 |
"sha256": "54918b8f29330299e18b8924e9cd93d05807e7f8cd99b2efd4805e9be94003c1",
|
| 44 |
"bytes": 55
|
| 45 |
},
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 46 |
{
|
| 47 |
"path": "training/trackformer_1_2/source/data_preparation/v164_download_pressure_geography.py",
|
| 48 |
"sha256": "b8602cd2fba131a287dc9e25781c8f037d3325fec8a23203576916c2f7d2189c",
|
|
|
|
| 68 |
"sha256": "2da0078d4a74faa93f8cf874e42707b01f286406f0ceaa4db1814aab0eceb7d1",
|
| 69 |
"bytes": 18414
|
| 70 |
},
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 71 |
{
|
| 72 |
"path": "training/trackformer_1_2/source/v173/baseline_model.py",
|
| 73 |
"sha256": "6c05941fd93baaca04508877edc69f8f0394b70e0a64e734ef0d2c1e8832246a",
|
|
|
|
| 170 |
"sha256": "2ff4dfe29509fa62e5cffd91e4046d5ca2d436e606cd8ea0c14f15df33adf7b1",
|
| 171 |
"pages": 106,
|
| 172 |
"source_snapshot_huggingface_revision": "1f4ed985a05589076d37f170cc8053e26cec9fba"
|
| 173 |
+
},
|
| 174 |
+
"archived_documentation": [
|
| 175 |
+
{
|
| 176 |
+
"path": "training/trackformer_1_2/PUBLICATION.md",
|
| 177 |
+
"sha256": "1bcaa51a1999422925db2a100927985b927b454a2ff4250e318f7049ceec81c8",
|
| 178 |
+
"bytes": 1505,
|
| 179 |
+
"archive_path": "docs/archive/documentation-before-consolidation-20261030.zip",
|
| 180 |
+
"archive_member": "training/trackformer_1_2/PUBLICATION.md"
|
| 181 |
+
},
|
| 182 |
+
{
|
| 183 |
+
"path": "training/trackformer_1_2/README.md",
|
| 184 |
+
"sha256": "97ac00bff15891218956a4743ae0d12cddad56106683dbc3c14b4922a463023c",
|
| 185 |
+
"bytes": 1328,
|
| 186 |
+
"archive_path": "docs/archive/documentation-before-consolidation-20261030.zip",
|
| 187 |
+
"archive_member": "training/trackformer_1_2/README.md"
|
| 188 |
+
},
|
| 189 |
+
{
|
| 190 |
+
"path": "training/trackformer_1_2/THIRD_PARTY_NOTICES.md",
|
| 191 |
+
"sha256": "f764dd203eb124fa6192edd3b7b49a7a546cd5d69d3a8a250589a14ce43e2115",
|
| 192 |
+
"bytes": 1697,
|
| 193 |
+
"archive_path": "docs/archive/documentation-before-consolidation-20261030.zip",
|
| 194 |
+
"archive_member": "training/trackformer_1_2/THIRD_PARTY_NOTICES.md"
|
| 195 |
+
},
|
| 196 |
+
{
|
| 197 |
+
"path": "training/trackformer_1_2/source/data_preparation/v164_dataset_contract.md",
|
| 198 |
+
"sha256": "cfd838651e632aba0d2439ba202a59895bd4df22137185974df1d97f0e2525d4",
|
| 199 |
+
"bytes": 4664,
|
| 200 |
+
"archive_path": "docs/archive/documentation-before-consolidation-20261030.zip",
|
| 201 |
+
"archive_member": "training/trackformer_1_2/source/data_preparation/v164_dataset_contract.md"
|
| 202 |
+
},
|
| 203 |
+
{
|
| 204 |
+
"path": "training/trackformer_1_2/source/data_preparation/v164_structure_research.md",
|
| 205 |
+
"sha256": "a4b40e50b7d0d1433a162438f14ed110541702c4cec2a386959857293ea78141",
|
| 206 |
+
"bytes": 17119,
|
| 207 |
+
"archive_path": "docs/archive/documentation-before-consolidation-20261030.zip",
|
| 208 |
+
"archive_member": "training/trackformer_1_2/source/data_preparation/v164_structure_research.md"
|
| 209 |
+
},
|
| 210 |
+
{
|
| 211 |
+
"path": "training/trackformer_1_2/source/v173/DESIGN.md",
|
| 212 |
+
"sha256": "163ae39ff572f23c519d65e27ee6a8db23b47df9461670709229ccf95130049a",
|
| 213 |
+
"bytes": 3278,
|
| 214 |
+
"archive_path": "docs/archive/documentation-before-consolidation-20261030.zip",
|
| 215 |
+
"archive_member": "training/trackformer_1_2/source/v173/DESIGN.md"
|
| 216 |
+
}
|
| 217 |
+
],
|
| 218 |
+
"documentation_index": "docs/archive/documentation-manifest.json"
|
| 219 |
}
|
training/trackformer_1_2/THIRD_PARTY_NOTICES.md
DELETED
|
@@ -1,9 +0,0 @@
|
|
| 1 |
-
# Data sources and attribution
|
| 2 |
-
|
| 3 |
-
The MIT license covers project code and project model artifacts; it does not replace upstream data terms.
|
| 4 |
-
|
| 5 |
-
- **NOAA PSL NCEP/NCAR Reanalysis 1**: basin mean sea-level pressure, 500 hPa height and winds at 850/500/200 hPa. Data provided by NOAA PSL, Boulder, Colorado, USA. https://psl.noaa.gov/data/gridded/data.ncep.reanalysis.html
|
| 6 |
-
- **NOAA/NCEI IBTrACS**: derived storm observations, labels and masks; credit the original contributing meteorological agencies and NOAA/NCEI. Knapp et al. (2010), doi:10.1175/2009BAMS2755.1. https://www.ncei.noaa.gov/products/international-best-track-archive . Exact derived archive SHA is in assets.json; its original raw snapshot ID was not recorded.
|
| 7 |
-
- **Copernicus Climate Change Service (C3S), ERA5, accessed through Google's ARCO-ERA5 mirror**: native mean-sea-level-pressure patches and static land fraction/elevation. Contains modified Copernicus Climate Change Service information (2026). ERA5 hourly single-level data, doi:10.24381/cds.adbb2d47, https://cds.climate.copernicus.eu/datasets/reanalysis-era5-single-levels . The current dataset lists a CC-BY licence; attribution and upstream terms apply. ECMWF or the European Commission does not endorse this model. ARCO mirror and recipes: https://github.com/google-research/arco-era5 . The mirror's code licence is separate from the underlying meteorological data licence.
|
| 8 |
-
|
| 9 |
-
The published arrays are transformed subsets: quantized/coarse basin grids, float16 pressure, issue-centred patches, normalized derived track features and masks. These transformations do not create new native observational resolution. No claim of endorsement by NOAA, ECMWF, C3S or Google is made.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
training/trackformer_1_2/source/data_preparation/v164_dataset_contract.md
DELETED
|
@@ -1,41 +0,0 @@
|
|
| 1 |
-
# v1.2.64 Combined-Archive Training
|
| 2 |
-
|
| 3 |
-
## Active Workflow
|
| 4 |
-
|
| 5 |
-
The same `ai_train.ipynb` has two cells: version information, then fetching and preflight. It uses Google Drive only, installs no packages, and does not start training. The trainer remains `v164_global_regional_train.py` for command compatibility. Its active architecture is `wp-coarse8-native-pressure1-v164`, with 53,435,692 parameters. The old 31-channel global experiment and six-window pilot are not used.
|
| 6 |
-
|
| 7 |
-
Run in the Colab terminal after setup reports READY:
|
| 8 |
-
|
| 9 |
-
```sh
|
| 10 |
-
cd /content/v164_colab_bundle
|
| 11 |
-
python v164_global_regional_train.py 20 3 1
|
| 12 |
-
```
|
| 13 |
-
|
| 14 |
-
Numbers mean effective batch, workers, and CUDA AMP. Physical microbatch is one; workers are capped at two. Twelve epochs use rollout horizons 2, 4, 8, 12, then 20 steps. First two epochs train fields; later epochs add route and intensity objectives. Best selection starts at epoch five with the stable full-horizon objective. Validation is never training data; test is excluded from selection.
|
| 15 |
-
|
| 16 |
-
## Data Coverage
|
| 17 |
-
|
| 18 |
-
- Old archives supply 13,949 train, 1,041 validation, and 1,195 test windows with complete six-hour context/target weather coverage inside the Western Pacific domain.
|
| 19 |
-
- Train years 2000-2021; validation 2022-2023; test 2024-2025. Whole storms and window boundaries cannot cross splits. Tip 1979 is excluded from all three.
|
| 20 |
-
- Eight 2.5-degree context variables: mean sea-level pressure, 500-hPa height, and u/v winds at 850/500/200 hPa. Domain: 0-60 N, 100-180 E. These are old NCEP reanalysis fields, not high-resolution ERA5.
|
| 21 |
-
- New native 0.25-degree ERA5 pressure patches provide 800/100/100 detailed windows within those splits. Each patch is 121x121 over 30 degrees, fixed at its observed issue-time center, with nine history and twenty target frames. These 1,000 windows are additional fine supervision, NOT the entire old corpus.
|
| 22 |
-
- Static geographic channels: latitude, longitude, land fraction and elevation. Fine pressure is never synthesized by enlarging the old coarse map.
|
| 23 |
-
- SST, ONI and fine upper-air variables are not consumed by this configuration; they are not silently substituted or claimed present.
|
| 24 |
-
|
| 25 |
-
## Causality And Outputs
|
| 26 |
-
|
| 27 |
-
Inputs are analyses at -48,-42,...,0 hours, static geography, observed issue-time center/intensity and past six-hour motion. Future analysis fields and best-track positions/intensities are labels only. Issue-time retrospective best-track estimates are a historical hindcast assumption, not proof of real-time operational availability. Normalization uses only the training years. No official forecasts or future weather are model inputs.
|
| 28 |
-
|
| 29 |
-
The model autoregressively predicts eight basin fields and a native pressure patch when native history is available. No future recentering or route-painted vortex is used. Its track head uses generated basin pressure/winds/features. Fine data outside the parent domain receives an explicit missing-coverage mask, not extrapolated fields. Central pressure is a sampled generated field plus a separately learned subgrid correction; maximum wind is a scalar head, not a predicted 10m-wind map. No exact physical conservation is claimed.
|
| 30 |
-
|
| 31 |
-
The archive-wide training processes every eligible coarse window. Only the detailed subset trains the fine branch. A case lacking native pressure history can run the coarse branch but is not represented as a genuine high-resolution pressure forecast.
|
| 32 |
-
|
| 33 |
-
## Storage And Resume
|
| 34 |
-
|
| 35 |
-
Drive folder: `typhoon_predict/data/v164_reuse` contains the manifest and bundle descriptor referencing existing archived data plus `data/v164_pressure_geography` parts. No large archives are duplicated in Drive. Total payload about 2.13 GB. Four independent Drive HTTP clients fetch parts concurrently. Existing verified files are reused; the manifest is published after complete-file checksums pass.
|
| 36 |
-
|
| 37 |
-
Checkpoints are written directly to `MyDrive/typhoon_predict/training/v164_reuse_pressure/last.pt` and `best.pt`, with history and run manifest. Resume restores the last completed epoch, optimizer, scaler, and RNG; an interrupted epoch restarts. Architecture and dataset hashes must match. A stale lock after runtime death requires checking that no other training process is active before removing it. Pilot checkpoints are incompatible and are not auto-loaded.
|
| 38 |
-
|
| 39 |
-
## Verification Limits
|
| 40 |
-
|
| 41 |
-
Mac checks cover all archive hashes and split counts, real coarse and fine samples, tiny-model +120h finite rollouts, backward gradients, completed-epoch checkpoint resume, and full-width two-step MPS gradients. These are software checks, not model-skill validation. CUDA/T4 execution and a complete training run have not been tested; Colab was not accessed.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
training/trackformer_1_2/source/data_preparation/v164_structure_research.md
DELETED
|
@@ -1,142 +0,0 @@
|
|
| 1 |
-
# TrackFormer v1.2.64: field-first atmosphere and cyclone redesign
|
| 2 |
-
|
| 3 |
-
Date: 2026-09-13. Status: investigated proposal, NOT a trained or validated v164 model.
|
| 4 |
-
No Colab interaction and no training were performed. Diagnostics ran on this Mac's MPS GPU.
|
| 5 |
-
|
| 6 |
-
## Decision
|
| 7 |
-
|
| 8 |
-
Change the state representation, atmospheric training coverage, and field/track contract before scaling the current architecture. Do not merely widen v163, remove its tracker, or add more geometric route losses.
|
| 9 |
-
|
| 10 |
-
The recommended v164 prototype is a global-to-regional, multi-variable atmospheric predictor with a storm-object decoder tied to its generated fields. It must genuinely forecast pressure, not move or paint a pressure template onto a separately predicted route. A stochastic residual extension should follow a verified deterministic baseline, not obscure an input bug.
|
| 11 |
-
|
| 12 |
-
## What the Tip investigation actually established
|
| 13 |
-
|
| 14 |
-
Case: 1979-10-12 00 UTC, nine analyses from t-48 through t0, 20 six-hour forecast steps. Checkpoint metadata says epoch 7. The model has **27,179,604 parameters**, despite the `20m` configuration/file label.
|
| 15 |
-
|
| 16 |
-
Evidence files:
|
| 17 |
-
|
| 18 |
-
- `diagnostics/v164_tip_audit/report.json`: actual baseline and two controlled inference ablations.
|
| 19 |
-
- `diagnostics/v164_tip_audit/baseline.npz`: unmodified checkpoint tensor outputs.
|
| 20 |
-
- `diagnostics/v164_tip_audit/grib_verification.json`: issue-time channel checks against original NOAA CFSR GRIB.
|
| 21 |
-
- `audit_tip_v163_for_v164.py`: repeatable local diagnostic, with labels excluded from forward inputs.
|
| 22 |
-
|
| 23 |
-
### Confirmed data/display problems
|
| 24 |
-
|
| 25 |
-
1. The demo plotted six-hour truth movements as if they were cumulative displacements. All 20 corrected coordinates now match the archived time-tagged realized route within 0.0001 degrees. The true +120 location is 21.5 N, 128.1 E, not approximately 17 N, 138 E. This was a renderer defect, not proof that the training labels were wrong: training already accumulates the step targets.
|
| 26 |
-
2. The forecast renderer used predicted latitude to convert eastward displacement, whereas model field sampling uses an issue-latitude tangent plane. The renderer now uses the model's transform. The training step-coordinate versus fixed-grid geometry remains an architectural consistency issue to remove in v164 using one tested spherical coordinate API.
|
| 27 |
-
3. The displayed v1.1 and v1.2.26 legacy routes match exactly at every lead. The v1.2.26 renderer imports a legacy v62 artifact, not a freshly identified v1.2.26 checkpoint. They are not two independently verified model replays. A provenance warning was added; this audit does not certify their independent skill or modify the global benchmark.
|
| 28 |
-
4. Tip has no SST patch in this demo, so it receives zeros. The training loader filters on `sst_ok`; it does not train this exact missing-data condition. This is a confirmed train/inference contract gap. Its causal contribution requires a matched-source SST replay, not an invented warm ocean patch.
|
| 29 |
-
|
| 30 |
-
### Checks that passed
|
| 31 |
-
|
| 32 |
-
- All forward inputs are finite. Weather ranges are plausible: pressure about 972-1031 hPa over history; height about 5116-5922 gpm; winds in m/s. There is no demonstrated Pa/hPa or height/geopotential factor error.
|
| 33 |
-
- The nine weather timestamps stop exactly at the issue time.
|
| 34 |
-
- All eight issue-time map channels were checked against the original GRIB analysis. Pressure differs by at most 0.000059 hPa after float conversion; height and wind differences are zero on the sampled native grid. GRIB time is the requested issue and lead is zero. Older snapshots were timestamp-checked, not all independently re-decoded here.
|
| 35 |
-
- The issue-pressure mask is zero. The grid-pressure fallback therefore follows the training behavior. Missing pressure is not evidence of an incorrectly supplied 870-hPa observation.
|
| 36 |
-
- ONI is intentionally masked in both training and this replay until release-dated vintages exist. An empty ONI dictionary is not an accidental missing feature for this checkpoint.
|
| 37 |
-
|
| 38 |
-
### Measured model failure
|
| 39 |
-
|
| 40 |
-
These are forecast-structure diagnostics, not future-field accuracy scores. Core minimum means the lowest grid value within 500 km of the predicted center; the ring is 500-1000 km. That region contains only about 9-13 native grid points in the core.
|
| 41 |
-
|
| 42 |
-
| Lead | Whole-grid minimum, hPa | Core minimum, hPa | Ring mean minus core minimum, hPa |
|
| 43 |
-
| --- | ---: | ---: | ---: |
|
| 44 |
-
| 0 | 980.425 | 980.425 | 24.993 |
|
| 45 |
-
| 24 | 983.234 | 983.234 | 20.906 |
|
| 46 |
-
| 48 | 987.474 | 987.474 | 16.002 |
|
| 47 |
-
| 72 | 992.260 | 994.568 | 9.411 |
|
| 48 |
-
| 120 | 999.023 | 1008.427 | 2.562 |
|
| 49 |
-
|
| 50 |
-
The generated route ends near 34.30 N, 139.66 E. The scalar central-pressure head says 992.23 hPa, but the field near that route has no comparably deep resolved low. Some difference between a subgrid scalar and coarse reanalysis is legitimate; the combination of lost core contrast, weakening gradients, and diverging route/field location is the issue.
|
| 51 |
-
|
| 52 |
-
After correcting the displayed truth and projection, descriptive Tip mean geodesic route error is **677.7 km**, with mean heading error **54.6 degrees**. The stored DeepMind route has 140.1 km mean error on those leads, but uses a different analysis pipeline and this historical case is not established as held out for its pretrained weights. Do not call that a controlled generalization benchmark.
|
| 53 |
-
|
| 54 |
-
Controlled MPS ablations (same checkpoint/inputs, local-distance approximation for this diagnostic):
|
| 55 |
-
|
| 56 |
-
| Variant | Mean route error, km | Interpretation |
|
| 57 |
-
| --- | ---: | --- |
|
| 58 |
-
| Original model | 678.1 | Baseline |
|
| 59 |
-
| Learned acceleration forced to zero | 694.9 | Does not repair the failure |
|
| 60 |
-
| Field-tracker correction disabled | 1106.4 | Tracker helps on this case |
|
| 61 |
-
|
| 62 |
-
The baseline mean tracker correction is about 21.3 km per step. These interventions do NOT isolate every possible cause, but disprove the easy claim that deleting the tracker or acceleration is the demonstrated solution. They are diagnostics, not new tuned models.
|
| 63 |
-
|
| 64 |
-
### What remains unproved
|
| 65 |
-
|
| 66 |
-
- A future, matched-source Tip atmospheric verification sequence was not evaluated here; no whole-map Tip MAE claim is made. Some real weakening is expected, so simply requiring a deep low forever would be wrong.
|
| 67 |
-
- CFSR versus the training analysis source is a distribution shift. Finite/range checks do not eliminate it; compare source-matched replays and normalized distributions on many storms.
|
| 68 |
-
- This one storm cannot prove overfitting, an optimal parameter count, or guaranteed improvements from any proposed architecture.
|
| 69 |
-
|
| 70 |
-
## Why another v163-style network is unlikely to address the main issue
|
| 71 |
-
|
| 72 |
-
The actual code uses a 25 x 33, 2.5-degree grid with eight variables: MSLP, height at 500 hPa, and winds at 850/500/200 hPa. A multi-scale encoder cannot recover missing fine-scale or thermodynamic information from that grid. The 0.5-degree source already has a relatively weak resolved low (minimum 976.435 hPa); sampling to 2.5 degrees weakens it further to 980.425 hPa. These are grid values, not observed cyclone central pressure.
|
| 73 |
-
|
| 74 |
-
It predicts bounded pointwise state increments on a fixed regional grid without a global boundary forecast. The two pressure update branches permit at most about 3.52 hPa per six-hour step at a cell (0.50 times the checkpoint pressure standard deviation). This is a potential translation/gradient limitation, not proof that saturation is the observed cause.
|
| 75 |
-
|
| 76 |
-
It lacks a prognostic humidity/temperature column and evolving ocean coupling. The current physics loss matches derivatives, vorticity, divergence and shear against targets; it is not a numerical conservation-law solver. Increasing that weight cannot supply missing thermodynamics or guarantee physical pressure evolution.
|
| 77 |
-
|
| 78 |
-
The scalar intensity head, route proposal and field tracker can disagree. Numerous route losses coexist with comparatively small field objectives. Their numerical weights alone do not establish gradient dominance: measure shared-layer gradient norms and conflict before reweighting.
|
| 79 |
-
|
| 80 |
-
Training uses storm windows that have 120-hour targets and valid SST. That removes many partial lifecycle samples and does not teach general atmospheric evolution from every available analysis time. Parameter scaling leaves these problems intact.
|
| 81 |
-
|
| 82 |
-
## Proposed architecture and data contract
|
| 83 |
-
|
| 84 |
-
### 1. Atmospheric state, not pressure-only images
|
| 85 |
-
|
| 86 |
-
Use a real global 1-degree parent state and a genuine 0.25-degree western-Pacific regional state with a surrounding halo. Start with MSLP, surface pressure, 10-m winds, 2-m temperature, SST, and u/v/T/q/geopotential at 850, 700, 500, 300 and 200 hPa. Include land fraction, orography, solar geometry, availability and age channels. Add ocean heat-content/mixed-layer information only when a causal, consistently covered archive exists.
|
| 87 |
-
|
| 88 |
-
Use physical units and one canonical regridding/normalization implementation for training, replay and benchmarking. A larger interpolated copy of the current 2.5-degree labels is NOT a high-resolution dataset. Reanalysis does not fully resolve the eye even at 0.25 degrees; retain a separately identified observation-scale intensity output.
|
| 89 |
-
|
| 90 |
-
Train from 2000 onward on all eligible atmospheric timestamps, including non-storm periods, with predeclared storm-disjoint/year splits. Older years can supplement training only when sources and masks are consistent, and Tip remains excluded. No future reanalysis, best-track values or operational forecast product may enter a rollout input. A global parent generated internally from t0 is a prediction, not a future observation.
|
| 91 |
-
|
| 92 |
-
### 2. Global-to-regional atmospheric dynamics backbone
|
| 93 |
-
|
| 94 |
-
Use a spherical multi-resolution graph or spherical neural-operator backbone for the global state, with vertically coupled tokens. It forecasts evolving ridges, troughs and steering winds, not just a global pooled context vector. A regional windowed-attention/ConvNeXt operator consumes the parent forecast, its own prior predicted fields, and lateral boundary state at each step. Decode all atmospheric variables jointly.
|
| 95 |
-
|
| 96 |
-
Start with supervised six-hour state evolution and multi-step training. A differentiable transport operator can be tested for prognostic quantities with a learned physical-process residual, but do not treat MSLP as a passive scalar or claim conservation from a simple image warp. If implementing a full dynamics solver, use an established tested solver rather than an unverified hand-written approximation.
|
| 97 |
-
|
| 98 |
-
Do not require a moving cyclone crop for the first prototype: a fixed regional field avoids using true future centers to choose inputs and avoids route-driven template placement. Predicted-center local attention is allowed, but cannot move the output field to force agreement.
|
| 99 |
-
|
| 100 |
-
### 3. One coupled storm-object and field representation
|
| 101 |
-
|
| 102 |
-
Maintain a persistent storm-identity token initialized from the observed issue-time storm. Cross-attend to generated pressure, vorticity, wind and warm-core features. Forecast a spatial association distribution over the generated field, retaining multiple candidate lows where necessary. The final route is a subgrid center associated with a generated meteorological object, not an independent trajectory plus a pasted low.
|
| 103 |
-
|
| 104 |
-
Use a motion prior only as a soft association aid. Avoid a global minimum tracker, and avoid an expectation between two distinct lows. Model cyclone existence, weakening, transition and association confidence explicitly; do not force a tropical vortex to survive after genuine dissipation.
|
| 105 |
-
|
| 106 |
-
The pressure field remains the atmospheric decoder's raw output. Grid-scale pressure, observation-scale central pressure, and maximum surface wind have different contracts. Train the subgrid intensity decoder against available observations and calibrate its relation to the resolved vortex. Do not force central pressure to equal a coarse cell or claim a head offset creates a resolved eye.
|
| 107 |
-
|
| 108 |
-
### 4. Conditional probabilistic extension, after the deterministic baseline
|
| 109 |
-
|
| 110 |
-
Add a spatially coherent stochastic residual to the atmospheric evolution, shared across variables and time, using a conditional flow or FGN-style formulation. Each ensemble member must contain its own full field, storm association, track and intensity. The purpose is to represent uncertain storm positions without averaging several locations into a diffuse low.
|
| 111 |
-
|
| 112 |
-
Keep the deterministic baseline for ablation. Score members/distributions properly; never choose the best-looking member against future truth. In the demo distinguish a fixed-seed member from the ensemble mean. A smooth ensemble-mean field is not itself proof of a bad probabilistic forecast.
|
| 113 |
-
|
| 114 |
-
### 5. Training stages and sizes
|
| 115 |
-
|
| 116 |
-
1. Pretrain atmospheric next-state prediction on non-storm and storm times, then train genuinely free-running 48/120-hour rollouts with predicted lateral boundaries.
|
| 117 |
-
2. Add the storm-object and intensity decoders and joint training. Use field accuracy/structure and storm association losses with logged gradient scales, not ten overlapping route terms by default.
|
| 118 |
-
3. Add stochastic evolution only if deterministic members lose structure under uncertainty, and evaluate calibration as well as sharpness.
|
| 119 |
-
|
| 120 |
-
Initial trainable target: **40-60M total**, for example a roughly 25-30M global core plus 15-25M regional/object components. These are architecture budgets, not measured parameter counts or confirmed T4 fits. Test the same architecture at approximately 25M, 50M and 100M under matched data/optimization before attributing gains to size. A large global dataset can require substantially more training than previous storm-only runs.
|
| 121 |
-
|
| 122 |
-
Practical alternative: freeze an existing pretrained global weather backbone and train only the regional/object refinement. This may be the fastest skillful system, but label it a hybrid and disclose its pretrained-data coverage, license and full parameter count. Do not present its accuracy as a standalone TrackFormer win over that same parent. No parent product or remote service should be silently introduced.
|
| 123 |
-
|
| 124 |
-
## Promotion gates before calling v164 better
|
| 125 |
-
|
| 126 |
-
- Shared input tests: exact timestamps, field units, channel order, coordinate round trip, longitude wrapping, missingness, normalization and source checksums. Delete future-label tensors from the forward interface and verify predictions are unchanged when verification labels are shuffled.
|
| 127 |
-
- Missing SST: either fetch a verified causal patch or use a masked encoder trained with modality dropout. No unmasked all-zero fallback on an always-present training feature.
|
| 128 |
-
- Genuine field skill: matched-grid area-weighted MSLP MAE/RMSE plus anomaly and gradient error, spectral diagnostics, cyclone-centered pressure contrast and radial profiles, low-center displacement, and co-location with vorticity/winds. Evaluate core and background separately.
|
| 129 |
-
- Route: per-step direction, turning/recurvature timing, cross-track and along-track error, and full-path mean distance. Do not substitute a single terminal-distance score or optimize only appearance.
|
| 130 |
-
- Intensity: separate observed central-pressure/wind errors from gridded MSLP error; keep wind averaging conventions explicit.
|
| 131 |
-
- Multi-storm free rollouts: tropical, recurving, landfalling and dissipating cases; report failure rate and confidence. A persistence baseline must be included.
|
| 132 |
-
- Frozen validation selection, storm-clustered bootstrap uncertainty, and a new untouched test cohort. Tip is now a development diagnostic due repeated inspection; it cannot remain an untouched test merely because it is excluded from gradient training. The repeatedly used 270 cases likewise require an independent final test.
|
| 133 |
-
- Gates must not be tuned to force the DeepMind ranking. Compare the same initialization/verification sources and disclose pretrained train/test overlap where known.
|
| 134 |
-
|
| 135 |
-
## Research basis and limits
|
| 136 |
-
|
| 137 |
-
- [GraphCast](https://arxiv.org/abs/2212.12794): useful basis for multi-scale global atmospheric forecasting rather than a storm-only latent backbone.
|
| 138 |
-
- [WeatherNext Cyclones, official explanation](https://deepmind.google/blog/weathernext-ai-model-achieves-breakthrough-in-forecasting-cyclones/): combines global atmospheric prediction with cyclone observations and coherent ensembles. Its reported Mini performance also shows that resolution alone is not a sufficient explanation for our failure. Adopting the design principles does not reproduce its skill or training corpus.
|
| 139 |
-
- [StormCast](https://arxiv.org/abs/2408.10958): supports conditioning regional probabilistic evolution on synoptic variables and evaluating realistic spectra. Its short-range kilometer-scale experiments are not proof of 120-hour western-Pacific cyclone skill.
|
| 140 |
-
- [NeuralGCM](https://www.nature.com/articles/s41586-024-07744-y): supports investigating tested hybrid dynamics and learned physics, not adding an arbitrary pressure-balance penalty and calling it physically guaranteed.
|
| 141 |
-
|
| 142 |
-
Bottom line: the defensible v164 change is stronger atmospheric evolution plus a genuinely coupled storm representation, with verified data and evaluation contracts. Tip supports investigating this direction; it does not prove that more parameters or a particular new block will win.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
training/trackformer_1_2/source/v173/DESIGN.md
DELETED
|
@@ -1,52 +0,0 @@
|
|
| 1 |
-
# v1.2.73: controlled moving-core improvement
|
| 2 |
-
|
| 3 |
-
Baseline: v168 E3, SHA256
|
| 4 |
-
99ed776097e05ad6df03375465d9618741bed63a4ee57724d3ba98e295039d30.
|
| 5 |
-
The baseline is frozen as an archived reference, not frozen during v173 training.
|
| 6 |
-
|
| 7 |
-
The 21,452,595-parameter model transfers every baseline parameter. A two-scale
|
| 8 |
-
environment encoder pools physical state, recurrent memory and geographic
|
| 9 |
-
features. Two transformer blocks exchange regional context; cross attention
|
| 10 |
-
returns a correction to the environmental recurrent state. A zero-initialized
|
| 11 |
-
output makes the initial forecast equal the baseline. The moving core continues
|
| 12 |
-
to evolve pressure through relative transport and learned tendencies.
|
| 13 |
-
|
| 14 |
-
Both objectives matter: spatial pressure evolution and derived storm numbers.
|
| 15 |
-
The existing area-weighted atmospheric, outer-core pressure/gradient, track,
|
| 16 |
-
central-pressure and auxiliary wind losses remain. New masked losses supervise
|
| 17 |
-
six-hour pressure change and displacement against corresponding label changes.
|
| 18 |
-
Future fields and best tracks enter only the objective. Validation always uses
|
| 19 |
-
120 hours; training horizons are 24, 48, 72 then 120 hours. Best selection uses
|
| 20 |
-
validation, never Tip or the displayed benchmark storms.
|
| 21 |
-
|
| 22 |
-
The route is a pressure association within the moving core; central pressure is
|
| 23 |
-
sampled from that same field at that location. Maximum wind remains an auxiliary
|
| 24 |
-
scalar estimate: MSLP alone does not uniquely determine surface wind. The fixed
|
| 25 |
-
regional rendering is a resampled composite; the core is authoritative for the
|
| 26 |
-
extracted central pressure. Display +0 from the initialized core, and distinguish
|
| 27 |
-
it from raw input analysis. Do not connect raw +0 to conditioned +6.
|
| 28 |
-
|
| 29 |
-
Deliberately retained limitations: the v168 initialization includes a 100 km
|
| 30 |
-
Gaussian correction using issue-time observed pressure. It is a reconstruction,
|
| 31 |
-
not resolved truth. Fine inner geometry is underdetermined with only 800 native
|
| 32 |
-
training windows. Replacing the initializer, tracker, transport and backbone all
|
| 33 |
-
at once would destroy the controlled baseline. Learned initialization and a
|
| 34 |
-
probabilistic ensemble are later separate ablations, not implemented claims.
|
| 35 |
-
|
| 36 |
-
Only the verified full existing corpus is accepted: 13949/1041/1195 windows and
|
| 37 |
-
800/100/100 native windows. No new weather downloads. Tip is excluded, and its
|
| 38 |
-
repeatedly inspected forecasts are development diagnostics. Region is Western
|
| 39 |
-
Pacific, not a global weather model. Report native/coarse cases separately and
|
| 40 |
-
validate on untouched storms before calling this an improvement.
|
| 41 |
-
|
| 42 |
-
Mac Metal tests passed: baseline equivalence before optimization, coarse/native
|
| 43 |
-
four-step finite backward with attention gradients, field-to-pressure equality,
|
| 44 |
-
120h finite rollout, label invariance, rejected future inputs, missing intensity,
|
| 45 |
-
optimizer resume and full sampler coverage. CUDA/T4 and skill remain unverified.
|
| 46 |
-
|
| 47 |
-
Training command after the existing notebook's setup cell:
|
| 48 |
-
`python v173_pressure_train.py 10 3 1 --microbatch 4`
|
| 49 |
-
The final positional flag is accepted for compatibility; dynamics use FP32.
|
| 50 |
-
Increase physical microbatch only after measuring throughput and memory. Effective
|
| 51 |
-
batch alone does not increase GPU memory. Checkpoints and train.log persist to
|
| 52 |
-
MyDrive/typhoon_predict/training/v173; restarting resumes last.pt.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|