Spaces:

FINAL-Bench
/

Weather-Model-Leaderboard

Running

App Files Files Community

SeaWolf-AI commited on about 17 hours ago

Commit

230ca5d

verified ·

1 Parent(s): b35b7c1

License: Apache-2.0 for the leaderboard application

Browse files

Files changed (1) hide show

README.md +148 -148

README.md CHANGED Viewed

@@ -1,148 +1,148 @@
----
-title: Global Weather Model Leaderboard
-emoji: 🌦️
-colorFrom: blue
-colorTo: indigo
-sdk: static
-pinned: true
-license: cc-by-nc-4.0
-short_description: Which weather model is most accurate? Scored daily.
-tags:
-- weather
-- weather-forecasting
-- leaderboard
-- benchmark
-- ai-weather
-- numerical-weather-prediction
-- forecast-verification
-- ecmwf
-- aifs
-- gfs
-- era5
-- darwin
-datasets:
-- FINAL-Bench/Weather-Model-Forecast-Archive
-- VIDraft/Darwin-Prediction-Weather-WB2
-models:
-- FINAL-Bench/Darwin-Prediction-Weather
----
-# Global Weather Model Leaderboard
-**Which weather forecast model is the most accurate, AI or physics?** This leaderboard answers that question with data. Every day it records what each major global weather model predicted for 24 world cities, compares those predictions with what actually happened, and updates a cumulative accuracy ranking that goes back to 1 January 2026.
-- **Live board:** https://huggingface.co/spaces/FINAL-Bench/Weather-Model-Leaderboard
-- **Open data:** [FINAL-Bench/Weather-Model-Forecast-Archive](https://huggingface.co/datasets/FINAL-Bench/Weather-Model-Forecast-Archive) (every forecast and every observed value, as CSV)
-- **Engine:** [FINAL-Bench/Darwin-Prediction-Weather](https://huggingface.co/FINAL-Bench/Darwin-Prediction-Weather)
-- **Maintained by:** VIDRAFT. Contact: [vidraft@mail.vidraft.net](mailto:vidraft@mail.vidraft.net)
-## Key results (2026 year to date, verified through 2026-11-07)
-Root-mean-square error of the daily maximum 2 m temperature, in degrees Celsius, across 24 cities. Lower is better.
-| Rank | Model | Developer | Type | 1-day | 3-day | 5-day | 7-day |
-|---|---|---|---|---|---|---|---|
-| 1 | **Darwin-Prediction-Weather** | VIDRAFT (South Korea) | AI | **0.84** | **1.17** | **1.59** | **2.09** |
-| 2 | ECMWF AIFS | ECMWF (Europe) | AI | 1.35 | 1.59 | 1.90 | 2.42 |
-| 3 | ECMWF IFS | ECMWF (Europe) | Physics (Supercomputer) | 1.44 | 1.77 | 2.21 | 2.85 |
-| 4 | ICON | DWD (Germany) | Physics (Supercomputer) | 1.47 | 1.82 | 2.39 | n/a |
-| 5 | AIGFS | NOAA NCEP (United States) | AI | 1.54 | 1.84 | 2.26 | 2.82 |
-| 6 | MET Nordic / IFS | MET Norway (Norway) | Physics (Supercomputer) | 1.60 | 1.92 | 2.35 | 2.96 |
-| 7 | GFS | NOAA NCEP (United States) | Physics (Supercomputer) | 1.83 | 2.16 | 2.64 | 3.22 |
-| 8 | JMA GSM | JMA (Japan) | Physics (Supercomputer) | 1.90 | 2.31 | 2.72 | 3.12 |
-| 9 | GEM | ECCC (Canada) | Physics (Supercomputer) | 2.03 | 2.45 | 2.83 | 3.23 |
-Rank order shown for the 1-day forecast. The live board is authoritative and changes every day.
-**Two findings stand out.** Darwin-Prediction-Weather has the lowest error at every forecast lead, 14 to 38 percent below the next model. Among the models from national and international weather centres, the AI model ECMWF AIFS leads the physics-based ECMWF IFS at every lead, by 6 to 15 percent.
-## What the leaderboard shows
-| Panel | What you see |
-|---|---|
-| Most accurate model | The current no. 1 for the selected forecast lead, period and city, with no. 2 and no. 3 |
-| Champions at a glance | Two tables of winners: by period and forecast day, and by city and forecast day |
-| Accuracy board | Every model ranked by error, with developer, country flag and type (AI or Physics) |
-| Error by forecast day | How each model's error grows from 1 day ahead to 7 days ahead |
-| Most accurate model by city | A map of 24 cities coloured by the winning model |
-| Results and 10-day outlook | For any city: the last 7 days with forecast errors, today, and each model's forecast for the next 10 days |
-| Monthly champions | The winning model for each month of 2026 |
-Controls: forecast lead (1 to 10 days), period (single day, 7, 14, 30, 90 days, year to date), verification date and city.
-## Models compared
-| Model | Developer | Type |
-|---|---|---|
-| Darwin-Prediction-Weather | VIDRAFT | AI: fusion engine with local bias correction and skill-weighted fusion of global forecasts |
-| ECMWF AIFS | European Centre for Medium-Range Weather Forecasts | AI: data-driven global forecast model |
-| ECMWF IFS | European Centre for Medium-Range Weather Forecasts | Physics: Integrated Forecasting System |
-| GFS | NOAA National Centers for Environmental Prediction | Physics: Global Forecast System |
-| AIGFS | NOAA National Centers for Environmental Prediction | AI: data-driven global forecast model |
-| ICON | Deutscher Wetterdienst | Physics: Icosahedral Nonhydrostatic model |
-| JMA GSM | Japan Meteorological Agency | Physics: Global Spectral Model |
-| GEM | Environment and Climate Change Canada | Physics: Global Environmental Multiscale model |
-| MET Nordic / IFS | MET Norway | Physics: MET Norway forecast product |
-**AI** models learn the forecast from data. **Physics (Supercomputer)** models solve the equations of the atmosphere on supercomputers.
-## Methodology
-1. **Forecasts.** Every day each model's forecast is recorded for up to 10 days ahead for 24 cities: Seoul, Busan, Tokyo, Beijing, Singapore, Delhi, Dubai, Moscow, Berlin, London, Paris, Madrid, Cairo, Nairobi, Johannesburg, New York, Chicago, Los Angeles, Mexico City, Sao Paulo, Buenos Aires, Sydney, Anchorage and Reykjavik.
-2. **Quantity.** Daily maximum 2 m temperature for the UTC day, in degrees Celsius.
-3. **Truth.** ERA5 reanalysis from the Copernicus Climate Change Service. ERA5 is published about five days after the fact. In the results timeline, more recent days use preliminary values aligned to ERA5 and are replaced when ERA5 arrives.
-4. **Score.** Root-mean-square error (RMSE). A forecast issued N days earlier is compared with the value for the same day. Lower RMSE ranks higher.
-5. **Same cases for every model.** Models are compared only on days and cities where every listed model issued a forecast. A model with less than 60 percent coverage for a selection is not listed for that selection, and the board says so.
-6. **No look-ahead.** Darwin-Prediction-Weather is fitted only on days whose observed value was already known before the forecast day being scored.
-7. **Daily update.** When the observed value for a day is published, every forecast made for that day is scored and the cumulative record is updated, once a day.
-Forecasts 1 to 7 days ahead are scored from 2026-01-01. Forecasts 8 to 10 days ahead are archived from 2026-11-13 and enter the ranking as their target days are verified.
-## Frequently asked questions
-**Which weather model is the most accurate in 2026?**
-On this board, for daily maximum temperature across 24 world cities, Darwin-Prediction-Weather has the lowest error at every lead from 1 to 7 days. Among the models from national and international weather centres, ECMWF AIFS ranks first.
-**Are AI weather models more accurate than traditional physics models?**
-In this record, yes for the leading models. ECMWF AIFS (AI) has lower error than ECMWF IFS (physics) at every lead, and NOAA AIGFS (AI) has lower error than NOAA GFS (physics) at every lead.
-**What is the difference between ECMWF AIFS and ECMWF IFS?**
-IFS is ECMWF's physics-based numerical model that solves atmospheric equations on a supercomputer. AIFS is ECMWF's data-driven AI model. Both are scored here side by side.
-**How accurate is a 7-day weather forecast?**
-For daily maximum temperature in 2026, the leading weather-centre model has an error of about 2.4 °C at 7 days, against about 1.35 °C at 1 day. Darwin-Prediction-Weather records about 2.1 °C at 7 days and 0.84 °C at 1 day.
-**What is Darwin-Prediction-Weather?**
-VIDRAFT's weather prediction system. Its forecast engine applies local bias correction and skill-weighted fusion to global forecasts. On the WeatherBench 2 deterministic protocol for 2020 it records the lowest error in all 15 evaluated cells (2 m temperature, 500 hPa geopotential, 850 hPa temperature at 1, 3, 5, 7 and 10 days). See the [model page](https://huggingface.co/FINAL-Bench/Darwin-Prediction-Weather) and the [WeatherBench 2 forecast files](https://huggingface.co/datasets/VIDraft/Darwin-Prediction-Weather-WB2).
-**How often is the leaderboard updated?**
-Once a day. New forecasts are archived, newly published observations are scored, and the cumulative record and the 10-day outlook are refreshed.
-**Can I download the data?**
-Yes. The full daily record since 2026-01-01 is in [Weather-Model-Forecast-Archive](https://huggingface.co/datasets/FINAL-Bench/Weather-Model-Forecast-Archive): one row per date, city and forecast lead, with the observed value and each model's forecast.
-**Which cities and variables are covered?**
-24 cities on six continents and daily maximum 2 m temperature. Minimum temperature, precipitation and wind are being added.
-## Data sources and attribution
-Weather data by [Open-Meteo.com](https://open-meteo.com/) (CC BY 4.0). Verification data: ERA5, Copernicus Climate Change Service. Original model sources: ECMWF, NOAA NCEP, DWD, JMA, ECCC and MET Norway, under their respective licences.
-## License
-This leaderboard and its compiled record are released under CC BY-NC 4.0. Non-commercial use only. For commercial licensing contact [vidraft@mail.vidraft.net](mailto:vidraft@mail.vidraft.net).
-## Citation
-```bibtex
-@misc{vidraft2026weatherleaderboard,
-  title  = {Global Weather Model Leaderboard},
-  author = {VIDRAFT},
-  year   = {2026},
-  url    = {https://huggingface.co/spaces/FINAL-Bench/Weather-Model-Leaderboard}
-}
-```
-## Contact
-VIDRAFT: [vidraft@mail.vidraft.net](mailto:vidraft@mail.vidraft.net)

+---
+title: Global Weather Model Leaderboard
+emoji: 🌦️
+colorFrom: blue
+colorTo: indigo
+sdk: static
+pinned: true
+license: apache-2.0
+short_description: Which weather model is most accurate? Scored daily.
+tags:
+- weather
+- weather-forecasting
+- leaderboard
+- benchmark
+- ai-weather
+- numerical-weather-prediction
+- forecast-verification
+- ecmwf
+- aifs
+- gfs
+- era5
+- darwin
+datasets:
+- FINAL-Bench/Weather-Model-Forecast-Archive
+- VIDraft/Darwin-Prediction-Weather-WB2
+models:
+- FINAL-Bench/Darwin-Prediction-Weather
+---
+# Global Weather Model Leaderboard
+**Which weather forecast model is the most accurate, AI or physics?** This leaderboard answers that question with data. Every day it records what each major global weather model predicted for 24 world cities, compares those predictions with what actually happened, and updates a cumulative accuracy ranking that goes back to 1 January 2026.
+- **Live board:** https://huggingface.co/spaces/FINAL-Bench/Weather-Model-Leaderboard
+- **Open data:** [FINAL-Bench/Weather-Model-Forecast-Archive](https://huggingface.co/datasets/FINAL-Bench/Weather-Model-Forecast-Archive) (every forecast and every observed value, as CSV)
+- **Engine:** [FINAL-Bench/Darwin-Prediction-Weather](https://huggingface.co/FINAL-Bench/Darwin-Prediction-Weather)
+- **Maintained by:** VIDRAFT. Contact: [vidraft@mail.vidraft.net](mailto:vidraft@mail.vidraft.net)
+## Key results (2026 year to date, verified through 2026-11-07)
+Root-mean-square error of the daily maximum 2 m temperature, in degrees Celsius, across 24 cities. Lower is better.
+| Rank | Model | Developer | Type | 1-day | 3-day | 5-day | 7-day |
+|---|---|---|---|---|---|---|---|
+| 1 | **Darwin-Prediction-Weather** | VIDRAFT (South Korea) | AI | **0.84** | **1.17** | **1.59** | **2.09** |
+| 2 | ECMWF AIFS | ECMWF (Europe) | AI | 1.35 | 1.59 | 1.90 | 2.42 |
+| 3 | ECMWF IFS | ECMWF (Europe) | Physics (Supercomputer) | 1.44 | 1.77 | 2.21 | 2.85 |
+| 4 | ICON | DWD (Germany) | Physics (Supercomputer) | 1.47 | 1.82 | 2.39 | n/a |
+| 5 | AIGFS | NOAA NCEP (United States) | AI | 1.54 | 1.84 | 2.26 | 2.82 |
+| 6 | MET Nordic / IFS | MET Norway (Norway) | Physics (Supercomputer) | 1.60 | 1.92 | 2.35 | 2.96 |
+| 7 | GFS | NOAA NCEP (United States) | Physics (Supercomputer) | 1.83 | 2.16 | 2.64 | 3.22 |
+| 8 | JMA GSM | JMA (Japan) | Physics (Supercomputer) | 1.90 | 2.31 | 2.72 | 3.12 |
+| 9 | GEM | ECCC (Canada) | Physics (Supercomputer) | 2.03 | 2.45 | 2.83 | 3.23 |
+Rank order shown for the 1-day forecast. The live board is authoritative and changes every day.
+**Two findings stand out.** Darwin-Prediction-Weather has the lowest error at every forecast lead, 14 to 38 percent below the next model. Among the models from national and international weather centres, the AI model ECMWF AIFS leads the physics-based ECMWF IFS at every lead, by 6 to 15 percent.
+## What the leaderboard shows
+| Panel | What you see |
+|---|---|
+| Most accurate model | The current no. 1 for the selected forecast lead, period and city, with no. 2 and no. 3 |
+| Champions at a glance | Two tables of winners: by period and forecast day, and by city and forecast day |
+| Accuracy board | Every model ranked by error, with developer, country flag and type (AI or Physics) |
+| Error by forecast day | How each model's error grows from 1 day ahead to 7 days ahead |
+| Most accurate model by city | A map of 24 cities coloured by the winning model |
+| Results and 10-day outlook | For any city: the last 7 days with forecast errors, today, and each model's forecast for the next 10 days |
+| Monthly champions | The winning model for each month of 2026 |
+Controls: forecast lead (1 to 10 days), period (single day, 7, 14, 30, 90 days, year to date), verification date and city.
+## Models compared
+| Model | Developer | Type |
+|---|---|---|
+| Darwin-Prediction-Weather | VIDRAFT | AI: fusion engine with local bias correction and skill-weighted fusion of global forecasts |
+| ECMWF AIFS | European Centre for Medium-Range Weather Forecasts | AI: data-driven global forecast model |
+| ECMWF IFS | European Centre for Medium-Range Weather Forecasts | Physics: Integrated Forecasting System |
+| GFS | NOAA National Centers for Environmental Prediction | Physics: Global Forecast System |
+| AIGFS | NOAA National Centers for Environmental Prediction | AI: data-driven global forecast model |
+| ICON | Deutscher Wetterdienst | Physics: Icosahedral Nonhydrostatic model |
+| JMA GSM | Japan Meteorological Agency | Physics: Global Spectral Model |
+| GEM | Environment and Climate Change Canada | Physics: Global Environmental Multiscale model |
+| MET Nordic / IFS | MET Norway | Physics: MET Norway forecast product |
+**AI** models learn the forecast from data. **Physics (Supercomputer)** models solve the equations of the atmosphere on supercomputers.
+## Methodology
+1. **Forecasts.** Every day each model's forecast is recorded for up to 10 days ahead for 24 cities: Seoul, Busan, Tokyo, Beijing, Singapore, Delhi, Dubai, Moscow, Berlin, London, Paris, Madrid, Cairo, Nairobi, Johannesburg, New York, Chicago, Los Angeles, Mexico City, Sao Paulo, Buenos Aires, Sydney, Anchorage and Reykjavik.
+2. **Quantity.** Daily maximum 2 m temperature for the UTC day, in degrees Celsius.
+3. **Truth.** ERA5 reanalysis from the Copernicus Climate Change Service. ERA5 is published about five days after the fact. In the results timeline, more recent days use preliminary values aligned to ERA5 and are replaced when ERA5 arrives.
+4. **Score.** Root-mean-square error (RMSE). A forecast issued N days earlier is compared with the value for the same day. Lower RMSE ranks higher.
+5. **Same cases for every model.** Models are compared only on days and cities where every listed model issued a forecast. A model with less than 60 percent coverage for a selection is not listed for that selection, and the board says so.
+6. **No look-ahead.** Darwin-Prediction-Weather is fitted only on days whose observed value was already known before the forecast day being scored.
+7. **Daily update.** When the observed value for a day is published, every forecast made for that day is scored and the cumulative record is updated, once a day.
+Forecasts 1 to 7 days ahead are scored from 2026-01-01. Forecasts 8 to 10 days ahead are archived from 2026-11-13 and enter the ranking as their target days are verified.
+## Frequently asked questions
+**Which weather model is the most accurate in 2026?**
+On this board, for daily maximum temperature across 24 world cities, Darwin-Prediction-Weather has the lowest error at every lead from 1 to 7 days. Among the models from national and international weather centres, ECMWF AIFS ranks first.
+**Are AI weather models more accurate than traditional physics models?**
+In this record, yes for the leading models. ECMWF AIFS (AI) has lower error than ECMWF IFS (physics) at every lead, and NOAA AIGFS (AI) has lower error than NOAA GFS (physics) at every lead.
+**What is the difference between ECMWF AIFS and ECMWF IFS?**
+IFS is ECMWF's physics-based numerical model that solves atmospheric equations on a supercomputer. AIFS is ECMWF's data-driven AI model. Both are scored here side by side.
+**How accurate is a 7-day weather forecast?**
+For daily maximum temperature in 2026, the leading weather-centre model has an error of about 2.4 °C at 7 days, against about 1.35 °C at 1 day. Darwin-Prediction-Weather records about 2.1 °C at 7 days and 0.84 °C at 1 day.
+**What is Darwin-Prediction-Weather?**
+VIDRAFT's weather prediction system. Its forecast engine applies local bias correction and skill-weighted fusion to global forecasts. On the WeatherBench 2 deterministic protocol for 2020 it records the lowest error in all 15 evaluated cells (2 m temperature, 500 hPa geopotential, 850 hPa temperature at 1, 3, 5, 7 and 10 days). See the [model page](https://huggingface.co/FINAL-Bench/Darwin-Prediction-Weather) and the [WeatherBench 2 forecast files](https://huggingface.co/datasets/VIDraft/Darwin-Prediction-Weather-WB2).
+**How often is the leaderboard updated?**
+Once a day. New forecasts are archived, newly published observations are scored, and the cumulative record and the 10-day outlook are refreshed.
+**Can I download the data?**
+Yes. The full daily record since 2026-01-01 is in [Weather-Model-Forecast-Archive](https://huggingface.co/datasets/FINAL-Bench/Weather-Model-Forecast-Archive): one row per date, city and forecast lead, with the observed value and each model's forecast.
+**Which cities and variables are covered?**
+24 cities on six continents and daily maximum 2 m temperature. Minimum temperature, precipitation and wind are being added.
+## Data sources and attribution
+Weather data by [Open-Meteo.com](https://open-meteo.com/) (CC BY 4.0). Verification data: ERA5, Copernicus Climate Change Service. Original model sources: ECMWF, NOAA NCEP, DWD, JMA, ECCC and MET Norway, under their respective licences.
+## License
+The leaderboard application is released under the Apache License 2.0. The compiled daily record ([Weather-Model-Forecast-Archive](https://huggingface.co/datasets/FINAL-Bench/Weather-Model-Forecast-Archive)) and the [Darwin-Prediction-Weather](https://huggingface.co/FINAL-Bench/Darwin-Prediction-Weather) model are released under CC BY-NC 4.0, non-commercial use only; for commercial licensing contact [vidraft@mail.vidraft.net](mailto:vidraft@mail.vidraft.net). Source weather data keep the attribution terms listed above.
+## Citation
+```bibtex
+@misc{vidraft2026weatherleaderboard,
+  title  = {Global Weather Model Leaderboard},
+  author = {VIDRAFT},
+  year   = {2026},
+  url    = {https://huggingface.co/spaces/FINAL-Bench/Weather-Model-Leaderboard}
+}
+```
+## Contact
+VIDRAFT: [vidraft@mail.vidraft.net](mailto:vidraft@mail.vidraft.net)