Add live leaderboard section with captured panels
Browse files- .gitattributes +4 -0
- README.md +59 -0
- assets/leaderboard/01_champion.png +3 -0
- assets/leaderboard/02_accuracy_board.png +0 -0
- assets/leaderboard/03_error_by_day_and_city_map.png +0 -0
- assets/leaderboard/04_results_and_10day_outlook.png +3 -0
- assets/leaderboard/05_monthly_champions.png +0 -0
- assets/leaderboard/06_champions_by_period_and_lead.png +3 -0
- assets/leaderboard/07_live_earth_city_champions.png +3 -0
.gitattributes
CHANGED
|
@@ -34,3 +34,7 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
|
| 34 |
*.zst filter=lfs diff=lfs merge=lfs -text
|
| 35 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
| 36 |
tokenizer.json filter=lfs diff=lfs merge=lfs -text
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 34 |
*.zst filter=lfs diff=lfs merge=lfs -text
|
| 35 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
| 36 |
tokenizer.json filter=lfs diff=lfs merge=lfs -text
|
| 37 |
+
assets/leaderboard/01_champion.png filter=lfs diff=lfs merge=lfs -text
|
| 38 |
+
assets/leaderboard/04_results_and_10day_outlook.png filter=lfs diff=lfs merge=lfs -text
|
| 39 |
+
assets/leaderboard/06_champions_by_period_and_lead.png filter=lfs diff=lfs merge=lfs -text
|
| 40 |
+
assets/leaderboard/07_live_earth_city_champions.png filter=lfs diff=lfs merge=lfs -text
|
README.md
CHANGED
|
@@ -21,6 +21,65 @@ datasets:
|
|
| 21 |
- **Forecast engine**: produces the global numerical forecast. It records the lowest error in all 15 evaluated cells of the WeatherBench 2 deterministic protocol (2020), ahead of every model on the official scorecard.
|
| 22 |
- **Decision core (the weights in this repository)**: Darwin-27B with the ZTC calibrated readout on the Darwin-27B-RSI backbone. Given a state and a typed question it returns a calibrated probability for every option in one forward pass. In this system it turns forecast fields into calibrated decisions: go or no-go, threshold exceedance, category probabilities.
|
| 23 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 24 |
## WeatherBench 2 results (forecast engine)
|
| 25 |
|
| 26 |
RMSE vs ERA5, global, latitude-weighted, 1.5 degree, 2020, 702 initializations.
|
|
|
|
| 21 |
- **Forecast engine**: produces the global numerical forecast. It records the lowest error in all 15 evaluated cells of the WeatherBench 2 deterministic protocol (2020), ahead of every model on the official scorecard.
|
| 22 |
- **Decision core (the weights in this repository)**: Darwin-27B with the ZTC calibrated readout on the Darwin-27B-RSI backbone. Given a state and a typed question it returns a calibrated probability for every option in one forward pass. In this system it turns forecast fields into calibrated decisions: go or no-go, threshold exceedance, category probabilities.
|
| 23 |
|
| 24 |
+
## Live leaderboard: verified every day against the world's weather centers
|
| 25 |
+
|
| 26 |
+
The [Global Weather Model Leaderboard](https://huggingface.co/spaces/FINAL-Bench/Weather-Model-Leaderboard) scores Darwin-Prediction-Weather every day against the operational models of ECMWF, NOAA, JMA, ECCC and MET Norway in 24 world cities. Each forecast is recorded when it is issued and scored when the observed value is published. Screens below were captured on 2026-11-13; the board updates daily.
|
| 27 |
+
|
| 28 |
+
### 1. No. 1 overall
|
| 29 |
+
|
| 30 |
+

|
| 31 |
+
|
| 32 |
+
7-day forecast of daily maximum temperature, 24 cities, 2026-02-01 to 2026-11-07, 4,848 verified forecasts. Darwin-Prediction-Weather records 2.09 °C RMSE, 13.2% lower error than No. 2 (ECMWF AIFS, 2.41 °C). Against the best physics model run on a supercomputer (ECMWF IFS, 2.84 °C) the lead is 26.4%.
|
| 33 |
+
|
| 34 |
+
### 2. Accuracy board
|
| 35 |
+
|
| 36 |
+

|
| 37 |
+
|
| 38 |
+
All eight models on one scale, scored on the cases every listed model covers.
|
| 39 |
+
|
| 40 |
+
| Rank | Model | Provider | Type | RMSE [°C] |
|
| 41 |
+
|---|---|---|---|---|
|
| 42 |
+
| 1 | **Darwin-Prediction-Weather** | VIDRAFT | AI | **2.09** |
|
| 43 |
+
| 2 | ECMWF AIFS | ECMWF | AI | 2.41 |
|
| 44 |
+
| 3 | AIGFS | NOAA NCEP | AI | 2.81 |
|
| 45 |
+
| 4 | ECMWF IFS | ECMWF | Physics (supercomputer) | 2.84 |
|
| 46 |
+
| 5 | MET Nordic / IFS | MET Norway | Physics (supercomputer) | 2.95 |
|
| 47 |
+
| 6 | JMA GSM | JMA | Physics (supercomputer) | 3.11 |
|
| 48 |
+
| 7 | GEM | ECCC | Physics (supercomputer) | 3.20 |
|
| 49 |
+
| 8 | GFS | NOAA NCEP | Physics (supercomputer) | 3.21 |
|
| 50 |
+
|
| 51 |
+
### 3. Lowest error at every forecast day, and the city map
|
| 52 |
+
|
| 53 |
+

|
| 54 |
+
|
| 55 |
+
Left: error grows with lead time for every model, and the Darwin-Prediction-Weather curve (dashed white) is the lowest from day 1 to day 7. Right: the most accurate model in each of the 24 cities.
|
| 56 |
+
|
| 57 |
+
### 4. No. 1 in every period and every lead time
|
| 58 |
+
|
| 59 |
+

|
| 60 |
+
|
| 61 |
+
Five periods (latest day, last 7 days, last 30 days, last 90 days, 2026 year to date) by seven lead times (1 to 7 days). Darwin-Prediction-Weather is No. 1 in all 35 cells.
|
| 62 |
+
|
| 63 |
+
### 5. Monthly champion, every month on record
|
| 64 |
+
|
| 65 |
+

|
| 66 |
+
|
| 67 |
+
Darwin-Prediction-Weather holds every monthly title on the 2026 record.
|
| 68 |
+
|
| 69 |
+
### 6. Daily results and the 10-day outlook
|
| 70 |
+
|
| 71 |
+

|
| 72 |
+
|
| 73 |
+
For each city: the observed value and the winning model for each of the last seven days, followed by the Darwin-Prediction-Weather outlook for the next ten days. Darwin-Prediction-Weather has the most daily wins of the eight models.
|
| 74 |
+
|
| 75 |
+
### 7. Live Earth
|
| 76 |
+
|
| 77 |
+

|
| 78 |
+
|
| 79 |
+
The board's Live Earth view shows the 2026 champion of each city on a satellite map with live rain radar. Click a city on the board to see its forecast for the next days.
|
| 80 |
+
|
| 81 |
+
Leaderboard data: weather data by [Open-Meteo](https://open-meteo.com/) (CC BY 4.0); verification with ERA5, Copernicus Climate Change Service. Live Earth imagery: NASA Global Imagery Browse Services (GIBS), part of NASA's Earth Science Data and Information System (ESDIS); rain radar by [RainViewer](https://www.rainviewer.com/).
|
| 82 |
+
|
| 83 |
## WeatherBench 2 results (forecast engine)
|
| 84 |
|
| 85 |
RMSE vs ERA5, global, latitude-weighted, 1.5 degree, 2020, 702 initializations.
|
assets/leaderboard/01_champion.png
ADDED
|
Git LFS Details
|
assets/leaderboard/02_accuracy_board.png
ADDED
|
assets/leaderboard/03_error_by_day_and_city_map.png
ADDED
|
assets/leaderboard/04_results_and_10day_outlook.png
ADDED
|
Git LFS Details
|
assets/leaderboard/05_monthly_champions.png
ADDED
|
assets/leaderboard/06_champions_by_period_and_lead.png
ADDED
|
Git LFS Details
|
assets/leaderboard/07_live_earth_city_champions.png
ADDED
|
Git LFS Details
|