SeaWolf-AI commited on
Commit
d216069
·
verified ·
1 Parent(s): a315651

Add live leaderboard section with captured panels

Browse files
.gitattributes CHANGED
@@ -34,3 +34,7 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
  tokenizer.json filter=lfs diff=lfs merge=lfs -text
 
 
 
 
 
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
  tokenizer.json filter=lfs diff=lfs merge=lfs -text
37
+ assets/leaderboard/01_champion.png filter=lfs diff=lfs merge=lfs -text
38
+ assets/leaderboard/04_results_and_10day_outlook.png filter=lfs diff=lfs merge=lfs -text
39
+ assets/leaderboard/06_champions_by_period_and_lead.png filter=lfs diff=lfs merge=lfs -text
40
+ assets/leaderboard/07_live_earth_city_champions.png filter=lfs diff=lfs merge=lfs -text
README.md CHANGED
@@ -21,6 +21,65 @@ datasets:
21
  - **Forecast engine**: produces the global numerical forecast. It records the lowest error in all 15 evaluated cells of the WeatherBench 2 deterministic protocol (2020), ahead of every model on the official scorecard.
22
  - **Decision core (the weights in this repository)**: Darwin-27B with the ZTC calibrated readout on the Darwin-27B-RSI backbone. Given a state and a typed question it returns a calibrated probability for every option in one forward pass. In this system it turns forecast fields into calibrated decisions: go or no-go, threshold exceedance, category probabilities.
23
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
24
  ## WeatherBench 2 results (forecast engine)
25
 
26
  RMSE vs ERA5, global, latitude-weighted, 1.5 degree, 2020, 702 initializations.
 
21
  - **Forecast engine**: produces the global numerical forecast. It records the lowest error in all 15 evaluated cells of the WeatherBench 2 deterministic protocol (2020), ahead of every model on the official scorecard.
22
  - **Decision core (the weights in this repository)**: Darwin-27B with the ZTC calibrated readout on the Darwin-27B-RSI backbone. Given a state and a typed question it returns a calibrated probability for every option in one forward pass. In this system it turns forecast fields into calibrated decisions: go or no-go, threshold exceedance, category probabilities.
23
 
24
+ ## Live leaderboard: verified every day against the world's weather centers
25
+
26
+ The [Global Weather Model Leaderboard](https://huggingface.co/spaces/FINAL-Bench/Weather-Model-Leaderboard) scores Darwin-Prediction-Weather every day against the operational models of ECMWF, NOAA, JMA, ECCC and MET Norway in 24 world cities. Each forecast is recorded when it is issued and scored when the observed value is published. Screens below were captured on 2026-11-13; the board updates daily.
27
+
28
+ ### 1. No. 1 overall
29
+
30
+ ![Champion panel: Darwin-Prediction-Weather is No. 1 with 2.09 degrees C RMSE](assets/leaderboard/01_champion.png)
31
+
32
+ 7-day forecast of daily maximum temperature, 24 cities, 2026-02-01 to 2026-11-07, 4,848 verified forecasts. Darwin-Prediction-Weather records 2.09 °C RMSE, 13.2% lower error than No. 2 (ECMWF AIFS, 2.41 °C). Against the best physics model run on a supercomputer (ECMWF IFS, 2.84 °C) the lead is 26.4%.
33
+
34
+ ### 2. Accuracy board
35
+
36
+ ![Accuracy board: eight models ranked by RMSE](assets/leaderboard/02_accuracy_board.png)
37
+
38
+ All eight models on one scale, scored on the cases every listed model covers.
39
+
40
+ | Rank | Model | Provider | Type | RMSE [°C] |
41
+ |---|---|---|---|---|
42
+ | 1 | **Darwin-Prediction-Weather** | VIDRAFT | AI | **2.09** |
43
+ | 2 | ECMWF AIFS | ECMWF | AI | 2.41 |
44
+ | 3 | AIGFS | NOAA NCEP | AI | 2.81 |
45
+ | 4 | ECMWF IFS | ECMWF | Physics (supercomputer) | 2.84 |
46
+ | 5 | MET Nordic / IFS | MET Norway | Physics (supercomputer) | 2.95 |
47
+ | 6 | JMA GSM | JMA | Physics (supercomputer) | 3.11 |
48
+ | 7 | GEM | ECCC | Physics (supercomputer) | 3.20 |
49
+ | 8 | GFS | NOAA NCEP | Physics (supercomputer) | 3.21 |
50
+
51
+ ### 3. Lowest error at every forecast day, and the city map
52
+
53
+ ![Error by forecast day, and the most accurate model in each city](assets/leaderboard/03_error_by_day_and_city_map.png)
54
+
55
+ Left: error grows with lead time for every model, and the Darwin-Prediction-Weather curve (dashed white) is the lowest from day 1 to day 7. Right: the most accurate model in each of the 24 cities.
56
+
57
+ ### 4. No. 1 in every period and every lead time
58
+
59
+ ![Champions at a glance: Darwin is No. 1 in all 35 cells](assets/leaderboard/06_champions_by_period_and_lead.png)
60
+
61
+ Five periods (latest day, last 7 days, last 30 days, last 90 days, 2026 year to date) by seven lead times (1 to 7 days). Darwin-Prediction-Weather is No. 1 in all 35 cells.
62
+
63
+ ### 5. Monthly champion, every month on record
64
+
65
+ ![Monthly champions 2026](assets/leaderboard/05_monthly_champions.png)
66
+
67
+ Darwin-Prediction-Weather holds every monthly title on the 2026 record.
68
+
69
+ ### 6. Daily results and the 10-day outlook
70
+
71
+ ![Results for the last seven days and the 10-day outlook for all 24 cities](assets/leaderboard/04_results_and_10day_outlook.png)
72
+
73
+ For each city: the observed value and the winning model for each of the last seven days, followed by the Darwin-Prediction-Weather outlook for the next ten days. Darwin-Prediction-Weather has the most daily wins of the eight models.
74
+
75
+ ### 7. Live Earth
76
+
77
+ ![Live Earth: the most accurate model in each city over live rain radar](assets/leaderboard/07_live_earth_city_champions.png)
78
+
79
+ The board's Live Earth view shows the 2026 champion of each city on a satellite map with live rain radar. Click a city on the board to see its forecast for the next days.
80
+
81
+ Leaderboard data: weather data by [Open-Meteo](https://open-meteo.com/) (CC BY 4.0); verification with ERA5, Copernicus Climate Change Service. Live Earth imagery: NASA Global Imagery Browse Services (GIBS), part of NASA's Earth Science Data and Information System (ESDIS); rain radar by [RainViewer](https://www.rainviewer.com/).
82
+
83
  ## WeatherBench 2 results (forecast engine)
84
 
85
  RMSE vs ERA5, global, latitude-weighted, 1.5 degree, 2020, 702 initializations.
assets/leaderboard/01_champion.png ADDED

Git LFS Details

  • SHA256: c441e4c2c42892a92930e16e8ec83b28af3bd25286c79236a21897a905324743
  • Pointer size: 131 Bytes
  • Size of remote file: 217 kB
assets/leaderboard/02_accuracy_board.png ADDED
assets/leaderboard/03_error_by_day_and_city_map.png ADDED
assets/leaderboard/04_results_and_10day_outlook.png ADDED

Git LFS Details

  • SHA256: a53cb9a120da9b62fec74a0d22e31224f4dbcbcc11e202a06a532db901f299a5
  • Pointer size: 131 Bytes
  • Size of remote file: 187 kB
assets/leaderboard/05_monthly_champions.png ADDED
assets/leaderboard/06_champions_by_period_and_lead.png ADDED

Git LFS Details

  • SHA256: 9a56f2d7a1fee97bde8965108d5397c0df768b65362113ad53ed570681db7730
  • Pointer size: 131 Bytes
  • Size of remote file: 193 kB
assets/leaderboard/07_live_earth_city_champions.png ADDED

Git LFS Details

  • SHA256: 21d37e050988c7f2a98518ef1eb0a20e5a4460a42d420ab202c04ab2daaad2df
  • Pointer size: 131 Bytes
  • Size of remote file: 807 kB