Add final weights and normalization constants, correct variable order
Browse filessonny_final.safetensors holds the weights of the revised paper; model.safetensors (epoch 50) is unchanged. config.json now lists the training order of the variables.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- README.md +55 -1
- config.json +19 -3
- normalization/normalize_diff_std_12.npz +3 -0
- normalization/normalize_diff_std_24.npz +3 -0
- normalization/normalize_diff_std_6.npz +3 -0
- normalization/normalize_mean.npz +3 -0
- normalization/normalize_std.npz +3 -0
- normalization_note.txt +1 -1
- sonny_final.safetensors +3 -0
README.md
CHANGED
|
@@ -12,7 +12,61 @@ pipeline_tag: other
|
|
| 12 |
|
| 13 |
# Sonny
|
| 14 |
|
| 15 |
-
StepNets one-step weather backbone: variable-aware embedding and a two-stage trunk, 69 ERA5
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 16 |
|
| 17 |
## Citation
|
| 18 |
|
|
|
|
| 12 |
|
| 13 |
# Sonny
|
| 14 |
|
| 15 |
+
StepNets one-step weather backbone: variable-aware embedding and a two-stage trunk, 69 ERA5 variables, **1.5° grid (120×240 lat×lon)**. Rows run from 88.5°S to 90°N (the south-pole row of the 121×240 WeatherBench 2 grid is dropped), and columns from 0° eastward.
|
| 16 |
+
|
| 17 |
+
## Files
|
| 18 |
+
|
| 19 |
+
| File | Contents |
|
| 20 |
+
| :--- | :--- |
|
| 21 |
+
| `sonny_final.safetensors` | **Sonny**: EMA (decay 0.999) after a five-epoch learning-rate cooldown from epoch 50. These are the weights evaluated in the revised paper. |
|
| 22 |
+
| `model.safetensors` | **Sonny (epoch 50)**: EMA (decay 0.9) at epoch 50, the weights of arXiv:2603.21284v1. |
|
| 23 |
+
| `config.json` | Constructor arguments. `init_args.variables` is the training order, which sets the order of the output channels. |
|
| 24 |
+
| `normalization/` | `normalize_mean`, `normalize_std`, and `normalize_diff_std_{6,12,24}`, one value per variable. |
|
| 25 |
+
|
| 26 |
+
**Correction (November 2026):** earlier versions of `config.json` listed the first three variables as `2m_temperature, 10m_u_component_of_wind, 10m_v_component_of_wind`. The training order is `10m_u_component_of_wind, 10m_v_component_of_wind, 2m_temperature`. Outputs obtained with the earlier list have these three channels mislabeled. This applies to both weight files.
|
| 27 |
+
|
| 28 |
+
## Usage
|
| 29 |
+
|
| 30 |
+
The model code is at [github.com/jmj2316/Sonny](https://github.com/jmj2316/Sonny) and needs `torch`, `timm`, `xformers`, and a GPU. Run this from a clone of that repository:
|
| 31 |
+
|
| 32 |
+
```python
|
| 33 |
+
import json
|
| 34 |
+
|
| 35 |
+
import numpy as np
|
| 36 |
+
import torch
|
| 37 |
+
from huggingface_hub import snapshot_download
|
| 38 |
+
from safetensors.torch import load_file
|
| 39 |
+
|
| 40 |
+
from networks.sonny import Sonny
|
| 41 |
+
|
| 42 |
+
repo = snapshot_download("Francismj/sonny")
|
| 43 |
+
args = json.load(open(f"{repo}/config.json"))["init_args"]
|
| 44 |
+
variables = args["variables"]
|
| 45 |
+
model = Sonny(**args).cuda().eval()
|
| 46 |
+
model.load_state_dict(load_file(f"{repo}/sonny_final.safetensors"), strict=True)
|
| 47 |
+
|
| 48 |
+
|
| 49 |
+
def constant(name):
|
| 50 |
+
z = np.load(f"{repo}/normalization/{name}.npz")
|
| 51 |
+
values = np.concatenate([z[v] for v in variables]).astype("float32")
|
| 52 |
+
return torch.tensor(values).cuda().view(1, -1, 1, 1)
|
| 53 |
+
|
| 54 |
+
|
| 55 |
+
mean, std = constant("normalize_mean"), constant("normalize_std")
|
| 56 |
+
|
| 57 |
+
|
| 58 |
+
def step(x, hours):
|
| 59 |
+
"""x: physical units, (batch, 69, 120, 240), channels in the order of `variables`. hours: 6, 12, or 24."""
|
| 60 |
+
diff_std = constant(f"normalize_diff_std_{hours}")
|
| 61 |
+
interval = torch.full((x.shape[0],), hours / 10, device=x.device)
|
| 62 |
+
with torch.no_grad(), torch.autocast("cuda", dtype=torch.float16):
|
| 63 |
+
delta = model((x - mean) / std, variables, interval)
|
| 64 |
+
return x + delta.float() * diff_std
|
| 65 |
+
```
|
| 66 |
+
|
| 67 |
+
The scores in the paper average three rollouts, with steps of 6, 12, and 24 h, at every 24 h lead.
|
| 68 |
+
|
| 69 |
+
This procedure was run on the 355 daily 00 UTC starts of 2022 with `sonny_final.safetensors`. Over nine variables and ten lead days, the mean RMSE differs from the values in the paper by at most 0.47% in any cell, and by −0.13% on average. The paper's evaluation code scaled the increments in float16, which also left `specific_humidity_{50,100,150,200}` unchanged during rollouts; the code above scales them in float32.
|
| 70 |
|
| 71 |
## Citation
|
| 72 |
|
config.json
CHANGED
|
@@ -8,9 +8,9 @@
|
|
| 8 |
"source_epoch": 50,
|
| 9 |
"init_args": {
|
| 10 |
"variables": [
|
| 11 |
-
"2m_temperature",
|
| 12 |
"10m_u_component_of_wind",
|
| 13 |
"10m_v_component_of_wind",
|
|
|
|
| 14 |
"mean_sea_level_pressure",
|
| 15 |
"geopotential_50",
|
| 16 |
"geopotential_100",
|
|
@@ -90,5 +90,21 @@
|
|
| 90 |
"num_heads": 6,
|
| 91 |
"mlp_ratio": 4.0,
|
| 92 |
"step_ratio": 0.5
|
| 93 |
-
}
|
| 94 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 8 |
"source_epoch": 50,
|
| 9 |
"init_args": {
|
| 10 |
"variables": [
|
|
|
|
| 11 |
"10m_u_component_of_wind",
|
| 12 |
"10m_v_component_of_wind",
|
| 13 |
+
"2m_temperature",
|
| 14 |
"mean_sea_level_pressure",
|
| 15 |
"geopotential_50",
|
| 16 |
"geopotential_100",
|
|
|
|
| 90 |
"num_heads": 6,
|
| 91 |
"mlp_ratio": 4.0,
|
| 92 |
"step_ratio": 0.5
|
| 93 |
+
},
|
| 94 |
+
"weights": {
|
| 95 |
+
"sonny_final.safetensors": {
|
| 96 |
+
"name": "Sonny",
|
| 97 |
+
"epoch": 55,
|
| 98 |
+
"ema_decay": 0.999,
|
| 99 |
+
"description": "Weights evaluated in the revised paper: EMA after a five-epoch learning-rate cooldown from epoch 50."
|
| 100 |
+
},
|
| 101 |
+
"model.safetensors": {
|
| 102 |
+
"name": "Sonny (epoch 50)",
|
| 103 |
+
"epoch": 50,
|
| 104 |
+
"ema_decay": 0.9,
|
| 105 |
+
"description": "EMA weights at epoch 50, the weights of arXiv:2603.21284v1."
|
| 106 |
+
}
|
| 107 |
+
},
|
| 108 |
+
"time_interval_input": "hours / 10",
|
| 109 |
+
"normalization": "normalization/"
|
| 110 |
+
}
|
normalization/normalize_diff_std_12.npz
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:4d9cebc5b0a31ef8b3580558372ee43aeb79a5967aecb28a04bc943b6c9ebc79
|
| 3 |
+
size 33012
|
normalization/normalize_diff_std_24.npz
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:78edd41274ce1ad95d182cdb77e4469a8046f1180131a95ddfbafb197379aaea
|
| 3 |
+
size 19020
|
normalization/normalize_diff_std_6.npz
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:ed232f7501ac1ee53d77fdb6ca34c7f8433bc3ba52815cd17016cc64eb2b3a71
|
| 3 |
+
size 33012
|
normalization/normalize_mean.npz
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:00c9caffac219440dcb03f794ec32bf6658c819f8434ea1e9bb985ba00d15684
|
| 3 |
+
size 19296
|
normalization/normalize_std.npz
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:516dd83d268d7ab2b98194092a6b200553bf86fe39bb64aeb087f21d2496255b
|
| 3 |
+
size 19296
|
normalization_note.txt
CHANGED
|
@@ -1 +1 @@
|
|
| 1 |
-
|
|
|
|
| 1 |
+
Normalization constants are in normalization/. Inputs use normalize_mean and normalize_std; the model output for a 6, 12, or 24 h interval is multiplied by normalize_diff_std_6, _12, or _24 to give the increment in physical units.
|
sonny_final.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:f79585141b6eab07947f8906b4101a7c5d93a21403b9af0e7de55929665ec137
|
| 3 |
+
size 94337776
|