yzt15806542928 commited on
Commit
3fef26f
·
verified ·
1 Parent(s): f9f58b8

Upload folder using huggingface_hub

Browse files
README.md ADDED
@@ -0,0 +1,149 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ frameworks: ONNX Runtime
3
+ language:
4
+ - en
5
+ license: cc-by-nc-nd-4.0
6
+ tags:
7
+ - OneScience
8
+ - Earth Science
9
+ - Weather Forecast
10
+ - Subseasonal Forecast
11
+ - Ensemble Forecasting
12
+ - ERA5
13
+ - ONNX
14
+ tasks: []
15
+ datasets:
16
+ - OneScience/ERA5
17
+ ---
18
+ <p align="center">
19
+ <strong>
20
+ <span style="font-size: 30px;">FuXi-S2S</span>
21
+ </strong>
22
+ </p>
23
+
24
+ # Model Introduction
25
+
26
+ FuXi-S2S is a global subseasonal forecasting model proposed by researchers from Fudan University and collaborating institutions.
27
+
28
+ Paper: A machine learning model that outperforms conventional global subseasonal forecast models
29
+
30
+ https://doi.org/10.1038/s41467-024-50714-1
31
+
32
+ # Model Description
33
+
34
+ FuXi-S2S takes two consecutive daily mean atmospheric states as input and targets the two-week to two-month forecast range, where conventional numerical models remain challenging to use effectively. This model package exposes the official ONNX inference graph through a small ONNX Runtime adapter.
35
+
36
+ # Use Cases
37
+
38
+ | Scenario | Description |
39
+ | :---: | :--- |
40
+ | Global subseasonal forecasting | Run the official FuXi-S2S ONNX weights with ERA5 inputs following the fixed 76-channel order. |
41
+ | Local quick validation | Use synthetic HDF5 data to check data loading, ONNX Runtime execution, and visualization. |
42
+ | ModelScope / OneCode execution | Download the standalone model package, configure an ONNX Runtime provider, and run the scripts directly. |
43
+
44
+ # Usage Guide
45
+
46
+ ## 1. OneCode Usage
47
+
48
+ Experience intelligent one-click AI4S programming through the OneCode online environment:
49
+
50
+ [Click to Experience Intelligent One-Click AI4S Programming](https://web-2069360198568017922-iaaj.ksai.scnet.cn:58043/home)
51
+
52
+ ## 2. Manual Installation and Usage
53
+
54
+ **Hardware Requirements**
55
+
56
+ - A GPU or DCU is recommended for practical inference. CPU can be used for import and small-scale connectivity checks, but full-resolution inference will be slow.
57
+ - Install the ONNX Runtime build that provides the execution provider required by your hardware.
58
+ - DCU users must install DTK in advance. DTK 25.04.2 or above, or the OneScience recommended version matching your cluster, is recommended.
59
+
60
+ ### Download the Model Package
61
+
62
+ ```bash
63
+ hf download OneScience-Group/FuXi-S2S --local-dir ./FuXi-S2S
64
+ cd FuXi-S2S
65
+ ```
66
+
67
+ ### Install the Runtime Environment
68
+
69
+ **DCU Environment**
70
+
71
+ ```bash
72
+ # Please activate DTK and CONDA first
73
+ conda create -n onescience311 python=3.11 -y
74
+ conda activate onescience311
75
+ # uv installation is supported
76
+ pip install onescience[earth-dcu] -i http://mirrors.onescience.ai:3141/pypi/simple/ --trusted-host mirrors.onescience.ai
77
+ ```
78
+
79
+ **GPU Environment**
80
+ ```bash
81
+ # Please activate CONDA first
82
+ conda create -n onescience311 python=3.11 -y libstdcxx-ng=12 libgcc-ng=12 gcc_linux-64=12 gxx_linux-64=12
83
+ conda activate onescience311
84
+ # uv installation is supported
85
+ pip install onescience[earth-gpu] -i http://mirrors.onescience.ai:3141/pypi/simple/ --trusted-host mirrors.onescience.ai
86
+ ```
87
+
88
+ Install or select an ONNX Runtime provider matching the target hardware, then update `model.providers` in `conf/config.yaml` if necessary. The default configuration targets a DCU-compatible provider list.
89
+
90
+ ### Training Data Introduction
91
+
92
+ The official model uses daily mean ERA5 states with a fixed 76-channel order. The OneScience community provides an ERA5 data slice:
93
+
94
+ ```bash
95
+ hf download --repo-type dataset OneScience-Group/ERA5 --local-dir ./data
96
+ ```
97
+
98
+ The adapter expects yearly files under `data/data/` and normalization arrays under `data/stats/`. Confirm the variable order in `conf/config.yaml` before inference.
99
+
100
+ ### Generate Synthetic Data
101
+
102
+ When real ERA5 data is unavailable, generate native-grid HDF5 files for interface checks:
103
+
104
+ ```bash
105
+ python scripts/fake_data.py
106
+ ```
107
+
108
+ For a smaller smoke fixture, pass `--height 32 --width 64`; synthetic data does not reproduce the official forecast quality.
109
+
110
+ ### Pre-trained Weights
111
+
112
+ The official ONNX graph requires both files below:
113
+
114
+ ```text
115
+ weight/fuxi_s2s.onnx
116
+ weight/fuxi_s2s
117
+ ```
118
+
119
+ The large weight files are not bundled in this working copy and must be supplied from the authorized release. The `weight/` directory is reserved for these files.
120
+
121
+ ### Inference
122
+
123
+ Inference reads `weight/fuxi_s2s.onnx` and its external data file by default. It converts ERA5 fields to the model's `121x240` grid and writes ONNX outputs to `result/output/`:
124
+
125
+ ```bash
126
+ python scripts/inference.py
127
+ ```
128
+
129
+ Use `--device cpu`, `--device cuda`, or `--device dcu` and configure `model.providers` for the selected runtime.
130
+
131
+ ### Evaluation and Visualization
132
+
133
+ ```bash
134
+ python scripts/result.py
135
+ ```
136
+
137
+ The result script reads the newest NPY output and writes multi-variable forecast figures to `result/visualization/`.
138
+
139
+ # Official OneScience Resources
140
+
141
+ | Platform | OneScience Main Repository | Skills Repository |
142
+ | --- | --- | --- |
143
+ | Gitee | https://gitee.com/onescience-ai/onescience | https://gitee.com/onescience-ai/oneskills |
144
+ | GitHub | https://github.com/onescience-ai/OneScience | https://github.com/onescience-ai/oneskills |
145
+
146
+ # Citation and License
147
+
148
+ - This model package contains an adapter for the official FuXi-S2S ONNX release.
149
+ - The official ONNX graph, external data file, and related data are subject to the CC BY-NC-ND 4.0 terms stated by the authorized release.
conf/config.yaml ADDED
@@ -0,0 +1,107 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ project:
2
+ name: fuxi_s2s
3
+
4
+ data:
5
+ virtual_dir: ./data
6
+ raw_height: 721
7
+ raw_width: 1440
8
+ model_height: 121
9
+ model_width: 240
10
+ time_step_hours: 24
11
+ input_steps: 2
12
+ output_steps: 1
13
+ virtual_time_steps: 3
14
+ virtual_dtype: float16
15
+ variables:
16
+ - z1000
17
+ - z925
18
+ - z850
19
+ - z700
20
+ - z600
21
+ - z500
22
+ - z400
23
+ - z300
24
+ - z250
25
+ - z200
26
+ - z150
27
+ - z100
28
+ - z50
29
+ - t1000
30
+ - t925
31
+ - t850
32
+ - t700
33
+ - t600
34
+ - t500
35
+ - t400
36
+ - t300
37
+ - t250
38
+ - t200
39
+ - t150
40
+ - t100
41
+ - t50
42
+ - u1000
43
+ - u925
44
+ - u850
45
+ - u700
46
+ - u600
47
+ - u500
48
+ - u400
49
+ - u300
50
+ - u250
51
+ - u200
52
+ - u150
53
+ - u100
54
+ - u50
55
+ - v1000
56
+ - v925
57
+ - v850
58
+ - v700
59
+ - v600
60
+ - v500
61
+ - v400
62
+ - v300
63
+ - v250
64
+ - v200
65
+ - v150
66
+ - v100
67
+ - v50
68
+ - q1000
69
+ - q925
70
+ - q850
71
+ - q700
72
+ - q600
73
+ - q500
74
+ - q400
75
+ - q300
76
+ - q250
77
+ - q200
78
+ - q150
79
+ - q100
80
+ - q50
81
+ - t2m
82
+ - d2m
83
+ - sst
84
+ - ttr
85
+ - 10u
86
+ - 10v
87
+ - 100u
88
+ - 100v
89
+ - msl
90
+ - tcwv
91
+ - tp
92
+
93
+ model:
94
+ path: ./weight/fuxi_s2s.onnx
95
+ device: dcu
96
+ # Replace DCUExecutionProvider with the provider name supplied by the installed DCU ONNX Runtime build.
97
+ providers: [ROCMExecutionProvider, CPUExecutionProvider]
98
+
99
+ runtime:
100
+ seed: 42
101
+
102
+ inference:
103
+ year: 2000
104
+ sample_index: 0
105
+ output_dir: ./result/output
106
+ visualization_dir: ./result/visualization
107
+ visualization_title: FuXi-S2S output
config.json ADDED
@@ -0,0 +1,45 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "model_name": "FuXi-S2S",
3
+ "model_type": "fuxi_s2s",
4
+ "architectures": [
5
+ "FuXiS2SModel"
6
+ ],
7
+ "framework": "ONNX Runtime",
8
+ "domain": "atmosphere",
9
+ "task": "global-subseasonal-weather-forecasting",
10
+ "implementation": {
11
+ "entry_point": "model/fuxi_s2s.py",
12
+ "scope": "official FuXi-S2S ONNX inference adapter; both the ONNX graph and its external data file are required"
13
+ },
14
+ "architecture": {
15
+ "family": "ONNX autoregressive weather forecast model",
16
+ "input_steps": 2,
17
+ "output_steps": 1,
18
+ "input_channels": 76,
19
+ "spatial_source_shape": [
20
+ 721,
21
+ 1440
22
+ ],
23
+ "model_shape": [
24
+ 121,
25
+ 240
26
+ ],
27
+ "time_step_hours": 24,
28
+ "forecast_range": "two weeks to two months"
29
+ },
30
+ "data": {
31
+ "dataset": "ERA5",
32
+ "protocol": "fixed 76-channel daily-mean input order",
33
+ "normalization": "official FuXi-S2S preprocessing"
34
+ },
35
+ "weights": {
36
+ "onnx_graph": "weight/fuxi_s2s.onnx",
37
+ "external_data": "weight/fuxi_s2s"
38
+ },
39
+ "configuration_sources": [
40
+ "conf/config.yaml",
41
+ "model/fuxi_s2s.py",
42
+ "scripts/inference.py",
43
+ "scripts/config.py"
44
+ ]
45
+ }
configuration.json ADDED
@@ -0,0 +1,11 @@
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "framework": "ONNX Runtime",
3
+ "task": "subseasonal_forecasting",
4
+ "model": "FuXi-S2S",
5
+ "input_format": "BTCHW",
6
+ "protocol": "official_onnx_inference",
7
+ "default_config": "conf/config.yaml",
8
+ "inference": "scripts/inference.py",
9
+ "evaluation": "scripts/result.py",
10
+ "visualization": "scripts/result.py"
11
+ }
download.sh ADDED
@@ -0,0 +1,5 @@
 
 
 
 
 
 
1
+ #!/bin/bash
2
+ # Download files larger than 1MB, excluding .sh .py .md .yaml .yml
3
+ # Total large files: 2
4
+ modelscope download --model OneScience/FuXi-S2S weight/fuxi_s2s --local_dir ./
5
+ modelscope download --model OneScience/FuXi-S2S weight/fuxi_s2s.onnx --local_dir ./
model/fuxi_s2s.py ADDED
@@ -0,0 +1,80 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """Official FuXi-S2S ONNX adapter.
2
+
3
+ The public FuXi-S2S release contains an ONNX model and inference code. This
4
+ module exposes the official inference contract through ONNX Runtime.
5
+ """
6
+
7
+ from pathlib import Path
8
+
9
+ import numpy as np
10
+
11
+
12
+ class FuXiS2SModel:
13
+ def __init__(self, model_path, device="cpu", providers=None):
14
+ self.model_path = Path(model_path)
15
+ self.device = device
16
+ self.providers = list(providers) if providers else self._default_providers(device)
17
+ try:
18
+ import onnxruntime as ort
19
+ except ImportError as exc:
20
+ raise RuntimeError("onnxruntime is required for official FuXi-S2S inference") from exc
21
+ if not self.model_path.is_file():
22
+ raise FileNotFoundError(f"FuXi-S2S ONNX model not found: {self.model_path}")
23
+ external_data_path = self.model_path.parent / "fuxi_s2s"
24
+ if not external_data_path.is_file():
25
+ raise FileNotFoundError(
26
+ f"FuXi-S2S ONNX external data not found: {external_data_path}. "
27
+ "The official archive requires both fuxi_s2s.onnx and fuxi_s2s."
28
+ )
29
+ available_providers = ort.get_available_providers()
30
+ missing_providers = [provider for provider in self.providers if provider not in available_providers]
31
+ if missing_providers:
32
+ raise RuntimeError(
33
+ f"ONNX Runtime provider(s) unavailable: {missing_providers}. "
34
+ f"Available providers: {available_providers}. "
35
+ "Install the matching ONNX Runtime build and update model.providers."
36
+ )
37
+ session_options = ort.SessionOptions()
38
+ if device == "dcu":
39
+ # CPU constant folding can warn on kernels that are intended for the
40
+ # accelerator. Let the configured DCU provider execute the graph.
41
+ session_options.graph_optimization_level = ort.GraphOptimizationLevel.ORT_DISABLE_ALL
42
+ self.session = ort.InferenceSession(
43
+ str(self.model_path),
44
+ sess_options=session_options,
45
+ providers=self.providers,
46
+ )
47
+
48
+ @staticmethod
49
+ def _default_providers(device):
50
+ if device == "cpu":
51
+ return ["CPUExecutionProvider"]
52
+ if device == "cuda":
53
+ return ["CUDAExecutionProvider", "CPUExecutionProvider"]
54
+ if device == "dcu":
55
+ raise ValueError(
56
+ "DCU inference requires an explicit ONNX Runtime provider in "
57
+ "model.providers, for example ['DCUExecutionProvider', "
58
+ "'CPUExecutionProvider']"
59
+ )
60
+ raise ValueError("device must be 'cpu', 'cuda', or 'dcu'")
61
+
62
+ @property
63
+ def input_names(self):
64
+ return [item.name for item in self.session.get_inputs()]
65
+
66
+ @property
67
+ def output_names(self):
68
+ return [item.name for item in self.session.get_outputs()]
69
+
70
+ def step(self, inputs):
71
+ payload = {}
72
+ for name in self.input_names:
73
+ if name not in inputs:
74
+ raise KeyError(f"Missing FuXi-S2S ONNX input: {name}")
75
+ payload[name] = np.asarray(inputs[name], dtype=np.float32)
76
+ outputs = self.session.run(None, payload)
77
+ return dict(zip(self.output_names, outputs))
78
+
79
+ def __call__(self, inputs):
80
+ return self.step(inputs)
scripts/config.py ADDED
@@ -0,0 +1,19 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """Load project defaults from conf/config.yaml."""
2
+
3
+ from pathlib import Path
4
+
5
+ import yaml
6
+
7
+
8
+ PROJECT_DIR = Path(__file__).resolve().parents[1]
9
+ CONFIG_PATH = PROJECT_DIR / "conf" / "config.yaml"
10
+
11
+
12
+ def load_config():
13
+ with CONFIG_PATH.open(encoding="utf-8") as handle:
14
+ return yaml.safe_load(handle)
15
+
16
+
17
+ def project_path(value):
18
+ path = Path(value)
19
+ return path if path.is_absolute() else PROJECT_DIR / path
scripts/fake_data.py ADDED
@@ -0,0 +1,59 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """Generate native-grid virtual ERA5 data for OneScience ERA5Dataset."""
2
+
3
+ import argparse
4
+ from pathlib import Path
5
+
6
+ import numpy as np
7
+
8
+ from config import load_config, project_path
9
+
10
+ def generate(dataset_dir, year, time_steps, height, width, variables, time_step_hours, seed, dtype):
11
+ import h5py
12
+
13
+ dataset_dir = Path(dataset_dir)
14
+ data_dir = dataset_dir / "data"
15
+ stats_dir = dataset_dir / "stats"
16
+ data_dir.mkdir(parents=True, exist_ok=True)
17
+ stats_dir.mkdir(parents=True, exist_ok=True)
18
+ path = data_dir / f"{year}.h5"
19
+ rng = np.random.default_rng(seed + year)
20
+ lat = np.linspace(90.0, -90.0, height, dtype=np.float32)[:, None]
21
+ lon = np.linspace(0.0, 360.0, width, endpoint=False, dtype=np.float32)[None, :]
22
+ with h5py.File(path, "w") as handle:
23
+ fields = handle.create_dataset(
24
+ "fields", shape=(time_steps, len(variables), height, width), dtype=dtype,
25
+ chunks=(1, 1, min(height, 64), min(width, 128)),
26
+ )
27
+ fields.attrs["variables"] = np.asarray(variables, dtype=h5py.string_dtype())
28
+ fields.attrs["time_step"] = time_step_hours
29
+ for step in range(time_steps):
30
+ phase = 2.0 * np.pi * (step / max(time_steps, 1))
31
+ base = np.sin(np.deg2rad(lat)) + 0.25 * np.cos(np.deg2rad(lon) + phase)
32
+ noise = rng.normal(0.0, 0.01, size=(len(variables), height, width)).astype(np.float32)
33
+ fields[step] = np.asarray([(index + 1) * base + noise[index] for index in range(len(variables))], dtype=dtype)
34
+ # ERA5Dataset always initializes these files, even when normalize=False.
35
+ np.save(stats_dir / "global_means.npy", np.zeros((1, len(variables), 1, 1), dtype=np.float32))
36
+ np.save(stats_dir / "global_stds.npy", np.ones((1, len(variables), 1, 1), dtype=np.float32))
37
+ return path
38
+
39
+
40
+ def main():
41
+ config = load_config()
42
+ data_config = config["data"]
43
+ runtime_config = config["runtime"]
44
+ inference_config = config["inference"]
45
+ parser = argparse.ArgumentParser(description="Generate native-grid virtual ERA5 HDF5 data.")
46
+ parser.add_argument("--dataset-dir", default=str(project_path(data_config["virtual_dir"])))
47
+ parser.add_argument("--year", type=int, default=inference_config["year"])
48
+ parser.add_argument("--time-steps", type=int, default=data_config["virtual_time_steps"])
49
+ parser.add_argument("--height", type=int, default=data_config["raw_height"])
50
+ parser.add_argument("--width", type=int, default=data_config["raw_width"])
51
+ parser.add_argument("--seed", type=int, default=runtime_config["seed"])
52
+ parser.add_argument("--dtype", choices=["float16", "float32"], default=data_config["virtual_dtype"])
53
+ args = parser.parse_args()
54
+ print(generate(args.dataset_dir, args.year, args.time_steps, args.height, args.width,
55
+ data_config["variables"], data_config["time_step_hours"], args.seed, args.dtype))
56
+
57
+
58
+ if __name__ == "__main__":
59
+ main()
scripts/inference.py ADDED
@@ -0,0 +1,101 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ import argparse
2
+ from datetime import datetime, timedelta, timezone
3
+ import sys
4
+ from pathlib import Path
5
+
6
+ import numpy as np
7
+
8
+ PROJECT_DIR = Path(__file__).resolve().parents[1]
9
+ sys.path.insert(0, str(PROJECT_DIR / "scripts"))
10
+ from config import load_config, project_path
11
+
12
+
13
+ VARIABLES = load_config()["data"]["variables"]
14
+
15
+
16
+ def prepare_input(dataset_dir, year, sample_index, onescience_src, input_steps, output_steps, model_height, model_width):
17
+ import torch.nn.functional as F
18
+
19
+ dataset_dir = Path(dataset_dir)
20
+ required_paths = [
21
+ dataset_dir / "data" / f"{year}.h5",
22
+ dataset_dir / "stats" / "global_means.npy",
23
+ dataset_dir / "stats" / "global_stds.npy",
24
+ ]
25
+ missing_paths = [path for path in required_paths if not path.is_file()]
26
+ if missing_paths:
27
+ missing = ", ".join(str(path) for path in missing_paths)
28
+ raise FileNotFoundError(
29
+ f"ERA5Dataset files are missing: {missing}. "
30
+ "Run 'python scripts/fake_data.py' or update data.virtual_dir and "
31
+ "inference.year in conf/config.yaml."
32
+ )
33
+ if onescience_src:
34
+ sys.path.insert(0, onescience_src)
35
+ from onescience.datapipes.climate.era5 import ERA5Dataset
36
+
37
+ dataset = ERA5Dataset(
38
+ dataset_dir=dataset_dir,
39
+ used_years=[year],
40
+ used_variables=VARIABLES,
41
+ input_steps=input_steps,
42
+ output_steps=output_steps,
43
+ normalize=False,
44
+ )
45
+ fields, _, _, step_idx, _ = dataset[sample_index]
46
+ if fields.ndim != 4 or tuple(fields.shape[:2]) != (input_steps, len(VARIABLES)):
47
+ raise ValueError(f"Unexpected ERA5Dataset input shape: {tuple(fields.shape)}")
48
+ fields = fields.float()
49
+ fields[:, VARIABLES.index("tp")] = fields[:, VARIABLES.index("tp")].mul(1000.0).clamp(0.0, 1000.0)
50
+ fields[:, VARIABLES.index("ttr")] /= 3600.0
51
+ fields = F.interpolate(fields, size=(model_height, model_width), mode="bilinear", align_corners=False)
52
+ return fields.unsqueeze(0).numpy().astype(np.float32), step_idx
53
+
54
+
55
+ def main():
56
+ config = load_config()
57
+ data_config = config["data"]
58
+ model_config = config["model"]
59
+ inference_config = config["inference"]
60
+ parser = argparse.ArgumentParser(description="Run FuXi-S2S inference from OneScience ERA5Dataset data.")
61
+ parser.add_argument("--model", default=str(project_path(model_config["path"])))
62
+ parser.add_argument("--dataset-dir", default=str(project_path(data_config["virtual_dir"])))
63
+ parser.add_argument("--year", type=int, default=inference_config["year"])
64
+ parser.add_argument("--sample-index", type=int, default=inference_config["sample_index"])
65
+ parser.add_argument("--onescience-src", help="Path containing the onescience Python package")
66
+ parser.add_argument("--output-dir", default=str(project_path(inference_config["output_dir"])))
67
+ parser.add_argument("--device", default=model_config["device"], choices=["cpu", "cuda", "dcu"])
68
+ args = parser.parse_args()
69
+ sys.path.insert(0, str(Path(__file__).resolve().parents[1] / "model"))
70
+ from fuxi_s2s import FuXiS2SModel
71
+
72
+ model = FuXiS2SModel(args.model, device=args.device, providers=model_config["providers"])
73
+ fields, step_idx = prepare_input(
74
+ args.dataset_dir,
75
+ args.year,
76
+ args.sample_index,
77
+ args.onescience_src,
78
+ data_config["input_steps"],
79
+ data_config["output_steps"],
80
+ data_config["model_height"],
81
+ data_config["model_width"],
82
+ )
83
+ if fields.shape[-2:] != (data_config["model_height"], data_config["model_width"]):
84
+ raise ValueError(f"Unexpected FuXi-S2S input grid: {fields.shape[-2:]}")
85
+ inputs = {"input": fields}
86
+ if "step" in model.input_names:
87
+ inputs["step"] = np.asarray([step_idx], dtype=np.float32)
88
+ if "doy" in model.input_names:
89
+ valid_time = datetime(args.year, 1, 1, tzinfo=timezone.utc) + timedelta(days=step_idx + 1)
90
+ inputs["doy"] = np.asarray([min(365, valid_time.timetuple().tm_yday) / 365.0], dtype=np.float32)
91
+ outputs = model(inputs)
92
+ output_dir = Path(args.output_dir)
93
+ output_dir.mkdir(parents=True, exist_ok=True)
94
+ for name, value in outputs.items():
95
+ np.save(output_dir / f"{name}.npy", value)
96
+ print("Inference completed")
97
+ print({name: tuple(value.shape) for name, value in outputs.items()})
98
+
99
+
100
+ if __name__ == "__main__":
101
+ main()
scripts/result.py ADDED
@@ -0,0 +1,182 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ import argparse
2
+ from pathlib import Path
3
+
4
+ import matplotlib.pyplot as plt
5
+ import numpy as np
6
+
7
+ from config import load_config, project_path
8
+
9
+
10
+ DEFAULT_VARIABLES = ("t2m", "msl", "tp", "tcwv")
11
+ VARIABLE_STYLES = {
12
+ "t2m": ("2 m temperature", "Temperature (K)", "coolwarm"),
13
+ "d2m": ("2 m dew point", "Temperature (K)", "coolwarm"),
14
+ "sst": ("Sea-surface temperature", "Temperature (K)", "coolwarm"),
15
+ "msl": ("Mean sea-level pressure", "Pressure (model units)", "viridis"),
16
+ "tp": ("Total precipitation", "Precipitation (model units)", "Blues"),
17
+ "tcwv": ("Total column water vapour", "Water vapour (model units)", "YlGnBu"),
18
+ }
19
+
20
+
21
+ def find_input(output_dir):
22
+ candidates = sorted(Path(output_dir).glob("*.npy"), key=lambda path: path.stat().st_mtime)
23
+ if not candidates:
24
+ raise FileNotFoundError(
25
+ f"No inference result (*.npy) was found in {output_dir}. "
26
+ "Run scripts/inference.py first or pass --input explicitly."
27
+ )
28
+ return candidates[-1]
29
+
30
+
31
+ def select_forecast(array, step, ensemble):
32
+ if array.ndim == 5:
33
+ if ensemble >= array.shape[0]:
34
+ raise IndexError(f"Ensemble index {ensemble} is outside array shape {array.shape}")
35
+ array = array[ensemble]
36
+ if array.ndim == 4:
37
+ if step >= array.shape[0]:
38
+ raise IndexError(f"Forecast step {step} is outside array shape {array.shape}")
39
+ array = array[step]
40
+ if array.ndim != 3:
41
+ raise ValueError(
42
+ "Expected an inference result shaped [time, channel, lat, lon], "
43
+ f"[channel, lat, lon], or [ensemble, time, channel, lat, lon], got {array.shape}"
44
+ )
45
+ return np.asarray(array, dtype=np.float32)
46
+
47
+
48
+ def parse_variables(value, variables):
49
+ names = [name.strip() for name in value.split(",") if name.strip()]
50
+ if not names:
51
+ raise ValueError("At least one variable must be selected with --variables")
52
+ unknown = [name for name in names if name not in variables]
53
+ if unknown:
54
+ raise ValueError(f"Unknown variables: {unknown}. Available variables are configured in config.yaml")
55
+ return names
56
+
57
+
58
+ def plot_field(axis, colorbar_axis, field, variable, latitudes, longitudes):
59
+ finite = field[np.isfinite(field)]
60
+ if finite.size == 0:
61
+ axis.set_title(f"{VARIABLE_STYLES.get(variable, (variable,))[0]} (no finite data)")
62
+ axis.set_facecolor("#d9dde3")
63
+ axis.text(
64
+ 0.5,
65
+ 0.5,
66
+ "No finite values",
67
+ transform=axis.transAxes,
68
+ ha="center",
69
+ va="center",
70
+ fontsize=12,
71
+ color="#263238",
72
+ )
73
+ axis.set_xlabel("Longitude (degrees)")
74
+ axis.set_ylabel("Latitude (degrees)")
75
+ axis.set_xticks(np.arange(0, 361, 60))
76
+ axis.set_yticks(np.arange(-90, 91, 30))
77
+ colorbar_axis.set_visible(False)
78
+ return
79
+ low, high = np.percentile(finite, [2, 98])
80
+ if low == high:
81
+ low, high = float(finite.min()), float(finite.max() + 1.0)
82
+ label, colorbar_label, colormap = VARIABLE_STYLES.get(
83
+ variable, (variable, "Model output", "viridis")
84
+ )
85
+ image = axis.imshow(
86
+ field,
87
+ cmap=colormap,
88
+ vmin=low,
89
+ vmax=high,
90
+ extent=(longitudes[0], longitudes[-1], latitudes[-1], latitudes[0]),
91
+ interpolation="nearest",
92
+ aspect="auto",
93
+ )
94
+ axis.set_title(label)
95
+ axis.set_xlabel("Longitude (degrees)")
96
+ axis.set_ylabel("Latitude (degrees)")
97
+ axis.set_xticks(np.arange(0, 361, 60))
98
+ axis.set_yticks(np.arange(-90, 91, 30))
99
+ axis.grid(color="white", alpha=0.3, linewidth=0.6)
100
+ axis.text(
101
+ 0.02,
102
+ 0.03,
103
+ f"range {finite.min():.3g} to {finite.max():.3g}\n"
104
+ f"mean {finite.mean():.3g} | p02-p98 {low:.3g}-{high:.3g}",
105
+ transform=axis.transAxes,
106
+ color="white",
107
+ fontsize=8,
108
+ va="bottom",
109
+ bbox={"facecolor": "black", "alpha": 0.55, "pad": 3, "edgecolor": "none"},
110
+ )
111
+ colorbar = plt.colorbar(image, cax=colorbar_axis, orientation="horizontal")
112
+ colorbar.set_label(colorbar_label)
113
+
114
+
115
+ def main():
116
+ config = load_config()
117
+ data_config = config["data"]
118
+ inference_config = config["inference"]
119
+ variables = data_config["variables"]
120
+ parser = argparse.ArgumentParser(
121
+ description="Render FuXi-S2S forecast fields as a geographic multi-panel image."
122
+ )
123
+ parser.add_argument(
124
+ "--input",
125
+ default=None,
126
+ help="Inference .npy file; defaults to the newest file in inference.output_dir",
127
+ )
128
+ parser.add_argument(
129
+ "--output",
130
+ default=None,
131
+ help="PNG output path; defaults to inference.visualization_dir/<input-stem>.png",
132
+ )
133
+ parser.add_argument("--title", default=inference_config["visualization_title"])
134
+ parser.add_argument("--step", type=int, default=0, help="Forecast time index to display")
135
+ parser.add_argument("--ensemble", type=int, default=0, help="Ensemble member to display")
136
+ parser.add_argument(
137
+ "--variables",
138
+ default=",".join(name for name in DEFAULT_VARIABLES if name in variables),
139
+ help="Comma-separated variable names to display",
140
+ )
141
+ args = parser.parse_args()
142
+
143
+ input_path = Path(args.input) if args.input else find_input(project_path(inference_config["output_dir"]))
144
+ array = select_forecast(np.load(input_path), args.step, args.ensemble)
145
+ selected_variables = parse_variables(args.variables, variables)
146
+ if array.shape[0] != len(variables):
147
+ raise ValueError(
148
+ f"Inference result has {array.shape[0]} channels, but config.yaml defines {len(variables)} variables"
149
+ )
150
+ output = Path(args.output) if args.output else project_path(inference_config["visualization_dir"]) / f"{input_path.stem}.png"
151
+ output.parent.mkdir(parents=True, exist_ok=True)
152
+
153
+ height, width = array.shape[-2:]
154
+ latitudes = np.linspace(90.0, -90.0, height)
155
+ longitudes = np.linspace(0.0, 360.0, width, endpoint=False)
156
+ figure = plt.figure(figsize=(16, 9), constrained_layout=True)
157
+ layout = figure.add_gridspec(
158
+ 2,
159
+ len(selected_variables),
160
+ height_ratios=[12, 1],
161
+ width_ratios=[1] * len(selected_variables),
162
+ )
163
+ figure.suptitle(
164
+ f"{args.title}\n{input_path.name} | forecast step {args.step} | ensemble {args.ensemble}",
165
+ fontsize=15,
166
+ fontweight="bold",
167
+ )
168
+ for index, variable in enumerate(selected_variables):
169
+ axis = figure.add_subplot(layout[0, index])
170
+ colorbar_axis = figure.add_subplot(layout[1, index])
171
+ variable_index = variables.index(variable)
172
+ plot_field(axis, colorbar_axis, array[variable_index], variable, latitudes, longitudes)
173
+ colorbar_axis.set_xticks([])
174
+ colorbar_axis.set_yticks([])
175
+ figure.savefig(output, dpi=180, facecolor="white")
176
+ plt.close(figure)
177
+ print(f"Input: {input_path}")
178
+ print(f"Output: {output}")
179
+
180
+
181
+ if __name__ == "__main__":
182
+ main()
weight/.gitkeep ADDED
File without changes