Upload folder using huggingface_hub
Browse files- README.md +149 -0
- conf/config.yaml +107 -0
- config.json +45 -0
- configuration.json +11 -0
- download.sh +5 -0
- model/fuxi_s2s.py +80 -0
- scripts/config.py +19 -0
- scripts/fake_data.py +59 -0
- scripts/inference.py +101 -0
- scripts/result.py +182 -0
- weight/.gitkeep +0 -0
README.md
ADDED
|
@@ -0,0 +1,149 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
frameworks: ONNX Runtime
|
| 3 |
+
language:
|
| 4 |
+
- en
|
| 5 |
+
license: cc-by-nc-nd-4.0
|
| 6 |
+
tags:
|
| 7 |
+
- OneScience
|
| 8 |
+
- Earth Science
|
| 9 |
+
- Weather Forecast
|
| 10 |
+
- Subseasonal Forecast
|
| 11 |
+
- Ensemble Forecasting
|
| 12 |
+
- ERA5
|
| 13 |
+
- ONNX
|
| 14 |
+
tasks: []
|
| 15 |
+
datasets:
|
| 16 |
+
- OneScience/ERA5
|
| 17 |
+
---
|
| 18 |
+
<p align="center">
|
| 19 |
+
<strong>
|
| 20 |
+
<span style="font-size: 30px;">FuXi-S2S</span>
|
| 21 |
+
</strong>
|
| 22 |
+
</p>
|
| 23 |
+
|
| 24 |
+
# Model Introduction
|
| 25 |
+
|
| 26 |
+
FuXi-S2S is a global subseasonal forecasting model proposed by researchers from Fudan University and collaborating institutions.
|
| 27 |
+
|
| 28 |
+
Paper: A machine learning model that outperforms conventional global subseasonal forecast models
|
| 29 |
+
|
| 30 |
+
https://doi.org/10.1038/s41467-024-50714-1
|
| 31 |
+
|
| 32 |
+
# Model Description
|
| 33 |
+
|
| 34 |
+
FuXi-S2S takes two consecutive daily mean atmospheric states as input and targets the two-week to two-month forecast range, where conventional numerical models remain challenging to use effectively. This model package exposes the official ONNX inference graph through a small ONNX Runtime adapter.
|
| 35 |
+
|
| 36 |
+
# Use Cases
|
| 37 |
+
|
| 38 |
+
| Scenario | Description |
|
| 39 |
+
| :---: | :--- |
|
| 40 |
+
| Global subseasonal forecasting | Run the official FuXi-S2S ONNX weights with ERA5 inputs following the fixed 76-channel order. |
|
| 41 |
+
| Local quick validation | Use synthetic HDF5 data to check data loading, ONNX Runtime execution, and visualization. |
|
| 42 |
+
| ModelScope / OneCode execution | Download the standalone model package, configure an ONNX Runtime provider, and run the scripts directly. |
|
| 43 |
+
|
| 44 |
+
# Usage Guide
|
| 45 |
+
|
| 46 |
+
## 1. OneCode Usage
|
| 47 |
+
|
| 48 |
+
Experience intelligent one-click AI4S programming through the OneCode online environment:
|
| 49 |
+
|
| 50 |
+
[Click to Experience Intelligent One-Click AI4S Programming](https://web-2069360198568017922-iaaj.ksai.scnet.cn:58043/home)
|
| 51 |
+
|
| 52 |
+
## 2. Manual Installation and Usage
|
| 53 |
+
|
| 54 |
+
**Hardware Requirements**
|
| 55 |
+
|
| 56 |
+
- A GPU or DCU is recommended for practical inference. CPU can be used for import and small-scale connectivity checks, but full-resolution inference will be slow.
|
| 57 |
+
- Install the ONNX Runtime build that provides the execution provider required by your hardware.
|
| 58 |
+
- DCU users must install DTK in advance. DTK 25.04.2 or above, or the OneScience recommended version matching your cluster, is recommended.
|
| 59 |
+
|
| 60 |
+
### Download the Model Package
|
| 61 |
+
|
| 62 |
+
```bash
|
| 63 |
+
hf download OneScience-Group/FuXi-S2S --local-dir ./FuXi-S2S
|
| 64 |
+
cd FuXi-S2S
|
| 65 |
+
```
|
| 66 |
+
|
| 67 |
+
### Install the Runtime Environment
|
| 68 |
+
|
| 69 |
+
**DCU Environment**
|
| 70 |
+
|
| 71 |
+
```bash
|
| 72 |
+
# Please activate DTK and CONDA first
|
| 73 |
+
conda create -n onescience311 python=3.11 -y
|
| 74 |
+
conda activate onescience311
|
| 75 |
+
# uv installation is supported
|
| 76 |
+
pip install onescience[earth-dcu] -i http://mirrors.onescience.ai:3141/pypi/simple/ --trusted-host mirrors.onescience.ai
|
| 77 |
+
```
|
| 78 |
+
|
| 79 |
+
**GPU Environment**
|
| 80 |
+
```bash
|
| 81 |
+
# Please activate CONDA first
|
| 82 |
+
conda create -n onescience311 python=3.11 -y libstdcxx-ng=12 libgcc-ng=12 gcc_linux-64=12 gxx_linux-64=12
|
| 83 |
+
conda activate onescience311
|
| 84 |
+
# uv installation is supported
|
| 85 |
+
pip install onescience[earth-gpu] -i http://mirrors.onescience.ai:3141/pypi/simple/ --trusted-host mirrors.onescience.ai
|
| 86 |
+
```
|
| 87 |
+
|
| 88 |
+
Install or select an ONNX Runtime provider matching the target hardware, then update `model.providers` in `conf/config.yaml` if necessary. The default configuration targets a DCU-compatible provider list.
|
| 89 |
+
|
| 90 |
+
### Training Data Introduction
|
| 91 |
+
|
| 92 |
+
The official model uses daily mean ERA5 states with a fixed 76-channel order. The OneScience community provides an ERA5 data slice:
|
| 93 |
+
|
| 94 |
+
```bash
|
| 95 |
+
hf download --repo-type dataset OneScience-Group/ERA5 --local-dir ./data
|
| 96 |
+
```
|
| 97 |
+
|
| 98 |
+
The adapter expects yearly files under `data/data/` and normalization arrays under `data/stats/`. Confirm the variable order in `conf/config.yaml` before inference.
|
| 99 |
+
|
| 100 |
+
### Generate Synthetic Data
|
| 101 |
+
|
| 102 |
+
When real ERA5 data is unavailable, generate native-grid HDF5 files for interface checks:
|
| 103 |
+
|
| 104 |
+
```bash
|
| 105 |
+
python scripts/fake_data.py
|
| 106 |
+
```
|
| 107 |
+
|
| 108 |
+
For a smaller smoke fixture, pass `--height 32 --width 64`; synthetic data does not reproduce the official forecast quality.
|
| 109 |
+
|
| 110 |
+
### Pre-trained Weights
|
| 111 |
+
|
| 112 |
+
The official ONNX graph requires both files below:
|
| 113 |
+
|
| 114 |
+
```text
|
| 115 |
+
weight/fuxi_s2s.onnx
|
| 116 |
+
weight/fuxi_s2s
|
| 117 |
+
```
|
| 118 |
+
|
| 119 |
+
The large weight files are not bundled in this working copy and must be supplied from the authorized release. The `weight/` directory is reserved for these files.
|
| 120 |
+
|
| 121 |
+
### Inference
|
| 122 |
+
|
| 123 |
+
Inference reads `weight/fuxi_s2s.onnx` and its external data file by default. It converts ERA5 fields to the model's `121x240` grid and writes ONNX outputs to `result/output/`:
|
| 124 |
+
|
| 125 |
+
```bash
|
| 126 |
+
python scripts/inference.py
|
| 127 |
+
```
|
| 128 |
+
|
| 129 |
+
Use `--device cpu`, `--device cuda`, or `--device dcu` and configure `model.providers` for the selected runtime.
|
| 130 |
+
|
| 131 |
+
### Evaluation and Visualization
|
| 132 |
+
|
| 133 |
+
```bash
|
| 134 |
+
python scripts/result.py
|
| 135 |
+
```
|
| 136 |
+
|
| 137 |
+
The result script reads the newest NPY output and writes multi-variable forecast figures to `result/visualization/`.
|
| 138 |
+
|
| 139 |
+
# Official OneScience Resources
|
| 140 |
+
|
| 141 |
+
| Platform | OneScience Main Repository | Skills Repository |
|
| 142 |
+
| --- | --- | --- |
|
| 143 |
+
| Gitee | https://gitee.com/onescience-ai/onescience | https://gitee.com/onescience-ai/oneskills |
|
| 144 |
+
| GitHub | https://github.com/onescience-ai/OneScience | https://github.com/onescience-ai/oneskills |
|
| 145 |
+
|
| 146 |
+
# Citation and License
|
| 147 |
+
|
| 148 |
+
- This model package contains an adapter for the official FuXi-S2S ONNX release.
|
| 149 |
+
- The official ONNX graph, external data file, and related data are subject to the CC BY-NC-ND 4.0 terms stated by the authorized release.
|
conf/config.yaml
ADDED
|
@@ -0,0 +1,107 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
project:
|
| 2 |
+
name: fuxi_s2s
|
| 3 |
+
|
| 4 |
+
data:
|
| 5 |
+
virtual_dir: ./data
|
| 6 |
+
raw_height: 721
|
| 7 |
+
raw_width: 1440
|
| 8 |
+
model_height: 121
|
| 9 |
+
model_width: 240
|
| 10 |
+
time_step_hours: 24
|
| 11 |
+
input_steps: 2
|
| 12 |
+
output_steps: 1
|
| 13 |
+
virtual_time_steps: 3
|
| 14 |
+
virtual_dtype: float16
|
| 15 |
+
variables:
|
| 16 |
+
- z1000
|
| 17 |
+
- z925
|
| 18 |
+
- z850
|
| 19 |
+
- z700
|
| 20 |
+
- z600
|
| 21 |
+
- z500
|
| 22 |
+
- z400
|
| 23 |
+
- z300
|
| 24 |
+
- z250
|
| 25 |
+
- z200
|
| 26 |
+
- z150
|
| 27 |
+
- z100
|
| 28 |
+
- z50
|
| 29 |
+
- t1000
|
| 30 |
+
- t925
|
| 31 |
+
- t850
|
| 32 |
+
- t700
|
| 33 |
+
- t600
|
| 34 |
+
- t500
|
| 35 |
+
- t400
|
| 36 |
+
- t300
|
| 37 |
+
- t250
|
| 38 |
+
- t200
|
| 39 |
+
- t150
|
| 40 |
+
- t100
|
| 41 |
+
- t50
|
| 42 |
+
- u1000
|
| 43 |
+
- u925
|
| 44 |
+
- u850
|
| 45 |
+
- u700
|
| 46 |
+
- u600
|
| 47 |
+
- u500
|
| 48 |
+
- u400
|
| 49 |
+
- u300
|
| 50 |
+
- u250
|
| 51 |
+
- u200
|
| 52 |
+
- u150
|
| 53 |
+
- u100
|
| 54 |
+
- u50
|
| 55 |
+
- v1000
|
| 56 |
+
- v925
|
| 57 |
+
- v850
|
| 58 |
+
- v700
|
| 59 |
+
- v600
|
| 60 |
+
- v500
|
| 61 |
+
- v400
|
| 62 |
+
- v300
|
| 63 |
+
- v250
|
| 64 |
+
- v200
|
| 65 |
+
- v150
|
| 66 |
+
- v100
|
| 67 |
+
- v50
|
| 68 |
+
- q1000
|
| 69 |
+
- q925
|
| 70 |
+
- q850
|
| 71 |
+
- q700
|
| 72 |
+
- q600
|
| 73 |
+
- q500
|
| 74 |
+
- q400
|
| 75 |
+
- q300
|
| 76 |
+
- q250
|
| 77 |
+
- q200
|
| 78 |
+
- q150
|
| 79 |
+
- q100
|
| 80 |
+
- q50
|
| 81 |
+
- t2m
|
| 82 |
+
- d2m
|
| 83 |
+
- sst
|
| 84 |
+
- ttr
|
| 85 |
+
- 10u
|
| 86 |
+
- 10v
|
| 87 |
+
- 100u
|
| 88 |
+
- 100v
|
| 89 |
+
- msl
|
| 90 |
+
- tcwv
|
| 91 |
+
- tp
|
| 92 |
+
|
| 93 |
+
model:
|
| 94 |
+
path: ./weight/fuxi_s2s.onnx
|
| 95 |
+
device: dcu
|
| 96 |
+
# Replace DCUExecutionProvider with the provider name supplied by the installed DCU ONNX Runtime build.
|
| 97 |
+
providers: [ROCMExecutionProvider, CPUExecutionProvider]
|
| 98 |
+
|
| 99 |
+
runtime:
|
| 100 |
+
seed: 42
|
| 101 |
+
|
| 102 |
+
inference:
|
| 103 |
+
year: 2000
|
| 104 |
+
sample_index: 0
|
| 105 |
+
output_dir: ./result/output
|
| 106 |
+
visualization_dir: ./result/visualization
|
| 107 |
+
visualization_title: FuXi-S2S output
|
config.json
ADDED
|
@@ -0,0 +1,45 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"model_name": "FuXi-S2S",
|
| 3 |
+
"model_type": "fuxi_s2s",
|
| 4 |
+
"architectures": [
|
| 5 |
+
"FuXiS2SModel"
|
| 6 |
+
],
|
| 7 |
+
"framework": "ONNX Runtime",
|
| 8 |
+
"domain": "atmosphere",
|
| 9 |
+
"task": "global-subseasonal-weather-forecasting",
|
| 10 |
+
"implementation": {
|
| 11 |
+
"entry_point": "model/fuxi_s2s.py",
|
| 12 |
+
"scope": "official FuXi-S2S ONNX inference adapter; both the ONNX graph and its external data file are required"
|
| 13 |
+
},
|
| 14 |
+
"architecture": {
|
| 15 |
+
"family": "ONNX autoregressive weather forecast model",
|
| 16 |
+
"input_steps": 2,
|
| 17 |
+
"output_steps": 1,
|
| 18 |
+
"input_channels": 76,
|
| 19 |
+
"spatial_source_shape": [
|
| 20 |
+
721,
|
| 21 |
+
1440
|
| 22 |
+
],
|
| 23 |
+
"model_shape": [
|
| 24 |
+
121,
|
| 25 |
+
240
|
| 26 |
+
],
|
| 27 |
+
"time_step_hours": 24,
|
| 28 |
+
"forecast_range": "two weeks to two months"
|
| 29 |
+
},
|
| 30 |
+
"data": {
|
| 31 |
+
"dataset": "ERA5",
|
| 32 |
+
"protocol": "fixed 76-channel daily-mean input order",
|
| 33 |
+
"normalization": "official FuXi-S2S preprocessing"
|
| 34 |
+
},
|
| 35 |
+
"weights": {
|
| 36 |
+
"onnx_graph": "weight/fuxi_s2s.onnx",
|
| 37 |
+
"external_data": "weight/fuxi_s2s"
|
| 38 |
+
},
|
| 39 |
+
"configuration_sources": [
|
| 40 |
+
"conf/config.yaml",
|
| 41 |
+
"model/fuxi_s2s.py",
|
| 42 |
+
"scripts/inference.py",
|
| 43 |
+
"scripts/config.py"
|
| 44 |
+
]
|
| 45 |
+
}
|
configuration.json
ADDED
|
@@ -0,0 +1,11 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"framework": "ONNX Runtime",
|
| 3 |
+
"task": "subseasonal_forecasting",
|
| 4 |
+
"model": "FuXi-S2S",
|
| 5 |
+
"input_format": "BTCHW",
|
| 6 |
+
"protocol": "official_onnx_inference",
|
| 7 |
+
"default_config": "conf/config.yaml",
|
| 8 |
+
"inference": "scripts/inference.py",
|
| 9 |
+
"evaluation": "scripts/result.py",
|
| 10 |
+
"visualization": "scripts/result.py"
|
| 11 |
+
}
|
download.sh
ADDED
|
@@ -0,0 +1,5 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
#!/bin/bash
|
| 2 |
+
# Download files larger than 1MB, excluding .sh .py .md .yaml .yml
|
| 3 |
+
# Total large files: 2
|
| 4 |
+
modelscope download --model OneScience/FuXi-S2S weight/fuxi_s2s --local_dir ./
|
| 5 |
+
modelscope download --model OneScience/FuXi-S2S weight/fuxi_s2s.onnx --local_dir ./
|
model/fuxi_s2s.py
ADDED
|
@@ -0,0 +1,80 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Official FuXi-S2S ONNX adapter.
|
| 2 |
+
|
| 3 |
+
The public FuXi-S2S release contains an ONNX model and inference code. This
|
| 4 |
+
module exposes the official inference contract through ONNX Runtime.
|
| 5 |
+
"""
|
| 6 |
+
|
| 7 |
+
from pathlib import Path
|
| 8 |
+
|
| 9 |
+
import numpy as np
|
| 10 |
+
|
| 11 |
+
|
| 12 |
+
class FuXiS2SModel:
|
| 13 |
+
def __init__(self, model_path, device="cpu", providers=None):
|
| 14 |
+
self.model_path = Path(model_path)
|
| 15 |
+
self.device = device
|
| 16 |
+
self.providers = list(providers) if providers else self._default_providers(device)
|
| 17 |
+
try:
|
| 18 |
+
import onnxruntime as ort
|
| 19 |
+
except ImportError as exc:
|
| 20 |
+
raise RuntimeError("onnxruntime is required for official FuXi-S2S inference") from exc
|
| 21 |
+
if not self.model_path.is_file():
|
| 22 |
+
raise FileNotFoundError(f"FuXi-S2S ONNX model not found: {self.model_path}")
|
| 23 |
+
external_data_path = self.model_path.parent / "fuxi_s2s"
|
| 24 |
+
if not external_data_path.is_file():
|
| 25 |
+
raise FileNotFoundError(
|
| 26 |
+
f"FuXi-S2S ONNX external data not found: {external_data_path}. "
|
| 27 |
+
"The official archive requires both fuxi_s2s.onnx and fuxi_s2s."
|
| 28 |
+
)
|
| 29 |
+
available_providers = ort.get_available_providers()
|
| 30 |
+
missing_providers = [provider for provider in self.providers if provider not in available_providers]
|
| 31 |
+
if missing_providers:
|
| 32 |
+
raise RuntimeError(
|
| 33 |
+
f"ONNX Runtime provider(s) unavailable: {missing_providers}. "
|
| 34 |
+
f"Available providers: {available_providers}. "
|
| 35 |
+
"Install the matching ONNX Runtime build and update model.providers."
|
| 36 |
+
)
|
| 37 |
+
session_options = ort.SessionOptions()
|
| 38 |
+
if device == "dcu":
|
| 39 |
+
# CPU constant folding can warn on kernels that are intended for the
|
| 40 |
+
# accelerator. Let the configured DCU provider execute the graph.
|
| 41 |
+
session_options.graph_optimization_level = ort.GraphOptimizationLevel.ORT_DISABLE_ALL
|
| 42 |
+
self.session = ort.InferenceSession(
|
| 43 |
+
str(self.model_path),
|
| 44 |
+
sess_options=session_options,
|
| 45 |
+
providers=self.providers,
|
| 46 |
+
)
|
| 47 |
+
|
| 48 |
+
@staticmethod
|
| 49 |
+
def _default_providers(device):
|
| 50 |
+
if device == "cpu":
|
| 51 |
+
return ["CPUExecutionProvider"]
|
| 52 |
+
if device == "cuda":
|
| 53 |
+
return ["CUDAExecutionProvider", "CPUExecutionProvider"]
|
| 54 |
+
if device == "dcu":
|
| 55 |
+
raise ValueError(
|
| 56 |
+
"DCU inference requires an explicit ONNX Runtime provider in "
|
| 57 |
+
"model.providers, for example ['DCUExecutionProvider', "
|
| 58 |
+
"'CPUExecutionProvider']"
|
| 59 |
+
)
|
| 60 |
+
raise ValueError("device must be 'cpu', 'cuda', or 'dcu'")
|
| 61 |
+
|
| 62 |
+
@property
|
| 63 |
+
def input_names(self):
|
| 64 |
+
return [item.name for item in self.session.get_inputs()]
|
| 65 |
+
|
| 66 |
+
@property
|
| 67 |
+
def output_names(self):
|
| 68 |
+
return [item.name for item in self.session.get_outputs()]
|
| 69 |
+
|
| 70 |
+
def step(self, inputs):
|
| 71 |
+
payload = {}
|
| 72 |
+
for name in self.input_names:
|
| 73 |
+
if name not in inputs:
|
| 74 |
+
raise KeyError(f"Missing FuXi-S2S ONNX input: {name}")
|
| 75 |
+
payload[name] = np.asarray(inputs[name], dtype=np.float32)
|
| 76 |
+
outputs = self.session.run(None, payload)
|
| 77 |
+
return dict(zip(self.output_names, outputs))
|
| 78 |
+
|
| 79 |
+
def __call__(self, inputs):
|
| 80 |
+
return self.step(inputs)
|
scripts/config.py
ADDED
|
@@ -0,0 +1,19 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Load project defaults from conf/config.yaml."""
|
| 2 |
+
|
| 3 |
+
from pathlib import Path
|
| 4 |
+
|
| 5 |
+
import yaml
|
| 6 |
+
|
| 7 |
+
|
| 8 |
+
PROJECT_DIR = Path(__file__).resolve().parents[1]
|
| 9 |
+
CONFIG_PATH = PROJECT_DIR / "conf" / "config.yaml"
|
| 10 |
+
|
| 11 |
+
|
| 12 |
+
def load_config():
|
| 13 |
+
with CONFIG_PATH.open(encoding="utf-8") as handle:
|
| 14 |
+
return yaml.safe_load(handle)
|
| 15 |
+
|
| 16 |
+
|
| 17 |
+
def project_path(value):
|
| 18 |
+
path = Path(value)
|
| 19 |
+
return path if path.is_absolute() else PROJECT_DIR / path
|
scripts/fake_data.py
ADDED
|
@@ -0,0 +1,59 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Generate native-grid virtual ERA5 data for OneScience ERA5Dataset."""
|
| 2 |
+
|
| 3 |
+
import argparse
|
| 4 |
+
from pathlib import Path
|
| 5 |
+
|
| 6 |
+
import numpy as np
|
| 7 |
+
|
| 8 |
+
from config import load_config, project_path
|
| 9 |
+
|
| 10 |
+
def generate(dataset_dir, year, time_steps, height, width, variables, time_step_hours, seed, dtype):
|
| 11 |
+
import h5py
|
| 12 |
+
|
| 13 |
+
dataset_dir = Path(dataset_dir)
|
| 14 |
+
data_dir = dataset_dir / "data"
|
| 15 |
+
stats_dir = dataset_dir / "stats"
|
| 16 |
+
data_dir.mkdir(parents=True, exist_ok=True)
|
| 17 |
+
stats_dir.mkdir(parents=True, exist_ok=True)
|
| 18 |
+
path = data_dir / f"{year}.h5"
|
| 19 |
+
rng = np.random.default_rng(seed + year)
|
| 20 |
+
lat = np.linspace(90.0, -90.0, height, dtype=np.float32)[:, None]
|
| 21 |
+
lon = np.linspace(0.0, 360.0, width, endpoint=False, dtype=np.float32)[None, :]
|
| 22 |
+
with h5py.File(path, "w") as handle:
|
| 23 |
+
fields = handle.create_dataset(
|
| 24 |
+
"fields", shape=(time_steps, len(variables), height, width), dtype=dtype,
|
| 25 |
+
chunks=(1, 1, min(height, 64), min(width, 128)),
|
| 26 |
+
)
|
| 27 |
+
fields.attrs["variables"] = np.asarray(variables, dtype=h5py.string_dtype())
|
| 28 |
+
fields.attrs["time_step"] = time_step_hours
|
| 29 |
+
for step in range(time_steps):
|
| 30 |
+
phase = 2.0 * np.pi * (step / max(time_steps, 1))
|
| 31 |
+
base = np.sin(np.deg2rad(lat)) + 0.25 * np.cos(np.deg2rad(lon) + phase)
|
| 32 |
+
noise = rng.normal(0.0, 0.01, size=(len(variables), height, width)).astype(np.float32)
|
| 33 |
+
fields[step] = np.asarray([(index + 1) * base + noise[index] for index in range(len(variables))], dtype=dtype)
|
| 34 |
+
# ERA5Dataset always initializes these files, even when normalize=False.
|
| 35 |
+
np.save(stats_dir / "global_means.npy", np.zeros((1, len(variables), 1, 1), dtype=np.float32))
|
| 36 |
+
np.save(stats_dir / "global_stds.npy", np.ones((1, len(variables), 1, 1), dtype=np.float32))
|
| 37 |
+
return path
|
| 38 |
+
|
| 39 |
+
|
| 40 |
+
def main():
|
| 41 |
+
config = load_config()
|
| 42 |
+
data_config = config["data"]
|
| 43 |
+
runtime_config = config["runtime"]
|
| 44 |
+
inference_config = config["inference"]
|
| 45 |
+
parser = argparse.ArgumentParser(description="Generate native-grid virtual ERA5 HDF5 data.")
|
| 46 |
+
parser.add_argument("--dataset-dir", default=str(project_path(data_config["virtual_dir"])))
|
| 47 |
+
parser.add_argument("--year", type=int, default=inference_config["year"])
|
| 48 |
+
parser.add_argument("--time-steps", type=int, default=data_config["virtual_time_steps"])
|
| 49 |
+
parser.add_argument("--height", type=int, default=data_config["raw_height"])
|
| 50 |
+
parser.add_argument("--width", type=int, default=data_config["raw_width"])
|
| 51 |
+
parser.add_argument("--seed", type=int, default=runtime_config["seed"])
|
| 52 |
+
parser.add_argument("--dtype", choices=["float16", "float32"], default=data_config["virtual_dtype"])
|
| 53 |
+
args = parser.parse_args()
|
| 54 |
+
print(generate(args.dataset_dir, args.year, args.time_steps, args.height, args.width,
|
| 55 |
+
data_config["variables"], data_config["time_step_hours"], args.seed, args.dtype))
|
| 56 |
+
|
| 57 |
+
|
| 58 |
+
if __name__ == "__main__":
|
| 59 |
+
main()
|
scripts/inference.py
ADDED
|
@@ -0,0 +1,101 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
import argparse
|
| 2 |
+
from datetime import datetime, timedelta, timezone
|
| 3 |
+
import sys
|
| 4 |
+
from pathlib import Path
|
| 5 |
+
|
| 6 |
+
import numpy as np
|
| 7 |
+
|
| 8 |
+
PROJECT_DIR = Path(__file__).resolve().parents[1]
|
| 9 |
+
sys.path.insert(0, str(PROJECT_DIR / "scripts"))
|
| 10 |
+
from config import load_config, project_path
|
| 11 |
+
|
| 12 |
+
|
| 13 |
+
VARIABLES = load_config()["data"]["variables"]
|
| 14 |
+
|
| 15 |
+
|
| 16 |
+
def prepare_input(dataset_dir, year, sample_index, onescience_src, input_steps, output_steps, model_height, model_width):
|
| 17 |
+
import torch.nn.functional as F
|
| 18 |
+
|
| 19 |
+
dataset_dir = Path(dataset_dir)
|
| 20 |
+
required_paths = [
|
| 21 |
+
dataset_dir / "data" / f"{year}.h5",
|
| 22 |
+
dataset_dir / "stats" / "global_means.npy",
|
| 23 |
+
dataset_dir / "stats" / "global_stds.npy",
|
| 24 |
+
]
|
| 25 |
+
missing_paths = [path for path in required_paths if not path.is_file()]
|
| 26 |
+
if missing_paths:
|
| 27 |
+
missing = ", ".join(str(path) for path in missing_paths)
|
| 28 |
+
raise FileNotFoundError(
|
| 29 |
+
f"ERA5Dataset files are missing: {missing}. "
|
| 30 |
+
"Run 'python scripts/fake_data.py' or update data.virtual_dir and "
|
| 31 |
+
"inference.year in conf/config.yaml."
|
| 32 |
+
)
|
| 33 |
+
if onescience_src:
|
| 34 |
+
sys.path.insert(0, onescience_src)
|
| 35 |
+
from onescience.datapipes.climate.era5 import ERA5Dataset
|
| 36 |
+
|
| 37 |
+
dataset = ERA5Dataset(
|
| 38 |
+
dataset_dir=dataset_dir,
|
| 39 |
+
used_years=[year],
|
| 40 |
+
used_variables=VARIABLES,
|
| 41 |
+
input_steps=input_steps,
|
| 42 |
+
output_steps=output_steps,
|
| 43 |
+
normalize=False,
|
| 44 |
+
)
|
| 45 |
+
fields, _, _, step_idx, _ = dataset[sample_index]
|
| 46 |
+
if fields.ndim != 4 or tuple(fields.shape[:2]) != (input_steps, len(VARIABLES)):
|
| 47 |
+
raise ValueError(f"Unexpected ERA5Dataset input shape: {tuple(fields.shape)}")
|
| 48 |
+
fields = fields.float()
|
| 49 |
+
fields[:, VARIABLES.index("tp")] = fields[:, VARIABLES.index("tp")].mul(1000.0).clamp(0.0, 1000.0)
|
| 50 |
+
fields[:, VARIABLES.index("ttr")] /= 3600.0
|
| 51 |
+
fields = F.interpolate(fields, size=(model_height, model_width), mode="bilinear", align_corners=False)
|
| 52 |
+
return fields.unsqueeze(0).numpy().astype(np.float32), step_idx
|
| 53 |
+
|
| 54 |
+
|
| 55 |
+
def main():
|
| 56 |
+
config = load_config()
|
| 57 |
+
data_config = config["data"]
|
| 58 |
+
model_config = config["model"]
|
| 59 |
+
inference_config = config["inference"]
|
| 60 |
+
parser = argparse.ArgumentParser(description="Run FuXi-S2S inference from OneScience ERA5Dataset data.")
|
| 61 |
+
parser.add_argument("--model", default=str(project_path(model_config["path"])))
|
| 62 |
+
parser.add_argument("--dataset-dir", default=str(project_path(data_config["virtual_dir"])))
|
| 63 |
+
parser.add_argument("--year", type=int, default=inference_config["year"])
|
| 64 |
+
parser.add_argument("--sample-index", type=int, default=inference_config["sample_index"])
|
| 65 |
+
parser.add_argument("--onescience-src", help="Path containing the onescience Python package")
|
| 66 |
+
parser.add_argument("--output-dir", default=str(project_path(inference_config["output_dir"])))
|
| 67 |
+
parser.add_argument("--device", default=model_config["device"], choices=["cpu", "cuda", "dcu"])
|
| 68 |
+
args = parser.parse_args()
|
| 69 |
+
sys.path.insert(0, str(Path(__file__).resolve().parents[1] / "model"))
|
| 70 |
+
from fuxi_s2s import FuXiS2SModel
|
| 71 |
+
|
| 72 |
+
model = FuXiS2SModel(args.model, device=args.device, providers=model_config["providers"])
|
| 73 |
+
fields, step_idx = prepare_input(
|
| 74 |
+
args.dataset_dir,
|
| 75 |
+
args.year,
|
| 76 |
+
args.sample_index,
|
| 77 |
+
args.onescience_src,
|
| 78 |
+
data_config["input_steps"],
|
| 79 |
+
data_config["output_steps"],
|
| 80 |
+
data_config["model_height"],
|
| 81 |
+
data_config["model_width"],
|
| 82 |
+
)
|
| 83 |
+
if fields.shape[-2:] != (data_config["model_height"], data_config["model_width"]):
|
| 84 |
+
raise ValueError(f"Unexpected FuXi-S2S input grid: {fields.shape[-2:]}")
|
| 85 |
+
inputs = {"input": fields}
|
| 86 |
+
if "step" in model.input_names:
|
| 87 |
+
inputs["step"] = np.asarray([step_idx], dtype=np.float32)
|
| 88 |
+
if "doy" in model.input_names:
|
| 89 |
+
valid_time = datetime(args.year, 1, 1, tzinfo=timezone.utc) + timedelta(days=step_idx + 1)
|
| 90 |
+
inputs["doy"] = np.asarray([min(365, valid_time.timetuple().tm_yday) / 365.0], dtype=np.float32)
|
| 91 |
+
outputs = model(inputs)
|
| 92 |
+
output_dir = Path(args.output_dir)
|
| 93 |
+
output_dir.mkdir(parents=True, exist_ok=True)
|
| 94 |
+
for name, value in outputs.items():
|
| 95 |
+
np.save(output_dir / f"{name}.npy", value)
|
| 96 |
+
print("Inference completed")
|
| 97 |
+
print({name: tuple(value.shape) for name, value in outputs.items()})
|
| 98 |
+
|
| 99 |
+
|
| 100 |
+
if __name__ == "__main__":
|
| 101 |
+
main()
|
scripts/result.py
ADDED
|
@@ -0,0 +1,182 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
import argparse
|
| 2 |
+
from pathlib import Path
|
| 3 |
+
|
| 4 |
+
import matplotlib.pyplot as plt
|
| 5 |
+
import numpy as np
|
| 6 |
+
|
| 7 |
+
from config import load_config, project_path
|
| 8 |
+
|
| 9 |
+
|
| 10 |
+
DEFAULT_VARIABLES = ("t2m", "msl", "tp", "tcwv")
|
| 11 |
+
VARIABLE_STYLES = {
|
| 12 |
+
"t2m": ("2 m temperature", "Temperature (K)", "coolwarm"),
|
| 13 |
+
"d2m": ("2 m dew point", "Temperature (K)", "coolwarm"),
|
| 14 |
+
"sst": ("Sea-surface temperature", "Temperature (K)", "coolwarm"),
|
| 15 |
+
"msl": ("Mean sea-level pressure", "Pressure (model units)", "viridis"),
|
| 16 |
+
"tp": ("Total precipitation", "Precipitation (model units)", "Blues"),
|
| 17 |
+
"tcwv": ("Total column water vapour", "Water vapour (model units)", "YlGnBu"),
|
| 18 |
+
}
|
| 19 |
+
|
| 20 |
+
|
| 21 |
+
def find_input(output_dir):
|
| 22 |
+
candidates = sorted(Path(output_dir).glob("*.npy"), key=lambda path: path.stat().st_mtime)
|
| 23 |
+
if not candidates:
|
| 24 |
+
raise FileNotFoundError(
|
| 25 |
+
f"No inference result (*.npy) was found in {output_dir}. "
|
| 26 |
+
"Run scripts/inference.py first or pass --input explicitly."
|
| 27 |
+
)
|
| 28 |
+
return candidates[-1]
|
| 29 |
+
|
| 30 |
+
|
| 31 |
+
def select_forecast(array, step, ensemble):
|
| 32 |
+
if array.ndim == 5:
|
| 33 |
+
if ensemble >= array.shape[0]:
|
| 34 |
+
raise IndexError(f"Ensemble index {ensemble} is outside array shape {array.shape}")
|
| 35 |
+
array = array[ensemble]
|
| 36 |
+
if array.ndim == 4:
|
| 37 |
+
if step >= array.shape[0]:
|
| 38 |
+
raise IndexError(f"Forecast step {step} is outside array shape {array.shape}")
|
| 39 |
+
array = array[step]
|
| 40 |
+
if array.ndim != 3:
|
| 41 |
+
raise ValueError(
|
| 42 |
+
"Expected an inference result shaped [time, channel, lat, lon], "
|
| 43 |
+
f"[channel, lat, lon], or [ensemble, time, channel, lat, lon], got {array.shape}"
|
| 44 |
+
)
|
| 45 |
+
return np.asarray(array, dtype=np.float32)
|
| 46 |
+
|
| 47 |
+
|
| 48 |
+
def parse_variables(value, variables):
|
| 49 |
+
names = [name.strip() for name in value.split(",") if name.strip()]
|
| 50 |
+
if not names:
|
| 51 |
+
raise ValueError("At least one variable must be selected with --variables")
|
| 52 |
+
unknown = [name for name in names if name not in variables]
|
| 53 |
+
if unknown:
|
| 54 |
+
raise ValueError(f"Unknown variables: {unknown}. Available variables are configured in config.yaml")
|
| 55 |
+
return names
|
| 56 |
+
|
| 57 |
+
|
| 58 |
+
def plot_field(axis, colorbar_axis, field, variable, latitudes, longitudes):
|
| 59 |
+
finite = field[np.isfinite(field)]
|
| 60 |
+
if finite.size == 0:
|
| 61 |
+
axis.set_title(f"{VARIABLE_STYLES.get(variable, (variable,))[0]} (no finite data)")
|
| 62 |
+
axis.set_facecolor("#d9dde3")
|
| 63 |
+
axis.text(
|
| 64 |
+
0.5,
|
| 65 |
+
0.5,
|
| 66 |
+
"No finite values",
|
| 67 |
+
transform=axis.transAxes,
|
| 68 |
+
ha="center",
|
| 69 |
+
va="center",
|
| 70 |
+
fontsize=12,
|
| 71 |
+
color="#263238",
|
| 72 |
+
)
|
| 73 |
+
axis.set_xlabel("Longitude (degrees)")
|
| 74 |
+
axis.set_ylabel("Latitude (degrees)")
|
| 75 |
+
axis.set_xticks(np.arange(0, 361, 60))
|
| 76 |
+
axis.set_yticks(np.arange(-90, 91, 30))
|
| 77 |
+
colorbar_axis.set_visible(False)
|
| 78 |
+
return
|
| 79 |
+
low, high = np.percentile(finite, [2, 98])
|
| 80 |
+
if low == high:
|
| 81 |
+
low, high = float(finite.min()), float(finite.max() + 1.0)
|
| 82 |
+
label, colorbar_label, colormap = VARIABLE_STYLES.get(
|
| 83 |
+
variable, (variable, "Model output", "viridis")
|
| 84 |
+
)
|
| 85 |
+
image = axis.imshow(
|
| 86 |
+
field,
|
| 87 |
+
cmap=colormap,
|
| 88 |
+
vmin=low,
|
| 89 |
+
vmax=high,
|
| 90 |
+
extent=(longitudes[0], longitudes[-1], latitudes[-1], latitudes[0]),
|
| 91 |
+
interpolation="nearest",
|
| 92 |
+
aspect="auto",
|
| 93 |
+
)
|
| 94 |
+
axis.set_title(label)
|
| 95 |
+
axis.set_xlabel("Longitude (degrees)")
|
| 96 |
+
axis.set_ylabel("Latitude (degrees)")
|
| 97 |
+
axis.set_xticks(np.arange(0, 361, 60))
|
| 98 |
+
axis.set_yticks(np.arange(-90, 91, 30))
|
| 99 |
+
axis.grid(color="white", alpha=0.3, linewidth=0.6)
|
| 100 |
+
axis.text(
|
| 101 |
+
0.02,
|
| 102 |
+
0.03,
|
| 103 |
+
f"range {finite.min():.3g} to {finite.max():.3g}\n"
|
| 104 |
+
f"mean {finite.mean():.3g} | p02-p98 {low:.3g}-{high:.3g}",
|
| 105 |
+
transform=axis.transAxes,
|
| 106 |
+
color="white",
|
| 107 |
+
fontsize=8,
|
| 108 |
+
va="bottom",
|
| 109 |
+
bbox={"facecolor": "black", "alpha": 0.55, "pad": 3, "edgecolor": "none"},
|
| 110 |
+
)
|
| 111 |
+
colorbar = plt.colorbar(image, cax=colorbar_axis, orientation="horizontal")
|
| 112 |
+
colorbar.set_label(colorbar_label)
|
| 113 |
+
|
| 114 |
+
|
| 115 |
+
def main():
|
| 116 |
+
config = load_config()
|
| 117 |
+
data_config = config["data"]
|
| 118 |
+
inference_config = config["inference"]
|
| 119 |
+
variables = data_config["variables"]
|
| 120 |
+
parser = argparse.ArgumentParser(
|
| 121 |
+
description="Render FuXi-S2S forecast fields as a geographic multi-panel image."
|
| 122 |
+
)
|
| 123 |
+
parser.add_argument(
|
| 124 |
+
"--input",
|
| 125 |
+
default=None,
|
| 126 |
+
help="Inference .npy file; defaults to the newest file in inference.output_dir",
|
| 127 |
+
)
|
| 128 |
+
parser.add_argument(
|
| 129 |
+
"--output",
|
| 130 |
+
default=None,
|
| 131 |
+
help="PNG output path; defaults to inference.visualization_dir/<input-stem>.png",
|
| 132 |
+
)
|
| 133 |
+
parser.add_argument("--title", default=inference_config["visualization_title"])
|
| 134 |
+
parser.add_argument("--step", type=int, default=0, help="Forecast time index to display")
|
| 135 |
+
parser.add_argument("--ensemble", type=int, default=0, help="Ensemble member to display")
|
| 136 |
+
parser.add_argument(
|
| 137 |
+
"--variables",
|
| 138 |
+
default=",".join(name for name in DEFAULT_VARIABLES if name in variables),
|
| 139 |
+
help="Comma-separated variable names to display",
|
| 140 |
+
)
|
| 141 |
+
args = parser.parse_args()
|
| 142 |
+
|
| 143 |
+
input_path = Path(args.input) if args.input else find_input(project_path(inference_config["output_dir"]))
|
| 144 |
+
array = select_forecast(np.load(input_path), args.step, args.ensemble)
|
| 145 |
+
selected_variables = parse_variables(args.variables, variables)
|
| 146 |
+
if array.shape[0] != len(variables):
|
| 147 |
+
raise ValueError(
|
| 148 |
+
f"Inference result has {array.shape[0]} channels, but config.yaml defines {len(variables)} variables"
|
| 149 |
+
)
|
| 150 |
+
output = Path(args.output) if args.output else project_path(inference_config["visualization_dir"]) / f"{input_path.stem}.png"
|
| 151 |
+
output.parent.mkdir(parents=True, exist_ok=True)
|
| 152 |
+
|
| 153 |
+
height, width = array.shape[-2:]
|
| 154 |
+
latitudes = np.linspace(90.0, -90.0, height)
|
| 155 |
+
longitudes = np.linspace(0.0, 360.0, width, endpoint=False)
|
| 156 |
+
figure = plt.figure(figsize=(16, 9), constrained_layout=True)
|
| 157 |
+
layout = figure.add_gridspec(
|
| 158 |
+
2,
|
| 159 |
+
len(selected_variables),
|
| 160 |
+
height_ratios=[12, 1],
|
| 161 |
+
width_ratios=[1] * len(selected_variables),
|
| 162 |
+
)
|
| 163 |
+
figure.suptitle(
|
| 164 |
+
f"{args.title}\n{input_path.name} | forecast step {args.step} | ensemble {args.ensemble}",
|
| 165 |
+
fontsize=15,
|
| 166 |
+
fontweight="bold",
|
| 167 |
+
)
|
| 168 |
+
for index, variable in enumerate(selected_variables):
|
| 169 |
+
axis = figure.add_subplot(layout[0, index])
|
| 170 |
+
colorbar_axis = figure.add_subplot(layout[1, index])
|
| 171 |
+
variable_index = variables.index(variable)
|
| 172 |
+
plot_field(axis, colorbar_axis, array[variable_index], variable, latitudes, longitudes)
|
| 173 |
+
colorbar_axis.set_xticks([])
|
| 174 |
+
colorbar_axis.set_yticks([])
|
| 175 |
+
figure.savefig(output, dpi=180, facecolor="white")
|
| 176 |
+
plt.close(figure)
|
| 177 |
+
print(f"Input: {input_path}")
|
| 178 |
+
print(f"Output: {output}")
|
| 179 |
+
|
| 180 |
+
|
| 181 |
+
if __name__ == "__main__":
|
| 182 |
+
main()
|
weight/.gitkeep
ADDED
|
File without changes
|