README.md

---
frameworks: PyTorch
language:
- en
license: apache-2.0
tags:
- OneScience
- Earth Science
- Weather Forecast
- Masked Autoencoder
- ERA5
- W-MAE
tasks: []
datasets:
- OneScience/ERA5
---
<p align="center">
  <strong>
    <span style="font-size: 30px;">W-MAE</span>
  </strong>
</p>

# Model Introduction

W-MAE (Weather Masked AutoEncoder) is a pretraining model for multivariable weather forecasting.

Paper: W-MAE: Pre-trained weather model with masked autoencoder for multi-variable weather forecasting

https://arxiv.org/abs/2304.08754

# Model Description

W-MAE first learns spatial relationships among weather variables through masked reconstruction, then learns temporal dependencies by fine-tuning on a forecasting task.

# Use Cases

| Scenario | Description |
| :---: | :--- |
| Masked weather-field pretraining | Train the W-MAE reconstruction model with ERA5 HDF5 data that follows this project's protocol. |
| Local quick validation | Use synthetic HDF5 data to check loading, training, inference, and visualization of inference results. |
| ModelScope / OneCode execution | Download the standalone model package, configure data, and run the scripts directly. |
| Multi-GPU training | Launch PyTorch DistributedDataParallel with `torchrun`. |

# Usage Guide

## 1. OneCode Usage

Experience intelligent one-click AI4S programming through the OneCode online environment:

[Click to Experience Intelligent One-Click AI4S Programming](https://web-2069360198568017922-iaaj.ksai.scnet.cn:58043/home)

## 2. Manual Installation and Usage

**Hardware Requirements**

- A GPU or DCU is recommended.
- CPU can be used for import and small-scale connectivity verification; full training and inference will be slow.
- DCU users must install DTK in advance. DTK 25.04.2 or above, or the OneScience recommended version matching your cluster, is recommended.

### Download the Model Package

```bash
hf download OneScience-Group/W-MAE --local-dir ./W-MAE
cd W-MAE
```

### Install the Runtime Environment

**DCU Environment**

```bash
# Please activate DTK and CONDA first
conda create -n onescience311 python=3.11 -y
conda activate onescience311
# uv installation is supported
pip install onescience[earth-dcu] -i http://mirrors.onescience.ai:3141/pypi/simple/  --trusted-host mirrors.onescience.ai
```

**GPU Environment**
```bash
# Please activate CONDA first
conda create -n onescience311 python=3.11 -y libstdcxx-ng=12 libgcc-ng=12 gcc_linux-64=12 gxx_linux-64=12
conda activate onescience311
# uv installation is supported
pip install onescience[earth-gpu] -i http://mirrors.onescience.ai:3141/pypi/simple/  --trusted-host mirrors.onescience.ai
```

### Training Data Introduction

The OneScience community provides an ERA5 data slice for training. Download it to the directory configured by `data.dataset_dir` in `conf/config.yaml`:

```bash
hf download --repo-type dataset OneScience-Group/ERA5 --local-dir ./data/era5
```

The W-MAE adapter expects yearly HDF5 files under `data/era5/data/`. Each file must contain a `fields` dataset, an ordered list of 20 channel names, six-hour time steps, and normalization statistics. Verify the physical ERA5 variable order before scientific training.

### Generate Synthetic Data

When real ERA5 data is unavailable, generate protocol-compatible files for pipeline checks:

```bash
python scripts/fake_data.py
```

The synthetic files use placeholder channel names and must not be used for scientific evaluation.

### Training

Single GPU:

```bash
python scripts/train.py
```

Multi-GPU:

```bash
torchrun --nproc_per_node=8 scripts/train.py
```

Training starts from random initialization and saves `data/checkpoint/model_bak.pth` by default. A compatible checkpoint can be supplied explicitly when continuing training.

### Training Weights

This repository provides a `weight/` directory for W-MAE checkpoints. The weight files will be uploaded soon and are expected to be available in the near future.

### Inference

Inference reads `data/checkpoint/model_bak.pth` by default and writes compressed reconstruction samples to `outputs/inference/`:

```bash
python scripts/inference.py
```

### Evaluation and Visualization

```bash
python scripts/result.py
```

The result script validates reconstruction files and writes diagnostic figures under `outputs/inference/diagnostics/`.

# Official OneScience Resources

| Platform | OneScience Main Repository | Skills Repository |
| --- | --- | --- |
| Gitee | https://gitee.com/onescience-ai/onescience | https://gitee.com/onescience-ai/oneskills |
| GitHub | https://github.com/onescience-ai/OneScience | https://github.com/onescience-ai/oneskills |

# Citation and License

- This repository is an independent reproduction of the original W-MAE paper.