zhangrenchao commited on
Commit
a094e43
·
verified ·
1 Parent(s): 387a20d

Use English Hugging Face model card

Browse files
Files changed (1) hide show
  1. README.md +58 -59
README.md CHANGED
@@ -1,7 +1,6 @@
1
  ---
2
  frameworks: PyTorch
3
  language:
4
- - zh
5
  - en
6
  license: apache-2.0
7
  tags:
@@ -21,176 +20,176 @@ datasets: []
21
  </strong>
22
  </p>
23
 
24
- # 模型介绍
25
 
26
- SpectralGPT 通过三维空间-光谱掩码建模和多目标重建,从大规模多光谱遥感影像中学习通用表征,主要用于少标签条件下的场景分类、语义分割和变化检测。
27
 
28
- 论文:SpectralGPT: Spectral Remote Sensing Foundation Model
29
 
30
  https://arxiv.org/abs/2311.07113
31
 
32
- # 模型描述
33
 
34
- SpectralGPT 由中国科学院空天信息创新研究院等机构提出,使用 fMoW-S2 和 BigEarthNet-S2 的 12 波段 Sentinel-2 影像训练。模型适用于单标签或多标签场景分类、语义分割和变化检测等遥感任务。
35
 
36
- # 适用场景
37
 
38
- | 场景 | 说明 |
39
  | :---: | :--- |
40
- | 多光谱掩码预训练 | 使用归一化的 12 波段 Sentinel-2 多光谱影像训练 SpectralGPT。 |
41
- | 遥感场景理解 | 使用学习到的多光谱表征进行场景分类、语义分割和变化检测。 |
42
- | 本地快速验证 | 使用虚拟数据检查数据读取、掩码重建训练、推理、评估和可视化流程。 |
43
- | Hugging Face / OneCode 运行 | 作为独立模型包下载后直接安装依赖并运行脚本。 |
44
- | 多卡训练 | 通过 `torchrun` 启动多进程训练。 |
45
 
46
- # 使用说明
47
 
48
- ## 1. OneCode 使用
49
 
50
- 可通过 OneCode 在线环境体验智能化一键式 AI4S 编程:
51
 
52
- [点击体验智能化一键式 AI4S 编程](https://web-2069360198568017922-iaaj.ksai.scnet.cn:58043/home)
53
 
54
- ## 2. 手动安装使用
55
 
56
- **硬件要求**
57
 
58
- - 推荐使用 GPU 或 DCU 运行。
59
- - CPU 可以用于导入和小配置连通性验证,完整训练和推理速度较慢。
60
- - DCU 用户需要预先安装 DTK,建议使用 DTK 25.04.2 以上版本或与当前集群匹配的 OneScience 推荐版本。
61
 
62
- ### 下载模型包
63
 
64
  ```bash
65
  hf download OneScience-Group/SpectralGPT --local-dir ./SpectralGPT
66
  cd SpectralGPT
67
  ```
68
 
69
- ### 安装运行环境
70
 
71
- **DCU环境**
72
 
73
  ```bash
74
- # 请首先激活 DTK 及 CONDA
75
  conda create -n onescience311 python=3.11 -y
76
  conda activate onescience311
77
- # 支持 uv 安装
78
  pip install onescience[earth-dcu] -i http://mirrors.onescience.ai:3141/pypi/simple/ --trusted-host mirrors.onescience.ai
79
  ```
80
 
81
- **GPU环境**
82
 
83
  ```bash
84
- # 请首先激活 CONDA
85
  conda create -n onescience311 python=3.11 -y libstdcxx-ng=12 libgcc-ng=12 gcc_linux-64=12 gxx_linux-64=12
86
  conda activate onescience311
87
- # 支持 uv 安装
88
  pip install onescience[earth-gpu] -i http://mirrors.onescience.ai:3141/pypi/simple/ --trusted-host mirrors.onescience.ai
89
  ```
90
 
91
- ### 训练数据介绍
92
 
93
- 论文使用 Sentinel-2 的 12 个主波段并排除 B10,各波段缩放到 `[0,1]`。fMoW-S2 包含 882,779 张影像,其中 712,874 张用于第���阶段预训练;BigEarthNet-S2 包含 590,326 张影像,其中 354,196 张用于后续预训练。OneScience 社区目前未提供可直接用于本仓库训练的 fMoW-S2、BigEarthNet-S2 或 EuroSAT 数据,因此本仓库默认使用 `scripts/fake_data.py` 生成 8 张小规模虚拟多光谱影像进行流程验证。
94
 
95
- EuroSAT 官方数据资源:
96
 
97
  ```text
98
- 官方 Zenodo 数据页:
99
  https://zenodo.org/records/7711810
100
 
101
- 多光谱数据下载:
102
  https://madm.dfki.de/files/sentinel/EuroSATallBands.zip
103
  ```
104
 
105
- 解压 EuroSAT 多光谱数据后,可使用以下命令排除 B10、缩放到 `[0,1]` 并转换为本仓库使用的 12 波段 NPZ 数据:
106
 
107
  ```bash
108
  python scripts/fake_data.py --real-dir ./data/EuroSAT_MS
109
  ```
110
 
111
- 虚拟数据保存为 `data/fake_spectralgpt.npz`,只用于验证数据读取、掩码重建训练、检查点保存与加载、推理、评估和可视化流程,不代表真实 Sentinel-2 影像,也不能用于复现论文指标。真实数据训练不应先运行不带 `--real-dir` 的虚拟数据命令,并需要使用一致的波段顺序、空间尺寸和数值缩放方式。
112
 
113
- ### 训练
114
 
115
- 使用虚拟数据快速验证:
116
 
117
  ```bash
118
  python scripts/fake_data.py
119
  python scripts/train.py
120
  ```
121
 
122
- 使用真实 EuroSAT 多光谱 TIFF 训练:
123
 
124
  ```bash
125
  python scripts/fake_data.py --real-dir ./data/EuroSAT_MS
126
  python scripts/train.py
127
  ```
128
 
129
- 虚拟数据和真实数据共用同一个训练脚本。需要调整训练规模时,可修改 `conf/config.yaml` 中的 `data.samples`、`model.image_size`、`training.epochs` 和 `training.batch_size`。
130
 
131
- 单卡:
132
 
133
  ```bash
134
  python scripts/train.py
135
  ```
136
 
137
- 多卡:
138
 
139
  ```bash
140
  torchrun --nproc_per_node=8 --nnodes=1 --rdzv_id=1000 --rdzv_backend=c10d --max_restarts=0 --master_addr="localhost" --master_port=29500 scripts/train.py
141
  ```
142
 
143
- 训练输出:
144
 
145
  ```text
146
  result/checkpoints/best.pth
147
  result/checkpoints/last.pth
148
- 每轮输出 reconstruction_loss。
149
  ```
150
 
151
- 训练日志和检查点中的 `data_source` 与 `protocol` 用于标识当前使用的是虚拟数据还是真实数据。使用虚拟数据生成的检查点只用于流程验证,不具备真实遥感表征能力。
152
 
153
- ### 训练权重
154
 
155
- 本仓库将在 `weight/` 文件夹内提供基于 Sentinel-2 多光谱遥感数据训练的权重,权重文件即将上传,预计将于近期完成。
156
 
157
- ### 推理
158
 
159
  ```bash
160
  python scripts/inference.py
161
  ```
162
 
163
- 推理读取训练数据路径配置和训练生成的检查点,并输出多光谱影像重建结果。
164
 
165
- 推理结果输出到:
166
 
167
  ```text
168
  result/output/reconstruction.npz
169
  ```
170
 
171
- ### 评估和可视化
172
 
173
  ```bash
174
  python scripts/result.py
175
  ```
176
 
177
- 评估和可视化输出到:
178
 
179
  ```text
180
  result/output/metrics.json
181
  result/output/reconstruction.ppm
182
  ```
183
 
184
- 指标包括重建 MSE、MAE、PSNR 和逐波段 RMSE;可视化并排显示输入与重建的假彩色合成图。使用虚拟数据时,评估结果只用于确认代码流程能够运行,不展示或报告虚拟数据数值;使用真实数据时,指标表示当前真实影像上的掩码重建结果。
185
 
186
- # OneScience 官方信息
187
 
188
- | 平台 | OneScience 主仓库 | Skills 仓库 |
189
  | --- | --- | --- |
190
  | Gitee | https://gitee.com/onescience-ai/onescience | https://gitee.com/onescience-ai/oneskills |
191
  | GitHub | https://github.com/onescience-ai/OneScience | https://github.com/onescience-ai/oneskills |
192
 
193
- # 引用与许可证
194
 
195
- - 本仓库为 SpectralGPT 原始论文的复现版本。
196
- - 本模型包采用 Apache License 2.0;原始数据与论文作者发布的模型权重遵循各自来源的许可条款。
 
1
  ---
2
  frameworks: PyTorch
3
  language:
 
4
  - en
5
  license: apache-2.0
6
  tags:
 
20
  </strong>
21
  </p>
22
 
23
+ # Model Introduction
24
 
25
+ SpectralGPT learns general-purpose representations from large-scale multispectral remote sensing imagery through three-dimensional spatial-spectral masked modeling and multi-objective reconstruction. It is designed for scene classification, semantic segmentation, and change detection under limited-label conditions.
26
 
27
+ Paper: SpectralGPT: Spectral Remote Sensing Foundation Model
28
 
29
  https://arxiv.org/abs/2311.07113
30
 
31
+ # Model Description
32
 
33
+ SpectralGPT was proposed by researchers from the Aerospace Information Research Institute, Chinese Academy of Sciences, and related institutions. It is trained with 12-band Sentinel-2 imagery from fMoW-S2 and BigEarthNet-S2 and is suitable for single-label or multi-label scene classification, semantic segmentation, and change detection.
34
 
35
+ # Use Cases
36
 
37
+ | Scenario | Description |
38
  | :---: | :--- |
39
+ | Multispectral masked pretraining | Train SpectralGPT with normalized 12-band Sentinel-2 multispectral imagery. |
40
+ | Remote sensing scene understanding | Use learned multispectral representations for scene classification, semantic segmentation, and change detection. |
41
+ | Local quick validation | Use synthetic data to validate data loading, masked reconstruction training, inference, evaluation, and visualization. |
42
+ | Hugging Face / OneCode execution | Download the standalone model package, install dependencies, and run the scripts directly. |
43
+ | Multi-GPU training | Launch distributed multi-process training with `torchrun`. |
44
 
45
+ # Usage Guide
46
 
47
+ ## 1. OneCode Usage
48
 
49
+ Experience intelligent one-click AI4S programming through the OneCode online environment:
50
 
51
+ [Click to Experience Intelligent One-Click AI4S Programming](https://web-2069360198568017922-iaaj.ksai.scnet.cn:58043/home)
52
 
53
+ ## 2. Manual Installation and Usage
54
 
55
+ **Hardware Requirements**
56
 
57
+ - A GPU or DCU is recommended.
58
+ - CPU can be used for import and small-scale connectivity verification; full training and inference will be slower.
59
+ - DCU users must install DTK in advance. DTK 25.04.2 or above, or the OneScience recommended version matching the cluster, is recommended.
60
 
61
+ ### Download the Model Package
62
 
63
  ```bash
64
  hf download OneScience-Group/SpectralGPT --local-dir ./SpectralGPT
65
  cd SpectralGPT
66
  ```
67
 
68
+ ### Install the Runtime Environment
69
 
70
+ **DCU Environment**
71
 
72
  ```bash
73
+ # Please activate DTK and CONDA first
74
  conda create -n onescience311 python=3.11 -y
75
  conda activate onescience311
76
+ # uv installation is supported
77
  pip install onescience[earth-dcu] -i http://mirrors.onescience.ai:3141/pypi/simple/ --trusted-host mirrors.onescience.ai
78
  ```
79
 
80
+ **GPU Environment**
81
 
82
  ```bash
83
+ # Please activate CONDA first
84
  conda create -n onescience311 python=3.11 -y libstdcxx-ng=12 libgcc-ng=12 gcc_linux-64=12 gxx_linux-64=12
85
  conda activate onescience311
86
+ # uv installation is supported
87
  pip install onescience[earth-gpu] -i http://mirrors.onescience.ai:3141/pypi/simple/ --trusted-host mirrors.onescience.ai
88
  ```
89
 
90
+ ### Training Data Introduction
91
 
92
+ The paper uses 12 Sentinel-2 spectral bands and excludes B10, with each band scaled to `[0,1]`. The fMoW-S2 dataset contains 882,779 images, including 712,874 images used for the first pretraining stage. BigEarthNet-S2 contains 590,326 images, including 354,196 images used for subsequent pretraining. OneScience does not currently provide fMoW-S2, BigEarthNet-S2, or EuroSAT data that can be downloaded directly for this repository, so the default workflow uses a small synthetic multispectral dataset for pipeline validation.
93
 
94
+ Official EuroSAT data resources:
95
 
96
  ```text
97
+ Official Zenodo page:
98
  https://zenodo.org/records/7711810
99
 
100
+ Multispectral data download:
101
  https://madm.dfki.de/files/sentinel/EuroSATallBands.zip
102
  ```
103
 
104
+ After extracting the EuroSAT multispectral data, use the following command to exclude B10, scale the bands to `[0,1]`, and convert the 13-band TIFF files into the 12-band NPZ data used by this repository:
105
 
106
  ```bash
107
  python scripts/fake_data.py --real-dir ./data/EuroSAT_MS
108
  ```
109
 
110
+ Synthetic data is saved to `data/fake_spectralgpt.npz` and is used only to validate data loading, masked reconstruction training, checkpoint saving and loading, inference, evaluation, and visualization. It does not represent real Sentinel-2 imagery and cannot reproduce the paper results. Real-data training should not run `scripts/fake_data.py` without `--real-dir`, and the input TIFF files must use a consistent band order, spatial size, and value scaling method.
111
 
112
+ ### Training
113
 
114
+ Local quick validation with synthetic data:
115
 
116
  ```bash
117
  python scripts/fake_data.py
118
  python scripts/train.py
119
  ```
120
 
121
+ Training with real EuroSAT multispectral TIFF files:
122
 
123
  ```bash
124
  python scripts/fake_data.py --real-dir ./data/EuroSAT_MS
125
  python scripts/train.py
126
  ```
127
 
128
+ Synthetic and real data use the same training script. Adjust `data.samples`, `model.image_size`, `training.epochs`, and `training.batch_size` in `conf/config.yaml` as needed.
129
 
130
+ Single GPU or CPU:
131
 
132
  ```bash
133
  python scripts/train.py
134
  ```
135
 
136
+ Multi-GPU:
137
 
138
  ```bash
139
  torchrun --nproc_per_node=8 --nnodes=1 --rdzv_id=1000 --rdzv_backend=c10d --max_restarts=0 --master_addr="localhost" --master_port=29500 scripts/train.py
140
  ```
141
 
142
+ Training outputs:
143
 
144
  ```text
145
  result/checkpoints/best.pth
146
  result/checkpoints/last.pth
147
+ Each epoch prints reconstruction_loss.
148
  ```
149
 
150
+ The `data_source` and `protocol` fields in the training logs and checkpoints identify whether synthetic or real data was used. Checkpoints generated from synthetic data are only for pipeline validation and do not provide real remote sensing representations.
151
 
152
+ ### Training Weights
153
 
154
+ Multispectral Sentinel-2 training weights will be provided under `weight/` in a future update. No trained weights are included in the current package.
155
 
156
+ ### Inference
157
 
158
  ```bash
159
  python scripts/inference.py
160
  ```
161
 
162
+ Inference reads the configured dataset and the checkpoint generated during training, then produces multispectral image reconstruction results.
163
 
164
+ Prediction output:
165
 
166
  ```text
167
  result/output/reconstruction.npz
168
  ```
169
 
170
+ ### Evaluation and Visualization
171
 
172
  ```bash
173
  python scripts/result.py
174
  ```
175
 
176
+ Evaluation and visualization outputs:
177
 
178
  ```text
179
  result/output/metrics.json
180
  result/output/reconstruction.ppm
181
  ```
182
 
183
+ The metrics include reconstruction MSE, MAE, PSNR, and per-band RMSE. The visualization displays input and reconstructed false-color composites. Synthetic data is used only to confirm that the evaluation and visualization pipeline runs successfully; its numerical values are not reported as model performance. Real-data metrics represent masked reconstruction on the user-provided imagery.
184
 
185
+ # Official OneScience Resources
186
 
187
+ | Platform | OneScience Main Repository | Skills Repository |
188
  | --- | --- | --- |
189
  | Gitee | https://gitee.com/onescience-ai/onescience | https://gitee.com/onescience-ai/oneskills |
190
  | GitHub | https://github.com/onescience-ai/OneScience | https://github.com/onescience-ai/oneskills |
191
 
192
+ # Citation and License
193
 
194
+ - This repository is a reproduction of the original SpectralGPT paper.
195
+ - The model package is released under Apache License 2.0. The original datasets and model weights remain subject to their respective source licenses.