Update English model card
Browse files
README.md
CHANGED
|
@@ -1,19 +1,19 @@
|
|
| 1 |
---
|
| 2 |
-
|
| 3 |
language:
|
| 4 |
- en
|
| 5 |
-
license: apache-2.0
|
| 6 |
tags:
|
| 7 |
- OneScience
|
| 8 |
- Earth Science
|
| 9 |
-
- Remote Sensing
|
| 10 |
-
- Multi-
|
| 11 |
-
|
| 12 |
-
- arxiv:2211.15660
|
| 13 |
-
tasks: []
|
| 14 |
datasets:
|
| 15 |
-
-
|
|
|
|
|
|
|
| 16 |
---
|
|
|
|
| 17 |
<p align="center">
|
| 18 |
<strong>
|
| 19 |
<span style="font-size: 30px;">SatlasPretrain</span>
|
|
@@ -22,50 +22,48 @@ datasets:
|
|
| 22 |
|
| 23 |
# Model Introduction
|
| 24 |
|
| 25 |
-
SatlasPretrain is a
|
| 26 |
-
|
| 27 |
-
Paper: SatlasPretrain: A Large-Scale Dataset for Remote Sensing Image Understanding
|
| 28 |
|
|
|
|
| 29 |
https://arxiv.org/abs/2211.15660
|
| 30 |
|
| 31 |
# Model Description
|
| 32 |
|
| 33 |
-
SatlasPretrain was proposed by the Allen Institute for AI
|
| 34 |
-
The model is suitable for multi-temporal remote sensing segmentation, regression, detection, property prediction, and classification tasks.
|
| 35 |
|
| 36 |
# Use Cases
|
| 37 |
|
| 38 |
| Scenario | Description |
|
| 39 |
| :---: | :--- |
|
| 40 |
-
|
|
| 41 |
-
|
|
| 42 |
-
|
|
| 43 |
-
|
|
|
|
|
|
|
|
| 44 |
|
| 45 |
# Usage Guide
|
| 46 |
|
| 47 |
-
## 1. OneCode
|
| 48 |
|
| 49 |
Experience intelligent one-click AI4S programming through the OneCode online environment:
|
| 50 |
|
| 51 |
[Click to Experience Intelligent One-Click AI4S Programming](https://web-2069360198568017922-iaaj.ksai.scnet.cn:58043/home)
|
| 52 |
|
| 53 |
-
## 2.
|
| 54 |
-
|
| 55 |
-
**Hardware Requirements**
|
| 56 |
-
|
| 57 |
-
- A GPU or DCU is recommended.
|
| 58 |
-
- CPU can be used for import and small-scale connectivity verification; full training and inference will be slow.
|
| 59 |
-
- DCU users must install DTK in advance. DTK 25.04.2 or above, or the OneScience recommended version matching the cluster, is recommended.
|
| 60 |
-
|
| 61 |
-
### Download the Model Package
|
| 62 |
|
| 63 |
```bash
|
| 64 |
hf download OneScience-Group/SatlasPretrain --local-dir ./SatlasPretrain
|
| 65 |
cd SatlasPretrain
|
| 66 |
```
|
| 67 |
|
| 68 |
-
###
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 69 |
|
| 70 |
**DCU Environment**
|
| 71 |
|
|
@@ -89,49 +87,55 @@ pip install onescience[earth-gpu] -i http://mirrors.onescience.ai:3141/pypi/simp
|
|
| 89 |
|
| 90 |
### Training Data Introduction
|
| 91 |
|
| 92 |
-
|
| 93 |
|
| 94 |
-
|
| 95 |
|
| 96 |
-
|
| 97 |
-
- https://huggingface.co/datasets/allenai/satlas-pretrain
|
| 98 |
|
| 99 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 100 |
|
| 101 |
-
|
| 102 |
|
| 103 |
```bash
|
| 104 |
python scripts/fake_data.py
|
| 105 |
```
|
| 106 |
|
| 107 |
-
The generated `data/fake_satlaspretrain.npz` contains four samples with `images` in `[B, T, C, H, W]` layout and labels for all seven task heads. Verify that `conf/config.yaml` points to this file before training.
|
| 108 |
-
|
| 109 |
### Training
|
| 110 |
|
| 111 |
-
Single GPU or CPU:
|
| 112 |
-
|
| 113 |
```bash
|
| 114 |
python scripts/train.py
|
| 115 |
```
|
| 116 |
|
| 117 |
-
|
| 118 |
|
| 119 |
```bash
|
| 120 |
-
torchrun --nproc_per_node=8
|
| 121 |
```
|
| 122 |
|
| 123 |
-
Training
|
| 124 |
|
| 125 |
```text
|
| 126 |
-
|
| 127 |
-
|
| 128 |
```
|
| 129 |
|
| 130 |
-
`best.pth` stores the checkpoint with the lowest joint multi-task validation loss, while `last.pth` stores the final training epoch. Checkpoints generated from synthetic data validate the pipeline only and are not pretrained weights reproducing the paper results.
|
| 131 |
-
|
| 132 |
### Training Weights
|
| 133 |
|
| 134 |
-
This repository
|
| 135 |
|
| 136 |
### Inference
|
| 137 |
|
|
@@ -139,35 +143,25 @@ This repository does not include synthetic, trained, or official pretrained weig
|
|
| 139 |
python scripts/inference.py
|
| 140 |
```
|
| 141 |
|
| 142 |
-
|
| 143 |
|
| 144 |
```text
|
| 145 |
-
result/output/
|
| 146 |
-
result/output/regression.npy
|
| 147 |
-
result/output/point.npy
|
| 148 |
-
result/output/polygon.npy
|
| 149 |
-
result/output/polyline.npy
|
| 150 |
-
result/output/property.npy
|
| 151 |
-
result/output/classification.npy
|
| 152 |
```
|
| 153 |
|
| 154 |
-
These files store class probabilities, dense regression values, confidence maps, and category probabilities for the seven compact task heads.
|
| 155 |
-
|
| 156 |
### Evaluation and Visualization
|
| 157 |
|
| 158 |
```bash
|
| 159 |
python scripts/result.py
|
| 160 |
```
|
| 161 |
|
| 162 |
-
Evaluation
|
| 163 |
|
| 164 |
```text
|
| 165 |
-
result/
|
| 166 |
-
result/
|
| 167 |
```
|
| 168 |
|
| 169 |
-
The metrics follow the task families reported in the paper: segmentation F1, regression MAE, point mAP, polygon mAP, polyline GEO, property accuracy, and classification accuracy. The qualitative comparison displays input imagery and paired ground-truth/prediction panels. Metrics and figures produced from synthetic data are pipeline checks, not paper results.
|
| 170 |
-
|
| 171 |
# Official OneScience Resources
|
| 172 |
|
| 173 |
| Platform | OneScience Main Repository | Skills Repository |
|
|
@@ -177,5 +171,6 @@ The metrics follow the task families reported in the paper: segmentation F1, reg
|
|
| 177 |
|
| 178 |
# Citation and License
|
| 179 |
|
| 180 |
-
|
| 181 |
-
|
|
|
|
|
|
| 1 |
---
|
| 2 |
+
license: apache-2.0
|
| 3 |
language:
|
| 4 |
- en
|
|
|
|
| 5 |
tags:
|
| 6 |
- OneScience
|
| 7 |
- Earth Science
|
| 8 |
+
- Remote Sensing Multi-Task Learning
|
| 9 |
+
- Multi-Temporal Remote Sensing
|
| 10 |
+
frameworks: PyTorch
|
|
|
|
|
|
|
| 11 |
datasets:
|
| 12 |
+
- SatlasPretrain
|
| 13 |
+
- NAIP
|
| 14 |
+
- Sentinel-2
|
| 15 |
---
|
| 16 |
+
|
| 17 |
<p align="center">
|
| 18 |
<strong>
|
| 19 |
<span style="font-size: 30px;">SatlasPretrain</span>
|
|
|
|
| 22 |
|
| 23 |
# Model Introduction
|
| 24 |
|
| 25 |
+
SatlasPretrain is a multi-task pretraining model for large-scale remote sensing understanding. It uses a dual-stream hierarchical backbone to process multi-temporal high-resolution RGB and Sentinel-2 multispectral imagery and jointly learns dense prediction and global classification tasks.
|
|
|
|
|
|
|
| 26 |
|
| 27 |
+
Paper: SatlasPretrain: A Large-Scale Dataset for Remote Sensing Image Understanding
|
| 28 |
https://arxiv.org/abs/2211.15660
|
| 29 |
|
| 30 |
# Model Description
|
| 31 |
|
| 32 |
+
SatlasPretrain was proposed by a research team from the Allen Institute for AI. The model is trained with NAIP-style high-resolution imagery, Sentinel-2 multispectral imagery, and multiple types of remote sensing labels from SatlasPretrain. It is suitable for remote sensing multi-task learning, including semantic segmentation, regression, point, polyline, and polygon object prediction, property recognition, and scene classification.
|
|
|
|
| 33 |
|
| 34 |
# Use Cases
|
| 35 |
|
| 36 |
| Scenario | Description |
|
| 37 |
| :---: | :--- |
|
| 38 |
+
| Multi-temporal remote sensing fusion | Jointly process 4 time steps of high-resolution RGB and 8 time steps of Sentinel-2 data. |
|
| 39 |
+
| Remote sensing multi-task learning | Simultaneously predict segmentation, regression, point, polygon, polyline, property, and classification tasks. |
|
| 40 |
+
| Land-cover and crop mapping | Use multi-temporal imagery for semantic segmentation tasks such as land cover and crop type mapping. |
|
| 41 |
+
| Infrastructure feature extraction | Identify point, polyline, and polygon geographic objects such as buildings, roads, railways, airports, and energy facilities. |
|
| 42 |
+
| Local engineering validation | Use a small amount of synthetic data to check the training, inference, and evaluation workflows. |
|
| 43 |
+
| Multi-GPU training | Launch distributed training with `torchrun`. |
|
| 44 |
|
| 45 |
# Usage Guide
|
| 46 |
|
| 47 |
+
## 1. OneCode
|
| 48 |
|
| 49 |
Experience intelligent one-click AI4S programming through the OneCode online environment:
|
| 50 |
|
| 51 |
[Click to Experience Intelligent One-Click AI4S Programming](https://web-2069360198568017922-iaaj.ksai.scnet.cn:58043/home)
|
| 52 |
|
| 53 |
+
## 2. Download and Installation
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 54 |
|
| 55 |
```bash
|
| 56 |
hf download OneScience-Group/SatlasPretrain --local-dir ./SatlasPretrain
|
| 57 |
cd SatlasPretrain
|
| 58 |
```
|
| 59 |
|
| 60 |
+
### Environment Dependencies
|
| 61 |
+
|
| 62 |
+
**Hardware Requirements**
|
| 63 |
+
|
| 64 |
+
- A GPU or DCU is recommended.
|
| 65 |
+
- CPU can be used for small-configuration connectivity validation; full training and inference will be slow.
|
| 66 |
+
- DCU users must install DTK in advance. DTK 25.04.2 or above, or the OneScience recommended version matching the current cluster, is recommended.
|
| 67 |
|
| 68 |
**DCU Environment**
|
| 69 |
|
|
|
|
| 87 |
|
| 88 |
### Training Data Introduction
|
| 89 |
|
| 90 |
+
By default, 1 training and 1 test synthetic sample are used to validate the engineering workflow. They are saved as `data/train.npz` and `data/test.npz`, respectively.
|
| 91 |
|
| 92 |
+
The synthetic data preserves the official multi-temporal configuration of 4 NAIP time steps, 8 Sentinel-2 time steps, a `512x512` spatial size, and seven task-label specifications.
|
| 93 |
|
| 94 |
+
Real data must be preprocessed and converted to the following NPZ training protocol. This protocol is consistent with the model input specification but is not the download format of the original datasets.
|
|
|
|
| 95 |
|
| 96 |
+
```text
|
| 97 |
+
highres_images: float32 [N,4,3,512,512]
|
| 98 |
+
lowres_images: float32 [N,8,9,512,512]
|
| 99 |
+
valid_highres_times: bool [N,4]
|
| 100 |
+
valid_lowres_times: bool [N,8]
|
| 101 |
+
sample_ids: string [N]
|
| 102 |
+
segmentation: int64 [N,512,512]
|
| 103 |
+
regression: float32 [N,1,512,512]
|
| 104 |
+
point: float32 [N,1,512,512]
|
| 105 |
+
polygon: float32 [N,1,512,512]
|
| 106 |
+
polyline: float32 [N,1,512,512]
|
| 107 |
+
property: int64 [N]
|
| 108 |
+
classification: int64 [N]
|
| 109 |
+
```
|
| 110 |
|
| 111 |
+
`fake_data.py` automatically writes the `protocol` and `source` protocol metadata. These fields must be retained when using real data.
|
| 112 |
|
| 113 |
```bash
|
| 114 |
python scripts/fake_data.py
|
| 115 |
```
|
| 116 |
|
|
|
|
|
|
|
| 117 |
### Training
|
| 118 |
|
|
|
|
|
|
|
| 119 |
```bash
|
| 120 |
python scripts/train.py
|
| 121 |
```
|
| 122 |
|
| 123 |
+
For multi-GPU training, use:
|
| 124 |
|
| 125 |
```bash
|
| 126 |
+
torchrun --nproc_per_node=8 scripts/train.py
|
| 127 |
```
|
| 128 |
|
| 129 |
+
Training jointly optimizes seven remote sensing task types and saves a checkpoint and aggregate training metrics. The default configuration is intended for quick workflow validation. Formal experiments should use the multi-temporal data, complete task labels, model configuration, and training duration corresponding to the paper.
|
| 130 |
|
| 131 |
```text
|
| 132 |
+
result/checkpoints/satlaspretrain.pt
|
| 133 |
+
result/training/metrics.json
|
| 134 |
```
|
| 135 |
|
|
|
|
|
|
|
| 136 |
### Training Weights
|
| 137 |
|
| 138 |
+
This repository will provide SatlasPretrain training weights in the `weight/` folder. The weight files will be uploaded soon and are expected to be available in the near future.
|
| 139 |
|
| 140 |
### Inference
|
| 141 |
|
|
|
|
| 143 |
python scripts/inference.py
|
| 144 |
```
|
| 145 |
|
| 146 |
+
Inference loads the training checkpoint, generates predictions for seven task types, retains sample identity and protocol metadata, and saves the results to:
|
| 147 |
|
| 148 |
```text
|
| 149 |
+
result/output/predictions.npz
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 150 |
```
|
| 151 |
|
|
|
|
|
|
|
| 152 |
### Evaluation and Visualization
|
| 153 |
|
| 154 |
```bash
|
| 155 |
python scripts/result.py
|
| 156 |
```
|
| 157 |
|
| 158 |
+
Evaluation covers segmentation, regression, point, polyline, and polygon objects, property, and classification tasks, and generates a multi-task prediction figure. Results on synthetic data are only for engineering workflow validation and do not represent full-paper performance.
|
| 159 |
|
| 160 |
```text
|
| 161 |
+
result/evaluation/metrics.json
|
| 162 |
+
result/evaluation/multitask_predictions.png
|
| 163 |
```
|
| 164 |
|
|
|
|
|
|
|
| 165 |
# Official OneScience Resources
|
| 166 |
|
| 167 |
| Platform | OneScience Main Repository | Skills Repository |
|
|
|
|
| 171 |
|
| 172 |
# Citation and License
|
| 173 |
|
| 174 |
+
This repository is a reproduction of the original SatlasPretrain paper.
|
| 175 |
+
|
| 176 |
+
Use of the code and data in this repository remains subject to the licenses and terms of use of their respective projects.
|