README / README.md
blumenstiel's picture
Update README.md
481afbc verified
|
Raw History Blame Contribute Delete
2.44 kB
---
title: README
emoji: 🌍
colorFrom: blue
colorTo: green
sdk: static
pinned: false
---
# TerraVerse
**Multimodal generative models for Earth observation simulation and analysis.**
TerraVerse is a generative foundation model that simulates what a satellite records at
the top of the atmosphere. Given a surface reflectance image (Level-2A), the acquisition
geometry and the atmospheric state, it generates the corresponding top-of-atmosphere
observation (Level-1C). Input and output are described by band centre wavelengths and
widths, so the same model works for arbitrary optical instruments, from three-band RGB
cameras to hyperspectral sensors with more than 200 bands.
```
L2A surface reflectance + atmosphere and geometry -> L1C top-of-atmosphere
(any band set) (AOT, water vapour, angles) (any requested band set)
```
## Why
New missions need realistic simulated data before launch, and existing missions often
have sparse coverage. TerraVerse learns the atmospheric and sensor forward model from
real paired observations and can then generate data for instruments with few or no
acquisitions. The two mission scenarios of the project are:
- **Φsat-2** (MIS1): very-high-resolution multispectral imagery, simulated from Sentinel-2.
- **CHIME** (MIS2): the upcoming Copernicus hyperspectral mission, simulated before operations.
## Approach
- **Data:** about 2 M paired EnMAP and EMIT L2A / L1C patches with acquisition metadata, based on
SpectralEarth-MM.
- **Instrument simulation:** during training, every sample is shown as the view of an
instrument drawn from a pool of real (Sentinel-2, Landsat 8/9, PlanetScope, SPOT-6,
WorldView-3, MODIS, VIIRS, CHIME, ...) and randomly generated instruments.
- **Model:** a FLUX-style diffusion transformer trained as a stochastic bridge from the
L2A image to the L1C image, directly in pixel and band space, with a wavelength-conditioned
spectral front end.
## Repositories
| Repository | Content |
|---|---|
| [TerraVerse-0.1-base](https://huggingface.co/ESA-TerraVerse/TerraVerse-0.1-base) | Beta model weights and model card |
| [TerraVerse-Inference](https://huggingface.co/ESA-TerraVerse/TerraVerse-Inference) | Inference code, band presets for common sensors, example notebook |
## Consortium
Developed under ESA contract 4000150135/25/I-DT by **ESA Φ-lab**,
**DLR** (German Aerospace Center), **IBM Research Europe** and **VITO**.