fatmur/gedi-biomass

GEDI biomass regression

An experiment workspace for pixelwise aboveground biomass density (AGBD, Mg/ha) prediction from satellite imagery using sparse GEDI L4A supervision. This repository contains training code, not trained weights. The default small CNN + U-Net + regression head is a baseline; it does not load IBM Granite weights.

Run

Use the development environment of the parent GeoSave checkout:

cd workspace/gedi-biomass
uv run python main.py fit --config configs/gedi-biomass.yaml
uv run python main.py test --config configs/gedi-biomass.yaml --ckpt_path /path/to/model.ckpt

The project imports the local GeoSave library. Run uv sync in the GeoSave root first. This workspace is not a standalone installable package. Its starting implementation was adapted from GeoSave's supervised regression recipe and Dynamic World's paired TIFF workflow.

Prepared data

data/gedi_biomass/
├── train/{images,labels}/<scene>.tif
├── val/{images,labels}/<scene>.tif
└── test/{images,labels}/<scene>.tif

Every image must have a matching label on exactly the same projected CRS, transform, and shape. The configured default resolution is 30 map units in a meter-based CRS. Prepare the grid explicitly before training; the Dataset rejects grid mismatches and origins off the resolution lattice to prevent implicit label resampling. A scene is read chip by chip, without retaining open raster readers.

Image variable names must match bands in the config, in the intended input order. The six default names are blue, green, red, nir, swir1, and swir2. Rename source bands during preparation. Store reflectance scale/offset and nodata metadata in the TIFF, or write decoded reflectance. Cloud/shadow-invalid imagery must already be masked as nodata. The loader decodes scale/offset without an implicit normalization recipe.

Labels contain one float variable named agbd, in Mg/ha, with NaN or an explicitly encoded nodata value for unobserved pixels. A zero target is valid biomass. Training loss and metrics use only finite targets with valid imagery. Invalid image values are zero-filled for the network and masked in targets and exported predictions. Missing labels do not mask predictions: outputs are spatially dense where imagery is valid. Entirely unlabelled chips contribute zero loss/gradient; validation needs some valid observations for checkpoint monitoring.

Split geographically before creating chips. Filter GEDI quality flags, retain acquisition dates and uncertainty in preparation metadata, and match imagery in time. Do not interpolate sparse GEDI shots into invented training labels. If multiple shots land in one pixel, choose and document an aggregation policy.

Evaluation and export

Validation and test export single-band agbd COGs with CRS, grid, nodata, and Mg/ha units into artifacts/gedi-biomass/predictions/{val,test}. Overlapping predictions are merged before writing. Validation exports are replaced each epoch. The callback requires one device and complete evaluation passes; remove it when using batch limits or distributed training. This initial recipe is validated on one device only. Metrics score chip observations, so overlapping evaluation chips repeat observations; these are not independent scene-level accuracy estimates.

Training flips imagery and sparse labels together without interpolation. No ModelSpec or publication of a portable inference artifact is implemented here. Checkpoints are local and excluded from Git; uploading trained weights is a separate release step.

Data sources and baseline reference

• Original GEDI L4A V3 footprints (HDF5): https://doi.org/10.3334/ORNLDAAC/2508
• Rasterized monthly L4A V2.1: https://developers.google.com/earth-engine/datasets/catalog/LARSE_GEDI_GEDI04_A_002_MONTHLY
• IBM Granite biomass reference: https://huggingface.co/ibm-granite/granite-geospatial-biomass

GEDI download and label preparation are available below. Imagery acquisition, location/biome split assignment, and IBM checkpoint integration remain separate steps. Label-only tiles cannot yet be passed directly to the paired-image Dataset.

Smoke tests

From the parent GeoSave repository root:

PYTHONPATH=workspace/gedi-biomass uv run pytest workspace/gedi-biomass/tests -q --confcutdir=workspace/gedi-biomass/tests

These tests use synthetic local rasters and a short CPU fit/test run, including sparse-target masking and georeferenced prediction round trips. They do not establish accuracy on real GEDI data.

Prepare GEDI labels first

Run from this workspace using the parent GeoSave environment. The example polygon near Leuser, Indonesia, is a small discovery example, not a chosen experiment ROI or a claimed biome boundary. Edit/copy it for your extraction areas and dates.

# Public discovery only; no account or file downloads.
uv run --no-sync --with-requirements requirements-ingest.txt python -m scripts.download_gedi \
  --regions configs/regions.example.geojson

# Download original HDF5 granules after authenticating with NASA Earthdata.
uv run --no-sync --with-requirements requirements-ingest.txt python -m scripts.download_gedi \
  --regions configs/regions.example.geojson --download

# Prepare labels and a location index, without retrieving imagery.
uv run --no-sync python -m scripts.ingest_gedi \
  --regions configs/regions.example.geojson --size 512 --resolution 30

Create a free Earthdata Login account. earthaccess.login() supports environment credentials (EARTHDATA_USERNAME and EARTHDATA_PASSWORD), your local .netrc, or an interactive login. Credentials are never written into the source manifests. Original granules can be large even for a small polygon. --limit rejects an oversized search instead of truncating it. The download manifest pins exact CMR granule IDs and URLs.

--gedi-version 2.1 is the default paper-era product; --gedi-version 3 selects the newer release with its own quality flags. Use separate raw/output directories for different versions. Direct sources: V2.1 and V3.

Each GeoJSON feature requires id, start, end and crs properties. Dates are start-inclusive and end-exclusive. Polygon coordinates are WGS84; the output CRS must be projected in metres. Optional biome is a user-supplied annotation, not inferred from the polygon. Use short ecological/seasonal time windows; do not average observations over years of land-cover change without an explicit reason.

data/raw/<region>/manifest.json
data/raw/<region>/gedi/*.h5

data/gedi_labels/
├── labels/<tile>.tif          # one agbd band, Mg/ha; unobserved pixels are NaN
├── shots/<tile>.parquet      # unique shot IDs, dates, raw AGBD and standard error
├── metadata/<tile>.json      # source and filtering provenance
├── catalogs/<region>.parquet # completed region tile index
├── manifests/<region>.json   # completion state and content fingerprint
└── tiles.parquet             # combined spatial index

Only tiles containing selected shots are saved. Every tile has the requested shape, including tiles at region boundaries: pixels outside observed coverage stay nodata. There is no interpolation between footprints. Multiple unique shots falling in one pixel contribute their arithmetic mean. Their individual uncertainties remain in the shot table; they are not combined under an unsupported independence assumption. Valid zero AGBD is preserved.

For V2.1, use the L2/L4 quality flags and undegraded positioning. For V3, the L4 quality flag includes a land criterion, and preparation additionally checks degrade_include_flag and the elevation outlier flag. These are biomass-quality land observations, not a forest-only mask or a guarantee of zero water/shoreline contamination. See the filter definitions.

The default 512 x 512 at 30 m spans 15.36 km per side. --size 510 is also supported (15.3 km). Neither is claimed to be IBM's original stored dataset size: its public configuration only specifies optional 224 x 224 inference crops. Storage size and later model crop size are independent.

Inspect the labels before deciding splits or fetching imagery:

import geopandas as gpd

tiles = gpd.read_parquet("data/gedi_labels/tiles.parquet")
print(tiles[["location_id", "region", "biome", "shot_count", "observed_fraction"]])
locations = tiles.groupby("location_id")

Tile geometry is the full footprint; start_datetime/end_datetime describe the selected observations. The index also stores exact CRS/transform, dimensions, resolution, and relative asset paths. This is a native GeoParquet inventory, not an automatically generated STAC Item catalog. Exact location IDs are stable for the same CRS, resolution and tile size; use one grid convention for an experiment. Nearby locations still need geographic grouping/buffers before train/test splits. Avoid overlapping extraction polygons for the same dates, which can create multiple partial records for one location. When imagery is selected later, load it onto each label's exact grid and preserve these location group assignments.

Reruns with identical inputs resume; changing source contents, dates, product or tiling settings requires a new output directory. These scripts are intended for bounded regions processed in batches: HDF5 is read in chunks, but the retained shot table for each extraction region resides in memory. They are not a one-shot 15 TB global ingest implementation.