README.md

---
license: mit
pipeline_tag: image-to-image
tags:
- hirise
- mars
- planetary-science
- image-reconstruction
- pytorch
datasets:
- ardan-S/HiPredict
---
# HiPredict

**Paper:** preprint coming soon · **Code:** [ardan-S/HiPredict](https://github.com/ardan-S/HiPredict) · **Dataset:** [ardan-S/HiPredict](https://huggingface.co/datasets/ardan-S/HiPredict)

HiPredict reconstructs the RED4 band of the HiRISE camera on the Mars
Reconnaissance Orbiter. The RED4 detector has partially failed, and many
observations have no usable RED4 band as a result. HiPredict predicts RED4 for
those observations from the instrument's other channels.

## Model

A U-Net (2,861,331 parameters) with a brightness head. For each RED4 location it uses:

- the registered IR10 and BG12 colour bands, which image the same ground as RED4;
- 32-pixel-wide context strips from the adjacent edges of the RED3 and RED5 detectors;
- nine observation metadata values: TDI, incidence and phase angles, local time,
  solar longitude and distance, orbit number, and two instrument temperatures.

The U-Net reconstructs spatial detail, and the brightness head sets the local
level from the neighbouring RED context. Inputs are normalised with fixed
dataset-wide statistics, and predictions are returned in the same calibrated
units as the processed input cubes.

## Usage

Install the code from [GitHub](https://github.com/ardan-S/HiPredict), then:

```python
from hipredict import get_model, HiPredict

model, config = get_model("hipredict", device="cpu")
predictor = HiPredict(model, config, device="cpu")
```

`get_model` downloads this repository on first use, pinned to the revision the
installed code was validated against. To download, preprocess and predict one
observation:

```bash
uv run -m hipredict.download --obs ESP_028693_1535 --workers 2
uv run -m hipredict.process --obs ESP_028693_1535 --workers 2
uv run -m hipredict.inference.predict --obs ESP_028693_1535 --model hipredict
```

Preprocessing needs a separate HiPROC + ISIS environment; see the code
repository's `docs/preprocessing.md`.

## Training data

HiPredict was trained on the [HiPredict dataset](https://huggingface.co/datasets/ardan-S/HiPredict).
Its patch sets are split by date: the earliest 70% for training, the next 10%
for validation and the last 20% for testing. The model was trained on 81,717
sets, at a point when the record held 51 fewer patch sets than the published
version; the same split of the published record gives 81,753 training sets.
The observations were processed with HiPROC 0.11.0 and ISIS 9.0.0.

## Performance

On the held-out test split, compared with SYN4, a baseline that scales IR10 to
the brightness of RED5 (↓ lower is better, ↑ higher is better):

| Model | NMAE mean (%) ↓ | NMAE median (%) ↓ | SSIM mean ↑ | SSIM median ↑ | PWT 1% mean (%) ↑ | PWT 1% median (%) ↑ |
| --- | ---: | ---: | ---: | ---: | ---: | ---: |
| SYN4 | 3.118 | 1.545 | 0.663 | 0.722 | 11.069 | 9.148 |
| **HiPredict** | **1.110** | **0.943** | **0.725** | **0.788** | **20.844** | **18.801** |

- **NMAE**: normalised mean absolute error, the mean absolute error as a percentage of the image's mean brightness.
- **SSIM**: structural similarity index.
- **PWT 1%**: percentage of pixels whose absolute error is within 1% of the image's dynamic range.

## Limitations

- Inputs must be processed the same way as the training data, with the pipeline
  in the code repository (HiPROC 0.11.0 and ISIS 9.0.0). Cubes from other
  pipelines or versions can differ radiometrically.
- Only observations whose five detectors use binning 1 or 2 and TDI 64 or 128,
  with matching binning across the RED detectors and across the colour
  detectors, are supported.
- On the simplest images, where SYN4's error is below about 1.5%, SYN4 is as
  good as HiPredict or slightly better.
- Much of the remaining error is pixel-level noise, likely from binning
  artefacts and genuine instrument noise in the true RED4, which the model is
  not expected to reproduce.

## Files

| File | Contents |
| --- | --- |
| `model.pt` | Model weights |
| `meta_scalar.joblib` | Metadata scaler fitted on the training split |
| `config.yaml` | Model and training configuration |
| `global_stats.json` | Dataset-wide normalisation statistics |
| `provenance.json` | Source checkpoint and SHA-256 checksums, verified when the model is loaded |

Keep these files together.

## Licence and credit

The model and code are released under the MIT License. HiRISE data:
NASA/JPL/University of Arizona.

## Citation

A citation will be added here when the preprint is released.