File size: 4,513 Bytes
82449fe | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 | ---
license: apache-2.0
base_model: facebook/VGGT-1B
tags:
- georay
- satellite-imagery
- satellite-photogrammetry
- remote-sensing
- 3d-reconstruction
- multi-view-stereo
- rational-polynomial-camera
- digital-surface-model
- pytorch
- research
arxiv: 2608.29680
---
# GeoRay
**Gauge-Aware Feed-Forward Satellite 3D Reconstruction in the Geodetic Frame**
[Paper](https://arxiv.org/abs/2608.29680) · [Code](https://github.com/HIT-SIRS/GeoRay) · [Issues](https://github.com/HIT-SIRS/GeoRay/issues)
GeoRay is a feed-forward framework for reconstructing dense satellite surface height directly in the absolute geodetic frame under rational polynomial cameras (RPCs). It combines RPC height-ray reasoning, gauge-aware relief–datum separation, and uncertainty-calibrated monocular/multi-view fusion.
## Model details
- **Architecture:** frozen VGGT-1B and MoGe-2 backbones with lightweight GeoRay adaptation, relief, datum, calibration, and uncertainty modules
- **Inputs:** three-view satellite image patches and forward RPC coefficients
- **Outputs:** absolute surface height and predictive uncertainty; optional longitude/latitude/height back-projection
- **Framework:** PyTorch 2.7.x
- **Code license:** Apache-2.0
- **Checkpoint file:** `georay_ckpt.pt`
- **Checkpoint SHA-256:** `7bbe1fc5f04d471d1ecd843be94b73571d8e6aded9a407965277cf37a1a3f330`
The released checkpoint contains GeoRay's trainable modules. Frozen backbone weights are obtained separately and remain subject to their upstream licenses.
## Results
On 26 held-out US3D tiles, GeoRay reports:
| Metric | Result |
|:--|--:|
| Absolute MAE ↓ | **2.99 m** |
| Coverage ↑ | **91.9%** |
| Completeness-aware accuracy gain ↑ | **+46.4 points** over the strongest compliant feed-forward baseline |
| Model-forward time ↓ | **24.0 s / tile** in the reported setting |
See the [paper](https://arxiv.org/abs/2608.29680) for the complete evaluation protocol, cross-dataset and cross-city results, and baseline definitions.
## Installation
```bash
git clone https://github.com/HIT-SIRS/GeoRay.git
cd GeoRay
conda create -n georay python=3.11 -y
conda activate georay
# Install the CUDA-enabled PyTorch 2.7.x build appropriate for your system first.
pip install -r requirements.txt
pip install -e .
```
Follow the repository [installation guide](https://github.com/HIT-SIRS/GeoRay#installation) to prepare the pinned VGGT code and external frozen weights.
## Download
After this model repository is published, download the checkpoint with:
```bash
hf download hit-sirs/GeoRay georay_ckpt.pt --local-dir checkpoints/GeoRay
export GEORAY_CKPT="$PWD/checkpoints/GeoRay/georay_ckpt.pt"
```
## Inference
```bash
python scripts/infer.py \
--ckpt "$GEORAY_CKPT" \
--config configs/train.yaml \
--input /path/to/tile.tar \
--out_dir outputs/tile
python scripts/export_dsm.py \
--infer_out outputs/tile \
--out_dir outputs/tile \
--input /path/to/tile.tar
```
The input WebDataset format and RPC requirements are documented in the [GeoRay README](https://github.com/HIT-SIRS/GeoRay#data-format).
## Intended use
GeoRay is intended for research on satellite multi-view 3D reconstruction, absolute-frame surface-height estimation, uncertainty-aware photogrammetry, and related geospatial studies.
## Limitations
- The released model expects the preprocessing and three-view WebDataset structure documented by the project.
- Accuracy is not guaranteed for unseen sensors, geographic regions, RPC conventions, or height datums; validate outputs before downstream use.
- The exported product is a predicted digital surface model, not a surveyed digital terrain model.
- Predictive uncertainty is a model estimate and must not be treated as a formal safety guarantee.
- Inference requires an NVIDIA GPU and separately obtained frozen backbone weights.
## Third-party components
GeoRay source code and its lightweight checkpoint are released under Apache-2.0. VGGT-1B, MoGe-2, and other third-party components are governed by their own licenses. In particular, users must review the applicable VGGT weight license before use or redistribution.
## Citation
```bibtex
@article{dong2026georay,
title = {GeoRay: Gauge-Aware Feed-Forward Satellite 3D Reconstruction in the Geodetic Frame},
author = {Dong, Zhe and Wu, Wanqing and Sun, Yuzhe and Jiang, Haochen and Ma, Yuchen and Ren, Lecheng and Liu, Tianzhu and Gu, Yanfeng},
journal = {arXiv preprint arXiv:2608.29680},
year = {2026}
}
```
|