| --- |
| license: apache-2.0 |
| base_model: facebook/VGGT-1B |
| tags: |
| - georay |
| - satellite-imagery |
| - satellite-photogrammetry |
| - remote-sensing |
| - 3d-reconstruction |
| - multi-view-stereo |
| - rational-polynomial-camera |
| - digital-surface-model |
| - pytorch |
| - research |
| arxiv: 2608.29680 |
| --- |
| |
| # GeoRay |
|
|
| **Gauge-Aware Feed-Forward Satellite 3D Reconstruction in the Geodetic Frame** |
|
|
| [Paper](https://arxiv.org/abs/2608.29680) · [Code](https://github.com/HIT-SIRS/GeoRay) · [Issues](https://github.com/HIT-SIRS/GeoRay/issues) |
|
|
| GeoRay is a feed-forward framework for reconstructing dense satellite surface height directly in the absolute geodetic frame under rational polynomial cameras (RPCs). It combines RPC height-ray reasoning, gauge-aware relief–datum separation, and uncertainty-calibrated monocular/multi-view fusion. |
|
|
| ## Model details |
|
|
| - **Architecture:** frozen VGGT-1B and MoGe-2 backbones with lightweight GeoRay adaptation, relief, datum, calibration, and uncertainty modules |
| - **Inputs:** three-view satellite image patches and forward RPC coefficients |
| - **Outputs:** absolute surface height and predictive uncertainty; optional longitude/latitude/height back-projection |
| - **Framework:** PyTorch 2.7.x |
| - **Code license:** Apache-2.0 |
| - **Checkpoint file:** `georay_ckpt.pt` |
| - **Checkpoint SHA-256:** `7bbe1fc5f04d471d1ecd843be94b73571d8e6aded9a407965277cf37a1a3f330` |
|
|
| The released checkpoint contains GeoRay's trainable modules. Frozen backbone weights are obtained separately and remain subject to their upstream licenses. |
|
|
| ## Results |
|
|
| On 26 held-out US3D tiles, GeoRay reports: |
|
|
| | Metric | Result | |
| |:--|--:| |
| | Absolute MAE ↓ | **2.99 m** | |
| | Coverage ↑ | **91.9%** | |
| | Completeness-aware accuracy gain ↑ | **+46.4 points** over the strongest compliant feed-forward baseline | |
| | Model-forward time ↓ | **24.0 s / tile** in the reported setting | |
|
|
| See the [paper](https://arxiv.org/abs/2608.29680) for the complete evaluation protocol, cross-dataset and cross-city results, and baseline definitions. |
|
|
| ## Installation |
|
|
| ```bash |
| git clone https://github.com/HIT-SIRS/GeoRay.git |
| cd GeoRay |
| conda create -n georay python=3.11 -y |
| conda activate georay |
| |
| # Install the CUDA-enabled PyTorch 2.7.x build appropriate for your system first. |
| pip install -r requirements.txt |
| pip install -e . |
| ``` |
|
|
| Follow the repository [installation guide](https://github.com/HIT-SIRS/GeoRay#installation) to prepare the pinned VGGT code and external frozen weights. |
|
|
| ## Download |
|
|
| After this model repository is published, download the checkpoint with: |
|
|
| ```bash |
| hf download hit-sirs/GeoRay georay_ckpt.pt --local-dir checkpoints/GeoRay |
| export GEORAY_CKPT="$PWD/checkpoints/GeoRay/georay_ckpt.pt" |
| ``` |
|
|
| ## Inference |
|
|
| ```bash |
| python scripts/infer.py \ |
| --ckpt "$GEORAY_CKPT" \ |
| --config configs/train.yaml \ |
| --input /path/to/tile.tar \ |
| --out_dir outputs/tile |
| |
| python scripts/export_dsm.py \ |
| --infer_out outputs/tile \ |
| --out_dir outputs/tile \ |
| --input /path/to/tile.tar |
| ``` |
|
|
| The input WebDataset format and RPC requirements are documented in the [GeoRay README](https://github.com/HIT-SIRS/GeoRay#data-format). |
|
|
| ## Intended use |
|
|
| GeoRay is intended for research on satellite multi-view 3D reconstruction, absolute-frame surface-height estimation, uncertainty-aware photogrammetry, and related geospatial studies. |
|
|
| ## Limitations |
|
|
| - The released model expects the preprocessing and three-view WebDataset structure documented by the project. |
| - Accuracy is not guaranteed for unseen sensors, geographic regions, RPC conventions, or height datums; validate outputs before downstream use. |
| - The exported product is a predicted digital surface model, not a surveyed digital terrain model. |
| - Predictive uncertainty is a model estimate and must not be treated as a formal safety guarantee. |
| - Inference requires an NVIDIA GPU and separately obtained frozen backbone weights. |
|
|
| ## Third-party components |
|
|
| GeoRay source code and its lightweight checkpoint are released under Apache-2.0. VGGT-1B, MoGe-2, and other third-party components are governed by their own licenses. In particular, users must review the applicable VGGT weight license before use or redistribution. |
|
|
| ## Citation |
|
|
| ```bibtex |
| @article{dong2026georay, |
| title = {GeoRay: Gauge-Aware Feed-Forward Satellite 3D Reconstruction in the Geodetic Frame}, |
| author = {Dong, Zhe and Wu, Wanqing and Sun, Yuzhe and Jiang, Haochen and Ma, Yuchen and Ren, Lecheng and Liu, Tianzhu and Gu, Yanfeng}, |
| journal = {arXiv preprint arXiv:2608.29680}, |
| year = {2026} |
| } |
| ``` |
|
|
|
|
