GeoUG-Depth

Project page · Paper

Metric depth from a single RGB image.

GeoUG-Depth is the standalone metric-depth generation expert released with GeoUG. It generates depth in meters for geometric measurement and downstream 3D reasoning. It is a separate checkpoint from the adaptive spatial RL policy.

Property	Value
Task	Monocular metric-depth generation
Output	Depth array in meters and a visualization
Weights	EMA metric-depth expert
Precision	BF16 release converted from the FP32 source EMA
Normalization	Multiscale disparity, c: [2.0, 8.0, 32.0]

Download

from huggingface_hub import snapshot_download

snapshot_download(
    repo_id="rayruiyang/GeoUG-Depth",
    local_dir="ckpt/geoug-metric-depth",
)

These are custom BAGEL/MoT weights. Use the GeoUG runtime rather than a generic AutoModel.from_pretrained call. The weight repository contains the checkpoint assets; inference and training code live in the GeoUG code checkout.

Verify the download:

(cd ckpt/geoug-metric-depth && sha256sum -c checksums.sha256)

Quick start

Follow the GeoUG checkout's standalone inference installation instructions (Python 3.10/3.11, CUDA, and the documented FlashAttention environment). From the checkout root, after downloading the package:

python -m inference.depth \
  --model-path ckpt/geoug-metric-depth \
  --image /path/to/image.jpg \
  --num-timesteps 50 \
  --cfg-text-scale 4.0 \
  --cfg-img-scale 1.5 \
  --output-dir outputs/inference/depth

The loader selects the package's transform_config.yaml and EMA weights by default. The default prompt is:

<GENERATE_DEPTH>Estimate the metric depth map.</GENERATE_DEPTH>

Use the .npy output for metric depth. The PNG output is a percentile-normalized visualization and does not encode metric depth values. The source FP32 weights were preserved; this repository distributes the BF16 inference copy.

Reported results and evaluation

The project reports the following affine-aligned results:

Dataset	AbsRel (%) ↓	δ1 (%) ↑
NYU-v2	3.32	98.46
KITTI	4.84	97.27
ETH3D	3.75	97.85

These aligned metrics should not be interpreted as unaligned metric-scale accuracy. Keep alignment, image preprocessing, and benchmark configuration fixed when comparing results.

For evaluation, obtain the benchmark data separately, configure local dataset paths, and run from the GeoUG checkout:

python -m evaluation.depth.eval_depth \
  --model-path ckpt/geoug-metric-depth \
  --dataset-config /path/to/depth_datasets.json \
  --depth-type metric

See evaluation/depth/README.md for dataset configuration and scoring protocols. Generated depth is an estimate; geometry recovered from it also depends on camera calibration and the scene distribution.

Model family

Checkpoint	Intended use
GeoUG-Spatial	Final adaptive spatial reasoning policy
GeoUG-Depth	Standalone metric-depth generation
GeoUG-ColdStart	SFT initialization for spatial-policy RL

Package and integrity

File	Contents
ema.safetensors	BF16 EMA model weights, approximately 27.21 GiB
ae.safetensors	VAE weights
llm_config.json, vit_config.json, config.json, generation_config.json	Model and generation configuration
tokenizer.json, tokenizer_config.json, vocab.json, merges.txt	Tokenizer assets
transform_config.yaml	Image/depth preprocessing and normalization
checksums.sha256	SHA-256 checksums for the package files

Download the complete repository. The release uses EMA weights and does not require a separate non-EMA checkpoint or a shard index.

Checkpoint provenance

The release is the metric-depth expert EMA checkpoint at training step 6000. The published weights are the BF16 conversion of the original FP32 EMA.

License and attribution

GeoUG builds on the BAGEL/MoT architecture and its pretrained components. This checkpoint accompanies the mixed-license GeoUG code release. Consult LICENSE and THIRD_PARTY_NOTICES.md in the GeoUG code checkout for the applicable component terms, including the CC BY-NC restrictions on the listed BAGEL/DiT-derived code files. This card does not assign a new blanket license to upstream components. The data repositories document their own source and usage terms separately.

Contact

For questions about this release, contact rayyang0116@connect.hku.hk.