| --- |
| language: |
| - en |
| library_name: bagel-mot |
| license: other |
| pipeline_tag: depth-estimation |
| tags: |
| - geoug |
| - geometry |
| - depth-estimation |
| --- |
| |
| # GeoUG-Depth |
|
|
| [Project page](https://yangr116.github.io/geoug-project-page/) · |
| [Paper](https://yangr116.github.io/geoug-project-page/assets/geoug-paper.pdf) |
|
|
| **Metric depth from a single RGB image.** |
|
|
| GeoUG-Depth is the standalone metric-depth generation expert released with |
| GeoUG. It generates depth in meters for geometric measurement and downstream |
| 3D reasoning. It is a separate checkpoint from the adaptive spatial RL policy. |
|
|
| | Property | Value | |
| |---|---| |
| | Task | Monocular metric-depth generation | |
| | Output | Depth array in meters and a visualization | |
| | Weights | EMA metric-depth expert | |
| | Precision | BF16 release converted from the FP32 source EMA | |
| | Normalization | Multiscale disparity, `c: [2.0, 8.0, 32.0]` | |
|
|
| ## Download |
|
|
| ```python |
| from huggingface_hub import snapshot_download |
| |
| snapshot_download( |
| repo_id="rayruiyang/GeoUG-Depth", |
| local_dir="ckpt/geoug-metric-depth", |
| ) |
| ``` |
|
|
| These are custom BAGEL/MoT weights. Use the GeoUG runtime rather than a generic |
| `AutoModel.from_pretrained` call. The weight repository contains the checkpoint |
| assets; inference and training code live in the GeoUG code checkout. |
|
|
| Verify the download: |
|
|
| ```bash |
| (cd ckpt/geoug-metric-depth && sha256sum -c checksums.sha256) |
| ``` |
|
|
| ## Quick start |
|
|
| Follow the GeoUG checkout's standalone inference installation instructions |
| (Python 3.10/3.11, CUDA, and the documented FlashAttention environment). From |
| the checkout root, after downloading the package: |
|
|
| ```bash |
| python -m inference.depth \ |
| --model-path ckpt/geoug-metric-depth \ |
| --image /path/to/image.jpg \ |
| --num-timesteps 50 \ |
| --cfg-text-scale 4.0 \ |
| --cfg-img-scale 1.5 \ |
| --output-dir outputs/inference/depth |
| ``` |
|
|
| The loader selects the package's `transform_config.yaml` and EMA weights by |
| default. The default prompt is: |
|
|
| ```text |
| <GENERATE_DEPTH>Estimate the metric depth map.</GENERATE_DEPTH> |
| ``` |
|
|
| Use the `.npy` output for metric depth. The PNG output is a percentile-normalized |
| visualization and does not encode metric depth values. The source FP32 weights |
| were preserved; this repository distributes the BF16 inference copy. |
|
|
| ## Reported results and evaluation |
|
|
| The project reports the following **affine-aligned** results: |
|
|
| | Dataset | AbsRel (%) ↓ | δ1 (%) ↑ | |
| |---|---:|---:| |
| | NYU-v2 | 3.32 | 98.46 | |
| | KITTI | 4.84 | 97.27 | |
| | ETH3D | 3.75 | 97.85 | |
|
|
| These aligned metrics should not be interpreted as unaligned metric-scale |
| accuracy. Keep alignment, image preprocessing, and benchmark configuration |
| fixed when comparing results. |
|
|
| For evaluation, obtain the benchmark data separately, configure local dataset |
| paths, and run from the GeoUG checkout: |
|
|
| ```bash |
| python -m evaluation.depth.eval_depth \ |
| --model-path ckpt/geoug-metric-depth \ |
| --dataset-config /path/to/depth_datasets.json \ |
| --depth-type metric |
| ``` |
|
|
| See `evaluation/depth/README.md` for dataset configuration and scoring protocols. |
| Generated depth is an estimate; geometry recovered from it also depends on |
| camera calibration and the scene distribution. |
|
|
| ## Model family |
|
|
| | Checkpoint | Intended use | |
| |---|---| |
| | [GeoUG-Spatial](https://huggingface.co/rayruiyang/GeoUG-Spatial) | Final adaptive spatial reasoning policy | |
| | [GeoUG-Depth](https://huggingface.co/rayruiyang/GeoUG-Depth) | Standalone metric-depth generation | |
| | [GeoUG-ColdStart](https://huggingface.co/rayruiyang/GeoUG-ColdStart) | SFT initialization for spatial-policy RL | |
|
|
| ## Package and integrity |
|
|
| | File | Contents | |
| |---|---| |
| | `ema.safetensors` | BF16 EMA model weights, approximately 27.21 GiB | |
| | `ae.safetensors` | VAE weights | |
| | `llm_config.json`, `vit_config.json`, `config.json`, `generation_config.json` | Model and generation configuration | |
| | `tokenizer.json`, `tokenizer_config.json`, `vocab.json`, `merges.txt` | Tokenizer assets | |
| | `transform_config.yaml` | Image/depth preprocessing and normalization | |
| | `checksums.sha256` | SHA-256 checksums for the package files | |
|
|
| Download the complete repository. The release uses EMA weights and does not |
| require a separate non-EMA checkpoint or a shard index. |
|
|
| ## Checkpoint provenance |
|
|
| The release is the metric-depth expert EMA checkpoint at training step 6000. |
| The published weights are the BF16 conversion of the original FP32 EMA. |
|
|
| ## License and attribution |
|
|
| GeoUG builds on the BAGEL/MoT architecture and its pretrained components. |
| This checkpoint accompanies the mixed-license GeoUG code release. Consult |
| `LICENSE` and `THIRD_PARTY_NOTICES.md` in the GeoUG code checkout for the |
| applicable component terms, including the CC BY-NC restrictions on the listed |
| BAGEL/DiT-derived code files. This card does not assign a new blanket license |
| to upstream components. The data repositories document their own source and |
| usage terms separately. |
|
|
| ## Contact |
|
|
| For questions about this release, contact |
| [rayyang0116@connect.hku.hk](mailto:rayyang0116@connect.hku.hk). |
