| --- |
| license: other |
| library_name: bagel-mot |
| language: |
| - en |
| tags: |
| - geoug |
| - geometry |
| - reinforcement-learning |
| --- |
| |
| # GeoUG-Spatial |
|
|
| [Project page](https://yangr116.github.io/geoug-project-page/) · |
| [Paper](https://yangr116.github.io/geoug-project-page/assets/geoug-paper.pdf) |
|
|
| **Generate geometry when it helps, then reason with it.** |
|
|
| GeoUG-Spatial is the final adaptive spatial reasoning model from |
| *GeoUG: Learning and Reasoning with Geometry — How Generation Helps Spatial |
| Understanding?* Given an image and a grounded question, the policy can answer |
| directly or generate metric depth, query geometry tools, and revise its answer. |
|
|
| | Property | Value | |
| |---|---| |
| | Stage | Spatial-policy RL, initialized from GeoUG-ColdStart | |
| | Objective | `spatial_gate_draft_v3`, generation cost λ = 0.1 | |
| | Weights | Merged EMA spatial policy | |
| | Precision | Original BF16 EMA weights | |
| | Runtime | GeoUG custom BAGEL/MoT loader | |
|
|
| ## Download |
|
|
| ```python |
| from huggingface_hub import snapshot_download |
| |
| snapshot_download( |
| repo_id="rayruiyang/GeoUG-Spatial", |
| local_dir="ckpt/geoug-spatial-rl-lambda0.1", |
| ) |
| ``` |
|
|
| These are custom BAGEL/MoT weights. Use the GeoUG runtime rather than a generic |
| `AutoModel.from_pretrained` call. The weight repository contains the checkpoint |
| assets; inference and training code live in the GeoUG code checkout. |
|
|
| Verify the download: |
|
|
| ```bash |
| (cd ckpt/geoug-spatial-rl-lambda0.1 && sha256sum -c checksums.sha256) |
| ``` |
|
|
| ## Quick start |
|
|
| Follow the GeoUG checkout's standalone inference installation instructions |
| (Python 3.10/3.11, CUDA, and the documented FlashAttention environment). Run the |
| following commands from the checkout root after downloading the package: |
|
|
| ```bash |
| python -m inference.spatial \ |
| --model-path ckpt/geoug-spatial-rl-lambda0.1 \ |
| --image /path/to/image.jpg \ |
| --question 'Which point is closer to the camera? A: <point>120 400</point>. B: <point>730 420</point>. Answer A or B.' \ |
| --mode adaptive \ |
| --no-do-sample \ |
| --num-timesteps 50 \ |
| --cfg-text-scale 4.0 \ |
| --cfg-img-scale 2.0 \ |
| --output-dir outputs/inference/spatial |
| ``` |
|
|
| Replace the example points with references appropriate for your image. The |
| GeoUG/BAGEL point convention normalizes x and y independently to 0–1000; |
| benchmark adapters convert their canonical coordinates at runtime. |
|
|
| The CLI enables the required reasoning tokens automatically. Custom runners |
| must set `BAGEL_REASONING_TOKENS=1`. `force_depth` and `force_answer` expose the |
| two policy endpoints for inspection; use `adaptive` for the adaptive model. |
| The spatial model contains its own depth generator; downloading GeoUG-Depth |
| is not required for spatial inference. |
|
|
| ## Reported results |
|
|
| | Benchmark | Adaptive circular accuracy (%) | |
| |---|---:| |
| | 3DCueBench | 90.75 | |
| | CV-Bench-3D Textual | 97.92 | |
| | DA-2K Textual | 94.20 | |
|
|
| Mean depth-trigger rate across the three benchmarks: **63.4%**. Circular |
| accuracy credits a question only if every tested answer-option order is correct. |
|
|
| These results are reported in the paper. Full adaptive-policy reproduction |
| with the current evaluation code remains pending. |
|
|
| ## Training and evaluation resources |
|
|
| The RL reward balances answer correctness against depth-generation cost: |
| `reward = answer_correct - 0.1 * used_depth`. See `docs/RL_TRAINING.md` in the |
| GeoUG checkout for the full method and training setup. |
|
|
| - [GeoUG training data](https://huggingface.co/datasets/rayruiyang/GeoUG_data) |
| - [3DCueBench](https://huggingface.co/datasets/rayruiyang/3DCueBench) |
| - [CV-Bench-3D Textual](https://huggingface.co/datasets/rayruiyang/CVBench3DTexutal) |
| - DA-2K Textual / Visual preparation: `evaluation/vlmevalkit/README.md` |
|
|
| Generated depth and spatial answers are model predictions. Their accuracy |
| depends on the input, coordinate interpretation, and geometric calibration. |
|
|
| ## Model family |
|
|
| | Checkpoint | Intended use | |
| |---|---| |
| | [GeoUG-Spatial](https://huggingface.co/rayruiyang/GeoUG-Spatial) | Final adaptive spatial reasoning policy | |
| | [GeoUG-Depth](https://huggingface.co/rayruiyang/GeoUG-Depth) | Standalone metric-depth generation | |
| | [GeoUG-ColdStart](https://huggingface.co/rayruiyang/GeoUG-ColdStart) | SFT initialization for spatial-policy RL | |
|
|
| ## Package and integrity |
|
|
| | File | Contents | |
| |---|---| |
| | `ema.safetensors` | BF16 EMA model weights, approximately 27.21 GiB | |
| | `ae.safetensors` | VAE weights | |
| | `llm_config.json`, `vit_config.json`, `config.json`, `generation_config.json` | Model and generation configuration | |
| | `tokenizer.json`, `tokenizer_config.json`, `vocab.json`, `merges.txt` | Tokenizer assets | |
| | `transform_config.yaml` | Image/depth preprocessing and normalization | |
| | `checksums.sha256` | SHA-256 checksums for the package files | |
|
|
| Download the complete repository. The release uses EMA weights and does not |
| require a separate non-EMA checkpoint or a shard index. |
|
|
| ## License and attribution |
|
|
| GeoUG builds on the BAGEL/MoT architecture and its pretrained components. |
| This checkpoint accompanies the mixed-license GeoUG code release. Consult |
| `LICENSE` and `NOTICE` in the GeoUG code checkout for the |
| applicable component terms, including the CC BY-NC restrictions on the listed |
| BAGEL/DiT-derived code files. This card does not assign a new blanket license |
| to upstream components. The data repositories document their own source and |
| usage terms separately. |
|
|
| ## Contact |
|
|
| For questions about this release, contact |
| [rayyang0116@connect.hku.hk](mailto:rayyang0116@connect.hku.hk). |
|
|
