| --- |
| base_model: Wan-AI/Wan2.2-TI2V-5B |
| library_name: pytorch |
| license: mit |
| pipeline_tag: any-to-any |
| tags: |
| - 4d |
| - video-generation |
| - depth-estimation |
| - camera-pose-estimation |
| - novel-view-synthesis |
| - world-model |
| --- |
| |
| # UGG |
|
|
| This model was presented in the paper [UGG: Unified Generative Geometry Model](https://huggingface.co/papers/2611.14440). |
| Project page: https://ugg-model.github.io/ |
|
|
| UGG is a video diffusion model that generates RGB, per-frame camera rays and depth together. |
| It is built on Wan2.2-TI2V-5B, extended to a mixture of transformers with RGB, ray and depth |
| experts (12.6B parameters in total). Whatever subset of the three modalities you supply, the |
| model fills in the rest. That makes one checkpoint cover six tasks: |
|
|
| | task | given | generated | |
| |---|---|---| |
| | video → 4D | all frames | depth + cameras | |
| | image + camera → 4D | first frame + trajectory | video + depth | |
| | first/last image + camera → 4D | first and last frames + trajectory | video + depth | |
| | text → 4D | prompt | video + depth + cameras | |
| | camera → 4D | trajectory (+ prompt) | video + depth | |
| | depth → 4D | depth (+ prompt, optional frames) | video + cameras | |
|
|
| Code, samples and the training recipe: **https://github.com/amanikiruga/UGG** |
|
|
| ## Files |
|
|
| | file | contents | |
| |---|---| |
| | `ugg_release.safetensors` | the full UGG transformer (bf16) plus VAE latent normalisation statistics | |
|
|
| The Wan2.2 VAE and UMT5-XXL text encoder are not duplicated here. The code fetches them from |
| [`Wan-AI/Wan2.2-TI2V-5B`](https://huggingface.co/Wan-AI/Wan2.2-TI2V-5B). |
|
|
| ## Usage |
|
|
| ```bash |
| git clone https://github.com/amanikiruga/UGG && cd UGG |
| # environment: see the repo README |
| python infer.py task=video input=assets/bonn_balloon2/frames.mp4 |
| ``` |
|
|
| ```python |
| from ugg import UGG |
| |
| model = UGG.from_pretrained() # downloads ugg_release.safetensors on first use |
| pred = model.video_to_4d("clip.mp4") # pred.rgb, pred.depth, pred.c2w, pred.intrinsics |
| pred.save("outputs/clip") |
| ``` |
|
|
| Output resolution is 288×384. Clip length must satisfy `num_frames % 4 == 2` (for example 10, 26 or 50). |
| Inference takes one 80 GB GPU. |
|
|
| ## Intended Use and Responsible Use |
| UGG is intended for research on video generation and 4D scene understanding. |
| Outputs may contain inaccurate geometry or synthetic content. Users should |
| disclose generated content and avoid deceptive, privacy-invasive, or |
| safety-critical applications without independent validation. |
|
|
| ## License |
|
|
| The UGG weights are released under the MIT license. The Wan2.2 base components they rely on |
| are distributed by Wan-AI under their own license. |
