---
license: mit
base_model: Wan-AI/Wan2.2-TI2V-5B
library_name: pytorch
pipeline_tag: image-to-video
tags:
  - 4d
  - video-generation
  - depth-estimation
  - camera-pose-estimation
  - novel-view-synthesis
  - world-model
---

# UGG

UGG is a video diffusion model that generates RGB, per-frame camera rays and depth together.
It is built on Wan2.2-TI2V-5B, extended to a mixture of transformers with RGB, ray and depth
experts (12.6B parameters in total). Whatever subset of the three modalities you supply, the
model fills in the rest. That makes one checkpoint cover six tasks:

| task | given | generated |
|---|---|---|
| video → 4D | all frames | depth + cameras |
| image + camera → 4D | first frame + trajectory | video + depth |
| first/last image + camera → 4D | first and last frames + trajectory | video + depth |
| text → 4D | prompt | video + depth + cameras |
| camera → 4D | trajectory (+ prompt) | video + depth |
| depth → 4D | depth (+ prompt, optional frames) | video + cameras |

Code, samples and the training recipe: **https://github.com/amanikiruga/UGG**

## Files

| file | contents |
|---|---|
| `ugg_release.safetensors` | the full UGG transformer (bf16) plus VAE latent normalisation statistics |

The Wan2.2 VAE and UMT5-XXL text encoder are not duplicated here. The code fetches them from
[`Wan-AI/Wan2.2-TI2V-5B`](https://huggingface.co/Wan-AI/Wan2.2-TI2V-5B).

## Usage

```bash
git clone https://github.com/amanikiruga/UGG && cd UGG
# environment: see the repo README
python infer.py task=video input=assets/bonn_balloon2/frames.mp4
```

```python
from ugg import UGG

model = UGG.from_pretrained()          # downloads ugg_release.safetensors on first use
pred = model.video_to_4d("clip.mp4")    # pred.rgb, pred.depth, pred.c2w, pred.intrinsics
pred.save("outputs/clip")
```

Output resolution is 288×384. Clip length must satisfy `num_frames % 4 == 2` (for example 10, 26 or 50).
Inference takes one 80 GB GPU.

## Intended Use and Responsible Use
UGG is intended for research on video generation and 4D scene understanding.
Outputs may contain inaccurate geometry or synthetic content. Users should
disclose generated content and avoid deceptive, privacy-invasive, or
safety-critical applications without independent validation.

## License

The UGG weights are released under the MIT license. The Wan2.2 base components they rely on
are distributed by Wan-AI under their own license.