Model card

README.md

@@ -0,0 +1,64 @@
+---
+license: mit
+base_model: Wan-AI/Wan2.2-TI2V-5B
+library_name: pytorch
+pipeline_tag: image-to-video
+tags:
+  - 4d
+  - video-generation
+  - depth-estimation
+  - camera-pose-estimation
+  - novel-view-synthesis
+  - world-model
+---
+
+# UGG
+
+UGG is a video diffusion model that generates RGB, per-frame camera rays and depth together.
+It is built on Wan2.2-TI2V-5B, extended to a mixture of transformers with RGB, ray and depth
+experts (12.6B parameters in total). Whatever subset of the three modalities you supply, the
+model fills in the rest. That makes one checkpoint cover six tasks:
+
+| task | given | generated |
+|---|---|---|
+| video → 4D | all frames | depth + cameras |
+| image + camera → 4D | first frame + trajectory | video + depth |
+| first/last image + camera → 4D | first and last frames + trajectory | video + depth |
+| text → 4D | prompt | video + depth + cameras |
+| camera → 4D | trajectory (+ prompt) | video + depth |
+| depth → 4D | depth (+ prompt, optional frames) | video + cameras |
+
+Code, samples and the training recipe: **https://github.com/amanikiruga/UGG**
+
+## Files
+
+| file | contents |
+|---|---|
+| `ugg_release.safetensors` | the full UGG transformer (bf16) plus VAE latent normalisation statistics |
+
+The Wan2.2 VAE and UMT5-XXL text encoder are not duplicated here. The code fetches them from
+[`Wan-AI/Wan2.2-TI2V-5B`](https://huggingface.co/Wan-AI/Wan2.2-TI2V-5B).
+
+## Usage
+
+```bash
+git clone https://github.com/amanikiruga/UGG && cd UGG
+# environment: see the repo README
+python infer.py task=video input=assets/bonn_balloon2/frames.mp4
+```
+
+```python
+from ugg import UGG
+
+model = UGG.from_pretrained()          # downloads ugg_release.safetensors on first use
+pred = model.video_to_4d("clip.mp4")    # pred.rgb, pred.depth, pred.c2w, pred.intrinsics
+pred.save("outputs/clip")
+```
+
+Output resolution is 288×384. Clip length must satisfy `num_frames % 4 == 2` (for example 10, 26 or 50).
+Inference takes one 80 GB GPU.
+
+## License
+
+The UGG weights are released under the MIT license. The Wan2.2 base components they rely on
+are distributed by Wan-AI under their own license.