README.md

---
license: apache-2.0
pretty_name: "rest2art Model Checkpoints"
base_model:
- Wan-AI/Wan2.2-I2V-A14B
task_categories:
- image-to-video
language:
- en
tags:
- articulated-objects
- reconstruction
- image-to-video
- lora
- wan2.2
---

# [ECCV'26] Articulated Object Reconstruction from Rest-State Observation

<!-- TODO: replace with real links once public -->
[![Project Page](https://img.shields.io/badge/Project_Page-Rest2Art-blue)](https://da-eun07.github.io/rest2art/)
[![arXiv](https://img.shields.io/badge/arXiv-TODO-B31B1B?logo=arxiv)](https://arxiv.org/abs/2607.27749)
[![GitHub](https://img.shields.io/badge/GitHub-Repository-black?logo=github)](https://github.com/da-eun07/rest2art)
[![Dataset](https://img.shields.io/badge/Dataset-rest2art-yellow)](https://huggingface.co/datasets/da-eun07/rest2art)

## Model Checkpoints

Pretrained models used by the rest2art pipeline. We only re-host the Wan2.2 video-generation
LoRAs, whose inference is config-sensitive and hard to reproduce. Every other model is linked to
its original repo (cleaner attribution and licensing), so fetch those from the source.

| Model | Hosting | Source | License |
|---|---|---|---|
| GPT (OpenAI API) | API, no ckpt | OpenAI | n/a |
| SAM 3 | link | [facebook/sam3](https://huggingface.co/facebook/sam3) | see repo |
| Qwen3-VL-8B-Instruct | link | [Qwen/Qwen3-VL-8B-Instruct](https://huggingface.co/Qwen/Qwen3-VL-8B-Instruct) | Apache-2.0 |
| Wan2.2-I2V-A14B + 3 LoRAs | hosted (`wan2.2-i2v-loras/`) | [Wan-AI](https://huggingface.co/Wan-AI/Wan2.2-I2V-A14B), [lightx2v](https://huggingface.co/lightx2v/Wan2.2-Distill-Loras), [Kijai](https://huggingface.co/Kijai/WanVideo_comfy) | Apache-2.0 |
| CoTracker3 (scaled_offline) | link | [facebook/cotracker3](https://huggingface.co/facebook/cotracker3) | CC BY-NC 4.0 |


## Get started

Put every checkpoint under one `ckpt/` root, then point the pipeline flags at it.

### 1. Download checkpoints

```bash
mkdir -p ckpt && cd ckpt

# this collection: Wan2.2 LoRAs + config + loader
huggingface-cli download da-eun07/rest2art-models --local-dir ./rest2art-models

# Wan2.2 base model, UMT5-XXL T5, VAE (~119 GB) from Wan-AI
bash ./rest2art-models/wan2.2-i2v-loras/scripts/download_assets.sh ./Wan-AI

# CoTracker3 offline checkpoint
huggingface-cli download facebook/cotracker3 scaled_offline.pth --local-dir ./co-tracker
```

SAM 3 (`facebook/sam3`) and Qwen3-VL-8B (`Qwen/Qwen3-VL-8B-Instruct`) are pulled from their repos
on first run, so no manual download is needed. Resulting layout:

```
ckpt/
  rest2art-models/wan2.2-i2v-loras/loras/*.safetensors
  Wan-AI/Wan2.2-I2V-A14B/
  co-tracker/scaled_offline.pth
```

### 2. Render the Wan2.2 config

The LightX2V config needs absolute LoRA paths, so render a local copy:

```bash
cd ckpt/rest2art-models/wan2.2-i2v-loras
python scripts/render_config.py --repo . --base ../../Wan-AI/Wan2.2-I2V-A14B --out config.local.json
```

### 3. Run the pipeline

From the rest2art project root, point the flags at your `ckpt/` root:

```bash
python -m rest2art.pipeline.main \
    --scene <scene> \
    --wan_path       ckpt/Wan-AI/Wan2.2-I2V-A14B \
    --config_json    ckpt/rest2art-models/wan2.2-i2v-loras/config.local.json \
    --cotracker_ckpt ckpt/co-tracker/scaled_offline.pth \
    --vlm_path       Qwen/Qwen3-VL-8B-Instruct
```

To run only the Wan2.2 video step, follow [`wan2.2-i2v-loras/`](./wan2.2-i2v-loras).

## License

Hosted LoRAs are Apache-2.0 (Wan-AI, lightx2v); the VBVR LoRA follows
[Kijai/WanVideo_comfy](https://huggingface.co/Kijai/WanVideo_comfy) terms. Linked models keep
their own licenses (see table). Config and scripts are Apache-2.0. To request removal from this
mirror, open a discussion.

## Citation

```bibtex
@inproceedings{lee2026rest2art,
  title={Articulated Object Reconstruction from Rest-State Observation},
  author={Lee, Daeun and Lee, Jaeah and Kim, Woosung and Jung, Haebeom and Park, Jaesik},
  booktitle={European Conference on Computer Vision (ECCV)},
  year={2026}
}
```