| --- |
| library_name: diffusers |
| base_model: stabilityai/stable-video-diffusion-img2vid |
| tags: |
| - video-restoration |
| - video-refocusing |
| - video-diffusion |
| - lora |
| license: other |
| license_name: stable-video-diffusion-community |
| license_link: https://huggingface.co/nickchiu/Refocusing-Any-Video/blob/main/LICENSE-SVD.txt |
| --- |
| |
| # Refocusing Any Video: Temporally Consistent Video Refocusing for Dynamic Focal Planes |
|
|
| **SIGGRAPH Asia 2026** |
|
|
| Chee Kim Gan, [Pin-Yen Chiu](https://itsnickchiu.github.io/), I-Sheng Fang, Jun-Cheng Chen |
| Research Center for Information Technology Innovation, Academia Sinica |
|
|
| Inference checkpoints for Refocusing Any Video (RAV). **Video Defocus Deblurring (VDD)** recovers an all-in-focus video; **Video Bokeh Rendering (VBR)** refocuses it using depth and a chosen focal plane. Optional identity adapters guide face restoration. |
|
|
| [Source code and inference instructions](https://github.com/aiiu-lab/Refocusing-Any-Video) |
|
|
| ## Release status |
|
|
| - [x] Inference code and pretrained checkpoints |
| - [ ] Training code — being prepared for a future release |
|
|
| ## Checkpoints |
|
|
| | Folder | Contents | |
| | --- | --- | |
| | `vdd_weights/checkpoint-90000/` | VDD LoRA weights | |
| | `vbr_weights/checkpoint-100000/` | VBR LoRA weights and required input convolution | |
| | `identity_adapter/` | Identity Adapter | |
| | `identity_adapter_track4gen/` | Identity Adapter with Track4Gen Refiner | |
| | `facenet/` | MTCNN face detector weights | |
|
|
| ## Download and inference |
|
|
| After setting up the [code environment](https://github.com/aiiu-lab/Refocusing-Any-Video#environment), run from the cloned code repository root: |
|
|
| ```bash |
| python tools/checkpoints/transfer_checkpoints.py download \ |
| --repo-id nickchiu/Refocusing-Any-Video --local-dir ./checkpoints |
| ``` |
|
|
| Download SVD, Video Depth Anything, and the auxiliary face models separately using the [checkpoint guide](https://github.com/aiiu-lab/Refocusing-Any-Video/blob/main/checkpoints/README.md), then follow the [inference example](https://github.com/aiiu-lab/Refocusing-Any-Video#inference). These adapters require the code and SVD base model. Generated details may differ from the original scene. |
|
|
| ## Models and licenses |
|
|
| RAV uses Stable Video Diffusion (SVD), Video Depth Anything, FaRL, ArcFace, OpenAI CLIP, and facenet-pytorch. |
|
|
| SVD and applicable derivatives are subject to the [SVD license](LICENSE-SVD.txt) and [NOTICE](NOTICE.txt). **Powered by Stability AI** (attribution required by the SVD license). MTCNN retains its [MIT license](facenet/LICENSE.md); other models follow their providers' terms. An additional project-specific license has not yet been specified. |
|
|
| ## Citation |
|
|
| If you use this work, please cite: |
|
|
| ```bibtex |
| @misc{gan2026refocusinganyvideo, |
| title = {Refocusing Any Video: Temporally Consistent Video Refocusing for Dynamic Focal Planes}, |
| author = {Gan, Chee Kim and Chiu, Pin-Yen and Fang, I-Sheng and Chen, Jun-Cheng}, |
| year = {2026}, |
| note = {SIGGRAPH Asia 2026}, |
| url = {https://github.com/aiiu-lab/Refocusing-Any-Video} |
| } |
| ``` |
|
|
