| --- |
| tags: |
| - relayvsr |
| - lightx2v |
| - video-super-resolution |
| - super-resolution |
| - video-restoration |
| - video-enhancement |
| - 2x-upscaling |
| - safetensors |
| --- |
| |
| # RelayVSR_X2 |
| |
| We trained this RelayVSR model for **native, fixed 2× video super-resolution**. |
| It doubles the input video's width and height. |
| |
| We found that RelayVSR's model design is effective at suppressing temporal flicker |
| in upscaled videos. This is particularly useful for videos produced by generative |
| video models: the generated video may occasionally contain subtle flicker, which |
| super-resolution algorithms can amplify. |
| |
| The [original RelayVSR model](https://huggingface.co/kopper/RelayVSR) uses a fixed |
| 4× super-resolution scale, which incurs substantial inference time. To reduce this |
| cost, we modified the model architecture and continued training to obtain a |
| **fixed 2× RelayVSR model**, with inference supported by |
| [LightX2V](https://github.com/ModelTC/LightX2V). |
| |
| We recommend RelayVSR as a super-resolution post-processing model for video |
| generators such as **MiniMax-H3**. For example, a MiniMax-H3 video generated at |
| **768 × 1344** can be upscaled by 2× to **1536 × 2688**. |
| |
| For inference instructions, see the |
| [LightX2V RelayVSR documentation](https://github.com/ModelTC/LightX2V/tree/main/scripts/relayvsr). |
| |
| | Model | Native scale | Output dimensions | |
| | --- | --- | --- | |
| | [Original RelayVSR](https://huggingface.co/kopper/RelayVSR) | 4× | 4W × 4H | |
| | [RelayVSR X2](https://huggingface.co/lightx2v/RelayVSR_X2) | 2× | 2W × 2H | |
| |
| ## Inference Benchmark |
| |
| On the tested clip, **RelayVSR X2 with LightX2V processes 243 frames in 5.348 seconds |
| (45.44 inference FPS)** on a single NVIDIA H100 80GB. |
| |
| Using the same |
| [drone rooftop input video](assets/benchmark/originals/input.mp4): |
| **1024 × 768**, **243 frames**, **24 fps**, **10.125 seconds**. |
| |
| The input video was generated by **MiniMax-H3** and contains some mild temporal |
| flicker. This flicker is already present in the source video before super-resolution. |
| |
| All cases ran sequentially on one **NVIDIA H100 80GB HBM3** with |
| **PyTorch 2.11.0 + CUDA 13.0**. Each row reports **one measured run after |
| same-resolution warmup**. |
| |
| | Model / implementation | Scale | Output resolution | Inference time (s) | Inference FPS | Output | |
| | --- | --- | --- | ---: | ---: | --- | |
| | [SeedVR2-3B](https://huggingface.co/ByteDance-Seed/SeedVR2-3B) | 2× | 2048 × 1536 | 352.820 | 0.69 | [MP4](assets/benchmark/originals/seedvr2_3b_x2.mp4) | |
| | [FlashVSR+ Turbo](https://huggingface.co/ruchangyao/FlashVSR_plus) | 2× | 2048 × 1536 | 54.024 | 4.50 | [MP4](assets/benchmark/originals/flashvsr_plus_turbo_x2.mp4) | |
| | [Open-H3-Regenerate-2K](https://huggingface.co/Efficient-Large-Model/Open-H3-Regenerate-2K) | 2× | 2048 × 1536 | 784.954 | 0.31 | [MP4](assets/benchmark/originals/open_h3_regenerate_x2.mp4) | |
| | [SwiftVR](https://huggingface.co/H-oliday/SwiftVR) | 2× | 2048 × 1536 | 7.252 | 33.51 | [MP4](assets/benchmark/originals/swiftvr_x2.mp4) | |
| | [SwiftVR (LightX2V)](https://github.com/ModelTC/LightX2V/tree/main/scripts/swiftvr) | 2× | 2048 × 1536 | 6.399 | 37.98 | [MP4](assets/benchmark/originals/swiftvr_lightx2v_x2.mp4) | |
| | [RelayVSR](https://huggingface.co/kopper/RelayVSR) | 4× | 4096 × 3072 | 26.332 | 9.23 | [MP4](assets/benchmark/originals/relayvsr_x4.mp4) | |
| | [RelayVSR (LightX2V)](https://github.com/ModelTC/LightX2V/tree/main/scripts/relayvsr) | 4× | 4096 × 3072 | 19.304 | 12.59 | [MP4](assets/benchmark/originals/relayvsr_x4_lightx2v.mp4) | |
| | **[RelayVSR X2 (LightX2V)](https://huggingface.co/lightx2v/RelayVSR_X2)** | 2× | 2048 × 1536 | **5.348** 🚀 | **45.44** 🚀 | [MP4](assets/benchmark/originals/relayvsr_x2_lightx2v.mp4) | |
| |
| ## Video Comparison |
| |
| <video controls src="https://huggingface.co/lightx2v/RelayVSR_X2/resolve/main/assets/benchmark/comparison_grid.mp4"></video> |
| |
| Many super-resolution methods can amplify the temporal flicker already present in |
| AI-generated input videos. **RelayVSR can substantially alleviate this flicker**, |
| producing more temporally consistent results. |
| |
| The original RelayVSR model is limited to fixed **4× upscaling**, which incurs a |
| higher computational cost and slower inference. To reduce this cost, we trained a |
| **native 2× super-resolution model based on RelayVSR**. Our **RelayVSR X2** model |
| combines effective temporal-flicker suppression with **fast inference**, making it |
| suited to upscaling AI-generated videos. |
| |
| ## Model Files |
| |
| Download both checkpoints into the same model directory: |
| |
| | File | Description | |
| | --- | --- | |
| | `wan_x2.safetensors` | Wan checkpoint for the 2× model | |
| | `flash_x2.safetensors` | Flash checkpoint for the 2× model | |
|
|
| ## Usage with LightX2V |
|
|
| You can use the LightX2V Docker image: |
|
|
| ```bash |
| docker pull lightx2v/lightx2v:26062001-cu130 |
| ``` |
|
|
| Run the following steps inside the container. First, clone the latest LightX2V source: |
|
|
| ```bash |
| git clone https://github.com/ModelTC/LightX2V.git |
| cd LightX2V |
| ``` |
|
|
| Download the 2× weights: |
|
|
| ```bash |
| hf download lightx2v/RelayVSR_X2 \ |
| wan_x2.safetensors flash_x2.safetensors \ |
| --local-dir ./models/RelayVSR_X2 |
| ``` |
|
|
| Edit |
| [`scripts/relayvsr/run_relayvsr_x2_sr.sh`](https://github.com/ModelTC/LightX2V/blob/main/scripts/relayvsr/run_relayvsr_x2_sr.sh) |
| and set the absolute paths and GPU to use: |
|
|
| ```bash |
| lightx2v_path="/absolute/path/to/LightX2V" |
| model_path="/absolute/path/to/LightX2V/models/RelayVSR_X2" |
| video_path="/absolute/path/to/input.mp4" |
| export CUDA_VISIBLE_DEVICES=0 |
| ``` |
|
|
| Keep the script's X2 settings: `--model_cls relayvsr_x2`, |
| `--config_json "${lightx2v_path}/configs/relayvsr/relayvsr_x2.json"`, and |
| `--sr_ratio 2`. Use both X2 checkpoints together. |
|
|
| Run from the LightX2V repository root: |
|
|
| ```bash |
| bash scripts/relayvsr/run_relayvsr_x2_sr.sh |
| ``` |
|
|
| The default output is `save_results/relayvsr_x2.mp4` inside the LightX2V repository. |
| Change `--save_result_path` in the script to choose another output location. |
|
|
| For block offloading or multi-GPU execution, see the |
| [LightX2V RelayVSR usage guide](https://github.com/ModelTC/LightX2V/tree/main/scripts/relayvsr). |
| Use the corresponding `relayvsr_x2_block_offload.json` or `relayvsr_x2_sp.json` |
| configuration. The default configuration enables compilation and warmup. |
|
|
| ## Upstream Sources |
|
|
| This model builds on the original RelayVSR project. Credit goes to the RelayVSR |
| authors for their method and implementation. |
|
|
| - Original Hugging Face model: [kopper/RelayVSR](https://huggingface.co/kopper/RelayVSR) |
| - Original GitHub repository: [kopperx/RelayVSR](https://github.com/kopperx/RelayVSR) |
|
|
|
|
