README.md

---
library_name: pytorch
pipeline_tag: image-to-video
license: apache-2.0
base_model: Wan-AI/Wan2.2-I2V-A14B
tags:
  - video-generation
  - image-to-video
  - camera-control
  - wan2.2
  - safetensors
---

# SCoPE: Sightline-Coordinate Positional Encoding for Video Diffusion Transformers

Minghao Yin · Jiahao Lu · Wenbo Hu · Wang Zhao · Ying Shan · Kai Han

[![Project Page](https://img.shields.io/badge/Project%20Page-visual--ai.github.io%2Fscope-1E90FF)](https://visual-ai.github.io/scope/)
[![arXiv](https://img.shields.io/badge/arXiv-2606.27345-B31B1B?logo=arxiv)](https://arxiv.org/abs/2606.27345)
[![GitHub](https://img.shields.io/badge/GitHub-SCoPE-181717?logo=github)](https://github.com/TencentARC/SCoPE)
[![License](https://img.shields.io/badge/License-Apache%202.0-green)](https://github.com/TencentARC/SCoPE/blob/main/LICENSE.txt)

![SCoPE teaser](assets/teaser.png)

SCoPE adds camera sightlines as positional coordinates to a pretrained video diffusion
transformer. Given a first frame, a text prompt, and a camera trajectory, it generates a video
that follows the requested camera motion while preserving the original image-to-video prior. This
repository is a self-contained release for **Wan2.2-I2V-A14B**: it contains everything required for
inference, so a separate Wan2.2 checkpoint download is not needed.

## Download

```bash
pip install -U huggingface_hub
hf download TencentARC/SCoPE --local-dir checkpoints/SCoPE
```

The checkpoint is approximately 67 GB. Keep both the checkpoint and the Hugging Face cache on local
storage.

## Usage

Install the SCoPE code. The released weights were trained and evaluated with **PyTorch 2.9.1
(CUDA 12.8)**; because changing the PyTorch version can change the numerical output, we recommend
reproducing this exact environment with [uv](https://docs.astral.sh/uv/):

```bash
git clone https://github.com/TencentARC/SCoPE.git
cd SCoPE
uv sync
source .venv/bin/activate
```

Generate a video with an example camera trajectory:

```bash
python inference.py \
  --model_path checkpoints/SCoPE \
  --case omni-misty-forest \
  --trajectory truck_right \
  --output_path outputs/omni-misty-forest.mp4
```

For custom inputs:

```bash
python inference.py \
  --model_path checkpoints/SCoPE \
  --input_image path/to/first_frame.png \
  --prompt "A person walks along a misty forest trail." \
  --camera_path path/to/camera_poses.npy \
  --x_fov 1.11847 \
  --output_path outputs/custom.mp4
```

Camera poses use OpenCV camera-to-world coordinates and must have shape `[81, 3, 4]` or
`[81, 4, 4]`. `x_fov` is the horizontal field of view in radians; pinhole cameras use `xi=0`. See
the [GitHub repository](https://github.com/TencentARC/SCoPE) for the full documentation, options,
and demos.

## Training data

SCoPE is trained with RealEstate10K, DL3DV, PanShot, and OmniWorld. The datasets use a common
camera protocol: poses are expressed relative to the first camera and translation is normalized
with per-clip near depth, while absolute scale is handled inside the model by a learned scale gate.
Users are responsible for following the licenses and terms of the corresponding datasets.

## Intended use and limitations

This model is intended for research on image-to-video generation and controllable camera motion.
It inherits the visual capabilities, biases, safety limitations, and computational requirements of
Wan2.2. Results may degrade for inaccurate camera poses or intrinsics, trajectories far outside the
training distribution, large occlusions, or unusually fast camera motion.

## Citation

```bibtex
@article{yin2026scope,
  title={SCoPE: Sightline-Coordinate Positional Encoding for Video Diffusion Transformers},
  author={Yin, Minghao and Lu, Jiahao and Hu, Wenbo and Zhao, Wang and Shan, Ying and Han, Kai},
  year={2026}
}
```

## Acknowledgements

SCoPE is built on [Wan2.2](https://github.com/Wan-Video/Wan2.2) and
[DiffSynth-Studio](https://github.com/modelscope/DiffSynth-Studio). We thank the authors and
contributors of these projects.

## License

SCoPE is released under the [Apache-2.0 License](https://github.com/TencentARC/SCoPE/blob/main/LICENSE.txt).