| --- |
| license: apache-2.0 |
| pipeline_tag: video-to-video |
| tags: |
| - video |
| - object-removal |
| - video-inpainting |
| - wan |
| - vace |
| --- |
| |
| # Effecteraser-VOR Inference |
|
|
| Inference code and distilled checkpoints for video object removal. This repository contains separate 1-step and 2-step DMD students and a single shared copy of the text encoder, tokenizer, VAE, and LightVAE weights. |
|
|
| The associated gated dataset is [Kry4ta1/Effecteraser-VOR](https://huggingface.co/datasets/Kry4ta1/Effecteraser-VOR). |
|
|
| ## Layout |
|
|
| ```text |
| checkpoints/ |
| common/ # shared T5, tokenizer, Wan VAE, and LightVAE |
| dmd_1step/ # 1-step transformer checkpoint |
| dmd_2step/ # 2-step transformer checkpoint |
| configs/ |
| scripts/ |
| src/ |
| ``` |
|
|
| ## Installation |
|
|
| Python 3.11 or 3.12, CUDA-capable PyTorch, FFmpeg, and Git LFS are recommended. |
|
|
| ```bash |
| git lfs install |
| git clone https://huggingface.co/Kry4ta1/Effecteraser-VOR-Inference |
| cd Effecteraser-VOR-Inference |
| python -m pip install -r requirements.txt |
| ``` |
|
|
| ## Input format |
|
|
| Provide one directory of source videos and one directory of binary mask videos. Every source video must have an exactly matching mask filename. Supported video extensions are MP4, AVI, MOV, MKV, and WebM. |
|
|
| The default inference settings use the first 81 frames, an orientation-aware 480x832 canvas, 16 FPS output, mask dilation of 6 pixels, and guidance scale 1.0. |
|
|
| ## Run |
|
|
| One-step inference: |
|
|
| ```bash |
| INPUT_DIR=/path/to/videos MASK_DIR=/path/to/masks GPU=0 \ |
| bash scripts/infer_1step.sh |
| ``` |
|
|
| Two-step inference: |
|
|
| ```bash |
| INPUT_DIR=/path/to/videos MASK_DIR=/path/to/masks GPU=0 \ |
| bash scripts/infer_2step.sh |
| ``` |
|
|
| Run both variants concurrently on two GPUs: |
|
|
| ```bash |
| INPUT_DIR=/path/to/videos MASK_DIR=/path/to/masks \ |
| GPU_1STEP=0 GPU_2STEP=1 bash scripts/infer_all.sh |
| ``` |
|
|
| Each run writes generated videos to `outputs/<variant>/meta/` and side-by-side mask/input/result review videos to `outputs/<variant>/cat/`. |
|
|
| ## Notes |
|
|
| - The two DMD checkpoints are separately distilled; do not run the 1-step weights with a 2-step schedule or vice versa. |
| - `guidance_scale` must remain 1.0 for these distilled students. |
| - The repository uses the shared checkpoint directory through `--common_model_name`; this avoids publishing duplicate copies of the large T5 and VAE weights. |
| - This package is intended for research use. Users are responsible for evaluating outputs and complying with applicable licenses and policies. |
|
|
