UniVOE — A Removal-Driven Framework for Interaction-Aware Universal Video Object Editing

Paper: UniVOE: A Removal-Driven Framework for Interaction-Aware Universal Video Object Editing · Shuzhou Yang, Lanqing Guo, Lingen Li, Weiqi Li, Zhao Zhong, Liefeng Bo, Jian Zhang · Peking University (SECE), Tencent Hunyuan, The Chinese University of Hong Kong

Code: https://github.com/Tencent-Hunyuan/UniVOE

Weights for UniVOE, a video object editing framework built on Wan2.1-VACE-14B. Object removal and injection are treated as reciprocal operations between an object-present and an object-absent video; besides the object itself, UniVOE edits the interaction effects it leaves on the scene — shadows, reflections, footprints and lighting.

Removal: "remove the woman" · Injection: "Inject the woman" — the model also updates the effects tied to the object.

Files

File	Model	Steps	Where it goes
remove_12step_vace.safetensors	remove-12step (control branch)	12	checkpoints/UniVOE/
remove_12step_dit_lora.safetensors	remove-12step (DiT LoRA, rank 32)	12	checkpoints/UniVOE/
remove_50step_vace.safetensors	remove-50step (control branch)	50	checkpoints/UniVOE/
inject_50step_vace.safetensors	inject-50step (control branch, after GRAR)	50	checkpoints/UniVOE/

remove-12step needs both of its files; the two 50-step models need only their single file.

Model size

The full system is 23B parameters: a 14B frozen Transformer backbone, a 3B trainable control branch (VACE) and the text encoder / VAE. The files here are the 3B control branch — 6.1 GB in bf16 each — plus a 153M DiT LoRA for the 12-step remover. The 14B backbone, text encoder and VAE are the public Wan2.1-VACE-14B base model, which must be downloaded separately. Removal additionally needs GroundingDINO, SAM2 and bert-base-uncased — see the code repository for links and setup.

File	Size	SHA-256
remove_12step_vace.safetensors	6.1 GB	5b8f224841232c26abe403774920b9fd0ae5b3c674bd8faf4680c181e31c66a8
remove_12step_dit_lora.safetensors	307 MB	beffa706a6b0261c6c92fa270ac466a33deda2ea4e703a384a30ac01d3c6bd8c
remove_50step_vace.safetensors	6.1 GB	3971aaa457aaeb977c40aa4e1b7d58238ccf15a80ed9224851f95d6edd4eb4ea
inject_50step_vace.safetensors	6.1 GB	098936173cb0e8f40e8e4712c047052287b09d3b82f133cda63a1caaf04d75cd

Download

huggingface-cli download Ysz2022/UniVOE --local-dir checkpoints/UniVOE
# or a single model:
huggingface-cli download Ysz2022/UniVOE --include "remove_12step_*" --local-dir checkpoints/UniVOE

Then clone the inference code, place this folder under its checkpoints/, and follow its README.

Usage

• Removal — input video + instruction, optionally a first-frame reference with the object already removed. remove-12step is the default; remove-50step is ~4× slower.
• Injection — object-absent video + instruction, optionally a first-frame reference with the object inserted. Injection is mask-free: the object may appear anywhere in the video.

bash run_erase.sh                 # removal, 12-step
STEPS=50 bash run_erase.sh        # removal, 50-step
bash run_inject.sh                # injection

Settings

Item	remove-12step	remove-50step	inject-50step
Denoising steps	12	50	50
Resolution	832 × 480	832 × 480	832 × 480
Frames	4n+1, at most 237	4n+1, at most 237	4n+1, at most 237
CFG scale / sigma shift / seed	5.0 / 5.0 / 42	5.0 / 5.0 / 42	5.0 / 5.0 / 42
References	none / first / first+tail / first+mid+tail	none / first / first+tail	none / first / first+tail

Method at a glance

1. Effect-aware remover — an inpainting prior is transferred to removal and finetuned on object-present / object-absent pairs with an inflated object mask, so the model recovers the whole object-absent scene, not just the mask interior.
2. 12-step distillation — the remover is distilled from 50 to 12 steps with a rank-32 backbone LoRA, enabling large-scale object-absent generation at a fraction of the cost.
3. Removal-driven injection — the trained remover converts real object-present videos into object-absent counterparts, giving paired injection supervision with real motion and interaction effects, together with curated open-source data.
4. GRAR post-training — Group-Relative Advantage Regression updates the injector online with signed group-relative rewards from a preference-aligned reward model, without a reference policy.

Intended use and limitations

• Intended for: research and creative video editing — object removal and insertion in short clips (up to 237 frames at 832×480).
• Not intended for: deceptive editing of real people or events, or any use that misrepresents footage of identifiable individuals. Videos and instructions are supplied by the user.
• Output quality depends on the instruction and on the reference image; crowded scenes and heavy camera motion may show artifacts. Instructions are in English.

Citation

@article{yang2026univoe,
  title   = {UniVOE: A Removal-Driven Framework for Interaction-Aware Universal Video Object Editing},
  author  = {Yang, Shuzhou and Guo, Lanqing and Li, Lingen and Li, Weiqi and Zhong, Zhao and Bo, Liefeng and Zhang, Jian},
  journal = {arXiv preprint arXiv:2610.31641},
  year    = {2026}
}

Licence and attribution

• These weights are released under [LICENSE-PLACEHOLDER].
• Built on Wan2.1-VACE-14B; its licence applies to the base model.
• The inference code builds on DiffSynth-Studio (Apache-2.0) and uses SAM2 and GroundingDINO for object localisation.