| --- |
| license: apache-2.0 |
| language: |
| - en |
| tags: |
| - video-generation |
| - human-object-interaction |
| - sparse-control |
| --- |
| |
| # SparseCtrl-HOI-14B |
|
|
| Official checkpoints for **SparseCtrl-HOI: Sparse Temporal Control for Human-Object Interaction Video Generation**. |
|
|
| SparseCtrl-HOI generates human-object interaction videos conditioned on a reference image, audio, and sparse interaction keyframes with designated timestamps. |
|
|
| [Paper](https://arxiv.org/abs/2607.05994) 路 [Project Page](https://mpi-lab.github.io/SparseCtrl-HOI/) 路 [Code](https://github.com/MPI-Lab/SparseCtrl-HOI) |
|
|
| ## Checkpoints |
|
|
| - `stage1.ckpt`: Stage-1 weights (LoRA and TiRoPE input adapter). |
| - `stage2.ckpt`: Stage-2 weights (Motion Prior Injection Module). |
| - `prompt_emb.pt`: Precomputed T5 embedding of the positive prompt below. |
| - `prompt_emb_negative.pt`: Precomputed T5 embedding of the negative prompt below. |
|
|
| **Positive prompt** |
|
|
| ```text |
| A person is introducing a product. |
| ``` |
|
|
| **Negative prompt** |
|
|
| ```text |
| Vivid color tones, background/camera moving quickly, screen switching, subtitles and special effects, mutation, overexposed, static, blurred details, subtitles, style, work, painting, image, still, overall grayish, worst quality, low quality, JPEG compression residue, ugly, incomplete, extra fingers, poorly drawn hands, poorly drawn face, deformed, disfigured, malformed limbs, fingers merging, motionless image, chaotic background, three legs, crowded background with many people, walking backward |
| ``` |
|
|
| Both stage checkpoints are required for inference. These files are not a standalone model; the additional pretrained models listed in the code repository are also required. |
|
|
| ## Usage |
|
|
| From the SparseCtrl-HOI project root, download the checkpoints: |
|
|
| ```bash |
| hf download Moon-jack/SparseCtrl-HOI-14B \ |
| --local-dir checkpoints/SparseCtrl-HOI-14B |
| ``` |
|
|
| Follow the [GitHub README](https://github.com/MPI-Lab/SparseCtrl-HOI#setup) to install dependencies, download the additional models and run. |
|
|
| ## License |
|
|
| These checkpoints are released under the Apache License 2.0. |
| Third-party models remain subject to their respective licenses. |
|
|
