File size: 6,359 Bytes
0b7a872 c11e680 0b7a872 c11e680 0b7a872 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 | ---
license: apache-2.0
library_name: pytorch
pipeline_tag: text-to-video
tags:
- video-generation
- diffusion
- sgf
- sgf-plus
---
<h1 style="display:flex;align-items:center;gap:8px;font-size:26px;line-height:1.3;margin:16px 0"><img src="assets/logo.png" width="32" height="32" style="display:block;flex-shrink:0;margin:0" alt="SGF+ logo"> Self Gradient Forcing Plus</h1>
**Decoupling Gradient Flows for Autoregressive Video Generation**
[Paper](https://arxiv.org/abs/2610.10429) · [Project Page](https://zihan-su.github.io/self-gradient-forcing-plus) · [GitHub](https://github.com/Zihan-Su/Self_Gradient_Forcing_Plus)
SGF+ separates context-writing and denoising parameters to resolve conflicting gradient updates, improving autoregressive video generation and enabling rollouts of up to 24 hours from only 5s training windows.
## Model Checkpoints
| Checkpoint | Description |
| :--- | :--- |
| [`chunkwise/model.pt`](chunkwise/model.pt) | SGF+ for chunkwise video generation. |
| [`framewise/model.pt`](framewise/model.pt) | SGF+ for framewise video generation. |
| [`diagnostics/sgf.pt`](diagnostics/sgf.pt) | SGF checkpoint used for gradient-conflict reproduction, containing the raw generator and paired critic. |
## Installation
The environment follows the SGF setup.
```bash
git clone https://github.com/Zihan-Su/Self_Gradient_Forcing_Plus.git
cd Self_Gradient_Forcing_Plus
conda create -n sgf_plus python=3.10 -y
conda activate sgf_plus
pip install -r requirements.txt
pip install flash-attn --no-build-isolation
python setup.py develop
```
Run the following commands from the repository root.
## Download Weights
```bash
bash scripts/download_weights.sh
```
The script uses the Hugging Face CLI command `hf` by default. Set `HF_CLI=huggingface-cli` if your environment still uses the older command name.
It downloads:
- Wan base models to `wan_models/Wan2.1-T2V-1.3B` and `wan_models/Wan2.1-T2V-14B`.
- [Causal-Forcing](https://github.com/thu-ml/Causal-Forcing) AR initialization checkpoints to `checkpoints/init/chunkwise/ar_diffusion.pt` and `checkpoints/init/framewise/ar_diffusion.pt`.
- Released SGF+ inference checkpoints to `hf_weights/chunkwise/model.pt` and `hf_weights/framewise/model.pt`.
- The training prompt list to `prompts/vidprom_filtered_extended.txt`.
Run `hf auth login` first if authentication is required.
## Inference
The default prompt file is `prompts/test_prompt.txt` with 8 prompts. The launcher uses 8 GPUs when at least 8 GPUs are visible; otherwise it falls back to single-GPU serial inference. By default it generates `963` latent frames, which decode to about 240 seconds of video at 16 fps.
The inference script takes the release setting name (`chunkwise` or `framewise`) and a checkpoint path, and selects the matching config automatically:
- chunkwise config: `configs/sgf_plus_chunkwise.yaml`
- framewise config: `configs/sgf_plus_framewise.yaml`
The long-video KV-cache geometry is set in `scripts/infer_self_gradient_forcing.sh`, which is called by the SGF+ launcher. Chunkwise defaults to `KV_CACHE_SINK=3`, `KV_CACHE_FIFO_FRAMES=6`, and `KV_CACHE_CURRENT_FRAMES=3`, so `--kv_cache_max_frames` is `12`. Framewise defaults to `KV_CACHE_SINK=4`, `KV_CACHE_FIFO_FRAMES=16`, and `KV_CACHE_CURRENT_FRAMES=1`, so `--kv_cache_max_frames` is `21`.
### Chunkwise
```bash
bash scripts/infer_sgf_plus.sh chunkwise hf_weights/chunkwise/model.pt
```
This uses:
```text
configs/sgf_plus_chunkwise.yaml
hf_weights/chunkwise/model.pt
```
### Framewise
```bash
bash scripts/infer_sgf_plus.sh framewise hf_weights/framewise/model.pt
```
This uses:
```text
configs/sgf_plus_framewise.yaml
hf_weights/framewise/model.pt
```
### Custom checkpoint or prompt file
```bash
bash scripts/infer_sgf_plus.sh \
chunkwise \
hf_weights/chunkwise/model.pt \
prompts/test_prompt.txt
```
Useful overrides:
```bash
NUM_OUTPUT_FRAMES=963 SEED=42 OUTPUT_ROOT=outputs/demo \
bash scripts/infer_sgf_plus.sh chunkwise hf_weights/chunkwise/model.pt
```
For trained checkpoints, pass the release setting first and the produced `logs/.../checkpoint_model_*/model.pt` path as the second argument. The script uses EMA weights by default; set `USE_EMA=0` to use the non-EMA `generator` weights.
## Training
### Chunkwise SGF+
```bash
bash scripts/train_sgf_plus_chunkwise.sh
```
Equivalent explicit form:
```bash
bash scripts/train_sgf_plus_chunkwise.sh \
configs/sgf_plus_chunkwise.yaml \
logs/sgf_plus_chunkwise
```
### Framewise SGF+
```bash
bash scripts/train_sgf_plus_framewise.sh
```
Equivalent explicit form:
```bash
bash scripts/train_sgf_plus_framewise.sh \
configs/sgf_plus_framewise.yaml \
logs/sgf_plus_framewise
```
The launchers accept `[config.yaml] [logdir] [extra train.py args...]` and support single-node and multi-node training. Without explicit or scheduler-provided topology settings, nodes auto-register through `.rendezvous/` on the shared filesystem and launch static `torchrun` with an IP master address. For multi-node jobs, run the same command on every node within the gather window.
Useful overrides:
```bash
GATHER_WINDOW=90 NUM_GPUS=8 MASTER_PORT=29501 ENABLE_WANDB=1 \
bash scripts/train_sgf_plus_chunkwise.sh logs/sgf_plus_chunkwise
```
To specify the topology manually, set `NNODES`, `NODE_RANK`, `MASTER_ADDR` and `MASTER_PORT`. Use a distinct `NODE_RANK` on each node and matching values for the other settings.
## Gradient Conflict Experiments
To reproduce our gradient-conflict experiments, see the [experiment guide](https://github.com/Zihan-Su/Self_Gradient_Forcing_Plus/blob/main/gradient_conflict/README.md).
## Citation
```bibtex
@misc{su2026sgfdecouplinggradientflows,
title={SGF+: Decoupling Gradient Flows for Autoregressive Video Generation},
author={Zihan Su and Junhao Zhuang and Yaowei Li and Siwen Lu and Haoran Li and Lingen Li and Haoyu Wu and Weiyang Jin and Songchun Zhang and Haoyang Huang and Chun Yuan and Zeyue Xue and Nan Duan},
year={2026},
eprint={2610.10429},
archivePrefix={arXiv},
primaryClass={cs.CV},
url={https://arxiv.org/abs/2610.10429},
}
```
## License
This project is released under the [Apache-2.0 license](https://huggingface.co/ZihanSu/Self_Gradient_Forcing_Plus/blob/main/LICENSE).
|