--- license: apache-2.0 library_name: pytorch pipeline_tag: text-to-video tags: - video-generation - diffusion - sgf - sgf-plus ---

SGF+ logo Self Gradient Forcing Plus

**Decoupling Gradient Flows for Autoregressive Video Generation** [Paper](https://arxiv.org/abs/2610.10429) · [Project Page](https://zihan-su.github.io/self-gradient-forcing-plus) · [GitHub](https://github.com/Zihan-Su/Self_Gradient_Forcing_Plus) SGF+ separates context-writing and denoising parameters to resolve conflicting gradient updates, improving autoregressive video generation and enabling rollouts of up to 24 hours from only 5s training windows. ![SGF+ video generation comparisons and a 24-hour rollout](assets/teaser.png) ## Model Checkpoints | Checkpoint | Description | | :--- | :--- | | [`chunkwise/model.pt`](chunkwise/model.pt) | SGF+ for chunkwise video generation. | | [`framewise/model.pt`](framewise/model.pt) | SGF+ for framewise video generation. | | [`diagnostics/sgf.pt`](diagnostics/sgf.pt) | SGF checkpoint used for gradient-conflict reproduction, containing the raw generator and paired critic. | ## Installation The environment follows the SGF setup. ```bash git clone https://github.com/Zihan-Su/Self_Gradient_Forcing_Plus.git cd Self_Gradient_Forcing_Plus conda create -n sgf_plus python=3.10 -y conda activate sgf_plus pip install -r requirements.txt pip install flash-attn --no-build-isolation python setup.py develop ``` Run the following commands from the repository root. ## Download Weights ```bash bash scripts/download_weights.sh ``` The script uses the Hugging Face CLI command `hf` by default. Set `HF_CLI=huggingface-cli` if your environment still uses the older command name. It downloads: - Wan base models to `wan_models/Wan2.1-T2V-1.3B` and `wan_models/Wan2.1-T2V-14B`. - [Causal-Forcing](https://github.com/thu-ml/Causal-Forcing) AR initialization checkpoints to `checkpoints/init/chunkwise/ar_diffusion.pt` and `checkpoints/init/framewise/ar_diffusion.pt`. - Released SGF+ inference checkpoints to `hf_weights/chunkwise/model.pt` and `hf_weights/framewise/model.pt`. - The training prompt list to `prompts/vidprom_filtered_extended.txt`. Run `hf auth login` first if authentication is required. ## Inference The default prompt file is `prompts/test_prompt.txt` with 8 prompts. The launcher uses 8 GPUs when at least 8 GPUs are visible; otherwise it falls back to single-GPU serial inference. By default it generates `963` latent frames, which decode to about 240 seconds of video at 16 fps. The inference script takes the release setting name (`chunkwise` or `framewise`) and a checkpoint path, and selects the matching config automatically: - chunkwise config: `configs/sgf_plus_chunkwise.yaml` - framewise config: `configs/sgf_plus_framewise.yaml` The long-video KV-cache geometry is set in `scripts/infer_self_gradient_forcing.sh`, which is called by the SGF+ launcher. Chunkwise defaults to `KV_CACHE_SINK=3`, `KV_CACHE_FIFO_FRAMES=6`, and `KV_CACHE_CURRENT_FRAMES=3`, so `--kv_cache_max_frames` is `12`. Framewise defaults to `KV_CACHE_SINK=4`, `KV_CACHE_FIFO_FRAMES=16`, and `KV_CACHE_CURRENT_FRAMES=1`, so `--kv_cache_max_frames` is `21`. ### Chunkwise ```bash bash scripts/infer_sgf_plus.sh chunkwise hf_weights/chunkwise/model.pt ``` This uses: ```text configs/sgf_plus_chunkwise.yaml hf_weights/chunkwise/model.pt ``` ### Framewise ```bash bash scripts/infer_sgf_plus.sh framewise hf_weights/framewise/model.pt ``` This uses: ```text configs/sgf_plus_framewise.yaml hf_weights/framewise/model.pt ``` ### Custom checkpoint or prompt file ```bash bash scripts/infer_sgf_plus.sh \ chunkwise \ hf_weights/chunkwise/model.pt \ prompts/test_prompt.txt ``` Useful overrides: ```bash NUM_OUTPUT_FRAMES=963 SEED=42 OUTPUT_ROOT=outputs/demo \ bash scripts/infer_sgf_plus.sh chunkwise hf_weights/chunkwise/model.pt ``` For trained checkpoints, pass the release setting first and the produced `logs/.../checkpoint_model_*/model.pt` path as the second argument. The script uses EMA weights by default; set `USE_EMA=0` to use the non-EMA `generator` weights. ## Training ### Chunkwise SGF+ ```bash bash scripts/train_sgf_plus_chunkwise.sh ``` Equivalent explicit form: ```bash bash scripts/train_sgf_plus_chunkwise.sh \ configs/sgf_plus_chunkwise.yaml \ logs/sgf_plus_chunkwise ``` ### Framewise SGF+ ```bash bash scripts/train_sgf_plus_framewise.sh ``` Equivalent explicit form: ```bash bash scripts/train_sgf_plus_framewise.sh \ configs/sgf_plus_framewise.yaml \ logs/sgf_plus_framewise ``` The launchers accept `[config.yaml] [logdir] [extra train.py args...]` and support single-node and multi-node training. Without explicit or scheduler-provided topology settings, nodes auto-register through `.rendezvous/` on the shared filesystem and launch static `torchrun` with an IP master address. For multi-node jobs, run the same command on every node within the gather window. Useful overrides: ```bash GATHER_WINDOW=90 NUM_GPUS=8 MASTER_PORT=29501 ENABLE_WANDB=1 \ bash scripts/train_sgf_plus_chunkwise.sh logs/sgf_plus_chunkwise ``` To specify the topology manually, set `NNODES`, `NODE_RANK`, `MASTER_ADDR` and `MASTER_PORT`. Use a distinct `NODE_RANK` on each node and matching values for the other settings. ## Gradient Conflict Experiments To reproduce our gradient-conflict experiments, see the [experiment guide](https://github.com/Zihan-Su/Self_Gradient_Forcing_Plus/blob/main/gradient_conflict/README.md). ## Citation ```bibtex @article{zhuang2026self, title={Self Gradient Forcing: Native Long Video Extrapolation}, author={Zhuang, Junhao and Zhang, Shiyi and Bian, Yuxuan and Li, Yaowei and Luo, Yawen and Liu, Yijun and Jin, Weiyang and Zhang, Songchun and He, Xianglong and Zhang, Xuying and others}, journal={arXiv preprint arXiv:2607.20368}, year={2026} } @article{su2026sgf+, title={SGF+: Decoupling Gradient Flows for Autoregressive Video Generation}, author={Su, Zihan and Zhuang, Junhao and Li, Yaowei and Lu, Siwen and Li, Haoran and Li, Lingen and Wu, Haoyu and Jin, Weiyang and Zhang, Songchun and Huang, Haoyang and others}, journal={arXiv preprint arXiv:2610.10429}, year={2026} } ``` ## License This project is released under the [Apache-2.0 license](https://huggingface.co/ZihanSu/Self_Gradient_Forcing_Plus/blob/main/LICENSE).