Add AGF model card with paper checkpoint details and project links
Browse files
README.md
CHANGED
|
@@ -1,3 +1,85 @@
|
|
| 1 |
---
|
| 2 |
license: apache-2.0
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 3 |
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
---
|
| 2 |
license: apache-2.0
|
| 3 |
+
language:
|
| 4 |
+
- en
|
| 5 |
+
pipeline_tag: text-to-video
|
| 6 |
+
base_model:
|
| 7 |
+
- Wan-AI/Wan2.1-T2V-1.3B
|
| 8 |
+
- gdhe17/Self-Forcing
|
| 9 |
+
tags:
|
| 10 |
+
- video-generation
|
| 11 |
+
- autoregressive
|
| 12 |
+
- diffusion
|
| 13 |
+
- adversarial-gradient-forcing
|
| 14 |
---
|
| 15 |
+
|
| 16 |
+
# Adversarial Gradient Forcing
|
| 17 |
+
|
| 18 |
+
**Robust Few-Step Autoregressive Video Generation**
|
| 19 |
+
|
| 20 |
+
[GitHub](https://github.com/wuli-art/Adversarial-Gradient-Forcing) 路 **arXiv (coming soon)** 路 [Project Page](https://shark-console.oss-cn-shanghai.aliyuncs.com/shuju/project/kv_reasoning/runs/AGF/reports/project-page/20261010-explainer-ea1f9094/index.html)
|
| 21 |
+
|
| 22 |
+
**AGF** improves long-video generation by training models to withstand adversarial perturbations in their generated history. Trained on five-second rollouts, it supports generation up to four minutes with the same four-step sampler.
|
| 23 |
+
|
| 24 |
+
[](https://shark-console.oss-cn-shanghai.aliyuncs.com/shuju/project/kv_reasoning/runs/AGF/reports/project-page/20261010-explainer-ea1f9094/index.html#gallery)
|
| 25 |
+
|
| 26 |
+
[Watch the method overview and full video comparisons 鈫抅(https://shark-console.oss-cn-shanghai.aliyuncs.com/shuju/project/kv_reasoning/runs/AGF/reports/project-page/20261010-explainer-ea1f9094/index.html)
|
| 27 |
+
|
| 28 |
+
## Model details
|
| 29 |
+
|
| 30 |
+
AGF uses future prediction gradients to construct challenging history perturbations, then jointly trains on clean and perturbed histories. An early prediction consistency regularizer helps preserve scene structure and is removed in the second stage.
|
| 31 |
+
|
| 32 |
+
| Property | Paper checkpoint |
|
| 33 |
+
| --- | --- |
|
| 34 |
+
| Architecture | Wan2.1 1.3B autoregressive video generator |
|
| 35 |
+
| Initialization | Self-Forcing ODE initialization |
|
| 36 |
+
| Training | 200 regularized AGF iterations + 200 pure AGF iterations |
|
| 37 |
+
| Selected weights | Global step 400, `generator_ema` |
|
| 38 |
+
| Training rollouts | 5 seconds; effective batch size 64 |
|
| 39 |
+
| Training prompts | Filtered, LLM-extended VidProM prompts used by Self-Forcing |
|
| 40 |
+
| Inference | 832 脳 480, 16 fps, four denoising steps per block |
|
| 41 |
+
|
| 42 |
+
## Usage
|
| 43 |
+
|
| 44 |
+
**Release status:** model weights are not yet available in this Hugging Face repository. The GitHub repository currently requires access. The project page is available for viewing examples and results.
|
| 45 |
+
|
| 46 |
+
Once `model.pt` is available, follow the [installation and inference guide](https://github.com/wuli-art/Adversarial-Gradient-Forcing/blob/main/docs/inference.md), then run from the code repository root:
|
| 47 |
+
|
| 48 |
+
```bash
|
| 49 |
+
hf download Wuli-art/Adversarial-Gradient-Forcing model.pt \
|
| 50 |
+
--local-dir checkpoints/AGF
|
| 51 |
+
|
| 52 |
+
AGF_RUN_ID=preview bash scripts/infer.sh \
|
| 53 |
+
--checkpoint checkpoints/AGF/model.pt \
|
| 54 |
+
--prompts prompts/examples.txt --latent-frames 960 --gpus 0
|
| 55 |
+
```
|
| 56 |
+
|
| 57 |
+
The inference script selects **`generator_ema` by default**, matching the paper. Use `--latent-frames 21`, `240`, or `960` for approximately 5 seconds, 1 minute, or 4 minutes. Inference also requires the Wan 1.3B text encoder and VAE assets described in the guide. `model.pt` alone is not a complete checkpoint for exact training resumption.
|
| 58 |
+
|
| 59 |
+
## Results
|
| 60 |
+
|
| 61 |
+
Scores reported in the paper (higher is better):
|
| 62 |
+
|
| 63 |
+
| Model | 5 s 路 VBench Total | 60 s 路 VBench Total | 240 s 路 Quality 5 |
|
| 64 |
+
| --- | ---: | ---: | ---: |
|
| 65 |
+
| AGF | 82.03 | 81.47 | 84.82 |
|
| 66 |
+
|
| 67 |
+
Quality 5 averages subject consistency, background consistency, motion smoothness, aesthetic quality, and imaging quality. It excludes dynamic degree and is distinct from the official VBench Total. See the [project page](https://shark-console.oss-cn-shanghai.aliyuncs.com/shuju/project/kv_reasoning/runs/AGF/reports/project-page/20261010-explainer-ea1f9094/index.html#results) for comparisons and the [evaluation protocol](https://github.com/wuli-art/Adversarial-Gradient-Forcing/blob/main/docs/evaluation.md) for details.
|
| 68 |
+
|
| 69 |
+
## Intended use and limitations
|
| 70 |
+
|
| 71 |
+
AGF is intended for text-to-video generation and research on autoregressive video models. Long sequences can still contain visual artifacts, motion inconsistencies, or departures from the prompt; reported results are benchmark averages and do not guarantee every sample's quality. Review generated content before use or sharing.
|
| 72 |
+
|
| 73 |
+
## Citation
|
| 74 |
+
|
| 75 |
+
```bibtex
|
| 76 |
+
@misc{xia2026adversarial,
|
| 77 |
+
title = {Adversarial Gradient Forcing for Robust Few-Step Autoregressive Video Generation},
|
| 78 |
+
author = {Chenxi Xia and Yixuan Xu and Shan Liu and Jiapeng Wang
|
| 79 |
+
and Maohua Li and Hanlin Tang and Kan Liu and Tao Lan
|
| 80 |
+
and Weiyi Lu and Xinggao Liu},
|
| 81 |
+
year = {2026}
|
| 82 |
+
}
|
| 83 |
+
```
|
| 84 |
+
|
| 85 |
+
License: Apache 2.0. Third-party models, datasets, and other assets retain their respective licenses.
|