Polish release model card and link the upcoming GitHub Pages site
Browse files- README.md +15 -15
- assets/project.svg +3 -3
README.md
CHANGED
|
@@ -18,18 +18,16 @@ tags:
|
|
| 18 |
<p align="center" style="margin: 8px 0 14px;">
|
| 19 |
<sub>Zhejiang University · Alibaba Group · Nanjing University</sub>
|
| 20 |
</p>
|
| 21 |
-
<div align="center" style="display: flex; flex-wrap: wrap; justify-content: center; align-items: center; gap: 8px; margin: 12px 0 22px;"> <a href="https://github.com/wuli-art/Adversarial-Gradient-Forcing"><img src="assets/github.svg" height="38" style="display: inline-block; vertical-align: middle; margin: 4px 3px;" alt="GitHub — source code"></a> <a href="https://
|
| 22 |
</div>
|
| 23 |
|
| 24 |
**AGF** improves long-video generation by training models to withstand **adversarial perturbations in their generated history**. Trained on five-second rollouts, it supports generation up to four minutes with the same four-step sampler.
|
| 25 |
|
| 26 |
-
<p align="center"> <a href="https://
|
| 27 |
<br>
|
| 28 |
<sub>Generated with AGF · Selected video excerpts</sub>
|
| 29 |
</p>
|
| 30 |
-
<p align="center">
|
| 31 |
-
· <a href="https://shark-console.oss-cn-shanghai.aliyuncs.com/shuju/project/kv_reasoning/runs/AGF/reports/project-page/20261010-explainer-ea1f9094/index.html#gallery">Explore full videos ↗</a>
|
| 32 |
-
</p>
|
| 33 |
|
| 34 |
<table style="display: table; width: 100%; table-layout: fixed;">
|
| 35 |
<tr>
|
|
@@ -41,23 +39,25 @@ tags:
|
|
| 41 |
|
| 42 |
## 🧩 Model details
|
| 43 |
|
| 44 |
-
AGF
|
| 45 |
|
| 46 |
| Property | Paper checkpoint |
|
| 47 |
| --- | --- |
|
| 48 |
| Architecture | Wan2.1 1.3B autoregressive video generator |
|
| 49 |
| Initialization | Self-Forcing ODE initialization |
|
| 50 |
-
| Training | 200 regularized AGF
|
| 51 |
| Selected weights | Global step 400, `generator_ema` |
|
| 52 |
| Training rollouts | 5 seconds; effective batch size 64 |
|
| 53 |
| Training prompts | Filtered, LLM-extended VidProM prompts used by Self-Forcing |
|
| 54 |
| Inference | 832 × 480, 16 fps, four denoising steps per block |
|
| 55 |
|
| 56 |
-
|
|
|
|
|
|
|
| 57 |
|
| 58 |
-
|
| 59 |
|
| 60 |
-
|
| 61 |
|
| 62 |
```bash
|
| 63 |
hf download Wuli-art/Adversarial-Gradient-Forcing model.pt \
|
|
@@ -65,24 +65,24 @@ hf download Wuli-art/Adversarial-Gradient-Forcing model.pt \
|
|
| 65 |
|
| 66 |
AGF_RUN_ID=preview bash scripts/infer.sh \
|
| 67 |
--checkpoint checkpoints/AGF/model.pt \
|
| 68 |
-
--prompts prompts/examples.txt --latent-frames
|
| 69 |
```
|
| 70 |
|
| 71 |
-
The
|
| 72 |
|
| 73 |
## 📊 Results
|
| 74 |
|
| 75 |
-
|
| 76 |
|
| 77 |
| Model | 5 s · VBench Total | 60 s · VBench Total | 240 s · Quality 5 |
|
| 78 |
| --- | ---: | ---: | ---: |
|
| 79 |
| **AGF** | **82.03** | **81.47** | **84.82** |
|
| 80 |
|
| 81 |
-
Quality 5
|
| 82 |
|
| 83 |
## Intended use and limitations
|
| 84 |
|
| 85 |
-
AGF is intended for text-to-video generation and research on autoregressive video models. Long sequences
|
| 86 |
|
| 87 |
## 📝 Citation
|
| 88 |
|
|
|
|
| 18 |
<p align="center" style="margin: 8px 0 14px;">
|
| 19 |
<sub>Zhejiang University · Alibaba Group · Nanjing University</sub>
|
| 20 |
</p>
|
| 21 |
+
<div align="center" style="display: flex; flex-wrap: wrap; justify-content: center; align-items: center; gap: 8px; margin: 12px 0 22px;"> <a href="https://github.com/wuli-art/Adversarial-Gradient-Forcing"><img src="assets/github.svg" height="38" style="display: inline-block; vertical-align: middle; margin: 4px 3px;" alt="GitHub — source code"></a> <a href="https://wuli-art.github.io/Adversarial-Gradient-Forcing/"><img src="assets/project.svg" height="38" style="display: inline-block; vertical-align: middle; margin: 4px 3px;" alt="Project page — coming soon"></a> <img src="assets/arxiv.svg" height="38" style="display: inline-block; vertical-align: middle; margin: 4px 3px;" alt="arXiv — coming soon">
|
| 22 |
</div>
|
| 23 |
|
| 24 |
**AGF** improves long-video generation by training models to withstand **adversarial perturbations in their generated history**. Trained on five-second rollouts, it supports generation up to four minutes with the same four-step sampler.
|
| 25 |
|
| 26 |
+
<p align="center"> <a href="https://wuli-art.github.io/Adversarial-Gradient-Forcing/#gallery"><img src="https://shark-console.oss-cn-shanghai.aliyuncs.com/shuju/project/kv_reasoning/runs/AGF/reports/project-page/20261010-9024de6/assets/showcase/agf-scenes.gif" width="960" alt="Selected AGF video excerpts: pirate ships, a robot, felt creatures, a creature by candlelight, a watercolor dance, and a yawning child."></a>
|
| 27 |
<br>
|
| 28 |
<sub>Generated with AGF · Selected video excerpts</sub>
|
| 29 |
</p>
|
| 30 |
+
<p align="center"><a href="https://wuli-art.github.io/Adversarial-Gradient-Forcing/#gallery">Full videos and comparisons ↗</a> · <sub>Coming soon</sub></p>
|
|
|
|
|
|
|
| 31 |
|
| 32 |
<table style="display: table; width: 100%; table-layout: fixed;">
|
| 33 |
<tr>
|
|
|
|
| 39 |
|
| 40 |
## 🧩 Model details
|
| 41 |
|
| 42 |
+
AGF selects adversarial history perturbations using distribution-matching gradients from later video blocks, then trains jointly on clean and perturbed histories. A first-step prediction consistency regularizer helps preserve scene structure during the initial training stage.
|
| 43 |
|
| 44 |
| Property | Paper checkpoint |
|
| 45 |
| --- | --- |
|
| 46 |
| Architecture | Wan2.1 1.3B autoregressive video generator |
|
| 47 |
| Initialization | Self-Forcing ODE initialization |
|
| 48 |
+
| Training | 200 regularized AGF + 200 pure AGF outer iterations |
|
| 49 |
| Selected weights | Global step 400, `generator_ema` |
|
| 50 |
| Training rollouts | 5 seconds; effective batch size 64 |
|
| 51 |
| Training prompts | Filtered, LLM-extended VidProM prompts used by Self-Forcing |
|
| 52 |
| Inference | 832 × 480, 16 fps, four denoising steps per block |
|
| 53 |
|
| 54 |
+
Iterations count outer training loops; the full schedule contains **80 generator updates**.
|
| 55 |
+
|
| 56 |
+
## 🚀 Inference
|
| 57 |
|
| 58 |
+
**Code and pretrained weights: coming soon.**
|
| 59 |
|
| 60 |
+
Follow the [installation and inference guide](https://github.com/wuli-art/Adversarial-Gradient-Forcing/blob/main/docs/inference.md) for the Python/CUDA environment and Wan 1.3B assets. The release will use the following download and inference commands, run from the code repository root:
|
| 61 |
|
| 62 |
```bash
|
| 63 |
hf download Wuli-art/Adversarial-Gradient-Forcing model.pt \
|
|
|
|
| 65 |
|
| 66 |
AGF_RUN_ID=preview bash scripts/infer.sh \
|
| 67 |
--checkpoint checkpoints/AGF/model.pt \
|
| 68 |
+
--prompts prompts/examples.txt --latent-frames 21 --gpus 0
|
| 69 |
```
|
| 70 |
|
| 71 |
+
The script selects **`generator_ema` by default**, matching the paper. Use `--latent-frames 21`, `240`, or `960` for approximately **5 seconds**, **1 minute**, or **4 minutes**. The default policy uses 12 latent frames of local attention, including 3 persistent sink frames, with relative sink positions enabled. See the [training guide](https://github.com/wuli-art/Adversarial-Gradient-Forcing/blob/main/docs/training.md) for the full training and checkpoint-resumption workflow.
|
| 72 |
|
| 73 |
## 📊 Results
|
| 74 |
|
| 75 |
+
Paper results under the shared evaluation protocol (higher is better). The 5-second and 60-second evaluations use 944 and 240 VBench prompts, respectively; the 240-second evaluation uses 120 MovieGenVideoBench extended prompts.
|
| 76 |
|
| 77 |
| Model | 5 s · VBench Total | 60 s · VBench Total | 240 s · Quality 5 |
|
| 78 |
| --- | ---: | ---: | ---: |
|
| 79 |
| **AGF** | **82.03** | **81.47** | **84.82** |
|
| 80 |
|
| 81 |
+
Quality 5 (Q5) is the arithmetic mean of subject consistency, background consistency, motion smoothness, aesthetic quality, and imaging quality. It excludes dynamic degree and is not an official VBench Total score. See the [evaluation protocol](https://github.com/wuli-art/Adversarial-Gradient-Forcing/blob/main/docs/evaluation.md) for metric definitions and reproducibility details.
|
| 82 |
|
| 83 |
## Intended use and limitations
|
| 84 |
|
| 85 |
+
AGF is intended for text-to-video generation and research on autoregressive video models. Long sequences may exhibit visual artifacts, motion inconsistencies, or prompt drift.
|
| 86 |
|
| 87 |
## 📝 Citation
|
| 88 |
|
assets/project.svg
CHANGED
|
|
|
|