chenxib commited on
Commit
2d7cab3
·
verified ·
1 Parent(s): d6f1591

Polish release model card and link the upcoming GitHub Pages site

Browse files
Files changed (2) hide show
  1. README.md +15 -15
  2. assets/project.svg +3 -3
README.md CHANGED
@@ -18,18 +18,16 @@ tags:
18
  <p align="center" style="margin: 8px 0 14px;">
19
  <sub>Zhejiang University &nbsp;·&nbsp; Alibaba Group &nbsp;·&nbsp; Nanjing University</sub>
20
  </p>
21
- <div align="center" style="display: flex; flex-wrap: wrap; justify-content: center; align-items: center; gap: 8px; margin: 12px 0 22px;"> <a href="https://github.com/wuli-art/Adversarial-Gradient-Forcing"><img src="assets/github.svg" height="38" style="display: inline-block; vertical-align: middle; margin: 4px 3px;" alt="GitHub — source code"></a> <a href="https://shark-console.oss-cn-shanghai.aliyuncs.com/shuju/project/kv_reasoning/runs/AGF/reports/project-page/20261010-explainer-ea1f9094/index.html"><img src="assets/project.svg" height="38" style="display: inline-block; vertical-align: middle; margin: 4px 3px;" alt="Project page — videos and results"></a> <img src="assets/arxiv.svg" height="38" style="display: inline-block; vertical-align: middle; margin: 4px 3px;" alt="arXiv — coming soon">
22
  </div>
23
 
24
  **AGF** improves long-video generation by training models to withstand **adversarial perturbations in their generated history**. Trained on five-second rollouts, it supports generation up to four minutes with the same four-step sampler.
25
 
26
- <p align="center"> <a href="https://shark-console.oss-cn-shanghai.aliyuncs.com/shuju/project/kv_reasoning/runs/AGF/reports/project-page/20261010-explainer-ea1f9094/index.html#gallery"><img src="https://shark-console.oss-cn-shanghai.aliyuncs.com/shuju/project/kv_reasoning/runs/AGF/reports/project-page/20261010-9024de6/assets/showcase/agf-scenes.gif" width="960" alt="Selected AGF video excerpts: pirate ships, a robot, felt creatures, a creature by candlelight, a watercolor dance, and a yawning child."></a>
27
  <br>
28
  <sub>Generated with AGF · Selected video excerpts</sub>
29
  </p>
30
- <p align="center"> <a href="https://shark-console.oss-cn-shanghai.aliyuncs.com/shuju/project/kv_reasoning/runs/AGF/reports/project-page/20261010-explainer-ea1f9094/index.html">▶ Watch the method overview</a>
31
- &nbsp; · &nbsp; <a href="https://shark-console.oss-cn-shanghai.aliyuncs.com/shuju/project/kv_reasoning/runs/AGF/reports/project-page/20261010-explainer-ea1f9094/index.html#gallery">Explore full videos ↗</a>
32
- </p>
33
 
34
  <table style="display: table; width: 100%; table-layout: fixed;">
35
  <tr>
@@ -41,23 +39,25 @@ tags:
41
 
42
  ## 🧩 Model details
43
 
44
- AGF uses future prediction gradients to construct challenging history perturbations, then jointly trains on clean and perturbed histories. An early prediction consistency regularizer helps preserve scene structure and is removed in the second stage.
45
 
46
  | Property | Paper checkpoint |
47
  | --- | --- |
48
  | Architecture | Wan2.1 1.3B autoregressive video generator |
49
  | Initialization | Self-Forcing ODE initialization |
50
- | Training | 200 regularized AGF iterations + 200 pure AGF iterations |
51
  | Selected weights | Global step 400, `generator_ema` |
52
  | Training rollouts | 5 seconds; effective batch size 64 |
53
  | Training prompts | Filtered, LLM-extended VidProM prompts used by Self-Forcing |
54
  | Inference | 832 × 480, 16 fps, four denoising steps per block |
55
 
56
- ## 🚀 Quick start
 
 
57
 
58
- > **Release status** — model weights are not yet available in this Hugging Face repository. The GitHub repository currently requires access. The project page is available for viewing examples and results.
59
 
60
- Once `model.pt` is available, follow the [installation and inference guide](https://github.com/wuli-art/Adversarial-Gradient-Forcing/blob/main/docs/inference.md), then run from the code repository root:
61
 
62
  ```bash
63
  hf download Wuli-art/Adversarial-Gradient-Forcing model.pt \
@@ -65,24 +65,24 @@ hf download Wuli-art/Adversarial-Gradient-Forcing model.pt \
65
 
66
  AGF_RUN_ID=preview bash scripts/infer.sh \
67
  --checkpoint checkpoints/AGF/model.pt \
68
- --prompts prompts/examples.txt --latent-frames 960 --gpus 0
69
  ```
70
 
71
- The inference script selects **`generator_ema` by default**, matching the paper. Use `--latent-frames 21`, `240`, or `960` for approximately 5 seconds, 1 minute, or 4 minutes. Inference also requires the Wan 1.3B text encoder and VAE assets described in the guide. `model.pt` alone is not a complete checkpoint for exact training resumption.
72
 
73
  ## 📊 Results
74
 
75
- Scores reported in the paper (higher is better):
76
 
77
  | Model | 5 s · VBench Total | 60 s · VBench Total | 240 s · Quality 5 |
78
  | --- | ---: | ---: | ---: |
79
  | **AGF** | **82.03** | **81.47** | **84.82** |
80
 
81
- Quality 5 averages subject consistency, background consistency, motion smoothness, aesthetic quality, and imaging quality. It excludes dynamic degree and is distinct from the official VBench Total. See the [project page](https://shark-console.oss-cn-shanghai.aliyuncs.com/shuju/project/kv_reasoning/runs/AGF/reports/project-page/20261010-explainer-ea1f9094/index.html#results) for comparisons and the [evaluation protocol](https://github.com/wuli-art/Adversarial-Gradient-Forcing/blob/main/docs/evaluation.md) for details.
82
 
83
  ## Intended use and limitations
84
 
85
- AGF is intended for text-to-video generation and research on autoregressive video models. Long sequences can still contain visual artifacts, motion inconsistencies, or departures from the prompt; reported results are benchmark averages and do not guarantee every sample's quality. Review generated content before use or sharing.
86
 
87
  ## 📝 Citation
88
 
 
18
  <p align="center" style="margin: 8px 0 14px;">
19
  <sub>Zhejiang University &nbsp;·&nbsp; Alibaba Group &nbsp;·&nbsp; Nanjing University</sub>
20
  </p>
21
+ <div align="center" style="display: flex; flex-wrap: wrap; justify-content: center; align-items: center; gap: 8px; margin: 12px 0 22px;"> <a href="https://github.com/wuli-art/Adversarial-Gradient-Forcing"><img src="assets/github.svg" height="38" style="display: inline-block; vertical-align: middle; margin: 4px 3px;" alt="GitHub — source code"></a> <a href="https://wuli-art.github.io/Adversarial-Gradient-Forcing/"><img src="assets/project.svg" height="38" style="display: inline-block; vertical-align: middle; margin: 4px 3px;" alt="Project page — coming soon"></a> <img src="assets/arxiv.svg" height="38" style="display: inline-block; vertical-align: middle; margin: 4px 3px;" alt="arXiv — coming soon">
22
  </div>
23
 
24
  **AGF** improves long-video generation by training models to withstand **adversarial perturbations in their generated history**. Trained on five-second rollouts, it supports generation up to four minutes with the same four-step sampler.
25
 
26
+ <p align="center"> <a href="https://wuli-art.github.io/Adversarial-Gradient-Forcing/#gallery"><img src="https://shark-console.oss-cn-shanghai.aliyuncs.com/shuju/project/kv_reasoning/runs/AGF/reports/project-page/20261010-9024de6/assets/showcase/agf-scenes.gif" width="960" alt="Selected AGF video excerpts: pirate ships, a robot, felt creatures, a creature by candlelight, a watercolor dance, and a yawning child."></a>
27
  <br>
28
  <sub>Generated with AGF · Selected video excerpts</sub>
29
  </p>
30
+ <p align="center"><a href="https://wuli-art.github.io/Adversarial-Gradient-Forcing/#gallery">Full videos and comparisons ↗</a> &nbsp;·&nbsp; <sub>Coming soon</sub></p>
 
 
31
 
32
  <table style="display: table; width: 100%; table-layout: fixed;">
33
  <tr>
 
39
 
40
  ## 🧩 Model details
41
 
42
+ AGF selects adversarial history perturbations using distribution-matching gradients from later video blocks, then trains jointly on clean and perturbed histories. A first-step prediction consistency regularizer helps preserve scene structure during the initial training stage.
43
 
44
  | Property | Paper checkpoint |
45
  | --- | --- |
46
  | Architecture | Wan2.1 1.3B autoregressive video generator |
47
  | Initialization | Self-Forcing ODE initialization |
48
+ | Training | 200 regularized AGF + 200 pure AGF outer iterations |
49
  | Selected weights | Global step 400, `generator_ema` |
50
  | Training rollouts | 5 seconds; effective batch size 64 |
51
  | Training prompts | Filtered, LLM-extended VidProM prompts used by Self-Forcing |
52
  | Inference | 832 × 480, 16 fps, four denoising steps per block |
53
 
54
+ Iterations count outer training loops; the full schedule contains **80 generator updates**.
55
+
56
+ ## 🚀 Inference
57
 
58
+ **Code and pretrained weights: coming soon.**
59
 
60
+ Follow the [installation and inference guide](https://github.com/wuli-art/Adversarial-Gradient-Forcing/blob/main/docs/inference.md) for the Python/CUDA environment and Wan 1.3B assets. The release will use the following download and inference commands, run from the code repository root:
61
 
62
  ```bash
63
  hf download Wuli-art/Adversarial-Gradient-Forcing model.pt \
 
65
 
66
  AGF_RUN_ID=preview bash scripts/infer.sh \
67
  --checkpoint checkpoints/AGF/model.pt \
68
+ --prompts prompts/examples.txt --latent-frames 21 --gpus 0
69
  ```
70
 
71
+ The script selects **`generator_ema` by default**, matching the paper. Use `--latent-frames 21`, `240`, or `960` for approximately **5 seconds**, **1 minute**, or **4 minutes**. The default policy uses 12 latent frames of local attention, including 3 persistent sink frames, with relative sink positions enabled. See the [training guide](https://github.com/wuli-art/Adversarial-Gradient-Forcing/blob/main/docs/training.md) for the full training and checkpoint-resumption workflow.
72
 
73
  ## 📊 Results
74
 
75
+ Paper results under the shared evaluation protocol (higher is better). The 5-second and 60-second evaluations use 944 and 240 VBench prompts, respectively; the 240-second evaluation uses 120 MovieGenVideoBench extended prompts.
76
 
77
  | Model | 5 s · VBench Total | 60 s · VBench Total | 240 s · Quality 5 |
78
  | --- | ---: | ---: | ---: |
79
  | **AGF** | **82.03** | **81.47** | **84.82** |
80
 
81
+ Quality 5 (Q5) is the arithmetic mean of subject consistency, background consistency, motion smoothness, aesthetic quality, and imaging quality. It excludes dynamic degree and is not an official VBench Total score. See the [evaluation protocol](https://github.com/wuli-art/Adversarial-Gradient-Forcing/blob/main/docs/evaluation.md) for metric definitions and reproducibility details.
82
 
83
  ## Intended use and limitations
84
 
85
+ AGF is intended for text-to-video generation and research on autoregressive video models. Long sequences may exhibit visual artifacts, motion inconsistencies, or prompt drift.
86
 
87
  ## 📝 Citation
88
 
assets/project.svg CHANGED