Qixin Hu commited on
Commit
e4f959c
Β·
verified Β·
1 Parent(s): 89f1ad1

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +110 -0
README.md ADDED
@@ -0,0 +1,110 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ library_name: pytorch
4
+ pipeline_tag: text-to-video
5
+ tags:
6
+ - long-video-generation
7
+ - autoregressive
8
+ - retrieval-augmented-generation
9
+ - video-generation
10
+ - wan
11
+ - longlive
12
+ language:
13
+ - en
14
+ ---
15
+
16
+ # LongLive-RAG β€” Checkpoints & Toy Data
17
+
18
+ This repository hosts the model checkpoints, prompt files, and a toy latent set for
19
+ **LongLive-RAG: A General Retrieval-Augmented Framework for Long Video Generation**.
20
+
21
+ - πŸ“„ **Paper:** arXiv (coming soon)
22
+ - πŸ’» **Code:** https://github.com/qixinhu11/LongLive-RAG
23
+ - πŸ” **TL;DR:** LongLive-RAG turns long video generation into a *retrieval problem*. An
24
+ autoregressive (AR) video generator looks back over the video it has *already* generated
25
+ and pulls in the most relevant past latents as extra context β€” cutting error accumulation,
26
+ identity drift, and background flicker over long horizons, **without retraining the base
27
+ generator**.
28
+
29
+ ## What's in here
30
+
31
+ ```
32
+ checkpoints/
33
+ β”œβ”€β”€ causal_forcing.pt # Causal-Forcing AR backbone
34
+ β”œβ”€β”€ self_forcing.pt # Self-Forcing AR backbone
35
+ β”œβ”€β”€ longlive_base.pt # LongLive AR backbone
36
+ β”œβ”€β”€ longlive_lora.pt # LongLive LoRA (paired with longlive_base.pt)
37
+ β”œβ”€β”€ ae_latent_mem.pt # Retrieval autoencoder (default for inference)
38
+ β”œβ”€β”€ moviegenbench_128_refined.txt # 128 MovieGenBench prompts
39
+ └── vidprom_filtered_extended.txt # Self-Forcing prompt pool (for generate_latent.py)
40
+ toydatasets/
41
+ └── latent_0000xx.pt # tiny example latent set for the training demo
42
+ ```
43
+
44
+ - **AR backbones** (`causal_forcing.pt`, `self_forcing.pt`, `longlive_base.pt` + `longlive_lora.pt`)
45
+ are the frozen base generators LongLive-RAG plugs into.
46
+ - **`ae_latent_mem.pt`** is the trainable retrieval encoder (a small latent autoencoder) β€”
47
+ the only component LongLive-RAG actually trains.
48
+ - **`toydatasets/`** is a tiny set of clean latent blocks, enough to smoke-test the AE
49
+ training pipeline end-to-end.
50
+
51
+ > The base **WAN VAE** (`Wan-AI/Wan2.1-T2V-1.3B`) that LongLive-RAG operates in the latent
52
+ > space of is **not** included here β€” download it separately (see below).
53
+
54
+ ## Download
55
+
56
+ Everything restores into the code repo's expected layout with two commands:
57
+
58
+ ```bash
59
+ # Base WAN VAE β€” LongLive-RAG operates in its latent space
60
+ hf download Wan-AI/Wan2.1-T2V-1.3B --local-dir wan_models/Wan2.1-T2V-1.3B
61
+
62
+ # All LongLive-RAG assets β€” restores checkpoints/ and toydatasets/ in place
63
+ hf download qixinhu11/LongLive-RAG --local-dir .
64
+ ```
65
+
66
+ Older setups can swap `hf download` for `huggingface-cli download` (same arguments).
67
+
68
+ To pull a single file instead of the whole repo:
69
+
70
+ ```bash
71
+ hf download qixinhu11/LongLive-RAG checkpoints/ae_latent_mem.pt --local-dir .
72
+ ```
73
+
74
+ ## Usage
75
+
76
+ Clone the [code repository](https://github.com/qixinhu11/LongLive-RAG), download the assets
77
+ above, then run the shipped 3 Γ— 2 grid (three backbones Γ— two context-assembly methods):
78
+
79
+ ```bash
80
+ # Main result: LongLive backbone + LongLive-RAG retrieval
81
+ bash inference.sh longlive latentmem
82
+
83
+ # Baseline: native sliding-window
84
+ bash inference.sh causal_forcing native
85
+ ```
86
+
87
+ See the [GitHub README](https://github.com/qixinhu11/LongLive-RAG) for full installation,
88
+ inference, and training instructions.
89
+
90
+ ## License
91
+
92
+ Released under the [Apache 2.0](https://github.com/qixinhu11/LongLive-RAG/blob/main/LICENSE)
93
+ license. The included AR backbones and WAN VAE latent space derive from their respective
94
+ upstream projects ([LongLive](https://github.com/NVlabs/LongLive),
95
+ [Self-Forcing](https://github.com/guandeh17/Self-Forcing),
96
+ [Causal-Forcing](https://github.com/thu-ml/Causal-Forcing),
97
+ [Wan](https://github.com/Wan-Video/Wan2.1)) β€” please also respect their original licenses.
98
+
99
+ ## Citation
100
+
101
+ ```bibtex
102
+ @article{longliverag2026,
103
+ title = {LongLive-RAG: A General Retrieval-Augmented Framework for Long Video Generation},
104
+ author = {Hu, Qixin and Yang, Shuai and Huang, Wei and Han, Song and Chen, Yukang},
105
+ journal = {arXiv preprint arXiv:},
106
+ archivePrefix = {arXiv},
107
+ eprint = {},
108
+ year = {2026}
109
+ }
110
+ ```