Blame for README.md

---
license: apache-2.0
library_name: pytorch
pipeline_tag: text-to-video
tags:
  - long-video-generation
  - autoregressive
  - retrieval-augmented-generation
  - video-generation
  - wan
  - longlive
  - arxiv:2606.02553
language:
  - en
---

# LongLive-RAG — Checkpoints & Toy Data

This repository hosts the model checkpoints, prompt files, and a toy latent set for  
**LongLive-RAG: A General Retrieval-Augmented Framework for Long Video Generation**.

- 📄 **Paper:** https://arxiv.org/abs/2606.02553
- 🤗 **HF Paper Page:** https://huggingface.co/papers/2606.02553
- 💻 **Code:** https://github.com/qixinhu11/LongLive-RAG
- 🔍 **TL;DR:** LongLive-RAG turns long video generation into a *retrieval problem*. An
  autoregressive (AR) video generator looks back over the video it has *already* generated
  and retrieves the most relevant past latents as additional context — reducing error
  accumulation, identity drift, and background flicker over long horizons, **without
  retraining the base generator**.

## What's in here

```text
checkpoints/
├── causal_forcing.pt              # Causal-Forcing AR backbone
├── self_forcing.pt                # Self-Forcing AR backbone
├── longlive_base.pt               # LongLive AR backbone
├── longlive_lora.pt               # LongLive LoRA, paired with longlive_base.pt
├── ae_latent_mem.pt               # Retrieval autoencoder, default for inference
├── moviegenbench_128_refined.txt  # 128 MovieGenBench prompts
└── vidprom_filtered_extended.txt  # Self-Forcing prompt pool for generate_latent.py

toydatasets/
└── latent_0000xx.pt               # Tiny example latent set for the training demo
```

- **AR backbones** — `causal_forcing.pt`, `self_forcing.pt`, and
  `longlive_base.pt` + `longlive_lora.pt` — are the frozen base generators that
  LongLive-RAG plugs into.
- **`ae_latent_mem.pt`** is the trainable retrieval encoder, implemented as a small latent
  autoencoder. This is the only component LongLive-RAG trains.
- **`toydatasets/`** contains a tiny set of clean latent blocks for smoke-testing the
  autoencoder training pipeline end-to-end.

> The base **WAN VAE** (`Wan-AI/Wan2.1-T2V-1.3B`) that LongLive-RAG operates in the latent
> space of is **not** included here. Please download it separately, as described below.

## Download

Everything restores into the expected layout of the code repository. Run the following
commands from the root of your local
[LongLive-RAG](https://github.com/qixinhu11/LongLive-RAG) checkout:

```bash
# Base WAN VAE — LongLive-RAG operates in its latent space
hf download Wan-AI/Wan2.1-T2V-1.3B --local-dir wan_models/Wan2.1-T2V-1.3B

# All LongLive-RAG assets — restores checkpoints/ and toydatasets/ in place
hf download qixinhu11/LongLive-RAG --local-dir . --include "checkpoints/*" "toydatasets/*"
```

The `--include` filter pulls only `checkpoints/` and `toydatasets/`, so it will not overwrite
your local `README.md`. Older setups can replace `hf download` with
`huggingface-cli download` using the same arguments.

To pull a single file instead:

```bash
hf download qixinhu11/LongLive-RAG checkpoints/ae_latent_mem.pt --local-dir .
```

## Usage

Clone the [code repository](https://github.com/qixinhu11/LongLive-RAG), download the assets
above, then run the shipped 3 × 2 grid, covering three AR backbones and two context-assembly
methods:

```bash
# Main result: LongLive backbone + LongLive-RAG retrieval
bash inference.sh longlive latentmem

# Baseline: native sliding-window context
bash inference.sh causal_forcing native
```

See the [GitHub README](https://github.com/qixinhu11/LongLive-RAG) for full installation,
inference, and training instructions.

## Paper

LongLive-RAG is described in the following paper:

**LongLive-RAG: A General Retrieval-Augmented Framework for Long Video Generation**  
Qixin Hu, Shuai Yang, Wei Huang, Song Han, Yukang Chen

- arXiv: https://arxiv.org/abs/2606.02553
- Hugging Face Papers: https://huggingface.co/papers/2606.02553

If you find this repository useful, please cite our paper.

## License

Released under the [Apache 2.0](https://github.com/qixinhu11/LongLive-RAG/blob/main/LICENSE)
license.

The included AR backbones and WAN VAE latent space derive from their respective upstream
projects:

- [LongLive](https://github.com/NVlabs/LongLive)
- [Self-Forcing](https://github.com/guandeh17/Self-Forcing)
- [Causal-Forcing](https://github.com/thu-ml/Causal-Forcing)
- [Wan](https://github.com/Wan-Video/Wan2.1)

Please also respect the original licenses of these upstream projects.

## Citation

```bibtex
@article{hu2026longliverag,
  title         = {LongLive-RAG: A General Retrieval-Augmented Framework for Long Video Generation},
  author        = {Hu, Qixin and Yang, Shuai and Huang, Wei and Han, Song and Chen, Yukang},
  journal       = {arXiv preprint arXiv:2606.02553},
  year          = {2026},
  eprint        = {2606.02553},
  archivePrefix = {arXiv},
  primaryClass  = {cs.CV}
}
```