|
Download README.md from apple/FastFrames-L: direct link, hf CLI and curl.
- Browser
- Download file 5.79 kB
-
https://huggingface.co/apple/FastFrames-L/resolve/main/README.md
- Command line
-
hf download hf://apple/FastFrames-L/README.md
-
curl -L -o README.md https://huggingface.co/apple/FastFrames-L/resolve/main/README.md
5.79 kB
| license: apple-amlr | |
| pipeline_tag: text-to-video | |
| base_model: | |
| - Wan-AI/Wan2.1-T2V-14B | |
| # FastFrames: Video Generation at Interactive Latency | |
| [](https://apple-aiml-research.github.io/ml-fastframes/) | |
| [](https://arxiv.org/abs/2611.23226) | |
| [](https://huggingface.co/apple/FastFrames-S) | |
| [](https://huggingface.co/apple/FastFrames-L) | |
| This repository contains the official implementation of *FastFrames: Video Generation at | |
| Interactive Latency*. We release [FastFrames-S](https://huggingface.co/apple/FastFrames-S) | |
| (1.3B) and [FastFrames-L](https://huggingface.co/apple/FastFrames-L) (14B). Both are | |
| autoregressive, two-step video diffusion models that generate 480p video at interactive | |
| latency. We also release an optimized version of LingBot 1.3B that is 16x faster than the | |
| official model and allows fluid camera-conditioned control. | |
| - [Latency profiling](#latency-profiling) | |
| - [Interactive demo with LingBot](#interactive-demo-with-lingbot) | |
| - [T2V generation](#t2v-generation) | |
| - [VBench evaluation](#vbench-evaluation) | |
| - [Training](#training) | |
| - [Citation](#citation) | |
| [](assets/fastframes_demo.mp4) | |
| ## Installation | |
| Linux x86-64 with an NVIDIA GPU and a CUDA 12.8-compatible driver. From the repository root: | |
| ```bash | |
| uv sync | |
| ``` | |
| Download FastFrames and third-party weights using the [checkpoint guide](https://github.com/apple-aiml-research/ml-fastframes/blob/main/docs/installation.md#checkpoints). | |
| ## Latency Profiling | |
| ```bash | |
| uv run python evaluation/latency/benchmark.py ff-s | |
| uv run python evaluation/latency/benchmark.py ff-l --gpus 8 | |
| ``` | |
| Example component timings for **FastFrames-S, 1 GPU, 480p**, with a full KV cache. | |
| | Component | Calls | Latency per call | Total latency | | |
| |---|---:|---:|---:| | |
| | DiT evaluation, t=1000 | 1 | 16.1 ms | 16.1 ms | | |
| | DiT evaluation, t=833 | 1 | 16.2 ms | 16.2 ms | | |
| | KV refinement | 1 | 2.2 ms | 2.2 ms | | |
| | Pixel decoder | 1 | 0.9 ms | 0.9 ms | | |
| | **Total** | | | **35.4 ms** (paper: 36 ms) | | |
| See the [profiling guide](https://github.com/apple-aiml-research/ml-fastframes/blob/main/docs/inference.md#profiling) for details and baselines. | |
| ## Interactive Demo with LingBot | |
| Complete the [demo setup](https://github.com/apple-aiml-research/ml-fastframes/blob/main/docs/lingbot.md#interactive-demo-13b), then launch one of: | |
| ```bash | |
| CUDA_VISIBLE_DEVICES=0 uv run --extra lingbot --extra demo python apps/lingbot/demo.py --gpus 1 | |
| CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7 uv run --extra lingbot --extra demo python apps/lingbot/demo.py --gpus 8 | |
| ``` | |
| and open `http://localhost:8000`. | |
| ## T2V Generation | |
| ```bash | |
| uv run python scripts/generate.py --model ff-s --checkpoint-dir checkpoints \ | |
| --prompt "A corgi surfing a wave at sunset" --seed 0 --output outputs/corgi.mp4 | |
| ``` | |
| Outputs 832 × 480 video, 81 frames at 16 fps. Use `--model ff-l` for FastFrames-L. | |
| To generate the videos on our project page (20+ videos): | |
| ```bash | |
| uv run python scripts/generate_project_page.py --model ff-s --output-dir outputs/project_page/ff-s | |
| uv run python scripts/generate_project_page.py --model ff-l --output-dir outputs/project_page/ff-l | |
| ``` | |
| See the [generation guide](https://github.com/apple-aiml-research/ml-fastframes/blob/main/docs/inference.md#generation) for details. | |
| ## VBench Evaluation | |
| Follow the [evaluation guide](https://github.com/apple-aiml-research/ml-fastframes/blob/main/docs/inference.md#evaluation) to reproduce Table 1's | |
| Quality / Semantic / Total scores. | |
| ## Training | |
| ### [Causal Forcing++ -> FastFrames-S](https://github.com/apple-aiml-research/ml-fastframes/blob/main/docs/training.md#causal-forcing---fastframes-s) | |
| FastFrames-S keeps the two-step frame-wise Causal Forcing++ generator frozen and trains the | |
| networks around it. | |
| ### [Wan2.1-14B -> FastFrames-L](https://github.com/apple-aiml-research/ml-fastframes/blob/main/docs/training.md#wan21-14b---fastframes-l) | |
| We first distill Wan2.1-14B into a two-step causal model following the Causal Forcing++ recipe, | |
| then train its pixel decoder and KV refinement network as for FastFrames-S. We release all | |
| checkpoints: 1) a causal teacher trained on latents generated by Wan2.1-14B; 2) the | |
| consistency-distilled model; 3) the DMD-distilled model; 4) the final generator, pixel decoder | |
| and KV refinement network. | |
| ## Citation | |
| If you find this work useful in your research, please cite: | |
| ```bibtex | |
| @article{chou2026fastframes, | |
| title = {FastFrames: Video Generation at Interactive Latency}, | |
| author = {Chou, Gene and Yang, Guandao and Luan, Fujun and Mescheder, Lars and | |
| Dong, Wei and Richter, Stephan R. and Koltun, Vladlen}, | |
| journal = {arXiv preprint arXiv:2611.23226}, | |
| year = {2026} | |
| } | |
| ``` | |
| ## License and Acknowledgments | |
| LingBot-World v2 code and weights follow their upstream [CC BY-NC-SA 4.0 license](https://github.com/apple-aiml-research/ml-fastframes/blob/main/LICENSE_LINGBOT). | |
| All other FastFrames code is licensed under the [Apple Sample Code License](https://github.com/apple-aiml-research/ml-fastframes/blob/main/LICENSE). | |
| Also see [ACKNOWLEDGEMENTS](https://github.com/apple-aiml-research/ml-fastframes/blob/main/ACKNOWLEDGEMENTS) for third-party code. | |
| We thank the authors of [Wan2.1](https://github.com/Wan-Video/Wan2.1), | |
| [Self-Forcing](https://github.com/guandeh17/Self-Forcing), and | |
| [Causal Forcing++](https://github.com/thu-ml/Causal-Forcing) for releasing their code, on which the | |
| FastFrames code and models build. | |