Text-to-Video
Safetensors
File size: 5,794 Bytes
18f30cb
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
---
license: apple-amlr
pipeline_tag: text-to-video
base_model:
- Wan-AI/Wan2.1-T2V-14B
---
# FastFrames: Video Generation at Interactive Latency

[![Project Page](https://img.shields.io/badge/Project-Page-green)](https://apple-aiml-research.github.io/ml-fastframes/)
[![arXiv](https://img.shields.io/badge/arXiv-2611.23226-b31b1b.svg)](https://arxiv.org/abs/2611.23226)
[![FastFrames-S](https://img.shields.io/badge/%F0%9F%A4%97_Model-FastFrames--S-yellow)](https://huggingface.co/apple/FastFrames-S)
[![FastFrames-L](https://img.shields.io/badge/%F0%9F%A4%97_Model-FastFrames--L-yellow)](https://huggingface.co/apple/FastFrames-L)

This repository contains the official implementation of *FastFrames: Video Generation at
Interactive Latency*. We release [FastFrames-S](https://huggingface.co/apple/FastFrames-S)
(1.3B) and [FastFrames-L](https://huggingface.co/apple/FastFrames-L) (14B). Both are
autoregressive, two-step video diffusion models that generate 480p video at interactive
latency. We also release an optimized version of LingBot 1.3B that is 16x faster than the
official model and allows fluid camera-conditioned control.

- [Latency profiling](#latency-profiling)
- [Interactive demo with LingBot](#interactive-demo-with-lingbot)
- [T2V generation](#t2v-generation)
- [VBench evaluation](#vbench-evaluation)
- [Training](#training)
- [Citation](#citation)

[![FastFrames demo: LingBot camera control with and without FastFrames optimizations](assets/fastframes_demo_preview.gif)](assets/fastframes_demo.mp4)

## Installation

Linux x86-64 with an NVIDIA GPU and a CUDA 12.8-compatible driver. From the repository root:

```bash
uv sync
```

Download FastFrames and third-party weights using the [checkpoint guide](https://github.com/apple-aiml-research/ml-fastframes/blob/main/docs/installation.md#checkpoints).

## Latency Profiling

```bash
uv run python evaluation/latency/benchmark.py ff-s
uv run python evaluation/latency/benchmark.py ff-l --gpus 8
```

Example component timings for **FastFrames-S, 1 GPU, 480p**, with a full KV cache.

| Component | Calls | Latency per call | Total latency |
|---|---:|---:|---:|
| DiT evaluation, t=1000 | 1 | 16.1 ms | 16.1 ms |
| DiT evaluation, t=833 | 1 | 16.2 ms | 16.2 ms |
| KV refinement | 1 | 2.2 ms | 2.2 ms |
| Pixel decoder | 1 | 0.9 ms | 0.9 ms |
| **Total** | | | **35.4 ms** (paper: 36 ms) |

See the [profiling guide](https://github.com/apple-aiml-research/ml-fastframes/blob/main/docs/inference.md#profiling) for details and baselines.

## Interactive Demo with LingBot

Complete the [demo setup](https://github.com/apple-aiml-research/ml-fastframes/blob/main/docs/lingbot.md#interactive-demo-13b), then launch one of:

```bash
CUDA_VISIBLE_DEVICES=0 uv run --extra lingbot --extra demo python apps/lingbot/demo.py --gpus 1
CUDA_VISIBLE_DEVICES=0,1,2,3,4,5,6,7 uv run --extra lingbot --extra demo python apps/lingbot/demo.py --gpus 8
```

and open `http://localhost:8000`.

## T2V Generation

```bash
uv run python scripts/generate.py --model ff-s --checkpoint-dir checkpoints \
    --prompt "A corgi surfing a wave at sunset" --seed 0 --output outputs/corgi.mp4
```

Outputs 832 × 480 video, 81 frames at 16 fps. Use `--model ff-l` for FastFrames-L.

To generate the videos on our project page (20+ videos):

```bash
uv run python scripts/generate_project_page.py --model ff-s --output-dir outputs/project_page/ff-s
uv run python scripts/generate_project_page.py --model ff-l --output-dir outputs/project_page/ff-l
```

See the [generation guide](https://github.com/apple-aiml-research/ml-fastframes/blob/main/docs/inference.md#generation) for details.

## VBench Evaluation

Follow the [evaluation guide](https://github.com/apple-aiml-research/ml-fastframes/blob/main/docs/inference.md#evaluation) to reproduce Table 1's
Quality / Semantic / Total scores.

## Training

### [Causal Forcing++ -> FastFrames-S](https://github.com/apple-aiml-research/ml-fastframes/blob/main/docs/training.md#causal-forcing---fastframes-s)

FastFrames-S keeps the two-step frame-wise Causal Forcing++ generator frozen and trains the
networks around it.

### [Wan2.1-14B -> FastFrames-L](https://github.com/apple-aiml-research/ml-fastframes/blob/main/docs/training.md#wan21-14b---fastframes-l)

We first distill Wan2.1-14B into a two-step causal model following the Causal Forcing++ recipe,
then train its pixel decoder and KV refinement network as for FastFrames-S. We release all
checkpoints: 1) a causal teacher trained on latents generated by Wan2.1-14B; 2) the
consistency-distilled model; 3) the DMD-distilled model; 4) the final generator, pixel decoder
and KV refinement network.

## Citation

If you find this work useful in your research, please cite:

```bibtex
@article{chou2026fastframes,
  title   = {FastFrames: Video Generation at Interactive Latency},
  author  = {Chou, Gene and Yang, Guandao and Luan, Fujun and Mescheder, Lars and
             Dong, Wei and Richter, Stephan R. and Koltun, Vladlen},
  journal = {arXiv preprint arXiv:2611.23226},
  year    = {2026}
}
```

## License and Acknowledgments

LingBot-World v2 code and weights follow their upstream [CC BY-NC-SA 4.0 license](https://github.com/apple-aiml-research/ml-fastframes/blob/main/LICENSE_LINGBOT).
All other FastFrames code is licensed under the [Apple Sample Code License](https://github.com/apple-aiml-research/ml-fastframes/blob/main/LICENSE).
Also see [ACKNOWLEDGEMENTS](https://github.com/apple-aiml-research/ml-fastframes/blob/main/ACKNOWLEDGEMENTS) for third-party code.

We thank the authors of [Wan2.1](https://github.com/Wan-Video/Wan2.1),
[Self-Forcing](https://github.com/guandeh17/Self-Forcing), and
[Causal Forcing++](https://github.com/thu-ml/Causal-Forcing) for releasing their code, on which the
FastFrames code and models build.