Text-to-Video
MLX
apple-silicon

FastFrames-S-MLX

Unofficial MLX port of Apple's FastFrames-S (1.3B), a two-step autoregressive text-to-video model. It generates 832ร—480 video, 81 frames at 16 fps (about 5 seconds).

This repository contains the fastframes_mlx inference package and an example clip. It does not re-host model weights. At runtime the generate script downloads the official checkpoints (see below), converts them in memory to MLX, and runs on Apple Silicon.

A full converted MLX weight dump is not included: the stack is large (the umT5 encoder alone is about 11 GB), conversion is in-memory at load time rather than a saved MLX checkpoint format, and producing that dump was not practical on the machine used to publish this repo. The code loads the official weights directly.

Example clip, prompt "A corgi surfing a wave at sunset", seed 0, generated on a 64GB M5 Pro Mac mini:

If the player does not show, download examples/corgi.mp4.

Credit

This is an unofficial port. All credit for FastFrames goes to Apple and the authors: Gene Chou, Guandao Yang, Fujun Luan, Lars Mescheder, Wei Dong, Stephan R. Richter, and Vladlen Koltun.

Apple Machine Learning Research Model is licensed under the Apple Machine Learning Research Model License Agreement.

The name, trademarks, service marks, or logos of Apple may not be used to endorse or promote this port. "Apple" here is attribution only.

The port also depends on:

License

Two licenses apply, and they are not the same.

Piece License Where
This Python package (fastframes_mlx) MIT LICENSE
FastFrames-S KV refiner and pixel decoder weights, loaded from apple/FastFrames-S Apple Machine Learning Research Model License (apple-amlr) LICENSE_APPLE_AMLR
Official FastFrames Python/CUDA code Apple Sample Code License not vendored here; this is a reimplementation
Generator (generator_ema in zhuhz22/Causal-Forcing) Apache-2.0 upstream repo
umT5-XXL encoder, tokenizer, optional Wan VAE (Wan-AI/Wan2.1-T2V-1.3B) Apache-2.0 upstream repo

The Apple Machine Learning Research Model License allows use, modification, creation of model derivatives, and redistribution only for research purposes (non-commercial scientific research and academic development). Redistribution of those weights or derivatives must include a copy of that agreement, the attribution notice above, and a clear statement of modifications. This port does not redistribute those weights; if you use them, you must accept that license.

This repository identifies the port as an unofficial MLX reimplementation of the FastFrames-S inference path. Changes relative to the official CUDA release include: MLX kernels instead of PyTorch/CUDA, in-memory layout remapping of checkpoint keys and 3D conv weights, float64 tables computed in NumPy and stored as float32 so they can run on Metal, and optional post-training quantization of nn.Linear layers (--bits, untested).

Install and run

Needs an Apple Silicon Mac, Python 3.12, and ffmpeg.

brew install ffmpeg
uv venv --python 3.12
source .venv/bin/activate
uv pip install -e .
python -m fastframes_mlx.generate --prompt "A corgi surfing a wave at sunset" --out clip.mp4 --seed 0

The first run downloads official checkpoints into the Hugging Face cache, roughly:

  • apple/FastFrames-S fastframes-s/* (refiner + decoder, about 1 GB)
  • zhuhz22/Causal-Forcing causal-forcing++/framewise-2step.pt (the 1.3B generator)
  • Wan-AI/Wan2.1-T2V-1.3B models_t5_umt5-xxl-enc-bf16.pth (about 11 GB) and google/umt5-xxl/tokenizer.json

If a source repo requires login, run hf auth login first.

Useful flags: --latent-frames (default 21 โ†’ 81 RGB frames), --bits 4 or --bits 8 (untested), --decoder vae to decode with Wan2.1-VAE instead of the FastFrames pixel decoder, --smoke for a tiny random network that does not download weights.

Measured result (64GB M5 Pro Mac mini)

One run, seed 0, official prompt "A corgi surfing a wave at sunset", FastFrames-S, 832ร—480, 81 frames at 16 fps. --bits was not used. A background inference app was paused for this run.

Generation 30.4 s (30.43 s measured)
Per output frame 375.7 ms (~2.7 fps, about 6ร— off real time)
MLX peak memory 22.93 GB
RSS peak 30.37 GB
Wall clock 35.9 s (includes load)
Text encoder 871 ms
Decoder 1,116 ms

Full logs, the speed table, and the clip are in the benchmark dataset: pratikgx/fastframes-s-m5-pro-mac-mini-bench.

Caveats

  • Verified by 13 CPU parity tests against the official PyTorch reference, plus one Mac mini run.
  • No side-by-side output comparison versus the official CUDA implementation yet.
  • --bits quantization is untested.
  • Single run, single seed.
  • FastFrames-L (14B), the LingBot camera demo, and training are not implemented.

More detail on what the CPU parity tests did and did not cover is in docs/port-notes.md. The package in this repo is commit 99755c01bf645fce1ae95ef564104ce3fea666cf of the port (the revision used for the Mac mini run).

@article{chou2026fastframes,
  title   = {FastFrames: Video Generation at Interactive Latency},
  author  = {Chou, Gene and Yang, Guandao and Luan, Fujun and Mescheder, Lars and
             Dong, Wei and Richter, Stephan R. and Koltun, Vladlen},
  journal = {arXiv preprint arXiv:2611.23226},
  year    = {2026}
}
Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for pratikgx/FastFrames-S-MLX

Finetuned
(132)
this model

Paper for pratikgx/FastFrames-S-MLX