---
title: FrameForge
emoji: 🖼️
colorFrom: purple
colorTo: indigo
sdk: gradio
app_file: app.py
pinned: false
tags:
  - ml-intern
  - frameforge
  - image-editing
  - hunyuanvideo
---

[![Built with ML Intern](https://huggingface.co/datasets/huggingface/badges/resolve/main/built-with-ml-intern-sm.svg)](https://hf.co/chat?mode=ml-intern)

# FrameForge — demo

Unofficial interactive demo for **FrameForge: Harnessing Priors of Video Foundation
Models for Unified Image Generation** ([arXiv 2610.36545](https://arxiv.org/abs/2610.36545)),
built on the authors' released
[inference code](https://github.com/Lyne1/FrameForge) and
[official LoRA weights](https://huggingface.co/lyneeeeeeeee/FrameForge) over
[HunyuanVideo 1.5](https://huggingface.co/tencent/HunyuanVideo-1.5).

Three modes in one model:

- **Instruction-guided editing** — "change the sky to sunset" style edits
- **Subject-driven generation** — place a subject from a photo into a new scene
- **Canny-to-image** — generate from an edge map (auto-extraction available)

## Running it

Built for **ZeroGPU** (`xlarge` slices, 96 GB): generation functions are
decorated with `@spaces.GPU`, and torch is pinned to **2.8.0** (the closest
ZeroGPU-supported version to the authors' 2.6.0 pin). It also runs unchanged
on a persistent GPU (A100 40 GB+ recommended; `FF_OFFLOAD=true` for smaller
GPUs at a large speed cost) — the decorator is a no-op off ZeroGPU.

**First run downloads ~40 GB of checkpoints.** (byt5-small/Glyph-SDXL-v2
from the authors' README are skipped — the released config sets `disable_by5=True`,
so they are never loaded.) On Spaces, attach persistent
storage and set `FF_CKPT_DIR=/data/ckpts` so this happens once.

**Quota note:** a 50-step generation takes a few minutes of GPU time and
`xlarge` consumes 2× daily quota — free accounts (5 min/day) get roughly one
image per day, PRO accounts (40 min/day) get several.

Required secret: `HF_TOKEN` — the pipeline pulls the **gated**
[FLUX.1-Redux-dev](https://huggingface.co/black-forest-labs/FLUX.1-Redux-dev)
vision encoder, so the token's account must have accepted its license.

Useful env vars: `FF_CKPT_DIR` (checkpoint root), `FF_LORA_PATH`,
`FF_REPO_DIR` (code cache), `FF_OFFLOAD` (`true`/`false`).

## Local run

```bash
pip install -r requirements.txt
python app.py
```

## Notes

- Outputs are 512×512; ~50 denoising steps on a video-model backbone, so expect
  a couple of minutes per image.
- License: Tencent Hunyuan Community License (see the base model and the
  [FrameForge repo](https://github.com/Lyne1/FrameForge)) — review before
  redistribution or commercial use.
- If you use this in research, cite the paper: https://arxiv.org/abs/2610.36545