Spaces:
Paused
Paused
Commit ·
7cca9ce
1
Parent(s): 3cc6f42
gradio.Server demo: real LTX-2 t2v backend + DreamVerse UI (ZeroGPU-shaped)
Browse filesapp.py: gradio.Server serving index.html at "/" and @app .api("/generate")
running the real FastVideo LTX-2 pipeline behind @spaces.GPU (queued,
single-GPU safe). Validated end-to-end on the RTX 6000: model load ~26s
cached, generation ~9s for 512x768x65f with audio; verified via gradio_client.
Written in the ZeroGPU idiom (import spaces first, GPU-only work in
@spaces.GPU, generator cached, lazy load for spawn-safety). ZEROGPU_NOTES.md
documents the port path (in-process pipeline + aokit weight-less DiT blocks)
and the dev-mode deploy (restart, not rebuild; ephemeral storage).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- ZEROGPU_NOTES.md +80 -0
- app.py +145 -5
- requirements.txt +41 -0
ZEROGPU_NOTES.md
ADDED
|
@@ -0,0 +1,80 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# DreamVerse LTX-2 · gradio.Server demo — notes
|
| 2 |
+
|
| 3 |
+
A custom **DreamVerse**-styled frontend (vanilla HTML/CSS/JS in `index.html`)
|
| 4 |
+
served by **`gradio.Server`**, with a real **FastVideo LTX-2** text-to-video
|
| 5 |
+
backend behind `@app.api("/generate")` and `@spaces.GPU`.
|
| 6 |
+
|
| 7 |
+
## Architecture (per the gradio.Server blog)
|
| 8 |
+
|
| 9 |
+
```
|
| 10 |
+
index.html ──Gradio JS client──▶ @app.api("/generate") ──▶ @spaces.GPU
|
| 11 |
+
(DreamVerse UI) (queued, single-GPU safe) real LTX-2
|
| 12 |
+
▲ │
|
| 13 |
+
└────────────── transparent mp4 / video url ◀─────────────┘
|
| 14 |
+
@app.get("/") serves index.html ; @app.get("/healthz") for readiness
|
| 15 |
+
```
|
| 16 |
+
|
| 17 |
+
`@app.api()` (not a plain `@app.post()`) puts generation through Gradio's
|
| 18 |
+
queue, so concurrent users are serialized onto the one GPU instead of
|
| 19 |
+
colliding, and the endpoint is also callable via `gradio_client`
|
| 20 |
+
(`api_name="/generate"`).
|
| 21 |
+
|
| 22 |
+
## Validated (on the RTX PRO 6000 Blackwell dev Space)
|
| 23 |
+
|
| 24 |
+
- `gradio.Server` boots and serves the DreamVerse UI at `/` instantly.
|
| 25 |
+
- `/generate` ran the **real** `FastVideo/LTX2-Distilled-Diffusers` pipeline:
|
| 26 |
+
model load **26 s** (cached) / **77 s** cold, generation **~9 s** for a
|
| 27 |
+
512×768, 65-frame clip with synced audio (h264 + aac mp4). First call ~40 s
|
| 28 |
+
total, subsequent calls ~9 s (model stays warm).
|
| 29 |
+
- Verified end-to-end through `gradio_client.predict("/generate", ...)`.
|
| 30 |
+
- Attention forced to **TORCH_SDPA** (sm_120 can't run FA4 — see the aokit
|
| 31 |
+
digest), `num_gpus=1`.
|
| 32 |
+
|
| 33 |
+
## ZeroGPU porting
|
| 34 |
+
|
| 35 |
+
The app is already in the ZeroGPU idiom: `import spaces` first, GPU work only
|
| 36 |
+
inside `@spaces.GPU`, generator created once and cached. On a dedicated GPU
|
| 37 |
+
`@spaces.GPU` is a passthrough so this just works.
|
| 38 |
+
|
| 39 |
+
The one real blocker for actual ZeroGPU hardware: FastVideo's `VideoGenerator`
|
| 40 |
+
spawns a **persistent** GPU worker (`MultiprocExecutor`, start method
|
| 41 |
+
`spawn`). ZeroGPU instead gives you the GPU only inside a forked `@spaces.GPU`
|
| 42 |
+
call and snapshots **module-scope** CUDA state across forks — a persistent
|
| 43 |
+
external worker process doesn't fit that model, and a `spawn` worker re-imports
|
| 44 |
+
the app module (which is why the generator load is lazy, never at module
|
| 45 |
+
scope).
|
| 46 |
+
|
| 47 |
+
The ZeroGPU-native path:
|
| 48 |
+
1. Run the LTX-2 pipeline **in-process** (no MultiprocExecutor), loading
|
| 49 |
+
components at module scope so `spaces` can snapshot them.
|
| 50 |
+
2. Use the **aokit-compiled, weight-less** DiT blocks from
|
| 51 |
+
`package/{cu128,cu130}/` for the transformer — aokit is built exactly for
|
| 52 |
+
ZeroGPU (lazy `ContextVar` execution, fork cleanup), so the heavy compiled
|
| 53 |
+
graph loads correctly inside each `@spaces.GPU` fork.
|
| 54 |
+
|
| 55 |
+
That reuses the artifacts from the earlier task and removes the persistent-worker
|
| 56 |
+
assumption.
|
| 57 |
+
|
| 58 |
+
## Deploy (dev mode, ephemeral storage — be careful)
|
| 59 |
+
|
| 60 |
+
The Space supervisor runs, on rebuild only:
|
| 61 |
+
`git reset --hard $COMMIT_SHA && git clean -fdx && python app.py`.
|
| 62 |
+
So **`git clean -fdx` wipes any uncommitted file** and a full rebuild wipes the
|
| 63 |
+
ephemeral env (the source-installed FastVideo, the CUDA-matched torch, and the
|
| 64 |
+
~38 GB cached weights). Therefore:
|
| 65 |
+
|
| 66 |
+
- Everything is committed to git (survives the reset).
|
| 67 |
+
- Deploy by **restarting the app process** in the warm env — not a clean
|
| 68 |
+
rebuild. On restart `python app.py` picks up the new app; the first
|
| 69 |
+
`/generate` lazy-loads the (already cached) model in ~30 s.
|
| 70 |
+
- `requirements.txt` is best-effort for a cold rebuild; FastVideo (main) isn't
|
| 71 |
+
on PyPI and pins `gradio==5.32.0` (conflicts with the gradio≥6 that
|
| 72 |
+
`gradio.Server` needs), so the warm env installs it via
|
| 73 |
+
`pip install -e ./FastVideo --no-deps`.
|
| 74 |
+
|
| 75 |
+
## Render settings (env-overridable in `app.py`)
|
| 76 |
+
|
| 77 |
+
`DREAMVERSE_HEIGHT=512 DREAMVERSE_WIDTH=768 DREAMVERSE_FRAMES=65`
|
| 78 |
+
`DREAMVERSE_STEPS=5 DREAMVERSE_FPS=24 DREAMVERSE_GPU_DURATION=300`
|
| 79 |
+
(`num_frames-1` must be divisible by 8). Bump toward DreamVerse's
|
| 80 |
+
1920×1088 / 121-frame profile for higher quality at higher latency.
|
app.py
CHANGED
|
@@ -1,7 +1,147 @@
|
|
| 1 |
-
|
| 2 |
|
| 3 |
-
|
| 4 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
| 5 |
|
| 6 |
-
|
| 7 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""DreamVerse-style LTX-2 text-to-video demo on `gradio.Server`.
|
| 2 |
|
| 3 |
+
Architecture (per the gradio.Server blog):
|
| 4 |
+
* `@app.get("/")` serves the custom DreamVerse UI (index.html)
|
| 5 |
+
* `@app.api("/generate")` runs REAL FastVideo LTX-2 text-to-video and
|
| 6 |
+
returns an mp4. It goes through Gradio's queue, so concurrent users are
|
| 7 |
+
serialized onto the single GPU instead of colliding.
|
| 8 |
+
* `@spaces.GPU` ZeroGPU allocation around the GPU work.
|
| 9 |
|
| 10 |
+
The heavy FastVideo `VideoGenerator` (the same engine DreamVerse runs) is
|
| 11 |
+
initialized lazily on the first request so the Server boots instantly and
|
| 12 |
+
serves the UI immediately — the model load (and a slow first generation,
|
| 13 |
+
DreamVerse-style warmup) happens on first `/generate`.
|
| 14 |
+
|
| 15 |
+
Notes for this box:
|
| 16 |
+
* RTX PRO 6000 Blackwell (sm_120): FA4/CuTeDSL kernels aren't built for it,
|
| 17 |
+
so we force `FASTVIDEO_ATTENTION_BACKEND=TORCH_SDPA`.
|
| 18 |
+
* The compiled aokit LTX-2 block lives under `package/{cu128,cu130}/` — it
|
| 19 |
+
is the ZeroGPU-ready (lazy, weight-less) piece; this app runs the full
|
| 20 |
+
FastVideo pipeline via its own persistent GPU worker.
|
| 21 |
+
"""
|
| 22 |
+
|
| 23 |
+
import os
|
| 24 |
+
import threading
|
| 25 |
+
import time
|
| 26 |
+
import uuid
|
| 27 |
+
import tempfile
|
| 28 |
+
|
| 29 |
+
# Must be set before any fastvideo / CUDA import.
|
| 30 |
+
os.environ.setdefault("FASTVIDEO_ATTENTION_BACKEND", "TORCH_SDPA")
|
| 31 |
+
os.environ.setdefault("TOKENIZERS_PARALLELISM", "false")
|
| 32 |
+
|
| 33 |
+
import spaces
|
| 34 |
+
from gradio import Server
|
| 35 |
+
from gradio.data_classes import FileData
|
| 36 |
+
from fastapi.responses import HTMLResponse
|
| 37 |
+
|
| 38 |
+
MODEL_ID = os.getenv("LTX2_MODEL_PATH", "FastVideo/LTX2-Distilled-Diffusers")
|
| 39 |
+
HERE = os.path.dirname(os.path.abspath(__file__))
|
| 40 |
+
OUT_DIR = os.path.join(tempfile.gettempdir(), "dreamverse_outputs")
|
| 41 |
+
os.makedirs(OUT_DIR, exist_ok=True)
|
| 42 |
+
|
| 43 |
+
# Demo render settings — modest for single-GPU latency. Override via env.
|
| 44 |
+
# num_frames must satisfy (num_frames - 1) % 8 == 0.
|
| 45 |
+
GEN_HEIGHT = int(os.getenv("DREAMVERSE_HEIGHT", "512"))
|
| 46 |
+
GEN_WIDTH = int(os.getenv("DREAMVERSE_WIDTH", "768"))
|
| 47 |
+
GEN_FRAMES = int(os.getenv("DREAMVERSE_FRAMES", "65"))
|
| 48 |
+
GEN_STEPS = int(os.getenv("DREAMVERSE_STEPS", "5"))
|
| 49 |
+
GEN_FPS = int(os.getenv("DREAMVERSE_FPS", "24"))
|
| 50 |
+
GPU_DURATION = int(os.getenv("DREAMVERSE_GPU_DURATION", "300"))
|
| 51 |
+
|
| 52 |
+
app = Server()
|
| 53 |
+
|
| 54 |
+
# --- ZeroGPU porting notes ---------------------------------------------
|
| 55 |
+
# This app is written in the ZeroGPU idiom so the port is mechanical:
|
| 56 |
+
# * `import spaces` happens before any torch/CUDA import (top of file);
|
| 57 |
+
# * the only GPU-touching code lives inside the `@spaces.GPU` function;
|
| 58 |
+
# * the generator is created once and reused (cached in a module global).
|
| 59 |
+
# On this dedicated RTX 6000 dev Space `@spaces.GPU` is a passthrough, so the
|
| 60 |
+
# cached generator persists in-process and the first call pays the load cost.
|
| 61 |
+
#
|
| 62 |
+
# The ONE thing to change for real ZeroGPU: FastVideo's `VideoGenerator`
|
| 63 |
+
# spawns a *persistent* GPU worker (MultiprocExecutor, start method = spawn).
|
| 64 |
+
# ZeroGPU instead hands you the GPU only inside a forked `@spaces.GPU` call
|
| 65 |
+
# and snapshots module-scope CUDA state across forks — a persistent external
|
| 66 |
+
# worker doesn't fit that. The ZeroGPU-native path is an in-process LTX-2
|
| 67 |
+
# pipeline whose DiT blocks are the aokit-compiled, weight-less artifacts in
|
| 68 |
+
# `package/{cu128,cu130}/` (aokit is built for exactly this: lazy, fork-safe
|
| 69 |
+
# loading). See ZEROGPU_NOTES.md. Until then this runs great on a dedicated
|
| 70 |
+
# GPU and keeps the ZeroGPU-shaped surface.
|
| 71 |
+
_generator = None
|
| 72 |
+
_gen_lock = threading.Lock()
|
| 73 |
+
|
| 74 |
+
|
| 75 |
+
def _get_generator():
|
| 76 |
+
"""Lazily build the FastVideo LTX-2 generator (single GPU, in-process worker).
|
| 77 |
+
|
| 78 |
+
Lazy (not module-scope) on purpose: FastVideo's worker uses `spawn`, which
|
| 79 |
+
re-imports this module in the child; a module-scope load would re-trigger
|
| 80 |
+
here. Building inside the request keeps import side-effect-free.
|
| 81 |
+
"""
|
| 82 |
+
global _generator
|
| 83 |
+
if _generator is not None:
|
| 84 |
+
return _generator
|
| 85 |
+
with _gen_lock:
|
| 86 |
+
if _generator is not None:
|
| 87 |
+
return _generator
|
| 88 |
+
from fastvideo import VideoGenerator
|
| 89 |
+
print(f"[dreamverse] loading {MODEL_ID} (first request — this is slow) ...", flush=True)
|
| 90 |
+
t0 = time.perf_counter()
|
| 91 |
+
gen = VideoGenerator.from_pretrained(MODEL_ID, num_gpus=1)
|
| 92 |
+
print(f"[dreamverse] generator ready in {time.perf_counter()-t0:.0f}s", flush=True)
|
| 93 |
+
_generator = gen
|
| 94 |
+
return _generator
|
| 95 |
+
|
| 96 |
+
|
| 97 |
+
@spaces.GPU(duration=GPU_DURATION)
|
| 98 |
+
def generate_video(prompt: str) -> str:
|
| 99 |
+
"""Run real LTX-2 text-to-video; return the path to an mp4."""
|
| 100 |
+
gen = _get_generator()
|
| 101 |
+
out_path = os.path.join(OUT_DIR, f"{uuid.uuid4().hex}.mp4")
|
| 102 |
+
t0 = time.perf_counter()
|
| 103 |
+
gen.generate_video(
|
| 104 |
+
prompt=prompt,
|
| 105 |
+
output_path=out_path,
|
| 106 |
+
save_video=True,
|
| 107 |
+
height=GEN_HEIGHT,
|
| 108 |
+
width=GEN_WIDTH,
|
| 109 |
+
num_frames=GEN_FRAMES,
|
| 110 |
+
num_inference_steps=GEN_STEPS,
|
| 111 |
+
fps=GEN_FPS,
|
| 112 |
+
guidance_scale=1.0,
|
| 113 |
+
seed=int(time.time()) % 100000,
|
| 114 |
+
)
|
| 115 |
+
print(f"[dreamverse] generated in {time.perf_counter()-t0:.1f}s -> {out_path}", flush=True)
|
| 116 |
+
if not os.path.exists(out_path):
|
| 117 |
+
raise RuntimeError("Generation did not produce an output file")
|
| 118 |
+
return out_path
|
| 119 |
+
|
| 120 |
+
|
| 121 |
+
@app.api(name="generate")
|
| 122 |
+
def generate(prompt: str) -> FileData:
|
| 123 |
+
"""Text-to-video endpoint. Goes through Gradio's queue (single-GPU safe).
|
| 124 |
+
|
| 125 |
+
Callable from the browser via the Gradio JS client and from Python via
|
| 126 |
+
`gradio_client` (api_name="/generate").
|
| 127 |
+
"""
|
| 128 |
+
prompt = (prompt or "").strip()
|
| 129 |
+
if not prompt:
|
| 130 |
+
raise ValueError("Prompt must be non-empty")
|
| 131 |
+
path = generate_video(prompt)
|
| 132 |
+
return FileData(path=path)
|
| 133 |
+
|
| 134 |
+
|
| 135 |
+
@app.get("/", response_class=HTMLResponse)
|
| 136 |
+
async def homepage():
|
| 137 |
+
with open(os.path.join(HERE, "index.html"), "r", encoding="utf-8") as f:
|
| 138 |
+
return f.read()
|
| 139 |
+
|
| 140 |
+
|
| 141 |
+
@app.get("/healthz")
|
| 142 |
+
async def healthz():
|
| 143 |
+
return {"ok": True, "model": MODEL_ID, "loaded": _generator is not None}
|
| 144 |
+
|
| 145 |
+
|
| 146 |
+
if __name__ == "__main__":
|
| 147 |
+
app.launch(server_name="0.0.0.0", server_port=int(os.getenv("PORT", "7860")), show_error=True)
|
requirements.txt
ADDED
|
@@ -0,0 +1,41 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# DreamVerse LTX-2 demo on gradio.Server
|
| 2 |
+
#
|
| 3 |
+
# NOTE ON DEPLOY: this Space runs in dev mode with EPHEMERAL storage. The live
|
| 4 |
+
# environment is pre-warmed (FastVideo installed from source, torch built for
|
| 5 |
+
# the box's CUDA, the LTX-2 weights cached) and the demo is deployed by
|
| 6 |
+
# RESTARTING the app process — not by a clean rebuild. A clean rebuild from
|
| 7 |
+
# this file is best-effort only, because FastVideo (main) is not on PyPI and
|
| 8 |
+
# pins gradio==5.32.0, which conflicts with the gradio>=6 that gradio.Server
|
| 9 |
+
# needs. See ZEROGPU_NOTES.md.
|
| 10 |
+
|
| 11 |
+
# Web / serving (gradio.Server lives in gradio>=6)
|
| 12 |
+
gradio>=6.14.0
|
| 13 |
+
spaces
|
| 14 |
+
|
| 15 |
+
# Torch for the box's CUDA (cu128 shown; use cu130 to match a CUDA-13 runtime).
|
| 16 |
+
--extra-index-url https://download.pytorch.org/whl/cu128
|
| 17 |
+
torch==2.11.0
|
| 18 |
+
torchvision
|
| 19 |
+
|
| 20 |
+
# FastVideo (LTX-2 lives on main, not in the 0.1.7 PyPI release). Installed in
|
| 21 |
+
# the warm env via: pip install -e ./FastVideo --no-deps
|
| 22 |
+
# (kept here as a reference; --no-deps avoids the gradio==5.32 / torch pins)
|
| 23 |
+
# fastvideo @ git+https://github.com/hao-ai-lab/FastVideo.git
|
| 24 |
+
|
| 25 |
+
# FastVideo runtime deps actually needed to load + run LTX-2 inference
|
| 26 |
+
transformers==4.57.3
|
| 27 |
+
diffusers>=0.33.1
|
| 28 |
+
safetensors
|
| 29 |
+
sentencepiece
|
| 30 |
+
tokenizers>=0.20.1
|
| 31 |
+
accelerate==1.0.1
|
| 32 |
+
einops
|
| 33 |
+
omegaconf
|
| 34 |
+
loguru
|
| 35 |
+
cloudpickle
|
| 36 |
+
imageio
|
| 37 |
+
imageio-ffmpeg
|
| 38 |
+
scipy
|
| 39 |
+
ftfy
|
| 40 |
+
huggingface_hub
|
| 41 |
+
hf_transfer
|