Spaces:
Running on Zero
Running on Zero
Stream: render every edit (no debounce), 0.5 s segments, 0.6 s lead; reuse KV cache
Browse files
README.md
CHANGED
|
@@ -22,8 +22,8 @@ models:
|
|
| 22 |
Real-time, prompt-reactive text-to-video with [apple/FastFrames-S](https://huggingface.co/apple/FastFrames-S).
|
| 23 |
Press Go live and keep editing the prompt; the stream follows it.
|
| 24 |
|
| 25 |
-
- One held ZeroGPU session per user streams
|
| 26 |
-
-
|
| 27 |
and the shot stays comparable.
|
| 28 |
- An idle prompt keeps streaming past 5 s with a sink + rolling KV window (keys re-rotated so RoPE positions stay in
|
| 29 |
0..20) and gets a fresh scene every 20 s (adjustable) to stop long-rollout drift.
|
|
|
|
| 22 |
Real-time, prompt-reactive text-to-video with [apple/FastFrames-S](https://huggingface.co/apple/FastFrames-S).
|
| 23 |
Press Go live and keep editing the prompt; the stream follows it.
|
| 24 |
|
| 25 |
+
- One held ZeroGPU session per user streams 0.5 s segments; prompt edits reach it through a per-session file.
|
| 26 |
+
- Every prompt edit regenerates the scene from frame 0 with the same seed, so the new prompt is followed exactly
|
| 27 |
and the shot stays comparable.
|
| 28 |
- An idle prompt keeps streaming past 5 s with a sink + rolling KV window (keys re-rotated so RoPE positions stay in
|
| 29 |
0..20) and gets a fresh scene every 20 s (adjustable) to stop long-rollout drift.
|
app.py
CHANGED
|
@@ -4,7 +4,7 @@ One held ZeroGPU session (like the Qwen-Image 2.1 Realtime Space) runs the offic
|
|
| 4 |
per-frame rollout and streams it as 1 s segments. UI edits reach the running GPU process through a
|
| 5 |
small per-session JSON file.
|
| 6 |
|
| 7 |
-
*
|
| 8 |
seed, so the shot stays comparable and the new prompt is followed exactly.
|
| 9 |
* While the prompt is idle the scene keeps going past the trained 5 s with a sink + rolling KV
|
| 10 |
window (slot 0 keeps the first frame; when the 21-slot cache is full the oldest window frames are
|
|
@@ -41,12 +41,12 @@ WIDTH, HEIGHT = 832, 480
|
|
| 41 |
MAX_SEED = 2**31 - 1
|
| 42 |
MAX_CHARS = 1500
|
| 43 |
|
| 44 |
-
CHUNK = int(os.environ.get("CHUNK",
|
| 45 |
ROLL_STRIDE = int(os.environ.get("ROLL_STRIDE", 4)) # window frames dropped each time the cache is full
|
| 46 |
SINK = os.environ.get("SINK", "1") == "1" # keep frame 0 as an attention sink
|
| 47 |
-
LEAD = float(os.environ.get("LEAD",
|
| 48 |
REGEN_SECONDS = int(os.environ.get("REGEN_SECONDS", 20)) # idle prompt: fresh scene after this much video
|
| 49 |
-
DEBOUNCE = float(os.environ.get("DEBOUNCE", 0.
|
| 50 |
GPU_SIZE = os.environ.get("GPU_SIZE", "xlarge")
|
| 51 |
SESSION_SECONDS = int(os.environ.get("SESSION_SECONDS", 90))
|
| 52 |
RT_WINDOW = SESSION_SECONDS - 6 # leave headroom under the @spaces.GPU duration
|
|
@@ -75,8 +75,10 @@ print(f"[load] FastFrames-S ready: tokens/frame={TOKENS} cap={CAP} temporal_pair
|
|
| 75 |
class LiveRollout:
|
| 76 |
"""FastFrames.rollout, one latent frame at a time, forever, with a sink + rolling KV window."""
|
| 77 |
|
| 78 |
-
def __init__(self, seed):
|
| 79 |
-
|
|
|
|
|
|
|
| 80 |
self.gen = torch.Generator(device=PIPE.device).manual_seed(int(seed))
|
| 81 |
self.used = 0 # occupied cache slots
|
| 82 |
self.total = 0 # latent frames generated
|
|
@@ -254,9 +256,11 @@ def live_session(sid):
|
|
| 254 |
scenes = 0
|
| 255 |
last_ver = -1
|
| 256 |
|
|
|
|
|
|
|
| 257 |
def new_scene(reason):
|
| 258 |
nonlocal rollout, scenes
|
| 259 |
-
rollout = LiveRollout(seed)
|
| 260 |
scenes += 1
|
| 261 |
print(f"[live] {sid[:6]} scene {scenes} ({reason}) seed={seed} {prompt[:60]!r}", flush=True)
|
| 262 |
|
|
|
|
| 4 |
per-frame rollout and streams it as 1 s segments. UI edits reach the running GPU process through a
|
| 5 |
small per-session JSON file.
|
| 6 |
|
| 7 |
+
* Every prompt edit regenerates the scene from frame 0 under the new prompt with the same
|
| 8 |
seed, so the shot stays comparable and the new prompt is followed exactly.
|
| 9 |
* While the prompt is idle the scene keeps going past the trained 5 s with a sink + rolling KV
|
| 10 |
window (slot 0 keeps the first frame; when the 21-slot cache is full the oldest window frames are
|
|
|
|
| 41 |
MAX_SEED = 2**31 - 1
|
| 42 |
MAX_CHARS = 1500
|
| 43 |
|
| 44 |
+
CHUNK = int(os.environ.get("CHUNK", 2)) # latent frames per streamed segment (2 = 0.5 s of video)
|
| 45 |
ROLL_STRIDE = int(os.environ.get("ROLL_STRIDE", 4)) # window frames dropped each time the cache is full
|
| 46 |
SINK = os.environ.get("SINK", "1") == "1" # keep frame 0 as an attention sink
|
| 47 |
+
LEAD = float(os.environ.get("LEAD", 0.6)) # max seconds of video generated ahead of playback
|
| 48 |
REGEN_SECONDS = int(os.environ.get("REGEN_SECONDS", 20)) # idle prompt: fresh scene after this much video
|
| 49 |
+
DEBOUNCE = float(os.environ.get("DEBOUNCE", 0.0)) # a prompt must stay unchanged this long before it applies
|
| 50 |
GPU_SIZE = os.environ.get("GPU_SIZE", "xlarge")
|
| 51 |
SESSION_SECONDS = int(os.environ.get("SESSION_SECONDS", 90))
|
| 52 |
RT_WINDOW = SESSION_SECONDS - 6 # leave headroom under the @spaces.GPU duration
|
|
|
|
| 75 |
class LiveRollout:
|
| 76 |
"""FastFrames.rollout, one latent frame at a time, forever, with a sink + rolling KV window."""
|
| 77 |
|
| 78 |
+
def __init__(self, seed, kv):
|
| 79 |
+
# one cache per session, reused across scenes: slots are overwritten in order and attention only
|
| 80 |
+
# reads filled slots, so no clearing is needed
|
| 81 |
+
self.kv = kv
|
| 82 |
self.gen = torch.Generator(device=PIPE.device).manual_seed(int(seed))
|
| 83 |
self.used = 0 # occupied cache slots
|
| 84 |
self.total = 0 # latent frames generated
|
|
|
|
| 256 |
scenes = 0
|
| 257 |
last_ver = -1
|
| 258 |
|
| 259 |
+
kv = allocate_kv_cache(PIPE.dit, CAP, TOKENS, device=PIPE.device)
|
| 260 |
+
|
| 261 |
def new_scene(reason):
|
| 262 |
nonlocal rollout, scenes
|
| 263 |
+
rollout = LiveRollout(seed, kv)
|
| 264 |
scenes += 1
|
| 265 |
print(f"[live] {sid[:6]} scene {scenes} ({reason}) seed={seed} {prompt[:60]!r}", flush=True)
|
| 266 |
|