basyx commited on
Commit
166e621
·
verified ·
1 Parent(s): e0df3c7

Upload 6 files

Browse files
Files changed (6) hide show
  1. README.md +8 -7
  2. TRAIN_B200.md +41 -0
  3. aoti.py +35 -0
  4. app.py +675 -0
  5. packages.txt +1 -0
  6. requirements.txt +19 -0
README.md CHANGED
@@ -1,13 +1,14 @@
1
  ---
2
- title: Wan
3
- emoji: 🔥
4
- colorFrom: red
5
- colorTo: indigo
6
  sdk: gradio
7
- sdk_version: 6.27.0
8
- python_version: '3.12'
9
  app_file: app.py
10
  pinned: false
 
11
  ---
12
 
13
- Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
 
1
  ---
2
+ title: Wan2.2 14B Fast Preview
3
+ emoji: 🐌
4
+ colorFrom: yellow
5
+ colorTo: pink
6
  sdk: gradio
7
+ sdk_version: 5.44.1
8
+ python_version: "3.10"
9
  app_file: app.py
10
  pinned: false
11
+ short_description: generate a video from an image with a text prompt
12
  ---
13
 
14
+ Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
TRAIN_B200.md ADDED
@@ -0,0 +1,41 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: other
3
+ task_categories:
4
+ - text-to-video
5
+ tags:
6
+ - nsfw
7
+ - wan
8
+ - lora
9
+ pretty_name: letsnude LoRA train clips
10
+ ---
11
+
12
+ # Antrian train B200 (revisi catatan papan)
13
+
14
+ VPS tetap hidup. Setelah sewa B200, kirim SSH ke sesi ini — `scripts/b200/queue.sh` langsung jalan.
15
+
16
+ LoRA FN yang sudah jadi **relabel jadi cowgirl** (`cowgirv1`). File bobot tetap `fnelsonv2` / repo `kulkas2pintu/fnelsonv-wan22-i2v`, trigger generate: **`cowgirv1`**. Style full nelson **train ulang** (kunci lengan popliteal→leher, bukan cowgirl).
17
+
18
+ ## Target 1–6
19
+
20
+ | # | Scene | Acc train | Trigger | B200 high+low | Panduan dari catatan papan |
21
+ |---|---|---|---|---|---|
22
+ | 1 | Side fuck | 31 | `sidefuckv1` | ~5–7 jam | Miring, **satu kaki diangkat, satu kaki ke bawah**, pria di belakang, ventral penis, wajah+payudara, mid-thrust. Start generate: lepas baju → posisi → penis masuk. |
23
+ | 2 | Cumshot | 30 | `cumshotv1` | ~5–7 jam | Generate: **awal tanpa semen di wajah**, lalu semen banyak mendarat sampai akhir. Clip Acc banyak sudah ada semen di frame 1 — I2V lanjut dari frame itu. |
24
+ | 3 | DVP telentang | 30 | `dvpv1` | ~5–7 jam | Dua penis di SATU vagina, telentang, pria bawah + depan, keluar-masuk cepat, kasar. Bukan DP vagina+anal. |
25
+ | 4 | Deepthroat | 29 set / 30 Acc | `deepthroatv1` | ~5–7 jam | Dia berdiri, lutut, tangan di rambut supaya rambut tidak nutup wajah, kepala diam, shaft sampai pangkal, tangan dia lepas penis. 1 clip pendek Acc tidak masuk set. |
26
+ | 5 | Full nelson | 30 | `fnelsonv` | ~5–7 jam **fresh** | **Bukan cowgirl.** Pria di bawah, wanita di panggul, wajah kamera. Lengan lewat popliteal, kunci paha posterior naik ke sisi leher. Tangan wanita bebas. Jangan init dari LoRA lama. |
27
+ | 6 | Standing FN | 24 | `sfnelsonv` | ~5–7 jam | 24 clip OK (tidak tunggu 30). Berdiri, angkat, punggung ke dada, kunci sama, kaki di udara. 21 clip `sfn_su_*` wanita sama — tetap masuk karena kamu izinkan. Init dari FN baru jika sudah ada. |
28
+
29
+ Cowgirl: **tidak di-train**. LoRA ada, trigger `cowgirv1`. Clip cowgirl belum 30.
30
+
31
+ Total 6 pose ≈ **30–42 jam** B200.
32
+
33
+ ## Perintah
34
+
35
+ ```bash
36
+ cd /home/ubuntu/grok-work/workspace/fullnelson
37
+ python3 scripts/assemble_train_sets.py
38
+ QUEUE="side_fuck cumshot dvp_supine deepthroat fn sfn" bash scripts/b200/queue.sh
39
+ ```
40
+
41
+ B200: 192 GB VRAM, RAM host ≥ 64 GB, `blocks_to_swap 0`. Backup clip: `kulkas2pintu/letsnude-lora-backup`.
aoti.py ADDED
@@ -0,0 +1,35 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """
2
+ """
3
+
4
+ from typing import cast
5
+
6
+ import torch
7
+ from huggingface_hub import hf_hub_download
8
+ from spaces.zero.torch.aoti import ZeroGPUCompiledModel
9
+ from spaces.zero.torch.aoti import ZeroGPUWeights
10
+ from torch._functorch._aot_autograd.subclass_parametrization import unwrap_tensor_subclass_parameters
11
+
12
+
13
+ def _shallow_clone_module(module: torch.nn.Module) -> torch.nn.Module:
14
+ clone = object.__new__(module.__class__)
15
+ clone.__dict__ = module.__dict__.copy()
16
+ clone._parameters = module._parameters.copy()
17
+ clone._buffers = module._buffers.copy()
18
+ clone._modules = {k: _shallow_clone_module(v) for k, v in module._modules.items() if v is not None}
19
+ return clone
20
+
21
+
22
+ def aoti_blocks_load(module: torch.nn.Module, repo_id: str, variant: str | None = None):
23
+ repeated_blocks = cast(list[str], module._repeated_blocks)
24
+ aoti_files = {name: hf_hub_download(
25
+ repo_id=repo_id,
26
+ filename='package.pt2',
27
+ subfolder=name if variant is None else f'{name}.{variant}',
28
+ ) for name in repeated_blocks}
29
+ for block_name, aoti_file in aoti_files.items():
30
+ for block in module.modules():
31
+ if block.__class__.__name__ == block_name:
32
+ block_ = _shallow_clone_module(block)
33
+ unwrap_tensor_subclass_parameters(block_)
34
+ weights = ZeroGPUWeights(block_.state_dict())
35
+ block.forward = ZeroGPUCompiledModel(aoti_file, weights)
app.py ADDED
@@ -0,0 +1,675 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ import os; os.system('pip install --upgrade --no-deps spaces')
2
+ import spaces
3
+ import shutil
4
+ import subprocess
5
+ import sys
6
+ import copy
7
+ import random
8
+ import tempfile
9
+ import warnings
10
+ import time
11
+ import gc
12
+ import uuid
13
+ from tqdm import tqdm
14
+
15
+ import cv2
16
+ import numpy as np
17
+ import torch
18
+ from torch.nn import functional as F
19
+ from PIL import Image
20
+
21
+ import gradio as gr
22
+ from diffusers import (
23
+ FlowMatchEulerDiscreteScheduler,
24
+ SASolverScheduler,
25
+ DEISMultistepScheduler,
26
+ DPMSolverMultistepInverseScheduler,
27
+ UniPCMultistepScheduler,
28
+ DPMSolverMultistepScheduler,
29
+ DPMSolverSinglestepScheduler,
30
+ )
31
+ from diffusers.pipelines.wan.pipeline_wan_i2v import WanImageToVideoPipeline
32
+ from diffusers.utils.export_utils import export_to_video
33
+
34
+ from torchao.quantization import quantize_, Float8DynamicActivationFloat8WeightConfig, Int8WeightOnlyConfig
35
+ import aoti
36
+
37
+ os.environ["TOKENIZERS_PARALLELISM"] = "true"
38
+ warnings.filterwarnings("ignore")
39
+ IS_ZERO_GPU = bool(os.getenv("SPACES_ZERO_GPU"))
40
+
41
+ # if IS_ZERO_GPU:
42
+ # print("Loading...")
43
+ # subprocess.run("rm -rf /data-nvme/zerogpu-offload/*", env={}, shell=True)
44
+
45
+ # --- FRAME EXTRACTION JS & LOGIC ---
46
+
47
+ # JS to grab timestamp from the output video
48
+ get_timestamp_js = """
49
+ function() {
50
+ // Select the video element specifically inside the component with id 'generated-video'
51
+ const video = document.querySelector('#generated-video video');
52
+
53
+ if (video) {
54
+ console.log("Video found! Time: " + video.currentTime);
55
+ return video.currentTime;
56
+ } else {
57
+ console.log("No video element found.");
58
+ return 0;
59
+ }
60
+ }
61
+ """
62
+
63
+
64
+ def extract_frame(video_path, timestamp):
65
+ # Safety check: if no video is present
66
+ if not video_path:
67
+ return None
68
+
69
+ print(f"Extracting frame at timestamp: {timestamp}")
70
+
71
+ cap = cv2.VideoCapture(video_path)
72
+
73
+ if not cap.isOpened():
74
+ return None
75
+
76
+ # Calculate frame number
77
+ fps = cap.get(cv2.CAP_PROP_FPS)
78
+ target_frame_num = int(float(timestamp) * fps)
79
+
80
+ # Cap total frames to prevent errors at the very end of video
81
+ total_frames = int(cap.get(cv2.CAP_PROP_FRAME_COUNT))
82
+ if target_frame_num >= total_frames:
83
+ target_frame_num = total_frames - 1
84
+
85
+ # Set position
86
+ cap.set(cv2.CAP_PROP_POS_FRAMES, target_frame_num)
87
+ ret, frame = cap.read()
88
+ cap.release()
89
+
90
+ if ret:
91
+ # Convert from BGR (OpenCV) to RGB (Gradio)
92
+ # Gradio Image component handles Numpy array -> PIL conversion automatically
93
+ return cv2.cvtColor(frame, cv2.COLOR_BGR2RGB)
94
+
95
+ return None
96
+
97
+ # --- END FRAME EXTRACTION LOGIC ---
98
+
99
+
100
+ def clear_vram():
101
+ gc.collect()
102
+ torch.cuda.empty_cache()
103
+
104
+
105
+ # RIFE
106
+ if not os.path.exists("RIFEv4.26_0921.zip"):
107
+ print("Downloading RIFE Model...")
108
+ subprocess.run([
109
+ "wget", "-q",
110
+ "https://huggingface.co/r3gm/RIFE/resolve/main/RIFEv4.26_0921.zip",
111
+ "-O", "RIFEv4.26_0921.zip"
112
+ ], check=True)
113
+ subprocess.run(["unzip", "-o", "RIFEv4.26_0921.zip"], check=True)
114
+
115
+ # sys.path.append(os.getcwd())
116
+
117
+ from train_log.RIFE_HDv3 import Model
118
+ device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
119
+ rife_model = Model()
120
+ rife_model.load_model("train_log", -1)
121
+ rife_model.eval()
122
+
123
+
124
+ @torch.no_grad()
125
+ def interpolate_bits(frames_np, multiplier=2, scale=1.0):
126
+ """
127
+ Interpolation maintaining Numpy Float 0-1 format.
128
+ Args:
129
+ frames_np: Numpy Array (Time, Height, Width, Channels) - Float32[0.0, 1.0]
130
+ multiplier: int (2, 4, 8)
131
+ Returns:
132
+ List of Numpy Arrays (Height, Width, Channels) - Float32[0.0, 1.0]
133
+ """
134
+
135
+ # Handle input shape
136
+ if isinstance(frames_np, list):
137
+ # Convert list of arrays to one big array for easier shape handling if needed,
138
+ # but here we just grab dims from first frame
139
+ T = len(frames_np)
140
+ H, W, C = frames_np[0].shape
141
+ else:
142
+ T, H, W, C = frames_np.shape
143
+
144
+ # 1. No Interpolation Case
145
+ if multiplier < 2:
146
+ # Just convert 4D array to list of 3D arrays
147
+ if isinstance(frames_np, np.ndarray):
148
+ return list(frames_np)
149
+ return frames_np
150
+
151
+ n_interp = multiplier - 1
152
+
153
+ # Pre-calc padding for RIFE (requires dimensions divisible by 32/scale)
154
+ tmp = max(128, int(128 / scale))
155
+ ph = ((H - 1) // tmp + 1) * tmp
156
+ pw = ((W - 1) // tmp + 1) * tmp
157
+ padding = (0, pw - W, 0, ph - H)
158
+
159
+ # Helper: Numpy (H, W, C) Float -> Tensor (1, C, H, W) Half
160
+ def to_tensor(frame_np):
161
+ # frame_np is float32 0-1
162
+ t = torch.from_numpy(frame_np).to(device)
163
+ # HWC -> CHW
164
+ t = t.permute(2, 0, 1).unsqueeze(0)
165
+ return F.pad(t, padding).half()
166
+
167
+ # Helper: Tensor (1, C, H, W) Half -> Numpy (H, W, C) Float
168
+ def from_tensor(tensor):
169
+ # Crop padding
170
+ t = tensor[0, :, :H, :W]
171
+ # CHW -> HWC
172
+ t = t.permute(1, 2, 0)
173
+ # Keep as float32, range 0-1
174
+ return t.float().cpu().numpy()
175
+
176
+ def make_inference(I0, I1, n):
177
+ if rife_model.version >= 3.9:
178
+ res =[]
179
+ for i in range(n):
180
+ res.append(rife_model.inference(I0, I1, (i+1) * 1. / (n+1), scale))
181
+ return res
182
+ else:
183
+ middle = rife_model.inference(I0, I1, scale)
184
+ if n == 1:
185
+ return [middle]
186
+ first_half = make_inference(I0, middle, n=n//2)
187
+ second_half = make_inference(middle, I1, n=n//2)
188
+ if n % 2:
189
+ return[*first_half, middle, *second_half]
190
+ else:
191
+ return[*first_half, *second_half]
192
+
193
+ output_frames =[]
194
+
195
+ # Process Frames
196
+ # Load first frame into GPU
197
+ I1 = to_tensor(frames_np[0])
198
+
199
+ total_steps = T - 1
200
+
201
+ with tqdm(total=total_steps, desc="Interpolating", unit="frame") as pbar:
202
+
203
+ for i in range(total_steps):
204
+ I0 = I1
205
+ # Add original frame to output
206
+ output_frames.append(from_tensor(I0))
207
+
208
+ # Load next frame
209
+ I1 = to_tensor(frames_np[i+1])
210
+
211
+ # Generate intermediate frames
212
+ mid_tensors = make_inference(I0, I1, n_interp)
213
+
214
+ # Append intermediate frames
215
+ for mid in mid_tensors:
216
+ output_frames.append(from_tensor(mid))
217
+
218
+ if (i + 1) % 50 == 0:
219
+ pbar.update(50)
220
+ pbar.update(total_steps % 50)
221
+
222
+ # Add the very last frame
223
+ output_frames.append(from_tensor(I1))
224
+
225
+ # Cleanup
226
+ del I0, I1, mid_tensors
227
+ torch.cuda.empty_cache()
228
+
229
+ return output_frames
230
+
231
+
232
+ # WAN
233
+
234
+ MODEL_ID = "Wan-AI/Wan2.2-I2V-A14B-Diffusers"
235
+ CACHE_DIR = os.path.expanduser("~/.cache/huggingface/")
236
+
237
+ MAX_DIM = 832
238
+ MIN_DIM = 480
239
+ SQUARE_DIM = 640
240
+ MULTIPLE_OF = 16
241
+ MAX_SEED = np.iinfo(np.int32).max
242
+
243
+ FIXED_FPS = 16
244
+ MIN_FRAMES_MODEL = 8
245
+ MAX_FRAMES_MODEL = 160
246
+
247
+ MIN_DURATION = round(MIN_FRAMES_MODEL / FIXED_FPS, 1)
248
+ MAX_DURATION = round(MAX_FRAMES_MODEL / FIXED_FPS, 1)
249
+
250
+ SCHEDULER_MAP = {
251
+ "FlowMatchEulerDiscrete": FlowMatchEulerDiscreteScheduler,
252
+ "SASolver": SASolverScheduler,
253
+ "DEISMultistep": DEISMultistepScheduler,
254
+ "DPMSolverMultistepInverse": DPMSolverMultistepInverseScheduler,
255
+ "UniPCMultistep": UniPCMultistepScheduler,
256
+ "DPMSolverMultistep": DPMSolverMultistepScheduler,
257
+ "DPMSolverSinglestep": DPMSolverSinglestepScheduler,
258
+ }
259
+
260
+ pipe = WanImageToVideoPipeline.from_pretrained(
261
+ "TestOrganizationPleaseIgnore/WAMU_v3_WAN2.2_I2V_LIGHTNING",
262
+ torch_dtype=torch.bfloat16,
263
+ ).to('cuda')
264
+ original_scheduler = copy.deepcopy(pipe.scheduler)
265
+
266
+ # if os.path.exists(CACHE_DIR):
267
+ # shutil.rmtree(CACHE_DIR)
268
+ # print("Deleted Hugging Face cache.")
269
+ # else:
270
+ # print("No hub cache found.")
271
+
272
+ quantize_(pipe.text_encoder, Int8WeightOnlyConfig())
273
+ torch._dynamo.reset()
274
+ quantize_(pipe.transformer, Float8DynamicActivationFloat8WeightConfig())
275
+ torch._dynamo.reset()
276
+ quantize_(pipe.transformer_2, Float8DynamicActivationFloat8WeightConfig())
277
+ torch._dynamo.reset()
278
+
279
+ spaces.aoti_load(
280
+ module=pipe.transformer,
281
+ repo_id='cbensimon/WanTransformer3DModel-sm120-cu130-raa',
282
+ )
283
+ spaces.aoti_load(
284
+ module=pipe.transformer_2,
285
+ repo_id='cbensimon/WanTransformer3DModel-sm120-cu130-raa',
286
+ )
287
+
288
+ # pipe.vae.enable_slicing()
289
+ # pipe.vae.enable_tiling()
290
+
291
+ default_prompt_i2v = "make this image come alive, cinematic motion, smooth animation"
292
+ default_negative_prompt = "色调艳丽, 过曝, 静态, 细节模糊不清, 字幕, 风格, 作品, 画作, 画面, 静止, 整体发灰, 最差质量, 低质量, JPEG压缩残留, 丑陋的, 残缺的, 多余的手指, 画得不好的手部, 画得不好的脸部, 畸形的, 毁容的, 形态畸形的肢体, 手指融合, 静止不动的画面, 杂乱的背景, 三条腿, 背景人很多, 倒着走"
293
+
294
+
295
+ def resize_image(image: Image.Image) -> Image.Image:
296
+ """
297
+ Resizes an image to fit within the model's constraints, preserving aspect ratio as much as possible.
298
+ """
299
+ width, height = image.size
300
+ if width == height:
301
+ return image.resize((SQUARE_DIM, SQUARE_DIM), Image.LANCZOS)
302
+
303
+ aspect_ratio = width / height
304
+ MAX_ASPECT_RATIO = MAX_DIM / MIN_DIM
305
+ MIN_ASPECT_RATIO = MIN_DIM / MAX_DIM
306
+
307
+ image_to_resize = image
308
+ if aspect_ratio > MAX_ASPECT_RATIO:
309
+ target_w, target_h = MAX_DIM, MIN_DIM
310
+ crop_width = int(round(height * MAX_ASPECT_RATIO))
311
+ left = (width - crop_width) // 2
312
+ image_to_resize = image.crop((left, 0, left + crop_width, height))
313
+ elif aspect_ratio < MIN_ASPECT_RATIO:
314
+ target_w, target_h = MIN_DIM, MAX_DIM
315
+ crop_height = int(round(width / MIN_ASPECT_RATIO))
316
+ top = (height - crop_height) // 2
317
+ image_to_resize = image.crop((0, top, width, top + crop_height))
318
+ else:
319
+ if width > height:
320
+ target_w = MAX_DIM
321
+ target_h = int(round(target_w / aspect_ratio))
322
+ else:
323
+ target_h = MAX_DIM
324
+ target_w = int(round(target_h * aspect_ratio))
325
+
326
+ final_w = round(target_w / MULTIPLE_OF) * MULTIPLE_OF
327
+ final_h = round(target_h / MULTIPLE_OF) * MULTIPLE_OF
328
+ final_w = max(MIN_DIM, min(MAX_DIM, final_w))
329
+ final_h = max(MIN_DIM, min(MAX_DIM, final_h))
330
+ return image_to_resize.resize((final_w, final_h), Image.LANCZOS)
331
+
332
+
333
+ def resize_and_crop_to_match(target_image, reference_image):
334
+ """Resizes and center-crops the target image to match the reference image's dimensions."""
335
+ ref_width, ref_height = reference_image.size
336
+ target_width, target_height = target_image.size
337
+ scale = max(ref_width / target_width, ref_height / target_height)
338
+ new_width, new_height = int(target_width * scale), int(target_height * scale)
339
+ resized = target_image.resize((new_width, new_height), Image.Resampling.LANCZOS)
340
+ left, top = (new_width - ref_width) // 2, (new_height - ref_height) // 2
341
+ return resized.crop((left, top, left + ref_width, top + ref_height))
342
+
343
+
344
+ def get_num_frames(duration_seconds: float):
345
+ return 1 + int(np.clip(
346
+ int(round(duration_seconds * FIXED_FPS)),
347
+ MIN_FRAMES_MODEL,
348
+ MAX_FRAMES_MODEL,
349
+ ))
350
+
351
+
352
+ def get_inference_duration(
353
+ resized_image,
354
+ processed_last_image,
355
+ prompt,
356
+ steps,
357
+ negative_prompt,
358
+ num_frames,
359
+ guidance_scale,
360
+ guidance_scale_2,
361
+ current_seed,
362
+ scheduler_name,
363
+ flow_shift,
364
+ frame_multiplier,
365
+ quality,
366
+ duration_seconds,
367
+ safe_mode,
368
+ progress
369
+ ):
370
+ BASE_FRAMES_HEIGHT_WIDTH = 81 * 832 * 624
371
+ BASE_STEP_DURATION = 15
372
+ width, height = resized_image.size
373
+ factor = num_frames * width * height / BASE_FRAMES_HEIGHT_WIDTH
374
+ step_duration = BASE_STEP_DURATION * factor ** 1.5
375
+ gen_time = int(steps) * step_duration
376
+
377
+ if guidance_scale > 1:
378
+ gen_time = gen_time * 2.4
379
+
380
+ frame_factor = frame_multiplier // FIXED_FPS
381
+ if frame_factor > 1:
382
+ total_out_frames = (num_frames * frame_factor) - num_frames
383
+ inter_time = (total_out_frames * 0.02)
384
+ gen_time += inter_time
385
+
386
+ total_time = 15 + gen_time
387
+ if safe_mode:
388
+ total_time = total_time * 1.3
389
+
390
+ # Cap the reserved GPU duration so the reported reservation (×2 for xlarge)
391
+ # stays under free-tier ZeroGPU's ~300s per-call limit. The padded worst-case
392
+ # estimate ran 3-10x the real p90 (~107s) and tripped "The requested GPU
393
+ # duration (…) is larger than the maximum allowed". 145 → ~290s reported,
394
+ # still covers p90 and matches the caller's 150s hard timeout. Revert to 140
395
+ # if the cap turns out stricter than 300s.
396
+ total_time = min(total_time, 145)
397
+ return total_time
398
+
399
+
400
+
401
+ @spaces.GPU(duration=get_inference_duration, size='xlarge')
402
+ def run_inference(
403
+ resized_image,
404
+ processed_last_image,
405
+ prompt,
406
+ steps,
407
+ negative_prompt,
408
+ num_frames,
409
+ guidance_scale,
410
+ guidance_scale_2,
411
+ current_seed,
412
+ scheduler_name,
413
+ flow_shift,
414
+ frame_multiplier,
415
+ quality,
416
+ duration_seconds,
417
+ safe_mode,
418
+ progress=gr.Progress(track_tqdm=True),
419
+ ):
420
+ scheduler_class = SCHEDULER_MAP.get(scheduler_name)
421
+ if scheduler_class.__name__ != pipe.scheduler.config._class_name or flow_shift != pipe.scheduler.config.get("flow_shift", "shift"):
422
+ config = copy.deepcopy(original_scheduler.config)
423
+ if scheduler_class == FlowMatchEulerDiscreteScheduler:
424
+ config['shift'] = flow_shift
425
+ else:
426
+ config['flow_shift'] = flow_shift
427
+ pipe.scheduler = scheduler_class.from_config(config)
428
+
429
+ clear_vram()
430
+
431
+ task_name = str(uuid.uuid4())[:8]
432
+ print(f"Task: {task_name}, {duration_seconds}, {resized_image.size}, FM={frame_multiplier}")
433
+ start = time.time()
434
+ result = pipe(
435
+ image=resized_image,
436
+ last_image=processed_last_image,
437
+ prompt=prompt,
438
+ negative_prompt=negative_prompt,
439
+ height=resized_image.height,
440
+ width=resized_image.width,
441
+ num_frames=num_frames,
442
+ guidance_scale=float(guidance_scale),
443
+ guidance_scale_2=float(guidance_scale_2),
444
+ num_inference_steps=int(steps),
445
+ generator=torch.Generator(device="cuda").manual_seed(current_seed),
446
+ output_type="np"
447
+ )
448
+
449
+ raw_frames_np = result.frames[0] # Returns (T, H, W, C) float32
450
+ pipe.scheduler = original_scheduler
451
+
452
+ frame_factor = frame_multiplier // FIXED_FPS
453
+ if frame_factor > 1:
454
+ start = time.time()
455
+ rife_model.device()
456
+ rife_model.flownet = rife_model.flownet.half()
457
+ final_frames = interpolate_bits(raw_frames_np, multiplier=int(frame_factor))
458
+ else:
459
+ final_frames = list(raw_frames_np)
460
+
461
+ final_fps = FIXED_FPS * int(frame_factor)
462
+
463
+ with tempfile.NamedTemporaryFile(suffix=".mp4", delete=False) as tmpfile:
464
+ video_path = tmpfile.name
465
+
466
+ start = time.time()
467
+ with tqdm(total=3, desc="Rendering Media", unit="clip") as pbar:
468
+ pbar.update(2)
469
+ export_to_video(final_frames, video_path, fps=final_fps, quality=quality)
470
+ pbar.update(1)
471
+
472
+ return video_path, task_name
473
+
474
+
475
+ def generate_video(
476
+ input_image,
477
+ last_image,
478
+ prompt,
479
+ steps=4,
480
+ negative_prompt=default_negative_prompt,
481
+ duration_seconds=MAX_DURATION,
482
+ guidance_scale=1,
483
+ guidance_scale_2=1,
484
+ seed=42,
485
+ randomize_seed=False,
486
+ quality=5,
487
+ scheduler="UniPCMultistep",
488
+ flow_shift=6.0,
489
+ frame_multiplier=16,
490
+ video_component=True,
491
+ safe_mode=False,
492
+ progress=gr.Progress(track_tqdm=True),
493
+ ):
494
+ """
495
+ Generate a video from an input image using the Wan 2.2 14B I2V model with Lightning LoRA.
496
+ This function takes an input image and generates a video animation based on the provided
497
+ prompt and parameters. It uses an FP8 qunatized Wan 2.2 14B Image-to-Video model in with Lightning LoRA
498
+ for fast generation in 4-8 steps.
499
+ Args:
500
+ input_image (PIL.Image): The input image to animate. Will be resized to target dimensions.
501
+ last_image (PIL.Image, optional): The optional last image for the video.
502
+ prompt (str): Text prompt describing the desired animation or motion.
503
+ steps (int, optional): Number of inference steps. More steps = higher quality but slower.
504
+ Defaults to 4. Range: 1-30.
505
+ negative_prompt (str, optional): Negative prompt to avoid unwanted elements.
506
+ Defaults to default_negative_prompt (contains unwanted visual artifacts).
507
+ duration_seconds (float, optional): Duration of the generated video in seconds.
508
+ Defaults to 2. Clamped between MIN_FRAMES_MODEL/FIXED_FPS and MAX_FRAMES_MODEL/FIXED_FPS.
509
+ guidance_scale (float, optional): Controls adherence to the prompt. Higher values = more adherence.
510
+ Defaults to 1.0. Range: 0.0-20.0.
511
+ guidance_scale_2 (float, optional): Controls adherence to the prompt. Higher values = more adherence.
512
+ Defaults to 1.0. Range: 0.0-20.0.
513
+ seed (int, optional): Random seed for reproducible results. Defaults to 42.
514
+ Range: 0 to MAX_SEED (2147483647).
515
+ randomize_seed (bool, optional): Whether to use a random seed instead of the provided seed.
516
+ Defaults to False.
517
+ quality (float, optional): Video output quality. Default is 5. Uses variable bit rate.
518
+ Highest quality is 10, lowest is 1.
519
+ scheduler (str, optional): The name of the scheduler to use for inference. Defaults to "UniPCMultistep".
520
+ flow_shift (float, optional): The flow shift value for compatible schedulers. Defaults to 6.0.
521
+ frame_multiplier (int, optional): The int value for fps enhancer
522
+ video_component(bool, optional): Show video player in output.
523
+ Defaults to True.
524
+ progress (gr.Progress, optional): Gradio progress tracker. Defaults to gr.Progress(track_tqdm=True).
525
+ Returns:
526
+ tuple: A tuple containing:
527
+ - video_path (str): Path for the video component.
528
+ - video_path (str): Path for the file download component. Attempt to avoid reconversion in video component.
529
+ - current_seed (int): The seed used for generation.
530
+ Raises:
531
+ gr.Error: If input_image is None (no image uploaded).
532
+ Note:
533
+ - Frame count is calculated as duration_seconds * FIXED_FPS (24)
534
+ - Output dimensions are adjusted to be multiples of MOD_VALUE (32)
535
+ - The function uses GPU acceleration via the @spaces.GPU decorator
536
+ - Generation time varies based on steps and duration (see get_duration function)
537
+ """
538
+
539
+ if input_image is None:
540
+ raise gr.Error("Please upload an input image.")
541
+
542
+ num_frames = get_num_frames(duration_seconds)
543
+ current_seed = random.randint(0, MAX_SEED) if randomize_seed else int(seed)
544
+ resized_image = resize_image(input_image)
545
+
546
+ processed_last_image = None
547
+ if last_image:
548
+ processed_last_image = resize_and_crop_to_match(last_image, resized_image)
549
+
550
+ video_path, task_n = run_inference(
551
+ resized_image,
552
+ processed_last_image,
553
+ prompt,
554
+ steps,
555
+ negative_prompt,
556
+ num_frames,
557
+ guidance_scale,
558
+ guidance_scale_2,
559
+ current_seed,
560
+ scheduler,
561
+ flow_shift,
562
+ frame_multiplier,
563
+ quality,
564
+ duration_seconds,
565
+ safe_mode,
566
+ progress,
567
+ )
568
+
569
+ print(f"GPU complete: {task_n}")
570
+
571
+ return (video_path if video_component else None), video_path, current_seed
572
+
573
+
574
+ CSS = """
575
+ #hidden-timestamp {
576
+ opacity: 0;
577
+ height: 0px;
578
+ width: 0px;
579
+ margin: 0px;
580
+ padding: 0px;
581
+ overflow: hidden;
582
+ position: absolute;
583
+ pointer-events: none;
584
+ }
585
+ """
586
+
587
+
588
+ with gr.Blocks(theme=gr.themes.Soft(), css=CSS, delete_cache=(3600, 3700)) as demo:
589
+ gr.Markdown("## Wan 2.2 I2V (14B) — Fast ⚡")
590
+ gr.Markdown("Run Wan 2.2 in just 4-8 steps, fp8 quantization & AoT compilation - compatible with 🧨 diffusers and ZeroGPU.")
591
+
592
+ with gr.Row():
593
+ with gr.Column():
594
+ input_image_component = gr.Image(type="pil", label="Input Image", sources=["upload", "clipboard"])
595
+ prompt_input = gr.Textbox(label="Prompt", value=default_prompt_i2v)
596
+ duration_seconds_input = gr.Slider(minimum=MIN_DURATION, maximum=MAX_DURATION, step=0.1, value=3.5, label="Duration (seconds)", info=f"Clamped to model's {MIN_FRAMES_MODEL}-{MAX_FRAMES_MODEL} frames at {FIXED_FPS}fps.")
597
+ frame_multi = gr.Dropdown(
598
+ choices=[FIXED_FPS, FIXED_FPS*2, FIXED_FPS*4, FIXED_FPS*8],
599
+ value=FIXED_FPS,
600
+ label="Video Fluidity (Frames per Second)",
601
+ info="Extra frames will be generated using flow estimation, which estimates motion between frames to make the video smoother."
602
+ )
603
+ safe_mode_checkbox = gr.Checkbox(
604
+ label="🛠️ Safe Mode",
605
+ value=True,
606
+ info="Safe Mode: Requests 30% extra processing time to try to prevent unfinished tasks when the server is busy."
607
+ )
608
+ with gr.Accordion("Advanced Settings", open=False):
609
+ last_image_component = gr.Image(type="pil", label="Last Image (Optional)", sources=["upload", "clipboard"])
610
+ negative_prompt_input = gr.Textbox(label="Negative Prompt", value=default_negative_prompt, info="Used if any Guidance Scale > 1.", lines=3)
611
+ quality_slider = gr.Slider(minimum=1, maximum=10, step=1, value=6, label="Video Quality", info="If set to 10, the generated video may be too large and won't play in the Gradio preview.")
612
+ seed_input = gr.Slider(label="Seed", minimum=0, maximum=MAX_SEED, step=1, value=42, interactive=True)
613
+ randomize_seed_checkbox = gr.Checkbox(label="Randomize seed", value=True, interactive=True)
614
+ steps_slider = gr.Slider(minimum=1, maximum=30, step=1, value=4, label="Inference Steps", info="4 steps = fastest (Lightning default). Raise to 6-8 for slightly more detail.")
615
+ guidance_scale_input = gr.Slider(minimum=0.0, maximum=10.0, step=0.5, value=1, label="Guidance Scale - high noise stage", info="Values above 1 increase GPU usage and may take longer to process.")
616
+ guidance_scale_2_input = gr.Slider(minimum=0.0, maximum=10.0, step=0.5, value=1, label="Guidance Scale 2 - low noise stage")
617
+ scheduler_dropdown = gr.Dropdown(
618
+ label="Scheduler",
619
+ choices=list(SCHEDULER_MAP.keys()),
620
+ value="UniPCMultistep",
621
+ info="Select a custom scheduler."
622
+ )
623
+ flow_shift_slider = gr.Slider(minimum=0.5, maximum=15.0, step=0.1, value=3.0, label="Flow Shift")
624
+ play_result_video = gr.Checkbox(label="Display result", value=True, interactive=True)
625
+
626
+ generate_button = gr.Button("Generate Video", variant="primary")
627
+
628
+ with gr.Column():
629
+ # ASSIGNED elem_id="generated-video" so JS can find it
630
+ video_output = gr.Video(label="Generated Video", autoplay=True, sources=["upload"], show_download_button=True, show_share_button=True, interactive=False, elem_id="generated-video")
631
+
632
+ # --- Frame Grabbing UI ---
633
+ with gr.Row():
634
+ grab_frame_btn = gr.Button("📸 Use Current Frame as Input", variant="secondary")
635
+ timestamp_box = gr.Number(value=0, label="Timestamp", visible=True, elem_id="hidden-timestamp")
636
+ # -------------------------
637
+
638
+ file_output = gr.File(label="Download Video")
639
+
640
+ ui_inputs =[
641
+ input_image_component, last_image_component, prompt_input, steps_slider,
642
+ negative_prompt_input, duration_seconds_input,
643
+ guidance_scale_input, guidance_scale_2_input, seed_input, randomize_seed_checkbox,
644
+ quality_slider, scheduler_dropdown, flow_shift_slider, frame_multi,
645
+ play_result_video, safe_mode_checkbox
646
+ ]
647
+
648
+ generate_button.click(
649
+ fn=generate_video,
650
+ inputs=ui_inputs,
651
+ outputs=[video_output, file_output, seed_input]
652
+ )
653
+
654
+ # --- Frame Grabbing Events ---
655
+ # 1. Click button -> JS runs -> puts time in hidden number box
656
+ grab_frame_btn.click(
657
+ fn=None,
658
+ inputs=None,
659
+ outputs=[timestamp_box],
660
+ js=get_timestamp_js
661
+ )
662
+
663
+ # 2. Hidden number box changes -> Python runs -> puts frame in Input Image
664
+ timestamp_box.change(
665
+ fn=extract_frame,
666
+ inputs=[video_output, timestamp_box],
667
+ outputs=[input_image_component]
668
+ )
669
+
670
+ if __name__ == "__main__":
671
+ demo.queue().launch(
672
+ mcp_server=True,
673
+ ssr_mode=False,
674
+ show_error=True,
675
+ )
packages.txt ADDED
@@ -0,0 +1 @@
 
 
1
+ ffmpeg
requirements.txt ADDED
@@ -0,0 +1,19 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ diffusers==0.38.0
2
+ transformers==4.57.6
3
+ accelerate===1.13.0
4
+ safetensors
5
+ sentencepiece
6
+ peft==0.19.1
7
+ ftfy
8
+ imageio
9
+ imageio-ffmpeg
10
+ opencv-python
11
+ torchao==0.17.0
12
+
13
+ numpy>=1.16, <=1.23.5
14
+ # tqdm>=4.35.0
15
+ # sk-video>=1.1.10
16
+ # opencv-python>=4.1.2
17
+ # moviepy>=1.0.3
18
+ torch==2.11.0
19
+ torchvision==0.26.0