someone-in-the-world Claude Sonnet 5 commited on
Commit
3d9fab2
·
unverified ·
1 Parent(s): 4578f11

Log the final served video, not the pre-upscale result (#46)

Browse files

* Sync develop into main (#43)

* Check Upscale 4x by default and drop model name from its label (#19)

Simplifies the option to a plain checkbox toggle without exposing the
underlying model name in the UI.


Claude-Session: https://claude.ai/code/session_01W9GLmk1R3mFfb7otDoGiWX

Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>

* Add detailed export/traceback logging to diagnose post-pipeline UI errors (#20)

The Space logs cut off right after the pipeline stages finish, with no
trace of what happens next in generate_video() (export_to_video and the
run_inference call boundary), even though the UI shows a generic Error.
Log frame metadata and full tracebacks at those points so the next
failure is diagnosable from the Space logs alone.

* Pre-download RIFE/upscale weights at startup instead of inside @spaces.GPU (#22)

The RIFE and 4xLSDIRCompact weight fetches are pure network/disk I/O with no
CUDA dependency, but were only downloaded lazily on first use inside the
metered @spaces.GPU allocation — costing a few seconds of ZeroGPU quota on
every cold container start. Move both fetches to app startup (alongside
load_pipeline()), mirroring how the main video model is already preloaded.


Claude-Session: https://claude.ai/code/session_011keM1seSWsBZqPHrzKTeCG

Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>

* Batch frames and same-shaped tiles through the upscale model (#23)

upscale_frames processed one frame at a time, and within each frame
_tile_process called the model once per tile (batch=1 every time). With
SRVGGNetCompact this leaves most of the GPU idle per call, since a single
~256px tile is far under typical GPU parallel capacity.

Group frames of equal size into batches (FRAME_BATCH_SIZE) before tiling,
and within _tile_process group tile jobs by input-patch shape (interior
tiles are all the same size; only the last row/column differs, clipped
against the image edge) so each model() call is one batched forward pass
over same-shaped patches, chunked to MAX_TILE_BATCH to bound memory. The
per-tile crop/placement math is unchanged — verified index-for-index
equivalent to the prior per-tile loop via a numpy simulation with a
synthetic nonlinear "model" across several batch/ragged-size cases.

No GPU available in this environment to benchmark; needs validation on
the dev Space before merge.

* Increase upscale tile size from 256 to 512 (#24)

Larger tiles mean fewer, bigger model() calls per frame, which cuts
pad/crop overhead and Python/kernel-launch count and improves GPU
occupancy for this small SRVGGNetCompact network. 512 is still
conservative for a 16GB+ Spaces GPU (T4/L4/A10G).

No GPU available in this environment to benchmark memory headroom or
speedup; needs validation on the dev Space before merge.

* torch.compile the upscale model on CUDA (#26)

Wraps the model with torch.compile(dynamic=True) on the CUDA path for
kernel fusion. dynamic=True avoids a fresh recompile for every distinct
tile shape (interior tiles vs. the clipped last row/column) — there's
only a handful of those per video, but without it each new shape would
trigger its own recompile. Wrapped in try/except: if torch.compile fails
to initialize (e.g. missing/incompatible triton on the target GPU image),
the eager model is used as-is.

No GPU available in this environment to benchmark, and this is the
riskiest of the four upscale perf changes (compile warmup cost vs.
steady-state speedup, triton availability on the Space's GPU image) —
needs careful validation on the dev Space before merge.

* Log when torch.compile falls back to the eager upscale model (#31)

The try/except guard added in #26 silently swallowed torch.compile
initialization failures (e.g. missing/incompatible triton on the Space's
GPU image), so a fallback to the eager model would be invisible in the
logs — leaving no way to tell, from the deployed Space's console output,
whether the compiled path is actually active. Print the exception instead
of passing silently.

* Add liability disclaimer to the Space UI (#30)

Notes that outputs are provided "as is" and that users are solely
responsible for generated content and how it's used/shared.


Claude-Session: https://claude.ai/code/session_015woCw9matvyXW3Xnx2JwXG

Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>

* Stop Inductor compilation progress from leaking into the Gradio UI (#33)

gr.Progress(track_tqdm=True) monkey-patches tqdm globally, so it was
picking up TorchInductor's own "Inductor Compilation" progress bars
(triggered by torch.compile on the upscale model, and by the quantized
transformer/text_encoder) regardless of their own disable flag. Drop
track_tqdm and report the generation stage's progress explicitly via
callback_on_step_end instead, matching the interpolation/upscale stages.

* Quantize transformer with weight-only fp8 instead of dynamic-activation fp8 (#37)

Float8DynamicActivationFloat8WeightConfig quantizes activations on every
F.linear call, which under eager execution (AOT deferred, #3) meant an
unplanned CUDA allocation per linear layer per step. That allocator churn
reliably crashed the prod ZeroGPU Space with an NVML assert in the caching
allocator (#35). Float8WeightOnlyConfig keeps the weight-memory savings
without any runtime activation-quantization allocations, trading away the
fp8x8 matmul speedup until AOT compilation (#36) makes the dynamic-activation
config safe again.


Claude-Session: https://claude.ai/code/session_01GRZRv1d2KinVjvMhJ4c7fF

Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>

* AOT-compile the transformer to re-enable dynamic-activation fp8 quant (#39)

Loads pipe.transformer/transformer_2 via spaces.aoti_load() against
thornmaze/WanTransformer3DModel-sm120-cu130-raa, confirmed reusable for
WAMU_v3 (#36's architecture/hardware investigation), and switches the
transformer quantization back from Float8WeightOnlyConfig to
Float8DynamicActivationFloat8WeightConfig now that AOT pre-plans memory
for the per-layer activation-quantize casts that config adds — the
allocator churn that caused #35's eager-mode NVML crash.

Needs real-GPU validation on the dev Space before rolling to prod: the
AOT package was compiled against torch==2.9.1+cu130 vs. this repo's
torch==2.11.0 pin, and no GPU is available in this environment to test
that ABI compatibility directly.


Claude-Session: https://claude.ai/code/session_01NbKYSwLurCaLK6SZ3b5EJQ

Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>

* Support specifying an end frame (last image) for I2V generation (#41)

Mirrors wan2-2-i2v-v3's Last Image input: adds an optional end-frame
upload in Advanced Settings, resized/cropped to match the input
image's dimensions, and passes it through to
WanImageToVideoPipeline(..., last_image=...). This repo pins the
same diffusers==0.38.0 as the reference project, so the pipeline
already supported last_image — it just wasn't wired up here.

Closes #40.


Claude-Session: https://claude.ai/code/session_01929npp9MCfTtGF4tyfEE5n

Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>

* Fix deploy crash: gr.Image doesn't accept an info kwarg in Gradio 6.0.1 (#42)

TypeError: Image.__init__() got an unexpected keyword argument 'info'
on Space startup. Unlike gr.Slider/gr.Dropdown, gr.Image has no info
param in this Gradio version — fold the hint into the label instead.


Claude-Session: https://claude.ai/code/session_01929npp9MCfTtGF4tyfEE5n

Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>

* Log the final served video, not the pre-upscale result

The debug logger recorded the pre-upscale/post-interpolation frames
even when upscaling succeeded, so the logged video never matched what
was actually served to the user. Log the same final_frames/final_fps
used for export_to_video instead, and drop the now-unused pre-upscale
return values from run_inference.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Y7x4tsmRCqNBzsi87N216U

---------

Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>

Files changed (2) hide show
  1. app.py +6 -10
  2. logging_utils.py +2 -2
app.py CHANGED
@@ -206,12 +206,10 @@ def run_inference(
206
 
207
  if upscaled_frames is None:
208
  print_infer_done(_time.perf_counter() - t_start, final_fps, len(final_frames))
209
- return final_frames, final_fps, None, final_fps
210
 
211
- # LOG-4: the debug log records the pre-upscale (post-interpolation) result, not the final
212
- # upscaled video and not the raw un-interpolated base output.
213
  print_infer_done(_time.perf_counter() - t_start, final_fps, len(upscaled_frames))
214
- return upscaled_frames, final_fps, final_frames, final_fps
215
 
216
 
217
  def _spawn_log(
@@ -275,7 +273,7 @@ def generate_video(
275
 
276
  t0 = _time.perf_counter()
277
  try:
278
- final_frames, final_fps, pre_upscale_frames, pre_upscale_fps = run_inference(
279
  resized_image,
280
  processed_last_image,
281
  prompt,
@@ -304,7 +302,6 @@ def generate_video(
304
  duration = _time.perf_counter() - t0
305
 
306
  print_frames_info("final_frames", final_frames, final_fps)
307
- print_frames_info("pre_upscale_frames", pre_upscale_frames, pre_upscale_fps)
308
 
309
  with tempfile.NamedTemporaryFile(suffix=".mp4", delete=False) as tmpfile:
310
  video_path = tmpfile.name
@@ -323,9 +320,8 @@ def generate_video(
323
  # run_inference — a video was still produced, so this still counts as success=True.
324
  # interpolation_enabled/upscale_output reflect what the user requested, not whether an
325
  # internal fallback fired.
326
- log_frames = pre_upscale_frames if pre_upscale_frames is not None else final_frames
327
  _spawn_log(
328
- resized_image, log_frames, pre_upscale_fps, prompt, negative_prompt, current_seed,
329
  steps, guidance_scale, interpolation_enabled, interpolation_multiplier, upscale_output,
330
  resized_image.width, resized_image.height, duration, True,
331
  )
@@ -378,11 +374,11 @@ with gr.Blocks() as demo:
378
  gr.Markdown(
379
  f"**Space:** High Quality Video Generation \n"
380
  f"**Operator:** {_OPERATOR_MD} \n"
381
- f"**Last updated:** 2026-08-30\n"
382
  "\n"
383
  "#### What data is collected\n"
384
  "When you submit a request, the following is logged: your uploaded input image, the generated "
385
- "output video (pre-upscale, post-interpolation), your prompt and negative prompt, seed, inference "
386
  "steps, guidance scale, frame-interpolation and upscaling settings, output resolution/fps/duration, "
387
  "generation duration, and success/error status. No user account, IP address, or session data is "
388
  "collected.\n"
 
206
 
207
  if upscaled_frames is None:
208
  print_infer_done(_time.perf_counter() - t_start, final_fps, len(final_frames))
209
+ return final_frames, final_fps
210
 
 
 
211
  print_infer_done(_time.perf_counter() - t_start, final_fps, len(upscaled_frames))
212
+ return upscaled_frames, final_fps
213
 
214
 
215
  def _spawn_log(
 
273
 
274
  t0 = _time.perf_counter()
275
  try:
276
+ final_frames, final_fps = run_inference(
277
  resized_image,
278
  processed_last_image,
279
  prompt,
 
302
  duration = _time.perf_counter() - t0
303
 
304
  print_frames_info("final_frames", final_frames, final_fps)
 
305
 
306
  with tempfile.NamedTemporaryFile(suffix=".mp4", delete=False) as tmpfile:
307
  video_path = tmpfile.name
 
320
  # run_inference — a video was still produced, so this still counts as success=True.
321
  # interpolation_enabled/upscale_output reflect what the user requested, not whether an
322
  # internal fallback fired.
 
323
  _spawn_log(
324
+ resized_image, final_frames, final_fps, prompt, negative_prompt, current_seed,
325
  steps, guidance_scale, interpolation_enabled, interpolation_multiplier, upscale_output,
326
  resized_image.width, resized_image.height, duration, True,
327
  )
 
374
  gr.Markdown(
375
  f"**Space:** High Quality Video Generation \n"
376
  f"**Operator:** {_OPERATOR_MD} \n"
377
+ f"**Last updated:** 2026-09-18\n"
378
  "\n"
379
  "#### What data is collected\n"
380
  "When you submit a request, the following is logged: your uploaded input image, the generated "
381
+ "output video, your prompt and negative prompt, seed, inference "
382
  "steps, guidance scale, frame-interpolation and upscaling settings, output resolution/fps/duration, "
383
  "generation duration, and success/error status. No user account, IP address, or session data is "
384
  "collected.\n"
logging_utils.py CHANGED
@@ -3,8 +3,8 @@ Face Hub dataset repo (LOG-2 through LOG-9).
3
 
4
  Modeled on the FireRed-Image-Edit-1.0-Fast reference project's logging_utils.py/inference.py
5
  pipeline, adapted for video: logged media is referenced by path rather than embedded as bytes
6
- (LOG-5), the logged output video is the pre-upscale result (LOG-4), and retention prunes on
7
- both a total-storage-size cap and a per-directory file-count cap (LOG-7) — the latter exists
8
  because the Hub commit endpoint rejects pushes once a directory (data/, images/, videos/)
9
  holds more than 10000 files, which a size-only cap doesn't bound since data/ and images/
10
  files are tiny compared to videos/.
 
3
 
4
  Modeled on the FireRed-Image-Edit-1.0-Fast reference project's logging_utils.py/inference.py
5
  pipeline, adapted for video: logged media is referenced by path rather than embedded as bytes
6
+ (LOG-5), the logged output video is the same final result served to the user, and retention
7
+ prunes on both a total-storage-size cap and a per-directory file-count cap (LOG-7) — the latter exists
8
  because the Hub commit endpoint rejects pushes once a directory (data/, images/, videos/)
9
  holds more than 10000 files, which a size-only cap doesn't bound since data/ and images/
10
  files are tiny compared to videos/.