Spaces:
Running on Zero
Running on Zero
Commit History
-
Drop torch.compile from the upscale model; run eager (#50) (#70)
Unverified signatureDev-Space A/B via UPSCALE_COMPILE (#69), 53 frames, back-to-back runs: - compiled (832x624 / 624x832): model_time 10.56-11.04s cold, 2.72-5.42s warm, almost entirely the per-request first-call (compile/guard) penalty - eager (544x832): model_time 0.96s cold and 0.96s warm, first call ~0.1s Eager's steady compute is ~25% slower per pixel (~+0.2s per request at 832x624), far below the 1.9-10s compile penalty it removes. Remove the compile path and the now-unneeded UPSCALE_COMPILE flag, and update the profiler docstring that assumed compile. Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>d7bfd39 -
Add UPSCALE_COMPILE env flag to run upscale eager for comparison (#50) (#69)
Unverified signatureAfter #67/#68, torch.compile's first-call-at-shape cost (9.8s cold, 1.9s warm) dwarfs the 0.79s of steady compiled compute per 53-frame request, so eager may be faster end to end. UPSCALE_COMPILE=0 skips torch.compile and logs that the eager model is in use; default behavior is unchanged. Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>0aab626 -
Upscale frames untiled and pad short batches to one compiled shape (#51) (#68)
Unverified signatureDev-Space profiling after #67 shows the remaining upscale cost is almost all torch.compile first-call-at-shape penalties: 8 distinct (tile-shape, batch-size) pairs cost 28.9s cold / 3.8s warm, vs. 0.73s for all 48 steady calls. The 8 come from a 2x2 grid of edge-clipped 512 tiles (4 shapes) times two batch sizes (4, plus 1 for the tail frame). - TILE_SIZE 512 -> 1024, covering the largest generated frame (MAX_DIM=832 plus TILE_PAD) in one tile. #51's numbers show ~13 GiB free vs. ~1 GiB used. - Pad a short tail batch up to FRAME_BATCH_SIZE by repeating its last frame, and drop the extra outputs, so it reuses the full batch's compiled shape. Expected: 1 first-at-shape call per resolution instead of 8. Output is no longer tile-stitched, so it can differ slightly (no seams) from before. Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>2ba97e4 -
Quantize upscaled frames to uint8 on the GPU before host transfer (#61) (#67)
Unverified signatureDev-Space profiling (see #61) showed pil_convert at ~9.1s of ~12s non-model time per 45-frame request: the per-frame numpy `(x * 255).round().astype(uint8)` on 3328x2496 float32 arrays, ~200ms/frame. host_transfer was another ~2.6s, copying float32 to the host when only uint8 is needed. Do the clamp/*255/round/uint8 on the device, then transfer a contiguous uint8 batch (1/4 the bytes) and hand it straight to Image.fromarray. Same arithmetic as before (fp32 multiply, round-half-to-even), so output is unchanged. Adds a `quantize` profiling span so the next dev-Space run shows the new split. Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>b730447
-
Break down upscale's non-model overhead into named profiling spans (#61)
UpscaleProfiler previously only measured model() calls, leaving the ~74% of upscale wall time spent outside them as one opaque "non_model" number. Adds a span() context manager and wraps job_grouping, tile_merge, frames_to_tensor, pre_pad, host_transfer, and pil_convert so the next UPSCALE_PROFILE=1 run on the dev Space localizes where that time actually goes. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>ea4ada4
-
Merge origin/develop, resolving upscale.py profiling conflict
Both branches touched upscale_frames(): develop added opt-in profiling (UpscaleProfiler) for #48, this branch added the mypy --strict type annotation on out_frames. Keep both. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>4bc4218 -
Add opt-in profiling to the upscale stage for #48 benchmarking
Gated by UPSCALE_PROFILE=1 (off by default, zero overhead when unset). Reports, per upscale_frames() call: wall time vs. model time vs. non-model overhead, split into first-call-at-a-(tile-shape,batch-size) vs. steady-state repeat calls (to expose torch.compile recompilation cost — the #50 question), plus peak CUDA memory alongside the current TILE_SIZE/FRAME_BATCH_SIZE/MAX_TILE_BATCH constants (the #51 headroom question). Profiling logic lives in its own UpscaleProfiler class (postprocess/upscale/profiling.py) so the tiling/inference hot path in upscale.py isn't cluttered with timing bookkeeping. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>678f3ac -
Merge remote-tracking branch 'origin/develop' into add-mypy-strict-typecheck
# Conflicts: # app.py # postprocess/interpolation.py # postprocess/upscale/upscale.pyc8ec0f7 -
Build a tensor-only RIFE -> upscale pipeline (#53)
Interpolation can now hand upscale (C,H,W) fp16 GPU tensors directly instead of round-tripping every frame through CPU numpy in between, when both stages run in the same @spaces.GPU allocation. The "no upscale" and "no interpolation" paths are unchanged. Adds a print_frames_info log line after interpolation so the handoff mode (tensor vs. numpy) is visible in production logs. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>fed6bc3 -
Remove redundant numpy->PIL->numpy round trip between interpolation and upscale
upscale_frames (and _frames_to_tensor) now accept the numpy float32 [0,1] (H,W,C) frames that _apply_interpolation already produces directly, instead of requiring a PIL.Image round trip solely to cross the function boundary. export_to_video accepts numpy or PIL either way, so the PIL detour bought nothing but two CPU-side per-pixel conversions and a PIL allocation per frame. Part of #47, closes #49. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>55d9178
-
Log when torch.compile falls back to the eager upscale model (#31)
Unverified signatureThe try/except guard added in #26 silently swallowed torch.compile initialization failures (e.g. missing/incompatible triton on the Space's GPU image), so a fallback to the eager model would be invisible in the logs — leaving no way to tell, from the deployed Space's console output, whether the compiled path is actually active. Print the exception instead of passing silently.1ceec92 -
torch.compile the upscale model on CUDA (#26)
Unverified signatureWraps the model with torch.compile(dynamic=True) on the CUDA path for kernel fusion. dynamic=True avoids a fresh recompile for every distinct tile shape (interior tiles vs. the clipped last row/column) — there's only a handful of those per video, but without it each new shape would trigger its own recompile. Wrapped in try/except: if torch.compile fails to initialize (e.g. missing/incompatible triton on the target GPU image), the eager model is used as-is. No GPU available in this environment to benchmark, and this is the riskiest of the four upscale perf changes (compile warmup cost vs. steady-state speedup, triton availability on the Space's GPU image) — needs careful validation on the dev Space before merge.a5e5374 -
Add mypy --strict type-check CI gate on every PR
Annotates the previously-untyped functions in app.py, model/pipeline.py, and postprocess/interpolation.py, fixes the couple of real bugs strict type-checking surfaced (deprecated Image.LANCZOS constant, an int-or-None mixing in preview_effect, a numpy float**float Any leak), and adds a pyproject.toml mypy config that exempts vendored third-party files instead of relaxing strictness project-wide. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CRfYRYPEtM8UA6mtHoKL422682589 -
Increase upscale tile size from 256 to 512 (#24)
Unverified signatureLarger tiles mean fewer, bigger model() calls per frame, which cuts pad/crop overhead and Python/kernel-launch count and improves GPU occupancy for this small SRVGGNetCompact network. 512 is still conservative for a 16GB+ Spaces GPU (T4/L4/A10G). No GPU available in this environment to benchmark memory headroom or speedup; needs validation on the dev Space before merge.e284828 -
Batch frames and same-shaped tiles through the upscale model (#23)
Unverified signatureupscale_frames processed one frame at a time, and within each frame _tile_process called the model once per tile (batch=1 every time). With SRVGGNetCompact this leaves most of the GPU idle per call, since a single ~256px tile is far under typical GPU parallel capacity. Group frames of equal size into batches (FRAME_BATCH_SIZE) before tiling, and within _tile_process group tile jobs by input-patch shape (interior tiles are all the same size; only the last row/column differs, clipped against the image edge) so each model() call is one batched forward pass over same-shaped patches, chunked to MAX_TILE_BATCH to bound memory. The per-tile crop/placement math is unchanged — verified index-for-index equivalent to the prior per-tile loop via a numpy simulation with a synthetic nonlinear "model" across several batch/ragged-size cases. No GPU available in this environment to benchmark; needs validation on the dev Space before merge.60ca9a8 -
Pre-download RIFE/upscale weights at startup instead of inside @spaces.GPU (#22)
Unverified signatureThe RIFE and 4xLSDIRCompact weight fetches are pure network/disk I/O with no CUDA dependency, but were only downloaded lazily on first use inside the metered @spaces.GPU allocation — costing a few seconds of ZeroGPU quota on every cold container start. Move both fetches to app startup (alongside load_pipeline()), mirroring how the main video model is already preloaded. Claude-Session: https://claude.ai/code/session_011keM1seSWsBZqPHrzKTeCG Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>7123409
-
Move 4x upscaling from CPU to GPU (fixes #11)
Runs upscale_frames() inside run_inference()'s @spaces.GPU call, right after RIFE interpolation, instead of outside it in generate_video(). postprocess/upscale/upscale.py now uses CUDA/fp16 when available (fp32 CPU fallback), matching the RIFE module's device pattern. _gpu_duration adds an upscaling-time estimate when the option is enabled. Avoids contending for the Space's single shared CPU container under concurrent load, at the cost of spending ZeroGPU quota on upscaling. SRS amendment tracked in issue #12.3546667 -
Add 4x super-resolution upscaling stage (FR-4, FR-5, ID-2, CR-2)
postprocess/upscale/: 4xLSDIRCompact (SRVGGNetCompact, num_conv=16) tiled CPU upscaling, vendored from the reference project (inference structure adapted from xinntao/Real-ESRGAN, BSD-3-Clause). Runs after interpolation and outside the @spaces.GPU call, so it never bids for ZeroGPU time. Wired into app.py's "Upscale 4x" checkbox. Also adds .gitignore for __pycache__/ and the RIFE weights downloaded at runtime. Closes #6 Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>819311d