Spaces:
Runtime error
Runtime error
Commit History
-
Add custom user-LoRA (Quality mode) for the MPS route
Unverified signatureUsers can supply a LoRA by HF repo id or by uploading a .safetensors file, with a weight slider (0-1.5, default 0.9). Quality-only: in Fast the Lightning 4-step distillation fights a content LoRA, so a user LoRA is ignored there. - lora.py: security-critical resolver. .safetensors only (pickle refused), size-capped from repo metadata BEFORE download (anti-DoS), public-only repo access on Spaces (no owner token for user-supplied repos). Friendly LoraError on every failure. - modes.py: load adapter 'user' per-request (off the GPU clock), activate it in the Quality path, ALWAYS delete it in finally (mandatory MPS cleanup -- the long-lived process never re-forks). Cleanup keyed off a user_lora_loaded flag so an OOM degrade-to-Fast can't leak the adapter; lora_active recomputed after an in-loop degrade so calibration peaks key correctly. - Fix: call enable_lora() before every set_adapters(). disable_lora() leaves _disable_adapters=True on all layers and set_adapters() only …47baa19 -
feat(zerogpu): get all modes working on ZeroGPU
Unverified signatureRoot cause behind the gray output / ~31s-per-step slowness / OOM-refusals: the ~58GB model never landed on CUDA inside the @spaces.GPU fork. A module-level pipe.to("cuda") gets snapshotted and on per-call restore the params land back on CPU while a ~36GB orphaned cuda copy stays resident, so the model ran on CPU (slow) and the bf16 conv3d VAE decode on CPU returned all-zeros (the uniform-gray image). - Build the pipeline on CPU; move to cuda INSIDE the @spaces.GPU fork (one clean ~55GB resident copy, ~37GB free for activation). - Load the pipeline directly (drop the separately-constructed transformer). - CUDA activation preflight: budget on (total - allocated) VRAM, not driver-free (reads ~0 under the caching allocator); device-specific CUDA activation coefficients ~8x lower than MPS (measured peak ~4GB). - launch(ssr_mode=False, show_error=True): SSR was masking errors. - Open the Space: remove the active-development "don't run inference" banner. Verified on ZeroGPU xlarge: Ed…22b5c43 -
feat(mps): run locally on Apple Silicon MPS with a pre-inference OOM preflight
Unverified signatureMake the app functional end-to-end on MPS (bf16, no fp8), additive to the existing CUDA/ZeroGPU path which is left untouched. - backend: gate fp8/torchao quantization behind device=='cuda'; on MPS/CPU load the transformer + pipeline in bf16, pipe.to(device), enable VAE tiling/slicing, use a CPU generator. No cpu_offload on unified memory (it saves no physical RAM). Add a LOAD gate that refuses cleanly if the ~58 GB weights won't fit. - memory (new): pre-inference OOM preflight (the "calculate before inference so it never OOMs" requirement). Activation-centric dynamic budget — the weights are already resident, so only the activation must fit free RAM — recalibrated from MEASURED MPS peaks (~14 GB/input-image dominates; CFG double-pass negligible). Auto-degrades (drop reference -> drop CFG double-pass -> resolution) when memory is tight; an OOM-catch-retry plus an MPS high-watermark backstop guarantee it never OOMs even if the estimate is off. Calibration is an upward-…09ffd50