Spaces:
Running on Zero
Running on Zero
Commit History
-
Work around missing /etc/host-cgroup breaking ZeroGPU scheduling Running
Weijie Lyu +1: Claude Opus 5 (1M context)Every GPU request has failed since the container restarted on 2026-09-07 (46/46), all with the same traceback: spaces/zero/client.py:131 in schedule cgroup_path=utils.self_cgroup_device_path(), spaces/zero/utils.py:86 in self_cgroup_device_path cgroup_content = Path(Config.zerogpu_proc_self_cgroup_path).read_text() FileNotFoundError: [Errno 2] No such file or directory: '/etc/host-cgroup' It fails inside spaces' GPU-scheduling handshake, before generate_3d_head runs, so it is unrelated to the xformers/SDPA/OOM fixes below it. Both halves of this are HF's: the /etc/host-cgroup path comes from the ZEROGPU_PROC_SELF_CGROUP_PATH env var the runtime injects (upstream's default in spaces/config.py is /proc/self/cgroup), and the runtime is the thing that should bind-mount the file. Restarting and a factory rebuild both changed nothing -- the rebuild hit CACHED on every layer, and HF's generated Dockerfile pins spaces==0.50.4 on the pip command line anyway. Upgrading is not a …b69bb2c
-
Force SDPA in GS-LRM attention (xformers unusable on Blackwell)
Weijie Lyu +1: Claude Opus 4.8 (1M context)Diffusion now completes (50/50), but GS-LRM's transformer used xformers memory_efficient_attention, which raises 'xFormers wasn't built with CUDA support' on this GPU. The class already has an equivalent PyTorch SDPA branch (use_flashatt_v2=False); force it on, since xformers can't run here. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>268d06a -
Fix real OOM: 4-D SDPA layout; cu130 is ZeroGPU-native, drop cu128 cruft
Weijie Lyu +1: Claude Opus 4.8 (1M context)GPU-DIAG showed torch 2.11.0+cu130 persists and diff-gaussian compiles against the system CUDA 13.0 toolkit -- i.e. ZeroGPU natively ships cu130 + a matching CUDA 13.0 toolkit. So cu130 was never the problem and the cu128/nvcc machinery was unnecessary: remove the pip-nvcc CUDA_HOME logic and the nvidia-cuda-nvcc/cccl requirements; compile against the matching system toolkit. The actual failure (42 GiB free yet OOM at first attention) was the SDPA call receiving 3-D tensors, which forces SDPA's math backend to materialize the full multi-view attention matrix (tens of GB). Reshape q/k/v to 4-D (batch, heads, seq, head_dim) so SDPA uses its flash / memory-efficient backend -- the efficient path xformers used to provide. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>d0d5411 -
Make pip nvcc detection robust (glob, not hardcoded path)
Weijie Lyu +1: Claude Opus 4.8 (1M context)Search site-packages for nvidia/cuda_nvcc*/bin/nvcc instead of assuming the exact path, and only add include/lib dirs that exist. Also note in the fallback warning that a stale build cache (lingering cu130 torch + missing nvcc package) needs a Factory Reboot. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>4ff86b7 -
Revert cu130 forcing; use ZeroGPU-native cu128 torch + pip cu128 nvcc
Weijie Lyu +1: Claude Opus 4.8 (1M context)Root cause found via [GPU-DIAG]: 42 GiB free yet allocation fails -- not a real OOM. The forced cu130 torch is incompatible with ZeroGPU's fork-based GPU memory management (ZeroGPU targets the cu128 builds of its supported torch versions). Stop forcing cu130: drop the startup torch reinstall and the allocator-env band-aids, and run the cu128 torch the base image provides. diff-gaussian then needs a CUDA 12.8 compiler (container only has 13.0), so install nvidia-cuda-nvcc-cu12 / cccl and point the build's CUDA_HOME at it (arch list incl. sm_120). Net: fewer hacks than before, aligned with ZeroGPU's supported stack. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>fdba6dc -
Install cu130 CUDA runtime libs (drop --no-deps) to fix spurious OOM
Weijie Lyu +1: Claude Opus 4.8 (1M context)The cu130 torch was force-installed with --no-deps, so its matching nvidia-*-cu13 runtime libraries were never installed -- the container kept the base image's nvidia-*-cu12 (12.8) libs. Running cu130 torch on cu12 CUDA libraries is a broken mix: convs limp along via cuDNN but allocations fail, surfacing as the cudaMalloc OOM that PyTorch's NVML-based reporter then asserts on. Reinstall the cu130 stack WITH deps (extra-index-url PyPI for the non-NVIDIA deps) so torch gets the CUDA 13 runtime it was built against. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>06957f5 -
Add GPU memory/build diagnostic before inference
Weijie Lyu +1: Claude Opus 4.8 (1M context)The NVML assert is PyTorch's OOM reporter failing in the ZeroGPU fork, so the underlying failure is a cudaMalloc OOM at the first attention -- implausible on a ~70GB GPU with SDPA. Log torch build and mem_get_info (free/total) inside the worker to tell a real memory cap apart from a spurious malloc failure (likely the forced cu130 torch fighting ZeroGPU's fork-based memory management). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>b330272 -
Use SDPA in MVAttnProcessor to fix OOM on multi-view attention
Weijie Lyu +1: Claude Opus 4.8 (1M context)The NVML assert was a red herring: it fires only in PyTorch's out-of-memory *reporter* (reportProcessMemoryInfo), whose NVML handle lookup fails in ZeroGPU's forked GPU process. The real failure is an OOM at the first attention op. Dropping enable_xformers left attn1 on MVAttnProcessor's full baddbmm+bmm path, which materializes the entire multi-view attention matrix (upcast fp32) and exhausts VRAM at 512-res. Replace it with F.scaled_dot_product_attention -- same math, no full matrix, and unlike xformers it runs on the Blackwell GPU. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>5efd06f -
Disable expandable_segments instead of the whole caching allocator
Weijie Lyu +1: Claude Opus 4.8 (1M context)PYTORCH_NO_CUDA_MEMORY_CACHING=1 dodged the NVML assert but OOM'd (no allocation pooling/reuse). The NVML query comes from the expandable_segments (CUDA VMM) path, which can't reinit NVML in ZeroGPU's forked GPU subprocess. Set PYTORCH_CUDA_ALLOC_CONF=expandable_segments:False: keeps the normal caching allocator (memory pooled, no OOM) while avoiding the NVML path (no assert). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>b9a9f95
-
Disable CUDA caching allocator to dodge ZeroGPU NVML assert
Weijie Lyu +1: Claude Opus 4.8 (1M context)The NVML_SUCCESS assert is the caching allocator's NVML memory query failing on ZeroGPU's sliced/virtualized GPU (same failure mode as MIG GPUs in upstream pytorch issues, which remain unfixed). Set PYTORCH_NO_CUDA_MEMORY_CACHING=1 in the main process (before torch imports) so allocations bypass the caching allocator and its NVML path. ssr_mode=False alone didn't resolve it. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>aee9f62 -
Disable Gradio SSR mode to fix ZeroGPU NVML allocator assert
Weijie Lyu +1: Claude Opus 4.8 (1M context)Inference failed with 'NVML_SUCCESS == r INTERNAL ASSERT FAILED' in the CUDA caching allocator. Gradio's SSR front-server forks/shares the CUDA context so NVML can't initialize in the ZeroGPU worker. Launch with ssr_mode=False; also drop share=True (unsupported on Spaces, was only emitting a warning). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>60ce5bb -
Drop enable_xformers call; use PyTorch SDPA on Blackwell GPU
Weijie Lyu +1: Claude Opus 4.8 (1M context)ZeroGPU's GPU is Blackwell (compute capability 12.0 / sm_120), too new for xformers' prebuilt memory-efficient-attention kernels (fa3F/cutlassF support <= 9.0), so enable_xformers_memory_efficient_ attention() raised NotImplementedError at inference. Remove the call and let diffusers fall back to PyTorch's built-in SDPA, which supports sm_120. xformers stays installed but unused. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>ca04977 -
Move torch stack to 2.11.0 to resolve xformers conflict
Weijie Lyu +1: Claude Opus 4.8 (1M context)xformers 0.0.33 hard-pins torch==2.9.0 (ZeroGPU-unsupported), which conflicted with torchvision 0.24.1 (torch==2.9.1) and broke pip resolution. xformers 0.0.35 relaxed to torch>=2.10, so move the whole trio to the agreeing, ZeroGPU-supported line: torch 2.11.0, torchvision 0.26.0, xformers 0.0.35 Update the startup cu130 reinstall to the same versions. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>4f828ee -
Force cu130 torch at startup before importing torch
Weijie Lyu +1: Claude Opus 4.8 (1M context)Build-time pip kept installing a cu128 torch (torch.version.cuda 12.8) no matter the index config, so diff-gaussian-rasterization failed to compile against the container's CUDA 13.0 toolkit. Enforce it in-process instead: before 'import torch', check the installed torch metadata and, if it isn't a +cu130 build, force-reinstall the cu130 torch/torchvision/xformers stack. Also log torch version/CUDA so the container log shows what's actually installed. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>d7cc8a7 -
Make cu130 the primary pip index, keep torch pin bare
Weijie Lyu +1: Claude Opus 4.8 (1M context)ZeroGPU's config validator rejects torch==2.9.1+cu130 (only bare 2.8.0/2.9.1/2.10.0/2.11.0 allowed), but PyPI's bare 2.9.1 is a cu128 build and --extra-index-url won't outrank PyPI for the same base version. Flip priority: cu130 index primary, PyPI extra. pip resolves the bare torch==2.9.1 to the cu130 wheel and the validator passes. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>a823266 -
Pin torch/torchvision to explicit +cu130 local version
Weijie Lyu +1: Claude Opus 4.8 (1M context)--extra-index-url alone didn't win: pip still pulled PyPI's cu128 torch 2.9.1 (version.cuda 12.8), so diff-gaussian-rasterization still saw a 13.0-vs-12.8 mismatch. Pin the exact +cu130 local version, which exists only on the cu130 index, forcing pip there. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>547b11d -
Move torch stack to cu130 to match ZeroGPU CUDA 13.0 toolkit
Weijie Lyu +1: Claude Opus 4.8 (1M context)diff-gaussian-rasterization failed to compile: torch 2.8.0 is built for CUDA 12.8 but the build container's toolkit is CUDA 13.0, and torch's cpp_extension hard-errors on a major-version mismatch. torch 2.8 has no cu130 wheel, so bump to the cu130 builds: torch 2.9.1+cu130, torchvision 0.24.1+cu130, xformers 0.0.33 pulled via --extra-index-url for the cu130 channel. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>36edc22 -
Use --no-build-isolation for diff-gaussian-rasterization
Weijie Lyu +1: Claude Opus 4.8 (1M context)The package's setup.py imports torch to detect CUDA arch, but pip's isolated build env lacks torch (ModuleNotFoundError: No module named 'torch'). Build against the main env where torch and ninja already live. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>a1e3266 -
Pin sdk_version to 5.49.1 to match gradio requirement
Weijie Lyu +1: Claude Opus 4.8 (1M context)Without sdk_version, HF injected gradio 6.15.2 which conflicted with the gradio==5.49.1 pin in requirements.txt. Pin the Space SDK version so HF installs the matching gradio build. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>41c6383 -
Bump torch to 2.8.0 for ZeroGPU compatibility
Weijie Lyu +1: Claude Opus 4.8 (1M context)ZeroGPU rejected the build with torch==2.4.0 (supported: 2.8.0, 2.9.1, 2.10.0, 2.11.0). Move torch/torchvision/xformers together to the matching torch 2.8.0 build set. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>4c43d53
-
Update README.md
f62503b -
change readme
802371b -
demo
cb4cbb0 -
demo
9922080 -
demo
2f82b20 -
demo
f13510a -
demo
5095eb7 -
demo
f81f220 -
demo
e799c2d -
demo
e8c4ba0 -
demo
e44e69a -
demo
32621a8 -
demo
f9b802b -
demo
6272050 -
demo
9b72c9a -
demo
343399b -
demo
6132ce7 -
demo
f423985 -
demo
918c308 -
demo
ac3e055 -
demo
e40964f -
demo
0704fd9 -
demo
bee7b3f -
demo
e5c64d1 -
demo
45fe0fa -
demo
465f735 -
Change thumnail
226b07f -
Update demo
25d0eb4 -
Update demo
3916713
-
Update demo
957b0df