Spaces:
Running on Zero
Running on Zero
Commit History
30 days ago
-
Recover when the checkpoint volume returns zero-filled bytes (#2) Running
- Recover when the checkpoint volume returns zero-filled bytes (6d71687ed10fa4251fde921cd1ce05da2928ee81) - Catch corrupt pickle records up front; clear message on GPU OOM (1c9e2512bae201186a9f6c79137c7f4d08afbaab)69b9d6e
-
GPU: lower requested ZeroGPU duration 180 -> 120s (free-tier limit)
Umut Kocasari +1: Claude Opus 4.8Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>9d09f64
-
Face crop: square again (better for the model), but tight to face+hair
Umut Kocasari +1: Claude Opus 4.8The model performs better on square inputs, so square the crop again — but base it on the tight face+hair mask (not the expanded detection box), center horizontally, anchor at the chin (face always kept), and cap the side at 1.5x head width so a tall head doesn't blow up the side margins (only a little hair-top is dropped then; hair is mostly in). Minimizes the inherent left/right background of a square head crop. Verified on the example: 517x517 square, face+most hair in, much less side background than the old full-width square. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>6b32ef4 -
Face crop: tighter, non-square box around face+hair (less background)
Umut Kocasari +1: Claude Opus 4.8Squaring a tall head filled the sides/top with background. The model's input processor uses upper_bound_resize (longest side -> process_res, aspect preserved; no square center-crop), so a square crop isn't needed. Crop tightly to the face+hair segmentation bbox with small margins (top/side 4%, bottom 3%), non-square, and save without forcing a square (only downscale if larger than needed). Removes the large left/right/top white-background margins while still covering all the hair; bottom unchanged. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>da04fcf -
Face crop: make it hair-aware (don't clip top hair; cover long hair)
Umut Kocasari +1: Claude Opus 4.8The RetinaFace box starts ~at the hairline, so the pixel3dmm crop clipped the top of the head. Grow the crop to also enclose the face+hair segmentation bbox (reusing the facer parser already used for tracks), with extra headroom on top. Square the box anchored at the TOP so, when it can't fit everything in a narrow portrait, the hair is kept and the bottom rides up to the jaw (still excludes shoulders). Width now follows the hair extent, so long hair on the sides is covered too. Verified on the example: top raised 143->61 (full headroom), hair fully inside. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>fb2f19a -
Inputs: add default-on face-centric crop (pixel3dmm-style)
Umut Kocasari +1: Claude Opus 4.8New 'Face crop' checkbox (default ON). Before inference, each frame is cropped to a face-centred square, mirroring pixel3dmm's get_cstm_crop (square the face box, expand ~1.42x — or 1.1x of the clip's union box when the face moves a lot, clamp to the image, resize). A single static box is computed over the clip so the crop is temporally stable. Reuses facer's RetinaFace detector for the box (no extra weights). Cropping runs before background removal so masks stay aligned. Falls back to full frames if facer finds no face. Verified on the NeRSemble example: a 536x536 face-centred square covering hair..chin, excluding the shoulders. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>3d9392a -
UI: reorder 2D video tabs to Normals, Depth, Canonical, Tracks
Umut Kocasari +1: Claude Opus 4.8Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>62331c3 -
Viewer: start the camera closer to the point cloud (1.7x -> 1.15x fit)
Umut Kocasari +1: Claude Opus 4.8Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>b3ac791 -
Viewer: default the colorful-tracks toggle to OFF (show RGB by default)
Umut Kocasari +1: Claude Opus 4.8Per request: start on plain image-RGB points; toggling 'Colorful tracks' on reveals the face+hair tracks. Checkbox unchecked + viewer default mode = rgb. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>044d9f2 -
Viewer: tracks/RGB toggle + restrict colorful tracks to face+hair (facer)
Umut Kocasari +1: Claude Opus 4.8Two changes to the dynamic point-cloud viewer: 1) Toggle between two colorings, switched instantly client-side: - colorful point tracks (default), and - plain image-RGB points. run() now emits both a track-colored and an RGB .glb per frame (<=2*N, i.e. up to 80 files); the three.js player preloads both sets, keyed by frame index, and the 'Colorful tracks' checkbox flips the active set. 2) Restrict the colorful tracks to the facial area + hair (never neck / shoulders / clothing) using FacePerceiver/facer face parsing (CelebAMask-HQ classes; keep all except background/neck/cloth/hat/necklace). The per-frame face+hair mask scopes both the track seeds and the recoloring via a new optional 'regions' arg on compute_track_colors. Falls back to unrestricted tracks if facer is unavailable / finds no face. Adds pyfacer to requirements and libgl1/libglib2.0-0 to packages.txt (pyfacer pulls in the GUI opencv build, which needs libGL). Co-Authored-By: Claude Opus 4…e948811 -
Viewer: preload all frames in a client-side three.js player (smooth playback)
Umut Kocasari +1: Claude Opus 4.8Replace gradio Model3D + Timer with a custom three.js point-cloud player embedded via gr.HTML(head=..., js_on_load=...). All per-frame .glb point clouds are loaded once into the scene; playback animates by toggling frame visibility, so the points stay on screen the whole time (no per-frame network/parse → no white flashes) and the full, un-subsampled cloud is kept. Play/Pause, Speed (fps) and a scrub slider run entirely client-side; a hidden file list bridges the served .glb URLs to the viewer. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>1f2eb37 -
Viewer: animate the point cloud over time (play/pause + speed)
Umut Kocasari +1: Claude Opus 4.8Add a gr.Timer-driven playback to the 3D point-cloud viewer so the track point cloud plays through the sequence over time, with Play/Pause buttons and a Speed (fps) slider. Each tick advances to the next frame's .glb (looping) and moves the scrub slider in sync; pause to orbit freely. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>715dbba
-
Examples: show a NeRSemble thumbnail directly + bold dataset label
Umut Kocasari +1: Claude Opus 4.8Replace gr.Examples (text-only labels hid the image) with a small custom layout: each example shows the first image as a visible thumbnail (no click needed) with a '**NeRSemble** 40 images' / '**NeRSemble** 1 image' caption (dataset name bold, count normal), and a Load & run button that sets the inputs then runs the pipeline. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>ead828b -
Default to one-by-one, robust video decode, add 40-img/1-img examples
Umut Kocasari +1: Claude Opus 4.8- Default inference mode is now One-by-one (was Joint). - Decode videos with cv2.VideoCapture instead of imageio's extension-based plugin pick, which only yielded the first frame of webcam recordings; now the first MAX_FRAMES frames are used (imageio fallback kept). - Add two click-to-run gr.Examples (cache_examples=False so the build never runs on the GPU-less builder): a 40-image sequence and a single image, vendored under examples/seq40/ (first 40 frames of nersemble 109 EMO-1-shout+laugh), LFS-tracked. Verified: examples build on gradio 6.19; cv2 extracts 40 frames from a 50-frame clip. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>94c4dfa
-
Viewer: keep the full point cloud (no subsampling)
Umut Kocasari +1: Claude Opus 4.8Per feedback, don't shrink the viewer cloud — render all points so it looks dense and filled instead of sparse. Drop the per-frame downsample cap (now only a 1M safety ceiling, matching DA3) and the cube-splat experiment; keep the sRGB->linear color fix. Verified: full-frame 504 run yields a 176k-point .glb (2.8MB) that renders as points. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>dae4cfd -
Demo feedback: video upload, fix washed-out points, smoother scrub, rename folder
Umut Kocasari +1: Claude Opus 4.8- Accept a video upload (gr.Video); its first MAX_FRAMES frames are decoded and used (via io_utils.load_frame_paths). Images or video both work. - Fix washed-out viewer colors: glTF COLOR_0 is linear, so convert our sRGB point colors to linear before writing the .glb (the viewer re-applies display gamma). - Smoother frame scrubbing: bind the slider's .release (reload on let-go, not on every drag tick) and shrink the viewer point cloud to <=60k points per frame. - Rename the download subfolder colored_points/ -> points/. Verified on a real run incl. a video input: first-N sampling, linearized GLB colors, <=60k pts, zip has tracks/ + points/; UI builds on gradio 6.19. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>35598db -
Apply demo feedback: predicted poses, title, dual point clouds, working viewer
Umut Kocasari +1: Claude Opus 4.8- Always use predicted camera poses (remove the toggle); point clouds are built in the world frame from pred.extrinsics. - Title uses the paper name directly ('Face Anything: 4D Face Reconstruction from Any Image Sequence'), no em dash. - Download zip now contains BOTH colorings: tracks/ (colorful tracks) and colored_points/ (plain RGB) point clouds per frame. - Fix the 3D viewer: gradio Model3D renders a vertex-only .ply as an empty solid mesh (the blank/black viewer). Export the point cloud as a .glb via trimesh (glTF-aligned: flip Y/Z, center by median) and set a white background, matching the pattern DA3's own demo uses. Verified on a real run: 42k-pt colored PointCloud GLB centered at origin; both ply sets in the zip; UI builds on 6.19. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>434c817 -
Add torchvision (not in the ZeroGPU base image)
Umut Kocasari +1: Claude Opus 4.8depth_anything_3.utils.io.input_processor imports torchvision.transforms, but the ZeroGPU base image ships only torch (2.11.0, CUDA 13), not torchvision. Pin torchvision==0.26.0, which requires exactly torch==2.11.0 (verified on PyPI), so it installs onto the existing runtime torch without pulling a different CUDA build. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>138ce2c -
Fix missing model deps on the Space (lazy export + addict/evo)
Umut Kocasari +1: Claude Opus 4.8depth_anything_3.api imported the whole utils.export chain at module top, pulling moviepy (export-only, not installed) and crashing load_model. Make that import lazy (only when export_dir is set, which the demo never does). The remaining core imports need addict (config dicts in registry/model heads) and evo (pose_align PosePath3D), which were missing from requirements — add them (pinned to the known-good versions). Verified by tracing a real load_model+inference run against this exact code: full pipeline completes, all outputs produced. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>6d76d9e -
Fix 'No images found' on upload (robust input normalization)
Umut Kocasari +1: Claude Opus 4.8The old _prepare_inputs only handled str/.name file values and re-globbed by extension, so Gradio uploads that arrive as dict/FileData (or temp files without a usable extension) copied nothing -> empty dir -> FileNotFoundError. Normalize each upload to (temp_path, orig_name), natural-sort by orig_name for temporal order, cap, then copy as frame_XXXX.<ext> with a content-sniffed extension. Verified on gradio 6.19 for dict+orig_name, string paths, extension-less temps, cap, and empty. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>fe4c53e -
Fix RVM background removal crash on the Space (trust_repo + fallback)
Umut Kocasari +1: Claude Opus 4.8torch.hub.load('PeterL1n/RobustVideoMatting') prompts 'trust this repo?', which raises EOFError in the non-interactive Space. Pass trust_repo=True. Also wrap the background-removal call so any failure logs a warning and falls back to full-frame reconstruction instead of aborting the whole run. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>d35e4df -
Fix Slider min==max crash on gradio 6.19
Umut Kocasari +1: Claude Opus 4.8gr.Slider requires minimum < maximum (enforced in 6.19, not 6.0.1). The initial hidden frame slider used (0, 0) and the post-run update used maximum=max(0, N-1) =0 for a single frame. Use maximum=1 initially and max(1, N-1) after, and only show the slider when N>1. Verified build_demo() on gradio==6.19.0. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>e92e6bd -
Docs: mount model repo as a read-only volume for the checkpoint
Umut Kocasari +1: Claude Opus 4.8HF replaced the fixed persistent-storage disk with mounted volumes. Recommend mounting the model repo (read-only) so the checkpoint is a local file at /models/checkpoint.pt (FACEANYTHING_CHECKPOINT) — no download, nothing to re-fetch on cold start. Keep the hf_hub_download path as an alternative, with a Storage Bucket volume backing HF_HOME for cache persistence. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>bce9a48 -
Pull checkpoint from a HF model repo at startup
Umut Kocasari +1: Claude Opus 4.8Add FACEANYTHING_CHECKPOINT_REPO (+ _FILE/_REPO_TYPE/_REVISION, HF_TOKEN) so the ~15 GB checkpoint lives in a separate HF model repo and is fetched once via hf_hub_download on the CPU node at startup (never spending ZeroGPU seconds). Falls back to an explicit local path or checkpoints/checkpoint.pt. README documents the recommended storage + Persistent Storage / HF_HOME setup. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>10fa3d7 -
Add FaceAnything Gradio demo app
Umut Kocasari +1: Claude Opus 4.8Up to 40 input images; Joint (all-at-once) vs One-by-one inference modes. Outputs: canonical/depth/normals 2D videos, a colorful 3D point-track point cloud (.ply) in the 3D viewer with a per-frame slider + downloadable .zip, and a bonus 2D track overlay video. Exposes the repo's hyperparameters (process_res, background removal, predicted poses, conf cut, track count/k/ threshold, fps, frame cap). Vendors src/ (faceanything + depth_anything_3); checkpoint added separately via LFS. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>4db294e -
initial commit
Verified signature7dd9e15