Spaces:
Runtime error
Runtime error
Commit History
-
fix: simplify round 2 — deduplicate update_masks, reduce VRAM, clear cache
Zhen Ye +1: Claude Opus 4.6 (1M context)- update_masks: now a thin wrapper over update_masks_with_remapping (was a full copy-paste of the same logic) - _mask_nms: use .half() instead of .float() to halve GPU memory spike (~40MB vs ~80MB for 10 masks at 1080p) - Export clearExplainCache() from explain.js, call it in resetToReady() so stale explain data doesn't persist across jobs Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>75516e9 -
fix: simplify review — fix mask NMS bug, remove dead code, hoist imports
Zhen Ye +1: Claude Opus 4.6 (1M context)- _mask_nms: fix greedy suppression bug (j<=i compared raw indices, not sort positions — masks escaped suppression). Use order[pos+1:] - _mask_nms: batch GPU computation (single matmul + single CPU sync instead of O(n^2) CUDA syncs per mask pair) - _apply_nms: hoist torch import (already module-level) and batched_nms import to module scope - real-backend.js: remove dead isNotRelevant (always false after mission_relevant filter) - explain.js: clear _explainCache on job change to prevent stale data across different videos Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>d6c4e26 -
fix: add mask-level NMS in GSAM2/YSAM2 to deduplicate overlapping masks
Zhen Ye +1: Claude Opus 4.6 (1M context)Within a single keyframe, YOLO can detect the same object with slightly different bounding boxes (e.g., cab vs full truck body) that survive box-level NMS but produce overlapping SAM2 masks. These all get unique IDs and render as stacked labels. Added _mask_nms() to MaskDictionary.add_new_frame_annotation(): - Computes pairwise mask IoU for same-label detections - Suppresses smaller masks when IoU > 0.5 with a larger one - Runs before masks enter the SAM2 video predictor pipeline Fixes duplicate "truck truck truck" labels on single objects. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>8d09cca
-
Eliminate redundant JPEG frame loading via shared frame store
Zhen Yef013501
-
feat(ttfs): instrument segmentation time-to-first-stream
Zhen Yecffad78 -
refactor: rename hf_yolov8 → yolo11 across codebase
Zhen Ye +1: Claude Opus 4.6Rename file yolov8.py → yolov11.py, class HuggingFaceYoloV8Detector → Yolo11Detector, and registry key "hf_yolov8" → "yolo11" in all 11 files. Loads YOLO11m COCO-pretrained via hf://Ultralytics/YOLO11. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>02d40a4 -
feat(segmentation): add YSAM2 variants and mission-aware detector mapping
Zhen Ye7d48e41 -
chore: remove dead code and unused frontend modules
Zhen Ye5a72443 -
refactor(gsam2): make SAM2 detector-agnostic
Zhen Yed23c4ea
-
fix: SAM2 _prep_prompts() signature
Zhen Ye339d4ac -
perf: defer GPU->CPU mask transfer to render thread via LazyFrameObjects
Zhen Ye +1: Claude Opus 4.6Replace bulk segment_output.to_object_dicts() CPU transfer with lazy materialization — only the last frame per segment is materialized for IoU tracking continuity, while render frames use LazyFrameObjects that defer GPU->CPU copy until the render thread needs them. Reduces CUDA syncs from 100+ per segment to ~3. Also removes deployment_logs.txt. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>97d32b8 -
fix: correct benchmark metrics — remove double-counting and add missing timers
Zhen Ye +1: Claude Opus 4.6- Fix sam_video_total_ms double-counting in single-GPU process_video() path (propagate_segment already accumulates, outer timer was adding it twice) - Fix id_reconciliation_ms in multi-GPU path to measure actual IoU work instead of wall-clock queue waits - Add model_load_ms and init_state_ms metrics for both single/multi-GPU paths Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>3e8ca01
-
perf: pipeline GSAM2 tracking + rendering with startup buffer
Zhen Ye +1: Claude Opus 4.6Pipeline tracking and rendering so segments stream as they're tracked instead of waiting for all tracking to complete before rendering begins. - Add on_segment callback to process_video() for incremental feeding - Hoist render/writer infrastructure above tracking branch - Single-GPU: callback-based incremental feeding via on_segment - Multi-GPU: streaming reconciliation with segment_buffer pattern - Writer startup buffer (60 frames) before streaming begins - 3x frame duplication for 18 FPS stream (6 FPS processing throughput) - Safety threshold: pause streaming when buffer < 20 frames - try/finally sentinel safety for render worker shutdown - Increase render_out queue from 64 to 128 for buffer headroom Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>108b51b -
feat: add num_maskmem parameter to /benchmark endpoint
Zhen Ye +1: Claude Opus 4.6Thread num_maskmem override through the full pipeline so benchmark runs can test reduced memory bank sizes (e.g. 3 instead of default 7). Patches the SAM2 video predictor's maskmem_tpos_enc at runtime. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>a47c9fa -
revert: remove torch.compile — runtime failures on HF Space
Zhen Ye +1: Claude Opus 4.6Reverts all torch.compile changes (b17bd6d..10eb3c6). The HF Space container lacks Triton/inductor support, causing 500 errors. The GPU-resident tensor pipeline from 5aec47c is retained. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>81548c3 -
fix: drop fullgraph=True from torch.compile for broader compatibility
Zhen Ye +1: Claude Opus 4.6fullgraph=True requires no graph breaks, which fails without SAM2VideoPredictorVOS's .clone() patches. Removing it allows torch.compile to handle graph breaks gracefully via partial compilation. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>40d31e0 -
fix: add runtime fallback for torch.compile inductor/triton failures
Zhen Ye +1: Claude Opus 4.6torch.compile wrapping succeeds immediately but actual compilation happens lazily on first forward pass. Now catches runtime errors on the first propagate_in_video yield, reverts all compiled forwards to eager originals, and retries the propagation transparently. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>534727c -
fix: manual torch.compile with graceful fallback for SAM2 components
Zhen Ye +1: Claude Opus 4.6Replace vos_optimized=True (caused 500 error) with manual per-component torch.compile matching Facebook's official recipe. Compiles image_encoder, memory_encoder, memory_attention (dynamic=True), sam_prompt_encoder, and sam_mask_decoder with max-autotune. Falls back to eager mode silently if PyTorch version or CUDA compiler is unavailable. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>43ebce8 -
perf: enable torch.compile for SAM2 via vos_optimized flag
Zhen Ye +1: Claude Opus 4.6Uses Facebook's official SAM2VideoPredictorVOS which compiles all five model components (image_encoder, memory_encoder, memory_attention, sam_prompt_encoder, sam_mask_decoder) with torch.compile max-autotune. First inference has ~30s warmup cost; subsequent frames benefit from fused kernels and reduced memory round-trips. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>703e1a0 -
perf: GPU-resident tensor pipeline for SAM2 video propagation
Zhen Ye +1: Claude Opus 4.6Eliminate all CUDA synchronization from propagate_segment() by keeping masks, bboxes, and validity flags on GPU in pre-allocated buffers. CPU materialization is deferred to a single bulk transfer via SegmentOutput.to_object_dicts() at the consumer. - Add _bbox_gpu() for zero-sync bounding box computation on GPU - Add SegmentOutput dataclass for GPU-resident segment results - Rewrite propagate_segment() with inline bbox + pre-allocated tensors - Refactor process_video() to reuse propagate_segment() - Update Phase 4 reconciliation: 3 CUDA syncs per segment vs 100+ Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>abf2d0f
-
perf: eliminate CUDA sync points in SAM2 video propagation hot-path
Zhen Ye +1: Claude Opus 4.6Replace per-object-per-frame torch.nonzero + .item() calls (7 CUDA syncs each) with batched GPU-native argmax reductions (2 syncs per segment). Move deepcopy from per-frame to once per segment on last frame only. Reduces total CUDA sync points from ~28,700 to ~82 per pipeline run. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>012d815 -
feat: add detailed timing metrics to GroundedSAM2 segmenter
Zhen Yed9cce19
-
Fix segmentation mission routing and first-frame GPT alignment
Zhen Ye2fa425d
-
Fix cross-GPU mask device mismatch in GSAM2 reconciliation
Zhen Ye55a2cb0 -
Harden GSAM2 parallel pipeline and tracking reconciliation
Zhen Ye824c842 -
Refine Grounded-SAM2 tracking behavior and video frame handling
Zhen Ye89e1672 -
Remove SAM3 and standardize segmentation on Grounded-SAM2
Zhen Yea637bd3
-
perf: Tune batch sizes and queues for A10 GPUs
Zhen Ye- Increased YOLO (HF/Drone) batch size to 32 - Increased Depth Anything batch size to 16 - Increased SAM3 batch size to 8 - Increased DETR batch size to 8 - Expanded inference.py queue_out to handle 32-frame batch bursts5ed99ac
-
Fix SAM3 batch prediction shape mismatch and add InternVL2 to frontend
Zhen Ye59562bd -
feat: Integrate InternVL2 and fix SAM3 segmentation batch size issue
Zhen Yed53b346
-
Implement Batch Inference & Queue Backpressure Fixes
Zhen Ye62cb625
-
feat(inference): enable full multi-GPU support for all models
Zhen Ye- Update inference.py to parallelize detection, segmentation, and depth estimation across all available GPUs - Update detectors (YOLOv8, DETR, GroundingDINO, DroneYolo) to accept device argument - Update SAM3 and DepthAnythingV2 to accept device argument - Add device-specific model loading to all model loaders - Remove OwlV2 support99b9e51
-
inital commit
Zhen Yef297b77