Spaces:
Sleeping
Sleeping
Commit History
10 days ago
-
No frame limit: MUSt3R picks its memory stride automatically; memory stays flat on long videos
Jay +1: Claude Opus 5.5- Frames are decoded and written to disk a few at a time, and the SAM2 / MUSt3R inputs are built into memory-mapped arrays, with progress shown while loading. - MUSt3R keeps every frame in its memory up to 300 frames (the evaluation setting) and writes every k-th frame beyond that, so the memory stays at about 300 frames. - MUSt3R outputs go frame by frame to memory-mapped files (also the per-video cache), SAM2 features are computed as tracking reaches each frame (same aligned batches: bit-identical masks), and the masks are written to a memory-mapped file: GPU and host memory no longer grow with the length. - Local sessions are pruned by disk use; long videos get a clear message when the disk or the visitor's ZeroGPU quota is not enough. The frame strip shows at most 240 evenly spaced frames. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012ZguskrS95ci1qFaiZp8HP8ef5c98 -
Instant click feedback, no 3D scene view
Jay +1: Claude Opus 5.5- Prompt clicks are drawn in the browser at once and replaced by the server-rendered frame; frame and prompt events skip the queue (one HTTP round trip), and fast clicks are all kept (trigger_mode="multiple", handled one at a time per browser session). - The MUSt3R point-cloud view is removed (plotly is no longer a dependency); Results sit next to the viewer. - The light theme is enforced from <head>, where it runs reliably. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012ZguskrS95ci1qFaiZp8HP3733f66 -
Track-only UI: drop the SAM 2.1 comparison and the 3D memory settings; examples accept new prompts
Jay +1: Claude Opus 5.5- The viewer, results, and downloads show 3AM only; the SAM 2.1 comparison run, overlay toggle, and side-by-side video are removed. - MUSt3R always keeps every frame in its memory (the evaluation setting); no user options. - Examples open with an empty prompt: the first click starts a new prompt, the viewer shows the plain frame with the clicks instead of the cached mask, and Track is enabled once there is a new prompt to track. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012ZguskrS95ci1qFaiZp8HP401cd46 -
UI: drop the checkpoint and GPU details from the page
Jay +1: Claude Opus 5.5Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012ZguskrS95ci1qFaiZp8HPa7573c8 -
Redesign the UI and split the SAM 2.1 comparison into its own button
Jay +1: Claude Opus 5.5- Light, card-based layout: examples, video, and prompt cards on the left; viewer with frame slider, thumbnail strip, and overlay toggle; 3D scene; results with downloads. Black primary actions, segmented controls, plain "3AM" wordmark. - "Track with 3AM" runs only MUSt3R + 3AM (one ZeroGPU run). "Compare with SAM 2.1" runs the official SAM 2.1 predictor from the same prompt mask as a second run and adds the side-by-side video. - Comparison videos label each panel with a title band above the frame instead of in-frame boxes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012ZguskrS95ci1qFaiZp8HP6c13e2d -
Workspace UI: live tracking, 3AM vs SAM 2.1 comparison, 3D scene view, downloads
Jay +1: Claude Opus 5.5- One ZeroGPU run (a generator) reconstructs with MUSt3R, tracks with 3AM, and optionally tracks with the official SAM 2.1 video predictor; tracked frames stream to the viewer. - Viewer with frame slider and thumbnail strip, Include / Exclude / Box prompts with undo, overlay of 3AM, SAM 2.1, or both; side-by-side comparison video; MUSt3R point cloud (plotly) with the tracked object highlighted; MP4 and per-frame PNG mask downloads. - Examples open with cached results (no GPU); the first one loads with the page. - engine.py: generator forms of MUSt3R streaming and tracking, and sam2_track_iter for the comparison. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012ZguskrS95ci1qFaiZp8HPb73d282 -
Track straight from the clicks in one ZeroGPU run; Preview mask becomes optional
Jay +1: Claude Opus 5.5Anonymous visitors hit "You have exceeded your ZeroGPU runs limit" after a few GPU calls, and Preview + Track used two. Track now computes the prompt mask (SAM 2.1, as Preview does) inside the same GPU call unless a preview mask is already set; new clicks invalidate a previewed mask. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012ZguskrS95ci1qFaiZp8HP8d51561 -
Pin torch 2.8.0 (supported by ZeroGPU) and store downloaded weights in weights/
Jay +1: Claude Opus 5.5ZeroGPU rejects torch 2.7.0 (supported: 2.8.0 and newer). The app was re-verified end to end with torch 2.8.0 / torchvision 0.23.0 in a clean environment. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012ZguskrS95ci1qFaiZp8HP1b01552 -
Use the step-1486336 checkpoint, speed up the pipeline, add 3D memory options
Jay +1: Claude Opus 5.5- 3AM weights: sam2.1-3am-routing-fusion-v1-contiguous-step1486336.pt (best ScanNet++ validation pIoU). - Fix tracking on ZeroGPU: the duration function now takes the GPU function's arguments (it raised "TypeError: _dur_track() takes 1 positional argument but 3 were given"). - MUSt3R streams with an explicit KV memory: two anchor frames, a write stride, and an optional maximum memory size (no keyframe search, no KD-tree). Defaults keep every frame. - Decoding, resizing, and video encoding run outside the GPU calls; MUSt3R runs inside the tracking call and is cached per memory setting; features are captured with forward hooks instead of copying all 13 decoder layers to the host. MUSt3R features are bit-identical to the previous engine. - Vendored SAM 2 / MUSt3R / DUSt3R are unmodified upstream subsets; unused code and the January app files are removed. Requirements are pinned to the verified versions. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: h…95d0361
-
Update the Space to the 3AM routing-fusion model
Yang-Che Sun +1: Claude Fable 5.1- engine.py: SAM2.1 Hiera-L + RoutingMemoryAttention/RoutingFusion (routing token), MUSt3R-512 with memory_mode=kv keeping every frame (no keyframe selection, no KD-tree, no retrieval), letterboxed frames, memory policy keeping the previous frame plus recent positive masks. - app.py: models built once on CPU at import, per-video MUSt3R features cached on local disk, MUSt3R KV memory kept on the GPU, regenerated examples, private repo download limited to needed files. - requirements: drop faiss-cpu and xformers. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PDHB2Z73hzKGTdXax7jT7H7145eb7
-
description
3ab2b45
-
typo
7855b0c -
apps
6fd7427 -
examples
d16fb5c -
examples
fb1055c
-
ui improve
9d941d0 -
build
03d1a10 -
change asmk
9cbcd7c -
app building
e6f20b8 -
building
Jay54e04d2 -
building
Jayb5cdc06 -
building
Jay439421e -
building
Jay0d8008d -
bug fix
98d7801 -
yaml
399411e -
bug fix
fd8f5cb -
sam2
8404eb0 -
sam2 and must3r
19be62c -
folders
4e89a51
-
fix bug
897c690 -
app overall
0bb5fcf -
initial commit
Verified signature14aac1a