Spaces:
Running on Zero
Running on Zero
Commit History
-
feat(mps): chunk the frame processor() call — bound the long/HD memory transient
Unverified signatureThe single processor(all-unique-frames) call was an O(#frames × source-res) resize transient the per-CLIP memguard couldn't see (it runs between clips): a 71s 720x1280 clip peaked ~96GB and only the external watchdog caught it. Process frames in resolution-adaptive chunks instead. tribe-brain verified BIT-IDENTICAL from transformers source: the VJEPA2 processor is per-frame spatial (resize+center_crop(256)+normalize, fixed params, no temporal op / no frame sampling), so cat([processor(chunk_i)]) == processor(all) exactly -> zero output change, Space parity preserved. Must-fixes applied: incremental free of the raw-frame list (so it + parts don't both peak), resolution-adaptive chunk size (~<=3GB/chunk: 256@720p, 32@4K; TRIBE_PROC_CHUNK override), and an IN-LOOP memguard so a regression aborts gracefully instead of needing kill -9. Expected peak ~96GB -> ~55-60GB. Pending live verification on full-res neeraj_reel_2.6a7f0c3 -
fix(dedup): correct the cache-key rationale comment (Theory A proven)
Unverified signatureReview (workflow wpdbh7b73) caught the old block comment still asserting the disproven Theory B ('per-window __call__ passes a WINDOWED event') and 'NO duration' — contradicting the hardened key 13 lines below and risking a future revert of the offset+duration key. Rewrite to the proven full-clip-event + safe-by-construction rationale. Comment-only; no behavior change.8565b75 -
feat(mps): dedup-cache machinery — hardened key, clear-per-Score, n_pass (gated)
Unverified signatureImplements the parity-gated dedup OUTPUT cache plan. TRIBE_DEDUP_CACHE stays OFF until the parity gate (max|Δ|==0) passes; this lands the machinery only. tribe-brain-reviewed. - fast_encode.py: harden the cache key with event.offset+duration (mirror exca's item_uid) so it is safe BY CONSTRUCTION — a windowed event would miss and re-extract, never serve a wrong slice. Add clear_dedup_cache(). - app.py: clear the cache at Score start (local) so a stale extraction (Gradio recycles temp upload paths across Scores) is never served; set n_pass=1 when the cache is on (one extraction -> clean 0->100% bar, retiring the double-extract saturation wart), window-count when off. Parity gate harness lives at ~/Projects/tests/parity_dedup.py (not in repo): cache-OFF vs ON, raw pre-stitch preds max|Δ|==0 + sweeps_on<sweeps_off, run locally before enabling.583bd05 -
feat(ui): timeline playhead follows video playback
Unverified signatureThe seek line was drawn once on click and stayed static. Now it tracks video.currentTime during playback: seek_js gains a bindPlayhead() that attaches a rAF loop to tm-video moving shapes[0] via partial relayout (~30Hz gated) with a single global rafId. Live-verified on a quality result: the playhead snapped to currentTime on play, tracked 3->20s, and moved to the seek position when scrubbed while paused. tribe-brain APPROVED (disproved the relayout feedback-storm from Gradio source). 6 must-fixes applied: create-or-move robust (Plotly.react clears shapes each Score), global rafId kill, both-bind retry (video may mount after the plot), shared gd resolver, xref:x (zoom-correct), 0.03s gate.efaaa37 -
feat(ui): in-card determinate progress bar via gr.Timer (Option T)
Unverified signatureThe native gr.Progress bar does not render in this loading-card layout (smoke-test: the sink fires every clip but Gradio's status-tracker stays empty), so drive the card's OWN HTML instead. The encode sink writes per-clip counters into a module-global state dict; a gr.Timer(0.3) reads it and re-renders the loading card with a determinate amber bar + live "clip k/N · window p/P" label. Live-verified: the Timer ticks in parallel with the concurrency_limit=1 Score (per-concurrency-id queue) — card showed "Encoding video · clip 4/30" at 13% advancing per clip; after the Score the card hides, no stuck bar, timeline + summary render cleanly (Timer -> gr.skip when idle). - progress.py: rewrite to a dict-based live state (begin/sink/snapshot/end + encode_frac); the 0.3s Timer is the throttle (no separate per-clip throttle). - ui.py: loading_card_html renderer + addressable loading_body gr.HTML. - theme.py: .co-prog-track/fill/pct amber bar styles (drop dead native CSS). - app.py: gr.Timer …03fb6f3 -
feat(ui): determinate per-clip encode progress bar (Option X)
Unverified signatureReplace the indeterminate "Scoring…" loader with a real determinate bar that advances one tick per V-JEPA2 clip across a monotonic 0.15→0.70 band, with a live "clip k/N · window p/P" label. Drives Gradio's native gr.Progress (CSS-themed amber); the custom card is kept as a themed header. - progress.py: pure encode_frac + make_clip_sink — P-window monotonic mapping (no reset between windows; degrades to a single sweep when P=1), ~0.4s throttle (final clip of each window always emits), injectable clock for tests. - fast_encode.py: module-global progress sink + per-clip _emit_progress (best-effort, never raises into the encode; no-op when unset). - app.py: register the sink LOCAL-only (video & not on_spaces), clear in a finally; P = len(plan_windows(dur)). The Space keeps the coarse stage progress — its forked GPU worker can't see a parent-set sink. - ui.py / theme.py: drop the fake indeterminate co-loading-bar; theme the native progress widget to the amber accent. - tests/te…ef94760
-
refine(mps): blocked attention BLOCK=1024 + stock-fallback for edge cases (mask/causal/dropout/training/small-N)
Unverified signature9a21d17 -
feat(mps): blocked/streaming V-JEPA2 attention (Approach C) — structural memory bound, replaces per-layer empty_cache hook
Unverified signature7859981 -
fix(windowing): global z-score post-stitch (cross-window scale consistency) to cure WIN-1 seam
Unverified signature801bb0e -
feat(quality): surface 'no speech track -> video-only' notice when no Word events
Unverified signaturef3abd43 -
feat(mps): wire vm_stat memguard into the dedup encode loop (start gate + periodic abort)
Unverified signaturea01623e -
perf(mps): HIGH watermark 0.6->0.45 (graceful OOM under the gate) + document B=1
Unverified signature5079aba -
feat(mps): per-layer empty_cache hook on V-JEPA2 encoder layers (TRIBE_MPS_EMPTY_EVERY)
Unverified signaturec2d76ae -
feat(memguard): vm_stat system-memory governor for local MPS runs
Unverified signaturea81c26f -
feat(text): preview synthesized speech + remove profiler harness
Unverified signature- Text mode now renders the gTTS speech the model actually scored as a seekable audio player (id=tm-video), like Audio mode — clicking a timeline spike seeks it. run_inference surfaces the synthesized Audio event's filepath via out_info; _gpu_infer threads it back out of the GPU fork to _score_impl. - Remove the temporary ZeroGPU profiler harness from the UI (import + accordion + /profile_* endpoints) to de-clutter before the Space goes public. - Document the PREWARM_QUALITY Space variable + HF_HOME writable-cache in README. Verified on HF (billed): text/quality returns a valid 4-trace timeline + the audio.mp3 player; scoring unchanged.0b87f18 -
feat: local MPS all-modes-working (audio + text); video deferred to HF
Unverified signatureAll 4 practical modes verified locally on M5 Max MPS via the Gradio UI (Playwright): audio fast + quality (whisperx ASR + Llama text features) and text (gTTS -> ASR -> Llama). Every change is MPS-gated / Space-safe (no-op on the ZeroGPU Space). - mps.py: whisperx ASR compute_type float16 -> int8 (CTranslate2 has no efficient fp16 on a CPU/MPS host; float16 is GPU-only). Subprocess-level patch, off-Space only, so it doesn't duplicate tribev2's transcript fn. - app.py: _ensure_writable_hub is now SPACE-ONLY -- off-Space it copied the entire ~197GB dev HF cache (all unrelated models) into ./cache and filled the disk. Plus PYTORCH_MPS_LOW/HIGH_WATERMARK_RATIO env to bound the Metal allocator. - fast_encode.py (MPS-gated): empty_cache per clip; eager attention for V-JEPA2 (sdpa falls back to CPU and OOMs at 8192 tokens; eager == sdpa numerically); bf16-on-MPS; Trap A (MPS OOM surfaces cleanly, not via the heavier fallback); dedup output cache behind TRIBE_DEDUP_CACHE (default…1005ddb
-
feat: local MPS port + ZeroGPU perf toolkit (dedup, backbone-reuse, pre-warm)
Unverified signatureSync GitHub with the deployed Space — work done since the last commit: - src/tribescore/mps.py: device coercion so the app runs locally on Apple-silicon MPS (config hardcodes cuda; bypass the pydantic Literal via object.__setattr__) + TRIBE_FORCE_CPU. - src/tribescore/inference.py: load_model() honours MPS + a loader-time config override (drops batch on MPS, keeps overlap/num_workers). - src/tribescore/fast_encode.py: numerically-exact frame-dedup V-JEPA2 encode (GPU-validated max|Δ|=0, keystone intact) + startup-built backbone reuse — un-bills the per-Score from_pretrained; .to(cuda) only inside the @spaces.GPU fork (CUDA can't be forked). Default-on (TRIBE_NO_DEDUP opts out). - src/tribescore/prewarm.py: cache everything OUTSIDE @spaces.GPU at startup so no Score triggers a download/build — atlas always; V-JEPA2 backbone; Llama/whisperx/spacy gated by PREWARM_QUALITY. - profiler.py: ZeroGPU billing + pipeline-breakdown harness (gradio_client). - app.py: wire pre-warm…11ab6a3
-
revert: disable torch.compile on V-JEPA2 (dynamo tracer bug broke video runs)
torch.compile hard-fails tracing V-JEPA2 on torch 2.8 + this transformers: AssertionError "Cannot construct ConstantVariable for value of type torch.device" at vjepa2 get_position_ids (device = x.device). fullgraph=False does NOT catch it (internal dynamo assertion, not a clean graph break) and the __init__ try/except guards only setup, not the forward-time trace -> every video run raised + flooded the logs. Restore last-known-good bf16+TF32 (~173s, clean) from 0c2af6a. AOT returns behind an opt-in flag + hard fallback so a compile failure can never reach the app. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CjPK7gQJfUBjPtiUmijstz47caa8e -
perf: torch.compile (static AOTInductor graph) on V-JEPA2 encode
Compile the compute-bound V-JEPA2 forward with torch.compile(dynamic=False, mode=max-autotune-no-cudagraphs). Every clip has an identical shape, so one static graph (fused attention + autotuned GEMMs) serves all ~104 clips — the right lever for a compute-bound ViT-g (clip-batching was slower: B=4 ~231s vs per-clip bf16 ~173s). Stacks on the existing bf16 autocast + TF32. vjepa2-only, fullgraph=False (self-degrades on graph breaks), disk-cached so only the first inference per container pays the compile. - GPU duration 480->900 to cover the once-per-container compile (bills actual use) - route TORCHINDUCTOR_CACHE_DIR into CACHE_DIR (cold-start insurance if /data) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CjPK7gQJfUBjPtiUmijstzb3147e1 -
fix(perf): batched encode B=4 + correct exca-descriptor fallback
B=8 exceeded 48GB (~11GB/clip) and threw a hard NVML INTERNAL ASSERT (not a catchable OutOfMemoryError), then the fallback failed because _orig=cls._get_data is the exca @infra.apply descriptor (not directly callable). Fixes: batch_size=4 (~32GB, safe margin) and fallback via _orig.__get__(self, type(self))(events). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CjPK7gQJfUBjPtiUmijstza368d74 -
perf+fix: batched V-JEPA2 encode (~3-8x) + writable hub cache for opt-in ASR/LLaMA
- patches.apply_batched_video_encode: replaces neuralset HuggingFaceVideo._get_data (vjepa2) hardcoded batch=1 loop with a B=8 batched bf16 forward; byte-identical per-clip post-processing (verified vs source) so stitch/metrics are unchanged; OOM auto-halves; falls back to the original on any error. - app._ensure_writable_hub: copies the preload-baked (build-owned) HF hub cache to a writable dir + redirects HF_HUB_CACHE, so the opt-in full-multimodal path's whisper + LLaMA downloads no longer hit "Permission denied (os error 13)". Invoked lazily only when full ASR is requested (fast default stays lean). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CjPK7gQJfUBjPtiUmijstzb05a413 -
perf: bf16 V-JEPA2 encode via scoped monkeypatch (~2x, fp32 output)
The "Encoding video" loop is hardcoded batch=1; the per-clip ViT-g forward is the bottleneck (~2s/it x ~104). New src/tribescore/patches.py monkeypatches _HFVideoModel.predict_hidden_states to run under bf16 autocast and return fp32 hidden states -> ~2x faster encode (measured 2.02s/it -> ~0.88s/it) with no numpy break (numpy has no bfloat16; a blanket autocast had broken .numpy()). V-JEPA2 only; audio/text/head stay fp32. Applied idempotently in the GPU worker. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CjPK7gQJfUBjPtiUmijstz49e3e84 -
perf: raise extractor batch sizes to exploit ZeroGPU VRAM (~48GB) + VRAM log
Bottleneck is per-modality backbone feature extraction at small batch sizes (V-JEPA2 8, LLaMA 4), underusing the GPU. config_update now also sets data.video_feature.image.batch_size=16, data.text_feature.batch_size=16, data.batch_size=16. Throughput-only (outputs/stitch unchanged). Added a PEAK_VRAM_GB log in _gpu_infer to guide ramping (16->24->32). Per brain verdict; ffmpeg-prechunk rejected (would regress via N x backbone reload). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CjPK7gQJfUBjPtiUmijstz8920ab3 -
fix(zerogpu): force data.num_workers=0 so predict() works in the GPU worker
The @spaces.GPU function runs in a daemonic subprocess; tribev2's shipped config uses num_workers=20, so predict()'s DataLoader tried to spawn worker children -> "AssertionError: daemonic processes are not allowed to have children". config_update now also sets data.num_workers=0. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CjPK7gQJfUBjPtiUmijstz6d3ed1d -
Integrate full app: 3-mode UI + windowed inference + ROI metrics + timeline
- app.py: Video/Audio/Text mode-switcher wired end-to-end (validate -> @spaces.GPU predict [approach B] -> windowing.stitch -> metrics.to_metrics -> plotting.timeline_figure), eager model load on Spaces, lazy ROI masks, audio_only debug path, UI<->PARCELS name mapping, summary peak_time->peak_t adapter, empty/loading/error/result states, click-to-seek JS, queue(1) + show_error + ssr_mode=False. - theme.py / ui.py: "Cortical Observatory" frontend-design (3 tabs + states). - src/tribescore/{inference,windowing,metrics,plotting}.py: engine, ROI, plot. - 36 synthetic tests pass (numpy/scipy); ast.parse clean across all files. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CjPK7gQJfUBjPtiUmijstz3202062 -
Scaffold TRIBE v2 Video Brain-Score Space
Clean open-source skeleton for a Gradio + HF ZeroGPU Space that wraps facebook/tribev2 to score a 4-5 min video and plot derived brain-metric curves (attention, virality, engagement) over the full timeline. - app.py: Gradio Blocks shell; model loaded lazily, Space-only (on_spaces guard); spaces.GPU shim so it imports without torch/tribev2/spaces. - src/tribescore: windowing (sliding-window + stitch, injectable infer_fn), metrics (ROI reduction to named curves), inference (import-safe tribev2 wrapper), plotting (timeline figure). Heavy imports deferred throughout. - tests: pure-numpy pytest for run_windowed (shape + monotonic time axis) and ROI reduction; 16 tests pass with just numpy + pytest. - requirements.txt pinned to the proven cbensimon/tribe-v2-demo reference (tribev2[plotting] @ git+..., torch==2.6.0, gradio==6.11.0, spaces, ...). - README with HF Space YAML front-matter, methodology, ZeroGPU note, attribution; Apache-2.0 LICENSE for our code; NOTICE for non-commer…7059c53