Spaces:
Running on Zero
Download docs/ARCHITECTURE.md from fffiloni/viggle-mine-workflow: direct link, hf CLI and curl.
- Browser
- Download file 10.8 kB
-
https://huggingface.co/spaces/fffiloni/viggle-mine-workflow/resolve/main/docs/ARCHITECTURE.md
- Command line
-
hf download hf://spaces/fffiloni/viggle-mine-workflow/docs/ARCHITECTURE.md
-
curl -L -o ARCHITECTURE.md https://huggingface.co/spaces/fffiloni/viggle-mine-workflow/resolve/main/docs/ARCHITECTURE.md
A newer version of the Gradio SDK is available: 6.32.0
Canonical V22 execution and recovered graph
Authority: the attached canonical HANDOFF.md, app.py, backend and engine files. Experimental workflow.json was used only as a feature inventory. The canonical event bindings near the end of app.py determine the mapping below. v22_core.py preserves the relevant definitions verbatim.
Coordinate vocabulary
| Space | Meaning |
|---|---|
upload_original |
Uploaded source and full-frame Reference A geometry; timestamps are absolute seconds in the original source |
normalized_source |
Selected clip, normalized V22 canvas, 24 fps; TAP points are pixel coordinates, not fractions in [0,1] |
stabilized |
Virtual-camera input coordinates after the saved transforms; Reference B must match this frame geometry |
stabilized_generated |
Viggle-generated Pass A in that same stabilized raster; retained for recovery/fusion |
normalized_source final |
Generated/composited result on V22's normalized original-side canvas, with original clip audio muxed as in V22 |
Frame index is clip-relative: round((absolute_source_time - clip_start) * 24). Anchor timestamps and frame indices are canonical preparation outputs and cannot be edited independently in the picker. βSource-sideβ final does not mean the uploaded source's original pixel dimensions are restored; V22 uses its normalized canvas.
Precise action mapping
Inputs named below are canonical arguments, in canonical order. The Workflow controls may be reordered for readability; the adapters call the original functions in their original order.
| Canonical UI action | V22 function | Inputs | Outputs | Downstream consumers | Coordinates | Allocation |
|---|---|---|---|---|---|---|
| Upload/browse source | Native gr.Video input |
Uploaded media | Source path, visible source browser | Prepare, Standard, A/B source audio | upload_original |
CPU/browser |
| Prepare clip / anchors | prepare |
video, duration, start, ref_time, auto_anchors, t1, t2, t3 | 3 display anchors; same 3 originals; 3 empty point JSON strings; normalized video; prep metadata; status | Point picker; TAP; SAM; camera; A/B metadata/audio | Original source β normalized_source |
CPU/FFmpeg |
| Select subject point | add_click in Blocks; equivalent standalone picker coordinate mapping in Workflow |
Original anchor, current points, click position | Annotated display image, point list, count | TAP and optional SAM use original anchor geometry + point list | normalized_source pixels |
Browser/CPU |
| Clear anchor | clear_clicks; picker clear/undo controls |
Original anchor | Original display, empty list, count | Next saved point revision | normalized_source |
Browser/CPU |
| Track subject | run_tracking β tap_tracking.track_anchors |
normalized, meta_path, p1, p2, p3, progress | TAP preview, tracks NPZ, status | Camera solver; optional SAM wrapper reuses point groups, not TAP trajectories as SAM prompts | normalized_source |
One large GPU job; CPU preview |
| Optional camera guard | run_sam_guard β sam_guard.track_guard |
normalized, meta_path, p1, p2, p3, progress | SAM preview, masks NPY, status | solve_transforms guard input only |
normalized_source |
One large GPU job; CPU preview; bypass is CPU only |
| Build virtual camera | build_zoom |
normalized, meta_path, tracks_path, optional masks_path, Reference A, ref_time, target_pct, smoothing, peripheral_blur, guard_margin, sam_center_assist, max_zoom_change, max_center_speed, progress | Stabilized video and Reference A (each duplicated for preview/state), transforms NPY, status | Pass A; Recovery browser; Pass B; final inverse transform | Source β stabilized |
CPU/FFmpeg/NumPy |
| Standard generation | generate_standard |
video, Reference A, start, duration, segment_seconds, seed, audio_mode, progress | Standard final video, status | Final display/download only | Source β normalized generated output | CPU prep + one xlarge job for all windows + CPU assembly |
| Generate Pass A | generate_pass_a |
video, meta_path, stable_video, stable_ref, transforms_path, ref_time, pass_a_backward, segment_seconds, seed, feather, audio_mode, progress | Recomposed final; generated stabilized video; status | Final preview; optional Recovery uses generated stabilized output plus camera/preparation checkpoint | stabilized generation; normalized_source final |
CPU prep + one xlarge job encompassing every internal window and A branch + CPU recomposition |
| Extract Reference B candidate | extract_b_candidate |
stable input video, meta_path, reacq_time | Exact stabilized frame, status | External repaint, then Reference B upload | stabilized |
CPU/FFmpeg |
| Reacquire identity | reacquire_bidirectional |
source video, meta_path, stable input, transforms_path, Pass A stabilized generated crop, Reference B, reacq_time, backward_compute, transition_center, transition_duration, segment_seconds, seed, feather, audio_mode, progress | Recovered final, status | Final display/download | B generation and A/B fusion in stabilized space; then one inverse transform | CPU prep + a separate xlarge job encompassing B backward/forward windows + CPU fusion/inverse/audio |
add_click is retained byte-for-byte in the extracted core but is not bound to a Workflow .select() event. The standalone browser performs the equivalent pixel scaling and saves the structured list. It never asks SAM to composite a final matte.
Computation inside a pass
generate_pass_a and reacquire_bidirectional each call _render_branches_one_gpu, which builds the complete branch/window lists and calls _render_gpu_with_ui_progress once. That invokes the original backend.render_all_segments, decorated spaces.GPU(size="xlarge", duration=batch_gpu_budget). Technical windows default to five seconds; the ten-second V22 option remains. They are not graph nodes and are not separate allocations.
Pass A forward-only is the default. Its optional backward and forward branches still share the single A allocation. Pass B's backward computation and short visible transition remain separate controls. B keeps V22's internal seed offset of +7001. A/B are merged using dynamic_zoom.merge_reacquisition; only the merged stabilized result is passed through dynamic_zoom.recompose once for B's final. A has its own independently useful recomposed final, which is never fed to B.
Startup imports spaces before model-related imports. The canonical backend creates its CPU-side global pipeline before ZeroGPU fork, retains its fixed prompt/timestep/scheduler compatibility code, including the pinned_loop(..., i, t) signature, and moves/uses the pipeline inside its unchanged decorated rendering function. Generic documentation examples do not override this application-specific initialization strategy.
Recovered graph
The schema has 8 operators, 3 media references, 9 subjects and 20 edges. Every operator's immediate successor is a subject checkpoint or final, not another operator. This is purposeful: native single-node Run includes one-hop downstream nodes.
| Path | Actual connected nodes |
|---|---|
| Source/spatial | Source β Prepare Source β Point Picker checkpoint β Track Subject β Tracking preview checkpoint β SAM/bypass β Guard checkpoint β Virtual Camera β Stabilized input checkpoint |
| Generation | Reference A β Virtual Camera; stabilized checkpoint β Pass A β final A + saved A checkpoint |
| Recovery | Saved A β Prepare Recovery β candidate/checkpoint β Pass B β Reference B β recovered final |
| Standard | Shared Source + Reference A β Standard β Standard final; no spatial/generation ancestor |
The two browser interactions are explicit side transactions tied to immutable IDs: Point Picker writes points for a prepared ID; Recovery Browser writes a candidate selection for a completed A ID. Track/Prepare Recovery consume those selections on their next explicit Run. The browser pages neither synthesize a graph edge nor inject events into Gradio.
Artifacts and ownership
| Kind | Stored/reused contents | Contract / next consumer |
|---|---|---|
| prepared | Canonical metadata, normalized clip, anchors, absolute reference time, source digest | normalized_source; picker/TAP |
| tracked | Preparation ID, point revision, anchor groups, tracks, preview | normalized_source; SAM/bypass |
| guarded | Tracking/preparation IDs, optional masks, point revision | normalized_source; camera |
| camera | Preparation/guard IDs, stabilized input, transformed Reference A, transforms, point revision | stabilized; Pass A |
| pass_a | Camera/preparation IDs, final video and separate stabilized generated video, generation settings, take | stabilized_generated primary checkpoint; optional Recovery |
| recovery | Saved A ID, camera/preparation IDs, candidate frame, exact absolute time and index | stabilized; Pass B |
| pass_b / standard | Final path, settings/status provenance | normalized_source final |
Checkpoint cards render native Markdown previews and an Inspect link. Only the opaque capability ID is consumed by downstream adapters. Server-owned JSON stores the typed manifest and lineage; browser-provided paths, metadata or coordinate labels are not accepted as authoritative checkpoints. Media inputs/outputs use native file descriptors, resolved only within upload/output directories. This avoids relying on Workflow preserving arbitrary fields in file dictionaries.
User inputs are the three media references and saved point/frame selections. Computed artifacts are immutable checkpoints. Configurations are scalar native ports. Optional overrides include manual anchor times, SAM enable, camera tuning, A backward mode, and B settings. B inherits the completed A's window/seed/feather/audio values by default and exposes native overrides without editing A's saved inputs. A -1 transition center selects a UI convenience default up to two seconds before B; an explicit center is forwarded unchanged. This is a documented UI default, not a new fusion algorithm.
Why the experimental layout was not retained
The prior 46-card graph exposed prepared data and generation intermediates as repeated standalone inputs, used raw JSON control bundles, and separated many technical outputs from their consumers. Edges could be schema-valid while Recovery still required users to manually provide its source/camera/A files. The replacement carries these as one typed lineage and reuses real edges. Native output subjects remain where needed for execution stops and useful previews. Reference A remains shared with Standard; this leaves a long fan-out connection. Rendered wire crossings, spacing and mobile navigation are still an explicit acceptance gate, not a claimed visual success.