# Canonical V22 execution and recovered graph Authority: the attached canonical `HANDOFF.md`, `app.py`, backend and engine files. Experimental `workflow.json` was used only as a feature inventory. The canonical event bindings near the end of `app.py` determine the mapping below. `v22_core.py` preserves the relevant definitions verbatim. ## Coordinate vocabulary | Space | Meaning | |---|---| | `upload_original` | Uploaded source and full-frame Reference A geometry; timestamps are absolute seconds in the original source | | `normalized_source` | Selected clip, normalized V22 canvas, 24 fps; TAP points are **pixel coordinates**, not fractions in [0,1] | | `stabilized` | Virtual-camera input coordinates after the saved transforms; Reference B must match this frame geometry | | `stabilized_generated` | Viggle-generated Pass A in that same stabilized raster; retained for recovery/fusion | | `normalized_source` final | Generated/composited result on V22's normalized original-side canvas, with original clip audio muxed as in V22 | Frame index is clip-relative: `round((absolute_source_time - clip_start) * 24)`. Anchor timestamps and frame indices are canonical preparation outputs and cannot be edited independently in the picker. “Source-side” final does not mean the uploaded source's original pixel dimensions are restored; V22 uses its normalized canvas. ## Precise action mapping Inputs named below are canonical arguments, in canonical order. The Workflow controls may be reordered for readability; the adapters call the original functions in their original order. | Canonical UI action | V22 function | Inputs | Outputs | Downstream consumers | Coordinates | Allocation | |---|---|---|---|---|---|---| | Upload/browse source | Native `gr.Video` input | Uploaded media | Source path, visible source browser | Prepare, Standard, A/B source audio | `upload_original` | CPU/browser | | Prepare clip / anchors | `prepare` | video, duration, start, ref_time, auto_anchors, t1, t2, t3 | 3 display anchors; same 3 originals; 3 empty point JSON strings; normalized video; prep metadata; status | Point picker; TAP; SAM; camera; A/B metadata/audio | Original source → `normalized_source` | CPU/FFmpeg | | Select subject point | `add_click` in Blocks; equivalent standalone picker coordinate mapping in Workflow | Original anchor, current points, click position | Annotated display image, point list, count | TAP and optional SAM use original anchor geometry + point list | `normalized_source` pixels | Browser/CPU | | Clear anchor | `clear_clicks`; picker clear/undo controls | Original anchor | Original display, empty list, count | Next saved point revision | `normalized_source` | Browser/CPU | | Track subject | `run_tracking` → `tap_tracking.track_anchors` | normalized, meta_path, p1, p2, p3, progress | TAP preview, tracks NPZ, status | Camera solver; optional SAM wrapper reuses point groups, not TAP trajectories as SAM prompts | `normalized_source` | One `large` GPU job; CPU preview | | Optional camera guard | `run_sam_guard` → `sam_guard.track_guard` | normalized, meta_path, p1, p2, p3, progress | SAM preview, masks NPY, status | `solve_transforms` guard input only | `normalized_source` | One `large` GPU job; CPU preview; bypass is CPU only | | Build virtual camera | `build_zoom` | normalized, meta_path, tracks_path, optional masks_path, Reference A, ref_time, target_pct, smoothing, peripheral_blur, guard_margin, sam_center_assist, max_zoom_change, max_center_speed, progress | Stabilized video and Reference A (each duplicated for preview/state), transforms NPY, status | Pass A; Recovery browser; Pass B; final inverse transform | Source → `stabilized` | CPU/FFmpeg/NumPy | | Standard generation | `generate_standard` | video, Reference A, start, duration, segment_seconds, seed, audio_mode, progress | Standard final video, status | Final display/download only | Source → normalized generated output | CPU prep + one `xlarge` job for all windows + CPU assembly | | Generate Pass A | `generate_pass_a` | video, meta_path, stable_video, stable_ref, transforms_path, ref_time, pass_a_backward, segment_seconds, seed, feather, audio_mode, progress | Recomposed final; **generated stabilized video**; status | Final preview; optional Recovery uses generated stabilized output plus camera/preparation checkpoint | `stabilized` generation; `normalized_source` final | CPU prep + one `xlarge` job encompassing every internal window and A branch + CPU recomposition | | Extract Reference B candidate | `extract_b_candidate` | **stable input video**, meta_path, reacq_time | Exact stabilized frame, status | External repaint, then Reference B upload | `stabilized` | CPU/FFmpeg | | Reacquire identity | `reacquire_bidirectional` | source video, meta_path, **stable input**, transforms_path, **Pass A stabilized generated crop**, Reference B, reacq_time, backward_compute, transition_center, transition_duration, segment_seconds, seed, feather, audio_mode, progress | Recovered final, status | Final display/download | B generation and A/B fusion in stabilized space; then one inverse transform | CPU prep + a **separate** `xlarge` job encompassing B backward/forward windows + CPU fusion/inverse/audio | `add_click` is retained byte-for-byte in the extracted core but is not bound to a Workflow `.select()` event. The standalone browser performs the equivalent pixel scaling and saves the structured list. It never asks SAM to composite a final matte. ## Computation inside a pass `generate_pass_a` and `reacquire_bidirectional` each call `_render_branches_one_gpu`, which builds the complete branch/window lists and calls `_render_gpu_with_ui_progress` once. That invokes the original `backend.render_all_segments`, decorated `spaces.GPU(size="xlarge", duration=batch_gpu_budget)`. Technical windows default to five seconds; the ten-second V22 option remains. They are not graph nodes and are not separate allocations. Pass A forward-only is the default. Its optional backward and forward branches still share the single A allocation. Pass B's backward computation and short visible transition remain separate controls. B keeps V22's internal seed offset of +7001. A/B are merged using `dynamic_zoom.merge_reacquisition`; only the merged stabilized result is passed through `dynamic_zoom.recompose` once for B's final. A has its own independently useful recomposed final, which is never fed to B. Startup imports `spaces` before model-related imports. The canonical backend creates its CPU-side global pipeline before ZeroGPU fork, retains its fixed prompt/timestep/scheduler compatibility code, including the `pinned_loop(..., i, t)` signature, and moves/uses the pipeline inside its unchanged decorated rendering function. Generic documentation examples do not override this application-specific initialization strategy. ## Recovered graph The schema has 8 operators, 3 media references, 9 subjects and 20 edges. Every operator's immediate successor is a subject checkpoint or final, not another operator. This is purposeful: native single-node Run includes one-hop downstream nodes. | Path | Actual connected nodes | |---|---| | Source/spatial | Source → Prepare Source → Point Picker checkpoint → Track Subject → Tracking preview checkpoint → SAM/bypass → Guard checkpoint → Virtual Camera → Stabilized input checkpoint | | Generation | Reference A → Virtual Camera; stabilized checkpoint → Pass A → final A + saved A checkpoint | | Recovery | Saved A → Prepare Recovery → candidate/checkpoint → Pass B ← Reference B → recovered final | | Standard | Shared Source + Reference A → Standard → Standard final; no spatial/generation ancestor | The two browser interactions are explicit side transactions tied to immutable IDs: Point Picker writes points for a prepared ID; Recovery Browser writes a candidate selection for a completed A ID. Track/Prepare Recovery consume those selections on their next explicit Run. The browser pages neither synthesize a graph edge nor inject events into Gradio. ## Artifacts and ownership | Kind | Stored/reused contents | Contract / next consumer | |---|---|---| | prepared | Canonical metadata, normalized clip, anchors, absolute reference time, source digest | `normalized_source`; picker/TAP | | tracked | Preparation ID, point revision, anchor groups, tracks, preview | `normalized_source`; SAM/bypass | | guarded | Tracking/preparation IDs, optional masks, point revision | `normalized_source`; camera | | camera | Preparation/guard IDs, stabilized input, transformed Reference A, transforms, point revision | `stabilized`; Pass A | | pass_a | Camera/preparation IDs, final video and separate stabilized generated video, generation settings, take | `stabilized_generated` primary checkpoint; optional Recovery | | recovery | Saved A ID, camera/preparation IDs, candidate frame, exact absolute time and index | `stabilized`; Pass B | | pass_b / standard | Final path, settings/status provenance | `normalized_source` final | Checkpoint cards render native Markdown previews and an Inspect link. Only the opaque capability ID is consumed by downstream adapters. Server-owned JSON stores the typed manifest and lineage; browser-provided paths, metadata or coordinate labels are not accepted as authoritative checkpoints. Media inputs/outputs use native file descriptors, resolved only within upload/output directories. This avoids relying on Workflow preserving arbitrary fields in file dictionaries. User inputs are the three media references and saved point/frame selections. Computed artifacts are immutable checkpoints. Configurations are scalar native ports. Optional overrides include manual anchor times, SAM enable, camera tuning, A backward mode, and B settings. B inherits the completed A's window/seed/feather/audio values by default and exposes native overrides without editing A's saved inputs. A `-1` transition center selects a UI convenience default up to two seconds before B; an explicit center is forwarded unchanged. This is a documented UI default, not a new fusion algorithm. ## Why the experimental layout was not retained The prior 46-card graph exposed prepared data and generation intermediates as repeated standalone inputs, used raw JSON control bundles, and separated many technical outputs from their consumers. Edges could be schema-valid while Recovery still required users to manually provide its source/camera/A files. The replacement carries these as one typed lineage and reuses real edges. Native output subjects remain where needed for execution stops and useful previews. Reference A remains shared with Standard; this leaves a long fan-out connection. Rendered wire crossings, spacing and mobile navigation are still an explicit acceptance gate, not a claimed visual success.