| # Version 0.5 workflow architecture |
|
|
| This document freezes the contracts and migration boundaries for the 0.5 |
| workflow-UX release. Implementations may evolve behind these contracts, but a |
| later step must not silently change their meaning. |
|
|
| ## Release invariant |
|
|
| A normal workflow selects source media once, chooses its audio intent once, |
| chooses an incoming transition preset per scene, and can validate or resume |
| without wiring the same full media value to several nodes. |
|
|
| Version 0.5 remains compatible with saved 0.4 workflow JSON and checkpoint |
| metadata. Existing node class ids and positional output slots are not removed |
| or reordered during this release. |
|
|
| ## Source Timeline contract |
|
|
| `H3_SOURCE_TIMELINE` is the single source-media contract. Its serialized |
| version is `h3_source_timeline_v1`. |
|
|
| It records: |
|
|
| - a file-backed video descriptor when picture is present; |
| - embedded, external-path, deferred-tensor, or absent audio; |
| - native FPS, stream PTS origin, duration, and the derived 24-fps extent; |
| - a native-frame skip offset and its exact time origin; |
| - independent video, audio, and combined timeline fingerprints; |
| - original, archived, and run-owned recovery locations. |
|
|
| The descriptor never requires a decoded full-video IMAGE batch. Scene consumers |
| request one overlap-aware 24-fps window. Path-backed audio is also decoded only |
| for the requested scene. If the only available audio is a ComfyUI tensor, Loop |
| Start materializes one normalized run-owned file and places only its descriptor |
| in recursive state and manifests. |
|
|
| Primary consumers are Loop Start and Tagged Motion Ref. Current Shot obtains |
| scene slices from chain state; Review, Manifest Load, and Assemble obtain the |
| descriptor from the saved manifest. Legacy VIDEO/AUDIO sockets remain adapters. |
|
|
| ## Audio intent contract |
|
|
| Audio intent uses `h3_audio_policy_v1`, three independent axes, and an optional |
| exact-target switch: |
|
|
| | Axis | Values | Meaning | |
| |---|---|---| |
| | Final audio | `generated`, `source`, `none` | What Assemble places in the final MP4 | |
| | Source reference | `on`, `off` | Whether the active source window guides H3 generation | |
| | Generated continuity | `on`, `off` | Whether the prior sampled audio latent continues into the next scene, including AV-mask prefixes | |
| | Source audio target | omitted, `locked` | Whether the scene-local source window replaces and fully protects H3's complete target audio latent | |
|
|
| The compact **Lock source audio** switch emits `source_audio_target=locked` and |
| canonically resolves Source reference and Generated continuity to `off`. This |
| prevents a loose Ref2VA audio bank or predecessor prefix from competing with the |
| exact target clock. Chain Context audio-VAE encodes the current source window, |
| crops only right-side encoder padding to the target's authoritative 40 Hz grid, |
| and composes an all-zero audio denoise mask with the selected video boundary. |
| Final-audio selection remains independent; `source` uses the Source Timeline |
| track for assembly. |
|
|
| Paired audio on a tagged motion reference is a fourth, reference-local decision: |
| `embedded` or `off`. It never selects the final soundtrack implicitly. |
|
|
| Legacy Plan modes migrate exactly: |
|
|
| | 0.4 `audio_mode` | Final | Source reference | Generated continuity | |
| |---|---|---|---| |
| | `source_track` | source | on | off | |
| | `generated_audio` | generated | off | on | |
| | `source_plus_timeline` | source | on | on | |
|
|
| New Plan nodes default to generated final audio, no source reference, generated |
| continuity, and no source target lock. Saved 0.4 widget values retain their old |
| behavior. |
|
|
| Normal workflows author both contracts through `h3_chain_policy_v1`, a |
| one-wire wrapper. Plan immediately expands it back into the canonical Audio |
| Policy and Transition Policy records before hashing, so compatibility follows |
| resolved behavior rather than the authoring graph. The compact node derives |
| audio overlap from the selected boundary (Cut 0, Guide 22, AV 39). An |
| Advanced Policy Override composes a named experimental transition over that |
| policy without replacing its audio record. An independent numeric audio |
| overlap is available only through the Legacy 0.4 Policy Adapter. |
| Policy nodes are composable wrappers over the same `h3_chain_policy_v1` |
| record. Chain Policy owns audio intent and a normal transition. Advanced |
| Policy replaces only the transition with a named semantic recipe. A connected |
| Legacy Adapter preserves the incoming audio record and replaces only its raw |
| transition/audio-context fields; standalone, it also translates 0.4 |
| `audio_mode`. The policy layer nearest Plan owns the effective boundary. |
|
|
| ## Incoming transition contract |
|
|
| Transitions use `h3_transition_policy_v1`. A scene setting always describes |
| how that scene consumes its predecessor; it does not retroactively redefine |
| the completed predecessor. |
|
|
| | Preset | Continuation implementation | Context | |
| |---|---|---:| |
| | `cut` | guide with no carried picture | 0 frames | |
| | `guide` | guide rows | 22 frames | |
| | `hard_av` | protected AV prefix | 39 frames | |
| | `soft_av` | exact picture prefix with feathered audio release | 39 frames | |
|
|
| The normal selector intentionally exposes only those four tested choices. |
| Advanced Policy Override selects named semantic recipes including |
| `tone_guide`, `latent_guide`, `detail_guide`, `detail_av`, `drift_av`, and |
| `color_drift_av` while preserving the upstream audio intent. The Legacy 0.4 |
| Policy Adapter owns raw implementation, visual-context, and audio-context |
| values, including the old dual-stream `feathered_av` implementation. |
| `latent_guide` requires video encode mode and at least five positive context |
| frames. `tapered_guide` accepts the listed Guide context lengths; only the |
| 22-frame preset has published validation, so other lengths remain |
| experimental. The resolved values, not merely the preset name, enter scene |
| metadata. |
|
|
| Detail AV is deliberately narrower than the generic AV implementations. Its |
| v2 recipe requires exactly 39 frames / 12 video-latent steps, applies |
| matched-standard-deviation Gaussian noise only to a disposable copy of the |
| carried video prefix, and tapers 0.30 -> 0.225 -> 0.15 -> 0.075 -> 0.00 over |
| the final four steps. Audio and masks retain Hard AV semantics. The recipe |
| version, |
| parameters, and deterministic seed rule are part of `incoming_boundary`, so a |
| resume cannot silently cross an implementation change. |
|
|
| Drift-Control AV keeps the predecessor latent clean and changes only the |
| disposable incoming prefix mask. For each model evaluation it resolves the next |
| strictly lower scheduler sigma, applies `next/current` to the oldest eight |
| video steps, and tapers the final four `.75/.50/.25/.00`. Chain Context's MODEL |
| output couples the sampler-side inpaint blend to H3's per-row timestep mask; |
| using only one of those paths would give the model a mislabeled prefix. The |
| recipe is dependency-hashed and initially validated at 20 steps. |
|
|
| Color-Stable Drift AV keeps that exact mask/sampler behavior and adds a |
| video-only scene-one color anchor to the disposable copied prefix. It measures |
| the retained RGB tail of the first generated scene and the current predecessor, |
| decodes the 12-step prefix, and computes |
| `E(weakly graded decode) - E(original decode)`. Only that delta is added to the |
| sampled prefix, so ordinary VAE round-trip bias is cancelled instead of |
| replacing the sampled latent with a re-encode. A 3x3 latent low-pass limits the |
| change to broad tone/chroma, and a full-prefix smoothstep keeps the oldest |
| overlap step exact while reaching the bounded correction beside the generated |
| future. Brightness is capped at six code values, saturation at six percent, |
| and both use half the measured correction. The predecessor checkpoint and |
| audio latent are never modified. The complete recipe is dependency-hashed. |
|
|
| When a workflow splits sigmas and switches H3 models, Chain Context receives |
| the original unsplit schedule but no MODEL. Each raw model passes through its |
| own lightweight Drift-Control Model Patch with the same Current Shot state, |
| Chain Context latent, and full schedule. This keeps next-sigma selection |
| continuous across the sampler boundary without making Chain Context own either |
| model loader or retaining duplicate weight tensors. |
|
|
| ## Scene dependency contract |
|
|
| Each accepted scene stores `h3_scene_dependency_v1`. Dependencies have four |
| scopes: |
|
|
| 1. `global_generation`: model, VAE, LoRA, sampler, scheduler, CFG, and other |
| generation-body configuration. |
| 2. `scene_generation`: prompt, seed, raw length, steps, active references, and |
| scene-local media windows. |
| 3. `incoming_boundary`: the transition and context used to enter that scene. |
| 4. `assembly_only`: final mux media that did not guide generation. |
|
|
| Resume through scene N compares scopes 1–3 only for accepted scenes 1..N. |
| Changing scene N+1 or its incoming boundary cannot invalidate scene N. Changing |
| an assembly-only source track does not invalidate sampled video. If source |
| audio guided a scene, only that scene's canonical PCM window is a generation |
| dependency; unrelated future audio is not. |
|
|
| Every mismatch report identifies its scope, scene, field, saved value, current |
| value, and whether regeneration is required. |
|
|
| ## Compatibility rules |
|
|
| - Existing node ids remain registered. |
| - Existing outputs retain their positional indices. |
| - Existing required widget order remains readable. |
| - Plan's retired `audio_mode`, `continuation_mode`, `context_length`, and |
| `audio_context_length` widgets remain serialized for 0.4 compatibility but |
| are hidden in the normal 0.5 presentation. Legacy 0.4 Policy Adapter exposes |
| those choices as an explicit compatibility route and emits one combined |
| policy output. |
| - New policy fields are appended or introduced through frontend-backed |
| migration rather than inserted into old positional layouts. |
| - Legacy `audio_mode`, full AUDIO fan-out, direct media paths, and manual |
| generation fingerprints remain accepted for 0.5. |
| - Existing checkpoint formats retain their current generic hash fallback when |
| structured dependency records are unavailable. |
| - Diagnostic and legacy sockets may be visually hidden, but the backend slots |
| remain until a later breaking release. |
|
|
| ## Preflight contract |
|
|
| The same pure preflight implementation serves Loop Start and Plan Studio. It |
| runs before model-dependent sampling and reports: |
|
|
| - resolved scene frame counts, durations, source windows, and overlap trims; |
| - native-FPS/PTS to 24-fps mapping and skip origin; |
| - source duration, required duration, exact shortfall, and last complete scene; |
| - active/unresolved reference tags and their scene windows; |
| - source and archive availability; |
| - runtime guide, mask, and known attention-wrapper compatibility; |
| - automatic generation-body fingerprint coverage; |
| - resume eligibility plus structured mismatches. |
|
|
| Errors include a user action. Sample counts and latent-grid details may appear |
| under diagnostics, not as the primary explanation. |
|
|
| Plan Studio also consumes those same reference-window results for its motion |
| comparison track. It does not decode the full reference into IMAGE tensors. |
| The server seeks and transcodes only the selected scene window to a cached |
| low-resolution MP4; the comparison player offsets Guide windows past the |
| incoming context that is removed from the delivered scene. |
|
|
| ## Socket presentation rules |
|
|
| The primary graph displays only generation-bearing connections. Status, |
| manifest JSON, booleans used only for inspection, legacy passthroughs, and |
| conditional audio sockets are Advanced. Conditional inputs appear when their |
| policy needs them. Superseded Plan policy and layout widgets are also hidden; |
| the node menu can reveal them for diagnosis. Hiding a socket or widget must not |
| change its backend index or serialized position. |
|
|
| ## Delivery order |
|
|
| 1. Freeze contracts and 0.4 fixtures. |
| 2. Implement Source Timeline and legacy adapters. |
| 3. Introduce independent audio policies. |
| 4. Add transition presets. |
| 5. Migrate all consumers to the timeline. |
| 6. Add preflight. |
| 7. Store and compare structured scene dependencies. |
| 8. Hide redundant sockets without changing positions. |
| 9. Clarify Run Manager and prompt-history state. |
| 10. Run migration, integration, and release validation. |
|
|
| ## Release validation |
|
|
| All ten delivery stages are implemented. The maintained workflow catalog uses |
| one-wire Chain Policy, Source Timeline, and model-free |
| preflight. The migration tool is idempotent, the frozen 0.4 positional contract |
| is covered by regression tests, and backend/frontend release checks enforce a |
| single package version. Archived 0.4 workflows remain unchanged examples of the |
| supported compatibility route. |
|
|
