dzmdd's picture
Upload folder using huggingface_hub
e488ce7 verified
|
Raw History Blame Contribute Delete
6.22 kB

MAI video tools (alpha)

Three nodes and one widget that remove drudgery around renders without taking any decision away from you. None of them runs a model.

MAI Video Out (file + sidecars + draft preview)

Save Video with the record beside it. Frames (+ audio) go through ComfyUI's own encoder (h264 / av1, CRF), exactly like Create Video + Save Video; the extras are each off unless wired or toggled:

  • hold_map -> <prefix>_00001_.holdmap.json: the clock that produced these frames. H3 Load Hold Map and the deck's hold-map staircase read it.
  • write_meta -> <prefix>_00001_.meta.json: seeds, models, LoRAs, steps, samplers, inject schedule, encode time, your notes, all read out of the graph that ran. The compare viewer's metadata strip and the scorer's labels come from here instead of from file names.
  • draft_preview + latent: decode the sampled latent through the tiny VAE in models/vae_approx (taeh3 for H3; any stem) into <prefix>_draft.mp4. The tiny VAE is loaded for that call and released after; with the toggle off nothing is loaded and nothing runs. Cost measured 2026-08-21: 8 ms per latent frame at 864x480, 21 ms at 1344x768, 42 ms at 1080p.

MAI Load Video (path) / MAI Select Every Nth

The two things the graphs still took from VideoHelperSuite: a video by absolute or input-relative path (as ComfyUI's VIDEO type, frames and fps alongside), and every n-th frame from an offset. For a smeared clip use H3 Exact Recover instead; it is exact.

MAI Video Compare (2-6, synchronized)

The player has grown since this page was written: a waveform timeline with loop brackets, a frame stepper, and a frame-exact export. The README's MAI Video Compare section is the current reference; this page keeps the original wiring notes.

A media UI, not a processing stage. The node writes each wired VIDEO once as a small h264 preview into the temp directory (CPU, no VAE, no tensors kept) and the widget plays them in the browser with the flipbook's player: one set of live <video> elements, buffered start-together, flip / wipe / side-by-side for a pair, a grid for more.

  • hover a source to hear it, click to lock its audio
  • space play/pause all, arrows step all one frame (shift: 12), F flicker A/B
  • 1-6 pick the pair (last two pressed), Enter stars the B side
  • the star sets the node's winner widget; the NEXT queue passes that source through winner_video (and winner_index). Pick, then finalize in a second execution: a graph never waits on a human mid-run.

The flipbook deck (https://matlowai.github.io/flipbook/derope.html) is the offline twin, same player code.

MAI Seed Hunter (compare + winner seed)

Video Compare that knows about seeds: wire 2-6 cheap candidates and the seed each ran on (labels default to the seed), star the keeper, and the NEXT queue passes the starred candidate's seed out of winner_seed for the finalize pass's RandomNoise (and its video out of winner_video). The seed-hunt pass and the finalize pass are two executions by design.

MAI Motion Editor additions

Alpha. The editor's rows, modal, undo and run buttons have only been exercised by an automated browser so far; expect rough edges, and treat anything surprising as a bug to report rather than a design choice.

Verified headless against a lab instance (2026-08-22, Playwright): run-to-here renders base + oracle + filmstrip and holds the graph; a drag on the block lane creates exactly the dragged range (snapped to the token grid); the live envelope drives the dilated clock, the play loop and the price before any render; the modal applies as one undo step; run-from-here resumes at the editor with the whole base chain from ComfyUI's cache (14 nodes cached, editor + downstream executed); a fresh page load restores the last run's filmstrip from the server.

  • run to here / run from here on the node: the first is partial execution (nothing downstream), the second a normal queue where unchanged upstream nodes come from the cache, so it resumes at this node. Editing a row then "run from here" is the retry loop.

  • Undo: the node's undo/redo is the fine-grained one (every row edit, drag, modal apply). ComfyUI's own ctrl+z jumps back to the graph's last snapshot (the widget re-hydrates from it and says so); the graph tracker does not see edits made inside the widget individually, so ctrl+shift+z at graph level has nothing to redo after that. Use the node's buttons or ctrl+z/ctrl+shift+z while the editor has focus.

  • hold_until_edited (default on): with no rows, a run stops after the filmstrip; lay out rows and run again.

  • Cache reality: ComfyUI keeps only the previous prompt's outputs, so running other graphs in between evicts the base pass and the next run-from-here re-renders it. The other way to lose the cache is a seed that moves: the frontend's default "control after generate" is randomize on nodes it builds, so the plain Run button re-renders the base every time until both RandomNoise nodes are set to fixed. The node's run buttons pin every seed control to fixed before queueing.

  • Play (space): the filmstrip plays at fps; in the dilated clock each frame lingers for its hold, so the slowdown is seen, not inferred.

  • clock: world / dilated: the timeline's x-axis reads as "when" (frame) or as "how long it costs" (cumulative holds). The compiled hold curve from the last run is drawn over the jerk profile in both.

Paste an API workflow

With the pack installed, Ctrl-V of API-format JSON on the canvas works: the stock paste handler only takes UI-format workflows, so web/api_paste.js catches API JSON first and routes it through the frontend's own loadApiJson (widgets built from the server schema, never by position). It replaces the graph, so a non-empty canvas asks first. The deck's "api" links and every *_api.json in examples/ paste directly now.

What these replace

Our graphs used VHS_LoadVideoPath and VHS_SelectEveryNthImage; both are covered. Create Video + Save Video stay valid; MAI Video Out is those plus the sidecars. The compare node replaces opening three files and scrubbing them to roughly the same place.