Upload folder using huggingface_hub
Browse filesThis view is limited to 50 files because it contains too many changes. See raw diff
- .gitattributes +25 -0
- ComfyUI-MiniMaxH3-Context-Loop/.github/workflows/publish_action.yml +24 -0
- ComfyUI-MiniMaxH3-Context-Loop/.gitignore +3 -0
- ComfyUI-MiniMaxH3-Context-Loop/AGENT_PROMPT_ASSISTANT_STUDY.md +328 -0
- ComfyUI-MiniMaxH3-Context-Loop/CHANGELOG.md +1347 -0
- ComfyUI-MiniMaxH3-Context-Loop/CONTRIBUTING.md +75 -0
- ComfyUI-MiniMaxH3-Context-Loop/ComfyUI-MAINodes/.github/FUNDING.yml +1 -0
- ComfyUI-MiniMaxH3-Context-Loop/ComfyUI-MAINodes/.github/workflows/nightly-publish.yml +97 -0
- ComfyUI-MiniMaxH3-Context-Loop/ComfyUI-MAINodes/.github/workflows/publish-node.yml +22 -0
- ComfyUI-MiniMaxH3-Context-Loop/ComfyUI-MAINodes/.gitignore +2 -0
- ComfyUI-MiniMaxH3-Context-Loop/ComfyUI-MAINodes/ALPHA.md +294 -0
- ComfyUI-MiniMaxH3-Context-Loop/ComfyUI-MAINodes/DEROPE_ANY_MODEL.md +156 -0
- ComfyUI-MiniMaxH3-Context-Loop/ComfyUI-MAINodes/FASTER.md +49 -0
- ComfyUI-MiniMaxH3-Context-Loop/ComfyUI-MAINodes/HARDWARE.md +102 -0
- ComfyUI-MiniMaxH3-Context-Loop/ComfyUI-MAINodes/LICENSE +679 -0
- ComfyUI-MiniMaxH3-Context-Loop/ComfyUI-MAINodes/LOWVRAM.md +323 -0
- ComfyUI-MiniMaxH3-Context-Loop/ComfyUI-MAINodes/LOWVRAM_lines.html +53 -0
- ComfyUI-MiniMaxH3-Context-Loop/ComfyUI-MAINodes/README.md +1119 -0
- ComfyUI-MiniMaxH3-Context-Loop/ComfyUI-MAINodes/RESEARCH_NOTES_ATOS.md +234 -0
- ComfyUI-MiniMaxH3-Context-Loop/ComfyUI-MAINodes/ROADMAP.md +355 -0
- ComfyUI-MiniMaxH3-Context-Loop/ComfyUI-MAINodes/TESTING_ALPHA.md +152 -0
- ComfyUI-MiniMaxH3-Context-Loop/ComfyUI-MAINodes/TUNING.md +618 -0
- ComfyUI-MiniMaxH3-Context-Loop/ComfyUI-MAINodes/VIDEO_TOOLS.md +116 -0
- ComfyUI-MiniMaxH3-Context-Loop/ComfyUI-MAINodes/__init__.py +53 -0
- ComfyUI-MiniMaxH3-Context-Loop/ComfyUI-MAINodes/assets/adapter_chaos_derope_triptych.mp4 +3 -0
- ComfyUI-MiniMaxH3-Context-Loop/ComfyUI-MAINodes/assets/adapter_drop_full3x_triptych.mp4 +3 -0
- ComfyUI-MiniMaxH3-Context-Loop/ComfyUI-MAINodes/assets/adapter_mage_pinned_quad.mp4 +3 -0
- ComfyUI-MiniMaxH3-Context-Loop/ComfyUI-MAINodes/assets/adapter_ref2va_colour_muting.mp4 +3 -0
- ComfyUI-MiniMaxH3-Context-Loop/ComfyUI-MAINodes/assets/adapter_rooftop_ood_triptych.mp4 +3 -0
- ComfyUI-MiniMaxH3-Context-Loop/ComfyUI-MAINodes/assets/adapter_t2c_insert_quad.mp4 +3 -0
- ComfyUI-MiniMaxH3-Context-Loop/ComfyUI-MAINodes/assets/baseline_vs_regenerated_sbs.mp4 +3 -0
- ComfyUI-MiniMaxH3-Context-Loop/ComfyUI-MAINodes/assets/derope_sbs.gif +3 -0
- ComfyUI-MiniMaxH3-Context-Loop/ComfyUI-MAINodes/assets/featherweight_flf_triptych.mp4 +3 -0
- ComfyUI-MiniMaxH3-Context-Loop/ComfyUI-MAINodes/assets/featherweight_i2v_triptych.mp4 +3 -0
- ComfyUI-MiniMaxH3-Context-Loop/ComfyUI-MAINodes/assets/featherweight_int8_vs_w4a8_quad.mp4 +3 -0
- ComfyUI-MiniMaxH3-Context-Loop/ComfyUI-MAINodes/assets/featherweight_ref_triptych.mp4 +3 -0
- ComfyUI-MiniMaxH3-Context-Loop/ComfyUI-MAINodes/assets/kitsune_threeway.gif +3 -0
- ComfyUI-MiniMaxH3-Context-Loop/ComfyUI-MAINodes/assets/kitsune_threeway_sbs.mp4 +3 -0
- ComfyUI-MiniMaxH3-Context-Loop/ComfyUI-MAINodes/assets/oracle_map.gif +3 -0
- ComfyUI-MiniMaxH3-Context-Loop/ComfyUI-MAINodes/assets/oracle_map.mp4 +3 -0
- ComfyUI-MiniMaxH3-Context-Loop/ComfyUI-MAINodes/assets/panrun_sbs.mp4 +3 -0
- ComfyUI-MiniMaxH3-Context-Loop/ComfyUI-MAINodes/assets/preview_ladder_grid.mp4 +3 -0
- ComfyUI-MiniMaxH3-Context-Loop/ComfyUI-MAINodes/assets/time_burst_plate.mp4 +3 -0
- ComfyUI-MiniMaxH3-Context-Loop/ComfyUI-MAINodes/assets/time_burst_result.mp4 +3 -0
- ComfyUI-MiniMaxH3-Context-Loop/ComfyUI-MAINodes/assets/uniform_vs_adaptive.gif +3 -0
- ComfyUI-MiniMaxH3-Context-Loop/ComfyUI-MAINodes/assets/uniform_vs_adaptive_sbs.mp4 +3 -0
- ComfyUI-MiniMaxH3-Context-Loop/ComfyUI-MAINodes/capsule_types.py +185 -0
- ComfyUI-MiniMaxH3-Context-Loop/ComfyUI-MAINodes/concept_lab/DECISIONS.md +543 -0
- ComfyUI-MiniMaxH3-Context-Loop/ComfyUI-MAINodes/concept_lab/README.md +224 -0
- ComfyUI-MiniMaxH3-Context-Loop/ComfyUI-MAINodes/concept_lab/__init__.py +55 -0
.gitattributes
CHANGED
|
@@ -1264,3 +1264,28 @@ checkpoint_h3_context_loop/h3_chains/twerk_sur_lit_fl2va_2/reviews/clip_0003.2d3
|
|
| 1264 |
checkpoint_h3_context_loop/h3_chains/twerk_sur_lit_fl2va_2/segments/clip_0001.6b9aeb22d21146e3811ef0e8c3d3f9be.mp4 filter=lfs diff=lfs merge=lfs -text
|
| 1265 |
checkpoint_h3_context_loop/h3_chains/twerk_sur_lit_fl2va_2/segments/clip_0002.2f3e5fc154f04b70aa603272e3509f79.mp4 filter=lfs diff=lfs merge=lfs -text
|
| 1266 |
checkpoint_h3_context_loop/h3_chains/twerk_sur_lit_fl2va_2/segments/clip_0003.bb8250496ae84b96ae7a727f88420d5f.mp4 filter=lfs diff=lfs merge=lfs -text
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1264 |
checkpoint_h3_context_loop/h3_chains/twerk_sur_lit_fl2va_2/segments/clip_0001.6b9aeb22d21146e3811ef0e8c3d3f9be.mp4 filter=lfs diff=lfs merge=lfs -text
|
| 1265 |
checkpoint_h3_context_loop/h3_chains/twerk_sur_lit_fl2va_2/segments/clip_0002.2f3e5fc154f04b70aa603272e3509f79.mp4 filter=lfs diff=lfs merge=lfs -text
|
| 1266 |
checkpoint_h3_context_loop/h3_chains/twerk_sur_lit_fl2va_2/segments/clip_0003.bb8250496ae84b96ae7a727f88420d5f.mp4 filter=lfs diff=lfs merge=lfs -text
|
| 1267 |
+
ComfyUI-MiniMaxH3-Context-Loop/ComfyUI-MAINodes/assets/adapter_chaos_derope_triptych.mp4 filter=lfs diff=lfs merge=lfs -text
|
| 1268 |
+
ComfyUI-MiniMaxH3-Context-Loop/ComfyUI-MAINodes/assets/adapter_drop_full3x_triptych.mp4 filter=lfs diff=lfs merge=lfs -text
|
| 1269 |
+
ComfyUI-MiniMaxH3-Context-Loop/ComfyUI-MAINodes/assets/adapter_mage_pinned_quad.mp4 filter=lfs diff=lfs merge=lfs -text
|
| 1270 |
+
ComfyUI-MiniMaxH3-Context-Loop/ComfyUI-MAINodes/assets/adapter_ref2va_colour_muting.mp4 filter=lfs diff=lfs merge=lfs -text
|
| 1271 |
+
ComfyUI-MiniMaxH3-Context-Loop/ComfyUI-MAINodes/assets/adapter_rooftop_ood_triptych.mp4 filter=lfs diff=lfs merge=lfs -text
|
| 1272 |
+
ComfyUI-MiniMaxH3-Context-Loop/ComfyUI-MAINodes/assets/adapter_t2c_insert_quad.mp4 filter=lfs diff=lfs merge=lfs -text
|
| 1273 |
+
ComfyUI-MiniMaxH3-Context-Loop/ComfyUI-MAINodes/assets/baseline_vs_regenerated_sbs.mp4 filter=lfs diff=lfs merge=lfs -text
|
| 1274 |
+
ComfyUI-MiniMaxH3-Context-Loop/ComfyUI-MAINodes/assets/derope_sbs.gif filter=lfs diff=lfs merge=lfs -text
|
| 1275 |
+
ComfyUI-MiniMaxH3-Context-Loop/ComfyUI-MAINodes/assets/featherweight_flf_triptych.mp4 filter=lfs diff=lfs merge=lfs -text
|
| 1276 |
+
ComfyUI-MiniMaxH3-Context-Loop/ComfyUI-MAINodes/assets/featherweight_i2v_triptych.mp4 filter=lfs diff=lfs merge=lfs -text
|
| 1277 |
+
ComfyUI-MiniMaxH3-Context-Loop/ComfyUI-MAINodes/assets/featherweight_int8_vs_w4a8_quad.mp4 filter=lfs diff=lfs merge=lfs -text
|
| 1278 |
+
ComfyUI-MiniMaxH3-Context-Loop/ComfyUI-MAINodes/assets/featherweight_ref_triptych.mp4 filter=lfs diff=lfs merge=lfs -text
|
| 1279 |
+
ComfyUI-MiniMaxH3-Context-Loop/ComfyUI-MAINodes/assets/kitsune_threeway.gif filter=lfs diff=lfs merge=lfs -text
|
| 1280 |
+
ComfyUI-MiniMaxH3-Context-Loop/ComfyUI-MAINodes/assets/kitsune_threeway_sbs.mp4 filter=lfs diff=lfs merge=lfs -text
|
| 1281 |
+
ComfyUI-MiniMaxH3-Context-Loop/ComfyUI-MAINodes/assets/oracle_map.gif filter=lfs diff=lfs merge=lfs -text
|
| 1282 |
+
ComfyUI-MiniMaxH3-Context-Loop/ComfyUI-MAINodes/assets/oracle_map.mp4 filter=lfs diff=lfs merge=lfs -text
|
| 1283 |
+
ComfyUI-MiniMaxH3-Context-Loop/ComfyUI-MAINodes/assets/panrun_sbs.mp4 filter=lfs diff=lfs merge=lfs -text
|
| 1284 |
+
ComfyUI-MiniMaxH3-Context-Loop/ComfyUI-MAINodes/assets/preview_ladder_grid.mp4 filter=lfs diff=lfs merge=lfs -text
|
| 1285 |
+
ComfyUI-MiniMaxH3-Context-Loop/ComfyUI-MAINodes/assets/time_burst_plate.mp4 filter=lfs diff=lfs merge=lfs -text
|
| 1286 |
+
ComfyUI-MiniMaxH3-Context-Loop/ComfyUI-MAINodes/assets/time_burst_result.mp4 filter=lfs diff=lfs merge=lfs -text
|
| 1287 |
+
ComfyUI-MiniMaxH3-Context-Loop/ComfyUI-MAINodes/assets/uniform_vs_adaptive.gif filter=lfs diff=lfs merge=lfs -text
|
| 1288 |
+
ComfyUI-MiniMaxH3-Context-Loop/ComfyUI-MAINodes/assets/uniform_vs_adaptive_sbs.mp4 filter=lfs diff=lfs merge=lfs -text
|
| 1289 |
+
ComfyUI-MiniMaxH3-Context-Loop/ComfyUI-MAINodes/docs/latents/frame0.png filter=lfs diff=lfs merge=lfs -text
|
| 1290 |
+
ComfyUI-MiniMaxH3-Context-Loop/ComfyUI-MAINodes/docs/latents/frame72.png filter=lfs diff=lfs merge=lfs -text
|
| 1291 |
+
ComfyUI-MiniMaxH3-Context-Loop/__pycache__/chain_nodes.cpython-312.pyc filter=lfs diff=lfs merge=lfs -text
|
ComfyUI-MiniMaxH3-Context-Loop/.github/workflows/publish_action.yml
ADDED
|
@@ -0,0 +1,24 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
name: Publish to Comfy Registry
|
| 2 |
+
|
| 3 |
+
on:
|
| 4 |
+
workflow_dispatch:
|
| 5 |
+
push:
|
| 6 |
+
branches:
|
| 7 |
+
- main
|
| 8 |
+
paths:
|
| 9 |
+
- pyproject.toml
|
| 10 |
+
|
| 11 |
+
permissions:
|
| 12 |
+
contents: read
|
| 13 |
+
|
| 14 |
+
jobs:
|
| 15 |
+
publish-node:
|
| 16 |
+
name: Publish custom node
|
| 17 |
+
runs-on: ubuntu-latest
|
| 18 |
+
steps:
|
| 19 |
+
- name: Check out code
|
| 20 |
+
uses: actions/checkout@v4
|
| 21 |
+
- name: Publish custom node
|
| 22 |
+
uses: Comfy-Org/publish-node-action@main
|
| 23 |
+
with:
|
| 24 |
+
personal_access_token: ${{ secrets.REGISTRY_ACCESS_TOKEN }}
|
ComfyUI-MiniMaxH3-Context-Loop/.gitignore
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
__pycache__/
|
| 2 |
+
*.pyc
|
| 3 |
+
.release-drafts/
|
ComfyUI-MiniMaxH3-Context-Loop/AGENT_PROMPT_ASSISTANT_STUDY.md
ADDED
|
@@ -0,0 +1,328 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Agent-backed prompt assistant study
|
| 2 |
+
|
| 3 |
+
## Finding
|
| 4 |
+
|
| 5 |
+
Yes, the Scene Prompt Editor can host an interactive prompt assistant for Codex
|
| 6 |
+
and Hermes.
|
| 7 |
+
|
| 8 |
+
The first implementation now exists on this study branch together with the
|
| 9 |
+
`feature/prompt-assistant-bridge` comfyui-mcp branch. It follows the isolated,
|
| 10 |
+
staged design below. Codex uses app-server; the initial Hermes adapter uses its
|
| 11 |
+
tool-restricted one-shot interface while a persistent ACP adapter remains a
|
| 12 |
+
possible later optimization.
|
| 13 |
+
|
| 14 |
+
The recommended architecture is:
|
| 15 |
+
|
| 16 |
+
```text
|
| 17 |
+
Scene Prompt Editor DOM widget
|
| 18 |
+
|
|
| 19 |
+
| prompt-assist request / progress / staged result
|
| 20 |
+
v
|
| 21 |
+
comfyui-mcp local orchestrator
|
| 22 |
+
|
|
| 23 |
+
+-- Codex app-server backend
|
| 24 |
+
+-- Hermes tool-restricted one-shot adapter (initial)
|
| 25 |
+
+-- persistent ACP adapter (later option)
|
| 26 |
+
```
|
| 27 |
+
|
| 28 |
+
The assistant should have its own restricted conversation and a correlated
|
| 29 |
+
request/response protocol. It should not borrow the sidebar panel's main chat
|
| 30 |
+
session, open a second socket under the panel's workflow identity, or launch an
|
| 31 |
+
agent process from ComfyUI's Python server.
|
| 32 |
+
|
| 33 |
+
## What exists today
|
| 34 |
+
|
| 35 |
+
### This pack
|
| 36 |
+
|
| 37 |
+
`MiniMaxH3ChainScenePromptEditor` is a pass-through Python node. Its useful
|
| 38 |
+
behavior lives in `web/h3_chain_scene_prompt_editor.js` as a DOM widget.
|
| 39 |
+
|
| 40 |
+
The editor finds the upstream `MiniMaxH3ChainPlan`, parses its `plan_json`, and
|
| 41 |
+
writes the active scene's real `shots[n].prompt` back to that Plan widget on
|
| 42 |
+
every edit. There is no duplicate prompt value. This is a good base for an
|
| 43 |
+
assistant because a draft can be staged beside the textarea and applied through
|
| 44 |
+
the existing `writePlan()` path.
|
| 45 |
+
|
| 46 |
+
### comfyui-mcp and its panel
|
| 47 |
+
|
| 48 |
+
The panel already connects a browser client to a local `comfyui-mcp`
|
| 49 |
+
orchestrator over WebSocket. The orchestrator owns provider authentication,
|
| 50 |
+
agent sessions, streaming, model discovery, interruption, and live ComfyUI
|
| 51 |
+
tools. Codex is a native orchestrator backend.
|
| 52 |
+
|
| 53 |
+
Hermes support currently means `comfyui-mcp setup hermes`: it registers the
|
| 54 |
+
ComfyUI MCP server in Hermes so a Hermes console agent can control ComfyUI.
|
| 55 |
+
Hermes is not currently a panel-orchestrator backend and does not speak the
|
| 56 |
+
panel bridge protocol.
|
| 57 |
+
|
| 58 |
+
The panel bridge cannot safely be reused as-is by this node:
|
| 59 |
+
|
| 60 |
+
- its bridge client is private to the panel bundle;
|
| 61 |
+
- a workflow route is designed to have one live browser socket;
|
| 62 |
+
- the orchestrator deliberately shares one conversation per provider across
|
| 63 |
+
panel tabs and workflows;
|
| 64 |
+
- `user_message` replies are chat frames, not request-correlated rewrite
|
| 65 |
+
results.
|
| 66 |
+
|
| 67 |
+
A second node client using the current `hello`/`user_message` protocol could
|
| 68 |
+
replace the panel's route or mix rewrite output into the user's main panel
|
| 69 |
+
conversation.
|
| 70 |
+
|
| 71 |
+
### Agent-native integration surfaces
|
| 72 |
+
|
| 73 |
+
Codex documents app-server as the interface for a deep product integration. It
|
| 74 |
+
provides threads, turns, streamed events, approvals, interruption, and JSON-RPC.
|
| 75 |
+
`codex exec` is intended for non-interactive scripts and is useful for a small
|
| 76 |
+
one-shot proof of concept, but app-server (or the Codex SDK which controls it)
|
| 77 |
+
is the appropriate persistent implementation. `comfyui-mcp` already contains a
|
| 78 |
+
Codex app-server adapter, so that work should be reused.
|
| 79 |
+
|
| 80 |
+
The installed Hermes CLI exposes three relevant surfaces:
|
| 81 |
+
|
| 82 |
+
- `hermes --oneshot` for a scriptable one-shot response;
|
| 83 |
+
- `hermes --resume <session>` for continuing a stored session;
|
| 84 |
+
- `hermes acp` and `hermes serve` for editor/backend integrations.
|
| 85 |
+
|
| 86 |
+
ACP or the backend service is preferable for the final persistent assistant.
|
| 87 |
+
One-shot mode is sufficient to validate prompt quality and UX before adding a
|
| 88 |
+
full Hermes backend adapter.
|
| 89 |
+
|
| 90 |
+
## Recommended user experience
|
| 91 |
+
|
| 92 |
+
Add a collapsible **Prompt Assistant** area below the scene textarea:
|
| 93 |
+
|
| 94 |
+
- provider: Codex or Hermes;
|
| 95 |
+
- action presets: Rewrite, Improve continuity, Shorten, Critique;
|
| 96 |
+
- a free-form instruction box and Send/Stop buttons;
|
| 97 |
+
- a compact transcript for follow-up requests;
|
| 98 |
+
- a staged draft view with Apply, Copy, and Discard;
|
| 99 |
+
- connection, working, and error status that does not replace the editor's
|
| 100 |
+
existing Plan synchronization status.
|
| 101 |
+
|
| 102 |
+
The agent's result must be staged. Streaming tokens must never mutate the Plan.
|
| 103 |
+
Only an explicit Apply/Replace action should call the existing `writePlan()`
|
| 104 |
+
path.
|
| 105 |
+
|
| 106 |
+
Suggested interaction:
|
| 107 |
+
|
| 108 |
+
1. The user edits the current scene prompt.
|
| 109 |
+
2. The user asks, for example, "Make the camera movement more precise and keep
|
| 110 |
+
the unfinished action for scene 4."
|
| 111 |
+
3. The node sends the current scene, shared prompt, and bounded neighboring
|
| 112 |
+
continuity context.
|
| 113 |
+
4. The UI shows working/progress state while the assistant builds a structured
|
| 114 |
+
result.
|
| 115 |
+
5. The node shows the proposed prompt separately.
|
| 116 |
+
6. The user applies or discards it, then can continue chatting about the staged
|
| 117 |
+
draft.
|
| 118 |
+
|
| 119 |
+
The entire plan should not be sent by default. Useful bounded context is:
|
| 120 |
+
|
| 121 |
+
- shared prompt;
|
| 122 |
+
- current scene id, duration, and prompt;
|
| 123 |
+
- previous scene's ending and next scene's opening (or the complete neighboring
|
| 124 |
+
prompts while they remain under a strict size limit);
|
| 125 |
+
- reference tags available to the workflow;
|
| 126 |
+
- the prompt-writing rules bundled with this pack.
|
| 127 |
+
|
| 128 |
+
## Implemented bridge protocol
|
| 129 |
+
|
| 130 |
+
The implementation adds a prompt-assistant client kind and correlated frames to
|
| 131 |
+
`comfyui-mcp`.
|
| 132 |
+
|
| 133 |
+
Client request:
|
| 134 |
+
|
| 135 |
+
```json
|
| 136 |
+
{
|
| 137 |
+
"type": "prompt_assist_request",
|
| 138 |
+
"request_id": "pa-uuid",
|
| 139 |
+
"conversation_id": "h3-browser-instance-uuid",
|
| 140 |
+
"provider": "codex",
|
| 141 |
+
"mode": "rewrite",
|
| 142 |
+
"instruction": "Improve camera precision and ending continuity.",
|
| 143 |
+
"source_revision": "opaque-source-text-and-scene-id-revision",
|
| 144 |
+
"context": {
|
| 145 |
+
"scene_id": "clip_0003",
|
| 146 |
+
"source_prompt": "...",
|
| 147 |
+
"selected_text": null,
|
| 148 |
+
"shared_prompt": "...",
|
| 149 |
+
"previous_prompt": "...",
|
| 150 |
+
"next_prompt": "..."
|
| 151 |
+
}
|
| 152 |
+
}
|
| 153 |
+
```
|
| 154 |
+
|
| 155 |
+
Server frames:
|
| 156 |
+
|
| 157 |
+
```json
|
| 158 |
+
{"type":"prompt_assist_started","request_id":"pa-uuid","provider":"codex"}
|
| 159 |
+
{"type":"prompt_assist_progress","request_id":"pa-uuid"}
|
| 160 |
+
{
|
| 161 |
+
"type": "prompt_assist_result",
|
| 162 |
+
"request_id": "pa-uuid",
|
| 163 |
+
"source_revision": "opaque-source-text-and-scene-id-revision",
|
| 164 |
+
"message": "What changed and why.",
|
| 165 |
+
"rewritten_prompt": "The complete proposed scene prompt."
|
| 166 |
+
}
|
| 167 |
+
{"type":"prompt_assist_error","request_id":"pa-uuid","error":"..."}
|
| 168 |
+
```
|
| 169 |
+
|
| 170 |
+
Cancellation:
|
| 171 |
+
|
| 172 |
+
```json
|
| 173 |
+
{"type":"prompt_assist_cancel","request_id":"pa-uuid"}
|
| 174 |
+
```
|
| 175 |
+
|
| 176 |
+
The canonical final result is schema-constrained. A conversational message and
|
| 177 |
+
the complete replacement prompt are separate fields. Codex enforces the schema;
|
| 178 |
+
the initial Hermes adapter also accepts a plain-text fallback so older local
|
| 179 |
+
Hermes models remain usable.
|
| 180 |
+
|
| 181 |
+
For an HTTPS-hosted ComfyUI, use the panel's existing advertised secure bridge
|
| 182 |
+
mechanism. A page served over HTTPS cannot directly use an insecure localhost
|
| 183 |
+
WebSocket.
|
| 184 |
+
|
| 185 |
+
## Session and tool boundaries
|
| 186 |
+
|
| 187 |
+
Use a session key such as:
|
| 188 |
+
|
| 189 |
+
```text
|
| 190 |
+
prompt-assistant::<client-id>::<node-id>::<provider>
|
| 191 |
+
```
|
| 192 |
+
|
| 193 |
+
This session must be separate from `orchestrator::<provider>`, the panel's main
|
| 194 |
+
conversation. Switching providers should preserve each provider's assistant
|
| 195 |
+
session, as the panel already does for its provider conversations.
|
| 196 |
+
|
| 197 |
+
The prompt assistant should be text-only and tool-free by default:
|
| 198 |
+
|
| 199 |
+
- no shell or file editing;
|
| 200 |
+
- no graph mutation tools;
|
| 201 |
+
- no workflow execution;
|
| 202 |
+
- no web access unless the user explicitly enables it later;
|
| 203 |
+
- read-only, minimal sandbox for Codex;
|
| 204 |
+
- bounded prompt, response, turn duration, and queued request count.
|
| 205 |
+
|
| 206 |
+
This keeps "rewrite my prompt" from becoming an implicit authorization for an
|
| 207 |
+
agent to edit the workspace or run the graph. If graph-aware tools are wanted
|
| 208 |
+
later, they should be a separate explicit mode.
|
| 209 |
+
|
| 210 |
+
The current console session should not be scraped through a pseudo-terminal or
|
| 211 |
+
shared concurrently. A console and the node may resume the same stored session
|
| 212 |
+
by id at different times, but the normal node experience should use a dedicated
|
| 213 |
+
assistant session. The UI may expose Copy session id / Reset conversation for
|
| 214 |
+
advanced use.
|
| 215 |
+
|
| 216 |
+
## Edit-safety rules
|
| 217 |
+
|
| 218 |
+
Every request records the active scene id and a revision hash of the source
|
| 219 |
+
text. When a result arrives:
|
| 220 |
+
|
| 221 |
+
- if both still match, enable Apply;
|
| 222 |
+
- if the user changed scenes or edited the source, keep the result staged but
|
| 223 |
+
show "Source changed while the assistant was working";
|
| 224 |
+
- never apply a late result to whichever scene happens to be active;
|
| 225 |
+
- keep the user's current text available as a one-step local undo after Apply;
|
| 226 |
+
- discard or cancel outstanding requests when the node is removed;
|
| 227 |
+
- cap transcript persistence and do not serialize auth data or bridge tokens in
|
| 228 |
+
the workflow.
|
| 229 |
+
|
| 230 |
+
Prompt text is untrusted data. Put it in a clearly delimited data field and keep
|
| 231 |
+
the rewrite instruction outside that field. The structured result schema is the
|
| 232 |
+
authority for what may be applied.
|
| 233 |
+
|
| 234 |
+
## Implementation phases
|
| 235 |
+
|
| 236 |
+
### Phase 0: one-shot UX proof (superseded)
|
| 237 |
+
|
| 238 |
+
The original proposal was a manually started local development sidecar outside
|
| 239 |
+
ComfyUI's Python process. The native bridge was implemented directly instead,
|
| 240 |
+
so no separate sidecar is required.
|
| 241 |
+
|
| 242 |
+
- Codex: `codex exec --ephemeral --sandbox read-only` with structured output.
|
| 243 |
+
- Hermes: `hermes --oneshot` with tools disabled and a structured-output
|
| 244 |
+
contract where supported.
|
| 245 |
+
- Node: provider picker, instruction, staged result, conflict detection, Apply.
|
| 246 |
+
|
| 247 |
+
This proves prompt quality and editor ergonomics. It should be marked a local
|
| 248 |
+
development mode, not the shipping architecture.
|
| 249 |
+
|
| 250 |
+
### Phase 1: native comfyui-mcp capability (implemented for Codex and initial Hermes)
|
| 251 |
+
|
| 252 |
+
- add the prompt-assist frame family and dedicated session manager;
|
| 253 |
+
- reuse the current Codex backend/app-server lifecycle;
|
| 254 |
+
- use Hermes' tool-restricted one-shot adapter initially; ACP remains the
|
| 255 |
+
persistent-session optimization;
|
| 256 |
+
- expose provider readiness specifically for prompt assistance;
|
| 257 |
+
- support progress state, cancellation, reset, and structured final output;
|
| 258 |
+
- keep the existing panel protocol and shared panel sessions unchanged.
|
| 259 |
+
|
| 260 |
+
### Phase 2: production editor UI (initial implementation complete)
|
| 261 |
+
|
| 262 |
+
- implement the assistant panel in `web/h3_chain_scene_prompt_editor.js`;
|
| 263 |
+
- factor transport and revision helpers into testable `.mjs` modules;
|
| 264 |
+
- keep a transient node transcript plus bounded orchestrator transcript state;
|
| 265 |
+
neither is serialized into the workflow;
|
| 266 |
+
- include selected text as special context and provide an editable staged
|
| 267 |
+
replacement with original/proposal comparison;
|
| 268 |
+
- document startup and unavailable-provider states.
|
| 269 |
+
|
| 270 |
+
## Test plan
|
| 271 |
+
|
| 272 |
+
Current frontend tests cover bounded context construction, source revisions,
|
| 273 |
+
stale-draft detection, bridge discovery, deferred conversation reset, Plan
|
| 274 |
+
round-trips, and the editor's staged controls. Current orchestrator tests cover
|
| 275 |
+
request validation, output parsing, correlated result/cancellation, transcript
|
| 276 |
+
reset and provider separation, Codex isolation settings, auxiliary socket
|
| 277 |
+
routing, and existing conversation-boundary behavior.
|
| 278 |
+
|
| 279 |
+
A later browser-level automation pass should additionally cover:
|
| 280 |
+
|
| 281 |
+
- request context for first, middle, and last scenes;
|
| 282 |
+
- multiline prompt round trips;
|
| 283 |
+
- request/result correlation with out-of-order replies;
|
| 284 |
+
- navigation and source edits while a request is in flight;
|
| 285 |
+
- Apply, Discard, and one-step undo through real DOM events;
|
| 286 |
+
- cancellation and node removal;
|
| 287 |
+
- bridge loss/reconnect without duplicate Apply;
|
| 288 |
+
- no bridge token, credential, or transcript leakage into `plan_json`.
|
| 289 |
+
|
| 290 |
+
Orchestrator tests should cover:
|
| 291 |
+
|
| 292 |
+
- assistant sessions never resolve to the panel's shared session key;
|
| 293 |
+
- no panel/graph tools are registered for assistant turns;
|
| 294 |
+
- provider switch and per-provider resume;
|
| 295 |
+
- schema validation and size/time limits;
|
| 296 |
+
- cancellation before and during generation;
|
| 297 |
+
- Codex and Hermes adapters produce the same canonical result frames;
|
| 298 |
+
- a malicious prompt cannot change the requested response shape or tool policy.
|
| 299 |
+
|
| 300 |
+
A browser smoke test should edit a real Plan prompt, request a rewrite, verify
|
| 301 |
+
that agent progress does not change `plan_json`, Apply the draft, and confirm
|
| 302 |
+
both the large editor and the Plan editor show the same saved text.
|
| 303 |
+
|
| 304 |
+
## Decision
|
| 305 |
+
|
| 306 |
+
Proceed with Phase 0 only as a short-lived UX experiment. The production target
|
| 307 |
+
should be Phase 1 plus Phase 2: a provider-neutral prompt-assist capability in
|
| 308 |
+
`comfyui-mcp` and a staged assistant UI in this editor.
|
| 309 |
+
|
| 310 |
+
This gives Codex and Hermes one consistent node experience, keeps user
|
| 311 |
+
authentication and agent processes out of ComfyUI, preserves Registry-friendly
|
| 312 |
+
separation, avoids interference with the sidebar agent, and leaves room for the
|
| 313 |
+
same assistant transport to serve other prompt-editor nodes later.
|
| 314 |
+
|
| 315 |
+
## Evidence reviewed
|
| 316 |
+
|
| 317 |
+
- this pack's `chain_nodes.py`, `web/h3_chain_scene_prompt_editor.js`, and
|
| 318 |
+
`web/h3_chain_plan_core.mjs`;
|
| 319 |
+
- `/media/p5/comfyui-mcp/src/orchestrator/agent-backend.ts`, `index.ts`,
|
| 320 |
+
`codex-backend.ts`, and `src/services/agent-setup.ts`;
|
| 321 |
+
- `/media/p5/comfyui-mcp-panel/__init__.py` and
|
| 322 |
+
`web/js/comfyui-mcp-panel.js`;
|
| 323 |
+
- the locally installed `codex exec`, `hermes`, `hermes acp`, and
|
| 324 |
+
`hermes serve` command contracts;
|
| 325 |
+
- OpenAI's current [Codex App Server](https://learn.chatgpt.com/docs/app-server.md),
|
| 326 |
+
[Codex SDK](https://learn.chatgpt.com/docs/codex-sdk.md), and
|
| 327 |
+
[non-interactive mode](https://learn.chatgpt.com/docs/non-interactive-mode.md)
|
| 328 |
+
documentation.
|
ComfyUI-MiniMaxH3-Context-Loop/CHANGELOG.md
ADDED
|
@@ -0,0 +1,1347 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Changelog
|
| 2 |
+
|
| 3 |
+
Newest first. The [README](README.md#changelog) keeps a short highlight reel;
|
| 4 |
+
this file records the detailed changes.
|
| 5 |
+
|
| 6 |
+
## v0.6.5 — Explicit upscale anchor settings and source summaries
|
| 7 |
+
|
| 8 |
+
- Added semantic-anchor size and presentation-mode overrides to latent, pixel,
|
| 9 |
+
and inherited VIDEO upscale conditioning. Choices match Tagged Scene Options,
|
| 10 |
+
with `inherit` preserving saved settings or the connected anchor bundle.
|
| 11 |
+
- Overrides apply to cached, rebuilt, and connected references. Changed anchor
|
| 12 |
+
settings rebuild presentation and prompt labels from verified saved media in
|
| 13 |
+
a separate reusable cache without modifying the original checkpoint/cache.
|
| 14 |
+
- Checkpoint Manager now names the output branch, available scenes, Original or
|
| 15 |
+
DeRoPE source, exact Original fallback scenes, and workflow pin state. Preview
|
| 16 |
+
browsing stays separate from the source; downstream range controls still
|
| 17 |
+
determine which scenes are processed.
|
| 18 |
+
- Preserved existing positional pixel/VIDEO widget values. Added CPU recovery,
|
| 19 |
+
cache reuse, source-preservation, schema and frontend output-summary tests.
|
| 20 |
+
|
| 21 |
+
## v0.6.4 — Preserve reference sizing during recovery
|
| 22 |
+
|
| 23 |
+
- Recover saved Match/Max sizing through static input links and modern Tagged
|
| 24 |
+
Scene Options, including when the reference tensors have been lost.
|
| 25 |
+
- Apply upscale sizing overrides to cached and reconstructed references, not
|
| 26 |
+
only connected Tagged overrides. Match-to-Max changes re-encode original
|
| 27 |
+
picture masters; legacy caches recover those masters from saved media.
|
| 28 |
+
- Preserve source caches and unrelated reference roles; report the actual
|
| 29 |
+
effective policy and rebuild count. Added CPU policy/geometry regressions.
|
| 30 |
+
|
| 31 |
+
## v0.6.3 — Cancellation-safe processing saves
|
| 32 |
+
|
| 33 |
+
- DeRoPE/upscale publication retains media after uncertain checkpoint commits;
|
| 34 |
+
a failed manifest refresh no longer deletes a successfully saved scene.
|
| 35 |
+
- VIDEO PNG exports journal staged publication and recover interrupted frame
|
| 36 |
+
copies or index commits on retry. Earlier scenes remain untouched; conflicting
|
| 37 |
+
incomplete files are preserved in the recovery staging folder.
|
| 38 |
+
- Recreated VIDEO containers reuse existing PNGs when their delivered pixels
|
| 39 |
+
match at the selected precision, including legacy exports. Changed renders,
|
| 40 |
+
branches, and edited committed PNGs are still protected from silent adoption.
|
| 41 |
+
- Added CPU cancellation/fault-injection and real process-exit regression tests.
|
| 42 |
+
See [processing cancellation and resume](docs/processing-resume.md).
|
| 43 |
+
|
| 44 |
+
## v0.6.2 — Upscale reference recovery and durable exports
|
| 45 |
+
|
| 46 |
+
- Backported reference-cache deduplication, legacy conversion, reconstruction,
|
| 47 |
+
and upscale saving fixes from nightly without adding workflow-ownership
|
| 48 |
+
locking or unrelated nightly review features.
|
| 49 |
+
- Missing cache manifests or tensors can be rebuilt from verified saved media
|
| 50 |
+
and reference identities, including semantic anchors, saved presentation
|
| 51 |
+
settings, and timed references when their immutable timing snapshot exists.
|
| 52 |
+
Recovery never executes an archived workflow, resolves an old tag against
|
| 53 |
+
today's replacement asset, or silently drops references to text-only.
|
| 54 |
+
- New caches store content-addressed per-reference safetensors. Existing bundles
|
| 55 |
+
remain readable; `tools/convert_reference_caches.py` provides dry-run/apply
|
| 56 |
+
conversion. Legacy bundles are removed only after their converted tensors
|
| 57 |
+
have been verified and used by a successfully saved render.
|
| 58 |
+
- Each new generation or alternate take keeps an immutable Plan/API/workflow
|
| 59 |
+
recovery snapshot. Failed scene publication removes only its uncommitted
|
| 60 |
+
artifacts; failed advisory root/editorial refreshes retain the committed take.
|
| 61 |
+
Historical and attributed checkpoint selections use their own recovery Plan,
|
| 62 |
+
with root-archive compatibility for older saves.
|
| 63 |
+
- Added Original/DeRoPE/Latent Upscale/Pixel Upscale checkpoint views, saved
|
| 64 |
+
DeRoPE source selection with original fallback, explicit processing ranges,
|
| 65 |
+
and final-cut alternate selection across full-sequence deferred upscales.
|
| 66 |
+
- Added preview-confirmed processed-take deletion and chapter-snapshot retirement.
|
| 67 |
+
Later independent pixel takes remain intact; actual source/context dependencies,
|
| 68 |
+
original checkpoints, shared references, and unrelated exports stay protected.
|
| 69 |
+
- PNG export now accepts file-backed VIDEO plus upscale state, publishes one
|
| 70 |
+
scene at a time, and passes the same VIDEO downstream only after publication.
|
| 71 |
+
Existing output slots remain unchanged; VIDEO is appended. Users choose 8- or
|
| 72 |
+
16-bit RGB PNGs. Frames are decoded incrementally with bounded worker queues;
|
| 73 |
+
no full-chain pixel batch or VAE is required in VIDEO mode.
|
| 74 |
+
- PNG publication falls back to exclusive buffered copies when network storage
|
| 75 |
+
denies hard links. Timestamp drift triggers content-hash verification instead
|
| 76 |
+
of rejection. Changed/missing PNG contents still prevent reuse without overwriting
|
| 77 |
+
the user's files; timestamp tolerance is not permission to adopt edited frames.
|
| 78 |
+
|
| 79 |
+
- Grouped full-mix/vocal/instrumental sources in the Carousel and a reusable
|
| 80 |
+
Audio Tracks node. Full mixes are never doubled with stems; stems-only mixes
|
| 81 |
+
are frame-aligned, peak-protected and previewable. Audio archives include
|
| 82 |
+
every track and validate their recovery hashes.
|
| 83 |
+
- Per-scene Lip-sync Inherit/On/Off in Plan Studio and the Plan editor, using
|
| 84 |
+
the established source-target/reference/carry fields. Grouped lip-sync uses
|
| 85 |
+
vocals only; delivery remains the full mix. Existing single-track workflows
|
| 86 |
+
are unchanged. Missing vocals and mismatched stem durations fail clearly.
|
| 87 |
+
- Chapter PNG/WAV exports can reuse verified unchanged PNGs and append newly
|
| 88 |
+
generated scenes in the same folder. Identical exports skip VAE decoding;
|
| 89 |
+
growing soundtracks are rebuilt atomically to preserve audio joins. Changed
|
| 90 |
+
takes, trims, placements, settings, damaged/missing files, and interrupted or
|
| 91 |
+
unverifiable legacy exports use a new folder without replacing prior frames.
|
| 92 |
+
A fresh-export switch, per-export process lock, file integrity records, and
|
| 93 |
+
export-index history preserve explicit recovery and whole-Run behaviour.
|
| 94 |
+
|
| 95 |
+
## v0.6.1 — Chapter output and authoring polish
|
| 96 |
+
|
| 97 |
+
- Ported the standalone Prompt IDE's marker editing, context-aware completions,
|
| 98 |
+
timestamps, and presentation settings while retaining connected Plan editing,
|
| 99 |
+
reference discovery, prompt revisions, and optimizer support. No task-aware
|
| 100 |
+
template panel was added.
|
| 101 |
+
- Added chapter width/height controls in Plan Studio. Locked saved scenes pin
|
| 102 |
+
their chapter's original size; incompatible cross-resolution video context
|
| 103 |
+
still requires an independent boundary. Export differently sized chapters
|
| 104 |
+
separately rather than silently resizing saved media.
|
| 105 |
+
- Added immutable Chapter Delivery and Chapter Recovery manifests, including
|
| 106 |
+
export of generated scenes from unfinished chapters. Saved chapter snapshots
|
| 107 |
+
retain their editorial state, audio timing, and exact checkpoint lineage.
|
| 108 |
+
Checkpoint deletion protects artifacts referenced by these snapshots.
|
| 109 |
+
PNG exports retain main's separate numbered export folders.
|
| 110 |
+
- Added workflow-local checkpoint output and **Selected chapter only** scope.
|
| 111 |
+
Pins do not change project-wide active branches or the connected Plan.
|
| 112 |
+
Local/chapter selection is read-only, including legacy checkpoint scanning;
|
| 113 |
+
a missing pinned revision is rejected rather than silently recreated.
|
| 114 |
+
- Added chapter-scoped deferred upscale runs with original scene numbering,
|
| 115 |
+
source-audio offsets, isolated profiles, and compatibility validation limited
|
| 116 |
+
to the selected chapter's media. Improved saved context-independence evidence,
|
| 117 |
+
chapter-boundary validation, and compatibility fingerprint handling.
|
| 118 |
+
- Added optional pixel-upscale conditioning width/height (`0 = auto`) to the
|
| 119 |
+
nodes and example recipe. Overrides affect conditioning and reference size,
|
| 120 |
+
not the actual image batch dimensions.
|
| 121 |
+
- Fixed narrow Plan Studio controls overlapping, improved light-theme contrast,
|
| 122 |
+
and refreshed shared helper cache keys consistently.
|
| 123 |
+
- Fixed scene duplication rewriting context links and preserving the copied
|
| 124 |
+
prompt instead of inheriting the following scene's prompt.
|
| 125 |
+
|
| 126 |
+
Workflow-ownership locking remains nightly-only; these ports do not add it to main.
|
| 127 |
+
|
| 128 |
+
## v0.6.0 — Production context authoring and synchronized export
|
| 129 |
+
|
| 130 |
+
### Release preparation
|
| 131 |
+
|
| 132 |
+
- Rebuilt the 0.6 example catalog from named recipes and this release's node
|
| 133 |
+
definitions, rather than reserializing archived workflows. The 19 examples
|
| 134 |
+
have typed widget validation, required-input and bidirectional-link checks,
|
| 135 |
+
title-aware layout checks, and companion setup guides. Studio Plan outputs
|
| 136 |
+
are connected to Loop Start; the Carousel and Checkpoint Manager are included.
|
| 137 |
+
Older examples remain in `example_workflows/Archive/0.5/`.
|
| 138 |
+
|
| 139 |
+
- Corrected release-workflow execution settings and switched maintained source
|
| 140 |
+
video examples to native VIDEO inputs. Bounded smoke renders exercised the
|
| 141 |
+
15 non-upscale examples; the test setup and its Ref2VA model-substitution
|
| 142 |
+
limits are recorded in [Workflow validation](tools/v06/README.md#layout-and-release-checks).
|
| 143 |
+
These checks are not a full-resolution quality or GPU-memory guarantee.
|
| 144 |
+
|
| 145 |
+
- Added an experimental pixel-upscale backend and DLSS5 + USDU example, adapted
|
| 146 |
+
from **Illynir's** contribution. Pixel Current Scene decodes saved video
|
| 147 |
+
checkpoints one scene at a time; Pixel Conditioning uses the actual upscaled
|
| 148 |
+
image dimensions, rebuilds matching references at that size, and passes the
|
| 149 |
+
same images to the refiner without applying a second upscale. Image-only
|
| 150 |
+
segment saving preserves original audio and supports resume and assembly.
|
| 151 |
+
Pixel mode does not require or splice HQ latent continuity. CPU integration
|
| 152 |
+
tests cover geometry, audio, trimming, recursion, and resume; external
|
| 153 |
+
DLSS5/USDU GPU refinement remains untested. See the
|
| 154 |
+
[experimental workflow guide](<example_workflows/guides/Deferred Upscale - Pixel DLSS5 + USDU - EXPERIMENTAL - MiniMax H3 0.6.md>).
|
| 155 |
+
|
| 156 |
+
- Added visible Carousel semantic-anchor size and mode controls, with an
|
| 157 |
+
**inherit** choice that delegates to downstream Tagged Scene Options without
|
| 158 |
+
requiring another Options wire into the Carousel. Older explicit values
|
| 159 |
+
remain readable. The controls can be hidden without changing their values.
|
| 160 |
+
|
| 161 |
+
- Hardened asynchronous project switching, run restoration, and editorial
|
| 162 |
+
saving. Responses from an old run or detached workflow no longer update the
|
| 163 |
+
newly selected graph; saves serialize with revision checks so late responses
|
| 164 |
+
cannot overwrite newer edits. Switching or closing a project flushes pending
|
| 165 |
+
editorial changes. Unused stale Studio scene entries can be reconciled,
|
| 166 |
+
while entries carrying saved work remain protected.
|
| 167 |
+
|
| 168 |
+
- Fixed source-audio preview and waveform mapping after scene-length changes,
|
| 169 |
+
including clearing the previous run's source preview on project switches.
|
| 170 |
+
Empty-scene navigation and seeking now preserve the selected scene and
|
| 171 |
+
playhead instead of jumping to a rendered or prompted scene.
|
| 172 |
+
|
| 173 |
+
- Clear a selected final-cut alternate when its base scene is regenerated,
|
| 174 |
+
retaining the saved alternate revision without applying it to the wrong
|
| 175 |
+
generation lineage. Connected Review Gate previews now replace the redundant
|
| 176 |
+
silent Segment + Checkpoint preview.
|
| 177 |
+
|
| 178 |
+
- Restored the concise README changelog, corrected the release banner from
|
| 179 |
+
NIGHTLY to 0.6, and updated the installation URL to the renamed Context-Loop
|
| 180 |
+
repository. Existing custom-node folder names and serialized node IDs do not
|
| 181 |
+
need to change.
|
| 182 |
+
|
| 183 |
+
### Features and fixes
|
| 184 |
+
|
| 185 |
+
- Fixed Windows mapped network drives used as ComfyUI input/output directories
|
| 186 |
+
(#45). Root and artifact paths now resolve consistently before calculating
|
| 187 |
+
relative paths, preventing drive-letter/UNC mount errors during segment and
|
| 188 |
+
recovery-archive saving, Run Manager access, and Asset Carousel imports.
|
| 189 |
+
|
| 190 |
+
- Refreshed browser cache keys for Checkpoint Manager, prompt editors, Plan
|
| 191 |
+
Studio, and Asset Carousel helpers. Added checks to prevent cache keys from
|
| 192 |
+
being downgraded to stale versions during future updates.
|
| 193 |
+
|
| 194 |
+
- Fixed Prompt Editor reference discovery when the Studio render engine lives
|
| 195 |
+
inside a subgraph. Editors now read the connected upstream Project Asset
|
| 196 |
+
Carousel directly instead of requiring a traversable downstream Ref2VA
|
| 197 |
+
wrapper.
|
| 198 |
+
|
| 199 |
+
- Fixed Project Asset Carousel Run-name editing so pauses after a space,
|
| 200 |
+
underscore, or hyphen no longer let an asynchronous catalog refresh erase
|
| 201 |
+
the separator. The field now preserves the exact draft while typing and
|
| 202 |
+
commits its folder-safe project name on Enter or blur.
|
| 203 |
+
|
| 204 |
+
- Fixed Checkpoint Manager hiding the safe rollback operation for an earlier
|
| 205 |
+
checkpoint that was already on the active branch. It now offers **Roll
|
| 206 |
+
active branch back**, retires later active pointers in one operation, and
|
| 207 |
+
keeps all immutable revisions. The deletion inspector now distinguishes
|
| 208 |
+
this non-destructive rollback from leaf-by-leaf permanent deletion.
|
| 209 |
+
|
| 210 |
+
- Hardened Scene Prompt Editor and Rich Scene Prompt Editor workflow teardown.
|
| 211 |
+
Closing a workflow now cancels pending Plan propagation, prompt analysis,
|
| 212 |
+
history saves, timers, and listeners without repainting or mutating a graph
|
| 213 |
+
while ComfyUI is removing it. Cleanup failures are isolated so one editor
|
| 214 |
+
cannot prevent the remaining workflow nodes from closing.
|
| 215 |
+
|
| 216 |
+
- Replaced the Project Asset Carousel's filtered browser suggestion popup with
|
| 217 |
+
a real project picker attached to the editable Run name field. The arrow now
|
| 218 |
+
always lists every existing project with its asset and unassigned counts,
|
| 219 |
+
while the text field still accepts a new Run name.
|
| 220 |
+
|
| 221 |
+
- Fixed Review Gate delivery when Project Assets owns the Plan run name. The
|
| 222 |
+
hidden, stale Plan widget no longer rejects the exact pending review or
|
| 223 |
+
participates in run-name fallback matching. Report and original fix by
|
| 224 |
+
**Psylent_Gamer (4090)** via Banodoco, adapted to the current Plan/Modern Plan
|
| 225 |
+
routing implementation.
|
| 226 |
+
|
| 227 |
+
- Fixed path-backed video and motion references in `restart_each_scene` mode
|
| 228 |
+
being cropped or padded to the generated scene length. They now retain
|
| 229 |
+
their own complete 24 fps reference duration; only `sequential` references
|
| 230 |
+
use scene-sized timeline windows.
|
| 231 |
+
|
| 232 |
+
- Extended **Export PNG Sequence** into an independent picture/audio exporter.
|
| 233 |
+
Its video and audio VAE sockets are now both optional: connect either one for
|
| 234 |
+
PNG-only or WAV-only output, or both for a continuous PNG sequence plus a
|
| 235 |
+
synchronized `audio.wav`. The WAV is decoded from the saved H3 audio latents,
|
| 236 |
+
preserves AV overlap ownership, and follows the same selected scene order and
|
| 237 |
+
latent-safe trims as the exported frames. Existing video-VAE connections and
|
| 238 |
+
output indexes remain valid; `audio_path` is appended as a new output.
|
| 239 |
+
|
| 240 |
+
- Accelerated **Export PNG Sequence** with chunked batch conversion and up to
|
| 241 |
+
eight bounded parallel atomic PNG writers, inspired by JSON-Dynamic's Fast
|
| 242 |
+
Absolute Saver. New nodes default to lossless compression level 1; existing
|
| 243 |
+
saved compression values remain unchanged. The exporter now shows ComfyUI
|
| 244 |
+
progress, logs per-scene verify/load/GPU-decode/convert/save timings, updates
|
| 245 |
+
`export.partial.json` after every chunk, honors cancellation between chunks,
|
| 246 |
+
and caches successful checkpoint hashes by immutable hash, size, and mtime.
|
| 247 |
+
Strict verification remains available as an optional input.
|
| 248 |
+
|
| 249 |
+
- Corrected the user-visible **Contex Loop** typo to **Context Loop** across
|
| 250 |
+
node titles, Add Node categories, settings, documentation, branding, and
|
| 251 |
+
bundled workflow titles. Serialized node IDs and settings keys remain
|
| 252 |
+
unchanged for compatibility.
|
| 253 |
+
|
| 254 |
+
- Fixed Review candidate acceptance and rerolls leaving Plan Studio and
|
| 255 |
+
prompt-editor companions on stale scene metadata. Selecting a 10x candidate
|
| 256 |
+
now refreshes the complete Plan—including seed, length, steps, context, and
|
| 257 |
+
scene order—while prompt-only edits retain their selection- and undo-safe
|
| 258 |
+
fast path. The shared companion module now has an explicit cache revision,
|
| 259 |
+
with safe fallbacks for partially refreshed browser sessions.
|
| 260 |
+
|
| 261 |
+
- Added the multi-block **Picture Context Builder** to Plan Studio. Scene Context
|
| 262 |
+
now owns picture/audio totals, boundary implementation, and spatial proxy;
|
| 263 |
+
selecting 1–N picture blocks exposes every H3-native repartition through
|
| 264 |
+
ordered boundary selectors plus an independent earlier-scene source and
|
| 265 |
+
latent window for every block. Repeated sources are supported, audio stays
|
| 266 |
+
locked to its continuous predecessor path by default, and checkpoint,
|
| 267 |
+
resume, trim, review, and dependency metadata track every selected block.
|
| 268 |
+
Released one/two-block Plan fields remain a read-only compatibility path.
|
| 269 |
+
|
| 270 |
+
- Added **MiniMax H3 Plan (Modern)** as a clean, opt-in replacement for the
|
| 271 |
+
original Plan node. It preserves the familiar vertical scene-column editor
|
| 272 |
+
and exact Plan output sockets, but organizes its controls into Project,
|
| 273 |
+
Canvas, Generation defaults, and Delivery while requiring Generation Profile
|
| 274 |
+
and exposing no legacy context/audio/continuation fallbacks. The original
|
| 275 |
+
Plan remains unchanged; its **Upgrade to Modern Plan…** action preserves
|
| 276 |
+
scenes, supported settings, placement, and graph connections. Plan Studio,
|
| 277 |
+
prompt editors, Review, reroll, Run/Checkpoint Managers, project assets, and
|
| 278 |
+
recovery archives recognize both Plan types.
|
| 279 |
+
|
| 280 |
+
- Fixed zero-picture-context resume validation. When a visually new scene
|
| 281 |
+
carries only the preceding generated audio, strict history checks now ignore
|
| 282 |
+
that predecessor's unrelated incoming visual-boundary recipe while retaining
|
| 283 |
+
prompt/model/source identity and artifact-integrity checks. Resume errors now
|
| 284 |
+
report the consumed streams and selected scene's effective video/audio
|
| 285 |
+
context lengths.
|
| 286 |
+
|
| 287 |
+
- Hardened the Manager-facing HTTP surface. Direct prompt optimization now
|
| 288 |
+
uses a server-owned exact-origin allow-list and validates every redirect;
|
| 289 |
+
custom or local providers require `H3_PROMPT_OPTIMIZER_ALLOWED_ORIGINS`.
|
| 290 |
+
Carousel imports can read only enumerated ComfyUI input, project, or chain
|
| 291 |
+
media; Plan Studio browser media is confined to configured ComfyUI roots
|
| 292 |
+
with real-path/symlink checks; and host folder opening is limited to
|
| 293 |
+
loopback clients.
|
| 294 |
+
|
| 295 |
+
- Composed visual context now supports two independently positioned native
|
| 296 |
+
latent windows from the same saved scene. A split such as 5+34 can reuse one
|
| 297 |
+
source movie for both blocks, including on Scene 2, while generated-audio
|
| 298 |
+
continuity remains attached to the immediate timeline predecessor.
|
| 299 |
+
|
| 300 |
+
- Completed the post-Issue-42 payload audit. Delegated Motion Context now
|
| 301 |
+
verifies the live video/audio Guide merge before calling every upstream
|
| 302 |
+
provider version, including older providers that do not fail preflight.
|
| 303 |
+
The behavioral probe now uses real tiny tensors, and release cache,
|
| 304 |
+
workflow, and regression fixtures are synchronized.
|
| 305 |
+
|
| 306 |
+
- Fixed Issue #42 on partially updated ComfyUI builds and internal Motion
|
| 307 |
+
Context fallback paths. Chain now behaviorally verifies both video and
|
| 308 |
+
audio keyframe/reference payload merging before sampling and installs its
|
| 309 |
+
marker-gated compatibility merge only when native Guide layout exists but
|
| 310 |
+
the payload still drops Guide tensors. This covers the reported 37-step
|
| 311 |
+
stereo mismatch (`714` reserved rows versus `640` supplied rows), Patch
|
| 312 |
+
Priority, longer/audio-only carry, and other internal-engine routes.
|
| 313 |
+
|
| 314 |
+
- Made prompt reference tags case-sensitive from display through conversion.
|
| 315 |
+
The rich editor no longer presents a wrong-case alias as a valid asset, so
|
| 316 |
+
`@Maison` and `@maison` remain distinct. Completion search is still
|
| 317 |
+
case-insensitive and inserts the asset's canonical spelling.
|
| 318 |
+
|
| 319 |
+
- Fixed a Loop Start runtime crash introduced by structured source-duration
|
| 320 |
+
diagnostics. Source Timeline binding now derives the active audio-source
|
| 321 |
+
requirements before calculating required frames, matching the preflight
|
| 322 |
+
path and eliminating the `source_requirements` NameError.
|
| 323 |
+
|
| 324 |
+
- Completed the structured preflight diagnostic contract. All hard preflight
|
| 325 |
+
categories now carry their precise trigger, including scene ID, setting,
|
| 326 |
+
value, origin, and tag where applicable; duration/capacity failures report
|
| 327 |
+
required versus available frames, the last complete scene, and the first
|
| 328 |
+
affected scene. Diagnostics retain the compatible primary `action` while
|
| 329 |
+
adding ranked alternative solutions, and validation failures are separated
|
| 330 |
+
by Plan, scene selection, source, reference, runtime, and resume stage rather
|
| 331 |
+
than collapsing into one generic internal error. Source-duration validation
|
| 332 |
+
now measures only through the last scene that actually consumes source audio
|
| 333 |
+
unless the final soundtrack itself is sourced.
|
| 334 |
+
|
| 335 |
+
- Made blocking preflight errors identify their actionable origin. Missing
|
| 336 |
+
source-audio diagnostics now list every triggering scene ID, the exact
|
| 337 |
+
effective setting (`source_reference`, `source_audio_target`, or final
|
| 338 |
+
output audio), and whether it came from a scene override or Chain Policy;
|
| 339 |
+
they also name the disconnected Loop Start sockets. Other structured
|
| 340 |
+
preflight errors now render their available scene, setting, origin, and tag
|
| 341 |
+
context on separate readable lines before the corrective action.
|
| 342 |
+
|
| 343 |
+
- Fixed Issue #38's remaining Scene 2 failure on installations where
|
| 344 |
+
ComfyUI retains a compatible H3 payload wrapper after its temporary module
|
| 345 |
+
alias is removed. Patch Priority now recovers that wrapper's captured stock
|
| 346 |
+
method from the function's own execution globals, allowing the guarded
|
| 347 |
+
Guide-audio plus Ref2VA-audio merge to take ownership without requiring the
|
| 348 |
+
user to remove a valid node pack. Unknown payload wrappers remain refused.
|
| 349 |
+
|
| 350 |
+
- Fixed a selected final-cut ALT incorrectly retargeting later queues back to
|
| 351 |
+
its original scene. The server-side editorial record now owns whether an
|
| 352 |
+
alternate generation draft is armed; the hidden Plan Studio widget may only
|
| 353 |
+
update an already-armed matching draft and cannot resurrect one cleared by
|
| 354 |
+
Review. Selecting Original/ALT explicitly disarms that scene's draft, and
|
| 355 |
+
Plan Studio synchronizes the one-shot queue widget on every workflow load,
|
| 356 |
+
including Plans without chapters.
|
| 357 |
+
|
| 358 |
+
- Fixed delayed and inconsistent workflow hydration. Project Asset Carousel
|
| 359 |
+
now paints its serialized catalog immediately while refreshing in the
|
| 360 |
+
background, and Plan Studio keeps a lightweight serialized presentation
|
| 361 |
+
cache so saved thumbnails, placements, and latent-safe trims appear on its
|
| 362 |
+
first render instead of snapping into place after the checkpoint scan.
|
| 363 |
+
Player transport now advances at the trimmed endpoint instead of continuing
|
| 364 |
+
through discarded media. Scene renaming is atomic, rejects duplicate IDs,
|
| 365 |
+
and migrates chapters, visual-context links, placements, trims, locks, and
|
| 366 |
+
alternate final-cut metadata so an ungenerated rename cannot hide a scene.
|
| 367 |
+
|
| 368 |
+
- Plan Studio can now create a prompt-word alternate for an already accepted
|
| 369 |
+
scene. Queueing an enabled draft renders only that scene and saves an
|
| 370 |
+
immutable `editorial_alternate` revision without changing the active
|
| 371 |
+
generation checkpoint. Accepting it selects its picture for preview,
|
| 372 |
+
assembly, PNG export, and whole-chain latent finishing, while later scenes
|
| 373 |
+
continue to depend on the original take and final audio remains original.
|
| 374 |
+
The timeline marks selected clips `ALT`; Checkpoint Manager nests alternates
|
| 375 |
+
under their immutable base and distinguishes generation lineage from the
|
| 376 |
+
final-cut choice. Selected base/alternate revisions are deletion-protected.
|
| 377 |
+
|
| 378 |
+
- Plan Studio can now shorten a rendered scene non-destructively at a
|
| 379 |
+
latent-safe endpoint. Its right edge and Scene panel expose only cuts shared
|
| 380 |
+
by H3's native video-latent cycle and the delivered 24 fps / 40 Hz audio
|
| 381 |
+
clock; the original complete checkpoint and scene movie remain untouched.
|
| 382 |
+
Continuation state, generated audio, standard assembly, PNG export, and the
|
| 383 |
+
Full-Chain Latent Video adapter use the shortened prefix. If an earlier cut
|
| 384 |
+
changes after dependent scenes were generated, those checkpoints are marked
|
| 385 |
+
stale and resume requires regeneration from the first affected scene instead
|
| 386 |
+
of silently mixing incompatible endpoints.
|
| 387 |
+
|
| 388 |
+
- Fixed scene-2 continuation with simultaneous Ref2VA/source audio. On a
|
| 389 |
+
partially native ComfyUI runtime that drops Guide audio when reference audio
|
| 390 |
+
is also present, Chain now activates its marker-gated payload merge and uses
|
| 391 |
+
the retained internal Motion Context engine. The compatibility merge now
|
| 392 |
+
preserves keyframe audio followed by reference audio, matching H3 layout
|
| 393 |
+
order, instead of accidentally retaining only the reference-audio rows.
|
| 394 |
+
|
| 395 |
+
- Scene LoRA routing now grows with the workflow instead of exposing four
|
| 396 |
+
fixed branches. A new scheduler shows one empty LoRA A socket; connecting it
|
| 397 |
+
reveals B, then C, through Z. Plan Editor and Plan Studio offer only routes
|
| 398 |
+
connected to their scheduler plus a scene's already-saved route. Existing
|
| 399 |
+
A-D plans and scheduler links remain compatible, and every route stays lazy.
|
| 400 |
+
|
| 401 |
+
- Run name now suggests existing live Asset Carousel projects. Choosing a
|
| 402 |
+
suggestion switches the Carousel and its connected Plan, while the same
|
| 403 |
+
field still accepts a new Run name and remains the only serialized source of
|
| 404 |
+
truth. Empty projects are included in the history; output-only H3 backups
|
| 405 |
+
remain available through Import instead of appearing as switch targets.
|
| 406 |
+
|
| 407 |
+
- Project Asset Carousel can duplicate its complete asset project under a new
|
| 408 |
+
Run name. The copy keeps asset order, folders, roles, tags, lyrics, options,
|
| 409 |
+
reference slots, and independent project-owned media files plus their asset
|
| 410 |
+
recovery mirror; generated clips, checkpoints, assembled renders, previews,
|
| 411 |
+
and upload scratch files are deliberately excluded. Existing Run names are
|
| 412 |
+
never overwritten, and the source Carousel remains selected.
|
| 413 |
+
|
| 414 |
+
- Made both scene prompt editors responsive while typing long prompts. Each
|
| 415 |
+
keystroke still updates the connected Plan value immediately, but expensive
|
| 416 |
+
Plan-card rebuilding, canvas redraw, companion broadcasting, and strict
|
| 417 |
+
schema analysis are now coalesced after a short idle interval and flushed on
|
| 418 |
+
blur, navigation, or removal. Consecutive keystrokes also avoid reparsing an
|
| 419 |
+
unchanged full Plan JSON.
|
| 420 |
+
|
| 421 |
+
- Fixed Tagged Audio `source_timeline` identity validation for the shipped
|
| 422 |
+
full-track workflow. Loop Start's route fingerprint and Tagged Audio's
|
| 423 |
+
waveform-content fingerprint are now recognized as two identities of the
|
| 424 |
+
same Source Timeline audio, while genuinely different tracks remain blocked.
|
| 425 |
+
|
| 426 |
+
- Plan Studio Player now uses its preloaded media as a real second playback
|
| 427 |
+
buffer. At adjacent saved-scene boundaries the decoded video and separate
|
| 428 |
+
generated-audio elements are promoted immediately, the previous elements
|
| 429 |
+
become the following preload buffers, and the existing last-frame veil fades
|
| 430 |
+
only while the promoted player starts. This removes the redundant same-URL
|
| 431 |
+
reload pause without changing clips, editorial timing, or final assembly.
|
| 432 |
+
|
| 433 |
+
- Project Asset Carousel can now import from another Run's live Carousel, not
|
| 434 |
+
only from recovery backups. The copy preserves the source role, tag,
|
| 435 |
+
reference options, and audio lyrics, receives an independent project-owned
|
| 436 |
+
media copy and catalog identity, and never modifies the source Run. Selecting
|
| 437 |
+
an Unassigned card binds the imported media to that slot instead.
|
| 438 |
+
|
| 439 |
+
- Fixed Plan Studio transport synchronization. Ruler ticks, scrubbing, scene
|
| 440 |
+
widths, and the red play line now use the same pixels-per-second scale, so
|
| 441 |
+
extending the open timeline or changing zoom cannot skew the playhead.
|
| 442 |
+
Generated-video and Source Track playback also refresh the visible clock and
|
| 443 |
+
line every animation frame; the browser's coarse `timeupdate` event remains
|
| 444 |
+
only as a fallback.
|
| 445 |
+
|
| 446 |
+
- Updated the retained internal Motion Context engine to current upstream
|
| 447 |
+
Guide behavior: arbitrary off-grid visual lengths remain exact, head-mode
|
| 448 |
+
Guides can be placed at an interior `target_start`, unrelated anchors are
|
| 449 |
+
preserved outside that interval, and decoded source audio is end-aligned on
|
| 450 |
+
an exact 40 Hz PCM grid before VAE encoding. Native H3 run lengths still use
|
| 451 |
+
one efficient video Guide. Loop-only latent Guide, audio-only carry,
|
| 452 |
+
longer-audio windows, fractional audio correction, and the
|
| 453 |
+
guarded legacy-core path remain available.
|
| 454 |
+
|
| 455 |
+
- Chain Context now transparently delegates compatible Guide scenes to
|
| 456 |
+
seitanism's installed **H3 Motion Context** node. Existing workflows and
|
| 457 |
+
sockets do not change. The adapter preserves Loop keyframe arbitration,
|
| 458 |
+
Guide markers, and exact latent-audio placement; Latent Guide, audio-only or
|
| 459 |
+
longer-audio continuity, and legacy ComfyUI
|
| 460 |
+
remain on the internal compatibility engine until upstream exposes those
|
| 461 |
+
capabilities.
|
| 462 |
+
|
| 463 |
+
- Fixed the bundled Tagged Source Audio workflow's `source_timeline` path.
|
| 464 |
+
Tagged Audio now receives the same full Load Audio track as Source Timeline,
|
| 465 |
+
derives its aligned current-scene slice internally, and returns the complete
|
| 466 |
+
static picture-and-audio fingerprint to Plan. The example also serializes
|
| 467 |
+
Tagged Ref2VA's current `native_ref2va` widget and `refmod_sources` output so
|
| 468 |
+
its controls no longer appear shifted after node updates.
|
| 469 |
+
|
| 470 |
+
- Checkpoint Manager can now delete an entire selected
|
| 471 |
+
`output/h3_chains/<run>` folder. It previews the file, folder, and byte
|
| 472 |
+
impact, requires both an explicit warning confirmation and the exact Run
|
| 473 |
+
name, then revalidates the folder snapshot immediately before deletion.
|
| 474 |
+
Original assets under `input/h3_projects/<run>` are deliberately kept.
|
| 475 |
+
|
| 476 |
+
- Semantic pictures now accept bare `#tag` as an untimed Qwen-only visual;
|
| 477 |
+
append `[time]` only when explicit approximate scene placement is wanted.
|
| 478 |
+
Prompt completion, reference conversion, rich chips, preflight, native
|
| 479 |
+
Ref2VA hybrid conditioning, and external RefMod hybrid conditioning share
|
| 480 |
+
the same syntax. `@tag` remains the native Ref2VA/RefMod namespace, and
|
| 481 |
+
neither `#` form creates a VAE reference.
|
| 482 |
+
|
| 483 |
+
- Added optional **Lip-Sync Options** for Generation Profile's exact-source
|
| 484 |
+
audio mode. Scene audio can be encoded with discarded real-song pre-roll and
|
| 485 |
+
lookahead instead of artificial hard boundaries. An aligned isolated vocal
|
| 486 |
+
stem can drive a conservative 40 Hz voice gate so vocal regions remain exact
|
| 487 |
+
while instrumental gaps receive a configurable amount of denoising. The
|
| 488 |
+
options and stem fingerprint are stored in generation dependencies; leaving
|
| 489 |
+
the node disconnected preserves the previous hard-cut/full-freeze result.
|
| 490 |
+
|
| 491 |
+
- Checkpoint Manager now adopts active checkpoints written before immutable
|
| 492 |
+
revision sidecars were introduced. It recovers the original transaction id
|
| 493 |
+
from the existing versioned MP4 and safetensors filenames, writes only the
|
| 494 |
+
missing lightweight JSON revision records, and links the legacy linear
|
| 495 |
+
lineage without copying or rewriting media, latents, or active pointers.
|
| 496 |
+
|
| 497 |
+
- Added **Generation Profile**, a two-control replacement for the cryptic Chain
|
| 498 |
+
Policy switches. Clear scene-continuity and audio choices include Generate
|
| 499 |
+
audio, fresh per-scene audio, source-guided generation, source soundtrack,
|
| 500 |
+
silence, and exact source-audio lip-sync. The former node remains compatible
|
| 501 |
+
as deprecated **Manual Chain Policy (Legacy)**.
|
| 502 |
+
|
| 503 |
+
- Plan Studio's horizontal timeline position is now inert during passive
|
| 504 |
+
rerenders, workspace growth, and prompt-editor scene synchronization. It
|
| 505 |
+
restores the exact pixel position, cancels stale layout callbacks, and only
|
| 506 |
+
reveals a scene after an explicit timeline selection; automatic restoration
|
| 507 |
+
can no longer recursively grow or throw the scrollbar to either end.
|
| 508 |
+
|
| 509 |
+
- Both prompt editors now discover tagged and Project Asset references through
|
| 510 |
+
the compact **Current Tagged Ref2VA Scene** node, just as they already did
|
| 511 |
+
through the original standalone Tagged Ref2VA wrapper.
|
| 512 |
+
|
| 513 |
+
- Added a modern **Current Tagged Ref2VA Scene** composite that replaces the
|
| 514 |
+
visible Current Shot plus Tagged Ref2VA pair while reusing both established
|
| 515 |
+
execution paths internally. Its public outputs are only state, conditioning,
|
| 516 |
+
latent, and typed scene data. A separate versioned options node carries
|
| 517 |
+
reference/semantic/backend settings, and **Scene Data Extract** selects former
|
| 518 |
+
secondary outputs—including `refmod_sources`—with a concrete UI socket type.
|
| 519 |
+
The new route deliberately rejects the legacy 0.4 state contract and exposes
|
| 520 |
+
no legacy source-audio or blend socket; both original nodes remain unchanged.
|
| 521 |
+
|
| 522 |
+
- Tagged Ref2VA now has an opt-in `external_refmod` conditioning backend for
|
| 523 |
+
testing ComfyUI-MiniMaxH3Mod without duplicating native references. It keeps
|
| 524 |
+
scene-local `@tag` selection and video slicing, emits text-only H3
|
| 525 |
+
conditioning/latent, and exposes the active pictures/videos as the upstream
|
| 526 |
+
`H3_REF_LIST` contract through `refmod_sources`. Native Ref2VA remains the
|
| 527 |
+
default. Active semantic `#anchors` now use a hybrid path: ordinary `@visuals`
|
| 528 |
+
remain compressed RefMods, while an anchor-only Qwen presentation preserves
|
| 529 |
+
each anchor's timestamp and storyboard mode without re-encoding all ordinary
|
| 530 |
+
reference media. Active reference audio still fails explicitly because the
|
| 531 |
+
external format is visual-only. External Extract/Apply settings remain
|
| 532 |
+
outside Plan fingerprinting and native deferred-upscale caching.
|
| 533 |
+
|
| 534 |
+
- Plan Studio's generated track is now an editorial timeline rather than a
|
| 535 |
+
delayed view of generation order. Absolute scene placements can move any
|
| 536 |
+
scene—including the terminal scene—before or after other clips; collisions
|
| 537 |
+
resolve as non-overlapping inserts, uncovered time stays black, and raw
|
| 538 |
+
placement requests are no longer rewritten during layout. Playback, motion
|
| 539 |
+
references, source-audio waveforms, generated scene audio, and final video
|
| 540 |
+
assembly share the resolved editorial order. Generation lineage and saved
|
| 541 |
+
checkpoints remain unchanged, while boundaries whose generation predecessor
|
| 542 |
+
changed assemble as safe hard cuts.
|
| 543 |
+
|
| 544 |
+
- Audio assets in the Project Asset Carousel now open a lyrics workspace beside
|
| 545 |
+
the audio player. Lyrics can be pasted or edited in place, are persisted in
|
| 546 |
+
both the project catalog and its H3-chain backup, and remain presentation-only:
|
| 547 |
+
they do not alter prompts, reference fingerprints, or generation behavior.
|
| 548 |
+
|
| 549 |
+
- Chapter sections in Checkpoint Manager's **All scenes** view can now be
|
| 550 |
+
collapsed from their headers. Expanded/collapsed state is remembered per
|
| 551 |
+
run and chapter in workflow presentation state, and collapsed chapters skip
|
| 552 |
+
rendering their branch cards until reopened.
|
| 553 |
+
|
| 554 |
+
- Checkpoint Manager now treats editorial chapters as independent revision
|
| 555 |
+
scopes. **All scenes** renders a separate branch graph per chapter, and
|
| 556 |
+
making a branch active rewrites or retires pointers only inside that
|
| 557 |
+
chapter. Earlier and later chapters retain their active takes; a selected
|
| 558 |
+
manifest combines the selected chapter with the active takes from preceding
|
| 559 |
+
chapters without requiring a structural parent edge across the chapter
|
| 560 |
+
boundary. Explicit visual/audio context metadata remains intact.
|
| 561 |
+
|
| 562 |
+
- Carousel folders now behave like compact Discord folders. Each folder is a
|
| 563 |
+
card in the asset strip with a four-item miniature and count; clicking
|
| 564 |
+
expands or collapses its assets inline, and dropping an asset onto the card
|
| 565 |
+
moves it into that folder. Expansion persists as workflow presentation state
|
| 566 |
+
without affecting generation, prompts, or fingerprints.
|
| 567 |
+
|
| 568 |
+
- Project Asset previews now resolve correctly in both prompt editors. The
|
| 569 |
+
Carousel media URL is retained as a preview record instead of being mistaken
|
| 570 |
+
for a preview object, restoring hover players, inline picture thumbnails,
|
| 571 |
+
reference-tray thumbnails, and the owning Carousel source label.
|
| 572 |
+
|
| 573 |
+
- The Project Asset Carousel is now a direct media drop target. Dropping one
|
| 574 |
+
or several image, video, or audio files creates ordered project assets
|
| 575 |
+
immediately. External file drops are isolated from internal card dragging,
|
| 576 |
+
thumbnails no longer start native browser drags, and the compact drop cue
|
| 577 |
+
leaves reorder targets visible. The
|
| 578 |
+
source browser now defaults explicitly to ComfyUI input, so existing input
|
| 579 |
+
files can be turned into assets through **Import** without uploading them
|
| 580 |
+
again.
|
| 581 |
+
|
| 582 |
+
- Project Asset image variants now accept exact megapixel targets and presets
|
| 583 |
+
without requiring a crop. Output aspect locking couples width and height in
|
| 584 |
+
both directions, **Use full image** preserves the megapixel target, and a
|
| 585 |
+
connected upscale model reports its exact input and final saved dimensions.
|
| 586 |
+
Output snapping now offers Off/8/16/32/64 (default 8), preserves the locked
|
| 587 |
+
aspect ratio instead of producing arbitrary odd sizes, and **Reset all**
|
| 588 |
+
restores the full editor state. Megapixel and multiple presets use compact
|
| 589 |
+
fixed grids so their final choices no longer stretch across wrapped rows.
|
| 590 |
+
|
| 591 |
+
- Reference discovery in Plan Studio and both scene prompt editors now crosses
|
| 592 |
+
ComfyUI subgraph input/output rails. Tagged references, Semantic Anchor
|
| 593 |
+
Bundles, their internal semantic picture anchors, and externally supplied
|
| 594 |
+
preview images remain visible when the reference registry is packed into a
|
| 595 |
+
subgraph for presentation or reusable storage.
|
| 596 |
+
|
| 597 |
+
- Plan editing now exposes only two timing levels: the Plan-wide default
|
| 598 |
+
seconds/steps widgets and optional per-scene overrides. The redundant
|
| 599 |
+
JSON-level default controls were removed from both Scene Plan and Plan
|
| 600 |
+
Studio, and scene placeholders now inherit directly from the Plan widgets.
|
| 601 |
+
|
| 602 |
+
- Checkpoint Manager branch rows keep their compact shared-revision colors and
|
| 603 |
+
badges without drawing long SVG lineage rails across the branch list.
|
| 604 |
+
|
| 605 |
+
- Plan Studio now persists its discovered motion-reference and Source Timeline
|
| 606 |
+
audio presentation beside the run, then restores it automatically after a
|
| 607 |
+
workflow, browser, or server reload. This UI-only record does not participate
|
| 608 |
+
in generation fingerprints, checkpoint branches, or resume validation.
|
| 609 |
+
|
| 610 |
+
- Long Plan timelines no longer rebuild one live video element per saved scene
|
| 611 |
+
on every checkpoint poll. Generated cards use immutable, lazy, disk-cached
|
| 612 |
+
JPEG thumbnails and update in place, substantially reducing browser media
|
| 613 |
+
churn and the associated Windows Proactor `WinError 10054` disconnect noise.
|
| 614 |
+
|
| 615 |
+
- The two-clip masked AV bridge now accepts silent source videos. Start and end
|
| 616 |
+
audio are independent optional inputs: an available endpoint is encoded and
|
| 617 |
+
protected, while a missing endpoint remains unmasked so H3 generates audio
|
| 618 |
+
there instead of failing or forcing artificial silence.
|
| 619 |
+
|
| 620 |
+
- **Use this take & stop batch** now pins the accepted candidate preview across
|
| 621 |
+
targeted cancellation and the automatic next-scene queue. Late payloads from
|
| 622 |
+
an earlier scene can no longer replace it with the predecessor preview; a new
|
| 623 |
+
run of the same scene, the next scene, or an explicit checkpoint choice
|
| 624 |
+
releases the pin normally.
|
| 625 |
+
|
| 626 |
+
- Review Gate candidate polling no longer reloads an unchanged preview or
|
| 627 |
+
jumps back to the newest take when a live candidate batch reaches its final
|
| 628 |
+
review token. The selected take, playback position, and playing/paused state
|
| 629 |
+
survive progress refreshes and synchronized-audio preview upgrades.
|
| 630 |
+
|
| 631 |
+
- Plan Studio's scene lanes now behave as one horizontally zoomable timeline.
|
| 632 |
+
**Fit** compresses the complete Plan into the available width; zooming in
|
| 633 |
+
expands duration-proportional scene spans beneath a shared ruler and single
|
| 634 |
+
scrollbar. Generated clips, motion references, Source Timeline audio, and
|
| 635 |
+
the playhead remain aligned while resizing, zooming around the pointer, and
|
| 636 |
+
navigating long Plans.
|
| 637 |
+
|
| 638 |
+
- Live generation can now route **Current Shot** directly through the four
|
| 639 |
+
H3 de-rope adapter nodes, recover the exact AV clock, and save that recovered
|
| 640 |
+
latent back into the normal chain. The same nodes still accept deferred
|
| 641 |
+
Upscale Loop state. Inline continuity resolves the Plan's actual linear,
|
| 642 |
+
non-linear, or composed visual-context source, and a recovered scene tail is
|
| 643 |
+
protected only when a later scene really consumes it.
|
| 644 |
+
|
| 645 |
+
- Added ordered **Composed visual context** in Plan and Plan Studio. A
|
| 646 |
+
continuation can assemble one native H3 context total from two saved scene
|
| 647 |
+
tails. The UI exposes both orientations of every valid split—for example
|
| 648 |
+
`39 = 17+22` as well as `22+17`, then all splits for `56`, `73`, and the
|
| 649 |
+
longer native totals. Native-phase layouts splice latent blocks directly;
|
| 650 |
+
inverse layouts are normalized through the connected video VAE only when a
|
| 651 |
+
latent continuation consumes them. Plan Studio also remembers whether its
|
| 652 |
+
Advanced Boundary controls are open while settings re-render. The complete
|
| 653 |
+
generated-audio latent stays
|
| 654 |
+
continuous from the immediately previous timeline scene, the two visual
|
| 655 |
+
sources are tracked independently by checkpoint preflight, and checkpoint
|
| 656 |
+
activation restores the exact source IDs, total, and split.
|
| 657 |
+
|
| 658 |
+
- Unified source and deferred-upscale finishing in **H3 Chain Assemble**.
|
| 659 |
+
Upscale Loop End now emits the common manifest wire; Assemble dispatches by
|
| 660 |
+
document format, keeps HQ finals under the child profile, and exposes the
|
| 661 |
+
same optional regular-output copy, filename, audio, and assembly controls.
|
| 662 |
+
The former Upscale Merger remains as a deprecated compatibility wrapper.
|
| 663 |
+
|
| 664 |
+
- Ported the strict authoring layer from the standalone H3 Prompt IDE into
|
| 665 |
+
both scene prompt editors. They now share Auto/T2VA/I2VA/FL2VA/L2VA/Ref2VA
|
| 666 |
+
schema selection, ordered-section diagnostics and repair, exact keyframe
|
| 667 |
+
alignment helpers, live word/character counts, and mode-aware completion.
|
| 668 |
+
Completion and rich presentation now also understand 12 shot markers,
|
| 669 |
+
multilingual dialogue markers, stable speaker IDs, `<scenetrans>`, and
|
| 670 |
+
`<cutoff>`; the Rich Scene Prompt Editor gained the same Plain/Rich source
|
| 671 |
+
toggle as the focused editor.
|
| 672 |
+
|
| 673 |
+
- Reference insertion from the Scene Prompt Editor's top menu now preserves
|
| 674 |
+
the last rich-text or plain-text selection, so a chosen reference replaces
|
| 675 |
+
the selection or appears at the requested caret instead of jumping to the
|
| 676 |
+
prompt start. `#` autocomplete and the reference tray now also include
|
| 677 |
+
dedicated semantic-only picture anchors and select their timestamp for
|
| 678 |
+
immediate editing.
|
| 679 |
+
|
| 680 |
+
- Added deterministic alternatives in scene prompts with ComfyUI-style
|
| 681 |
+
`{first|second}` syntax. Every scene now has a **Prompt alternatives** policy:
|
| 682 |
+
derive a stable scene seed, use an exact fixed scene seed, or generate a fresh
|
| 683 |
+
choice each time that Plan is queued. The redundant Plan-wide prompt seed was
|
| 684 |
+
removed. Sampler seeds remain completely
|
| 685 |
+
independent. The authored template, selected text, and exact choice seed are
|
| 686 |
+
saved together. Resume treats a new choice or policy from the same template
|
| 687 |
+
as neutral for completed predecessors, but still rejects an actual template
|
| 688 |
+
edit.
|
| 689 |
+
|
| 690 |
+
- Reference chips in both rich scene-prompt editors are now directly
|
| 691 |
+
editable. Clicking a chip can replace that exact occurrence with another
|
| 692 |
+
compatible connected reference, switch a tagged picture between native
|
| 693 |
+
`@tag` and semantic `#tag[time]` syntax, and edit the semantic timestamp in
|
| 694 |
+
a dedicated seconds field without rewriting the prompt by hand.
|
| 695 |
+
|
| 696 |
+
- Split Qwen-only `#semantic[timestamp]` pictures from native H3 `@reference`
|
| 697 |
+
media. New Semantic Picture Anchor nodes feed one Semantic Anchor Bundle
|
| 698 |
+
with centralized scale/mode controls; the bundle connects once to Tagged
|
| 699 |
+
Ref2VA, Plan Studio, and Run Manager. Semantic pictures no longer consume
|
| 700 |
+
native picture/video/audio capacity, while their source loaders remain
|
| 701 |
+
recoverable through the saved-run asset manifest and their combined
|
| 702 |
+
incremental fingerprint remains resume-safe.
|
| 703 |
+
|
| 704 |
+
- Added **MiniMax H3 Scene LoRA Scheduler**, a lazy per-scene MODEL router
|
| 705 |
+
that deliberately does not load or patch LoRAs. Plan and Plan Studio expose
|
| 706 |
+
Base plus LoRA A-D on every scene; each branch comes from ordinary ComfyUI
|
| 707 |
+
LoRA loaders and may contain a stack or strength variant. Only the selected
|
| 708 |
+
input is evaluated. Route choices are saved as Plan/revision provenance but
|
| 709 |
+
deliberately excluded from checkpoint/resume verification, and Base remains
|
| 710 |
+
absent from serialization for compatibility with existing plans.
|
| 711 |
+
|
| 712 |
+
- The bundled LBH 3D deferred-upscale workflow now leaves the pass-2 prompt
|
| 713 |
+
override blank, preserving the exact compiled scene prompt by default. The
|
| 714 |
+
former neutral replacement text remains available in the How To Run note
|
| 715 |
+
for explicit copy/paste comparisons.
|
| 716 |
+
|
| 717 |
+
- Hid the Upscale Adapter's provenance-only `recipe_json` in the default
|
| 718 |
+
presentation. It remains serialized for resume validation and can be edited
|
| 719 |
+
through **Show advanced H3 controls**.
|
| 720 |
+
|
| 721 |
+
- Fixed Checkpoint Manager branch clicks so the selected lineage is committed
|
| 722 |
+
through the hidden Comfy widget callback and immediately available to
|
| 723 |
+
`selected_manifest`. Branch headers now select their final revision, and
|
| 724 |
+
the UI distinguishes the current selection from the saved active lineage.
|
| 725 |
+
|
| 726 |
+
- Added an inline **Upscale Reference + Prompt Override** on the existing
|
| 727 |
+
`H3_TAGGED_REFERENCES` line. Connected Tagged refs replace the automatic
|
| 728 |
+
cache for pass 2, optional tag filtering can remove unwanted refs, and the
|
| 729 |
+
paired prompt output recompiles their native/Qwen conditioning coherently.
|
| 730 |
+
Blank reference input keeps automatic cache restore active.
|
| 731 |
+
- Fixed deferred H3 conditioning sync enlarging `max` picture-reference
|
| 732 |
+
latents with the output canvas. Max refs now retain their Core H3 capped
|
| 733 |
+
geometry, while `match` refs remain canvas-aware and cache-v2 masters that
|
| 734 |
+
were already rebuilt at pass-2 size are protected from double scaling.
|
| 735 |
+
|
| 736 |
+
- Added a guarded compatibility backport for ComfyUI PR #15808. Older core
|
| 737 |
+
builds now register MiniMax H3's seven released dialogue, cutoff, lyrics,
|
| 738 |
+
and caption tokens on the MiniMax-only Qwen tokenizer; updated ComfyUI
|
| 739 |
+
builds are detected and remain fully core-owned.
|
| 740 |
+
|
| 741 |
+
- Added an opt-in **Reference Video Fade** MODEL patch for native H3 Ref2VA
|
| 742 |
+
video blocks. It keeps the complete 24 fps reference at full early
|
| 743 |
+
influence, then applies a full-schedule half-cosine attention-value fade
|
| 744 |
+
without touching still refs, Qwen presentation, reference audio,
|
| 745 |
+
continuation guides, or target streams. Split samplers can share the
|
| 746 |
+
original unsplit sigma schedule, and existing SolAttn/Comfy Kitchen
|
| 747 |
+
attention overrides remain chained. This complements rather than replaces
|
| 748 |
+
Color-Stable Drift AV: one controls external native video references, while
|
| 749 |
+
the other controls the previous-scene continuation prefix.
|
| 750 |
+
- Added a whole-branch SeedVR2 route: Checkpoint Manager → Full-Chain Latent
|
| 751 |
+
Video Adapter → SeedVR2 Direct. It decodes the immutable H3 video latents,
|
| 752 |
+
resolves scene overlap blends before upscaling, and exposes one continuous
|
| 753 |
+
native file-backed VIDEO instead of restarting an upscaler per scene.
|
| 754 |
+
- Added a disk-backed MiniMax VAE output buffer and content-addressed source
|
| 755 |
+
cache under `upscaled/seedvr2/source/`. The decoder retains its native
|
| 756 |
+
temporal chunks while ordinary RAM holds only the boundary window; saved
|
| 757 |
+
source/generated audio is embedded automatically for the SeedVR2 pass.
|
| 758 |
+
- Added the standalone whole-chain SeedVR2 workflow and the LBH 3D H3 latent
|
| 759 |
+
workflow with video-only learned upscale, locked source audio, and a
|
| 760 |
+
conservative two-step 0.24 refinement pass.
|
| 761 |
+
- Added a backend-neutral recursive upscale child run: Checkpoint Manager
|
| 762 |
+
selected manifest → Upscale Adapter → Current Scene → H3/LTX/custom backend →
|
| 763 |
+
Segment Save → Loop End → Merger. Each profile is isolated under the parent
|
| 764 |
+
run's `upscaled/<profile>` folder with verified resume metadata.
|
| 765 |
+
- Upscale Current Scene prefers an explicitly saved denoised H3 x0 and exposes
|
| 766 |
+
joint AV plus split video/audio latent routes for combined and video-only
|
| 767 |
+
learned H3 upscalers. Older terminal-latent checkpoints remain supported,
|
| 768 |
+
and decoded-video LTX 2.5 passes use the same orchestration.
|
| 769 |
+
- Made persistence of the large HQ latent optional and off by default. A small
|
| 770 |
+
self-contained assembly/audio checkpoint is still written for reliable
|
| 771 |
+
resume and final merge.
|
| 772 |
+
- Checkpoint Manager now serializes its selected immutable lineage and emits a
|
| 773 |
+
verified generated-tip manifest, including partial runs whose later planned
|
| 774 |
+
scenes do not exist yet. The standalone upscale workflow therefore needs
|
| 775 |
+
no source Plan, Chain Policy, or live Source Timeline connection.
|
| 776 |
+
- Added cache-v2 pass-2 conditioning: original picture masters are retained so
|
| 777 |
+
`match` image refs and their Qwen presentation can be rebuilt at the exact
|
| 778 |
+
LBH target canvas. V1 caches remain readable; max/video/audio refs preserve
|
| 779 |
+
their native geometry instead of being blindly scaled.
|
| 780 |
+
|
| 781 |
+
## v0.5.5 — Modern editable-install metadata
|
| 782 |
+
|
| 783 |
+
- Added an explicit setuptools build backend and disabled accidental discovery
|
| 784 |
+
of asset and workflow folders as Python packages, so `pip install -e .`
|
| 785 |
+
succeeds with current setuptools releases.
|
| 786 |
+
- Replaced deprecated license metadata with its SPDX form while leaving the
|
| 787 |
+
repository's GPL v3 license text unchanged. Thanks to @ed45626 in PR #27.
|
| 788 |
+
|
| 789 |
+
## v0.5.4 — Cleaner shared-checkpoint links
|
| 790 |
+
|
| 791 |
+
- Moved shared-revision connectors into a dedicated side gutter with thin
|
| 792 |
+
solid rails and short taps to each matching card, keeping lineage marks away
|
| 793 |
+
from branch names, revision text, and status labels.
|
| 794 |
+
|
| 795 |
+
## v0.5.3 — Review candidate batches and checkpoint refresh
|
| 796 |
+
|
| 797 |
+
- Review Gate can now generate 1–20 different-seed takes for each scene,
|
| 798 |
+
present them together, and continue from the exact saved video/audio
|
| 799 |
+
checkpoint selected by the user. The default remains one take.
|
| 800 |
+
- Selecting an earlier take atomically promotes its checkpoint and recovery
|
| 801 |
+
Plan before the loop continues, so later scenes and interrupted-run recovery
|
| 802 |
+
follow the chosen branch rather than the last generated take.
|
| 803 |
+
- Fixed overlapping checkpoint refresh requests adding duplicate choices after
|
| 804 |
+
workflow reload, and clarified that rejected, rerolled, and candidate takes
|
| 805 |
+
remain as intentional immutable recovery revisions until explicitly deleted.
|
| 806 |
+
|
| 807 |
+
## v0.5.2 — Shared checkpoint lineage visibility
|
| 808 |
+
|
| 809 |
+
- Repeated checkpoint revisions now keep the existing branch layout while
|
| 810 |
+
receiving a consistent color label and a vertical connector between every
|
| 811 |
+
branch line that shares the same clip.
|
| 812 |
+
- Loop Trim now resolves `video_blend_frames` from Current Shot state. This
|
| 813 |
+
removes the ambiguous Plan-default versus per-scene integer wiring that could
|
| 814 |
+
discard the requested overlap and fail Segment Save only after sampling.
|
| 815 |
+
Legacy integer sockets remain compatible, while maintained 0.5 workflows and
|
| 816 |
+
the migration tool use the authoritative state route.
|
| 817 |
+
|
| 818 |
+
## v0.5.1 — Checkpoint Manager and workflow clarity
|
| 819 |
+
|
| 820 |
+
- Added experimental **Drift-Control AV** for recursive same-shot chains. It
|
| 821 |
+
keeps predecessor checkpoints clean, but at every sampler evaluation moves
|
| 822 |
+
the disposable 39-frame video prefix to the next scheduler sigma using the
|
| 823 |
+
sampler's existing noise field. Eight video-latent steps receive the full
|
| 824 |
+
matched ratio and the final four taper `.75/.50/.25/.00` to an exact seam.
|
| 825 |
+
Chain Context now exposes an optional MODEL passthrough so the sampler blend
|
| 826 |
+
and H3 per-row timestep labels receive the same live mask. The 20-step path
|
| 827 |
+
is the initial validated baseline; existing continuation modes are unchanged.
|
| 828 |
+
|
| 829 |
+
- Corrected the scheduled Guide 5/6 spatial proxy to reproduce the observed
|
| 830 |
+
mixed-resolution chain operation: reduce the complete saved predecessor
|
| 831 |
+
video latent, VAE-decode it on the 5/6 grid, select the delivered RGB tail,
|
| 832 |
+
and only then restore it through Motion Context. The UI now calls this
|
| 833 |
+
**Low-grid 5/6 · Guide** instead of implying a simple RGB resize. Existing
|
| 834 |
+
AV latent-prefix proxy behavior is unchanged.
|
| 835 |
+
|
| 836 |
+
- Added a Plan-passthrough Checkpoint Manager that browses every saved run by
|
| 837 |
+
scene and inferred revision branch, previews saved video/audio, and exposes
|
| 838 |
+
prompts, seeds, frame counts, compatibility data, storage, lineage, and the
|
| 839 |
+
exact incoming video/audio context for each revision.
|
| 840 |
+
- Added dependency-aware cleanup. Active revisions and revisions used by later
|
| 841 |
+
scenes are protected; the manager identifies every dependent scene and lets
|
| 842 |
+
users work backward from a leaf one revision at a time.
|
| 843 |
+
- Added a two-step deletion contract shared with Review Gate. The server
|
| 844 |
+
previews every owned/shared file and preserved archive category, then rejects
|
| 845 |
+
confirmation if files, active pointers, or descendants changed in between.
|
| 846 |
+
- New checkpoints persist creation time, effective continuation context, and a
|
| 847 |
+
stable branch identity so future runs need less lineage inference while old
|
| 848 |
+
checkpoint folders remain fully discoverable.
|
| 849 |
+
|
| 850 |
+
- Made titles, active controls, and rich reference tags derive their semantic
|
| 851 |
+
colors from the active ComfyUI foreground. Both scene prompt editors retain
|
| 852 |
+
their pastel dark-theme palette while gaining readable contrast in light
|
| 853 |
+
themes.
|
| 854 |
+
|
| 855 |
+
- Added shared text-level Ctrl/Cmd+Z and redo history to both dedicated scene
|
| 856 |
+
prompt editors. Undo now survives rich-tag DOM decoration, plain-text paste,
|
| 857 |
+
toolbar insertion, Plan synchronization, and switching between rich and
|
| 858 |
+
plain presentation.
|
| 859 |
+
|
| 860 |
+
- Added a maintained two-scene Ref2V masked-inpaint demo that uses the bundled
|
| 861 |
+
CC0 crab picture as replacement appearance while keeping the source movie as
|
| 862 |
+
the authoritative joint AV target. Documented its exact mask-contract
|
| 863 |
+
compatibility with Ablejones/droz's v3.1 MaskVidExperiments workflow and the
|
| 864 |
+
separate, optional roles of crop/uncrop, tracking, and mask growth.
|
| 865 |
+
- Reworked the README as a task-oriented quick start and workflow chooser,
|
| 866 |
+
moving implementation detail into focused guides. Added a feature
|
| 867 |
+
traceability matrix that distinguishes original, adapted, inspired,
|
| 868 |
+
integrated, and compatibility work with upstream, code, and commit links.
|
| 869 |
+
- Added exact H3 causal/token mask conversion to Apply Target Mask. Static or
|
| 870 |
+
correctly sliced tracked masks now max-reduce through the VAE's repeating
|
| 871 |
+
`1,4,4,4,4` source-frame groups and the model's 2×2 latent-token cells,
|
| 872 |
+
avoiding temporal interpolation drift and loss of thin moving regions. The
|
| 873 |
+
bundled inpaint workflow uses the exact mode; legacy trilinear conversion
|
| 874 |
+
remains selectable for controlled compatibility comparisons. Ordinary AV
|
| 875 |
+
extension remains unchanged and needs no user-supplied mask.
|
| 876 |
+
- Revised experimental Detail AV to a clean-boundary v2 taper. Video-latent
|
| 877 |
+
noise now starts at 0.30 and falls through 0.225, 0.15, and 0.075 to an exact
|
| 878 |
+
zero on the boundary-adjacent step, reducing seam displacement while keeping
|
| 879 |
+
the predecessor, carried audio, denoise masks, and final overlap clean.
|
| 880 |
+
- Recognized the final merged ComfyUI PR #15375 helper-based mask API and left
|
| 881 |
+
it fully native. The old-build fallback now mirrors merge-time commit
|
| 882 |
+
`c676536`, including pooled token-grid masks and ceil-quantized 8-bit mask
|
| 883 |
+
strengths, and removes obsolete pre-merge wrappers before capability checks.
|
| 884 |
+
- Restored saved prompts, Plan widgets, and connected 0.5 audio/transition
|
| 885 |
+
policy nodes together when loading a run or checkpoint revision. Revision
|
| 886 |
+
recovery now uses the selected checkpoint chain's policy metadata instead of
|
| 887 |
+
silently retaining newer controls from the active workflow.
|
| 888 |
+
- Disabled the legacy Review Gate prompt textarea by default and added an
|
| 889 |
+
Interface setting to restore it. While disabled, Retry and Reroll use the
|
| 890 |
+
active Plan prompt authored in Scene Prompt Editor or Rich Scene Prompt
|
| 891 |
+
Editor.
|
| 892 |
+
- Added the typed Source Timeline contract so path-backed picture and audio are
|
| 893 |
+
registered once, remain lazy, and are sliced per scene without full-track
|
| 894 |
+
AUDIO fan-out.
|
| 895 |
+
- Split audio intent into final soundtrack, source-reference, and generated-
|
| 896 |
+
continuity axes while preserving the exact behavior of saved 0.4 modes.
|
| 897 |
+
- Added Cut, Guide, Latent Guide, Detail Guide, Hard AV, and Soft AV incoming-
|
| 898 |
+
transition policies with advanced access to the existing low-level controls.
|
| 899 |
+
- Aligned Soft AV with the upstream tested recipe: an exact 39-frame picture
|
| 900 |
+
prefix plus a half-cosine audio release. The older dual-stream
|
| 901 |
+
`feathered_av` implementation remains available through Expert override for
|
| 902 |
+
compatibility, and the former Audio Feather AV preset remains as an alias.
|
| 903 |
+
- AV continuation now validates the shared 24 fps picture / 40 Hz audio clock
|
| 904 |
+
before model loading. Only 39, 90, 141, 192, or 243 context frames are
|
| 905 |
+
accepted, eliminating silently rounded prefixes such as 22 or 56 frames.
|
| 906 |
+
- Preserved masked-AV decoded overlap audio through Loop Trim and Segment Save
|
| 907 |
+
without adding a public socket. Generated-audio assembly now gives the
|
| 908 |
+
incoming AV scene ownership of that overlap at both generated-scene and
|
| 909 |
+
external-prelude boundaries, so Soft AV's audio feather reaches the final
|
| 910 |
+
track; older checkpoints retain delivered-only fallback behavior.
|
| 911 |
+
- Updated both maintained soldier-crab extension workflows to the 0.5 Soft AV
|
| 912 |
+
policy with a full 39-frame visual seam blend, matching the tested upstream
|
| 913 |
+
extension topology while retaining recursive Plan/Run Manager operation.
|
| 914 |
+
- Added opt-in Latent Guide continuation. Generated boundaries reuse the
|
| 915 |
+
phase-aligned tail of the accepted predecessor's sampled video latent as
|
| 916 |
+
persistent Guide conditioning without an RGB/VAE round trip; imported or
|
| 917 |
+
incompatible context retains the original Guide fallback.
|
| 918 |
+
- Added opt-in Detail Guide continuation, adapting MacroSony's deterministic
|
| 919 |
+
tapered chroma-noise context recipe. The 22-frame preset uses 19 frames at
|
| 920 |
+
0.45 and a three-frame taper to 0.10; expert mode supports other Guide
|
| 921 |
+
lengths without modifying the accepted predecessor checkpoint.
|
| 922 |
+
- Added experimental Detail AV continuation, adapting beijinren's latent
|
| 923 |
+
context-noise recipe to the recursive hard-AV path. It treats a disposable
|
| 924 |
+
copy of the 39-frame / 12-step video prefix with matched-variance Gaussian
|
| 925 |
+
noise using a boundary-clean taper, leaves audio and masks unchanged, trims
|
| 926 |
+
the complete treated overlap, and fingerprints the exact recipe for resume.
|
| 927 |
+
- Added model-free Chain Preflight and shared Plan Studio validation for source
|
| 928 |
+
duration, scene windows, references, runtime compatibility, and resume
|
| 929 |
+
eligibility.
|
| 930 |
+
- Added a second Plan Studio track for the exact active motion-reference
|
| 931 |
+
windows. Selected scenes are transcoded lazily to cached low-resolution
|
| 932 |
+
previews, and the player provides synchronized generated/reference wipe
|
| 933 |
+
comparison with incoming-context offsets applied automatically.
|
| 934 |
+
- Replaced opaque whole-Plan resume hashes with structured scene dependencies
|
| 935 |
+
and actionable diffs. Future-scene, incoming-boundary, and assembly-only
|
| 936 |
+
changes no longer force regeneration of an accepted predecessor.
|
| 937 |
+
- Simplified conditional socket presentation, clarified active Plan versus
|
| 938 |
+
selected Run Manager archive state, and made prompt-history activation
|
| 939 |
+
explicit.
|
| 940 |
+
- Migrated every maintained workflow to explicit policies and preflight. The
|
| 941 |
+
source-audio demo now uses canonical Source Timeline wiring and Current
|
| 942 |
+
Shot's scene-local reference slice.
|
| 943 |
+
- Added an idempotent workflow migration tool, frozen 0.4 positional fixtures,
|
| 944 |
+
and backend/frontend release validation. Existing nodes, workflow JSON,
|
| 945 |
+
manifests, and checkpoints remain supported.
|
| 946 |
+
- Completed input and output tooltips across the 0.5 policy, Source Timeline,
|
| 947 |
+
prompt-driven reference, semantic-anchor, masking, and bridge nodes. The
|
| 948 |
+
integration smoke fixture now follows the real Current Shot state wire so
|
| 949 |
+
strict per-scene source-window provenance is exercised during recovery.
|
| 950 |
+
|
| 951 |
+
## v0.4.9 — Run Manager active asset target
|
| 952 |
+
|
| 953 |
+
- Made the Run Manager show the connected Plan `run_name` beside Reference
|
| 954 |
+
assets so the destination of **Save/update assets** is explicit.
|
| 955 |
+
- After a manual asset save, the saved-run browser now selects the actual
|
| 956 |
+
destination run instead of remaining on a previously browsed run.
|
| 957 |
+
- Kept asset-only run folders selectable for inspection and opening while
|
| 958 |
+
leaving **Load into Plan** disabled until a restorable Plan archive exists.
|
| 959 |
+
|
| 960 |
+
## v0.4.8 — Feathered AV continuation
|
| 961 |
+
|
| 962 |
+
- Added `feathered_av` as a third continuation mode. It uses the same aligned
|
| 963 |
+
video/audio target prefix as `masked_av`, but ramps the end of that prefix
|
| 964 |
+
from preservation toward generation to soften the temporal handoff.
|
| 965 |
+
- For the recommended 39-frame prefix, the first 8 video latent steps and 42
|
| 966 |
+
audio steps remain fully protected. The final 4 video steps and 23 audio
|
| 967 |
+
steps use monotonic denoise ramps; all future rows remain fully denoisable.
|
| 968 |
+
- Added the mode to Plan, per-scene overrides, Plan Studio, timing validation,
|
| 969 |
+
resume provenance, and the native-first PR #15375 compatibility path.
|
| 970 |
+
- Added Loop Start `verify_resume_history`, enabled by default. Disabling it
|
| 971 |
+
explicitly permits reuse of a saved predecessor after intentional Plan
|
| 972 |
+
changes while retaining file, SHA-256, tensor-shape, and internal metadata
|
| 973 |
+
integrity checks.
|
| 974 |
+
- Corrected sequential Tagged Motion Ref timing under masked AV continuation.
|
| 975 |
+
Scene 2+ now excludes the repeated prefix from the unpositioned native video
|
| 976 |
+
bank and applies the identical delivered-frame window to paired audio,
|
| 977 |
+
removing the context-length motion delay without changing generic video refs.
|
| 978 |
+
- Added path-backed Tagged Motion Ref for long CFR control videos. It keeps
|
| 979 |
+
compressed media on disk and decodes/resizes only the active scene, converts
|
| 980 |
+
that window to 24 fps, and extracts its exact embedded-audio time range. A
|
| 981 |
+
separate Plan-aware scene-counter preview blocks media output when no Plan is
|
| 982 |
+
connected or the tag is inactive. A native-frame start offset seeks near the
|
| 983 |
+
requested source point and shifts video and audio together without building
|
| 984 |
+
tensors for the skipped prefix. It also accepts native file-backed `VIDEO`
|
| 985 |
+
from core Load Video, allowing that loader to fan out normally to both the
|
| 986 |
+
tagged reference and Run Manager asset registration. A dedicated Lazy
|
| 987 |
+
Motion AV Loader now exposes that same disk-backed native VIDEO plus the
|
| 988 |
+
complete post-skip AUDIO track without decoding any video frames. The tagged
|
| 989 |
+
node accepts and passes through that full track, while retaining direct-path
|
| 990 |
+
audio extraction for compatibility.
|
| 991 |
+
|
| 992 |
+
## v0.4.7 — General masked targets
|
| 993 |
+
|
| 994 |
+
- Updated the vendored PR #15375 fallback through upstream commit `989e7a9`.
|
| 995 |
+
Arbitrary masks now retain their original per-pixel sampler blend while H3
|
| 996 |
+
pools only its internal video/audio timestep labels to the token grid. Older
|
| 997 |
+
ComfyUI builds receive a scoped per-step sampler bridge; post-PR builds keep
|
| 998 |
+
their native implementation untouched.
|
| 999 |
+
- Made Plan-wide continuation mode and context length next-scene choices for
|
| 1000 |
+
resume validation. Switching either after a completed scene now reuses that
|
| 1001 |
+
verified predecessor instead of demanding a pointless regeneration. When a
|
| 1002 |
+
longer tail was not cached, it is recovered by re-decoding the saved full
|
| 1003 |
+
video latent. Explicit per-scene overrides remain generation-significant,
|
| 1004 |
+
and new segment records retain their effective values for provenance.
|
| 1005 |
+
- Fixed master-audio encoding when H3's rounded 40 Hz target requires one
|
| 1006 |
+
latent step beyond a floor-style VAE encode of the exact picture duration.
|
| 1007 |
+
The node uses real timeline-audio lookahead, crops to the authoritative
|
| 1008 |
+
target length, and keeps `clip_audio` at exact picture duration.
|
| 1009 |
+
- Added an exact master-audio target composer for timeline-driven FL2VA. It
|
| 1010 |
+
VAE-encodes the current interval from music, dialogue, narration, or effects
|
| 1011 |
+
into the complete H3 audio target, protects every audio row, and optionally
|
| 1012 |
+
protects a native previous-video prefix while future video rows remain
|
| 1013 |
+
denoisable.
|
| 1014 |
+
- Added public source-AV trim, 32px mask-grid preview, and arbitrary H3 masked
|
| 1015 |
+
target nodes for static or tracked video inpainting.
|
| 1016 |
+
- Added preserve, generate, follow-video, and custom audio-mask modes. New
|
| 1017 |
+
masks intersect existing nested masks, allowing spatial edits to compose
|
| 1018 |
+
with the chain's protected `masked_av` prefix.
|
| 1019 |
+
- Adapted the complete masked-video inpaint workflow to the existing
|
| 1020 |
+
native-first PR #15375 compatibility. The old per-row MODEL patch is not
|
| 1021 |
+
required, and distinct public IDs allow both packs to remain installed.
|
| 1022 |
+
- Documented how the same mask contract extends to prepared outpaint canvases,
|
| 1023 |
+
temporal repairs, and two-ended clip-bridge targets.
|
| 1024 |
+
|
| 1025 |
+
## v0.4.10 — Final-output publishing
|
| 1026 |
+
|
| 1027 |
+
- Fixed scene and final MP4 publication in ComfyUI's global Media Assets panel
|
| 1028 |
+
by using the native animated-video output descriptor recognized by job
|
| 1029 |
+
history.
|
| 1030 |
+
- Added optional Final Assemble controls to copy the completed MP4 into the
|
| 1031 |
+
regular ComfyUI output tree. The relative subfolder supports nested folders
|
| 1032 |
+
and date tokens, an empty value targets the output root, and collisions are
|
| 1033 |
+
versioned without replacing the canonical chain final.
|
| 1034 |
+
|
| 1035 |
+
## v0.4.9 — Defensive Sol-Attn observer detection
|
| 1036 |
+
|
| 1037 |
+
- Fixed folder-independent Sol-Attn recognition for the defensive
|
| 1038 |
+
`getattr(self, "segments", ...)` observer used by the Kitchen PR helper,
|
| 1039 |
+
while continuing to require the complete `_video_span`, `_SPANS`,
|
| 1040 |
+
`position_ids`, and `segments` fingerprint. Thanks to @tsolful in PR #16
|
| 1041 |
+
for identifying the bytecode lookup difference.
|
| 1042 |
+
|
| 1043 |
+
## v0.4.8 — Renamed Sol-Attn observer compatibility
|
| 1044 |
+
|
| 1045 |
+
- Made Sol-Attn H3 layout-observer compatibility independent of the custom
|
| 1046 |
+
node's install-folder name. Renamed PR helpers such as
|
| 1047 |
+
`sol_attn_minimax_v2`, lazy installation between scene 1 and scene 2, and
|
| 1048 |
+
nested read-only observer copies now preserve native Add Guide detection;
|
| 1049 |
+
unrelated layout-mutating wrappers remain refused.
|
| 1050 |
+
|
| 1051 |
+
## v0.4.7 — Review/editor synchronization and compatibility
|
| 1052 |
+
|
| 1053 |
+
- Synchronized Review Gate with prompt editors bound to the same Plan. A new
|
| 1054 |
+
review activates its scene in the connected editor, live editor changes are
|
| 1055 |
+
reflected in the Gate, and retry/reroll reads the current Plan prompt unless
|
| 1056 |
+
the Gate field was explicitly edited as a fallback.
|
| 1057 |
+
- Removed the core Ref2VA right-click conversion into Scheduled Ref2VA and its
|
| 1058 |
+
graph-rewriting implementation. Scheduled and Tagged Ref2VA remain available
|
| 1059 |
+
as explicit nodes.
|
| 1060 |
+
- Recognized the audited path-valued H3 layout observer installed by the
|
| 1061 |
+
`ComfyUI-SolAttn-CUDA-PR117` development checkout, while continuing to reject
|
| 1062 |
+
unknown wrappers.
|
| 1063 |
+
- Rotated every browser helper-module cache token with the release so the new
|
| 1064 |
+
Review Gate/core export contract cannot be mixed with cached v0.4.6 modules.
|
| 1065 |
+
|
| 1066 |
+
## v0.4.6 — Asset publishing and media fallbacks
|
| 1067 |
+
|
| 1068 |
+
- Scene Segment Save and Final Assemble now publish their MP4 paths through
|
| 1069 |
+
ComfyUI's standard output descriptor contract, so newly produced scene clips
|
| 1070 |
+
and final videos appear promptly in the global Assets sidebar when enabled.
|
| 1071 |
+
- FFmpeg executables found on `PATH` are now launch-tested once before use. A
|
| 1072 |
+
broken Windows build (including `0xC0000139` DLL entry-point failures) is
|
| 1073 |
+
treated as unavailable so review muxing and final assembly use PyAV instead.
|
| 1074 |
+
- Versioned every production `.mjs` import with the package release and added
|
| 1075 |
+
a consistency regression check, preventing stale browser helper modules from
|
| 1076 |
+
disabling the Plan DOM editor after an update.
|
| 1077 |
+
|
| 1078 |
+
## v0.4.5 — Scene context and checkpoint Plan recovery
|
| 1079 |
+
|
| 1080 |
+
- Added per-scene context-length overrides to the Plan's existing Advanced
|
| 1081 |
+
controls and Plan Studio. Blank inherits the Plan default; `0` creates a
|
| 1082 |
+
visually independent scene.
|
| 1083 |
+
- Added an independent per-scene generated-audio context override. A guide
|
| 1084 |
+
scene can now use zero video context while retaining preceding dialogue,
|
| 1085 |
+
ambience, or music; explicit audio `0` disables the carry, while masked AV
|
| 1086 |
+
remains locked to one synchronized prefix.
|
| 1087 |
+
- Made timing, masked/guide behavior, imported-video scene 1 handling, resume
|
| 1088 |
+
hashes, checkpoint tail storage, Run Manager recovery, and checkpoint
|
| 1089 |
+
revision recovery preserve the effective per-scene context.
|
| 1090 |
+
|
| 1091 |
+
- Made Review Gate's ordinary **Load checkpoint** action restore the saved
|
| 1092 |
+
run's complete Plan before arming Loop Start. Prompts and Plan settings no
|
| 1093 |
+
longer remain from whichever workflow happened to be open.
|
| 1094 |
+
- Reapplied the exact selected checkpoint metadata for every saved predecessor
|
| 1095 |
+
scene, including active revisions, while keeping plan-only recovery free of
|
| 1096 |
+
archived-asset materialization side effects.
|
| 1097 |
+
|
| 1098 |
+
## v0.4.4 — Review retry persistence
|
| 1099 |
+
|
| 1100 |
+
- Kept each Review Gate retry's edited scene prompt, seed, and length bound to
|
| 1101 |
+
the same submitted scene while the server request is in flight.
|
| 1102 |
+
- Made the server return the accepted prompt and synchronized it into the Plan
|
| 1103 |
+
and any open prompt companion editor, preventing stale UI state from
|
| 1104 |
+
replacing the retry prompt.
|
| 1105 |
+
|
| 1106 |
+
## v0.4.3 — Per-scene continuation modes
|
| 1107 |
+
|
| 1108 |
+
- Added per-scene `continuation_mode` overrides in Plan JSON, the compact Plan
|
| 1109 |
+
editor advanced controls, and Plan Studio. The Plan node setting remains the
|
| 1110 |
+
inherited default, so one chain can use flexible `guide` transitions for new
|
| 1111 |
+
shots and exact `masked_av` transitions for continued shots.
|
| 1112 |
+
- Fixed ComfyUI Stop/Cancel while execution is waiting indefinitely at Review
|
| 1113 |
+
Gate; the gate heartbeat now observes the processing-interrupted flag and
|
| 1114 |
+
resolves its browser controls before propagating the normal interruption.
|
| 1115 |
+
- Added experimental Plan `continuation_mode=masked_av`. Chain Context
|
| 1116 |
+
VAE-encodes the preceding scene's decoded tail into the next target video
|
| 1117 |
+
latent, copies the matching sampled audio-latent tail, and emits nested
|
| 1118 |
+
per-stream masks where `0` preserves the prefix and `1` generates the future.
|
| 1119 |
+
- Appended a sampler-ready LATENT output to Chain Context without changing its
|
| 1120 |
+
existing output indices; guide mode and scene 1 pass the original latent
|
| 1121 |
+
through unchanged.
|
| 1122 |
+
- Added lazy native-first PR #15375 compatibility for H3 mask payloads,
|
| 1123 |
+
preprocessing, inpaint scaling, and per-row diffusion timesteps.
|
| 1124 |
+
- Converted Studio Tagged into the wired 39-frame / 65-audio-step masked AV
|
| 1125 |
+
example while retaining Tagged Ref2VA and Plan Studio authoring.
|
| 1126 |
+
|
| 1127 |
+
## v0.4.2 — Checkpoint revision recovery
|
| 1128 |
+
|
| 1129 |
+
- Extended the existing Review Gate checkpoint browser to discover every
|
| 1130 |
+
retained scene revision, preview it, and restore a selected predecessor
|
| 1131 |
+
chain before resuming the next scene.
|
| 1132 |
+
- Restoring revisions updates the editable Plan's scene prompts, seeds,
|
| 1133 |
+
lengths, steps, identifiers, and shared prompt before Loop Start is armed.
|
| 1134 |
+
- Added guarded cleanup for inactive revisions. Review Gate shows each
|
| 1135 |
+
revision's estimated storage and requires an explicit permanent-delete
|
| 1136 |
+
confirmation; active pointers and unrelated run files cannot be removed.
|
| 1137 |
+
|
| 1138 |
+
## v0.4.1 — Tagged source audio and UI fixes
|
| 1139 |
+
|
| 1140 |
+
- Added `source_timeline` playback to Tagged Audio Ref. It fingerprints the full
|
| 1141 |
+
Loop source track while Tagged Ref2VA derives each exact scene slice from
|
| 1142 |
+
Current Shot state, avoiding the circular dependency caused by returning a
|
| 1143 |
+
dynamically sliced audio fingerprint to Plan.
|
| 1144 |
+
- Corrected every maintained Ref2V example to load the Ref2VA diffusion model
|
| 1145 |
+
instead of the FL2VA checkpoint inherited from the workflow template.
|
| 1146 |
+
- Kept the Run Manager's serialized asset-binding state hidden across legacy
|
| 1147 |
+
and current canvas renderers so its internal JSON field cannot leak into the
|
| 1148 |
+
visible node layout.
|
| 1149 |
+
- Constrained the main Plan editor to its assigned DOM-widget bounds and fully
|
| 1150 |
+
suppressed its internal `plan_json` canvas widget, preventing an intermittent
|
| 1151 |
+
invisible hit layer from blocking the native Plan settings.
|
| 1152 |
+
|
| 1153 |
+
## v0.4.0 — Prompt-driven Ref2VA and Studio authoring
|
| 1154 |
+
|
| 1155 |
+
- Added Tagged Picture, Video, and Audio references. Register stable aliases
|
| 1156 |
+
such as `@hero`, `@motion`, and `@voice`; only tags present in the resolved
|
| 1157 |
+
scene prompt are sent to H3 and compacted to native media labels.
|
| 1158 |
+
- Retained numeric-range Scheduled Ref2VA under the legacy-schedule category
|
| 1159 |
+
for workflows that need explicit scene selectors.
|
| 1160 |
+
- Added optional Plan Studio and Rich Scene Prompt Editor authoring, including
|
| 1161 |
+
synchronized scene selection, rich reference chips and previews, prompt
|
| 1162 |
+
revisions, and configurable Direct API or MCP prompt optimization.
|
| 1163 |
+
- Added maintained T2V and I2V Normal/Studio pairs, indexed A→B→A FL2V, Basic /
|
| 1164 |
+
Tagged / Studio Tagged Ref2V, and an experimental advancing motion-reference
|
| 1165 |
+
workflow. Previous examples remain archived rather than deleted.
|
| 1166 |
+
- Added cumulative disk-backed visual blending, final-assembly playback in
|
| 1167 |
+
Review Gate, editable retry duration, and exact scene retiming.
|
| 1168 |
+
- Added native first/last-frame reference previews that follow the active frame
|
| 1169 |
+
index, plus prompt-driven image, video, and audio miniatures.
|
| 1170 |
+
- Added portable Run Manager asset restoration to the Studio Tagged example.
|
| 1171 |
+
- Updated compatibility for merged ComfyUI PR #15439: current ComfyUI owns H3
|
| 1172 |
+
guide placement and payload merging; older builds receive one warning before
|
| 1173 |
+
the guarded fallback is used.
|
| 1174 |
+
- Added an optional external Plan JSON STRING input for provider-independent
|
| 1175 |
+
story-director and LLM workflows.
|
| 1176 |
+
|
| 1177 |
+
Credit: native H3 guide support is by **drozbay**; cumulative audio budgeting
|
| 1178 |
+
was inspired by **seitanism**; the editor interaction pattern was inspired by
|
| 1179 |
+
**nkxx188's ComfyUI-MiniMaxH3-Easy**. See
|
| 1180 |
+
[THIRD_PARTY_NOTICES.md](THIRD_PARTY_NOTICES.md) for the complete attribution.
|
| 1181 |
+
|
| 1182 |
+
## v0.3.28 — PyAV audio rounding tolerance
|
| 1183 |
+
|
| 1184 |
+
Final PyAV muxing tolerates and zero-pads a single missing sample caused by
|
| 1185 |
+
frame-to-sample rounding while continuing to reject larger audio deficits.
|
| 1186 |
+
|
| 1187 |
+
## v0.3.27 — True disabled scheduler compliance
|
| 1188 |
+
|
| 1189 |
+
Disabled policy reaches upstream Schedule nodes, converts scheduler-owned
|
| 1190 |
+
validation into warnings, and omits unusable media. An empty
|
| 1191 |
+
`source_audio_slice` left wired in `generated_audio` mode no longer stops a
|
| 1192 |
+
render.
|
| 1193 |
+
|
| 1194 |
+
## v0.3.26 — Three-level prompt compliance
|
| 1195 |
+
|
| 1196 |
+
Scheduled Ref2VA offers strict, soft, and disabled policy. Strict blocks
|
| 1197 |
+
scheduler mistakes; soft relaxes prompt-alias failures; disabled passes prompt
|
| 1198 |
+
text through unchanged and makes scheduler checks non-blocking.
|
| 1199 |
+
|
| 1200 |
+
## v0.3.25 — Portable run assets and optional tag warnings
|
| 1201 |
+
|
| 1202 |
+
Run Manager accepts dynamic loader-asset connections, records persistent
|
| 1203 |
+
binding identities and original input paths, and can retain content-addressed
|
| 1204 |
+
image/audio/video fallbacks under the run folder. Restore prefers the original
|
| 1205 |
+
input file and materializes an archived fallback only when needed. Scheduled
|
| 1206 |
+
Ref2VA can downgrade unresolved prompt-tag failures to visible log warnings.
|
| 1207 |
+
|
| 1208 |
+
## v0.3.24 — Saved Run Manager
|
| 1209 |
+
|
| 1210 |
+
A companion node browses projects under `output/h3_chains`, reports scene and
|
| 1211 |
+
checkpoint details, and restores archived prompts and Plan settings after
|
| 1212 |
+
confirmation. Exact API/workflow inputs are preferred, with `plan.json` as the
|
| 1213 |
+
older-run fallback.
|
| 1214 |
+
|
| 1215 |
+
## v0.3.23 — Branching scene-prompt history
|
| 1216 |
+
|
| 1217 |
+
The Scene Prompt Editor keeps lazy per-scene revisions outside workflow and
|
| 1218 |
+
Plan JSON. Its compact `‹ 2 / 5 ›` control navigates versions, shows execution
|
| 1219 |
+
state and timestamp, and creates a child branch when an executed revision is
|
| 1220 |
+
edited.
|
| 1221 |
+
|
| 1222 |
+
## v0.3.22 — Optional floating reroll control
|
| 1223 |
+
|
| 1224 |
+
A ComfyUI setting under **MiniMax H3 Context Loop → Interface → Cancel &
|
| 1225 |
+
reroll** can hide the floating in-progress action. Review Gate controls remain
|
| 1226 |
+
available.
|
| 1227 |
+
|
| 1228 |
+
## v0.3.21 — Upstream continuity update and exact assembly
|
| 1229 |
+
|
| 1230 |
+
Motion Context preserves a stock H3 `last_frame` target while replacing a
|
| 1231 |
+
conflicting first-frame anchor with its carried head. Added 56-frame context,
|
| 1232 |
+
the in-graph Seam Probe, cumulative generated-audio sample budgeting, and
|
| 1233 |
+
stitcher-ready retained visual overlap. The cumulative-audio approach was
|
| 1234 |
+
inspired by **seitanism's**
|
| 1235 |
+
[MultiRef implementation](https://github.com/seitanism/ComfyUI-H3-Motion-Context-MultiRef).
|
| 1236 |
+
|
| 1237 |
+
## v0.3.20 — Cancel and reroll the active scene
|
| 1238 |
+
|
| 1239 |
+
During generation, a guarded floating action can cancel only the active prompt,
|
| 1240 |
+
assign a new explicit scene seed, preserve the selected range end, and requeue
|
| 1241 |
+
through ComfyUI's normal queue.
|
| 1242 |
+
|
| 1243 |
+
## v0.3.19 — Plan and review UX pass
|
| 1244 |
+
|
| 1245 |
+
Plan controls retain pointer input, editor and preview sizes persist, scene
|
| 1246 |
+
seeds remain visible, and reference menus show only sources active in the
|
| 1247 |
+
selected scene. Documentation clarifies that `@aliases` are optional.
|
| 1248 |
+
|
| 1249 |
+
## v0.3.14 — Explicit compatible patch priority
|
| 1250 |
+
|
| 1251 |
+
The optional wired **MiniMax H3 Patch Priority** pass-through can promote this
|
| 1252 |
+
pack over an older compatible Motion Context copy while retaining recognized
|
| 1253 |
+
H3-Multishot and SolAttn behavior.
|
| 1254 |
+
|
| 1255 |
+
## v0.3.13 — Open a Plan's output folder
|
| 1256 |
+
|
| 1257 |
+
A compact **Output** action creates and opens
|
| 1258 |
+
`output/h3_chains/<run_name>` on the ComfyUI host. Headless hosts fall back to
|
| 1259 |
+
copying the host path into the browser clipboard.
|
| 1260 |
+
|
| 1261 |
+
## v0.3.12 — Clearer Plan guidance and looping I2VA
|
| 1262 |
+
|
| 1263 |
+
Expanded Plan tooltips, clarified audio modes and seed rerolls, and added a
|
| 1264 |
+
single-image I2VA example plus First-Scene Image Gate.
|
| 1265 |
+
|
| 1266 |
+
## v0.3.11 — Invisible legacy widget-width repair
|
| 1267 |
+
|
| 1268 |
+
While a Context Loop node is on the canvas, the pack repairs the LiteGraph
|
| 1269 |
+
widget-width regression across all nodes. Regenerated scenes retain previous
|
| 1270 |
+
segment and checkpoint revisions instead of deleting the superseded take.
|
| 1271 |
+
|
| 1272 |
+
## v0.3.10 — Scene-scheduled Ref2VA
|
| 1273 |
+
|
| 1274 |
+
Added chained picture, video, paired-video-audio, and standalone-audio
|
| 1275 |
+
references under stable `@tags`, with per-scene activation and compact native
|
| 1276 |
+
label numbering. A right-click converter migrates an already-wired core Ref2VA
|
| 1277 |
+
node.
|
| 1278 |
+
|
| 1279 |
+
## v0.3.8 — One-pass performance re-filming
|
| 1280 |
+
|
| 1281 |
+
Reference Video Prep converts native VIDEO or decoded IMAGE/AUDIO to exact
|
| 1282 |
+
24 fps Ref2VA input, copies its soundtrack without padding or time-stretching,
|
| 1283 |
+
and powers the experimental three-angle guitar workflow.
|
| 1284 |
+
|
| 1285 |
+
## v0.3.7 — Flexible video loaders
|
| 1286 |
+
|
| 1287 |
+
Existing Video Context accepts either native ComfyUI VIDEO or separate
|
| 1288 |
+
IMAGE + AUDIO + FPS outputs.
|
| 1289 |
+
|
| 1290 |
+
## v0.3.6 — Extend an existing video
|
| 1291 |
+
|
| 1292 |
+
A typed adapter turns decoded video and optional audio into scene 1 context,
|
| 1293 |
+
with optional normalized-source prepend for partial and final output.
|
| 1294 |
+
|
| 1295 |
+
## v0.3.5 — Native guides and portable assembly
|
| 1296 |
+
|
| 1297 |
+
Added automatic support for ComfyUI's native arbitrary-position AV guides,
|
| 1298 |
+
retained the guarded legacy path, and added PyAV fallback when `ffmpeg` is not
|
| 1299 |
+
available.
|
| 1300 |
+
|
| 1301 |
+
## v0.3.4 — Scene Prompt Editor
|
| 1302 |
+
|
| 1303 |
+
Added the synchronized large-format scene editor with navigation, reference and
|
| 1304 |
+
dialogue shortcuts, and adjustable type size.
|
| 1305 |
+
|
| 1306 |
+
## v0.3.3 — Reliable preview resizing
|
| 1307 |
+
|
| 1308 |
+
Review video sizing remains stable when the ComfyUI canvas is zoomed.
|
| 1309 |
+
|
| 1310 |
+
## v0.3.2 — Resizable review video
|
| 1311 |
+
|
| 1312 |
+
The bar beneath Review Gate's player adjusts preview height.
|
| 1313 |
+
|
| 1314 |
+
## v0.3.1 — Friendlier JSON defaults
|
| 1315 |
+
|
| 1316 |
+
Top-level `duration_seconds` and `steps` shorthand populate visual Plan defaults
|
| 1317 |
+
correctly.
|
| 1318 |
+
|
| 1319 |
+
## v0.3.0 — Archival PNG export
|
| 1320 |
+
|
| 1321 |
+
Saved scene checkpoints can be re-decoded into a continuous lossless PNG
|
| 1322 |
+
sequence without holding the complete production in RAM.
|
| 1323 |
+
|
| 1324 |
+
## v0.2.0 — Recovery, metadata, and compatibility
|
| 1325 |
+
|
| 1326 |
+
- Persisted each scene prompt, effective plan, workflow, and API prompt beside
|
| 1327 |
+
the rendered chain.
|
| 1328 |
+
- Added scene-range rendering, resumable review checkpoints, partial assembly,
|
| 1329 |
+
notification/timeout controls, and Firefox-safe Review Gate recovery.
|
| 1330 |
+
- Added guarded compatibility with H3-Multishot, SolAttn, Ref2VA, and upstream
|
| 1331 |
+
H3 Motion Context.
|
| 1332 |
+
- Added Comfy Registry publishing and a shorter project-focused README.
|
| 1333 |
+
|
| 1334 |
+
## v0.1.0 — The production loop takes shape
|
| 1335 |
+
|
| 1336 |
+
- Introduced the visual scene-plan editor, multiline prompts, automatic scene
|
| 1337 |
+
colors, responsive layout, and collapsible raw JSON.
|
| 1338 |
+
- Added the recursive one-body chain, frame-locked audio trimming, per-scene
|
| 1339 |
+
checkpoints, interactive review/retry, and looping Ref2VA example.
|
| 1340 |
+
- Renamed the expanded project **MiniMax H3 Context Loop** so it could coexist
|
| 1341 |
+
clearly with NikoDemon80's manual Motion Context tools.
|
| 1342 |
+
|
| 1343 |
+
## Origins — Motion Context and Ref2VA continuation
|
| 1344 |
+
|
| 1345 |
+
The project began with MiniMax H3 clip chaining and generated-audio
|
| 1346 |
+
continuation, then added Ref2VA Motion Context, opt-in compatibility patches,
|
| 1347 |
+
and the resumable disk-backed production loop.
|
ComfyUI-MiniMaxH3-Context-Loop/CONTRIBUTING.md
ADDED
|
@@ -0,0 +1,75 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Contributing
|
| 2 |
+
|
| 3 |
+
Focused fixes, workflow improvements, compatibility updates, and documentation
|
| 4 |
+
corrections are welcome. Open an issue before a large redesign so its runtime
|
| 5 |
+
and saved-workflow compatibility can be discussed first.
|
| 6 |
+
|
| 7 |
+
## Keep version 0.5 compatible
|
| 8 |
+
|
| 9 |
+
Version 0.5 preserves saved 0.4 workflows and checkpoints. Unless a change is
|
| 10 |
+
explicitly planned as a breaking release:
|
| 11 |
+
|
| 12 |
+
- do not reuse or rename an existing public node class ID;
|
| 13 |
+
- do not reorder existing positional inputs or outputs;
|
| 14 |
+
- append new optional inputs and outputs instead of inserting them;
|
| 15 |
+
- retain readable legacy widget values and checkpoint formats;
|
| 16 |
+
- keep compatible H3-Multishot, SolAttn, and shared patch markers intact.
|
| 17 |
+
|
| 18 |
+
The frozen rules are documented in
|
| 19 |
+
[Version 0.5 architecture](docs/V0_5_ARCHITECTURE.md) and enforced by
|
| 20 |
+
`tests/fixtures/v0_4_public_contract.json`.
|
| 21 |
+
|
| 22 |
+
## Document behavior from evidence
|
| 23 |
+
|
| 24 |
+
Use the implementation, tests, maintained workflows, and upstream source as
|
| 25 |
+
the authority. A user-facing claim should point to a working example or a
|
| 26 |
+
specific implementation path. Mark experiments as experimental and describe
|
| 27 |
+
their fallback or compatibility behavior.
|
| 28 |
+
|
| 29 |
+
When a change is derived from another project, update
|
| 30 |
+
[Feature traceability](docs/FEATURE_TRACEABILITY.md) and
|
| 31 |
+
[Third-party notices](THIRD_PARTY_NOTICES.md) together. Record:
|
| 32 |
+
|
| 33 |
+
1. the upstream repository and author;
|
| 34 |
+
2. the revision or pull request used;
|
| 35 |
+
3. the upstream license;
|
| 36 |
+
4. the local files that implement the feature;
|
| 37 |
+
5. whether the relationship is **adapted**, **inspired**, **integrated**, or
|
| 38 |
+
**compatibility-only**.
|
| 39 |
+
|
| 40 |
+
Do not describe an integration with an upstream API as copied code, and do not
|
| 41 |
+
describe an adaptation as an original implementation.
|
| 42 |
+
|
| 43 |
+
## Validate a change
|
| 44 |
+
|
| 45 |
+
Run the focused test for the area you changed. Before release or a broad pull
|
| 46 |
+
request, run at least:
|
| 47 |
+
|
| 48 |
+
```bash
|
| 49 |
+
python tests/_node_smoke_test.py
|
| 50 |
+
python tests/_chain_smoke_test.py
|
| 51 |
+
python tests/_workflow_catalog_unit_test.py
|
| 52 |
+
python tests/_v05_contract_unit_test.py
|
| 53 |
+
```
|
| 54 |
+
|
| 55 |
+
Masking, source-timeline, frontend, and migration changes have additional
|
| 56 |
+
targeted scripts under `tests/`. JavaScript tests use Node directly, for
|
| 57 |
+
example:
|
| 58 |
+
|
| 59 |
+
```bash
|
| 60 |
+
node tests/_plan_editor_js_test.mjs
|
| 61 |
+
```
|
| 62 |
+
|
| 63 |
+
Tests intentionally use mock ComfyUI modules where possible. Real generation
|
| 64 |
+
still needs a current ComfyUI installation, H3 models, and appropriate media.
|
| 65 |
+
|
| 66 |
+
## Keep the README approachable
|
| 67 |
+
|
| 68 |
+
The README is a workflow chooser and quick start. Put algorithms, complete
|
| 69 |
+
node settings, and recovery internals in the focused guides under `docs/`.
|
| 70 |
+
Prefer short tables, task-based headings, and links to deeper explanations.
|
| 71 |
+
|
| 72 |
+
When adding or retiring a maintained workflow, update
|
| 73 |
+
`example_workflows/README.md` and `tests/_workflow_catalog_unit_test.py` in the
|
| 74 |
+
same change. Keep prompt and media attribution beside the workflow that uses
|
| 75 |
+
it.
|
ComfyUI-MiniMaxH3-Context-Loop/ComfyUI-MAINodes/.github/FUNDING.yml
ADDED
|
@@ -0,0 +1 @@
|
|
|
|
|
|
|
| 1 |
+
ko_fi: matlowai
|
ComfyUI-MiniMaxH3-Context-Loop/ComfyUI-MAINodes/.github/workflows/nightly-publish.yml
ADDED
|
@@ -0,0 +1,97 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
name: Nightly Registry Publish
|
| 2 |
+
|
| 3 |
+
# Batches a day's commits into ONE registry version. Registry versions are
|
| 4 |
+
# immutable and permanent, so publishing per-push would leave users scrolling
|
| 5 |
+
# through a version per commit.
|
| 6 |
+
#
|
| 7 |
+
# Self-correcting: it asks the registry what is already published rather than
|
| 8 |
+
# assuming. If the version in pyproject.toml was never published, it ships that
|
| 9 |
+
# as-is (covers a hand-made release). If it was, and main has moved since, it
|
| 10 |
+
# bumps the patch and ships that. If main has not moved, it does nothing.
|
| 11 |
+
on:
|
| 12 |
+
schedule:
|
| 13 |
+
- cron: "0 9 * * *" # 09:00 UTC daily
|
| 14 |
+
workflow_dispatch:
|
| 15 |
+
inputs:
|
| 16 |
+
force:
|
| 17 |
+
description: "Publish even if main has not moved"
|
| 18 |
+
type: boolean
|
| 19 |
+
default: false
|
| 20 |
+
|
| 21 |
+
permissions:
|
| 22 |
+
contents: write
|
| 23 |
+
|
| 24 |
+
concurrency:
|
| 25 |
+
group: registry-publish
|
| 26 |
+
cancel-in-progress: false
|
| 27 |
+
|
| 28 |
+
jobs:
|
| 29 |
+
publish:
|
| 30 |
+
runs-on: ubuntu-latest
|
| 31 |
+
steps:
|
| 32 |
+
- uses: actions/checkout@v4
|
| 33 |
+
with: { fetch-depth: 0 }
|
| 34 |
+
|
| 35 |
+
- name: Decide whether to publish, and at what version
|
| 36 |
+
id: decide
|
| 37 |
+
run: |
|
| 38 |
+
set -euo pipefail
|
| 39 |
+
CUR=$(python3 -c "import tomllib;print(tomllib.load(open('pyproject.toml','rb'))['project']['version'])")
|
| 40 |
+
NAME=$(python3 -c "import tomllib;print(tomllib.load(open('pyproject.toml','rb'))['project']['name'])")
|
| 41 |
+
echo "pyproject version: $CUR"
|
| 42 |
+
|
| 43 |
+
PUBLISHED=$(curl -sf "https://api.comfy.org/nodes/$NAME/versions" \
|
| 44 |
+
| python3 -c "import json,sys;d=json.load(sys.stdin);v=d if isinstance(d,list) else d.get('versions',[]);print(' '.join(x.get('version','') for x in v))" || echo "")
|
| 45 |
+
echo "already published: [$PUBLISHED]"
|
| 46 |
+
|
| 47 |
+
LAST_TAG=$(git tag --list 'published-*' --sort=-v:refname | head -1)
|
| 48 |
+
MOVED=1
|
| 49 |
+
if [ -n "$LAST_TAG" ] && [ "$(git rev-list -n1 "$LAST_TAG")" = "$(git rev-parse HEAD)" ]; then
|
| 50 |
+
MOVED=0
|
| 51 |
+
fi
|
| 52 |
+
echo "last published tag: ${LAST_TAG:-none} | main moved: $MOVED"
|
| 53 |
+
|
| 54 |
+
if ! echo " $PUBLISHED " | grep -q " $CUR "; then
|
| 55 |
+
echo "version=$CUR" >> "$GITHUB_OUTPUT"; echo "bump=no" >> "$GITHUB_OUTPUT"
|
| 56 |
+
echo "go=yes" >> "$GITHUB_OUTPUT"
|
| 57 |
+
echo "-> $CUR is not on the registry yet; publishing it as-is"
|
| 58 |
+
elif [ "$MOVED" = "1" ] || [ "${{ inputs.force }}" = "true" ]; then
|
| 59 |
+
NEXT=$(python3 -c "p='$CUR'.split('.');p[-1]=str(int(p[-1])+1);print('.'.join(p))")
|
| 60 |
+
echo "version=$NEXT" >> "$GITHUB_OUTPUT"; echo "bump=yes" >> "$GITHUB_OUTPUT"
|
| 61 |
+
echo "go=yes" >> "$GITHUB_OUTPUT"
|
| 62 |
+
echo "-> main moved since ${LAST_TAG:-the last publish}; bumping $CUR -> $NEXT"
|
| 63 |
+
else
|
| 64 |
+
echo "go=no" >> "$GITHUB_OUTPUT"
|
| 65 |
+
echo "-> nothing to do: $CUR is published and main has not moved"
|
| 66 |
+
fi
|
| 67 |
+
|
| 68 |
+
- name: Bump the patch version
|
| 69 |
+
if: steps.decide.outputs.go == 'yes' && steps.decide.outputs.bump == 'yes'
|
| 70 |
+
run: |
|
| 71 |
+
set -euo pipefail
|
| 72 |
+
python3 - <<'PY'
|
| 73 |
+
import re, os
|
| 74 |
+
v = os.environ["V"]
|
| 75 |
+
s = open("pyproject.toml").read()
|
| 76 |
+
s = re.sub(r'(?m)^version\s*=\s*".*"$', f'version = "{v}"', s, count=1)
|
| 77 |
+
open("pyproject.toml", "w").write(s)
|
| 78 |
+
PY
|
| 79 |
+
git config user.name "github-actions[bot]"
|
| 80 |
+
git config user.email "41898282+github-actions[bot]@users.noreply.github.com"
|
| 81 |
+
git commit -qam "release: v${{ steps.decide.outputs.version }}"
|
| 82 |
+
git push -q origin HEAD:main
|
| 83 |
+
env:
|
| 84 |
+
V: ${{ steps.decide.outputs.version }}
|
| 85 |
+
|
| 86 |
+
- name: Publish
|
| 87 |
+
if: steps.decide.outputs.go == 'yes'
|
| 88 |
+
uses: Comfy-Org/publish-node-action@main
|
| 89 |
+
with:
|
| 90 |
+
personal_access_token: ${{ secrets.COMFY_REGISTRY }}
|
| 91 |
+
skip_checkout: 'true'
|
| 92 |
+
|
| 93 |
+
- name: Mark what was published
|
| 94 |
+
if: steps.decide.outputs.go == 'yes'
|
| 95 |
+
run: |
|
| 96 |
+
git tag -f "published-${{ steps.decide.outputs.version }}"
|
| 97 |
+
git push -qf origin "published-${{ steps.decide.outputs.version }}"
|
ComfyUI-MiniMaxH3-Context-Loop/ComfyUI-MAINodes/.github/workflows/publish-node.yml
ADDED
|
@@ -0,0 +1,22 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
name: Publish to Comfy Registry
|
| 2 |
+
|
| 3 |
+
# Publishes when pyproject.toml changes on main (i.e. a version bump), and on
|
| 4 |
+
# demand. The registry rejects a re-publish of an existing version, so the
|
| 5 |
+
# version field is the release switch.
|
| 6 |
+
on:
|
| 7 |
+
push:
|
| 8 |
+
branches: [main]
|
| 9 |
+
paths: ["pyproject.toml"]
|
| 10 |
+
workflow_dispatch:
|
| 11 |
+
|
| 12 |
+
permissions:
|
| 13 |
+
contents: read
|
| 14 |
+
|
| 15 |
+
jobs:
|
| 16 |
+
publish:
|
| 17 |
+
runs-on: ubuntu-latest
|
| 18 |
+
steps:
|
| 19 |
+
- name: Publish custom node # the action checks out for itself
|
| 20 |
+
uses: Comfy-Org/publish-node-action@main
|
| 21 |
+
with:
|
| 22 |
+
personal_access_token: ${{ secrets.COMFY_REGISTRY }}
|
ComfyUI-MiniMaxH3-Context-Loop/ComfyUI-MAINodes/.gitignore
ADDED
|
@@ -0,0 +1,2 @@
|
|
|
|
|
|
|
|
|
|
| 1 |
+
__pycache__/
|
| 2 |
+
*.pyc
|
ComfyUI-MiniMaxH3-Context-Loop/ComfyUI-MAINodes/ALPHA.md
ADDED
|
@@ -0,0 +1,294 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# What is alpha here, and how alpha is it
|
| 2 |
+
|
| 3 |
+
This pack ships finished work and unfinished work in the same install. The
|
| 4 |
+
finished work is the Motion Lab de-rope pipeline and the contact sheet; it
|
| 5 |
+
has been measured, it has example graphs, and its dials have numbers behind
|
| 6 |
+
them in TUNING.md. Everything on this page is the other kind.
|
| 7 |
+
|
| 8 |
+
Alpha here means one or more of: it has run on one machine and no others,
|
| 9 |
+
the interesting half of it is not built, its interface will change, or it
|
| 10 |
+
is a real capability that simply has not been tested on enough material to
|
| 11 |
+
know where it stops working. Each entry below says which.
|
| 12 |
+
|
| 13 |
+
Nothing on this page changes existing behaviour or defaults. Alpha nodes
|
| 14 |
+
carry `(alpha)` in the name you see in the ComfyUI menu, and the subsystems
|
| 15 |
+
load behind a guarded loader, so a broken alpha module cannot take the rest
|
| 16 |
+
of the pack down with it.
|
| 17 |
+
|
| 18 |
+
If you find something wrong, an issue with the symptom and the graph is
|
| 19 |
+
worth more than a fix. Symptom-and-dial pairs belong in TUNING.md.
|
| 20 |
+
|
| 21 |
+
---
|
| 22 |
+
|
| 23 |
+
## Concept Lab
|
| 24 |
+
|
| 25 |
+
`concept_lab/`, and the nodes `MAI Concept Capture Arm / Flush` and
|
| 26 |
+
`MAI Concept Inject Delta`.
|
| 27 |
+
|
| 28 |
+
A research subsystem on the bet that a reusable concept does not have to
|
| 29 |
+
live in trained weights and can instead live in a measured functional
|
| 30 |
+
delta: measure what a piece of conditioning actually does to the model,
|
| 31 |
+
factor that into reusable components, compile them back through the model's
|
| 32 |
+
own conditioning channels.
|
| 33 |
+
|
| 34 |
+
**State: the interesting half is not built.** What exists is the data
|
| 35 |
+
layer (contracts, workspace, verbs, three surfaces over them) and an H3
|
| 36 |
+
capture tap that rides along on a render. What does not exist is anything
|
| 37 |
+
that turns a capture into a factor, or a factor into conditioning. Unbuilt
|
| 38 |
+
verbs raise and name the task that blocks them rather than returning an
|
| 39 |
+
empty result, because in a subsystem whose output is evidence a quiet
|
| 40 |
+
nothing is worse than a refusal.
|
| 41 |
+
|
| 42 |
+
Full status table, the layering rules, and a Contributing section naming
|
| 43 |
+
the four things most worth receiving: **`concept_lab/README.md`**. The
|
| 44 |
+
design decisions and their reasoning are in `concept_lab/DECISIONS.md`.
|
| 45 |
+
|
| 46 |
+
## Extension (long clips from short renders)
|
| 47 |
+
|
| 48 |
+
`h3_extend.py`: `H3 Extension Plan`, `H3 Tail Context`, `H3 Protect Prefix`,
|
| 49 |
+
`H3 Prefix Freeze Mask`, `H3 Trim`, `H3 Seam Normalize`; the shared types
|
| 50 |
+
in `capsule_types.py`; the graph `examples/motion_pipeline_extend_api.json`;
|
| 51 |
+
edge-protection dials on `H3 Jerk Oracle` (`protect_tail`) and
|
| 52 |
+
`H3 Window Plan` (`edge_protect`).
|
| 53 |
+
|
| 54 |
+
Short segments generated and de-roped one at a time, the last 39 frames
|
| 55 |
+
carried into the next segment, the overlap trimmed at assembly. Integer
|
| 56 |
+
time throughout: the 141/39 atom adds exactly 102 frames and 170 audio
|
| 57 |
+
ticks per segment. On a core with per-token masks (#15375) the handle is
|
| 58 |
+
WRITTEN INTO the next segment's own pass-1 latent under a time-varying
|
| 59 |
+
mask, with the audio handle on an audio-only `MiniMaxH3AddGuide`; on older
|
| 60 |
+
cores the whole handle rides the guide. The de-rope holds the prefix at 1
|
| 61 |
+
AND freezes it in pass 2; a non-final segment's last 17 frames are also
|
| 62 |
+
held at 1, because a gesture that runs into the cut has no "after" to slow
|
| 63 |
+
into and comes back fast otherwise. `H3 Seam Normalize` fits per-channel
|
| 64 |
+
linear-light gains on the hidden prefix (each VAE round trip darkens it
|
| 65 |
+
~2.4%) and applies them to the new material; its audio rms gain measured
|
| 66 |
+
NEGATIVE and defaults off.
|
| 67 |
+
|
| 68 |
+
**State: measured on two content sets, one machine, two segments.** The
|
| 69 |
+
masked path beats the image guide everywhere we measured: join jerk 0.86x
|
| 70 |
+
to 1.1x the clip's ordinary frame-to-frame motion vs 5x to 6.5x for the
|
| 71 |
+
guide, camera velocity continuous through the join instead of reversing,
|
| 72 |
+
handle within 2.2 to 3.3/255 of the carried tail, ~20% less wall (no
|
| 73 |
+
guide rows in every block). The closing-gesture fix measured 1.55x -> 1.06x.
|
| 74 |
+
Unmeasured: more than two segments (the drift curve), the 192/90 atom,
|
| 75 |
+
lower inject on continuation segments, dialogue across a join, and the
|
| 76 |
+
audio ambience bed, which still steps at the cut (a per-tick audio mask
|
| 77 |
+
is the planned fix). The API graph is the only form shipped so far.
|
| 78 |
+
|
| 79 |
+
## The timeline surface
|
| 80 |
+
|
| 81 |
+
Nodes: `H3 Drawn Plan`, `H3 Plan Settings`, `H3 Plan Estimate`,
|
| 82 |
+
`H3 Timeline Analyze`, `H3 Timeline Render`, `H3 Flight Recorder Start` and
|
| 83 |
+
`Stop`.
|
| 84 |
+
|
| 85 |
+
A plan document as the single source of truth: something proposes a plan,
|
| 86 |
+
a human edits it, a compiler turns it into a legal graph. The node surface
|
| 87 |
+
reaches parity with the compiler route, and the splice is proven on tensors.
|
| 88 |
+
|
| 89 |
+
**State: real but young.** The interface is expected to move, and the
|
| 90 |
+
editing experience around it is not built. If you are looking for the
|
| 91 |
+
production route, use the ordinary de-rope graphs.
|
| 92 |
+
|
| 93 |
+
## The audio init for dialogue
|
| 94 |
+
|
| 95 |
+
`H3 Audio Smear`, plus `audio_latent` / `audio_mode` on `H3 V2V Init` and
|
| 96 |
+
`audio_source` on `H3 Audio Recover`.
|
| 97 |
+
|
| 98 |
+
Fixes a real defect: a de-rope breaks speech, because the picture gets an
|
| 99 |
+
init and obeys it while the audio starts from zeros, so pass 2 writes a
|
| 100 |
+
fresh performance at natural rate and moves the mouth to that. Seeding the
|
| 101 |
+
audio rows with the baseline performance stretched onto the same dilated
|
| 102 |
+
clock makes pass 2 render a genuinely slowed take. The README's
|
| 103 |
+
"Dialogue through a de-rope" section has the adoption steps.
|
| 104 |
+
|
| 105 |
+
**State: it works, on material we have not varied enough.** The geometry
|
| 106 |
+
is solid: the smear/recover round trip is sample-exact in both directions
|
| 107 |
+
with envelope correlation 0.975, and the init costs nothing measurable in
|
| 108 |
+
picture quality. But it has been heard on a handful of clips, all
|
| 109 |
+
two-speaker sword-fight material at hold factors around 4 and dilations
|
| 110 |
+
between 2.4x and 2.7x. A single speaker, speech over music, non-anime
|
| 111 |
+
footage, and other hold factors are all unmeasured. If you try one of
|
| 112 |
+
those, that result is worth reporting whichever way it goes.
|
| 113 |
+
|
| 114 |
+
## The motion adapter (pilot)
|
| 115 |
+
|
| 116 |
+
A rank-16 LoRA trained on the de-rope task itself, applied to the de-rope
|
| 117 |
+
pass only. Documented with its measured settings and its known costs in the
|
| 118 |
+
README's "The motion adapter (pilot)" section, and released as an
|
| 119 |
+
intermediate option rather than a finished one. A more ambitious all-in-one
|
| 120 |
+
adapter is in progress and may not work.
|
| 121 |
+
|
| 122 |
+
## Manual mask paths
|
| 123 |
+
|
| 124 |
+
The `(alpha)`-tagged inputs on `H3 V2V Init` and `H3 Motion Composite`:
|
| 125 |
+
manual region masks, final-alpha masks, and the time-varying mask path.
|
| 126 |
+
These are exercised by the checklist in **`TESTING_ALPHA.md`**, which
|
| 127 |
+
covers the 2026-08-09 node batch and still wants human passes on the
|
| 128 |
+
interactive widget and the drag-in workflows.
|
| 129 |
+
|
| 130 |
+
---
|
| 131 |
+
|
| 132 |
+
## VRAM Lab
|
| 133 |
+
|
| 134 |
+
`H3 Streamed Blocks`, `H3 Memory Probe`, `H3 Free Cache`, `H3 Evict Text
|
| 135 |
+
Encoder` (`vram_lab.py`, 2026-08-18). Alpha because it has run on one
|
| 136 |
+
machine (RTX PRO 6000 Blackwell, 188 SMs) and its exactness is a kernel
|
| 137 |
+
property, not a code property: query chunking is bit-equal only while every
|
| 138 |
+
chunk keeps PyTorch's flash kernel off its split-KV path (measured boundary
|
| 139 |
+
`heads x ceil(L/64) >= 0.8 x 2 x SMs`; the node sizes chunks from the SM
|
| 140 |
+
count with a 2.6x margin and carries a `self_check` that compares stock and
|
| 141 |
+
streamed on the first block's real input). int8 and W4A8 checkpoints are
|
| 142 |
+
bit-equal by mechanism (row-wise activation scales, int32 accumulate);
|
| 143 |
+
NVFP4/FP8 would need one shared activation scale and are untested; bf16 is
|
| 144 |
+
numerically equivalent, not identical. `kv_block` is experimental and as
|
| 145 |
+
built does not lower memory (leave it at 0). The output head has an exact
|
| 146 |
+
mode (default) and a chunked-GEMM mode that changes the clip after 25 steps
|
| 147 |
+
by ~1e-6 per step of fp32 difference; that mode is labelled and off.
|
| 148 |
+
|
| 149 |
+
What was measured, one graph (294 f -> 702 f de-rope at 1376x768, ~217k
|
| 150 |
+
tokens): torch activations peak +11.9 GiB over the weight floor at any
|
| 151 |
+
card; a 16 GB card in a 32 GB machine renders it with ComfyUI's dynamic VRAM
|
| 152 |
+
and `--fast-disk` at 316 s/step; a 96 GB card resident is 311 s/step. RAM is
|
| 153 |
+
the small-machine ceiling in normal mode (CPU copies of the models without
|
| 154 |
+
`--fast-disk`, and CPU-side IMAGE intermediates); the text encoder costs
|
| 155 |
+
1 to 2.5 GiB of peak on a 16 GB card and about 3 s per new prompt, and
|
| 156 |
+
either eviction or the conditioning bank removes it. Untested: other GPUs,
|
| 157 |
+
Windows, ROCm, non-flash attention backends (cuDNN and mem-efficient are
|
| 158 |
+
chunk-invariant at every length here but 1.1 to 2x slower; no two backends
|
| 159 |
+
are bit-equal to each other).
|
| 160 |
+
|
| 161 |
+
The `kv_store` options (`kvi8r`, `kvi8s`) are the approximation tier of the
|
| 162 |
+
same node: half the K/V bytes (kvi8r measured +9.5 GiB forward peak against
|
| 163 |
+
the exact +11.9 at 217k tokens; kvi8s attends on int8/fp8 tensor cores
|
| 164 |
+
straight from the store, ~1.6x faster attention standalone), same-seed
|
| 165 |
+
renders are sibling takes, and only kvi8r's first cut has been in front of
|
| 166 |
+
eyes ("almost perfect" on the de-rope side by side, one clip, one viewer).
|
| 167 |
+
The node defaults to the exact store; the low-VRAM example graph turns kvi8r
|
| 168 |
+
on (operator's call for that graph); the table with the numbers and what is
|
| 169 |
+
still owed is in LOWVRAM.md. The int8_convrot video VAE now in the example graph is also an
|
| 170 |
+
approximation (60 dB PSNR against the fp16 decoder on the same latent).
|
| 171 |
+
|
| 172 |
+
|
| 173 |
+
## Time burst (examples/experimental/time_burst_api.json)
|
| 174 |
+
|
| 175 |
+
Alpha because: one clip, one seed, one machine, and the effect is a defect
|
| 176 |
+
worn as a feature. The windowed insert with noise-born in-betweens and no
|
| 177 |
+
guide has no clock inside the window, so it rushes the span (about 2.7x the
|
| 178 |
+
source speed on the shipped clip) and invents flares. The graph is the exact
|
| 179 |
+
one that made `assets/time_burst_result.mp4` from `assets/time_burst_plate.mp4`,
|
| 180 |
+
so the first run is a reproduction, not an experiment. What is not built:
|
| 181 |
+
any control over how hard it rushes beyond the hold map and inject; a
|
| 182 |
+
second clip; a null pair. If you want the ordinary insert, start from the
|
| 183 |
+
motion pipeline graphs instead.
|
| 184 |
+
|
| 185 |
+
## Where to look next
|
| 186 |
+
|
| 187 |
+
| | |
|
| 188 |
+
|---|---|
|
| 189 |
+
| `concept_lab/README.md` | Concept Lab status, layering, contributing |
|
| 190 |
+
| `TESTING_ALPHA.md` | hands-on checklist for the manual mask and editor batch |
|
| 191 |
+
| `TUNING.md` | dials with measured numbers, for the finished pipeline |
|
| 192 |
+
| `ROADMAP.md` | where the unfinished parts are meant to go |
|
| 193 |
+
|
| 194 |
+
---
|
| 195 |
+
|
| 196 |
+
## Repair verb
|
| 197 |
+
|
| 198 |
+
`h3_repair.py`, nodes `H3 Repair Plan (alpha)` and `H3 Repair Splice (alpha)`.
|
| 199 |
+
|
| 200 |
+
Type a bad frame range; the plan snaps it outward to whole tokens, runs the
|
| 201 |
+
regeneration through the nearest shot cut, and the splice puts every
|
| 202 |
+
untouched pixel back bit-exact, printing its seam numbers. Validated on one
|
| 203 |
+
cell and one in-graph rerun (entry seam 0.26x the clip's median frame delta,
|
| 204 |
+
exit ON the source's own cut within 1.8%). Alpha because: one scene family
|
| 205 |
+
so far, and AUDIO IS STILL A WORK IN PROGRESS: the splice passes audio
|
| 206 |
+
through untouched, which is only right when the repair changed no performance.
|
| 207 |
+
|
| 208 |
+
## DyRoPE
|
| 209 |
+
|
| 210 |
+
`motion.py`, node `H3 DyRoPE (alpha)`.
|
| 211 |
+
|
| 212 |
+
An instrument that hands the de-rope's two clock geometries (true world time
|
| 213 |
+
vs the trained grid) to different transformer blocks or different denoise
|
| 214 |
+
steps. The dose map is measured and in TUNING.md: two settings keep the
|
| 215 |
+
world-time correction, everything below them loses it, everything above adds
|
| 216 |
+
shimmer. Alpha because: one scene measured deeply, everything else untested.
|
| 217 |
+
|
| 218 |
+
## Audio prefix freeze
|
| 219 |
+
|
| 220 |
+
`motion.py`, inputs `audio_prefix_ticks` and `audio_prefix_release_ticks` on
|
| 221 |
+
`H3 V2V Init`.
|
| 222 |
+
|
| 223 |
+
The audio twin of the video prefix freeze, with a half-cosine release.
|
| 224 |
+
Bench-tested on chained music (overlap correlation ~0.92 in the full
|
| 225 |
+
chain, 0.98 on the pass-1-only control; beat phase 0.0 ms). It carries local continuity (level, timbre, beat),
|
| 226 |
+
not a track's global phrase structure. Alpha because: the caveat list in the
|
| 227 |
+
README section is load-bearing; read it before wiring.
|
| 228 |
+
|
| 229 |
+
## Drift Control
|
| 230 |
+
|
| 231 |
+
`h3_drift.py`, node `H3 Drift Control (alpha)`.
|
| 232 |
+
|
| 233 |
+
Schedule-matched noise on a carried video prefix so a chain's later joins
|
| 234 |
+
stop drifting. Second-join delta 0.680 to 0.860 with the first join
|
| 235 |
+
unchanged; replicated across two chains and new seeds; joins hold 0.85-0.89 through link 4 (the earlier chain sat in the 0.65 class at its second
|
| 236 |
+
join before the recipe; cross-chain comparison, not a paired arm). Refuses to stack with
|
| 237 |
+
any other dynamic denoise-mask patch; sigma-split samplers unsupported.
|
| 238 |
+
|
| 239 |
+
## Color Carry
|
| 240 |
+
|
| 241 |
+
`h3_color_carry.py`, nodes `H3 Delta Color Carry (alpha)` and
|
| 242 |
+
`H3 Scene Color Stats (alpha)`.
|
| 243 |
+
|
| 244 |
+
Cancels the VAE round-trip color bias on a carried prefix by adding only
|
| 245 |
+
E(corrected) minus E(original) to the latent, so the encode bias cancels and
|
| 246 |
+
the weak scene-one grade survives. Active path exercised on a real 4-link render 2026-08-24: it fires, corrects in the anchor direction, and stays sub-visible under its clamps on mildly-drifted content; on the deepest tested link it fires on both channels at or near its clamp. A strong-drift bench is still owed.
|
| 247 |
+
|
| 248 |
+
## Video Compare viewer (upgraded player)
|
| 249 |
+
|
| 250 |
+
`video_compare.py` + `web/video_compare.js`. The review player grew a
|
| 251 |
+
timeline, loop brackets, waveform, blips and a frame-exact export this
|
| 252 |
+
release. KNOWN ISSUES, alpha honesty: multi-video playback inside the
|
| 253 |
+
ComfyUI node surface does not hold sync reliably (browser video elements
|
| 254 |
+
each run their own clock; every correction strategy trades stutter for
|
| 255 |
+
offset), and the transport can wedge after scrubbing during playback -
|
| 256 |
+
re-queue or reload the page to recover. The standalone deck pages built
|
| 257 |
+
by `tools/compare_deck/` do not share these problems. Two more, both about
|
| 258 |
+
audio: in clock sync mode audio follows only the clock-owning side, so
|
| 259 |
+
locking audio to the other pane is silent until sync mode is flipped; and
|
| 260 |
+
the realtime export records audio from the locked side even when it is not
|
| 261 |
+
one of the two playing panes, so prefer the precise export. Planned fix,
|
| 262 |
+
next iteration: the node bakes each pair into ONE combined preview stream
|
| 263 |
+
and every view mode becomes a crop of it - one decoder, one clock, sync by
|
| 264 |
+
construction. Deferred with it: the player is duplicated (~330 lines)
|
| 265 |
+
between `web/video_compare.js` and the deck template, and a single-source
|
| 266 |
+
refactor is the next iteration's job (the deck's export bitrate selector
|
| 267 |
+
having been honored on only one of the two export paths was the first bite
|
| 268 |
+
of that divergence).
|
| 269 |
+
|
| 270 |
+
## Mid-denoise insert
|
| 271 |
+
|
| 272 |
+
`motion.py`, node `H3 MidInsert (alpha)`.
|
| 273 |
+
|
| 274 |
+
Inserts time-dilation tokens into a still-noisy latent at a handoff sigma
|
| 275 |
+
instead of between passes, with a measured variance top-up. Kept for
|
| 276 |
+
research honesty: at its one tested operating point (handoff sigma 0.53,
|
| 277 |
+
four dense steps) it breaks clip continuity in playback, and it is
|
| 278 |
+
documented as that negative rather than removed. Do not reach for it in
|
| 279 |
+
production; the two-pass insertion path is the shipped construction.
|
| 280 |
+
|
| 281 |
+
## VRAM lab instruments
|
| 282 |
+
|
| 283 |
+
`vram_lab.py`, node `H3 FakeQuant (alpha)`. `H3 Sol Attention` is in the
|
| 284 |
+
code but NOT registered this release.
|
| 285 |
+
|
| 286 |
+
Measurement instruments from the quantization review work: FakeQuant
|
| 287 |
+
fake-quantizes a chosen region of the model's activations so a precision
|
| 288 |
+
regime can be auditioned without real kernels; Sol Attention switches the
|
| 289 |
+
attention path for the same kind of audition. Instruments, not
|
| 290 |
+
accelerators: they exist to make honest comparison pages, and their
|
| 291 |
+
numbers only mean something next to a reference arm. FakeQuant ships;
|
| 292 |
+
Sol Attention stays unregistered until it gets the DyRoPE arm/disarm
|
| 293 |
+
pattern, because it rebinds attention process-globally with no restore
|
| 294 |
+
path.
|
ComfyUI-MiniMaxH3-Context-Loop/ComfyUI-MAINodes/DEROPE_ANY_MODEL.md
ADDED
|
@@ -0,0 +1,156 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# De-rope for any model (branch `generic-derope`, not shipped)
|
| 2 |
+
|
| 3 |
+
This branch carries two things: the LTX-2.5 graphs and tools behind the public
|
| 4 |
+
de-rope deck (https://matlowai.github.io/flipbook/derope.html), and the
|
| 5 |
+
any-model layer that lets one hold map drive any regenerating model through
|
| 6 |
+
per-model presets. It is a branch on purpose: what ships on `main` is the H3
|
| 7 |
+
Motion Lab; this merges once the oracle-shaped LTX arm has been measured.
|
| 8 |
+
|
| 9 |
+
## What it does today: a FLAT clock
|
| 10 |
+
|
| 11 |
+
An H3 de-rope (`H3JerkOracle` -> `H3TimeSmear` -> regenerate -> `H3ExactRecover`)
|
| 12 |
+
slows each frame by its own hold (1 to 4, with ramps and bridges), which is what
|
| 13 |
+
makes the regenerated motion land on the right frames. The LTX-2.5 graphs here
|
| 14 |
+
use the H3 oracle only to choose WHERE the window sits (its hot span), and inside
|
| 15 |
+
the window every frame is held exactly d times (`ImageBatchRepeatInterleaving`,
|
| 16 |
+
d = 8 or 16), padded to LTX's legal length (8k+1), regenerated from a given
|
| 17 |
+
starting sigma, and recovered by taking every d-th frame.
|
| 18 |
+
|
| 19 |
+
Why flat: LTX compresses time by 8, so a held frame only owns its own latent
|
| 20 |
+
block when its hold is a multiple of 8 frames. Holds of 1 to 4 share blocks, and
|
| 21 |
+
measured on two scenes that is exactly where d=8 barely responds to the denoise
|
| 22 |
+
dial (0.94 to 0.90 across 0.1 to 0.9) while d=16 moves it (0.90 to 0.28).
|
| 23 |
+
|
| 24 |
+
Per-pass cap: 57 source frames at d=16 (`positional_embedding_max_pos[0] = 20`
|
| 25 |
+
with time in absolute seconds). Full coverage of a longer clip is several
|
| 26 |
+
overlapping windows joined by `tools/ltx25/splice_windows.py`, which measures
|
| 27 |
+
PSNR across each overlap before it blends.
|
| 28 |
+
|
| 29 |
+
## The any-model layer (new nodes)
|
| 30 |
+
|
| 31 |
+
The smear, recover and audio nodes were already pixel-level and model-agnostic;
|
| 32 |
+
what was H3-specific was the grid law (17k+5 lengths, holds in H3 frames). That
|
| 33 |
+
law is now data, `model_profiles.py`, and one node applies it:
|
| 34 |
+
|
| 35 |
+
- **H3 Clock Remap (any model, presets)** - wire any hold map (H3 Jerk Oracle,
|
| 36 |
+
H3 Manual Hold Map, H3 Drawn Plan), pick the model that will regenerate the
|
| 37 |
+
frames. Holds of 2+ are scaled by the profile's `hold_scale` and quantised
|
| 38 |
+
to whole latent time blocks; holds of 1 stay 1 (plain video anywhere). The
|
| 39 |
+
output map carries the target's legal grid, so **H3 Time Smear pads to that
|
| 40 |
+
grid** (LTX-2.5: 8k+1, not 17k+5) and H3 Exact Recover / H3 Audio Recover
|
| 41 |
+
invert it unchanged. `minimax-h3` is the identity: existing graphs do not
|
| 42 |
+
change by construction (tests assert it). The report is the price tag: frames,
|
| 43 |
+
seconds at the model's fps, windows needed under its cap, and a loud
|
| 44 |
+
UNMEASURED line for any profile whose numbers did not come from a ladder.
|
| 45 |
+
- **H3 Save Hold Map / H3 Load Hold Map (sidecar)** - write `<prefix>.holdmap.json`
|
| 46 |
+
beside the render; read it back by prefix (newest wins) in another graph, so
|
| 47 |
+
the H3 pass decides the clock and an LTX-2.5 or Wan 2.2 graph regenerates on
|
| 48 |
+
it without both models loaded at once. Load hands back the ORIGINAL clock
|
| 49 |
+
(`source_holds`), never a previous target's grid. Without it a render's clock is lost when the graph closes; with it the
|
| 50 |
+
staircase ruler can read H3 arms and a later remap can retime the same plan.
|
| 51 |
+
- **H3 Jerk Oracle `profile_mode`: "value |d3| camera-compensated"** - aligns
|
| 52 |
+
each latent frame to its predecessor by the best small integer shift before
|
| 53 |
+
differencing, so a pan or scroll stops scoring as jerk (the documented cause
|
| 54 |
+
of panrun dilating 124 -> 345 frames). Same compiler, different signal.
|
| 55 |
+
|
| 56 |
+
- **H3 Jerk Oracle `model_profile`** - the oracle can read ANOTHER model's
|
| 57 |
+
video latent: pick the preset whose latent is wired in (LTX-2.5: 128
|
| 58 |
+
channels on a 1+8k token clock; Wan 2.2: 16 channels, 1+4k). Same planner,
|
| 59 |
+
that model's frame<->token mapping, no H3 phase normalisation, holds per
|
| 60 |
+
source frame as before. Wrong channel counts and wrong lengths are refused
|
| 61 |
+
with a message that says what was expected. `minimax-h3` is the shipped
|
| 62 |
+
path, bit-identical (tested). An independent port of the three motion nodes
|
| 63 |
+
onto LTX-2.3/2.5 latents appeared publicly in August 2026 and confirmed this
|
| 64 |
+
is the right seam: the oracle, the smear and the recover carried over with
|
| 65 |
+
only the grid law changed, which is exactly what the profile holds.
|
| 66 |
+
|
| 67 |
+
### Profiles
|
| 68 |
+
|
| 69 |
+
| id | block | hold_scale | legal | fps | cap | latent (channels, clock) | measured |
|
| 70 |
+
|---|---|---|---|---|---|---|---|
|
| 71 |
+
| `minimax-h3` | 1 | 1 | 17k+5 | 24 | VRAM | 24, (1,4,4,4,4) per 17 (H3 code path) | yes |
|
| 72 |
+
| `ltx-2.5` | 8 | 4 | 8k+1 | 48 (the conditioning frame_rate) | 20 s / pass = 960 frames | 128, 1+8k | yes (2026-08-21 ladder) |
|
| 73 |
+
| `wan-2.2 (unmeasured)` | 4 | 4 | 4k+1 | 16 | none | 16, 1+4k | NO - placeholder |
|
| 74 |
+
|
| 75 |
+
`hold_scale` is measured, never derived: LTX-2.5 needed two 8-frame blocks per
|
| 76 |
+
hot hold (d=16) before the dial responded at all, which its compression ratio
|
| 77 |
+
alone would not have told you.
|
| 78 |
+
|
| 79 |
+
### Adding a model without touching code
|
| 80 |
+
|
| 81 |
+
Pick `custom (use the fields below)` on the node and type the five numbers; the
|
| 82 |
+
report stamps the result unmeasured. When a row works, save it as a preset:
|
| 83 |
+
create `<ComfyUI user dir>/mainodes_models.json` (or point
|
| 84 |
+
`$MAINODES_MODELS_JSON` at a file) with
|
| 85 |
+
|
| 86 |
+
```json
|
| 87 |
+
{"my-model": {"name": "My Model", "block": 4, "hold_scale": 4,
|
| 88 |
+
"legal": [4, 1], "fps": 16, "cap_seconds": null, "measured": false,
|
| 89 |
+
"latent": {"channels": 16, "first": 1, "block": 4}}}
|
| 90 |
+
```
|
| 91 |
+
|
| 92 |
+
and restart: it appears in the dropdown, and a row with a preset's id
|
| 93 |
+
overrides that preset. Send us the row and the ladder behind it.
|
| 94 |
+
|
| 95 |
+
### First measurements (2026-08-21, one pass on two lab instances)
|
| 96 |
+
|
| 97 |
+
- H3 two-stage graph with the identity remap in the chain: **bit-identical**
|
| 98 |
+
to the production render (PSNR inf on baseline, dilated and recovered).
|
| 99 |
+
- LTX-2.5 on the camera-compensated H3 clock at x4: panrun needs a window
|
| 100 |
+
(the live clock is 1201 dilated frames at x4, over the 960 cap; a 98-frame
|
| 101 |
+
window fits at 951), swordspin fits whole (866). Hold-map staircase 0.864
|
| 102 |
+
(panrun) / 0.818 (swordspin) at starting sigma 0.4, against 0.80 / 0.78 for
|
| 103 |
+
the uniform d=16 windows: **the oracle-shaped clock does not fix LTX's weak
|
| 104 |
+
response at that denoise; the denoise is the limiter, not the clock.**
|
| 105 |
+
298 s / 247 s, peak 59-60 GiB.
|
| 106 |
+
- Wan 2.2, first ladder cell (x1, 21-source-frame window on the densest
|
| 107 |
+
hold-4 run, low-noise 14B expert, denoise 0.4, 640^2): staircase **0.693**
|
| 108 |
+
on swordspin (the lowest of the set), 0.877 on panrun's window; 119-126 s
|
| 109 |
+
including the model load, peak 33-52 GiB. The `wan-2.2` preset stays
|
| 110 |
+
unmeasured until the x1/x2 ladder settles its hold_scale; x4 cannot fit
|
| 111 |
+
Wan's 81-frame training length.
|
| 112 |
+
- Camera-compensated oracle: identical to the default on the static swordspin
|
| 113 |
+
shot; trims panrun's held span 80 -> 76 frames.
|
| 114 |
+
|
| 115 |
+
### Sizing the oracle-shaped LTX arm
|
| 116 |
+
|
| 117 |
+
On the saved oracle profiles the x4 rule gives panrun 712 / swordspin 816
|
| 118 |
+
dilated frames over the hot span, under the 960-frame cap (913 was the largest legal length used), so it fits one
|
| 119 |
+
pass at the same cost as the uniform d=16 window with the time budget spent
|
| 120 |
+
where the oracle put it. That arm is the one measurement left before this
|
| 121 |
+
merges. The x8 rule (hold 4 -> 32x) needs two windows per scene.
|
| 122 |
+
|
| 123 |
+
## Files
|
| 124 |
+
|
| 125 |
+
- `examples/ltx25/derope_any_model_ltx25_api.json` - the any-model path on
|
| 126 |
+
LTX-2.5: H3 Manual Hold Map (or an oracle) -> H3 Clock Remap (`ltx-2.5`) ->
|
| 127 |
+
H3 Time Smear -> the LTX-2.5 pass -> H3 Exact Recover, with H3 Save Hold Map
|
| 128 |
+
beside the render. API JSON (drag onto the canvas); the UI twin lands once
|
| 129 |
+
an instance with these nodes can convert it.
|
| 130 |
+
- `examples/ltx25/derope_any_model_h3_oracles_api.json` - the H3 base pass with
|
| 131 |
+
both oracle modes (default, camera-compensated), each saved as a sidecar, plus
|
| 132 |
+
the identity remap saved beside them for a bit-exact check.
|
| 133 |
+
- `examples/ltx25/derope_any_model_ltx25_fromsidecar_api.json` - the whole clip
|
| 134 |
+
on LTX-2.5 in ONE pass from the camera-compensated sidecar (panrun: 124 frames
|
| 135 |
+
-> ~785 dilated under the 960-frame cap, quiet frames at native rate).
|
| 136 |
+
- `examples/wan22/derope_any_model_wan22_api.json` - the same clock on Wan 2.2
|
| 137 |
+
(low-noise 14B expert, v2v from the smeared frames at denoise 0.4, 640^2,
|
| 138 |
+
16 fps, 4k+1 grid from the `wan-2.2 (unmeasured)` preset): the first ladder
|
| 139 |
+
cell that turns that preset's numbers into measured ones.
|
| 140 |
+
- `examples/ltx25/` - four graphs as run for the deck, API JSON (`*_api.json`,
|
| 141 |
+
what actually executed) and UI JSON (paste on the canvas):
|
| 142 |
+
`panrun_or_d16_d040` (uniform d=16 de-rope of the oracle window),
|
| 143 |
+
`swordspin_or_d8_d050` (d=8), `swordspin_or_x2_d8_d050_d030` (one-pass x2
|
| 144 |
+
latent upsample of a d=8 pass), `panrun_ltx_d040_full` (first of the three
|
| 145 |
+
windows behind the spliced full-length clip).
|
| 146 |
+
Source clips are referenced by basename: put yours in ComfyUI `input/`.
|
| 147 |
+
- `tools/ltx25/mint_ltx25_derope.py` - mints a uniform de-rope graph from the
|
| 148 |
+
stock LTX-2.5 T2V template (`--frames --fps --dilation --seed ...`).
|
| 149 |
+
- `tools/ltx25/mint_ltx25.py` - the plain LTX-2.5 graph minter the ladder used.
|
| 150 |
+
- `tools/ltx25/ltx25_oracle_window.py` - reads a saved H3 oracle profile and
|
| 151 |
+
returns the legal LTX window over its hot span.
|
| 152 |
+
- `tools/ltx25/splice_windows.py` - joins window renders, checking alignment.
|
| 153 |
+
- `tools/ltx25/ui2api.py` - the UI -> API converter the minters depend on.
|
| 154 |
+
|
| 155 |
+
Model files are the stock Comfy-Org LTX-2.5 set (22B transformer, its VAE and
|
| 156 |
+
audio VAE, the x2 latent upsampler) plus the MiniMax-H3 set for the H3 arms.
|
ComfyUI-MiniMaxH3-Context-Loop/ComfyUI-MAINodes/FASTER.md
ADDED
|
@@ -0,0 +1,49 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# FASTER.md, or: how to make it less awesome but still better than nothing
|
| 2 |
+
|
| 3 |
+
The Motion Lab defaults are tuned for quality. Every dial below trades some of
|
| 4 |
+
that quality for wall-clock time, in descending order of how much time it buys.
|
| 5 |
+
Numbers come from measured runs on the example scenes; your content will move
|
| 6 |
+
them, the ordering should hold. The plan report prices most of these before you
|
| 7 |
+
spend a second of GPU: queue the windowed graph with `window: 0`, read the
|
| 8 |
+
report, then decide.
|
| 9 |
+
|
| 10 |
+
## Where the time actually goes
|
| 11 |
+
|
| 12 |
+
A de-rope render is pass 1 (generate) plus pass 2 (regenerate the dilated
|
| 13 |
+
timeline at higher effective resolution). Pass 2 dominates, and its cost is
|
| 14 |
+
roughly quadratic in token count: tokens = frames x pixels. So the dials that
|
| 15 |
+
remove frames or pixels beat the dials that make steps cheaper.
|
| 16 |
+
|
| 17 |
+
## The dials, best value first
|
| 18 |
+
|
| 19 |
+
| dial | where | default | cheaper setting | what you give up |
|
| 20 |
+
|---|---|---|---|---|
|
| 21 |
+
| `dilation` / `d_max` | H3 Time Smear / H3 Jerk Oracle | 4 | 3 or 2 | the slow-motion budget on bursts; 2 still beats no de-rope by a wide margin |
|
| 22 |
+
| `q` | H3 Jerk Oracle | 0.75 | 0.85 | fewer frames count as bursts, so borderline motion stays baseline |
|
| 23 |
+
| `expand_to_end` | H3 Time Smear | on in some graphs | off | the span after the last burst stays baseline; fine when the tail is calm |
|
| 24 |
+
| `inject` | H3 Inject Schedule | 0.48 | keep 0.48 | it is already the cheap setting: only the steps below the inject point run. 0.7 is the deeper, slower variant |
|
| 25 |
+
| pass 2 resolution | the second ResolutionSelector | 1.5 MP | 1.0 MP | the upscale half of upscale-de-rope; tokens scale with pixels |
|
| 26 |
+
| `coverage` | H3 Window Plan | full clip | burst only | on NON-upscale graphs only: calm spans keep their baseline render. On upscale graphs leave it on full clip or calm spans stay low-res |
|
| 27 |
+
| `kv_store` | H3 Streamed Blocks | kvi8r | kvi8s | needs the sageattention package; measured 219 vs 318 s/step at long context, judged side by side as a keeper |
|
| 28 |
+
| `handle_frames` | H3 Window Plan | 12 | 8 | less duplicated context at window seams; the edge pins carry more of the tracking load |
|
| 29 |
+
| `steps` (pass 1) | BasicScheduler | 12 (fast flow) | 12 | going below 12 shows; save elsewhere first |
|
| 30 |
+
|
| 31 |
+
## What not to touch
|
| 32 |
+
|
| 33 |
+
- The window edge pins (`MiniMax H3 Add Guide` at the first and last frame).
|
| 34 |
+
They cost nothing measurable and they are what holds the regenerated windows
|
| 35 |
+
on the baseline's motion and look.
|
| 36 |
+
- The audio seed (`H3 Audio Smear` into `H3 V2V Init`). Also near-free, and
|
| 37 |
+
pass 2's foley tracks the original performance because of it.
|
| 38 |
+
- `max_dilated_frames` does not buy total time. Smaller windows lower the PEAK
|
| 39 |
+
cost so a smaller card can run at all; the total goes slightly up from
|
| 40 |
+
duplicated handles.
|
| 41 |
+
|
| 42 |
+
## A worked example
|
| 43 |
+
|
| 44 |
+
The rolling-window example scene (107 frames, 4.5 s) plans 243 dilated frames
|
| 45 |
+
at the defaults: 2.27x the frames and 4.0x the time per step of a plain pass.
|
| 46 |
+
Dropping `d_max` to 3 and `q` to 0.85 with `expand_to_end` off re-plans the
|
| 47 |
+
same clip in the low 100s of dilated frames; the burst keeps its slow-motion
|
| 48 |
+
budget and the calm frames stop paying for it. Queue with `window: 0` and the
|
| 49 |
+
report will show you the exact count for your clip before you commit.
|
ComfyUI-MiniMaxH3-Context-Loop/ComfyUI-MAINodes/HARDWARE.md
ADDED
|
@@ -0,0 +1,102 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Hardware vs settings
|
| 2 |
+
|
| 3 |
+
Which graph and which dials for the machine you have. Two numbers matter
|
| 4 |
+
and they are not the same number: **VRAM** decides whether the dilated
|
| 5 |
+
pass fits, **system RAM** decides whether the whole pipeline fits, because
|
| 6 |
+
the decoded frames of pass 1, the smeared frames and the recovered frames
|
| 7 |
+
all live on the host as float images. People with a 24 GB card and 32 GB
|
| 8 |
+
of RAM hit the RAM wall first.
|
| 9 |
+
|
| 10 |
+
Everything here was measured on one box (RTX PRO 6000 Blackwell 96 GB,
|
| 11 |
+
91 GB RAM) with the smaller cards simulated by fencing VRAM and RAM to a
|
| 12 |
+
hard budget, so a "16 GB" row means a real 16 GiB ceiling, desktop
|
| 13 |
+
included. Cells marked *extrapolated* come from the fitted line in
|
| 14 |
+
[`LOWVRAM.md`](LOWVRAM.md#how-far-you-can-go-extrapolated-from-the-lines-above)
|
| 15 |
+
and have not been rendered. If you run one of those, your seconds per step
|
| 16 |
+
is the most useful thing you can send back.
|
| 17 |
+
|
| 18 |
+
## Pick a row
|
| 19 |
+
|
| 20 |
+
| VRAM | RAM | start from | checkpoint | what to expect |
|
| 21 |
+
|---|---|---|---|---|
|
| 22 |
+
| 96 GB | 64 GB+ | [`examples/motion_pipeline_ref2va_audioinit.json`](examples/motion_pipeline_ref2va_audioinit.json) | int8 | 192 frames at 1 MP in ~12 min; a 702-frame de-rope at 1376x768 renders with everything resident |
|
| 23 |
+
| 32 GB | 64 GB | the same, or [`motion_pipeline_lowvram.json`](examples/motion_pipeline_lowvram.json) past ~10 s | int8 or W4A8 | the 702-frame de-rope at 25 GiB peak, 318 s/step, same speed as the 96 GB card (the step is attention-bound at that length) |
|
| 24 |
+
| 24 GB | 32 GB | [`motion_pipeline_lowvram.json`](examples/motion_pipeline_lowvram.json) | W4A8 | 702 frames at 23.5 GiB peak, 25.8 GB RSS; 906 frames renders at 515 s/step with RAM at the edge (29.8 GB) |
|
| 25 |
+
| 16 GB | 32 GB | [`motion_pipeline_lowvram.json`](examples/motion_pipeline_lowvram.json) | W4A8 + NVFP4 text encoder | 702 frames at 15.5 GiB peak, 27.4 GB RSS, 316 s/step; 906 frames does not fit |
|
| 26 |
+
| 15 GiB (a 16 GB card with its desktop up) | 48 GB | the same with `kv_store` = `kvi8s` | W4A8 | the exact K/V store runs out of memory at the second forward; `kvi8r` renders (+9.5 GiB, 374 s/step), `kvi8s` renders (219 s/step), `kvi8s` + `trim_forward` renders with zero allocator trims |
|
| 27 |
+
| 12 GB | any | not measured | | the line says ~150k packed tokens, roughly a 5 s clip de-roped at 1 MP. [`motion_pipeline_rolling_window_lowvram.json`](examples/motion_pipeline_rolling_window_lowvram.json) makes peak memory follow the window, not the clip, and is the graph to try; reports wanted |
|
| 28 |
+
|
| 29 |
+
The ComfyUI flags the small-card rows were measured with: dynamic VRAM on
|
| 30 |
+
(the default since 0.33), `--fast-disk`, and nothing else. `--reserve-vram`
|
| 31 |
+
is a global hammer and was not needed on any row.
|
| 32 |
+
|
| 33 |
+
## How long a clip fits
|
| 34 |
+
|
| 35 |
+
Packed tokens is the unit the card cares about, not seconds. A frame at
|
| 36 |
+
1376x768 is about 310 tokens after dilation, and a de-rope at `d_max` 4
|
| 37 |
+
on a bursty clip roughly doubles to triples the frame count, so the
|
| 38 |
+
dilated pass is what you are sizing for, not the clip you typed.
|
| 39 |
+
|
| 40 |
+
| VRAM | max packed tokens | dilated frames at 1376x768, one pass | RAM at that length, single pass / full pipeline |
|
| 41 |
+
|---|---|---|---|
|
| 42 |
+
| 16 GB | ~230k | ~740 (measured: 702 renders, 906 does not) | ~28 / ~40 GB |
|
| 43 |
+
| 24 GB | ~380k | ~1230 (measured: 906 renders) | ~33 / ~45 GB |
|
| 44 |
+
| 32 GB | ~530k | ~1720 *extrapolated* | ~39 / ~51 GB *extrapolated* |
|
| 45 |
+
|
| 46 |
+
Beyond the row you are on, the answers in order of cheapness: lower
|
| 47 |
+
`d_max` (3 instead of 4 is a third fewer dilated frames), raise `q` (0.85
|
| 48 |
+
dilates only the hottest spans), drop the resolution of pass 1 and let
|
| 49 |
+
[`motion_pipeline_upscale_derope.json`](examples/motion_pipeline_upscale_derope.json)
|
| 50 |
+
do the de-rope at the size you want, then the rolling window.
|
| 51 |
+
|
| 52 |
+
## Seconds per step
|
| 53 |
+
|
| 54 |
+
The only per-step number we can give you is ours, and it scales
|
| 55 |
+
superlinearly with tokens (exponent ~1.7), so double the frames costs
|
| 56 |
+
more than double the time:
|
| 57 |
+
|
| 58 |
+
| clip | card | s/step |
|
| 59 |
+
|---|---|---|
|
| 60 |
+
| 5 s at 1024x1024, plain pipeline | RTX PRO 6000 | ~11.5 |
|
| 61 |
+
| 702 dilated frames at 1376x768 (~217k tokens), pass 2 | RTX PRO 6000, or a fenced 16 to 32 GB budget of it | 311 to 318 |
|
| 62 |
+
|
| 63 |
+
A 5090 user reported the VAE encode of 120 frames at ~1 MP at 20.5 s
|
| 64 |
+
(fp16) and 0.41 s with the TAE encoder; our own encoder line is 0.35 s per
|
| 65 |
+
dilated frame at 1.2 MP, flat 5 GiB, and the int8 VAE is no faster at
|
| 66 |
+
encoding (it only halves the weights). The encode is the stage that hurts
|
| 67 |
+
most on a small card, which is why the TAE matters: it needs a ComfyUI
|
| 68 |
+
newer than 2026-08-17 (core PR #15695).
|
| 69 |
+
|
| 70 |
+
## Quality tiers on the same hardware
|
| 71 |
+
|
| 72 |
+
Every row above also has a time dial that is not hardware at all:
|
| 73 |
+
|
| 74 |
+
| want | pass 1 | pass 2 | cost vs one baseline |
|
| 75 |
+
|---|---|---|---|
|
| 76 |
+
| scout a prompt | 0.2 MP, 12 steps | turbo, 0.4 MP | ~1x, 95 s on our card |
|
| 77 |
+
| the normal final | 12 steps, base model | turbo, 6-step `beta`, inject 0.70 | ~1.3x |
|
| 78 |
+
| the slow final | 25 steps | base model, inject 0.70 | ~3.5x |
|
| 79 |
+
|
| 80 |
+
The middle row is the August 19 recipe and it wants the whole recipe
|
| 81 |
+
(`linear_quadratic` + `gradient_estimation` on pass 1, `beta` on pass 2,
|
| 82 |
+
seeded audio): the turbo LoRA dropped into the 25-step graph on its own
|
| 83 |
+
renders jerky. [`examples/README.md`](examples/README.md) has the two
|
| 84 |
+
recipes side by side.
|
| 85 |
+
|
| 86 |
+
## System RAM, specifically
|
| 87 |
+
|
| 88 |
+
Measured on the 702-frame de-rope: 25.8 to 27.4 GB RSS on the small-card
|
| 89 |
+
rows, with the text encoder's CPU mirror the avoidable part. What helps,
|
| 90 |
+
in order: `H3 Evict Text Encoder` after each prompt encode (1 to 2.5 GiB
|
| 91 |
+
of VRAM peak back on a 16 GB card, and the mirror goes with it), keeping
|
| 92 |
+
pass 1 at the resolution you need and no higher, and not running two
|
| 93 |
+
ComfyUI instances that each hold a copy of the weights on the host. A
|
| 94 |
+
`--gpu-only` instance removes the host mirror entirely but makes unload a
|
| 95 |
+
no-op, so it only suits a card that holds everything.
|
| 96 |
+
|
| 97 |
+
## Send numbers back
|
| 98 |
+
|
| 99 |
+
An issue with your card, RAM, ComfyUI version, the graph, the packed token
|
| 100 |
+
count from the `H3 Jerk Oracle` cost readout, and your seconds per step
|
| 101 |
+
fills in a row of this page for everyone after you. Failures on content
|
| 102 |
+
unlike ours are worth more than successes on content like ours.
|
ComfyUI-MiniMaxH3-Context-Loop/ComfyUI-MAINodes/LICENSE
ADDED
|
@@ -0,0 +1,679 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
ComfyUI-MAINodes
|
| 2 |
+
Copyright (C) 2026 MATLOWAI
|
| 3 |
+
|
| 4 |
+
This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version. See below for the full text.
|
| 5 |
+
|
| 6 |
+
GNU GENERAL PUBLIC LICENSE
|
| 7 |
+
Version 3, 29 June 2007
|
| 8 |
+
|
| 9 |
+
Copyright (C) 2007 Free Software Foundation, Inc. <https://fsf.org/>
|
| 10 |
+
Everyone is permitted to copy and distribute verbatim copies
|
| 11 |
+
of this license document, but changing it is not allowed.
|
| 12 |
+
|
| 13 |
+
Preamble
|
| 14 |
+
|
| 15 |
+
The GNU General Public License is a free, copyleft license for
|
| 16 |
+
software and other kinds of works.
|
| 17 |
+
|
| 18 |
+
The licenses for most software and other practical works are designed
|
| 19 |
+
to take away your freedom to share and change the works. By contrast,
|
| 20 |
+
the GNU General Public License is intended to guarantee your freedom to
|
| 21 |
+
share and change all versions of a program--to make sure it remains free
|
| 22 |
+
software for all its users. We, the Free Software Foundation, use the
|
| 23 |
+
GNU General Public License for most of our software; it applies also to
|
| 24 |
+
any other work released this way by its authors. You can apply it to
|
| 25 |
+
your programs, too.
|
| 26 |
+
|
| 27 |
+
When we speak of free software, we are referring to freedom, not
|
| 28 |
+
price. Our General Public Licenses are designed to make sure that you
|
| 29 |
+
have the freedom to distribute copies of free software (and charge for
|
| 30 |
+
them if you wish), that you receive source code or can get it if you
|
| 31 |
+
want it, that you can change the software or use pieces of it in new
|
| 32 |
+
free programs, and that you know you can do these things.
|
| 33 |
+
|
| 34 |
+
To protect your rights, we need to prevent others from denying you
|
| 35 |
+
these rights or asking you to surrender the rights. Therefore, you have
|
| 36 |
+
certain responsibilities if you distribute copies of the software, or if
|
| 37 |
+
you modify it: responsibilities to respect the freedom of others.
|
| 38 |
+
|
| 39 |
+
For example, if you distribute copies of such a program, whether
|
| 40 |
+
gratis or for a fee, you must pass on to the recipients the same
|
| 41 |
+
freedoms that you received. You must make sure that they, too, receive
|
| 42 |
+
or can get the source code. And you must show them these terms so they
|
| 43 |
+
know their rights.
|
| 44 |
+
|
| 45 |
+
Developers that use the GNU GPL protect your rights with two steps:
|
| 46 |
+
(1) assert copyright on the software, and (2) offer you this License
|
| 47 |
+
giving you legal permission to copy, distribute and/or modify it.
|
| 48 |
+
|
| 49 |
+
For the developers' and authors' protection, the GPL clearly explains
|
| 50 |
+
that there is no warranty for this free software. For both users' and
|
| 51 |
+
authors' sake, the GPL requires that modified versions be marked as
|
| 52 |
+
changed, so that their problems will not be attributed erroneously to
|
| 53 |
+
authors of previous versions.
|
| 54 |
+
|
| 55 |
+
Some devices are designed to deny users access to install or run
|
| 56 |
+
modified versions of the software inside them, although the manufacturer
|
| 57 |
+
can do so. This is fundamentally incompatible with the aim of
|
| 58 |
+
protecting users' freedom to change the software. The systematic
|
| 59 |
+
pattern of such abuse occurs in the area of products for individuals to
|
| 60 |
+
use, which is precisely where it is most unacceptable. Therefore, we
|
| 61 |
+
have designed this version of the GPL to prohibit the practice for those
|
| 62 |
+
products. If such problems arise substantially in other domains, we
|
| 63 |
+
stand ready to extend this provision to those domains in future versions
|
| 64 |
+
of the GPL, as needed to protect the freedom of users.
|
| 65 |
+
|
| 66 |
+
Finally, every program is threatened constantly by software patents.
|
| 67 |
+
States should not allow patents to restrict development and use of
|
| 68 |
+
software on general-purpose computers, but in those that do, we wish to
|
| 69 |
+
avoid the special danger that patents applied to a free program could
|
| 70 |
+
make it effectively proprietary. To prevent this, the GPL assures that
|
| 71 |
+
patents cannot be used to render the program non-free.
|
| 72 |
+
|
| 73 |
+
The precise terms and conditions for copying, distribution and
|
| 74 |
+
modification follow.
|
| 75 |
+
|
| 76 |
+
TERMS AND CONDITIONS
|
| 77 |
+
|
| 78 |
+
0. Definitions.
|
| 79 |
+
|
| 80 |
+
"This License" refers to version 3 of the GNU General Public License.
|
| 81 |
+
|
| 82 |
+
"Copyright" also means copyright-like laws that apply to other kinds of
|
| 83 |
+
works, such as semiconductor masks.
|
| 84 |
+
|
| 85 |
+
"The Program" refers to any copyrightable work licensed under this
|
| 86 |
+
License. Each licensee is addressed as "you". "Licensees" and
|
| 87 |
+
"recipients" may be individuals or organizations.
|
| 88 |
+
|
| 89 |
+
To "modify" a work means to copy from or adapt all or part of the work
|
| 90 |
+
in a fashion requiring copyright permission, other than the making of an
|
| 91 |
+
exact copy. The resulting work is called a "modified version" of the
|
| 92 |
+
earlier work or a work "based on" the earlier work.
|
| 93 |
+
|
| 94 |
+
A "covered work" means either the unmodified Program or a work based
|
| 95 |
+
on the Program.
|
| 96 |
+
|
| 97 |
+
To "propagate" a work means to do anything with it that, without
|
| 98 |
+
permission, would make you directly or secondarily liable for
|
| 99 |
+
infringement under applicable copyright law, except executing it on a
|
| 100 |
+
computer or modifying a private copy. Propagation includes copying,
|
| 101 |
+
distribution (with or without modification), making available to the
|
| 102 |
+
public, and in some countries other activities as well.
|
| 103 |
+
|
| 104 |
+
To "convey" a work means any kind of propagation that enables other
|
| 105 |
+
parties to make or receive copies. Mere interaction with a user through
|
| 106 |
+
a computer network, with no transfer of a copy, is not conveying.
|
| 107 |
+
|
| 108 |
+
An interactive user interface displays "Appropriate Legal Notices"
|
| 109 |
+
to the extent that it includes a convenient and prominently visible
|
| 110 |
+
feature that (1) displays an appropriate copyright notice, and (2)
|
| 111 |
+
tells the user that there is no warranty for the work (except to the
|
| 112 |
+
extent that warranties are provided), that licensees may convey the
|
| 113 |
+
work under this License, and how to view a copy of this License. If
|
| 114 |
+
the interface presents a list of user commands or options, such as a
|
| 115 |
+
menu, a prominent item in the list meets this criterion.
|
| 116 |
+
|
| 117 |
+
1. Source Code.
|
| 118 |
+
|
| 119 |
+
The "source code" for a work means the preferred form of the work
|
| 120 |
+
for making modifications to it. "Object code" means any non-source
|
| 121 |
+
form of a work.
|
| 122 |
+
|
| 123 |
+
A "Standard Interface" means an interface that either is an official
|
| 124 |
+
standard defined by a recognized standards body, or, in the case of
|
| 125 |
+
interfaces specified for a particular programming language, one that
|
| 126 |
+
is widely used among developers working in that language.
|
| 127 |
+
|
| 128 |
+
The "System Libraries" of an executable work include anything, other
|
| 129 |
+
than the work as a whole, that (a) is included in the normal form of
|
| 130 |
+
packaging a Major Component, but which is not part of that Major
|
| 131 |
+
Component, and (b) serves only to enable use of the work with that
|
| 132 |
+
Major Component, or to implement a Standard Interface for which an
|
| 133 |
+
implementation is available to the public in source code form. A
|
| 134 |
+
"Major Component", in this context, means a major essential component
|
| 135 |
+
(kernel, window system, and so on) of the specific operating system
|
| 136 |
+
(if any) on which the executable work runs, or a compiler used to
|
| 137 |
+
produce the work, or an object code interpreter used to run it.
|
| 138 |
+
|
| 139 |
+
The "Corresponding Source" for a work in object code form means all
|
| 140 |
+
the source code needed to generate, install, and (for an executable
|
| 141 |
+
work) run the object code and to modify the work, including scripts to
|
| 142 |
+
control those activities. However, it does not include the work's
|
| 143 |
+
System Libraries, or general-purpose tools or generally available free
|
| 144 |
+
programs which are used unmodified in performing those activities but
|
| 145 |
+
which are not part of the work. For example, Corresponding Source
|
| 146 |
+
includes interface definition files associated with source files for
|
| 147 |
+
the work, and the source code for shared libraries and dynamically
|
| 148 |
+
linked subprograms that the work is specifically designed to require,
|
| 149 |
+
such as by intimate data communication or control flow between those
|
| 150 |
+
subprograms and other parts of the work.
|
| 151 |
+
|
| 152 |
+
The Corresponding Source need not include anything that users
|
| 153 |
+
can regenerate automatically from other parts of the Corresponding
|
| 154 |
+
Source.
|
| 155 |
+
|
| 156 |
+
The Corresponding Source for a work in source code form is that
|
| 157 |
+
same work.
|
| 158 |
+
|
| 159 |
+
2. Basic Permissions.
|
| 160 |
+
|
| 161 |
+
All rights granted under this License are granted for the term of
|
| 162 |
+
copyright on the Program, and are irrevocable provided the stated
|
| 163 |
+
conditions are met. This License explicitly affirms your unlimited
|
| 164 |
+
permission to run the unmodified Program. The output from running a
|
| 165 |
+
covered work is covered by this License only if the output, given its
|
| 166 |
+
content, constitutes a covered work. This License acknowledges your
|
| 167 |
+
rights of fair use or other equivalent, as provided by copyright law.
|
| 168 |
+
|
| 169 |
+
You may make, run and propagate covered works that you do not
|
| 170 |
+
convey, without conditions so long as your license otherwise remains
|
| 171 |
+
in force. You may convey covered works to others for the sole purpose
|
| 172 |
+
of having them make modifications exclusively for you, or provide you
|
| 173 |
+
with facilities for running those works, provided that you comply with
|
| 174 |
+
the terms of this License in conveying all material for which you do
|
| 175 |
+
not control copyright. Those thus making or running the covered works
|
| 176 |
+
for you must do so exclusively on your behalf, under your direction
|
| 177 |
+
and control, on terms that prohibit them from making any copies of
|
| 178 |
+
your copyrighted material outside their relationship with you.
|
| 179 |
+
|
| 180 |
+
Conveying under any other circumstances is permitted solely under
|
| 181 |
+
the conditions stated below. Sublicensing is not allowed; section 10
|
| 182 |
+
makes it unnecessary.
|
| 183 |
+
|
| 184 |
+
3. Protecting Users' Legal Rights From Anti-Circumvention Law.
|
| 185 |
+
|
| 186 |
+
No covered work shall be deemed part of an effective technological
|
| 187 |
+
measure under any applicable law fulfilling obligations under article
|
| 188 |
+
11 of the WIPO copyright treaty adopted on 20 December 1996, or
|
| 189 |
+
similar laws prohibiting or restricting circumvention of such
|
| 190 |
+
measures.
|
| 191 |
+
|
| 192 |
+
When you convey a covered work, you waive any legal power to forbid
|
| 193 |
+
circumvention of technological measures to the extent such circumvention
|
| 194 |
+
is effected by exercising rights under this License with respect to
|
| 195 |
+
the covered work, and you disclaim any intention to limit operation or
|
| 196 |
+
modification of the work as a means of enforcing, against the work's
|
| 197 |
+
users, your or third parties' legal rights to forbid circumvention of
|
| 198 |
+
technological measures.
|
| 199 |
+
|
| 200 |
+
4. Conveying Verbatim Copies.
|
| 201 |
+
|
| 202 |
+
You may convey verbatim copies of the Program's source code as you
|
| 203 |
+
receive it, in any medium, provided that you conspicuously and
|
| 204 |
+
appropriately publish on each copy an appropriate copyright notice;
|
| 205 |
+
keep intact all notices stating that this License and any
|
| 206 |
+
non-permissive terms added in accord with section 7 apply to the code;
|
| 207 |
+
keep intact all notices of the absence of any warranty; and give all
|
| 208 |
+
recipients a copy of this License along with the Program.
|
| 209 |
+
|
| 210 |
+
You may charge any price or no price for each copy that you convey,
|
| 211 |
+
and you may offer support or warranty protection for a fee.
|
| 212 |
+
|
| 213 |
+
5. Conveying Modified Source Versions.
|
| 214 |
+
|
| 215 |
+
You may convey a work based on the Program, or the modifications to
|
| 216 |
+
produce it from the Program, in the form of source code under the
|
| 217 |
+
terms of section 4, provided that you also meet all of these conditions:
|
| 218 |
+
|
| 219 |
+
a) The work must carry prominent notices stating that you modified
|
| 220 |
+
it, and giving a relevant date.
|
| 221 |
+
|
| 222 |
+
b) The work must carry prominent notices stating that it is
|
| 223 |
+
released under this License and any conditions added under section
|
| 224 |
+
7. This requirement modifies the requirement in section 4 to
|
| 225 |
+
"keep intact all notices".
|
| 226 |
+
|
| 227 |
+
c) You must license the entire work, as a whole, under this
|
| 228 |
+
License to anyone who comes into possession of a copy. This
|
| 229 |
+
License will therefore apply, along with any applicable section 7
|
| 230 |
+
additional terms, to the whole of the work, and all its parts,
|
| 231 |
+
regardless of how they are packaged. This License gives no
|
| 232 |
+
permission to license the work in any other way, but it does not
|
| 233 |
+
invalidate such permission if you have separately received it.
|
| 234 |
+
|
| 235 |
+
d) If the work has interactive user interfaces, each must display
|
| 236 |
+
Appropriate Legal Notices; however, if the Program has interactive
|
| 237 |
+
interfaces that do not display Appropriate Legal Notices, your
|
| 238 |
+
work need not make them do so.
|
| 239 |
+
|
| 240 |
+
A compilation of a covered work with other separate and independent
|
| 241 |
+
works, which are not by their nature extensions of the covered work,
|
| 242 |
+
and which are not combined with it such as to form a larger program,
|
| 243 |
+
in or on a volume of a storage or distribution medium, is called an
|
| 244 |
+
"aggregate" if the compilation and its resulting copyright are not
|
| 245 |
+
used to limit the access or legal rights of the compilation's users
|
| 246 |
+
beyond what the individual works permit. Inclusion of a covered work
|
| 247 |
+
in an aggregate does not cause this License to apply to the other
|
| 248 |
+
parts of the aggregate.
|
| 249 |
+
|
| 250 |
+
6. Conveying Non-Source Forms.
|
| 251 |
+
|
| 252 |
+
You may convey a covered work in object code form under the terms
|
| 253 |
+
of sections 4 and 5, provided that you also convey the
|
| 254 |
+
machine-readable Corresponding Source under the terms of this License,
|
| 255 |
+
in one of these ways:
|
| 256 |
+
|
| 257 |
+
a) Convey the object code in, or embodied in, a physical product
|
| 258 |
+
(including a physical distribution medium), accompanied by the
|
| 259 |
+
Corresponding Source fixed on a durable physical medium
|
| 260 |
+
customarily used for software interchange.
|
| 261 |
+
|
| 262 |
+
b) Convey the object code in, or embodied in, a physical product
|
| 263 |
+
(including a physical distribution medium), accompanied by a
|
| 264 |
+
written offer, valid for at least three years and valid for as
|
| 265 |
+
long as you offer spare parts or customer support for that product
|
| 266 |
+
model, to give anyone who possesses the object code either (1) a
|
| 267 |
+
copy of the Corresponding Source for all the software in the
|
| 268 |
+
product that is covered by this License, on a durable physical
|
| 269 |
+
medium customarily used for software interchange, for a price no
|
| 270 |
+
more than your reasonable cost of physically performing this
|
| 271 |
+
conveying of source, or (2) access to copy the
|
| 272 |
+
Corresponding Source from a network server at no charge.
|
| 273 |
+
|
| 274 |
+
c) Convey individual copies of the object code with a copy of the
|
| 275 |
+
written offer to provide the Corresponding Source. This
|
| 276 |
+
alternative is allowed only occasionally and noncommercially, and
|
| 277 |
+
only if you received the object code with such an offer, in accord
|
| 278 |
+
with subsection 6b.
|
| 279 |
+
|
| 280 |
+
d) Convey the object code by offering access from a designated
|
| 281 |
+
place (gratis or for a charge), and offer equivalent access to the
|
| 282 |
+
Corresponding Source in the same way through the same place at no
|
| 283 |
+
further charge. You need not require recipients to copy the
|
| 284 |
+
Corresponding Source along with the object code. If the place to
|
| 285 |
+
copy the object code is a network server, the Corresponding Source
|
| 286 |
+
may be on a different server (operated by you or a third party)
|
| 287 |
+
that supports equivalent copying facilities, provided you maintain
|
| 288 |
+
clear directions next to the object code saying where to find the
|
| 289 |
+
Corresponding Source. Regardless of what server hosts the
|
| 290 |
+
Corresponding Source, you remain obligated to ensure that it is
|
| 291 |
+
available for as long as needed to satisfy these requirements.
|
| 292 |
+
|
| 293 |
+
e) Convey the object code using peer-to-peer transmission, provided
|
| 294 |
+
you inform other peers where the object code and Corresponding
|
| 295 |
+
Source of the work are being offered to the general public at no
|
| 296 |
+
charge under subsection 6d.
|
| 297 |
+
|
| 298 |
+
A separable portion of the object code, whose source code is excluded
|
| 299 |
+
from the Corresponding Source as a System Library, need not be
|
| 300 |
+
included in conveying the object code work.
|
| 301 |
+
|
| 302 |
+
A "User Product" is either (1) a "consumer product", which means any
|
| 303 |
+
tangible personal property which is normally used for personal, family,
|
| 304 |
+
or household purposes, or (2) anything designed or sold for incorporation
|
| 305 |
+
into a dwelling. In determining whether a product is a consumer product,
|
| 306 |
+
doubtful cases shall be resolved in favor of coverage. For a particular
|
| 307 |
+
product received by a particular user, "normally used" refers to a
|
| 308 |
+
typical or common use of that class of product, regardless of the status
|
| 309 |
+
of the particular user or of the way in which the particular user
|
| 310 |
+
actually uses, or expects or is expected to use, the product. A product
|
| 311 |
+
is a consumer product regardless of whether the product has substantial
|
| 312 |
+
commercial, industrial or non-consumer uses, unless such uses represent
|
| 313 |
+
the only significant mode of use of the product.
|
| 314 |
+
|
| 315 |
+
"Installation Information" for a User Product means any methods,
|
| 316 |
+
procedures, authorization keys, or other information required to install
|
| 317 |
+
and execute modified versions of a covered work in that User Product from
|
| 318 |
+
a modified version of its Corresponding Source. The information must
|
| 319 |
+
suffice to ensure that the continued functioning of the modified object
|
| 320 |
+
code is in no case prevented or interfered with solely because
|
| 321 |
+
modification has been made.
|
| 322 |
+
|
| 323 |
+
If you convey an object code work under this section in, or with, or
|
| 324 |
+
specifically for use in, a User Product, and the conveying occurs as
|
| 325 |
+
part of a transaction in which the right of possession and use of the
|
| 326 |
+
User Product is transferred to the recipient in perpetuity or for a
|
| 327 |
+
fixed term (regardless of how the transaction is characterized), the
|
| 328 |
+
Corresponding Source conveyed under this section must be accompanied
|
| 329 |
+
by the Installation Information. But this requirement does not apply
|
| 330 |
+
if neither you nor any third party retains the ability to install
|
| 331 |
+
modified object code on the User Product (for example, the work has
|
| 332 |
+
been installed in ROM).
|
| 333 |
+
|
| 334 |
+
The requirement to provide Installation Information does not include a
|
| 335 |
+
requirement to continue to provide support service, warranty, or updates
|
| 336 |
+
for a work that has been modified or installed by the recipient, or for
|
| 337 |
+
the User Product in which it has been modified or installed. Access to a
|
| 338 |
+
network may be denied when the modification itself materially and
|
| 339 |
+
adversely affects the operation of the network or violates the rules and
|
| 340 |
+
protocols for communication across the network.
|
| 341 |
+
|
| 342 |
+
Corresponding Source conveyed, and Installation Information provided,
|
| 343 |
+
in accord with this section must be in a format that is publicly
|
| 344 |
+
documented (and with an implementation available to the public in
|
| 345 |
+
source code form), and must require no special password or key for
|
| 346 |
+
unpacking, reading or copying.
|
| 347 |
+
|
| 348 |
+
7. Additional Terms.
|
| 349 |
+
|
| 350 |
+
"Additional permissions" are terms that supplement the terms of this
|
| 351 |
+
License by making exceptions from one or more of its conditions.
|
| 352 |
+
Additional permissions that are applicable to the entire Program shall
|
| 353 |
+
be treated as though they were included in this License, to the extent
|
| 354 |
+
that they are valid under applicable law. If additional permissions
|
| 355 |
+
apply only to part of the Program, that part may be used separately
|
| 356 |
+
under those permissions, but the entire Program remains governed by
|
| 357 |
+
this License without regard to the additional permissions.
|
| 358 |
+
|
| 359 |
+
When you convey a copy of a covered work, you may at your option
|
| 360 |
+
remove any additional permissions from that copy, or from any part of
|
| 361 |
+
it. (Additional permissions may be written to require their own
|
| 362 |
+
removal in certain cases when you modify the work.) You may place
|
| 363 |
+
additional permissions on material, added by you to a covered work,
|
| 364 |
+
for which you have or can give appropriate copyright permission.
|
| 365 |
+
|
| 366 |
+
Notwithstanding any other provision of this License, for material you
|
| 367 |
+
add to a covered work, you may (if authorized by the copyright holders of
|
| 368 |
+
that material) supplement the terms of this License with terms:
|
| 369 |
+
|
| 370 |
+
a) Disclaiming warranty or limiting liability differently from the
|
| 371 |
+
terms of sections 15 and 16 of this License; or
|
| 372 |
+
|
| 373 |
+
b) Requiring preservation of specified reasonable legal notices or
|
| 374 |
+
author attributions in that material or in the Appropriate Legal
|
| 375 |
+
Notices displayed by works containing it; or
|
| 376 |
+
|
| 377 |
+
c) Prohibiting misrepresentation of the origin of that material, or
|
| 378 |
+
requiring that modified versions of such material be marked in
|
| 379 |
+
reasonable ways as different from the original version; or
|
| 380 |
+
|
| 381 |
+
d) Limiting the use for publicity purposes of names of licensors or
|
| 382 |
+
authors of the material; or
|
| 383 |
+
|
| 384 |
+
e) Declining to grant rights under trademark law for use of some
|
| 385 |
+
trade names, trademarks, or service marks; or
|
| 386 |
+
|
| 387 |
+
f) Requiring indemnification of licensors and authors of that
|
| 388 |
+
material by anyone who conveys the material (or modified versions of
|
| 389 |
+
it) with contractual assumptions of liability to the recipient, for
|
| 390 |
+
any liability that these contractual assumptions directly impose on
|
| 391 |
+
those licensors and authors.
|
| 392 |
+
|
| 393 |
+
All other non-permissive additional terms are considered "further
|
| 394 |
+
restrictions" within the meaning of section 10. If the Program as you
|
| 395 |
+
received it, or any part of it, contains a notice stating that it is
|
| 396 |
+
governed by this License along with a term that is a further
|
| 397 |
+
restriction, you may remove that term. If a license document contains
|
| 398 |
+
a further restriction but permits relicensing or conveying under this
|
| 399 |
+
License, you may add to a covered work material governed by the terms
|
| 400 |
+
of that license document, provided that the further restriction does
|
| 401 |
+
not survive such relicensing or conveying.
|
| 402 |
+
|
| 403 |
+
If you add terms to a covered work in accord with this section, you
|
| 404 |
+
must place, in the relevant source files, a statement of the
|
| 405 |
+
additional terms that apply to those files, or a notice indicating
|
| 406 |
+
where to find the applicable terms.
|
| 407 |
+
|
| 408 |
+
Additional terms, permissive or non-permissive, may be stated in the
|
| 409 |
+
form of a separately written license, or stated as exceptions;
|
| 410 |
+
the above requirements apply either way.
|
| 411 |
+
|
| 412 |
+
8. Termination.
|
| 413 |
+
|
| 414 |
+
You may not propagate or modify a covered work except as expressly
|
| 415 |
+
provided under this License. Any attempt otherwise to propagate or
|
| 416 |
+
modify it is void, and will automatically terminate your rights under
|
| 417 |
+
this License (including any patent licenses granted under the third
|
| 418 |
+
paragraph of section 11).
|
| 419 |
+
|
| 420 |
+
However, if you cease all violation of this License, then your
|
| 421 |
+
license from a particular copyright holder is reinstated (a)
|
| 422 |
+
provisionally, unless and until the copyright holder explicitly and
|
| 423 |
+
finally terminates your license, and (b) permanently, if the copyright
|
| 424 |
+
holder fails to notify you of the violation by some reasonable means
|
| 425 |
+
prior to 60 days after the cessation.
|
| 426 |
+
|
| 427 |
+
Moreover, your license from a particular copyright holder is
|
| 428 |
+
reinstated permanently if the copyright holder notifies you of the
|
| 429 |
+
violation by some reasonable means, this is the first time you have
|
| 430 |
+
received notice of violation of this License (for any work) from that
|
| 431 |
+
copyright holder, and you cure the violation prior to 30 days after
|
| 432 |
+
your receipt of the notice.
|
| 433 |
+
|
| 434 |
+
Termination of your rights under this section does not terminate the
|
| 435 |
+
licenses of parties who have received copies or rights from you under
|
| 436 |
+
this License. If your rights have been terminated and not permanently
|
| 437 |
+
reinstated, you do not qualify to receive new licenses for the same
|
| 438 |
+
material under section 10.
|
| 439 |
+
|
| 440 |
+
9. Acceptance Not Required for Having Copies.
|
| 441 |
+
|
| 442 |
+
You are not required to accept this License in order to receive or
|
| 443 |
+
run a copy of the Program. Ancillary propagation of a covered work
|
| 444 |
+
occurring solely as a consequence of using peer-to-peer transmission
|
| 445 |
+
to receive a copy likewise does not require acceptance. However,
|
| 446 |
+
nothing other than this License grants you permission to propagate or
|
| 447 |
+
modify any covered work. These actions infringe copyright if you do
|
| 448 |
+
not accept this License. Therefore, by modifying or propagating a
|
| 449 |
+
covered work, you indicate your acceptance of this License to do so.
|
| 450 |
+
|
| 451 |
+
10. Automatic Licensing of Downstream Recipients.
|
| 452 |
+
|
| 453 |
+
Each time you convey a covered work, the recipient automatically
|
| 454 |
+
receives a license from the original licensors, to run, modify and
|
| 455 |
+
propagate that work, subject to this License. You are not responsible
|
| 456 |
+
for enforcing compliance by third parties with this License.
|
| 457 |
+
|
| 458 |
+
An "entity transaction" is a transaction transferring control of an
|
| 459 |
+
organization, or substantially all assets of one, or subdividing an
|
| 460 |
+
organization, or merging organizations. If propagation of a covered
|
| 461 |
+
work results from an entity transaction, each party to that
|
| 462 |
+
transaction who receives a copy of the work also receives whatever
|
| 463 |
+
licenses to the work the party's predecessor in interest had or could
|
| 464 |
+
give under the previous paragraph, plus a right to possession of the
|
| 465 |
+
Corresponding Source of the work from the predecessor in interest, if
|
| 466 |
+
the predecessor has it or can get it with reasonable efforts.
|
| 467 |
+
|
| 468 |
+
You may not impose any further restrictions on the exercise of the
|
| 469 |
+
rights granted or affirmed under this License. For example, you may
|
| 470 |
+
not impose a license fee, royalty, or other charge for exercise of
|
| 471 |
+
rights granted under this License, and you may not initiate litigation
|
| 472 |
+
(including a cross-claim or counterclaim in a lawsuit) alleging that
|
| 473 |
+
any patent claim is infringed by making, using, selling, offering for
|
| 474 |
+
sale, or importing the Program or any portion of it.
|
| 475 |
+
|
| 476 |
+
11. Patents.
|
| 477 |
+
|
| 478 |
+
A "contributor" is a copyright holder who authorizes use under this
|
| 479 |
+
License of the Program or a work on which the Program is based. The
|
| 480 |
+
work thus licensed is called the contributor's "contributor version".
|
| 481 |
+
|
| 482 |
+
A contributor's "essential patent claims" are all patent claims
|
| 483 |
+
owned or controlled by the contributor, whether already acquired or
|
| 484 |
+
hereafter acquired, that would be infringed by some manner, permitted
|
| 485 |
+
by this License, of making, using, or selling its contributor version,
|
| 486 |
+
but do not include claims that would be infringed only as a
|
| 487 |
+
consequence of further modification of the contributor version. For
|
| 488 |
+
purposes of this definition, "control" includes the right to grant
|
| 489 |
+
patent sublicenses in a manner consistent with the requirements of
|
| 490 |
+
this License.
|
| 491 |
+
|
| 492 |
+
Each contributor grants you a non-exclusive, worldwide, royalty-free
|
| 493 |
+
patent license under the contributor's essential patent claims, to
|
| 494 |
+
make, use, sell, offer for sale, import and otherwise run, modify and
|
| 495 |
+
propagate the contents of its contributor version.
|
| 496 |
+
|
| 497 |
+
In the following three paragraphs, a "patent license" is any express
|
| 498 |
+
agreement or commitment, however denominated, not to enforce a patent
|
| 499 |
+
(such as an express permission to practice a patent or covenant not to
|
| 500 |
+
sue for patent infringement). To "grant" such a patent license to a
|
| 501 |
+
party means to make such an agreement or commitment not to enforce a
|
| 502 |
+
patent against the party.
|
| 503 |
+
|
| 504 |
+
If you convey a covered work, knowingly relying on a patent license,
|
| 505 |
+
and the Corresponding Source of the work is not available for anyone
|
| 506 |
+
to copy, free of charge and under the terms of this License, through a
|
| 507 |
+
publicly available network server or other readily accessible means,
|
| 508 |
+
then you must either (1) cause the Corresponding Source to be so
|
| 509 |
+
available, or (2) arrange to deprive yourself of the benefit of the
|
| 510 |
+
patent license for this particular work, or (3) arrange, in a manner
|
| 511 |
+
consistent with the requirements of this License, to extend the patent
|
| 512 |
+
license to downstream recipients. "Knowingly relying" means you have
|
| 513 |
+
actual knowledge that, but for the patent license, your conveying the
|
| 514 |
+
covered work in a country, or your recipient's use of the covered work
|
| 515 |
+
in a country, would infringe one or more identifiable patents in that
|
| 516 |
+
country that you have reason to believe are valid.
|
| 517 |
+
|
| 518 |
+
If, pursuant to or in connection with a single transaction or
|
| 519 |
+
arrangement, you convey, or propagate by procuring conveyance of, a
|
| 520 |
+
covered work, and grant a patent license to some of the parties
|
| 521 |
+
receiving the covered work authorizing them to use, propagate, modify
|
| 522 |
+
or convey a specific copy of the covered work, then the patent license
|
| 523 |
+
you grant is automatically extended to all recipients of the covered
|
| 524 |
+
work and works based on it.
|
| 525 |
+
|
| 526 |
+
A patent license is "discriminatory" if it does not include within
|
| 527 |
+
the scope of its coverage, prohibits the exercise of, or is
|
| 528 |
+
conditioned on the non-exercise of one or more of the rights that are
|
| 529 |
+
specifically granted under this License. You may not convey a covered
|
| 530 |
+
work if you are a party to an arrangement with a third party that is
|
| 531 |
+
in the business of distributing software, under which you make payment
|
| 532 |
+
to the third party based on the extent of your activity of conveying
|
| 533 |
+
the work, and under which the third party grants, to any of the
|
| 534 |
+
parties who would receive the covered work from you, a discriminatory
|
| 535 |
+
patent license (a) in connection with copies of the covered work
|
| 536 |
+
conveyed by you (or copies made from those copies), or (b) primarily
|
| 537 |
+
for and in connection with specific products or compilations that
|
| 538 |
+
contain the covered work, unless you entered into that arrangement,
|
| 539 |
+
or that patent license was granted, prior to 28 March 2007.
|
| 540 |
+
|
| 541 |
+
Nothing in this License shall be construed as excluding or limiting
|
| 542 |
+
any implied license or other defenses to infringement that may
|
| 543 |
+
otherwise be available to you under applicable patent law.
|
| 544 |
+
|
| 545 |
+
12. No Surrender of Others' Freedom.
|
| 546 |
+
|
| 547 |
+
If conditions are imposed on you (whether by court order, agreement or
|
| 548 |
+
otherwise) that contradict the conditions of this License, they do not
|
| 549 |
+
excuse you from the conditions of this License. If you cannot convey a
|
| 550 |
+
covered work so as to satisfy simultaneously your obligations under this
|
| 551 |
+
License and any other pertinent obligations, then as a consequence you may
|
| 552 |
+
not convey it at all. For example, if you agree to terms that obligate you
|
| 553 |
+
to collect a royalty for further conveying from those to whom you convey
|
| 554 |
+
the Program, the only way you could satisfy both those terms and this
|
| 555 |
+
License would be to refrain entirely from conveying the Program.
|
| 556 |
+
|
| 557 |
+
13. Use with the GNU Affero General Public License.
|
| 558 |
+
|
| 559 |
+
Notwithstanding any other provision of this License, you have
|
| 560 |
+
permission to link or combine any covered work with a work licensed
|
| 561 |
+
under version 3 of the GNU Affero General Public License into a single
|
| 562 |
+
combined work, and to convey the resulting work. The terms of this
|
| 563 |
+
License will continue to apply to the part which is the covered work,
|
| 564 |
+
but the special requirements of the GNU Affero General Public License,
|
| 565 |
+
section 13, concerning interaction through a network will apply to the
|
| 566 |
+
combination as such.
|
| 567 |
+
|
| 568 |
+
14. Revised Versions of this License.
|
| 569 |
+
|
| 570 |
+
The Free Software Foundation may publish revised and/or new versions of
|
| 571 |
+
the GNU General Public License from time to time. Such new versions will
|
| 572 |
+
be similar in spirit to the present version, but may differ in detail to
|
| 573 |
+
address new problems or concerns.
|
| 574 |
+
|
| 575 |
+
Each version is given a distinguishing version number. If the
|
| 576 |
+
Program specifies that a certain numbered version of the GNU General
|
| 577 |
+
Public License "or any later version" applies to it, you have the
|
| 578 |
+
option of following the terms and conditions either of that numbered
|
| 579 |
+
version or of any later version published by the Free Software
|
| 580 |
+
Foundation. If the Program does not specify a version number of the
|
| 581 |
+
GNU General Public License, you may choose any version ever published
|
| 582 |
+
by the Free Software Foundation.
|
| 583 |
+
|
| 584 |
+
If the Program specifies that a proxy can decide which future
|
| 585 |
+
versions of the GNU General Public License can be used, that proxy's
|
| 586 |
+
public statement of acceptance of a version permanently authorizes you
|
| 587 |
+
to choose that version for the Program.
|
| 588 |
+
|
| 589 |
+
Later license versions may give you additional or different
|
| 590 |
+
permissions. However, no additional obligations are imposed on any
|
| 591 |
+
author or copyright holder as a result of your choosing to follow a
|
| 592 |
+
later version.
|
| 593 |
+
|
| 594 |
+
15. Disclaimer of Warranty.
|
| 595 |
+
|
| 596 |
+
THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY
|
| 597 |
+
APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT
|
| 598 |
+
HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY
|
| 599 |
+
OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO,
|
| 600 |
+
THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
|
| 601 |
+
PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM
|
| 602 |
+
IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF
|
| 603 |
+
ALL NECESSARY SERVICING, REPAIR OR CORRECTION.
|
| 604 |
+
|
| 605 |
+
16. Limitation of Liability.
|
| 606 |
+
|
| 607 |
+
IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING
|
| 608 |
+
WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MODIFIES AND/OR CONVEYS
|
| 609 |
+
THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY
|
| 610 |
+
GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE
|
| 611 |
+
USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF
|
| 612 |
+
DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD
|
| 613 |
+
PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS),
|
| 614 |
+
EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF
|
| 615 |
+
SUCH DAMAGES.
|
| 616 |
+
|
| 617 |
+
17. Interpretation of Sections 15 and 16.
|
| 618 |
+
|
| 619 |
+
If the disclaimer of warranty and limitation of liability provided
|
| 620 |
+
above cannot be given local legal effect according to their terms,
|
| 621 |
+
reviewing courts shall apply local law that most closely approximates
|
| 622 |
+
an absolute waiver of all civil liability in connection with the
|
| 623 |
+
Program, unless a warranty or assumption of liability accompanies a
|
| 624 |
+
copy of the Program in return for a fee.
|
| 625 |
+
|
| 626 |
+
END OF TERMS AND CONDITIONS
|
| 627 |
+
|
| 628 |
+
How to Apply These Terms to Your New Programs
|
| 629 |
+
|
| 630 |
+
If you develop a new program, and you want it to be of the greatest
|
| 631 |
+
possible use to the public, the best way to achieve this is to make it
|
| 632 |
+
free software which everyone can redistribute and change under these terms.
|
| 633 |
+
|
| 634 |
+
To do so, attach the following notices to the program. It is safest
|
| 635 |
+
to attach them to the start of each source file to most effectively
|
| 636 |
+
state the exclusion of warranty; and each file should have at least
|
| 637 |
+
the "copyright" line and a pointer to where the full notice is found.
|
| 638 |
+
|
| 639 |
+
<one line to give the program's name and a brief idea of what it does.>
|
| 640 |
+
Copyright (C) <year> <name of author>
|
| 641 |
+
|
| 642 |
+
This program is free software: you can redistribute it and/or modify
|
| 643 |
+
it under the terms of the GNU General Public License as published by
|
| 644 |
+
the Free Software Foundation, either version 3 of the License, or
|
| 645 |
+
(at your option) any later version.
|
| 646 |
+
|
| 647 |
+
This program is distributed in the hope that it will be useful,
|
| 648 |
+
but WITHOUT ANY WARRANTY; without even the implied warranty of
|
| 649 |
+
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
| 650 |
+
GNU General Public License for more details.
|
| 651 |
+
|
| 652 |
+
You should have received a copy of the GNU General Public License
|
| 653 |
+
along with this program. If not, see <https://www.gnu.org/licenses/>.
|
| 654 |
+
|
| 655 |
+
Also add information on how to contact you by electronic and paper mail.
|
| 656 |
+
|
| 657 |
+
If the program does terminal interaction, make it output a short
|
| 658 |
+
notice like this when it starts in an interactive mode:
|
| 659 |
+
|
| 660 |
+
<program> Copyright (C) <year> <name of author>
|
| 661 |
+
This program comes with ABSOLUTELY NO WARRANTY; for details type `show w'.
|
| 662 |
+
This is free software, and you are welcome to redistribute it
|
| 663 |
+
under certain conditions; type `show c' for details.
|
| 664 |
+
|
| 665 |
+
The hypothetical commands `show w' and `show c' should show the appropriate
|
| 666 |
+
parts of the General Public License. Of course, your program's commands
|
| 667 |
+
might be different; for a GUI interface, you would use an "about box".
|
| 668 |
+
|
| 669 |
+
You should also get your employer (if you work as a programmer) or school,
|
| 670 |
+
if any, to sign a "copyright disclaimer" for the program, if necessary.
|
| 671 |
+
For more information on this, and how to apply and follow the GNU GPL, see
|
| 672 |
+
<https://www.gnu.org/licenses/>.
|
| 673 |
+
|
| 674 |
+
The GNU General Public License does not permit incorporating your program
|
| 675 |
+
into proprietary programs. If your program is a subroutine library, you
|
| 676 |
+
may consider it more useful to permit linking proprietary applications with
|
| 677 |
+
the library. If this is what you want to do, use the GNU Lesser General
|
| 678 |
+
Public License instead of this License. But first, please read
|
| 679 |
+
<https://www.gnu.org/licenses/why-not-lgpl.html>.
|
ComfyUI-MiniMaxH3-Context-Loop/ComfyUI-MAINodes/LOWVRAM.md
ADDED
|
@@ -0,0 +1,323 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Low VRAM, low RAM: the de-rope on a 16 GB card
|
| 2 |
+
|
| 3 |
+
Status 2026-08-18: alpha. Measured on one machine fenced down to a 16 GB
|
| 4 |
+
card / 32 GB box (how, and how the fence was read, at the bottom); not yet
|
| 5 |
+
run on a physically different 16 GB card, which is the next thing.
|
| 6 |
+
|
| 7 |
+
**Get started** (16 GB card, 32 GB machine; measured, alpha):
|
| 8 |
+
|
| 9 |
+
1. Start ComfyUI **without** `--gpu-only` and **with**
|
| 10 |
+
`--fast-disk --fp16-intermediates --disable-pinned-memory` (and
|
| 11 |
+
`--cache-none` if RAM is tight; it re-runs every node on each queue).
|
| 12 |
+
What each does, measured on the 702-frame de-rope (RSS peak, same VRAM and
|
| 13 |
+
speed in every case): fp32 IMAGEs and a pinned DiT = 41.9 GB; `--fp16-intermediates`
|
| 14 |
+
halves every held IMAGE = 31.8; `--disable-pinned-memory` drops the DiT's
|
| 15 |
+
~10 GB pinned staging copy = 21.6; `--cache-none` frees node outputs after
|
| 16 |
+
their consumers = 24.2 alone, **14.6 GB with all four**. `--fast-disk` reads
|
| 17 |
+
the weights from the NVMe page cache instead of RAM copies. Speed did not
|
| 18 |
+
move (369-374 s/step throughout).
|
| 19 |
+
2. Load `examples/motion_pipeline_lowvram.json` (API twin alongside), point
|
| 20 |
+
the LoadImage at your reference, set the prompt and canvas.
|
| 21 |
+
3. Queue. The 124-frame reference clip takes about 7 minutes end to end on
|
| 22 |
+
the fenced 15 GiB card (RSS 25 GB).
|
| 23 |
+
|
| 24 |
+
If `H3 Streamed Blocks` is not in the node search (category `MAINodes/VRAM
|
| 25 |
+
Lab`, display name "H3 Streamed Blocks (exact low-VRAM, alpha)"), look at
|
| 26 |
+
the ComfyUI console for one line at startup: `[MAINodes] vram_lab not
|
| 27 |
+
loaded: ...`. The module needs a recent ComfyUI (developed on 0.33.0; it
|
| 28 |
+
imports `comfy.ldm.minimax.model` and comfy_kitchen's native rope op via
|
| 29 |
+
`comfy.quant_ops.ck`) and PyTorch flash attention; the pack loads the rest
|
| 30 |
+
of its nodes without it and prints why. Paste that line in an issue.
|
| 31 |
+
|
| 32 |
+
**What the graph does to make a small card happy:**
|
| 33 |
+
|
| 34 |
+
- the DiT at 4-bit weights / 8-bit activations (`minimax_h3_ref2va_pruned_w4a8_mixed`, 11 GB), the text encoder in NVFP4, the video VAE as int8_convrot (2.2 GiB less than fp16, decode 1.5x faster, 60 dB PSNR against it);
|
| 35 |
+
- `H3 Streamed Blocks` runs every DiT block in 16k-token chunks so the whole-sequence QKV and SwiGLU tensors (8.6 and 15.4 GiB at 217k tokens) never exist, and streams the output head; same math as stock (bit-equal on int8 / W4A8 with the exact K/V store);
|
| 36 |
+
- **`kv_store` = kvi8r**: the K/V of the whole sequence held as rotated int8, forward peak +9.5 GiB at 217k tokens instead of +11.9 (with `trim_forward`, on by default, another 2.3 GiB comes off); a sibling take of the exact result that passed the operator's eyes on the de-rope ("perfect for both"); flip it to `bf16 (exact)` for the bit-equal path;
|
| 37 |
+
- `H3 Evict Text Encoder` after each prompt encode, `H3 Free Cache` before the pass-2 decode: the 15 GB TE and the allocator pool leave the card before the VAE needs it;
|
| 38 |
+
- the turbo LoRA with a 12-step base pass and a 6-step / inject 0.7 de-rope pass, so the long pass is a handful of forwards.
|
| 39 |
+
|
| 40 |
+
Everything below is the why and the numbers.
|
| 41 |
+
|
| 42 |
+
## The workflow that ships: `examples/motion_pipeline_lowvram.json`
|
| 43 |
+
|
| 44 |
+
On a Blackwell card there is a free speed lever on top of all of this: swap
|
| 45 |
+
the example's W4A8 UNET for the NVFP4 build
|
| 46 |
+
([MATLOWAI/minimax-h3-nvfp4](https://huggingface.co/MATLOWAI/minimax-h3-nvfp4)).
|
| 47 |
+
Same file size, measured 0.84 to 0.91x wall on 25-step graphs and 0.88x with
|
| 48 |
+
the turbo LoRA stacked, stock ComfyUI. A same-seed render is a sibling take;
|
| 49 |
+
judge it on your own content. Non-Blackwell cards should not use it.
|
| 50 |
+
|
| 51 |
+
Two things the example already includes that are easy to miss: the AUDIO INIT
|
| 52 |
+
(`H3 V2V Init` takes the source performance's audio latent at strength 0.5, so
|
| 53 |
+
pass 2 follows the original delivery) and the smear-aware audio recover. For
|
| 54 |
+
clips whose dilated pass does not fit your card in one piece, the same stack is
|
| 55 |
+
wired into the windowed graph:
|
| 56 |
+
[`examples/motion_pipeline_rolling_window_lowvram.json`](examples/motion_pipeline_rolling_window_lowvram.json)
|
| 57 |
+
(API twin alongside; plan report first, then queue per window; inject 0.48). Each
|
| 58 |
+
window pins its first and last frame to the baseline via `MiniMax H3 Add Guide`,
|
| 59 |
+
which holds the regenerated windows on the baseline's motion and look so the
|
| 60 |
+
splice seams carry no state jumps; the pin frames come straight from
|
| 61 |
+
`H3 Window Plan`'s first/last frame outputs.
|
| 62 |
+
|
| 63 |
+
One graph, the whole motion pipeline for a small card, with the rotated int8
|
| 64 |
+
K/V store on. What is in it and why, in the order it runs (2026-08-18):
|
| 65 |
+
|
| 66 |
+
| node | setting | what it buys |
|
| 67 |
+
|---|---|---|
|
| 68 |
+
| `UNETLoader` | `minimax_h3_ref2va_pruned_w4a8_mixed` (11 GB) | the DiT at 4-bit weights / 8-bit activations, exact under chunking (int32 accumulate) |
|
| 69 |
+
| `CLIPLoader` | `qwen3vl_32b_minimax_h3_nvfp4_awq` | the text encoder in NVFP4 (~15 GB file) |
|
| 70 |
+
| `VAELoader` (video) | `minimax_h3_video_vae_int8_convrot` | 2.2 GiB less on the card than fp16, decode 1.5x faster, 60 dB PSNR against the fp16 decoder on the same latent |
|
| 71 |
+
| `VAELoader` (audio) | `minimax_h3_audio_vae_fp32` | unchanged, 0.6 GB |
|
| 72 |
+
| `LoraLoaderModelOnly` | lightx2v turbo 4-step v1.0 768p, strength 1.0 | pass 1 at 12 steps (`linear_quadratic`), pass 2 on a 6-step `beta` schedule with `inject 0.7` (the "faithful detail" preset), so the long de-rope pass costs a handful of forwards |
|
| 73 |
+
| **`H3 Streamed Blocks`** | q/kv/mlp chunks 16384, `min_tokens` 32768, `final_layer_chunk` 16384, `final_layer_gemm` exact, **`kv_store` = kvi8r** | every DiT block in token chunks (never the whole-sequence QKV or SwiGLU), the output head streamed, and the K/V of the whole sequence held as rotated int8: forward peak +9.5 GiB at 217k tokens instead of +11.9 (exact store) or +21 (stock). Bit-equal to stock with `kv_store` = bf16; with kvi8r a sibling take that the operator judged "perfect for both" on the de-rope side by side |
|
| 74 |
+
| `H3 Evict Text Encoder` (x2) | after each prompt encode | the 15 GB TE leaves the card as soon as its conditioning exists (1 to 2.5 GiB of peak on a 16 GB card, ~3 s per new prompt to bring back) |
|
| 75 |
+
| `H3 Jerk Oracle`, `H3 Time Smear`, `H3 Audio Smear`, `H3 V2V Init` | `d_max 4`, dilation 4, audio 0.5 | the de-rope itself (see README) |
|
| 76 |
+
| `H3 Free Cache` | between the pass-2 sampler and its decodes | returns the allocator pool before the VAE runs (17 GiB on the long pass under `--gpu-only`; the small-card win is that decode never competes with a stale pool) |
|
| 77 |
+
| `H3 Exact Recover`, `H3 Audio Recover` | | back to real time, frame-exact |
|
| 78 |
+
|
| 79 |
+
Measured as it ships (kvi8r + int8 VAE) at the 15.0 GiB fence / 48 GB
|
| 80 |
+
cgroup: the 124-frame reference clip end to end in **6:56**, VRAM 15.4 (the
|
| 81 |
+
fence, filled during the TE load), torch reserved max 6.2 GiB, RSS 25.2 GB,
|
| 82 |
+
zero allocator trims; pass 1 14.2 s/step, the de-rope pass 56 s/step. (With
|
| 83 |
+
the exact store and the fp16 VAE at the earlier 15.4 fence: 6:14, RSS 24.8.)
|
| 84 |
+
On the 702-frame de-rope, a bigger graph than this example, the exact store
|
| 85 |
+
no longer fits at 15.0 GiB while kvi8r (374 s/step) and kvi8s (219) render.
|
| 86 |
+
|
| 87 |
+
The KJNodes sage-attention patches and FFN chunker from the parent graph
|
| 88 |
+
are not in it: sage attention is an approximation and the exactness gates
|
| 89 |
+
here were run on PyTorch's flash attention.
|
| 90 |
+
3. Set your prompt, reference image and canvas as usual.
|
| 91 |
+
|
| 92 |
+
## What to expect
|
| 93 |
+
|
| 94 |
+
**VRAM.** The activation peak inside a forward is card-independent:
|
| 95 |
+
`0.3 GiB + ~53 KB per packed token` (K/V buffers 28.7 KB/token, the residual
|
| 96 |
+
10.8, chunk transients and two tensors ComfyUI's forward keeps alive).
|
| 97 |
+
At 702 frames / 217k tokens that is +11.9 GiB. Dynamic VRAM fills whatever is
|
| 98 |
+
left with resident weights, so the process peak reads close to the card size
|
| 99 |
+
on every card; what decides whether a length fits is `weights it must keep +
|
| 100 |
+
activations + ~1 GB context`. Measured: 702 frames fits a 16 GB card, 906 does
|
| 101 |
+
not (275k tokens: +14.9 GiB of activations plus the minimum resident weights).
|
| 102 |
+
Frames here are 1376x768; other canvases scale by tokens
|
| 103 |
+
(`latent_frames x (W/32) x (H/32)`).
|
| 104 |
+
|
| 105 |
+
**RAM. This is the tight budget, not VRAM.** With `--fast-disk`, process RSS
|
| 106 |
+
was 27.4 GB on the 16 GB card and 25.8 GB on the 24 GB card for a single
|
| 107 |
+
702-frame pass, 36 to 40 GB for the full de-rope pipeline (it holds the pass-1
|
| 108 |
+
decodes, the smear IMAGE, the heat map: a 702-frame fp32 IMAGE is 8.9 GB), and
|
| 109 |
+
59 GB without `--fast-disk`. Add your own headroom: an idle Ubuntu desktop is
|
| 110 |
+
~2 GB, an idle Windows 11 desktop 4 to 8 GB, ComfyUI itself 3 to 5 GB before
|
| 111 |
+
any model. So, measured: a single 702-frame pass and the 124-frame example
|
| 112 |
+
graph fit a 16 GB card + 32 GB RAM (lean Linux; already paging on Windows);
|
| 113 |
+
**the full 702-frame de-rope pipeline does not fit 32 GB** (it was OOM-killed at
|
| 114 |
+
the smear under a 32 GB fence, 2026-08-18 19:00) and needs ~48 GB today, 64 GB
|
| 115 |
+
to be comfortable. Holding intermediates as fp16/uint8 by consumer type is
|
| 116 |
+
the next change and should bring the full pipeline back under 32 GB. Under our 64 GB fence
|
| 117 |
+
the kernel reclaimed page cache before killing anything; a Windows box would
|
| 118 |
+
pagefile instead, which is slower but alive.
|
| 119 |
+
|
| 120 |
+
**Time.** 316 to 318 s per pass-2 step at 217k tokens on the fenced 16, 24 and
|
| 121 |
+
32 GB cards versus 311 resident on 96 GB. On the 90-frame clip the 16 GB card
|
| 122 |
+
did 8.2 s/step. The text encoder costs 3 to 4.5 s per new prompt from a warm
|
| 123 |
+
page cache (a cold read of the 15 GB file is longer), and 1 to 2.5 GiB of
|
| 124 |
+
VRAM peak on a 16 GB card, which the evict node removes.
|
| 125 |
+
|
| 126 |
+
**The text encoder on tiny cards**, measured on the fenced 16 GB card, 90 frames:
|
| 127 |
+
|
| 128 |
+
| arm | wall | encode | VRAM peak | RSS |
|
| 129 |
+
|---|---|---|---|---|
|
| 130 |
+
| stock CLIPLoader | 221 s | 3.0 s | 15.7 | 14.3 |
|
| 131 |
+
| second prompt in the same process | 218 s | 1.7 s | 16.6 | 15.5 |
|
| 132 |
+
| `H3 Evict Text Encoder` after the encode | 221 s | 3.1 s | 13.9 | 14.7 |
|
| 133 |
+
| `CLIPLoader device=cpu` | 272 s | 55 s | 14.0 | 26.9 |
|
| 134 |
+
| conditioning bank hit (encoder never runs) | 218 s | none | 14.7 | 12.4 |
|
| 135 |
+
|
| 136 |
+
Nothing thrashed. Evict is the default in the example graph because a normal
|
| 137 |
+
run types a new prompt; `H3 Conditioning Bank` is for repeated prompts (seed
|
| 138 |
+
hunts, windows, extension chains) and needs the sampler's latent to come from
|
| 139 |
+
somewhere other than the encode node (`EmptyMiniMaxH3LatentAV` for T2VA, the
|
| 140 |
+
latent bank for a banked pass 1), or the encode node runs anyway for its
|
| 141 |
+
latent output.
|
| 142 |
+
|
| 143 |
+
## The numbers
|
| 144 |
+
|
| 145 |
+
The de-rope pass at `d_max 4` on an 8 to 12 second clip is ~200k packed tokens.
|
| 146 |
+
The stock H3 block builds its fused QKV and SwiGLU tensors for the whole
|
| 147 |
+
sequence at once, 8.6 and 15.4 GiB at that length, and that is what OOMs a
|
| 148 |
+
24 GB card, and at 1376x768 a 96 GB one. `H3 Streamed Blocks` runs every DiT
|
| 149 |
+
block in token chunks with the same math. Together with two things ComfyUI
|
| 150 |
+
already ships (dynamic VRAM, `--fast-disk`) it changes the small-card story
|
| 151 |
+
from "OOM" to "same speed":
|
| 152 |
+
|
| 153 |
+
| card | machine | 294 f -> 702 f de-rope at 1376x768 (~217k tokens) | s/step (pass 2) |
|
| 154 |
+
|---|---|---|---|
|
| 155 |
+
| 96 GB, everything resident (`--gpu-only`) | 91 GB | renders | 311 |
|
| 156 |
+
| 32 GB (fenced), normal mode, `--fast-disk` | 64 GB (cgroup) | renders, 25 GiB VRAM peak | 318 |
|
| 157 |
+
| 24 GB (fenced) | 32 GB (cgroup) | renders, 23.5 GiB VRAM peak, 25.8 GB RSS | 318 |
|
| 158 |
+
| **16 GB (fenced)** | **32 GB (cgroup)** | **renders, 15.5 GiB VRAM peak, 27.4 GB RSS** | **316** |
|
| 159 |
+
| **15.0 GiB fence (a 5070 Ti with its desktop up)** | 48 GB (cgroup) | exact store: **out of memory** at the second forward (+12.1 GiB needed above the resident weights, 1990 allocator trims first); **kvi8r: renders** (+9.5 GiB, 22:33); **kvi8s: renders** (+9.5 GiB, 14:50); **kvi8s + `trim_forward`: renders with zero allocator trims (+7.2 GiB, 14:51)** | exact - / kvi8r 374 / kvi8s 219 |
|
| 160 |
+
| 16 GB (fenced) | 32 or 64 GB | 906 f (~275k tokens): out of memory | |
|
| 161 |
+
| 24 GB (fenced) | 32 GB | 906 f: renders, RSS 29.8 GB (RAM at the edge) | 515 |
|
| 162 |
+
|
| 163 |
+
Same seed, the streamed result is bit-equal to the stock result on the int8
|
| 164 |
+
and W4A8 checkpoints (video and audio latents compared tensor by tensor). It
|
| 165 |
+
is not a quality dial. It is also not slower: at these lengths the step is
|
| 166 |
+
attention-bound and the weight traffic hides under it (see "Why no speed
|
| 167 |
+
penalty").
|
| 168 |
+
|
| 169 |
+
## How far you can go (extrapolated from the lines above)
|
| 170 |
+
|
| 171 |
+
The activation line predicted the 906-frame point on the 24 GB card before it
|
| 172 |
+
was measured (14.9 GiB both ways), so the table below is the line, not a
|
| 173 |
+
guess; treat it as +-1 frame-block. "Max tokens" assumes dynamic VRAM keeps
|
| 174 |
+
the minimum of resident weights (2 to 3 GB measured) and ~1 GB of context.
|
| 175 |
+
|
| 176 |
+
| card | max packed tokens | frames at 1376x768, single de-rope pass | RAM at that length, single pass / full pipeline |
|
| 177 |
+
|---|---|---|---|
|
| 178 |
+
| 16 GB | ~230k | ~740 (measured: 702 renders, 906 does not) | ~28 / ~40 GB |
|
| 179 |
+
| 24 GB | ~380k | ~1230 (measured: 906 renders at 515 s/step) | ~33 / ~45 GB |
|
| 180 |
+
| 32 GB | ~530k | ~1720 | ~39 / ~51 GB |
|
| 181 |
+
|
| 182 |
+
RAM per frame is ~12 MB (one fp32 IMAGE at 1376x768) on top of a ~19 GB base
|
| 183 |
+
for a single pass with `--fast-disk`; the full pipeline holds two to three
|
| 184 |
+
more IMAGEs of the clip. Past ~1100 frames the step time (attention grows with
|
| 185 |
+
the square of the sequence: 316 s at 702 frames, 515 at 906, ~650 at 1110)
|
| 186 |
+
is the practical limit before memory is. Other canvases: convert to tokens
|
| 187 |
+
with `latent_frames x (W/32) x (H/32)`, where `latent_frames = (frames-5)/17*5+2`.
|
| 188 |
+
|
| 189 |
+
**The example graph itself, end to end on the fenced 16 GB card in a 32 GB
|
| 190 |
+
machine** (2026-08-18 18:25): rendered in 6:14 wall. Pass 1 (Ref2VA, 12 base
|
| 191 |
+
steps, 124 frames) 147 s, oracle + smear + audio smear + VAE encode 30 s,
|
| 192 |
+
pass 2 (6 turbo steps) 150 s, decode 21 s, recover and three saves. Process
|
| 193 |
+
VRAM peak 16.1 GiB (dynamic VRAM sized itself to the card), RSS peak 24.8 GB,
|
| 194 |
+
cgroup peak 30.9 GB (RSS plus the reclaimable page cache of the weight files:
|
| 195 |
+
that is how little air a 32 GB box has even at 124 frames). The text encoder
|
| 196 |
+
held 13 GB of the card until `H3 Evict Text Encoder` freed it
|
| 197 |
+
(`device free 1.3 -> 14.6 GiB`).
|
| 198 |
+
|
| 199 |
+
## Why no speed penalty
|
| 200 |
+
|
| 201 |
+
At 217k tokens about 250 of the ~315 seconds per forward are attention, which
|
| 202 |
+
grows with the square of the sequence. Streaming the whole 11 to 12.5 GB DiT
|
| 203 |
+
once per forward is ~0.5 s over PCIe from RAM and ~2 s from an NVMe page
|
| 204 |
+
cache, and dynamic VRAM prefetches the next layers while the current ones
|
| 205 |
+
compute. Weight traffic is a fixed cost per forward; attention is quadratic;
|
| 206 |
+
the long clip hides the traffic completely, and on the short one prefetch
|
| 207 |
+
did the same (8.2 s/step on 16 GB versus ~9 resident on 96 GB). This box has
|
| 208 |
+
PCIe 5 x16 and a fast NVMe; a PCIe 4 desktop halves the bandwidth, which is
|
| 209 |
+
still nothing against a 300 s step. A machine whose RAM cannot hold the
|
| 210 |
+
weight file's page cache and whose SSD is slow could show it on short clips.
|
| 211 |
+
|
| 212 |
+
## The K/V store options (approximation tier, default off)
|
| 213 |
+
|
| 214 |
+
At 217k tokens the exact block's forward peak is +11.9 GiB above the resident
|
| 215 |
+
weights, and 5.8 GiB of that is the K/V of the whole sequence held in bf16
|
| 216 |
+
while the query chunks attend against it. `H3 Streamed Blocks` has a
|
| 217 |
+
`kv_store` option with two ways of halving those bytes. Neither is bit-equal
|
| 218 |
+
to stock: a same-seed render is a sibling take. The node's default is the
|
| 219 |
+
exact store; the low-VRAM example graph ships with kvi8r on (operator's
|
| 220 |
+
call); kvi8s has now been rendered and seen ("looks perfect", 219 s/step
|
| 221 |
+
against kvi8r's 374) and is the faster option for anyone with the
|
| 222 |
+
`sageattention` package installed; the example keeps kvi8r because it needs
|
| 223 |
+
no extra package.
|
| 224 |
+
|
| 225 |
+
| `kv_store` | what it is | forward peak at 217k tokens | s/step (pass 2, 702 f de-rope) | quality so far |
|
| 226 |
+
|---|---|---|---|---|
|
| 227 |
+
| `bf16 (exact)` | stock math, chunked | +11.9 GiB | 318 | bit-equal |
|
| 228 |
+
| `kvi8r: rotated int8 K/V` | K and V int8 with one fp16 scale per (token, head) row, in a fixed Hadamard-rotated basis of the head dim; the query chunk is rotated to match and the output un-rotated once, so a K/V block dequant is one int8->fp16 cast times a scale, no GEMM; attention in fp16 blockwise (`kv_block`, default 16384) with an online-softmax combine | **+9.5 GiB** (second cut, measured live 2026-08-18 evening; the first cut was +12.5) | **374** (first cut 400) | first cut: operator's eyes "almost perfect" on the de-rope side by side, "looks fine" on a 90 f T2VA; second cut: side by side rendered, verdict pending; sensor bank over seeds owed |
|
| 229 |
+
| `kvi8s: Sage int8/fp8 K/V, rotated` | the same bytes kept in SageAttention's kernel layout (int8 K with one scale per 64-token block after mean smoothing, fp8 e4m3 V per channel), Q and K Hadamard-rotated first, attended on int8/fp8 tensor cores straight from the store, no dequant; needs the `sageattention` package (2.x, sm120 works) | **+9.5 GiB** (measured live 2026-08-18 evening) | **219** (31% faster than the exact store, 42% faster than kvi8r) | one rung more approximate than kvi8r on a synthetic proxy (rel-rms 3.1e-2 vs 1.3e-2; the shipped `sageattn()` scores 5.5e-2 on the same inputs, so the rotation is worth 1.8x to Sage); operator's eyes on the de-rope side by side: "looks perfect" (one clip, one viewer; sensor bank over seeds owed) |
|
| 230 |
+
|
| 231 |
+
Why the rotation: the fixed orthonormal Hadamard spreads the outlier channels
|
| 232 |
+
of K and V over all 128 dims before rounding, so a per-row int8 scale wastes
|
| 233 |
+
less range; scores are invariant (q.k = (qH).(kH)) and P.V comes back through
|
| 234 |
+
H^T once on the output. Why fp16 and not bf16 inside kvi8r: in the rotated
|
| 235 |
+
basis every value has the same magnitude and bf16's 8-bit mantissa alone costs
|
| 236 |
+
a 4.8e-3 rel-rms floor; fp16's 11 bits cost 1.7e-3 (measured on random data).
|
| 237 |
+
|
| 238 |
+
## What is exact, and what is not
|
| 239 |
+
|
| 240 |
+
- `H3 Streamed Blocks` on int8 and W4A8 checkpoints: bit-equal to stock. The
|
| 241 |
+
mechanism is that comfy_kitchen quantises activations per row and the int8
|
| 242 |
+
GEMM accumulates in int32, which is associative, so chunking rows cannot
|
| 243 |
+
change the sum. NVFP4/FP8 would need one shared activation scale and are
|
| 244 |
+
untested; bf16 checkpoints are numerically equivalent (fp accumulation
|
| 245 |
+
order), not identical.
|
| 246 |
+
- Query chunking is exact only while every chunk keeps PyTorch's flash
|
| 247 |
+
kernel off its split-KV path (measured boundary `heads x ceil(L/64) >=
|
| 248 |
+
0.8 x 2 x SMs`; a 259-token tail once made a different clip). The node sizes
|
| 249 |
+
chunks from the SM count with a 2.6x margin and has a `self_check` that
|
| 250 |
+
compares stock and streamed on the first block's real input.
|
| 251 |
+
- The output head has an exact mode (default) and a chunked-GEMM mode that
|
| 252 |
+
differs by ~1e-6 per step in fp32 and produced a different clip after 25
|
| 253 |
+
steps. It is labelled and off. `kv_block` on the exact store is experimental
|
| 254 |
+
and as built does not lower memory; leave it at 0 (it is the block size for
|
| 255 |
+
the kvi8r store).
|
| 256 |
+
- `H3 Free Cache` and `H3 Evict Text Encoder` change no math.
|
| 257 |
+
- `trim_forward` (default on) runs a copy of the stock model forward that
|
| 258 |
+
releases the patch-embed buffers once the packed sequence is assembled
|
| 259 |
+
(stock holds them for the whole forward: 2.3 GiB at 217k tokens, measured
|
| 260 |
+
live). Same math, bit-equal on the patcher gate; it checks the installed
|
| 261 |
+
ComfyUI's forward against the copy by source hash and switches itself off
|
| 262 |
+
with a log line if ComfyUI has changed. With kvi8s it takes the forward
|
| 263 |
+
peak to +7.2 GiB (the streamed output head is now the peak, not attention).
|
| 264 |
+
- The int8_convrot video VAE is an approximation by construction (60 dB PSNR
|
| 265 |
+
against the fp16 decoder on the same latent, encoder identical); the fp16
|
| 266 |
+
VAE is what the exactness gates were run with.
|
| 267 |
+
|
| 268 |
+
## What is next (the roadmap, in the order it will be tested)
|
| 269 |
+
|
| 270 |
+
1. **Host RAM was the binding limit on the small machine; four flags moved it.**
|
| 271 |
+
Every held IMAGE is 8.9 GB per 702 frames at 1376x768 in fp32, ComfyUI pins
|
| 272 |
+
a ~10 GB staging copy of the DiT, and the execution cache holds every
|
| 273 |
+
output until the prompt ends: 41.9 GB. With `--fp16-intermediates
|
| 274 |
+
--disable-pinned-memory --cache-none` (all measured one at a time and
|
| 275 |
+
together) the same pipeline peaks at 14.6 GB. Note the fp16 IMAGE is
|
| 276 |
+
normalised in fp16 before the encoder, so the pass-2 result is a sibling
|
| 277 |
+
take of the fp32 run (28 dB between them, judged equal by eye). What is
|
| 278 |
+
left for code: streaming the smear straight into the encoder (never
|
| 279 |
+
holding the dilated IMAGE), uint8 for view-only outputs, and an fp32
|
| 280 |
+
normalise on the encode input if anyone wants the fp32 sibling back. VAE
|
| 281 |
+
activations themselves are not the problem (0.6 GiB decode / 2.2 GiB
|
| 282 |
+
encode transient at the 256 px tiles).
|
| 283 |
+
2. **kvi8s live**, then the sensor bank over seeds for both stores; a per-32-value
|
| 284 |
+
scale variant of kvi8r (finer than per row); fp8 K/V if a kernel takes it
|
| 285 |
+
without dequant; int4/nvfp4 K/V behind the same gate.
|
| 286 |
+
3. **The weight side**: where the streamed block forces extra weight decode
|
| 287 |
+
per token chunk (W4A8's int4->int8 decode is repeated per chunk: measured
|
| 288 |
+
+25% wall at 29k tokens, +6% at 217k); decode once per block and reuse,
|
| 289 |
+
larger chunks where the flash split-KV rule allows, kitchen's fused paths
|
| 290 |
+
(SwiGLU fold, `convrot_w4a4` for the MLP).
|
| 291 |
+
4. **A real 16 GB card**, someone else's, end to end on the example graph.
|
| 292 |
+
5. Later, after the static rungs: dynamic precision routing (per block, per
|
| 293 |
+
step, FP4 vs FP8 by measured reconstruction error; the first experiment is
|
| 294 |
+
a heatmap on real activations, no kernels), and the ViT VAE decoder's own
|
| 295 |
+
low-bit rungs (already on int8 tensor cores in the int8_convrot file; the
|
| 296 |
+
remaining case is speed, ~40 s per 702 f decode).
|
| 297 |
+
|
| 298 |
+
## The environment this was measured in
|
| 299 |
+
|
| 300 |
+
Everything below was measured on one
|
| 301 |
+
machine, fenced down to look like small ones (real renders on the real GPU with
|
| 302 |
+
the VRAM held by a balloon and the RAM capped by a cgroup). **How the fence
|
| 303 |
+
was read:** the first day's "16 GB" rows ran with 15.4 GiB device-free as
|
| 304 |
+
`torch.cuda.mem_get_info` sees it (a headless 5070 Ti reports 15.92 GiB total
|
| 305 |
+
and about 15.5 free); the evening's kvi8r run used a stricter 15.0 GiB fence,
|
| 306 |
+
which is a 5070 Ti with its desktop attached. Dynamic VRAM fills whatever the
|
| 307 |
+
card has, so process peaks always read as the fence; the numbers that carry a
|
| 308 |
+
memory claim are the in-process forward peaks from `H3 Memory Probe`. Nothing
|
| 309 |
+
has been run on a physically different 16 GB card yet; that is the next thing,
|
| 310 |
+
and it is a dogfood ask, not a research question.
|
| 311 |
+
|
| 312 |
+
One workstation: two RTX PRO 6000 Blackwell (96 GB, 188 SMs), EPYC 9554,
|
| 313 |
+
91 GB RAM, NVMe, Ubuntu, ComfyUI 0.33.0, PyTorch 2.14 nightly cu132,
|
| 314 |
+
comfy_kitchen native ops, PyTorch flash attention (comfy default), MiniMax-H3
|
| 315 |
+
W4A8 and int8 checkpoints. Small cards were simulated by holding VRAM with a
|
| 316 |
+
balloon process so ComfyUI, dynamic VRAM and torch all saw a real 16 / 24 /
|
| 317 |
+
32 GB card; small machines by running ComfyUI in a cgroup with a hard memory
|
| 318 |
+
limit and swap off. Only ComfyUI ran during the measurements. Not measured:
|
| 319 |
+
Windows, other GPUs, ROCm, non-flash attention backends, real PCIe 4 boxes.
|
| 320 |
+
|
| 321 |
+
The graph of the lines (VRAM activation line with per-card ceilings, RSS by
|
| 322 |
+
graph type) is `LOWVRAM_lines.html` next to this file; it grows as the sweep
|
| 323 |
+
does.
|
ComfyUI-MiniMaxH3-Context-Loop/ComfyUI-MAINodes/LOWVRAM_lines.html
ADDED
|
@@ -0,0 +1,53 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
<!doctype html><meta charset="utf-8"><title>H3 De-Rope Memory Lines</title>
|
| 2 |
+
<style>
|
| 3 |
+
:root{--bg:#ffffff;--ink:#1a1a19;--ink2:#5c5b55;--ink3:#8a8982;--grid:#ececea;--rule:#c8c7c0;--torch:#2a78d6;--c16:#eb6834;--c24:#1baf7a;--c31:#eda100;--c32:#e87ba4;--single:#2a78d6;--pipe:#eb6834;--nofd:#4a3aa7}
|
| 4 |
+
@media (prefers-color-scheme: dark){:root:not([data-theme="light"]){--bg:#1a1a19;--ink:#ffffff;--ink2:#c3c2b7;--ink3:#8f8e86;--grid:#2c2c2a;--rule:#4a4a46;--torch:#3987e5;--c16:#d95926;--c24:#199e70;--c31:#c98500;--c32:#d55181;--single:#3987e5;--pipe:#d95926;--nofd:#9085e9}}
|
| 5 |
+
:root[data-theme="dark"]{--bg:#1a1a19;--ink:#ffffff;--ink2:#c3c2b7;--ink3:#8f8e86;--grid:#2c2c2a;--rule:#4a4a46;--torch:#3987e5;--c16:#d95926;--c24:#199e70;--c31:#c98500;--c32:#d55181;--single:#3987e5;--pipe:#d95926;--nofd:#9085e9}
|
| 6 |
+
body{background:var(--bg);color:var(--ink);font:14px/1.5 system-ui,sans-serif;margin:24px;max-width:960px}
|
| 7 |
+
h1{font-size:22px;margin:0 0 4px} p{color:var(--ink2);max-width:70ch} svg{width:100%;height:auto;display:block;margin:12px 0 28px}
|
| 8 |
+
.t1{font-size:15px;font-weight:600;fill:var(--ink)} .t3{font-size:11px;fill:var(--ink2)} .t4{font-size:10px;fill:var(--ink3)}
|
| 9 |
+
.grid{stroke:var(--grid)} .rule{stroke-dasharray:4 3;stroke-width:1} .pt{stroke:var(--bg);stroke-width:2;cursor:default}
|
| 10 |
+
table{border-collapse:collapse;font-size:12px;font-variant-numeric:tabular-nums} th,td{border-bottom:1px solid var(--grid);padding:4px 8px;text-align:left} th{color:var(--ink2);font-weight:600}
|
| 11 |
+
</style>
|
| 12 |
+
<h1>How long can a de-rope go on your card</h1>
|
| 13 |
+
<p>Measured 2026-08-18 on one machine (RTX PRO 6000 Blackwell fenced to smaller cards with a VRAM balloon, RAM fenced with cgroups), MiniMax-H3 W4A8 DiT, MAINodes <code>H3 Streamed Blocks</code> (exact), ComfyUI dynamic VRAM, <code>--fast-disk</code>, PyTorch flash attention. Hover a point. Frames are at 1376x768; other canvases scale by tokens.</p>
|
| 14 |
+
<svg viewBox="0 0 900 400" role="img" aria-label="VRAM: activations are card-independent; the card only sets the ceiling"><text x="70" y="18" class="t1">VRAM: activations are card-independent; the card only sets the ceiling</text><line x1="70" x2="860" y1="330.0" y2="330.0" class="grid"/><text x="62" y="334.0" text-anchor="end" class="t3">0</text><line x1="70" x2="860" y1="285.0" y2="285.0" class="grid"/><text x="62" y="289.0" text-anchor="end" class="t3">6</text><line x1="70" x2="860" y1="240.0" y2="240.0" class="grid"/><text x="62" y="244.0" text-anchor="end" class="t3">12</text><line x1="70" x2="860" y1="195.0" y2="195.0" class="grid"/><text x="62" y="199.0" text-anchor="end" class="t3">18</text><line x1="70" x2="860" y1="150.0" y2="150.0" class="grid"/><text x="62" y="154.0" text-anchor="end" class="t3">24</text><line x1="70" x2="860" y1="105.0" y2="105.0" class="grid"/><text x="62" y="109.0" text-anchor="end" class="t3">30</text><line x1="70" x2="860" y1="60.0" y2="60.0" class="grid"/><text x="62" y="64.0" text-anchor="end" class="t3">36</text><text x="70.0" y="346" text-anchor="middle" class="t3">0</text><text x="70.0" y="360" text-anchor="middle" class="t4">0k</text><text x="204.9" y="346" text-anchor="middle" class="t3">294</text><text x="204.9" y="360" text-anchor="middle" class="t4">90k</text><text x="392.1" y="346" text-anchor="middle" class="t3">702</text><text x="392.1" y="360" text-anchor="middle" class="t4">214k</text><text x="485.6" y="346" text-anchor="middle" class="t3">906</text><text x="485.6" y="360" text-anchor="middle" class="t4">276k</text><text x="579.2" y="346" text-anchor="middle" class="t3">1110</text><text x="579.2" y="360" text-anchor="middle" class="t4">339k</text><text x="719.6" y="346" text-anchor="middle" class="t3">1416</text><text x="719.6" y="360" text-anchor="middle" class="t4">432k</text><text x="860.0" y="346" text-anchor="middle" class="t3">1722</text><text x="860.0" y="360" text-anchor="middle" class="t4">525k</text><text x="465" y="376" text-anchor="middle" class="t3">frames at 1376x768 (packed tokens below)</text><text transform="rotate(-90)" x="-180" y="16" text-anchor="middle" class="t3">GiB</text><line x1="70" x2="860" y1="210.0" y2="210.0" class="rule" style="stroke:var(--c16)"/><text x="864" y="214.0" class="t3">16 GB</text><line x1="70" x2="860" y1="150.0" y2="150.0" class="rule" style="stroke:var(--c24)"/><text x="864" y="154.0" class="t3">24 GB</text><line x1="70" x2="860" y1="90.0" y2="90.0" class="rule" style="stroke:var(--c32)"/><text x="864" y="94.0" class="t3">32 GB</text><circle cx="111.3" cy="316.5" r="5" style="fill:var(--torch)" class="pt"><title>torch activation peak (in-forward, any card)
|
| 15 |
+
90 f / 28k tokens: torch peak +1.8 GiB (TE matrix A, 25 steps)</title></circle><circle cx="111.3" cy="316.5" r="5" style="fill:var(--torch)" class="pt"><title>torch activation peak (in-forward, any card)
|
| 16 |
+
90 f / 28k tokens: torch peak +1.8 GiB (TE matrix A, 25 steps)</title></circle><circle cx="392.1" cy="240.8" r="5" style="fill:var(--torch)" class="pt"><title>torch activation peak (in-forward, any card)
|
| 17 |
+
702 f / 214k tokens: torch peak +11.9 GiB (v014: bank, fast-disk, balloon 63)</title></circle><circle cx="392.1" cy="240.8" r="5" style="fill:var(--torch)" class="pt"><title>torch activation peak (in-forward, any card)
|
| 18 |
+
702 f / 214k tokens: torch peak +11.9 GiB (v012: planner+cap card)</title></circle><circle cx="392.1" cy="240.8" r="5" style="fill:var(--torch)" class="pt"><title>torch activation peak (in-forward, any card)
|
| 19 |
+
702 f / 214k tokens: torch peak +11.9 GiB (v011: normal mode, no fast-disk)</title></circle><circle cx="392.1" cy="240.8" r="5" style="fill:var(--torch)" class="pt"><title>torch activation peak (in-forward, any card)
|
| 20 |
+
702 f / 214k tokens: torch peak +11.9 GiB (line sweep, 2 steps)</title></circle><circle cx="392.1" cy="240.8" r="5" style="fill:var(--torch)" class="pt"><title>torch activation peak (in-forward, any card)
|
| 21 |
+
702 f / 214k tokens: torch peak +11.9 GiB (line sweep, 2 steps)</title></circle><circle cx="392.1" cy="240.8" r="5" style="fill:var(--torch)" class="pt"><title>torch activation peak (in-forward, any card)
|
| 22 |
+
702 f / 214k tokens: torch peak +11.9 GiB (line sweep, 2 steps)</title></circle><circle cx="392.1" cy="240.8" r="5" style="fill:var(--torch)" class="pt"><title>torch activation peak (in-forward, any card)
|
| 23 |
+
702 f / 214k tokens: torch peak +11.9 GiB (line sweep, 2 steps)</title></circle><circle cx="485.6" cy="218.2" r="5" style="fill:var(--torch)" class="pt"><title>torch activation peak (in-forward, any card)
|
| 24 |
+
906 f / 276k tokens: torch peak +14.9 GiB (line sweep, 2 steps)</title></circle><path d="M111.3,316.2 L204.9,291.5 L392.1,242.0 L485.6,217.3 L579.2,192.6 L719.6,155.5 L860.0,118.4" fill="none" style="stroke:var(--torch)" stroke-width="2"/><circle cx="111.3" cy="316.2" r="5" style="fill:var(--torch)" class="pt"><title>fit
|
| 25 |
+
fit: 0.34 + 53 KB/token</title></circle><circle cx="204.9" cy="291.5" r="5" style="fill:var(--torch)" class="pt"><title>fit
|
| 26 |
+
fit: 0.34 + 53 KB/token</title></circle><circle cx="392.1" cy="242.0" r="5" style="fill:var(--torch)" class="pt"><title>fit
|
| 27 |
+
fit: 0.34 + 53 KB/token</title></circle><circle cx="485.6" cy="217.3" r="5" style="fill:var(--torch)" class="pt"><title>fit
|
| 28 |
+
fit: 0.34 + 53 KB/token</title></circle><circle cx="579.2" cy="192.6" r="5" style="fill:var(--torch)" class="pt"><title>fit
|
| 29 |
+
fit: 0.34 + 53 KB/token</title></circle><circle cx="719.6" cy="155.5" r="5" style="fill:var(--torch)" class="pt"><title>fit
|
| 30 |
+
fit: 0.34 + 53 KB/token</title></circle><circle cx="860.0" cy="118.4" r="5" style="fill:var(--torch)" class="pt"><title>fit
|
| 31 |
+
fit: 0.34 + 53 KB/token</title></circle><circle cx="111.3" cy="212.2" r="5" style="fill:var(--c16)" class="pt"><title>16 GB card, process VRAM peak
|
| 32 |
+
90 f: process VRAM peak 15.7 GiB on a 16 GB card (TE matrix A, 25 steps)</title></circle><circle cx="392.1" cy="213.8" r="5" style="fill:var(--c16)" class="pt"><title>16 GB card, process VRAM peak
|
| 33 |
+
702 f: process VRAM peak 15.5 GiB on a 16 GB card (line sweep, 2 steps)</title></circle><circle cx="392.1" cy="213.8" r="5" style="fill:var(--c16)" class="pt"><title>16 GB card, process VRAM peak
|
| 34 |
+
702 f: process VRAM peak 15.5 GiB on a 16 GB card (line sweep, 2 steps)</title></circle><circle cx="111.3" cy="157.5" r="5" style="fill:var(--c24)" class="pt"><title>24 GB card, process VRAM peak
|
| 35 |
+
90 f: process VRAM peak 23.0 GiB on a 24 GB card (TE matrix A, 25 steps)</title></circle><circle cx="392.1" cy="153.8" r="5" style="fill:var(--c24)" class="pt"><title>24 GB card, process VRAM peak
|
| 36 |
+
702 f: process VRAM peak 23.5 GiB on a 24 GB card (line sweep, 2 steps)</title></circle><circle cx="392.1" cy="154.5" r="5" style="fill:var(--c24)" class="pt"><title>24 GB card, process VRAM peak
|
| 37 |
+
702 f: process VRAM peak 23.4 GiB on a 24 GB card (line sweep, 2 steps)</title></circle><circle cx="485.6" cy="154.5" r="5" style="fill:var(--c24)" class="pt"><title>24 GB card, process VRAM peak
|
| 38 |
+
906 f: process VRAM peak 23.4 GiB on a 24 GB card (line sweep, 2 steps)</title></circle><circle cx="392.1" cy="141.0" r="5" style="fill:var(--c31)" class="pt"><title>31 GB card, process VRAM peak
|
| 39 |
+
702 f: process VRAM peak 25.2 GiB on a 31 GB card (v014: bank, fast-disk, balloon 63)</title></circle><circle cx="392.1" cy="75.8" r="5" style="fill:var(--c32)" class="pt"><title>32 GB card, process VRAM peak
|
| 40 |
+
702 f: process VRAM peak 33.9 GiB on a 32 GB card (v012: planner+cap card)</title></circle><circle cx="392.1" cy="75.0" r="5" style="fill:var(--c32)" class="pt"><title>32 GB card, process VRAM peak
|
| 41 |
+
702 f: process VRAM peak 34.0 GiB on a 32 GB card (v011: normal mode, no fast-disk)</title></circle><rect x="70" y="388" width="12" height="12" rx="2" style="fill:var(--torch)"/><text x="86" y="398" class="t3">torch activation peak (in-forward, any card)</text><rect x="418" y="388" width="12" height="12" rx="2" style="fill:var(--torch)"/><text x="434" y="398" class="t3">fit</text><rect x="479" y="388" width="12" height="12" rx="2" style="fill:var(--c16)"/><text x="495" y="398" class="t3">16 GB card, process VRAM peak</text><rect x="722" y="388" width="12" height="12" rx="2" style="fill:var(--c24)"/><text x="738" y="398" class="t3">24 GB card, process VRAM peak</text><rect x="965" y="388" width="12" height="12" rx="2" style="fill:var(--c31)"/><text x="981" y="398" class="t3">31 GB card, process VRAM peak</text><rect x="1208" y="388" width="12" height="12" rx="2" style="fill:var(--c32)"/><text x="1224" y="398" class="t3">32 GB card, process VRAM peak</text><text x="70" y="392" class="t4">fit: torch peak = 0.34 GiB + 53 KB per packed token (K/V 28.7 + x 10.8 + residents/chunks). Points not rendered are listed in the table.</text></svg><svg viewBox="0 0 900 400" role="img" aria-label="Host RAM (process RSS peak): add your own OS + ComfyUI headroom"><text x="70" y="18" class="t1">Host RAM (process RSS peak): add your own OS + ComfyUI headroom</text><line x1="70" x2="860" y1="330.0" y2="330.0" class="grid"/><text x="62" y="334.0" text-anchor="end" class="t3">0</text><line x1="70" x2="860" y1="282.9" y2="282.9" class="grid"/><text x="62" y="286.9" text-anchor="end" class="t3">11</text><line x1="70" x2="860" y1="235.7" y2="235.7" class="grid"/><text x="62" y="239.7" text-anchor="end" class="t3">22</text><line x1="70" x2="860" y1="188.6" y2="188.6" class="grid"/><text x="62" y="192.6" text-anchor="end" class="t3">33</text><line x1="70" x2="860" y1="141.4" y2="141.4" class="grid"/><text x="62" y="145.4" text-anchor="end" class="t3">44</text><line x1="70" x2="860" y1="94.3" y2="94.3" class="grid"/><text x="62" y="98.3" text-anchor="end" class="t3">55</text><line x1="70" x2="860" y1="47.1" y2="47.1" class="grid"/><text x="62" y="51.1" text-anchor="end" class="t3">66</text><text x="70.0" y="346" text-anchor="middle" class="t3">0</text><text x="70.0" y="360" text-anchor="middle" class="t4">0k</text><text x="204.9" y="346" text-anchor="middle" class="t3">294</text><text x="204.9" y="360" text-anchor="middle" class="t4">90k</text><text x="392.1" y="346" text-anchor="middle" class="t3">702</text><text x="392.1" y="360" text-anchor="middle" class="t4">214k</text><text x="485.6" y="346" text-anchor="middle" class="t3">906</text><text x="485.6" y="360" text-anchor="middle" class="t4">276k</text><text x="579.2" y="346" text-anchor="middle" class="t3">1110</text><text x="579.2" y="360" text-anchor="middle" class="t4">339k</text><text x="719.6" y="346" text-anchor="middle" class="t3">1416</text><text x="719.6" y="360" text-anchor="middle" class="t4">432k</text><text x="860.0" y="346" text-anchor="middle" class="t3">1722</text><text x="860.0" y="360" text-anchor="middle" class="t4">525k</text><text x="465" y="376" text-anchor="middle" class="t3">frames at 1376x768 (packed tokens below)</text><text transform="rotate(-90)" x="-180" y="16" text-anchor="middle" class="t3">GB</text><line x1="70" x2="860" y1="192.9" y2="192.9" class="rule" style="stroke:var(--c32)"/><text x="864" y="196.9" class="t3">32 GB</text><line x1="70" x2="860" y1="55.7" y2="55.7" class="rule" style="stroke:var(--c24)"/><text x="864" y="59.7" class="t3">64 GB</text><circle cx="111.3" cy="268.7" r="5" style="fill:var(--single)" class="pt"><title>single T2VA pass, --fast-disk
|
| 42 |
+
90 f: RSS 14.3 GB, single pass, 16 GB card, 64 GB machine (TE matrix A, 25 steps)</title></circle><circle cx="111.3" cy="289.7" r="5" style="fill:var(--single)" class="pt"><title>single T2VA pass, --fast-disk
|
| 43 |
+
90 f: RSS 9.4 GB, single pass, 24 GB card, 64 GB machine (TE matrix A, 25 steps)</title></circle><circle cx="392.1" cy="212.6" r="5" style="fill:var(--single)" class="pt"><title>single T2VA pass, --fast-disk
|
| 44 |
+
702 f: RSS 27.4 GB, single pass, 16 GB card, 32 GB machine (line sweep, 2 steps)</title></circle><circle cx="392.1" cy="212.6" r="5" style="fill:var(--single)" class="pt"><title>single T2VA pass, --fast-disk
|
| 45 |
+
702 f: RSS 27.4 GB, single pass, 16 GB card, 64 GB machine (line sweep, 2 steps)</title></circle><circle cx="392.1" cy="219.4" r="5" style="fill:var(--single)" class="pt"><title>single T2VA pass, --fast-disk
|
| 46 |
+
702 f: RSS 25.8 GB, single pass, 24 GB card, 32 GB machine (line sweep, 2 steps)</title></circle><circle cx="392.1" cy="219.4" r="5" style="fill:var(--single)" class="pt"><title>single T2VA pass, --fast-disk
|
| 47 |
+
702 f: RSS 25.8 GB, single pass, 24 GB card, 64 GB machine (line sweep, 2 steps)</title></circle><circle cx="485.6" cy="202.3" r="5" style="fill:var(--single)" class="pt"><title>single T2VA pass, --fast-disk
|
| 48 |
+
906 f: RSS 29.8 GB, single pass, 24 GB card, 32 GB machine (line sweep, 2 steps)</title></circle><circle cx="392.1" cy="157.7" r="5" style="fill:var(--pipe)" class="pt"><title>full de-rope pipeline, --fast-disk
|
| 49 |
+
702 f: RSS 40.2 GB, de-rope pipeline, 31 GB card (v014: bank, fast-disk, balloon 63)</title></circle><circle cx="392.1" cy="174.0" r="5" style="fill:var(--pipe)" class="pt"><title>full de-rope pipeline, --fast-disk
|
| 50 |
+
702 f: RSS 36.4 GB, de-rope pipeline, 32 GB card (v012: planner+cap card)</title></circle><circle cx="392.1" cy="75.4" r="5" style="fill:var(--nofd)" class="pt"><title>de-rope pipeline, no --fast-disk
|
| 51 |
+
702 f: RSS 59.4 GB WITHOUT --fast-disk (v011: normal mode, no fast-disk)</title></circle><rect x="70" y="388" width="12" height="12" rx="2" style="fill:var(--single)"/><text x="86" y="398" class="t3">single T2VA pass, --fast-disk</text><rect x="313" y="388" width="12" height="12" rx="2" style="fill:var(--pipe)"/><text x="329" y="398" class="t3">full de-rope pipeline, --fast-disk</text><rect x="591" y="388" width="12" height="12" rx="2" style="fill:var(--nofd)"/><text x="607" y="398" class="t3">de-rope pipeline, no --fast-disk</text><text x="70" y="392" class="t4">RSS with --fast-disk = ComfyUI base + CPU-side intermediates (a 702-frame fp32 IMAGE is 8.9 GB); without it, add every model's CPU copy (~33 GB here).</text></svg>
|
| 52 |
+
<h2 style="font-size:16px">Points</h2><table><thead><tr><th>frames</th><th>tokens</th><th>card</th><th>machine</th><th>graph</th><th>torch peak</th><th>VRAM peak</th><th>RSS</th><th>note</th></tr></thead><tbody><tr><td>90</td><td>28k</td><td>16 GB</td><td>64 GB</td><td>single pass</td><td>1.8</td><td>15.7</td><td>14.3</td><td>TE matrix A, 25 steps</td></tr><tr><td>90</td><td>28k</td><td>24 GB</td><td>64 GB</td><td>single pass</td><td>1.8</td><td>23.0</td><td>9.4</td><td>TE matrix A, 25 steps</td></tr><tr><td>702</td><td>214k</td><td>16 GB</td><td>32 GB</td><td>single pass</td><td>11.9</td><td>15.5</td><td>27.4</td><td>line sweep, 2 steps</td></tr><tr><td>702</td><td>214k</td><td>16 GB</td><td>64 GB</td><td>single pass</td><td>11.9</td><td>15.5</td><td>27.4</td><td>line sweep, 2 steps</td></tr><tr><td>702</td><td>214k</td><td>24 GB</td><td>32 GB</td><td>single pass</td><td>11.9</td><td>23.5</td><td>25.8</td><td>line sweep, 2 steps</td></tr><tr><td>702</td><td>214k</td><td>24 GB</td><td>64 GB</td><td>single pass</td><td>11.9</td><td>23.4</td><td>25.8</td><td>line sweep, 2 steps</td></tr><tr><td>702</td><td>214k</td><td>31 GB</td><td>64 GB</td><td>de-rope pipeline</td><td>11.9</td><td>25.2</td><td>40.2</td><td>v014: bank, fast-disk, balloon 63</td></tr><tr><td>702</td><td>214k</td><td>32 GB</td><td>64 GB</td><td>de-rope pipeline</td><td>11.9</td><td>33.9</td><td>36.4</td><td>v012: planner+cap card</td></tr><tr><td>702</td><td>214k</td><td>32 GB</td><td>96 GB</td><td>de-rope pipeline (no fast-disk)</td><td>11.9</td><td>34.0</td><td>59.4</td><td>v011: normal mode, no fast-disk</td></tr><tr><td>906</td><td>276k</td><td>16 GB</td><td>32 GB</td><td>single pass</td><td>-</td><td>-</td><td>-</td><td>OOM</td></tr><tr><td>906</td><td>276k</td><td>16 GB</td><td>64 GB</td><td>single pass</td><td>-</td><td>-</td><td>-</td><td>OOM</td></tr><tr><td>906</td><td>276k</td><td>24 GB</td><td>32 GB</td><td>single pass</td><td>14.9</td><td>23.4</td><td>29.8</td><td>line sweep, 2 steps</td></tr></tbody></table>
|
| 53 |
+
<p>Reading it: VRAM needed = resident weights (dynamic VRAM shrinks these to a few GB when it must) + the blue line + ~1 GB context; a card fits while that stays under it. RAM needed = the RSS series for your graph type + your OS and ComfyUI headroom, which you know and we do not.</p>
|
ComfyUI-MiniMaxH3-Context-Loop/ComfyUI-MAINodes/README.md
ADDED
|
@@ -0,0 +1,1119 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# ComfyUI-MAINodes
|
| 2 |
+
|
| 3 |
+
Custom nodes for MiniMax-H3. Two groups: **Motion Lab** (a test-time fix
|
| 4 |
+
for fast-motion smearing) and **Contact-Sheet diffusion** (five views of
|
| 5 |
+
one subject from one reference image).
|
| 6 |
+
|
| 7 |
+
Like it? A star helps. Want to feed the GPU? [Sponsor the experiments](#support-mainodes).
|
| 8 |
+
|
| 9 |
+
## Start here
|
| 10 |
+
|
| 11 |
+
- **Which graph:** [`examples/README.md`](examples/README.md). Four
|
| 12 |
+
starting points, the essential node chain as a diagram, and what each
|
| 13 |
+
specialised graph adds to it.
|
| 14 |
+
- **Which settings for your card:** [`HARDWARE.md`](HARDWARE.md). VRAM and
|
| 15 |
+
system RAM rows, measured.
|
| 16 |
+
- **Which dial for which symptom:** [`TUNING.md`](TUNING.md).
|
| 17 |
+
- **What is still alpha:** [`ALPHA.md`](ALPHA.md).
|
| 18 |
+
|
| 19 |
+
The short version: start from
|
| 20 |
+
[`examples/motion_pipeline_ref2va_audioinit.json`](examples/motion_pipeline_ref2va_audioinit.json),
|
| 21 |
+
keep pass 1 at 12 steps on the base model, let the turbo LoRA do pass 2,
|
| 22 |
+
and reach for `inject` (0.50 sharper, 0.70 safer) before any other dial.
|
| 23 |
+
|
| 24 |
+
## What's new
|
| 25 |
+
|
| 26 |
+
Cloned before 2026-08-19 and `git pull` errors? Run
|
| 27 |
+
`git fetch origin && git reset --hard origin/main`, or re-clone.
|
| 28 |
+
|
| 29 |
+
**Repair, a per-block clock, and the chaining nodes** (2026-08-24, alpha).
|
| 30 |
+
Five additions, all opt-in and all alpha. `H3 Repair Plan` / `H3 Repair
|
| 31 |
+
Splice` re-render a few bad frames of a finished clip and hand back the same
|
| 32 |
+
clip with only those frames different ([below](#repairing-a-few-bad-frames)).
|
| 33 |
+
`H3 DyRoPE` gives the true-duration RoPE clock to SOME blocks or SOME steps
|
| 34 |
+
instead of all of them, which is what separates the timing correction from
|
| 35 |
+
the flash and jitter it used to cost
|
| 36 |
+
([below](#dyrope-who-gets-the-true-clock)). `H3 V2V Init` grew
|
| 37 |
+
`audio_prefix_ticks`, the audio twin of the video prefix freeze, so a
|
| 38 |
+
continuation can carry the previous segment's sound across the join
|
| 39 |
+
([below](#anchoring-audio-across-a-join-alpha)). `H3 Drift Control` and
|
| 40 |
+
`H3 Delta Color Carry` attack the two things that decay down a chain of
|
| 41 |
+
segments, clean-conditioning error and VAE round-trip colour
|
| 42 |
+
([below](#chained-segments-drift-and-colour-alpha)). And window mode, the
|
| 43 |
+
excerpt-shaped graph where only part of a long clip is retimed, finally has
|
| 44 |
+
a page: [docs/WINDOW_MODE.md](docs/WINDOW_MODE.md).
|
| 45 |
+
|
| 46 |
+
**Long de-ropes on small cards** (2026-08-18, alpha). A de-rope pass at
|
| 47 |
+
d_max 4 on an 8 to 12 second clip is ~200k packed tokens, and the stock H3
|
| 48 |
+
block materialises its fused QKV and SwiGLU tensors for the whole sequence
|
| 49 |
+
(8.6 and 15.4 GiB at that length), which is what OOMs 24 GB cards and, at
|
| 50 |
+
1376x768, a 96 GB one. `H3 Streamed Blocks` runs every DiT block in token
|
| 51 |
+
chunks with the same math (int8 and W4A8 activations quantise per row and
|
| 52 |
+
accumulate in int32, so the result is bit-equal to stock; measured on
|
| 53 |
+
same-seed renders, video and audio) and streams the output head the same
|
| 54 |
+
way. With ComfyUI's dynamic VRAM and `--fast-disk`, the same 702-frame pass
|
| 55 |
+
that OOMed a 96 GB card renders on a 16 GB card in a 32 GB machine at the
|
| 56 |
+
same seconds per step (316 vs 311), because the step is attention-bound at
|
| 57 |
+
that length and the weight traffic hides under it. Alongside it: `H3 Memory
|
| 58 |
+
Probe` (per-block, per-phase VRAM and RSS ledger with a hoverable timeline,
|
| 59 |
+
plus an optional allocator trace), `H3 Free Cache` (return the allocator
|
| 60 |
+
pool before VAE decode; 17 GiB back on the long pass), `H3 Evict Text
|
| 61 |
+
Encoder`. All opt-in, nothing changes unless the node is in the graph. The
|
| 62 |
+
memory numbers behind them are in the docstrings of `vram_lab.py`; the
|
| 63 |
+
small-card recipe, what to expect and the environment it was measured in are
|
| 64 |
+
in [LOWVRAM.md](LOWVRAM.md), with the example graph
|
| 65 |
+
`examples/motion_pipeline_lowvram.json`; the alpha status is in
|
| 66 |
+
[ALPHA.md](ALPHA.md#vram-lab). Evening additions: two opt-in K/V store
|
| 67 |
+
options in the same node (`kvi8r`, rotated int8 K/V, forward peak +9.5 GiB
|
| 68 |
+
instead of +11.9; `kvi8s`, the same bytes in SageAttention's layout attended
|
| 69 |
+
on int8/fp8 tensor cores with no dequant), both approximation tier with the
|
| 70 |
+
measurements and what is still owed in LOWVRAM.md, kvi8r on in the low-VRAM
|
| 71 |
+
example graph; the example graph moves to
|
| 72 |
+
the int8_convrot video VAE (2.2 GiB less, decode 1.5x faster, 60 dB PSNR
|
| 73 |
+
against fp16 on the same latent); and a fix for the `H3 Audio Smear` /
|
| 74 |
+
`H3 Audio Recover` istft crash on short stretched tails (user report).
|
| 75 |
+
|
| 76 |
+
**Dialogue survives a de-rope now** (2026-08-17). A de-rope used to break
|
| 77 |
+
speech, and the way it broke was confusing: the body came back at the right
|
| 78 |
+
speed and the mouth did not, so held regions sounded rushed while the tail
|
| 79 |
+
sounded fine. The cause is that the picture gets an init and obeys it while
|
| 80 |
+
the audio starts from zeros, so pass 2 writes a fresh performance at natural
|
| 81 |
+
rate and moves the mouth to that. Recovery then compresses something that was
|
| 82 |
+
never slow. New node `H3 Audio Smear` stretches the baseline track onto the
|
| 83 |
+
same dilated clock so pass 2 renders a genuinely slowed take instead. See
|
| 84 |
+
[Dialogue through a de-rope](#dialogue-through-a-de-rope) for the three nodes
|
| 85 |
+
to add to a graph you already have.
|
| 86 |
+
|
| 87 |
+
**The graph to start from** is
|
| 88 |
+
[`motion_pipeline_ref2va_audioinit.json`](examples/motion_pipeline_ref2va_audioinit.json):
|
| 89 |
+
full-reference mode with the audio init wired, 12 steps on the base model for
|
| 90 |
+
pass 1 and the turbo LoRA at 6 steps for the de-rope. On 192 frames at 1 MP
|
| 91 |
+
that is **12 minutes end to end**, against 36 for the same graph at 25 steps
|
| 92 |
+
both passes without turbo. The audio init itself is free in both currencies,
|
| 93 |
+
costing no measurable time and no measurable sharpness.
|
| 94 |
+
|
| 95 |
+
## What is stable, and what is not
|
| 96 |
+
|
| 97 |
+
**Stable:** the Motion Lab de-rope pipeline and Contact-Sheet diffusion, both
|
| 98 |
+
described below. Their dials have measured numbers behind them in
|
| 99 |
+
[TUNING.md](TUNING.md), and their example graphs run out of the box.
|
| 100 |
+
|
| 101 |
+
**Not stable:** Concept Lab, the timeline surface, the motion adapter pilot,
|
| 102 |
+
the manual mask paths, and the audio init above, which works but has only
|
| 103 |
+
been heard on a narrow slice of material. [ALPHA.md](ALPHA.md) says which is
|
| 104 |
+
which and where each has and has not been tested. Alpha nodes carry
|
| 105 |
+
`(alpha)` in the ComfyUI menu and load behind a guarded loader, so a broken
|
| 106 |
+
one cannot take the pack down.
|
| 107 |
+
|
| 108 |
+
## Motion Lab
|
| 109 |
+
|
| 110 |
+
H3 smears bursty motion: backflips, fast sword arcs, whip-fast reversals.
|
| 111 |
+
The cause is structural. One latent token spans four pixel frames, and at
|
| 112 |
+
high motion speed those four frames need four distinct poses that a single
|
| 113 |
+
token can't hold. Re-denoising the affected region doesn't help, because
|
| 114 |
+
the missing poses were never generated in the first place.
|
| 115 |
+
|
| 116 |
+
This pipeline works around that at inference time. It re-generates the
|
| 117 |
+
clip as a slowed-down version of itself, seeded from the original. Frames
|
| 118 |
+
where motion is too fast get held (repeated) so the model has more
|
| 119 |
+
temporal room, the result is generated video-to-video from that retimed
|
| 120 |
+
init at partial denoise, and the original frame rate is recovered
|
| 121 |
+
afterward by dropping the held frames. The oracle that decides where to
|
| 122 |
+
slow down reads the clip's own latent. No extra model, no training.
|
| 123 |
+
|
| 124 |
+

|
| 125 |
+
|
| 126 |
+
Baseline on the left smears the aerial spin into a blob; the two
|
| 127 |
+
regenerated settings render it clean and keep the choreography. New scene,
|
| 128 |
+
default knobs, no per-clip tuning
|
| 129 |
+
([view](https://matlowai.github.io/ComfyUI-MAINodes/#kitsune) /
|
| 130 |
+
[download](https://github.com/matlowai/ComfyUI-MAINodes/raw/main/assets/kitsune_threeway_sbs.mp4)).
|
| 131 |
+
|
| 132 |
+
All demos play on one page: **https://matlowai.github.io/ComfyUI-MAINodes/**
|
| 133 |
+
|
| 134 |
+
Demo clips:
|
| 135 |
+
- baseline vs regenerated, same seed, real time: left smears through the
|
| 136 |
+
backflip, right doesn't
|
| 137 |
+
([view](https://matlowai.github.io/ComfyUI-MAINodes/#derope) /
|
| 138 |
+
[download](https://github.com/matlowai/ComfyUI-MAINodes/raw/main/assets/baseline_vs_regenerated_sbs.mp4))
|
| 139 |
+
- uniform vs adaptive hold maps, the bridge trade-off described below
|
| 140 |
+
([view](https://matlowai.github.io/ComfyUI-MAINodes/#adaptive) /
|
| 141 |
+
[download](https://github.com/matlowai/ComfyUI-MAINodes/raw/main/assets/uniform_vs_adaptive_sbs.mp4))
|
| 142 |
+
- the oracle, watching: heat pools where motion runs too hot, and the
|
| 143 |
+
strip lights up as the burst arrives
|
| 144 |
+
([view](https://matlowai.github.io/ComfyUI-MAINodes/#oracle) /
|
| 145 |
+
[download](https://github.com/matlowai/ComfyUI-MAINodes/raw/main/assets/oracle_map.mp4))
|
| 146 |
+
- fast motion under a panning camera: parasol burst mid-pan
|
| 147 |
+
([view](https://matlowai.github.io/ComfyUI-MAINodes/#panrun) /
|
| 148 |
+
[download](https://github.com/matlowai/ComfyUI-MAINodes/raw/main/assets/panrun_sbs.mp4))
|
| 149 |
+
- nine tiers, one seed: every quality/speed rung with render times in the
|
| 150 |
+
header
|
| 151 |
+
([view](https://matlowai.github.io/ComfyUI-MAINodes/#ladder) /
|
| 152 |
+
[download](https://github.com/matlowai/ComfyUI-MAINodes/raw/main/assets/preview_ladder_grid.mp4))
|
| 153 |
+
- the featherweight stack on 24-32 GB cards, with a Ref2VA subject drop-in
|
| 154 |
+
([view](https://matlowai.github.io/ComfyUI-MAINodes/#featherweight) /
|
| 155 |
+
[download](https://github.com/matlowai/ComfyUI-MAINodes/raw/main/assets/featherweight_ref_triptych.mp4))
|
| 156 |
+
|
| 157 |
+
| baseline vs regenerated | the oracle, watching |
|
| 158 |
+
|---|---|
|
| 159 |
+
|  |  |
|
| 160 |
+
|
| 161 |
+
Already good, and slightly better: uniform dilation, then the adaptive
|
| 162 |
+
map without and with `bridge`. Same seed all three.
|
| 163 |
+
|
| 164 |
+

|
| 165 |
+
|
| 166 |
+
```
|
| 167 |
+
(baseline video) -> VAEDecode frames (baseline latent)
|
| 168 |
+
| |
|
| 169 |
+
v v
|
| 170 |
+
H3TimeSmear <-- hold_map ------------- H3JerkOracle
|
| 171 |
+
| (integer holds) |
|
| 172 |
+
v |
|
| 173 |
+
VAEEncode -> H3V2VInit -> SamplerCustomAdvanced
|
| 174 |
+
^
|
| 175 |
+
H3InjectSchedule ---/
|
| 176 |
+
|
|
| 177 |
+
VAEDecode -> H3ExactRecover -> original fps
|
| 178 |
+
```
|
| 179 |
+
|
| 180 |
+
Three ready-made graphs live in [`examples/`](examples/):
|
| 181 |
+
[`motion_pipeline.json`](examples/motion_pipeline.json) drags straight
|
| 182 |
+
onto the ComfyUI canvas;
|
| 183 |
+
[`motion_pipeline_api.json`](examples/motion_pipeline_api.json) is the
|
| 184 |
+
same graph in API format for scripted use; and
|
| 185 |
+
Three flows cover almost every job, and the examples keep one graph per job:
|
| 186 |
+
the full-quality pipeline (25-step pass 1, 25-step pass 2, no turbo), the fast
|
| 187 |
+
flow (12-step turbo pass 1, 6-step turbo pass 2, inject 0.48 or 0.7), and the
|
| 188 |
+
low-VRAM stack in [LOWVRAM.md](LOWVRAM.md). Inject 0.48 replaced 0.50 as the
|
| 189 |
+
shipped default: at 0.50 and above some scenes measurably loosen their hold on
|
| 190 |
+
the reference. Historical dial studies and superseded variants live in
|
| 191 |
+
[`examples/archive/`](examples/archive/), still runnable as committed. When the
|
| 192 |
+
defaults are too slow for your card or your patience, [FASTER.md](FASTER.md)
|
| 193 |
+
prices every speed dial from measured runs.
|
| 194 |
+
|
| 195 |
+
[`motion_pipeline_turbo.json`](examples/archive/motion_pipeline_turbo.json) is the
|
| 196 |
+
same pipeline with the regeneration pass running on the LightX2V 4-step
|
| 197 |
+
turbo LoRA (Kijai's ComfyUI conversion, strength 0.8, er_sde with a beta
|
| 198 |
+
schedule, 3 of 4 steps after injection). Point the LoRA loader at
|
| 199 |
+
wherever you saved the conversion; community strength range is 0.65 to
|
| 200 |
+
0.8, and v0.1 of that LoRA is a preview, so judge results accordingly.
|
| 201 |
+
|
| 202 |
+
How we actually use this after trying every combination: **turbo is for
|
| 203 |
+
getting your prompt right, the pipeline is for the keeper, and mixing
|
| 204 |
+
them is a waste of time.** Iterate prompts and seeds on plain turbo
|
| 205 |
+
generations to learn what you will get globally, then run the winner
|
| 206 |
+
through the base pipeline. Putting the turbo LoRA inside the
|
| 207 |
+
regeneration pass saves a few minutes on a clip you have already decided
|
| 208 |
+
deserves the full treatment, and it costs quality on exactly that clip;
|
| 209 |
+
we do not recommend it. The turbo-inside graphs remain for people with
|
| 210 |
+
different budgets. One wrinkle that did earn its keep: starting the
|
| 211 |
+
turbo PREVIEW on the base model for the first couple of steps before
|
| 212 |
+
handing off to turbo (H3 Expert Schedule with inject 1.0) may buy
|
| 213 |
+
preview fidelity cheaply; we are still testing it. A unified workflow
|
| 214 |
+
with a preview/final toggle (H3 Mode Switch, lazy: only the chosen path
|
| 215 |
+
executes) is the intended end state.
|
| 216 |
+
|
| 217 |
+
A fourth graph,
|
| 218 |
+
[`examples/archive/motion_pipeline_probe_expert.json`](examples/archive/motion_pipeline_probe_expert.json),
|
| 219 |
+
is the fast path: instead of finishing the baseline it runs only the
|
| 220 |
+
first 6 steps (H3 Probe Schedule, configurable) and reads the oracle and
|
| 221 |
+
the init from the early x0 estimate, then regenerates with a base-model
|
| 222 |
+
head and a turbo tail (H3 Expert Schedule). Cheapest of the set; no
|
| 223 |
+
full-speed audio track to blend, and the saved preview is intentionally
|
| 224 |
+
rough.
|
| 225 |
+
|
| 226 |
+
There is also [`examples/archive/motion_pipeline_i50.json`](examples/archive/motion_pipeline_i50.json),
|
| 227 |
+
the same finals graph with the inject 0.50 preset selected, for people who
|
| 228 |
+
prefer the sharper flavor without touching a dropdown.
|
| 229 |
+
|
| 230 |
+
For 24 to 32 GB cards there is
|
| 231 |
+
[`examples/archive/motion_pipeline_featherweight.json`](examples/archive/motion_pipeline_featherweight.json):
|
| 232 |
+
the same finals graph pointed at the smallest community-published models
|
| 233 |
+
(w4a8 DiT, int8_convrot VAE, nvfp4 text encoder). Needs ComfyUI 0.31+.
|
| 234 |
+
Measured numbers and the honest caveats live in
|
| 235 |
+
[TUNING.md](TUNING.md#featherweight-stack-measured-comfyui-031).
|
| 236 |
+
|
| 237 |
+
Everything from here through the editor and segment graphs is new
|
| 238 |
+
(2026-08-09) and marked **alpha**: the node interfaces may still move,
|
| 239 |
+
and the interactive widget is young. The classic pipeline nodes above
|
| 240 |
+
are untouched and differentially regression-tested against the
|
| 241 |
+
previous release (same inputs, identical outputs).
|
| 242 |
+
|
| 243 |
+
And for the "the oracle is overzealous, I know exactly where the
|
| 244 |
+
problem is" crowd:
|
| 245 |
+
[`motion_pipeline_targeted.json`](examples/motion_pipeline_targeted.json)
|
| 246 |
+
(API twin alongside) puts a human in the loop. The oracle still
|
| 247 |
+
proposes, but its hold map passes through **H3 Manual Hold Map**, which
|
| 248 |
+
keeps holds only inside the time ranges you type (`36-60, 1.5s-2.4s:3`,
|
| 249 |
+
frames or seconds, optional per-range hold count; leave the oracle
|
| 250 |
+
unwired to author holds directly). Queue once and watch the saved
|
| 251 |
+
oraclemap video to see where the heat pools; type your ranges; queue
|
| 252 |
+
again. The node's report output prices the pass before you pay for it:
|
| 253 |
+
world length in, effective regeneration length out, with an optional
|
| 254 |
+
minutes estimate from your measured s/step. Since cost scales with the
|
| 255 |
+
held spans, targeting one burst in a long clip is also the biggest
|
| 256 |
+
speed lever this pack has. The drag-in graph additionally carries a
|
| 257 |
+
muted spatial branch: export a frame, paint a mask, load it, unmute,
|
| 258 |
+
and H3 Motion Composite returns your masked region to baseline timing
|
| 259 |
+
with a feathered seam (see below).
|
| 260 |
+
|
| 261 |
+
The full editing experience is
|
| 262 |
+
[`motion_pipeline_editor.json`](examples/motion_pipeline_editor.json)
|
| 263 |
+
(API twin alongside): the **H3 Motion Editor** node puts a DAW-style
|
| 264 |
+
editor right on the canvas. Queue once to load the filmstrip, then
|
| 265 |
+
work on the node: drag bracket blocks on the timeline (multiple
|
| 266 |
+
blocks, snapped to the model's token grid, jerk profile drawn
|
| 267 |
+
underneath so you can see what the oracle sees), click a block and
|
| 268 |
+
step frame by frame painting the problem areas with brush and eraser
|
| 269 |
+
(onion skin included), set per-block dials for hold, feather size,
|
| 270 |
+
feather profile and direction, edge grow, and temporal fade, and
|
| 271 |
+
toggle `A` on hold, feather, or strength to draw an automation
|
| 272 |
+
envelope with draggable breakpoints, exactly like an automation lane
|
| 273 |
+
in a DAW. A block with no strokes regenerates its whole time span;
|
| 274 |
+
strokes narrow it to the painted region. Queue again and only the
|
| 275 |
+
regeneration side re-runs; the baseline stays cached. The node's
|
| 276 |
+
outputs are ordinary `hold_map` and MASK wires, the mask arriving
|
| 277 |
+
pre-feathered and envelope-scaled (`mask_is_soft` on the composite),
|
| 278 |
+
so everything downstream is the same pipeline. Agents skip the GUI
|
| 279 |
+
and write the same `editor_state` JSON directly; the contract is in
|
| 280 |
+
the node's docstring.
|
| 281 |
+
|
| 282 |
+
And the compute payoff of targeting:
|
| 283 |
+
[`motion_pipeline_editor_segment.json`](examples/motion_pipeline_editor_segment.json)
|
| 284 |
+
(API twin alongside) adds **H3 Segment Crop** and **H3 Segment Splice**
|
| 285 |
+
around the editor. The regeneration chain runs only on the editor's
|
| 286 |
+
held window plus a few real-time handle frames, then the recovered
|
| 287 |
+
segment splices back into the baseline with video and sample-accurate
|
| 288 |
+
audio crossfades inside the handles. Cost scales with dilated frame
|
| 289 |
+
count, so a one-burst window in a longer clip regenerates severalfold
|
| 290 |
+
faster than the whole world; the crop node's report output states the
|
| 291 |
+
exact ratio for your selection. On FL2VA checkpoints, wire the crop's
|
| 292 |
+
first/last frame outputs into the regeneration conditioning to pin the
|
| 293 |
+
seam poses.
|
| 294 |
+
|
| 295 |
+
Four shorter paths, each measured on one prompt and seed:
|
| 296 |
+
|
| 297 |
+
[`examples/archive/motion_pipeline_split_lora.json`](examples/archive/motion_pipeline_split_lora.json)
|
| 298 |
+
splits pass 1 mid-trajectory. The bare model runs the early, high-sigma
|
| 299 |
+
steps -- where the motion is actually decided -- and a turbo LoRA takes
|
| 300 |
+
over for the low-sigma steps, off one schedule with no re-noising
|
| 301 |
+
between them (`SplitSigmas` + `DisableNoise`). Running a turbo LoRA
|
| 302 |
+
across the whole of pass 1 cost about 22% of mean subject motion and 30%
|
| 303 |
+
of the peak in our measurements, and the second pass never gave it back;
|
| 304 |
+
splitting recovered 99% of it, and still finished faster than a plain
|
| 305 |
+
12-step pass 1.
|
| 306 |
+
|
| 307 |
+
[`motion_pipeline_upscale_derope.json`](examples/motion_pipeline_upscale_derope.json)
|
| 308 |
+
does the de-rope and a spatial upscale in the same second pass: pass 1
|
| 309 |
+
renders at 0.4 MP, the smeared frames are resampled to the target size,
|
| 310 |
+
and the regeneration runs there. The second pass rebuilds detail rather
|
| 311 |
+
than interpolating it -- measured 89% of a native 1.5 MP render's
|
| 312 |
+
high-frequency detail for 83% of the wall time. Most of what the cheap
|
| 313 |
+
pass 1 saves, the larger second pass gives back: pass 1 drops from 179 s
|
| 314 |
+
to 31 s, and pass 2 rises from 305 s to 372 s. The cost is jerk
|
| 315 |
+
removal: a soft pass 1 gives the oracle blurrier evidence, so it cuts
|
| 316 |
+
less of it.
|
| 317 |
+
|
| 318 |
+
[`motion_pipeline_rolling_window.json`](examples/motion_pipeline_rolling_window.json)
|
| 319 |
+
(alpha) is the upscale de-rope split into budgeted windows, for cards
|
| 320 |
+
that cannot hold the whole dilated pass at once. **H3 Window Plan**
|
| 321 |
+
divides the clip into as many windows as your `max_dilated_frames`
|
| 322 |
+
budget requires and emits one per queue item: queue once, read the plan
|
| 323 |
+
report (wired to a preview node; it prices every window and names each
|
| 324 |
+
cut cold or hot before anything runs), then set the queue batch count
|
| 325 |
+
to the window count and queue once more -- the `window` widget
|
| 326 |
+
increments itself per batch item, seed-widget style, so the whole set
|
| 327 |
+
renders from one click. **H3 Window Collect** banks each rendered window to disk
|
| 328 |
+
(`output/h3_windows`, so a crash or reboot costs one window, not the
|
| 329 |
+
run) and splices the full set into the baseline once the last one
|
| 330 |
+
lands. On our test clip the peak window was 62 latent tokens against
|
| 331 |
+
77 for the one-pass version, paid for as 1.47x the one-pass total in
|
| 332 |
+
generated frames at the budget that forced the split. `coverage`
|
| 333 |
+
defaults to `full clip` so calm spans get the same second-pass repaint
|
| 334 |
+
as the action; `held span` is cheaper but leaves everything outside the
|
| 335 |
+
held span at baseline resolution, which on this graph means visibly
|
| 336 |
+
soft the moment motion calms.
|
| 337 |
+
|
| 338 |
+
**H3 Conditioning Bank** (alpha) keeps the text encoder out of the
|
| 339 |
+
window items. Wire it between the encode node and the guider: the first
|
| 340 |
+
item encodes and banks the conditioning to disk, and every later item
|
| 341 |
+
reads the tensors back. Its `conditioning` input is lazy, so on a bank
|
| 342 |
+
hit ComfyUI never executes the encode node or the CLIP loader behind
|
| 343 |
+
it, and the encoder (14.96 GB resident on the int8 ref2va stack, 21.2
|
| 344 |
+
GB peak on a 16 GB-simulated card) is never loaded for that item. To be
|
| 345 |
+
exact about what it buys: requeueing the same graph with only the
|
| 346 |
+
`window` widget changed already hits ComfyUI's own node cache, so
|
| 347 |
+
nothing is re-encoded there. What loses that cache is queueing any
|
| 348 |
+
other workflow in between, restarting ComfyUI, or editing anything
|
| 349 |
+
upstream of the encode, and the bank survives all three. It is not
|
| 350 |
+
window-specific: a seed hunt and an extension chain on the same prompt
|
| 351 |
+
encode once between them too.
|
| 352 |
+
|
| 353 |
+
**H3 Latent Bank** (alpha) is the same trick one stage later, and on
|
| 354 |
+
this graph it is worth more. The pass-1 baseline render is cached
|
| 355 |
+
exactly like the conditioning is, and lost exactly as easily, so an
|
| 356 |
+
interleaved workflow makes window item 2 re-render the whole baseline
|
| 357 |
+
before it starts its own window. Bank the pass-1 LATENT and every
|
| 358 |
+
consumer is served from it: the video decode, the audio decode, and H3
|
| 359 |
+
Jerk Oracle, which reads the latent directly. Latents are the cheap
|
| 360 |
+
thing to keep: the AV latent of a 107-frame 480x832 clip is 4.8 MB,
|
| 361 |
+
against 513 MB for the same clip as float32 frames. The staleness
|
| 362 |
+
contract is stated plainly on the node: `bank_key` plus a hash of
|
| 363 |
+
`seed` and `fingerprint`, and nothing else is fingerprinted for you.
|
| 364 |
+
|
| 365 |
+
Measured on a simulated 32 GB card (a large card fenced down with
|
| 366 |
+
`--reserve-vram`; int8 stack, 0.4 to 1.5 MP, 107 frames): the one-pass
|
| 367 |
+
de-rope peaked at 30.7 GB, touched the ceiling, and streamed the DiT
|
| 368 |
+
layer by layer (208 lowvram patches). Split into two windows the peaks
|
| 369 |
+
were 22.5 and 24.6 GB and the DiT stayed resident, so the two windows
|
| 370 |
+
together (440 s) matched the one-pass wall time (460 s) while
|
| 371 |
+
generating 1.4x the frames. The lower peak is what buys the extra work
|
| 372 |
+
back: on a card at the offload cliff, not streaming beats not
|
| 373 |
+
splitting.
|
| 374 |
+
|
| 375 |
+
[`motion_pipeline_fast_iterate.json`](examples/motion_pipeline_fast_iterate.json)
|
| 376 |
+
is the same idea sized for iteration: 0.2 MP in, 0.4 MP out, about 95
|
| 377 |
+
seconds end to end. Use it to find out whether the choreography lands
|
| 378 |
+
before paying for a final.
|
| 379 |
+
|
| 380 |
+
[`motion_pipeline_ref2va_audioinit.json`](examples/motion_pipeline_ref2va_audioinit.json)
|
| 381 |
+
runs the pipeline in full-reference mode, with the six-section prompt
|
| 382 |
+
contract and a reference image (wired to ComfyUI's stock `example.png`
|
| 383 |
+
so it runs out of the box -- swap in your own). It also seeds pass 2's
|
| 384 |
+
AUDIO rows with the baseline performance, which is what keeps dialogue
|
| 385 |
+
intact through a de-rope: without a seed, pass 2 writes fresh speech at
|
| 386 |
+
natural rate and drags the mouth to match it, and recovery then
|
| 387 |
+
compresses those lips by the hold factor, so held regions come back
|
| 388 |
+
rushed while the unheld tail sounds fine. `H3 Audio Smear` stretches the
|
| 389 |
+
baseline track onto the dilated clock, `VAEEncodeAudio` encodes it, and
|
| 390 |
+
`H3 V2V Init`'s `audio_latent` seeds it at `follow the original
|
| 391 |
+
performance (0.5)`.
|
| 392 |
+
|
| 393 |
+
**This is the balanced setting, and it is the one to start from.** Pass 1
|
| 394 |
+
runs 12 steps on the base model, pass 2 runs the turbo LoRA at
|
| 395 |
+
`total_steps 6` with inject 0.50, so about three steps actually execute on
|
| 396 |
+
the de-rope. On 192 frames at 1 MP on an RTX PRO 6000 Blackwell that is
|
| 397 |
+
**12 minutes end to end**. The same graph without turbo, at 25 steps for
|
| 398 |
+
both passes, took 36 minutes on the Max-Q card, which runs roughly 13%
|
| 399 |
+
slower on sustained 1 MP work, so call it 32 minutes equivalent. Nearly
|
| 400 |
+
three times the wall time, and the turbo arm was the one that got the
|
| 401 |
+
playback verdict.
|
| 402 |
+
|
| 403 |
+
**The audio init itself is free.** Pass 2 measured 6:06 with the seed
|
| 404 |
+
against about 7:20 without it on the same card, which is inside run to run
|
| 405 |
+
variation. It also costs nothing in picture quality: 91.8 against 91.3 on
|
| 406 |
+
laplacian variance, inside the noise floor of the video encoder itself.
|
| 407 |
+
And because the seed only touches pass 2, re-running a graph after wiring
|
| 408 |
+
it serves pass 1 from cache.
|
| 409 |
+
|
| 410 |
+
It writes two finals so you can hear the difference the seed makes:
|
| 411 |
+
`_recovered` keeps the original performance (the safe route, and the one
|
| 412 |
+
we ship as the default), and `_seededfoley` takes pass 2's own foley
|
| 413 |
+
retimed to the world clock -- legitimate ONLY because the rows were
|
| 414 |
+
seeded. Alpha: measured on a handful of clips, all sword-fight material
|
| 415 |
+
with two speakers. Speech over music, a single speaker and hold factors
|
| 416 |
+
other than 4 are untested.
|
| 417 |
+
|
| 418 |
+
[`motion_pipeline_ref2va.json`](examples/motion_pipeline_ref2va.json)
|
| 419 |
+
is the same graph WITHOUT the audio init, kept as the archived version.
|
| 420 |
+
Reach for it only if you want the older behaviour; on any clip with
|
| 421 |
+
dialogue, prefer the audioinit graph above.
|
| 422 |
+
|
| 423 |
+
**There is a resolution floor.** Below roughly 0.4 MP the subject smears
|
| 424 |
+
regardless of configuration, and every quality judgement we took from a
|
| 425 |
+
448x448 render turned out to be worthless. The small-canvas paths are for
|
| 426 |
+
iteration, not finals. Separately, a distilled LoRA wants the size it was
|
| 427 |
+
trained at: ours was mush at 448x448 and clean at its native 768x768.
|
| 428 |
+
|
| 429 |
+
**Do not run a turbo LoRA at its distilled step count in pass 2.** A
|
| 430 |
+
distilled LoRA's step budget is sized for a full denoise from noise;
|
| 431 |
+
pass 2 is a partial re-denoise from a v2v init, so those steps land far
|
| 432 |
+
too finely and the subject dissolves into a coarse mosaic. Running an
|
| 433 |
+
8-step LoRA for 8 steps of a 0.50 injection failed at both 0.2 and
|
| 434 |
+
1.0 MP and with either v1.0 file; 4 steps of the same schedule was
|
| 435 |
+
clean. Budget pass-2 steps against the fraction of the schedule you are
|
| 436 |
+
actually running, not against the LoRA's name.
|
| 437 |
+
|
| 438 |
+
All of them generate or probe a baseline, read its oracle, regenerate,
|
| 439 |
+
and recover, in one queue item. The oracle's length and the regeneration
|
| 440 |
+
length are wired dynamically, so changing the clip duration needs no
|
| 441 |
+
other edits. Each node's info button documents its inputs.
|
| 442 |
+
|
| 443 |
+
### Nodes
|
| 444 |
+
|
| 445 |
+
| node | knob | default | notes |
|
| 446 |
+
|---|---|---|---|
|
| 447 |
+
| H3 Jerk Oracle | `q` | 0.75 | jerk quantile treated as "hot"; higher = tighter span, lower cost |
|
| 448 |
+
| | `d_max` | 4 | peak hold count; below 4, smearing starts returning in our tests |
|
| 449 |
+
| | `ramp` | on | smooth shoulders on the hold curve; hard steps caused visible stutter |
|
| 450 |
+
| | `bridge` | 8 | fill dips between peaks of the same burst (see below); 0 = off |
|
| 451 |
+
| | `preset` | balanced | balanced / max quality / economy; `custom` uses the knobs |
|
| 452 |
+
| H3 Time Smear | `dilation` | 4 | uniform hold count, used when no hold_map is wired |
|
| 453 |
+
| H3 Inject Schedule | `inject` | 0.70 | fraction of the denoise schedule that runs. Lower keeps more of the init (including its artifacts); higher lets the model drift from the source choreography. 0.5 to 0.8 is the useful range |
|
| 454 |
+
| | `preset` | 0.70 | 0.70 / 0.50 / 0.80; `custom` uses the knob |
|
| 455 |
+
| H3 V2V Init | `length` | 0 (auto) | wraps the encoded init as H3's joint AV latent; audio regenerates with the video |
|
| 456 |
+
| H3 Exact Recover | | | drops held frames per the hold map; recovery is frame selection, not resampling |
|
| 457 |
+
| H3 Audio Recover | `fps` | 24 | retimes the regenerated audio to the original clock with the same hold map, pitch preserved, so the recovered video keeps its own foley |
|
| 458 |
+
| | `reference_mix` | 1 | whose track survives: 1 = the pass-1 baseline audio intact (default; regenerated audio quality varies, especially off turbo passes), 0 = the regenerated foley (leaner, one performance). The two performances are different takes, so mid values blend them; the dial is happiest near its ends |
|
| 459 |
+
| H3 Jerk Heatmap | `alpha`, `strip_height` | 0.55, 96 | the oracle-watching overlay from the demo clip, as a node |
|
| 460 |
+
| H3 Probe Schedule | `probe_steps` | 6 | run only the head of the baseline; the early x0 feeds the oracle and the init. Raise it if the init loses choreography |
|
| 461 |
+
| H3 Expert Schedule | `base_head` | 2 | split the injected schedule: base-model head for structure, turbo tail for refinement (tail defaults to turbo's native 4 steps) |
|
| 462 |
+
| H3 Trajectory Bank | `every_n` | 1 | wraps a sampler and checkpoints the trajectory latent each step (~7 MB per step for a 5 s clip) |
|
| 463 |
+
| H3 Trajectory Load | `step` | 5 | resume a banked run from any step with its remaining schedule; swap the model, LoRA, or guider and continue without recomputing the head |
|
| 464 |
+
|
| 465 |
+
#### Alpha nodes
|
| 466 |
+
|
| 467 |
+
Added 2026-08-09 and after. These are the research surface: they
|
| 468 |
+
work and are documented, but their names, defaults and outputs may
|
| 469 |
+
change, and they have had far less playback mileage than the nodes
|
| 470 |
+
above. `TESTING_ALPHA.md` is the manual checklist; `ROADMAP.md` and
|
| 471 |
+
`RESEARCH_NOTES_ATOS.md` carry the open questions and what we
|
| 472 |
+
rejected. If you want the settled pipeline, everything above this
|
| 473 |
+
line is it.
|
| 474 |
+
|
| 475 |
+
| node | knob | default | notes |
|
| 476 |
+
|---|---|---|---|
|
| 477 |
+
| H3 Motion Editor | timeline, brushes, lanes | | the GUI: time blocks with bracket handles, per-frame mask painting, per-block dials, automation envelopes for hold/feather/strength. Compiles to a hold map and a soft mask; state is plain JSON that agents can author without the GUI |
|
| 478 |
+
| H3 Segment Crop | `handle_frames` | 12 | cut the world to the held window plus context handles; the regen chain then pays only for the window. Report output states the speedup |
|
| 479 |
+
| H3 Segment Splice | `feather_frames` | 6 | reassemble after recovery: baseline outside, segment inside, video + audio crossfades inside the handles |
|
| 480 |
+
| H3 Window Plan (alpha) | `max_dilated_frames` | 209 | per-window budget in smeared frames, the number that sets cost and peak memory; read your card's ceiling off a run that survived |
|
| 481 |
+
| | `coverage` | full clip | `full clip` tiles windows over every frame so calm spans are repainted too (they cost 1 dilated frame each and cut cold); `held span` regenerates only where the hold map fires, cheaper, but passed-through frames keep baseline resolution on upscale graphs |
|
| 482 |
+
| | `window` | 0 | which window this queue item renders; 0, queue, then 1, queue. The report output is the interface: read it |
|
| 483 |
+
| H3 Window Collect (alpha) | `store_dir` | output/h3_windows | windows bank here between queue items and survive a reboot; avoid /tmp, it is a RAM disk on most Linux installs |
|
| 484 |
+
| | `run_name` | window_run | keys the banked set; change it whenever the plan changes |
|
| 485 |
+
| H3 Conditioning Bank (alpha) | `bank_key` | run | banks the encoded prompt to disk between queue items. Its `conditioning` input is lazy, so on a hit the encode node and the ~15 GB text encoder are never executed. One key per (prompt, reference, canvas, length): only the prompt is fingerprinted for you, and only if you wire it |
|
| 486 |
+
| | `mode` | use bank if present | `refresh` re-encodes and overwrites, which is what you press after changing anything the key does not cover |
|
| 487 |
+
| H3 Latent Bank (alpha) | `bank_key` | pass1 | same idea one stage later: banks a sampled LATENT so the pass is not re-rendered. Seat it after the pass-1 sampler, where the video decode, the audio decode and the jerk oracle all read from it. Lazy input, so a hit never stages the sampler |
|
| 488 |
+
| | `seed`, `fingerprint` | unwired | the ONLY things folded into the filename beyond `bank_key`. Wire the noise seed; put steps, scheduler, LoRA strength and resolution into `fingerprint` yourself |
|
| 489 |
+
| | `store_dtype` | float32 (exact) | float16 halves the file and sits below a VAE decode's noise, but it is not bit-identical: keep float32 while you are comparing takes |
|
| 490 |
+
| H3 Manual Hold Map | `ranges` | | manual time targeting: `start-end[:hold]` pairs, frames or seconds. Wire the oracle's hold_map in and its holds survive only inside your ranges (gate mode); leave it unwired to author holds directly. The report output prices the regeneration before you run it |
|
| 491 |
+
| H3 V2V Init | `freeze_threshold` | 0 (off) | automatic background freeze, not recommended: it fixes background timing but degraded other artifacts in our playback tests. Kept for content where the trade goes the other way |
|
| 492 |
+
| | `mask`, `mask_feather` | off, 0 | manual freeze region: you paint what regenerates (`invert_mask` to paint the frozen background instead). Static union over time, so the boundary never moves. Default is hard latent cells (each ~16 px cell fully frozen or fully live; the decode smooths the edge); raise `mask_feather` for a pixel-space ramp pooled to fractional cells if a seam shows |
|
| 493 |
+
| H3 Motion Composite | `mask` | oracle heat | spatial recovery: regenerated pixels inside the mask, baseline outside. The automatic oracle-heat mode stays deprecated for moving background objects (they pop at its boundary); with a hand-drawn mask the seam goes where you hide it, along a real edge |
|
| 494 |
+
| H3 Indecision Oracle (experimental) | `mode` | indecision | which signal drives the hold map: `indecision`, `jerk passthrough`, `blend max`, `blend weighted w`. Outputs mirror H3 Jerk Oracle and compile through the same threshold/bridge/ramp code, so the switch changes the signal and nothing else. See below |
|
| 495 |
+
| | `step_a`, `step_b` | 6, 12 | which two X0 Tap dumps to difference. 6 to 12 on a 25-step run is the validated pair; 0 to 1 is degenerate |
|
| 496 |
+
| | `blend_w` | 0.5 | weight on indecision in `blend weighted w`; blending happens after per-source rank normalization |
|
| 497 |
+
| H3 Timeline Analyze (alpha) | `auto` | True | oracle profiles in, a PLAN DOCUMENT out: a per-clip JSON saying in generation densities over time how the shot should be retimed. `auto` compiles the proposal straight to a minted graph; off stops at the plan so you can edit it |
|
| 498 |
+
| H3 Timeline Render (alpha) | `plan_path` | | a plan in, a legal graph minted and priced. It emits the graph path and the launch line rather than executing: launching stays with your own queue script |
|
| 499 |
+
| H3 Drawn Plan (alpha) | `plan_path`, `plan_json` | | load a plan and get its compiled geometry as wireable outputs: `hold_map` (the compiler's map verbatim, for H3 Temporal Insert), `ranges` (the same map in H3 Manual Hold Map's language, which re-shapes it through that node's own snapping), window start/len, dilated length, guide frame, and a `splice_map` for H3 Segment Splice at feather 0 |
|
| 500 |
+
| | `ignore_uncompiled_lanes` | False | a plan may carry lanes this backend cannot compile yet. Off refuses the plan so nothing is dropped silently; on compiles the density lane and names every lane it skipped |
|
| 501 |
+
| H3 Plan Settings (alpha) | `plan_path`, `plan_json` | | the same plan's execution knobs as typed outputs: inject, steps, seed, prompt, width, height, output prefix. Wire these instead of retyping them and the graph cannot drift from the plan. `expand_to_end` is reported for reading only: the compiler already baked it into the map, so H3 Temporal Insert stays False |
|
| 502 |
+
| H3 Plan Estimate (alpha) | `recorder_path` | flight recorder | what the plan costs before it runs: equivalent clip time (exact from the plan), work units, VRAM band. `seconds` comes back -1 when your box has no recorded runs, because an uncalibrated guess is worse than no answer |
|
| 503 |
+
| H3 Repair Plan (alpha) | `bad_start`, `bad_end` | 0, 0 | 0-based inclusive frame range that is wrong. The node snaps it OUTWARD to whole latent time tokens and emits the hold map, the time-varying regenerate mask and the splice bounds |
|
| 504 |
+
| | `hold` | 3 | dilation on the repaired span only; set H3 Time Smear's `dilation` to the same number, and its `expand_to_end` off |
|
| 505 |
+
| | `shot_cuts`, `cut_reach` | "", 8 | 0-based first frame of each new shot. A cut inside the span, or within `cut_reach` of its end, becomes the exit: the splice hands back to the original AT the cut |
|
| 506 |
+
| H3 Repair Splice (alpha) | `plan` | | frame selection, nothing resampled or blended. Asserts that every frame outside the splice is bit-identical to the original, and prints entry/exit seam deltas next to the original's own delta at the same frames |
|
| 507 |
+
| H3 DyRoPE (experimental) | `mode` | physical_blocks | which blocks or steps see the physical (true-duration) grid instead of the stock uniform one. `physical_all` reproduces H3 True Clock, `compact_all` reproduces having no node at all |
|
| 508 |
+
| | `block_lo`, `block_hi` | 0, 24 | the block range for the `*_blocks` modes. Two recommended doses, neither a default: 30-49 (timing-fidelity, slight flicker in the span) and 40-49 (minimal shimmer, can read over-smooth on some content) |
|
| 509 |
+
| | `fade_end` | 0.5 | for the `fade_*` modes: the sigma at which the interpolation completes. 0.7 is the recommended fade dose; 0.8 is already past the benefit boundary |
|
| 510 |
+
| H3 V2V Init | `audio_prefix_ticks` | 0 (off) | (alpha) freeze the first n audio-latent ticks to the seeded `audio_latent`. A 39-frame carried handle is exactly 65 ticks. Needs `audio_latent` wired |
|
| 511 |
+
| | `audio_prefix_release_ticks` | 0 | (alpha) half-cosine release over the last n ticks of the frozen prefix instead of a hard edge; 8 ticks (0.2 s) is the upstream lineage's tested recipe |
|
| 512 |
+
| H3 Drift Control (alpha) | `prefix_frames` | 39 | schedule-matched noise on a chained segment's carried VIDEO prefix, tapering to exact at the seam end. 39 frames = 12 latent steps = `matched_steps` 8 + `taper_steps` 4, and those two must sum to the prefix's step count |
|
| 513 |
+
| H3 Delta Color Carry (alpha) | `strength` | 0.50 | adds only E(corrected) - E(original) to the carried prefix, so the encode bias cancels and only the grade survives. Inert until the two stats inputs differ |
|
| 514 |
+
| H3 Scene Color Stats (alpha) | | | centre-weighted luma/saturation percentiles of a frame batch as JSON, for the node above |
|
| 515 |
+
|
| 516 |
+
#### The indecision oracle (experimental, 2026-08-14)
|
| 517 |
+
|
| 518 |
+
A second, independent oracle. The jerk oracle reads a clip's motion out
|
| 519 |
+
of a finished latent; this one reads the model's own uncertainty out of
|
| 520 |
+
two mid-schedule x0 predictions:
|
| 521 |
+
|
| 522 |
+
J[token] = avgpool2x2( mean over the 24 latent channels of |x0_b - x0_a| )
|
| 523 |
+
|
| 524 |
+
High J means tokens whose predicted clean latent is still moving between
|
| 525 |
+
denoise steps, i.e. where the model has not made up its mind. A desk
|
| 526 |
+
study over 7 scenes found it carries genuinely independent signal:
|
| 527 |
+
controlling for pixel motion it still correlates +0.41 with static
|
| 528 |
+
detail energy, and on the quietest third of token-times (where
|
| 529 |
+
frame-diff has nothing to say) +0.51. It also misses things the jerk
|
| 530 |
+
oracle catches: one fast swinging prop read motion rank 0.97 and jitter
|
| 531 |
+
rank 0.04. The two disagree in both directions, so `blend max` is the
|
| 532 |
+
recommended experiment rather than a straight substitution.
|
| 533 |
+
|
| 534 |
+
**Nothing else in the pack changed its defaults.** This node exists so
|
| 535 |
+
the comparison can be made on real renders first.
|
| 536 |
+
|
| 537 |
+
Wiring the A/B:
|
| 538 |
+
|
| 539 |
+
1. Pass 1 goes through `X0 Tap (SAMPLER wrapper)` from `h3-motion-lab`
|
| 540 |
+
with `dump_steps` including both steps you want (`6,12` at minimum).
|
| 541 |
+
2. Wire the pass-1 latent into `samples` and the tap's `dump_dir` into
|
| 542 |
+
`dump_dir`, and set `length`/`width`/`height` to the tapped clip.
|
| 543 |
+
3. Wire `hold_map` where you would have wired the jerk oracle's, and
|
| 544 |
+
flip `mode` to compare. `jerk passthrough` reproduces H3 Jerk Oracle
|
| 545 |
+
exactly (unit-tested for byte-identical output), so the A/B is one
|
| 546 |
+
widget and not two graphs.
|
| 547 |
+
4. Read the `comparison` output for the overlap and divergence numbers,
|
| 548 |
+
and preview `heat` for the side-by-side map (indecision left, jerk
|
| 549 |
+
right; wire `images` to get it overlaid on frames instead of tiles).
|
| 550 |
+
|
| 551 |
+
Two traps the node shouts about in its report:
|
| 552 |
+
|
| 553 |
+
- **The cheap pair is the useless one.** Step 0 to 1 is dominated by the
|
| 554 |
+
(1,4,4,4,4) chunk-phase ramp, not by content, and correlated at or
|
| 555 |
+
below zero with the picture on 6 of 7 test scenes.
|
| 556 |
+
- **Masked, pinned and repaint runs give you a picture of the mask.**
|
| 557 |
+
Composited token rows read as exactly zero jitter. If more than 30% of
|
| 558 |
+
token rows are exactly zero the report says so in capitals; believe it
|
| 559 |
+
and do not drive a hold map off that map.
|
| 560 |
+
|
| 561 |
+
Some graphs tap `0,1,12,24` rather than including 6. There 12 to 24 is
|
| 562 |
+
the usable pair. With `auto_fallback` on (default) the node picks the
|
| 563 |
+
closest available pair and puts what it did in the report; turn it off
|
| 564 |
+
to hard-fail on a mis-tapped graph instead.
|
| 565 |
+
| | `feather_profile`, `feather_direction` | linear, centered | seam control: linear/smoothstep/gaussian falloff; centered straddles the boundary, inward eats into the masked side, outward into the kept side |
|
| 566 |
+
|
| 567 |
+
### What to expect, time-wise
|
| 568 |
+
|
| 569 |
+
Measured on a 5 second 1024x1024 clip at about 11.5 s/step (RTX PRO 6000);
|
| 570 |
+
scale to your card and clip:
|
| 571 |
+
|
| 572 |
+
| path | time | what you get |
|
| 573 |
+
|---|---|---|
|
| 574 |
+
| starting point, turbo pass 2 (`examples/motion_pipeline_ref2va_audioinit.json`) | ~12 min for 192 frames at 1 MP; the same graph at 25 steps both passes without turbo is ~36 | the normal finals since 2026-08-19 |
|
| 575 |
+
| pipeline, inject 0.70 (`examples/motion_pipeline.json`) | ~19 min incl. its own baseline | the slow finals: 25 steps both passes, base model throughout |
|
| 576 |
+
| pipeline, inject 0.50 (`examples/archive/motion_pipeline_i50.json`) | ~15 min incl. baseline | sharper, tracks the source motion closer; try both |
|
| 577 |
+
| probe + expert turbo (`examples/archive/motion_pipeline_probe_expert.json`) | ~8.5 min, no full baseline | the fast full de-rope; preview output is intentionally rough |
|
| 578 |
+
| featherweight (`examples/archive/motion_pipeline_featherweight.json`, ComfyUI 0.31+) | 4-6 min for 3 s clips; ~29 min for 5 s at 1.0 MP | the 24-32 GB card path; fits where int8 thrashes. See TUNING for measured peaks |
|
| 579 |
+
| split LoRA pass 1 (`examples/archive/motion_pipeline_split_lora.json`) | ~7.5 min at 1.5 MP | most motion retained of the pass-1 recipes we measured |
|
| 580 |
+
| upscale de-rope (`examples/motion_pipeline_upscale_derope.json`) | ~6.75 min at 0.4 -> 1.5 MP | 89% of native detail, 83% of the time |
|
| 581 |
+
| fast iterate (`examples/motion_pipeline_fast_iterate.json`) | ~95 s at 0.2 -> 0.4 MP | prompt and choreography loop, not a final |
|
| 582 |
+
|
| 583 |
+
The rows pointing into `examples/archive/` are earlier recipes, kept so
|
| 584 |
+
these numbers still resolve; start from the live graphs.
|
| 585 |
+
|
| 586 |
+
Start with a short clip, 2 to 3 seconds, and scale up once you like what
|
| 587 |
+
you see. Durations snap to the model's legal frame counts automatically
|
| 588 |
+
(the closest ones are 1.6 s, 2.3 s, and 3.0 s at 24fps), a short clip
|
| 589 |
+
keeps VRAM and wait times friendly, and how far you scale is really a
|
| 590 |
+
question of how much fast action the clip contains: cost follows the
|
| 591 |
+
burst spans, not the runtime.
|
| 592 |
+
|
| 593 |
+
A tuning guide for all of this, written for humans and for AI assistants
|
| 594 |
+
working on a user's behalf, is in [`TUNING.md`](TUNING.md).
|
| 595 |
+
|
| 596 |
+
Where the method is honestly weak, and what we are doing about it, is split
|
| 597 |
+
across two documents: [`RESEARCH_NOTES_ATOS.md`](RESEARCH_NOTES_ATOS.md) for
|
| 598 |
+
what has been measured (including the finding that the oracle can rank but
|
| 599 |
+
cannot abstain), and [`ROADMAP.md`](ROADMAP.md) for the methods under
|
| 600 |
+
investigation, what would count as success for each, and the approaches we
|
| 601 |
+
tried and rejected.
|
| 602 |
+
|
| 603 |
+
### bridge and inject
|
| 604 |
+
|
| 605 |
+
Both settings change the output in ways that are a preference, not a
|
| 606 |
+
ranking. From same-seed comparisons on our test clips:
|
| 607 |
+
|
| 608 |
+
- `bridge: 8` (default): the hold plateau covers each burst fully.
|
| 609 |
+
Sharpest output, motion tracking equal to uniform dilation, about 2.9x
|
| 610 |
+
frame budget. Poses can drift slightly from the baseline (a head angle
|
| 611 |
+
on a landing, that kind of thing).
|
| 612 |
+
- `bridge: 0`: holds follow the raw oracle curve. Closest to the
|
| 613 |
+
baseline's poses; a few soft frames can remain where the curve dips
|
| 614 |
+
inside a burst.
|
| 615 |
+
- no hold_map (uniform `dilation: 4`): most conservative, highest cost.
|
| 616 |
+
- `inject 0.70` vs `0.50`: 0.50 measured sharper with closer motion
|
| 617 |
+
tracking on our clips; 0.70 has been the safer default in playback.
|
| 618 |
+
Try both on your content.
|
| 619 |
+
|
| 620 |
+
### Manual spatial masks, or: the birds
|
| 621 |
+
|
| 622 |
+
Time warping overcranks steady background movers (birds, crowds,
|
| 623 |
+
traffic) inside dilated spans, and both automatic remedies failed our
|
| 624 |
+
playback bar: compositing on the oracle's own heat mask popped at the
|
| 625 |
+
boundary, and the latent freeze degraded other artifacts. The
|
| 626 |
+
mechanisms were fine; the mask author wasn't. The oracle cannot hide a
|
| 627 |
+
seam. You can, by lassoing the sky down to a rooftop line and letting
|
| 628 |
+
the feather blend along an edge where nothing moves.
|
| 629 |
+
|
| 630 |
+
So both mechanisms now take a hand-drawn MASK. On the demo clip, whose
|
| 631 |
+
regeneration invented an extra flock and a small pagoda, a two-box
|
| 632 |
+
keep-baseline mask (`invert_mask` on) returned both regions to the
|
| 633 |
+
baseline take and kept the regenerated subject, seam along the cloud
|
| 634 |
+
deck. Prefer the composite (pixel space, fine feather control) unless
|
| 635 |
+
background and subject share lighting or contact, then use the V2V
|
| 636 |
+
Init freeze (latent space, coarser feather, but the model renders the
|
| 637 |
+
interaction). A single mask is a static boundary and cannot pop by
|
| 638 |
+
construction; mask batches are supported for per-frame control but
|
| 639 |
+
bring the moving-boundary risk back, so feather harder and judge in
|
| 640 |
+
playback.
|
| 641 |
+
|
| 642 |
+
### Notes on the approach
|
| 643 |
+
|
| 644 |
+
- A reference conditions every step at full strength and will copy the
|
| 645 |
+
source's artifacts. An init decays with noise: at `inject 0.70` the
|
| 646 |
+
baseline's smear detail is destroyed while its coarse motion survives.
|
| 647 |
+
- The model's clock stays uniform. The slowdown exists only in the
|
| 648 |
+
content, as a speed ramp, so there is no boundary where the DiT and the
|
| 649 |
+
VAE disagree about time. (Warping the RoPE time axis directly was
|
| 650 |
+
tried; it produced boundary stutter.)
|
| 651 |
+
- Holds are integer, so recovering the original frame rate is exact frame
|
| 652 |
+
selection.
|
| 653 |
+
|
| 654 |
+
### Dialogue through a de-rope
|
| 655 |
+
|
| 656 |
+
If your clip has speech in it, the de-rope will break it, and the way it
|
| 657 |
+
breaks is not obvious from the output.
|
| 658 |
+
|
| 659 |
+
Here is what happens. The de-rope stretches time, regenerates, then
|
| 660 |
+
compresses back. The picture goes along with that: the smeared init tells
|
| 661 |
+
pass 2 to move slowly, and it does. The audio has no init. It starts from
|
| 662 |
+
zeros, so pass 2 writes a fresh performance at natural speaking rate and
|
| 663 |
+
moves the mouth to match that. Recovery then compresses the whole clip by
|
| 664 |
+
the hold factor. The body comes back at the right speed because it really
|
| 665 |
+
was slowed. The mouth does not, because it never was. On a clip that
|
| 666 |
+
dilated 2.4x, held regions come back sounding rushed while the tail sounds
|
| 667 |
+
fine, which is a confusing symptom because half the clip is correct.
|
| 668 |
+
|
| 669 |
+
The fix is to give the audio an init too. `H3 Audio Smear` stretches the
|
| 670 |
+
baseline track onto the same dilated clock the video init lives on, using
|
| 671 |
+
the same hold map. Encode that with `VAEEncodeAudio`, wire it into
|
| 672 |
+
`H3 V2V Init`'s `audio_latent`, and set `audio_mode` to
|
| 673 |
+
`follow the original performance (0.5)`. Pass 2 now renders a genuinely
|
| 674 |
+
slowed take, so compressing it afterwards is a valid thing to do.
|
| 675 |
+
|
| 676 |
+
Two graphs ship with this, both full-reference mode:
|
| 677 |
+
[`motion_pipeline_ref2va_audioinit.json`](examples/motion_pipeline_ref2va_audioinit.json)
|
| 678 |
+
has the seed wired and writes both audio routes as separate finals, and
|
| 679 |
+
[`motion_pipeline_ref2va.json`](examples/motion_pipeline_ref2va.json) is the
|
| 680 |
+
same graph without it, kept as the archived version. Diff the two if you
|
| 681 |
+
would rather read the change than follow the steps below.
|
| 682 |
+
|
| 683 |
+
**To adopt it in a graph you already have, add three things:**
|
| 684 |
+
|
| 685 |
+
1. `H3 Audio Smear` -- `audio` from the VAEDecodeAudio of your FIRST pass
|
| 686 |
+
(the baseline performance, not pass 2's), `hold_map` from the same
|
| 687 |
+
`H3 Time Smear` output you already feed to `H3 Audio Recover`, `fps` 24.
|
| 688 |
+
2. `VAEEncodeAudio` -- audio from the smear, vae is your audio VAE.
|
| 689 |
+
3. On `H3 V2V Init`, wire `audio_latent` from that encode and set
|
| 690 |
+
`audio_mode` to `follow the original performance (0.5)`.
|
| 691 |
+
|
| 692 |
+
Nothing else changes. Both new inputs are optional and default to the old
|
| 693 |
+
behaviour, so a graph without them behaves exactly as it did before.
|
| 694 |
+
|
| 695 |
+
**Which audio to ship.** `H3 Audio Recover` now takes plain-language
|
| 696 |
+
options instead of a bare 0 to 1 dial, because the two ends mean different
|
| 697 |
+
things depending on whether you seeded the rows:
|
| 698 |
+
|
| 699 |
+
- `keep the original performance (safe default)` is the one to use. Your
|
| 700 |
+
first pass's audio is already on the world clock and has never been
|
| 701 |
+
through a vocoder, so it is the best-sounding track you have. The seed's
|
| 702 |
+
job is to make the picture agree with it.
|
| 703 |
+
- `use pass 2's foley - ONLY IF the audio rows were seeded` gives you
|
| 704 |
+
foley scored for the new motion. Without a seed this is the rushed
|
| 705 |
+
defect described above. With one it is a real option, though it comes
|
| 706 |
+
back thinner: two phase-vocoder passes and a VAE round trip cost about a
|
| 707 |
+
third of the presence band between 3.4 and 8 kHz.
|
| 708 |
+
|
| 709 |
+
**What it costs the picture: nothing measurable.** Same graph with and
|
| 710 |
+
without the seed, everything else held, came out 91.8 against 91.3 on
|
| 711 |
+
laplacian variance, which is inside the noise floor of the video encoder
|
| 712 |
+
itself. It moves most pixels, because it is steering a joint audio-video
|
| 713 |
+
latent and the two halves are denoised together, but it does not soften.
|
| 714 |
+
|
| 715 |
+
**Where this has and has not been tested.** It has been run on a handful
|
| 716 |
+
of clips, all sword-fight material with two speakers trading lines over
|
| 717 |
+
fast motion, at hold factors around 4 and dilations from 2.4x to 2.7x.
|
| 718 |
+
It has not been tried on a single speaker, on speech over music, on
|
| 719 |
+
non-anime footage, or at other hold factors. Treat it as alpha, and
|
| 720 |
+
listen to the tail of a clip as well as the start, because the tail is
|
| 721 |
+
where an unheld region will sound normal whether or not anything is wrong.
|
| 722 |
+
|
| 723 |
+
### The motion adapter (pilot)
|
| 724 |
+
|
| 725 |
+
A rank-16 LoRA trained on the de-rope task itself: frames inside a motion
|
| 726 |
+
burst held out, the rest kept as clean context, the model asked to fill
|
| 727 |
+
the burst back in. Applied to the de-rope pass only, it teaches the base
|
| 728 |
+
model to spend the stretched clock on smoothness instead of invention.
|
| 729 |
+
Weights and the measurement write-up:
|
| 730 |
+
[huggingface.co/matlowai/MiniMax-H3-Motion-Adapter](https://huggingface.co/matlowai/MiniMax-H3-Motion-Adapter).
|
| 731 |
+
Graphs: [`examples/experimental/motion_pipeline_adapter_api.json`](examples/experimental/motion_pipeline_adapter_api.json)
|
| 732 |
+
(text to video, then the standard de-rope with one `LoraLoaderModelOnly`
|
| 733 |
+
on the pass-2 model) and
|
| 734 |
+
[`examples/experimental/motion_window_pinned_adapter_api.json`](examples/experimental/motion_window_pinned_adapter_api.json)
|
| 735 |
+
(a clip you already have: the window is regenerated at denoise 0.70 with
|
| 736 |
+
its first and last frames pinned, adapter on the pass-2 model; this is the
|
| 737 |
+
graph behind the demo page's headline quad). Clips on the
|
| 738 |
+
[demo page](https://matlowai.github.io/ComfyUI-MAINodes/#adapter).
|
| 739 |
+
|
| 740 |
+
Settings, measured on clips it never saw: strength 1.0 (strength and
|
| 741 |
+
inject are the same dial; 1.0 wins alternation in every paired cell);
|
| 742 |
+
inject is what you tune, 0.45 for character work where the base
|
| 743 |
+
over-produces, 0.30 where identity or props are the deliverable, anything
|
| 744 |
+
on very fast anime; keep the tail guide on. Known costs: about 1 dB of
|
| 745 |
+
anchor fidelity on native keyframes, over-correction of calm chains, and
|
| 746 |
+
muted colour on strong-colour or particle-heavy subjects, worst on the
|
| 747 |
+
Ref2VA checkpoint and on full-clip 3x passes (a prismatic creature came
|
| 748 |
+
out as a plain calico both ways). It is a pilot released as an intermediate option; a
|
| 749 |
+
more ambitious all-in-one adapter is in progress and may not work.
|
| 750 |
+
|
| 751 |
+
### Repairing a few bad frames
|
| 752 |
+
|
| 753 |
+
Sometimes a finished clip is fine except for half a second of it. Repair is
|
| 754 |
+
the bilateral sibling of the extension nodes: extension asks what comes
|
| 755 |
+
after a clip, repair says these frames are wrong, render them again, and
|
| 756 |
+
give the clip back with only those frames different.
|
| 757 |
+
|
| 758 |
+
`H3 Repair Plan` does the arithmetic. You give it the clip length, the bad
|
| 759 |
+
frame range (0-based, inclusive) and a hold, and it emits four things: a
|
| 760 |
+
hold map for `H3 Time Smear`, a time-varying regenerate mask for
|
| 761 |
+
`H3 V2V Init` in DILATED coordinates, the splice bounds, and a report that
|
| 762 |
+
prices the pass. `H3 Repair Splice` then reassembles: original pixels
|
| 763 |
+
outside the splice, repaired pixels inside, nothing resampled and nothing
|
| 764 |
+
blended.
|
| 765 |
+
|
| 766 |
+
Two rules run through both nodes, and they pull in opposite directions:
|
| 767 |
+
|
| 768 |
+
- **Masks snap to tokens.** A latent time token cannot be half
|
| 769 |
+
regenerated, so the regenerate mask has to cover whole tokens. Frames
|
| 770 |
+
group in 17s, each group is 5 tokens covering (1, 4, 4, 4, 4) frames, so
|
| 771 |
+
"frames 45 to 47 are wrong" becomes "regenerate 43 to 50".
|
| 772 |
+
- **Splices are free.** Reassembly is pure frame selection and may cut
|
| 773 |
+
anywhere. So extend the mask where the grid forces it, then choose the
|
| 774 |
+
splice points for picture reasons: enter in quiet motion, and exit ON a
|
| 775 |
+
shot cut whenever one is near.
|
| 776 |
+
|
| 777 |
+
The cut rule is the one that earns its keep. On the validation clip the
|
| 778 |
+
token snap pushed the regenerated span across a hard cut and the model
|
| 779 |
+
reinvented the first frames of the next shot; the exit seam popped at 2.6x
|
| 780 |
+
the clip's median frame delta. Extending the mask through the cut did not
|
| 781 |
+
fix it (2.3x). Handing back to the original AT its own cut did. Type the
|
| 782 |
+
cut into `shot_cuts` and the plan carries the mask through it while the
|
| 783 |
+
splice exits on it, so the seam becomes the edit the shot always had.
|
| 784 |
+
|
| 785 |
+
Measured on that cell (90 frames, bad 45 to 47, hold 3, cut at 53): entry
|
| 786 |
+
seam 3.99/255, which is 0.26x the clip's median frame delta; exit 138.04
|
| 787 |
+
against the original's own 135.57 at the same cut, a difference of 1.8%;
|
| 788 |
+
maximum absolute pixel difference outside the splice 0, asserted by the
|
| 789 |
+
node. The plan's own entry check called frame 43 quiet (2.0 against a
|
| 790 |
+
median of 15.2). `tests/test_repair_plan.py` pins that arithmetic and runs
|
| 791 |
+
on CPU with no models.
|
| 792 |
+
|
| 793 |
+
**AUDIO IS STILL A WORK IN PROGRESS for the repair verb.** The video splice
|
| 794 |
+
is frame-exact; audio handling across the splice is not yet finished.
|
| 795 |
+
`H3 Repair Splice` passes an AUDIO input straight through, untouched and
|
| 796 |
+
the same length, which is right when the repair did not change the
|
| 797 |
+
performance and is not a general answer. Repair picture, keep your original
|
| 798 |
+
track, and do not expect the node to solve a repaired line of dialogue yet.
|
| 799 |
+
|
| 800 |
+
Two things to set on the rest of the graph: `H3 Time Smear`'s `dilation`
|
| 801 |
+
must equal the plan's `hold`, and its `expand_to_end` must be OFF, because
|
| 802 |
+
an end-expansion rewrite moves every dilated coordinate the plan just
|
| 803 |
+
computed. The plan says so in its report if it detects the case.
|
| 804 |
+
|
| 805 |
+
### DyRoPE: who gets the true clock
|
| 806 |
+
|
| 807 |
+
`H3 True Clock` tells the model the true world duration of a time-smeared
|
| 808 |
+
clip, and it works: background agents hold world speed through a held span.
|
| 809 |
+
It was also measured to cost seam flash (7.17x against a control's 1.87x)
|
| 810 |
+
and jitter (5.06x against 0.85x), because the same non-uniform time grid
|
| 811 |
+
goes to all 50 blocks at every step, and that is off-distribution.
|
| 812 |
+
|
| 813 |
+
`H3 DyRoPE` splits the two geometries apart so they can be handed out
|
| 814 |
+
selectively. Physical is True Clock's grid, compact is the stock uniform
|
| 815 |
+
grid the model was trained on. `physical_all` reproduces True Clock exactly
|
| 816 |
+
and `compact_all` reproduces having no node in the graph at all; those are
|
| 817 |
+
the identity arms. The `*_blocks` modes give a named block range one
|
| 818 |
+
geometry and every other block the other, and the `fade_*` modes give every
|
| 819 |
+
block one interpolated grid per step, as a function of sigma, complete at
|
| 820 |
+
`fade_end`.
|
| 821 |
+
|
| 822 |
+
What a twelve-arm dose ladder on one clip says, meters first:
|
| 823 |
+
|
| 824 |
+
| arm | flash | jitter |
|
| 825 |
+
|---|---|---|
|
| 826 |
+
| stock clock everywhere (control) | 2.09x | 1.77x |
|
| 827 |
+
| physical everywhere (= True Clock) | 8.14x | 2.93x |
|
| 828 |
+
| physical in blocks 0-9 only | 7.84x | 2.95x |
|
| 829 |
+
| physical in blocks 25-49 only | 1.87x | 2.61x |
|
| 830 |
+
| physical in blocks 30-49 only | 1.84x | 2.33x |
|
| 831 |
+
| physical in blocks 40-49 only | 1.94x | 1.39x |
|
| 832 |
+
| fade to stock by sigma 0.3 | 1.76x | 2.72x |
|
| 833 |
+
| fade to stock by sigma 0.7 | 1.80x | 2.06x |
|
| 834 |
+
| stock at high sigma, physical by sigma 0.5 | 2.58x | 8.32x |
|
| 835 |
+
|
| 836 |
+
Two readings. **Flash is an early-block effect**: ten early blocks with
|
| 837 |
+
true time reproduce the whole True Clock flash, and every arm that keeps
|
| 838 |
+
true time out of blocks 0-9 sits at the control's level. **Jitter is a dose
|
| 839 |
+
effect**: it scales with how much physical-time exposure the run gets, once
|
| 840 |
+
the early blocks are excluded.
|
| 841 |
+
|
| 842 |
+
The meters cannot pick the winner, though, because they say nothing about
|
| 843 |
+
whether an arm still delivers what the true clock was FOR. Playback maps a
|
| 844 |
+
benefit boundary the meters do not see: world timing survives at blocks
|
| 845 |
+
40-49 for all steps, and at all blocks above sigma 0.7, and at every larger
|
| 846 |
+
dose. It dies at `fade_end` 0.8 (the window gets too short), and at any
|
| 847 |
+
block restriction stacked on top of the sigma window. Every attempt to cut
|
| 848 |
+
jitter below those two doses lost the timing.
|
| 849 |
+
|
| 850 |
+
So there are two settings worth recommending, and neither is a winner. They
|
| 851 |
+
trade the same two things against each other, and which one is right is a
|
| 852 |
+
property of your clip. Neither is a code default; the node ships with its
|
| 853 |
+
defaults unchanged.
|
| 854 |
+
|
| 855 |
+
- **`physical_blocks` 30-49, the timing-fidelity setting.** The held span
|
| 856 |
+
holds world speed neutrally, without over- or undershooting it. The cost
|
| 857 |
+
is a slight flicker or hitch inside the span (this dose metered 2.33x
|
| 858 |
+
jitter against the stock control's 1.77x).
|
| 859 |
+
- **`physical_blocks` 40-49, the minimal-shimmer setting.** The smoothest
|
| 860 |
+
span of the pair and the cleanest meters on the board (1.17x to 1.39x
|
| 861 |
+
jitter across three takes, always below the stock control). On some
|
| 862 |
+
content it slightly overshoots the correction, so the span can read
|
| 863 |
+
over-smooth or slowed.
|
| 864 |
+
- **`fade_physical_to_compact` at `fade_end 0.7` is the flash-free
|
| 865 |
+
alternative.** No flash at all in playback, visible jitter (2.06x to
|
| 866 |
+
2.87x across takes), timing held.
|
| 867 |
+
- The mechanism to keep in mind while you choose: **less dose than these
|
| 868 |
+
loses the world-time correction entirely, more dose adds shimmer.** There
|
| 869 |
+
is no setting that is quieter than 40-49 and still on time; every arm
|
| 870 |
+
tried below these doses came back on the stock clock's speed.
|
| 871 |
+
- The failure being fixed is worth naming, because it is easy to mistake
|
| 872 |
+
for a de-rope defect: on the stock clock a dilated span PLAYS
|
| 873 |
+
ACCELERATED. Background agents in the held region move too fast, because
|
| 874 |
+
the model was told the span is shorter than it really is.
|
| 875 |
+
|
| 876 |
+
`H3 DyRoPE` is not composable with `H3 Streamed Blocks` in the `*_blocks`
|
| 877 |
+
modes: both own the double-block replacement slot and ComfyUI keeps one per
|
| 878 |
+
block. The capability probe reports the collision. The fade modes use no
|
| 879 |
+
block patches and compose fine.
|
| 880 |
+
|
| 881 |
+
The clips these numbers came from are on the public deck:
|
| 882 |
+
<https://matlowai.github.io/flipbook/trueclock.html>.
|
| 883 |
+
|
| 884 |
+
### Anchoring audio across a join (alpha)
|
| 885 |
+
|
| 886 |
+
`H3 V2V Init`'s `audio_strength` is one number for the whole track.
|
| 887 |
+
`audio_prefix_ticks` makes it a shape: the first n audio-latent ticks are
|
| 888 |
+
frozen to the seeded `audio_latent` content (noise mask 0 there), and every
|
| 889 |
+
later tick keeps `audio_strength`. That is the audio twin of the video
|
| 890 |
+
prefix freeze, and it is what lets a continuation carry the previous
|
| 891 |
+
segment's sound into the new one instead of starting a new performance.
|
| 892 |
+
|
| 893 |
+
The four things to know before wiring it:
|
| 894 |
+
|
| 895 |
+
- **It needs `audio_latent` wired.** With nothing seeded there is nothing
|
| 896 |
+
to freeze to, and the setting does nothing but confuse the log. The tick
|
| 897 |
+
clock is the audio latent's, not the video's: a 39-frame carried handle
|
| 898 |
+
is exactly 65 ticks.
|
| 899 |
+
- **An AddGuide AUDIO guide and `ref_audios` cannot coexist.** They collide
|
| 900 |
+
in conditioning-audio packing. (Measured on core 82f839f5; treat as a current-core rule, not a model property.) Pick one route for a graph and stay on it.
|
| 901 |
+
- **`audio_prefix_release_ticks` changes what assembly has to do.** With a
|
| 902 |
+
release above 0 the last n ticks of the frozen prefix ramp from 0 up to
|
| 903 |
+
`audio_strength` on a half cosine instead of cutting hard (8 ticks, 0.2
|
| 904 |
+
s, is the upstream lineage's tested recipe). If you use it, the assembly step must let the
|
| 905 |
+
CONTINUATION own the overlap tail, or the trim throws the ramp away and
|
| 906 |
+
you paid for nothing.
|
| 907 |
+
- **Scope: it carries local continuity, not structure.** Level, timbre and
|
| 908 |
+
beat come across the join. It does not carry global phrase structure. A
|
| 909 |
+
longer handle helps at the seam (90 frames measured 0.953 overlap
|
| 910 |
+
correlation vs 0.92 at 39) and gives the model a full musical phrase of
|
| 911 |
+
context, but only a pinned master track GUARANTEES the structure; that
|
| 912 |
+
guarantee is by construction.
|
| 913 |
+
|
| 914 |
+
Alpha, and the honest reading is that this has been exercised on a narrow
|
| 915 |
+
slice of material. Both inputs default to 0 and the scalar path is
|
| 916 |
+
unchanged at that value, so a graph without them behaves exactly as it did
|
| 917 |
+
before.
|
| 918 |
+
|
| 919 |
+
### Chained segments: drift and colour (alpha)
|
| 920 |
+
|
| 921 |
+
Two different things decay when you chain segments, and they have separate
|
| 922 |
+
nodes.
|
| 923 |
+
|
| 924 |
+
**`H3 Drift Control`** attacks clean-conditioning error. A chained
|
| 925 |
+
segment's carried prefix is bit-clean, which one seam loves and a chain
|
| 926 |
+
does not: repeated clean conditioning accumulates contrast and texture
|
| 927 |
+
error, and a seam probe watches the join decay from about 0.9 at the first
|
| 928 |
+
join to about 0.65 by the second across every variant tested. The node
|
| 929 |
+
gives the disposable carried VIDEO prefix a small, schedule-matched amount
|
| 930 |
+
of the sampler's own noise at every model evaluation, tapering to exact at
|
| 931 |
+
the seam end, so the model never treats the prefix as impossibly clean.
|
| 932 |
+
Audio is deliberately untouched: the frozen per-tick audio prefix stays
|
| 933 |
+
hard.
|
| 934 |
+
|
| 935 |
+
What it demands, all of which it enforces rather than guesses:
|
| 936 |
+
|
| 937 |
+
- `prefix_frames` must equal the plan's handle and sit on the 17k+5 grid.
|
| 938 |
+
39 frames is 12 latent steps, and `matched_steps` + `taper_steps` must
|
| 939 |
+
sum to exactly that (8 + 4 is the field-validated recipe).
|
| 940 |
+
- The latent must already carry a noise mask: wire `H3 V2V Init` first. The
|
| 941 |
+
node rewrites that mask, it does not create one.
|
| 942 |
+
- It **refuses to stack** on another dynamic denoise-mask patch. Two of
|
| 943 |
+
them cannot share the model.
|
| 944 |
+
- Sigma-split samplers are not supported; our graphs run one
|
| 945 |
+
`SamplerCustomAdvanced` per pass.
|
| 946 |
+
- **Replicated across two chains and new seeds.** Joins hold 0.85-0.89
|
| 947 |
+
through link 4 on the newer chain; the earlier chain, before this node,
|
| 948 |
+
fell to the 0.65 class by its second join. That is a cross-chain
|
| 949 |
+
comparison, not a paired drift-off arm. The matched+taper recipe itself
|
| 950 |
+
is still one recipe, not a swept space.
|
| 951 |
+
|
| 952 |
+
**`H3 Delta Color Carry`** (with `H3 Scene Color Stats`) attacks the other
|
| 953 |
+
decay: every carried handle is a VAE round trip, each round trip darkens
|
| 954 |
+
about 2.4%, and chains accumulate it. Rather than regrading the sampled
|
| 955 |
+
latent, it decodes the prefix once, applies a weak scene-one
|
| 956 |
+
exposure/saturation correction in RGB, encodes BOTH the original and the
|
| 957 |
+
corrected frames, and adds only E(corrected) - E(original) to the latent.
|
| 958 |
+
The encode bias appears in both terms and cancels; only the intended grade
|
| 959 |
+
survives. The delta is spatially low-passed and tapered from zero at the
|
| 960 |
+
old edge to full strength beside the generated future, and audio is never
|
| 961 |
+
touched. Wire the delivered tail of scene one through one
|
| 962 |
+
`H3 Scene Color Stats` into `anchor_stats`, the current predecessor's tail
|
| 963 |
+
through another into `current_stats`, and the node between the extend
|
| 964 |
+
graph's `VAEEncode` and `H3 V2V Init`. On the first link the two stats are
|
| 965 |
+
the same tail, the transform is the identity, and the latent passes
|
| 966 |
+
through untouched.
|
| 967 |
+
|
| 968 |
+
Status, stated plainly: Active path exercised on a real 4-link render 2026-08-24: it fires, corrects in the anchor direction, and stays sub-visible under its clamps on mildly-drifted content; on the deepest tested link it fires on both channels at or near its clamp. A strong-drift bench (where the 2.4% per-round-trip signature actually accrues) is still owed.
|
| 969 |
+
|
| 970 |
+
Both nodes are adapted, with permission of the licence, from
|
| 971 |
+
ComfyUI-MiniMaxH3-Contex-Loop (GPL-3.0, the same licence as this pack):
|
| 972 |
+
Drift Control from its Drift-Control AV, Delta Color Carry from its
|
| 973 |
+
Color-Stable Drift AV. The per-module credit lines in `h3_drift.py` and
|
| 974 |
+
`h3_color_carry.py` say exactly what was taken and what was deliberately
|
| 975 |
+
not ported.
|
| 976 |
+
|
| 977 |
+
### Retiming part of a clip (window mode)
|
| 978 |
+
|
| 979 |
+
When only an excerpt of a long clip needs retiming, the graph passes the
|
| 980 |
+
rest through untouched and expands the excerpt. That shape is worth its own
|
| 981 |
+
page, because the window is encoded in four coupled places (two crops, the
|
| 982 |
+
hold map, and the regeneration's conditioning length) and editing three of
|
| 983 |
+
them produces a broken render whose damage sits at the OLD boundary. Window
|
| 984 |
+
starts also quantize to a 17-frame lattice, 0.708 s at 24 fps, which is why
|
| 985 |
+
"start at second 1" is not a thing the graph can do.
|
| 986 |
+
|
| 987 |
+
The arithmetic, the failure signature and the procedure are in
|
| 988 |
+
[docs/WINDOW_MODE.md](docs/WINDOW_MODE.md).
|
| 989 |
+
|
| 990 |
+
## Contact-Sheet diffusion
|
| 991 |
+
|
| 992 |
+
Five standalone image latents packed on the model's time axis, jointly
|
| 993 |
+
denoised, decoded independently. Use with a Turnaround LoRA from
|
| 994 |
+
[matlod/minimax-h3-turnaround](https://huggingface.co/matlod/minimax-h3-turnaround).
|
| 995 |
+
Nodes: **H3 Contact Sheet**, **H3 Contact Sheet Decode**.
|
| 996 |
+
Drag-in workflow: [`examples/contact_sheet.json`](examples/contact_sheet.json)
|
| 997 |
+
(API twin alongside) — point the LoadImage at your reference image, pick
|
| 998 |
+
your downloaded LoRA file, queue. `ref_image` is optional: leave it
|
| 999 |
+
unconnected for a text-only sheet (drop the `<Picture 1>` tag from the
|
| 1000 |
+
prompt); [`examples/contact_sheet_t2i.json`](examples/contact_sheet_t2i.json)
|
| 1001 |
+
is that graph. Stock loaders and sampler throughout;
|
| 1002 |
+
28 steps of res_multistep at denoise 1.0, LoRA strength 0.75. A scripted
|
| 1003 |
+
example is in [`example_api_workflow.py`](example_api_workflow.py).
|
| 1004 |
+
Previously published as ComfyUI-H3-ContactSheet; that repo remains up for
|
| 1005 |
+
existing installs.
|
| 1006 |
+
|
| 1007 |
+
## Alpha work
|
| 1008 |
+
|
| 1009 |
+
This pack ships finished work and unfinished work in the same install.
|
| 1010 |
+
The Motion Lab pipeline and the contact sheet above are the finished part.
|
| 1011 |
+
Alongside them are a research subsystem (Concept Lab), a timeline surface,
|
| 1012 |
+
the audio init for dialogue, the motion adapter pilot, and the manual mask
|
| 1013 |
+
paths, all at varying degrees of unfinished.
|
| 1014 |
+
|
| 1015 |
+
**[ALPHA.md](ALPHA.md) says which is which**, what each one can and cannot
|
| 1016 |
+
do today, and where each has and has not been tested. Alpha nodes carry
|
| 1017 |
+
`(alpha)` in the ComfyUI menu, they load behind a guarded loader so a
|
| 1018 |
+
broken one cannot take the pack down, and none of them changes existing
|
| 1019 |
+
behaviour or defaults.
|
| 1020 |
+
|
| 1021 |
+
## Comparing a batch of renders
|
| 1022 |
+
|
| 1023 |
+
[`tools/compare_deck/`](tools/compare_deck/) builds the review page used for
|
| 1024 |
+
the sweeps in this pack: a manifest JSON in, one self-contained HTML file
|
| 1025 |
+
out, with a card grid, a sortable metric table, a synced A/B (side by side
|
| 1026 |
+
or wipe) on a waveform timeline, and a frame-exact export of the current
|
| 1027 |
+
view. `make_manifest.py` drafts the manifest from a folder of clips; you
|
| 1028 |
+
fill in the groups and descriptions. The clips stay where they are, so
|
| 1029 |
+
nothing but HTML is produced. A live deck built with it:
|
| 1030 |
+
<https://matlowai.github.io/flipbook/trueclock.html>. Standalone Python, no
|
| 1031 |
+
ComfyUI import, see [`tools/compare_deck/README.md`](tools/compare_deck/README.md).
|
| 1032 |
+
|
| 1033 |
+
**MAI Video Compare** is the same review surface inside ComfyUI, and it grew
|
| 1034 |
+
into a full player. Wire 2 to 6 renders into the node and queue once: it
|
| 1035 |
+
writes each one as a small h264 preview (CPU, no VAE, no tensors kept) and
|
| 1036 |
+
the widget plays them as live `<video>` elements, so it costs no VRAM.
|
| 1037 |
+
Driving it, once the previews load:
|
| 1038 |
+
|
| 1039 |
+
- **Pick the pair.** Keys 1 to 6 choose sources (the last two pressed are A
|
| 1040 |
+
and B), and the mode buttons switch between side by side, flip, wipe and,
|
| 1041 |
+
for more than two, a synchronized grid. Hover a source to hear it, click
|
| 1042 |
+
to lock its audio there.
|
| 1043 |
+
- **Move together.** Space plays and pauses everything, the arrows step one
|
| 1044 |
+
frame (shift steps 12), F flickers A against B. All sources stay on the
|
| 1045 |
+
lead's clock.
|
| 1046 |
+
- **Loop the moment that matters.** The timeline draws the A side's stereo
|
| 1047 |
+
waveform, and the loop brackets are always live: `i` and `o` set them from
|
| 1048 |
+
the playhead, or drag the grab tabs, or double-click to reset to the whole
|
| 1049 |
+
clip. Playback stays inside the brackets, which is how you watch one seam
|
| 1050 |
+
fifty times without touching anything.
|
| 1051 |
+
- **Let the graph mark it up.** Wire `H3 Time Smear`'s `hold_map_used` into
|
| 1052 |
+
the node's `hold_map` and the regenerated window is drawn as a band with
|
| 1053 |
+
enter and exit blips as the playhead crosses it. `curves` takes
|
| 1054 |
+
`{name: [per-frame floats]}` and draws each as a lane under the playhead.
|
| 1055 |
+
Both inputs are optional and were appended last, so older graphs keep
|
| 1056 |
+
their widget order.
|
| 1057 |
+
- **Keep the verdict.** Enter stars the B side, which sets the node's
|
| 1058 |
+
`winner` widget; the NEXT queue passes that source through `winner_video`
|
| 1059 |
+
and `winner_index`. Pick, then finalize in a second execution: a graph
|
| 1060 |
+
never waits on a human mid-run.
|
| 1061 |
+
- **Export what you just watched.** The export button renders the current
|
| 1062 |
+
pair, in the current mode, over the current loop, plus a two second end
|
| 1063 |
+
card carrying each side's label, seed, frame count, fps and size. It is
|
| 1064 |
+
frame-exact through WebCodecs (VP9, AV1 or H.264, up to 100 Mbps, Opus
|
| 1065 |
+
audio) and falls back to real-time capture where WebCodecs or the muxers
|
| 1066 |
+
are unavailable, which the dialog says before you press anything. The two
|
| 1067 |
+
muxers are vendored under `web/vendor/` (MIT, see its README).
|
| 1068 |
+
- The gear panel holds the rest: region edit, the frame counter overlay, and
|
| 1069 |
+
which blips fire. Its settings are per node and survive a reload.
|
| 1070 |
+
|
| 1071 |
+
Known issues (alpha): in-node playback sync between sources is
|
| 1072 |
+
approximate and can drift or stagger, and scrubbing during playback can
|
| 1073 |
+
wedge the transport (re-queue or reload recovers). In clock sync mode
|
| 1074 |
+
audio follows only the clock-owning side, so locking audio to the other
|
| 1075 |
+
pane is silent until sync mode is flipped. The realtime export records
|
| 1076 |
+
audio from the locked side even when it is not one of the two playing
|
| 1077 |
+
panes; prefer the precise export. The standalone deck
|
| 1078 |
+
pages (`tools/compare_deck/`) are the reliable review surface today; a
|
| 1079 |
+
combined-single-stream preview that removes the sync problem by
|
| 1080 |
+
construction is the planned next iteration.
|
| 1081 |
+
|
| 1082 |
+
## Install
|
| 1083 |
+
|
| 1084 |
+
```
|
| 1085 |
+
cd ComfyUI/custom_nodes
|
| 1086 |
+
git clone https://github.com/matlowai/ComfyUI-MAINodes
|
| 1087 |
+
```
|
| 1088 |
+
|
| 1089 |
+
**The example graphs also need [ComfyUI-KJNodes](https://github.com/kijai/ComfyUI-KJNodes).**
|
| 1090 |
+
They use `PathchSageAttentionKJ`,
|
| 1091 |
+
`MiniMaxH3MemoryEfficientSageAttentionPatch` and `MiniMaxChunkFeedForward`
|
| 1092 |
+
from it. Without KJNodes a graph loads with those three missing and will
|
| 1093 |
+
not run. The nodes in this pack themselves have no such dependency.
|
| 1094 |
+
|
| 1095 |
+
Restart ComfyUI. Nodes appear under `latent/minimax/motion`,
|
| 1096 |
+
`image/minimax/motion`, and `sampling/custom_sampling/schedulers`.
|
| 1097 |
+
|
| 1098 |
+
## Support MAINodes
|
| 1099 |
+
|
| 1100 |
+
MAINodes is free and open source. If it saved you time, fixed a workflow
|
| 1101 |
+
or made something cool possible, you can help fund the next round of GPU
|
| 1102 |
+
time, benchmarking and increasingly questionable experiments:
|
| 1103 |
+
|
| 1104 |
+
- [Support MAINodes on Ko-fi](https://ko-fi.com/matlowai)
|
| 1105 |
+
- Or just star the repo; that helps too.
|
| 1106 |
+
|
| 1107 |
+
## License
|
| 1108 |
+
|
| 1109 |
+
GPL-3.0-or-later. Copyright (C) 2026 MATLOWAI. See LICENSE.
|
| 1110 |
+
|
| 1111 |
+
## De-rope for any model (branch `generic-derope`)
|
| 1112 |
+
|
| 1113 |
+
One hold map, any regenerating model: `H3 Clock Remap` retimes the oracle's
|
| 1114 |
+
plan onto a model's clock from a preset (`minimax-h3`, `ltx-2.5`, a
|
| 1115 |
+
user-editable registry for the rest), `H3 Time Smear` pads to that model's
|
| 1116 |
+
grid, and `H3 Save Hold Map` keeps the clock beside the render. The LTX-2.5
|
| 1117 |
+
graphs behind https://matlowai.github.io/flipbook/derope.html live there too.
|
| 1118 |
+
See `DEROPE_ANY_MODEL.md`. Alpha, on a branch until the oracle-shaped LTX arm
|
| 1119 |
+
is measured.
|
ComfyUI-MiniMaxH3-Context-Loop/ComfyUI-MAINodes/RESEARCH_NOTES_ATOS.md
ADDED
|
@@ -0,0 +1,234 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Research notes: Adaptive Temporal Over-Sampling (ATOS)
|
| 2 |
+
|
| 3 |
+
Working notes for turning Motion Lab into a paper. Everything here is
|
| 4 |
+
hypothesis and plan, not established result; the demos in the README are
|
| 5 |
+
same-seed comparisons on our test clips, and the whole point of this
|
| 6 |
+
document is to close the gap between "it looks great here" and evidence.
|
| 7 |
+
|
| 8 |
+
## The claim we think we can defend
|
| 9 |
+
|
| 10 |
+
When motion exceeds the temporal representational capacity of a
|
| 11 |
+
temporally compressed video generator, do not interpolate the bad output
|
| 12 |
+
afterward. Detect the overloaded spans from the model's own latent,
|
| 13 |
+
temporally dilate exactly those spans with integer frame holds, let the
|
| 14 |
+
generative prior re-solve the motion video-to-video at partial denoise
|
| 15 |
+
with more temporal room, then return to the original clock by exact
|
| 16 |
+
frame selection. Training-free, no auxiliary model.
|
| 17 |
+
|
| 18 |
+
Candidate title: "Adaptive Temporal Over-Sampling for Training-Free
|
| 19 |
+
Fast-Motion Repair in Latent Video Diffusion".
|
| 20 |
+
|
| 21 |
+
Three contributions as implemented today:
|
| 22 |
+
|
| 23 |
+
1. Latent motion-overload detection. The jerk oracle reads a
|
| 24 |
+
phase-normalized third temporal difference of the video latent. No
|
| 25 |
+
optical flow, no segmentation, no extra network.
|
| 26 |
+
2. Generative temporal over-sampling with exact recovery. Integer holds,
|
| 27 |
+
partial-denoise regeneration, frame-selection recovery. The slowdown
|
| 28 |
+
lives in the content as a speed ramp; the model clock stays uniform.
|
| 29 |
+
3. Adaptive and early allocation. Quantile threshold, bridge and ramp
|
| 30 |
+
turn the signal into a compute policy (about 2.9x frame budget vs
|
| 31 |
+
uniform 4x on our clips); the probe variant reads the oracle from an
|
| 32 |
+
early x0 estimate instead of a finished baseline.
|
| 33 |
+
|
| 34 |
+
## Positioning vs prior art (as of 2026-08)
|
| 35 |
+
|
| 36 |
+
- DLFR-VAE and VGDFR recognize that temporal information density is not
|
| 37 |
+
uniform, and spend FEWER resources where motion is low, to accelerate.
|
| 38 |
+
ATOS inverts the direction: it spends MORE effective temporal capacity
|
| 39 |
+
where motion overloads the representation, to repair. Same
|
| 40 |
+
observation, opposite allocation.
|
| 41 |
+
- DiffuseSlide is training-free high-FPS generation via keyframes and
|
| 42 |
+
sliding-window denoising; it raises output frame rate rather than
|
| 43 |
+
repairing a native-rate clip and returning to the original clock.
|
| 44 |
+
- Large-motion VAE work documents that temporal compression degrades
|
| 45 |
+
under large motion, which is the underlying problem, addressed there
|
| 46 |
+
by training better VAEs rather than at inference time.
|
| 47 |
+
- A fresh literature sweep is required before drafting. Search terms:
|
| 48 |
+
training-free temporal super-resolution inside latent video diffusion,
|
| 49 |
+
motion-adaptive resampling at inference, work citing VGDFR.
|
| 50 |
+
|
| 51 |
+
## What honesty requires
|
| 52 |
+
|
| 53 |
+
- The README currently implies causality (one token cannot hold four
|
| 54 |
+
distinct poses, therefore the smear). Until the probe below runs, the
|
| 55 |
+
paper says: we hypothesize these failures arise in part from
|
| 56 |
+
fixed-rate temporal compression becoming insufficient for
|
| 57 |
+
high-information motion.
|
| 58 |
+
- The latent third difference is a heuristic overload proxy, not
|
| 59 |
+
physical jerk. Whether it beats simpler signals is an experiment, not
|
| 60 |
+
an assumption.
|
| 61 |
+
- Everything is currently demonstrated on one model family. The method's
|
| 62 |
+
components (grid constants, phase normalization, token mapping) are
|
| 63 |
+
model-specific in code even if the idea is not.
|
| 64 |
+
|
| 65 |
+
## Evidence plan
|
| 66 |
+
|
| 67 |
+
1. Fast-motion benchmark: 30 to 50 prompts x 3 seeds. Categories: rigid
|
| 68 |
+
fast motion, articulated humans, rotations and backflips,
|
| 69 |
+
projectiles, camera pans, camera plus subject motion, plus 10 to 20
|
| 70 |
+
low-motion negative controls. Vary every axis except the one being
|
| 71 |
+
measured.
|
| 72 |
+
2. Baselines: native generation; plain partial-denoise V2V with no
|
| 73 |
+
dilation at the same inject; uniform 2x/3x/4x dilation; adaptive
|
| 74 |
+
(ours); adaptive gated to user ranges; post-hoc frame interpolation
|
| 75 |
+
on the baseline; a dynamic-frame-rate method as a reference point if
|
| 76 |
+
feasible.
|
| 77 |
+
3. Ablations: difference order (first, second, third), phase
|
| 78 |
+
normalization on/off, quantile q, bridge, ramp, d_max, inject 0.5 to
|
| 79 |
+
0.8, final-latent oracle vs early-x0 oracle across probe step counts.
|
| 80 |
+
Include an external detector baseline: optical-flow magnitude and
|
| 81 |
+
flow acceleration on decoded frames.
|
| 82 |
+
4. Causal probes:
|
| 83 |
+
- VAE-only roundtrip: encode/decode real fast-motion footage at
|
| 84 |
+
native speed vs slowed variants and measure reconstruction error
|
| 85 |
+
against local motion magnitude. Tests the compression-bottleneck
|
| 86 |
+
hypothesis with no sampler in the loop.
|
| 87 |
+
- Failure severity vs oracle score on human-labeled clips.
|
| 88 |
+
- Trajectory-bank instrument: checkpoint a run every step, run the
|
| 89 |
+
oracle offline on each step's estimate, and plot agreement with the
|
| 90 |
+
final map as a function of denoising step. This is the figure that
|
| 91 |
+
justifies the early-probe variant.
|
| 92 |
+
5. Metrics: separate scores for repair (did the smear go) and
|
| 93 |
+
preservation (choreography, identity, background), plus a blinded
|
| 94 |
+
human A/B in playback. Still frames mislead about temporal quality in
|
| 95 |
+
both directions; playback judgment is primary.
|
| 96 |
+
6. Quality/compute Pareto: artifact reduction vs dilated tokens and wall
|
| 97 |
+
time, with uniform, adaptive, gated and segment-cropped variants as
|
| 98 |
+
points on one curve.
|
| 99 |
+
7. Cross-model transfer: port oracle, smear and recovery to at least one
|
| 100 |
+
other temporally compressed video model and report the outcome either
|
| 101 |
+
way. This is the highest-value single experiment.
|
| 102 |
+
|
| 103 |
+
## Ideas ledger (directions, not commitments)
|
| 104 |
+
|
| 105 |
+
- Spatiotemporal allocation D(x, y, t). The pipeline already carries
|
| 106 |
+
soft spatial masks end to end, but spatial masks only control
|
| 107 |
+
blending; the DiT still pays for every token. A real spatial compute
|
| 108 |
+
claim needs either token dropping/merging inside the model or a
|
| 109 |
+
spatial analog of the segment crop (crop a subject window, regenerate,
|
| 110 |
+
splice with feathered seams).
|
| 111 |
+
- Human-seeded motion attachment. The user marks the subject once; a
|
| 112 |
+
segmenter propagates it; an attachment envelope then grows from
|
| 113 |
+
co-motion rather than object identity, so swords, boards, hair and
|
| 114 |
+
splashes join the repair region while a panning background does not.
|
| 115 |
+
Keep the detector (when) and the targeting (where) separable so
|
| 116 |
+
ablations stay clean.
|
| 117 |
+
- Latent co-motion test. Correlate per-cell temporal-difference series
|
| 118 |
+
against the subject-averaged series to find regions participating in
|
| 119 |
+
the same motion event, directly in the latent, no optical flow. The
|
| 120 |
+
existing drift overlay (velocity high, jerk low) is a crude ancestor.
|
| 121 |
+
- Duplicate-motion (mirror) detection: PROBED AND NOT SOLVED by the
|
| 122 |
+
co-motion statistic, 2026-08-09. Two synthetic results worth keeping.
|
| 123 |
+
(1) The oracle is scale-invariant to duplication: because `q` is a
|
| 124 |
+
quantile of the profile, a reflection showing the same motion at the
|
| 125 |
+
same time yields a bit-identical hold map (profile correlation
|
| 126 |
+
0.9999). The old "mirrors make the oracle over-dilate" line in TUNING
|
| 127 |
+
was wrong and is now corrected; the real amplifier is `bridge`
|
| 128 |
+
welding a temporally separated reflection burst to the body's burst
|
| 129 |
+
(+26% dilated time in the probe, +18% of it from bridge alone).
|
| 130 |
+
(2) Lagged correlation between per-cell jerk profiles CANNOT separate
|
| 131 |
+
a mirror from two independent dancers: one body's own parts already
|
| 132 |
+
correlate at ~1.0, so the confuser floor sits above the signal
|
| 133 |
+
(margin -0.145 with heterogeneous limbs). Sharpening: the same
|
| 134 |
+
statistic that is right for motion ATTACHMENT (you want the whole
|
| 135 |
+
body, sword included, to score as one event) is structurally wrong
|
| 136 |
+
for DE-DUPLICATION. A duplicate detector needs the geometric fact a
|
| 137 |
+
mirror actually provides, approximate reflection symmetry of the
|
| 138 |
+
spatial jerk map about some axis, not temporal correlation. Untested;
|
| 139 |
+
needs a real mirror clip, since a synthetic with uniform blocks
|
| 140 |
+
passes any symmetry test trivially.
|
| 141 |
+
- **The oracle has no "nothing is wrong here" state, and that is the
|
| 142 |
+
mechanical explanation for the overzealousness complaint.** Measured
|
| 143 |
+
2026-08-09 on synthetic latents. Two clips, same object, same average
|
| 144 |
+
speed: one at constant velocity (true trajectory jerk exactly 0), one
|
| 145 |
+
in stop/go lurches (trajectory jerk 0.347). The oracle is not blind to
|
| 146 |
+
the difference, its profile peak contrast is about 2x higher on the
|
| 147 |
+
jerky clip. But it still asked for **3.06x dilation on the clip whose
|
| 148 |
+
jerk is exactly zero**, versus 3.47x on the jerky one. The cause is
|
| 149 |
+
structural: `q` is a quantile of the profile, so the oracle always
|
| 150 |
+
selects the top (1-q) of tokens no matter what the clip contains. It
|
| 151 |
+
can rank, but it cannot abstain. A clip that needs no repair still
|
| 152 |
+
gets a 3x budget spent on its fastest quarter.
|
| 153 |
+
Second measurement, real clips: in the value domain the third
|
| 154 |
+
difference is nearly the same signal as the first, corr(|d1|,|d3|) =
|
| 155 |
+
0.960 and 0.977 on two of our renders. That is expected analytically,
|
| 156 |
+
since a textured object passing a location makes the value there
|
| 157 |
+
pulse, and a pulse has large differences of every order even at
|
| 158 |
+
constant velocity. So the score is heavily contaminated by motion
|
| 159 |
+
energy: it is closer to "how much is moving" than to "how abruptly
|
| 160 |
+
the motion changes", and the name oversells it.
|
| 161 |
+
Two consequences. (1) An **absolute** gate belongs next to the
|
| 162 |
+
relative one, so a smooth clip can be told it needs nothing, which
|
| 163 |
+
would also stop the pipeline inventing beats in dilated time it never
|
| 164 |
+
needed. (2) The trajectory-domain measurement is the more honest
|
| 165 |
+
instrument: tracking a body's centroid and differentiating THAT gave
|
| 166 |
+
jerk a visibly narrower profile than velocity (top-decile share 0.29
|
| 167 |
+
vs 0.22, and 0.47 vs 0.37 on a second clip) with only moderate
|
| 168 |
+
correlation between them (r = 0.46 and 0.72), which is the separation
|
| 169 |
+
the value-domain score fails to make. Method, so the numbers are reproducible without our scripts: build two
|
| 170 |
+
synthetic latent sequences with matched mean speed, one constant-velocity
|
| 171 |
+
and one lurching, run the shipped oracle on both; then for the real clips,
|
| 172 |
+
track a bright-region centroid per frame and differentiate the TRAJECTORY
|
| 173 |
+
rather than the latent values.
|
| 174 |
+
- **PARKED FOR A SECOND OPINION, 2026-08-09. Reframe: we
|
| 175 |
+
built an allocator, not a meter.** The shipped oracle answers "given
|
| 176 |
+
that I am going to spend a budget, where should it go", a relative
|
| 177 |
+
allocation between roughly 5x slow and not at all, spreading the extra
|
| 178 |
+
time as well as it can. The question it does not answer is "how much
|
| 179 |
+
extra time does THIS clip actually need". Those are different
|
| 180 |
+
problems and only the second one can decline to act. Degenerate case
|
| 181 |
+
that shows the gap plainly: a clip that is high jerk from start to
|
| 182 |
+
finish. The quantile still picks a top quarter, which is arbitrary,
|
| 183 |
+
and the other three quarters go unhelped even though they need help
|
| 184 |
+
just as much. Uniform dilation is the only current answer and it is a
|
| 185 |
+
blunt one.
|
| 186 |
+
Seed for the meter, not a decision: the project's founding sentence
|
| 187 |
+
already contains an absolute unit. A token spans four frames, and the
|
| 188 |
+
failure is that those four frames need four distinct poses the token
|
| 189 |
+
cannot hold. So the natural absolute quantity is **demand over
|
| 190 |
+
capacity**, how much pose change a span requires against how much one
|
| 191 |
+
token can represent, with the hold count falling out as the ratio
|
| 192 |
+
rounded up. That gives dilate-everything on a uniformly violent clip,
|
| 193 |
+
dilate-nothing on a smooth one, and it is calibratable once against
|
| 194 |
+
the speed at which smear actually appears rather than being fixed by
|
| 195 |
+
a quantile forever.
|
| 196 |
+
**Second parked observation (an impression from playback, explicitly not
|
| 197 |
+
yet tested): global frame jerk, for example the background decelerating,
|
| 198 |
+
seems to produce less dangerous artifacts than articulated subject
|
| 199 |
+
jerk.** If that survives testing, a plausible reason is that a global
|
| 200 |
+
motion is low dimensional, a whole-frame shift is one simple thing a
|
| 201 |
+
token can represent cheaply, whereas an articulated body needs many
|
| 202 |
+
independent degrees of freedom in the same budget. That would mean
|
| 203 |
+
the quantity that matters is not jerk alone but jerk weighted by the
|
| 204 |
+
complexity of whatever is jerking, and it strengthens the case for
|
| 205 |
+
measuring residual jerk after global motion is subtracted rather than
|
| 206 |
+
raw jerk. Do not act on this before it is measured; both halves are open.
|
| 207 |
+
- Mask hysteresis. Once a region becomes attached, keep it attached for
|
| 208 |
+
a short window unless evidence says it separated; prevents flicker on
|
| 209 |
+
hands, props and cloth. Attachment-then-separation events (a thrown
|
| 210 |
+
ball spawning its own track) are future work.
|
| 211 |
+
- Latent freeze without feathering (now the default). Feathering may be
|
| 212 |
+
unnecessary or even harmful in latent space: the VAE decode and
|
| 213 |
+
overlapping receptive fields already smooth a hard cell edge, while
|
| 214 |
+
fractional cells denoise as frozen/live blends that may sit
|
| 215 |
+
off-manifold and mush the seam. Hard 0/1 cells keep each cell in one
|
| 216 |
+
regime and let attention reconcile the boundary. mask_feather 0 now
|
| 217 |
+
snaps boundary cells to 0/1; the ratifying same-seed A/B (0 vs 32 vs
|
| 218 |
+
64) is on the alpha checklist. The pixel-space composite feather is a
|
| 219 |
+
separate mechanism and still needed there.
|
| 220 |
+
- Camera-compensated jerk. Subtract the dominant global motion component
|
| 221 |
+
per token before thresholding, to stop camera pans from inflating the
|
| 222 |
+
dilated spans. Progression: global jerk, camera-compensated jerk,
|
| 223 |
+
spatially localized overload.
|
| 224 |
+
- Beat-anchored recovery. Dilated time sometimes gets filled with extra
|
| 225 |
+
action beats instead of slower ones (two backflips become four). That
|
| 226 |
+
failure is itself evidence about the model's action-density prior and
|
| 227 |
+
deserves measurement, plus a recovery mode that pins beat count.
|
| 228 |
+
|
| 229 |
+
## Status
|
| 230 |
+
|
| 231 |
+
The targeting/editor/segment nodes that make the adaptive and targeted
|
| 232 |
+
variants usable are alpha (2026-08-09) and under manual test; see
|
| 233 |
+
TESTING_ALPHA.md. The classic pipeline nodes are unchanged and
|
| 234 |
+
regression-tested against the previous release.
|
ComfyUI-MiniMaxH3-Context-Loop/ComfyUI-MAINodes/ROADMAP.md
ADDED
|
@@ -0,0 +1,355 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Roadmap: methods we are investigating
|
| 2 |
+
|
| 3 |
+
> Status ledger for what has shipped vs what is open:
|
| 4 |
+
> [`docs/roadmaps/ATOS_ALPHA_STATUS.md`](docs/roadmaps/ATOS_ALPHA_STATUS.md).
|
| 5 |
+
> The persistent-world design and its status live there too.
|
| 6 |
+
|
| 7 |
+
Companion to `RESEARCH_NOTES_ATOS.md` (the claim and the evidence plan),
|
| 8 |
+
`TUNING.md` (symptom to dial, including the rejections) and
|
| 9 |
+
`TESTING_ALPHA.md` (what the alpha nodes still need from human hands).
|
| 10 |
+
|
| 11 |
+
Everything below is a direction, not a promise. Items are grouped by the
|
| 12 |
+
problem they attack, and each says what would count as success and what the
|
| 13 |
+
cheapest first measurement is. Where we have already measured something that
|
| 14 |
+
contradicts an earlier claim of ours, it is written down as such.
|
| 15 |
+
|
| 16 |
+
House rules that constrain every item here: motion is judged in playback
|
| 17 |
+
because stills lie in both directions; a measurement earns its compute only
|
| 18 |
+
if it beats the cheap baseline it replaces; and anything shipped as a default
|
| 19 |
+
has to survive a same-seed A/B.
|
| 20 |
+
|
| 21 |
+
---
|
| 22 |
+
|
| 23 |
+
## 1. The oracle: from allocator to meter
|
| 24 |
+
|
| 25 |
+
**The problem, measured and published in the research notes:** the jerk
|
| 26 |
+
oracle ranks but cannot abstain. Because the threshold is a quantile of the
|
| 27 |
+
profile, a clip with exactly zero trajectory jerk still received 3.06x
|
| 28 |
+
dilation. And in the value domain the third difference is nearly the same
|
| 29 |
+
signal as the first (corr 0.960 and 0.977 on real clips), so the score is
|
| 30 |
+
contaminated by motion energy. This is the single most important open
|
| 31 |
+
problem in the method.
|
| 32 |
+
|
| 33 |
+
Directions, roughly in order of expected value per hour:
|
| 34 |
+
|
| 35 |
+
- **Paired-speed reachability.** Generate the same choreography at two
|
| 36 |
+
content speeds and diff the latents. Tokens whose state actually changes
|
| 37 |
+
between the two are the tokens that motion speed reaches; tokens that look
|
| 38 |
+
hot in both are hot for some other reason, which is exactly the
|
| 39 |
+
textured-object-passing-a-location confound. Borrowed from the practice of
|
| 40 |
+
probing a system with paired semantic perturbations rather than noise.
|
| 41 |
+
Success: a per-token relevance score that separates the two synthetic
|
| 42 |
+
clips (zero jerk versus lurching) where the quantile does not. Cost: two
|
| 43 |
+
renders plus offline analysis of saved latents.
|
| 44 |
+
- **Demand over capacity as an absolute unit.** A token spans four frames;
|
| 45 |
+
the failure is that those four frames need more distinct poses than one
|
| 46 |
+
token can hold. So the natural absolute quantity is required pose change
|
| 47 |
+
divided by representable pose change, with the hold count falling out as
|
| 48 |
+
that ratio rounded up. Calibrated once against the speed at which smear
|
| 49 |
+
actually appears, instead of fixed by a quantile forever. Success: the
|
| 50 |
+
meter says "dilate nothing" on a smooth clip and "dilate everything" on a
|
| 51 |
+
uniformly violent one, both of which the quantile gets wrong today.
|
| 52 |
+
- **Trajectory-domain measurement.** Showing promise on real clips
|
| 53 |
+
(top-decile share 0.29 versus 0.22 for velocity, with only moderate
|
| 54 |
+
correlation between them). Track a region centroid and differentiate the
|
| 55 |
+
trajectory rather than the latent values. Now selectable on the oracle as
|
| 56 |
+
`profile_mode`. **Caveat measured while shipping it:** on a smooth
|
| 57 |
+
synthetic blob the centroid path is nearly noise-free, so its peak-to-mean
|
| 58 |
+
contrast is not comparable to the value domain's there (1.21x versus 1.88x
|
| 59 |
+
separation between a constant-velocity and a lurching toy). Treat it as an
|
| 60 |
+
ablation option, not as an established improvement, until it is measured on
|
| 61 |
+
real textured content.
|
| 62 |
+
- **Knee-finding instead of a fixed quantile.** Run the cheap early-x0 probe
|
| 63 |
+
at several dilation factors and take the knee of the curve per clip.
|
| 64 |
+
Unlike a quantile, a knee can land at 1.0x, which is the abstention we
|
| 65 |
+
currently cannot express.
|
| 66 |
+
- **Absolute calibration by temporal capacity profiling.** Feed the VAE
|
| 67 |
+
synthetic controlled-speed content (moving gratings, rotating props, speed
|
| 68 |
+
sweeps) and measure reconstruction fidelity against speed to get the
|
| 69 |
+
model's temporal transfer curve. Useful beyond this repo: it would be a
|
| 70 |
+
reusable way to compare base models on temporal capacity.
|
| 71 |
+
|
| 72 |
+
**Ablation owed regardless:** delta order 1 versus 2 versus 3, phase
|
| 73 |
+
normalization on and off, against external baselines (optical flow magnitude
|
| 74 |
+
and flow acceleration on decoded frames). If the third difference wins or
|
| 75 |
+
ties while being free, that is the result. If it loses, the honest move is to
|
| 76 |
+
say so and rename the thing.
|
| 77 |
+
|
| 78 |
+
## 2. Detection quality
|
| 79 |
+
|
| 80 |
+
- **Camera-compensated jerk.** Subtract the dominant global motion component
|
| 81 |
+
per token before thresholding, to stop pans inflating the dilated spans.
|
| 82 |
+
Progression: global jerk, then camera-compensated, then spatially
|
| 83 |
+
localized overload.
|
| 84 |
+
- **Stabilize, repair, unstabilize.** The alternative to teaching the oracle
|
| 85 |
+
to ignore pans: remove the pan first with a coarse global warp, run the
|
| 86 |
+
pipeline where subject motion is the only motion, then invert the warp.
|
| 87 |
+
A classic VFX move that turns the hardest confound into preprocessing.
|
| 88 |
+
Worth comparing directly against camera compensation.
|
| 89 |
+
- **Cross-modal detection.** The model co-generates audio, so audio onset
|
| 90 |
+
density is a second, independent account of where the action beats are.
|
| 91 |
+
Where video-hot and audio-quiet disagree is a good candidate signature for
|
| 92 |
+
camera-pan inflation. This also gives the oracle a **free baseline it must
|
| 93 |
+
beat**: a latent detector that cannot outperform onset detection on
|
| 94 |
+
co-generated audio has not earned its complexity.
|
| 95 |
+
- **Mirror and reflection double counting.** Published correction in TUNING:
|
| 96 |
+
a synchronized reflection leaves the hold map bit-identical, and the real
|
| 97 |
+
amplifier is `bridge` welding a temporally separated reflection to the
|
| 98 |
+
body's burst. A real detector needs approximate reflection symmetry of the
|
| 99 |
+
spatial jerk map, not temporal correlation, which we probed and found
|
| 100 |
+
cannot work (one body's own limbs correlate at ~1.0, so the confuser floor
|
| 101 |
+
sits above the signal). Untested on real mirror footage.
|
| 102 |
+
|
| 103 |
+
## 3. Allocation and cost
|
| 104 |
+
|
| 105 |
+
- **Fixed-budget bidirectional retiming.** Today the method BUYS temporal
|
| 106 |
+
capacity for bursts. The general object is a monotone retiming function
|
| 107 |
+
whose two signs are dilation and decimation: compress the quiet spans and
|
| 108 |
+
spend exactly what you saved on the bursts, for a constant token budget.
|
| 109 |
+
"Repair at zero marginal cost" is a much stronger claim than "3.5x for
|
| 110 |
+
quality." Open risk: decimated frames have no generated source, so
|
| 111 |
+
recovery there is nearest-held-frame and may judder in quiet spans.
|
| 112 |
+
Playback decides.
|
| 113 |
+
- **Honest cost accounting, already shipped.** Per-step time is superlinear
|
| 114 |
+
in token count (measured exponent 1.75 here, ~1.64 in a field report,
|
| 115 |
+
`COST_EXP 1.7` in the code). Reports now state time multipliers rather than
|
| 116 |
+
frame ratios. The practical consequence is counter-intuitive and worth
|
| 117 |
+
repeating: many small windows can be cheaper than one large one, so
|
| 118 |
+
pass-count is the wrong unit for comparing any of these approaches.
|
| 119 |
+
- **Rolling-window regeneration, for cards that OOM on the dilated pass.**
|
| 120 |
+
What runs people out of memory is not the clip, it is the dilated span: a
|
| 121 |
+
5 s action clip regenerates as 11 to 13 s of frame data, so peak VRAM
|
| 122 |
+
tracks the burst budget rather than the runtime. Field reports put the
|
| 123 |
+
wall around 1 MP past 10 s on a 3090, 128 GB of system RAM
|
| 124 |
+
notwithstanding. Sliding a fixed-size window across world time,
|
| 125 |
+
regenerating each window and splicing, turns peak memory into a knob
|
| 126 |
+
instead of a consequence: the user sets the window, and clip length stops
|
| 127 |
+
mattering. `H3 Segment Crop` and `H3 Segment Splice` already do exactly
|
| 128 |
+
this for one window, with context handles and video plus sample-accurate
|
| 129 |
+
audio crossfades, so what is missing is orchestration and a seam policy
|
| 130 |
+
across N windows, not new machinery. A community continuation graph
|
| 131 |
+
(`seamless_extension_derope.json`, 2026-08-12) already wraps the de-rope
|
| 132 |
+
chain in a segment loop for a different purpose and is worth reading
|
| 133 |
+
before designing ours.
|
| 134 |
+
It is also not the slow option, though by less than the bare exponent
|
| 135 |
+
suggests. Naive arithmetic says N windows cost N^-0.75 of the single-window
|
| 136 |
+
time, four windows for about a third. Fitting 266 corpus runs and then
|
| 137 |
+
charging for duplicated handle frames and grid padding gives **0.69x at 300
|
| 138 |
+
dilated frames, 0.45x at 600 and 0.37x at 1000**: real, and roughly half
|
| 139 |
+
the advertised saving. Handle duplication, not per-window overhead, is what
|
| 140 |
+
eats it, and it turns back into a loss somewhere around 8 to 12 windows.
|
| 141 |
+
The same fit found the step-independent cost is not constant either, it
|
| 142 |
+
scales at t^1.66 against the step's t^1.70, so it splits along with
|
| 143 |
+
everything else.
|
| 144 |
+
Two things are already true that this item assumed were missing. Chaining
|
| 145 |
+
`H3 Segment Splice` so each output becomes the next one's baseline
|
| 146 |
+
reconstructs the world to 6e-8 today, with no new code, so N-window
|
| 147 |
+
reassembly is not the gap. And core's graph-expansion API means owning the
|
| 148 |
+
iteration does not require a fat node that swallows the sampler.
|
| 149 |
+
The real difficulty is the seam, and specifically the HOT cut, a window
|
| 150 |
+
boundary landing inside a burst, which is exactly the case the feature is
|
| 151 |
+
requested for. Handle frames are forced to hold 1, so a crossfade there
|
| 152 |
+
blends repaired against un-repaired content at peak motion, and the audio
|
| 153 |
+
problem is worse: two independent foley takes of one impact, which no
|
| 154 |
+
crossfade width fixes. Cold cuts, where the boundary sits at hold 1, are
|
| 155 |
+
just the shipped single-window path N times and carry no new risk.
|
| 156 |
+
Success: peak VRAM flat in clip length, seams invisible in playback. Note
|
| 157 |
+
we cannot currently measure the first half of that, `peak_vram_mb` in our
|
| 158 |
+
own telemetry is flat at 55-62 GB from 5 to 294 frames. Cheapest first
|
| 159 |
+
measurement: run the shipped crop/splice pair over two adjacent windows of
|
| 160 |
+
one clip and look at the join. That needs no new code at all.
|
| 161 |
+
Worth saying plainly: two cheaper answers to "I OOM" already ship, lowering
|
| 162 |
+
`d_max` or raising `q`, and `H3 Segment Crop` as it stands. Rolling windows
|
| 163 |
+
only win for someone insisting on full `d_max` on a burst larger than their
|
| 164 |
+
card, and nobody has produced that clip yet.
|
| 165 |
+
- **Extension: make a long clip from short renders (shipped alpha 2026-08-23).**
|
| 166 |
+
The other long-video job: not one long world chunked, but a chain of
|
| 167 |
+
short worlds with a carried handle. `examples/motion_pipeline_extend_api.json`
|
| 168 |
+
and `ALPHA.md`. What the first cells taught, in order: protecting the
|
| 169 |
+
handle from RETIMING is not enough, pass 2 still re-denoises it (20/255
|
| 170 |
+
drift) until it is also frozen (3.3); the REPRESENTATION beats the
|
| 171 |
+
prompt - the tail written into the next segment's own latent under a
|
| 172 |
+
per-token mask makes the join an ordinary frame transition (0.86x to
|
| 173 |
+
1.1x vs 5x to 6.5x for the image guide) and keeps the camera's velocity
|
| 174 |
+
continuous, while an explicit camera-transition sentence changed nothing;
|
| 175 |
+
each VAE round trip darkens the frozen prefix ~2.4% (per-channel gains
|
| 176 |
+
fitted on the hidden overlap fix it, an audio rms gain measured
|
| 177 |
+
negative); and a burst that runs into the cut comes back fast unless the
|
| 178 |
+
last token group of a non-final segment is protected (1.55x -> 1.06x).
|
| 179 |
+
Open: the drift curve over many segments, 192/90 vs 141/39, inject on
|
| 180 |
+
continuation segments, dialogue across a join, the audio ambience bed
|
| 181 |
+
(wants a per-tick audio mask).
|
| 182 |
+
- **A ceiling on dilated frames** (field request, 2026-08-12). A max-length
|
| 183 |
+
option that drops lower-priority frames to keep people off the OOM rail
|
| 184 |
+
while the highest-priority ones still reach pass 2. In our terms a hard
|
| 185 |
+
cap on the dilated budget, filled greedily from the top of the jerk
|
| 186 |
+
profile, so the pass fits the card and the hottest spans keep their holds.
|
| 187 |
+
Small change at the `_compile_hold_map` seam. The blocker sits upstream of
|
| 188 |
+
the cap: at today's false-positive rate a budget can be spent entirely on
|
| 189 |
+
a camera pan, so this wants the detector work in section 2 first, or it
|
| 190 |
+
will confidently protect the wrong frames.
|
| 191 |
+
- **Spatial crop regeneration.** Time targeting and segment crop reduce
|
| 192 |
+
FLOPs; spatial masks do not, they only blend. A spatial crop analog of
|
| 193 |
+
segment crop would make "foveated" literal. Risks are context loss and
|
| 194 |
+
resolution mismatch at the seam.
|
| 195 |
+
|
| 196 |
+
## 4. Beat duplication and anchoring
|
| 197 |
+
|
| 198 |
+
The model fills dilated time with extra beats instead of slowing the existing
|
| 199 |
+
ones (two backflips become four). This is the most visible remaining artifact
|
| 200 |
+
and it has a clear line of attack.
|
| 201 |
+
|
| 202 |
+
- **Sparse pinned time.** We already have the choreography: the baseline.
|
| 203 |
+
Sampling sparse pose anchors from it and pinning them at their retimed
|
| 204 |
+
positions gives the regeneration a ladder to climb instead of room to
|
| 205 |
+
improvise. Success: beat count stops scaling with dilation factor.
|
| 206 |
+
- **Dependency: resolved upstream (checked against core 2026-08-20).**
|
| 207 |
+
When this item was written, stock ComfyUI restricted H3 keyframe anchors
|
| 208 |
+
to first and last, and two community packs rewrote the temporal column
|
| 209 |
+
after packing to get around it. Upstream
|
| 210 |
+
[PR #15439](https://github.com/Comfy-Org/ComfyUI/pull/15439) (merged
|
| 211 |
+
commit `e01fb4c5`) made arbitrary guide positions native:
|
| 212 |
+
`MiniMaxH3AddGuide` takes a still or a clip at any frame index
|
| 213 |
+
(negative counts from the end), a clip guide snaps DOWN the 17k+5 grid
|
| 214 |
+
(5, 22, 39, ...; fewer than five frames becomes a single-frame guide),
|
| 215 |
+
and an audio guide rides along. The two bugs we had recorded here (a
|
| 216 |
+
keyframe mis-timed by the reference advance; the refs branch overwriting
|
| 217 |
+
the keyframe `cond_video_latents` list) are both fixed: references now
|
| 218 |
+
APPEND to the keyframe list in `model_base.py`, and `PackedLayout` places
|
| 219 |
+
each keyframe at its resolved frame index. On v0.33.1 and earlier the
|
| 220 |
+
slide still happens, so a graph that combines references with anchors
|
| 221 |
+
should probe the running core rather than trust a version string. Since
|
| 222 |
+
2026-08-18, [PR #15375](https://github.com/Comfy-Org/ComfyUI/pull/15375)
|
| 223 |
+
also offers per-token video and audio latent masks (snapped to the 2x2
|
| 224 |
+
latent patch grid and to whole audio latent frames, rounded to binary by
|
| 225 |
+
the generic mask path), which is the better tool for PINNING existing
|
| 226 |
+
content; guides remain the tool for creative anchors.
|
| 227 |
+
- **Audio onsets as beat anchors.** Align the retimed audio's impacts to the
|
| 228 |
+
baseline's impact times and let video recovery inherit those anchor points.
|
| 229 |
+
Attacks the same problem from the other modality.
|
| 230 |
+
|
| 231 |
+
## 5. Byproducts worth shipping
|
| 232 |
+
|
| 233 |
+
- **The slow-motion insert reel.** Exact recovery discards most of what the
|
| 234 |
+
expensive pass generated, but those are genuine intermediate poses at up to
|
| 235 |
+
4x temporal density, with pitch-preserved slowed audio available from the
|
| 236 |
+
same machinery. Exporting each repaired burst as a slow-motion insert is
|
| 237 |
+
nearly free and is production-useful on its own.
|
| 238 |
+
- **Quantization agreement as an artifact pre-filter.** Running the same seed
|
| 239 |
+
through two quantizations of the same weights is a controlled perturbation:
|
| 240 |
+
content that survives it is something the model committed to, content that
|
| 241 |
+
moves is underdetermined. If that holds, it is an automatic way to
|
| 242 |
+
pre-sort renders before a human looks, which is the real bottleneck in
|
| 243 |
+
any comparison-heavy workflow. Speculative and cheap to test.
|
| 244 |
+
|
| 245 |
+
## 6. Validation we owe
|
| 246 |
+
|
| 247 |
+
- **VAE-only roundtrip probe.** Take real fast-motion footage, encode and
|
| 248 |
+
decode at native speed versus slowed variants, and measure reconstruction
|
| 249 |
+
error against local motion magnitude. This tests "temporal compression is
|
| 250 |
+
the bottleneck" with no sampler and no DiT in the loop. Until it runs, the
|
| 251 |
+
causality language in the README stays soft.
|
| 252 |
+
- **Cross-model transfer.** The largest external-validity risk is that
|
| 253 |
+
everything here is fitted to one model's packing geometry. Porting the
|
| 254 |
+
oracle, the smear and the recovery to a second temporally compressed video
|
| 255 |
+
model is the experiment that upgrades this from a trick to a method, even
|
| 256 |
+
as a partial result.
|
| 257 |
+
- **Blinded playback A/B.** Repair score and preservation score reported
|
| 258 |
+
separately, because a method that removes the smear by removing the motion
|
| 259 |
+
is not a fix.
|
| 260 |
+
|
| 261 |
+
## 7. An adjacent approach we are evaluating, not pursuing yet
|
| 262 |
+
|
| 263 |
+
Rather than dilating time so the compressed representation can hold the
|
| 264 |
+
motion, one could refuse the compression: generate every frame as its own
|
| 265 |
+
standalone image latent by sliding the model's native five-slot stencil
|
| 266 |
+
across world time and anchoring each pass on the previous pass's frames.
|
| 267 |
+
Four passes shifted by one frame cover 17 contiguous frames at full rate.
|
| 268 |
+
|
| 269 |
+
We think this is a genuinely interesting sibling to ATOS rather than a rival,
|
| 270 |
+
and the two would compose (use the cheap detector to decide where the
|
| 271 |
+
expensive representation is worth it). Two things to know before anyone
|
| 272 |
+
sinks time into it: priced in passes it sounds absurd, but priced in the
|
| 273 |
+
measured superlinear cost model it lands much closer to adaptive dilation
|
| 274 |
+
than the pass count suggests; and its likely failure mode is per-frame decode
|
| 275 |
+
incoherence, since decoding each frame independently discards the temporal
|
| 276 |
+
smoothing that overlapping receptive fields normally provide.
|
| 277 |
+
|
| 278 |
+
## 8. Explicitly not doing, with reasons
|
| 279 |
+
|
| 280 |
+
Kept here so nobody re-derives them:
|
| 281 |
+
|
| 282 |
+
- **RoPE time warping.** Warping per-token time spans was implemented and
|
| 283 |
+
rejected: it produced boundary stutter, and the root cause was traced to
|
| 284 |
+
chunked VAE decode on rate-warped content. Content-level slowdown on a
|
| 285 |
+
uniform model clock is the approach that survived.
|
| 286 |
+
|
| 287 |
+
**Reopened as a narrower question, 2026-08-13.** Two facts sharpen it.
|
| 288 |
+
First, verified by executing `comfy/ldm/minimax/model.py:_video_t_grid`:
|
| 289 |
+
H3's MM-RoPE time coordinate is already PHYSICAL, 5/3 RoPE units per
|
| 290 |
+
world frame (token spans 5/3 x (1, 4, 4, 4, 4)), so the positional
|
| 291 |
+
clock is a frame clock, not a token index. That means content dilation
|
| 292 |
+
currently misreports time to the model: a dilated pass gets RoPE time
|
| 293 |
+
proportional to dilated frames, and the model is generating what it
|
| 294 |
+
believes is a longer clip rather than slow motion. Second, the original
|
| 295 |
+
rejection was rooted in the decode path, not in RoPE itself, so it does
|
| 296 |
+
not settle this. The narrow experiment now on the table: keep content
|
| 297 |
+
dilation exactly as shipped, divide the RoPE t-grid by the local hold
|
| 298 |
+
density so the time axis reports TRUE world time, decode normally
|
| 299 |
+
(nothing rate-warped reaches the decoder), and A/B against the shipped
|
| 300 |
+
pipeline for beat duplication, which is the failure this would most
|
| 301 |
+
plausibly dissolve. Proposed, not run.
|
| 302 |
+
- **Automatic spatial masks.** Rejected in playback. Automatic
|
| 303 |
+
oracle-heat-driven compositing degraded other artifacts even where it
|
| 304 |
+
fixed background timing. Hand-authored masks are a different question and
|
| 305 |
+
are still open.
|
| 306 |
+
- **Post-hoc frame interpolation.** It is a baseline to compare against, not
|
| 307 |
+
a component. Interpolating a smeared frame cannot recover a pose the
|
| 308 |
+
generator never created.
|
| 309 |
+
- **Feathering latent freeze masks.** Now defaulting to hard cells, on the
|
| 310 |
+
argument that fractional edge cells denoise as half-frozen blends while the
|
| 311 |
+
decode smooths a hard cell edge for free. Ratification A/B is on the alpha
|
| 312 |
+
checklist and the default reverts if the ramp wins.
|
| 313 |
+
|
| 314 |
+
---
|
| 315 |
+
|
| 316 |
+
## How much work each of these is
|
| 317 |
+
|
| 318 |
+
Rough sizing against this codebase, for anyone deciding where to jump in.
|
| 319 |
+
The architecture helps more than it looks: `_jerk_profile` is a dozen lines
|
| 320 |
+
of numpy and `_compile_hold_map` is a shared seam, so **any new detector is a
|
| 321 |
+
drop-in** and most detection work is hours rather than days. The expensive
|
| 322 |
+
items are the ones that change the time REPRESENTATION or reach into ComfyUI
|
| 323 |
+
internals.
|
| 324 |
+
|
| 325 |
+
**Already shipped on this branch (alpha, defaults unchanged):** the
|
| 326 |
+
abstention gate (`abstain_below`) and alternative detectors (`profile_mode`:
|
| 327 |
+
value |d3|, value |d1| baseline, trajectory centroid) on H3 Jerk Oracle. The
|
| 328 |
+
default path is bit-identical to what shipped before.
|
| 329 |
+
|
| 330 |
+
| item | size | why |
|
| 331 |
+
|---|---|---|
|
| 332 |
+
| delta-order ablation | done | now a dropdown on the oracle |
|
| 333 |
+
| absolute abstention gate | done | contrast floor, off by default |
|
| 334 |
+
| camera-compensated profile | hours | another `_jerk_profile` mode, same seam |
|
| 335 |
+
| audio-onset profile as the cheap baseline | ~half day | solved DSP, emits the same profile shape |
|
| 336 |
+
| slow-motion insert export | ~half day | the recover node already selects frames; emit the discarded ones |
|
| 337 |
+
| paired-speed reachability | ~1 day | two renders plus offline latent diff, no new node |
|
| 338 |
+
| knee-finding over dilation factors | ~1 day | orchestration around the existing probe schedule |
|
| 339 |
+
| quantization agreement pre-filter | ~1 day | two renders per clip plus a divergence map |
|
| 340 |
+
| mirror symmetry detector | ~1 day | needs real mirror footage; synthetics pass trivially |
|
| 341 |
+
| spatial crop regeneration | ~2 days | mirrors the existing time crop/splice pair; seam risk |
|
| 342 |
+
| fixed-budget bidirectional retiming | ~2 days | holds below 1 change the hold-map representation AND recovery semantics |
|
| 343 |
+
| sparse pinned time / anchor ladder | ~2-3 days | needs the packed-layout temporal patch or upstream #15439, plus maintenance risk against upstream churn |
|
| 344 |
+
| stabilize, repair, unstabilize | ~2-3 days | a whole preprocessing stage with warp and inverse warp |
|
| 345 |
+
| VAE roundtrip probe | ~1 day | a script, not a node |
|
| 346 |
+
| temporal capacity profiling | ~2-3 days | synthetic content generation plus a sweep |
|
| 347 |
+
| cross-model transfer | large | a port, not a feature |
|
| 348 |
+
|
| 349 |
+
## What would actually help
|
| 350 |
+
|
| 351 |
+
If you are running this pack, the most useful things you can send back are
|
| 352 |
+
same-seed before and after pairs on content that breaks it, a note on which
|
| 353 |
+
dial you reached for, and your seconds-per-step so cost estimates can be
|
| 354 |
+
calibrated across hardware. Failure cases on content unlike ours are worth
|
| 355 |
+
more to us than successes on content like ours.
|
ComfyUI-MiniMaxH3-Context-Loop/ComfyUI-MAINodes/TESTING_ALPHA.md
ADDED
|
@@ -0,0 +1,152 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Alpha test checklist (2026-08-09 nodes)
|
| 2 |
+
|
| 3 |
+
Covers H3 Manual Hold Map, H3 Motion Editor (and its widget), H3 Segment
|
| 4 |
+
Crop, H3 Segment Splice, and the manual mask paths on H3 Motion
|
| 5 |
+
Composite / H3 V2V Init. The Python compile paths have been exercised via
|
| 6 |
+
the API; the interactive widget and the drag-in workflows need human
|
| 7 |
+
passes. Findings welcome as issues; symptom/dial pairs belong in
|
| 8 |
+
TUNING.md.
|
| 9 |
+
|
| 10 |
+
Setup: any ComfyUI with the H3 models working (0.30 and 0.31 front ends
|
| 11 |
+
both need coverage). Keep the browser dev console open; any red line
|
| 12 |
+
during widget use is a finding. All three graphs wire clip length from
|
| 13 |
+
one source node; drop to a short legal duration (56 or 73 frames at
|
| 14 |
+
24fps) while iterating. Reminder when reviewing saved clips elsewhere:
|
| 15 |
+
H3 audio is 32 kHz and some players decode that as silence; resample to
|
| 16 |
+
48 kHz AAC if you share files.
|
| 17 |
+
|
| 18 |
+
## 1. Targeted graph (motion_pipeline_targeted.json)
|
| 19 |
+
|
| 20 |
+
- [ ] First queue with the shipped range saves baseline / oraclemap /
|
| 21 |
+
dilated / final, no errors, final plays clean
|
| 22 |
+
- [ ] The Manual Hold Map report (the price tag) is actually visible
|
| 23 |
+
somewhere; if you had to wire your own text display, note it
|
| 24 |
+
- [ ] Two-pass rhythm: watch oraclemap, retype ranges to the real burst,
|
| 25 |
+
requeue; baseline sampler must NOT re-run; dilated length and
|
| 26 |
+
report shrink
|
| 27 |
+
- [ ] Syntax: multi-range, seconds ("0.5s-1.1s"), per-range ":hold",
|
| 28 |
+
mixed; segments output echoes token-snapped (wider) spans
|
| 29 |
+
- [ ] Error UX: reversed range and out-of-bounds range produce readable
|
| 30 |
+
messages, not a stack wall
|
| 31 |
+
- [ ] Gate mode semantics: with the oracle wired, your ranges choose
|
| 32 |
+
WHERE and the oracle chooses hold strength (typed :hold values are
|
| 33 |
+
ignored); unwire the oracle and your holds apply directly
|
| 34 |
+
- [ ] s_per_step set to a measured value gives a plausible minutes
|
| 35 |
+
estimate
|
| 36 |
+
- [ ] Spatial branch: unmute, paint a background lasso mask, queue;
|
| 37 |
+
with invert_mask on, the painted region returns to baseline
|
| 38 |
+
timing; the seam (feather 64 smoothstep on a real edge) is
|
| 39 |
+
invisible in playback
|
| 40 |
+
- [ ] Latent freeze hard-cell ratification: hard 0/1 cells are now the
|
| 41 |
+
DEFAULT (mask_feather 0 snaps boundary cells instead of leaving
|
| 42 |
+
them fractional). Route the same mask through H3 V2V Init's
|
| 43 |
+
freeze path and run same-seed mask_feather 0 vs 32 vs 64. The
|
| 44 |
+
bet: hard cells tie or win, because half-frozen blend cells were
|
| 45 |
+
the seam mush and the decode smooths a hard edge anyway. If 32/64
|
| 46 |
+
clearly wins in playback, revert the default and say so in
|
| 47 |
+
TUNING
|
| 48 |
+
|
| 49 |
+
## 2. Editor graph (motion_pipeline_editor.json)
|
| 50 |
+
|
| 51 |
+
Cold load
|
| 52 |
+
- [ ] Node renders with toolbar, timeline, paint hint, price line; the
|
| 53 |
+
raw editor_state textarea is hidden; console clean
|
| 54 |
+
- [ ] Save and reload the workflow before any queue; node intact
|
| 55 |
+
|
| 56 |
+
First queue (no blocks)
|
| 57 |
+
- [ ] Behaves like the classic adaptive pipeline (oracle passes through,
|
| 58 |
+
mask is all ones)
|
| 59 |
+
- [ ] Widget populates: filmstrip, green jerk profile, "last run" report
|
| 60 |
+
|
| 61 |
+
Timeline
|
| 62 |
+
- [ ] Scrub on filmstrip/ruler; playhead and frame label track
|
| 63 |
+
- [ ] Drag on the block lane creates a block snapped to the token grid;
|
| 64 |
+
"+ block" works at the playhead; snap toggle off allows free edges
|
| 65 |
+
- [ ] Move and resize via bracket handles (are the 4 px hit zones big
|
| 66 |
+
enough?); multiple blocks get distinct colors; labels show
|
| 67 |
+
"oracle"/"hN" and a star once painted
|
| 68 |
+
- [ ] Delete button and Delete key; drags under 2 frames create nothing
|
| 69 |
+
- [ ] Undo button and ctrl+z; ctrl+z inside the editor must NOT undo the
|
| 70 |
+
ComfyUI graph, and editor keys must not trigger canvas hotkeys;
|
| 71 |
+
clicking outside restores normal hotkeys
|
| 72 |
+
|
| 73 |
+
Painting
|
| 74 |
+
- [ ] Unpainted selected block shows the red tint and the "paint to
|
| 75 |
+
narrow it" notice
|
| 76 |
+
- [ ] Brush paints with the circle cursor; erase removes; size slider
|
| 77 |
+
works at both extremes; "clear frame" wipes only the current frame
|
| 78 |
+
- [ ] Playhead outside the selected block: painting no-ops with notice
|
| 79 |
+
- [ ] "paint: whole block" strokes persist across the block's frames
|
| 80 |
+
- [ ] Onion skin ghosts previous (blue) and next (green) frame strokes;
|
| 81 |
+
toggle kills them
|
| 82 |
+
- [ ] Mouse wheel over the paint area steps frames without zooming the
|
| 83 |
+
graph canvas; arrow keys step too
|
| 84 |
+
|
| 85 |
+
Dials and automation
|
| 86 |
+
- [ ] All per-block dials respond; hold changes move the price line
|
| 87 |
+
- [ ] "A" buttons open lanes for hold/feather/strength; click adds
|
| 88 |
+
points, drag shapes, double-click deletes; values clamp to range
|
| 89 |
+
and block span; the button lights when an envelope exists
|
| 90 |
+
- [ ] Known cosmetic: pre-queue price estimates oracle blocks at hold 4,
|
| 91 |
+
so it can overshoot the post-run report
|
| 92 |
+
|
| 93 |
+
Edit and requeue (the core promise)
|
| 94 |
+
- [ ] After an edit, only the regeneration side re-runs; if the baseline
|
| 95 |
+
sampler re-runs, that is the P1 finding
|
| 96 |
+
- [ ] Painted region regenerates, unpainted stays baseline; block edges
|
| 97 |
+
ease via the fade dial with no hard pop
|
| 98 |
+
- [ ] Trap to confirm and document: adding ANY block gates the oracle,
|
| 99 |
+
so bursts outside your blocks are not de-roped
|
| 100 |
+
- [ ] invert_mask and outside_blocks=regenerated behave as labeled
|
| 101 |
+
|
| 102 |
+
Persistence
|
| 103 |
+
- [ ] Save + browser refresh mid-edit: blocks/strokes/envelopes survive;
|
| 104 |
+
the filmstrip may be blank until the next queue (temp thumbnails);
|
| 105 |
+
confirm it degrades to the hint, not broken images
|
| 106 |
+
- [ ] After a ComfyUI restart, the first queue repopulates everything
|
| 107 |
+
|
| 108 |
+
## 3. Segment graph (motion_pipeline_editor_segment.json)
|
| 109 |
+
|
| 110 |
+
- [ ] First queue with an empty editor crops to the oracle's window plus
|
| 111 |
+
handles; the crop report states the speedup ratio
|
| 112 |
+
- [ ] A tight block around the burst improves the ratio and the wall
|
| 113 |
+
time vs the plain editor graph
|
| 114 |
+
- [ ] Video seam: step frames at both window boundaries; look for tone
|
| 115 |
+
or brightness drift across the crossfade; compare feather_frames
|
| 116 |
+
0 vs 6
|
| 117 |
+
- [ ] Audio seam: headphones at both splice points; no clicks; total
|
| 118 |
+
duration matches baseline; sync holds after the splice
|
| 119 |
+
- [ ] Edge cases: block touching frame 0 or the last frame; two separate
|
| 120 |
+
bursts produce ONE window spanning both (current behavior); the
|
| 121 |
+
report should make that obvious
|
| 122 |
+
|
| 123 |
+
## Triage
|
| 124 |
+
|
| 125 |
+
- P1: baseline re-runs on edit; state loss on save/reload; crashes;
|
| 126 |
+
unreadable errors on common mistakes
|
| 127 |
+
- P2: interaction jank, price/report confusion, seams below the playback
|
| 128 |
+
bar
|
| 129 |
+
- P3: cosmetics (canvas text is not hidpi-scaled yet, known)
|
| 130 |
+
|
| 131 |
+
## 5. Oracle alpha knobs (added 2026-08-10)
|
| 132 |
+
|
| 133 |
+
- [ ] `profile_mode` default (`value |d3|`) must reproduce your previous
|
| 134 |
+
results exactly: same clip, same seed, same hold map and same segments
|
| 135 |
+
string as before this change. Verified bit-identical offline; confirm
|
| 136 |
+
once in the graph.
|
| 137 |
+
- [ ] `profile_mode` = `value |d1| (energy baseline)`: the honest cheap
|
| 138 |
+
baseline. On real clips it correlates 0.96-0.98 with |d3|, so if it
|
| 139 |
+
produces a materially similar hold map, that IS the finding and the
|
| 140 |
+
default detector is not earning its name.
|
| 141 |
+
- [ ] `profile_mode` = `trajectory centroid |d3|`: expect a different hold
|
| 142 |
+
map. Judge in playback whether it targets the burst better or worse.
|
| 143 |
+
Do not judge from the profile string alone.
|
| 144 |
+
- [ ] `abstain_below` (absolute gate, 0 = off): run it on a clip you KNOW is
|
| 145 |
+
calm. Raise until the oracle returns a flat hold map and empty
|
| 146 |
+
segments, then check it still dilates a genuinely fast clip at that
|
| 147 |
+
setting. Measured contrast ran ~1.9x higher on a lurching toy than a
|
| 148 |
+
constant-velocity one, so the useful range is likely 1.5-2.5. If a
|
| 149 |
+
single threshold cannot separate your calm and fast clips, say so:
|
| 150 |
+
that is evidence the contrast statistic is the wrong absolute unit and
|
| 151 |
+
the demand-over-capacity route in ROADMAP.md section 1 is needed
|
| 152 |
+
instead.
|
ComfyUI-MiniMaxH3-Context-Loop/ComfyUI-MAINodes/TUNING.md
ADDED
|
@@ -0,0 +1,618 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Tuning the Motion Lab pipeline
|
| 2 |
+
|
| 3 |
+
A working guide for dialing the pipeline to a specific user's content and
|
| 4 |
+
taste. Written for humans and for AI assistants doing the tuning on a
|
| 5 |
+
user's behalf. Everything here comes from same-seed comparisons on our
|
| 6 |
+
test clips; expect your content to move the numbers a little.
|
| 7 |
+
|
| 8 |
+
## First: pick the graph
|
| 9 |
+
|
| 10 |
+
The start-here index with diagrams is `examples/README.md`; the hardware
|
| 11 |
+
rows are `HARDWARE.md`. The short table:
|
| 12 |
+
|
| 13 |
+
| priority | graph | cost (vs one baseline render) |
|
| 14 |
+
|---|---|---|
|
| 15 |
+
| the normal starting point (since 2026-08-19): 12-step base pass 1, turbo 6-step pass 2, audio seeded | `examples/motion_pipeline_ref2va_audioinit.json` | ~1.3x; 192 frames at 1 MP in ~12 min on our card |
|
| 16 |
+
| best quality, base model throughout, audio dial available | `examples/motion_pipeline.json` | ~3.5x; the same clip at 25 steps both passes is ~3x the row above |
|
| 17 |
+
| a 16 to 24 GB card (12 reported working in the field, just slower) | `examples/motion_pipeline_lowvram.json` | as the first row; see `LOWVRAM.md` for the fenced measurements |
|
| 18 |
+
| prompt and seed scouting, not a final | `examples/motion_pipeline_fast_iterate.json` | ~1x or less (0.2 -> 0.4 MP, ~95 s) |
|
| 19 |
+
| user knows where the problem is (two-pass: review the oraclemap, type ranges, requeue) | `examples/motion_pipeline_targeted.json` | baseline + regen cost of YOUR spans only |
|
| 20 |
+
| GUI editing on the node (blocks, painting, automation) | `examples/motion_pipeline_editor.json` | as targeted |
|
| 21 |
+
| maximum speed for one burst in a longer clip (segment crop + splice) | `examples/motion_pipeline_editor_segment.json` | baseline + regen of window+handles only; the crop report states the ratio |
|
| 22 |
+
| cheap pass 1, de-rope at the delivery size | `examples/motion_pipeline_upscale_derope.json` | 89% of native detail in 83% of the time (0.4 -> 1.5 MP) |
|
| 23 |
+
| de-roping footage that already exists (no baseline render) | `examples/archive/motion_pipeline_v2v_source.json` [alpha] | regen only, ~2.5x one baseline-equivalent render; there is no baseline pass |
|
| 24 |
+
| the dilated pass does not fit your card (OOM, or steps balloon while weights stream) | `examples/motion_pipeline_rolling_window.json` [alpha] | peak memory scales with the biggest WINDOW, not the clip. At the budget that forces a split: ~1.4x the generated frames, and on a card at the offload cliff wall time comes back to parity because the DiT stays resident instead of streaming. Measured on a fenced 32 GB budget: 30.7 GB peak + layer streaming one-pass vs 24.6 GB resident windowed, 460 s vs 440 s |
|
| 25 |
+
| earlier recipes (turbo-in-the-old-graph, probe + expert, featherweight, i50, split LoRA) | `examples/archive/` | kept so old write-ups resolve; do not start from these |
|
| 26 |
+
|
| 27 |
+
On turbo, the rule moved on 2026-08-19 and it is worth stating exactly.
|
| 28 |
+
Pass 1 decides the choreography and wants the base model at 12 steps or
|
| 29 |
+
more. Pass 2 repairs bursts from a known init and is where the turbo LoRA
|
| 30 |
+
belongs, **provided the whole pass-2 recipe comes with it**: `gradient_estimation`
|
| 31 |
+
and `linear_quadratic` on pass 1, a 6-step `beta` schedule on pass 2, the
|
| 32 |
+
audio seeded through `H3 Audio Smear`. The turbo LoRA dropped into the
|
| 33 |
+
25-step `res_multistep` graph with nothing else changed renders jerky and
|
| 34 |
+
pixelated (measured 2026-08-22, seven arms, all scrapped). That is what the
|
| 35 |
+
old "no turbo in the pipeline" rule was actually measuring.
|
| 36 |
+
|
| 37 |
+
The working rhythm: iterate prompts and seeds on the fast-iterate graph to
|
| 38 |
+
learn what a prompt gives you globally, then run the keeper through the
|
| 39 |
+
starting-point graph. The base model on pass 2 (the 25-step graph) is
|
| 40 |
+
better quality still, at three times the wall; reach for it when a final
|
| 41 |
+
has earned the time.
|
| 42 |
+
|
| 43 |
+
## Second: ask the user what they actually care about
|
| 44 |
+
|
| 45 |
+
The dials trade four things against each other: sharpness in the bursts,
|
| 46 |
+
pose fidelity to the source, render time, and audio feel. Get a ranking
|
| 47 |
+
before touching anything. A user who says "the flip looks melted" wants a
|
| 48 |
+
different dial than one who says "it changed my character's pose."
|
| 49 |
+
|
| 50 |
+
## Symptom to dial
|
| 51 |
+
|
| 52 |
+
| user says | do this |
|
| 53 |
+
|---|---|
|
| 54 |
+
| the reference image stops influencing pass 2 (or an upscale pass) after a graph change | before touching any reference dial, put `BasicScheduler` / stock sigmas back. One field report (2026-08-23) lost all reference influence to a third-party scheduler node and got it back with `BasicScheduler`; not a same-sigma comparison, so it is a diagnostic first step, not a theorem about that scheduler. Only then: reference strength, prompt roles, `ref_image_size` |
|
| 55 |
+
| dialogue or audio sounds wrong: mangled words, tags read aloud, audio that copies the audio reference instead of borrowing its voice | check the core first: `H3 Capability Probe` prints `tokenizer_special_tokens`; before ComfyUI PR #15808 (2026-08-22) `<d>` and six other H3 tags tokenized as two tokens each, and every dialogue reading on such a core is suspect. Update, re-render, then tune steps, turbo and audio refs |
|
| 56 |
+
| motion still smears inside a burst | raise `d_max` toward 4 if lowered; lower `q` toward 0.70; check `bridge` is not 0 |
|
| 57 |
+
| brief stutter or soft frames mid-burst | `bridge: 8` (fills the dip between burst peaks) |
|
| 58 |
+
| poses drift from the source (head angle, hand position) | `bridge: 0` first; if not enough, lower `inject` toward 0.5 |
|
| 59 |
+
| output ignores the source choreography, invents moves | `inject` is too high; come down toward 0.6 |
|
| 60 |
+
| source artifacts leak into the output | `inject` is too low; go up toward 0.7 |
|
| 61 |
+
| too slow / too expensive | raise `q` toward 0.85 (tighter spans); switch to the turbo graph; then the probe graph |
|
| 62 |
+
| coarse mosaic / tiling in a turbo pass 2 when pass 1 was ALSO turbo (the seed-hunt iterate-then-finish flow) | a turbo init only tolerates a SHALLOW turbo pass 2. Judged in playback, one fight clip, fl2va, 0.7 to 1.5 MP, rank-21 at 1.0: 1 step at inject 0.20 clean and good; 3 steps at 0.50 and 6 steps at 0.50 both out of quality bounds. A later playback check on a second content class found 1 step at 0.30 already visibly griddy, so 0.20 is the measured ceiling, not a conservative pick, and 0.30 to 0.50 is progressive tiling onset. The same 3-at-0.50 recipe is clean over a bare 12-step pass 1, so the init is the variable, not the recipe. For keepers from turbo seed picks: stay at 1 step / 0.20, or hand pass 2 to the base model. Do NOT stack a second shallow tap through pixel space: each extra tap pays a VAE encode/decode roundtrip that costs more than a 0.20 solve returns (judged: 2 taps a step below 1) | split the pass with the rolling-window graph: `H3 Window Plan` + `H3 Window Collect`. Set `max_dilated_frames` below the dilated count your card last survived, read the plan report before rendering, queue one window per item. Leave `coverage` on `full clip` on upscale graphs or the calm spans stay at baseline resolution |
|
| 63 |
+
| turbo output changes appearance (adds ornament, shifts style) | lower LoRA strength 0.8 toward 0.65; or raise `base_head` in the expert schedule so more structure forms on the base model |
|
| 64 |
+
| audio feels thin | raise `reference_mix` (needs the non-probe graphs); it is happiest near 0 or 1, mid values can double misaligned impacts |
|
| 65 |
+
| audio impacts feel soft | known vocoder trait; try `reference_mix: 1` for baseline foley, or accept lean |
|
| 66 |
+
| unprompted speech or vocal sounds appear | any voice your prompt describes without verbatim words gets gap-filled: sometimes a coherent invented line, sometimes non-language vocalizing. Script the words in the dialogue format, declare the voice indistinct by design, or cut the voice mention from the prompt |
|
| 67 |
+
| dialog sounds processed | check whether the speech overlaps a burst; unheld spans pass through untouched, so only speech during bursts is affected. `reference_mix: 1` restores the original line |
|
| 68 |
+
| probe init loses choreography | raise `probe_steps` from 6 toward 10 |
|
| 69 |
+
| background details change in regenerated spans (a flag recolors, props swap) | known limitation: detailed backgrounds re-roll during dilation. Try `inject` toward 0.5 (closer init tracking); simple backgrounds barely show it. A subject-only "foveated" mode that never regenerates the background is on the roadmap |
|
| 70 |
+
| smooth or slowly panning footage gets dilated spans much wider than the action | measured cause: on smooth content the latent jerk profile is nearly flat (max/min about 1.7 where the pixel jerk spans 20+), so a quantile threshold marks a fixed fraction of tokens hot by construction and the ramp+bridge fuse them into one wide window. Camera motion itself explains only a few percent of the jerk, so camera compensation does not help. Raise `q` toward 0.85, or gate the oracle through `H3 Manual Hold Map` with the ranges where the real action is |
|
| 71 |
+
| the oracle dilates spans the user does not care about / the pass is too expensive for one burst | `H3 Manual Hold Map` in gate mode: wire the oracle's hold_map, type the ranges that matter, everything else recovers to hold 1. The report output shows the effective length before the expensive pass runs |
|
| 72 |
+
| an action repeats or doubles in regenerated spans (two backflips become four) | the model fills dilated time with extra beats instead of slowing the existing ones; its prior for action density wins over the init. Try `inject` toward 0.5 (closer init tracking), and state exact counts positively in the prompt ("performs exactly two backflips in total"). On the roadmap as beat-anchored recovery |
|
| 73 |
+
| the clip contains a mirror or a big reflection and the beats double | do NOT blame the oracle first. Synthetic probe, 2026-08-09: a reflection showing the SAME motion at the SAME time leaves the oracle's output bit-identical (its threshold is a quantile of the profile, so duplicating the motion scales every token equally and selects exactly the same ones; profile shape correlation 0.9999). What does bite is a reflection prominent at a DIFFERENT time than the body, which a camera tilt produces (mirror early, subject late): that leaves two separated hot clusters and `bridge` welds them into one long plateau, +26% dilated time in the probe, of which `bridge` alone is +18%. So on mirror content try `bridge: 0` first, then `inject` toward 0.5, then explicit counts in the prompt. Everything left over after that is the model's action-density prior, not the oracle |
|
| 74 |
+
| background elements (birds, crowds, traffic) speed up during bursts | the AUTOMATIC remedies stay rejected (oracle-mask compositing popped at the boundary, blanket freeze degraded other artifacts). The manual path is new and promising: draw the keep-baseline region yourself (`show_drift` on the heatmap shows what to lasso in blue), wire it into `H3 Motion Composite` `mask` with `invert_mask` on, feather 48-64 smoothstep, and put the seam on a real edge (horizon, rooftop) where nothing moves. Use the `H3 V2V Init` `mask` freeze instead when background and subject share lighting or contact. First internal A/B (demo clip, invented flock + pagoda both removed, static seam) awaits playback ratification |
|
| 75 |
+
| half a second of an otherwise good clip is wrong (a face, a prop, one bad beat) | do not re-render the clip. `H3 Repair Plan` + `H3 Repair Splice`: type the bad frame range, let the plan snap it outward to whole tokens, and splice the rest back as original pixels. Type any nearby shot cut into `shot_cuts`; the exit belongs ON a cut. AUDIO IS STILL A WORK IN PROGRESS here, the splice is video-exact and passes audio through untouched |
|
| 76 |
+
| the de-roped span still plays accelerated: background agents run fast inside the held region | that is the stock clock, not the oracle. `H3 DyRoPE` gives the true-duration grid to some blocks or some steps. Two settings worth trying, `physical_blocks` 30-49 and 40-49; see the dose section below, and expect to judge it in playback because the meters cannot rank the two |
|
| 77 |
+
| a de-rope with DyRoPE on shows flicker at the span boundary | flash is an early-block effect: keep the physical grid out of blocks 0-9. Any `*_blocks` range starting at 25 or above meters at the stock control's flash level. If flicker still bothers you more than jitter does, switch to `fade_physical_to_compact` at `fade_end 0.7`, which is flash-free in playback |
|
| 78 |
+
| the sound restarts, jumps level or changes room at a segment join | wire the previous segment's audio as `audio_latent` and set `H3 V2V Init`'s `audio_prefix_ticks` to the handle (a 39-frame handle is exactly 65 ticks). Add `audio_prefix_release_ticks 8` for a soft handover, and then the ASSEMBLY must let the continuation own the overlap tail or the trim discards the ramp. It carries level, timbre and beat, not phrase structure |
|
| 79 |
+
| contrast and texture wash out further with every link of a chain | `H3 Drift Control` on the pass-1 model: schedule-matched noise on the carried video prefix so the model stops treating it as impossibly clean. `prefix_frames` must equal the handle and `matched_steps` + `taper_steps` must equal its latent step count (39 f = 12 = 8 + 4). replicated across two chains and new seeds; joins hold 0.85-0.89 through link 4 (previously 0.65-class at link 2 without it).4% and handles accumulate it. `H3 Delta Color Carry` cancels the encode bias by construction. Active path exercised on a real 4-link render 2026-08-24: it fires, corrects in the anchor direction, and stays sub-visible under its clamps on mildly-drifted content; on the deepest tested link it fires on both channels at or near its clamp. A strong-drift bench (where the 2.4% per-round-trip signature actually accrues) is still owed. |
|
| 80 |
+
| a window edit renders garbage, and the damage sits where the window USED to be | the conditioning length was not moved with the crops and the hold map. All four numbers move together: see `docs/WINDOW_MODE.md` |
|
| 81 |
+
|
| 82 |
+
## VRAM expectations (measured)
|
| 83 |
+
|
| 84 |
+
An honesty note on the peak numbers in this section: they were read
|
| 85 |
+
from `nvidia-smi` on a large card, where the reading includes whatever
|
| 86 |
+
the caching allocator felt like keeping, so treat them as upper bounds
|
| 87 |
+
rather than requirements (we have since measured a smaller model
|
| 88 |
+
peaking HIGHER than a bigger one this way). The portable units are
|
| 89 |
+
latent token count for activations, and the `loaded partially` /
|
| 90 |
+
`lowvram patches` lines in the ComfyUI log for whether your card is
|
| 91 |
+
actually streaming weights. When the dilated pass is what does not
|
| 92 |
+
fit, the rolling-window graph caps the token count per pass: see the
|
| 93 |
+
table above.
|
| 94 |
+
|
| 95 |
+
Weights: the int8 DiT is 20 GB, the text encoder 15 GB (offloads after
|
| 96 |
+
encoding), the video VAE 4.9 GB. Activations at 1.0 MP run about 0.1 GB
|
| 97 |
+
per latent token: a 5 s clip (37 tokens) needs ~5 GB, and its dilated
|
| 98 |
+
regeneration pass (87 tokens) ~9 GB. Roughly half that at 0.5 MP.
|
| 99 |
+
|
| 100 |
+
- 96 GB: everything stays resident, ~11.5 s/step.
|
| 101 |
+
- 32 GB (5090 class): the DiT fits, but the dilated pass of a 5 s clip
|
| 102 |
+
lands around 30 GB total, right at the cliff, so ComfyUI starts
|
| 103 |
+
offloading and steps balloon (~20 s/it reports). What helps, in order:
|
| 104 |
+
start with 2 to 3 second clips (the dilated span is what hurts), drop
|
| 105 |
+
the regeneration to 0.5 MP, or use the probe graph so there are far
|
| 106 |
+
fewer of those expensive steps. Or switch stacks entirely: see the
|
| 107 |
+
featherweight numbers below.
|
| 108 |
+
- 24 GB: featherweight plus 0.5 MP plus short clips territory. Untested
|
| 109 |
+
by us so far.
|
| 110 |
+
|
| 111 |
+
Cost scales with the burst spans, not the clip length, so a long clip
|
| 112 |
+
with one short burst is far cheaper than these worst-case numbers.
|
| 113 |
+
|
| 114 |
+
### Featherweight stack, measured (ComfyUI 0.31+)
|
| 115 |
+
|
| 116 |
+
The smallest community-published variant of each piece: w4a8 DiT
|
| 117 |
+
(12.5 GB, Kijai/MiniMax-H3-experimental), int8_convrot video VAE
|
| 118 |
+
(3.2 GB), nvfp4 AWQ text encoder (15.7 GB, offloads after encoding).
|
| 119 |
+
Needs ComfyUI 0.31 with comfy-kitchen 0.2.28. Full de-roping pipeline
|
| 120 |
+
measured on one card (peaks are torch-allocated for the whole process):
|
| 121 |
+
|
| 122 |
+
| run | wall time | peak, TE resident | peak with TE offloaded |
|
| 123 |
+
|---|---|---|---|
|
| 124 |
+
| 3 s at 0.4 MP | 4.0 min | 34.4 GB | ~19 GB |
|
| 125 |
+
| 3 s at 0.7 MP | 6.3 min | 35.2 GB | ~20 GB |
|
| 126 |
+
| 5 s at 1.0 MP | 29 min | 43.8 GB | ~28 GB peak, ~20 GB sustained |
|
| 127 |
+
|
| 128 |
+
The last column is what consumer cards see: without `--gpu-only` the
|
| 129 |
+
text encoder leaves VRAM after encoding, which is default behavior.
|
| 130 |
+
The first column is the one **H3 Conditioning Bank** attacks: it banks
|
| 131 |
+
the encoded conditioning to disk and takes its `conditioning` input
|
| 132 |
+
lazily, so a queue item that hits the bank never executes the encode
|
| 133 |
+
node and the text encoder is not loaded at all. Wire it on any flow
|
| 134 |
+
that re-runs one prompt: rolling windows, seed hunts, extension chains.
|
| 135 |
+
Requeueing an unchanged graph does not need it (ComfyUI's node cache
|
| 136 |
+
already serves the conditioning); queueing a second workflow in
|
| 137 |
+
between, restarting, or editing anything upstream of the encode does,
|
| 138 |
+
and that is the normal shape of a session. **H3 Latent Bank** does the
|
| 139 |
+
same for a sampled pass: on the rolling-window graphs the pass-1
|
| 140 |
+
baseline is lost to the same evictions and re-renders in full before
|
| 141 |
+
window item 2 can start, so bank its LATENT (4.8 MB for a 107-frame
|
| 142 |
+
480x832 clip; the decoded frames would be 513 MB) and wire the noise
|
| 143 |
+
seed into the node's `seed` input so a new seed misses instead of
|
| 144 |
+
serving the old take. So
|
| 145 |
+
a 5 s, 1.0 MP de-rope fits a 32 GB card with a few GB of headroom (the
|
| 146 |
+
28 GB figure is a brief spike, under a minute total), and 3 s clips run
|
| 147 |
+
comfortably. Resolution barely moves the peak (0.4 to 0.7 MP added less
|
| 148 |
+
than 1 GB); frame count is what costs.
|
| 149 |
+
|
| 150 |
+
Two honest notes. Quality: the featherweight output is coherent and
|
| 151 |
+
sharp in our runs, but a given seed renders a DIFFERENT take of the
|
| 152 |
+
scene than the int8 stack; quantization changes the trajectory, so
|
| 153 |
+
compare quality, not pixels. Speed: on our card the w4a8 pipeline ran
|
| 154 |
+
somewhat slower per clip than int8 (29 vs low-20s minutes for the 5 s
|
| 155 |
+
case); on a 32 GB card that is the wrong comparison, because int8 does
|
| 156 |
+
not fit and offload-thrash costs far more.
|
| 157 |
+
|
| 158 |
+
## Chained clips and the audio clock
|
| 159 |
+
|
| 160 |
+
H3's video runs at 24 fps but its audio latent runs a 40 Hz clock, and
|
| 161 |
+
the two only agree on durations that are multiples of 1/8 s. Of the
|
| 162 |
+
legal `17k+5` frame counts, only k = 2, 5, 8, 11 ... (39, 90, 141, 192
|
| 163 |
+
frames) land exactly; every other length ships an audio stream up to
|
| 164 |
+
12.5 ms longer or shorter than its video (124 frames: audio 8.3 ms
|
| 165 |
+
long; 107: 8.3 ms short). One clip: harmless. Concatenate segments for
|
| 166 |
+
a long continuation and the error compounds into audible A/V drift --
|
| 167 |
+
we have watched a 10-minute extension from another tool visibly
|
| 168 |
+
separate by the end.
|
| 169 |
+
|
| 170 |
+
What to do about it:
|
| 171 |
+
- for anything destined for chaining, prefer the aligned lengths
|
| 172 |
+
(90 / 141 / 192 frames);
|
| 173 |
+
- when assembling segments, trim or pad each segment's audio to exactly
|
| 174 |
+
`frames / 24` seconds before concatenation, so no segment exports its
|
| 175 |
+
rounding error to the next;
|
| 176 |
+
- inside this pack you are already covered: H3 Audio Recover, H3
|
| 177 |
+
Segment Splice, and H3 Window Collect place and size audio on the
|
| 178 |
+
absolute world clock (sample-exact per segment), so seams do not
|
| 179 |
+
accumulate error.
|
| 180 |
+
|
| 181 |
+
`tests/test_audio_recover.py` is the regression guard for all of this:
|
| 182 |
+
run it bare for the synthetic sample-exact checks (no GPU), or hand it
|
| 183 |
+
a `baseline.mp4 recovered.mp4` pair from any `reference_mix: 1` render
|
| 184 |
+
and it verifies the recovered audio really is the baseline track --
|
| 185 |
+
duration within one AAC frame, zero lag, full correlation. It caught a
|
| 186 |
+
pre-fix render 64 ms short on its first outing.
|
| 187 |
+
|
| 188 |
+
### Known-good environment (this works for me)
|
| 189 |
+
|
| 190 |
+
Every number in this document was measured on this exact stack, on
|
| 191 |
+
Blackwell silicon, so it is a safe reference point for 5090-class cards:
|
| 192 |
+
|
| 193 |
+
```
|
| 194 |
+
GPU: RTX PRO 6000 Blackwell / driver 610.43.02
|
| 195 |
+
Python: 3.12.13
|
| 196 |
+
PyTorch: 2.14.0.dev20260801+cu132 (nightly; Blackwell wants the cu13x builds)
|
| 197 |
+
CUDA: 13.2, cuDNN 9.24, Triton 3.8.0
|
| 198 |
+
Attention: sageattention (no flash-attn, no xformers)
|
| 199 |
+
ComfyUI: 0.31.1 with comfy-kitchen 0.2.28
|
| 200 |
+
```
|
| 201 |
+
|
| 202 |
+
If a 5090 runs far slower than the table above, check the torch build
|
| 203 |
+
first: a stable cu126/cu128 wheel on Blackwell can cost you more than
|
| 204 |
+
any dial in this document.
|
| 205 |
+
|
| 206 |
+
### Conditioning modes (I2VA, FL2VA, Ref2VA)
|
| 207 |
+
|
| 208 |
+
The pipeline works with all of them, unmodified, at the shipped
|
| 209 |
+
defaults. Tested end to end on the featherweight stack: wire your
|
| 210 |
+
`first_frame` / `last_frame` / reference inputs into BOTH conditioning
|
| 211 |
+
nodes (the baseline's and the regeneration's) and run.
|
| 212 |
+
|
| 213 |
+
- I2VA: the first-frame pin lands at t=0 on any clock, so the
|
| 214 |
+
regeneration honors it exactly. Recovered frame 0 matched the
|
| 215 |
+
reference in our test.
|
| 216 |
+
- FL2VA: the last-frame pin names a timestamp that lands mid-clip on
|
| 217 |
+
the regeneration's dilated clock, and it does not matter: at inject
|
| 218 |
+
0.70 the injected trajectory owns the timing and the pins act as
|
| 219 |
+
pose guidance. The recovered clip hit the anchor pose at the true
|
| 220 |
+
end. At much lower inject values this could in principle fight; if
|
| 221 |
+
your FLF de-rope stalls mid-clip, raise inject back toward 0.70.
|
| 222 |
+
- Ref2VA (the ref2va checkpoint, w4a8 available): reference tokens
|
| 223 |
+
carry no timeline, so they pass through dilation untouched. Subject
|
| 224 |
+
identity held through the full de-rope in our test.
|
| 225 |
+
|
| 226 |
+
## Manual targeting (time ranges and spatial masks)
|
| 227 |
+
|
| 228 |
+
When the user can point at the problem, let them. Two tools, one per
|
| 229 |
+
axis:
|
| 230 |
+
|
| 231 |
+
- Time: `H3 Manual Hold Map` converts typed ranges (`36-60,
|
| 232 |
+
1.5s-2.4s:3`; frames or seconds, ends inclusive) into an
|
| 233 |
+
oracle-format hold map. Gate mode (oracle hold_map wired in) is the
|
| 234 |
+
default recommendation: the oracle decides hold strength, the user
|
| 235 |
+
decides where it is allowed to spend. Ranges snap outward to the
|
| 236 |
+
token grid, so trust the segments output over the typed numbers, and
|
| 237 |
+
show the user the report output: it states world length vs effective
|
| 238 |
+
regeneration length, and estimates minutes when given a measured
|
| 239 |
+
s/step. Cost scales with held spans, so this is also the speed
|
| 240 |
+
answer for long clips with one burst.
|
| 241 |
+
- Space: `H3 Motion Composite` `mask` (pixel space, full feather
|
| 242 |
+
control: size, linear/smoothstep/gaussian profile, in/out/centered
|
| 243 |
+
direction) pastes baseline timing back outside the mask after
|
| 244 |
+
recovery. `H3 V2V Init` `mask` freezes the region during generation
|
| 245 |
+
instead; its default is HARD latent cells (mask_feather 0): every
|
| 246 |
+
~16 px cell fully frozen or fully live, no fractional blend cells.
|
| 247 |
+
Rationale: half-frozen cells denoise as frozen/live mixtures that can
|
| 248 |
+
themselves mush the seam, while the decode's receptive-field overlap
|
| 249 |
+
smooths a hard cell edge for free. mask_feather above 0 restores the
|
| 250 |
+
pooled fractional ramp if a hard seam ever shows in playback (report
|
| 251 |
+
it; that reading is one A/B from ratified). Either way this path
|
| 252 |
+
cannot do fine pixel feathering; the composite is the fine-feather
|
| 253 |
+
tool.
|
| 254 |
+
|
| 255 |
+
The GUI for both axes at once is the `H3 Motion Editor` node
|
| 256 |
+
(`examples/motion_pipeline_editor.json`): timeline blocks, per-frame
|
| 257 |
+
painting, per-block dials, and automation envelopes (hold, feather,
|
| 258 |
+
strength) compiled into the same hold_map/mask wires. Queue once to
|
| 259 |
+
load the filmstrip, edit on the node, queue again; the baseline stays
|
| 260 |
+
cached. Its mask output is already feathered, so the composite runs
|
| 261 |
+
with `mask_is_soft` on and its own feather at 0. When tuning for a
|
| 262 |
+
user who cannot or will not paint, author `editor_state` JSON for
|
| 263 |
+
them; the contract is in the node docstring.
|
| 264 |
+
|
| 265 |
+
Compute honesty when a user asks where the speed comes from: TIME
|
| 266 |
+
targeting is the lever. Sampler cost scales with the dilated frame
|
| 267 |
+
count, so gating holds to the user's window helps, and `H3 Segment
|
| 268 |
+
Crop` multiplies that by dropping the un-held world from the regen
|
| 269 |
+
pass entirely (its report states the ratio). SPATIAL masks do not
|
| 270 |
+
reduce FLOPs; the DiT still processes every token and the mask only
|
| 271 |
+
controls blending. Sell the mask as quality control and the window as
|
| 272 |
+
the speedup.
|
| 273 |
+
|
| 274 |
+
A mask authored without frame-by-frame attention WILL cut off props
|
| 275 |
+
that swing with the subject (a sword mid-cartwheel leaves a static
|
| 276 |
+
blob and gets truncated at the boundary). Fixes, in order: paint
|
| 277 |
+
per-frame in the editor following the prop, enlarge the static region
|
| 278 |
+
to the prop's full arc, or feather outward. This is exactly the
|
| 279 |
+
static-vs-per-frame trade; a human on the timeline beats any static
|
| 280 |
+
region for rotating props.
|
| 281 |
+
|
| 282 |
+
Method rules for masks: static union masks cannot pop (the boundary
|
| 283 |
+
never moves); route the seam along a real image edge, not through sky
|
| 284 |
+
mid-gradient; lasso generously and let feather work; per-frame mask
|
| 285 |
+
batches reintroduce the moving-boundary pop and need harder feather
|
| 286 |
+
and a playback check. The two-pass rhythm for the targeted graph is
|
| 287 |
+
queue once, read `mainodes_targeted_oraclemap`, type ranges, requeue.
|
| 288 |
+
|
| 289 |
+
## When the expansion stops before the clip does (expand_to_end)
|
| 290 |
+
|
| 291 |
+
A hold map that expands a burst and then drops back to rate 1 for the
|
| 292 |
+
last few world frames reads as a small jump at the end of the shot: the
|
| 293 |
+
picture is in slow motion and then, a beat before the cut, it is not.
|
| 294 |
+
`H3 Time Smear` and `H3 Temporal Insert` both carry an `expand_to_end`
|
| 295 |
+
toggle, default ON, for exactly that shape.
|
| 296 |
+
|
| 297 |
+
What it does when it fires: the trailing rate-1 run is lifted to the
|
| 298 |
+
rate of the span in front of it, and the resulting length is put back
|
| 299 |
+
on the 17k+5 grid inside that same span, spending the deficit on the
|
| 300 |
+
LAST frames so rates only rise toward the end. The t2c fight window
|
| 301 |
+
`[1]*17+[2]*34+[1]*5` (56 world frames, 90 dilated) becomes
|
| 302 |
+
`[1]*17+[2]*27+[3]*12` (107 dilated); the same shape with the burst
|
| 303 |
+
later, `[1]*34+[2]*17+[1]*5`, becomes `[1]*34+[2]*10+[3]*12` (90
|
| 304 |
+
dilated). Mixed rates are the normal outcome, not a fallback: the added
|
| 305 |
+
frame count has to be a multiple of 17 and a single rate rarely lands
|
| 306 |
+
there.
|
| 307 |
+
|
| 308 |
+
What it never touches: uniform dilation, a map that already ends inside
|
| 309 |
+
an expansion, a map that is rate 1 all the way, and any rate-1 tail
|
| 310 |
+
longer than 17 world frames. That last one is the tail guard: one whole
|
| 311 |
+
group of frames or more of real time at the end of a shot is intended
|
| 312 |
+
rest, not an end jump, and adaptive oracle maps produce it constantly
|
| 313 |
+
(a 124-frame map ending in 39 quiet frames would otherwise go from 250
|
| 314 |
+
to 294 dilated frames to slow down a section nobody asked to slow
|
| 315 |
+
down). Those come back bit-identical, so existing graphs keep their
|
| 316 |
+
results. It also never
|
| 317 |
+
lowers a hold, so the user never loses expansion they asked for, and it
|
| 318 |
+
does not move where the expansion begins: only the tail is rewritten.
|
| 319 |
+
A rewrite prints one line to the console with the before and after map
|
| 320 |
+
and repeats it in the node's report, so nobody has to guess whether it
|
| 321 |
+
fired. `expand_to_end` off is the old behaviour exactly.
|
| 322 |
+
|
| 323 |
+
Cost warning worth saying out loud before turning it on: running the
|
| 324 |
+
span to the end usually buys frames. 90 to 107 dilated frames is about
|
| 325 |
+
19% more frame data and more than that in time, because per-step cost
|
| 326 |
+
is superlinear in tokens. The report output prices it.
|
| 327 |
+
|
| 328 |
+
Two traps that cost a round each while this was being built:
|
| 329 |
+
|
| 330 |
+
1. `H3 Inject Schedule`'s `preset` overrides the `inject` knob unless
|
| 331 |
+
it is set to `custom`. Typing 0.45 into `inject` and leaving the
|
| 332 |
+
preset at "balanced 0.70" renders 0.70, silently.
|
| 333 |
+
2. `H3 Time Smear` pads any segment under 39 frames. `_legal_ceil`
|
| 334 |
+
floors at 39, so a 20-frame window smeared at rate 2 comes out at 39
|
| 335 |
+
frames with the whole pad folded into the final hold, which is a
|
| 336 |
+
long freeze on the last frame. With `expand_to_end` on, that pad is
|
| 337 |
+
absorbed by the expansion instead (`[1]*5+[2]*10+[1]*5` becomes
|
| 338 |
+
`[1]*5+[2]*11+[3]*4`, still 39 frames, no freeze). Either way, do
|
| 339 |
+
not hand a sub-39-frame window to the smear and expect the length
|
| 340 |
+
you asked for.
|
| 341 |
+
|
| 342 |
+
## Source footage instead of a baseline render (alpha)
|
| 343 |
+
|
| 344 |
+
`examples/archive/motion_pipeline_v2v_source.json` swaps the first pass for a file
|
| 345 |
+
on disk: `LoadVideo -> GetVideoComponents -> H3 Video Fit -> ImageScale ->
|
| 346 |
+
VAEEncode`, and from there the graph is the standard one. The oracle reads
|
| 347 |
+
the source's own encoded latent; nothing else in the chain changes.
|
| 348 |
+
|
| 349 |
+
Two things to get right, in this order.
|
| 350 |
+
|
| 351 |
+
1. Frame count. H3's legal lengths are 5, 22, 39 ... 17k+5. An arbitrary
|
| 352 |
+
file lands between them, and the VAE does NOT complain: `encode_temporal`
|
| 353 |
+
pads the last chunk by repeating the final frame, so a 312-frame clip is
|
| 354 |
+
encoded as 323 with 11 frozen frames on the end. The oracle then reads
|
| 355 |
+
those tokens as calm and under-dilates the real ending. `H3 Video Fit`
|
| 356 |
+
trims to 311 instead and hands the count straight to the oracle's
|
| 357 |
+
`length`. Read its report; it states which end lost frames.
|
| 358 |
+
2. Canvas. The source is resized to the `ResolutionSelector` dimensions with
|
| 359 |
+
a centre crop, and the same width/height feed the conditioning. Pick the
|
| 360 |
+
aspect ratio that matches your footage or you will crop the subject out.
|
| 361 |
+
|
| 362 |
+
`max_frames` on the fit node is the cost lever. Per-step time goes as
|
| 363 |
+
tokens**1.7, so capping a long source before fitting saves more than it
|
| 364 |
+
looks: 192 frames capped to 150 fits to 141, which is 42 tokens instead of
|
| 365 |
+
57, about 1.7x faster per step.
|
| 366 |
+
|
| 367 |
+
Audio: the graph keeps YOUR track. Exact recovery restores the world clock
|
| 368 |
+
frame for frame, so the source audio still lines up with the regenerated
|
| 369 |
+
video; it is wired into `H3 Audio Recover`'s `reference` at `reference_mix
|
| 370 |
+
1.0`. Lower the mix to hear the model's own foley for the slowed
|
| 371 |
+
performance instead, or blend. If your source has no audio stream, the fit
|
| 372 |
+
node's audio output carries nothing and that link must be removed.
|
| 373 |
+
|
| 374 |
+
## Repairing a few frames instead of the clip
|
| 375 |
+
|
| 376 |
+
The repair verb is for a finished clip that is good except for a moment of
|
| 377 |
+
it. Two nodes: `H3 Repair Plan` does integer arithmetic, `H3 Repair Splice`
|
| 378 |
+
does frame selection.
|
| 379 |
+
|
| 380 |
+
The tuning is almost all in the plan's inputs.
|
| 381 |
+
|
| 382 |
+
- `bad_start` / `bad_end` are 0-BASED and inclusive. A file-numbered frame
|
| 383 |
+
46 is 45 here, and getting this wrong by one is the most common way to
|
| 384 |
+
repair the wrong token.
|
| 385 |
+
- `hold` is the dilation on the repaired span only; everything else stays
|
| 386 |
+
real time. Set `H3 Time Smear`'s `dilation` to the same number, and turn
|
| 387 |
+
its `expand_to_end` OFF: an end-expansion rewrite moves every dilated
|
| 388 |
+
coordinate the plan just computed, and the plan's report says so when it
|
| 389 |
+
detects the case.
|
| 390 |
+
- `shot_cuts` is where the quality is. Type the 0-based first frame of any
|
| 391 |
+
new shot near the span. A cut inside the span, or within `cut_reach`
|
| 392 |
+
frames of its end, becomes the exit, and the splice hands back to the
|
| 393 |
+
original AT the cut so the seam is an edit the shot already had. On the
|
| 394 |
+
validation clip, not doing this left the exit seam popping at 2.6x the
|
| 395 |
+
clip's median frame delta while the model reinvented the next shot's
|
| 396 |
+
opening frames; extending the mask through the cut did not fix it (2.3x);
|
| 397 |
+
exiting on the cut did.
|
| 398 |
+
- Wire `images` if you have the frames handy. It costs nothing and the
|
| 399 |
+
report then tells you whether your entry frame is quiet, and names a
|
| 400 |
+
quieter frame within four if it is not. The plan never moves the entry
|
| 401 |
+
for you: change `bad_start` yourself if you want it.
|
| 402 |
+
|
| 403 |
+
Read both reports rather than eyeballing the result. The plan states the
|
| 404 |
+
snapped token span, the splice bounds and the cost; the splice states the
|
| 405 |
+
entry and exit seam deltas next to the ORIGINAL clip's own delta at the same
|
| 406 |
+
frames, because at a hard cut a big number is the right answer. The measured
|
| 407 |
+
cell (90 frames, bad 45 to 47, hold 3, cut at 53) read entry 3.99/255 =
|
| 408 |
+
0.26x median, exit 138.04 against the original's own 135.57 (1.8% apart),
|
| 409 |
+
and maximum absolute difference 0 outside the splice, which the node
|
| 410 |
+
asserts rather than reports.
|
| 411 |
+
|
| 412 |
+
**Audio is still a work in progress for repair.** The video splice is
|
| 413 |
+
frame-exact; audio handling across the splice is not finished. The node
|
| 414 |
+
passes an AUDIO input through untouched, same length and same clock, which
|
| 415 |
+
is correct when the repair did not change the performance and is not a
|
| 416 |
+
general answer. If the bad frames carried dialogue, plan on repairing the
|
| 417 |
+
picture and keeping your original track.
|
| 418 |
+
|
| 419 |
+
## DyRoPE: choosing the dose
|
| 420 |
+
|
| 421 |
+
`H3 DyRoPE` is a dial on one question: how much of the run sees the clip's
|
| 422 |
+
true world duration instead of the stock uniform grid. That dose is what
|
| 423 |
+
buys the timing correction on a de-roped span, and it is also what costs
|
| 424 |
+
shimmer, so tuning it is picking a point on one curve rather than finding a
|
| 425 |
+
setting that is good at everything.
|
| 426 |
+
|
| 427 |
+
Two findings shape the whole dial. **Flash is an early-block effect**: the
|
| 428 |
+
physical grid in blocks 0 to 9 reproduces the entire flash penalty (7.84x
|
| 429 |
+
against a stock control's 2.09x), while every arm that keeps it out of those
|
| 430 |
+
blocks meters at the control's level. **Jitter is a dose effect**: once the
|
| 431 |
+
early blocks are excluded, it scales with total physical-time exposure
|
| 432 |
+
(blocks 25-49 2.61x, 30-49 2.33x, 40-49 1.39x; fading by sigma 0.3 2.72x,
|
| 433 |
+
0.5 2.07x, 0.7 2.06x).
|
| 434 |
+
|
| 435 |
+
The two settings worth starting from, with no winner between them:
|
| 436 |
+
|
| 437 |
+
| setting | what you get | what it costs |
|
| 438 |
+
|---|---|---|
|
| 439 |
+
| `physical_blocks`, blocks 30-49 | world speed held neutrally through the span | slight flicker or hitch inside the span |
|
| 440 |
+
| `physical_blocks`, blocks 40-49 | the smoothest span, and the cleanest meters measured (1.17x to 1.39x jitter across three takes) | on some content it slightly overshoots and reads over-smooth or slowed |
|
| 441 |
+
| `fade_physical_to_compact`, `fade_end 0.7` | flash-free in playback, timing held | visible jitter (2.06x to 2.87x across takes) |
|
| 442 |
+
|
| 443 |
+
Below those doses the world-time correction is gone entirely: `fade_end 0.8`
|
| 444 |
+
loses it because the window is too short, and stacking a block restriction
|
| 445 |
+
on top of a sigma window loses it too, even though both parents keep it.
|
| 446 |
+
Above them, shimmer rises. There is no quieter setting that is still on
|
| 447 |
+
time, so a clip whose timing slips wants MORE dose, not a cleverer one.
|
| 448 |
+
|
| 449 |
+
Three method notes.
|
| 450 |
+
|
| 451 |
+
- **The meters cannot pick for you.** They rank flash and jitter, and they
|
| 452 |
+
say nothing about whether the timing benefit survived. Every ranking
|
| 453 |
+
above came from a meter; every benefit boundary came from playback.
|
| 454 |
+
- **Match the arms.** `physical_all` reproduces `H3 True Clock` exactly and
|
| 455 |
+
`compact_all` reproduces having no node in the graph, so an A/B is one
|
| 456 |
+
widget rather than two graphs. Keep the seed, the hold map and the
|
| 457 |
+
schedule fixed across arms.
|
| 458 |
+
- **Watch out for the patch collision.** The `*_blocks` modes and
|
| 459 |
+
`H3 Streamed Blocks` both own the double-block replacement slot and
|
| 460 |
+
ComfyUI keeps one per block; the capability probe reports it. The fade
|
| 461 |
+
modes use no block patches and compose fine.
|
| 462 |
+
|
| 463 |
+
## Audio across a join (alpha)
|
| 464 |
+
|
| 465 |
+
`audio_prefix_ticks` on `H3 V2V Init` freezes the first n audio-latent ticks
|
| 466 |
+
to the seeded `audio_latent` and leaves every later tick at
|
| 467 |
+
`audio_strength`. It is the audio twin of the video prefix freeze, and it is
|
| 468 |
+
how a continuation carries the previous segment's sound instead of starting
|
| 469 |
+
a new performance.
|
| 470 |
+
|
| 471 |
+
Four constraints, all of which have bitten:
|
| 472 |
+
|
| 473 |
+
1. **It needs `audio_latent` wired.** The ticks are the audio latent's clock,
|
| 474 |
+
not the video's: a 39-frame carried handle is exactly 65 ticks.
|
| 475 |
+
2. **An AddGuide AUDIO guide and `ref_audios` cannot coexist** in the same
|
| 476 |
+
graph; they collide in conditioning-audio packing. (Measured on core 82f839f5; treat as a current-core rule, not a model property.) Pick one route.
|
| 477 |
+
3. **A release changes assembly.** With `audio_prefix_release_ticks` above 0
|
| 478 |
+
the last n ticks of the frozen prefix ramp up on a half cosine (8 ticks,
|
| 479 |
+
0.2 s, is the upstream lineage's tested recipe), and the assembly step must let the
|
| 480 |
+
CONTINUATION own the overlap tail. Trim the other way and the ramp is
|
| 481 |
+
discarded, which is worse than never asking for it.
|
| 482 |
+
4. **Scope.** It carries local continuity: level, timbre, beat. It does not
|
| 483 |
+
carry global phrase structure. A longer handle helps at the seam (90
|
| 484 |
+
frames measured 0.953 overlap correlation vs 0.92 at 39) and gives the
|
| 485 |
+
model a full musical phrase of context, but only a pinned master track
|
| 486 |
+
GUARANTEES the structure; that guarantee is by construction.
|
| 487 |
+
|
| 488 |
+
## Chained segments: drift and colour (alpha)
|
| 489 |
+
|
| 490 |
+
Two separate decays, two separate nodes, and neither is a substitute for the
|
| 491 |
+
other.
|
| 492 |
+
|
| 493 |
+
`H3 Drift Control` is for contrast and texture washing out down a chain. The
|
| 494 |
+
carried prefix is bit-clean conditioning, and repeated clean conditioning
|
| 495 |
+
accumulates error: the seam probe reads join one around 0.9 and join two
|
| 496 |
+
around 0.65 across every variant tested. The node gives the carried VIDEO
|
| 497 |
+
prefix a small schedule-matched amount of the sampler's own noise at every
|
| 498 |
+
model evaluation, tapering to exact at the seam end. Audio is untouched by
|
| 499 |
+
design; the frozen per-tick audio prefix stays hard.
|
| 500 |
+
|
| 501 |
+
Wiring and limits, in the order they trip people:
|
| 502 |
+
|
| 503 |
+
- Wire the `H3 V2V Init` latent in for shape (it must already carry the
|
| 504 |
+
noise mask; the node rewrites that mask, it does not create one), feed it
|
| 505 |
+
the SAME base model your pass-1 guider uses, and take the guider's model
|
| 506 |
+
from this node's output.
|
| 507 |
+
- `prefix_frames` must equal the plan's handle and sit on the 17k+5 grid.
|
| 508 |
+
39 frames is 12 latent steps, and `matched_steps` + `taper_steps` must sum
|
| 509 |
+
to exactly that: 8 + 4 is the field-validated recipe.
|
| 510 |
+
- It **refuses to stack** on another dynamic denoise-mask patch, and it does
|
| 511 |
+
not support sigma-split samplers.
|
| 512 |
+
- **Replicated across two chains and new seeds.** Joins hold 0.85-0.89
|
| 513 |
+
through link 4 on the newer chain; the earlier chain fell to the 0.65
|
| 514 |
+
class at its second join before the recipe existed. Cross-chain
|
| 515 |
+
comparison, not a paired drift-off arm. The recipe numbers are still
|
| 516 |
+
one recipe, not a swept space.
|
| 517 |
+
|
| 518 |
+
`H3 Delta Color Carry` (fed by two `H3 Scene Color Stats`) is for the other
|
| 519 |
+
decay: each carried handle is a VAE round trip, each round trip darkens
|
| 520 |
+
about 2.4%, and a chain accumulates it. It decodes the prefix once, applies
|
| 521 |
+
a weak scene-one exposure and saturation correction in RGB, encodes both the
|
| 522 |
+
original and the corrected frames, and adds only E(corrected) - E(original)
|
| 523 |
+
to the latent, so the encode bias cancels and only the grade survives. The
|
| 524 |
+
delta is low-passed spatially and tapered from zero at the old edge to full
|
| 525 |
+
strength beside the generated future; audio is never touched. Anchor is
|
| 526 |
+
scene one's delivered tail, current is the current predecessor's tail, and
|
| 527 |
+
on link one they are the same tail, so the transform is the identity.
|
| 528 |
+
|
| 529 |
+
Status, so nobody tunes against a number that does not exist yet:
|
| 530 |
+
Active path exercised on a real 4-link render 2026-08-24: it fires, corrects in the anchor direction, and stays sub-visible under its clamps on mildly-drifted content; on the deepest tested link it fires on both channels at or near its clamp. A strong-drift bench (where the 2.4% per-round-trip signature actually accrues) is still owed.
|
| 531 |
+
|
| 532 |
+
Both are adapted from ComfyUI-MiniMaxH3-Contex-Loop (GPL-3.0, the same
|
| 533 |
+
licence as this pack); the per-module credit lines in `h3_drift.py` and
|
| 534 |
+
`h3_color_carry.py` state what was taken and what was deliberately not
|
| 535 |
+
ported.
|
| 536 |
+
|
| 537 |
+
## Window mode (retiming an excerpt)
|
| 538 |
+
|
| 539 |
+
When only part of a long clip needs retiming, the window lives in four
|
| 540 |
+
coupled places: the prefix crop, the excerpt crop, the hold map, and the
|
| 541 |
+
regeneration's conditioning length. Move three of them and the render breaks
|
| 542 |
+
with the damage parked at the OLD boundary, because a latent longer than its
|
| 543 |
+
conditioning is garbage and the downstream masks resample to the old
|
| 544 |
+
proportions. Window starts also quantize to a 17-frame lattice (0.708 s at
|
| 545 |
+
24 fps), so a request to start at a given second lands on the nearest rung.
|
| 546 |
+
The arithmetic and the procedure are in `docs/WINDOW_MODE.md`.
|
| 547 |
+
|
| 548 |
+
## Method
|
| 549 |
+
|
| 550 |
+
- Change one dial at a time, same seed, and compare in playback. Still
|
| 551 |
+
frames lie about temporal quality in both directions; we have been
|
| 552 |
+
burned by this repeatedly.
|
| 553 |
+
- The oracle heatmap (`H3 Jerk Heatmap`) shows where the pipeline will
|
| 554 |
+
spend its budget. If the hot region misses the artifact the user is
|
| 555 |
+
pointing at, fix `q` before touching anything else.
|
| 556 |
+
- Sharpness metrics rank frames usefully within one clip but do not
|
| 557 |
+
decide between settings. The user's eyes decide.
|
| 558 |
+
- Cost scales with the burst span, not the clip length. A long clip with
|
| 559 |
+
one short burst is cheap to fix.
|
| 560 |
+
|
| 561 |
+
## Trying the indecision oracle (experimental)
|
| 562 |
+
|
| 563 |
+
`H3 Indecision Oracle` is a second signal for the same hold map: the
|
| 564 |
+
model's own x0 jitter instead of the clip's jerk. It is not on by
|
| 565 |
+
default and nothing else moved for it.
|
| 566 |
+
|
| 567 |
+
- Cost: free if pass 1 already ran through `X0 Tap` with two steps
|
| 568 |
+
dumped. It reads files, it does not sample.
|
| 569 |
+
- Compare, do not switch. Put it where the jerk oracle sits, flip `mode`
|
| 570 |
+
between `indecision`, `jerk passthrough` and `blend max` with
|
| 571 |
+
everything else fixed, and judge in playback. `jerk passthrough` is
|
| 572 |
+
the same numbers the jerk oracle produces, so the only variable is
|
| 573 |
+
the signal.
|
| 574 |
+
- Keep `q`, `d_max`, `ramp` and `bridge` matched across the arms. They
|
| 575 |
+
feed the same compiler, and an unmatched knob turns the A/B into a
|
| 576 |
+
comparison of two hold-map compilers.
|
| 577 |
+
- The `comparison` output is the measurement, not the vibe: whole-map
|
| 578 |
+
Spearman, top-decile IoU, and the token-times where the two sources
|
| 579 |
+
disagree most. Look at those token-times in playback first, since
|
| 580 |
+
that is where the choice actually shows.
|
| 581 |
+
- If the map looks like a rectangle rather than like the picture, check
|
| 582 |
+
the report for the degeneracy line. Masked and repaint runs pin token
|
| 583 |
+
rows to exactly zero jitter and the oracle then draws the mask.
|
| 584 |
+
- Expected failure mode: fast small props. Jitter under-ranks them
|
| 585 |
+
relative to frame-diff, which is the argument for `blend max` over
|
| 586 |
+
`indecision` alone.
|
| 587 |
+
|
| 588 |
+
## Defaults, and why
|
| 589 |
+
|
| 590 |
+
`q 0.75, d_max 4, ramp on, bridge 8, inject 0.70` is the playback-ratified
|
| 591 |
+
starting point. The shipped default is now `inject 0.48`: at 0.50 and above some
|
| 592 |
+
scenes measurably loosen reference adherence, and 0.48 keeps the sharpness
|
| 593 |
+
without that risk. `inject 0.50` measured sharper with closer motion tracking
|
| 594 |
+
on our clips and is one preset away; some of us prefer it. The expert
|
| 595 |
+
schedule defaults (`total 8, inject 0.70, base_head 2`) give the turbo
|
| 596 |
+
tail its native 4 steps.
|
| 597 |
+
|
| 598 |
+
## Identity vs strength: the reference rule
|
| 599 |
+
|
| 600 |
+
Whether a higher inject or denoise is safe depends on what is anchoring
|
| 601 |
+
the subject, not on the route.
|
| 602 |
+
|
| 603 |
+
- No substantial references in the graph: keep the repaint strength
|
| 604 |
+
under 0.5 when the subject's identity must match the source. Identity
|
| 605 |
+
drift is measurable from about 0.30 and structural past 0.5; below
|
| 606 |
+
that line the init still owns the subject, above it the model's prior
|
| 607 |
+
does, and no prompt wording buys it back.
|
| 608 |
+
- With substantial references (identity refs, FLF pins, a strong anchor
|
| 609 |
+
frame): higher strength is fine, and on the high-step routes (the
|
| 610 |
+
full de-rope at 25+ steps) it often helps more, because the
|
| 611 |
+
references carry identity so the extra strength buys motion quality
|
| 612 |
+
instead of drift. The pinned FLF de-rope holding identity at inject
|
| 613 |
+
0.70 above is the worked example.
|
| 614 |
+
|
| 615 |
+
## Refining this guide
|
| 616 |
+
|
| 617 |
+
When a tuning session finds a symptom/dial pair this table lacks, add it.
|
| 618 |
+
That is the point of the file.
|
ComfyUI-MiniMaxH3-Context-Loop/ComfyUI-MAINodes/VIDEO_TOOLS.md
ADDED
|
@@ -0,0 +1,116 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# MAI video tools (alpha)
|
| 2 |
+
|
| 3 |
+
Three nodes and one widget that remove drudgery around renders without taking
|
| 4 |
+
any decision away from you. None of them runs a model.
|
| 5 |
+
|
| 6 |
+
## MAI Video Out (file + sidecars + draft preview)
|
| 7 |
+
|
| 8 |
+
Save Video with the record beside it. Frames (+ audio) go through ComfyUI's
|
| 9 |
+
own encoder (h264 / av1, CRF), exactly like Create Video + Save Video; the
|
| 10 |
+
extras are each off unless wired or toggled:
|
| 11 |
+
|
| 12 |
+
- `hold_map` -> `<prefix>_00001_.holdmap.json`: the clock that produced these
|
| 13 |
+
frames. H3 Load Hold Map and the deck's hold-map staircase read it.
|
| 14 |
+
- `write_meta` -> `<prefix>_00001_.meta.json`: seeds, models, LoRAs, steps,
|
| 15 |
+
samplers, inject schedule, encode time, your `notes`, all read out of the
|
| 16 |
+
graph that ran. The compare viewer's metadata strip and the scorer's labels
|
| 17 |
+
come from here instead of from file names.
|
| 18 |
+
- `draft_preview` + `latent`: decode the sampled latent through the tiny VAE
|
| 19 |
+
in `models/vae_approx` (`taeh3` for H3; any stem) into `<prefix>_draft.mp4`.
|
| 20 |
+
The tiny VAE is loaded for that call and released after; with the toggle
|
| 21 |
+
off nothing is loaded and nothing runs. Cost measured 2026-08-21: 8 ms per
|
| 22 |
+
latent frame at 864x480, 21 ms at 1344x768, 42 ms at 1080p.
|
| 23 |
+
|
| 24 |
+
## MAI Load Video (path) / MAI Select Every Nth
|
| 25 |
+
|
| 26 |
+
The two things the graphs still took from VideoHelperSuite: a video by
|
| 27 |
+
absolute or input-relative path (as ComfyUI's VIDEO type, frames and fps
|
| 28 |
+
alongside), and every n-th frame from an offset. For a smeared clip use H3
|
| 29 |
+
Exact Recover instead; it is exact.
|
| 30 |
+
|
| 31 |
+
## MAI Video Compare (2-6, synchronized)
|
| 32 |
+
|
| 33 |
+
The player has grown since this page was written: a waveform timeline with
|
| 34 |
+
loop brackets, a frame stepper, and a frame-exact export. The README's
|
| 35 |
+
MAI Video Compare section is the current reference; this page keeps the
|
| 36 |
+
original wiring notes.
|
| 37 |
+
|
| 38 |
+
A media UI, not a processing stage. The node writes each wired VIDEO once as a
|
| 39 |
+
small h264 preview into the temp directory (CPU, no VAE, no tensors kept) and
|
| 40 |
+
the widget plays them in the browser with the flipbook's player: one set of
|
| 41 |
+
live `<video>` elements, buffered start-together, flip / wipe / side-by-side
|
| 42 |
+
for a pair, a grid for more.
|
| 43 |
+
|
| 44 |
+
- hover a source to hear it, click to lock its audio
|
| 45 |
+
- space play/pause all, arrows step all one frame (shift: 12), F flicker A/B
|
| 46 |
+
- 1-6 pick the pair (last two pressed), Enter stars the B side
|
| 47 |
+
- the star sets the node's `winner` widget; the NEXT queue passes that source
|
| 48 |
+
through `winner_video` (and `winner_index`). Pick, then finalize in a second
|
| 49 |
+
execution: a graph never waits on a human mid-run.
|
| 50 |
+
|
| 51 |
+
The flipbook deck (https://matlowai.github.io/flipbook/derope.html) is the
|
| 52 |
+
offline twin, same player code.
|
| 53 |
+
|
| 54 |
+
## MAI Seed Hunter (compare + winner seed)
|
| 55 |
+
|
| 56 |
+
Video Compare that knows about seeds: wire 2-6 cheap candidates and the seed
|
| 57 |
+
each ran on (labels default to the seed), star the keeper, and the NEXT queue
|
| 58 |
+
passes the starred candidate's seed out of `winner_seed` for the finalize
|
| 59 |
+
pass's RandomNoise (and its video out of `winner_video`). The seed-hunt pass
|
| 60 |
+
and the finalize pass are two executions by design.
|
| 61 |
+
|
| 62 |
+
## MAI Motion Editor additions
|
| 63 |
+
|
| 64 |
+
Alpha. The editor's rows, modal, undo and run buttons have only been exercised by
|
| 65 |
+
an automated browser so far; expect rough edges, and treat anything surprising
|
| 66 |
+
as a bug to report rather than a design choice.
|
| 67 |
+
|
| 68 |
+
Verified headless against a lab instance (2026-08-22, Playwright): run-to-here
|
| 69 |
+
renders base + oracle + filmstrip and holds the graph; a drag on the block lane
|
| 70 |
+
creates exactly the dragged range (snapped to the token grid); the live envelope
|
| 71 |
+
drives the dilated clock, the play loop and the price before any render; the
|
| 72 |
+
modal applies as one undo step; run-from-here resumes at the editor with the
|
| 73 |
+
whole base chain from ComfyUI's cache (14 nodes cached, editor + downstream
|
| 74 |
+
executed); a fresh page load restores the last run's filmstrip from the server.
|
| 75 |
+
|
| 76 |
+
- **run to here / run from here** on the node: the first is partial execution
|
| 77 |
+
(nothing downstream), the second a normal queue where unchanged upstream
|
| 78 |
+
nodes come from the cache, so it resumes at this node. Editing a row then
|
| 79 |
+
"run from here" is the retry loop.
|
| 80 |
+
- **Undo**: the node's undo/redo is the fine-grained one (every row edit, drag,
|
| 81 |
+
modal apply). ComfyUI's own ctrl+z jumps back to the graph's last snapshot
|
| 82 |
+
(the widget re-hydrates from it and says so); the graph tracker does not see
|
| 83 |
+
edits made inside the widget individually, so ctrl+shift+z at graph level
|
| 84 |
+
has nothing to redo after that. Use the node's buttons or ctrl+z/ctrl+shift+z
|
| 85 |
+
while the editor has focus.
|
| 86 |
+
- **hold_until_edited** (default on): with no rows, a run stops after the
|
| 87 |
+
filmstrip; lay out rows and run again.
|
| 88 |
+
- **Cache reality**: ComfyUI keeps only the previous prompt's outputs, so
|
| 89 |
+
running other graphs in between evicts the base pass and the next
|
| 90 |
+
run-from-here re-renders it. The other way to lose the cache is a seed that
|
| 91 |
+
moves: the frontend's default "control after generate" is randomize on
|
| 92 |
+
nodes it builds, so the plain Run button re-renders the base every time
|
| 93 |
+
until both RandomNoise nodes are set to fixed. The node's run buttons pin
|
| 94 |
+
every seed control to fixed before queueing.
|
| 95 |
+
|
| 96 |
+
- **Play** (space): the filmstrip plays at fps; in the dilated clock each
|
| 97 |
+
frame lingers for its hold, so the slowdown is seen, not inferred.
|
| 98 |
+
- **clock: world / dilated**: the timeline's x-axis reads as "when" (frame)
|
| 99 |
+
or as "how long it costs" (cumulative holds). The compiled hold curve from
|
| 100 |
+
the last run is drawn over the jerk profile in both.
|
| 101 |
+
|
| 102 |
+
## Paste an API workflow
|
| 103 |
+
|
| 104 |
+
With the pack installed, Ctrl-V of API-format JSON on the canvas works: the
|
| 105 |
+
stock paste handler only takes UI-format workflows, so `web/api_paste.js`
|
| 106 |
+
catches API JSON first and routes it through the frontend's own `loadApiJson`
|
| 107 |
+
(widgets built from the server schema, never by position). It replaces the
|
| 108 |
+
graph, so a non-empty canvas asks first. The deck's "api" links and every
|
| 109 |
+
`*_api.json` in `examples/` paste directly now.
|
| 110 |
+
|
| 111 |
+
## What these replace
|
| 112 |
+
|
| 113 |
+
Our graphs used VHS_LoadVideoPath and VHS_SelectEveryNthImage; both are
|
| 114 |
+
covered. Create Video + Save Video stay valid; MAI Video Out is those plus the
|
| 115 |
+
sidecars. The compare node replaces opening three files and scrubbing them to
|
| 116 |
+
roughly the same place.
|
ComfyUI-MiniMaxH3-Context-Loop/ComfyUI-MAINodes/__init__.py
ADDED
|
@@ -0,0 +1,53 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""ComfyUI-MAINodes — MatlowAI's MiniMax-H3 node collection.
|
| 2 |
+
|
| 3 |
+
Two families in one pack:
|
| 4 |
+
- Contact-Sheet diffusion (five coordinated views from one reference) —
|
| 5 |
+
V3-API nodes, unchanged from ComfyUI-H3-ContactSheet.
|
| 6 |
+
- Motion Lab (v2v time-smear de-roping pipeline: jerk oracle, time smear,
|
| 7 |
+
inject schedule, exact recovery, oracle heatmap) — V1 nodes, defaults
|
| 8 |
+
set to measured-best values.
|
| 9 |
+
|
| 10 |
+
Registration note: ComfyUI's loader treats NODE_CLASS_MAPPINGS and
|
| 11 |
+
comfy_entrypoint as either-or (nodes.py: `elif hasattr(module,
|
| 12 |
+
"comfy_entrypoint")`), and its V3 path registers node classes into the
|
| 13 |
+
same mappings dict keyed by schema.node_id — so we merge the V3 classes
|
| 14 |
+
in here ourselves instead of exporting an entrypoint.
|
| 15 |
+
|
| 16 |
+
The original ComfyUI-H3-ContactSheet repo remains for existing installs;
|
| 17 |
+
this is the consolidated home going forward.
|
| 18 |
+
"""
|
| 19 |
+
from .contact_sheet import H3ContactSheet, H3ContactSheetDecode
|
| 20 |
+
from .motion import TIMESMEAR_CLASS_MAPPINGS, TIMESMEAR_DISPLAY_MAPPINGS
|
| 21 |
+
|
| 22 |
+
WEB_DIRECTORY = "./web"
|
| 23 |
+
|
| 24 |
+
NODE_CLASS_MAPPINGS = dict(TIMESMEAR_CLASS_MAPPINGS)
|
| 25 |
+
NODE_DISPLAY_NAME_MAPPINGS = dict(TIMESMEAR_DISPLAY_MAPPINGS)
|
| 26 |
+
|
| 27 |
+
# Rolling-window drivers (alpha, 2026-08-12). Three implementations of the
|
| 28 |
+
# same idea, deliberately shipped side by side while they are compared on
|
| 29 |
+
# real renders: the requeue pair lives in motion.py, the other two in their
|
| 30 |
+
# own modules. They share the seam policy and the window planner's rules but
|
| 31 |
+
# not its code. Expect at most one of them to survive.
|
| 32 |
+
#
|
| 33 |
+
# timeline (alpha, 2026-08-14) is loaded the same guarded way: it is the
|
| 34 |
+
# plan-document surface, and a broken plan schema must not cost anyone the
|
| 35 |
+
# de-roping nodes. vram_lab (alpha, 2026-08-18) is the exact low-VRAM block
|
| 36 |
+
# executor; same rule.
|
| 37 |
+
for _mod in ("window_loop", "window_expand", "timeline.nodes",
|
| 38 |
+
"concept_lab.nodes", "vram_lab", "derope_any", "video_out", "video_compare",
|
| 39 |
+
"h3_capabilities", "h3_extend", "h3_repair", "h3_drift", "h3_color_carry"):
|
| 40 |
+
try:
|
| 41 |
+
_m = __import__(f"{__name__}.{_mod}", fromlist=["*"])
|
| 42 |
+
except Exception as _e: # an alpha module must never
|
| 43 |
+
print(f"[MAINodes] {_mod} not loaded: " # take the whole pack down
|
| 44 |
+
f"{type(_e).__name__}: {_e}")
|
| 45 |
+
continue
|
| 46 |
+
NODE_CLASS_MAPPINGS.update(getattr(_m, "NODE_CLASS_MAPPINGS", {}))
|
| 47 |
+
NODE_DISPLAY_NAME_MAPPINGS.update(getattr(_m, "NODE_DISPLAY_NAME_MAPPINGS", {}))
|
| 48 |
+
|
| 49 |
+
for _cls in (H3ContactSheet, H3ContactSheetDecode):
|
| 50 |
+
_schema = _cls.GET_SCHEMA()
|
| 51 |
+
NODE_CLASS_MAPPINGS[_schema.node_id] = _cls
|
| 52 |
+
if _schema.display_name is not None:
|
| 53 |
+
NODE_DISPLAY_NAME_MAPPINGS[_schema.node_id] = _schema.display_name
|
ComfyUI-MiniMaxH3-Context-Loop/ComfyUI-MAINodes/assets/adapter_chaos_derope_triptych.mp4
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:8cf8554e17223bfaf76d9d4669c6f8150c9b5e7a5a07dc55d8d07cc76e75c2b9
|
| 3 |
+
size 1806979
|
ComfyUI-MiniMaxH3-Context-Loop/ComfyUI-MAINodes/assets/adapter_drop_full3x_triptych.mp4
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:af8988c31d4cc51657769f91813a1a3813d7ad015432be3cf83c94cec74829dd
|
| 3 |
+
size 1310988
|
ComfyUI-MiniMaxH3-Context-Loop/ComfyUI-MAINodes/assets/adapter_mage_pinned_quad.mp4
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:6a02ce11fc434790e082ed8c2d21d8d29fcafcf54a2cc9ce2d01e7e2b94cf135
|
| 3 |
+
size 2666539
|
ComfyUI-MiniMaxH3-Context-Loop/ComfyUI-MAINodes/assets/adapter_ref2va_colour_muting.mp4
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:258ec7fdf7890946353c8eacc503ffe076f7742c958e065fa08ab328a2046774
|
| 3 |
+
size 1405217
|
ComfyUI-MiniMaxH3-Context-Loop/ComfyUI-MAINodes/assets/adapter_rooftop_ood_triptych.mp4
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:ac73362390fa7320b250404771d4c7b306bd0f69af9cf752739640edcbbba918
|
| 3 |
+
size 2539230
|
ComfyUI-MiniMaxH3-Context-Loop/ComfyUI-MAINodes/assets/adapter_t2c_insert_quad.mp4
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:980af606d0127786f4755d3b73133d452beb0acf16ed617c479163c15550225c
|
| 3 |
+
size 2904346
|
ComfyUI-MiniMaxH3-Context-Loop/ComfyUI-MAINodes/assets/baseline_vs_regenerated_sbs.mp4
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:2f8cb97cd0df399dcb69ac1683634a97474397c5e26d68e8c7f50d268ed8ca30
|
| 3 |
+
size 3584130
|
ComfyUI-MiniMaxH3-Context-Loop/ComfyUI-MAINodes/assets/derope_sbs.gif
ADDED
|
Git LFS Details
|
ComfyUI-MiniMaxH3-Context-Loop/ComfyUI-MAINodes/assets/featherweight_flf_triptych.mp4
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:e521ce137d84ba2a47eaec82d22b605a1066d9c7b7e22f3a2518e101eaff95a1
|
| 3 |
+
size 7522184
|
ComfyUI-MiniMaxH3-Context-Loop/ComfyUI-MAINodes/assets/featherweight_i2v_triptych.mp4
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:33d6dcab412a37ea2d1f5fcb166dda93e2a2f25de4a1e492f55d1823563b415e
|
| 3 |
+
size 4100757
|
ComfyUI-MiniMaxH3-Context-Loop/ComfyUI-MAINodes/assets/featherweight_int8_vs_w4a8_quad.mp4
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:820b3b625f8e81f597c1f4176470879d777e93f054af1477c590c089058bb12d
|
| 3 |
+
size 14623732
|
ComfyUI-MiniMaxH3-Context-Loop/ComfyUI-MAINodes/assets/featherweight_ref_triptych.mp4
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:61377cd4dba6388aa842652d4d8776cde06b18c023b708292fd38b2c21236777
|
| 3 |
+
size 4798651
|
ComfyUI-MiniMaxH3-Context-Loop/ComfyUI-MAINodes/assets/kitsune_threeway.gif
ADDED
|
Git LFS Details
|
ComfyUI-MiniMaxH3-Context-Loop/ComfyUI-MAINodes/assets/kitsune_threeway_sbs.mp4
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:554d8903b310c133c680d57db1f24fdb90b10c7da090f769180dbed95463b3c1
|
| 3 |
+
size 4442322
|
ComfyUI-MiniMaxH3-Context-Loop/ComfyUI-MAINodes/assets/oracle_map.gif
ADDED
|
Git LFS Details
|
ComfyUI-MiniMaxH3-Context-Loop/ComfyUI-MAINodes/assets/oracle_map.mp4
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:bebae5a6d9752a3ecd20fc7a66dfecaa22ec8d0ad82cf2795faa31fc72dc665e
|
| 3 |
+
size 1234068
|
ComfyUI-MiniMaxH3-Context-Loop/ComfyUI-MAINodes/assets/panrun_sbs.mp4
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:c51f3005d2797555b56853987da81c0c8bcad39999de509696aa632bf9580249
|
| 3 |
+
size 6833536
|
ComfyUI-MiniMaxH3-Context-Loop/ComfyUI-MAINodes/assets/preview_ladder_grid.mp4
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:e30bdc90eebc07885e4f4c6b189aa805e245cb57b91ab4defc57e1e47dc74da4
|
| 3 |
+
size 11178570
|
ComfyUI-MiniMaxH3-Context-Loop/ComfyUI-MAINodes/assets/time_burst_plate.mp4
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:2c16132fc3b37547657df847d87b040b8398e400454668b02516a5054573cfae
|
| 3 |
+
size 2025296
|
ComfyUI-MiniMaxH3-Context-Loop/ComfyUI-MAINodes/assets/time_burst_result.mp4
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:6200451eb76f988a461c7bab03576a17ccaf27a1f06c0dad900f04f52e745f0c
|
| 3 |
+
size 2010256
|
ComfyUI-MiniMaxH3-Context-Loop/ComfyUI-MAINodes/assets/uniform_vs_adaptive.gif
ADDED
|
Git LFS Details
|
ComfyUI-MiniMaxH3-Context-Loop/ComfyUI-MAINodes/assets/uniform_vs_adaptive_sbs.mp4
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:31da4b90a9d7c9c808babda85a67427f9d2e8db2f7985144f2056342d380e49c
|
| 3 |
+
size 2727214
|
ComfyUI-MiniMaxH3-Context-Loop/ComfyUI-MAINodes/capsule_types.py
ADDED
|
@@ -0,0 +1,185 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Shared data vocabulary for stage handoff (alpha, 2026-08-23).
|
| 2 |
+
|
| 3 |
+
No persistence here. These are the structures the extension planner emits
|
| 4 |
+
today and a Capsule will serialize tomorrow, frozen now so the first
|
| 5 |
+
feature built to consume a Capsule does not need an adapter around it.
|
| 6 |
+
|
| 7 |
+
Time is integer or rational everywhere. Seconds are a display field.
|
| 8 |
+
|
| 9 |
+
Timebase fps as a rational, the 40 Hz H3 audio-latent clock, sample rate
|
| 10 |
+
Span a half-open frame range with its tick and sample counts
|
| 11 |
+
Handle a typed span shared by extension prefix, segment-crop handles
|
| 12 |
+
and rolling-window overlap: role, where it comes from, where it
|
| 13 |
+
lands, whether it is protected from retiming
|
| 14 |
+
ConditioningFingerprint what the prompt MEANT on this runtime (#15808
|
| 15 |
+
made prompt text + model name insufficient)
|
| 16 |
+
ExecutionFingerprint what actually ran: commits, hashes, the
|
| 17 |
+
capability snapshot, requested / resolved / executed values
|
| 18 |
+
|
| 19 |
+
Every structure round-trips through ``to_dict`` / ``from_dict`` and hashes
|
| 20 |
+
through ``canonical_json`` (sorted keys, no whitespace) so two producers of
|
| 21 |
+
the same content get the same digest.
|
| 22 |
+
"""
|
| 23 |
+
from __future__ import annotations
|
| 24 |
+
|
| 25 |
+
import hashlib
|
| 26 |
+
import json
|
| 27 |
+
from dataclasses import asdict, dataclass, field
|
| 28 |
+
from fractions import Fraction
|
| 29 |
+
from typing import Optional
|
| 30 |
+
|
| 31 |
+
SCHEMA = "mai.h3.capsule-types/1"
|
| 32 |
+
|
| 33 |
+
H3_FRAME_GRID = (17, 5) # legal clip length = 17k + 5
|
| 34 |
+
H3_AUDIO_HZ = 40 # audio latent ticks per second
|
| 35 |
+
|
| 36 |
+
|
| 37 |
+
def canonical_json(obj) -> str:
|
| 38 |
+
return json.dumps(obj, sort_keys=True, separators=(",", ":"), ensure_ascii=True, default=str)
|
| 39 |
+
|
| 40 |
+
|
| 41 |
+
def digest(obj) -> str:
|
| 42 |
+
return "sha256:" + hashlib.sha256(canonical_json(obj).encode("utf-8")).hexdigest()
|
| 43 |
+
|
| 44 |
+
|
| 45 |
+
def align_up(n: int, k: int = H3_FRAME_GRID[0], r: int = H3_FRAME_GRID[1]) -> int:
|
| 46 |
+
"""Generation length: core rounds UP to 17k+5 (align_frame_count)."""
|
| 47 |
+
n = max(int(n), r)
|
| 48 |
+
while n % k != r:
|
| 49 |
+
n += 1
|
| 50 |
+
return n
|
| 51 |
+
|
| 52 |
+
|
| 53 |
+
def align_down(n: int, k: int = H3_FRAME_GRID[0], r: int = H3_FRAME_GRID[1]) -> int:
|
| 54 |
+
"""Clip guide / ref video length: core rounds DOWN to 17k+5; below 5 a
|
| 55 |
+
guide becomes a single frame (1), zero stays zero."""
|
| 56 |
+
n = int(n)
|
| 57 |
+
if n <= 0:
|
| 58 |
+
return 0
|
| 59 |
+
if n < r:
|
| 60 |
+
return 1
|
| 61 |
+
while n % k != r:
|
| 62 |
+
n -= 1
|
| 63 |
+
return n
|
| 64 |
+
|
| 65 |
+
|
| 66 |
+
@dataclass
|
| 67 |
+
class Timebase:
|
| 68 |
+
fps_num: int = 24
|
| 69 |
+
fps_den: int = 1
|
| 70 |
+
audio_hz: int = H3_AUDIO_HZ
|
| 71 |
+
sample_rate: int = 0 # 0 = unknown until an AUDIO arrives
|
| 72 |
+
|
| 73 |
+
@property
|
| 74 |
+
def fps(self) -> Fraction:
|
| 75 |
+
return Fraction(self.fps_num, self.fps_den)
|
| 76 |
+
|
| 77 |
+
def ticks(self, frames: int) -> Fraction:
|
| 78 |
+
return Fraction(frames) * self.audio_hz / self.fps
|
| 79 |
+
|
| 80 |
+
def samples(self, frames: int) -> Fraction:
|
| 81 |
+
return Fraction(frames) * self.sample_rate / self.fps
|
| 82 |
+
|
| 83 |
+
def seconds(self, frames: int) -> float:
|
| 84 |
+
return float(Fraction(frames) / self.fps)
|
| 85 |
+
|
| 86 |
+
def clock_aligned(self, frames: int) -> bool:
|
| 87 |
+
return self.ticks(frames).denominator == 1
|
| 88 |
+
|
| 89 |
+
def to_dict(self):
|
| 90 |
+
return asdict(self)
|
| 91 |
+
|
| 92 |
+
@classmethod
|
| 93 |
+
def from_dict(cls, d):
|
| 94 |
+
return cls(**{k: d[k] for k in ("fps_num", "fps_den", "audio_hz", "sample_rate") if k in d})
|
| 95 |
+
|
| 96 |
+
|
| 97 |
+
@dataclass
|
| 98 |
+
class Span:
|
| 99 |
+
"""Half-open [start, end) in frames, with the derived counts spelled out
|
| 100 |
+
so a reader never re-derives them from a different clock."""
|
| 101 |
+
start: int
|
| 102 |
+
end: int
|
| 103 |
+
ticks: Optional[str] = None # Fraction as "num/den" string, exact
|
| 104 |
+
samples: Optional[str] = None
|
| 105 |
+
|
| 106 |
+
@property
|
| 107 |
+
def frames(self) -> int:
|
| 108 |
+
return self.end - self.start
|
| 109 |
+
|
| 110 |
+
@staticmethod
|
| 111 |
+
def make(start: int, end: int, tb: Timebase) -> "Span":
|
| 112 |
+
n = end - start
|
| 113 |
+
s = Span(int(start), int(end), str(tb.ticks(n)),
|
| 114 |
+
str(tb.samples(n)) if tb.sample_rate else None)
|
| 115 |
+
return s
|
| 116 |
+
|
| 117 |
+
def to_dict(self):
|
| 118 |
+
return asdict(self)
|
| 119 |
+
|
| 120 |
+
@classmethod
|
| 121 |
+
def from_dict(cls, d):
|
| 122 |
+
return cls(d["start"], d["end"], d.get("ticks"), d.get("samples"))
|
| 123 |
+
|
| 124 |
+
|
| 125 |
+
@dataclass
|
| 126 |
+
class Handle:
|
| 127 |
+
"""One concept for extension prefix, segment-crop handle, window overlap."""
|
| 128 |
+
role: str # extension_prefix | segment_crop | window_overlap | edit_region
|
| 129 |
+
source: Span # where the pixels come from (previous clip coords)
|
| 130 |
+
destination: Span # where they land (new clip coords)
|
| 131 |
+
global_span: Optional[Span] = None # on the assembled timeline
|
| 132 |
+
protected: bool = True # the de-rope may not retime it
|
| 133 |
+
retime_allowed: bool = False
|
| 134 |
+
ownership: str = "source" # who owns the pixels at assembly: source | new | blend
|
| 135 |
+
visual_anchor: str = "guide" # per_token_mask | guide | none
|
| 136 |
+
audio_anchor: str = "guide" # guide | regenerated | none
|
| 137 |
+
notes: list = field(default_factory=list)
|
| 138 |
+
|
| 139 |
+
def to_dict(self):
|
| 140 |
+
d = asdict(self)
|
| 141 |
+
return d
|
| 142 |
+
|
| 143 |
+
@classmethod
|
| 144 |
+
def from_dict(cls, d):
|
| 145 |
+
return cls(role=d["role"], source=Span.from_dict(d["source"]),
|
| 146 |
+
destination=Span.from_dict(d["destination"]),
|
| 147 |
+
global_span=Span.from_dict(d["global_span"]) if d.get("global_span") else None,
|
| 148 |
+
protected=d.get("protected", True), retime_allowed=d.get("retime_allowed", False),
|
| 149 |
+
ownership=d.get("ownership", "source"), visual_anchor=d.get("visual_anchor", "guide"),
|
| 150 |
+
audio_anchor=d.get("audio_anchor", "guide"), notes=list(d.get("notes", [])))
|
| 151 |
+
|
| 152 |
+
|
| 153 |
+
@dataclass
|
| 154 |
+
class ConditioningFingerprint:
|
| 155 |
+
prompt_sha: str = ""
|
| 156 |
+
tokenizer_special_tokens: Optional[bool] = None # #15808 present on the runtime that encoded it
|
| 157 |
+
comfy_commit: Optional[str] = None
|
| 158 |
+
ref_roles: dict = field(default_factory=dict) # "<Picture 1>": "identity", "<Video 1>": "motion"
|
| 159 |
+
visual_cond_noise_aug: Optional[float] = None
|
| 160 |
+
audio_cond_noise_aug: Optional[float] = None
|
| 161 |
+
|
| 162 |
+
def to_dict(self):
|
| 163 |
+
return asdict(self)
|
| 164 |
+
|
| 165 |
+
@classmethod
|
| 166 |
+
def from_dict(cls, d):
|
| 167 |
+
return cls(**{k: d[k] for k in cls.__dataclass_fields__ if k in d})
|
| 168 |
+
|
| 169 |
+
|
| 170 |
+
@dataclass
|
| 171 |
+
class ExecutionFingerprint:
|
| 172 |
+
comfy_commit: Optional[str] = None
|
| 173 |
+
mainodes_commit: Optional[str] = None
|
| 174 |
+
capabilities: dict = field(default_factory=dict) # the probe snapshot that ran
|
| 175 |
+
requested: dict = field(default_factory=dict)
|
| 176 |
+
resolved: dict = field(default_factory=dict)
|
| 177 |
+
executed: dict = field(default_factory=dict) # filled after the run; empty = not yet
|
| 178 |
+
reasons: dict = field(default_factory=dict) # why resolved != requested
|
| 179 |
+
|
| 180 |
+
def to_dict(self):
|
| 181 |
+
return asdict(self)
|
| 182 |
+
|
| 183 |
+
@classmethod
|
| 184 |
+
def from_dict(cls, d):
|
| 185 |
+
return cls(**{k: d[k] for k in cls.__dataclass_fields__ if k in d})
|
ComfyUI-MiniMaxH3-Context-Loop/ComfyUI-MAINodes/concept_lab/DECISIONS.md
ADDED
|
@@ -0,0 +1,543 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Concept Lab decisions
|
| 2 |
+
|
| 3 |
+
Append only. Never edit an old entry: add a new one with `supersedes:`
|
| 4 |
+
instead. An edited history means the next person re-litigates a settled
|
| 5 |
+
argument without knowing they are doing it, and the path looking straighter
|
| 6 |
+
than it was is precisely the damage.
|
| 7 |
+
|
| 8 |
+
Format: `DEC-CL-nnnn · STATUS · date · title`, then Decision / Alternatives
|
| 9 |
+
/ Consequences.
|
| 10 |
+
|
| 11 |
+
---
|
| 12 |
+
|
| 13 |
+
## DEC-CL-0001 · ACCEPTED · 2026-08-15 · Fresh subsystem inside MAINodes, named `concept_lab/`
|
| 14 |
+
|
| 15 |
+
**Decision.** Functional-conditioning work lands as a new package inside
|
| 16 |
+
ComfyUI-MAINodes rather than as an extension of any existing
|
| 17 |
+
reference-format or adapter-format code. The name is `concept_lab/`.
|
| 18 |
+
|
| 19 |
+
**Alternatives.** `concept_surgery/` (the source bundle's first suggestion,
|
| 20 |
+
and it carries the ancestor program's vocabulary) — not chosen; it reads
|
| 21 |
+
sharper than the work is, on a public node pack.
|
| 22 |
+
`functional_conditioning/` — most accurate, longest imports, unwieldy node
|
| 23 |
+
names. Operator chose `concept_lab/` to sit alongside Motion Lab and Voice
|
| 24 |
+
Lab: a place you work rather than an operation you perform.
|
| 25 |
+
|
| 26 |
+
**Consequences.** Backend-specific formats are compiler *targets*, not core
|
| 27 |
+
types. Node display names will read "MAI Concept ...".
|
| 28 |
+
|
| 29 |
+
---
|
| 30 |
+
|
| 31 |
+
## DEC-CL-0002 · ACCEPTED · 2026-08-15 · The functional delta is the primary object
|
| 32 |
+
|
| 33 |
+
**Decision.** The primary scientific object is the matched change in model
|
| 34 |
+
function caused by a conditioning intervention, not the content of the
|
| 35 |
+
source asset and not a similarity between latents.
|
| 36 |
+
|
| 37 |
+
**Consequences.** Source-latent similarity alone can never certify a
|
| 38 |
+
semantic factor. Every factor claim has to reach an effect on the target
|
| 39 |
+
stream.
|
| 40 |
+
|
| 41 |
+
---
|
| 42 |
+
|
| 43 |
+
## DEC-CL-0003 · ACCEPTED · 2026-08-15 · Model-neutral core, H3 as first backend
|
| 44 |
+
|
| 45 |
+
**Decision.** Core data and factor APIs know nothing model-specific.
|
| 46 |
+
Segment names, block counts, sampler shapes and layout rules live behind
|
| 47 |
+
`backends/`. H3 is first because MAINodes already has H3 tooling and H3 has
|
| 48 |
+
a native image/video/audio reference path.
|
| 49 |
+
|
| 50 |
+
**Consequences.** `types.py` treats segment names as opaque strings. The
|
| 51 |
+
second backend should not require rewriting the first one's evidence.
|
| 52 |
+
|
| 53 |
+
---
|
| 54 |
+
|
| 55 |
+
## DEC-CL-0004 · ACCEPTED · 2026-08-15 · Three arms, because a reference changes more than its content
|
| 56 |
+
|
| 57 |
+
**Decision.** First reference experiments capture three arms: actual
|
| 58 |
+
reference `R`, shape-matched control `N`, and no reference `0`.
|
| 59 |
+
`Delta_content = R - N` is the primary object; `N - 0` and `R - 0` are kept
|
| 60 |
+
as diagnostics.
|
| 61 |
+
|
| 62 |
+
**Why.** Verified against the installed ComfyUI: `PackedLayout`
|
| 63 |
+
(`comfy/ldm/minimax/model.py:300`) advances its cursor past every reference
|
| 64 |
+
block before placing the target streams, and references also enter the
|
| 65 |
+
Qwen3-VL presentation path. So a two-arm on/off difference mixes semantic
|
| 66 |
+
content, the presence of a reference, and a moved time origin for the
|
| 67 |
+
target stream.
|
| 68 |
+
|
| 69 |
+
**Consequences.** `N` is a structural control, not a metaphysical zero.
|
| 70 |
+
Several control strata must be tried and the sensitivity reported before
|
| 71 |
+
`R - N` is called canonical. `backends/h3.py:plan_arms` returns the strata
|
| 72 |
+
rather than picking one.
|
| 73 |
+
|
| 74 |
+
---
|
| 75 |
+
|
| 76 |
+
## DEC-CL-0005 · ACCEPTED · 2026-08-15 · Capture the target stream, not only the reference rows
|
| 77 |
+
|
| 78 |
+
**Decision.** v0 captures include target-video DiT state after selected
|
| 79 |
+
blocks. Reference-row state is secondary diagnostic evidence.
|
| 80 |
+
|
| 81 |
+
**Why.** The question is not how the reference is represented. It is how
|
| 82 |
+
the reference changes what gets generated.
|
| 83 |
+
|
| 84 |
+
---
|
| 85 |
+
|
| 86 |
+
## DEC-CL-0006 · ACCEPTED · 2026-08-15 · Ids are computed over an explicit identity payload
|
| 87 |
+
|
| 88 |
+
**Decision.** Every id is a hash of a named identity payload, never of the
|
| 89 |
+
whole record. Adding a field to a manifest must not move ids that already
|
| 90 |
+
exist on disk. If a new field genuinely belongs to a capture's identity, it
|
| 91 |
+
joins `identity()` **and** `PROTOCOL_VERSION` is bumped in the same commit,
|
| 92 |
+
deliberately, with an entry here.
|
| 93 |
+
|
| 94 |
+
**Also decided.** Floats are rounded to 12 significant digits before
|
| 95 |
+
hashing, so a sigma arriving as float32 and the same sigma arriving as
|
| 96 |
+
float64 name one condition rather than two.
|
| 97 |
+
|
| 98 |
+
**Alternatives.** Hashing the whole record — rejected: every bank on disk
|
| 99 |
+
goes stale the next time anyone adds a note field, for no scientific
|
| 100 |
+
reason. Ignoring floats' representation — rejected: an id that disagrees
|
| 101 |
+
about whether two identical conditions are identical is an id that lies.
|
| 102 |
+
|
| 103 |
+
**Consequences.** `tests/test_concept_lab_contracts.py` check 2 asserts
|
| 104 |
+
both directions: identity fields move the id, notes and timestamps and
|
| 105 |
+
status do not.
|
| 106 |
+
|
| 107 |
+
---
|
| 108 |
+
|
| 109 |
+
## DEC-CL-0007 · ACCEPTED · 2026-08-15 · Same id with different content is refused
|
| 110 |
+
|
| 111 |
+
**Decision.** Writing a record under an existing id with different bytes
|
| 112 |
+
raises. Writing identical content is idempotent.
|
| 113 |
+
|
| 114 |
+
**Why.** A capture id names a condition. If the same condition produced
|
| 115 |
+
different bytes, one of the two runs is not what its manifest says, and
|
| 116 |
+
silently keeping the newer one destroys the only evidence that something is
|
| 117 |
+
wrong.
|
| 118 |
+
|
| 119 |
+
---
|
| 120 |
+
|
| 121 |
+
## DEC-CL-0008 · ACCEPTED · 2026-08-15 · Two-root workspace, neither root in the repo
|
| 122 |
+
|
| 123 |
+
**Decision.** Manifests default under the ComfyUI output directory
|
| 124 |
+
(`output/h3_concept`), tensors default to `/mnt/weights/ai/concept-space/tensors`
|
| 125 |
+
on a box that has it, falling back to `<index root>/tensors`. Both are
|
| 126 |
+
overridable per call and by `H3_CONCEPT_INDEX_DIR` / `H3_CONCEPT_TENSOR_DIR`.
|
| 127 |
+
|
| 128 |
+
**Why.** Manifests are for reading and diffing and belong where a human
|
| 129 |
+
browses. Tensors are scientific evidence, can be large, and belong on the
|
| 130 |
+
array. Neither belongs in a public repository, because a concept pack can
|
| 131 |
+
encode a likeness and the failure mode is somebody running `git add -A`.
|
| 132 |
+
|
| 133 |
+
---
|
| 134 |
+
|
| 135 |
+
## DEC-CL-0009 · ACCEPTED · 2026-08-15 · Imported evidence keeps its origin and never merges
|
| 136 |
+
|
| 137 |
+
**Decision.** An imported space lands under `imported/<origin space id>/`
|
| 138 |
+
with the origin stamped on each record. Imported records are never
|
| 139 |
+
re-exported under this space's id. Importing your own export back into
|
| 140 |
+
itself is refused.
|
| 141 |
+
|
| 142 |
+
**Why.** The whole subsystem turns measurements into claims. Somebody
|
| 143 |
+
else's measurement informing your work is good; becoming indistinguishable
|
| 144 |
+
from your own is how a claim loses its scope.
|
| 145 |
+
|
| 146 |
+
---
|
| 147 |
+
|
| 148 |
+
## DEC-CL-0010 · ACCEPTED · 2026-08-15 · Packs are private by default; export names what it held back
|
| 149 |
+
|
| 150 |
+
**Decision.** `ConceptPackManifest.visibility` defaults to `private`.
|
| 151 |
+
Export excludes private packs unless explicitly asked, and reports every
|
| 152 |
+
exclusion with its reason. Source assets are referenced by content hash and
|
| 153 |
+
never copied into a pack.
|
| 154 |
+
|
| 155 |
+
**Consequences.** The MCP surface can export the shareable set and can see
|
| 156 |
+
what was withheld; releasing a private pack is a human keystroke through
|
| 157 |
+
the CLI. A model cannot prove who is asking.
|
| 158 |
+
|
| 159 |
+
---
|
| 160 |
+
|
| 161 |
+
## DEC-CL-0011 · ACCEPTED · 2026-08-15 · Three surfaces, one verb layer
|
| 162 |
+
|
| 163 |
+
**Decision.** `api.py` holds one implementation of every operation and
|
| 164 |
+
returns plain JSON-able dicts. `cli.py`, `mcp.py` and a future `nodes.py`
|
| 165 |
+
are thin clients with no rules in them. No tensors, ComfyUI types or torch
|
| 166 |
+
objects cross the api boundary.
|
| 167 |
+
|
| 168 |
+
**Why.** It is what lets an agent drive the subsystem over a pipe, lets the
|
| 169 |
+
tests run with no GPU, and stops the three surfaces drifting into three
|
| 170 |
+
implementations.
|
| 171 |
+
|
| 172 |
+
**Consequences.** When a verb eventually touches tensors it returns
|
| 173 |
+
references, not handles. If that becomes painful, it gets decided here
|
| 174 |
+
rather than worked around in one surface.
|
| 175 |
+
|
| 176 |
+
---
|
| 177 |
+
|
| 178 |
+
## DEC-CL-0012 · ACCEPTED · 2026-08-15 · Unbuilt verbs refuse and name their task
|
| 179 |
+
|
| 180 |
+
**Decision.** A verb whose machinery does not exist raises
|
| 181 |
+
`NotYetImplemented` carrying the task id that blocks it. It never returns
|
| 182 |
+
an empty or placeholder result.
|
| 183 |
+
|
| 184 |
+
**Why.** The output of this subsystem is evidence. A quiet nothing gets
|
| 185 |
+
recorded and later read as a measurement.
|
| 186 |
+
|
| 187 |
+
---
|
| 188 |
+
|
| 189 |
+
## DEC-CL-0013 · ACCEPTED · 2026-08-15 · No ComfyUI nodes until there is something to run
|
| 190 |
+
|
| 191 |
+
**Decision.** `concept_lab` registers no nodes and the pack's `__init__.py`
|
| 192 |
+
is untouched. Nodes arrive at T12, wrapping proven verbs.
|
| 193 |
+
|
| 194 |
+
**Why.** Existing MAINodes behaviour and defaults must not move for
|
| 195 |
+
additive alpha work. And a half-built research surface in a UI is how ten
|
| 196 |
+
unstable knobs become somebody's saved workflow.
|
| 197 |
+
|
| 198 |
+
**Consequences.** The guarded-import line that adds `concept_lab.nodes` to
|
| 199 |
+
the pack's loader is a one-liner when the time comes; the loader already
|
| 200 |
+
does exactly this for `window_loop`, `window_expand` and `timeline.nodes`.
|
| 201 |
+
|
| 202 |
+
---
|
| 203 |
+
|
| 204 |
+
## DEC-CL-0014 · ACCEPTED · 2026-08-15 · Held-out, adversarial and null splits are not optional
|
| 205 |
+
|
| 206 |
+
**Decision.** A corpus carries four splits and `corpus_status` reports the
|
| 207 |
+
missing ones as warnings. No factor ships on extraction-set scores.
|
| 208 |
+
|
| 209 |
+
**Status of the enforcement.** The warnings exist; a *containment* check
|
| 210 |
+
(no source hash appearing in two splits) does not yet, and is proposed in
|
| 211 |
+
the review brief. Recorded here so the gap is visible rather than assumed
|
| 212 |
+
closed.
|
| 213 |
+
|
| 214 |
+
---
|
| 215 |
+
|
| 216 |
+
## DEC-CL-0015 · ACCEPTED · 2026-08-15 · Process launch is an identity dimension of a capture; arms of one comparison share one launch
|
| 217 |
+
|
| 218 |
+
**Decision.** `FunctionalCaptureManifest` carries a
|
| 219 |
+
`process` fingerprint and `process["launch_id"]` joins `identity()`. The
|
| 220 |
+
launch id is minted once per interpreter (`types.process_fingerprint`).
|
| 221 |
+
Host, pid, start time and Python version stay OUT of identity: a manifest
|
| 222 |
+
must be re-readable on another machine without its id moving.
|
| 223 |
+
|
| 224 |
+
Corollary, and it is a protocol constraint rather than a nicety: R, N and 0
|
| 225 |
+
are three separate forwards, so all arms of one comparison — and the
|
| 226 |
+
replicates that make a delta — must be captured inside one process launch.
|
| 227 |
+
A capture interrupted and resumed in a new process yields an arm 0 that is
|
| 228 |
+
not comparable row-for-row to the R and N from the old one.
|
| 229 |
+
`capture_plan` prints this in `protocol_notes` and stamps the current
|
| 230 |
+
`process_launch_id`.
|
| 231 |
+
|
| 232 |
+
**Why.** On this rig the H3 pipeline is bit-identical WITHIN a process and
|
| 233 |
+
merely close ACROSS processes: per-process kernel autotuning puts a
|
| 234 |
+
~41-48 dB PSNR floor between two cold-process renders at the same seed
|
| 235 |
+
(`.claude/skills/h3-latent-mechanics/SKILL.md` §7 in ComfyUI-ModelCatalog).
|
| 236 |
+
Before this entry, `identity()` excluded tensor hashes while `Space.put`
|
| 237 |
+
compared canonical JSON that includes `TensorRef.sha256`, so two honest
|
| 238 |
+
reruns at one seed in two processes produced the SAME capture id and
|
| 239 |
+
DIFFERENT bytes, and `put` accused the second of not being what its
|
| 240 |
+
manifest said. The first legitimate replicate anyone captured would have
|
| 241 |
+
hit it.
|
| 242 |
+
|
| 243 |
+
**Alternatives.** A mandatory explicit `replicate` set by the caller —
|
| 244 |
+
rejected: it defaults silently to 0, so the failure returns the first time
|
| 245 |
+
someone forgets, which is exactly the population that needs the guard. A
|
| 246 |
+
tolerance-based collision check ("close enough bytes are the same capture")
|
| 247 |
+
— rejected: within a launch the pipeline is exact, so a tolerance hides a
|
| 248 |
+
real defect in the case where the check is most informative.
|
| 249 |
+
|
| 250 |
+
**Consequences.** `PROTOCOL_VERSION` bumps to `concept_lab/0.2.0`
|
| 251 |
+
(DEC-CL-0006's rule: a field entering identity bumps the protocol,
|
| 252 |
+
deliberately). DEC-CL-0007's rationale is amended below. T3 will refuse
|
| 253 |
+
`alignment="exact_rows"` when `FunctionalDeltaManifest.same_process()` is
|
| 254 |
+
False; the two `*_process_launch_id` fields are there for it now, as
|
| 255 |
+
non-identity provenance. The cold-process floor gets MEASURED in E0 rather
|
| 256 |
+
than assumed: the ~41-48 dB figure was taken on decoded renders, not on
|
| 257 |
+
captured block states, and nobody has read the block-state number yet.
|
| 258 |
+
|
| 259 |
+
---
|
| 260 |
+
|
| 261 |
+
## DEC-CL-0007 · AMENDMENT (2026-08-15) · rationale narrowed, decision unchanged
|
| 262 |
+
|
| 263 |
+
The refusal stands: same id with different bytes still raises. The reason
|
| 264 |
+
it is correct is narrower than the original entry said. WITHIN one process
|
| 265 |
+
launch the pipeline is bit-identical, so different bytes under one id mean
|
| 266 |
+
one of the runs is not what its manifest says. ACROSS launches two honest
|
| 267 |
+
reruns differ by kernel autotuning alone, and calling that a defect would
|
| 268 |
+
have refused the first real replicate — which is why the launch id is now
|
| 269 |
+
part of a capture's identity (DEC-CL-0015) and those two runs are two
|
| 270 |
+
captures.
|
| 271 |
+
|
| 272 |
+
`Space.put`'s message now says both halves: "Same id, different bytes.
|
| 273 |
+
Within one process launch this means the id does not describe the
|
| 274 |
+
condition; if these came from different launches the capture is missing its
|
| 275 |
+
process fingerprint (DEC-CL-0015)."
|
| 276 |
+
|
| 277 |
+
---
|
| 278 |
+
|
| 279 |
+
## DEC-CL-0016 · ACCEPTED · 2026-08-15 · Corpora, tap specs and plans are definitions and want version control (amends DEC-CL-0008's rationale, not its default)
|
| 280 |
+
|
| 281 |
+
**Decision.** The default index root does not move (DEC-CL-0008 stands, and
|
| 282 |
+
it is the operator's call). But `Space.status()` reports
|
| 283 |
+
`index_root_tracked` and warns when the index root is not inside a git work
|
| 284 |
+
tree, and `corpus_new`, `corpus_add`, `corpus_status` and `capture_plan`
|
| 285 |
+
carry the same warning. The recommendation is to point
|
| 286 |
+
`H3_CONCEPT_INDEX_DIR` at a directory inside a PRIVATE repository, or to
|
| 287 |
+
copy `corpora/` and `plans/` into one.
|
| 288 |
+
|
| 289 |
+
**Why.** Manifests are not one kind of thing. Captures and deltas are
|
| 290 |
+
RESULTS and can be re-measured from a corpus plus a stack. Corpora, tap
|
| 291 |
+
specs and plans are DEFINITIONS: they encode which contrasts somebody chose
|
| 292 |
+
to shoot, and no tensor reconstructs them. The ancestor program's F30 is
|
| 293 |
+
the paid-for version of this: the definitions behind four findings lived
|
| 294 |
+
only under a gitignored output directory
|
| 295 |
+
(`docs/concept_lab/FABLE_SYNTHESIS_2026-08-15.md` §3.8).
|
| 296 |
+
|
| 297 |
+
**Alternatives.** Moving the default index root into a repo — not taken
|
| 298 |
+
here; DEC-CL-0008 settled the default deliberately and a public node pack
|
| 299 |
+
is the wrong place for it to land by accident. Silence — rejected: the
|
| 300 |
+
loss is invisible until the day someone wants to re-run the experiment.
|
| 301 |
+
|
| 302 |
+
**Consequences.** Status and the corpus verbs warn. Whether the default
|
| 303 |
+
moves is the operator's decision, and this entry does not pre-empt it.
|
| 304 |
+
|
| 305 |
+
---
|
| 306 |
+
|
| 307 |
+
## DEC-CL-0017 · ACCEPTED · 2026-08-15 · Split containment is enforced, and lineage counts as containment (closes DEC-CL-0014's open gap)
|
| 308 |
+
|
| 309 |
+
**Decision.** `ConceptCorpus.containment()` lists every `content_hash` and
|
| 310 |
+
every `source_id` that appears in more than one split, with the splits and
|
| 311 |
+
row ids. `corpus_status` reports it and warns per collision;
|
| 312 |
+
`capture_plan` REFUSES, naming the value and the splits, unless
|
| 313 |
+
`allow_containment=True` is passed — and then the plan carries
|
| 314 |
+
`containment_overridden` so the exception is in the record rather than in
|
| 315 |
+
somebody's memory. `ConceptCorpusRow.source_id` is new and enters the row
|
| 316 |
+
identity.
|
| 317 |
+
|
| 318 |
+
**Why.** DEC-CL-0014 recorded this as a visible gap. A content hash alone
|
| 319 |
+
does not close it: two crops of one take are different bytes and the same
|
| 320 |
+
observation, so held-out means nothing if a lineage id is not checked too.
|
| 321 |
+
Refusal rather than a warning at this one point, because plan time is the
|
| 322 |
+
last moment when the fix is free.
|
| 323 |
+
|
| 324 |
+
**Consequences.** A corpus written before this entry has no `source_id`
|
| 325 |
+
anywhere, which reads as "not recorded" and never as "no collision".
|
| 326 |
+
Absence is skipped, not treated as a shared value.
|
| 327 |
+
|
| 328 |
+
---
|
| 329 |
+
|
| 330 |
+
## DEC-CL-0018 · ACCEPTED · 2026-08-15 · The capture tap ships as two thin alpha nodes now (amends DEC-CL-0013's timing, not its rule)
|
| 331 |
+
|
| 332 |
+
**Decision.** Capture must run INSIDE the ComfyUI process — the states live
|
| 333 |
+
for microseconds inside the DiT block loop and the launch is part of the
|
| 334 |
+
condition (DEC-CL-0015) — so the seam needs a node to exist. Two ship now,
|
| 335 |
+
`MAIConceptCaptureArm` and `MAIConceptCaptureFlush`, under the category
|
| 336 |
+
`MAI/concept (alpha)`. They hold no rules: the Arm node parses its strings
|
| 337 |
+
into a `TapSpec`, reads a stack fingerprint off the live `ModelPatcher`,
|
| 338 |
+
opens a `TapSession` and installs it on a CLONE; the Flush node calls
|
| 339 |
+
`finalize()`. Every refusal comes from `concept_lab/`.
|
| 340 |
+
|
| 341 |
+
DEC-CL-0013's rule stands and is what makes this safe: existing MAINodes
|
| 342 |
+
behaviour and defaults do not move. The registration is the one-liner that
|
| 343 |
+
entry predicted, in the same guarded loader tuple as `window_loop`,
|
| 344 |
+
`window_expand` and `timeline.nodes`, so a broken alpha module cannot take
|
| 345 |
+
the pack down with it.
|
| 346 |
+
|
| 347 |
+
**Alternatives.** A monkeypatched sampler — rejected: `ModelPatcher`'s
|
| 348 |
+
`add_wrapper_with_key` / `set_model_patch_replace` are the sanctioned seam,
|
| 349 |
+
and the true-clock precedent is a monkeypatch we would rather not repeat.
|
| 350 |
+
Waiting for T12 — rejected: nothing else in the subsystem can produce a
|
| 351 |
+
capture, so waiting means the data layer keeps being tested against itself.
|
| 352 |
+
|
| 353 |
+
**Consequences.** E0's gates are run THROUGH these nodes: a baseline render,
|
| 354 |
+
a tap-installed-disabled render and a tap-enabled render at one seed in one
|
| 355 |
+
process must decode to pixel-identical frames. If the pass-through gate
|
| 356 |
+
fails, the nodes are pulled, not patched around. Until those gates run live,
|
| 357 |
+
the nodes are alpha and say so in their DESCRIPTION.
|
| 358 |
+
|
| 359 |
+
**Also decided here (mechanical, same change).** `TapSpec.store` becomes a
|
| 360 |
+
comma-joined subset of `("stats", "frame_mean", "sketch", "full")` and is
|
| 361 |
+
canonicalized to that order, because one recorded forward is expensive and
|
| 362 |
+
the four reductions answer different questions off the same rows. Single
|
| 363 |
+
words stay legal and keep their meaning, so no id already on disk moves.
|
| 364 |
+
|
| 365 |
+
---
|
| 366 |
+
|
| 367 |
+
## DEC-CL-0019 · ACCEPTED · 2026-08-15 · Capture identity is measured at the model boundary, not declared; the tap refuses before its first byte and streams into a partial dir
|
| 368 |
+
|
| 369 |
+
**Decision.** Three parts, one rule.
|
| 370 |
+
|
| 371 |
+
1. A capture's `ConditionVariant` is built by the tap on its FIRST
|
| 372 |
+
diffusion-model call, from what the model was handed: the text-encoder
|
| 373 |
+
output (`sha256` of `context` through float32) as the first
|
| 374 |
+
`ConditioningSource`, then one source per `minimax_payload["refs"]` block
|
| 375 |
+
and one per `keyframes` block, hashed over their latents in packed order.
|
| 376 |
+
`variant.name` stays a LABEL and stays out of identity; `variant.prompt`
|
| 377 |
+
stays `None` (the human text is not reachable at this seam, and the
|
| 378 |
+
encoder output the model consumed is the stronger identity). Noise-aug
|
| 379 |
+
levels, `audio_scale`, the token-tag digest and the payload seed ride in
|
| 380 |
+
the text source's `preprocess`, which `ConditioningSource.identity()`
|
| 381 |
+
already covers, so no schema changed and `PROTOCOL_VERSION` does not move.
|
| 382 |
+
2. The payload seed must equal the seed the Arm node was given. The node
|
| 383 |
+
cannot read the sampler's seed, so it is copied by hand; a mismatch now
|
| 384 |
+
raises naming both numbers instead of mislabelling the capture.
|
| 385 |
+
3. `_ensure_capture_id()` refuses BEFORE the first write when the id is
|
| 386 |
+
already on disk (manifest present, or a non-partial tensor directory),
|
| 387 |
+
naming the existing variant, replicate, launch and paths. Tensors stream
|
| 388 |
+
into `<tensor_dir>/<id>.partial/` while `TensorRef.path` records the final
|
| 389 |
+
`<id>/...`; `finalize()` files the manifest through `api.capture_record`
|
| 390 |
+
and only then renames the directory. `api.capture_exists()` is the stdlib
|
| 391 |
+
door the tap knocks on. `Space.doctor()` reports leftover `*.partial`
|
| 392 |
+
directories as `unfinished_capture` problems.
|
| 393 |
+
|
| 394 |
+
**Why.** Both halves are the reported-vs-consumed family the ancestor
|
| 395 |
+
program kept paying for, and both were caught live rather than reasoned
|
| 396 |
+
about. `MAIConceptCaptureArm` built `ConditionVariant(name, arm)` with
|
| 397 |
+
`sources=()` and `prompt=None`, and `ConditionVariant.identity()` is
|
| 398 |
+
`{arm, sources, prompt}`, so every R cell hashed to one id and every N cell
|
| 399 |
+
to another regardless of which plate the model actually saw: 12 of 15 E0.5
|
| 400 |
+
cells collided. Worse, the same-id refusal lived in `capture_record`, which
|
| 401 |
+
runs at the END, while `h3_tap` had already streamed tensors into
|
| 402 |
+
`<tensor_dir>/<id>/`. A collision therefore overwrote the earlier capture's
|
| 403 |
+
tensors before anything detected it, and E0 replicate 0's tensors were
|
| 404 |
+
destroyed that way. A refusal that fires after the damage is not a refusal.
|
| 405 |
+
|
| 406 |
+
**Alternatives.** Putting `variant_name` into identity — rejected: it makes
|
| 407 |
+
the label load-bearing, so a typo becomes a new condition and a copied node
|
| 408 |
+
silently merges two. Declaring sources from the node's inputs — rejected:
|
| 409 |
+
that is the reported number again, and the plate a node points at is not
|
| 410 |
+
necessarily the latent the model got. Buffering tensors until finalize —
|
| 411 |
+
rejected: a capture's worth of block states is exactly what T4 refuses to
|
| 412 |
+
hold in VRAM. Deleting the partial dir on failure — rejected: it is the
|
| 413 |
+
evidence that something failed.
|
| 414 |
+
|
| 415 |
+
**How to apply.** Arms of one comparison differ by what they feed the model,
|
| 416 |
+
not by what they are called; two arms with different plates now get
|
| 417 |
+
different ids automatically, and two arms with the same plate and the same
|
| 418 |
+
label get the same id and the second one is refused. An honest rerun of a
|
| 419 |
+
condition inside one launch increments `replicate`. A `*.partial` directory
|
| 420 |
+
in the tensor root is wreckage: `space doctor` lists it, the operator
|
| 421 |
+
deletes it, nothing else touches it.
|
| 422 |
+
|
| 423 |
+
**Consequences.** `tests/test_concept_lab_tap.py` grew sections 7-9
|
| 424 |
+
(identity from context and refs, refuse-before-writing and partial/promote,
|
| 425 |
+
render fingerprint). The render fingerprint is evidence and not identity
|
| 426 |
+
(DEC-CL-0006 stands): `MAIConceptCaptureFlush` hashes frame 0 and the whole
|
| 427 |
+
decoded batch as uint8 into the manifest notes, so E0's pass-through gate is
|
| 428 |
+
a hash comparison rather than two humans watching two videos.
|
| 429 |
+
|
| 430 |
+
---
|
| 431 |
+
|
| 432 |
+
## DEC-CL-0020 · ACCEPTED · 2026-08-15 · The first injection is additive at the tap, in frame_mean form, with its controls built in
|
| 433 |
+
|
| 434 |
+
**Decision.** The first injection is additive at the tap, in `frame_mean`
|
| 435 |
+
(and optionally separable frame + patch) form, on the target-video rows,
|
| 436 |
+
with `zero` / `time_shuffle` / `sign_flip` controls built in; it is an
|
| 437 |
+
instrument for E0.5's premise test, not the compiler.
|
| 438 |
+
|
| 439 |
+
Mechanically: `MAIConceptInjectDelta` loads a delta pack (a `.safetensors`
|
| 440 |
+
of `b<block>_s<step>/frame_mean` `[latent_t, hidden]` maps plus an optional
|
| 441 |
+
`patch_mean` `[frame_rows, hidden]`, with a sidecar JSON carrying
|
| 442 |
+
`latent_t`, `frame_rows`, `hidden`, the captured steps and the source
|
| 443 |
+
capture ids) and, at the post-block point where `TapSession._record` reads,
|
| 444 |
+
adds `alpha * d_frame[t]` to every row of latent frame `t` of the video
|
| 445 |
+
segment, in place in the block's OWN output dict. `separable` adds
|
| 446 |
+
`alpha * (d_frame[t] + d_patch[p] - grand)`, the rank-2 map with the shared
|
| 447 |
+
grand mean counted once. `captured_only` fires on the pack's steps;
|
| 448 |
+
`nearest_all` fires every step off the nearest captured entry, ties to the
|
| 449 |
+
earlier one. Step index, layout stash and `cond_or_uncond` handling are the
|
| 450 |
+
tap's, shared rather than copied (`h3_tap.schedule_and_sigma` /
|
| 451 |
+
`step_from_sigma`, `TapSession._layout_signature`).
|
| 452 |
+
|
| 453 |
+
**Why.** The capture side can be right and mean nothing: a delta that
|
| 454 |
+
separates R from N in a bank is a statistic until something is generated
|
| 455 |
+
with it. Additive-at-the-tap is the smallest thing that spends the delta
|
| 456 |
+
where it was measured, so a null result indicts the delta rather than a
|
| 457 |
+
translation layer. The controls ship WITH it rather than after it because
|
| 458 |
+
the two questions the first read has to answer are both control questions:
|
| 459 |
+
`zero` runs every line of the patch with zeroes (if that render moves, the
|
| 460 |
+
plumbing is the effect, not the delta), and `time_shuffle` keeps the
|
| 461 |
+
delta's energy while destroying its timing (if that renders like `none`,
|
| 462 |
+
the timing content is not what is doing the work). A control invented after
|
| 463 |
+
a promising render is a control chosen to survive it.
|
| 464 |
+
|
| 465 |
+
**Alternatives.** Replacing rather than adding (set the rows' frame means to
|
| 466 |
+
the pack's) — rejected for the first read: it destroys the run's own
|
| 467 |
+
content and makes a null result unattributable. Injecting through the
|
| 468 |
+
conditioning (a new ref block) — rejected: that is a different experiment
|
| 469 |
+
about the packer, and it cannot address a chosen block or step. Waiting for
|
| 470 |
+
a compiler that maps a factor to a schedule — rejected: the compiler's
|
| 471 |
+
first input is exactly the measurement this node produces. A distinct patch
|
| 472 |
+
NAME so a tap and an inject could share a block — rejected on the source:
|
| 473 |
+
the block loop reads only `patches_replace["dit"]`
|
| 474 |
+
(comfy/ldm/minimax/model.py:643-655), so a patch under any other name never
|
| 475 |
+
fires, and comfy keeps ONE callable per (name, block, index)
|
| 476 |
+
(model_patcher.py:93-112), so a second install on one block silently
|
| 477 |
+
replaces the first. The injector therefore refuses on block overlap and
|
| 478 |
+
says to put the tap at a later block, where it records the injected state.
|
| 479 |
+
|
| 480 |
+
**How to apply.** One pack, one arm per render, all arms in one launch
|
| 481 |
+
(DEC-CL-0015 binds injections too). Read `none` against `zero` before
|
| 482 |
+
reading `none` against anything else. The geometry guard is
|
| 483 |
+
(latent_t, frame_rows) recomputed off the live layout the way
|
| 484 |
+
`TapSession._write` computes them, checked before any block fires, because
|
| 485 |
+
the manifests carry segments and a position hash rather than the packed
|
| 486 |
+
signature tuple; text_len and audio_t are free to differ, so a delta may be
|
| 487 |
+
spent under a different prompt. `alpha` is the dose and it is not
|
| 488 |
+
calibrated: the report logs the applied norms per (block, step) on the
|
| 489 |
+
ComfyUI console at the last step, and that is the number to quote.
|
| 490 |
+
|
| 491 |
+
**Consequences.** `concept_lab/backends/h3_inject.py` is the second and last
|
| 492 |
+
file in the subsystem that imports torch; `types.py`, `space.py` and `api.py`
|
| 493 |
+
stay stdlib-only. `h3_tap`'s step arithmetic moved to two module functions so
|
| 494 |
+
both sides read one implementation; `TapSession` behaviour is unchanged and
|
| 495 |
+
its suite is untouched and green. `tests/test_concept_lab_inject.py` is the
|
| 496 |
+
new offline suite (pack reading, zero/frame/separable arithmetic, controls,
|
| 497 |
+
step mapping, geometry refusals, cond_only, tap+inject on one clone). The
|
| 498 |
+
node is ALPHA and has NOT been run live: E0.5's premise test is the first
|
| 499 |
+
run, and if the `zero` arm is not bit-identical to an unpatched render the
|
| 500 |
+
node gets pulled rather than patched around.
|
| 501 |
+
|
| 502 |
+
---
|
| 503 |
+
|
| 504 |
+
## DEC-CL-0021 · ACCEPTED · 2026-08-15 · Injection gains a full-state mode and a spatial-shuffle control
|
| 505 |
+
|
| 506 |
+
**Decision.** `MAIConceptInjectDelta` gains `mode='full'`, which adds the
|
| 507 |
+
whole stored block state (`b<block>_s<step>/full`, `[latent_t * frame_rows,
|
| 508 |
+
hidden]` in the video segment's packed row order) row for row instead of a
|
| 509 |
+
reduction, and `control='space_shuffle'`, which permutes the frame_rows axis
|
| 510 |
+
with one seeded permutation shared by every frame. `frame_mean` injection
|
| 511 |
+
was measured to act as a timing-blind global bias (E0.5, 2026-08-15), so the
|
| 512 |
+
object injected must be the object measured to agree.
|
| 513 |
+
|
| 514 |
+
**Why.** E0.5 spent a frame map and got a global push:
|
| 515 |
+
`/mnt/work/ai/apps/ComfyUI-ModelCatalog/docs/concept_lab/E05_RESULTS_2026-08-15.md`.
|
| 516 |
+
A frame map has already averaged the spatial axis away, so the arm has no
|
| 517 |
+
"where" to be wrong about, and `time_shuffle` is the only control the object
|
| 518 |
+
can even fail. The full state is the object the tap files and the object the
|
| 519 |
+
cosines were computed on, and `space_shuffle` is the control that asks
|
| 520 |
+
whether its placement is doing the work. A null on the reduction indicts the
|
| 521 |
+
reduction, not the delta, and that ambiguity is what this closes.
|
| 522 |
+
|
| 523 |
+
**Alternatives.** Reading the reduction harder (more blocks, more steps,
|
| 524 |
+
bigger alpha) — rejected: it varies the dose of a quantity already known to
|
| 525 |
+
be timing-blind. Holding the pack's full states in RAM the way the
|
| 526 |
+
reductions are held — rejected: ~160 MB per (block, step) at production
|
| 527 |
+
geometry. They are read one key at a time on first use and cached only in
|
| 528 |
+
the model's own device and dtype, only for the (block, step) pairs a session
|
| 529 |
+
selected, which is at most `blocks x steps` of them.
|
| 530 |
+
|
| 531 |
+
**How to apply.** `space_shuffle` in `mode='frame'` REFUSES: frame_mean has
|
| 532 |
+
no space axis and the arm would render identically to `none`, which is a
|
| 533 |
+
control that cannot fail and therefore is not one. `mode='full'` refuses at
|
| 534 |
+
session build if any selected (block, step) has no `full` tensor. Sidecar
|
| 535 |
+
keys are unchanged; a pack may carry reductions, full states, or both, and
|
| 536 |
+
the shape guard is `(latent_t, frame_rows, hidden)` as before.
|
| 537 |
+
|
| 538 |
+
**Consequences.** `tests/test_concept_lab_inject.py` grows section 10 (full
|
| 539 |
+
mode: exact add on video rows only, laziness, both shuffles as permutations
|
| 540 |
+
of the axis they name, fp16 packs applied at the model's dtype); the pack
|
| 541 |
+
reader now enumerates keys through `safe_open` rather than `load_file`, so a
|
| 542 |
+
full pack's bytes are never all resident. The node is still ALPHA and the
|
| 543 |
+
full mode has NOT been run live.
|
ComfyUI-MiniMaxH3-Context-Loop/ComfyUI-MAINodes/concept_lab/README.md
ADDED
|
@@ -0,0 +1,224 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Concept Lab (alpha, nothing measured yet)
|
| 2 |
+
|
| 3 |
+
A reusable concept does not have to live in trained weights. It can live in
|
| 4 |
+
a measured functional delta.
|
| 5 |
+
|
| 6 |
+
That is the whole bet. Measure what a piece of conditioning actually does to
|
| 7 |
+
the model, factor that measured effect into reusable components, and compile
|
| 8 |
+
the components back through the model's own conditioning channels.
|
| 9 |
+
|
| 10 |
+
```
|
| 11 |
+
examples -> functional deltas -> factorization -> portable operator
|
| 12 |
+
-> native conditioning
|
| 13 |
+
```
|
| 14 |
+
|
| 15 |
+
The primitive is a difference between two matched forward passes. For model
|
| 16 |
+
state `h` at layer `l`, sampling coordinate `s`, output segment `q`, base
|
| 17 |
+
input `x`, and a conditioning source `C`:
|
| 18 |
+
|
| 19 |
+
```
|
| 20 |
+
Delta(C; x, l, s, q) = h(x | C) - h(x | control(C))
|
| 21 |
+
```
|
| 22 |
+
|
| 23 |
+
The control is doing the real work in that line. On H3 the interesting first
|
| 24 |
+
delta is not "reference versus no reference", because adding a reference
|
| 25 |
+
changes the packed layout, the presentation tokens, and the time origin of
|
| 26 |
+
the target stream all at once. It is "actual reference versus a
|
| 27 |
+
shape-matched control reference", which holds all three fixed and leaves
|
| 28 |
+
only the content.
|
| 29 |
+
|
| 30 |
+
## Status
|
| 31 |
+
|
| 32 |
+
**Alpha, and deliberately narrow.** What exists is the data layer (the
|
| 33 |
+
contracts, the workspace, the verbs, and three surfaces over them) plus the
|
| 34 |
+
H3 capture tap that rides along on a render. What does not exist is anything
|
| 35 |
+
that turns a capture into a factor, or a factor into conditioning.
|
| 36 |
+
|
| 37 |
+
| built | not built |
|
| 38 |
+
|---|---|
|
| 39 |
+
| contracts with deterministic ids (`types.py`) | the delta engine (T3) |
|
| 40 |
+
| the workspace, export and import (`space.py`) | factorization strategies (T6+) |
|
| 41 |
+
| the verb layer (`api.py`) | the compiler (T9) |
|
| 42 |
+
| shell and agent surfaces (`cli.py`, `mcp.py`) | the rest of the node surface (T12) |
|
| 43 |
+
| the H3 backend's verified anchors (`backends/h3.py`) | |
|
| 44 |
+
| capture (T4/T5): tap + nodes built, E0 gates NOT yet run live | |
|
| 45 |
+
|
| 46 |
+
Unbuilt verbs raise and name the task that blocks them. They do not return
|
| 47 |
+
an empty result. In a subsystem whose output is evidence, a quiet nothing is
|
| 48 |
+
worse than a refusal, because the nothing gets written down.
|
| 49 |
+
|
| 50 |
+
Three alpha nodes are registered (`MAI Concept Capture Arm` / `Flush`,
|
| 51 |
+
DEC-CL-0018, and `MAI Concept Inject Delta`, DEC-CL-0020), through the pack's
|
| 52 |
+
guarded loader, so a broken alpha module cannot take the pack down. No
|
| 53 |
+
existing MAINodes behaviour or default moves.
|
| 54 |
+
|
| 55 |
+
## Layering, and why it is strict
|
| 56 |
+
|
| 57 |
+
```
|
| 58 |
+
types.py CONTRACTS ONLY. Stdlib. No torch, no comfy, no H3.
|
| 59 |
+
space.py the workspace: manifests you can read, tensors you can't.
|
| 60 |
+
api.py THE VERBS. One implementation of every operation.
|
| 61 |
+
backends/ everything model-specific. base.py is the seam, h3.py is first.
|
| 62 |
+
strategies/ factorization algorithms (they arrive with their experiments).
|
| 63 |
+
|
| 64 |
+
cli.py shell client over api.py
|
| 65 |
+
mcp.py typed agent client over api.py
|
| 66 |
+
nodes.py ComfyUI client over api.py (the capture pair, alpha)
|
| 67 |
+
```
|
| 68 |
+
|
| 69 |
+
Three surfaces, one authority. A verb that is not in `api.py` does not
|
| 70 |
+
exist, and no surface holds a rule the others cannot see. That is what makes
|
| 71 |
+
a graph, a shell and an agent interchangeable rather than three
|
| 72 |
+
reimplementations that drift.
|
| 73 |
+
|
| 74 |
+
`types.py` and `space.py` stay importable in a process with no torch and no
|
| 75 |
+
comfy in it, and the test suite asserts exactly that in a subprocess. The
|
| 76 |
+
point of a contract is that it is readable and writable by things that are
|
| 77 |
+
not ComfyUI.
|
| 78 |
+
|
| 79 |
+
## The workspace
|
| 80 |
+
|
| 81 |
+
Two roots, because the two kinds of thing want different homes.
|
| 82 |
+
|
| 83 |
+
```
|
| 84 |
+
<index root> small JSON you want to browse and diff
|
| 85 |
+
SPACE.json
|
| 86 |
+
corpora/ captures/ deltas/ packs/ plans/
|
| 87 |
+
imported/<origin space id>/...
|
| 88 |
+
<tensor root> the heavy evidence
|
| 89 |
+
<space id>/...
|
| 90 |
+
```
|
| 91 |
+
|
| 92 |
+
Defaults are `output/h3_concept` next to the renders and `<index
|
| 93 |
+
root>/tensors` for the heavy evidence (point the tensor root at a big disk
|
| 94 |
+
on a box that has one).
|
| 95 |
+
Override with `H3_CONCEPT_INDEX_DIR` and `H3_CONCEPT_TENSOR_DIR`.
|
| 96 |
+
|
| 97 |
+
Neither default is inside the repository. That is not tidiness: a concept
|
| 98 |
+
pack can encode a likeness, and the way that goes wrong is a research
|
| 99 |
+
artifact sitting in a working tree when somebody runs `git add -A`.
|
| 100 |
+
|
| 101 |
+
But the index root holds two different kinds of thing, and only one of them
|
| 102 |
+
is a result. Captures and deltas can be re-measured from a corpus and a
|
| 103 |
+
model stack. Corpora, tap specs and plans are **definitions**: they are the
|
| 104 |
+
record of which contrasts somebody chose, and no tensor reconstructs them.
|
| 105 |
+
So `space status` reports `index_root_tracked` and the corpus verbs warn
|
| 106 |
+
when the index root is not inside a git work tree. The fix is one
|
| 107 |
+
environment variable pointing at a **private** repository (or a copy of
|
| 108 |
+
`corpora/` and `plans/` into one):
|
| 109 |
+
|
| 110 |
+
```
|
| 111 |
+
export H3_CONCEPT_INDEX_DIR=~/research-private/concept_space
|
| 112 |
+
```
|
| 113 |
+
|
| 114 |
+
The default does not move (DEC-CL-0008 settled it); the warning exists
|
| 115 |
+
because losing the definitions is invisible until the day you want to run
|
| 116 |
+
the experiment again (DEC-CL-0016).
|
| 117 |
+
|
| 118 |
+
### Export and import
|
| 119 |
+
|
| 120 |
+
An export bundles manifests and their tensors into one zip regardless of
|
| 121 |
+
which root they sat in. An import lands somebody else's space **beside**
|
| 122 |
+
yours rather than in it:
|
| 123 |
+
|
| 124 |
+
```
|
| 125 |
+
<index root>/imported/<their space id>/...
|
| 126 |
+
<tensor root>/<their space id>/...
|
| 127 |
+
```
|
| 128 |
+
|
| 129 |
+
with the origin stamped on every record. Their numbers can inform your work
|
| 130 |
+
and can never be mistaken for your own measurements. Packs marked private
|
| 131 |
+
are held back from an export unless you ask for them in as many words, and
|
| 132 |
+
every exclusion is reported with its reason — nothing is dropped silently.
|
| 133 |
+
|
| 134 |
+
## Using it
|
| 135 |
+
|
| 136 |
+
From a shell:
|
| 137 |
+
|
| 138 |
+
```
|
| 139 |
+
python -m concept_lab.cli space init --label mine
|
| 140 |
+
python -m concept_lab.cli corpus new wave
|
| 141 |
+
python -m concept_lab.cli corpus add wave --kind video --ref alice_wave.mp4 \
|
| 142 |
+
--label subject=alice --label motion=wave --split extract --group alice
|
| 143 |
+
python -m concept_lab.cli corpus confounds wave --target motion
|
| 144 |
+
python -m concept_lab.cli capture plan wave
|
| 145 |
+
python -m concept_lab.cli space export mine
|
| 146 |
+
```
|
| 147 |
+
|
| 148 |
+
Every command prints JSON, so a pipeline and an agent read the same bytes.
|
| 149 |
+
|
| 150 |
+
From an agent, over MCP:
|
| 151 |
+
|
| 152 |
+
```
|
| 153 |
+
<venv>/bin/python -m concept_lab.mcp
|
| 154 |
+
```
|
| 155 |
+
|
| 156 |
+
Same verbs, typed. Two things are deliberately absent from that surface:
|
| 157 |
+
anything that runs a model (capture costs GPU time on a shared box, so the
|
| 158 |
+
agent plans and a human launches), and exporting a private pack (a model
|
| 159 |
+
cannot prove who is asking, and export is a publishing act). An agent that
|
| 160 |
+
can measure everything and publish nothing is the right shape here.
|
| 161 |
+
|
| 162 |
+
## The two checks that are cheap and worth running first
|
| 163 |
+
|
| 164 |
+
**`corpus confounds`** asks which other labels move exactly with the one you
|
| 165 |
+
care about. If every waving clip also smiles, then a "wave" factor is a
|
| 166 |
+
wave-and-smile factor, and no amount of solver will tell you that. The fix
|
| 167 |
+
is another clip, not another algorithm. This costs nothing and answers a
|
| 168 |
+
question that is expensive to answer after a capture.
|
| 169 |
+
|
| 170 |
+
**`capture plan`** costs a three-arm capture without running it: forward
|
| 171 |
+
count, tensors written, and every protocol gap the corpus carries. It says
|
| 172 |
+
plainly that it measured nothing.
|
| 173 |
+
|
| 174 |
+
## Rules this subsystem tries to make structural
|
| 175 |
+
|
| 176 |
+
Each of these is here because a previous research program paid for it.
|
| 177 |
+
|
| 178 |
+
- **A delta needs a matched control.** Not "the intervention off" — a
|
| 179 |
+
control that holds every structural nuisance fixed.
|
| 180 |
+
- **Same id, different bytes is an error.** If one condition produced two
|
| 181 |
+
different results, one of the runs is not what its manifest says, and
|
| 182 |
+
keeping the newer one destroys the evidence that something is wrong.
|
| 183 |
+
- **Extraction success is not a result.** Splits are `extract`, `validate`,
|
| 184 |
+
`adversarial`, `null`, and a factor that has only seen the first one is a
|
| 185 |
+
hypothesis.
|
| 186 |
+
- **Depth and time are coordinates, not metadata.** A factor collapsed to
|
| 187 |
+
one global vector before anyone measured the layer and timestep profile
|
| 188 |
+
has thrown away the answer.
|
| 189 |
+
- **Record the stack, never infer it from a filename.** A capture without
|
| 190 |
+
its condition is a number without a condition.
|
| 191 |
+
- **A diagnostic is not a behavioural result.** A decomposition is a
|
| 192 |
+
hypothesis until it predicts output the model was not fitted on.
|
| 193 |
+
- **Nothing gets promoted quietly.** Every claim carries one of PROPOSED,
|
| 194 |
+
MEASURED, SCOPED, FAILED, RETRACTED, SUPERSEDED, OPEN.
|
| 195 |
+
|
| 196 |
+
## Tests
|
| 197 |
+
|
| 198 |
+
```
|
| 199 |
+
python tests/test_concept_lab_contracts.py
|
| 200 |
+
python tests/test_concept_lab_tap.py # torch, no comfy, no GPU
|
| 201 |
+
```
|
| 202 |
+
|
| 203 |
+
Synthetic. No GPU, no models, no renders, no torch. Thirteen check groups
|
| 204 |
+
covering the import boundary, id determinism (including which fields are
|
| 205 |
+
*not* allowed to move an id, and the process launch that now belongs to
|
| 206 |
+
one), roundtrips, refusals, the collision check, confound detection,
|
| 207 |
+
export/import provenance, the H3 anchors read from the installed ComfyUI
|
| 208 |
+
source, unbuilt verbs refusing, the CLI surface, the version-control
|
| 209 |
+
warning, tensor hashing in `doctor`, and the two guards that have to fire:
|
| 210 |
+
split containment and the shape-matched control arm.
|
| 211 |
+
|
| 212 |
+
The tap suite is the second one and needs torch (it never imports ComfyUI):
|
| 213 |
+
six check groups over pass-through exactness, step selection, what each
|
| 214 |
+
store writes, the manifest and its id, the refusals, and the disabled arm
|
| 215 |
+
writing nothing. What it cannot check offline is the only thing that
|
| 216 |
+
matters in the end, which is why E0 exists: that a real render is
|
| 217 |
+
pixel-identical with the tap installed.
|
| 218 |
+
|
| 219 |
+
## Reading order for the design
|
| 220 |
+
|
| 221 |
+
`__init__.py` for the layering, `types.py` for the id rules and why they are
|
| 222 |
+
shaped that way, `backends/h3.py` for the H3 anchors with their source
|
| 223 |
+
citations, `DECISIONS.md` for what has been settled and should not be
|
| 224 |
+
quietly relitigated.
|
ComfyUI-MiniMaxH3-Context-Loop/ComfyUI-MAINodes/concept_lab/__init__.py
ADDED
|
@@ -0,0 +1,55 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Concept Lab: measure what conditioning does to the model, factor that
|
| 2 |
+
effect into reusable pieces, and compile the pieces back through the
|
| 3 |
+
model's own conditioning channels.
|
| 4 |
+
|
| 5 |
+
The thesis in one line: a reusable concept does not have to live in trained
|
| 6 |
+
weights. It can live in a measured functional delta.
|
| 7 |
+
|
| 8 |
+
examples -> functional deltas -> factorization -> portable operator
|
| 9 |
+
-> native conditioning
|
| 10 |
+
|
| 11 |
+
Layering, and it is load-bearing (the same discipline timeline/ uses):
|
| 12 |
+
|
| 13 |
+
concept_lab/types.py CONTRACTS ONLY. Stdlib. No torch, no comfy,
|
| 14 |
+
no H3. Dataclasses, canonical JSON, and the
|
| 15 |
+
deterministic ids everything else keys on.
|
| 16 |
+
concept_lab/space.py the workspace: a manifest root you can see, a
|
| 17 |
+
tensor root that holds the heavy evidence, and
|
| 18 |
+
the export/import that moves a space between
|
| 19 |
+
machines with provenance intact.
|
| 20 |
+
concept_lab/api.py THE VERBS. One implementation of every
|
| 21 |
+
operation. Returns plain JSON-able dicts.
|
| 22 |
+
concept_lab/backends/ everything model-specific. base.py is the
|
| 23 |
+
seam; h3.py is the first backend.
|
| 24 |
+
concept_lab/strategies/ factorization algorithms (arrive with their
|
| 25 |
+
experiments, not before).
|
| 26 |
+
|
| 27 |
+
concept_lab/cli.py shell client over api.py
|
| 28 |
+
concept_lab/mcp.py typed agent client over api.py
|
| 29 |
+
concept_lab/nodes.py ComfyUI client over api.py: the two alpha
|
| 30 |
+
capture nodes, which exist because a capture
|
| 31 |
+
can only be taken inside the ComfyUI process
|
| 32 |
+
(DEC-CL-0018).
|
| 33 |
+
|
| 34 |
+
Three surfaces, one authority. A verb that is not in api.py does not
|
| 35 |
+
exist, and no surface is allowed to hold a rule the others cannot see.
|
| 36 |
+
That is what makes a graph, a shell and an agent interchangeable here.
|
| 37 |
+
|
| 38 |
+
This file imports NOTHING on purpose. `concept_lab.types` and
|
| 39 |
+
`concept_lab.space` must stay importable in a process with no torch and no
|
| 40 |
+
comfy in it (tests/test_concept_lab_contracts.py asserts exactly that in a
|
| 41 |
+
subprocess), because the whole point of the contracts is that they are
|
| 42 |
+
readable and writable by things that are not ComfyUI.
|
| 43 |
+
|
| 44 |
+
Status vocabulary, used on every claim in this subsystem:
|
| 45 |
+
|
| 46 |
+
PROPOSED idea, not measured
|
| 47 |
+
MEASURED measured under a named condition
|
| 48 |
+
SCOPED measured, valid only on a named distribution
|
| 49 |
+
FAILED tested, did not pass its gate
|
| 50 |
+
RETRACTED an earlier reading was wrong
|
| 51 |
+
SUPERSEDED replaced by a better formulation
|
| 52 |
+
OPEN important unknown, no adequate test yet
|
| 53 |
+
|
| 54 |
+
Nothing gets promoted quietly.
|
| 55 |
+
"""
|