ComfyUI-MiniMaxH3-Context-Loop/docs/SCHEDULED_REFERENCES.md

Scheduled references

Use Scheduled Ref2VA when a picture, video, or audio reference should apply to selected scenes rather than every recursive iteration.

Scheduled Picture Ref ─→ Scheduled Video Ref ─→ Scheduled Audio Ref
                                                    ↓
Current Shot ─ prompt, clip_index, clip_count ─→ Scheduled Ref2VA
Current Shot ─ width, height, length ───────────→ Scheduled Ref2VA
CLIP + video VAE + audio VAE ──────────────────→ Scheduled Ref2VA

Scene selectors

Value	Active scenes
blank, all, or *	every scene
3	scene 3
1:5	scenes 1 through 5
1,3,5:8	scenes 1, 3, and 5 through 8

Disjoint selectors are safe for references because they do not skip the loop's motion dependency.

Aliases and native labels

Aliases such as @hero_face, @performance, and @voice are optional authoring conveniences. The wrapper replaces active aliases with the native H3 label assigned in that scene. It never inserts definitions or rewrites semantic prompt text.

Native numbering is compact and independent by media type:

1. active pictures become <Picture 1>, <Picture 2>, and so on;
2. active videos receive independent <Video N> labels;
3. paired video soundtracks are presented immediately before their video and consume <Audio N> labels;
4. standalone audio continues the independent audio numbering.

If an earlier picture becomes inactive, a later @hero may compile to <Picture 1>. This is why aliases are safer for changing schedules. Core workflows may continue to use native labels directly.

Write the meaning of every reference in the Plan prompt itself:

subject_definitions:
<Subject 1> uses @hero_face for facial identity and @performance for movement.

Compliance policies

• strict compiles valid active aliases and stops on unresolved/inactive tags or invalid scheduled media.
• soft compiles valid aliases, leaves unresolved prompt tags intact, logs a warning, and continues.
• disabled passes the prompt and all @tags through unchanged. Scheduler validation becomes warning-only; unusable media is omitted and excess slots are capped to stock H3 limits.

Errors outside the scheduler remain real errors, including invalid model/VAE wiring, sampling failures, continuation tensor incompatibility, and checkpoint integrity failures.

Preview and insertion

The Scene Prompt Editor's @ Reference tray shows only sources active for the selected scene. Hover a loader-backed tag to preview image, video, or audio, then click to insert it. Audio never autoplays.

Dedicated semantic picture anchors

For new workflows, register Qwen-only stills with Semantic Picture Anchor, chain those small nodes, and finish them with one Semantic Anchor Bundle. Connect the Bundle's references output to Tagged Ref2VA's references, Plan Studio's tagged_references, and Run Manager's tagged_references. That one reference carrier contains both native @references and Qwen-only #anchors, but semantic images remain excluded from native H3 capacity. The Bundle owns the shared presentation size and mode, so individual anchor nodes store their original picture and scale it only when a scene actually calls its #tag.

The prompt-driven Tagged route accepts both #picture and #picture[2.50s]. Bare #picture presents the matching Semantic Picture Anchor to Qwen as one untimed image. Adding [2.50s] gives that same Qwen-only visual an approximate scene-local placement. Neither form adds a native VAE reference. Timed anchors are useful for reinforcing a replacement character's identity later in a shot:

<Subject 1> is the replacement performer defined by @replacement.
#replacement[0.00s] #replacement[2.50s] #replacement[4.75s]

@replacement, bare #replacement, and #replacement[...] have distinct jobs and may be used together, including from different source nodes. The @ form is a native Ref2VA picture and counts toward H3's nine active Picture slots. The # form is Qwen-only semantic reinforcement and never counts toward native picture, video, or audio limits. A supplied time must fall inside the current scene. Timing is optional and approximate—not an exact frame, pose, spatial mask, motion control, or continuation seam.

Tagged Picture nodes remain accepted as #tags for existing workflows, but the dedicated semantic chain is preferred: inactive semantic images can stay in the project without crowding native reference accounting, while the Bundle emits one combined incremental fingerprint for Plan resume safety. Start with two or three sparse anchors—too many repeated pictures can resist the source video's changing pose.

Picture storyboard mode

Set Semantic Anchor Bundle's mode to picture_storyboard to compile timed #picture[time] syntax differently. Each distinct tagged image is added once as a separate Qwen-only <Picture N>, and the compiler adds an approximate scene-relative timing sentence for every requested time. No image is VAE encoded, spatially fused, or inserted into a fixed generated frame. Bare #picture is already an untimed <Picture N> in either mode.

This mode gives Qwen a high-detail visual shot plan while H3 remains free to invent motion and transitions. timestamped_video remains the default and is better for sparse temporal reinforcement. Storyboard mode is useful for a sequence of compositions or appearances, but its textual timing is softer.

The Bundle's semantic_anchor_size accepts 384, 512, 768, 1024, 1280, or source. Higher values preserve more face, wardrobe, prop, and environment detail at the cost of longer Qwen conditioning and greater VRAM/runtime pressure. Prefer 512 or 768 generally, 1024 for important detailed anchors, and 1280 for a small number of critical stills.

With core Ref2VA, the tray previews media and inserts native labels. With core Image to Video it exposes first and last frames as <Picture N> according to the active keyframes.

External RefMod experiment

Tagged Ref2VA can expose the exact active visual sources selected for the current scene to ComfyUI-MiniMaxH3Mod. Install that node pack, then set Tagged Ref2VA's conditioning_backend to external_refmod and wire:

Tagged Ref2VA refmod_sources ─→ Extract H3 RefMod refs_bundle
Tagged Ref2VA positive ───────→ Apply H3 RefMod conditioning
Extract H3 RefMod mods ───────→ Apply H3 RefMod mods
Apply H3 RefMod conditioning ─→ H3 Chain Context conditioning
Tagged Ref2VA latent ─────────→ H3 Chain Context latent

In this mode Tagged Ref2VA still selects scene-local @tags, resolves lazy project images, and slices sequential reference videos. It describes those RefMods in text using their readable tag names, creates base conditioning and a matching empty AV latent through core MiniMaxH3ImageToVideo, and skips stock Ref2VA. refmod_sources uses the external pack's existing H3_REF_LIST contract, so no direct Python import or patched external node is required. Native behavior remains the default.

Semantic #tag and #tag[timestamp] visuals use a hybrid path. Ordinary active @visuals remain exclusively in RefMod, while only the active anchor images are presented to Qwen and numbered in their own <Picture N> or <Video N> namespace. Bare tags are Pictures; timed tags are Video checkpoints in timestamped_video mode or Picture cues in storyboard mode. This preserves the supplied timing without paying Qwen's media-token cost for every ordinary reference again. An asset intentionally used as both @tag and either # form participates in both paths.

The external experiment remains unable to carry reference audio. A scene with active standalone tagged audio or paired reference-video audio stops with a clear error instead of silently losing it. Source Timeline/final audio managed outside Tagged Ref2VA remains unaffected.

The reference registry and backend choice participate in the Plan fingerprint, but the external Extract/Apply widgets and resulting mod bytes do not. Use a new run name when changing RefMod mode, resolution, pooling, frame retention, token cap, or strength. Native deferred-upscale reference caching is disabled for external_refmod; the deferred pass does not yet reconstruct external mods automatically.

Resume fingerprints

For static loaders, connect schedule_fingerprint to Plan's generation_fingerprint. Changing media bytes, tags, or selectors then invalidates incompatible checkpoints.

Do not create a fingerprint cycle when a scheduled entry consumes Current Shot, such as source_audio_slice. Source-track mode already fingerprints the full source waveform at Loop Start.

Automatic deferred-upscale cache

Tagged and Scheduled Ref2VA enable cache_for_upscale by default. The wrapper stores native H3 picture/video/audio reference latents, original picture masters for larger pass-2 canvases, and the resized Qwen image or 2 fps video presentation.

New caches use V3 per-reference tensor objects, not one large safetensors bundle per scene. Each original, presentation tensor, and encoded latent is stored independently, addressed by its exact dtype, shape, and bytes. Different scenes, prompts, reference ordering, and projects reuse identical objects. An original picture can be shared even when its resized presentations and encoded variants differ. Different VAE results, resolutions, video slices, and audio payloads never share an object unless the resulting tensors are identical.

Small scene JSON manifests retain the registry fingerprint, prompt, frame contract, canvas, reference sizing, roles, ordering, timestamps, and object links. Their signatures include the tensor-object map. This deduplicates disk storage; it does not skip VAE encoding based on guessed model identity, or spill the upscaler's working frames/model memory to disk.

Ref2VA writes objects to output/h3_reference_cache/objects/ and scene manifests to output/h3_reference_cache/<fingerprint>/ because it does not require a Plan or run_name input. Segment Save then hard-links each unique object (or copies it once per project when linking is unavailable) and publishes local metadata:

output/h3_chains/<run-name>/reference_cache/
  scene_0001.<scene-contract>.json
  scene_0002.<scene-contract>.json
  objects/<tensor-content-hash>.safetensors

The standalone deferred-upscale workflow reads generation_fingerprint from the selected checkpoint branch and restores the matching scene automatically. It does not need the source Plan or any original reference-media connection. The checkpoint points only to the run-local descriptor, making the run folder self-contained for copying, backup, and later upscale. Object maps and tensor files are SHA-256 verified before use; relocated projects do not need the shared store. Do not delete individual objects: multiple scenes can reference them. Disable cache_for_upscale only when the extra reference encode and disk cache are not wanted. Existing checkpoints made before this feature have no cache; the upscale conditioning node can either fall back to text-only conditioning or raise an explicit error.

V1/V2 bundled caches remain readable and are not automatically rewritten or deleted merely by installing the update. New writes use V3. Use the converter below to opt existing bundles into conversion and success-gated retirement. There is no automatic garbage collection of the new shared tensor objects.

Checkpoints with exact cache links made during the earlier global-cache-only implementation are adopted automatically on their next complete-branch selection in Checkpoint Manager. The verified tensor is hard-linked or copied into the run-local cache; the original output/h3_reference_cache/ object is deliberately left intact. No source render or manual file move is required. Workflow-local and chapter-only selection are read-only and do not migrate caches. Older checkpoints without an exact cache link still use the fingerprint-based shared-cache lookup.

Converting existing bundles

Update every ComfyUI instance sharing the output directory before enabling conversion. Old node versions cannot follow V3 redirects after retirement. Run this with a Python environment containing PyTorch and safetensors; no model load, VAE re-encoding, or generation is required to convert:

python tools/convert_reference_caches.py --output-root /path/to/ComfyUI/output --all --dry-run
python tools/convert_reference_caches.py --output-root /path/to/ComfyUI/output --all --apply

For narrower scope, replace --all with --run <run_name> (project-local caches only), or --metadata <relative/path/to/scene_cache.json> (repeatable). Dry-run is the default and writes nothing. --apply is resumable: completed conversions are verified and reused, and incomplete conversions retain their old bundle. Conversion needs temporary headroom for unique objects until successful use.

The converter retains the original scene_….json and large bundle, and writes a neighbouring scene_….converted.json with links to the per-tensor objects. Tensor values, dtypes, reference roles, prompt, order, and timestamps are preserved exactly; V1 caches do not gain original images they never contained. Immutable checkpoint JSON and source-manifest identities are not rewritten, so existing upscale profiles can still resume.

Automatic retirement happens after a successful scene save using converted conditioning, not during conversion, cache discovery, preview, or conditioning encoding. The H3 generation/upscale saver requires a same-execution use receipt on the conditioning path of a supported sampler leading to its saved pixels. Built-in SamplerCustom, SamplerCustomAdvanced, KSampler/KSamplerAdvanced, H3 USDU UltimateSDUpscale*, and the CAT USDU video adapter are recognized. Unknown wrappers, cached conditioning from a previous execution, failed or cancelled renders, and unprovable branches keep the legacy bundle.

After verifying the committed output, the complete converted tensors against the original bundle, and other cache JSON consumers of that pathname, the saver deletes only that legacy safetensors file. It retains the original JSON, converted manifest, tensor objects and a small .retired.json receipt so old checkpoint addresses continue to resolve. Other hard links are left intact; unlinking one path may not release physical space until its remaining links are also retired. The saver reports retirement in its status and log. Cleanup failure keeps the bundle and never invalidates a successfully saved render.

Patch priority

If an older compatible H3 Motion Context copy wins process load order, insert MiniMax H3 Patch Priority between Ref2VA/I2V conditioning and Context Loop Context. It passes conditioning unchanged while claiming only the recognized shared patch family. Known H3-Multishot and SolAttn hooks remain active; unknown wrappers produce a clear error rather than being overwritten.