docs/attention.md

Per-layer attention config LoRAs

On an RTX 5090, dense attention is ~2.4 s of every ~3.8 s step at 960×544 (5 s clip), so it, not the weights, decides speed. ComfyUI can pick an attention kernel per layer from a weightless "config LoRA":

• kitchen INT8 attention: ~2.5× faster, ~1% error per layer;
• kitchen SOL block-sparse attention: each 64-token query block attends a routed subset of key blocks exactly and the rest through one pooled term; tau (in standard deviations of a proxy score) sets the sparsity — 4–14× faster, 11–36% error per layer.

comfyanonymous published a test LoRA that sets 23 layers to INT8 and 27 to SOL. We searched for better allocations.

How the allocations were found

1. Profile the exact stack (INT8 ConvRot + dmad 4-step LoRA, 4 steps, 960×544): at every DiT attention call the dense result is kept and every candidate kernel (INT8, SOL at six tau) also runs on the same q/k/v, recording kernel time and error separately on the video rows and the audio rows of H3's packed sequence.
2. Search with an exact knapsack: least video error + w × audio error for an attention-time budget.
3. Verify each allocation end to end in ComfyUI: one-step output error against dense attention on held-out prompts, video and audio separately.

Results on the tuned stack (INT8 ConvRot + dmad 4-step)

Allocation	Attention speed-up	Video error	Audio error	Layers (INT8 / SOL)
comfyanonymous test LoRA (fast_sol_kitchen_attention)	3.32×	18.4%	13.6%	23 / 27
ours, same speed, video-row objective	3.24×	14.5%	13.8%	32 / 18
ours, same speed, audio weight 1	3.25×	14.6%	13.3%	32 / 18
balanced: same speed, audio weight 3	3.24×	15.1%	11.6%	31 / 19
ours, same speed, audio weight 10	3.25×	15.7%	11.7%	32 / 18
quality: x3	2.94×	12.2%	10.7%	39 / 11
ours, x3, audio weight 1	2.96×	12.5%	10.7%	39 / 11
ours, x3, audio weight 3	2.96×	12.5%	10.7%	39 / 11
ours, x3, audio weight 10	2.96×	14.8%	9.9%	39 / 11
ours, x3.5	3.40×	16.0%	17.5%	29 / 21
ours, x3.5, audio weight 1	3.41×	15.9%	15.0%	28 / 22
ours, x3.5, audio weight 3	3.42×	17.0%	14.3%	28 / 22
fast: x3.5, audio weight 10	3.42×	17.1%	13.2%	28 / 22

Measured speed on the same stack (RTX 5090, 4 timed generations each):

Attention LoRA	s/step	Time per clip (4 steps)
none (dense attention)	3.83	25 s
comfyanonymous test LoRA	2.16	21 s
balanced	2.16	20 s
quality	2.24	19 s

At the same real speed as the reference LoRA, balanced has ~18% less video error and ~15% less audio error. Quality gives up ~4% of the step time for ~35% less video and ~20% less audio error.

Without the 4-step LoRA (base INT8 ConvRot, 20 steps)

The same LoRAs, unchanged, on a stack they were not tuned on:

Attention LoRA	Attention speed-up	Video error	Audio error
comfyanonymous test LoRA	3.32×	15.1%	12.5%
fast (dmad 4-step)	3.42×	15.4%	11.4%
balanced	3.24×	13.5%	10.4%
quality	2.94×	11.2%	10.6%

Balanced and quality stay better than the reference on both video and audio without the distillation LoRA; the fast allocation is only worth it on the 4-step stack.

Files

File	Allocation	Use with
loras/minimax_h3_attn_balanced.safetensors	same speed as the reference, less error	any INT8/NVFP4 H3 stack, 4 or 20 steps
loras/minimax_h3_attn_quality.safetensors	~4% slower, clearly less error	any INT8/NVFP4 H3 stack
loras/minimax_h3_attn_fast_dmad4step.safetensors	slightly faster than the reference, less audio error, about the same video error	INT8 ConvRot + dmad 4-step

Load with Load LoRA (Model Only) at strength 1.0, after any weight LoRA. They hold no weights, so they also keep a calibrated NVFP4 checkpoint's scales. Disable attention-override nodes, or the configs are ignored.

Experimental: audio/text protection (needs a ComfyUI patch, not yet upstream)

ComfyUI's config passes only tau to SOL. A small patch (ComfyUI PR #16994) lets H3 publish its packed row spans so SOL keeps every query's text/conditioning/audio keys exact ("sink_keys": "prefix") and the audio queries fully exact ("sink_queries": "audio"): a few hundred of ~19.5K rows. It also passes through the other SOL options and, separately, allows per-step configs ("sigma_range"). Same stack, RTX 5090:

Attention LoRA	s/step	Video error	Audio error
comfyanonymous test LoRA	2.15	18.4%	13.6%
sinks, same speed	2.17	10.9%	7.5%
sinks, fast (4× attention kernels; ~6% per step)	2.03	13.7%	8.6%

Removing the sinks from the same allocations raises the error to 14.1% / 11.4% and 17.5% / 15.5%: the search and the sinks each account for roughly half of the gain over the reference. Per-step configs added at most ~8% on top and did worse at 4×.

To test, check out the PR (gh pr checkout 16994 in a ComfyUI clone) and load a sink LoRA as above, after the dmad 4-step LoRA:

File	Allocation
loras/sinks/minimax_h3_attn_sinks_same_speed_dmad4step.safetensors	reference speed, ~40% less error
loras/sinks/minimax_h3_attn_sinks_fast_dmad4step.safetensors	~6% faster per step than the reference, less error
loras/sinks/minimax_h3_fast_sol_kitchen_attention_sinks.safetensors	comfyanonymous' test LoRA with sinks added, otherwise unchanged (see below)

comfyanonymous' LoRA + sinks. The cleanest test of the patch: the original fast_sol_kitchen_attention_test_lora by comfyanonymous with sink_keys/sink_queries added to its 27 SOL layers and nothing else changed (same layers, same tau, same speed). Credit for the allocation is theirs; it is redistributed here unmodified apart from the sink keys, for testing the PR.

Attention LoRA	Attention speed-up	Video error	Audio error
comfyanonymous test LoRA	3.31×	18.4%	13.6%
same LoRA + sinks	3.31×	12.3%	8.1%

Without the 4-step LoRA (base INT8 ConvRot, 20 steps), unchanged LoRAs:

Attention LoRA	Attention speed-up	Video error	Audio error
comfyanonymous test LoRA	3.31×	15.1%	12.5%
same LoRA + sinks	3.31×	12.3%	7.1%
sinks, same speed (dmad-tuned)	3.22×	9.7%	7.2%
sinks, fast (dmad-tuned)	3.98×	12.9%	8.1%

Errors are per model call. Final clips from any attention LoRA diverge from dense attention about equally, so judge them by eye and ear. Two blind tests (3 prompts × 2 seeds each, 8-step workflow at 1280×736): our same-speed sink LoRA vs the reference 4–2; the reference with vs without sinks 3–3 on concert prompts and 5–1 on fine-texture prompts (lace, ribbed swimsuit, mesh), 12–6 for sinks overall. Audio was indistinguishable throughout; the wins were coherence in busy scenes (broken motion without sinks) and fine detail (hands, fabric noise). Small sample, consistent direction.

On stock ComfyUI these load and run, but the sink keys are ignored and the error is the "sinks removed" figures above: use the LoRAs in the table further up instead.

Ref2VA at 768×1344, 124 frames (sinks; needs the PR above)

Tuned on the ref2va pruned INT8 ConvRot transformer + dmad 4-step full-critic LoRA, dpmpp_sde_gpu / beta, 8 steps, one reference image. At this size dense attention is ~78% of each model call, so faster allocations pay off more than at 960×544. All files measured in one run (2 held-out prompts, same seed; DiT time per clip on an RTX 5090 with DynamicVRAM, as in a ComfyUI server):

Attention LoRA	Time per clip	Speed-up	Video error	Audio error
none (dense)	198 s	1.00×	–	–
comfyanonymous test LoRA	91 s	2.19×	16.4%	24.6%
quality (ref2va)	97 s	2.05×	9.2%	10.2%
balanced (ref2va)	90 s	2.20×	11.1%	11.3%
sinks fast (dmad 4-step, 960×544 tune)	83 s	2.40×	14.4%	14.7%
x6 (ref2va)	72 s	2.75×	22.2%	19.3%

At the reference LoRA's speed, balanced has a third less video error and half the audio error. Blind tests on two prompts: dense, fast, balanced and quality could not be told apart; between fast, x6 and an 8× allocation, only the 8× one was picked out as different, in both clips. Past ~6× the rest of the model dominates the step, so x6 is the fast end.

The same LoRAs on the SMD 8-step LoRA

SMD Ref2VA 8-step at its own settings (euler, its ManualSigmas, shift 12/3, 1344×768, two references) gives a different stack: different LoRA, sampler, schedule and reference count. We tuned it from scratch and compared:

Which layers tolerate sparse attention barely changes between the two stacks (rank correlation 0.93–0.98 per kernel), so a fresh SMD tune found nothing better than x6 as published: at the same speed it measured 63.9% / 23.8% final video / audio error against 67.2% / 26.3% for the SMD tune (2 prompts). Use the ref2va LoRAs on SMD as they are. Measured on SMD, DiT time per clip:

Attention LoRA	Time per clip	Speed-up
none (dense)	102 s	1.00×
x6 (ref2va)	36 s	2.81×

"Final error" is how far the finished latent lands from dense attention on the same seed. A few-step sampler turns small early differences into a different but equally valid sample, so these numbers are large for every attention LoRA. They separate allocations far apart in speed, but not ones at a similar speed (see below); they are not a quality scale.

File	Allocation
loras/sinks/minimax_h3_attn_sinks_x6_ref2va_hires.safetensors	~6× attention, ~2.8× per clip: the fast end
loras/sinks/minimax_h3_attn_sinks_balanced_ref2va_hires.safetensors	3.5× attention
loras/sinks/minimax_h3_attn_sinks_quality_ref2va_hires.safetensors	3× attention, least error

Load after the dmad or SMD LoRA, at strength 1.0. They were tuned at 768×1344 × 124 frames and work at other sizes, but there they are not the measured best.

All released LoRAs, measured together

Every file above re-measured in one run per stack, from the released files themselves: measured speed-up per clip against one-step error (2 held-out prompts, RTX 5090, DynamicVRAM).

At 960×544 × 4 steps the non-attention part of the model is a large share of the step, so every LoRA lands at 1.7–1.9× per clip; the sink LoRAs are the clear wins there. This run's margins over the reference are smaller than in the first table on this page (different prompts and loader): balanced has 9% less video and 25% less audio error at the same speed, and the non-sink fast is no better than the reference on video. The ranking of the sink LoRAs holds.

Final-latent error did not rank these LoRAs: at 960×544 the reference ended closer to dense (55%) than our non-sink LoRAs (67–73%) despite more one-step error. Between LoRAs of similar speed it is dominated by how a few-step sampler amplifies any difference, so judge those by eye and ear.