Attention page: ref2va 768x1344 sink LoRAs (quality, balanced, x6), all released LoRAs re-measured together, SMD 8-step transfer, layer map and speed-vs-error charts

README.md

@@ -66,7 +66,7 @@ LoRA's own error is measured separately on the [attention page](docs/attention.m
 | `loras/minimax_h3_attn_balanced.safetensors` | attention config LoRA: reference speed, less error | ~10 KB |
 | `loras/minimax_h3_attn_quality.safetensors` | attention config LoRA: ~4% slower, clearly less error | ~10 KB |
 | `loras/minimax_h3_attn_fast_dmad4step.safetensors` | attention config LoRA for the dmad 4-step stack | ~10 KB |
-| `loras/sinks/*.safetensors` | experimental attention LoRAs with audio/text sinks; need the ComfyUI PR in [Attention](docs/attention.md) | ~10 KB |
+| `loras/sinks/*.safetensors` | attention LoRAs with audio/text sinks, including ref2va 768×1344 ones that also suit the SMD 8-step LoRA (x6: ~2.8× faster sampling); need the ComfyUI PR in [Attention](docs/attention.md) | ~10 KB |
 
 **Checkpoint:** put it in `models/diffusion_models/` and pick it in the H3 template's **Load Diffusion Model** node;
 keep the usual H3 text encoder and VAEs. "Pruned" means the 13B AdaLN parameters are replaced by an equivalent

docs/attention.md

@@ -75,7 +75,7 @@ the **fast** allocation is only worth it on the 4-step stack.
 |---|---|---|
 | `loras/minimax_h3_attn_balanced.safetensors` | same speed as the reference, less error | any INT8/NVFP4 H3 stack, 4 or 20 steps |
 | `loras/minimax_h3_attn_quality.safetensors` | ~4% slower, clearly less error | any INT8/NVFP4 H3 stack |
-| `loras/minimax_h3_attn_fast_dmad4step.safetensors` | slightly faster than the reference, a little less error | INT8 ConvRot + dmad 4-step |
+| `loras/minimax_h3_attn_fast_dmad4step.safetensors` | slightly faster than the reference, less audio error, about the same video error | INT8 ConvRot + dmad 4-step |
 
 Load with **Load LoRA (Model Only)** at strength 1.0, after any weight LoRA. They hold no weights, so they also keep a
 calibrated NVFP4 checkpoint's scales. Disable attention-override nodes, or the configs are ignored.
@@ -133,3 +133,73 @@ in busy scenes (broken motion without sinks) and fine detail (hands, fabric nois
 
 On stock ComfyUI these load and run, but the sink keys are ignored and the error is the "sinks removed" figures above:
 use the LoRAs in the table further up instead.
+
+## Ref2VA at 768×1344, 124 frames (sinks; needs the PR above)
+
+Tuned on the ref2va pruned INT8 ConvRot transformer + dmad 4-step full-critic LoRA, `dpmpp_sde_gpu` / beta, 8 steps,
+one reference image. At this size dense attention is ~78% of each model call, so faster allocations pay off more than
+at 960×544. All files measured in one run (2 held-out prompts, same seed; DiT time per clip on an RTX 5090 with
+DynamicVRAM, as in a ComfyUI server):
+
+| Attention LoRA | Time per clip | Speed-up | Video error | Audio error |
+|---|---|---|---|---|
+| none (dense) | 198 s | 1.00× | – | – |
+| comfyanonymous test LoRA | 91 s | 2.19× | 16.4% | 24.6% |
+| **quality** (ref2va) | 97 s | 2.05× | 9.2% | 10.2% |
+| **balanced** (ref2va) | 90 s | 2.20× | 11.1% | 11.3% |
+| sinks fast (dmad 4-step, 960×544 tune) | 83 s | 2.40× | 14.4% | 14.7% |
+| **x6** (ref2va) | 72 s | 2.75× | 22.2% | 19.3% |
+
+At the reference LoRA's speed, **balanced** has a third less video error and half the audio error. Blind tests on two
+prompts: dense, fast, balanced and quality could not be told apart; between fast, x6 and an 8× allocation, only the 8×
+one was picked out as different, in both clips. Past ~6× the rest of the model dominates the step, so x6 is the fast
+end.
+
+### The same LoRAs on the SMD 8-step LoRA
+
+[SMD Ref2VA 8-step](https://huggingface.co/zjwfufu/SMD-MiniMax-H3-Ref2VA-8step-ComfyUI) at its own settings (euler,
+its ManualSigmas, shift 12/3, 1344×768, two references) gives a different stack: different LoRA, sampler, schedule and
+reference count. We tuned it from scratch and compared:
+
+![Per-layer attention allocations and sensitivity](../assets/attn_layer_map.png)
+
+Which layers tolerate sparse attention barely changes between the two stacks (rank correlation 0.93–0.98 per kernel),
+so a fresh SMD tune found nothing better than **x6** as published: at the same speed it measured 63.9% / 23.8% final
+video / audio error against 67.2% / 26.3% for the SMD tune (2 prompts). Use the ref2va LoRAs on SMD as they are. Measured on SMD,
+DiT time per clip:
+
+| Attention LoRA | Time per clip | Speed-up |
+|---|---|---|
+| none (dense) | 102 s | 1.00× |
+| **x6** (ref2va) | 36 s | 2.81× |
+
+"Final error" is how far the finished latent lands from dense attention on the same seed. A few-step sampler turns
+small early differences into a different but equally valid sample, so these numbers are large for every attention
+LoRA. They separate allocations far apart in speed, but not ones at a similar speed (see below); they are not a
+quality scale.
+
+| File | Allocation |
+|---|---|
+| `loras/sinks/minimax_h3_attn_sinks_x6_ref2va_hires.safetensors` | ~6× attention, ~2.8× per clip: the fast end |
+| `loras/sinks/minimax_h3_attn_sinks_balanced_ref2va_hires.safetensors` | 3.5× attention |
+| `loras/sinks/minimax_h3_attn_sinks_quality_ref2va_hires.safetensors` | 3× attention, least error |
+
+Load after the dmad or SMD LoRA, at strength 1.0. They were tuned at 768×1344 × 124 frames and work at other sizes,
+but there they are not the measured best.
+
+## All released LoRAs, measured together
+
+Every file above re-measured in one run per stack, from the released files themselves: measured speed-up per clip
+against one-step error (2 held-out prompts, RTX 5090, DynamicVRAM).
+
+![Speed vs error of the released attention LoRAs](../assets/attn_loras.png)
+
+At 960×544 × 4 steps the non-attention part of the model is a large share of the step, so every LoRA lands at
+1.7–1.9× per clip; the sink LoRAs are the clear wins there. This run's margins over the reference are smaller than
+in the first table on this page (different prompts and loader): **balanced** has 9% less video and 25% less audio
+error at the same speed, and the non-sink **fast** is no better than the reference on video. The ranking of the sink
+LoRAs holds.
+
+Final-latent error did not rank these LoRAs: at 960×544 the reference ended closer to dense (55%) than our non-sink
+LoRAs (67–73%) despite more one-step error. Between LoRAs of similar speed it is dominated by how a few-step sampler
+amplifies any difference, so judge those by eye and ear.

loras/sinks/minimax_h3_attn_sinks_balanced_ref2va_hires.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a03d3f070d0a9f4a9b196f85b216dc0ac84ac049c9c5f8f01749a2f4195980a9
+size 10080

loras/sinks/minimax_h3_attn_sinks_quality_ref2va_hires.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ca0557c0302d609a23562f5a82810e5f4b596311a0e198ced6aba202f6f49303
+size 9297

loras/sinks/minimax_h3_attn_sinks_x6_ref2va_hires.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a73c302617ccb77cd6e5e914fdd6f57b019fe279133a18216e65082c0b3f3c9e
+size 11145