Add experimental sink attention LoRAs (need ComfyUI PR #16994) and results

README.md

@@ -66,6 +66,7 @@ LoRA's own error is measured separately on the [attention page](docs/attention.m
 | `loras/minimax_h3_attn_balanced.safetensors` | attention config LoRA: reference speed, less error | ~10 KB |
 | `loras/minimax_h3_attn_quality.safetensors` | attention config LoRA: ~4% slower, clearly less error | ~10 KB |
 | `loras/minimax_h3_attn_fast_dmad4step.safetensors` | attention config LoRA for the dmad 4-step stack | ~10 KB |
+| `loras/sinks/*.safetensors` | experimental attention LoRAs with audio/text sinks; need the ComfyUI PR in [Attention](docs/attention.md) | ~10 KB |
 
 **Checkpoint:** put it in `models/diffusion_models/` and pick it in the H3 template's **Load Diffusion Model** node;
 keep the usual H3 text encoder and VAEs. "Pruned" means the 13B AdaLN parameters are replaced by an equivalent

docs/attention.md

@@ -80,15 +80,29 @@ the **fast** allocation is only worth it on the 4-step stack.
 Load with **Load LoRA (Model Only)** at strength 1.0, after any weight LoRA. They hold no weights, so they also keep a
 calibrated NVFP4 checkpoint's scales. Disable attention-override nodes, or the configs are ignored.
 
-## Next: audio/text protection and per-step configs (ComfyUI patch, in progress)
+## Experimental: audio/text protection (needs a ComfyUI patch, not yet upstream)
 
-ComfyUI's config passes only `tau` to SOL. A small patch (branch `h3-attn-config-ext`) adds:
+ComfyUI's config passes only `tau` to SOL. A small patch ([ComfyUI PR #16994](https://github.com/Comfy-Org/ComfyUI/pull/16994)) lets H3 publish its packed row
+spans so SOL keeps every query's text/conditioning/audio keys exact (`"sink_keys": "prefix"`) and the audio queries
+fully exact (`"sink_queries": "audio"`): a few hundred of ~19.5K rows. It also passes through the other SOL options
+and, separately, allows per-step configs (`"sigma_range"`). Same stack, RTX 5090:
 
-- **sinks**: H3 publishes its packed row spans, and SOL can keep every query's text/conditioning/audio keys exact
-  (`"sink_keys": "prefix"`) and the audio queries fully exact (`"sink_queries": "audio"`) — a few hundred of ~19.5K
-  rows, the ones the audio error says matter most;
-- **per-step configs**: a config can be a list of entries with `"sigma_range"`, e.g. dense on the first step and
-  sparser later;
-- pass-through of the other SOL options (`topk_ratio`, `tail`, `token_aug`).
+| Attention LoRA | s/step | Video error | Audio error |
+|---|---|---|---|
+| comfyanonymous test LoRA | 2.15 | 18.4% | 13.6% |
+| sinks, same speed | 2.17 | **10.9%** | **7.5%** |
+| sinks, fast (4× attention kernels; ~6% per step) | **2.03** | 13.7% | 8.6% |
+
+Removing the sinks from the same allocations raises the error to 14.1% / 11.4% and 17.5% / 15.5%: the search and the sinks
+each account for roughly half of the gain over the reference. Per-step configs added at most ~8% on top and did worse at 4×. 
+
+To test, check out the PR (`gh pr checkout 16994` in a ComfyUI clone) and load a sink LoRA as above, after the
+dmad 4-step LoRA:
+
+| File | Allocation |
+|---|---|
+| `loras/sinks/minimax_h3_attn_sinks_same_speed_dmad4step.safetensors` | reference speed, ~40% less error |
+| `loras/sinks/minimax_h3_attn_sinks_fast_dmad4step.safetensors` | ~6% faster per step than the reference, less error |
 
-Searches with these are running; results will be added here.
+On stock ComfyUI these load and run, but the sink keys are ignored and the error is the "sinks removed" figures above:
+use the LoRAs in the table further up instead.

loras/sinks/minimax_h3_attn_sinks_fast_dmad4step.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:067e30a836392b3acc60398161b7d4a31a940828103decec10e943ed7541e566
+size 10250

loras/sinks/minimax_h3_attn_sinks_same_speed_dmad4step.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b882723c13247929335c3ea937827be2a3c8b15c713a033ef127771ff06ab804
+size 9306