COMFYUI.md

ComfyUI plugin (SM89 and SM120)

This repository is also a ComfyUI custom node for the native MiniMax-H3 model. Its current implementation and cross-pipeline differences are described in the ComfyUI guide. The node uses architecture-specific comfy-kitchen Spark-H3 CUDA kernels on NVIDIA SM89 (RTX 4090) and SM120 (RTX 50-series) GPUs and keeps all packed text, image/video reference, and audio conditioning exact.

The integration is built on ComfyUI's official MiniMax-H3 sparse-attention architecture and the comfy-kitchen v0.2.36 Sol kernel stack. It installs dit/double_block replacements, follows the native sigma-based sparse window, resets state through ON_CLEANUP, and projects QKV in 4096-token chunks. The projected BTHD tensors then use comfy-kitchen kernels for Top-K routing and global reweighting. The fanout-16, 32-landmark, group-1 reblock permutation is consumed directly by Sol preprocessing and its output scatter, without four materialized Q/K/V/output gathers. Global reweighting uses the mean of all target-video queries as one anchor per head; conditioning queries remain dense. It does not replace each attention module's forward method.

Install

This release provides two Linux x86_64 CUDA 13.0 archives:

GPU	CUDA toolkit	Release archive
SM120 (RTX 50 series)	13.0	ComfyUI-Spark-H3-<version>-linux-x86_64-sm120-cu130.zip
SM89 (RTX 4090)	13.0	ComfyUI-Spark-H3-<version>-linux-x86_64-sm89-cu130.zip

Select the archive matching both the GPU architecture and torch.version.cuda. CUDA 12.8 is available only through the explicit --experimental-cuda local-wheel or source-build path and is not a release asset. Each archive contains only the custom node, its reblock runtime, workflows, and compatible backend wheels—not this repository's model or research assets. Extract it below custom_nodes and run its installer with the same Python interpreter that starts ComfyUI:

cd /path/to/ComfyUI/custom_nodes
unzip /path/to/ComfyUI-Spark-H3-<version>-linux-x86_64-sm120-cu130.zip  # SM120 example
/path/to/ComfyUI/.venv/bin/python ComfyUI-Spark-H3/install.py

After registry publication, ComfyUI Manager is the recommended discovery and installation UI, but it remains optional: Spark-H3 neither bundles nor installs Manager. The manual ZIP path is intentionally kept as an independently testable baseline.

The cross-platform installer first accepts an existing compatible backend, then selects a bundled or pinned-release wheel. It falls back to building the pinned, patched comfy-kitchen source only when no wheel matches. Source builds require Git, CMake, Ninja, a C++ compiler, and CUDA nvcc.

The backend wheel version encodes both architecture and CUDA toolchain: SM89 CU130 uses +spark.h3.sm89.cu130.1, SM120 CU130 uses +spark.h3.sm120.cu130.1, and experimental SM120 CU128 uses +spark.h3.sm120.cu128.1. SM89 is compiled with CUDA architecture 89, while SM120 CU130 is compiled with 120f. Archive names and spark_h3_build.json also encode linux-x86_64 or windows-x86_64; the packager and installer reject wheels or packages for the other OS. Python 3.10/3.11 also requires its own wheel or source compilation. Windows packaging is prepared, but the Windows runtime has not yet been validated. Other GPU architectures are unsupported.

Restart ComfyUI afterward; select MiniMax H3 Spark Attention (SM89) on an RTX 4090 or MiniMax H3 Spark Attention (SM120) on an RTX 50-series GPU. Both appear in model_patches/attention. See the dedicated ComfyUI installation and workflow guide for model files and examples.

Workflow

Insert the node after UNETLoader and before BasicGuider:

UNETLoader -> MiniMax H3 Spark Attention (SM89 or SM120) -> BasicGuider

The 14.4 s native model example uses the complete three-shot VDN prompt 8, 345 frames at 1344×768, and a 20-step sampler. It has one Spark patch in the model path, without a Turbo LoRA or a second sparse-attention patch. It uses the same 20% Top-K ratio as the LoRA examples, with four dense warmup evaluations. Bundled workflows default to the official minimax_h3_video_vae_fp16.safetensors.

Two matching 14.4-second, 8-step LoRA examples use the same full prompt, seed, resolution, Spark 20% Top-K ratio, and two dense warmup evaluations. The DMAD and Alibaba-PAI examples retain the sampling protocol required by their respective students:

Workflow	LoRA loader	Sampler
LightX2V 768p 8-step LoRA	LoraLoaderModelOnly	res_multistep
Larryvrh v4 8-step LoRA	LoraLoaderModelOnly (converted weight)	euler
DMAD 4-step LoRA	LoraLoaderModelOnly	MiniMaxH3DMADSampler
Official LBH two-pass + LightX2V 4-step	LoraLoaderModelOnly	two SamplerCustomAdvanced passes
Alibaba-PAI PDD Acc FL2VA 8-step LoRA	MiniMaxH3PDDAccApply	euler + node-provided sigmas

Larryvrh's original LoRA names its tensors differently and targets the full 2688-wide AdaLN time embedding, while the pruned ComfyUI model stores an 8-dimensional time-curve basis. Convert it once with tools/convert_larryvrh_lora_comfyui.py; the result preserves the 208 backbone updates and projects all 51 AdaLN updates into that curve basis. The bundled workflow then follows the same low-overhead path as LightX2V: stock LoraLoaderModelOnly plus a stock sampler, with no runtime-LoRA matmuls or Larryvrh custom node. The measured projection error before the AdaLN output matrix is about 0.10% mean and 0.22% worst-case for the published v4 weight.

python tools/convert_larryvrh_lora_comfyui.py \
  --input /path/to/minimax_h3_turbo_v4_step600_ema.safetensors \
  --output /path/to/ComfyUI/models/loras/minimax_h3_turbo_v4_step600_ema_comfyui_curve_bf16.safetensors

Both the LightX2V and Larryvrh examples remain in the standalone ZIP as explicitly labelled optional integrations.

DMAD publishes a Diffusers-format rank-128 LoRA. Convert it before selecting it in LoraLoaderModelOnly:

python scripts/convert_dmad_lora_to_comfyui.py \
  /path/to/dmad_minimax_h3_4step_lora_critic.safetensors \
  /path/to/ComfyUI/models/loras/dmad_minimax_h3_4step_lora_critic_comfyui_bf16.safetensors

The converter fuses Q/K/V adapters without changing their represented update and swaps the Diffusers SwiGLU halves to ComfyUI order. The DMAD workflow then applies MiniMaxH3SigmaShift with video shift 12 and audio shift 2 before Spark. Use the bundled MiniMaxH3DMADSampler; a regular Euler or res_multistep sampler does not implement the re-noise rule used to train the released student.

The LBH example follows the repository's h3_lbh implementation and LBH's published ComfyUI graph. It requires LBH-123-AI/Comfyui_Minimax_h3_latent_Upscaler and minimax_h3_latent_upscaler_3d_conv_v1_fp16.safetensors under ComfyUI/models/latent_upscale_models/. The first pass consumes four evaluations from an eight-point simple schedule at 0.2 MP. Its clean denoised_output is separated into video and audio; only the video latent is lifted. The second pass re-noises at 0.98 MP and runs three evaluations at video sigmas 0.9035, 0.6316, 0.3158, 0.0. Spark applies only to this high-resolution pass, with no warmup and the Diffusers-side 10% Top-K setting.

Alibaba-PAI's MiniMax-H3-Acc checkpoints are also Diffusers-side weights, but they are not ordinary PEFT LoRAs. Each file contains a rank-64 trunk LoRA and 32 PDD video/audio output heads. Diffusers can use the official file directly with scripts/minimax_h3_pdd.py: call apply_pdd_lora on pipeline.transformer for FL2VA or pipeline.transformer_ref for Ref2VA, then pass num_inference_steps=nfe + 1 to the modular pipeline. The pinned source revision, local file sizes, and SHA-256 checksums are recorded in the MiniMax-H3-Acc integration note.

For ComfyUI, install the Apache-2.0 BSAI-ComfyUI-MiniMax-H3-PDD-Acc node pack and place the original weights in models/pdd_acc. The node reads the official files directly, converts the trunk keys in memory, handles the pruned checkpoint's AdaLN curve basis, and installs the PDD head bank. Pair the FL2VA weight only with an FL2VA base and the Ref2VA weight only with a Ref2VA base. Use CFG 1.0, video/audio shifts 12/3, the plain euler sampler, and wire the Apply node's sigmas output to SamplerCustomAdvanced; do not substitute BasicScheduler, res_multistep, or another distillation LoRA.

The defaults select a 20% Top-K ratio, 20% dense warmup, and the first transformer block kept dense. Set steps to the number of model evaluations made by the sampler (normally the sampler's step count). The warmup ratio is converted to an exact number of model evaluations; minimum-token and dense-layer gating use the same policy object as ComfyUI's official sparse node. The public Spark node uses the full reweight path and a dense video tail; research ablation and tail-mode controls are not exposed in the node UI. The default reblock_layout=q_reuse_k builds the K-side landmark layout once and aliases that permutation for Q, matching Spark's q_from_k mode while avoiding the query-side M2 transform, tree plan, and duplicate index storage. Select independent to reproduce the former separately planned Q/K layouts. The default topk_mode=topk_ratio selects a fraction of the target video's 64-token key blocks with topk_ratio (default 0.2). Set topk_mode=topk_blocks to request a fixed number with topk_blocks (default 228); the ratio field is ignored in this mode. The block count is capped at the number of candidate video blocks, while conditioning/sink blocks remain exact and do not consume the Top-K budget. To use a 10% ratio, set topk_mode=topk_ratio and topk_ratio=0.1. warmup_mode=warmup_ratio (default) computes dense warmup as the ceiling of steps * warmup_ratio, where 0.2 means 20%; warmup_mode=warmup_steps uses the fixed warmup_steps count (default 4). Both modes cap warmup at steps. The frontend migrates legacy workflow values such as warmup_percent=20 to warmup_ratio=0.2 when loading the graph. strict=true is recommended: it reports an incompatible ComfyUI build, dtype, GPU, or kernel error instead of silently switching to dense attention.

Requirements:

• ComfyUI 0.38.x or 0.39.x with native MiniMax-H3 support.
• CUDA BF16 execution on a compute-capability 8.9 or 12.0 GPU.
• The ComfyUI-supported PyTorch/CUDA stack and a comfy-kitchen build containing the Spark Top-K, reblock, and global-reweight extension.

The ComfyUI integration never calls Spark-H3's research Triton/CuTe attention backends. The compatibility Sol node delegates to ComfyUI's official apply_block_sparse_attention, while the Spark node fails explicitly unless the comfy-kitchen global Spark backend is selected. The repository's original PyTorch pipeline and experimental kernels remain available only through the Python API.

The node is compatible with ComfyUI's T2VA, FL2VA, and Ref2VA packed layouts. It uses transformer_options["minimax_h3_layout"] rather than guessing token spans. Do not stack it with another node that replaces MiniMax-H3 dit/double_block entries; choose one sparse-attention implementation per workflow.