TaoMate-H3 FL2VA LoRA for ComfyUI

TaoMate-H3-FL2VA by the Alibaba TaoLive AIGC Team is a 3-step first/last-frame-to-audio-video LoRA for MiniMax H3 (step-300 online generator, rank 128, alpha 128), released in a PEFT-style layout for TaoMate's own streaming runtime. This is the same LoRA with its keys converted to ComfyUI's layout.

Unlike the attention config LoRAs, this is a weight LoRA: it changes the transformer's weights and holds ~620M parameters.

File	What	Size
`distill_loras/taomate_h3_fl2va_step300_bf16.safetensors`	TaoMate-H3-FL2VA step 300, full rank 128, ComfyUI keys	1.24 GB

Conversion

ComfyUI's H3 model uses the same module names and the same fused layouts as TaoMate's (attn.qkv_proj as q, k, v concatenated; mlp.fc1 as the fused SwiGLU input), so no weight is reordered:

keys renamed blocks.N.attn.qkv_proj.lora_a / lora_b → diffusion_model.blocks.N.attn.qkv_proj.lora_down.weight / lora_up.weight, plus .alpha = 128;
fp32 rounded to bf16 (the only lossy step; rank, alpha and shapes unchanged);
TaoMate's adapter_config.json is kept in the file's metadata.

Checked with ComfyUI's own LoRA loader against the H3 model: all 208 layers (50 DiT blocks and 2 token refiner blocks × qkv, out, fc1, fc2) are matched with the expected shapes, and no key is left unloaded.

Use

Put it in models/loras/ and load it with Load LoRA (Model Only) on the H3 FL2VA transformer, before any attention config LoRA. Alpha equals rank, so strength 1.0 is the adapter's own scale.

TaoMate has not released FL2VA inference code yet. Their released T2AV code (TaoMate-H3, denoise_schedule.py, model/pipeline.py, model/denoise.py) samples its 3-step LoRA like this:

3 steps at states 0, 16, 33, 49 of a 50-step shifted schedule: video shift 12 (sigmas 1.0, 0.9612, 0.8533, 0), audio shift 3 (sigmas 1.0, 0.8609, 0.5926, 0);
a deterministic Euler update on the flow;
positive prompt only, no CFG.

In ComfyUI that is ModelSamplingMiniMaxH3 (shift_video 12, shift_audio 3), ManualSigmas 1.0, 0.9612, 0.8533, 0.0, sampler euler, CFG 1. ComfyUI derives the audio sigmas from the video ones through the two shifts and gives 1.0, 0.861, 0.5925, 0. Whether FL2VA uses the same schedule is not confirmed until TaoMate's FL2VA code is out.

TaoMate's runtime generates in streamed chunks with a persistent audio-video KV cache; ComfyUI samples the whole clip in one pass.

License and attribution

TaoMate-H3 is by the Alibaba TaoLive AIGC Team, built on MiniMax H3, and distributed under the MiniMax H3 Community License Agreement. The converted file is a modified derivative (keys renamed, fp32 → bf16); see NOTICE.