Audio artifacts
I've been testing the various recent checkpoints including test4, and I'm noticing some strange audio behavior. Almost like an ASMR background noise. Is that expected in these early versions while testing? Just want to make sure it's not something I'm doing incorrectly.
Setup: RTX 5070 Ti (16GB), ComfyUI 0.31.0 native H3 nodes, standard FL2VA workflow (UNETLoader → MiniMaxH3ImageToVideo → SamplerCustomAdvanced), Comfy-Org qwen3vl_32b_minimax_h3_nvfp4_awq text encoder, official video/audio VAEs. Checkpoint is test4_pruned quantized to NVFP4 locally (same recipe and profile as the
community NVFP4-HQ releases). 16GB can't run the bf16, so I can't rule quantization out myself.
On the earlier build I had multiple audio issues: ASMR-like mouth sounds, unnaturally wet contact SFX, loose audio-action sync. Test4 is a big improvement. Character voice and foreground sounds are now cleaner. What remains is a constant low-level background bed of soft mouth-noise/breathy ASMR-like sound running under the whole clip.
What I've isolated so far:
- Present with the turbo lora (larryvrh v4-step600, 6-8 steps) and, reduced but still audible, without it (20 steps, res_multistep + simple). The lora amplifies it but does not originate it.
- Present across seeds and prompts. Prompt follows the official field format, the soundscape claims all audio space and includes explicit exclusions.
- Video quality and prompt adherence are excellent throughout. This is audio-only.
I'm still exploring audio. At this point I've isolated where audio lives heavily in blocks and heads and text layers, and am trying to map the real audio system inside the model. Test4 takes text layers from an LTX model as a huge experimental push and the idea was to show that it also does touch audio and audio prompting. The huge issue with audio and audio training as people are finding is that it is pretty much unified and spread throughout the entire model, with a lot of it's music weight present in blocks 20-30, which also is underneath and overlaps a ton of motion character that is present in blocks 25-40. I think this is the first thing I would call a flaw or weakness in the h3 model. The fact that those elements are unified works fine with their reinforcement run and data, but as soon as adverse training occurs it unwinds both audio, audio pairing, music, pretty much everything substantially if you show it new motion + new audio. All of that is so heavily concentrated in mid blocks and completely unified unfortunately.
Thanks for the detailed response. No worries, just making sure I'm not operating incorrectly. Glad to wait for more - good luck!
I did some intense test with the model 10Eros_Max_h3_TURBO-hybrid_beta3_int8_convrot vs minimax_h3_fl2va_int8_convrot, and i can confirm what's OP noticed. Some sounds are actually correctly matched with video but some other sound are just here to pollute the video. Also music adherence seems to be lost compared to native Minimax H3 gen. On the other end Video adherence and quality is just insanely good. It is as good the native model if not better. Just audio adherence need to be fixed and this will probably be the best fine tuned model. Thank for the hard work
I think I'd just shift blame on to the design of H3. Audio is unified, even if you train the models on just images you still disturb all audio attention. I could unfuse every lora, freeze part of mlp2 and attn_k to reduce audio shift but then it also loses the shapes of the loras to a certain extent. It's something that isn't fixed it will never be like base.
I actually see way less issues now with the turbo baked checkpoints - the audio drastically improved since earlier betas. The less LoRa stacked on top also, the better. And I've found I need far less LoRa in general with beta3