Best approach for maintaining likeness in I2V?
Previously it was straightforward - just feed the image and run it. Now, with the two-pass workflow, it’s not that simple. The default 0.3 MP starting resolution completely loses the likeness. Using a reference model helps bring it back, but in my case, it causes noticeable face boiling that wasn't an issue with older methods.
To anyone who knows their way around this: what should I try? Bump up the initial pass resolution? Tweak the refmod parameters?
I prefer I2V over T2V 99% of the time because Krea 2 can give me a much better initial image with higher likeness and all the exact elements I want already dialed in.
The time/quality I use most frequently lands me at ~0.8 MP final output.
btw: In my opinion the BF16 upscaler results are way better than the FP16, without much time difference.
I'm not having issues doing pure i2v with a starting reference image and holding it.
Try something like this in the prompt:
"<Picture 1> is the correct scene identity. All lighting, lighting hue, color palette, visual output style, camera angle, camera focal length, camera orbit, camera framing, background, background objects, and characters fully preserved from <Picture 1>. <Picture 1> is the initial frame for the video and the scene continues from there. Then the rest of the prompt"
This usually holds quite well as the conditioning is passed to both sampling phases, so the second refine pass should also hold the reference.
You can try bumping up the initial pass resolution and step count to 10 or 12 and see if that helps.
I did some testing and tweaking, and here is what I landed on:
- Changed Reference to Video from Match to Max (the impact scales with the megapixel setting used, ranging from minor to significant).
- Set Add Guide For MiniMax H3: 1x before the first Basic Guider, and 1x before the second Basic Guider (the one after the upscale pass), using the initial first-frame image.
This setup currently relies on just a single image. Next, I plan to tweak it by using a refmod instead of the reference image, keeping the raw image strictly for the two Add Guide passes.
I also benchmarked it with and without your prompt, both with and without Add Guide, and honestly... I won't be using the steering prompt. Add Guide locks down the first frame almost identically to how a native I2V node behaves (giving a true 1:1 match in framing, alignment, etc.), whereas the prompt only nudged the composition closer without fully nailing it. Layering the prompt on top of Add Guide introduced weird artifacts—like the character not blinking for several seconds. With two Add Guide passes already in place, the prompt just isn't worth the trouble.
Thanks for sharing this. Problem I see with this approach of "add guide" is that, if the image selected is used as a reference but not as a starting frame, this might inject the wrong conditioning in the sampler. And the node needs careful managing bc you cannot bypass the initial image like this to do pure t2v. But I might figure out something.
Good luck with your gens! :)
PS. Adapted the same basic approach slightly modified on my current workflows, if you want to take a look at them in my repo. Thanks!
Thanks for sharing this. Problem I see with this approach of "add guide" is that, if the image selected is used as a reference but not as a starting frame, this might inject the wrong conditioning in the sampler.
That's true. That's the difference - you're building workflow for you and others, I'm building it for me only. 😅 And now I simply have a couple with different combinations, just I2V, I2V + Refmod, etc.
Checked yours - I'm not an expert at all, but it looks similar to mine just easier to work with (more automatic). Glad I could've pointed you in a direction at least!
For me the 1:1 as a starting point was important simply because I have character LoRAs trained for Krea2, and for H3 I only have refmods, so combining these two techniques gives me way better likness than just relying on ref2vid.
Cool that we both learned something!