Instructions to use SagiPolaczek/LTX-2.3-Sync-LoRA with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Diffusers
How to use SagiPolaczek/LTX-2.3-Sync-LoRA with Diffusers:
pip install -U diffusers transformers accelerate
import torch from diffusers import DiffusionPipeline from diffusers.utils import load_image, export_to_video # switch to "mps" for apple devices pipe = DiffusionPipeline.from_pretrained("Lightricks/LTX-2", torch_dtype=torch.bfloat16, device_map="cuda") pipe.load_lora_weights("SagiPolaczek/LTX-2.3-Sync-LoRA") prompt = "A man with short gray hair plays a red electric guitar." input_image = load_image("https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/guitar-man.png") image = pipe(image=input_image, prompt=prompt).frames[0] export_to_video(output, "output.mp4") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- Draw Things
audion sync and lips sync
#2
by ttamementomori - opened
Thanks the lora is great for the moves !! but I noticed it doen't seams to take the original audio in the process and it generate a new one breaking lipsync with with original audio is there a way to fix that ?
ttamementomori changed discussion status to closed
I didn’t test it, but it will probably work:
Instead of generating the audio, “freeze” it to be your input audio.
Specifically: the audio is part of the generation, but it’s not being regenerated- ans the model will use it in its context
thats the weird thing i encode original audio instead of the blank default one but it generates a fresh audio. anyway the moves are working great !!!
edit : it works using this node for the audio latent : 🅛🅣🅧 Set Audio Ref Tokens