import torch
from diffusers import DiffusionPipeline
from diffusers.utils import load_image, export_to_video
# switch to "mps" for apple devices
pipe = DiffusionPipeline.from_pretrained("VAST-AI/EdenLive-14B-Diffusers", dtype=torch.bfloat16, device_map="cuda")
pipe.to("cuda")
prompt = "A man with short gray hair plays a red electric guitar."
image = load_image(
"https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/guitar-man.png"
)
output = pipe(image=image, prompt=prompt).frames[0]
export_to_video(output, "output.mp4")EdenLive 14B — Diffusers
EdenLive generates video from a proxy video, a reference image, and a text prompt. This repository includes the 14B bidirectional and causal models for Diffusers.
Usage
Install the EdenLive integration and dependencies:
pip install "git+https://github.com/skygoo2000/diffusers.git@edenlive" \
torch torchvision "transformers>=4.56.0" accelerate sentencepiece ftfy imageio imageio-ffmpeg
Prepare a reference image and a proxy video with at least 49 frames sampled at 16 fps.
import torch
from diffusers import EdenLivePipeline
from diffusers.utils import export_to_video, load_image, load_video
pipe = EdenLivePipeline.from_pretrained(
"VAST-AI/EdenLive-14B-Diffusers",
torch_dtype={"default": torch.bfloat16, "proxy_encoder": torch.float32, "proxy_projection": torch.float32},
).to("cuda")
frames = pipe(
image=load_image("reference.png"),
video=load_video("proxy-16fps.mp4"),
prompt="A character explores a snowy landscape.",
num_frames=49,
).frames[0]
export_to_video(frames, "edenlive.mp4", fps=16)
See the usage guide for causal generation, dynamic prompts, and LightTAE, and the inference examples for multi-GPU usage.
License
EdenLive components use Apache 2.0. DINOv3 components are covered by the DINOv3 License. CLIP components use the MIT license.
- Downloads last month
- 8
Model tree for VAST-AI/EdenLive-14B-Diffusers
Base model
VAST-AI/EdenLive