Hugging Face's logo Hugging Face
  • Models
  • Datasets
  • Spaces
  • Buckets
  • Docs
  • Enterprise
  • Pricing
    • Website
      • Tasks
      • HuggingChat
      • Collections
      • Languages
      • Organizations
    • Community
      • Blog
      • Posts
      • Daily Papers
      • Hardware
      • Learn
      • Discord
      • Forum
      • GitHub
    • Solutions
      • Team & Enterprise
      • Hugging Face PRO
      • Enterprise Support
      • Inference Providers
      • Inference Endpoints
      • Storage Buckets

  • Log In
  • Sign Up

FriendsforHumanity
/
Pika-H3-Flex-FRef2VA

Image-to-Video
Diffusers
MiniMax H3
Pika H3 Flex
comfyui
modular-diffusers
video-generation
reference-to-video
Model card Files Files and versions
xet
Community

Instructions to use FriendsforHumanity/Pika-H3-Flex-FRef2VA with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

  • Libraries
  • Diffusers

    How to use FriendsforHumanity/Pika-H3-Flex-FRef2VA with Diffusers:

    pip install -U diffusers transformers accelerate
    import torch
    from diffusers import DiffusionPipeline
    from diffusers.utils import load_image, export_to_video
    
    # switch to "mps" for apple devices
    pipe = DiffusionPipeline.from_pretrained("FriendsforHumanity/Pika-H3-Flex-FRef2VA", dtype=torch.bfloat16, device_map="cuda")
    pipe.to("cuda")
    
    prompt = "A man with short gray hair plays a red electric guitar."
    image = load_image(
        "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/guitar-man.png"
    )
    
    output = pipe(image=image, prompt=prompt).frames[0]
    export_to_video(output, "output.mp4")
  • Pika H3 Flex

    How to use FriendsforHumanity/Pika-H3-Flex-FRef2VA with Pika H3 Flex:

    # pip install -r examples/requirements-diffusers.txt (see the examples/ folder of this repo)
    import torch
    from diffusers import ComponentsManager, ModularPipeline
    from diffusers.utils.export_utils import encode_video
    from PIL import Image
    
    workflow = "fl2va"  # "fl2va" (first-frame to video+audio) or "ref2va" (reference images/video to video+audio)
    
    manager = ComponentsManager()
    manager.enable_auto_cpu_offload(device="cuda:0")
    pipe = ModularPipeline.from_pretrained("FriendsforHumanity/Pika-H3-Flex-FRef2VA", workflow=workflow, components_manager=manager)
    pipe.load_components(
        pretrained_model_name_or_path="FriendsforHumanity/Pika-H3-Flex-FRef2VA",
        dtype={"vae": torch.float32, "audio_vae": torch.float32, "default": torch.bfloat16},
    )
    # The bundled base components are not Pika-adapted weights: load the matching adapter.
    pipe.load_lora_weights("FriendsforHumanity/Pika-H3-Flex-FRef2VA", weight_name=f"{workflow}/adapter.safetensors")
    
    result = pipe(
        prompt="A cat walks through a sunlit cafe and jumps onto the counter",
        image=Image.open("first_frame.png").convert("RGB"),
        height=480,
        width=832,
        num_frames=121,
        generator=torch.Generator("cuda:0").manual_seed(0),
        output=["videos", "audio", "sampling_rate"],
    )
    encode_video(
        result["videos"][0], fps=24, output_path="output.mp4",
        audio=result["audio"][0].float().cpu(), audio_sample_rate=result["sampling_rate"],
    )
  • Notebooks
  • Google Colab
  • Kaggle
Pika-H3-Flex-FRef2VA
58.5 kB
Ctrl+K
Ctrl+K
  • 1 contributor
History: 1 commit
grindset's picture
grindset
FRef2VA for Pika H3 Flex: first/last frame with references and voice (code only)
6f5429e 13 days ago
  • comfyui
    FRef2VA for Pika H3 Flex: first/last frame with references and voice (code only) 13 days ago
  • tests
    FRef2VA for Pika H3 Flex: first/last frame with references and voice (code only) 13 days ago
  • .gitattributes
    1.52 kB
    FRef2VA for Pika H3 Flex: first/last frame with references and voice (code only) 13 days ago
  • LICENSE
    11.4 kB
    FRef2VA for Pika H3 Flex: first/last frame with references and voice (code only) 13 days ago
  • README.md
    4.46 kB
    FRef2VA for Pika H3 Flex: first/last frame with references and voice (code only) 13 days ago
  • block.py
    19.8 kB
    FRef2VA for Pika H3 Flex: first/last frame with references and voice (code only) 13 days ago
  • example.py
    5.97 kB
    FRef2VA for Pika H3 Flex: first/last frame with references and voice (code only) 13 days ago
  • modular_config.json
    156 Bytes
    FRef2VA for Pika H3 Flex: first/last frame with references and voice (code only) 13 days ago
  • requirements.txt
    387 Bytes
    FRef2VA for Pika H3 Flex: first/last frame with references and voice (code only) 13 days ago