How to use from the
Use from the
Diffusers library
pip install -U diffusers transformers accelerate
import torch
from diffusers import DiffusionPipeline

# switch to "mps" for apple devices
pipe = DiffusionPipeline.from_pretrained("Lakonik/pi-H3", dtype=torch.bfloat16, device_map="cuda")

prompt = "Astronaut in a jungle, cold color palette, muted colors, detailed, 8k"
image = pipe(prompt).images[0]

pi-H3: Video Generation in 2–4 Steps Without a VAE Decoder

Hansheng Chen1*, Shengqu Cai1, Jiaming Song†, Gordon Wetzstein1, Xinchen Yan†, Leonidas Guibas1
1Stanford University
*Part of work done during an internship at Luma AI, †Work done while at Luma AI
Blog | Code | ComfyUI | pi-H3 DemoπŸ€—

2-step and 4-step adapters of MiniMax H3, the fastest way to generate MiniMax H3 videos with synchronized audio. pi-H3 combines pi-Flow v2 few-step distillation with DWD (Decoding Without a Decoder), which predicts the video frames directly at the last step, so the video skips the VAE decoder.

Models

Adapter Steps Video output INT8 ConvRot version
pi_h3_2step_dwd_adapter 2 DWD pi_h3_2step_dwd_adapter_int8_convrot.safetensors
pi_h3_4step_dwd_adapter 4 DWD pi_h3_4step_dwd_adapter_int8_convrot.safetensors
pi_h3_2step_adapter 2 VAE decoder
pi_h3_4step_adapter 4 VAE decoder

All adapters support text, first-frame, and last-frame conditioning. We recommend the DWD adapters. The INT8 ConvRot versions are for ComfyUI only.

Usage

Diffusers

Please first install LakonLab v0.3 following the pi-H3 README.

import torch
from diffusers import ComponentsManager
from diffusers.utils import encode_video
from lakonlab.models.architectures import RGBColorEncoder
from lakonlab.models.architectures.diffusers.attention import default_attention_backend, set_attention_backends
from lakonlab.models.architectures.diffusers.minimax_h3 import MiniMaxH3SolAttnProcessor
from lakonlab.models.diffusions.schedulers import FlowEulerODEScheduler
from lakonlab.ops import quantize_frozen_linears
from lakonlab.pipelines.pipeline_minimax_h3 import MiniMaxH3DWDBlocks
from lakonlab.pipelines.utils import load_lakonlab_adapter

NUM_STEPS = 4  # 2 or 4
ADAPTER_NAME = f'pi_h3_{NUM_STEPS}step_dwd_adapter'
MAX_RAW_T = {2: 0.93, 4: 0.96}[NUM_STEPS]

# load base pipeline (the VAE only encodes the keyframes, the DWD head decodes the pixels)
manager = ComponentsManager()
pipe = MiniMaxH3DWDBlocks().init_pipeline('multimodalart/MiniMax-H3-Pruned', components_manager=manager)
pipe.load_components(
    names=['text_encoder', 'tokenizer', 'processor', 'vae', 'audio_vae', 'transformer'],
    dtype=torch.bfloat16, trust_remote_code=True)
# load adapter
load_lakonlab_adapter(pipe, 'Lakonik/pi-H3', subfolder=ADAPTER_NAME)

# FP8 W8A8
pipe.transformer.requires_grad_(False)
quantize_frozen_linears(pipe.transformer, exclude_keys=['lora', 'norm_out', 'proj_out', 'dwd_linear_head'])
pipe.text_encoder.requires_grad_(False)
quantize_frozen_linears(pipe.text_encoder, exclude_keys=['lm_head'])
torch.cuda.empty_cache()
# the fastest installed attention kernels for the GPU (the DWD pixel refiner takes variable-length windows)
set_attention_backends(pipe.transformer, backend=default_attention_backend('dense'), overrides={
    'self.dwd_pixel_refiner': default_attention_backend('varlen')})
# Sol sparse attention after the first 2 blocks (requires lakonlab[sol]; remove this loop for dense attention)
for block in pipe.transformer.transformer_blocks[2:]:
    block.attn.set_processor(MiniMaxH3SolAttnProcessor())
    block.attn.set_attention_backend(default_attention_backend('short'))  # for the short prefix query rows

pipe.update_components(
    scheduler=FlowEulerODEScheduler(
        num_train_timesteps=1, shift=12.0, max_raw_t=MAX_RAW_T, final_step_size_scale=0.6),
    audio_scheduler=FlowEulerODEScheduler(
        num_train_timesteps=1, shift=3.0, max_raw_t=MAX_RAW_T, final_step_size_scale=0.6))

pipe.to('cuda')

color_encoder = RGBColorEncoder(
    use_affine_norm=True, mean=(-0.03, -0.088, -0.188), std=(0.458, 0.448, 0.45)).cuda()

# Text-to-video+audio example
prompt = 'A red fox trotting through a snowy pine forest, snow crunching underfoot, birds chirping in the distance'
out = pipe(
    prompt=prompt,
    height=768,
    width=1344,
    num_frames=243,  # 10 s at H3's fixed 24 fps, rounded up to 17n + 5
    num_inference_steps=NUM_STEPS,
    generator=torch.Generator().manual_seed(42),
    output=['dwd_pixels', 'audio', 'sampling_rate'])
video = out.pop('dwd_pixels')
video = color_encoder.decode(video).add_(1).mul_(0.5)
encode_video(
    video[0].permute(1, 2, 3, 0).cpu().numpy(), fps=24, output_path='minimax_h3_dwd_t2va.mp4',
    audio=out['audio'][0], audio_sample_rate=out['sampling_rate'])

To condition on a first frame or a last frame, pass image=... or last_image=.... See the pi-H3 README for the non-DWD adapters, the Gradio app, and accelerated inference.

ComfyUI

See ComfyUI-piFlow for the workflows and model links.

License

The pi-H3 adapters are Model Derivatives of MiniMax H3 and are distributed under the MiniMax H3 Community License Agreement. See also NOTICE.

Citation

@misc{chen2026pih3,
  title        = {{pi-H3: Video Generation in 2–4 Steps Without a VAE Decoder}},
  author       = {Hansheng Chen and Shengqu Cai and Jiaming Song and Gordon Wetzstein and Xinchen Yan and Leonidas Guibas},
  year         = {2026},
  month        = nov,
  howpublished = {\url{https://hanshengchen.com/blog/pi-h3/}}
}

@article{chen2026decoding,
  title={Decoding Without a Decoder: Direct Pixel Prediction from Diffusion Transformers},
  author={Hansheng Chen and Shengqu Cai and Jan Ackermann and Rayhan Zirvi and Gordon Wetzstein and Xinchen Yan and Leonidas Guibas},
  journal={arXiv preprint arXiv:2611.22287},
  url={https://arxiv.org/abs/2611.22287},
  year={2026},
}
Downloads last month
-
Inference Providers NEW
This model isn't deployed by any Inference Provider. πŸ™‹ Ask for provider support

Model tree for Lakonik/pi-H3

Finetuned
(241)
this model
Finetunes
1 model

Spaces using Lakonik/pi-H3 4

Paper for Lakonik/pi-H3