Instructions to use Lakonik/pi-H3 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Diffusers
How to use Lakonik/pi-H3 with Diffusers:
pip install -U diffusers transformers accelerate
import torch from diffusers import DiffusionPipeline # switch to "mps" for apple devices pipe = DiffusionPipeline.from_pretrained("Lakonik/pi-H3", dtype=torch.bfloat16, device_map="cuda") prompt = "Astronaut in a jungle, cold color palette, muted colors, detailed, 8k" image = pipe(prompt).images[0] - Notebooks
- Google Colab
- Kaggle
import torch
from diffusers import DiffusionPipeline
# switch to "mps" for apple devices
pipe = DiffusionPipeline.from_pretrained("Lakonik/pi-H3", dtype=torch.bfloat16, device_map="cuda")
prompt = "Astronaut in a jungle, cold color palette, muted colors, detailed, 8k"
image = pipe(prompt).images[0]pi-H3: Video Generation in 2β4 Steps Without a VAE Decoder
Hansheng Chen1*,
Shengqu Cai1,
Jiaming Songβ ,
Gordon Wetzstein1,
Xinchen Yanβ ,
Leonidas Guibas1
1Stanford University
*Part of work done during an internship at Luma AI, β Work done while at Luma AI
Blog | Code | ComfyUI | pi-H3 Demoπ€
2-step and 4-step adapters of MiniMax H3, the fastest way to generate MiniMax H3 videos with synchronized audio. pi-H3 combines pi-Flow v2 few-step distillation with DWD (Decoding Without a Decoder), which predicts the video frames directly at the last step, so the video skips the VAE decoder.
Models
| Adapter | Steps | Video output | INT8 ConvRot version |
|---|---|---|---|
pi_h3_2step_dwd_adapter |
2 | DWD | pi_h3_2step_dwd_adapter_int8_convrot.safetensors |
pi_h3_4step_dwd_adapter |
4 | DWD | pi_h3_4step_dwd_adapter_int8_convrot.safetensors |
pi_h3_2step_adapter |
2 | VAE decoder | |
pi_h3_4step_adapter |
4 | VAE decoder |
All adapters support text, first-frame, and last-frame conditioning. We recommend the DWD adapters. The INT8 ConvRot versions are for ComfyUI only.
Usage
Diffusers
Please first install LakonLab v0.3 following the pi-H3 README.
import torch
from diffusers import ComponentsManager
from diffusers.utils import encode_video
from lakonlab.models.architectures import RGBColorEncoder
from lakonlab.models.architectures.diffusers.attention import default_attention_backend, set_attention_backends
from lakonlab.models.architectures.diffusers.minimax_h3 import MiniMaxH3SolAttnProcessor
from lakonlab.models.diffusions.schedulers import FlowEulerODEScheduler
from lakonlab.ops import quantize_frozen_linears
from lakonlab.pipelines.pipeline_minimax_h3 import MiniMaxH3DWDBlocks
from lakonlab.pipelines.utils import load_lakonlab_adapter
NUM_STEPS = 4 # 2 or 4
ADAPTER_NAME = f'pi_h3_{NUM_STEPS}step_dwd_adapter'
MAX_RAW_T = {2: 0.93, 4: 0.96}[NUM_STEPS]
# load base pipeline (the VAE only encodes the keyframes, the DWD head decodes the pixels)
manager = ComponentsManager()
pipe = MiniMaxH3DWDBlocks().init_pipeline('multimodalart/MiniMax-H3-Pruned', components_manager=manager)
pipe.load_components(
names=['text_encoder', 'tokenizer', 'processor', 'vae', 'audio_vae', 'transformer'],
dtype=torch.bfloat16, trust_remote_code=True)
# load adapter
load_lakonlab_adapter(pipe, 'Lakonik/pi-H3', subfolder=ADAPTER_NAME)
# FP8 W8A8
pipe.transformer.requires_grad_(False)
quantize_frozen_linears(pipe.transformer, exclude_keys=['lora', 'norm_out', 'proj_out', 'dwd_linear_head'])
pipe.text_encoder.requires_grad_(False)
quantize_frozen_linears(pipe.text_encoder, exclude_keys=['lm_head'])
torch.cuda.empty_cache()
# the fastest installed attention kernels for the GPU (the DWD pixel refiner takes variable-length windows)
set_attention_backends(pipe.transformer, backend=default_attention_backend('dense'), overrides={
'self.dwd_pixel_refiner': default_attention_backend('varlen')})
# Sol sparse attention after the first 2 blocks (requires lakonlab[sol]; remove this loop for dense attention)
for block in pipe.transformer.transformer_blocks[2:]:
block.attn.set_processor(MiniMaxH3SolAttnProcessor())
block.attn.set_attention_backend(default_attention_backend('short')) # for the short prefix query rows
pipe.update_components(
scheduler=FlowEulerODEScheduler(
num_train_timesteps=1, shift=12.0, max_raw_t=MAX_RAW_T, final_step_size_scale=0.6),
audio_scheduler=FlowEulerODEScheduler(
num_train_timesteps=1, shift=3.0, max_raw_t=MAX_RAW_T, final_step_size_scale=0.6))
pipe.to('cuda')
color_encoder = RGBColorEncoder(
use_affine_norm=True, mean=(-0.03, -0.088, -0.188), std=(0.458, 0.448, 0.45)).cuda()
# Text-to-video+audio example
prompt = 'A red fox trotting through a snowy pine forest, snow crunching underfoot, birds chirping in the distance'
out = pipe(
prompt=prompt,
height=768,
width=1344,
num_frames=243, # 10 s at H3's fixed 24 fps, rounded up to 17n + 5
num_inference_steps=NUM_STEPS,
generator=torch.Generator().manual_seed(42),
output=['dwd_pixels', 'audio', 'sampling_rate'])
video = out.pop('dwd_pixels')
video = color_encoder.decode(video).add_(1).mul_(0.5)
encode_video(
video[0].permute(1, 2, 3, 0).cpu().numpy(), fps=24, output_path='minimax_h3_dwd_t2va.mp4',
audio=out['audio'][0], audio_sample_rate=out['sampling_rate'])
To condition on a first frame or a last frame, pass image=... or last_image=.... See the pi-H3 README for the non-DWD adapters, the Gradio app, and accelerated inference.
ComfyUI
See ComfyUI-piFlow for the workflows and model links.
License
The pi-H3 adapters are Model Derivatives of MiniMax H3 and are distributed under the MiniMax H3 Community License Agreement. See also NOTICE.
Citation
@misc{chen2026pih3,
title = {{pi-H3: Video Generation in 2β4 Steps Without a VAE Decoder}},
author = {Hansheng Chen and Shengqu Cai and Jiaming Song and Gordon Wetzstein and Xinchen Yan and Leonidas Guibas},
year = {2026},
month = nov,
howpublished = {\url{https://hanshengchen.com/blog/pi-h3/}}
}
@article{chen2026decoding,
title={Decoding Without a Decoder: Direct Pixel Prediction from Diffusion Transformers},
author={Hansheng Chen and Shengqu Cai and Jan Ackermann and Rayhan Zirvi and Gordon Wetzstein and Xinchen Yan and Leonidas Guibas},
journal={arXiv preprint arXiv:2611.22287},
url={https://arxiv.org/abs/2611.22287},
year={2026},
}
- Downloads last month
- -
