| --- |
| license: apache-2.0 |
| pipeline_tag: other |
| tags: |
| - video-vae |
| - video-generation |
| - distillation |
| - image-to-video |
| - text-to-video |
| - comfyui |
| - wan2.1 |
| - wan2.2 |
| - minimax-h3 |
| --- |
| |
| <div align="center"> |
|
|
| # ⚡ LightVAE |
|
|
| **Towards Compact and Efficient Video Autoencoders** |
|
|
| Wan2.1 · Wan2.2 · MiniMax-H3 |
|
|
| [📄 Paper](https://huggingface.co/papers/2610.14726) · [💻 Code](https://github.com/ModelTC/LightVAE) · [🧩 ComfyUI](https://github.com/ModelTC/LightVAE/tree/main/comfyui) · [⚡ LightX2V](https://github.com/ModelTC/LightX2V) |
|
|
| </div> |
|
|
| LightVAE compresses video autoencoders while preserving their original latent interfaces, allowing compressed components to be paired with the corresponding original components. |
|
|
| - **Wan2.1 / Wan2.2:** Pro and Lite decoders, paired with the original encoder. Pro retains the original decoder depth with temporal low-rank compression; Lite adds block pruning. |
| - **MiniMax-H3:** independently block-pruned Lite encoder and decoder, with native ComfyUI integration. |
| - **Optional ConvRot-INT8:** a quantized H3 Lite decoder for ComfyUI. |
|
|
| Download the components you need from [Files and versions](https://huggingface.co/lightx2v/LightVAE/tree/main). Keep checkpoints matched to their backbone. |
|
|
| > **Wan2.2 model selection:** 14B models use the **Wan2.1 VAE**; TI2V-5B uses the **Wan2.2 VAE**. |
|
|
| ## 🧩 ComfyUI · MiniMax-H3 |
|
|
| The **Load H3 LightVAE** node reuses ComfyUI’s native H3 VAE. No separate LightVAE package installation, TLR or custom Triton kernels are needed. The current node supports **H3 only**. |
|
|
| ### 1. Install |
|
|
| Use a ComfyUI version with native MiniMax-H3 support. Run these commands in the Python environment that starts ComfyUI: |
|
|
| ```bash |
| cd /path/to/ComfyUI |
| |
| git clone https://github.com/ModelTC/LightVAE.git LightVAE |
| python LightVAE/comfyui/install.py --comfy-root . |
| |
| python -m pip install huggingface_hub |
| hf download lightx2v/LightVAE \ |
| lightvae-lite-h3-decoder.safetensors \ |
| lightvae-lite-h3-encoder.safetensors \ |
| --local-dir models/vae/LightVAE |
| ``` |
|
|
| Already cloned? Run `git -C LightVAE pull`, then rerun the installer. |
|
|
| The installer creates `custom_nodes/ComfyUI-LightVAE/` automatically. **Do not clone the entire repository directly into `custom_nodes`.** |
| |
| The original H3 video VAE is required. Download it if needed: |
| |
| ```bash |
| hf download Comfy-Org/MiniMax-H3 \ |
| vae/minimax_h3_video_vae_fp16.safetensors \ |
| --local-dir models |
| ``` |
| |
| ### 2. Connect |
| |
| Restart ComfyUI and search for **Load H3 LightVAE**. |
| |
| Load the **original H3 video VAE** with the standard **Load VAE** node and connect it to **`base_vae`**. Select Lite components in **Load H3 LightVAE**, not in the standard Load VAE node. |
|
|
| - **FL2V:** replace only the decoder. Leave `encoder = none`, keep the original VAE connected to first/last-frame encoding, and use LightVAE for **VAE Decode**. |
| - **REF2V:** replace the encoder, decoder, or both. Connect LightVAE to reference encoding and **VAE Decode**. |
| - **`none`:** use the corresponding original component. |
|
|
| **The audio VAE stays unchanged.** |
|
|
| Standard H3 checkpoints and our previous `*-h3-*-comfy.safetensors` exports are both accepted; downloading both formats is unnecessary. |
|
|
| ### 3. Load a workflow |
|
|
| Download a workflow JSON and drag it into ComfyUI: |
|
|
| [🎞️ First + last frame](https://github.com/ModelTC/LightVAE/blob/main/comfyui/workflows/h3_fl2v_lightvae.json) · [🖼️ Image references](https://github.com/ModelTC/LightVAE/blob/main/comfyui/workflows/h3_ref_image_lightvae.json) · [🎬 Video reference](https://github.com/ModelTC/LightVAE/blob/main/comfyui/workflows/h3_ref_video_lightvae.json) |
|
|
| Select your model files, upload the inputs and run. These are base-model workflows, not 4-/8-step turbo-LoRA workflows. |
|
|
| See the [workflow guide](https://github.com/ModelTC/LightVAE/tree/main/comfyui/workflows) for required diffusion models, text encoders and input preparation. |
|
|
| ### Optional · ConvRot-INT8 decoder |
|
|
| Update LightVAE, rerun the installer and download the quantized decoder: |
|
|
| ```bash |
| hf download lightx2v/LightVAE \ |
| lightvae-lite-h3-decoder-int8-convrot.safetensors \ |
| --local-dir /path/to/ComfyUI/models/vae/LightVAE |
| ``` |
|
|
| Restart ComfyUI and select this file in **Load H3 LightVAE → decoder**. Keep `base_vae`, encoder selection and audio VAE unchanged. |
|
|
| Native NVIDIA CUDA acceleration requires a **CUDA 13.0+ PyTorch build** and a compatible driver. Backend support and performance depend on your environment. |
|
|
| > This INT8 checkpoint is **ComfyUI-only**. Use standard, non-INT8 checkpoints with the standalone Python scripts. |
|
|
| ## 🐍 Python · Video Reconstruction |
|
|
| Example: **original Wan2.1 encoder + LightVAE-Lite decoder**. |
|
|
| ```bash |
| git clone https://github.com/ModelTC/LightVAE.git |
| cd LightVAE |
| python -m pip install -r requirements.txt |
| |
| hf download lightx2v/LightVAE \ |
| lightvae-lite-wan21-decoder.safetensors \ |
| --local-dir weights/LightVAE |
| |
| hf download Wan-AI/Wan2.1-T2V-1.3B \ |
| Wan2.1_VAE.pth --local-dir weights/Wan2.1 |
| |
| python infer.py \ |
| --model wan21 --variant lite \ |
| --encoder weights/Wan2.1/Wan2.1_VAE.pth \ |
| --decoder weights/LightVAE/lightvae-lite-wan21-decoder.safetensors \ |
| --input demo.mp4 --frames 81 --height 480 --width 832 \ |
| --output outputs/wan21_lite_recon.mp4 |
| ``` |
|
|
| `infer.py` accepts independent encoder and decoder checkpoints and saves a reconstructed video with a JSON report. |
|
|
| Find Wan2.2 and H3 examples, benchmarks and visual comparisons in [LightVAE](https://github.com/ModelTC/LightVAE). For video generation with LightVAE, see [LightX2V](https://github.com/ModelTC/LightX2V). |
|
|
| ## ⭐ Support |
|
|
| If LightVAE helps your work, please give [LightVAE](https://github.com/ModelTC/LightVAE) and [LightX2V](https://github.com/ModelTC/LightX2V) a star! |
