lightx2v commited on
Commit
dceba4d
·
verified ·
1 Parent(s): bdfd1e9

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +47 -2
README.md CHANGED
@@ -4,8 +4,53 @@ tags:
4
  - diffusion-single-file
5
  - distillation
6
  - video genration
 
 
 
 
 
7
  pipeline_tags:
8
  - image-to-video
9
  - text-to-video
10
- library_name: diffusers
11
- ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
4
  - diffusion-single-file
5
  - distillation
6
  - video genration
7
+ - video-vae
8
+ - video-generation
9
+ - wan2.1
10
+ - wan2.2
11
+ - minimax-h3
12
  pipeline_tags:
13
  - image-to-video
14
  - text-to-video
15
+ ---
16
+
17
+ # LightVAE: Towards Compact and Efficient Video Autoencoders
18
+
19
+ LightVAE accelerates pretrained video autoencoders using temporal low-rank compression and block pruning. It provides efficient checkpoints for Wan2.1, Wan2.2, and MiniMax-H3 while preserving their latent interfaces.
20
+
21
+ **Paper:** *LightVAE: Towards Compact and Efficient Video Autoencoders*
22
+ **Code and examples:** [ModelTC/LightVAE](https://github.com/ModelTC/LightVAE)
23
+
24
+ ## Checkpoints
25
+
26
+ | Backbone | Available checkpoints |
27
+ | --- | --- |
28
+ | Wan2.1 | `lightvae-pro-wan21-decoder.safetensors`, `lightvae-lite-wan21-decoder.safetensors` |
29
+ | Wan2.2 | `lightvae-pro-wan22-decoder.safetensors`, `lightvae-lite-wan22-decoder.safetensors` |
30
+ | MiniMax-H3 | `lightvae-lite-h3-decoder.safetensors`, `lightvae-lite-h3-encoder.safetensors` |
31
+
32
+ **Pro** retains the original decoder depth for higher reconstruction fidelity. **Lite** applies additional pruning for faster decoding. For Wan2.1 and Wan2.2, use the original VAE encoder with a LightVAE decoder. MiniMax-H3 can use either its original encoder or the provided Lite encoder.
33
+
34
+ ## Quick start
35
+
36
+ ```bash
37
+ git clone https://github.com/ModelTC/LightVAE.git
38
+ cd LightVAE
39
+ pip install -r requirements.txt
40
+ hf download lightx2v/LightVAE --local-dir weights/LightVAE
41
+ ```
42
+
43
+ For example, reconstruct a video with the Wan2.1 Lite decoder:
44
+
45
+ ```bash
46
+ hf download Wan-AI/Wan2.1-T2V-1.3B Wan2.1_VAE.pth --local-dir weights/Wan2.1
47
+
48
+ python infer.py \
49
+ --model wan21 --variant lite \
50
+ --encoder weights/Wan2.1/Wan2.1_VAE.pth \
51
+ --decoder weights/LightVAE/lightvae-lite-wan21-decoder.safetensors \
52
+ --input demo.mp4 --frames 81 --height 480 --width 832 \
53
+ --output outputs/wan21_lite_recon.mp4
54
+ ```
55
+
56
+ See the [GitHub repository](https://github.com/ModelTC/LightVAE) for Wan2.2 and MiniMax-H3 examples, benchmarks, and visual comparisons.