Video-to-Video
PyTorch
byeongjun-park commited on
Commit
a9a7638
·
verified ·
1 Parent(s): 8bc1a5b

Add stage 1/2/3 checkpoints, config.json, and model card

Browse files
README.md CHANGED
@@ -1,3 +1,42 @@
1
  ---
2
  license: apache-2.0
 
 
 
 
 
 
 
 
 
3
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  license: apache-2.0
3
+ datasets:
4
+ - KlingTeam/MultiCamVideo-Dataset
5
+ - KlingTeam/SynCamVideo-Dataset
6
+ base_model:
7
+ - alibaba-pai/Wan2.1-Fun-V1.1-1.3B-Control-Camera
8
+ base_model_relation: finetune
9
+ pipeline_tag: video-to-video
10
+ tags:
11
+ - pytorch
12
  ---
13
+
14
+ This model was presented in [FlashRender: Few-Step Generative Rendering via Camera-Controlled Video MeanFlow](https://huggingface.co/papers/2609.03563).
15
+
16
+ Please refer to the [Github](https://github.com/byeongjun-park/FlashRender) README for usage.
17
+
18
+ ## Checkpoints
19
+
20
+ | Stage | File | Sampling |
21
+ |:---:|---|---|
22
+ | 1 | `epoch_20.safetensors` | 50 steps, `inference_mode=mul` |
23
+ | 2 | `meanflow_epoch_20.safetensors` | 4 steps, `inference_mode=any` |
24
+ | 3 | `onpolicy_epoch_5.safetensors` | 4 steps, `inference_mode=any` |
25
+
26
+ **`onpolicy_epoch_5.safetensors` is the final model** and the default in `configs/base.yaml`.
27
+
28
+ Each file stores only the **trainable parameter subset**, not a stand-alone diffusers/transformers
29
+ checkpoint. Weights are `bfloat16`, except the relative-pose tokens and the per-block RoPE phase MLPs
30
+ (`rel_pose_{src,tgt}_token`, `self_attn.rope_phase_{qk,vo}`), which are stored in `float32`.
31
+ They are loaded with `strict=False` onto the base Wan DiT after it is patched by
32
+ `flashrender_utils.model_utils.adjust_to_FlashRender`, so use the code repository:
33
+
34
+ ```python
35
+ from huggingface_hub import snapshot_download
36
+
37
+ snapshot_download(
38
+ repo_id="byeongjun-park/FlashRender",
39
+ local_dir="models/checkpoints",
40
+ allow_patterns=["*.safetensors", "config.json"],
41
+ )
42
+ ```
config.json ADDED
@@ -0,0 +1,33 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "model_name": "FlashRender",
3
+ "description": "Few-step generative rendering via camera-controlled video MeanFlow.",
4
+ "task": "video-to-video",
5
+ "license": "apache-2.0",
6
+ "code_repository": "https://github.com/byeongjun-park/FlashRender",
7
+ "base_model": "alibaba-pai/Wan2.1-Fun-V1.1-1.3B-Control-Camera",
8
+ "torch_dtype": "bfloat16",
9
+ "height": 480,
10
+ "width": 832,
11
+ "num_frames": 81,
12
+ "state_dict_format": "partial (trainable subset only); load with strict=False onto the base Wan DiT patched by flashrender_utils.model_utils.adjust_to_FlashRender",
13
+ "checkpoints": {
14
+ "epoch_20.safetensors": {
15
+ "stage": 1,
16
+ "training": "multi-step flow matching (t = r) with RETA feature alignment",
17
+ "num_inference_steps": 50,
18
+ "inference_mode": "mul"
19
+ },
20
+ "meanflow_epoch_20.safetensors": {
21
+ "stage": 2,
22
+ "training": "few-step MeanFlow distillation, resumed from stage 1",
23
+ "num_inference_steps": 4,
24
+ "inference_mode": "any"
25
+ },
26
+ "onpolicy_epoch_5.safetensors": {
27
+ "stage": 3,
28
+ "training": "on-policy flow-map distillation (DMD/VSD + GAN) of the stage-2 student",
29
+ "num_inference_steps": 4,
30
+ "inference_mode": "any"
31
+ }
32
+ }
33
+ }
epoch_20.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b5b6bf14b6e45eb74cc3a29c47e52e7b1e2f71faf84e08ba0f9b8ab8657f58f8
3
+ size 1543319280
meanflow_epoch_20.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:17928eb068db1a118d00d0c82275c8d1094813250ff7133b7df213ba9efa8168
3
+ size 1582672592
onpolicy_epoch_5.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6152ef1bc25fc664674273e628e5a21dbdd9fe9f7a77264c865c3ade8e100a7c
3
+ size 1657823568