YixiangChen commited on
Commit
689cce6
·
verified ·
1 Parent(s): dd214aa

Add model card (RoboTwin flow-action IDM checkpoint + inference config)

Browse files
Files changed (1) hide show
  1. README.md +64 -3
README.md CHANGED
@@ -1,3 +1,64 @@
1
- ---
2
- license: apache-2.0
3
- ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ library_name: diffsynth
4
+ pipeline_tag: robotics
5
+ tags:
6
+ - robotics
7
+ - manipulation
8
+ - robotwin
9
+ - world-model
10
+ - flow-matching
11
+ - imitation-learning
12
+ ---
13
+
14
+ # FlowWAM
15
+
16
+ Checkpoints for **FlowWAM** — a dual-stream (RGB + optical-flow) Wan2.2 world
17
+ model paired with an IDM action expert for RoboTwin dual-arm manipulation.
18
+
19
+ - **Code:** https://github.com/YixiangChen515/FlowWAM
20
+ - **Dataset:** https://huggingface.co/datasets/YixiangChen/FlowWAM_RoboTwin
21
+
22
+ ## Files
23
+
24
+ | File | Description |
25
+ |------|-------------|
26
+ | `flowwam_robotwin.safetensors` | RoboTwin dual-stream flow-action **IDM** checkpoint (video DiT + flow_stream + action expert). |
27
+ | `flowwam_robotwin_action_norm_stats.npz` | 14-D action normalization stats (mean/std). **Required for inference.** |
28
+ | `flowwam_worldarena_stage1.safetensors` | FlowWAM WorldArena world-model checkpoint. |
29
+
30
+ ## RoboTwin flow-action IDM checkpoint
31
+
32
+ `flowwam_robotwin.safetensors` jointly denoises an RGB stream and a head-camera
33
+ optical-flow stream, and predicts a 14-D dual-arm action chunk by cross-attending
34
+ to the video DiT's per-layer hidden states. It is trained on the RoboTwin 2.0
35
+ `aloha-agilex` demonstrations (see the dataset repo above).
36
+
37
+ ### Inference
38
+
39
+ Keep `flowwam_robotwin.safetensors` and `flowwam_robotwin_action_norm_stats.npz`
40
+ **in the same folder** — the server derives the norm-stats path from the
41
+ checkpoint path. The decoding config must match the training config:
42
+
43
+ | Setting | Value |
44
+ |---|---|
45
+ | `num_frames` | 33 |
46
+ | `num_video_frames` | 9 |
47
+ | `num_action_layers` | 30 |
48
+ | `action_pred_target` | velocity |
49
+ | `action_pos_mode` | rope |
50
+ | `proprio_mode` | text |
51
+ | `action_snr_shift` | 5.0 |
52
+ | `action_dim` | 14 |
53
+
54
+ ```bash
55
+ # From the FlowWAM code repo (https://github.com/YixiangChen515/FlowWAM)
56
+ cd inference
57
+ CHECKPOINT=/path/to/flowwam_robotwin.safetensors bash start_server.sh
58
+ ```
59
+
60
+ See the code repo's `inference/README.md` for the full RoboTwin evaluation setup.
61
+
62
+ ## License
63
+
64
+ Released under the Apache-2.0 License.