JEdward commited on
Commit
94dd3ca
·
verified ·
1 Parent(s): 6de848e

Create README.md

Browse files
Files changed (1) hide show
  1. README.md +76 -0
README.md ADDED
@@ -0,0 +1,76 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ tags:
3
+ - robotics
4
+ - world-model
5
+ - action-conditioned-video-generation
6
+ - robot-manipulation
7
+ - coachworld
8
+ - arxiv:2609.39685
9
+ ---
10
+
11
+ # CoachWorld — Pretrained Robot World Model
12
+
13
+ [**Paper**](https://arxiv.org/abs/2609.39685) · [**Code**](https://github.com/RoboCoach-AI/CoachWorld) · [**Project Page & Videos**](https://robocoach-ai.github.io/)
14
+
15
+ **CoachWorld predicts future robot observations conditioned on visual history and robot actions.** It is the world-model component of **RoboCoach: World Models as Active Coaches for Compositional Robot Skills**.
16
+
17
+ This repository hosts the **CoachWorld pretrained checkpoint**, trained on heterogeneous single-arm and dual-arm robot data.
18
+
19
+ ## Overview
20
+
21
+ CoachWorld is an action-conditioned video world model built on an adapted Wan2.2 TI backbone. It uses visual history and end-effector action conditioning to generate future observations, with camera and action conventions that support learning across robot datasets.
22
+
23
+ The model supports autoregressive rollout: generated observations become part of the context for subsequent predictions. In the full RoboCoach framework, skill policies interact with the world model, while a separate progress judge identifies subtask failures. These diagnoses guide targeted demonstration collection and policy updates.
24
+
25
+ ## Model at a Glance
26
+
27
+ | Item | Description |
28
+ | --- | --- |
29
+ | Release | CoachWorld pretrain |
30
+ | Model type | Action-conditioned video world model |
31
+ | Backbone | Adapted Wan2.2 |
32
+ | Conditioning | Visual history and robot end-effector actions, using the prescribed camera/action representation |
33
+ | Output | Predicted future RGB observations |
34
+ | Rollout mode | Autoregressive video prediction |
35
+ | Training scope | Heterogeneous single-arm and dual-arm robot data |
36
+ | Implementation | Custom `coachworld` Python package |
37
+
38
+ ## Getting Started
39
+
40
+ Download the weights from the [Files and versions](https://huggingface.co/JEdward/CoachWorld/tree/main) tab and use the [CoachWorld code repository](https://github.com/RoboCoach-AI/CoachWorld) for environment setup and model integration.
41
+
42
+ The implementation includes:
43
+
44
+ - Model conditioning, inference, and training components.
45
+ - Modified Wan2.2 model and VAE components.
46
+ - Video-latent data interfaces and action/camera conventions.
47
+ - Camera and robot geometry helpers.
48
+ - A distributed training entry point and example configuration.
49
+
50
+ **Input preparation matters.** Robot actions must follow the implementation’s coordinate-frame, normalization, and temporal-sampling conventions. Raw joint commands or pixel-space trajectories are not interchangeable with the model’s expected conditioning.
51
+
52
+ Use the CoachWorld implementation to load and run this checkpoint; this release does not claim compatibility with a generic Transformers or Diffusers loading pipeline.
53
+
54
+ ## Citation
55
+
56
+ If you use CoachWorld in your research, please cite:
57
+
58
+ ```bibtex
59
+ @article{liu2026robocoach,
60
+ title={RoboCoach: World Models as Active Coaches for Compositional Robot Skills},
61
+ author={Liu, Jiajun and Chen, Yifan and Liu, Yichao and Zhang, Jiayi
62
+ and Chen, Ruoqu and Xie, Shaoxuan and Yao, Guocai
63
+ and Xu, Mengdi and Cui, Sen and Zhang, Changshui},
64
+ journal={arXiv preprint arXiv:2609.39685},
65
+ year={2026},
66
+ url={https://arxiv.org/abs/2609.39685}
67
+ }
68
+ ```
69
+
70
+ ## Acknowledgments
71
+
72
+ CoachWorld builds on Wan2.2 and other open-source components. See the code repository’s [third-party notices](https://github.com/RoboCoach-AI/CoachWorld/blob/main/THIRD_PARTY.md) for component attribution and applicable terms.
73
+
74
+ ## Contact
75
+
76
+ For implementation questions and reproducibility issues, please open an issue in the [CoachWorld repository](https://github.com/RoboCoach-AI/CoachWorld/issues).