Update model card
Browse files
README.md
CHANGED
|
@@ -1,12 +1,41 @@
|
|
| 1 |
---
|
| 2 |
-
|
| 3 |
-
tags:
|
|
|
|
|
|
|
|
|
|
|
|
|
| 4 |
---
|
|
|
|
| 5 |
# RoboTryOn
|
| 6 |
|
| 7 |
-
Checkpoints for
|
|
|
|
|
|
|
| 8 |
|
| 9 |
-
|
|
| 10 |
|---|---|---|
|
| 11 |
-
| `image/` |
|
| 12 |
-
| `video/` |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
---
|
| 2 |
+
pipeline_tag: robotics
|
| 3 |
+
tags:
|
| 4 |
+
- robotics
|
| 5 |
+
- human-to-robot
|
| 6 |
+
- video-generation
|
| 7 |
+
- inverse-dynamics
|
| 8 |
---
|
| 9 |
+
|
| 10 |
# RoboTryOn
|
| 11 |
|
| 12 |
+
Checkpoints for **Synthesizing Dexterous Robot Demonstrations from Human Videos via Virtual Try-On**.
|
| 13 |
+
|
| 14 |
+
[Paper](https://arxiv.org/abs/2611.05163) 路 [Project page](https://ryan-seungyong-lee.github.io/RoboTryOn) 路 [Code](https://github.com/ryan-seungyong-lee/RoboTryOn)
|
| 15 |
|
| 16 |
+
| Folder | Model | Base |
|
| 17 |
|---|---|---|
|
| 18 |
+
| `image/` | Keyframe translation (LoRA) | [FLUX.2-klein-base-9B](https://huggingface.co/black-forest-labs/FLUX.2-klein-base-9B) |
|
| 19 |
+
| `video/` | Video synthesis (in-context LoRA) | [LTX-2.5-Diffusers](https://huggingface.co/Lightricks/LTX-2.5-Diffusers) |
|
| 20 |
+
| `idm/` | Inverse dynamics model, shared by all embodiments | [Depth-Anything-V2-Large](https://huggingface.co/depth-anything/Depth-Anything-V2-Large-hf) |
|
| 21 |
+
|
| 22 |
+
All models are trained on teleoperated episodes of OpenArm, RB-Y1 and GR-1.
|
| 23 |
+
|
| 24 |
+
## Usage
|
| 25 |
+
|
| 26 |
+
```bash
|
| 27 |
+
hf download RyanL22/RoboTryOn --local-dir ckpt
|
| 28 |
+
python scripts/run_pipeline.py --config configs/pipeline.yaml ... \
|
| 29 |
+
checkpoints.image=ckpt/image checkpoints.video=ckpt/video checkpoints.idm=ckpt/idm
|
| 30 |
+
```
|
| 31 |
+
|
| 32 |
+
## Citation
|
| 33 |
+
|
| 34 |
+
```bibtex
|
| 35 |
+
@article{lee2026robotryon,
|
| 36 |
+
title={Synthesizing Dexterous Robot Demonstrations from Human Videos via Virtual Try-On},
|
| 37 |
+
author={Lee, Seungyong and Jang, Suhyeok and Kim, Minyoung and Shin, Jinwoo},
|
| 38 |
+
journal={arXiv preprint arXiv:2611.05163},
|
| 39 |
+
year={2026}
|
| 40 |
+
}
|
| 41 |
+
```
|