|
Download README.md from Boyun7/LEAP-VLA: direct link, hf CLI and curl.
- Browser
- Download file 3.36 kB
-
https://huggingface.co/Boyun7/LEAP-VLA/resolve/main/README.md
- Command line
-
hf download hf://Boyun7/LEAP-VLA/README.md
-
curl -L -o README.md https://huggingface.co/Boyun7/LEAP-VLA/resolve/main/README.md
3.36 kB
| license: mit | |
| base_model: OpenGVLab/InternVL3-1B-hf | |
| pipeline_tag: robotics | |
| tags: | |
| - vision-language-action | |
| - robotics | |
| - robot-manipulation | |
| - libero | |
| - calvin | |
| # 🐸 LEAP-VLA | |
| **Latent-Enhanced Action Prototyping via Continuous Residual Latent Spaces for Vision-Language-Action Models** (ECCV 2026) | |
| Bo-Yun Yu · Kuan-Chuan Peng · Jun-Wei Hsieh | |
| [**💻 Code (GitHub)**](https://github.com/AndersonYu7/LEAP-VLA) · | |
| [**📄 Paper**](https://eccv.ecva.net/virtual/2026/poster/3297) · | |
| [**⚡ Quick start**](https://github.com/AndersonYu7/LEAP-VLA#-quick-start) | |
| Pretrained Stage-1 (MSRQ) and Stage-2 (Latent Action Aligner + LoRA on InternVL3-1B) checkpoints for | |
| every benchmark in the paper. Installation, training, evaluation and the inference API are in the | |
| [GitHub repository](https://github.com/AndersonYu7/LEAP-VLA). | |
| ## Checkpoints | |
| | Folder | Evaluated on | Result | | |
| |---|---|:--:| | |
| | `libero_spatial/` | LIBERO-Spatial · LIBERO-plus (spatial) | 98.4% | | |
| | `libero_object/` | LIBERO-Object · LIBERO-plus (object) | 99.4% | | |
| | `libero_goal/` | LIBERO-Goal · LIBERO-plus (goal) | 97.6% | | |
| | `libero_10/` | LIBERO-Long · LIBERO-plus (10) | 93.6% | | |
| | `calvin/` | CALVIN ABC→D | 4.28 avg len | | |
| LIBERO-plus total (all 10,030 perturbed tasks): 78.3%. Full tables and logs: | |
| [Results](https://github.com/AndersonYu7/LEAP-VLA#-results). | |
| ``` | |
| <benchmark>/ | |
| ├── bridge/ # Stage-2 → --bridge_checkpoint | |
| │ ├── bridge.pt # Latent Action Aligner | |
| │ ├── vlm_lora/ # LoRA adapter for OpenGVLab/InternVL3-1B-hf | |
| │ ├── config.json | |
| │ ├── dataset_statistics.json # action de-normalization | |
| │ └── optimizer.pt · scheduler.pt · training_state.pt # only needed to resume training | |
| └── stage1/best.pt # Stage-1 MSRQ → --stage1_checkpoint | |
| ``` | |
| ## Usage | |
| ```bash | |
| git clone https://github.com/AndersonYu7/LEAP-VLA.git && cd LEAP-VLA | |
| conda create -n leap_vla python=3.10 -y && conda activate leap_vla | |
| bash install.sh base | |
| hf download Boyun7/LEAP-VLA --include "libero_spatial/*" --exclude "*/optimizer.pt" --local-dir checkpoints | |
| python scripts/inference.py \ | |
| --bridge_checkpoint checkpoints/libero_spatial/bridge \ | |
| --stage1_checkpoint checkpoints/libero_spatial/stage1/best.pt \ | |
| --image assets/quickstart/libero_spatial_agentview.png \ | |
| --wrist_image assets/quickstart/libero_spatial_wrist.png \ | |
| --instruction "pick up the black bowl between the plate and the ramekin and place it on the plate" | |
| ``` | |
| ```python | |
| from leap.policy import LEAPPolicy | |
| policy = LEAPPolicy("checkpoints/libero_spatial/bridge", "checkpoints/libero_spatial/stage1/best.pt") | |
| actions = policy.predict(["agentview.png", "wrist.png"], "put the bowl on the plate") # (8, 7) action chunk | |
| ``` | |
| Benchmark evaluation (LIBERO, LIBERO-plus, CALVIN): | |
| [docs/](https://github.com/AndersonYu7/LEAP-VLA/tree/main/docs). | |
| ## Citation | |
| ```bibtex | |
| @inproceedings{yu2026leapvla, | |
| title = {LEAP-VLA: Latent-Enhanced Action Prototyping via Continuous Residual Latent Spaces for Vision-Language-Action Models}, | |
| author = {Yu, Bo-Yun and Peng, Kuan-Chuan and Hsieh, Jun-Wei}, | |
| booktitle = {Proceedings of the European Conference on Computer Vision (ECCV)}, | |
| year = {2026}, | |
| url = {https://eccv.ecva.net/virtual/2026/poster/3297} | |
| } | |
| ``` | |