--- license: mit base_model: OpenGVLab/InternVL3-1B-hf pipeline_tag: robotics tags: - vision-language-action - robotics - robot-manipulation - libero - calvin --- # ๐Ÿธ LEAP-VLA **Latent-Enhanced Action Prototyping via Continuous Residual Latent Spaces for Vision-Language-Action Models** (ECCV 2026) Bo-Yun Yu ยท Kuan-Chuan Peng ยท Jun-Wei Hsieh [**๐Ÿ’ป Code (GitHub)**](https://github.com/AndersonYu7/LEAP-VLA) ยท [**๐Ÿ“„ Paper**](https://eccv.ecva.net/virtual/2026/poster/3297) ยท [**โšก Quick start**](https://github.com/AndersonYu7/LEAP-VLA#-quick-start) Pretrained Stage-1 (MSRQ) and Stage-2 (Latent Action Aligner + LoRA on InternVL3-1B) checkpoints for every benchmark in the paper. Installation, training, evaluation and the inference API are in the [GitHub repository](https://github.com/AndersonYu7/LEAP-VLA). ## Checkpoints | Folder | Evaluated on | Result | |---|---|:--:| | `libero_spatial/` | LIBERO-Spatial ยท LIBERO-plus (spatial) | 98.4% | | `libero_object/` | LIBERO-Object ยท LIBERO-plus (object) | 99.4% | | `libero_goal/` | LIBERO-Goal ยท LIBERO-plus (goal) | 97.6% | | `libero_10/` | LIBERO-Long ยท LIBERO-plus (10) | 93.6% | | `calvin/` | CALVIN ABCโ†’D | 4.28 avg len | LIBERO-plus total (all 10,030 perturbed tasks): 78.3%. Full tables and logs: [Results](https://github.com/AndersonYu7/LEAP-VLA#-results). ``` / โ”œโ”€โ”€ bridge/ # Stage-2 โ†’ --bridge_checkpoint โ”‚ โ”œโ”€โ”€ bridge.pt # Latent Action Aligner โ”‚ โ”œโ”€โ”€ vlm_lora/ # LoRA adapter for OpenGVLab/InternVL3-1B-hf โ”‚ โ”œโ”€โ”€ config.json โ”‚ โ”œโ”€โ”€ dataset_statistics.json # action de-normalization โ”‚ โ””โ”€โ”€ optimizer.pt ยท scheduler.pt ยท training_state.pt # only needed to resume training โ””โ”€โ”€ stage1/best.pt # Stage-1 MSRQ โ†’ --stage1_checkpoint ``` ## Usage ```bash git clone https://github.com/AndersonYu7/LEAP-VLA.git && cd LEAP-VLA conda create -n leap_vla python=3.10 -y && conda activate leap_vla bash install.sh base hf download Boyun7/LEAP-VLA --include "libero_spatial/*" --exclude "*/optimizer.pt" --local-dir checkpoints python scripts/inference.py \ --bridge_checkpoint checkpoints/libero_spatial/bridge \ --stage1_checkpoint checkpoints/libero_spatial/stage1/best.pt \ --image assets/quickstart/libero_spatial_agentview.png \ --wrist_image assets/quickstart/libero_spatial_wrist.png \ --instruction "pick up the black bowl between the plate and the ramekin and place it on the plate" ``` ```python from leap.policy import LEAPPolicy policy = LEAPPolicy("checkpoints/libero_spatial/bridge", "checkpoints/libero_spatial/stage1/best.pt") actions = policy.predict(["agentview.png", "wrist.png"], "put the bowl on the plate") # (8, 7) action chunk ``` Benchmark evaluation (LIBERO, LIBERO-plus, CALVIN): [docs/](https://github.com/AndersonYu7/LEAP-VLA/tree/main/docs). ## Citation ```bibtex @inproceedings{yu2026leapvla, title = {LEAP-VLA: Latent-Enhanced Action Prototyping via Continuous Residual Latent Spaces for Vision-Language-Action Models}, author = {Yu, Bo-Yun and Peng, Kuan-Chuan and Hsieh, Jun-Wei}, booktitle = {Proceedings of the European Conference on Computer Vision (ECCV)}, year = {2026}, url = {https://eccv.ecva.net/virtual/2026/poster/3297} } ```