| --- |
| license: apache-2.0 |
| pipeline_tag: robotics |
| tags: |
| - robotics |
| - vision-language-action |
| - vla |
| - imle |
| - openpi |
| - pi0.5 |
| - droid |
| - franka |
| --- |
| |
| # IMLE-VLA checkpoints (IROS 2026) |
|
|
| Checkpoints for **IMLE-VLA: Fast Single-Step Action Generation for Vision-Language-Action Policies** |
| ([paper](https://arxiv.org/abs/2609.10915), [project page](https://kianhk6.github.io/IMLE-VLA/), |
| [code](https://github.com/kianhk6/IMLE-VLA)). |
|
|
| <video controls muted playsinline width="100%" src="https://huggingface.co/kinster/IMLE-VLA/resolve/main/media/imle_vla_teaser.mp4"></video> |
|
|
| *Five-minute teaser: the setting, why regression collapses, why flow matching is slow and jerky, IMLE-VLA on the real robot, and how conditional IMLE works.* |
|
|
| IMLE-VLA replaces the 10-step flow-matching action head of Οβ.β
with a single-step generator trained with conditional |
| IMLE. Same frozen VLM backbone, one forward pass per action chunk, 55 Hz instead of 15 Hz on an NVIDIA L40S. |
|
|
| ## Contents |
|
|
| | path | what | size | |
| |---|---|---| |
| | `droid/jax/57000/` | JAX (orbax) checkpoint of the real-robot DROID model: `params/` + `assets/droid/norm_stats.json`. Training config `16_horrizon_05_lr_2_factor_96_batch`, step 57000. Serve with `scripts/serve_policy.py policy:checkpoint`. | 6.6 GB | |
| | `droid/triton/` | The same weights converted for the single-step Triton engine (`pi05_imle_droid_57000.pkl`) plus `assets/droid/norm_stats.json`, the PaliGemma tokenizer and a provenance copy of the engine. This is the 55 Hz path used on the robot. | 6.3 GB | |
|
|
| The DROID model: Οβ.β
fine-tuned on DROID with the cIMLE objective (sample factor m = 2), frozen VLM, trained action |
| expert only, joint-position action space, 16-step action chunks, two camera views (one exterior + wrist). It is the policy |
| behind the real-robot experiments in the paper (Franka Emika Panda, four tabletop tasks, 55 Hz at execution horizon 12). |
|
|
| ## Real-robot videos |
|
|
| Franka Emika Panda, both policies trained on DROID: Οβ.β
at 15 Hz versus IMLE-VLA at 55 Hz (the `droid/` checkpoint). |
|
|
| **Reactive task: pineapple on a moving plate.** A remote-controlled car moves the plate, so the target is in continuous motion. |
|
|
| | IMLE-VLA | Οβ.β
| |
| |---|---| |
| | <video controls muted playsinline width="100%" src="https://huggingface.co/kinster/IMLE-VLA/resolve/main/media/reactive_imle_vla_success.mp4"></video> | <video controls muted playsinline width="100%" src="https://huggingface.co/kinster/IMLE-VLA/resolve/main/media/reactive_pi05_fail.mp4"></video> | |
|
|
| **Static tasks** (side by side: Οβ.β
left, IMLE-VLA right) |
|
|
| Swap pineapple and cube |
|
|
| <video controls muted playsinline width="100%" src="https://huggingface.co/kinster/IMLE-VLA/resolve/main/media/swap_pineapple_cube.mp4"></video> |
|
|
| Pineapple in bowl |
|
|
| <video controls muted playsinline width="100%" src="https://huggingface.co/kinster/IMLE-VLA/resolve/main/media/pineapple_in_bowl.mp4"></video> |
|
|
| Pineapple in cabinet |
|
|
| <video controls muted playsinline width="100%" src="https://huggingface.co/kinster/IMLE-VLA/resolve/main/media/pineapple_in_cabinet.mp4"></video> |
|
|
| ## Use |
|
|
| ```bash |
| git clone --recurse-submodules https://github.com/kianhk6/IMLE-VLA.git && cd IMLE-VLA/openpi # install per openpi/README.md |
| pip install -U huggingface_hub |
| |
| # JAX checkpoint |
| huggingface-cli download kinster/IMLE-VLA --include "droid/jax/*" --local-dir ./ckpt |
| python scripts/serve_policy.py --env=DROID --port=8000 policy:checkpoint \ |
| --policy.config=16_horrizon_05_lr_2_factor_96_batch \ |
| --policy.dir=./ckpt/droid/jax/57000 \ |
| --policy.sample-kwargs='{"num_steps": 1}' |
| |
| # Triton package (55 Hz) |
| huggingface-cli download kinster/IMLE-VLA --include "droid/triton/*" --local-dir ./ckpt |
| scripts/serve_triton.sh ./ckpt/droid/triton 16_horrizon_05_lr_2_factor_96_batch 0 8000 DROID |
| ``` |
|
|
| Then run the DROID robot client (`examples/droid/main.py --open_loop_horizon 12`) or score the served policy against |
| ground-truth DROID actions with `scripts/validate_triton_droid.py`. Both are documented in `openpi/README.md` and `openpi/docs/triton.md`. |
|
|
| ## LIBERO |
|
|
| `libero/imle_vla_libero_h10_{spatial,object,goal,long,overall}` and `libero/imle_vla_libero_h30_{...}` hold LeRobot |
| checkpoints (`pretrained_model/`), per suite and overall, for execution horizon 10 and 30. Train and evaluate them with the |
| LeRobot code in the repository's [`lerobot/`](https://github.com/kianhk6/IMLE-VLA/tree/main/lerobot) folder. |
|
|
| ## Citation |
|
|
| ```bibtex |
| @inproceedings{Hosseinkhani2026IMLEVLAFS, |
| title = {{IMLE-VLA}: Fast Single-Step Action Generation for Vision-Language-Action Policies}, |
| author = {Hosseinkhani, Kian and Peng, Qinhe and Shramko, George and Aghabozorgi, Mehran and |
| Qian, Jianing and Engst, Tristan and Moazeni, Alireza and Jayaraman, Dinesh and Li, Ke}, |
| booktitle = {IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)}, |
| year = {2026}, |
| eprint = {2609.10915}, |
| archivePrefix = {arXiv}, |
| url = {https://arxiv.org/abs/2609.10915} |
| } |
| ``` |
|
|
| ## License |
|
|
| Apache 2.0, like the [openpi](https://github.com/Physical-Intelligence/openpi) code and Οβ.β
checkpoints these weights |
| are fine-tuned from. The PaliGemma backbone is subject to the |
| [Gemma Terms of Use](https://ai.google.dev/gemma/terms). |
|
|
