khanhnd61/impact_libero_object

IMPACT — libero_object

Instruction-Modulated Perception + ACTion chunking, trained from scratch on the libero_object suite of LIBERO. No robot pretraining: an ImageNet ResNet-18 and a frozen T5-small are the only pretrained weights in the model.

libero_object is ten tasks that place a different named object into a basket.

Result

	success rate	episodes
this checkpoint	83.5%	200

Evaluated on 10 tasks × 20 initial states = 200 rollouts, at 256×256, control_mode=relative, executing the full 50-action chunk open-loop.

This is the 30,000-step checkpoint, selected as the highest-scoring of the four checkpoints evaluated for this suite. It is also the final checkpoint of its run, so no selection was applied.

How it compares

Averages over all four LIBERO suites. IMPACT's 78.6% is the mean of each suite's selected checkpoint; shipping every suite's final checkpoint instead gives 76.4%.

policy	params	robot pretraining	LIBERO avg
Diffusion Policy	—	none	72.4%
Octo	93M	Open X-Embodiment	75.1%
OpenVLA	7B	Open X-Embodiment	76.5%
IMPACT	78M	none	78.6%
OpenVLA + parallel decoding & action chunking	7B	Open X-Embodiment	90.2%
π0	3B	proprietary + OXE	94.2%
OpenVLA-OFT	7B	Open X-Embodiment	97.1%

Baselines from Kim et al., Fine-Tuning Vision-Language-Action Models: Optimizing Speed and Success, Table I. IMPACT leads every policy that does not use robot pretraining, and both pretrained models of its own generation, at 78M parameters. The 90%+ tier is 3–7B models; the gap between OpenVLA at 76.5% and the same model at 90.2% comes from execution strategy, not capacity — which is the most promising thing to try here next, since IMPACT currently executes all 50 predicted actions open-loop.

Architecture

Two camera frames, an 8-D end-effector state and a natural-language instruction in; a 50-step action chunk out in one forward pass. The instruction enters twice — as 32 tokens appended to the transformer encoder sequence, and as FiLM scale/shift (1 + γ) ⊙ x + β on every ResNet stage, so language modulates perception and not only the fused memory.

backbone	ResNet-18, ImageNet init, BN folded
text tower	T5-small encoder, frozen, 32 tokens
transformer	d=512, 8 heads, ff 3200, 6 encoder / 4 decoder layers
chunk	50 actions
parameters	78M trainable, 113M total

Training

data	HuggingFaceVLA/libero, libero_object episodes only
steps	30,000 (of a 30k run) at batch 64
lr	1e-4 trunk, 1e-5 backbone, flat
hardware	1× RTX 3090, ~8 h

The suite split resolves each episode's instruction against the LIBERO benchmark's own per-suite task lists and raises on anything unplaceable, rather than splitting on task-index ranges.

Usage

from lerobot.policies.impact.modeling_impact import IMPACTPolicy

policy = IMPACTPolicy.from_pretrained("khanhnd61/impact_libero_object")

Requires the impact policy from LeRobot. Instructions are tokenized to a fixed 32 tokens; every LIBERO instruction fits (the longest, in libero_10, is 22 tokens), but longer instructions are silently truncated.

Limitations

• Single-suite. Trained on this suite's 10 tasks only; it is not a generalist policy.
• No held-out split. Training used every episode, so there is no validation signal separating learning from memorisation. The rollouts are the only out-of-sample evidence, and they use the same 10 tasks with different initial states.
• 200 rollouts per suite, not the 500 used by most published LIBERO tables, so sampling noise is roughly ±7 points (±2 SE).
• No FiLM ablation. Language use is measured, not assumed: on libero_goal, showing the policy another task's instruction drops it to 0.0% on the original goal over 600 episodes (control 88.5%) while it achieves the instructed goal 91.0% of the time. Which language path carries that is not isolated - shuffling perturbs FiLM and the encoder text tokens together - so use_film=false is still needed for the FiLM claim specifically.
• No test of language generalisation. Every instruction evaluated is one the policy trained on, so this is instruction following, not robustness to unseen wording.