---
license: other
license_name: mixed-apache-2.0-and-mit
tags:
- robotics
- world-model
- model-based-control
- jepa
- reinforcement-learning
- latent-planning
- iterative-control
---

# LePlanner: Amortized Iterative Control in Frozen Latent World Models

This repository contains the trained controllers, frozen world models, latent caches, and evaluation results for **LePlanner**:

> **LePlanner: Amortized Iterative Control in Frozen Latent World Models**
> [Authors], [Year].
> [Paper link — arXiv / venue]

LePlanner is an amortized iterative controller that plans through a **frozen JEPA world model** (LeWM) over a small number of learned refinement steps, trained with an **arrival–hold** objective. It matches or exceeds test-time search planners (CEM) while requiring an order-of-magnitude fewer predictor invocations and 8–15× lower wall-clock per decision.

## Results

| Environment | Method | Success | Notes |
|---|---|---|---|
| PushT | LePlanner (rh=1) | **98%** | arrival+hold checkpoint |
| PushT | LePlanner (rh=5) | **92%** | |
| PushT | CEM (rh=5) | 90% | baseline search planner |
| Reacher | LePlanner (rh=1) | **100%** | 50 episodes, seed 42 |
| TwoRooms | LePlanner (rh=1) | **100%** | 50 episodes, seed 42 |

LePlanner uses **~16× fewer predictor rows per decision** than CEM and **8–15× lower wall-clock** per decision.

## Quick Start

```python
from huggingface_hub import hf_hub_download

# Load a controller checkpoint
ckpt_path = hf_hub_download(
    "SaltedLemon/lejepa-control-pusht",
    "checkpoints/environment_controllers/reacher/controller.pt",
    repo_type="model",
)

# Load a frozen world model
wm_path = hf_hub_download(
    "SaltedLemon/lejepa-control-pusht",
    "checkpoints/base_world_models/reacher/weights.pt",
    repo_type="model",
)
```

The training code is available at [`github.com/SaltedLemon/lejepa_control`](https://github.com/SaltedLemon/lejepa_control).

## Repository Structure

```
checkpoints/
├── environment_controllers/   # Trained LePlanner controllers (per-environment)
│   ├── reacher/
│   │   ├── controller.pt      # 100% success, rh=1
│   │   └── density.pt         # Behavior density model
│   └── tworooms/
│       ├── controller.pt      # 100% success, rh=1
│       └── density.pt
├── controller/                # PushT iterative controller (phase-1)
├── ablations/                 # PushT training ablations
│   └── ah_hold0.5/
│       └── controller.pt      # Headline PushT checkpoint (94-96%)
├── base_world_models/         # Frozen LeWM world models
│   ├── pusht/                 # quentinll/lewm-pusht (MIT)
│   ├── reacher/               # quentinll/lewm-reacher (MIT)
│   └── tworooms/              # quentinll/lewm-tworooms (MIT)
├── planner_*/                 # Recursive & cross-attention planners
├── manifold_transfer/         # Latent transfer experiments
└── density/                   # PushT behavior density model

code/                          # Training & evaluation scripts
├── planner.py                 # LePlanner controller
├── solver.py                  # Training/eval harness
├── losses.py                  # Arrival-hold, action-Gaussian, support losses
└── scripts/                   # train_planner.py, eval_planner.py, ...

latents/                       # Pre-computed latent caches
├── reacher/                   # Latents, actions, episode metadata
└── tworoom_cls/

results/                       # Raw evaluation logs
├── eval/results.jsonl         # PushT eval across planners
└── eval_planner/results.jsonl
```

## Key Checkpoints

| File | Description |
|---|---|
| `checkpoints/ablations/ah_hold0.5/controller.pt` | **Headline PushT controller** — 98% at rh=1, 92% at rh=5 |
| `checkpoints/environment_controllers/reacher/controller.pt` | Reacher controller — 100% |
| `checkpoints/environment_controllers/tworooms/controller.pt` | TwoRooms controller — 100% |
| `checkpoints/base_world_models/pusht/weights.pt` | Frozen PushT world model (LeWM) |
| `checkpoints/base_world_models/reacher/weights.pt` | Frozen Reacher world model (LeWM) |
| `checkpoints/base_world_models/tworooms/weights.pt` | Frozen TwoRooms world model (LeWM) |

## Training & Evaluation

The controller and planner checkpoints are trained in this project; the LeWM world model is **always frozen** while those planners are trained.

Training code: [`github.com/SaltedLemon/lejepa_control`](https://github.com/SaltedLemon/lejepa_control)

Each controller checkpoint embeds its training arguments, action statistics, step, and validation profile.

## Citation

If you use this code or these checkpoints, please cite:

```bibtex
@article{leplanner,
  title   = {LePlanner: Amortized Iterative Control in Frozen Latent World Models},
  author  = {[Authors]},
  journal = {[Venue / arXiv]},
  year    = {[Year]},
  url     = {https://huggingface.co/SaltedLemon/lejepa-control-pusht}
}
```

## License

- **Controller/planner code and checkpoints:** as specified in the linked `lejepa_control` repository.
- **LeWM world model weights** (`base_world_models/pusht`, `reacher`, `tworooms`): MIT licensed upstream from `quentinll/lewm-*`.