Patch Policy diffusion on LIBERO Spatial: epoch 122 (confirmed 0.944 / 500 episodes) + results
5b0c01e verified | license: mit | |
| tags: | |
| - robotics | |
| - libero | |
| - diffusion-policy | |
| - patch-policy | |
| - dinov2 | |
| datasets: | |
| - ParsaSharifi/patch-policy-libero-spatial | |
| # Patch Policy diffusion on LIBERO Spatial | |
| A [Patch Policy](https://patch-policy.github.io/) diffusion head trained on **LIBERO Spatial**: frozen DINOv2 ViT-S/14 | |
| patch tokens from both cameras (512 tokens), conditioned on a per-task goal image, predicting 7-d end-effector | |
| actions. Checkpoint of epoch 122, the best of a 200-epoch run after a 500-episode re-evaluation. | |
| ## Result | |
| | checkpoint | success | episodes | | |
| |---|---|---| | |
| | epoch 122 (this checkpoint) | **0.944** | 500 (50 per task) | | |
| | epoch 122 | 0.99 | 100 (10 per task) | | |
| Per task, epoch 122, 500 episodes: | |
| | task: pick up the black bowl ... | success | | |
| |---|---| | |
| | between the plate and the ramekin | 0.88 | | |
| | from table center | 1.00 | | |
| | in the top drawer of the wooden cabinet | 0.98 | | |
| | next to the cookie box | 1.00 | | |
| | next to the plate | 0.80 | | |
| | next to the ramekin | 0.98 | | |
| | on the cookie box | 0.96 | | |
| | on the ramekin | 0.96 | | |
| | on the stove | 0.98 | | |
| | on the wooden cabinet | 0.90 | | |
| - Evaluation protocol of the patch-policy code, not the LIBERO standard: a fresh seeded reset per episode | |
| (seed 42 + k, objects jittered within the task's regions), 300-step limit. The LIBERO standard uses 50 fixed initial | |
| states per task and 220 steps, so this number is not directly comparable to published LIBERO Spatial results. | |
| - All sweep and confirmation results are in `results/` (`summary.json`, `eval/model_<epoch>.json`, | |
| `eval/confirm/`). The 10 goal images are in `results/goal_images/`. | |
| ## Training | |
| | setting | value | | |
| |---|---| | |
| | code | patch_policy_cleanup, branch `feat/libero-spatial` (commit `7fda225`), config `libero_spatial/diffusion` | | |
| | data | [ParsaSharifi/patch-policy-libero-spatial](https://huggingface.co/datasets/ParsaSharifi/patch-policy-libero-spatial): the 500 official LIBERO Spatial demos re-rendered at 224x224 | | |
| | observation / action windows | 2 / 3 (prediction horizon 4) | | |
| | batch, epochs, optimiser | 255 (85 x 3 GPUs), 200, AdamW lr 5e-5, weight decay 2e-4 | | |
| | denoiser | transformer 8 layers x 4 heads x 256; DDPM 100 steps, EMA weights used for evaluation | | |
| | precision | bf16 autocast, torch.compile, TF32 (DINOv2 runs in fp32) | | |
| | goal | last frame of each task's first-listed demo (`demo_36`), both cameras, concatenated per token | | |
| | hardware, time | 3 x RTX 3090, 4 h | | |
| ## Files | |
| ``` | |
| outputs/libero_spatial/diffusion/dino_patch-2026-09-21_212317/checkpoints/model_0122.pt # epoch, config, policy_state_dict | |
| results/ # evaluation results, goal images, frame strips | |
| ``` | |
| The checkpoint is a PyTorch pickle (`torch.load(..., weights_only=False)`): load it only if you trust this repo. | |
| DINOv2 is not included; the code loads it from `torch.hub` (`facebookresearch/dinov2`). | |
| ## Evaluate | |
| From a checkout of the code on branch `feat/libero-spatial`, with the dataset downloaded: | |
| ```bash | |
| hf download ParsaSharifi/patch-policy-libero-spatial-diffusion --local-dir ~/pp_libero_spatial | |
| MUJOCO_GL=egl uv run python evaluate.py \ | |
| ~/pp_libero_spatial/outputs/libero_spatial/diffusion/dino_patch-2026-09-21_212317 --epochs 122 --no-wandb | |
| ``` | |
| - The config stored in the checkpoint names the dataset directory it was trained with | |
| (`dataset.data_directory: /media/sda1/parsa/original_patch_policy/patch_policy_datasets/libero_dataset`). The evaluation | |
| loads the goal images from there, so the dataset must be at that path (or a symlink to it). | |
| - The goal image of each task is the last frame of the first demo its folder lists. On the training machine that is | |
| `demo_36` for every task; directory listing order depends on the filesystem, so check it on a new machine. | |