planning-baselines / README.md
greycat111's picture
Complete PLDM PushT five-seed planning evaluation and report mean/std
2ffd52b verified
|
Raw History Blame Contribute Delete
3.24 kB
metadata
tags:
  - world-model
  - planning
  - dinowm
  - pldm
  - fast-lewm

Planning baseline checkpoint archive

This repository archives checkpoints and measured planning results for TwoRoom, Cube and PushT. Reacher is excluded and LeJEPA is canceled. Original checkpoints retain their upstream licenses; see licenses/.

Directory Contents Status
fast-lewm/tworoom, cube, pusht Exact original naiverer/fast-leworldmodel checkpoints All five-seed planning evaluations completed
dinowm/pusht Original DINO-WM OSF PushT checkpoint and frozen DINOv2-S/14 backbone Published model trained on pusht_noise; evaluation incomplete
pldm/cube Final epoch-10 model, epoch-8 resume checkpoint, configuration and provenance Training and all five-seed planning evaluations complete
pldm/pusht Final epoch-10 model, configuration and provenance Training and all five-seed planning evaluations complete
pldm/tworoom Configuration and results only The trained weights were deleted after evaluation under the earlier disk-saving instruction and are unavailable

Newly completed local checkpoints are archived under <method>/<task>/model.pt, with provenance alongside each file. Each local checkpoint uses training seed 42. Five evaluation seeds do not mean five independently trained models.

Results and comparison protocol

results/planning_success.xlsx contains the detailed per-seed results and the seed summary. results/planning_summary.csv contains columns for seeds 42, 43, 44, 45, 46 and mean/sample standard deviation (ddof=1). Success percentages are reported only for 200 completed episodes per seed and goal offset. Goal offsets are 25, 50, 75 and 100. Mean and std require all five completed seeds; missing values are null.

The planning budget is 50 environment steps, with a 25-step planning window and CEM using 300 candidates, 30 iterations and 30 elites. Models share the persisted task/offset/seed evaluation pairs. Newly trained PLDM and DINO-WM checkpoints use a fixed 10-epoch training budget, while published Fast-LeWM and DINO-WM checkpoints have their original training histories. This is not an equal-training-budget comparison or an exact paper reproduction. Evaluation pairs come from the supplied dataset and are not a separate held-out test split.

Loading

Original Fast-LeWM object checkpoints depend on the upstream Fast-LeWorldModel Python source. Original DINO-WM objects depend on the original DINO-WM source. Locally trained objects additionally depend on code/train_models.py and stable-worldmodel 0.1.1. The Cube resume checkpoint holds state dictionaries, optimizer/RNG state and completed-epoch metadata; it is not a directly callable model object. See code/train_baseline.py for restoration. Loading pickled Python model objects requires their matching source and a trusted checkpoint source.

Source repositories:

No source datasets are uploaded. No paper-reported number is substituted for a measured score. For complete protocol details and known implementation differences, see results/PROTOCOL.md.