LiLa-WAM LIBERO baseline: remove evaluation, open-loop and latency results (weights unchanged)

baselines/LiLa-WAM_LIBERO_ours/README.md

@@ -59,48 +59,5 @@ Each run is the public `train.py`, unchanged except that its dataset factory rea
 
 ## Status
 
-- **LIBERO success rates under our matched protocol are pending:** 4 suites × 10 tasks × 50 trials × seeds 42/43/44.
 - **Paper reference:** the paper reports 98.0 / 98.8 / 97.2 / 94.2 (Spatial / Object / Goal / Long), 97.1 average. Those come from the authors' own data and training length, not from these weights.
 - **License:** the LiLa-WAM ModelScope release states Apache-2.0. DINOv3 is under Meta's DINOv3 license and is not redistributed here.
-
-## Results so far (generated from the result files)
-
-### Training
-
-| Suite | Stage-1 epoch mean loss (5 epochs) | Stage-2 epoch mean loss (2 epochs) | Final file | sha256 (HF LFS verified) |
-|---|---|---|---|---|
-| Long (libero_10) | 0.2737 → 0.1731 → 0.1546 → 0.1430 → 0.1371 | 0.1348 → 0.1318 | `libero_10/stage2_epoch2_final.pt` | `27ad99a458d97a68…` (match) |
-| Goal | 0.3463 → 0.1971 → 0.1710 → 0.1591 → 0.1520 | 0.1484 → 0.1456 | `libero_goal/stage2_epoch2_final.pt` | `5ac2bfd3aad9c2ca…` (match) |
-| Object | 0.2800 → 0.1623 → 0.1444 → 0.1352 → 0.1299 | 0.1277 → 0.1247 | `libero_object/stage2_epoch2_final.pt` | `4571a5f45f02dc61…` (match) |
-| Spatial | 0.3419 → 0.1951 → 0.1742 → 0.1626 → 0.1555 | 0.1528 → 0.1503 | `libero_spatial/stage2_epoch2_final.pt` | `0836ecc4ceefb7fe…` (match) |
-
-### Open-loop check on held-in demonstration frames (not a success rate)
-
-512 random training samples per suite, seed 0, public sampling procedure (10 Euler steps). MAE in raw action units over the 6 end-effector deltas; the baseline always predicts the suite's mean action.
-
-| Suite | Checkpoint | MAE model | MAE baseline | Gripper sign agreement model / baseline |
-|---|---|---|---|---|
-| Long (libero_10) | stage-1 epoch 5 | 0.0729 | 0.1452 | 0.937 / 0.534 |
-| Long (libero_10) | final | 0.0702 | 0.1452 | 0.949 / 0.534 |
-| Goal | stage-1 epoch 5 | 0.0889 | 0.1845 | 0.948 / 0.633 |
-| Goal | final | 0.0853 | 0.1845 | 0.956 / 0.633 |
-| Object | stage-1 epoch 5 | 0.0709 | 0.1600 | 0.939 / 0.537 |
-| Object | final | 0.0682 | 0.1600 | 0.939 / 0.537 |
-| Spatial | stage-1 epoch 5 | 0.0885 | 0.1939 | 0.949 / 0.536 |
-| Spatial | final | 0.0855 | 0.1939 | 0.950 / 0.536 |
-
-### Latency
-
-Batch 1 on an H100 80GB (evc103, GPU otherwise idle), 500 recorded raw LIBERO observations after 20 warmup queries, from raw observation to the smoothed host action chunk (12×7): mean 52.0 ms, p50 51.1, p95 61.0, peak 1.12 GB; 0.515B parameters including the frozen DINOv3 (212.1M trainable action model).
-
-### LIBERO success under our matched protocol
-
-Protocol lightwam_20261009_matched: 4 suites × 10 tasks × official initial states 0–49 × seeds 42/43/44; 30 settling steps; at most 400 policy steps (700 for Long); success = environment done flag. Each suite uses its own final policy. 6 of 12 predicted actions are executed per query, and the gripper is executed as sign(g). Inference is batched, in bf16.
-
-| Suite | Successes / episodes | Success % | Seeds 42 / 43 / 44 % |
-|---|---|---|---|
-| Spatial | 554 / 1500 | 36.93 | 9.6 / 68.6 / 32.6 |
-| Object | 293 / 1500 | 19.53 | 18.6 / 21.6 / 18.4 |
-| Goal | 495 / 1500 | 33.00 | 11.0 / 60.8 / 27.2 |
-| Long (libero_10) | 27 / 1500 | 1.80 | 0.2 / 5.2 / 0.0 |
-| **All** | 1369 / 6000 | **22.82** | 9.8 / 39.1 / 19.6 |

baselines/LiLa-WAM_LIBERO_ours/results/eval_matched_manifest.json

@@ -1,111 +0,0 @@
-{
- "run": "lilawam_libero_ours_paper_recipe",
- "output": "/groups/yshang/an221229/checkpoints/wam_baselines/lilawam/libero_ours_paper_recipe/eval_matched",
- "checkpoint": "4 per-suite policies: libero_spatial=baselines/LiLa-WAM_LIBERO_ours/libero_spatial/stage2_epoch2_final.pt, libero_object=baselines/LiLa-WAM_LIBERO_ours/libero_object/stage2_epoch2_final.pt, libero_goal=baselines/LiLa-WAM_LIBERO_ours/libero_goal/stage2_epoch2_final.pt, libero_10=baselines/LiLa-WAM_LIBERO_ours/libero_10/stage2_epoch2_final.pt",
- "sha256": "5f73fc1a4b850e6dfcaca8208a5ffbe11847c9f735de5b9a2e41da3bb3b116b0",
- "step": "stage 2 epoch 2 (per suite)",
- "backend": "batched-bf16-server",
- "suite_checkpoints": {
-  "libero_spatial": {
-   "config": "/groups/yshang/an221229/checkpoints/wam_baselines/lilawam/libero_ours_paper_recipe/libero_spatial/stage2/sft_2026-10-30_12-14-57/config.yaml",
-   "checkpoint": "/groups/yshang/an221229/checkpoints/wam_baselines/lilawam/libero_ours_paper_recipe/libero_spatial/stage2/sft_2026-10-30_12-14-57/checkpoint_epoch_2.pt",
-   "stats": "/groups/yshang/an221229/data/LiLaWAM_libero_ours_stats_libero_spatial.json",
-   "hf_path": "baselines/LiLa-WAM_LIBERO_ours/libero_spatial/stage2_epoch2_final.pt",
-   "sha256": "0836ecc4ceefb7feacc4ef92c5e76f0683cb46819a8b900a14f5febc69b90af1",
-   "stats_sha256": "638bdff4b942999608cfadee9a696ac8c7ee40373aa8e2824c24822c4b68036b"
-  },
-  "libero_object": {
-   "config": "/groups/yshang/an221229/checkpoints/wam_baselines/lilawam/libero_ours_paper_recipe/libero_object/stage2/sft_2026-10-30_11-40-12/config.yaml",
-   "checkpoint": "/groups/yshang/an221229/checkpoints/wam_baselines/lilawam/libero_ours_paper_recipe/libero_object/stage2/sft_2026-10-30_11-40-12/checkpoint_epoch_2.pt",
-   "stats": "/groups/yshang/an221229/data/LiLaWAM_libero_ours_stats_libero_object.json",
-   "hf_path": "baselines/LiLa-WAM_LIBERO_ours/libero_object/stage2_epoch2_final.pt",
-   "sha256": "4571a5f45f02dc614c64a396112fcc0dc22687df8305740d40c1ab8c044efbce",
-   "stats_sha256": "e2387fa963cb1971c132729e3c87b0d5e105332c8a383ed87d94a9efaee82f37"
-  },
-  "libero_goal": {
-   "config": "/groups/yshang/an221229/checkpoints/wam_baselines/lilawam/libero_ours_paper_recipe/libero_goal/stage2/sft_2026-10-30_10-59-26/config.yaml",
-   "checkpoint": "/groups/yshang/an221229/checkpoints/wam_baselines/lilawam/libero_ours_paper_recipe/libero_goal/stage2/sft_2026-10-30_10-59-26/checkpoint_epoch_2.pt",
-   "stats": "/groups/yshang/an221229/data/LiLaWAM_libero_ours_stats_libero_goal.json",
-   "hf_path": "baselines/LiLa-WAM_LIBERO_ours/libero_goal/stage2_epoch2_final.pt",
-   "sha256": "5ac2bfd3aad9c2caa9082bd6d9a78d363c683673472ec6dea60660507ec464ba",
-   "stats_sha256": "23a7711c1648546c3e6fe5711ae9c4a2feb3c61e2ecfd74463777334fa1d9092"
-  },
-  "libero_10": {
-   "config": "/groups/yshang/an221229/checkpoints/wam_baselines/lilawam/libero_ours_paper_recipe/libero_10/stage2/sft_2026-10-30_10-20-08/config.yaml",
-   "checkpoint": "/groups/yshang/an221229/checkpoints/wam_baselines/lilawam/libero_ours_paper_recipe/libero_10/stage2/sft_2026-10-30_10-20-08/checkpoint_epoch_2.pt",
-   "stats": "/groups/yshang/an221229/data/LiLaWAM_libero_ours_stats_libero_10.json",
-   "hf_path": "baselines/LiLa-WAM_LIBERO_ours/libero_10/stage2_epoch2_final.pt",
-   "sha256": "27ad99a458d97a68e5bfb5503e7e69f9ddc8c1a80d09bc73a3306dd33211d537",
-   "stats_sha256": "8b69db3db1954e51e7cdbdb1eebc8c8d1d4ebb29eb31206f0114f80fd6b9f6b8"
-  }
- },
- "protocol": {
-  "name": "lightwam_20261009_matched",
-  "suites": [
-   "libero_spatial",
-   "libero_object",
-   "libero_goal",
-   "libero_10"
-  ],
-  "seeds": [
-   42,
-   43,
-   44
-  ],
-  "tasks": [
-   0,
-   1,
-   2,
-   3,
-   4,
-   5,
-   6,
-   7,
-   8,
-   9
-  ],
-  "episodes": 50,
-  "max_policy_steps": {
-   "libero_spatial": 400,
-   "libero_object": 400,
-   "libero_goal": 400,
-   "libero_10": 700
-  },
-  "settling_steps": 30,
-  "settling_action": [
-   0,
-   0,
-   0,
-   0,
-   0,
-   0,
-   -1
-  ],
-  "settling_done": "ignored",
-  "replan_steps": 6,
-  "action_horizon": 12,
-  "denoising_steps": 10,
-  "seed_rule": "global seed once per task/seed group; env.seed(base) at construction, env reused for 50 sequential initial states; every query starts from the same noise drawn from a CPU generator seeded with the base seed",
-  "success": "environment done flag",
-  "precision": "bf16 (public LiLa-WAM inference dtype)",
-  "camera": 256,
-  "model_image": "[agentview | wrist] 2x256x256 side by side (512x256), both rotated 180 deg",
-  "state": "eef pos (3), axis-angle (3), gripper qpos (2)",
-  "smoothing": "public B-spline (degree 3, 8 control points)",
-  "gripper": "sign(raw predicted gripper); policy trained on raw LIBERO gripper (-1 open, +1 close)",
-  "videos": false
- },
- "runner_sha256": "0cab3ce6ea7c37852a61267db0071b6528f3632726d4c47e591f3f4acf9e37a3",
- "adapter_sha256": "a2c0ef722786b6925046fdf2d79743d39e9acc46ee72cccedcc45ee5e5087c5b",
- "host": "evc103",
- "slurm_job_id": "1102933",
- "mujoco_gl": "egl",
- "torch": "2.7.1+cu128",
- "gpus": "0,1",
- "env_workers": 80,
- "max_batch": 64,
- "batch_wait": 0.004,
- "cuda_graphs_requested": true,
- "egl_gpus": "0,1",
- "render_skip": true
-}

baselines/LiLa-WAM_LIBERO_ours/results/eval_matched_summary.json

@@ -1,124 +0,0 @@
-{
- "run": "lilawam_libero_ours_paper_recipe",
- "checkpoint": "4 per-suite policies: libero_spatial=baselines/LiLa-WAM_LIBERO_ours/libero_spatial/stage2_epoch2_final.pt, libero_object=baselines/LiLa-WAM_LIBERO_ours/libero_object/stage2_epoch2_final.pt, libero_goal=baselines/LiLa-WAM_LIBERO_ours/libero_goal/stage2_epoch2_final.pt, libero_10=baselines/LiLa-WAM_LIBERO_ours/libero_10/stage2_epoch2_final.pt",
- "checkpoint_sha256": "5f73fc1a4b850e6dfcaca8208a5ffbe11847c9f735de5b9a2e41da3bb3b116b0",
- "checkpoint_step": "stage 2 epoch 2 (per suite)",
- "backend": "batched-bf16-server",
- "episodes": 6000,
- "successes": 1369,
- "success_rate": 0.22816666666666666,
- "per_suite": {
-  "libero_spatial": {
-   "successes": 554,
-   "episodes": 1500,
-   "success_rate": 0.36933333333333335,
-   "seeds": {
-    "42": {
-     "successes": 48,
-     "episodes": 500,
-     "success_rate": 0.096
-    },
-    "43": {
-     "successes": 343,
-     "episodes": 500,
-     "success_rate": 0.686
-    },
-    "44": {
-     "successes": 163,
-     "episodes": 500,
-     "success_rate": 0.326
-    }
-   }
-  },
-  "libero_object": {
-   "successes": 293,
-   "episodes": 1500,
-   "success_rate": 0.19533333333333333,
-   "seeds": {
-    "42": {
-     "successes": 93,
-     "episodes": 500,
-     "success_rate": 0.186
-    },
-    "43": {
-     "successes": 108,
-     "episodes": 500,
-     "success_rate": 0.216
-    },
-    "44": {
-     "successes": 92,
-     "episodes": 500,
-     "success_rate": 0.184
-    }
-   }
-  },
-  "libero_goal": {
-   "successes": 495,
-   "episodes": 1500,
-   "success_rate": 0.33,
-   "seeds": {
-    "42": {
-     "successes": 55,
-     "episodes": 500,
-     "success_rate": 0.11
-    },
-    "43": {
-     "successes": 304,
-     "episodes": 500,
-     "success_rate": 0.608
-    },
-    "44": {
-     "successes": 136,
-     "episodes": 500,
-     "success_rate": 0.272
-    }
-   }
-  },
-  "libero_10": {
-   "successes": 27,
-   "episodes": 1500,
-   "success_rate": 0.018,
-   "seeds": {
-    "42": {
-     "successes": 1,
-     "episodes": 500,
-     "success_rate": 0.002
-    },
-    "43": {
-     "successes": 26,
-     "episodes": 500,
-     "success_rate": 0.052
-    },
-    "44": {
-     "successes": 0,
-     "episodes": 500,
-     "success_rate": 0.0
-    }
-   }
-  }
- },
- "per_seed": {
-  "42": {
-   "successes": 197,
-   "episodes": 2000,
-   "success_rate": 0.0985
-  },
-  "43": {
-   "successes": 781,
-   "episodes": 2000,
-   "success_rate": 0.3905
-  },
-  "44": {
-   "successes": 391,
-   "episodes": 2000,
-   "success_rate": 0.1955
-  }
- },
- "suite_mean_success_rate": 0.22816666666666668,
- "total_policy_steps": 2488241,
- "total_queries": 416811,
- "total_query_seconds": 89363.90853575757,
- "group_seconds": 165549.12861748622,
- "first_group_completed_at": 1793380893.469947,
- "last_group_completed_at": 1793382806.4206002
-}

baselines/LiLa-WAM_LIBERO_ours/results/latency_lilawam_libero_evc103.json

@@ -1,547 +0,0 @@
-{
- "model": "LiLa-WAM (ours, LIBERO paper recipe, libero_10 policy)",
- "repo": "/groups/yshang/an221229/code/wam_baselines_20261030/LiLa-WAM",
- "repo_commit": "339860f65d5cde9e229c037b672889bc4f70497c",
- "checkpoint": "/groups/yshang/an221229/checkpoints/wam_baselines/lilawam/libero_ours_paper_recipe/libero_10/stage2/sft_2026-10-30_10-20-08/checkpoint_epoch_2.pt",
- "config": "/groups/yshang/an221229/checkpoints/wam_baselines/lilawam/libero_ours_paper_recipe/libero_10/stage2/sft_2026-10-30_10-20-08/config.yaml",
- "stats": "/groups/yshang/an221229/data/LiLaWAM_libero_ours_stats_libero_10.json",
- "task_cond": "libero_10/pick_up_the_book_and_place_it_in_the_back_compartment_of_the_caddy",
- "dtype": "bfloat16",
- "num_inference_steps": 10,
- "action_chunk_shape": [
-  12,
-  7
- ],
- "image_size_wh": [
-  512,
-  256
- ],
- "smooth_actions": true,
- "parameters": 515223559,
- "parameters_action_model": 212093959,
- "load_seconds": 8.11405046703294,
- "host": "evc103",
- "gpu": "NVIDIA H100 80GB HBM3",
- "torch": "2.7.1+cu128",
- "transformers": "5.0.0rc0",
- "observations_sha256": "5815c6fdc22d2dfe0c241f89c687a6f2e83804f9bd298332867258eb7160e0f0",
- "warmup": 20,
- "slurm_job": "1094056",
- "actions_finite": true,
- "actions_abs_max": 1.358736331074679,
- "modes": {
-  "native": {
-   "n": 500,
-   "mean_ms": 51.99800907028839,
-   "median_ms": 51.081929937936366,
-   "p5_ms": 50.43525520013645,
-   "p95_ms": 60.98149734316394,
-   "std_ms": 2.935952774707154,
-   "min_ms": 50.31171697191894,
-   "max_ms": 63.4146670345217,
-   "peak_allocated_gb": 1.122252288,
-   "per_query_ms": [
-    51.046,
-    50.826,
-    50.985,
-    50.893,
-    50.846,
-    50.817,
-    50.854,
-    51.519,
-    51.333,
-    51.121,
-    51.141,
-    51.148,
-    50.995,
-    51.077,
-    51.019,
-    50.969,
-    51.134,
-    51.094,
-    52.193,
-    51.317,
-    51.178,
-    51.076,
-    51.132,
-    51.114,
-    51.166,
-    51.14,
-    50.961,
-    51.188,
-    51.164,
-    51.721,
-    51.431,
-    51.468,
-    51.204,
-    51.429,
-    51.191,
-    51.141,
-    51.209,
-    51.257,
-    51.254,
-    51.138,
-    51.875,
-    51.21,
-    51.165,
-    51.424,
-    51.262,
-    51.269,
-    51.212,
-    51.274,
-    51.179,
-    51.075,
-    51.121,
-    51.019,
-    52.479,
-    51.313,
-    51.111,
-    51.118,
-    51.194,
-    51.212,
-    51.198,
-    51.125,
-    51.228,
-    51.199,
-    51.299,
-    56.089,
-    51.192,
-    51.133,
-    51.215,
-    51.177,
-    51.157,
-    51.114,
-    51.182,
-    51.125,
-    51.099,
-    51.018,
-    51.119,
-    52.335,
-    51.172,
-    51.168,
-    51.12,
-    51.036,
-    51.136,
-    51.092,
-    51.2,
-    51.142,
-    51.044,
-    51.119,
-    51.194,
-    52.083,
-    51.242,
-    51.07,
-    51.23,
-    51.098,
-    51.206,
-    51.208,
-    51.117,
-    51.366,
-    51.171,
-    51.189,
-    52.197,
-    51.308,
-    51.205,
-    51.168,
-    51.112,
-    51.115,
-    51.114,
-    51.138,
-    51.213,
-    51.14,
-    51.03,
-    51.787,
-    51.107,
-    51.005,
-    50.984,
-    50.983,
-    51.059,
-    51.02,
-    51.035,
-    50.902,
-    51.122,
-    50.953,
-    51.089,
-    51.085,
-    51.159,
-    51.103,
-    51.81,
-    52.597,
-    55.374,
-    51.072,
-    51.055,
-    51.131,
-    51.033,
-    51.067,
-    51.08,
-    51.109,
-    51.145,
-    51.034,
-    50.963,
-    51.079,
-    51.246,
-    51.161,
-    51.078,
-    51.125,
-    50.933,
-    50.985,
-    51.022,
-    51.019,
-    50.982,
-    51.092,
-    51.071,
-    51.088,
-    51.156,
-    50.955,
-    51.072,
-    51.615,
-    51.073,
-    51.004,
-    51.007,
-    51.079,
-    51.055,
-    51.159,
-    50.975,
-    51.082,
-    51.031,
-    51.077,
-    51.095,
-    51.056,
-    51.107,
-    51.856,
-    51.295,
-    50.988,
-    50.873,
-    51.082,
-    50.972,
-    51.027,
-    51.071,
-    51.031,
-    51.085,
-    51.065,
-    51.032,
-    51.18,
-    51.035,
-    51.223,
-    52.365,
-    51.11,
-    51.103,
-    51.171,
-    51.01,
-    51.128,
-    51.119,
-    51.081,
-    51.14,
-    51.179,
-    50.972,
-    51.084,
-    51.136,
-    50.995,
-    52.24,
-    51.207,
-    51.133,
-    51.075,
-    51.01,
-    51.561,
-    51.032,
-    51.311,
-    51.16,
-    51.139,
-    51.169,
-    51.043,
-    50.976,
-    50.994,
-    50.953,
-    51.442,
-    51.006,
-    51.07,
-    51.107,
-    50.979,
-    50.996,
-    50.902,
-    51.114,
-    50.994,
-    51.078,
-    51.078,
-    51.004,
-    51.012,
-    51.126,
-    51.187,
-    51.308,
-    50.928,
-    51.073,
-    52.583,
-    51.021,
-    51.282,
-    51.08,
-    51.099,
-    51.238,
-    51.161,
-    51.12,
-    51.064,
-    51.088,
-    51.039,
-    51.265,
-    51.132,
-    50.95,
-    55.026,
-    50.808,
-    50.585,
-    50.738,
-    50.673,
-    50.708,
-    50.683,
-    50.687,
-    50.708,
-    50.764,
-    50.683,
-    50.851,
-    50.644,
-    50.633,
-    50.642,
-    50.575,
-    50.576,
-    50.65,
-    50.663,
-    50.67,
-    50.601,
-    50.524,
-    50.555,
-    50.472,
-    50.573,
-    50.591,
-    50.944,
-    50.619,
-    50.55,
-    50.627,
-    50.681,
-    50.722,
-    50.673,
-    50.547,
-    50.575,
-    50.499,
-    50.586,
-    50.514,
-    50.557,
-    50.692,
-    50.686,
-    50.802,
-    50.673,
-    50.603,
-    50.614,
-    50.522,
-    50.692,
-    50.615,
-    50.546,
-    50.566,
-    50.484,
-    50.607,
-    50.554,
-    50.578,
-    50.519,
-    50.455,
-    50.507,
-    50.438,
-    50.631,
-    50.552,
-    50.469,
-    50.444,
-    50.501,
-    50.503,
-    50.741,
-    50.736,
-    50.56,
-    50.64,
-    50.61,
-    50.621,
-    50.687,
-    50.649,
-    50.708,
-    50.535,
-    50.635,
-    50.66,
-    50.71,
-    50.565,
-    50.551,
-    50.481,
-    50.613,
-    50.472,
-    52.953,
-    51.102,
-    51.139,
-    50.831,
-    50.823,
-    50.756,
-    52.686,
-    55.719,
-    63.415,
-    60.568,
-    60.492,
-    60.542,
-    60.497,
-    60.598,
-    60.485,
-    60.53,
-    60.572,
-    60.539,
-    61.804,
-    61.057,
-    60.878,
-    60.944,
-    60.903,
-    61.081,
-    61.084,
-    60.981,
-    60.992,
-    60.971,
-    60.927,
-    61.231,
-    61.033,
-    61.002,
-    61.038,
-    60.979,
-    61.61,
-    61.177,
-    61.136,
-    61.058,
-    61.097,
-    61.063,
-    61.871,
-    52.265,
-    50.426,
-    50.373,
-    52.174,
-    52.925,
-    51.855,
-    51.58,
-    52.282,
-    50.48,
-    52.157,
-    51.893,
-    54.285,
-    50.457,
-    51.828,
-    50.58,
-    50.405,
-    52.093,
-    52.322,
-    52.267,
-    50.649,
-    63.083,
-    60.69,
-    62.628,
-    52.567,
-    52.359,
-    50.596,
-    52.252,
-    50.508,
-    52.668,
-    50.608,
-    54.353,
-    50.46,
-    50.51,
-    52.112,
-    50.6,
-    52.458,
-    51.242,
-    53.771,
-    50.565,
-    50.482,
-    50.547,
-    63.3,
-    50.723,
-    50.593,
-    50.744,
-    50.665,
-    50.58,
-    51.354,
-    51.123,
-    51.367,
-    51.086,
-    51.342,
-    51.023,
-    51.122,
-    51.181,
-    51.024,
-    51.314,
-    51.077,
-    51.155,
-    50.974,
-    62.534,
-    62.731,
-    62.756,
-    62.541,
-    63.29,
-    50.877,
-    50.814,
-    50.852,
-    50.891,
-    51.2,
-    52.649,
-    50.678,
-    50.698,
-    50.595,
-    50.613,
-    50.605,
-    50.597,
-    53.407,
-    50.572,
-    50.575,
-    50.536,
-    50.596,
-    51.4,
-    52.791,
-    50.582,
-    50.698,
-    50.744,
-    50.549,
-    52.544,
-    51.408,
-    51.046,
-    51.042,
-    51.069,
-    51.033,
-    50.856,
-    53.291,
-    52.57,
-    51.901,
-    51.896,
-    51.919,
-    52.318,
-    54.529,
-    52.181,
-    52.977,
-    51.783,
-    52.005,
-    52.08,
-    53.025,
-    50.544,
-    50.348,
-    50.396,
-    50.388,
-    50.383,
-    50.312,
-    50.331,
-    50.42,
-    50.314,
-    50.43,
-    50.352,
-    50.402,
-    50.499,
-    50.343,
-    50.402,
-    50.406,
-    50.348,
-    50.318,
-    50.436,
-    50.568,
-    50.391,
-    50.412,
-    50.364,
-    50.398,
-    50.411,
-    50.352
-   ]
-  }
- }
-}

baselines/LiLa-WAM_LIBERO_ours/results/openloop_libero_10_stage1.json

@@ -1,54 +0,0 @@
-{
- "checkpoint": "/groups/yshang/an221229/checkpoints/wam_baselines/lilawam/libero_ours_paper_recipe/libero_10/stage1/sft_2026-10-30_09-37-52/checkpoint_epoch_5.pt",
- "config": "/groups/yshang/an221229/checkpoints/wam_baselines/lilawam/libero_ours_paper_recipe/libero_10/stage1/sft_2026-10-30_09-37-52/config.yaml",
- "stats": "/groups/yshang/an221229/data/LiLaWAM_libero_ours_stats_libero_10.json",
- "samples": 512,
- "seed": 0,
- "valid_action_steps": 6030,
- "model_ms_per_sample": null,
- "seconds": 36.1,
- "raw": {
-  "mae_xyzrot": 0.07294327020645142,
-  "mae_xyzrot_first_action": 0.07593155652284622,
-  "gripper_sign_agreement": 0.93681592039801,
-  "mae_per_dim": [
-   0.12131507694721222,
-   0.096018947660923,
-   0.12239743024110794,
-   0.0223709587007761,
-   0.040025077760219574,
-   0.03553217276930809,
-   0.13211806118488312
-  ]
- },
- "bspline_smoothed": {
-  "mae_xyzrot": 0.07291633569440528,
-  "mae_xyzrot_first_action": 0.07593133606053604,
-  "gripper_sign_agreement": 0.9371475953565506,
-  "mae_per_dim": [
-   0.1212827699023029,
-   0.09596802225460056,
-   0.1223576577822977,
-   0.022359602577212426,
-   0.04001482875250262,
-   0.035515132897515456,
-   0.13843009248642849
-  ]
- },
- "max_abs_raw_minus_smoothed": 1.1661126731455715,
- "mean_abs_raw_minus_smoothed": 0.0037333345285003945,
- "baseline_mean_action": {
-  "mae_xyzrot": 0.14520248770713806,
-  "mae_xyzrot_first_action": 0.14539091289043427,
-  "gripper_sign_agreement": 0.5343283582089552,
-  "mae_per_dim": [
-   0.20340023934841156,
-   0.2761094570159912,
-   0.2801450192928314,
-   0.02172059565782547,
-   0.03549468517303467,
-   0.05434494838118553,
-   0.993684709072113
-  ]
- }
-}

baselines/LiLa-WAM_LIBERO_ours/results/openloop_libero_10_stage2.json

@@ -1,54 +0,0 @@
-{
- "checkpoint": "/groups/yshang/an221229/checkpoints/wam_baselines/lilawam/libero_ours_paper_recipe/libero_10/stage2/sft_2026-10-30_10-20-08/checkpoint_epoch_2.pt",
- "config": "/groups/yshang/an221229/checkpoints/wam_baselines/lilawam/libero_ours_paper_recipe/libero_10/stage2/sft_2026-10-30_10-20-08/config.yaml",
- "stats": "/groups/yshang/an221229/data/LiLaWAM_libero_ours_stats_libero_10.json",
- "samples": 512,
- "seed": 0,
- "valid_action_steps": 6030,
- "model_ms_per_sample": null,
- "seconds": 26.9,
- "raw": {
-  "mae_xyzrot": 0.07022982090711594,
-  "mae_xyzrot_first_action": 0.07304584980010986,
-  "gripper_sign_agreement": 0.9485903814262023,
-  "mae_per_dim": [
-   0.11697820574045181,
-   0.0930483415722847,
-   0.11718197166919708,
-   0.02172759361565113,
-   0.038496293127536774,
-   0.03394647687673569,
-   0.1060362234711647
-  ]
- },
- "bspline_smoothed": {
-  "mae_xyzrot": 0.07021153836757546,
-  "mae_xyzrot_first_action": 0.07304456162289721,
-  "gripper_sign_agreement": 0.948424543946932,
-  "mae_per_dim": [
-   0.11693636606603112,
-   0.09302810015611977,
-   0.11715296432141627,
-   0.021720655927819122,
-   0.038489977267252215,
-   0.03394116646681417,
-   0.11396457074150271
-  ]
- },
- "max_abs_raw_minus_smoothed": 1.1441281347125898,
- "mean_abs_raw_minus_smoothed": 0.003856766595313692,
- "baseline_mean_action": {
-  "mae_xyzrot": 0.14520248770713806,
-  "mae_xyzrot_first_action": 0.14539091289043427,
-  "gripper_sign_agreement": 0.5343283582089552,
-  "mae_per_dim": [
-   0.20340023934841156,
-   0.2761094570159912,
-   0.2801450192928314,
-   0.02172059565782547,
-   0.03549468517303467,
-   0.05434494838118553,
-   0.993684709072113
-  ]
- }
-}

baselines/LiLa-WAM_LIBERO_ours/results/openloop_libero_goal_stage1.json

@@ -1,54 +0,0 @@
-{
- "checkpoint": "/groups/yshang/an221229/checkpoints/wam_baselines/lilawam/libero_ours_paper_recipe/libero_goal/stage1/sft_2026-10-30_10-37-29/checkpoint_epoch_5.pt",
- "config": "/groups/yshang/an221229/checkpoints/wam_baselines/lilawam/libero_ours_paper_recipe/libero_goal/stage1/sft_2026-10-30_10-37-29/config.yaml",
- "stats": "/groups/yshang/an221229/data/LiLaWAM_libero_ours_stats_libero_goal.json",
- "samples": 512,
- "seed": 0,
- "valid_action_steps": 5852,
- "model_ms_per_sample": null,
- "seconds": 23.7,
- "raw": {
-  "mae_xyzrot": 0.08890368789434433,
-  "mae_xyzrot_first_action": 0.08953068405389786,
-  "gripper_sign_agreement": 0.948051948051948,
-  "mae_per_dim": [
-   0.14938004314899445,
-   0.11821720004081726,
-   0.15685448050498962,
-   0.025271615013480186,
-   0.048691462725400925,
-   0.03500727191567421,
-   0.11095432192087173
-  ]
- },
- "bspline_smoothed": {
-  "mae_xyzrot": 0.08886729274275057,
-  "mae_xyzrot_first_action": 0.08953038006620494,
-  "gripper_sign_agreement": 0.948051948051948,
-  "mae_per_dim": [
-   0.14934976655423415,
-   0.11813900059314215,
-   0.15678745927042992,
-   0.025257630085772104,
-   0.04867197730850233,
-   0.03499792264442286,
-   0.11743164499945619
-  ]
- },
- "max_abs_raw_minus_smoothed": 1.1997946691122572,
- "mean_abs_raw_minus_smoothed": 0.00403382928876949,
- "baseline_mean_action": {
-  "mae_xyzrot": 0.18453039228916168,
-  "mae_xyzrot_first_action": 0.18440485000610352,
-  "gripper_sign_agreement": 0.6327751196172249,
-  "mae_per_dim": [
-   0.29741108417510986,
-   0.2667995095252991,
-   0.38410162925720215,
-   0.03460879251360893,
-   0.05827276408672333,
-   0.06598860770463943,
-   0.9285557270050049
-  ]
- }
-}

baselines/LiLa-WAM_LIBERO_ours/results/openloop_libero_goal_stage2.json

@@ -1,54 +0,0 @@
-{
- "checkpoint": "/groups/yshang/an221229/checkpoints/wam_baselines/lilawam/libero_ours_paper_recipe/libero_goal/stage2/sft_2026-10-30_10-59-26/checkpoint_epoch_2.pt",
- "config": "/groups/yshang/an221229/checkpoints/wam_baselines/lilawam/libero_ours_paper_recipe/libero_goal/stage2/sft_2026-10-30_10-59-26/config.yaml",
- "stats": "/groups/yshang/an221229/data/LiLaWAM_libero_ours_stats_libero_goal.json",
- "samples": 512,
- "seed": 0,
- "valid_action_steps": 5852,
- "model_ms_per_sample": null,
- "seconds": 36.9,
- "raw": {
-  "mae_xyzrot": 0.08530567586421967,
-  "mae_xyzrot_first_action": 0.08639468997716904,
-  "gripper_sign_agreement": 0.9557416267942583,
-  "mae_per_dim": [
-   0.14491380751132965,
-   0.11155447363853455,
-   0.15003924071788788,
-   0.0248557198792696,
-   0.04706612974405289,
-   0.03340467810630798,
-   0.09414315968751907
-  ]
- },
- "bspline_smoothed": {
-  "mae_xyzrot": 0.08528253864759758,
-  "mae_xyzrot_first_action": 0.0863953195847703,
-  "gripper_sign_agreement": 0.9557416267942583,
-  "mae_per_dim": [
-   0.14490496840973593,
-   0.1115045280217428,
-   0.14998854252865573,
-   0.024841943785789276,
-   0.04705381943120802,
-   0.033401429708453736,
-   0.10050620861325198
-  ]
- },
- "max_abs_raw_minus_smoothed": 0.9326507185694056,
- "mean_abs_raw_minus_smoothed": 0.0036275973994723165,
- "baseline_mean_action": {
-  "mae_xyzrot": 0.18453039228916168,
-  "mae_xyzrot_first_action": 0.18440485000610352,
-  "gripper_sign_agreement": 0.6327751196172249,
-  "mae_per_dim": [
-   0.29741108417510986,
-   0.2667995095252991,
-   0.38410162925720215,
-   0.03460879251360893,
-   0.05827276408672333,
-   0.06598860770463943,
-   0.9285557270050049
-  ]
- }
-}

baselines/LiLa-WAM_LIBERO_ours/results/openloop_libero_object_stage1.json

@@ -1,54 +0,0 @@
-{
- "checkpoint": "/groups/yshang/an221229/checkpoints/wam_baselines/lilawam/libero_ours_paper_recipe/libero_object/stage1/sft_2026-10-30_11-10-16/checkpoint_epoch_5.pt",
- "config": "/groups/yshang/an221229/checkpoints/wam_baselines/lilawam/libero_ours_paper_recipe/libero_object/stage1/sft_2026-10-30_11-10-16/config.yaml",
- "stats": "/groups/yshang/an221229/data/LiLaWAM_libero_ours_stats_libero_object.json",
- "samples": 512,
- "seed": 0,
- "valid_action_steps": 5884,
- "model_ms_per_sample": null,
- "seconds": 32.5,
- "raw": {
-  "mae_xyzrot": 0.07092098891735077,
-  "mae_xyzrot_first_action": 0.07137373834848404,
-  "gripper_sign_agreement": 0.9394969408565602,
-  "mae_per_dim": [
-   0.11681602150201797,
-   0.08651068061590195,
-   0.13443148136138916,
-   0.018160240724682808,
-   0.03875875100493431,
-   0.03084881417453289,
-   0.12645787000656128
-  ]
- },
- "bspline_smoothed": {
-  "mae_xyzrot": 0.07090260858533572,
-  "mae_xyzrot_first_action": 0.07137057126040812,
-  "gripper_sign_agreement": 0.9393269884432359,
-  "mae_per_dim": [
-   0.11677656529687021,
-   0.08646329587294171,
-   0.1344257848647293,
-   0.018155414073619907,
-   0.03875172077158669,
-   0.030842870632266536,
-   0.13535093777411733
-  ]
- },
- "max_abs_raw_minus_smoothed": 0.9205889029113743,
- "mean_abs_raw_minus_smoothed": 0.004192262122229111,
- "baseline_mean_action": {
-  "mae_xyzrot": 0.15997077524662018,
-  "mae_xyzrot_first_action": 0.1590776890516281,
-  "gripper_sign_agreement": 0.5365397688647179,
-  "mae_per_dim": [
-   0.19809450209140778,
-   0.34645289182662964,
-   0.3369031846523285,
-   0.013075307011604309,
-   0.03415495157241821,
-   0.03114379197359085,
-   0.9949068427085876
-  ]
- }
-}

baselines/LiLa-WAM_LIBERO_ours/results/openloop_libero_object_stage2.json

@@ -1,54 +0,0 @@
-{
- "checkpoint": "/groups/yshang/an221229/checkpoints/wam_baselines/lilawam/libero_ours_paper_recipe/libero_object/stage2/sft_2026-10-30_11-40-12/checkpoint_epoch_2.pt",
- "config": "/groups/yshang/an221229/checkpoints/wam_baselines/lilawam/libero_ours_paper_recipe/libero_object/stage2/sft_2026-10-30_11-40-12/config.yaml",
- "stats": "/groups/yshang/an221229/data/LiLaWAM_libero_ours_stats_libero_object.json",
- "samples": 512,
- "seed": 0,
- "valid_action_steps": 5884,
- "model_ms_per_sample": null,
- "seconds": 32.4,
- "raw": {
-  "mae_xyzrot": 0.06822021305561066,
-  "mae_xyzrot_first_action": 0.06855077296495438,
-  "gripper_sign_agreement": 0.938817131203263,
-  "mae_per_dim": [
-   0.11383122950792313,
-   0.08429605513811111,
-   0.1268875002861023,
-   0.017427468672394753,
-   0.03742528706789017,
-   0.02945375256240368,
-   0.1259048581123352
-  ]
- },
- "bspline_smoothed": {
-  "mae_xyzrot": 0.06821094450463312,
-  "mae_xyzrot_first_action": 0.06854974035023415,
-  "gripper_sign_agreement": 0.9389870836165873,
-  "mae_per_dim": [
-   0.11380894168087226,
-   0.08425937678102108,
-   0.12689931038120597,
-   0.017421580840100405,
-   0.037420560266965774,
-   0.029455897077633256,
-   0.1349420050661866
-  ]
- },
- "max_abs_raw_minus_smoothed": 1.0650691145549156,
- "mean_abs_raw_minus_smoothed": 0.004098033818130006,
- "baseline_mean_action": {
-  "mae_xyzrot": 0.15997077524662018,
-  "mae_xyzrot_first_action": 0.1590776890516281,
-  "gripper_sign_agreement": 0.5365397688647179,
-  "mae_per_dim": [
-   0.19809450209140778,
-   0.34645289182662964,
-   0.3369031846523285,
-   0.013075307011604309,
-   0.03415495157241821,
-   0.03114379197359085,
-   0.9949068427085876
-  ]
- }
-}

baselines/LiLa-WAM_LIBERO_ours/results/openloop_libero_spatial_stage1.json

@@ -1,54 +0,0 @@
-{
- "checkpoint": "/groups/yshang/an221229/checkpoints/wam_baselines/lilawam/libero_ours_paper_recipe/libero_spatial/stage1/sft_2026-10-30_11-52-40/checkpoint_epoch_5.pt",
- "config": "/groups/yshang/an221229/checkpoints/wam_baselines/lilawam/libero_ours_paper_recipe/libero_spatial/stage1/sft_2026-10-30_11-52-40/config.yaml",
- "stats": "/groups/yshang/an221229/data/LiLaWAM_libero_ours_stats_libero_spatial.json",
- "samples": 512,
- "seed": 0,
- "valid_action_steps": 5839,
- "model_ms_per_sample": null,
- "seconds": 14.3,
- "raw": {
-  "mae_xyzrot": 0.08847805857658386,
-  "mae_xyzrot_first_action": 0.09094158560037613,
-  "gripper_sign_agreement": 0.948621339270423,
-  "mae_per_dim": [
-   0.14923778176307678,
-   0.11272138357162476,
-   0.15589170157909393,
-   0.02719828113913536,
-   0.049992628395557404,
-   0.03582650050520897,
-   0.10927130281925201
-  ]
- },
- "bspline_smoothed": {
-  "mae_xyzrot": 0.08843251468032237,
-  "mae_xyzrot_first_action": 0.0909412777092345,
-  "gripper_sign_agreement": 0.9484500770679911,
-  "mae_per_dim": [
-   0.14915458627126654,
-   0.11263683723105061,
-   0.15581527575209342,
-   0.0271890842956625,
-   0.049977631253884026,
-   0.035821673277977195,
-   0.11848453348205581
-  ]
- },
- "max_abs_raw_minus_smoothed": 1.1656631476437784,
- "mean_abs_raw_minus_smoothed": 0.004558494304498524,
- "baseline_mean_action": {
-  "mae_xyzrot": 0.19394469261169434,
-  "mae_xyzrot_first_action": 0.19752240180969238,
-  "gripper_sign_agreement": 0.535708169207056,
-  "mae_per_dim": [
-   0.3348146378993988,
-   0.27155423164367676,
-   0.43856778740882874,
-   0.024212533608078957,
-   0.05417237430810928,
-   0.040346596390008926,
-   0.9938967227935791
-  ]
- }
-}

baselines/LiLa-WAM_LIBERO_ours/results/openloop_libero_spatial_stage2.json

@@ -1,54 +0,0 @@
-{
- "checkpoint": "/groups/yshang/an221229/checkpoints/wam_baselines/lilawam/libero_ours_paper_recipe/libero_spatial/stage2/sft_2026-10-30_12-14-57/checkpoint_epoch_2.pt",
- "config": "/groups/yshang/an221229/checkpoints/wam_baselines/lilawam/libero_ours_paper_recipe/libero_spatial/stage2/sft_2026-10-30_12-14-57/config.yaml",
- "stats": "/groups/yshang/an221229/data/LiLaWAM_libero_ours_stats_libero_spatial.json",
- "samples": 512,
- "seed": 0,
- "valid_action_steps": 5839,
- "model_ms_per_sample": null,
- "seconds": 15.7,
- "raw": {
-  "mae_xyzrot": 0.08550859987735748,
-  "mae_xyzrot_first_action": 0.0879758819937706,
-  "gripper_sign_agreement": 0.9499914368898784,
-  "mae_per_dim": [
-   0.14433318376541138,
-   0.1079985499382019,
-   0.15060068666934967,
-   0.026742860674858093,
-   0.04849477857351303,
-   0.03488151356577873,
-   0.1060594692826271
-  ]
- },
- "bspline_smoothed": {
-  "mae_xyzrot": 0.08547713147525254,
-  "mae_xyzrot_first_action": 0.08797538113201629,
-  "gripper_sign_agreement": 0.9499914368898784,
-  "mae_per_dim": [
-   0.14425030538972225,
-   0.10793496089985437,
-   0.15058512147232217,
-   0.02673761927000959,
-   0.04847956796643034,
-   0.03487521385317656,
-   0.11615406858671087
-  ]
- },
- "max_abs_raw_minus_smoothed": 1.1563497193107835,
- "mean_abs_raw_minus_smoothed": 0.004443378828302518,
- "baseline_mean_action": {
-  "mae_xyzrot": 0.19394469261169434,
-  "mae_xyzrot_first_action": 0.19752240180969238,
-  "gripper_sign_agreement": 0.535708169207056,
-  "mae_per_dim": [
-   0.3348146378993988,
-   0.27155423164367676,
-   0.43856778740882874,
-   0.024212533608078957,
-   0.05417237430810928,
-   0.040346596390008926,
-   0.9938967227935791
-  ]
- }
-}