plan-bind-d4 / results.md
thanhkt's picture
Upload results.md with huggingface_hub
dd7839b verified
|
Raw History Blame Contribute Delete
2.21 kB

Plan-Bind D4 results: LIBERO

Source: /home/25thanh.tk/plan-bind/experiments/261105-plan-bind-libero/outputs/full/eval. Training seeds 1000, 1001; success in percent, mean over seeds with the per-seed values in brackets.

Main comparison (latency 0, last checkpoint)

Policy All suites libero_spatial libero_object libero_goal libero_10 Episodes
bsp10 85.0 (84, 86) 80.0 (75, 85) 92.5 (95, 90) 87.5 (85, 90) 80.0 (80, 80) 80
onestep 86.9 (89, 85) 82.5 (85, 80) 95.0 (90, 100) 92.5 (95, 90) 77.5 (85, 70) 80
planbind 84.4 (81, 88) 87.5 (90, 85) 85.0 (70, 100) 87.5 (80, 95) 77.5 (85, 70) 80

Call rate and commit (latency 0, success)

Policy Commit Every 2 frames
onestep latency 86.9 (89, 85)
onestep none 86.2 (84, 89)
planbind latency 84.4 (81, 88)
planbind none 93.1 (94, 92)

Latency (commit latency, success)

bsp10 runs through the spline runtime here; its classic action queue is in the main comparison.

Policy 0 ms 200 ms
bsp10 85.6 (82, 89) 84.4 (84, 85)
onestep 86.9 (89, 85) 90.6 (89, 92)
planbind 84.4 (81, 88) 80.0 (86, 74)

All cells

Policy Latency Commit Every Seed Episodes Success
bsp10 0 - 2 1000 80 83.8
bsp10 0 - 2 1001 80 86.2
bsp10 0 latency 2 1000 80 82.5
bsp10 0 latency 2 1001 80 88.8
bsp10 2 latency 2 1000 80 83.8
bsp10 2 latency 2 1001 80 85.0
onestep 0 latency 2 1000 80 88.8
onestep 0 latency 2 1001 80 85.0
onestep 0 none 2 1000 80 83.8
onestep 0 none 2 1001 80 88.8
onestep 2 latency 2 1000 80 88.8
onestep 2 latency 2 1001 80 92.5
planbind 0 latency 2 1000 80 81.2
planbind 0 latency 2 1001 80 87.5
planbind 0 none 2 1000 80 93.8
planbind 0 none 2 1001 80 92.5
planbind 2 latency 2 1000 80 86.2
planbind 2 latency 2 1001 80 73.8