|
Download results.md from thanhkt/plan-bind-d3: direct link, hf CLI and curl.
- Browser
- Download file 6.72 kB
-
https://huggingface.co/thanhkt/plan-bind-d3/resolve/main/results.md
- Command line
-
hf download hf://thanhkt/plan-bind-d3/results.md
-
curl -L -o results.md https://huggingface.co/thanhkt/plan-bind-d3/resolve/main/results.md
6.72 kB
Plan-Bind D3 results: Push-T
Source: /home/25thanh.tk/plan-bind/experiments/261028-plan-bind-d3-pusht/outputs/full/eval. Training seeds 1000, 1001; values are the mean over seeds, per-seed values in brackets. Score: mean best coverage in percent.
Main comparison (latency 0, last checkpoint)
| Policy | Score | Coverage >= 0.9 (%) | Env success (%) | Paper score | Episodes |
|---|---|---|---|---|---|
| diffusion | 83.5 (86, 81) | 73.0 (76, 70) | 38.2 (40, 37) | 72 | 200 |
| bsp10 | 66.4 (68, 64) | 56.0 (58, 54) | 34.5 (36, 33) | 75 | 200 |
| onestep | 72.6 (71, 74) | 60.8 (58, 63) | 34.8 (30, 40) | - | 200 |
| planbind | 53.2 (55, 51) | 39.0 (41, 37) | 20.0 (21, 19) | - | 200 |
Checkpoint curve (latency 0, score)
| Policy | Seed | 25000 | 50000 | 75000 | 100000 |
|---|---|---|---|---|---|
| diffusion | 1000 | 79.4 | 85.1 | 86.4 | 86.3 |
| diffusion | 1001 | 78.7 | 77.7 | 81.7 | 80.8 |
| bsp10 | 1000 | 64.3 | 69.1 | 71.3 | 70.3 |
| bsp10 | 1001 | 61.0 | 67.1 | 64.9 | 64.3 |
| onestep | 1000 | 69.9 | 69.9 | 74.4 | 72.4 |
| onestep | 1001 | 76.5 | 70.6 | 74.2 | 75.0 |
| planbind | 1000 | 54.0 | 61.8 | 62.0 | 57.4 |
| planbind | 1001 | 53.4 | 51.4 | 52.7 | 47.8 |
Call rate and commit (latency 0, score)
A call every 2 frames is the default (0.2 s); every 8 is diffusion's rate (8 actions at 10 fps).
| Policy | Commit | Every 2 | Every 8 |
|---|---|---|---|
| onestep | latency | 72.6 (71, 74) | 71.7 (73, 70) |
| onestep | none | 38.4 (38, 39) | 78.7 (76, 81) |
| planbind | latency | 53.2 (55, 51) | 48.6 (50, 47) |
| planbind | none | 61.4 (63, 60) | 53.3 (56, 51) |
Latency (commit latency, score)
bsp10 runs through the spline runtime here; its classic action queue is in the main comparison.
| Policy | 0 ms | 100 ms | 200 ms |
|---|---|---|---|
| bsp10 | 63.4 (63, 64) | 62.2 (64, 60) | 62.5 (64, 61) |
| onestep | 72.6 (71, 74) | 71.3 (73, 70) | 72.0 (71, 73) |
| planbind | 53.2 (55, 51) | 49.8 (51, 49) | 49.7 (51, 48) |
All cells
| Policy | Latency | Commit | Every | Seed | Checkpoint | Episodes | Score | >= 0.9 | Env success |
|---|---|---|---|---|---|---|---|---|---|
| bsp10 | 0 | - | 2 | 1000 | 25000 | 100 | 64.3 | 54.0 | 27.0 |
| bsp10 | 0 | - | 2 | 1001 | 25000 | 100 | 61.0 | 49.0 | 22.0 |
| bsp10 | 0 | - | 2 | 1000 | 50000 | 100 | 69.1 | 60.0 | 31.0 |
| bsp10 | 0 | - | 2 | 1001 | 50000 | 100 | 67.1 | 59.0 | 37.0 |
| bsp10 | 0 | - | 2 | 1000 | 75000 | 100 | 71.3 | 62.0 | 34.0 |
| bsp10 | 0 | - | 2 | 1001 | 75000 | 100 | 64.9 | 55.0 | 30.0 |
| bsp10 | 0 | - | 2 | 1000 | 100000 | 100 | 70.3 | 61.0 | 40.0 |
| bsp10 | 0 | - | 2 | 1001 | 100000 | 100 | 64.3 | 57.0 | 37.0 |
| bsp10 | 0 | - | 2 | 1000 | last | 200 | 68.4 | 58.5 | 36.0 |
| bsp10 | 0 | - | 2 | 1001 | last | 200 | 64.3 | 53.5 | 33.0 |
| bsp10 | 0 | latency | 2 | 1000 | last | 200 | 63.3 | 52.5 | 29.5 |
| bsp10 | 0 | latency | 2 | 1001 | last | 200 | 63.6 | 53.0 | 30.0 |
| bsp10 | 1 | latency | 2 | 1000 | last | 200 | 64.4 | 53.5 | 26.5 |
| bsp10 | 1 | latency | 2 | 1001 | last | 200 | 60.0 | 48.5 | 19.5 |
| bsp10 | 2 | latency | 2 | 1000 | last | 200 | 64.4 | 53.5 | 26.5 |
| bsp10 | 2 | latency | 2 | 1001 | last | 200 | 60.6 | 48.5 | 22.5 |
| diffusion | 0 | - | 2 | 1000 | 25000 | 100 | 79.4 | 68.0 | 31.0 |
| diffusion | 0 | - | 2 | 1001 | 25000 | 100 | 78.7 | 65.0 | 26.0 |
| diffusion | 0 | - | 2 | 1000 | 50000 | 100 | 85.1 | 75.0 | 37.0 |
| diffusion | 0 | - | 2 | 1001 | 50000 | 100 | 77.7 | 68.0 | 38.0 |
| diffusion | 0 | - | 2 | 1000 | 75000 | 100 | 86.4 | 76.0 | 40.0 |
| diffusion | 0 | - | 2 | 1001 | 75000 | 100 | 81.7 | 70.0 | 31.0 |
| diffusion | 0 | - | 2 | 1000 | 100000 | 100 | 86.3 | 76.0 | 43.0 |
| diffusion | 0 | - | 2 | 1001 | 100000 | 100 | 80.8 | 72.0 | 37.0 |
| diffusion | 0 | - | 2 | 1000 | last | 200 | 86.0 | 76.5 | 39.5 |
| diffusion | 0 | - | 2 | 1001 | last | 200 | 81.0 | 69.5 | 37.0 |
| onestep | 0 | latency | 2 | 1000 | 25000 | 100 | 69.9 | 60.0 | 21.0 |
| onestep | 0 | latency | 2 | 1001 | 25000 | 100 | 76.5 | 69.0 | 27.0 |
| onestep | 0 | latency | 2 | 1000 | 50000 | 100 | 69.9 | 57.0 | 26.0 |
| onestep | 0 | latency | 2 | 1001 | 50000 | 100 | 70.6 | 58.0 | 26.0 |
| onestep | 0 | latency | 2 | 1000 | 75000 | 100 | 74.4 | 60.0 | 35.0 |
| onestep | 0 | latency | 2 | 1001 | 75000 | 100 | 74.2 | 66.0 | 35.0 |
| onestep | 0 | latency | 2 | 1000 | 100000 | 100 | 72.4 | 60.0 | 32.0 |
| onestep | 0 | latency | 2 | 1001 | 100000 | 100 | 75.0 | 66.0 | 41.0 |
| onestep | 0 | latency | 2 | 1000 | last | 200 | 71.4 | 58.5 | 30.0 |
| onestep | 0 | latency | 2 | 1001 | last | 200 | 73.8 | 63.0 | 39.5 |
| onestep | 0 | latency | 8 | 1000 | last | 200 | 73.4 | 60.0 | 34.5 |
| onestep | 0 | latency | 8 | 1001 | last | 200 | 70.0 | 62.5 | 33.5 |
| onestep | 0 | none | 2 | 1000 | last | 200 | 38.3 | 26.0 | 11.5 |
| onestep | 0 | none | 2 | 1001 | last | 200 | 38.5 | 25.0 | 8.0 |
| onestep | 0 | none | 8 | 1000 | last | 200 | 76.0 | 66.0 | 36.0 |
| onestep | 0 | none | 8 | 1001 | last | 200 | 81.3 | 72.5 | 44.5 |
| onestep | 1 | latency | 2 | 1000 | last | 200 | 72.7 | 61.0 | 33.0 |
| onestep | 1 | latency | 2 | 1001 | last | 200 | 69.9 | 60.0 | 33.0 |
| onestep | 2 | latency | 2 | 1000 | last | 200 | 71.4 | 59.0 | 30.5 |
| onestep | 2 | latency | 2 | 1001 | last | 200 | 72.6 | 65.5 | 34.0 |
| planbind | 0 | latency | 2 | 1000 | 25000 | 100 | 54.0 | 41.0 | 10.0 |
| planbind | 0 | latency | 2 | 1001 | 25000 | 100 | 53.4 | 39.0 | 12.0 |
| planbind | 0 | latency | 2 | 1000 | 50000 | 100 | 61.8 | 51.0 | 20.0 |
| planbind | 0 | latency | 2 | 1001 | 50000 | 100 | 51.4 | 37.0 | 11.0 |
| planbind | 0 | latency | 2 | 1000 | 75000 | 100 | 62.0 | 47.0 | 24.0 |
| planbind | 0 | latency | 2 | 1001 | 75000 | 100 | 52.7 | 39.0 | 15.0 |
| planbind | 0 | latency | 2 | 1000 | 100000 | 100 | 57.4 | 43.0 | 17.0 |
| planbind | 0 | latency | 2 | 1001 | 100000 | 100 | 47.8 | 30.0 | 19.0 |
| planbind | 0 | latency | 2 | 1000 | last | 200 | 55.4 | 41.0 | 21.0 |
| planbind | 0 | latency | 2 | 1001 | last | 200 | 51.0 | 37.0 | 19.0 |
| planbind | 0 | latency | 8 | 1000 | last | 200 | 50.2 | 35.0 | 14.0 |
| planbind | 0 | latency | 8 | 1001 | last | 200 | 47.0 | 32.5 | 17.0 |
| planbind | 0 | none | 2 | 1000 | last | 200 | 62.7 | 50.5 | 25.0 |
| planbind | 0 | none | 2 | 1001 | last | 200 | 60.1 | 44.5 | 20.0 |
| planbind | 0 | none | 8 | 1000 | last | 200 | 55.8 | 41.5 | 17.5 |
| planbind | 0 | none | 8 | 1001 | last | 200 | 50.8 | 37.5 | 16.0 |
| planbind | 1 | latency | 2 | 1000 | last | 200 | 51.0 | 35.5 | 15.5 |
| planbind | 1 | latency | 2 | 1001 | last | 200 | 48.5 | 35.0 | 18.5 |
| planbind | 2 | latency | 2 | 1000 | last | 200 | 51.3 | 34.0 | 15.5 |
| planbind | 2 | latency | 2 | 1001 | last | 200 | 48.2 | 34.5 | 11.5 |