Replace WanOFT latency PPO with accepted 10 FPS policy
Browse filesSelected at 23,040,000 steps from W&B run s463nqus; independent success_at_end 240/256 (93.75%).
- latency-aware/mikasa-intercept-grab-fast/small-policy/ppo-wanoft-rtx3090-v1/README.md +10 -10
- latency-aware/mikasa-intercept-grab-fast/small-policy/ppo-wanoft-rtx3090-v1/checkpoint.pt +1 -1
- latency-aware/mikasa-intercept-grab-fast/small-policy/ppo-wanoft-rtx3090-v1/eval.yaml +3 -3
- latency-aware/mikasa-intercept-grab-fast/small-policy/ppo-wanoft-rtx3090-v1/provenance.json +5 -5
- latency-aware/mikasa-intercept-grab-fast/small-policy/ppo-wanoft-rtx3090-v1/strict-eval.json +10 -10
- latency-aware/mikasa-intercept-grab-fast/small-policy/ppo-wanoft-rtx3090-v1/train.yaml +3 -3
latency-aware/mikasa-intercept-grab-fast/small-policy/ppo-wanoft-rtx3090-v1/README.md
CHANGED
|
@@ -19,17 +19,17 @@ temporal latency profile measured for WanOFT on one RTX 3090. The policy uses
|
|
| 19 |
- Environment: `InterceptGrabFast-VLA-v0`, `normalized_dense` reward, 60-step episodes.
|
| 20 |
- Observation: 49D privileged state; actor and critic each use three 256-unit tanh layers.
|
| 21 |
- Action: 7D `pd_ee_delta_pose`, one action per accepted request.
|
| 22 |
-
-
|
| 23 |
- Seed: 123; 1,024 environments; 60-step rollouts; 32 minibatches; eight update epochs.
|
| 24 |
- Learning rate: `1e-4`; PPO target KL: `0.2`; gamma: `0.99`; GAE lambda: `0.95`.
|
| 25 |
- Latency: `temporal`, training latency seed 0, one simulated worker,
|
| 26 |
`latest_ready` ordering, and `hold` between action arrivals.
|
| 27 |
- The profile's mean observation-to-action latency is 300.54 ms. The joint model
|
| 28 |
represents both worker occupancy and action-ready time.
|
| 29 |
-
- Selected checkpoint:
|
| 30 |
-
the highest periodic `eval/success_at_end`
|
| 31 |
|
| 32 |
-
Training run: [
|
| 33 |
|
| 34 |
The exact latency profile is
|
| 35 |
[`profiles/wanoft/1x-rtx3090/mikasa_intercept_grab_fast/instance_3a0d42681a03715c`](https://huggingface.co/datasets/latency-sensitive-bench/profiles/tree/66414463f2a609316da63935941a79abf0a73ce9/profiles/wanoft/1x-rtx3090/mikasa_intercept_grab_fast/instance_3a0d42681a03715c),
|
|
@@ -37,17 +37,17 @@ pinned to revision `66414463f2a609316da63935941a79abf0a73ce9`.
|
|
| 37 |
|
| 38 |
## Independent evaluation
|
| 39 |
|
| 40 |
-
The evaluation used 64 parallel environments, 240
|
| 41 |
(four complete episodes each), environment seed `20260910`, and independent
|
| 42 |
latency seed `271828`. Actions were deterministic policy means.
|
| 43 |
|
| 44 |
| Metric | Result |
|
| 45 |
| --- | --- |
|
| 46 |
-
| Success at least once |
|
| 47 |
-
| Success at the end |
|
| 48 |
-
| Mean episode return |
|
| 49 |
-
| Mean episode length | 60
|
| 50 |
-
| Accepted / capacity-dropped requests |
|
| 51 |
|
| 52 |
## Download and evaluate
|
| 53 |
|
|
|
|
| 19 |
- Environment: `InterceptGrabFast-VLA-v0`, `normalized_dense` reward, 60-step episodes.
|
| 20 |
- Observation: 49D privileged state; actor and critic each use three 256-unit tanh layers.
|
| 21 |
- Action: 7D `pd_ee_delta_pose`, one action per accepted request.
|
| 22 |
+
- Benchmark and observation clocks: 10 FPS; native control: 20 Hz; physics: 100 Hz.
|
| 23 |
- Seed: 123; 1,024 environments; 60-step rollouts; 32 minibatches; eight update epochs.
|
| 24 |
- Learning rate: `1e-4`; PPO target KL: `0.2`; gamma: `0.99`; GAE lambda: `0.95`.
|
| 25 |
- Latency: `temporal`, training latency seed 0, one simulated worker,
|
| 26 |
`latest_ready` ordering, and `hold` between action arrivals.
|
| 27 |
- The profile's mean observation-to-action latency is 300.54 ms. The joint model
|
| 28 |
represents both worker occupancy and action-ready time.
|
| 29 |
+
- Selected checkpoint: 23,040,000 environment steps, iteration 375, chosen by
|
| 30 |
+
the highest periodic `eval/success_at_end`; training stopped at 31,150,080 steps.
|
| 31 |
|
| 32 |
+
Training run: [s463nqus](https://wandb.ai/saberrr-zju/MIKASA-Robo-dataset-collectors/runs/s463nqus).
|
| 33 |
|
| 34 |
The exact latency profile is
|
| 35 |
[`profiles/wanoft/1x-rtx3090/mikasa_intercept_grab_fast/instance_3a0d42681a03715c`](https://huggingface.co/datasets/latency-sensitive-bench/profiles/tree/66414463f2a609316da63935941a79abf0a73ce9/profiles/wanoft/1x-rtx3090/mikasa_intercept_grab_fast/instance_3a0d42681a03715c),
|
|
|
|
| 37 |
|
| 38 |
## Independent evaluation
|
| 39 |
|
| 40 |
+
The evaluation used 64 parallel environments, 240 benchmark steps per environment
|
| 41 |
(four complete episodes each), environment seed `20260910`, and independent
|
| 42 |
latency seed `271828`. Actions were deterministic policy means.
|
| 43 |
|
| 44 |
| Metric | Result |
|
| 45 |
| --- | --- |
|
| 46 |
+
| Success at least once | 252 / 256 = **98.44%** |
|
| 47 |
+
| Success at the end | 240 / 256 = **93.75%** |
|
| 48 |
+
| Mean episode return | 30.8664 |
|
| 49 |
+
| Mean episode length | 60 benchmark steps |
|
| 50 |
+
| Accepted / capacity-dropped requests | 5,119 / 10,241 |
|
| 51 |
|
| 52 |
## Download and evaluate
|
| 53 |
|
latency-aware/mikasa-intercept-grab-fast/small-policy/ppo-wanoft-rtx3090-v1/checkpoint.pt
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
size 1169937
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:ba0f485b8dec4da11836eb64f6863df172744b0922e654ace644dd4668d82e4b
|
| 3 |
size 1169937
|
latency-aware/mikasa-intercept-grab-fast/small-policy/ppo-wanoft-rtx3090-v1/eval.yaml
CHANGED
|
@@ -1,6 +1,6 @@
|
|
| 1 |
env:
|
| 2 |
-
env_fps:
|
| 3 |
-
obs_fps:
|
| 4 |
noop_action: [0, 0, 0, 0, 0, 0, 0]
|
| 5 |
executor:
|
| 6 |
simulated_worker_capacity: 1
|
|
@@ -16,7 +16,7 @@ latency:
|
|
| 16 |
seed: 271828
|
| 17 |
profile_path: profiles/wanoft/1x-rtx3090/mikasa_intercept_grab_fast/instance_3a0d42681a03715c/profile.json
|
| 18 |
training:
|
| 19 |
-
exp_name: small-ppo-mikasa-intercept-grab-fast-wanoft-rtx3090-temporal-scratch
|
| 20 |
checkpoint: checkpoint.pt
|
| 21 |
resume: null
|
| 22 |
seed: 20260910
|
|
|
|
| 1 |
env:
|
| 2 |
+
env_fps: 10
|
| 3 |
+
obs_fps: 10
|
| 4 |
noop_action: [0, 0, 0, 0, 0, 0, 0]
|
| 5 |
executor:
|
| 6 |
simulated_worker_capacity: 1
|
|
|
|
| 16 |
seed: 271828
|
| 17 |
profile_path: profiles/wanoft/1x-rtx3090/mikasa_intercept_grab_fast/instance_3a0d42681a03715c/profile.json
|
| 18 |
training:
|
| 19 |
+
exp_name: small-ppo-mikasa-intercept-grab-fast-wanoft-rtx3090-temporal-clock10-scratch
|
| 20 |
checkpoint: checkpoint.pt
|
| 21 |
resume: null
|
| 22 |
seed: 20260910
|
latency-aware/mikasa-intercept-grab-fast/small-policy/ppo-wanoft-rtx3090-v1/provenance.json
CHANGED
|
@@ -2,12 +2,12 @@
|
|
| 2 |
"task": "mikasa-intercept-grab-fast",
|
| 3 |
"model": "ppo-mlp-state",
|
| 4 |
"status": "published",
|
| 5 |
-
"run_id": "
|
| 6 |
-
"wandb_url": "https://wandb.ai/saberrr-zju/MIKASA-Robo-dataset-collectors/runs/
|
| 7 |
"checkpoint": "checkpoint.pt",
|
| 8 |
"checkpoint_bytes": 1169937,
|
| 9 |
-
"checkpoint_global_step":
|
| 10 |
-
"checkpoint_iteration":
|
| 11 |
"selection_metric": "eval/success_at_end",
|
| 12 |
"initialization": "from_scratch",
|
| 13 |
"training_seed": 123,
|
|
@@ -17,7 +17,7 @@
|
|
| 17 |
"path": "profiles/wanoft/1x-rtx3090/mikasa_intercept_grab_fast/instance_3a0d42681a03715c"
|
| 18 |
},
|
| 19 |
"trainer_source": {
|
| 20 |
-
"wandb_run_id": "
|
| 21 |
"path": "code/third_party/MIKASA-Robo/mikasa_robo_suite/vla/dataset_collectors/get_dataset_collectors_ckpt.py"
|
| 22 |
},
|
| 23 |
"evaluation": "strict-eval.json"
|
|
|
|
| 2 |
"task": "mikasa-intercept-grab-fast",
|
| 3 |
"model": "ppo-mlp-state",
|
| 4 |
"status": "published",
|
| 5 |
+
"run_id": "s463nqus",
|
| 6 |
+
"wandb_url": "https://wandb.ai/saberrr-zju/MIKASA-Robo-dataset-collectors/runs/s463nqus",
|
| 7 |
"checkpoint": "checkpoint.pt",
|
| 8 |
"checkpoint_bytes": 1169937,
|
| 9 |
+
"checkpoint_global_step": 23040000,
|
| 10 |
+
"checkpoint_iteration": 375,
|
| 11 |
"selection_metric": "eval/success_at_end",
|
| 12 |
"initialization": "from_scratch",
|
| 13 |
"training_seed": 123,
|
|
|
|
| 17 |
"path": "profiles/wanoft/1x-rtx3090/mikasa_intercept_grab_fast/instance_3a0d42681a03715c"
|
| 18 |
},
|
| 19 |
"trainer_source": {
|
| 20 |
+
"wandb_run_id": "s463nqus",
|
| 21 |
"path": "code/third_party/MIKASA-Robo/mikasa_robo_suite/vla/dataset_collectors/get_dataset_collectors_ckpt.py"
|
| 22 |
},
|
| 23 |
"evaluation": "strict-eval.json"
|
latency-aware/mikasa-intercept-grab-fast/small-policy/ppo-wanoft-rtx3090-v1/strict-eval.json
CHANGED
|
@@ -1,21 +1,21 @@
|
|
| 1 |
{
|
| 2 |
"episodes": 256,
|
| 3 |
"metrics": {
|
| 4 |
-
"success_once": 0.
|
| 5 |
-
"return":
|
| 6 |
"episode_len": 60.0,
|
| 7 |
-
"reward": 0.
|
| 8 |
-
"success_at_end": 0.
|
| 9 |
},
|
| 10 |
-
"admitted_count":
|
| 11 |
-
"dropped_count":
|
| 12 |
"checkpoint": "checkpoint.pt",
|
| 13 |
-
"checkpoint_global_step":
|
| 14 |
"evaluation_config": "eval.yaml",
|
| 15 |
-
"success_rate": 0.
|
| 16 |
"success_metric": "success_at_end",
|
| 17 |
-
"success_once_count":
|
| 18 |
-
"success_at_end_count":
|
| 19 |
"environment_seed": 20260910,
|
| 20 |
"latency_seed": 271828,
|
| 21 |
"deterministic": true
|
|
|
|
| 1 |
{
|
| 2 |
"episodes": 256,
|
| 3 |
"metrics": {
|
| 4 |
+
"success_once": 0.984375,
|
| 5 |
+
"return": 30.86635398864746,
|
| 6 |
"episode_len": 60.0,
|
| 7 |
+
"reward": 0.5144392251968384,
|
| 8 |
+
"success_at_end": 0.9375
|
| 9 |
},
|
| 10 |
+
"admitted_count": 5119,
|
| 11 |
+
"dropped_count": 10241,
|
| 12 |
"checkpoint": "checkpoint.pt",
|
| 13 |
+
"checkpoint_global_step": 23040000,
|
| 14 |
"evaluation_config": "eval.yaml",
|
| 15 |
+
"success_rate": 0.9375,
|
| 16 |
"success_metric": "success_at_end",
|
| 17 |
+
"success_once_count": 252,
|
| 18 |
+
"success_at_end_count": 240,
|
| 19 |
"environment_seed": 20260910,
|
| 20 |
"latency_seed": 271828,
|
| 21 |
"deterministic": true
|
latency-aware/mikasa-intercept-grab-fast/small-policy/ppo-wanoft-rtx3090-v1/train.yaml
CHANGED
|
@@ -1,6 +1,6 @@
|
|
| 1 |
env:
|
| 2 |
-
env_fps:
|
| 3 |
-
obs_fps:
|
| 4 |
noop_action: [0, 0, 0, 0, 0, 0, 0]
|
| 5 |
executor:
|
| 6 |
simulated_worker_capacity: 1
|
|
@@ -16,7 +16,7 @@ latency:
|
|
| 16 |
seed: 0
|
| 17 |
profile_path: profiles/wanoft/1x-rtx3090/mikasa_intercept_grab_fast/instance_3a0d42681a03715c/profile.json
|
| 18 |
training:
|
| 19 |
-
exp_name: small-ppo-mikasa-intercept-grab-fast-wanoft-rtx3090-temporal-scratch
|
| 20 |
checkpoint: null
|
| 21 |
resume: null
|
| 22 |
seed: 123
|
|
|
|
| 1 |
env:
|
| 2 |
+
env_fps: 10
|
| 3 |
+
obs_fps: 10
|
| 4 |
noop_action: [0, 0, 0, 0, 0, 0, 0]
|
| 5 |
executor:
|
| 6 |
simulated_worker_capacity: 1
|
|
|
|
| 16 |
seed: 0
|
| 17 |
profile_path: profiles/wanoft/1x-rtx3090/mikasa_intercept_grab_fast/instance_3a0d42681a03715c/profile.json
|
| 18 |
training:
|
| 19 |
+
exp_name: small-ppo-mikasa-intercept-grab-fast-wanoft-rtx3090-temporal-clock10-scratch
|
| 20 |
checkpoint: null
|
| 21 |
resume: null
|
| 22 |
seed: 123
|