Caesarrr commited on
Commit
65f4d88
·
verified ·
1 Parent(s): c72713d

Replace WanOFT latency PPO with accepted 10 FPS policy

Browse files

Selected at 23,040,000 steps from W&B run s463nqus; independent success_at_end 240/256 (93.75%).

latency-aware/mikasa-intercept-grab-fast/small-policy/ppo-wanoft-rtx3090-v1/README.md CHANGED
@@ -19,17 +19,17 @@ temporal latency profile measured for WanOFT on one RTX 3090. The policy uses
19
  - Environment: `InterceptGrabFast-VLA-v0`, `normalized_dense` reward, 60-step episodes.
20
  - Observation: 49D privileged state; actor and critic each use three 256-unit tanh layers.
21
  - Action: 7D `pd_ee_delta_pose`, one action per accepted request.
22
- - Control frequency: 20 Hz; physics frequency: 100 Hz.
23
  - Seed: 123; 1,024 environments; 60-step rollouts; 32 minibatches; eight update epochs.
24
  - Learning rate: `1e-4`; PPO target KL: `0.2`; gamma: `0.99`; GAE lambda: `0.95`.
25
  - Latency: `temporal`, training latency seed 0, one simulated worker,
26
  `latest_ready` ordering, and `hold` between action arrivals.
27
  - The profile's mean observation-to-action latency is 300.54 ms. The joint model
28
  represents both worker occupancy and action-ready time.
29
- - Selected checkpoint: 69,120,000 environment steps, iteration 1125, chosen by
30
- the highest periodic `eval/success_at_end` during the 150M-step run.
31
 
32
- Training run: [sw3b981e](https://wandb.ai/saberrr-zju/MIKASA-Robo-dataset-collectors/runs/sw3b981e).
33
 
34
  The exact latency profile is
35
  [`profiles/wanoft/1x-rtx3090/mikasa_intercept_grab_fast/instance_3a0d42681a03715c`](https://huggingface.co/datasets/latency-sensitive-bench/profiles/tree/66414463f2a609316da63935941a79abf0a73ce9/profiles/wanoft/1x-rtx3090/mikasa_intercept_grab_fast/instance_3a0d42681a03715c),
@@ -37,17 +37,17 @@ pinned to revision `66414463f2a609316da63935941a79abf0a73ce9`.
37
 
38
  ## Independent evaluation
39
 
40
- The evaluation used 64 parallel environments, 240 control steps per environment
41
  (four complete episodes each), environment seed `20260910`, and independent
42
  latency seed `271828`. Actions were deterministic policy means.
43
 
44
  | Metric | Result |
45
  | --- | --- |
46
- | Success at least once | 161 / 256 = **62.89%** |
47
- | Success at the end | 122 / 256 = **47.66%** |
48
- | Mean episode return | 22.5840 |
49
- | Mean episode length | 60 control steps |
50
- | Accepted / capacity-dropped requests | 2,560 / 12,800 |
51
 
52
  ## Download and evaluate
53
 
 
19
  - Environment: `InterceptGrabFast-VLA-v0`, `normalized_dense` reward, 60-step episodes.
20
  - Observation: 49D privileged state; actor and critic each use three 256-unit tanh layers.
21
  - Action: 7D `pd_ee_delta_pose`, one action per accepted request.
22
+ - Benchmark and observation clocks: 10 FPS; native control: 20 Hz; physics: 100 Hz.
23
  - Seed: 123; 1,024 environments; 60-step rollouts; 32 minibatches; eight update epochs.
24
  - Learning rate: `1e-4`; PPO target KL: `0.2`; gamma: `0.99`; GAE lambda: `0.95`.
25
  - Latency: `temporal`, training latency seed 0, one simulated worker,
26
  `latest_ready` ordering, and `hold` between action arrivals.
27
  - The profile's mean observation-to-action latency is 300.54 ms. The joint model
28
  represents both worker occupancy and action-ready time.
29
+ - Selected checkpoint: 23,040,000 environment steps, iteration 375, chosen by
30
+ the highest periodic `eval/success_at_end`; training stopped at 31,150,080 steps.
31
 
32
+ Training run: [s463nqus](https://wandb.ai/saberrr-zju/MIKASA-Robo-dataset-collectors/runs/s463nqus).
33
 
34
  The exact latency profile is
35
  [`profiles/wanoft/1x-rtx3090/mikasa_intercept_grab_fast/instance_3a0d42681a03715c`](https://huggingface.co/datasets/latency-sensitive-bench/profiles/tree/66414463f2a609316da63935941a79abf0a73ce9/profiles/wanoft/1x-rtx3090/mikasa_intercept_grab_fast/instance_3a0d42681a03715c),
 
37
 
38
  ## Independent evaluation
39
 
40
+ The evaluation used 64 parallel environments, 240 benchmark steps per environment
41
  (four complete episodes each), environment seed `20260910`, and independent
42
  latency seed `271828`. Actions were deterministic policy means.
43
 
44
  | Metric | Result |
45
  | --- | --- |
46
+ | Success at least once | 252 / 256 = **98.44%** |
47
+ | Success at the end | 240 / 256 = **93.75%** |
48
+ | Mean episode return | 30.8664 |
49
+ | Mean episode length | 60 benchmark steps |
50
+ | Accepted / capacity-dropped requests | 5,119 / 10,241 |
51
 
52
  ## Download and evaluate
53
 
latency-aware/mikasa-intercept-grab-fast/small-policy/ppo-wanoft-rtx3090-v1/checkpoint.pt CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:3ec32a1d150f295d7f27d8d2dabdda34966b91389ff3d6568c09a75da98334ea
3
  size 1169937
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ba0f485b8dec4da11836eb64f6863df172744b0922e654ace644dd4668d82e4b
3
  size 1169937
latency-aware/mikasa-intercept-grab-fast/small-policy/ppo-wanoft-rtx3090-v1/eval.yaml CHANGED
@@ -1,6 +1,6 @@
1
  env:
2
- env_fps: 20
3
- obs_fps: 20
4
  noop_action: [0, 0, 0, 0, 0, 0, 0]
5
  executor:
6
  simulated_worker_capacity: 1
@@ -16,7 +16,7 @@ latency:
16
  seed: 271828
17
  profile_path: profiles/wanoft/1x-rtx3090/mikasa_intercept_grab_fast/instance_3a0d42681a03715c/profile.json
18
  training:
19
- exp_name: small-ppo-mikasa-intercept-grab-fast-wanoft-rtx3090-temporal-scratch
20
  checkpoint: checkpoint.pt
21
  resume: null
22
  seed: 20260910
 
1
  env:
2
+ env_fps: 10
3
+ obs_fps: 10
4
  noop_action: [0, 0, 0, 0, 0, 0, 0]
5
  executor:
6
  simulated_worker_capacity: 1
 
16
  seed: 271828
17
  profile_path: profiles/wanoft/1x-rtx3090/mikasa_intercept_grab_fast/instance_3a0d42681a03715c/profile.json
18
  training:
19
+ exp_name: small-ppo-mikasa-intercept-grab-fast-wanoft-rtx3090-temporal-clock10-scratch
20
  checkpoint: checkpoint.pt
21
  resume: null
22
  seed: 20260910
latency-aware/mikasa-intercept-grab-fast/small-policy/ppo-wanoft-rtx3090-v1/provenance.json CHANGED
@@ -2,12 +2,12 @@
2
  "task": "mikasa-intercept-grab-fast",
3
  "model": "ppo-mlp-state",
4
  "status": "published",
5
- "run_id": "sw3b981e",
6
- "wandb_url": "https://wandb.ai/saberrr-zju/MIKASA-Robo-dataset-collectors/runs/sw3b981e",
7
  "checkpoint": "checkpoint.pt",
8
  "checkpoint_bytes": 1169937,
9
- "checkpoint_global_step": 69120000,
10
- "checkpoint_iteration": 1125,
11
  "selection_metric": "eval/success_at_end",
12
  "initialization": "from_scratch",
13
  "training_seed": 123,
@@ -17,7 +17,7 @@
17
  "path": "profiles/wanoft/1x-rtx3090/mikasa_intercept_grab_fast/instance_3a0d42681a03715c"
18
  },
19
  "trainer_source": {
20
- "wandb_run_id": "sw3b981e",
21
  "path": "code/third_party/MIKASA-Robo/mikasa_robo_suite/vla/dataset_collectors/get_dataset_collectors_ckpt.py"
22
  },
23
  "evaluation": "strict-eval.json"
 
2
  "task": "mikasa-intercept-grab-fast",
3
  "model": "ppo-mlp-state",
4
  "status": "published",
5
+ "run_id": "s463nqus",
6
+ "wandb_url": "https://wandb.ai/saberrr-zju/MIKASA-Robo-dataset-collectors/runs/s463nqus",
7
  "checkpoint": "checkpoint.pt",
8
  "checkpoint_bytes": 1169937,
9
+ "checkpoint_global_step": 23040000,
10
+ "checkpoint_iteration": 375,
11
  "selection_metric": "eval/success_at_end",
12
  "initialization": "from_scratch",
13
  "training_seed": 123,
 
17
  "path": "profiles/wanoft/1x-rtx3090/mikasa_intercept_grab_fast/instance_3a0d42681a03715c"
18
  },
19
  "trainer_source": {
20
+ "wandb_run_id": "s463nqus",
21
  "path": "code/third_party/MIKASA-Robo/mikasa_robo_suite/vla/dataset_collectors/get_dataset_collectors_ckpt.py"
22
  },
23
  "evaluation": "strict-eval.json"
latency-aware/mikasa-intercept-grab-fast/small-policy/ppo-wanoft-rtx3090-v1/strict-eval.json CHANGED
@@ -1,21 +1,21 @@
1
  {
2
  "episodes": 256,
3
  "metrics": {
4
- "success_once": 0.62890625,
5
- "return": 22.58401107788086,
6
  "episode_len": 60.0,
7
- "reward": 0.3764001727104187,
8
- "success_at_end": 0.4765625
9
  },
10
- "admitted_count": 2560,
11
- "dropped_count": 12800,
12
  "checkpoint": "checkpoint.pt",
13
- "checkpoint_global_step": 69120000,
14
  "evaluation_config": "eval.yaml",
15
- "success_rate": 0.4765625,
16
  "success_metric": "success_at_end",
17
- "success_once_count": 161,
18
- "success_at_end_count": 122,
19
  "environment_seed": 20260910,
20
  "latency_seed": 271828,
21
  "deterministic": true
 
1
  {
2
  "episodes": 256,
3
  "metrics": {
4
+ "success_once": 0.984375,
5
+ "return": 30.86635398864746,
6
  "episode_len": 60.0,
7
+ "reward": 0.5144392251968384,
8
+ "success_at_end": 0.9375
9
  },
10
+ "admitted_count": 5119,
11
+ "dropped_count": 10241,
12
  "checkpoint": "checkpoint.pt",
13
+ "checkpoint_global_step": 23040000,
14
  "evaluation_config": "eval.yaml",
15
+ "success_rate": 0.9375,
16
  "success_metric": "success_at_end",
17
+ "success_once_count": 252,
18
+ "success_at_end_count": 240,
19
  "environment_seed": 20260910,
20
  "latency_seed": 271828,
21
  "deterministic": true
latency-aware/mikasa-intercept-grab-fast/small-policy/ppo-wanoft-rtx3090-v1/train.yaml CHANGED
@@ -1,6 +1,6 @@
1
  env:
2
- env_fps: 20
3
- obs_fps: 20
4
  noop_action: [0, 0, 0, 0, 0, 0, 0]
5
  executor:
6
  simulated_worker_capacity: 1
@@ -16,7 +16,7 @@ latency:
16
  seed: 0
17
  profile_path: profiles/wanoft/1x-rtx3090/mikasa_intercept_grab_fast/instance_3a0d42681a03715c/profile.json
18
  training:
19
- exp_name: small-ppo-mikasa-intercept-grab-fast-wanoft-rtx3090-temporal-scratch
20
  checkpoint: null
21
  resume: null
22
  seed: 123
 
1
  env:
2
+ env_fps: 10
3
+ obs_fps: 10
4
  noop_action: [0, 0, 0, 0, 0, 0, 0]
5
  executor:
6
  simulated_worker_capacity: 1
 
16
  seed: 0
17
  profile_path: profiles/wanoft/1x-rtx3090/mikasa_intercept_grab_fast/instance_3a0d42681a03715c/profile.json
18
  training:
19
+ exp_name: small-ppo-mikasa-intercept-grab-fast-wanoft-rtx3090-temporal-clock10-scratch
20
  checkpoint: null
21
  resume: null
22
  seed: 123