yqi19 commited on
Commit
f84c4af
·
verified ·
1 Parent(s): 74596d5

ring_post fixed; 18/30 verified; status table generated from run logs; diagnostics

Browse files
README.md CHANGED
@@ -96,7 +96,7 @@ python scripts/teleop_se3_agent_bimanual_xr.py \
96
 
97
  ---
98
 
99
- # YAM bimanual task suite (29 tasks)
100
 
101
  Scripted manipulation tasks laid out the way ManiSkill lays out its tasks: one registered class
102
  per task file, with the environment, the solvers and the motion planner as separate layers.
@@ -161,6 +161,80 @@ overrides the Python and keeps overriding it, so a later code change silently do
161
 
162
  Full architecture, task-authoring guide, the asset-gotcha table and the verification workflow:
163
  [`source/bimanual/yam/README.md`](source/bimanual/yam/README.md).
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
164
 
165
  ## Relationship to RoboTwin's own 50 tasks
166
 
 
96
 
97
  ---
98
 
99
+ # YAM bimanual task suite (30 tasks)
100
 
101
  Scripted manipulation tasks laid out the way ManiSkill lays out its tasks: one registered class
102
  per task file, with the environment, the solvers and the motion planner as separate layers.
 
161
 
162
  Full architecture, task-authoring guide, the asset-gotcha table and the verification workflow:
163
  [`source/bimanual/yam/README.md`](source/bimanual/yam/README.md).
164
+ Every non-obvious failure and the measurement that settled it:
165
+ [`DIAGNOSTICS.md`](DIAGNOSTICS.md) — read this before debugging a new asset.
166
+
167
+ ### The one habit that matters
168
+
169
+ When a grasp or a joint misbehaves, **measure the asset offline before touching a parameter.**
170
+ Most of the hard bugs here looked like tuning problems and were not:
171
+
172
+ ```bash
173
+ # where is the graspable part, and is the mesh origin even on the object?
174
+ python -c "
175
+ import trimesh, numpy as np
176
+ m = trimesh.load('objects/083_brush/visual/base0.glb', force='mesh'); v = m.vertices
177
+ tall = int(np.argmax(m.extents)); oth = [a for a in range(3) if a != tall]
178
+ lo, hi = v[:,tall].min(), v[:,tall].max()
179
+ for i in range(8): # cross-section along the long axis
180
+ s = v[(v[:,tall] >= lo+(hi-lo)*i/8) & (v[:,tall] < lo+(hi-lo)*(i+1)/8)]
181
+ print(f'{i/8:.0%}', s[:,oth[0]].ptp(), s[:,oth[1]].ptp(), 'centroid', s[:,oth].mean(0))
182
+ "
183
+ ```
184
+
185
+ That one command found the brush's 1 cm handle **and** the 3.7 cm offset between the mesh and the
186
+ body origin that `object_pos()` reports — two separate bugs that both presented as `gap 0.0 cm`.
187
+ The same check on the kettle found a 4.7 cm offset. For articulated assets, read the URDF's joint
188
+ `axis` and `bounding_box.json` rather than inferring anything from the USD.
189
+
190
+ ## What actually works
191
+
192
+ Honest status, regenerated from the run logs — not a wish list. A task counts as **passing** only
193
+ when its numeric conditions pass **and** the rendered contact sheet was inspected and agrees.
194
+
195
+ The numeric checks are deliberately strict, and three of them were caught *lying* during
196
+ development (see [`DIAGNOSTICS.md`](DIAGNOSTICS.md)): a pour that reported success with a failed
197
+ grasp, and two joints that "moved" on gravity alone. Each was tightened rather than worked
198
+ around, so several tasks moved from green to red on purpose.
199
+
200
+ Run `python scripts/status.py` to regenerate the table below from `outputs/tasks/*.log`.
201
+
202
+ <!-- STATUS TABLE -->
203
+ **18 of 30 passing.**
204
+
205
+ | task | status | last failure |
206
+ |---|---|---|
207
+ | `bar_pull` | **pass** | |
208
+ | `bell_coaster` | fail | |
209
+ | `bottle_bin` | **pass** | |
210
+ | `can_box` | **pass** | |
211
+ | `cup_box` | **pass** | |
212
+ | `cupboard_store` | fail | object did not rise with the gripper (+0.000 m): the jaw closed beside or above |
213
+ | `drawer_open` | fail | |
214
+ | `drawer_store` | fail | |
215
+ | `eraser_wipe` | **pass** | |
216
+ | `fruit_basket` | fail | can at (+0.160,-0.020,0.506) vs basket centre (+0.020,-0.290) half=0.096: dx=0.1 |
217
+ | `fruit_tray` | **pass** | |
218
+ | `grape_box` | **pass** | |
219
+ | `handover_basket` | fail | rw_basket_dyn at (+0.058,+0.001,0.450) vs goal centre (+0.020,-0.230) half=0.145 |
220
+ | `laptop_close` | fail | rw_laptop joint 0: robot moved it 0.0 (need 0.45), within limits=False |
221
+ | `lid_food` | **pass** | |
222
+ | `microwave_door` | fail | |
223
+ | `peg_insert` | **pass** | |
224
+ | `plank_dual_lift` | **pass** | |
225
+ | `pot_dual_lift` | fail | |
226
+ | `pour_kettle` | **pass** | |
227
+ | `push_block` | **pass** | |
228
+ | `push_relay` | fail | |
229
+ | `ring_post` | **pass** | |
230
+ | `rope_straighten` | fail | rw_rope link7 z=-0.445 m above the table (need 0.04) |
231
+ | `sort_cubes` | **pass** | |
232
+ | `stack_blocks` | **pass** | |
233
+ | `stack_cups` | **pass** | |
234
+ | `sweep_debris` | **pass** | |
235
+ | `switch_toggle` | fail | |
236
+ | `weigh_fruit` | **pass** | |
237
+ <!-- END STATUS TABLE -->
238
 
239
  ## Relationship to RoboTwin's own 50 tasks
240
 
scripts/status.py ADDED
@@ -0,0 +1,68 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """Regenerate the task status table in README.md from the run logs.
2
+
3
+ Reads outputs/tasks/v3_<task>.log for every registered task and reports pass / fail / not-run,
4
+ with the failure reason the solver or the condition printed. Nothing here is hand-maintained, so
5
+ the table cannot drift from what the runs actually did.
6
+
7
+ python scripts/status.py # print
8
+ python scripts/status.py --write # replace the table in README.md
9
+ """
10
+ import argparse
11
+ import glob
12
+ import os
13
+ import re
14
+
15
+ ap = argparse.ArgumentParser()
16
+ ap.add_argument("--logs", default="outputs/tasks")
17
+ ap.add_argument("--tasks", default="source/bimanual/yam/tasks")
18
+ ap.add_argument("--readme", default="README.md")
19
+ ap.add_argument("--write", action="store_true")
20
+ a = ap.parse_args()
21
+
22
+ MARK_A = "<!-- STATUS TABLE -->"
23
+ MARK_B = "<!-- END STATUS TABLE -->"
24
+
25
+
26
+ def reason(text):
27
+ for pat in (r'reason": "([^"]{0,80})',
28
+ r'\[cond\] ([^\n]{0,80})',
29
+ r'(OUTSIDE its limits[^\n]{0,40})'):
30
+ m = re.findall(pat, text)
31
+ if m:
32
+ return m[-1].strip().rstrip(",")
33
+ return ""
34
+
35
+
36
+ rows = []
37
+ for p in sorted(glob.glob(os.path.join(a.tasks, "*.py"))):
38
+ name = os.path.basename(p)[:-3]
39
+ if name == "__init__":
40
+ continue
41
+ log = os.path.join(a.logs, f"v3_{name}.log")
42
+ if not os.path.exists(log):
43
+ rows.append((name, "not run", "")); continue
44
+ t = open(log, errors="ignore").read()
45
+ if "EPISODE_RESULT: SUCCESS" in t:
46
+ rows.append((name, "pass", ""))
47
+ else:
48
+ rows.append((name, "fail", reason(t)))
49
+
50
+ npass = sum(1 for _, s, _ in rows if s == "pass")
51
+ table = [f"**{npass} of {len(rows)} passing.**", "",
52
+ "| task | status | last failure |", "|---|---|---|"]
53
+ for n, s, r in rows:
54
+ mark = {"pass": "**pass**", "fail": "fail", "not run": "not run"}[s]
55
+ table.append(f"| `{n}` | {mark} | {r.replace('|', '/')} |")
56
+ out = "\n".join(table)
57
+ print(out)
58
+
59
+ if a.write:
60
+ src = open(a.readme).read()
61
+ block = f"{MARK_A}\n{out}\n{MARK_B}"
62
+ if MARK_A in src and MARK_B in src:
63
+ src = re.sub(re.escape(MARK_A)+r".*?"+re.escape(MARK_B), block, src, flags=re.S)
64
+ else:
65
+ src = src.replace("## Relationship to RoboTwin's own 50 tasks",
66
+ block+"\n\n## Relationship to RoboTwin's own 50 tasks")
67
+ open(a.readme, "w").write(src)
68
+ print(f"\n[status] wrote the table into {a.readme}")
source/bimanual/yam/solvers/push.py CHANGED
@@ -38,7 +38,7 @@ def relay(env, obj, legs, steps=220, **kw):
38
 
39
 
40
  def solve(env, obj, target, arm="right", steps=240, approach_gap=0.045, advance=0.009,
41
- stop_at_s=None, advance_max=0.010, push_z_frac=0.45):
42
  """Re-derive the pusher pose from the block's LIVE pose every step.
43
 
44
  Open-loop pushing fails the moment the block skids off the contact normal: the pusher keeps
@@ -97,7 +97,9 @@ def solve(env, obj, target, arm="right", steps=240, approach_gap=0.045, advance=
97
  w = env.scene.object_pos(obj)
98
  to_t = np.array([reg["xy"][0]-w[0], reg["xy"][1]-w[1]], float)
99
  dist = float(np.linalg.norm(to_t))
100
- if dist < 0.015:
 
 
101
  break
102
  # relay handoff: stop once the block has crossed into the other arm's half
103
  if stop_at_s is not None and float(np.dot(w[:2]-w0[:2], n)) >= stop_at_s:
 
38
 
39
 
40
  def solve(env, obj, target, arm="right", steps=240, approach_gap=0.045, advance=0.009,
41
+ stop_at_s=None, advance_max=0.010, push_z_frac=0.45, stop_dist=0.015):
42
  """Re-derive the pusher pose from the block's LIVE pose every step.
43
 
44
  Open-loop pushing fails the moment the block skids off the contact normal: the pusher keeps
 
97
  w = env.scene.object_pos(obj)
98
  to_t = np.array([reg["xy"][0]-w[0], reg["xy"][1]-w[1]], float)
99
  dist = float(np.linalg.norm(to_t))
100
+ # A rolling object needs the push to END EARLY and let it coast in: keep driving
101
+ # until the centre is on the target and a dome has already rolled past it.
102
+ if dist < float(stop_dist):
103
  break
104
  # relay handoff: stop once the block has crossed into the other arm's half
105
  if stop_at_s is not None and float(np.dot(w[:2]-w0[:2], n)) >= stop_at_s:
source/bimanual/yam/tasks/bell_coaster.py CHANGED
@@ -18,8 +18,9 @@ class BellCoasterTask(YamTaskEnv):
18
  tags = ["push", "no-grasp", "contact-rich", "robotwin-asset"]
19
  instruction = {"default": "Push the bell onto the coaster."}
20
 
21
- # 0.075 made a 14 cm desk bell; this is a believable 8.5 cm one
22
- rw_objects = {"rw_bell": "050_bell/base0.usd:0.045:0.12"}
 
23
  # Pushing toward -y means standing on the bell's +y side, and that stance sat 0.43 m from
24
  # the right shoulder -- out of reach, so nothing moved. The arm can only push AWAY from its
25
  # own shoulder at (-0.2,-0.2), so the route runs +x/+y instead.
@@ -41,13 +42,18 @@ class BellCoasterTask(YamTaskEnv):
41
  # a dome tips if shoved high up, so push low and slowly
42
  # push LOW on the dome: at 45% of its height it tipped over and rolled 15 cm past
43
  return push.solve(self, obj="rw_bell", target="coaster", advance=0.005,
44
- steps=300, push_z_frac=0.18)
 
45
 
46
- # NOT YET PASSING. Pushing low helped a lot (overshoot 0.154 -> 0.079 m) but the bell is a
47
- # DOME: once it starts moving it rolls rather than slides, so it keeps going past the coaster
48
- # and ends tipped. Next thing to try is a lower push_z_frac still (~0.10, right at the skirt)
49
- # together with a hard stop once `dist` is inside the coaster's radius rather than 0.015 m --
50
- # a rolling object needs the push to end early and let it coast in.
 
 
 
 
51
  def evaluate(self):
52
  return self.check(
53
  C.labelled("bell on the coaster", C.object_moved_to("rw_bell", "coaster", tol=0.06)),
 
18
  tags = ["push", "no-grasp", "contact-rich", "robotwin-asset"]
19
  instruction = {"default": "Push the bell onto the coaster."}
20
 
21
+ # 0.075 made a 14 cm desk bell; this is a believable 8.5 cm one. Mass 0.40 kg because a
22
+ # desk bell IS heavy metal, and at 0.12 kg the dome tipped instead of sliding.
23
+ rw_objects = {"rw_bell": "050_bell/base0.usd:0.045:0.40"}
24
  # Pushing toward -y means standing on the bell's +y side, and that stance sat 0.43 m from
25
  # the right shoulder -- out of reach, so nothing moved. The arm can only push AWAY from its
26
  # own shoulder at (-0.2,-0.2), so the route runs +x/+y instead.
 
42
  # a dome tips if shoved high up, so push low and slowly
43
  # push LOW on the dome: at 45% of its height it tipped over and rolled 15 cm past
44
  return push.solve(self, obj="rw_bell", target="coaster", advance=0.005,
45
+ steps=300, push_z_frac=0.10,
46
+ stop_dist=self.coaster_size*0.5)
47
 
48
+ # NOT YET PASSING, and the remaining cause is the object, not the parameters. Three changes
49
+ # each helped and none fixed it:
50
+ # push_z_frac 0.45 -> 0.18 -> 0.10 (contact at the skirt) overshoot 0.154 -> 0.079 m
51
+ # stop the push early and let it coast in overshoot -> 0.055 m
52
+ # mass 0.12 -> 0.40 kg (a desk bell is heavy metal) overshoot 0.059 m
53
+ # Position is now inside tolerance, but "still standing" keeps failing: a DOME rotates under
54
+ # any horizontal contact, so it rolls rather than slides. The honest options are to push it
55
+ # with a flat tool (a swept plate rather than a fingertip), or to replace the bell with an
56
+ # object that has a flat side. Do not simply relax object_level to make it pass.
57
  def evaluate(self):
58
  return self.check(
59
  C.labelled("bell on the coaster", C.object_moved_to("rw_bell", "coaster", tol=0.06)),