| # Diagnostics log |
|
|
| Every failure in this suite that took more than one attempt to understand, with the *measurement* |
| that settled it. Written down because the diagnosis is worth more than the fix: almost all of |
| these look like tuning problems and are not. |
|
|
| The recurring lesson: **when a grasp or a joint does not behave, measure the asset offline |
| (trimesh on the source mesh, or the URDF's own numbers) before touching a parameter.** |
|
|
| --- |
|
|
| ## Asset geometry |
|
|
| ### The mesh origin is often at the bbox EDGE, not the centre |
| `object_pos()` returns the rigid body's origin. For several RoboTwin meshes the geometry is not |
| centred on it, so the grasp lands beside the object. Symptom: `gap 0.0 cm` (jaws fully closed on |
| nothing) or `object did not rise` — never anything that points at geometry. |
|
|
| | asset | offset from origin | how it was found | |
| |---|---|---| |
| | `083_brush` | **+3.7 cm in y** (its Y runs 0..0.46, not −0.23..0.23) | trimesh: centroid of the handle slice | |
| | `091_kettle` | **+4.7 cm in y** | trimesh: per-slice centroid along the tall axis | |
|
|
| Fix: measure the centroid of the part you intend to grip and pass `grasp_offset`. |
|
|
| ### Slice the mesh to find the graspable feature |
| Both the brush and the kettle were being gripped in the wrong place entirely. Slicing the mesh |
| along its long axis and printing each slice's cross-section shows where the thin part is: |
|
|
| * `083_brush` — 0–40 % of its length is a **1.0 cm handle**, 40–100 % is the 2.7–3.6 cm head. |
| * `091_kettle` — top 25 % is a **1.5 × 3.5 cm neck**; the rest is a smooth 7 cm shell. |
|
|
| (The kettle's neck stalls the jaws but slips out under load — the body grip works once the y |
| offset is corrected. A stall is not a grip.) |
|
|
| ### The bbox is pose-independent — `object_size()` ignores yaw |
| `handover_basket` sets `yaw=90` so the basket's short axis faces the jaws, but `auto_jaw()` reads |
| the **authored** bbox and so chose the wrong closing axis. Anything that yaws an object must |
| state its jaw axis and its rotated half-extent explicitly. |
| |
| ### RoboTwin GLBs are Y-up — and so are the SAPIEN articulations |
| Long known for the rigid meshes (`rpy: (90,0,0)`). It is **also** true of the articulated assets, |
| which was not handled and broke every one of them: |
| |
| * `036_cabinet` — prismatic axis `(0,0,1)` with drawers stacked along **Y** ⇒ the drawers were |
| being asked to slide straight **up**. |
| * `044_microwave` — hinge axis `(0,-1,0)` ⇒ a microwave door hinging about a **horizontal** axis. |
| * `060_kitchenpot` — lid slides along **Y**, i.e. vertically. Consistent. |
|
|
| `place_articulation` now applies `roll=90` by default, mapping asset `(x,y,z)` → sim `(x,-z,y)`. |
|
|
| ### Seat articulations from `bounding_box.json`, never from the USD |
| `ComputeWorldBound` on an articulation root returns an **empty range**, whose min is `-3.4e38`. |
| Using it as a seat height threw every fixture to negative infinity. The shipped |
| `bounding_box.json × scale` is deterministic and correct. |
|
|
| --- |
|
|
| ## Asset conversion |
|
|
| ### A hyphen in a mesh filename silently produces an empty USD |
| SAPIEN names collision meshes `original-7.obj`. Hyphens are illegal in USD prim paths. Isaac's |
| URDF importer half-handles it: renames the prim to `original_7`, then still tries to create |
| `</colliders/base/original-7>`, fails to parse, dies with `Used null prim` — **writes a 492-byte |
| stub and returns normally**. The breakage only surfaces much later as |
| `Failed to find an articulation ... ArticulationRootAPI`. |
|
|
| Fix: `scripts/sanitize_urdf_asset.py`, plus a payload-size check in the converter that FAILS. |
|
|
| ### Renaming `.mtl` files without rewriting `mtllib` inside the `.obj` |
| An OBJ names its material library **inside the file**. Renaming the `.mtl` on disk leaves every |
| mesh material-less and the whole asset imports as featureless grey — the microwave rendered as a |
| plain white cube. The sanitizer now rewrites `mtllib` / `usemtl` / `map_*` lines too. |
|
|
| ### `UrdfConverterCfg.collider_type` defaults to `convex_hull` |
| That collapses a whole microwave into one box, swallowing the door, and the door's collider then |
| interpenetrates the body's so the hinge cannot move. Forced to `convex_decomposition`. |
|
|
| ### A carried vessel cannot have an interior |
| PhysX will not simulate a hollow triangle mesh as a **dynamic** body, so anything liftable is |
| convex and solid inside. Beads placed "in" the kettle were pushed onto the table before the |
| episode started (measured at `z=0.459`, i.e. table height, every run). Static containers can be |
| hollow — convert them with `--collision none --suffix _mesh`. |
|
|
| Consequence: `pour_kettle` is scored on the **pour** (vessel over the target, tipped past the |
| angle its contents would leave it), not on contents landing. |
|
|
| --- |
|
|
| ## The checking code lying |
|
|
| Three false positives, all of which reported SUCCESS for episodes that did nothing. These were |
| worse than the task failures, because they hide them. |
|
|
| | check | why it lied | fix | |
| |---|---|---| |
| | `count_in_region` | no height bound — a bead still inside the carried vessel counted as poured, so an episode whose **grasp failed** reported SUCCESS | added `below_top`, plus a per-object position print | |
| | `joint_moved` | baselined at build time, so the fixture settling under gravity counted as progress — `switch_toggle` "passed" on 0.44 rad of falling and 0.008 rad of robot work | `joint_driven`: the delta the solver measured across the action itself | |
| | `joint_driven` (first version) | still passed a joint that had blown to **−7.99 rad** against a `[−2.339, 0.754]` limit | also require the final value to be inside the joint's limits | |
|
|
| ### `ast.parse` does not catch duplicate keyword arguments |
| My syntax check passed a call with `grip_tilt=` written twice; it only failed at run time. Use |
| `compile()`. |
|
|
| ### Verifying only with `--no-randomize` hides pose bugs |
| `lid_food` placed the lid at the class constant `self.pot_xy` while the pot itself was jittered — |
| so with randomization on, the lid landed beside the pot. Invisible for as long as every check run |
| used nominal poses. Always derive from the value the builder **returned**. |
|
|
| --- |
|
|
| ## Robot workspace |
|
|
| Shoulders are at `(-0.2, +0.2)` and `(-0.2, -0.2)`; the wrist parks at `[0.078, -0.19]`. |
|
|
| * **Comfortable reach is ~0.25 m; ~0.34 m still works; 0.38–0.40 m stalls.** Several "grasp |
| failed" bugs were only this — `sweep_debris` (0.38 m), `cupboard_store` (0.40 m), |
| `pour_kettle` (0.39 m). |
| * **Nearest arm is not the best arm.** `sort_cubes` chose the right arm for a pad 9 cm from its |
| own shoulder; folded up that far it could not servo (0.54 m tracking error). Score on |
| `|reach − 0.25|` instead. |
| * **Each arm can only push away from its own shoulder.** To push in −y the hand must stand on the |
| block's +y side, which the right arm (at y=−0.2) can only do by reaching over the block — |
| the approach swipes it off the table. `push_relay` is an L-shaped route so each arm pushes its |
| own natural direction. |
| * **The home pose sits inside the only reachable placement band.** This is why `cupboard_store` |
| is parked: moved in, the unit's walls block the arm before it can reach the object; moved out, |
| the shelf is past reach. |
|
|
| ## Motion |
|
|
| * **The jaw opens 9.4 cm and clips above ~7.8 cm.** `check_graspable()` warns. |
| * **Push: the loop must be parameterised by CONTROL STEPS, not distance.** Sizing it as |
| `span/advance` gave 25 iterations = 25 sim steps for a 22 cm push, which reads exactly like |
| "the block will not slide". |
| * **Push penetration is capped at 10 mm.** 17 mm into a 25 mm half-block launched it 1.8 m. |
| * **Re-aim the pusher every step from the live pose.** A line fixed at t=0 has no authority to |
| steer a block that has skidded (13 cm off in x). |
| * **Grip height for anything resting on a fixture must come from physics** (`live_z=True`), not |
| from the table. A lid on a 7.5 cm pot rim was grasped 7.5 cm too low. |
| * **`env.render()` needs warm-up.** A single render straight after `build()` returns black — all |
| 29 tiles of the first overview sheet were black. Step and render repeatedly, keep the last. |
|
|
| ## Still open |
|
|
| * `rope_straighten` — the rope never leaves its off-scene spawn. It is the only **floating-base** |
| articulation (the fixed-base ones all move). Try `write_root_state_to_sim()`. Separately, the |
| chain interleaves massless spacer links, so body index 13 is **not** `seg13` — use |
| `scene.link_index()`. |
| * `microwave_door`, `drawer_open`, `drawer_store`, `laptop_close`, `switch_toggle` — all load, |
| seat and report their joints correctly now; the remaining problem is the grip pose against a |
| flat, handle-less panel, and (for the switch) that the button's link frame **is** the hinge, so |
| pressing there has zero moment arm. |
| * `handover_basket` — both hands grip, but the basket is empty by the time the second hand |
| arrives. 0.062 scale (8.8 cm short axis) is the largest this basket can be and still be |
| graspable at all. |
| * `ring_post`, `push_relay`, `pour_cup`, `cupboard_store` — see the notes in each task file. |
|
|
