# Diagnostics log Every failure in this suite that took more than one attempt to understand, with the *measurement* that settled it. Written down because the diagnosis is worth more than the fix: almost all of these look like tuning problems and are not. The recurring lesson: **when a grasp or a joint does not behave, measure the asset offline (trimesh on the source mesh, or the URDF's own numbers) before touching a parameter.** --- ## Asset geometry ### The mesh origin is often at the bbox EDGE, not the centre `object_pos()` returns the rigid body's origin. For several RoboTwin meshes the geometry is not centred on it, so the grasp lands beside the object. Symptom: `gap 0.0 cm` (jaws fully closed on nothing) or `object did not rise` — never anything that points at geometry. | asset | offset from origin | how it was found | |---|---|---| | `083_brush` | **+3.7 cm in y** (its Y runs 0..0.46, not −0.23..0.23) | trimesh: centroid of the handle slice | | `091_kettle` | **+4.7 cm in y** | trimesh: per-slice centroid along the tall axis | Fix: measure the centroid of the part you intend to grip and pass `grasp_offset`. ### Slice the mesh to find the graspable feature Both the brush and the kettle were being gripped in the wrong place entirely. Slicing the mesh along its long axis and printing each slice's cross-section shows where the thin part is: * `083_brush` — 0–40 % of its length is a **1.0 cm handle**, 40–100 % is the 2.7–3.6 cm head. * `091_kettle` — top 25 % is a **1.5 × 3.5 cm neck**; the rest is a smooth 7 cm shell. (The kettle's neck stalls the jaws but slips out under load — the body grip works once the y offset is corrected. A stall is not a grip.) ### The bbox is pose-independent — `object_size()` ignores yaw `handover_basket` sets `yaw=90` so the basket's short axis faces the jaws, but `auto_jaw()` reads the **authored** bbox and so chose the wrong closing axis. Anything that yaws an object must state its jaw axis and its rotated half-extent explicitly. ### RoboTwin GLBs are Y-up — and so are the SAPIEN articulations Long known for the rigid meshes (`rpy: (90,0,0)`). It is **also** true of the articulated assets, which was not handled and broke every one of them: * `036_cabinet` — prismatic axis `(0,0,1)` with drawers stacked along **Y** ⇒ the drawers were being asked to slide straight **up**. * `044_microwave` — hinge axis `(0,-1,0)` ⇒ a microwave door hinging about a **horizontal** axis. * `060_kitchenpot` — lid slides along **Y**, i.e. vertically. Consistent. `place_articulation` now applies `roll=90` by default, mapping asset `(x,y,z)` → sim `(x,-z,y)`. ### Seat articulations from `bounding_box.json`, never from the USD `ComputeWorldBound` on an articulation root returns an **empty range**, whose min is `-3.4e38`. Using it as a seat height threw every fixture to negative infinity. The shipped `bounding_box.json × scale` is deterministic and correct. --- ## Asset conversion ### A hyphen in a mesh filename silently produces an empty USD SAPIEN names collision meshes `original-7.obj`. Hyphens are illegal in USD prim paths. Isaac's URDF importer half-handles it: renames the prim to `original_7`, then still tries to create ``, fails to parse, dies with `Used null prim` — **writes a 492-byte stub and returns normally**. The breakage only surfaces much later as `Failed to find an articulation ... ArticulationRootAPI`. Fix: `scripts/sanitize_urdf_asset.py`, plus a payload-size check in the converter that FAILS. ### Renaming `.mtl` files without rewriting `mtllib` inside the `.obj` An OBJ names its material library **inside the file**. Renaming the `.mtl` on disk leaves every mesh material-less and the whole asset imports as featureless grey — the microwave rendered as a plain white cube. The sanitizer now rewrites `mtllib` / `usemtl` / `map_*` lines too. ### `UrdfConverterCfg.collider_type` defaults to `convex_hull` That collapses a whole microwave into one box, swallowing the door, and the door's collider then interpenetrates the body's so the hinge cannot move. Forced to `convex_decomposition`. ### A carried vessel cannot have an interior PhysX will not simulate a hollow triangle mesh as a **dynamic** body, so anything liftable is convex and solid inside. Beads placed "in" the kettle were pushed onto the table before the episode started (measured at `z=0.459`, i.e. table height, every run). Static containers can be hollow — convert them with `--collision none --suffix _mesh`. Consequence: `pour_kettle` is scored on the **pour** (vessel over the target, tipped past the angle its contents would leave it), not on contents landing. --- ## The checking code lying Three false positives, all of which reported SUCCESS for episodes that did nothing. These were worse than the task failures, because they hide them. | check | why it lied | fix | |---|---|---| | `count_in_region` | no height bound — a bead still inside the carried vessel counted as poured, so an episode whose **grasp failed** reported SUCCESS | added `below_top`, plus a per-object position print | | `joint_moved` | baselined at build time, so the fixture settling under gravity counted as progress — `switch_toggle` "passed" on 0.44 rad of falling and 0.008 rad of robot work | `joint_driven`: the delta the solver measured across the action itself | | `joint_driven` (first version) | still passed a joint that had blown to **−7.99 rad** against a `[−2.339, 0.754]` limit | also require the final value to be inside the joint's limits | ### `ast.parse` does not catch duplicate keyword arguments My syntax check passed a call with `grip_tilt=` written twice; it only failed at run time. Use `compile()`. ### Verifying only with `--no-randomize` hides pose bugs `lid_food` placed the lid at the class constant `self.pot_xy` while the pot itself was jittered — so with randomization on, the lid landed beside the pot. Invisible for as long as every check run used nominal poses. Always derive from the value the builder **returned**. --- ## Robot workspace Shoulders are at `(-0.2, +0.2)` and `(-0.2, -0.2)`; the wrist parks at `[0.078, -0.19]`. * **Comfortable reach is ~0.25 m; ~0.34 m still works; 0.38–0.40 m stalls.** Several "grasp failed" bugs were only this — `sweep_debris` (0.38 m), `cupboard_store` (0.40 m), `pour_kettle` (0.39 m). * **Nearest arm is not the best arm.** `sort_cubes` chose the right arm for a pad 9 cm from its own shoulder; folded up that far it could not servo (0.54 m tracking error). Score on `|reach − 0.25|` instead. * **Each arm can only push away from its own shoulder.** To push in −y the hand must stand on the block's +y side, which the right arm (at y=−0.2) can only do by reaching over the block — the approach swipes it off the table. `push_relay` is an L-shaped route so each arm pushes its own natural direction. * **The home pose sits inside the only reachable placement band.** This is why `cupboard_store` is parked: moved in, the unit's walls block the arm before it can reach the object; moved out, the shelf is past reach. ## Motion * **The jaw opens 9.4 cm and clips above ~7.8 cm.** `check_graspable()` warns. * **Push: the loop must be parameterised by CONTROL STEPS, not distance.** Sizing it as `span/advance` gave 25 iterations = 25 sim steps for a 22 cm push, which reads exactly like "the block will not slide". * **Push penetration is capped at 10 mm.** 17 mm into a 25 mm half-block launched it 1.8 m. * **Re-aim the pusher every step from the live pose.** A line fixed at t=0 has no authority to steer a block that has skidded (13 cm off in x). * **Grip height for anything resting on a fixture must come from physics** (`live_z=True`), not from the table. A lid on a 7.5 cm pot rim was grasped 7.5 cm too low. * **`env.render()` needs warm-up.** A single render straight after `build()` returns black — all 29 tiles of the first overview sheet were black. Step and render repeatedly, keep the last. ## Still open * `rope_straighten` — the rope never leaves its off-scene spawn. It is the only **floating-base** articulation (the fixed-base ones all move). Try `write_root_state_to_sim()`. Separately, the chain interleaves massless spacer links, so body index 13 is **not** `seg13` — use `scene.link_index()`. * `microwave_door`, `drawer_open`, `drawer_store`, `laptop_close`, `switch_toggle` — all load, seat and report their joints correctly now; the remaining problem is the grip pose against a flat, handle-less panel, and (for the switch) that the button's link frame **is** the hinge, so pressing there has zero moment arm. * `handover_basket` — both hands grip, but the basket is empty by the time the second hand arrives. 0.062 scale (8.8 cm short axis) is the largest this basket can be and still be graspable at all. * `ring_post`, `push_relay`, `pour_cup`, `cupboard_store` — see the notes in each task file.