DIAGNOSTICS.md

Diagnostics log

Every failure in this suite that took more than one attempt to understand, with the measurement that settled it. Written down because the diagnosis is worth more than the fix: almost all of these look like tuning problems and are not.

The recurring lesson: when a grasp or a joint does not behave, measure the asset offline (trimesh on the source mesh, or the URDF's own numbers) before touching a parameter.

---

Asset geometry

The mesh origin is often at the bbox EDGE, not the centre

object_pos() returns the rigid body's origin. For several RoboTwin meshes the geometry is not centred on it, so the grasp lands beside the object. Symptom: gap 0.0 cm (jaws fully closed on nothing) or object did not rise — never anything that points at geometry.

asset	offset from origin	how it was found
083_brush	+3.7 cm in y (its Y runs 0..0.46, not −0.23..0.23)	trimesh: centroid of the handle slice
091_kettle	+4.7 cm in y	trimesh: per-slice centroid along the tall axis

Fix: measure the centroid of the part you intend to grip and pass grasp_offset.

Slice the mesh to find the graspable feature

Both the brush and the kettle were being gripped in the wrong place entirely. Slicing the mesh along its long axis and printing each slice's cross-section shows where the thin part is:

• 083_brush — 0–40 % of its length is a 1.0 cm handle, 40–100 % is the 2.7–3.6 cm head.
• 091_kettle — top 25 % is a 1.5 × 3.5 cm neck; the rest is a smooth 7 cm shell.

(The kettle's neck stalls the jaws but slips out under load — the body grip works once the y offset is corrected. A stall is not a grip.)

The bbox is pose-independent — object_size() ignores yaw

handover_basket sets yaw=90 so the basket's short axis faces the jaws, but auto_jaw() reads the authored bbox and so chose the wrong closing axis. Anything that yaws an object must state its jaw axis and its rotated half-extent explicitly.

RoboTwin GLBs are Y-up — and so are the SAPIEN articulations

Long known for the rigid meshes (rpy: (90,0,0)). It is also true of the articulated assets, which was not handled and broke every one of them:

• 036_cabinet — prismatic axis (0,0,1) with drawers stacked along Y ⇒ the drawers were being asked to slide straight up.
• 044_microwave — hinge axis (0,-1,0) ⇒ a microwave door hinging about a horizontal axis.
• 060_kitchenpot — lid slides along Y, i.e. vertically. Consistent.

place_articulation now applies roll=90 by default, mapping asset (x,y,z) → sim (x,-z,y).

Seat articulations from bounding_box.json, never from the USD

ComputeWorldBound on an articulation root returns an empty range, whose min is -3.4e38. Using it as a seat height threw every fixture to negative infinity. The shipped bounding_box.json × scale is deterministic and correct.

---

Asset conversion

A hyphen in a mesh filename silently produces an empty USD

SAPIEN names collision meshes original-7.obj. Hyphens are illegal in USD prim paths. Isaac's URDF importer half-handles it: renames the prim to original_7, then still tries to create </colliders/base/original-7>, fails to parse, dies with Used null prim — writes a 492-byte stub and returns normally. The breakage only surfaces much later as Failed to find an articulation ... ArticulationRootAPI.

Fix: scripts/sanitize_urdf_asset.py, plus a payload-size check in the converter that FAILS.

Renaming .mtl files without rewriting mtllib inside the .obj

An OBJ names its material library inside the file. Renaming the .mtl on disk leaves every mesh material-less and the whole asset imports as featureless grey — the microwave rendered as a plain white cube. The sanitizer now rewrites mtllib / usemtl / map_* lines too.

UrdfConverterCfg.collider_type defaults to convex_hull

That collapses a whole microwave into one box, swallowing the door, and the door's collider then interpenetrates the body's so the hinge cannot move. Forced to convex_decomposition.

A carried vessel cannot have an interior

PhysX will not simulate a hollow triangle mesh as a dynamic body, so anything liftable is convex and solid inside. Beads placed "in" the kettle were pushed onto the table before the episode started (measured at z=0.459, i.e. table height, every run). Static containers can be hollow — convert them with --collision none --suffix _mesh.

Consequence: pour_kettle is scored on the pour (vessel over the target, tipped past the angle its contents would leave it), not on contents landing.

---

The checking code lying

Three false positives, all of which reported SUCCESS for episodes that did nothing. These were worse than the task failures, because they hide them.

check	why it lied	fix
count_in_region	no height bound — a bead still inside the carried vessel counted as poured, so an episode whose grasp failed reported SUCCESS	added below_top, plus a per-object position print
joint_moved	baselined at build time, so the fixture settling under gravity counted as progress — switch_toggle "passed" on 0.44 rad of falling and 0.008 rad of robot work	joint_driven: the delta the solver measured across the action itself
joint_driven (first version)	still passed a joint that had blown to −7.99 rad against a [−2.339, 0.754] limit	also require the final value to be inside the joint's limits

ast.parse does not catch duplicate keyword arguments

My syntax check passed a call with grip_tilt= written twice; it only failed at run time. Use compile().

Verifying only with --no-randomize hides pose bugs

lid_food placed the lid at the class constant self.pot_xy while the pot itself was jittered — so with randomization on, the lid landed beside the pot. Invisible for as long as every check run used nominal poses. Always derive from the value the builder returned.

---

Robot workspace

Shoulders are at (-0.2, +0.2) and (-0.2, -0.2); the wrist parks at [0.078, -0.19].

• Comfortable reach is ~0.25 m; ~0.34 m still works; 0.38–0.40 m stalls. Several "grasp failed" bugs were only this — sweep_debris (0.38 m), cupboard_store (0.40 m), pour_kettle (0.39 m).
• Nearest arm is not the best arm. sort_cubes chose the right arm for a pad 9 cm from its own shoulder; folded up that far it could not servo (0.54 m tracking error). Score on |reach − 0.25| instead.
• Each arm can only push away from its own shoulder. To push in −y the hand must stand on the block's +y side, which the right arm (at y=−0.2) can only do by reaching over the block — the approach swipes it off the table. push_relay is an L-shaped route so each arm pushes its own natural direction.
• The home pose sits inside the only reachable placement band. This is why cupboard_store is parked: moved in, the unit's walls block the arm before it can reach the object; moved out, the shelf is past reach.

Motion

• The jaw opens 9.4 cm and clips above ~7.8 cm. check_graspable() warns.
• Push: the loop must be parameterised by CONTROL STEPS, not distance. Sizing it as span/advance gave 25 iterations = 25 sim steps for a 22 cm push, which reads exactly like "the block will not slide".
• Push penetration is capped at 10 mm. 17 mm into a 25 mm half-block launched it 1.8 m.
• Re-aim the pusher every step from the live pose. A line fixed at t=0 has no authority to steer a block that has skidded (13 cm off in x).
• Grip height for anything resting on a fixture must come from physics (live_z=True), not from the table. A lid on a 7.5 cm pot rim was grasped 7.5 cm too low.
• env.render() needs warm-up. A single render straight after build() returns black — all 29 tiles of the first overview sheet were black. Step and render repeatedly, keep the last.

Still open

• rope_straighten — the rope never leaves its off-scene spawn. It is the only floating-base articulation (the fixed-base ones all move). Try write_root_state_to_sim(). Separately, the chain interleaves massless spacer links, so body index 13 is not seg13 — use scene.link_index().
• microwave_door, drawer_open, drawer_store, laptop_close, switch_toggle — all load, seat and report their joints correctly now; the remaining problem is the grip pose against a flat, handle-less panel, and (for the switch) that the button's link frame is the hinge, so pressing there has zero moment arm.
• handover_basket — both hands grip, but the basket is empty by the time the second hand arrives. 0.062 scale (8.8 cm short axis) is the largest this basket can be and still be graspable at all.
• ring_post, push_relay, pour_cup, cupboard_store — see the notes in each task file.