real_deploy_script/DEPLOY_FIXES.md

Real-robot deploy — findings & required fixes (2026-08-11)

Symptom investigated: qpos and wrench policies behave normally on hardware; HTT (in-graph), T3, and SITR policies do not.

A. T3 / SITR — root cause found, new runner provided

run_ACT.py feeds obs["tactile"] = wrench_1 (30-d wrench) to EVERY policy. The T3/SITR checkpoints expect the 3840-d encoder feature vector; no encoder exists anywhere in the deploy scripts. Use the new run_ACT_t3_sitr.py, which computes features live via tools/live_features.py (validated against the training features). Needs TACTILE_FUSION_ROOT/GSRL_ROOT env vars and the encoder weights (see SETUP_ENCODERS_REMOTE.md in the HF bundle).

B. HTT in-graph (run_ACT_tactile_checkpoint.py) — 3 line-level fixes

1. Backgrounds captured at the wrong pose/time (primary suspect). Current order: command hand_initial_pose → sleep(1) → set_backgrounds (l.130) → 'c' → move to pose.json["pose1"] → run. Training bg (tactile_bg_224) = resting frame AT the episode start configuration. FIX: move the two bg lines (l.130–131) to AFTER the pose.json move and its rospy.sleep(5) (i.e., after l.144), so bg is captured settled, at the pose the episode actually starts from, immediately before stepping:

   rospy.sleep(5)
   bg = {key: finger_image(sharpa_tactile[i]) for i, key in enumerate(FINGER_KEYS)}
   infer.set_backgrounds(bg)
   

   Also verify the fingers are NOT in contact at that pose — bg must be a free/rest frame, as in the dataset.

2. inference_tactile_checkpoint.py l.112: act_cfg["n_action_steps"] = 15 hard-overrides the trained value (30). Delete the line (keep the setdefault above it) unless the 15-step re-query was a deliberate, validated choice.

3. Control rate: rospy.Rate(50) → rospy.Rate(30) in both runners. Demos are ~31 Hz; at 50 Hz chunks execute 1.6× too fast AND the in-graph branch's rolling frame-pair history gets 20 ms spacing vs ~32 ms at training — the image-history branch is the component most sensitive to this. (qpos/wrench tolerating 50 Hz does not mean the HTT branch does.)

C. Hygiene (not the bug, but will bite eventually)

• run_ACT.py l.41 hardcodes screw_qpos_50k while policies[arg] suggests selection — the log filename (l.52) records policies[3] regardless of what actually ran. Make the checkpoint path derive from the selection.
• finger_image's arr.max() <= 1.5 float heuristic mis-scales a genuinely dark uint8-as-float frame; prefer an explicit dtype contract from the publisher.

Recommended bring-up order after fixes

1. T3/SITR: run the live_features self-test on the robot machine first (PASS required), then run_ACT_t3_sitr.py with the hand in free space — confirm actions look like the demo's approach motion before touching the object.
2. HTT: after fixes B1–B3, log policy's tactile-branch feature norm for a few seconds at rest and compare to a training episode's feature norms — a mismatch >2× means bg is still wrong.