zhouqh/act-dump-bin-interrupt-real

ACT — dump_bin_interrupt_real (real-rig calibrated xArm7)

Action Chunking Transformer trained on Genesis HR Bench's real-rig calibrated dump_bin_interrupt_real task: an xArm7 in the identity frame of the physical rig, with the RealSense D455 head-camera extrinsics and hand-eye wrist transform taken from the 2026-08 calibration (real/sim_setup.py).

Inputs / outputs

observation.image	256x256x3 — head camera (D455, world pose (0.925, -0.267, 0.293))
observation.wrist_image	256x256x3 — right wrist camera, attached to link7
observation.state	8-D [7 arm qpos, gripper], gripper 1.0 = open
action	8-D [7 commanded target qpos, gripper] — absolute, not deltas

Action space is qpos_target_abs: the recorded PD targets, not measured qpos and not deltas. Serve it with --action-type qpos_abs.

Training

Data	300 episodes, runs/real/vla_data_0808/dump_bin_interrupt_real (scripted rollouts, rasterizer)
Steps	25,000 (batch 80, 27.2 epochs)
Optimizer	AdamW, lr 1e-05, wd 0.0001, grad clip 10.0
Chunk	chunk_size=100, n_action_steps=1, temporal_ensemble_coeff=0.01
VAE	disabled (use_vae=false)
Normalization	MEAN_STD for visual / state / action
Final loss	0.028
Hardware	1x H100 80GB, 1h12m

Usage

# serve (lerobot v0.1.0 venv — this is a v0.1.0-format checkpoint:
# config.json + model.safetensors, no processor files)
python baseline/servers/lerobot_server.py \
    --policy-type act --checkpoint <local_dir> --port 8775

# evaluate
python scripts/eval_vla.py --task dump_bin_interrupt_real --model act \
    --server-url http://127.0.0.1:8775 --action-type qpos_abs

Status — read before using the benchmark number

The weights reproduce their training targets closely. Replaying the training episodes' own observations through the served policy (scripts/check_act_replay.py, frame-by-frame):

	joint MAE	gripper MAE	gripper range predicted
with temporal ensembler	0.012-0.014 rad	0.003	0.377 - 1.00
single-shot (ensembler cleared each step)	0.012-0.017 rad	0.004	0.375 - 1.01

(ground-truth gripper is 0.375 closed / 1.0 open, so the closed command is being reproduced correctly.)

However, in-simulator benchmark success is currently 0% — and that is an eval-harness bug, not a property of these weights. In the eval path the human avatar is left at its home pose, which sits directly in the calibrated head camera's field of view: the policy sees a torso where the robot arm should be, from step 0 onward. Re-running the collection pipeline on the same seed produces a clean view (the avatar's forearm only enters frame during the interrupt), so the training data is unaffected. Until that is fixed, do not read 0% as a capability measurement.

Provenance

• Task registry: real/tasks.py -> dump_bin_interrupt_real
• Training run: SLURM job 62715183, exp act_dump_bin_interrupt_real_20260524_qpos_target_abs_300ep_real0808_c100_s25000_novae_v1
• W&B project: act_genesis_hr_bench_real_0808 (entity multi-agent-world-model)
• Source dataset (LeRobot): ghb/dump_bin_interrupt_real_qt24_300_c100