GR00T-N1.7-CubeStack-megamix

GR00T N1.7 (nvidia/GR00T-N1.7-3B) finetuned on a large multitask mixture: all 13 publicly available unitreerobotics/G1_Dex3_* demonstration datasets plus 16 of our own synthetic Isaac Lab cube-stacking episodes, head camera only (cam_left_high), absolute joint-position targets for all action groups.

Part of the G1 Dex3 Cube Stacking Masterprojekt (Fichtl00/Masterprojekt_G1_CubeStack).

Training data (3168 episodes, 2,592,840 frames)

Dataset	Episodes
Own synthetic Isaac Lab cube-stacking	16
unitreerobotics/G1_Dex3_BlockStacking_Dataset	301
unitreerobotics/G1_Dex3_CameraPackaging_Dataset	201
unitreerobotics/G1_Dex3_GraspSquare_Dataset	301
unitreerobotics/G1_Dex3_ObjectPlacement_Dataset	210
unitreerobotics/G1_Dex3_Pouring_Dataset	311
unitreerobotics/G1_Dex3_ToastedBread_Dataset	418
unitreerobotics/G1_Dex3_PickApple_Dataset	201
unitreerobotics/G1_Dex3_PickBottle_Dataset	202
unitreerobotics/G1_Dex3_PickCharger_Dataset	200
unitreerobotics/G1_Dex3_PickGum_Dataset	199
unitreerobotics/G1_Dex3_PickSnack_Dataset	200
unitreerobotics/G1_Dex3_PickTissue_Dataset	205
unitreerobotics/G1_Dex3_PickDoll_Dataset	203

All real datasets were remapped from the DEX3_ARM_HAND real-dataset hand-joint convention into the GR00T-LEAPP unitree_g1 registry convention (index/middle/thumb per side) to match our own synthetic data and modality config.

• Base model: nvidia/GR00T-N1.7-3B
• Modality config: left_arm, right_arm, left_hand, right_hand, all ActionRepresentation.ABSOLUTE, video key cam_left_high only
• Training: 324,000 steps (~1 epoch), global batch size 8, lr 1e-4, single L40S GPU, ~38.2h wall time
• Final train_loss: 0.0532

Evaluation

Open-loop (reconstruction of held-in trajectories, one per dataset): average MSE 0.0126, average MAE 0.0548 across all 14 task configurations. Own cube-stacking task scored best (MSE 0.0045), PickDoll scored worst (MSE 0.0425) -- see the Masterprojekt repo for the full per-task breakdown.

Closed-loop (Isaac Lab sim, JointSpaceV2 env, head camera only): 0/5 successful stacking episodes across the base checkpoint and an extensive stability/robustness sweep (actuator damping, action-chunking horizon, joint velocity limits, action smoothing, hand-closing multiplier -- 23 configurations tested in total). Reach behavior is consistently plausible (the hand moves toward the correct object), but the grasp itself is not reliably executed: depending on configuration the hand stays too open, over-closes into a fist, or knocks the object away on contact. See the Masterprojekt repo's eval_results/groot_cubestack/ for full videos, per-config settings, and results JSON.

Status

Not yet suitable for real-robot deployment. Provided as-is for research/reproduction purposes and as a reference point for further grasp-execution debugging (see the Masterprojekt repo for the full investigation log).