jren313/starvla-pipette-5task-v2

GR00T-N1.7 · G1 pipette, ONE policy for all five phases (five-step v2)

A GR00T-N1.7-3B policy fine-tuned with starVLA on ALL FIVE phases of a bench pipetting workflow on a Unitree G1 humanoid with Inspire hands. The task sentence tells it which phase it is in:

phase	sentence
1 pipette pick	Pick up the pipette from the holder on the right with the right hand.
2 tube pick	Pick up the tube from the blue rack on the left with the left hand.
3 pipette aim	Aim the pipette held in the right hand at the tube held in the left hand.
4 tube return	Put the tube held in the left hand back into the blue rack on the left.
5 pipette return	Put the pipette held in the right hand back into the holder on the right.

It is trained exactly like the per-phase GR00T checkpoints (per-phase recipe v2), on the union of their five training splits: the one change is one model for five phases instead of five models. Status: research checkpoint, not yet run on the robot.

Files

path	what
gr00t/checkpoints/steps_9000_pytorch_model.pt	the SERVED checkpoint
gr00t/config.yaml, gr00t/config.full.yaml	the run's resolved config
gr00t/dataset_statistics.json	normalisation statistics — serve with exactly this file
gr00t/starvla_gr00t_n1d7_pipette_2view_5task_wrist12rel_lthumbrel_joint14_sd08.yaml	training yaml
gr00t/run_starvla_GR00T_N1d7_pipette_train.sh	launcher
gr00t/train.log, gr00t/summary.jsonl	training log with every holdout evaluation

Recipe

• Model: starVLA CosmosGR00TN1d7 from nvidia/GR00T-N1.7-3B (all 1,031 tensors load 1:1; Cosmos-Reason2-2B, select_layer 16, 32-layer alternate-VL DiT flow-matching head, 4 inference steps, embodiment slot 25), full fine-tune.
• Observation: head RGB + eye-in-hand wrist_left, 256 px; the phase's sentence; 59-D state (29 joints, measured and COMMANDED wrist pose6 per side, commanded left thumb_bend and right hand) with state dropout 0.8.
• Action (32-D, 30 steps = 0.5 s at 60 Hz): both wrists chunk-relative to the commanded pose (pose6), LEFT thumb_bend relative, right hand (5 registers) absolute, 14 absolute arm joints that only warm-start the bridge IK (DataConfig unitree_g1_pipette_2view_wrist12rel_lthumbrel_joint14).
• Data: the five-step capture's recording-level split, all phases: 202 train / 35 test episodes (= the union of the five per-phase splits), 137,741 training anchors after the 1 mm pause filter; statistics over the 202 training episodes.
• Optimisation: paged AdamW 8-bit (0.9, 0.95); LR 2e-5 backbone / VL interface, 2e-4 action head; cosine to 1e-6 after 100 warmup steps; clip 1.0; batch 128 (16 × 8 B200); 10 epochs = 10,761 steps (every phase sees as many samples as in its own run); seed 42.
• Selection: holdout MSE on a fixed 960-sample slice of all 35 test episodes every 900 steps (lowest 0.0361 at step 6300, unsaved; flat within noise from step 3600), then each saved checkpoint scored per phase on the per-phase runs' OWN 480 held-out samples in physical units.

Holdout curve (mse / ratio vs hold-still; the per-phase runs ended at 1.22-1.28 because the 14 joint dims' hold baseline replays the commanded joints -- this run is below 1 from step 1800): 900 0.0546/1.23, 1800 0.0432/0.98, 2700 0.0397/0.90, 3600 0.0364/0.82, 4500 0.0391/0.88, 5400 0.0374/0.84, 6300 0.0361/0.82, 7200 0.0378/0.85, 8100 0.0367/0.83, 9000 0.0372/0.84, 9900 0.0382/0.86.

Per phase, against the per-phase checkpoints (same 480 samples each, true labels)

Acting wrist position / rotation RMSE over the 30-step chunk; hand channel RMSE in registers; gain = median projection of the predicted chunk-end displacement on the true one (1 = right amount); stall = share of truly moving chunks (> 10 mm) predicted to cover < 30% of it.

phase	model	wrist mm	wrist deg	hand reg	gain	stall %
1 pipette pick	one policy (steps_9000)	4.34	1.17	206.4	0.76	7.9
1 pipette pick	per-phase GR00T (pick R3, steps_2000)	4.50	1.22	195.6	0.73	8.6
1 pipette pick	round 10 five-step GR00T (18-D, both hands rel)	4.26	1.17	196.2	0.62	17.2
1 pipette pick	hold still	6.40	1.74	201.5	0	100
2 tube pick	one policy (steps_9000)	3.77	1.26	33.1	0.96	3.5
2 tube pick	per-phase GR00T (tube pick R2, steps_1000)	4.03	1.31	31.7	0.95	3.9
2 tube pick	round 10 five-step GR00T (18-D, both hands rel)	3.87	1.29	35.9	0.90	7.9
2 tube pick	hold still	6.68	2.07	51.0	0	100
3 pipette aim	one policy (steps_9000)	3.39	1.11	–	0.82	14.1
3 pipette aim	per-phase GR00T (aim R1, steps_1350)	3.70	1.20	–	0.79	16.1
3 pipette aim	round 10 five-step GR00T (18-D, both hands rel)	3.56	1.21	–	0.70	13.8
3 pipette aim	hold still	4.60	1.58	–	0	100
4 tube return	one policy (steps_9000)	4.49	1.63	35.7	0.80	8.2
4 tube return	per-phase GR00T (tube return R1, steps_500)	4.70	1.62	41.5	0.84	4.3
4 tube return	round 10 five-step GR00T (18-D, both hands rel)	5.00	1.77	41.3	0.76	9.6
4 tube return	hold still	7.55	2.34	55.2	0	100
5 pipette return	one policy (steps_9000)	4.15	1.27	106.1	0.82	1.5
5 pipette return	per-phase GR00T (return R1, steps_3000)	4.39	1.32	124.6	0.87	0.9
5 pipette return	round 10 five-step GR00T (18-D, both hands rel)	3.82	1.25	109.2	0.77	0.9
5 pipette return	hold still	6.08	1.77	118.6	0	100

The sentence picks the phase

Each phase's 480 held-out samples scored with every phase's sentence (acting-wrist RMSE mm, gain in brackets; diagonal = the right sentence). The right sentence gives the highest gain in every row; the REVERSE sentence of the same object (pick <-> return, tube pick <-> tube return) drives the wrist the opposite way (negative gain), worse than holding still. A chained run must switch the sentence at each phase boundary.

frames / sentence	pick	tube pick	aim	tube return	return	hold still
pick	4.34 (+0.76)	4.23 (+0.55)	5.52 (+0.09)	8.07 (-0.24)	9.88 (-0.54)	6.40
tube pick	4.76 (+0.37)	3.77 (+0.96)	6.90 (+0.08)	13.16 (-0.44)	10.13 (-0.13)	6.68
aim	3.49 (+0.49)	5.11 (+0.34)	3.39 (+0.82)	4.96 (+0.21)	3.98 (+0.58)	4.60
tube return	8.79 (-0.06)	12.43 (-0.51)	7.37 (+0.05)	4.49 (+0.80)	5.45 (+0.39)	7.55
return	11.60 (-0.65)	9.31 (-0.34)	5.91 (+0.21)	4.17 (+0.57)	4.15 (+0.82)	6.08

Serving

Through VLAPolicyBridge, one lane per phase (pipette_5taskv2_*_gr00t_starvla), each the per-phase GR00T lane with only the checkpoint swapped: same sentence, staging, held hand channels and envelope. One policy server serves all five lanes.