GR00T N1.7 post-trained on real FR3 + LEAP whiteboard wiping β€” from the base checkpoint

GR00T N1.7 fully fine-tuned on real FR3 + LEAP hand demonstrations. It starts from nvidia/GR00T-N1.7-3B. The same recipe from the CEI mid-training is Geoffrey19/cei2-groot-n17-fr3v2-leaphand-112h-wipe200.

License. Derived from GR00T N1.7 and distributed under the NVIDIA License in LICENSE, which limits use to non-commercial research and evaluation.

Data

Real teleoperated demonstrations on a Franka FR3 arm (7 joints) with a LEAP hand (16 joints): zwx/wipe_whiteboard_franka_leap_15hz_absolute_200, task "Pick up the eraser and wipe the whiteboard clean.".

  • 199 episodes, 99,133 frames at 15 Hz. The release lists 200 episodes; the data file of episode 36 was missing from the copy this was trained on, so the remaining 199 were renumbered 0–198.
  • One exterior RGB camera, 640Γ—480.
  • State and action: 23 absolute joint positions in radians. action[t] is the measured joint position 1/15 s later (resampled), not the command sent to the robot.
  • All 199 episodes were used for training. There is no held-out split, so no evaluation is reported.

Training

embodiment NEW_EMBODIMENT; state/action fields arm[0:7], hand[7:23]; video exterior_1; annotation annotation.human.task_description
actions 16-step chunks of state-relative joint targets for arm and hand (NON_EEF); the policy adds the state back and outputs absolute positions
normalisation percentiles, q01/q99 mapped to βˆ’1/+1 and clipped; relative-action statistics computed per chunk step from this dataset (statistics.json)
tuning full fine-tune: --tune-llm --tune-visual, projector and action head
steps / batch 20,000 / 32, one GPU
learning rate 2e-5 peak, 2.5 % warmup, cosine decay; weight decay 1e-5
precision bfloat16 mixed precision over float32 weights
weights float32, 3.14 B parameters (1,030 tensors)

Training loss of this run and the three related runs:

steps pi05 from base pi05 from CEI GR00T from base GR00T from CEI
0–100 0.1284 0.1352 1.0587 0.1817
4,900–5,000 0.0150 0.0130 0.0596 0.0344
9,900–10,000 0.0113 0.0102 0.0439 0.0237
14,900–15,000 0.0084 0.0076 0.0270 0.0146
19,000–20,000 0.0066 0.0060 0.0236 0.0129

Mean training loss (each model's own flow-matching objective). The values compare runs of the same model only, and they are fits to the training episodes, not measures of policy quality.

Joint order

The recordings list the LEAP joints in LEAP motor order (per finger: mcp_side, mcp_forward, pip, dip). The CEI fr3v2_leaphand robot model, and so every CEI render and checkpoint, puts each finger's mcp_forward before its mcp_side. The model therefore works in this order:

model dims joints
0–6 franka_joint1 … franka_joint7
7–10 index: mcp_forward, mcp_side, pip, dip
11–14 middle: mcp_forward, mcp_side, pip, dip
15–18 ring: mcp_forward, mcp_side, pip, dip
19–22 thumb: cmc_side, cmc_forward, mcp, ip

As an index map, model[i] = robot[P[i]] with P = [0, 1, 2, 3, 4, 5, 6, 8, 7, 9, 10, 12, 11, 13, 14, 16, 15, 17, 18, 19, 20, 21, 22]. The same P converts back. With this order all 23 joint limits in the recording's robot configuration equal the robot model's exactly.

This model works in the model order. A client must apply P to the robot's state before sending it, and apply P again to the returned actions. joint_order.json lists both orders. The GR00T-format training data were written in the model order by scripts/lerobot_images_to_groot.py --joint_order … in the CEI-2.0 repository.

Files

model-0000{1,2,3}-of-00003.safetensors   step 20,000 (final)
model.safetensors.index.json
config.json  processor_config.json  embodiment_id.json
statistics.json                        the normalisation the model was trained with
experiment_cfg/                        resolved training and processor config
trainer_state.json                     training log
cei2_groot_config.py                   the modality config used for training (--modality-config-path)
dataset_modality.json                  the training data's meta/modality.json: arm = dims 0-6, hand = dims 7-22
joint_order.json                       robot order, model order and the swap between them
LICENSE                                NVIDIA License (applies to this model)

cei2_groot_config.py is the file the training loaded (/opt/Isaac-GR00T/cei2_groot_config.py in the CEI-2.0 GR00T image, docker/groot_cei2_configs.py in the repository). Its comment mentions UR7e + XHand because it was written for that robot; it defines only the field names (arm, hand), the camera, the 16-step relative actions and the prompt key, and the joint split comes from dataset_modality.json. The resolved result is in experiment_cfg/conf.yaml, and processor_config.json contains it too.

The optimiser, scheduler and RNG state are not included. backbone.model.lm_head.weight, the language model's output head, is not stored; the action model does not use it.

Usage

Needs Isaac-GR00T and the modality config for this embodiment, cei2_groot_config.py in this repo (pass it as --modality-config-path to fine-tune further). To score the model offline on a GR00T-format copy of the data:

python gr00t/eval/open_loop_eval.py \
    --dataset-path <GR00T-format dataset, joints in the model order> \
    --embodiment-tag NEW_EMBODIMENT \
    --model-path <local copy of this repo> \
    --traj-ids 0 1 2 --execution-horizon 16

config.json records model_name as the backbone path used during training. Point it at a local copy of nvidia/Cosmos-Reason2-2B; GR00T picks the backbone class by checking whether the name contains nvidia/Cosmos-Reason2, so the directory must be named accordingly.

Limitations

  • No held-out evaluation and no real-robot results. Every number here is training loss.
  • One task, one scene, one camera.
  • Timing is aligned on the host clock, not by a hardware trigger (see the dataset's alignment/ files).
  • Per-embodiment layers start untrained. The base checkpoint has no embodiment mapped to the NEW_EMBODIMENT slot, so its state encoder and action decoder for this robot start from scratch. The CEI-initialised run starts with them trained on the rendered FR3 + LEAP data. This explains much of the difference between the two runs in the first few thousand steps.
Downloads last month
9
Safetensors
Model size
3B params
Tensor type
F32
Β·
Video Preview
loading

Model tree for Geoffrey19/cei2-groot-n17-base-wipe200

Finetuned
(330)
this model