Instructions to use Geoffrey19/cei2-groot-n17-base-wipe200 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- LeRobot
How to use Geoffrey19/cei2-groot-n17-base-wipe200 with LeRobot:
# No code snippets available yet for this library. # To use this model, check the repository files and the library's documentation. # Want to help? PRs adding snippets are welcome at: # https://github.com/huggingface/huggingface.js
- Notebooks
- Google Colab
- Kaggle
GR00T N1.7 post-trained on real FR3 + LEAP whiteboard wiping β from the base checkpoint
GR00T N1.7 fully fine-tuned on real FR3 + LEAP hand demonstrations.
It starts from nvidia/GR00T-N1.7-3B.
The same recipe from the CEI mid-training is Geoffrey19/cei2-groot-n17-fr3v2-leaphand-112h-wipe200.
License. Derived from GR00T N1.7 and distributed under the NVIDIA License in LICENSE, which limits use
to non-commercial research and evaluation.
Data
Real teleoperated demonstrations on a Franka FR3 arm (7 joints) with a LEAP hand (16 joints):
zwx/wipe_whiteboard_franka_leap_15hz_absolute_200, task "Pick up the eraser and wipe the whiteboard clean.".
- 199 episodes, 99,133 frames at 15 Hz. The release lists 200 episodes; the data file of episode 36 was missing from the copy this was trained on, so the remaining 199 were renumbered 0β198.
- One exterior RGB camera, 640Γ480.
- State and action: 23 absolute joint positions in radians.
action[t]is the measured joint position 1/15 s later (resampled), not the command sent to the robot. - All 199 episodes were used for training. There is no held-out split, so no evaluation is reported.
Training
| embodiment | NEW_EMBODIMENT; state/action fields arm[0:7], hand[7:23]; video exterior_1; annotation annotation.human.task_description |
| actions | 16-step chunks of state-relative joint targets for arm and hand (NON_EEF); the policy adds the state back and outputs absolute positions |
| normalisation | percentiles, q01/q99 mapped to β1/+1 and clipped; relative-action statistics computed per chunk step from this dataset (statistics.json) |
| tuning | full fine-tune: --tune-llm --tune-visual, projector and action head |
| steps / batch | 20,000 / 32, one GPU |
| learning rate | 2e-5 peak, 2.5 % warmup, cosine decay; weight decay 1e-5 |
| precision | bfloat16 mixed precision over float32 weights |
| weights | float32, 3.14 B parameters (1,030 tensors) |
Training loss of this run and the three related runs:
| steps | pi05 from base | pi05 from CEI | GR00T from base | GR00T from CEI |
|---|---|---|---|---|
| 0β100 | 0.1284 | 0.1352 | 1.0587 | 0.1817 |
| 4,900β5,000 | 0.0150 | 0.0130 | 0.0596 | 0.0344 |
| 9,900β10,000 | 0.0113 | 0.0102 | 0.0439 | 0.0237 |
| 14,900β15,000 | 0.0084 | 0.0076 | 0.0270 | 0.0146 |
| 19,000β20,000 | 0.0066 | 0.0060 | 0.0236 | 0.0129 |
Mean training loss (each model's own flow-matching objective). The values compare runs of the same model only, and they are fits to the training episodes, not measures of policy quality.
Joint order
The recordings list the LEAP joints in LEAP motor order (per finger: mcp_side, mcp_forward, pip, dip).
The CEI fr3v2_leaphand robot model, and so every CEI render and checkpoint, puts each finger's
mcp_forward before its mcp_side. The model therefore works in this order:
| model dims | joints |
|---|---|
| 0β6 | franka_joint1 β¦ franka_joint7 |
| 7β10 | index: mcp_forward, mcp_side, pip, dip |
| 11β14 | middle: mcp_forward, mcp_side, pip, dip |
| 15β18 | ring: mcp_forward, mcp_side, pip, dip |
| 19β22 | thumb: cmc_side, cmc_forward, mcp, ip |
As an index map, model[i] = robot[P[i]] with
P = [0, 1, 2, 3, 4, 5, 6, 8, 7, 9, 10, 12, 11, 13, 14, 16, 15, 17, 18, 19, 20, 21, 22].
The same P converts back. With this order all 23 joint limits in the recording's robot
configuration equal the robot model's exactly.
This model works in the model order. A client must apply P to the robot's state before sending it,
and apply P again to the returned actions. joint_order.json lists both orders. The GR00T-format training data were written in the model
order by scripts/lerobot_images_to_groot.py --joint_order β¦ in the CEI-2.0 repository.
Files
model-0000{1,2,3}-of-00003.safetensors step 20,000 (final)
model.safetensors.index.json
config.json processor_config.json embodiment_id.json
statistics.json the normalisation the model was trained with
experiment_cfg/ resolved training and processor config
trainer_state.json training log
cei2_groot_config.py the modality config used for training (--modality-config-path)
dataset_modality.json the training data's meta/modality.json: arm = dims 0-6, hand = dims 7-22
joint_order.json robot order, model order and the swap between them
LICENSE NVIDIA License (applies to this model)
cei2_groot_config.py is the file the training loaded (/opt/Isaac-GR00T/cei2_groot_config.py in the
CEI-2.0 GR00T image, docker/groot_cei2_configs.py in the repository). Its comment mentions UR7e + XHand
because it was written for that robot; it defines only the field names (arm, hand), the camera, the
16-step relative actions and the prompt key, and the joint split comes from dataset_modality.json. The
resolved result is in experiment_cfg/conf.yaml, and processor_config.json contains it too.
The optimiser, scheduler and RNG state are not included. backbone.model.lm_head.weight, the language
model's output head, is not stored; the action model does not use it.
Usage
Needs Isaac-GR00T and the modality config for this embodiment, cei2_groot_config.py in this repo (pass it as
--modality-config-path to fine-tune further). To score the model offline on a GR00T-format copy of the data:
python gr00t/eval/open_loop_eval.py \
--dataset-path <GR00T-format dataset, joints in the model order> \
--embodiment-tag NEW_EMBODIMENT \
--model-path <local copy of this repo> \
--traj-ids 0 1 2 --execution-horizon 16
config.json records model_name as the backbone path used during training. Point it at a local copy of
nvidia/Cosmos-Reason2-2B; GR00T picks the backbone class by checking whether the name contains
nvidia/Cosmos-Reason2, so the directory must be named accordingly.
Limitations
- No held-out evaluation and no real-robot results. Every number here is training loss.
- One task, one scene, one camera.
- Timing is aligned on the host clock, not by a hardware trigger (see the dataset's
alignment/files). - Per-embodiment layers start untrained. The base checkpoint has no embodiment mapped to the
NEW_EMBODIMENTslot, so its state encoder and action decoder for this robot start from scratch. The CEI-initialised run starts with them trained on the rendered FR3 + LEAP data. This explains much of the difference between the two runs in the first few thousand steps.
- Downloads last month
- 9
Model tree for Geoffrey19/cei2-groot-n17-base-wipe200
Base model
nvidia/GR00T-N1.7-3B