dita_object / GRIPPER_MODES.md

albus2024

Update Dita training configuration and bundled runtime to 2.1.0

3be9ef3 verified about 1 month ago

5.94 kB

# DITA gripper modes

Runtime **2.1.0** supports `policy.gripper_mode=binary` and `continuous`. The mode is
saved in `config.json` and both processor JSON files. The public LeRobot processor
factory applies the active policy mode when loading pretrained templates, including
CLI overrides. Conflicting explicit processor modes fail. New fine-tuning binds the
target dataset statistics; resume keeps the saved statistics.

The config schema replaces the serialized `action_normalization_mask` and
`gripper_sign_flip` fields with `gripper_mode`. Old policy and processor configs are
not supported. Use the updated Hub revisions and their 2.1.0 wheel together.

| Contract | `binary` | `continuous` |
| --- | --- | --- |
| Target dataset | LIBERO signed commands | SO101 EEF gripper positions |
| Training gripper | `-1=open, +1=closed` becomes `1=open, 0=closed` | Dataset `action.min/max` maps position to `[-1, 1]` |
| Model output | Continuous diffusion prediction | Continuous diffusion prediction |
| Policy output | `prediction > 0.5` becomes `+1`, otherwise `-1` | No threshold |
| Postprocessor output | Sign flip restores `-1=open, +1=closed` | Inverse min/max mapping restores recorded position units |
| Hub defaults | Spatial, Object, Goal, Long | DROID fine-tuning initialization |

Both modes use target-dataset `action.q01/q99` for the first six pose dimensions.
Continuous gripper additionally requires seven-element finite `action.min/max`.
Gripper outputs are bounded to the recorded min/max; a constant gripper range maps
to zero during training and restores the constant position during inference.

The SO101 merged dataset has gripper min/max approximately `0.377/47.472` while its
q01/q99 are approximately `1.227/28.432`. Using the full gripper min/max retains
rare fully open positions that would exceed the diffusion sampler's normalized
range with quantile scaling. Dataset extrema describe recorded commands, not new
hardware calibration limits. Use the same robot calibration for recording and rollout.

Binary mode accepts raw LIBERO signed labels. Already transformed RLDS `0/1`
labels must first be converted to LIBERO commands (`1 - 2 * openness`) before using
this processor. This restores the original DITA learning convention while keeping
all converted LIBERO inference outputs unchanged.

## Fine-tune

Install the runtime bundled with the updated initialization:

```bash
pip install 'lerobot_policy_dita[training] @ https://huggingface.co/albus2024/dita_droid_pretrain/resolve/main/lerobot_policy_dita-2.1.0-py3-none-any.whl'
```

LIBERO:

```bash
lerobot-train \
  --policy.path=albus2024/dita_droid_pretrain \
  --policy.gripper_mode=binary \
  --policy.device=cuda --policy.dtype=bfloat16 \
  --policy.input_features=null --policy.output_features=null \
  --policy.push_to_hub=false \
  --dataset.repo_id=HuggingFaceVLA/libero \
  --output_dir=outputs/train/dita_libero_binary
```

SO101 EEF, using the local merged dataset and front camera:

```bash
lerobot-train \
  --policy.path=albus2024/dita_droid_pretrain \
  --policy.gripper_mode=continuous \
  --policy.device=cuda --policy.dtype=bfloat16 \
  --policy.input_features='{"observation.images.image":{"type":"VISUAL","shape":[3,480,640]}}' \
  --policy.output_features=null \
  --rename_map='{"observation.images.front":"observation.images.image"}' \
  --policy.push_to_hub=false \
  --dataset.repo_id=albus2024/so101_merged \
  --dataset.root=/home/lyanam/project/lerobot_real_robot/datasets/so101_merged \
  --dataset.video_backend=pyav \
  --output_dir=outputs/train/dita_so101_continuous
```

Keep `traj_length=11`, `chunk_size=10`, `n_action_steps=1`, and the existing camera
and temporal contract. The DROID artifact is initialization without target-dataset
statistics. Fine-tune with the chosen mode; changing modes after training does not
convert learned gripper targets or align EEF coordinate frames.

## EEF deployment

Load the fine-tuned policy **and its saved processors** in `lerobot-rollout-eef`.
After the policy postprocessor, the seven action channels must be ordered as:

```text
ee.x, ee.y, ee.z, ee.wx, ee.wy, ee.wz, ee.gripper_pos
```

`ee.gripper_pos` is already a continuous value in the recorded servo units. The
EEF IK processor forwards it to `gripper.pos`; no additional threshold, sign flip,
or multiplication by 100 is needed. SO101 data uses absolute position and rotvec
orientation; preserve that action interpretation, robot calibration, and camera mapping.

## Reproduce the real-data mechanics checks

From the repository root, with an updated local DROID artifact:

```bash
CUDA_VISIBLE_DEVICES=0 python scripts/validate_gripper_modes.py \
  --checkpoint /path/to/updated/dita_droid_pretrain \
  --dataset-root /path/to/libero \
  --dataset-repo-id HuggingFaceVLA/libero \
  --mode binary --output workspace/validation/gripper_libero
CUDA_VISIBLE_DEVICES=1 python scripts/validate_gripper_modes.py \
  --checkpoint /path/to/updated/dita_droid_pretrain \
  --dataset-root /home/lyanam/project/lerobot_real_robot/datasets/so101_merged \
  --dataset-repo-id albus2024/so101_merged \
  --camera-key observation.images.front \
  --mode continuous --output workspace/validation/gripper_so101
```

The script invokes native `lerobot.scripts.lerobot_train` for ten optimizer steps,
resumes for one more step, and checks finite weights/optimizer state, changed weights,
saved statistics, real action encode/decode, `[4, 10, 7]` chunk inference, and
`[4, 7]` online single-frame inference. It records exact commands, logs, and
`verification.json`. `--verify-only` rechecks existing checkpoints without training.

The 2026-09-14 checks passed on three episodes of each real dataset with BF16 and
batch size four. They validate training and inference mechanics, not convergence,
LIBERO task success, or a physical SO101 rollout.

See the [validation results](GRIPPER_VALIDATION.md) for the checked datasets and scope.

	# DITA gripper modes

	Runtime 2.1.0 supports `policy.gripper_mode=binary` and `continuous`. The mode is
	saved in `config.json` and both processor JSON files. The public LeRobot processor
	factory applies the active policy mode when loading pretrained templates, including
	CLI overrides. Conflicting explicit processor modes fail. New fine-tuning binds the
	target dataset statistics; resume keeps the saved statistics.

	The config schema replaces the serialized `action_normalization_mask` and
	`gripper_sign_flip` fields with `gripper_mode`. Old policy and processor configs are
	not supported. Use the updated Hub revisions and their 2.1.0 wheel together.

	\| Contract \| `binary` \| `continuous` \|
	\| --- \| --- \| --- \|
	\| Target dataset \| LIBERO signed commands \| SO101 EEF gripper positions \|
	\| Training gripper \| `-1=open, +1=closed` becomes `1=open, 0=closed` \| Dataset `action.min/max` maps position to `[-1, 1]` \|
	\| Model output \| Continuous diffusion prediction \| Continuous diffusion prediction \|
	\| Policy output \| `prediction > 0.5` becomes `+1`, otherwise `-1` \| No threshold \|
	\| Postprocessor output \| Sign flip restores `-1=open, +1=closed` \| Inverse min/max mapping restores recorded position units \|
	\| Hub defaults \| Spatial, Object, Goal, Long \| DROID fine-tuning initialization \|

	Both modes use target-dataset `action.q01/q99` for the first six pose dimensions.
	Continuous gripper additionally requires seven-element finite `action.min/max`.
	Gripper outputs are bounded to the recorded min/max; a constant gripper range maps
	to zero during training and restores the constant position during inference.

	The SO101 merged dataset has gripper min/max approximately `0.377/47.472` while its
	q01/q99 are approximately `1.227/28.432`. Using the full gripper min/max retains
	rare fully open positions that would exceed the diffusion sampler's normalized
	range with quantile scaling. Dataset extrema describe recorded commands, not new
	hardware calibration limits. Use the same robot calibration for recording and rollout.

	Binary mode accepts raw LIBERO signed labels. Already transformed RLDS `0/1`
	labels must first be converted to LIBERO commands (`1 - 2 * openness`) before using
	this processor. This restores the original DITA learning convention while keeping
	all converted LIBERO inference outputs unchanged.

	## Fine-tune

	Install the runtime bundled with the updated initialization:

	```bash
	pip install 'lerobot_policy_dita[training] @ https://huggingface.co/albus2024/dita_droid_pretrain/resolve/main/lerobot_policy_dita-2.1.0-py3-none-any.whl'
	```

	LIBERO:

	```bash
	lerobot-train \
	--policy.path=albus2024/dita_droid_pretrain \
	--policy.gripper_mode=binary \
	--policy.device=cuda --policy.dtype=bfloat16 \
	--policy.input_features=null --policy.output_features=null \
	--policy.push_to_hub=false \
	--dataset.repo_id=HuggingFaceVLA/libero \
	--output_dir=outputs/train/dita_libero_binary
	```

	SO101 EEF, using the local merged dataset and front camera:

	```bash
	lerobot-train \
	--policy.path=albus2024/dita_droid_pretrain \
	--policy.gripper_mode=continuous \
	--policy.device=cuda --policy.dtype=bfloat16 \
	--policy.input_features='{"observation.images.image":{"type":"VISUAL","shape":[3,480,640]}}' \
	--policy.output_features=null \
	--rename_map='{"observation.images.front":"observation.images.image"}' \
	--policy.push_to_hub=false \
	--dataset.repo_id=albus2024/so101_merged \
	--dataset.root=/home/lyanam/project/lerobot_real_robot/datasets/so101_merged \
	--dataset.video_backend=pyav \
	--output_dir=outputs/train/dita_so101_continuous
	```

	Keep `traj_length=11`, `chunk_size=10`, `n_action_steps=1`, and the existing camera
	and temporal contract. The DROID artifact is initialization without target-dataset
	statistics. Fine-tune with the chosen mode; changing modes after training does not
	convert learned gripper targets or align EEF coordinate frames.

	## EEF deployment

	Load the fine-tuned policy and its saved processors in `lerobot-rollout-eef`.
	After the policy postprocessor, the seven action channels must be ordered as:

	```text
	ee.x, ee.y, ee.z, ee.wx, ee.wy, ee.wz, ee.gripper_pos
	```

	`ee.gripper_pos` is already a continuous value in the recorded servo units. The
	EEF IK processor forwards it to `gripper.pos`; no additional threshold, sign flip,
	or multiplication by 100 is needed. SO101 data uses absolute position and rotvec
	orientation; preserve that action interpretation, robot calibration, and camera mapping.

	## Reproduce the real-data mechanics checks

	From the repository root, with an updated local DROID artifact:

	```bash
	CUDA_VISIBLE_DEVICES=0 python scripts/validate_gripper_modes.py \
	--checkpoint /path/to/updated/dita_droid_pretrain \
	--dataset-root /path/to/libero \
	--dataset-repo-id HuggingFaceVLA/libero \
	--mode binary --output workspace/validation/gripper_libero

	CUDA_VISIBLE_DEVICES=1 python scripts/validate_gripper_modes.py \
	--checkpoint /path/to/updated/dita_droid_pretrain \
	--dataset-root /home/lyanam/project/lerobot_real_robot/datasets/so101_merged \
	--dataset-repo-id albus2024/so101_merged \
	--camera-key observation.images.front \
	--mode continuous --output workspace/validation/gripper_so101
	```

	The script invokes native `lerobot.scripts.lerobot_train` for ten optimizer steps,
	resumes for one more step, and checks finite weights/optimizer state, changed weights,
	saved statistics, real action encode/decode, `[4, 10, 7]` chunk inference, and
	`[4, 7]` online single-frame inference. It records exact commands, logs, and
	`verification.json`. `--verify-only` rechecks existing checkpoints without training.

	The 2026-09-14 checks passed on three episodes of each real dataset with BF16 and
	batch size four. They validate training and inference mechanics, not convergence,
	LIBERO task success, or a physical SO101 rollout.

	See the [validation results](GRIPPER_VALIDATION.md) for the checked datasets and scope.