File size: 2,928 Bytes
4072761 eb759e1 4072761 eb759e1 4072761 eb759e1 4072761 dd4f491 4072761 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 | # RoboCOIN RIFT inference
Existing final RIFT checkpoint from 10 epochs / 9,340 optimizer steps on object
classification (191 episodes), red/yellow test-tube mixing (50), and basket stacking
(101). Original source provenance is in [checkpoint_manifest.json](../checkpoint_manifest.json).
```bash
hf download PoopBear/RIFT --include 'robocoin_multitask_mm/rift_10ep/*' 'assets/*' --local-dir ./checkpoints
export DIFFSYNTH_MODEL_BASE_PATH=./checkpoints/assets
```
Load `./checkpoints/robocoin_multitask_mm/rift_10ep/rift_step009340.pt` with the adjacent
`config.yaml` and `dataset_stats.json`. The checkpoint includes anticipation/probe
parameters; preserve all model state and the packaged processor settings. For full
RIFT checkpoint loading, skip separate pretrained video/ActionDiT initialization
(`skip_dit_load_from_pretrain=true`, `action_dit_pretrained_path=null`) and enable
`load_text_encoder=true` for uncached prompts. Use `weights_only=True` for the policy
checkpoint. No raw dataset or training-view archive is needed for live inference.
## Observation and action contract
- Policy action and proprioceptive state order is
`[left_arm (6), left_gripper (1), right_arm (6), right_gripper (1)]` (14 values).
The raw dataset's packed order is `[left_arm (6), right_arm (6), left_gripper (1), right_gripper (1)]`;
reorder by named fields before policy preprocessing.
- Arm joints remain in radians. When starting from the original RoboCOIN source
representations, multiply
classification and test-tube gripper values by `180/pi` to obtain the policy's
millimeter convention; basket gripper values remain unchanged. Apply this to both
actions and states, without clipping. Already-converted millimeter values must
not be converted again. This conversion follows the original
preprocessing's inverse degree-to-radian interpretation; per-subset export logs
were not available. For a live robot, map its measured gripper units to the same
policy convention, then invert that mapping when sending commands.
- Cameras, in order: `head_rgb`, `left_wrist_rgb`, `right_wrist_rgb`. Dataset aliases
are `cam_head_left_rgb` for classification/test-tube heads and `cam_high_left_rgb`
for basket heads; wrists are `cam_left_wrist_rgb` and `cam_right_wrist_rgb`.
- Preserve the packaged 224 x 224 per-camera resizing, three-camera composition,
relative-joint transform for both arms, stepwise action normalization, and gripper
action normalization `0/100`. Use this checkpoint's statistics for normalization
and inverse action processing; do not substitute another dataset's statistics.
The saved config retains loader-compatible fields and original architecture settings.
Its normalization path points to the packaged statistics under `./checkpoints/`;
relative paths resolve from the working directory. Dataset/cache/output fields are
retained metadata rather than inference setup instructions.
|