Organize RoboCOIN and Galaxea checkpoints with matching configs and stats

Add both verified RoboCOIN RIFT variants from research-vla/RIFT-RoboCOIN. Group the three existing Galaxea checkpoints by dataset and include the matching saved training configurations, normalization files, and SHA-256 checksums. Add source provenance and download instructions while preserving existing download URLs.

README.md

@@ -1,5 +1,5 @@
 ---
-pretty_name: RIFT Checkpoint
+pretty_name: RIFT Checkpoints
 license: apache-2.0
 library_name: pytorch
 base_model: Wan-AI/Wan2.2-TI2V-5B
@@ -11,56 +11,61 @@ tags:
   - world-action-model
   - flow-matching
   - libero
+  - robocoin
+  - galaxea
   - arxiv:2608.11521
 ---
 
-# RIFT Checkpoint
+# RIFT Checkpoints
 
-Official checkpoint for
-**[Keep the Future, Drop the Rollout: RIFT for World Action Models](https://arxiv.org/abs/2608.11521)**.
+Checkpoints for **[Keep the Future, Drop the Rollout: RIFT for World Action Models](https://arxiv.org/abs/2608.11521)**.
+[Code and evaluation](https://github.com/ChushanZhang/RIFT).
 
-## Project
+## Checkpoints
 
-- Paper: https://arxiv.org/abs/2608.11521
-- Code: https://github.com/ChushanZhang/RIFT
-- Dataset: https://huggingface.co/datasets/yuanty/LIBERO-fastwam
+| Dataset | Directory | Models |
+| --- | --- | --- |
+| LIBERO | Repository root | RIFT, step 21,700 |
+| RoboCOIN, three tasks | [robocoin_multitask_mm](https://huggingface.co/PoopBear/RIFT/tree/main/robocoin_multitask_mm) | RIFT fused and separate, 10 epochs / step 9,340 |
+| Galaxea indoor cleaning | [galaxea_indoor_cleaning_3cam224](https://huggingface.co/PoopBear/RIFT/tree/main/galaxea_indoor_cleaning_3cam224) | FastWAM, FastWAM-Joint and RIFT, 10 epochs / step 14,830 |
 
-## Files
+Each real-world model directory contains the actual `.pt` checkpoint, its
+`config.yaml`, matching `dataset_stats.json`, and `SHA256SUMS`. Exact file
+hashes and source revisions are listed in [checkpoint_manifest.json](checkpoint_manifest.json).
+The existing Galaxea download paths under `fastwam/`, `fastwam_joint/`, and `rift/` remain valid.
 
-```text
-rift_step021700.pt
-dataset_stats.json
-config.yaml
-```
+## Download
 
-- `rift_step021700.pt`: released RIFT checkpoint for LIBERO
-- `dataset_stats.json`: normalization statistics used by the checkpoint
-- `config.yaml`: current Hydra model configuration using the `rift.*` namespace
+LIBERO:
 
-Keep `dataset_stats.json` with the checkpoint during evaluation.
+```bash
+hf download PoopBear/RIFT \
+  rift_step021700.pt dataset_stats.json config.yaml \
+  --local-dir ./checkpoints/rift
+```
 
-Install the standalone [RIFT repository](https://github.com/ChushanZhang/RIFT)
-with `pip install -e .`. Compose `config.yaml` with the repository's task and
-data configs; it contains interpolations such as `${data.train...}` and is not
-a complete training configuration by itself.
+RoboCOIN RIFT, fused variant:
 
-`rift_step021700.pt` uses PyTorch serialization. Download it only from this
-repository and load it with `weights_only=True`.
+```bash
+hf download PoopBear/RIFT \
+  --include 'robocoin_multitask_mm/rift_fused_z1_10ep/*' \
+  --local-dir ./checkpoints
+```
 
-## Download
+Galaxea RIFT:
 
 ```bash
-pip install -U huggingface_hub
-
 hf download PoopBear/RIFT \
-  rift_step021700.pt \
-  dataset_stats.json \
-  config.yaml \
-  --local-dir ./checkpoints/rift
+  --include 'galaxea_indoor_cleaning_3cam224/rift_10ep/*' \
+  --local-dir ./checkpoints
 ```
 
-Evaluation commands are provided in the
-[RIFT repository](https://github.com/ChushanZhang/RIFT#benchmark-evaluation).
+Use each checkpoint's own normalization statistics. Real-world directories
+include the full saved training configuration; set local dataset and pretrained
+asset paths for your machine. The LIBERO root `config.yaml` is a model
+configuration with `${data.train...}` interpolations and must be composed with
+the [RIFT repository](https://github.com/ChushanZhang/RIFT)'s task and data configs.
+Load PyTorch checkpoints with `weights_only=True`.
 
 ## Citation
 

checkpoint_manifest.json

@@ -0,0 +1,89 @@
+{
+  "checkpoints": [
+    {
+      "dataset": "robocoin_multitask_mm",
+      "variant": "rift_fused_z1_10ep",
+      "checkpoint": "robocoin_multitask_mm/rift_fused_z1_10ep/rift_step_009340.pt",
+      "bytes": 12061290937,
+      "sha256": "c6ca1db0a7e4f06c0867a2d7e06caf1775308e8ddfc51013cbaf5459ba7dcaad",
+      "step": 9340,
+      "epochs": 10,
+      "config": "robocoin_multitask_mm/rift_fused_z1_10ep/config.yaml",
+      "dataset_stats": "robocoin_multitask_mm/rift_fused_z1_10ep/dataset_stats.json",
+      "model_target": "rift.model_factory.create_rift",
+      "source": {
+        "repo": "research-vla/RIFT-RoboCOIN",
+        "revision": "debae88e14f2c0f32c0e86eafd4037a10e7b7123",
+        "path": "fused_z1_10ep/rift_step_009340.pt"
+      }
+    },
+    {
+      "dataset": "robocoin_multitask_mm",
+      "variant": "rift_separate_z1_10ep",
+      "checkpoint": "robocoin_multitask_mm/rift_separate_z1_10ep/rift_step_009340.pt",
+      "bytes": 12061290937,
+      "sha256": "8d049fdd5bc5928912cb62685eb907234644509114d2167fe63eab00cc74b7a7",
+      "step": 9340,
+      "epochs": 10,
+      "config": "robocoin_multitask_mm/rift_separate_z1_10ep/config.yaml",
+      "dataset_stats": "robocoin_multitask_mm/rift_separate_z1_10ep/dataset_stats.json",
+      "model_target": "rift.model_factory.create_rift",
+      "source": {
+        "repo": "research-vla/RIFT-RoboCOIN",
+        "revision": "debae88e14f2c0f32c0e86eafd4037a10e7b7123",
+        "path": "separate_z1_10ep/rift_step_009340.pt"
+      }
+    },
+    {
+      "dataset": "galaxea_indoor_cleaning_3cam224",
+      "variant": "fastwam_10ep",
+      "checkpoint": "galaxea_indoor_cleaning_3cam224/fastwam_10ep/fastwam_step014830.pt",
+      "bytes": 12041813369,
+      "sha256": "9e324cad3dbb13d623e4aed51b8534226a13079f1d9e3f3fb59f1f67ec0ec19f",
+      "step": 14830,
+      "epochs": 10,
+      "config": "galaxea_indoor_cleaning_3cam224/fastwam_10ep/config.yaml",
+      "dataset_stats": "galaxea_indoor_cleaning_3cam224/fastwam_10ep/dataset_stats.json",
+      "model_target": "fastwam.runtime.create_fastwam",
+      "source": {
+        "repo": "PoopBear/RIFT",
+        "revision": "584b9c80199ab223705f15a75cd02a7d8dfad754",
+        "path": "fastwam/galaxea_indoor_cleaning_3cam224_10ep/fastwam_step014830.pt"
+      }
+    },
+    {
+      "dataset": "galaxea_indoor_cleaning_3cam224",
+      "variant": "fastwam_joint_10ep",
+      "checkpoint": "galaxea_indoor_cleaning_3cam224/fastwam_joint_10ep/fastwam_joint_step014830.pt",
+      "bytes": 12041813369,
+      "sha256": "7616e7861c240360d6fab1ccca75d54b4857734264210c6a1c1a2f017cbcd869",
+      "step": 14830,
+      "epochs": 10,
+      "config": "galaxea_indoor_cleaning_3cam224/fastwam_joint_10ep/config.yaml",
+      "dataset_stats": "galaxea_indoor_cleaning_3cam224/fastwam_joint_10ep/dataset_stats.json",
+      "model_target": "fastwam.runtime.create_fastwam_joint",
+      "source": {
+        "repo": "PoopBear/RIFT",
+        "revision": "584b9c80199ab223705f15a75cd02a7d8dfad754",
+        "path": "fastwam_joint/galaxea_indoor_cleaning_3cam224_10ep/fastwam_joint_step014830.pt"
+      }
+    },
+    {
+      "dataset": "galaxea_indoor_cleaning_3cam224",
+      "variant": "rift_10ep",
+      "checkpoint": "galaxea_indoor_cleaning_3cam224/rift_10ep/rift_step014830.pt",
+      "bytes": 12061290937,
+      "sha256": "052c0713bfd1f04222599deea43f02cebff963e7453ec7f2e53f81863d8d3939",
+      "step": 14830,
+      "epochs": 10,
+      "config": "galaxea_indoor_cleaning_3cam224/rift_10ep/config.yaml",
+      "dataset_stats": "galaxea_indoor_cleaning_3cam224/rift_10ep/dataset_stats.json",
+      "model_target": "rift.model_factory.create_rift",
+      "source": {
+        "repo": "PoopBear/RIFT",
+        "revision": "584b9c80199ab223705f15a75cd02a7d8dfad754",
+        "path": "rift/galaxea_indoor_cleaning_3cam224_10ep/rift_step014830.pt"
+      }
+    }
+  ]
+}

galaxea_indoor_cleaning_3cam224/README.md

@@ -0,0 +1,28 @@
+# Galaxea indoor-cleaning checkpoints
+
+Final weights-only checkpoints after 10 epochs / 14,830 optimizer steps.
+
+| Model | Directory | Checkpoint |
+| --- | --- | --- |
+| FastWAM | [fastwam_10ep](https://huggingface.co/PoopBear/RIFT/tree/main/galaxea_indoor_cleaning_3cam224/fastwam_10ep) | `fastwam_step014830.pt` |
+| FastWAM-Joint | [fastwam_joint_10ep](https://huggingface.co/PoopBear/RIFT/tree/main/galaxea_indoor_cleaning_3cam224/fastwam_joint_10ep) | `fastwam_joint_step014830.pt` |
+| RIFT | [rift_10ep](https://huggingface.co/PoopBear/RIFT/tree/main/galaxea_indoor_cleaning_3cam224/rift_10ep) | `rift_step014830.pt` |
+
+Each directory also includes the corresponding saved `config.yaml`,
+`dataset_stats.json`, and `SHA256SUMS`. Checkpoint hashes were verified against
+the final training-run files. All three use the Galaxea statistics whose SHA-256 is
+`52ab72e1185d0fd5c15b7e29187b9606f7e610f5d7d386e4c59ba217442673b2`.
+
+```bash
+hf download PoopBear/RIFT \
+  --include 'galaxea_indoor_cleaning_3cam224/rift_10ep/*' \
+  --local-dir ./checkpoints
+```
+
+Use `fastwam_10ep` or `fastwam_joint_10ep` in the include pattern for a baseline.
+RIFT uses `rift.model_factory.create_rift`; the baseline configs use the
+`fastwam.runtime` factories. Set local dataset and pretrained-asset paths while
+keeping the model configuration and matching normalization statistics.
+
+The previous checkpoint URLs under `fastwam/`, `fastwam_joint/`, and `rift/`
+remain available and contain identical weights.

galaxea_indoor_cleaning_3cam224/fastwam_10ep/SHA256SUMS

@@ -0,0 +1,3 @@
+9e324cad3dbb13d623e4aed51b8534226a13079f1d9e3f3fb59f1f67ec0ec19f  fastwam_step014830.pt
+a99f015236820aef4da01ab349659c6282c865d930a7acd51022fb3c5e40c7d9  config.yaml
+52ab72e1185d0fd5c15b7e29187b9606f7e610f5d7d386e4c59ba217442673b2  dataset_stats.json

galaxea_indoor_cleaning_3cam224/fastwam_10ep/config.yaml

@@ -0,0 +1,384 @@
+output_dir: /data/chushan/fastwam-runs/galaxea_indoor_cleaning_fastwam_3cam224_1e-4/20260819_100750_bs32_10ep_from_scratch_zero1
+batch_size: 32
+num_workers: 6
+lr_scheduler_type: cosine
+learning_rate: 0.0001
+num_epochs: 10
+max_steps: 14830
+log_every: 10
+save_every: 1483
+eval_every: 0
+eval_num_inference_steps: 10
+gradient_accumulation_steps: 1
+mixed_precision: bf16
+seed: 42
+max_grad_norm: 1.0
+weight_decay: 0.01
+resume: null
+wandb:
+  enabled: false
+  workspace: null
+  project: fast-wam
+  name: galaxea_indoor_cleaning_fastwam_3cam224_1e-4
+  group: null
+  mode: online
+data:
+  train:
+    _target_: fastwam.datasets.lerobot.robot_video_dataset.RobotVideoDataset
+    dataset_dirs:
+    - /cfsdata/chushan/Galaxea-Dataset/manipulation-clean/Clean_Cutting_Board_20250627_002
+    - /cfsdata/chushan/Galaxea-Dataset/manipulation-clean/Clean_The_Cutting_Board_20250711_006
+    - /cfsdata/chushan/Galaxea-Dataset/manipulation-clean/Clean_The_Cutting_Board_20250804_012
+    - /cfsdata/chushan/Galaxea-Dataset/manipulation-clean/Clean_The_Sink_20250710_006
+    - /cfsdata/chushan/Galaxea-Dataset/manipulation-clean/Clean_The_Sink_20250721_008
+    - /cfsdata/chushan/Galaxea-Dataset/manipulation-clean/Clean_The_Sink_20250730_012
+    - /cfsdata/chushan/Galaxea-Dataset/manipulation-clean/Wipe_The_Desktop_20250801_012
+    - /cfsdata/chushan/Galaxea-Dataset/manipulation-clean/Wipe_The_Desktop_20250804_011
+    expected_total_episodes:
+      Clean_Cutting_Board_20250627_002: 89
+      Clean_The_Cutting_Board_20250711_006: 56
+      Clean_The_Cutting_Board_20250804_012: 51
+      Clean_The_Sink_20250710_006: 79
+      Clean_The_Sink_20250721_008: 58
+      Clean_The_Sink_20250730_012: 46
+      Wipe_The_Desktop_20250801_012: 55
+      Wipe_The_Desktop_20250804_011: 127
+    exclude_episodes:
+      Clean_Cutting_Board_20250627_002:
+      - 1
+      - 2
+      - 3
+      - 4
+      - 5
+      - 6
+      - 7
+      - 8
+      - 9
+      - 10
+      - 11
+      - 12
+      - 13
+      - 14
+      - 15
+      - 16
+      - 17
+      - 18
+      - 20
+      - 21
+      - 23
+      - 24
+      - 25
+      - 31
+      - 34
+      - 45
+      - 56
+      - 67
+      - 78
+      - 85
+      - 88
+      Clean_The_Cutting_Board_20250711_006:
+      - 5
+      - 6
+      - 8
+      - 9
+      - 10
+      - 11
+      - 13
+      - 14
+      - 15
+      - 16
+      - 17
+      - 18
+      - 19
+      - 20
+      - 21
+      - 24
+      - 30
+      - 31
+      - 32
+      - 33
+      - 35
+      - 36
+      - 37
+      - 38
+      - 39
+      - 40
+      - 41
+      - 42
+      - 43
+      - 44
+      - 46
+      - 47
+      - 48
+      - 49
+      - 50
+      - 51
+      Clean_The_Cutting_Board_20250804_012:
+      - 0
+      - 1
+      Clean_The_Sink_20250710_006: []
+      Clean_The_Sink_20250721_008: []
+      Clean_The_Sink_20250730_012: []
+      Wipe_The_Desktop_20250801_012:
+      - 2
+      - 3
+      - 4
+      - 5
+      - 6
+      - 7
+      - 8
+      - 9
+      - 10
+      - 11
+      - 22
+      - 25
+      - 26
+      - 27
+      - 28
+      - 29
+      - 30
+      - 31
+      - 32
+      - 33
+      - 35
+      - 36
+      - 37
+      - 38
+      - 39
+      - 40
+      - 41
+      - 42
+      - 43
+      Wipe_The_Desktop_20250804_011:
+      - 3
+      - 4
+      - 8
+      - 11
+      - 21
+      - 70
+      - 74
+      - 84
+      - 86
+      - 89
+      - 100
+      - 103
+      - 104
+      - 107
+      - 108
+      - 111
+      - 121
+      - 123
+    shape_meta:
+      images:
+      - key: head_rgb
+        raw_shape:
+        - 3
+        - 224
+        - 224
+        shape:
+        - 3
+        - 224
+        - 224
+      - key: left_wrist_rgb
+        raw_shape:
+        - 3
+        - 224
+        - 224
+        shape:
+        - 3
+        - 224
+        - 224
+      - key: right_wrist_rgb
+        raw_shape:
+        - 3
+        - 224
+        - 224
+        shape:
+        - 3
+        - 224
+        - 224
+      action:
+      - key: left_arm
+        raw_shape: 6
+        shape: 6
+      - key: left_gripper
+        raw_shape: 1
+        shape: 1
+      - key: right_arm
+        raw_shape: 6
+        shape: 6
+      - key: right_gripper
+        raw_shape: 1
+        shape: 1
+      state:
+      - key: left_arm
+        raw_shape: 6
+        shape: 6
+      - key: left_gripper
+        raw_shape: 1
+        shape: 1
+      - key: right_arm
+        raw_shape: 6
+        shape: 6
+      - key: right_gripper
+        raw_shape: 1
+        shape: 1
+    num_frames: 33
+    global_sample_stride: 1
+    action_video_freq_ratio: 4
+    video_size:
+    - 384
+    - 320
+    camera_key: null
+    val_set_proportion: 0.0
+    is_training_set: true
+    pretrained_norm_stats: /cfsdata/chushan/Galaxea-Dataset/manipulation-clean/stat-norm/dataset_stats.json
+    skip_padding_as_possible: false
+    concat_multi_camera: robotwin
+    processor:
+      _target_: fastwam.datasets.lerobot.processors.fastwam_processor.FastWAMProcessor
+      shape_meta:
+        images:
+        - key: head_rgb
+          raw_shape:
+          - 3
+          - 224
+          - 224
+          shape:
+          - 3
+          - 224
+          - 224
+        - key: left_wrist_rgb
+          raw_shape:
+          - 3
+          - 224
+          - 224
+          shape:
+          - 3
+          - 224
+          - 224
+        - key: right_wrist_rgb
+          raw_shape:
+          - 3
+          - 224
+          - 224
+          shape:
+          - 3
+          - 224
+          - 224
+        action:
+        - key: left_arm
+          raw_shape: 6
+          shape: 6
+        - key: left_gripper
+          raw_shape: 1
+          shape: 1
+        - key: right_arm
+          raw_shape: 6
+          shape: 6
+        - key: right_gripper
+          raw_shape: 1
+          shape: 1
+        state:
+        - key: left_arm
+          raw_shape: 6
+          shape: 6
+        - key: left_gripper
+          raw_shape: 1
+          shape: 1
+        - key: right_arm
+          raw_shape: 6
+          shape: 6
+        - key: right_gripper
+          raw_shape: 1
+          shape: 1
+      num_obs_steps: 33
+      num_output_cameras: 3
+      action_output_dim: 14
+      proprio_output_dim: 14
+      action_state_transforms:
+      - _target_: fastwam.datasets.lerobot.transforms.relative_action.RelativeJointTransform
+        keys:
+        - left_arm
+        - right_arm
+      use_stepwise_action_norm: true
+      norm_default_mode: z-score
+      norm_exception_mode:
+        action:
+          left_gripper: 0/100
+          right_gripper: 0/100
+      action_state_merger:
+        _target_: fastwam.datasets.lerobot.transforms.action_state_merger.ConcatLeftAlign
+      train_transforms:
+      - _target_: fastwam.datasets.lerobot.transforms.image.ToTensor
+      - _target_: torchvision.transforms.Resize
+        size:
+        - 224
+        - 224
+      val_transforms:
+      - _target_: fastwam.datasets.lerobot.transforms.image.ToTensor
+      - _target_: torchvision.transforms.Resize
+        size:
+        - 224
+        - 224
+      drop_high_level_prob: 1.0
+      use_zh_instruction: false
+    text_embedding_cache_dir: /cfsdata/chushan/Galaxea-Dataset/manipulation-clean/t5-cache
+    context_len: 128
+model:
+  _target_: fastwam.runtime.create_fastwam
+  model_id: Wan-AI/Wan2.2-TI2V-5B
+  tokenizer_model_id: Wan-AI/Wan2.1-T2V-1.3B
+  tokenizer_max_len: 128
+  load_text_encoder: false
+  proprio_dim: 14
+  redirect_common_files: true
+  mot_checkpoint_mixed_attn: true
+  action_dit_pretrained_path: /data/chushan/rift-assets/checkpoints/ActionDiT_linear_interp_Wan22_alphascale_1024hdim.pt
+  skip_dit_load_from_pretrain: false
+  video_dit_config:
+    has_image_input: false
+    patch_size:
+    - 1
+    - 2
+    - 2
+    in_dim: 48
+    hidden_dim: 3072
+    ffn_dim: 14336
+    freq_dim: 256
+    text_dim: 4096
+    out_dim: 48
+    num_heads: 24
+    attn_head_dim: 128
+    num_layers: 30
+    eps: 1.0e-06
+    seperated_timestep: true
+    require_clip_embedding: false
+    require_vae_embedding: false
+    fuse_vae_embedding_in_latents: true
+    use_gradient_checkpointing: true
+    video_attention_mask_mode: first_frame_causal
+    action_conditioned: false
+    action_dim: 14
+    action_group_causal_mask_mode: group_diagonal
+  action_dit_config:
+    action_dim: 14
+    hidden_dim: 1024
+    ffn_dim: 4096
+    num_heads: 24
+    attn_head_dim: 128
+    num_layers: 30
+    text_dim: 4096
+    freq_dim: 256
+    eps: 1.0e-06
+    use_gradient_checkpointing: true
+  video_scheduler:
+    train_shift: 5.0
+    infer_shift: 5.0
+    num_train_timesteps: 1000
+  action_scheduler:
+    train_shift: 5.0
+    infer_shift: 5.0
+    num_train_timesteps: 1000
+  loss:
+    lambda_action: 1.0

galaxea_indoor_cleaning_3cam224/fastwam_10ep/fastwam_step014830.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9e324cad3dbb13d623e4aed51b8534226a13079f1d9e3f3fb59f1f67ec0ec19f
+size 12041813369

galaxea_indoor_cleaning_3cam224/fastwam_joint_10ep/SHA256SUMS

@@ -0,0 +1,3 @@
+7616e7861c240360d6fab1ccca75d54b4857734264210c6a1c1a2f017cbcd869  fastwam_joint_step014830.pt
+564f2a86dd4da6b730e9180695e7083402cef9c1c0b8c06b08ceae5e59be20d8  config.yaml
+52ab72e1185d0fd5c15b7e29187b9606f7e610f5d7d386e4c59ba217442673b2  dataset_stats.json

galaxea_indoor_cleaning_3cam224/fastwam_joint_10ep/config.yaml

@@ -0,0 +1,384 @@
+output_dir: /data/chushan/fastwam-runs/galaxea_indoor_cleaning_fastwam_joint_3cam224_1e-4/20260821_095942_bs16_gas2_10ep_from_scratch_zero1_gc
+batch_size: 16
+num_workers: 6
+lr_scheduler_type: cosine
+learning_rate: 0.0001
+num_epochs: 10
+max_steps: 14830
+log_every: 10
+save_every: 0
+eval_every: 0
+eval_num_inference_steps: 10
+gradient_accumulation_steps: 2
+mixed_precision: bf16
+seed: 42
+max_grad_norm: 1.0
+weight_decay: 0.01
+resume: null
+wandb:
+  enabled: false
+  workspace: null
+  project: fast-wam
+  name: galaxea_indoor_cleaning_fastwam_joint_3cam224_1e-4
+  group: null
+  mode: online
+data:
+  train:
+    _target_: fastwam.datasets.lerobot.robot_video_dataset.RobotVideoDataset
+    dataset_dirs:
+    - /cfsdata/chushan/Galaxea-Dataset/manipulation-clean/Clean_Cutting_Board_20250627_002
+    - /cfsdata/chushan/Galaxea-Dataset/manipulation-clean/Clean_The_Cutting_Board_20250711_006
+    - /cfsdata/chushan/Galaxea-Dataset/manipulation-clean/Clean_The_Cutting_Board_20250804_012
+    - /cfsdata/chushan/Galaxea-Dataset/manipulation-clean/Clean_The_Sink_20250710_006
+    - /cfsdata/chushan/Galaxea-Dataset/manipulation-clean/Clean_The_Sink_20250721_008
+    - /cfsdata/chushan/Galaxea-Dataset/manipulation-clean/Clean_The_Sink_20250730_012
+    - /cfsdata/chushan/Galaxea-Dataset/manipulation-clean/Wipe_The_Desktop_20250801_012
+    - /cfsdata/chushan/Galaxea-Dataset/manipulation-clean/Wipe_The_Desktop_20250804_011
+    expected_total_episodes:
+      Clean_Cutting_Board_20250627_002: 89
+      Clean_The_Cutting_Board_20250711_006: 56
+      Clean_The_Cutting_Board_20250804_012: 51
+      Clean_The_Sink_20250710_006: 79
+      Clean_The_Sink_20250721_008: 58
+      Clean_The_Sink_20250730_012: 46
+      Wipe_The_Desktop_20250801_012: 55
+      Wipe_The_Desktop_20250804_011: 127
+    exclude_episodes:
+      Clean_Cutting_Board_20250627_002:
+      - 1
+      - 2
+      - 3
+      - 4
+      - 5
+      - 6
+      - 7
+      - 8
+      - 9
+      - 10
+      - 11
+      - 12
+      - 13
+      - 14
+      - 15
+      - 16
+      - 17
+      - 18
+      - 20
+      - 21
+      - 23
+      - 24
+      - 25
+      - 31
+      - 34
+      - 45
+      - 56
+      - 67
+      - 78
+      - 85
+      - 88
+      Clean_The_Cutting_Board_20250711_006:
+      - 5
+      - 6
+      - 8
+      - 9
+      - 10
+      - 11
+      - 13
+      - 14
+      - 15
+      - 16
+      - 17
+      - 18
+      - 19
+      - 20
+      - 21
+      - 24
+      - 30
+      - 31
+      - 32
+      - 33
+      - 35
+      - 36
+      - 37
+      - 38
+      - 39
+      - 40
+      - 41
+      - 42
+      - 43
+      - 44
+      - 46
+      - 47
+      - 48
+      - 49
+      - 50
+      - 51
+      Clean_The_Cutting_Board_20250804_012:
+      - 0
+      - 1
+      Clean_The_Sink_20250710_006: []
+      Clean_The_Sink_20250721_008: []
+      Clean_The_Sink_20250730_012: []
+      Wipe_The_Desktop_20250801_012:
+      - 2
+      - 3
+      - 4
+      - 5
+      - 6
+      - 7
+      - 8
+      - 9
+      - 10
+      - 11
+      - 22
+      - 25
+      - 26
+      - 27
+      - 28
+      - 29
+      - 30
+      - 31
+      - 32
+      - 33
+      - 35
+      - 36
+      - 37
+      - 38
+      - 39
+      - 40
+      - 41
+      - 42
+      - 43
+      Wipe_The_Desktop_20250804_011:
+      - 3
+      - 4
+      - 8
+      - 11
+      - 21
+      - 70
+      - 74
+      - 84
+      - 86
+      - 89
+      - 100
+      - 103
+      - 104
+      - 107
+      - 108
+      - 111
+      - 121
+      - 123
+    shape_meta:
+      images:
+      - key: head_rgb
+        raw_shape:
+        - 3
+        - 224
+        - 224
+        shape:
+        - 3
+        - 224
+        - 224
+      - key: left_wrist_rgb
+        raw_shape:
+        - 3
+        - 224
+        - 224
+        shape:
+        - 3
+        - 224
+        - 224
+      - key: right_wrist_rgb
+        raw_shape:
+        - 3
+        - 224
+        - 224
+        shape:
+        - 3
+        - 224
+        - 224
+      action:
+      - key: left_arm
+        raw_shape: 6
+        shape: 6
+      - key: left_gripper
+        raw_shape: 1
+        shape: 1
+      - key: right_arm
+        raw_shape: 6
+        shape: 6
+      - key: right_gripper
+        raw_shape: 1
+        shape: 1
+      state:
+      - key: left_arm
+        raw_shape: 6
+        shape: 6
+      - key: left_gripper
+        raw_shape: 1
+        shape: 1
+      - key: right_arm
+        raw_shape: 6
+        shape: 6
+      - key: right_gripper
+        raw_shape: 1
+        shape: 1
+    num_frames: 33
+    global_sample_stride: 1
+    action_video_freq_ratio: 4
+    video_size:
+    - 384
+    - 320
+    camera_key: null
+    val_set_proportion: 0.0
+    is_training_set: true
+    pretrained_norm_stats: /cfsdata/chushan/Galaxea-Dataset/manipulation-clean/stat-norm/dataset_stats.json
+    skip_padding_as_possible: false
+    concat_multi_camera: robotwin
+    processor:
+      _target_: fastwam.datasets.lerobot.processors.fastwam_processor.FastWAMProcessor
+      shape_meta:
+        images:
+        - key: head_rgb
+          raw_shape:
+          - 3
+          - 224
+          - 224
+          shape:
+          - 3
+          - 224
+          - 224
+        - key: left_wrist_rgb
+          raw_shape:
+          - 3
+          - 224
+          - 224
+          shape:
+          - 3
+          - 224
+          - 224
+        - key: right_wrist_rgb
+          raw_shape:
+          - 3
+          - 224
+          - 224
+          shape:
+          - 3
+          - 224
+          - 224
+        action:
+        - key: left_arm
+          raw_shape: 6
+          shape: 6
+        - key: left_gripper
+          raw_shape: 1
+          shape: 1
+        - key: right_arm
+          raw_shape: 6
+          shape: 6
+        - key: right_gripper
+          raw_shape: 1
+          shape: 1
+        state:
+        - key: left_arm
+          raw_shape: 6
+          shape: 6
+        - key: left_gripper
+          raw_shape: 1
+          shape: 1
+        - key: right_arm
+          raw_shape: 6
+          shape: 6
+        - key: right_gripper
+          raw_shape: 1
+          shape: 1
+      num_obs_steps: 33
+      num_output_cameras: 3
+      action_output_dim: 14
+      proprio_output_dim: 14
+      action_state_transforms:
+      - _target_: fastwam.datasets.lerobot.transforms.relative_action.RelativeJointTransform
+        keys:
+        - left_arm
+        - right_arm
+      use_stepwise_action_norm: true
+      norm_default_mode: z-score
+      norm_exception_mode:
+        action:
+          left_gripper: 0/100
+          right_gripper: 0/100
+      action_state_merger:
+        _target_: fastwam.datasets.lerobot.transforms.action_state_merger.ConcatLeftAlign
+      train_transforms:
+      - _target_: fastwam.datasets.lerobot.transforms.image.ToTensor
+      - _target_: torchvision.transforms.Resize
+        size:
+        - 224
+        - 224
+      val_transforms:
+      - _target_: fastwam.datasets.lerobot.transforms.image.ToTensor
+      - _target_: torchvision.transforms.Resize
+        size:
+        - 224
+        - 224
+      drop_high_level_prob: 1.0
+      use_zh_instruction: false
+    text_embedding_cache_dir: /cfsdata/chushan/Galaxea-Dataset/manipulation-clean/t5-cache
+    context_len: 128
+model:
+  _target_: fastwam.runtime.create_fastwam_joint
+  model_id: Wan-AI/Wan2.2-TI2V-5B
+  tokenizer_model_id: Wan-AI/Wan2.1-T2V-1.3B
+  tokenizer_max_len: 128
+  load_text_encoder: false
+  proprio_dim: 14
+  redirect_common_files: true
+  mot_checkpoint_mixed_attn: true
+  action_dit_pretrained_path: /data/chushan/rift-assets/checkpoints/ActionDiT_linear_interp_Wan22_alphascale_1024hdim.pt
+  skip_dit_load_from_pretrain: false
+  video_dit_config:
+    has_image_input: false
+    patch_size:
+    - 1
+    - 2
+    - 2
+    in_dim: 48
+    hidden_dim: 3072
+    ffn_dim: 14336
+    freq_dim: 256
+    text_dim: 4096
+    out_dim: 48
+    num_heads: 24
+    attn_head_dim: 128
+    num_layers: 30
+    eps: 1.0e-06
+    seperated_timestep: true
+    require_clip_embedding: false
+    require_vae_embedding: false
+    fuse_vae_embedding_in_latents: true
+    use_gradient_checkpointing: true
+    video_attention_mask_mode: first_frame_causal
+    action_conditioned: false
+    action_dim: 14
+    action_group_causal_mask_mode: group_diagonal
+  action_dit_config:
+    action_dim: 14
+    hidden_dim: 1024
+    ffn_dim: 4096
+    num_heads: 24
+    attn_head_dim: 128
+    num_layers: 30
+    text_dim: 4096
+    freq_dim: 256
+    eps: 1.0e-06
+    use_gradient_checkpointing: true
+  video_scheduler:
+    train_shift: 5.0
+    infer_shift: 5.0
+    num_train_timesteps: 1000
+  action_scheduler:
+    train_shift: 5.0
+    infer_shift: 5.0
+    num_train_timesteps: 1000
+  loss:
+    lambda_action: 1.0

galaxea_indoor_cleaning_3cam224/fastwam_joint_10ep/fastwam_joint_step014830.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7616e7861c240360d6fab1ccca75d54b4857734264210c6a1c1a2f017cbcd869
+size 12041813369

galaxea_indoor_cleaning_3cam224/rift_10ep/SHA256SUMS

@@ -0,0 +1,3 @@
+052c0713bfd1f04222599deea43f02cebff963e7453ec7f2e53f81863d8d3939  rift_step014830.pt
+2b15f88fc9e2146993b0d0ca187f666ac5f8e966b676bef14a1e371f4d2204d8  config.yaml
+52ab72e1185d0fd5c15b7e29187b9606f7e610f5d7d386e4c59ba217442673b2  dataset_stats.json

galaxea_indoor_cleaning_3cam224/rift_10ep/config.yaml

@@ -0,0 +1,398 @@
+output_dir: /data/chushan/rift-runs/galaxea_indoor_cleaning_rift_3cam224_1e-4/20260824_132036_bs16_gas2_10ep_zero2
+batch_size: 16
+num_workers: 6
+lr_scheduler_type: cosine
+learning_rate: 0.0001
+num_epochs: 10
+max_steps: 14830
+log_every: 10
+save_every: 0
+gradient_accumulation_steps: 2
+mixed_precision: bf16
+seed: 42
+max_grad_norm: 1.0
+weight_decay: 0.01
+resume: null
+wandb:
+  enabled: false
+  workspace: null
+  project: rift
+  name: rift_galaxea_10ep_bs16_gas2
+  group: null
+  mode: online
+data:
+  train:
+    _target_: rift.datasets.lerobot.robot_video_dataset.RobotVideoDataset
+    dataset_dirs:
+    - /cfsdata/chushan/Galaxea-Dataset/manipulation-clean/Clean_Cutting_Board_20250627_002
+    - /cfsdata/chushan/Galaxea-Dataset/manipulation-clean/Clean_The_Cutting_Board_20250711_006
+    - /cfsdata/chushan/Galaxea-Dataset/manipulation-clean/Clean_The_Cutting_Board_20250804_012
+    - /cfsdata/chushan/Galaxea-Dataset/manipulation-clean/Clean_The_Sink_20250710_006
+    - /cfsdata/chushan/Galaxea-Dataset/manipulation-clean/Clean_The_Sink_20250721_008
+    - /cfsdata/chushan/Galaxea-Dataset/manipulation-clean/Clean_The_Sink_20250730_012
+    - /cfsdata/chushan/Galaxea-Dataset/manipulation-clean/Wipe_The_Desktop_20250801_012
+    - /cfsdata/chushan/Galaxea-Dataset/manipulation-clean/Wipe_The_Desktop_20250804_011
+    expected_total_episodes:
+      Clean_Cutting_Board_20250627_002: 89
+      Clean_The_Cutting_Board_20250711_006: 56
+      Clean_The_Cutting_Board_20250804_012: 51
+      Clean_The_Sink_20250710_006: 79
+      Clean_The_Sink_20250721_008: 58
+      Clean_The_Sink_20250730_012: 46
+      Wipe_The_Desktop_20250801_012: 55
+      Wipe_The_Desktop_20250804_011: 127
+    exclude_episodes:
+      Clean_Cutting_Board_20250627_002:
+      - 1
+      - 2
+      - 3
+      - 4
+      - 5
+      - 6
+      - 7
+      - 8
+      - 9
+      - 10
+      - 11
+      - 12
+      - 13
+      - 14
+      - 15
+      - 16
+      - 17
+      - 18
+      - 20
+      - 21
+      - 23
+      - 24
+      - 25
+      - 31
+      - 34
+      - 45
+      - 56
+      - 67
+      - 78
+      - 85
+      - 88
+      Clean_The_Cutting_Board_20250711_006:
+      - 5
+      - 6
+      - 8
+      - 9
+      - 10
+      - 11
+      - 13
+      - 14
+      - 15
+      - 16
+      - 17
+      - 18
+      - 19
+      - 20
+      - 21
+      - 24
+      - 30
+      - 31
+      - 32
+      - 33
+      - 35
+      - 36
+      - 37
+      - 38
+      - 39
+      - 40
+      - 41
+      - 42
+      - 43
+      - 44
+      - 46
+      - 47
+      - 48
+      - 49
+      - 50
+      - 51
+      Clean_The_Cutting_Board_20250804_012:
+      - 0
+      - 1
+      Clean_The_Sink_20250710_006: []
+      Clean_The_Sink_20250721_008: []
+      Clean_The_Sink_20250730_012: []
+      Wipe_The_Desktop_20250801_012:
+      - 2
+      - 3
+      - 4
+      - 5
+      - 6
+      - 7
+      - 8
+      - 9
+      - 10
+      - 11
+      - 22
+      - 25
+      - 26
+      - 27
+      - 28
+      - 29
+      - 30
+      - 31
+      - 32
+      - 33
+      - 35
+      - 36
+      - 37
+      - 38
+      - 39
+      - 40
+      - 41
+      - 42
+      - 43
+      Wipe_The_Desktop_20250804_011:
+      - 3
+      - 4
+      - 8
+      - 11
+      - 21
+      - 70
+      - 74
+      - 84
+      - 86
+      - 89
+      - 100
+      - 103
+      - 104
+      - 107
+      - 108
+      - 111
+      - 121
+      - 123
+    shape_meta:
+      images:
+      - key: head_rgb
+        raw_shape:
+        - 3
+        - 224
+        - 224
+        shape:
+        - 3
+        - 224
+        - 224
+      - key: left_wrist_rgb
+        raw_shape:
+        - 3
+        - 224
+        - 224
+        shape:
+        - 3
+        - 224
+        - 224
+      - key: right_wrist_rgb
+        raw_shape:
+        - 3
+        - 224
+        - 224
+        shape:
+        - 3
+        - 224
+        - 224
+      action:
+      - key: left_arm
+        raw_shape: 6
+        shape: 6
+      - key: left_gripper
+        raw_shape: 1
+        shape: 1
+      - key: right_arm
+        raw_shape: 6
+        shape: 6
+      - key: right_gripper
+        raw_shape: 1
+        shape: 1
+      state:
+      - key: left_arm
+        raw_shape: 6
+        shape: 6
+      - key: left_gripper
+        raw_shape: 1
+        shape: 1
+      - key: right_arm
+        raw_shape: 6
+        shape: 6
+      - key: right_gripper
+        raw_shape: 1
+        shape: 1
+    num_frames: 33
+    global_sample_stride: 1
+    action_video_freq_ratio: 4
+    video_size:
+    - 384
+    - 320
+    camera_key: null
+    val_set_proportion: 0.0
+    is_training_set: true
+    pretrained_norm_stats: /cfsdata/chushan/Galaxea-Dataset/manipulation-clean/stat-norm/dataset_stats.json
+    skip_padding_as_possible: false
+    concat_multi_camera: robotwin
+    processor:
+      _target_: rift.datasets.lerobot.processors.rift_processor.RIFTProcessor
+      shape_meta:
+        images:
+        - key: head_rgb
+          raw_shape:
+          - 3
+          - 224
+          - 224
+          shape:
+          - 3
+          - 224
+          - 224
+        - key: left_wrist_rgb
+          raw_shape:
+          - 3
+          - 224
+          - 224
+          shape:
+          - 3
+          - 224
+          - 224
+        - key: right_wrist_rgb
+          raw_shape:
+          - 3
+          - 224
+          - 224
+          shape:
+          - 3
+          - 224
+          - 224
+        action:
+        - key: left_arm
+          raw_shape: 6
+          shape: 6
+        - key: left_gripper
+          raw_shape: 1
+          shape: 1
+        - key: right_arm
+          raw_shape: 6
+          shape: 6
+        - key: right_gripper
+          raw_shape: 1
+          shape: 1
+        state:
+        - key: left_arm
+          raw_shape: 6
+          shape: 6
+        - key: left_gripper
+          raw_shape: 1
+          shape: 1
+        - key: right_arm
+          raw_shape: 6
+          shape: 6
+        - key: right_gripper
+          raw_shape: 1
+          shape: 1
+      num_obs_steps: 33
+      num_output_cameras: 3
+      action_output_dim: 14
+      proprio_output_dim: 14
+      action_state_transforms:
+      - _target_: rift.datasets.lerobot.transforms.relative_joint.RelativeJointTransform
+        keys:
+        - left_arm
+        - right_arm
+      use_stepwise_action_norm: true
+      norm_default_mode: z-score
+      norm_exception_mode:
+        action:
+          left_gripper: 0/100
+          right_gripper: 0/100
+      action_state_merger:
+        _target_: rift.datasets.lerobot.transforms.action_state_merger.ConcatLeftAlign
+      train_transforms:
+      - _target_: rift.datasets.lerobot.transforms.image.ToTensor
+      - _target_: torchvision.transforms.Resize
+        size:
+        - 224
+        - 224
+      val_transforms:
+      - _target_: rift.datasets.lerobot.transforms.image.ToTensor
+      - _target_: torchvision.transforms.Resize
+        size:
+        - 224
+        - 224
+      drop_high_level_prob: 1.0
+      use_zh_instruction: false
+    text_embedding_cache_dir: /cfsdata/chushan/Galaxea-Dataset/manipulation-clean/t5-cache
+    context_len: 128
+model:
+  _target_: rift.model_factory.create_rift
+  anticip_future_tokens: 240
+  anticip_latent_t: 3
+  anticip_lambda: 1.0
+  anticip_action_lambda: 1.0
+  anticip_delta_mix: 0.8
+  anticip_lambda_final: 0.2
+  anticip_anneal_start_frac: 0.7
+  anticip_anneal_total_steps: 14830
+  anticip_cond_noise_p: 0.3
+  anticip_cond_noise_sigma: 0.06
+  anticip_cond_noise_ramp_start_frac: 0.7
+  anticip_cond_noise_action_free: true
+  anticip_fm_lambda: 1.0
+  anticip_fm_width: 512
+  anticip_fm_blocks: 2
+  anticip_fm_lambda_final: 0.2
+  model_id: Wan-AI/Wan2.2-TI2V-5B
+  tokenizer_model_id: Wan-AI/Wan2.1-T2V-1.3B
+  tokenizer_max_len: 128
+  load_text_encoder: false
+  proprio_dim: 14
+  redirect_common_files: true
+  mot_checkpoint_mixed_attn: true
+  action_dit_pretrained_path: /data/chushan/rift-assets/checkpoints/ActionDiT_linear_interp_Wan22_alphascale_1024hdim.pt
+  skip_dit_load_from_pretrain: false
+  video_dit_config:
+    has_image_input: false
+    patch_size:
+    - 1
+    - 2
+    - 2
+    in_dim: 48
+    hidden_dim: 3072
+    ffn_dim: 14336
+    freq_dim: 256
+    text_dim: 4096
+    out_dim: 48
+    num_heads: 24
+    attn_head_dim: 128
+    num_layers: 30
+    eps: 1.0e-06
+    seperated_timestep: true
+    require_clip_embedding: false
+    require_vae_embedding: false
+    fuse_vae_embedding_in_latents: true
+    use_gradient_checkpointing: true
+    video_attention_mask_mode: first_frame_causal
+    action_conditioned: false
+    action_dim: 14
+    action_group_causal_mask_mode: group_diagonal
+  action_dit_config:
+    action_dim: 14
+    hidden_dim: 1024
+    ffn_dim: 4096
+    num_heads: 24
+    attn_head_dim: 128
+    num_layers: 30
+    text_dim: 4096
+    freq_dim: 256
+    eps: 1.0e-06
+    use_gradient_checkpointing: true
+  video_scheduler:
+    train_shift: 5.0
+    infer_shift: 5.0
+    num_train_timesteps: 1000
+  action_scheduler:
+    train_shift: 5.0
+    infer_shift: 5.0
+    num_train_timesteps: 1000
+  loss:
+    lambda_action: 1.0

galaxea_indoor_cleaning_3cam224/rift_10ep/rift_step014830.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:052c0713bfd1f04222599deea43f02cebff963e7453ec7f2e53f81863d8d3939
+size 12061290937

robocoin_multitask_mm/README.md

@@ -0,0 +1,27 @@
+# RoboCOIN RIFT checkpoints
+
+RIFT trained for 10 epochs / 9,340 optimizer steps on three RoboCOIN tasks:
+object classification (191 episodes), red/yellow test-tube mixing (50), and
+basket stacking (101). The numerical training view is in [training_view](https://huggingface.co/PoopBear/RIFT/tree/main/robocoin_multitask_mm/training_view).
+
+| Variant | Directory | `fuse_training_paths` |
+| --- | --- | --- |
+| Fused | [rift_fused_z1_10ep](https://huggingface.co/PoopBear/RIFT/tree/main/robocoin_multitask_mm/rift_fused_z1_10ep) | `true` |
+| Separate | [rift_separate_z1_10ep](https://huggingface.co/PoopBear/RIFT/tree/main/robocoin_multitask_mm/rift_separate_z1_10ep) | `false` |
+
+Each directory contains `rift_step_009340.pt`, `config.yaml`,
+`dataset_stats.json`, and `SHA256SUMS`. These are complete weights-only
+checkpoints, copied without modification from [research-vla/RIFT-RoboCOIN](https://huggingface.co/research-vla/RIFT-RoboCOIN/tree/debae88e14f2c0f32c0e86eafd4037a10e7b7123).
+Both variants use the same inference architecture and include the anticipation/probe parameters.
+
+```bash
+hf download PoopBear/RIFT \
+  --include 'robocoin_multitask_mm/rift_fused_z1_10ep/*' \
+  --local-dir ./checkpoints
+```
+
+Replace `rift_fused_z1_10ep` with `rift_separate_z1_10ep` to download the other variant.
+Use the statistics packaged with the selected checkpoint. The policy field order
+is `[left_arm (6), left_gripper, right_arm (6), right_gripper]`; grippers use the
+0–100 mm stroke. Preserve the corresponding image, state, and action preprocessing.
+The configs record training paths; update these for your local assets.

robocoin_multitask_mm/rift_fused_z1_10ep/SHA256SUMS

@@ -0,0 +1,3 @@
+c6ca1db0a7e4f06c0867a2d7e06caf1775308e8ddfc51013cbaf5459ba7dcaad  rift_step_009340.pt
+880f9a93f816415cfa1f5a3792ff50241926d17cba6094b626db2c4624663021  config.yaml
+b3bc2d0d090e84ef78ace8c4388d517b53a8bc01a5ea95a82b0cf9a5f42cf41a  dataset_stats.json

robocoin_multitask_mm/rift_fused_z1_10ep/config.yaml

@@ -0,0 +1,271 @@
+output_dir: /scratch3/ton068/datasets/isaac/runs/robocoin_rift_2x4_fused_z1_10ep_001/train
+batch_size: 16
+num_workers: 6
+vae_cache_dir: /scratch3/ton068/datasets/isaac/RoboCOIN-downloads/vae-cache
+lr_scheduler_type: cosine
+learning_rate: 0.0001
+num_epochs: 10
+max_steps: 9340
+log_every: 10
+save_every: 1000
+gradient_accumulation_steps: 2
+mixed_precision: bf16
+seed: 42
+max_grad_norm: 1.0
+weight_decay: 0.01
+resume: null
+wandb:
+  enabled: false
+  workspace: null
+  project: rift
+  name: galaxea_indoor_cleaning_rift_3cam224_1e-4
+  group: null
+  mode: online
+data:
+  train:
+    _target_: rift.datasets.lerobot.robot_video_dataset.RobotVideoDataset
+    dataset_dirs:
+    - /scratch3/ton068/datasets/isaac/RoboCOIN-downloads/training-views/galaxea_mm_v1/Galaxea_R1_Lite_classify_object_four
+    - /scratch3/ton068/datasets/isaac/RoboCOIN-downloads/training-views/galaxea_mm_v1/Galaxea_R1_Lite_mix_red_yellow_large_test_tube
+    - /scratch3/ton068/datasets/isaac/RoboCOIN-downloads/training-views/galaxea_mm_v1/R1_Lite_stack_baskets
+    expected_total_episodes:
+      Galaxea_R1_Lite_classify_object_four: 191
+      Galaxea_R1_Lite_mix_red_yellow_large_test_tube: 50
+      R1_Lite_stack_baskets: 101
+    exclude_episodes: null
+    shape_meta:
+      images:
+      - key: head_rgb
+        raw_shape:
+        - 3
+        - 224
+        - 224
+        shape:
+        - 3
+        - 224
+        - 224
+      - key: left_wrist_rgb
+        raw_shape:
+        - 3
+        - 224
+        - 224
+        shape:
+        - 3
+        - 224
+        - 224
+      - key: right_wrist_rgb
+        raw_shape:
+        - 3
+        - 224
+        - 224
+        shape:
+        - 3
+        - 224
+        - 224
+      action:
+      - key: left_arm
+        raw_shape: 6
+        shape: 6
+      - key: left_gripper
+        raw_shape: 1
+        shape: 1
+      - key: right_arm
+        raw_shape: 6
+        shape: 6
+      - key: right_gripper
+        raw_shape: 1
+        shape: 1
+      state:
+      - key: left_arm
+        raw_shape: 6
+        shape: 6
+      - key: left_gripper
+        raw_shape: 1
+        shape: 1
+      - key: right_arm
+        raw_shape: 6
+        shape: 6
+      - key: right_gripper
+        raw_shape: 1
+        shape: 1
+    num_frames: 33
+    global_sample_stride: 1
+    action_video_freq_ratio: 4
+    sample_video_before_transforms: true
+    vae_cache_index_dir: /scratch3/ton068/datasets/isaac/RoboCOIN-downloads/vae-cache/_precompute/20260922T030557Z
+    vae_cache_dir: /scratch3/ton068/datasets/isaac/RoboCOIN-downloads/vae-cache
+    video_size:
+    - 384
+    - 320
+    camera_key: null
+    val_set_proportion: 0.0
+    is_training_set: true
+    pretrained_norm_stats: /scratch3/ton068/datasets/isaac/RoboCOIN-downloads/stat-norm/robocoin_multitask_mm/dataset_stats.json
+    skip_padding_as_possible: false
+    concat_multi_camera: robotwin
+    processor:
+      _target_: rift.datasets.lerobot.processors.rift_processor.RIFTProcessor
+      shape_meta:
+        images:
+        - key: head_rgb
+          raw_shape:
+          - 3
+          - 224
+          - 224
+          shape:
+          - 3
+          - 224
+          - 224
+        - key: left_wrist_rgb
+          raw_shape:
+          - 3
+          - 224
+          - 224
+          shape:
+          - 3
+          - 224
+          - 224
+        - key: right_wrist_rgb
+          raw_shape:
+          - 3
+          - 224
+          - 224
+          shape:
+          - 3
+          - 224
+          - 224
+        action:
+        - key: left_arm
+          raw_shape: 6
+          shape: 6
+        - key: left_gripper
+          raw_shape: 1
+          shape: 1
+        - key: right_arm
+          raw_shape: 6
+          shape: 6
+        - key: right_gripper
+          raw_shape: 1
+          shape: 1
+        state:
+        - key: left_arm
+          raw_shape: 6
+          shape: 6
+        - key: left_gripper
+          raw_shape: 1
+          shape: 1
+        - key: right_arm
+          raw_shape: 6
+          shape: 6
+        - key: right_gripper
+          raw_shape: 1
+          shape: 1
+      num_obs_steps: 33
+      num_output_cameras: 3
+      action_output_dim: 14
+      proprio_output_dim: 14
+      action_state_transforms:
+      - _target_: rift.datasets.lerobot.transforms.relative_joint.RelativeJointTransform
+        keys:
+        - left_arm
+        - right_arm
+      use_stepwise_action_norm: true
+      norm_default_mode: z-score
+      norm_exception_mode:
+        action:
+          left_gripper: 0/100
+          right_gripper: 0/100
+      action_state_merger:
+        _target_: rift.datasets.lerobot.transforms.action_state_merger.ConcatLeftAlign
+      train_transforms:
+      - _target_: rift.datasets.lerobot.transforms.image.ToTensor
+      - _target_: torchvision.transforms.Resize
+        size:
+        - 224
+        - 224
+      val_transforms:
+      - _target_: rift.datasets.lerobot.transforms.image.ToTensor
+      - _target_: torchvision.transforms.Resize
+        size:
+        - 224
+        - 224
+      drop_high_level_prob: 1.0
+      use_zh_instruction: false
+    text_embedding_cache_dir: /scratch3/ton068/datasets/isaac/RoboCOIN-downloads/t5-cache
+    context_len: 128
+    text_memory_cache_size: 128
+model:
+  _target_: rift.model_factory.create_rift
+  anticip_future_tokens: 240
+  anticip_latent_t: 3
+  anticip_lambda: 1.0
+  anticip_action_lambda: 1.0
+  anticip_delta_mix: 0.8
+  anticip_lambda_final: 0.2
+  anticip_anneal_start_frac: 0.7
+  anticip_anneal_total_steps: 9340
+  anticip_cond_noise_p: 0.3
+  anticip_cond_noise_sigma: 0.06
+  anticip_cond_noise_ramp_start_frac: 0.7
+  anticip_cond_noise_action_free: true
+  anticip_fm_lambda: 1.0
+  anticip_fm_width: 512
+  anticip_fm_blocks: 2
+  anticip_fm_lambda_final: 0.2
+  model_id: Wan-AI/Wan2.2-TI2V-5B
+  tokenizer_model_id: Wan-AI/Wan2.1-T2V-1.3B
+  tokenizer_max_len: 128
+  load_text_encoder: false
+  proprio_dim: 14
+  redirect_common_files: true
+  mot_checkpoint_mixed_attn: true
+  fuse_training_paths: true
+  train_probe_head: true
+  action_dit_pretrained_path: /scratch3/ton068/datasets/isaac/rift-assets/checkpoints/ActionDiT_linear_interp_Wan22_alphascale_1024hdim.pt
+  skip_dit_load_from_pretrain: false
+  video_dit_config:
+    has_image_input: false
+    patch_size:
+    - 1
+    - 2
+    - 2
+    in_dim: 48
+    hidden_dim: 3072
+    ffn_dim: 14336
+    freq_dim: 256
+    text_dim: 4096
+    out_dim: 48
+    num_heads: 24
+    attn_head_dim: 128
+    num_layers: 30
+    eps: 1.0e-06
+    seperated_timestep: true
+    require_clip_embedding: false
+    require_vae_embedding: false
+    fuse_vae_embedding_in_latents: true
+    use_gradient_checkpointing: true
+    video_attention_mask_mode: first_frame_causal
+    action_conditioned: false
+    action_dim: 14
+    action_group_causal_mask_mode: group_diagonal
+  action_dit_config:
+    action_dim: 14
+    hidden_dim: 1024
+    ffn_dim: 4096
+    num_heads: 24
+    attn_head_dim: 128
+    num_layers: 30
+    text_dim: 4096
+    freq_dim: 256
+    eps: 1.0e-06
+    use_gradient_checkpointing: true
+  video_scheduler:
+    train_shift: 5.0
+    infer_shift: 5.0
+    num_train_timesteps: 1000
+  action_scheduler:
+    train_shift: 5.0
+    infer_shift: 5.0
+    num_train_timesteps: 1000
+  loss:
+    lambda_action: 1.0

robocoin_multitask_mm/rift_fused_z1_10ep/rift_step_009340.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c6ca1db0a7e4f06c0867a2d7e06caf1775308e8ddfc51013cbaf5459ba7dcaad
+size 12061290937

robocoin_multitask_mm/rift_separate_z1_10ep/SHA256SUMS

@@ -0,0 +1,3 @@
+8d049fdd5bc5928912cb62685eb907234644509114d2167fe63eab00cc74b7a7  rift_step_009340.pt
+915a468956797513cbb4cba4cd88219e2055f77476735f786a5937253dddcb85  config.yaml
+b3bc2d0d090e84ef78ace8c4388d517b53a8bc01a5ea95a82b0cf9a5f42cf41a  dataset_stats.json

robocoin_multitask_mm/rift_separate_z1_10ep/config.yaml

@@ -0,0 +1,271 @@
+output_dir: /scratch3/ton068/datasets/isaac/runs/robocoin_rift_2x4_separate_z1_10ep_001/train
+batch_size: 16
+num_workers: 6
+vae_cache_dir: /scratch3/ton068/datasets/isaac/RoboCOIN-downloads/vae-cache
+lr_scheduler_type: cosine
+learning_rate: 0.0001
+num_epochs: 10
+max_steps: 9340
+log_every: 10
+save_every: 1000
+gradient_accumulation_steps: 2
+mixed_precision: bf16
+seed: 42
+max_grad_norm: 1.0
+weight_decay: 0.01
+resume: null
+wandb:
+  enabled: false
+  workspace: null
+  project: rift
+  name: galaxea_indoor_cleaning_rift_3cam224_1e-4
+  group: null
+  mode: online
+data:
+  train:
+    _target_: rift.datasets.lerobot.robot_video_dataset.RobotVideoDataset
+    dataset_dirs:
+    - /scratch3/ton068/datasets/isaac/RoboCOIN-downloads/training-views/galaxea_mm_v1/Galaxea_R1_Lite_classify_object_four
+    - /scratch3/ton068/datasets/isaac/RoboCOIN-downloads/training-views/galaxea_mm_v1/Galaxea_R1_Lite_mix_red_yellow_large_test_tube
+    - /scratch3/ton068/datasets/isaac/RoboCOIN-downloads/training-views/galaxea_mm_v1/R1_Lite_stack_baskets
+    expected_total_episodes:
+      Galaxea_R1_Lite_classify_object_four: 191
+      Galaxea_R1_Lite_mix_red_yellow_large_test_tube: 50
+      R1_Lite_stack_baskets: 101
+    exclude_episodes: null
+    shape_meta:
+      images:
+      - key: head_rgb
+        raw_shape:
+        - 3
+        - 224
+        - 224
+        shape:
+        - 3
+        - 224
+        - 224
+      - key: left_wrist_rgb
+        raw_shape:
+        - 3
+        - 224
+        - 224
+        shape:
+        - 3
+        - 224
+        - 224
+      - key: right_wrist_rgb
+        raw_shape:
+        - 3
+        - 224
+        - 224
+        shape:
+        - 3
+        - 224
+        - 224
+      action:
+      - key: left_arm
+        raw_shape: 6
+        shape: 6
+      - key: left_gripper
+        raw_shape: 1
+        shape: 1
+      - key: right_arm
+        raw_shape: 6
+        shape: 6
+      - key: right_gripper
+        raw_shape: 1
+        shape: 1
+      state:
+      - key: left_arm
+        raw_shape: 6
+        shape: 6
+      - key: left_gripper
+        raw_shape: 1
+        shape: 1
+      - key: right_arm
+        raw_shape: 6
+        shape: 6
+      - key: right_gripper
+        raw_shape: 1
+        shape: 1
+    num_frames: 33
+    global_sample_stride: 1
+    action_video_freq_ratio: 4
+    sample_video_before_transforms: true
+    vae_cache_index_dir: /scratch3/ton068/datasets/isaac/RoboCOIN-downloads/vae-cache/_precompute/20260922T030557Z
+    vae_cache_dir: /scratch3/ton068/datasets/isaac/RoboCOIN-downloads/vae-cache
+    video_size:
+    - 384
+    - 320
+    camera_key: null
+    val_set_proportion: 0.0
+    is_training_set: true
+    pretrained_norm_stats: /scratch3/ton068/datasets/isaac/RoboCOIN-downloads/stat-norm/robocoin_multitask_mm/dataset_stats.json
+    skip_padding_as_possible: false
+    concat_multi_camera: robotwin
+    processor:
+      _target_: rift.datasets.lerobot.processors.rift_processor.RIFTProcessor
+      shape_meta:
+        images:
+        - key: head_rgb
+          raw_shape:
+          - 3
+          - 224
+          - 224
+          shape:
+          - 3
+          - 224
+          - 224
+        - key: left_wrist_rgb
+          raw_shape:
+          - 3
+          - 224
+          - 224
+          shape:
+          - 3
+          - 224
+          - 224
+        - key: right_wrist_rgb
+          raw_shape:
+          - 3
+          - 224
+          - 224
+          shape:
+          - 3
+          - 224
+          - 224
+        action:
+        - key: left_arm
+          raw_shape: 6
+          shape: 6
+        - key: left_gripper
+          raw_shape: 1
+          shape: 1
+        - key: right_arm
+          raw_shape: 6
+          shape: 6
+        - key: right_gripper
+          raw_shape: 1
+          shape: 1
+        state:
+        - key: left_arm
+          raw_shape: 6
+          shape: 6
+        - key: left_gripper
+          raw_shape: 1
+          shape: 1
+        - key: right_arm
+          raw_shape: 6
+          shape: 6
+        - key: right_gripper
+          raw_shape: 1
+          shape: 1
+      num_obs_steps: 33
+      num_output_cameras: 3
+      action_output_dim: 14
+      proprio_output_dim: 14
+      action_state_transforms:
+      - _target_: rift.datasets.lerobot.transforms.relative_joint.RelativeJointTransform
+        keys:
+        - left_arm
+        - right_arm
+      use_stepwise_action_norm: true
+      norm_default_mode: z-score
+      norm_exception_mode:
+        action:
+          left_gripper: 0/100
+          right_gripper: 0/100
+      action_state_merger:
+        _target_: rift.datasets.lerobot.transforms.action_state_merger.ConcatLeftAlign
+      train_transforms:
+      - _target_: rift.datasets.lerobot.transforms.image.ToTensor
+      - _target_: torchvision.transforms.Resize
+        size:
+        - 224
+        - 224
+      val_transforms:
+      - _target_: rift.datasets.lerobot.transforms.image.ToTensor
+      - _target_: torchvision.transforms.Resize
+        size:
+        - 224
+        - 224
+      drop_high_level_prob: 1.0
+      use_zh_instruction: false
+    text_embedding_cache_dir: /scratch3/ton068/datasets/isaac/RoboCOIN-downloads/t5-cache
+    context_len: 128
+    text_memory_cache_size: 128
+model:
+  _target_: rift.model_factory.create_rift
+  anticip_future_tokens: 240
+  anticip_latent_t: 3
+  anticip_lambda: 1.0
+  anticip_action_lambda: 1.0
+  anticip_delta_mix: 0.8
+  anticip_lambda_final: 0.2
+  anticip_anneal_start_frac: 0.7
+  anticip_anneal_total_steps: 9340
+  anticip_cond_noise_p: 0.3
+  anticip_cond_noise_sigma: 0.06
+  anticip_cond_noise_ramp_start_frac: 0.7
+  anticip_cond_noise_action_free: true
+  anticip_fm_lambda: 1.0
+  anticip_fm_width: 512
+  anticip_fm_blocks: 2
+  anticip_fm_lambda_final: 0.2
+  model_id: Wan-AI/Wan2.2-TI2V-5B
+  tokenizer_model_id: Wan-AI/Wan2.1-T2V-1.3B
+  tokenizer_max_len: 128
+  load_text_encoder: false
+  proprio_dim: 14
+  redirect_common_files: true
+  mot_checkpoint_mixed_attn: true
+  fuse_training_paths: false
+  train_probe_head: true
+  action_dit_pretrained_path: /scratch3/ton068/datasets/isaac/rift-assets/checkpoints/ActionDiT_linear_interp_Wan22_alphascale_1024hdim.pt
+  skip_dit_load_from_pretrain: false
+  video_dit_config:
+    has_image_input: false
+    patch_size:
+    - 1
+    - 2
+    - 2
+    in_dim: 48
+    hidden_dim: 3072
+    ffn_dim: 14336
+    freq_dim: 256
+    text_dim: 4096
+    out_dim: 48
+    num_heads: 24
+    attn_head_dim: 128
+    num_layers: 30
+    eps: 1.0e-06
+    seperated_timestep: true
+    require_clip_embedding: false
+    require_vae_embedding: false
+    fuse_vae_embedding_in_latents: true
+    use_gradient_checkpointing: true
+    video_attention_mask_mode: first_frame_causal
+    action_conditioned: false
+    action_dim: 14
+    action_group_causal_mask_mode: group_diagonal
+  action_dit_config:
+    action_dim: 14
+    hidden_dim: 1024
+    ffn_dim: 4096
+    num_heads: 24
+    attn_head_dim: 128
+    num_layers: 30
+    text_dim: 4096
+    freq_dim: 256
+    eps: 1.0e-06
+    use_gradient_checkpointing: true
+  video_scheduler:
+    train_shift: 5.0
+    infer_shift: 5.0
+    num_train_timesteps: 1000
+  action_scheduler:
+    train_shift: 5.0
+    infer_shift: 5.0
+    num_train_timesteps: 1000
+  loss:
+    lambda_action: 1.0

robocoin_multitask_mm/rift_separate_z1_10ep/rift_step_009340.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8d049fdd5bc5928912cb62685eb907234644509114d2167fe63eab00cc74b7a7
+size 12061290937