Hugging Face's logo

RoboColosseum
/
so101-stack-cubes-molmoact2

Robotics
LeRobot
Safetensors
robocolosseum
stack-cubes
so101
molmoact2
Model card Files
xet
 Community
aarontung commited on
Commit
cbaa985
·
verified ·
1 Parent(s): 634c37c

Copy SO101 model selected in FineTune Track

Browse files

Source: tsangb34/molmoact2-so101-stack-cubes-40k at 27489e5defaeb02ed86148c9b02a5149105d6c96; workbook cell E18

Files changed (32) hide show
  1. README.md +166 -0
  2. checkpoints/020000/pretrained_model/config.json +139 -0
  3. checkpoints/020000/pretrained_model/model.safetensors +3 -0
  4. checkpoints/020000/pretrained_model/policy_postprocessor.json +43 -0
  5. checkpoints/020000/pretrained_model/policy_postprocessor_step_1_molmoact2_masked_unnormalizer.safetensors +3 -0
  6. checkpoints/020000/pretrained_model/policy_preprocessor.json +100 -0
  7. checkpoints/020000/pretrained_model/policy_preprocessor_step_3_molmoact2_masked_normalizer.safetensors +3 -0
  8. checkpoints/020000/pretrained_model/train_config.json +275 -0
  9. checkpoints/020000/training_state/optimizer_param_groups.json +1370 -0
  10. checkpoints/020000/training_state/optimizer_state.safetensors +3 -0
  11. checkpoints/020000/training_state/rng_state.safetensors +3 -0
  12. checkpoints/020000/training_state/scheduler_state.json +24 -0
  13. checkpoints/020000/training_state/training_step.json +5 -0
  14. checkpoints/040000/pretrained_model/config.json +139 -0
  15. checkpoints/040000/pretrained_model/model.safetensors +3 -0
  16. checkpoints/040000/pretrained_model/policy_postprocessor.json +43 -0
  17. checkpoints/040000/pretrained_model/policy_postprocessor_step_1_molmoact2_masked_unnormalizer.safetensors +3 -0
  18. checkpoints/040000/pretrained_model/policy_preprocessor.json +100 -0
  19. checkpoints/040000/pretrained_model/policy_preprocessor_step_3_molmoact2_masked_normalizer.safetensors +3 -0
  20. checkpoints/040000/pretrained_model/train_config.json +275 -0
  21. checkpoints/040000/training_state/optimizer_param_groups.json +1370 -0
  22. checkpoints/040000/training_state/optimizer_state.safetensors +3 -0
  23. checkpoints/040000/training_state/rng_state.safetensors +3 -0
  24. checkpoints/040000/training_state/scheduler_state.json +24 -0
  25. checkpoints/040000/training_state/training_step.json +5 -0
  26. config.json +139 -0
  27. model.safetensors +3 -0
  28. policy_postprocessor.json +43 -0
  29. policy_postprocessor_step_1_molmoact2_masked_unnormalizer.safetensors +3 -0
  30. policy_preprocessor.json +100 -0
  31. policy_preprocessor_step_3_molmoact2_masked_normalizer.safetensors +3 -0
  32. train_config.json +275 -0
README.md ADDED
@@ -0,0 +1,166 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ datasets: Jiamo0912/so101-stack-cubes
3
+ library_name: lerobot
4
+ license: apache-2.0
5
+ model_name: molmoact2
6
+ pipeline_tag: robotics
7
+ tags:
8
+ - robocolosseum
9
+ - lerobot
10
+ - stack-cubes
11
+ - so101
12
+ - robotics
13
+ - molmoact2
14
+ ---
15
+
16
+ # Model Card for molmoact2
17
+
18
+ <!-- Provide a quick summary of what the model is/does. -->
19
+
20
+
21
+ [MolmoAct2](https://allenai.org/blog/molmoact2) is an open robotics foundation model from the Allen Institute for AI (Ai2) that maps camera images and language instructions to robot action chunks. The LeRobot implementation supports training and evaluation of the regular MolmoAct2 model.
22
+
23
+
24
+
25
+
26
+ <!-- A short demo is worth more than any description! Record a GIF/video of the policy
27
+ running on your robot, upload it to this repo, and embed it here:
28
+ <p align="center">
29
+ <img src="https://huggingface.co/<hf_user>/<policy_repo_id>/resolve/main/demo.gif" width="60%"/>
30
+ </p>
31
+ -->
32
+
33
+ This policy has been trained and pushed to the Hub using [LeRobot](https://github.com/huggingface/lerobot).
34
+
35
+ Learn how to train and run it in the [LeRobot molmoact2 guide](https://huggingface.co/docs/lerobot/main/en/molmoact2), or browse the [full documentation](https://huggingface.co/docs/lerobot/index).
36
+
37
+
38
+ ---
39
+
40
+ ## Model Details
41
+
42
+ - **License:** apache-2.0
43
+ - **Robot type:** `so_follower`
44
+ - **Cameras:** `front`, `wrist`
45
+
46
+
47
+ ## Inputs & Outputs
48
+
49
+ The policy consumes these observation features and produces these action features.
50
+
51
+ **Inputs**
52
+
53
+ | Feature | Type | Shape |
54
+ | --- | --- | --- |
55
+ | `observation.state` | STATE | `(6,)` |
56
+ | `observation.images.front` | VISUAL | `(3, 480, 640)` |
57
+ | `observation.images.wrist` | VISUAL | `(3, 480, 640)` |
58
+
59
+ **Outputs**
60
+
61
+ | Feature | Type | Shape |
62
+ | --- | --- | --- |
63
+ | `action` | ACTION | `(6,)` |
64
+
65
+
66
+ ## Training Dataset
67
+
68
+ - **Repository:** [Jiamo0912/so101-stack-cubes](https://huggingface.co/datasets/Jiamo0912/so101-stack-cubes)
69
+ - **Episodes:** 100
70
+ - **Frames:** 41359
71
+ - **Frame rate:** 30 FPS
72
+ - **Task(s):** "Stack the blue cube on the red cube, then stack the white cube on the blue cube."
73
+
74
+ <a class="flex" href="https://huggingface.co/spaces/lerobot/visualize_dataset?path=Jiamo0912/so101-stack-cubes">
75
+ <img class="block dark:hidden" src="https://huggingface.co/datasets/huggingface/badges/resolve/main/visualize-this-dataset-xl.svg"/>
76
+ <img class="hidden dark:block" src="https://huggingface.co/datasets/huggingface/badges/resolve/main/visualize-this-dataset-xl-dark.svg"/>
77
+ </a>
78
+
79
+
80
+ ## Training Configuration
81
+
82
+ | Setting | Value |
83
+ | --- | --- |
84
+ | Training steps | 40000 |
85
+ | Batch size | 16 |
86
+ | Optimizer | adamw |
87
+ | Learning rate | 1e-05 |
88
+ | Seed | 1000 |
89
+ | LeRobot version | 0.6.1 |
90
+
91
+ ---
92
+
93
+ ## How to Get Started with the Model
94
+
95
+ New to LeRobot? These guides cover the full workflow:
96
+
97
+ - **[Install LeRobot](https://huggingface.co/docs/lerobot/main/en/installation)** — set up the `lerobot` package.
98
+ - **[Hardware setup](https://huggingface.co/docs/lerobot/main/en/hardware_guide)** — assemble, wire, and calibrate your robot and cameras.
99
+ - **[Record data & train a policy](https://huggingface.co/docs/lerobot/en/il_robots)** — the end-to-end imitation-learning walkthrough.
100
+ - **[CLI cheat-sheet](https://huggingface.co/docs/lerobot/main/en/cheat-sheet)** — quick reference for the `lerobot-*` commands.
101
+
102
+ The short version to run and train this policy:
103
+
104
+ ### Run the policy on your robot
105
+
106
+ ```bash
107
+ lerobot-rollout \
108
+ --strategy.type=base \
109
+ --robot.type=so_follower \
110
+ --robot.port=<your_robot_port> \
111
+ --robot.cameras="{ <camera_1>: {type: opencv, index_or_path: <index_or_path>, width: 640, height: 480, fps: 30}, <camera_2>: {type: opencv, index_or_path: <index_or_path>, width: 640, height: 480, fps: 30}}" \
112
+ --policy.path=tsangb34/molmoact2-so101-stack-cubes-40k \
113
+ --task="Stack the blue cube on the red cube, then stack the white cube on the blue cube." \
114
+ --duration=60
115
+ ```
116
+
117
+ Replace the remaining `<...>` placeholders with your own values: `--robot.port` and the camera names/indices are specific to your machine, and the camera names must match the observation keys this policy was trained on.
118
+
119
+ When `--strategy.type=base` is used the script doesn't record the episodes. Skipping duration will make the policy run indefinitely. For more information look at [rollout documentation](https://huggingface.co/docs/lerobot/main/en/inference).
120
+
121
+ ### Train your own policy
122
+
123
+ ```bash
124
+ lerobot-train \
125
+ --dataset.repo_id=${HF_USER}/<dataset> \
126
+ --policy.type=molmoact2 \
127
+ --output_dir=outputs/train/<policy_repo_id> \
128
+ --job_name=lerobot_training \
129
+ --policy.device=cuda \
130
+ --policy.repo_id=${HF_USER}/<policy_repo_id> \
131
+ --wandb.enable=true
132
+ ```
133
+
134
+ _Writes checkpoints to `outputs/train/<policy_repo_id>/checkpoints/`._
135
+
136
+ ---
137
+
138
+ ## Evaluation
139
+
140
+ <!-- Report real-robot results here: run the policy several times per task and count the
141
+ successes. Delete the "No evaluation results" line and fill in this table instead:
142
+
143
+ | Task | Trials | Successes | Success rate |
144
+ | ---- | ------ | --------- | ------------ |
145
+ | pick the lego brick | 10 | 8 | 80% |
146
+
147
+ Also worth noting: anything that affects difficulty (new object positions, lighting,
148
+ distractors, a different robot of the same type, ...).
149
+ -->
150
+
151
+ _No evaluation results have been provided for this policy yet._
152
+
153
+ ---
154
+
155
+ ## Citation
156
+
157
+ If you use this policy, please cite the method linked in the description above, along with LeRobot:
158
+
159
+ ```bibtex
160
+ @misc{cadene2024lerobot,
161
+ author = {Cadene, Remi and Alibert, Simon and Soare, Alexander and Gallouedec, Quentin and Zouitine, Adil and Palma, Steven and Kooijmans, Pepijn and Aractingi, Michel and Shukor, Mustafa and Aubakirova, Dana and Russi, Martino and Capuano, Francesco and Pascal, Caroline and Choghari, Jade and Moss, Jess and Wolf, Thomas},
162
+ title = {LeRobot: State-of-the-art Machine Learning for Real-World Robotics in Pytorch},
163
+ howpublished = "\url{https://github.com/huggingface/lerobot}",
164
+ year = {2024}
165
+ }
166
+ ```
checkpoints/020000/pretrained_model/config.json ADDED
@@ -0,0 +1,139 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "type": "molmoact2",
3
+ "n_obs_steps": 1,
4
+ "input_features": {
5
+ "observation.state": {
6
+ "type": "STATE",
7
+ "shape": [
8
+ 6
9
+ ]
10
+ },
11
+ "observation.images.front": {
12
+ "type": "VISUAL",
13
+ "shape": [
14
+ 3,
15
+ 480,
16
+ 640
17
+ ]
18
+ },
19
+ "observation.images.wrist": {
20
+ "type": "VISUAL",
21
+ "shape": [
22
+ 3,
23
+ 480,
24
+ 640
25
+ ]
26
+ }
27
+ },
28
+ "output_features": {
29
+ "action": {
30
+ "type": "ACTION",
31
+ "shape": [
32
+ 6
33
+ ]
34
+ }
35
+ },
36
+ "device": "cuda",
37
+ "use_amp": false,
38
+ "use_peft": false,
39
+ "push_to_hub": true,
40
+ "repo_id": "tsangb34/molmoact2-so101-stack-cubes-40k",
41
+ "private": false,
42
+ "tags": [
43
+ "molmoact2",
44
+ "so101",
45
+ "robocolosseum",
46
+ "stack-cubes"
47
+ ],
48
+ "license": null,
49
+ "pretrained_path": null,
50
+ "pretrained_revision": null,
51
+ "checkpoint_path": "allenai/MolmoAct2",
52
+ "checkpoint_revision": null,
53
+ "checkpoint_force_download": false,
54
+ "chunk_size": 32,
55
+ "n_action_steps": 32,
56
+ "action_mode": "continuous",
57
+ "inference_action_mode": "continuous",
58
+ "discrete_action_tokenizer": "allenai/MolmoAct2-FAST-Tokenizer",
59
+ "discrete_generation_max_steps": null,
60
+ "norm_tag": null,
61
+ "setup_type": "single so100/so101 robotic arm in molmoact2",
62
+ "control_mode": "absolute joint pose",
63
+ "image_keys": [
64
+ "observation.images.front",
65
+ "observation.images.wrist"
66
+ ],
67
+ "normalize_language": true,
68
+ "add_setup_tokens": true,
69
+ "add_control_tokens": true,
70
+ "normalize_gripper": true,
71
+ "num_state_tokens": 256,
72
+ "max_sequence_length": null,
73
+ "expected_max_action_dim": 32,
74
+ "num_flow_timesteps": 8,
75
+ "flow_matching_cutoff": 1.0,
76
+ "flow_matching_time_offset": 0.001,
77
+ "flow_matching_time_scale": 0.999,
78
+ "flow_matching_beta_alpha": 1.0,
79
+ "flow_matching_beta_beta": 1.5,
80
+ "num_inference_steps": null,
81
+ "mask_action_dim_padding": true,
82
+ "enable_inference_cuda_graph": true,
83
+ "per_episode_seed": false,
84
+ "eval_seed": null,
85
+ "rtc_config": null,
86
+ "joint_signs": null,
87
+ "joint_offsets": null,
88
+ "enable_lora_vlm": false,
89
+ "lora_rank": 64,
90
+ "lora_alpha": 16,
91
+ "lora_dropout": 0.05,
92
+ "lora_bias": "none",
93
+ "enable_lora_action_expert": false,
94
+ "enable_knowledge_insulation": false,
95
+ "freeze_embedding": true,
96
+ "train_action_expert_only": false,
97
+ "gradient_checkpointing": true,
98
+ "model_dtype": "bfloat16",
99
+ "softmax_auxiliary_loss": true,
100
+ "softmax_auxiliary_loss_scale": 0.0001,
101
+ "discrete_loss_token_weighting": "root_subsegments_root_tokens",
102
+ "optimizer_lr": 1e-05,
103
+ "optimizer_vit_lr": 5e-06,
104
+ "optimizer_connector_lr": 5e-06,
105
+ "optimizer_action_expert_lr": 5e-05,
106
+ "optimizer_betas": [
107
+ 0.9,
108
+ 0.95
109
+ ],
110
+ "optimizer_eps": 1e-06,
111
+ "optimizer_weight_decay": 0.0,
112
+ "optimizer_grad_clip_norm": 1.0,
113
+ "scheduler_warmup_steps": 200,
114
+ "scheduler_decay_steps": 40000,
115
+ "scheduler_decay_lr": 1e-06,
116
+ "normalization_mapping": {
117
+ "ACTION": "QUANTILES",
118
+ "STATE": "QUANTILES",
119
+ "VISUAL": "IDENTITY"
120
+ },
121
+ "dataset_feature_names": {
122
+ "action": [
123
+ "shoulder_pan.pos",
124
+ "shoulder_lift.pos",
125
+ "elbow_flex.pos",
126
+ "wrist_flex.pos",
127
+ "wrist_roll.pos",
128
+ "gripper.pos"
129
+ ],
130
+ "observation.state": [
131
+ "shoulder_pan.pos",
132
+ "shoulder_lift.pos",
133
+ "elbow_flex.pos",
134
+ "wrist_flex.pos",
135
+ "wrist_roll.pos",
136
+ "gripper.pos"
137
+ ]
138
+ }
139
+ }
checkpoints/020000/pretrained_model/model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0a99929beea64a503b49f35bb293c5f63527b100498518737558ecab9ba1cbd5
3
+ size 10884573720
checkpoints/020000/pretrained_model/policy_postprocessor.json ADDED
@@ -0,0 +1,43 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "name": "policy_postprocessor",
3
+ "steps": [
4
+ {
5
+ "registry_name": "molmoact2_clamp_action",
6
+ "config": {}
7
+ },
8
+ {
9
+ "registry_name": "molmoact2_masked_unnormalizer",
10
+ "config": {
11
+ "eps": 1e-08,
12
+ "features": {
13
+ "action": {
14
+ "type": "ACTION",
15
+ "shape": [
16
+ 6
17
+ ]
18
+ }
19
+ },
20
+ "norm_map": {
21
+ "ACTION": "QUANTILES",
22
+ "STATE": "QUANTILES",
23
+ "VISUAL": "IDENTITY"
24
+ }
25
+ },
26
+ "state_file": "policy_postprocessor_step_1_molmoact2_masked_unnormalizer.safetensors"
27
+ },
28
+ {
29
+ "registry_name": "molmoact2_action_frame_transform",
30
+ "config": {
31
+ "joint_signs": null,
32
+ "joint_offsets": null
33
+ }
34
+ },
35
+ {
36
+ "registry_name": "device_processor",
37
+ "config": {
38
+ "device": "cpu",
39
+ "float_dtype": null
40
+ }
41
+ }
42
+ ]
43
+ }
checkpoints/020000/pretrained_model/policy_postprocessor_step_1_molmoact2_masked_unnormalizer.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ee6a8cf8d7533d272953fddc06a419c5b7c89cac01d3dba8db5b3cee5efb107f
3
+ size 7744
checkpoints/020000/pretrained_model/policy_preprocessor.json ADDED
@@ -0,0 +1,100 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "name": "policy_preprocessor",
3
+ "steps": [
4
+ {
5
+ "registry_name": "rename_observations_processor",
6
+ "config": {
7
+ "rename_map": {}
8
+ }
9
+ },
10
+ {
11
+ "registry_name": "to_batch_processor",
12
+ "config": {}
13
+ },
14
+ {
15
+ "registry_name": "molmoact2_state_frame_transform",
16
+ "config": {
17
+ "joint_signs": null,
18
+ "joint_offsets": null
19
+ }
20
+ },
21
+ {
22
+ "registry_name": "molmoact2_masked_normalizer",
23
+ "config": {
24
+ "eps": 1e-08,
25
+ "features": {
26
+ "observation.state": {
27
+ "type": "STATE",
28
+ "shape": [
29
+ 6
30
+ ]
31
+ },
32
+ "observation.images.front": {
33
+ "type": "VISUAL",
34
+ "shape": [
35
+ 3,
36
+ 480,
37
+ 640
38
+ ]
39
+ },
40
+ "observation.images.wrist": {
41
+ "type": "VISUAL",
42
+ "shape": [
43
+ 3,
44
+ 480,
45
+ 640
46
+ ]
47
+ },
48
+ "action": {
49
+ "type": "ACTION",
50
+ "shape": [
51
+ 6
52
+ ]
53
+ }
54
+ },
55
+ "norm_map": {
56
+ "ACTION": "QUANTILES",
57
+ "STATE": "QUANTILES",
58
+ "VISUAL": "IDENTITY"
59
+ }
60
+ },
61
+ "state_file": "policy_preprocessor_step_3_molmoact2_masked_normalizer.safetensors"
62
+ },
63
+ {
64
+ "registry_name": "molmoact2_clamp_normalized",
65
+ "config": {}
66
+ },
67
+ {
68
+ "registry_name": "molmoact2_pack_inputs",
69
+ "config": {
70
+ "checkpoint_path": "allenai/MolmoAct2",
71
+ "checkpoint_revision": null,
72
+ "checkpoint_force_download": false,
73
+ "action_mode": "continuous",
74
+ "discrete_action_tokenizer": "allenai/MolmoAct2-FAST-Tokenizer",
75
+ "image_keys": [
76
+ "observation.images.front",
77
+ "observation.images.wrist"
78
+ ],
79
+ "allow_image_key_fallback": false,
80
+ "setup_type": "single so100/so101 robotic arm in molmoact2",
81
+ "control_mode": "absolute joint pose",
82
+ "normalize_language": true,
83
+ "add_setup_tokens": true,
84
+ "add_control_tokens": true,
85
+ "num_state_tokens": 256,
86
+ "max_sequence_length": null,
87
+ "chunk_size": 32,
88
+ "max_action_dim": 32,
89
+ "env_action_dim": 6
90
+ }
91
+ },
92
+ {
93
+ "registry_name": "device_processor",
94
+ "config": {
95
+ "device": "cuda",
96
+ "float_dtype": null
97
+ }
98
+ }
99
+ ]
100
+ }
checkpoints/020000/pretrained_model/policy_preprocessor_step_3_molmoact2_masked_normalizer.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:95f69bcc0e1bee273f9668382626418fd89e92d69d6e42aa99c4a458f9a06c31
3
+ size 7752
checkpoints/020000/pretrained_model/train_config.json ADDED
@@ -0,0 +1,275 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "dataset": {
3
+ "repo_id": "Jiamo0912/so101-stack-cubes",
4
+ "repo_type": "dataset",
5
+ "root": "/scratch/e1583535/datasets/so101-stack-cubes",
6
+ "episodes": null,
7
+ "image_transforms": {
8
+ "enable": true,
9
+ "max_num_transforms": 3,
10
+ "random_order": false,
11
+ "tfs": {
12
+ "crop": {
13
+ "weight": 1.0,
14
+ "type": "RandomResizedCrop",
15
+ "kwargs": {
16
+ "size": [
17
+ 480,
18
+ 640
19
+ ],
20
+ "scale": [
21
+ 0.9025,
22
+ 0.9025
23
+ ],
24
+ "ratio": [
25
+ 1.3333333,
26
+ 1.3333333
27
+ ]
28
+ }
29
+ },
30
+ "rotation": {
31
+ "weight": 1.0,
32
+ "type": "RandomRotation",
33
+ "kwargs": {
34
+ "degrees": [
35
+ -5.0,
36
+ 5.0
37
+ ],
38
+ "interpolation": 2
39
+ }
40
+ },
41
+ "color": {
42
+ "weight": 1.0,
43
+ "type": "ColorJitter",
44
+ "kwargs": {
45
+ "brightness": 0.2,
46
+ "contrast": [
47
+ 0.8,
48
+ 1.2
49
+ ],
50
+ "saturation": [
51
+ 0.8,
52
+ 1.2
53
+ ],
54
+ "hue": 0.05
55
+ }
56
+ }
57
+ }
58
+ },
59
+ "revision": null,
60
+ "use_imagenet_stats": true,
61
+ "video_backend": "pyav",
62
+ "return_uint8": false,
63
+ "depth_output_unit": "mm",
64
+ "streaming": false,
65
+ "eval_split": 0.0
66
+ },
67
+ "env": null,
68
+ "policy": {
69
+ "type": "molmoact2",
70
+ "n_obs_steps": 1,
71
+ "input_features": {
72
+ "observation.state": {
73
+ "type": "STATE",
74
+ "shape": [
75
+ 6
76
+ ]
77
+ },
78
+ "observation.images.front": {
79
+ "type": "VISUAL",
80
+ "shape": [
81
+ 3,
82
+ 480,
83
+ 640
84
+ ]
85
+ },
86
+ "observation.images.wrist": {
87
+ "type": "VISUAL",
88
+ "shape": [
89
+ 3,
90
+ 480,
91
+ 640
92
+ ]
93
+ }
94
+ },
95
+ "output_features": {
96
+ "action": {
97
+ "type": "ACTION",
98
+ "shape": [
99
+ 6
100
+ ]
101
+ }
102
+ },
103
+ "device": "cuda",
104
+ "use_amp": false,
105
+ "use_peft": false,
106
+ "push_to_hub": true,
107
+ "repo_id": "tsangb34/molmoact2-so101-stack-cubes-40k",
108
+ "private": false,
109
+ "tags": [
110
+ "molmoact2",
111
+ "so101",
112
+ "robocolosseum",
113
+ "stack-cubes"
114
+ ],
115
+ "license": null,
116
+ "pretrained_path": null,
117
+ "pretrained_revision": null,
118
+ "checkpoint_path": "allenai/MolmoAct2",
119
+ "checkpoint_revision": null,
120
+ "checkpoint_force_download": false,
121
+ "chunk_size": 32,
122
+ "n_action_steps": 32,
123
+ "action_mode": "continuous",
124
+ "inference_action_mode": "continuous",
125
+ "discrete_action_tokenizer": "allenai/MolmoAct2-FAST-Tokenizer",
126
+ "discrete_generation_max_steps": null,
127
+ "norm_tag": null,
128
+ "setup_type": "single so100/so101 robotic arm in molmoact2",
129
+ "control_mode": "absolute joint pose",
130
+ "image_keys": [
131
+ "observation.images.front",
132
+ "observation.images.wrist"
133
+ ],
134
+ "normalize_language": true,
135
+ "add_setup_tokens": true,
136
+ "add_control_tokens": true,
137
+ "normalize_gripper": true,
138
+ "num_state_tokens": 256,
139
+ "max_sequence_length": null,
140
+ "expected_max_action_dim": 32,
141
+ "num_flow_timesteps": 8,
142
+ "flow_matching_cutoff": 1.0,
143
+ "flow_matching_time_offset": 0.001,
144
+ "flow_matching_time_scale": 0.999,
145
+ "flow_matching_beta_alpha": 1.0,
146
+ "flow_matching_beta_beta": 1.5,
147
+ "num_inference_steps": null,
148
+ "mask_action_dim_padding": true,
149
+ "enable_inference_cuda_graph": true,
150
+ "per_episode_seed": false,
151
+ "eval_seed": null,
152
+ "rtc_config": null,
153
+ "joint_signs": null,
154
+ "joint_offsets": null,
155
+ "enable_lora_vlm": false,
156
+ "lora_rank": 64,
157
+ "lora_alpha": 16,
158
+ "lora_dropout": 0.05,
159
+ "lora_bias": "none",
160
+ "enable_lora_action_expert": false,
161
+ "enable_knowledge_insulation": false,
162
+ "freeze_embedding": true,
163
+ "train_action_expert_only": false,
164
+ "gradient_checkpointing": true,
165
+ "model_dtype": "bfloat16",
166
+ "softmax_auxiliary_loss": true,
167
+ "softmax_auxiliary_loss_scale": 0.0001,
168
+ "discrete_loss_token_weighting": "root_subsegments_root_tokens",
169
+ "optimizer_lr": 1e-05,
170
+ "optimizer_vit_lr": 5e-06,
171
+ "optimizer_connector_lr": 5e-06,
172
+ "optimizer_action_expert_lr": 5e-05,
173
+ "optimizer_betas": [
174
+ 0.9,
175
+ 0.95
176
+ ],
177
+ "optimizer_eps": 1e-06,
178
+ "optimizer_weight_decay": 0.0,
179
+ "optimizer_grad_clip_norm": 1.0,
180
+ "scheduler_warmup_steps": 200,
181
+ "scheduler_decay_steps": 40000,
182
+ "scheduler_decay_lr": 1e-06,
183
+ "normalization_mapping": {
184
+ "ACTION": "QUANTILES",
185
+ "STATE": "QUANTILES",
186
+ "VISUAL": "IDENTITY"
187
+ },
188
+ "dataset_feature_names": {
189
+ "action": [
190
+ "shoulder_pan.pos",
191
+ "shoulder_lift.pos",
192
+ "elbow_flex.pos",
193
+ "wrist_flex.pos",
194
+ "wrist_roll.pos",
195
+ "gripper.pos"
196
+ ],
197
+ "observation.state": [
198
+ "shoulder_pan.pos",
199
+ "shoulder_lift.pos",
200
+ "elbow_flex.pos",
201
+ "wrist_flex.pos",
202
+ "wrist_roll.pos",
203
+ "gripper.pos"
204
+ ]
205
+ }
206
+ },
207
+ "reward_model": null,
208
+ "output_dir": "/scratch/Projects/CFP-05/CFP05-CF-002/robocolosseum-finetuned-models/molmoact2-so101-stack-cubes-40k",
209
+ "job_name": "molmoact2-so101-stack-cubes-40k",
210
+ "resume": false,
211
+ "seed": 1000,
212
+ "cudnn_deterministic": false,
213
+ "num_workers": 4,
214
+ "batch_size": 16,
215
+ "prefetch_factor": 4,
216
+ "persistent_workers": true,
217
+ "dataloader_multiprocessing_context": "spawn",
218
+ "steps": 40000,
219
+ "env_eval_freq": 20000,
220
+ "log_freq": 20,
221
+ "eval_steps": 0,
222
+ "max_eval_samples": 0,
223
+ "tolerance_s": 0.0001,
224
+ "save_checkpoint": true,
225
+ "save_freq": 20000,
226
+ "use_policy_training_preset": true,
227
+ "optimizer": {
228
+ "type": "adamw",
229
+ "lr": 1e-05,
230
+ "weight_decay": 0.0,
231
+ "grad_clip_norm": 1.0,
232
+ "betas": [
233
+ 0.9,
234
+ 0.95
235
+ ],
236
+ "eps": 1e-06
237
+ },
238
+ "scheduler": {
239
+ "type": "cosine_decay_with_warmup",
240
+ "num_warmup_steps": 200,
241
+ "num_decay_steps": 40000,
242
+ "peak_lr": 1e-05,
243
+ "decay_lr": 1e-06
244
+ },
245
+ "eval": {
246
+ "n_episodes": 50,
247
+ "batch_size": 50,
248
+ "use_async_envs": true,
249
+ "recording": false,
250
+ "recording_repo_id": null,
251
+ "recording_private": false
252
+ },
253
+ "wandb": {
254
+ "enable": true,
255
+ "disable_artifact": true,
256
+ "project": "RoboColosseum",
257
+ "entity": "tsangb34-national-university-of-singapore-students-union",
258
+ "notes": null,
259
+ "run_id": "molmoact2-so101-stack-cubes-40k",
260
+ "mode": null,
261
+ "add_tags": true
262
+ },
263
+ "peft": null,
264
+ "job": {
265
+ "target": null,
266
+ "image": "huggingface/lerobot-gpu:latest",
267
+ "timeout": "2d",
268
+ "detach": false,
269
+ "tags": []
270
+ },
271
+ "save_checkpoint_to_hub": true,
272
+ "sample_weighting": null,
273
+ "rename_map": {},
274
+ "checkpoint_path": null
275
+ }
checkpoints/020000/training_state/optimizer_param_groups.json ADDED
@@ -0,0 +1,1370 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ [
2
+ {
3
+ "lr": 5.500000000000001e-06,
4
+ "betas": [
5
+ 0.9,
6
+ 0.95
7
+ ],
8
+ "eps": 1e-06,
9
+ "weight_decay": 0.0,
10
+ "amsgrad": false,
11
+ "maximize": false,
12
+ "foreach": null,
13
+ "capturable": false,
14
+ "differentiable": false,
15
+ "fused": null,
16
+ "decoupled_weight_decay": true,
17
+ "initial_lr": 1e-05,
18
+ "params": [
19
+ 0,
20
+ 1,
21
+ 2,
22
+ 3,
23
+ 4,
24
+ 5,
25
+ 6,
26
+ 7,
27
+ 8,
28
+ 9,
29
+ 10,
30
+ 11,
31
+ 12,
32
+ 13,
33
+ 14,
34
+ 15,
35
+ 16,
36
+ 17,
37
+ 18,
38
+ 19,
39
+ 20,
40
+ 21,
41
+ 22,
42
+ 23,
43
+ 24,
44
+ 25,
45
+ 26,
46
+ 27,
47
+ 28,
48
+ 29,
49
+ 30,
50
+ 31,
51
+ 32,
52
+ 33,
53
+ 34,
54
+ 35,
55
+ 36,
56
+ 37,
57
+ 38,
58
+ 39,
59
+ 40,
60
+ 41,
61
+ 42,
62
+ 43,
63
+ 44,
64
+ 45,
65
+ 46,
66
+ 47,
67
+ 48,
68
+ 49,
69
+ 50,
70
+ 51,
71
+ 52,
72
+ 53,
73
+ 54,
74
+ 55,
75
+ 56,
76
+ 57,
77
+ 58,
78
+ 59,
79
+ 60,
80
+ 61,
81
+ 62,
82
+ 63,
83
+ 64,
84
+ 65,
85
+ 66,
86
+ 67,
87
+ 68,
88
+ 69,
89
+ 70,
90
+ 71,
91
+ 72,
92
+ 73,
93
+ 74,
94
+ 75,
95
+ 76,
96
+ 77,
97
+ 78,
98
+ 79,
99
+ 80,
100
+ 81,
101
+ 82,
102
+ 83,
103
+ 84,
104
+ 85,
105
+ 86,
106
+ 87,
107
+ 88,
108
+ 89,
109
+ 90,
110
+ 91,
111
+ 92,
112
+ 93,
113
+ 94,
114
+ 95,
115
+ 96,
116
+ 97,
117
+ 98,
118
+ 99,
119
+ 100,
120
+ 101,
121
+ 102,
122
+ 103,
123
+ 104,
124
+ 105,
125
+ 106,
126
+ 107,
127
+ 108,
128
+ 109,
129
+ 110,
130
+ 111,
131
+ 112,
132
+ 113,
133
+ 114,
134
+ 115,
135
+ 116,
136
+ 117,
137
+ 118,
138
+ 119,
139
+ 120,
140
+ 121,
141
+ 122,
142
+ 123,
143
+ 124,
144
+ 125,
145
+ 126,
146
+ 127,
147
+ 128,
148
+ 129,
149
+ 130,
150
+ 131,
151
+ 132,
152
+ 133,
153
+ 134,
154
+ 135,
155
+ 136,
156
+ 137,
157
+ 138,
158
+ 139,
159
+ 140,
160
+ 141,
161
+ 142,
162
+ 143,
163
+ 144,
164
+ 145,
165
+ 146,
166
+ 147,
167
+ 148,
168
+ 149,
169
+ 150,
170
+ 151,
171
+ 152,
172
+ 153,
173
+ 154,
174
+ 155,
175
+ 156,
176
+ 157,
177
+ 158,
178
+ 159,
179
+ 160,
180
+ 161,
181
+ 162,
182
+ 163,
183
+ 164,
184
+ 165,
185
+ 166,
186
+ 167,
187
+ 168,
188
+ 169,
189
+ 170,
190
+ 171,
191
+ 172,
192
+ 173,
193
+ 174,
194
+ 175,
195
+ 176,
196
+ 177,
197
+ 178,
198
+ 179,
199
+ 180,
200
+ 181,
201
+ 182,
202
+ 183,
203
+ 184,
204
+ 185,
205
+ 186,
206
+ 187,
207
+ 188,
208
+ 189,
209
+ 190,
210
+ 191,
211
+ 192,
212
+ 193,
213
+ 194,
214
+ 195,
215
+ 196,
216
+ 197,
217
+ 198,
218
+ 199,
219
+ 200,
220
+ 201,
221
+ 202,
222
+ 203,
223
+ 204,
224
+ 205,
225
+ 206,
226
+ 207,
227
+ 208,
228
+ 209,
229
+ 210,
230
+ 211,
231
+ 212,
232
+ 213,
233
+ 214,
234
+ 215,
235
+ 216,
236
+ 217,
237
+ 218,
238
+ 219,
239
+ 220,
240
+ 221,
241
+ 222,
242
+ 223,
243
+ 224,
244
+ 225,
245
+ 226,
246
+ 227,
247
+ 228,
248
+ 229,
249
+ 230,
250
+ 231,
251
+ 232,
252
+ 233,
253
+ 234,
254
+ 235,
255
+ 236,
256
+ 237,
257
+ 238,
258
+ 239,
259
+ 240,
260
+ 241,
261
+ 242,
262
+ 243,
263
+ 244,
264
+ 245,
265
+ 246,
266
+ 247,
267
+ 248,
268
+ 249,
269
+ 250,
270
+ 251,
271
+ 252,
272
+ 253,
273
+ 254,
274
+ 255,
275
+ 256,
276
+ 257,
277
+ 258,
278
+ 259,
279
+ 260,
280
+ 261,
281
+ 262,
282
+ 263,
283
+ 264,
284
+ 265,
285
+ 266,
286
+ 267,
287
+ 268,
288
+ 269,
289
+ 270,
290
+ 271,
291
+ 272,
292
+ 273,
293
+ 274,
294
+ 275,
295
+ 276,
296
+ 277,
297
+ 278,
298
+ 279,
299
+ 280,
300
+ 281,
301
+ 282,
302
+ 283,
303
+ 284,
304
+ 285,
305
+ 286,
306
+ 287,
307
+ 288,
308
+ 289
309
+ ]
310
+ },
311
+ {
312
+ "lr": 2.7500000000000004e-06,
313
+ "betas": [
314
+ 0.9,
315
+ 0.95
316
+ ],
317
+ "eps": 1e-06,
318
+ "weight_decay": 0.0,
319
+ "amsgrad": false,
320
+ "maximize": false,
321
+ "foreach": null,
322
+ "capturable": false,
323
+ "differentiable": false,
324
+ "fused": null,
325
+ "decoupled_weight_decay": true,
326
+ "initial_lr": 5e-06,
327
+ "params": [
328
+ 290,
329
+ 291,
330
+ 292,
331
+ 293,
332
+ 294,
333
+ 295,
334
+ 296,
335
+ 297,
336
+ 298,
337
+ 299,
338
+ 300,
339
+ 301,
340
+ 302,
341
+ 303,
342
+ 304,
343
+ 305,
344
+ 306,
345
+ 307,
346
+ 308,
347
+ 309,
348
+ 310,
349
+ 311,
350
+ 312,
351
+ 313,
352
+ 314,
353
+ 315,
354
+ 316,
355
+ 317,
356
+ 318,
357
+ 319,
358
+ 320,
359
+ 321,
360
+ 322,
361
+ 323,
362
+ 324,
363
+ 325,
364
+ 326,
365
+ 327,
366
+ 328,
367
+ 329,
368
+ 330,
369
+ 331,
370
+ 332,
371
+ 333,
372
+ 334,
373
+ 335,
374
+ 336,
375
+ 337,
376
+ 338,
377
+ 339,
378
+ 340,
379
+ 341,
380
+ 342,
381
+ 343,
382
+ 344,
383
+ 345,
384
+ 346,
385
+ 347,
386
+ 348,
387
+ 349,
388
+ 350,
389
+ 351,
390
+ 352,
391
+ 353,
392
+ 354,
393
+ 355,
394
+ 356,
395
+ 357,
396
+ 358,
397
+ 359,
398
+ 360,
399
+ 361,
400
+ 362,
401
+ 363,
402
+ 364,
403
+ 365,
404
+ 366,
405
+ 367,
406
+ 368,
407
+ 369,
408
+ 370,
409
+ 371,
410
+ 372,
411
+ 373,
412
+ 374,
413
+ 375,
414
+ 376,
415
+ 377,
416
+ 378,
417
+ 379,
418
+ 380,
419
+ 381,
420
+ 382,
421
+ 383,
422
+ 384,
423
+ 385,
424
+ 386,
425
+ 387,
426
+ 388,
427
+ 389,
428
+ 390,
429
+ 391,
430
+ 392,
431
+ 393,
432
+ 394,
433
+ 395,
434
+ 396,
435
+ 397,
436
+ 398,
437
+ 399,
438
+ 400,
439
+ 401,
440
+ 402,
441
+ 403,
442
+ 404,
443
+ 405,
444
+ 406,
445
+ 407,
446
+ 408,
447
+ 409,
448
+ 410,
449
+ 411,
450
+ 412,
451
+ 413,
452
+ 414,
453
+ 415,
454
+ 416,
455
+ 417,
456
+ 418,
457
+ 419,
458
+ 420,
459
+ 421,
460
+ 422,
461
+ 423,
462
+ 424,
463
+ 425,
464
+ 426,
465
+ 427,
466
+ 428,
467
+ 429,
468
+ 430,
469
+ 431,
470
+ 432,
471
+ 433,
472
+ 434,
473
+ 435,
474
+ 436,
475
+ 437,
476
+ 438,
477
+ 439,
478
+ 440,
479
+ 441,
480
+ 442,
481
+ 443,
482
+ 444,
483
+ 445,
484
+ 446,
485
+ 447,
486
+ 448,
487
+ 449,
488
+ 450,
489
+ 451,
490
+ 452,
491
+ 453,
492
+ 454,
493
+ 455,
494
+ 456,
495
+ 457,
496
+ 458,
497
+ 459,
498
+ 460,
499
+ 461,
500
+ 462,
501
+ 463,
502
+ 464,
503
+ 465,
504
+ 466,
505
+ 467,
506
+ 468,
507
+ 469,
508
+ 470,
509
+ 471,
510
+ 472,
511
+ 473,
512
+ 474,
513
+ 475,
514
+ 476,
515
+ 477,
516
+ 478,
517
+ 479,
518
+ 480,
519
+ 481,
520
+ 482,
521
+ 483,
522
+ 484,
523
+ 485,
524
+ 486,
525
+ 487,
526
+ 488,
527
+ 489,
528
+ 490,
529
+ 491,
530
+ 492,
531
+ 493,
532
+ 494,
533
+ 495,
534
+ 496,
535
+ 497,
536
+ 498,
537
+ 499,
538
+ 500,
539
+ 501,
540
+ 502,
541
+ 503,
542
+ 504,
543
+ 505,
544
+ 506,
545
+ 507,
546
+ 508,
547
+ 509,
548
+ 510,
549
+ 511,
550
+ 512,
551
+ 513,
552
+ 514,
553
+ 515,
554
+ 516,
555
+ 517,
556
+ 518,
557
+ 519,
558
+ 520,
559
+ 521,
560
+ 522,
561
+ 523,
562
+ 524,
563
+ 525,
564
+ 526,
565
+ 527,
566
+ 528,
567
+ 529,
568
+ 530,
569
+ 531,
570
+ 532,
571
+ 533,
572
+ 534,
573
+ 535,
574
+ 536,
575
+ 537,
576
+ 538,
577
+ 539,
578
+ 540,
579
+ 541,
580
+ 542,
581
+ 543,
582
+ 544,
583
+ 545,
584
+ 546,
585
+ 547,
586
+ 548,
587
+ 549,
588
+ 550,
589
+ 551,
590
+ 552,
591
+ 553,
592
+ 554,
593
+ 555,
594
+ 556,
595
+ 557,
596
+ 558,
597
+ 559,
598
+ 560,
599
+ 561,
600
+ 562,
601
+ 563,
602
+ 564,
603
+ 565,
604
+ 566,
605
+ 567,
606
+ 568,
607
+ 569,
608
+ 570,
609
+ 571,
610
+ 572,
611
+ 573,
612
+ 574,
613
+ 575,
614
+ 576,
615
+ 577,
616
+ 578,
617
+ 579,
618
+ 580,
619
+ 581,
620
+ 582,
621
+ 583,
622
+ 584,
623
+ 585,
624
+ 586,
625
+ 587,
626
+ 588,
627
+ 589,
628
+ 590,
629
+ 591,
630
+ 592,
631
+ 593,
632
+ 594,
633
+ 595,
634
+ 596,
635
+ 597,
636
+ 598,
637
+ 599,
638
+ 600,
639
+ 601,
640
+ 602,
641
+ 603,
642
+ 604,
643
+ 605,
644
+ 606,
645
+ 607,
646
+ 608,
647
+ 609,
648
+ 610,
649
+ 611,
650
+ 612,
651
+ 613,
652
+ 614,
653
+ 615,
654
+ 616,
655
+ 617,
656
+ 618,
657
+ 619,
658
+ 620,
659
+ 621,
660
+ 622,
661
+ 623,
662
+ 624,
663
+ 625,
664
+ 626,
665
+ 627,
666
+ 628,
667
+ 629,
668
+ 630,
669
+ 631,
670
+ 632,
671
+ 633,
672
+ 634,
673
+ 635,
674
+ 636,
675
+ 637,
676
+ 638,
677
+ 639,
678
+ 640,
679
+ 641,
680
+ 642,
681
+ 643,
682
+ 644,
683
+ 645,
684
+ 646,
685
+ 647,
686
+ 648,
687
+ 649,
688
+ 650,
689
+ 651,
690
+ 652,
691
+ 653,
692
+ 654,
693
+ 655,
694
+ 656,
695
+ 657,
696
+ 658,
697
+ 659,
698
+ 660,
699
+ 661,
700
+ 662,
701
+ 663,
702
+ 664,
703
+ 665,
704
+ 666,
705
+ 667,
706
+ 668,
707
+ 669,
708
+ 670,
709
+ 671,
710
+ 672,
711
+ 673,
712
+ 674,
713
+ 675,
714
+ 676,
715
+ 677,
716
+ 678,
717
+ 679,
718
+ 680,
719
+ 681,
720
+ 682,
721
+ 683,
722
+ 684,
723
+ 685,
724
+ 686,
725
+ 687,
726
+ 688,
727
+ 689,
728
+ 690,
729
+ 691,
730
+ 692
731
+ ]
732
+ },
733
+ {
734
+ "lr": 2.7500000000000004e-06,
735
+ "betas": [
736
+ 0.9,
737
+ 0.95
738
+ ],
739
+ "eps": 1e-06,
740
+ "weight_decay": 0.0,
741
+ "amsgrad": false,
742
+ "maximize": false,
743
+ "foreach": null,
744
+ "capturable": false,
745
+ "differentiable": false,
746
+ "fused": null,
747
+ "decoupled_weight_decay": true,
748
+ "initial_lr": 5e-06,
749
+ "params": [
750
+ 693,
751
+ 694,
752
+ 695,
753
+ 696,
754
+ 697,
755
+ 698,
756
+ 699,
757
+ 700,
758
+ 701,
759
+ 702,
760
+ 703
761
+ ]
762
+ },
763
+ {
764
+ "lr": 2.7500000000000004e-05,
765
+ "betas": [
766
+ 0.9,
767
+ 0.95
768
+ ],
769
+ "eps": 1e-06,
770
+ "weight_decay": 0.0,
771
+ "amsgrad": false,
772
+ "maximize": false,
773
+ "foreach": null,
774
+ "capturable": false,
775
+ "differentiable": false,
776
+ "fused": null,
777
+ "decoupled_weight_decay": true,
778
+ "initial_lr": 5e-05,
779
+ "params": [
780
+ 704,
781
+ 705,
782
+ 706,
783
+ 707,
784
+ 708,
785
+ 709,
786
+ 710,
787
+ 711,
788
+ 712,
789
+ 713,
790
+ 714,
791
+ 715,
792
+ 716,
793
+ 717,
794
+ 718,
795
+ 719,
796
+ 720,
797
+ 721,
798
+ 722,
799
+ 723,
800
+ 724,
801
+ 725,
802
+ 726,
803
+ 727,
804
+ 728,
805
+ 729,
806
+ 730,
807
+ 731,
808
+ 732,
809
+ 733,
810
+ 734,
811
+ 735,
812
+ 736,
813
+ 737,
814
+ 738,
815
+ 739,
816
+ 740,
817
+ 741,
818
+ 742,
819
+ 743,
820
+ 744,
821
+ 745,
822
+ 746,
823
+ 747,
824
+ 748,
825
+ 749,
826
+ 750,
827
+ 751,
828
+ 752,
829
+ 753,
830
+ 754,
831
+ 755,
832
+ 756,
833
+ 757,
834
+ 758,
835
+ 759,
836
+ 760,
837
+ 761,
838
+ 762,
839
+ 763,
840
+ 764,
841
+ 765,
842
+ 766,
843
+ 767,
844
+ 768,
845
+ 769,
846
+ 770,
847
+ 771,
848
+ 772,
849
+ 773,
850
+ 774,
851
+ 775,
852
+ 776,
853
+ 777,
854
+ 778,
855
+ 779,
856
+ 780,
857
+ 781,
858
+ 782,
859
+ 783,
860
+ 784,
861
+ 785,
862
+ 786,
863
+ 787,
864
+ 788,
865
+ 789,
866
+ 790,
867
+ 791,
868
+ 792,
869
+ 793,
870
+ 794,
871
+ 795,
872
+ 796,
873
+ 797,
874
+ 798,
875
+ 799,
876
+ 800,
877
+ 801,
878
+ 802,
879
+ 803,
880
+ 804,
881
+ 805,
882
+ 806,
883
+ 807,
884
+ 808,
885
+ 809,
886
+ 810,
887
+ 811,
888
+ 812,
889
+ 813,
890
+ 814,
891
+ 815,
892
+ 816,
893
+ 817,
894
+ 818,
895
+ 819,
896
+ 820,
897
+ 821,
898
+ 822,
899
+ 823,
900
+ 824,
901
+ 825,
902
+ 826,
903
+ 827,
904
+ 828,
905
+ 829,
906
+ 830,
907
+ 831,
908
+ 832,
909
+ 833,
910
+ 834,
911
+ 835,
912
+ 836,
913
+ 837,
914
+ 838,
915
+ 839,
916
+ 840,
917
+ 841,
918
+ 842,
919
+ 843,
920
+ 844,
921
+ 845,
922
+ 846,
923
+ 847,
924
+ 848,
925
+ 849,
926
+ 850,
927
+ 851,
928
+ 852,
929
+ 853,
930
+ 854,
931
+ 855,
932
+ 856,
933
+ 857,
934
+ 858,
935
+ 859,
936
+ 860,
937
+ 861,
938
+ 862,
939
+ 863,
940
+ 864,
941
+ 865,
942
+ 866,
943
+ 867,
944
+ 868,
945
+ 869,
946
+ 870,
947
+ 871,
948
+ 872,
949
+ 873,
950
+ 874,
951
+ 875,
952
+ 876,
953
+ 877,
954
+ 878,
955
+ 879,
956
+ 880,
957
+ 881,
958
+ 882,
959
+ 883,
960
+ 884,
961
+ 885,
962
+ 886,
963
+ 887,
964
+ 888,
965
+ 889,
966
+ 890,
967
+ 891,
968
+ 892,
969
+ 893,
970
+ 894,
971
+ 895,
972
+ 896,
973
+ 897,
974
+ 898,
975
+ 899,
976
+ 900,
977
+ 901,
978
+ 902,
979
+ 903,
980
+ 904,
981
+ 905,
982
+ 906,
983
+ 907,
984
+ 908,
985
+ 909,
986
+ 910,
987
+ 911,
988
+ 912,
989
+ 913,
990
+ 914,
991
+ 915,
992
+ 916,
993
+ 917,
994
+ 918,
995
+ 919,
996
+ 920,
997
+ 921,
998
+ 922,
999
+ 923,
1000
+ 924,
1001
+ 925,
1002
+ 926,
1003
+ 927,
1004
+ 928,
1005
+ 929,
1006
+ 930,
1007
+ 931,
1008
+ 932,
1009
+ 933,
1010
+ 934,
1011
+ 935,
1012
+ 936,
1013
+ 937,
1014
+ 938,
1015
+ 939,
1016
+ 940,
1017
+ 941,
1018
+ 942,
1019
+ 943,
1020
+ 944,
1021
+ 945,
1022
+ 946,
1023
+ 947,
1024
+ 948,
1025
+ 949,
1026
+ 950,
1027
+ 951,
1028
+ 952,
1029
+ 953,
1030
+ 954,
1031
+ 955,
1032
+ 956,
1033
+ 957,
1034
+ 958,
1035
+ 959,
1036
+ 960,
1037
+ 961,
1038
+ 962,
1039
+ 963,
1040
+ 964,
1041
+ 965,
1042
+ 966,
1043
+ 967,
1044
+ 968,
1045
+ 969,
1046
+ 970,
1047
+ 971,
1048
+ 972,
1049
+ 973,
1050
+ 974,
1051
+ 975,
1052
+ 976,
1053
+ 977,
1054
+ 978,
1055
+ 979,
1056
+ 980,
1057
+ 981,
1058
+ 982,
1059
+ 983,
1060
+ 984,
1061
+ 985,
1062
+ 986,
1063
+ 987,
1064
+ 988,
1065
+ 989,
1066
+ 990,
1067
+ 991,
1068
+ 992,
1069
+ 993,
1070
+ 994,
1071
+ 995,
1072
+ 996,
1073
+ 997,
1074
+ 998,
1075
+ 999,
1076
+ 1000,
1077
+ 1001,
1078
+ 1002,
1079
+ 1003,
1080
+ 1004,
1081
+ 1005,
1082
+ 1006,
1083
+ 1007,
1084
+ 1008,
1085
+ 1009,
1086
+ 1010,
1087
+ 1011,
1088
+ 1012,
1089
+ 1013,
1090
+ 1014,
1091
+ 1015,
1092
+ 1016,
1093
+ 1017,
1094
+ 1018,
1095
+ 1019,
1096
+ 1020,
1097
+ 1021,
1098
+ 1022,
1099
+ 1023,
1100
+ 1024,
1101
+ 1025,
1102
+ 1026,
1103
+ 1027,
1104
+ 1028,
1105
+ 1029,
1106
+ 1030,
1107
+ 1031,
1108
+ 1032,
1109
+ 1033,
1110
+ 1034,
1111
+ 1035,
1112
+ 1036,
1113
+ 1037,
1114
+ 1038,
1115
+ 1039,
1116
+ 1040,
1117
+ 1041,
1118
+ 1042,
1119
+ 1043,
1120
+ 1044,
1121
+ 1045,
1122
+ 1046,
1123
+ 1047,
1124
+ 1048,
1125
+ 1049,
1126
+ 1050,
1127
+ 1051,
1128
+ 1052,
1129
+ 1053,
1130
+ 1054,
1131
+ 1055,
1132
+ 1056,
1133
+ 1057,
1134
+ 1058,
1135
+ 1059,
1136
+ 1060,
1137
+ 1061,
1138
+ 1062,
1139
+ 1063,
1140
+ 1064,
1141
+ 1065,
1142
+ 1066,
1143
+ 1067,
1144
+ 1068,
1145
+ 1069,
1146
+ 1070,
1147
+ 1071,
1148
+ 1072,
1149
+ 1073,
1150
+ 1074,
1151
+ 1075,
1152
+ 1076,
1153
+ 1077,
1154
+ 1078,
1155
+ 1079,
1156
+ 1080,
1157
+ 1081,
1158
+ 1082,
1159
+ 1083,
1160
+ 1084,
1161
+ 1085,
1162
+ 1086,
1163
+ 1087,
1164
+ 1088,
1165
+ 1089,
1166
+ 1090,
1167
+ 1091,
1168
+ 1092,
1169
+ 1093,
1170
+ 1094,
1171
+ 1095,
1172
+ 1096,
1173
+ 1097,
1174
+ 1098,
1175
+ 1099,
1176
+ 1100,
1177
+ 1101,
1178
+ 1102,
1179
+ 1103,
1180
+ 1104,
1181
+ 1105,
1182
+ 1106,
1183
+ 1107,
1184
+ 1108,
1185
+ 1109,
1186
+ 1110,
1187
+ 1111,
1188
+ 1112,
1189
+ 1113,
1190
+ 1114,
1191
+ 1115,
1192
+ 1116,
1193
+ 1117,
1194
+ 1118,
1195
+ 1119,
1196
+ 1120,
1197
+ 1121,
1198
+ 1122,
1199
+ 1123,
1200
+ 1124,
1201
+ 1125,
1202
+ 1126,
1203
+ 1127,
1204
+ 1128,
1205
+ 1129,
1206
+ 1130,
1207
+ 1131,
1208
+ 1132,
1209
+ 1133,
1210
+ 1134,
1211
+ 1135,
1212
+ 1136,
1213
+ 1137,
1214
+ 1138,
1215
+ 1139,
1216
+ 1140,
1217
+ 1141,
1218
+ 1142,
1219
+ 1143,
1220
+ 1144,
1221
+ 1145,
1222
+ 1146,
1223
+ 1147,
1224
+ 1148,
1225
+ 1149,
1226
+ 1150,
1227
+ 1151,
1228
+ 1152,
1229
+ 1153,
1230
+ 1154,
1231
+ 1155,
1232
+ 1156,
1233
+ 1157,
1234
+ 1158,
1235
+ 1159,
1236
+ 1160,
1237
+ 1161,
1238
+ 1162,
1239
+ 1163,
1240
+ 1164,
1241
+ 1165,
1242
+ 1166,
1243
+ 1167,
1244
+ 1168,
1245
+ 1169,
1246
+ 1170,
1247
+ 1171,
1248
+ 1172,
1249
+ 1173,
1250
+ 1174,
1251
+ 1175,
1252
+ 1176,
1253
+ 1177,
1254
+ 1178,
1255
+ 1179,
1256
+ 1180,
1257
+ 1181,
1258
+ 1182,
1259
+ 1183,
1260
+ 1184,
1261
+ 1185,
1262
+ 1186,
1263
+ 1187,
1264
+ 1188,
1265
+ 1189,
1266
+ 1190,
1267
+ 1191,
1268
+ 1192,
1269
+ 1193,
1270
+ 1194,
1271
+ 1195,
1272
+ 1196,
1273
+ 1197,
1274
+ 1198,
1275
+ 1199,
1276
+ 1200,
1277
+ 1201,
1278
+ 1202,
1279
+ 1203,
1280
+ 1204,
1281
+ 1205,
1282
+ 1206,
1283
+ 1207,
1284
+ 1208,
1285
+ 1209,
1286
+ 1210,
1287
+ 1211,
1288
+ 1212,
1289
+ 1213,
1290
+ 1214,
1291
+ 1215,
1292
+ 1216,
1293
+ 1217,
1294
+ 1218,
1295
+ 1219,
1296
+ 1220,
1297
+ 1221,
1298
+ 1222,
1299
+ 1223,
1300
+ 1224,
1301
+ 1225,
1302
+ 1226,
1303
+ 1227,
1304
+ 1228,
1305
+ 1229,
1306
+ 1230,
1307
+ 1231,
1308
+ 1232,
1309
+ 1233,
1310
+ 1234,
1311
+ 1235,
1312
+ 1236,
1313
+ 1237,
1314
+ 1238,
1315
+ 1239,
1316
+ 1240,
1317
+ 1241,
1318
+ 1242,
1319
+ 1243,
1320
+ 1244,
1321
+ 1245,
1322
+ 1246,
1323
+ 1247,
1324
+ 1248,
1325
+ 1249,
1326
+ 1250,
1327
+ 1251,
1328
+ 1252,
1329
+ 1253,
1330
+ 1254,
1331
+ 1255,
1332
+ 1256,
1333
+ 1257,
1334
+ 1258,
1335
+ 1259,
1336
+ 1260,
1337
+ 1261,
1338
+ 1262,
1339
+ 1263,
1340
+ 1264,
1341
+ 1265,
1342
+ 1266,
1343
+ 1267,
1344
+ 1268,
1345
+ 1269,
1346
+ 1270,
1347
+ 1271,
1348
+ 1272,
1349
+ 1273,
1350
+ 1274,
1351
+ 1275,
1352
+ 1276,
1353
+ 1277,
1354
+ 1278,
1355
+ 1279,
1356
+ 1280,
1357
+ 1281,
1358
+ 1282,
1359
+ 1283,
1360
+ 1284,
1361
+ 1285,
1362
+ 1286,
1363
+ 1287,
1364
+ 1288,
1365
+ 1289,
1366
+ 1290,
1367
+ 1291
1368
+ ]
1369
+ }
1370
+ ]
checkpoints/020000/training_state/optimizer_state.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0c4c24208173118c09fb7af3fa28687989b6104408e6be6575c65d6ecadc0d1c
3
+ size 18260308060
checkpoints/020000/training_state/rng_state.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:2a5adad55ec2c81ad416821e0fc58f8e49fff318c69e63ffe4d933d81a2561cf
3
+ size 15708
checkpoints/020000/training_state/scheduler_state.json ADDED
@@ -0,0 +1,24 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "base_lrs": [
3
+ 1e-05,
4
+ 5e-06,
5
+ 5e-06,
6
+ 5e-05
7
+ ],
8
+ "last_epoch": 20000,
9
+ "_step_count": 20001,
10
+ "_is_initial": false,
11
+ "_get_lr_called_within_step": false,
12
+ "_last_lr": [
13
+ 5.500000000000001e-06,
14
+ 2.7500000000000004e-06,
15
+ 2.7500000000000004e-06,
16
+ 2.7500000000000004e-05
17
+ ],
18
+ "lr_lambdas": [
19
+ null,
20
+ null,
21
+ null,
22
+ null
23
+ ]
24
+ }
checkpoints/020000/training_state/training_step.json ADDED
@@ -0,0 +1,5 @@
 
 
 
 
 
 
1
+ {
2
+ "step": 20000,
3
+ "num_processes": 1,
4
+ "batch_size": 16
5
+ }
checkpoints/040000/pretrained_model/config.json ADDED
@@ -0,0 +1,139 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "type": "molmoact2",
3
+ "n_obs_steps": 1,
4
+ "input_features": {
5
+ "observation.state": {
6
+ "type": "STATE",
7
+ "shape": [
8
+ 6
9
+ ]
10
+ },
11
+ "observation.images.front": {
12
+ "type": "VISUAL",
13
+ "shape": [
14
+ 3,
15
+ 480,
16
+ 640
17
+ ]
18
+ },
19
+ "observation.images.wrist": {
20
+ "type": "VISUAL",
21
+ "shape": [
22
+ 3,
23
+ 480,
24
+ 640
25
+ ]
26
+ }
27
+ },
28
+ "output_features": {
29
+ "action": {
30
+ "type": "ACTION",
31
+ "shape": [
32
+ 6
33
+ ]
34
+ }
35
+ },
36
+ "device": "cuda",
37
+ "use_amp": false,
38
+ "use_peft": false,
39
+ "push_to_hub": true,
40
+ "repo_id": "tsangb34/molmoact2-so101-stack-cubes-40k",
41
+ "private": false,
42
+ "tags": [
43
+ "molmoact2",
44
+ "so101",
45
+ "robocolosseum",
46
+ "stack-cubes"
47
+ ],
48
+ "license": null,
49
+ "pretrained_path": null,
50
+ "pretrained_revision": null,
51
+ "checkpoint_path": "allenai/MolmoAct2",
52
+ "checkpoint_revision": null,
53
+ "checkpoint_force_download": false,
54
+ "chunk_size": 32,
55
+ "n_action_steps": 32,
56
+ "action_mode": "continuous",
57
+ "inference_action_mode": "continuous",
58
+ "discrete_action_tokenizer": "allenai/MolmoAct2-FAST-Tokenizer",
59
+ "discrete_generation_max_steps": null,
60
+ "norm_tag": null,
61
+ "setup_type": "single so100/so101 robotic arm in molmoact2",
62
+ "control_mode": "absolute joint pose",
63
+ "image_keys": [
64
+ "observation.images.front",
65
+ "observation.images.wrist"
66
+ ],
67
+ "normalize_language": true,
68
+ "add_setup_tokens": true,
69
+ "add_control_tokens": true,
70
+ "normalize_gripper": true,
71
+ "num_state_tokens": 256,
72
+ "max_sequence_length": null,
73
+ "expected_max_action_dim": 32,
74
+ "num_flow_timesteps": 8,
75
+ "flow_matching_cutoff": 1.0,
76
+ "flow_matching_time_offset": 0.001,
77
+ "flow_matching_time_scale": 0.999,
78
+ "flow_matching_beta_alpha": 1.0,
79
+ "flow_matching_beta_beta": 1.5,
80
+ "num_inference_steps": null,
81
+ "mask_action_dim_padding": true,
82
+ "enable_inference_cuda_graph": true,
83
+ "per_episode_seed": false,
84
+ "eval_seed": null,
85
+ "rtc_config": null,
86
+ "joint_signs": null,
87
+ "joint_offsets": null,
88
+ "enable_lora_vlm": false,
89
+ "lora_rank": 64,
90
+ "lora_alpha": 16,
91
+ "lora_dropout": 0.05,
92
+ "lora_bias": "none",
93
+ "enable_lora_action_expert": false,
94
+ "enable_knowledge_insulation": false,
95
+ "freeze_embedding": true,
96
+ "train_action_expert_only": false,
97
+ "gradient_checkpointing": true,
98
+ "model_dtype": "bfloat16",
99
+ "softmax_auxiliary_loss": true,
100
+ "softmax_auxiliary_loss_scale": 0.0001,
101
+ "discrete_loss_token_weighting": "root_subsegments_root_tokens",
102
+ "optimizer_lr": 1e-05,
103
+ "optimizer_vit_lr": 5e-06,
104
+ "optimizer_connector_lr": 5e-06,
105
+ "optimizer_action_expert_lr": 5e-05,
106
+ "optimizer_betas": [
107
+ 0.9,
108
+ 0.95
109
+ ],
110
+ "optimizer_eps": 1e-06,
111
+ "optimizer_weight_decay": 0.0,
112
+ "optimizer_grad_clip_norm": 1.0,
113
+ "scheduler_warmup_steps": 200,
114
+ "scheduler_decay_steps": 40000,
115
+ "scheduler_decay_lr": 1e-06,
116
+ "normalization_mapping": {
117
+ "ACTION": "QUANTILES",
118
+ "STATE": "QUANTILES",
119
+ "VISUAL": "IDENTITY"
120
+ },
121
+ "dataset_feature_names": {
122
+ "action": [
123
+ "shoulder_pan.pos",
124
+ "shoulder_lift.pos",
125
+ "elbow_flex.pos",
126
+ "wrist_flex.pos",
127
+ "wrist_roll.pos",
128
+ "gripper.pos"
129
+ ],
130
+ "observation.state": [
131
+ "shoulder_pan.pos",
132
+ "shoulder_lift.pos",
133
+ "elbow_flex.pos",
134
+ "wrist_flex.pos",
135
+ "wrist_roll.pos",
136
+ "gripper.pos"
137
+ ]
138
+ }
139
+ }
checkpoints/040000/pretrained_model/model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b4a641939db28ef73392c98dcb7116767cd2cbabe7cbdca68836ba88efc793bc
3
+ size 10884573720
checkpoints/040000/pretrained_model/policy_postprocessor.json ADDED
@@ -0,0 +1,43 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "name": "policy_postprocessor",
3
+ "steps": [
4
+ {
5
+ "registry_name": "molmoact2_clamp_action",
6
+ "config": {}
7
+ },
8
+ {
9
+ "registry_name": "molmoact2_masked_unnormalizer",
10
+ "config": {
11
+ "eps": 1e-08,
12
+ "features": {
13
+ "action": {
14
+ "type": "ACTION",
15
+ "shape": [
16
+ 6
17
+ ]
18
+ }
19
+ },
20
+ "norm_map": {
21
+ "ACTION": "QUANTILES",
22
+ "STATE": "QUANTILES",
23
+ "VISUAL": "IDENTITY"
24
+ }
25
+ },
26
+ "state_file": "policy_postprocessor_step_1_molmoact2_masked_unnormalizer.safetensors"
27
+ },
28
+ {
29
+ "registry_name": "molmoact2_action_frame_transform",
30
+ "config": {
31
+ "joint_signs": null,
32
+ "joint_offsets": null
33
+ }
34
+ },
35
+ {
36
+ "registry_name": "device_processor",
37
+ "config": {
38
+ "device": "cpu",
39
+ "float_dtype": null
40
+ }
41
+ }
42
+ ]
43
+ }
checkpoints/040000/pretrained_model/policy_postprocessor_step_1_molmoact2_masked_unnormalizer.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ee6a8cf8d7533d272953fddc06a419c5b7c89cac01d3dba8db5b3cee5efb107f
3
+ size 7744
checkpoints/040000/pretrained_model/policy_preprocessor.json ADDED
@@ -0,0 +1,100 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "name": "policy_preprocessor",
3
+ "steps": [
4
+ {
5
+ "registry_name": "rename_observations_processor",
6
+ "config": {
7
+ "rename_map": {}
8
+ }
9
+ },
10
+ {
11
+ "registry_name": "to_batch_processor",
12
+ "config": {}
13
+ },
14
+ {
15
+ "registry_name": "molmoact2_state_frame_transform",
16
+ "config": {
17
+ "joint_signs": null,
18
+ "joint_offsets": null
19
+ }
20
+ },
21
+ {
22
+ "registry_name": "molmoact2_masked_normalizer",
23
+ "config": {
24
+ "eps": 1e-08,
25
+ "features": {
26
+ "observation.state": {
27
+ "type": "STATE",
28
+ "shape": [
29
+ 6
30
+ ]
31
+ },
32
+ "observation.images.front": {
33
+ "type": "VISUAL",
34
+ "shape": [
35
+ 3,
36
+ 480,
37
+ 640
38
+ ]
39
+ },
40
+ "observation.images.wrist": {
41
+ "type": "VISUAL",
42
+ "shape": [
43
+ 3,
44
+ 480,
45
+ 640
46
+ ]
47
+ },
48
+ "action": {
49
+ "type": "ACTION",
50
+ "shape": [
51
+ 6
52
+ ]
53
+ }
54
+ },
55
+ "norm_map": {
56
+ "ACTION": "QUANTILES",
57
+ "STATE": "QUANTILES",
58
+ "VISUAL": "IDENTITY"
59
+ }
60
+ },
61
+ "state_file": "policy_preprocessor_step_3_molmoact2_masked_normalizer.safetensors"
62
+ },
63
+ {
64
+ "registry_name": "molmoact2_clamp_normalized",
65
+ "config": {}
66
+ },
67
+ {
68
+ "registry_name": "molmoact2_pack_inputs",
69
+ "config": {
70
+ "checkpoint_path": "allenai/MolmoAct2",
71
+ "checkpoint_revision": null,
72
+ "checkpoint_force_download": false,
73
+ "action_mode": "continuous",
74
+ "discrete_action_tokenizer": "allenai/MolmoAct2-FAST-Tokenizer",
75
+ "image_keys": [
76
+ "observation.images.front",
77
+ "observation.images.wrist"
78
+ ],
79
+ "allow_image_key_fallback": false,
80
+ "setup_type": "single so100/so101 robotic arm in molmoact2",
81
+ "control_mode": "absolute joint pose",
82
+ "normalize_language": true,
83
+ "add_setup_tokens": true,
84
+ "add_control_tokens": true,
85
+ "num_state_tokens": 256,
86
+ "max_sequence_length": null,
87
+ "chunk_size": 32,
88
+ "max_action_dim": 32,
89
+ "env_action_dim": 6
90
+ }
91
+ },
92
+ {
93
+ "registry_name": "device_processor",
94
+ "config": {
95
+ "device": "cuda",
96
+ "float_dtype": null
97
+ }
98
+ }
99
+ ]
100
+ }
checkpoints/040000/pretrained_model/policy_preprocessor_step_3_molmoact2_masked_normalizer.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:95f69bcc0e1bee273f9668382626418fd89e92d69d6e42aa99c4a458f9a06c31
3
+ size 7752
checkpoints/040000/pretrained_model/train_config.json ADDED
@@ -0,0 +1,275 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "dataset": {
3
+ "repo_id": "Jiamo0912/so101-stack-cubes",
4
+ "repo_type": "dataset",
5
+ "root": "/scratch/e1583535/datasets/so101-stack-cubes",
6
+ "episodes": null,
7
+ "image_transforms": {
8
+ "enable": true,
9
+ "max_num_transforms": 3,
10
+ "random_order": false,
11
+ "tfs": {
12
+ "crop": {
13
+ "weight": 1.0,
14
+ "type": "RandomResizedCrop",
15
+ "kwargs": {
16
+ "size": [
17
+ 480,
18
+ 640
19
+ ],
20
+ "scale": [
21
+ 0.9025,
22
+ 0.9025
23
+ ],
24
+ "ratio": [
25
+ 1.3333333,
26
+ 1.3333333
27
+ ]
28
+ }
29
+ },
30
+ "rotation": {
31
+ "weight": 1.0,
32
+ "type": "RandomRotation",
33
+ "kwargs": {
34
+ "degrees": [
35
+ -5.0,
36
+ 5.0
37
+ ],
38
+ "interpolation": 2
39
+ }
40
+ },
41
+ "color": {
42
+ "weight": 1.0,
43
+ "type": "ColorJitter",
44
+ "kwargs": {
45
+ "brightness": 0.2,
46
+ "contrast": [
47
+ 0.8,
48
+ 1.2
49
+ ],
50
+ "saturation": [
51
+ 0.8,
52
+ 1.2
53
+ ],
54
+ "hue": 0.05
55
+ }
56
+ }
57
+ }
58
+ },
59
+ "revision": null,
60
+ "use_imagenet_stats": true,
61
+ "video_backend": "pyav",
62
+ "return_uint8": false,
63
+ "depth_output_unit": "mm",
64
+ "streaming": false,
65
+ "eval_split": 0.0
66
+ },
67
+ "env": null,
68
+ "policy": {
69
+ "type": "molmoact2",
70
+ "n_obs_steps": 1,
71
+ "input_features": {
72
+ "observation.state": {
73
+ "type": "STATE",
74
+ "shape": [
75
+ 6
76
+ ]
77
+ },
78
+ "observation.images.front": {
79
+ "type": "VISUAL",
80
+ "shape": [
81
+ 3,
82
+ 480,
83
+ 640
84
+ ]
85
+ },
86
+ "observation.images.wrist": {
87
+ "type": "VISUAL",
88
+ "shape": [
89
+ 3,
90
+ 480,
91
+ 640
92
+ ]
93
+ }
94
+ },
95
+ "output_features": {
96
+ "action": {
97
+ "type": "ACTION",
98
+ "shape": [
99
+ 6
100
+ ]
101
+ }
102
+ },
103
+ "device": "cuda",
104
+ "use_amp": false,
105
+ "use_peft": false,
106
+ "push_to_hub": true,
107
+ "repo_id": "tsangb34/molmoact2-so101-stack-cubes-40k",
108
+ "private": false,
109
+ "tags": [
110
+ "molmoact2",
111
+ "so101",
112
+ "robocolosseum",
113
+ "stack-cubes"
114
+ ],
115
+ "license": null,
116
+ "pretrained_path": null,
117
+ "pretrained_revision": null,
118
+ "checkpoint_path": "allenai/MolmoAct2",
119
+ "checkpoint_revision": null,
120
+ "checkpoint_force_download": false,
121
+ "chunk_size": 32,
122
+ "n_action_steps": 32,
123
+ "action_mode": "continuous",
124
+ "inference_action_mode": "continuous",
125
+ "discrete_action_tokenizer": "allenai/MolmoAct2-FAST-Tokenizer",
126
+ "discrete_generation_max_steps": null,
127
+ "norm_tag": null,
128
+ "setup_type": "single so100/so101 robotic arm in molmoact2",
129
+ "control_mode": "absolute joint pose",
130
+ "image_keys": [
131
+ "observation.images.front",
132
+ "observation.images.wrist"
133
+ ],
134
+ "normalize_language": true,
135
+ "add_setup_tokens": true,
136
+ "add_control_tokens": true,
137
+ "normalize_gripper": true,
138
+ "num_state_tokens": 256,
139
+ "max_sequence_length": null,
140
+ "expected_max_action_dim": 32,
141
+ "num_flow_timesteps": 8,
142
+ "flow_matching_cutoff": 1.0,
143
+ "flow_matching_time_offset": 0.001,
144
+ "flow_matching_time_scale": 0.999,
145
+ "flow_matching_beta_alpha": 1.0,
146
+ "flow_matching_beta_beta": 1.5,
147
+ "num_inference_steps": null,
148
+ "mask_action_dim_padding": true,
149
+ "enable_inference_cuda_graph": true,
150
+ "per_episode_seed": false,
151
+ "eval_seed": null,
152
+ "rtc_config": null,
153
+ "joint_signs": null,
154
+ "joint_offsets": null,
155
+ "enable_lora_vlm": false,
156
+ "lora_rank": 64,
157
+ "lora_alpha": 16,
158
+ "lora_dropout": 0.05,
159
+ "lora_bias": "none",
160
+ "enable_lora_action_expert": false,
161
+ "enable_knowledge_insulation": false,
162
+ "freeze_embedding": true,
163
+ "train_action_expert_only": false,
164
+ "gradient_checkpointing": true,
165
+ "model_dtype": "bfloat16",
166
+ "softmax_auxiliary_loss": true,
167
+ "softmax_auxiliary_loss_scale": 0.0001,
168
+ "discrete_loss_token_weighting": "root_subsegments_root_tokens",
169
+ "optimizer_lr": 1e-05,
170
+ "optimizer_vit_lr": 5e-06,
171
+ "optimizer_connector_lr": 5e-06,
172
+ "optimizer_action_expert_lr": 5e-05,
173
+ "optimizer_betas": [
174
+ 0.9,
175
+ 0.95
176
+ ],
177
+ "optimizer_eps": 1e-06,
178
+ "optimizer_weight_decay": 0.0,
179
+ "optimizer_grad_clip_norm": 1.0,
180
+ "scheduler_warmup_steps": 200,
181
+ "scheduler_decay_steps": 40000,
182
+ "scheduler_decay_lr": 1e-06,
183
+ "normalization_mapping": {
184
+ "ACTION": "QUANTILES",
185
+ "STATE": "QUANTILES",
186
+ "VISUAL": "IDENTITY"
187
+ },
188
+ "dataset_feature_names": {
189
+ "action": [
190
+ "shoulder_pan.pos",
191
+ "shoulder_lift.pos",
192
+ "elbow_flex.pos",
193
+ "wrist_flex.pos",
194
+ "wrist_roll.pos",
195
+ "gripper.pos"
196
+ ],
197
+ "observation.state": [
198
+ "shoulder_pan.pos",
199
+ "shoulder_lift.pos",
200
+ "elbow_flex.pos",
201
+ "wrist_flex.pos",
202
+ "wrist_roll.pos",
203
+ "gripper.pos"
204
+ ]
205
+ }
206
+ },
207
+ "reward_model": null,
208
+ "output_dir": "/scratch/Projects/CFP-05/CFP05-CF-002/robocolosseum-finetuned-models/molmoact2-so101-stack-cubes-40k",
209
+ "job_name": "molmoact2-so101-stack-cubes-40k",
210
+ "resume": false,
211
+ "seed": 1000,
212
+ "cudnn_deterministic": false,
213
+ "num_workers": 4,
214
+ "batch_size": 16,
215
+ "prefetch_factor": 4,
216
+ "persistent_workers": true,
217
+ "dataloader_multiprocessing_context": "spawn",
218
+ "steps": 40000,
219
+ "env_eval_freq": 20000,
220
+ "log_freq": 20,
221
+ "eval_steps": 0,
222
+ "max_eval_samples": 0,
223
+ "tolerance_s": 0.0001,
224
+ "save_checkpoint": true,
225
+ "save_freq": 20000,
226
+ "use_policy_training_preset": true,
227
+ "optimizer": {
228
+ "type": "adamw",
229
+ "lr": 1e-05,
230
+ "weight_decay": 0.0,
231
+ "grad_clip_norm": 1.0,
232
+ "betas": [
233
+ 0.9,
234
+ 0.95
235
+ ],
236
+ "eps": 1e-06
237
+ },
238
+ "scheduler": {
239
+ "type": "cosine_decay_with_warmup",
240
+ "num_warmup_steps": 200,
241
+ "num_decay_steps": 40000,
242
+ "peak_lr": 1e-05,
243
+ "decay_lr": 1e-06
244
+ },
245
+ "eval": {
246
+ "n_episodes": 50,
247
+ "batch_size": 50,
248
+ "use_async_envs": true,
249
+ "recording": false,
250
+ "recording_repo_id": null,
251
+ "recording_private": false
252
+ },
253
+ "wandb": {
254
+ "enable": true,
255
+ "disable_artifact": true,
256
+ "project": "RoboColosseum",
257
+ "entity": "tsangb34-national-university-of-singapore-students-union",
258
+ "notes": null,
259
+ "run_id": "molmoact2-so101-stack-cubes-40k",
260
+ "mode": null,
261
+ "add_tags": true
262
+ },
263
+ "peft": null,
264
+ "job": {
265
+ "target": null,
266
+ "image": "huggingface/lerobot-gpu:latest",
267
+ "timeout": "2d",
268
+ "detach": false,
269
+ "tags": []
270
+ },
271
+ "save_checkpoint_to_hub": true,
272
+ "sample_weighting": null,
273
+ "rename_map": {},
274
+ "checkpoint_path": null
275
+ }
checkpoints/040000/training_state/optimizer_param_groups.json ADDED
@@ -0,0 +1,1370 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ [
2
+ {
3
+ "lr": 1e-06,
4
+ "betas": [
5
+ 0.9,
6
+ 0.95
7
+ ],
8
+ "eps": 1e-06,
9
+ "weight_decay": 0.0,
10
+ "amsgrad": false,
11
+ "maximize": false,
12
+ "foreach": null,
13
+ "capturable": false,
14
+ "differentiable": false,
15
+ "fused": null,
16
+ "decoupled_weight_decay": true,
17
+ "initial_lr": 1e-05,
18
+ "params": [
19
+ 0,
20
+ 1,
21
+ 2,
22
+ 3,
23
+ 4,
24
+ 5,
25
+ 6,
26
+ 7,
27
+ 8,
28
+ 9,
29
+ 10,
30
+ 11,
31
+ 12,
32
+ 13,
33
+ 14,
34
+ 15,
35
+ 16,
36
+ 17,
37
+ 18,
38
+ 19,
39
+ 20,
40
+ 21,
41
+ 22,
42
+ 23,
43
+ 24,
44
+ 25,
45
+ 26,
46
+ 27,
47
+ 28,
48
+ 29,
49
+ 30,
50
+ 31,
51
+ 32,
52
+ 33,
53
+ 34,
54
+ 35,
55
+ 36,
56
+ 37,
57
+ 38,
58
+ 39,
59
+ 40,
60
+ 41,
61
+ 42,
62
+ 43,
63
+ 44,
64
+ 45,
65
+ 46,
66
+ 47,
67
+ 48,
68
+ 49,
69
+ 50,
70
+ 51,
71
+ 52,
72
+ 53,
73
+ 54,
74
+ 55,
75
+ 56,
76
+ 57,
77
+ 58,
78
+ 59,
79
+ 60,
80
+ 61,
81
+ 62,
82
+ 63,
83
+ 64,
84
+ 65,
85
+ 66,
86
+ 67,
87
+ 68,
88
+ 69,
89
+ 70,
90
+ 71,
91
+ 72,
92
+ 73,
93
+ 74,
94
+ 75,
95
+ 76,
96
+ 77,
97
+ 78,
98
+ 79,
99
+ 80,
100
+ 81,
101
+ 82,
102
+ 83,
103
+ 84,
104
+ 85,
105
+ 86,
106
+ 87,
107
+ 88,
108
+ 89,
109
+ 90,
110
+ 91,
111
+ 92,
112
+ 93,
113
+ 94,
114
+ 95,
115
+ 96,
116
+ 97,
117
+ 98,
118
+ 99,
119
+ 100,
120
+ 101,
121
+ 102,
122
+ 103,
123
+ 104,
124
+ 105,
125
+ 106,
126
+ 107,
127
+ 108,
128
+ 109,
129
+ 110,
130
+ 111,
131
+ 112,
132
+ 113,
133
+ 114,
134
+ 115,
135
+ 116,
136
+ 117,
137
+ 118,
138
+ 119,
139
+ 120,
140
+ 121,
141
+ 122,
142
+ 123,
143
+ 124,
144
+ 125,
145
+ 126,
146
+ 127,
147
+ 128,
148
+ 129,
149
+ 130,
150
+ 131,
151
+ 132,
152
+ 133,
153
+ 134,
154
+ 135,
155
+ 136,
156
+ 137,
157
+ 138,
158
+ 139,
159
+ 140,
160
+ 141,
161
+ 142,
162
+ 143,
163
+ 144,
164
+ 145,
165
+ 146,
166
+ 147,
167
+ 148,
168
+ 149,
169
+ 150,
170
+ 151,
171
+ 152,
172
+ 153,
173
+ 154,
174
+ 155,
175
+ 156,
176
+ 157,
177
+ 158,
178
+ 159,
179
+ 160,
180
+ 161,
181
+ 162,
182
+ 163,
183
+ 164,
184
+ 165,
185
+ 166,
186
+ 167,
187
+ 168,
188
+ 169,
189
+ 170,
190
+ 171,
191
+ 172,
192
+ 173,
193
+ 174,
194
+ 175,
195
+ 176,
196
+ 177,
197
+ 178,
198
+ 179,
199
+ 180,
200
+ 181,
201
+ 182,
202
+ 183,
203
+ 184,
204
+ 185,
205
+ 186,
206
+ 187,
207
+ 188,
208
+ 189,
209
+ 190,
210
+ 191,
211
+ 192,
212
+ 193,
213
+ 194,
214
+ 195,
215
+ 196,
216
+ 197,
217
+ 198,
218
+ 199,
219
+ 200,
220
+ 201,
221
+ 202,
222
+ 203,
223
+ 204,
224
+ 205,
225
+ 206,
226
+ 207,
227
+ 208,
228
+ 209,
229
+ 210,
230
+ 211,
231
+ 212,
232
+ 213,
233
+ 214,
234
+ 215,
235
+ 216,
236
+ 217,
237
+ 218,
238
+ 219,
239
+ 220,
240
+ 221,
241
+ 222,
242
+ 223,
243
+ 224,
244
+ 225,
245
+ 226,
246
+ 227,
247
+ 228,
248
+ 229,
249
+ 230,
250
+ 231,
251
+ 232,
252
+ 233,
253
+ 234,
254
+ 235,
255
+ 236,
256
+ 237,
257
+ 238,
258
+ 239,
259
+ 240,
260
+ 241,
261
+ 242,
262
+ 243,
263
+ 244,
264
+ 245,
265
+ 246,
266
+ 247,
267
+ 248,
268
+ 249,
269
+ 250,
270
+ 251,
271
+ 252,
272
+ 253,
273
+ 254,
274
+ 255,
275
+ 256,
276
+ 257,
277
+ 258,
278
+ 259,
279
+ 260,
280
+ 261,
281
+ 262,
282
+ 263,
283
+ 264,
284
+ 265,
285
+ 266,
286
+ 267,
287
+ 268,
288
+ 269,
289
+ 270,
290
+ 271,
291
+ 272,
292
+ 273,
293
+ 274,
294
+ 275,
295
+ 276,
296
+ 277,
297
+ 278,
298
+ 279,
299
+ 280,
300
+ 281,
301
+ 282,
302
+ 283,
303
+ 284,
304
+ 285,
305
+ 286,
306
+ 287,
307
+ 288,
308
+ 289
309
+ ]
310
+ },
311
+ {
312
+ "lr": 5e-07,
313
+ "betas": [
314
+ 0.9,
315
+ 0.95
316
+ ],
317
+ "eps": 1e-06,
318
+ "weight_decay": 0.0,
319
+ "amsgrad": false,
320
+ "maximize": false,
321
+ "foreach": null,
322
+ "capturable": false,
323
+ "differentiable": false,
324
+ "fused": null,
325
+ "decoupled_weight_decay": true,
326
+ "initial_lr": 5e-06,
327
+ "params": [
328
+ 290,
329
+ 291,
330
+ 292,
331
+ 293,
332
+ 294,
333
+ 295,
334
+ 296,
335
+ 297,
336
+ 298,
337
+ 299,
338
+ 300,
339
+ 301,
340
+ 302,
341
+ 303,
342
+ 304,
343
+ 305,
344
+ 306,
345
+ 307,
346
+ 308,
347
+ 309,
348
+ 310,
349
+ 311,
350
+ 312,
351
+ 313,
352
+ 314,
353
+ 315,
354
+ 316,
355
+ 317,
356
+ 318,
357
+ 319,
358
+ 320,
359
+ 321,
360
+ 322,
361
+ 323,
362
+ 324,
363
+ 325,
364
+ 326,
365
+ 327,
366
+ 328,
367
+ 329,
368
+ 330,
369
+ 331,
370
+ 332,
371
+ 333,
372
+ 334,
373
+ 335,
374
+ 336,
375
+ 337,
376
+ 338,
377
+ 339,
378
+ 340,
379
+ 341,
380
+ 342,
381
+ 343,
382
+ 344,
383
+ 345,
384
+ 346,
385
+ 347,
386
+ 348,
387
+ 349,
388
+ 350,
389
+ 351,
390
+ 352,
391
+ 353,
392
+ 354,
393
+ 355,
394
+ 356,
395
+ 357,
396
+ 358,
397
+ 359,
398
+ 360,
399
+ 361,
400
+ 362,
401
+ 363,
402
+ 364,
403
+ 365,
404
+ 366,
405
+ 367,
406
+ 368,
407
+ 369,
408
+ 370,
409
+ 371,
410
+ 372,
411
+ 373,
412
+ 374,
413
+ 375,
414
+ 376,
415
+ 377,
416
+ 378,
417
+ 379,
418
+ 380,
419
+ 381,
420
+ 382,
421
+ 383,
422
+ 384,
423
+ 385,
424
+ 386,
425
+ 387,
426
+ 388,
427
+ 389,
428
+ 390,
429
+ 391,
430
+ 392,
431
+ 393,
432
+ 394,
433
+ 395,
434
+ 396,
435
+ 397,
436
+ 398,
437
+ 399,
438
+ 400,
439
+ 401,
440
+ 402,
441
+ 403,
442
+ 404,
443
+ 405,
444
+ 406,
445
+ 407,
446
+ 408,
447
+ 409,
448
+ 410,
449
+ 411,
450
+ 412,
451
+ 413,
452
+ 414,
453
+ 415,
454
+ 416,
455
+ 417,
456
+ 418,
457
+ 419,
458
+ 420,
459
+ 421,
460
+ 422,
461
+ 423,
462
+ 424,
463
+ 425,
464
+ 426,
465
+ 427,
466
+ 428,
467
+ 429,
468
+ 430,
469
+ 431,
470
+ 432,
471
+ 433,
472
+ 434,
473
+ 435,
474
+ 436,
475
+ 437,
476
+ 438,
477
+ 439,
478
+ 440,
479
+ 441,
480
+ 442,
481
+ 443,
482
+ 444,
483
+ 445,
484
+ 446,
485
+ 447,
486
+ 448,
487
+ 449,
488
+ 450,
489
+ 451,
490
+ 452,
491
+ 453,
492
+ 454,
493
+ 455,
494
+ 456,
495
+ 457,
496
+ 458,
497
+ 459,
498
+ 460,
499
+ 461,
500
+ 462,
501
+ 463,
502
+ 464,
503
+ 465,
504
+ 466,
505
+ 467,
506
+ 468,
507
+ 469,
508
+ 470,
509
+ 471,
510
+ 472,
511
+ 473,
512
+ 474,
513
+ 475,
514
+ 476,
515
+ 477,
516
+ 478,
517
+ 479,
518
+ 480,
519
+ 481,
520
+ 482,
521
+ 483,
522
+ 484,
523
+ 485,
524
+ 486,
525
+ 487,
526
+ 488,
527
+ 489,
528
+ 490,
529
+ 491,
530
+ 492,
531
+ 493,
532
+ 494,
533
+ 495,
534
+ 496,
535
+ 497,
536
+ 498,
537
+ 499,
538
+ 500,
539
+ 501,
540
+ 502,
541
+ 503,
542
+ 504,
543
+ 505,
544
+ 506,
545
+ 507,
546
+ 508,
547
+ 509,
548
+ 510,
549
+ 511,
550
+ 512,
551
+ 513,
552
+ 514,
553
+ 515,
554
+ 516,
555
+ 517,
556
+ 518,
557
+ 519,
558
+ 520,
559
+ 521,
560
+ 522,
561
+ 523,
562
+ 524,
563
+ 525,
564
+ 526,
565
+ 527,
566
+ 528,
567
+ 529,
568
+ 530,
569
+ 531,
570
+ 532,
571
+ 533,
572
+ 534,
573
+ 535,
574
+ 536,
575
+ 537,
576
+ 538,
577
+ 539,
578
+ 540,
579
+ 541,
580
+ 542,
581
+ 543,
582
+ 544,
583
+ 545,
584
+ 546,
585
+ 547,
586
+ 548,
587
+ 549,
588
+ 550,
589
+ 551,
590
+ 552,
591
+ 553,
592
+ 554,
593
+ 555,
594
+ 556,
595
+ 557,
596
+ 558,
597
+ 559,
598
+ 560,
599
+ 561,
600
+ 562,
601
+ 563,
602
+ 564,
603
+ 565,
604
+ 566,
605
+ 567,
606
+ 568,
607
+ 569,
608
+ 570,
609
+ 571,
610
+ 572,
611
+ 573,
612
+ 574,
613
+ 575,
614
+ 576,
615
+ 577,
616
+ 578,
617
+ 579,
618
+ 580,
619
+ 581,
620
+ 582,
621
+ 583,
622
+ 584,
623
+ 585,
624
+ 586,
625
+ 587,
626
+ 588,
627
+ 589,
628
+ 590,
629
+ 591,
630
+ 592,
631
+ 593,
632
+ 594,
633
+ 595,
634
+ 596,
635
+ 597,
636
+ 598,
637
+ 599,
638
+ 600,
639
+ 601,
640
+ 602,
641
+ 603,
642
+ 604,
643
+ 605,
644
+ 606,
645
+ 607,
646
+ 608,
647
+ 609,
648
+ 610,
649
+ 611,
650
+ 612,
651
+ 613,
652
+ 614,
653
+ 615,
654
+ 616,
655
+ 617,
656
+ 618,
657
+ 619,
658
+ 620,
659
+ 621,
660
+ 622,
661
+ 623,
662
+ 624,
663
+ 625,
664
+ 626,
665
+ 627,
666
+ 628,
667
+ 629,
668
+ 630,
669
+ 631,
670
+ 632,
671
+ 633,
672
+ 634,
673
+ 635,
674
+ 636,
675
+ 637,
676
+ 638,
677
+ 639,
678
+ 640,
679
+ 641,
680
+ 642,
681
+ 643,
682
+ 644,
683
+ 645,
684
+ 646,
685
+ 647,
686
+ 648,
687
+ 649,
688
+ 650,
689
+ 651,
690
+ 652,
691
+ 653,
692
+ 654,
693
+ 655,
694
+ 656,
695
+ 657,
696
+ 658,
697
+ 659,
698
+ 660,
699
+ 661,
700
+ 662,
701
+ 663,
702
+ 664,
703
+ 665,
704
+ 666,
705
+ 667,
706
+ 668,
707
+ 669,
708
+ 670,
709
+ 671,
710
+ 672,
711
+ 673,
712
+ 674,
713
+ 675,
714
+ 676,
715
+ 677,
716
+ 678,
717
+ 679,
718
+ 680,
719
+ 681,
720
+ 682,
721
+ 683,
722
+ 684,
723
+ 685,
724
+ 686,
725
+ 687,
726
+ 688,
727
+ 689,
728
+ 690,
729
+ 691,
730
+ 692
731
+ ]
732
+ },
733
+ {
734
+ "lr": 5e-07,
735
+ "betas": [
736
+ 0.9,
737
+ 0.95
738
+ ],
739
+ "eps": 1e-06,
740
+ "weight_decay": 0.0,
741
+ "amsgrad": false,
742
+ "maximize": false,
743
+ "foreach": null,
744
+ "capturable": false,
745
+ "differentiable": false,
746
+ "fused": null,
747
+ "decoupled_weight_decay": true,
748
+ "initial_lr": 5e-06,
749
+ "params": [
750
+ 693,
751
+ 694,
752
+ 695,
753
+ 696,
754
+ 697,
755
+ 698,
756
+ 699,
757
+ 700,
758
+ 701,
759
+ 702,
760
+ 703
761
+ ]
762
+ },
763
+ {
764
+ "lr": 4.9999999999999996e-06,
765
+ "betas": [
766
+ 0.9,
767
+ 0.95
768
+ ],
769
+ "eps": 1e-06,
770
+ "weight_decay": 0.0,
771
+ "amsgrad": false,
772
+ "maximize": false,
773
+ "foreach": null,
774
+ "capturable": false,
775
+ "differentiable": false,
776
+ "fused": null,
777
+ "decoupled_weight_decay": true,
778
+ "initial_lr": 5e-05,
779
+ "params": [
780
+ 704,
781
+ 705,
782
+ 706,
783
+ 707,
784
+ 708,
785
+ 709,
786
+ 710,
787
+ 711,
788
+ 712,
789
+ 713,
790
+ 714,
791
+ 715,
792
+ 716,
793
+ 717,
794
+ 718,
795
+ 719,
796
+ 720,
797
+ 721,
798
+ 722,
799
+ 723,
800
+ 724,
801
+ 725,
802
+ 726,
803
+ 727,
804
+ 728,
805
+ 729,
806
+ 730,
807
+ 731,
808
+ 732,
809
+ 733,
810
+ 734,
811
+ 735,
812
+ 736,
813
+ 737,
814
+ 738,
815
+ 739,
816
+ 740,
817
+ 741,
818
+ 742,
819
+ 743,
820
+ 744,
821
+ 745,
822
+ 746,
823
+ 747,
824
+ 748,
825
+ 749,
826
+ 750,
827
+ 751,
828
+ 752,
829
+ 753,
830
+ 754,
831
+ 755,
832
+ 756,
833
+ 757,
834
+ 758,
835
+ 759,
836
+ 760,
837
+ 761,
838
+ 762,
839
+ 763,
840
+ 764,
841
+ 765,
842
+ 766,
843
+ 767,
844
+ 768,
845
+ 769,
846
+ 770,
847
+ 771,
848
+ 772,
849
+ 773,
850
+ 774,
851
+ 775,
852
+ 776,
853
+ 777,
854
+ 778,
855
+ 779,
856
+ 780,
857
+ 781,
858
+ 782,
859
+ 783,
860
+ 784,
861
+ 785,
862
+ 786,
863
+ 787,
864
+ 788,
865
+ 789,
866
+ 790,
867
+ 791,
868
+ 792,
869
+ 793,
870
+ 794,
871
+ 795,
872
+ 796,
873
+ 797,
874
+ 798,
875
+ 799,
876
+ 800,
877
+ 801,
878
+ 802,
879
+ 803,
880
+ 804,
881
+ 805,
882
+ 806,
883
+ 807,
884
+ 808,
885
+ 809,
886
+ 810,
887
+ 811,
888
+ 812,
889
+ 813,
890
+ 814,
891
+ 815,
892
+ 816,
893
+ 817,
894
+ 818,
895
+ 819,
896
+ 820,
897
+ 821,
898
+ 822,
899
+ 823,
900
+ 824,
901
+ 825,
902
+ 826,
903
+ 827,
904
+ 828,
905
+ 829,
906
+ 830,
907
+ 831,
908
+ 832,
909
+ 833,
910
+ 834,
911
+ 835,
912
+ 836,
913
+ 837,
914
+ 838,
915
+ 839,
916
+ 840,
917
+ 841,
918
+ 842,
919
+ 843,
920
+ 844,
921
+ 845,
922
+ 846,
923
+ 847,
924
+ 848,
925
+ 849,
926
+ 850,
927
+ 851,
928
+ 852,
929
+ 853,
930
+ 854,
931
+ 855,
932
+ 856,
933
+ 857,
934
+ 858,
935
+ 859,
936
+ 860,
937
+ 861,
938
+ 862,
939
+ 863,
940
+ 864,
941
+ 865,
942
+ 866,
943
+ 867,
944
+ 868,
945
+ 869,
946
+ 870,
947
+ 871,
948
+ 872,
949
+ 873,
950
+ 874,
951
+ 875,
952
+ 876,
953
+ 877,
954
+ 878,
955
+ 879,
956
+ 880,
957
+ 881,
958
+ 882,
959
+ 883,
960
+ 884,
961
+ 885,
962
+ 886,
963
+ 887,
964
+ 888,
965
+ 889,
966
+ 890,
967
+ 891,
968
+ 892,
969
+ 893,
970
+ 894,
971
+ 895,
972
+ 896,
973
+ 897,
974
+ 898,
975
+ 899,
976
+ 900,
977
+ 901,
978
+ 902,
979
+ 903,
980
+ 904,
981
+ 905,
982
+ 906,
983
+ 907,
984
+ 908,
985
+ 909,
986
+ 910,
987
+ 911,
988
+ 912,
989
+ 913,
990
+ 914,
991
+ 915,
992
+ 916,
993
+ 917,
994
+ 918,
995
+ 919,
996
+ 920,
997
+ 921,
998
+ 922,
999
+ 923,
1000
+ 924,
1001
+ 925,
1002
+ 926,
1003
+ 927,
1004
+ 928,
1005
+ 929,
1006
+ 930,
1007
+ 931,
1008
+ 932,
1009
+ 933,
1010
+ 934,
1011
+ 935,
1012
+ 936,
1013
+ 937,
1014
+ 938,
1015
+ 939,
1016
+ 940,
1017
+ 941,
1018
+ 942,
1019
+ 943,
1020
+ 944,
1021
+ 945,
1022
+ 946,
1023
+ 947,
1024
+ 948,
1025
+ 949,
1026
+ 950,
1027
+ 951,
1028
+ 952,
1029
+ 953,
1030
+ 954,
1031
+ 955,
1032
+ 956,
1033
+ 957,
1034
+ 958,
1035
+ 959,
1036
+ 960,
1037
+ 961,
1038
+ 962,
1039
+ 963,
1040
+ 964,
1041
+ 965,
1042
+ 966,
1043
+ 967,
1044
+ 968,
1045
+ 969,
1046
+ 970,
1047
+ 971,
1048
+ 972,
1049
+ 973,
1050
+ 974,
1051
+ 975,
1052
+ 976,
1053
+ 977,
1054
+ 978,
1055
+ 979,
1056
+ 980,
1057
+ 981,
1058
+ 982,
1059
+ 983,
1060
+ 984,
1061
+ 985,
1062
+ 986,
1063
+ 987,
1064
+ 988,
1065
+ 989,
1066
+ 990,
1067
+ 991,
1068
+ 992,
1069
+ 993,
1070
+ 994,
1071
+ 995,
1072
+ 996,
1073
+ 997,
1074
+ 998,
1075
+ 999,
1076
+ 1000,
1077
+ 1001,
1078
+ 1002,
1079
+ 1003,
1080
+ 1004,
1081
+ 1005,
1082
+ 1006,
1083
+ 1007,
1084
+ 1008,
1085
+ 1009,
1086
+ 1010,
1087
+ 1011,
1088
+ 1012,
1089
+ 1013,
1090
+ 1014,
1091
+ 1015,
1092
+ 1016,
1093
+ 1017,
1094
+ 1018,
1095
+ 1019,
1096
+ 1020,
1097
+ 1021,
1098
+ 1022,
1099
+ 1023,
1100
+ 1024,
1101
+ 1025,
1102
+ 1026,
1103
+ 1027,
1104
+ 1028,
1105
+ 1029,
1106
+ 1030,
1107
+ 1031,
1108
+ 1032,
1109
+ 1033,
1110
+ 1034,
1111
+ 1035,
1112
+ 1036,
1113
+ 1037,
1114
+ 1038,
1115
+ 1039,
1116
+ 1040,
1117
+ 1041,
1118
+ 1042,
1119
+ 1043,
1120
+ 1044,
1121
+ 1045,
1122
+ 1046,
1123
+ 1047,
1124
+ 1048,
1125
+ 1049,
1126
+ 1050,
1127
+ 1051,
1128
+ 1052,
1129
+ 1053,
1130
+ 1054,
1131
+ 1055,
1132
+ 1056,
1133
+ 1057,
1134
+ 1058,
1135
+ 1059,
1136
+ 1060,
1137
+ 1061,
1138
+ 1062,
1139
+ 1063,
1140
+ 1064,
1141
+ 1065,
1142
+ 1066,
1143
+ 1067,
1144
+ 1068,
1145
+ 1069,
1146
+ 1070,
1147
+ 1071,
1148
+ 1072,
1149
+ 1073,
1150
+ 1074,
1151
+ 1075,
1152
+ 1076,
1153
+ 1077,
1154
+ 1078,
1155
+ 1079,
1156
+ 1080,
1157
+ 1081,
1158
+ 1082,
1159
+ 1083,
1160
+ 1084,
1161
+ 1085,
1162
+ 1086,
1163
+ 1087,
1164
+ 1088,
1165
+ 1089,
1166
+ 1090,
1167
+ 1091,
1168
+ 1092,
1169
+ 1093,
1170
+ 1094,
1171
+ 1095,
1172
+ 1096,
1173
+ 1097,
1174
+ 1098,
1175
+ 1099,
1176
+ 1100,
1177
+ 1101,
1178
+ 1102,
1179
+ 1103,
1180
+ 1104,
1181
+ 1105,
1182
+ 1106,
1183
+ 1107,
1184
+ 1108,
1185
+ 1109,
1186
+ 1110,
1187
+ 1111,
1188
+ 1112,
1189
+ 1113,
1190
+ 1114,
1191
+ 1115,
1192
+ 1116,
1193
+ 1117,
1194
+ 1118,
1195
+ 1119,
1196
+ 1120,
1197
+ 1121,
1198
+ 1122,
1199
+ 1123,
1200
+ 1124,
1201
+ 1125,
1202
+ 1126,
1203
+ 1127,
1204
+ 1128,
1205
+ 1129,
1206
+ 1130,
1207
+ 1131,
1208
+ 1132,
1209
+ 1133,
1210
+ 1134,
1211
+ 1135,
1212
+ 1136,
1213
+ 1137,
1214
+ 1138,
1215
+ 1139,
1216
+ 1140,
1217
+ 1141,
1218
+ 1142,
1219
+ 1143,
1220
+ 1144,
1221
+ 1145,
1222
+ 1146,
1223
+ 1147,
1224
+ 1148,
1225
+ 1149,
1226
+ 1150,
1227
+ 1151,
1228
+ 1152,
1229
+ 1153,
1230
+ 1154,
1231
+ 1155,
1232
+ 1156,
1233
+ 1157,
1234
+ 1158,
1235
+ 1159,
1236
+ 1160,
1237
+ 1161,
1238
+ 1162,
1239
+ 1163,
1240
+ 1164,
1241
+ 1165,
1242
+ 1166,
1243
+ 1167,
1244
+ 1168,
1245
+ 1169,
1246
+ 1170,
1247
+ 1171,
1248
+ 1172,
1249
+ 1173,
1250
+ 1174,
1251
+ 1175,
1252
+ 1176,
1253
+ 1177,
1254
+ 1178,
1255
+ 1179,
1256
+ 1180,
1257
+ 1181,
1258
+ 1182,
1259
+ 1183,
1260
+ 1184,
1261
+ 1185,
1262
+ 1186,
1263
+ 1187,
1264
+ 1188,
1265
+ 1189,
1266
+ 1190,
1267
+ 1191,
1268
+ 1192,
1269
+ 1193,
1270
+ 1194,
1271
+ 1195,
1272
+ 1196,
1273
+ 1197,
1274
+ 1198,
1275
+ 1199,
1276
+ 1200,
1277
+ 1201,
1278
+ 1202,
1279
+ 1203,
1280
+ 1204,
1281
+ 1205,
1282
+ 1206,
1283
+ 1207,
1284
+ 1208,
1285
+ 1209,
1286
+ 1210,
1287
+ 1211,
1288
+ 1212,
1289
+ 1213,
1290
+ 1214,
1291
+ 1215,
1292
+ 1216,
1293
+ 1217,
1294
+ 1218,
1295
+ 1219,
1296
+ 1220,
1297
+ 1221,
1298
+ 1222,
1299
+ 1223,
1300
+ 1224,
1301
+ 1225,
1302
+ 1226,
1303
+ 1227,
1304
+ 1228,
1305
+ 1229,
1306
+ 1230,
1307
+ 1231,
1308
+ 1232,
1309
+ 1233,
1310
+ 1234,
1311
+ 1235,
1312
+ 1236,
1313
+ 1237,
1314
+ 1238,
1315
+ 1239,
1316
+ 1240,
1317
+ 1241,
1318
+ 1242,
1319
+ 1243,
1320
+ 1244,
1321
+ 1245,
1322
+ 1246,
1323
+ 1247,
1324
+ 1248,
1325
+ 1249,
1326
+ 1250,
1327
+ 1251,
1328
+ 1252,
1329
+ 1253,
1330
+ 1254,
1331
+ 1255,
1332
+ 1256,
1333
+ 1257,
1334
+ 1258,
1335
+ 1259,
1336
+ 1260,
1337
+ 1261,
1338
+ 1262,
1339
+ 1263,
1340
+ 1264,
1341
+ 1265,
1342
+ 1266,
1343
+ 1267,
1344
+ 1268,
1345
+ 1269,
1346
+ 1270,
1347
+ 1271,
1348
+ 1272,
1349
+ 1273,
1350
+ 1274,
1351
+ 1275,
1352
+ 1276,
1353
+ 1277,
1354
+ 1278,
1355
+ 1279,
1356
+ 1280,
1357
+ 1281,
1358
+ 1282,
1359
+ 1283,
1360
+ 1284,
1361
+ 1285,
1362
+ 1286,
1363
+ 1287,
1364
+ 1288,
1365
+ 1289,
1366
+ 1290,
1367
+ 1291
1368
+ ]
1369
+ }
1370
+ ]
checkpoints/040000/training_state/optimizer_state.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:e8f363d7bd040a29cb2cc6f38a42083fd3c425f5a7ff062a180fca9f11c8c1ab
3
+ size 18260308060
checkpoints/040000/training_state/rng_state.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f369902ca48cb1afe3c3d68128eaf91829de8eb8e725a54a3a426d77f2c807e6
3
+ size 15708
checkpoints/040000/training_state/scheduler_state.json ADDED
@@ -0,0 +1,24 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "base_lrs": [
3
+ 1e-05,
4
+ 5e-06,
5
+ 5e-06,
6
+ 5e-05
7
+ ],
8
+ "last_epoch": 40000,
9
+ "_step_count": 40001,
10
+ "_is_initial": false,
11
+ "_get_lr_called_within_step": false,
12
+ "_last_lr": [
13
+ 1e-06,
14
+ 5e-07,
15
+ 5e-07,
16
+ 4.9999999999999996e-06
17
+ ],
18
+ "lr_lambdas": [
19
+ null,
20
+ null,
21
+ null,
22
+ null
23
+ ]
24
+ }
checkpoints/040000/training_state/training_step.json ADDED
@@ -0,0 +1,5 @@
 
 
 
 
 
 
1
+ {
2
+ "step": 40000,
3
+ "num_processes": 1,
4
+ "batch_size": 16
5
+ }
config.json ADDED
@@ -0,0 +1,139 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "type": "molmoact2",
3
+ "n_obs_steps": 1,
4
+ "input_features": {
5
+ "observation.state": {
6
+ "type": "STATE",
7
+ "shape": [
8
+ 6
9
+ ]
10
+ },
11
+ "observation.images.front": {
12
+ "type": "VISUAL",
13
+ "shape": [
14
+ 3,
15
+ 480,
16
+ 640
17
+ ]
18
+ },
19
+ "observation.images.wrist": {
20
+ "type": "VISUAL",
21
+ "shape": [
22
+ 3,
23
+ 480,
24
+ 640
25
+ ]
26
+ }
27
+ },
28
+ "output_features": {
29
+ "action": {
30
+ "type": "ACTION",
31
+ "shape": [
32
+ 6
33
+ ]
34
+ }
35
+ },
36
+ "device": "cuda",
37
+ "use_amp": false,
38
+ "use_peft": false,
39
+ "push_to_hub": true,
40
+ "repo_id": "tsangb34/molmoact2-so101-stack-cubes-40k",
41
+ "private": false,
42
+ "tags": [
43
+ "molmoact2",
44
+ "so101",
45
+ "robocolosseum",
46
+ "stack-cubes"
47
+ ],
48
+ "license": null,
49
+ "pretrained_path": null,
50
+ "pretrained_revision": null,
51
+ "checkpoint_path": "allenai/MolmoAct2",
52
+ "checkpoint_revision": null,
53
+ "checkpoint_force_download": false,
54
+ "chunk_size": 32,
55
+ "n_action_steps": 32,
56
+ "action_mode": "continuous",
57
+ "inference_action_mode": "continuous",
58
+ "discrete_action_tokenizer": "allenai/MolmoAct2-FAST-Tokenizer",
59
+ "discrete_generation_max_steps": null,
60
+ "norm_tag": null,
61
+ "setup_type": "single so100/so101 robotic arm in molmoact2",
62
+ "control_mode": "absolute joint pose",
63
+ "image_keys": [
64
+ "observation.images.front",
65
+ "observation.images.wrist"
66
+ ],
67
+ "normalize_language": true,
68
+ "add_setup_tokens": true,
69
+ "add_control_tokens": true,
70
+ "normalize_gripper": true,
71
+ "num_state_tokens": 256,
72
+ "max_sequence_length": null,
73
+ "expected_max_action_dim": 32,
74
+ "num_flow_timesteps": 8,
75
+ "flow_matching_cutoff": 1.0,
76
+ "flow_matching_time_offset": 0.001,
77
+ "flow_matching_time_scale": 0.999,
78
+ "flow_matching_beta_alpha": 1.0,
79
+ "flow_matching_beta_beta": 1.5,
80
+ "num_inference_steps": null,
81
+ "mask_action_dim_padding": true,
82
+ "enable_inference_cuda_graph": true,
83
+ "per_episode_seed": false,
84
+ "eval_seed": null,
85
+ "rtc_config": null,
86
+ "joint_signs": null,
87
+ "joint_offsets": null,
88
+ "enable_lora_vlm": false,
89
+ "lora_rank": 64,
90
+ "lora_alpha": 16,
91
+ "lora_dropout": 0.05,
92
+ "lora_bias": "none",
93
+ "enable_lora_action_expert": false,
94
+ "enable_knowledge_insulation": false,
95
+ "freeze_embedding": true,
96
+ "train_action_expert_only": false,
97
+ "gradient_checkpointing": true,
98
+ "model_dtype": "bfloat16",
99
+ "softmax_auxiliary_loss": true,
100
+ "softmax_auxiliary_loss_scale": 0.0001,
101
+ "discrete_loss_token_weighting": "root_subsegments_root_tokens",
102
+ "optimizer_lr": 1e-05,
103
+ "optimizer_vit_lr": 5e-06,
104
+ "optimizer_connector_lr": 5e-06,
105
+ "optimizer_action_expert_lr": 5e-05,
106
+ "optimizer_betas": [
107
+ 0.9,
108
+ 0.95
109
+ ],
110
+ "optimizer_eps": 1e-06,
111
+ "optimizer_weight_decay": 0.0,
112
+ "optimizer_grad_clip_norm": 1.0,
113
+ "scheduler_warmup_steps": 200,
114
+ "scheduler_decay_steps": 40000,
115
+ "scheduler_decay_lr": 1e-06,
116
+ "normalization_mapping": {
117
+ "ACTION": "QUANTILES",
118
+ "STATE": "QUANTILES",
119
+ "VISUAL": "IDENTITY"
120
+ },
121
+ "dataset_feature_names": {
122
+ "action": [
123
+ "shoulder_pan.pos",
124
+ "shoulder_lift.pos",
125
+ "elbow_flex.pos",
126
+ "wrist_flex.pos",
127
+ "wrist_roll.pos",
128
+ "gripper.pos"
129
+ ],
130
+ "observation.state": [
131
+ "shoulder_pan.pos",
132
+ "shoulder_lift.pos",
133
+ "elbow_flex.pos",
134
+ "wrist_flex.pos",
135
+ "wrist_roll.pos",
136
+ "gripper.pos"
137
+ ]
138
+ }
139
+ }
model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:b4a641939db28ef73392c98dcb7116767cd2cbabe7cbdca68836ba88efc793bc
3
+ size 10884573720
policy_postprocessor.json ADDED
@@ -0,0 +1,43 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "name": "policy_postprocessor",
3
+ "steps": [
4
+ {
5
+ "registry_name": "molmoact2_clamp_action",
6
+ "config": {}
7
+ },
8
+ {
9
+ "registry_name": "molmoact2_masked_unnormalizer",
10
+ "config": {
11
+ "eps": 1e-08,
12
+ "features": {
13
+ "action": {
14
+ "type": "ACTION",
15
+ "shape": [
16
+ 6
17
+ ]
18
+ }
19
+ },
20
+ "norm_map": {
21
+ "ACTION": "QUANTILES",
22
+ "STATE": "QUANTILES",
23
+ "VISUAL": "IDENTITY"
24
+ }
25
+ },
26
+ "state_file": "policy_postprocessor_step_1_molmoact2_masked_unnormalizer.safetensors"
27
+ },
28
+ {
29
+ "registry_name": "molmoact2_action_frame_transform",
30
+ "config": {
31
+ "joint_signs": null,
32
+ "joint_offsets": null
33
+ }
34
+ },
35
+ {
36
+ "registry_name": "device_processor",
37
+ "config": {
38
+ "device": "cpu",
39
+ "float_dtype": null
40
+ }
41
+ }
42
+ ]
43
+ }
policy_postprocessor_step_1_molmoact2_masked_unnormalizer.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ee6a8cf8d7533d272953fddc06a419c5b7c89cac01d3dba8db5b3cee5efb107f
3
+ size 7744
policy_preprocessor.json ADDED
@@ -0,0 +1,100 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "name": "policy_preprocessor",
3
+ "steps": [
4
+ {
5
+ "registry_name": "rename_observations_processor",
6
+ "config": {
7
+ "rename_map": {}
8
+ }
9
+ },
10
+ {
11
+ "registry_name": "to_batch_processor",
12
+ "config": {}
13
+ },
14
+ {
15
+ "registry_name": "molmoact2_state_frame_transform",
16
+ "config": {
17
+ "joint_signs": null,
18
+ "joint_offsets": null
19
+ }
20
+ },
21
+ {
22
+ "registry_name": "molmoact2_masked_normalizer",
23
+ "config": {
24
+ "eps": 1e-08,
25
+ "features": {
26
+ "observation.state": {
27
+ "type": "STATE",
28
+ "shape": [
29
+ 6
30
+ ]
31
+ },
32
+ "observation.images.front": {
33
+ "type": "VISUAL",
34
+ "shape": [
35
+ 3,
36
+ 480,
37
+ 640
38
+ ]
39
+ },
40
+ "observation.images.wrist": {
41
+ "type": "VISUAL",
42
+ "shape": [
43
+ 3,
44
+ 480,
45
+ 640
46
+ ]
47
+ },
48
+ "action": {
49
+ "type": "ACTION",
50
+ "shape": [
51
+ 6
52
+ ]
53
+ }
54
+ },
55
+ "norm_map": {
56
+ "ACTION": "QUANTILES",
57
+ "STATE": "QUANTILES",
58
+ "VISUAL": "IDENTITY"
59
+ }
60
+ },
61
+ "state_file": "policy_preprocessor_step_3_molmoact2_masked_normalizer.safetensors"
62
+ },
63
+ {
64
+ "registry_name": "molmoact2_clamp_normalized",
65
+ "config": {}
66
+ },
67
+ {
68
+ "registry_name": "molmoact2_pack_inputs",
69
+ "config": {
70
+ "checkpoint_path": "allenai/MolmoAct2",
71
+ "checkpoint_revision": null,
72
+ "checkpoint_force_download": false,
73
+ "action_mode": "continuous",
74
+ "discrete_action_tokenizer": "allenai/MolmoAct2-FAST-Tokenizer",
75
+ "image_keys": [
76
+ "observation.images.front",
77
+ "observation.images.wrist"
78
+ ],
79
+ "allow_image_key_fallback": false,
80
+ "setup_type": "single so100/so101 robotic arm in molmoact2",
81
+ "control_mode": "absolute joint pose",
82
+ "normalize_language": true,
83
+ "add_setup_tokens": true,
84
+ "add_control_tokens": true,
85
+ "num_state_tokens": 256,
86
+ "max_sequence_length": null,
87
+ "chunk_size": 32,
88
+ "max_action_dim": 32,
89
+ "env_action_dim": 6
90
+ }
91
+ },
92
+ {
93
+ "registry_name": "device_processor",
94
+ "config": {
95
+ "device": "cuda",
96
+ "float_dtype": null
97
+ }
98
+ }
99
+ ]
100
+ }
policy_preprocessor_step_3_molmoact2_masked_normalizer.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:95f69bcc0e1bee273f9668382626418fd89e92d69d6e42aa99c4a458f9a06c31
3
+ size 7752
train_config.json ADDED
@@ -0,0 +1,275 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "dataset": {
3
+ "repo_id": "Jiamo0912/so101-stack-cubes",
4
+ "repo_type": "dataset",
5
+ "root": "/scratch/e1583535/datasets/so101-stack-cubes",
6
+ "episodes": null,
7
+ "image_transforms": {
8
+ "enable": true,
9
+ "max_num_transforms": 3,
10
+ "random_order": false,
11
+ "tfs": {
12
+ "crop": {
13
+ "weight": 1.0,
14
+ "type": "RandomResizedCrop",
15
+ "kwargs": {
16
+ "size": [
17
+ 480,
18
+ 640
19
+ ],
20
+ "scale": [
21
+ 0.9025,
22
+ 0.9025
23
+ ],
24
+ "ratio": [
25
+ 1.3333333,
26
+ 1.3333333
27
+ ]
28
+ }
29
+ },
30
+ "rotation": {
31
+ "weight": 1.0,
32
+ "type": "RandomRotation",
33
+ "kwargs": {
34
+ "degrees": [
35
+ -5.0,
36
+ 5.0
37
+ ],
38
+ "interpolation": 2
39
+ }
40
+ },
41
+ "color": {
42
+ "weight": 1.0,
43
+ "type": "ColorJitter",
44
+ "kwargs": {
45
+ "brightness": 0.2,
46
+ "contrast": [
47
+ 0.8,
48
+ 1.2
49
+ ],
50
+ "saturation": [
51
+ 0.8,
52
+ 1.2
53
+ ],
54
+ "hue": 0.05
55
+ }
56
+ }
57
+ }
58
+ },
59
+ "revision": null,
60
+ "use_imagenet_stats": true,
61
+ "video_backend": "pyav",
62
+ "return_uint8": false,
63
+ "depth_output_unit": "mm",
64
+ "streaming": false,
65
+ "eval_split": 0.0
66
+ },
67
+ "env": null,
68
+ "policy": {
69
+ "type": "molmoact2",
70
+ "n_obs_steps": 1,
71
+ "input_features": {
72
+ "observation.state": {
73
+ "type": "STATE",
74
+ "shape": [
75
+ 6
76
+ ]
77
+ },
78
+ "observation.images.front": {
79
+ "type": "VISUAL",
80
+ "shape": [
81
+ 3,
82
+ 480,
83
+ 640
84
+ ]
85
+ },
86
+ "observation.images.wrist": {
87
+ "type": "VISUAL",
88
+ "shape": [
89
+ 3,
90
+ 480,
91
+ 640
92
+ ]
93
+ }
94
+ },
95
+ "output_features": {
96
+ "action": {
97
+ "type": "ACTION",
98
+ "shape": [
99
+ 6
100
+ ]
101
+ }
102
+ },
103
+ "device": "cuda",
104
+ "use_amp": false,
105
+ "use_peft": false,
106
+ "push_to_hub": true,
107
+ "repo_id": "tsangb34/molmoact2-so101-stack-cubes-40k",
108
+ "private": false,
109
+ "tags": [
110
+ "molmoact2",
111
+ "so101",
112
+ "robocolosseum",
113
+ "stack-cubes"
114
+ ],
115
+ "license": null,
116
+ "pretrained_path": null,
117
+ "pretrained_revision": null,
118
+ "checkpoint_path": "allenai/MolmoAct2",
119
+ "checkpoint_revision": null,
120
+ "checkpoint_force_download": false,
121
+ "chunk_size": 32,
122
+ "n_action_steps": 32,
123
+ "action_mode": "continuous",
124
+ "inference_action_mode": "continuous",
125
+ "discrete_action_tokenizer": "allenai/MolmoAct2-FAST-Tokenizer",
126
+ "discrete_generation_max_steps": null,
127
+ "norm_tag": null,
128
+ "setup_type": "single so100/so101 robotic arm in molmoact2",
129
+ "control_mode": "absolute joint pose",
130
+ "image_keys": [
131
+ "observation.images.front",
132
+ "observation.images.wrist"
133
+ ],
134
+ "normalize_language": true,
135
+ "add_setup_tokens": true,
136
+ "add_control_tokens": true,
137
+ "normalize_gripper": true,
138
+ "num_state_tokens": 256,
139
+ "max_sequence_length": null,
140
+ "expected_max_action_dim": 32,
141
+ "num_flow_timesteps": 8,
142
+ "flow_matching_cutoff": 1.0,
143
+ "flow_matching_time_offset": 0.001,
144
+ "flow_matching_time_scale": 0.999,
145
+ "flow_matching_beta_alpha": 1.0,
146
+ "flow_matching_beta_beta": 1.5,
147
+ "num_inference_steps": null,
148
+ "mask_action_dim_padding": true,
149
+ "enable_inference_cuda_graph": true,
150
+ "per_episode_seed": false,
151
+ "eval_seed": null,
152
+ "rtc_config": null,
153
+ "joint_signs": null,
154
+ "joint_offsets": null,
155
+ "enable_lora_vlm": false,
156
+ "lora_rank": 64,
157
+ "lora_alpha": 16,
158
+ "lora_dropout": 0.05,
159
+ "lora_bias": "none",
160
+ "enable_lora_action_expert": false,
161
+ "enable_knowledge_insulation": false,
162
+ "freeze_embedding": true,
163
+ "train_action_expert_only": false,
164
+ "gradient_checkpointing": true,
165
+ "model_dtype": "bfloat16",
166
+ "softmax_auxiliary_loss": true,
167
+ "softmax_auxiliary_loss_scale": 0.0001,
168
+ "discrete_loss_token_weighting": "root_subsegments_root_tokens",
169
+ "optimizer_lr": 1e-05,
170
+ "optimizer_vit_lr": 5e-06,
171
+ "optimizer_connector_lr": 5e-06,
172
+ "optimizer_action_expert_lr": 5e-05,
173
+ "optimizer_betas": [
174
+ 0.9,
175
+ 0.95
176
+ ],
177
+ "optimizer_eps": 1e-06,
178
+ "optimizer_weight_decay": 0.0,
179
+ "optimizer_grad_clip_norm": 1.0,
180
+ "scheduler_warmup_steps": 200,
181
+ "scheduler_decay_steps": 40000,
182
+ "scheduler_decay_lr": 1e-06,
183
+ "normalization_mapping": {
184
+ "ACTION": "QUANTILES",
185
+ "STATE": "QUANTILES",
186
+ "VISUAL": "IDENTITY"
187
+ },
188
+ "dataset_feature_names": {
189
+ "action": [
190
+ "shoulder_pan.pos",
191
+ "shoulder_lift.pos",
192
+ "elbow_flex.pos",
193
+ "wrist_flex.pos",
194
+ "wrist_roll.pos",
195
+ "gripper.pos"
196
+ ],
197
+ "observation.state": [
198
+ "shoulder_pan.pos",
199
+ "shoulder_lift.pos",
200
+ "elbow_flex.pos",
201
+ "wrist_flex.pos",
202
+ "wrist_roll.pos",
203
+ "gripper.pos"
204
+ ]
205
+ }
206
+ },
207
+ "reward_model": null,
208
+ "output_dir": "/scratch/Projects/CFP-05/CFP05-CF-002/robocolosseum-finetuned-models/molmoact2-so101-stack-cubes-40k",
209
+ "job_name": "molmoact2-so101-stack-cubes-40k",
210
+ "resume": false,
211
+ "seed": 1000,
212
+ "cudnn_deterministic": false,
213
+ "num_workers": 4,
214
+ "batch_size": 16,
215
+ "prefetch_factor": 4,
216
+ "persistent_workers": true,
217
+ "dataloader_multiprocessing_context": "spawn",
218
+ "steps": 40000,
219
+ "env_eval_freq": 20000,
220
+ "log_freq": 20,
221
+ "eval_steps": 0,
222
+ "max_eval_samples": 0,
223
+ "tolerance_s": 0.0001,
224
+ "save_checkpoint": true,
225
+ "save_freq": 20000,
226
+ "use_policy_training_preset": true,
227
+ "optimizer": {
228
+ "type": "adamw",
229
+ "lr": 1e-05,
230
+ "weight_decay": 0.0,
231
+ "grad_clip_norm": 1.0,
232
+ "betas": [
233
+ 0.9,
234
+ 0.95
235
+ ],
236
+ "eps": 1e-06
237
+ },
238
+ "scheduler": {
239
+ "type": "cosine_decay_with_warmup",
240
+ "num_warmup_steps": 200,
241
+ "num_decay_steps": 40000,
242
+ "peak_lr": 1e-05,
243
+ "decay_lr": 1e-06
244
+ },
245
+ "eval": {
246
+ "n_episodes": 50,
247
+ "batch_size": 50,
248
+ "use_async_envs": true,
249
+ "recording": false,
250
+ "recording_repo_id": null,
251
+ "recording_private": false
252
+ },
253
+ "wandb": {
254
+ "enable": true,
255
+ "disable_artifact": true,
256
+ "project": "RoboColosseum",
257
+ "entity": "tsangb34-national-university-of-singapore-students-union",
258
+ "notes": null,
259
+ "run_id": "molmoact2-so101-stack-cubes-40k",
260
+ "mode": null,
261
+ "add_tags": true
262
+ },
263
+ "peft": null,
264
+ "job": {
265
+ "target": null,
266
+ "image": "huggingface/lerobot-gpu:latest",
267
+ "timeout": "2d",
268
+ "detach": false,
269
+ "tags": []
270
+ },
271
+ "save_checkpoint_to_hub": true,
272
+ "sample_weighting": null,
273
+ "rename_map": {},
274
+ "checkpoint_path": null
275
+ }