--- datasets: jstm/mpc_lerobot_pd_ee_pose_200 library_name: lerobot license: apache-2.0 model_name: molmoact2 pipeline_tag: robotics tags: - lerobot - robotics - molmoact2 - maniskill - mpc base_model: allenai/MolmoAct2-LIBERO --- # Model Card for molmoact2_mpc_200 MolmoAct2 fine-tuned on motion-planning demos for four MPC tabletop ManiSkill tasks (absolute end-effector pose / `pd_ee_pose`). Fine-tuned from [`allenai/MolmoAct2-LIBERO`](https://huggingface.co/allenai/MolmoAct2-LIBERO) using [LeRobot](https://github.com/huggingface/lerobot). ## Training Dataset - **Repository:** [`jstm/mpc_lerobot_pd_ee_pose_200`](https://huggingface.co/datasets/jstm/mpc_lerobot_pd_ee_pose_200) - **Control mode:** absolute end-effector pose (`pd_ee_pose`) - **Cameras:** `camera_center`, `camera_left`, `camera_wrist` (378×378) - **Total demonstrations:** 767 | Env | Demonstrations | | --- | ---: | | `PickCube-v2-wrist` | 192 | | `PushCube-v2` | 192 | | `LiftPegUpright-v2` | 191 | | `PullCubeTool-v2` | 192 | ## Model Details - **License:** apache-2.0 - **Base model:** `allenai/MolmoAct2-LIBERO` - **Policy type:** `molmoact2` - **Action mode:** `both` (use `--policy.inference_action_mode=continuous` at eval) - **Setup type:** `single franka robotic arm` - **Control mode:** `absolute end-effector pose` - **Chunk size / n_action_steps:** 10 - **Training steps:** 10000 ## Inputs & Outputs **Inputs** | Feature | Type | Shape | | --- | --- | --- | | `observation.state` | STATE | `(9,)` | | `observation.images.camera_center` | VISUAL | `(3, 378, 378)` | | `observation.images.camera_left` | VISUAL | `(3, 378, 378)` | | `observation.images.camera_wrist` | VISUAL | `(3, 378, 378)` | **Outputs** | Feature | Type | Shape | | --- | --- | --- | | `action` | ACTION | `(7,)` | ## Evaluation Example eval on an MPC tabletop task: ```bash lerobot-eval \ --policy.path=jstm/molmoact2_mpc_200 \ --policy.device=cuda \ --policy.inference_action_mode=continuous \ --trust_remote_code=true \ --env.type=maniskill \ --env.control_mode=pd_ee_pose \ --env.task="PickCube-v2-wrist::lift the red cube" \ --env.episode_length=100 \ --env.observation_height=378 \ --env.observation_width=378 \ --env.goal_height=0.1 \ --eval.n_episodes=100 \ --eval.batch_size=20 \ --output_dir=outputs/molmoact2_mpc_200__PickCube-v2-wrist ```