| --- |
| license: other |
| license_name: ncsl |
| license_link: https://github.com/NVlabs/ProtoMotions/blob/main/LICENSE |
| datasets: |
| - realdream-ai/AMASS |
| - TeoGchx/HumanML3D |
| tags: |
| - humanoid |
| - reinforcementlearning |
| - robot |
| --- |
| |
| # Full codebase: |
| This model works with the [ProtoMotions](https://github.com/NVlabs/ProtoMotions/tree/main) environment. |
|
|
| # Masked Mimic -- SMPL Humanoid |
|
|
| <div float="center"> |
| <img src="https://github.com/NVlabs/ProtoMotions/blob/main/data/pretrained_models/masked_mimic/smpl/assets/inbetweening.gif?raw=true" width="300"/> |
| </div> |
| |
| # What is this? |
|
|
| - Pre-trained Masked Mimic agent for the SMPL (no fingers) humanoid. |
| - The goal of this model is to generate novel motions from partial constraints. |
| - It observes: |
| - Current pose |
| - 15 future poses |
| - Projected surrounding heightmap |
| - It can be constrained using: |
| - Any-joint-any-time. Any number of future states (defined via time). For each state, any subset of joints. Each joint constraint supports translation and/or rotation constraints. |
| - It predicts: |
| - Next action (PD target for each joint) |
|
|
|
|
| - Trained in IsaacLab. The model may not perform as well in IsaacGym/Genesis. |
|
|
| # Evaluating the model |
| To evaluate the model run the following command: |
|
|
| ``` |
| PYTHON_PATH protomotions/eval_agent.py +robot=smpl +simulator=isaaclab +motion_file=<path to motion file> +checkpoint=data/pretrained_models/masked_mimic/smpl/last.ckpt |
| ``` |
|
|
| - You should pick which `motion_file` to load. |
| - The model was trained and performs best in IsaacLab. Simulator can selected using the `simulator` flag --- performance may vary. |
| - For faster loading times use a flat terrain (config defaults to random heightmap) `+terrain=flat`. |
|
|
| For easy evaluation of a target-pose inbetween objective add the following flags |
| ``` |
| +opt=masked_mimic/constraints/no_constraint env.config.masked_mimic.masked_mimic_masking.target_pose_visible_prob=1 |
| ``` |
| The `no_constraint` yaml file turns off all constraints. Then we only enable the target_pose visibility. |
