| # MSK-Bench Weights and Experiment Artifacts |
|
|
| The `data/` directory contains policy weights for MyoSuite/MSK-Bench tasks, together with their configuration files, training logs, and evaluation results. The top-level folders identify algorithms or experiment series. Weights from different folders are not interchangeable, even when their task names look similar. |
|
|
| ## Where to find weights |
|
|
| | Directory | Contents | Start with | |
| | --- | --- | --- | |
| | [`deprl/`](./deprl/) | DEPRL walking, running, and stair-climbing baselines; the configurations use DEP + MPO | `walk/`, `run/`, or `stair/` → `checkpoints/best.pt` | |
| | [`msgym/`](./msgym/) | `SAC_DynSyn` policies organized by MyoSuite environment and experiment run | `<environment>/<run>/checkpoint/best_model.zip` | |
| | [`ppo/`](./ppo/) | PPO weights, normalization state, evaluation reports, and training logs | `ppo_<task>_checkpoints/best_model.zip` | |
| | [`residualrl/`](./residualrl/) | ResidualRL walking, running, and stair-climbing experiments; the configurations use DEP + MPO | `<task>/checkpoints/best.pt` | |
| | [`sac/`](./sac/) | SAC weights, normalization state, evaluation reports, and training logs | `sac_<task>_checkpoints/best_model.zip` | |
|
|
| The word `best` is part of the original filenames; this README does not infer which metric was used to select those checkpoints. To reproduce an evaluation, also check the configuration, environment version, and any associated preprocessing or normalization state. |
|
|
| ## `deprl/`: DEPRL baselines |
|
|
| This directory has three task folders. Each numbered subdirectory contains one experiment run. The environment names below come from the corresponding `config.yaml` files. |
|
|
| | Subdirectory | Environment in the configuration | Purpose | |
| | --- | --- | --- | |
| | `walk/260521.131030/` | `myoLegWalk-v0` | Leg-walking policy and evaluation results | |
| | `run/260502.131631/` | `MyoRun-v0` | Running policy and evaluation results | |
| | `stair/260202.174137/` | `myoLegStair-v0` | Stair-climbing policy and evaluation results | |
|
|
| A run typically contains: |
|
|
| - `config.yaml`: environment, DEP, MPO, training, and checkpoint settings. |
| - `script.py`: the saved training script, useful when inspecting or reproducing the experiment. |
| - `checkpoints/best.pt`: the checkpoint labeled best; `step_*.pt` files are checkpoints saved at specific training steps. |
| - `checkpoints/logger.pt` and `checkpoints/time.pt`: saved logging or training-progress state, not standalone policy weights. |
| - `log.csv`: training and test metrics. |
| - `eval_*.png` and `*_compliance_report.csv`: plots and numerical reports for energy, jitter, muscle actions, and physical joints. |
| - `EMG_*`: electromyography-related results, where present. |
|
|
| ## `msgym/`: Policies grouped by environment and run |
|
|
| The first level is a MyoSuite environment, such as `myoLegWalk-v0/` or `MyoRun-v0/`. The next level, such as `0625-005737_0/`, contains the artifacts from one run. Some environments have several runs; keep a model with the configuration and environment state from the same run. All current run configuration JSON files identify the agent as `SAC_DynSyn`. |
|
|
| Environment folders: |
|
|
| ```text |
| MyoBunnyHop-v0 MyoCatch-v0 MyoChinUp-v0 |
| MyoCrawl-v0 MyoDoorOpen-v0 MyoHurdle-v0 |
| myoLegBalance-v0 myoLegSquat-v0 myoLegStair-v0 |
| myoLegStandRandom-v0 myoLegWalk-v0 MyoMazeWalk-v0 |
| MyoPoleWalk-v0 myoPowerlift-v0 MyoRun-v0 |
| MyoSidestep-v0 MyoSingleLegStance-v0 MyoSteppingStones-v0 |
| myoWalkAndSit-v0 |
| ``` |
|
|
| Common files in a run: |
|
|
| | Path or file | Purpose | |
| | --- | --- | |
| | `<task>.json` | Algorithm, environment, network, and training parameters; the filename varies by task. | |
| | `checkpoint/best_model.zip` and `final_model.zip` | Stable-Baselines3 policy archives labeled best and final by the original experiment. Some runs may contain only one of them. | |
| | `checkpoint/best_env.zip` and `final_env.zip` | Environment state saved alongside the models. **Despite their `.zip` extensions, these are Python pickle streams, not ordinary ZIP archives.** Load them using the original experiment's code. | |
| | `eval/evaluations.npz` | Periodic evaluation data in NumPy format. | |
| | `monitor/*.monitor.csv` | Episode records, including reward `r`, length `l`, and time `t`. | |
| | `events.out.tfevents.*` | TensorBoard event logs. | |
|
|
| ## `ppo/` and `sac/`: Task-specific checkpoints and logs |
|
|
| Both directories pair a `<algorithm>_<task>_checkpoints/` folder (models and evaluation results) with a `<algorithm>_<task>_logs/` folder (evaluation and training logs). For example, `ppo_balance_checkpoints/` pairs with `ppo_balance_logs/`. SAC uses the `sac_` prefix. |
|
|
| Current task suffixes in `ppo/`: |
|
|
| ```text |
| balance, bunny, catch, chinup, crawl, door, hurdle, maze, pole_walk, |
| powerlift, reach, run, sidestep, sit, slidehard, squat, stair, stance, |
| stepping_stones, walkandsit |
| ``` |
|
|
| Current task suffixes in `sac/`: |
|
|
| ```text |
| balance, bunnyhop, catch, chinup, crawl, door, hurdle, maze, pole_walk, |
| powerlift, reach, run, sidestep, singlelegstance, sit, slidehard, squat, |
| stair, stepping_stones, walkandsit |
| ``` |
|
|
| These suffixes preserve the original directory names. The two algorithms do not have exactly the same task set or spelling. Key files in each `*_checkpoints/` folder are: |
|
|
| - `best_model.zip`: a Stable-Baselines3 policy archive. |
| - `vec_normalize.pkl`: vector-environment normalization state associated with that task's model; check and use the matching file when evaluating. |
| - `eval_*_energy_plot.png` and `eval_*_jitter_report.png`: energy and jitter evaluation plots. |
| - `Muscle_Actions_compliance_report.csv` and `Physical_Joints_compliance_report.csv`: per-dimension muscle-action and joint statistics, including derivative-energy and jerk fields. |
|
|
| The matching `*_logs/` folder typically contains `eval_logs/evaluations.npz` (evaluation data) and `events.out.tfevents.*` under `PPO_1/` or `SAC_1/` (TensorBoard training logs). |
|
|
| ## `residualrl/`: ResidualRL experiments |
|
|
| The `run/`, `stair/`, and `walk/` folders correspond to `MyoRunResidual-v0`, `MyoStairResidual-v0`, and `MyoWalkResidual-v0` in their configurations. Each task folder contains: |
|
|
| - `config.yaml`: environment, DEP, MPO, and training parameters. |
| - `script.py`: the saved training script. |
| - `checkpoints/best.pt`: the checkpoint labeled best. |
| - `checkpoints/step_*.pt`: checkpoints saved at specific training steps; `walk/` retains more intermediate steps. |
| - `checkpoints/logger.pt` and `checkpoints/time.pt`: logging or training-progress state. |
| - `log.csv`: training and test metrics. |
|
|
| ## Keeping files together |
|
|
| 1. Choose the **algorithm and environment/task** first, then select a weight file inside that directory. Similar task names do not make weights from different algorithms interchangeable. |
| 2. For `ppo/` and `sac/`, check `best_model.zip` together with the task's `vec_normalize.pkl`. For `msgym/`, keep the model with the JSON configuration and environment state from the same run. |
| 3. Loading `.pt`, `.pkl`, or `msgym`'s `*_env.zip` files may require the corresponding Python classes and library versions. These are experiment artifacts; the weight files alone are not guaranteed to provide a working inference setup. |
| 4. `log.csv`, `*.npz`, `*.monitor.csv`, `events.out.tfevents.*`, and the evaluation plots are for inspecting training or comparing experiments. They are not policy weights. |
|
|
|
|
