File size: 7,373 Bytes
1e2c60a | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 | # MSK-Bench Weights and Experiment Artifacts
The `data/` directory contains policy weights for MyoSuite/MSK-Bench tasks, together with their configuration files, training logs, and evaluation results. The top-level folders identify algorithms or experiment series. Weights from different folders are not interchangeable, even when their task names look similar.
## Where to find weights
| Directory | Contents | Start with |
| --- | --- | --- |
| [`deprl/`](./deprl/) | DEPRL walking, running, and stair-climbing baselines; the configurations use DEP + MPO | `walk/`, `run/`, or `stair/` → `checkpoints/best.pt` |
| [`msgym/`](./msgym/) | `SAC_DynSyn` policies organized by MyoSuite environment and experiment run | `<environment>/<run>/checkpoint/best_model.zip` |
| [`ppo/`](./ppo/) | PPO weights, normalization state, evaluation reports, and training logs | `ppo_<task>_checkpoints/best_model.zip` |
| [`residualrl/`](./residualrl/) | ResidualRL walking, running, and stair-climbing experiments; the configurations use DEP + MPO | `<task>/checkpoints/best.pt` |
| [`sac/`](./sac/) | SAC weights, normalization state, evaluation reports, and training logs | `sac_<task>_checkpoints/best_model.zip` |
The word `best` is part of the original filenames; this README does not infer which metric was used to select those checkpoints. To reproduce an evaluation, also check the configuration, environment version, and any associated preprocessing or normalization state.
## `deprl/`: DEPRL baselines
This directory has three task folders. Each numbered subdirectory contains one experiment run. The environment names below come from the corresponding `config.yaml` files.
| Subdirectory | Environment in the configuration | Purpose |
| --- | --- | --- |
| `walk/260521.131030/` | `myoLegWalk-v0` | Leg-walking policy and evaluation results |
| `run/260502.131631/` | `MyoRun-v0` | Running policy and evaluation results |
| `stair/260202.174137/` | `myoLegStair-v0` | Stair-climbing policy and evaluation results |
A run typically contains:
- `config.yaml`: environment, DEP, MPO, training, and checkpoint settings.
- `script.py`: the saved training script, useful when inspecting or reproducing the experiment.
- `checkpoints/best.pt`: the checkpoint labeled best; `step_*.pt` files are checkpoints saved at specific training steps.
- `checkpoints/logger.pt` and `checkpoints/time.pt`: saved logging or training-progress state, not standalone policy weights.
- `log.csv`: training and test metrics.
- `eval_*.png` and `*_compliance_report.csv`: plots and numerical reports for energy, jitter, muscle actions, and physical joints.
- `EMG_*`: electromyography-related results, where present.
## `msgym/`: Policies grouped by environment and run
The first level is a MyoSuite environment, such as `myoLegWalk-v0/` or `MyoRun-v0/`. The next level, such as `0625-005737_0/`, contains the artifacts from one run. Some environments have several runs; keep a model with the configuration and environment state from the same run. All current run configuration JSON files identify the agent as `SAC_DynSyn`.
Environment folders:
```text
MyoBunnyHop-v0 MyoCatch-v0 MyoChinUp-v0
MyoCrawl-v0 MyoDoorOpen-v0 MyoHurdle-v0
myoLegBalance-v0 myoLegSquat-v0 myoLegStair-v0
myoLegStandRandom-v0 myoLegWalk-v0 MyoMazeWalk-v0
MyoPoleWalk-v0 myoPowerlift-v0 MyoRun-v0
MyoSidestep-v0 MyoSingleLegStance-v0 MyoSteppingStones-v0
myoWalkAndSit-v0
```
Common files in a run:
| Path or file | Purpose |
| --- | --- |
| `<task>.json` | Algorithm, environment, network, and training parameters; the filename varies by task. |
| `checkpoint/best_model.zip` and `final_model.zip` | Stable-Baselines3 policy archives labeled best and final by the original experiment. Some runs may contain only one of them. |
| `checkpoint/best_env.zip` and `final_env.zip` | Environment state saved alongside the models. **Despite their `.zip` extensions, these are Python pickle streams, not ordinary ZIP archives.** Load them using the original experiment's code. |
| `eval/evaluations.npz` | Periodic evaluation data in NumPy format. |
| `monitor/*.monitor.csv` | Episode records, including reward `r`, length `l`, and time `t`. |
| `events.out.tfevents.*` | TensorBoard event logs. |
## `ppo/` and `sac/`: Task-specific checkpoints and logs
Both directories pair a `<algorithm>_<task>_checkpoints/` folder (models and evaluation results) with a `<algorithm>_<task>_logs/` folder (evaluation and training logs). For example, `ppo_balance_checkpoints/` pairs with `ppo_balance_logs/`. SAC uses the `sac_` prefix.
Current task suffixes in `ppo/`:
```text
balance, bunny, catch, chinup, crawl, door, hurdle, maze, pole_walk,
powerlift, reach, run, sidestep, sit, slidehard, squat, stair, stance,
stepping_stones, walkandsit
```
Current task suffixes in `sac/`:
```text
balance, bunnyhop, catch, chinup, crawl, door, hurdle, maze, pole_walk,
powerlift, reach, run, sidestep, singlelegstance, sit, slidehard, squat,
stair, stepping_stones, walkandsit
```
These suffixes preserve the original directory names. The two algorithms do not have exactly the same task set or spelling. Key files in each `*_checkpoints/` folder are:
- `best_model.zip`: a Stable-Baselines3 policy archive.
- `vec_normalize.pkl`: vector-environment normalization state associated with that task's model; check and use the matching file when evaluating.
- `eval_*_energy_plot.png` and `eval_*_jitter_report.png`: energy and jitter evaluation plots.
- `Muscle_Actions_compliance_report.csv` and `Physical_Joints_compliance_report.csv`: per-dimension muscle-action and joint statistics, including derivative-energy and jerk fields.
The matching `*_logs/` folder typically contains `eval_logs/evaluations.npz` (evaluation data) and `events.out.tfevents.*` under `PPO_1/` or `SAC_1/` (TensorBoard training logs).
## `residualrl/`: ResidualRL experiments
The `run/`, `stair/`, and `walk/` folders correspond to `MyoRunResidual-v0`, `MyoStairResidual-v0`, and `MyoWalkResidual-v0` in their configurations. Each task folder contains:
- `config.yaml`: environment, DEP, MPO, and training parameters.
- `script.py`: the saved training script.
- `checkpoints/best.pt`: the checkpoint labeled best.
- `checkpoints/step_*.pt`: checkpoints saved at specific training steps; `walk/` retains more intermediate steps.
- `checkpoints/logger.pt` and `checkpoints/time.pt`: logging or training-progress state.
- `log.csv`: training and test metrics.
## Keeping files together
1. Choose the **algorithm and environment/task** first, then select a weight file inside that directory. Similar task names do not make weights from different algorithms interchangeable.
2. For `ppo/` and `sac/`, check `best_model.zip` together with the task's `vec_normalize.pkl`. For `msgym/`, keep the model with the JSON configuration and environment state from the same run.
3. Loading `.pt`, `.pkl`, or `msgym`'s `*_env.zip` files may require the corresponding Python classes and library versions. These are experiment artifacts; the weight files alone are not guaranteed to provide a working inference setup.
4. `log.csv`, `*.npz`, `*.monitor.csv`, `events.out.tfevents.*`, and the evaluation plots are for inspecting training or comparing experiments. They are not policy weights.
|