Reinforcement Learning
stable-baselines3
LunarLander-v2
deep-reinforcement-learning
Eval Results (legacy)
Instructions to use drap/ppo-LunarLander-v2 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- stable-baselines3
How to use drap/ppo-LunarLander-v2 with stable-baselines3:
from huggingface_sb3 import load_from_hub checkpoint = load_from_hub( repo_id="drap/ppo-LunarLander-v2", filename="{MODEL FILENAME}.zip", ) - Notebooks
- Google Colab
- Kaggle
Added more training iterations; adjusted hyperparams
Browse files- README.md +1 -1
- config.json +1 -1
- ppo-LunarLander-v2.zip +2 -2
- ppo-LunarLander-v2/data +67 -45
- ppo-LunarLander-v2/policy.optimizer.pth +2 -2
- ppo-LunarLander-v2/policy.pth +2 -2
- replay.mp4 +0 -0
- results.json +1 -1
README.md
CHANGED
|
@@ -16,7 +16,7 @@ model-index:
|
|
| 16 |
type: LunarLander-v2
|
| 17 |
metrics:
|
| 18 |
- type: mean_reward
|
| 19 |
-
value: -
|
| 20 |
name: mean_reward
|
| 21 |
verified: false
|
| 22 |
---
|
|
|
|
| 16 |
type: LunarLander-v2
|
| 17 |
metrics:
|
| 18 |
- type: mean_reward
|
| 19 |
+
value: -3.83 +/- 140.81
|
| 20 |
name: mean_reward
|
| 21 |
verified: false
|
| 22 |
---
|
config.json
CHANGED
|
@@ -1 +1 @@
|
|
| 1 |
-
{"policy_class": {":type:": "<class 'abc.ABCMeta'>", ":serialized:": "gAWVOwAAAAAAAACMIXN0YWJsZV9iYXNlbGluZXMzLmNvbW1vbi5wb2xpY2llc5SMEUFjdG9yQ3JpdGljUG9saWN5lJOULg==", "__module__": "stable_baselines3.common.policies", "__doc__": "\n Policy class for actor-critic algorithms (has both policy and value prediction).\n Used by A2C, PPO and the likes.\n\n :param observation_space: Observation space\n :param action_space: Action space\n :param lr_schedule: Learning rate schedule (could be constant)\n :param net_arch: The specification of the policy and value networks.\n :param activation_fn: Activation function\n :param ortho_init: Whether to use or not orthogonal initialization\n :param use_sde: Whether to use State Dependent Exploration or not\n :param log_std_init: Initial value for the log standard deviation\n :param full_std: Whether to use (n_features x n_actions) parameters\n for the std instead of only (n_features,) when using gSDE\n :param use_expln: Use ``expln()`` function instead of ``exp()`` to ensure\n a positive standard deviation (cf paper). It allows to keep variance\n above zero and prevent it from growing too fast. In practice, ``exp()`` is usually enough.\n :param squash_output: Whether to squash the output using a tanh function,\n this allows to ensure boundaries when using gSDE.\n :param features_extractor_class: Features extractor to use.\n :param features_extractor_kwargs: Keyword arguments\n to pass to the features extractor.\n :param share_features_extractor: If True, the features extractor is shared between the policy and value networks.\n :param normalize_images: Whether to normalize images or not,\n dividing by 255.0 (True by default)\n :param optimizer_class: The optimizer to use,\n ``th.optim.Adam`` by default\n :param optimizer_kwargs: Additional keyword arguments,\n excluding the learning rate, to pass to the optimizer\n ", "__init__": "<function ActorCriticPolicy.__init__ at 0x79f32a820790>", "_get_constructor_parameters": "<function ActorCriticPolicy._get_constructor_parameters at 0x79f32a820820>", "reset_noise": "<function ActorCriticPolicy.reset_noise at 0x79f32a8208b0>", "_build_mlp_extractor": "<function ActorCriticPolicy._build_mlp_extractor at 0x79f32a820940>", "_build": "<function ActorCriticPolicy._build at 0x79f32a8209d0>", "forward": "<function ActorCriticPolicy.forward at 0x79f32a820a60>", "extract_features": "<function ActorCriticPolicy.extract_features at 0x79f32a820af0>", "_get_action_dist_from_latent": "<function ActorCriticPolicy._get_action_dist_from_latent at 0x79f32a820b80>", "_predict": "<function ActorCriticPolicy._predict at 0x79f32a820c10>", "evaluate_actions": "<function ActorCriticPolicy.evaluate_actions at 0x79f32a820ca0>", "get_distribution": "<function ActorCriticPolicy.get_distribution at 0x79f32a820d30>", "predict_values": "<function ActorCriticPolicy.predict_values at 0x79f32a820dc0>", "__abstractmethods__": "frozenset()", "_abc_impl": "<_abc._abc_data object at 0x79f2ae5c2c40>"}, "verbose": 1, "policy_kwargs": {}, "num_timesteps": 1507328, "_total_timesteps": 1500000, "_num_timesteps_at_start": 0, "seed": null, "action_noise": null, "start_time": 1730703176396799499, "learning_rate": 0.0003, "tensorboard_log": null, "_last_obs": {":type:": "<class 'numpy.ndarray'>", ":serialized:": "gAWVdQIAAAAAAACMEm51bXB5LmNvcmUubnVtZXJpY5SMC19mcm9tYnVmZmVylJOUKJYAAgAAAAAAAMDWPT7peym8qWMOO/Gw57gSsZy9jJUsugAAgD8AAIA/M2WmvPAJuj9m/r+7DNO5vPisAL8aH6e+AAAAAAAAAAAaFVy9S7yZP5J7VL3avt68CYGYvYFqxL0AAAAAAAAAADOzNzt7Luw+o0TavZHcvr4aACg/iHDIvgAAAAAAAAAAmjxVPrMX2j7cl5g9/Qe+vkfobL9CZFw+AAAAAAAAAABmDPM8NlygP7uidj1bkl+9qPkwv6qmYb4AAAAAAAAAAE0Xkr2E7qY/rkS7u8Ly+byZw8M8RaamPAAAAAAAAAAAsDdZvl9RVz8eING+mzrWvmmP3D03/Tq/AAAAAAAAAABmA8G9bbKGPxYesb7uKoe9shngvR74Eb8AAAAAAAAAAAAQujpMyK4/w2viPJC3Ubyqwq07j8wGPQAAAAAAAAAApow7PsrDHzymsbq3FZGttXwUrT1LuOc2AACAPwAAgD+zZS699iguOUbnxLYiVsQvU8bru2tC9DUAAIA/AACAP2bV0jw3lqU/Ijy1vBOTo7zzLjC8oH54vAAAAAAAAAAA5gLgvhqBIT7CvnE+A0MYv9oy9745Fjg/AAAAAAAAAACtiwo+DchgP2iK075a8nO+RXhbPHNHF78AAAAAAAAAAJpUyryVZ6Y/62D3vKHCyTxCyim+qujOPAAAAAAAAAAAlIwFbnVtcHmUjAVkdHlwZZSTlIwCZjSUiYiHlFKUKEsDjAE8lE5OTkr/////Sv////9LAHSUYksQSwiGlIwBQ5R0lFKULg=="}, "_last_episode_starts": {":type:": "<class 'numpy.ndarray'>", ":serialized:": "gAWVgwAAAAAAAACMEm51bXB5LmNvcmUubnVtZXJpY5SMC19mcm9tYnVmZmVylJOUKJYQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAACUjAVudW1weZSMBWR0eXBllJOUjAJiMZSJiIeUUpQoSwOMAXyUTk5OSv////9K/////0sAdJRiSxCFlIwBQ5R0lFKULg=="}, "_last_original_obs": null, "_episode_num": 0, "use_sde": false, "sde_sample_freq": -1, "_current_progress_remaining": -0.004885333333333408, "_stats_window_size": 100, "ep_info_buffer": {":type:": "<class 'collections.deque'>", ":serialized:": "gAWVNwwAAAAAAACMC2NvbGxlY3Rpb25zlIwFZGVxdWWUk5QpS2SGlFKUKH2UKIwBcpRHwGZFtKyv9tOMAWyUS/GMAXSUR0CvSkied07sdX2UKGgGR8Bi2g4n4O+aaAdL42gIR0CvS3ez+m3wdX2UKGgGR8BjZtdJJ5E/aAdN6ANoCEdAr0xmq94/vHV9lChoBkdAb2ZF3pwCKmgHTWkBaAhHQK9Md4NZvDR1fZQoaAZHQG6VCosI3R5oB03GAWgIR0CvUIiWeHzpdX2UKGgGR8BAiC8nNPgvaAdL+GgIR0CvVLICdSVGdX2UKGgGR0BmcuUdJaq0aAdNLwJoCEdAr1SzWiDdxnV9lChoBkfAZP2YLsrupmgHTegDaAhHQK9VANWluWN1fZQoaAZHQG0NzUI9kjJoB03jAWgIR0CvVQlPJq7AdX2UKGgGR0BtRT4N7SiNaAdNwwFoCEdAr1Vk30f5lHV9lChoBkfAR2aZ+hGpdmgHTbABaAhHQK9ZfgWJrL11fZQoaAZHwEHEvexfOUtoB00GAWgIR0Cvacmhdt2tdX2UKGgGR8BjjK+vhZQpaAdN6ANoCEdAr2oUVk+X7nV9lChoBkfAILynk1dgOWgHTR0BaAhHQK9q6h0yP+51fZQoaAZHQFcxSK3uuzRoB03oA2gIR0CvaxD3/PxAdX2UKGgGR8BeTKMm4RVZaAdN6ANoCEdAr2vZtJnQIHV9lChoBkdAcFmyWiUPhGgHTVsBaAhHQK9uIekHlfZ1fZQoaAZHwGBBqYiPhhpoB03oA2gIR0CvcNhKlHjIdX2UKGgGR0Bx924SYgJUaAdNNwFoCEdAr3FcfYBeX3V9lChoBkdAa04z7di2D2gHTfUBaAhHQK90NiONo8J1fZQoaAZHwGT4NTcZccFoB03oA2gIR0CvdQqWszVMdX2UKGgGR8BmDvLq2SdOaAdNEwFoCEdAr3cYVuaWonV9lChoBkfAaDh3Gn4wiGgHTegDaAhHQK93wglnh891fZQoaAZHwDJSCuloDgZoB0vvaAhHQK97jB9kSVZ1fZQoaAZHwElFUaQ3gk1oB02NAWgIR0CvfFCKrJbMdX2UKGgGR8BiV2PtD2J0aAdN6ANoCEdAr32DiADq4nV9lChoBkfARR6+i8FpwmgHTUwBaAhHQK9/F1VYISl1fZQoaAZHQGuTd8zAN5NoB03HAWgIR0Cvf4k8A7xNdX2UKGgGR8BjqXta6jFiaAdN6ANoCEdAr4AI/RmbsnV9lChoBkfAZQ87muDBdmgHTegDaAhHQK+AGmdAgPp1fZQoaAZHQF5PMHbAUL5oB03oA2gIR0Cvg8UpEx7BdX2UKGgGR0BtHskv9LpSaAdNMAJoCEdAr4Si8nNPg3V9lChoBkfAZq5DFZPl+2gHTegDaAhHQK+HSbT+ee51fZQoaAZHwD6tnHvMKTloB0v+aAhHQK+HojO9nK51fZQoaAZHQGvUp9RaX8hoB03CAWgIR0Cvh95FocrBdX2UKGgGR8BpbBFkQPI5aAdNVQFoCEdAr4k94X40uXV9lChoBkdAbVDS1mapgmgHTX8BaAhHQK+MeUQkHD91fZQoaAZHwGFdyZSeiBZoB00sAWgIR0Cvl/m5c1O1dX2UKGgGR0AwVTW5H3DfaAdL92gIR0Cvmgy9EkSmdX2UKGgGR0BrleHP/rB1aAdN4gFoCEdAr5qsihWYGHV9lChoBkfAZETOclPac2gHS/RoCEdAr5qvlr/KhnV9lChoBkdAUMcZflZHNGgHTegDaAhHQK+bCrp7kXF1fZQoaAZHwGVbNw71ZkloB03oA2gIR0Cvm0GxUvPDdX2UKGgGR0BsD90o0ALiaAdNDwJoCEdAr5xdU83dbnV9lChoBkfAOKrA+IMz/WgHTQIBaAhHQK+cst3fQ8h1fZQoaAZHwGPSaAFxGUhoB008AWgIR0CvnPMijcmCdX2UKGgGR8BieXMbFS88aAdL7mgIR0CvnvM1baAXdX2UKGgGR8BjbrMaCL/CaAdN6ANoCEdAr6MuocaOxXV9lChoBkdAWVi+sYEW7GgHTegDaAhHQK+j5gpjMFF1fZQoaAZHwGSV2f9P1thoB00aAWgIR0CvpM9/rjYJdX2UKGgGR8BAM0+LWI43aAdNKwFoCEdAr6TrBInSfHV9lChoBkfAYWobNr0rb2gHTegDaAhHQK+mTbGm1pl1fZQoaAZHwGPBGCZnctZoB01WAWgIR0Cvpxez+m3wdX2UKGgGR8AhpQ3xWkrPaAdNKgFoCEdAr6eVapxWDHV9lChoBkfAZtRilSCOFWgHTegDaAhHQK+p4r8zhxZ1fZQoaAZHwGFEyNOuaF5oB03oA2gIR0CvqoYEwFkhdX2UKGgGR8BlIVc2R7qqaAdL32gIR0CvrUatDD0ldX2UKGgGR0AmfQPZqVQiaAdL62gIR0CvraKZDzAfdX2UKGgGR0BqD0THsC1aaAdNxQFoCEdAr7A2+XZ5A3V9lChoBkfAJv7gKneiz2gHTWABaAhHQK+xjqO938p1fZQoaAZHwGMJIIF/x2BoB03oA2gIR0CvtA7O/tY0dX2UKGgGR0BoCvvDxb0OaAdNigJoCEdAr7QUONHYpXV9lChoBkfAJ7J6QeV9nmgHS+xoCEdAr7QyAH3UQXV9lChoBkdAaEJqxC6YmmgHTXoCaAhHQK+0sy7f51x1fZQoaAZHQG4fw8OkLx9oB01hAWgIR0CvtbDxLCemdX2UKGgGR8A2527nPmgbaAdNcgFoCEdAr7XU+5e7c3V9lChoBkfAZTw8yN4qw2gHS+VoCEdAr7qg6U7jk3V9lChoBkdAbbJc+JP69GgHTbYBaAhHQK+7NBnjABV1fZQoaAZHwEBsNb1RLsdoB01vAWgIR0CvvCzFVDKHdX2UKGgGR8BmBiKLsKLLaAdN6ANoCEdAr774A0bcXXV9lChoBkfANeJ/9YOlPGgHTTIBaAhHQK/Keyzolld1fZQoaAZHwGKJtIsiB5JoB03oA2gIR0Cvy2hqsU7CdX2UKGgGR8BqRZkAggX/aAdNYAFoCEdAr8uxbOeJ53V9lChoBkdARDI2Kl54W2gHTTgBaAhHQK/L19xZMcp1fZQoaAZHQG8JYwh4dIZoB01tAWgIR0CvzELbxmTUdX2UKGgGR0AkRVG0/nnuaAdNSgFoCEdAr8yXnGKhtnV9lChoBkfAYUN+1jRUm2gHTegDaAhHQK/NW51eSjh1fZQoaAZHQGHMD9OymhxoB03oA2gIR0Cv06sU7CBPdX2UKGgGR8Bir1yLhrFgaAdL3WgIR0Cv1L+t0V8DdX2UKGgGR0BpjN7pmmLtaAdNpAFoCEdAr9Y7Wbwz+HV9lChoBkfAIFlVDKHO8mgHTRcBaAhHQK/WcSUTtb91fZQoaAZHQGhYnRkVerxoB03KAWgIR0Cv1vBqCYkWdX2UKGgGR8A0NivxH5JsaAdNeQFoCEdAr9cmgBcRlHV9lChoBkfAZe3vaURnOGgHTegDaAhHQK/XahgVoHt1fZQoaAZHwEMegSOBDohoB02kAWgIR0Cv2nwo1DSgdX2UKGgGR8A2aHtF8XvZaAdNZwFoCEdAr9sa8xsVL3V9lChoBkdAbpjs54nndWgHTZ4BaAhHQK/bUdFOO811fZQoaAZHQHAnASJ0nw5oB02gAWgIR0Cv2/OQQtjDdX2UKGgGR8BjdU3fhuO0aAdN6ANoCEdAr96IuwosqnV9lChoBkdAMmGNedCmdmgHTQABaAhHQK/enDjR2KV1fZQoaAZHQGGiMkyDZlFoB03oA2gIR0Cv4KxwZOzqdX2UKGgGR8BnFPZ5AyEdaAdNKAFoCEdAr+Gi6z3RHHV9lChoBkfAZQpUI9kjHGgHTegDaAhHQK/j3A44p+d1fZQoaAZHwGXPmbCrLhdoB00xAWgIR0Cv5ZK5sj3VdX2UKGgGR8BkDjSVnmJWaAdNIwFoCEdAr+aXzOHFgnV9lChoBkfAa0FBJI1+AmgHTeEBaAhHQK/oM6Kcd5p1fZQoaAZHQGlJb2tdRixoB01LAmgIR0Cv6is1TBIndX2UKGgGR8BmeFuJk5IZaAdN6ANoCEdAr+qp20Re1XV9lChoBkfAbhxSBshxHWgHTTACaAhHQK/r9tvXK8t1fZQoaAZHQEDEZNwiqyZoB01DAWgIR0Cv7Ps5GSZCdWUu"}, "ep_success_buffer": {":type:": "<class 'collections.deque'>", ":serialized:": "gAWVIAAAAAAAAACMC2NvbGxlY3Rpb25zlIwFZGVxdWWUk5QpS2SGlFKULg=="}, "_n_updates": 368, "observation_space": {":type:": "<class 'gymnasium.spaces.box.Box'>", ":serialized:": "gAWVdgIAAAAAAACMFGd5bW5hc2l1bS5zcGFjZXMuYm94lIwDQm94lJOUKYGUfZQojAVkdHlwZZSMBW51bXB5lIwFZHR5cGWUk5SMAmY0lImIh5RSlChLA4wBPJROTk5K/////0r/////SwB0lGKMDWJvdW5kZWRfYmVsb3eUjBJudW1weS5jb3JlLm51bWVyaWOUjAtfZnJvbWJ1ZmZlcpSTlCiWCAAAAAAAAAABAQEBAQEBAZRoCIwCYjGUiYiHlFKUKEsDjAF8lE5OTkr/////Sv////9LAHSUYksIhZSMAUOUdJRSlIwNYm91bmRlZF9hYm92ZZRoESiWCAAAAAAAAAABAQEBAQEBAZRoFUsIhZRoGXSUUpSMBl9zaGFwZZRLCIWUjANsb3eUaBEoliAAAAAAAAAAAAC0wgAAtMIAAKDAAACgwNsPScAAAKDAAAAAgAAAAICUaAtLCIWUaBl0lFKUjARoaWdolGgRKJYgAAAAAAAAAAAAtEIAALRCAACgQAAAoEDbD0lAAACgQAAAgD8AAIA/lGgLSwiFlGgZdJRSlIwIbG93X3JlcHKUjFtbLTkwLiAgICAgICAgLTkwLiAgICAgICAgIC01LiAgICAgICAgIC01LiAgICAgICAgIC0zLjE0MTU5MjcgIC01LgogIC0wLiAgICAgICAgIC0wLiAgICAgICBdlIwJaGlnaF9yZXBylIxTWzkwLiAgICAgICAgOTAuICAgICAgICAgNS4gICAgICAgICA1LiAgICAgICAgIDMuMTQxNTkyNyAgNS4KICAxLiAgICAgICAgIDEuICAgICAgIF2UjApfbnBfcmFuZG9tlE51Yi4=", "dtype": "float32", "bounded_below": "[ True True True True True True True True]", "bounded_above": "[ True True True True True True True True]", "_shape": [8], "low": "[-90. -90. -5. -5. -3.1415927 -5.\n -0. -0. ]", "high": "[90. 90. 5. 5. 3.1415927 5.\n 1. 1. ]", "low_repr": "[-90. -90. -5. -5. -3.1415927 -5.\n -0. -0. ]", "high_repr": "[90. 90. 5. 5. 3.1415927 5.\n 1. 1. ]", "_np_random": null}, "action_space": {":type:": "<class 'gymnasium.spaces.discrete.Discrete'>", ":serialized:": "gAWV2wAAAAAAAACMGWd5bW5hc2l1bS5zcGFjZXMuZGlzY3JldGWUjAhEaXNjcmV0ZZSTlCmBlH2UKIwBbpSMFW51bXB5LmNvcmUubXVsdGlhcnJheZSMBnNjYWxhcpSTlIwFbnVtcHmUjAVkdHlwZZSTlIwCaTiUiYiHlFKUKEsDjAE8lE5OTkr/////Sv////9LAHSUYkMIBAAAAAAAAACUhpRSlIwFc3RhcnSUaAhoDkMIAAAAAAAAAACUhpRSlIwGX3NoYXBllCmMBWR0eXBllGgOjApfbnBfcmFuZG9tlE51Yi4=", "n": "4", "start": "0", "_shape": [], "dtype": "int64", "_np_random": null}, "n_envs": 16, "n_steps": 1024, "gamma": 0.8, "gae_lambda": 0.98, "ent_coef": 0.01, "vf_coef": 0.5, "max_grad_norm": 0.5, "batch_size": 64, "n_epochs": 4, "clip_range": {":type:": "<class 'function'>", ":serialized:": "gAWVrQIAAAAAAACMF2Nsb3VkcGlja2xlLmNsb3VkcGlja2xllIwOX21ha2VfZnVuY3Rpb26Uk5QoaACMDV9idWlsdGluX3R5cGWUk5SMCENvZGVUeXBllIWUUpQoSwFLAEsASwFLAUsTQwSIAFMAlE6FlCmMAV+UhZSMSS91c3IvbG9jYWwvbGliL3B5dGhvbjMuMTAvZGlzdC1wYWNrYWdlcy9zdGFibGVfYmFzZWxpbmVzMy9jb21tb24vdXRpbHMucHmUjARmdW5jlEuEQwIEAZSMA3ZhbJSFlCl0lFKUfZQojAtfX3BhY2thZ2VfX5SMGHN0YWJsZV9iYXNlbGluZXMzLmNvbW1vbpSMCF9fbmFtZV9flIwec3RhYmxlX2Jhc2VsaW5lczMuY29tbW9uLnV0aWxzlIwIX19maWxlX1+UjEkvdXNyL2xvY2FsL2xpYi9weXRob24zLjEwL2Rpc3QtcGFja2FnZXMvc3RhYmxlX2Jhc2VsaW5lczMvY29tbW9uL3V0aWxzLnB5lHVOTmgAjBBfbWFrZV9lbXB0eV9jZWxslJOUKVKUhZR0lFKUaACMEl9mdW5jdGlvbl9zZXRzdGF0ZZSTlGgffZR9lChoFowEZnVuY5SMDF9fcXVhbG5hbWVfX5SMGWNvbnN0YW50X2ZuLjxsb2NhbHM+LmZ1bmOUjA9fX2Fubm90YXRpb25zX1+UfZSMDl9fa3dkZWZhdWx0c19flE6MDF9fZGVmYXVsdHNfX5ROjApfX21vZHVsZV9flGgXjAdfX2RvY19flE6MC19fY2xvc3VyZV9flGgAjApfbWFrZV9jZWxslJOURz/JmZmZmZmahZRSlIWUjBdfY2xvdWRwaWNrbGVfc3VibW9kdWxlc5RdlIwLX19nbG9iYWxzX1+UfZR1hpSGUjAu"}, "clip_range_vf": null, "normalize_advantage": true, "target_kl": null, "lr_schedule": {":type:": "<class 'function'>", ":serialized:": "gAWVrQIAAAAAAACMF2Nsb3VkcGlja2xlLmNsb3VkcGlja2xllIwOX21ha2VfZnVuY3Rpb26Uk5QoaACMDV9idWlsdGluX3R5cGWUk5SMCENvZGVUeXBllIWUUpQoSwFLAEsASwFLAUsTQwSIAFMAlE6FlCmMAV+UhZSMSS91c3IvbG9jYWwvbGliL3B5dGhvbjMuMTAvZGlzdC1wYWNrYWdlcy9zdGFibGVfYmFzZWxpbmVzMy9jb21tb24vdXRpbHMucHmUjARmdW5jlEuEQwIEAZSMA3ZhbJSFlCl0lFKUfZQojAtfX3BhY2thZ2VfX5SMGHN0YWJsZV9iYXNlbGluZXMzLmNvbW1vbpSMCF9fbmFtZV9flIwec3RhYmxlX2Jhc2VsaW5lczMuY29tbW9uLnV0aWxzlIwIX19maWxlX1+UjEkvdXNyL2xvY2FsL2xpYi9weXRob24zLjEwL2Rpc3QtcGFja2FnZXMvc3RhYmxlX2Jhc2VsaW5lczMvY29tbW9uL3V0aWxzLnB5lHVOTmgAjBBfbWFrZV9lbXB0eV9jZWxslJOUKVKUhZR0lFKUaACMEl9mdW5jdGlvbl9zZXRzdGF0ZZSTlGgffZR9lChoFowEZnVuY5SMDF9fcXVhbG5hbWVfX5SMGWNvbnN0YW50X2ZuLjxsb2NhbHM+LmZ1bmOUjA9fX2Fubm90YXRpb25zX1+UfZSMDl9fa3dkZWZhdWx0c19flE6MDF9fZGVmYXVsdHNfX5ROjApfX21vZHVsZV9flGgXjAdfX2RvY19flE6MC19fY2xvc3VyZV9flGgAjApfbWFrZV9jZWxslJOURz8zqSowVTJhhZRSlIWUjBdfY2xvdWRwaWNrbGVfc3VibW9kdWxlc5RdlIwLX19nbG9iYWxzX1+UfZR1hpSGUjAu"}, "system_info": {"OS": "Linux-6.1.85+-x86_64-with-glibc2.35 # 1 SMP PREEMPT_DYNAMIC Thu Jun 27 21:05:47 UTC 2024", "Python": "3.10.12", "Stable-Baselines3": "2.0.0a5", "PyTorch": "2.5.0+cu121", "GPU Enabled": "True", "Numpy": "1.26.4", "Cloudpickle": "3.1.0", "Gymnasium": "0.28.1", "OpenAI Gym": "0.25.2"}}
|
|
|
|
| 1 |
+
{"policy_class": {":type:": "<class 'abc.ABCMeta'>", ":serialized:": "gAWVMAAAAAAAAACMHnN0YWJsZV9iYXNlbGluZXMzLmRxbi5wb2xpY2llc5SMCURRTlBvbGljeZSTlC4=", "__module__": "stable_baselines3.dqn.policies", "__annotations__": "{'q_net': <class 'stable_baselines3.dqn.policies.QNetwork'>, 'q_net_target': <class 'stable_baselines3.dqn.policies.QNetwork'>}", "__doc__": "\n Policy class with Q-Value Net and target net for DQN\n\n :param observation_space: Observation space\n :param action_space: Action space\n :param lr_schedule: Learning rate schedule (could be constant)\n :param net_arch: The specification of the policy and value networks.\n :param activation_fn: Activation function\n :param features_extractor_class: Features extractor to use.\n :param features_extractor_kwargs: Keyword arguments\n to pass to the features extractor.\n :param normalize_images: Whether to normalize images or not,\n dividing by 255.0 (True by default)\n :param optimizer_class: The optimizer to use,\n ``th.optim.Adam`` by default\n :param optimizer_kwargs: Additional keyword arguments,\n excluding the learning rate, to pass to the optimizer\n ", "__init__": "<function DQNPolicy.__init__ at 0x79f32a84ee60>", "_build": "<function DQNPolicy._build at 0x79f32a84eef0>", "make_q_net": "<function DQNPolicy.make_q_net at 0x79f32a84ef80>", "forward": "<function DQNPolicy.forward at 0x79f32a84f010>", "_predict": "<function DQNPolicy._predict at 0x79f32a84f0a0>", "_get_constructor_parameters": "<function DQNPolicy._get_constructor_parameters at 0x79f32a84f130>", "set_training_mode": "<function DQNPolicy.set_training_mode at 0x79f32a84f1c0>", "__abstractmethods__": "frozenset()", "_abc_impl": "<_abc._abc_data object at 0x79f32a8569c0>"}, "verbose": 1, "policy_kwargs": {}, "num_timesteps": 750000, "_total_timesteps": 750000, "_num_timesteps_at_start": 0, "seed": 42, "action_noise": null, "start_time": 1730705438188553600, "learning_rate": 0.0001, "tensorboard_log": null, "_last_obs": {":type:": "<class 'numpy.ndarray'>", ":serialized:": "gAWVlQAAAAAAAACMEm51bXB5LmNvcmUubnVtZXJpY5SMC19mcm9tYnVmZmVylJOUKJYgAAAAAAAAALrGyD6r7zg/LrIIPh2l5b23KnQ8guCsOwAAAAAAAAAAlIwFbnVtcHmUjAVkdHlwZZSTlIwCZjSUiYiHlFKUKEsDjAE8lE5OTkr/////Sv////9LAHSUYksBSwiGlIwBQ5R0lFKULg=="}, "_last_episode_starts": {":type:": "<class 'numpy.ndarray'>", ":serialized:": "gAWVdAAAAAAAAACMEm51bXB5LmNvcmUubnVtZXJpY5SMC19mcm9tYnVmZmVylJOUKJYBAAAAAAAAAAGUjAVudW1weZSMBWR0eXBllJOUjAJiMZSJiIeUUpQoSwOMAXyUTk5OSv////9K/////0sAdJRiSwGFlIwBQ5R0lFKULg=="}, "_last_original_obs": {":type:": "<class 'numpy.ndarray'>", ":serialized:": "gAWVlQAAAAAAAACMEm51bXB5LmNvcmUubnVtZXJpY5SMC19mcm9tYnVmZmVylJOUKJYgAAAAAAAAAGYXyD4BlTk/WtgIPqcy971W2G888rqYOwAAAAAAAAAAlIwFbnVtcHmUjAVkdHlwZZSTlIwCZjSUiYiHlFKUKEsDjAE8lE5OTkr/////Sv////9LAHSUYksBSwiGlIwBQ5R0lFKULg=="}, "_episode_num": 1355, "use_sde": false, "sde_sample_freq": -1, "_current_progress_remaining": 0.0, "_stats_window_size": 100, "ep_info_buffer": {":type:": "<class 'collections.deque'>", ":serialized:": "gAWVNgwAAAAAAACMC2NvbGxlY3Rpb25zlIwFZGVxdWWUk5QpS2SGlFKUKH2UKIwBcpRHQGPzF+uvECOMAWyUTWUCjAF0lEdAmBSAJHAh0XV9lChoBkdAG49d/rjYI2gHTegDaAhHQJggmqBEroZ1fZQoaAZHQEevSncclw9oB03oA2gIR0CYKqcz67/XdX2UKGgGR0Bp7KUkfLcLaAdN2wFoCEdAmC5H6ZYxL3V9lChoBkdAY1z6KtPpIWgHTQgDaAhHQJg2gJF9a2Z1fZQoaAZHwFZUBLwnYxtoB03oA2gIR0CYQMcyWRigdX2UKGgGR8BBl9d3Sro4aAdN6ANoCEdAmErcbJfYz3V9lChoBkdAcABsYl6Z6WgHTb0BaAhHQJhOLrv9cbB1fZQoaAZHwEj0hZha1TloB03oA2gIR0CYVxt6X0GvdX2UKGgGR0AxnETg2qDLaAdN6ANoCEdAmF8xttQ9BHV9lChoBkdAZKaWpIczZmgHTd4DaAhHQJhn5dcB2fV1fZQoaAZHQF7yAJswco9oB01+AmgIR0CYbOe4TbnHdX2UKGgGR0Bicytga3qiaAdNewNoCEdAmHcXxaxHG3V9lChoBkdAYWj6SDAaemgHTZ4DaAhHQJh+JyFPBSF1fZQoaAZHwEwCE8JUo8ZoB00CAmgIR0CYgksaKk2xdX2UKGgGR0Bj3/sTnJT3aAdNWAJoCEdAmIbbl/6O53V9lChoBkfAKBVxbSqlxmgHTegBaAhHQJiKYrvsqrl1fZQoaAZHwEKURigCfYloB01CAmgIR0CYjsc2itaIdX2UKGgGR8BVvNWEK3NLaAdNEAFoCEdAmJCdZq20A3V9lChoBkfAX4VlZowmFGgHTV4DaAhHQJiYSHj6vaF1fZQoaAZHQGcUqDTSb6RoB00oA2gIR0CYnwg5R0lrdX2UKGgGR8BCf8mShakiaAdN+AFoCEdAmKMQnUlRg3V9lChoBkdAaVPnvlU6xWgHTWcCaAhHQJiolfBvaUR1fZQoaAZHwFPtG0eEIxBoB0uqaAhHQJippG3F1jl1fZQoaAZHQGMxStmtheBoB01WAmgIR0CYrqobGWD6dX2UKGgGR0BZqfR/mT1TaAdN2AJoCEdAmLVHGOuJUHV9lChoBkdAZ0ScI7eVLWgHTR4CaAhHQJi5eKtPpIN1fZQoaAZHQFoKiZv1lGxoB03oA2gIR0CYwJuw5eZ5dX2UKGgGR0BkDuXPZ7HAaAdNXQFoCEdAmMMVC5VfeHV9lChoBkdAWqhxffGdZ2gHTUkDaAhHQJjK8yAQQMB1fZQoaAZHQGX6ojGDL8toB031AWgIR0CYzroAXEZSdX2UKGgGR0BpV4XoC+10aAdNiAJoCEdAmNQjVx0dR3V9lChoBkfAVpe01IiC8WgHTTEBaAhHQJjWPTnaFmF1fZQoaAZHQFecNaQmu1ZoB01bA2gIR0CY3zRP420idX2UKGgGR0Bmwym8/UvxaAdNUQJoCEdAmOQ7mMfignV9lChoBkdAZqBy/bj942gHTWgCaAhHQJjotgOSW7h1fZQoaAZHQFUwt5D7ZWdoB00XA2gIR0CY8IOfNA1OdX2UKGgGR0BppSjDbah6aAdNwAFoCEdAmPO5tSAH3XV9lChoBkdAZVJnIyTINmgHTeUCaAhHQJj5hKsdT5x1fZQoaAZHwFyGMIu5BkZoB0v7aAhHQJj7H5mAbyZ1fZQoaAZHQGXC35N47ihoB00GAmgIR0CY/ur8zhxYdX2UKGgGR0Bt1jakAPupaAdN4gFoCEdAmQJDRD1GsnV9lChoBkdAZRDUI9kjHGgHTW4CaAhHQJkHfVRUFSt1fZQoaAZHQGQZ5tvXK8toB00XA2gIR0CZDnYeDFqBdX2UKGgGR0Bo5uK4x1xLaAdNyAFoCEdAmRGhgiNbT3V9lChoBkfAUyQSkCV8kWgHS8doCEdAmRLkXgtOEnV9lChoBkfAUKQNy5qdpmgHS7doCEdAmRP9gfEGaHV9lChoBkfAZZLVjqfOEGgHTa0CaAhHQJkZaEoOQQt1fZQoaAZHwFenr1dxAB1oB0v0aAhHQJka9BUrCnB1fZQoaAZHQCVVpItlI3BoB03oA2gIR0CZJot4A0bcdX2UKGgGR8BTnOumrKeTaAdL5WgIR0CZJ/UPhAGCdX2UKGgGR0Bv8HbKzRhMaAdNswJoCEdAmS2+oHcDbXV9lChoBkdAWZkkVvddmmgHTUEDaAhHQJk2WoUBXCF1fZQoaAZHwFLdornTy8VoB00fAWgIR0CZOC+kP+XJdX2UKGgGR8BpkzIzWPLgaAdNhANoCEdAmT9ljZtelnV9lChoBkdAYScCrcTJyWgHTUkDaAhHQJlIgA+6iCd1fZQoaAZHwFG9F6Rhc7hoB01GAWgIR0CZSwTINmUXdX2UKGgGR8BZexCdBjWkaAdNPAFoCEdAmU00nkT6BXV9lChoBkfAWI+26TW5H2gHTS8CaAhHQJlRQwnH/951fZQoaAZHQGvtHrhR64VoB00OAmgIR0CZVSnZ00WNdX2UKGgGR0Bk5NaMaS9vaAdN6wJoCEdAmVudAkcCHXV9lChoBkdAYNn/LDAJs2gHTZMDaAhHQJlitNpM6BB1fZQoaAZHwFRhlyBCladoB0uaaAhHQJljoX/HYHx1fZQoaAZHwDQEVRDTjNpoB01oAWgIR0CZZgdf9gnddX2UKGgGR0BkN/BP9DQaaAdNXQJoCEdAmWwDMmnfmHV9lChoBkdAZzmXHim2s2gHTYECaAhHQJlx21G9YfZ1fZQoaAZHwFpS19v0h/1oB03EAWgIR0CZdWMglnh9dX2UKGgGR8BS0LdJrcj8aAdNAQFoCEdAmXcGCAc1fnV9lChoBkfAVAqrzXjEN2gHS4hoCEdAmXfSQxN7B3V9lChoBkdAaMrYSQHRkWgHTQADaAhHQJl9RcC5mRN1fZQoaAZHQGtEhOxjawloB01FAmgIR0CZgazvJA+qdX2UKGgGR0BptJw2l2vCaAdNDAJoCEdAmYVtfsu3+nV9lChoBkfAVy6xD9fkWGgHS+9oCEdAmYbqCxu89XV9lChoBkdAY9edgfEGaGgHTU4DaAhHQJmN6suFpPB1fZQoaAZHwFlIyYoiLVFoB0unaAhHQJmO6Ef1Yhd1fZQoaAZHwFdD0l7dBSloB0uOaAhHQJmPxCiRGMJ1fZQoaAZHwFeatapxWDJoB0uTaAhHQJmQpU0elsR1fZQoaAZHQGdZwLE1l5JoB03nAmgIR0CZl2mWMS9NdX2UKGgGR0BnsA5myxA0aAdN5AFoCEdAmZuFzuF6A3V9lChoBkfAVzgSsbNr02gHS/toCEdAmZ0jO1OTJXV9lChoBkdAb9xFOO8012gHTc8BaAhHQJmgnPhQ3xZ1fZQoaAZHQFm2Sa3I+4doB03WAmgIR0CZpwTG5tm+dX2UKGgGR0Bqh2RJVbRnaAdNMwJoCEdAmasH5BTn73V9lChoBkdAM3bP2PDHfmgHTWoCaAhHQJmvSC04R291fZQoaAZHQGxdR8lXzUZoB03iAWgIR0CZsrkpI+W4dX2UKGgGR0BnjhjjJdSmaAdNAAJoCEdAmbcj101ZT3V9lChoBkfAXvnwCr92o2gHTfkBaAhHQJm7CtaIN3J1fZQoaAZHQGLyxHf/FR5oB02bA2gIR0CZxADXOGCadX2UKGgGR0BmvOV9nbqRaAdNTwJoCEdAmchchTwUg3V9lChoBkdAYc80fozN2WgHTf8CaAhHQJnPWEBbOeJ1fZQoaAZHwFcZ/wAlv61oB0vHaAhHQJnQkaXKKYR1fZQoaAZHwGqgO4oZydZoB02vA2gIR0CZ2QzSThYOdX2UKGgGR0BoPkZm7J4jaAdN/wJoCEdAmeCdmQKa5XV9lChoBkdAPKdKh+OOsGgHTXgBaAhHQJnjU7LdN351fZQoaAZHQGMstSydFv1oB03TAmgIR0CZ6csabWmQdX2UKGgGR8BOuPHT7VJ+aAdNKgFoCEdAmeu+gctGu3V9lChoBkdAaIVFPznRs2gHTY0CaAhHQJnyWj7ALzB1fZQoaAZHQGH+oLPUrkNoB02sAmgIR0CZ9/FuvUz9dX2UKGgGR8BQ2HiWE9McaAdN6ANoCEdAmgEvVqesgnV9lChoBkdAYQclRgqmTGgHTRwCaAhHQJoFHm8ujAV1ZS4="}, "ep_success_buffer": {":type:": "<class 'collections.deque'>", ":serialized:": "gAWVIAAAAAAAAACMC2NvbGxlY3Rpb25zlIwFZGVxdWWUk5QpS2SGlFKULg=="}, "_n_updates": 187250, "observation_space": {":type:": "<class 'gymnasium.spaces.box.Box'>", ":serialized:": "gAWVdgIAAAAAAACMFGd5bW5hc2l1bS5zcGFjZXMuYm94lIwDQm94lJOUKYGUfZQojAVkdHlwZZSMBW51bXB5lIwFZHR5cGWUk5SMAmY0lImIh5RSlChLA4wBPJROTk5K/////0r/////SwB0lGKMDWJvdW5kZWRfYmVsb3eUjBJudW1weS5jb3JlLm51bWVyaWOUjAtfZnJvbWJ1ZmZlcpSTlCiWCAAAAAAAAAABAQEBAQEBAZRoCIwCYjGUiYiHlFKUKEsDjAF8lE5OTkr/////Sv////9LAHSUYksIhZSMAUOUdJRSlIwNYm91bmRlZF9hYm92ZZRoESiWCAAAAAAAAAABAQEBAQEBAZRoFUsIhZRoGXSUUpSMBl9zaGFwZZRLCIWUjANsb3eUaBEoliAAAAAAAAAAAAC0wgAAtMIAAKDAAACgwNsPScAAAKDAAAAAgAAAAICUaAtLCIWUaBl0lFKUjARoaWdolGgRKJYgAAAAAAAAAAAAtEIAALRCAACgQAAAoEDbD0lAAACgQAAAgD8AAIA/lGgLSwiFlGgZdJRSlIwIbG93X3JlcHKUjFtbLTkwLiAgICAgICAgLTkwLiAgICAgICAgIC01LiAgICAgICAgIC01LiAgICAgICAgIC0zLjE0MTU5MjcgIC01LgogIC0wLiAgICAgICAgIC0wLiAgICAgICBdlIwJaGlnaF9yZXBylIxTWzkwLiAgICAgICAgOTAuICAgICAgICAgNS4gICAgICAgICA1LiAgICAgICAgIDMuMTQxNTkyNyAgNS4KICAxLiAgICAgICAgIDEuICAgICAgIF2UjApfbnBfcmFuZG9tlE51Yi4=", "dtype": "float32", "bounded_below": "[ True True True True True True True True]", "bounded_above": "[ True True True True True True True True]", "_shape": [8], "low": "[-90. -90. -5. -5. -3.1415927 -5.\n -0. -0. ]", "high": "[90. 90. 5. 5. 3.1415927 5.\n 1. 1. ]", "low_repr": "[-90. -90. -5. -5. -3.1415927 -5.\n -0. -0. ]", "high_repr": "[90. 90. 5. 5. 3.1415927 5.\n 1. 1. ]", "_np_random": null}, "action_space": {":type:": "<class 'gymnasium.spaces.discrete.Discrete'>", ":serialized:": "gAWVowEAAAAAAACMGWd5bW5hc2l1bS5zcGFjZXMuZGlzY3JldGWUjAhEaXNjcmV0ZZSTlCmBlH2UKIwBbpSMFW51bXB5LmNvcmUubXVsdGlhcnJheZSMBnNjYWxhcpSTlIwFbnVtcHmUjAVkdHlwZZSTlIwCaTiUiYiHlFKUKEsDjAE8lE5OTkr/////Sv////9LAHSUYkMIBAAAAAAAAACUhpRSlIwFc3RhcnSUaAhoDkMIAAAAAAAAAACUhpRSlIwGX3NoYXBllCmMBWR0eXBllGgOjApfbnBfcmFuZG9tlIwUbnVtcHkucmFuZG9tLl9waWNrbGWUjBBfX2dlbmVyYXRvcl9jdG9ylJOUjAVQQ0c2NJRoG4wUX19iaXRfZ2VuZXJhdG9yX2N0b3KUk5SGlFKUfZQojA1iaXRfZ2VuZXJhdG9ylIwFUENHNjSUjAVzdGF0ZZR9lChoJooRXAzIT2ucUDgplpt+Ybl+xgCMA2luY5SKEf+rJY3i8spmbkGoyTZUUPoAdYwKaGFzX3VpbnQzMpRLAYwIdWludGVnZXKUSqkYhRJ1YnViLg==", "n": "4", "start": "0", "_shape": [], "dtype": "int64", "_np_random": "Generator(PCG64)"}, "n_envs": 1, "buffer_size": 100000, "batch_size": 64, "learning_starts": 1000, "tau": 1.0, "gamma": 0.99, "gradient_steps": 1, "optimize_memory_usage": false, "replay_buffer_class": {":type:": "<class 'abc.ABCMeta'>", ":serialized:": "gAWVNQAAAAAAAACMIHN0YWJsZV9iYXNlbGluZXMzLmNvbW1vbi5idWZmZXJzlIwMUmVwbGF5QnVmZmVylJOULg==", "__module__": "stable_baselines3.common.buffers", "__doc__": "\n Replay buffer used in off-policy algorithms like SAC/TD3.\n\n :param buffer_size: Max number of element in the buffer\n :param observation_space: Observation space\n :param action_space: Action space\n :param device: PyTorch device\n :param n_envs: Number of parallel environments\n :param optimize_memory_usage: Enable a memory efficient variant\n of the replay buffer which reduces by almost a factor two the memory used,\n at a cost of more complexity.\n See https://github.com/DLR-RM/stable-baselines3/issues/37#issuecomment-637501195\n and https://github.com/DLR-RM/stable-baselines3/pull/28#issuecomment-637559274\n Cannot be used in combination with handle_timeout_termination.\n :param handle_timeout_termination: Handle timeout termination (due to timelimit)\n separately and treat the task as infinite horizon task.\n https://github.com/DLR-RM/stable-baselines3/issues/284\n ", "__init__": "<function ReplayBuffer.__init__ at 0x79f32a8232e0>", "add": "<function ReplayBuffer.add at 0x79f32a823370>", "sample": "<function ReplayBuffer.sample at 0x79f32a823400>", "_get_samples": "<function ReplayBuffer._get_samples at 0x79f32a823490>", "__abstractmethods__": "frozenset()", "_abc_impl": "<_abc._abc_data object at 0x79f2aea16080>"}, "replay_buffer_kwargs": {}, "train_freq": {":type:": "<class 'stable_baselines3.common.type_aliases.TrainFreq'>", ":serialized:": "gAWVYQAAAAAAAACMJXN0YWJsZV9iYXNlbGluZXMzLmNvbW1vbi50eXBlX2FsaWFzZXOUjAlUcmFpbkZyZXGUk5RLBGgAjBJUcmFpbkZyZXF1ZW5jeVVuaXSUk5SMBHN0ZXCUhZRSlIaUgZQu"}, "use_sde_at_warmup": false, "exploration_initial_eps": 1.0, "exploration_final_eps": 0.01, "exploration_fraction": 0.1, "target_update_interval": 1000, "_n_calls": 750000, "max_grad_norm": 10, "exploration_rate": 0.01, "lr_schedule": {":type:": "<class 'function'>", ":serialized:": "gAWVrQIAAAAAAACMF2Nsb3VkcGlja2xlLmNsb3VkcGlja2xllIwOX21ha2VfZnVuY3Rpb26Uk5QoaACMDV9idWlsdGluX3R5cGWUk5SMCENvZGVUeXBllIWUUpQoSwFLAEsASwFLAUsTQwSIAFMAlE6FlCmMAV+UhZSMSS91c3IvbG9jYWwvbGliL3B5dGhvbjMuMTAvZGlzdC1wYWNrYWdlcy9zdGFibGVfYmFzZWxpbmVzMy9jb21tb24vdXRpbHMucHmUjARmdW5jlEuEQwIEAZSMA3ZhbJSFlCl0lFKUfZQojAtfX3BhY2thZ2VfX5SMGHN0YWJsZV9iYXNlbGluZXMzLmNvbW1vbpSMCF9fbmFtZV9flIwec3RhYmxlX2Jhc2VsaW5lczMuY29tbW9uLnV0aWxzlIwIX19maWxlX1+UjEkvdXNyL2xvY2FsL2xpYi9weXRob24zLjEwL2Rpc3QtcGFja2FnZXMvc3RhYmxlX2Jhc2VsaW5lczMvY29tbW9uL3V0aWxzLnB5lHVOTmgAjBBfbWFrZV9lbXB0eV9jZWxslJOUKVKUhZR0lFKUaACMEl9mdW5jdGlvbl9zZXRzdGF0ZZSTlGgffZR9lChoFowEZnVuY5SMDF9fcXVhbG5hbWVfX5SMGWNvbnN0YW50X2ZuLjxsb2NhbHM+LmZ1bmOUjA9fX2Fubm90YXRpb25zX1+UfZSMDl9fa3dkZWZhdWx0c19flE6MDF9fZGVmYXVsdHNfX5ROjApfX21vZHVsZV9flGgXjAdfX2RvY19flE6MC19fY2xvc3VyZV9flGgAjApfbWFrZV9jZWxslJOURz8aNuLrHEMthZRSlIWUjBdfY2xvdWRwaWNrbGVfc3VibW9kdWxlc5RdlIwLX19nbG9iYWxzX1+UfZR1hpSGUjAu"}, "batch_norm_stats": [], "batch_norm_stats_target": [], "exploration_schedule": {":type:": "<class 'function'>", ":serialized:": "gAWVTQMAAAAAAACMF2Nsb3VkcGlja2xlLmNsb3VkcGlja2xllIwOX21ha2VfZnVuY3Rpb26Uk5QoaACMDV9idWlsdGluX3R5cGWUk5SMCENvZGVUeXBllIWUUpQoSwFLAEsASwFLBEsTQyhkAXwAGACIAWsEcgiIAFMAiAJkAXwAGACIAIgCGAAUAIgBGwAXAFMAlE5LAYaUKYwScHJvZ3Jlc3NfcmVtYWluaW5nlIWUjEkvdXNyL2xvY2FsL2xpYi9weXRob24zLjEwL2Rpc3QtcGFja2FnZXMvc3RhYmxlX2Jhc2VsaW5lczMvY29tbW9uL3V0aWxzLnB5lIwEZnVuY5RLckMGDAEEARgClIwDZW5klIwMZW5kX2ZyYWN0aW9ulIwFc3RhcnSUh5QpdJRSlH2UKIwLX19wYWNrYWdlX1+UjBhzdGFibGVfYmFzZWxpbmVzMy5jb21tb26UjAhfX25hbWVfX5SMHnN0YWJsZV9iYXNlbGluZXMzLmNvbW1vbi51dGlsc5SMCF9fZmlsZV9flIxJL3Vzci9sb2NhbC9saWIvcHl0aG9uMy4xMC9kaXN0LXBhY2thZ2VzL3N0YWJsZV9iYXNlbGluZXMzL2NvbW1vbi91dGlscy5weZR1Tk5oAIwQX21ha2VfZW1wdHlfY2VsbJSTlClSlGgdKVKUaB0pUpSHlHSUUpRoAIwSX2Z1bmN0aW9uX3NldHN0YXRllJOUaCN9lH2UKGgYjARmdW5jlIwMX19xdWFsbmFtZV9flIwbZ2V0X2xpbmVhcl9mbi48bG9jYWxzPi5mdW5jlIwPX19hbm5vdGF0aW9uc19flH2UKGgKjAhidWlsdGluc5SMBWZsb2F0lJOUjAZyZXR1cm6UaC91jA5fX2t3ZGVmYXVsdHNfX5ROjAxfX2RlZmF1bHRzX1+UTowKX19tb2R1bGVfX5RoGYwHX19kb2NfX5ROjAtfX2Nsb3N1cmVfX5RoAIwKX21ha2VfY2VsbJSTlEc/hHrhR64Ue4WUUpRoN0c/uZmZmZmZmoWUUpRoN0c/8AAAAAAAAIWUUpSHlIwXX2Nsb3VkcGlja2xlX3N1Ym1vZHVsZXOUXZSMC19fZ2xvYmFsc19flH2UdYaUhlIwLg=="}, "system_info": {"OS": "Linux-6.1.85+-x86_64-with-glibc2.35 # 1 SMP PREEMPT_DYNAMIC Thu Jun 27 21:05:47 UTC 2024", "Python": "3.10.12", "Stable-Baselines3": "2.0.0a5", "PyTorch": "2.5.0+cu121", "GPU Enabled": "True", "Numpy": "1.26.4", "Cloudpickle": "3.1.0", "Gymnasium": "0.28.1", "OpenAI Gym": "0.25.2"}}
|
ppo-LunarLander-v2.zip
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
-
size
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:57df4ea2ab4d3cea60c6afc9acf08c417095a2e699087f3124c6b82b72bacde1
|
| 3 |
+
size 106640
|
ppo-LunarLander-v2/data
CHANGED
|
@@ -1,57 +1,56 @@
|
|
| 1 |
{
|
| 2 |
"policy_class": {
|
| 3 |
":type:": "<class 'abc.ABCMeta'>",
|
| 4 |
-
":serialized:": "
|
| 5 |
-
"__module__": "stable_baselines3.
|
| 6 |
-
"
|
| 7 |
-
"
|
| 8 |
-
"
|
| 9 |
-
"
|
| 10 |
-
"
|
| 11 |
-
"
|
| 12 |
-
"
|
| 13 |
-
"
|
| 14 |
-
"
|
| 15 |
-
"_predict": "<function ActorCriticPolicy._predict at 0x79f32a820c10>",
|
| 16 |
-
"evaluate_actions": "<function ActorCriticPolicy.evaluate_actions at 0x79f32a820ca0>",
|
| 17 |
-
"get_distribution": "<function ActorCriticPolicy.get_distribution at 0x79f32a820d30>",
|
| 18 |
-
"predict_values": "<function ActorCriticPolicy.predict_values at 0x79f32a820dc0>",
|
| 19 |
"__abstractmethods__": "frozenset()",
|
| 20 |
-
"_abc_impl": "<_abc._abc_data object at
|
| 21 |
},
|
| 22 |
"verbose": 1,
|
| 23 |
"policy_kwargs": {},
|
| 24 |
-
"num_timesteps":
|
| 25 |
-
"_total_timesteps":
|
| 26 |
"_num_timesteps_at_start": 0,
|
| 27 |
-
"seed":
|
| 28 |
"action_noise": null,
|
| 29 |
-
"start_time":
|
| 30 |
-
"learning_rate": 0.
|
| 31 |
"tensorboard_log": null,
|
| 32 |
"_last_obs": {
|
| 33 |
":type:": "<class 'numpy.ndarray'>",
|
| 34 |
-
":serialized:": "
|
| 35 |
},
|
| 36 |
"_last_episode_starts": {
|
| 37 |
":type:": "<class 'numpy.ndarray'>",
|
| 38 |
-
":serialized:": "
|
| 39 |
},
|
| 40 |
-
"_last_original_obs":
|
| 41 |
-
|
|
|
|
|
|
|
|
|
|
| 42 |
"use_sde": false,
|
| 43 |
"sde_sample_freq": -1,
|
| 44 |
-
"_current_progress_remaining":
|
| 45 |
"_stats_window_size": 100,
|
| 46 |
"ep_info_buffer": {
|
| 47 |
":type:": "<class 'collections.deque'>",
|
| 48 |
-
":serialized:": "
|
| 49 |
},
|
| 50 |
"ep_success_buffer": {
|
| 51 |
":type:": "<class 'collections.deque'>",
|
| 52 |
":serialized:": "gAWVIAAAAAAAAACMC2NvbGxlY3Rpb25zlIwFZGVxdWWUk5QpS2SGlFKULg=="
|
| 53 |
},
|
| 54 |
-
"_n_updates":
|
| 55 |
"observation_space": {
|
| 56 |
":type:": "<class 'gymnasium.spaces.box.Box'>",
|
| 57 |
":serialized:": "gAWVdgIAAAAAAACMFGd5bW5hc2l1bS5zcGFjZXMuYm94lIwDQm94lJOUKYGUfZQojAVkdHlwZZSMBW51bXB5lIwFZHR5cGWUk5SMAmY0lImIh5RSlChLA4wBPJROTk5K/////0r/////SwB0lGKMDWJvdW5kZWRfYmVsb3eUjBJudW1weS5jb3JlLm51bWVyaWOUjAtfZnJvbWJ1ZmZlcpSTlCiWCAAAAAAAAAABAQEBAQEBAZRoCIwCYjGUiYiHlFKUKEsDjAF8lE5OTkr/////Sv////9LAHSUYksIhZSMAUOUdJRSlIwNYm91bmRlZF9hYm92ZZRoESiWCAAAAAAAAAABAQEBAQEBAZRoFUsIhZRoGXSUUpSMBl9zaGFwZZRLCIWUjANsb3eUaBEoliAAAAAAAAAAAAC0wgAAtMIAAKDAAACgwNsPScAAAKDAAAAAgAAAAICUaAtLCIWUaBl0lFKUjARoaWdolGgRKJYgAAAAAAAAAAAAtEIAALRCAACgQAAAoEDbD0lAAACgQAAAgD8AAIA/lGgLSwiFlGgZdJRSlIwIbG93X3JlcHKUjFtbLTkwLiAgICAgICAgLTkwLiAgICAgICAgIC01LiAgICAgICAgIC01LiAgICAgICAgIC0zLjE0MTU5MjcgIC01LgogIC0wLiAgICAgICAgIC0wLiAgICAgICBdlIwJaGlnaF9yZXBylIxTWzkwLiAgICAgICAgOTAuICAgICAgICAgNS4gICAgICAgICA1LiAgICAgICAgIDMuMTQxNTkyNyAgNS4KICAxLiAgICAgICAgIDEuICAgICAgIF2UjApfbnBfcmFuZG9tlE51Yi4=",
|
|
@@ -69,31 +68,54 @@
|
|
| 69 |
},
|
| 70 |
"action_space": {
|
| 71 |
":type:": "<class 'gymnasium.spaces.discrete.Discrete'>",
|
| 72 |
-
":serialized:": "
|
| 73 |
"n": "4",
|
| 74 |
"start": "0",
|
| 75 |
"_shape": [],
|
| 76 |
"dtype": "int64",
|
| 77 |
-
"_np_random":
|
| 78 |
},
|
| 79 |
-
"n_envs":
|
| 80 |
-
"
|
| 81 |
-
"gamma": 0.8,
|
| 82 |
-
"gae_lambda": 0.98,
|
| 83 |
-
"ent_coef": 0.01,
|
| 84 |
-
"vf_coef": 0.5,
|
| 85 |
-
"max_grad_norm": 0.5,
|
| 86 |
"batch_size": 64,
|
| 87 |
-
"
|
| 88 |
-
"
|
| 89 |
-
|
| 90 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 91 |
},
|
| 92 |
-
"
|
| 93 |
-
"
|
| 94 |
-
"
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 95 |
"lr_schedule": {
|
| 96 |
":type:": "<class 'function'>",
|
| 97 |
-
":serialized:": "gAWVrQIAAAAAAACMF2Nsb3VkcGlja2xlLmNsb3VkcGlja2xllIwOX21ha2VfZnVuY3Rpb26Uk5QoaACMDV9idWlsdGluX3R5cGWUk5SMCENvZGVUeXBllIWUUpQoSwFLAEsASwFLAUsTQwSIAFMAlE6FlCmMAV+UhZSMSS91c3IvbG9jYWwvbGliL3B5dGhvbjMuMTAvZGlzdC1wYWNrYWdlcy9zdGFibGVfYmFzZWxpbmVzMy9jb21tb24vdXRpbHMucHmUjARmdW5jlEuEQwIEAZSMA3ZhbJSFlCl0lFKUfZQojAtfX3BhY2thZ2VfX5SMGHN0YWJsZV9iYXNlbGluZXMzLmNvbW1vbpSMCF9fbmFtZV9flIwec3RhYmxlX2Jhc2VsaW5lczMuY29tbW9uLnV0aWxzlIwIX19maWxlX1+UjEkvdXNyL2xvY2FsL2xpYi9weXRob24zLjEwL2Rpc3QtcGFja2FnZXMvc3RhYmxlX2Jhc2VsaW5lczMvY29tbW9uL3V0aWxzLnB5lHVOTmgAjBBfbWFrZV9lbXB0eV9jZWxslJOUKVKUhZR0lFKUaACMEl9mdW5jdGlvbl9zZXRzdGF0ZZSTlGgffZR9lChoFowEZnVuY5SMDF9fcXVhbG5hbWVfX5SMGWNvbnN0YW50X2ZuLjxsb2NhbHM+LmZ1bmOUjA9fX2Fubm90YXRpb25zX1+
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 98 |
}
|
| 99 |
}
|
|
|
|
| 1 |
{
|
| 2 |
"policy_class": {
|
| 3 |
":type:": "<class 'abc.ABCMeta'>",
|
| 4 |
+
":serialized:": "gAWVMAAAAAAAAACMHnN0YWJsZV9iYXNlbGluZXMzLmRxbi5wb2xpY2llc5SMCURRTlBvbGljeZSTlC4=",
|
| 5 |
+
"__module__": "stable_baselines3.dqn.policies",
|
| 6 |
+
"__annotations__": "{'q_net': <class 'stable_baselines3.dqn.policies.QNetwork'>, 'q_net_target': <class 'stable_baselines3.dqn.policies.QNetwork'>}",
|
| 7 |
+
"__doc__": "\n Policy class with Q-Value Net and target net for DQN\n\n :param observation_space: Observation space\n :param action_space: Action space\n :param lr_schedule: Learning rate schedule (could be constant)\n :param net_arch: The specification of the policy and value networks.\n :param activation_fn: Activation function\n :param features_extractor_class: Features extractor to use.\n :param features_extractor_kwargs: Keyword arguments\n to pass to the features extractor.\n :param normalize_images: Whether to normalize images or not,\n dividing by 255.0 (True by default)\n :param optimizer_class: The optimizer to use,\n ``th.optim.Adam`` by default\n :param optimizer_kwargs: Additional keyword arguments,\n excluding the learning rate, to pass to the optimizer\n ",
|
| 8 |
+
"__init__": "<function DQNPolicy.__init__ at 0x79f32a84ee60>",
|
| 9 |
+
"_build": "<function DQNPolicy._build at 0x79f32a84eef0>",
|
| 10 |
+
"make_q_net": "<function DQNPolicy.make_q_net at 0x79f32a84ef80>",
|
| 11 |
+
"forward": "<function DQNPolicy.forward at 0x79f32a84f010>",
|
| 12 |
+
"_predict": "<function DQNPolicy._predict at 0x79f32a84f0a0>",
|
| 13 |
+
"_get_constructor_parameters": "<function DQNPolicy._get_constructor_parameters at 0x79f32a84f130>",
|
| 14 |
+
"set_training_mode": "<function DQNPolicy.set_training_mode at 0x79f32a84f1c0>",
|
|
|
|
|
|
|
|
|
|
|
|
|
| 15 |
"__abstractmethods__": "frozenset()",
|
| 16 |
+
"_abc_impl": "<_abc._abc_data object at 0x79f32a8569c0>"
|
| 17 |
},
|
| 18 |
"verbose": 1,
|
| 19 |
"policy_kwargs": {},
|
| 20 |
+
"num_timesteps": 750000,
|
| 21 |
+
"_total_timesteps": 750000,
|
| 22 |
"_num_timesteps_at_start": 0,
|
| 23 |
+
"seed": 42,
|
| 24 |
"action_noise": null,
|
| 25 |
+
"start_time": 1730705438188553600,
|
| 26 |
+
"learning_rate": 0.0001,
|
| 27 |
"tensorboard_log": null,
|
| 28 |
"_last_obs": {
|
| 29 |
":type:": "<class 'numpy.ndarray'>",
|
| 30 |
+
":serialized:": "gAWVlQAAAAAAAACMEm51bXB5LmNvcmUubnVtZXJpY5SMC19mcm9tYnVmZmVylJOUKJYgAAAAAAAAALrGyD6r7zg/LrIIPh2l5b23KnQ8guCsOwAAAAAAAAAAlIwFbnVtcHmUjAVkdHlwZZSTlIwCZjSUiYiHlFKUKEsDjAE8lE5OTkr/////Sv////9LAHSUYksBSwiGlIwBQ5R0lFKULg=="
|
| 31 |
},
|
| 32 |
"_last_episode_starts": {
|
| 33 |
":type:": "<class 'numpy.ndarray'>",
|
| 34 |
+
":serialized:": "gAWVdAAAAAAAAACMEm51bXB5LmNvcmUubnVtZXJpY5SMC19mcm9tYnVmZmVylJOUKJYBAAAAAAAAAAGUjAVudW1weZSMBWR0eXBllJOUjAJiMZSJiIeUUpQoSwOMAXyUTk5OSv////9K/////0sAdJRiSwGFlIwBQ5R0lFKULg=="
|
| 35 |
},
|
| 36 |
+
"_last_original_obs": {
|
| 37 |
+
":type:": "<class 'numpy.ndarray'>",
|
| 38 |
+
":serialized:": "gAWVlQAAAAAAAACMEm51bXB5LmNvcmUubnVtZXJpY5SMC19mcm9tYnVmZmVylJOUKJYgAAAAAAAAAGYXyD4BlTk/WtgIPqcy971W2G888rqYOwAAAAAAAAAAlIwFbnVtcHmUjAVkdHlwZZSTlIwCZjSUiYiHlFKUKEsDjAE8lE5OTkr/////Sv////9LAHSUYksBSwiGlIwBQ5R0lFKULg=="
|
| 39 |
+
},
|
| 40 |
+
"_episode_num": 1355,
|
| 41 |
"use_sde": false,
|
| 42 |
"sde_sample_freq": -1,
|
| 43 |
+
"_current_progress_remaining": 0.0,
|
| 44 |
"_stats_window_size": 100,
|
| 45 |
"ep_info_buffer": {
|
| 46 |
":type:": "<class 'collections.deque'>",
|
| 47 |
+
":serialized:": "gAWVNgwAAAAAAACMC2NvbGxlY3Rpb25zlIwFZGVxdWWUk5QpS2SGlFKUKH2UKIwBcpRHQGPzF+uvECOMAWyUTWUCjAF0lEdAmBSAJHAh0XV9lChoBkdAG49d/rjYI2gHTegDaAhHQJggmqBEroZ1fZQoaAZHQEevSncclw9oB03oA2gIR0CYKqcz67/XdX2UKGgGR0Bp7KUkfLcLaAdN2wFoCEdAmC5H6ZYxL3V9lChoBkdAY1z6KtPpIWgHTQgDaAhHQJg2gJF9a2Z1fZQoaAZHwFZUBLwnYxtoB03oA2gIR0CYQMcyWRigdX2UKGgGR8BBl9d3Sro4aAdN6ANoCEdAmErcbJfYz3V9lChoBkdAcABsYl6Z6WgHTb0BaAhHQJhOLrv9cbB1fZQoaAZHwEj0hZha1TloB03oA2gIR0CYVxt6X0GvdX2UKGgGR0AxnETg2qDLaAdN6ANoCEdAmF8xttQ9BHV9lChoBkdAZKaWpIczZmgHTd4DaAhHQJhn5dcB2fV1fZQoaAZHQF7yAJswco9oB01+AmgIR0CYbOe4TbnHdX2UKGgGR0Bicytga3qiaAdNewNoCEdAmHcXxaxHG3V9lChoBkdAYWj6SDAaemgHTZ4DaAhHQJh+JyFPBSF1fZQoaAZHwEwCE8JUo8ZoB00CAmgIR0CYgksaKk2xdX2UKGgGR0Bj3/sTnJT3aAdNWAJoCEdAmIbbl/6O53V9lChoBkfAKBVxbSqlxmgHTegBaAhHQJiKYrvsqrl1fZQoaAZHwEKURigCfYloB01CAmgIR0CYjsc2itaIdX2UKGgGR8BVvNWEK3NLaAdNEAFoCEdAmJCdZq20A3V9lChoBkfAX4VlZowmFGgHTV4DaAhHQJiYSHj6vaF1fZQoaAZHQGcUqDTSb6RoB00oA2gIR0CYnwg5R0lrdX2UKGgGR8BCf8mShakiaAdN+AFoCEdAmKMQnUlRg3V9lChoBkdAaVPnvlU6xWgHTWcCaAhHQJiolfBvaUR1fZQoaAZHwFPtG0eEIxBoB0uqaAhHQJippG3F1jl1fZQoaAZHQGMxStmtheBoB01WAmgIR0CYrqobGWD6dX2UKGgGR0BZqfR/mT1TaAdN2AJoCEdAmLVHGOuJUHV9lChoBkdAZ0ScI7eVLWgHTR4CaAhHQJi5eKtPpIN1fZQoaAZHQFoKiZv1lGxoB03oA2gIR0CYwJuw5eZ5dX2UKGgGR0BkDuXPZ7HAaAdNXQFoCEdAmMMVC5VfeHV9lChoBkdAWqhxffGdZ2gHTUkDaAhHQJjK8yAQQMB1fZQoaAZHQGX6ojGDL8toB031AWgIR0CYzroAXEZSdX2UKGgGR0BpV4XoC+10aAdNiAJoCEdAmNQjVx0dR3V9lChoBkfAVpe01IiC8WgHTTEBaAhHQJjWPTnaFmF1fZQoaAZHQFecNaQmu1ZoB01bA2gIR0CY3zRP420idX2UKGgGR0Bmwym8/UvxaAdNUQJoCEdAmOQ7mMfignV9lChoBkdAZqBy/bj942gHTWgCaAhHQJjotgOSW7h1fZQoaAZHQFUwt5D7ZWdoB00XA2gIR0CY8IOfNA1OdX2UKGgGR0BppSjDbah6aAdNwAFoCEdAmPO5tSAH3XV9lChoBkdAZVJnIyTINmgHTeUCaAhHQJj5hKsdT5x1fZQoaAZHwFyGMIu5BkZoB0v7aAhHQJj7H5mAbyZ1fZQoaAZHQGXC35N47ihoB00GAmgIR0CY/ur8zhxYdX2UKGgGR0Bt1jakAPupaAdN4gFoCEdAmQJDRD1GsnV9lChoBkdAZRDUI9kjHGgHTW4CaAhHQJkHfVRUFSt1fZQoaAZHQGQZ5tvXK8toB00XA2gIR0CZDnYeDFqBdX2UKGgGR0Bo5uK4x1xLaAdNyAFoCEdAmRGhgiNbT3V9lChoBkfAUyQSkCV8kWgHS8doCEdAmRLkXgtOEnV9lChoBkfAUKQNy5qdpmgHS7doCEdAmRP9gfEGaHV9lChoBkfAZZLVjqfOEGgHTa0CaAhHQJkZaEoOQQt1fZQoaAZHwFenr1dxAB1oB0v0aAhHQJka9BUrCnB1fZQoaAZHQCVVpItlI3BoB03oA2gIR0CZJot4A0bcdX2UKGgGR8BTnOumrKeTaAdL5WgIR0CZJ/UPhAGCdX2UKGgGR0Bv8HbKzRhMaAdNswJoCEdAmS2+oHcDbXV9lChoBkdAWZkkVvddmmgHTUEDaAhHQJk2WoUBXCF1fZQoaAZHwFLdornTy8VoB00fAWgIR0CZOC+kP+XJdX2UKGgGR8BpkzIzWPLgaAdNhANoCEdAmT9ljZtelnV9lChoBkdAYScCrcTJyWgHTUkDaAhHQJlIgA+6iCd1fZQoaAZHwFG9F6Rhc7hoB01GAWgIR0CZSwTINmUXdX2UKGgGR8BZexCdBjWkaAdNPAFoCEdAmU00nkT6BXV9lChoBkfAWI+26TW5H2gHTS8CaAhHQJlRQwnH/951fZQoaAZHQGvtHrhR64VoB00OAmgIR0CZVSnZ00WNdX2UKGgGR0Bk5NaMaS9vaAdN6wJoCEdAmVudAkcCHXV9lChoBkdAYNn/LDAJs2gHTZMDaAhHQJlitNpM6BB1fZQoaAZHwFRhlyBCladoB0uaaAhHQJljoX/HYHx1fZQoaAZHwDQEVRDTjNpoB01oAWgIR0CZZgdf9gnddX2UKGgGR0BkN/BP9DQaaAdNXQJoCEdAmWwDMmnfmHV9lChoBkdAZzmXHim2s2gHTYECaAhHQJlx21G9YfZ1fZQoaAZHwFpS19v0h/1oB03EAWgIR0CZdWMglnh9dX2UKGgGR8BS0LdJrcj8aAdNAQFoCEdAmXcGCAc1fnV9lChoBkfAVAqrzXjEN2gHS4hoCEdAmXfSQxN7B3V9lChoBkdAaMrYSQHRkWgHTQADaAhHQJl9RcC5mRN1fZQoaAZHQGtEhOxjawloB01FAmgIR0CZgazvJA+qdX2UKGgGR0BptJw2l2vCaAdNDAJoCEdAmYVtfsu3+nV9lChoBkfAVy6xD9fkWGgHS+9oCEdAmYbqCxu89XV9lChoBkdAY9edgfEGaGgHTU4DaAhHQJmN6suFpPB1fZQoaAZHwFlIyYoiLVFoB0unaAhHQJmO6Ef1Yhd1fZQoaAZHwFdD0l7dBSloB0uOaAhHQJmPxCiRGMJ1fZQoaAZHwFeatapxWDJoB0uTaAhHQJmQpU0elsR1fZQoaAZHQGdZwLE1l5JoB03nAmgIR0CZl2mWMS9NdX2UKGgGR0BnsA5myxA0aAdN5AFoCEdAmZuFzuF6A3V9lChoBkfAVzgSsbNr02gHS/toCEdAmZ0jO1OTJXV9lChoBkdAb9xFOO8012gHTc8BaAhHQJmgnPhQ3xZ1fZQoaAZHQFm2Sa3I+4doB03WAmgIR0CZpwTG5tm+dX2UKGgGR0Bqh2RJVbRnaAdNMwJoCEdAmasH5BTn73V9lChoBkdAM3bP2PDHfmgHTWoCaAhHQJmvSC04R291fZQoaAZHQGxdR8lXzUZoB03iAWgIR0CZsrkpI+W4dX2UKGgGR0BnjhjjJdSmaAdNAAJoCEdAmbcj101ZT3V9lChoBkfAXvnwCr92o2gHTfkBaAhHQJm7CtaIN3J1fZQoaAZHQGLyxHf/FR5oB02bA2gIR0CZxADXOGCadX2UKGgGR0BmvOV9nbqRaAdNTwJoCEdAmchchTwUg3V9lChoBkdAYc80fozN2WgHTf8CaAhHQJnPWEBbOeJ1fZQoaAZHwFcZ/wAlv61oB0vHaAhHQJnQkaXKKYR1fZQoaAZHwGqgO4oZydZoB02vA2gIR0CZ2QzSThYOdX2UKGgGR0BoPkZm7J4jaAdN/wJoCEdAmeCdmQKa5XV9lChoBkdAPKdKh+OOsGgHTXgBaAhHQJnjU7LdN351fZQoaAZHQGMstSydFv1oB03TAmgIR0CZ6csabWmQdX2UKGgGR8BOuPHT7VJ+aAdNKgFoCEdAmeu+gctGu3V9lChoBkdAaIVFPznRs2gHTY0CaAhHQJnyWj7ALzB1fZQoaAZHQGH+oLPUrkNoB02sAmgIR0CZ9/FuvUz9dX2UKGgGR8BQ2HiWE9McaAdN6ANoCEdAmgEvVqesgnV9lChoBkdAYQclRgqmTGgHTRwCaAhHQJoFHm8ujAV1ZS4="
|
| 48 |
},
|
| 49 |
"ep_success_buffer": {
|
| 50 |
":type:": "<class 'collections.deque'>",
|
| 51 |
":serialized:": "gAWVIAAAAAAAAACMC2NvbGxlY3Rpb25zlIwFZGVxdWWUk5QpS2SGlFKULg=="
|
| 52 |
},
|
| 53 |
+
"_n_updates": 187250,
|
| 54 |
"observation_space": {
|
| 55 |
":type:": "<class 'gymnasium.spaces.box.Box'>",
|
| 56 |
":serialized:": "gAWVdgIAAAAAAACMFGd5bW5hc2l1bS5zcGFjZXMuYm94lIwDQm94lJOUKYGUfZQojAVkdHlwZZSMBW51bXB5lIwFZHR5cGWUk5SMAmY0lImIh5RSlChLA4wBPJROTk5K/////0r/////SwB0lGKMDWJvdW5kZWRfYmVsb3eUjBJudW1weS5jb3JlLm51bWVyaWOUjAtfZnJvbWJ1ZmZlcpSTlCiWCAAAAAAAAAABAQEBAQEBAZRoCIwCYjGUiYiHlFKUKEsDjAF8lE5OTkr/////Sv////9LAHSUYksIhZSMAUOUdJRSlIwNYm91bmRlZF9hYm92ZZRoESiWCAAAAAAAAAABAQEBAQEBAZRoFUsIhZRoGXSUUpSMBl9zaGFwZZRLCIWUjANsb3eUaBEoliAAAAAAAAAAAAC0wgAAtMIAAKDAAACgwNsPScAAAKDAAAAAgAAAAICUaAtLCIWUaBl0lFKUjARoaWdolGgRKJYgAAAAAAAAAAAAtEIAALRCAACgQAAAoEDbD0lAAACgQAAAgD8AAIA/lGgLSwiFlGgZdJRSlIwIbG93X3JlcHKUjFtbLTkwLiAgICAgICAgLTkwLiAgICAgICAgIC01LiAgICAgICAgIC01LiAgICAgICAgIC0zLjE0MTU5MjcgIC01LgogIC0wLiAgICAgICAgIC0wLiAgICAgICBdlIwJaGlnaF9yZXBylIxTWzkwLiAgICAgICAgOTAuICAgICAgICAgNS4gICAgICAgICA1LiAgICAgICAgIDMuMTQxNTkyNyAgNS4KICAxLiAgICAgICAgIDEuICAgICAgIF2UjApfbnBfcmFuZG9tlE51Yi4=",
|
|
|
|
| 68 |
},
|
| 69 |
"action_space": {
|
| 70 |
":type:": "<class 'gymnasium.spaces.discrete.Discrete'>",
|
| 71 |
+
":serialized:": "gAWVowEAAAAAAACMGWd5bW5hc2l1bS5zcGFjZXMuZGlzY3JldGWUjAhEaXNjcmV0ZZSTlCmBlH2UKIwBbpSMFW51bXB5LmNvcmUubXVsdGlhcnJheZSMBnNjYWxhcpSTlIwFbnVtcHmUjAVkdHlwZZSTlIwCaTiUiYiHlFKUKEsDjAE8lE5OTkr/////Sv////9LAHSUYkMIBAAAAAAAAACUhpRSlIwFc3RhcnSUaAhoDkMIAAAAAAAAAACUhpRSlIwGX3NoYXBllCmMBWR0eXBllGgOjApfbnBfcmFuZG9tlIwUbnVtcHkucmFuZG9tLl9waWNrbGWUjBBfX2dlbmVyYXRvcl9jdG9ylJOUjAVQQ0c2NJRoG4wUX19iaXRfZ2VuZXJhdG9yX2N0b3KUk5SGlFKUfZQojA1iaXRfZ2VuZXJhdG9ylIwFUENHNjSUjAVzdGF0ZZR9lChoJooRXAzIT2ucUDgplpt+Ybl+xgCMA2luY5SKEf+rJY3i8spmbkGoyTZUUPoAdYwKaGFzX3VpbnQzMpRLAYwIdWludGVnZXKUSqkYhRJ1YnViLg==",
|
| 72 |
"n": "4",
|
| 73 |
"start": "0",
|
| 74 |
"_shape": [],
|
| 75 |
"dtype": "int64",
|
| 76 |
+
"_np_random": "Generator(PCG64)"
|
| 77 |
},
|
| 78 |
+
"n_envs": 1,
|
| 79 |
+
"buffer_size": 100000,
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 80 |
"batch_size": 64,
|
| 81 |
+
"learning_starts": 1000,
|
| 82 |
+
"tau": 1.0,
|
| 83 |
+
"gamma": 0.99,
|
| 84 |
+
"gradient_steps": 1,
|
| 85 |
+
"optimize_memory_usage": false,
|
| 86 |
+
"replay_buffer_class": {
|
| 87 |
+
":type:": "<class 'abc.ABCMeta'>",
|
| 88 |
+
":serialized:": "gAWVNQAAAAAAAACMIHN0YWJsZV9iYXNlbGluZXMzLmNvbW1vbi5idWZmZXJzlIwMUmVwbGF5QnVmZmVylJOULg==",
|
| 89 |
+
"__module__": "stable_baselines3.common.buffers",
|
| 90 |
+
"__doc__": "\n Replay buffer used in off-policy algorithms like SAC/TD3.\n\n :param buffer_size: Max number of element in the buffer\n :param observation_space: Observation space\n :param action_space: Action space\n :param device: PyTorch device\n :param n_envs: Number of parallel environments\n :param optimize_memory_usage: Enable a memory efficient variant\n of the replay buffer which reduces by almost a factor two the memory used,\n at a cost of more complexity.\n See https://github.com/DLR-RM/stable-baselines3/issues/37#issuecomment-637501195\n and https://github.com/DLR-RM/stable-baselines3/pull/28#issuecomment-637559274\n Cannot be used in combination with handle_timeout_termination.\n :param handle_timeout_termination: Handle timeout termination (due to timelimit)\n separately and treat the task as infinite horizon task.\n https://github.com/DLR-RM/stable-baselines3/issues/284\n ",
|
| 91 |
+
"__init__": "<function ReplayBuffer.__init__ at 0x79f32a8232e0>",
|
| 92 |
+
"add": "<function ReplayBuffer.add at 0x79f32a823370>",
|
| 93 |
+
"sample": "<function ReplayBuffer.sample at 0x79f32a823400>",
|
| 94 |
+
"_get_samples": "<function ReplayBuffer._get_samples at 0x79f32a823490>",
|
| 95 |
+
"__abstractmethods__": "frozenset()",
|
| 96 |
+
"_abc_impl": "<_abc._abc_data object at 0x79f2aea16080>"
|
| 97 |
+
},
|
| 98 |
+
"replay_buffer_kwargs": {},
|
| 99 |
+
"train_freq": {
|
| 100 |
+
":type:": "<class 'stable_baselines3.common.type_aliases.TrainFreq'>",
|
| 101 |
+
":serialized:": "gAWVYQAAAAAAAACMJXN0YWJsZV9iYXNlbGluZXMzLmNvbW1vbi50eXBlX2FsaWFzZXOUjAlUcmFpbkZyZXGUk5RLBGgAjBJUcmFpbkZyZXF1ZW5jeVVuaXSUk5SMBHN0ZXCUhZRSlIaUgZQu"
|
| 102 |
},
|
| 103 |
+
"use_sde_at_warmup": false,
|
| 104 |
+
"exploration_initial_eps": 1.0,
|
| 105 |
+
"exploration_final_eps": 0.01,
|
| 106 |
+
"exploration_fraction": 0.1,
|
| 107 |
+
"target_update_interval": 1000,
|
| 108 |
+
"_n_calls": 750000,
|
| 109 |
+
"max_grad_norm": 10,
|
| 110 |
+
"exploration_rate": 0.01,
|
| 111 |
"lr_schedule": {
|
| 112 |
":type:": "<class 'function'>",
|
| 113 |
+
":serialized:": "gAWVrQIAAAAAAACMF2Nsb3VkcGlja2xlLmNsb3VkcGlja2xllIwOX21ha2VfZnVuY3Rpb26Uk5QoaACMDV9idWlsdGluX3R5cGWUk5SMCENvZGVUeXBllIWUUpQoSwFLAEsASwFLAUsTQwSIAFMAlE6FlCmMAV+UhZSMSS91c3IvbG9jYWwvbGliL3B5dGhvbjMuMTAvZGlzdC1wYWNrYWdlcy9zdGFibGVfYmFzZWxpbmVzMy9jb21tb24vdXRpbHMucHmUjARmdW5jlEuEQwIEAZSMA3ZhbJSFlCl0lFKUfZQojAtfX3BhY2thZ2VfX5SMGHN0YWJsZV9iYXNlbGluZXMzLmNvbW1vbpSMCF9fbmFtZV9flIwec3RhYmxlX2Jhc2VsaW5lczMuY29tbW9uLnV0aWxzlIwIX19maWxlX1+UjEkvdXNyL2xvY2FsL2xpYi9weXRob24zLjEwL2Rpc3QtcGFja2FnZXMvc3RhYmxlX2Jhc2VsaW5lczMvY29tbW9uL3V0aWxzLnB5lHVOTmgAjBBfbWFrZV9lbXB0eV9jZWxslJOUKVKUhZR0lFKUaACMEl9mdW5jdGlvbl9zZXRzdGF0ZZSTlGgffZR9lChoFowEZnVuY5SMDF9fcXVhbG5hbWVfX5SMGWNvbnN0YW50X2ZuLjxsb2NhbHM+LmZ1bmOUjA9fX2Fubm90YXRpb25zX1+UfZSMDl9fa3dkZWZhdWx0c19flE6MDF9fZGVmYXVsdHNfX5ROjApfX21vZHVsZV9flGgXjAdfX2RvY19flE6MC19fY2xvc3VyZV9flGgAjApfbWFrZV9jZWxslJOURz8aNuLrHEMthZRSlIWUjBdfY2xvdWRwaWNrbGVfc3VibW9kdWxlc5RdlIwLX19nbG9iYWxzX1+UfZR1hpSGUjAu"
|
| 114 |
+
},
|
| 115 |
+
"batch_norm_stats": [],
|
| 116 |
+
"batch_norm_stats_target": [],
|
| 117 |
+
"exploration_schedule": {
|
| 118 |
+
":type:": "<class 'function'>",
|
| 119 |
+
":serialized:": "gAWVTQMAAAAAAACMF2Nsb3VkcGlja2xlLmNsb3VkcGlja2xllIwOX21ha2VfZnVuY3Rpb26Uk5QoaACMDV9idWlsdGluX3R5cGWUk5SMCENvZGVUeXBllIWUUpQoSwFLAEsASwFLBEsTQyhkAXwAGACIAWsEcgiIAFMAiAJkAXwAGACIAIgCGAAUAIgBGwAXAFMAlE5LAYaUKYwScHJvZ3Jlc3NfcmVtYWluaW5nlIWUjEkvdXNyL2xvY2FsL2xpYi9weXRob24zLjEwL2Rpc3QtcGFja2FnZXMvc3RhYmxlX2Jhc2VsaW5lczMvY29tbW9uL3V0aWxzLnB5lIwEZnVuY5RLckMGDAEEARgClIwDZW5klIwMZW5kX2ZyYWN0aW9ulIwFc3RhcnSUh5QpdJRSlH2UKIwLX19wYWNrYWdlX1+UjBhzdGFibGVfYmFzZWxpbmVzMy5jb21tb26UjAhfX25hbWVfX5SMHnN0YWJsZV9iYXNlbGluZXMzLmNvbW1vbi51dGlsc5SMCF9fZmlsZV9flIxJL3Vzci9sb2NhbC9saWIvcHl0aG9uMy4xMC9kaXN0LXBhY2thZ2VzL3N0YWJsZV9iYXNlbGluZXMzL2NvbW1vbi91dGlscy5weZR1Tk5oAIwQX21ha2VfZW1wdHlfY2VsbJSTlClSlGgdKVKUaB0pUpSHlHSUUpRoAIwSX2Z1bmN0aW9uX3NldHN0YXRllJOUaCN9lH2UKGgYjARmdW5jlIwMX19xdWFsbmFtZV9flIwbZ2V0X2xpbmVhcl9mbi48bG9jYWxzPi5mdW5jlIwPX19hbm5vdGF0aW9uc19flH2UKGgKjAhidWlsdGluc5SMBWZsb2F0lJOUjAZyZXR1cm6UaC91jA5fX2t3ZGVmYXVsdHNfX5ROjAxfX2RlZmF1bHRzX1+UTowKX19tb2R1bGVfX5RoGYwHX19kb2NfX5ROjAtfX2Nsb3N1cmVfX5RoAIwKX21ha2VfY2VsbJSTlEc/hHrhR64Ue4WUUpRoN0c/uZmZmZmZmoWUUpRoN0c/8AAAAAAAAIWUUpSHlIwXX2Nsb3VkcGlja2xlX3N1Ym1vZHVsZXOUXZSMC19fZ2xvYmFsc19flH2UdYaUhlIwLg=="
|
| 120 |
}
|
| 121 |
}
|
ppo-LunarLander-v2/policy.optimizer.pth
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
-
size
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:4d15656fe6c373c5015568a0fdcfd10be0691670e1db8b0e0e7f1681d82b172e
|
| 3 |
+
size 45344
|
ppo-LunarLander-v2/policy.pth
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
-
size
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:5d6203e0e7ee4bed140a4a971e5ed42b434fb97e47bb5f03f46579abe52e96fe
|
| 3 |
+
size 44466
|
replay.mp4
CHANGED
|
Binary files a/replay.mp4 and b/replay.mp4 differ
|
|
|
results.json
CHANGED
|
@@ -1 +1 @@
|
|
| 1 |
-
{"mean_reward": -
|
|
|
|
| 1 |
+
{"mean_reward": -3.8334901999999986, "std_reward": 140.81302644124065, "is_deterministic": true, "n_eval_episodes": 10, "eval_datetime": "2024-11-04T07:58:31.766632"}
|