LaithSakk/ppo-LunarLander-v2

LunarLander-v2 Reinforcement Learning with Stable Baselines3 PPO

This project demonstrates how to train a reinforcement learning agent to master the LunarLander-v2 environment using the Proximal Policy Optimization (PPO) algorithm from the Stable Baselines3 library.

Environment

The agent is trained in the LunarLander-v2 environment provided by Gymnasium. The goal in this environment is to land a lunar module safely between two flags on the landing pad.

Algorithm

We use the Proximal Policy Optimization (PPO) algorithm, a popular and effective on-policy algorithm for reinforcement learning.

Project Structure

• LunarLander.py: Contains the code for training the PPO agent.
• push.py: Script to package and push the trained model to the Hugging Face Hub.
• test.py: Script to load a trained model and visualize its performance.

Setup

To run this project, you'll need to install the necessary libraries:

pip install gymnasium stable-baselines3[extra] huggingface_sb3

Training the Agent

The LunarLander.py script handles the training process. It initializes a PPO model with a multi-layer perceptron (MLP) policy and trains it for a specified number of timesteps.

# Example snippet from LunarLander.py
env = make_vec_env('LunarLander-v2', n_envs=16) # Using vectorized environments for faster training

model = PPO(
    "MlpPolicy",
    env=env,
    n_steps=1024,
    batch_size=64,
    n_epochs=8,
    gamma=0.996,
    gae_lambda=0.98,
    ent_coef=0.001,                               
    device="cuda", # Use "cpu" if you don't have a GPU
    policy_kwargs = dict(net_arch=dict(pi=,vf=) ),
    verbose=1,
)

model.learn(total_timesteps=1000000)
model.save("ppo-LunarLander-v2")

To train the model, simply run:

python /home/latih/Rainforcment Leaning Hugging face/LunarLander.py

Evaluating the Agent

After training, you can evaluate the agent's performance using the evaluate_policy function. The LunarLander.py script already includes an evaluation step. For a more dedicated evaluation and visualization, you can use test.py.

python /home/latih/Rainforcment Leaning Hugging face/test.py

Pushing to Hugging Face Hub

The push.py script allows you to upload your trained model to the Hugging Face Hub, making it easy to share and collaborate.

Before pushing, make sure to set your repo_id in push.py to your desired repository name on the Hugging Face Hub.

# Example snippet from push.py
repo_id = "your_username/ppo-LunarLander-v2"  # CHANGE THIS
model = PPO.load("ppo-LunarLander-v2")
package_to_hub(
    model=model,
    model_name="ppo-LunarLander-v2",
    model_architecture="PPO",
    env_id="LunarLander-v2",
    eval_env=eval_env,
    repo_id=repo_id,
    commit_message="Upload PPO LunarLander-v2 trained agent",
)

To push your model, run:

python /home/latih/Rainforcment Leaning Hugging face/push.py

Usage of a Trained Model

Once a model is trained and saved (or downloaded from the Hugging Face Hub), you can load it and use it to interact with the environment:

import gymnasium as gym
from stable_baselines3 import PPO

env = gym.make("LunarLander-v2", render_mode="human")
model = PPO.load("ppo-LunarLander-v2") # Or from Hugging Face Hub

obs, info = env.reset()
done = False
while not done:
    action, _ = model.predict(obs, deterministic=True)
    obs, reward, terminated, truncated, info = env.step(action)
    done = terminated or truncated
env.close()

Feel free to experiment with different hyperparameters and network architectures to improve the agent's performance!