PPO on CarRacing-v3, with my own reward (BULSHY.FM build #1)

A PPO agent that drives Gymnasium's CarRacing-v3 from pixels. It is run 4 of a series where every run changed one thing in the reward, and each one was benchmarked on the same 100 tracks.

Benchmark

100 fixed tracks (seeds 1000-1099), deterministic actions, scored with the game's own reward (the shaped training reward plays no part):

Run Mean ± 95% CI vs the game's own reward (paired)
1. the game's own reward 540 ± 34 —
4. this model 742 ± 31 +202 ± 35 per track, better on 87 of 100

Median 783; 15% of tracks score 900 or more ("solved" is a mean of 900, so not solved yet). Its main limit is speed: on 85% of tracks it runs out of the 1000-frame limit before finishing the lap.

Training reward

The game's reward (+1000/N per new track tile, -0.1 per frame), plus per game frame:

  • grass: -0.1 when no wheel touches the road
  • centre band: up to -0.05 for leaving a band around the centre line (half the road on straights, the whole road before curves)
  • speed: up to +0.05 for speed along the track, on the road only
  • steering: -0.1 per unit of steering change between decisions (against wobbling)

Setup

  • Observation: 64x64 grayscale, the last 2 frames stacked, 1 decision every 2 frames.
  • PPO (Stable-Baselines3 2.9.2) CnnPolicy, gSDE, lr 1e-4 linear to 0, n_steps 4096 total, batch 128, 10 epochs, gamma 0.99, GAE lambda 0.95, clip 0.2, actor and critic heads of 256 (GELU), rewards normalized with VecNormalize. 4M decisions.

Files

  • final.zip: the SB3 model. final_vecnormalize.pkl: the reward normalization (not needed to drive).
  • bench.csv: every benchmark track (return, frames, share of tiles visited, lap finished, wrong-way share).

To drive it you need the same observation wrappers (resize to 64x64, grayscale, skip 2 frames, stack 2), which are in the training code: https://github.com/GOATsann/goatsan-lab.

Made by @_goatsan.

Downloads last month
27
Video Preview
loading

Evaluation results