racingmania-size-1-seed1

Trained on 1 track, seed 1: PPO trained on only 1 generated tracks

Status: evaluated (evaluated; scores available); physics version v1; card generated 2026-11-15 from the result files.

RacingMania is an original 3D time-attack racing game for the browser (three.js + cannon-es RaycastVehicle) with reinforcement-learning drivers trained on a batched PyTorch port of the same car physics. Code, paper and full results: https://github.com/BurnyCoder/reinforcement-learning-in-trackmania-like-racing-game.

Videos

Official evaluation runs of this model replayed in the browser game ("Watch replay"): the car states the evaluator recorded while scoring the run, so each clip is exactly the run its caption describes. More clips, including the controls: https://github.com/BurnyCoder/reinforcement-learning-in-trackmania-like-racing-game#videos.

Best lap of the single-track ablation

• What you see: The training-set-size ablation with one track: the standard recipe trained on a single generated track from the pool. The hand-made track was never part of its training; the results compare how far one training track generalizes against the full pool (hypothesis H2).
• Model: size-1/seed1: PPO, trained by reinforcement learning for 1 billion agent steps on 1 track (seed 1, 1.3 h on one GPU).
• Map: First Flight (campaign-01), a hand-made campaign track, 628 m long with 1 jump and up to 6° of banking; this model never saw it during training.
• Evaluation: official evaluation in results/size-1/seed1/eval_campaign.csv (standing start, most probable action, final checkpoint): finished in 20.14 s; the scripted driver needs 30.17 s, so this is 1.50× its speed. The clip replays this run.
• Why this clip: chosen by a fixed rule as the highest-scoring finished track of the 2 campaign tracks it finished in its official evaluation.

Model

Item	Value
Algorithm	PPO
Architecture	MLP 2x256 tanh (actor and critic), two 3-way heads
Objective	centreline progress
Observation	car state + 20 look-ahead points
Training data	1 generated tracks
Start states	standing + random centreline (p=0.5)
Agent steps trained	1,000,341,504
Wall-clock hours	1.33
Git commit	3e75ae37db322a34f61d1adaf7435d76f72c821b

Evaluation

Success-weighted speed (SWS): mean over tracks of finished x (scripted driver's time / agent's time); 1.0 means finishing every track as fast as the scripted reference driver. One deterministic run per track from a standing start with the most probable action; intervals are 95 % percentile bootstrap intervals over tracks (one seed).

Track set	Tracks	SWS [95 % CI]	Finished
Campaign (hand-made)	5	0.584 [0.000, 1.184]	40.0 %
Out-of-range test	187	0.214 [0.142, 0.291]	14.4 %
Test (unseen generated)	971	0.539 [0.495, 0.583]	36.5 %
Unfiltered test	200	0.544 [0.445, 0.644]	36.0 %
Own training tracks	1	1.636 [1.636, 1.636]	100.0 %
Training sample	1000	0.499 [0.455, 0.542]	33.5 %
Validation	197	0.508 [0.408, 0.607]	34.0 %

How to use

Files: final.pt (PyTorch checkpoint: agent and norm state dicts, ppo and env_cfg settings, obs_dim), policy.onnx (observation function + frozen normaliser + actor in one graph, as run by the browser game), run.json, training and validation curves, and per-track evaluation CSVs under eval/.

PyTorch, inside a clone of the repository (uv sync):

import torch
from racingmania.env.obs import obs_size
from racingmania.rl.networks import ActorCritic
from racingmania.rl.normalizer import RunningNorm

sd = torch.load("final.pt", map_location="cpu", weights_only=False)
k = sd["env_cfg"]["window_k"]
agent = ActorCritic(obs_size(k), tuple(sd["ppo"]["hidden"]), sd["ppo"]["action_mode"])
agent.load_state_dict(sd["agent"]); agent.eval()
norm = RunningNorm(obs_size(k)); norm.load_state_dict(sd["norm"]); norm.frozen = True
# action = agent.act(norm(env.observe().float()), deterministic=True)[1]   (see racingmania.runs.load_policy)

ONNX (inputs and outputs are specified in docs/formats.md, section "Policy model"):

import numpy as np, onnxruntime as ort
sess = ort.InferenceSession("policy.onnx")
inputs = {i.name: np.zeros([d if isinstance(d, int) else 1 for d in i.shape], np.float32) for i in sess.get_inputs()}
steer_logits, pedal_logits = sess.run(None, inputs)
steer, pedal = int(steer_logits.argmax()), int(pedal_logits.argmax())  # 0/1/2 = left/none/right, gas/coast/brake

To watch it drive, open the game (web/index.html, or the GitHub Pages site), choose it in a track's AI drivers table and press "Watch AI" (live in the browser) or "Watch replay" (its official evaluation run).

Training configuration (from run.json)

Setting	Value
total_steps	1000000000
num_envs	16384
num_steps	32
lr	0.001
adaptive_lr	True
desired_kl	0.01
lr_min	1e-05
lr_max	0.01
gamma	0.99
gae_lambda	0.95
num_minibatches	4
update_epochs	5
clip_coef	0.2
ent_coef	0.01
vf_coef	1.0
max_grad_norm	1.0
weight_decay	0.0
hidden	[256, 256]
action_mode	factored
eval_every_minutes	10.0
checkpoint_every_minutes	10.0
rotate_tracks_every	20
loaded_tracks	4096
seed	1
compile	True
env.random_start_prob	0.5
env.random_start_speed	[0.0, 30.0]
env.random_start_lateral	0.5
env.max_steps	1500
env.reward_scale	0.1
env.window_k	20
env.window_stride	3
env.blind	False
env.time_cost	0.0
env.fall_penalty	0.0
env.hint_weight	0.0
env.scripted_starts	False

Licence and citation

MIT. Author: Libor "Burny" Burian. Repository: https://github.com/BurnyCoder/reinforcement-learning-in-trackmania-like-racing-game.