cartmind-1.0-223K
cartmind is a 223,190-parameter convolutional network that plays Cart Surfer, the mine-cart minigame from Club Penguin, using only the pixels on screen. It finishes randomly generated courses of any length without losing a life, grinds every corner, and keeps up an alternating backflip/spin trick rhythm the whole way down.
The game runs in Club Pyguin, a faithful Python recreation of the original Flash minigame. The model sees only the last four frames, downscaled to 84Γ84 grayscale. It has no access to the game's internal state.
βΆοΈ Watch the video on YouTube
Results
All numbers come from deterministic greedy play. A "course" is a randomly generated track with 7, 14, or 28 turns (the classic track has 7).
| Evaluation | Courses finished | Lives lost | Turns cleared | Corners ground | Mean score (7 / 14 / 28 turns) |
|---|---|---|---|---|---|
| Validation (seeds 1,051,000+) | 30 / 30 | 0 | 490 / 490 | 490 / 490 | 3,694 / 7,184 / 13,266 |
| Untouched final test (seeds 2,060,000+) | 150 / 150 | 0 | 2,450 / 2,450 | 2,450 / 2,450 | 3,595 / 6,851 / 12,781 |
| Classic track | 20 / 20 | 0 | 140 / 140 | 140 / 140 | 3,280 |
- The final-test courses were never used for training, validation, or model selection. The test ran once, after the weights were frozen.
- Over the 150 final-test courses, the model performed 6,350 backflips and 6,604 spins (about 86 tricks per course). 95% of them landed for points.
- The model outscores its own teacher on the classic track (3,280 vs. 2,853).
Quick start
You need Python 3.11 or newer.
pip install -U huggingface_hub # provides the `hf` command
hf download benank/cartmind-1.0-223K --local-dir cartmind-1.0-223K
cd cartmind-1.0-223K
python -m venv .venv && source .venv/bin/activate
pip install torch safetensors .
python play.py --check
pip install . installs the bundled Cart Surfer recreation (see
Watch it play). --check loads the weights and runs one
decision on a dummy frame. It needs no game assets.
Use the model in your own loop
play.py is one self-contained file. Import CartSurferAgent and give it one
game frame per tick, either a pygame Surface or an H Γ W Γ 3 uint8 RGB
array. It returns an action index.
from play import ACTIONS, CartSurferAgent
agent = CartSurferAgent() # loads model.safetensors
action = agent.reset(first_frame) # first frame of a run
# Pseudocode: `game` is your Cart Surfer loop.
while running:
frame = game.step(action) # advance one tick (24 per second)
action = agent.step(frame)
print(ACTIONS[action]) # nothing, left, right, up, down, space
play() in play.py is a complete working example that uses the bundled
game.
Watch it play
The club_pyguin/ folder holds the Cart Surfer part of Club Pyguin, a
Python recreation of the original Flash minigame. It includes the game engine,
renderer, random-course generator, and asset extractor. The model was trained
and evaluated in this exact code.
This repository contains no game graphics or sounds. You extract them once from your own copy of the original 2006 game file.
1. Get the original game file
You need CartSurfer2006.swf, the original version of the minigame from
before stamps were added:
- size: 77,163 bytes;
- SHA-256:
fd30e04c8de51fbfd970841fac9e761d0f9c41481f77b3bee74524a42fe188f9.
The extractor rejects any other file, because other versions draw and play
differently. Check yours with shasum -a 256 CartSurfer2006.swf.
2. Get JPEXS FFDec (the Flash decompiler)
Install a Java runtime (java -version should work). Download and unzip the
latest JPEXS release from
github.com/jindrapetrik/jpexs-decompiler/releases.
You'll point the extractor at ffdec-cli.jar inside the unzipped folder.
Keep it there, because it needs the lib/ folder next to it.
3. Extract the assets
cart-surfer-extract extract \
--source /path/to/CartSurfer2006.swf \
--ffdec /path/to/ffdec/ffdec-cli.jar \
--output cart-surfer-assets
This takes a few seconds. It writes the sprites, shapes, sounds, and a
manifest.json to cart-surfer-assets/.
4. Watch the model play
python play.py --assets cart-surfer-assets # classic track, in a window
python play.py --assets cart-surfer-assets --turns 28 # random 28-turn course
python play.py --assets cart-surfer-assets --turns 14 --seed 7 --headless
The window stays on the final frame when the run ends. Press Esc to close it.
--headless plays without a window and prints the result. Each --turns/--seed pair is a different, reproducible course.
5. Play it yourself
The same install gives you the full recreation, with the title screen, instructions, sounds, and game over:
cart-surfer --assets cart-surfer-assets
| Key | Action |
|---|---|
| Enter / Space | Start, and advance the instructions (press I on the title for instructions) |
| β / β | Lean, or turn through a corner |
| β then β / β | Wheelie, then grind around a corner (the move the model uses on every corner) |
| β then β / β | Surf stance, then slide |
| Space | Jump. From a wheelie: backflip. From surf stance: ollie. Also recovers after a crash |
| P / D / Esc | Pause / debug overlay / quit |
Model
| Input | 4 stacked 84Γ84 grayscale frames (uint8), scaled to [0, 1] inside the model |
| Architecture | Conv 8Γ8/4 (16) β Conv 4Γ4/2 (32) β Conv 3Γ3/1 (32) β Linear 1568β128 β Linear 128β6, ReLU between layers |
| Output | Logits for 6 actions: nothing, left, right, up, down, space. The agent takes the argmax |
| Parameters | 223,190 (894 KB as float32 safetensors) |
| Decision rate | One action per game tick, 24 per second |
Preprocessing must match training exactly. play.py resizes each 760Γ480
frame with pygame's smoothscale, then converts it to grayscale as
(77R + 150G + 29B) >> 8. At the start of a run, the first frame fills all
four stack slots.
The agent adds one small rule on top of the network. Crash recovery needs a fresh Space key press, so if the model outputs Space three ticks in a row, the third tick becomes "release all keys". The rule uses only the agent's own past outputs, never game state.
How it was made
Training ran in three stages, about 40 minutes in total on one Apple M3 Pro laptop.
State-based teacher (Double DQN). A small Q-network read the game's 17-value internal state and learned through a curriculum:
- safe completion with score reward off (1,200 episodes);
- grind discovery with shaped bonuses for grinding corners (Down + direction) (600 episodes);
- stabilization with score reward restored and smaller grind bonuses (750 episodes).
Snapshots were saved every 25 episodes and screened on 30 held-out courses. The chosen teacher finished 150/150 courses with zero lives lost and ground all 2,450 corners.
Behavior cloning. The teacher played 80 random courses (40 at 7 turns, 20 at 14, 20 at 28) while every rendered frame was recorded with the teacher's action: 169,139 labeled frames. A fresh CNN trained on them for 10 epochs (Adam, learning rate 1e-3, batch 128). It already finished 30/30 validation courses, but lost 4 lives.
DAgger, one round. The student CNN drove 30 new 28-turn courses itself (95% of the time), while the teacher labeled every frame it visited. That added 123,338 frames, mostly the student's own mistakes and near misses. Fine-tuning on all 292,477 frames for 5 epochs (learning rate 1e-4) removed the remaining deaths.
Model selection used rollout results (completion, then lives, then score), never training accuracy. The training, validation, and final-test courses come from disjoint seed ranges.
Limitations
- The model was trained and tested only on the Club Pyguin recreation's renderer. It has not been tested on the original Flash game (for example through Ruffle), where small rendering differences could hurt it.
- Behavior is deterministic. The same course and seed always give the same run.
- It plays only Cart Surfer. It does not navigate menus: the run starts already playing.
Files
| File | Purpose |
|---|---|
model.safetensors |
Network weights (SHA-256 c0b0d3dffaee11f63235c89a2f111cf418f8d4f9ffe64df3278a0237c7714175) |
play.py |
Self-contained model definition, preprocessing, agent, and player |
club_pyguin/ |
The Cart Surfer recreation (engine, renderer, random courses, asset extractor) |
pyproject.toml |
Installs club_pyguin and its dependencies with pip install ., and adds the cart-surfer and cart-surfer-extract commands |
README.md |
This card |
Disclaimer
This is an unofficial fan project for learning about reinforcement and imitation learning. It is not affiliated with or endorsed by Disney or Club Penguin. Club Penguin and Cart Surfer are trademarks of their owners. This repository contains no original game files, graphics, or sounds. The bundled code is an independent Python recreation, and it needs assets that you extract from your own copy of the game.
