cartmind-1.0-223K

cartmind is a 223,190-parameter convolutional network that plays Cart Surfer, the mine-cart minigame from Club Penguin, using only the pixels on screen. It finishes randomly generated courses of any length without losing a life, grinds every corner, and keeps up an alternating backflip/spin trick rhythm the whole way down.

The game runs in Club Pyguin, a faithful Python recreation of the original Flash minigame. The model sees only the last four frames, downscaled to 84Γ—84 grayscale. It has no access to the game's internal state.

Watch the video: how this AI learned to play Cart Surfer

▢️ Watch the video on YouTube

Results

All numbers come from deterministic greedy play. A "course" is a randomly generated track with 7, 14, or 28 turns (the classic track has 7).

Evaluation Courses finished Lives lost Turns cleared Corners ground Mean score (7 / 14 / 28 turns)
Validation (seeds 1,051,000+) 30 / 30 0 490 / 490 490 / 490 3,694 / 7,184 / 13,266
Untouched final test (seeds 2,060,000+) 150 / 150 0 2,450 / 2,450 2,450 / 2,450 3,595 / 6,851 / 12,781
Classic track 20 / 20 0 140 / 140 140 / 140 3,280
  • The final-test courses were never used for training, validation, or model selection. The test ran once, after the weights were frozen.
  • Over the 150 final-test courses, the model performed 6,350 backflips and 6,604 spins (about 86 tricks per course). 95% of them landed for points.
  • The model outscores its own teacher on the classic track (3,280 vs. 2,853).

Quick start

You need Python 3.11 or newer.

pip install -U huggingface_hub    # provides the `hf` command
hf download benank/cartmind-1.0-223K --local-dir cartmind-1.0-223K
cd cartmind-1.0-223K
python -m venv .venv && source .venv/bin/activate
pip install torch safetensors .
python play.py --check

pip install . installs the bundled Cart Surfer recreation (see Watch it play). --check loads the weights and runs one decision on a dummy frame. It needs no game assets.

Use the model in your own loop

play.py is one self-contained file. Import CartSurferAgent and give it one game frame per tick, either a pygame Surface or an H Γ— W Γ— 3 uint8 RGB array. It returns an action index.

from play import ACTIONS, CartSurferAgent

agent = CartSurferAgent()                 # loads model.safetensors
action = agent.reset(first_frame)         # first frame of a run

# Pseudocode: `game` is your Cart Surfer loop.
while running:
    frame = game.step(action)             # advance one tick (24 per second)
    action = agent.step(frame)
    print(ACTIONS[action])                # nothing, left, right, up, down, space

play() in play.py is a complete working example that uses the bundled game.

Watch it play

The club_pyguin/ folder holds the Cart Surfer part of Club Pyguin, a Python recreation of the original Flash minigame. It includes the game engine, renderer, random-course generator, and asset extractor. The model was trained and evaluated in this exact code.

This repository contains no game graphics or sounds. You extract them once from your own copy of the original 2006 game file.

1. Get the original game file

You need CartSurfer2006.swf, the original version of the minigame from before stamps were added:

  • size: 77,163 bytes;
  • SHA-256: fd30e04c8de51fbfd970841fac9e761d0f9c41481f77b3bee74524a42fe188f9.

The extractor rejects any other file, because other versions draw and play differently. Check yours with shasum -a 256 CartSurfer2006.swf.

2. Get JPEXS FFDec (the Flash decompiler)

Install a Java runtime (java -version should work). Download and unzip the latest JPEXS release from github.com/jindrapetrik/jpexs-decompiler/releases. You'll point the extractor at ffdec-cli.jar inside the unzipped folder. Keep it there, because it needs the lib/ folder next to it.

3. Extract the assets

cart-surfer-extract extract \
  --source /path/to/CartSurfer2006.swf \
  --ffdec /path/to/ffdec/ffdec-cli.jar \
  --output cart-surfer-assets

This takes a few seconds. It writes the sprites, shapes, sounds, and a manifest.json to cart-surfer-assets/.

4. Watch the model play

python play.py --assets cart-surfer-assets                # classic track, in a window
python play.py --assets cart-surfer-assets --turns 28     # random 28-turn course
python play.py --assets cart-surfer-assets --turns 14 --seed 7 --headless

The window stays on the final frame when the run ends. Press Esc to close it. --headless plays without a window and prints the result. Each --turns/--seed pair is a different, reproducible course.

5. Play it yourself

The same install gives you the full recreation, with the title screen, instructions, sounds, and game over:

cart-surfer --assets cart-surfer-assets
Key Action
Enter / Space Start, and advance the instructions (press I on the title for instructions)
← / β†’ Lean, or turn through a corner
↓ then ← / β†’ Wheelie, then grind around a corner (the move the model uses on every corner)
↑ then ← / β†’ Surf stance, then slide
Space Jump. From a wheelie: backflip. From surf stance: ollie. Also recovers after a crash
P / D / Esc Pause / debug overlay / quit

Model

Input 4 stacked 84Γ—84 grayscale frames (uint8), scaled to [0, 1] inside the model
Architecture Conv 8Γ—8/4 (16) β†’ Conv 4Γ—4/2 (32) β†’ Conv 3Γ—3/1 (32) β†’ Linear 1568β†’128 β†’ Linear 128β†’6, ReLU between layers
Output Logits for 6 actions: nothing, left, right, up, down, space. The agent takes the argmax
Parameters 223,190 (894 KB as float32 safetensors)
Decision rate One action per game tick, 24 per second

Preprocessing must match training exactly. play.py resizes each 760Γ—480 frame with pygame's smoothscale, then converts it to grayscale as (77R + 150G + 29B) >> 8. At the start of a run, the first frame fills all four stack slots.

The agent adds one small rule on top of the network. Crash recovery needs a fresh Space key press, so if the model outputs Space three ticks in a row, the third tick becomes "release all keys". The rule uses only the agent's own past outputs, never game state.

How it was made

Training ran in three stages, about 40 minutes in total on one Apple M3 Pro laptop.

  1. State-based teacher (Double DQN). A small Q-network read the game's 17-value internal state and learned through a curriculum:

    • safe completion with score reward off (1,200 episodes);
    • grind discovery with shaped bonuses for grinding corners (Down + direction) (600 episodes);
    • stabilization with score reward restored and smaller grind bonuses (750 episodes).

    Snapshots were saved every 25 episodes and screened on 30 held-out courses. The chosen teacher finished 150/150 courses with zero lives lost and ground all 2,450 corners.

  2. Behavior cloning. The teacher played 80 random courses (40 at 7 turns, 20 at 14, 20 at 28) while every rendered frame was recorded with the teacher's action: 169,139 labeled frames. A fresh CNN trained on them for 10 epochs (Adam, learning rate 1e-3, batch 128). It already finished 30/30 validation courses, but lost 4 lives.

  3. DAgger, one round. The student CNN drove 30 new 28-turn courses itself (95% of the time), while the teacher labeled every frame it visited. That added 123,338 frames, mostly the student's own mistakes and near misses. Fine-tuning on all 292,477 frames for 5 epochs (learning rate 1e-4) removed the remaining deaths.

Model selection used rollout results (completion, then lives, then score), never training accuracy. The training, validation, and final-test courses come from disjoint seed ranges.

Limitations

  • The model was trained and tested only on the Club Pyguin recreation's renderer. It has not been tested on the original Flash game (for example through Ruffle), where small rendering differences could hurt it.
  • Behavior is deterministic. The same course and seed always give the same run.
  • It plays only Cart Surfer. It does not navigate menus: the run starts already playing.

Files

File Purpose
model.safetensors Network weights (SHA-256 c0b0d3dffaee11f63235c89a2f111cf418f8d4f9ffe64df3278a0237c7714175)
play.py Self-contained model definition, preprocessing, agent, and player
club_pyguin/ The Cart Surfer recreation (engine, renderer, random courses, asset extractor)
pyproject.toml Installs club_pyguin and its dependencies with pip install ., and adds the cart-surfer and cart-surfer-extract commands
README.md This card

Disclaimer

This is an unofficial fan project for learning about reinforcement and imitation learning. It is not affiliated with or endorsed by Disney or Club Penguin. Club Penguin and Cart Surfer are trademarks of their owners. This repository contains no original game files, graphics, or sounds. The bundled code is an independent Python recreation, and it needs assets that you extract from your own copy of the game.

Downloads last month

-

Downloads are not tracked for this model. How to track
Safetensors
Model size
223k params
Tensor type
F32
Β·
Inference Providers NEW
This model isn't deployed by any Inference Provider. πŸ™‹ Ask for provider support