laya-2048

convaiinnovations/laya fine-tuned to play the game 2048. It reads the board as text and picks one of four moves (up / down / left / right) in a single forward pass.

• Code, training scripts and the browser player: https://github.com/ctfpanda17/laya-2048
• Speedrun: reaches the 2048 tile in ~11 seconds (~950 moves, ~12 ms per move on an RTX 4090 with CUDA graphs)

Results

64 self-played games in a simulator (same random seeds for every row):

Model	Avg. score	Reached 2048	Reached 4096
Original laya (zero-shot)	~1,000	0/64	0/64
After imitation learning	15,000	17/64	1/64
After 5 DAgger rounds (this model)	~45,000–48,000	51–55/64	33/64

The expectimax teacher it learned from averages ~108,000 points, so there is still a gap.

How it was trained

1. Imitation learning: an expectimax search played 200 games; laya was fine-tuned (cross-entropy on the four option markers) to pick the same move on ~90k boards.
2. DAgger, 5 rounds: laya plays 64 games by itself, a stronger expectimax teacher labels every board laya visited, the data is aggregated (~590k boards in total) and laya is fine-tuned again. Agreement with the teacher went from 61% to 85%.

Full ModernBERT-large backbone fine-tuned (421M parameters), AdamW, lr 1e-5–2e-5, bf16 autocast, RTX 4090. Weights are stored in float16 (~840 MB); laya.load upcasts them to float32.

Usage

The board must be formatted exactly like this, and the question must be the same, or the model will not recognise the task.

import laya
from huggingface_hub import snapshot_download

# local_dir avoids the Hugging Face cache symlinks, which fail on Windows without developer mode
path = snapshot_download("linpanda/laya-2048", local_dir="laya-2048")
agent = laya.load(path, device="cuda")  # or "cpu"

QUESTIONS = {
    "move": {
        "type": "choice",
        "instructions": (
            "You are playing 2048. Tiles with equal numbers merge when they slide into each other. "
            "Keep the largest tile in a corner and keep empty cells. Which move is best?"
        ),
        "criteria": {
            "up": "slide all tiles up",
            "down": "slide all tiles down",
            "left": "slide all tiles left",
            "right": "slide all tiles right",
        },
    }
}

def laya_state(grid):
    """grid: 4x4 list of ints, 0 = empty cell."""
    board = "\n".join(f"row{i + 1}: " + " ".join(str(v) if v else "." for v in row)
                      for i, row in enumerate(grid))
    return {"game": "2048 puzzle, 4x4 board, '.' is empty", "board": board}

grid = [[1024, 512, 256, 64],
        [32,   64,  128, 32],
        [16,   32,  16,  8],
        [4,    8,   2,   0]]
probs = agent.predict(laya_state(grid), QUESTIONS)["answers"]["move"]["probabilities"]
print(probs)  # pick the most probable move that actually changes the board

The model sometimes ranks a move that cannot change the board first, so in practice choose the most probable legal move. The GitHub repo has a browser player (play_2048.py) and the speedrun script (speedrun.py).

Limitations

• Trained only on the standard 4x4 board with the text format above.
• Plays clearly weaker than its expectimax teacher; it imitates rather than searches.
• Not eligible for speedrun.com's 2048 leaderboards, which are for human players only.

License

Apache-2.0, the same as the base model convaiinnovations/laya.