---
license: apache-2.0
library_name: transformers
pipeline_tag: text-generation
tags:
- chess
- gpt_neox
- pythia
---

# Checkmate-5M

A tiny chess move model built for the [Chess LLM Arena](https://huggingface.co/spaces/mlabonne/chessllm).

- **Architecture:** Pythia / GPT-NeoX (6 layers, hidden size 128, 4 heads), trained from scratch.
- **Parameters:** 4.8M
- **Tokenizer:** move-level chess tokenizer: one token per SAN move (without `+`/`#`), plus the
  move-number prompt `1.` and single SAN characters. Every legal SAN move is a single token.
- **Training:** the move-selection policy was optimized with reinforcement learning (policy
  gradient) on millions of simulated games against the models at the top of the arena
  leaderboard, then distilled into the network.
- **Best used as White.**

## Usage in the arena

Type `DedeProGames/Checkmate-5M` as the **White** model in the arena and press *Fight!*.

## Usage with transformers + outlines

```python
import chess, re
import outlines.models as models
from outlines import generate

model = models.transformers("DedeProGames/Checkmate-5M")
board = chess.Board()
legal = "|".join(re.escape(re.sub(r"[+#]", "", board.san(m))) for m in board.legal_moves)
print(generate.regex(model, legal)("1."))
```