| --- |
| license: apache-2.0 |
| library_name: transformers |
| pipeline_tag: text-generation |
| tags: |
| - chess |
| - reinforcement-learning |
| - self-play |
| - qwen3 |
| - causal-lm |
| - pytorch |
| --- |
| |
| # Plynder-1 |
|
|
| Plynder-1 is a randomly initialized 50M-parameter Qwen3 architecture trained |
| to play chess through reinforcement learning and self-play. |
| It does not use pretrained Qwen language-model weights and was not trained by imitating a PGN corpus. |
|
|
| Plynder represents a game as UCI move tokens. Its public vocabulary has 1,972 |
| entries: 1,968 possible moves and four game-state tokens: `<bos>`, `<draw>`, |
| `<white_win>`, and `<black_win>`. The causal model produces logits for the full |
| vocabulary; legal-move filtering is required before selecting an action. |
|
|
| > [!Note] |
| > Plynder-1 is a chess policy model. It |
| > predicts moves in Plynder's custom token space. Select actions from the legal |
| > moves in the current position as shown in the inference example. |
|
|
|
|
| ## Model overview |
|
|
| | Property | Value | |
| |---|---| |
| | Architecture | `Qwen3ForCausalLM` | |
| | Parameters | ~50M | |
| | Hidden size | 512 | |
| | Transformer layers | 16 | |
| | Attention heads | 8 query heads, 4 key-value heads | |
| | Head dimension | 64 | |
| | Feed-forward dimension | 1,534 | |
| | Vocabulary | 1,972 chess and game-state tokens | |
| | Maximum sequence length | 8,000 tokens | |
| | Training precision | bfloat16 | |
|
|
| The policy model uses full attention in all transformer layers. |
|
|
| ## Training recipe |
|
|
| | Setting | Value | |
| |---|---:| |
| | Steps | 265k | |
| | Global batch size | 8192 | |
| | Optimizer | Muon and AdamW | |
| | Learning rate | `7e-4` | |
| | Rollout group size | 16 continuations | |
| | Policy objective | CISPO | |
| | CISPO lower and upper bounds | `0.1` and `0.2` | |
| | CISPO importance-sampling mask | Enabled | |
| | Rules objective | KL distillation from legal-move masks | |
|
|
| During training, a separate linear rules head reads the intermediate hidden state at layer 6 and |
| predicts the legal-move mask. The rules head is an auxiliary training component; |
| standard Transformers inference uses the published causal language model. |
| For each opening position, rollout generates a group of continuations. Their |
| outcomes provide group-relative advantages, separately for White and Black. The |
| policy is updated with CISPO, while the rules head receives a dense legal-move |
| signal from the Rust chess engine. |
|
|
| ## Training statistics |
|
|
|
|
| | Statistic | Value | |
| |---|---:| |
| | Tokens processed by trainer | **245.3B** padded token slots | |
| | Non-padding trainer tokens | **232.7B** | |
| | Tokens generated by rollout | **134.7B** kept-group tokens | |
| | Games generated | **1.62B** sampled continuations (**1.27B** in kept groups) | |
| | Average padding ratio | **5.16%** | |
|
|
| The rollout-token total counts the full token sequence for every member of a |
| kept 16-game group, including the shared opening prefix each time. Groups whose |
| continuations all have the same outcome are discarded. The |
| game total includes every sampled continuation; the kept-group figure excludes |
| discarded groups. |
|
|
| ### Training progress |
|
|
| <p align="center"> |
| <img |
| src="https://huggingface.co/aLocks/plynder-1/resolve/main/plynder_1_training_progress.png" |
| alt="Plynder-1 training progress" |
| width="860" |
| > |
| </p> |
|
|
| The plot shows the smoothed internal strength trajectory. Its horizontal axis is |
| kept rollout tokens, with the shared prefix counted for every continuation. |
|
|
| ### Data staleness |
|
|
| The estimated mean rollout-data staleness is **5.6 optimizer steps**: |
|
|
| - **0.6 steps**: measured p50 trajectory/group latency |
| - **1.0 step**: expected delay from the 16,384-trajectory sampler buffer |
| - **4.0 steps**: expected delay from refreshing rollout weights every 8 optimizer steps |
|
|
| The latter two values use half the corresponding buffering/refresh interval as |
| the expected delay. |
|
|
| ## Evaluation |
|
|
| Plynder-1 measured an internal ranking of **1,433**. The ranking is an Elo-like project metric |
| calibrated against Stockfish with 4,096-game evaluations. It is not a Lichess |
| rating. |
|
|
| A deployment of the trained model on Lichess reached approximately **1,550 Elo** |
| in the Bullet time control. |
|
|
| ## Usage |
|
|
| Install the inference dependencies: |
|
|
| ```bash |
| pip install torch transformers chess |
| ``` |
|
|
| The following example loads the tokenizer from the model repository and chooses |
| the highest-scoring legal UCI move: |
|
|
| ```python |
| import chess |
| import torch |
| from transformers import AutoModelForCausalLM, AutoTokenizer |
| |
| CASTLING_MAP = {"e1g1": "e1h1", "e1c1": "e1a1", "e8g8": "e8h8", "e8c8": "e8a8"} |
| |
| def to_plynder_uci(move: chess.Move, board: chess.Board) -> str: |
| """Convert a python-chess move to Plynder's token notation.""" |
| uci = move.uci() |
| return CASTLING_MAP.get(uci, uci) if board.is_castling(move) else uci |
| |
| |
| model_name = "aLocks/plynder-1" |
| |
| tokenizer = AutoTokenizer.from_pretrained(model_name) |
| model = AutoModelForCausalLM.from_pretrained(model_name) |
| |
| prompt = "<bos> e2e3 f7f6 b1c3 g7g5" |
| |
| # Get legal moves |
| board = chess.Board() |
| for uci in prompt.split(" ")[1:]: |
| board.push_uci(uci) |
| |
| legal = list(board.legal_moves) |
| legal_ids = torch.tensor( |
| tokenizer.convert_tokens_to_ids([to_plynder_uci(move, board) for move in legal]) |
| ) |
| |
| # Infer policy |
| inputs = tokenizer(prompt, return_tensors="pt") |
| with torch.inference_mode(): |
| logits = model(**inputs).logits[0, -1] |
| |
| played_uci = legal[logits.index_select(0, legal_ids).argmax().item()].uci() |
| |
| print("played:", played_uci) |
| ``` |
|
|
| Plynder uses UCI-like move tokens. For castling, its vocabulary uses the |
| king-to-rook forms `e1h1` or `e1a1` for White and `e8h8` or `e8a8` for Black. |
|
|
| ## Limitations |
|
|
| - Plynder-1 is train to continue games with at least 2 moves (4 plies). To get variability, use an opening book |
| - Legal filtering is part of the inference procedure. The raw language-model |
| logits also contain illegal moves and game-state tokens. |
| - A board created directly from a FEN has no prior move history. Provide the |
| history when it is available. |
|
|
|
|
|
|
| ## Source and license |
|
|
| Plynder-1 is released under the Apache License 2.0. The training system, Rust |
| legal-move engine, and configuration are available in the |
| [source repository](https://github.com/AntoineLorentz/plynder). |
|
|
| ## Citation |
|
|
| ``` |
| @software{plynder, |
| author = {AntoineLorentz}, |
| title = {Plynder}, |
| url = {https://github.com/AntoineLorentz/plynder}, |
| license = {Apache-2.0} |
| } |
| ``` |
|
|
