| --- |
| license: mit |
| library_name: tensorflow |
| tags: |
| - othello |
| - reversi |
| - game-ai |
| - transformer |
| - mixture-of-experts |
| --- |
| |
| ## Transformer Reversi AI |
| This Reversi AI is a Transformer-based model that uses Mixture-of-Experts architecture for decision-making instead of CNNs. |
| [Github - rsu-Suba/ReversiGPT](https://github.com/rsu-Suba/ReversiGPT) |
|
|
| ## Architecture |
|
|
| - Parameters: 1.68M |
| - Model size: 20.6MB |
| - Input: `(8, 8, 2)` |
| - Own stones |
| - Opponent stones |
| - Board representation: 64 tokens |
| - Embedding dimension: 128 |
| - Transformer blocks: 4 |
| - Attention heads: 4 |
| - MHA experts: 2 |
| - FFN experts: 2 |
| - Routing: softmax-based dynamic routing |
| - Iterative routing steps: 2 |
| - Normalization: Pre-LN LayerNorm |
| - FFN activation: GELU |
|
|
| ### Diagram |
| ``` |
| Input (8×8×2) |
| ↓ |
| 64 Tokens |
| ↓ |
| Dense 128 |
| ↓ |
| Token + Row + Column + Move Embeddings |
| ↓ |
| DynamicAssembly × 4 |
| ├─ MHA Expert × 2 |
| └─ FFN Expert × 2 |
| ↓ |
| ├─ Policy Head → 64 moves |
| └─ Value Head → Win-rate |
| ``` |
|
|
| ### DynamicAssembly |
|
|
| Each DynamicAssembly block contains a shared pool of MHA and FFN |
| experts. At each iteration step, the mean-pooled board representation |
| is combined with a step embedding and passed to a router. |
|
|
| The router produces a probability distribution over the expert pool. |
| All experts are evaluated and their outputs are combined using the |
| router probabilities. |
|
|
| ### Policy Head |
|
|
| The policy head predicts a probability distribution over the 64 |
| board positions. |
|
|
| ### Value Head |
|
|
| The value head predicts a scalar value in `[-1, 1]`, representing |
| the estimated game outcome. |
|
|
| ## Training |
|
|
| This model was trained using self-play game records generated by a previous CNN-based Othello AI. |
|
|
| ### Self-Play data generation |
| - MCTS simulations: 600 |
| - MCTS batch size: 100 |
| - PUCT constant: 2.3 |
| - Total games: 10,000 |
|
|
| ### Model Training |
| - Optimizer: AdamW |
| - Clipnorm: 1.0 |
| - Weight decay: 0.05 |
| - Learning rate: 6.0e-5 |
| - Learning rate scheduler: ReduceLROnPlateau |
| - Training batch size: 256 |
| - Epochs: 75 |
|
|
| ## Evaluation |
|
|
| A 63M-parameter ResNet CNN model was used as the previous model. |
|
|
| ### Win rate |
| - vs Random : 100% (100 / 100 games) |
| - vs Prev model : 63% (63 / 100 games) |
|
|
| ### Average final stone count |
| - vs Random : 58 |
| - vs Prev model : 47 |
|
|
| ## Usage / Quick start |
| ```python |
| import numpy as np |
| import tensorflow as tf |
| from huggingface_hub import hf_hub_download |
| from model import TokenAndPositionEmbedding, MHA, FFN, DynamicAssembly |
| |
| model_path = hf_hub_download(repo_id="rsu/Reversi-Transformer-1", filename="M1.h5", repo_type="model") |
| |
| custom_objects = { |
| "TokenAndPositionEmbedding": TokenAndPositionEmbedding, |
| "MHA": MHA, |
| "FFN": FFN, |
| "DynamicAssembly": DynamicAssembly, |
| } |
| model = tf.keras.models.load_model(model_path, custom_objects=custom_objects, compile=False) |
| |
| board = np.zeros((1, 8, 8, 2), dtype=np.float32) |
| board[0, 3, 3, 1] = 1.0 |
| board[0, 3, 4, 1] = 1.0 |
| board[0, 3, 3, 1] = 1.0 |
| board[0, 4, 4, 1] = 1.0 |
| |
| policy, value = model(board, training=False) |
| print("Policy (64 moves prob): ", policy.numpy()) |
| print("Value (win rate est [-1, 1]): ", value.numpy().item()) |
| ``` |
