---
license: mit
library_name: tensorflow
tags:
  - othello
  - reversi
  - game-ai
  - transformer
  - mixture-of-experts
---

## Transformer Reversi AI
This Reversi AI is a Transformer-based model that uses Mixture-of-Experts architecture for decision-making instead of CNNs.  
[Github - rsu-Suba/ReversiGPT](https://github.com/rsu-Suba/ReversiGPT)

## Architecture

- Parameters: 1.68M
- Model size: 20.6MB
- Input: `(8, 8, 2)`
  - Own stones
  - Opponent stones
- Board representation: 64 tokens
- Embedding dimension: 128
- Transformer blocks: 4
- Attention heads: 4
- MHA experts: 2
- FFN experts: 2
- Routing: softmax-based dynamic routing
- Iterative routing steps: 2
- Normalization: Pre-LN LayerNorm
- FFN activation: GELU

### Diagram
```
Input (8×8×2)
    ↓
64 Tokens
    ↓
Dense 128
    ↓
Token + Row + Column + Move Embeddings
    ↓
DynamicAssembly × 4
  ├─ MHA Expert × 2
  └─ FFN Expert × 2
    ↓
    ├─ Policy Head → 64 moves
    └─ Value Head → Win-rate
```

### DynamicAssembly

Each DynamicAssembly block contains a shared pool of MHA and FFN
experts. At each iteration step, the mean-pooled board representation
is combined with a step embedding and passed to a router.

The router produces a probability distribution over the expert pool.
All experts are evaluated and their outputs are combined using the
router probabilities.

### Policy Head

The policy head predicts a probability distribution over the 64
board positions.

### Value Head

The value head predicts a scalar value in `[-1, 1]`, representing
the estimated game outcome.

## Training

This model was trained using self-play game records generated by a previous CNN-based Othello AI.

### Self-Play data generation
- MCTS simulations: 600
- MCTS batch size: 100
- PUCT constant: 2.3
- Total games: 10,000

### Model Training
- Optimizer: AdamW
- Clipnorm: 1.0
- Weight decay: 0.05
- Learning rate: 6.0e-5
- Learning rate scheduler: ReduceLROnPlateau
- Training batch size: 256
- Epochs: 75

## Evaluation

A 63M-parameter ResNet CNN model was used as the previous model.

### Win rate
- vs Random : 100% (100 / 100 games)
- vs Prev model : 63% (63 / 100 games)

### Average final stone count
- vs Random : 58
- vs Prev model : 47

## Usage / Quick start
```python
import numpy as np
import tensorflow as tf 
from huggingface_hub import hf_hub_download
from model import TokenAndPositionEmbedding, MHA, FFN, DynamicAssembly

model_path = hf_hub_download(repo_id="rsu/Reversi-Transformer-1", filename="M1.h5", repo_type="model")

custom_objects = {
  "TokenAndPositionEmbedding": TokenAndPositionEmbedding,
  "MHA": MHA,
  "FFN": FFN,
  "DynamicAssembly": DynamicAssembly,
}
model = tf.keras.models.load_model(model_path, custom_objects=custom_objects, compile=False)

board = np.zeros((1, 8, 8, 2), dtype=np.float32)
board[0, 3, 3, 1] = 1.0
board[0, 3, 4, 1] = 1.0
board[0, 3, 3, 1] = 1.0
board[0, 4, 4, 1] = 1.0

policy, value = model(board, training=False)
print("Policy (64 moves prob): ", policy.numpy())
print("Value (win rate est [-1, 1]): ", value.numpy().item())
```