avewright/chess-transformer-100m-squares64

Chess transformer 100M (squares64 recurrent)

~99M parameter recurrent chess policy/value net. Attention is only over the 64 board squares. Turn, castling, and en passant are FiLM on the square stream.

This file is latest.pt at FT step 4000 (2026-09-08). Polar-NorMuon continuation from run1 step 1500. Train loss ~3.13.

Playing strength vs the original public weights is not established. A search-free greedy head-to-head (8 openings × both colors, no book / no Syzygy) had the original init.pt beating run1 step 1500 (9.0–7.0) and SWA (9.5–6.5). This checkpoint has not been re-laddered under that protocol. Do not treat the older book+tablebase ~2050 number as search-free Elo.

Architecture

Params	98.97M
Hidden / heads	736d / 8
Encoder dim	256
Trunk	prefix 4 + bank 7×3 unrolls + suffix 4
Effective depth	29 (15 unique layer modules)
Attention	64×64 squares only
Vocab	1968 compact
Heads	spatial policy + 3-way WDL

Config is in model_config.json (Squares64RecurrentConfig).

This run (sf19_ft/run2)

Weights-only warm start from run1 step_001500.pt, then 4000 Polar-NorMuon steps (muon_lr=0.002, Adam aux 3e-5, warmup 100, batch 64). Mix:

Stream	Fraction	Data
SF19	75%	avewright/chess-soft-sf19 train + harvest expand (~1.84M unique after blocking)
Lichess MultiPV	20%	avewright/chess-soft-multipv-lichess 1.5M phase-balanced, depth ≥ 12
Syzygy	5%	avewright/chess-soft-syzygy 400k

Frozen SF19 eval (saved_split_v1, 9,992 positions, flip-equivalents blocked) was not rewritten. Final holdout:

• val/soft hard_ce=1.751 teacher_kl=0.657
• val/deep hard_ce=1.387 teacher_kl=0.322

Inference

Requires this repo (chess_inference.py, chess_squares64.py) and compact vocab 1968.

export MOVE_VOCAB_VERSION=compact

import os
os.environ["MOVE_VOCAB_VERSION"] = "compact"

import chess
from huggingface_hub import hf_hub_download
from chess_inference import load_checkpoint, get_model_move

path = hf_hub_download("avewright/chess-transformer-100m-squares64", "latest.pt")
model = load_checkpoint(path, device="cpu")
move, info = get_model_move(model, chess.Board(), "cpu")
print(move, info["top_moves"], info["wdl"])

Files

• latest.pt — full train checkpoint at step 4000 (weights + optimizer)
• model_config.json — architecture
• run_config.json — mix / LR used for this run
• train.log — step/loss/val log