Hugging Face's logo

Kedhareswer
/
depth-5

numpy
chess
nnue
game-playing
alpha-beta
trained-from-scratch
cpu
Model card Files
xet
 Community
depth-5 / README.md
Kedhareswer's picture
Kedhareswer
UCI engine: python -m chess_nnue.uci
255fab2 verified
preview code
|
Raw History Blame Contribute Delete
12.5 kB
metadata 
license: mit
library_name: numpy
tags:
  - chess
  - nnue
  - game-playing
  - alpha-beta
  - trained-from-scratch
  - cpu
datasets:
  - Lichess/chess-position-evaluations

depth 5: a 4.8 MB neural chess engine

4.8 MB weights, CPU only, plain numpy. On the ChessBench LLM scale it rates 2446 (95%: 2288-2666), ahead of every LLM on that leaderboard (best: Gemini 3.1 Pro, 2144; the intervals overlap). It solves 96.6% of Kagi's 1,000 LLM chess puzzles (best LLM there: GPT-4.5, 84.8%) and plays at about 2110 against Stockfish's calibrated strength levels.

The name is a name, not a setting: the search is limited by time, not depth. At the benchmark time control (60 s + 0.6 s per move) it reached depth 5.2 to 5.8 on average.

Results

Every game is saved in evidence/chess_nnue/games_*.jsonl and every summary in games_*.json. Openings: 4-8 random plies that Stockfish scores within 100 cp, each played twice with colours swapped; games are drawn at 200 plies.

Against LLMs, live: LLM Chess Arena

depth 5 now plays LLMs directly in LLM Chess Arena as the built-in bot local/neural, rated on the arena's live leaderboard next to Claude, GPT, Gemini and Llama models. It joined on 2026-11-01 with no rated games yet, so its arena rating is still the 1500 starting value; the benchmarks below are the measured numbers until games accumulate there.

Against LLMs: the ChessBench LLM scale

ChessBench LLM rates LLMs (on the Lichess Classical scale) from games against fixed anchors: a random mover (400), Maia 1100 (1628) and Maia 1900 (1816), Maia running in lc0 at 1 node per move. We played the same anchors with the same settings, so our rating sits on the same scale as the LLMs. Leaderboard snapshot: evidence/chess_nnue/chessbench_leaderboard_2026-11-01.json.

ChessBench scale

Opponent Games Won Lost Drawn
random mover (400) 100 100 0 0
Maia 1100 (1628) 100 97 0 3
Maia 1900 (1816) 100 96 0 4
Rating (maximum likelihood, 95% interval) 300 2446 (2288-2666)

Honest reading: depth 5 never lost to an anchor, so the rating rests on 7 draws against opponents far below it. It is an extrapolation and the interval is wide; it overlaps the top LLM's interval (Gemini 3.1 Pro high, 1963-2325). It is clearly above GPT-6 Astra (1975), Grok, DeepSeek, o3 and every Claude model on the board. We did not play ChessBench's strongest anchor (Eubos, 2357).

Against LLMs: Kagi llm-chess-puzzles

Kagi's benchmark gives LLMs the same 1,000 Lichess puzzles and checks the first move (exact match). Same file, same rule, 1 s per move. Kagi's table (2025-04-26): evidence/chess_nnue/kagi_llm_chess_puzzles_2025-04-26.csv.

Kagi puzzles

Player Solved Adjusted rating (Glicko-2)
depth 5 96.6% (966) 2221
gpt-4.5-preview 84.8% 1866
gpt-4.1 57.9% 1345
gpt-4o 53.6% 1321
claude-3-opus 7.2% 521

Kagi's table is from April 2025 and has no newer models.

Against Stockfish's calibrated levels

Stockfish 19 with UCI_LimitStrength (its UCI_Elo is calibrated to CCRL 40/4 at 60 s + 0.6 s), same time control for both sides.

Opponent Games Won Lost Drawn Score
Stockfish UCI_Elo 1800 50 32 14 4 68%
Stockfish UCI_Elo 2100 50 21 19 10 52%
Stockfish UCI_Elo 2400 50 7 24 19 33%
Rating (95% interval) 150 2110 (2044-2178)

The two scales differ (Lichess Classical vs CCRL), so 2446 and 2110 are not comparable numbers.

Lichess puzzles (whole solution line must match)

Set Puzzles depth 5 Puzzle Arena chess model
Held-out Lichess puzzles 3,000 90.4% 58.7%
Lichess daily puzzles 297 90.2% 30.0%

By puzzle rating (held-out set): under 1500: 98.8%, 1500-2000: 90.4%, 2000+: 62.6%.

Ablation: what the network adds (and a regression)

Against LLM Chess Arena's local/engine-4 (run unchanged from that repo: depth 4, material + piece-square tables, 3 s/move), both sides 3 s/move.

Ablation

Evaluation + search Games W / L / D Elo vs engine-4 (95%) Nodes/s Avg depth
old eval + old search 50 12 / 16 / 22 -28 (-102 to +44) 40k 4.7
NN eval + old search 50 24 / 21 / 5 +21 (-71 to +116) 13k 3.6
old eval + new search 50 34 / 6 / 10 +220 (+134 to +341) 28k 8.2
depth 5: NN eval + new search 100 55 / 38 / 7 +60 (-6 to +129) 9k 5.2

Head to head, depth 5 vs old eval + new search (60 s + 0.6 s): 15 wins, 18 losses, 4 draws after 37 games (Elo -28, 95%: -141 to +78; the run was stopped before its planned 60 games). No clear winner at this time control.

The regression, plainly: at 3 s/move in Python, the network costs more than it gives. Scoring a position with the network is about 2-3x slower than with the piece-square table, so the same search reaches depth 5 instead of 8. The network is a better judge per position (the "old search" rows show it), but here depth wins. The fix is speed, not a different network: a compiled (Rust/WASM) build of the same weights searches 50x more positions per second. (These games ran while other benchmarks shared the CPU, which lowered nodes/s for both sides.)

The model

An NNUE ("efficiently updatable neural network") evaluator, the design modern chess engines use, trained from scratch here. Search finds the move; the network scores the positions search reaches.

Part What it is
Inputs 768 piece-square features (6 piece types x 2 colours x 64 squares) per perspective, in 12 buckets keyed on that side's own king square (mirrored so the king is always on files a-d). 9,216 inputs per side.
Feature transformer 9,216 -> 256 per perspective, int16 weights (scale 255). Two perspectives (side to move first) -> 512. Activation: clipped and squared (SCReLU).
Layers 512 -> 32 -> 32 -> 1, float32, clipped ReLU. Output x 400 = centipawns for the side to move.
Parameters 2,377,057 (feature transformer 2,359,552)
File chess.nnue, 4,789,188 bytes: header with magic, version, shapes and a SHA-256 of the payload; the loader rejects any mismatch
Inference numpy, no compiled code. An accumulator per perspective is updated incrementally on every move (add/remove the moved, captured and promoted pieces' rows); only a king move refreshes that side. Exact: float32 holds these integer sums exactly, and a test checks incremental == full refresh.
Search python-chess move generation; iterative deepening, aspiration windows, PVS, transposition table, killer + history ordering, null move, late move reductions, check extension, quiescence with MVV-LVA, delta pruning and losing-capture pruning. Stops at a deadline and plays the best move of the last finished depth, or a move from the unfinished depth only if it beat that depth's window.
Speed 13-20k positions/s on one core of a desktop CPU; RAM about 100-150 MB per engine (weights 9.4 MB in memory).

Data and licence

Training labels come from the Lichess evaluation database (lichess_db_eval.jsonl.zst, CC0; also on Hugging Face as Lichess/chess-position-evaluations): positions analysed by Stockfish in Lichess's cloud, with the deepest evaluation's best line. chess_nnue.data streams the file over HTTP and stops early, so the 22 GB file is never stored. We read the first 2.19 GB (32.0M positions) and kept 24.0M.

Kept: the score from the side to move's view (mates clamped to +-3000 cp), only for positions whose best move is quiet (not a capture or promotion), because the engine evaluates quiet positions at the end of its search.

Stockfish (GPL-3) is used only as a benchmark opponent and, through the Lichess database, as the source of labels. No Stockfish code or weights are copied. Maia and lc0 (GPL-3) are only benchmark opponents. The board page serves python-chess's built-in piece images (the cburnett set).

Train or regenerate the weights 

cd reasoning_harness
python -m chess_nnue.data --out runs/nnue/data/evals24.npz --max-gb 2.8 --max-positions 24000000   # ~2.2 GB streamed, 0.84 GB on disk
python -m chess_nnue.train --data runs/nnue/data/evals24.npz --run runs/nnue/v2 --epochs 4 --threads 10
cp runs/nnue/v2/chess.nnue artifacts/chess_nnue/chess.nnue
python -m unittest tests.test_chess_nnue

Training: 4 epochs, batch 16,384, Adam (lr 1e-3, cosine), 27 minutes on 10 CPU threads, about 3 GB RAM. Loss: MSE between sigmoid(output) and sigmoid(cp/400). The last 100,000 positions (a contiguous block) are held out; the exported weights are the checkpoint with the lowest held-out loss (0.01203, the final step). A first run on 12M positions overfit (held-out loss stalled at 0.0148 while training loss kept falling), which is why the data was doubled.

Use it 

import chess
from chess_nnue.engine import Engine
from chess_nnue.net import NNUE

engine = Engine(NNUE("artifacts/chess_nnue/chess.nnue"))
move, info = engine.search(chess.Board(), 1000)   # ms budget
print(move, info)   # e.g. d2d4 {'depth': 6, 'nodes': ..., 'score': ..., 'ms': ...}
  • Browser board: python -m chess_nnue.ui (http://127.0.0.1:7863): drag or click to move, promotion picker, eval bar, take back, flip.
  • UCI engine (any chess GUI or harness, e.g. Cute Chess, Arena, python-chess): python -m chess_nnue.uci from reasoning_harness/.
  • Lichess bot: chess_nnue/lichess_bot.py is a lichess-bot "homemade" engine.

Run the benchmarks 

python -m chess_nnue.bench games --a nnue:full --weights artifacts/chess_nnue/chess.nnue \
  --b random maia1100 maia1900 --games 100 --lc0 lc0.exe --maia-dir maia/ --tag chessbench
python -m chess_nnue.bench games --a nnue:full --weights ... --b sf1800 sf2100 sf2400 --games 50 --stockfish stockfish.exe --tag stockfish
python -m chess_nnue.bench games --a nnue:full --weights ... --b engine4 --ms 3000 --arena ../llm-chess-agentic --tag ablation_nnue_full
python -m chess_nnue.bench puzzles --a nnue:full --weights ... --ms 1000 --kagi puzzles.csv
python scripts/plot_chess_nnue.py

Limits

  • Speed. Python search reaches only 13-20k positions/s, so the engine is far weaker than the same network would be in a compiled engine. This is also why the piece-square-table version beats it against engine-4 at 3 s/move.
  • The ChessBench rating is an extrapolation. No losses to the anchors; the rating comes from 7 draws. We did not play ChessBench's strong anchor (Eubos).
  • Comparisons with LLMs are indirect. We never played an LLM; we played the same anchors (ChessBench) or the same puzzles (Kagi) under the same rules.
  • Stockfish's strength limiter is noisy: the 1800 and 2100 results are less far apart than their labels. The rating uses all 150 games together.
  • Data: only the first 2.2 GB of the Lichess evaluation file; those positions lean towards whatever Lichess users analysed most.
  • Not tested: time controls under 60 s per side, Chess960, and other hardware.