PBatch23888/chessq-chesscam

ChessQ ChessCam

A ChessQueries model for reading chess positions from ChessCam footage, exported to LiteRT for the browser. It is the model the ChessCam web app uses: a DINOv3 ViT-S/16 encoder with a ChessQueries decoder, at 320 × 320.

Built with ChessQueries (paper).

Files

File	Encoder	Input	Format
chessq-chesscam-dinov3-320-v4-fp16.tflite	DINOv3 ViT-S/16	320 × 320	LiteRT, FP16 weights (62.4 MB)
chessq-chesscam-dinov3-320-v4-fp32.safetensors	DINOv3 ViT-S/16	320 × 320	PyTorch, FP32 weights (124 MB)

The .tflite file runs with LiteRT.js on WebGPU or on the CPU (WebAssembly). The .safetensors file is in the ChessQueries checkpoint format. The model only works well at 320 × 320.

Quick start (PyTorch)

With Python 3.12:

python -m pip install "git+https://github.com/JSeytre/chessqueries.git@f5aaaf38212cc7839b03ccf383455053905b3f7e"
chessqueries-predict photo.jpg --checkpoint chessq-chesscam-dinov3-320-v4-fp32.safetensors --resolution 320

Input and output

• Input: uint8 (1, 320, 320, 4), RGBA pixels (for example, canvas ImageData.data as is). The image is resized to 320 × 320 without keeping its aspect ratio. Normalisation happens inside the model.
• Outputs (their order is up to the runtime, so tell them apart by shape):
  • float32 (1, 64, 13): softmax probabilities for each square, with squares in FEN order (a8, b8, …, h1). The classes are . P N B R Q K p n b r q k, where . is an empty square.
  • float32 (1, 64, 2): where each square's query attends in the input image, as (x, y) in [0, 1], squares in the same order. It is the attention-weighted centre of the image patches in the last decoder layer's cross-attention, so it is pulled towards the board centre: use it for directions (for example, which way rank 8 lies), not exact square positions.

The probabilities are overconfident. The web app's move tracking softens them as softmax(log p / 3.0), with its evidence weight scaled to match.

Training

The encoder starts from Meta's pretrained DINOv3 ViT-S/16 (LVD-1689M) weights; the ChessQueries decoder (4 layers) starts from scratch. The model was trained in three stages:

1. 45 epochs, at 448 × 448, on the training splits of ChessReD, ChessCog, ChessCam and SLCC, 4,527 synthetic renders (Blender) and 10,778 frames from 98 YunkerChess blitz videos.
2. 10 more epochs, at 448 × 448, on every image of ten datasets, with no validation or test split held back (64,820 images):

Dataset	Images	Source
ChessCam	17,936	Frames from recorded ChessCam games, labelled by converting the app's piece boxes and board grids to FENs
YunkerChess	21,430	Frames from 197 blitz videos by The Yunkers' Chess & Co, labelled from the 2D board the videos show beside the real one, keeping only frames where the position had stood still for at least half a second
ChessReD	10,800	ChessReD
Synthetic	5,000	Blender renders
ChessCog	4,888	ChessCog
SLCC	2,174	SLCC
Digitizer	1,356	Webcam frames of 20 games from Oliver Butler's Realtime-Chess-Digitizer (CC0)
Samryan	500	Overhead photos from Sam Ryan and Mukund Vaidya's chess-dataset (MIT)
Fenit	384	Phone photos from "Annotated Image Dataset for Fenit: Automatic Chessboard Digitization and FEN Generation" (Harjanto, Oei and Susanto, Binus University; CC BY 4.0)
CVChess	352	CVChess, with the labels released with ChessQueries (CC BY-NC 4.0)

3. 10 more epochs on the same images at 320 × 320, which takes about half the compute per frame.

Evaluation

As every image of those datasets was trained on, there is no held-out test set for them, and test-split scores would not be meaningful.

What it is tested on instead is the web app's move tracking, replayed on recorded YunkerChess blitz games that are not in the training set, against ChessQ Lite V4 (cq-vit-s-644-v4-seed2, at its 644 × 644 input). The results are for the 43 games (3,571 moves) in which no moves were taken back, each model at the tracker settings that suit it best, at the app's 4 frames a second:

Metric	This model	ChessQ Lite V4
Games with every move right	38 (88%)	27 (63%)
Moves right before each game's first mistake	3,308 (92.6%)	2,761 (77.3%)
Corrections needed to fix the moves found	110 (2.6 per game)	405 (9.4 per game)
Squares misread per frame	0.95	1.81

A correction is one move added, removed or changed (the edit distance from the true game); moves found after a game's last move, as the board is cleared, are not counted. Squares misread per frame are counted over the 57,164 frames where the position stood still (at least 1.5 s from any move). The tracker settings are a temperature and an evidence weight: 3.0 and 1.5 for this model, and for ChessQ Lite V4 3.5 and 1.05, the best of a sweep from 2.5 to 4.5.

Speed

Median time per run with LiteRT.js 2.6.0 in Chrome on a Pixel 7a, including reading the outputs back, against ChessQ Lite V4 (cq-vit-s-644-v4-seed2, exported to LiteRT the same way, at its 644 × 644 input):

Pixel 7a	This model (320 × 320)	ChessQ Lite V4 (644 × 644)	Speed-up
WebGPU (FP16)	196 ms	1,375 ms	7×
CPU (WebAssembly, 8 threads)	383 ms	3,400 ms	9×

Times are from a cool phone. Under sustained load the phone throttles its CPU within seconds, and the CPU times grow to about 680 ms for this model and 4,500 ms for ChessQ Lite V4; the WebGPU times barely change.

License

The model is a derivative of Meta's DINOv3 weights and is made with ChessQueries. It is distributed under the DINOv3 License and under the PolyForm Noncommercial License 1.0.0; use must comply with both.

Third-party components keep their upstream terms; see the notices.

Required Notice: Copyright (c) 2026 Joël Seytre.