0xrushi commited on
Commit
d2debd4
·
verified ·
1 Parent(s): 42570e9

Add experiment model card and evaluation summary

Browse files
Files changed (1) hide show
  1. README.md +54 -1
README.md CHANGED
@@ -1,3 +1,56 @@
1
  ---
2
- license: mit
 
 
 
 
 
 
 
 
 
 
3
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
+ library_name: transformers
3
+ license: other
4
+ license_name: lfm1.0
5
+ license_link: LICENSE
6
+ base_model: mlabonne/LFM2.5-230M-Chess
7
+ pipeline_tag: text-generation
8
+ tags:
9
+ - chess
10
+ - lfm2
11
+ - research
12
+ - transformers
13
  ---
14
+
15
+ # Chess language-model experiments
16
+
17
+ Five chess language-model checkpoints and evaluation summaries from experiments in [0xrushi/ChessLFM](https://github.com/0xrushi/ChessLFM). These checkpoints are provided for research and reproducibility.
18
+
19
+ **Important provenance:** the released baseline is the public [mlabonne/LFM2.5-230M-Chess](https://huggingface.co/mlabonne/LFM2.5-230M-Chess) checkpoint. The four experiment checkpoints were fine-tuned from that released chess model. **None of these checkpoints was trained from random initialization or from scratch.** The baseline is included as an upstream reference, not claimed as our work.
20
+
21
+ ## Checkpoints and observed results
22
+
23
+ | Rank by highest observed score | Folder | Checkpoint | Match score | W/D/L | Opening block | Descriptive Elo* |
24
+ | ---: | --- | --- | ---: | ---: | --- | ---: |
25
+ | 1 | `released-baseline/` | Released ChessLFM baseline | 17.5/32 | 12/11/9 | Later (16–31) | ~2033 |
26
+ | 2 | `branch-ranking-treatment/` | Branch-ranking treatment | 15.5/32 | 12/7/13 | Later (16–31) | ~1989 |
27
+ | 3 | `branch-ranking-control/` | Matched branch-ranking control | 13/32 | 5/16/11 | Later (16–31) | ~1934 |
28
+ | 4 (tie) | `self-play-refresh/` | Self-play refresh, round 1 | 12/32 | 9/6/17 | Initial (0–15) | ~1911 |
29
+ | 4 (tie) | `action-value-finetune/` | Action-value fine-tune, standard search | 12/32 | 10/4/18 | Initial (0–15) | ~1911 |
30
+
31
+ *The Elo column applies the logistic score transform relative to Stockfish configured at Elo 2000. It is descriptive only: 8–32 games per condition gives substantial uncertainty. Opening blocks differ, so ranks across blocks are only an ordering of the highest observed scores, not a controlled cross-model comparison. In the later block, the paired bootstrap interval for branch-ranking treatment versus control was [−4.0, +9.5] points; it includes no advantage. See the detailed [experiment report](https://github.com/0xrushi/ChessLFM/blob/main/EXPERIMENT_REPORT_2026-10-01.md).
32
+
33
+ ## What is included
34
+
35
+ Each folder contains Transformers model weights, model configuration, the chess tokenizer, and `token_ids.json`. Training recovery states, optimizer files, raw datasets, and match journals are not included. Use the project code for the chess-specific 80-token prompt, legal-move masking, and depth-three search; these weights alone are not a complete chess engine.
36
+
37
+ Example loading one checkpoint:
38
+
39
+ ```python
40
+ from transformers import AutoModelForCausalLM, AutoTokenizer
41
+
42
+ repo = "0xrushi/chesslfm-experiments"
43
+ subfolder = "released-baseline"
44
+ tokenizer = AutoTokenizer.from_pretrained(repo, subfolder=subfolder)
45
+ model = AutoModelForCausalLM.from_pretrained(repo, subfolder=subfolder)
46
+ ```
47
+
48
+ ## License
49
+
50
+ The model weights are derivatives of [LiquidAI/LFM2.5-230M-Base](https://huggingface.co/LiquidAI/LFM2.5-230M-Base) through the upstream ChessLFM checkpoint. They are distributed under the [LFM Open License v1.0](LICENSE), including its commercial-use threshold. The repository's training code is separately available under MIT in the GitHub project.
51
+
52
+ ## References
53
+
54
+ - Upstream chess model: [mlabonne/LFM2.5-230M-Chess](https://huggingface.co/mlabonne/LFM2.5-230M-Chess)
55
+ - Base model and license: [LiquidAI/LFM2.5-230M-Base](https://huggingface.co/LiquidAI/LFM2.5-230M-Base)
56
+ - ChessLFM article: [ChessLFM: 2000 Elo in 230M params](https://maximelabonne.substack.com/p/chesslfm-2000-elo-in-230m-params)