Add Stackcraft technical report, figures and reproducible source link
Browse files- .gitattributes +1 -0
- README.md +13 -0
- paper/README.md +27 -0
- paper/build-manifest.json +46 -0
- paper/figures/latency.svg +1860 -0
- paper/figures/outcomes.svg +2089 -0
- paper/stackcraft-technical-report.md +313 -0
- paper/stackcraft-technical-report.pdf +3 -0
.gitattributes
CHANGED
|
@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
|
| 33 |
*.zip filter=lfs diff=lfs merge=lfs -text
|
| 34 |
*.zst filter=lfs diff=lfs merge=lfs -text
|
| 35 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
|
|
|
|
|
| 33 |
*.zip filter=lfs diff=lfs merge=lfs -text
|
| 34 |
*.zst filter=lfs diff=lfs merge=lfs -text
|
| 35 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
| 36 |
+
paper/stackcraft-technical-report.pdf filter=lfs diff=lfs merge=lfs -text
|
README.md
CHANGED
|
@@ -15,6 +15,19 @@ tags:
|
|
| 15 |
|
| 16 |
# Stackcraft Clef-Flash
|
| 17 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 18 |
Selected checkpoint: **epoch-02**, chosen on validation before final tests.
|
| 19 |
This is a small, synthetic imitation-learning study of a falling-block placement
|
| 20 |
policy. It is not a generally capable game agent, reinforcement-learning result,
|
|
|
|
| 15 |
|
| 16 |
# Stackcraft Clef-Flash
|
| 17 |
|
| 18 |
+
## Technical report
|
| 19 |
+
|
| 20 |
+
**[Read the technical report (PDF)](paper/stackcraft-technical-report.pdf)** ·
|
| 21 |
+
[Read online](paper/stackcraft-technical-report.md) ·
|
| 22 |
+
[Source and reproducible build](https://github.com/kkarimi/stackcraft/tree/c5c7ada5878b0e6cf7e00387a7fdcca1445651a0/paper)
|
| 23 |
+
|
| 24 |
+
*Stackcraft: Adapting Clef-flash to a Falling-Block Decision Task* — Nima (nima1),
|
| 25 |
+
6 October 2026. A seven-page technical report with methods, measured results,
|
| 26 |
+
figures, limitations and AI-assistance disclosure. **Not peer reviewed.**
|
| 27 |
+
The paper reports 16.81 mean lines for the trained policy versus 0.07 for native
|
| 28 |
+
base and 76.53 for the heuristic. It is an additive document release; the model
|
| 29 |
+
weights, dataset and original experimental evidence are unchanged.
|
| 30 |
+
|
| 31 |
Selected checkpoint: **epoch-02**, chosen on validation before final tests.
|
| 32 |
This is a small, synthetic imitation-learning study of a falling-block placement
|
| 33 |
policy. It is not a generally capable game agent, reinforcement-learning result,
|
paper/README.md
ADDED
|
@@ -0,0 +1,27 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Stackcraft technical report
|
| 2 |
+
|
| 3 |
+
[Read the PDF](stackcraft-technical-report.pdf) · [Read online](stackcraft-technical-report.md)
|
| 4 |
+
|
| 5 |
+
Nima (nima1) · 6 October 2026 · Version 1.0 · **Not peer reviewed**
|
| 6 |
+
|
| 7 |
+
The seven-page report presents the frozen Stackcraft study with two vector figures,
|
| 8 |
+
three measured-results tables, limitations, references and AI-assistance disclosure.
|
| 9 |
+
|
| 10 |
+
[Exact source and build instructions](https://github.com/kkarimi/stackcraft/tree/c5c7ada5878b0e6cf7e00387a7fdcca1445651a0/paper) ·
|
| 11 |
+
[Milestone tutorial](https://github.com/kkarimi/stackcraft/blob/c5c7ada5878b0e6cf7e00387a7fdcca1445651a0/docs/tutorials/07-technical-report.md)
|
| 12 |
+
|
| 13 |
+
Rebuild from that GitHub checkout with:
|
| 14 |
+
|
| 15 |
+
```bash
|
| 16 |
+
uv run --project paper --locked python paper/build.py
|
| 17 |
+
```
|
| 18 |
+
|
| 19 |
+
This directory adds documentation only. The report cites the original model
|
| 20 |
+
revision c4272310bd6c63ee97a941255abf8f36ff162229 and dataset revision
|
| 21 |
+
211708dd56f1a8af711c060c2ab1ecab35e7166d; those remain the immutable experimental
|
| 22 |
+
artifacts. The new document commit does not modify the checkpoint, dataset,
|
| 23 |
+
original code bundle, or evidence archive. `build-manifest.json` records source
|
| 24 |
+
report hashes, build inputs, document/figure hashes and dependency versions.
|
| 25 |
+
Native renderer libraries may affect PDF bytes when rebuilding on another platform.
|
| 26 |
+
|
| 27 |
+
Apache-2.0. No arXiv ID, DOI or indexed HF Paper Page is claimed.
|
paper/build-manifest.json
ADDED
|
@@ -0,0 +1,46 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"schema_version": 1,
|
| 3 |
+
"scope": "Presentation of frozen evidence; no new experiments or statistical estimates",
|
| 4 |
+
"frozen_evidence": {
|
| 5 |
+
"reports/final-summary.json": "63cdd8c6f3a51a943041714d255cbd9c3d6711f25e028dc6cfd7042ff08ff066",
|
| 6 |
+
"reports/published-checkpoint-reload.json": "a89b2e65f465f97966de78d9107671c647b1b1b2546b4f39840d1f941cd0b86b",
|
| 7 |
+
"reports/publication.json": "f2e32a4379c4334ee33ca345e6167a21610ef3c0d13e17ed97c4dbe3a819bfd0"
|
| 8 |
+
},
|
| 9 |
+
"inputs": {
|
| 10 |
+
"paper/report.md": "641733f78eeda30c311805a8571736649c2f51b61107458425ff4d30e6a940cb",
|
| 11 |
+
"paper/metadata.json": "93e9a55b493cd9dd74f1846e7b47d4545290f31583e200140bcbbdb328913c15",
|
| 12 |
+
"paper/style.css": "be4ce64f842794b229e7b2a742b5a09d2218e320fbd8acaf8ef640ebf7cb6442",
|
| 13 |
+
"paper/build.py": "6f55036d9039fc24948d3c609275305d2ad32cfa55fe99f2d7b68e5fde9af06c",
|
| 14 |
+
"paper/pyproject.toml": "656c6a0eb0768df38ba263a0fdab9283cc7a95487fe485162ef5919b876e00ee",
|
| 15 |
+
"paper/uv.lock": "216fea2d72c52d181642590774f43f3945998d7c099bfa59aae9c5e0661bc0dd",
|
| 16 |
+
"paper/evidence.json": "e31ebb7384e80efd68f6ecdd5800f7cc8ff5821e3db74c21fa2e24e36c9ac33c"
|
| 17 |
+
},
|
| 18 |
+
"outputs": {
|
| 19 |
+
"stackcraft-technical-report.pdf": {
|
| 20 |
+
"sha256": "bc006f185507965483576254ce241d74afcb8515f8fa56b766cade464b672829",
|
| 21 |
+
"bytes": 101109
|
| 22 |
+
},
|
| 23 |
+
"stackcraft-technical-report.md": {
|
| 24 |
+
"sha256": "95822e9954457f621f8e0173776a24d7390d8a2662e1c6ea94ceaaf4c6331da8",
|
| 25 |
+
"bytes": 19820
|
| 26 |
+
},
|
| 27 |
+
"figures/latency.svg": {
|
| 28 |
+
"sha256": "805d33917441ca9ded9f06dd24e078406a071df9676b8cec73759c2e0f74bd9e",
|
| 29 |
+
"bytes": 54775
|
| 30 |
+
},
|
| 31 |
+
"figures/outcomes.svg": {
|
| 32 |
+
"sha256": "27ffe1afc9dc5a0022aa0227b0f3b6c41e94bae5412af54e2e4cdd69170fee1b",
|
| 33 |
+
"bytes": 68942
|
| 34 |
+
}
|
| 35 |
+
},
|
| 36 |
+
"python": "3.13.16",
|
| 37 |
+
"versions": {
|
| 38 |
+
"matplotlib": "3.11.2",
|
| 39 |
+
"markdown-it-py": "4.2.0",
|
| 40 |
+
"weasyprint": "70.0",
|
| 41 |
+
"pypdf": "6.19.0"
|
| 42 |
+
},
|
| 43 |
+
"pages": 7,
|
| 44 |
+
"pdf_text_checks": "passed",
|
| 45 |
+
"fonts": "DejaVu fonts from the locked Matplotlib distribution; embedded in PDF"
|
| 46 |
+
}
|
paper/figures/latency.svg
ADDED
|
|
paper/figures/outcomes.svg
ADDED
|
|
paper/stackcraft-technical-report.md
ADDED
|
@@ -0,0 +1,313 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Stackcraft: Adapting Clef-flash to a Falling-Block Decision Task
|
| 2 |
+
|
| 3 |
+
Nima (nima1) · 2026-10-06 · Version 1.0
|
| 4 |
+
|
| 5 |
+
**Technical report · Not peer reviewed**
|
| 6 |
+
|
| 7 |
+
|
| 8 |
+
## Abstract
|
| 9 |
+
|
| 10 |
+
Can a general decision model learn a useful game policy from a small synthetic
|
| 11 |
+
dataset on one workstation? Stackcraft studies this question in a deterministic,
|
| 12 |
+
turn-based falling-block game. A Clef-flash model was adapted using rank-4 LoRA
|
| 13 |
+
and its full native decision head, with 827 training positions and 215 validation
|
| 14 |
+
positions labeled by a bounded search teacher. One training run produced two
|
| 15 |
+
checkpoint candidates; validation negative log likelihood selected the second.
|
| 16 |
+
Across 200 previously reserved piece sequences per policy, the selected model
|
| 17 |
+
cleared 16.81 lines per game, versus 0.07 for unchanged Clef-flash. The paired
|
| 18 |
+
improvement was 16.74 lines, with a 95% bootstrap interval of [15.74, 17.77].
|
| 19 |
+
However, a fixed arithmetic heuristic cleared 76.53 lines and was substantially
|
| 20 |
+
faster. All 1,000 evaluated episodes completed without inference errors or invalid
|
| 21 |
+
decisions. The study demonstrates task adaptation under a constrained local
|
| 22 |
+
budget, while showing why improvement over a weak base model is insufficient
|
| 23 |
+
evidence of practical superiority. Code, data, learned parameters, recorded games,
|
| 24 |
+
and verification evidence are public. This is a technical report, not a
|
| 25 |
+
peer-reviewed publication.
|
| 26 |
+
|
| 27 |
+
## 1. Motivation and scope
|
| 28 |
+
|
| 29 |
+
A falling-block game offers a concrete test of decision quality: choices alter
|
| 30 |
+
the board, mistakes accumulate, and complete games expose failures that isolated
|
| 31 |
+
classification scores can hide. Stackcraft asks whether a small imitation-learning
|
| 32 |
+
dataset can improve a general decision model, and whether that improvement is
|
| 33 |
+
competitive with a simple task-specific policy. The project also makes the
|
| 34 |
+
experiment inspectable through replayable games and milestone tutorials.
|
| 35 |
+
|
| 36 |
+
Clef-flash is a roughly nine-billion-parameter model derived from Qwen3.5-9B.
|
| 37 |
+
Its native interface accepts a state and typed questions, then produces
|
| 38 |
+
probabilities over supplied options through a joint decision head. This avoids
|
| 39 |
+
requiring a generated command or a text parser to select a legal move. Stackcraft
|
| 40 |
+
uses text and structured state only; it does not test the model's visual
|
| 41 |
+
capabilities. The implementation follows the pinned upstream interface. [[1]](https://huggingface.co/Cloudflare/clef-flash/blob/17f0b0ad64efb65d273590632833508766b2aae6/README.md)
|
| 42 |
+
|
| 43 |
+
The contribution is an engineering case study with released evidence. No new
|
| 44 |
+
adaptation algorithm, general game-playing advance, or superiority over
|
| 45 |
+
specialized game agents is claimed. The central distinction is between learning
|
| 46 |
+
to play better than the unchanged model and building the best bot for these rules.
|
| 47 |
+
|
| 48 |
+
## 2. Game and policy interface
|
| 49 |
+
|
| 50 |
+
The board has ten columns and twenty rows. Seven tetromino shapes arrive in
|
| 51 |
+
seeded seven-bag sequences: each shuffled bag contains every shape once. A turn
|
| 52 |
+
chooses a legal orientation and column, then drops the piece vertically. The
|
| 53 |
+
rules omit hold, wall kicks, tucks, T-spin bonuses, and real-time movement
|
| 54 |
+
requirements. Clearing one, two, three, or four lines awards 100, 300, 500, or
|
| 55 |
+
800 points. Lines cleared per episode is the primary outcome.
|
| 56 |
+
|
| 57 |
+
Every policy receives the board, current piece, one next-piece preview, and the
|
| 58 |
+
same legal placement options. Hidden future pieces, the episode seed, sequence
|
| 59 |
+
index, and generator state are excluded from the observation. Python owns the
|
| 60 |
+
authoritative transitions, so browser play, offline tournaments, and replay
|
| 61 |
+
validation use the same rules. This is a structured placement task rather than a
|
| 62 |
+
benchmark of full competitive Tetris.
|
| 63 |
+
|
| 64 |
+
Five policies were evaluated. **Native base** retains unchanged Clef-flash weights
|
| 65 |
+
and its original BF16 head. **FP32 base** uses unchanged weights with the FP32 head
|
| 66 |
+
wrapper used for training. **Trained** combines the selected LoRA adapter with the
|
| 67 |
+
learned FP32 head. All three share the pinned backbone, observation encoding,
|
| 68 |
+
option ordering, BF16 backbone precision, and 4,096-token limit. Full observations
|
| 69 |
+
are checked against that limit before inference.
|
| 70 |
+
|
| 71 |
+
**Random** samples uniformly from legal options using a recorded random generator.
|
| 72 |
+
**Heuristic** greedily evaluates the resulting board with a fixed formula:
|
| 73 |
+
negative aggregate column height, minus four times the number of holes, minus
|
| 74 |
+
adjacent-column height differences, plus eight times the number of cleared lines.
|
| 75 |
+
A hole is an empty cell below an occupied cell in its column. The heuristic does
|
| 76 |
+
not use the available preview; its weights were frozen before final testing. It
|
| 77 |
+
is distinct from the lookahead teacher that generated training labels. [[3]](https://github.com/kkarimi/stackcraft/blob/82a8854258f228a214f0964bbbd3d2b52e2be1e5/reports/final-study.md)
|
| 78 |
+
|
| 79 |
+
## 3. Synthetic data and adaptation
|
| 80 |
+
|
| 81 |
+
### Data construction
|
| 82 |
+
|
| 83 |
+
The dataset contains 827 training positions from seeds 10000–10023 and 215
|
| 84 |
+
validation positions from seeds 20000–20005. Collection cycles among random,
|
| 85 |
+
greedy-heuristic, and search-expert behavior, recording at most forty moves per
|
| 86 |
+
episode. This exposes the learner to boards produced by different policies rather
|
| 87 |
+
than only successful expert trajectories. Every episode belongs to one split.
|
| 88 |
+
Occupancy-normalized duplicate detection removed six repeated training positions
|
| 89 |
+
and two validation positions that overlapped the training split.
|
| 90 |
+
|
| 91 |
+
For each retained observation, the teacher enumerates legal placements of the
|
| 92 |
+
current piece and the one visible preview, then evaluates line clears and board
|
| 93 |
+
quality. It cannot inspect unseen pieces. Its chosen action is a deterministic,
|
| 94 |
+
bounded-search recommendation, not a globally optimal label. The released dataset
|
| 95 |
+
records legal choices, teacher values, labels, source identity, and split
|
| 96 |
+
provenance. It is programmatically generated synthetic data; it is not a corpus
|
| 97 |
+
of human gameplay or human-certified optimal decisions. The reserved test seeds
|
| 98 |
+
30000–30199 were excluded from collection and checkpoint selection. [[4]](https://huggingface.co/datasets/nima1/stackcraft-data/tree/211708dd56f1a8af711c060c2ab1ecab35e7166d)
|
| 99 |
+
|
| 100 |
+
### Training configuration
|
| 101 |
+
|
| 102 |
+
LoRA adds trainable low-rank weight updates while retaining frozen backbone
|
| 103 |
+
parameters. [[2]](https://arxiv.org/abs/2106.09685) Here it was applied to the text backbone alongside full training
|
| 104 |
+
of Clef's native joint head. A single seed-42 run used rank 4, alpha 8, zero
|
| 105 |
+
adapter dropout, a BF16 unquantized backbone, and an FP32 head. The combined
|
| 106 |
+
trainable parameter count was 132,582,404. This count includes the full head;
|
| 107 |
+
“LoRA” should not be read as implying that only a tiny adapter was trained.
|
| 108 |
+
|
| 109 |
+
Training ran for exactly two epochs. The objective combined cross-entropy with
|
| 110 |
+
0.05 label smoothing and a Brier-loss term weighted by 0.1. AdamW used a learning
|
| 111 |
+
rate of 0.00001 and weight decay 0.01. Batch size was one, gradient accumulation
|
| 112 |
+
was eight, and gradient norm clipping was 1.0. Each epoch processed all 827
|
| 113 |
+
positions in 104 optimizer steps. The final accumulation group used its actual
|
| 114 |
+
three examples for normalization. Recorded hashes confirmed that frozen
|
| 115 |
+
parameters remained unchanged while intended trainable parameters changed.
|
| 116 |
+
|
| 117 |
+
### Selection and reload checks
|
| 118 |
+
|
| 119 |
+
The two epoch checkpoints were the entire preregistered selection search.
|
| 120 |
+
Eligibility required complete, finite predictions on all 215 validation positions;
|
| 121 |
+
selection minimized mean target negative log likelihood (NLL), with exact ties
|
| 122 |
+
going to the earlier epoch. Both candidates qualified, and epoch 02 was selected.
|
| 123 |
+
Teacher agreement and Brier scores were diagnostics, not alternative selection
|
| 124 |
+
criteria. The local preregistration was preserved in source; it was not a signed
|
| 125 |
+
external registration.
|
| 126 |
+
|
| 127 |
+
| Condition | Teacher agreement | Mean NLL | Mean Brier |
|
| 128 |
+
| --- | --- | --- | --- |
|
| 129 |
+
| Native base | 21/215 (9.77%) | 2.952702 | 0.923173 |
|
| 130 |
+
| FP32 base | 22/215 (10.23%) | 2.952393 | 0.923102 |
|
| 131 |
+
| Epoch 01 | 89/215 (41.40%) | 1.860870 | 0.722190 |
|
| 132 |
+
| Epoch 02 (selected) | 92/215 (42.79%) | 1.775923 | 0.708980 |
|
| 133 |
+
|
| 134 |
+
*Table 1. Validation results on all 215 positions. Lower NLL and Brier values are
|
| 135 |
+
better. The second epoch was selected before test games were opened.*
|
| 136 |
+
|
| 137 |
+
The selected model agreed with 92 teacher actions, or 42.79%. That percentage is
|
| 138 |
+
not a game-quality score. Alternative moves may be useful, teacher tie-breaking
|
| 139 |
+
can affect agreement, and a wrong move changes the states encountered later.
|
| 140 |
+
Complete held-out games therefore provide the behavioral evaluation.
|
| 141 |
+
|
| 142 |
+
The checkpoint saves both the adapter and native head. A fresh-process reload
|
| 143 |
+
matched four fixed training-position reference distributions exactly, with a
|
| 144 |
+
maximum absolute probability difference of 0.0 against a declared tolerance of
|
| 145 |
+
0.0001. The same four-reference check later passed using anonymously downloaded
|
| 146 |
+
published code and weights. These checks establish serialization consistency on
|
| 147 |
+
those references, not correctness on every board. [[3]](https://github.com/kkarimi/stackcraft/blob/82a8854258f228a214f0964bbbd3d2b52e2be1e5/reports/final-study.md), [[5]](https://github.com/kkarimi/stackcraft/blob/ad348c20820061b130abfc2890ab8de00b62e7e4/reports/publication.md)
|
| 148 |
+
|
| 149 |
+
## 4. Held-out evaluation
|
| 150 |
+
|
| 151 |
+
Each policy played the same 200 held-out sequences, seeds 30000–30199, with a cap
|
| 152 |
+
of 200 placed pieces per episode. The tournament therefore contains 1,000
|
| 153 |
+
episodes. A cap hit means survival for at least 200 pieces; it is not a loss on
|
| 154 |
+
piece 200. No model, prompt, policy, dataset, or checkpoint was tuned after opening
|
| 155 |
+
this final test.
|
| 156 |
+
|
| 157 |
+
Comparisons subtract outcomes within each seed. The analysis resamples the 200
|
| 158 |
+
paired episode differences with replacement 10,000 times, using bootstrap seed
|
| 159 |
+
2026, and takes percentile 95% intervals. Resampling episodes retains dependence
|
| 160 |
+
among decisions in the same game. The primary contrast is trained minus native
|
| 161 |
+
base in lines cleared. Score and survival intervals are descriptive secondary
|
| 162 |
+
results without a multiplicity correction.
|
| 163 |
+
|
| 164 |
+
| Player | Mean lines | Median lines | Mean score | Mean pieces | Cap hits |
|
| 165 |
+
| --- | --- | --- | --- | --- | --- |
|
| 166 |
+
| Native base | 0.070 | 0 | 7.0 | 26.020 | 0/200 |
|
| 167 |
+
| FP32 base | 0.105 | 0 | 10.5 | 26.000 | 0/200 |
|
| 168 |
+
| Trained | 16.810 | 16 | 1745.5 | 83.565 | 0/200 |
|
| 169 |
+
| Random | 0.140 | 0 | 14.0 | 26.645 | 0/200 |
|
| 170 |
+
| Heuristic | 76.530 | 77 | 8306.5 | 200.000 | 200/200 |
|
| 171 |
+
|
| 172 |
+
*Table 2. Held-out outcomes for 200 episodes per policy. Survival is pieces
|
| 173 |
+
placed. All five policies recorded zero inference errors and invalid decisions.*
|
| 174 |
+
|
| 175 |
+

|
| 176 |
+
|
| 177 |
+
*Figure 1. Mean lines cleared and paired differences for the selected checkpoint.
|
| 178 |
+
Intervals describe variability across held-out sequences for this checkpoint,
|
| 179 |
+
not variation across independently trained models.*
|
| 180 |
+
|
| 181 |
+
The trained model cleared 16.81 lines on average, compared with 0.07 for native
|
| 182 |
+
base: a gain of 16.74 lines, 95% interval [15.74, 17.77]. The gain against FP32 base
|
| 183 |
+
was 16.705 [15.705, 17.740]. The FP32-minus-native difference was only 0.035
|
| 184 |
+
[−0.020, 0.090], so changing head precision alone does not explain the observed
|
| 185 |
+
adaptation gain. This comparison does not separate learned adapter effects from
|
| 186 |
+
learned head effects; both were optimized together.
|
| 187 |
+
|
| 188 |
+
The heuristic remained substantially stronger, clearing 76.53 lines on average.
|
| 189 |
+
The trained-minus-heuristic difference was −59.720 [−60.805, −58.619875]. Every
|
| 190 |
+
heuristic episode reached the cap; no other policy did. Its unrestricted survival
|
| 191 |
+
is consequently unknown. No failed episodes were removed. The frozen analysis
|
| 192 |
+
would assign zero lines, score, and survival to an errored episode while retaining
|
| 193 |
+
its partial outcome separately; because there were no errors, adjusted and
|
| 194 |
+
observed results coincide.
|
| 195 |
+
|
| 196 |
+
An independently implemented audit reconstructed all 1,000 saved episodes,
|
| 197 |
+
checked observations and legal choices, verified source and checkpoint bindings,
|
| 198 |
+
and recomputed all twelve paired intervals across lines, score, and survival.
|
| 199 |
+
It matched the published results. This was a separate code-path check by another
|
| 200 |
+
AI agent, not external scientific replication. [[3]](https://github.com/kkarimi/stackcraft/blob/82a8854258f228a214f0964bbbd3d2b52e2be1e5/reports/final-study.md)
|
| 201 |
+
|
| 202 |
+
## 5. Runtime and practical value
|
| 203 |
+
|
| 204 |
+
Experiments ran on an NVIDIA RTX 5090 with roughly 32 GB of VRAM, an AMD Ryzen 9
|
| 205 |
+
9950X3D, and about 123 GiB of host RAM. The recorded environment used Python
|
| 206 |
+
3.13.16, Torch 2.14.1+cu130, CUDA 13.0, Transformers 5.18.0, and PEFT 0.21.2.
|
| 207 |
+
Torch used eight CPU threads and TF32 was disabled.
|
| 208 |
+
|
| 209 |
+
| Player | Mean ms | Median ms | p95 ms | Decisions |
|
| 210 |
+
| --- | --- | --- | --- | --- |
|
| 211 |
+
| Native base | 149.109957 | 139.381274 | 244.916266 | 5,204 |
|
| 212 |
+
| FP32 base | 150.188366 | 140.968013 | 247.879591 | 5,200 |
|
| 213 |
+
| Trained | 206.814401 | 171.844287 | 309.000088 | 16,713 |
|
| 214 |
+
| Random | 0.001497 | 0.001250 | 0.002879 | 5,329 |
|
| 215 |
+
| Heuristic | 0.198510 | 0.139489 | 0.280528 | 40,000 |
|
| 216 |
+
|
| 217 |
+
*Table 3. Per-decision timings over each policy's actual tournament workload.
|
| 218 |
+
Initialization is excluded; no warmup calls within the tournament were removed.*
|
| 219 |
+
|
| 220 |
+

|
| 221 |
+
|
| 222 |
+
*Figure 2. Observed mean decision latency. Neural policies use the GPU; random and
|
| 223 |
+
heuristic policies use the CPU. Different policies visit different boards, so
|
| 224 |
+
this is not a controlled same-state speed benchmark.*
|
| 225 |
+
|
| 226 |
+
Decision timings include encoding and probability transfer, but exclude model
|
| 227 |
+
loading, rendering, and replay serialization. The trained mean was 206.814 ms,
|
| 228 |
+
versus approximately 0.198510 ms for the heuristic: about 1,042 times longer on
|
| 229 |
+
these workloads. The trained policy also saw longer inputs on average than either
|
| 230 |
+
unchanged model. These measurements support choosing the heuristic for a practical
|
| 231 |
+
bot under the tested rules; they do not isolate a hardware-independent inference
|
| 232 |
+
speed ratio.
|
| 233 |
+
|
| 234 |
+
Training took 1,511.25 seconds, or 25.2 minutes, including checkpoint and hash
|
| 235 |
+
work. Peak CUDA allocation was 23.23 GB (21.64 GiB), with 24.18 GB reserved.
|
| 236 |
+
The final evaluation wrapper took 5,103.11 seconds, or 85.1 minutes, including
|
| 237 |
+
loading, reporting, and service restoration checks. These durations exclude
|
| 238 |
+
previous development, validation, and data generation. Energy use and monetary
|
| 239 |
+
cost were not measured. [[3]](https://github.com/kkarimi/stackcraft/blob/82a8854258f228a214f0964bbbd3d2b52e2be1e5/reports/final-study.md)
|
| 240 |
+
|
| 241 |
+
## 6. Limitations and next experiments
|
| 242 |
+
|
| 243 |
+
The strongest limitation is the single training seed. The bootstrap intervals
|
| 244 |
+
measure variability across episode sequences for one selected model; they do not
|
| 245 |
+
quantify training instability. The dataset is small and synthetic, and its teacher
|
| 246 |
+
optimizes a short-horizon approximation. Collection stops after at most forty
|
| 247 |
+
moves, whereas test games can last two hundred. Errors may therefore lead the
|
| 248 |
+
policy into poorly represented states, although this study does not establish a
|
| 249 |
+
causal failure mechanism.
|
| 250 |
+
|
| 251 |
+
Only one structured encoding, one adaptation configuration, and two epoch
|
| 252 |
+
candidates were studied. The weak unchanged-model result characterizes that pinned
|
| 253 |
+
model under this interface, not every possible use of Clef-flash. The rules omit
|
| 254 |
+
many skills relevant to real-time Tetris. The heuristic's universal cap hits limit
|
| 255 |
+
claims about its full survival distribution. None of these findings establishes
|
| 256 |
+
performance on a different game, board representation, or deployment platform.
|
| 257 |
+
|
| 258 |
+
Useful follow-up studies could compare head-only training with joint head/LoRA
|
| 259 |
+
training, repeat training across seeds, and evaluate a compact game-specific
|
| 260 |
+
student against the same inexpensive heuristic. Gathering recovery states from
|
| 261 |
+
the learned policy could test whether broader state coverage improves complete
|
| 262 |
+
games. Those are proposed experiments, not measured improvements. Any follow-up
|
| 263 |
+
should declare its selection procedure and reserve fresh test sequences rather
|
| 264 |
+
than repeatedly optimizing against the published test set.
|
| 265 |
+
|
| 266 |
+
## 7. Reproducibility and availability
|
| 267 |
+
|
| 268 |
+
The public code repository includes the game, experiment scripts, tests, a locked
|
| 269 |
+
uv environment, and tutorials. The model release includes the adapter, learned
|
| 270 |
+
head, reproduction source, and a lossless evidence archive with 1,017 records.
|
| 271 |
+
The archive's inventory preserves uncompressed file hashes. Anonymous downloads
|
| 272 |
+
of the pinned model and dataset passed byte and schema verification; all archive
|
| 273 |
+
entries were read and hashed. The published reload check used downloaded source
|
| 274 |
+
and checkpoint files. [[5]](https://github.com/kkarimi/stackcraft/blob/ad348c20820061b130abfc2890ab8de00b62e7e4/reports/publication.md)
|
| 275 |
+
|
| 276 |
+
The frozen model release is
|
| 277 |
+
`c4272310bd6c63ee97a941255abf8f36ff162229`; the dataset release is
|
| 278 |
+
`211708dd56f1a8af711c060c2ab1ecab35e7166d`. The bundled release source is
|
| 279 |
+
`82a8854258f228a214f0964bbbd3d2b52e2be1e5`. These identify the experimental
|
| 280 |
+
artifacts, not the later additive PDF publication. The upstream Clef-flash
|
| 281 |
+
revision is `17f0b0ad64efb65d273590632833508766b2aae6`. [[1]](https://huggingface.co/Cloudflare/clef-flash/blob/17f0b0ad64efb65d273590632833508766b2aae6/README.md), [[4]](https://huggingface.co/datasets/nima1/stackcraft-data/tree/211708dd56f1a8af711c060c2ab1ecab35e7166d), [[6]](https://huggingface.co/nima1/stackcraft-clef-flash-lora/tree/c4272310bd6c63ee97a941255abf8f36ff162229)
|
| 282 |
+
|
| 283 |
+
Provenance distinguishes stages rather than assigning every result to the release
|
| 284 |
+
commit. Data generation used commit `70d84bd8dc5d4a60f3b96455a57d9e6f9416d109`.
|
| 285 |
+
Training recorded `d9fcd05c02e9327d4d791f2338725092cede8bdd` with a dirty working
|
| 286 |
+
tree and explicit source-file hashes; that commit alone is not the full training
|
| 287 |
+
source identity. Final evaluation used clean commit
|
| 288 |
+
`e524f0c15c2dd60ebfdc1374d66836c79e1e6dd6`. The retained reports and source hashes
|
| 289 |
+
are the authoritative record of these distinctions. [[3]](https://github.com/kkarimi/stackcraft/blob/82a8854258f228a214f0964bbbd3d2b52e2be1e5/reports/final-study.md)
|
| 290 |
+
|
| 291 |
+
The local browser demo pairs live human play with clearly labeled recordings from
|
| 292 |
+
the actual base, trained, and heuristic policies. Playback speed is not live
|
| 293 |
+
neural inference speed. Hosted CPU deployment is deferred; no public running
|
| 294 |
+
Space is claimed. Readers can inspect evidence and replay games without retraining
|
| 295 |
+
or renting a GPU.
|
| 296 |
+
|
| 297 |
+
## Assistance disclosure
|
| 298 |
+
|
| 299 |
+
AI coding agents assisted with implementation, experiment orchestration,
|
| 300 |
+
documentation, manuscript preparation, and computational review under the project
|
| 301 |
+
owner's direction. The separate audit used another agent and an independent
|
| 302 |
+
implementation, but was part of this project. Synthetic labels came from the
|
| 303 |
+
programmatic search teacher. No external peer review, independent laboratory
|
| 304 |
+
replication, or exhaustive human labeling review is claimed.
|
| 305 |
+
|
| 306 |
+
## References
|
| 307 |
+
|
| 308 |
+
1. Cloudflare. [Clef-Flash model card and native interface, pinned revision](https://huggingface.co/Cloudflare/clef-flash/blob/17f0b0ad64efb65d273590632833508766b2aae6/README.md).
|
| 309 |
+
2. Hu, E. J., et al. (2021). [LoRA: Low-Rank Adaptation of Large Language Models](https://arxiv.org/abs/2106.09685).
|
| 310 |
+
3. Stackcraft. [Frozen study report and linked experimental evidence](https://github.com/kkarimi/stackcraft/blob/82a8854258f228a214f0964bbbd3d2b52e2be1e5/reports/final-study.md).
|
| 311 |
+
4. Stackcraft. [Synthetic dataset and provenance, pinned release](https://huggingface.co/datasets/nima1/stackcraft-data/tree/211708dd56f1a8af711c060c2ab1ecab35e7166d).
|
| 312 |
+
5. Stackcraft. [Publication and fresh-download verification report](https://github.com/kkarimi/stackcraft/blob/ad348c20820061b130abfc2890ab8de00b62e7e4/reports/publication.md).
|
| 313 |
+
6. Stackcraft. [Adapter, native head, source, and evidence, pinned release](https://huggingface.co/nima1/stackcraft-clef-flash-lora/tree/c4272310bd6c63ee97a941255abf8f36ff162229).
|
paper/stackcraft-technical-report.pdf
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:bc006f185507965483576254ce241d74afcb8515f8fa56b766cade464b672829
|
| 3 |
+
size 101109
|