leehack/laya-tetris-head

Laya Tetris head

A decision head for Laya, fine-tuned to pick Tetris placements. It is used by llamadart's laya_tetris example through DecisionEngine (llamadart#604).

laya-head-tetris.safetensors holds the 36 head tensors under Laya's PyTorch names, in F32, with no encoder tensors. Its laya.config metadata is the unmodified rl_agent_config.json of convaiinnovations/laya at revision 1c5edc17a7acd8701df6fc341c0d179f1c62c982, so DecisionEngine.load needs no configPath. Pair it with a Laya ModernBERT backbone GGUF, such as laya-Q8_0.gguf from fr0stbit3/laya-gguf.

final decisions = await DecisionEngine.load(
  engine, // a LlamaEngine with the Laya backbone GGUF loaded
  headPath: 'laya-head-tetris.safetensors',
);

Training

• Base: the head of convaiinnovations/laya at 1c5edc17, with the encoder frozen. Only head.*, type_emb.* and scorer.* were trained; the act head and the temperatures are unchanged.
• Data: 16,000 training and 2,000 validation choice questions from bin/make_dataset.dart in the example. Each question asks "Which placement of the falling Tetris piece is best?" and lists 2 to 6 candidate placements with their lines cleared, new holes, stack height and bumpiness. Targets are softmax(h / 0.3) over the example heuristic's value for each placement.
• Recipe: 12 epochs; AdamW at lr 3e-4, weight decay 0.01; 100 warmup steps, then cosine decay; batch 32. The best validation epoch (12) was kept. This is the default run of the example's fine-tuning notebook, proposed in llamadart#619.

Results

On the 2,000 validation questions: accuracy counts a pick among the heuristic-best placements as correct; regret is the heuristic value lost against the best placement.

Head	Accuracy	Regret
Uniform random pick	0.275
Laya base head	0.305	1.621
This head, PyTorch F32 encoder	0.757	0.234
This head, llamadart with laya-Q8_0.gguf on Metal	0.760	0.236

Turn-based games in the example's bin/bench.dart: laya-Q8_0.gguf on Metal, Apple M4 Max, mean over 5 seeds, capped at 150 pieces.

Player	Candidates	Pieces	Lines	Questions per piece
This head, one choice question	3 heuristic-best + 3 random	150	48	1
This head, knockout rounds of choice questions	all legal placements	135	40	4.8
Base head, yes/no checklist	3 heuristic-best + 3 random	120	35	11.9
Base head, one choice question	3 heuristic-best + 3 random	33	1	1
Heuristic alone	all legal placements	150	56

With one choice question, every game reached the 150-piece cap.

Revisions

• 83794e0bdd5526420997279a5d3dc9810a445217: the first head, trained for 8 epochs. Validation accuracy 0.750 (PyTorch) and 0.747 (llamadart); 122 pieces and 34 lines with one choice question, 134 and 40 with knockout rounds.

Limitations

• Trained only on the question format above. Other questions, including general Laya decisions, have not been evaluated with this head.
• English only, like the base model.

License

Apache-2.0, like the base model. This is a modified version of the convaiinnovations/laya head: the weights listed under Training were fine-tuned on the Tetris data.