Spaces:
Sleeping
Sleeping
Commit History
-
Rewrite notebook: per-piece GRPO training on Qwen 3B
Complete rewrite of training approach: - Model sees board before EVERY piece (not 100 blind actions) - Generates up to 20 action tokens per piece, stops when piece locks - Custom REINFORCE/GRPO loop (no GRPOTrainer dependency) - Two-phase: rollout without grad, then recompute log_probs with grad - 8 games per iteration (same seed, GRPO-style comparison) - 100 iterations with different seeds Model changes: - Qwen2.5-3B-Instruct (was 7B) - LoRA r=16 (was r=32) - Target GPU: L4 (was A100) Reward structure: - Engine per-step rewards summed across entire game - L/R penalty: -0.1 per move - Piece placed bonus: +1.0 - No-place penalty: -10.0 (forced drop after 20 tokens) - Line clear bonus: +100 per line Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>b61d866
