-
Restructure to src/ layout with attention, per-layer MoE, and working chat
geoore +1: Claude Sonnet 4.6 - Add GQA + RoPE attention (AttentionLayer) using config's n_heads/n_kv_heads/head_dim
- Wire 28 transformer layers (attention + shared MoE FFN per layer) in forward()
- Fix expert dispatch weight bug (per-token weights, not flattened scalar)
- Fix chat() stub: tokenize with GPT-2, generate, decode new tokens only
- Fix GRPO reward: remove proxy metrics, correctness + CoT bonus only
- Add load balance loss to GRPO train_step to prevent expert collapse
- Fix JSONLibrary recall() write-back and add 1000-entry cap per category
- Add tests/test_core.py with 11 tests covering all core components
- Add .gitignore excluding venv/, outputs/, data/, .vscode/, .claude/
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>