Commit History
-
add KG empirical-feedback loop + per-competition robustness column Running
Pegah +1: Claude Opus 4.7 (1M context)BM-16 — Empirical-agentic feedback loop: - New savant.knowledge_graph.empirical: attach_benchmark_results() walks Track-B artefacts and attaches per-method empirical_variants dicts to the KG's Method nodes (T1 Brier/AUC, T2 log-loss, T3 ρ with 95% CI, T4 Δρ + provenance). - Multi-variant handling: singh_2018_xt keeps both xt_state_singh and xt_action_sumpter under one Method node. - scripts/enrich_kg_with_benchmark.py: standalone CLI that rebuilds the KG JSON + GraphML + pyvis HTML with empirical enrichment baked in. - 8/24 corpus Method nodes now carry measured metrics (all Track-B papers). Gap-agent and critic-agent prompts updated to reference empirical_results in their input contract. - build_kg.save_kg() now JSON-encodes nested containers before GraphML write so Gephi/yEd don't choke on list-valued attributes. - 11 new tests covering wrapper→paper_id mapping, single/multi-variant attachment, no-op on empty bench dir, gap-agent input surfacing, GraphML round-tri…50e29fd -
add Track-B empirical benchmark (9 methods × 4 tasks on StatsBomb Open + 360)
Pegah +1: Claude Opus 4.7 (1M context)- 9 corpus-derived methods: xt_state_singh, xt_action_sumpter, vaep_socceraction, daxt_merhej, vanroy_mdp, vanroy_leaving_goals, playerank_pappalardo, obso_ff_spearman, liu_2020_ttdp. Each cites a paper in extractions/. - 3 trivial baselines: goals_per_90, assists_per_90, xg_per_shot. - 4 evaluation tasks: T1 (per-action goal prediction, Brier/log-loss/ECE/AUC), T2 (multiclass possession outcome), T3 (team-match xG differential, Pearson/Spearman/MAE), T4 (cross-competition Δ on WSL + AFCON holdout). - Match-level 80/10/10 splits + fail-fast leakage audit. - Match-level bootstrap 95% CIs (n=1000) + Wilcoxon signed-rank pairwise with Benjamini-Hochberg correction. - By-competition orchestrator (scripts/run_benchmark_by_competition.py) so large runs don't OOM. - Full-corpus results: 660 train matches + 183 holdout matches. - Streamlit UI: 9_🏁_Leaderboard page renders T3/T1/T2/T4 with CIs, Wilcoxon heatmap, small-sample warning, empty-state fallback. - 186 fast tests coverin…4b35d65