Commit History
-
add KG empirical-feedback loop + per-competition robustness column Running
Pegah +1: Claude Opus 4.7 (1M context)BM-16 — Empirical-agentic feedback loop: - New savant.knowledge_graph.empirical: attach_benchmark_results() walks Track-B artefacts and attaches per-method empirical_variants dicts to the KG's Method nodes (T1 Brier/AUC, T2 log-loss, T3 ρ with 95% CI, T4 Δρ + provenance). - Multi-variant handling: singh_2018_xt keeps both xt_state_singh and xt_action_sumpter under one Method node. - scripts/enrich_kg_with_benchmark.py: standalone CLI that rebuilds the KG JSON + GraphML + pyvis HTML with empirical enrichment baked in. - 8/24 corpus Method nodes now carry measured metrics (all Track-B papers). Gap-agent and critic-agent prompts updated to reference empirical_results in their input contract. - build_kg.save_kg() now JSON-encodes nested containers before GraphML write so Gephi/yEd don't choke on list-valued attributes. - 11 new tests covering wrapper→paper_id mapping, single/multi-variant attachment, no-op on empty bench dir, gap-agent input surfacing, GraphML round-tri…50e29fd -
add Track-B empirical benchmark (9 methods × 4 tasks on StatsBomb Open + 360)
Pegah +1: Claude Opus 4.7 (1M context)- 9 corpus-derived methods: xt_state_singh, xt_action_sumpter, vaep_socceraction, daxt_merhej, vanroy_mdp, vanroy_leaving_goals, playerank_pappalardo, obso_ff_spearman, liu_2020_ttdp. Each cites a paper in extractions/. - 3 trivial baselines: goals_per_90, assists_per_90, xg_per_shot. - 4 evaluation tasks: T1 (per-action goal prediction, Brier/log-loss/ECE/AUC), T2 (multiclass possession outcome), T3 (team-match xG differential, Pearson/Spearman/MAE), T4 (cross-competition Δ on WSL + AFCON holdout). - Match-level 80/10/10 splits + fail-fast leakage audit. - Match-level bootstrap 95% CIs (n=1000) + Wilcoxon signed-rank pairwise with Benjamini-Hochberg correction. - By-competition orchestrator (scripts/run_benchmark_by_competition.py) so large runs don't OOM. - Full-corpus results: 660 train matches + 183 holdout matches. - Streamlit UI: 9_🏁_Leaderboard page renders T3/T1/T2/T4 with CIs, Wilcoxon heatmap, small-sample warning, empty-state fallback. - 186 fast tests coverin…4b35d65
-
chore: initial commit — SAVANT corpus, KG, agentic pipeline, 9-page UI
Pegah +1: Claude Opus 4.7 (1M context)The initial commit captures the full SAVANT system as of 2026-06-30: - Stage 0/1 scaffolding: configs/, prompts/, src/savant/ package, tests - Phase-1 corpus: 28 papers (24 candidate + 4 held-out), schema v0.3.0 (52 fields per record, action_coverage + practical block + relationships) - Track A: knowledge graph (165 nodes / 476 edges), coverage matrices, ranked gaps (62 actionable), top-5 hand-written rationale, 3 manuscript figures, pyvis interactive HTML, CLI orchestrator - WP 2: agentic loop — extraction / critic / taxonomy / gap-agent runners with shared AgentRunner base, AnthropicClient + MockClient, validation hooks, JSON-mode parsing, retry policy - 9-page Streamlit UI (Home, Corpus, KG, Coverage, Gaps, Agentic Pipeline, Live Demo with 4 modes, Chat with citation-grounded answers, About) with heavy custom CSS theme + HF Spaces deploy plumbing (streamlit_app.py + README_HF.md + docs/deploy_hf_spaces.md) Tests: 92 passing across schema, paper metadata, KG, gap an…c63cf78