Commit History
-
[NOTICKET] planner: group-labelled recipes + the feasibility gate the eval caught (A18)
Rifqi Hafizuddin +1: Claude Opus 5Recipes, part 2 of A18. The table goes R1-R6 -> 18 rows, each labelled with its GROUP so the tool list (organised by group, A17) and the rules (organised by recipe) stop being two overlapping schemes the planner has to reconcile. New: R7 data-quality, R8 forecast, R9 importance, R10 pareto, R11 segmentation, R12 clustering, R13 A/B test, R14 anomaly, R15 method-infeasible. Plus a "telling the close pairs apart" block for the six intra-group confusions the group hint cannot separate -- trend vs forecast, correlation vs importance, importance vs pareto, segmentation vs clustering, A/B vs aggregate, anomaly vs descriptive. R15 is not a softer R6: R6 is "the column does not exist", R15 is "the column exists but no tool does that". Numbering note -- the pre-implementation spec called the method-infeasible recipe R12; the implemented table gives R12 to clustering and R15 to method-infeasible. Stale references in DEV_PLAN, examples.py and groups.py corrected rather than renumbering the live …b56fb4c
-
[NOTICKET] planner: rebalance few-shot exemplars EN/ID, demo the reply-language rule
Rifqi Hafizuddin +1: Claude Opus 5A19. The exemplar set was 13/13 English while the eval dataset -- and real traffic -- are majority Indonesian (17 ID / 14 EN). In-context learning over-predicts whatever the examples show most (majority-label bias, Zhao et al. ICML 2021), so an all-English set teaches the planner that Indonesian questions are out-of-distribution. Converted four questions to Indonesian (now 9 EN / 4 ID), chosen so each conversion also breaks a near-duplicate: D breaks the 3x check_data -> retrieve_data -> analyze_aggregate stack (A/D/E) M stops reading as a near-duplicate of J's English bar-chart ask L R2b "berapa banyak ..." is the recipe most often asked in Indonesian H the infeasible example -- see below H now demonstrates the reply-language rule: an Indonesian question gets an Indonesian infeasible_reason while every other field stays English. prompt.py:151 has always mandated this ("If (and only if) you return an infeasible plan, write infeasible_reason in {reply_language}") but NO e…fb30aec -
[NOTICKET] eval(planner): skip cases whose tool is withheld, don't fail them
Rifqi Hafizuddin +1: Claude Opus 5A16. Moving a tool to DEFERRED_ANALYTICS_TOOLS turns its eval cases red, and that noise hides a real planner regression in the same run. Those cases are now reported [SKIP] and leave the denominator, so a score stays comparable to a run taken when a different set of tools was active. The requirement is DERIVED from each case's own assertions rather than a hand-kept field, so the dataset stays self-maintaining: has_tool: X -> skipped when X is not planner-visible any_tool: [...] -> skipped only when every listed tool is withheld no_tool: X -> asserts absence, never a reason to skip Cases stay in planner_dataset.json and reactivate the moment the tool returns to ACTIVE. The skip is checked before the LLM call, so a withheld tool costs no tokens. --rescore carries a saved run's skips through unchanged. Verified without spending a run: - 0 skips against today's 10-tool registry (a no-op until A15 lands) - deferring analyze_correlation skips exactly corr_mttr_pa_id + c…845284b
-
/feat knowledge management (#20)
- [NOTICKET] docs: knowledge pipeline test integration — context, audited task checklist, DEV_PLAN §0.8 (8e69dc6c638509f4dd7b6775d520fb4aa4efbd3e) - [NOTICKET] docs: knowledge pipeline — reframe kex as prototype, not port target (7cce5ffc9ab88791ac139dbef8e88e124a3d049e) - [NOTICKET] feat(eval): knowledge-extraction gold set, scorer + frozen prototype baseline (60cf4a885ef31a8aa64916f5a70ac5a0e8062c5f) - [NOTICKET] chore: add missing planner and readiness eval results (bd6659b3bf523597e4482b04ab1fcff5e1284d25) - [NOTICKET] feat(knowledge_extraction): free stages — models, seam adapter, filters, cluster, ranking (024c30abd1189119d33508eded334fe5a3ed5011) - [NOTICKET] fix(knowledge_extraction): move runner into the package; fix two ranking defects (72605ff6876a7936ab685d417c766ce820f6214b) - [NOTICKET] feat(knowledge_extraction): span filter — E1 recall 0.8537, matches the frozen baseline (3a016343398753cf730d50548c0e92ff47c6716c) - [NOTICKET] feat(knowledge_extraction): paid extraction …b68816f
-
/fix planner count and report (#19)
- /fix planner count question (11de970ff65be9b4f81f773cfb355d3dc57358d7) - [noticket] fix report bug where some answered analysis isnt included in report summary answers (8dec1825cf803ad28f4ff0400b9c373250a7d90a) - [NOTICKET] fix: split report floor from report body; scope analysis reads by user_id (fd4865b20dfc46782706be9aab318f909b9903c3) - [NOTICKET] docs: sync report floor/body + security posture; add review remediation tracker (fc5200c994441ec505ea7a82469da577afe170a5) - [NOTICKET] test: planner eval harness (rule-compliance, --selfcheck/--rescore) (4292522aa3fa3a7af8f933307334cb7e5eb0b377) - [NOTICKET] eval: repair readiness harness; commit intent/help/readiness baselines (08a0a4ad5eacc6c90df22fab5be69f2d33579f29) - [NOTICKET] fix: compiler parity — F-17 fabricated column, F-18 LIKE nulls, F-25 empty in/not_in (8f736f90f1958960a83847d332b45b13fa346168) - [NOTICKET] feat: bound catalog render + blob read; scope charts/traceability; mark degraded seams (0cb7d53cb6f7f7e7b687dd6d9c27…f282b15