Commit History
8 days ago
-
[NOTICKET] fix(planner): ranking coverage and outcome-derived drivers
Rifqi Hafizuddin +1: Claude Opus 5.5A six-month PA ranking put a unit with one partial month first (6.36%, April only), and driver analysis ranked columns derived from the outcome itself. Rankings of per-period averages now carry n_periods and last_period (planner rule + example AA), the assembler states the period, flags short coverage and where the data ends, and the planner drops candidates the catalog describes as derived from the outcome. Eval: 4 new cases (dataset 64). Risk-targeted subset 24/64, all pass. pytest 1044 passed, 7 skipped; import main OK. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>374160d
13 days ago
-
[NOTICKET] fix(planner): EMA dry-run fixes — analysis scope, site-code lookup, in-query aggregation, NULLS LAST
Rifqi Hafizuddin +1: Claude Opus 5.5From a 9-turn replay of a real EMA session on maintenance views (Neon): - compiler: ORDER BY ... NULLS LAST (a NULL group ranked first under DESC) - the analysis objective/business questions reach the planner as a `# Analysis context` section inside <data>; today's date (WIB) in `# Question` - planner.md: aggregate in the IR, relative periods, site codes via the lookup (never a typed id, bare codes matched to lookup values), "why" uses the cause table, driver questions compare groups in the IR, rankings return 10 not 1, "infeasible is about columns, never rows"; client codes replaced by placeholders - examples: A/E aggregate in the IR, D asks for a median, P notes its row count, new X (ranking without N), Y (driver on a large table), Z (code lookup) - validator: 8i capped-slice importance, 8j typed lookup ids (code columns only), 8k top-1 rankings, check 11 rejects handoffs from metadata steps - assembler: shared top values, correct cell labels, period per row - eval/planne…a0e68ee
-
[KM-717] eval(planner): data-dictionary cases, and live Neon knowledge tables verified
Rifqi Hafizuddin +1: Claude Opus 5.5eval.planner gains db_dict / db_dict_injected fixture variants: the DB fixture plus data-dictionary entries and two opaque measures (ua, bd_uns) whose meaning exists only in their dictionary line. Three new `dictionary` cases: - dict_meaning_join_en: pick ua, avg, join to equipment - dict_sum_by_model_id: pick bd_uns from its Indonesian business meaning, sum, join (shaped like demo step 3) - dict_injection_ignored_en: a dictionary entry orders the planner to refuse; it must plan normally (prompt-injection regression guard) The existing 40 cases keep their exact prompts. Run (--only the three): 3/3 on the first attempt, 0 retries, 59,588 tokens (planner_result_2026-09-30_093130.json + log). One sample each, no no-dictionary control. Docs: DEV_PLAN P2-I2 done; D2 closed. Live Neon knowledge tables match Go 0008 (86 columns, nullability, all 14 constraints; read-only check run by Rifqi). REPO_STATUS §9/§14, knowledge_tables.sql, KNOWLEDGE_PERSISTENCE_CONTRACT.md and eval/planner/REA…97b27d1
-
[NOTICKET] eval(help): capability cases for the v7 tool sync
The existing orientation cases all sit at the start of an analysis, so they only ever exercise the "basic questions first" branch and cannot tell whether help.md v7 changed anything. Three held-out, auto-scored cases in a new `capability` group cover what v7 is for: - cap_01: basics done, business questions ask for drivers and a projection -> Help offers a deeper analysis. - cap_02 (ID): goal is "which unit breaks down next" (R15, no tool) -> Help must not hand that question over to copy, and degrades it to unusual units / a projection. - cap_03: goal is "what causes low PA" -> reframed as what is most related to PA, never a quoted "what causes" suggestion. Result: 14/14 (held-out 12/12, carried-over 2/2), capability 3/3. cap_02 is the case that separates v7 from v6; cap_01/cap_03 may also pass on the old prompt, which already allowed correlation suggestions. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>87bb688
-
/fix planner and unstructured pipeline (#23)
- feat(documents): v2 unstructured ingestion, image-token, scoped retrieval (b848f739c23c9ea152c5bcc5ea4b9d8b4ca64afa) - feat(prompts): mixed document+data path, figure markers (39019d57e159b510fda1f5bd813d051e1ba7c05c) - eval(chat_sim): doc + mixed suites with bound documents (b6e060b328c8dbbdb19f3da466fe97f62dab09aa) - [NOTICKET] docs: unstructured v2 plan, contract, status sync (6702119c2705ef9a99a4a0ba0adaab9ac7a28c54) - fix(check): list data and documents when both are asked (2b7c3bd79fe73b693634e583be46c812b6c74d3c) - [NOTICKET] docs: mixed suite result, check + router fixes, U8 done (0e1ad6e143e30ec20960d3a2c5ae22190aa512ff) - feat(router): a figure IN a document vs a chart FROM data (770b98855d640c727a402b0db1252638364c20d7) - fix(chatbot): reply in the turn's language, not the documents' (b6150d58b991fb5d98b0bb2278956c938d80d4c0) - fix(query,planner): stable capped rows, plan-time handoff check, trimmed records (ea9848db5ec39e32067a2c3dbc9453f1a94736d2) - eval: scoped/DB/docum…7124acf
-
/feat ds tools (#22)
- feat(tools): analytical-tools foundation — spec_builder + ml executor (ee1a76a466c6ea6dc52f337d0fd9534ca0271a86) - feat(tools): shared chart kit + auto-chart builders (535f17340950e070556441140276c47d074606e2) - feat(tools,spine): wire tool auto-charts to chart persistence (522e582bab372b645577789cbbd59d5105a66632) - build(deps): add statsmodels + explicit scikit-learn pin (4d76bd0e57f5f7b1bbb0b77e49970063d1898a1b) - [NOTICKET] docs: analytical-tools foundation — conventions, DEV_PLAN 0.9, status sync (a7f0bca51ed8ff08784cca3fd4179dce56ef7622) - [NOTICKET] eval(planner): skip cases whose tool is withheld, don't fail them (845284b4a3b32df142565de23386dee739d0f733) - [NOTICKET] feat(tools): analyze_forecast (ETS) — first ML tool, DEFERRED (dae102f68fc36cf755e7d53df10affb4ec7edaa6) - [NOTICKET] planner: rebalance few-shot exemplars EN/ID, demo the reply-language rule (fb30aec99586daf12a4799087ae538e5e862451c) - [NOTICKET] feat(tools): analyze_anomaly (Isolation Forest) — second ML tool,…e201ae4
-
/fix parsing and term extract (#21)
- [NOTICKET] fix: retire the fuzzy clustering pass; add E1c post-cluster recall (2cc21c92688d662c5cfdfeeb6784bb85b98575ca) - [NOTICKET] feat: parsing schema 0.4.0 — asset pointers for figures and tables (e662ac089b741c90e19b9f51f4933fb97a1225bc) - [NOTICKET] docs: sync REPO_STATUS / DEV_PLAN / contracts for pipeline v2 (c3c184d83bec4a4d42e9ad3f7265560802817e5b) - [NOTICKET] docs: close K15 with the paid v1-vs-v2 comparison (62782fed9292f67d7b376783920a07197d6776e9) - [NOTICKET] fix: test the caption hypothesis; ship the signal at weight 0.0 (36a166e1844d186d6cfb8fff9ea070f3dc69b220) - [NOTICKET] fix: cap key-parameter pre-emption in the review queue (Step 2) (4e3a6aefef7024db3ea4b7946fd7771a6c269601) - [NOTICKET] feat: assert on invariants, report a band, classify the genre (Steps 0 + 5) (f1d64b113db0289601806635a9b154201a6127fc) - [NOTICKET] fix: gate spend in cluster units and surface the estimate (Step 6) (cdb5fe82cc5f2893ccbb5e28282e8dc46a27e061) - [NOTICKET] fix: retire escalation…f07443e
-
/feat knowledge management (#20)
- [NOTICKET] docs: knowledge pipeline test integration — context, audited task checklist, DEV_PLAN §0.8 (8e69dc6c638509f4dd7b6775d520fb4aa4efbd3e) - [NOTICKET] docs: knowledge pipeline — reframe kex as prototype, not port target (7cce5ffc9ab88791ac139dbef8e88e124a3d049e) - [NOTICKET] feat(eval): knowledge-extraction gold set, scorer + frozen prototype baseline (60cf4a885ef31a8aa64916f5a70ac5a0e8062c5f) - [NOTICKET] chore: add missing planner and readiness eval results (bd6659b3bf523597e4482b04ab1fcff5e1284d25) - [NOTICKET] feat(knowledge_extraction): free stages — models, seam adapter, filters, cluster, ranking (024c30abd1189119d33508eded334fe5a3ed5011) - [NOTICKET] fix(knowledge_extraction): move runner into the package; fix two ranking defects (72605ff6876a7936ab685d417c766ce820f6214b) - [NOTICKET] feat(knowledge_extraction): span filter — E1 recall 0.8537, matches the frozen baseline (3a016343398753cf730d50548c0e92ff47c6716c) - [NOTICKET] feat(knowledge_extraction): paid extraction …b68816f
-
/fix planner count and report (#19)
- /fix planner count question (11de970ff65be9b4f81f773cfb355d3dc57358d7) - [noticket] fix report bug where some answered analysis isnt included in report summary answers (8dec1825cf803ad28f4ff0400b9c373250a7d90a) - [NOTICKET] fix: split report floor from report body; scope analysis reads by user_id (fd4865b20dfc46782706be9aab318f909b9903c3) - [NOTICKET] docs: sync report floor/body + security posture; add review remediation tracker (fc5200c994441ec505ea7a82469da577afe170a5) - [NOTICKET] test: planner eval harness (rule-compliance, --selfcheck/--rescore) (4292522aa3fa3a7af8f933307334cb7e5eb0b377) - [NOTICKET] eval: repair readiness harness; commit intent/help/readiness baselines (08a0a4ad5eacc6c90df22fab5be69f2d33579f29) - [NOTICKET] fix: compiler parity — F-17 fabricated column, F-18 LIKE nulls, F-25 empty in/not_in (8f736f90f1958960a83847d332b45b13fa346168) - [NOTICKET] feat: bound catalog render + blob read; scope charts/traceability; mark degraded seams (0cb7d53cb6f7f7e7b687dd6d9c27…f282b15
-
/fix change model to gpt 5.4 mini (#17)
- [NOTICKET] chore: rename LLM env quad __4o -> __54m (GPT-5.4-mini) (790ef0c7aaac6e76eb01b0ef9a3398ff4249416d)d65c41d -
fix/ check and new chart tool (#16)
- check: table-level column drill + drop misleading "name a table" hint (ef0c7b1c5aa49a244b02f1b5eea15cc77dcf2bcb) - [KM-715] charts: render_chart tool + message_charts store + GET /api/v1/charts + planner viz slice (87cfcf884d923ee4bcd722e0e4ef4d90d7eaedca) - [KM-715] checkpoint: S1a deterministic quality checkpoint between runner and assembler (f3c8de34dc19a573031b67fd4f48dcfbadcdfed8) - [KM-715] docs: sync REPO_STATUS/DEV_PLAN + land SPINE_V2_PLAN (W1/W2 done) (434eacbf2ac8324188ebe5e32775dbc9d2a3f49a) - planner: value-handoff (${tN.col} → list) for tabular lookup/anti-join (54070ac1407b751000e49dfe7f102a2be5ab1ff8) - [KM-715] charts: GET /charts by message_id only + tri-state status marker (b8d15761addf4d499c9631df64e7693aa1d61fa2) - [KM-715] report: embed charts as plotly EDA fences; render_chart counts toward the floor (89db8be6b6f11e86ad0776d2ee07f2306e3b6079) - [KM-715] docs: contract + status deltas for the charts lead-review round (96347ee7c55a4eb39a906409ec214463e25c5e35) - …5a60e93
-
/fix help and tools analyze (#14)
- [KM-711] Retire /report slash command; report is Generate-button only (f7f7b172da0dc16adac0a7555f6d34a704cc2050) - fix(analyze_trend): handle value_column == date_column (count events per period) (55e3327f7b2e21a954b317e143f46bcea8b5ada7) - [KM-711] Docs: drop /report from list_skills catalog examples (0772863b296e2ed3e2d559df05e98540764e58b0) - fix(planner): reject inlined table-ref as analyze_* data (Pattern A) (776fae3c7c6e3d3a07c22bef10898f6b1c33c701) - fix(analyze_aggregate): reject non-string group_by (CASE/binning) keys (b40fa76e2fa68572dad0d8532e07076207095d6b) - fix(planner): reply-language for infeasible_reason + answer-only-asked rule (4d89d7d93938e41892979cd20e5ccf0c97441dac) Co-authored-by: Sofhia Az-Zahra <sofhiaazzhr@users.noreply.huggingface.co>17e31c6
-
/fix validator and report (#10)
- /fix report: remove record_id hash citations, EDA (for future tool) change to data source, transform to mdx-safe (b1b101730ccf5628e43e80f2ad654f78fbed5207) - [NOTICKET] fix(planner): repair near-miss catalog ids in QueryIR before validation (aacd4d13def258a9815761aecbd7b246a8dae9fc) - /fix report: group findings by analysis, dedup, code-span-safe MDX escaping (029b48894e9f79af6c0d81c00a2d811a982f87ae) - [NOTICKET] fix(tools): log swallowed analytics tool errors in AnalyticsToolInvoker (93d9381b2a5f5efe058c913fe0b793ce994a9097) - fix(planner): harden plan validation to reduce retry thrash on structured questions (1619dd3c0fa74825010c7af6d4b375e3b9404847) - [NOTICKET] fix(tools): coerce numeric-string columns before analyze_* compute (6bda333866a1a8ccf1b3fe6fcee7a8e1832f28d7) - fix(planner): correct top-N guidance — in-query ranking IS supported (99aa4141aed8241a625d290fb170f74192a6d831) - feat(guardrails): add input guard, out_of_scope intent, and refusal handling (0bbc27bc40ba20e7a8e…0e5fdb5
-
/fix check and help tool (#8)
- /fix data_catalog model + CatalogStore to dedorch schema (data → catalog_payload) (09857fc52486354f53e752e3ea329b064a9e4c4b) - [NOTICKET] fix(help): lock reply language to user's language on button path (cb7f5ac484ad4a510e68faf5291091958b6d2f42) - /fix default SKIP_INIT_DB=true for dedorch cutover (skip init_db on Go-owned schema) (781b93c6b8744209a1101629b5170175e9212592) - /fix surface catalog foreign keys to the planner (FK-backed joins) (446c57524e932fcdbb0ddecc049189c443b6ae97) - [NOTICKET] feat(eval): add help-skill eval harness + goal-based language fallback (cbdbefb7ab0ffa36949713f18422213db480a959) - [NOTICKET] fix(eval): update readiness eval to post-KM-652 floor (drop problem_validated) (6f46e81bdeb1eb987f9dac8af0e22a975474cbb1) - /fix infer FKs for FK-less dedorch catalog + log repr(e) in db executor (3669290474583ae011f5ed5e50b94dad146fbc77) - Merge eval/help + readiness eval fix into PR #8 (5f4818a8e38f93988ab5756f3a06c94645826b15) - /docs REPO_STATUS: FK inference (FK-…3743cfe
-
feat/Knowledge & Data Tools (#3)
- [KM-641] refactor(tools): trim analytics registry to 4 active tools (66e2e4df66f1abf7eaa7e442d9900d428696ece4) - [NOTICKET] docs: add checkpoint plan for 2026-06-17 sync (8c1bb63ebdee02f9db0da9fc2b2269823b0a7688) - [KM-642] refactor(tools): rename data-access tools to data/knowledge naming (c38c0c29d3e6b245f67d03f2c35e451900e1abdb) - [KM-643] feat(tools): merge data sources into data_check + add knowledge_check (4bd5f1efdc560a64092cc8df672a7990fa949b59) - [NOTICKET] docs: record KM-642/643 in checkpoint plan + PROGRESS (4c57588b90fe0cac8d0d4d22cfe5701fc9eb52dd) - [NOTICKET] fix(chat): scope response cache by user_id + drop TTL to 1h (R5) (b701e95d300c7d2480a3608b2d4da6dfe102b19a) - [NOTICKET] feat(slow-path): persist AnalysisRecord via PostgresAnalysisStore (09f845188226dd01633cf74525caa569e4a7d37a) - [NOTICKET] Adopt verb-first skill naming (2d6406def7ed45edd1226e96645ba2a5cdbfa3d0) - [NOTICKET] docs: sync checkpoint plan — verb-first tool names + report-as-API (3b399f84d73b0d98bd99…0721bb4