- The RANKING paragraph (#10) read as "any ranking goes here"; on the
relative_last6_months_id eval case 5.4 mini once pulled raw rows and ranked them with
analyze_aggregate instead of grouping in the query (Rifqi, audit A1-F7)
- Now: only a median ranking sets sort_by/top_n here; a sum/avg/count/min/max ranking
is a grouped retrieve_data with order_by + limit, matching planner.md:107
- sort_by example changed from is_bad_actor_sum to pa_median
- Median rankings (units by PA, sections by MTTR): analyze_aggregate + sort_by + top_n
6/6, same as before; 6 avg/count ranking eval cases 6/6 via retrieve_data;
relative_last6_months_id 3/3 via retrieve_data
- Full suite 403 passed, 6 pre-existing failures
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
[NOTICKET] fix(validator): reject an aggregate alias that names another column (#16)
- Test demo #16 (chat 2 Q6, Q8): `AVG(mtbf) AS avg_mtbs` passed, and the answer
reported MTBF as MTBS; only Traceability showed it
- _reject_misleading_aliases only caught an alias equal to another column's name
- New _alias_stem(): strip one aggregate prefix/suffix (avg_, sum_, total_, min_, max_,
_avg, _mean, ...) and reject when the rest names a different column; retry follows
- Replay of 346 stored retrieve_data IRs (all analyses): only Q6, Q8 and the old
`ma AS backlog` are rejected, no new false rejects
- Local tests; full suite 403 passed, 6 pre-existing failures
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
[NOTICKET] feat(report): trend evidence shows rows per period and stub periods
Rifqi Hafizuddin+1: Claude Opus 5.5
493b0b8
When analyze_trend reports per-point `n` / `stub` (P0-8), the evidence table
reads period · value · n · note ("few rows" on a stub); older records keep
period · value.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
[NOTICKET] feat(assembler): stub periods and weak results are worded as such (P0-11)
Rifqi Hafizuddin+1: Claude Opus 5.5
632caf8
Sofhia's local run (analysis a67625f2) after P0-8: the trend answer opened
with "puncak pada September" on a 9-row stub month, and a weak correlation
(n=4) opened with "Ya" without p. One assembler.md bullet: a stub period is
never the peak, the lowest point or where a trend ends; a weak result does not
open with a plain yes/no and gives n and p when present. Prompt change: a
small chat check is still to run (tracker).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
[NOTICKET] fix(query): GROUP BY without an aggregate is DISTINCT on uploaded files (P0-10)
Rifqi Hafizuddin+1: Claude Opus 5.5
d20cd21
The pandas compiler applied group_by only with an aggregate, so
`SELECT site_code GROUP BY site_code LIMIT 50` on a tabular source returned
50 raw rows. The scope value index for uploaded files therefore held only the
first rows' values (analysis da688944: site_code ['LAT','BIN','SDJ'] of 7;
section missing OB LOADER), marked complete, and the objective's site ADT was
never pinned — Sofhia's "ADT lost" turns. Now drop_duplicates() after the
select, matching SQL; the rendered query shows it. Same turn's value index now
loads all 7 sites and the objective resolves to site_code ADT + OB HAULER.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- P0-8 row set to done with the three commit ids (f28898b0, 50cae402, 1ac2267a)
- Evidence: what changed, the IPR OB LOADER example, test counts
- Notes it is not yet checked through a chat and the report trend table has no n/stub
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
[NOTICKET] feat(correlation): pairwise n, p-value and a quality block (P0-8)
- Each pair now carries n (rows complete for both columns), p_value (scipy
pearsonr / spearmanr / kendalltau) and weak (n < 10 or p >= 0.05)
- Reserved `quality` block for the strongest pair, so CK7 surfaces a weak correlation;
a caveat says one point can drive r on few rows, or that a large-n r is not significant
- With no column_ids, `n_*` row-count columns (the repair's n_periods) are skipped
- ADT OB HAULER monthly means: PA-MTTR r=-0.42 now reads n=9, p=0.26, weak; unit-month
rows (n=2,379) stay strong; a ranking table no longer correlates n_periods
- Reported by Rifqi (audit P0-8, decision D4)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
[NOTICKET] feat(trend,forecast): row count per period and stub flag (P0-8)
- Each trend point and forecast history point now carries `n` (rows) and `stub`
- A stub has fewer rows than half the median non-empty period (needs >= 3 periods);
empty periods are not stubs
- analyze_trend leaves stubs out of first/last, direction, slope and R^2, with a caveat
naming them; it falls back to all periods if every one is a stub
- analyze_forecast still fits every period but caveats a thin last actual period
- IPR OB LOADER (Sep 9 rows vs 63-66): sum(mtbs) "down" -> "up" (R^2 0.01 -> 0.91);
mean(pa) last 99.95 (Sep) -> 92.45 (Jul)
- Reported by Rifqi (audit P0-8, decision D4)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
[NOTICKET] fix(trend,forecast): a period with no rows stays empty (P0-8)
- resample().agg("sum") turned a month with no rows into 0: the trend read it as a
collapse and the forecast fitted it as a real value
- New temporal._resample(): sum uses min_count=1, so an empty period is None; count
keeps 0 (zero events is the true count); other aggs were already None
- analyze_forecast uses the same helper, so the empty month is interpolated for the fit
- Example (April empty): trend R^2 0.06 -> 0.90, forecast no longer dips after "April 0"
- Chart already draws None as a gap; the report table shows it as a dash
- Reported by Rifqi (audit P0-8, decision D4)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
[NOTICKET] fix(planner): a failed plan is not a data gap (P0-9)
Rifqi Hafizuddin+1: Claude Opus 5.5
863bf8e
TaskList._exhausted (private attr, not in the model's schema) marks a plan
that failed validation on every attempt; the coordinator shows its own text
without the data-gap lead. New EN/ID texts (register "Anda"): the plan failed
its checks, which does not mean the data is missing; ask more specifically.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
[NOTICKET] feat(report): cap answers on a weak model fit, not on a non-significant test (P0-7)
Rifqi Hafizuddin+1: Claude Opus 5.5
5595f0e
anchored._weak_fit: when analyze_importance / cluster / forecast / anomaly mark
their own quality weak, the answer is at most "partially answered" with the
fit in the reason; a weak primary sorts behind a clean one; limitation line.
analyze_ab_test is excluded (weak there = "not significant", a valid answer).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
[NOTICKET] feat(report): summary written after the per-question answers (P0-6)
Rifqi Hafizuddin+1: Claude Opus 5.5
d8c4759
ReportSummaryNarrative field order = generation order: bq_answers → title →
executive_summary → next_analyses; report_summary.md: the summary restates
the answers only. Analysis 2d841e2f's summary had contradicted its own Q4 band
counts. Prompt change: report fixture + 7-question gate pending (tracker).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
[NOTICKET] feat(scope): relative periods resolved against today and applied by code (P0-5)
Rifqi Hafizuddin+1: Claude Opus 5.5
621aed1
"6 bulan terakhir" ran as week 1–44 (the whole year); the planner.md rule was
prose only. period.relative_window (D7, WIB): "N bulan terakhir" = the N
complete months before today's month; bulan lalu/ini, tahun lalu/ini, kuartal
lalu/ini. Released to the planner: windows across two years, period
comparisons (YoY, "dibanding tahun lalu"), "N bulan lalu", open ranges, weekly.
A period comparison releases even when one side parses (review round 3).
Comparisons against a target or between sites keep the window. All-time wording
("selama ini", "seluruh data", "ever") releases. The scope hook takes `today`;
the planner eval passes each case's date. Scope line hides only `*_id` next to
its code column.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
[NOTICKET] eval(intent): follow-up cases with chat history
- run_eval.py sent every case without history, so follow-up routing was never tested
- New case_history(): an optional `history` (user/ai turns, oldest first) goes to the
router like the app's chat history; cases without it run as before
- 7 followup_* rows: analysis follow-ups -> structured_flow, "detailnya?" after a
document answer -> unstructured_flow, column/source questions stay check
- followup_01 uses the real chat-1 history before Test demo Q2 (a short made-up
history did not reproduce the miss); followup_02 is "sitenya mana aja?"
- Old router prompt 61/64 (both fail), new prompt 63/64; unstructured_04 predates this
- Dataset schema/_next_layer and README note the new `history` field
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
[NOTICKET] fix(router): short follow-ups take the route of the answer they follow
- Test demo Q2: "unitnya yg mana aja?" after a model ranking (and a chart turn) was
routed to `check` and listed the data sources instead of the units
- Reproduced with the real chat-1 history: `check` 3/3 runs before, `structured_flow`
3/3 after; "sitenya mana aja?" the same
- `check` definition: asking which rows/entities in the data match is not inventory
- New follow-up rule: a short follow-up keeps the previous answer's route (analysis ->
structured_flow, document -> unstructured_flow, source/column list -> check)
- One few-shot in another domain (sales branches), not the demo sentence
- Intent eval 56/57 before and after; the one miss (unstructured_04) predates this
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
[NOTICKET] docs(plans): tracker — P0-1b and P0-4 done, commit ids
Rifqi Hafizuddin+1: Claude Opus 5.5
67af228
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
[NOTICKET] fix(scope): lookup artefacts and typed site ids (P0-4)
Rifqi Hafizuddin+1: Claude Opus 5.5
effd6af
- only a `${tN.col}` value handoff marks tN as a lookup; a Pattern-A
`${t2}` data feed made the answering query a "lookup", so the line fell back
to the site lookup and read "(sebagian kueri)".
- _strip_broken_id_filters strips both FK ends: a typed
`vw_dim_site.site_id = 2009` (Check 8j rejected it, costing a retry).
- the scope line no longer prints `site_id 2009` next to `site_code IPR`.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
[NOTICKET] feat(scope): apply the user's own clean window by code too (P0-1b)
Rifqi Hafizuddin+1: Claude Opus 5.5
fc63682
Sofhia's analysis a67625f2: two turns typed "Juni–September 2026" and the
planner still ran `month 6–9` with no year, reading June–September of every
year. The P0-1 release ("the user named a period → planner keeps it") left
that unfixed.
period.turn_window: the user's own clean window → applied; months without a
year → the objective's year; an unparsed period of the user's ("6 bulan
terakhir", two years) → released to the planner; no period → the objective.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
[NOTICKET] feat(assembler): show what ran; label planner notes as pre-execution (P0-3)
Rifqi Hafizuddin+1: Claude Opus 5.5
04d356d
A planner note ("no site-code mapping for IPR") reached the answer and the
report under a scope line showing IPR applied; a "last 6 months" objective
was repeated over a query on weeks 1-44.
- render each retrieve_data as "(ran on <table> · <n> rows · filters: …)".
- carried open_questions labelled "Planner notes, written BEFORE the queries
ran (unverified …)"; assembler.md open_questions line matches.
- Prompt change: 7-question chat_sim gate still to run (tracker).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
[NOTICKET] feat(planner): rows of a per-period table are not entities (P0-2)
Rifqi Hafizuddin+1: Claude Opus 5.5
63e17f7
"equipment mana yang PA-nya paling rendah tahun 2026?" ranked raw month rows
(one 8-day April row read as the unit's 2026 PA).
- validator Check 8m: an ordered+limited ungrouped query on a year/month/
week/quarter table must group by the entity or select the period column.
- validator Check 8n: analyze_segmentation may not band raw period rows.
- IRRepairer: a grouped ranking over such a table gets COUNT(*) AS n_periods
(not when grouped by a period, not when it already counts).
- Replay of 135 stored planner-eval plans: 0 new rejects.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
[NOTICKET] feat(scope): the analysis objective's period is applied by code (P0-1)
Rifqi Hafizuddin+1: Claude Opus 5.5
5e5b0e3
Site/section were code-owned; the period rested on one planner.md line and
was dropped or invented on casual questions (Jun–Sep 2026 objective read
Jan–Sep; A/B test and correlation read every year; a planner typed 2025).
- scope/period.py: parse the objective window ("tahun 2026", "Jun–Sep 2026",
"Juni s/d September 2026"); no default for Q/semester/lists/"mulai Juni" or
4-digit ids ("site 2009"). Applied to every retrieve_data, replacing the
planner's filter on the same period columns; ISO weeks on week-only tables.
- Released when the user's own words name a period, when the plan forecasts,
or when the queries compare periods (by column name).
- TurnScope.periodic + "semua periode / all periods" in the scope line;
period filters render first and outside the other-filters cap.
- Never-throw; two independent review rounds (5 blockers fixed).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Tracks the P0 fixes from the 3 Nov prompt & tool-logic audit. P0-0 done: the
ranking regression (rank_units_worst_pa_*) passes 3/4 with the pre-71c6f2b2
planner.md + examples vs 3/8 with the current ones, so the 10-28 cut is the
likely cause; the fixture-only fix moves to P1.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
[NOTICKET] eval(chat_sim): ema_demo Q1 asks for the average PA
Rifqi Hafizuddin+1: Claude Opus 5.5
d0f8980
Without "rata-rata", GPT-5.4-mini answered "equipment mana yang PA-nya paling
rendah tahun 2026?" with one raw monthly row (EXHC36007 6.36% = April, 8 days).
With it, a one-turn check returned the averaged ranking with period counts
(EXKM12064 76.569%, 28 periods), matching analysis 2d841e2f.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- validator Check 8h looked `month` up by NAME across the catalog and took the
last one; on the EMA source that is a decimal column in another view, so a
month-without-year trend passed and failed at execution (Luna demo run
2026-11-03, Q2). 8h and 8l now share _upstream_column, the exact column the
upstream retrieve selected.
- analyze_ab_test: when the exact group match finds no rows, a group written
as text matches the same number and vice versa ("1"/"0" on an integer 0/1
flag raised GroupNotFoundError, Q5). Exact matches take the old path.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
[NOTICKET] feat(llm): LLM_PROFILE selector for the chat LLM quad (54m default, 6l trial)
Rifqi Hafizuddin+1: Claude Opus 5.5
268ef5d
- settings: azureai__*__6l quad + LLM_PROFILE (default "54m"); call sites read
settings.llm_* instead of the _54m fields. Fallback = one env var + restart.
- GPT-6 Luna on chat/completions refuses function tools (every
with_structured_output) unless reasoning_effort="none", and accepts
temperature 0 or 1 only: llm_reasoning_effort / llm_temperature() handle
both. For 54m they send exactly the previous request (reasoning_effort=None
is omitted, temperature unchanged).
- eval: intent/help runners label the active deployment; chat_sim gains the
ema_demo suite (the BMDP 7-question script) and Suite.business_questions.
- CONTEXT.md: profile note. The demo stays on 54m.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
[NOTICKET] fix(report): no default chart for descriptive evidence
- Since 0e64f6ba a descriptive answer's report evidence is its statistics
table (column, count, mean, median, std, min, max).
- analyze_descriptive draws no chart, so _default_chart built one from the
first measure, `count`: "Count by column" with identical bars (1080 x 4 on
the test-demo Q13 record).
- The other statistics mix units (PA in %, MTBS/MTTR in hours), so no single
bar chart reads right.
- _default_chart now returns None when the record ran analyze_descriptive;
the statistics table stays, rankings keep their default chart.
- Reported by Rifqi in the review of 3 Nov.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
[NOTICKET] feat(report): evidence tables for cluster, anomaly, descriptive, data-quality and forecast
Fixes from the 2026-11-02 demo test (analysis 2d841e2f):
- report: evidence table comes from the analysis tool's result (pareto,
segmentation, ab_test, trend, correlation, importance), not raw rows
- analyze_trend: auto line chart + value_column in the output
- validator Check 8l: reject trend/forecast on week-number date columns
- analyze_importance: weak when accuracy <= majority-class baseline
- IRRepairer: source name in source_id -> that source's id (unique only)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
[NOTICKET] feat(cluster): name example members per cluster
- [NOTICKET] feat(scope): code-owned conversation scope — focus from value lists, follow-up carry, scope line (111ef6b9f94acaae6932782799d7fc3c0364e845)
- [NOTICKET] feat(chat): status pings always in English (144494152cfba32196cb64e75b44548a031778cc)
- [NOTICKET] fix(planner): match the question's time grain; reject non-task placeholders (ac68af5f1fedb87cd64b206a25c539136e1e491e)
- [NOTICKET] docs(plans): report reframe and prompt cleanup plan (1acca723005e9ce364981a6e8e20684dc41dddee)
- [NOTICKET] feat(report): question-anchored report with scope, checks and an analysis ledger (564c8270c31e0729288f251c918ea075ed042ae7)
- [NOTICKET] fix(report): English structure, content in the user's language (99ed8a1a0af610180bd77a65bb952d79aa7fd906)
- [NOTICKET] refactor(prompts): one home per rule — planner.md 7.1k -> 3.9k tokens (71c6f2b2bf958186f1d0b430b237229f5fdcf03d)
- [NOTICKET] fix(help): suggest the analytical tools that are now active (b1ef6f048c8dd469a7f3d2214fae3bbfe7da4e46)
- [NOTICKET] eval(help): capability cases for the v7 tool sync (87bb688ca6d8b1de28e3f911bf8502ab504012d3)
- [NOTICKET] fix(chat): load the latest history, not the first ten messages (b3a1a3557a8946a0d493084be741105e0293f45e)
- [NOTICKET] docs: sync REPO_STATUS, DEV_PLAN and contract with the code (19587bc335e857a2d3c992b9d2fd16e0b7e37159)
- [NOTICKET] docs: close DEV_PLAN #32 — live Neon matches Go 0007 (17b54e212416e06fe32e3942ebf951856b03792e)
- [NOTICKET] fix(report): no user-scope catalog fallback in the data-source appendix (35a6aedb48870a8c6007b845c7a5ff1d0f57c324)
- [KM-717] feat(planner): read the catalog data dictionary into the planner's catalog summary (2d70efebcdfecd9f1fce6cfa43c5e304bf6edbe4)
- [KM-717] eval(planner): data-dictionary cases, and live Neon knowledge tables verified (97b27d162f937dfe302e17c432a7e76e3cdfb937)
- Merge remote-tracking branch 'origi…
Merge branch 'main' of https://huggingface.co/spaces/DataEyond/Agentic-Service-Data-Eyond-Catalog
- feat(documents): v2 unstructured ingestion, image-token, scoped retrieval (b848f739c23c9ea152c5bcc5ea4b9d8b4ca64afa)
- feat(prompts): mixed document+data path, figure markers (39019d57e159b510fda1f5bd813d051e1ba7c05c)
- eval(chat_sim): doc + mixed suites with bound documents (b6e060b328c8dbbdb19f3da466fe97f62dab09aa)
- [NOTICKET] docs: unstructured v2 plan, contract, status sync (6702119c2705ef9a99a4a0ba0adaab9ac7a28c54)
- fix(check): list data and documents when both are asked (2b7c3bd79fe73b693634e583be46c812b6c74d3c)
- [NOTICKET] docs: mixed suite result, check + router fixes, U8 done (0e1ad6e143e30ec20960d3a2c5ae22190aa512ff)
- feat(router): a figure IN a document vs a chart FROM data (770b98855d640c727a402b0db1252638364c20d7)
- fix(chatbot): reply in the turn's language, not the documents' (b6150d58b991fb5d98b0bb2278956c938d80d4c0)
- fix(query,planner): stable capped rows, plan-time handoff check, trimmed records (ea9848db5ec39e32067a2c3dbc9453f1a94736d2)
- eval: scoped/DB/docum…
