Commit History
-
[NOTICKET] feat(knowledge): two review modes, recorded against a person (r1, r2, r3)
Rifqi Hafizuddin +1: Claude Opus 5Closes Sistem review. Section 7 landed in dedorch, so the columns are declared and written; verified read-only through the ORM against live before committing. r1 `review_mode` on the batch, chosen at ingest. "expert" leaves entries waiting for a human; "llm" approves them on arrival with NO reviewer step. Worth being plain about what the second one is: there is no second model call and no second opinion. The entries were written by a model already, and this mode means accepting them without a human looking. Default is "expert" - nobody gets LLM approval by accident. r2/r3 What makes that offerable rather than reckless is the record. `decided_by` says whether a human looked; `authorised_by` says on whose authority it happened. They are separate columns because the admin who clicks ingest and the expert who reviews can be different people, and in "llm" mode there is no reviewer at all - so without the clicker's id reaching the review row the de…6ef7990 -
[NOTICKET] feat(knowledge): review diff + the rule/domain boundary (r4, o2)
Rifqi Hafizuddin +1: Claude Opus 5r4 `changed_fields()` and `review_diff()`. The queue can now say WHAT moved rather than only that something did - which was the ask: show before / after, do not send the expert back to read the entry from the start. `changed` is empty when the approval still stands, so the screen can say "nothing to re-read" instead of rendering two identical panes. `before` is what the expert actually had in front of them; for an `edited` decision that is their own corrected text, not the model's claim. A test pins changed_fields() and has_mutated() to the same answer - two notions of "has this changed" that can disagree is the same defect as two notions of readiness. o2 The rule/domain boundary, written down in the output contract. The two blurred on screen during the 8 Sep review and a boundary nobody has recorded is one the prompts cannot hold. The test: if removing it would change how a NUMBER IS COMPUTED OR SOURCED it is a rule; if it would only …5cb2e2d -
[NOTICKET] feat(knowledge): promoted columns + withdrawal (m3, m4, m5, i7)
Rifqi Hafizuddin +1: Claude Opus 5The section-6 migration landed in dedorch, so the columns are now declared and written. Verified read-only through the ORM against live before committing: knowledge_entries.doc_ids/content/provenance and knowledge_documents.withdrawn_at all resolve. m4 `content` - the entry's body, promoted out of payload so a consumer does not have to open jsonb. Per kind, first non-empty wins, so a kind that abstained on its primary field still promotes something usable. m5 `provenance` - a LIST of {doc_id, page, page_no, section_no, chunk_id}. The extraction contract emits one object; it is wrapped rather than reshaped, because a list is what a fused entry needs and one shape means every consumer reads the same thing. The span is deliberately NOT promoted: it is evidence, not a locator, and duplicating it out of payload would give the span check two copies to disagree about. m3 `doc_ids` - derived from doc_id + provenance rather than declared, so it cannot drift…e2ae28f -
[NOTICKET] feat(knowledge): incremental ingest — versions, reuse, staleness (i1-i7)
Rifqi Hafizuddin +1: Claude Opus 5The 2026-09-08 "ingest bertahap" items, against four rulings taken 2026-09-14. i5 Identity is `doc_id`; the same document arriving with different bytes is a NEW VERSION of it, not a new document. That is what keeps an amended file's approvals attached - entity_id derives from doc_id, which does not move. The reuse check moves to find_document_by_hash (any version): computing the next version first and then looking it up would miss the row that already holds those bytes and re-parse a document we already have, turning the idempotency guarantee into a paid no-op. The artifact's own `version` is ignored - parsing stamps 1 on every parse and cannot know what the tenant already holds. i2 A term this company has already approved is no longer re-extracted. One LLM call per cluster is the entire cost of the glossary branch, so every recognised term is a call not made - and it is also what stops one concept arriving as term 1.1 and term 1.2. Ids s…72e18a5 -
[NOTICKET] feat(knowledge): readable concept keys + key-value lookup (m1, m2, m6, m8)
Rifqi Hafizuddin +1: Claude Opus 5From the 2026-09-08 discussion. `label` now carries a TYPE_SUBJECT key - GLOSSARY_PA - so an agent that meets an unknown term writes the key and hits the row: no scan, no embedding, no model call. Searching a list is O(N), searching a dictionary is O(1), which was the ask. m1 `src/knowledge_extraction/keys.py`. Prefixes come from the `kind` column vocabulary (English, singular) so the system holds one spelling per branch rather than two to keep in sync. Subjects cap at 40 chars, cut on a word boundary. A rule has no canonical surface of its own, so it borrows the term it governs plus a short digest (RULE_PA_C07BE4); asking the model to name it is the option ids.py already rules out, and a model-written rule_id was a real defect here. m2 A key is a LABEL, never an identity. `entity_id` stays the hash an expert's approval hangs on; a key is built from a surface and moves when clustering canonicalises differently. Pinned by a test: one term from two …0ba89b1
-
[NOTICKET] fix: add a fault-code label, measured; re-diagnose the Komatsu tables
Rifqi Hafizuddin +1: Claude Opus 5The Komatsu manual prints 27 service codes ([CA559], [CB2249]) in plain prose and the pipeline returned ZERO mentions for them. Not a parsing problem -- the codes are in `Chunk.text` verbatim. No label was asking: a code is not a component, a symptom, a check or an action, so neither `broad` nor the parked `diagnostic` set proposed one. GLiNER finds them readily once asked, 6 of 6 on the probe chunk at 0.54-0.69 against a 0.25 floor. `fault code` added to `diagnostic`, plus a `broad_codes` A/B arm. It is deliberately NOT in `broad`, because a label is not additive -- GLiNER runs flat NER and a new label competes for spans: mentions clusters code clusters E1c_strict BUMA 251 -> 200 86 -> 67 0 -> 0 0.8537 -> 0.7805 Komatsu 449 -> 548 93 -> 161 0 -> 17 -- Open Pit 787 -> 711 427 -> 377 0 -> 0 -- BUMA loses 51 mentions, 19 clusters and three gold terms; Open Pit loses 76 mentions…366697c -
[NOTICKET] feat: pull one linked partner chunk into the evidence window
Rifqi Hafizuddin +1: Claude Opus 50.4.0 pairs a table chunk with the prose that introduces it (`referenced_by`, both directions) and nothing consumed the pointer. Measured what it would buy, ranking only on the three v2 artifacts -- free, no GLiNER, no spend: clusters top-5 touches a pair BOTH sides only ONE BUMA 83 31 0 31 Komatsu 93 55 0 55 Open Pit 427 70 0 70 Zero of 603 clusters saw both sides. The ranker never co-selects a table and the prose that names it, so the partner is unseen content rather than a duplicate -- and for a bare grid that prose is often its only title ("Troubleshooting of engine (S-mode)" above "S-1 | 6"). Cost at the cap of 1: evidence read +14.5%/+24.5%/+9.9%, i.e. +1.6%/+4.1%/+4.2% of total prompt spend; Open Pit moves 71% -> ~74% of the 2M ceiling. The expansion lives in `top_k`, not `rank_evidence`, for two rea…5d51025 -
[NOTICKET] fix: retire escalation, budget windows by subword (Finding 6 + the over-cap window)
Rifqi Hafizuddin +1: Claude Opus 5Two measurements, two constants. Finding 6 asked for a number either way on escalation. Counted off all six persisted paid runs (v1 K=3 and v2 K=5 x three documents): glossary calls clusters extra definitions from escalation BUMA v1 69 66 3 0 BUMA v2 83 83 0 0 Komatsu v1 94 75 19 0 Komatsu v2 102 93 9 0 Open Pit v1 361 349 12 0 Open Pit v2 430 425 5 0 48 escalation calls, and not one produced a definition the first round had missed. `extraction_status == "escalated"` is set only when a later round succeeds where round 0 failed, and it appears zero times in every file. Why it cannot work as built: escalation hands the model the NEX…1faca81 -
[NOTICKET] fix: gate spend in cluster units and surface the estimate (Step 6)
Rifqi Hafizuddin +1: Claude Opus 5Found on the way in, and it is the bigger half of this commit: the pre-flight ceiling was gating on a PARTIAL number. `estimate_cost` summed the glossary branch alone while reporting a field named `estimated_prompt_tokens`, so every consumer read a part as a whole. Measured on the three v2 artifacts, free: clusters calls old (glossary) new (all four) undercount BUMA 83 106 170,466 201,751 18.4% Komatsu 93 95 208,993 227,329 8.8% Open Pit 427 494 1,354,283 1,560,548 15.2% The Open Pit chapter is at 78.0% of the 2M ceiling, not 71%. Adds a SECOND ceiling on cluster count, which is what actually drives spend -- one LLM call per term cluster -- and unlike the token figure is exact and known for free. Two things the token ceiling cannot see: the cache hit rate degrades as a run grows, so the real bill rises faster than the estimate; and part of that estimate is…cdb5fe8 -
[NOTICKET] feat: assert on invariants, report a band, classify the genre (Steps 0 + 5)
Rifqi Hafizuddin +1: Claude Opus 5Step 0 -- measurement hardening. `temperature=0` is argmax over logits a shared inference server does not hand back identically twice, and this corpus has a recorded case of identical input scoring 6/7 one day and 0/12 the next. Three changes follow from that: * `eval/knowledge/invariants.py` asserts on FIELDS AND INVARIANTS, never on exact text: a span that cannot be located verbatim, a missing or duplicate entity id, a rejected field that still carries a value (rejected-never-repaired is only checkable from outside), a page out of range, queue ranks that are not contiguous, a leaked sort key. A violation means a control leaked, so it fails the run (exit 5) regardless of how the scores looked. * `--runs N` repeats ONLY the paid stage -- the free stages are deterministic -- and reports min/median/max with the MEDIAN as the value, because one failed run drags a mean into a region no run occupied. It also checks every run minted identical entity ids, which is the K11…f1d64b1 -
[NOTICKET] fix: cap key-parameter pre-emption in the review queue (Step 2)
Rifqi Hafizuddin +1: Claude Opus 5Unresolved key parameters held their own sort tier beneath conflicts, so EVERY one of them outranked EVERY routine term regardless of how many there were. The branch emits them in bulk on exactly the documents it understands least, so the promotion fired hardest where it was least deserved: on the Komatsu shop manual that is 24 junk rows standing in front of `Engine`, a term with 108 mentions, at rank 25. The original reason for promoting them is preserved and still sound -- that branch has no span check, so an unresolvable surface is the one hallucination signal it can raise and it must not sink beneath sixty routine rows. A CAP expresses that without letting the tail take the queue over: at most one parameter per five term rows. Also heads the parameter name. The branch picks the right CONCEPTS and returns them wrapped in their whole defining sentence (185 chars in one measured case); `term` now carries the head and the new `surface_full` carries the untruncated text. Split only pa…4e3a6ae -
[NOTICKET] fix: test the caption hypothesis; ship the signal at weight 0.0
Rifqi Hafizuddin +1: Claude Opus 5The first paid v2 run produced `definitions_citing_a_caption = 0` on all three documents. Two candidate explanations. Both are now measured and BOTH ARE FALSE. 1. "Captions never reach the term filter." FALSE. A caption is part of Chunk.text, so GLiNER already scans it and already produces mentions from it: 2 of 2 captions on BUMA, 3 of 8 on Open Pit. Feeding captions to the filter would have been a no-op. 2. "Figure-bearing chunks are never selected as evidence." FALSE. A captioned chunk is already inside the top-5 for 22 of 83 BUMA clusters and 64 of 427 Open Pit clusters. To test (2) properly this adds the lever that genuinely was missing: an `asset_caption_names_term` ranker signal. A/B at 2.5 vs 0.0 across all three documents changed the top-5 of ONE cluster out of 603, and moved zero clusters from "no figure" to "has a figure". The real reason is visible by reading them: not one caption in the corpus is a definition. They are figure titles -- "Figure 1.3. Re…36a166e -
[NOTICKET] feat: parsing schema 0.4.0 — asset pointers for figures and tables
Rifqi Hafizuddin +1: Claude Opus 5A figure used to become its own chunk with an EMPTY `text`, which made it unreachable by construction: no text -> no term mentions -> joins no cluster -> never selected as evidence Measured across three documents: 13 figures, every one already described by a vision model we pay for, and ZERO of them appearing in any cluster's evidence. Meanwhile the prose that said "Figure 1.1" had its  reference stripped and the caption landed in a NEIGHBOURING chunk, so nothing connected the sentence to the picture. Now the reference is folded into the prose chunk at its reading-order position and the figure travels as an `Asset` on that chunk. image chunks 2/3/8 -> 0/0/0 dangling asset:// refs 0 description leaks 0 captions kept in text 17/17 (Open Pit) — unchanged E1 / E1c identical v1 -> v2; recall did not move Tables deliberately DO NOT change: unlike a figure, a table's linearised text earns its evidence slot (74.6% of the Komatsu manual's t…e662ac0 -
[NOTICKET] fix: retire the fuzzy clustering pass; add E1c post-cluster recall
Rifqi Hafizuddin +1: Claude Opus 5`rapidfuzz.token_set_ratio` returns exactly 100 whenever one string's token set is a SUBSET of the other's, so a head noun always absorbed its own compounds and FUZZY_THRESHOLD=92 was inert. Measured across three documents (BUMA standard, Komatsu shop manual, Open Pit textbook Ch.1), the pass made 140 merges: 131 token-subset matches -> all wrong 6 singular/plural pairs -> correct, now folded deterministically 3 gain/loss pty|qty|ua -> wrong; three distinct metrics 94% of its output was wrong and it produced no correct merge that plural folding does not now do. Matching is exact (plural-folded) -> abbreviation<->expansion. token-subset merges 15/18/97 -> 0/0/0 E1c_strict (BUMA) 0.8293 -> 0.8537 (`EWH`, absorbed by `Working Hours`, recovered) After the fix BUMA's remaining unowned gold terms are all ones the FILTER never proposed, so clustering destroys nothing. `normalize(lemma=True)` is scoped to the clustering key path only. rank/evidence.py imports `norm…2cc21c9
-
/feat knowledge management (#20)
- [NOTICKET] docs: knowledge pipeline test integration — context, audited task checklist, DEV_PLAN §0.8 (8e69dc6c638509f4dd7b6775d520fb4aa4efbd3e) - [NOTICKET] docs: knowledge pipeline — reframe kex as prototype, not port target (7cce5ffc9ab88791ac139dbef8e88e124a3d049e) - [NOTICKET] feat(eval): knowledge-extraction gold set, scorer + frozen prototype baseline (60cf4a885ef31a8aa64916f5a70ac5a0e8062c5f) - [NOTICKET] chore: add missing planner and readiness eval results (bd6659b3bf523597e4482b04ab1fcff5e1284d25) - [NOTICKET] feat(knowledge_extraction): free stages — models, seam adapter, filters, cluster, ranking (024c30abd1189119d33508eded334fe5a3ed5011) - [NOTICKET] fix(knowledge_extraction): move runner into the package; fix two ranking defects (72605ff6876a7936ab685d417c766ce820f6214b) - [NOTICKET] feat(knowledge_extraction): span filter — E1 recall 0.8537, matches the frozen baseline (3a016343398753cf730d50548c0e92ff47c6716c) - [NOTICKET] feat(knowledge_extraction): paid extraction …b68816f
-
/fix planner count and report (#19)
- /fix planner count question (11de970ff65be9b4f81f773cfb355d3dc57358d7) - [noticket] fix report bug where some answered analysis isnt included in report summary answers (8dec1825cf803ad28f4ff0400b9c373250a7d90a) - [NOTICKET] fix: split report floor from report body; scope analysis reads by user_id (fd4865b20dfc46782706be9aab318f909b9903c3) - [NOTICKET] docs: sync report floor/body + security posture; add review remediation tracker (fc5200c994441ec505ea7a82469da577afe170a5) - [NOTICKET] test: planner eval harness (rule-compliance, --selfcheck/--rescore) (4292522aa3fa3a7af8f933307334cb7e5eb0b377) - [NOTICKET] eval: repair readiness harness; commit intent/help/readiness baselines (08a0a4ad5eacc6c90df22fab5be69f2d33579f29) - [NOTICKET] fix: compiler parity — F-17 fabricated column, F-18 LIKE nulls, F-25 empty in/not_in (8f736f90f1958960a83847d332b45b13fa346168) - [NOTICKET] feat: bound catalog render + blob read; scope charts/traceability; mark degraded seams (0cb7d53cb6f7f7e7b687dd6d9c27…f282b15
-
/fix report not working (#18)
- [NOTICKET] fix: write reports.user_id (Neon NOT NULL column broke report persistence) (13f74e11df6058800f71e539fe135eeba7576f24) - [NOTICKET] docs: correct 13f74e1 root cause — Go migrations are non-convergent (86d0805130637d66082c5a75b8da71fae36002ad)9070d67
-
/fix change model to gpt 5.4 mini (#17)
- [NOTICKET] chore: rename LLM env quad __4o -> __54m (GPT-5.4-mini) (790ef0c7aaac6e76eb01b0ef9a3398ff4249416d)d65c41d -
fix/ check and new chart tool (#16)
- check: table-level column drill + drop misleading "name a table" hint (ef0c7b1c5aa49a244b02f1b5eea15cc77dcf2bcb) - [KM-715] charts: render_chart tool + message_charts store + GET /api/v1/charts + planner viz slice (87cfcf884d923ee4bcd722e0e4ef4d90d7eaedca) - [KM-715] checkpoint: S1a deterministic quality checkpoint between runner and assembler (f3c8de34dc19a573031b67fd4f48dcfbadcdfed8) - [KM-715] docs: sync REPO_STATUS/DEV_PLAN + land SPINE_V2_PLAN (W1/W2 done) (434eacbf2ac8324188ebe5e32775dbc9d2a3f49a) - planner: value-handoff (${tN.col} → list) for tabular lookup/anti-join (54070ac1407b751000e49dfe7f102a2be5ab1ff8) - [KM-715] charts: GET /charts by message_id only + tri-state status marker (b8d15761addf4d499c9631df64e7693aa1d61fa2) - [KM-715] report: embed charts as plotly EDA fences; render_chart counts toward the floor (89db8be6b6f11e86ad0776d2ee07f2306e3b6079) - [KM-715] docs: contract + status deltas for the charts lead-review round (96347ee7c55a4eb39a906409ec214463e25c5e35) - …5a60e93
-
/fix traceability (#15)
- [KM-691] Traceability: add resolved data_used layer for the FE (a646735e33f6ac725d0756a88956b0aec15fe495) - check: don't leak user-scope catalog when analysis catalog missing (c0ba82938383169b41059c4125765db5275885e7) Co-authored-by: Sofhia Az-Zahra <sofhiaazzhr@users.noreply.huggingface.co>32abc41 -
/fix help and tools analyze (#14)
- [KM-711] Retire /report slash command; report is Generate-button only (f7f7b172da0dc16adac0a7555f6d34a704cc2050) - fix(analyze_trend): handle value_column == date_column (count events per period) (55e3327f7b2e21a954b317e143f46bcea8b5ada7) - [KM-711] Docs: drop /report from list_skills catalog examples (0772863b296e2ed3e2d559df05e98540764e58b0) - fix(planner): reject inlined table-ref as analyze_* data (Pattern A) (776fae3c7c6e3d3a07c22bef10898f6b1c33c701) - fix(analyze_aggregate): reject non-string group_by (CASE/binning) keys (b40fa76e2fa68572dad0d8532e07076207095d6b) - fix(planner): reply-language for infeasible_reason + answer-only-asked rule (4d89d7d93938e41892979cd20e5ccf0c97441dac) Co-authored-by: Sofhia Az-Zahra <sofhiaazzhr@users.noreply.huggingface.co>17e31c6
-
/fix planner and report (#13)
- [KM-644] planner: infeasible plan outcome + entity-ranking few-shots (089b69d9b6111e813307c6afbd3e7613bb823f9d) - [KM-644] harden query path: grouped-select rule, int-date trends, sample decode, trace guard (74ad5261736bda0bd72bccdb41565ea67313993a) - [KM-703] add analyze_merge tool: two-table combine (analytics.combine) (8abf635532366d670367557a3ffa03d42371d831) - [KM-567] planner support for analyze_merge: data_right guard + two-retrieve merge few-shot (a029a843247e6ea4d3c79a4e2c1b7571ba4cda69) - [KM-644] report v2: BQ answers, unresolved/excluded sections, evidence tables + curation/readiness endpoints (4ceb058d85c5a2a289443b9411d2b345027fa226) - [NOTICKET] refactor: mint message_id as UUID to mirror analyses_messages id shape (4253733b6b8db0449a82442a915c732ddd55f528) - [KM-644] report: drop Notes/Unresolved/Method sections from rendered markdown (compact report) (eef6de811a139402c79fd74ad4adfc00cf7aff01) - /edit md file (8198cc188ff05a2798ae2056b90b6e56e1bea87b) Co-authored-by…49b0848
-
fix/ check (#12)
- [NOTICKET] check: scope to analysis catalog + list database tables (db2a10f61e1f6ff9579deaaeafed8fdf0cdf5bb7) - feat(planner/help): graceful degradation for gated analytics capabilities (efc0af7c05308acb564bffdd7d8ab8abd2dd2e81) - Scope structured_flow + report to analysis catalog; drop dead data_sources binding (c333ec84a96b171484af7a87bdf07fd1fb0e9c69) - update md file (c5e2e1b1adcba46b56eaf6bbb85d9b8bb0a0d277) - [NOTICKET] check: read documents from analysis-scope catalog (6977394de88d7944dd2504beeed3e00eafe7857a) Co-authored-by: Sofhia Az-Zahra <sofhiaazzhr@users.noreply.huggingface.co>b9dfc76
-
/feat traceability (#11)
- [NOTICKET] Fix check skill: reply language + tabular inventory routing (1367f41269cf9ef42fd91f311b24d4edf5885c7b) - [KM-691] Develop traceability (23cc2079989fef931a6e146980fe6bf528ee6e92) - [NOTICKET] Humanize check output: hide UUIDs, add lead-in, list format (2bdda8fc3fc4b4e17b4df7fb53aee5abede45add) - [NOTICKET] check: schema-detail view for column/type questions (9e248d9cc5688d6ee622be523a6bedddf5010995) - [KM-691] Make sources trace-authoritative; drop stream sources (b43ecc77f26c1537cbbfddb6800ec4ff62d16441) - [KM-691] Bound traceability payload + reject contradictory IR filter ranges (db6db73d6ab0c2175447946742038443d67b2f78) - /fix planner can now plan quarter comparison (839e32c8d6d5895029072074ab7aba8bb4fcdb6e) Co-authored-by: Sofhia Az-Zahra <sofhiaazzhr@users.noreply.huggingface.co>f873f92
-
/fix validator and report (#10)
- /fix report: remove record_id hash citations, EDA (for future tool) change to data source, transform to mdx-safe (b1b101730ccf5628e43e80f2ad654f78fbed5207) - [NOTICKET] fix(planner): repair near-miss catalog ids in QueryIR before validation (aacd4d13def258a9815761aecbd7b246a8dae9fc) - /fix report: group findings by analysis, dedup, code-span-safe MDX escaping (029b48894e9f79af6c0d81c00a2d811a982f87ae) - [NOTICKET] fix(tools): log swallowed analytics tool errors in AnalyticsToolInvoker (93d9381b2a5f5efe058c913fe0b793ce994a9097) - fix(planner): harden plan validation to reduce retry thrash on structured questions (1619dd3c0fa74825010c7af6d4b375e3b9404847) - [NOTICKET] fix(tools): coerce numeric-string columns before analyze_* compute (6bda333866a1a8ccf1b3fe6fcee7a8e1832f28d7) - fix(planner): correct top-N guidance — in-query ranking IS supported (99aa4141aed8241a625d290fb170f74192a6d831) - feat(guardrails): add input guard, out_of_scope intent, and refusal handling (0bbc27bc40ba20e7a8e…0e5fdb5
-
/fix analysis + report_inputs and storage update (#9)
- fix(chat): drop ENABLE_SLOW_PATH flag + make chat persistence read-only (b8b76c4fb8db21de183cc6f80c0acba0d35bdad1) - [NOTICKET] feat(storage): read tabular Parquet from Supabase S3 via provider toggle (3f48226adaf2ed107a4eb2c0373cbb73fc62f552) Co-authored-by: Sofhia Az-Zahra <sofhiaazzhr@users.noreply.huggingface.co>079782d -
/fix check and help tool (#8)
- /fix data_catalog model + CatalogStore to dedorch schema (data → catalog_payload) (09857fc52486354f53e752e3ea329b064a9e4c4b) - [NOTICKET] fix(help): lock reply language to user's language on button path (cb7f5ac484ad4a510e68faf5291091958b6d2f42) - /fix default SKIP_INIT_DB=true for dedorch cutover (skip init_db on Go-owned schema) (781b93c6b8744209a1101629b5170175e9212592) - /fix surface catalog foreign keys to the planner (FK-backed joins) (446c57524e932fcdbb0ddecc049189c443b6ae97) - [NOTICKET] feat(eval): add help-skill eval harness + goal-based language fallback (cbdbefb7ab0ffa36949713f18422213db480a959) - [NOTICKET] fix(eval): update readiness eval to post-KM-652 floor (drop problem_validated) (6f46e81bdeb1eb987f9dac8af0e22a975474cbb1) - /fix infer FKs for FK-less dedorch catalog + log repr(e) in db executor (3669290474583ae011f5ed5e50b94dad146fbc77) - Merge eval/help + readiness eval fix into PR #8 (5f4818a8e38f93988ab5756f3a06c94645826b15) - /docs REPO_STATUS: FK inference (FK-…3743cfe
-
/fix v2 chat stream api update to analyses_messages (#7)
- [NOTICKET] fix(db): reconcile analyses schema + migrate chat to analyses_messages (283eb0e4653acfe9311f42e3bb12d0490b8c9751)0066161 -
/fix message_id for chat/stream Endpoint (#6)
- [NOTICKET] refactor(api): server-mint message_id only; unwire v1 chat (729086150b982cbd747829074658bc798d8e18f4)d8e7745 -
feat/ Endpoint Restructure (#5)
- [KM-686] Unwire Non-AI Routers from Main + Swagger (0b2d6785801ecc4d58a32580b1a9ef6238059de2) - [NOTICKET] docs: pr/5 sprint tracker + endpoint-restructure contract + REPO_STATUS banner (ce7be178428cadf1884a77d3d1f1398dccf32f6e) - [KM-687] Add Dedicated /tools/help Endpoint + Regroup Tools List (a881d38851470bf685990fdb554417da4fc1eccd) - [KM-688] feat(api): v2 chat pilot + regroup report under /tools; adopt lead API contracts (ad2ce619b51c234a8cd1c3772417a9a542a9a40d) Co-authored-by: Sofhia Az-Zahra <sofhiaazzhr@users.noreply.huggingface.co>3bacc1d
-
feat/Analysis State & Report Rework (#4)
- [NOTICKET] Add project repo documentation (ade9c0c635b1bb6f171a3f4e710e6d0f9bae28f3) - [KM-674] feat(report): adapt report generator to objective + business_questions (9bd8ec51fdcd5057b5e8ba6b219a7d9dce945144) - [NOTICKET] Add revision plan md file (e0384e1dc600b2496f96cbba152e616d65650f07) - chore(docs): tidy root markdown — archive stale docs, delete superseded ones (4835fb85915bd69079fb43d7c602774d514e02ae) - [KM-652] refactor: remove problem_statement skill + problem_validated gate (057e0b24b56dd697603d183f508cb1e8a87268fd) - [KM-678] feat(tools): trim FE slash menu to /help + /report (fed17ccc98ec523088c482673132763826f86701) - [NOTICKET] docs: sync DEV_PLAN status (#8/#9/#20) (e936cfafdfde1385fe16bd7e3796e7d46c193197) - [KM-679] fix(help): drop problem_statement refs after KM-652 (0bec4725db572d492634ced6a80ef6e65d075edc) - [KM-680] feat(report): show author display name (users.fullname) (9bccc122878fbdbae0ac1bd0b4316f5ea8af658c) - [NOTICKET] docs: add API_ENDPOINTS.md (FE surf…0e02a0f
-
feat/Knowledge & Data Tools (#3)
- [KM-641] refactor(tools): trim analytics registry to 4 active tools (66e2e4df66f1abf7eaa7e442d9900d428696ece4) - [NOTICKET] docs: add checkpoint plan for 2026-06-17 sync (8c1bb63ebdee02f9db0da9fc2b2269823b0a7688) - [KM-642] refactor(tools): rename data-access tools to data/knowledge naming (c38c0c29d3e6b245f67d03f2c35e451900e1abdb) - [KM-643] feat(tools): merge data sources into data_check + add knowledge_check (4bd5f1efdc560a64092cc8df672a7990fa949b59) - [NOTICKET] docs: record KM-642/643 in checkpoint plan + PROGRESS (4c57588b90fe0cac8d0d4d22cfe5701fc9eb52dd) - [NOTICKET] fix(chat): scope response cache by user_id + drop TTL to 1h (R5) (b701e95d300c7d2480a3608b2d4da6dfe102b19a) - [NOTICKET] feat(slow-path): persist AnalysisRecord via PostgresAnalysisStore (09f845188226dd01633cf74525caa569e4a7d37a) - [NOTICKET] Adopt verb-first skill naming (2d6406def7ed45edd1226e96645ba2a5cdbfa3d0) - [NOTICKET] docs: sync checkpoint plan — verb-first tool names + report-as-API (3b399f84d73b0d98bd99…0721bb4
-
/fix redis connection code for general redis connection
Rifqi Hafizuddin0810be5 -
feat/Planner Agent (#2)
- [KM-567][AI] Planner agent: contracts, schemas, inputs, registry (be96601ad46c00e8e789768dd6e89c2db827bde4) - [KM-567][AI] Planner agent: prompt + few-shot examples (83ba6b1743370145454d9315224eeb7b2d12ac18) - [KM-567][AI] Planner agent: validator + service (72306d0ce92c76d60117a20c701c7f8185bcfeef) - [KM-567] docs: record Phase 3 Planner agent in PROGRESS.md (ba2fa8892392028c8d13646c09d1155bbf34e703) - [KM-567][AI] Planner agent: fix PII leak + placeholder/retry robustness (6b3637dda097f16cd98d4b9ad6094fdaed6d1346) - [KM-624] Implement Compute Functions Tools (78c598c22d2f2a14229a83c3f43fa98c5d520ae7) - [KM-624][AI] Planner: realign stub registry + examples to composite analyze_* tools (93a24dabf48d8121e981cc470c835cb10badb62a) - [KM-624] docs: record tool taxonomy + ownership revision in PROGRESS (f3461142542f6216aff23b30a1321dfb4654afdc) - [KM-626][AI] Slow-path agent: seam contracts + TaskRunner (ac310de551d0888ccfef31e981e22d88f54c6a38) - [KM-626][AI] Slow-path agent: Assembler…81e5fe7
-
[KM-582][DED][AI] Fix Retrieval in Agentic Service
ticket: https://bukittechnology.atlassian.net/browse/KM-582 fix: replace LangChain ORM retrieval with raw SQL and fix pgvector collection name - Rewrite DocumentRetriever.retrieve() using raw SQL cosine/manhattan queries instead of LangChain PGVector ORM, bypassing asyncpg type-mapping issues (id String vs UUID column, jsonb_path_match binding quirks) - Fix _COLLECTION_NAME from "document_embeddings" to "documents" to match the collection name set by the Golang ingestion service - Fix collection_name in vector_store.py to match consistently - Fix Redis chat cache to store {response, sources} dict so cached replies also populate message_sources table - Add cache management endpoints: DELETE /chat/cache, /chat/cache/room/{id}, /retrieval/cache/{user_id} - Invalidate retrieval cache automatically after document processing - Update intent_router prompt: route topical/knowledge questions to unstructured even without explicit document mention; prefer unstructured when ambiguous…61c746f
-
fix build error in importing packages langchain_postgres
0e9263a -
Fix schema table langchain_pg_embedding
079dea0
-
[NOTICKET] fix: add SKIP_INIT_DB flag and remove pymongo dependency
- Add SKIP_INIT_DB env variable to skip database initialization on startup, useful for local dev when DB is not available - Remove pymongo dependency (unused, replaced by PostgreSQL-based user storage) - Comment out unused MongoDB settings in Settings class - Update .gitignore: add API_CONTRACT_AGENT*.md and migratego/ - Remove .vscode/launch.json (superseded by uv run fastapi dev)19bd037
-
feat/Catalog Retrieval System (#1)
- initial commit (8557a2045ca2c0873e07efdf6227064b83e4a1e5) - initial structure (efc0c0a8b2444014cbe74f04617c3c974d9915be) - [KM-553] initialize shared contract (f31f6731692b0199d8cb5c5d29dffa466d6d4ca0) - [KM-553] migrate retrieval layer and remove obsolete rag/tools modules (df220ea47c96eb8a1efa07c6eebef4604fc308d6) - [NOTICKET] catalog enricher + structured pipeline + on_db_registered (77ea82d29d86400c5ed30ac154be8df6a7e897eb) - Merge commit 'refs/pr/1' of https://huggingface.co/spaces/DataEyond/Agentic-Service-Data-Eyond-Catalog into pr/1 (0ae8717c7efce2cda14fe4b8caecd774459b1f6e) - [NOTICKET] SqlCompiler (Postgres) + DbExecutor (212dad360714508c1aa607995039e6818fa40f78) - Merge commit 'refs/pr/1' of https://huggingface.co/spaces/DataEyond/Agentic-Service-Data-Eyond-Catalog into pr/1 (faf8a4f6d7e06ccf1ce84c2e7f4990c79f27f843) - TabularIntrospector + on_tabular_uploaded trigger + 31 tests (ca425208969b72a20ed76f86ef0faa1a0e912339) - [KM 556] add columns field to QueryResult + update…6bff5d9
-
[KM-438][KM-439] Improve Retrieval and Querying feature (#15)
- [noticket] add gitignore (c87f27f458854787102eebc110b178c2572ef8a6) - [NOTICKET]: add document pipeline, simplify document API (fb871f3d7db9c5a9c95a012761c4180a432e7e4b) - [NOTICKET]: update folder document_pipelines after pipelines (a4cf97ab3890e35831c0043a1d411c17d84f374b) - [NOTICKET][DB] refactor code to new repo (7f3bb9782bb9b6ace69e6be35377c12757a2f81d) - [KM-441] add mean and median (9b5933420821187e73bb72a896e28d7adf6d6137) - [NOTICKET] new metadata format for cleaner code (6b590d94c398ac4966a0e376f2809aa52c5ba022) - update document (5a69e0eecf80dd807565dbc369374e41450c37fc) - delete duplicate file (3848d7b262745b758069e3ab8089437ced42a45f) - edit document for new pipeline (425e021053c4de65e2598b0f5554b7c1044a904c) - [NOTICKET]: add CSV and XLSX file type (31920c3bc6f9faf9e07095a02fc1faf08adc246e) - [DB] fix/rename db_pipeline.py (d913315c356d741cc7a5a5b1f8e36ba41397f2ab) - [NOTICKET][DB] menyesuaikan format struktur db_pipeline sesuai dengan file lain (e13a9017e0f7fd26ef9b8a…c93ec90
-
Make a query for tabular (XLSX and CSV) (#14)
- [NOTICKET] add software to gitignore (d43ecb180036339cf287e93f1d9116bd6eff9b9d) - [NOTICKET] add pyarrow (e50eadc82e160012e3319267f5bfe084cd9034d4) - [KM-515][document] Make Query for Tabular Type (XLSX & CSV) (695ca0a154a68077c51499ed83b7507b988be065) - [KM-455][document] decided methods retrieval for document (8c9cc79223eb1d96c8622d129219a83a4ba2500b) Co-authored-by: Sofhia Az-Zahra <sofhiaazzhr@users.noreply.huggingface.co>8802920
-
Merge dev_new to main (improve retriever and add querying) (#13)
- [noticket] add gitignore (c87f27f458854787102eebc110b178c2572ef8a6) - [NOTICKET]: add document pipeline, simplify document API (fb871f3d7db9c5a9c95a012761c4180a432e7e4b) - [NOTICKET]: update folder document_pipelines after pipelines (a4cf97ab3890e35831c0043a1d411c17d84f374b) - [NOTICKET][DB] refactor code to new repo (7f3bb9782bb9b6ace69e6be35377c12757a2f81d) - [KM-441] add mean and median (9b5933420821187e73bb72a896e28d7adf6d6137) - [NOTICKET] new metadata format for cleaner code (6b590d94c398ac4966a0e376f2809aa52c5ba022) - update document (5a69e0eecf80dd807565dbc369374e41450c37fc) - delete duplicate file (3848d7b262745b758069e3ab8089437ced42a45f) - edit document for new pipeline (425e021053c4de65e2598b0f5554b7c1044a904c) - [NOTICKET]: add CSV and XLSX file type (31920c3bc6f9faf9e07095a02fc1faf08adc246e) - [DB] fix/rename db_pipeline.py (d913315c356d741cc7a5a5b1f8e36ba41397f2ab) - [NOTICKET][DB] menyesuaikan format struktur db_pipeline sesuai dengan file lain (e13a9017e0f7fd26ef9b8a…52999bc
-
[NOTICKET] fixed single source into multiple sources (#12)
- [NOTICKET] fixed single source into multiple sources (9d826225424dd8fb0044ce6f4a010ab8eac1075b) - [NOTICKET] remove source_hint line (d1f034edff5a6a78c0a7b9f3402ca2a46baa4f43) Co-authored-by: Sofhia Az-Zahra <sofhiaazzhr@users.noreply.huggingface.co>c8e3046 -
[NOTICKET] add updated_at metadata & prevent duplicate while adding new embedding (#10)
- [NOTICKET] add updated_at & delete old embedding before adding new embedding ingestion (da83e0111d3fd220fcfdd717018ce4794415e020)bd0029e
-
[NOTICKET] add total token logging (#9)
- [NOTICKET] add token logging (34eb194ed4c3f31ccd968390d1c05ca9547edafe)d973099
-
[KM-496] [DED][AI] Fix Response data structure for field sources (db flow)
5c4067e -
[NOTICKET] [document]: add doctypes endpoint (#7)
- [NOTIKCET][document]: add doctypes endpoint and 10MB file size limit (9694cc0635e4b59a0d6291e613d83262ee3277f3) Co-authored-by: Sofhia Az-Zahra <sofhiaazzhr@users.noreply.huggingface.co>b1f1ccd -
[NOTICKET][DB] add endpoints for dbtypes (#6)
- [noticket] add gitignore (c87f27f458854787102eebc110b178c2572ef8a6) - [NOTICKET]: add document pipeline, simplify document API (fb871f3d7db9c5a9c95a012761c4180a432e7e4b) - [NOTICKET]: update folder document_pipelines after pipelines (a4cf97ab3890e35831c0043a1d411c17d84f374b) - [NOTICKET][DB] refactor code to new repo (7f3bb9782bb9b6ace69e6be35377c12757a2f81d) - [KM-441] add mean and median (9b5933420821187e73bb72a896e28d7adf6d6137) - [NOTICKET] new metadata format for cleaner code (6b590d94c398ac4966a0e376f2809aa52c5ba022) - update document (5a69e0eecf80dd807565dbc369374e41450c37fc) - delete duplicate file (3848d7b262745b758069e3ab8089437ced42a45f) - edit document for new pipeline (425e021053c4de65e2598b0f5554b7c1044a904c) - [NOTICKET]: add CSV and XLSX file type (31920c3bc6f9faf9e07095a02fc1faf08adc246e) - [DB] fix/rename db_pipeline.py (d913315c356d741cc7a5a5b1f8e36ba41397f2ab) - [NOTICKET][DB] menyesuaikan format struktur db_pipeline sesuai dengan file lain (e13a9017e0f7fd26ef9b8a…c7d59cf
-
[KM-493] [DED][AI] Fix source extraction in chat
changes: - Add .vscode/launch.json with debugpy config for FastAPI via uvicorn on port 7860 - Fix _extract_sources in chat.py to read metadata from nested `data` key - Add debug log for extracted sources tikcet: https://bukittechnology.atlassian.net/browse/KM-4937da6db5 -
[KM-436][KM-437] add and modify for database and document pipeline (#3)
- [noticket] add gitignore (c87f27f458854787102eebc110b178c2572ef8a6) - [NOTICKET]: add document pipeline, simplify document API (fb871f3d7db9c5a9c95a012761c4180a432e7e4b) - [NOTICKET]: update folder document_pipelines after pipelines (a4cf97ab3890e35831c0043a1d411c17d84f374b) - [NOTICKET][DB] refactor code to new repo (7f3bb9782bb9b6ace69e6be35377c12757a2f81d) - [KM-441] add mean and median (9b5933420821187e73bb72a896e28d7adf6d6137) - [NOTICKET] new metadata format for cleaner code (6b590d94c398ac4966a0e376f2809aa52c5ba022) - update document (5a69e0eecf80dd807565dbc369374e41450c37fc) - delete duplicate file (3848d7b262745b758069e3ab8089437ced42a45f) - edit document for new pipeline (425e021053c4de65e2598b0f5554b7c1044a904c) - [NOTICKET]: add CSV and XLSX file type (31920c3bc6f9faf9e07095a02fc1faf08adc246e) - [DB] fix/rename db_pipeline.py (d913315c356d741cc7a5a5b1f8e36ba41397f2ab) - [NOTICKET][DB] menyesuaikan format struktur db_pipeline sesuai dengan file lain (e13a9017e0f7fd26ef9b8a…2ba0613