Commit History
-
/fix planner can now plan quarter comparison
Rifqi Hafizuddin839e32c -
[KM-691] Bound traceability payload + reject contradictory IR filter ranges
Rifqi HafizuddinTraceability: _truncate now summarizes embedded ToolOutputs (Pattern-A analyze_* inputs) instead of re-embedding the full upstream table, and executed queries get a dedicated 2000-char cap (was clipped at 300). Planner: IRValidator rejects >=2 disjoint BETWEEN ranges on one column (all filters are ANDed -> 0 rows), and planner.md teaches multi-period comparisons to use one spanning range + group-by or one query per period.db6db73 -
[KM-691] Make sources trace-authoritative; drop stream sources
Rifqi HafizuddinSources move fully to GET /api/v1/traceability. The SSE sources event now always emits [] (kept for backward-compat); removed the dead _build_sources helper and the structured-path source TODO. Updates the contract + v2 chat docstring accordingly.b43ecc7 -
[NOTICKET] check: schema-detail view for column/type questions
"kolom apa aja + tipe datanya?" now resolves straight to the data — no filename required — instead of a bare source listing. A schema cue (kolom/tipe/struktur/field) drills into the named source(s), or ALL structured sources when none is named (documents excluded — no columns). The rendered schema is humanized, ChatGPT-style and fully deterministic: - lean per-table columns (Column · Data type); friendly type words (teks/bilangan bulat/desimal); row count moved to the header line. - adaptive columns: "Boleh kosong" only when a table mixes nullable values, "PII" only when a column is flagged. - per-source Ringkasan bucketing columns (Categorical / Numeric integer / Numeric decimal / Date) — only buckets that exist; int vs float split. - heuristic notes: a date-named string column → "convert to datetime"; a numeric id-named column → "likely an identifier". - big-database guard: a DB with >3 tables is summarised at the table level (name + column/row counts) instead of dumping e…9e248d9 -
[NOTICKET] Humanize check output: hide UUIDs, add lead-in, list format
Turn the deterministic check reply into something that reads like an answer instead of a raw catalog dump: - Hide internal id columns (source_id/table_id/column_id) from rendered output. - Humanize + localize (EN/ID) headers and the source_type enum (tabular -> "File tabular"/"Tabular file", schema -> "Database", etc.). - Add localized lead-in sentences ("Berikut data yang kamu punya:", "Kamu punya N sumber data terstruktur:", "Berikut isi X:"). - Render the source *listing* as a grouped bullet list (structured vs documents) rather than a table; the column-level schema drill-down stays a table since that data is genuinely tabular. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>2bdda8f -
[KM-691] Develop traceability
Rifqi Hafizuddin23cc207 -
[NOTICKET] Fix check skill: reply language + tabular inventory routing
The `check` intent path never applied reply-language detection and hardcoded English strings, so inventory questions ("data apa saja yang aku punya?") always answered in English regardless of the room's language. It also routed generic words ("uploaded", "file") to the document-only branch, so a user with only tabular (CSV/XLSX) sources got "nothing registered" — their data lives under check_data, not check_knowledge. - chat_handler: detect reply_language from the original message and pass it to run_check (mirrors the structured_flow branch). - check.py: localize fixed prose (no_match, helicopter labels, error) via an EN/ID string bundle; thread reply_language through render helpers. - check.py: drop "uploaded"/"file" from knowledge cues so bare inventory questions fall through to the helicopter view (structured + documents). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>1367f41
-
/fix validator and report (#10)
- /fix report: remove record_id hash citations, EDA (for future tool) change to data source, transform to mdx-safe (b1b101730ccf5628e43e80f2ad654f78fbed5207) - [NOTICKET] fix(planner): repair near-miss catalog ids in QueryIR before validation (aacd4d13def258a9815761aecbd7b246a8dae9fc) - /fix report: group findings by analysis, dedup, code-span-safe MDX escaping (029b48894e9f79af6c0d81c00a2d811a982f87ae) - [NOTICKET] fix(tools): log swallowed analytics tool errors in AnalyticsToolInvoker (93d9381b2a5f5efe058c913fe0b793ce994a9097) - fix(planner): harden plan validation to reduce retry thrash on structured questions (1619dd3c0fa74825010c7af6d4b375e3b9404847) - [NOTICKET] fix(tools): coerce numeric-string columns before analyze_* compute (6bda333866a1a8ccf1b3fe6fcee7a8e1832f28d7) - fix(planner): correct top-N guidance — in-query ranking IS supported (99aa4141aed8241a625d290fb170f74192a6d831) - feat(guardrails): add input guard, out_of_scope intent, and refusal handling (0bbc27bc40ba20e7a8e…0e5fdb5
-
/fix analysis + report_inputs and storage update (#9)
- fix(chat): drop ENABLE_SLOW_PATH flag + make chat persistence read-only (b8b76c4fb8db21de183cc6f80c0acba0d35bdad1) - [NOTICKET] feat(storage): read tabular Parquet from Supabase S3 via provider toggle (3f48226adaf2ed107a4eb2c0373cbb73fc62f552) Co-authored-by: Sofhia Az-Zahra <sofhiaazzhr@users.noreply.huggingface.co>079782d -
/fix check and help tool (#8)
- /fix data_catalog model + CatalogStore to dedorch schema (data → catalog_payload) (09857fc52486354f53e752e3ea329b064a9e4c4b) - [NOTICKET] fix(help): lock reply language to user's language on button path (cb7f5ac484ad4a510e68faf5291091958b6d2f42) - /fix default SKIP_INIT_DB=true for dedorch cutover (skip init_db on Go-owned schema) (781b93c6b8744209a1101629b5170175e9212592) - /fix surface catalog foreign keys to the planner (FK-backed joins) (446c57524e932fcdbb0ddecc049189c443b6ae97) - [NOTICKET] feat(eval): add help-skill eval harness + goal-based language fallback (cbdbefb7ab0ffa36949713f18422213db480a959) - [NOTICKET] fix(eval): update readiness eval to post-KM-652 floor (drop problem_validated) (6f46e81bdeb1eb987f9dac8af0e22a975474cbb1) - /fix infer FKs for FK-less dedorch catalog + log repr(e) in db executor (3669290474583ae011f5ed5e50b94dad146fbc77) - Merge eval/help + readiness eval fix into PR #8 (5f4818a8e38f93988ab5756f3a06c94645826b15) - /docs REPO_STATUS: FK inference (FK-…3743cfe
-
/fix v2 chat stream api update to analyses_messages (#7)
- [NOTICKET] fix(db): reconcile analyses schema + migrate chat to analyses_messages (283eb0e4653acfe9311f42e3bb12d0490b8c9751)0066161 -
/fix message_id for chat/stream Endpoint (#6)
- [NOTICKET] refactor(api): server-mint message_id only; unwire v1 chat (729086150b982cbd747829074658bc798d8e18f4)d8e7745 -
feat/ Endpoint Restructure (#5)
- [KM-686] Unwire Non-AI Routers from Main + Swagger (0b2d6785801ecc4d58a32580b1a9ef6238059de2) - [NOTICKET] docs: pr/5 sprint tracker + endpoint-restructure contract + REPO_STATUS banner (ce7be178428cadf1884a77d3d1f1398dccf32f6e) - [KM-687] Add Dedicated /tools/help Endpoint + Regroup Tools List (a881d38851470bf685990fdb554417da4fc1eccd) - [KM-688] feat(api): v2 chat pilot + regroup report under /tools; adopt lead API contracts (ad2ce619b51c234a8cd1c3772417a9a542a9a40d) Co-authored-by: Sofhia Az-Zahra <sofhiaazzhr@users.noreply.huggingface.co>3bacc1d
-
feat/Analysis State & Report Rework (#4)
- [NOTICKET] Add project repo documentation (ade9c0c635b1bb6f171a3f4e710e6d0f9bae28f3) - [KM-674] feat(report): adapt report generator to objective + business_questions (9bd8ec51fdcd5057b5e8ba6b219a7d9dce945144) - [NOTICKET] Add revision plan md file (e0384e1dc600b2496f96cbba152e616d65650f07) - chore(docs): tidy root markdown — archive stale docs, delete superseded ones (4835fb85915bd69079fb43d7c602774d514e02ae) - [KM-652] refactor: remove problem_statement skill + problem_validated gate (057e0b24b56dd697603d183f508cb1e8a87268fd) - [KM-678] feat(tools): trim FE slash menu to /help + /report (fed17ccc98ec523088c482673132763826f86701) - [NOTICKET] docs: sync DEV_PLAN status (#8/#9/#20) (e936cfafdfde1385fe16bd7e3796e7d46c193197) - [KM-679] fix(help): drop problem_statement refs after KM-652 (0bec4725db572d492634ced6a80ef6e65d075edc) - [KM-680] feat(report): show author display name (users.fullname) (9bccc122878fbdbae0ac1bd0b4316f5ea8af658c) - [NOTICKET] docs: add API_ENDPOINTS.md (FE surf…0e02a0f
-
feat/Knowledge & Data Tools (#3)
- [KM-641] refactor(tools): trim analytics registry to 4 active tools (66e2e4df66f1abf7eaa7e442d9900d428696ece4) - [NOTICKET] docs: add checkpoint plan for 2026-06-17 sync (8c1bb63ebdee02f9db0da9fc2b2269823b0a7688) - [KM-642] refactor(tools): rename data-access tools to data/knowledge naming (c38c0c29d3e6b245f67d03f2c35e451900e1abdb) - [KM-643] feat(tools): merge data sources into data_check + add knowledge_check (4bd5f1efdc560a64092cc8df672a7990fa949b59) - [NOTICKET] docs: record KM-642/643 in checkpoint plan + PROGRESS (4c57588b90fe0cac8d0d4d22cfe5701fc9eb52dd) - [NOTICKET] fix(chat): scope response cache by user_id + drop TTL to 1h (R5) (b701e95d300c7d2480a3608b2d4da6dfe102b19a) - [NOTICKET] feat(slow-path): persist AnalysisRecord via PostgresAnalysisStore (09f845188226dd01633cf74525caa569e4a7d37a) - [NOTICKET] Adopt verb-first skill naming (2d6406def7ed45edd1226e96645ba2a5cdbfa3d0) - [NOTICKET] docs: sync checkpoint plan — verb-first tool names + report-as-API (3b399f84d73b0d98bd99…0721bb4
-
/fix redis connection code for general redis connection
Rifqi Hafizuddin0810be5 -
feat/Planner Agent (#2)
- [KM-567][AI] Planner agent: contracts, schemas, inputs, registry (be96601ad46c00e8e789768dd6e89c2db827bde4) - [KM-567][AI] Planner agent: prompt + few-shot examples (83ba6b1743370145454d9315224eeb7b2d12ac18) - [KM-567][AI] Planner agent: validator + service (72306d0ce92c76d60117a20c701c7f8185bcfeef) - [KM-567] docs: record Phase 3 Planner agent in PROGRESS.md (ba2fa8892392028c8d13646c09d1155bbf34e703) - [KM-567][AI] Planner agent: fix PII leak + placeholder/retry robustness (6b3637dda097f16cd98d4b9ad6094fdaed6d1346) - [KM-624] Implement Compute Functions Tools (78c598c22d2f2a14229a83c3f43fa98c5d520ae7) - [KM-624][AI] Planner: realign stub registry + examples to composite analyze_* tools (93a24dabf48d8121e981cc470c835cb10badb62a) - [KM-624] docs: record tool taxonomy + ownership revision in PROGRESS (f3461142542f6216aff23b30a1321dfb4654afdc) - [KM-626][AI] Slow-path agent: seam contracts + TaskRunner (ac310de551d0888ccfef31e981e22d88f54c6a38) - [KM-626][AI] Slow-path agent: Assembler…81e5fe7
-
[KM-582][DED][AI] Fix Retrieval in Agentic Service
ticket: https://bukittechnology.atlassian.net/browse/KM-582 fix: replace LangChain ORM retrieval with raw SQL and fix pgvector collection name - Rewrite DocumentRetriever.retrieve() using raw SQL cosine/manhattan queries instead of LangChain PGVector ORM, bypassing asyncpg type-mapping issues (id String vs UUID column, jsonb_path_match binding quirks) - Fix _COLLECTION_NAME from "document_embeddings" to "documents" to match the collection name set by the Golang ingestion service - Fix collection_name in vector_store.py to match consistently - Fix Redis chat cache to store {response, sources} dict so cached replies also populate message_sources table - Add cache management endpoints: DELETE /chat/cache, /chat/cache/room/{id}, /retrieval/cache/{user_id} - Invalidate retrieval cache automatically after document processing - Update intent_router prompt: route topical/knowledge questions to unstructured even without explicit document mention; prefer unstructured when ambiguous…61c746f
-
fix build error in importing packages langchain_postgres
0e9263a -
Fix schema table langchain_pg_embedding
079dea0
-
[NOTICKET] fix: add SKIP_INIT_DB flag and remove pymongo dependency
- Add SKIP_INIT_DB env variable to skip database initialization on startup, useful for local dev when DB is not available - Remove pymongo dependency (unused, replaced by PostgreSQL-based user storage) - Comment out unused MongoDB settings in Settings class - Update .gitignore: add API_CONTRACT_AGENT*.md and migratego/ - Remove .vscode/launch.json (superseded by uv run fastapi dev)19bd037
-
feat/Catalog Retrieval System (#1)
- initial commit (8557a2045ca2c0873e07efdf6227064b83e4a1e5) - initial structure (efc0c0a8b2444014cbe74f04617c3c974d9915be) - [KM-553] initialize shared contract (f31f6731692b0199d8cb5c5d29dffa466d6d4ca0) - [KM-553] migrate retrieval layer and remove obsolete rag/tools modules (df220ea47c96eb8a1efa07c6eebef4604fc308d6) - [NOTICKET] catalog enricher + structured pipeline + on_db_registered (77ea82d29d86400c5ed30ac154be8df6a7e897eb) - Merge commit 'refs/pr/1' of https://huggingface.co/spaces/DataEyond/Agentic-Service-Data-Eyond-Catalog into pr/1 (0ae8717c7efce2cda14fe4b8caecd774459b1f6e) - [NOTICKET] SqlCompiler (Postgres) + DbExecutor (212dad360714508c1aa607995039e6818fa40f78) - Merge commit 'refs/pr/1' of https://huggingface.co/spaces/DataEyond/Agentic-Service-Data-Eyond-Catalog into pr/1 (faf8a4f6d7e06ccf1ce84c2e7f4990c79f27f843) - TabularIntrospector + on_tabular_uploaded trigger + 31 tests (ca425208969b72a20ed76f86ef0faa1a0e912339) - [KM 556] add columns field to QueryResult + update…6bff5d9
-
[KM-438][KM-439] Improve Retrieval and Querying feature (#15)
- [noticket] add gitignore (c87f27f458854787102eebc110b178c2572ef8a6) - [NOTICKET]: add document pipeline, simplify document API (fb871f3d7db9c5a9c95a012761c4180a432e7e4b) - [NOTICKET]: update folder document_pipelines after pipelines (a4cf97ab3890e35831c0043a1d411c17d84f374b) - [NOTICKET][DB] refactor code to new repo (7f3bb9782bb9b6ace69e6be35377c12757a2f81d) - [KM-441] add mean and median (9b5933420821187e73bb72a896e28d7adf6d6137) - [NOTICKET] new metadata format for cleaner code (6b590d94c398ac4966a0e376f2809aa52c5ba022) - update document (5a69e0eecf80dd807565dbc369374e41450c37fc) - delete duplicate file (3848d7b262745b758069e3ab8089437ced42a45f) - edit document for new pipeline (425e021053c4de65e2598b0f5554b7c1044a904c) - [NOTICKET]: add CSV and XLSX file type (31920c3bc6f9faf9e07095a02fc1faf08adc246e) - [DB] fix/rename db_pipeline.py (d913315c356d741cc7a5a5b1f8e36ba41397f2ab) - [NOTICKET][DB] menyesuaikan format struktur db_pipeline sesuai dengan file lain (e13a9017e0f7fd26ef9b8a…c93ec90
-
Make a query for tabular (XLSX and CSV) (#14)
- [NOTICKET] add software to gitignore (d43ecb180036339cf287e93f1d9116bd6eff9b9d) - [NOTICKET] add pyarrow (e50eadc82e160012e3319267f5bfe084cd9034d4) - [KM-515][document] Make Query for Tabular Type (XLSX & CSV) (695ca0a154a68077c51499ed83b7507b988be065) - [KM-455][document] decided methods retrieval for document (8c9cc79223eb1d96c8622d129219a83a4ba2500b) Co-authored-by: Sofhia Az-Zahra <sofhiaazzhr@users.noreply.huggingface.co>8802920
-
Merge dev_new to main (improve retriever and add querying) (#13)
- [noticket] add gitignore (c87f27f458854787102eebc110b178c2572ef8a6) - [NOTICKET]: add document pipeline, simplify document API (fb871f3d7db9c5a9c95a012761c4180a432e7e4b) - [NOTICKET]: update folder document_pipelines after pipelines (a4cf97ab3890e35831c0043a1d411c17d84f374b) - [NOTICKET][DB] refactor code to new repo (7f3bb9782bb9b6ace69e6be35377c12757a2f81d) - [KM-441] add mean and median (9b5933420821187e73bb72a896e28d7adf6d6137) - [NOTICKET] new metadata format for cleaner code (6b590d94c398ac4966a0e376f2809aa52c5ba022) - update document (5a69e0eecf80dd807565dbc369374e41450c37fc) - delete duplicate file (3848d7b262745b758069e3ab8089437ced42a45f) - edit document for new pipeline (425e021053c4de65e2598b0f5554b7c1044a904c) - [NOTICKET]: add CSV and XLSX file type (31920c3bc6f9faf9e07095a02fc1faf08adc246e) - [DB] fix/rename db_pipeline.py (d913315c356d741cc7a5a5b1f8e36ba41397f2ab) - [NOTICKET][DB] menyesuaikan format struktur db_pipeline sesuai dengan file lain (e13a9017e0f7fd26ef9b8a…52999bc
-
[NOTICKET] fixed single source into multiple sources (#12)
- [NOTICKET] fixed single source into multiple sources (9d826225424dd8fb0044ce6f4a010ab8eac1075b) - [NOTICKET] remove source_hint line (d1f034edff5a6a78c0a7b9f3402ca2a46baa4f43) Co-authored-by: Sofhia Az-Zahra <sofhiaazzhr@users.noreply.huggingface.co>c8e3046 -
[NOTICKET] add updated_at metadata & prevent duplicate while adding new embedding (#10)
- [NOTICKET] add updated_at & delete old embedding before adding new embedding ingestion (da83e0111d3fd220fcfdd717018ce4794415e020)bd0029e
-
[NOTICKET] add total token logging (#9)
- [NOTICKET] add token logging (34eb194ed4c3f31ccd968390d1c05ca9547edafe)d973099
-
[KM-496] [DED][AI] Fix Response data structure for field sources (db flow)
5c4067e -
[NOTICKET] [document]: add doctypes endpoint (#7)
- [NOTIKCET][document]: add doctypes endpoint and 10MB file size limit (9694cc0635e4b59a0d6291e613d83262ee3277f3) Co-authored-by: Sofhia Az-Zahra <sofhiaazzhr@users.noreply.huggingface.co>b1f1ccd -
[NOTICKET][DB] add endpoints for dbtypes (#6)
- [noticket] add gitignore (c87f27f458854787102eebc110b178c2572ef8a6) - [NOTICKET]: add document pipeline, simplify document API (fb871f3d7db9c5a9c95a012761c4180a432e7e4b) - [NOTICKET]: update folder document_pipelines after pipelines (a4cf97ab3890e35831c0043a1d411c17d84f374b) - [NOTICKET][DB] refactor code to new repo (7f3bb9782bb9b6ace69e6be35377c12757a2f81d) - [KM-441] add mean and median (9b5933420821187e73bb72a896e28d7adf6d6137) - [NOTICKET] new metadata format for cleaner code (6b590d94c398ac4966a0e376f2809aa52c5ba022) - update document (5a69e0eecf80dd807565dbc369374e41450c37fc) - delete duplicate file (3848d7b262745b758069e3ab8089437ced42a45f) - edit document for new pipeline (425e021053c4de65e2598b0f5554b7c1044a904c) - [NOTICKET]: add CSV and XLSX file type (31920c3bc6f9faf9e07095a02fc1faf08adc246e) - [DB] fix/rename db_pipeline.py (d913315c356d741cc7a5a5b1f8e36ba41397f2ab) - [NOTICKET][DB] menyesuaikan format struktur db_pipeline sesuai dengan file lain (e13a9017e0f7fd26ef9b8a…c7d59cf
-
[KM-493] [DED][AI] Fix source extraction in chat
changes: - Add .vscode/launch.json with debugpy config for FastAPI via uvicorn on port 7860 - Fix _extract_sources in chat.py to read metadata from nested `data` key - Add debug log for extracted sources tikcet: https://bukittechnology.atlassian.net/browse/KM-4937da6db5 -
[KM-436][KM-437] add and modify for database and document pipeline (#3)
- [noticket] add gitignore (c87f27f458854787102eebc110b178c2572ef8a6) - [NOTICKET]: add document pipeline, simplify document API (fb871f3d7db9c5a9c95a012761c4180a432e7e4b) - [NOTICKET]: update folder document_pipelines after pipelines (a4cf97ab3890e35831c0043a1d411c17d84f374b) - [NOTICKET][DB] refactor code to new repo (7f3bb9782bb9b6ace69e6be35377c12757a2f81d) - [KM-441] add mean and median (9b5933420821187e73bb72a896e28d7adf6d6137) - [NOTICKET] new metadata format for cleaner code (6b590d94c398ac4966a0e376f2809aa52c5ba022) - update document (5a69e0eecf80dd807565dbc369374e41450c37fc) - delete duplicate file (3848d7b262745b758069e3ab8089437ced42a45f) - edit document for new pipeline (425e021053c4de65e2598b0f5554b7c1044a904c) - [NOTICKET]: add CSV and XLSX file type (31920c3bc6f9faf9e07095a02fc1faf08adc246e) - [DB] fix/rename db_pipeline.py (d913315c356d741cc7a5a5b1f8e36ba41397f2ab) - [NOTICKET][DB] menyesuaikan format struktur db_pipeline sesuai dengan file lain (e13a9017e0f7fd26ef9b8a…2ba0613
-
[NOTICKET] Init db_client endpoint
a7d8798 -
[KM-467] [DED][AI] Orchestration Agent - Init
- https://bukittechnology.atlassian.net/browse/KM-467027123c