Commit History
-
[KM-652] refactor(db): align analysis-family models to the dedorch schema
Rifqi Hafizuddin +1: Claude Opus 4.8Repoint our SQLAlchemy models to the Go-owned dedorch tables (Go owns migrations; we are consumer-only). Base tables already matched; only the analysis family differed. - analysis_states -> `analysis`: table rename; id/report_id -> Postgres uuid. - analysis_data_sources -> `data_sources`: richer shape (id, type, name, reference_id, bound_by, bound_at, metadata, created_at). binding_store reads `reference_id`; /analysis/create snapshots each source's type+name from the catalog (fail-open). `metadata` mapped as `source_metadata` (reserved name). - analysis_reports -> `reports`: flatten to title + content (rendered markdown) + generated_at + version; drop user_id/jsonb data. ReportStore writes title+content and rebuilds a minimal AnalysisReport on read (rendered_markdown = content; structured fields empty — markdown-only per the 2026-06-23 checkpoint). Deferred (pending lead): analysis_records has no dedorch table, so the report generator / record persistence / readiness flo…4f0ff12 -
[NOTICKET] fix(api): un-break Swagger /docs (OpenAPI schema generation)
analysis.py had 'from __future__ import annotations', which turned the CreateAnalysisRequest body annotation into a string forward-ref. FastAPI then mis-classified it as a query param and OpenAPI schema generation raised PydanticUserError, 500-ing /openapi.json and bricking the whole Swagger UI. Removed the future import (all annotations here are native py312); /docs, /openapi.json, /redoc now return 200. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>e7f2703 -
[NOTICKET] feat(api): add list_tools endpoint (GET /api/v1/tools)
Static slash-command catalog (10 commands) for the Golang backend to cache and surface on the FE without calling the agent per request. Each command maps 1:1 to a real internal intent/tool name (the dispatch key); display command names follow Harry's verb-first kebab-case convention. Mirrors the 15/06 list; deferred analytics tools excluded. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>4b7010a
-
[KM-652] fix(report,chat): align report gate with readiness + cleanup
Rifqi Hafizuddin +1: Claude Opus 4.8- T-D: generate_report only 409'd on zero records — it ignored problem_validated and whether any analysis actually succeeded, so the button could 201 a findings-empty report while Help said "not ready". Extract report_floor() (validated goal + >=1 substantive analyze_* success) as the single source of truth; is_report_ready composes it (+ its delta-check), and the report API gates on it (no delta — a new version is always allowed, decision 4A). - T-E: document that reports require ENABLE_SLOW_PATH=on (only the slow path persists AnalysisRecords; no records => 409 by design), and return a clear "still needs ..." 409 instead of a bare "no analyses recorded". - T-F: drop the dead source_hint derivation + _intent_to_source_hint — the intent event is consumed by the endpoint's cache gate and never forwarded to the frontend; fix the misleading comment. - T-G: document the history-blind chat cache key (message-only) limitation; history-aware key deferred to the endpoint…9c06bea -
[NOTICKET] fix(chat): cache only the stateless 'chat' intent
The response cache was keyed on (room_id, user_id, message) with no analysis- state/data version and checked before the gate, so a repeated message replayed a stale answer and could bypass the gate (e.g. the same analytical question asked pre- and post-validation returned the cached 'define your goal' redirect). Now the write is gated on the intent the handler already emits: only 'chat' (stateless) is cached; help/problem_statement/check/structured/unstructured are never cached, so they always reflect current state/data. Read path unchanged (safe by construction — only 'chat' is ever stored). Caching analysis answers needs proper invalidation on data/state change — deferred. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>e58cc23
-
[KM-652] feat(analysis): GET /analysis/{id} + GET /analysis (read endpoints)
Rifqi Hafizuddin +1: Claude Opus 4.8The FE workspace + sidebar need to read analysis state back; only create existed. Add: - GET /analysis/{id} — full state (8 fields) + bound data_source_ids; 404 if absent - GET /analysis?user_id= — a user's analyses, most-recently-updated first (summary) Extract _serialize_state so create + detail share one payload shape. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>4c5cff8 -
[KM-652] feat(analysis): #10 data-source binding (analysis-owned join table)
Rifqi Hafizuddin +1: Claude Opus 4.8Persist which catalog sources an analysis is scoped to, and consume it: - New analysis_data_sources(analysis_id, source_id) join table (Option A: analysis-owned, leaves the 8-field state contract untouched). Written atomically with state + room in /analysis/create. - AnalysisDataSourceStore read seam. structured_flow scopes its catalog to the bound sources (chat_handler._scope_to_binding); the report's data appendix scopes _build_data_sources to them. Both fail-open: no binding / legacy room / read error -> whole catalog, so nothing breaks until the FE sends ids. - /analysis/create dedupes + returns data_source_ids. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>4c143c4 -
[KM-652] fix(report): connect record->report->state loop + readiness refinement
Rifqi Hafizuddin +1: Claude Opus 4.8Three subsystems were built in parallel and never wired together: - Stamp analysis_id (+ user_id) onto the persisted AnalysisRecord in the slow path. Without it records were orphaned (analysis_id NULL) so list_for_analysis found nothing — the report API and is_report_ready were both silently blind. - Report API now loads the validated problem statement from analysis_states (was an empty stub) and writes report_id back onto the state after persist (feeds Help's has_report + the readiness delta-check). Both never-throw. - is_report_ready: count *substantive* records (>=1 finding, not raw turns) and add delta-since-report (a prior report + no newer analysis -> not ready). Also fix stale docstrings made false by this change: get_analysis_state (no longer a stub — reads the real store), the slow-path AnalysisStore seam (no longer a no-op), and SlowPathCoordinator (now wired behind ENABLE_SLOW_PATH). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>53d197f
-
[KM-652] feat(orchestrator): analysis state + gate wiring + problem_statement skill
Rifqi Hafizuddin +1: Claude Opus 4.8Wire the gated analysis flow end-to-end. The gate is OFF by default (ENABLE_GATE) so live structured_flow is unchanged until the frontend creates analyses. - Analysis State: `analysis_states` table (shared id with `rooms` — analysis_id == room_id), `AnalysisStateStore` (get/create/update), `get_analysis_state` seam (fails closed to not-validated). 8-field contract: id, analysis_title, problem_statement, problem_validated, owner_id, report_id, created_at, updated_at. - POST /api/v1/analysis/create — creates Room + analysis_states atomically with a shared id; data-first check. - gate.py: rule keys on `problem_validated`; stub fixtures for build/test. - problem_statement skill: LLM drafts/refines, deterministic validator decides the pass, writes problem_statement + problem_validated to the store. - chat_handler: thread `analysis_id` (= room_id); apply gate before dispatch behind `enable_gate` (redirect structured_flow -> problem_statement until validated). - settings: ENABL…adcc635
-
[KM-644] feat(report): button-triggered report generator + API
Rifqi Hafizuddin +1: Claude Opus 4.8A dedicated report service (not a chat skill/agent): reads a session's persisted AnalysisRecords + Problem Statement, makes ONE LLM call (the executive summary only), assembles everything else verbatim from records (INV-4), renders markdown, and persists an immutable versioned artifact. - src/agents/report/: schemas (AnalysisReport, DataSourceRef, ReportFinding + AttributedNote with plural record_ids, ProblemStatement stub), errors, ReportGenerator (deterministic assembly + 1 LLM call w/ fallback on failure), ReportStore (per-analysis advisory-lock versioning; unique(analysis_id,version)). - analysis_reports table (jsonb, one immutable row per version) + init_db register. - API: POST /report (generate), GET /report/{analysis_id} (list versions), GET /report/{analysis_id}/{version} (fetch); models/api/report.py; main.py wiring. - prompt: config/prompts/report_summary.md (synthesis-only exec summary). - record hygiene: Assembler caps raw retrieve_data table rows (kind="table") i…1570c5d
-
[NOTICKET] fix(chat): scope response cache by user_id + drop TTL to 1h (R5)
Rifqi Hafizuddin +1: Claude Fable 5Cache key was chat:{room_id}:{message} with a 24h TTL - no user in the key, so one user's cached answer could be replayed to another. Urgent now that Redis is a shared Cloud instance across the team. - key -> chat:{room_id}:{user_id}:{message} via _chat_cache_key() helper (room_id stays first so the room-wide clear pattern is unchanged) - TTL 86400 -> 3600 per the 2026-06-11 checkpoint decision - DELETE /chat/cache now requires user_id (frontend heads-up) - still open from R5: catalog-version in key / invalidate-on-ingest Docs: checkpoint plan + PROGRESS updated - Redis Cloud done, planner tool list verified, R3 closed as won't-do (team decision: tests stay local), Rifqi owns analysis_records/AnalysisStore + generate_report. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>b701e95
-
feat/Planner Agent (#2)
- [KM-567][AI] Planner agent: contracts, schemas, inputs, registry (be96601ad46c00e8e789768dd6e89c2db827bde4) - [KM-567][AI] Planner agent: prompt + few-shot examples (83ba6b1743370145454d9315224eeb7b2d12ac18) - [KM-567][AI] Planner agent: validator + service (72306d0ce92c76d60117a20c701c7f8185bcfeef) - [KM-567] docs: record Phase 3 Planner agent in PROGRESS.md (ba2fa8892392028c8d13646c09d1155bbf34e703) - [KM-567][AI] Planner agent: fix PII leak + placeholder/retry robustness (6b3637dda097f16cd98d4b9ad6094fdaed6d1346) - [KM-624] Implement Compute Functions Tools (78c598c22d2f2a14229a83c3f43fa98c5d520ae7) - [KM-624][AI] Planner: realign stub registry + examples to composite analyze_* tools (93a24dabf48d8121e981cc470c835cb10badb62a) - [KM-624] docs: record tool taxonomy + ownership revision in PROGRESS (f3461142542f6216aff23b30a1321dfb4654afdc) - [KM-626][AI] Slow-path agent: seam contracts + TaskRunner (ac310de551d0888ccfef31e981e22d88f54c6a38) - [KM-626][AI] Slow-path agent: Assembler…81e5fe7
-
[KM-582][DED][AI] Fix Retrieval in Agentic Service
ticket: https://bukittechnology.atlassian.net/browse/KM-582 fix: replace LangChain ORM retrieval with raw SQL and fix pgvector collection name - Rewrite DocumentRetriever.retrieve() using raw SQL cosine/manhattan queries instead of LangChain PGVector ORM, bypassing asyncpg type-mapping issues (id String vs UUID column, jsonb_path_match binding quirks) - Fix _COLLECTION_NAME from "document_embeddings" to "documents" to match the collection name set by the Golang ingestion service - Fix collection_name in vector_store.py to match consistently - Fix Redis chat cache to store {response, sources} dict so cached replies also populate message_sources table - Add cache management endpoints: DELETE /chat/cache, /chat/cache/room/{id}, /retrieval/cache/{user_id} - Invalidate retrieval cache automatically after document processing - Update intent_router prompt: route topical/knowledge questions to unstructured even without explicit document mention; prefer unstructured when ambiguous…61c746f
-
feat/Catalog Retrieval System (#1)
- initial commit (8557a2045ca2c0873e07efdf6227064b83e4a1e5) - initial structure (efc0c0a8b2444014cbe74f04617c3c974d9915be) - [KM-553] initialize shared contract (f31f6731692b0199d8cb5c5d29dffa466d6d4ca0) - [KM-553] migrate retrieval layer and remove obsolete rag/tools modules (df220ea47c96eb8a1efa07c6eebef4604fc308d6) - [NOTICKET] catalog enricher + structured pipeline + on_db_registered (77ea82d29d86400c5ed30ac154be8df6a7e897eb) - Merge commit 'refs/pr/1' of https://huggingface.co/spaces/DataEyond/Agentic-Service-Data-Eyond-Catalog into pr/1 (0ae8717c7efce2cda14fe4b8caecd774459b1f6e) - [NOTICKET] SqlCompiler (Postgres) + DbExecutor (212dad360714508c1aa607995039e6818fa40f78) - Merge commit 'refs/pr/1' of https://huggingface.co/spaces/DataEyond/Agentic-Service-Data-Eyond-Catalog into pr/1 (faf8a4f6d7e06ccf1ce84c2e7f4990c79f27f843) - TabularIntrospector + on_tabular_uploaded trigger + 31 tests (ca425208969b72a20ed76f86ef0faa1a0e912339) - [KM 556] add columns field to QueryResult + update…6bff5d9
-
[KM-438][KM-439] Improve Retrieval and Querying feature (#15)
- [noticket] add gitignore (c87f27f458854787102eebc110b178c2572ef8a6) - [NOTICKET]: add document pipeline, simplify document API (fb871f3d7db9c5a9c95a012761c4180a432e7e4b) - [NOTICKET]: update folder document_pipelines after pipelines (a4cf97ab3890e35831c0043a1d411c17d84f374b) - [NOTICKET][DB] refactor code to new repo (7f3bb9782bb9b6ace69e6be35377c12757a2f81d) - [KM-441] add mean and median (9b5933420821187e73bb72a896e28d7adf6d6137) - [NOTICKET] new metadata format for cleaner code (6b590d94c398ac4966a0e376f2809aa52c5ba022) - update document (5a69e0eecf80dd807565dbc369374e41450c37fc) - delete duplicate file (3848d7b262745b758069e3ab8089437ced42a45f) - edit document for new pipeline (425e021053c4de65e2598b0f5554b7c1044a904c) - [NOTICKET]: add CSV and XLSX file type (31920c3bc6f9faf9e07095a02fc1faf08adc246e) - [DB] fix/rename db_pipeline.py (d913315c356d741cc7a5a5b1f8e36ba41397f2ab) - [NOTICKET][DB] menyesuaikan format struktur db_pipeline sesuai dengan file lain (e13a9017e0f7fd26ef9b8a…c93ec90
-
Merge dev_new to main (improve retriever and add querying) (#13)
- [noticket] add gitignore (c87f27f458854787102eebc110b178c2572ef8a6) - [NOTICKET]: add document pipeline, simplify document API (fb871f3d7db9c5a9c95a012761c4180a432e7e4b) - [NOTICKET]: update folder document_pipelines after pipelines (a4cf97ab3890e35831c0043a1d411c17d84f374b) - [NOTICKET][DB] refactor code to new repo (7f3bb9782bb9b6ace69e6be35377c12757a2f81d) - [KM-441] add mean and median (9b5933420821187e73bb72a896e28d7adf6d6137) - [NOTICKET] new metadata format for cleaner code (6b590d94c398ac4966a0e376f2809aa52c5ba022) - update document (5a69e0eecf80dd807565dbc369374e41450c37fc) - delete duplicate file (3848d7b262745b758069e3ab8089437ced42a45f) - edit document for new pipeline (425e021053c4de65e2598b0f5554b7c1044a904c) - [NOTICKET]: add CSV and XLSX file type (31920c3bc6f9faf9e07095a02fc1faf08adc246e) - [DB] fix/rename db_pipeline.py (d913315c356d741cc7a5a5b1f8e36ba41397f2ab) - [NOTICKET][DB] menyesuaikan format struktur db_pipeline sesuai dengan file lain (e13a9017e0f7fd26ef9b8a…52999bc
-
[NOTICKET] fixed single source into multiple sources (#12)
- [NOTICKET] fixed single source into multiple sources (9d826225424dd8fb0044ce6f4a010ab8eac1075b) - [NOTICKET] remove source_hint line (d1f034edff5a6a78c0a7b9f3402ca2a46baa4f43) Co-authored-by: Sofhia Az-Zahra <sofhiaazzhr@users.noreply.huggingface.co>c8e3046
-
[KM-496] [DED][AI] Fix Response data structure for field sources (db flow)
5c4067e -
[NOTICKET] [document]: add doctypes endpoint (#7)
- [NOTIKCET][document]: add doctypes endpoint and 10MB file size limit (9694cc0635e4b59a0d6291e613d83262ee3277f3) Co-authored-by: Sofhia Az-Zahra <sofhiaazzhr@users.noreply.huggingface.co>b1f1ccd -
[NOTICKET][DB] add endpoints for dbtypes (#6)
- [noticket] add gitignore (c87f27f458854787102eebc110b178c2572ef8a6) - [NOTICKET]: add document pipeline, simplify document API (fb871f3d7db9c5a9c95a012761c4180a432e7e4b) - [NOTICKET]: update folder document_pipelines after pipelines (a4cf97ab3890e35831c0043a1d411c17d84f374b) - [NOTICKET][DB] refactor code to new repo (7f3bb9782bb9b6ace69e6be35377c12757a2f81d) - [KM-441] add mean and median (9b5933420821187e73bb72a896e28d7adf6d6137) - [NOTICKET] new metadata format for cleaner code (6b590d94c398ac4966a0e376f2809aa52c5ba022) - update document (5a69e0eecf80dd807565dbc369374e41450c37fc) - delete duplicate file (3848d7b262745b758069e3ab8089437ced42a45f) - edit document for new pipeline (425e021053c4de65e2598b0f5554b7c1044a904c) - [NOTICKET]: add CSV and XLSX file type (31920c3bc6f9faf9e07095a02fc1faf08adc246e) - [DB] fix/rename db_pipeline.py (d913315c356d741cc7a5a5b1f8e36ba41397f2ab) - [NOTICKET][DB] menyesuaikan format struktur db_pipeline sesuai dengan file lain (e13a9017e0f7fd26ef9b8a…c7d59cf
-
[KM-493] [DED][AI] Fix source extraction in chat
changes: - Add .vscode/launch.json with debugpy config for FastAPI via uvicorn on port 7860 - Fix _extract_sources in chat.py to read metadata from nested `data` key - Add debug log for extracted sources tikcet: https://bukittechnology.atlassian.net/browse/KM-4937da6db5 -
[KM-436][KM-437] add and modify for database and document pipeline (#3)
- [noticket] add gitignore (c87f27f458854787102eebc110b178c2572ef8a6) - [NOTICKET]: add document pipeline, simplify document API (fb871f3d7db9c5a9c95a012761c4180a432e7e4b) - [NOTICKET]: update folder document_pipelines after pipelines (a4cf97ab3890e35831c0043a1d411c17d84f374b) - [NOTICKET][DB] refactor code to new repo (7f3bb9782bb9b6ace69e6be35377c12757a2f81d) - [KM-441] add mean and median (9b5933420821187e73bb72a896e28d7adf6d6137) - [NOTICKET] new metadata format for cleaner code (6b590d94c398ac4966a0e376f2809aa52c5ba022) - update document (5a69e0eecf80dd807565dbc369374e41450c37fc) - delete duplicate file (3848d7b262745b758069e3ab8089437ced42a45f) - edit document for new pipeline (425e021053c4de65e2598b0f5554b7c1044a904c) - [NOTICKET]: add CSV and XLSX file type (31920c3bc6f9faf9e07095a02fc1faf08adc246e) - [DB] fix/rename db_pipeline.py (d913315c356d741cc7a5a5b1f8e36ba41397f2ab) - [NOTICKET][DB] menyesuaikan format struktur db_pipeline sesuai dengan file lain (e13a9017e0f7fd26ef9b8a…2ba0613
-
[NOTICKET] Init db_client endpoint
a7d8798 -
[KM-467] [DED][AI] Orchestration Agent - Init
- https://bukittechnology.atlassian.net/browse/KM-467027123c