Commit History
-
[NOTICKET] fix: F-5 — isolate DB threads; stop treating session hardening as best-effort
Rifqi Hafizuddin +1: Claude Opus 4.8`asyncio.wait_for` cancels the awaiting COROUTINE. The `to_thread` worker underneath is not cancellable — it runs to completion, holding a thread and a connection on the CUSTOMER's database long after we returned "timed out" to the user. Degrading their production database is our liability, and it is invisible from our side because we already answered. (a) Dedicated bounded pool. DB work moves off `asyncio.to_thread` (which always uses the shared default executor) onto its own ThreadPoolExecutor(50, "dbexec") via `run_in_executor`. Abandoned workers previously accumulated in a pool of min(32, cpu_count+4) shared with every other to_thread caller in the process — notably the tabular Parquet loader — so a handful of slow customer queries could stall unrelated work service-wide. Sized to the engine cache's own ceiling (_MAX_ENGINES=50 x _POOL_SIZE=1): more threads than that cannot make progress anyway. Timeout semantics are unchanged. (b) Session hardening is no longer best-effort — and…f3025d4 -
[NOTICKET] fix: review cheap batch — F-19 SSE order, F-22/F-24 analysis_id, F-26, F-16, F-4
Rifqi Hafizuddin +1: Claude Opus 4.8Six low-risk review findings, batched. No behaviour anyone depends on is removed. F-19 — SSE order/presence did not match the contract. `check` and router-`help` emitted NO `sources` event at all, and `structured_flow` emitted `status` BEFORE `sources`, inverting the documented order on exactly the turns that take longest. A frontend that initializes per-turn state on `sources` (documented always-present) therefore never initialized on check/help turns and initialized late on slow ones — every one of those a plausible FE bug that would have been blamed on the frontend. Note `stream_help` (the /tools/help endpoint) already emitted it, so the two help paths disagreed with each other. Purely additive; `sources` stays `[]`. F-22 — `analysis_id` now 422s unless it parses as a UUID, on BOTH live endpoints (chat and help kept deliberately identical — two endpoints taking the same field should not disagree on what is valid). It was never usable: `analyses.id` is `uuid NOT NULL`, so a non-UUI…cbc8d6a
-
[NOTICKET] feat: bound catalog render + blob read; scope charts/traceability; mark degraded seams
Rifqi Hafizuddin +1: Claude Opus 4.8DEV_PLAN #40, #41, #42. #41 F-13 — TabularExecutor buffered an entire Parquet blob and read_parquet'd it with no size check. Filtering and the 10k row cap both happen after the frame exists, so they bound the RESULT, never the working set. An OOM here is the one failure in this service that escapes every never-throw seam: the process dies, taking every concurrent request with it. Neither storage backend could report an object size, so `object_size()` is added to both (S3 head_object -> ContentLength, Azure get_blob_properties().size), each returning None rather than raising. The executor now refuses >500 MB BEFORE downloading, with a post-download byte check as the fallback when the probe is unavailable — that one cannot prevent the download, but it does stop read_parquet from multiplying the bytes into a frame several times larger. #41 F-12 — CatalogSummary.render() had no truncation at all; 400 tables x 30 cols measured ~241k tokens per planner call, x3 retries. Two ceilings now, w…0cb7d53 -
[NOTICKET] fix: compiler parity — F-17 fabricated column, F-18 LIKE nulls, F-25 empty in/not_in
Rifqi Hafizuddin +1: Claude Opus 4.8Three execution-verified findings from CODE_REVIEW_2026-07-23 that had no DEV_PLAN row — the tracker went from #39 straight to #40 and lost them. Same IR must not mean two different things depending on whether the source is a database or a file. F-17 (High) — the bare-select check was gated on `if ir.group_by`, so a select mixing a bare column with an aggregate and group_by=[] passed validation. Postgres then rejects it loudly, but the pandas path silently DROPS the column while `output_columns` (built from the select list, not the result) still advertises it. `data_access` maps rows by name, so the missing name becomes None: the user gets a real-looking table whose first column is entirely em-dash, presented as data. Wrong answers presented as correct is the worst outcome for this product. The check now fires whenever any aggregate is present. No false positives are possible — a mixed select with no group_by is invalid SQL in every dialect — and the planner's retry loop self-correct…8f736f9
-
/fix change model to gpt 5.4 mini (#17)
- [NOTICKET] chore: rename LLM env quad __4o -> __54m (GPT-5.4-mini) (790ef0c7aaac6e76eb01b0ef9a3398ff4249416d)d65c41d -
fix/ check and new chart tool (#16)
- check: table-level column drill + drop misleading "name a table" hint (ef0c7b1c5aa49a244b02f1b5eea15cc77dcf2bcb) - [KM-715] charts: render_chart tool + message_charts store + GET /api/v1/charts + planner viz slice (87cfcf884d923ee4bcd722e0e4ef4d90d7eaedca) - [KM-715] checkpoint: S1a deterministic quality checkpoint between runner and assembler (f3c8de34dc19a573031b67fd4f48dcfbadcdfed8) - [KM-715] docs: sync REPO_STATUS/DEV_PLAN + land SPINE_V2_PLAN (W1/W2 done) (434eacbf2ac8324188ebe5e32775dbc9d2a3f49a) - planner: value-handoff (${tN.col} → list) for tabular lookup/anti-join (54070ac1407b751000e49dfe7f102a2be5ab1ff8) - [KM-715] charts: GET /charts by message_id only + tri-state status marker (b8d15761addf4d499c9631df64e7693aa1d61fa2) - [KM-715] report: embed charts as plotly EDA fences; render_chart counts toward the floor (89db8be6b6f11e86ad0776d2ee07f2306e3b6079) - [KM-715] docs: contract + status deltas for the charts lead-review round (96347ee7c55a4eb39a906409ec214463e25c5e35) - …5a60e93
-
/fix planner and report (#13)
- [KM-644] planner: infeasible plan outcome + entity-ranking few-shots (089b69d9b6111e813307c6afbd3e7613bb823f9d) - [KM-644] harden query path: grouped-select rule, int-date trends, sample decode, trace guard (74ad5261736bda0bd72bccdb41565ea67313993a) - [KM-703] add analyze_merge tool: two-table combine (analytics.combine) (8abf635532366d670367557a3ffa03d42371d831) - [KM-567] planner support for analyze_merge: data_right guard + two-retrieve merge few-shot (a029a843247e6ea4d3c79a4e2c1b7571ba4cda69) - [KM-644] report v2: BQ answers, unresolved/excluded sections, evidence tables + curation/readiness endpoints (4ceb058d85c5a2a289443b9411d2b345027fa226) - [NOTICKET] refactor: mint message_id as UUID to mirror analyses_messages id shape (4253733b6b8db0449a82442a915c732ddd55f528) - [KM-644] report: drop Notes/Unresolved/Method sections from rendered markdown (compact report) (eef6de811a139402c79fd74ad4adfc00cf7aff01) - /edit md file (8198cc188ff05a2798ae2056b90b6e56e1bea87b) Co-authored-by…49b0848
-
/feat traceability (#11)
- [NOTICKET] Fix check skill: reply language + tabular inventory routing (1367f41269cf9ef42fd91f311b24d4edf5885c7b) - [KM-691] Develop traceability (23cc2079989fef931a6e146980fe6bf528ee6e92) - [NOTICKET] Humanize check output: hide UUIDs, add lead-in, list format (2bdda8fc3fc4b4e17b4df7fb53aee5abede45add) - [NOTICKET] check: schema-detail view for column/type questions (9e248d9cc5688d6ee622be523a6bedddf5010995) - [KM-691] Make sources trace-authoritative; drop stream sources (b43ecc77f26c1537cbbfddb6800ec4ff62d16441) - [KM-691] Bound traceability payload + reject contradictory IR filter ranges (db6db73d6ab0c2175447946742038443d67b2f78) - /fix planner can now plan quarter comparison (839e32c8d6d5895029072074ab7aba8bb4fcdb6e) Co-authored-by: Sofhia Az-Zahra <sofhiaazzhr@users.noreply.huggingface.co>f873f92
-
/fix validator and report (#10)
- /fix report: remove record_id hash citations, EDA (for future tool) change to data source, transform to mdx-safe (b1b101730ccf5628e43e80f2ad654f78fbed5207) - [NOTICKET] fix(planner): repair near-miss catalog ids in QueryIR before validation (aacd4d13def258a9815761aecbd7b246a8dae9fc) - /fix report: group findings by analysis, dedup, code-span-safe MDX escaping (029b48894e9f79af6c0d81c00a2d811a982f87ae) - [NOTICKET] fix(tools): log swallowed analytics tool errors in AnalyticsToolInvoker (93d9381b2a5f5efe058c913fe0b793ce994a9097) - fix(planner): harden plan validation to reduce retry thrash on structured questions (1619dd3c0fa74825010c7af6d4b375e3b9404847) - [NOTICKET] fix(tools): coerce numeric-string columns before analyze_* compute (6bda333866a1a8ccf1b3fe6fcee7a8e1832f28d7) - fix(planner): correct top-N guidance — in-query ranking IS supported (99aa4141aed8241a625d290fb170f74192a6d831) - feat(guardrails): add input guard, out_of_scope intent, and refusal handling (0bbc27bc40ba20e7a8e…0e5fdb5
-
/fix analysis + report_inputs and storage update (#9)
- fix(chat): drop ENABLE_SLOW_PATH flag + make chat persistence read-only (b8b76c4fb8db21de183cc6f80c0acba0d35bdad1) - [NOTICKET] feat(storage): read tabular Parquet from Supabase S3 via provider toggle (3f48226adaf2ed107a4eb2c0373cbb73fc62f552) Co-authored-by: Sofhia Az-Zahra <sofhiaazzhr@users.noreply.huggingface.co>079782d -
/fix check and help tool (#8)
- /fix data_catalog model + CatalogStore to dedorch schema (data → catalog_payload) (09857fc52486354f53e752e3ea329b064a9e4c4b) - [NOTICKET] fix(help): lock reply language to user's language on button path (cb7f5ac484ad4a510e68faf5291091958b6d2f42) - /fix default SKIP_INIT_DB=true for dedorch cutover (skip init_db on Go-owned schema) (781b93c6b8744209a1101629b5170175e9212592) - /fix surface catalog foreign keys to the planner (FK-backed joins) (446c57524e932fcdbb0ddecc049189c443b6ae97) - [NOTICKET] feat(eval): add help-skill eval harness + goal-based language fallback (cbdbefb7ab0ffa36949713f18422213db480a959) - [NOTICKET] fix(eval): update readiness eval to post-KM-652 floor (drop problem_validated) (6f46e81bdeb1eb987f9dac8af0e22a975474cbb1) - /fix infer FKs for FK-less dedorch catalog + log repr(e) in db executor (3669290474583ae011f5ed5e50b94dad146fbc77) - Merge eval/help + readiness eval fix into PR #8 (5f4818a8e38f93988ab5756f3a06c94645826b15) - /docs REPO_STATUS: FK inference (FK-…3743cfe
-
feat/Knowledge & Data Tools (#3)
- [KM-641] refactor(tools): trim analytics registry to 4 active tools (66e2e4df66f1abf7eaa7e442d9900d428696ece4) - [NOTICKET] docs: add checkpoint plan for 2026-06-17 sync (8c1bb63ebdee02f9db0da9fc2b2269823b0a7688) - [KM-642] refactor(tools): rename data-access tools to data/knowledge naming (c38c0c29d3e6b245f67d03f2c35e451900e1abdb) - [KM-643] feat(tools): merge data sources into data_check + add knowledge_check (4bd5f1efdc560a64092cc8df672a7990fa949b59) - [NOTICKET] docs: record KM-642/643 in checkpoint plan + PROGRESS (4c57588b90fe0cac8d0d4d22cfe5701fc9eb52dd) - [NOTICKET] fix(chat): scope response cache by user_id + drop TTL to 1h (R5) (b701e95d300c7d2480a3608b2d4da6dfe102b19a) - [NOTICKET] feat(slow-path): persist AnalysisRecord via PostgresAnalysisStore (09f845188226dd01633cf74525caa569e4a7d37a) - [NOTICKET] Adopt verb-first skill naming (2d6406def7ed45edd1226e96645ba2a5cdbfa3d0) - [NOTICKET] docs: sync checkpoint plan — verb-first tool names + report-as-API (3b399f84d73b0d98bd99…0721bb4
-
feat/Planner Agent (#2)
- [KM-567][AI] Planner agent: contracts, schemas, inputs, registry (be96601ad46c00e8e789768dd6e89c2db827bde4) - [KM-567][AI] Planner agent: prompt + few-shot examples (83ba6b1743370145454d9315224eeb7b2d12ac18) - [KM-567][AI] Planner agent: validator + service (72306d0ce92c76d60117a20c701c7f8185bcfeef) - [KM-567] docs: record Phase 3 Planner agent in PROGRESS.md (ba2fa8892392028c8d13646c09d1155bbf34e703) - [KM-567][AI] Planner agent: fix PII leak + placeholder/retry robustness (6b3637dda097f16cd98d4b9ad6094fdaed6d1346) - [KM-624] Implement Compute Functions Tools (78c598c22d2f2a14229a83c3f43fa98c5d520ae7) - [KM-624][AI] Planner: realign stub registry + examples to composite analyze_* tools (93a24dabf48d8121e981cc470c835cb10badb62a) - [KM-624] docs: record tool taxonomy + ownership revision in PROGRESS (f3461142542f6216aff23b30a1321dfb4654afdc) - [KM-626][AI] Slow-path agent: seam contracts + TaskRunner (ac310de551d0888ccfef31e981e22d88f54c6a38) - [KM-626][AI] Slow-path agent: Assembler…81e5fe7
-
feat/Catalog Retrieval System (#1)
- initial commit (8557a2045ca2c0873e07efdf6227064b83e4a1e5) - initial structure (efc0c0a8b2444014cbe74f04617c3c974d9915be) - [KM-553] initialize shared contract (f31f6731692b0199d8cb5c5d29dffa466d6d4ca0) - [KM-553] migrate retrieval layer and remove obsolete rag/tools modules (df220ea47c96eb8a1efa07c6eebef4604fc308d6) - [NOTICKET] catalog enricher + structured pipeline + on_db_registered (77ea82d29d86400c5ed30ac154be8df6a7e897eb) - Merge commit 'refs/pr/1' of https://huggingface.co/spaces/DataEyond/Agentic-Service-Data-Eyond-Catalog into pr/1 (0ae8717c7efce2cda14fe4b8caecd774459b1f6e) - [NOTICKET] SqlCompiler (Postgres) + DbExecutor (212dad360714508c1aa607995039e6818fa40f78) - Merge commit 'refs/pr/1' of https://huggingface.co/spaces/DataEyond/Agentic-Service-Data-Eyond-Catalog into pr/1 (faf8a4f6d7e06ccf1ce84c2e7f4990c79f27f843) - TabularIntrospector + on_tabular_uploaded trigger + 31 tests (ca425208969b72a20ed76f86ef0faa1a0e912339) - [KM 556] add columns field to QueryResult + update…6bff5d9
-
[KM-438][KM-439] Improve Retrieval and Querying feature (#15)
- [noticket] add gitignore (c87f27f458854787102eebc110b178c2572ef8a6) - [NOTICKET]: add document pipeline, simplify document API (fb871f3d7db9c5a9c95a012761c4180a432e7e4b) - [NOTICKET]: update folder document_pipelines after pipelines (a4cf97ab3890e35831c0043a1d411c17d84f374b) - [NOTICKET][DB] refactor code to new repo (7f3bb9782bb9b6ace69e6be35377c12757a2f81d) - [KM-441] add mean and median (9b5933420821187e73bb72a896e28d7adf6d6137) - [NOTICKET] new metadata format for cleaner code (6b590d94c398ac4966a0e376f2809aa52c5ba022) - update document (5a69e0eecf80dd807565dbc369374e41450c37fc) - delete duplicate file (3848d7b262745b758069e3ab8089437ced42a45f) - edit document for new pipeline (425e021053c4de65e2598b0f5554b7c1044a904c) - [NOTICKET]: add CSV and XLSX file type (31920c3bc6f9faf9e07095a02fc1faf08adc246e) - [DB] fix/rename db_pipeline.py (d913315c356d741cc7a5a5b1f8e36ba41397f2ab) - [NOTICKET][DB] menyesuaikan format struktur db_pipeline sesuai dengan file lain (e13a9017e0f7fd26ef9b8a…c93ec90
-
Make a query for tabular (XLSX and CSV) (#14)
- [NOTICKET] add software to gitignore (d43ecb180036339cf287e93f1d9116bd6eff9b9d) - [NOTICKET] add pyarrow (e50eadc82e160012e3319267f5bfe084cd9034d4) - [KM-515][document] Make Query for Tabular Type (XLSX & CSV) (695ca0a154a68077c51499ed83b7507b988be065) - [KM-455][document] decided methods retrieval for document (8c9cc79223eb1d96c8622d129219a83a4ba2500b) Co-authored-by: Sofhia Az-Zahra <sofhiaazzhr@users.noreply.huggingface.co>8802920
-
Merge dev_new to main (improve retriever and add querying) (#13)
- [noticket] add gitignore (c87f27f458854787102eebc110b178c2572ef8a6) - [NOTICKET]: add document pipeline, simplify document API (fb871f3d7db9c5a9c95a012761c4180a432e7e4b) - [NOTICKET]: update folder document_pipelines after pipelines (a4cf97ab3890e35831c0043a1d411c17d84f374b) - [NOTICKET][DB] refactor code to new repo (7f3bb9782bb9b6ace69e6be35377c12757a2f81d) - [KM-441] add mean and median (9b5933420821187e73bb72a896e28d7adf6d6137) - [NOTICKET] new metadata format for cleaner code (6b590d94c398ac4966a0e376f2809aa52c5ba022) - update document (5a69e0eecf80dd807565dbc369374e41450c37fc) - delete duplicate file (3848d7b262745b758069e3ab8089437ced42a45f) - edit document for new pipeline (425e021053c4de65e2598b0f5554b7c1044a904c) - [NOTICKET]: add CSV and XLSX file type (31920c3bc6f9faf9e07095a02fc1faf08adc246e) - [DB] fix/rename db_pipeline.py (d913315c356d741cc7a5a5b1f8e36ba41397f2ab) - [NOTICKET][DB] menyesuaikan format struktur db_pipeline sesuai dengan file lain (e13a9017e0f7fd26ef9b8a…52999bc