Commit History
-
[NOTICKET] perf(slow-path): DB3 pre-connect + R4 progress events
Rifqi Hafizuddin +1: Claude Opus 4.8DB3 (DB owner): DbExecutor.prewarm(catalog, user_id) warms the pooled engine for schema sources, fired (best-effort, never raises) at slow-path entry so the cold first-query TCP+TLS+auth handshake overlaps the ~4s Planner LLM call. Gated to the default path (skipped when a coordinator factory is injected, so tests stay hermetic). Reuses DbExecutor's existing client resolution + the DB2 engine cache. R4 (agent): SlowPathCoordinator.run gained an optional `progress` callback; ChatHandler bridges it to SSE `status` events via an asyncio queue, and chat.py forwards them. The slow path no longer streams ~13s of silence — live timeline now shows Planning -> Running N steps -> Composing, max wire gap ~4.6s, fixing proxy idle-timeout + UX. Status events only appear when the coordinator calls back, so the existing wiring test's event sequence is unchanged (fake coordinator just accepts **kwargs). Token-streaming the Assembler answer is deferred: it would require splitting the Assembler into a …735c421 -
[NOTICKET][DB] perf(query): DB2 - pooled engine cache for user DBs
Rifqi Hafizuddin +1: Claude Opus 4.8DbExecutor built a fresh engine and disposed it on every query, paying a full TCP+TLS+auth handshake per call (~6-8s measured, dominating slow-path latency). engine_scope's connect-once-dispose is right for ingestion, wrong for the query path. - New UserEngineCache (src/database_client/engine.py): process-wide cache of pooled engines keyed by client_id + a hash of the decrypted creds (rotation auto- invalidates the key). Bounded LRU (50) + 600s idle TTL; pool_pre_ping + pool_recycle=300; small pool (1 + 2 overflow). invalidate(client_id) disposes eagerly. Scope: postgres/supabase only; other db_types keep the legacy per-call path, so nothing regresses. - DbExecutor._run_sync reuses the warm connection (no per-query SET, no dispose). - Read-only + statement_timeout moved from libpq startup `options` to a per- connection `connect` event: Neon's transaction pooler rejects default_transaction_read_only as a startup parameter (caught in a live run) but accepts it as a SET. …4c911cb -
[NOTICKET][DB] fix(query): R2 - always compile a bounded LIMIT
Rifqi Hafizuddin +1: Claude Opus 4.8An IR with no limit compiled to a LIMIT-less SELECT, and DbExecutor materialized the entire result set before capping at 10k rows - an unbounded fetch that can OOM on a large user table. - SqlCompiler now bounds every query: an explicit IR limit is honored (clamped to MAX_RESULT_ROWS=10000); an unbounded query gets LIMIT cap+1 so the executor can distinguish exactly-the-cap from more-rows-existed and flag truncation. - CompiledSql.row_cap carries the effective cap; DbExecutor caps + derives truncated from it and drops its own _ROW_HARD_CAP constant. - test_sql.py: no-limit golden strings get the safety bound; +3 cases (no-limit bounded, explicit-limit clamp, row_cap). Restored S608 to tests/** ruff ignore (documented in PR3-DB but missing from pyproject) - golden tests assert literal SQL. Also logs R13: a pre-existing, unrelated test_prompt failure found while running R2. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>6974c5b
-
feat/Catalog Retrieval System (#1)
- initial commit (8557a2045ca2c0873e07efdf6227064b83e4a1e5) - initial structure (efc0c0a8b2444014cbe74f04617c3c974d9915be) - [KM-553] initialize shared contract (f31f6731692b0199d8cb5c5d29dffa466d6d4ca0) - [KM-553] migrate retrieval layer and remove obsolete rag/tools modules (df220ea47c96eb8a1efa07c6eebef4604fc308d6) - [NOTICKET] catalog enricher + structured pipeline + on_db_registered (77ea82d29d86400c5ed30ac154be8df6a7e897eb) - Merge commit 'refs/pr/1' of https://huggingface.co/spaces/DataEyond/Agentic-Service-Data-Eyond-Catalog into pr/1 (0ae8717c7efce2cda14fe4b8caecd774459b1f6e) - [NOTICKET] SqlCompiler (Postgres) + DbExecutor (212dad360714508c1aa607995039e6818fa40f78) - Merge commit 'refs/pr/1' of https://huggingface.co/spaces/DataEyond/Agentic-Service-Data-Eyond-Catalog into pr/1 (faf8a4f6d7e06ccf1ce84c2e7f4990c79f27f843) - TabularIntrospector + on_tabular_uploaded trigger + 31 tests (ca425208969b72a20ed76f86ef0faa1a0e912339) - [KM 556] add columns field to QueryResult + update…6bff5d9
-
[KM-438][KM-439] Improve Retrieval and Querying feature (#15)
- [noticket] add gitignore (c87f27f458854787102eebc110b178c2572ef8a6) - [NOTICKET]: add document pipeline, simplify document API (fb871f3d7db9c5a9c95a012761c4180a432e7e4b) - [NOTICKET]: update folder document_pipelines after pipelines (a4cf97ab3890e35831c0043a1d411c17d84f374b) - [NOTICKET][DB] refactor code to new repo (7f3bb9782bb9b6ace69e6be35377c12757a2f81d) - [KM-441] add mean and median (9b5933420821187e73bb72a896e28d7adf6d6137) - [NOTICKET] new metadata format for cleaner code (6b590d94c398ac4966a0e376f2809aa52c5ba022) - update document (5a69e0eecf80dd807565dbc369374e41450c37fc) - delete duplicate file (3848d7b262745b758069e3ab8089437ced42a45f) - edit document for new pipeline (425e021053c4de65e2598b0f5554b7c1044a904c) - [NOTICKET]: add CSV and XLSX file type (31920c3bc6f9faf9e07095a02fc1faf08adc246e) - [DB] fix/rename db_pipeline.py (d913315c356d741cc7a5a5b1f8e36ba41397f2ab) - [NOTICKET][DB] menyesuaikan format struktur db_pipeline sesuai dengan file lain (e13a9017e0f7fd26ef9b8a…c93ec90
-
Make a query for tabular (XLSX and CSV) (#14)
- [NOTICKET] add software to gitignore (d43ecb180036339cf287e93f1d9116bd6eff9b9d) - [NOTICKET] add pyarrow (e50eadc82e160012e3319267f5bfe084cd9034d4) - [KM-515][document] Make Query for Tabular Type (XLSX & CSV) (695ca0a154a68077c51499ed83b7507b988be065) - [KM-455][document] decided methods retrieval for document (8c9cc79223eb1d96c8622d129219a83a4ba2500b) Co-authored-by: Sofhia Az-Zahra <sofhiaazzhr@users.noreply.huggingface.co>8802920
-
Merge dev_new to main (improve retriever and add querying) (#13)
- [noticket] add gitignore (c87f27f458854787102eebc110b178c2572ef8a6) - [NOTICKET]: add document pipeline, simplify document API (fb871f3d7db9c5a9c95a012761c4180a432e7e4b) - [NOTICKET]: update folder document_pipelines after pipelines (a4cf97ab3890e35831c0043a1d411c17d84f374b) - [NOTICKET][DB] refactor code to new repo (7f3bb9782bb9b6ace69e6be35377c12757a2f81d) - [KM-441] add mean and median (9b5933420821187e73bb72a896e28d7adf6d6137) - [NOTICKET] new metadata format for cleaner code (6b590d94c398ac4966a0e376f2809aa52c5ba022) - update document (5a69e0eecf80dd807565dbc369374e41450c37fc) - delete duplicate file (3848d7b262745b758069e3ab8089437ced42a45f) - edit document for new pipeline (425e021053c4de65e2598b0f5554b7c1044a904c) - [NOTICKET]: add CSV and XLSX file type (31920c3bc6f9faf9e07095a02fc1faf08adc246e) - [DB] fix/rename db_pipeline.py (d913315c356d741cc7a5a5b1f8e36ba41397f2ab) - [NOTICKET][DB] menyesuaikan format struktur db_pipeline sesuai dengan file lain (e13a9017e0f7fd26ef9b8a…52999bc