# Backend Agentic Service API Contract **Last updated**: 2026-07-14 This document describes the Python agentic backend used by the frontend for AI chat, help/report tools, charts, and traceability data shown alongside chat answers. Base path examples use relative URLs. Configure the frontend with the deployed Python service base URL. ## Overview The Python backend owns the generative AI interaction surface: 1. Stream chat answers from the AI agent. 2. Execute tool-style actions for help and report generation. 3. Return report versions, report details, and report-readiness signals. 4. Return traceability for a completed assistant answer. 5. Return chart specifications produced for a completed assistant answer. The frontend uses this service during the analysis conversation flow: 1. User sends a chat message. 2. Frontend calls `POST /api/v2/chat/stream` and renders the streamed answer. 3. When the stream emits `done`, frontend uses the returned `message_id` as the assistant answer correlation id. 4. Frontend calls `GET /api/v1/traceability` for planning, tool calls, and source provenance. 5. Frontend calls `GET /api/v1/charts` with the same `message_id` and renders any returned charts under the answer. 6. Frontend calls `/api/v1/tools/help` for guided help and `/api/v1/tools/report` for report generation (the Generate-Report button; `GET …/readiness` drives the button state). ## Endpoint Summary | Method | Path | Purpose | | --- | --- | --- | | `POST` | `/api/v2/chat/stream` | Stream an AI chat answer for one analysis conversation. | | `GET` | `/api/v1/tools/list` | List available frontend tools. | | `POST` | `/api/v1/tools/help` | Stream contextual help for the current analysis conversation. | | `POST` | `/api/v1/tools/report` | Generate and persist a new report version. | | `GET` | `/api/v1/tools/report/{analysis_id}` | List report versions for an analysis. | | `GET` | `/api/v1/tools/report/{analysis_id}/records` | List analysis records for report curation. | | `GET` | `/api/v1/tools/report/{analysis_id}/readiness` | Report-readiness signal for the Generate-Report button. | | `GET` | `/api/v1/tools/report/{analysis_id}/{version}` | Retrieve one report version. | | `GET` | `/api/v1/traceability` | Retrieve provenance for one assistant answer. | | `GET` | `/api/v1/charts` | Retrieve chart(s) produced for one assistant answer. | ## Common Concepts ### Identifiers - `user_id`: user identifier passed by the frontend. - `analysis_id`: analysis conversation identifier. - `message_id`: assistant answer identifier generated by Python and returned in the stream `done` event; used to correlate chat streaming, Golang message persistence, traceability, and charts. It is a UUID string (e.g. `77f06761-0fdf-4cc5-84f8-5f81bcbb6f84`); the `msg_…` values in the examples below are illustrative placeholders only. Never generate or send it from the frontend. ### Server-Sent Events Chat and help endpoints return `text/event-stream`. Frontend should parse events by `event` name and `data` payload. Blank lines separate SSE events. Common event types: | Event | Data | Meaning | | --- | --- | --- | | `sources` | JSON array | Always `[]` — real sources moved to `GET /api/v1/traceability`. Event kept for backward compatibility. | | `status` | text | Optional progress update for slower paths. | | `chunk` | text | Answer text fragment. Concatenate chunks in order. | | `done` | JSON object | Terminal success event. Includes `message_id`. | | `error` | text | Terminal error event. Stream stops after this. | The stream carries answer text only. Planning, tool call details, and full provenance are fetched from `GET /api/v1/traceability` after the stream is done; charts are fetched from `GET /api/v1/charts`. The `done` event carries no chart hint — fetch `GET /api/v1/charts` unconditionally on every `done` (the response tells you if there is nothing to render). ## Chat ### `POST /api/v2/chat/stream` Streams an AI answer for one user message in an analysis conversation. The frontend must not generate or send `message_id`; Python generates it and returns it in the terminal `done` event. Request body: ```json { "user_id": "u_1a2b3c", "analysis_id": "an_42", "message": "What were total sales by region last quarter?" } ``` Fields: | Field | Required | Description | | --- | --- | --- | | `user_id` | Yes | User identifier. | | `analysis_id` | Yes | Analysis conversation identifier. | | `message` | Yes | User message text. | Response: `text/event-stream`. Example structured answer: ```text event: sources data: [] event: status data: Planning analysis... event: status data: Running 3 steps... event: chunk data: Total sales by region last quarter: event: chunk data: Central led at $1.21M (38%), East $0.74M, West $0.55M (down 12% QoQ). event: done data: {"message_id":"msg_88f1"} ``` Example simple chat answer: ```text event: sources data: [] event: chunk data: I'm your AI data analyst. Connect a source or ask a question to get started. event: done data: {"message_id":"msg_12"} ``` Behavior notes: - Greeting and farewell messages may use a fast canned path. - Stateless `chat` intent may use a 1-hour Redis response cache. - The router may classify messages into intents such as `chat`, `help`, `check`, `unstructured_flow`, `structured_flow`, or `out_of_scope`. - `sources` in the stream is **always `[]`** — read the real `sources[]` from `GET /api/v1/traceability` after `done`. - `status` events are optional and should be safe for the frontend to ignore. - When the user explicitly asks to plot/visualize ("show me a bar chart of…", "buatkan grafik…"), the answer text describes the result and the chart itself is delivered via `GET /api/v1/charts` — it is never embedded in `chunk` text. ## Tools ### `GET /api/v1/tools/list` Returns the deterministic list of tools available to the frontend. Request: none. Response `200`: ```json { "count": 1, "tools": [ { "command": "/help", "name": "help", "type": "skill", "description": "Show what the assistant can do and guide your next step." } ] } ``` The catalog is `/help` only. `/report` is not a slash command — report generation is a right-side **Generate** button; the button calls `POST /api/v1/tools/report`. Tool item shape: ```json { "command": "/help", "name": "help", "type": "skill", "description": "Show what the assistant can do and guide your next step." } ``` Frontend behavior: - Surface `/help` in the slash menu. - Surface report generation as a button or explicit UI action, driven by `GET /tools/report/{analysis_id}/readiness`. ### `POST /api/v1/tools/help` Streams contextual guidance for the current analysis conversation. Request body: ```json { "user_id": "u_1a2b3c", "analysis_id": "an_42" } ``` Response: `text/event-stream` using the same event shape as chat. Help responses usually emit `sources: []` and no `status` pings. Example: ```text event: sources data: [] event: chunk data: Your goal is set. You can start exploring now. Try a question like "average order value by month", then I can generate a report. event: done data: {"message_id":"msg_h7"} ``` ## Reports ### `POST /api/v1/tools/report` Generates, persists, and returns a new report version for an analysis. Query params: | Query | Required | Description | | --- | --- | --- | | `analysis_id` | Yes | Analysis identifier. | | `user_id` | Yes | User identifier. | | `exclude_record_ids` | No | Record ids to leave out of this version (repeat the param per id). Get ids from `GET /tools/report/{analysis_id}/records`. Excluded runs are listed in the report's "Excluded Analyses" section. Excluding every substantive record returns `409`. | Example: ```text POST /api/v1/tools/report?analysis_id=an_42&user_id=u_1a2b3c POST /api/v1/tools/report?analysis_id=an_42&user_id=u_1a2b3c&exclude_record_ids=rec_a1&exclude_record_ids=rec_c3 ``` Status codes: | Status | Meaning | | --- | --- | | `201` | New report version generated. | | `409` | Report floor/precondition not met. | | `500` | Generation or persistence failed. | Response `201`: ```json { "report_id": "8f3a2b1c9d4e4f6a8b0c1d2e3f4a5b6c", "analysis_id": "an_42", "user_id": "u_1a2b3c", "version": 2, "generated_at": "2026-06-30T09:14:33.512Z", "problem_statement": { "objective": "Understand which regions drive revenue and why Q1 dipped.", "business_questions": [ "Which regions contribute most to total revenue?", "Did any region decline quarter-over-quarter?" ] }, "record_ids": ["rec_a1", "rec_b2"], "executive_summary": "Revenue is concentrated in the Central region (38% of total). The West was the only region to contract, down 12% QoQ, the main driver of the Q1 dip.", "bq_answers": [ { "question": "Which regions contribute most to total revenue?", "answer": "The Central region leads with 38% of total revenue.", "status": "answered", "record_ids": ["rec_a1"] }, { "question": "Did any region decline quarter-over-quarter?", "answer": "Yes — the West region fell 12% QoQ.", "status": "answered", "record_ids": ["rec_b2"] } ], "findings": [ { "text": "Central region contributed 38% of total revenue, the largest share.", "record_ids": ["rec_a1"], "supporting_data": null }, { "text": "West region revenue fell 12% quarter-over-quarter.", "record_ids": ["rec_b2"], "supporting_data": null } ], "caveats": [ { "text": "March data for the East region was partially missing, around 6% of rows.", "record_ids": ["rec_b2"] } ], "open_questions": [ { "text": "What drove the West region's QoQ decline?", "record_ids": ["rec_b2"] } ], "unresolved": [ { "text": "Correlate churn with tenure — churn column not found in the source.", "record_ids": ["rec_d4"] } ], "excluded": [], "evidence_tables": { "rec_a1": [ { "title": "Aggregate revenue by region", "columns": ["region", "total_revenue"], "rows": [["Central", "18321"], ["West", "9954"]], "truncated": false } ] }, "charts": { "rec_a1": [ { "schema": "dataeyond.chart.v1", "chart_type": "bar", "title": "Revenue by region", "plotly": { "data": [{ "type": "bar", "x": ["Central", "East", "West"], "y": [1210000, 740000, 550000], "name": "revenue" }], "layout": { "title": { "text": "Revenue by region" }, "xaxis": { "title": { "text": "region" } }, "yaxis": { "title": { "text": "revenue" } } } } } ] }, "data_sources": [ { "source_id": "src_sales_db", "name": "orders", "source_type": "postgres", "detail": { "tables": ["orders"], "row_count": 48213, "columns": ["region", "amount", "ordered_at"] } } ], "method_steps": [ { "task_id": "t1", "stage": "data_understanding", "objective": "Inventory the sales source", "status": "success", "tools_used": ["check_data"] }, { "task_id": "t2", "stage": "modeling", "objective": "Aggregate revenue by region", "status": "success", "tools_used": ["analyze_aggregate"] } ], "rendered_markdown": "# Analysis Report\n\n*Generated 2026-06-30 by u_1a2b3c*\n\n## Objective\nUnderstand which regions drive revenue..." } ``` Field notes: - `bq_answers` — one entry per business question. `status` is `answered` | `partial` | `unanswered`; `record_ids` cite the backing analyses. Written in the analysis's language (Indonesian objective → Indonesian answers). - `unresolved` — runs that were attempted but produced no usable evidence. Not part of the findings body. - `excluded` — runs the caller excluded via `exclude_record_ids`. - `evidence_tables` — `record_id` → small result tables copied from the run's stored outputs (max 3 tables per record, max 10 rows each; `truncated: true` when rows were capped). Rendered as markdown tables under the matching Key Findings group in `rendered_markdown`. - `charts` — `record_id` → chart envelopes (same `dataeyond.chart.v1` shape as `GET /api/v1/charts` returns; max 3 per record) copied from the run's stored outputs. `rendered_markdown` contains an `## EDA` section where each chart appears as a fenced block: ````text ```plotly { "schema": "dataeyond.chart.v1", "chart_type": "bar", "title": "…", "plotly": { "data": [ … ], "layout": { … } } } ``` ```` The fence content is the full envelope — parse it and render `Plotly.newPlot(el, parsed.plotly.data, parsed.plotly.layout)`. A bold caption line (the chart title) precedes each fence. Precondition: - Reports require at least one completed analysis record for the session. A run counts when an analysis step succeeded **or** a chart was produced — a chart-only session can generate a report. - If slow-path analysis recording is disabled, report generation can return `409` by design. Response `409`: ```json { "detail": "Not ready to generate a report - still needs at least one completed analysis." } ``` ### `GET /api/v1/tools/report/{analysis_id}` Lists report versions for one analysis, oldest first. Response `200`: ```json [ { "report_id": "1b2c3d4e", "version": 1, "generated_at": "2026-06-24T15:02:11Z", "record_count": 1 }, { "report_id": "8f3a2b1c", "version": 2, "generated_at": "2026-06-25T09:14:33Z", "record_count": 2 } ] ``` If no reports exist, returns `[]`. ### `GET /api/v1/tools/report/{analysis_id}/records` Lists the persisted analysis runs a report would be built from, oldest first. The frontend shows this before generating so the user can deselect runs; the chosen ids go to `POST /tools/report` as `exclude_record_ids`. Response `200`: ```json [ { "record_id": "rec_a1", "goal_restated": "Rank regions by total revenue", "created_at": "2026-06-30T08:55:02Z", "substantive": true, "findings_count": 2 }, { "record_id": "rec_d4", "goal_restated": "Correlate churn with tenure", "created_at": "2026-06-30T09:01:47Z", "substantive": false, "findings_count": 1 } ] ``` `substantive: false` means the run produced no usable result (no analysis step or chart succeeded) — that run is listed in the report's `unresolved` JSON field rather than the findings body. If no runs exist, returns `[]`. ### `GET /api/v1/tools/report/{analysis_id}/readiness` Deterministic report-readiness signal for the Generate-Report button — the same producer as Help's readiness signal, so the button and Help never disagree. Response `200`: ```json { "ready": false, "missing": ["a new analysis since the last report"] } ``` Note: `POST /tools/report` itself only enforces the floor (at least one completed analysis). The delta gap in `missing` is a soft warning the frontend can surface ("nothing new since the last report") without blocking the button. ### `GET /api/v1/tools/report/{analysis_id}/{version}` Returns one report version. Shape is the same as the `201` response from `POST /api/v1/tools/report`. Response `404`: ```json { "detail": "No report v3 for analysis 'an_42'." } ``` ## Charts A chart is produced when the user explicitly asks to plot/visualize something in chat. The chart is never embedded in the streamed text — it is fetched separately after the stream completes, then rendered with plotly.js under the assistant message. ### `GET /api/v1/charts` Returns every chart produced during one assistant answer. Call this after the chat stream emits `done`, using the `message_id` from the `done` event (same fetch-on-`done` pattern as traceability). Chart rows are written before `done`, so there is no polling race. Every response is HTTP `200` — branch on the `status` field, not the HTTP code. Query params: | Query | Required | Description | | --- | --- | --- | | `message_id` | Yes | Assistant answer identifier returned by the stream's `done` event. | Example: ```text GET /api/v1/charts?message_id=88f10c3a-6f03-4204-bf98-41ffc20388b2 ``` Response `200` — `status: "success"` (≥1 chart to render): ```json { "status": "success", "message": "1 chart(s) for this message.", "count": 1, "charts": [ { "chart_id": "3fbd8e2e-8e21-4d4b-9b21-9e6b6a0a6a6e", "chart_type": "bar", "title": "Revenue by region", "spec": { "schema": "dataeyond.chart.v1", "chart_type": "bar", "title": "Revenue by region", "plotly": { "data": [{ "type": "bar", "x": ["Central", "East", "West"], "y": [1210000, 740000, 550000], "name": "revenue" }], "layout": { "title": { "text": "Revenue by region" }, "xaxis": { "title": { "text": "region" } }, "yaxis": { "title": { "text": "revenue" } } } } }, "created_at": "2026-07-13T03:21:09.114Z" } ] } ``` Response `200` — `status: "empty"` (the turn completed but produced no charts; the common case, not an error): ```json { "status": "empty", "message": "This message completed without producing charts.", "count": 0, "charts": [] } ``` Response `200` — `status: "not_found"` (no completed turn is known for this `message_id`; usually a stale or mistyped id): ```json { "status": "not_found", "message": "No completed turn is known for this message_id.", "count": 0, "charts": [] } ``` The `dataeyond.chart.v1` envelope (the shape of `charts[].spec`): ```json { "schema": "dataeyond.chart.v1", "chart_type": "bar", "title": "Revenue by region", "plotly": { "data": [{ "type": "bar", "x": ["A", "B"], "y": [1, 2], "name": "revenue" }], "layout": { "title": { "text": "Revenue by region" } } } } ``` Field rules: - `status` is the outcome marker: `success` | `empty` | `not_found`. Branch on it; do not parse `message` (human-readable, for logs only). - `spec` is the full envelope, unmodified — render straight from it: `Plotly.newPlot(el, spec.plotly.data, spec.plotly.layout)`. - `chart_type` / `title` are copied out of `spec` for convenience (list rendering without parsing `spec`); `title` may be `null`. Chart types: `bar`, `line`, `pie`, `scatter`. - A turn can produce more than one chart; `charts` is ordered by creation time. Frontend rendering guidance: - Fetch unconditionally on every `done`; `status: "empty"` means render nothing extra. - Render each chart under the assistant message it belongs to. - Treat `status: "not_found"` as a signal worth logging (stale id or fetch bug) — not as a user-facing error. - Chart iteration is a follow-up chat turn (e.g. "make it a line chart") — there is no separate edit endpoint. - The same envelope shape appears inside report markdown as ` ```plotly ` fenced blocks (see Reports → `charts`), so one renderer can serve both surfaces. ## Traceability ### `GET /api/v1/traceability` Returns user-facing provenance for one assistant answer. The frontend should call this after the chat/help stream emits `done`, using the `message_id` from the `done` event. The row is written **before** `done`, so an immediate GET returns `200` (no polling race). A `404` means the id is unknown or the turn errored before completing (error turns never produce a row). Query params: | Query | Required | Description | | --- | --- | --- | | `analysis_id` | Yes | Analysis identifier. | | `message_id` | Yes | Assistant answer identifier returned by the stream. | Example: ```text GET /api/v1/traceability?analysis_id=an_42&message_id=msg_88f1 ``` `intent` values the frontend may see: `chat` · `help` · `check` · `unstructured_flow` · `structured_flow` · `out_of_scope` · `blocked` (`blocked` = input-guard or content-filter refusal; `chat` also covers the greeting fast-path and cache replays). Field rules: - `planning`: present only when the planner ran (`structured_flow`); otherwise `null`. - `thinking`: **always `null` in v1** — the field stays in the payload so it can be populated later without a contract change. - `tool_calls`: every invoked tool with `summary` (plain-English one-liner), `input`, `output`, `status`, `task_id` (nullable), and `error` (nullable); empty for chat / help / greeting / refusal paths. `input`/`output` are the raw tool I/O (opaque ids) — render them in a collapsible "technical details" section, not the headline; use `summary` for the headline. - A `render_chart` tool call reports a compact chart summary in `output` (`chart_type`, `title`, `trace_count`, `point_count`) — the full spec is served by `GET /api/v1/charts`, not here. - `data_used`: one entry per structured data pull, resolved to **real names** for display (empty when no structured pull ran). Split into `columns_read` (columns read straight from the user's data, each tagged with its `roles`) and `output_columns` (`kind: "column"` = read from data, `kind: "computed"` = calculated, carrying a `formula` and no id). Also carries `tables` (all touched, including join targets), `joins`, `filters` (with a plain-language `description`), `group_by`, `order_by`, `limit`, `rows_returned`, and the executed `query`. - **`id` fields are machine-only.** Every `id` in `data_used` (`source.id`, `tables[].id`, `columns_read[].id`) is for linking/audit — **the frontend must never render it.** Show `name` (qualified as `table.name`). A `computed` output column has no id by design. - `sources`: required for retrieval flows; empty for chat / help / refusal paths and for `check`. Database sources also carry `source_name` (the DB's real name) and `tables` (every table touched). - Summaries and filter descriptions are built from fixed templates, never an LLM — traceability adds no latency and cannot hallucinate. - The payload also carries an internal `user_id` (ownership); the frontend may ignore it. - Truncation: `preview` ≤ 5 rows; any string inside `input`/`output`/`preview`/`snippet` ≤ 300 chars (executed `query` ≤ 2000); rows beyond the preview are dropped (`row_count` is preserved). Response `200` for `structured_flow`: ```json { "analysis_id": "an_42", "message_id": "msg_88f1", "user_id": "user_7", "intent": "structured_flow", "generated_at": "2026-07-06T03:21:09.114Z", "planning": { "goal_restated": "Find which regions drive revenue and why Q1 dipped.", "assumptions": [], "steps": [ { "step": 1, "stage": "data_understanding", "objective": "Inventory the sales source", "status": "success", "tools_used": ["check_data"] }, { "step": 2, "stage": "modeling", "objective": "Aggregate revenue by region", "status": "success", "tools_used": ["retrieve_data", "analyze_aggregate"] } ] }, "thinking": null, "tool_calls": [ { "order": 1, "task_id": null, "name": "check_data", "summary": "Inspected your data source structure", "input": { "source_hint": "structured" }, "output": { "kind": "table", "columns": ["source_id", "name", "source_type", "table_count"], "row_count": 1, "preview": [["src_sales_db", "orders", "schema", 1]] }, "status": "success", "error": null }, { "order": 2, "task_id": null, "name": "retrieve_data", "summary": "Retrieved 4 rows across 2 columns from orders", "input": { "ir": { "source_id": "src_sales_db", "table_id": "orders", "select": ["region", "amount"], "group_by": ["region"] } }, "output": { "kind": "table", "columns": ["region", "total"], "row_count": 4, "preview": [["Central", 1210000], ["East", 740000]] }, "status": "success", "error": null } ], "data_used": [ { "source": { "id": "src_sales_db", "name": "sales db", "type": "schema" }, "tables": [ { "id": "orders", "name": "orders", "role": "base" } ], "joins": [], "columns_read": [ { "id": "c_region", "name": "region", "table": "orders", "data_type": "string", "pii": false, "roles": ["selected", "grouped"] }, { "id": "c_amount", "name": "amount", "table": "orders", "data_type": "decimal", "pii": false, "roles": ["aggregated"] } ], "output_columns": [ { "name": "region", "kind": "column", "from": "orders.region" }, { "name": "total", "kind": "computed", "from": "orders.amount", "formula": "SUM(orders.amount)" } ], "filters": [], "group_by": ["orders.region"], "order_by": [], "limit": null, "rows_returned": 4, "query": "SELECT region, SUM(amount) AS total FROM orders GROUP BY region" } ], "sources": [ { "type": "database", "source_id": "src_sales_db", "source_name": "sales db", "name": "orders", "tables": ["orders"], "query": "SELECT region, SUM(amount) AS total FROM orders GROUP BY region", "detail": { "table": "orders", "row_count": 4 } } ] } ``` > Note: `retrieve_data`'s real `input` is the compiled query IR under an `ir` key (the planner builds an IR, never raw SQL). The executed SQL/rendered query appears on the corresponding `sources[].query`. Response `200` for `unstructured_flow`: ```json { "analysis_id": "an_42", "message_id": "msg_55", "user_id": "user_7", "intent": "unstructured_flow", "generated_at": "2026-07-06T03:40:02.001Z", "planning": null, "thinking": null, "tool_calls": [ { "order": 1, "task_id": null, "name": "retrieve_knowledge", "input": { "query": "technology stack used in this project" }, "output": { "kind": "documents", "row_count": 4 }, "status": "success", "error": null } ], "sources": [ { "type": "document", "document_id": "doc_7", "filename": "tech_handbook.pdf", "page_label": "12", "query": "technology stack used in this project", "snippet": "The backend is built on FastAPI with async SQLAlchemy...", "score": 0.83 } ] } ``` Response `200` for chat / greeting / help / refusals (`out_of_scope`, `blocked`): ```json { "analysis_id": "an_42", "message_id": "msg_12", "user_id": "user_7", "intent": "chat", "generated_at": "2026-07-06T03:05:00.000Z", "planning": null, "thinking": null, "tool_calls": [], "sources": [] } ``` Response `404`: ```json { "detail": "No traceability for message 'msg_88f1' yet." } ``` Frontend rendering guidance: - Render traceability separately from the streamed answer. - Default state can be collapsed. - Show planning, tool calls, and sources as separate sections. - Treat `planning: null`, `tool_calls: []`, and `sources: []` as valid states.