Agentic-Service-Data-Eyond-Catalog

Running

Agentic-Service-Data-Eyond-Catalog / src /agents /planner /examples.py
/fix demo prep (#25)
4a5f122 7 days ago
65.7 kB
"""Few-shot examples for the planner prompt.
Illustrative (question -> TaskList) pairs that teach the OUTPUT SHAPE:
stages, dependency edges, ordered tool-call chains, inline QueryIR,
"${t<id>}" placeholders, and the assumed data-flow convention — `retrieve_data`
pulls rows, then a composite `analyze_*` tool consumes them via a `data` placeholder
referencing the upstream result's column aliases (Pattern A; the tool team may
instead pick self-fetch by `source_id`, in which case these examples are reshaped
to match — see registry.py). They reference a hypothetical sales catalog
(`src_sales` / `t_orders`); these ids are part of the illustration and are not
validated against the user's real catalog. v1 is descriptive/diagnostic — no
modeling tasks.
**Exemplar balance (2026-09-17).** In-context learning over-predicts whatever a
prompt's examples show most and shows last (majority-label and recency bias —
Zhao et al., ICML 2021), so the mix here is deliberate, not incidental:
- **Question language is mixed EN/ID on purpose — do NOT translate these back to
  English.** Real traffic and the `eval/planner` dataset are majority Indonesian
  (17 ID / 14 EN); an all-English exemplar set teaches the planner that Indonesian
  questions are out-of-distribution.
- **Plan internals stay English regardless of question language** — `goal_restated`,
  `objective`, `success_criteria` are internal artifacts. The ONE field that follows
  the user's language is `infeasible_reason` (see `prompt.py`: "If (and only if) you
  return an infeasible plan, write `infeasible_reason` in {reply_language}").
  Example H demonstrates that pairing; keep it demonstrated.
- **Don't stack near-identical exemplars.** Only D still shows the
  `retrieve_data -> analyze_aggregate` shape (A and E aggregate in the IR since
  2026-10-22); adding more on one shape and
  the same fictional domain makes that plan the model's default answer to everything.
  A new example should earn its place by teaching a shape or a rule no other one does.
- **Feasible/infeasible pairs are load-bearing.** G↔H and J↔K each pair a capability
  with its refusal so an explicit ask can't talk the planner past a missing column.
  Keep both halves, and keep each pair's phrasing parallel.
See AGENT_ARCHITECTURE_CONTEXT_new.md §7.3 (Examples A and B).
"""

from __future__ import annotations

from .schemas import Task, TaskList, ToolCall

# --------------------------------------------------------------------------- #
# Example A — exploratory, no modeling.
# "Which product categories drove last quarter's revenue?"
# Shows: a total per category is computed INSIDE the retrieve_data IR (group_by +
# sum, ordered) — one row per category over ALL matching orders. It used to pull
# raw rows and sum them in analyze_aggregate, which caps at 10,000 rows, so on a
# larger table the "total" was the total of an arbitrary slice (EMA dry run
# 2026-10-22: "lowest PA per equipment" averaged 10,000 of 26,305 rows).
# --------------------------------------------------------------------------- #

_EXAMPLE_A = TaskList(
    plan_id="example_a",
    goal_restated="Identify which product categories contributed most to last quarter's revenue.",
    assumptions=["'last quarter' = 2026-01-01 to 2026-03-31."],
    open_questions=[],
    tasks=[
        Task(
            id="t1",
            stage="data_preparation",
            objective="Total last quarter's revenue per category, largest first.",
            tool_calls=[
                ToolCall(
                    tool="retrieve_data",
                    args={
                        "ir": {
                            "source_id": "src_sales",
                            "table_id": "t_orders",
                            "select": [
                                {"kind": "column", "column_id": "c_category", "alias": "category"},
                                {
                                    "kind": "agg",
                                    "fn": "sum",
                                    "column_id": "c_revenue",
                                    "alias": "total_revenue",
                                },
                            ],
                            "filters": [
                                {
                                    "column_id": "c_order_date",
                                    "op": "between",
                                    "value": ["2026-01-01", "2026-03-31"],
                                    "value_type": "date",
                                }
                            ],
                            "group_by": ["c_category"],
                            "order_by": [{"column_id": "total_revenue", "dir": "desc"}],
                        }
                    },
                )
            ],
            expected_output="category_revenue",
            success_criteria="Produced total revenue per category, one row each, largest first.",
            depends_on=[],
            estimated_cost="low",
        ),
    ],
)

# --------------------------------------------------------------------------- #
# Example B — descriptive / trend.
# "How has monthly revenue trended by region this year, and what's unusual?"
# --------------------------------------------------------------------------- #

_EXAMPLE_B = TaskList(
    plan_id="example_b",
    goal_restated="Describe this year's monthly revenue trend and flag unusual months.",
    assumptions=["'this year' starts 2026-01-01."],
    open_questions=["'Unusual' is interpreted as months far from the typical monthly revenue."],
    tasks=[
        Task(
            id="t1",
            stage="data_preparation",
            objective="Pull this year's order dates, revenue, and region.",
            tool_calls=[
                ToolCall(
                    tool="retrieve_data",
                    args={
                        "ir": {
                            "source_id": "src_sales",
                            "table_id": "t_orders",
                            "select": [
                                {
                                    "kind": "column",
                                    "column_id": "c_order_date",
                                    "alias": "order_date",
                                },
                                {"kind": "column", "column_id": "c_revenue", "alias": "revenue"},
                                {"kind": "column", "column_id": "c_region", "alias": "region"},
                            ],
                            "filters": [
                                {
                                    "column_id": "c_order_date",
                                    "op": ">=",
                                    "value": "2026-01-01",
                                    "value_type": "date",
                                }
                            ],
                            "limit": 10000,
                        }
                    },
                )
            ],
            expected_output="ytd_rows",
            success_criteria="Produced this year's order-level rows with date, revenue, region.",
            depends_on=[],
            estimated_cost="medium",
        ),
        Task(
            id="t2",
            stage="evaluation",
            objective="Bucket revenue into months and summarize the trend and movement.",
            tool_calls=[
                ToolCall(
                    tool="analyze_trend",
                    args={
                        "data": "${t1}",
                        "date_column": "order_date",
                        "value_column": "revenue",
                        "freq": "month",
                        "agg": "sum",
                    },
                )
            ],
            expected_output="monthly_trend",
            success_criteria=(
                "Produced a per-month revenue series with direction and change rate to "
                "flag months above/below the typical level."
            ),
            depends_on=["t1"],
            estimated_cost="low",
        ),
    ],
)


# --------------------------------------------------------------------------- #
# Example C — mixed structured + unstructured.
# "Revenue dipped in Q1 — what happened?"
# Shows: a structured branch (query -> analyze_trend) runs alongside an
# INDEPENDENT retrieve_knowledge branch that pulls qualitative context. Note
# retrieve_knowledge takes a natural-language `query` (NOT a `${t<id>}` data
# placeholder — it is a source, not a consumer) and can run in parallel; the
# Assembler folds the document context into the explanation.
# --------------------------------------------------------------------------- #

_EXAMPLE_C = TaskList(
    plan_id="example_c",
    goal_restated="Explain Q1's revenue dip using both the numbers and the qualitative record.",
    assumptions=["'Q1' = 2026-01-01 to 2026-03-31."],
    open_questions=[],
    tasks=[
        Task(
            id="t1",
            stage="data_preparation",
            objective="Pull Q1 order dates and revenue.",
            tool_calls=[
                ToolCall(
                    tool="retrieve_data",
                    args={
                        "ir": {
                            "source_id": "src_sales",
                            "table_id": "t_orders",
                            "select": [
                                {
                                    "kind": "column",
                                    "column_id": "c_order_date",
                                    "alias": "order_date",
                                },
                                {"kind": "column", "column_id": "c_revenue", "alias": "revenue"},
                            ],
                            "filters": [
                                {
                                    "column_id": "c_order_date",
                                    "op": "between",
                                    "value": ["2026-01-01", "2026-03-31"],
                                    "value_type": "date",
                                }
                            ],
                            "limit": 10000,
                        }
                    },
                )
            ],
            expected_output="q1_rows",
            success_criteria="Produced Q1 order rows with date and revenue.",
            depends_on=[],
            estimated_cost="medium",
        ),
        Task(
            id="t2",
            stage="evaluation",
            objective="Summarize the Q1 monthly revenue trend to locate the dip.",
            tool_calls=[
                ToolCall(
                    tool="analyze_trend",
                    args={
                        "data": "${t1}",
                        "date_column": "order_date",
                        "value_column": "revenue",
                        "freq": "month",
                        "agg": "sum",
                    },
                )
            ],
            expected_output="q1_trend",
            success_criteria="Produced a per-month revenue series showing where revenue fell.",
            depends_on=["t1"],
            estimated_cost="low",
        ),
        Task(
            id="t3",
            stage="data_understanding",
            objective="Retrieve qualitative context on Q1 operational events behind a dip.",
            tool_calls=[
                ToolCall(
                    tool="retrieve_knowledge",
                    args={
                        "query": "operational issues, outages, or notable events in Q1 2026",
                        "top_k": 5,
                    },
                )
            ],
            expected_output="q1_context_chunks",
            success_criteria="Produced relevant document chunks about Q1 operations.",
            depends_on=[],
            estimated_cost="low",
        ),
    ],
)


# --------------------------------------------------------------------------- #
# Example D — group-by aggregation (analyze_aggregate arg shape).
# "Berapa median dan rata-rata nilai order per region?"  [ID — see the exemplar
# balance note in the module docstring; the plan itself stays English.]
# Shows the EXACT analyze_aggregate args: `aggregations` is an OBJECT mapping each
# column to a LIST of functions ({"revenue": ["median", "mean"]}), and `group_by` is a
# SEPARATE array — NOT a nested list of metric specs. Supported funcs: sum, mean,
# count, min, max, median, nunique.
# The question asks for a MEDIAN on purpose (2026-10-22): the IR has no median, so
# this is a case where analyze_aggregate on pulled rows is the right plan. It used
# to ask for "rata-rata dan total", which the IR computes itself over every row —
# teaching the raw-pull shape for exactly the questions that should not use it.
# --------------------------------------------------------------------------- #

_EXAMPLE_D = TaskList(
    plan_id="example_d",
    goal_restated="Report the median and average order value for each region.",
    assumptions=[],
    open_questions=[],
    tasks=[
        Task(
            id="t1",
            stage="data_preparation",
            objective="Pull order-level region and revenue.",
            tool_calls=[
                ToolCall(
                    tool="retrieve_data",
                    args={
                        "ir": {
                            "source_id": "src_sales",
                            "table_id": "t_orders",
                            "select": [
                                {"kind": "column", "column_id": "c_region", "alias": "region"},
                                {"kind": "column", "column_id": "c_revenue", "alias": "revenue"},
                            ],
                            "limit": 10000,
                        }
                    },
                )
            ],
            expected_output="region_rows",
            success_criteria="Produced order rows with region and revenue.",
            depends_on=[],
            estimated_cost="medium",
        ),
        Task(
            id="t2",
            stage="evaluation",
            objective="Median and mean revenue per region (the IR has no median).",
            tool_calls=[
                ToolCall(
                    tool="analyze_aggregate",
                    args={
                        "data": "${t1}",
                        "aggregations": {"revenue": ["median", "mean"]},
                        "group_by": ["region"],
                    },
                )
            ],
            expected_output="region_aggregates",
            success_criteria="Produced one row per region with median and mean revenue.",
            depends_on=["t1"],
            estimated_cost="low",
        ),
    ],
)


# --------------------------------------------------------------------------- #
# Example E — non-date filters (value_type is the ELEMENT type, never a container).
# "Total revenue for the East and West regions, counting orders of at least 100."
# Shows: an `in` filter over a list of strings uses value_type "string" (NOT
# "list"); a numeric comparison uses "decimal"; and every filter carries a
# value_type copied from the column's catalog [data_type]. The total per region is
# computed in the same IR (2026-10-22; it used to pull raw rows and sum them in
# analyze_aggregate, the capped-slice shape planner.md now forbids).
# --------------------------------------------------------------------------- #

_EXAMPLE_E = TaskList(
    plan_id="example_e",
    goal_restated="Total revenue for the East and West regions, orders of at least 100.",
    assumptions=["'at least 100' filters order revenue >= 100."],
    open_questions=[],
    tasks=[
        Task(
            id="t1",
            stage="data_preparation",
            objective="Total revenue per region for East/West orders of at least 100.",
            tool_calls=[
                ToolCall(
                    tool="retrieve_data",
                    args={
                        "ir": {
                            "source_id": "src_sales",
                            "table_id": "t_orders",
                            "select": [
                                {"kind": "column", "column_id": "c_region", "alias": "region"},
                                {
                                    "kind": "agg",
                                    "fn": "sum",
                                    "column_id": "c_revenue",
                                    "alias": "total_revenue",
                                },
                            ],
                            "filters": [
                                {
                                    "column_id": "c_region",
                                    "op": "in",
                                    "value": ["East", "West"],
                                    "value_type": "string",
                                },
                                {
                                    "column_id": "c_revenue",
                                    "op": ">=",
                                    "value": 100,
                                    "value_type": "decimal",
                                },
                            ],
                            "group_by": ["c_region"],
                        }
                    },
                )
            ],
            expected_output="region_revenue",
            success_criteria="Produced total revenue per region, one row each.",
            depends_on=[],
            estimated_cost="low",
        ),
    ],
)


# --------------------------------------------------------------------------- #
# Example F — descriptive statistics (analyze_descriptive).
# "Give me the summary statistics for order revenue and quantity."
# Shows: retrieve_data pulls the numeric columns -> analyze_descriptive summarizes
# them. The `data` comes from the retrieve_data task (t1), NEVER from a check_data
# inspection step — check_data returns column metadata, not data rows.
# --------------------------------------------------------------------------- #

_EXAMPLE_F = TaskList(
    plan_id="example_f",
    goal_restated="Summarize the distribution of order revenue and quantity.",
    assumptions=[],
    open_questions=[],
    tasks=[
        Task(
            id="t1",
            stage="data_preparation",
            objective="Pull the order revenue and quantity rows.",
            tool_calls=[
                ToolCall(
                    tool="retrieve_data",
                    args={
                        "ir": {
                            "source_id": "src_sales",
                            "table_id": "t_orders",
                            "select": [
                                {"kind": "column", "column_id": "c_revenue", "alias": "revenue"},
                                {"kind": "column", "column_id": "c_quantity", "alias": "quantity"},
                            ],
                            "limit": 10000,
                        }
                    },
                )
            ],
            expected_output="order_rows",
            success_criteria="Produced order-level rows with revenue and quantity.",
            depends_on=[],
            estimated_cost="medium",
        ),
        Task(
            id="t2",
            stage="evaluation",
            objective="Summarize the distribution of revenue and quantity.",
            tool_calls=[
                ToolCall(
                    tool="analyze_descriptive",
                    args={
                        # `data` references t1 (retrieve_data rows), never check_data.
                        "data": "${t1}",
                        # column refs are the retrieve_data output aliases.
                        "column_ids": ["revenue", "quantity"],
                    },
                )
            ],
            expected_output="summary_stats",
            success_criteria=(
                "Produced mean/median/std/quartiles for revenue and quantity, above/below "
                "the typical range."
            ),
            depends_on=["t1"],
            estimated_cost="low",
        ),
    ],
)


# --------------------------------------------------------------------------- #
# Example G — top-N ranking.
# "Top 3 product categories by total revenue."
# Shows: top-N is ONE retrieve_data query — group by the entity, aggregate the
# measure with an alias, order by that alias, limit N. NEVER a bare
# order-by-measure + limit (that ranks raw rows, so the same entity can appear
# twice — observed in production: "top 3 models" returned one model twice).
# --------------------------------------------------------------------------- #

_EXAMPLE_G = TaskList(
    plan_id="example_g",
    goal_restated="Rank product categories by total revenue and return the top 3.",
    assumptions=[],
    open_questions=[],
    tasks=[
        Task(
            id="t1",
            stage="data_preparation",
            objective="Aggregate revenue per category, rank descending, keep the top 3.",
            tool_calls=[
                ToolCall(
                    tool="retrieve_data",
                    args={
                        "ir": {
                            "source_id": "src_sales",
                            "table_id": "t_orders",
                            "select": [
                                {"kind": "column", "column_id": "c_category", "alias": "category"},
                                {
                                    "kind": "agg",
                                    "fn": "sum",
                                    "column_id": "c_revenue",
                                    "alias": "total_revenue",
                                },
                            ],
                            "group_by": ["c_category"],
                            "order_by": [{"column_id": "total_revenue", "dir": "desc"}],
                            "limit": 3,
                        }
                    },
                )
            ],
            expected_output="top3_categories",
            success_criteria=(
                "Produced at most 3 rows, one distinct category each, ranked by "
                "total revenue."
            ),
            depends_on=[],
            estimated_cost="low",
        ),
    ],
)

# --------------------------------------------------------------------------- #
# Example H — infeasible question (see planner.md "When the catalog cannot
# answer"). "Berapa tingkat churn pelanggan kita?" against a sales catalog with no
# subscription/churn data: no task list is forced onto unrelated columns;
# instead `infeasible_reason` states the gap + the nearest available data.
# ALSO the one example of the reply-language rule: the question is Indonesian, so
# `infeasible_reason` is Indonesian — while every other field stays English,
# because only this field is user-facing prose (see `prompt.py`). Keep it that way.
# --------------------------------------------------------------------------- #

_EXAMPLE_H = TaskList(
    plan_id="example_h",
    goal_restated="Measure the customer churn rate.",
    assumptions=[],
    open_questions=[],
    tasks=[],
    infeasible_reason=(
        "Sumber data yang terhubung tidak memuat data churn atau status langganan — "
        "tabel orders hanya menyimpan kategori, revenue, kuantitas, dan tanggal di "
        "level order. Analisis terdekat yang tersedia: perilaku pembelian ulang, "
        "atau revenue per pelanggan dari waktu ke waktu."
    ),
)


# --------------------------------------------------------------------------- #
# Example I — combine two measures per entity (KM-703).
# "Which category has both the highest revenue and the highest average order
# quantity?" Shows: each measure is computed in its OWN grouped retrieve_data
# task (a "${t<id>}" placeholder resolves to a task's LAST output, so the two
# retrievals must be separate tasks), then analyze_merge aligns them on the
# shared entity alias. The merged table answers "both A and B" questions that
# a single query cannot express.
# --------------------------------------------------------------------------- #

_EXAMPLE_I = TaskList(
    plan_id="example_i",
    goal_restated=(
        "Identify the product category with both the highest total revenue and the "
        "highest average order quantity."
    ),
    assumptions=[],
    open_questions=[],
    tasks=[
        Task(
            id="t1",
            stage="data_preparation",
            objective="Total revenue per category.",
            tool_calls=[
                ToolCall(
                    tool="retrieve_data",
                    args={
                        "ir": {
                            "source_id": "src_sales",
                            "table_id": "t_orders",
                            "select": [
                                {"kind": "column", "column_id": "c_category", "alias": "category"},
                                {
                                    "kind": "agg",
                                    "fn": "sum",
                                    "column_id": "c_revenue",
                                    "alias": "total_revenue",
                                },
                            ],
                            "group_by": ["c_category"],
                        }
                    },
                )
            ],
            expected_output="revenue_per_category",
            success_criteria="Produced one total-revenue row per category.",
            depends_on=[],
            estimated_cost="low",
        ),
        Task(
            id="t2",
            stage="data_preparation",
            objective="Average order quantity per category.",
            tool_calls=[
                ToolCall(
                    tool="retrieve_data",
                    args={
                        "ir": {
                            "source_id": "src_sales",
                            "table_id": "t_orders",
                            "select": [
                                {"kind": "column", "column_id": "c_category", "alias": "category"},
                                {
                                    "kind": "agg",
                                    "fn": "avg",
                                    "column_id": "c_quantity",
                                    "alias": "avg_quantity",
                                },
                            ],
                            "group_by": ["c_category"],
                        }
                    },
                )
            ],
            expected_output="quantity_per_category",
            success_criteria="Produced one average-quantity row per category.",
            depends_on=[],
            estimated_cost="low",
        ),
        Task(
            id="t3",
            stage="evaluation",
            objective="Align both measures per category to find the category leading on both.",
            tool_calls=[
                ToolCall(
                    tool="analyze_merge",
                    args={
                        "data": "${t1}",
                        "data_right": "${t2}",
                        "on": ["category"],
                    },
                )
            ],
            expected_output="combined_measures",
            success_criteria=(
                "Produced one row per category carrying both total_revenue and "
                "avg_quantity."
            ),
            depends_on=["t1", "t2"],
            estimated_cost="low",
        ),
    ],
)


# --------------------------------------------------------------------------- #
# Example J — visualization tail (SPINE_V2_PLAN §4.3, recipe "viz tail").
# "Show me a bar chart of total revenue per product category."
# Shows: render_chart is planned ONLY on an explicit ask ("bar chart"), and is
# ALWAYS a tail step — its `data` consumes a TABLE-producing upstream (here the
# grouped top-N-style retrieve from Example G, without the limit), never stats/
# series/metadata. `x`/`y` reference that table's column aliases; the chart
# carries the already-aggregated rows (one bar per category), never raw order
# rows. The `assumptions` line carries the feasibility check on purpose: a chart
# ask never licenses aliasing a stand-in column (see Example K).
# --------------------------------------------------------------------------- #

_EXAMPLE_J = TaskList(
    plan_id="example_j",
    goal_restated="Chart total revenue per product category as a bar chart.",
    assumptions=[
        "The chart dimension exists in the catalog (c_category) — a chart ask is "
        "planned only when the asked-for dimension and measure have real catalog "
        "columns; otherwise it is infeasible, chart or no chart."
    ],
    open_questions=[],
    tasks=[
        Task(
            id="t1",
            stage="data_preparation",
            objective="Aggregate total revenue per product category.",
            tool_calls=[
                ToolCall(
                    tool="retrieve_data",
                    args={
                        "ir": {
                            "source_id": "src_sales",
                            "table_id": "t_orders",
                            "select": [
                                {"kind": "column", "column_id": "c_category", "alias": "category"},
                                {
                                    "kind": "agg",
                                    "fn": "sum",
                                    "column_id": "c_revenue",
                                    "alias": "total_revenue",
                                },
                            ],
                            "group_by": ["c_category"],
                        }
                    },
                )
            ],
            expected_output="revenue_per_category",
            success_criteria="Produced one total-revenue row per category.",
            depends_on=[],
            estimated_cost="low",
        ),
        Task(
            id="t2",
            stage="evaluation",
            objective="Render the per-category revenue table as a bar chart.",
            tool_calls=[
                ToolCall(
                    tool="render_chart",
                    args={
                        # `data` references the TABLE task (t1) — a chart is always a
                        # tail on an aggregated table, never on raw rows or stats.
                        "data": "${t1}",
                        "chart_type": "bar",
                        "x": "category",
                        "y": "total_revenue",
                        "title": "Total revenue by product category",
                    },
                )
            ],
            expected_output="revenue_bar_chart",
            success_criteria="Produced a bar-chart spec with one bar per category.",
            depends_on=["t1"],
            estimated_cost="low",
        ),
    ],
)


# --------------------------------------------------------------------------- #
# Example K — a chart ask that is INFEASIBLE (SPINE_V2_PLAN §4.3 + planner.md
# "Charts ... never relaxes feasibility"). "Plot the customer churn rate by
# month as a line chart" against the sales catalog: churn does not exist, and
# the explicit chart request does NOT license mapping some other column into
# the asked-for measure — the verdict is the same infeasible_reason Example H
# would give, chart or no chart.
# --------------------------------------------------------------------------- #

_EXAMPLE_K = TaskList(
    plan_id="example_k",
    goal_restated="Chart the monthly customer churn rate as a line chart.",
    assumptions=[],
    open_questions=[],
    tasks=[],
    infeasible_reason=(
        "The connected source has no churn or subscription-status data — the "
        "orders table only carries order-level category, revenue, quantity, and "
        "dates — so there is nothing to chart. Nearest chartable analyses: "
        "monthly revenue trend, or order counts per category."
    ),
)


# --------------------------------------------------------------------------- #
# Example L — scalar count with a filter (no grouping, no analyze step).
# "Ada berapa order yang revenue-nya nol?"  [ID — R2b "berapa banyak …" is the
# recipe users most often reach for in Indonesian; the plan stays English.]
# Shows: a "how many rows match X" question is answered by a SINGLE retrieve_data
# IR whose select is a COUNT(*) aggregate ({"kind": "agg", "fn": "count"}, with
# column_id omitted) plus the filter — it returns the exact number in one row.
# Do NOT select the raw column and count the returned rows: that caps at `limit`
# and leaves the tally to be eyeballed. count(*) omits column_id (the validator
# lets only 'count' do so); no group_by is needed for a scalar count.
# --------------------------------------------------------------------------- #

_EXAMPLE_L = TaskList(
    plan_id="example_l",
    goal_restated="Count how many orders have revenue equal to 0.",
    assumptions=[],
    open_questions=[],
    tasks=[
        Task(
            id="t1",
            stage="evaluation",
            objective="Count the orders whose revenue equals 0.",
            tool_calls=[
                ToolCall(
                    tool="retrieve_data",
                    args={
                        "ir": {
                            "source_id": "src_sales",
                            "table_id": "t_orders",
                            "select": [
                                {"kind": "agg", "fn": "count", "alias": "order_count"}
                            ],
                            "filters": [
                                {
                                    "column_id": "c_revenue",
                                    "op": "=",
                                    "value": 0,
                                    "value_type": "decimal",
                                }
                            ],
                        }
                    },
                )
            ],
            expected_output="zero_revenue_count",
            success_criteria="Produced one row holding the count of zero-revenue orders.",
            depends_on=[],
            estimated_cost="low",
        ),
    ],
)


# --------------------------------------------------------------------------- #
# Example M — line/TREND chart over time (viz tail on a time series).
# "Tampilkan grafik garis rata-rata revenue dari waktu ke waktu."  [ID — also keeps
# this from reading as a near-duplicate of Example J's English bar-chart ask.]
# Shows: a trend line chart aggregates PER TIME PERIOD first — group_by the date
# column + aggregate the measure in the retrieve_data IR (one row per date), THEN
# tail render_chart. Never chart raw per-record rows over time (many rows per date
# = an unreadable vertical smear). Mirrors Example J (bar) but grouping by the date
# instead of a category. (analyze_trend -> render_chart is an equally valid shape.)
# --------------------------------------------------------------------------- #

_EXAMPLE_M = TaskList(
    plan_id="example_m",
    goal_restated="Chart average order revenue per day as a line chart over time.",
    assumptions=["The date and revenue columns exist in the catalog (c_order_date, c_revenue)."],
    open_questions=[],
    tasks=[
        Task(
            id="t1",
            stage="data_preparation",
            objective="Aggregate average revenue per day (one row per date).",
            tool_calls=[
                ToolCall(
                    tool="retrieve_data",
                    args={
                        "ir": {
                            "source_id": "src_sales",
                            "table_id": "t_orders",
                            "select": [
                                {"kind": "column", "column_id": "c_order_date", "alias": "order_date"},
                                {"kind": "agg", "fn": "avg", "column_id": "c_revenue", "alias": "avg_revenue"},
                            ],
                            "group_by": ["c_order_date"],
                            "order_by": [{"column_id": "c_order_date", "dir": "asc"}],
                        }
                    },
                )
            ],
            expected_output="daily_revenue",
            success_criteria="Produced one average-revenue row per date, ordered by date.",
            depends_on=[],
            estimated_cost="low",
        ),
        Task(
            id="t2",
            stage="evaluation",
            objective="Render the daily average-revenue series as a line chart.",
            tool_calls=[
                ToolCall(
                    tool="render_chart",
                    args={
                        # `data` is the AGGREGATED daily table (t1) — one point per date,
                        # never the raw per-order rows.
                        "data": "${t1}",
                        "chart_type": "line",
                        "x": "order_date",
                        "y": "avg_revenue",
                        "title": "Average revenue over time",
                    },
                )
            ],
            expected_output="revenue_line_chart",
            success_criteria="Produced a line-chart spec with one point per date.",
            depends_on=["t1"],
            estimated_cost="low",
        ),
    ],
)


# =========================================================================== #
# Examples N–W — one per tool that had no exemplar (A18, 2026-09-17).
#
# These deliberately do NOT open with `check_data`. The 13 earlier examples all
# did — a prior strong enough that 44% of eval plans emitted a catalog task they
# rarely need (planner.md: "smallest plan that answers the question"). Since
# 2026-10-28 (prompt audit F3) none of the examples opens with it: the catalog is
# already in the prompt.
#
# Stages follow the rule rewritten in planner.md: `modeling` ONLY for a task that
# actually FITS a model (forecast, importance, cluster, anomaly). A statistic
# (correlation), an arithmetic breakdown (pareto), a rule-based split
# (segmentation) and a hypothesis test (ab_test) are `evaluation`; profiling is
# `data_understanding`.
#
# None carries a `render_chart` tail: the ML tools ship their own chart, so a tail
# would demonstrate exactly the double-chart rule 5 forbids.
# =========================================================================== #


def _ir(select, filters=None, group_by=None, limit=10000, order_by=None,
        table_id="t_orders"):
    """Compact inline QueryIR for the examples below (same shape as A–M).
    `limit=None` omits it: an aggregated IR bounds itself (2026-10-22).
    """
    ir = {"source_id": "src_sales", "table_id": table_id, "select": select}
    if limit is not None:
        ir["limit"] = limit
    if filters:
        ir["filters"] = filters
    if group_by:
        ir["group_by"] = group_by
    if order_by:
        ir["order_by"] = order_by
    return {"ir": ir}


def _col(cid, alias):
    return {"kind": "column", "column_id": cid, "alias": alias}


def _agg(fn, alias, cid=None):
    out = {"kind": "agg", "fn": fn, "alias": alias}
    if cid:
        out["column_id"] = cid
    return out


def _pull(objective, select, expected, criteria, **kw):
    # `kw` passes straight to `_ir` (filters, group_by, order_by, limit, table_id).
    return Task(
        id="t1",
        stage="data_preparation",
        objective=objective,
        tool_calls=[ToolCall(tool="retrieve_data", args=_ir(select, **kw))],
        expected_output=expected,
        success_criteria=criteria,
        depends_on=[],
        estimated_cost="medium",
    )


def _step(stage, objective, tool, args, expected, criteria, cost="low"):
    return Task(
        id="t2",
        stage=stage,
        objective=objective,
        tool_calls=[ToolCall(tool=tool, args={"data": "${t1}", **args})],
        expected_output=expected,
        success_criteria=criteria,
        depends_on=["t1"],
        estimated_cost=cost,
    )


# --------------------------------------------------------------------------- #
# Example N — forecast (timeseries). "Proyeksikan revenue 6 bulan ke depan."
# Pairs against Example B (trend): the SAME retrieve shape — raw date + value
# rows, the tool buckets them itself via `freq`/`agg` — but the question projects
# forward instead of describing the past. Past vs project is the entire
# distinction between `analyze_trend` and `analyze_forecast`.
# --------------------------------------------------------------------------- #

_EXAMPLE_N = TaskList(
    plan_id="example_n",
    goal_restated="Project monthly revenue six months beyond the last actual month.",
    assumptions=[],
    open_questions=[],
    tasks=[
        _pull(
            "Pull order dates and revenue across the available history.",
            [_col("c_order_date", "order_date"), _col("c_revenue", "revenue")],
            "history_rows",
            "Produced dated revenue rows covering the available history.",
        ),
        _step(
            "modeling",
            "Fit a forecast on monthly revenue and project six months ahead.",
            "analyze_forecast",
            {
                "date_column": "order_date",
                "value_column": "revenue",
                "horizon": 6,
                "freq": "month",
                "agg": "sum",
            },
            "revenue_forecast",
            "Produced six future monthly points plus the backtest score.",
            cost="medium",
        ),
    ],
)

# --------------------------------------------------------------------------- #
# Example O — correlation (relationship). "Is order quantity related to revenue?"
# Pairs against Example P, which uses an identical retrieve: correlation answers
# how TWO NAMED columns move together, so `column_ids` lists exactly that pair.
# --------------------------------------------------------------------------- #

_EXAMPLE_O = TaskList(
    plan_id="example_o",
    goal_restated="Measure how order quantity and revenue move together.",
    assumptions=[],
    open_questions=[],
    tasks=[
        _pull(
            "Pull order-level quantity and revenue.",
            [_col("c_quantity", "quantity"), _col("c_revenue", "revenue")],
            "pair_rows",
            "Produced order rows carrying both quantity and revenue.",
        ),
        _step(
            "evaluation",
            "Correlate quantity against revenue.",
            "analyze_correlation",
            {"column_ids": ["quantity", "revenue"]},
            "quantity_revenue_correlation",
            "Produced a correlation coefficient for the pair.",
        ),
    ],
)

# --------------------------------------------------------------------------- #
# Example P — importance (relationship). "Apa yang paling mempengaruhi revenue?"
# Contrast with Example O, and note the contrast is TWO-part: the question AND the
# retrieve width. Correlation takes TWO NAMED columns, so O pulls exactly two.
# Importance takes ONE TARGET and ranks every OTHER numeric column against it, so P
# has to pull the target plus SEVERAL candidates — a ranking needs something to rank.
#
# This example used to reuse O's two-column retrieve verbatim, which left importance
# exactly one candidate feature: a "ranking" of one, and an exemplar teaching the
# wrong retrieve width for the tool. (A23 D-12.)
#
# "How do X and Y relate" -> O; "what most affects X" -> P. A `modeling` task.
# --------------------------------------------------------------------------- #

_EXAMPLE_P = TaskList(
    plan_id="example_p",
    goal_restated="Rank which order attributes most explain revenue.",
    assumptions=["The orders table (about 8,000 rows) fits under row_budget, so "
                 "importance can rank on every order."],
    open_questions=[],
    tasks=[
        _pull(
            "Pull revenue alongside its candidate explanatory columns.",
            [
                _col("c_revenue", "revenue"),
                _col("c_quantity", "quantity"),
                _col("c_discount", "discount"),
                _col("c_unit_price", "unit_price"),
            ],
            "feature_rows",
            "Produced rows carrying revenue and several candidate explainers.",
        ),
        _step(
            "modeling",
            "Rank the columns that explain revenue.",
            "analyze_importance",
            {"target_column": "revenue"},
            "revenue_drivers",
            "Produced ranked features with a holdout score.",
            cost="medium",
        ),
    ],
)

# --------------------------------------------------------------------------- #
# Example Q — pareto (relationship). "80% revenue kita dari kategori apa?"
# Contrast with Example P: pareto is ARITHMETIC share of a total — which few
# categories add up to most of it — where importance is statistical explanation.
# A "which few account for most" question is always pareto, never importance.
# --------------------------------------------------------------------------- #

_EXAMPLE_Q = TaskList(
    plan_id="example_q",
    goal_restated="Identify the few categories that make up most of total revenue.",
    assumptions=[],
    open_questions=[],
    tasks=[
        _pull(
            "Pull category and revenue per order.",
            [_col("c_category", "category"), _col("c_revenue", "revenue")],
            "category_rows",
            "Produced order rows with category and revenue.",
        ),
        _step(
            "evaluation",
            "Break total revenue down by category, largest first.",
            "analyze_pareto",
            {"dimension": "category", "value_column": "revenue", "agg": "sum"},
            "revenue_pareto",
            "Produced categories with share and cumulative share of the total.",
        ),
    ],
)

# --------------------------------------------------------------------------- #
# Example R — segmentation (grouping). "Split customers into spend tiers
# 0-100, 100-500, 500+."
# Pairs against Example S. The user STATED the rule, so the split is
# deterministic: explicit `bins` edges and `labels`. Note the retrieve groups to
# one row per customer first — the tiers are per customer, not per order.
# --------------------------------------------------------------------------- #

_EXAMPLE_R = TaskList(
    plan_id="example_r",
    goal_restated="Split customers into the three stated spend tiers.",
    assumptions=["Tier boundaries are the ones the user gave: 0-100, 100-500, 500+."],
    open_questions=[],
    tasks=[
        _pull(
            "Total revenue per customer.",
            [
                _col("c_customer_id", "customer_id"),
                _agg("sum", "total_revenue", "c_revenue"),
            ],
            "customer_totals",
            "Produced one row per customer with their total revenue.",
            group_by=["c_customer_id"],
        ),
        _step(
            "evaluation",
            "Bin customers into the stated spend tiers.",
            "analyze_segmentation",
            {
                "column": "total_revenue",
                "bins": [0, 100, 500, 1000000],
                "method": "edges",
                "labels": ["0-100", "100-500", "500+"],
            },
            "spend_tiers",
            "Produced a customer count per stated tier.",
        ),
    ],
)

# --------------------------------------------------------------------------- #
# Example S — cluster (grouping). "Ada berapa kelompok pelanggan secara alami?"
# Pairs against Example R. The user gave NO rule, so the groups are discovered:
# no bins, no labels — just the columns describing each customer, and the tool
# picks k itself. Rule given -> R; no rule -> S. A `modeling` task.
# --------------------------------------------------------------------------- #

_EXAMPLE_S = TaskList(
    plan_id="example_s",
    goal_restated="Discover how many natural customer groups the data contains.",
    assumptions=[],
    open_questions=[],
    tasks=[
        _pull(
            "Build a per-customer profile of spend and volume.",
            [
                _col("c_customer_id", "customer_id"),
                _agg("sum", "total_revenue", "c_revenue"),
                _agg("sum", "total_quantity", "c_quantity"),
            ],
            "customer_profiles",
            "Produced one row per customer with spend and volume.",
            group_by=["c_customer_id"],
        ),
        _step(
            "modeling",
            "Discover the natural customer groups.",
            "analyze_cluster",
            {"column_ids": ["total_revenue", "total_quantity"]},
            "customer_clusters",
            "Produced a chosen k with its silhouette and one profile per cluster.",
            cost="medium",
        ),
    ],
)

# --------------------------------------------------------------------------- #
# Example T — A/B test (evaluate). "Apakah selisih revenue East dan West signifikan secara
# statistik?" (rephrased 2026-10-28 away from "bedanya", which pulled "apa yang membedakan
# unit X dengan unit lain" — a group comparison, Example Y — onto this shape).
# The question names TWO specific groups and asks whether the gap is real, so
# the plan retrieves the dimension plus the measure and hands both group values
# to the tool. "Is the difference significant" is always this tool — a plain
# total per region is a grouped retrieve_data instead (Example E).
# --------------------------------------------------------------------------- #

_EXAMPLE_T = TaskList(
    plan_id="example_t",
    goal_restated="Test whether East and West differ significantly on revenue.",
    assumptions=[],
    open_questions=[],
    tasks=[
        _pull(
            "Pull region and revenue for the two regions being compared.",
            [_col("c_region", "region"), _col("c_revenue", "revenue")],
            "region_rows",
            "Produced order rows carrying region and revenue.",
            filters=[
                {
                    "column_id": "c_region",
                    "op": "in",
                    "value": ["East", "West"],
                    "value_type": "string",
                }
            ],
        ),
        _step(
            "evaluation",
            "Test East against West on revenue.",
            "analyze_ab_test",
            {
                "dimension": "region",
                "value_column": "revenue",
                "group_a": "East",
                "group_b": "West",
                "agg": "sum",
            },
            "east_west_test",
            "Produced both group values, the gap, and a significance verdict.",
        ),
    ],
)

# --------------------------------------------------------------------------- #
# Example U — anomaly (evaluate). "Any unusual orders this month?"
# Unusual ROWS, not a comparison between groups: the plan pulls the numeric
# columns that define "normal" and the tool flags the outliers among them. A
# `modeling` task — it fits a model.
#
# Note the retrieve pulls an IDENTIFIER (`order_id`) alongside the numerics, and
# passes it as `label_columns`. Without it the flagged rows are a list of numbers
# and a score, and the answer to "WHICH orders are unusual?" cannot name one — the
# question is about rows, so the rows have to be identifiable. Identifier columns
# are never scored; they ride along so the answer can point at something. (A23 D-3.)
# --------------------------------------------------------------------------- #

_EXAMPLE_U = TaskList(
    plan_id="example_u",
    goal_restated="Find this month's orders that look unusual.",
    assumptions=["'this month' = 2026-03-01 to 2026-03-31."],
    open_questions=[],
    tasks=[
        _pull(
            "Pull this month's orders with their id, revenue and quantity.",
            [
                _col("c_order_id", "order_id"),
                _col("c_revenue", "revenue"),
                _col("c_quantity", "quantity"),
            ],
            "month_rows",
            "Produced this month's orders with their id, revenue and quantity.",
            filters=[
                {
                    "column_id": "c_order_date",
                    "op": "between",
                    "value": ["2026-03-01", "2026-03-31"],
                    "value_type": "date",
                }
            ],
        ),
        _step(
            "modeling",
            "Flag the unusual orders.",
            "analyze_anomaly",
            {
                "column_ids": ["revenue", "quantity"],
                "label_columns": ["order_id"],
            },
            "unusual_orders",
            "Produced the flagged orders, named by id, with their anomaly scores.",
            cost="medium",
        ),
    ],
)

# --------------------------------------------------------------------------- #
# Example V — data quality (explore). "Data ordernya bersih nggak? Ada yang kosong?"
# A question ABOUT the data rather than an answer computed from it. Still needs a
# retrieve — the tool profiles rows it is handed, it never self-fetches — and the
# profiling task is `data_understanding`, not `evaluation`.
# --------------------------------------------------------------------------- #

_EXAMPLE_V = TaskList(
    plan_id="example_v",
    goal_restated="Assess how complete and clean the order data is.",
    assumptions=[],
    open_questions=[],
    tasks=[
        _pull(
            "Pull the order columns to be profiled.",
            [
                _col("c_order_date", "order_date"),
                _col("c_category", "category"),
                _col("c_revenue", "revenue"),
                _col("c_quantity", "quantity"),
            ],
            "profile_rows",
            "Produced order rows across the columns to be profiled.",
        ),
        _step(
            "data_understanding",
            "Profile the columns for nulls, types and outliers.",
            "analyze_data_quality",
            {},
            "order_data_quality",
            "Produced per-column completeness and any flagged issues.",
        ),
    ],
)

# --------------------------------------------------------------------------- #
# Example W — infeasible METHOD (recipe R15). "Which customer will churn next?"
# Distinct from Example H, where the measure simply does not exist. Here the data
# is fine; the METHOD does not apply. This product forecasts a measure over time
# and ranks what explains one — it does not predict which individual entity a
# future event will happen to. An explicit ask for a prediction never relaxes
# that (planner.md rule 5), exactly as a chart ask never relaxes feasibility.
# --------------------------------------------------------------------------- #

_EXAMPLE_W = TaskList(
    plan_id="example_w",
    goal_restated="Predict which individual customer will churn next.",
    assumptions=[],
    open_questions=[],
    tasks=[],
    infeasible_reason=(
        "Naming the next customer to churn means predicting a future event for one "
        "entity, which none of the available tools do — they project a measure over "
        "time and rank what explains one. The orders table also carries no churn or "
        "subscription-status column to learn from. Nearest available analyses: revenue "
        "per customer over time, or which attributes most explain order value."
    ),
)


# --------------------------------------------------------------------------- #
# Examples X, Y, Z (2026-10-22, EMA dry run). Each fixes one failure the replay
# showed that no other example demonstrates — and A19 found the examples win when
# they and planner.md disagree, so the rules alone did not move these cases:
#   X — a ranking with NO number asked returns 10 rows, never 1 (limit 1 hid 20 of
#       21 units tied at the top).
#   Y — "what affects <flag>" on a table over row_budget compares groups IN the
#       query. Pairs against P: the same kind of question, and the row count decides
#       the plan. On a capped raw slice the flag never varied and importance could
#       rank nothing.
#   Z — a code the user types (here a store code) is looked up, never typed as an
#       id: the replay filtered "SDJ" as another site's number and labelled it SDJ.
# --------------------------------------------------------------------------- #

_EXAMPLE_X = TaskList(
    plan_id="example_x",
    goal_restated="Find the product category with the highest total revenue.",
    assumptions=["No number was asked for, so the top 10 are returned: categories tied "
                 "at the top, or close behind it, stay visible."],
    open_questions=[],
    tasks=[
        _pull(
            "Total revenue per category, highest first.",
            [_col("c_category", "category"), _agg("sum", "total_revenue", "c_revenue")],
            "category_ranking",
            "Produced up to 10 categories ranked by total revenue.",
            group_by=["c_category"],
            order_by=[{"column_id": "total_revenue", "dir": "desc"}],
            limit=10,
        ),
    ],
)

_EXAMPLE_Y = TaskList(
    plan_id="example_y",
    goal_restated="Compare returned and kept order lines on their candidate drivers.",
    assumptions=["order_lines holds about 250,000 rows, over row_budget: a raw pull "
                 "would be a capped slice, so the two groups are compared in the query "
                 "over every row."],
    open_questions=[],
    tasks=[
        _pull(
            "Average discount, unit price and quantity for returned vs kept lines.",
            [
                _col("c_is_returned", "is_returned"),
                _agg("avg", "avg_discount", "c_discount"),
                _agg("avg", "avg_unit_price", "c_unit_price"),
                _agg("avg", "avg_quantity", "c_quantity"),
                _agg("count", "lines"),
            ],
            "returned_vs_kept",
            "Produced one row per is_returned value with the averages and line count.",
            group_by=["c_is_returned"],
            limit=None,
            table_id="t_order_lines",
        ),
    ],
)

_EXAMPLE_Z = TaskList(
    plan_id="example_z",
    goal_restated="Total March 2026 revenue for store JKT.",
    assumptions=["JKT is a store code: its store_id is looked up from "
                 "stores.store_code, never typed as a number."],
    open_questions=[],
    tasks=[
        _pull(
            "Look up the id of store JKT.",
            [_col("c_store_id", "store_id")],
            "store_id",
            "Produced the store_id row matching code JKT.",
            filters=[{"column_id": "c_store_code", "op": "=", "value": "JKT",
                      "value_type": "string"}],
            limit=None,
            table_id="t_stores",
        ),
        Task(
            id="t2",
            stage="data_preparation",
            objective="Total March 2026 revenue for that store.",
            tool_calls=[ToolCall(tool="retrieve_data", args=_ir(
                [_agg("sum", "total_revenue", "c_revenue")],
                filters=[
                    {"column_id": "c_store_id", "op": "in", "value": "${t1.store_id}",
                     "value_type": "int"},
                    {"column_id": "c_order_date", "op": "between",
                     "value": ["2026-03-01", "2026-03-31"], "value_type": "date"},
                ],
                limit=None,
            ))],
            expected_output="store_revenue",
            success_criteria="Produced one total revenue row for store JKT in March 2026.",
            depends_on=["t1"],
            estimated_cost="low",
        ),
    ],
)


# Example AA (2026-10-27, Q1 hand-check). A ranking of a per-month measure carries
# how many months each entity has and its latest month: a unit with one partial
# month topped a six-month PA ranking at 6.36% and the answer could not tell.
# Rule-only did not move it (X ranks a SUM with no periods), so it is shown here.
_EXAMPLE_AA = TaskList(
    plan_id="example_aa",
    goal_restated="Rank stores by average monthly margin over the last 3 complete months.",
    assumptions=[
        "Last 3 months = July to September 2026, the complete months before today.",
        "Margin is a percentage, so it is averaged per store; n_periods and "
        "last_period show how many months each average rests on and where the "
        "data ends.",
    ],
    open_questions=[],
    tasks=[
        _pull(
            "Average monthly margin per store, lowest first, with its month count.",
            [
                _col("c_sm_store_id", "store_id"),
                _agg("avg", "avg_margin_pct", "c_sm_margin_pct"),
                _agg("count", "n_periods"),
                _agg("max", "last_period", "c_sm_month"),
            ],
            "store_margin_ranking",
            "Produced up to 10 stores ranked by average margin, each with its month "
            "count and latest month.",
            filters=[
                {"column_id": "c_sm_year", "op": "=", "value": 2026, "value_type": "int"},
                {"column_id": "c_sm_month", "op": "between", "value": [7, 9],
                 "value_type": "int"},
            ],
            group_by=["c_sm_store_id"],
            order_by=[{"column_id": "avg_margin_pct", "dir": "asc"}],
            limit=10,
            table_id="t_store_monthly",
        ),
    ],
)


EXAMPLES: list[tuple[str, TaskList]] = [
    ("Which product categories drove last quarter's revenue?", _EXAMPLE_A),
    ("How has monthly revenue trended by region this year, and what's unusual?", _EXAMPLE_B),
    ("Revenue dipped in Q1 — what happened?", _EXAMPLE_C),
    ("Berapa median dan rata-rata nilai order per region?", _EXAMPLE_D),
    ("Total revenue for the East and West regions, counting orders of at least 100.", _EXAMPLE_E),
    ("Give me the summary statistics for order revenue and quantity.", _EXAMPLE_F),
    ("Which 3 product categories have the best revenue performance?", _EXAMPLE_G),
    ("Kategori mana yang revenue-nya paling tinggi?", _EXAMPLE_X),
    ("Toko mana yang margin bulanannya paling rendah 3 bulan terakhir?", _EXAMPLE_AA),
    ("Berapa tingkat churn pelanggan kita?", _EXAMPLE_H),
    (
        "Which product category has both the highest revenue and the highest average "
        "order quantity?",
        _EXAMPLE_I,
    ),
    ("Show me a bar chart of total revenue per product category.", _EXAMPLE_J),
    ("Plot the customer churn rate by month as a line chart.", _EXAMPLE_K),
    ("Ada berapa order yang revenue-nya nol?", _EXAMPLE_L),
    ("Tampilkan grafik garis rata-rata revenue dari waktu ke waktu.", _EXAMPLE_M),
    ("Proyeksikan revenue 6 bulan ke depan.", _EXAMPLE_N),
    ("Is order quantity related to revenue?", _EXAMPLE_O),
    ("Apa yang paling mempengaruhi revenue?", _EXAMPLE_P),
    ("Order line yang di-retur dipengaruhi apa aja — apa yang membedakannya dari yang tidak?",
     _EXAMPLE_Y),
    ("80% revenue kita dari kategori apa?", _EXAMPLE_Q),
    ("Split customers into spend tiers 0-100, 100-500, and 500+.", _EXAMPLE_R),
    ("Ada berapa kelompok pelanggan secara alami?", _EXAMPLE_S),
    ("Apakah selisih revenue East dan West signifikan secara statistik?", _EXAMPLE_T),
    ("Any unusual orders this month?", _EXAMPLE_U),
    ("Data ordernya bersih nggak? Ada yang kosong?", _EXAMPLE_V),
    ("Which customer will churn next?", _EXAMPLE_W),
    ("What was store JKT's revenue in March 2026?", _EXAMPLE_Z),
]


def _tools_used(plan: TaskList) -> set[str]:
    """Every tool name a plan calls (empty for an infeasible plan)."""
    return {call.tool for task in plan.tasks for call in task.tool_calls}


def render_examples(visible_tools: set[str] | None = None) -> str:
    """Render the few-shots as text for the planner prompt.
    `visible_tools` is the name set the planner can actually call this turn. Group
    scoping (A17) shows the planner ~8 of 18 tools, so an example demonstrating a tool
    outside that slice teaches it to reach for something the validator will reject —
    and A19 established that these examples dominate when they and the rules disagree.
    Examples whose tools are not all visible are therefore dropped.
    Dropping is safe because it never starves the prompt: the narrowest group still
    keeps 7 of 13, and the two infeasible examples survive EVERY scope for free, since
    an infeasible plan calls no tools. Those are the ones that matter most — they teach
    the refusal discipline, and the feasible/infeasible pairs stay intact.
    `None` renders everything (the unscoped/fail-open path, and back-compat).
    """
    shown = [
        (q, p)
        for q, p in EXAMPLES
        if visible_tools is None or _tools_used(p) <= visible_tools
    ]
    blocks: list[str] = []
    for i, (question, plan) in enumerate(shown, start=1):
        # Compact JSON (PR23 part 2 D5, 2026-09-23): same content, no indentation —
        # measured −34–36% of exemplar tokens, the largest zero-semantics cut in the
        # planner call. The model reads structure from the braces, not the whitespace.
        blocks.append(
            f"## Example {i}\n\n"
            f"Question:\n{question}\n\n"
            f"TaskList:\n{plan.model_dump_json()}"
        )
    return "\n\n".join(blocks)