[NOTICKET] anomaly: bound the returned sample at source, not just the assembler

analyze_anomaly's `anomalies` list could carry up to 200 records (all scored
columns + score each). A30 capped the same bloat at the assembler (what reaches
the LLM), but the tool output is ALSO persisted to results_snapshot and
Langfuse-traced, so the 50k blob still landed there.

Cap _MAX_ANOMALY_ROWS 200 -> 50 (a bounded top-by-score SAMPLE, not the full
set) and add anomalies_total / anomalies_shown so "N of M" stays honest. The
exact count (n_anomalies), rate, and chart are unchanged. Defense-in-depth with
A30's assembler cap, not a duplicate.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

DEV_PLAN.md

@@ -528,7 +528,7 @@ ambiguous, "feature importance" as too technical); `analyze_comparison` → **`a
 | A6 | Authoring conventions doc | R | ✅ | [docs/tools/ANALYTICAL_TOOLS_CONVENTIONS.md](docs/tools/ANALYTICAL_TOOLS_CONVENTIONS.md), written against the shipped code |
 | A7 | `analyze_forecast` — ETS, **no AutoML**, line chart actual vs predicted | S | ✅ | `dae102f6`. ETS Holt-Winters (additive trend); reuses `analyze_trend` date parsing/resampling; DEFERRED via `spec_from_model`. **First `run_fit` user** — invoker now awaits coroutine compute fns (`inspect.isawaitable`, opt-in), resolving the conventions §4/§6 contradiction toward "async only for the fit"; doc §6 wording ("never async") needs the follow-up tweak. Headline demo item |
 | A8 | `analyze_importance` — ranked impact on a target, bar chart | R | ✅ | written by Rifqi 2026-09-16; **registered DEFERRED by Sofhia** (invoker + registry via `spec_from_model`, output_kind `stats`, category `analytics.relationship`); smoke-tested invokable. Standardised coefficients, not permutation importance: for a linear model the coefficient IS the importance, and permutation (the E4 tall pole) buys nothing. Ridge for a numeric target, Logistic for a categorical one, auto-detected. **Leakage-safe**: scaler fitted on the TRAIN split only. Reports a holdout `quality` block. Verified on planted signal — recovered breakdowns 0.86 / age 0.14 / noise 0.003 at r2 0.982; classification path accuracy 0.92; 5 error paths raise correctly |
-| A9 | `analyze_anomaly` — Isolation Forest, scatter **with anomaly points highlighted** | S | ✅ | `9f3267d5`. output_kind `stats` (summary + flagged rows); fixed `random_state`, fit via `run_fit`; DEFERRED via `spec_from_model`. Showing the points was explicit in the discussion |
+| A9 | `analyze_anomaly` — Isolation Forest, scatter **with anomaly points highlighted** | S | ✅ | `9f3267d5`. output_kind `stats` (summary + flagged rows); fixed `random_state`, fit via `run_fit`; DEFERRED via `spec_from_model`. Showing the points was explicit in the discussion. **Source-side sample cap (post-A30):** `_MAX_ANOMALY_ROWS` 200 → **50** and new `anomalies_total`/`anomalies_shown` fields — the `anomalies` list is a bounded top-by-score SAMPLE, count stays exact. A30 capped the same bloat at the assembler (what reaches the LLM); this bounds it at the SOURCE (the tool output is also persisted to `results_snapshot` + Langfuse-traced), so the 50k blob no longer lands there either. Defense-in-depth, not a duplicate |
 | A10 | `analyze_cluster` — KMeans + silhouette, cluster scatter | R | ✅ | written by Rifqi 2026-09-16; **registered DEFERRED by Sofhia** (invoker + registry, output_kind `stats`, category `analytics.clustering`); smoke-tested invokable. k chosen by silhouette over 2..8 with every score returned; one profile row per cluster; scatter on two REAL columns, not PCA components — a user can read "PA vs MTTR" and cannot read "PC1 vs PC2". Reports a `quality` block. Verified on 3 planted blobs — found k=3 (silhouette 0.768), cluster centres matched the planted centroids, 3 chart traces; 4 error/edge paths correct |
 | A11 | `analyze_ab_test` — rename `comparison.py`, add the significance test | S | ✅ | `analyze_comparison`/`comparison.py` → `analyze_ab_test`/`ab_test.py`. Kept the two-group aggregate gap; **added a Welch two-sample t-test** (t, p-value, significant at alpha) on the raw values + a bar chart of the two group means with 95% error bars. `group_a`/`group_b` typed `Any` (schema keeps them untyped, as before). DEFERRED via `spec_from_model`. Old name removed — 5 cross-ref DESCRIPTIONs + `contracts.py` comment repointed |
 | A12 | `analyze_pareto` — rename `decomposition.py`, reframe to 80/20 | S | ✅ | `analyze_contribution`/`decomposition.py` → `analyze_pareto`/`pareto.py`. Kept the share + cumulative compute; added `categories_for_80pct` and the `pareto_chart` (bars + cumulative-% line). **DEFERRED** via `spec_from_model`. Old name fully removed (no eval/agent refs; 3 cross-ref DESCRIPTIONs repointed). Local tests gitignored — Rifqi to rename any local `analyze_contribution` test refs after pulling |

src/tools/analytics/anomaly.py

@@ -25,8 +25,12 @@ from src.tools.spec_builder import build_description
 
 # IsolationForest is meaningless on a handful of rows; below this we raise.
 _MIN_ROWS = 10
-# Cap the returned anomaly rows (the count is still exact) to keep the payload sane.
-_MAX_ANOMALY_ROWS = 200
+# Cap the returned anomaly SAMPLE (the count stays exact) so a tool output can't
+# balloon to tens of thousands of tokens — the list is illustrative, not the result.
+# The full total is reported alongside as `anomalies_total`. A23: was 200; a bounded
+# top-by-score sample is the right payload for an LLM-facing tool (defense-in-depth
+# with the assembler's own cap).
+_MAX_ANOMALY_ROWS = 50
 # Fixed so the same data always flags the same rows (family-B determinism promise).
 _RANDOM_STATE = 42
 # Internal column names for the highlighted scatter (never collide with user data).
@@ -109,8 +113,9 @@ async def analyze_anomaly(
     `df`.
 
     Returns a dict with: method, column_ids, contamination, n_rows, n_anomalies,
-    anomaly_rate, `anomalies` (the flagged rows, top by score, capped), and `charts`
-    (the reserved auto-chart key).
+    anomaly_rate, anomalies_total (= n_anomalies) / anomalies_shown (how many the
+    capped sample actually carries), `anomalies` (the flagged rows, top by score,
+    capped at _MAX_ANOMALY_ROWS), and `charts` (the reserved auto-chart key).
 
     Raises (all wrapped by the invoker's never-throw seam):
         ColumnNotFoundError, EmptyColumnSelectionError, NotEnoughDataError.
@@ -159,6 +164,11 @@ async def analyze_anomaly(
         "n_rows": int(scored.shape[0]),
         "n_anomalies": int(flags.sum()),
         "anomaly_rate": float(flags.mean()) if scored.shape[0] else 0.0,
+        # `anomalies` is a bounded top-by-score SAMPLE, not the full set. These two
+        # keep it honest: total says how many there really are, shown says how many
+        # the sample carries. When total > shown the caller knows it's truncated.
+        "anomalies_total": int(flags.sum()),
+        "anomalies_shown": len(anomalies),
         "anomalies": anomalies,
         "charts": [_anomaly_chart(df, column_ids, is_anomaly)],
     }