sofhiaazzhr Claude Opus 5.5 commited on
Commit
b1ef6f0
Β·
1 Parent(s): 7124acf

[NOTICKET] fix(help): suggest the analytical tools that are now active

Browse files

help.md's Capability boundary was written 2026-07-07, when only descriptive,
group-by, correlation and trend existed, and it explicitly forbade suggesting
significance tests, forecasting, clustering/segmentation and share-of-total.
Since A15 (2026-09-17) every one of those tools is ACTIVE, so Help steered
users away from half the product.

The MAY list now follows the tool groups (explore, relationships, over time,
grouping, evaluate, utility). The MUST NOT list mirrors the planner's R15
(method-infeasible): predicting which individual entity a future event will
hit, causal claims, recommendations, and a model as the deliverable, so Help
never proposes a question the planner will refuse.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Files changed (1) hide show
  1. src/config/prompts/help.md +57 -22
src/config/prompts/help.md CHANGED
@@ -1,4 +1,12 @@
1
- <!-- help.md Β· v6 Β· Help skill prompt.
 
 
 
 
 
 
 
 
2
  v6 (2026-07-09): report trigger changed β€” the `/report` skill and the old report button
3
  are GONE. A report is now generated ONE way only: the **Generate** button on the Report
4
  panel (single source of truth). Help must no longer tell users to run `/report` or "use
@@ -94,28 +102,55 @@ Every question you propose must be answerable by the system's live analysis capa
94
  Suggesting an analysis the tools cannot perform sets the user up to fail: they copy your
95
  suggestion, ask it, and hit a dead end. Stay strictly inside this list.
96
 
97
- **You MAY suggest** (supported):
98
- - Descriptive summaries of a column β€” average, median, spread, min/max, distribution.
99
- - Group-by breakdowns β€” a total, average, or count of a metric **per category**. This is also
100
- how to compare groups (e.g. "the average retention for online vs offline").
101
- - Correlation / relationship between numeric columns ("which factors relate to exam score?").
102
- - Trends over time.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
103
  - Inventory β€” what data / tables / documents exist.
104
 
105
- **You must NOT suggest** (no tool exists β€” do not propose these even when they fit the goal):
106
- - **Statistical significance / hypothesis tests** β€” never use "significant", "statistically
107
- significant", "significant difference", t-test, ANOVA, or p-value. To compare groups, suggest
108
- a group-by average that shows the gap (e.g. "compare the average retention of online vs
109
- offline"), NOT "is the difference significant".
110
- - Predictive modeling, forecasting, or regression models.
111
- - Causal claims β€” avoid "cause", "impact of", "effect of" framed as causation; keep it to
112
- relationship / correlation.
113
- - Clustering or segmentation into discovered groups.
114
- - Share-of-total / contribution breakdowns.
115
-
116
- When a goal naturally invites a forbidden analysis (e.g. the user wants to know if a gap is
117
- "real"), degrade the suggestion to the nearest supported one β€” the group-by average that shows
118
- the gap β€” rather than promising the unsupported analysis.
 
 
 
 
 
 
 
119
 
120
  ## How-to phrasing (degrade gracefully)
121
 
@@ -145,7 +180,7 @@ spam, no overselling. A few sentences is usually enough.
145
  - Never suggest an action that the signals say isn't available or isn't ready.
146
  - One step at a time β€” give the next step, not the whole roadmap.
147
  - When you suggest questions, **dedupe against `chat_history`** β€” only propose analyses not yet run that move the goal forward; a question that already has an answer adds no fresh evidence.
148
- - **Stay inside the Capability boundary above** β€” never propose significance tests, forecasting, modeling, causal claims, clustering/segmentation, or share-of-total; there is no tool for them, so suggesting them sends the user into a dead end.
149
  - No markdown headers or code fences in your reply; short prose (and an inline `/command` or a tiny bullet list) is fine.
150
 
151
  ## Examples
 
1
+ <!-- help.md Β· v7 Β· Help skill prompt.
2
+ v7 (2026-09-28): Capability boundary synced to the live tool registry. Since A15
3
+ (2026-09-17) every analytics tool is ACTIVE β€” significance tests (analyze_ab_test),
4
+ forecasting (analyze_forecast), key drivers (analyze_importance), share-of-total
5
+ (analyze_pareto), segmentation + clustering, anomaly detection, data quality and charts β€”
6
+ but v5's list still forbade suggesting them, so Help steered users away from half the
7
+ product. The MUST NOT list now mirrors the planner's R15 (method-infeasible): individual
8
+ future-event prediction, causal claims, recommendations. This list is hand-maintained β€”
9
+ update it whenever a tool is activated or retired (src/tools/registry.py).
10
  v6 (2026-07-09): report trigger changed β€” the `/report` skill and the old report button
11
  are GONE. A report is now generated ONE way only: the **Generate** button on the Report
12
  panel (single source of truth). Help must no longer tell users to run `/report` or "use
 
102
  Suggesting an analysis the tools cannot perform sets the user up to fail: they copy your
103
  suggestion, ask it, and hit a dead end. Stay strictly inside this list.
104
 
105
+ **You MAY suggest** (supported β€” grouped the way the analysis tools are grouped):
106
+ - **Explore**
107
+ - Descriptive summaries of a column β€” average, median, spread, min/max, distribution.
108
+ - Group-by breakdowns β€” a total, average, or count of a metric **per category**.
109
+ - Data quality β€” missing values, duplicate rows, columns that never change.
110
+ - **Relationships**
111
+ - Correlation between two named numeric columns ("is operating hours related to PA?").
112
+ - Key drivers β€” which columns most explain ONE target metric ("what most affects
113
+ downtime?"). Phrase it as *explains* / *most related to*, never as *causes*.
114
+ - Pareto / share of total β€” which categories make up most of a total ("which components
115
+ account for 80% of downtime?").
116
+ - **Over time**
117
+ - Trends over time.
118
+ - Forecasting a metric's future values ("project PA for the next 3 months") β€” needs a date
119
+ column and a history of past periods.
120
+ - **Grouping**
121
+ - Segmentation by a rule the user gives (tiers, ranges, bins).
122
+ - Clustering into natural groups nobody has defined ("what natural groups of units are there?").
123
+ - **Evaluate**
124
+ - Whether the gap between **two named groups** is statistically significant ("is the PA
125
+ difference between site A and site B significant?").
126
+ - Anomaly detection β€” which individual rows look unusual ("which units look unusual?").
127
+ - **Utility**
128
+ - Combining two measures per entity ("which unit has both the highest downtime and the
129
+ lowest PA?").
130
+ - Charts β€” a chart of any of the above, when the user wants one.
131
  - Inventory β€” what data / tables / documents exist.
132
 
133
+ **Start basic, then go deeper.** In the first suggestions of an analysis, lead with explore
134
+ questions; offer the deeper analyses (drivers, forecast, significance, clustering, anomalies)
135
+ once the basics are covered, or when the objective and business questions clearly ask for them.
136
+ You cannot see the user's columns, so tie each suggestion to the objective and business
137
+ questions rather than naming specific columns you have not been told about.
138
+
139
+ **You must NOT suggest** (no tool does this β€” the planner will refuse it, so the user hits a
140
+ dead end):
141
+ - **Predicting which individual entity a future event will hit** β€” "which customer will churn
142
+ next", "which unit will break down tomorrow". Forecasting a metric over time is supported;
143
+ naming the specific future case is not.
144
+ - **Causal claims** β€” avoid "cause", "impact of", "effect of" framed as causation. Key drivers
145
+ and correlation show what is *related to* or *explains* a metric, not what causes it.
146
+ - **Recommendations / prescriptions** β€” "what should we do", "which action will fix it". The
147
+ analysis can surface evidence; it does not decide actions.
148
+ - **Building a model as the deliverable** β€” training a regression/classification model for
149
+ the user to keep, AutoML, or text analysis of free-text columns.
150
+
151
+ When a goal naturally invites a forbidden analysis, degrade the suggestion to the nearest
152
+ supported one β€” e.g. "which unit will break down next" β†’ "which units look unusual" or "project
153
+ downtime for the next 3 months"; "what causes low PA" β†’ "what most affects PA".
154
 
155
  ## How-to phrasing (degrade gracefully)
156
 
 
180
  - Never suggest an action that the signals say isn't available or isn't ready.
181
  - One step at a time β€” give the next step, not the whole roadmap.
182
  - When you suggest questions, **dedupe against `chat_history`** β€” only propose analyses not yet run that move the goal forward; a question that already has an answer adds no fresh evidence.
183
+ - **Stay inside the Capability boundary above** β€” never propose predicting which individual entity a future event will hit, causal claims, recommendations, or building a model as the deliverable; there is no tool for them, so suggesting them sends the user into a dead end.
184
  - No markdown headers or code fences in your reply; short prose (and an inline `/command` or a tiny bullet list) is fine.
185
 
186
  ## Examples