fsanyoto commited on
Commit
64268da
Β·
verified Β·
1 Parent(s): 149aa87

Deploy AIOS web (React glide grid + FastAPI slice)

Browse files
This view is limited to 50 files because it contains too many changes. Β  See raw diff
Files changed (50) hide show
  1. RELEASES.json +19 -1
  2. VERSION +1 -1
  3. api/ai_enrich.py +550 -0
  4. api/ai_review.py +314 -9
  5. api/automation_engine.py +0 -0
  6. api/deps.py +421 -1
  7. api/main.py +183 -14
  8. api/odoo_relational.py +233 -15
  9. api/routes_admin.py +224 -12
  10. api/routes_alerts.py +137 -6
  11. api/routes_automation.py +648 -21
  12. api/routes_brand.py +140 -0
  13. api/routes_changes.py +85 -1
  14. api/routes_connectors.py +42 -13
  15. api/routes_customers.py +11 -0
  16. api/routes_feedback.py +180 -0
  17. api/routes_grid.py +123 -1
  18. api/routes_keychain.py +212 -4
  19. api/routes_nav.py +212 -10
  20. api/routes_odoo_tables.py +12 -1
  21. api/routes_publish.py +917 -0
  22. api/routes_query.py +0 -0
  23. api/routes_records.py +59 -12
  24. api/routes_shares.py +9 -1
  25. api/routes_slack.py +756 -0
  26. api/routes_starred.py +828 -0
  27. api/routes_statements.py +104 -3
  28. api/routes_tables.py +307 -44
  29. api/routes_usage.py +51 -0
  30. api/usage_ledger.py +281 -0
  31. api/web_agent.py +41 -23
  32. jobs/web_agent_job.py +117 -2
  33. platform/aios_grid.py +41 -1
  34. platform/aios_grid_fields.json +70 -0
  35. platform/core/alerts.py +21 -2
  36. platform/core/data_binding.py +270 -0
  37. platform/core/grid_events.py +113 -10
  38. platform/core/keychain.py +46 -0
  39. platform/core/perm_scope.py +130 -0
  40. platform/core/perms.py +110 -0
  41. platform/core/registry.py +141 -20
  42. platform/core/shared_overlay.py +38 -0
  43. platform/core/shares.py +26 -4
  44. platform/core/store.py +448 -57
  45. platform/core/user_tables.py +428 -5
  46. platform/core/users.py +45 -25
  47. platform/harness/datastore.py +23 -2
  48. platform/harness/tools.py +38 -9
  49. platform/model/topics/odoo_accounts.yml +60 -0
  50. platform/model/topics/odoo_agents.yml +59 -0
RELEASES.json CHANGED
@@ -1,6 +1,24 @@
1
  {
2
- "current": "v25 (bc24829)",
3
  "releases": [
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
4
  {
5
  "version": "v25",
6
  "sha": "bc24829",
 
1
  {
2
+ "current": "v29 (55ced50)",
3
  "releases": [
4
+ {
5
+ "version": "v29",
6
+ "sha": "55ced50",
7
+ "date": "2026-08-18",
8
+ "subject": "v29: the build staging has been running since 2026-08-18 (55ced50)."
9
+ },
10
+ {
11
+ "version": "v28",
12
+ "sha": "2793e4b",
13
+ "date": "2026-08-18",
14
+ "subject": "release v28"
15
+ },
16
+ {
17
+ "version": "v27",
18
+ "sha": "c05a794",
19
+ "date": "2026-08-18",
20
+ "subject": "release v27"
21
+ },
22
  {
23
  "version": "v25",
24
  "sha": "bc24829",
VERSION CHANGED
@@ -1 +1 @@
1
- v25 (bc24829)
 
1
+ v29 (55ced50)
api/ai_enrich.py ADDED
@@ -0,0 +1,550 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """ai_enrich.py -- the AI ENRICHMENT column (wave 34, owner ruling R13).
2
+
3
+ R13: *"A field kind called AI enrichment: a prompt per row that populates text. Detailed
4
+ configuration in the field's own config, including a token-usage limit."*
5
+
6
+ WHERE THE HALVES LIVE, and why the line is drawn here rather than anywhere else:
7
+
8
+ core/user_tables.py the CONTRACT and the STORAGE -- `UT_FIELD_TYPES`, `_clean_ai_enrich`,
9
+ the per-cell provenance stratum, and the law about what an automatic
10
+ run may overwrite. It owns the store document, so it owns the marks.
11
+ api/ai_enrich.py THIS file: the PROMPT vocabulary, the run PLAN, and the token ceiling's
12
+ own report. `core` may not import upward, so anything that needs to
13
+ reach a vendor or an HTTP session lives here.
14
+
15
+ Everything in this module is a PURE FUNCTION of data it is handed. That is deliberate: it is the
16
+ half a gate can exercise without a network, a store, or a running server, which is what makes
17
+ `api_ai_enrich` a real gate rather than a mount check.
18
+
19
+ TOKEN ACCOUNTING, and the hole it is filling. `api/ai_review.py::decide` -- the product's only
20
+ other LLM entry -- has no token accounting of ANY kind: `max_tokens: 300` per call, one 20 s
21
+ timeout, no retry, and nothing anywhere counts what a run spent. R13 asks for a usage limit, so
22
+ there was nothing to build on and the ledger starts here. The two measured community complaints
23
+ about this feature elsewhere are both REPORTING gaps rather than capability gaps (a preview said
24
+ 15 credits and the run spent 360; a finished run gave no completion signal), which is why
25
+ `ceiling_report` exists beside the ceiling itself.
26
+ """
27
+ from __future__ import annotations
28
+
29
+ import os
30
+ import re
31
+
32
+ import requests
33
+
34
+ #: A prompt names other columns the way a formula does, so a user who has written one already
35
+ #: knows this syntax. ⚠ ONE parser, exported, because `user_tables.ai_enrich_input_hash` fingerprints
36
+ #: the referenced cells and a second copy of this pattern would disagree with it the first time the
37
+ #: token syntax gained a form ([[one-question-two-normalizers]]).
38
+ _REF = re.compile(r'\{([a-zA-Z0-9_]{1,60})\}')
39
+
40
+ #: What a cell can say about where its value came from. Mirrors
41
+ #: `user_tables.AI_ENRICH_STATES` plus the two states that are DERIVED rather than stored:
42
+ #: `stale` (the inputs moved since the value was written) and `empty` (nothing has run).
43
+ #: β›” Re-exported rather than re-declared: the stored vocabulary has exactly one owner.
44
+ CELL_STATES = ('empty', 'agent', 'human', 'stale', 'error')
45
+
46
+ #: β›” THERE IS DELIBERATELY NO PROVIDER LIST IN THIS FILE. The product already has TWO opinions
47
+ #: about provider order (`harness/analyst.PROVIDERS` says groq-first, `routes_query.
48
+ #: QUERY_PROVIDER_ORDER` says cerebras-first) and `W34-T37` exists to reconcile them. Declaring a
49
+ #: third here would make that ticket harder and this feature wrong in a new way, so the catalogue
50
+ #: is BORROWED from `ai_review` (`PROVIDERS` + `ladder()` + the key names + the cheap-first order
51
+ #: owner ruling R14 set). ⚠ The CALL is ours because the requirements differ: `ai_review` hardcodes
52
+ #: `max_tokens: 300` and reads no `usage`, and R13 needs a per-column ceiling and a real ledger.
53
+ #: One provider catalogue, two call shapes, and the reason is written down rather than inferred.
54
+
55
+ #: The run's per-cell wall clock. Separate from `ai_review.TIMEOUT_SECONDS` because a review is
56
+ #: one classification a person is waiting on and this is a loop over rows.
57
+ TIMEOUT_SECONDS = float(os.environ.get('AIOS_AI_ENRICH_TIMEOUT') or 30)
58
+ #: How many failing rows carry their error into the report before it is summarised. A report that
59
+ #: lists 900 identical `429`s is a report nobody reads.
60
+ MAX_REPORTED_ERRORS = 10
61
+ #: Bound on what one row hands the model, mirroring `ai_review`'s own two caps: a table can hold a
62
+ #: 32 KB JSON cell per row and no prompt needs it.
63
+ MAX_CELL_CHARS = 400
64
+
65
+
66
+ def prompt_refs(prompt):
67
+ """The column keys a prompt names, in first-appearance order, de-duplicated.
68
+
69
+ `"Summarise {name} for {segment}"` -> `['name', 'segment']`.
70
+ """
71
+ seen, out = set(), []
72
+ for key in _REF.findall(str(prompt or '')):
73
+ if key not in seen:
74
+ seen.add(key)
75
+ out.append(key)
76
+ return out
77
+
78
+
79
+ def render_prompt(prompt, row, field_keys=()):
80
+ """`(text, unresolved)` -- the prompt with `{token}` replaced by this row's cells.
81
+
82
+ β›” AN UNKNOWN TOKEN IS REPORTED, NOT SILENTLY BLANKED. A prompt naming a column that does not
83
+ exist is a prompt that will quietly ask a different question on every row, and the resulting
84
+ text looks exactly like a correct answer. `unresolved` is what the caller shows the user;
85
+ R6's standing rule is the same sentence one layer up (a limit that cannot be removed must be
86
+ reported with its cause).
87
+
88
+ ⚠ A token naming a real column whose CELL is empty is NOT unresolved: an empty cell is data.
89
+ """
90
+ known = set(field_keys or ())
91
+ unresolved = []
92
+
93
+ def _sub(match):
94
+ key = match.group(1)
95
+ if known and key not in known:
96
+ unresolved.append(key)
97
+ return match.group(0)
98
+ return str((row or {}).get(key, ''))
99
+
100
+ return _REF.sub(_sub, str(prompt or '')), sorted(set(unresolved))
101
+
102
+
103
+ def ceiling_report(spent, ceiling, rows_done, rows_total):
104
+ """R6's second sentence as DATA, or None while the run is still inside its budget.
105
+
106
+ `{subject, limit, spent, effect, cause, recommendation}` -- the same shape
107
+ `user_tables.limit_report` returns, so the client renders one vocabulary for every limit the
108
+ product enforces rather than a second one for this feature.
109
+
110
+ β›” `effect` is `stopped`, never `truncated`. A run that quietly stopped filling cells leaves a
111
+ column that looks complete and is not, which is the failure the whole reporting rule is
112
+ arranged against. The caller STOPS and says so.
113
+ """
114
+ if ceiling is None or spent < ceiling:
115
+ return None
116
+ left = max(0, int(rows_total or 0) - int(rows_done or 0))
117
+ rest = f'{left:,} rows' if left != 1 else '1 row'
118
+ return {
119
+ 'subject': 'tokens',
120
+ 'limit': int(ceiling),
121
+ 'spent': int(spent),
122
+ 'effect': 'stopped',
123
+ 'cause': f'this run reached its token ceiling of {int(ceiling):,} after filling '
124
+ f'{int(rows_done or 0):,} of {int(rows_total or 0):,} rows',
125
+ 'recommendation': f'raise the column\'s token limit, shorten its prompt, or run the '
126
+ f'remaining {rest} again. Nothing was truncated and no cell holds a '
127
+ f'partial answer',
128
+ }
129
+
130
+
131
+ def plan_rows(rows, col_id, config, marks, cost_of=None):
132
+ """Which rows an automatic run should fill, and why the others were left alone.
133
+
134
+ Returns `{'run': [row_id, ...], 'skipped': {reason: count}, 'estimate': int}`.
135
+
136
+ β›” THE SELECTION IS SEPARATE FROM THE RUN ON PURPOSE. It is pure, so a gate can prove the
137
+ never-overwrite-a-human-edit law without a vendor, and so the client can PREVIEW a run
138
+ honestly before it spends anything -- which is exactly the complaint R13 cites (a preview
139
+ that said 15 credits against a run that spent 360).
140
+
141
+ `cost_of(row)` estimates one row's tokens; absent, the column's own ceiling is used per row,
142
+ which is the pessimistic direction and the right one for a preview.
143
+ """
144
+ import core.user_tables as ut
145
+
146
+ cfg = config if isinstance(config, dict) else {}
147
+ prompt = str(cfg.get('prompt') or '')
148
+ policy = str(cfg.get('overwrite') or 'blank')
149
+ refs = prompt_refs(prompt)
150
+ per_row = int(cfg.get('maxTokens') or 0)
151
+ run, skipped, estimate = [], {}, 0
152
+ for row_id, row in (rows or {}).items():
153
+ row = row if isinstance(row, dict) else {}
154
+ mark = (marks or {}).get(str(row_id)) or {}
155
+ fresh = ut.ai_enrich_input_hash(prompt, row, refs)
156
+ has_value = bool(str(row.get(str(col_id)) or '').strip())
157
+ if ut.ai_enrich_may_write(mark, policy, fresh, has_value=has_value):
158
+ run.append(str(row_id))
159
+ estimate += int(cost_of(row)) if callable(cost_of) else per_row
160
+ else:
161
+ # The same predicate again, so the count a preview shows and the cells a run leaves
162
+ # alone can never be two different sets.
163
+ reason = ('human_edited' if ut.ai_enrich_human_authored(mark, has_value)
164
+ else 'unchanged' if has_value else 'skipped')
165
+ skipped[reason] = skipped.get(reason, 0) + 1
166
+ return {'run': run, 'skipped': skipped, 'estimate': estimate}
167
+
168
+
169
+ def cell_state(mark, config, row, col_id):
170
+ """What ONE cell should say about itself: one of `CELL_STATES`.
171
+
172
+ ⚠ `stale` and `empty` are computed here and never stored -- see the stratum's own note in
173
+ `core/user_tables.py`. A stored staleness flag needs a sweep to stay true, and a sweep nobody
174
+ runs is a flag shipped without its writer.
175
+ """
176
+ import core.user_tables as ut
177
+
178
+ mark = mark if isinstance(mark, dict) else {}
179
+ has_value = bool(str((row or {}).get(str(col_id)) or '').strip())
180
+ if str(mark.get('state') or '') == 'error':
181
+ return 'error'
182
+ # β›” THE SAME PREDICATE THE RUNNER OBEYS, called rather than restated. A cell this reports as
183
+ # the human's while `ai_enrich_may_write` would overwrite it is the worst of both answers.
184
+ if ut.ai_enrich_human_authored(mark, has_value):
185
+ return 'human'
186
+ if str(mark.get('state') or '') != 'agent':
187
+ return 'empty'
188
+ cfg = config if isinstance(config, dict) else {}
189
+ prompt = str(cfg.get('prompt') or '')
190
+ fresh = ut.ai_enrich_input_hash(prompt, row or {}, prompt_refs(prompt))
191
+ return 'stale' if ut.ai_enrich_is_stale(mark, fresh) else 'agent'
192
+
193
+
194
+ # ══════════════════════════════════════════════════════════════════════ THE RUN
195
+ # οΏ½οΏ½οΏ½β›” THE LEDGER IS THE POINT OF THIS HALF. `ai_review.decide` (the product's only other LLM entry)
196
+ # counts nothing: it sends `max_tokens: 300` and never reads `usage` off the response, so no run in
197
+ # this product has ever been able to say what it cost. R13 asks for a token-usage limit, so the
198
+ # accounting starts here, and every leg below reads `usage` even where it is not needed, because a
199
+ # ledger with one blind provider is not a ledger.
200
+
201
+
202
+ def _usage_tokens(body):
203
+ """Total tokens for one call, from whichever shape the provider answered in.
204
+
205
+ ⚠ RETURNS None WHEN THE PROVIDER DID NOT SAY, and the caller treats that as a real unknown
206
+ rather than as zero. A ledger that silently books an unmeasured call at 0 reports a cheaper
207
+ run than happened, which is exactly the cost-surprise complaint R13 cites.
208
+
209
+ ⭐⭐ W35-T41 / C7 β€” THE READER MOVED AND THIS IS NOW ONE LINE OVER IT. `usage_ledger.total_from`
210
+ is the same logic, in the module that owns the meter; the alternative was a second reader in a
211
+ second file answering the same question about the same key, which is the shape this repo pays
212
+ for most often ([[one-question-two-normalizers]]). ⚠ The NAME and the None-semantics stay
213
+ exactly as wave 34 wrote them, because `run_field`'s ceiling arithmetic reads this and a gate
214
+ injects against its shape β€” lifting the body, not the signature.
215
+ """
216
+ import usage_ledger
217
+ return usage_ledger.total_from(body)
218
+
219
+
220
+ #: R6's subject: the em dash (U+2014) and the en dash (U+2013), as a regex character class.
221
+ #: β›” BUILT FROM CODE POINTS, NOT TYPED, AND THIS IS NOT COSMETIC. `web_prose` reads a Python
222
+ #: string's VALUE off the AST, so a class written `'[<em><en>]'` is reported as two findings in
223
+ #: the one function whose entire job is to REMOVE those characters. That is a false positive, and
224
+ #: the wrong fix would be to silence the gate or to weaken the pattern. `chr()` keeps the value
225
+ #: byte-identical while putting no dash in a literal, so the gate sees what it should see and the
226
+ #: normalizer keeps working. ⚠ Lane D hit the same wall in `routes_query` (note D-13).
227
+ _DASH = '[' + chr(0x2014) + chr(0x2013) + ']'
228
+
229
+
230
+ def no_dashes(text):
231
+ """R6 applied to MODEL-AUTHORED text, before it is stored.
232
+
233
+ β›”β›” A PROMPT INSTRUCTION DOES NOT ENFORCE THIS, AND THE MEASUREMENT IS LANE D'S, NOT A GUESS
234
+ (mailbox note D-4, 2026-08-16): a system prompt ending *"Never use an em dash or an en dash"*
235
+ was answered on the very next live turn with *"Which view type would you like-grid, chart,
236
+ list, or another?"* carrying U+2014 (cerebras). `web_prose` scans SOURCE, so it is
237
+ structurally blind to a dash that arrives at runtime.
238
+ ⚠ AND THIS COLUMN IS THE WORST CASE IN THE WAVE, which is why D routed the note here: chat
239
+ prose is read once, but an enrichment value is WRITTEN INTO A CELL, then re-read and
240
+ re-rendered forever. Normalising after the write would never reach the rows already stored.
241
+
242
+ ⚠ A DIGIT RANGE IS A DIFFERENT SENTENCE and gets the first rule: "10-20" means "10 to 20", and
243
+ rewriting it as "10, 20" states two numbers where the model stated a span. That is a wrong
244
+ value, not a punctuation fix.
245
+
246
+ β›”β›” THIS IS THE SECOND IMPLEMENTATION IN THE PRODUCT and that is a known cost, taken
247
+ deliberately rather than by drift. The first is `routes_query._no_dashes` (lane D's, shipped
248
+ first). Importing it would point a LIBRARY module at a ROUTE module, which is backwards, and
249
+ lane D's file is not in this lane's fence to move it. So the two are held in step by a PARITY
250
+ CHECK instead of by hope: `verify_ai_enrich` section 10 imports D's function and asserts both
251
+ agree over a corpus including every shape either docstring names. One shared home is owed and
252
+ is booked as a PENDING line, not left implicit ([[one-question-two-normalizers]]).
253
+ """
254
+ text = str(text or '')
255
+ text = re.sub(rf'(?<=\d)\s*{_DASH}\s*(?=\d)', ' to ', text)
256
+ text = re.sub(rf'\s*{_DASH}\s*(?=[,.;:!?])', '', text) # abutting punctuation: it just goes
257
+ text = re.sub(rf'(?<=[,;:])\s*{_DASH}\s*', ' ', text) # already punctuated: one space
258
+ return re.sub(rf'\s*{_DASH}\s*', ', ', text)
259
+
260
+
261
+ #: What the model is told about its job. It fills ONE cell, so anything conversational it adds is
262
+ #: a defect in the column rather than a nicety. `UNKNOWN` is the honest out: a row the data cannot
263
+ #: answer is recorded as an error against that row, never as an invented value.
264
+ #: ⚠ The dash sentence is here for the same reason lane D kept theirs: it costs nothing and it
265
+ #: reduces how often `no_dashes` has to act. It is NOT the enforcement; `no_dashes` is.
266
+ _SYSTEM = ('You fill in ONE cell of a spreadsheet. Answer with the value only: no preamble, no '
267
+ 'quotes, no markdown, no explanation. Never use an em dash or an en dash; use a comma, '
268
+ 'a colon or a full stop. If the information given does not let you answer, reply with '
269
+ 'exactly: UNKNOWN')
270
+
271
+
272
+ def _ask(provider, model, prompt, max_tokens, timeout):
273
+ """One cell's answer. `(text, tokens_or_None, problem)`; a truthy `problem` means no text.
274
+
275
+ ⚠ Both request shapes are `ai_review`'s, changed in exactly two ways: they carry the COLUMN's
276
+ ceiling instead of a hardcoded 300, and they read `usage` back.
277
+ """
278
+ key = (os.environ.get(provider['env']) or '').strip()
279
+ if not key:
280
+ return '', None, f"{provider['name']} has no key on this deployment"
281
+ try:
282
+ if provider['shape'] == 'anthropic':
283
+ r = requests.post(
284
+ provider['url'], timeout=timeout,
285
+ headers={'x-api-key': key, 'anthropic-version': '2023-06-01',
286
+ 'content-type': 'application/json'},
287
+ json={'model': model, 'max_tokens': max_tokens, 'system': _SYSTEM,
288
+ 'messages': [{'role': 'user', 'content': prompt}]})
289
+ if r.status_code >= 400:
290
+ return '', None, f"{provider['name']} answered {r.status_code}"
291
+ body = r.json()
292
+ # β›” stop_reason FIRST. A safety refusal is a 200 with an EMPTY content list, so
293
+ # reading content[0] before this turns a refusal into an IndexError inside the loop.
294
+ if body.get('stop_reason') == 'refusal':
295
+ return '', _usage_tokens(body), f"{provider['name']} declined this row"
296
+ parts = [b.get('text') or '' for b in (body.get('content') or [])
297
+ if isinstance(b, dict) and b.get('type') == 'text']
298
+ return ''.join(parts).strip(), _usage_tokens(body), ''
299
+ r = requests.post(
300
+ provider['url'], timeout=timeout,
301
+ headers={'Authorization': f"Bearer {key}", 'Content-Type': 'application/json'},
302
+ json={'model': model, 'max_tokens': max_tokens, 'temperature': 0,
303
+ 'messages': [{'role': 'system', 'content': _SYSTEM},
304
+ {'role': 'user', 'content': prompt}]})
305
+ if r.status_code >= 400:
306
+ return '', None, f"{provider['name']} answered {r.status_code}"
307
+ body = r.json()
308
+ choices = body.get('choices') or []
309
+ if not choices:
310
+ return '', _usage_tokens(body), f"{provider['name']} returned no choices"
311
+ text = ((choices[0] or {}).get('message') or {}).get('content') or ''
312
+ return str(text).strip(), _usage_tokens(body), ''
313
+ except Exception as exc: # noqa: BLE001
314
+ return '', None, f"{provider['name']} failed: {type(exc).__name__}"
315
+
316
+
317
+ def on_change_fields(defn, changed_keys):
318
+ """Which `ai_enrich` columns in this table want a run because one of their inputs moved.
319
+
320
+ A column qualifies when its trigger is `on_change` AND its prompt names at least one of the
321
+ columns that just changed. β›” A column that names NOTHING (a prompt with no `{token}`) never
322
+ fires on change, whatever its trigger says: it would re-ask the same question of the same row
323
+ forever, once per unrelated edit, and bill for every one.
324
+
325
+ ⚠ PURE, AND HANDED THE DEFINITION IT SHOULD USE. `automation_engine.grid_hook` calls
326
+ `all_definitions(st)` once PER EVENT, which turns a 20,000-row import into 20,000 whole
327
+ document reads (`D-134`); this function reads no store at all, so the caller can do one read
328
+ for a whole batch. Rebuilding that shape here was the one thing T54's ticket said not to do.
329
+ """
330
+ import core.user_tables as ut
331
+
332
+ touched = {str(k) for k in (changed_keys or ())}
333
+ if not touched:
334
+ return []
335
+ out = []
336
+ for field in ut.ai_enrich_fields(defn):
337
+ cfg = field.get('aiEnrich') or {}
338
+ if (cfg.get('trigger') or {}).get('mode') != 'on_change':
339
+ continue
340
+ refs = set(prompt_refs(cfg.get('prompt')))
341
+ # ⚠ A column never fires on a change to ITSELF. Its own value moving is either the run
342
+ # that just wrote it or a person typing over it, and both would loop.
343
+ refs.discard(str(field.get('key') or ''))
344
+ if refs & touched:
345
+ out.append(field)
346
+ return out
347
+
348
+
349
+ def scheduled_fields(defn):
350
+ """`[(field, cron)]` for every enrichment column that runs on a cadence.
351
+
352
+ β›” IT DOES NOT PARSE THE CRON AND MUST NOT. `core.user_tables` stores that string opaque
353
+ precisely so the cadence vocabulary has ONE owner, and the owner is the scheduler
354
+ (`automation_engine`), not the field layer. A second parser here would be a door that starts
355
+ refusing a cadence the other one accepts, which is exactly the divergence the whole field
356
+ contract keeps paying for ([[one-question-two-normalizers]]).
357
+
358
+ ⚠ SO THIS IS HALF A FEATURE ON PURPOSE, and the other half is a cross-fence ask: something has
359
+ to TICK. Nothing in lane F's fence runs on a timer.
360
+ """
361
+ import core.user_tables as ut
362
+
363
+ out = []
364
+ for field in ut.ai_enrich_fields(defn):
365
+ trigger = ((field.get('aiEnrich') or {}).get('trigger') or {})
366
+ if trigger.get('mode') == 'schedule' and str(trigger.get('cron') or '').strip():
367
+ out.append((field, str(trigger['cron'])))
368
+ return out
369
+
370
+
371
+ def run_field(table_key, col_id, *, st, rows=None, manual=False, policy=None, budget=None,
372
+ timeout=None, ask=None, user=''):
373
+ """Fill one `ai_enrich` column over the rows an automatic run is allowed to touch.
374
+
375
+ Returns a REPORT and never raises for a vendor problem:
376
+ `{planned, filled, failed, skipped{reason: count}, tokens, provider, model,
377
+ limit: <ceiling_report or None>, errors[], problem}`
378
+
379
+ β›” THE RUN BUDGET IS DERIVED FROM THE COLUMN'S OWN CEILING, and the arithmetic is the point:
380
+ `budget = maxTokens * planned`. `maxTokens` bounds the OUTPUT of one call while spend counts
381
+ INPUT tokens too, so a real run crosses this before it finishes every row. That is intended:
382
+ R13 asked for a token-usage limit and a ceiling that can never bite is not one. A caller may
383
+ pass a TIGHTER `budget`; nothing may pass a looser one.
384
+
385
+ β›” TWO STORE WRITES FOR THE WHOLE RUN, not two per row: one `patch_many_cells` for the values
386
+ and one `stamp_ai_enrich` for the marks. Each is a read-modify-write of the whole tenant
387
+ document, which is why `patch_cells` in a loop was never an option (D-134's shape).
388
+
389
+ ⚠ `ask` is injectable so a gate can exercise the ledger, the ceiling and every failure path
390
+ with no network and no vendor bill. It defaults to the real `_ask`.
391
+
392
+ ⭐⭐ W35-T41 / C7 β€” ONE USAGE-LEDGER LINE PER RUN, NOT PER ROW, and the arithmetic is the reason:
393
+ this loop makes one LLM call per row, so a ledger write per call would be hundreds of store
394
+ writes for one click. The run already accumulates `report['tokens']`; `calls` and `unmeasured`
395
+ join it so the meter can say "40 calls, 12,000 tokens, 3 of them unmeasured" instead of a total
396
+ with an unknown shortfall. `user` is the ledger's attribution and is passed by this file's own
397
+ callers in `routes_tables`.
398
+ """
399
+ import ai_review
400
+ import core.user_tables as ut
401
+ import usage_ledger
402
+
403
+ defn = ut.get(table_key, st=st) or {}
404
+ field = next((f for f in ut.ai_enrich_fields(defn) if f.get('key') == str(col_id)), None)
405
+ if field is None:
406
+ return {'problem': f'{col_id!r} is not an AI enrichment column on this database',
407
+ 'planned': 0, 'filled': 0, 'failed': 0, 'skipped': {}, 'tokens': 0,
408
+ 'errors': [], 'limit': None, 'provider': '', 'model': ''}
409
+ cfg = field.get('aiEnrich') or {}
410
+ # ⭐ THE BULK MENU'S TWO CHOICES ARE THIS ONE OVERRIDE (`W34-T54`): "Rows never filled" is
411
+ # `blank` and "All rows" is `always`. β›” It overrides the column's SAVED POLICY for one run and
412
+ # NOTHING ELSE: `ai_enrich_may_write` still refuses a human-edited cell under every value, so
413
+ # "All rows" means every row the AGENT wrote, which is exactly what the menu label says.
414
+ # ⚠ An unrecognised value falls back to the column's own setting rather than to a default: a
415
+ # typo'd override must not quietly widen what a run touches.
416
+ if policy in ut.AI_ENRICH_OVERWRITE:
417
+ cfg = {**cfg, 'overwrite': policy}
418
+ all_rows = defn.get('rows') or {}
419
+ marks = ut.ai_enrich_marks(table_key, col_id, st=st)
420
+ # β›”β›” `rows` NARROWS; `manual` DECIDES WHETHER THE POLICY APPLIES. They are two questions and
421
+ # an early draft of this function conflated them, which would have let an ON-CHANGE fire (an
422
+ # automatic act, scoped to one row) overwrite a cell a person had typed into. That is the one
423
+ # law this whole feature is arranged around, broken by a parameter name.
424
+ # rows=None -> the automatic plan over every row
425
+ # rows=[...], manual=False -> the automatic plan, INTERSECTED with those rows (on_change)
426
+ # rows=[...], manual=True -> exactly those rows, policy skipped
427
+ # `manual` is skippable because a person clicking "run this cell" has asked, in front of the
428
+ # value being replaced; refusing them would be the product overruling an explicit act with a
429
+ # rule written for an implicit one. `ai_enrich_may_write`'s own docstring says exactly this.
430
+ # ⚠ A named row is always INTERSECTED with the table's own row set, so a stale caller cannot
431
+ # name a record that has since been deleted.
432
+ named = None if rows is None else [str(r) for r in rows if str(r) in all_rows]
433
+ if named is not None and manual:
434
+ plan = {'run': named, 'skipped': {},
435
+ 'estimate': int(cfg.get('maxTokens') or 0) * len(named)}
436
+ else:
437
+ plan = plan_rows(all_rows, col_id, cfg, marks)
438
+ if named is not None:
439
+ keep = set(named)
440
+ dropped = [r for r in plan['run'] if r not in keep]
441
+ plan = {**plan, 'run': [r for r in plan['run'] if r in keep]}
442
+ if dropped:
443
+ plan['skipped'] = {**plan['skipped'], 'not_in_this_run': len(dropped)}
444
+ rows = all_rows
445
+ keys = [f.get('key') for f in (defn.get('fields') or []) if f.get('key')]
446
+ report = {'planned': len(plan['run']), 'filled': 0, 'failed': 0,
447
+ 'skipped': dict(plan['skipped']), 'tokens': 0, 'errors': [], 'limit': None,
448
+ 'provider': '', 'model': '', 'problem': '',
449
+ # ⭐ C7's two counters. `calls` is what the meter bills; `unmeasured` is how many of
450
+ # them the provider declined to report tokens for, so a total is never presented as
451
+ # complete when it is a floor.
452
+ 'calls': 0, 'unmeasured': 0}
453
+ if not plan['run']:
454
+ return report
455
+
456
+ caller = ask or _ask
457
+ ceiling = int(cfg.get('maxTokens') or 0) * len(plan['run'])
458
+ if isinstance(budget, int) and not isinstance(budget, bool):
459
+ ceiling = min(ceiling, max(0, budget))
460
+ live = ai_review.ladder()
461
+ if not live and ask is None:
462
+ # β›” FAIL CLOSED AND SAY SO. A run that quietly filled nothing looks identical to a run
463
+ # with nothing to do; `planned` above is the number that distinguishes them.
464
+ report['problem'] = 'no AI provider is configured on this deployment'
465
+ return report
466
+ provider = (live or [{'name': 'injected', 'model': 'injected', 'env': '',
467
+ 'shape': 'openai', 'url': ''}])[0]
468
+ model = str(cfg.get('model') or provider['model'])
469
+ report['provider'], report['model'] = provider['name'], model
470
+ tmo = float(timeout or TIMEOUT_SECONDS)
471
+
472
+ values, stamps = {}, {}
473
+ for row_id in plan['run']:
474
+ if ceiling and report['tokens'] >= ceiling:
475
+ # β›” STOP AND REPORT (R6's second sentence). Cells already written STAY written and
476
+ # carry honest marks; nothing is truncated and no cell holds half an answer.
477
+ report['limit'] = ceiling_report(report['tokens'], ceiling,
478
+ report['filled'], report['planned'])
479
+ left = report['planned'] - report['filled'] - report['failed']
480
+ if left > 0:
481
+ report['skipped']['over_budget'] = left
482
+ break
483
+ row = rows.get(row_id) or {}
484
+ text, unresolved = render_prompt(cfg.get('prompt'), row, keys)
485
+ if unresolved:
486
+ # Reported per row rather than refused up front: the prompt is valid for the column,
487
+ # and naming the missing token is what lets somebody fix it.
488
+ why = ('the prompt names ' + ', '.join(unresolved)
489
+ + ', which is not a column on this database')
490
+ report['failed'] += 1
491
+ stamps[row_id] = {'state': 'error', 'error': why}
492
+ if len(report['errors']) < MAX_REPORTED_ERRORS:
493
+ report['errors'].append({'row': row_id, 'error': why})
494
+ continue
495
+ answer, tokens, problem = caller(provider, model, text[:MAX_CELL_CHARS * 8],
496
+ int(cfg.get('maxTokens') or 300), tmo)
497
+ # ⚠ COUNTED HERE, ON EVERY OUTCOME β€” before the `problem` branch below, because a call that
498
+ # errored or answered UNKNOWN still reached the vendor and is still billed. A meter that
499
+ # counts only filled cells reports a cheap run for one that spent its whole budget failing.
500
+ report['calls'] += 1
501
+ if isinstance(tokens, int) and not isinstance(tokens, bool):
502
+ report['tokens'] += tokens
503
+ else:
504
+ report['unmeasured'] += 1
505
+ if problem or not answer or answer.strip().upper() == 'UNKNOWN':
506
+ # β›” THE CELL IS LEFT ALONE. An errored row keeps whatever it held; only the MARK
507
+ # changes, so a failed run never destroys a value it could not replace.
508
+ why = problem or ('the model had nothing to answer from' if not answer
509
+ else 'the model answered UNKNOWN for this row')
510
+ report['failed'] += 1
511
+ stamps[row_id] = {'state': 'error', 'error': why, 'model': model}
512
+ if len(report['errors']) < MAX_REPORTED_ERRORS:
513
+ report['errors'].append({'row': row_id, 'error': why})
514
+ continue
515
+ # β›” NORMALISED BEFORE THE WRITE, NOT AFTER (note D-6). This is the ONE line where model
516
+ # prose becomes stored product data; a cell fixed after the fact leaves every row already
517
+ # written carrying the dash, and `web_prose` can never see it because it scans source.
518
+ values[row_id] = no_dashes(answer)[:MAX_CELL_CHARS]
519
+ stamps[row_id] = {'state': 'agent', 'model': model,
520
+ 'hash': ut.ai_enrich_input_hash(cfg.get('prompt'), row,
521
+ prompt_refs(cfg.get('prompt'))),
522
+ 'tokens': tokens if isinstance(tokens, int) else 0}
523
+ report['filled'] += 1
524
+
525
+ if values:
526
+ ut.patch_many_cells(table_key, {r: {str(col_id): v} for r, v in values.items()}, st=st)
527
+ if stamps:
528
+ ut.stamp_ai_enrich(table_key, col_id, stamps, st=st)
529
+ # ⚠ The ceiling is re-asked AFTER the loop too: a run whose LAST row crossed the budget spent
530
+ # over it without re-entering the guard, and staying silent there would under-report the one
531
+ # run most likely to surprise somebody.
532
+ if report['limit'] is None and ceiling and report['tokens'] >= ceiling:
533
+ report['limit'] = ceiling_report(report['tokens'], ceiling,
534
+ report['filled'], report['planned'])
535
+ # ⭐⭐ C7's LEDGER LINE β€” one per run, after both store writes, and only when the run actually
536
+ # called something. `total=` rather than a split: `_ask`'s contract returns a total and a gate
537
+ # injects against that shape, so splitting it here would mean inventing two numbers from one.
538
+ #
539
+ # ⚠ TWO LINES WHEN A RUN HAS BOTH KINDS, and it is not tidiness. `record` marks a line
540
+ # `unmeasured` only when its total is None, so ONE line carrying 40 calls and a partial total
541
+ # would book all forty as measured and lose the shortfall β€” a total presented as complete when
542
+ # it is a floor, which is the exact failure `_usage_tokens`' None-semantics exists to prevent.
543
+ _measured = report['calls'] - report['unmeasured']
544
+ if _measured > 0:
545
+ usage_ledger.record('field_agent', report['provider'], report['model'],
546
+ total=report['tokens'], calls=_measured, st=st, user=user)
547
+ if report['unmeasured'] > 0:
548
+ usage_ledger.record('field_agent', report['provider'], report['model'],
549
+ total=None, calls=report['unmeasured'], st=st, user=user)
550
+ return report
api/ai_review.py CHANGED
@@ -130,6 +130,17 @@ def _parse(text, options):
130
  return "", reason
131
 
132
 
 
 
 
 
 
 
 
 
 
 
 
133
  def _call_openai(p, model, system, user, timeout):
134
  r = requests.post(p["url"], timeout=timeout,
135
  headers={"Authorization": f"Bearer {os.environ[p['env']].strip()}",
@@ -138,12 +149,14 @@ def _call_openai(p, model, system, user, timeout):
138
  "messages": [{"role": "system", "content": system},
139
  {"role": "user", "content": user}]})
140
  if r.status_code >= 400:
141
- return "", f"{p['name']} answered {r.status_code}"
 
 
142
  body = r.json()
143
  choices = body.get("choices") or []
144
  if not choices:
145
- return "", f"{p['name']} returned no choices"
146
- return str(((choices[0] or {}).get("message") or {}).get("content") or ""), ""
147
 
148
 
149
  def _call_anthropic(p, model, system, user, timeout):
@@ -154,25 +167,34 @@ def _call_anthropic(p, model, system, user, timeout):
154
  json={"model": model, "max_tokens": 300, "system": system,
155
  "messages": [{"role": "user", "content": user}]})
156
  if r.status_code >= 400:
157
- return "", f"anthropic answered {r.status_code}"
158
  body = r.json()
159
  # β›” stop_reason FIRST. A safety refusal is a successful 200 with an EMPTY content list, so
160
  # reading content[0] before this check turns a refusal into an IndexError inside a run.
 
161
  if body.get("stop_reason") == "refusal":
162
- return "", "anthropic declined to answer this record"
163
  parts = [b.get("text") or "" for b in (body.get("content") or [])
164
  if isinstance(b, dict) and b.get("type") == "text"]
165
  if not parts:
166
- return "", "anthropic returned no text"
167
- return "".join(parts), ""
168
 
169
 
170
- def decide(*, prompt, options, row, fields=(), label="Review", timeout=None):
171
  """Pick this record's next stage. Returns `(choice, meta)`.
172
 
173
  `choice` is "" whenever a person should decide β€” which is every failure mode there is.
174
  `meta` carries `provider`, `model`, `reason` on success, and `problem` on refusal to answer.
 
 
 
 
 
 
 
175
  """
 
176
  opts = [str(o) for o in (options or []) if str(o).strip()]
177
  if not opts:
178
  return "", {"problem": "the review offers no next stages"}
@@ -188,11 +210,22 @@ def decide(*, prompt, options, row, fields=(), label="Review", timeout=None):
188
  problems = []
189
  for p in live:
190
  model = override or p["model"]
 
191
  try:
192
- text, err = (_call_anthropic if p["shape"] == "anthropic" else _call_openai)(
193
  p, model, system, user, tmo)
194
  except Exception as e: # noqa: BLE001
195
  text, err = "", f"{p['name']} failed: {type(e).__name__}"
 
 
 
 
 
 
 
 
 
 
196
  if err:
197
  problems.append(err)
198
  continue # ladder: a dead provider degrades to the next one
@@ -205,3 +238,275 @@ def decide(*, prompt, options, row, fields=(), label="Review", timeout=None):
205
  "problem": reason or "the model did not choose one of the stages"}
206
  return choice, {"provider": p["name"], "model": model, "reason": reason}
207
  return "", {"problem": "; ".join(problems)[:300] or "no provider answered"}
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
130
  return "", reason
131
 
132
 
133
+ # ⭐⭐ WAVE 35 Β· W35-T41 / C7 β€” THE TWO TRANSPORTS RETURN THE RESPONSE BODY'S `usage`.
134
+ #
135
+ # β›” THIS FILE'S OWN HEADER USED TO BE THE PRODUCT'S CONFESSION THAT NOTHING COUNTED: `ai_enrich`'s
136
+ # docstring says *"`ai_review.decide` -- the product's only other LLM entry -- has no token
137
+ # accounting of ANY kind"*. R9 asks for ONE meter for every AI surface, so the counting has to reach
138
+ # this transport rather than be bolted onto its caller β€” the caller never sees the body.
139
+ #
140
+ # ⚠ A THIRD RETURN VALUE, NOT A MUTATED ARGUMENT, and both call sites are in `decide` below. The
141
+ # tuple grew from `(text, err)` to `(text, err, usage)`; `usage` is the raw `usage` OBJECT (or None),
142
+ # because reading it is `usage_ledger`'s job and a second reader here would be the two-normalizers
143
+ # shape this repo keeps paying for.
144
  def _call_openai(p, model, system, user, timeout):
145
  r = requests.post(p["url"], timeout=timeout,
146
  headers={"Authorization": f"Bearer {os.environ[p['env']].strip()}",
 
149
  "messages": [{"role": "system", "content": system},
150
  {"role": "user", "content": user}]})
151
  if r.status_code >= 400:
152
+ # ⚠ NO BODY ON A 4xx/5xx: an error envelope carries no usage, and a provider that refused
153
+ # before running the model has nothing to bill. The call is still COUNTED by `decide`.
154
+ return "", f"{p['name']} answered {r.status_code}", None
155
  body = r.json()
156
  choices = body.get("choices") or []
157
  if not choices:
158
+ return "", f"{p['name']} returned no choices", body
159
+ return str(((choices[0] or {}).get("message") or {}).get("content") or ""), "", body
160
 
161
 
162
  def _call_anthropic(p, model, system, user, timeout):
 
167
  json={"model": model, "max_tokens": 300, "system": system,
168
  "messages": [{"role": "user", "content": user}]})
169
  if r.status_code >= 400:
170
+ return "", f"anthropic answered {r.status_code}", None
171
  body = r.json()
172
  # β›” stop_reason FIRST. A safety refusal is a successful 200 with an EMPTY content list, so
173
  # reading content[0] before this check turns a refusal into an IndexError inside a run.
174
+ # ⚠ A refusal IS billed and its body carries `usage`, so the body rides back on this branch too.
175
  if body.get("stop_reason") == "refusal":
176
+ return "", "anthropic declined to answer this record", body
177
  parts = [b.get("text") or "" for b in (body.get("content") or [])
178
  if isinstance(b, dict) and b.get("type") == "text"]
179
  if not parts:
180
+ return "", "anthropic returned no text", body
181
+ return "".join(parts), "", body
182
 
183
 
184
+ def decide(*, prompt, options, row, fields=(), label="Review", timeout=None, st=None, user=""):
185
  """Pick this record's next stage. Returns `(choice, meta)`.
186
 
187
  `choice` is "" whenever a person should decide β€” which is every failure mode there is.
188
  `meta` carries `provider`, `model`, `reason` on success, and `problem` on refusal to answer.
189
+
190
+ ⭐ W35-T41 / C7 β€” `st` and `user` are the USAGE LEDGER's target. Both default to absent because
191
+ this function's caller is `automation_engine.ai_decide(rt, ...)`, in another lane's fence: it
192
+ HAS the runtime and does not pass it yet, so until it does, a review's tokens are counted as
193
+ unattributed and REPORTED by `GET /usage` rather than dropped. See `usage_ledger`'s header for
194
+ why they cannot be resolved implicitly (measured: a contextvar does not survive a FastAPI
195
+ dependency).
196
  """
197
+ import usage_ledger # noqa: PLC0415
198
  opts = [str(o) for o in (options or []) if str(o).strip()]
199
  if not opts:
200
  return "", {"problem": "the review offers no next stages"}
 
210
  problems = []
211
  for p in live:
212
  model = override or p["model"]
213
+ body = None
214
  try:
215
+ text, err, body = (_call_anthropic if p["shape"] == "anthropic" else _call_openai)(
216
  p, model, system, user, tmo)
217
  except Exception as e: # noqa: BLE001
218
  text, err = "", f"{p['name']} failed: {type(e).__name__}"
219
+ # ⭐⭐ C7 β€” THE LEDGER LINE, BEFORE ANY BRANCH BELOW READS THE ANSWER.
220
+ # β›” IT IS WRITTEN ON EVERY OUTCOME THAT REACHED A PROVIDER, including a refusal and an
221
+ # unusable answer. A meter that books only successes reports a cheap week for a run that
222
+ # spent its budget being declined β€” and a declined call is billed. The only path that does
223
+ # NOT record is a transport that never reached the vendor (`body is None`), which spent
224
+ # nothing.
225
+ if body is not None:
226
+ ins, outs = usage_ledger.tokens_from(body)
227
+ usage_ledger.record("ai_review", p["name"], model, ins, outs,
228
+ total=usage_ledger.total_from(body), st=st, user=user)
229
  if err:
230
  problems.append(err)
231
  continue # ladder: a dead provider degrades to the next one
 
238
  "problem": reason or "the model did not choose one of the stages"}
239
  return choice, {"provider": p["name"], "model": model, "reason": reason}
240
  return "", {"problem": "; ".join(problems)[:300] or "no provider answered"}
241
+
242
+
243
+ # ══════════════════════════════════════════════════════════════════ the FLOW WRITER
244
+ # ⭐⭐ WAVE 33 Β· W33-T54/T55 (owner item 7, ruling R3) β€” A PROMPT BECOMES A DRAFT AUTOMATION.
245
+ #
246
+ # Owner, verbatim: *"lay out the foundation with custom tools we provide"* β€” and the tools are the
247
+ # action catalog, which has been machine-readable since wave 23. So this writes no new vocabulary:
248
+ # it hands the model the SAME `ACTION_CATALOG` the menu paints, the SAME `ACTION_REQUIRED` the
249
+ # runner blocks on, and the SAME trigger keys `clean_trigger` accepts, and asks for one JSON object
250
+ # in that vocabulary.
251
+ #
252
+ # β›” WHY IT LIVES IN `ai_review.py` RATHER THAN A NEW FILE. Two reasons, both about ownership. This
253
+ # is the one module in the API package that already holds an LLM ladder and its credentials, so a
254
+ # second one would be a second place a rotated key has to be noticed; and a new `routes_*.py` needs
255
+ # a `main.py` line from another lane (contract C2) to be reachable at all, which is how wave 23
256
+ # shipped three finished routers 404-dead behind green gates. The DOOR is a route on
257
+ # `routes_automation.py`, which is already mounted.
258
+ #
259
+ # β›”β›” CONTRACT C7 IS THE DESIGN, NOT A CHECK AT THE END: *"the flow writer may emit only kinds
260
+ # present in `ACTION_CATALOG`, and its output must pass `clean_actions` unchanged."* Both halves are
261
+ # enforced mechanically rather than asked for politely β€” the kind list is an `enum` in the schema
262
+ # the model fills, and the caller runs `clean_actions` and DIFFS the result. That diff matters
263
+ # because `clean_actions` has no disclosure channel (D-75): it drops a key it does not recognise
264
+ # and answers 200, so a draft accepted without the diff would show a person a flow that is not the
265
+ # flow the model described, with nothing anywhere saying so.
266
+ #
267
+ # ⚠ `chat` IS INJECTABLE, exactly as `routes_query._call_model`'s is, and it is the reason this door
268
+ # can be proven end to end with no API key and no spend. A gate that can only run where a
269
+ # credential exists is a gate that never runs.
270
+
271
+ #: Cerebras first, NOT the cheap-first order `ladder()` uses. Same reason `routes_query` inverts it:
272
+ #: this contract includes a REFUSAL, and refusing honestly ("I cannot build that from the steps you
273
+ #: have") is a model property. A cheaper rung answers an impossible request with a plausible flow,
274
+ #: which is worse than no flow. Override with `AIOS_FLOW_PROVIDER`.
275
+ FLOW_PROVIDER_ORDER = ("cerebras", "groq", "openrouter", "anthropic")
276
+ MAX_DRAFT_ACTIONS = 12 # a draft a person reads in one screen; the engine's own cap is 25
277
+ MAX_PROMPT_CHARS = 2000
278
+
279
+
280
+ def flow_providers():
281
+ """The rungs usable here, in THIS module's refusal-first order. Empty = the feature is off."""
282
+ pin = (os.environ.get("AIOS_FLOW_PROVIDER") or "").strip().lower()
283
+ by_name = {p["name"]: p for p in PROVIDERS}
284
+ order = [pin] if pin else list(FLOW_PROVIDER_ORDER)
285
+ return [by_name[n] for n in order
286
+ if n in by_name and (os.environ.get(by_name[n]["env"]) or "").strip()]
287
+
288
+
289
+ def flow_schema(kinds, trigger_keys, table_keys):
290
+ """The tool schema the model fills β€” the catalog's OWN key lists as enums.
291
+
292
+ β›” `kind` AND `trigger` ARE ENUMS, not free strings, and that is C7's first half enforced by the
293
+ transport rather than by a check afterwards. A model asked for "any action name" invents
294
+ `send_slack_message` on a deployment that has no such kind, and the failure then surfaces as a
295
+ 400 from `clean_actions` carrying a sentence about a word the person never typed.
296
+ ⚠ `required` IS `["kind"]` ALONE, deliberately: a REFUSAL carries no name and no actions, and a
297
+ schema demanding them turns an honest refusal into a provider-side 400 that reads exactly like a
298
+ transport failure (`routes_query._spec_schema` carries the same note, for the same measured
299
+ reason).
300
+ """
301
+ return {
302
+ "type": "object",
303
+ "properties": {
304
+ "kind": {"type": "string", "enum": ["flow", "refused"],
305
+ "description": "refused = this cannot be built from the steps available"},
306
+ "refusal": {"type": "string",
307
+ "description": "when kind=refused: ONE plain sentence naming what is "
308
+ "missing, in the words a non-technical person would use"},
309
+ "name": {"type": "string", "description": "a short title for the automation"},
310
+ "trigger": {"type": "string", "enum": sorted(trigger_keys),
311
+ "description": "what starts this automation"},
312
+ "table": {"type": "string", "enum": sorted(table_keys),
313
+ "description": "the database whose records this flow walks, if any"},
314
+ "actions": {
315
+ "type": "array",
316
+ "maxItems": MAX_DRAFT_ACTIONS,
317
+ "items": {
318
+ "type": "object",
319
+ "properties": {
320
+ "kind": {"type": "string", "enum": sorted(kinds)},
321
+ "why": {"type": "string",
322
+ "description": "one short sentence: why this step is here"},
323
+ "config": {"type": "object", "additionalProperties": True,
324
+ "description": "the step's settings, using ONLY the keys named "
325
+ "for that kind in the system message"},
326
+ },
327
+ "required": ["kind"],
328
+ },
329
+ },
330
+ },
331
+ "required": ["kind"],
332
+ }
333
+
334
+
335
+ def flow_system_prompt(catalog, required, triggers, tables):
336
+ """What the model is told it may build with β€” DERIVED, never written down twice.
337
+
338
+ Every list here is the server's own: the catalog rows the menu paints, the required-key table
339
+ the runner blocks on, the trigger keys `clean_trigger` accepts, and this tenant's real databases
340
+ with their real columns. Nothing about the vocabulary is restated by hand, so a kind added to
341
+ the catalog is offered here on the same deploy and a kind removed stops being offered.
342
+ """
343
+ lines = ["You build small automations for a business tool. You are given the EXACT set of "
344
+ "steps this tool can perform. You may use nothing else.",
345
+ "", "THE STEPS YOU MAY USE:"]
346
+ # ⚠ BOTH HALVES, `key (phrase)`. The KEY is what the model must write into `config` and the
347
+ # PHRASE is the only human wording of that requirement anywhere β€” the one the runner's own
348
+ # refusal sentence is built from. Keys alone leave the model guessing what `field` means on a
349
+ # `web_read` (it is a column to write into, not a form field); phrases alone leave it guessing
350
+ # what to call the setting. Handing it one and hoping is how a draft comes back configured
351
+ # against a key the validator drops.
352
+ req = {str(k): [f"{key} ({phrase})" for phrase, key in v] for k, v in (required or {}).items()}
353
+ for row in catalog or []:
354
+ if not row.get("ready"):
355
+ continue
356
+ kind = str(row.get("kind") or "")
357
+ need = req.get(kind) or []
358
+ lines.append(f"- {kind}: {row.get('label')} - {row.get('detail') or ''}"
359
+ + (f" REQUIRED settings: {', '.join(need)}" if need else ""))
360
+ lines += ["", "WHAT CAN START AN AUTOMATION:"]
361
+ for t in triggers or []:
362
+ if t.get("planned") or not t.get("ready", True):
363
+ continue
364
+ lines.append(f"- {t.get('key')}: {t.get('label')}")
365
+ lines += ["", "THE DATABASES THIS PERSON HAS:"]
366
+ for t in (tables or [])[:40]:
367
+ cols = ", ".join(str(f.get("key")) for f in (t.get("fields") or [])[:25])
368
+ lines.append(f"- {t.get('key')} ({t.get('label')}): {cols or 'no columns yet'}")
369
+ if not tables:
370
+ lines.append("- (none - do not name a database)")
371
+ lines += [
372
+ "",
373
+ "RULES:",
374
+ "1. Use ONLY the step kinds listed above. If what is asked needs a step that is not there, "
375
+ "answer kind=refused and say plainly which capability is missing.",
376
+ "2. Fill in every REQUIRED setting you can from what the person told you. Leave one blank "
377
+ "rather than inventing a web address, a CSS selector or a column name.",
378
+ "3. A web address must start with http:// or https://.",
379
+ "4. To use a value from the record the flow is walking, write it as {{Column name}}.",
380
+ "5. Name a database only from the list above, by its key.",
381
+ "6. Keep it short. Fewer steps that work beat more steps that guess.",
382
+ ]
383
+ return "\n".join(lines)
384
+
385
+
386
+ def _flow_from_tool_call(obj):
387
+ """The model's tool arguments -> `(draft, refusal)`. A shape error is a refusal, never a crash."""
388
+ if not isinstance(obj, dict):
389
+ return None, "the assistant's answer could not be read"
390
+ if str(obj.get("kind") or "") == "refused":
391
+ return None, (str(obj.get("refusal") or "").strip()
392
+ or "the assistant could not build this from the steps available")
393
+ acts = obj.get("actions")
394
+ if not isinstance(acts, list) or not acts:
395
+ return None, ("the assistant did not produce any steps - try describing what should "
396
+ "happen, one action at a time")
397
+ out = []
398
+ for a in acts[:MAX_DRAFT_ACTIONS]:
399
+ if not isinstance(a, dict) or not str(a.get("kind") or "").strip():
400
+ continue
401
+ cfg = a.get("config")
402
+ out.append({"kind": str(a["kind"]).strip(),
403
+ "config": cfg if isinstance(cfg, dict) else {},
404
+ "why": str(a.get("why") or "").strip()[:200]})
405
+ if not out:
406
+ return None, "the assistant's steps could not be read"
407
+ return {"name": str(obj.get("name") or "").strip()[:80] or "New automation",
408
+ "trigger": str(obj.get("trigger") or "").strip(),
409
+ "table": str(obj.get("table") or "").strip(),
410
+ # β›”β›” THE TRUNCATION IS REPORTED, and it is a STANDING RULE that it must be (owner,
411
+ # 2026-08-12, W30/R6 second sentence): *"if there is lag or it can't be done, you need
412
+ # to explicitly tell me why and recommend a fix"* β€” **a silent truncation IS the
413
+ # violation, not the limit.** `acts[:MAX_DRAFT_ACTIONS]` above dropped everything past
414
+ # the twelfth and said nothing, so a model that answered with a twenty-step journey had
415
+ # eight steps deleted between the answer and the screen, with no key anywhere in the
416
+ # response naming them. `MAX_ACTIONS` is 20, so those steps were STORABLE β€” this
417
+ # ceiling is the draft door's own, which makes reporting it the whole obligation.
418
+ # ⚠ Found by the verifier that checked this ticket, not by a gate.
419
+ "asked": len([a for a in acts if isinstance(a, dict)]),
420
+ "actions": out}, ""
421
+
422
+
423
+ def draft_flow(*, prompt, catalog, required, triggers, tables, chat=None, timeout=None,
424
+ st=None, user=""):
425
+ """A sentence -> `(draft, refusal_sentence, provider)`. β›” NOTHING IS SAVED HERE.
426
+
427
+ Exactly one of `draft` and `refusal_sentence` is truthy β€” the same contract
428
+ `web_agent.run_step` keeps, so a caller has no third case to get wrong.
429
+
430
+ ⭐ W35-T41 / C7 β€” `st`/`user` are the usage ledger's target, exactly as on `decide` above and for
431
+ the same reason: both of this function's callers (`automation_engine._ai_agent_plan` and
432
+ `routes_automation`'s draft door) are in lane D's fence. C7 says E adds the ledger line here and
433
+ D asserts it; the two keywords are what D has to pass for the line to be attributable.
434
+ """
435
+ import usage_ledger # noqa: PLC0415
436
+ text = str(prompt or "").strip()[:MAX_PROMPT_CHARS]
437
+ if not text:
438
+ return None, "type what you want the automation to do", None
439
+ kinds = sorted({str(r.get("kind")) for r in (catalog or []) if r.get("ready")})
440
+ if not kinds:
441
+ return None, "this deployment offers no automation steps to build with", None
442
+ trigger_keys = sorted({str(t.get("key")) for t in (triggers or [])
443
+ if t.get("key") and not t.get("planned")}) or ["manual"]
444
+ table_keys = sorted({str(t.get("key")) for t in (tables or []) if t.get("key")}) or [""]
445
+ tools = [{"type": "function", "function": {
446
+ "name": "build_automation",
447
+ "description": "Emit the automation, or refuse.",
448
+ "parameters": flow_schema(kinds, trigger_keys, table_keys)}}]
449
+ messages = [{"role": "system",
450
+ "content": flow_system_prompt(catalog, required, triggers, tables)},
451
+ {"role": "user", "content": text}]
452
+
453
+ if chat is not None:
454
+ # ⚠ THE INJECTED PATH IS THE PROVEN PATH. It runs the SAME parse and the SAME refusal
455
+ # branches as a live call; only the transport is replaced.
456
+ draft, refusal = _flow_from_tool_call(chat(messages, tools))
457
+ return draft, refusal, "injected"
458
+
459
+ provs = flow_providers()
460
+ if not provs:
461
+ # β›” SAY SO. An AI feature that silently does nothing is indistinguishable from one that was
462
+ # never built [[flag-shipped-without-its-writer]].
463
+ return None, ("the assistant is not configured on this deployment, so an automation cannot "
464
+ "be drafted from a description yet"), None
465
+ tmo = float(timeout or TIMEOUT_SECONDS)
466
+ problems = []
467
+ for p in provs:
468
+ if p["shape"] != "openai":
469
+ # Anthropic's tool transport differs; it stays on the ladder for `decide()` and is
470
+ # SKIPPED here rather than half-supported. Named in `problems` so it is not invisible.
471
+ problems.append(f"{p['name']}: tool calls are not wired for this shape")
472
+ continue
473
+ try:
474
+ r = requests.post(p["url"], timeout=tmo,
475
+ headers={"Authorization": f"Bearer {os.environ[p['env']].strip()}",
476
+ "Content-Type": "application/json"},
477
+ json={"model": p["model"], "messages": messages, "tools": tools,
478
+ "tool_choice": "required", "temperature": 0.1,
479
+ "max_tokens": 1500})
480
+ except Exception as e: # noqa: BLE001
481
+ problems.append(f"{p['name']}: {type(e).__name__}")
482
+ continue
483
+ if r.status_code != 200:
484
+ problems.append(f"{p['name']}: HTTP {r.status_code}")
485
+ continue
486
+ try:
487
+ body = r.json()
488
+ calls = (((body.get("choices") or [{}])[0].get("message") or {})
489
+ .get("tool_calls") or [])
490
+ args = json.loads(calls[0]["function"]["arguments"]) if calls else None
491
+ except Exception as e: # noqa: BLE001
492
+ problems.append(f"{p['name']}: unreadable answer ({type(e).__name__})")
493
+ continue
494
+ # ⭐⭐ C7 β€” THE LEDGER LINE. Placed after the parse rather than before it so an UNREADABLE
495
+ # answer is not double-counted by the `continue` above... ⚠ which means an unreadable answer
496
+ # is NOT counted at all, and that is a deliberate, disclosed loss: a body this code cannot
497
+ # parse is a body whose `usage` it also cannot trust, and `body` is out of scope in that
498
+ # branch by construction. The 200-with-junk case is rare and named here rather than
499
+ # silently rounded to zero.
500
+ ins, outs = usage_ledger.tokens_from(body)
501
+ usage_ledger.record("automation_draft", p["name"], p["model"], ins, outs,
502
+ total=usage_ledger.total_from(body), st=st, user=user)
503
+ draft, refusal = _flow_from_tool_call(args)
504
+ if draft is None and isinstance(args, dict) and str(args.get("kind")) == "refused":
505
+ # β›” A REFUSAL IS AN ANSWER, NOT A FAULT, so it does NOT fall through to a more
506
+ # expensive rung β€” the model has just said this cannot be built. `decide()` takes the
507
+ # same posture for the same reason.
508
+ return None, refusal, p["name"]
509
+ if draft is not None:
510
+ return draft, "", p["name"]
511
+ problems.append(f"{p['name']}: {refusal}")
512
+ return None, ("; ".join(problems)[:300] or "no provider answered"), None
api/automation_engine.py CHANGED
The diff for this file is too large to render. See raw diff
 
api/deps.py CHANGED
@@ -10,6 +10,7 @@ epoch-revoked, or naming an unknown tenant β†’ **401**. Authenticated but not gr
10
  **403**. Never an empty 200: an empty list is a legitimate answer meaning "no rows", and using it
11
  to mean "you are not allowed" is how a permission bug becomes invisible.
12
  """
 
13
  import os
14
  import sys
15
  import time
@@ -68,6 +69,418 @@ class Session:
68
  f"this workspace does not include {module_key}")
69
 
70
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
71
  def _user_for(claims):
72
  """The account named by a verified cookie, or None β€” with the epoch check that makes a
73
  stateless cookie revocable.
@@ -110,7 +523,14 @@ def _user_for(claims):
110
  # (login validated the slug's existence, never the membership) β€” with
111
  # AIOS_ENABLE_QA_TENANT=1 that was a working cross-tenant session. A signature proves
112
  # the cookie is ours; THIS proves the account belongs where the cookie says it does.
113
- if claim_tenant != pub.get("tenant", "royal-imports"):
 
 
 
 
 
 
 
114
  return None
115
  return pub
116
  if (uname == "admin" and os.environ.get("APP_PASSWORD", "")
 
10
  **403**. Never an empty 200: an empty list is a legitimate answer meaning "no rows", and using it
11
  to mean "you are not allowed" is how a permission bug becomes invisible.
12
  """
13
+ import copy
14
  import os
15
  import sys
16
  import time
 
69
  f"this workspace does not include {module_key}")
70
 
71
 
72
+ # ── Assistant: app-stored, permissioned snapshot boundary (W33-T71 / C8) ───────────────────────
73
+ #
74
+ # This is deliberately an ordinary function rather than a route. E owns the Assistant's tool
75
+ # routes and model/provider choices; every one of those tools must come through this one boundary
76
+ # before it can see a database. Keeping it here gives the reader the already-verified Session and
77
+ # prevents Query's schema-only `_target` helper from becoming an accidental data authority.
78
+ #
79
+ # The important absence is as much part of the contract as the code below: this module never calls
80
+ # the Customer/Product pool builders, connector routes, provider routes, Odoo, or an environment
81
+ # lookup from this boundary. A cache miss is a named refusal, never permission to refresh from the
82
+ # source of truth. That is what "data the app already stores or serves" means in executable form.
83
+ _ASSISTANT_BUILT_INS = {
84
+ "customer_data": {"label": "Odoo customers", "cache_prefix": "pool"},
85
+ "product_data": {"label": "Odoo products", "cache_prefix": "product_pool"},
86
+ }
87
+
88
+
89
+ def _assistant_refuse(status, code, message):
90
+ """Raise the public error shape without exposing a source implementation detail."""
91
+ raise err(status, code, message)
92
+
93
+
94
+ def _assistant_tenant_allows(session: Session, database: str) -> bool:
95
+ """Read the served tenant catalogue without reusing the legacy account gate."""
96
+ tenant = getattr(getattr(session, "runtime", None), "tenant", None)
97
+ config = getattr(tenant, "config", None) or {}
98
+ modules = config.get("modules", "all") if isinstance(config, dict) else "all"
99
+ return modules == "all" or str(database) in {str(key) for key in (modules or ())}
100
+
101
+
102
+ def _assistant_preflight_request(session: Session, database: str, canonical_fields,
103
+ requested_fields, filters, *, require_explicit_grant: bool):
104
+ """Refuse an unresolved grant, field or filter before a stored record is touched."""
105
+ import core.perm_scope as perm_scope
106
+
107
+ grant = perm_scope.assistant_entry(session.user, database)
108
+ if require_explicit_grant and grant is None:
109
+ _assistant_refuse(403, "assistant_forbidden",
110
+ "your current permissions do not allow this database")
111
+ visible = (perm_scope.assistant_visible_fields(canonical_fields, session.user, database)
112
+ if grant is not None else list(canonical_fields or ()))
113
+ try:
114
+ perm_scope.validate_assistant_filter((grant or {}).get("filter"), canonical_fields)
115
+ _assistant_requested_fields(requested_fields, visible)
116
+ perm_scope.validate_assistant_filter(filters, visible)
117
+ except ValueError as exc:
118
+ _assistant_refuse(400, "assistant_bad_filter", str(exc))
119
+ return grant, visible
120
+
121
+
122
+ def _assistant_cached_builtin(session: Session, database: str, requested_fields=None, filters=None):
123
+ """The exact in-process snapshot for one built-in database, or a loud cache-miss refusal.
124
+
125
+ This intentionally reads `runtime.pool_cache` directly. Calling either route's `_pool_for`
126
+ would make a first Assistant request call Odoo; calling a cache helper would make a stale entry
127
+ refresh. Neither is data already stored/served by this process.
128
+ """
129
+ import core.perm_scope as perm_scope
130
+
131
+ # Unlike an interactive grid, the Assistant has no legacy-grant compatibility path: the
132
+ # caller needs a declared, migrated database grant. `may_access` retains the old wall for
133
+ # the grid rollout, so it is necessary but not sufficient at this boundary.
134
+ declared_access = perm_scope.assistant_entry(session.user, database) is not None
135
+ if not _assistant_tenant_allows(session, database) or not declared_access:
136
+ _assistant_refuse(403, "assistant_forbidden",
137
+ "your current permissions do not allow this database")
138
+
139
+ # These are static field contracts, not the route assemblies: route assemblies can add
140
+ # per-user workspace state and can call a source builder. The Assistant begins with the
141
+ # already-stored base rows only; resolve their grant/field/filter contract *before* the cache
142
+ # read so a denied field or malformed operand cannot touch a stored record.
143
+ import aios_grid
144
+ fields = aios_grid.FIELDS if database == "customer_data" else aios_grid.product_fields()
145
+ _assistant_preflight_request(session, database, fields, requested_fields, filters,
146
+ require_explicit_grant=True)
147
+
148
+ team_id, agent = perm_scope.derive_pool_scope(session.user, database)
149
+ cache_key = (("pool", team_id, agent) if database == "customer_data"
150
+ else ("product_pool", team_id))
151
+ cache = getattr(session.runtime, "pool_cache", None)
152
+ entry = cache.get(cache_key) if isinstance(cache, dict) else None
153
+ if (not isinstance(entry, tuple) or len(entry) < 2
154
+ or not isinstance(entry[1], (list, tuple))
155
+ or any(not isinstance(row, dict) for row in entry[1])):
156
+ _assistant_refuse(409, "assistant_snapshot_unavailable",
157
+ "this database has no app-stored snapshot for your current scope")
158
+ return list(entry[1]), [dict(field) for field in fields if isinstance(field, dict)], entry[0]
159
+
160
+
161
+ def _assistant_user_table(session: Session, database: str, requested_fields=None, filters=None):
162
+ """Read a materialised user-table document after a rows-free database grant check."""
163
+ from core import user_tables
164
+
165
+ # A guessed `ut_*` key must never make `user_tables.get()` copy the tenant's rows before the
166
+ # table resolver refuses it. `all_defs` can fall back to a whole-document read for older
167
+ # stores, so use the runtime's projection primitive directly and fail closed if it is absent.
168
+ # The existing `may_open` remains THE resolver; it is lent this rows-dropped document.
169
+ try:
170
+ definitions = session.runtime.get_projection('user_tables', drop=('rows',))
171
+ except Exception:
172
+ _assistant_refuse(409, "assistant_snapshot_unavailable",
173
+ "this database has no readable definition snapshot")
174
+ if not isinstance(definitions, dict):
175
+ _assistant_refuse(409, "assistant_snapshot_unavailable",
176
+ "this database has no readable definition snapshot")
177
+ definition = definitions.get(database)
178
+ if not isinstance(definition, dict):
179
+ _assistant_refuse(404, "assistant_unknown_database", "that database is not available")
180
+ definition_runtime = user_tables.lend(session.runtime, user_tables=definitions)
181
+ if not user_tables.may_open(database, session.uname, session.admin, st=definition_runtime):
182
+ # Deliberately no label/owner in this response: knowing a key must not become an oracle
183
+ # for somebody else's database.
184
+ _assistant_refuse(403, "assistant_forbidden",
185
+ "your current permissions do not allow this database")
186
+
187
+ fields = [dict(field) for field in (definition.get("fields") or [])
188
+ if isinstance(field, dict) and field.get("key")]
189
+ _assistant_preflight_request(session, database, fields, requested_fields, filters,
190
+ require_explicit_grant=False)
191
+
192
+ # Only an admitted table may now cross the app-stored record seam. This call still has no
193
+ # upstream fallback: it is the tenant's materialised document already held by the runtime.
194
+ table = user_tables.get(database, st=session.runtime)
195
+ if not isinstance(table, dict):
196
+ _assistant_refuse(409, "assistant_snapshot_unavailable",
197
+ "this database changed before its stored snapshot could be read")
198
+ if not user_tables.materialises(database, st=session.runtime, defn=table):
199
+ # Read-through rows belong to the app's DuckDB mirror, but this fence does not own its
200
+ # tenant-checked query binding. Returning an empty table would be a false answer; E must
201
+ # consume the mirror through its own `runtime.mirror_cursor`-guarded seam.
202
+ _assistant_refuse(409, "assistant_snapshot_unavailable",
203
+ "this database is mirror-served and needs its materialised reader")
204
+
205
+ # A table's field contract was authorized from the rows-free definition. If it changed while
206
+ # the materialised document was fetched, fail rather than projecting a schema that was never
207
+ # checked.
208
+ actual_fields = [dict(field) for field in (table.get("fields") or [])
209
+ if isinstance(field, dict) and field.get("key")]
210
+ if actual_fields != fields:
211
+ _assistant_refuse(409, "assistant_snapshot_unavailable",
212
+ "this database changed before its stored snapshot could be read")
213
+ raw_rows = table.get("rows") or {}
214
+ if not isinstance(raw_rows, dict):
215
+ _assistant_refuse(409, "assistant_snapshot_unavailable",
216
+ "this database does not have a readable stored snapshot")
217
+ keys = {field["key"] for field in fields}
218
+ rows = []
219
+ for record_id, raw in raw_rows.items():
220
+ if not str(record_id).isdigit() or not isinstance(raw, dict):
221
+ continue
222
+ # Matches `routes_tables.scoped_pool`: user-table permission is table-level, and only
223
+ # declared fields cross this boundary. `pid` is the opaque record identity required to
224
+ # bind a returned record to a single-source Query artefact; it is not a selectable field.
225
+ rows.append({"pid": int(record_id),
226
+ **{key: raw.get(key) for key in keys if key in raw}})
227
+ rows.sort(key=lambda row: row["pid"])
228
+ return rows, fields, (table.get("updated") or table.get("created")), table.get("label") or database
229
+
230
+
231
+ def _assistant_requested_fields(fields, visible_fields):
232
+ """Resolve an explicit field selection without turning an unknown key into an omission."""
233
+ by_key = {str(field.get("key")): field for field in visible_fields
234
+ if isinstance(field, dict) and field.get("key")}
235
+ if fields is None:
236
+ return list(by_key)
237
+ if not isinstance(fields, (list, tuple)):
238
+ _assistant_refuse(400, "assistant_bad_fields", "fields must be a list of field keys")
239
+ out = []
240
+ for raw in fields:
241
+ key = raw.strip() if isinstance(raw, str) else ""
242
+ if not key or key not in by_key or key in out:
243
+ _assistant_refuse(400, "assistant_unknown_field",
244
+ "every requested field must be visible in this database")
245
+ out.append(key)
246
+ return out
247
+
248
+
249
+ def assistant_read_scope(session: Session, database, fields=None, filters=None):
250
+ """Return a detached, permissioned snapshot for one Assistant tool call.
251
+
252
+ Signature (the C8 contract for E):
253
+ assistant_read_scope(session, database, fields, filters) -> {
254
+ database, label, source_kind, source_version, retrieved_at,
255
+ permission_scope_applied, fields, filters, records
256
+ }
257
+
258
+ `database` is exactly one source. `fields` is an explicit list of visible field keys (or
259
+ ``None`` for every visible field). `filters` is ``None`` or the strict tree accepted by
260
+ `core.perm_scope.validate_assistant_filter`; it rejects unknown, hidden, inactive and
261
+ unsupported operands before a row is read. The returned dictionaries are deep copies, never
262
+ a live cache/store object, cursor, connector, route, provider, or credential.
263
+ """
264
+ import core.perm_scope as perm_scope
265
+ from harness import filter_eval
266
+
267
+ key = str(database or "").strip()
268
+ if key in _ASSISTANT_BUILT_INS:
269
+ rows, canonical_fields, source_version = _assistant_cached_builtin(session, key, fields, filters)
270
+ label = _ASSISTANT_BUILT_INS[key]["label"]
271
+ source_kind = "runtime_cache"
272
+ elif key.startswith("ut_"):
273
+ rows, canonical_fields, source_version, label = _assistant_user_table(session, key, fields, filters)
274
+ source_kind = "materialised_user_table"
275
+ else:
276
+ _assistant_refuse(404, "assistant_unknown_database", "that database is not available")
277
+
278
+ # Permanent permissions run against the full source contract. A permission filter may name
279
+ # a field that is intentionally hidden from the returned schema; resolving it after the field
280
+ # projection would make that filter unanswerable and (depending on the evaluator) could widen.
281
+ # Validate even that grant before evaluating it: `permits` would deny an unknown operand row
282
+ # by row, but the Assistant contract requires an explicit refusal for unresolved grants.
283
+ grant = perm_scope.assistant_entry(session.user, key)
284
+ if source_kind != "materialised_user_table" and grant is None:
285
+ # `_assistant_cached_builtin` already made this decision before touching its cache. Keep
286
+ # this guard adjacent to the field/row projection so a future source cannot forget it.
287
+ _assistant_refuse(403, "assistant_forbidden",
288
+ "your current permissions do not allow this database")
289
+ try:
290
+ perm_scope.validate_assistant_filter((grant or {}).get("filter"), canonical_fields)
291
+ except ValueError as exc:
292
+ _assistant_refuse(400, "assistant_bad_filter", str(exc))
293
+ permitted_rows = (perm_scope.assistant_apply_row_scope(rows, session.user, key, canonical_fields)
294
+ if grant is not None else list(rows or ()))
295
+ visible = (perm_scope.assistant_visible_fields(canonical_fields, session.user, key)
296
+ if grant is not None else list(canonical_fields))
297
+ selected_keys = _assistant_requested_fields(fields, visible)
298
+ try:
299
+ requested_filter = perm_scope.validate_assistant_filter(filters, visible)
300
+ except ValueError as exc:
301
+ _assistant_refuse(400, "assistant_bad_filter", str(exc))
302
+ if requested_filter is not None:
303
+ permitted_rows = [row for row in permitted_rows
304
+ if filter_eval.permits(requested_filter, row, visible)]
305
+
306
+ records = []
307
+ for row in permitted_rows:
308
+ if not isinstance(row, dict) or row.get("pid") is None:
309
+ continue
310
+ records.append({"pid": copy.deepcopy(row["pid"]),
311
+ **{field: copy.deepcopy(row.get(field)) for field in selected_keys}})
312
+
313
+ return {
314
+ "database": key,
315
+ "label": str(label),
316
+ "source_kind": source_kind,
317
+ "source_version": copy.deepcopy(source_version),
318
+ "retrieved_at": time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime()),
319
+ "permission_scope_applied": True,
320
+ "fields": [copy.deepcopy(field) for field in visible
321
+ if field.get("key") in selected_keys],
322
+ "filters": copy.deepcopy(requested_filter),
323
+ "records": records,
324
+ }
325
+
326
+
327
+ def assistant_permitted_sources(session: Session, databases):
328
+ """Which of `databases` the caller may still read β€” WITHOUT reading one record.
329
+
330
+ β›” THIS EXISTS BECAUSE `GET /query` LISTS ARTEFACTS, AND AN ARTEFACT IS NOT A READ.
331
+ `routes_query._public_state` has to drop every saved view whose source the caller has since
332
+ lost. The obvious way to ask that question is `assistant_read_scope(..., fields=[])` β€” and it
333
+ is the wrong one: for a `ut_*` source that call runs `user_tables.get()`, i.e. a WHOLE-DOCUMENT
334
+ read of the tenant store (measured at **28.6 MB / 703 ms warm on tenant #0**, and 99.9% of those
335
+ bytes are `rows` this question does not look at), then projects every row down to its `pid` and
336
+ throws the result away. Once per saved view, up to `MAX_ARTIFACTS = 200` of them, on a route
337
+ that BOTH the Assistant and Query call on mount. That is `/nav`'s old 1+N whole-document read
338
+ (D-175, an 11.0 s median) rebuilt inside a new route, and every fixture with two artefacts
339
+ passes it [[measure-the-real-call]].
340
+
341
+ So this asks the permission question and ONLY the permission question:
342
+ * the rows-free `user_tables` projection is read at most ONCE for the whole batch, and lent
343
+ to `may_open` exactly as `_assistant_user_table` lends it;
344
+ * a built-in resolves from the tenant catalogue and the declared grant, with no `pool_cache`
345
+ read β€” a cache MISS is a data-availability fact, not a permission one, and refusing to list
346
+ an artefact because its snapshot is cold would silently delete the user's history from the
347
+ rail every time the process restarted [[empty-answer-vs-unfinished-answer]].
348
+
349
+ ⚠ It is deliberately NOT a second permission wall. Every predicate here is the same call
350
+ `assistant_read_scope` makes for the same source; nothing may be admitted here that the read
351
+ boundary would refuse, which is what `verify_query.py` asserts in both directions.
352
+ """
353
+ return {key for key, row in assistant_source_status(session, databases).items()
354
+ if row["permitted"]}
355
+
356
+
357
+ def assistant_source_status(session: Session, databases=None):
358
+ """`{key: {permitted, answerable, reason}}` β€” WITHOUT reading one record.
359
+
360
+ ⭐⭐ `answerable` EXISTS BECAUSE `permitted` IS NOT THE QUESTION THE READER IS ASKING.
361
+ Measured on tenant #0, signed in, 2026-08-15: the Assistant offered **twelve** source chips
362
+ and could answer for **two**. The other ten are the `ut_odoo_*` grids, and
363
+ `_assistant_user_table` refuses every one of them β€” *"this database is mirror-served and needs
364
+ its materialised reader"* β€” because their rows live UNCAPPED in the DuckDB mirror and are read
365
+ through it (W30/R6 is exactly why). The refusal is correct, named and fail-closed. The defect
366
+ is that it arrives one spent prompt at a time: a picker that offers ten doors which cannot
367
+ open is the same shape as the rail that painted two rows differently from five.
368
+ ⚠ It is R6's SECOND SENTENCE, in the family that rule was written for β€” a limit that cannot be
369
+ removed must be REPORTED with its cause, never silently enforced.
370
+
371
+ β›” AND IT COSTS NOTHING, WHICH IS WHY IT IS HERE RATHER THAN IN THE REGISTER.
372
+ `user_tables.materialises()` reads the `readThrough` flag off the DEFINITION, and the
373
+ rows-free projection this function already reads carries it. So the batch that answers
374
+ "may they?" answers "can it?" in the same pass, with no extra store read.
375
+
376
+ ⚠ `databases=None` enumerates every source this tenant HAS (the two built-ins plus every
377
+ `ut_*` in the projection) rather than taking a caller-supplied list. A client list would make
378
+ the answer depend on what the caller thought to ask about, and the nav has already told them
379
+ what they hold.
380
+ """
381
+ import core.perm_scope as perm_scope
382
+ from core import user_tables
383
+
384
+ definitions = None
385
+ read_definitions = False
386
+
387
+ def _definitions():
388
+ nonlocal definitions, read_definitions
389
+ if not read_definitions:
390
+ read_definitions = True
391
+ try:
392
+ candidate = session.runtime.get_projection('user_tables', drop=('rows',))
393
+ except Exception:
394
+ candidate = None
395
+ definitions = candidate if isinstance(candidate, dict) else None
396
+ return definitions
397
+
398
+ if databases is None:
399
+ wanted = list(_ASSISTANT_BUILT_INS) + [
400
+ key for key in sorted((_definitions() or {}))
401
+ if isinstance(key, str) and key.startswith("ut_")
402
+ ]
403
+ else:
404
+ wanted = [str(key or "").strip() for key in (databases or ())]
405
+ wanted = list(dict.fromkeys(key for key in wanted if key))
406
+
407
+ out = {}
408
+ for key in wanted:
409
+ # `visible` = "the caller can open this database elsewhere in the product". D-276 uses it
410
+ # to decide whether an unusable source is REPORTED or stays absent; default False, so a
411
+ # branch that never sets it keeps today's silence rather than inheriting a disclosure.
412
+ row = {"permitted": False, "answerable": False, "reason": "", "visible": False}
413
+ out[key] = row
414
+ if key in _ASSISTANT_BUILT_INS:
415
+ # ⭐⭐ D-276 (2026-08-15) β€” AN OMISSION MUST STATE ITS CAUSE, and this branch used to
416
+ # `continue` in silence. MEASURED on tenant #0, both `admin` and `leadership`:
417
+ # `GET /query` listed nine sources and `product_data` was not among them, while
418
+ # `POST /query/chat {database:"product_data"}` answered 403 `assistant_forbidden` β€”
419
+ # for a database the SAME account opens from the nav, holds saved views on, and can
420
+ # see in the Database flyout. The two mirror-served grids are also un-askable and they
421
+ # ARE listed, with their cause in the chip; this one said nothing at all.
422
+ # β›” THE ASYMMETRY IS THE DEFECT, NOT THE REFUSAL. R6's second sentence: a limit that
423
+ # cannot be removed must be REPORTED with its cause, never silently enforced.
424
+ # ⚠ `visible` IS WHAT KEEPS THIS FROM BEING A DISCLOSURE. A source the caller cannot
425
+ # open ANYWHERE stays absent β€” naming it would tell them a database exists that they
426
+ # were never allowed to know about. `visible` is true only when the ordinary product
427
+ # wall already lets them open it, i.e. when the silence was incoherent rather than
428
+ # protective. `permitted` is untouched: it gates which stored artefacts survive, and
429
+ # widening it here would resurrect views the read boundary refuses.
430
+ if not _assistant_tenant_allows(session, key):
431
+ row["visible"] = bool(perm_scope.may_access(session.user, key))
432
+ row["reason"] = ("this workspace's catalogue does not list it as an assistant "
433
+ "source β€” an administrator can add it")
434
+ continue
435
+ if perm_scope.assistant_entry(session.user, key) is None:
436
+ row["visible"] = bool(perm_scope.may_access(session.user, key))
437
+ # β›”β›” THIS SENTENCE DELIBERATELY PROMISES NO REMEDY, AND THAT IS A CORRECTION.
438
+ # Its first version said *"an administrator can re-save them"*. MEASURED on the
439
+ # deployed build the same hour: re-saving `leadership`'s permissions through
440
+ # `PUT /admin/users/{u}/perms` returned 200 with the block byte-identical and
441
+ # `perms_v` unchanged at 1 β€” and the source stayed refused. So the advice was
442
+ # wrong, and a refusal that hands the reader a fix which does not work is worse
443
+ # than one that admits it does not know: they spend the trip and land back here.
444
+ # ⚠ The cause is still open (D-276). `assistant_entry` reads only `entry`,
445
+ # `is_migrated` and `may_access`, and `product_data` carries the SAME grant as
446
+ # `customer_data` under one record β€” so the divergence is not visible in the perms
447
+ # document and guessing at it once already produced a wrong answer.
448
+ row["reason"] = ("the assistant cannot resolve a data grant for it yet β€” this is a "
449
+ "known gap (D-276), not something you have set wrongly")
450
+ continue
451
+ row["permitted"] = True
452
+ # ⚠ A CACHE MISS IS A DATA-AVAILABILITY FACT, NOT A PERMISSION ONE, and it must not
453
+ # drop the source from the list β€” an artefact already built on it stays listed. It is
454
+ # reported as not-yet-answerable, with the action that fixes it. MEASURED: the pool is
455
+ # filled by a grid REQUEST, not by `AIOS_PREWARM` β€” a fresh process refuses until
456
+ # somebody opens the database once.
457
+ team_id, agent = perm_scope.derive_pool_scope(session.user, key)
458
+ cache_key = (("pool", team_id, agent) if key == "customer_data"
459
+ else ("product_pool", team_id))
460
+ cache = getattr(session.runtime, "pool_cache", None)
461
+ entry = cache.get(cache_key) if isinstance(cache, dict) else None
462
+ if isinstance(entry, tuple) and len(entry) >= 2 and isinstance(entry[1], (list, tuple)):
463
+ row["answerable"] = True
464
+ else:
465
+ row["reason"] = "no snapshot loaded yet β€” open this database once, then ask"
466
+ continue
467
+ if not key.startswith("ut_"):
468
+ continue
469
+ defs = _definitions()
470
+ definition = (defs or {}).get(key)
471
+ if not isinstance(definition, dict):
472
+ continue
473
+ lent = user_tables.lend(session.runtime, user_tables=defs)
474
+ if not user_tables.may_open(key, session.uname, session.admin, st=lent):
475
+ continue
476
+ row["permitted"] = True
477
+ if user_tables.materialises(key, st=lent, defn=definition):
478
+ row["answerable"] = True
479
+ else:
480
+ row["reason"] = "served live from its connector mirror β€” the assistant cannot read it yet"
481
+ return out
482
+
483
+
484
  def _user_for(claims):
485
  """The account named by a verified cookie, or None β€” with the epoch check that makes a
486
  stateless cookie revocable.
 
523
  # (login validated the slug's existence, never the membership) β€” with
524
  # AIOS_ENABLE_QA_TENANT=1 that was a working cross-tenant session. A signature proves
525
  # the cookie is ours; THIS proves the account belongs where the cookie says it does.
526
+ # ⚠ WAVE 33 (W33-T37) β€” the default was DELETED, not changed. It read
527
+ # `pub.get("tenant", "royal-imports")` and could never fire: `users._public` normalises a
528
+ # missing tenant to `royal-imports` before this line ever sees it. A dead default that
529
+ # names a real tenant reads like a live fallback, and the cross-tenant audit spent a row
530
+ # proving it was not one. Bare `.get` is also the safer shape if that ever stops being
531
+ # true: `None != claim_tenant` denies, where the old default would have ADMITTED a
532
+ # tenant-less record into tenant #0's session.
533
+ if claim_tenant != pub.get("tenant"):
534
  return None
535
  return pub
536
  if (uname == "admin" and os.environ.get("APP_PASSWORD", "")
api/main.py CHANGED
@@ -79,7 +79,18 @@ import routes_odoo_tables # noqa: E402 (wave 27 item 17 β€” Odoo relational; S
79
  import routes_connected_tables # noqa: E402 (wave 31 T49/C4 β€” the source-neutral grid door; D's)
80
  import routes_web_agent # noqa: E402 (wave 31 R10/C5 β€” the web-browsing agent; E's router, A's line)
81
  import routes_query # noqa: E402 (wave 32 R1/C5 β€” the Query module; E's router, A's line)
 
 
 
 
 
 
82
  from core import grid_events # noqa: E402
 
 
 
 
 
83
  from deps import Session, module_gate # noqa: E402
84
 
85
  _WEB_DIST = _HERE.parents[1] / "web" / "dist"
@@ -164,7 +175,49 @@ def _store_unavailable(request: Request, exc: grid_events.StoreUnavailable):
164
  """
165
  return JSONResponse(
166
  {"error": {"code": "store_unavailable",
167
- "message": "the tenant store is unavailable β€” no change was saved"}},
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
168
  status_code=503)
169
 
170
 
@@ -269,6 +322,60 @@ app.include_router(routes_web_agent.router) # R10 / C5 β€” E's router, A's lin
269
  # ⚠ `verify_api` asserts `/api/v1/query` in `app.openapi()["paths"]` β€” NEVER `{r.path for r in
270
  # app.routes}`, which finds nothing in this app because FastAPI wraps included routers (W31).
271
  app.include_router(routes_query.router) # R1 / C5 β€” E's router, A's line
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
272
 
273
 
274
  # --- DEPRECATED ALIASES (removed when S2's shell flips; kept so the current bundle keeps working)
@@ -302,18 +409,18 @@ def _assert_contract_floor():
302
  if [f["key"] for f in base] != [f["key"] for f in FIELDS]:
303
  raise RuntimeError(
304
  "AIOS web API: the canonical field contract and aios_grid.fields_from_workspace({}) "
305
- "disagree on the base field set β€” embed and standalone would serve different schemas. "
306
  "Run aios-web/verify_fields_contract.py.")
307
 
308
 
309
  def _startup_notes():
310
  """Say the two things an operator must know, once, at import β€” never in a response body."""
311
  if aios_session.EPHEMERAL_SECRET:
312
- print("[aios-api] AIOS_SESSION_SECRET is not set β€” signing with a random per-process "
313
  "key. Sessions will not survive a restart and will not work across workers. "
314
  "Set it in production.")
315
  if os.environ.get("AIOS_INSECURE_SSL") == "1":
316
- print("[aios-api] AIOS_INSECURE_SSL=1 β€” TLS verification is DISABLED for outbound "
317
  "requests. Local development only; never in a deployed environment.")
318
 
319
 
@@ -431,6 +538,12 @@ def _seed_and_sync_store():
431
  # is D-107's own hypothesis 3: a partial population trips `MAX_SHRINK` and the rebuild
432
  # refuses β€” correctly, but for a reason that reads like a data loss scare.
433
  # ⚠ Same thread on purpose: it is already a daemon and nothing serves requests behind it.
 
 
 
 
 
 
434
  _pull_meta("boot")
435
  _rebuild_odoo_relational("boot")
436
  # ⭐⭐ W32-T07 β€” owner items 13 and 15, DELIVERED. See `_sweep_automation_schemas`.
@@ -442,6 +555,35 @@ def _seed_and_sync_store():
442
  print(f"[aios-api] datastore seed/sync skipped: {e}")
443
 
444
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
445
  def _pull_meta(why):
446
  """Pull Meta Ads into THIS container's mirror, before the relational rebuild reads it.
447
 
@@ -546,18 +688,18 @@ def _sweep_automation_schemas(why):
546
  import automation_engine as _eng
547
  from harness import runtime as _runtime
548
  except Exception as e: # noqa: BLE001
549
- print(f"[aios-api] schema sweep ({why}) SKIPPED β€” import failed: {e}")
550
  return
551
  try:
552
  schemas = _eng.platform_schemas()
553
  except Exception as e: # noqa: BLE001
554
- print(f"[aios-api] schema sweep ({why}) SKIPPED β€” no declarations: {e}")
555
  return
556
  tenants = []
557
  try:
558
  tenants = _runtime.known_tenants()
559
  except Exception as e: # noqa: BLE001
560
- print(f"[aios-api] schema sweep ({why}) SKIPPED β€” tenant list unreadable: {e}")
561
  return
562
  for slug in tenants:
563
  try:
@@ -599,7 +741,7 @@ def _sweep_automation_schemas(why):
599
  # which is how the previous behaviour survived review in the first place.
600
  print(f"[aios-api] schema sweep ({why}) {slug}: {len(skipped)} child table(s) not "
601
  f"present in this tenant, so nothing was created for them "
602
- f"({', '.join(sorted(skipped))}) β€” they are spawned by an automation that "
603
  f"needs them, never by this sweep")
604
  # ⭐ THE RETRACTION (D's `retract_foreign_presets`, D-152), AFTER the children loop.
605
  # β›” THE SWEEP ABOVE MAKES COLUMNS **ARRIVE** AND CANNOT MAKE STALE ONES **LEAVE**, and
@@ -714,11 +856,31 @@ def _store_resync_loop():
714
  # next cycle rather than at the next container restart, which is what makes it a setting.
715
  # ⚠ FAIL-SAFE TO 1800 β€” an unreadable config must not become a tight loop. The floor is
716
  # enforced in `sync_seconds` as well as at the write door, for the same reason.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
717
  _every = 1800
 
718
  try:
719
  import odoo_relational as _rel_cad
720
  from harness import runtime as _rt_cad
 
 
 
 
721
  _secs = _rel_cad.sync_seconds(_rt_cad.get_runtime("royal-imports"))
 
722
  _every = 1800 if _secs is None else max(int(_secs), 60)
723
  except Exception: # noqa: BLE001
724
  pass
@@ -731,12 +893,19 @@ def _store_resync_loop():
731
  _odoo._tlocal.client = _odoo.OdooClient()
732
  except Exception:
733
  pass
734
- _ds.sync_all(log=lambda *a, **k: None)
735
- # Wave 21 β€” every 4th pass (~2h): purge hard-deleted rows the cursor sync cannot
736
- # see (the boot pass's comment has the measured case). Cheap id-sweep per entity;
737
- # without it deleted Odoo lines inflate every sum on the mirror FOREVER.
738
- if passes % 4 == 0:
739
- _ds.reconcile_deletes(log=lambda *a, **k: None)
 
 
 
 
 
 
 
740
  except Exception as e: # noqa: BLE001
741
  print(f"[aios-api] store resync failed: {e}")
742
  # ⭐ WAVE 27 item 17 (E's ASK ->A, resolved): the Odoo RELATIONAL tables are rebuilt
 
79
  import routes_connected_tables # noqa: E402 (wave 31 T49/C4 β€” the source-neutral grid door; D's)
80
  import routes_web_agent # noqa: E402 (wave 31 R10/C5 β€” the web-browsing agent; E's router, A's line)
81
  import routes_query # noqa: E402 (wave 32 R1/C5 β€” the Query module; E's router, A's line)
82
+ import routes_publish # noqa: E402 (wave 33 R5/C2 β€” the publish door; C's router, A's line)
83
+ import routes_brand # noqa: E402 (wave 33 R4/C2/C6 β€” connector brand marks; G's router, A's line)
84
+ import routes_slack # noqa: E402 (wave 33 R4/C2 β€” Manage agent + the Slack door; D's router, A's line)
85
+ import routes_starred # noqa: E402 (wave 35 R4/C2/C3/C5 β€” the star; E's router, E's line)
86
+ import routes_usage # noqa: E402 (wave 35 R9/C7 β€” the AI usage meter; E's router, E's line)
87
+ import routes_feedback # noqa: E402 (wave 35 R8/C6 β€” feedback to the operator plane; E's router)
88
  from core import grid_events # noqa: E402
89
+ # D-315 / D-305 (2026-08-18) β€” the two store REFUSALS get their own app-level handlers below.
90
+ # ⚠ Neither is a `StoreUnavailable` subclass, on purpose: routes that degrade a store outage into a
91
+ # session fallback must NOT degrade a refusal, or the user is told a change was saved that was not.
92
+ from core import data_binding # noqa: E402
93
+ from core import store # noqa: E402
94
  from deps import Session, module_gate # noqa: E402
95
 
96
  _WEB_DIST = _HERE.parents[1] / "web" / "dist"
 
175
  """
176
  return JSONResponse(
177
  {"error": {"code": "store_unavailable",
178
+ "message": "the tenant store is unavailable. No change was saved"}},
179
+ status_code=503)
180
+
181
+
182
+ @app.exception_handler(data_binding.StoreWriteRefused)
183
+ def _store_write_refused(request: Request, exc: data_binding.StoreWriteRefused):
184
+ """D-315 β€” this deployment may not write that store. 503, with the REASON.
185
+
186
+ β›” IT ECHOES THE EXCEPTION'S OWN MESSAGE, WHERE ITS NEIGHBOUR ABOVE USES A FIXED SENTENCE, and
187
+ the difference is the point. "The store is unavailable" is the whole truth about an outage; a
188
+ refusal has a cause the operator cannot otherwise discover, because per D-160 the Space
189
+ environment cannot be read from outside. A refusal that said only "unavailable" would send
190
+ somebody hunting for a network fault that does not exist ([[report-the-cause-before-you-fix-it]]).
191
+ The message names the deployment, the store and the fix, and carries no customer data.
192
+
193
+ β›” AND IT IS ITS OWN HANDLER RATHER THAN A SUBCLASS OF `StoreUnavailable`. Several routes carry
194
+ `except StoreUnavailable:` blocks that DEGRADE to a session-scoped fallback workspace β€” correct
195
+ for an outage, catastrophic here, because the user would be told the change was saved. A
196
+ distinct type falls through every one of them to this handler.
197
+ """
198
+ return JSONResponse(
199
+ {"error": {"code": "store_write_refused",
200
+ "message": f"this deployment may not write the tenant store. No change was "
201
+ f"saved. {exc}"}},
202
+ status_code=503)
203
+
204
+
205
+ @app.exception_handler(store.StoreConflict)
206
+ def _store_conflict(request: Request, exc: store.StoreConflict):
207
+ """D-305 β€” the document moved under this write more times than it could be rebased. 503.
208
+
209
+ ⚠ 503 RATHER THAN 409, DELIBERATELY. A 409 invites the client to resolve a conflict, and there
210
+ is nothing here for it to resolve: the store already re-read and re-applied this change up to
211
+ `_MAX_REBASE` times before giving up. What the caller needs to know is that the write did not
212
+ land and nothing was damaged, which is the same contract as the outage above and the same
213
+ retry-later shape. β›” The one answer this must never be is 200: reaching this handler means the
214
+ only way to have "succeeded" was to overwrite somebody else's work.
215
+ """
216
+ return JSONResponse(
217
+ {"error": {"code": "store_conflict",
218
+ "message": f"somebody else changed this database while your change was being "
219
+ f"saved, and it could not be merged. Nothing was saved or "
220
+ f"overwritten. Try again. {exc}"}},
221
  status_code=503)
222
 
223
 
 
322
  # ⚠ `verify_api` asserts `/api/v1/query` in `app.openapi()["paths"]` β€” NEVER `{r.path for r in
323
  # app.routes}`, which finds nothing in this app because FastAPI wraps included routers (W31).
324
  app.include_router(routes_query.router) # R1 / C5 β€” E's router, A's line
325
+ # ⭐⭐ WAVE 33 (C2) β€” THE SIXTH CONSECUTIVE WAVE IN WHICH THIS BLOCK IS THE THING THE PROTOCOL LOSES.
326
+ # Contract C2 is written for exactly that: a lane creating a `routes_*.py` posts an ASK, and the
327
+ # INTEGRATOR adds the line in the SAME wave. Both of these arrived that way (C's `ASK C-1`, G's
328
+ # `ASK G-1`), and `verify_api::section_w23_mounts` asserts each path in `app.openapi()["paths"]`
329
+ # with an NC that comments a mount out and goes RED.
330
+ # ⚠ Same placement rule as the line above β€” ABOVE the `app.mount("/", _AppStatic(...), html=True)`
331
+ # at the end of the file, never after it.
332
+ # ⭐⭐ THE PUBLISH DOOR IS MOUNTED AGAIN (2026-08-15). It was withheld for the wave-33 deploy
333
+ # because QA reproduced three HIGH defects that are armed ONLY by mounting it. All three are fixed
334
+ # in `routes_publish.py`, each at its cause rather than at its symptom:
335
+ # 1. W33-T68 β€” `form` LEFT `PUBLISHABLE_MODES`. A form view's rows ARE the submissions people
336
+ # sent it, so publishing one served other people's answers to anyone holding the link. A form
337
+ # still has its own public door (`#/form/<token>`) which serves the BLANK form and never rows.
338
+ # 2. W33-T69 β€” `_visible_keys` returns `[]` when a view STORED a `visible` list and none of its
339
+ # keys survive, instead of falling back to the table default. The fallback still applies to a
340
+ # view that never stored one, which is what it was written for.
341
+ # 3. W33-T70 β€” three separate leaks closed: the rate limit no longer keys on the caller-supplied
342
+ # `x-forwarded-for` (it keys on the socket peer and counts FAILURES only, so a shared proxy
343
+ # peer cannot become one global bucket); every failing path now spends the same PBKDF2 the
344
+ # success path spends, closing the 206x timing gap; and an unknown token on the sibling GET
345
+ # answers the LOCKED shape rather than a 403, so that route stops sorting real tokens from
346
+ # fake ones for free.
347
+ # ⚠ `verify_api::section_w23_mounts` asserts this path in `app.openapi()["paths"]`, so its four
348
+ # EXPECTED reds should now go GREEN. A red here after this line means the mount broke, not the gate.
349
+ app.include_router(routes_publish.router) # R5 / C2 β€” C's router, A's line (the publish door)
350
+ # β›” NOT BEHIND `module_gate("product_data")`, WHICH IS THE WHOLE REASON IT IS A SECOND ASSET DOOR.
351
+ # `routes_assets.py` gates EVERY one of its routes on that module, so a connector logo served from
352
+ # there would 403 for any account without the product-data grant β€” i.e. the Connectors directory
353
+ # would lose its logos for exactly the accounts most likely to be setting a connector up (C6).
354
+ app.include_router(routes_brand.router) # R4 / C2 / C6 β€” G's router, A's line (brand marks)
355
+ # β›” TWO OF ITS SIX PATHS ARE UNAUTHENTICATED BY DESIGN (`/slack/events`, `/slack/interact`) and
356
+ # that is the DOOR, not an omission β€” Slack posts to them with no session and could not carry one.
357
+ # They are built on `routes_forms`' proven public shape: signing-secret verification, a sliding-window
358
+ # rate limit, a body cap and ONE non-oracle 403, so a bad signature cannot distinguish "no such
359
+ # workspace" from "not yours". ⚠ There is no auth middleware and no exempt-path allow-list in this
360
+ # file to register them in: "public" here IS the absence of `Depends(require_session)`, which is why
361
+ # `verify_api` asserts the absence rather than an entry in a list that does not exist.
362
+ app.include_router(routes_slack.router) # R4 / C2 β€” D's router, A's line (Manage agent + Slack)
363
+ # ⭐⭐ WAVE 35 (R4/R5/R7, contracts C2/C3/C5) β€” THE STAR. Mounted in the SAME change that created
364
+ # `routes_starred.py`, which is the seventh consecutive wave in which this block is the artefact the
365
+ # protocol nearly loses β€” and the first in which the router's own lane also owns `main.py`, so there
366
+ # is no ask/mount pair to drop. `W35-T46` asserts every one of this wave's paths in
367
+ # `app.openapi()["paths"]` and CALLS one route from each.
368
+ # ⚠ Same placement rule as every line above: ABOVE `app.mount("/", _AppStatic(...), html=True)`, or
369
+ # the router answers 404 on GET and 405 on POST while every one of its own tests passes.
370
+ app.include_router(routes_starred.router) # R4 / C2 β€” the star, counts, and record stars
371
+ # β›” R9 SAYS THIS ROUTE REPORTS AND NEVER ENFORCES, so mounting it cannot cut anybody off β€” there is
372
+ # no ceiling anywhere behind it (`usage_ledger` has no refusal in it). The one AI limit the product
373
+ # enforces is per COLUMN and is unchanged (`ai_enrich.ceiling_report`).
374
+ app.include_router(routes_usage.router) # R9 / C7 β€” GET /usage, the one AI meter
375
+ # β›” ITS TWO DOORS CARRY DIFFERENT WALLS ON PURPOSE (R8, and D-221 is the booked precedent): the
376
+ # POST is any authenticated session's own act, the GET is `is_platform_admin` only. Mounting it does
377
+ # not widen anything a tenant admin can reach β€” `verify_api` proves that by having one try.
378
+ app.include_router(routes_feedback.router) # R8 / C6 β€” feedback to the operator plane
379
 
380
 
381
  # --- DEPRECATED ALIASES (removed when S2's shell flips; kept so the current bundle keeps working)
 
409
  if [f["key"] for f in base] != [f["key"] for f in FIELDS]:
410
  raise RuntimeError(
411
  "AIOS web API: the canonical field contract and aios_grid.fields_from_workspace({}) "
412
+ "disagree on the base field set. Embed and standalone would serve different schemas. "
413
  "Run aios-web/verify_fields_contract.py.")
414
 
415
 
416
  def _startup_notes():
417
  """Say the two things an operator must know, once, at import β€” never in a response body."""
418
  if aios_session.EPHEMERAL_SECRET:
419
+ print("[aios-api] AIOS_SESSION_SECRET is not set. Signing with a random per-process "
420
  "key. Sessions will not survive a restart and will not work across workers. "
421
  "Set it in production.")
422
  if os.environ.get("AIOS_INSECURE_SSL") == "1":
423
+ print("[aios-api] AIOS_INSECURE_SSL=1. TLS verification is DISABLED for outbound "
424
  "requests. Local development only; never in a deployed environment.")
425
 
426
 
 
538
  # is D-107's own hypothesis 3: a partial population trips `MAX_SHRINK` and the rebuild
539
  # refuses β€” correctly, but for a reason that reads like a data loss scare.
540
  # ⚠ Same thread on purpose: it is already a daemon and nothing serves requests behind it.
541
+ # ⭐⭐ W35-T45 / R11 β€” TENANT #0'S ODOO CREDENTIAL MOVES ONTO ITS KEYCHAIN, IN THE CONTAINER.
542
+ # ⚠ BEFORE the relational rebuild, deliberately: the rebuild resolves its connector through
543
+ # `rt.odoo_source()`, so running the migration first means the very next read already goes
544
+ # through the keychain branch and a broken migration is visible in THIS boot's log rather
545
+ # than in tomorrow's. It is idempotent, so every later boot is one `list_entries` read.
546
+ _migrate_env_odoo("boot")
547
  _pull_meta("boot")
548
  _rebuild_odoo_relational("boot")
549
  # ⭐⭐ W32-T07 β€” owner items 13 and 15, DELIVERED. See `_sweep_automation_schemas`.
 
555
  print(f"[aios-api] datastore seed/sync skipped: {e}")
556
 
557
 
558
+ def _migrate_env_odoo(why):
559
+ """⭐⭐ W35-T45 / R11 β€” move tenant #0's environment Odoo credential onto its keychain.
560
+
561
+ β›”β›” IN THE CONTAINER, WHICH IS THE WHOLE REASON THIS IS A BOOT LINE AND NOT A SCRIPT. D-195,
562
+ measured three times: a developer's CLI write to the tenant store is reverted by the running
563
+ Space within a minute (download-modify-upload, last-write-wins) β€” and the write REPORTS SUCCESS
564
+ every time, then a fresh read confirms it, and it is gone by the next poll. A CLI migration would
565
+ be a dry run that lies, and what it would lie about here is a credential.
566
+
567
+ ⚠ IDEMPOTENT AND SCOPED TO TENANT #0 by `routes_keychain.env_odoo_available`, so on every other
568
+ tenant and on every later boot this is one `list_entries` read and a line.
569
+ ⚠ FAIL-QUIET: this must never take a boot down. But it is never SILENT β€” a skip prints its reason,
570
+ because "already migrated", "no keychain key on this deployment" and "the env is incomplete" are
571
+ three different operator actions and a blank Keychain page cannot tell them apart.
572
+ """
573
+ try:
574
+ import routes_keychain as _kc_routes
575
+ from harness import runtime as _runtime
576
+ rep = _kc_routes.migrate_env_odoo(_runtime.get_runtime("royal-imports"))
577
+ if rep.get("done"):
578
+ print(f"[aios-api] odoo credential migrated onto the keychain ({why}): "
579
+ f"entry={rep['entry']} carried_pause={rep['carried_pause']}"
580
+ + (f" PROBLEM: {rep['why']}" if rep.get("why") else ""))
581
+ else:
582
+ print(f"[aios-api] odoo keychain migration skipped ({why}): {rep.get('why')}")
583
+ except Exception as e: # noqa: BLE001
584
+ print(f"[aios-api] odoo keychain migration FAILED ({why}): {type(e).__name__}: {e}")
585
+
586
+
587
  def _pull_meta(why):
588
  """Pull Meta Ads into THIS container's mirror, before the relational rebuild reads it.
589
 
 
688
  import automation_engine as _eng
689
  from harness import runtime as _runtime
690
  except Exception as e: # noqa: BLE001
691
+ print(f"[aios-api] schema sweep ({why}) SKIPPED. Import failed: {e}")
692
  return
693
  try:
694
  schemas = _eng.platform_schemas()
695
  except Exception as e: # noqa: BLE001
696
+ print(f"[aios-api] schema sweep ({why}) SKIPPED. No declarations: {e}")
697
  return
698
  tenants = []
699
  try:
700
  tenants = _runtime.known_tenants()
701
  except Exception as e: # noqa: BLE001
702
+ print(f"[aios-api] schema sweep ({why}) SKIPPED. Tenant list unreadable: {e}")
703
  return
704
  for slug in tenants:
705
  try:
 
741
  # which is how the previous behaviour survived review in the first place.
742
  print(f"[aios-api] schema sweep ({why}) {slug}: {len(skipped)} child table(s) not "
743
  f"present in this tenant, so nothing was created for them "
744
+ f"({', '.join(sorted(skipped))}). They are spawned by an automation that "
745
  f"needs them, never by this sweep")
746
  # ⭐ THE RETRACTION (D's `retract_foreign_presets`, D-152), AFTER the children loop.
747
  # β›” THE SWEEP ABOVE MAKES COLUMNS **ARRIVE** AND CANNOT MAKE STALE ONES **LEAVE**, and
 
856
  # next cycle rather than at the next container restart, which is what makes it a setting.
857
  # ⚠ FAIL-SAFE TO 1800 β€” an unreadable config must not become a tight loop. The floor is
858
  # enforced in `sync_seconds` as well as at the write door, for the same reason.
859
+ # β›”β›” WAVE 34 Β· W34-T46 / D-246 β€” THE PARAGRAPH ABOVE DESCRIBED A BRANCH THAT DID NOT
860
+ # EXIST, AND THAT IS WHY NOBODY LOOKED. It said `manual` "parks at the default cadence
861
+ # instead and simply DOES NO WORK". The first half was true; the second half was not
862
+ # implemented anywhere: `_secs is None` set the sleep to 1800 and then fell into an
863
+ # UNCONDITIONAL `sync_all()`. So a tenant who chose "manual" was synced every thirty
864
+ # minutes exactly like everyone else, with the setting stored, displayed, clamped and
865
+ # obeyed by nothing β€” the identical failure this block's own W32/R11 note is about, one
866
+ # layer further in. A comment asserting a fix is why the second reader stops reading.
867
+ # ⚠ FIXED, AND THE FIX IS NARROW ON PURPOSE. `manual` is an ODOO connector setting, so it
868
+ # skips the Odoo mirror sync and its reconcile and NOTHING ELSE: the Meta pull and the
869
+ # relational rebuild below still run, because a tenant asking not to have Odoo polled has
870
+ # not asked for the other connector to stop.
871
+ # ⚠ AND IT STILL SLEEPS AND STILL LOOPS. The config is re-read every pass, so switching
872
+ # back off `manual` takes effect on the next cycle rather than at the next restart.
873
  _every = 1800
874
+ _manual = False
875
  try:
876
  import odoo_relational as _rel_cad
877
  from harness import runtime as _rt_cad
878
+ # ⚠ D-245, KNOWN AND NOT FIXED HERE: the slug is hardcoded, so this reads TENANT #0's
879
+ # cadence and applies it to a loop that syncs the shared mirror. Correct while one
880
+ # tenant has Odoo; wrong the moment a second one does. Out of this ticket's scope and
881
+ # named rather than silently inherited.
882
  _secs = _rel_cad.sync_seconds(_rt_cad.get_runtime("royal-imports"))
883
+ _manual = _secs is None
884
  _every = 1800 if _secs is None else max(int(_secs), 60)
885
  except Exception: # noqa: BLE001
886
  pass
 
893
  _odoo._tlocal.client = _odoo.OdooClient()
894
  except Exception:
895
  pass
896
+ # ⚠ A PLAIN BRANCH, NOT AN EARLY EXIT AND NOT AN EXCEPTION. A `raise` here would be
897
+ # caught by this block's own `except` and printed as "store resync failed", turning a
898
+ # setting a person chose into a recurring error in the log; a `continue` would skip
899
+ # the Meta pull and the relational rebuild below, which `manual` says nothing about.
900
+ if not _manual:
901
+ _ds.sync_all(log=lambda *a, **k: None)
902
+ # Wave 21 β€” every 4th pass (~2h): purge hard-deleted rows the cursor sync cannot
903
+ # see (the boot pass's comment has the measured case). Cheap id-sweep per entity;
904
+ # without it deleted Odoo lines inflate every sum on the mirror FOREVER.
905
+ # ⚠ INSIDE the branch: a reconcile is a pass over the mirror this loop was just
906
+ # told not to advance.
907
+ if passes % 4 == 0:
908
+ _ds.reconcile_deletes(log=lambda *a, **k: None)
909
  except Exception as e: # noqa: BLE001
910
  print(f"[aios-api] store resync failed: {e}")
911
  # ⭐ WAVE 27 item 17 (E's ASK ->A, resolved): the Odoo RELATIONAL tables are rebuilt
api/odoo_relational.py CHANGED
@@ -117,6 +117,13 @@ def _ut():
117
  return user_tables
118
 
119
 
 
 
 
 
 
 
 
120
  def _iso_today():
121
  return _dt.date.today().strftime("%Y-%m-%d")
122
 
@@ -180,9 +187,14 @@ def agent_fields():
180
  "source": "overlay", "default": True},
181
  # ⭐ THE INVERSE HALF: the customers whose `agent_id` names this agent. MEASURED: 2,093
182
  # customers carry one and ALL 2,093 resolve to a row in this table (zero dangling).
183
- {"key": "customers", "label": "Customers", "type": "link", "source": "overlay",
184
- "default": True,
185
- "link": {"table": CUSTOMERS_KEY, "on": AGENT_JOIN_KEY, "from": AGENT_JOIN_KEY}},
 
 
 
 
 
186
  _refreshed_field(),
187
  )]
188
 
@@ -237,8 +249,34 @@ def vendor_fields():
237
  "default": False, "description": "The `res.partner` id. Also this row's id."},
238
  {"key": VENDOR_JOIN_KEY, "label": "Odoo vendor id", "type": "int", "source": "overlay",
239
  "default": False},
 
 
 
 
 
 
240
  {"key": "country", "label": "Country", "type": "text", "source": "overlay",
241
  "default": True},
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
242
  {"key": "bills", "label": "Bills", "type": "link", "source": "overlay", "default": True,
243
  "link": {"table": BILLS_KEY, "on": VENDOR_JOIN_KEY, "from": VENDOR_JOIN_KEY}},
244
  _refreshed_field(),
@@ -316,9 +354,21 @@ def invoice_fields():
316
  _scope_field(),
317
  # ⭐ THE RECIPROCAL HALF (owner item 2, 2026-08-09). DERIVED (`on` declared), exactly like
318
  # its twin, so the engine owns the cell and no human can edit a relation Odoo decided.
319
- {"key": "customer_link", "label": "Customer record", "type": "link", "source": "overlay",
320
- "default": False,
321
- "link": {"table": CUSTOMERS_KEY, "on": JOIN_KEY, "from": JOIN_KEY}},
 
 
 
 
 
 
 
 
 
 
 
 
322
  # ⭐ DEBT D-88, closed 2026-08-09. `invoice_origin` carries the ORDER NAME an invoice was
323
  # raised from, and the mirror did not sync it until this wave β€” so order->invoice was a
324
  # two-hop join through 963,783 `account_move_line` rows and wave 27 shipped no link at all.
@@ -358,9 +408,9 @@ def order_fields():
358
  {"key": "invoice_status", "label": "Invoice status", "type": "select", "source": "overlay",
359
  "default": True, "options": ["invoiced", "to invoice", "upselling", "no"]},
360
  _scope_field(),
361
- {"key": "customer_link", "label": "Customer record", "type": "link", "source": "overlay",
362
- "default": False,
363
- "link": {"table": CUSTOMERS_KEY, "on": JOIN_KEY, "from": JOIN_KEY}},
364
  # The reciprocal of the invoice's `order_link` (D-88): the invoices raised from THIS
365
  # order, matched on the order's own name.
366
  {"key": "invoices", "label": "Invoices", "type": "link", "source": "overlay",
@@ -1071,18 +1121,44 @@ def read_vendors(cur):
1071
  have = columns(cur, "res_partner")
1072
  if not have or not columns(cur, "account_move"):
1073
  return []
1074
- sql = (f"SELECT p.id, p.name, {_col(have, 'p.country_name', chr(39) + chr(39))} "
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1075
  "FROM res_partner p WHERE p.id IN "
1076
  f" (SELECT partner_id FROM account_move WHERE {_VENDOR_DOCS} "
1077
  " AND partner_id IS NOT NULL)")
1078
  out = []
1079
- for (pid, name, country) in cur.execute(sql).fetchall():
 
1080
  out.append({
1081
  "_id": str(pid),
1082
  "vendor": str(name or ""),
1083
  "odoo_id": int(pid),
1084
  VENDOR_JOIN_KEY: int(pid),
1085
  "country": str(country or ""),
 
 
 
 
 
 
 
 
 
 
1086
  })
1087
  return out
1088
 
@@ -1118,8 +1194,14 @@ class Refused(Exception):
1118
  #: point: adding an Odoo entity is a spec row plus a reader, not a fifth copy of the spawn code.
1119
  #: ⚠ ORDER MATTERS ONLY FOR THE REFUSAL MESSAGE; `plan` checks every cap before anything commits.
1120
  TABLES = (
1121
- ("customers", CUSTOMERS_KEY, "Odoo customers", customer_fields),
1122
- ("products", PRODUCTS_KEY, "Odoo products", product_fields),
 
 
 
 
 
 
1123
  ("invoices", INVOICES_KEY, "Odoo invoices", invoice_fields),
1124
  ("orders", ORDERS_KEY, "Odoo orders", order_fields),
1125
  # ⭐ WAVE 28 / R1. Measured populations: 19 / 192 / 6,538 / 393 β€” every one of them two orders
@@ -1139,6 +1221,66 @@ TABLES = (
1139
  ("gl_lines", GL_LINES_KEY, "Odoo GL lines", gl_line_fields),
1140
  )
1141
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1142
  #: The bucket -> reader map. β›” ITS ABSENCES ARE LOAD-BEARING: a bucket with no reader has no
1143
  #: python row builder ANYWHERE, which is what makes "never materialised" structural rather than a
1144
  #: policy `plan()` could forget. The two line grains are absent for that reason and no other.
@@ -1412,7 +1554,7 @@ def plan(cur, rt=None):
1412
  return built
1413
 
1414
 
1415
- def apply_plan(rt, built, username="automation", today=None):
1416
  """Create-or-merge every table in `built` and its rows. Idempotent by construction.
1417
 
1418
  Row ids ARE the Odoo ids, so a re-run updates in place and never appends a second copy of the
@@ -1425,6 +1567,9 @@ def apply_plan(rt, built, username="automation", today=None):
1425
  raise Refused("; ".join(built["problems"]))
1426
  stamp = today or _iso_today()
1427
  written = {}
 
 
 
1428
  plans = [(key, label, fields(), built[bucket])
1429
  for bucket, key, label, fields in TABLES if bucket in built]
1430
 
@@ -1442,11 +1587,29 @@ def apply_plan(rt, built, username="automation", today=None):
1442
  # create. Inside one updater, a raise aborts before anything is persisted.
1443
  def _apply_all(cur):
1444
  cur = cur if isinstance(cur, dict) else {}
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1445
  for key, label, fields, rows in plans:
1446
  written[key] = _ensure_table_inplace(cur, key, label, fields, rows, username, stamp)
1447
  return cur
1448
 
1449
  rt.update(_ut().STORE_KEY, _apply_all, flush="sync")
 
 
1450
  return written
1451
 
1452
 
@@ -1478,12 +1641,37 @@ def _ensure_table_inplace(cur, key, label, fields, rows, username, stamp):
1478
  for row in wanted.values():
1479
  row["refreshed"] = stamp
1480
  counts = {"added": 0, "updated": 0, "removed": 0, "rows": len(wanted)}
 
1481
  table = cur.get(key)
1482
  if table is None:
1483
  if len(cur) >= ut.MAX_TABLES:
1484
  # ⚠ `ut_ensure` returns silently at this cap; a silent no-op here would report a
1485
  # successful refresh over a table that does not exist.
1486
  raise Refused(f"{key}: tenant is at MAX_TABLES={ut.MAX_TABLES}; nothing created")
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1487
  table = cur[key] = {
1488
  "key": key, "label": label, "source": ut.AUTOMATION_SOURCE,
1489
  "createdBy": username, "created": stamp, "fields": [], "rows": {},
@@ -1499,8 +1687,33 @@ def _ensure_table_inplace(cur, key, label, fields, rows, username, stamp):
1499
  # an empty grid.
1500
  if key in READ_THROUGH_KEYS:
1501
  table["readThrough"] = True
 
 
 
 
 
 
 
 
1502
  have = {str(f.get("key")): f for f in (table.get("fields") or [])}
1503
  for field in fields:
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1504
  fkey = str(field.get("key"))
1505
  if fkey not in have:
1506
  table.setdefault("fields", []).append(dict(field))
@@ -1599,8 +1812,13 @@ def refresh(rt, tenant, username="automation", cur=None, today=None):
1599
  built.pop(bucket, None)
1600
  else:
1601
  skipped = []
1602
- written = apply_plan(rt, built, username=username, today=today)
 
 
 
 
1603
  return {"tables": written,
 
1604
  # R6's second sentence: a set that was deliberately not built SAYS SO, with the keys.
1605
  **({"skipped": skipped} if skipped else {}),
1606
  **{bucket: len(built[bucket]) for bucket, _k, _l, _f in TABLES if bucket in built}}
 
117
  return user_tables
118
 
119
 
120
+ def _registry():
121
+ """`core.registry`, imported the same lazy way `_ut` is β€” this module is imported by the
122
+ route layer before `platform/` is necessarily on the path."""
123
+ import core.registry as registry
124
+ return registry
125
+
126
+
127
  def _iso_today():
128
  return _dt.date.today().strftime("%Y-%m-%d")
129
 
 
187
  "source": "overlay", "default": True},
188
  # ⭐ THE INVERSE HALF: the customers whose `agent_id` names this agent. MEASURED: 2,093
189
  # customers carry one and ALL 2,093 resolve to a row in this table (zero dangling).
190
+ # β›” W33-T43 / AMENDMENT A2 β€” the `customers` REVERSE link DELETED. See `invoice_fields`
191
+ # for the ruling. ⚠ This one costs more than the other two and the difference is worth
192
+ # recording: those were a click-through to a record whose id stays on the row, while this
193
+ # was an agent's BOOK β€” the list of customers assigned to them. The id side survives
194
+ # (`agent_id` here, and `agent_id` on `customer_data`), so the relationship is intact in
195
+ # the data and only the rendered list is gone; the same question is answerable on the
196
+ # customer grid by filtering `agent_id`, and at analytical grain via the `agent` dim on
197
+ # `sales_lines` / `sales_orders`.
198
  _refreshed_field(),
199
  )]
200
 
 
249
  "default": False, "description": "The `res.partner` id. Also this row's id."},
250
  {"key": VENDOR_JOIN_KEY, "label": "Odoo vendor id", "type": "int", "source": "overlay",
251
  "default": False},
252
+ # ⭐⭐ W33-T48 / owner item 13 β€” the columns the census proved are there and were not shown.
253
+ # ⚠ `default` is deliberately split: the ones a vendor list is READ for (contact + tax id)
254
+ # arrive visible; the postal lines arrive HIDDEN, because four address columns turned on by
255
+ # default would push the useful ones off the first screen, and DESIGN.md's "never
256
+ # over-explain" applies to columns as much as to prose. Every one is one click away in the
257
+ # field picker, and a locked database still allows fields (item 4's vocabulary).
258
  {"key": "country", "label": "Country", "type": "text", "source": "overlay",
259
  "default": True},
260
+ {"key": "email", "label": "Email", "type": "text", "source": "overlay", "default": True},
261
+ {"key": "phone", "label": "Phone", "type": "text", "source": "overlay", "default": True},
262
+ {"key": "mobile", "label": "Mobile", "type": "text", "source": "overlay",
263
+ "default": False},
264
+ {"key": "vat", "label": "Tax ID", "type": "text", "source": "overlay", "default": True,
265
+ "description": "Odoo `vat` β€” the vendor's tax/VAT registration number."},
266
+ # ⚠ `vendor_ref`, not `ref`: `ref` is Odoo's own name for it, and the bills grid already
267
+ # uses `ref` for the VENDOR'S INVOICE NUMBER on a document. Two different facts, and a
268
+ # shared spelling across two linked grids is how a rollup ends up summing the wrong column.
269
+ {"key": "vendor_ref", "label": "Vendor reference", "type": "text", "source": "overlay",
270
+ "default": False,
271
+ "description": "Odoo `res.partner.ref` β€” our internal reference for this vendor."},
272
+ {"key": "website", "label": "Website", "type": "url", "source": "overlay",
273
+ "default": False},
274
+ {"key": "street", "label": "Street", "type": "text", "source": "overlay",
275
+ "default": False},
276
+ {"key": "street2", "label": "Street 2", "type": "text", "source": "overlay",
277
+ "default": False},
278
+ {"key": "city", "label": "City", "type": "text", "source": "overlay", "default": False},
279
+ {"key": "zip", "label": "ZIP", "type": "text", "source": "overlay", "default": False},
280
  {"key": "bills", "label": "Bills", "type": "link", "source": "overlay", "default": True,
281
  "link": {"table": BILLS_KEY, "on": VENDOR_JOIN_KEY, "from": VENDOR_JOIN_KEY}},
282
  _refreshed_field(),
 
354
  _scope_field(),
355
  # ⭐ THE RECIPROCAL HALF (owner item 2, 2026-08-09). DERIVED (`on` declared), exactly like
356
  # its twin, so the engine owns the cell and no human can edit a relation Odoo decided.
357
+ # β›”β›” W33-T43 / AMENDMENT A2 β€” `customer_link` DELETED, and the loss is stated here rather
358
+ # than left to be inferred from a green gate.
359
+ #
360
+ # It pointed at `ut_odoo_customers`, which W33-T44 retires (R2: one identity per subject).
361
+ # A2 ruled OPTION 2 β€” retire both twins, DROP the three link columns, keep every data
362
+ # column β€” so this grid loses the CLICK-THROUGH to a customer record and NOTHING else:
363
+ # `customer` (the name) and `partner_id` (the Odoo id) are plain columns on this same row,
364
+ # and `customer_data` now carries `partner_id` too, so both ends of the join still exist.
365
+ #
366
+ # β›” THE ALTERNATIVE WAS REFUSED IN WRITING, and the reason belongs beside the deletion:
367
+ # re-pointing this bag at `customer_data` needs `core/user_tables.py::_clean_link`'s `ut_`
368
+ # prefix test relaxed β€” which makes EVERY GATE GREEN while
369
+ # `automation_engine::compute_relation_cells` still resolves out of the `user_tables`
370
+ # document alone and returns `{}`. Empty cells, blank rollups, nothing red. A2 forbids
371
+ # touching that line this wave for exactly that reason.
372
  # ⭐ DEBT D-88, closed 2026-08-09. `invoice_origin` carries the ORDER NAME an invoice was
373
  # raised from, and the mirror did not sync it until this wave β€” so order->invoice was a
374
  # two-hop join through 963,783 `account_move_line` rows and wave 27 shipped no link at all.
 
408
  {"key": "invoice_status", "label": "Invoice status", "type": "select", "source": "overlay",
409
  "default": True, "options": ["invoiced", "to invoice", "upselling", "no"]},
410
  _scope_field(),
411
+ # β›” W33-T43 / AMENDMENT A2 β€” `customer_link` DELETED here too; see the note on the same
412
+ # column in `invoice_fields` above for the ruling and the refused alternative. This row
413
+ # keeps `customer` and `partner_id`, so only the click-through is gone.
414
  # The reciprocal of the invoice's `order_link` (D-88): the invoices raised from THIS
415
  # order, matched on the order's own name.
416
  {"key": "invoices", "label": "Invoices", "type": "link", "source": "overlay",
 
1121
  have = columns(cur, "res_partner")
1122
  if not have or not columns(cur, "account_move"):
1123
  return []
1124
+ # ⭐⭐ W33-T48 (owner item 13). The grid served FIVE columns off a partner the census measured
1125
+ # at SEVENTY-SIX populated fields. The nine added here are the ones a buyer actually asks a
1126
+ # vendor record for β€” how to reach them, who they are for tax, and where they are.
1127
+ # β›” EVERY ONE GOES THROUGH `_col`, which substitutes a literal when the mirror lacks the
1128
+ # column. That is not defensive habit: this projection has to keep working against a mirror
1129
+ # that has not been re-synced since the widening, and the alternative is a reader that raises
1130
+ # on the exact box the widening was meant to help. The columns arrive when the backfill runs;
1131
+ # until then these read blank rather than failing.
1132
+ blank = chr(39) + chr(39)
1133
+ sql = ("SELECT p.id, p.name, "
1134
+ f"{_col(have, 'p.country_name', blank)}, {_col(have, 'p.email', blank)}, "
1135
+ f"{_col(have, 'p.phone', blank)}, {_col(have, 'p.mobile', blank)}, "
1136
+ f"{_col(have, 'p.website', blank)}, {_col(have, 'p.vat', blank)}, "
1137
+ f"{_col(have, 'p.ref', blank)}, {_col(have, 'p.street', blank)}, "
1138
+ f"{_col(have, 'p.street2', blank)}, {_col(have, 'p.city', blank)}, "
1139
+ f"{_col(have, 'p.zip', blank)} "
1140
  "FROM res_partner p WHERE p.id IN "
1141
  f" (SELECT partner_id FROM account_move WHERE {_VENDOR_DOCS} "
1142
  " AND partner_id IS NOT NULL)")
1143
  out = []
1144
+ for (pid, name, country, email, phone, mobile, website, vat, ref,
1145
+ street, street2, city, zipc) in cur.execute(sql).fetchall():
1146
  out.append({
1147
  "_id": str(pid),
1148
  "vendor": str(name or ""),
1149
  "odoo_id": int(pid),
1150
  VENDOR_JOIN_KEY: int(pid),
1151
  "country": str(country or ""),
1152
+ "email": str(email or ""),
1153
+ "phone": str(phone or ""),
1154
+ "mobile": str(mobile or ""),
1155
+ "website": str(website or ""),
1156
+ "vat": str(vat or ""),
1157
+ "vendor_ref": str(ref or ""),
1158
+ "street": str(street or ""),
1159
+ "street2": str(street2 or ""),
1160
+ "city": str(city or ""),
1161
+ "zip": str(zipc or ""),
1162
  })
1163
  return out
1164
 
 
1194
  #: point: adding an Odoo entity is a spec row plus a reader, not a fifth copy of the spawn code.
1195
  #: ⚠ ORDER MATTERS ONLY FOR THE REFUSAL MESSAGE; `plan` checks every cap before anything commits.
1196
  TABLES = (
1197
+ # β›”β›” `customers` AND `products` ARE GONE FROM THIS TUPLE β€” W33-T44 / owner item 12 /
1198
+ # AMENDMENT A2. They presented the same SUBJECTS as the compiled registry modules
1199
+ # `customer_data` and `product_data` ("why do we have 'Odoo products' already with the current
1200
+ # Products database? The Unique ID is redundant"), and R2 ruled the LEGACY key survives: it
1201
+ # keeps its store bucket, so no saved view, grant, cohort or formula moves, and it now carries
1202
+ # the twins' join keys (`partner_id`; `product_id` pending the ask in `mailbox/E.md`).
1203
+ # Dropping the row here is what stops them being planned, built or re-created;
1204
+ # `RETIRED_KEYS` below is what removes the rows a tenant already has.
1205
  ("invoices", INVOICES_KEY, "Odoo invoices", invoice_fields),
1206
  ("orders", ORDERS_KEY, "Odoo orders", order_fields),
1207
  # ⭐ WAVE 28 / R1. Measured populations: 19 / 192 / 6,538 / 393 β€” every one of them two orders
 
1221
  ("gl_lines", GL_LINES_KEY, "Odoo GL lines", gl_line_fields),
1222
  )
1223
 
1224
+ #: store key -> the REAL-WORLD POPULATION that table presents (`core.registry`'s `subject`
1225
+ #: vocabulary, same strings, one namespace). ⭐ W33-T41 / item 12a.
1226
+ #:
1227
+ #: β›” A SEPARATE MAP RATHER THAN A FIFTH ELEMENT ON EACH `TABLES` ROW, and that is not tidiness:
1228
+ #: six sites in this file and two in the gate unpack `for bucket, key, _l, _f in TABLES`, so
1229
+ #: widening the tuple is eight edits that all fail loudly at once and one β€” in a gate β€” that
1230
+ #: would fail QUIETLY, having already been rewritten to match. The subject is a fact ABOUT the
1231
+ #: key, and this is the shape that says so.
1232
+ #:
1233
+ #: ⚠ THE THREE res.partner SUBSETS ARE THREE SUBJECTS, NOT ONE. Customers, agents and vendors all
1234
+ #: read `res_partner`, and they are different POPULATIONS of it β€” the claim is over who is in the
1235
+ #: database, never over which Odoo model was queried. Same for `account_move`, which is the
1236
+ #: customer-invoice book under one WHERE and the vendor-bill book under another.
1237
+ TABLE_SUBJECTS = {
1238
+ # β›” `CUSTOMERS_KEY` and `PRODUCTS_KEY` are absent β€” W33-T44 retired them, and their subjects
1239
+ # (`odoo:res.partner`, `odoo:product.product`) are claimed by `core.registry`'s `customer_data`
1240
+ # and `product_data` rows, which is now the ONLY claim on each. That is item 12 satisfied: one
1241
+ # subject, one database, and `subject_conflict` would REFUSE either of these keys if a future
1242
+ # spec row tried to bring it back.
1243
+ INVOICES_KEY: "odoo:account.move.customer",
1244
+ ORDERS_KEY: "odoo:sale.order",
1245
+ AGENTS_KEY: "odoo:res.partner.agent",
1246
+ ACCOUNTS_KEY: "odoo:account.account",
1247
+ VENDORS_KEY: "odoo:res.partner.vendor",
1248
+ BILLS_KEY: "odoo:account.move.vendor",
1249
+ ORDER_LINES_KEY: "odoo:sale.order.line",
1250
+ GL_LINES_KEY: "odoo:account.move.line",
1251
+ }
1252
+
1253
+ #: β›”β›” THE TWO RETIRED KEYS β€” W33-T44 / AMENDMENT A2. The rows a tenant ALREADY HAS.
1254
+ #:
1255
+ #: Dropping the `TABLES` rows above stops these being planned or re-created; it does NOT remove the
1256
+ #: definitions and rows already sitting in a tenant's `user_tables` document, which is what the
1257
+ #: owner actually sees in the nav. This set is what removes them, and it is applied inside
1258
+ #: `apply_plan`'s single atomic updater β€” i.e. BY THE CONTAINER, on the boot rebuild and every
1259
+ #: resync.
1260
+ #:
1261
+ #: β›” IT MUST BE THE CONTAINER AND NOT A CLI, AND THIS IS MEASURED, NOT CAUTIOUS (D-195): a
1262
+ #: developer's script CAN write the tenant store, the write returns clean, a fresh read confirms
1263
+ #: it β€” and the running Space reverts it within a minute, because the container holds the document
1264
+ #: and re-uploads its own copy (download-modify-upload, last write wins). A removal shipped as a
1265
+ #: script is a dry run that reports success.
1266
+ #:
1267
+ #: β›” AND A SWEEP THAT DELIVERS MUST NOT CREATE. Wave 32's `ut_ensure` was handed a merge-only job
1268
+ #: and minted 8 empty databases in every tenant, because the door it used creates when absent. This
1269
+ #: is a `pop`, it runs only over keys already present, and the gate asserts BOTH halves β€” removed,
1270
+ #: AND not re-created on the next pass. Gating only the removal would pass on a tree that deletes
1271
+ #: and re-adds the table every 30 minutes.
1272
+ RETIRED_KEYS = (CUSTOMERS_KEY, PRODUCTS_KEY)
1273
+
1274
+ #: β›” THE GRANDFATHER LIST IS DELETED β€” W33-T44 did what it was written to force.
1275
+ #:
1276
+ #: It existed for exactly one wave, to keep the product runnable between W33-T41 (the uniqueness
1277
+ #: check) and W33-T44 (the retirement): the check is CORRECT and the collision it forbids was
1278
+ #: LIVE, so without a named exemption the spawn refused on any fresh document and tenant #0 wrote
1279
+ #: nothing at all. The exemption was ratcheted BOTH ways β€” a collision outside it was a new
1280
+ #: duplicate, a member that stopped colliding was a stale exemption β€” so retiring the twins turned
1281
+ #: the gate RED until this constant went with them. It did, and that is the ratchet working.
1282
+ #: `verify_odoo_relational::_prove_subject_uniqueness` now asserts the collision set is EMPTY.
1283
+
1284
  #: The bucket -> reader map. β›” ITS ABSENCES ARE LOAD-BEARING: a bucket with no reader has no
1285
  #: python row builder ANYWHERE, which is what makes "never materialised" structural rather than a
1286
  #: policy `plan()` could forget. The two line grains are absent for that reason and no other.
 
1554
  return built
1555
 
1556
 
1557
+ def apply_plan(rt, built, username="automation", today=None, report=None):
1558
  """Create-or-merge every table in `built` and its rows. Idempotent by construction.
1559
 
1560
  Row ids ARE the Odoo ids, so a re-run updates in place and never appends a second copy of the
 
1567
  raise Refused("; ".join(built["problems"]))
1568
  stamp = today or _iso_today()
1569
  written = {}
1570
+ # ⚠ An OPTIONAL out-parameter, not a return-shape change: `written` has one value shape and
1571
+ # keeps it. A caller that wants to SAY what was retired passes a dict; `refresh` does.
1572
+ retired = []
1573
  plans = [(key, label, fields(), built[bucket])
1574
  for bucket, key, label, fields in TABLES if bucket in built]
1575
 
 
1587
  # create. Inside one updater, a raise aborts before anything is persisted.
1588
  def _apply_all(cur):
1589
  cur = cur if isinstance(cur, dict) else {}
1590
+ # β›”β›” W33-T44 β€” THE RETIREMENT, INSIDE THE SAME ATOMIC WRITE. This updater is the ONE
1591
+ # sync write the whole spawn makes, and it runs in the CONTAINER (boot rebuild + every
1592
+ # resync), which is the only place a change to the tenant document survives (D-195: the
1593
+ # same edit from a CLI reports success and is reverted within a minute).
1594
+ # ⚠ It POPS and never creates: `RETIRED_KEYS` are absent from `TABLES`, so `plans` cannot
1595
+ # contain them, and there is no door here that could re-add one. `removed` is reported so
1596
+ # a caller can SAY what happened rather than infer it from a table going missing.
1597
+ # β›” THE REMOVED KEYS DO **NOT** GO INTO `written`, and the first cut of this put them
1598
+ # there. `written` maps table key -> a COUNTS dict, and every consumer iterates its
1599
+ # `.values()` expecting `c["added"]`; a list under `"_retired"` made the very next leg die
1600
+ # with `TypeError: list indices must be integers`. One dict, two value shapes, is a
1601
+ # sentinel in a result set [[sentinel-in-a-sort-key]] β€” so the retirement reports through
1602
+ # its OWN channel and the return type stays exactly what it was.
1603
+ for key in RETIRED_KEYS:
1604
+ if cur.pop(key, None) is not None:
1605
+ retired.append(key)
1606
  for key, label, fields, rows in plans:
1607
  written[key] = _ensure_table_inplace(cur, key, label, fields, rows, username, stamp)
1608
  return cur
1609
 
1610
  rt.update(_ut().STORE_KEY, _apply_all, flush="sync")
1611
+ if report is not None and retired:
1612
+ report["retired"] = sorted(retired)
1613
  return written
1614
 
1615
 
 
1641
  for row in wanted.values():
1642
  row["refreshed"] = stamp
1643
  counts = {"added": 0, "updated": 0, "removed": 0, "rows": len(wanted)}
1644
+ subject = TABLE_SUBJECTS.get(key)
1645
  table = cur.get(key)
1646
  if table is None:
1647
  if len(cur) >= ut.MAX_TABLES:
1648
  # ⚠ `ut_ensure` returns silently at this cap; a silent no-op here would report a
1649
  # successful refresh over a table that does not exist.
1650
  raise Refused(f"{key}: tenant is at MAX_TABLES={ut.MAX_TABLES}; nothing created")
1651
+ # ⭐⭐ W33-T41 / item 12a β€” THE UNIQUENESS CHECK, ON THE LINE THAT MINTED THE DUPLICATE.
1652
+ #
1653
+ # β›” IT GUARDS **CREATION ONLY**, AND THAT IS THE WHOLE DESIGN, NOT A WEAKENING. This
1654
+ # function is the boot rebuild and the 1800 s resync; it adopts an existing table by key
1655
+ # on every pass. A claim test outside this `if` refuses the table it created last boot,
1656
+ # `_apply_all` raises inside the updater, and tenant #0 spawns NOTHING β€” the check would
1657
+ # take the product down to prevent a duplicate that already exists. Refusing the SECOND
1658
+ # birth is what "make sure this never happens" asks for; the FIRST one is T44's job to
1659
+ # remove, and it is removed by dropping its `TABLES` row, not by a guard here.
1660
+ #
1661
+ # ⚠ Claims are gathered from THIS TENANT'S document (`cur`) plus the compiled registry.
1662
+ # Never a module-level cache: one Space process serves every tenant, and two tenants both
1663
+ # holding an "Odoo customers" is correct (that is D-169's shape, and it is not repeated
1664
+ # here). A stored definition with no `subject` claims nothing.
1665
+ held = {t["subject"]: k for k, t in cur.items()
1666
+ if isinstance(t, dict) and t.get("subject")}
1667
+ # ⭐ NO EXEMPTION ANY MORE. The grandfather list is deleted with the two tables it covered
1668
+ # (W33-T44), so this is now the plain rule the owner asked for: one subject, one database.
1669
+ other = _registry().subject_conflict(subject, key, claimed=held)
1670
+ if other:
1671
+ raise Refused(
1672
+ f"{key}: refusing to create a second database for {subject!r} β€” {other!r} "
1673
+ f"already presents it. One subject, one database (item 12a); if this table is "
1674
+ f"meant to replace {other!r}, retire {other!r} first rather than shipping both")
1675
  table = cur[key] = {
1676
  "key": key, "label": label, "source": ut.AUTOMATION_SOURCE,
1677
  "createdBy": username, "created": stamp, "fields": [], "rows": {},
 
1687
  # an empty grid.
1688
  if key in READ_THROUGH_KEYS:
1689
  table["readThrough"] = True
1690
+ # W33-T41 β€” the claim, made DURABLE on the definition. Written on every pass for the same
1691
+ # reason `readThrough` is: it is derived from this module's own structure, so the code is what
1692
+ # it has to agree with, and a definition that lost the stamp should regain it rather than go
1693
+ # on being invisible to the next table's claim test. ⚠ It is also the ONLY way the check sees
1694
+ # a `ut_*` table at all β€” those are per-tenant DATA, never compiled registry rows, so a cold
1695
+ # process reading a fresh document has nothing else to read the claim off.
1696
+ if subject:
1697
+ table["subject"] = subject
1698
  have = {str(f.get("key")): f for f in (table.get("fields") or [])}
1699
  for field in fields:
1700
+ # β›”β›” THE ONE DOOR THAT BYPASSES THE FIELD VALIDATOR, HARDENED WHERE IT BYPASSES IT.
1701
+ #
1702
+ # `user_tables._clean_field` stamps `source: 'overlay'` unconditionally on create AND
1703
+ # patch, so every field that goes through the normal door has one. This function does NOT
1704
+ # go through that door β€” it writes definitions straight into the document β€” and
1705
+ # `aios_grid.rows_from_pool` reads `field["source"]` as a HARD KEY, so a contract that
1706
+ # ever omitted it would 500 the entire rows route rather than degrade one column. Measured
1707
+ # today across 9 automation-owned tables and 173 fields: zero are missing it, so this is
1708
+ # LATENT, not live (A's PENDING row, raised as D-9 and corrected by D-23 β€” the crash that
1709
+ # prompted it came from a hand-written test fixture, not from production).
1710
+ #
1711
+ # ⚠ Fixed HERE rather than by softening `rows_from_pool` to `.get`, deliberately: a
1712
+ # missing `source` means the definition does not say which stratum owns the column, and
1713
+ # rendering it as blank would bury that. The default matches what the validator would
1714
+ # have stamped, so the bypass stops being a hole without inventing a second rule.
1715
+ if isinstance(field, dict) and not field.get("source"):
1716
+ field = {**field, "source": "overlay"}
1717
  fkey = str(field.get("key"))
1718
  if fkey not in have:
1719
  table.setdefault("fields", []).append(dict(field))
 
1812
  built.pop(bucket, None)
1813
  else:
1814
  skipped = []
1815
+ # ⭐ W33-T44: a table this pass REMOVED is reported, not inferred from a grid going missing.
1816
+ # Same rule as `skipped` below β€” R6's second sentence: what was deliberately not built (or no
1817
+ # longer built) SAYS SO, with the keys.
1818
+ plan_report = {}
1819
+ written = apply_plan(rt, built, username=username, today=today, report=plan_report)
1820
  return {"tables": written,
1821
+ **({"retired": plan_report["retired"]} if plan_report.get("retired") else {}),
1822
  # R6's second sentence: a set that was deliberately not built SAYS SO, with the keys.
1823
  **({"skipped": skipped} if skipped else {}),
1824
  **{bucket: len(built[bucket]) for bucket, _k, _l, _f in TABLES if bucket in built}}
api/routes_admin.py CHANGED
@@ -356,7 +356,78 @@ def set_password(username: str, body: dict = Body(default=None),
356
  #: so a migrated account would be locked out of Product with no control anywhere to grant it:
357
  #: fail-closed, but unadministrable. The editor grows its section with zero client edits
358
  #: (S3 built it off this route's `modules` + `fields_by_module`).
359
- _PERM_MODULES = ("customer_data", "product_data")
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
360
 
361
 
362
  def _module_fields(key):
@@ -374,23 +445,98 @@ def _module_fields(key):
374
  a filter naming a field the product rows lack would DENY every row via `permits()`.
375
  The product contract is served CONSOLIDATED (every column) on purpose: a BU-scoped reader
376
  is served fewer columns, but the ADMIN is choosing what may ever be hidden.
 
 
 
 
 
 
 
 
 
377
  """
378
  import aios_grid
379
- src = aios_grid.product_fields() if key == "product_data" else aios_grid.FIELDS
 
 
 
 
 
 
 
 
 
 
380
  out = []
381
  for f in src:
382
  if not isinstance(f, dict) or not f.get("key"):
383
  continue
 
 
 
 
 
384
  out.append({"key": f["key"], "label": f.get("label") or f["key"],
385
  "type": f.get("type") or "text",
386
- **({"options": f["options"]} if isinstance(f.get("options"), list) else {}),
387
  **({"pinned": True} if f.get("pinned") else {})})
388
  return out
389
 
390
 
391
- def _clean_perms(v):
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
392
  """Validate a whole `perms` block. FAIL-CLOSED, and LOUD rather than lenient.
393
 
 
 
 
 
 
 
 
 
 
 
 
 
 
394
  β›” WHY THIS REFUSES WHERE `clean_filter_tree` DROPS. The view sanitiser is deliberately
395
  drop-per-node: one bad rule must never cost a user their whole saved view. A PERMISSION
396
  filter is the opposite situation β€” an admin types "agent is Tara", a leaf is silently
@@ -409,11 +555,21 @@ def _clean_perms(v):
409
  return None
410
  if not isinstance(v, dict):
411
  raise err(400, "bad_perms", "perms must be an object keyed by module")
412
- unknown = sorted(set(v) - set(_PERM_MODULES))
 
 
 
 
 
 
 
 
 
 
413
  if unknown:
414
  raise err(400, "bad_perms",
415
- f"unknown module keys: {unknown} β€” the permission editor governs "
416
- f"{list(_PERM_MODULES)}")
417
  out = {}
418
  for key, raw in v.items():
419
  if not isinstance(raw, dict):
@@ -730,8 +886,16 @@ def get_perms(username: str, session: Session = Depends(admin_gate)):
730
  # omitted from `perms` would become an explicit DENY the moment anyone pressed Save. The
731
  # default sent is the EFFECTIVE one (`may_access`), so an un-migrated record shows the
732
  # access its legacy grant currently gives rather than a guess in either direction.
 
 
 
 
 
 
 
 
733
  perms_out = {}
734
- for k in _PERM_MODULES:
735
  e = stored.get(k)
736
  perms_out[k] = e if isinstance(e, dict) else {
737
  "access": bool(perm_scope.may_access(rec, k)), "filter": None, "hiddenFields": []}
@@ -744,11 +908,20 @@ def get_perms(username: str, session: Session = Depends(admin_gate)):
744
  # `role == 'admin'` bypasses perms entirely (amendment 4) β€” sent so the editor can
745
  # say "Everything (admin)" instead of rendering stored rules that do not apply.
746
  "is_admin": perms.is_admin(rec),
747
- "modules": [{"key": k, "label": _MODULE_LABELS.get(k, k)} for k in _PERM_MODULES],
 
 
 
 
 
 
748
  # S3 ask 1 β€” CANONICAL SHAPE: a BARE ARRAY of fields per module key. Not the nav
749
  # schema envelope; the editor needs the field list and nothing else, and one shape
750
  # beats two readings of a sentence.
751
- "fields_by_module": {k: _module_fields(k) for k in _PERM_MODULES}}
 
 
 
752
 
753
 
754
  @router.put("/admin/users/{username}/perms")
@@ -781,7 +954,10 @@ def put_perms(username: str, body: dict = Body(default=None),
781
  raise err(400, "self_perms",
782
  "you cannot set permissions on your own account β€” ask another admin")
783
 
784
- cleaned = _clean_perms(body.get("perms")) or {}
 
 
 
785
  cleaned = _fold_legacy_scope(reg[uname], cleaned)
786
  _guard_bu_widening(reg[uname], cleaned, bool(body.get("confirm_widen")))
787
  users.set_access(uname, perms=cleaned)
@@ -796,7 +972,29 @@ def put_perms(username: str, body: dict = Body(default=None),
796
  "perms_v": int(rec.get("perms_v") or 0)}
797
 
798
 
799
- _MODULE_LABELS = {"customer_data": "Customer", "product_data": "Product"}
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
800
 
801
 
802
  @router.get("/settings")
@@ -827,6 +1025,20 @@ def settings(session: Session = Depends(require_session)):
827
  },
828
  "tenant": {"key": session.tenant, "name": getattr(session.runtime, "name",
829
  session.tenant)},
 
 
 
 
 
 
 
 
 
 
 
 
 
 
830
  # CHROME ONLY. Every /admin route re-checks the role server-side; this exists so the client
831
  # does not paint a "Manage users" button that 403s.
832
  "admin": perms.is_admin(session.user),
 
356
  #: so a migrated account would be locked out of Product with no control anywhere to grant it:
357
  #: fail-closed, but unadministrable. The editor grows its section with zero client edits
358
  #: (S3 built it off this route's `modules` + `fields_by_module`).
359
+ #:
360
+ #: ⭐⭐ WAVE 33 (owner item 11, W33-T33) β€” **THIS IS A CATALOGUE, NOT AN ANSWER.** It used to be
361
+ #: both, and that was the defect the owner flagged four times: `get_perms` served this tuple
362
+ #: verbatim and never touched `session.runtime`, so EVERY tenant was handed tenant #0's two grid
363
+ #: modules. What this names now is a fact about THIS FILE β€” the registry topics `_module_fields`
364
+ #: below has a field-schema arm for β€” and the answer a tenant receives is `_perm_modules(session)`,
365
+ #: which filters this by the tenant's own provisioning and merges the tenant's own `ut_*`
366
+ #: databases. β›” Serving this list directly again re-opens item 11; `verify_perm_scope`'s
367
+ #: `section_tenant_derived` NC does exactly that and reds.
368
+ #: ⭐ CONTRACT C3, CONSUMED (lane E's `W33-T46`, answered as `E-3`). The topic list is DERIVED from
369
+ #: `registry.governable_modules()` β€” `{topic: {'key','label','subject'}}` over every non-archived,
370
+ #: non-group_only registry row that declares a `topic`. What stays hand-written is
371
+ #: `_FIELD_PROVIDER_KEYS` below, and that is CODE, not policy: each name has its own `aios_grid`
372
+ #: contract behind it in `_module_fields`, and a topic with no arm cannot be governed by this
373
+ #: editor at all. The intersection is the honest answer to "what can this module build pickers for".
374
+ #:
375
+ #: ⚠ KEPT AS A SYMBOL RATHER THAN DELETED, deliberately. `verify_api`'s W30a reads
376
+ #: `routes_admin._PERM_MODULES` to assert `perm_scope.BU_FILTERABLE_MODULES` names exactly the
377
+ #: governed topics whose contract carries `dba`. Deleting the name would make that gate raise
378
+ #: `AttributeError` β€” a CRASH, which scores as "the gate is broken" rather than as a red
379
+ #: [[gate-must-go-red-not-crash]] β€” and `verify_api.py` is lane A's fence. Derived-and-kept gives
380
+ #: that assertion a truer subject than the literal ever did, and costs nobody a cross-fence edit.
381
+ _FIELD_PROVIDER_KEYS = frozenset({"customer_data", "product_data"})
382
+
383
+
384
+ def _governable_catalogue():
385
+ """The REGISTRY topics this module can govern: `[{key, label}]`, in registry order.
386
+
387
+ β›” No tenant filtering here β€” that is `perms.tenant_governable_modules`' job, and E's map is
388
+ tenant-blind by design. Two filters in two files answering one question is how the perms route
389
+ and `routes_nav` came apart in the first place.
390
+ """
391
+ import core.registry as registry
392
+ return [{"key": v["key"], "label": v["label"]}
393
+ for v in registry.governable_modules().values()
394
+ if v.get("key") in _FIELD_PROVIDER_KEYS]
395
+
396
+
397
+ _PERM_MODULES = tuple(m["key"] for m in _governable_catalogue())
398
+
399
+
400
+ def _perm_modules(session):
401
+ """The databases THIS tenant's permission editor governs: `[{key, label, enforced}]`.
402
+
403
+ Three inputs, none of them a literal: the topic catalogue above, the tenant's provisioned
404
+ module set (`tenant.config['modules']`, exactly the rule `routes_nav.py::nav` applies), and
405
+ the tenant's own `ut_*` databases read through the DEFINITIONS projection.
406
+
407
+ ⭐ `user_tables.nav_entries` rather than a fresh listing, deliberately: it is already the ONE
408
+ resolver for "which databases may this account see", it is `may_open`-filtered, and it reads
409
+ the projected document (`all_defs`) so this costs definitions, never the 28.6 MB of rows.
410
+ The caller is admin-gated, so the filter admits the whole tenant β€” but asking the resolver
411
+ instead of assuming that is what keeps this from becoming the SECOND idea of who may see a
412
+ table (`user_tables.may_open`'s own docstring is the record of the first time that happened).
413
+ """
414
+ import core.perms as _perms
415
+ import core.user_tables as user_tables
416
+ try:
417
+ ut = user_tables.nav_entries(viewer=session.uname, is_admin=True,
418
+ st=session.runtime)
419
+ except Exception:
420
+ # A store that cannot list the tenant's databases must not take the whole editor down β€”
421
+ # the topic half is still administrable, and an empty `ut_*` half is visibly empty.
422
+ ut = ()
423
+ return _perms.tenant_governable_modules(session.runtime, _governable_catalogue(),
424
+ ut_entries=ut)
425
+
426
+
427
+ def _enforced_keys(session):
428
+ """The module keys a permission block may actually be STORED against for this tenant."""
429
+ import core.perms as _perms
430
+ return _perms.enforced_module_keys(_perm_modules(session))
431
 
432
 
433
  def _module_fields(key):
 
445
  a filter naming a field the product rows lack would DENY every row via `permits()`.
446
  The product contract is served CONSOLIDATED (every column) on purpose: a BU-scoped reader
447
  is served fewer columns, but the ADMIN is choosing what may ever be hidden.
448
+
449
+ β›”β›” WAVE 33 β€” THE `else` USED TO BE A FALL-THROUGH, AND IT WAS SILENT-WRONG. The body read
450
+ `aios_grid.product_fields() if key == "product_data" else aios_grid.FIELDS`, so ANY key that
451
+ was not `product_data` got the CUSTOMER contract: the moment the governed list stopped being
452
+ a two-element literal, a `ut_*` database or a third topic would have been handed customer
453
+ columns, `_clean_perms` would have validated an admin's `hiddenFields` against them, and the
454
+ stored wall would name columns that module does not have β€” which `permits()` resolves by
455
+ DENYING every row. No error at any point. So the map is explicit and an unknown key RAISES:
456
+ a topic without a schema arm is a missing arm, never a customer grid in disguise.
457
  """
458
  import aios_grid
459
+ providers = {"customer_data": lambda: aios_grid.FIELDS,
460
+ "product_data": aios_grid.product_fields}
461
+ provider = providers.get(str(key or ""))
462
+ if provider is None:
463
+ # Reached only by a caller that skipped `_perm_modules`. Loud, because the alternative
464
+ # (`[]`) is an empty option list read as an ANSWER β€” wave 26 item 24, `if ([])` is truthy.
465
+ raise err(400, "ungoverned_module",
466
+ f"{key!r} has no permission-editor field schema β€” the editor governs "
467
+ f"{sorted(providers)}")
468
+ src = provider()
469
+ known = _server_side_vocabularies()
470
  out = []
471
  for f in src:
472
  if not isinstance(f, dict) or not f.get("key"):
473
  continue
474
+ opts = f.get("options")
475
+ if not isinstance(opts, list) or not opts:
476
+ # ⭐ D-76 / W33-T35 β€” SUPPLY WHAT THE SERVER ALREADY KNOWS. See
477
+ # `_server_side_vocabularies`.
478
+ opts = known.get(f["key"])
479
  out.append({"key": f["key"], "label": f.get("label") or f["key"],
480
  "type": f.get("type") or "text",
481
+ **({"options": list(opts)} if isinstance(opts, list) and opts else {}),
482
  **({"pinned": True} if f.get("pinned") else {})})
483
  return out
484
 
485
 
486
+ def _server_side_vocabularies():
487
+ """`{field key: [choice, …]}` for choice columns whose vocabulary is CLOSED and known here,
488
+ but which the grid contract does not declare.
489
+
490
+ β›” D-76 β€” WHY THE PERMISSION EDITOR'S DROPDOWNS WERE EMPTY, AND WHY THE FIX IS HERE. The grid
491
+ resolves a choice column through `types.ts::choiceVocabulary`: the DECLARED list when the
492
+ field has one, otherwise the values seen in the loaded ROWS. This editor loads no rows, so
493
+ every column that leans on the second branch renders `Select…` with nothing in it β€” the
494
+ identical shape wave 26 item 24 closed on the Product grid, one surface over. ⚠ And the fix
495
+ belongs at the CALLER, never in the panel: `FilterBuilderPanel::choicesFor` treats a supplied
496
+ list as authoritative *including when it is empty* ("this column has no values" is an answer),
497
+ which is correct and must not be weakened. So the caller stops handing it nothing.
498
+
499
+ ⭐ IMPORTED, NEVER RESTATED. `stock_bucket`'s labels are `modules/inventory.COVERAGE_LABELS`,
500
+ the same constant `_bucket()` mints from β€” a copy here would be a second vocabulary that
501
+ drifts the first time a band is renamed [[constant-two-features-share]]. The import is lazy
502
+ for the same reason `aios_grid`'s is: this route should not pull the analytics stack at
503
+ module load.
504
+
505
+ ⚠ TWO COLUMNS ARE DELIBERATELY ABSENT, and R6's second sentence says to name them rather than
506
+ let them look handled:
507
+ Β· `category` (product) β€” an OPEN vocabulary read off Odoo's product categories. There is no
508
+ closed list to declare, and discovering it would mean a pool read on an admin route.
509
+ Β· `odoo_status` (customer) β€” WAS bare for the same reason and is NOT any more: lane E
510
+ answered `ASK D-15` by declaring `options: ['Active','Archived']` on the CONTRACT, which
511
+ is the right home (the grid picks a declared list up for free) and is why this function
512
+ never needed an arm for it. Recorded because the ask, not a guess, is what settled it.
513
+ """
514
+ try:
515
+ import modules.inventory as inventory
516
+ labels = list(inventory.COVERAGE_LABELS or ())
517
+ except Exception:
518
+ # A missing analytics import must not take the permission editor down; the dropdown
519
+ # degrades to today's empty one rather than the page to a 500.
520
+ return {}
521
+ return {"stock_bucket": labels} if labels else {}
522
+
523
+
524
+ def _clean_perms(v, enforced_keys=None, listed_keys=None):
525
  """Validate a whole `perms` block. FAIL-CLOSED, and LOUD rather than lenient.
526
 
527
+ ⭐ WAVE 33 (W33-T33) β€” `enforced_keys` IS PER TENANT AND IT IS NOT OPTIONAL IN PRACTICE. It
528
+ used to be `set(_PERM_MODULES)`, a literal, which is the same tenant-blindness owner item 11
529
+ flagged: an admin of a tenant provisioned for neither grid module could still store a wall
530
+ for both. It now comes from `_enforced_keys(session)`, i.e. from the tenant's own catalogue.
531
+ The default is the catalogue itself so a test or a future caller cannot silently widen it.
532
+
533
+ β›” AND "LISTED" IS NOT "ENFORCED". `listed_keys` is every database the editor SHOWS, which
534
+ includes this tenant's `ut_*` ones. `perm_scope` is never consulted on those (visibility
535
+ there is `user_tables.may_open` β€” creator, admin, or a `core.shares` grant), so a wall stored
536
+ against one would be INERT: the editor would say DENY and the table routes would keep
537
+ serving. It is refused with a message that says WHICH of the two it is, because
538
+ "unknown module key" for a database the admin can see on screen is the wrong sentence.
539
+
540
  β›” WHY THIS REFUSES WHERE `clean_filter_tree` DROPS. The view sanitiser is deliberately
541
  drop-per-node: one bad rule must never cost a user their whole saved view. A PERMISSION
542
  filter is the opposite situation β€” an admin types "agent is Tara", a leaf is silently
 
555
  return None
556
  if not isinstance(v, dict):
557
  raise err(400, "bad_perms", "perms must be an object keyed by module")
558
+ governed = set(enforced_keys) if enforced_keys is not None else set(_PERM_MODULES)
559
+ shown = set(listed_keys) if listed_keys is not None else governed
560
+ inert = sorted(set(v) & (shown - governed))
561
+ if inert:
562
+ raise err(400, "unenforced_module",
563
+ f"{inert}: this tenant can SEE these databases, but a permission wall stored "
564
+ f"against them would never be applied β€” `routes_tables` gates a `ut_*` "
565
+ f"database on `user_tables.may_open` (creator, admin, or a share), not on "
566
+ f"`perm_scope`. Saved silently, it would read as a restriction that is not "
567
+ f"there. Share the database instead, or arm `perm_scope` over `ut_*` first.")
568
+ unknown = sorted(set(v) - shown)
569
  if unknown:
570
  raise err(400, "bad_perms",
571
+ f"unknown module keys: {unknown} β€” this tenant's permission editor governs "
572
+ f"{sorted(governed)}")
573
  out = {}
574
  for key, raw in v.items():
575
  if not isinstance(raw, dict):
 
886
  # omitted from `perms` would become an explicit DENY the moment anyone pressed Save. The
887
  # default sent is the EFFECTIVE one (`may_access`), so an un-migrated record shows the
888
  # access its legacy grant currently gives rather than a guess in either direction.
889
+ #
890
+ # ⭐⭐ WAVE 33 (W33-T33, owner item 11) β€” THE LIST IS THIS TENANT'S, NOT A LITERAL. Everything
891
+ # below was built from `_PERM_MODULES` and never touched `session.runtime`, which is why a
892
+ # nurilab or gtmlab admin opened Manage user and saw Royal Imports' Customer and Product
893
+ # databases. `_perm_modules` applies the tenant's own provisioning (the rule `routes_nav`
894
+ # has applied since wave 18) and merges the tenant's own `ut_*` databases.
895
+ modules = _perm_modules(session)
896
+ enforced = [m["key"] for m in modules if m.get("enforced")]
897
  perms_out = {}
898
+ for k in enforced:
899
  e = stored.get(k)
900
  perms_out[k] = e if isinstance(e, dict) else {
901
  "access": bool(perm_scope.may_access(rec, k)), "filter": None, "hiddenFields": []}
 
908
  # `role == 'admin'` bypasses perms entirely (amendment 4) β€” sent so the editor can
909
  # say "Everything (admin)" instead of rendering stored rules that do not apply.
910
  "is_admin": perms.is_admin(rec),
911
+ # ⭐ `enforced` rides every row. A `ut_*` database is LISTED because the owner asked to
912
+ # see every database, and declared UNENFORCED because `perm_scope` is not consulted on
913
+ # that path β€” so the editor can say which it is instead of painting a dead panel
914
+ # (wave 26 item 24: an empty answer that reads as an answer). The LABEL comes from the
915
+ # registry / the table's own definition; `_MODULE_LABELS` was a hardcoded copy of two
916
+ # registry labels and is deleted with this change.
917
+ "modules": modules,
918
  # S3 ask 1 β€” CANONICAL SHAPE: a BARE ARRAY of fields per module key. Not the nav
919
  # schema envelope; the editor needs the field list and nothing else, and one shape
920
  # beats two readings of a sentence.
921
+ # ENFORCED MODULES ONLY, and that is the honest shape: a picker for a database whose
922
+ # wall is never applied is a control that lies. An unenforced row carries no entry
923
+ # here at all, so a client cannot mistake `[]` for "this database has no fields".
924
+ "fields_by_module": {k: _module_fields(k) for k in enforced}}
925
 
926
 
927
  @router.put("/admin/users/{username}/perms")
 
954
  raise err(400, "self_perms",
955
  "you cannot set permissions on your own account β€” ask another admin")
956
 
957
+ _mods = _perm_modules(session)
958
+ cleaned = _clean_perms(body.get("perms"),
959
+ enforced_keys={m["key"] for m in _mods if m.get("enforced")},
960
+ listed_keys={m["key"] for m in _mods}) or {}
961
  cleaned = _fold_legacy_scope(reg[uname], cleaned)
962
  _guard_bu_widening(reg[uname], cleaned, bool(body.get("confirm_widen")))
963
  users.set_access(uname, perms=cleaned)
 
972
  "perms_v": int(rec.get("perms_v") or 0)}
973
 
974
 
975
+ def _store_binding(session):
976
+ """`data_binding.describe()` for the store THIS SESSION's tenant actually writes.
977
+
978
+ ⭐ THE TENANT'S STORE, NOT THE PROCESS DEFAULT, AND THAT DISTINCTION IS THE POINT. Tenant #0
979
+ rides `core.store`'s module default, but nurilab/gtmlab/loopable are bound to their OWN dataset
980
+ repos by `harness/runtime.py:470`, straight out of the control-plane record β€” which is the path
981
+ that carried three tenants' PRODUCTION stores onto staging while `--data-repo` isolated only
982
+ tenant #0. Reporting the process default here would show "staging store, all fine" to exactly
983
+ the tenants that were not fine.
984
+
985
+ ⚠ Degrades to a stated `unknown` rather than raising: a settings page that 500s because it
986
+ could not describe the store is worse than one that says it does not know.
987
+ """
988
+ try:
989
+ import core.data_binding as data_binding # noqa: PLC0415
990
+ import core.store as store # noqa: PLC0415
991
+ bound = getattr(session.runtime, "store_handle", None)
992
+ repo = getattr(bound, "repo", None) or store.REPO
993
+ return data_binding.describe(repo)
994
+ except Exception as e: # noqa: BLE001
995
+ return {"deployment": "unknown", "production": False, "repo": "",
996
+ "writable": False, "refusal": f"could not resolve the store binding: {e}",
997
+ "localOverride": False, "allowlist": [], "observed": {}}
998
 
999
 
1000
  @router.get("/settings")
 
1025
  },
1026
  "tenant": {"key": session.tenant, "name": getattr(session.runtime, "name",
1027
  session.tenant)},
1028
+ # ⭐⭐ D-315 β€” WHICH STORE THIS CONTAINER IS BOUND TO, AND WHETHER IT MAY WRITE IT.
1029
+ #
1030
+ # β›” THIS IS NOT DIAGNOSTIC GARNISH; IT IS THE ONLY WAY THE QUESTION CAN BE ANSWERED.
1031
+ # D-160 established that a Space's environment cannot be read from outside, so "is staging
1032
+ # pointed at production data?" is a question only the container can answer β€” and it went
1033
+ # unanswered for the whole window in which two builds wrote tenant #0's real store.
1034
+ # `verify_live` asserts this field after logging in, which is a check that survives a role
1035
+ # swap in a way that reading a deploy log never did.
1036
+ #
1037
+ # ⚠ Session-gated, beside `version`, for `main.py::health`'s reason: a Space id and a
1038
+ # dataset repo id are public names, but a health endpoint is the first URL a scanner finds
1039
+ # and it may not describe the deployment. No customer data is here β€” only which store, and
1040
+ # yes or no.
1041
+ "store": _store_binding(session),
1042
  # CHROME ONLY. Every /admin route re-checks the role server-side; this exists so the client
1043
  # does not paint a "Manage users" button that 403s.
1044
  "admin": perms.is_admin(session.user),
api/routes_alerts.py CHANGED
@@ -172,8 +172,36 @@ def notification_view(item):
172
  # ⚠ `kind` is OVERWRITTEN, not merged. There was one stored value (`automation_review`) and it
173
  # is D-101's dead one; leaving it through would give the client two vocabularies for one
174
  # question, which is the defect this wave's item 6 is about in a different file.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
175
  out = {**item, "read": bool(item.get("read")), "kind": kind,
176
- "subject": subject or "Notification"}
177
  if target is not None:
178
  out["target"] = target
179
  return out
@@ -191,7 +219,61 @@ def inbox_view(box):
191
  items = box.get("items")
192
  if not isinstance(items, list):
193
  return box
194
- return {**box, "items": [notification_view(n) for n in items]}
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
195
 
196
 
197
  def _topic_or_400(raw):
@@ -283,7 +365,7 @@ def _evaluate(session: Session, rec: dict, assemblies=None):
283
  if isinstance(assemblies, dict):
284
  assemblies[memo_key] = g
285
 
286
- view = (g.get("views") or {}).get(str(rec.get("viewId")))
287
  if not isinstance(view, dict):
288
  # Deleted, or un-shared out from under the alert. Say so on the RECORD rather than
289
  # deleting the alert: an alert that silently vanishes is indistinguishable from one that
@@ -293,7 +375,40 @@ def _evaluate(session: Session, rec: dict, assemblies=None):
293
  # The SAME row build the grid and `/customers` use β€” `rows_from_pool` is what puts derived
294
  # and overlay values on a row. Evaluating a filter against raw pool dicts would silently
295
  # never match any condition on a user-created or measure column.
296
- rows = aios_grid.rows_from_pool(g["rows_src"], g["fields"], g["ws"].get("overlays"),
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
297
  derived=g.get("derived"))
298
  config = view.get("config") or view
299
  ctx = filter_eval.EvalCtx(
@@ -391,12 +506,28 @@ def _require_filtered_view(session: Session, topic: str, view_id: str):
391
  # the one path that reaches this branch is also the one path with nothing to debug from.
392
  raise err(503, "unavailable",
393
  f"the table could not be read ({type(e).__name__}) β€” try again in a moment")
394
- view = (g.get("views") or {}).get(str(view_id))
395
  if not isinstance(view, dict):
396
  raise err(404, "no_view", "that view does not exist on this table")
397
  nodes, _conj = filter_eval.tree_parts((view.get("config") or view).get("filters") or [])
398
 
399
- # β›”β›” WAVE 32 Β· T22 β€” **THE CALL BELOW WAS MISSING AN ARGUMENT, AND THAT IS OWNER ITEM 17.**
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
400
  #
401
  # `is_rule_active(rule, columns)` takes TWO parameters (`harness/filter_sql.py`; every other
402
  # caller in the repo passes both). This one passed ONE, so the moment the walk reached a LEAF
 
172
  # ⚠ `kind` is OVERWRITTEN, not merged. There was one stored value (`automation_review`) and it
173
  # is D-101's dead one; leaving it through would give the client two vocabularies for one
174
  # question, which is the defect this wave's item 6 is about in a different file.
175
+ # ⭐⭐ W33-T28 (owner: "the Inbox reads like email") β€” THE SENDER, WHICH DID NOT EXIST.
176
+ #
177
+ # β›” A `verifier` reading the finished wave-32 surface found that the row's sender POSITION was
178
+ # occupied by `kindLabel(n.kind)` β€” the literals "Alert" / "Automation" / "Shared with you" β€”
179
+ # i.e. a CATEGORY standing where a who belongs, and no sender field anywhere on the wire, in
180
+ # the model or in the markup. Mail has a from. This is it.
181
+ #
182
+ # ⚠ IT IS DERIVED HERE, NOT STORED, FOR EVERY KIND BUT ONE β€” and the exception is the point.
183
+ # An alert firing and an automation landing rows have no person behind them; their honest
184
+ # sender is the machine that did it, named as the thing the reader recognises. A SHARE has a
185
+ # real person, and only the producer knows who: `routes_shares.py` writes it as `actor` and
186
+ # this reads it back. β›” It is NOT parsed out of the body prose ("<name> shared this with
187
+ # you") β€” a sender recovered by regexing a sentence breaks the first time the sentence is
188
+ # reworded, and it would break silently, in the header.
189
+ #
190
+ # ⚠ FALLS BACK, NEVER BLANK. A share queued BEFORE `actor` existed has none, and a row with an
191
+ # empty from column reads as a broken inbox rather than as an old notification.
192
+ actor = str(item.get("actor") or "").strip()
193
+ if kind == NOTIF_KIND_SHARE:
194
+ sender = actor or "A teammate"
195
+ elif kind == NOTIF_KIND_AUTOMATION:
196
+ # β›” "Agents", not "Automation" (W34-T40, corrected at QA 2026-08-17). The client's
197
+ # `senderOf` already falls back to `AGENTS_MODULE_LABEL` β€” but `if (sent) return sent`
198
+ # runs FIRST, so this server literal won and every actor-less automation notification
199
+ # showed the retired module name in the inbox's From column.
200
+ sender = actor or "Agents"
201
+ else:
202
+ sender = actor or "Alerts"
203
  out = {**item, "read": bool(item.get("read")), "kind": kind,
204
+ "subject": subject or "Notification", "sender": sender}
205
  if target is not None:
206
  out["target"] = target
207
  return out
 
219
  items = box.get("items")
220
  if not isinstance(items, list):
221
  return box
222
+ # ⭐⭐ W33-T28 / D-208 β€” THE SERVER'S CLOCK RIDES WITH THE PAGE, and it is what lets the client
223
+ # render a mail-shaped stamp ("09:41" today, "Aug 12" beyond) instead of `2026-08-13 09:41`.
224
+ #
225
+ # β›” THE CLIENT MUST NOT READ ITS OWN CLOCK, which is D-208's exit condition word for word and
226
+ # is why this key exists rather than a `new Date()` in the browser. `at` is sent as UTC WITH
227
+ # its offset (D-18) precisely so every reader sees the same instant; deciding "is this today?"
228
+ # against a browser clock would re-introduce the drift the offset exists to remove β€” a reader a
229
+ # day ahead being told an event happened tomorrow [[date-window-vocabulary]]. Both operands
230
+ # now come from the same machine.
231
+ # ⚠ Same funnel as the enrichment, so the read door and the mark-read door cannot disagree β€”
232
+ # the note two lines up records what happened last time only one of them was enriched.
233
+ return {**box, "now": alerts._now_iso(),
234
+ "items": [notification_view(n) for n in items]}
235
+
236
+
237
+ def _view_by_id(g, view_id):
238
+ """One saved view out of an assembly, by id. `None` when there is no such view.
239
+
240
+ β›”β›” W33-T29 (owner: *"Alert me about new records"* answering "Something went wrong") β€” THIS
241
+ FUNCTION EXISTS BECAUSE TWO CALL SITES BOTH WROTE `(g.get("views") or {}).get(view_id)`, AND
242
+ `g["views"] IS A LIST`. `aios_grid.views_from_defs` returns `[{...}]`, `workspace_wire` passes
243
+ it straight out and both `ut_assembly` and `grid_assembly` return it unchanged β€” so `.get` on
244
+ it raises `AttributeError`, and `views_from_defs` always returns at least one element, so the
245
+ `or {}` never fires. **It raised on EVERY call, on every topic, since wave 20.**
246
+
247
+ β›” AND THE TWO SITES FAILED DIFFERENTLY, WHICH IS WHY ONLY ONE WAS EVER REPORTED. In
248
+ `_require_filtered_view` the raise lands ABOVE the handler's own `try`, so it leaves as a bare
249
+ FastAPI 500 and the client's `errorMessage` turns any 5xx into *"Something went wrong on our
250
+ side"* β€” the exact sentence the owner reported (D-107's shape, again: an attribute error above
251
+ the guard arrives as plain text rather than as our envelope). In `_evaluate` the identical
252
+ line is swallowed by `/notifications`' `except Exception: continue`, so **every stored
253
+ view-alert was silently dropped from the Inbox** and nobody had anything to report at all.
254
+ One expression, one loud symptom and one silent one.
255
+
256
+ ⚠ SO IT IS A FUNCTION, NOT TWO FIXED LINES. Two copies of "find the view" is what let one site
257
+ be discussed for three waves while its twin went unnoticed [[one-question-two-normalizers]].
258
+
259
+ ⚠ It accepts a dict too, and that is not defensive noise: `verify_alerts`' door fixture was
260
+ keyed `{id: view}` β€” which is precisely why the gate was green while production raised on
261
+ every call. The fixture is moving to the production shape in this same change, and tolerating
262
+ both here means a caller that legitimately holds one cannot resurrect the bug.
263
+ """
264
+ want = str(view_id or "")
265
+ if not want:
266
+ return None
267
+ views = (g or {}).get("views")
268
+ if isinstance(views, dict):
269
+ found = views.get(want)
270
+ return found if isinstance(found, dict) else None
271
+ if not isinstance(views, list):
272
+ return None
273
+ for v in views:
274
+ if isinstance(v, dict) and str(v.get("id") or "") == want:
275
+ return v
276
+ return None
277
 
278
 
279
  def _topic_or_400(raw):
 
365
  if isinstance(assemblies, dict):
366
  assemblies[memo_key] = g
367
 
368
+ view = _view_by_id(g, rec.get("viewId"))
369
  if not isinstance(view, dict):
370
  # Deleted, or un-shared out from under the alert. Say so on the RECORD rather than
371
  # deleting the alert: an alert that silently vanishes is indistinguishable from one that
 
375
  # The SAME row build the grid and `/customers` use β€” `rows_from_pool` is what puts derived
376
  # and overlay values on a row. Evaluating a filter against raw pool dicts would silently
377
  # never match any condition on a user-created or measure column.
378
+ #
379
+ # β›”β›” AND "THE SAME ROW BUILD" WAS NOT TRUE, WHICH MADE EVERY ALERT ON A `ut_*` DATABASE BLIND
380
+ # TO IMPORTED DATA. Found by a verifier driving one real assembly through both paths.
381
+ #
382
+ # `routes_tables.table_rows` β€” the grid the person is looking at β€” merges the DEFINITION rows
383
+ # underneath the overlay ("base first, overlay wins"; that merge is itself the fix for owner
384
+ # item 3, *"it all got reseted"*). `_evaluate` is a second copy of that read and never got it:
385
+ # it handed `ws['overlays']` to `rows_from_pool` raw, so for a `ut_*` table every base cell
386
+ # evaluated as BLANK. Measured on one assembly, same view, same rows:
387
+ # rows_src state='unpaid' / 'paid'
388
+ # _evaluate saw state='' / '' ⇐ every base cell blank
389
+ # the GRID saw state='unpaid' / 'paid'
390
+ # so `state eq unpaid` matched NOTHING while the view showed one row, and `state isEmpty`
391
+ # matched EVERYTHING while the view showed none. **The alert did not merely miss rows β€” it
392
+ # inverted.** End to end: a row whose value arrived by import, automation, paste or the create
393
+ # door never fired; only a value typed as a hand EDIT did.
394
+ #
395
+ # ⚠ Scope, so nobody widens the fix past its cause: materialised `ut_*` tables are hit;
396
+ # `customer`/`product` are not (their fields are `source: "odoo"` and read off `rows_src`);
397
+ # `ut_odoo_*` never reaches here (`with_rows=True` refuses first and returns
398
+ # `skipped: window_required`).
399
+ # β›” ORDER IS LOAD-BEARING AND IS THE GRID'S: base underneath, overlay ON TOP. Inverting it
400
+ # would let a stale definition value shadow an edit the user has just made β€” the same defect
401
+ # `table_rows`' own note records, arriving from the other side.
402
+ _ov = (g.get("ws") or {}).get("overlays") or {}
403
+ _merged = {}
404
+ for _r in g["rows_src"]:
405
+ _pid = str(_r.get("pid"))
406
+ _cells = {k: v for k, v in _r.items() if k != "pid"}
407
+ _o = _ov.get(_pid)
408
+ if isinstance(_o, dict):
409
+ _cells.update(_o)
410
+ _merged[_pid] = _cells
411
+ rows = aios_grid.rows_from_pool(g["rows_src"], g["fields"], _merged,
412
  derived=g.get("derived"))
413
  config = view.get("config") or view
414
  ctx = filter_eval.EvalCtx(
 
506
  # the one path that reaches this branch is also the one path with nothing to debug from.
507
  raise err(503, "unavailable",
508
  f"the table could not be read ({type(e).__name__}) β€” try again in a moment")
509
+ view = _view_by_id(g, view_id)
510
  if not isinstance(view, dict):
511
  raise err(404, "no_view", "that view does not exist on this table")
512
  nodes, _conj = filter_eval.tree_parts((view.get("config") or view).get("filters") or [])
513
 
514
+ # β›”β›” WAVE 33 Β· T29 β€” **CORRECTION: THE BLOCK BELOW IS TRUE ABOUT THE CODE AND FALSE ABOUT
515
+ # PRODUCTION, AND IT MUST BE READ SECOND.** It claims the missing-argument `TypeError` "IS
516
+ # owner item 17" β€” the owner's *"Something went wrong"*. It was not, and it could not have
517
+ # been: at `cbcf005`, the build the owner was using, the dict-read on `views` sat ~10 lines
518
+ # ABOVE this call and raised `AttributeError` on EVERY request, so the walk never reached the
519
+ # leaf and the arity bug was unreachable. `_view_by_id`'s own header records that fix.
520
+ #
521
+ # ⚠ WHY THE STALE PARAGRAPH STAYS RATHER THAN GETTING DELETED: the arity bug was real, the
522
+ # fix was right, and the three reasons it hid are the most transferable thing in this file.
523
+ # What was wrong is only its CLAIM TO BE THE CAUSE. Two comment blocks in one function each
524
+ # naming themselves as the origin of the same screenshot are mutually exclusive, and the next
525
+ # reader believes whichever they meet first β€” which is why this correction sits above rather
526
+ # than below. Caught by a verifier that read the SHIPPED file at the deployed commit instead
527
+ # of the working tree. [[grep-output-is-not-source]]
528
+ #
529
+ # β›”β›” WAVE 32 Β· T22 β€” **THE CALL BELOW WAS MISSING AN ARGUMENT** (and wave 32 believed, wrongly,
530
+ # that this was owner item 17 β€” see the correction directly above).
531
  #
532
  # `is_rule_active(rule, columns)` takes TWO parameters (`harness/filter_sql.py`; every other
533
  # caller in the repo passes both). This one passed ONE, so the moment the walk reached a LEAF
api/routes_automation.py CHANGED
@@ -12,10 +12,12 @@ rather than admitting everyone. A "no token configured means no check" default i
12
  trigger becomes a public one β€” the same class of mistake as an empty-200 permission answer.
13
  """
14
  import os
 
15
  import time
16
 
17
  from fastapi import APIRouter, Body, Depends, Header, Request
18
 
 
19
  import automation_engine as engine
20
  import oauth_connect
21
  import routes_oauth
@@ -27,11 +29,22 @@ router = APIRouter(prefix="/api/v1")
27
  # C5 (wave 22): the OAuth connector surface rides INSIDE this router β€” main.py belongs to no
28
  # session this wave, and this router is already mounted there. `/api/v1` + `/oauth/...`.
29
  router.include_router(routes_oauth.router)
30
- # ⭐ WAVE 23 (C11): the connectors directory rides this router for the same reason the OAuth
31
- # routes do β€” it is mounted already, so E's page needs no main.py change to reach it. (C still
32
- # owns main.py; this avoids an unnecessary ask across a fence.)
33
- import routes_connectors # noqa: E402
34
- router.include_router(routes_connectors.router)
 
 
 
 
 
 
 
 
 
 
 
35
 
36
  #: The registry key this surface carries (C-AUTONAV β€” A adds the row; the gate is live now, so
37
  #: the day the row lands the wall is already the one that was tested).
@@ -78,6 +91,27 @@ def _wire(defn, tenant):
78
  # C3 (wave 22): the trigger config rides whole β€” the webhook token included, because
79
  # the person configuring the external caller has to be shown the URL somewhere, and
80
  # this payload is session-gated behind the same wall as everything else here.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
81
  "trigger": defn.get("trigger") or None,
82
  "statusNote": defn.get("statusNote") or "",
83
  # The one-sentence summary (airtable-brief rec 7), composed from the definition so it
@@ -110,6 +144,302 @@ def _wire(defn, tenant):
110
  }
111
 
112
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
113
  def _triggers_vocab(session):
114
  """C3's server-owned trigger list: `[{key, label, ready, needs, planned}]`, keys EXACTLY
115
  `engine.TRIGGER_KEYS` + `engine.TRIGGER_PLANNED`. `ready:false` + `needs` renders as a
@@ -241,6 +571,13 @@ def _tick_state():
241
  #: or delete β€” a new database cannot contain the legacy stage cells this retires.
242
  _BOARD_RETIRED = set()
243
 
 
 
 
 
 
 
 
244
  #: ⭐⭐ WAVE 30 Β· T12 β€” THE PICKER DEFAULT, MEMOISED. `{tenant: (stamp, table_key)}`, the shape
245
  #: `scope_cache` stores.
246
  #:
@@ -393,9 +730,31 @@ def list_automations(session: Session = Depends(_GATE)):
393
  # Idempotent Board retirement removes only engine-marked stage fields and legacy Board
394
  # state. User-created Status/Stage columns remain intact.
395
  engine.retire_automation_board_state(session.runtime, tables=tables)
 
 
 
 
 
 
 
 
 
 
 
 
 
 
396
  defs = engine.all_definitions(session.runtime)
397
  items = [_wire(d, session.tenant) for _, d in
398
  sorted(defs.items(), key=lambda kv: (kv[1].get("name") or "").lower())]
 
 
 
 
 
 
 
 
399
  return {"automations": items,
400
  # ⭐ WAVE 24 β€” DERIVED from the engine's own `KINDS`, not a hand-written trio. It was
401
  # three literals that happened to match, which is a second copy of a server
@@ -484,7 +843,11 @@ def list_automations(session: Session = Depends(_GATE)):
484
  # own reason instead of omitting them β€” the owner asked for Airtable's full menu, and
485
  # a shorter list would imply those actions do not exist. `clean_actions` REFUSES an
486
  # unready kind, so the faded state is a wall rather than a styling choice.
487
- "actionsCatalog": engine.action_catalog(),
 
 
 
 
488
  # The builder's own vocabulary: how deep a condition tree may nest, how deep groups
489
  # may nest, and the ceilings. B reads these instead of hard-coding the same numbers
490
  # into its "+ Add condition" affordance.
@@ -505,7 +868,25 @@ def list_automations(session: Session = Depends(_GATE)):
505
  # The ceiling a group's limit is judged against. `clean_post_groups` bounds a
506
  # group by the action's OWN `maxPosts`, not by a constant, so the control can
507
  # only warn honestly if it reads the same number the validator uses.
508
- "maxPostsPerPull": engine.MAX_POSTS_PER_PULL},
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
509
  # Wave 21 C6-A1: {"enabled": bool, "source": "in-process"|"external"|""} β€” see
510
  # `_tick_state` for why one boolean off AIOS_AUTOMATIONS alone would lie.
511
  "tick": _tick_state()}
@@ -714,8 +1095,203 @@ def automation_seed_derive(body: dict = Body(default=None), session: Session = D
714
  "derived": derived, "basis": basis}
715
 
716
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
717
  @router.get("/automations/{auto_id}")
718
  def get_automation(auto_id: str, session: Session = Depends(_GATE)):
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
719
  defn = engine.all_definitions(session.runtime).get(str(auto_id))
720
  if defn is None:
721
  raise err(404, "unknown_automation", "no automation with that id")
@@ -729,33 +1305,84 @@ def create_automation(body: dict = Body(default=None), session: Session = Depend
729
  answered before the kind. `new` mints the blank database in the same call, so the automation
730
  is never saved pointing at a table that does not exist yet."""
731
  if not session.runtime.available():
732
- raise err(503, "store_unavailable", "the tenant store is unavailable β€” nothing was saved")
733
- defn, error = engine.create(session.runtime, body or {}, username=session.uname)
 
 
 
 
 
 
 
 
734
  if error:
735
  raise err(400, "invalid_automation", error)
736
- return {"automation": _wire(defn, session.tenant)}
737
 
738
 
739
  @router.patch("/automations/{auto_id}")
740
  def patch_automation(auto_id: str, body: dict = Body(default=None),
741
  session: Session = Depends(_GATE)):
742
  if not session.runtime.available():
743
- raise err(503, "store_unavailable", "the tenant store is unavailable β€” nothing was saved")
744
- defn, error = engine.patch(session.runtime, auto_id, body or {}, username=session.uname)
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
745
  if error:
746
  raise err(400 if error != "no such automation" else 404,
747
  "invalid_automation" if error != "no such automation" else "unknown_automation",
748
  error)
749
- return {"automation": _wire(defn, session.tenant)}
750
 
751
 
752
  @router.delete("/automations/{auto_id}")
753
  def delete_automation(auto_id: str, session: Session = Depends(_GATE)):
754
  """Delete the DEFINITION. ⚠ The database it filled is NOT touched β€” an automation is the
755
  thing that writes rows, not the thing that owns them, and deleting a job must never be a way
756
- to lose data (the same rule the orphan count encodes)."""
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
757
  if engine.running(session.tenant, auto_id):
758
- raise err(409, "automation_running", "it is running β€” wait for it to finish")
 
 
 
 
 
759
  engine.remove(session.runtime, auto_id)
760
  return {"deleted": str(auto_id)}
761
 
@@ -773,7 +1400,7 @@ def preview_source(body: dict = Body(default=None), session: Session = Depends(_
773
  except engine.Refused as e:
774
  raise err(400, "refused_url", str(e))
775
  except Exception as e: # noqa: BLE001
776
- raise err(502, "fetch_failed", f"could not read that page β€” {type(e).__name__}: "
777
  f"{str(e)[:160]}")
778
 
779
 
@@ -804,9 +1431,9 @@ def toggle_automation_node(auto_id: str, node_id: str, session: Session = Depend
804
  nothing: a control that appears to work and does not is worse than one that refuses.
805
  """
806
  if not session.runtime.available():
807
- raise err(503, "store_unavailable", "the tenant store is unavailable β€” nothing was saved")
808
  if engine.running(session.tenant, auto_id):
809
- raise err(409, "automation_running", "it is running β€” wait for it to finish")
810
  defn, error = engine.toggle_node(session.runtime, auto_id, node_id)
811
  if error:
812
  raise err(404 if error == "no such automation" else 400,
@@ -873,7 +1500,7 @@ def tick(request: Request, x_aios_tick_token: str = Header(default="")):
873
  want = os.environ.get("AIOS_AUTOMATION_TICK_TOKEN") or ""
874
  if not want:
875
  raise err(403, "tick_disabled",
876
- "AIOS_AUTOMATION_TICK_TOKEN is not configured β€” the tick endpoint is closed")
877
  got = x_aios_tick_token or request.headers.get("X-AIOS-TICK-TOKEN") or ""
878
  if got != want:
879
  raise err(403, "bad_tick_token", "that token is not valid for this deployment")
@@ -891,7 +1518,7 @@ def purge_ig_subject(body: dict = Body(default=None), session: Session = Depends
891
  if not session.admin:
892
  raise err(403, "admin_only", "erasing a subject is an admin action")
893
  if not session.runtime.available():
894
- raise err(503, "store_unavailable", "the tenant store is unavailable β€” nothing was "
895
  "purged")
896
  handle = str((body or {}).get("handle") or "").strip()
897
  if not handle:
@@ -930,7 +1557,7 @@ def metric_drill(table_key: str, field_key: str, row_id: str,
930
  "measure": bag.get("measure"), "window": bag.get("window"),
931
  "value": value, "rows": rows[:200],
932
  "note": "" if value is not None else
933
- "no master data answers this window β€” the cell is honestly blank"}
934
 
935
 
936
  @router.get("/automations/{auto_id}/rows")
 
12
  trigger becomes a public one β€” the same class of mistake as an empty-200 permission answer.
13
  """
14
  import os
15
+ import re
16
  import time
17
 
18
  from fastapi import APIRouter, Body, Depends, Header, Request
19
 
20
+ import ai_review
21
  import automation_engine as engine
22
  import oauth_connect
23
  import routes_oauth
 
29
  # C5 (wave 22): the OAuth connector surface rides INSIDE this router β€” main.py belongs to no
30
  # session this wave, and this router is already mounted there. `/api/v1` + `/oauth/...`.
31
  router.include_router(routes_oauth.router)
32
+ # β›”β›” D-203 β€” THE NESTED INCLUDE OF `routes_connectors` IS REMOVED, and the reason it existed is
33
+ # worth keeping because it was a GOOD reason that expired.
34
+ #
35
+ # Wave 23 (C11) mounted the connectors directory here rather than in `main.py`, because this router
36
+ # was already mounted and `main.py` belonged to another session β€” a sensible way to avoid a
37
+ # cross-fence ask. `main.py:227` includes `routes_connectors` DIRECTLY now, so this line stopped
38
+ # being the only door and became a SECOND one.
39
+ # ⚠ AND A ROUTER MOUNTED TWICE DOES NOT SERVE THE SAME PATHS TWICE β€” it serves the prefix twice.
40
+ # `routes_connectors` carries its own `/api/v1`, so nesting it inside this router's `/api/v1`
41
+ # produced **`/api/v1/api/v1/connectors/directory`**: a live, session-gated, entirely dead path
42
+ # that nothing links to and every route audit has to explain. MEASURED before removal: 123 served
43
+ # paths, exactly 1 of them doubled.
44
+ # ⚠ `routes_oauth` above is NOT the same case and stays: `main.py` does not mount it, so this
45
+ # router is genuinely its only door. Deleting it because its neighbour was wrong is how a real
46
+ # route dies for a tidy-up.
47
+ import routes_connectors # noqa: E402,F401
48
 
49
  #: The registry key this surface carries (C-AUTONAV β€” A adds the row; the gate is live now, so
50
  #: the day the row lands the wall is already the one that was tested).
 
91
  # C3 (wave 22): the trigger config rides whole β€” the webhook token included, because
92
  # the person configuring the external caller has to be shown the URL somewhere, and
93
  # this payload is session-gated behind the same wall as everything else here.
94
+ # ⭐⭐ WAVE 35 Β· T36 / R10 β€” THE PARKED BATCH, SUMMARISED. Without this line the whole
95
+ # ticket would be unreachable: `_wire` is a WHITELIST, so a key the engine parks on the
96
+ # definition simply never arrives, and the detail pane could not know a batch was waiting
97
+ # ([[reachable-is-not-the-same-as-built]] β€” the exact shape `flow` was caught in above).
98
+ # ⚠ COUNT AND NOTES ONLY, NEVER `items`. A 200-statement batch carries 200 rendered HTML
99
+ # mails; this payload rides on the automations LIST, which every rail paint reads.
100
+ # `GET /admin/statements/agent/{id}` serves the detail when somebody opens the batch.
101
+ "pendingStatements": ({
102
+ "count": int((defn.get("pendingStatements") or {}).get("count") or 0),
103
+ "ts": (defn.get("pendingStatements") or {}).get("ts") or "",
104
+ "notes": list((defn.get("pendingStatements") or {}).get("notes") or [])[:25],
105
+ } if isinstance(defn.get("pendingStatements"), dict) else None),
106
+ # ⭐⭐ WAVE 35 Β· T37 β€” `system` FOR A **STORED** ROW. Until now only SYNTHETIC rows carried
107
+ # it, stamped by their builders AFTER this function (`_odoo_sync_row`, `_field_agent_rows`),
108
+ # so a stored row with the marker had it enforced on the server (`delete_automation` 409s)
109
+ # and INVISIBLE to the client β€” which paints a live-looking Delete that answers with a
110
+ # refusal. `AutomationDetail` already reads `automation.system` to disable that button and
111
+ # explain why; this is the line that lets it.
112
+ # ⚠ Absent stays absent rather than becoming `""`: every ordinary automation is not-a-system
113
+ # -agent, and an empty string is a value a client could accidentally treat as one.
114
+ **({"system": str(defn.get("system") or "")} if defn.get("system") else {}),
115
  "trigger": defn.get("trigger") or None,
116
  "statusNote": defn.get("statusNote") or "",
117
  # The one-sentence summary (airtable-brief rec 7), composed from the definition so it
 
144
  }
145
 
146
 
147
+ #: ⭐⭐ WAVE 34 Β· CONTRACTS C3 + C4 β€” WHY A ROW CANNOT BE DELETED, as a slug.
148
+ #:
149
+ #: ⚠ ALWAYS A STRING, NEVER A BOOLEAN, and the two contracts disagreed about that (C3 says
150
+ #: `system: true`, C4 says `system: "<slug>"`). Raised as ask `E-13`; built as a slug because one
151
+ #: key with two types is [[one-question-two-normalizers]] before a line is written, and because a
152
+ #: refusal has to be able to NAME the reason. Truthiness is unchanged for any reader that only
153
+ #: asks "is this a system agent".
154
+ SYSTEM_FIELD_AGENT = "field_agent"
155
+ SYSTEM_ODOO_SYNC = "odoo_sync"
156
+
157
+
158
+ def _field_agent_rows(session):
159
+ """⭐⭐ CONTRACT C3 β€” one synthetic Agent row per `ai_enrich` column in the tenant.
160
+
161
+ R13: *"Every field agent must be VIEWABLE under the Agent module as a simple Trigger whose
162
+ first step is the field enrichment, with the Field and Database shown under Config."*
163
+
164
+ β›”β›” THESE ARE DERIVED, NOT STORED, AND NOTHING HERE MAY WRITE. A field agent's only source of
165
+ truth is the column definition F's `_clean_field` stamps; a copy in the automations bucket
166
+ would be a second one, and the first PATCH through the enrichment editor would silently
167
+ diverge from it. `delete_automation` refuses these ids for the same reason: accepting a delete
168
+ that cannot delete anything is the `view_upsert` failure mode (200 OK, nothing written) in a
169
+ new place.
170
+
171
+ β›”β›” AND IT USES THE ROWS-FREE PROJECTION, WHICH IS THE WHOLE ENGINEERING PROBLEM OF THIS
172
+ TICKET. `GET /automations` is the route W29-T01 and W30-T12 spent two waves taking the
173
+ whole-document read OFF: it reads `user_tables` exactly ONCE PER TENANT PER PROCESS now, and
174
+ this file's own `unconfigured` note says *"a label is not worth putting it back"*. Walking
175
+ every table's fields for `ai_enrich` columns is exactly that read. So this uses
176
+ `user_tables.all_defs` β€” the same projection `/nav` uses, MEASURED there at 1,750 ms -> 1.4 ms,
177
+ because tenant #0's document is 99.89% rows and a field definition is not a row.
178
+ ⚠ `all_defs` RAISES on `rows` rather than answering `{}`. Nothing here wants a row; if that
179
+ ever changes, the error names the projection instead of painting an empty grid.
180
+
181
+ ⚠ THE PERMISSION WALL IS CALLED, NEVER RE-IMPLEMENTED. `may_open` decides which databases this
182
+ caller may see, lent the projected document exactly as `/nav` lends it. This route's own
183
+ `automation_tables` carries the scar that makes that non-negotiable: it once re-implemented
184
+ the wall, got it WIDER, and showed people databases they were refused the moment they clicked.
185
+ """
186
+ import core.user_tables as user_tables
187
+
188
+ try:
189
+ defs = user_tables.all_defs(st=session.runtime) or {}
190
+ lent = user_tables.lend(session.runtime, **{user_tables.STORE_KEY: defs})
191
+ except Exception: # noqa: BLE001
192
+ # A store blip must not take the whole rail down with it. The stored automations are the
193
+ # payload's real subject; field agents are an addition to it.
194
+ return []
195
+
196
+ out = []
197
+ for key, defn in sorted(defs.items(), key=lambda kv: (kv[1].get("label") or "").lower()):
198
+ if not user_tables.may_open(key, session.uname, session.admin, st=lent):
199
+ continue
200
+ table_label = defn.get("label") or key
201
+ for f in user_tables.ai_enrich_fields(defn):
202
+ bag = f.get("automation") if isinstance(f.get("automation"), dict) else {}
203
+ if bag.get("kind") != SYSTEM_FIELD_AGENT:
204
+ continue
205
+ col = str(f.get("key") or "")
206
+ if not col:
207
+ continue
208
+ # ⚠ THE ID CARRIES THE TABLE AND THE COLUMN, and `table` is NOT read off the bag:
209
+ # F's `_clean_field` is handed ONE field dict with no table context, so it cannot
210
+ # stamp one (mailbox `F-3`). E holds that key because E is the side that iterates
211
+ # tables. Two sources for one fact was the alternative, and it drifts.
212
+ trig = bag.get("trigger") if isinstance(bag.get("trigger"), dict) else {}
213
+ mode = str(trig.get("mode") or "manual")
214
+ cron = str(trig.get("cron") or "").strip()
215
+ defn_syn = {
216
+ "id": f"field:{key}:{col}",
217
+ "name": f.get("label") or col,
218
+ "kind": SYSTEM_FIELD_AGENT,
219
+ # β›”β›” TWO TRIGGER VOCABULARIES MEET HERE AND THEY ARE NOT THE SAME ONE. A field
220
+ # agent's trigger is the ENRICHMENT vocabulary (`{mode, cron}`, values
221
+ # manual/on_change/schedule); an automation's is the ENGINE's (`{key}` plus a
222
+ # separate `schedule {cron, enabled}`). Handing the engine's shape a `mode` it
223
+ # cannot read is not a type error β€” it is SILENT: `compose_sentence` fell back to
224
+ # "When you press Run now" for a column scheduled at 06:00, and `graph()` drew a
225
+ # trigger node with no trigger. Measured before this mapping existed, which is the
226
+ # only reason it does.
227
+ # ⚠ THE MAP IS EXACT WHERE AN EQUIVALENT EXISTS AND HONEST WHERE IT DOES NOT:
228
+ # `manual` and `schedule` are the engine's own keys, and `on_change` becomes
229
+ # `event_field` ("When a record matches conditions"), the nearest thing the engine
230
+ # has to "a cell this column reads has moved". `mode` is kept verbatim beside it so
231
+ # nothing is lost in translation and the enrichment editor stays the source.
232
+ "trigger": {"key": {"manual": "manual", "schedule": "schedule",
233
+ "on_change": "event_field"}.get(mode, "manual"),
234
+ "mode": mode, **({"cron": cron} if cron else {})},
235
+ **({"schedule": {"cron": cron, "enabled": True}}
236
+ if mode == "schedule" and cron else {}),
237
+ "flow": {"actions": [{"id": "act_1", "kind": "ai_enrich",
238
+ "config": {"table": key, "field": col,
239
+ "tableLabel": table_label}}]},
240
+ }
241
+ # ⭐ THROUGH `_wire`, NOT BESIDE IT. A synthetic row hand-built to "look like" a
242
+ # stored one is a second shape that agrees until somebody adds a key to the real one
243
+ # β€” the exact way `awaitingResults` shipped inert for a whole wave. Passing the
244
+ # synthetic DEFINITION through the same function makes them identical by
245
+ # construction; only `system` is stamped afterwards, because no stored row has it.
246
+ row = _wire(defn_syn, session.tenant)
247
+ row["system"] = SYSTEM_FIELD_AGENT
248
+ out.append(row)
249
+ return out
250
+
251
+
252
+ #: ⭐⭐ WAVE 34 Β· W34-T47 (mailbox D-5) β€” R6 APPLIED TO WORDS A MODEL WROTE.
253
+ #:
254
+ #: β›” `web_prose` READS FILES, so it is structurally blind to a dash that arrives at RUN TIME.
255
+ #: `POST /automations/draft` returns three strings a language model authored (the flow's name,
256
+ #: each step's `why`, and the dropped/notes sentences) and `W34-T42` is what put them on a screen.
257
+ #: Lane D MEASURED that a prompt instruction does not hold: their system prompt ends "Never use an
258
+ #: em dash" and the very next live cerebras turn came back with one. So this is code at the
259
+ #: boundary, not a better prompt.
260
+ #: ⚠ A DIGIT RANGE IS A DIFFERENT SENTENCE and gets the first rule: "10-20" means "10 to 20", and
261
+ #: turning it into "10, 20" states two numbers where the model stated a span.
262
+ _DRAFT_DASH = "[" + chr(0x2014) + chr(0x2013) + "]"
263
+
264
+
265
+ def _draft_no_dashes(text):
266
+ """Model prose with no em or en dash, and no meaning changed on the way."""
267
+ text = str(text or "")
268
+ text = re.sub(rf"(?<=\d)\s*{_DRAFT_DASH}\s*(?=\d)", " to ", text)
269
+ text = re.sub(rf"\s*{_DRAFT_DASH}\s*(?=[,.;:!?])", "", text)
270
+ text = re.sub(rf"(?<=[,;:])\s*{_DRAFT_DASH}\s*", " ", text)
271
+ return re.sub(rf"\s*{_DRAFT_DASH}\s*", ", ", text)
272
+
273
+
274
+ ODOO_SYNC_ID = "system:odoo_sync"
275
+ #: The cadence presets in the words a person reads. ⚠ Keyed on `odoo_relational.SYNC_PRESETS`, and
276
+ #: a key with no phrase here falls back to "Every <key>" rather than raising β€” a missing caption
277
+ #: must degrade to something readable, not take the agents list down.
278
+ ODOO_CADENCE_PHRASE = {"30m": "Every 30 minutes", "1h": "Every hour",
279
+ "4h": "Every 4 hours", "daily": "Every day"}
280
+
281
+
282
+ def _odoo_sync_row(session, detail=False):
283
+ """⭐⭐ CONTRACT C4 (ruling R22) β€” the Odoo sync, as an agent you can SEE.
284
+
285
+ R22: *"Implicit time-triggered work (Odoo syncing) becomes EXPLICIT: a pre-set agent that
286
+ cannot be deleted, whose Canvas shows Trigger = time and Action = data syncing from Odoo, with
287
+ the keychain/API configuration shown."*
288
+
289
+ β›”β›” DERIVED, NOT STORED, AND C4 SAYS "a stored automation may carry `system:`". THE MECHANISM
290
+ C4 ASKS FOR IS BUILT (`delete_automation` refuses any stored row carrying `system`); this
291
+ particular agent does not use it, for three measured reasons, and the deviation is raised as
292
+ ask `E-15` rather than taken quietly:
293
+ 1. NOTHING CAN SEED IT. There is no system-automation seeding path anywhere, and `D-195`
294
+ measured three times that a CLI write to the tenant store while the Space is live REPORTS
295
+ SUCCESS and is reverted within a minute. A stored row would have to be minted by the
296
+ container, i.e. a write-on-read on the busiest list route in the product.
297
+ 2. STORED MEANS RUNNABLE BY MACHINERY BUILT FOR USER AUTOMATIONS. If its runs went through
298
+ `_commit_run`, `CONSECUTIVE_FAILURE_PAUSE` would flip `schedule.enabled` off after K bad
299
+ passes β€” silently disabling the tenant's REAL Odoo cadence, not a cosmetic card.
300
+ 3. TWO COPIES OF ONE CADENCE. The loop reads `odoo_relational.sync_seconds`; a stored
301
+ automation would carry its own `schedule.cron`, and C4's own requirement is that the card
302
+ "cannot drift from the running loop". Deriving makes drift impossible instead of
303
+ forbidden.
304
+
305
+ ⚠ `detail=False` IS THE LIST PATH AND IT STAYS CHEAP. The keychain label and the last-sync
306
+ stamp each cost their own store read, and `GET /automations` is the route two waves were spent
307
+ taking reads OFF. They are resolved only on the DETAIL fetch, which is one automation and one
308
+ click. Returns `None` when this tenant has no Odoo at all β€” an agent for a connector nobody
309
+ connected is a card that lies.
310
+ """
311
+ try:
312
+ import odoo_relational as rel
313
+ except Exception: # noqa: BLE001
314
+ return None
315
+ try:
316
+ cfg = rel.read_config(session.runtime)
317
+ secs = rel.sync_seconds(session.runtime)
318
+ frozen = bool(rel.frozen(session.runtime))
319
+ except Exception: # noqa: BLE001
320
+ return None
321
+
322
+ every = str(cfg.get("syncEvery") or rel.DEFAULT_SYNC)
323
+ manual = secs is None
324
+ defn = {
325
+ "id": ODOO_SYNC_ID,
326
+ "name": "Odoo sync",
327
+ "kind": SYSTEM_ODOO_SYNC,
328
+ # ⚠ THE CADENCE IS READ, NEVER STORED HERE. `everySeconds` is what the loop will actually
329
+ # sleep for, so the card cannot claim a schedule the loop is not keeping.
330
+ "trigger": {"key": "manual" if manual else "schedule", "preset": every,
331
+ "everySeconds": secs, "presets": sorted(rel.SYNC_PRESETS)},
332
+ **({} if manual else {"schedule": {"cron": "", "enabled": True}}),
333
+ "flow": {"actions": [{"id": "act_1", "kind": "odoo_sync",
334
+ "config": {"connector": "odoo", "every": every,
335
+ "frozen": frozen}}]},
336
+ }
337
+ row = _wire(defn, session.tenant)
338
+ row["system"] = SYSTEM_ODOO_SYNC
339
+ # β›” THE SENTENCE IS OVERRIDDEN, AND IT IS A FIX RATHER THAN A PREFERENCE. `compose_sentence`
340
+ # speaks the automation vocabulary β€” it builds "When <cron phrase>, run N actions" out of
341
+ # `schedule.cron` β€” and this agent has no cron: its cadence is a PRESET in seconds that the
342
+ # resync loop sleeps on. MEASURED before this line existed, the card read exactly
343
+ # `", run 1 action."`: a leading comma where the trigger phrase should have been. Two
344
+ # vocabularies again, and this one fails in punctuation rather than in behaviour, which is
345
+ # why only reading the output catches it.
346
+ # ⚠ AND THE CADENCE IS SPELLED OUT RATHER THAN INTERPOLATED. `f"Every {every}"` produced
347
+ # "Every daily, sync data from Odoo." β€” the preset KEYS are storage tokens ("30m", "daily"),
348
+ # not English, and a sentence built by pasting one in is only accidentally readable for the
349
+ # three that happen to be durations. Caught by reading the output, not by any assertion.
350
+ row["sentence"] = ("Odoo is disconnected, so nothing syncs." if frozen else
351
+ "Manually, sync data from Odoo." if manual else
352
+ f"{ODOO_CADENCE_PHRASE.get(every, f'Every {every}')}, sync data from Odoo.")
353
+ # β›” R6's SECOND SENTENCE, WHICH IS THE HALF THAT GETS DROPPED: a limit that cannot be removed
354
+ # must be REPORTED with its cause. Two are reported here rather than left for somebody to
355
+ # discover by watching a mirror not move.
356
+ notes = []
357
+ if frozen:
358
+ notes.append("Odoo is disconnected, so this agent is not syncing. Reconnect it in "
359
+ "Connectors.")
360
+ if manual:
361
+ notes.append("The cadence is set to manual, so nothing syncs on a timer.")
362
+ row["statusNote"] = " ".join(notes)
363
+ if not detail:
364
+ return row
365
+
366
+ # ── the DETAIL half: two store reads, on a route that fetches one automation ──────────────
367
+ try:
368
+ import routes_keychain as kc_routes # noqa: PLC0415
369
+ row["flow"]["actions"][0]["config"]["keychain"] = (
370
+ kc_routes.odoo_config(session).get("label") or "Odoo")
371
+ except Exception: # noqa: BLE001
372
+ # A label is a nicety; failing to read one must not make the agent unopenable.
373
+ row["flow"]["actions"][0]["config"]["keychain"] = "Odoo"
374
+ try:
375
+ from harness import datastore as ds # noqa: PLC0415
376
+ # ⭐ THE LAST RUN IS THE MIRROR'S OWN `_sync_state`, not a run history we would have to
377
+ # write. That table already carries a real per-entity `updated` stamp, so the card reports
378
+ # what actually happened rather than what this module remembers happening.
379
+ st = ds.status() or {}
380
+ stamps = sorted(str((v or {}).get("updated") or "") for v in st.values() if v)
381
+ last = [s for s in stamps if s]
382
+ if last:
383
+ row["status"] = {**(row.get("status") or {}), "state": "ok", "lastRunAt": last[-1],
384
+ "lastSummary": f"{len(last)} datasets synced"}
385
+ except Exception: # noqa: BLE001
386
+ pass
387
+ return row
388
+
389
+
390
+ def _patch_odoo_sync(session, body):
391
+ """⭐⭐ CONTRACT C4 β€” THE ONE THING ON THIS CARD THAT IS EDITABLE: how often it runs.
392
+
393
+ C4: *"Its Canvas is READ-ONLY except `trigger.config.everySeconds`, which writes through to the
394
+ same value `odoo_relational.sync_seconds` reads, so the card cannot drift from the running
395
+ loop."*
396
+
397
+ β›” IT WRITES THE CONNECTOR'S OWN CONFIG KEY, NOT AN AUTOMATION. `rel.CONFIG_KEY` /
398
+ `rel.SYNC_PRESETS` are the same key and the same vocabulary `sync_seconds` reads and
399
+ `_store_resync_loop` sleeps on, so there is one value and the card reads back exactly what the
400
+ loop will use. Storing a cron on a synthetic automation would have been a second copy with a
401
+ guaranteed drift date.
402
+
403
+ ⚠ EVERY OTHER FIELD IS REFUSED RATHER THAN IGNORED. A PATCH that silently kept only the part
404
+ it liked would let somebody rename this agent, watch the 200, and find the name gone on reload
405
+ β€” the failure shape this module has already paid for twice.
406
+
407
+ ⚠ AND THE LATENCY IS REPORTED, because it is real and a person would otherwise call it a bug:
408
+ the loop re-reads the cadence at the TOP of its cycle and then sleeps, so a change takes effect
409
+ on the NEXT pass β€” up to one full OLD interval away (24 hours if it was on `daily`).
410
+ """
411
+ import odoo_relational as rel # noqa: PLC0415
412
+
413
+ known = {"trigger", "every", "syncEvery"}
414
+ extra = sorted(k for k in body if k not in known)
415
+ if extra:
416
+ raise err(409, "system_agent",
417
+ "this agent is the Odoo connection's own sync. Only how often it runs can be "
418
+ f"changed here; {', '.join(extra)} belongs to the connector in Connectors")
419
+ every = str(body.get("syncEvery") or body.get("every")
420
+ or ((body.get("trigger") or {}) if isinstance(body.get("trigger"), dict) else {})
421
+ .get("preset") or "").strip().lower()
422
+ if every not in rel.SYNC_PRESETS:
423
+ raise err(400, "bad_cadence",
424
+ f"pick one of: {', '.join(sorted(rel.SYNC_PRESETS))}")
425
+
426
+ def _up(cur):
427
+ cur = dict(cur) if isinstance(cur, dict) else {}
428
+ cur["syncEvery"] = every
429
+ return cur
430
+
431
+ session.runtime.update(rel.CONFIG_KEY, _up, flush="sync")
432
+ row = _odoo_sync_row(session, detail=True)
433
+ if row is None:
434
+ raise err(404, "unknown_automation", "no automation with that id")
435
+ secs = row["trigger"].get("everySeconds")
436
+ note = ("It syncs on a timer again from the next cycle." if secs
437
+ else "Nothing will sync on a timer now.")
438
+ return {"automation": row,
439
+ "notes": [f"The Odoo sync is set to {every}. {note} A change takes effect on the "
440
+ f"loop's next pass, so it can be up to one of the OLD intervals away."]}
441
+
442
+
443
  def _triggers_vocab(session):
444
  """C3's server-owned trigger list: `[{key, label, ready, needs, planned}]`, keys EXACTLY
445
  `engine.TRIGGER_KEYS` + `engine.TRIGGER_PLANNED`. `ready:false` + `needs` renders as a
 
571
  #: or delete β€” a new database cannot contain the legacy stage cells this retires.
572
  _BOARD_RETIRED = set()
573
 
574
+ #: ⭐ WAVE 35 Β· T37 β€” tenants whose statements agent has been considered THIS PROCESS. Same shape as
575
+ #: `_BOARD_RETIRED` and, like it, purely a cost saver: the DURABLE idempotency is the row's own
576
+ #: existence, which `engine.seed_statements_agent` re-checks inside its own store mutation.
577
+ #: ⚠ SO A RESTART RE-CHECKING IS HARMLESS BY CONSTRUCTION, which is the property to preserve. If
578
+ #: this set were ever the only guard, a process restart would mint a second agent.
579
+ _STATEMENTS_SEEDED = set()
580
+
581
  #: ⭐⭐ WAVE 30 Β· T12 β€” THE PICKER DEFAULT, MEMOISED. `{tenant: (stamp, table_key)}`, the shape
582
  #: `scope_cache` stores.
583
  #:
 
730
  # Idempotent Board retirement removes only engine-marked stage fields and legacy Board
731
  # state. User-created Status/Stage columns remain intact.
732
  engine.retire_automation_board_state(session.runtime, tables=tables)
733
+ # ⭐⭐ WAVE 35 Β· T37 / OWNER RULING R10 β€” ROYAL IMPORTS' STATEMENTS AGENT EXISTS BEFORE ANYBODY
734
+ # ASKS FOR IT. It is minted ONCE, by the container, SWITCHED OFF; every subsequent call finds
735
+ # it present and returns immediately. See `engine.seed_statements_agent` for why this one is
736
+ # STORED where wave 34's Odoo agent is derived, and why the row's existence is a sufficient
737
+ # idempotency key (it cannot be deleted).
738
+ # ⚠ GUARDED SO IT CAN NEVER TAKE THE RAIL DOWN. A tenant with no Odoo, a store mid-outage or a
739
+ # validator change must degrade to "no statements agent", never to a 500 on the one route the
740
+ # whole Agents surface polls. The same posture `_odoo_sync_row` takes for the same reason.
741
+ if session.tenant not in _STATEMENTS_SEEDED:
742
+ try:
743
+ engine.seed_statements_agent(session.runtime, username=session.uname or "system")
744
+ except Exception: # noqa: BLE001
745
+ pass
746
+ _STATEMENTS_SEEDED.add(session.tenant)
747
  defs = engine.all_definitions(session.runtime)
748
  items = [_wire(d, session.tenant) for _, d in
749
  sorted(defs.items(), key=lambda kv: (kv[1].get("name") or "").lower())]
750
+ # ⭐⭐ WAVE 34 Β· CONTRACT C3 (W34-T48) β€” field agents join the list as SYNTHETIC rows.
751
+ # ⚠ MERGED AND RE-SORTED, not appended in a block at the end. R13 asks for a field agent to be
752
+ # "VIEWABLE under the Agent module", i.e. one list of agents, not a list with a second list
753
+ # stapled to it β€” and a rail that sorts by name everywhere except its last few rows reads as a
754
+ # rendering bug. `_field_agent_rows` costs no whole-document read; see its docstring.
755
+ _sys_rows = [r for r in (_odoo_sync_row(session),) if r]
756
+ items = sorted(items + _field_agent_rows(session) + _sys_rows,
757
+ key=lambda r: str(r.get("name") or "").lower())
758
  return {"automations": items,
759
  # ⭐ WAVE 24 β€” DERIVED from the engine's own `KINDS`, not a hand-written trio. It was
760
  # three literals that happened to match, which is a second copy of a server
 
843
  # own reason instead of omitting them β€” the owner asked for Airtable's full menu, and
844
  # a shorter list would imply those actions do not exist. `clean_actions` REFUSES an
845
  # unready kind, so the faded state is a wall rather than a styling choice.
846
+ # ⭐ W35-T35 (C8): `session.runtime` is what withholds a TENANT-GATED row. Passing it
847
+ # here is the whole of "the menu does not offer Send statements to another tenant";
848
+ # the STORE-side wall is `clean_actions(rt=)`, deliberately separate, because a picker
849
+ # is not a security boundary [[opening-a-route-widens-every-field]].
850
+ "actionsCatalog": engine.action_catalog(session.runtime),
851
  # The builder's own vocabulary: how deep a condition tree may nest, how deep groups
852
  # may nest, and the ceilings. B reads these instead of hard-coding the same numbers
853
  # into its "+ Add condition" affordance.
 
868
  # The ceiling a group's limit is judged against. `clean_post_groups` bounds a
869
  # group by the action's OWN `maxPosts`, not by a constant, so the control can
870
  # only warn honestly if it reads the same number the validator uses.
871
+ "maxPostsPerPull": engine.MAX_POSTS_PER_PULL,
872
+ # ⭐⭐ W33-T52 (owner item 16) β€” WHICH CONFIG KEYS A KIND REQUIRES, on the
873
+ # wire, for exactly the reason `postTypes` above is: the panel paints a red
874
+ # `*` beside a required control, and a client-side table of which keys those
875
+ # are would be a SECOND copy of `ACTION_REQUIRED` living in another file. The
876
+ # two would agree on the day they were written and diverge the first time a
877
+ # kind gained a key β€” the panel would then mark a control optional that the
878
+ # runner blocks on, and the person would read "this is fine" from the one
879
+ # surface that is meant to tell them it is not.
880
+ # β›” BOTH HALVES RIDE, phrase AND key, because the phrase is the ONLY human
881
+ # wording of that requirement anywhere ("a column to write into"), and the
882
+ # runner's own refusal sentence is built from it. A client that re-worded it
883
+ # would give the same requirement two names.
884
+ # ⚠ It is `ACTION_REQUIRED`, not `WEB_REQUIRED`: the table is the one the
885
+ # run refusal and `unconfigured_actions` already read, so a kind added to it
886
+ # later paints its `*` with no client change at all.
887
+ "actionRequired": {kind: [{"phrase": phrase, "key": key}
888
+ for phrase, key in reqs]
889
+ for kind, reqs in engine.ACTION_REQUIRED.items()}},
890
  # Wave 21 C6-A1: {"enabled": bool, "source": "in-process"|"external"|""} β€” see
891
  # `_tick_state` for why one boolean off AIOS_AUTOMATIONS alone would lie.
892
  "tick": _tick_state()}
 
1095
  "derived": derived, "basis": basis}
1096
 
1097
 
1098
+ #: The injected chat transport, for gates only β€” `None` in every shipped path, so the route takes
1099
+ #: the real ladder. Set by `verify_automation.py` to prove this door end to end with NO API key and
1100
+ #: NO spend (`routes_query._call_model`'s `chat` argument is the same idea and the same reason).
1101
+ _DRAFT_CHAT = [None]
1102
+
1103
+
1104
+ @router.post("/automations/draft")
1105
+ def draft_automation(body: dict = Body(default=None), session: Session = Depends(_GATE)):
1106
+ """⭐⭐ WAVE 33 Β· W33-T54/T55 (owner item 7, ruling R3) β€” A DESCRIPTION BECOMES A DRAFT FLOW.
1107
+
1108
+ `{prompt}` β†’ `{draft: {name, trigger, table, actions:[{id,kind,config,why}]}, provider,
1109
+ dropped: [...], saved: false}` β€” or a 400 carrying ONE plain sentence.
1110
+
1111
+ β›”β›” NOTHING IS SAVED HERE, AND THE `saved: false` ON THE WIRE IS NOT DECORATION. T54's
1112
+ `done-when` is *"gets back a DRAFT flow they can see BEFORE anything is saved"*; a door that
1113
+ wrote first and showed second would satisfy every check about the flow's contents and none about
1114
+ the promise. Accepting a draft is the ORDINARY `POST /automations` β€” R3's *"indistinguishable
1115
+ from a hand-built one"* is achieved by there being no second write path, not by making the
1116
+ second one look similar.
1117
+
1118
+ β›”β›” CONTRACT C7, AND THIS ROUTE IS WHERE IT IS ENFORCED: *"the output must pass `clean_actions`
1119
+ unchanged."* So it is RUN, here, before the person ever sees the draft β€” and the result is
1120
+ DIFFED against what the model wrote. `clean_actions` has no disclosure channel (D-75): it drops
1121
+ a config key it does not recognise and answers 200. Without this diff a person would accept a
1122
+ flow, open it, and find a step configured differently from the one they were shown, with nothing
1123
+ anywhere having said so. `dropped` is that channel, built here because this is the first caller
1124
+ that needed one.
1125
+ ⚠ A draft whose actions `clean_actions` REFUSES outright is a 400, never a partial flow: half a
1126
+ flow presented as a whole one is the one outcome worse than a refusal.
1127
+
1128
+ ⚠ DECLARED ABOVE `/automations/{auto_id}`, and that is load-bearing rather than tidy β€” FastAPI
1129
+ matches in DECLARATION order, so a literal path registered after a sibling path-parameter route
1130
+ is never reached, and `get_automation` would answer *"no automation with that id"* for the id
1131
+ `draft`. The same reason `/automations/tables` and `/automations/presets` sit up here.
1132
+ """
1133
+ prompt = str((body or {}).get("prompt") or "").strip()
1134
+ if not prompt:
1135
+ raise err(400, "no_prompt", "type what you want the automation to do")
1136
+ # ⚠ THE TENANT'S OWN TABLES, THROUGH THE EXISTING WALL. `automation_tables` applies `may_open`
1137
+ # per table, so the model is shown exactly the databases this caller may already see and cannot
1138
+ # name one they were not granted β€” the permission wall re-used, never a second one built beside
1139
+ # it. (`QueryPage`'s `granted` prop carries the same clause on the client side.)
1140
+ tables = (automation_tables(session) or {}).get("tables") or []
1141
+ # ⭐⭐ WAVE 34 Β· W34-T42 / R18 β€” THE DRAFTER IS OFFERED WHAT THE PICKER OFFERS, nothing more.
1142
+ #
1143
+ # R18 collapsed six web actions into one on the menu. The other five stay in the catalog
1144
+ # because the surviving "Web agent" composes its own steps out of them server-side β€” but a
1145
+ # DRAFT is read and accepted by a PERSON, who then edits it on the Canvas. Handing the model
1146
+ # the unfiltered catalog would let it draft a step kind that is not in the action picker: the
1147
+ # reader could not add a second one, could not reason about where it came from, and the flow
1148
+ # they accepted would not be one they could have built by hand. R3's *"indistinguishable from
1149
+ # a hand-built one"* is a claim about what a person can REACH, not only about the write path.
1150
+ # ⚠ This is the ONE place the filter is applied on this route, and it is applied to the
1151
+ # drafter's vocabulary only. `clean_actions` below still validates against the FULL catalog,
1152
+ # so a flow that already holds a hidden kind is unaffected and nothing 400s (D-65).
1153
+ # ⚠ AND `required` IS NARROWED WITH IT, which is the half that is easy to miss. `draft_flow`
1154
+ # builds the ENUM from `catalog` and the "settings you must write" prose from `required`, so
1155
+ # filtering one and not the other describes settings for kinds the model cannot choose. That is
1156
+ # not merely untidy: it spends prompt on unreachable options and invites the model to reach for
1157
+ # one. `_ai_agent_plan` already narrows both in exactly this way; this follows it.
1158
+ # W35-T35 (C8): tenant-gated rows are withheld here too, or the DRAFTER would offer a kind
1159
+ # the same tenant's own save door then refuses β€” a flow the assistant proposes and the product
1160
+ # will not accept.
1161
+ menu_catalog = [r for r in engine.action_catalog(session.runtime) if r.get("menu") is not False]
1162
+ _offered = {r["kind"] for r in menu_catalog}
1163
+ draft, refusal, provider = ai_review.draft_flow(
1164
+ prompt=prompt,
1165
+ catalog=menu_catalog,
1166
+ required={k: v for k, v in engine.ACTION_REQUIRED.items() if k in _offered},
1167
+ triggers=_triggers_vocab(session),
1168
+ tables=tables,
1169
+ # ⭐ W35 Β· CONTRACT C7 (`NOTE E-16`) β€” the spend is ATTRIBUTED. Unlike the two engine call
1170
+ # sites, this one has a real person behind it: somebody typed the sentence, so `user` is
1171
+ # the caller rather than the automation's owner.
1172
+ st=session.runtime, user=getattr(session, "uname", "") or "",
1173
+ chat=_DRAFT_CHAT[0])
1174
+ if refusal or not draft:
1175
+ raise err(400, "draft_refused", refusal or "no automation could be drafted from that")
1176
+
1177
+ # ── C7: run the real save-door validator and DIFF it ──────────────────────────────────
1178
+ wrote = draft.get("actions") or []
1179
+ # W35-T35 (C8): the SAME `rt` the menu was built with. Without it this validator would refuse
1180
+ # a tenant-gated kind it had just offered the model β€” the draft door disagreeing with the save
1181
+ # door about one tenant, which reads as "the assistant produced an invalid flow".
1182
+ cleaned, why = engine.clean_actions([{"kind": a.get("kind"), "config": a.get("config") or {}}
1183
+ for a in wrote], rt=session.runtime)
1184
+ if why or cleaned is None:
1185
+ raise err(400, "draft_invalid",
1186
+ f"the assistant produced a flow this deployment will not accept: {why}")
1187
+ dropped = []
1188
+ for before, after in zip(wrote, cleaned):
1189
+ b_cfg = before.get("config") or {}
1190
+ a_cfg = after.get("config") or {}
1191
+ gone = sorted(k for k in b_cfg
1192
+ if k not in a_cfg or str(a_cfg.get(k)) != str(b_cfg.get(k)))
1193
+ if gone:
1194
+ dropped.append({"kind": str(after.get("kind") or ""), "keys": gone})
1195
+ # β›”β›” WHAT IS STILL MISSING FROM EACH STEP, PER STEP β€” and this is the fix for the sharpest
1196
+ # thing the verifier found. The system prompt TELLS the model *"leave one blank rather than
1197
+ # inventing a web address, a CSS selector or a column name"* (which is right), and the panel
1198
+ # skips empty values (which is also right, on its own). Together they meant a `web_read` with
1199
+ # no selector and no target column painted as a FINISHED step β€” url, attr, timeout, nothing
1200
+ # else β€” and the ordinary create door then accepted it. A person approved a step that reads
1201
+ # nothing into nowhere, having been shown no blank at all.
1202
+ # ⚠ `engine.action_needs` IS THE PREDICATE, not a second list: the same function the builder's
1203
+ # Configured/Unconfigured label and the run refusal read. Three readers, one answer.
1204
+ _tables_by_key = {str(t.get("key")): t for t in tables}
1205
+ _target_cols = {str(f.get("key")) for f
1206
+ in (_tables_by_key.get(str(draft.get("table") or "")) or {}).get("fields") or []}
1207
+ shown = []
1208
+ for i, after in enumerate(cleaned):
1209
+ row = dict(after, why=str((wrote[i] or {}).get("why") or ""),
1210
+ needs=engine.action_needs(after))
1211
+ # β›” AND A COLUMN NAME THE DATABASE DOES NOT HAVE. `field` is a free string that nobody
1212
+ # validated at draft, at accept or at store β€” so a model writing the column's LABEL
1213
+ # ("Price") instead of its key ("price"), which is exactly what a person would say and an
1214
+ # obedient model would echo, stored a write into a column that does not exist. No schema
1215
+ # was violated; the flow was well-formed and did something other than what was asked.
1216
+ _f = str((after.get("config") or {}).get("field") or "")
1217
+ if _f and _target_cols and _f not in _target_cols:
1218
+ row["unknownField"] = _f
1219
+ shown.append(row)
1220
+ # ⚠ A TRIGGER THE CREATE DOOR WOULD REFUSE IS CORRECTED HERE, NOT SHOWN AND THEN 400'd. The
1221
+ # enum makes this need a model that ignores its own schema, but the failure mode is ugly: the
1222
+ # draft paints, the person clicks Accept, and the save refuses with a sentence about a word
1223
+ # they never chose. Falling back to `manual` and SAYING SO keeps the draft usable.
1224
+ _trig = str(draft.get("trigger") or "").strip()
1225
+ _notes = []
1226
+ if _trig and _trig not in tuple(engine.TRIGGER_KEYS):
1227
+ _notes.append(f"the assistant asked for a trigger this deployment does not have "
1228
+ f"({_trig}). Set to manual instead")
1229
+ _trig = ""
1230
+ _asked = int(draft.get("asked") or len(wrote))
1231
+ if _asked > len(shown):
1232
+ _notes.append(f"the assistant wrote {_asked} steps and a draft carries at most "
1233
+ f"{ai_review.MAX_DRAFT_ACTIONS}; the last {_asked - len(shown)} were not "
1234
+ f"kept. Describe the job in two automations, or shorten it.")
1235
+ # ⭐⭐ WAVE 34 Β· W34-T47 (mailbox D-5) β€” THE MODEL'S OWN WORDS ARE NORMALISED, BECAUSE R6
1236
+ # CANNOT BE ENFORCED BY A SOURCE SCAN.
1237
+ #
1238
+ # β›” `web_prose` READS FILES. Every string below is written by a language model at run time,
1239
+ # so the gate is structurally blind to it: the sweep every lane did this wave is undone by our
1240
+ # own drafter the first time it answers with an em dash. Lane D MEASURED that a prompt
1241
+ # instruction does not hold (their system prompt says "Never use an em dash" and the very next
1242
+ # cerebras turn came back with one), so this is code, at the boundary, not a nicer prompt.
1243
+ # ⚠ THREE STRINGS REACH A SCREEN FROM HERE and all three are covered: the flow's NAME, each
1244
+ # step's `why`, and the `dropped`/`notes` sentences. `W34-T42` is what made them visible.
1245
+ # ⚠ A DIGIT RANGE IS A DIFFERENT SENTENCE: "10-20" means "10 to 20", and rewriting it as
1246
+ # "10, 20" states two numbers where the model stated a span. It gets its own rule, first.
1247
+ # Same shape as `routes_query._no_dashes`, deliberately: one behaviour, two doors.
1248
+ # ⚠ MODULE-LEVEL, not a closure. A normaliser defined inside this handler is unreachable to
1249
+ # anything that is not an HTTP request, so the only way to test it would be to drive the whole
1250
+ # route and read the answer β€” which is how a rule ends up asserted by a grep instead of a call.
1251
+ # β›” `dropped` IS NOT NORMALISED AND THAT IS DELIBERATE: it is `[{kind, keys}]`, an action kind
1252
+ # from OUR catalog and config keys from OUR allowlist, never a model's sentence. The first
1253
+ # version of this block mapped the normaliser over it and turned each dict into the string
1254
+ # `"{'kind': ..., 'keys': [...]}"` β€” `verify_automation`'s own W33 section caught it with a
1255
+ # `TypeError` on `d["kind"]`. Applying a text rule to a structure is how a channel stops
1256
+ # carrying what its reader expects, and the reader here is a person's list of what changed.
1257
+ shown = [dict(r, why=_draft_no_dashes(r.get("why"))) for r in shown]
1258
+ _notes = [_draft_no_dashes(n) for n in _notes]
1259
+ return {"draft": {"name": _draft_no_dashes(draft.get("name")) or "New automation",
1260
+ "trigger": _trig or "manual",
1261
+ "table": draft.get("table") or "",
1262
+ "actions": shown},
1263
+ "provider": provider, "dropped": dropped, "notes": _notes, "saved": False}
1264
+
1265
+
1266
  @router.get("/automations/{auto_id}")
1267
  def get_automation(auto_id: str, session: Session = Depends(_GATE)):
1268
+ """One automation, by id.
1269
+
1270
+ β›”β›” WAVE 34 Β· CONTRACT C3 β€” A SYNTHETIC ROW MUST BE FETCHABLE BY ID, NOT ONLY LISTABLE, AND
1271
+ THIS IS THE HALF THAT IS EASY TO SHIP BROKEN. `list_automations` and this route are SEPARATE
1272
+ lookups: a row appended only to the list appears in the rail, looks entirely real, and 404s
1273
+ the instant somebody clicks it. `W34-T48`'s `done-when` is *"opening it shows a Trigger and
1274
+ one step"* β€” i.e. this route, not the list β€” so the field agents are resolved here too, from
1275
+ the same builder, and a person cannot reach a row the detail route does not know.
1276
+ ⚠ THE SAME BUILDER, NOT A SECOND ONE. `_field_agent_rows` is called and the id looked up in
1277
+ its output rather than re-deriving one row from the field: two derivations of one row is how
1278
+ the list and the editor start disagreeing about a name.
1279
+ """
1280
+ if str(auto_id) == ODOO_SYNC_ID:
1281
+ # ⚠ `detail=True` β€” the keychain label and the last-sync stamp cost a store read each and
1282
+ # are resolved HERE, on a route that fetches one automation, never on the list.
1283
+ row = _odoo_sync_row(session, detail=True)
1284
+ if row is None:
1285
+ raise err(404, "unknown_automation", "no automation with that id")
1286
+ return {"automation": row}
1287
+ if str(auto_id).startswith("field:"):
1288
+ row = next((r for r in _field_agent_rows(session) if r.get("id") == str(auto_id)), None)
1289
+ if row is None:
1290
+ # ⚠ The SAME 404 as any other unknown id. A field agent whose column has been deleted
1291
+ # is genuinely gone, and inventing a different error for it would make a normal
1292
+ # outcome look like a fault.
1293
+ raise err(404, "unknown_automation", "no automation with that id")
1294
+ return {"automation": row}
1295
  defn = engine.all_definitions(session.runtime).get(str(auto_id))
1296
  if defn is None:
1297
  raise err(404, "unknown_automation", "no automation with that id")
 
1305
  answered before the kind. `new` mints the blank database in the same call, so the automation
1306
  is never saved pointing at a table that does not exist yet."""
1307
  if not session.runtime.available():
1308
+ raise err(503, "store_unavailable", "the tenant store is unavailable. Nothing was saved")
1309
+ # ⭐⭐ D-75 β€” THE DISCLOSURE CHANNEL, CONSUMED. `clean_actions` silently drops a
1310
+ # `create_record` condition (D-65's law: dropping is recoverable, refusing locks the door) and
1311
+ # any config key an arm's allowlist does not keep. Both are deliberate; what was missing was
1312
+ # anybody being TOLD, so a person saved a flow and got a different one with nothing said.
1313
+ # ⚠ It rides the SAVE's own response, beside `unconfigured`, because that is the moment the
1314
+ # person is looking β€” a note in a log they never open is the same silence with extra steps.
1315
+ _notes = []
1316
+ defn, error = engine.create(session.runtime, body or {}, username=session.uname,
1317
+ notes=_notes)
1318
  if error:
1319
  raise err(400, "invalid_automation", error)
1320
+ return {"automation": _wire(defn, session.tenant), "notes": _notes}
1321
 
1322
 
1323
  @router.patch("/automations/{auto_id}")
1324
  def patch_automation(auto_id: str, body: dict = Body(default=None),
1325
  session: Session = Depends(_GATE)):
1326
  if not session.runtime.available():
1327
+ raise err(503, "store_unavailable", "the tenant store is unavailable. Nothing was saved")
1328
+ # β›”β›” WAVE 34 Β· CONTRACT C3 β€” A SYNTHETIC ROW IS NOT PATCHABLE, AND IT MUST SAY SO. A field
1329
+ # agent is DERIVED from a column definition; there is nothing in the automations bucket to
1330
+ # write. `engine.patch` would answer "no such automation" (a 404 that reads as "your agent
1331
+ # vanished") or, worse for a future id shape, find nothing and report success. The refusal
1332
+ # names where the setting actually lives, because a wall that does not say what to do instead
1333
+ # sends somebody looking for a bug in a decision made on purpose (D-65's own lesson).
1334
+ if str(auto_id).startswith("field:"):
1335
+ raise err(409, "system_agent",
1336
+ "this agent is an AI enrichment column. Change its prompt, model or schedule on "
1337
+ "the column itself and this page follows")
1338
+ if str(auto_id) == ODOO_SYNC_ID:
1339
+ return _patch_odoo_sync(session, body or {})
1340
+ # D-75, same channel as `create` above β€” an EDIT is where this matters most, because the
1341
+ # person has just typed the thing that gets dropped.
1342
+ _notes = []
1343
+ defn, error = engine.patch(session.runtime, auto_id, body or {}, username=session.uname,
1344
+ notes=_notes)
1345
  if error:
1346
  raise err(400 if error != "no such automation" else 404,
1347
  "invalid_automation" if error != "no such automation" else "unknown_automation",
1348
  error)
1349
+ return {"automation": _wire(defn, session.tenant), "notes": _notes}
1350
 
1351
 
1352
  @router.delete("/automations/{auto_id}")
1353
  def delete_automation(auto_id: str, session: Session = Depends(_GATE)):
1354
  """Delete the DEFINITION. ⚠ The database it filled is NOT touched β€” an automation is the
1355
  thing that writes rows, not the thing that owns them, and deleting a job must never be a way
1356
+ to lose data (the same rule the orphan count encodes).
1357
+
1358
+ ⭐⭐ WAVE 34 Β· CONTRACTS C3 + C4 β€” A SYSTEM AGENT REFUSES, LOUDLY, AND THE REFUSAL SAYS WHERE
1359
+ TO GO INSTEAD. Two families are undeletable and they are undeletable for different reasons, so
1360
+ the sentence names the reason rather than saying "no": a FIELD AGENT is a column (delete the
1361
+ column), and the ODOO SYNC is the connector's own schedule (disconnect the connector).
1362
+
1363
+ β›” REFUSED, NOT IGNORED, AND THAT IS THE POINT OF THE FIRST BRANCH. A synthetic id is not in
1364
+ the automations bucket, so `engine.remove` would find nothing, do nothing, and this route
1365
+ would answer `200 {"deleted": ...}` β€” a delete that reports success and changes nothing, which
1366
+ is the `view_upsert` failure mode (200 OK, zero writes) arriving in a new place. The row would
1367
+ then reappear on the next poll and read as a bug in the rail.
1368
+ """
1369
+ # β›”β›” EVERY SYNTHETIC ID, NOT JUST THE FIELD ONES. The first version of this guard listed
1370
+ # `field:` alone, and `verify_automation`'s own C4 leg caught what that left open: deleting
1371
+ # `system:odoo_sync` fell straight through to `engine.remove`, which popped a key that was
1372
+ # never in the bucket and answered `200 {"deleted": ...}`. A delete that reports success and
1373
+ # changes nothing, on the one row the ruling says must be undeletable β€” the exact defect the
1374
+ # guard exists for, one id shape away. Both prefixes are refused by the same branch now.
1375
+ if str(auto_id).startswith("field:"):
1376
+ raise err(409, "system_agent", engine.system_agent_refusal(SYSTEM_FIELD_AGENT))
1377
+ if str(auto_id) == ODOO_SYNC_ID:
1378
+ raise err(409, "system_agent", engine.system_agent_refusal(SYSTEM_ODOO_SYNC))
1379
  if engine.running(session.tenant, auto_id):
1380
+ raise err(409, "automation_running", "it is running. Wait for it to finish")
1381
+ # ⚠ READ BEFORE THE REMOVE, not after: `engine.remove` is the thing being refused.
1382
+ _defn = (engine.all_definitions(session.runtime) or {}).get(str(auto_id)) or {}
1383
+ _sys = str(_defn.get("system") or "").strip()
1384
+ if _sys:
1385
+ raise err(409, "system_agent", engine.system_agent_refusal(_sys))
1386
  engine.remove(session.runtime, auto_id)
1387
  return {"deleted": str(auto_id)}
1388
 
 
1400
  except engine.Refused as e:
1401
  raise err(400, "refused_url", str(e))
1402
  except Exception as e: # noqa: BLE001
1403
+ raise err(502, "fetch_failed", f"could not read that page. {type(e).__name__}: "
1404
  f"{str(e)[:160]}")
1405
 
1406
 
 
1431
  nothing: a control that appears to work and does not is worse than one that refuses.
1432
  """
1433
  if not session.runtime.available():
1434
+ raise err(503, "store_unavailable", "the tenant store is unavailable. Nothing was saved")
1435
  if engine.running(session.tenant, auto_id):
1436
+ raise err(409, "automation_running", "it is running. Wait for it to finish")
1437
  defn, error = engine.toggle_node(session.runtime, auto_id, node_id)
1438
  if error:
1439
  raise err(404 if error == "no such automation" else 400,
 
1500
  want = os.environ.get("AIOS_AUTOMATION_TICK_TOKEN") or ""
1501
  if not want:
1502
  raise err(403, "tick_disabled",
1503
+ "AIOS_AUTOMATION_TICK_TOKEN is not configured. The tick endpoint is closed")
1504
  got = x_aios_tick_token or request.headers.get("X-AIOS-TICK-TOKEN") or ""
1505
  if got != want:
1506
  raise err(403, "bad_tick_token", "that token is not valid for this deployment")
 
1518
  if not session.admin:
1519
  raise err(403, "admin_only", "erasing a subject is an admin action")
1520
  if not session.runtime.available():
1521
+ raise err(503, "store_unavailable", "the tenant store is unavailable. Nothing was "
1522
  "purged")
1523
  handle = str((body or {}).get("handle") or "").strip()
1524
  if not handle:
 
1557
  "measure": bag.get("measure"), "window": bag.get("window"),
1558
  "value": value, "rows": rows[:200],
1559
  "note": "" if value is not None else
1560
+ "no master data answers this window. The cell is honestly blank"}
1561
 
1562
 
1563
  @router.get("/automations/{auto_id}/rows")
api/routes_brand.py ADDED
@@ -0,0 +1,140 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """routes_brand.py β€” THE CONNECTOR BRAND-ASSET DOOR (wave 33, owner item 6 / ruling R4 / contract C6).
2
+
3
+ ⭐ WHAT THE OWNER ASKED FOR, verbatim: *"Do not generate your own logo for Connectors, they look
4
+ really uncanny /browse them the PNG, cut appropriately for our needs, verify visually."* So the marks
5
+ are the VENDORS' OWN artwork, on disk, with their provenance recorded β€” `web/public/brand/` and its
6
+ `SOURCES.md`. This module is the door R4 asks for and nothing else: it maps a brand KEY to a file,
7
+ serves it, and refuses anything else.
8
+
9
+ β›” WHY A NEW DOOR AND NOT `routes_assets.py`. That module gates EVERY one of its routes behind
10
+ `module_gate("product_data")` (`routes_assets.py:45`) because it serves product imagery and the grant
11
+ that owns product imagery is the right wall for it. A connector logo is not product data: an account
12
+ with no product grant still opens Connectors, still opens the automation pickers, and would have got
13
+ a 403 where a picture should be. Contract C6 says session-gated, NOT product-gated, and that is the
14
+ whole reason this file exists rather than one more path over there.
15
+
16
+ ⚠ SESSION-GATED, AND THAT IS AN HONEST WALL RATHER THAN A SECRET. Public brand artwork is public by
17
+ definition, and the same bytes are ALSO reachable through the static bundle β€” `web/public/**` is
18
+ copied into `web/dist/**` by `vite build`, which `main.py` mounts unauthenticated at `/`. So the
19
+ session gate here is consistency with every other `/api/v1` route, not a claim that these files are
20
+ confidential. What the door actually buys, and what a raw static path could not:
21
+
22
+ Β· ONE KEY VOCABULARY. The client asks for `instagram`, the same string `BRAND_OF_CONNECTOR` maps
23
+ to β€” never a filename. The extension is ours to change (a vendor publishing SVG tomorrow moves
24
+ one line here) and no caller learns about it.
25
+ Β· A CLEAN 404 for a key we have no art for, which is what lets the client keep its two-letter tile
26
+ fallback honest instead of painting a broken-image glyph.
27
+ Β· IMMUTABLE CACHING on a response the browser then never re-fetches.
28
+
29
+ β›” THE FALLBACK IS NOT DEAD CODE AND MUST NOT BE "TIDIED". `ConnectorsPage.tsx:204 tileText` draws two
30
+ letters for any connector with no entry in `BRAND_OF_CONNECTOR` β€” which is most of the planned list,
31
+ by design, because the directory deliberately lists things we have not built (R8). A 404 from here is
32
+ the *correct* answer for those rows, not a failure.
33
+ """
34
+ from __future__ import annotations
35
+
36
+ import os
37
+ import re
38
+ from pathlib import Path
39
+
40
+ from fastapi import APIRouter, Depends
41
+ from fastapi.responses import FileResponse
42
+
43
+ from deps import Session, err, require_session
44
+
45
+ router = APIRouter(prefix="/api/v1")
46
+
47
+ #: β›” CLOSED CHARSET, CHECKED BEFORE THE KEY TOUCHES A PATH β€” same discipline as
48
+ #: `routes_assets._CODE_RE`. No dot, no slash, no backslash, no colon, so `..` and every absolute
49
+ #: path are refused at the door (400) rather than resolved and then defended against.
50
+ _KEY_RE = re.compile(r"^[a-z][a-z0-9_]{0,31}$")
51
+
52
+ #: A year, immutable. The artwork is versioned by its own key: a NEW mark for a vendor is a new file
53
+ #: with the same name, so a rebrand ships with a deploy and a hard reload, not with a cache bust.
54
+ #: ⚠ Deliberately the same string as `routes_assets._IMMUTABLE` β€” two asset doors telling a browser
55
+ #: two different things about the same class of file would be a defect nobody could see.
56
+ _IMMUTABLE = "public, max-age=31536000, immutable"
57
+
58
+ _MEDIA = {".png": "image/png", ".svg": "image/svg+xml"}
59
+
60
+ #: ⭐ THE ROOTS, IN ORDER, AND THE ORDER IS THE POINT.
61
+ #:
62
+ #: `web/dist/brand` FIRST because that is what a container has: `deploy_web.py::_web_source_files`
63
+ #: SKIPS `dist`, so the Space uploads `web/public/**` as source and its own `vite build` produces
64
+ #: `dist/`. `web/public/brand` SECOND because that is what a DEV BOX has before anything is built β€”
65
+ #: and a door that only works after `npm run build` is a door that fails for the person writing it.
66
+ #: `AIOS_BRAND_DIR` FIRST OF ALL, so an operator can point at either without a redeploy.
67
+ #:
68
+ #: ⚠ RESOLVED PER REQUEST, NOT AT IMPORT. A module-level constant computed at import is a fact about
69
+ #: the process's startup, and this one would be wrong for the whole life of any container that built
70
+ #: its bundle after the API module was first imported.
71
+ #:
72
+ #: ⚠ THE ONE TRAP THIS ORDER CREATES, AND IT ONLY BITES A DEV BOX. `dist` winning means a mark
73
+ #: replaced in `public/brand/` is NOT served until the next `npm run build` β€” locally the two can
74
+ #: disagree, and the door will confidently serve yesterday's logo. In the image they cannot: the
75
+ #: Space uploads `public/` (`deploy_web.py::_web_source_files` SKIPS `dist`) and builds `dist/` from
76
+ #: it in the same step. If a local mark looks stale, rebuild before believing the door is wrong.
77
+ _HERE = Path(__file__).resolve().parent
78
+
79
+
80
+ def _roots():
81
+ env = (os.environ.get("AIOS_BRAND_DIR") or "").strip()
82
+ out = [Path(env)] if env else []
83
+ web = _HERE.parent / "web" # _HERE is `aios-web/api`; its parent is `aios-web`
84
+ out.append(web / "dist" / "brand")
85
+ out.append(web / "public" / "brand")
86
+ return out
87
+
88
+
89
+ def brand_path(key):
90
+ """The file for a brand key, or None. Pure and importable, so the gate answers this question
91
+ without a request β€” the same shape `routes_connectors.directory` uses for the same reason."""
92
+ k = str(key or "")
93
+ if not _KEY_RE.match(k):
94
+ return None
95
+ for root in _roots():
96
+ for ext in (".png", ".svg"):
97
+ p = root / f"{k}{ext}"
98
+ # ⚠ `is_file()` and not `exists()`: a DIRECTORY named `instagram.png` would otherwise
99
+ # resolve and then fail inside FileResponse as a 500.
100
+ if p.is_file():
101
+ return p
102
+ return None
103
+
104
+
105
+ def available():
106
+ """Every brand key that has artwork on disk, sorted. The directory listing this module can
107
+ answer cheaply β€” used by `verify_meta` to assert parity against `BRAND_OF_CONNECTOR` rather than
108
+ against a list somebody typed twice."""
109
+ keys = set()
110
+ for root in _roots():
111
+ if not root.is_dir():
112
+ continue
113
+ for p in root.iterdir():
114
+ if p.is_file() and p.suffix in _MEDIA and _KEY_RE.match(p.stem):
115
+ keys.add(p.stem)
116
+ return sorted(keys)
117
+
118
+
119
+ @router.get("/brand/{key}")
120
+ def brand_asset(key: str, session: Session = Depends(require_session)):
121
+ """R4/C6: one connector's official mark, to any authenticated session.
122
+
123
+ β›” THREE ANSWERS, AND THE THIRD IS THE ONE THE TICKET ASKS ABOUT: 400 for a key that is not a
124
+ key (it never reaches the filesystem), **404 for a key we simply have no art for**, and the file
125
+ otherwise. A stack trace is not one of the answers β€” an unknown key is an ordinary, expected
126
+ request from a client whose connector list is longer than our logo set.
127
+ ⚠ `session` is required and then unused, which is intentional: the dependency IS the wall, and
128
+ naming it in the signature is what puts it there. Deleting the "unused" parameter would open the
129
+ route to the internet with no other symptom.
130
+ """
131
+ if not _KEY_RE.match(str(key or "")):
132
+ raise err(400, "bad_brand_key", "that is not a brand key")
133
+ p = brand_path(key)
134
+ if p is None:
135
+ raise err(404, "brand_not_found", f"no brand mark for '{key}'")
136
+ return FileResponse(
137
+ str(p),
138
+ media_type=_MEDIA.get(p.suffix, "application/octet-stream"),
139
+ headers={"Cache-Control": _IMMUTABLE},
140
+ )
api/routes_changes.py CHANGED
@@ -55,7 +55,7 @@ So: a valid session, a scope sanitised to a bucket-safe shape, and β€” for a `ut
55
  cached verdict that this session may open it.
56
  """
57
  import time
58
- from fastapi import APIRouter, Depends
59
 
60
  import core.shared_overlay as shared_overlay
61
  import core.store as store
@@ -224,3 +224,87 @@ def changes(scope: str = "", session: Session = Depends(require_session)):
224
  # a token frozen at 0 would promise liveness the backend is not delivering.
225
  tokens[label] = rev.get("token") if isinstance(rev, dict) else None
226
  return {"scope": _clean_scope(scope), "tokens": tokens, "backend": store.backend()}
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
55
  cached verdict that this session may open it.
56
  """
57
  import time
58
+ from fastapi import APIRouter, Depends, Request
59
 
60
  import core.shared_overlay as shared_overlay
61
  import core.store as store
 
224
  # a token frozen at 0 would promise liveness the backend is not delivering.
225
  tokens[label] = rev.get("token") if isinstance(rev, dict) else None
226
  return {"scope": _clean_scope(scope), "tokens": tokens, "backend": store.backend()}
227
+
228
+
229
+ # ── ⭐⭐ D-144 (W33-T32) β€” A CAUGHT RENDER ERROR TELLS SOMEBODY ────────────────────────────────
230
+ #
231
+ # β›” THE DEBT, in its own words: *"A caught render error tells nobody."* W30-T39 gave the app its
232
+ # first error boundary β€” measured at the time, `componentDidCatch|getDerivedStateFromError` was
233
+ # **0 hits** across all of `web/src`, so any throw anywhere blanked the whole product. The
234
+ # boundary CONTAINS the failure; it does not REPORT it, so the next user complaint still arrives
235
+ # as a sentence rather than as a stack. That is why owner item 2's crash could not be diagnosed
236
+ # and had to be fixed at both candidate causes.
237
+ #
238
+ # ⚠ IT LIVES HERE, IN THE CHANGE-TOKEN ROUTER, AND THE REASON IS WORTH ONE LINE. It is not about
239
+ # change tokens. It is here because this router is already mounted, already in one lane's fence,
240
+ # and a NEW `routes_*.py` costs a cross-fence mount (contract C2) that this wave has already spent
241
+ # three times β€” and a fourth finished-but-unmounted router is precisely the failure wave 23 shipped
242
+ # three of. Subject purity would have been bought with the exact risk the contract exists to
243
+ # prevent. ⚠ It performs no `store.get()`, so this file's zero-deep-copy claim (and
244
+ # `verify_live_workspace`'s count of it) is untouched: `telemetry.log` appends one line to a
245
+ # container-local JSONL.
246
+ #
247
+ # β›”β›” AND IT CARRIES NO ROW DATA, WHICH IS THE DEBT'S OWN HARD CONDITION. The body is rebuilt KEY
248
+ # BY KEY from exactly two strings; there is no `**body` anywhere below. A crash reporter that
249
+ # forwarded props or state would quietly ship a customer's cell values into a log β€” the one way
250
+ # this feature could do more harm than the silence it replaces.
251
+ _CLIENT_ERR_MAX = 4 * 1024
252
+ _CLIENT_ERR_WINDOW_S = 60
253
+ _CLIENT_ERR_PER_WINDOW = 12
254
+ _CLIENT_ERR_HITS: dict = {}
255
+
256
+
257
+ @router.post("/client-error")
258
+ async def client_error(request: Request, session: Session = Depends(require_session)):
259
+ """One caught render error, forwarded to telemetry. `{message, componentStack}`.
260
+
261
+ ⚠ SESSION-GATED, deliberately. A public crash sink is an unauthenticated append to a log file,
262
+ which is a disk-filling primitive with a URL. Only signed-in accounts can crash this app in a
263
+ way we need to hear about.
264
+ ⚠ RATE-LIMITED PER ACCOUNT, because the failure mode is a render LOOP: a boundary that catches,
265
+ resets and re-throws would otherwise post several times a second, for ever, from every open
266
+ tab. Beyond the window the report is DROPPED and the request still answers 200 β€” the client
267
+ must never learn to retry a report.
268
+ """
269
+ now = time.time()
270
+ who = session.uname or "?"
271
+ seen = [t for t in _CLIENT_ERR_HITS.get(who, ()) if now - t < _CLIENT_ERR_WINDOW_S]
272
+ seen.append(now)
273
+ _CLIENT_ERR_HITS[who] = seen
274
+ if len(_CLIENT_ERR_HITS) > 2048:
275
+ for k in [k for k, v in _CLIENT_ERR_HITS.items()
276
+ if not v or now - v[-1] > _CLIENT_ERR_WINDOW_S]:
277
+ _CLIENT_ERR_HITS.pop(k, None)
278
+ if len(seen) > _CLIENT_ERR_PER_WINDOW:
279
+ return {"ok": True, "recorded": False}
280
+
281
+ # The body, BOUNDED BY THE STREAM. `Body(...)` would already have been read and parsed before
282
+ # this line ran, and `content-length` is caller-supplied β€” the same argument `routes_forms`
283
+ # and `routes_publish` both carry.
284
+ size, chunks = 0, []
285
+ async for chunk in request.stream():
286
+ size += len(chunk)
287
+ if size > _CLIENT_ERR_MAX:
288
+ raise err(413, "body_too_large", "that report is too large")
289
+ chunks.append(chunk)
290
+ import json as _json
291
+ try:
292
+ body = _json.loads(b"".join(chunks) or b"{}")
293
+ except ValueError:
294
+ body = {}
295
+ if not isinstance(body, dict):
296
+ body = {}
297
+
298
+ # β›” TWO KEYS, BOTH TRUNCATED, NOTHING ELSE. See the block note above.
299
+ message = str(body.get("message") or "")[:500]
300
+ stack = str(body.get("componentStack") or "")[:4000]
301
+ if not message and not stack:
302
+ return {"ok": True, "recorded": False}
303
+ try:
304
+ import harness.telemetry as _tel
305
+ _tel.log("client_error", user=session.uname, tenant=session.tenant,
306
+ message=message, componentStack=stack)
307
+ except Exception: # noqa: BLE001
308
+ # Telemetry may never break a request β€” `deps.py` carries the same rule at its own site.
309
+ return {"ok": True, "recorded": False}
310
+ return {"ok": True, "recorded": True}
api/routes_connectors.py CHANGED
@@ -30,7 +30,7 @@ from deps import Session, require_session
30
  #: one question, and this module used to answer it with an unguarded `os.environ.get("ODOO_URL")`
31
  #: while `routes_keychain` answered it correctly. Importing the correct one is the fix; writing a
32
  #: second correct one here would leave the same two-copies shape that produced the defect.
33
- from routes_keychain import env_odoo_available
34
 
35
  # β›” THE PREFIX IS LOAD-BEARING AND IT WAS MISSING (found at the wave-23 close-out audit).
36
  # This router declared a bare `APIRouter()`, so its one path was served at `/connectors/directory`
@@ -396,15 +396,31 @@ def directory(rt, uname, is_admin=False, incomplete=None):
396
  # reader. A token connector is the case R4 is really about: a member's own Stripe
397
  # key is theirs, and the card must say which kind of connection it is looking at.
398
  "scope": scopes.get(t["entry"]) or "business",
399
- # ⚠ `manage` STAYS ADMIN-ONLY HERE, deliberately, and it is a smaller door than
400
- # R4 allows. A member may now hold a PERSONAL connection β€” the room for it is
401
- # Settings β†’ Keychains, which W32-T11 opens to them β€” but this card's action word
402
- # has been "the tenant-wide credential door" since wave 23 and a sibling gate
403
- # asserts a member is offered none. Widening it is a one-word change once that
404
- # assertion is retargeted; booked rather than done, because reddening another
405
- # lane's gate to add a second door to a room that is already reachable is a bad
406
- # trade.
407
- "manage": "keychain" if is_admin else "",
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
408
  # Said once, on the row, rather than in a paragraph somewhere (R13): storing the
409
  # key is real and useful, and it is not the same as the data being on screen.
410
  "note": "Storing the key connects the account; reading its data lands in a later "
@@ -566,9 +582,22 @@ def connectors_directory(session: Session = Depends(require_session)):
566
  """C11: every connector this platform knows about, with its state for THIS reader.
567
 
568
  Session-gated rather than admin-gated on purpose: a non-admin should be able to see what the
569
- workspace is connected to (and connect their OWN per-user accounts, which is the whole point
570
- of R7's per-user slots). `manage` is what carries admin-ness β€” it is empty for a non-admin,
571
- so the client renders the card without an action rather than offering a door that 403s.
 
 
 
 
 
 
 
 
 
 
 
 
 
572
  """
573
  # ⚠ `session.admin`, NOT `is_admin`. A `getattr(session, "is_admin", False)` here reads as
574
  # careful and is silently False for every admin on earth β€” the manage actions would simply
 
30
  #: one question, and this module used to answer it with an unguarded `os.environ.get("ODOO_URL")`
31
  #: while `routes_keychain` answered it correctly. Importing the correct one is the fix; writing a
32
  #: second correct one here would leave the same two-copies shape that produced the defect.
33
+ from routes_keychain import TENANT_WIDE_TYPES, env_odoo_available
34
 
35
  # β›” THE PREFIX IS LOAD-BEARING AND IT WAS MISSING (found at the wave-23 close-out audit).
36
  # This router declared a bare `APIRouter()`, so its one path was served at `/connectors/directory`
 
396
  # reader. A token connector is the case R4 is really about: a member's own Stripe
397
  # key is theirs, and the card must say which kind of connection it is looking at.
398
  "scope": scopes.get(t["entry"]) or "business",
399
+ # ⭐⭐ W33-T66 / D-204 β€” THE MEMBER'S DOOR, AND IT IS A THIRD WORD RATHER THAN A
400
+ # WIDENED ONE. This route's own docstring has promised since wave 23 that a
401
+ # non-admin may *"connect their OWN per-user accounts"*, and the next sentence
402
+ # then handed them an empty `manage` β€” a contradiction inside one paragraph, and
403
+ # D-204's subject. W32-T11 opened the ROOM (`admin_gate` is gone from
404
+ # `routes_keychain`; a member may POST `scope: "personal"`), so the only thing
405
+ # still missing was a way to reach it from the map.
406
+ #
407
+ # β›” `keychain` DOES NOT WIDEN, and that is deliberate rather than timid.
408
+ # `verify_automation`'s NC asserts a non-admin is offered no `manage == "keychain"`
409
+ # row, and its own comment says why in the right words: *"the line is TENANT-WIDE
410
+ # vs PER-USER, not admin vs everyone … what they must never be offered is the
411
+ # KEYCHAIN door, which edits credentials the whole tenant uses."* That assertion
412
+ # is CORRECT and stays green. A member's personal connection is a different door
413
+ # into the same room, so it gets its own word.
414
+ #
415
+ # ⚠ THE PREDICATE IS THE CREDENTIAL'S OWN RULE, IMPORTED, NOT RE-STATED HERE.
416
+ # `TENANT_WIDE_TYPES` is what `routes_keychain` REFUSES a personal scope for, so
417
+ # offering a personal door for one of those types would be offering a button that
418
+ # 400s β€” the exact failure R6 removed from the Google card. Today that means
419
+ # Stripe and Shopify open for a member and Meta Ads does not
420
+ # ([[one-question-two-normalizers]]).
421
+ "manage": ("keychain" if is_admin
422
+ else "" if t["entry"] in TENANT_WIDE_TYPES
423
+ else "keychain_personal"),
424
  # Said once, on the row, rather than in a paragraph somewhere (R13): storing the
425
  # key is real and useful, and it is not the same as the data being on screen.
426
  "note": "Storing the key connects the account; reading its data lands in a later "
 
582
  """C11: every connector this platform knows about, with its state for THIS reader.
583
 
584
  Session-gated rather than admin-gated on purpose: a non-admin should be able to see what the
585
+ workspace is connected to, and connect their OWN per-user accounts β€” which is the whole point
586
+ of R7's per-user slots.
587
+
588
+ ⭐⭐ W33-T66 / D-204 β€” AND THAT SECOND CLAUSE IS NOW TRUE. This docstring used to promise it and
589
+ then say, one sentence later, that `manage` *"is empty for a non-admin"* β€” a contradiction
590
+ inside a single paragraph, which is how the promise survived three waves without a door behind
591
+ it. `manage` carries WHICH ROOM, and there are three answers for a keychain-backed row:
592
+
593
+ Β· `keychain` β€” the tenant-wide credential door. **Admins only**, and it must stay
594
+ that way: it edits the key the whole workspace reads through.
595
+ Β· `keychain_personal` β€” the same room, for a connection that is this reader's own. Offered
596
+ to a non-admin **only** where a personal credential is actually
597
+ allowed (`routes_keychain.TENANT_WIDE_TYPES` is the refusal list, so
598
+ Odoo and Meta Ads never get it).
599
+ Β· `""` β€” nothing this reader can do here; the client renders no control
600
+ rather than a door that would 403.
601
  """
602
  # ⚠ `session.admin`, NOT `is_admin`. A `getattr(session, "is_admin", False)` here reads as
603
  # careful and is silently False for every admin on earth β€” the manage actions would simply
api/routes_customers.py CHANGED
@@ -209,6 +209,17 @@ def grid_assembly(session: Session, scope: str = "customer", storage_key: str =
209
  # The derived channel: cohort membership cells + measure column values, ONE dict β€” the
210
  # same read-only channel the embed host hands to rows_from_pool.
211
  derived = aios_grid.cohort_cells(lists)
 
 
 
 
 
 
 
 
 
 
 
212
  for pid, cells in measure_resolve.column_values(
213
  fields, team_id, pids, today, stamp, rt.measure_memo,
214
  on_error=_measure_err).items():
 
209
  # The derived channel: cohort membership cells + measure column values, ONE dict β€” the
210
  # same read-only channel the embed host hands to rows_from_pool.
211
  derived = aios_grid.cohort_cells(lists)
212
+ # ⭐ W33-T43 / owner item 12 ("One unique ID per database always"). The customer grid carried
213
+ # NO Odoo id column at all, while its retiring twin `ut_odoo_customers` carried `partner_id`
214
+ # as its join key β€” so the merge would have lost the one value every Odoo document joins on.
215
+ #
216
+ # β›” IT IS DERIVED, NOT A POOL COLUMN, AND THAT IS THE WHOLE POINT: a customer row's `pid` IS
217
+ # the `res.partner` id (`modules/customer_data.pool` mints it that way and the identity is
218
+ # asserted against Odoo in that module). Adding it to the pool would be a SECOND source for
219
+ # one fact, which is the class of defect item 12 is about. This channel exists for exactly
220
+ # this β€” a value the host knows per render and the pool has no business storing.
221
+ for pid in pids:
222
+ derived.setdefault(pid, {})["partner_id"] = pid
223
  for pid, cells in measure_resolve.column_values(
224
  fields, team_id, pids, today, stamp, rt.measure_memo,
225
  on_error=_measure_err).items():
api/routes_feedback.py ADDED
@@ -0,0 +1,180 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """routes_feedback.py β€” WAVE 35 (ruling R8, contract C6): FEEDBACK GOES TO THE OPERATOR PLANE.
2
+
3
+ R8: *"Feedback goes to the platform-wide `loopable` operator plane. No tenant sees another's, and
4
+ the submitting tenant cannot edit or delete what it sent."*
5
+
6
+ β›”β›” TWO DOORS, TWO DIFFERENT WALLS, AND THAT ASYMMETRY IS THE WHOLE TICKET. D-221 is already on the
7
+ register for a cross-tenant WRITE gated on `admin_gate` alone, so getting this backwards is a repeat
8
+ rather than a novelty:
9
+
10
+ POST /feedback ANY authenticated session, any tenant. It is the TENANT'S OWN ACT.
11
+ GET /feedback `core.platform_admin.is_platform_admin` ONLY. It is the OPERATOR'S read.
12
+
13
+ A single wall cannot express that. Gating the write on the operator predicate would mean nobody can
14
+ send feedback; gating the read on `admin_gate` would hand every tenant admin every other tenant's
15
+ words. So the walls are declared per route, and `verify_api` drives BOTH from a tenant admin, a
16
+ plain user and a platform admin rather than asserting the source.
17
+
18
+ β›” THE PROVENANCE IS STAMPED SERVER-SIDE AND NEVER READ FROM THE BODY. `tenant`, `user` and `ts`
19
+ come from the verified session and the clock. A body-supplied tenant would let any account file
20
+ words against another company's name, in a store only the operator reads and therefore only the
21
+ operator could ever be misled by.
22
+
23
+ β›” NO EDIT AND NO DELETE EXIST AT ALL β€” not for the tenant (R8 says so) and not for the operator
24
+ either. R8 grants the operator a READ; a delete door is a capability nobody asked for on a store
25
+ whose whole value is that it is append-only. There is nothing here to forget to wall.
26
+
27
+ ⚠ ONE STORE, AND WHERE IT PHYSICALLY LIVES IS STATED RATHER THAN IMPLIED. This uses `core.store`
28
+ with a raw key, which is the same thing `routes_platform_admin` does for `runtime.TENANTS_KEY` and
29
+ for the same reason: a platform fact belongs in the default store, not inside any tenant's
30
+ namespace. ⚠ That default is currently tenant #0's dataset repo (`OS_DATA_REPO`), so "the platform
31
+ store" and "tenant #0's repo" are the same bucket today. That is pre-existing (the control plane's
32
+ tenant records are already there) and it is a hosting fact, not a permission one β€” no tenant route
33
+ can read this key, because none of them names it.
34
+ """
35
+ import time
36
+
37
+ from fastapi import APIRouter, Body, Depends
38
+
39
+ import core.platform_admin as platform_admin
40
+ import core.store as store
41
+ from deps import Session, err, require_session
42
+
43
+ router = APIRouter(prefix="/api/v1")
44
+
45
+ #: The platform-wide bucket. `{"rows": [ {id, tenant, user, category, text, ts} ]}` β€” newest LAST
46
+ #: in the store, newest FIRST on the wire.
47
+ STORE_KEY = "platform_feedback"
48
+
49
+ #: ⭐ THE CATEGORY VOCABULARY IS SERVER-OWNED AND SERVED (B's `ASK B-1 (3)`, agreed). A client
50
+ #: constant beside a server enum drifts, and the drift shows up as a dropdown offering a value the
51
+ #: door refuses. B renders exactly this list; the door accepts exactly these keys.
52
+ CATEGORIES = (
53
+ {"key": "bug", "label": "Something is broken"},
54
+ {"key": "idea", "label": "An idea or a request"},
55
+ {"key": "data", "label": "The numbers look wrong"},
56
+ {"key": "speed", "label": "Something is slow"},
57
+ {"key": "other", "label": "Something else"},
58
+ )
59
+
60
+ MAX_CHARS = 4000
61
+
62
+ #: A ceiling on the whole store. β›” REPORTED AT BOTH ENDS, never a silent drop: the submitter is
63
+ #: refused with the cause, and the operator's own payload says the store is full so they can act.
64
+ #: Dropping the OLDEST rows instead would delete the operator's earliest feedback to make room for
65
+ #: the newest, which is the one direction this store must never move in.
66
+ MAX_ROWS = 5000
67
+
68
+
69
+ def _rows():
70
+ try:
71
+ raw = store.get(STORE_KEY) or {}
72
+ except Exception:
73
+ return []
74
+ rows = raw.get("rows") if isinstance(raw, dict) else None
75
+ return [r for r in rows if isinstance(r, dict)] if isinstance(rows, list) else []
76
+
77
+
78
+ @router.get("/feedback/form")
79
+ def feedback_form(session: Session = Depends(require_session)):
80
+ """What the composer renders. Any session β€” every account may send feedback.
81
+
82
+ ⚠ It deliberately returns NO submissions. R8 gives the tenant no read of what it sent, so a
83
+ "your feedback" list would be a second door to a thing the ruling closed. The 201 is the receipt.
84
+ """
85
+ return {"categories": [dict(c) for c in CATEGORIES], "maxChars": MAX_CHARS}
86
+
87
+
88
+ @router.post("/feedback", status_code=201)
89
+ def submit_feedback(body: dict = Body(default=None),
90
+ session: Session = Depends(require_session)):
91
+ """Send one piece of feedback to the operator plane. `{"category": key, "text": str}`.
92
+
93
+ ⚠ ANY authenticated session, deliberately β€” see the header. The provenance is stamped from the
94
+ VERIFIED session, never from the body.
95
+ """
96
+ body = body if isinstance(body, dict) else {}
97
+ category = str(body.get("category") or "").strip()
98
+ text = str(body.get("text") or "").strip()
99
+ if category not in {c["key"] for c in CATEGORIES}:
100
+ # Named, and it names the fix: a client whose dropdown has drifted from this list is a
101
+ # client whose user is about to lose what they typed.
102
+ raise err(400, "unknown_category",
103
+ "that is not one of the feedback categories. Reload the page and try again")
104
+ if not text:
105
+ raise err(400, "bad_request", "type what you want to tell us")
106
+ if len(text) > MAX_CHARS:
107
+ raise err(400, "too_long",
108
+ f"feedback is limited to {MAX_CHARS:,} characters. Yours is {len(text):,}")
109
+ if not store.available():
110
+ raise err(503, "store_unavailable",
111
+ "feedback could not be sent right now. Nothing was saved.")
112
+ if len(_rows()) >= MAX_ROWS:
113
+ raise err(503, "feedback_full",
114
+ "the feedback store is full and we have been told. Nothing was saved, so please "
115
+ "send this again later")
116
+ row = {
117
+ # β›” EVERY ONE OF THESE FOUR IS SERVER-SIDE. A body-supplied tenant or user would let any
118
+ # account file words against somebody else's name in a store only the operator reads.
119
+ "tenant": str(session.tenant),
120
+ "user": str(session.uname),
121
+ "ts": int(time.time()),
122
+ "category": category,
123
+ "text": text,
124
+ }
125
+ row["id"] = f"fb_{row['ts']}_{abs(hash((row['tenant'], row['user'], text))) % 10**8:08d}"
126
+
127
+ def _up(data):
128
+ data = data if isinstance(data, dict) else {}
129
+ rows = data.get("rows")
130
+ data["rows"] = (rows if isinstance(rows, list) else []) + [row]
131
+ return data
132
+
133
+ try:
134
+ # `flush='sync'`: a person pressed Send and is being told it landed. The coalescing mode is
135
+ # for autosaves nobody is waiting on.
136
+ store.update(STORE_KEY, _up)
137
+ except Exception:
138
+ raise err(503, "store_unavailable",
139
+ "feedback could not be sent right now. Nothing was saved.")
140
+ return {"ok": True, "id": row["id"]}
141
+
142
+
143
+ @router.get("/feedback")
144
+ def list_feedback(tenant: str = "", session: Session = Depends(require_session)):
145
+ """R8's operator read: every tenant's feedback, newest first. PLATFORM ADMIN ONLY.
146
+
147
+ β›” THE WALL IS `core.platform_admin.is_platform_admin`, IMPORTED AND CALLED β€” the double lock
148
+ (the record flag AND the `loopable` tenant), never re-expressed here. A tenant admin is not
149
+ admitted: `role: 'admin'` is one company's authority over its own workspace, and this is every
150
+ company's words in one list.
151
+ ⚠ `?tenant=` NARROWS an already-admitted operator's view. It is a convenience, not a wall, and
152
+ it is applied after the gate so it can never be the thing that admits anybody.
153
+ """
154
+ if not platform_admin.is_platform_admin(session.user):
155
+ raise err(403, "forbidden", "this is a Loopable operator surface")
156
+ rows = _rows()
157
+ if tenant:
158
+ want = str(tenant).strip().lower()
159
+ rows = [r for r in rows if str(r.get("tenant") or "").lower() == want]
160
+ # β›” THE TIEBREAK IS THE STORE'S OWN ORDER, AND WITHOUT IT "NEWEST FIRST" IS FALSE FOR ANY TWO
161
+ # ROWS FILED IN THE SAME SECOND (wave-35 QA). `ts` is `int(time.time())` β€” SECOND resolution β€”
162
+ # and Python's sort is STABLE, so under `reverse=True` a tie keeps INSERTION order, which is
163
+ # oldest-first: the one direction this route promises never to return. It is not hypothetical
164
+ # and it is not a flake: `api_api`'s R8 row failed on it in the close-out battery, in the
165
+ # wave-35 QA battery and in a solo re-run, and the baseline recorded the true count anyway, so
166
+ # a REAL regression would have read as the known-flake line.
167
+ # ⚠ ENUMERATE OVER THE ALREADY-FILTERED LIST: `_rows()` appends, so a row's index IS its arrival
168
+ # order (the module docstring's "newest LAST in the store"), and `?tenant=` only drops rows, so
169
+ # the surviving indexes stay monotonic. Raising `ts` to sub-second resolution instead would
170
+ # break every id already minted as `fb_<ts>_…`.
171
+ rows = [r for _, r in sorted(enumerate(rows),
172
+ key=lambda p: (int(p[1].get("ts") or 0), p[0]), reverse=True)]
173
+ out = {"rows": [dict(r) for r in rows], "total": len(rows), "capacity": MAX_ROWS,
174
+ "tenants": sorted({str(r.get("tenant") or "") for r in _rows() if r.get("tenant")})}
175
+ if len(_rows()) >= MAX_ROWS:
176
+ # R6's second sentence, aimed at the person who can act: submissions are being REFUSED right
177
+ # now, and the operator is the only one who can see that from here.
178
+ out["note"] = ("the feedback store is at capacity, so new submissions are being refused. "
179
+ "Archive what you have read")
180
+ return out
api/routes_grid.py CHANGED
@@ -726,6 +726,111 @@ def grid_calendar_metrics(body: dict = Body(default=None), scope: str = "custome
726
  return out
727
 
728
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
729
  @router.post("/grid/events")
730
  def grid_events_route(body: dict = Body(default=None),
731
  session: Session = Depends(require_session)):
@@ -822,13 +927,30 @@ def grid_events_route(body: dict = Body(default=None),
822
  try:
823
  for one in events:
824
  eid = str(one.get("id") or "") if isinstance(one, dict) else ""
 
 
 
 
 
 
825
  rerender = grid_events.handle_one(one, ctx)
826
- results.append({"id": eid, "rerender": bool(rerender)})
 
 
 
 
 
 
827
  except grid_events.StoreUnavailable:
828
  raise err(503, "store_unavailable",
829
  "the tenant store is unavailable β€” none of your changes were saved")
830
 
831
  out = {"results": results, "rerender": any(r["rerender"] for r in results)}
 
 
 
 
 
832
  if ctx.out.doc is not None:
833
  # ⭐ C4 / W30-T27 β€” `docPayload` IS THE NAME THE CLIENT ALREADY DECLARES. `types.ts` has
834
  # carried `docPayload?: {pid, docId, name, mime, data_b64}` since C5, and `Documents.tsx`
 
726
  return out
727
 
728
 
729
+ #: ⭐⭐ THE BULK CELL DOOR'S OWN CEILING, and it is REPORTED rather than silently enforced. The
730
+ #: standing no-cap rule governs data read from a connected source; this is a WRITE of team-typed
731
+ #: values, so a bound is right β€” but R6's second sentence still binds, which is why exceeding it
732
+ #: is a 400 naming the number and the count, never a truncation.
733
+ #: Sized against the real job with headroom: the 2027 catalog is 1,397 rows.
734
+ MAX_BULK_ROWS = 10_000
735
+
736
+
737
+ @router.post("/grid/bulk-cells")
738
+ def bulk_cells(body: dict = Body(default=None),
739
+ session: Session = Depends(require_session)):
740
+ """`{scopeKey, rows: {"<pid>": {key: value}}}` β†’ shared cells on many rows, ONE transaction.
741
+
742
+ ⭐⭐ WHY THIS EXISTS AT ALL, because "we already have `/grid/events`" is the obvious objection.
743
+ That door is capped at `_MAX_EVENTS = 24` (it is sized for the client's resend window), so a
744
+ 1,397-row import is ~59 sequential POSTs against ONE JSON document β€” and this repo has a
745
+ MEASURED scar for exactly that: 18 writes against one document under the store's coalescing
746
+ single-flight landed **zero** while answering 200 eighteen times. Raising `_MAX_EVENTS` would
747
+ have widened the browser's own resend window to fix a script's problem. One door, one
748
+ transaction, is the honest shape.
749
+
750
+ β›” TENANT-WIDE KEYS ONLY, AND THAT IS THE POINT RATHER THAN A LIMITATION. It writes through
751
+ `shared_overlay`, which every account in the tenant reads. A per-user bulk write would be a
752
+ contradiction: nobody imports 1,397 rows of the team's work so that one account can see it β€”
753
+ that is precisely the failure owner ruling R6 was made to avoid. A key that is not declared
754
+ `shared: true` in the canonical contract is REFUSED BY NAME.
755
+
756
+ β›” ADMIN ONLY. This mutates data every account in the tenant reads, in bulk, in one call.
757
+ D-172 already records that the shared stratum has an asymmetric wall (anyone may create a
758
+ tenant-wide column, only the creator or an admin may delete one); a new door does not get to
759
+ inherit the loose half of an asymmetry somebody already flagged.
760
+ """
761
+ import core.shared_overlay as shared_overlay
762
+ import modules.product_data as pd
763
+ import core.perm_scope as perm_scope
764
+ from routes_products import MODULE as PRODUCT_MODULE, scoped_pool
765
+
766
+ scope = _scope_or_400((body or {}).get("scopeKey"))
767
+ if scope != "product":
768
+ # Fail-closed with the reason: the shared stratum is a PRODUCT-topic mechanism today.
769
+ raise err(400, "scope_not_bulk_writable",
770
+ f"bulk cell writes are available on the product database; '{scope}' has no "
771
+ f"tenant-wide cell stratum")
772
+ session.require(PRODUCT_MODULE)
773
+ if not session.admin:
774
+ raise err(403, "admin_only",
775
+ "a bulk write changes values every account in this workspace reads")
776
+
777
+ rows = (body or {}).get("rows")
778
+ if not isinstance(rows, dict):
779
+ raise err(400, "bad_rows", "expected {rows: {\"<pid>\": {field: value}}}")
780
+ if len(rows) > MAX_BULK_ROWS:
781
+ raise err(400, "too_many_rows",
782
+ f"at most {MAX_BULK_ROWS} rows per request; this one carried {len(rows)}")
783
+
784
+ pids, _team, _src, fields_base = scoped_pool(session)
785
+ allowed = {int(p) for p in pids}
786
+ shared_keys = set(pd.SHARED_KEYS())
787
+ hidden = perm_scope.hidden_keys(session.user, PRODUCT_MODULE, fields_base)
788
+
789
+ clean, unknown_pid, refused_keys = {}, [], {}
790
+ for raw_pid, values in rows.items():
791
+ try:
792
+ pid = int(raw_pid)
793
+ except (TypeError, ValueError):
794
+ unknown_pid.append(str(raw_pid)[:40])
795
+ continue
796
+ # ⚠ THE POOL IS THE WALL. `scoped_pool` is the same predicate the read door uses, so a
797
+ # caller cannot write a row they could not see β€” including a row in another BU.
798
+ if pid not in allowed:
799
+ unknown_pid.append(str(raw_pid)[:40])
800
+ continue
801
+ keep = {}
802
+ for key, value in dict(values or {}).items():
803
+ key = str(key)
804
+ if key not in shared_keys:
805
+ refused_keys.setdefault("not_shared", set()).add(key)
806
+ continue
807
+ if key in hidden:
808
+ refused_keys.setdefault("hidden_by_permissions", set()).add(key)
809
+ continue
810
+ if not isinstance(value, (str, int, float)) or isinstance(value, bool):
811
+ refused_keys.setdefault("unsupported_value", set()).add(key)
812
+ continue
813
+ keep[key] = value
814
+ if keep:
815
+ clean[pid] = keep
816
+
817
+ written = shared_overlay.put_rows(pd.TABLE_KEY, clean, st=pd.TABLE_OPS.st) if clean else {}
818
+ out = {
819
+ "rows_written": len(written),
820
+ "cells_written": sum(len(v) for v in written.values()),
821
+ "rows_requested": len(rows),
822
+ }
823
+ # ⭐ R6's second sentence: what did NOT land, and why. A bulk door that reports only its
824
+ # successes is how 701 unmatched SKUs disappear quietly.
825
+ if unknown_pid:
826
+ out["rows_not_in_your_pool"] = {"count": len(unknown_pid),
827
+ "sample": sorted(unknown_pid)[:10]}
828
+ if refused_keys:
829
+ out["refused_fields"] = {reason: sorted(keys)
830
+ for reason, keys in refused_keys.items()}
831
+ return out
832
+
833
+
834
  @router.post("/grid/events")
835
  def grid_events_route(body: dict = Body(default=None),
836
  session: Session = Depends(require_session)):
 
927
  try:
928
  for one in events:
929
  eid = str(one.get("id") or "") if isinstance(one, dict) else ""
930
+ # ⭐⭐ D-291 β€” WHICH event was refused, not merely THAT something was. The handler
931
+ # appends to a shared list, so the refusals belonging to THIS event are exactly the
932
+ # ones that appeared across THIS call. Reading the list once after the loop would
933
+ # answer "the batch was refused" and leave the caller to guess which member, which is
934
+ # the same class of unfalsifiable answer the channel exists to end.
935
+ _before = len(ctx.out.refusals)
936
  rerender = grid_events.handle_one(one, ctx)
937
+ row = {"id": eid, "rerender": bool(rerender)}
938
+ mine = ctx.out.refusals[_before:]
939
+ if mine:
940
+ # `refused` is the SHAPE a caller branches on; the first reason is the one that
941
+ # stopped this write (a handler returns at its first refusal).
942
+ row["refused"] = mine[0]
943
+ results.append(row)
944
  except grid_events.StoreUnavailable:
945
  raise err(503, "store_unavailable",
946
  "the tenant store is unavailable β€” none of your changes were saved")
947
 
948
  out = {"results": results, "rerender": any(r["rerender"] for r in results)}
949
+ if ctx.out.refusals:
950
+ # ⚠ ALSO AT THE TOP LEVEL, because a batch that was wholly refused must not read as a
951
+ # batch that wholly landed. A caller that only checks the envelope still learns something
952
+ # is wrong, and a caller that walks `results` learns exactly which member.
953
+ out["refusals"] = list(ctx.out.refusals)
954
  if ctx.out.doc is not None:
955
  # ⭐ C4 / W30-T27 β€” `docPayload` IS THE NAME THE CLIENT ALREADY DECLARES. `types.ts` has
956
  # carried `docPayload?: {pid, docId, name, mime, data_b64}` since C5, and `Documents.tsx`
api/routes_keychain.py CHANGED
@@ -296,6 +296,114 @@ def _rel_reconnect(rt):
296
  return True
297
 
298
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
299
  def _own_row(session, entry_id):
300
  """The visible row for `entry_id`, or a 404. β›” A 404 rather than a 403 for an entry the
301
  caller cannot see: telling a member that somebody else's personal credential EXISTS is the
@@ -703,6 +811,31 @@ def odoo_config_put(body: dict = Body(default=None),
703
  if server_db:
704
  notes.append(_rewrite_server_db(session, server_db))
705
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
706
  out = odoo_config(session)
707
  return {**out, "notes": [n for n in notes if n]}
708
 
@@ -783,6 +916,32 @@ def odoo_disconnect(session: Session = Depends(require_session)):
783
  cur["frozenBy"] = str(session.uname)
784
  return cur
785
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
786
  # ⚠ THE FLAG FIRST, THE CREDENTIAL SECOND. If the flag write fails, nothing has happened and
787
  # the connector is still live; if the delete failed AFTER the flag landed, the tenant is
788
  # frozen with an unused key, which is recoverable from the Keychains pane. The reverse order
@@ -791,6 +950,19 @@ def odoo_disconnect(session: Session = Depends(require_session)):
791
  session.runtime.update(rel.CONFIG_KEY, _up, flush="sync")
792
  except Exception:
793
  raise err(503, "store_unavailable", "nothing was disconnected β€” try again")
 
 
 
 
 
 
 
 
 
 
 
 
 
794
  removed = ""
795
  if entry is not None:
796
  try:
@@ -801,10 +973,28 @@ def odoo_disconnect(session: Session = Depends(require_session)):
801
  raise err(503, "store_unavailable",
802
  "the databases are frozen but the stored credential was not removed β€” "
803
  "delete it under Keychains")
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
804
  return {"frozen": True, "removedEntry": removed,
805
- "note": "Your Odoo databases are frozen: every row and every column you had is still "
806
- "there and still readable, and nothing is being refreshed. Reconnecting adds "
807
- "the key back and resumes into the same databases."}
 
 
808
 
809
 
810
  @router.post("/admin/connectors/odoo/reconnect")
@@ -820,8 +1010,26 @@ def odoo_reconnect(session: Session = Depends(require_session)):
820
  _rel_reconnect(session.runtime)
821
  except Exception:
822
  raise err(503, "store_unavailable", "the change was not saved β€” try again")
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
823
  connected = bool(_odoo_entry(session)) or env_odoo_available(session.runtime)
824
- return {"frozen": False, "connected": connected,
825
  "note": ("Odoo is connected again and the databases you already had will refresh in "
826
  "place." if connected else
827
  "The freeze is lifted, but there is no Odoo credential yet β€” add one under "
 
296
  return True
297
 
298
 
299
+ # ══════════════════════════════════════════════════ W35-T45 / R11: THE ENV -> KEYCHAIN MIGRATION
300
+ #
301
+ # R11: *"Tenant #0's Odoo credential MIGRATES onto the keychain, with the environment kept as
302
+ # fallback."* The owner chose this over a read-only display row WITH THE MIGRATION RISK STATED, so
303
+ # the risk is what this block is mostly about.
304
+ #
305
+ # ⭐ WHAT WAS ALREADY TRUE, CHECKED BEFORE ANY OF IT WAS WRITTEN: the keychain-first-then-env
306
+ # RESOLVER has existed since the 2026-08-04 cutover. `harness/runtime.py::odoo_source` is already
307
+ # (1) a keychain `odoo` entry, (2) tenant #0's compiled env connector, (3) None for anybody else,
308
+ # and `visible_entries` already serves a non-admin the row WITHOUT a preview. So R11 is not "build
309
+ # a resolver" β€” it is "give tenant #0 the ROW", which is the only reason its Keychain page looks
310
+ # empty while its Odoo grids work.
311
+ #
312
+ # β›”β›” AND THE ONE REAL HAZARD IS NOT THE CREDENTIAL, IT IS THE PAUSE FLAG. `odoo_flag_key()` returns
313
+ # the first keychain `odoo` entry id when one exists and `ENV_ODOO_FLAG_KEY` ("odoo-env") otherwise β€”
314
+ # so CREATING THE ENTRY MOVES THE ADDRESS THE PAUSE FLAG LIVES AT. A tenant #0 that was paused under
315
+ # `odoo-env` would come back UNPAUSED, silently, at the first boot after this ships: the connector
316
+ # resumes pulling live Odoo because a migration changed which key the freeze was stored under. That
317
+ # is D-259's exact shape (a pause written under one key and read under another) and
318
+ # [[a-guard-bound-to-a-role-stops-guarding-when-the-role-moves]]. The flag is carried across in the
319
+ # SAME pass, and the carry is asserted.
320
+
321
+
322
+ def _env_odoo_fields():
323
+ """The four env values `odoo_client` reads, or `(None, why)` when they are not all present.
324
+
325
+ β›” ALL FOUR OR NOTHING, and this completeness check is load-bearing rather than defensive.
326
+ Creating the entry makes `odoo_source` resolve through branch 1 INSTEAD of the env β€” so a
327
+ PARTIAL migration would hand the connector `{url, db}` and no key and take tenant #0's Odoo
328
+ offline, on a deployment where it had been working. The env fallback cannot save it, because the
329
+ entry's existence is what turns the fallback off.
330
+ ⚠ The names are `odoo_client.py`'s own (`ODOO_URL`/`ODOO_DB`/`ODOO_USER`/`ODOO_API_KEY`) and the
331
+ field names are `harness/connectors/odoo.py`'s stored shape (`{url, db, user, api_key}`). Two
332
+ vocabularies meet here; nowhere else.
333
+ """
334
+ want = (("url", "ODOO_URL"), ("db", "ODOO_DB"),
335
+ ("user", "ODOO_USER"), ("api_key", "ODOO_API_KEY"))
336
+ got = {field: (os.environ.get(env) or "").strip() for field, env in want}
337
+ missing = sorted(env for field, env in want if not got[field])
338
+ if missing:
339
+ return None, (f"the environment is missing {', '.join(missing)}, and a partial credential "
340
+ f"would take this tenant's Odoo offline rather than migrate it")
341
+ return got, ""
342
+
343
+
344
+ def migrate_env_odoo(rt):
345
+ """R11 β€” put tenant #0's environment Odoo credential on its keychain, once. Returns a report.
346
+
347
+ `{"done": bool, "entry": id|"", "carried_pause": bool, "why": str}` β€” `why` is filled on every
348
+ path including the skips, because "already migrated", "no keychain key on this deployment" and
349
+ "the env is incomplete" are three different operator actions.
350
+
351
+ β›”β›” IT MUST RUN IN THE CONTAINER, WHICH IS WHY `main.py` CALLS IT AND NO SCRIPT DOES. D-195,
352
+ measured three times: a developer's CLI write to the tenant store is reverted by the running
353
+ Space within a minute (download-modify-upload, last-write-wins) β€” and **the write reports success
354
+ every time**, then a fresh read confirms it, and it is gone by the next poll. A CLI migration here
355
+ would be a dry run that lies, and the thing it would lie about is a credential.
356
+
357
+ ⚠ FAIL-QUIET AND IDEMPOTENT. It runs on EVERY boot; the second one must be a no-op and a
358
+ third-party failure must not take the boot down.
359
+ """
360
+ out = {"done": False, "entry": "", "carried_pause": False, "why": ""}
361
+ if not env_odoo_available(rt):
362
+ # Not tenant #0, or this deployment has no env Odoo at all. Both are normal states.
363
+ out["why"] = "this tenant has no environment Odoo credential to migrate"
364
+ return out
365
+ fields, why = _env_odoo_fields()
366
+ if not fields:
367
+ out["why"] = why
368
+ return out
369
+ # β›” READ THE PAUSE FLAG BEFORE THE WRITE. After the entry exists, `odoo_flag_key()` answers the
370
+ # NEW key and the old one is unreachable through the resolver β€” so the only moment this fact can
371
+ # be observed is now. [[undo-capture-before-the-write]] applied to a guard rather than to data.
372
+ try:
373
+ was_paused = bool(((rt.get(_CONNECTOR_FLAGS_KEY) or {})
374
+ .get(_ENV_ODOO_FLAG_KEY) or {}).get("paused"))
375
+ except Exception: # noqa: BLE001
376
+ was_paused = False
377
+ row, why = _kc().ensure_entry_of_type(
378
+ rt, "odoo", "Odoo (migrated from this deployment)", fields, "system")
379
+ if row is None:
380
+ out["why"] = why
381
+ return out
382
+ out["done"], out["entry"] = True, row["id"]
383
+ if was_paused:
384
+ # The freeze followed the credential. Without this the connector silently RESUMES pulling
385
+ # live Odoo at the first boot after the migration.
386
+ def _up(cur):
387
+ cur = cur if isinstance(cur, dict) else {}
388
+ entry = dict(cur.get(row["id"]) or {})
389
+ entry["paused"] = True
390
+ entry["pausedBy"] = "system"
391
+ entry["pausedNote"] = ("carried over from the environment source when the credential "
392
+ "was migrated onto the keychain")
393
+ cur[row["id"]] = entry
394
+ return cur
395
+
396
+ try:
397
+ rt.update(_CONNECTOR_FLAGS_KEY, _up, flush="sync")
398
+ out["carried_pause"] = True
399
+ except Exception as exc: # noqa: BLE001
400
+ # β›” SAID OUT LOUD. A migration that moved the credential and lost the freeze is worse
401
+ # than one that did not run, so this is the one failure that must never be silent.
402
+ out["why"] = (f"the credential migrated but the PAUSE could not be carried over "
403
+ f"({type(exc).__name__}), so this tenant's Odoo is no longer frozen")
404
+ return out
405
+
406
+
407
  def _own_row(session, entry_id):
408
  """The visible row for `entry_id`, or a 404. β›” A 404 rather than a 403 for an entry the
409
  caller cannot see: telling a member that somebody else's personal credential EXISTS is the
 
811
  if server_db:
812
  notes.append(_rewrite_server_db(session, server_db))
813
 
814
+ # ⭐⭐ W33-T65 / W30-R6's SECOND SENTENCE β€” THE CADENCE IS SET AND ONLY PARTLY OBEYED, AND THE
815
+ # PERSON SETTING IT IS THE ONE WHO HAS TO BE TOLD. Measured, not guessed:
816
+ # Β· `main.py::_store_resync_loop` reads the interval from `sync_seconds(get_runtime(
817
+ # "royal-imports"))` β€” a HARDCODED slug β€” and then sleeps ONCE for the whole process. So
818
+ # for tenant #0 this control moves EVERYBODY's sync, and for every other tenant the value
819
+ # is stored, clamped, displayed and never read.
820
+ # Β· `manual` stores and reads back as `None`, and the loop has no branch on it: it sleeps a
821
+ # default 1800 s and syncs anyway. "Only when I ask" asks all the same.
822
+ # β›” NEITHER IS FIXABLE FROM THIS FILE β€” the loop lives in `main.py`, which this lane does not
823
+ # own β€” and shipping a setting that silently does nothing is the exact failure R6 names. So it
824
+ # is REPORTED here, at the moment of the change, with what it really controls. Delete these
825
+ # notes when the loop becomes per-tenant, not before.
826
+ if clean_every is not None:
827
+ if not rel.is_royal(session.tenant):
828
+ notes.append("this interval is saved, but the sync loop currently reads its schedule "
829
+ "from one workspace for the whole deployment β€” so it will not change how "
830
+ "often YOUR data refreshes until per-workspace scheduling ships")
831
+ else:
832
+ notes.append("this interval is saved and it is the one the deployment's sync loop "
833
+ "uses β€” it changes the refresh rate for every workspace on this "
834
+ "deployment, not only this one")
835
+ if clean_every == "manual":
836
+ notes.append("⚠ 'manual' does not yet stop the background sync: the loop has no "
837
+ "manual-only branch, so data still refreshes on the default interval")
838
+
839
  out = odoo_config(session)
840
  return {**out, "notes": [n for n in notes if n]}
841
 
 
916
  cur["frozenBy"] = str(session.uname)
917
  return cur
918
 
919
+ # ⭐⭐ W33-T65 β€” THE FREEZE HAD A HOLE, AND THE NOTE BELOW WAS THE THING THAT MADE IT A DEFECT
920
+ # RATHER THAN A LIMIT. `rel.frozen` has exactly ONE consumer, `odoo_relational.refresh`, which
921
+ # materialises the EIGHT copied grids. The other two (`ut_odoo_order_lines`,
922
+ # `ut_odoo_gl_lines` β€” `READ_THROUGH_KEYS`) do not go through `refresh` at all: they read
923
+ # THROUGH the per-tenant DuckDB mirror, and the mirror is advanced by `datastore.sync_all`,
924
+ # which gates on the connector PAUSE flag and has never heard of `frozen`. So a disconnected
925
+ # workspace kept serving LIVE, still-moving rows in its two biggest grids while this route's
926
+ # own sentence promised *"nothing is being refreshed"*.
927
+ #
928
+ # β›” THE FIX IS TO MAKE THE SENTENCE TRUE, not to soften it. Disconnect now flips the pause
929
+ # flag on the RESOLVED source as well, which is the switch `sync_all` and `reconcile_deletes`
930
+ # actually read β€” so both halves of "frozen" mean the same thing. Two orderings are borrowed
931
+ # from `pause_connector` because it learned them the hard way:
932
+ # Β· the flag key is resolved BEFORE the credential is deleted β€” after the delete there is no
933
+ # resolved source left to name, and the flag would land under a key nothing reads (D-10).
934
+ # Β· the snapshot is captured BEFORE the flag flips, so there is a last-successful-sync to
935
+ # serve; a failed capture leaves the connector live rather than paused-with-nothing.
936
+ _, flag_key = _resolved_odoo_key(session.runtime)
937
+ snapshots = 0
938
+ if flag_key:
939
+ try:
940
+ snapshots = save_pool_snapshots(session.runtime, taken_by=session.uname)
941
+ except Exception: # noqa: BLE001
942
+ # A snapshot is a nicety; the freeze is the promise. Reported, never fatal.
943
+ snapshots = 0
944
+
945
  # ⚠ THE FLAG FIRST, THE CREDENTIAL SECOND. If the flag write fails, nothing has happened and
946
  # the connector is still live; if the delete failed AFTER the flag landed, the tenant is
947
  # frozen with an unused key, which is recoverable from the Keychains pane. The reverse order
 
950
  session.runtime.update(rel.CONFIG_KEY, _up, flush="sync")
951
  except Exception:
952
  raise err(503, "store_unavailable", "nothing was disconnected β€” try again")
953
+
954
+ paused_mirror = False
955
+ if flag_key:
956
+ def _pause(cur):
957
+ cur = cur if isinstance(cur, dict) else {}
958
+ cur[str(flag_key)] = {"paused": True}
959
+ return cur
960
+ try:
961
+ session.runtime.update(_CONNECTOR_FLAGS_KEY, _pause, flush="sync")
962
+ paused_mirror = True
963
+ except Exception: # noqa: BLE001
964
+ paused_mirror = False
965
+
966
  removed = ""
967
  if entry is not None:
968
  try:
 
973
  raise err(503, "store_unavailable",
974
  "the databases are frozen but the stored credential was not removed β€” "
975
  "delete it under Keychains")
976
+
977
+ # β›” THE SENTENCE IS COMPOSED, NOT CONSTANT, because the two sources genuinely differ and the
978
+ # old fixed string was wrong about one of them. A tenant whose Odoo came from the DEPLOYMENT
979
+ # ENVIRONMENT has no credential for this route to remove β€” `.env` is the container's, not a
980
+ # tenant screen's β€” so it said "the key back" about a key it never held. R6's second sentence:
981
+ # the limit that cannot be removed is REPORTED, with what to do instead.
982
+ note = ("Your Odoo databases are frozen: every row and every column you had is still there and "
983
+ "still readable, and nothing is being refreshed.")
984
+ if not paused_mirror and flag_key:
985
+ note += (" ⚠ The live mirror could not be paused, so the two read-through databases "
986
+ "(order lines and GL lines) may keep advancing β€” pause the Odoo connector under "
987
+ "Keychains to stop them.")
988
+ note += (" Reconnecting adds the key back and resumes into the same databases." if entry
989
+ else " This workspace's Odoo credential comes from the deployment environment, so "
990
+ "there was no stored key to remove β€” the databases are frozen and Reconnect "
991
+ "resumes them into the same tables.")
992
  return {"frozen": True, "removedEntry": removed,
993
+ # ⚠ ON THE WIRE, so the client and a gate can both see which half happened. A boolean
994
+ # nobody returns is a guarantee nobody can check.
995
+ "pausedMirror": paused_mirror, "snapshots": snapshots,
996
+ "source": "keychain" if entry else "env",
997
+ "note": note}
998
 
999
 
1000
  @router.post("/admin/connectors/odoo/reconnect")
 
1010
  _rel_reconnect(session.runtime)
1011
  except Exception:
1012
  raise err(503, "store_unavailable", "the change was not saved β€” try again")
1013
+ # ⭐⭐ W33-T65 β€” AND THE PAUSE DISCONNECT SET, or the freeze would be one-way. Clearing only
1014
+ # `frozen` restores the eight materialised grids and leaves the mirror pinned forever, so the
1015
+ # two read-through grids would sit at the disconnect date while the panel said "connected
1016
+ # again" β€” the same disagreement between the halves of "frozen", pointing the other way.
1017
+ # ⚠ Resolved AFTER `_rel_reconnect`: a tenant reconnects by adding the key back FIRST, so the
1018
+ # resolved source only exists again by this point.
1019
+ _, flag_key = _resolved_odoo_key(session.runtime)
1020
+ resumed = False
1021
+ if flag_key:
1022
+ def _unpause(cur):
1023
+ cur = cur if isinstance(cur, dict) else {}
1024
+ cur[str(flag_key)] = {"paused": False}
1025
+ return cur
1026
+ try:
1027
+ session.runtime.update(_CONNECTOR_FLAGS_KEY, _unpause, flush="sync")
1028
+ resumed = True
1029
+ except Exception: # noqa: BLE001
1030
+ resumed = False
1031
  connected = bool(_odoo_entry(session)) or env_odoo_available(session.runtime)
1032
+ return {"frozen": False, "connected": connected, "resumedMirror": resumed,
1033
  "note": ("Odoo is connected again and the databases you already had will refresh in "
1034
  "place." if connected else
1035
  "The freeze is lifted, but there is no Odoo credential yet β€” add one under "
api/routes_nav.py CHANGED
@@ -248,6 +248,183 @@ def _read_recents(runtime, uname, allowed):
248
  return out[:_MAX_RECENTS]
249
 
250
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
251
  @router.get("/nav")
252
  def nav(session: Session = Depends(require_session)):
253
  """`{pages: [{key,label,source?,chrome}], landing}`.
@@ -336,7 +513,7 @@ def nav(session: Session = Depends(require_session)):
336
  # empty rather than leaving 200-[] ambiguous.
337
  return {"pages": [], "landing": None, "empty": "no_databases"}
338
  raise err(403, "no_surfaces",
339
- "your account has no dashboards assigned β€” ask an administrator")
340
  # WAVE 19 (R8 / C1): the tenant's name + icon overrides, merged LAST β€” after the registry
341
  # rows, after the tenant module filter, after the user tables. Merged HERE rather than
342
  # applied by the client for one reason: `label` is what every reader of this payload shows,
@@ -350,6 +527,21 @@ def nav(session: Session = Depends(require_session)):
350
  # leak with no symptom until two tenants rename the same registry key. Copy the rows before
351
  # merging on the day that invariant changes.
352
  meta = _read_nav_meta(session.runtime)
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
353
  # Wave 21 (C3): the definitions, once β€” `manage`/`canDelete` below answer from `createdBy`.
354
  # WAVE 27 (C9): `locked` answers from `recordMode`, off the same one read.
355
  # ⭐ W31-T10: that read is now the SAME one the merge above did β€” it used to be a second,
@@ -398,6 +590,13 @@ def nav(session: Session = Depends(require_session)):
398
  p["locked"] = True
399
  elif session.admin:
400
  p["manage"] = True
 
 
 
 
 
 
 
401
  entry = meta.get(p.get("key")) if meta else None
402
  if not entry:
403
  continue
@@ -422,8 +621,11 @@ def nav(session: Session = Depends(require_session)):
422
  # both doors finally agree on what a placeable key is. This block keeps using the assembled
423
  # list because it already holds it: re-enumerating here would be a second store read for an
424
  # answer sitting in a local variable.
425
- recents = _read_recents(session.runtime, session.uname,
426
- {str(p.get("key", "")) for p in pages})
 
 
 
427
  # ⭐ W31-T11 β€” TWO KINDS OF ABSENCE, NAMED SEPARATELY, and both keys are ALWAYS PRESENT.
428
  # `omitted` β€” this workspace's catalogue does not include these modules. Deliberate.
429
  # `degraded` β€” a part of this payload could not be read. NOT deliberate, and the shell says
@@ -460,7 +662,7 @@ def nav_opened(body: dict = Body(default=None),
460
  raise err(400, "bad_request", "no page was named")
461
  if not session.runtime.available():
462
  raise err(503, "store_unavailable",
463
- "the tenant store is unavailable β€” nothing was recorded")
464
  stamp, uname = _now(), session.uname
465
 
466
  def _up(data):
@@ -484,7 +686,7 @@ def nav_opened(body: dict = Body(default=None),
484
  session.runtime.update(_NAV_RECENTS_KEY, _up, flush='async')
485
  except Exception:
486
  raise err(503, "store_unavailable",
487
- "the tenant store refused the write β€” nothing was recorded")
488
  return {"key": key, "at": stamp}
489
 
490
 
@@ -549,7 +751,7 @@ def save_nav_meta(body: dict = Body(default=None),
549
  if "name" in body:
550
  if not key.startswith("ut_"):
551
  raise err(400, "name_not_allowed",
552
- "only a database you created can be renamed β€” this one's name comes "
553
  "from the module registry")
554
  name = " ".join(str(body.get("name") or "").split())[:_MAX_NAV_NAME]
555
  if not name:
@@ -560,7 +762,7 @@ def save_nav_meta(body: dict = Body(default=None),
560
 
561
  if not session.runtime.available():
562
  raise err(503, "store_unavailable",
563
- "the tenant store is unavailable β€” nothing was saved")
564
 
565
  def _up(data):
566
  data = data if isinstance(data, dict) else {}
@@ -583,7 +785,7 @@ def save_nav_meta(body: dict = Body(default=None),
583
  session.runtime.update(_NAV_META_KEY, _up)
584
  except Exception:
585
  raise err(503, "store_unavailable",
586
- "the change was not saved β€” the store refused the write")
587
  return {"key": key, "meta": _read_nav_meta(session.runtime).get(key, {})}
588
 
589
 
@@ -608,7 +810,7 @@ def save_nav_prefs(body: dict = Body(default=None),
608
  clean = _clean_nav_prefs(body or {}, keys, keep_unknown_ut=not enumerated)
609
  if not session.runtime.available():
610
  raise err(503, "store_unavailable",
611
- "the tenant store is unavailable β€” nothing was saved")
612
 
613
  def _up(data):
614
  data = data if isinstance(data, dict) else {}
@@ -622,7 +824,7 @@ def save_nav_prefs(body: dict = Body(default=None),
622
  session.runtime.update(_NAV_PREFS_KEY, _up)
623
  except Exception:
624
  raise err(503, "store_unavailable",
625
- "the folder change was not saved β€” the store refused the write")
626
  return {"prefs": clean}
627
 
628
 
 
248
  return out[:_MAX_RECENTS]
249
 
250
 
251
+ # ── VIEW READERS β€” shared with `routes_starred` (W35-T39/T42) ────────────────────────────────
252
+ #
253
+ # β›”β›” W35-T43 (rulings R4/R7) β€” THE "MARK IMPORTANT" COUNTS ARE GONE FROM THIS ROUTE, and the
254
+ # block that computed them (`_important_counts`) went with them. It read one store bucket PER
255
+ # DATABASE on the route D-175 spent a whole wave reducing to a single read: MEASURED on a 13-database
256
+ # fixture, `GET /nav` performed **14 per-database view-bucket reads** and stamped `important` on
257
+ # **14** pages. It is **0 and 0** now.
258
+ #
259
+ # Those reads were bounded by a 2.5 s wall clock (`_IMPORTANT_BUDGET_S`) whose own note conceded the
260
+ # cost: twelve buckets are 6.2 ms WARM and **7,331 ms COLD**, so the budget existed to stop a cold
261
+ # container converting a slow success into a manufactured failure against `nav.ts`'s 20 s deadline.
262
+ # A budget that can be exhausted is a number that can be silently short, which is why it also had to
263
+ # publish a `degraded` entry. R7 removes the whole apparatus by moving the question to
264
+ # `GET /starred/counts`, called AFTER the page paints β€” where a slow answer costs a late badge
265
+ # instead of a late rail. Closes D-288 and D-289.
266
+ #
267
+ # ⚠ WHAT SURVIVES AND WHY: `_visible_views`, `_view_record_count` and `_granted_view_ids` are the
268
+ # READERS, and `routes_starred` calls all three. They were never the cost β€” the per-database store
269
+ # read was β€” and duplicating them into the new route would have put two answers behind one badge.
270
+
271
+ #: Suffix `core.view_templates.workspace_key` appends. Stripping it is how a page key becomes the
272
+ #: TOPIC key the cohort bucket is named from β€” `customer_data` -> `customer_table_workspace` ->
273
+ #: `customer` -> `customer_cohorts`. Derived rather than re-listed on purpose: `_WS_KEYS` is
274
+ #: already the one place `customer_data`/`product_data` are mapped to their topic, and a second
275
+ #: copy here would be free to drift from it ([[one-question-two-normalizers]]).
276
+ _WS_SUFFIX = "_table_workspace"
277
+
278
+
279
+ def _visible_views(doc, uname, is_admin, granted_ids):
280
+ """Every view on ONE database that THIS caller can see, from an already-read workspace doc.
281
+
282
+ β›” PER-CALLER, NOT TENANT-WIDE, and this is the half a server-side count is most likely to get
283
+ wrong. `<key>_table_workspace` is `{username: {views, fields, overlays}, '__shared__': {...}}`
284
+ β€” one home per view, never both β€” so "how many views are marked important here" has a
285
+ DIFFERENT answer for every account. Measured on tenant #0: `leadership` has one marked view and
286
+ `admin` has none, on the same database. A tenant-wide count would put a number in the owner's
287
+ rail that no view sidebar they can open would ever add up to.
288
+
289
+ ⚠ `table_store._may_see` is imported rather than re-expressed. It is private, and reaching for
290
+ it is still the right call: the alternative is `TableOps.shared_views`, which re-reads the whole
291
+ bucket per database (the N whole reads this block exists to avoid), and the only other option is
292
+ a second copy of a permission predicate. A wrong copy of `_may_see` widens what a user is told
293
+ exists; a private import cannot.
294
+ """
295
+ import core.table_store as table_store
296
+ out = {}
297
+ for vid, view in ((doc.get(uname) or {}).get("views") or {}).items():
298
+ if isinstance(view, dict):
299
+ out[str(vid)] = view
300
+ for vid, view in ((doc.get(table_store.SHARED_KEY) or {}).get("views") or {}).items():
301
+ if isinstance(view, dict) and table_store._may_see(view, uname, is_admin):
302
+ out.setdefault(str(vid), view)
303
+ # The wave-21 named-user grants. The ids come from ONE tenant-wide bucket read once for the
304
+ # whole request; the RECORD is already in hand, in whichever stratum its owner keeps it.
305
+ if granted_ids:
306
+ for stratum, blob in doc.items():
307
+ if stratum == uname or not isinstance(blob, dict):
308
+ continue
309
+ for vid, view in (blob.get("views") or {}).items():
310
+ if str(vid) in granted_ids and isinstance(view, dict):
311
+ out.setdefault(str(vid), view)
312
+ return out
313
+
314
+
315
+ def _view_record_count(cfg, cohorts):
316
+ """How many RECORDS this view resolves to, or None when that cannot be answered for free.
317
+
318
+ β›” `None` IS AN ANSWER AND IT IS THE IMPORTANT ONE. Two of the three shapes below are exact
319
+ because the view CARRIES its row set; the third β€” an ordinary filtered view β€” can only be
320
+ counted by running its filters over the records, and the records are the one thing this route
321
+ must never read (`D-175`/`D-185`: `/nav` is a rows-free projection, and reaching for `rows`
322
+ here raises by that projection's own contract). So a filtered view is reported as UNCOUNTED and
323
+ the database's `partial` flag says so, which is the whole of R6's second sentence applied to a
324
+ badge: a limit that cannot be removed is REPORTED with its cause, never papered over with a
325
+ number that is short by an unknown amount.
326
+
327
+ ⚠ THIS IS ALSO WHY THE SERVER DOES NOT SIMPLY MIRROR THE CLIENT. `CustomerGrid::alertCounts`
328
+ counts a filtered view fine and gives up on a SERVER-WINDOWED one (`D-205`'s `Important 0+`);
329
+ this end is the exact inverse β€” it has no rows at all and no window either. The two are honest
330
+ about different halves, which is why `partial` had to be on the wire rather than derived.
331
+ """
332
+ if not isinstance(cfg, dict):
333
+ return None
334
+ # A cohort-locked view IS its cohort: the lock and the id are the same fact (`grid_events`
335
+ # re-stamps it on every write), and a cohort's membership is a stored pid LIST, not a query.
336
+ lock = str(cfg.get("cohortLock") or "").strip()
337
+ if lock:
338
+ n = cohorts.get(lock)
339
+ return int(n) if isinstance(n, int) else None
340
+ # A curated row set carries its own count.
341
+ pids = cfg.get("memberPids")
342
+ if isinstance(pids, list) and pids:
343
+ return len(pids)
344
+ return None
345
+
346
+
347
+ def _visible_database_keys(session):
348
+ """`(keys, enumerated)` β€” every DATABASE this session may open that HAS a view bucket.
349
+
350
+ ⭐ ADDED BY W35-T39 because neither existing enumeration in this file answers this question:
351
+
352
+ Β· `_placeable_top_keys` applies the ACCOUNT grant and `may_open`, and **not the TENANT
353
+ CATALOGUE** β€” correctly, because a folder placement is cosmetic and pruning one is loss.
354
+ A star scan cannot borrow that: it would open the view bucket of a database this workspace's
355
+ catalogue does not include. `nav()` applies the catalogue filter; that helper never has.
356
+ Β· `nav()`'s own `_page_keys` is assembled from the page DICTS it is building for the wire, so
357
+ it cannot be reached from another route without rebuilding the payload.
358
+
359
+ So this is the KEY-SET question on its own, with the same three walls `nav()` applies in the same
360
+ order: the account grant (`perms.nav_pages`), the TENANT catalogue, and `may_open` for the
361
+ tenant's own databases. `view_templates.workspace_key` is the last filter and it is what makes
362
+ the answer honest for a caller about to read a view bucket: a module surface with no workspace
363
+ (`sales`, `ar`) is not a database and has no views to star.
364
+
365
+ ⚠ THE `parent` CLAUSE BELOW IS A NO-OP TODAY AND IS KEPT ONLY TO MIRROR `nav()`. `perms.nav_pages`
366
+ excludes `group_only` rows and therefore **emits no `parent` at all** (its own docstring says so:
367
+ a dangling reference would be worse than a flat list), so `customer_data` arrives here FLAT even
368
+ though the registry gives it `parent: 'customers'`. Stated because the opposite is the obvious
369
+ reading β€” this ticket's first draft assumed the clause was excluding children and wrote a
370
+ docstring around a defect that does not exist.
371
+
372
+ ⚠ `enumerated` is False when the `ut_` listing raised β€” the caller must not treat the set as
373
+ authoritative for `ut_` keys, exactly as `_placeable_top_keys` requires.
374
+ """
375
+ import core.view_templates as view_templates
376
+ keys, enumerated = set(), True
377
+ tcfg = getattr(session.runtime.tenant, "config", None) or {}
378
+ tmods = tcfg.get("modules", "all")
379
+ allowed = None if tmods == "all" else {str(k) for k in (tmods or [])}
380
+ for p in (perms.nav_pages(session.user) or []):
381
+ key = str(p.get("key") or "")
382
+ if allowed is not None and key not in allowed \
383
+ and str(p.get("parent") or "") not in allowed:
384
+ continue
385
+ if view_templates.workspace_key(key):
386
+ keys.add(key)
387
+ try:
388
+ import core.user_tables as user_tables
389
+ # The SAME `may_open`-filtered listing `nav()` renders from, over the ROWS-FREE projection
390
+ # (W32-T02) β€” so this costs ~0.1% of the document rather than a 28.6 MB deep copy.
391
+ for e in user_tables.nav_entries(viewer=session.uname, is_admin=session.admin,
392
+ st=session.runtime):
393
+ k = str(e.get("key") or "")
394
+ if k:
395
+ keys.add(k)
396
+ except Exception:
397
+ enumerated = False
398
+ return keys, enumerated
399
+
400
+
401
+ def _granted_view_ids(session):
402
+ """Every view id a wave-21 NAMED-USER GRANT lets this caller see. Fail-closed to `set()`.
403
+
404
+ ⭐ EXTRACTED (W35-T39) SO THE STAR AND THE COUNT SHARE ONE ANSWER. `_visible_views` above
405
+ takes this set as its third stratum, and it had exactly one caller (`_important_counts`) with
406
+ the derivation inline β€” which W35-T43 deletes. `routes_starred` needs the identical set to
407
+ answer "which views has this person starred", so the derivation moves here beside the reader
408
+ it feeds rather than being copied into a second file ([[one-evaluator-per-question]]).
409
+
410
+ ⚠ THE ROLE FILTER IS NOT BELT-AND-BRACES. `shared_with` answers "is there an entry naming
411
+ me", and `grid_events._granted_views` β€” the reader whose answer any consumer of this has to
412
+ agree with β€” then requires `role_for(...) in ('view','edit')`. Dropping that second test
413
+ would admit a view the sidebar does not list. One tenant-wide bucket, read once per call, so
414
+ it costs a dict lookup per id.
415
+ ⚠ Fail-closed: no grants is a NARROWER answer, never a wider one.
416
+ """
417
+ import core.shares as shares
418
+ try:
419
+ return {str(v) for v in
420
+ ((shares.shared_with(session.uname, kind="view", st=session.runtime) or {})
421
+ .get("view") or [])
422
+ if shares.role_for("view", str(v), session.uname,
423
+ st=session.runtime) in ("view", "edit")}
424
+ except Exception:
425
+ return set()
426
+
427
+
428
  @router.get("/nav")
429
  def nav(session: Session = Depends(require_session)):
430
  """`{pages: [{key,label,source?,chrome}], landing}`.
 
513
  # empty rather than leaving 200-[] ambiguous.
514
  return {"pages": [], "landing": None, "empty": "no_databases"}
515
  raise err(403, "no_surfaces",
516
+ "your account has no dashboards assigned. Ask an administrator.")
517
  # WAVE 19 (R8 / C1): the tenant's name + icon overrides, merged LAST β€” after the registry
518
  # rows, after the tenant module filter, after the user tables. Merged HERE rather than
519
  # applied by the client for one reason: `label` is what every reader of this payload shows,
 
527
  # leak with no symptom until two tenants rename the same registry key. Copy the rows before
528
  # merging on the day that invariant changes.
529
  meta = _read_nav_meta(session.runtime)
530
+ # β›”β›” W35-T43 (rulings R4/R7) β€” THE MARK-IMPORTANT COUNTS ARE NO LONGER COMPUTED HERE.
531
+ #
532
+ # W34-T10 put them on this payload with a 2.5 s budget and a `degraded` entry, spent in RECENTS
533
+ # ORDER so a cold container would reach the databases somebody actually works in first. Every
534
+ # one of those was a correct answer to the wrong question: the block was `N` per-database store
535
+ # reads on the route D-175 spent a wave reducing to ONE, and no ordering makes an N-read block
536
+ # free. MEASURED on a 13-database fixture: **14 view-bucket reads and 14 `important` stamps per
537
+ # `/nav`** before this ticket, **0 and 0** after.
538
+ #
539
+ # R7 moves the question to `GET /starred/counts`, called AFTER the page paints. A slow answer
540
+ # there is a late badge; a slow answer here was a late RAIL. Closes D-288 and D-289.
541
+ # ⚠ `recents` STAYS and is still ONE read β€” it is the Home landing's own payload (C10/R7 of wave
542
+ # 23), and it was only computed this early so the deleted budget could spend itself in its order.
543
+ _page_keys = {str(p.get("key", "")) for p in pages}
544
+ recents = _read_recents(session.runtime, session.uname, _page_keys)
545
  # Wave 21 (C3): the definitions, once β€” `manage`/`canDelete` below answer from `createdBy`.
546
  # WAVE 27 (C9): `locked` answers from `recordMode`, off the same one read.
547
  # ⭐ W31-T10: that read is now the SAME one the merge above did β€” it used to be a second,
 
590
  p["locked"] = True
591
  elif session.admin:
592
  p["manage"] = True
593
+ # β›” W35-T43 β€” `important` IS NO LONGER STAMPED HERE. W34-T10 emitted it for every database
594
+ # this route could read, including `{marked: 0, counted: 0, partial: false}`, so a consumer
595
+ # never had to test for the key. R4 retires the count from the rail and the flyout entirely
596
+ # (A's W35-T07 is the client half) and R7 moves the numbers to `GET /starred/counts`.
597
+ # ⚠ `nav.ts::NavPage` still DECLARES `important` on the client until A's ticket lands; an
598
+ # absent key reads as `undefined` there, which is the same thing the optional field already
599
+ # meant for a non-database row. Flagged to A rather than assumed harmless.
600
  entry = meta.get(p.get("key")) if meta else None
601
  if not entry:
602
  continue
 
621
  # both doors finally agree on what a placeable key is. This block keeps using the assembled
622
  # list because it already holds it: re-enumerating here would be a second store read for an
623
  # answer sitting in a local variable.
624
+ #
625
+ # ⚠ W34-T10 MOVED THE READ, NOT THE RULE. `recents` is now computed ABOVE the enrichment loop,
626
+ # because the important-count block spends its budget in RECENTS ORDER (see there). It is still
627
+ # ONE read of `nav_recents`, still pruned against the assembled page list, and it is used here
628
+ # unchanged β€” a second `_read_recents` call would be the extra store read this comment forbids.
629
  # ⭐ W31-T11 β€” TWO KINDS OF ABSENCE, NAMED SEPARATELY, and both keys are ALWAYS PRESENT.
630
  # `omitted` β€” this workspace's catalogue does not include these modules. Deliberate.
631
  # `degraded` β€” a part of this payload could not be read. NOT deliberate, and the shell says
 
662
  raise err(400, "bad_request", "no page was named")
663
  if not session.runtime.available():
664
  raise err(503, "store_unavailable",
665
+ "the tenant store is unavailable. Nothing was recorded.")
666
  stamp, uname = _now(), session.uname
667
 
668
  def _up(data):
 
686
  session.runtime.update(_NAV_RECENTS_KEY, _up, flush='async')
687
  except Exception:
688
  raise err(503, "store_unavailable",
689
+ "the tenant store refused the write. Nothing was recorded.")
690
  return {"key": key, "at": stamp}
691
 
692
 
 
751
  if "name" in body:
752
  if not key.startswith("ut_"):
753
  raise err(400, "name_not_allowed",
754
+ "only a database you created can be renamed. This one's name comes "
755
  "from the module registry")
756
  name = " ".join(str(body.get("name") or "").split())[:_MAX_NAV_NAME]
757
  if not name:
 
762
 
763
  if not session.runtime.available():
764
  raise err(503, "store_unavailable",
765
+ "the tenant store is unavailable. Nothing was saved.")
766
 
767
  def _up(data):
768
  data = data if isinstance(data, dict) else {}
 
785
  session.runtime.update(_NAV_META_KEY, _up)
786
  except Exception:
787
  raise err(503, "store_unavailable",
788
+ "the change was not saved: the store refused the write.")
789
  return {"key": key, "meta": _read_nav_meta(session.runtime).get(key, {})}
790
 
791
 
 
810
  clean = _clean_nav_prefs(body or {}, keys, keep_unknown_ut=not enumerated)
811
  if not session.runtime.available():
812
  raise err(503, "store_unavailable",
813
+ "the tenant store is unavailable. Nothing was saved.")
814
 
815
  def _up(data):
816
  data = data if isinstance(data, dict) else {}
 
824
  session.runtime.update(_NAV_PREFS_KEY, _up)
825
  except Exception:
826
  raise err(503, "store_unavailable",
827
+ "the folder change was not saved: the store refused the write.")
828
  return {"prefs": clean}
829
 
830
 
api/routes_odoo_tables.py CHANGED
@@ -841,7 +841,18 @@ def odoo_table_rows(table_key: str,
841
  # THE WALL FIRST, and it is the SAME one the materialised path uses β€” 404 for a key that does
842
  # not exist, 403 for one this session may not open. Reused rather than re-stated: a second
843
  # idea of "may this session open this database" is a permission bug waiting to happen.
844
- defn = _defn_or_refuse(session, table_key)
 
 
 
 
 
 
 
 
 
 
 
845
  fields = None
846
  for _bucket, key, _label, mk_fields in rel.TABLES:
847
  if key == table_key:
 
841
  # THE WALL FIRST, and it is the SAME one the materialised path uses β€” 404 for a key that does
842
  # not exist, 403 for one this session may not open. Reused rather than re-stated: a second
843
  # idea of "may this session open this database" is a permission bug waiting to happen.
844
+ #
845
+ # ⭐⭐ W33-T02 / D-213 β€” `defs_only=True`, AND THIS ROUTE IS THE CLEANEST OPT-IN ON THE BOARD.
846
+ # `defn` is read EXACTLY ONCE below, for `recordMode`; the fields come from `rel.TABLES`'s
847
+ # `mk_fields()` and the rows from `datastore.window`, so nothing here has ever touched
848
+ # `defn["rows"]`. That is what makes it safe by inspection rather than by argument β€” and it is
849
+ # why the same change must NOT be swept across this file: `odoo_tables_status` a few hundred
850
+ # lines down reads `table["rows"]` and its per-row `refreshed` stamps for the MATERIALISED
851
+ # grids, and a projection there raises `KeyError` on every one of them.
852
+ # ⚠ `mirror_stamp`'s bare `except: return ""` would convert exactly that raise into a silently
853
+ # BLANK `refreshed` column rather than a red β€” which is why the boundary is drawn here, at the
854
+ # one function that provably needs no row, instead of at the file.
855
+ defn = _defn_or_refuse(session, table_key, defs_only=True)
856
  fields = None
857
  for _bucket, key, _label, mk_fields in rel.TABLES:
858
  if key == table_key:
api/routes_publish.py ADDED
@@ -0,0 +1,917 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """routes_publish.py β€” PUBLISH AN INTERFACE VIEW AS A LINK (wave 33, owner item 8b, ruling R5).
2
+
3
+ The owner, verbatim: *"Have the ability to publish interface, when a user's view is an interface
4
+ (e.g. Map or Catalog), we should have the ability to publish a link that can be either accessed
5
+ publicly or with a password that the user that share it can toggle."*
6
+
7
+ R5, in five clauses, and every one of them is load-bearing:
8
+ * publish = a per-view secret token in a **server-only bucket**;
9
+ * a `public | password` toggle the SHARER owns, with a **hashed** passphrase beside it;
10
+ * an **unauthenticated** read-only route plus a `#/v/<token>` client route;
11
+ * the publish surface **projects only the columns the view shows** β€” never the whole table;
12
+ * **creator-or-admin** may publish, and **revoke ROTATES** the token.
13
+
14
+ β›” THIS FILE COPIES `routes_forms.py`'S CONSTRUCTION ON PURPOSE, AND THE TICKET SAID TO. That door
15
+ has been public since wave 23 and carries scar tissue no fresh design would reproduce: ONE refusal
16
+ for every resolution failure (so the route is not an oracle for which tokens are real), a
17
+ constant-time compare, a STREAMING body bound rather than `Body(...)` (FastAPI reads and parses the
18
+ whole body before the handler's first line, and `content-length` is caller-supplied), a sliding
19
+ rate window rather than a fixed bucket, and an index that is a POINTER, never a permission.
20
+
21
+ ⚠ WHAT IS DELIBERATELY *NOT* SHARED WITH `routes_forms.py`: the bucket. A form token buys a WRITE
22
+ door into a table; a publish token buys a READ projection of one view. Folding them into one index
23
+ would make a single leaked string ambiguous about which of the two it opens, and would make
24
+ `_resolve` return an object whose capability depends on a field rather than on which door was
25
+ knocked. Two buckets, two resolvers, one shape.
26
+
27
+ ⚠ AND NOT SHARED WITH SHARING. **Three systems now answer some version of "who can see this"** and
28
+ they are not the same question (audit S-4): the grant registry drives *Shared with me*,
29
+ `table_store.is_shared` GATES opening a view inside the app, and a published link is a THIRD thing
30
+ β€” an unauthenticated read of a projection, bound to a secret rather than to an account. A published
31
+ link is NOT a grant: it creates no registry row, appears in nobody's *Shared with me*, and cannot
32
+ be revoked by removing a person, because there is no person.
33
+
34
+ python verify_forms.py # this file's gate; section 6 onward
35
+ """
36
+ import hashlib
37
+ import hmac
38
+ import os
39
+ import secrets
40
+ import time
41
+
42
+ from fastapi import APIRouter, Depends, Request
43
+
44
+ from deps import Session, err, require_session
45
+
46
+ router = APIRouter(prefix="/api/v1")
47
+
48
+ #: The SERVER-ONLY index: `{token: {table, view, access, pw?, salt?, iter?, createdBy, createdAt}}`.
49
+ #: β›” IT IS NEVER `display.*`. Contract C1 puts two PRESENTATIONAL flags on the view spec
50
+ #: (`published`, `publishAccess`) precisely so the client has something to render, and the browser
51
+ #: writes that spec on every autosave β€” so anything secret living there would be echoed back to the
52
+ #: browser by construction. `aios_grid._clean_display`'s allowlist enforces the other half.
53
+ TOKENS_KEY = "publish_tokens"
54
+ TOKEN_BYTES = 24
55
+
56
+ #: Passphrase storage. PBKDF2-HMAC-SHA256 with a per-link salt.
57
+ #: ⚠ D-130 IS THE SCAR THIS AVOIDS: a form's "invited addresses" list is IDENTIFICATION β€” it says
58
+ #: who you claim to be and anyone may claim it. A passphrase is AUTHENTICATION. The difference is
59
+ #: not a stronger string, it is that the secret is never stored, never logged and never echoed.
60
+ PW_ITERATIONS = 240_000
61
+ PW_SALT_BYTES = 16
62
+ MIN_PASSPHRASE = 6
63
+ MAX_PASSPHRASE = 128
64
+
65
+ #: v1 request protections, the same shape and the same numbers as the form door (contract C9), so
66
+ #: the two public routes cannot drift into different postures. ⚠ In-process, therefore PER WORKER.
67
+ RATE_WINDOW_S = 60
68
+ RATE_PER_WINDOW = 30
69
+ MAX_BODY_BYTES = 16 * 1024
70
+
71
+ _HITS: dict = {}
72
+
73
+ #: The view kinds that may be published, i.e. R5's "interface".
74
+ #: ⚠ THE CLIENT'S SOURCE OF TRUTH IS `customer-grid/iconShapes.ts::MODE_GROUP` (wave 33 item 9
75
+ #: moved `swipe` and `timeseries` into this group). This constant is the SERVER's copy and the two
76
+ #: are held in step by `verify_forms.py`, which parses `MODE_GROUP` and compares β€” because a
77
+ #: server list that silently drifts from the picker means a mode a user can create and cannot
78
+ #: publish, with nothing anywhere going red. A `grid` or `kanban` view is a re-shaping of a row
79
+ #: set; publishing one would be publishing the table, which is exactly what R5's projection clause
80
+ #: exists to prevent.
81
+ #: β›”β›” W33-T68 β€” `form` IS DELIBERATELY ABSENT, AND ITS ABSENCE IS THE FIX.
82
+ #: A `form` view's rows ARE the submissions people have sent it. Every other mode here re-shapes a
83
+ #: row set the publisher already curated; a form's row set is other people's answers, gathered under
84
+ #: an implicit promise that they go to the owner. Publishing one turned "share this interface" into
85
+ #: "serve the responses to anyone holding the link" β€” on the one unauthenticated door in the
86
+ #: product, with no wall between the link and the data.
87
+ #: ⚠ AND A FORM ALREADY HAS ITS OWN PUBLIC DOOR: `#/form/<token>` through `routes_forms.py`, which
88
+ #: serves the BLANK form for submitting and never the stored rows. So this is not a capability
89
+ #: removed, it is a second door onto the same object that should never have existed beside the
90
+ #: first. Publishing a form to be filled in still works, at the URL that was always for it.
91
+ #: ⚠ `verify_forms.py` holds this list in step with the client's `MODE_GROUP`; the client must not
92
+ #: offer Publish on a form view, or the picker promises what this refuses.
93
+ PUBLISHABLE_MODES = ("map", "catalog", "swipe", "timeseries")
94
+
95
+
96
+ def _refuse():
97
+ """THE ONE REFUSAL, for every way a published link can fail to resolve.
98
+
99
+ Wrong token, revoked link, deleted view, deleted table, a view that stopped being an
100
+ interface, a wrong passphrase. β›” Callers must not branch a more specific message out of it:
101
+ the difference between "no such link" and "that link was revoked" tells an enumerator which
102
+ tokens are real, and the difference between "no such link" and "wrong passphrase" tells them
103
+ which links are worth guessing at.
104
+ """
105
+ return err(403, "bad_publish_token", "that link is not valid")
106
+
107
+
108
+ def _same(a: str, b: str) -> bool:
109
+ """Constant-time. A `==` leaks a secret's prefix through timing, one character at a time."""
110
+ return hmac.compare_digest(str(a or ""), str(b or ""))
111
+
112
+
113
+ def _client_ip(request: Request) -> str:
114
+ """β›”β›” W33-T70 β€” `x-forwarded-for` IS GONE FROM THIS FUNCTION, AND THAT WAS THE WHOLE HOLE.
115
+
116
+ The header is chosen by the caller. Keying a rate limit on it means an enumerator writes a new
117
+ value per request and every request lands in a fresh bucket: MEASURED at **100 of 100 admitted
118
+ through a 30-per-window ceiling**. A limiter with a caller-chosen key is not a limiter, and its
119
+ old docstring said the header was "a rate-limit key and NOTHING else" β€” which was exactly the
120
+ use it could not support. It is safe as a LOG field and as nothing else.
121
+
122
+ The socket peer is the only thing here the caller cannot choose, so it is the key.
123
+ ⚠ AND BEHIND HF'S PROXY EVERY CALLER SHARES ONE PEER, which is why `_rate_ok` counts FAILURES
124
+ ONLY (see there). A per-peer ceiling over ALL traffic would be one global bucket, i.e. an alarm
125
+ that fires for everyone [[alarm-that-fires-for-everyone]] β€” the honest reading of a shared peer
126
+ is that we cannot separate callers, not that we should throttle them together.
127
+ """
128
+ return request.client.host if request.client else "?"
129
+
130
+
131
+ def _rate_ok(key: str, now: float) -> bool:
132
+ """Is this caller UNDER the failure ceiling? Pure check β€” call `_note_failure` to count.
133
+
134
+ A SLIDING window. A fixed bucket lets a caller spend a whole allowance at 11:59:59 and the
135
+ whole next one at 12:00:00 β€” i.e. double the limit, back to back, against a door whose entire
136
+ protection is that guessing a 24-byte token is slow.
137
+
138
+ β›” W33-T70 β€” IT COUNTS FAILURES, NOT REQUESTS, and the split is what makes a shared proxy peer
139
+ survivable. A legitimate reader opens a link that RESOLVES, so they never touch the counter at
140
+ all; an enumerator produces nothing but misses. Counting every request under one shared peer
141
+ would have denied service to everybody the moment one guesser showed up β€” trading a token
142
+ oracle for an outage is not a fix.
143
+ """
144
+ seen = [t for t in _HITS.get(key, ()) if now - t < RATE_WINDOW_S]
145
+ _HITS[key] = seen
146
+ return len(seen) < RATE_PER_WINDOW
147
+
148
+
149
+ def _note_failure(key: str, now: float) -> None:
150
+ """Record one failed resolution against this peer, and keep the table bounded."""
151
+ seen = [t for t in _HITS.get(key, ()) if now - t < RATE_WINDOW_S]
152
+ seen.append(now)
153
+ _HITS[key] = seen
154
+ if len(_HITS) > 4096:
155
+ for k in [k for k, v in _HITS.items() if not v or now - v[-1] > RATE_WINDOW_S]:
156
+ _HITS.pop(k, None)
157
+
158
+
159
+ #: A salt used ONLY to burn the same PBKDF2 a real verification would, when there is no stored hash
160
+ #: to check against. Its value is irrelevant; its COST is the point.
161
+ _DUMMY_SALT = b"\x00" * 16
162
+
163
+
164
+ def _equalise_pw_cost(entry) -> None:
165
+ """β›”β›” W33-T70 β€” SPEND THE PBKDF2 EVEN WHEN THERE IS NOTHING TO VERIFY.
166
+
167
+ MEASURED before this existed: a wrong TOKEN answered in ~0.4 ms and a wrong PASSPHRASE in
168
+ ~82 ms β€” a **206x gap with zero overlap across 24 samples**. So the refusal's careful wording
169
+ ("that link is not valid", identical for both) was undone by the clock: anyone could sort real
170
+ tokens from fake ones by timing alone, then spend their guesses only on the real ones. The
171
+ docstring on `_refuse` describes exactly the leak the code then handed over for free.
172
+
173
+ Called on every path that fails BEFORE a passphrase check would have happened, so the cheap
174
+ branch costs what the expensive one costs. `iter` is taken from the entry when there is one, so
175
+ a link minted under a different iteration count stays indistinguishable too.
176
+ """
177
+ iterations = PW_ITERATIONS
178
+ if isinstance(entry, dict):
179
+ try:
180
+ iterations = int(entry.get("iter") or PW_ITERATIONS)
181
+ except (TypeError, ValueError):
182
+ iterations = PW_ITERATIONS
183
+ _hash_pw("", _DUMMY_SALT, iterations)
184
+
185
+
186
+ def _public_base() -> str:
187
+ return (os.environ.get("AIOS_PUBLIC_BASE") or os.environ.get("APP_BASE_URL") or "").rstrip("/")
188
+
189
+
190
+ def _link_of(token: str) -> str:
191
+ return f"{_public_base()}/#/v/{token}"
192
+
193
+
194
+ def _tokens(rt) -> dict:
195
+ """This tenant's publish index, always a dict."""
196
+ try:
197
+ found = rt.get(TOKENS_KEY) or {}
198
+ except Exception: # noqa: BLE001
199
+ return {}
200
+ return found if isinstance(found, dict) else {}
201
+
202
+
203
+ def _hash_pw(passphrase: str, salt: bytes, iterations: int = PW_ITERATIONS) -> str:
204
+ return hashlib.pbkdf2_hmac("sha256", str(passphrase).encode("utf-8"),
205
+ salt, int(iterations)).hex()
206
+
207
+
208
+ def _pw_ok(entry: dict, passphrase: str) -> bool:
209
+ """Constant-time verify against the stored digest.
210
+
211
+ β›” RETURNS FALSE, NEVER RAISES, and never distinguishes "this link has no passphrase stored"
212
+ from "the passphrase is wrong" β€” a `password` link whose hash went missing must refuse, not
213
+ fall open. [[default-must-pass-its-own-guard]]
214
+ """
215
+ stored, salt_hex = str(entry.get("pw") or ""), str(entry.get("salt") or "")
216
+ if not stored or not salt_hex:
217
+ return False
218
+ try:
219
+ salt = bytes.fromhex(salt_hex)
220
+ except ValueError:
221
+ return False
222
+ got = _hash_pw(passphrase, salt, int(entry.get("iter") or PW_ITERATIONS))
223
+ return _same(got, stored)
224
+
225
+
226
+ def _find_view(rt, table_key: str, view_id: str):
227
+ """`(view, owner_username)` from the table's workspace bucket, or `(None, "")`.
228
+
229
+ The bucket shape is `{username: {views: {id: view}}}` β€” the same walk `routes_forms._find_view`
230
+ does, and the reason a view is addressable by id ALONE here: the id is unique within the table,
231
+ while the owner is what we are trying to discover.
232
+ """
233
+ if not table_key or not view_id:
234
+ return None, ""
235
+ try:
236
+ bucket = rt.get(f"{table_key}_table_workspace") or {}
237
+ except Exception: # noqa: BLE001
238
+ return None, ""
239
+ if not isinstance(bucket, dict):
240
+ return None, ""
241
+ for owner, blob in bucket.items():
242
+ views = (blob or {}).get("views") if isinstance(blob, dict) else None
243
+ if isinstance(views, dict) and isinstance(views.get(view_id), dict):
244
+ return views[view_id], str(owner)
245
+ return None, ""
246
+
247
+
248
+ def _mode_of(view: dict) -> str:
249
+ """The view's display mode. Absent means `grid` β€” the stored default is "say nothing"."""
250
+ cfg = (view or {}).get("config") if isinstance(view, dict) else None
251
+ disp = (cfg or {}).get("display") if isinstance(cfg, dict) else None
252
+ return str((disp or {}).get("mode") or "grid") if isinstance(disp, dict) else "grid"
253
+
254
+
255
+ def _may_administer_view(session: Session, table_key: str, view_id: str):
256
+ """`(view, mode)` when this caller may publish THIS view, else raises. R5's creator-or-admin.
257
+
258
+ β›” `session.uname` / `session.admin`, NOT `username` / `is_admin`. `Session` has no such
259
+ attributes, and reading one raises ABOVE this route's own `try:` β€” which is how D-107 arrived
260
+ as a bare plain-text 500 rather than as our JSON envelope.
261
+ """
262
+ import core.table_store as table_store
263
+
264
+ if not table_key.startswith("ut_"):
265
+ # The Odoo/registry grids are read-through mirrors with their own permission wall; a
266
+ # published projection of one would be a second, secret-gated door onto tenant data whose
267
+ # visibility the module gate is supposed to decide.
268
+ raise err(400, "not_a_database", "you can publish views on your own databases only")
269
+ # β›” A READ-THROUGH DATABASE CANNOT BE PUBLISHED, AND THE REFUSAL SAYS SO RATHER THAN MINTING
270
+ # A LINK THAT WILL NOT OPEN. `startswith("ut_")` admits `ut_odoo_*` and `ut_meta_*`, whose rows
271
+ # do not live in the tenant document at all β€” they are windowed out of the DuckDB mirror
272
+ # through a Session-bound path (`ut_assembly`, which 409s `window_required` on the big ones,
273
+ # D-174's lineage). An unauthenticated route has no session and therefore no such path, so a
274
+ # token minted here would resolve to a page that could never render.
275
+ # ⚠ THIS IS R6's SECOND SENTENCE, WHICH IS THE HALF THAT GETS DROPPED: a limit that genuinely
276
+ # cannot be removed must be REPORTED, with its cause, never silently enforced. Refusing at the
277
+ # MINT β€” where a person is standing in front of the answer β€” is the only place that reads as a
278
+ # sentence rather than as an empty page.
279
+ import core.user_tables as user_tables
280
+
281
+ if user_tables.is_connected(table_key, st=session.runtime):
282
+ raise err(400, "connected_source",
283
+ "a database that reads through a connected source (Odoo, Meta Ads) cannot be "
284
+ "published as a public link: its rows are served from the tenant's mirror by a "
285
+ "signed-in request, and a public link has no session to serve them with")
286
+ view, owner = _find_view(session.runtime, table_key, view_id)
287
+ if not isinstance(view, dict):
288
+ raise err(404, "no_such_view", "that view no longer exists")
289
+ if not (owner == session.uname
290
+ or table_store._may_administer(view, session.uname, session.admin)):
291
+ raise err(403, "not_yours", "only this view's creator or an admin can publish it")
292
+ mode = _mode_of(view)
293
+ if mode not in PUBLISHABLE_MODES:
294
+ raise err(400, "not_an_interface",
295
+ "only an interface view (Map, Catalog, Swipe, Time-series, Form) can be "
296
+ "published as a link")
297
+ cfg = view.get("config") or {}
298
+ if cfg.get("cohortLock"):
299
+ raise err(400, "reader_scoped",
300
+ "this view is locked to a cohort, and a cohort's membership is resolved for "
301
+ "the person reading it β€” a public link has no reader, so the page would show "
302
+ "no rows at all. Publish a copy without the cohort lock.")
303
+ if _needs_a_reader(cfg.get("filters")):
304
+ raise err(400, "reader_scoped",
305
+ "this view filters on a cohort, a measure rule or a top-N slice, and each of "
306
+ "those is resolved for the person reading it β€” a public link has no reader, so "
307
+ "the page would show no rows at all. Publish a copy filtered on columns.")
308
+ return view, mode
309
+
310
+
311
+ #: How many rows a published page will serve. ⚠ R6 (no cap on connected-source data) does not
312
+ #: reach here twice over: a publishable database is `records_mutable`, i.e. the EDITABLE substrate
313
+ #: that keeps its bound (`user_tables.MAX_ROWS`), and a connected one is refused at the mint. What
314
+ #: R6's SECOND sentence does reach here is the reporting duty β€” a page that serves fewer rows than
315
+ #: the view has must SAY SO on the wire, with the cause, never just stop.
316
+ PUBLIC_ROW_CAP = 5000
317
+
318
+
319
+ def _needs_a_reader(tree) -> bool:
320
+ """Does this filter tree contain a leaf that only a SIGNED-IN reader could resolve?
321
+
322
+ β›” COHORTS, MEASURE RULES AND RANK SLICES ARE NOT PROPERTIES OF THE VIEW. Each is a SET the
323
+ host computes for the person asking β€” `filter_eval.EvalCtx`'s own docstring: *"each is an
324
+ answer a single ROW cannot compute … absent, the condition matches NOTHING rather than
325
+ everything"*. That default is right (it fails closed) and it is unusable here: an anonymous
326
+ page whose every row was silently filtered out is indistinguishable from a broken link, and
327
+ the reader has nobody to ask. Worse, `rank_sets` has NO server-side resolver anywhere in this
328
+ repo β€” `routes_alerts._evaluate` passes cohort/measure/today and nothing else β€” so a `topN`
329
+ view would serve zero rows on the server while showing twenty in the browser.
330
+
331
+ ⭐ So such a view is refused AT THE MINT, where a person is standing in front of the answer.
332
+ That is R6's second sentence: a limit that genuinely cannot be removed is REPORTED with its
333
+ cause, never silently enforced.
334
+
335
+ ⚠ THE PREDICATES ARE `filter_eval`'S OWN. Re-implementing "is this a cohort leaf?" here would
336
+ be a second evaluator that agrees today and drifts the day the leaf shape changes β€” and it
337
+ would drift SILENTLY, because the two answers only differ on views nobody has published yet.
338
+ [[one-evaluator-per-question]]
339
+ """
340
+ from harness import filter_eval
341
+
342
+ def walk(node) -> bool:
343
+ if isinstance(node, list):
344
+ return any(walk(n) for n in node)
345
+ if not isinstance(node, dict):
346
+ return False
347
+ if node.get("kind") == "group" or isinstance(node.get("children"), list):
348
+ return any(walk(n) for n in (node.get("children") or []))
349
+ if filter_eval._is_cohort(node) or filter_eval._is_measure(node):
350
+ return True
351
+ return node.get("op") in filter_eval.RANK_OPS
352
+
353
+ return walk(tree)
354
+
355
+
356
+ def _mirror_display(rt, table_key: str, view_id: str, published: bool, access: str = "password"):
357
+ """Keep contract C1's two PRESENTATIONAL flags on the stored view in step with this bucket.
358
+
359
+ β›” WHY THIS EXISTS AT ALL, and it was found by a reviewer rather than by a gate: revoking a
360
+ link dropped the token and left `config.display.published: true` on the view, so the grid's
361
+ own UI would go on saying "published" about a link that no longer resolves. Two records of one
362
+ fact, and only one of them moving, is the [[flag-shipped-without-its-writer]] shape β€” here with
363
+ the writer present and the OTHER half forgotten.
364
+
365
+ ⚠ THE BUCKET REMAINS THE TRUTH. These flags exist so the client has something to render
366
+ without asking; they are a MIRROR, never a source, and nothing in this module reads them back
367
+ to decide anything. `read_view_link` deliberately reports both so a drift is visible rather
368
+ than assumed away.
369
+
370
+ ⚠ ONLY THE TWO LEGAL KEYS ARE WRITTEN, with E's fail-closed coercion reproduced exactly
371
+ (`aios_grid._clean_display`: `published: True` always carries a `publishAccess`, and anything
372
+ that is not the literal `public` stores as `password`) β€” so a value written here and a value
373
+ written by the browser cannot disagree.
374
+ """
375
+ def _set(cur):
376
+ cur = dict(cur or {})
377
+ for owner, blob in list(cur.items()):
378
+ views = (blob or {}).get("views") if isinstance(blob, dict) else None
379
+ view = views.get(view_id) if isinstance(views, dict) else None
380
+ if not isinstance(view, dict):
381
+ continue
382
+ cfg = dict(view.get("config") or {})
383
+ disp = dict(cfg.get("display") or {})
384
+ if not disp.get("mode"):
385
+ # No display block means no interface view; nothing here should invent one.
386
+ continue
387
+ if published:
388
+ disp["published"] = True
389
+ disp["publishAccess"] = "public" if access == "public" else "password"
390
+ else:
391
+ disp.pop("published", None)
392
+ disp.pop("publishAccess", None)
393
+ cfg["display"] = disp
394
+ cur[owner] = {**blob, "views": {**views, view_id: {**view, "config": cfg}}}
395
+ return cur
396
+
397
+ try:
398
+ rt.update(f"{table_key}_table_workspace", _set, flush="sync")
399
+ except Exception: # noqa: BLE001
400
+ # ⚠ SWALLOWED, and deliberately: the token bucket is the truth and it has already been
401
+ # written. A mirror that failed to update leaves the UI one refresh out of date, which is
402
+ # strictly better than a 500 on a publish that actually succeeded.
403
+ pass
404
+
405
+
406
+ def _entry_for(rt, table_key: str, view_id: str):
407
+ """`(token, entry)` for a view's existing link, or `(None, None)`."""
408
+ for token, where in _tokens(rt).items():
409
+ if (isinstance(where, dict) and where.get("table") == table_key
410
+ and where.get("view") == view_id):
411
+ return str(token), where
412
+ return None, None
413
+
414
+
415
+ def _state_of(token, entry) -> dict:
416
+ """The link's state as the SHARER may see it. Built key by key.
417
+
418
+ β›” NO `**entry`. The stored blob carries `pw` and `salt`; a spread would put both on the wire
419
+ to the browser, which is the whole failure this file's bucket exists to avoid, and it would do
420
+ it silently the first time somebody added a field.
421
+ """
422
+ if not token or not isinstance(entry, dict):
423
+ return {"published": False, "access": "password", "token": "", "url": ""}
424
+ return {
425
+ "published": True,
426
+ "access": "public" if entry.get("access") == "public" else "password",
427
+ "token": str(token),
428
+ "url": _link_of(str(token)),
429
+ # A boolean, never the digest and never the salt.
430
+ "hasPassphrase": bool(entry.get("pw")),
431
+ "createdBy": str(entry.get("createdBy") or ""),
432
+ }
433
+
434
+
435
+ def _resolve(token: str):
436
+ """`(runtime, tenant_slug, table_key, view, entry)` for a token, or None.
437
+
438
+ β›” THE INDEX IS A POINTER, NEVER A PERMISSION. Holding a token means the sharer minted it for
439
+ THIS view; it does not mean the holder may read anything else, and nothing downstream of here
440
+ may widen the subject beyond the `(table, view)` pair the entry names.
441
+
442
+ ⚠ IT WALKS EVERY TENANT, because an unauthenticated request carries no tenant. That is the
443
+ form door's shape too, and the reason `_same` is constant-time: the walk compares the caller's
444
+ string against every stored token in the deployment.
445
+ """
446
+ from harness import runtime as _rt
447
+
448
+ if not token or len(token) < 16:
449
+ return None
450
+ for slug in _rt.known_tenants():
451
+ try:
452
+ rt = _rt.get_runtime(slug)
453
+ except Exception: # noqa: BLE001
454
+ continue
455
+ for stored, where in _tokens(rt).items():
456
+ if not _same(str(stored), token) or not isinstance(where, dict):
457
+ continue
458
+ table_key = str(where.get("table") or "")
459
+ view, _owner = _find_view(rt, table_key, str(where.get("view") or ""))
460
+ # A view deleted, or re-saved as a grid, since the link was minted. Both answer the
461
+ # ONE refusal β€” "that link is not valid" β€” rather than explaining which.
462
+ if not table_key or not isinstance(view, dict):
463
+ return None
464
+ if _mode_of(view) not in PUBLISHABLE_MODES:
465
+ return None
466
+ return rt, slug, table_key, view, where
467
+ return None
468
+
469
+
470
+ def _coord(value, limit: float):
471
+ """A real coordinate, or `None`. THE WALL that lets a map publish without publishing a column.
472
+
473
+ β›” A VALUE TEST, NEVER A NAME TEST. A field merely CALLED `lat` proves nothing β€” a verifier put
474
+ the string `CANARY-LAT-AAA` in one and watched it reach the wire under the first version of
475
+ this code. Anything that is not a finite number inside the earth's range is not a location and
476
+ does not travel.
477
+ ⚠ `bool` is rejected explicitly: `isinstance(True, int)` is True in Python, and `float(True)`
478
+ is `1.0` β€” a checkbox column named `lat` would otherwise publish as a point off the coast of
479
+ Ghana.
480
+ """
481
+ if value is None or isinstance(value, bool):
482
+ return None
483
+ try:
484
+ n = float(value)
485
+ except (TypeError, ValueError):
486
+ return None
487
+ return n if n == n and abs(n) <= limit else None
488
+
489
+
490
+ def _visible_keys(view: dict, fields: list) -> list:
491
+ """The columns this view SHOWS, in the view's own order. R5's projection clause, in one place.
492
+
493
+ β›” `config.order` IS NOT THE ANSWER AND IS THE OBVIOUS WRONG ONE. `aios_grid._default_view_config`
494
+ builds `order` as `shown + hidden`, and `grid_events`' `view_upsert` APPENDS every remaining
495
+ field to it β€” so `order` is every column the table has, hidden ones included. `visible` is the
496
+ only allowlist that exists; there is no stored "hidden" key to subtract.
497
+
498
+ ⚠ AND AN EMPTY `visible` IS NOT "NO COLUMNS". `CustomerGrid` falls back to the table's default
499
+ set when a saved view carries none, so a public page that read `[]` as an empty allowlist would
500
+ render blank β€” and one that read it as "all fields" would LEAK. The fallback is the same
501
+ predicate the default config uses (`field.default is not False`), taken from `aios_grid` rather
502
+ than restated here.
503
+ """
504
+ import aios_grid
505
+
506
+ by_key = {str(f.get("key")): f for f in fields if isinstance(f, dict)}
507
+ stored = [str(k) for k in (((view or {}).get("config") or {}).get("visible") or [])]
508
+ keep = [k for k in stored if k in by_key]
509
+ if keep:
510
+ return keep
511
+ # β›”β›” W33-T69 β€” A STALE `visible` MUST SERVE NOTHING, NOT THE TABLE DEFAULT.
512
+ # `delete_field` never prunes a view's stored `visible`, so a published view whose columns were
513
+ # later deleted and replaced arrives here with a NON-EMPTY `stored` of which nothing survives β€”
514
+ # and the fallback below then WIDENS the public payload to whatever the table declares by
515
+ # default. The publisher chose five columns; the anonymous reader gets the table's idea of
516
+ # sensible. That is a widening on the one unauthenticated door in the product.
517
+ # ⚠ THE DISTINCTION IS `stored` NON-EMPTY, NOT `keep` EMPTY. A view that never stored `visible`
518
+ # at all (a legacy publish, a view saved before the key existed) has no intent to honour and
519
+ # the default IS the right answer for it β€” that is what the fallback was written for. A view
520
+ # that stored five keys and has none left DID state an intent, and every column it named is
521
+ # gone: the honest answer is no columns, which renders as an empty published view rather than
522
+ # somebody else's data.
523
+ if stored:
524
+ return []
525
+ try:
526
+ default_visible = (aios_grid._default_view_config(fields) or {}).get("visible") or []
527
+ except Exception: # noqa: BLE001
528
+ default_visible = []
529
+ fallback = [str(k) for k in default_visible if str(k) in by_key]
530
+ return fallback or [str(f.get("key")) for f in fields if f.get("default") is not False]
531
+
532
+
533
+ def _public_view(rt, table_key: str, view: dict) -> dict:
534
+ """The ONLY bytes a valid token buys. Built KEY BY KEY β€” there is no `**row` in this function.
535
+
536
+ β›” THE REASON THAT IS A RULE AND NOT A STYLE. `aios_grid.rows_from_pool` puts `pid`, `_created`,
537
+ `lat` and `lon` on EVERY row regardless of what the view shows, and the stored row dict carries
538
+ every column the table has. Serialising a row and deleting the fields we do not want inverts
539
+ the failure: a column added next wave is INCLUDED by default and nobody notices, whereas an
540
+ allowlist that has not learned about it merely omits it. `routes_forms._public_form` is the
541
+ shipped precedent and says the same thing about itself.
542
+ """
543
+ import core.user_tables as user_tables
544
+ from harness import filter_eval
545
+
546
+ defn = user_tables.get(table_key, st=rt) or {}
547
+ fields = [f for f in (defn.get("fields") or []) if isinstance(f, dict)]
548
+ by_key = {str(f.get("key")): f for f in fields}
549
+ keys = _visible_keys(view, fields)
550
+ cfg = (view or {}).get("config") or {}
551
+
552
+ # The row pool: the table's own rows, narrowed to the fields it declares, exactly as
553
+ # `routes_tables.scoped_pool` builds it for a materialised table. ⚠ A read-through table has
554
+ # no rows here and is refused at the MINT, so this branch is the only one that can be reached.
555
+ rows_src = []
556
+ for rid, row in (defn.get("rows") or {}).items():
557
+ if not str(rid).isdigit():
558
+ continue
559
+ r = {k: v for k, v in (row or {}).items() if k in by_key}
560
+ r["pid"] = int(rid)
561
+ rows_src.append(r)
562
+ rows_src.sort(key=lambda r: r["pid"])
563
+
564
+ # The view's own row selection, through the SHARED evaluator. `_needs_a_reader` has already
565
+ # refused anything this context could not answer, so an empty result here means the filter
566
+ # genuinely matches nothing β€” not that we failed to resolve it.
567
+ ctx = filter_eval.EvalCtx(today=time.strftime("%Y-%m-%d"))
568
+ keep = set(filter_eval.visible_pids(cfg.get("filters"), rows_src, fields, ctx,
569
+ member_pids=cfg.get("memberPids")))
570
+ chosen = [r for r in rows_src if r.get("pid") in keep]
571
+
572
+ # β›”β›” COORDINATES RIDE A MAP WHEN THEY ARE COORDINATES β€” NOT WHEN THEY ARE VISIBLE, AND NOT
573
+ # BECAUSE OF WHAT A COLUMN IS CALLED. Two verifiers, one from each side, are why this reads
574
+ # the way it does; the first fix I wrote was wrong and the second report proved it.
575
+ #
576
+ # ⚠ THE LEAK (verifier #1, driven): a HIDDEN field keyed `lat` holding the string
577
+ # `CANARY-LAT-AAA` came out on the wire, because the pair was emitted before the `keys`
578
+ # projection and the bypass keyed on the FIELD NAME rather than on the value being a
579
+ # coordinate. So a column called `lat` could carry anything β€” a note, an address β€” and publish
580
+ # it. That is the real defect.
581
+ #
582
+ # β›” MY FIRST FIX GATED ON VISIBILITY, AND IT BROKE THE FEATURE (verifier #2): hiding the raw
583
+ # decimals is the NORMAL way somebody builds a Map view β€” nobody wants `38.7223` in the column
584
+ # list β€” so gating on `visible` meant an ordinary map published a page with no map, under two
585
+ # messages that contradicted each other ("no rows carry a location" vs "this view hides its
586
+ # location columns").
587
+ #
588
+ # ⭐ THE RULE THAT SATISFIES BOTH: publishing a MAP is publishing WHERE THE ROWS ARE β€” that is
589
+ # what the sharer chose β€” so a real coordinate rides whether or not its column is shown, and a
590
+ # value that is not a coordinate never rides at all. `_coord` is the whole wall, and it is a
591
+ # VALUE test, so no naming convention can smuggle anything past it.
592
+ # ⚠ It mirrors `PublishedView.MapPlot`'s own `coord()` deliberately: the client must not plot
593
+ # what the server would not send, and the server must not send what the client would discard.
594
+ # Two normalizers on one question is a smell [[one-question-two-normalizers]] β€” kept here
595
+ # because they sit on opposite sides of a trust boundary, where the server's copy is the wall
596
+ # and the client's is display hygiene.
597
+ mode = _mode_of(view)
598
+ plotted = 0
599
+ if mode == "map":
600
+ for r in chosen:
601
+ if _coord(r.get("lat"), 90) is not None and _coord(r.get("lon"), 180) is not None:
602
+ plotted += 1
603
+
604
+ limits = []
605
+ if mode == "map" and chosen and not plotted:
606
+ # R6's second sentence. A map with nothing on it must say WHY β€” and this says the true
607
+ # why, which is about the DATA, because visibility is no longer part of the answer.
608
+ limits.append({
609
+ "subject": "map", "effect": "not_plotted",
610
+ "detail": f"none of these {len(chosen)} rows carry a usable location",
611
+ "recommendation": "add `lat` and `lon` values to the records, then reload this link",
612
+ })
613
+ if len(chosen) > PUBLIC_ROW_CAP:
614
+ # R6's second sentence. A short page that does not say it is short is the silent
615
+ # truncation the rule is actually about.
616
+ limits.append({
617
+ "subject": "rows", "effect": "windowed",
618
+ "detail": f"this view has {len(chosen)} rows and a published page serves the first "
619
+ f"{PUBLIC_ROW_CAP}",
620
+ "recommendation": "narrow the view's filters, or share it with named people instead "
621
+ "of publishing a link",
622
+ })
623
+ chosen = chosen[:PUBLIC_ROW_CAP]
624
+
625
+ # β›” THE DISPLAY REFS ARE INTERSECTED WITH `visible`, NOT UNIONED INTO IT. A Map that colours
626
+ # by a column the view HIDES would otherwise put that column's value on every public row β€”
627
+ # the projection leak, arriving through the renderer rather than through the column list. The
628
+ # fail-closed choice is to drop the ref and render the map without colour; a published page
629
+ # that is slightly plainer beats one that ships a hidden column.
630
+ disp_in = (cfg.get("display") or {})
631
+ display = {"mode": mode}
632
+ for ref in ("dateField", "stackField", "titleField", "colorField", "sizeField"):
633
+ if disp_in.get(ref) in keys:
634
+ display[ref] = disp_in[ref]
635
+
636
+ return {
637
+ "title": str((view or {}).get("name") or "")[:200],
638
+ "mode": display["mode"],
639
+ "display": display,
640
+ "columns": [{"key": k,
641
+ "label": str(by_key[k].get("label") or k),
642
+ "type": str(by_key[k].get("type") or "text"),
643
+ **({"options": [str(o) for o in by_key[k]["options"]][:200]}
644
+ if isinstance(by_key[k].get("options"), list) and by_key[k].get("options")
645
+ else {})}
646
+ for k in keys],
647
+ # KEY BY KEY. `pid` rides because the client needs a stable row identity to render a list;
648
+ # it is a row NUMBER within this table and names nothing outside it.
649
+ # β›”β›” `lat`/`lon` RIDE ONLY ON A MAP **AND ONLY WHEN THE VIEW SHOWS THEM** β€” and the second
650
+ # half was missing, which was a LEAK. Found by a verifier that drove this route with a
651
+ # hidden field keyed `lat` carrying the string `CANARY-LAT-AAA`, and watched it come out
652
+ # on the wire.
653
+ #
654
+ # The first version emitted the pair BEFORE the `keys` projection, so `_visible_keys` never
655
+ # gated it. On a `ut_*` table coordinates are not magic: `routes_tables.scoped_pool` builds
656
+ # its row as `{k: v for k, v in row.items() if k in field_keys}`, so a value only survives
657
+ # if the table DECLARES a field keyed `lat`/`lon` β€” i.e. they are ORDINARY COLUMNS, and a
658
+ # view can hide them like any other. Hiding them therefore has to work here, because on a
659
+ # published page **the projection is the only wall there is** (`ut_*` databases have no
660
+ # hidden-field closure behind it, `routes_shares.py`'s docstring).
661
+ #
662
+ # ⚠ The bypass was keyed on the FIELD NAME, never on the value being a coordinate, so it
663
+ # forwarded whatever a column called `lat` happened to hold β€” a string, a note, anything.
664
+ # ⚠ And the shipped gate could not see it: it asserted the key NAMES rode on a map and not
665
+ # on a catalog, over a fixture whose rows carried no `lat` key at all β€” so it pinned the
666
+ # names while both values were `None` [[gate-answers-the-wrong-question]].
667
+ # KEY BY KEY. `pid` rides because the client needs a stable row identity; it is a row
668
+ # NUMBER within this table and names nothing outside it. `lat`/`lon` ride only on a map,
669
+ # and only when they PARSE as coordinates β€” see the block above for why that is the test.
670
+ "rows": [{"pid": r.get("pid"),
671
+ **({"lat": _coord(r.get("lat"), 90), "lon": _coord(r.get("lon"), 180)}
672
+ if mode == "map"
673
+ and _coord(r.get("lat"), 90) is not None
674
+ and _coord(r.get("lon"), 180) is not None else {}),
675
+ **{k: r.get(k) for k in keys}} for r in chosen],
676
+ "total": len(chosen),
677
+ **({"limits": limits} if limits else {}),
678
+ }
679
+
680
+
681
+ async def _bounded_body(request: Request) -> dict:
682
+ """The request body, read WITH A BOUND β€” never `Body(...)`, never `await request.body()`.
683
+
684
+ FastAPI reads and JSON-parses the WHOLE body before the handler's first line runs, so a
685
+ declared model is not a bound at all; `content-length` is caller-supplied, so checking it is
686
+ not one either. Streaming with a running total is the actual bound.
687
+ """
688
+ size, chunks = 0, []
689
+ async for chunk in request.stream():
690
+ size += len(chunk)
691
+ if size > MAX_BODY_BYTES:
692
+ raise err(413, "body_too_large", "that request is too large")
693
+ chunks.append(chunk)
694
+ import json
695
+ try:
696
+ parsed = json.loads(b"".join(chunks) or b"{}")
697
+ except ValueError:
698
+ raise err(400, "bad_request", "that request could not be read")
699
+ return parsed if isinstance(parsed, dict) else {}
700
+
701
+
702
+ # ── THE SHARER'S DOOR (authenticated, creator-or-admin) ──────────────────────────────────────
703
+ #
704
+ # ⚠ THE NOUN IS `/view-link`, DELIBERATELY NOT `/views/{...}/publish`, and it mirrors the form
705
+ # door's `/form-link` for the same reason: a literal path segment sitting beside a `/{token}`
706
+ # wildcard is resolved by DECLARATION ORDER, and a noun that cannot collide with a token has no
707
+ # order to get wrong.
708
+
709
+
710
+ @router.get("/view-link")
711
+ def read_view_link(topic: str = "", view: str = "",
712
+ session: Session = Depends(require_session)):
713
+ """This view's link state. β›” A GET MUST NOT MINT β€” opening the panel is not publishing."""
714
+ v, _mode = _may_administer_view(session, str(topic or ""), str(view or ""))
715
+ token, entry = _entry_for(session.runtime, str(topic), str(view))
716
+ out = _state_of(token, entry)
717
+ # The presentational flags contract C1 put on the view spec, echoed back so the client can
718
+ # tell whether the two agree. They are a MIRROR of this bucket, never its source.
719
+ disp = ((v.get("config") or {}).get("display") or {}) if isinstance(v, dict) else {}
720
+ out["displayPublished"] = disp.get("published") is True
721
+ return out
722
+
723
+
724
+ @router.post("/view-link")
725
+ async def mint_view_link(request: Request, session: Session = Depends(require_session)):
726
+ """Publish this view, or change its access. `{topic, view, access?, passphrase?, rotate?}`.
727
+
728
+ IDEMPOTENT WITHOUT `rotate`: opening the panel twice, or switching public↔password, must not
729
+ invalidate a link somebody already sent. `rotate: true` mints a fresh token and the previous
730
+ one dies in the SAME write, so there is never a window where both open the view.
731
+ """
732
+ body = await _bounded_body(request)
733
+ topic, view = str(body.get("topic") or ""), str(body.get("view") or "")
734
+ _v, _mode = _may_administer_view(session, topic, view)
735
+
736
+ access = "public" if body.get("access") == "public" else "password"
737
+ raw_pw = body.get("passphrase")
738
+ passphrase = "" if raw_pw is None else str(raw_pw)
739
+ if len(passphrase) > MAX_PASSPHRASE:
740
+ raise err(400, "passphrase_too_long",
741
+ f"a passphrase can be at most {MAX_PASSPHRASE} characters")
742
+
743
+ existing_token, existing = _entry_for(session.runtime, topic, view)
744
+ rotate = bool(body.get("rotate"))
745
+ fresh = secrets.token_urlsafe(TOKEN_BYTES)
746
+
747
+ # β›” A `password` LINK MUST END UP WITH A HASH, and there are exactly two ways to have one:
748
+ # the caller supplied a passphrase now, or one was already stored and is being kept. Anything
749
+ # else is refused HERE rather than stored and refused later β€” a link that cannot be opened by
750
+ # anybody is not a safe default, it is a broken feature that reads as a permission bug.
751
+ if access == "password":
752
+ if passphrase and len(passphrase) < MIN_PASSPHRASE:
753
+ raise err(400, "passphrase_too_short",
754
+ f"a passphrase needs at least {MIN_PASSPHRASE} characters")
755
+ if not passphrase and not (existing or {}).get("pw"):
756
+ raise err(400, "passphrase_required",
757
+ "a password-protected link needs a passphrase")
758
+
759
+ minted = {"token": existing_token or fresh}
760
+
761
+ def _set(cur):
762
+ cur = dict(cur or {})
763
+ prev = None
764
+ for stored, where in list(cur.items()):
765
+ if (isinstance(where, dict) and where.get("table") == topic
766
+ and where.get("view") == view):
767
+ prev = dict(where)
768
+ cur.pop(stored, None)
769
+ if not rotate:
770
+ minted["token"] = str(stored)
771
+ if rotate or prev is None:
772
+ minted["token"] = fresh
773
+ entry = {"table": topic, "view": view, "access": access,
774
+ "createdBy": str((prev or {}).get("createdBy") or session.uname),
775
+ "createdAt": float((prev or {}).get("createdAt") or time.time())}
776
+ # β›” THE STORED PASSPHRASE SURVIVES A TRIP THROUGH `public`, and the first version DROPPED
777
+ # it β€” found by a verifier that traced the toggle rather than the happy path. Publishing
778
+ # as `public` skipped this block entirely, so the hash was destroyed while the TOKEN was
779
+ # kept; switching back to `password` then demanded a new passphrase, silently, while the
780
+ # panel's own sentence promised the opposite ("leave blank to keep the current one").
781
+ # ⚠ Carrying it is inert, not lax: `_pw_ok` is consulted ONLY when `access == "password"`
782
+ # (`get_published`/`open_published` both test it first), and `_state_of` exposes a boolean,
783
+ # never the digest. A hash nobody can reach is not a secret in use β€” but a promise the UI
784
+ # makes and the store breaks is a defect either way, and the honest fix is to keep the
785
+ # promise rather than to reword it.
786
+ if passphrase:
787
+ salt = secrets.token_bytes(PW_SALT_BYTES)
788
+ entry["salt"] = salt.hex()
789
+ entry["iter"] = PW_ITERATIONS
790
+ entry["pw"] = _hash_pw(passphrase, salt)
791
+ elif (prev or {}).get("pw"):
792
+ # Carried key by key, so a future field on the entry is not silently inherited.
793
+ entry["salt"] = str((prev or {}).get("salt") or "")
794
+ entry["iter"] = int((prev or {}).get("iter") or PW_ITERATIONS)
795
+ entry["pw"] = str((prev or {}).get("pw") or "")
796
+ cur[minted["token"]] = entry
797
+ return cur
798
+
799
+ session.runtime.update(TOKENS_KEY, _set, flush="sync")
800
+ _mirror_display(session.runtime, topic, view, True, access)
801
+ token, entry = _entry_for(session.runtime, topic, view)
802
+ # β›” NO FABRICATED FALLBACK ENTRY HERE. The first version answered
803
+ # `_state_of(token or minted["token"], entry or {"access": access, "pw": "x"})`, and that
804
+ # `"pw": "x"` would have reported `hasPassphrase: true` about a PUBLIC link if the read-back
805
+ # ever came back empty β€” a lie in the safe-looking direction, which is the kind that survives.
806
+ # If the write cannot be read back, say so; do not describe a state nobody verified.
807
+ if not entry:
808
+ raise err(503, "not_saved",
809
+ "the link was minted but could not be read back β€” reload and try again")
810
+ return _state_of(token, entry)
811
+
812
+
813
+ @router.delete("/view-link")
814
+ async def revoke_view_link(request: Request, session: Session = Depends(require_session)):
815
+ """Unpublish. `{topic, view}`.
816
+
817
+ β›” REVOKE ROTATES β€” it does not merely unset a flag. The token STRING is dropped from the
818
+ index in this write, so the old link stops resolving immediately; and because a later publish
819
+ mints `secrets.token_urlsafe(24)` afresh, the revoked string can never come back. An
820
+ implementation that kept the token and flipped an `enabled` flag would leave the secret live
821
+ in the store, one bug away from working again, and would make "revoked" a property somebody
822
+ could forget to check on a code path added later.
823
+ """
824
+ body = await _bounded_body(request)
825
+ topic, view = str(body.get("topic") or ""), str(body.get("view") or "")
826
+ _may_administer_view(session, topic, view)
827
+
828
+ def _drop(cur):
829
+ cur = dict(cur or {})
830
+ for stored, where in list(cur.items()):
831
+ if (isinstance(where, dict) and where.get("table") == topic
832
+ and where.get("view") == view):
833
+ cur.pop(stored, None)
834
+ return cur
835
+
836
+ session.runtime.update(TOKENS_KEY, _drop, flush="sync")
837
+ # β›” AND THE MIRROR COMES DOWN IN THE SAME BREATH. Without this the grid goes on showing
838
+ # "published" about a link that no longer resolves β€” the half a reviewer caught.
839
+ _mirror_display(session.runtime, topic, view, False)
840
+ return {"published": False, "access": "password", "token": "", "url": ""}
841
+
842
+
843
+ # ── THE PUBLIC DOOR (no session β€” this is the whole feature) ─────────────────────────────────
844
+ #
845
+ # β›” NO `Depends(require_session)` ON EITHER ROUTE BELOW, DELIBERATELY. "Public" in this app is not
846
+ # a flag or an allow-list entry β€” `main.py` has no auth middleware and no exempt-path table; a
847
+ # route is public exactly by omitting the dependency. Which is why the two are kept together,
848
+ # under one banner, rather than filed beside the sharer's routes they resemble.
849
+
850
+
851
+ @router.get("/published/{token}")
852
+ def get_published(token: str, request: Request):
853
+ """The published view. Read-only, unauthenticated, projected to the view's own columns."""
854
+ if not _rate_ok(_client_ip(request), time.time()):
855
+ raise err(429, "too_many_requests", "too many requests β€” wait a moment and try again")
856
+ found = _resolve(token)
857
+ if not found:
858
+ # β›”β›” W33-T70 β€” AN UNKNOWN TOKEN ANSWERS EXACTLY WHAT A LOCKED ONE ANSWERS.
859
+ # This used to `raise _refuse()`, and that 403 was a free oracle: one unauthenticated GET,
860
+ # no passphrase, no cost, told an enumerator whether a token was REAL. `_refuse`'s own
861
+ # docstring says the difference between "no such link" and "that link was revoked" must
862
+ # never be observable β€” and the route beside it published that difference in its status
863
+ # code. Sorting real tokens from fake ones is the whole of the work; once it is free, the
864
+ # passphrase is all that is left and it can be attacked offline-cheap.
865
+ # ⚠ SO THE UNKNOWN TOKEN GETS THE LOCKED SHAPE: `{"locked": true}` and nothing else β€” no
866
+ # title, no columns, no count, the same bytes a real password link returns before anyone
867
+ # has tried to open it. The guess then costs a POST with a passphrase, which is rate
868
+ # limited and PBKDF2-priced. A PUBLIC link still opens on this GET, which is what a public
869
+ # link is for; what stops being visible is which PASSWORD tokens exist.
870
+ _note_failure(_client_ip(request), time.time())
871
+ _equalise_pw_cost(None)
872
+ return {"locked": True}
873
+ rt, _slug, table_key, view, entry = found
874
+ if entry.get("access") == "password":
875
+ # β›” THE SHAPE OF THE PASSWORD ANSWER, and it is not a refusal. A locked link must render
876
+ # a passphrase prompt, so this 200 says "there is something here and it is locked" and
877
+ # NOTHING else β€” no title, no column names, no row count. A 403 here would be
878
+ # indistinguishable from a bad token, which is right for a WRONG passphrase and wrong for
879
+ # a link the holder has not tried to open yet.
880
+ return {"locked": True}
881
+ return _public_view(rt, table_key, view)
882
+
883
+
884
+ @router.post("/published/{token}")
885
+ async def open_published(token: str, request: Request):
886
+ """Open a password-protected link. `{passphrase}`.
887
+
888
+ β›” A WRONG PASSPHRASE AND A WRONG TOKEN ANSWER THE SAME 403, from the same `_refuse`. If they
889
+ differed, the route would confirm which tokens are real to anybody willing to send one guess β€”
890
+ and a 24-byte token's entire protection is that it cannot be found by guessing.
891
+ ⚠ The passphrase is compared against a PBKDF2 digest with `hmac.compare_digest`, and is never
892
+ stored, logged or echoed. D-130's scar: a form's invited-address list is IDENTIFICATION and
893
+ anyone may claim an identity; this is AUTHENTICATION and is treated as one.
894
+ """
895
+ now = time.time()
896
+ if not _rate_ok(_client_ip(request), now):
897
+ raise err(429, "too_many_requests", "too many requests β€” wait a moment and try again")
898
+ body = await _bounded_body(request)
899
+ found = _resolve(token)
900
+ if not found:
901
+ # β›”β›” W33-T70 β€” SPEND THE PBKDF2 ANYWAY. A wrong token skipped the hash entirely and
902
+ # answered in ~0.4 ms while a wrong passphrase paid ~82 ms: a 206x gap, zero overlap in 24
903
+ # samples, and a clean separation of real tokens from fake ones for anyone with a stopwatch.
904
+ # Both arms now cost the same, so the identical 403 above is finally identical in practice
905
+ # rather than only in wording.
906
+ _note_failure(_client_ip(request), now)
907
+ _equalise_pw_cost(None)
908
+ raise _refuse()
909
+ rt, _slug, table_key, view, entry = found
910
+ if entry.get("access") == "password" and not _pw_ok(entry, str(body.get("passphrase") or "")):
911
+ _note_failure(_client_ip(request), now)
912
+ raise _refuse()
913
+ # ⚠ A link with NO passphrase must still pay, or "this token is public" is readable from the
914
+ # clock on a route whose whole job is to be uninformative.
915
+ if entry.get("access") != "password":
916
+ _equalise_pw_cost(entry)
917
+ return _public_view(rt, table_key, view)
api/routes_query.py CHANGED
The diff for this file is too large to render. See raw diff
 
api/routes_records.py CHANGED
@@ -15,8 +15,12 @@ one here:
15
 
16
  customer / cohort `routes_customers.allowed_pids` behind the `customer_data` grant
17
  product `routes_products.scoped_pool` behind the `product_data` grant
18
- ut_<slug> `routes_tables.scoped_pool`, whose `_defn_or_refuse` IS the wall
19
- (404 unknown / 403 not yours β€” a user table has no module grant)
 
 
 
 
20
 
21
  ⚠ THE PATH KEEPS ITS `/customers/` SEGMENT. It is the shipped URL and `verify_api.py`'s E1a
22
  section pins it; the scope now travels beside it explicitly. A nicer noun is not worth churning
@@ -49,31 +53,74 @@ def _scope_or_400(raw):
49
  def _pool_or_refuse(session: Session, scope: str):
50
  """The pids this session may attach comments to ON THIS DATABASE β€” grant wall included.
51
 
52
- Returns the frozenset. Raises the topic's own 403/404/503, so a caller who may not open the
53
- surface never learns anything about the row they asked about.
 
 
 
 
 
 
 
 
 
 
54
  """
55
  if scope == "product":
56
  from routes_products import MODULE as PRODUCT_MODULE, scoped_pool
57
 
58
  session.require(PRODUCT_MODULE)
59
  pids, _team, _rows, _fields = scoped_pool(session)
60
- return pids
 
 
 
 
61
  if scope.startswith("ut_"):
62
- # No module grant exists for a user table β€” `_defn_or_refuse` inside `scoped_pool` IS
63
  # the wall (creator or admin, fail-closed), and it answers 404 before 403 exactly as the
64
  # rows routes do.
65
- from routes_tables import scoped_pool
66
-
67
- pids, _rows, _fields, _defn = scoped_pool(session, scope)
68
- return pids
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
69
  from routes_customers import MODULE as CUSTOMER_MODULE, allowed_pids
70
 
71
  session.require(CUSTOMER_MODULE)
72
- return frozenset(allowed_pids(session))
73
 
74
 
75
  def _in_book(pid, session, scope):
76
- if pid not in _pool_or_refuse(session, scope):
 
 
 
 
 
 
 
 
 
77
  # 403, not 404: the record may exist, but this session may not inspect it.
78
  raise err(403, "out_of_scope", "that record is not in your book")
79
 
 
15
 
16
  customer / cohort `routes_customers.allowed_pids` behind the `customer_data` grant
17
  product `routes_products.scoped_pool` behind the `product_data` grant
18
+ ut_<slug> `routes_tables.scoped_pids`, whose `_defn_or_refuse` IS the wall
19
+ (404 unknown / 403 not yours β€” a user table has no module grant).
20
+ ⭐ W33-T03/D-183: `scoped_pIDs`, not `scoped_pOOL` β€” the pool builds every
21
+ ROW to derive a pid set this module discards, and RAISES 409 on a
22
+ read-through grid past one window, which is why the record drawer painted
23
+ an error page on `ut_odoo_gl_lines`.
24
 
25
  ⚠ THE PATH KEEPS ITS `/customers/` SEGMENT. It is the shipped URL and `verify_api.py`'s E1a
26
  section pins it; the scope now travels beside it explicitly. A nicer noun is not worth churning
 
53
  def _pool_or_refuse(session: Session, scope: str):
54
  """The pids this session may attach comments to ON THIS DATABASE β€” grant wall included.
55
 
56
+ Returns **`(pids, unbounded)`** β€” a 2-tuple on EVERY branch. Raises the topic's own 403/404/503,
57
+ so a caller who may not open the surface never learns anything about the row they asked about.
58
+
59
+ β›” `unbounded` is TRUE only when the row set could not be ENUMERATED (a read-through grid past
60
+ one window), never when it is merely EMPTY. Those are opposite answers and `scoped_pids` returns
61
+ `frozenset()` for both β€” see the branch below.
62
+ ⚠ THE SHAPE IS A CONTRACT EVEN THOUGH THIS FUNCTION IS PRIVATE, and it has two consumers that
63
+ do not travel together: `_in_book` here, and a NEGATIVE CONTROL in `aios-web/api/verify_scopes.py`
64
+ that REPLACES this function with its own lambda. A gate's test double is a caller
65
+ ([[test-double-patched-by-a-name-list]]); when this signature moved, that double kept returning a
66
+ bare frozenset and the section died on `ValueError: too many values to unpack` β€” no tally, no
67
+ failing name. Change the shape here and that double changes with it.
68
  """
69
  if scope == "product":
70
  from routes_products import MODULE as PRODUCT_MODULE, scoped_pool
71
 
72
  session.require(PRODUCT_MODULE)
73
  pids, _team, _rows, _fields = scoped_pool(session)
74
+ # ⚠ `(pids, unbounded)` on EVERY branch. This one returned a bare frozenset for ten minutes
75
+ # after the `ut_` branch grew its second element, and `_in_book`'s unpack would have raised
76
+ # a `TypeError` β€” a 500 on every product comment β€” while both other branches worked. A
77
+ # return shape is a contract even when the function is private.
78
+ return (pids, False)
79
  if scope.startswith("ut_"):
80
+ # No module grant exists for a user table β€” `_defn_or_refuse` inside `scoped_pids` IS
81
  # the wall (creator or admin, fail-closed), and it answers 404 before 403 exactly as the
82
  # rows routes do.
83
+ #
84
+ # ⭐⭐ W33-T03 / D-183 β€” `scoped_pids`, NOT `scoped_pool`, AND THAT ONE WORD IS THE BUG.
85
+ # `scoped_pool` builds every ROW to derive a pid set this function then throws away, and on
86
+ # a read-through grid larger than one window it RAISES `409 window_required`. So opening the
87
+ # record drawer on `ut_odoo_gl_lines` (975,137 rows) painted an error page β€” for a panel
88
+ # that renders comments about ONE row it already has. `scoped_pids` answers the identical
89
+ # question (its docstring: *"the pid set is IDENTICAL, not merely equivalent"*) and takes
90
+ # W31-T20's `limits` OUT-PARAMETER instead of raising, which is the same shape `/workspace`
91
+ # used to become openable on those two grids.
92
+ from routes_tables import scoped_pids
93
+
94
+ limits = []
95
+ pids, _fields, _defn = scoped_pids(session, scope, limits=limits)
96
+ # β›” AN EMPTY PID SET AND AN UNRESOLVABLE ONE ARE OPPOSITE ANSWERS, and collapsing them is
97
+ # how a fail-closed default becomes a lie. `scoped_pids` returns `frozenset()` BOTH for a
98
+ # database with no rows and for a read-through grid too big to enumerate β€” it distinguishes
99
+ # them by APPENDING R6's sentence to `limits`. Without this branch the drawer would move
100
+ # from a 409 error page to a 403 "not in your book" on a row the user is looking at, which
101
+ # is the same defect wearing a politer message ([[empty-answer-vs-unfinished-answer]]).
102
+ # ⚠ ADMITTING HERE IS NOT A WIDENING, and the ruling is wave 27 / D-72: on a `ut_*` database
103
+ # THE TENANT IS THE UNIT β€” `scoped_pool` itself carries "no per-row owner filter; the
104
+ # table-level wall is the WHOLE wall". `_defn_or_refuse` has already run inside
105
+ # `scoped_pids` and answered 404/403. The pid set was only ever an existence check.
106
+ return (pids, bool(limits))
107
  from routes_customers import MODULE as CUSTOMER_MODULE, allowed_pids
108
 
109
  session.require(CUSTOMER_MODULE)
110
+ return (frozenset(allowed_pids(session)), False)
111
 
112
 
113
  def _in_book(pid, session, scope):
114
+ pids, unbounded = _pool_or_refuse(session, scope)
115
+ if unbounded:
116
+ # The table wall passed and the row set is larger than this process will enumerate. Said
117
+ # out loud rather than silently admitting: R6's second sentence is that a limit which
118
+ # cannot be removed gets REPORTED, and this is the one place the report has no envelope to
119
+ # ride in.
120
+ print(f"[records] {scope}: pid membership unresolved (read-through beyond one window) β€” "
121
+ f"admitting on the table wall alone, per D-72")
122
+ return
123
+ if pid not in pids:
124
  # 403, not 404: the record may exist, but this session may not inspect it.
125
  raise err(403, "out_of_scope", "that record is not in your book")
126
 
api/routes_shares.py CHANGED
@@ -336,8 +336,16 @@ def _notify_new_grantees(session, kind, oid, before, after):
336
  # agree or the Inbox row is unclickable: `topic` selects the branch and `key` becomes
337
  # `alertId`, which that branch reads as the id to open. Both constants are IMPORTED
338
  # from there rather than typed again β€” one vocabulary, one owner.
 
 
 
 
 
 
 
339
  alerts.notify(user, label, topic=_SHARE_TOPIC, key=route, row_id=view_id,
340
- detail=f"{sharer} shared this with you", st=session.runtime)
 
341
  except Exception: # noqa: BLE001
342
  return
343
 
 
336
  # agree or the Inbox row is unclickable: `topic` selects the branch and `key` becomes
337
  # `alertId`, which that branch reads as the id to open. Both constants are IMPORTED
338
  # from there rather than typed again β€” one vocabulary, one owner.
339
+ # ⭐⭐ W33-T28 (`ASK C-14`, answered) β€” `actor` IS THE SENDER, AND IT IS THE ONLY WAY
340
+ # THE INBOX CAN NAME ONE. An alert and an automation have no person behind them and
341
+ # are honestly named by their machine; a SHARE has a real person, and only this call
342
+ # site knows who. β›” It is passed as its OWN field rather than recovered from the
343
+ # `detail` prose below: a sender parsed out of "<name> shared this with you" breaks
344
+ # the first time the sentence is reworded, silently, in the header
345
+ # [[grep-output-is-not-source]]. The prose stays as the body; this is the From.
346
  alerts.notify(user, label, topic=_SHARE_TOPIC, key=route, row_id=view_id,
347
+ detail=f"{sharer} shared this with you", actor=sharer,
348
+ st=session.runtime)
349
  except Exception: # noqa: BLE001
350
  return
351
 
api/routes_slack.py ADDED
@@ -0,0 +1,756 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """routes_slack.py β€” MANAGE AGENT: a bot per Slack channel, walled by the engine that walls a
2
+ person. (wave 33, owner item 10 Β· `W33-T38` / `W33-T39` / `W33-T67`.)
3
+
4
+ Owner, verbatim (2026-08-14): *"Expand Slack we need to be able to configure an AI bot PER channel
5
+ whose permissions we can toggle based on Fields etc of the databases, exactly like how we toggle
6
+ permissioning per user."*
7
+
8
+ ────────────────────────────────────────────────────────────────────────────────────────────────
9
+ β›”β›” D-84 SAID THIS WAS "adapted from OpenTag" AND THAT PREMISE IS FALSE (PRD amendment A1).
10
+
11
+ `PENDING.md` D-84 has named OpenTag as the model for per-Slack-channel permissioning since
12
+ 2026-08-08. **OpenTag has no permission model to adapt.** `app/runtime-host.ts` is
13
+ `identifyUser: () => OPENTAG_SERVICE_USER` β€” ONE constant identity for every request β€” and its own
14
+ `sender-context.ts` says the Slack sender is *"informational only… not gating access."* Its
15
+ authorization ceiling is whole-MCP-server, decided once at process boot from env vars. It is a
16
+ live specimen of identity that LOOKS like a permission and is not.
17
+
18
+ `core/perm_scope.py` is already strictly more expressive: per database, per row, per FIELD, per
19
+ principal. So item 10 is OUR engine projected onto a CHANNEL principal β€” see `agent_principal`
20
+ below, which is the whole projection and is nine lines. Study:
21
+ `.claude/wiki/research/opentag-adoption.md` (licence gate: MIT, PASSED).
22
+
23
+ What OpenTag genuinely contributes is the WRITE-APPROVAL interceptor (`W33-T67`, `approval_card`
24
+ below) and nothing else. Its TRANSPORT is AVOID wholesale β€” hosted CopilotKit Intelligence, a
25
+ second Node process, a persistent outbound socket and each tenant's Slack credentials held by a
26
+ third party, against ONE process on the HF free tier. The door here is `routes_forms.py`'s proven
27
+ unauthenticated-public shape plus Slack's signing-secret check.
28
+
29
+ ────────────────────────────────────────────────────────────────────────────────────────────────
30
+ ⚠ A CHANNEL IS A PRINCIPAL, NOT A PERSON. The wall binds to the Slack conversation id, so every
31
+ human in that channel reads through the same rules. That is the honest reading of "a bot per
32
+ channel" and the pane says it out loud, because the alternative is an admin assuming per-person
33
+ scoping from a screen that looks exactly like the per-person one.
34
+
35
+ ⚠ AND THE IDENTITY IS THE CHANNEL ID, NEVER ITS NAME. A channel can be renamed; its id cannot.
36
+ A wall bound to `#sales` silently re-points the day somebody renames the channel.
37
+ """
38
+ import hmac
39
+ import json
40
+ import os
41
+ import secrets
42
+ import time
43
+ from datetime import datetime, timezone
44
+
45
+ from fastapi import APIRouter, Body, Depends, Request
46
+
47
+ from deps import Session, err, require_session
48
+ from routes_admin import admin_gate
49
+
50
+ router = APIRouter(prefix="/api/v1")
51
+
52
+ #: The tenant's channel agents: `{id: record}`. A per-tenant bucket, so it rides
53
+ #: `runtime.store_key`'s prefix and never lands in tenant #0's namespace.
54
+ AGENTS_KEY = "slack_agents"
55
+
56
+ #: Pending mutations awaiting a human's click (`W33-T67`). Server-only, per tenant.
57
+ PENDING_KEY = "slack_pending_writes"
58
+
59
+ #: How long an approval card stays clickable. An approval is a statement about the world as it was
60
+ #: when the card was rendered; an hour later the values it names may no longer be the values it
61
+ #: would write. Expiring is the honest behaviour, and an expired card says so rather than 403ing.
62
+ APPROVAL_TTL_S = 15 * 60
63
+
64
+ #: Slack rejects a replayed request older than 5 minutes; so do we, before any signature work.
65
+ SLACK_MAX_SKEW_S = 60 * 5
66
+
67
+ MAX_BODY_BYTES = 64 * 1024
68
+
69
+ #: β›” ROW-CAPPED BECAUSE AN APPROVER WHO CANNOT READ THE CARD CANNOT APPROVE IT. Copied
70
+ #: deliberately from OpenTag's `agent/write_confirmation.py`, which caps for the same reason:
71
+ #: past a certain length Slack collapses a message behind "Show more", and a human clicking
72
+ #: Approve on a card whose tail they never saw is worse than no card at all.
73
+ APPROVAL_MAX_ROWS = 12
74
+
75
+
76
+ # ── the tenant's agent records ───────────────────────────────────────────────────────────────
77
+ def _agents(rt):
78
+ """`{id: record}` for one tenant. `{}` on any failure β€” an unreadable bucket must degrade to
79
+ "this tenant has no agents", never to a 500 on the settings pane."""
80
+ try:
81
+ found = rt.get(AGENTS_KEY) or {}
82
+ except Exception: # noqa: BLE001
83
+ return {}
84
+ return found if isinstance(found, dict) else {}
85
+
86
+
87
+ def _clean_agent(raw, prior=None):
88
+ """One stored record, built KEY BY KEY.
89
+
90
+ β›” Never `dict(raw)` and never `**raw`. This record is the SUBJECT of a permission decision;
91
+ a client-supplied key landing in it is a client-supplied input to `perm_scope`. The same rule
92
+ `routes_forms._public_form` follows on the way out, applied here on the way in.
93
+ """
94
+ prior = prior if isinstance(prior, dict) else {}
95
+ out = {
96
+ "id": str(prior.get("id") or raw.get("id") or ""),
97
+ "channel": str(raw.get("channel") or prior.get("channel") or "").strip()[:64],
98
+ "channelName": str(raw.get("channelName") or prior.get("channelName") or "").strip()[:80],
99
+ "label": " ".join(str(raw.get("label") or prior.get("label") or "").split())[:80],
100
+ "active": bool(raw.get("active", prior.get("active", True))),
101
+ # β›” DEFAULT TRUE, AND `is not False` RATHER THAN TRUTHINESS. A record written before this
102
+ # key existed must read as "approval required": the one direction a default here is
103
+ # allowed to be wrong in is the safe one, because an unattended write cannot be undone by
104
+ # revoking the bot afterwards [[default-must-pass-its-own-guard]].
105
+ "writeApproval": raw.get("writeApproval", prior.get("writeApproval", True)) is not False,
106
+ # The permission block. Same shape, same validator, same engine as a user's.
107
+ "perms": prior.get("perms") if isinstance(prior.get("perms"), dict) else {},
108
+ "perms_v": int(prior.get("perms_v") or 0),
109
+ "createdBy": str(prior.get("createdBy") or raw.get("createdBy") or ""),
110
+ "createdAt": str(prior.get("createdAt") or raw.get("createdAt") or ""),
111
+ }
112
+ return out
113
+
114
+
115
+ def _put_agent(session, agent_id, mutate):
116
+ """Read-modify-write ONE agent under the tenant's bucket, synchronously.
117
+
118
+ `flush="sync"` because the client re-reads the list immediately after every write here, and an
119
+ async flush would let the refetch win the race and paint the pre-write state β€” the
120
+ [[refetch-eats-its-own-write]] shape.
121
+ """
122
+ def _set(cur):
123
+ cur = dict(cur or {})
124
+ rec = cur.get(agent_id) if isinstance(cur.get(agent_id), dict) else None
125
+ nxt = mutate(rec)
126
+ if nxt is None:
127
+ cur.pop(agent_id, None)
128
+ else:
129
+ cur[agent_id] = nxt
130
+ return cur
131
+
132
+ session.runtime.update(AGENTS_KEY, _set, flush="sync")
133
+
134
+
135
+ # ── ⭐⭐ THE PROJECTION β€” THE WHOLE OF "the same engine that permissions a user" ───────────────
136
+ def agent_principal(agent):
137
+ """A channel agent, as a PRINCIPAL `core/perm_scope` already understands.
138
+
139
+ ⭐⭐ THIS IS THE ENTIRE POINT OF THE TICKET AND IT IS NINE LINES. `perm_scope` takes a user
140
+ RECORD β€” a plain dict with `perms`, `perms_v` and `role` β€” not a User object and not a session.
141
+ So a channel agent carrying a `perms` block simply IS such a record, and `may_access`,
142
+ `visible_fields`, `hidden_keys` and `apply_row_scope` work on it unmodified. There is no second
143
+ engine, no parallel vocabulary, and nothing about a channel to keep in step with anything about
144
+ a person. That is what the owner's *"exactly like how we toggle permissioning per user"* means
145
+ when it is true in code rather than in appearance.
146
+
147
+ β›” `role` IS HARDCODED NON-ADMIN AND MUST STAY SO. `perm_scope.may_access` returns True
148
+ unconditionally for `role == 'admin'` β€” the break-glass clause that keeps the owner out of a
149
+ locked store. A bot has no such emergency and no hands: an agent record that could carry
150
+ `role: 'admin'` would be one stored key away from bypassing every wall on this page. The key is
151
+ not copied from the record; it is written here, every time.
152
+
153
+ β›” AND `perms_v` IS FORCED TO THE CURRENT VERSION, which makes an UNDECLARED database DENY
154
+ (C-PERM amendment 4). Without it a fresh agent would fall through to `perms.may_open`'s legacy
155
+ grant wall and read `modules: 'all'`-shaped defaults β€” i.e. a brand-new bot would start with
156
+ access to everything. Fail-closed from the first byte.
157
+ """
158
+ import core.perm_scope as perm_scope
159
+ return {
160
+ "username": f"slack:{(agent or {}).get('channel') or '?'}",
161
+ "role": "agent",
162
+ "perms": (agent or {}).get("perms") or {},
163
+ "perms_v": perm_scope.PERMS_VERSION,
164
+ # No `bus`, no `agent`: a channel is not a business unit and not a salesperson. Absent
165
+ # rather than 'all' β€” `perms.allowed_bu_labels` narrows on absence, which is the direction
166
+ # this must fail in.
167
+ }
168
+
169
+
170
+ def agent_may_read(agent, module):
171
+ """May this channel open `module` at all?"""
172
+ import core.perm_scope as perm_scope
173
+ if not (agent or {}).get("active", True):
174
+ return False
175
+ return bool(perm_scope.may_access(agent_principal(agent), module))
176
+
177
+
178
+ def agent_visible_fields(agent, fields, module):
179
+ """`fields` minus this channel's hidden closure β€” the SAME transitive closure a person gets,
180
+ so a formula over a hidden column cannot leak it back through arithmetic."""
181
+ import core.perm_scope as perm_scope
182
+ return perm_scope.visible_fields(fields, agent_principal(agent), module)
183
+
184
+
185
+ def agent_rows(agent, rows, module, fields, ctx=None):
186
+ """The rows this channel may receive β€” `permits()`, so an unanswerable permanent filter DENIES
187
+ rather than being ignored."""
188
+ import core.perm_scope as perm_scope
189
+ p = agent_principal(agent)
190
+ hide = perm_scope.hidden_keys(p, module, fields)
191
+ kept = perm_scope.apply_row_scope(rows, p, module, fields, ctx)
192
+ # Both wires, never one: the field list and the row payload are separate, and stripping only
193
+ # the first leaves the value sitting in the second where anyone can read it.
194
+ return [perm_scope.strip_row(r, hide) for r in kept]
195
+
196
+
197
+ # ── the credential ───────────────────────────────────────────────────────────────────────────
198
+ def slack_creds(rt):
199
+ """This TENANT's Slack credentials from its own keychain, or None.
200
+
201
+ β›” NO ENVIRONMENT FALLBACK, DELIBERATELY, AND THIS IS D-202's LESSON APPLIED BEFORE IT
202
+ HAPPENS AGAIN. Meta Ads shipped loading end-to-end and was still not a connector, because
203
+ `keychain.meta_creds` had no caller and the token came from the owner's `.env` β€” making it a
204
+ tenant-#0 FACT indistinguishable from a screenshot. `keychain.meta_creds`' own docstring spells
205
+ out the rule: handing the environment's credential to a tenant whose admin has not stored one
206
+ is the leak the resolver exists to prevent. So a tenant with no entry gets None and the pane
207
+ SAYS so.
208
+
209
+ ⚠ Reads through `list_entries` + `read_fields` (both public) rather than `keychain._first_creds`
210
+ (private, and in a file outside this lane's fence). Same deterministic rule: entries are
211
+ id-sorted, so "first" is stable across reads rather than dict-order luck.
212
+ """
213
+ try:
214
+ import core.keychain as keychain
215
+ for row in keychain.list_entries(rt):
216
+ if str(row.get("type") or "") != "slack":
217
+ continue
218
+ fields = keychain.read_fields(rt, row.get("id"))
219
+ if isinstance(fields, dict) and fields.get("signing_secret"):
220
+ return fields
221
+ except Exception: # noqa: BLE001
222
+ # A locked or unreadable keychain reads as "not configured" for the PANE, which is honest;
223
+ # the signature check below fails closed regardless, so this cannot widen anything.
224
+ return None
225
+ return None
226
+
227
+
228
+ # ── the authenticated doors (the Manage agent surface) ───────────────────────────────────────
229
+ def _summary(agent, modules):
230
+ """One sentence per agent for the list row β€” computed here, because the alternative is one
231
+ round trip per row to fill one cell (the same reason `AdminUser.access` exists)."""
232
+ governed = [m for m in modules if m.get("enforced")]
233
+ perms = agent.get("perms") or {}
234
+ open_n = sum(1 for m in governed if (perms.get(m["key"]) or {}).get("access"))
235
+ if not governed:
236
+ return ""
237
+ if open_n == 0:
238
+ return "No access"
239
+ restricted = sum(1 for m in governed
240
+ if (perms.get(m["key"]) or {}).get("access")
241
+ and ((perms.get(m["key"]) or {}).get("filter")
242
+ or (perms.get(m["key"]) or {}).get("hiddenFields")))
243
+ head = (f"All {open_n} database{'' if open_n == 1 else 's'}" if open_n == len(governed)
244
+ else f"{open_n} of {len(governed)} databases")
245
+ return f"{head}, {restricted} restricted" if restricted else head
246
+
247
+
248
+ @router.get("/agents")
249
+ def list_channel_agents(session: Session = Depends(admin_gate)):
250
+ """This tenant's channel agents, plus whether Slack is reachable at all."""
251
+ import routes_admin
252
+ modules = routes_admin._perm_modules(session)
253
+ rows = []
254
+ for aid, a in sorted(_agents(session.runtime).items()):
255
+ if not isinstance(a, dict):
256
+ continue
257
+ rows.append({"id": aid, "channel": a.get("channel") or "",
258
+ "channelName": a.get("channelName") or "",
259
+ "label": a.get("label") or "", "active": a.get("active", True) is not False,
260
+ "writeApproval": a.get("writeApproval", True) is not False,
261
+ "summary": _summary(a, modules)})
262
+ return {"agents": rows, "configured": slack_creds(session.runtime) is not None}
263
+
264
+
265
+ @router.post("/agents")
266
+ def create_channel_agent(body: dict = Body(default=None),
267
+ session: Session = Depends(admin_gate)):
268
+ body = body if isinstance(body, dict) else {}
269
+ channel = str(body.get("channel") or "").strip()
270
+ if not channel:
271
+ raise err(400, "no_channel", "a Slack channel ID is required")
272
+ existing = _agents(session.runtime)
273
+ # ⚠ ONE AGENT PER CHANNEL. Two records for one conversation would mean two walls, and nothing
274
+ # anywhere decides which one applies β€” the [[one-question-two-normalizers]] shape, in the one
275
+ # place where the two answers are "may read" and "may not".
276
+ for a in existing.values():
277
+ if isinstance(a, dict) and str(a.get("channel") or "") == channel:
278
+ raise err(409, "channel_taken",
279
+ "that channel already has an agent β€” edit it instead of adding a second")
280
+ aid = secrets.token_urlsafe(9)
281
+ rec = _clean_agent(dict(body, id=aid, createdBy=session.uname,
282
+ createdAt=datetime.now(timezone.utc).isoformat(timespec="seconds")))
283
+ _put_agent(session, aid, lambda _prior: rec)
284
+ fresh = _agents(session.runtime).get(aid)
285
+ if not isinstance(fresh, dict):
286
+ # The store took the write and did not record it. A 200 here would tell an administrator
287
+ # the agent exists when it does not.
288
+ raise err(503, "store_unavailable", "the agent was NOT created")
289
+ return {"agent": {"id": aid, "channel": rec["channel"], "channelName": rec["channelName"],
290
+ "label": rec["label"], "active": rec["active"],
291
+ "writeApproval": rec["writeApproval"], "summary": "No access"}}
292
+
293
+
294
+ @router.get("/agents/{agent_id}")
295
+ def get_channel_agent(agent_id: str, session: Session = Depends(admin_gate)):
296
+ """The agent PLUS the permission envelope β€” the SAME shape `get_perms` answers with.
297
+
298
+ ⭐ ONE PAYLOAD SHAPE FOR TWO ROOMS. `permsModel.parsePermsPayload` parses this, and
299
+ `settings/ModulePermsList` renders it, because they are literally the same client code. A
300
+ second envelope here would mean a second parser and a second set of defaults within a wave.
301
+ """
302
+ import routes_admin
303
+ a = _agents(session.runtime).get(str(agent_id))
304
+ if not isinstance(a, dict):
305
+ raise err(404, "no_such_agent", "no agent with that id")
306
+ modules = routes_admin._perm_modules(session)
307
+ enforced = [m["key"] for m in modules if m.get("enforced")]
308
+ stored = a.get("perms") or {}
309
+ principal = agent_principal(a)
310
+ import core.perm_scope as perm_scope
311
+ perms_out = {}
312
+ for k in enforced:
313
+ e = stored.get(k)
314
+ perms_out[k] = e if isinstance(e, dict) else {
315
+ "access": bool(perm_scope.may_access(principal, k)), "filter": None,
316
+ "hiddenFields": []}
317
+ return {"id": str(agent_id), "channel": a.get("channel") or "",
318
+ "channelName": a.get("channelName") or "", "label": a.get("label") or "",
319
+ "active": a.get("active", True) is not False,
320
+ "writeApproval": a.get("writeApproval", True) is not False,
321
+ "perms": perms_out,
322
+ # β›” ALWAYS THE CURRENT VERSION, because `agent_principal` forces it: the editor must
323
+ # render "undeclared = denied", not the legacy-grant reading it would show at 0.
324
+ "perms_v": perm_scope.PERMS_VERSION,
325
+ # A bot is NEVER an admin β€” see `agent_principal`. Sent so the editor never paints
326
+ # the "Everything (admin)" state for a principal that cannot have it.
327
+ "is_admin": False,
328
+ "modules": modules,
329
+ "fields_by_module": {k: routes_admin._module_fields(k) for k in enforced}}
330
+
331
+
332
+ @router.put("/agents/{agent_id}/perms")
333
+ def put_channel_agent_perms(agent_id: str, body: dict = Body(default=None),
334
+ session: Session = Depends(admin_gate)):
335
+ """Replace this agent's permission block wholesale β€” the same validator a user's block gets."""
336
+ import routes_admin
337
+ body = body if isinstance(body, dict) else {}
338
+ if not isinstance(_agents(session.runtime).get(str(agent_id)), dict):
339
+ raise err(404, "no_such_agent", "no agent with that id")
340
+ if "perms" not in body:
341
+ raise err(400, "empty_patch", "no perms to save")
342
+ mods = routes_admin._perm_modules(session)
343
+ # ⭐ THE SAME `_clean_perms`, NOT A COPY OF IT. It refuses a filter this module cannot
344
+ # evaluate, refuses a hiddenFields key that names nothing, refuses an unenforced `ut_*` key,
345
+ # and refuses a BU condition the pushdown cannot read. Every one of those is exactly as true
346
+ # for a channel as for a person, and a second validator here is a second place for one of
347
+ # them to go missing.
348
+ cleaned = routes_admin._clean_perms(
349
+ body.get("perms"),
350
+ enforced_keys={m["key"] for m in mods if m.get("enforced")},
351
+ listed_keys={m["key"] for m in mods}) or {}
352
+
353
+ import core.perm_scope as perm_scope
354
+
355
+ def _mut(prior):
356
+ if not isinstance(prior, dict):
357
+ return None
358
+ nxt = dict(prior)
359
+ nxt["perms"] = cleaned
360
+ nxt["perms_v"] = perm_scope.PERMS_VERSION
361
+ return nxt
362
+
363
+ _put_agent(session, str(agent_id), _mut)
364
+ fresh = _agents(session.runtime).get(str(agent_id)) or {}
365
+ if (fresh.get("perms") or {}) != cleaned:
366
+ # The one direction this must never fail in: telling an administrator the wall is up when
367
+ # it is not.
368
+ raise err(503, "store_unavailable", "the permissions were not saved β€” the agent is UNCHANGED")
369
+ return {"id": str(agent_id), "perms": fresh.get("perms") or {},
370
+ "perms_v": int(fresh.get("perms_v") or 0)}
371
+
372
+
373
+ @router.patch("/agents/{agent_id}")
374
+ def patch_channel_agent(agent_id: str, body: dict = Body(default=None),
375
+ session: Session = Depends(admin_gate)):
376
+ body = body if isinstance(body, dict) else {}
377
+ if not isinstance(_agents(session.runtime).get(str(agent_id)), dict):
378
+ raise err(404, "no_such_agent", "no agent with that id")
379
+ #: β›” `perms` IS NOT PATCHABLE HERE. It has its own door with its own validator; accepting it
380
+ #: on the metadata route would be a second, unvalidated way to write the wall.
381
+ allowed = {k: v for k, v in body.items()
382
+ if k in ("label", "active", "writeApproval", "channelName")}
383
+ if not allowed:
384
+ raise err(400, "empty_patch", "nothing to change")
385
+
386
+ def _mut(prior):
387
+ return _clean_agent(allowed, prior=prior) if isinstance(prior, dict) else None
388
+
389
+ _put_agent(session, str(agent_id), _mut)
390
+ import routes_admin
391
+ a = _agents(session.runtime).get(str(agent_id)) or {}
392
+ return {"agent": {"id": str(agent_id), "channel": a.get("channel") or "",
393
+ "channelName": a.get("channelName") or "", "label": a.get("label") or "",
394
+ "active": a.get("active", True) is not False,
395
+ "writeApproval": a.get("writeApproval", True) is not False,
396
+ "summary": _summary(a, routes_admin._perm_modules(session))}}
397
+
398
+
399
+ @router.delete("/agents/{agent_id}")
400
+ def delete_channel_agent(agent_id: str, session: Session = Depends(admin_gate)):
401
+ if not isinstance(_agents(session.runtime).get(str(agent_id)), dict):
402
+ raise err(404, "no_such_agent", "no agent with that id")
403
+ _put_agent(session, str(agent_id), lambda _prior: None)
404
+ if isinstance(_agents(session.runtime).get(str(agent_id)), dict):
405
+ raise err(503, "store_unavailable", "the agent was NOT removed")
406
+ return {"ok": True}
407
+
408
+
409
+ # ── ⭐ W33-T67: THE WRITE-APPROVAL GATE ───────────────────────────────────────────────────────
410
+ def _is_empty(value):
411
+ """Is this value ABSENT, as opposed to falsy?
412
+
413
+ β›” COPIED EXACTLY FROM OpenTag's `agent/write_confirmation.py`, and the exactness is the point:
414
+ `0` and `False` are VALUES a human must see on the card. Setting a price to 0 or a flag to
415
+ False is precisely the kind of write somebody needs to approve, and a naive `if not value`
416
+ drops both rows silently β€” the approver then reads a card that does not describe the write
417
+ they are approving. The MIT licence gate for this borrowing is recorded in
418
+ `.claude/wiki/research/opentag-adoption.md`.
419
+ """
420
+ return value is None or (isinstance(value, str) and value.strip() == "")
421
+
422
+
423
+ def approval_card(action, values, agent=None):
424
+ """The card a human reads before a bot writes. `{action, rows, truncated, note}`.
425
+
426
+ β›” ROW-CAPPED, and the cap is a SAFETY property rather than a layout one: past a certain
427
+ length Slack collapses a message behind "Show more", and **an approver who cannot read the
428
+ card cannot approve it.** When rows are dropped the card SAYS how many β€” a truncation nobody
429
+ was told about is the violation, not the truncation (wave-30 R6's second sentence).
430
+ """
431
+ rows = [{"field": str(k), "value": v}
432
+ for k, v in (values or {}).items() if not _is_empty(v)]
433
+ rows.sort(key=lambda r: r["field"])
434
+ shown, dropped = rows[:APPROVAL_MAX_ROWS], max(0, len(rows) - APPROVAL_MAX_ROWS)
435
+ return {
436
+ "action": str(action or "")[:80],
437
+ "rows": shown,
438
+ "truncated": dropped,
439
+ "note": (f"{dropped} more field(s) not shown β€” open the record to review them before "
440
+ f"approving." if dropped else ""),
441
+ "agent": (agent or {}).get("label") or (agent or {}).get("channel") or "",
442
+ }
443
+
444
+
445
+ def intercept_write(agent, action, values, stash):
446
+ """THE INTERCEPTOR. Returns `("commit", None)` or `("await_approval", card)`.
447
+
448
+ ⭐ The one thing worth taking from OpenTag: a MUTATING tool call is halted, rendered as a card
449
+ naming the action and its values, and committed only on an explicit human accept. Everything
450
+ else in that repo β€” the transport, the runtime, the identity model β€” is AVOID (amendment A1).
451
+
452
+ β›” FAIL-CLOSED ON A MISSING FLAG: `is not False`, so a record written before `writeApproval`
453
+ existed requires approval. The unsafe direction here is not recoverable β€” revoking a bot does
454
+ not un-write what it wrote.
455
+ """
456
+ if (agent or {}).get("writeApproval", True) is not False:
457
+ card = approval_card(action, values, agent)
458
+ stash(card)
459
+ return "await_approval", card
460
+ return "commit", None
461
+
462
+
463
+ def _pending(rt):
464
+ try:
465
+ found = rt.get(PENDING_KEY) or {}
466
+ except Exception: # noqa: BLE001
467
+ return {}
468
+ return found if isinstance(found, dict) else {}
469
+
470
+
471
+ def stash_pending(rt, agent, action, values, now=None):
472
+ """Store one awaiting-approval mutation and return its token.
473
+
474
+ The token is what rides the card's button, so it is `secrets.token_urlsafe` and not the
475
+ action's id: a guessable value on an unauthenticated door is a way to approve somebody else's
476
+ write. Server-only bucket, per tenant, exactly as `routes_forms.TOKENS_KEY` is.
477
+ """
478
+ token = secrets.token_urlsafe(18)
479
+ row = {"at": float(now if now is not None else time.time()),
480
+ "channel": str((agent or {}).get("channel") or ""),
481
+ "action": str(action or "")[:80],
482
+ "values": values if isinstance(values, dict) else {}}
483
+ rt.update(PENDING_KEY, lambda cur: {**(cur or {}), token: row}, flush="sync")
484
+ return token
485
+
486
+
487
+ #: `{action kind: fn(rt, row) -> None}`. Registered by whatever can actually perform a mutation.
488
+ #:
489
+ #: β›”β›” EMPTY IN PRODUCTION TODAY, AND SAYING SO IS THE POINT. `commit_pending` below is the ONE
490
+ #: place an approved write may be performed, so it is the place a negative control can bite β€” but
491
+ #: a gate is only as honest as what it guards. Until something registers here, an approved card
492
+ #: performs NOTHING and `commit_pending` returns `no_committer` rather than pretending. That is a
493
+ #: declared absence, not a silent one [[flag-shipped-without-its-writer]].
494
+ _COMMITTERS = {}
495
+
496
+
497
+ def register_committer(kind, fn):
498
+ """Declare who may perform an approved mutation of `kind`. Idempotent by key."""
499
+ _COMMITTERS[str(kind)] = fn
500
+
501
+
502
+ def commit_pending(rt, token, approved, now=None):
503
+ """Resolve ONE approval card. Returns one of
504
+ `expired` Β· `unknown` Β· `rejected` Β· `no_committer` Β· `committed` Β· `failed`.
505
+
506
+ β›”β›” THIS IS THE WRITE GATE, AND IT IS A SEPARATE FUNCTION FROM THE ROUTE ON PURPOSE. T67's
507
+ `done-when` asks for an NC proving a REJECTED card writes nothing. A control aimed at the
508
+ route could not bite while the route had no write in it at all β€” it would stay green with the
509
+ guard deleted, because there would be nothing for the guard to be protecting anything from
510
+ ([[gate-can-report-green-on-nothing]], and a verifier caught exactly that on this ticket's
511
+ first draft). Putting the commit behind ONE named seam gives the control a real subject: the
512
+ gate registers a recorder, and `rejected` must leave it untouched while `approve` must call it
513
+ exactly once.
514
+
515
+ β›” THE TOKEN IS CONSUMED BEFORE EITHER BRANCH, and before any committer runs. A card that
516
+ survives its own click is a replayable write, and "approve twice" must never mean "write
517
+ twice". The delete is `flush="sync"` for the same reason.
518
+ """
519
+ now = float(now if now is not None else time.time())
520
+ row = _pending(rt).get(str(token))
521
+ if not isinstance(row, dict):
522
+ return "unknown"
523
+ # Consume FIRST β€” before the TTL verdict, before the approve/reject branch, before any write.
524
+ rt.update(PENDING_KEY,
525
+ lambda cur: {k: v for k, v in (cur or {}).items() if k != str(token)},
526
+ flush="sync")
527
+ if now - float(row.get("at") or 0) > APPROVAL_TTL_S:
528
+ return "expired"
529
+ if not approved:
530
+ # β›” NOTHING BELOW THIS LINE RUNS FOR A REJECTED CARD. The whole gate is this return.
531
+ return "rejected"
532
+ fn = _COMMITTERS.get(str(row.get("action") or ""))
533
+ if fn is None:
534
+ return "no_committer"
535
+ try:
536
+ fn(rt, row)
537
+ except Exception: # noqa: BLE001
538
+ # A committer that raises must not read as a commit. The card is already consumed, so the
539
+ # honest report is that it failed, and the human asks again.
540
+ return "failed"
541
+ return "committed"
542
+
543
+
544
+ # ── the UNAUTHENTICATED Slack door ───────────────────────────────────────────────────────────
545
+ #: Sliding window, keyed on the SLACK TEAM, never the client IP.
546
+ #:
547
+ #: β›” WHY NOT THE IP, WHICH IS WHAT `routes_forms` DOES. Every Slack event arrives from Slack's own
548
+ #: infrastructure, so an IP window is one shared bucket for every workspace on the platform: one
549
+ #: busy tenant would spend the allowance for all of them, and the symptom would be another
550
+ #: tenant's bot going quiet. The team id is the closest thing to the actual noisy party.
551
+ #: ⚠ It is attacker-CONTROLLED before the signature is checked, so the window is applied AFTER
552
+ #: verification, never before β€” an unsigned request is refused on cost grounds anyway (a signature
553
+ #: check is a single HMAC).
554
+ RATE_WINDOW_S, RATE_PER_WINDOW = 60, 60
555
+ _HITS: dict = {}
556
+
557
+
558
+ def _rate_ok(key, now):
559
+ seen = [t for t in _HITS.get(key, ()) if now - t < RATE_WINDOW_S]
560
+ seen.append(now)
561
+ _HITS[key] = seen
562
+ if len(_HITS) > 4096:
563
+ for k in [k for k, v in _HITS.items() if not v or now - v[-1] > RATE_WINDOW_S]:
564
+ _HITS.pop(k, None)
565
+ return len(seen) <= RATE_PER_WINDOW
566
+
567
+
568
+ async def _raw_body(request):
569
+ """The body as RAW BYTES, capped.
570
+
571
+ β›”β›” RAW, NOT PARSED, AND THAT IS NOT A STYLE CHOICE. Slack signs the literal string
572
+ `v0:{timestamp}:{body}` byte for byte. `routes_forms._bounded_body` β€” the function this is
573
+ otherwise copied from β€” returns only the parsed dict and DISCARDS the bytes, so a verbatim
574
+ copy of it could not verify a Slack signature at all: re-serialising the dict changes
575
+ whitespace and key order and the HMAC no longer matches. The bug would look like "Slack
576
+ signatures are always invalid", which is indistinguishable from a wrong secret.
577
+
578
+ ⚠ And it streams rather than calling `request.body()`: FastAPI would otherwise have buffered
579
+ the whole body before the handler's first line, so a content-length check inside the handler
580
+ caps nothing (`routes_forms`' own scar, recorded in its docstring).
581
+ """
582
+ size, chunks = 0, []
583
+ async for chunk in request.stream():
584
+ size += len(chunk)
585
+ if size > MAX_BODY_BYTES:
586
+ raise err(413, "body_too_large", "that request is too large")
587
+ chunks.append(chunk)
588
+ return b"".join(chunks)
589
+
590
+
591
+ def _refuse():
592
+ """ONE refusal for every reason. No oracle.
593
+
594
+ A wrong signature, a stale timestamp, an unknown team, an unconfigured tenant and a channel
595
+ with no agent all answer identically β€” otherwise the door tells an unauthenticated caller
596
+ which tenants exist and which channels are configured, which is the question it was built to
597
+ not answer. `routes_forms._refuse` takes the same posture for the same reason.
598
+ """
599
+ return err(403, "bad_request", "that request could not be verified")
600
+
601
+
602
+ def _verify(secret, timestamp, raw, signature, now):
603
+ """Slack's v0 signature, with the replay window checked FIRST."""
604
+ try:
605
+ ts = int(str(timestamp or "0"))
606
+ except ValueError:
607
+ return False
608
+ if abs(now - ts) > SLACK_MAX_SKEW_S:
609
+ return False
610
+ base = b"v0:" + str(ts).encode("ascii") + b":" + (raw or b"")
611
+ import hashlib
612
+ expected = "v0=" + hmac.new(str(secret).encode("utf-8"), base, hashlib.sha256).hexdigest()
613
+ return hmac.compare_digest(expected, str(signature or ""))
614
+
615
+
616
+ def _resolve_tenant(raw, timestamp, signature, now):
617
+ """`(slug, runtime, creds)` for the tenant whose signing secret verifies this request.
618
+
619
+ β›” THE TENANT IS DECIDED BY THE SIGNATURE, NEVER BY THE PAYLOAD. A body-supplied `team_id`
620
+ would let an unauthenticated caller name the tenant it wants to be β€” the exact widening
621
+ `deps._user_for`'s tenant-equality check closes on the authenticated side. So every configured
622
+ tenant's secret is tried and the one that VERIFIES names the tenant.
623
+
624
+ ⚠ NO EARLY `break` ON A FAILED CANDIDATE and no per-tenant error: the loop's cost must not
625
+ depend on which tenant matched. `routes_forms._resolve` walks tenants the same way, for the
626
+ same reason β€” a public door has no session to ask.
627
+ """
628
+ from harness import runtime as _rt
629
+ hit = None
630
+ try:
631
+ slugs = list(_rt.known_tenants())
632
+ except Exception: # noqa: BLE001
633
+ return None, None, None
634
+ for slug in slugs:
635
+ try:
636
+ rt = _rt.get_runtime(slug)
637
+ except Exception: # noqa: BLE001
638
+ continue
639
+ creds = slack_creds(rt)
640
+ if not creds:
641
+ continue
642
+ if _verify(creds.get("signing_secret"), timestamp, raw, signature, now) and hit is None:
643
+ hit = (slug, rt, creds)
644
+ return hit if hit else (None, None, None)
645
+
646
+
647
+ @router.post("/slack/events")
648
+ async def slack_events(request: Request):
649
+ """Slack's Events API. UNAUTHENTICATED BY CONSTRUCTION β€” no `Depends(require_session)`.
650
+
651
+ ⭐ Built like `routes_forms.py`'s public door and not like OpenTag's: one process, no vendor,
652
+ no second runtime, no persistent outbound socket, and this tenant's Slack secret never leaves
653
+ this deployment (amendment A1).
654
+ """
655
+ now = time.time()
656
+ raw = await _raw_body(request)
657
+ ts = request.headers.get("x-slack-request-timestamp")
658
+ sig = request.headers.get("x-slack-signature")
659
+ slug, rt, _creds = _resolve_tenant(raw, ts, sig, now)
660
+ if not rt:
661
+ raise _refuse()
662
+
663
+ try:
664
+ payload = json.loads(raw or b"{}")
665
+ except ValueError:
666
+ raise _refuse()
667
+ if not isinstance(payload, dict):
668
+ raise _refuse()
669
+
670
+ # Slack's one-time endpoint handshake. Answered ONLY after the signature verified β€” an
671
+ # unsigned challenge echo would confirm the endpoint exists to anyone who probes it.
672
+ if payload.get("type") == "url_verification":
673
+ return {"challenge": str(payload.get("challenge") or "")[:512]}
674
+
675
+ team = str(payload.get("team_id") or slug)
676
+ if not _rate_ok(f"{slug}:{team}", now):
677
+ raise err(429, "too_many_requests", "too many requests β€” wait a moment and try again")
678
+
679
+ event = payload.get("event") if isinstance(payload.get("event"), dict) else {}
680
+ channel = str(event.get("channel") or "")
681
+ agent = None
682
+ for a in _agents(rt).values():
683
+ if isinstance(a, dict) and str(a.get("channel") or "") == channel:
684
+ agent = a
685
+ break
686
+ # β›” NO AGENT, OR AN INACTIVE ONE, IS SILENCE β€” a 200 with no action. Not a 403: Slack retries
687
+ # a non-2xx up to three times, so refusing here would turn "this channel is not configured"
688
+ # into three refusals per message, and the retry storm would be the only visible symptom.
689
+ # ⚠ And it must not say WHICH β€” a distinguishable answer tells an unauthenticated caller which
690
+ # channels this workspace has configured.
691
+ if not isinstance(agent, dict) or agent.get("active", True) is False:
692
+ return {"ok": True}
693
+
694
+ # β›”β›” THE ANSWERING HALF IS NOT BUILT YET, AND SAYING SO IS THE POINT. What ships here is the
695
+ # DOOR and the WALL: a verified request, resolved to a tenant by signature, matched to a
696
+ # channel agent whose `perms` block is a `perm_scope` principal (`agent_principal`). Composing
697
+ # a reply means an LLM call on the cheap-first ladder plus the read path, and a half-built
698
+ # answerer that returns something plausible is worse than one that returns nothing.
699
+ # Booked rather than faked [[flag-shipped-without-its-writer]].
700
+ return {"ok": True}
701
+
702
+
703
+ @router.post("/slack/interact")
704
+ async def slack_interact(request: Request):
705
+ """The approval card's button (`W33-T67`). UNAUTHENTICATED, signature-verified, same shape."""
706
+ now = time.time()
707
+ raw = await _raw_body(request)
708
+ slug, rt, _creds = _resolve_tenant(raw, request.headers.get("x-slack-request-timestamp"),
709
+ request.headers.get("x-slack-signature"), now)
710
+ if not rt:
711
+ raise _refuse()
712
+ # Slack posts interactions as `application/x-www-form-urlencoded` with a `payload=` field.
713
+ from urllib.parse import parse_qs
714
+ try:
715
+ form = parse_qs(raw.decode("utf-8"))
716
+ payload = json.loads((form.get("payload") or ["{}"])[0])
717
+ except Exception: # noqa: BLE001
718
+ raise _refuse()
719
+ if not isinstance(payload, dict):
720
+ raise _refuse()
721
+ if not _rate_ok(f"{slug}:interact", now):
722
+ raise err(429, "too_many_requests", "too many requests β€” wait a moment and try again")
723
+
724
+ actions = payload.get("actions") if isinstance(payload.get("actions"), list) else []
725
+ choice = (actions[0] if actions and isinstance(actions[0], dict) else {})
726
+ token = str(choice.get("value") or "")
727
+ # ⭐ ANYTHING THAT IS NOT LITERALLY "approve" IS A REJECT. Fail-closed on a malformed,
728
+ # truncated or unknown action id: the direction that can be wrong here without anyone
729
+ # noticing is the one that writes.
730
+ approved = str(choice.get("action_id") or "") == "approve"
731
+ # β›” THE ROUTE DOES NOT DECIDE β€” `commit_pending` does, and it is the ONE seam a negative
732
+ # control can aim at. Consumption, the TTL, the reject branch and the committer lookup all
733
+ # live there; this handler only turns its verdict into a sentence.
734
+ verdict = commit_pending(rt, token, approved, now=now)
735
+ return {"text": {
736
+ # An unknown, already-used or expired token is not an error: a person clicking a stale
737
+ # card should be told it is stale, not shown a failure.
738
+ "unknown": "That approval is no longer available. Ask the bot again.",
739
+ "expired": "That approval expired before it was answered. Ask the bot again.",
740
+ "rejected": "Rejected. Nothing was changed.",
741
+ # ⚠ HONEST, not reassuring: the answering half that would register a committer is booked,
742
+ # so an approved card today performs nothing and says exactly that rather than "Approved."
743
+ "no_committer": "Approved β€” but this workspace has nothing configured to carry it out yet.",
744
+ "failed": "That change could not be completed. Nothing was saved; ask the bot again.",
745
+ "committed": "Approved.",
746
+ }.get(verdict, "That approval is no longer available. Ask the bot again.")}
747
+
748
+
749
+ @router.get("/slack/health")
750
+ def slack_health(session: Session = Depends(require_session)):
751
+ """Is Slack configured for THIS tenant? Session-gated, and it answers about one tenant only β€”
752
+ enumerating the others would answer a question about our customer list."""
753
+ creds = slack_creds(session.runtime)
754
+ return {"configured": creds is not None,
755
+ "agents": len(_agents(session.runtime)),
756
+ "hasBotToken": bool((creds or {}).get("bot_token"))}
api/routes_starred.py ADDED
@@ -0,0 +1,828 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """routes_starred.py β€” WAVE 35 (rulings R4/R5/R6/R7, contracts C2/C3/C5): THE STAR.
2
+
3
+ Owner items 8, 9 and 12. **Star and "mark important" are ONE idea and ONE flag** (R4): on screen
4
+ the word is Star, and starring an object lands it on Home and in the new Starred module. Four
5
+ kinds are starrable β€” `database`, `agent`, `query`, `view` β€” and RECORDS get their own door
6
+ (C5, `/starred/records`), because a record's identity is `(database, pid)` rather than a bare id.
7
+
8
+ β›”β›” THE ONE DESIGN RULE THAT DECIDES EVERYTHING BELOW: **A VIEW'S STAR IS NOT STORED HERE.**
9
+ C2 is explicit β€” `kind: "view"` writes the EXISTING `config.important` in that database's own
10
+ `<key>_table_workspace`, and never a second store. R4 says star and mark-important are one flag;
11
+ two stores for one flag is two flags that can disagree, and the badge inside the database and the
12
+ row on Home would then answer differently about the same view. So this file keeps THREE id lists
13
+ (`databases`, `agents`, `queries`) in its own per-user bucket, and for views it is a READER and a
14
+ WRITER of somebody else's bucket rather than an owner of anything.
15
+
16
+ ⚠ WHICH IS WHY `GET /starred` IS NOT FREE, AND EVERY CALLER MUST KNOW. Finding the starred views
17
+ means reading each visible database's view bucket β€” the same N reads `routes_nav._important_counts`
18
+ did before W35-T43 deleted it. Measured on tenant #0 (2026-08-16, recorded on that constant):
19
+ twelve buckets cost **6.2 ms WARM in total** and **7,331 ms of first fetch COLD**. So the scan is
20
+ budgeted, it is spent in the caller's own RECENTS order (a mark lives on a database somebody works
21
+ in), and a database it could not reach is named in `degraded` rather than quietly carrying no star.
22
+ That is R6's second sentence applied to a list: a limit that cannot be removed is REPORTED with its
23
+ cause, never silently enforced. **Call this AFTER first paint.**
24
+
25
+ ⚠ NO INDEX, DELIBERATELY. The obvious fix for the scan is a `starred.views` id list written beside
26
+ the three above. That is the second store C2 forbids: the moment the flag and the index can
27
+ disagree β€” a view deleted, a view unmarked from inside the database, a store write that lands on one
28
+ and not the other β€” the product has two answers to one question and no way to tell which is right.
29
+ The scan is the price of one flag, and it is paid off the critical path.
30
+ """
31
+ import time
32
+
33
+ from fastapi import APIRouter, Body, Depends
34
+
35
+ import routes_nav
36
+ from deps import Session, err, require_session
37
+
38
+ router = APIRouter(prefix="/api/v1")
39
+
40
+ #: The per-user bucket holding the three ID-LIST kinds. `{username: {databases, agents, queries}}`
41
+ #:
42
+ #: ⚠ PER USER (R6), like `nav_recents` and unlike `nav_meta`: what I care about is not a fact about
43
+ #: the object, so a tenant-wide star would put one person's shortlist on everybody's Home.
44
+ _STARRED_KEY = "starred"
45
+
46
+ #: The four kinds C2 declares. `view` is the odd one out β€” see the header.
47
+ KINDS = ("database", "agent", "query", "view")
48
+
49
+ #: The three kinds this file's own bucket holds, and the payload key each answers under.
50
+ _LIST_KINDS = {"database": "databases", "agent": "agents", "query": "queries"}
51
+
52
+ #: A bound on each id list. β›” REPORTED, NEVER SILENT: a POST that would cross it is refused with
53
+ #: the cause and the action (unstar something), rather than being accepted and truncated. A star
54
+ #: silently dropped is a star the user believes they set.
55
+ MAX_PER_KIND = 200
56
+
57
+ #: An id is a store key, a view id or an automation id β€” all of which are short. Bounded so a
58
+ #: hostile client cannot grow one user's document with one very long string.
59
+ _MAX_ID = 120
60
+
61
+ #: The wall-clock ceiling on the WHOLE view scan, checked between databases β€” the same number and
62
+ #: the same reason as the `_IMPORTANT_BUDGET_S` this replaces. It bounds the FIRST request after a
63
+ #: container starts; warm it is not reached.
64
+ VIEW_SCAN_BUDGET_S = 2.5
65
+
66
+
67
+ def _clean_id(raw):
68
+ """One id, or "" β€” the only shape this file stores."""
69
+ return str(raw or "").strip()[:_MAX_ID]
70
+
71
+
72
+ def _clean_list(raw):
73
+ """A stored id list, validated on the way OUT as well as in.
74
+
75
+ Re-validating a read is not redundant: the bucket outlives this code and an older build (or a
76
+ hand edit) could have written a shape this one does not accept. Prune, never invent β€” the
77
+ posture `_clean_nav_prefs` takes one file over.
78
+ """
79
+ if not isinstance(raw, list):
80
+ return []
81
+ out, seen = [], set()
82
+ for item in raw[:MAX_PER_KIND]:
83
+ got = _clean_id(item)
84
+ if got and got not in seen:
85
+ seen.add(got)
86
+ out.append(got)
87
+ return out
88
+
89
+
90
+ def _read_lists(session):
91
+ """This caller's three id lists. A store blip answers empty rather than taking the page down."""
92
+ try:
93
+ mine = (session.runtime.get(_STARRED_KEY) or {}).get(session.uname) or {}
94
+ except Exception:
95
+ return {name: [] for name in _LIST_KINDS.values()}
96
+ if not isinstance(mine, dict):
97
+ return {name: [] for name in _LIST_KINDS.values()}
98
+ return {name: _clean_list(mine.get(name)) for name in _LIST_KINDS.values()}
99
+
100
+
101
+ def _database_keys(session):
102
+ """`(keys, enumerated)` β€” every database this session may open that has a view bucket.
103
+
104
+ ⚠ `routes_nav._visible_database_keys` IS REUSED RATHER THAN RE-DERIVED, and that is not
105
+ tidiness: it merges the compiled registry (behind the account grant AND the tenant catalogue)
106
+ with `user_tables.nav_entries`, the `may_open`-filtered listing over the rows-free projection.
107
+ Writing a second enumeration here is how the folder-drop defect happened (wave 27, item 6): one
108
+ door knew about `ut_*` keys and the other did not, and the disagreeing door answered 200 while
109
+ storing nothing.
110
+ β›” AND IT IS NOT `_placeable_top_keys`, WHICH NEVER APPLIES THE TENANT CATALOGUE β€” right for a
111
+ cosmetic folder placement, wrong for a scan that then opens each database's view bucket. See
112
+ that helper's own note for the second difference (module surfaces have no views to star).
113
+ """
114
+ return routes_nav._visible_database_keys(session)
115
+
116
+
117
+ def _scan_order(session, keys):
118
+ """`keys`, RECENTS-FIRST β€” the order the scan budget is spent in.
119
+
120
+ β›” THE ORDER IS LOAD-BEARING AND W34-T10 LEARNED IT THE EXPENSIVE WAY. A budget spent in
121
+ whatever order the keys happen to arrive is a lottery: measured cold on tenant #0, that block
122
+ spent all of its first draft's budget on ten EMPTY Odoo buckets and was cut off two rows before
123
+ `customer_data`, which holds the only marked view in the tenant. The feature would have been
124
+ correct and shown nothing. A star lives on a database somebody works in, and `nav_recents` is
125
+ exactly that list, newest first.
126
+ """
127
+ recents = routes_nav._read_recents(session.runtime, session.uname, keys)
128
+ order = [r["key"] for r in recents]
129
+ seen = set(order)
130
+ order += [k for k in sorted(keys) if k not in seen]
131
+ return order
132
+
133
+
134
+ def _view_docs(session, order):
135
+ """`({key: workspace_doc}, unread)` β€” each database's view bucket, PROJECTED, within budget.
136
+
137
+ β›” PROJECTED, and the two dropped keys are what makes this affordable: `overlays` is per-record
138
+ cell values and `fields` is the schema stratum, neither of which says anything about a view. On
139
+ `customer_data` they are most of the bucket.
140
+ ⚠ `unread` is a store blip OR the budget β€” reported, never a confident absence.
141
+ """
142
+ import core.view_templates as view_templates
143
+ docs, unread = {}, []
144
+ started = time.perf_counter()
145
+ for key in order:
146
+ ws_key = view_templates.workspace_key(key)
147
+ if not ws_key:
148
+ continue # a module surface or a folder head, not a table at all
149
+ if time.perf_counter() - started > VIEW_SCAN_BUDGET_S:
150
+ unread.append(key)
151
+ continue
152
+ try:
153
+ docs[key] = session.runtime.get_projection(ws_key,
154
+ drop=("overlays", "fields")) or {}
155
+ except Exception:
156
+ unread.append(key)
157
+ return docs, unread
158
+
159
+
160
+ def _refresh_one(session, cache, key):
161
+ """Re-read ONE database's bucket inside a cache the caller already holds.
162
+
163
+ β›” THIS IS WHY A WRITE MAY REUSE A SCAN AT ALL. The star write changes exactly one bucket, so
164
+ every OTHER document in the cache is still true β€” but the one just written is stale by
165
+ definition, and serving it back would answer the POST with the flag's OLD value. That is a lost
166
+ write wearing the face of a failed read ([[refetch-eats-its-own-write]] in reverse), and it is
167
+ the trap that makes "just reuse the cache" wrong without this function.
168
+ """
169
+ import core.view_templates as view_templates
170
+ order, docs, unread = cache
171
+ ws_key = view_templates.workspace_key(key)
172
+ if not ws_key:
173
+ return cache
174
+ try:
175
+ docs[key] = session.runtime.get_projection(ws_key, drop=("overlays", "fields")) or {}
176
+ except Exception:
177
+ docs.pop(key, None)
178
+ if key not in unread:
179
+ unread.append(key)
180
+ return (order, docs, unread)
181
+
182
+
183
+ def _scan_starred_views(session, keys=None, cache=None):
184
+ """`(rows, unread)` β€” every view this caller has starred, newest-worked database first.
185
+
186
+ A row is `{"id", "database", "name", "config"}`. ⚠ `config` is carried so the COUNTS route does
187
+ not have to read every bucket a second time: `_view_docs` is the expensive half of this file
188
+ (7,331 ms cold for twelve buckets) and two consumers asking the same question twice per request
189
+ is the 1+N shape D-175 spent a wave removing from `/nav`. `_starred_views` below drops it for the
190
+ wire.
191
+
192
+ β›” `cache` IS `(order, docs, unread)` FROM AN EARLIER PASS IN THE SAME REQUEST, and it exists
193
+ because the view-star WRITE otherwise scanned every bucket TWICE: once in `_find_view` to locate
194
+ the view, then again to build the answer. That is the same 1+N shape, in the same file, on the
195
+ same expensive read β€” introduced an hour after it was removed one function away. A caller that
196
+ has written must `_refresh_one` the database it wrote before passing its cache.
197
+ """
198
+ if cache is not None:
199
+ order, docs, unread = cache
200
+ else:
201
+ if keys is None:
202
+ keys, _ = _database_keys(session)
203
+ order = _scan_order(session, keys)
204
+ docs, unread = _view_docs(session, order)
205
+ granted = routes_nav._granted_view_ids(session)
206
+ out = []
207
+ for key in order:
208
+ doc = docs.get(key)
209
+ if doc is None:
210
+ continue
211
+ for vid, view in routes_nav._visible_views(doc, session.uname, session.admin,
212
+ granted).items():
213
+ cfg = view.get("config") or {}
214
+ if cfg.get("important") is True:
215
+ out.append({"id": vid, "database": key,
216
+ "name": str(view.get("name") or vid)[:120], "config": cfg})
217
+ return out, unread
218
+
219
+
220
+ def _starred_views(session, keys=None, cache=None):
221
+ """The C2 wire rows β€” `{"id", "database", "name"}`, with the config dropped.
222
+
223
+ ⭐ `name` is a SUPERSET of C2, published to B as `NOTE E-1`: the scan already holds the record,
224
+ so the name is free, and without it Home would render a starred view as a bare id.
225
+ β›” THE CONFIG DOES NOT GO ON THE WIRE. It carries `memberPids` (a curated view's whole pid list)
226
+ and every filter leaf, which is somebody's row set travelling to a client that asked for a label.
227
+ """
228
+ rows, unread = _scan_starred_views(session, keys, cache=cache)
229
+ return ([{"id": r["id"], "database": r["database"], "name": r["name"]} for r in rows],
230
+ unread)
231
+
232
+
233
+ def _query_names(session, qids):
234
+ """`({qid: name}, resolved)` for the caller's own Query artefacts.
235
+
236
+ ⭐ ANSWERS `ASK B-1 (2)`. C2 declares `queries: [qid]` and there is no view/query id -> label map
237
+ anywhere in the client, so Home and Starred would render a starred Query as a bare id. B's only
238
+ alternative was `GET /query`, which drags threads, messages, citations and every saved view
239
+ across the wire to render two strings.
240
+
241
+ ⚠ IT READS LANE C's BUCKET AND CALLS LANE C's HELPERS, WITHOUT EDITING LANE C's FILE. Query
242
+ state is ONE small per-user document (`query_user_<hash>`: threads, messages, citations, views β€”
243
+ no grid rows), and `_namespace_key`/`_state` are pure functions of the session. Re-deriving the
244
+ key here would be a second spelling of a hash, which is the one thing guaranteed to drift.
245
+
246
+ β›” `resolved` IS FALSE WHEN THE READ FAILED, and the caller must not prune on it β€” a store blip
247
+ would otherwise delete somebody's whole starred-query list. Same posture as
248
+ `_clean_nav_prefs(keep_unknown_ut=...)`.
249
+ """
250
+ if not qids:
251
+ return {}, True
252
+ try:
253
+ import routes_query
254
+ views = (routes_query._state(session) or {}).get("views") or {}
255
+ except Exception:
256
+ return {}, False
257
+ out = {}
258
+ for qid in qids:
259
+ row = views.get(qid)
260
+ if isinstance(row, dict):
261
+ out[qid] = str(row.get("name") or qid)[:120]
262
+ return out, True
263
+
264
+
265
+ def _payload(session, keys=None, cache=None):
266
+ """The C2 answer: the three id lists, the starred views, and what could not be read.
267
+
268
+ ⚠ `degraded` SHIPS AS `[]` RATHER THAN BEING OMITTED WHEN EMPTY β€” the same rule `/nav`'s
269
+ `omitted`/`degraded` follow, for the same reason: a key a consumer has to test for is a key a
270
+ consumer forgets to test for.
271
+ ⚠ `cache` is a scan an earlier pass in THIS request already paid for β€” see `_scan_starred_views`.
272
+ """
273
+ lists = _read_lists(session)
274
+ if keys is None:
275
+ keys, _ = _database_keys(session)
276
+ # ⚠ PRUNED ON READ, NEVER ON WRITE β€” `_read_recents`' posture, and both halves are deliberate.
277
+ # A database whose grant was revoked must stop being offered without anybody running a
278
+ # migration, and must come back if the grant does. A key that is not visible therefore reaches
279
+ # the store and never reaches a screen.
280
+ # ⚠ ONLY `databases` is pruned. An agent id or a query id would each cost another store read to
281
+ # validate, and pruning them here buys nothing: this list is the CALLER'S OWN writes, so it can
282
+ # disclose nothing they did not already know, and B/C render only objects they can already see
283
+ # (the way `groupRecents` drops an unreachable recent). Said out loud rather than left as a gap.
284
+ lists["databases"] = [k for k in lists["databases"] if k in keys]
285
+ # ⭐ ASK B-1 (2): the names, and β€” because the read that answers "what is it called" also answers
286
+ # "does it still exist" β€” the same prune the databases get. β›” ONLY WHEN THE READ SUCCEEDED: a
287
+ # store blip that returned no artefacts must not delete the whole starred-query list.
288
+ qnames, qresolved = _query_names(session, lists["queries"])
289
+ if qresolved:
290
+ lists["queries"] = [q for q in lists["queries"] if q in qnames]
291
+ views, unread = _starred_views(session, keys, cache=cache)
292
+ out = {"databases": lists["databases"], "agents": lists["agents"],
293
+ "queries": lists["queries"], "queryNames": qnames,
294
+ "views": views, "degraded": unread}
295
+ if unread:
296
+ # R6's second sentence, in the payload: the CAUSE and what it means, not a bare list.
297
+ out["note"] = (f"{len(unread)} database(s) could not be read in time, so a view you "
298
+ f"starred there is missing from this list. Open one of them once and the "
299
+ f"next load will include it")
300
+ return out
301
+
302
+
303
+ @router.get("/starred")
304
+ def starred(session: Session = Depends(require_session)):
305
+ """C2 β€” everything this caller has starred.
306
+
307
+ ⚠ NOT ON `/nav`'s PATH, and R7 is the ruling: `/nav` is a rows-free projection whose budget was
308
+ already the subject of D-175/D-185/D-288. This route is called AFTER Home/Starred paint.
309
+ """
310
+ return _payload(session)
311
+
312
+
313
+ def _find_view(session, vid, database=""):
314
+ """`(key, ws_key, stratum, view, cache)` for a view this caller can SEE, else None.
315
+
316
+ `stratum` is the username whose block holds it, or `table_store.SHARED_KEY`. `cache` is the
317
+ `(order, docs, unread)` this call paid for β€” but **only when it scanned everything**. With a
318
+ `database` hint it read ONE bucket, which is not a complete answer for the caller's whole
319
+ starred list, so it hands back None and the caller pays for a full pass. Handing back a partial
320
+ cache would silently drop every starred view on every OTHER database.
321
+
322
+ ⚠ `database` IS AN OPTIONAL FAST PATH (a superset of C2, published to B and C): the client
323
+ almost always knows which database the view belongs to, and naming it turns an N-bucket scan
324
+ into one read. Absent, the scan runs β€” because C2's body carries only `kind`/`id`/`on`, and a
325
+ door that REQUIRED the hint would break the contract two lanes are building against.
326
+ """
327
+ import core.table_store as table_store
328
+ import core.view_templates as view_templates
329
+
330
+ keys, _ = _database_keys(session)
331
+ if database:
332
+ # An unknown or invisible key must not become a wider scan. It answers "not found", which
333
+ # is the same answer an id nobody holds gets β€” see the write door's note on why.
334
+ if database not in keys:
335
+ return None
336
+ order, cache = [database], None
337
+ else:
338
+ order = _scan_order(session, keys)
339
+ docs, unread = _view_docs(session, order)
340
+ if not database:
341
+ cache = (order, docs, unread)
342
+ granted = routes_nav._granted_view_ids(session)
343
+ for key in order:
344
+ doc = docs.get(key)
345
+ if doc is None:
346
+ continue
347
+ if str(vid) not in routes_nav._visible_views(doc, session.uname, session.admin, granted):
348
+ continue
349
+ ws_key = view_templates.workspace_key(key)
350
+ # WHICH stratum holds it decides the WALL, so it is resolved here rather than guessed.
351
+ # Personal first: a caller's own block is the common case and needs no grant.
352
+ if isinstance(doc.get(session.uname), dict) and \
353
+ str(vid) in ((doc[session.uname].get("views") or {})):
354
+ return (key, ws_key, session.uname, doc[session.uname]["views"][str(vid)], cache)
355
+ shared = (doc.get(table_store.SHARED_KEY) or {}).get("views") or {}
356
+ if str(vid) in shared:
357
+ return (key, ws_key, table_store.SHARED_KEY, shared[str(vid)], cache)
358
+ for stratum, blob in doc.items():
359
+ if stratum == session.uname or not isinstance(blob, dict):
360
+ continue
361
+ if str(vid) in (blob.get("views") or {}):
362
+ return (key, ws_key, str(stratum), blob["views"][str(vid)], cache)
363
+ return None
364
+
365
+
366
+ def _may_star_view(session, stratum, view):
367
+ """May this caller WRITE the star on this view? Fail-closed.
368
+
369
+ β›” A VIEW ID THE CALLER CANNOT WRITE MUST REFUSE **LOUDLY**, and this is a standing scar:
370
+ `view_upsert` answers **200 while writing nothing** when the id belongs to a view the caller
371
+ cannot see, so a pinned id is effectively tenant-scoped and every symptom of it is silent
372
+ ([[a-view-id-another-user-holds-refuses-silently]]). This function is the reason the write door
373
+ below raises instead of shrugging.
374
+
375
+ THREE HOMES, THREE WALLS:
376
+ Β· the caller's OWN stratum β€” theirs, no further question;
377
+ Β· `__shared__` β€” `table_store._may_edit`, which is the predicate the view door itself uses;
378
+ Β· another user's stratum, reached by a wave-21 grant β€” the grant must be `edit` or `owner`.
379
+ ⚠ A `view` ROLE IS NOT A WRITE GRANT. `_granted_view_ids` admits both roles because it
380
+ answers "can they SEE it"; this answers "may they CHANGE it", and collapsing the two would
381
+ let a read-only grantee mark somebody else's shared view.
382
+ """
383
+ import core.shares as shares
384
+ import core.table_store as table_store
385
+ if stratum == session.uname:
386
+ return True
387
+ if stratum == table_store.SHARED_KEY:
388
+ return bool(table_store._may_edit(view, session.uname, session.admin))
389
+ return shares.role_for("view", str((view or {}).get("id") or ""), session.uname,
390
+ is_admin=session.admin,
391
+ st=session.runtime) in ("edit", "owner")
392
+
393
+
394
+ def _write_view_star(session, ws_key, stratum, vid, on):
395
+ """Set `config.important` IN PLACE, in the view's own bucket. Returns True when it landed.
396
+
397
+ β›”β›” IN PLACE, AND **NOT** THROUGH `grid_events.view_upsert` OR `table_store.save_view` β€” three
398
+ separate reasons, each of which has cost this repo something:
399
+
400
+ 1. `view_upsert` REBUILDS `config` from a literal allowlist and filters `memberPids` against
401
+ `allowed_pids` and `cohortLock` against `cohort_ids`. Routing a flag toggle through it
402
+ means a cohort view whose pool this request did not assemble comes back with its curated
403
+ row set pruned β€” "the 40 accounts we agreed to call" quietly becoming fewer, under the
404
+ same name, with nothing going red.
405
+ 2. `table_store.save_view` re-runs `_unique_name` against every view name in the document.
406
+ A star has no business re-allocating a name; a collision would RENAME the starred view.
407
+ 3. Both are "write the whole view" doors. This writes one boolean.
408
+
409
+ ⚠ SET UNCONDITIONALLY, never "only when True" β€” `grid_events` records the same rule beside its
410
+ own `important` line: a key written only when present means unstarring is silently a no-op and
411
+ the stored `true` survives, i.e. a mark you can set and never clear.
412
+ ⚠ `bool(on)`, so a JSON `"false"` cannot become a star.
413
+ ⚠ FLUSH IS THE DEFAULT (sync). This is a deliberate one-at-a-time act like a folder rename, not
414
+ the autosave hot path `table_store` uses `async` for β€” and a coalesced write is exactly what
415
+ made eighteen 200s land zero rows in wave 33.
416
+ """
417
+ landed = {"ok": False}
418
+
419
+ def _up(data):
420
+ data = data if isinstance(data, dict) else {}
421
+ block = data.get(stratum)
422
+ if not isinstance(block, dict):
423
+ return data
424
+ views = block.get("views")
425
+ if not isinstance(views, dict) or str(vid) not in views:
426
+ return data
427
+ view = views[str(vid)]
428
+ if not isinstance(view, dict):
429
+ return data
430
+ cfg = view.get("config")
431
+ if not isinstance(cfg, dict):
432
+ cfg = {}
433
+ view["config"] = cfg
434
+ cfg["important"] = bool(on)
435
+ landed["ok"] = True
436
+ return data
437
+
438
+ session.runtime.update(ws_key, _up)
439
+ return landed["ok"]
440
+
441
+
442
+ @router.post("/starred")
443
+ def set_starred(body: dict = Body(default=None),
444
+ session: Session = Depends(require_session)):
445
+ """C2 β€” star or unstar ONE object, and answer with the caller's whole new list.
446
+
447
+ Body: `{"kind": "database"|"agent"|"query"|"view", "id": str, "on": bool}`, plus an OPTIONAL
448
+ `"database"` when `kind` is `view` (the fast path β€” see `_find_view`).
449
+
450
+ ⚠ THE WHOLE LIST COMES BACK, not an ack, because the client is optimistic-then-reconciled
451
+ (T15): an ack would leave the browser as the only place that knows what the new state is.
452
+ """
453
+ body = body if isinstance(body, dict) else {}
454
+ kind = str(body.get("kind") or "").strip()
455
+ oid = _clean_id(body.get("id"))
456
+ if kind not in KINDS:
457
+ raise err(400, "bad_request",
458
+ f"kind must be one of {', '.join(KINDS)}")
459
+ if not oid:
460
+ raise err(400, "bad_request", "no object was named")
461
+ if "on" not in body or not isinstance(body.get("on"), bool):
462
+ # Explicit rather than defaulted: a toggle whose default is "star" turns a malformed
463
+ # unstar into a star, which is the one direction a user cannot undo by repeating it.
464
+ raise err(400, "bad_request", "`on` must be true or false")
465
+ on = bool(body["on"])
466
+ if not session.runtime.available():
467
+ raise err(503, "store_unavailable",
468
+ "the tenant store is unavailable. Nothing was saved.")
469
+
470
+ if kind == "view":
471
+ found = _find_view(session, oid, _clean_id(body.get("database")))
472
+ if not found:
473
+ # β›” 404 FOR BOTH "no such view" AND "not yours to see", deliberately: a distinct answer
474
+ # would turn this route into an oracle that sorts real view ids from invented ones.
475
+ raise err(404, "unknown_view", "that view is not available")
476
+ db_key, ws_key, stratum, view, cache = found
477
+ if not _may_star_view(session, stratum, view):
478
+ raise err(403, "forbidden",
479
+ "that view belongs to somebody else and you have read access only")
480
+ try:
481
+ landed = _write_view_star(session, ws_key, stratum, oid, on)
482
+ except Exception:
483
+ raise err(503, "store_unavailable",
484
+ "the star was not saved: the store refused the write.")
485
+ if not landed:
486
+ # The view moved between the read and the write (deleted, shared, unshared). Loud,
487
+ # because a 200 over a write that changed nothing is the defect this file's own
488
+ # `_may_star_view` note is about.
489
+ raise err(409, "view_moved",
490
+ "that view changed while the star was being saved. Try again")
491
+ # β›” REUSE THE SCAN, REFRESH THE ONE BUCKET WE WROTE. Without this the POST scans every view
492
+ # bucket TWICE β€” once to find the view, once to build the answer β€” which is the 1+N shape
493
+ # this file removed from `/starred/counts` an hour earlier, reintroduced by the write path.
494
+ # ⚠ `_refresh_one` is not optional: the written document is stale by definition and serving
495
+ # it back would answer the POST with the flag's OLD value.
496
+ if cache is not None:
497
+ cache = _refresh_one(session, cache, db_key)
498
+ return _payload(session, cache=cache)
499
+
500
+ field = _LIST_KINDS[kind]
501
+ uname = session.uname
502
+ # β›” THE CAP IS ENFORCED INSIDE THE TRANSACTION, NOT BY A READ BEFORE IT, AND THAT IS NOT
503
+ # PEDANTRY. A pre-flight `len(current) >= MAX_PER_KIND` check is advisory: two concurrent stars
504
+ # both pass it, both append, and the document ends up at MAX+1 β€” where `_clean_list`'s
505
+ # `raw[:MAX_PER_KIND]` then drops one **on the next read**. That is a star the user set, saw
506
+ # confirmed, and cannot find, which is exactly the silent truncation R6's second sentence
507
+ # forbids. Refusing inside the read-modify-write makes the ceiling true rather than likely.
508
+ # ⚠ IT RECORDS THE REFUSAL RATHER THAN RAISING FROM INSIDE `_up`: an exception thrown through
509
+ # `store.update` would surface as this route's own 503 "the store refused the write", which is a
510
+ # different and wrong story about a limit the caller can act on.
511
+ refused = {"full": False}
512
+
513
+ def _up(data):
514
+ data = data if isinstance(data, dict) else {}
515
+ mine = dict(data.get(uname) or {}) if isinstance(data.get(uname), dict) else {}
516
+ ids = _clean_list(mine.get(field))
517
+ if on:
518
+ if oid not in ids:
519
+ if len(ids) >= MAX_PER_KIND:
520
+ refused["full"] = True
521
+ return data
522
+ ids.append(oid)
523
+ else:
524
+ ids = [i for i in ids if i != oid]
525
+ if ids:
526
+ mine[field] = ids
527
+ else:
528
+ mine.pop(field, None)
529
+ # A user with nothing starred is REMOVED rather than stored empty β€” a bucket that
530
+ # accumulates `{}` per account is a document that grows forever and says nothing.
531
+ if mine:
532
+ data[uname] = mine
533
+ else:
534
+ data.pop(uname, None)
535
+ return data
536
+
537
+ try:
538
+ session.runtime.update(_STARRED_KEY, _up)
539
+ except Exception:
540
+ raise err(503, "store_unavailable",
541
+ "the star was not saved: the store refused the write.")
542
+ if refused["full"]:
543
+ # R6's second sentence: the cause and the action, never a 200 over a star that was dropped.
544
+ raise err(400, "starred_full",
545
+ f"you have starred the maximum of {MAX_PER_KIND} items of this kind. "
546
+ f"Unstar one to make room")
547
+ return _payload(session)
548
+
549
+
550
+ # ══════════════════════════════════════════════════════════════════ STARRED COUNTS (R7, C3)
551
+ #
552
+ # ⭐ R7: *"Row counts come from a NEW on-demand endpoint called after Home/Starred paint, never from
553
+ # `/nav`. A view whose count is not free shows the CAUSE, never an invented number."*
554
+ #
555
+ # β›” SO EVERY VID LANDS IN EXACTLY ONE OF `counts` AND `notes`, AND THAT IS THE CONTRACT (C3). A vid
556
+ # in neither is a client rendering nothing with no idea why; a vid in both is two answers to one
557
+ # question. `notes` is not a fallback for "we did not get round to it" β€” it is the CAUSE, in a
558
+ # sentence, with what to do about it, which is R6's second sentence applied to a badge.
559
+
560
+
561
+ def _cohort_sizes(session, key, cache):
562
+ """`{cohort_id: size}` for THIS caller's own cohorts on `key`'s topic. Memoised per request.
563
+
564
+ β›” THIS CALLER'S OWN COHORTS ONLY, and the consequence is deliberate: a SHARED view locked to a
565
+ cohort somebody else owns finds no id here, so it is reported UNCOUNTED with that cause rather
566
+ than counted out of a stratum this session cannot see. Widening the read would make the number
567
+ disclose the SIZE of another person's private list, which is a leak wearing a bug fix.
568
+ ⚠ Read through `session.runtime`, NOT `modules.cohort`'s module-level helpers: those call
569
+ `core.store` directly and carry no tenant namespace. The MODULE is asked for the bucket NAME (it
570
+ owns that rule) and this route does the reading, which is the only tenant-correct combination.
571
+ """
572
+ import core.view_templates as view_templates
573
+ import modules.cohort as cohort_mod
574
+ ws_key = view_templates.workspace_key(key) or ""
575
+ suffix = "_table_workspace"
576
+ scope = ws_key[:-len(suffix)] if ws_key.endswith(suffix) else key
577
+ if scope in cache:
578
+ return cache[scope]
579
+ try:
580
+ bucket = session.runtime.get(cohort_mod.key_for(scope)) or {}
581
+ mine = bucket.get(session.uname) or {}
582
+ cache[scope] = {str(cid): len(c.get("members") or [])
583
+ for cid, c in mine.items() if isinstance(c, dict)}
584
+ except Exception:
585
+ cache[scope] = {}
586
+ return cache[scope]
587
+
588
+
589
+ def _count_note(cfg, cohorts):
590
+ """WHY this view's size is not free, as one sentence naming a cause AND a next step.
591
+
592
+ β›” R6's SECOND SENTENCE IS THE SPEC HERE, and a bare dash is the thing it forbids. D-253 was
593
+ booked once already for a limit whose explanation was hover-only, so this has to be a real
594
+ sentence a client can put in a `title` AND an `aria-label`.
595
+ ⚠ THREE DIFFERENT CAUSES, not one generic apology. A view locked to somebody else's list, a
596
+ filtered view and a whole-table view are three different facts, and only one of them is
597
+ something the reader can act on.
598
+ """
599
+ cfg = cfg if isinstance(cfg, dict) else {}
600
+ lock = str(cfg.get("cohortLock") or "").strip()
601
+ if lock:
602
+ return ("This view is locked to a list owned by somebody else, so its size is not "
603
+ "something your account can read. Ask whoever shared it.")
604
+ if cfg.get("filters"):
605
+ return ("Counting a filtered view means reading every record, which this page does not "
606
+ "do. Open the database and the number is beside the view.")
607
+ return ("This view covers the whole database, so its size is the record count. Open the "
608
+ "database to see it.")
609
+
610
+
611
+ @router.get("/starred/counts")
612
+ def starred_counts(session: Session = Depends(require_session)):
613
+ """C3 β€” `{"counts": {vid: int}, "notes": {vid: str}, "degraded": [key]}`.
614
+
615
+ β›” CALLED AFTER THE PAGE PAINTS, NEVER FROM `/nav` (R7). `/nav` is a rows-free projection whose
616
+ budget was the subject of D-175, D-185, D-288 and D-289; W35-T43 takes the last of that off it,
617
+ and putting a counting pass back on the nav's path would undo the whole point.
618
+
619
+ ⚠ THE COUNTER IS `routes_nav._view_record_count`, CALLED β€” not a second implementation. It is the
620
+ one function that knows which shapes are free (a cohort lock resolves to a stored member list; a
621
+ curated `memberPids` carries its own length) and returns None for everything else. A second
622
+ counter here would be free to disagree with the badge inside the database.
623
+ """
624
+ rows, unread = _scan_starred_views(session)
625
+ cache, counts, notes = {}, {}, {}
626
+ for row in rows:
627
+ cfg, key, vid = row["config"], row["database"], row["id"]
628
+ sizes = _cohort_sizes(session, key, cache)
629
+ n = routes_nav._view_record_count(cfg, sizes)
630
+ # β›” EXACTLY ONE OF THE TWO MAPS (C3). The `if/else` is what makes that structural rather
631
+ # than a rule somebody has to remember: there is no path that writes both and none that
632
+ # writes neither.
633
+ if isinstance(n, int):
634
+ counts[vid] = n
635
+ else:
636
+ notes[vid] = _count_note(cfg, sizes)
637
+ return {"counts": counts, "notes": notes, "degraded": unread}
638
+
639
+
640
+ # ══════════════════════════════════════════════════════════════════ RECORD STARS (R5/R6, C5)
641
+ #
642
+ # ⭐ R5: *"Records get their own star, feeding an undeletable 'Starred records' view beside
643
+ # 'All records' on every database."* R6: *"Record stars are PER USER, in the same per-username
644
+ # stratum views already live in."*
645
+ #
646
+ # β›”β›” THE ID LIST IS ITS OWN KEY IN THAT STRATUM AND IT DOES NOT TRAVEL THROUGH THE VIEW DOOR.
647
+ # D-170, measured: on a read-through grid a VIEW WRITE answers **409 `window_required`** β€”
648
+ # `/grid/events` -> `routes_tables.ut_write_ctx` -> `scoped_pids` -> `scoped_pool` ->
649
+ # `routes_odoo_tables.whole_pool` raises `TooBigToMaterialise`, because the shared read/write
650
+ # assembly insists on the pid set. `ut_odoo_order_lines` (255,286 rows) and `ut_odoo_gl_lines`
651
+ # (971,034) serve rows through a window correctly and refuse every view-config write. A record star
652
+ # stored inside a view record would therefore be IMPOSSIBLE on exactly the two biggest databases in
653
+ # the tenant. This writes one key in the workspace bucket and touches no pid set, so it answers 200
654
+ # there.
655
+ #
656
+ # ⚠ AND FOR THE SAME REASON IT DOES NOT VALIDATE THE ID AGAINST THE ROW SET. Checking that a pid
657
+ # exists means materialising the pool, which is the 409 again. `/nav/opened` takes the identical
658
+ # posture and says why: the WRITE is cheap and unvalidated, the READ is what prunes β€” a client
659
+ # intersects `memberPids` with the rows it can see, exactly as a cohort already does
660
+ # (`aios_grid.workspace_wire` drops absent members and reports `missing`).
661
+
662
+ #: The per-user key inside `<database>_table_workspace`. BESIDE `views`, never inside one (R6).
663
+ _RECORDS_KEY = "starredRecords"
664
+
665
+ #: The pinned id of the projected view. ⚠ C's rail pins this too (W35-T30) β€” it is a WIRE constant,
666
+ #: so it is exported rather than spelled twice.
667
+ STARRED_VIEW_ID = "starred-records"
668
+ STARRED_VIEW_NAME = "Starred records"
669
+
670
+ #: A bound per (user, database). Reported on overflow, never silently enforced.
671
+ MAX_STARRED_RECORDS = 500
672
+
673
+
674
+ def _record_ids(doc, uname):
675
+ """This caller's starred record ids on ONE database, validated on the way out.
676
+
677
+ β›” INTS, AND A NON-DIGIT ID IS NOT STORED. Every record identity in this product is a digit
678
+ string β€” a pool `pid` is an int and a `ut_*` row key is `str(record_id).isdigit()` β€” and the
679
+ CLIENT's view engine reads `memberPids` as numbers (a cohort's members are ints). A mixed list
680
+ would make the projection match nothing on the rows it was built for, silently.
681
+ """
682
+ raw = (doc.get(uname) or {}).get(_RECORDS_KEY) if isinstance(doc.get(uname), dict) else None
683
+ if not isinstance(raw, list):
684
+ return []
685
+ out, seen = [], set()
686
+ for item in raw[:MAX_STARRED_RECORDS]:
687
+ try:
688
+ pid = int(item)
689
+ except (TypeError, ValueError):
690
+ continue
691
+ if pid not in seen:
692
+ seen.add(pid)
693
+ out.append(pid)
694
+ return out
695
+
696
+
697
+ def records_view(ids):
698
+ """The **"Starred records"** view, as a SavedView β€” or None when nothing is starred.
699
+
700
+ ⭐ THE SERVER HANDS C THE WHOLE OBJECT (C5) rather than a bare id list, so the rail cannot grow
701
+ a second idea of what this view is. `kind: 'system'` and `locked: True` are what make it
702
+ undeletable, the same two facts that protect `all-customers`.
703
+
704
+ β›” `memberPids` IS THE WHOLE MECHANISM: it makes this a CURATED ROW SET, which is the one shape
705
+ `routes_nav._view_record_count` can count for FREE β€” no rows read, no filter run. That is why C3
706
+ can put a real number beside it and why R7's counts endpoint costs nothing here.
707
+ ⚠ None WHEN EMPTY, not an empty view: a view listing zero records under a name promising some is
708
+ the shape D-229 was booked for. C shows the row only when there is something in it.
709
+ """
710
+ pids = [int(p) for p in (ids or [])]
711
+ if not pids:
712
+ return None
713
+ return {
714
+ "id": STARRED_VIEW_ID,
715
+ "name": STARRED_VIEW_NAME,
716
+ "kind": "system",
717
+ "locked": True,
718
+ "note": "The records you starred on this database. Only you can see this list.",
719
+ "config": {"filters": [], "filterConj": "and", "sorts": [], "groupBy": None,
720
+ "colorBy": None, "rowHeightMode": "short", "order": [], "visible": [],
721
+ "widths": {}, "memberPids": pids},
722
+ }
723
+
724
+
725
+ def _database_or_refuse(session, key):
726
+ """The workspace bucket for a database this caller may OPEN, or a refusal.
727
+
728
+ ⚠ THE SPLIT IS `routes_nav.nav_schema`'s, COPIED IN SHAPE RATHER THAN INVENTED: a `ut_*`
729
+ database is walled by `user_tables.may_open` (its creator or an admin or a grantee), and a
730
+ built-in module by `session.require`, which would 403 every `ut_` key because a user table is
731
+ deliberately not a module.
732
+ """
733
+ import core.view_templates as view_templates
734
+ ws_key = view_templates.workspace_key(key)
735
+ if not ws_key:
736
+ raise err(404, "unknown_database", "that database is not available")
737
+ if key.startswith("ut_"):
738
+ import core.user_tables as user_tables
739
+ if not user_tables.may_open(key, session.uname, session.admin, st=session.runtime):
740
+ raise err(403, "forbidden", "that database belongs to another user")
741
+ else:
742
+ session.require(key)
743
+ return ws_key
744
+
745
+
746
+ def _records_payload(session, key, ws_key):
747
+ try:
748
+ doc = session.runtime.get_projection(ws_key, drop=("overlays", "fields")) or {}
749
+ except Exception:
750
+ doc = {}
751
+ ids = _record_ids(doc, session.uname)
752
+ return {"database": key, "ids": ids, "count": len(ids), "view": records_view(ids)}
753
+
754
+
755
+ @router.get("/starred/records")
756
+ def starred_records(database: str = "", session: Session = Depends(require_session)):
757
+ """C5 β€” the record ids THIS caller starred on ONE database, plus the view to render.
758
+
759
+ ⚠ ONE DATABASE PER CALL, deliberately. Answering "every database" would be the N-bucket scan
760
+ the view half already pays for, on a route a grid calls every time it opens.
761
+ """
762
+ key = _clean_id(database)
763
+ if not key:
764
+ raise err(400, "bad_request", "no database was named")
765
+ return _records_payload(session, key, _database_or_refuse(session, key))
766
+
767
+
768
+ @router.post("/starred/records")
769
+ def set_starred_record(body: dict = Body(default=None),
770
+ session: Session = Depends(require_session)):
771
+ """C5 β€” star or unstar ONE record. Body `{"database": key, "id": str, "on": bool}`.
772
+
773
+ β›” PER USER, IN THE PER-USERNAME STRATUM, AND IT MUST NEVER FALL BACK TO A TENANT-WIDE LIST
774
+ (R6). One person's stars becoming everyone's is the failure mode a shared column has here β€” a
775
+ custom field on these grids is written through a per-username stratum anyway, so a "shared"
776
+ answer would be a deliberate widening, not a shortcut.
777
+ """
778
+ body = body if isinstance(body, dict) else {}
779
+ key = _clean_id(body.get("database"))
780
+ raw_id = _clean_id(body.get("id"))
781
+ if not key:
782
+ raise err(400, "bad_request", "no database was named")
783
+ if not raw_id.isdigit():
784
+ # Named rather than coerced: see `_record_ids` on why a non-numeric id cannot be stored.
785
+ raise err(400, "bad_request", "a record id is a number")
786
+ if "on" not in body or not isinstance(body.get("on"), bool):
787
+ raise err(400, "bad_request", "`on` must be true or false")
788
+ pid, on = int(raw_id), bool(body["on"])
789
+ ws_key = _database_or_refuse(session, key)
790
+ if not session.runtime.available():
791
+ raise err(503, "store_unavailable",
792
+ "the tenant store is unavailable. Nothing was saved.")
793
+ current = _records_payload(session, key, ws_key)["ids"]
794
+ if on and pid not in current and len(current) >= MAX_STARRED_RECORDS:
795
+ raise err(400, "starred_full",
796
+ f"you have starred the maximum of {MAX_STARRED_RECORDS} records on this "
797
+ f"database. Unstar one to make room")
798
+ uname = session.uname
799
+
800
+ def _up(data):
801
+ data = data if isinstance(data, dict) else {}
802
+ block = data.get(uname)
803
+ block = dict(block) if isinstance(block, dict) else {}
804
+ ids = _record_ids({uname: block}, uname)
805
+ if on:
806
+ if pid not in ids:
807
+ ids.append(pid)
808
+ else:
809
+ ids = [i for i in ids if i != pid]
810
+ if ids:
811
+ block[_RECORDS_KEY] = ids
812
+ else:
813
+ block.pop(_RECORDS_KEY, None)
814
+ # ⚠ THE STRATUM IS KEPT WHEN IT STILL HOLDS ANYTHING ELSE. Popping a username whose views
815
+ # and overlays live in the same block would delete somebody's whole workspace to clear one
816
+ # star β€” which is why this writes `block` back rather than replacing it.
817
+ if block:
818
+ data[uname] = block
819
+ else:
820
+ data.pop(uname, None)
821
+ return data
822
+
823
+ try:
824
+ session.runtime.update(ws_key, _up)
825
+ except Exception:
826
+ raise err(503, "store_unavailable",
827
+ "the star was not saved: the store refused the write.")
828
+ return _records_payload(session, key, ws_key)
api/routes_statements.py CHANGED
@@ -31,6 +31,10 @@ from fastapi import APIRouter, Body, Depends
31
  from deps import Session, err
32
  from routes_admin import admin_gate
33
 
 
 
 
 
34
  router = APIRouter(prefix="/api/v1")
35
 
36
  #: Cache the Odoo follow-up pull briefly. The Streamlit page used `@st.cache_data(ttl=1800)`; the
@@ -48,8 +52,17 @@ def _royal_only(session: Session) -> Session:
48
  """β›” See the module docstring, guardrail 3. The send client is env-credentialed, so it is
49
  tenant #0's and only tenant #0's. Refuse for anyone else rather than send as the wrong company.
50
 
51
- Keyed on the runtime, never on a request field: a tenant is a property of the SESSION."""
52
- if getattr(session.runtime, "key", None) != "royal-imports":
 
 
 
 
 
 
 
 
 
53
  raise err(404, "not_found", "statements are not configured for this workspace")
54
  return session
55
 
@@ -91,7 +104,9 @@ def statements(refresh: int = 0, session: Session = Depends(_gate)):
91
  "replyTo": cs.REPLY_TO, "company": cs.COMPANY},
92
  "templates": {"subject": cs.DEFAULT_SUBJECT, "intro": cs.DEFAULT_INTRO,
93
  "footer": cs.DEFAULT_FOOTER},
94
- "tiers": ["A-Urgent", "B-Active", "C-Light", "Monitor"],
 
 
95
  }
96
 
97
 
@@ -166,3 +181,89 @@ def send(body: dict = Body(default=None), session: Session = Depends(_gate)):
166
  failed.append({"customer": name, "error": str(e)[:200]})
167
  return {"sent": sent, "failed": failed, "skipped": skipped,
168
  "safeMode": bool(cs.SAFE_MODE), "test": bool(override)}
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
31
  from deps import Session, err
32
  from routes_admin import admin_gate
33
 
34
+ # W35-T35 (C8): the tenant predicate, held once. See `_royal_only` for why the import runs this
35
+ # way round. `main.py` already imports both modules, so this adds no load.
36
+ import automation_engine as engine
37
+
38
  router = APIRouter(prefix="/api/v1")
39
 
40
  #: Cache the Odoo follow-up pull briefly. The Streamlit page used `@st.cache_data(ttl=1800)`; the
 
52
  """β›” See the module docstring, guardrail 3. The send client is env-credentialed, so it is
53
  tenant #0's and only tenant #0's. Refuse for anyone else rather than send as the wrong company.
54
 
55
+ Keyed on the runtime, never on a request field: a tenant is a property of the SESSION.
56
+
57
+ ⭐⭐ WAVE 35 Β· T35 / CONTRACT C8 β€” THE TEST ITSELF NOW LIVES IN ONE PLACE. Statements became an
58
+ agent step this wave, so the automation engine has to answer the same question at two more
59
+ doors (which tenants see the action, which tenants may store it). C8's words are "the predicate
60
+ is imported, never re-expressed", and this is the direction that import can run: the engine
61
+ imports no route module and no FastAPI, so reaching `_royal_only` FROM it would drag `deps` and
62
+ `routes_admin` into a route-free module. The engine holds the boolean; this holds the refusal.
63
+ ⚠ WHAT STAYS HERE IS THE HTTP SHAPE, and that is deliberate β€” a 404 rather than a 403, so the
64
+ surface is invisible rather than forbidden. The engine must not know about status codes."""
65
+ if not engine.is_statement_tenant(session.runtime):
66
  raise err(404, "not_found", "statements are not configured for this workspace")
67
  return session
68
 
 
104
  "replyTo": cs.REPLY_TO, "company": cs.COMPANY},
105
  "templates": {"subject": cs.DEFAULT_SUBJECT, "intro": cs.DEFAULT_INTRO,
106
  "footer": cs.DEFAULT_FOOTER},
107
+ # W35-T35: ONE literal, read from the engine, so the sender's worklist and a statements
108
+ # agent's tier filter cannot come to mean different things.
109
+ "tiers": list(engine.STATEMENT_TIERS),
110
  }
111
 
112
 
 
181
  failed.append({"customer": name, "error": str(e)[:200]})
182
  return {"sent": sent, "failed": failed, "skipped": skipped,
183
  "safeMode": bool(cs.SAFE_MODE), "test": bool(override)}
184
+
185
+
186
+ # ---------------------------------------------------------------------------------------------
187
+ # WAVE 35 Β· T36 / CONTRACT C8 / OWNER RULING R10 β€” THE AGENT'S PARKED BATCH.
188
+ #
189
+ # β›”β›” THE AGENT ASSEMBLES; A PERSON CLICKS SEND. `automation_engine.run_statements` renders a batch
190
+ # and parks it on the automation definition, and it imports no mail path at all. THIS is the only
191
+ # door that releases one, and it is deliberately built out of the parts already here:
192
+ # Β· `_gate` β€” `admin_gate` THEN `_royal_only`, byte-for-byte the dependency the other three
193
+ # endpoints use. Not a copy of the checks: the same object.
194
+ # Β· `cs.queue_statement` β€” the same call `send()` above makes, so SAFE_MODE's allow-list refusal
195
+ # happens in the DATA LAYER, where no route, payload or UI can reach around it.
196
+ # β‡’ Nothing about the guardrails is re-expressed here, which is what makes "unchanged" checkable.
197
+ #
198
+ # ⚠ WHY THE BATCH IS RE-RENDERED FROM THE LIVE WORKLIST RATHER THAN SENT AS STORED. The parked
199
+ # `html` is what a person APPROVED and is what the review screen shows; but a balance can move
200
+ # between the parking and the click, and mailing a figure we know to be stale is worse than mailing
201
+ # a fresh one. So the parked batch decides WHO and WITH WHAT WORDS, and the live row decides the
202
+ # NUMBERS β€” the same split `preview` already makes. A customer who has left the worklist entirely
203
+ # (they paid) is reported as skipped rather than invoiced.
204
+
205
+ @router.get("/admin/statements/agent/{auto_id}")
206
+ def agent_batch(auto_id: str, session: Session = Depends(_gate)):
207
+ """What is parked and waiting for a click, for ONE agent. `null` when nothing is."""
208
+ defn = (engine.all_definitions(session.runtime) or {}).get(str(auto_id)) or {}
209
+ pending = defn.get("pendingStatements")
210
+ if not isinstance(pending, dict):
211
+ return {"pending": None}
212
+ # ⚠ The rendered `html` is NOT returned in the list β€” a 200-statement batch of rendered mail is
213
+ # megabytes, and the review screen needs the count and the names to decide. `preview` already
214
+ # renders ONE on demand.
215
+ return {"pending": {
216
+ "ts": pending.get("ts"), "count": int(pending.get("count") or 0),
217
+ "notes": list(pending.get("notes") or []),
218
+ "items": [{k: v for k, v in it.items() if k != "html"}
219
+ for it in (pending.get("items") or [])],
220
+ "safeMode": bool(_cs().SAFE_MODE)}}
221
+
222
+
223
+ @router.post("/admin/statements/agent/{auto_id}/send")
224
+ def agent_send(auto_id: str, body: dict = Body(default=None),
225
+ session: Session = Depends(_gate)):
226
+ """Release a parked batch. THE CLICK R10 REQUIRES β€” nothing else in the system calls this.
227
+
228
+ β›” IT REFUSES AN EMPTY OR MISSING BATCH rather than answering 200 with nothing sent: a send
229
+ that reports success and mails nobody is the `view_upsert` failure mode (200 OK, zero writes)
230
+ arriving on the one route in this system that talks to a mail server.
231
+ """
232
+ cs = _cs()
233
+ defn = (engine.all_definitions(session.runtime) or {}).get(str(auto_id)) or {}
234
+ pending = defn.get("pendingStatements")
235
+ items = list((pending or {}).get("items") or []) if isinstance(pending, dict) else []
236
+ if not items:
237
+ raise err(404, "no_batch", "there is nothing parked for this agent to send")
238
+ # ⚠ An explicit subset is allowed (a person unticking a customer on the review screen) but it
239
+ # may only NARROW the parked batch. A name that was never parked cannot be introduced by the
240
+ # payload, or the review stops being what authorised the send.
241
+ want = {str(n) for n in (body or {}).get("customers") or []}
242
+ if want:
243
+ items = [it for it in items if str(it.get("customer")) in want]
244
+ if not items:
245
+ raise err(400, "bad_request", "none of those customers are in the parked batch")
246
+ rows, _ = _rows()
247
+ sent, failed, skipped = [], [], []
248
+ for it in items:
249
+ name = str(it.get("customer") or "")
250
+ row = _find(rows, name)
251
+ if row is None:
252
+ skipped.append({"customer": name,
253
+ "reason": "no longer on the collection list, so nothing is owed"})
254
+ continue
255
+ try:
256
+ # β›” THE SAME DATA-LAYER CALL `send()` MAKES. SafeModeBlocked is raised INSIDE
257
+ # `queue_statement`, so the guardrail cannot be argued with from here.
258
+ mid = cs.queue_statement(cs.Odoo(), row,
259
+ str(it.get("subject") or "") or cs.DEFAULT_SUBJECT,
260
+ cs.DEFAULT_INTRO, cs.DEFAULT_FOOTER)
261
+ sent.append({"customer": name, "to": row.get("Email"), "mailId": mid})
262
+ except Exception as e: # noqa: BLE001
263
+ failed.append({"customer": name, "error": str(e)[:200]})
264
+ # ⚠ CLEARED ONLY WHEN NOTHING IS LEFT TO RETRY. A batch dropped while some of it failed would
265
+ # lose the list of who still needs a statement, and nobody would know to look.
266
+ if sent and not failed:
267
+ engine.clear_statements(session.runtime, auto_id)
268
+ return {"sent": sent, "failed": failed, "skipped": skipped,
269
+ "safeMode": bool(cs.SAFE_MODE), "cleared": bool(sent and not failed)}
api/routes_tables.py CHANGED
@@ -105,7 +105,7 @@ def _refresh_relations(session):
105
  name=f"rel-refresh:{tenant or 'default'}").start()
106
 
107
 
108
- def _defn_or_refuse(session, table_key, st=None):
109
  """The per-table wall: 404 for a key that does not exist, 403 for one this session may not
110
  open. 404-before-403 leaks nothing useful β€” ut keys are guessable slugs, and 'exists but
111
  not yours' is exactly what may_open is for.
@@ -113,11 +113,30 @@ def _defn_or_refuse(session, table_key, st=None):
113
  ⭐ `st` LETS A CALLER LEND THE SNAPSHOT IT IS ALREADY HOLDING (W31 QA), the same `lend()` the
114
  nav and `/tables` use since W31-T10/T12. The WALL is untouched β€” the same `may_open`, asked
115
  about the same table β€” it is simply not re-reading a 28.5 MB document to ask it.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
116
  """
117
  ut = _ut()
118
  # ⭐⭐ W31 QA β€” THE SWEEP, AND IT IS ONE LINE BECAUSE IT IS DONE HERE RATHER THAN PER ROUTE.
119
  # Owner: *"this is just one case, I need you to check and apply the fix everywhere too."*
120
- # This guard has FOURTEEN call sites and cost TWO whole-document deep copies at every one
 
121
  # (`get`, then `may_open`) β€” on tenant #0 that is 2 x 28.5 MB (D-185) to answer two questions
122
  # about ONE table. Lending here fixes every caller at once, including the eight routes the
123
  # sweep enumerated (`PATCH /shared/{pid}` Β· `DELETE /shared/fields/{k}` Β· `GET|POST /rows` Β·
@@ -128,7 +147,8 @@ def _defn_or_refuse(session, table_key, st=None):
128
  # would hand that stale snapshot to `patch_row`'s and `add_row`'s post-write read-back, which
129
  # is [[refetch-eats-its-own-write]] with the sign flipped β€” a write that reports the value it
130
  # replaced. The remaining in-route reads are deliberately untouched and booked instead.
131
- st = st if st is not None else ut.lend(session.runtime)
 
132
  defn = ut.get(table_key, st=st)
133
  if not defn:
134
  raise err(404, "unknown_table", "that database does not exist")
@@ -145,7 +165,7 @@ def _records_or_refuse(session, table_key, st=None):
145
  defn = _defn_or_refuse(session, table_key, st=st)
146
  if not _ut().records_mutable(table_key, st=st):
147
  raise err(403, "records_read_only",
148
- "records in this automation-owned database are read-only β€” add Instagram "
149
  "handles in a Profile database and let enrichment populate this database")
150
  return defn
151
 
@@ -234,15 +254,21 @@ _PID_SCOPE_LIMIT = {
234
  }
235
 
236
 
237
- def scoped_pool(session: Session, table_key: str):
238
  """`(pids, rows_src, fields_base, defn)` β€” THE USER-TABLE WALL, on its own.
239
 
 
 
 
 
 
 
240
  `routes_products.scoped_pool`'s sibling, extracted for the same reason (wave 19, item 12): a
241
  caller that only needs "which rows of this database may this session touch" β€” record comments
242
  β€” must ask through `_defn_or_refuse` (404/403, the whole wall) rather than growing a second
243
  idea of what a user table's pool is.
244
  """
245
- defn = _defn_or_refuse(session, table_key)
246
  fields_base = [dict(f) for f in (defn.get("fields") or [])]
247
  field_keys = {f["key"] for f in fields_base}
248
  # ⭐⭐ W30-T31 / D-87 β€” A READ-THROUGH DATABASE HAS NO ROWS HERE, SO THEY COME FROM THE MIRROR.
@@ -350,7 +376,7 @@ def delete_shared_field(table_key: str, field_key: str,
350
  owner = str(defn.get("createdBy") or "")
351
  if not session.admin and owner != session.uname:
352
  raise err(403, "forbidden",
353
- f"a tenant-wide column can be removed by its creator or an admin β€” this one "
354
  f"was added by {owner or 'somebody else'}, and dropping it would delete the "
355
  f"value for every account")
356
  dropped = shared_overlay.drop_field(table_key, str(field_key), st=session.runtime)
@@ -375,8 +401,14 @@ def scoped_pids(session: Session, table_key: str, limits=None):
375
  middle. The row WALL is unchanged β€” a narrower or wider set here would be a permission
376
  change, and this is a performance change.
377
 
378
- ⚠ It does NOT make the write cheap on its own: `_defn_or_refuse` still costs two whole-document
379
- deep copies (`ut.get` then `may_open`), which is D-87 and W30-T31. This removes the row pass.
 
 
 
 
 
 
380
 
381
  ⭐⭐ W31-T20 / D-174 β€” `limits` IS AN OUT-PARAMETER, AND IT IS THE POINT OF THE TICKET. Pass a
382
  list and this function APPENDS R6's sentence to it when the pid set could not be resolved (a
@@ -386,7 +418,20 @@ def scoped_pids(session: Session, table_key: str, limits=None):
386
  write is expected to carry the sentence through. Omitting the list means the caller accepts an
387
  unannounced empty scope, which is only ever right for a caller that does not use the pids.
388
  """
389
- defn = _defn_or_refuse(session, table_key)
 
 
 
 
 
 
 
 
 
 
 
 
 
390
  fields_base = [dict(f) for f in (defn.get("fields") or [])]
391
  # ⚠ W30-T31: on a read-through database the stored `rows` is `{}` by construction, so the
392
  # comprehension below would answer an EMPTY pid set β€” and the promise this function makes is
@@ -407,8 +452,25 @@ def scoped_pids(session: Session, table_key: str, limits=None):
407
  return frozenset(), fields_base, defn
408
  except RuntimeError as e:
409
  raise err(503, "store_not_ready", str(e))
410
- pids = frozenset(int(rid) for rid in (defn.get("rows") or {}) if str(rid).isdigit())
411
- return pids, fields_base, defn
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
412
 
413
 
414
  def ut_write_ctx(session: Session, table_key: str):
@@ -441,7 +503,7 @@ def ut_write_ctx(session: Session, table_key: str):
441
 
442
 
443
  def ut_assembly(session: Session, table_key: str, storage_key: str = "",
444
- consume_corrections: bool = True, with_rows: bool = True):
445
  """The user-table mirror of `grid_assembly` / `product_assembly` β€” SAME g-dict keys, so
446
  `/workspace` and the events route consume any of the three interchangeably.
447
 
@@ -470,7 +532,9 @@ def ut_assembly(session: Session, table_key: str, storage_key: str = "",
470
 
471
  limits = []
472
  if with_rows:
473
- pids, rows_src, fields_base, defn = scoped_pool(session, table_key)
 
 
474
  else:
475
  pids, fields_base, defn = scoped_pids(session, table_key, limits=limits)
476
  rows_src = []
@@ -671,17 +735,17 @@ def create_table(body: dict = Body(default=None),
671
  raise err(400, "bad_label", "give the database a name")
672
  if not session.runtime.available():
673
  raise err(503, "store_unavailable",
674
- "the tenant store is unavailable β€” nothing was created")
675
  source = body.get("source")
676
  try:
677
  key = ut.create(label, session.uname, fields=body.get("fields"),
678
  source=source, st=session.runtime)
679
  except Exception:
680
  raise err(503, "store_unavailable",
681
- "the tenant store refused the write β€” nothing was created")
682
  if not key:
683
  raise err(400, "refused",
684
- f"could not create it β€” the name may be empty or this tenant already has "
685
  f"{ut.MAX_TABLES} databases")
686
  return {"key": key}
687
 
@@ -713,8 +777,11 @@ def table_footprint(table_key: str, session: Session = Depends(require_session))
713
 
714
  Counts drill to the SAME buckets `user_tables.delete` cleans; a dialog listing categories
715
  without numbers would break [[no-unverifiable-aggregates]] at the scariest moment. Walled
716
- like the delete itself: only someone who could delete may case the joint."""
717
- defn = _defn_or_refuse(session, table_key)
 
 
 
718
  if not (session.admin or defn.get("createdBy") == session.uname):
719
  raise err(403, "forbidden", "only the database's creator or an admin can delete it")
720
  s = session.runtime
@@ -737,7 +804,19 @@ def table_footprint(table_key: str, session: Session = Depends(require_session))
737
  auto.append({"id": str(aid), "name": d.get("name") or str(aid)})
738
  except Exception:
739
  pass
740
- return {"rows": len(defn.get("rows") or {}), "fields": fields, "views": len(views),
 
 
 
 
 
 
 
 
 
 
 
 
741
  "sharedUsers": len(g.get("entries") or []),
742
  "automations": sorted(auto, key=lambda a: a["name"].lower())}
743
 
@@ -766,7 +845,7 @@ def delete_table(table_key: str, session: Session = Depends(require_session)):
766
  try:
767
  _ut().delete(table_key, st=session.runtime)
768
  except Exception:
769
- raise err(503, "store_unavailable", "the delete did not land β€” try again")
770
  return {"ok": True}
771
 
772
 
@@ -856,7 +935,17 @@ def table_rows(table_key: str, session: Session = Depends(require_session)):
856
  import aios_grid
857
 
858
  _ig_forward(session)
859
- g = ut_assembly(session, table_key)
 
 
 
 
 
 
 
 
 
 
860
  # β›” THE OVERLAY WAS NEVER ACTUALLY MERGED, and the docstring above has described the merge
861
  # this line does not perform since the route was written (owner item 3, 2026-08-09:
862
  # *"Using the swipe, fast doesn't register the CHANGE. I went back and it all got reseted"*).
@@ -904,13 +993,16 @@ def table_rows(table_key: str, session: Session = Depends(require_session)):
904
  # so a tenant using `json` for a short config sees no change at all.
905
  rows = aios_grid.rows_from_pool(
906
  g["rows_src"], g["fields"], _thin_json(g["fields"], merged, table_key), derived=g["derived"])
 
 
 
907
  # ⭐ R6's SECOND SENTENCE, ON THE WIRE (W30-T29). *"if there is lag or it can't be done, you
908
  # need to explicitly tell me why and recommend a fix."* A ceiling that still applies to this
909
  # database says so here, with its cause and the recommendation, rather than waiting to be
910
  # discovered as a refused paste. `None` for a connected source, and an EMPTY LIST is the
911
  # honest answer for a table nothing limits β€” never an absent key, which a client cannot tell
912
  # apart from an older server.
913
- _report = _ut().limit_report(table_key, st=session.runtime)
914
  # ⭐ C4 / D-138 β€” THE DOCUMENTS PRODUCER. Absent since EXIT-6 deleted `app.py`, which was the
915
  # only thing that ever set this key; the write door never stopped working and every client
916
  # half is complete, but all six `onDoc*` handlers read `payload?.docs ? … : undefined`, so a
@@ -924,7 +1016,49 @@ def table_rows(table_key: str, session: Session = Depends(require_session)):
924
  "identity": {"pid": "pid"},
925
  "scope": {"table": table_key, "rowCount": len(rows)},
926
  "limits": [_report] if _report else [],
927
- "recordsMutable": _ut().records_mutable(table_key, st=session.runtime)}
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
928
 
929
 
930
  @router.get("/tables/{table_key}/rows/{pid}/fields/{fkey}")
@@ -973,7 +1107,7 @@ def add_row(table_key: str, body: dict = Body(default=None),
973
  rid = ut.add_row(table_key, values, session.uname, st=session.runtime,
974
  rid=(body or {}).get("rid"))
975
  except Exception:
976
- raise err(503, "store_unavailable", "the row was not saved β€” the store refused")
977
  if rid is None:
978
  # C3 (wave 25): `add_row` also refuses a profile cell that is not a handle, so the cap
979
  # sentence alone would misdirect β€” the reader would go and count rows. Ask the same
@@ -983,11 +1117,11 @@ def add_row(table_key: str, body: dict = Body(default=None),
983
  _h, ok = ut.normalize_profile(values[pf["key"]], pf["profile"].get("source"))
984
  if not ok:
985
  raise err(400, "refused",
986
- f"{str(values[pf['key']])[:80]!r} is not an Instagram profile β€” "
987
  f"{pf.get('label') or pf['key']!r} takes a handle (@name) or a "
988
  f"profile link (instagram.com/name)")
989
  raise err(400, "refused",
990
- f"row refused β€” the table may be at its {ut.MAX_ROWS}-row cap")
991
  _refresh_relations(session)
992
  return {"rid": rid, "pid": int(rid)}
993
 
@@ -1041,14 +1175,14 @@ def import_rows(table_key: str, body: dict = Body(default=None),
1041
  for key, value in row.items():
1042
  why = ut.cell_type_refusal(by_key[key], value)
1043
  if why:
1044
- raise err(400, "bad_value", f"row {index + 1}: {why} β€” nothing was imported")
1045
  try:
1046
  made = ut.add_rows(table_key, rows_in, session.uname, st=session.runtime)
1047
  except Exception:
1048
- raise err(503, "store_unavailable", "nothing was imported β€” the store refused")
1049
  if made is None:
1050
  raise err(400, "refused",
1051
- f"nothing was imported β€” {len(rows_in)} rows would take this database past "
1052
  f"its {ut.MAX_ROWS}-row cap, or a profile column rejected a value")
1053
  _refresh_relations(session)
1054
  return {"imported": len(made), "pids": [int(r) for r in made]}
@@ -1068,12 +1202,18 @@ def import_rows(table_key: str, body: dict = Body(default=None),
1068
  # as the rename (item 6a): two places to look, and the surfaces disagreed silently.
1069
 
1070
  def _field_or_refuse(session, table_key, fkey=""):
1071
- """The schema wall. `_defn_or_refuse` first (404/403 on the table), then the per-field rule."""
1072
- defn = _defn_or_refuse(session, table_key)
 
 
 
 
 
 
1073
  ut = _ut()
1074
  if not ut.is_user_table(table_key, st=session.runtime):
1075
  raise err(400, "not_a_user_table",
1076
- "only a user-created database has an editable schema β€” a connected source "
1077
  "owns its own columns")
1078
  if fkey and not ut.may_edit_field(table_key, fkey, session.uname, session.admin,
1079
  st=session.runtime):
@@ -1126,12 +1266,41 @@ def add_field(table_key: str, body: dict = Body(default=None),
1126
  # field relational", which is the question that was always meant.
1127
  if field.get("type") in ("link", "rollup"):
1128
  _refresh_relations(session)
1129
- return {"field": field}
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1130
 
1131
 
1132
  def _refusal_sentence(ut, session, body, table_key="", fkey=""):
1133
  """Why was this column refused? The specific reason when we can name one, the general list
1134
  otherwise β€” never a specific-sounding guess."""
 
 
 
 
 
 
 
 
 
 
1135
  # ⭐ C3 (wave 25, R7): the one-profile-per-table refusal NAMES THE EXISTING COLUMN, which is
1136
  # what the contract asks for and what makes it actionable β€” "at most one" sends the reader
1137
  # hunting through a 40-column schema for a flag they cannot see from the header.
@@ -1143,11 +1312,11 @@ def _refusal_sentence(ut, session, body, table_key="", fkey=""):
1143
  # real mistake was the column type sends them to fix the wrong thing, which is the
1144
  # misdirection D-46 closed one door over.
1145
  if str((body or {}).get("type") or "text").strip().lower() != "text":
1146
- return ("a profile column is a flag on an ordinary TEXT column β€” it validates what "
1147
  "is typed into it, which it can only do for text")
1148
  existing = ut.profile_field(table_key, st=session.runtime) if table_key else None
1149
  if existing and existing.get("key") != str(fkey):
1150
- return (f"this database already has a profile column β€” "
1151
  f"{existing.get('label') or existing.get('key')!r}. A database has at most "
1152
  f"one, so the automation knows which handle to enrich; edit that column, or "
1153
  f"take the flag off it first")
@@ -1155,16 +1324,16 @@ def _refusal_sentence(ut, session, body, table_key="", fkey=""):
1155
  if isinstance(bag, dict):
1156
  flow = str(bag.get("flowId") or "").strip()
1157
  if not flow:
1158
- return ("an automation column has to name the automation that fills it β€” pick a "
1159
  "flow, or make this an ordinary column")
1160
  if not ut.flow_bound(bag, st=session.runtime):
1161
  return (f"this column names automation {flow!r}, which does not exist in this "
1162
- f"workspace β€” it may have been deleted; pick a flow that is still there")
1163
  kind = str((body or {}).get("type") or "").strip()
1164
  if kind and kind not in ut.UT_FIELD_TYPES:
1165
  return (f"{kind!r} is not a column type here (types: "
1166
  f"{', '.join(sorted(ut.UT_FIELD_TYPES))})")
1167
- return (f"the column was refused β€” check the name and type, or the table may be at its "
1168
  f"{ut.MAX_FIELDS}-column cap (types: {', '.join(sorted(ut.UT_FIELD_TYPES))})")
1169
 
1170
 
@@ -1187,7 +1356,7 @@ def patch_field(table_key: str, fkey: str, body: dict = Body(default=None),
1187
  try:
1188
  migrated = ut.rename_choice_values(table_key, fkey, renames, st=session.runtime)
1189
  except Exception:
1190
- raise err(503, "store_unavailable", "the rename did not land β€” try again")
1191
  # The per-user workspace strata and any view filter naming the old value are the OTHER
1192
  # half of C-RENAME and belong to `core.table_store`. Called only if it is there: an
1193
  # enumerator's mirror waits for its counterpart rather than guessing at its shape, and a
@@ -1214,7 +1383,85 @@ def patch_field(table_key: str, fkey: str, body: dict = Body(default=None),
1214
  out = {"field": field}
1215
  if migrated is not None:
1216
  out["migrated"] = migrated
1217
- return out
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1218
 
1219
 
1220
  @router.delete("/tables/{table_key}/fields/{fkey}")
@@ -1222,7 +1469,7 @@ def delete_field(table_key: str, fkey: str, session: Session = Depends(require_s
1222
  _field_or_refuse(session, table_key, fkey)
1223
  if not _ut().delete_field(table_key, fkey, st=session.runtime):
1224
  raise err(400, "refused",
1225
- "that column could not be removed β€” a database must keep at least one")
1226
  _refresh_relations(session)
1227
  return {"deleted": fkey}
1228
 
@@ -1247,7 +1494,7 @@ def delete_row(table_key: str, rid: str, session: Session = Depends(require_sess
1247
  try:
1248
  ok = _ut().delete_row(table_key, rid, st=lent)
1249
  except Exception:
1250
- raise err(503, "store_unavailable", "the delete did not land β€” try again")
1251
  if not ok:
1252
  raise err(400, "refused", "rows can only be deleted from user-created databases")
1253
  _refresh_relations(session)
@@ -1280,7 +1527,7 @@ def patch_row(table_key: str, pid: int, body: dict = Body(default=None),
1280
  "pid": pid, "updates": updates}, ctx)
1281
  except grid_events.StoreUnavailable:
1282
  raise err(503, "store_unavailable",
1283
- "the tenant store is unavailable β€” your change was not saved")
1284
  # ⚠ THE READ-BACK IS THE DEFINITION ROW, AND ON A `ut_` SCOPE THAT IS THE WHOLE OF IT.
1285
  #
1286
  # β›” CORRECTED, wave-29 T22 (owner item 2a): this note used to say "THE READ-BACK SPANS BOTH
@@ -1319,6 +1566,22 @@ def patch_row(table_key: str, pid: int, body: dict = Body(default=None),
1319
  return False
1320
 
1321
  refused = sorted(k for k in updates if not _took(k))
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1322
  out = {"pid": pid, "updates": accepted}
1323
  if refused:
1324
  out["refused"] = refused
 
105
  name=f"rel-refresh:{tenant or 'default'}").start()
106
 
107
 
108
+ def _defn_or_refuse(session, table_key, st=None, defs_only=False):
109
  """The per-table wall: 404 for a key that does not exist, 403 for one this session may not
110
  open. 404-before-403 leaks nothing useful β€” ut keys are guessable slugs, and 'exists but
111
  not yours' is exactly what may_open is for.
 
113
  ⭐ `st` LETS A CALLER LEND THE SNAPSHOT IT IS ALREADY HOLDING (W31 QA), the same `lend()` the
114
  nav and `/tables` use since W31-T10/T12. The WALL is untouched β€” the same `may_open`, asked
115
  about the same table β€” it is simply not re-reading a 28.5 MB document to ask it.
116
+
117
+ ⭐⭐ W33-T01 (D-213) β€” `defs_only=True` MAKES THAT READ A PROJECTION, AND IT IS OPT-IN PER CALL
118
+ SITE ON PURPOSE. This wall asks two questions ("does the key exist", "may this session open
119
+ it") and neither has ever read a row, but three of its ten callers go on to read `rows` OFF THE
120
+ DEFINITION THIS RETURNS β€” so a blanket swap would turn `scoped_pool`, `scoped_pids` and
121
+ `table_footprint` into `KeyError`s on every materialised table. The flag is therefore the
122
+ CALLER's claim about what it will do next, not a global setting; each opt-in below is annotated
123
+ with why it can make that claim ([[reuse-and-delete-are-hypotheses]]).
124
+
125
+ β›” **`defs_only` IS FOR READS.** The six write walls (`_records_or_refuse`, `patch_shared_cell`,
126
+ `delete_shared_field`, `patch_table`, `delete_table`, `import_rows`) deliberately do NOT pass
127
+ it: a projected snapshot must never reach a post-write read-back (contract C5), and the
128
+ pre-write lend note below is the same argument one layer down.
129
+
130
+ ⚠ **WHAT COMES BACK IS A `store._Projected`, WHICH IS THE EVIDENCE, NOT AN IMPLEMENTATION
131
+ DETAIL.** `all_defs` falls back to the whole read on ANY failure, so "the answer was right"
132
+ proves nothing about which read produced it. `core.store.is_projected(defn)` is how a gate
133
+ asserts the fast path was TAKEN.
134
  """
135
  ut = _ut()
136
  # ⭐⭐ W31 QA β€” THE SWEEP, AND IT IS ONE LINE BECAUSE IT IS DONE HERE RATHER THAN PER ROUTE.
137
  # Owner: *"this is just one case, I need you to check and apply the fix everywhere too."*
138
+ # This guard has TEN direct call sites (counted 2026-08-14; the note said FOURTEEN, which was
139
+ # the route count, not the caller count) and cost TWO whole-document deep copies at every one
140
  # (`get`, then `may_open`) β€” on tenant #0 that is 2 x 28.5 MB (D-185) to answer two questions
141
  # about ONE table. Lending here fixes every caller at once, including the eight routes the
142
  # sweep enumerated (`PATCH /shared/{pid}` Β· `DELETE /shared/fields/{k}` Β· `GET|POST /rows` Β·
 
147
  # would hand that stale snapshot to `patch_row`'s and `add_row`'s post-write read-back, which
148
  # is [[refetch-eats-its-own-write]] with the sign flipped β€” a write that reports the value it
149
  # replaced. The remaining in-route reads are deliberately untouched and booked instead.
150
+ if st is None:
151
+ st = ut.lend_defs(session.runtime) if defs_only else ut.lend(session.runtime)
152
  defn = ut.get(table_key, st=st)
153
  if not defn:
154
  raise err(404, "unknown_table", "that database does not exist")
 
165
  defn = _defn_or_refuse(session, table_key, st=st)
166
  if not _ut().records_mutable(table_key, st=st):
167
  raise err(403, "records_read_only",
168
+ "records in this automation-owned database are read-only. Add Instagram "
169
  "handles in a Profile database and let enrichment populate this database")
170
  return defn
171
 
 
254
  }
255
 
256
 
257
+ def scoped_pool(session: Session, table_key: str, st=None):
258
  """`(pids, rows_src, fields_base, defn)` β€” THE USER-TABLE WALL, on its own.
259
 
260
+ ⭐ W33-T03 (D-214) β€” `st` LETS A CALLER LEND THE DOCUMENT IT IS ALREADY HOLDING, exactly as
261
+ `_defn_or_refuse` has since W31 QA. It is passed straight through to that wall and nowhere
262
+ else, so the permission question is answered by the same code against the same document.
263
+ β›” READ CALLERS ONLY. A lend is a PRE-WRITE snapshot; handing one to a path that writes and then
264
+ reads back is [[refetch-eats-its-own-write]] with the sign flipped (contract C5).
265
+
266
  `routes_products.scoped_pool`'s sibling, extracted for the same reason (wave 19, item 12): a
267
  caller that only needs "which rows of this database may this session touch" β€” record comments
268
  β€” must ask through `_defn_or_refuse` (404/403, the whole wall) rather than growing a second
269
  idea of what a user table's pool is.
270
  """
271
+ defn = _defn_or_refuse(session, table_key, st=st)
272
  fields_base = [dict(f) for f in (defn.get("fields") or [])]
273
  field_keys = {f["key"] for f in fields_base}
274
  # ⭐⭐ W30-T31 / D-87 β€” A READ-THROUGH DATABASE HAS NO ROWS HERE, SO THEY COME FROM THE MIRROR.
 
376
  owner = str(defn.get("createdBy") or "")
377
  if not session.admin and owner != session.uname:
378
  raise err(403, "forbidden",
379
+ f"a tenant-wide column can be removed by its creator or an admin. This one "
380
  f"was added by {owner or 'somebody else'}, and dropping it would delete the "
381
  f"value for every account")
382
  dropped = shared_overlay.drop_field(table_key, str(field_key), st=session.runtime)
 
401
  middle. The row WALL is unchanged β€” a narrower or wider set here would be a permission
402
  change, and this is a performance change.
403
 
404
+ ⚠ It does NOT make the write cheap on its own: `_defn_or_refuse` still costs a whole-document
405
+ read, which is D-87 and W30-T31. This removes the row pass.
406
+ ⭐ CORRECTED 2026-08-14 (W33-T01): that sentence said **two** deep copies (`ut.get` then
407
+ `may_open`) and had been stale since W31 QA taught the wall to `lend()` β€” the two questions
408
+ have shared ONE read since `routes_tables.py`'s lend line. And as of this ticket the read is a
409
+ PROJECTION on the read-through arm, so the sentence is now true only of the materialised one.
410
+ Booked because a stale performance note is how a wave re-fixes something twice
411
+ ([[stale-baseline-unreadable-deltas]]).
412
 
413
  ⭐⭐ W31-T20 / D-174 β€” `limits` IS AN OUT-PARAMETER, AND IT IS THE POINT OF THE TICKET. Pass a
414
  list and this function APPENDS R6's sentence to it when the pid set could not be resolved (a
 
418
  write is expected to carry the sentence through. Omitting the list means the caller accepts an
419
  unannounced empty scope, which is only ever right for a caller that does not use the pids.
420
  """
421
+ # ⭐⭐ W33-T01 / D-213 β€” THE DATABASE-SWITCH PATH, AND IT STARTS ON A PROJECTION.
422
+ #
423
+ # `GET /workspace?scope=<key>` reaches here through `ut_assembly(with_rows=False)`
424
+ # (`routes_grid.py`'s ut_ branch), which is what a person is waiting for when they click a
425
+ # database in the nav flyout: 1.8-7.3 s live for a 3-6 KB payload, of which one whole-document
426
+ # read is ~703 ms warm and 20.6 s cold. This function reads `fields` and (below) `readThrough`
427
+ # off the definition β€” no row β€” so the WALL can be answered from the 0.1% projection.
428
+ #
429
+ # β›” IT IS THE TRAP ON THIS BOARD, SO IT IS SAID TWICE: this function is NAMED and DOCUMENTED
430
+ # as the rows-free twin of `scoped_pool` and the materialised arm below still reads `rows`.
431
+ # The opt-in is therefore CONDITIONAL, and the condition is `materialises`, which reads
432
+ # `readThrough` β€” a definition key, safe under the projection, and already lent the defn so it
433
+ # costs no read of its own.
434
+ defn = _defn_or_refuse(session, table_key, defs_only=True)
435
  fields_base = [dict(f) for f in (defn.get("fields") or [])]
436
  # ⚠ W30-T31: on a read-through database the stored `rows` is `{}` by construction, so the
437
  # comprehension below would answer an EMPTY pid set β€” and the promise this function makes is
 
452
  return frozenset(), fields_base, defn
453
  except RuntimeError as e:
454
  raise err(503, "store_not_ready", str(e))
455
+ # β›”β›” MATERIALISED: THE PID SET *IS* `rows`, SO THIS ARM TAKES THE WHOLE READ β€” the projection
456
+ # above cannot serve it and would raise rather than answer `{}` (that is the whole design of
457
+ # `_Projected`). The wall has already passed on the projected document, so this re-reads the
458
+ # DEFINITION and does not re-ask `may_open`: re-walling would be a second, differently-shaped
459
+ # answer to a question already answered, which is how two ideas of ownership got into this file
460
+ # once before (see `may_open`'s own note in `core/user_tables.py`).
461
+ #
462
+ # ⚠ THE HONEST COST, STATED RATHER THAN BURIED: a materialised table now pays the projection
463
+ # PLUS the whole read β€” ~1.4 ms on top of ~703 ms on tenant #0, i.e. 0.2%. The pid set, the
464
+ # wall and the returned shape are byte-for-byte what they were; only tenant #0's ten
465
+ # read-through databases (every one of them, which is why the switch was slow) skip the big
466
+ # read entirely.
467
+ whole = _ut().get(table_key, st=session.runtime)
468
+ if whole is None:
469
+ # Between the wall and here the table was deleted by another request. Same refusal the
470
+ # wall gives, rather than an empty pid set nobody can distinguish from an empty table.
471
+ raise err(404, "unknown_table", "that database does not exist")
472
+ pids = frozenset(int(rid) for rid in (whole.get("rows") or {}) if str(rid).isdigit())
473
+ return pids, fields_base, whole
474
 
475
 
476
  def ut_write_ctx(session: Session, table_key: str):
 
503
 
504
 
505
  def ut_assembly(session: Session, table_key: str, storage_key: str = "",
506
+ consume_corrections: bool = True, with_rows: bool = True, st=None):
507
  """The user-table mirror of `grid_assembly` / `product_assembly` β€” SAME g-dict keys, so
508
  `/workspace` and the events route consume any of the three interchangeably.
509
 
 
532
 
533
  limits = []
534
  if with_rows:
535
+ # ⭐ W33-T03 (D-214): `st` is a caller's LEND, threaded to the wall and nowhere else. Only
536
+ # a READ route passes one β€” see `scoped_pool`'s own note and contract C5.
537
+ pids, rows_src, fields_base, defn = scoped_pool(session, table_key, st=st)
538
  else:
539
  pids, fields_base, defn = scoped_pids(session, table_key, limits=limits)
540
  rows_src = []
 
735
  raise err(400, "bad_label", "give the database a name")
736
  if not session.runtime.available():
737
  raise err(503, "store_unavailable",
738
+ "the tenant store is unavailable. Nothing was created")
739
  source = body.get("source")
740
  try:
741
  key = ut.create(label, session.uname, fields=body.get("fields"),
742
  source=source, st=session.runtime)
743
  except Exception:
744
  raise err(503, "store_unavailable",
745
+ "the tenant store refused the write. Nothing was created")
746
  if not key:
747
  raise err(400, "refused",
748
+ f"could not create it. The name may be empty or this tenant already has "
749
  f"{ut.MAX_TABLES} databases")
750
  return {"key": key}
751
 
 
777
 
778
  Counts drill to the SAME buckets `user_tables.delete` cleans; a dialog listing categories
779
  without numbers would break [[no-unverifiable-aggregates]] at the scariest moment. Walled
780
+ like the delete itself: only someone who could delete may case the joint.
781
+
782
+ ⭐ W33-T01 / D-213: the wall and `createdBy`/`fields` come off the PROJECTION; only the row
783
+ COUNT needs the whole document, and only on a materialised table."""
784
+ defn = _defn_or_refuse(session, table_key, defs_only=True)
785
  if not (session.admin or defn.get("createdBy") == session.uname):
786
  raise err(403, "forbidden", "only the database's creator or an admin can delete it")
787
  s = session.runtime
 
804
  auto.append({"id": str(aid), "name": d.get("name") or str(aid)})
805
  except Exception:
806
  pass
807
+ # β›” THE ROW COUNT IS THE ONE FIELD THAT NEEDS THE WHOLE DOCUMENT, and it needs it only where
808
+ # the rows are actually stored here. A read-through database keeps `rows: {}` by construction,
809
+ # so `len(...)` answered **0** for it before this change and answers 0 now β€” identical, and the
810
+ # projection is not what makes it wrong.
811
+ # ⚠ 0 IS A WRONG NUMBER FOR A READ-THROUGH GRID and always was (`ut_odoo_gl_lines` would say 0
812
+ # in a dialog headed "what dies with this database"). Booked rather than fixed here: this
813
+ # ticket is a read-path change and correcting it means asking the mirror for a `count(*)`
814
+ # inside a confirm dialog. See the `PENDING:` line in `mailbox/A.md`.
815
+ if _ut().materialises(table_key, st=s, defn=defn):
816
+ rows = len((_ut().get(table_key, st=s) or {}).get("rows") or {})
817
+ else:
818
+ rows = 0
819
+ return {"rows": rows, "fields": fields, "views": len(views),
820
  "sharedUsers": len(g.get("entries") or []),
821
  "automations": sorted(auto, key=lambda a: a["name"].lower())}
822
 
 
845
  try:
846
  _ut().delete(table_key, st=session.runtime)
847
  except Exception:
848
+ raise err(503, "store_unavailable", "the delete did not land. Try again")
849
  return {"ok": True}
850
 
851
 
 
935
  import aios_grid
936
 
937
  _ig_forward(session)
938
+ # ⭐⭐ W33-T03 / D-214 β€” ONE READ OF THE TENANT DOCUMENT FOR THE WHOLE REQUEST, MEASURED.
939
+ # This route asked for it FOUR times: the wall (via `scoped_pool`), then `limit_report` TWICE
940
+ # (`row_limit` calls `materialises` and then `is_connected`, and each takes its own whole copy),
941
+ # then `records_mutable` at the envelope. Each is ~703 ms warm on tenant #0, and all four ask
942
+ # about the SAME document in the SAME request. The lend is the fix W31 QA already built for
943
+ # `_defn_or_refuse`; this threads it through the three sites that never got it.
944
+ # ⚠ SAFE HERE FOR THE SAME REASON IT IS SAFE THERE: this is a pure READ route. A lend is a
945
+ # PRE-WRITE snapshot, and handing one to a path that writes and then reads back would report the
946
+ # value it replaced (contract C5).
947
+ _lent = _ut().lend(session.runtime)
948
+ g = ut_assembly(session, table_key, st=_lent)
949
  # β›” THE OVERLAY WAS NEVER ACTUALLY MERGED, and the docstring above has described the merge
950
  # this line does not perform since the route was written (owner item 3, 2026-08-09:
951
  # *"Using the swipe, fast doesn't register the CHANGE. I went back and it all got reseted"*).
 
993
  # so a tenant using `json` for a short config sees no change at all.
994
  rows = aios_grid.rows_from_pool(
995
  g["rows_src"], g["fields"], _thin_json(g["fields"], merged, table_key), derived=g["derived"])
996
+ # ⭐⭐ WAVE-34 (R13) β€” the per-cell enrichment STATE rides the row, beside `_created`/`lat`/
997
+ # `lon`. See `_stamp_ai_states` for why it is a row key rather than a map beside `rows`.
998
+ _stamp_ai_states(table_key, g["fields"], rows, st=_lent)
999
  # ⭐ R6's SECOND SENTENCE, ON THE WIRE (W30-T29). *"if there is lag or it can't be done, you
1000
  # need to explicitly tell me why and recommend a fix."* A ceiling that still applies to this
1001
  # database says so here, with its cause and the recommendation, rather than waiting to be
1002
  # discovered as a refused paste. `None` for a connected source, and an EMPTY LIST is the
1003
  # honest answer for a table nothing limits β€” never an absent key, which a client cannot tell
1004
  # apart from an older server.
1005
+ _report = _ut().limit_report(table_key, st=_lent)
1006
  # ⭐ C4 / D-138 β€” THE DOCUMENTS PRODUCER. Absent since EXIT-6 deleted `app.py`, which was the
1007
  # only thing that ever set this key; the write door never stopped working and every client
1008
  # half is complete, but all six `onDoc*` handlers read `payload?.docs ? … : undefined`, so a
 
1016
  "identity": {"pid": "pid"},
1017
  "scope": {"table": table_key, "rowCount": len(rows)},
1018
  "limits": [_report] if _report else [],
1019
+ "recordsMutable": _ut().records_mutable(table_key, st=_lent)}
1020
+
1021
+
1022
+ #: The per-cell provenance a row carries on the wire, one key per enrichment column.
1023
+ #: β›” A ROW KEY RATHER THAN A SIBLING MAP, and the choice is load-bearing rather than cosmetic.
1024
+ #: A `{colId: {pid: state}}` map beside `rows` would need a new PROP on `RecordDetail` and a new
1025
+ #: argument at `CustomerGrid`'s call site, both in another lane's fence, to reach the two surfaces
1026
+ #: that must paint it. The row already carries `_created`, `lat` and `lon` for exactly this
1027
+ #: reason, so every reader already tolerates keys that are not columns, and both surfaces hold the
1028
+ #: row already. ⚠ COLLISION-PROOF BY CONSTRUCTION: `_clean_field` strips leading underscores off
1029
+ #: every field key, so no column can ever be called `_ai_*`.
1030
+ AI_STATE_PREFIX = "_ai_"
1031
+
1032
+
1033
+ def _stamp_ai_states(table_key, fields, rows, st=None):
1034
+ """Add `_ai_<colId>` to each row for every `ai_enrich` column. Mutates and returns `rows`.
1035
+
1036
+ β›” A PROJECTION, NOT THE MARK SET. The stratum holds a hash, a model, a timestamp, a token
1037
+ count and an error per cell; a browser needs ONE WORD to paint a state, and shipping the rest
1038
+ would grow this payload by a dict per enriched cell for data no reader reads. The vocabulary
1039
+ is `agent`/`human`/`stale`/`error` (`api/ai_enrich.py::cell_state`), which is also what the
1040
+ RUNNER obeys, so the badge and the behaviour cannot disagree about whose cell it is.
1041
+
1042
+ ⚠ AN ABSENT KEY MEANS `empty`, and only non-empty states are stamped: a table with no
1043
+ enrichment column is untouched, and a freshly created column adds nothing until something
1044
+ runs. β›” `stale` is DERIVED here rather than stored, so it is computed against TODAY'S row
1045
+ instead of against whatever was true when the value was written.
1046
+ """
1047
+ cols = [f for f in (fields or []) if str(f.get("type") or "") == "ai_enrich"]
1048
+ if not cols:
1049
+ return rows
1050
+ import ai_enrich as _ae
1051
+ for field in cols:
1052
+ col = str(field.get("key") or "")
1053
+ marks = _ut().ai_enrich_marks(table_key, col, st=st)
1054
+ cfg = field.get("aiEnrich") if isinstance(field.get("aiEnrich"), dict) else {}
1055
+ for row in (rows or []):
1056
+ if not isinstance(row, dict):
1057
+ continue
1058
+ state = _ae.cell_state(marks.get(str(row.get("pid"))), cfg, row, col)
1059
+ if state != "empty":
1060
+ row[AI_STATE_PREFIX + col] = state
1061
+ return rows
1062
 
1063
 
1064
  @router.get("/tables/{table_key}/rows/{pid}/fields/{fkey}")
 
1107
  rid = ut.add_row(table_key, values, session.uname, st=session.runtime,
1108
  rid=(body or {}).get("rid"))
1109
  except Exception:
1110
+ raise err(503, "store_unavailable", "the row was not saved. The store refused")
1111
  if rid is None:
1112
  # C3 (wave 25): `add_row` also refuses a profile cell that is not a handle, so the cap
1113
  # sentence alone would misdirect β€” the reader would go and count rows. Ask the same
 
1117
  _h, ok = ut.normalize_profile(values[pf["key"]], pf["profile"].get("source"))
1118
  if not ok:
1119
  raise err(400, "refused",
1120
+ f"{str(values[pf['key']])[:80]!r} is not an Instagram profile. "
1121
  f"{pf.get('label') or pf['key']!r} takes a handle (@name) or a "
1122
  f"profile link (instagram.com/name)")
1123
  raise err(400, "refused",
1124
+ f"row refused. The table may be at its {ut.MAX_ROWS}-row cap")
1125
  _refresh_relations(session)
1126
  return {"rid": rid, "pid": int(rid)}
1127
 
 
1175
  for key, value in row.items():
1176
  why = ut.cell_type_refusal(by_key[key], value)
1177
  if why:
1178
+ raise err(400, "bad_value", f"row {index + 1}: {why}. Nothing was imported")
1179
  try:
1180
  made = ut.add_rows(table_key, rows_in, session.uname, st=session.runtime)
1181
  except Exception:
1182
+ raise err(503, "store_unavailable", "nothing was imported. The store refused")
1183
  if made is None:
1184
  raise err(400, "refused",
1185
+ f"nothing was imported. {len(rows_in)} rows would take this database past "
1186
  f"its {ut.MAX_ROWS}-row cap, or a profile column rejected a value")
1187
  _refresh_relations(session)
1188
  return {"imported": len(made), "pids": [int(r) for r in made]}
 
1202
  # as the rename (item 6a): two places to look, and the surfaces disagreed silently.
1203
 
1204
  def _field_or_refuse(session, table_key, fkey=""):
1205
+ """The schema wall. `_defn_or_refuse` first (404/403 on the table), then the per-field rule.
1206
+
1207
+ ⭐ W33-T01 / D-213: definitions only. Everything read off `defn` here is `fields` and
1208
+ `createdBy`; the returned value is DISCARDED by all three callers (they re-fetch what they
1209
+ write through the `user_tables` write doors), so no projected snapshot survives into a write.
1210
+ ⚠ It is a SCHEMA wall on a write route, not a row-write wall β€” the distinction contract C5
1211
+ draws is about the snapshot reaching a read-BACK, and this one does not escape the function."""
1212
+ defn = _defn_or_refuse(session, table_key, defs_only=True)
1213
  ut = _ut()
1214
  if not ut.is_user_table(table_key, st=session.runtime):
1215
  raise err(400, "not_a_user_table",
1216
+ "only a user-created database has an editable schema. A connected source "
1217
  "owns its own columns")
1218
  if fkey and not ut.may_edit_field(table_key, fkey, session.uname, session.admin,
1219
  st=session.runtime):
 
1266
  # field relational", which is the question that was always meant.
1267
  if field.get("type") in ("link", "rollup"):
1268
  _refresh_relations(session)
1269
+ return _with_dropped(ut, {"field": field}, body)
1270
+
1271
+
1272
+ def _with_dropped(ut, out, body):
1273
+ """Attach the NAMED list of config keys the validator did not keep (wave 34, R13 / W34-T51).
1274
+
1275
+ β›”β›” THIS EXISTS BECAUSE THE VALIDATOR HAS NO ERROR CHANNEL AND CANNOT GROW ONE. Every bag
1276
+ cleaner in `core/user_tables.py` returns `dict | None` and drops unknown keys in silence, and
1277
+ `verify_fields_contract` asserts that they do -- so the drop is correct and the SILENCE is the
1278
+ defect. T51's contract is that an unknown config key is dropped **and named**, so the naming
1279
+ rides the response beside the accepted field rather than inside the validator.
1280
+
1281
+ ⚠ OMITTED WHEN EMPTY, deliberately: an always-present `dropped: []` teaches every reader to
1282
+ ignore the key, which is how a report stops being read before it stops being true.
1283
+ """
1284
+ dropped = ut.ai_enrich_dropped_keys((body or {}).get("aiEnrich"))
1285
+ if dropped:
1286
+ out = dict(out)
1287
+ out["dropped"] = dropped
1288
+ return out
1289
 
1290
 
1291
  def _refusal_sentence(ut, session, body, table_key="", fkey=""):
1292
  """Why was this column refused? The specific reason when we can name one, the general list
1293
  otherwise β€” never a specific-sounding guess."""
1294
+ # ⭐ WAVE-34 (R13): the enrichment column's own sentence, named BEFORE the automation bag
1295
+ # below. `_clean_field` DERIVES `field.automation` for this kind, so a refused enrichment
1296
+ # column would otherwise be explained by the flow law -- "pick a flow, or make this an
1297
+ # ordinary column" -- which is the D-46 misdirection exactly, pointing at a control the user
1298
+ # never touched.
1299
+ if str((body or {}).get("type") or "").strip().lower() == "ai_enrich":
1300
+ bag = (body or {}).get("aiEnrich")
1301
+ if not isinstance(bag, dict) or not str(bag.get("prompt") or "").strip():
1302
+ return ("an AI enrichment column needs a prompt. It is the only thing that can "
1303
+ "produce a value here, so a column without one would stay empty forever")
1304
  # ⭐ C3 (wave 25, R7): the one-profile-per-table refusal NAMES THE EXISTING COLUMN, which is
1305
  # what the contract asks for and what makes it actionable β€” "at most one" sends the reader
1306
  # hunting through a 40-column schema for a flag they cannot see from the header.
 
1312
  # real mistake was the column type sends them to fix the wrong thing, which is the
1313
  # misdirection D-46 closed one door over.
1314
  if str((body or {}).get("type") or "text").strip().lower() != "text":
1315
+ return ("a profile column is a flag on an ordinary TEXT column. It validates what "
1316
  "is typed into it, which it can only do for text")
1317
  existing = ut.profile_field(table_key, st=session.runtime) if table_key else None
1318
  if existing and existing.get("key") != str(fkey):
1319
+ return (f"this database already has a profile column: "
1320
  f"{existing.get('label') or existing.get('key')!r}. A database has at most "
1321
  f"one, so the automation knows which handle to enrich; edit that column, or "
1322
  f"take the flag off it first")
 
1324
  if isinstance(bag, dict):
1325
  flow = str(bag.get("flowId") or "").strip()
1326
  if not flow:
1327
+ return ("an automation column has to name the automation that fills it. Pick a "
1328
  "flow, or make this an ordinary column")
1329
  if not ut.flow_bound(bag, st=session.runtime):
1330
  return (f"this column names automation {flow!r}, which does not exist in this "
1331
+ f"workspace. It may have been deleted; pick a flow that is still there")
1332
  kind = str((body or {}).get("type") or "").strip()
1333
  if kind and kind not in ut.UT_FIELD_TYPES:
1334
  return (f"{kind!r} is not a column type here (types: "
1335
  f"{', '.join(sorted(ut.UT_FIELD_TYPES))})")
1336
+ return (f"the column was refused. Check the name and type, or the table may be at its "
1337
  f"{ut.MAX_FIELDS}-column cap (types: {', '.join(sorted(ut.UT_FIELD_TYPES))})")
1338
 
1339
 
 
1356
  try:
1357
  migrated = ut.rename_choice_values(table_key, fkey, renames, st=session.runtime)
1358
  except Exception:
1359
+ raise err(503, "store_unavailable", "the rename did not land. Try again")
1360
  # The per-user workspace strata and any view filter naming the old value are the OTHER
1361
  # half of C-RENAME and belong to `core.table_store`. Called only if it is there: an
1362
  # enumerator's mirror waits for its counterpart rather than guessing at its shape, and a
 
1383
  out = {"field": field}
1384
  if migrated is not None:
1385
  out["migrated"] = migrated
1386
+ return _with_dropped(ut, out, body)
1387
+
1388
+
1389
+ def _fire_on_change(table_key, pid, changed, session):
1390
+ """Run any `on_change` enrichment column whose prompt names a cell that just moved.
1391
+
1392
+ β›” ONE DEFINITION READ FOR THE WHOLE WRITE, and that is the point rather than an optimisation.
1393
+ `automation_engine.grid_hook` calls `all_definitions(st)` once PER EVENT, which turns a
1394
+ 20,000-row import into 20,000 whole-document reads on the single process this product runs
1395
+ (`D-134`, and `W34-T54`'s own `how:` says not to rebuild it). `on_change_fields` is pure and
1396
+ takes the definition, so this reads once and asks about every column.
1397
+
1398
+ β›” AND IT NEVER FAILS THE WRITE. The cell edit has already succeeded and been acknowledged;
1399
+ an enrichment that could not run is a missing value, not a lost edit, and the run's own report
1400
+ carries the reason. ⚠ It is also deliberately SYNCHRONOUS and bounded to this one row: a
1401
+ fan-out here would put a vendor call on the critical path of every keystroke-commit.
1402
+ """
1403
+ import ai_enrich as _ae
1404
+
1405
+ try:
1406
+ defn = _ut().get(table_key, st=session.runtime) or {}
1407
+ wanted = _ae.on_change_fields(defn, changed.keys())
1408
+ for field in wanted:
1409
+ # ⭐ W35-T41 / C7 β€” `user` is the usage ledger's attribution. An on-change run is still
1410
+ # somebody's edit spending somebody's tokens, so it is booked against the person who
1411
+ # typed rather than left unattributed.
1412
+ _ae.run_field(table_key, field["key"], st=session.runtime, rows=[str(pid)],
1413
+ user=session.uname)
1414
+ except Exception: # noqa: BLE001
1415
+ pass
1416
+
1417
+
1418
+ @router.post("/tables/{table_key}/fields/{fkey}/enrich")
1419
+ def enrich_field(table_key: str, fkey: str, body: dict = Body(default=None),
1420
+ session: Session = Depends(require_session)):
1421
+ """Run an AI enrichment column (wave 34, owner ruling R13). Returns the run's own REPORT.
1422
+
1423
+ β›” THIS DOOR SPENDS MONEY, so it rides the same wall every other schema write rides
1424
+ (`_field_or_refuse`) rather than a looser one of its own. A read-only viewer cannot bill the
1425
+ tenant by opening a grid.
1426
+
1427
+ `{"rows": ["3"]}` is a MANUAL run of exactly those records: a person asked, in front of the
1428
+ value being replaced, so it skips the `overwrite` policy. An ABSENT `rows` is the automatic
1429
+ plan, where the policy and the never-overwrite-a-human law both apply. The two are one
1430
+ function with one flag, not two runners.
1431
+
1432
+ ⚠ THE REPORT IS THE PRODUCT, not a status code. It carries `filled`, `failed`, `skipped` by
1433
+ reason, `tokens` spent, the provider, per-row errors, and `limit` (R6's second sentence: a
1434
+ ceiling that stopped the run names its cause and a remedy). A 200 with `filled: 0` and a
1435
+ populated `skipped` is a correct, informative answer, and the client must render it rather
1436
+ than treat it as success.
1437
+ """
1438
+ _field_or_refuse(session, table_key, fkey)
1439
+ import ai_enrich as _ae
1440
+
1441
+ rows = (body or {}).get("rows")
1442
+ if rows is not None and not isinstance(rows, list):
1443
+ raise err(400, "bad_rows", "`rows` must be a list of record ids, or absent to run the "
1444
+ "rows this column's own settings choose")
1445
+ # The caller's permitted pool, the same one the row doors use. A named row outside it is
1446
+ # dropped rather than refused: a stale client naming a record that has been deleted or moved
1447
+ # out of scope should not fail a run over the rows it can legitimately fill.
1448
+ if rows is not None:
1449
+ allowed = {str(p) for p in scoped_pids(session, table_key)[0]}
1450
+ rows = [str(r) for r in rows if str(r) in allowed]
1451
+ # ⭐ `W34-T54`'s bulk menu is this one field: "Rows never filled" sends `blank`, "All rows"
1452
+ # sends `always`. Anything else falls back to the column's own saved policy rather than to a
1453
+ # default, so a typo cannot quietly widen what a run touches.
1454
+ report = _ae.run_field(table_key, fkey, st=session.runtime, rows=rows,
1455
+ policy=str((body or {}).get("scope") or "") or None,
1456
+ # A named row set through THIS door is a person asking.
1457
+ manual=rows is not None,
1458
+ # ⭐ W35-T41 / C7 β€” the usage ledger's attribution.
1459
+ user=session.uname)
1460
+ if report.get("problem"):
1461
+ # A run that could not start at all is not a 200: nothing was attempted, nothing was
1462
+ # spent, and the reason is actionable (no provider configured, or the wrong column).
1463
+ raise err(400, "enrich_refused", str(report["problem"]))
1464
+ return report
1465
 
1466
 
1467
  @router.delete("/tables/{table_key}/fields/{fkey}")
 
1469
  _field_or_refuse(session, table_key, fkey)
1470
  if not _ut().delete_field(table_key, fkey, st=session.runtime):
1471
  raise err(400, "refused",
1472
+ "that column could not be removed. A database must keep at least one")
1473
  _refresh_relations(session)
1474
  return {"deleted": fkey}
1475
 
 
1494
  try:
1495
  ok = _ut().delete_row(table_key, rid, st=lent)
1496
  except Exception:
1497
+ raise err(503, "store_unavailable", "the delete did not land. Try again")
1498
  if not ok:
1499
  raise err(400, "refused", "rows can only be deleted from user-created databases")
1500
  _refresh_relations(session)
 
1527
  "pid": pid, "updates": updates}, ctx)
1528
  except grid_events.StoreUnavailable:
1529
  raise err(503, "store_unavailable",
1530
+ "the tenant store is unavailable. Your change was not saved")
1531
  # ⚠ THE READ-BACK IS THE DEFINITION ROW, AND ON A `ut_` SCOPE THAT IS THE WHOLE OF IT.
1532
  #
1533
  # β›” CORRECTED, wave-29 T22 (owner item 2a): this note used to say "THE READ-BACK SPANS BOTH
 
1566
  return False
1567
 
1568
  refused = sorted(k for k in updates if not _took(k))
1569
+ # ⭐⭐ WAVE-34 (R13) β€” THE HUMAN-EDIT STAMP, AND IT HAS TO HAPPEN HERE. "Did a person write
1570
+ # this cell?" is not recoverable from the value afterwards, so the only place to record it is
1571
+ # the door a person writes through. `ai_enrich_may_write` then refuses to let any automatic
1572
+ # run overwrite it, whatever the column's `overwrite` policy says.
1573
+ # ⚠ STAMPED FROM THE CELLS THAT ACTUALLY TOOK, never from what was asked: marking a refused
1574
+ # write `human` would freeze a cell against the agent on the strength of an edit that never
1575
+ # landed. `note_human_edit` filters to the enrichment columns itself and is a no-op otherwise.
1576
+ took = {k: v for k, v in accepted.items() if k not in refused}
1577
+ if took:
1578
+ try:
1579
+ _ut().note_human_edit(table_key, took, pid, st=session.runtime)
1580
+ except Exception: # noqa: BLE001
1581
+ # Provenance is metadata about a write that has already succeeded. Failing the
1582
+ # request here would tell the user their edit was lost when it was not.
1583
+ pass
1584
+ _fire_on_change(table_key, pid, took, session)
1585
  out = {"pid": pid, "updates": accepted}
1586
  if refused:
1587
  out["refused"] = refused
api/routes_usage.py ADDED
@@ -0,0 +1,51 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """routes_usage.py β€” WAVE 35 (ruling R9, contract C7): `GET /api/v1/usage`, the AI meter.
2
+
3
+ Owner item 13's second door. R9: the dashboard REPORTS weekly usage per surface against an
4
+ allowance and does **not** cut anyone off this wave; over the allowance the product still works and
5
+ the bar is red.
6
+
7
+ β›” THIS ROUTE ENFORCES NOTHING AND MUST NOT START. It is the only reader of `usage_ledger`, and the
8
+ ledger has no ceiling in it (see that module's header). A `403 over_allowance` here would turn a
9
+ reporting feature into a wall the owner explicitly did not ask for this wave.
10
+
11
+ ⚠ TWO SCOPES, AND THE WIDER ONE IS ADMIN-ONLY. `?scope=me` (the default) is the caller's own week;
12
+ `?scope=tenant` folds every account and is refused for a non-admin β€” how much AI another employee
13
+ used is not a fact this product hands out sideways.
14
+ """
15
+ from fastapi import APIRouter, Depends
16
+
17
+ import usage_ledger
18
+ from deps import Session, err, require_session
19
+
20
+ router = APIRouter(prefix="/api/v1")
21
+
22
+
23
+ @router.get("/usage")
24
+ def usage(scope: str = "me", session: Session = Depends(require_session)):
25
+ """`{week, resets, allowance, total, over, calls, unmeasured, scope, surfaces: [...]}`.
26
+
27
+ Every one of the four surfaces is present whether or not it was used β€” see
28
+ `usage_ledger.usage`'s note on why a missing row and a quiet week must not look the same.
29
+ """
30
+ wanted = str(scope or "me").strip().lower()
31
+ if wanted not in ("me", "tenant"):
32
+ raise err(400, "bad_request", "scope must be me or tenant")
33
+ if wanted == "tenant" and not session.admin:
34
+ raise err(403, "forbidden",
35
+ "only an administrator can see the whole workspace's AI usage")
36
+ out = usage_ledger.usage(session.runtime, session.uname,
37
+ tenant_wide=(wanted == "tenant"))
38
+ if session.admin and usage_ledger.UNATTRIBUTED["calls"]:
39
+ # β›” THE METER REPORTS ITS OWN BLIND SPOT (R6's second sentence). A call `record()` could not
40
+ # attribute β€” no store handle reached it, or the store refused β€” is absent from every number
41
+ # above, and an operator has to be able to see that rather than infer it.
42
+ # ⚠ ADMIN ONLY, and labelled as a PROCESS figure rather than a tenant one: the counter is a
43
+ # module global in this container, so it spans every tenant this process has served and is
44
+ # not a fact about the workspace being asked about.
45
+ out["unattributed"] = {
46
+ "calls": usage_ledger.UNATTRIBUTED["calls"],
47
+ "surfaces": dict(usage_ledger.UNATTRIBUTED["surfaces"]),
48
+ "note": ("these AI calls reached no ledger target in this container and are missing "
49
+ "from every figure above. It is a wiring gap, not usage"),
50
+ }
51
+ return out
api/usage_ledger.py ADDED
@@ -0,0 +1,281 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """usage_ledger.py β€” WAVE 35 (ruling R9, contract C7): ONE METER FOR EVERY AI SURFACE.
2
+
3
+ R9: *"ONE meter for every AI surface. The dashboard REPORTS weekly usage against an allowance and
4
+ does NOT cut anyone off this wave; over the allowance it still works, shown red."*
5
+
6
+ β›” SO THIS FILE MEASURES AND NEVER ENFORCES. There is no ceiling here, no refusal, and no caller of
7
+ this module may branch on its answer. The one limit the product does enforce is per COLUMN and lives
8
+ where it was built (`ai_enrich.ceiling_report`); this is the tenant-wide REPORT, and mixing the two
9
+ would make an over-allowance week silently stop somebody's enrichment run.
10
+
11
+ **THE FOUR SURFACES.** `assistant` (a person asking the Assistant a question) Β· `field_agent` (an AI
12
+ enrichment column filling cells) Β· `ai_review` (a model deciding a review stage) Β· `automation_draft`
13
+ (a sentence becoming a draft flow). They are the four LLM entry points the product has, and the gate
14
+ in `verify_api.py` DERIVES them from the code rather than reading this list, so a fifth entry point
15
+ is caught by the gate and not by somebody remembering to edit a tuple.
16
+
17
+ β›”β›” **`record()` MUST BE HANDED ITS TARGET, AND THAT IS A MEASUREMENT, NOT A STYLE CHOICE.** The
18
+ obvious design is to bind `(runtime, username)` into a `contextvars.ContextVar` inside
19
+ `deps.require_session` β€” every session-authed request passes through it β€” so `record()` could take
20
+ C7's five arguments and resolve the rest itself. **It does not work here and it fails SILENTLY.**
21
+ Measured 2026-08-17 with a FastAPI `TestClient`: a `ContextVar` set inside a sync dependency reads
22
+ back `None` in the handler *and in every function the handler calls*, on the SAME OS thread (17820
23
+ both sides) β€” anyio copies the context per callable, so a `.set()` in the dependency's copy is
24
+ invisible to the endpoint's. An async endpoint reads `None` too. A ledger built that way would have
25
+ recorded nothing, answered no error, and shown a dashboard of zeros that looks exactly like a quiet
26
+ week ([[flag-shipped-without-its-writer]]).
27
+
28
+ ⚠ THEREFORE `st` AND `user` ARE KEYWORD ARGUMENTS AFTER C7'S FIVE, and a call that omits them is
29
+ COUNTED as unattributed and REPORTED rather than dropped β€” R6's second sentence applied to the meter
30
+ itself. A surface whose caller cannot reach a store handle is a gap somebody has to close; a meter
31
+ that hides the gap is worse than one that admits it.
32
+ """
33
+ from __future__ import annotations
34
+
35
+ import os
36
+ import time
37
+
38
+ #: The store bucket. Small by construction β€” counters only, never a per-call log: a log would grow
39
+ #: without bound on a route that runs per row, and nothing in R9 asks a question a log answers.
40
+ STORE_KEY = "ai_usage"
41
+
42
+ #: The four AI surfaces (C7). ⚠ A surface this list does not name is REFUSED rather than stored under
43
+ #: whatever string arrived, because a typo'd surface is a meter that reads low forever with a second
44
+ #: row nobody looks at.
45
+ SURFACES = ("assistant", "field_agent", "ai_review", "automation_draft")
46
+
47
+ #: The weekly allowance the dashboard measures against (R9). β›” IT IS A REPORTING LINE, NOT A GATE.
48
+ #: Env-tunable so a deployment can set a real number without a release.
49
+ WEEKLY_ALLOWANCE = int(os.environ.get("AIOS_AI_WEEKLY_TOKENS") or 1_000_000)
50
+
51
+ #: How many weeks stay in the bucket. A meter is about this week and the trend behind it; keeping
52
+ #: every week forever turns a counter document into an append-only log by another route.
53
+ MAX_WEEKS = 8
54
+
55
+ #: Calls this PROCESS could not attribute to a store and a user β€” see the header. Reported to an
56
+ #: admin by `GET /usage`, never silently zero.
57
+ UNATTRIBUTED = {"calls": 0, "surfaces": {}}
58
+
59
+
60
+ def week_of(when=None):
61
+ """The ISO week this instant belongs to, as `"2026-W33"`.
62
+
63
+ ⚠ UTC, EXPLICITLY, and never a naive local stamp. `routes_nav` records the same correction for
64
+ its own stamps: a naive local time written on a UTC host and parsed by a non-UTC browser misfiles
65
+ into the wrong bucket with nothing to go red. `%G`/`%V` are the ISO year and ISO week, which is
66
+ the only pair that agrees with itself across a year boundary (`%Y-%W` does not).
67
+ """
68
+ return time.strftime("%G-W%V", time.gmtime(when if when is not None else time.time()))
69
+
70
+
71
+ def resets_at(when=None):
72
+ """The UTC date the CURRENT week's counters roll over, as `"YYYY-MM-DD"`.
73
+
74
+ R9 asks the page to state when the allowance resets, so the server answers it β€” a client that
75
+ computed "next Monday" itself would be a second implementation of the week boundary, and the two
76
+ would disagree for anybody whose clock is not UTC ([[one-question-two-normalizers]]).
77
+ """
78
+ now = when if when is not None else time.time()
79
+ tm = time.gmtime(now)
80
+ # `tm_wday` is 0 for Monday, so days-until-next-Monday is 7 minus however far in we are.
81
+ return time.strftime("%Y-%m-%d", time.gmtime(now + (7 - tm.tm_wday) * 86400))
82
+
83
+
84
+ def tokens_from(body):
85
+ """`(tokens_in, tokens_out)` for one provider response. Either may be None.
86
+
87
+ β›” `None` MEANS THE PROVIDER DID NOT SAY, AND IT IS NOT ZERO. A ledger that books an unmeasured
88
+ call at zero reports a cheaper week than happened, which is exactly the cost-surprise complaint
89
+ R13 cited when the first token accounting was built. `ai_enrich._usage_tokens` has said this
90
+ since wave 34; this is that reader SPLIT, because C7 asks for input and output separately.
91
+
92
+ ⚠ THE TOTAL-ONLY SHAPE IS REAL AND IS NOT A SPLIT. Some providers answer only `total_tokens`.
93
+ That is reported by `record(total=...)` rather than by inventing a split, because halving a
94
+ total would put two numbers on a dashboard that were never measured.
95
+ """
96
+ usage = (body or {}).get("usage")
97
+ if not isinstance(usage, dict):
98
+ return (None, None)
99
+
100
+ def _int(*keys):
101
+ for key in keys:
102
+ got = usage.get(key)
103
+ if isinstance(got, int) and not isinstance(got, bool):
104
+ return got
105
+ return None
106
+
107
+ return (_int("prompt_tokens", "input_tokens"),
108
+ _int("completion_tokens", "output_tokens"))
109
+
110
+
111
+ def total_from(body):
112
+ """One call's TOTAL tokens, or None β€” the shape a caller with no use for the split wants.
113
+
114
+ ⚠ Prefers the provider's own `total_tokens` over a sum, because a provider that reports both may
115
+ count cached or reasoning tokens in the total and in neither half.
116
+ """
117
+ usage = (body or {}).get("usage")
118
+ if isinstance(usage, dict):
119
+ for key in ("total_tokens", "totalTokens"):
120
+ got = usage.get(key)
121
+ if isinstance(got, int) and not isinstance(got, bool):
122
+ return got
123
+ ins, outs = tokens_from(body)
124
+ if ins is None and outs is None:
125
+ return None
126
+ return (ins or 0) + (outs or 0)
127
+
128
+
129
+ def _int_or_none(value):
130
+ if isinstance(value, int) and not isinstance(value, bool):
131
+ return max(0, value)
132
+ return None
133
+
134
+
135
+ def record(surface, provider, model, tokens_in=None, tokens_out=None, *,
136
+ total=None, calls=1, st=None, user=""):
137
+ """Book ONE AI call. Returns True when the line landed durably, False when it could not.
138
+
139
+ C7's signature is the five positional parameters; everything after `*` is this module's own
140
+ extension and each one has a reason:
141
+
142
+ Β· `total` β€” for a caller that knows the total and not the split (`ai_enrich.run_field`
143
+ accumulates a per-run total through an `_ask` contract a gate injects against, and changing
144
+ that contract to carry a split would break the injected shape for no gain here).
145
+ Β· `calls` β€” so a RUN over many rows books one line with its real call count instead of one
146
+ store write per row.
147
+ Β· `st` / `user` β€” the target. Required in practice; see this module's header for the measured
148
+ reason a contextvar cannot supply them.
149
+
150
+ β›” IT NEVER RAISES. A meter that can break the feature it measures is worse than no meter: a
151
+ store blip during an enrichment run would abort the run and lose the cells.
152
+ β›” AND IT NEVER SILENTLY SUCCEEDS. A call it cannot attribute increments `UNATTRIBUTED`, which
153
+ `GET /usage` shows an admin.
154
+ """
155
+ key = str(surface or "").strip()
156
+ if key not in SURFACES:
157
+ # Refused rather than stored: a surface nobody named is a row nobody reads, and the meter
158
+ # would read low forever with no sign of why.
159
+ return False
160
+ ins, outs = _int_or_none(tokens_in), _int_or_none(tokens_out)
161
+ tot = _int_or_none(total)
162
+ if tot is None and (ins is not None or outs is not None):
163
+ tot = (ins or 0) + (outs or 0)
164
+ n = max(0, int(calls) if isinstance(calls, int) and not isinstance(calls, bool) else 1)
165
+ if not n:
166
+ return False
167
+ uname = str(user or "").strip().lower()
168
+ if st is None:
169
+ UNATTRIBUTED["calls"] += n
170
+ UNATTRIBUTED["surfaces"][key] = UNATTRIBUTED["surfaces"].get(key, 0) + n
171
+ return False
172
+ week = week_of()
173
+ prov, mdl = str(provider or "")[:40], str(model or "")[:80]
174
+
175
+ def _up(data):
176
+ data = data if isinstance(data, dict) else {}
177
+ weeks = {w: v for w, v in data.items() if isinstance(v, dict)}
178
+ row = ((weeks.setdefault(week, {})
179
+ .setdefault(uname or "-", {}))
180
+ .setdefault(key, {}))
181
+ row["calls"] = int(row.get("calls") or 0) + n
182
+ if tot is None:
183
+ # β›” "the call happened, the tokens are unknown" IS A STATE, and it has to be
184
+ # representable or the meter quietly under-reports. The dashboard shows it as a count of
185
+ # unmeasured calls beside the token total.
186
+ row["unmeasured"] = int(row.get("unmeasured") or 0) + n
187
+ else:
188
+ row["tokens"] = int(row.get("tokens") or 0) + tot
189
+ if ins is not None:
190
+ row["tokens_in"] = int(row.get("tokens_in") or 0) + ins
191
+ if outs is not None:
192
+ row["tokens_out"] = int(row.get("tokens_out") or 0) + outs
193
+ if prov:
194
+ row["provider"] = prov
195
+ if mdl:
196
+ row["model"] = mdl
197
+ # Oldest weeks first to go. Sorting ISO week strings sorts chronologically by construction,
198
+ # which is the second reason `%G-W%V` is the format rather than a prettier one.
199
+ for stale in sorted(weeks)[:-MAX_WEEKS]:
200
+ weeks.pop(stale, None)
201
+ return weeks
202
+
203
+ try:
204
+ # ⚠ `flush='async'` β€” the coalescing mode ([[store-async-flush]]). This is the hot path of
205
+ # every AI surface in the product (an enrichment run books one line per RUN, but the
206
+ # assistant books one per question), and a blocking hub upload per call would put a network
207
+ # round trip inside the answer a person is waiting for. The mutation applies to the
208
+ # in-process cache immediately, so `GET /usage` reads its own writes.
209
+ st.update(STORE_KEY, _up, flush="async")
210
+ return True
211
+ except Exception: # noqa: BLE001
212
+ UNATTRIBUTED["calls"] += n
213
+ UNATTRIBUTED["surfaces"][key] = UNATTRIBUTED["surfaces"].get(key, 0) + n
214
+ return False
215
+
216
+
217
+ def _blank_row():
218
+ return {"calls": 0, "tokens": 0, "tokens_in": 0, "tokens_out": 0, "unmeasured": 0}
219
+
220
+
221
+ def _fold(rows):
222
+ out = _blank_row()
223
+ for row in rows:
224
+ if not isinstance(row, dict):
225
+ continue
226
+ for field in ("calls", "tokens", "tokens_in", "tokens_out", "unmeasured"):
227
+ got = row.get(field)
228
+ if isinstance(got, int) and not isinstance(got, bool):
229
+ out[field] += max(0, got)
230
+ return out
231
+
232
+
233
+ def usage(st, user="", *, tenant_wide=False, week=None):
234
+ """This week's meter. `{week, resets, allowance, total, over, surfaces: [...], ...}`.
235
+
236
+ `tenant_wide=True` folds every account in the tenant β€” the admin view. Otherwise only `user`.
237
+
238
+ β›” EVERY SURFACE IS PRESENT WHETHER OR NOT IT WAS USED, at zero. A surface a consumer has to test
239
+ for is a surface a consumer forgets to test for, and R9 says ONE meter for every AI surface: a
240
+ missing row and a quiet week would render identically, and only one of them is true.
241
+ β›” NOTHING HERE IS ESTIMATED. A surface with no ledger line reports zero, never a guess
242
+ ([[no-unverifiable-aggregates]]).
243
+ """
244
+ wk = str(week or week_of())
245
+ try:
246
+ stored = st.get(STORE_KEY) or {}
247
+ except Exception: # noqa: BLE001
248
+ stored = {}
249
+ by_week = stored.get(wk) if isinstance(stored, dict) else {}
250
+ by_week = by_week if isinstance(by_week, dict) else {}
251
+ uname = str(user or "").strip().lower()
252
+ if tenant_wide:
253
+ buckets = [b for b in by_week.values() if isinstance(b, dict)]
254
+ else:
255
+ buckets = [by_week.get(uname or "-")] if isinstance(by_week.get(uname or "-"), dict) else []
256
+ surfaces = []
257
+ for key in SURFACES:
258
+ folded = _fold([b.get(key) for b in buckets])
259
+ folded["surface"] = key
260
+ surfaces.append(folded)
261
+ total = sum(s["tokens"] for s in surfaces)
262
+ unmeasured = sum(s["unmeasured"] for s in surfaces)
263
+ out = {
264
+ "week": wk,
265
+ "resets": resets_at(),
266
+ "allowance": WEEKLY_ALLOWANCE,
267
+ "total": total,
268
+ # R9: over the allowance the product STILL WORKS and the bar goes red. This flag is the
269
+ # colour, never a wall β€” nothing in the API reads it.
270
+ "over": bool(WEEKLY_ALLOWANCE and total > WEEKLY_ALLOWANCE),
271
+ "calls": sum(s["calls"] for s in surfaces),
272
+ "unmeasured": unmeasured,
273
+ "scope": "tenant" if tenant_wide else "me",
274
+ "surfaces": surfaces,
275
+ }
276
+ if unmeasured:
277
+ # R6's second sentence: the number is short by an unknown amount and the payload says so
278
+ # rather than presenting a confident total.
279
+ out["note"] = (f"{unmeasured} call(s) this week did not report a token count, so the total "
280
+ f"above is a floor. The provider decides whether to report usage")
281
+ return out
api/web_agent.py CHANGED
@@ -1,9 +1,10 @@
1
  """web_agent.py β€” THE SEAM between an automation step and a browser that runs somewhere else.
2
 
3
- CONTRACT C5 (wave 31, ruling R10 / D-51). E ships this; **C mounts it** in
4
- `automation_engine.py`'s action dispatch for `web_read` and adds the `verify_wiring` row β€”
5
- β›” E cannot verify its own mounting, and an unmounted runner is a whole, correct, UNREACHABLE
6
- feature, which is exactly how five wave-29 features shipped behind green gates.
 
7
 
8
  result, error = web_agent.run_step(step, ctx) # -> (dict|None, str)
9
 
@@ -30,22 +31,39 @@ foreground step of an automation RUN and far too slow for a route a person is wa
30
  cost is per JOB, not per step β€” `run_plan` sends a whole flow's steps in ONE job for that reason.
31
 
32
  ────────────────────────────────────────────────────────────────────────────────────────────────
33
- β›” THREE THINGS ARE TRUE OF PRODUCTION TODAY AND ARE NOT DEFECTS IN THIS FILE. `capability()`
34
- reports each as a sentence, and `verify_web_agent.py` holds each to a check, so none of them can
35
- be discovered by a customer instead of by us:
36
-
37
- 1. THE SPACE'S TOKEN CANNOT PAY FOR A JOB. `deploy_web.py` pushes `HF_TOKEN`, which is scoped to
38
- the org `royal-imports`; HF answers **402 `Pre-paid credit balance is insufficient`** there
39
- and 0 jobs have ever run under it. `AIOS_HF_TOKEN` (user `fsanyoto`, 100 jobs) works and is
40
- NOT pushed. Measured 2026-08-12.
41
- 2. THE SPACE DOES NOT SHIP THE JOB SCRIPT. `deploy_web.py` uploads `api/*.py` and
42
- `platform/{core,modules,harness}`; `jobs/` is in no manifest. And `check_upload()` walks
43
- IMPORTS, so it structurally cannot see a file opened by PATH β€” its own comments say so, about
44
- `aios_grid_fields.json`, which crashed a Space for exactly this reason.
45
- 3. `routes_web_agent.py` IS NOT MOUNTED. `main.py` is another lane's file this wave.
46
-
47
- Each needs one line somebody else owns. Until then this module answers with a sentence rather
48
- than a silence, which is the whole of R6's second half.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
49
  """
50
  import base64
51
  import hashlib
@@ -223,7 +241,7 @@ def _sentence_for(exc, namespace):
223
  f"point this deployment at one that has it ({ENV_NAMESPACE} / {ENV_TOKEN[0]}). "
224
  f"Nothing was read.")
225
  if status in (401, 403):
226
- return (f"Hugging Face refused the browser job as unauthorised for {where} β€” this "
227
  f"deployment's token does not carry `job.write` on that account. Point it at an "
228
  f"account the token owns ({ENV_NAMESPACE}), or configure a token that owns this "
229
  f"one ({ENV_TOKEN[0]}). Nothing was read.")
@@ -524,7 +542,7 @@ def _await(api, job_id, namespace, started, log):
524
  break
525
  if state in ("COMPLETED", "ERROR", "CANCELED"):
526
  return None, (f"The browser job {job_id} ended {state} without returning a result"
527
- f"{' β€” it never started printing' if not booted else ''}. "
528
  f"Nothing was read.")
529
  if not booted and waited > START_TIMEOUT:
530
  # The submitted-but-never-started case, named as itself.
@@ -534,7 +552,7 @@ def _await(api, job_id, namespace, started, log):
534
  pass
535
  return None, (f"The browser job {job_id} was accepted but had not started after "
536
  f"{int(START_TIMEOUT)}s (it stayed {state or 'queued'}), so it was "
537
- f"cancelled. Nothing was read β€” try again, or the platform is busy.")
538
  if waited > RUN_TIMEOUT:
539
  try:
540
  api.cancel_job(job_id=job_id, namespace=namespace)
 
1
  """web_agent.py β€” THE SEAM between an automation step and a browser that runs somewhere else.
2
 
3
+ CONTRACT C5 (wave 31, ruling R10 / D-51). E shipped this and C mounted it in
4
+ `automation_engine.py`'s action dispatch; both halves landed and the `verify_wiring` row exists.
5
+ ⚠ That sentence was written in the FUTURE TENSE as a wave-31 hand-off and stayed that way for
6
+ three waves, which is the smaller cousin of the correction below: a fence note describes one
7
+ wave's division of labour and reads, afterwards, as a description of the code.
8
 
9
  result, error = web_agent.run_step(step, ctx) # -> (dict|None, str)
10
 
 
31
  cost is per JOB, not per step β€” `run_plan` sends a whole flow's steps in ONE job for that reason.
32
 
33
  ────────────────────────────────────────────────────────────────────────────────────────────────
34
+ β›”β›” WAVE 34 Β· W34-T44 β€” THIS BLOCK LISTED "THREE THINGS TRUE OF PRODUCTION TODAY". ALL THREE WERE
35
+ FALSE BY THE TIME ANYBODY READ THEM, AND THIS DOCSTRING IS WHERE THE ERROR CAME FROM.
36
+
37
+ It said the Space token 402s, that `deploy_web.py` never uploads `jobs/`, and that
38
+ `routes_web_agent.py` is not mounted β€” so the whole web-action family was inert in production.
39
+ **Four later documents repeated it** (the wave-34 PRD's scouted index, the plan-time scout's
40
+ report, `W34-T44`'s own `how:`, and `run_gates.ps1`'s annotation telling a deploy to expect
41
+ `web_agent` at 48/50), and every one of them traced back here. Re-checked line by line, against
42
+ the files themselves, on 2026-08-16:
43
+
44
+ 1. THE TOKEN IS PUSHED. `deploy_web.py:1128` adds `AIOS_HF_TOKEN` to the Space's secrets when it
45
+ is set, and `ENV_TOKEN` below prefers it over `HF_TOKEN`, so the org token that 402s is never
46
+ the one reached. The 402 itself is still true OF THAT TOKEN and is why the order matters.
47
+ 2. THE JOB SCRIPT SHIPS. `deploy_web.py::_jobs_files` (:413) collects `jobs/` and :867 uploads
48
+ it. The `check_upload()` caveat stands as a caveat: it walks IMPORTS and cannot see a file
49
+ opened by PATH, so this upload is covered by the manifest, never by that check.
50
+ 3. IT IS MOUNTED. `main.py:264`, `app.include_router(routes_web_agent.router)` β€” and
51
+ `verify_web_agent.py` asserts the route is in the SERVED set (`app.openapi()["paths"]`),
52
+ which is green.
53
+
54
+ ⭐ MEASURED END TO END the same day, from a dev box: `capability()` = ready True / "configured" /
55
+ `AIOS_HF_TOKEN` / playwright 1.62.0, and a real `web_read` of `https://example.com h1` submitted in
56
+ 3.8 s, was alive at 21.4 s and returned `value "Example Domain"`, status 200, 106 ms in-browser,
57
+ 24.5 s wall clock cold.
58
+ ⚠ STILL UNPROVEN, so nobody over-corrects in the other direction: that run was NOT on the deployed
59
+ Space, which carries the secret only if the last deploy read this `.env`. Proving it needs a click
60
+ on staging, and until then "it works" means "it works from a box with the token".
61
+
62
+ β›” THE STANDING LESSON, kept here because this file is where it started: **a docstring stating
63
+ facts about PRODUCTION has no expiry and nothing re-reads it when the world changes.** Three
64
+ sentences about somebody else's file outlived all three of those files' fixes, and were quoted as
65
+ current by four documents and a ticket. `capability()` answers the same question in under a second
66
+ and returns TODAY's answer; prefer calling it to reading this.
67
  """
68
  import base64
69
  import hashlib
 
241
  f"point this deployment at one that has it ({ENV_NAMESPACE} / {ENV_TOKEN[0]}). "
242
  f"Nothing was read.")
243
  if status in (401, 403):
244
+ return (f"Hugging Face refused the browser job as unauthorised for {where}. This "
245
  f"deployment's token does not carry `job.write` on that account. Point it at an "
246
  f"account the token owns ({ENV_NAMESPACE}), or configure a token that owns this "
247
  f"one ({ENV_TOKEN[0]}). Nothing was read.")
 
542
  break
543
  if state in ("COMPLETED", "ERROR", "CANCELED"):
544
  return None, (f"The browser job {job_id} ended {state} without returning a result"
545
+ f"{'. It never started printing' if not booted else ''}. "
546
  f"Nothing was read.")
547
  if not booted and waited > START_TIMEOUT:
548
  # The submitted-but-never-started case, named as itself.
 
552
  pass
553
  return None, (f"The browser job {job_id} was accepted but had not started after "
554
  f"{int(START_TIMEOUT)}s (it stayed {state or 'queued'}), so it was "
555
+ f"cancelled. Nothing was read. Try again, or the platform is busy.")
556
  if waited > RUN_TIMEOUT:
557
  try:
558
  api.cancel_job(job_id=job_id, namespace=namespace)
jobs/web_agent_job.py CHANGED
@@ -113,6 +113,21 @@ INSTALL_DEPS = os.environ.get("WEB_AGENT_INSTALL_DEPS", "1") != "0"
113
  INSTALL_BROWSER = os.environ.get("WEB_AGENT_INSTALL_BROWSER", "1") != "0"
114
  MAX_VALUE_CHARS = 8000 # one cell's worth; a page is not a payload
115
  MAX_STEPS = 20 # matches the engine's MAX_ACTIONS β€” one flow, one job
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
116
 
117
 
118
  def _fail(sentence: str, *, kind: str = "refused", steps=None) -> None:
@@ -162,6 +177,8 @@ def _checkpoint(blob: str, payload: dict) -> None:
162
  repo_id=CHECKPOINT_REPO, repo_type="dataset",
163
  commit_message=f"web-agent run {run_id}")
164
  say(f"CHECKPOINT wrote runs/{run_id}.json to {CHECKPOINT_REPO}")
 
 
165
  except Exception as exc: # noqa: BLE001 β€” a boundary
166
  # β›” CATCH EVERYTHING. This is a report-don't-raise boundary and the ways an upload can
167
  # fail are not enumerable (429 commit budget, 403 scope, DNS, a repo that does not
@@ -171,8 +188,94 @@ def _checkpoint(blob: str, payload: dict) -> None:
171
  f"the stdout frame still carries the result")
172
 
173
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
174
  def _resume(run_id: str):
175
- """A prior run's result, if this exact run already finished. None otherwise."""
 
 
 
 
 
 
 
 
 
 
 
 
 
 
176
  if not (CHECKPOINT_REPO and HF_TOKEN and run_id):
177
  return None
178
  try:
@@ -181,9 +284,21 @@ def _resume(run_id: str):
181
  filename=f"runs/{run_id}.json", token=HF_TOKEN,
182
  force_download=True)
183
  with open(path, encoding="utf-8") as fh:
184
- return json.load(fh)
185
  except Exception: # noqa: BLE001
186
  return None # absent is the normal case
 
 
 
 
 
 
 
 
 
 
 
 
187
 
188
 
189
  # ── the plan ────────────────────────────────────────────────────────────────────────────────
 
113
  INSTALL_BROWSER = os.environ.get("WEB_AGENT_INSTALL_BROWSER", "1") != "0"
114
  MAX_VALUE_CHARS = 8000 # one cell's worth; a page is not a payload
115
  MAX_STEPS = 20 # matches the engine's MAX_ACTIONS β€” one flow, one job
116
+ #: β›” D-193 β€” HOW OLD A CHECKPOINT MAY BE AND STILL BE SERVED, in hours. **Deliberately SHORT.**
117
+ #: The key is content-addressed (`runId` + a hash of the steps), so a scheduled automation with a
118
+ #: stable run id hits the same key every time it runs; with no ceiling it would be served the same
119
+ #: stored page for as long as the file existed, which is a measurement of a web page presented with
120
+ #: today's date on it. Six hours makes a same-morning retry cheap and a next-day run honest.
121
+ #: ⚠ A CEILING, NOT A CACHE POLICY. The point of the checkpoint is that a RETRY of a job that
122
+ #: already paid for a fetch does not pay again; that window is minutes, not days.
123
+ try:
124
+ CHECKPOINT_MAX_AGE_H = float(os.environ.get("WEB_AGENT_CHECKPOINT_MAX_AGE_H", "6") or 6)
125
+ except (TypeError, ValueError):
126
+ CHECKPOINT_MAX_AGE_H = 6.0
127
+ #: How many expired `runs/*.json` one job may delete, in ONE commit. β›” Bounded because the repo
128
+ #: already spends a commit per run against HF's 256-commits/hr budget, and a sweep that spent one
129
+ #: commit per FILE would be a denial of service on the store it is tidying.
130
+ CHECKPOINT_SWEEP_MAX = 200
131
 
132
 
133
  def _fail(sentence: str, *, kind: str = "refused", steps=None) -> None:
 
177
  repo_id=CHECKPOINT_REPO, repo_type="dataset",
178
  commit_message=f"web-agent run {run_id}")
179
  say(f"CHECKPOINT wrote runs/{run_id}.json to {CHECKPOINT_REPO}")
180
+ # ⚠ AFTER the write, never before: the sweep is housekeeping and the result is the job.
181
+ _sweep_expired()
182
  except Exception as exc: # noqa: BLE001 β€” a boundary
183
  # β›” CATCH EVERYTHING. This is a report-don't-raise boundary and the ways an upload can
184
  # fail are not enumerable (429 commit budget, 403 scope, DNS, a repo that does not
 
188
  f"the stdout frame still carries the result")
189
 
190
 
191
+ def _too_old(payload: dict) -> str:
192
+ """"" if this checkpoint may still be served, else the SENTENCE saying why it may not.
193
+
194
+ β›” D-193 β€” A RESUME THAT IS A YEAR OLD IS NOT A RESUME, IT IS A STALE ANSWER SERVED AS FRESH.
195
+ The key is deliberately content-addressed (`runId` + a hash of the steps), so a scheduled
196
+ automation reusing a stable run id resumed the SAME STORED PAGE for as long as the file
197
+ existed. `finishedAt` was written by `_emit` from the first day and read by nothing.
198
+ ⚠ AN UNREADABLE OR ABSENT TIMESTAMP IS TREATED AS TOO OLD, not as fresh. A checkpoint written
199
+ before this field existed carries no date, and "no date" must not be the one input that gets
200
+ served forever β€” fail toward re-fetching, which costs a job, rather than toward a wrong answer,
201
+ which costs a number nobody can explain.
202
+ """
203
+ stamp = str(payload.get("finishedAt") or "")
204
+ if not stamp:
205
+ return "it carries no finishedAt, so its age cannot be established"
206
+ try:
207
+ when = datetime.fromisoformat(stamp)
208
+ if when.tzinfo is None:
209
+ when = when.replace(tzinfo=timezone.utc)
210
+ except Exception: # noqa: BLE001
211
+ return f"its finishedAt ({stamp[:40]}) is not a date this job can read"
212
+ hours = (datetime.now(timezone.utc) - when).total_seconds() / 3600.0
213
+ if hours > CHECKPOINT_MAX_AGE_H:
214
+ return (f"it finished {hours:.1f} h ago and the ceiling is {CHECKPOINT_MAX_AGE_H} h "
215
+ f"(WEB_AGENT_CHECKPOINT_MAX_AGE_H)")
216
+ return ""
217
+
218
+
219
+ def _sweep_expired() -> None:
220
+ """Delete checkpoints past the ceiling, ONE COMMIT, at most `CHECKPOINT_SWEEP_MAX` of them.
221
+
222
+ β›” D-193's SECOND CLAUSE, and it is a different problem from the first. Ageing out the READ
223
+ stops a stale answer being served; it does not stop the repo growing one file per run forever.
224
+ β›” ONE COMMIT FOR THE WHOLE SWEEP. `upload_file`/`delete_file` each spend a commit, and this
225
+ repo already spends one per run against HF's 256-per-hour budget β€” a per-file sweep would
226
+ exhaust the budget it exists to protect, and the failure would land on the next run's WRITE.
227
+ `create_commit` with a list of delete operations is one commit whatever the count.
228
+ ⚠ AGE IS READ FROM THE LISTING, not by downloading each file. `last_commit` is what the API
229
+ gives cheaply; downloading N files to read a timestamp inside them would make a tidy-up cost
230
+ more than the runs it is tidying.
231
+ ⚠ REPORT-DON'T-RAISE, like `_checkpoint`: this runs AFTER the result is in hand, and a
232
+ housekeeping failure must never destroy a result that has already been paid for.
233
+ """
234
+ if not (CHECKPOINT_REPO and HF_TOKEN):
235
+ return
236
+ try:
237
+ from huggingface_hub import CommitOperationDelete, HfApi
238
+ api = HfApi(token=HF_TOKEN)
239
+ cutoff = datetime.now(timezone.utc).timestamp() - CHECKPOINT_MAX_AGE_H * 3600.0
240
+ doomed = []
241
+ for info in api.list_repo_tree(CHECKPOINT_REPO, repo_type="dataset", path_in_repo="runs",
242
+ expand=True, recursive=False):
243
+ path = getattr(info, "path", "")
244
+ when = getattr(getattr(info, "last_commit", None), "date", None)
245
+ if not path.endswith(".json") or when is None:
246
+ continue
247
+ if when.timestamp() < cutoff:
248
+ doomed.append(path)
249
+ if len(doomed) >= CHECKPOINT_SWEEP_MAX:
250
+ break
251
+ if not doomed:
252
+ return
253
+ api.create_commit(repo_id=CHECKPOINT_REPO, repo_type="dataset",
254
+ operations=[CommitOperationDelete(path_in_repo=p) for p in doomed],
255
+ commit_message=f"sweep {len(doomed)} checkpoints past "
256
+ f"{CHECKPOINT_MAX_AGE_H} h")
257
+ say(f"CHECKPOINT swept {len(doomed)} expired run(s) in one commit")
258
+ except Exception as exc: # noqa: BLE001 β€” a boundary
259
+ say(f"CHECKPOINT sweep skipped ({type(exc).__name__}: {str(exc)[:160]}) β€” "
260
+ f"the result is unaffected")
261
+
262
+
263
  def _resume(run_id: str):
264
+ """A prior run's SUCCESSFUL and RECENT result, if one exists. None otherwise.
265
+
266
+ β›” D-199 β€” A FAILED RUN'S CHECKPOINT USED TO REPLAY FOREVER. `_fail` checkpoints too
267
+ (`_fail` β†’ `_emit` β†’ `_checkpoint`), so the stored envelope for a failure is exactly as
268
+ truthy as one for a success; the caller tested `if prior:` and nothing ever looked at `ok`.
269
+ Under a stable `runId` a failed fetch could then never be re-attempted β€” it replayed the
270
+ stored failure unchanged, with `resumedFromCheckpoint: True` on it. The defect was LATENT
271
+ only because no deployment set `WEB_AGENT_REPO`, i.e. it armed itself on the day somebody
272
+ switched checkpointing on, which is the day nobody would be reading this function.
273
+ ⚠ THE TEST BELONGS HERE, NOT AT THE CALL SITE. `_resume` is the one place that decides what a
274
+ resumable checkpoint IS; a caller-side `if prior and prior.get("ok")` would be a second
275
+ definition, and the next caller would be written from the first one.
276
+ ⚠ A REFUSED CHECKPOINT SAYS SO IN THE LOG. Silently re-running looks identical to never
277
+ having had a checkpoint, and those two need to be distinguishable at 3am.
278
+ """
279
  if not (CHECKPOINT_REPO and HF_TOKEN and run_id):
280
  return None
281
  try:
 
284
  filename=f"runs/{run_id}.json", token=HF_TOKEN,
285
  force_download=True)
286
  with open(path, encoding="utf-8") as fh:
287
+ prior = json.load(fh)
288
  except Exception: # noqa: BLE001
289
  return None # absent is the normal case
290
+ if not isinstance(prior, dict):
291
+ say(f"RESUME refused β€” runs/{run_id}.json is not an object; re-running")
292
+ return None
293
+ if not prior.get("ok"):
294
+ say(f"RESUME refused β€” the stored run FAILED "
295
+ f"({str(prior.get('error') or 'no reason recorded')[:120]}); re-running")
296
+ return None
297
+ stale = _too_old(prior)
298
+ if stale:
299
+ say(f"RESUME refused β€” {stale}; re-running")
300
+ return None
301
+ return prior
302
 
303
 
304
  # ── the plan ────────────────────────────────────────────────────────────────────────────────
platform/aios_grid.py CHANGED
@@ -120,9 +120,20 @@ def _round(v):
120
  #: surfaces that CREATE them are the user-table databases, where a relation between two tables is
121
  #: a thing that exists. On the Odoo-backed Customer/Product grids there is no second user table to
122
  #: point at, so the column menu there simply never offers one.
 
 
 
 
 
 
 
 
 
 
 
123
  CUSTOM_FIELD_TYPES = {"text", "select", "multiselect", "user", "int", "currency", "pct", "date",
124
  "checkbox", "phone", "email", "url", "rating", "created_time", "formula",
125
- "automation", "image", "json", "link", "rollup", "code"}
126
 
127
  #: ⭐ WAVE-27 item 13 (owner ruling R13) β€” the `code` field's LANGUAGES.
128
  #:
@@ -1344,6 +1355,13 @@ def _clean_display(raw, valid_keys):
1344
  dashboard `charts` (I19). A chart whose x/y names a deleted field is dropped INDIVIDUALLY β€”
1345
  never the whole array, because losing one column should not cost the user a dashboard they
1346
  spent time building.
 
 
 
 
 
 
 
1347
  """
1348
  if not isinstance(raw, dict):
1349
  return None
@@ -1522,6 +1540,28 @@ def _clean_display(raw, valid_keys):
1522
  form = _clean_form(raw.get('form'), valid_keys)
1523
  if form:
1524
  out['form'] = form
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1525
  return out
1526
 
1527
 
 
120
  #: surfaces that CREATE them are the user-table databases, where a relation between two tables is
121
  #: a thing that exists. On the Odoo-backed Customer/Product grids there is no second user table to
122
  #: point at, so the column menu there simply never offers one.
123
+ #: ⭐⭐ WAVE-34 (owner ruling R13) β€” `ai_enrich` JOINS, and it had to join HERE in the same change
124
+ #: that put it in `UT_FIELD_TYPES`, not a ticket later. The wave planned a SERVER-FIRST landing on
125
+ #: the reasoning that a kind the client does not offer is invisible while a kind the server refuses
126
+ #: deletes a column. That reasoning is sound and the conclusion was still wrong, because THREE
127
+ #: parity gates chain over these sets and none of them permits a partial landing:
128
+ #: `verify_api` W18-UT `UT_FIELD_TYPES - CUSTOM_FIELD_TYPES == set()` (this line)
129
+ #: `aw_fields_contract` Β§5 `CREATABLE_TYPES == CUSTOM_FIELD_TYPES`, EXACT set equality
130
+ #: `types.ts` `CREATABLE_TYPES: readonly FieldType[]`, so the union must carry it too
131
+ #: Measured live by lane B at 17:09: `api_api` went 969/969 to 968/969 the moment the kind entered
132
+ #: `core/user_tables.py` alone, printing `got {'ai_enrich'} want set()`. The comments on
133
+ #: `json`, `link`/`rollup` and `code` below all say the same thing in their own words.
134
  CUSTOM_FIELD_TYPES = {"text", "select", "multiselect", "user", "int", "currency", "pct", "date",
135
  "checkbox", "phone", "email", "url", "rating", "created_time", "formula",
136
+ "automation", "image", "json", "link", "rollup", "code", "ai_enrich"}
137
 
138
  #: ⭐ WAVE-27 item 13 (owner ruling R13) β€” the `code` field's LANGUAGES.
139
  #:
 
1355
  dashboard `charts` (I19). A chart whose x/y names a deleted field is dropped INDIVIDUALLY β€”
1356
  never the whole array, because losing one column should not cost the user a dashboard they
1357
  spent time building.
1358
+
1359
+ W33-T45 (contract C1, ruling R5) adds `published` + `publishAccess`. ⚠ THE HALF OF THE ROUND
1360
+ TRIP THIS FUNCTION CANNOT ENFORCE: `customer-grid/types.ts::cleanDisplay` is a SECOND
1361
+ normalizer, in the browser, which rebuilds the config key by key on every autosave and drops
1362
+ anything it does not name. A key accepted here and unknown there dies on the next column
1363
+ resize β€” silently, because the client then POSTs the stripped config and the host REPLACES
1364
+ the stored one. "The host accepts it" is half a round trip; that file is the other half.
1365
  """
1366
  if not isinstance(raw, dict):
1367
  return None
 
1540
  form = _clean_form(raw.get('form'), valid_keys)
1541
  if form:
1542
  out['form'] = form
1543
+ # ── ⭐⭐ W33-T45 / CONTRACT C1 / RULING R5 (owner item 8b): IS THIS INTERFACE PUBLISHED ────
1544
+ #
1545
+ # β›” TWO KEYS LIVE HERE AND TWO DELIBERATELY DO NOT. The `published` flag and the sharer's
1546
+ # `public | password` choice are DISPLAY state β€” the view says what it is, the client renders
1547
+ # a badge from it, and it travels with the view like every other key in this dict. The SECRET
1548
+ # TOKEN and the PASSPHRASE HASH do not: they live in the server-only bucket, exactly as
1549
+ # `routes_forms.py`'s `TOKENS_KEY` holds the form token. `config.display` is echoed back to
1550
+ # every user who can open the view, so a token in here is a token published to the audience
1551
+ # the password was meant to exclude. `_clean_form` above carries the same rule and the same
1552
+ # reason; this is the second door, not a new one.
1553
+ #
1554
+ # β›” AND THE COERCION IS FAIL-CLOSED, WHICH IS WHY THIS IS NOT A BARE ALLOWLIST. A plain
1555
+ # allowlist drops an unrecognised `publishAccess` and KEEPS `published: True` β€” leaving a
1556
+ # published view with no stated access, i.e. a third state neither the ruling nor the client
1557
+ # has a meaning for, on the one key where guessing wrong publishes a tenant's data to the
1558
+ # open internet. So: a published view ALWAYS carries an access, and anything that is not the
1559
+ # literal `'public'` reads as `'password'`. The unpublished case stores nothing at all β€”
1560
+ # absent means unpublished, and a `published: False` would be the second way to say nothing
1561
+ # that `kanbanClamp` and `mode: 'grid'` are both here to forbid.
1562
+ if raw.get('published') is True:
1563
+ out['published'] = True
1564
+ out['publishAccess'] = 'public' if raw.get('publishAccess') == 'public' else 'password'
1565
  return out
1566
 
1567
 
platform/aios_grid_fields.json CHANGED
@@ -10,12 +10,25 @@
10
  "default": true,
11
  "description": "The customer's name in Odoo. One row per customer who ordered in the last 24 months."
12
  },
 
 
 
 
 
 
 
 
 
13
  {
14
  "key": "odoo_status",
15
  "label": "Odoo record",
16
  "type": "status",
17
  "source": "odoo",
18
  "default": false,
 
 
 
 
19
  "description": "Whether this customer still exists in Odoo. Archived means deleted there."
20
  },
21
  {
@@ -458,6 +471,63 @@
458
  "type": "select",
459
  "source": "odoo",
460
  "description": "Dead / excess / healthy bucket from the inventory module. Consolidated; absent for a BU-scoped caller."
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
461
  }
462
  ]
463
  }
 
10
  "default": true,
11
  "description": "The customer's name in Odoo. One row per customer who ordered in the last 24 months."
12
  },
13
+ {
14
+ "key": "partner_id",
15
+ "label": "Odoo ID",
16
+ "type": "int",
17
+ "source": "odoo",
18
+ "derived": true,
19
+ "default": false,
20
+ "description": "The Odoo res.partner id β€” the key every Odoo document joins on. DERIVED: this row's pid IS the partner id, so a stored copy would be a second source."
21
+ },
22
  {
23
  "key": "odoo_status",
24
  "label": "Odoo record",
25
  "type": "status",
26
  "source": "odoo",
27
  "default": false,
28
+ "options": [
29
+ "Active",
30
+ "Archived"
31
+ ],
32
  "description": "Whether this customer still exists in Odoo. Archived means deleted there."
33
  },
34
  {
 
471
  "type": "select",
472
  "source": "odoo",
473
  "description": "Dead / excess / healthy bucket from the inventory module. Consolidated; absent for a BU-scoped caller."
474
+ },
475
+ {
476
+ "key": "needs_pricing",
477
+ "label": "Needs pricing",
478
+ "type": "select",
479
+ "source": "overlay",
480
+ "default": false,
481
+ "options": [
482
+ "Yes"
483
+ ],
484
+ "shared": true,
485
+ "description": "Team-maintained. A SKU carries β€œYes” when it appears on the NEEDS PRICING tab of the 2027 catalog workbook; no value means it is not on that list. Shared with everyone in the workspace β€” the whole point of R8 is that the TEAM's work lands in the system, not one importer's private column."
486
+ },
487
+ {
488
+ "key": "march_pricelist",
489
+ "label": "March pricelist",
490
+ "type": "select",
491
+ "source": "overlay",
492
+ "default": false,
493
+ "options": [
494
+ "Yes"
495
+ ],
496
+ "shared": true,
497
+ "description": "Team-maintained. A SKU carries β€œYes” when it appears on the March Pricelist tab of the 2027 catalog workbook; no value means it is not on that list. Shared with everyone in the workspace β€” the whole point of R8 is that the TEAM's work lands in the system, not one importer's private column."
498
+ },
499
+ {
500
+ "key": "price_changes",
501
+ "label": "Price changes",
502
+ "type": "select",
503
+ "source": "overlay",
504
+ "default": false,
505
+ "options": [
506
+ "Yes"
507
+ ],
508
+ "shared": true,
509
+ "description": "Team-maintained. A SKU carries β€œYes” when it appears on the Price Changes tab of the 2027 catalog workbook; no value means it is not on that list. Shared with everyone in the workspace β€” the whole point of R8 is that the TEAM's work lands in the system, not one importer's private column."
510
+ },
511
+ {
512
+ "key": "closeouts",
513
+ "label": "Closeouts",
514
+ "type": "select",
515
+ "source": "overlay",
516
+ "default": false,
517
+ "options": [
518
+ "Yes"
519
+ ],
520
+ "shared": true,
521
+ "description": "Team-maintained. A SKU carries β€œYes” when it appears on the Closeouts tab of the 2027 catalog workbook; no value means it is not on that list. Shared with everyone in the workspace β€” the whole point of R8 is that the TEAM's work lands in the system, not one importer's private column."
522
+ },
523
+ {
524
+ "key": "notes",
525
+ "label": "Notes",
526
+ "type": "text",
527
+ "source": "overlay",
528
+ "default": false,
529
+ "shared": true,
530
+ "description": "Team-maintained. The 2027 workbook's own non-Odoo columns (Product Description, Packing) folded into one field. Shared with everyone in the workspace."
531
  }
532
  ]
533
  }
platform/core/alerts.py CHANGED
@@ -209,7 +209,7 @@ def _queue(rec, pids, labels, st=None):
209
  _st(st).update(NOTIFICATIONS_KEY, _apply, flush='async')
210
 
211
 
212
- def notify(owner, label, *, topic='automation', key='', row_id='', detail='', st=None):
213
  """⭐ WAVE 23 (C6) β€” put ONE notification in a named user's inbox, for a producer that is not
214
  a view-alert evaluation.
215
 
@@ -222,6 +222,24 @@ def notify(owner, label, *, topic='automation', key='', row_id='', detail='', st
222
  The inbox shape is UNCHANGED, deliberately: `AlertsPane` reads one list, and a second row
223
  shape would mean a client that must branch on where a notification came from. `alertId`
224
  carries the producer's key so a click-through can route (W23-W5).
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
225
  """
226
  owner = str(owner or '').strip()
227
  if not owner or not label:
@@ -230,7 +248,8 @@ def notify(owner, label, *, topic='automation', key='', row_id='', detail='', st
230
  item = {'id': f"{key or topic}:{row_id or 'n'}:{at}",
231
  'alertId': str(key or ''), 'viewId': '', 'topic': str(topic),
232
  'rowId': str(row_id or ''), 'label': str(detail or label),
233
- 'alertLabel': str(label), 'at': at, 'read': False}
 
234
 
235
  def _apply(data):
236
  box = list(data.get(owner) or [])
 
209
  _st(st).update(NOTIFICATIONS_KEY, _apply, flush='async')
210
 
211
 
212
+ def notify(owner, label, *, topic='automation', key='', row_id='', detail='', actor='', st=None):
213
  """⭐ WAVE 23 (C6) β€” put ONE notification in a named user's inbox, for a producer that is not
214
  a view-alert evaluation.
215
 
 
222
  The inbox shape is UNCHANGED, deliberately: `AlertsPane` reads one list, and a second row
223
  shape would mean a client that must branch on where a notification came from. `alertId`
224
  carries the producer's key so a click-through can route (W23-W5).
225
+
226
+ ⭐⭐ W33-T28 (`ASK C-14`) β€” `actor` IS THE ONLY THING THAT KNOWS WHO. Owner item: *"the Inbox
227
+ reads like email"*, and a verifier reading the carried wave-32 surface found the SENDER position
228
+ occupied by `kindLabel(n.kind)` β€” the literals "Alert" / "Automation" / "Shared with you", i.e. a
229
+ CATEGORY where mail puts a who β€” with no sender field anywhere on the wire, in the model or in
230
+ the markup. An alert and an automation have no person behind them and are honestly named by
231
+ their machine; **a SHARE has a real person, and only the PRODUCER knows their name.** By the time
232
+ the read path sees the row that fact is gone, so it has to be carried at write time.
233
+
234
+ β›” ADDITIVE, AND THAT IS THE WHOLE DESIGN. No existing caller changes, no stored row changes
235
+ shape, and every reader already tolerates the key's absence β€” so an inbox written before this
236
+ line reads exactly as it did. The alternative C rejected is worth recording because it is the
237
+ tempting one: recovering the name by REGEXING the body prose (`"<name> shared this with you"`).
238
+ A sender parsed out of a sentence breaks the first time the sentence is reworded, and it breaks
239
+ SILENTLY, in the header β€” [[one-question-two-normalizers]] with the second normaliser being
240
+ English.
241
+ ⚠ Blank is a legitimate value and means *"no person did this"*, NOT "unknown". The client falls
242
+ back BY KIND rather than to an empty From column, so an automation never borrows a human's name.
243
  """
244
  owner = str(owner or '').strip()
245
  if not owner or not label:
 
248
  item = {'id': f"{key or topic}:{row_id or 'n'}:{at}",
249
  'alertId': str(key or ''), 'viewId': '', 'topic': str(topic),
250
  'rowId': str(row_id or ''), 'label': str(detail or label),
251
+ 'alertLabel': str(label), 'at': at, 'read': False,
252
+ 'actor': str(actor or '')}
253
 
254
  def _apply(data):
255
  box = list(data.get(owner) or [])
platform/core/data_binding.py ADDED
@@ -0,0 +1,270 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """WHICH STORE THIS DEPLOYMENT IS ALLOWED TO WRITE β€” the fail-closed binding (D-315 / D-298).
2
+
3
+ β›”β›” THE SENTENCE THIS FILE EXISTS TO MAKE IMPOSSIBLE. Owner, 2026-08-18, verbatim: *"this is so
4
+ critical that whenever we do staging, we don't fuck up anyone's data. EVER."* It had already
5
+ happened once: `fsanyoto/loopable` became STAGING on 2026-08-14 having never been given a store of
6
+ its own, so two different builds wrote tenant #0's real business data. Compose that with D-305
7
+ (the store is last-write-wins at whole-document granularity) and a routine staging deploy does not
8
+ merely add test rows β€” it can erase a real user's views, fields and typed cell values.
9
+
10
+ β›” A DEPLOY FLAG IS NOT THE FIX; THE FLAG IS THE THING THAT ALREADY FAILED. The isolation that
11
+ existed was bound to a ROLE ("the staging Space"), and a guard bound to a role stops guarding the
12
+ moment the role moves β€” which these two Spaces have done twice. So the binding here is on the
13
+ container's OWN IDENTITY, which no deploy can forget to pass and no role swap can invalidate.
14
+
15
+ ⭐⭐ THE RULE, AND IT CLASSIFIES THE DEPLOYMENT RATHER THAN THE REPO:
16
+
17
+ 1. the PRODUCTION deployment (identity in `PRODUCTION_SPACES`) -> may write anything
18
+ 2. any other deployment writing a KNOWN production store -> REFUSED
19
+ 3. any other deployment writing a repo on its OWN allowlist -> allowed
20
+ 4. anything else -> REFUSED
21
+
22
+ β›” WHY NOT CLASSIFY THE REPO, which is the design everybody reaches for first. A DENYLIST of
23
+ production stores fails OPEN for the next tenant nobody remembered to add. A SUFFIX RULE
24
+ (`*-staging-data`) pins a SPELLING rather than a claim β€” see [[gate-pins-a-spelling-not-a-claim]] β€”
25
+ and a store named without the suffix is unprotected while reading as protected. Rule 4 is the one
26
+ that survives a tenant nobody has provisioned yet: an unknown repo is refused, so a store this
27
+ deployment was never handed cannot be written by accident.
28
+
29
+ ⭐⭐ AND RULE 4 IS NOT THEORETICAL β€” IT IS THE HALF OF D-315 THE REGISTER ROW DID NOT KNOW ABOUT.
30
+ Measured 2026-08-18 by reading both control planes: `royal-imports/cfo-os-staging-data`'s own
31
+ `tenants.json` is byte-identical to live's and names **`royal-imports/aios-nurilab-data`,
32
+ `aios-gtmlab-data` and `aios-loopable-data`** β€” the three other tenants' REAL stores. So
33
+ `--data-repo=` only ever isolated tenant #0; `harness/runtime.py`'s `store.for_repo(repo)` path
34
+ resolved three PRODUCTION repos out of the staging store and wrote them, and no flag has ever
35
+ touched that path. The exposure is FOUR production stores, not one.
36
+
37
+ ⚠ READS ARE NOT REFUSED, AND THAT IS A DELIBERATE, STATED SCOPE DECISION. `Store.get` is lenient
38
+ by contract: a refused read would hand back `{}`, a caller would paint an empty grid, and the whole
39
+ Space would read as "no data" rather than as "refused" β€” the exact
40
+ [[empty-answer-vs-unfinished-answer]] shape this codebase has shipped before. A refused WRITE is
41
+ loud (503, with the reason). So staging still SEES production data when pointed at it; it can no
42
+ longer CHANGE it. Narrowing the read is a separate decision and is booked, not smuggled in here.
43
+
44
+ ⚠ ABSENT IDENTITY FAILS CLOSED, AND THE FAILURE LOOKS LIKE AN OUTAGE. If a container exposes none
45
+ of the identity variables, it is treated as "not the production deployment" and every write to a
46
+ production store refuses. That is the safe direction, but on a live Space it would read as a total
47
+ product outage β€” which is why `describe()` reports the RAW observations and not just the verdict:
48
+ per D-160 the Space environment cannot be read from outside, so the container has to say what it
49
+ saw. One deploy then answers "does SPACE_ID exist here", instead of three.
50
+ """
51
+ import os
52
+
53
+ #: The deployment that OWNS production data. A frozenset rather than a string because the product
54
+ #: has had two public addresses before and will again.
55
+ #:
56
+ #: ⚠ THIS SET IS DUPLICATED IN `aios-web/deploy_web.py` AS `PUBLIC_LIVE`, on purpose and with a
57
+ #: gate: `deploy_web` is a standalone CLI that must run without the platform package importable, so
58
+ #: it cannot import this. `verify_store_binding.py::section_constants` asserts the two literals
59
+ #: agree β€” a constant two features share is a constant that drifts ([[constant-two-features-share]]).
60
+ PRODUCTION_SPACES = frozenset({'fsanyoto/runloopable'})
61
+
62
+ #: The stores that hold REAL customer business data. Enumerated 2026-08-18 from the live control
63
+ #: plane (`royal-imports/cfo-os-data::tenants.json`), not guessed.
64
+ #:
65
+ #: β›” THIS LIST IS RULE 2, NOT THE WHOLE GUARD. It exists to make a DELIBERATE mis-pointing loud β€”
66
+ #: `OS_DATA_REPO=royal-imports/cfo-os-data` typed on a laptop, or a production repo added to a
67
+ #: sandbox allowlist by hand. Rule 4 is what covers a tenant provisioned after this line was
68
+ #: written, which is why the file does not depend on this staying complete.
69
+ PRODUCTION_STORES = frozenset({
70
+ 'royal-imports/cfo-os-data', # tenant #0 β€” Royal Imports
71
+ 'royal-imports/aios-nurilab-data', # Nurilab
72
+ 'royal-imports/aios-gtmlab-data', # GTM Lab
73
+ 'royal-imports/aios-loopable-data', # Loopable
74
+ })
75
+
76
+ #: The historical default of `OS_DATA_REPO`. Kept as a name so the resolution below can say what it
77
+ #: is refusing to assume, rather than repeating a literal.
78
+ PRODUCTION_DEFAULT_STORE = 'royal-imports/cfo-os-data'
79
+
80
+ #: The env vars a Hugging Face Space container sets to describe itself, in the order they are
81
+ #: trusted. `SPACE_ID` is the canonical one; the pair is the documented fallback. Read as a LIST so
82
+ #: `describe()` can report every one of them and what it held.
83
+ _IDENTITY_VARS = ('SPACE_ID', 'SPACE_AUTHOR_NAME', 'SPACE_REPO_NAME', 'SPACE_HOST')
84
+
85
+ #: The env a NON-production deployment uses to name the stores it may write, beyond `OS_DATA_REPO`.
86
+ #: Comma-separated.
87
+ #:
88
+ #: ⭐ THE POLARITY IS THE POINT, and it is what keeps trap 1 satisfied. Forgetting to set this
89
+ #: makes a deployment write LESS, never more β€” so an omitted deploy argument is a refusal, not a
90
+ #: silent grant. That is the exact inverse of `--data-repo`, whose omission printed
91
+ #: "OS_DATA_REPO unchanged" and left a Space on production.
92
+ _SANDBOX_STORES_VAR = 'AIOS_SANDBOX_STORES'
93
+
94
+ #: The local (no-identity) escape hatch. Present β‡’ writes to production stores are permitted from a
95
+ #: machine with no Space identity, AND every commit this process makes is stamped distinguishably
96
+ #: (see `local_commit_suffix`). Deliberately does NOT widen a Space: a container that HAS an
97
+ #: identity and is not production is refused with no override, because that is the case the owner
98
+ #: said "EVER" about.
99
+ _LOCAL_OPT_IN_VAR = 'AIOS_ALLOW_PRODUCTION_DATA'
100
+
101
+
102
+ class StoreWriteRefused(RuntimeError):
103
+ """A write this deployment is not allowed to make.
104
+
105
+ β›” DELIBERATELY NOT A SUBCLASS OF `grid_events.StoreUnavailable`, and the reason is the routes.
106
+ Several of them carry `except StoreUnavailable:` blocks that DEGRADE to a session-scoped
107
+ fallback workspace β€” correct for an outage, catastrophic for a refusal, because the user would
108
+ be told the change was saved somewhere. A distinct type falls through all of them to the
109
+ app-level handler, which answers 503 with the reason and never claims a save.
110
+ """
111
+
112
+
113
+ def _env(name):
114
+ return str(os.environ.get(name) or '').strip()
115
+
116
+
117
+ def deployment_id():
118
+ """This container's own identity, or `''` when it is not a Space (i.e. local dev).
119
+
120
+ ⚠ NOT DERIVED FROM ANY DEPLOY ARGUMENT. That is the whole design: `SPACE_ID` is set by the
121
+ platform inside the container, so it cannot be forgotten, mistyped, or left behind by a Space
122
+ that changed role without being redeployed.
123
+ """
124
+ sid = _env('SPACE_ID')
125
+ if sid:
126
+ return sid
127
+ author, repo = _env('SPACE_AUTHOR_NAME'), _env('SPACE_REPO_NAME')
128
+ if author and repo:
129
+ return f'{author}/{repo}'
130
+ return ''
131
+
132
+
133
+ def is_production_deployment():
134
+ """True only on the deployment that owns production data. Unknown β‡’ False (fail closed)."""
135
+ return deployment_id() in PRODUCTION_SPACES
136
+
137
+
138
+ def is_space():
139
+ """True when this process is running inside a Space at all.
140
+
141
+ The distinction matters exactly once: a Space that is not production gets NO override, while a
142
+ laptop does. `deployment_id()` conflates them into `''`, so ask separately.
143
+ """
144
+ return bool(deployment_id())
145
+
146
+
147
+ def sandbox_allowlist():
148
+ """The repos a non-production deployment may write: `OS_DATA_REPO` + `AIOS_SANDBOX_STORES`."""
149
+ allowed = {r.strip() for r in _env(_SANDBOX_STORES_VAR).split(',') if r.strip()}
150
+ own = _env('OS_DATA_REPO')
151
+ if own:
152
+ allowed.add(own)
153
+ return frozenset(allowed)
154
+
155
+
156
+ def local_override():
157
+ """True when a machine with no Space identity has explicitly opted in to production data."""
158
+ return (not is_space()) and _env(_LOCAL_OPT_IN_VAR) == '1'
159
+
160
+
161
+ def default_store():
162
+ """What `OS_DATA_REPO` resolves to when it is unset.
163
+
164
+ β›”β›” FACT (1) OF D-315 DIES HERE: `core/store.py` used to read
165
+ `os.environ.get('OS_DATA_REPO', 'royal-imports/cfo-os-data')`, so a Space that was never given
166
+ the key β€” every NEW Space, by construction β€” came up bound to tenant #0's real business data.
167
+ The default now exists ONLY on the deployment that owns it.
168
+
169
+ ⚠ IT STILL RETURNS THE PRODUCTION ID ELSEWHERE, RATHER THAN `None` OR `''`, AND THAT IS
170
+ DELIBERATE. A `None` repo would make `Store.__init__` β€” which runs at IMPORT time via
171
+ `_DEFAULT = for_repo(REPO)` β€” raise or bind something nonsensical, killing the app and every
172
+ gate on any non-production machine. Returning the real id keeps binding total and lets
173
+ `write_refusal` refuse the WRITE with a message that names the actual problem. Reads still
174
+ work, so a developer sees a populated app that will not let them damage it.
175
+ """
176
+ return PRODUCTION_DEFAULT_STORE
177
+
178
+
179
+ def write_refusal(repo):
180
+ """`None` if this deployment may write `repo`, else the sentence explaining why not.
181
+
182
+ ⭐ RETURNS A REASON RATHER THAN A BOOL, because [[report-the-cause-before-you-fix-it]]: a
183
+ silent refusal makes every theory about it unfalsifiable, and this one fires on a path where
184
+ the operator cannot read the container's environment (D-160).
185
+ """
186
+ rid = str(repo or '').strip()
187
+ if not rid:
188
+ return ('no store repo is bound. Refusing to guess: an unnamed store is how a deployment '
189
+ 'ends up on somebody else\'s data.')
190
+ if is_production_deployment():
191
+ return None # rule 1 β€” the live Space owns its data
192
+ if rid in PRODUCTION_STORES: # rule 2 β€” never, from anywhere else
193
+ if local_override():
194
+ return None
195
+ who = deployment_id() or 'this machine (no Space identity)'
196
+ if is_space():
197
+ return (f'{who} is not the production deployment, so it may not write {rid}, which '
198
+ f'holds real customer data. Point this Space at its own store with '
199
+ f'--data-repo=, or add it to {_SANDBOX_STORES_VAR}. There is no override for '
200
+ f'a Space: a build that is not live never writes live data.')
201
+ return (f'{who} may not write {rid}, which holds real customer data. Set OS_DATA_REPO to a '
202
+ f'sandbox store, or set {_LOCAL_OPT_IN_VAR}=1 to opt in deliberately (every commit '
203
+ f'is then stamped as a local write so it can be told apart afterwards).')
204
+ allow = sandbox_allowlist() # rule 3 β€” what it was explicitly handed
205
+ if rid in allow:
206
+ return None
207
+ who = deployment_id() or 'this machine (no Space identity)'
208
+ listed = ', '.join(sorted(allow)) or '(nothing)'
209
+ return (f'{who} was not given {rid}. A non-production deployment may write only the stores it '
210
+ f'was handed: {listed}. Add it to {_SANDBOX_STORES_VAR} if that is intended.')
211
+
212
+
213
+ def check_write(repo, operation='write'):
214
+ """Raise `StoreWriteRefused` unless this deployment may write `repo`. The one enforcement door."""
215
+ reason = write_refusal(repo)
216
+ if reason:
217
+ raise StoreWriteRefused(f'{operation} to {repo!r} refused: {reason}')
218
+
219
+
220
+ def local_commit_suffix():
221
+ """A marker appended to commit messages made under the local override, else `''`.
222
+
223
+ ⭐ THIS IS THE SECOND HALF OF D-298's EXIT CONDITION, satisfied rather than argued around: *"a
224
+ local run either cannot reach the live store without an explicit opt-in, OR stamps its commits
225
+ distinguishably"*. The measured complaint was that a laptop's commits were titled
226
+ `update user_tables` β€” byte-identical to what the live Space writes every few minutes β€” so a
227
+ local run and production were indistinguishable in the store's git history, which is the ONLY
228
+ audit trail this product has. Both clauses now hold: the opt-in is required AND it stamps.
229
+ """
230
+ if not local_override():
231
+ return ''
232
+ import getpass # noqa: PLC0415 β€” only on the opt-in path
233
+ import socket # noqa: PLC0415
234
+ try:
235
+ who = getpass.getuser()
236
+ except Exception: # noqa: BLE001
237
+ who = 'unknown'
238
+ try:
239
+ host = socket.gethostname()
240
+ except Exception: # noqa: BLE001
241
+ host = 'unknown'
242
+ return f' [LOCAL DEV {who}@{host}]'
243
+
244
+
245
+ def describe(repo=None):
246
+ """What this container sees about itself β€” the payload behind the status field.
247
+
248
+ β›” IT REPORTS THE RAW OBSERVATIONS, NOT ONLY THE VERDICT, and that is the point rather than
249
+ debug noise. D-160 established that a Space's environment cannot be read from outside, so
250
+ "is staging on the right store?" is a question only the container can answer. A verdict alone
251
+ ("production: false") cannot distinguish "SPACE_ID says staging" from "SPACE_ID does not exist
252
+ on this platform at all" β€” two states with the same verdict and completely different fixes.
253
+ Reporting the variables makes ONE deploy answer it.
254
+
255
+ ⚠ Carries no secret: a Space id and a dataset repo id are public names. The DATA is not, and
256
+ none of it is here. The field is session-gated anyway (see `routes_admin.settings`).
257
+ """
258
+ rid = str(repo or '').strip()
259
+ return {
260
+ 'deployment': deployment_id(),
261
+ 'production': is_production_deployment(),
262
+ 'repo': rid,
263
+ 'writable': write_refusal(rid) is None if rid else False,
264
+ 'refusal': write_refusal(rid) if rid else None,
265
+ 'localOverride': local_override(),
266
+ 'allowlist': sorted(sandbox_allowlist()),
267
+ # The raw half. `''` means "the variable is absent", which is itself the answer on a
268
+ # platform that does not set it.
269
+ 'observed': {name: _env(name) for name in _IDENTITY_VARS},
270
+ }
platform/core/grid_events.py CHANGED
@@ -40,7 +40,12 @@ down, and this note is the marker for that work.
40
  import datetime as dt
41
  import json as _json_ev # wave 23 (C7): the json field's parse wall
42
  from dataclasses import dataclass, field
43
- from typing import MutableMapping, Optional
 
 
 
 
 
44
 
45
  import core.store as store
46
  import core.table_store as _tstore # wave-9 I17: the shared-view authorisation predicates
@@ -92,6 +97,21 @@ class EventResult:
92
  rerender: bool = False
93
  doc: Optional[dict] = None # doc_fetch's answer β€” the bridge has no response channel
94
  toast: Optional[str] = None # add_to_list's confirmation
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
95
 
96
 
97
  @dataclass
@@ -579,6 +599,39 @@ def _known_usernames():
579
 
580
 
581
  # ------------------------------------------------------------------ THE SEAM
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
582
  def handle_events(events, ctx):
583
  """The event LOG entry point β†’ one EventResult. The `[-24:]` window lives here.
584
 
@@ -963,8 +1016,30 @@ def handle_one(event, ctx):
963
  # broken). Role `edit` (or admin) writes THROUGH to the owner's record β€” config only,
964
  # never identity or reach: `createdBy`/`permissions` are pinned from the stored record.
965
  # Role `view` (or no grant) refuses with a toast instead of forking.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
966
  _save_as = uname
967
- if _store_of(ctx).available():
968
  _foreign = _tops(ctx).find_view(view_id)
969
  if _foreign and _foreign[0] != uname:
970
  _o_uname, _stored = _foreign
@@ -1077,7 +1152,8 @@ def handle_one(event, ctx):
1077
  if kind == 'field_upsert':
1078
  raw = event.get('field')
1079
  if not isinstance(raw, dict):
1080
- return False
 
1081
  key = str(raw.get('key') or '')[:80]
1082
  import aios_grid as _ag2
1083
  if key.startswith(_ag2.MEASURE_FIELD_PREFIX):
@@ -1086,7 +1162,10 @@ def handle_one(event, ctx):
1086
  # measure this user could also filter by. Fail closed on anything else.
1087
  field = _ag2.clean_measure_field(raw, {m['key']: m for m in measure_offer})
1088
  if field is None:
1089
- return False
 
 
 
1090
  fmt = _ag2._clean_format(raw.get('format'), field.get('type'))
1091
  if fmt:
1092
  field['format'] = fmt
@@ -1138,13 +1217,19 @@ def handle_one(event, ctx):
1138
  # second, private, non-computing copy here would be a second definition of the same
1139
  # column. A refusal repaints, so a stale client says something rather than nothing.
1140
  if ftype in ('link', 'rollup'):
1141
- return False
 
 
 
1142
  readonly = ftype in _ag2.READONLY_CUSTOM_TYPES
1143
  # The editable family must SAY it is overlay (the client's own contract); the
1144
  # read-only pair (formula / created_time) is stored under the cohort column's
1145
  # mechanism instead, so no cell write can ever be accepted for it.
1146
  if not readonly and raw.get('source') != 'overlay':
1147
- return False
 
 
 
1148
  field = {'key': key, 'label': str(raw.get('label') or 'Untitled')[:120],
1149
  'type': ftype, 'source': 'odoo' if readonly else 'overlay',
1150
  'default': True, 'custom': True,
@@ -1170,7 +1255,10 @@ def handle_one(event, ctx):
1170
  if ftype in ('select', 'multiselect'):
1171
  field['options'] = _ag2._clean_options(raw.get('options'))
1172
  if not field['options']:
1173
- return False # a select with no choices can never hold a value
 
 
 
1174
  field.update(_ag2._choice_appearance(raw, field['options']))
1175
  if ftype == 'rating':
1176
  field['max'] = _ag2._clean_rating_max(raw.get('max'))
@@ -1179,7 +1267,13 @@ def handle_one(event, ctx):
1179
  # alone so a later-deleted ref blanks the CELLS, not the column.
1180
  formula = _ag2._clean_formula(raw.get('formula'), valid_keys)
1181
  if formula is None:
1182
- return False
 
 
 
 
 
 
1183
  field['formula'] = formula
1184
  if ftype == 'automation':
1185
  # C5-AUTOFIELD (wave 18): persist the validated config bag. No `return False`
@@ -1199,7 +1293,9 @@ def handle_one(event, ctx):
1199
  auto = _ag2._clean_automation(raw.get('automation'), valid_keys,
1200
  flow_ids=_flow_ids)
1201
  if auto is None:
1202
- return False
 
 
1203
  field['automation'] = auto
1204
  # ⭐ WAVE-27 item 13 (R13): the code column's language. OPTIONAL, like the
1205
  # `automation` bag above and unlike `formula` β€” an unconfigured code column is a
@@ -1216,7 +1312,14 @@ def handle_one(event, ctx):
1216
  if fmt:
1217
  field['format'] = fmt
1218
  else:
1219
- return False
 
 
 
 
 
 
 
1220
  # Wave-5 item 1 β€” createdBy + permissions on the CREATED strata (custom_/measure_).
1221
  # `createdBy` is stamped HOST-SIDE at create and preserved on every later write: the
1222
  # client never supplies it, so a browser cannot claim someone else's field. A
 
40
  import datetime as dt
41
  import json as _json_ev # wave 23 (C7): the json field's parse wall
42
  from dataclasses import dataclass, field
43
+ # ⚠ ALIASED ON PURPOSE. `field` is imported above for the dataclasses that need it, and this file
44
+ # uses `field` as a LOCAL VARIABLE in a dozen handlers (`field = {'key': key, …}`). A
45
+ # `field(default_factory=list)` written inside one of those functions would resolve to the local
46
+ # dict and raise. The alias is unshadowable, so D-291's channel cannot be broken by a rename.
47
+ from dataclasses import field as dataclass_field
48
+ from typing import List, MutableMapping, Optional
49
 
50
  import core.store as store
51
  import core.table_store as _tstore # wave-9 I17: the shared-view authorisation predicates
 
97
  rerender: bool = False
98
  doc: Optional[dict] = None # doc_fetch's answer β€” the bridge has no response channel
99
  toast: Optional[str] = None # add_to_list's confirmation
100
+ #: ⭐⭐ D-291 β€” WHY A WRITE WAS DECLINED. Until 2026-08-17 this door had NO disclosure channel
101
+ #: at all: `field_upsert` carries EIGHT distinct `return False` refusals and every one of them
102
+ #: answered `{"results":[{"id":"…","rerender":false}]}` β€” byte-identical to a success.
103
+ #: β›” THE COST WAS NOT THE MISSING MESSAGE, IT WAS THAT EVERY THEORY BECAME UNFALSIFIABLE.
104
+ #: D-275 (a custom field on the product grid answers 200 and is absent from a fresh read) sat
105
+ #: open across TWO waves because no amount of reading the source can say WHICH of the eight
106
+ #: fired on a given request β€” and the register's own advice is that reporting the cause IS the
107
+ #: diagnostic ([[report-the-cause-before-you-fix-it]]). D-276 is the precedent: shipping the
108
+ #: report first produced, in one deploy, a measurement that REFUTED a clean by-elimination
109
+ #: argument about the cause.
110
+ #: ⚠ SAFE BY CONSTRUCTION, and this is the check to re-run if a new refusal is added: every
111
+ #: entry describes the caller's OWN submitted event. It never names another user, another
112
+ #: tenant, or a row the caller cannot see β€” the failure mode D-276's `visible` flag exists to
113
+ #: avoid one layer up.
114
+ refusals: List[dict] = dataclass_field(default_factory=list)
115
 
116
 
117
  @dataclass
 
599
 
600
 
601
  # ------------------------------------------------------------------ THE SEAM
602
+ #: ⭐⭐ D-291's WRITER. Every refusal in `field_upsert` goes through here so the caller learns WHY
603
+ #: instead of receiving a success-shaped answer.
604
+ #:
605
+ #: β›” IT RETURNS **True**, NOT False, AND THAT HALF IS A SECOND DEFECT THIS CLOSES. `handle_one`'s
606
+ #: return value means *"this page must re-render from server state"* β€” so a refusal that returned
607
+ #: False left the browser's OPTIMISTIC column on screen indefinitely. The user typed a column, saw
608
+ #: it appear, and it was never stored; only a reload revealed it had never existed. The file already
609
+ #: states the rule for the one refusal that got it right (`refused_perms`): *"the client's optimistic
610
+ #: copy is a lie that must be repainted"*. Now every refusal obeys it.
611
+ #:
612
+ #: `code` is a stable machine token (the client may branch on it); `message` is the sentence a
613
+ #: person reads. ⚠ The message must name what the CALLER can change β€” a refusal a user cannot act
614
+ #: on is only marginally better than silence.
615
+ def _refuse(ctx, code, message, key=''):
616
+ """Record a NAMED refusal on the result and demand a repaint. Always returns True."""
617
+ try:
618
+ out = getattr(ctx, 'out', None)
619
+ if out is not None and hasattr(out, 'refusals'):
620
+ if len(out.refusals) < 24: # one per event in the window; never unbounded
621
+ out.refusals.append({
622
+ 'event': 'field_upsert',
623
+ 'key': str(key or '')[:80],
624
+ 'code': str(code)[:40],
625
+ 'message': str(message)[:300],
626
+ })
627
+ except Exception: # noqa: BLE001
628
+ # β›” REPORTING MUST NEVER BE THE THING THAT BREAKS THE WRITE PATH. A refusal that raises
629
+ # while explaining itself would turn a declined field into a 500 β€” strictly worse than the
630
+ # silence this replaces.
631
+ pass
632
+ return True
633
+
634
+
635
  def handle_events(events, ctx):
636
  """The event LOG entry point β†’ one EventResult. The `[-24:]` window lives here.
637
 
 
1016
  # broken). Role `edit` (or admin) writes THROUGH to the owner's record β€” config only,
1017
  # never identity or reach: `createdBy`/`permissions` are pinned from the stored record.
1018
  # Role `view` (or no grant) refuses with a toast instead of forking.
1019
+ #
1020
+ # β›”β›” WAVE 33 Β· T13 (owner item 3) β€” A PINNED PER-USER ID IS NOT A FOREIGN VIEW, AND
1021
+ # TREATING IT AS ONE IS A CROSS-USER WRITE THAT REPORTS SUCCESS.
1022
+ #
1023
+ # `find_view` scans EVERY personal stratum and returns the FIRST match. `all-customers`
1024
+ # and `tpl_overview` are PINNED ids that every user of a table has their own copy of β€”
1025
+ # so the moment ANY colleague autosaves one (a column resize is enough), the wall reads
1026
+ # "somebody else owns this id" for everybody else. MEASURED, on this exact path, after
1027
+ # one colleague resized a column on the Overview:
1028
+ # Β· a NON-ADMIN marking it is REFUSED, with a toast blaming sharing β€” a view nobody
1029
+ # ever shared, on their own database;
1030
+ # Β· an ADMIN marking it answers 200 and writes the mark into the OTHER USER'S
1031
+ # stratum. The admin's own workspace still serves the un-marked view, so the mark
1032
+ # "does not work"; the colleague silently gains a mark they never made.
1033
+ # That is item 3's second mechanism, and it is a residency bug wearing a sharing bug's
1034
+ # error message.
1035
+ #
1036
+ # ⚠ THIS FUNCTION ALREADY STATES THE RULE THE WALL BREAKS, seventy lines up: "A SYSTEM
1037
+ # VIEW IS PER-USER STATE AND MUST NEVER BE SHARED", enforced there by pinning
1038
+ # `perms = {'edit': 'personal'}`. A per-user view can therefore never BE a grant, which
1039
+ # is the only thing this wall exists to protect β€” so exempting it narrows nothing.
1040
+ _pinned_per_user = (_kind == 'system' or view_id in ('all-customers', _ag_overview_id()))
1041
  _save_as = uname
1042
+ if _store_of(ctx).available() and not _pinned_per_user:
1043
  _foreign = _tops(ctx).find_view(view_id)
1044
  if _foreign and _foreign[0] != uname:
1045
  _o_uname, _stored = _foreign
 
1152
  if kind == 'field_upsert':
1153
  raw = event.get('field')
1154
  if not isinstance(raw, dict):
1155
+ return _refuse(ctx, 'malformed',
1156
+ 'the event carried no field object')
1157
  key = str(raw.get('key') or '')[:80]
1158
  import aios_grid as _ag2
1159
  if key.startswith(_ag2.MEASURE_FIELD_PREFIX):
 
1162
  # measure this user could also filter by. Fail closed on anything else.
1163
  field = _ag2.clean_measure_field(raw, {m['key']: m for m in measure_offer})
1164
  if field is None:
1165
+ return _refuse(
1166
+ ctx, 'measure_not_offered',
1167
+ 'this measure is not one you can filter by on this page, so a column '
1168
+ 'cannot be built from it', key)
1169
  fmt = _ag2._clean_format(raw.get('format'), field.get('type'))
1170
  if fmt:
1171
  field['format'] = fmt
 
1217
  # second, private, non-computing copy here would be a second definition of the same
1218
  # column. A refusal repaints, so a stale client says something rather than nothing.
1219
  if ftype in ('link', 'rollup'):
1220
+ return _refuse(
1221
+ ctx, 'wrong_door',
1222
+ 'a link or rollup column belongs to the table schema, not to a personal '
1223
+ 'overlay - create it from the Fields panel instead', key)
1224
  readonly = ftype in _ag2.READONLY_CUSTOM_TYPES
1225
  # The editable family must SAY it is overlay (the client's own contract); the
1226
  # read-only pair (formula / created_time) is stored under the cohort column's
1227
  # mechanism instead, so no cell write can ever be accepted for it.
1228
  if not readonly and raw.get('source') != 'overlay':
1229
+ return _refuse(
1230
+ ctx, 'source_not_overlay',
1231
+ "an editable column must declare source 'overlay'; this one declared "
1232
+ '%r' % (raw.get('source'),), key)
1233
  field = {'key': key, 'label': str(raw.get('label') or 'Untitled')[:120],
1234
  'type': ftype, 'source': 'odoo' if readonly else 'overlay',
1235
  'default': True, 'custom': True,
 
1255
  if ftype in ('select', 'multiselect'):
1256
  field['options'] = _ag2._clean_options(raw.get('options'))
1257
  if not field['options']:
1258
+ # a select with no choices can never hold a value
1259
+ return _refuse(
1260
+ ctx, 'no_options',
1261
+ 'a select column needs at least one choice', key)
1262
  field.update(_ag2._choice_appearance(raw, field['options']))
1263
  if ftype == 'rating':
1264
  field['max'] = _ag2._clean_rating_max(raw.get('max'))
 
1267
  # alone so a later-deleted ref blanks the CELLS, not the column.
1268
  formula = _ag2._clean_formula(raw.get('formula'), valid_keys)
1269
  if formula is None:
1270
+ # ⭐ THE ONE D-275 IS MOST LIKELY TO BE. `valid_keys` is THIS page's column
1271
+ # set, so a formula naming a column that exists on another grid is refused
1272
+ # here and was, until now, indistinguishable from a save.
1273
+ return _refuse(
1274
+ ctx, 'formula_refs',
1275
+ 'the formula is empty or names a column this table does not have',
1276
+ key)
1277
  field['formula'] = formula
1278
  if ftype == 'automation':
1279
  # C5-AUTOFIELD (wave 18): persist the validated config bag. No `return False`
 
1293
  auto = _ag2._clean_automation(raw.get('automation'), valid_keys,
1294
  flow_ids=_flow_ids)
1295
  if auto is None:
1296
+ return _refuse(
1297
+ ctx, 'automation_flow',
1298
+ 'the automation this column names does not exist', key)
1299
  field['automation'] = auto
1300
  # ⭐ WAVE-27 item 13 (R13): the code column's language. OPTIONAL, like the
1301
  # `automation` bag above and unlike `formula` β€” an unconfigured code column is a
 
1312
  if fmt:
1313
  field['format'] = fmt
1314
  else:
1315
+ # β›” THE FALL-THROUGH, AND IT IS THE WIDEST OF THE EIGHT: the key is neither a base
1316
+ # contract field (a note/format override) nor a `custom_`/`measure_` key, so there is
1317
+ # no stratum it could be written to. A client sending an unprefixed new key lands
1318
+ # here β€” and until D-291 that answered exactly like a save.
1319
+ return _refuse(
1320
+ ctx, 'unknown_key',
1321
+ "a new column's key must start with 'custom_'; a base column can only take a "
1322
+ 'note or a format override', key)
1323
  # Wave-5 item 1 β€” createdBy + permissions on the CREATED strata (custom_/measure_).
1324
  # `createdBy` is stamped HOST-SIDE at create and preserved on every later write: the
1325
  # client never supplies it, so a browser cannot claim someone else's field. A
platform/core/keychain.py CHANGED
@@ -166,6 +166,52 @@ def read_fields(rt, entry_id):
166
  return None
167
 
168
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
169
  def odoo_creds(rt):
170
  """R3's resolver seam: the FIRST odoo-type entry's fields, else None (the caller falls
171
  back to the environment). Deterministic order = insertion-id sort, so 'first' is stable."""
 
166
  return None
167
 
168
 
169
+ def first_entry_of_type(rt, etype):
170
+ """The metadata row of the FIRST entry of `etype`, or None. Never decrypts anything.
171
+
172
+ ⭐ W35-T45 β€” the same "first, by insertion-id sort" rule `_first_creds` resolves on, exposed as
173
+ an IDENTITY question rather than a credential one. `ensure_entry_of_type` needs to know whether
174
+ an entry exists WITHOUT unlocking it, and `list_entries` + a hand-rolled `next()` at each caller
175
+ is how two places come to disagree about which entry is "the" one.
176
+ """
177
+ return next((e for e in list_entries(rt) if e.get('type') == str(etype or '')), None)
178
+
179
+
180
+ def ensure_entry_of_type(rt, etype, label, fields, username):
181
+ """Create ONE entry of `etype` only if the tenant has none. `(row, why)`; `row` is None on skip.
182
+
183
+ ⭐ W35-T45 / R11 β€” the primitive tenant #0's env-to-keychain migration is built from, and the
184
+ reason it lives here is that only this module knows what "already has one" means: `odoo_creds`
185
+ resolves to the FIRST entry of a type, so a second one would be stored, invisible in every
186
+ resolver, and impossible to tell from the one that serves.
187
+
188
+ β›” IDEMPOTENT BY TYPE, NOT BY A FLAG. A migration that keys idempotency on a marker it writes
189
+ itself is one lost marker away from running twice ([[a-migration-that-runs-on-the-next-write]]);
190
+ keying it on the thing it would create cannot double-apply, whatever else fails.
191
+
192
+ β›” AND IT REFUSES ON A LOCKED KEYCHAIN RATHER THAN STORING A SECRET IN THE CLEAR. `add_entry`
193
+ would raise `KeychainLocked`; this answers with a REASON instead, because "there is no Fernet key
194
+ on this deployment" is an operator fact to report, not an exception to propagate out of a boot
195
+ thread where nobody reads it.
196
+
197
+ ⚠ `why` IS RETURNED EVEN ON SUCCESS-BY-SKIP. "Already migrated" and "could not migrate" are
198
+ different operator actions and a bare None cannot tell them apart.
199
+ """
200
+ etype = str(etype or '').strip().lower()
201
+ if etype not in ENTRY_TYPES:
202
+ return None, f'{etype!r} is not a keychain entry type'
203
+ existing = first_entry_of_type(rt, etype)
204
+ if existing:
205
+ return None, f'this tenant already has a {etype} entry ({existing["id"]})'
206
+ if not unlocked():
207
+ return None, ('this deployment has no keychain key, and a credential must never be stored '
208
+ 'in the clear')
209
+ try:
210
+ return add_entry(rt, label, etype, fields, username), ''
211
+ except Exception as exc: # noqa: BLE001
212
+ return None, f'{type(exc).__name__}: {exc}'
213
+
214
+
215
  def odoo_creds(rt):
216
  """R3's resolver seam: the FIRST odoo-type entry's fields, else None (the caller falls
217
  back to the environment). Deterministic order = insertion-id sort, so 'first' is stable."""
platform/core/perm_scope.py CHANGED
@@ -87,6 +87,22 @@ def may_access(user, module):
87
  return perms.may_open(user, module) # legacy record: the old grant wall
88
 
89
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
90
  # ── FIELDS ───────────────────────────────────────────────────────────────────────────────────
91
  def hidden_keys(user, module, fields):
92
  """The TRANSITIVE closure of hidden field keys (C-PERM amendment 5).
@@ -143,6 +159,37 @@ def visible_fields(fields, user, module):
143
  if not (isinstance(f, dict) and f.get('key') in hide)]
144
 
145
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
146
  def strip_row(row, hide):
147
  """Drop hidden keys from ONE assembled row. Cheap enough to run per row, and it must run
148
  per row: the field list and the row payload are two different wires, and stripping only the
@@ -170,6 +217,89 @@ def apply_row_scope(rows, user, module, fields, ctx=None):
170
  return [r for r in (rows or ()) if fe.permits(tree, r, fields, ctx)]
171
 
172
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
173
  # ── THE PUSHDOWN ─────────────────────────────────────────────────────────────────────────────
174
  #: `dba` value sets that pin a BU, and the Odoo team they pin. `_dba_attrs` emits exactly
175
  #: {'Fisch','Royal','Both'} (blank for "no brand attributable in 24 months"), so a permission
 
87
  return perms.may_open(user, module) # legacy record: the old grant wall
88
 
89
 
90
+ def assistant_entry(user, module):
91
+ """The explicit database grant an Assistant snapshot may rely on, else ``None``.
92
+
93
+ The interactive application retains an admin break-glass path and a temporary legacy-grant
94
+ compatibility path. Neither is an answer at the Assistant's app-stored data boundary:
95
+ that reader must be able to name the migrated grant that admitted a database. In particular,
96
+ a store-outage admin identity with no ``perms`` document is not an unresolved permission that
97
+ may be widened into data access.
98
+ """
99
+ e = entry(user, module)
100
+ if (not is_migrated(user) or not isinstance(e, dict)
101
+ or not bool(e.get('access', True)) or not may_access(user, module)):
102
+ return None
103
+ return e
104
+
105
+
106
  # ── FIELDS ───────────────────────────────────────────────────────────────────────────────────
107
  def hidden_keys(user, module, fields):
108
  """The TRANSITIVE closure of hidden field keys (C-PERM amendment 5).
 
159
  if not (isinstance(f, dict) and f.get('key') in hide)]
160
 
161
 
162
+ def assistant_visible_fields(fields, user, module):
163
+ """Visible closure under one explicit Assistant grant, including for an admin caller."""
164
+ e = assistant_entry(user, module)
165
+ if e is None:
166
+ return []
167
+ # `hidden_keys` deliberately preserves the interactive admin break-glass behaviour. The
168
+ # Assistant uses its explicit grant instead, but shares the same transitive formula closure.
169
+ hidden = {str(k) for k in (e.get('hiddenFields') or ()) if k}
170
+ if not hidden:
171
+ return list(fields or ())
172
+
173
+ refs = {}
174
+ for field in fields or ():
175
+ if not isinstance(field, dict) or not field.get('key'):
176
+ continue
177
+ expr = field.get('formula')
178
+ if isinstance(expr, str) and expr:
179
+ refs[field['key']] = {match.strip() for match in _FORMULA_REF.findall(expr)
180
+ if match.strip()}
181
+ for _ in range(12):
182
+ grew = False
183
+ for key, deps in refs.items():
184
+ if key not in hidden and deps & hidden:
185
+ hidden.add(key)
186
+ grew = True
187
+ if not grew:
188
+ break
189
+ return [field for field in (fields or ())
190
+ if not (isinstance(field, dict) and field.get('key') in hidden)]
191
+
192
+
193
  def strip_row(row, hide):
194
  """Drop hidden keys from ONE assembled row. Cheap enough to run per row, and it must run
195
  per row: the field list and the row payload are two different wires, and stripping only the
 
217
  return [r for r in (rows or ()) if fe.permits(tree, r, fields, ctx)]
218
 
219
 
220
+ def assistant_apply_row_scope(rows, user, module, fields, ctx=None):
221
+ """Apply an explicit Assistant grant's permanent filter without an admin bypass."""
222
+ e = assistant_entry(user, module)
223
+ if e is None:
224
+ return []
225
+ tree = e.get('filter')
226
+ if not tree:
227
+ return list(rows or ())
228
+ from harness import filter_eval as fe
229
+ return [row for row in (rows or ()) if fe.permits(tree, row, fields, ctx)]
230
+
231
+
232
+ def validate_assistant_filter(tree, fields):
233
+ """Return a strict, detached Assistant filter tree or raise ``ValueError``.
234
+
235
+ The display filter cleaner is intentionally permissive: it drops a stale column or leaves an
236
+ inactive condition alone so an old saved view can still open. An Assistant data reader cannot
237
+ inherit that behaviour β€” dropping a predicate turns a request for a subset into a wider read.
238
+ This validator therefore admits only visible field operands that the existing evaluator can
239
+ answer from one stored row. Cohort, measure and rank conditions need separate materialised
240
+ set resolvers and are refused here rather than guessed.
241
+ """
242
+ if tree is None:
243
+ return None
244
+ from harness import filter_eval as fe
245
+
246
+ by_key = {str(field.get('key')): field for field in (fields or ())
247
+ if isinstance(field, dict) and field.get('key')}
248
+ if not by_key:
249
+ raise ValueError('assistant filter has no visible field contract')
250
+
251
+ def _leaf(raw):
252
+ if not isinstance(raw, dict):
253
+ raise ValueError('assistant filter leaf must be an object')
254
+ allowed = {'id', 'colId', 'op', 'value', 'value2', 'rhs'}
255
+ if set(raw) - allowed:
256
+ raise ValueError('assistant filter carries an unsupported operand')
257
+ col = raw.get('colId')
258
+ op = raw.get('op')
259
+ if not isinstance(col, str) or col not in by_key:
260
+ raise ValueError('assistant filter names an unknown or hidden field')
261
+ if (not isinstance(op, str) or op not in fe.FILTER_OPS or op in fe.RANK_OPS
262
+ or op in {'between', 'within'} or col == fe.COHORT_FIELD):
263
+ raise ValueError('assistant filter uses an unsupported operator')
264
+ rhs = raw.get('rhs')
265
+ if rhs is not None:
266
+ if (not isinstance(rhs, dict) or rhs.get('kind') != 'field'
267
+ or set(rhs) - {'kind', 'colId'}
268
+ or not isinstance(rhs.get('colId'), str)
269
+ or rhs['colId'] not in by_key):
270
+ raise ValueError('assistant filter names an unknown or hidden right-hand field')
271
+ out = {name: raw[name] for name in ('id', 'colId', 'op', 'value', 'value2')
272
+ if name in raw}
273
+ if rhs is not None:
274
+ out['rhs'] = {'kind': rhs.get('kind'), 'colId': rhs['colId']}
275
+ if not fe.is_rule_active(out, by_key):
276
+ raise ValueError('assistant filter is inactive or unanswerable')
277
+ return out
278
+
279
+ def _node(raw):
280
+ if not isinstance(raw, dict):
281
+ raise ValueError('assistant filter node must be an object')
282
+ if 'children' not in raw:
283
+ return _leaf(raw)
284
+ if set(raw) - {'conj', 'children'}:
285
+ raise ValueError('assistant filter group carries an unsupported operand')
286
+ children = raw.get('children')
287
+ if raw.get('conj') not in ('and', 'or') or not isinstance(children, list) or not children:
288
+ raise ValueError('assistant filter group must be a non-empty and/or group')
289
+ return {'conj': raw['conj'], 'children': [_node(child) for child in children]}
290
+
291
+ if isinstance(tree, list):
292
+ if not tree:
293
+ raise ValueError('assistant filter list must not be empty')
294
+ return {'conj': 'and', 'nodes': [_node(node) for node in tree]}
295
+ if not isinstance(tree, dict) or set(tree) - {'conj', 'nodes'}:
296
+ raise ValueError('assistant filters must be a tree with nodes')
297
+ nodes = tree.get('nodes')
298
+ if tree.get('conj') not in ('and', 'or') or not isinstance(nodes, list) or not nodes:
299
+ raise ValueError('assistant filter tree must be a non-empty and/or tree')
300
+ return {'conj': tree['conj'], 'nodes': [_node(node) for node in nodes]}
301
+
302
+
303
  # ── THE PUSHDOWN ─────────────────────────────────────────────────────────────────────────────
304
  #: `dba` value sets that pin a BU, and the Odoo team they pin. `_dba_attrs` emits exactly
305
  #: {'Fisch','Royal','Both'} (blank for "no brand attributable in 24 months"), so a permission
platform/core/perms.py CHANGED
@@ -199,6 +199,116 @@ def nav_pages(user):
199
  return out
200
 
201
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
202
  def landing_page(user):
203
  """Where a restricted user lands: the first MAIN-chrome page in their grant. None for
204
  unrestricted users (the client lands on the Analyst, as the host does).
 
199
  return out
200
 
201
 
202
+ # ------------------------------------------------------------------ governable modules (C3)
203
+ #: ⭐⭐ WAVE 33, OWNER ITEM 11 β€” *"its an error that ALL fucking tenant can see both Customer and
204
+ #: Product database under Manage User, this is only for Royal Imports tenant. Make sure per tenant
205
+ #: configuration always work, this is my fourth time I flag it."*
206
+ #:
207
+ #: The cause was two lines in `routes_admin.py`: `_PERM_MODULES = ("customer_data","product_data")`
208
+ #: and a `_MODULE_LABELS` dict beside it. `get_perms` built its answer from the literal and never
209
+ #: touched `session.runtime`, so the permission editor was TENANT-BLIND BY CONSTRUCTION β€” every
210
+ #: tenant was handed tenant #0's two grid modules whether or not it is provisioned for either.
211
+ #:
212
+ #: β›” WHY THE DERIVATION LIVES HERE AND NOT AT THE ROUTE. `routes_nav.py::nav` already applies
213
+ #: exactly this rule and has since wave 18; the perms route is the ONE surface that skipped it.
214
+ #: A second copy of the rule at a second call site is how the two came apart in the first place,
215
+ #: so the rule is stated once, as a pure function of (runtime, catalogue), and both the route and
216
+ #: `verify_perm_scope` ask THIS.
217
+ def tenant_modules(runtime):
218
+ """The registry keys this TENANT is provisioned for. None means "every key".
219
+
220
+ β›” FAIL-CLOSED ON THE EMPTY LIST, and that is not the obvious reading. `routes_admin`'s own
221
+ docstring records two shipped fail-OPENs of exactly this shape (`modules: []` and
222
+ `bus: [<unknown id>]` both read as UNRESTRICTED), so ONLY the literal `'all'` β€” or an absent
223
+ key, which is how the compiled-in tenant #0 record is written β€” means every module. A
224
+ bucket-provisioned tenant gets `{"modules": rec.get("modules", [])}`
225
+ (`harness/runtime.py::_builder_from_record`), so `[]` there means the tenant was provisioned
226
+ for NOTHING and must be told so, never handed the catalogue.
227
+
228
+ Legacy keys are migrated on read for the same reason `allowed_modules` does it: a tenant
229
+ record written before a rename must not silently lose the module it names.
230
+ """
231
+ cfg = getattr(getattr(runtime, 'tenant', None), 'config', None) or {}
232
+ raw = cfg.get('modules', 'all')
233
+ if raw is None or raw == 'all':
234
+ return None
235
+ if isinstance(raw, str):
236
+ raw = [raw]
237
+ return {_LEGACY_KEYS.get(str(k).strip(), str(k).strip())
238
+ for k in raw if str(k or '').strip()}
239
+
240
+
241
+ def tenant_has_module(runtime, key):
242
+ """Is `key` in this tenant's provisioned set? PARENT-AWARE, like `may_open`.
243
+
244
+ `routes_nav` keeps a page whose `parent` is granted even when the child key is not, and a
245
+ governable list that disagreed with the nav would offer an admin a wall for a database the
246
+ tenant cannot open β€” or hide one it can.
247
+ """
248
+ allowed = tenant_modules(runtime)
249
+ if allowed is None or key in allowed:
250
+ return True
251
+ parent = registry.BY_KEY.get(key, {}).get('parent')
252
+ return parent in allowed if parent else False
253
+
254
+
255
+ def tenant_governable_modules(runtime, topics, ut_entries=()):
256
+ """The databases the permission editor may render for THIS tenant, in stable order.
257
+
258
+ ⚠ NAMED `tenant_*` TO STAY DISTINCT FROM `registry.governable_modules()`, which lane E
259
+ published under contract C3 and which this consumes. They answer different questions β€”
260
+ registry's is "which topics are governable at all, platform-wide"; this one is "which
261
+ databases does THIS tenant govern" β€” and two same-named functions one import apart is the
262
+ [[one-question-two-normalizers]] shape waiting to happen.
263
+
264
+ `topics` β€” the registry-backed grid modules the editor can build pickers for, as
265
+ `[{'key': ..., 'label'?: ...}]`, i.e. `registry.governable_modules()` narrowed
266
+ to the topics the caller has a field-schema arm for. The CALLER owns that
267
+ narrowing because the schema behind each arm is code
268
+ (`routes_admin::_module_fields`), not a policy list. Tenant-blind on the way
269
+ in; this function is the only place the tenant is applied.
270
+ `ut_entries` β€” this tenant's own `ut_*` databases, in `user_tables.nav_entries` shape.
271
+
272
+ Returns `[{'key', 'label', 'enforced'}]`.
273
+
274
+ β›”β›” `enforced` IS THE HONEST HALF AND IT IS NOT DECORATION. `perm_scope` walls the REGISTRY
275
+ topics β€” `grid_assembly` calls `may_access`, `visible_fields` and `apply_row_scope` on every
276
+ read. It is never consulted for a `ut_*` database: `routes_tables`' visibility is
277
+ `user_tables.may_open` (creator, admin, or a `core.shares` grant) and there is no module
278
+ grant in that path at all. So a stored `ut_*` wall would be INERT β€” the editor would say
279
+ DENY, the table routes would keep serving, and nothing anywhere would say so. That is the
280
+ same class as the empty-dropdown defect (wave 26 item 24): a control that answers when it
281
+ cannot. The database is LISTED, because the owner asked to see every database; the wall is
282
+ declared unenforced, and `routes_admin::_clean_perms` REFUSES to store one against it rather
283
+ than accepting a rule nobody applies. Arming `perm_scope` over `ut_*` is a change to
284
+ `routes_tables`/`user_tables` (lane A's fence) β€” booked, not faked.
285
+ """
286
+ out = []
287
+ for t in topics or ():
288
+ key = str((t or {}).get('key') or '').strip()
289
+ if not key or not tenant_has_module(runtime, key):
290
+ continue
291
+ # The LABEL comes from the registry, never from a dict beside the list. `_MODULE_LABELS`
292
+ # was a hardcoded copy of two registry labels, so wave 33's R2 relabel ("Odoo customers" /
293
+ # "Odoo products") would have moved one of them and left the permission editor showing the
294
+ # old name with nothing to notice.
295
+ label = (t.get('label') or registry.BY_KEY.get(key, {}).get('label') or key)
296
+ out.append({'key': key, 'label': label, 'enforced': True})
297
+ seen = {r['key'] for r in out}
298
+ for e in ut_entries or ():
299
+ key = str((e or {}).get('key') or '').strip()
300
+ if not key or key in seen:
301
+ continue
302
+ seen.add(key)
303
+ out.append({'key': key, 'label': (e.get('label') or key), 'enforced': False})
304
+ return out
305
+
306
+
307
+ def enforced_module_keys(modules):
308
+ """The subset of `governable_modules`' answer a permission block may actually be stored for."""
309
+ return {m['key'] for m in (modules or ()) if m.get('enforced')}
310
+
311
+
312
  def landing_page(user):
313
  """Where a restricted user lands: the first MAIN-chrome page in their grant. None for
314
  unrestricted users (the client lands on the Analyst, as the host does).
platform/core/registry.py CHANGED
@@ -22,6 +22,22 @@ Fields:
22
  2026-07-27 pass (keep = Sales + Customers family + Agents + Procurement +
23
  Collections + Analyst/plumbing): products, assortment, financial, management,
24
  pricing, inventory, vendor, warehouse, returns, outreach, complexity, health.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
25
  note one-line scope note shown in the UI / docs
26
  """
27
  # WAVE-9 I8 β€” `'source'`: which connected system a module's data comes from.
@@ -41,7 +57,7 @@ REGISTRY = [
41
  # briefing still read it, and pages_sales.py survives UNREGISTERED as the Y1 envelope's
42
  # template + verify_api's fixture.
43
  {'key': 'sales', 'label': 'Sales', 'brand': True, 'hq': False, 'validate': True, 'source': 'Odoo', 'archived': True,
44
- 'note': 'Revenue, YoY, seasonality, reps, customers, SKUs β€” by BU. Retired wave 16: '
45
  'rebuild as grid chart/dashboard views (compare series, KPI deltas, tables).'},
46
  # ARCHIVED AS A PAGE (owner item 14, wave 8): "archive the current form of the Customer
47
  # dashboard completely, i want to redesign it, exactly with the backend we have". So the
@@ -65,7 +81,14 @@ REGISTRY = [
65
  # myday->customer_list was done. The STORE keys are deliberately NOT renamed:
66
  # 'customer_table_workspace' and 'customer_lists' hold real user data, and renaming them
67
  # would orphan every saved view. Label-only changes need none of that machinery.
68
- {'key': 'customer_data', 'label': 'Customer', 'brand': True, 'hq': False, 'validate': False, 'source': 'Odoo', 'parent': 'customers',
 
 
 
 
 
 
 
69
  'note': 'Build, filter and save customer lists: the whole scoped book with per-customer '
70
  'metrics; saved lists are visible, editable formulas (Call list and Win-back ship '
71
  'as templates) plus hand-picked members. Sub-module of Customers (owner nav redesign 2026-07-23; replaced My Day).'},
@@ -81,7 +104,7 @@ REGISTRY = [
81
  # _LEGACY_KEYS maps cohortβ†’customer_data so grants + ?page= deep links land on the grid
82
  # that now hosts the cohorts.
83
  {'key': 'cohort', 'label': 'Cohort', 'brand': True, 'hq': False, 'validate': False, 'source': 'Odoo', 'parent': 'customers', 'archived': True,
84
- 'note': 'Hand-curated, unchanging customer lists β€” folded into the Customer rail as '
85
  'locked views (wave 16). The set is fixed: it changes only when someone adds '
86
  'or removes a member. Open them from Customer > Views > Cohorts.'},
87
  # ARCHIVED wave 16 (owner item 7, R11, 2026-08-02) beside Sales. The agent DRAWER and the
@@ -101,13 +124,16 @@ REGISTRY = [
101
  'group_only': True,
102
  'note': 'Folder for the product surfaces (Product table). The archived SKU dashboard is '
103
  'not part of this family.'},
104
- {'key': 'product_data', 'label': 'Product', 'brand': True, 'hq': False, 'validate': True, 'source': 'Odoo', 'parent': 'products_family',
 
 
 
105
  'note': 'The SKU catalogue as a grid: per-product revenue, units and (consolidated) '
106
  'stock columns, with saved views and custom fields. Identity = the SKU code.'},
107
  {'key': 'products', 'label': 'SKU', 'brand': True, 'hq': False, 'validate': True, 'archived': True,
108
- 'note': 'SKU movers, zombies, velocity, coverage, drawers β€” by BU (sales-derived).'},
109
  {'key': 'assortment', 'label': 'Assortment', 'brand': True, 'hq': False, 'validate': True, 'archived': True,
110
- 'note': 'Facet-level performance (category/color/occasion/collection/season) + season readiness β€” by BU.'},
111
  {'key': 'financial', 'label': 'Financial', 'brand': True, 'hq': False, 'validate': True, 'archived': True,
112
  'note': 'Gross margin by BU/category/SKU. Cash-conversion cycle is HQ-consolidated.'},
113
  {'key': 'pricing', 'label': 'Pricing', 'brand': True, 'hq': False, 'validate': True, 'archived': True,
@@ -122,7 +148,7 @@ REGISTRY = [
122
  'note': 'PAGE RETIRED 2026-08-03 (owner wave-17 item 13: "We should be able to replace '
123
  'Procurement completely, and add it as part of the Product database"). The buy '
124
  'list is now a saved VIEW on the Product grid, filtered on a FORMULA field over '
125
- 'the supplier/lead-time columns and the demand measure β€” the owner\'s ruling R3: '
126
  '"Buy list is just a View, with a Filter from a Formula field that taps into '
127
  'Metrics Fields... the 8-month demand baseline etc. is just math in a Formula '
128
  'field." The ROW STAYS validate_only: `modules/procurement.py` still owns the '
@@ -136,7 +162,7 @@ REGISTRY = [
136
  'source': 'Odoo', 'api_surface': False, 'admin_only': True,
137
  'note': 'PAGE RETIRED 2026-08-03 (owner wave-17 item 15: "Collections should also be '
138
  'entirely replicable as just a View under Customer"). The worklist is the shared '
139
- '"Collections" view on the Customer grid β€” same numbers, from this module\'s own '
140
  'reconciled blocks (ar_open/ar_overdue/ar_exposure/days_to_pay + the four aging '
141
  'buckets). The ROW STAYS validate_only so validate.py keeps running ar.validate(), '
142
  'which is the proof those columns rest on; archiving it would have skipped the '
@@ -145,7 +171,7 @@ REGISTRY = [
145
  'sender (app._collections_statements -> modules/collections_send), the ONE '
146
  'sanctioned Odoo writer, which the same ruling says stays untouched. So the row '
147
  'keeps a Streamlit page for ADMINS ONLY (`admin_only`) and leaves the API payload '
148
- 'entirely (`api_surface: False`) β€” no "Collections" in the React nav, no second '
149
  'worklist, and the biweekly send keeps its door.'},
150
  {'key': 'returns', 'label': 'Returns', 'brand': False, 'hq': True, 'validate': True, 'archived': True,
151
  'note': 'Credit-note lens: refund concentration by SKU (quality) and customer (behavior). Company-level.'},
@@ -154,7 +180,7 @@ REGISTRY = [
154
  {'key': 'backorders', 'label': 'Backorders', 'brand': True, 'hq': False, 'validate': True, 'archived': True,
155
  'note': 'Confirmed-undelivered order lines aged vs promise date, valued, with a supply-aware next action per row (ship / expedite / call). Wholesale scope.'},
156
  {'key': 'pricecomp', 'label': 'Price Compliance', 'brand': True, 'hq': False, 'validate': True, 'archived': True,
157
- 'note': 'Selling below the customer pricelist tier (LTM, per customer x SKU): the pocket-price floor worklist with annualized leak $. Sub-30% ratios usually mean a stale or pack-basis RULE β€” fix the rule, not the rep.'},
158
  {'key': 'o2c', 'label': 'Cash Timing', 'brand': True, 'hq': False, 'validate': True, 'archived': True,
159
  'note': 'Order-to-cash stage decomposition (order-to-ship / ship-to-invoice / invoice-to-paid, each with its owner) + the terms-gap rollup: contractual vs actual days per payment term with the free-credit $ it strands.'},
160
  {'key': 'bookings', 'label': 'Order Book', 'brand': True, 'hq': False, 'validate': True, 'archived': True,
@@ -167,29 +193,37 @@ REGISTRY = [
167
  'note': 'Operating expense from the GL (all expense-type accounts; COGS excluded): trend, operating leverage (opex % of revenue), the YoY cost bridge, XmR control-limit spike watch list, fixed/variable split and a drill-to-ledger category directory. Company-level.'},
168
  {'key': 'health', 'label': 'Data Health', 'brand': False, 'hq': True, 'validate': True, 'archived': True,
169
  'note': 'Close/reconciliation scan; period-filtered, mostly company-level. ARCHITECTURE '
170
- '(owner 2026-07-23): every discrepancy / potential-error marker lives here β€” '
171
  'procurement mapping gaps, count-trust (unverified on-hand counts), untracked-on-'
172
- 'order β€” so operational workflows stay clean for doing the work.'},
173
  {'key': 'dictionary', 'label': 'Metric Dictionary', 'brand': False, 'hq': True, 'validate': True,
174
  'nav': False, 'validate_only': True,
175
- 'note': 'PAGE RETIRED 2026-07-23 (owner: broken/not customer-facing) β€” row kept ONLY so '
176
  'validate.py keeps running the semantic-layer contracts (the Analyst grounding '
177
- 'proof). No nav, no page. Wave 17 R8 ("I don\'t even know what it does β€” delete '
178
  'them") made that literal: `validate_only` takes it out of the account menu too, '
179
  'which is the last place it was still visible. The proof survives; the door does not.'},
180
- {'key': 'automation', 'label': 'Automation', 'brand': False, 'hq': True, 'validate': False,
181
  'nav': True,
182
- 'note': 'Wave 18 (C-AUTONAV): the Automation surface β€” scheduled jobs that create and '
183
- 'refresh user databases (website scrape-to-DB, the Instagram field). React-only '
184
- 'surface (no PAGE_FUNCS entry, the no-new-Streamlit rule); admins hold it via '
185
- '"all", other users need the explicit grant β€” fail-closed default.'},
 
 
 
 
 
 
 
 
186
  {'key': 'settings', 'label': 'Settings', 'brand': False, 'hq': True, 'validate': False,
187
  'nav': False, # sidebar sentinel (account group) β€” owner IA 2026-07-12
188
  'note': 'User scope settings: the Business Unit toggle (strict isolation) and the Data basis '
189
  'toggle (Orders vs Invoiced) moved here from the sidebar.'},
190
  {'key': 'analyst', 'label': 'AIOS Analyst', 'brand': False, 'hq': True, 'validate': False,
191
  'nav': False, # sidebar sentinel button; eval gate = harness/evals.py (run pre-ship, NOT in validate.py β€” live LLM cost)
192
- 'note': 'Ask the business a question in plain language: a small AI model calls governed tools over the semantic layer β€” answers carry their query trace and drill links. AI-generated output (Art. 50 labeled).'},
193
  ]
194
 
195
  BY_KEY = {m['key']: m for m in REGISTRY}
@@ -197,3 +231,90 @@ BY_KEY = {m['key']: m for m in REGISTRY}
197
 
198
  def brand_filterable(key) -> bool:
199
  return bool(BY_KEY.get(key, {}).get('brand'))
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
22
  2026-07-27 pass (keep = Sales + Customers family + Agents + Procurement +
23
  Collections + Analyst/plumbing): products, assortment, financial, management,
24
  pricing, inventory, vendor, warehouse, returns, outreach, complexity, health.
25
+ subject set -> the REAL-WORLD POPULATION this row presents, as a source-qualified id
26
+ ('odoo:res.partner'). ⭐ W33-T41 / item 12a: it is what makes "two databases present
27
+ the same thing" a statement code can refuse, and it exists because NO OTHER FIELD
28
+ CAN CARRY THAT TEST. Not the key (`product_data` and `ut_odoo_products` are different
29
+ keys by construction β€” that is the bug, not the check), and NOT THE LABEL: the owner's
30
+ own complaint is "why do we have 'Odoo products' ALREADY with the current Products
31
+ database?", i.e. the twin was already carrying the label the legacy row is about to
32
+ take (W33-T42). Between T42 and T44 the two rows legitimately share a label, so a
33
+ label-keyed check would refuse the rename that fixes the bug.
34
+ ⚠ A row without `subject` claims nothing and conflicts with nothing β€” silence is not
35
+ a claim, so adding this field cannot retro-refuse anything already shipped.
36
+ topic set -> this row is a GOVERNABLE DATABASE and this is the word the rest of the codebase
37
+ already calls it by (`routes_grid._SCOPES`, `routes_alerts._TOPICS`, `scope=` on the
38
+ grid routes). ⭐ W33-T46 / contract C3: it is what `governable_modules()` keys on, and
39
+ it exists so `routes_admin`'s permission list can be DERIVED from the registry instead
40
+ of being the hardcoded two-element tuple that made every tenant see Customer + Product.
41
  note one-line scope note shown in the UI / docs
42
  """
43
  # WAVE-9 I8 β€” `'source'`: which connected system a module's data comes from.
 
57
  # briefing still read it, and pages_sales.py survives UNREGISTERED as the Y1 envelope's
58
  # template + verify_api's fixture.
59
  {'key': 'sales', 'label': 'Sales', 'brand': True, 'hq': False, 'validate': True, 'source': 'Odoo', 'archived': True,
60
+ 'note': 'Revenue, YoY, seasonality, reps, customers, SKUs. By BU. Retired wave 16: '
61
  'rebuild as grid chart/dashboard views (compare series, KPI deltas, tables).'},
62
  # ARCHIVED AS A PAGE (owner item 14, wave 8): "archive the current form of the Customer
63
  # dashboard completely, i want to redesign it, exactly with the backend we have". So the
 
81
  # myday->customer_list was done. The STORE keys are deliberately NOT renamed:
82
  # 'customer_table_workspace' and 'customer_lists' hold real user data, and renaming them
83
  # would orphan every saved view. Label-only changes need none of that machinery.
84
+ # LABEL renamed 'Customer' -> 'Odoo customers' (owner item 12, 2026-08-14 / W33-T42): the
85
+ # naming convention of every other database from this connector ("Odoo orders", "Odoo vendor
86
+ # bills"). ⭐ STILL LABEL-ONLY, and the note above is the precedent β€” the KEY does not move, so
87
+ # no grant, saved view, cohort, formula, shared column or `?page=` deep link is touched.
88
+ # β›” `nav_meta` could not have done this: `routes_nav.py` raises `400 name_not_allowed` for any
89
+ # key that does not start with `ut_`, so a tenant rename cannot reach a registry module at all.
90
+ {'key': 'customer_data', 'label': 'Odoo customers', 'brand': True, 'hq': False, 'validate': False, 'source': 'Odoo', 'parent': 'customers',
91
+ 'subject': 'odoo:res.partner', 'topic': 'customer',
92
  'note': 'Build, filter and save customer lists: the whole scoped book with per-customer '
93
  'metrics; saved lists are visible, editable formulas (Call list and Win-back ship '
94
  'as templates) plus hand-picked members. Sub-module of Customers (owner nav redesign 2026-07-23; replaced My Day).'},
 
104
  # _LEGACY_KEYS maps cohortβ†’customer_data so grants + ?page= deep links land on the grid
105
  # that now hosts the cohorts.
106
  {'key': 'cohort', 'label': 'Cohort', 'brand': True, 'hq': False, 'validate': False, 'source': 'Odoo', 'parent': 'customers', 'archived': True,
107
+ 'note': 'Hand-curated, unchanging customer lists. Folded into the Customer rail as '
108
  'locked views (wave 16). The set is fixed: it changes only when someone adds '
109
  'or removes a member. Open them from Customer > Views > Cohorts.'},
110
  # ARCHIVED wave 16 (owner item 7, R11, 2026-08-02) beside Sales. The agent DRAWER and the
 
124
  'group_only': True,
125
  'note': 'Folder for the product surfaces (Product table). The archived SKU dashboard is '
126
  'not part of this family.'},
127
+ # LABEL renamed 'Product' -> 'Odoo products' (owner item 12, 2026-08-14 / W33-T42) β€” see the
128
+ # customer_data note above; label-only, the key stays `product_data`.
129
+ {'key': 'product_data', 'label': 'Odoo products', 'brand': True, 'hq': False, 'validate': True, 'source': 'Odoo', 'parent': 'products_family',
130
+ 'subject': 'odoo:product.product', 'topic': 'product',
131
  'note': 'The SKU catalogue as a grid: per-product revenue, units and (consolidated) '
132
  'stock columns, with saved views and custom fields. Identity = the SKU code.'},
133
  {'key': 'products', 'label': 'SKU', 'brand': True, 'hq': False, 'validate': True, 'archived': True,
134
+ 'note': 'SKU movers, zombies, velocity, coverage, drawers. By BU (sales-derived).'},
135
  {'key': 'assortment', 'label': 'Assortment', 'brand': True, 'hq': False, 'validate': True, 'archived': True,
136
+ 'note': 'Facet-level performance (category/color/occasion/collection/season) + season readiness. By BU.'},
137
  {'key': 'financial', 'label': 'Financial', 'brand': True, 'hq': False, 'validate': True, 'archived': True,
138
  'note': 'Gross margin by BU/category/SKU. Cash-conversion cycle is HQ-consolidated.'},
139
  {'key': 'pricing', 'label': 'Pricing', 'brand': True, 'hq': False, 'validate': True, 'archived': True,
 
148
  'note': 'PAGE RETIRED 2026-08-03 (owner wave-17 item 13: "We should be able to replace '
149
  'Procurement completely, and add it as part of the Product database"). The buy '
150
  'list is now a saved VIEW on the Product grid, filtered on a FORMULA field over '
151
+ 'the supplier/lead-time columns and the demand measure. The owner\'s ruling R3: '
152
  '"Buy list is just a View, with a Filter from a Formula field that taps into '
153
  'Metrics Fields... the 8-month demand baseline etc. is just math in a Formula '
154
  'field." The ROW STAYS validate_only: `modules/procurement.py` still owns the '
 
162
  'source': 'Odoo', 'api_surface': False, 'admin_only': True,
163
  'note': 'PAGE RETIRED 2026-08-03 (owner wave-17 item 15: "Collections should also be '
164
  'entirely replicable as just a View under Customer"). The worklist is the shared '
165
+ '"Collections" view on the Customer grid. Same numbers, from this module\'s own '
166
  'reconciled blocks (ar_open/ar_overdue/ar_exposure/days_to_pay + the four aging '
167
  'buckets). The ROW STAYS validate_only so validate.py keeps running ar.validate(), '
168
  'which is the proof those columns rest on; archiving it would have skipped the '
 
171
  'sender (app._collections_statements -> modules/collections_send), the ONE '
172
  'sanctioned Odoo writer, which the same ruling says stays untouched. So the row '
173
  'keeps a Streamlit page for ADMINS ONLY (`admin_only`) and leaves the API payload '
174
+ 'entirely (`api_surface: False`). No "Collections" in the React nav, no second '
175
  'worklist, and the biweekly send keeps its door.'},
176
  {'key': 'returns', 'label': 'Returns', 'brand': False, 'hq': True, 'validate': True, 'archived': True,
177
  'note': 'Credit-note lens: refund concentration by SKU (quality) and customer (behavior). Company-level.'},
 
180
  {'key': 'backorders', 'label': 'Backorders', 'brand': True, 'hq': False, 'validate': True, 'archived': True,
181
  'note': 'Confirmed-undelivered order lines aged vs promise date, valued, with a supply-aware next action per row (ship / expedite / call). Wholesale scope.'},
182
  {'key': 'pricecomp', 'label': 'Price Compliance', 'brand': True, 'hq': False, 'validate': True, 'archived': True,
183
+ 'note': 'Selling below the customer pricelist tier (LTM, per customer x SKU): the pocket-price floor worklist with annualized leak $. Sub-30% ratios usually mean a stale or pack-basis RULE. Fix the rule, not the rep.'},
184
  {'key': 'o2c', 'label': 'Cash Timing', 'brand': True, 'hq': False, 'validate': True, 'archived': True,
185
  'note': 'Order-to-cash stage decomposition (order-to-ship / ship-to-invoice / invoice-to-paid, each with its owner) + the terms-gap rollup: contractual vs actual days per payment term with the free-credit $ it strands.'},
186
  {'key': 'bookings', 'label': 'Order Book', 'brand': True, 'hq': False, 'validate': True, 'archived': True,
 
193
  'note': 'Operating expense from the GL (all expense-type accounts; COGS excluded): trend, operating leverage (opex % of revenue), the YoY cost bridge, XmR control-limit spike watch list, fixed/variable split and a drill-to-ledger category directory. Company-level.'},
194
  {'key': 'health', 'label': 'Data Health', 'brand': False, 'hq': True, 'validate': True, 'archived': True,
195
  'note': 'Close/reconciliation scan; period-filtered, mostly company-level. ARCHITECTURE '
196
+ '(owner 2026-07-23): every discrepancy / potential-error marker lives here. '
197
  'procurement mapping gaps, count-trust (unverified on-hand counts), untracked-on-'
198
+ 'order. So operational workflows stay clean for doing the work.'},
199
  {'key': 'dictionary', 'label': 'Metric Dictionary', 'brand': False, 'hq': True, 'validate': True,
200
  'nav': False, 'validate_only': True,
201
+ 'note': 'PAGE RETIRED 2026-07-23 (owner: broken/not customer-facing). Row kept ONLY so '
202
  'validate.py keeps running the semantic-layer contracts (the Analyst grounding '
203
+ 'proof). No nav, no page. Wave 17 R8 ("I don\'t even know what it does. Delete '
204
  'them") made that literal: `validate_only` takes it out of the account menu too, '
205
  'which is the last place it was still visible. The proof survives; the door does not.'},
206
+ {'key': 'automation', 'label': 'Agents', 'brand': False, 'hq': True, 'validate': False,
207
  'nav': True,
208
+ 'note': 'Wave 18 (C-AUTONAV): scheduled jobs that create and refresh user databases '
209
+ '(website scrape-to-DB, the Instagram field). React-only surface (no PAGE_FUNCS '
210
+ 'entry, the no-new-Streamlit rule); admins hold it via "all", other users need the '
211
+ 'explicit grant, fail-closed default. '
212
+ '⭐ WAVE 34 · R12. THE LABEL IS "Agents" AND THE KEY IS STILL `automation`. The '
213
+ 'owner ruled the module becomes Agent: this is our agent deployment surface going '
214
+ 'forward. Only the LABEL moved, because the key is a permission scope, a nav '
215
+ 'placement key, a grant name and the `#/automation` route; renaming it would revoke '
216
+ 'every stored grant and 404 every bookmark to buy a caption. ⚠ The plural is not a '
217
+ 'slip: R15 and the wave PRD both spell the rail row "Agents", and the degraded '
218
+ 'fallback row in Shell.tsx hard-codes that spelling, so a singular here would paint '
219
+ 'two different words in one list depending on whether /nav answered.'},
220
  {'key': 'settings', 'label': 'Settings', 'brand': False, 'hq': True, 'validate': False,
221
  'nav': False, # sidebar sentinel (account group) β€” owner IA 2026-07-12
222
  'note': 'User scope settings: the Business Unit toggle (strict isolation) and the Data basis '
223
  'toggle (Orders vs Invoiced) moved here from the sidebar.'},
224
  {'key': 'analyst', 'label': 'AIOS Analyst', 'brand': False, 'hq': True, 'validate': False,
225
  'nav': False, # sidebar sentinel button; eval gate = harness/evals.py (run pre-ship, NOT in validate.py β€” live LLM cost)
226
+ 'note': 'Ask the business a question in plain language: a small AI model calls governed tools over the semantic layer. Answers carry their query trace and drill links. AI-generated output (Art. 50 labeled).'},
227
  ]
228
 
229
  BY_KEY = {m['key']: m for m in REGISTRY}
 
231
 
232
  def brand_filterable(key) -> bool:
233
  return bool(BY_KEY.get(key, {}).get('brand'))
234
+
235
+
236
+ # ═════════════════════════════════════════════════════════════════════════════════════════════
237
+ # ⭐⭐ W33-T41 / item 12a β€” ONE SUBJECT, ONE DATABASE
238
+ # ═════════════════════════════════════════════════════════════════════════════════════════════
239
+ #
240
+ # The owner: *"why do we have 'Odoo products' already with the current Products database? The
241
+ # Unique ID is redundant, make sure this never happens. One unique ID per database always."*
242
+ #
243
+ # β›” THE CHECK HAD TO GO WHERE THE BUG WAS MADE, AND THAT IS NOT WHERE ANYONE WOULD LOOK.
244
+ # `core.user_tables.create` β€” the UI door β€” HAS had a collision loop all along: it slugs the
245
+ # label to a key and appends `_2 … _49` until the key is free. It never fired here for two
246
+ # reasons, and both matter. (1) It compares KEYS inside ONE tenant's `user_tables` document,
247
+ # and `customer_data` / `product_data` are compiled REGISTRY rows that never enter that
248
+ # document at all β€” the two halves of the duplicate live in different namespaces and only meet
249
+ # in `routes_nav.py::nav`, where the registry pages and `user_tables.nav_entries` are merged
250
+ # for display. (2) The connector path never calls `create()`: `odoo_relational.
251
+ # _ensure_table_inplace` writes `cur[key] = {...}` straight into the document. So a check bolted
252
+ # onto `create()` would have been green through the entire life of this defect.
253
+ #
254
+ # ⚠ THE SUFFIX LOOP IS ALSO WHY A RELABEL ALONE WOULD HAVE MADE THIS WORSE, MECHANICALLY: a
255
+ # naive "just rename Product to Odoo products" mints `ut_odoo_products_2` β€” the owner's
256
+ # redundant id again, from the fix for it.
257
+ #
258
+ #: subject -> the ONE registry key entitled to present it. Derived, never typed twice.
259
+ SUBJECT_CLAIMS = {m['subject']: m['key'] for m in REGISTRY if m.get('subject')}
260
+
261
+
262
+ # ═════════════════════════════════════════════════════════════════════════════════════════════
263
+ # ⭐⭐ W33-T46 / CONTRACT C3 β€” THE GOVERNABLE MODULE LIST IS DERIVED, NOT A LITERAL
264
+ # ═════════════════════════════════════════════════════════════════════════════════════════════
265
+ #
266
+ # The owner, on his FOURTH flag of the same thing: *"its an error that ALL fucking tenant can see
267
+ # both Customer and Product database under Manage User, this is only for Royal Imports tenant.
268
+ # Make sure per tenant configuration always work."*
269
+ #
270
+ # β›” THE CAUSE IS A TWO-ELEMENT TUPLE. `routes_admin._PERM_MODULES = ("customer_data",
271
+ # "product_data")` β€” a compiled literal that `get_perms` builds its whole list from without ever
272
+ # touching `session.runtime`. It is tenant-blind BY CONSTRUCTION, which is why no amount of
273
+ # per-tenant work elsewhere ever fixed it, and why the answer is not "filter the tuple" but
274
+ # "delete the tuple": a second hand-typed copy of a fact the registry already holds is a thing
275
+ # that CAN go stale, and this one did, four times.
276
+ #
277
+ # ⚠ THE LINE BETWEEN THE TWO HALVES, stated because both sides need it: this map is
278
+ # REGISTRY-SIDE and TENANT-BLIND β€” it answers *"which compiled modules are governable, and what
279
+ # are they called"*, nothing more. Applying the tenant's own `config["modules"]` filter and
280
+ # merging that tenant's `ut_*` databases is the CALLER's half (`routes_admin`, D's fence), which
281
+ # is exactly what `routes_nav.py` already does correctly and `routes_admin` never did.
282
+ def governable_modules():
283
+ """`{topic: {'key','label','subject'}}` β€” every registry row that is a governable DATABASE.
284
+
285
+ A fresh dict per call, deliberately: a module-level mapping handed to a route is one
286
+ `.pop()` away from being a different list for every request after the first.
287
+ """
288
+ return {m['topic']: {'key': m['key'], 'label': m['label'], 'subject': m.get('subject')}
289
+ for m in REGISTRY
290
+ if m.get('topic') and not m.get('archived') and not m.get('group_only')}
291
+
292
+
293
+ def subject_of(key):
294
+ """The subject a REGISTRY row claims, or None. (`ut_*` databases carry theirs on the stored
295
+ definition instead β€” they are per-tenant data, not compiled rows.)"""
296
+ return (BY_KEY.get(key) or {}).get('subject')
297
+
298
+
299
+ def subject_conflict(subject, key, claimed=None):
300
+ """The OTHER key already presenting `subject`, or None.
301
+
302
+ `claimed` is an optional {subject: key} of claims outside the registry β€” in practice the
303
+ caller's own tenant document, whose stored definitions carry a `subject` stamp. It is a
304
+ PARAMETER rather than a lookup on purpose: the store is per-tenant, two tenants both holding
305
+ an "Odoo customers" is entirely legitimate, and a module-level cache of somebody's tables is
306
+ how D-169 happened. This function knows nothing about which tenant it is serving.
307
+
308
+ ⭐ THE TEST IS "A DIFFERENT KEY", NOT "IS THIS SUBJECT TAKEN". `_ensure_table_inplace` and
309
+ `ut_ensure` re-run on every boot and every 1800 s resync, adopting their table BY KEY β€” so a
310
+ check phrased "is this subject already claimed?" goes red on the second boot against the very
311
+ table it created on the first, and the container fails to start. Idempotency is not an
312
+ exception to the rule here; it is the common case.
313
+ """
314
+ if not subject:
315
+ return None # silence is not a claim
316
+ for holder in (SUBJECT_CLAIMS, claimed or {}):
317
+ other = holder.get(subject)
318
+ if other and other != key:
319
+ return other
320
+ return None
platform/core/shared_overlay.py CHANGED
@@ -230,6 +230,44 @@ def put_cells(table_key, pid, values, st=None):
230
  return clean
231
 
232
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
233
  def clear_row(table_key, pid, st=None):
234
  """Forget every shared cell on one row. True when there was something to forget."""
235
  row_id = _pid(pid)
 
230
  return clean
231
 
232
 
233
+ def put_rows(table_key, rows, st=None):
234
+ """Write shared cells on MANY rows in ONE store update. Returns `{row_id: {key: value}}`.
235
+
236
+ ⭐⭐ THE WHOLE POINT IS THE *ONE*, AND IT IS NOT AN OPTIMISATION. `put_cells` is one row per
237
+ store write, so a 1,397-row catalog import is 1,397 download-modify-upload cycles against one
238
+ JSON document. This repo has a MEASURED scar for that shape: **18 writes against one document
239
+ under the store's coalescing single-flight landed ZERO while answering 200 eighteen times.**
240
+ Batching is what makes a bulk import land at all, not what makes it fast.
241
+
242
+ ⚠ It deliberately takes the WHOLE SET rather than accepting a stream: a caller that loops over
243
+ this function has simply rebuilt `put_cells` with extra steps, and the failure it reintroduces
244
+ is silent. If the set does not fit in memory it does not fit in this store either β€” that is a
245
+ signal to change substrate, not to chunk.
246
+
247
+ β›” NO CAP HERE, ON PURPOSE. The bound belongs at the DOOR, where a caller identity and a
248
+ reportable refusal exist (`routes_grid.bulk_cells`). A silent ceiling in a store primitive is
249
+ exactly the shape the standing no-cap rule forbids.
250
+ """
251
+ clean = {}
252
+ for pid, values in dict(rows or {}).items():
253
+ row_id = _pid(pid)
254
+ cells_for_row = {str(k): _value(v) for k, v in dict(values or {}).items() if str(k)}
255
+ if cells_for_row:
256
+ clean.setdefault(row_id, {}).update(cells_for_row)
257
+ if not clean:
258
+ return {}
259
+
260
+ def _patch(data):
261
+ for row_id, values in clean.items():
262
+ data['cells'].setdefault(row_id, {}).update(values)
263
+
264
+ # `flush='sync'` because a bulk import must be durable when the call returns: the caller is a
265
+ # script that will report "1,397 rows written" and exit, and an async flush would make that
266
+ # sentence a prediction rather than a fact.
267
+ _write(table_key, _patch, st, flush='sync')
268
+ return clean
269
+
270
+
271
  def clear_row(table_key, pid, st=None):
272
  """Forget every shared cell on one row. True when there was something to forget."""
273
  row_id = _pid(pid)
platform/core/shares.py CHANGED
@@ -43,10 +43,32 @@ SHARES_KEY = 'object_shares'
43
  #: namespace by typo, which would silently grant nothing to nobody and read as "sharing is broken".
44
  KINDS = ('view', 'folder', 'database')
45
 
46
- #: `*` is "everyone who can already open the surface". It is NOT "every account on the platform" β€”
47
- #: the module/table wall runs FIRST and this never widens past it. Spelled as a single character
48
- #: so it can never collide with a username (usernames are lower-case and non-empty by
49
- #: `core/users.py`, and are checked against this explicitly below).
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
50
  EVERYONE = '*'
51
 
52
  ROLES = ('view', 'edit')
 
43
  #: namespace by typo, which would silently grant nothing to nobody and read as "sharing is broken".
44
  KINDS = ('view', 'folder', 'database')
45
 
46
+ #: `*` is "everyone who can already open the surface". It is NOT "every account on the platform".
47
+ #: Spelled as a single character so it can never collide with a username (usernames are lower-case
48
+ #: and non-empty by `core/users.py`, and are checked against this explicitly below).
49
+ #:
50
+ #: β›”β›” **AND WHAT THAT MEANS DEPENDS ON THE KIND β€” THE LINE THIS NOTE USED TO CARRY WAS FALSE FOR
51
+ #: ONE OF THE THREE** (W33-T30, `waves/wave32/sharing-audit.md` S-8). It read *"the module/table
52
+ #: wall runs FIRST and this never widens past it"*, flatly, and the audit's own words for that are
53
+ #: *"the third docstring in this audit describing a check that is not on the path"*. Corrected
54
+ #: here rather than deleted, because the sentence is TRUE of two kinds and the difference is the
55
+ #: whole point:
56
+ #: * `kind='view'` / `kind='folder'` on a GOVERNED module (`customer_data`, `product_data`) β€”
57
+ #: the sentence holds. `require_session` plus the topic's own gate run first, and the
58
+ #: receiver's row scope and hidden-field closure are applied BEFORE any foreign view is
59
+ #: merged, so a grant can only narrow-or-equal what that account could already reach.
60
+ #: * `kind='database'` on a `ut_*` table β€” **THE SENTENCE IS FALSE AND THIS REGISTRY IS THE
61
+ #: ONLY WALL.** `routes_admin._PERM_MODULES` is `("customer_data", "product_data")` and
62
+ #: `_clean_perms` 400s anything else, so no row filter and no hidden field can even be
63
+ #: DECLARED for a user table; `routes_tables.py` makes zero `perm_scope` calls and passes
64
+ #: `hidden_keys=frozenset()`. A `database` grant is therefore ALL-OR-NOTHING β€” every row,
65
+ #: every column β€” and an `*` database grant admits every account in the tenant to all of it.
66
+ #:
67
+ #: ⚠ THAT IS A REAL CAPABILITY, DELIBERATELY KEPT, not a hole to plug in passing. What was wrong
68
+ #: was a docstring promising a second wall that does not exist for this kind; scoping user tables
69
+ #: is booked (S-8), not done. `routes_shares.py`'s module docstring says the same thing at the
70
+ #: OTHER door β€” the audit's fix is "say so at both", and one door saying it is how the next reader
71
+ #: gets the confident half [[one-question-two-normalizers]].
72
  EVERYONE = '*'
73
 
74
  ROLES = ('view', 'edit')
platform/core/store.py CHANGED
@@ -27,12 +27,35 @@ except Exception:
27
  pass
28
 
29
  from huggingface_hub import HfApi, hf_hub_download
30
- from huggingface_hub.utils import EntryNotFoundError, RepositoryNotFoundError
31
-
32
- REPO = os.environ.get('OS_DATA_REPO', 'royal-imports/cfo-os-data')
 
 
 
 
 
 
 
 
 
33
  _FLUSH_DELAY = 2.0 # coalesce a burst of edits into one commit
34
  _FLUSH_MIN_GAP = 20.0 # floor between commits of one key (the 256-commits/hr budget)
35
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
36
  #: WAVE 29 (item 20 / ruling R11, contract C6) β€” THE CHANGE TOKEN'S PROCESS STAMP.
37
  #:
38
  #: `Store`'s revision counter is IN-MEMORY, so a restarted Space would begin again at 0 and a
@@ -48,6 +71,51 @@ def _token():
48
  return os.environ.get('HF_TOKEN') or None
49
 
50
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
51
  class _Projected(dict):
52
  """One top-level value of a PROJECTED read β€” a dict that RAISES for the keys that were dropped.
53
 
@@ -111,6 +179,27 @@ def _project(doc, drop):
111
  return {k: (_Projected(v, drop) if isinstance(v, dict) else v) for k, v in copied.items()}
112
 
113
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
114
  class Store:
115
  """One repo's persistence: JSON-per-key + raw byte paths, with the wave-7 async flush.
116
 
@@ -156,23 +245,51 @@ class Store:
156
  self._key_locks = {}
157
  self._key_locks_guard = threading.Lock()
158
  self._rev_at = {}
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
159
 
160
  def available(self):
161
  """True when a token is configured β€” callers degrade gracefully without one."""
162
  return bool(_token())
163
 
164
  def _download(self, name):
165
- """Parse {name}.json into a dict. Returns {} ONLY when the file is genuinely absent.
166
  Any other failure (network, auth, rate-limit, corrupt JSON) RAISES β€” a transient read
167
  error must never be mistaken for an empty store, or a caller could overwrite real data
168
- with an empty dict (this is what wiped the user registry on restart)."""
 
 
 
 
 
 
 
 
 
 
 
 
 
169
  try:
170
  path = hf_hub_download(self.repo, f'{name}.json', repo_type='dataset',
171
  token=_token(), force_download=True)
172
  with open(path, encoding='utf-8') as f:
173
- return json.load(f)
174
  except (EntryNotFoundError, RepositoryNotFoundError, FileNotFoundError):
175
- return {}
176
  except ValueError as e:
177
  # Wave 18 (R2, found by SESSION D): current huggingface_hub wraps a missing REPO in
178
  # ValueError("Force download failed…") with the 404 as __cause__ β€” so a brand-new
@@ -180,7 +297,7 @@ class Store:
180
  # A genuinely-absent repo is the same fact as a genuinely-absent file: {}.
181
  cause = getattr(e, '__cause__', None)
182
  if isinstance(cause, (EntryNotFoundError, RepositoryNotFoundError)):
183
- return {}
184
  raise
185
 
186
  def get(self, name, fresh=False):
@@ -191,7 +308,7 @@ class Store:
191
  if not fresh and name in self._cache:
192
  return json.loads(json.dumps(self._cache[name])) # hand back a copy
193
  try:
194
- data = self._download(name)
195
  except Exception as e:
196
  try: # lazy import β€” core stays layer-clean
197
  import harness.telemetry as _tel # noqa: PLC0415
@@ -203,6 +320,10 @@ class Store:
203
  return json.loads(json.dumps(self._cache[name]))
204
  return {}
205
  self._cache[name] = data
 
 
 
 
206
  return json.loads(json.dumps(data))
207
 
208
  def get_projection(self, name, drop=()):
@@ -242,7 +363,7 @@ class Store:
242
  if name in self._cache:
243
  return _project(self._cache[name], drop)
244
  try:
245
- data = self._download(name)
246
  except Exception as e:
247
  try: # lazy import β€” core stays layer-clean
248
  import harness.telemetry as _tel # noqa: PLC0415
@@ -256,6 +377,7 @@ class Store:
256
  return {}
257
  with self._lock:
258
  self._cache[name] = data # β›” the WHOLE document, never the projection
 
259
  return _project(data, drop)
260
 
261
  def _key_lock(self, name):
@@ -280,15 +402,25 @@ class Store:
280
  So the dirty mark is re-read AFTER the download, under the short lock, and a key that
281
  became dirty keeps its cache.
282
  """
283
- data = self._download(name) # network, no lock held
284
  with self._lock:
285
  if self._dirty.get(name) and self._owned.get(name) and name in self._cache:
286
- # Our copy is stale by construction: a pending upload is going to overwrite the
287
- # hub anyway, so the fresh read cannot preserve a remote write β€” it can only
288
- # destroy a local one we have already acknowledged. Same argument, and the same
289
- # three conditions, as `update`'s pending-async branch below.
 
 
 
 
 
 
 
 
 
290
  return json.loads(json.dumps(self._cache[name]))
291
  self._cache[name] = data
 
292
  return json.loads(json.dumps(data))
293
 
294
  def exists(self, name):
@@ -345,30 +477,199 @@ class Store:
345
  at = self._rev_at.get(n)
346
  return {'rev': rev, 'updated_at': at, 'token': f'{REV_EPOCH}:{rev}'}
347
 
348
- def _upload(self, name, data):
349
- """The raw hub write. create_repo is confirmed ONCE per instance β€” it was a full extra
350
- API round-trip on every put(), pure latency after the first success."""
351
- api = HfApi(token=_token())
 
 
 
352
  if not self._repo_ok:
353
- api.create_repo(self.repo, repo_type='dataset', private=True, exist_ok=True)
 
354
  self._repo_ok = True
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
355
  buf = io.BytesIO(json.dumps(data, ensure_ascii=False, indent=2).encode('utf-8'))
356
- api.upload_file(path_or_fileobj=buf, path_in_repo=f'{name}.json', repo_id=self.repo,
357
- repo_type='dataset', commit_message=f'update {name}')
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
358
 
359
  def upload_bytes(self, path_in_repo, data, message=None):
360
  """Persist RAW BYTES at an arbitrary path in the tenant dataset (wave-8 C5, documents).
361
  Bytes are never cached in-process: large, cold, fetched on explicit user action."""
362
  if not self.available():
363
  raise RuntimeError('No HF_TOKEN configured β€” persistence is unavailable.')
 
 
 
 
 
364
  with self._lock:
365
- api = HfApi(token=_token())
366
- if not self._repo_ok:
367
- api.create_repo(self.repo, repo_type='dataset', private=True, exist_ok=True)
368
- self._repo_ok = True
369
- api.upload_file(path_or_fileobj=io.BytesIO(data), path_in_repo=path_in_repo,
370
- repo_id=self.repo, repo_type='dataset',
371
- commit_message=message or f'add {path_in_repo}')
372
 
373
  def download_bytes(self, path_in_repo):
374
  """Raw bytes back, or None when absent/unreachable β€” a missing attachment degrades to
@@ -385,13 +686,20 @@ class Store:
385
  return None
386
 
387
  def delete_path(self, path_in_repo):
388
- """Remove a stored file. Absent is SUCCESS β€” deleting what is already gone is the goal."""
 
 
 
 
 
 
389
  if not self.available():
390
  return False
 
391
  try:
392
- HfApi(token=_token()).delete_file(path_in_repo=path_in_repo, repo_id=self.repo,
393
- repo_type='dataset',
394
- commit_message=f'delete {path_in_repo}')
395
  return True
396
  except (EntryNotFoundError, RepositoryNotFoundError):
397
  return True
@@ -399,16 +707,44 @@ class Store:
399
  return False
400
 
401
  def put(self, name, data):
402
- """Persist {name}.json (creates the private repo on first write). Updates the cache."""
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
403
  if not self.available():
404
  raise RuntimeError('No HF_TOKEN configured β€” persistence is unavailable.')
405
- with self._lock:
406
- self._upload(name, data)
407
- self._cache[name] = json.loads(json.dumps(data))
408
- self._owned[name] = True
409
- self._last_flush[name] = _time.monotonic()
410
- self._bump(name) # C6 β€” AFTER the upload: a raise here is not a write
411
- return self._cache[name]
 
 
 
 
 
 
 
412
 
413
  def _flush_worker(self, name):
414
  """Coalescing uploader for one key. Runs while dirty; retries with backoff on failure β€”
@@ -433,11 +769,17 @@ class Store:
433
  this must too; taking `self._lock` first on this path is the deadlock. Both sleeps stay
434
  OUTSIDE the key lock β€” a worker that dozed while holding it would block every writer of
435
  that key for `_FLUSH_MIN_GAP`, which is the freeze this ticket exists to remove.
 
 
 
 
 
 
436
  """
437
  backoff = 2.0
438
  while True:
439
  _time.sleep(_FLUSH_DELAY)
440
- wait, err, snapshot = None, None, None
441
  with self._key_lock(name):
442
  with self._lock:
443
  gap = _time.monotonic() - self._last_flush.get(name, 0.0)
@@ -449,10 +791,17 @@ class Store:
449
  else:
450
  self._dirty[name] = False
451
  snapshot = json.loads(json.dumps(self._cache.get(name, {})))
 
 
 
452
  if wait is None:
453
  try:
454
- self._upload(name, snapshot)
455
  with self._lock:
 
 
 
 
456
  self._last_flush[name] = _time.monotonic()
457
  backoff = 2.0
458
  except Exception as e: # noqa: BLE001
@@ -507,15 +856,28 @@ class Store:
507
  finish, so waiting for its key lock could only turn a partial save into no save at all.
508
  Best-effort is the correct contract here, and it is the one case where taking the lock
509
  would be worse than skipping it.
 
 
 
 
 
 
510
  """
511
  with self._lock:
512
  dirty = [k for k, v in self._dirty.items() if v]
513
  snaps = {k: json.loads(json.dumps(self._cache.get(k, {}))) for k in dirty}
 
514
  for k in dirty:
515
  self._dirty[k] = False
516
  for k, d in snaps.items():
517
  try:
518
- self._upload(k, d)
 
 
 
 
 
 
519
  except Exception:
520
  pass # exiting anyway β€” nothing left to reschedule
521
 
@@ -528,9 +890,19 @@ class Store:
528
  returns immediately; a coalescing worker commits. The strict read still guards the
529
  FIRST async update of a key per process.
530
 
 
 
 
 
 
 
 
 
 
 
531
  ⭐⭐ W31-T13 (D-177(d)) β€” TWO LOCKS, AND THE SPLIT IS THE WHOLE TICKET.
532
  `self._key_lock(name)` is held for the entire read-modify-write, so two writers of one key
533
- still cannot interleave (last-write-wins is the contract; a lost update is not).
534
  `self._lock` β€” the one every `get()` takes β€” is now held only for the microseconds a cache
535
  read or write needs, never across a download or an upload. Before this, one sync write
536
  froze every reader in the process for two network round trips, on a store whose warm
@@ -538,6 +910,13 @@ class Store:
538
  ⚠ Uploading outside `self._lock` is not a new posture: `_flush_worker` has done exactly
539
  that since wave 7, and `_repo_ok` racing is benign because `create_repo` is `exist_ok=True`.
540
  """
 
 
 
 
 
 
 
541
  with self._key_lock(name):
542
  return self._update_locked(name, fn, flush)
543
 
@@ -596,6 +975,15 @@ class Store:
596
  result = fn(data)
597
  data = result if result is not None else data
598
 
 
 
 
 
 
 
 
 
 
599
  if flush == 'async':
600
  if not self.available():
601
  raise RuntimeError('No HF_TOKEN configured β€” persistence is unavailable.')
@@ -606,21 +994,24 @@ class Store:
606
  self._schedule_flush(name)
607
  return data
608
 
609
- # β›” THE SYNC COMMIT, SPELLED OUT HERE RATHER THAN THROUGH `put()`, AND THAT IS A CHOICE.
610
- # `put()` holds `self._lock` across its own `_upload` β€” which is exactly the freeze this
611
- # ticket removes β€” but it is PUBLIC with seven callers outside this file (`users.py`,
612
- # `modules/ar.py`, `modules/map.py`, `runtime.py`, `store_backend.py`,
613
- # `ops/seed_pg_from_hf.py`), and changing its locking changes all of their concurrency
614
- # semantics at once. Duplicating four assignments with this comment beside them is the
615
- # smaller blast radius; if `put` is ever split, this collapses back into a call.
616
  if not self.available():
617
  raise RuntimeError('No HF_TOKEN configured β€” persistence is unavailable.')
618
- self._upload(name, data) # network, no cache lock held
619
  with self._lock:
620
- self._cache[name] = json.loads(json.dumps(data))
 
 
 
621
  self._owned[name] = True
622
  self._last_flush[name] = _time.monotonic()
623
- # β›” AFTER THE UPLOAD, NEVER BEFORE β€” `put()`'s own law. `_upload` raises on failure,
624
  # and a bump on that path would have `revision()` report a change that never landed.
625
  self._bump(name)
626
  # The upload above CARRIED whatever was pending, so the key is no longer ahead of
@@ -629,9 +1020,9 @@ class Store:
629
  # ⚠ AFTER the upload, for the same reason: on a raise the mark MUST survive (the cache
630
  # is still the truth and the row must still be committed by the worker). Clearing it
631
  # first would turn a transient upload error into the very silent loss the branch above
632
- # exists to stop β€” which is why it sits here and not before `_upload`.
633
  self._dirty[name] = False
634
- return data
635
 
636
 
637
  # ------------------------------------------------------------------- instances + module API
 
27
  pass
28
 
29
  from huggingface_hub import HfApi, hf_hub_download
30
+ from huggingface_hub.utils import (EntryNotFoundError, HfHubHTTPError,
31
+ RepositoryNotFoundError)
32
+
33
+ import core.data_binding as data_binding
34
+
35
+ #: β›”β›” D-315 β€” THE BARE DEFAULT IS GONE. This line used to read
36
+ #: `os.environ.get('OS_DATA_REPO', 'royal-imports/cfo-os-data')`, i.e. every Space that was never
37
+ #: handed the key came up bound to tenant #0's REAL business data β€” which is how two different
38
+ #: builds wrote it on 2026-08-14. The value it resolves to is unchanged (see
39
+ #: `data_binding.default_store` for why returning the real id beats `None`); what changed is that
40
+ #: WRITING it is now refused unless this deployment is the one that owns it.
41
+ REPO = os.environ.get('OS_DATA_REPO') or data_binding.default_store()
42
  _FLUSH_DELAY = 2.0 # coalesce a burst of edits into one commit
43
  _FLUSH_MIN_GAP = 20.0 # floor between commits of one key (the 256-commits/hr budget)
44
 
45
+ #: ⭐⭐ D-305 β€” HOW MANY TIMES A WRITE MAY REBASE ONTO A MOVED HEAD BEFORE GIVING UP.
46
+ #:
47
+ #: Bounded rather than unbounded, and the cost is one extra DOWNLOAD per rebase. ⚠ Not an extra
48
+ #: commit: a refused write answers 412 and creates nothing, so only the attempt that finally lands
49
+ #: spends the 256-commits/hr budget. Four is generous for the measured case
50
+ #: (two humans editing one grid) and small enough that a genuinely hot key fails loudly instead of
51
+ #: spinning. β›” Hitting the bound RAISES β€” it never falls through to an unconditional upload, which
52
+ #: is the pre-fix behaviour wearing a retry loop ([[fallback-that-became-the-rule]]).
53
+ _MAX_REBASE = 4
54
+
55
+ #: How many unconfirmed intents one key may hold before the oldest are dropped. See
56
+ #: `_journal_append` for why a cap exists at all and what is reported when it bites.
57
+ _MAX_JOURNAL = 500
58
+
59
  #: WAVE 29 (item 20 / ruling R11, contract C6) β€” THE CHANGE TOKEN'S PROCESS STAMP.
60
  #:
61
  #: `Store`'s revision counter is IN-MEMORY, so a restarted Space would begin again at 0 and a
 
71
  return os.environ.get('HF_TOKEN') or None
72
 
73
 
74
+ class StoreConflict(RuntimeError):
75
+ """A write that could not be landed without overwriting somebody else's (D-305).
76
+
77
+ Raised only after `_MAX_REBASE` rebases, or when no parent commit can be resolved at all. Both
78
+ are "we could not make this safe", and both must reach the caller as a failure: the alternative
79
+ β€” uploading anyway β€” is exactly the lost update this whole mechanism exists to prevent, and it
80
+ would answer 200 while destroying an hour of another person's work.
81
+ """
82
+
83
+
84
+ def _is_conflict(exc):
85
+ """True for the ONE hub failure that means "the branch moved under you".
86
+
87
+ β›” NARROW ON PURPOSE, AND THE PROBE IS WHY. Measured against a scratch dataset repo on
88
+ 2026-08-18: a stale `parent_commit` returns **HTTP 412** with the server message *"The branch
89
+ was updated since you opened this page. Please refresh and try again."*, and the write does NOT
90
+ land. A NONSENSE `parent_commit` returns a **404 `RevisionNotFoundError`** β€” a different fault
91
+ that must not be retried, because rebasing on it would loop against a bad sha. A broad
92
+ `except HfHubHTTPError` would swallow both, plus every 401/429/500, and turn a network blip
93
+ into a silent rebase.
94
+ """
95
+ return getattr(getattr(exc, 'response', None), 'status_code', None) == 412
96
+
97
+
98
+ def _sha_from_path(path):
99
+ """The commit sha out of a hub cache path, or `None` if it does not look like one.
100
+
101
+ ⚠ VALIDATED, NOT ASSUMED. The layout is `…/snapshots/<40-hex>/<file>`, but a caller may have
102
+ an unusual `HF_HOME`, symlinks disabled (this repo's own machines warn about exactly that), or
103
+ a future hub version. A directory name that is not a 40-character hex sha is reported as "I do
104
+ not know", which routes the write through the explicit head lookup β€” the safe branch. Returning
105
+ a bad sha would surface as a 404 `RevisionNotFoundError` on every write instead.
106
+ """
107
+ try:
108
+ parent = Path(path).parent
109
+ if parent.parent.name != 'snapshots':
110
+ return None
111
+ name = parent.name
112
+ if len(name) == 40 and all(c in '0123456789abcdef' for c in name.lower()):
113
+ return name
114
+ except Exception: # noqa: BLE001
115
+ pass
116
+ return None
117
+
118
+
119
  class _Projected(dict):
120
  """One top-level value of a PROJECTED read β€” a dict that RAISES for the keys that were dropped.
121
 
 
179
  return {k: (_Projected(v, drop) if isinstance(v, dict) else v) for k, v in copied.items()}
180
 
181
 
182
+ def is_projected(value):
183
+ """Did this value come back from a PROJECTED read? β€” the TAKEN-PATH predicate (W33-T01/D-213).
184
+
185
+ ⭐⭐ WHY A PREDICATE AND NOT A COUNTER, and it is the whole reason this exists. `all_defs`
186
+ FALLS BACK to the whole read on any failure, silently and by design β€” so a caller that gets the
187
+ right answer has learned nothing about which read produced it. A gate asserting only "the
188
+ definitions came back" passes identically when the projection worked and when it quietly did
189
+ not, which is [[gate-can-report-green-on-nothing]] wearing a green tick.
190
+
191
+ ⭐ This is INTRINSIC rather than instrumented: `_project` stamps every top-level value as a
192
+ `_Projected`, so the evidence rides the document itself. Nothing has to be wrapped, no counter
193
+ can be installed on the wrong object ([[measure-the-real-call]]), and the answer is the same in
194
+ production as it is under a gate β€” a monkey-patched tally is true only while the patch is on.
195
+
196
+ ⚠ It answers about ONE VALUE (a table's definition), not about a document: `_project` returns a
197
+ plain dict OF `_Projected` values, so `is_projected(doc)` is False and `is_projected(doc[key])`
198
+ is True. That is the right grain β€” the thing a route holds and reads is the definition.
199
+ """
200
+ return isinstance(value, _Projected)
201
+
202
+
203
  class Store:
204
  """One repo's persistence: JSON-per-key + raw byte paths, with the wave-7 async flush.
205
 
 
245
  self._key_locks = {}
246
  self._key_locks_guard = threading.Lock()
247
  self._rev_at = {}
248
+ # --- ⭐⭐ D-305 β€” THE TWO FIELDS THAT MAKE A CROSS-PROCESS WRITE SURVIVABLE -----------
249
+ # `_base[name]` the hub commit sha this key's cached document was built ON. Learned for
250
+ # free from the download path and from `CommitInfo.oid` after a write, so
251
+ # it costs no extra round trip in the common case.
252
+ # `_journal[name]` the `fn`s applied to this key since the last CONFIRMED upload β€” our
253
+ # INTENT, not our result. On a conflict the intent is replayed onto the
254
+ # hub's newer document instead of the result being uploaded over it.
255
+ #
256
+ # β›” WHY A JOURNAL AND NOT A RETRY. The writers here are DIFFERENT PEOPLE in DIFFERENT
257
+ # CONTAINERS, and this instance's cache holds an hour of its own edits over a base that has
258
+ # moved. Re-uploading that snapshot β€” which is what a retry does β€” reproduces the loss
259
+ # exactly. Replaying "add this view" onto the fresher document keeps both writers' work,
260
+ # which is the only outcome that answers the defect.
261
+ self._base = {}
262
+ self._journal = {}
263
 
264
  def available(self):
265
  """True when a token is configured β€” callers degrade gracefully without one."""
266
  return bool(_token())
267
 
268
  def _download(self, name):
269
+ """`(document, commit_sha)` for {name}.json. `({}, None)` ONLY when it is genuinely absent.
270
  Any other failure (network, auth, rate-limit, corrupt JSON) RAISES β€” a transient read
271
  error must never be mistaken for an empty store, or a caller could overwrite real data
272
+ with an empty dict (this is what wiped the user registry on restart).
273
+
274
+ ⭐⭐ D-305 β€” THE SHA COMES FROM THE PATH, WHICH COSTS NOTHING. `hf_hub_download` writes into
275
+ `…/snapshots/<commit sha>/<file>`, so the revision the bytes came from is already in the
276
+ return value. Measured 2026-08-18 against a scratch repo: that directory name equals the
277
+ `CommitInfo.oid` of the commit that wrote the file, and equals
278
+ `get_hf_file_metadata(...).commit_hash` β€” which would have cost an extra HTTP round trip on
279
+ the hottest read in the product. **The parent of our write is therefore known by every
280
+ reader, for free.**
281
+
282
+ ⚠ A sha we cannot parse comes back as `None`, and `_commit` resolves that with an explicit
283
+ head lookup rather than uploading without a precondition. "I do not know my parent" must
284
+ take the careful path, never the fast one.
285
+ """
286
  try:
287
  path = hf_hub_download(self.repo, f'{name}.json', repo_type='dataset',
288
  token=_token(), force_download=True)
289
  with open(path, encoding='utf-8') as f:
290
+ return json.load(f), _sha_from_path(path)
291
  except (EntryNotFoundError, RepositoryNotFoundError, FileNotFoundError):
292
+ return {}, None
293
  except ValueError as e:
294
  # Wave 18 (R2, found by SESSION D): current huggingface_hub wraps a missing REPO in
295
  # ValueError("Force download failed…") with the 404 as __cause__ β€” so a brand-new
 
297
  # A genuinely-absent repo is the same fact as a genuinely-absent file: {}.
298
  cause = getattr(e, '__cause__', None)
299
  if isinstance(cause, (EntryNotFoundError, RepositoryNotFoundError)):
300
+ return {}, None
301
  raise
302
 
303
  def get(self, name, fresh=False):
 
308
  if not fresh and name in self._cache:
309
  return json.loads(json.dumps(self._cache[name])) # hand back a copy
310
  try:
311
+ data, sha = self._download(name)
312
  except Exception as e:
313
  try: # lazy import β€” core stays layer-clean
314
  import harness.telemetry as _tel # noqa: PLC0415
 
320
  return json.loads(json.dumps(self._cache[name]))
321
  return {}
322
  self._cache[name] = data
323
+ # D-305: the cached document and the revision it derives from are set TOGETHER, always.
324
+ # A base that can disagree with the cache it describes is worse than no base at all β€”
325
+ # it would make a stale write look like a fresh one and pass the precondition.
326
+ self._base[name] = sha
327
  return json.loads(json.dumps(data))
328
 
329
  def get_projection(self, name, drop=()):
 
363
  if name in self._cache:
364
  return _project(self._cache[name], drop)
365
  try:
366
+ data, sha = self._download(name)
367
  except Exception as e:
368
  try: # lazy import β€” core stays layer-clean
369
  import harness.telemetry as _tel # noqa: PLC0415
 
377
  return {}
378
  with self._lock:
379
  self._cache[name] = data # β›” the WHOLE document, never the projection
380
+ self._base[name] = sha # D-305 β€” set with the cache, never apart from it
381
  return _project(data, drop)
382
 
383
  def _key_lock(self, name):
 
402
  So the dirty mark is re-read AFTER the download, under the short lock, and a key that
403
  became dirty keeps its cache.
404
  """
405
+ data, sha = self._download(name) # network, no lock held
406
  with self._lock:
407
  if self._dirty.get(name) and self._owned.get(name) and name in self._cache:
408
+ # β›”β›” D-305 REWROTE THIS BRANCH'S JUSTIFICATION, NOT ITS BEHAVIOUR β€” AND THE OLD
409
+ # ONE HAD TO GO, because it asserted the defect. It read: *"a pending upload is
410
+ # going to overwrite the hub anyway, so the fresh read cannot preserve a remote
411
+ # write."* That sentence IS the lost update: "we are going to overwrite whatever
412
+ # the other person did" was stated as a reason to stop worrying about it.
413
+ #
414
+ # ⭐ It is no longer true. A pending upload now carries a `parent_commit`
415
+ # precondition, so it CANNOT overwrite a remote write β€” it will be refused and
416
+ # rebased onto it (`_commit`). The branch survives for its OTHER, still-correct
417
+ # reason: this cache holds an async write nobody has flushed yet, and replacing it
418
+ # with the hub's copy destroys a row we have already answered 201 for (D-177).
419
+ # β‡’ keep the local delta here; preserve the REMOTE work at commit time, which is
420
+ # the only place that can see it.
421
  return json.loads(json.dumps(self._cache[name]))
422
  self._cache[name] = data
423
+ self._base[name] = sha
424
  return json.loads(json.dumps(data))
425
 
426
  def exists(self, name):
 
477
  at = self._rev_at.get(n)
478
  return {'rev': rev, 'updated_at': at, 'token': f'{REV_EPOCH}:{rev}'}
479
 
480
+ def _ensure_repo(self):
481
+ """create_repo is confirmed ONCE per instance β€” it was a full extra API round-trip on
482
+ every put(), pure latency after the first success.
483
+
484
+ Lifted out of `_upload` (D-305) because `_commit` has to resolve a parent commit BEFORE the
485
+ first write, and a repo that does not exist yet has no head to resolve.
486
+ """
487
  if not self._repo_ok:
488
+ HfApi(token=_token()).create_repo(self.repo, repo_type='dataset', private=True,
489
+ exist_ok=True)
490
  self._repo_ok = True
491
+
492
+ def _head_sha(self):
493
+ """The repo's current head commit, or `None` if the repo does not exist yet.
494
+
495
+ ⚠ ONE EXTRA ROUND TRIP, AND IT IS THE COLD PATH ONLY. `_base[name]` is populated by every
496
+ download and by every successful write, so this is reached on a first write into a process
497
+ that has never read the key β€” and on the rebase path, where a round trip is already the
498
+ price of correctness.
499
+ """
500
+ try:
501
+ return HfApi(token=_token()).repo_info(self.repo, repo_type='dataset').sha
502
+ except RepositoryNotFoundError:
503
+ return None
504
+
505
+ def _upload(self, name, data, parent):
506
+ """The raw hub write, WITH A PRECONDITION. Returns the new head commit sha.
507
+
508
+ β›” `parent` IS MANDATORY AND HAS NO DEFAULT, deliberately. A default of `None` would make
509
+ "upload unconditionally" the thing you get by forgetting an argument β€” and unconditional
510
+ upload is precisely D-305. Every caller goes through `_commit`, which is the one place
511
+ allowed to decide what the parent is.
512
+ """
513
+ data_binding.check_write(self.repo, f'write {name!r}') # D-315 backstop, see `update`
514
+ self._ensure_repo()
515
  buf = io.BytesIO(json.dumps(data, ensure_ascii=False, indent=2).encode('utf-8'))
516
+ info = HfApi(token=_token()).upload_file(
517
+ path_or_fileobj=buf, path_in_repo=f'{name}.json', repo_id=self.repo,
518
+ repo_type='dataset',
519
+ # D-298: under the local opt-in this suffix is what makes a laptop's commit
520
+ # distinguishable from the live Space's in the store's git history β€” the only audit
521
+ # trail this product has.
522
+ commit_message=f'update {name}' + data_binding.local_commit_suffix(),
523
+ parent_commit=parent)
524
+ return getattr(info, 'oid', None)
525
+
526
+ def _commit(self, name, data, replay=(), max_rebase=None):
527
+ """Land `data` for `name` without overwriting anyone β€” D-305's whole mechanism.
528
+
529
+ ⭐⭐ THE SHAPE, and every step of it is measured rather than assumed (scratch-repo probe,
530
+ 2026-08-18):
531
+
532
+ 1. upload with `parent_commit = the revision our document was built on`;
533
+ 2. the hub answers **412** if the branch moved β€” and the write does NOT land, so there
534
+ is nothing to undo;
535
+ 3. re-read the head, REPLAY our intent (`replay`, the `fn`s applied since our last
536
+ confirmed upload) onto it, and try again against the new parent;
537
+ 4. after `_MAX_REBASE` attempts, RAISE.
538
+
539
+ ⭐ REPLAYING THE INTENT IS WHAT MAKES THIS DIFFERENT FROM A RETRY. Uploading our snapshot
540
+ again would reproduce the loss; re-applying "add this view" to the newer document keeps
541
+ both writers. The one place it changes an answer is a `fn` that ALLOCATES from the document
542
+ β€” `table_store._unique_name` picks the first free display name β€” which on a rebase may pick
543
+ a different name than the one already handed to the caller. That is a rename where the
544
+ pre-fix behaviour was a DELETION, and the trade is stated rather than hidden.
545
+
546
+ β›” NO PARENT β‡’ RAISE, NEVER UPLOAD BLIND. If neither the cached base nor `_head_sha()` can
547
+ name a revision, the safe answer is "this write cannot be made safe", not "make it the old
548
+ way". A fallback that quietly restores the unguarded path is how a fix becomes decorative
549
+ ([[fallback-that-became-the-rule]]).
550
+
551
+ Returns `(document_that_landed, new_sha)` β€” the document may differ from the one passed in,
552
+ because a rebase rebuilds it.
553
+ """
554
+ data_binding.check_write(self.repo, f'write {name!r}')
555
+ # ⚠ `_ensure_repo()` IS NOT CALLED HERE, and that is a deliberate reversal. It was, for one
556
+ # revision, so that a brand-new repo would have a head to name as a parent β€” and it broke
557
+ # three existing gates, because their fixtures replace `_upload` (which is where
558
+ # `create_repo` has always lived) and suddenly found themselves calling the real hub. The
559
+ # ordering works without it: a repo that does not exist resolves no head and no `exists`,
560
+ # so the first write goes out with `parent=None` and `_upload` creates the repo on its way
561
+ # through, exactly as before.
562
+ budget = _MAX_REBASE if max_rebase is None else int(max_rebase)
563
+ for attempt in range(budget + 1):
564
+ with self._lock:
565
+ parent = self._base.get(name)
566
+ if not parent:
567
+ parent = self._head_sha()
568
+ if not parent:
569
+ # A repo with no commits at all: `_upload` creates it on the way through, so this
570
+ # is the genuinely-first write and there is nothing that could be lost.
571
+ # ⚠ NO EM DASH IN THIS MESSAGE, OR THE ONE BELOW. Both are raised, and a raised
572
+ # store message reaches the client through the 503 handler, which puts them inside
573
+ # STANDING RULE 2's scope ("any Python string returned to the client"). `web_prose`
574
+ # caught exactly these two.
575
+ if self.exists(name):
576
+ raise StoreConflict(
577
+ f'cannot resolve a parent commit for {self.repo}:{name}, so this write '
578
+ f'cannot be made safe. Refusing to upload without a precondition: an '
579
+ f'unconditional write is what silently erases another session\'s work.')
580
+ parent = None
581
+ try:
582
+ oid = self._upload(name, data, parent)
583
+ except HfHubHTTPError as e:
584
+ if not _is_conflict(e):
585
+ raise # 401/404/429/500 β€” a different fault, never a rebase
586
+ if attempt >= budget:
587
+ # β›” THE EXHAUSTED CASE RAISES `StoreConflict`, NOT THE RAW 412, and this gate
588
+ # caught the first version doing the latter. A caller cannot be asked to know
589
+ # that one particular HTTP status out of the hub means "your change was not
590
+ # saved and nothing was damaged"; that is a domain fact and it needs a domain
591
+ # type. `from e` keeps the hub's own message in the traceback.
592
+ raise StoreConflict(
593
+ f'{self.repo}:{name} moved under this write {budget + 1} times in a '
594
+ f'row. Refusing to overwrite it. Nothing was saved.') from e
595
+ data = self._rebase(name, replay)
596
+ continue
597
+ with self._lock:
598
+ self._base[name] = oid
599
+ self._journal[name] = []
600
+ return data, oid
601
+ # Unreachable by construction β€” every iteration returns or raises. Kept so that an edit to
602
+ # the loop above cannot make this function fall out and return `None`, which callers would
603
+ # unpack into a crash two frames away from the cause.
604
+ raise StoreConflict(f'{self.repo}:{name}: the write loop exited without committing.')
605
+
606
+ def _journal_append(self, name, fn):
607
+ """Record one intent and return the replay list. THE CALLER HOLDS THE CACHE LOCK.
608
+
609
+ ⚠ CAPPED, BECAUSE AN UNCONFIRMED WRITE STREAM IS AN UNBOUNDED LIST OF CLOSURES. In normal
610
+ operation the cap is unreachable β€” `_commit` empties this every few seconds β€” so it only
611
+ bites during a sustained hub outage, where the alternative is a process that grows until it
612
+ dies. When it bites, the OLDEST intents are dropped and it is reported: a rebase afterwards
613
+ replays the most recent `_MAX_JOURNAL` changes rather than all of them. Stated rather than
614
+ silently enforced, per the standing rule about limits that cannot be removed.
615
+ """
616
+ j = self._journal.setdefault(name, [])
617
+ j.append(fn)
618
+ if len(j) > _MAX_JOURNAL:
619
+ dropped = len(j) - _MAX_JOURNAL
620
+ del j[:dropped]
621
+ try: # lazy import β€” core stays layer-clean
622
+ import harness.telemetry as _tel # noqa: PLC0415
623
+ _tel.error(f'store:journal_full:{name}', StoreConflict(
624
+ f'{self.repo}:{name} has {_MAX_JOURNAL}+ unconfirmed writes; dropped '
625
+ f'{dropped} oldest replay entr(ies). A rebase will re-apply only the most '
626
+ f'recent {_MAX_JOURNAL}.'))
627
+ except Exception: # noqa: BLE001
628
+ pass
629
+ return list(j)
630
+
631
+ def _rebase(self, name, replay):
632
+ """Re-read the hub's newer document and re-apply our intent onto it. Caller holds the key
633
+ lock, so nothing else can be mid-write on this key while we do.
634
+
635
+ ⚠ THE CACHE IS REPLACED WITH THE REBASED RESULT, and forgetting that line would make the
636
+ whole mechanism single-shot: the next write would start from the same stale snapshot and
637
+ conflict again, burning a rebase per write forever. It also has a second, welcome effect β€”
638
+ this instance now HOLDS the other container's work, which is the only moment a process
639
+ whose cache is never re-read gets to learn that anything changed.
640
+ """
641
+ fresh, sha = self._download(name)
642
+ for fn in list(replay):
643
+ out = fn(fresh)
644
+ fresh = out if out is not None else fresh
645
+ with self._lock:
646
+ self._cache[name] = json.loads(json.dumps(fresh))
647
+ self._base[name] = sha
648
+ try: # lazy import β€” core stays layer-clean
649
+ import harness.telemetry as _tel # noqa: PLC0415
650
+ _tel.error(f'store:rebase:{name}',
651
+ StoreConflict(f'{self.repo}:{name} moved; replayed {len(replay)} change(s)'))
652
+ except Exception: # noqa: BLE001
653
+ pass
654
+ return fresh
655
 
656
  def upload_bytes(self, path_in_repo, data, message=None):
657
  """Persist RAW BYTES at an arbitrary path in the tenant dataset (wave-8 C5, documents).
658
  Bytes are never cached in-process: large, cold, fetched on explicit user action."""
659
  if not self.available():
660
  raise RuntimeError('No HF_TOKEN configured β€” persistence is unavailable.')
661
+ # D-315: a blob is still this tenant's data. ⚠ No `parent_commit` here and that is correct,
662
+ # not an omission: a blob is written at its own path and read whole, so two writers cannot
663
+ # silently merge into one damaged document the way `{name}.json` can. The concurrency
664
+ # hazard D-305 describes is specific to the shared JSON blob.
665
+ data_binding.check_write(self.repo, f'write bytes {path_in_repo!r}')
666
  with self._lock:
667
+ self._ensure_repo()
668
+ HfApi(token=_token()).upload_file(
669
+ path_or_fileobj=io.BytesIO(data), path_in_repo=path_in_repo,
670
+ repo_id=self.repo, repo_type='dataset',
671
+ commit_message=(message or f'add {path_in_repo}')
672
+ + data_binding.local_commit_suffix())
 
673
 
674
  def download_bytes(self, path_in_repo):
675
  """Raw bytes back, or None when absent/unreachable β€” a missing attachment degrades to
 
686
  return None
687
 
688
  def delete_path(self, path_in_repo):
689
+ """Remove a stored file. Absent is SUCCESS β€” deleting what is already gone is the goal.
690
+
691
+ β›” THE REFUSAL IS RAISED, NOT SWALLOWED INTO THE `return False` BELOW, and the distinction
692
+ matters: `False` here means "the delete did not happen", which several callers treat as a
693
+ tolerable no-op. "You are not allowed to touch this store" is not a no-op β€” it is the
694
+ thing the operator has to be told.
695
+ """
696
  if not self.available():
697
  return False
698
+ data_binding.check_write(self.repo, f'delete {path_in_repo!r}')
699
  try:
700
+ HfApi(token=_token()).delete_file(
701
+ path_in_repo=path_in_repo, repo_id=self.repo, repo_type='dataset',
702
+ commit_message=f'delete {path_in_repo}' + data_binding.local_commit_suffix())
703
  return True
704
  except (EntryNotFoundError, RepositoryNotFoundError):
705
  return True
 
707
  return False
708
 
709
  def put(self, name, data):
710
+ """Persist {name}.json (creates the private repo on first write). Updates the cache.
711
+
712
+ ⭐ D-305 β€” `put` NOW TAKES THE KEY LOCK AND GOES THROUGH `_commit`, and both halves are
713
+ required. Without the precondition this method is the hole in the whole mechanism: seven
714
+ callers outside this file reach it (`users.py`, `modules/ar.py`, `modules/map.py`,
715
+ `runtime.py`, `store_backend.py`, `ops/seed_pg_from_hf.py`) and `users.json` is a document
716
+ every account shares, so an unguarded whole-file `put` clobbers exactly as `update` used to.
717
+
718
+ ⚠ THE CACHE LOCK NO LONGER SPANS THE NETWORK, which is a change to this method's
719
+ concurrency and is deliberate. The old body held `self._lock` β€” the lock every `get()`
720
+ takes β€” across the upload, so one `put` froze every reader for a round trip (D-181's
721
+ shape); with up to `_MAX_REBASE` extra round trips now possible that freeze would get
722
+ strictly worse. The per-key write lock gives the callers a STRONGER guarantee than they had
723
+ (two writers of one key can no longer interleave) while readers stop waiting. Lock order is
724
+ key-then-cache, as everywhere else in this class.
725
+
726
+ ⚠ REPLAY SEMANTICS FOR A WHOLE-DOCUMENT WRITE ARE LAST-WRITE-WINS, STATED RATHER THAN
727
+ IMPLIED. `put` means "the document is this now", so its journal entry replaces rather than
728
+ merges. What the precondition buys here is that the replacement is applied to a document we
729
+ have actually SEEN β€” and that any `update` queued behind it replays on top β€” instead of
730
+ being uploaded over an hour of unseen work.
731
+ """
732
  if not self.available():
733
  raise RuntimeError('No HF_TOKEN configured β€” persistence is unavailable.')
734
+ data_binding.check_write(self.repo, f'put {name!r}')
735
+ with self._key_lock(name):
736
+ def _replace(_current):
737
+ return json.loads(json.dumps(data))
738
+ with self._lock:
739
+ replay = self._journal_append(name, _replace)
740
+ landed, _oid = self._commit(name, data, replay)
741
+ with self._lock:
742
+ self._cache[name] = json.loads(json.dumps(landed))
743
+ self._owned[name] = True
744
+ self._last_flush[name] = _time.monotonic()
745
+ self._dirty[name] = False
746
+ self._bump(name) # C6 β€” AFTER the upload: a raise here is not a write
747
+ return self._cache[name]
748
 
749
  def _flush_worker(self, name):
750
  """Coalescing uploader for one key. Runs while dirty; retries with backoff on failure β€”
 
769
  this must too; taking `self._lock` first on this path is the deadlock. Both sleeps stay
770
  OUTSIDE the key lock β€” a worker that dozed while holding it would block every writer of
771
  that key for `_FLUSH_MIN_GAP`, which is the freeze this ticket exists to remove.
772
+
773
+ ⭐⭐ D-305 β€” THIS WORKER IS ALSO WHERE MOST CROSS-CONTAINER LOSS HAPPENED, because the
774
+ table workspace (views, fields, folders, overlays) is written `flush='async'`. It uploads a
775
+ SNAPSHOT of a cache that may be minutes old, so it takes the `replay` journal with it: on a
776
+ 412 the snapshot is thrown away and the journalled `fn`s are re-applied to the hub's newer
777
+ document instead. The journal is cleared inside `_commit`, only on success.
778
  """
779
  backoff = 2.0
780
  while True:
781
  _time.sleep(_FLUSH_DELAY)
782
+ wait, err, snapshot, replay = None, None, None, ()
783
  with self._key_lock(name):
784
  with self._lock:
785
  gap = _time.monotonic() - self._last_flush.get(name, 0.0)
 
791
  else:
792
  self._dirty[name] = False
793
  snapshot = json.loads(json.dumps(self._cache.get(name, {})))
794
+ # Snapshotted WITH the document and under the same lock: a journal that
795
+ # could drift from the snapshot it describes would replay the wrong set.
796
+ replay = list(self._journal.get(name) or ())
797
  if wait is None:
798
  try:
799
+ landed, _oid = self._commit(name, snapshot, replay)
800
  with self._lock:
801
+ # A rebase rebuilt the document, so the cache must take the result β€”
802
+ # `_rebase` already wrote it, and this keeps the no-conflict path
803
+ # identical to it rather than leaving two ways for the cache to be set.
804
+ self._cache[name] = json.loads(json.dumps(landed))
805
  self._last_flush[name] = _time.monotonic()
806
  backoff = 2.0
807
  except Exception as e: # noqa: BLE001
 
856
  finish, so waiting for its key lock could only turn a partial save into no save at all.
857
  Best-effort is the correct contract here, and it is the one case where taking the lock
858
  would be worse than skipping it.
859
+
860
+ ⭐ D-305 β€” IT STILL GOES THROUGH `_commit`, so the last write of a dying process cannot be
861
+ the one that erases somebody. Exiting is not a licence to overwrite: if the branch has
862
+ moved, this rebases like any other write, and if it cannot, it gives up rather than
863
+ clobbering. `_commit`'s own `data_binding` check applies too, so a container that was never
864
+ allowed to write this store does not get one last unguarded upload on the way out.
865
  """
866
  with self._lock:
867
  dirty = [k for k, v in self._dirty.items() if v]
868
  snaps = {k: json.loads(json.dumps(self._cache.get(k, {}))) for k in dirty}
869
+ replays = {k: list(self._journal.get(k) or ()) for k in dirty}
870
  for k in dirty:
871
  self._dirty[k] = False
872
  for k, d in snaps.items():
873
  try:
874
+ # ⚠ ONE REBASE, NOT `_MAX_REBASE`, and the docstring above is the argument: this
875
+ # runs at interpreter exit with the process going away. The full budget is up to
876
+ # four extra downloads and five upload attempts PER DIRTY KEY on a path that may be
877
+ # killed at any moment, which turns a best-effort save into a hang. One attempt
878
+ # keeps the safety this method exists for (a conflicting write still rebases rather
879
+ # than clobbering) at a cost the exit path can actually pay.
880
+ self._commit(k, d, replays.get(k, ()), max_rebase=1)
881
  except Exception:
882
  pass # exiting anyway β€” nothing left to reschedule
883
 
 
890
  returns immediately; a coalescing worker commits. The strict read still guards the
891
  FIRST async update of a key per process.
892
 
893
+ β›”β›” D-305 β€” "LAST-WRITE-WINS IS THE CONTRACT" IS NO LONGER TRUE ACROSS PROCESSES, AND THE
894
+ SENTENCE BELOW USED TO SAY IT WITHOUT THE QUALIFIER. Two writers in ONE process cannot lose
895
+ an update β€” they share `self._cache`, which is why every gate in this repo passed over the
896
+ defect. Two writers in two CONTAINERS could, and did: measured on tenant #0's live store in
897
+ commit `dee7481a77`, where one session's write rolled the document back an hour and cost a
898
+ real user 3 views, 49 typed cell values and a field id. The commit now carries a
899
+ `parent_commit` precondition and rebases (`_commit`), so the contract is: **within a
900
+ process, last write wins; across processes, a write that would overwrite unseen work is
901
+ refused and re-applied.** A lost update is not the contract at either grain.
902
+
903
  ⭐⭐ W31-T13 (D-177(d)) β€” TWO LOCKS, AND THE SPLIT IS THE WHOLE TICKET.
904
  `self._key_lock(name)` is held for the entire read-modify-write, so two writers of one key
905
+ still cannot interleave.
906
  `self._lock` β€” the one every `get()` takes β€” is now held only for the microseconds a cache
907
  read or write needs, never across a download or an upload. Before this, one sync write
908
  froze every reader in the process for two network round trips, on a store whose warm
 
910
  ⚠ Uploading outside `self._lock` is not a new posture: `_flush_worker` has done exactly
911
  that since wave 7, and `_repo_ok` racing is benign because `create_repo` is `exist_ok=True`.
912
  """
913
+ # β›” D-315 β€” REFUSED AT THE ACCEPTANCE POINT, NOT AT THE UPLOAD. `_upload` carries the same
914
+ # check as a backstop, but refusing only there would let the async branch accept the
915
+ # mutation into the cache, answer 200, mark the key dirty, and hand the flush worker a
916
+ # write that can never succeed β€” which retries with backoff forever and reports the refusal
917
+ # to nobody. Refuse before anything is accepted, and the caller gets a 503 that tells the
918
+ # truth: no change was saved.
919
+ data_binding.check_write(self.repo, f'update {name!r}')
920
  with self._key_lock(name):
921
  return self._update_locked(name, fn, flush)
922
 
 
975
  result = fn(data)
976
  data = result if result is not None else data
977
 
978
+ # ⭐⭐ D-305 β€” THE INTENT IS RECORDED, NOT JUST THE RESULT, and this one line is what makes
979
+ # a conflict survivable. `data` is our answer over OUR base; `fn` is what we MEANT, and
980
+ # only the intent can be re-applied to somebody else's newer document. The journal holds
981
+ # every `fn` since the last confirmed upload β€” which for an async key is a whole flush
982
+ # window of edits, not one β€” and `_commit` clears it only when the hub has accepted them.
983
+ # ⚠ Appended AFTER `fn` ran: a `fn` that raised changed nothing and must not be replayed.
984
+ with self._lock:
985
+ replay = self._journal_append(name, fn)
986
+
987
  if flush == 'async':
988
  if not self.available():
989
  raise RuntimeError('No HF_TOKEN configured β€” persistence is unavailable.')
 
994
  self._schedule_flush(name)
995
  return data
996
 
997
+ # οΏ½οΏ½οΏ½ THE SYNC COMMIT. It used to be spelled out here rather than calling `put()`, because
998
+ # `put()` held `self._lock` across its own upload β€” the freeze W31-T13 removed β€” and is
999
+ # PUBLIC with seven callers outside this file whose concurrency semantics changing it would
1000
+ # alter. D-305 made that duplication untenable: an unguarded whole-file write is the hole
1001
+ # the precondition exists to close, so BOTH now go through `_commit`, which is the single
1002
+ # place that resolves a parent, rebases and clears the journal. `put` still keeps its own
1003
+ # body (its op REPLACES rather than merges), but neither can upload unconditionally.
1004
  if not self.available():
1005
  raise RuntimeError('No HF_TOKEN configured β€” persistence is unavailable.')
1006
+ landed, _oid = self._commit(name, data, replay) # network, no cache lock held
1007
  with self._lock:
1008
+ # ⚠ `landed`, NOT `data`. A rebase rebuilt the document from the hub's newer copy, so
1009
+ # caching what we TRIED to write would leave the cache diverged from what is stored β€”
1010
+ # and the next write would rebase off that divergence forever.
1011
+ self._cache[name] = json.loads(json.dumps(landed))
1012
  self._owned[name] = True
1013
  self._last_flush[name] = _time.monotonic()
1014
+ # β›” AFTER THE UPLOAD, NEVER BEFORE β€” `put()`'s own law. `_commit` raises on failure,
1015
  # and a bump on that path would have `revision()` report a change that never landed.
1016
  self._bump(name)
1017
  # The upload above CARRIED whatever was pending, so the key is no longer ahead of
 
1020
  # ⚠ AFTER the upload, for the same reason: on a raise the mark MUST survive (the cache
1021
  # is still the truth and the row must still be committed by the worker). Clearing it
1022
  # first would turn a transient upload error into the very silent loss the branch above
1023
+ # exists to stop β€” which is why it sits here and not before the commit.
1024
  self._dirty[name] = False
1025
+ return landed
1026
 
1027
 
1028
  # ------------------------------------------------------------------- instances + module API
platform/core/user_tables.py CHANGED
@@ -336,7 +336,7 @@ def limit_report(table_key, st=None):
336
  'cause': f'this database is EDITABLE, so its rows live in the shared `{STORE_KEY}` '
337
  f'document that every request copies; at the widest shipped row (0.318 KB) '
338
  f'{cap:,} rows is 19.1 MB against a 32 MB per-table budget',
339
- 'recommendation': 'connect it to a source instead β€” connected data is served '
340
  'read-through from the mirror and is not bounded (R6); for typed data '
341
  'the increment is D-87, a per-table row key, not a bigger number',
342
  }
@@ -439,9 +439,15 @@ def records_mutable(table_key, st=None):
439
  #: waves because `verify_fields_contract` diffed the client's offer against the ODOO grid's
440
  #: acceptance and never against this set. Wave-27 item 13 adds that derived leg, which is what
441
  #: found it (measured, not reviewed: `_clean_field({'type':'image'})` returned None).
 
 
 
 
 
 
442
  UT_FIELD_TYPES = {'text', 'select', 'multiselect', 'user', 'int', 'currency', 'pct', 'date',
443
  'checkbox', 'phone', 'email', 'url', 'rating', 'automation', 'json',
444
- 'link', 'rollup', 'code', 'image', 'formula'}
445
  #: A cell is still a scalar on the wire (`Row` values are strings/numbers) β€” what this bounds is
446
  #: the DOCUMENT inside it. Source data is one already-paid provider response; keep it whole so a
447
  #: provider field is never silently discarded merely because the response is rich. 32 MiB matches
@@ -902,6 +908,19 @@ def clean_fields(raw):
902
  entry = None
903
  elif fx is not None:
904
  entry['formula'] = fx
 
 
 
 
 
 
 
 
 
 
 
 
 
905
  if entry is None:
906
  seen.discard(key)
907
  continue
@@ -949,13 +968,26 @@ def all_defs(st=None):
949
  fake store installed by a gate, a `_Lent` handle serving a document somebody else read). The
950
  fallback is CORRECT-but-slow by construction β€” it can only ever return MORE than was asked for
951
  β€” which is the one direction a fallback here is allowed to be wrong in.
 
 
 
 
 
 
 
 
 
 
 
 
952
  """
953
  try:
954
  return dict(_st(st).get_projection(STORE_KEY, drop=DEF_ONLY_DROPS) or {})
955
- except AttributeError:
956
- return all_tables(st)
957
  except Exception:
958
- return {}
 
 
 
959
 
960
 
961
  def get(table_key, st=None):
@@ -1651,6 +1683,308 @@ def _clean_rollup(raw):
1651
  return out
1652
 
1653
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1654
  def _clean_field(raw, previous=None):
1655
  """One field dict β†’ the stored shape, or None. The single validator for create AND patch, so
1656
  a column cannot be typed one way on the way in and another on the way back."""
@@ -1886,6 +2220,28 @@ def _clean_field(raw, previous=None):
1886
  out['formula'] = fx
1887
  elif ftype == 'formula':
1888
  return None
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1889
  return out
1890
 
1891
 
@@ -2081,6 +2437,16 @@ def flow_bound(bag, st=None):
2081
  lives in `aios_grid._clean_automation` β€” this guards the WRITE doors only.)"""
2082
  if not isinstance(bag, dict):
2083
  return True # no bag, no law β€” an ordinary column
 
 
 
 
 
 
 
 
 
 
2084
  flow = str(bag.get('flowId') or '').strip()
2085
  if not flow:
2086
  return False
@@ -2386,6 +2752,41 @@ def patch_cells(table_key, row_id, values, st=None):
2386
  return True
2387
 
2388
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
2389
  def patch_link_cell(table_key, row_id, fkey, value, st=None):
2390
  """Persist one user-picked Link cell in the shared row and return its canonical id string.
2391
 
@@ -2647,6 +3048,28 @@ def lend(st=None, **docs):
2647
  return _Lent(_st(st), {k: v for k, v in docs.items() if v is not None})
2648
 
2649
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
2650
  def nav_entries(viewer=None, is_admin=False, st=None):
2651
  """Registry-SHAPED dicts so the nav can render user tables beside real modules with no
2652
  special-casing: the same keys the flat nav reads (`key`, `label`, `source`).
 
336
  'cause': f'this database is EDITABLE, so its rows live in the shared `{STORE_KEY}` '
337
  f'document that every request copies; at the widest shipped row (0.318 KB) '
338
  f'{cap:,} rows is 19.1 MB against a 32 MB per-table budget',
339
+ 'recommendation': 'connect it to a source instead. Connected data is served '
340
  'read-through from the mirror and is not bounded (R6); for typed data '
341
  'the increment is D-87, a per-table row key, not a bigger number',
342
  }
 
439
  #: waves because `verify_fields_contract` diffed the client's offer against the ODOO grid's
440
  #: acceptance and never against this set. Wave-27 item 13 adds that derived leg, which is what
441
  #: found it (measured, not reviewed: `_clean_field({'type':'image'})` returned None).
442
+ #: ⭐ WAVE-34 (owner ruling R13) β€” `ai_enrich` JOINS, and it lands HERE FIRST rather than last.
443
+ #: The block above records that `image` reached four of the five surfaces and missed this one, so
444
+ #: the client offered a column the server deleted on the next read. `ai_enrich` is added to this
445
+ #: set in `W34-T51` (server) while the CLIENT half waits for `W34-T53`, which is the safe order:
446
+ #: a kind the server accepts and the client does not offer is invisible, whereas a kind the client
447
+ #: offers and the server refuses is a column created, named, configured and gone.
448
  UT_FIELD_TYPES = {'text', 'select', 'multiselect', 'user', 'int', 'currency', 'pct', 'date',
449
  'checkbox', 'phone', 'email', 'url', 'rating', 'automation', 'json',
450
+ 'link', 'rollup', 'code', 'image', 'formula', 'ai_enrich'}
451
  #: A cell is still a scalar on the wire (`Row` values are strings/numbers) β€” what this bounds is
452
  #: the DOCUMENT inside it. Source data is one already-paid provider response; keep it whole so a
453
  #: provider field is never silently discarded merely because the response is rich. 32 MiB matches
 
908
  entry = None
909
  elif fx is not None:
910
  entry['formula'] = fx
911
+ # ⭐⭐ WAVE-34 (R13) β€” the AI ENRICHMENT bag rides THIS door too, and it is the SIXTH time
912
+ # this seam has been fixed after the fact (`pinned`, `code`, `link`/`rollup`, `formula`,
913
+ # `agg`). It is written in the same change as `_clean_field`'s arm rather than a wave
914
+ # later, which is the only difference between this entry and the five above it.
915
+ if entry is not None:
916
+ ae = (_clean_ai_enrich(f.get('aiEnrich'))
917
+ if f.get('aiEnrich') is not None else None)
918
+ if (ftype == 'ai_enrich') != (ae is not None):
919
+ entry = None
920
+ elif ae is not None:
921
+ entry['aiEnrich'] = ae
922
+ # C3: derived, never typed beside the config. Same call as the other door.
923
+ entry['automation'] = _field_agent_binding(entry)
924
  if entry is None:
925
  seen.discard(key)
926
  continue
 
968
  fake store installed by a gate, a `_Lent` handle serving a document somebody else read). The
969
  fallback is CORRECT-but-slow by construction β€” it can only ever return MORE than was asked for
970
  β€” which is the one direction a fallback here is allowed to be wrong in.
971
+
972
+ β›”β›” W33-T01 (D-213) β€” **THAT PARAGRAPH WAS FALSE ON THIS FUNCTION'S OWN BODY UNTIL 2026-08-14,
973
+ AND THE LIE WAS THE DANGEROUS DIRECTION.** Only `AttributeError` fell back; every other
974
+ exception returned `{}` β€” the exact silent-empty answer the paragraph ABOVE it forbids, three
975
+ lines earlier, in bold. `/nav`, `/nav/prefs` and `routes_shares` would have rendered ZERO
976
+ databases and looked entirely correct doing it ([[empty-answer-vs-unfinished-answer]]).
977
+ Nothing had noticed because the projection does not fail on a healthy store β€” a fallback is
978
+ dead code until the day it is the only code running. **Every failure now falls back to the
979
+ whole read**, which keeps `all_tables`' own `except: return {}` as the single backstop instead
980
+ of two competing ones ([[one-question-two-normalizers]]). This ticket puts the table WALL on
981
+ this function, so an empty answer here stops being a blank nav and becomes a 404 on every
982
+ database in the tenant.
983
  """
984
  try:
985
  return dict(_st(st).get_projection(STORE_KEY, drop=DEF_ONLY_DROPS) or {})
 
 
986
  except Exception:
987
+ # ⚠ `AttributeError` (a store too old to project) is not special-cased any more β€” it was
988
+ # only ever ONE of the ways this can fail, and singling it out is what made the other ways
989
+ # silent. `all_tables` is correct for all of them.
990
+ return all_tables(st)
991
 
992
 
993
  def get(table_key, st=None):
 
1683
  return out
1684
 
1685
 
1686
+ # ---------------------------------------------------------------------------------------------
1687
+ # THE AI ENRICHMENT COLUMN (wave 34, owner ruling R13)
1688
+ # ---------------------------------------------------------------------------------------------
1689
+ # R13: *"a field kind called AI enrichment: a prompt per row that populates text. Detailed
1690
+ # configuration in the field's own config, including a token-usage limit."*
1691
+ #
1692
+ # β›” POSTURE: `link`/`rollup`/`formula`'s, NOT `code`'s. The bag DEFINES the column β€” a prompt is
1693
+ # the only thing that can ever produce a value here β€” so the type/bag pairing is enforced BOTH
1694
+ # WAYS and either half alone refuses the field. An `ai_enrich` column with no prompt is a column
1695
+ # that renders, sorts, filters and can never hold anything, which is the silent kind of broken
1696
+ # every other bag on this door is paired to prevent.
1697
+ #
1698
+ # ⚠ AND THE VALUE IS STORED, NOT COMPUTED. `formula` is client-recomputed on every paint and
1699
+ # `rollup` is read-through; neither persists. An enrichment that vanished on reload would not be
1700
+ # the feature (PRD assumption A6), so an `ai_enrich` cell is an ordinary stored string, a human
1701
+ # may type over it, and it is deliberately NOT in `is_computed_cell`.
1702
+
1703
+ #: The bag's own key set, read by BOTH the cleaner and the reporter below so they cannot disagree
1704
+ #: about what "unknown" means. β›” ONE constant, because T51's contract is that an unknown key is
1705
+ #: DROPPED **and NAMED** β€” two hand-maintained lists would eventually drop a key the report did
1706
+ #: not mention, which reads to a user as the config silently not saving.
1707
+ AI_ENRICH_KEYS = ('prompt', 'model', 'maxTokens', 'trigger', 'overwrite')
1708
+ #: When the column runs. `manual` = only when a person asks; `on_change` = when a referenced cell
1709
+ #: moves; `schedule` = on a cadence. The community ask R13 cites ("enrichment on a schedule") is
1710
+ #: the third of these, so it is in the vocabulary from the start rather than added later.
1711
+ AI_ENRICH_TRIGGER_MODES = ('manual', 'on_change', 'schedule')
1712
+ #: What an AUTOMATIC run may replace. β›” A HUMAN-EDITED CELL IS NEVER OVERWRITTEN AND THAT IS NOT
1713
+ #: IN THIS VOCABULARY β€” it is a law (`ai_enrich_may_write`), not a policy, so no configuration can
1714
+ #: turn it off. These three decide only what happens to cells the AGENT itself wrote.
1715
+ AI_ENRICH_OVERWRITE = ('blank', 'stale', 'always')
1716
+ #: Bounded because the ceiling is the feature (R13). The floor is one useful sentence; the roof is
1717
+ #: what one cell can cost before the run has to say so out loud.
1718
+ AI_ENRICH_MIN_TOKENS, AI_ENRICH_MAX_TOKENS = 16, 4000
1719
+ AI_ENRICH_DEFAULT_TOKENS = 300
1720
+ #: A prompt is a template, not an essay. Long enough for real instructions plus a few `{field}`
1721
+ #: tokens, short enough that the per-cell input cost is predictable.
1722
+ AI_ENRICH_PROMPT_MAX = 2000
1723
+ #: `cron` is stored as an opaque bounded string. ⚠ It is NOT parsed here on purpose: this module
1724
+ #: owns the field contract, and the scheduler that reads it owns the cadence vocabulary. Parsing
1725
+ #: it in two places is how one door starts refusing a cadence the other accepts.
1726
+ AI_ENRICH_CRON_MAX = 120
1727
+
1728
+
1729
+ def _clean_ai_enrich(raw):
1730
+ """The `ai_enrich` config bag β†’ the stored shape, or None if the column cannot run.
1731
+
1732
+ ⚠ Every key is normalised to a canonical form, so a definition saved and re-read is
1733
+ byte-identical (`_clean_ai_enrich(_clean_ai_enrich(x)) == _clean_ai_enrich(x)`). That
1734
+ idempotence is what makes the round-trip clause testable rather than a claim.
1735
+ """
1736
+ if not isinstance(raw, dict):
1737
+ return None
1738
+ prompt = str(raw.get('prompt') or '').strip()[:AI_ENRICH_PROMPT_MAX]
1739
+ if not prompt:
1740
+ return None # the prompt IS the column; see the posture note above
1741
+ out = {'prompt': prompt}
1742
+ # The model is OPTIONAL and an unknown name is DROPPED rather than refused: it selects a
1743
+ # PROVIDER for a column that already knows what to ask, so a stale name should fall back to
1744
+ # the ladder's own order, never destroy the column. (`code`'s language argument, same shape.)
1745
+ model = str(raw.get('model') or '').strip().lower()[:60]
1746
+ if model:
1747
+ out['model'] = model
1748
+ tokens = raw.get('maxTokens')
1749
+ # `bool` is an `int` subclass β€” `True` would otherwise validate as a 1-token ceiling.
1750
+ if isinstance(tokens, bool) or not isinstance(tokens, (int, float)):
1751
+ tokens = AI_ENRICH_DEFAULT_TOKENS
1752
+ out['maxTokens'] = max(AI_ENRICH_MIN_TOKENS, min(int(tokens), AI_ENRICH_MAX_TOKENS))
1753
+ trig = raw.get('trigger') if isinstance(raw.get('trigger'), dict) else {}
1754
+ mode = str(trig.get('mode') or '').strip().lower()
1755
+ trigger = {'mode': mode if mode in AI_ENRICH_TRIGGER_MODES else 'manual'}
1756
+ # β›” `cron` is kept ONLY on a scheduled column. A cadence stored under `manual` is a setting
1757
+ # the user can see and nothing will ever read β€” the shape D-246 wears on the Odoo sync, where
1758
+ # a `manual` preset does nothing because no branch reads it.
1759
+ cron = str(trig.get('cron') or '').strip()[:AI_ENRICH_CRON_MAX]
1760
+ if trigger['mode'] == 'schedule' and cron:
1761
+ trigger['cron'] = cron
1762
+ out['trigger'] = trigger
1763
+ overwrite = str(raw.get('overwrite') or '').strip().lower()
1764
+ out['overwrite'] = overwrite if overwrite in AI_ENRICH_OVERWRITE else 'blank'
1765
+ return out
1766
+
1767
+
1768
+ def ai_enrich_dropped_keys(raw):
1769
+ """The config keys `_clean_ai_enrich` will NOT keep, sorted β€” the NAMING half of T51's
1770
+ "an unknown config key is DROPPED and NAMED, never silently eaten".
1771
+
1772
+ β›” IT EXISTS BECAUSE EVERY OTHER BAG CLEANER IN THIS FILE HAS NO ERROR CHANNEL. `_clean_link`,
1773
+ `_clean_rollup`, `_clean_profile` and `_clean_format` all return `dict | None` and drop
1774
+ unknown keys in silence β€” `verify_fields_contract` even asserts that they do. Copying one of
1775
+ them as a template gets the DROP right and the NAMING wrong by construction, so the report is
1776
+ a separate function reading the SAME `AI_ENRICH_KEYS` constant rather than a second list.
1777
+
1778
+ ⚠ It reports UNKNOWN keys only. A known key whose VALUE was normalised (an out-of-range
1779
+ ceiling clamped, an unknown model dropped) is not a surprise worth a sentence β€” the stored
1780
+ definition comes straight back to the editor, so the user sees the accepted value itself.
1781
+ """
1782
+ if not isinstance(raw, dict):
1783
+ return []
1784
+ return sorted(str(k) for k in raw if k not in AI_ENRICH_KEYS)
1785
+
1786
+
1787
+ def ai_enrich_fields(defn):
1788
+ """Every `ai_enrich` column in this table definition, in declaration order.
1789
+
1790
+ One reader for "which columns are field agents", so the Agents module (contract C3), the
1791
+ runner and the scheduler cannot disagree about the set ([[one-evaluator-per-question]]).
1792
+ """
1793
+ return [f for f in ((defn or {}).get('fields') or [])
1794
+ if isinstance(f, dict) and f.get('type') == 'ai_enrich'
1795
+ and isinstance(f.get('aiEnrich'), dict)]
1796
+
1797
+
1798
+ def _field_agent_binding(field):
1799
+ """Contract C3's `field.automation` bag, DERIVED from the column rather than typed beside it.
1800
+
1801
+ β›”β›” THIS IS DERIVED ON PURPOSE AND IT IS THE ONE DESIGN CALL IN T51 WORTH ARGUING WITH.
1802
+ C3 as drafted has F *store* `{kind, table, field, trigger}` β€” but `field` is already
1803
+ `field['key']` and `trigger` is already `aiEnrich['trigger']`, so two of its four keys would
1804
+ be a second copy of a fact this same dict already carries. Two copies of one fact is
1805
+ [[one-question-two-normalizers]] with a guaranteed drift date: the first PATCH that changes
1806
+ the trigger through the enrichment editor and not through the automation editor.
1807
+ Computing it inside the SINGLE validator both write doors run means they cannot diverge.
1808
+
1809
+ ⚠ `table` IS ABSENT, and that is the deviation E must know about: this function is handed one
1810
+ field dict and no table context, while `_clean_field(f) == f` is asserted by two gates over
1811
+ engine-seeded field lists β€” stamping the table key from a caller that happens to know it would
1812
+ make a stored field unequal to its own validator's output. E iterates tables to build the
1813
+ synthetic rows, so E already holds the key that would go here.
1814
+ """
1815
+ bag = (field or {}).get('aiEnrich')
1816
+ if not isinstance(bag, dict):
1817
+ return None
1818
+ return {'kind': 'field_agent', 'field': str((field or {}).get('key') or ''),
1819
+ 'trigger': dict(bag.get('trigger') or {'mode': 'manual'})}
1820
+
1821
+
1822
+ # ---------------------------------------------------------------------------------------------
1823
+ # PER-CELL PROVENANCE (wave 34, W34-T51 β€” "stamp it FROM DAY ONE")
1824
+ # ---------------------------------------------------------------------------------------------
1825
+ # β›”β›” IT CANNOT LIVE IN THE ROW, AND THAT IS MEASURED RATHER THAN ASSUMED. `add_row` builds its
1826
+ # row as `{k: str(v) for k, v in values.items() if k in valid}` where `valid` is the declared
1827
+ # field keys β€” a row dict is field-keys-only by construction, so a reserved `__ai__` key inside
1828
+ # one would be stripped on the way in. Widening that filter to admit a private key would weaken
1829
+ # the wall that stops a client inventing columns. So provenance is a SIBLING STRATUM of `rows`,
1830
+ # exactly as `fields` and `rows` are siblings today.
1831
+ #
1832
+ # β›” AND `stale` IS DERIVED, NEVER STORED. A stored `stale` flag needs a sweep to stay true, and a
1833
+ # sweep nobody runs is [[flag-shipped-without-its-writer]]. What IS stored is the INPUT HASH the
1834
+ # value was produced from; `ai_enrich_is_stale` compares it to today's inputs, so staleness is a
1835
+ # question asked at read time and can never be out of date.
1836
+
1837
+ #: The stratum's key inside a table document, beside `fields` and `rows`.
1838
+ AI_ENRICH_MARK_KEY = 'aiEnrich'
1839
+ #: What a mark can say. ⚠ `stale` is absent ON PURPOSE (see above) and so is `generating`: an
1840
+ #: in-flight run is a fact about a REQUEST, not about stored data, and persisting it would leave
1841
+ #: a cell stuck on "generating" forever the first time a process dies mid-run.
1842
+ AI_ENRICH_STATES = ('agent', 'human', 'error')
1843
+ #: A bound on the stratum, in the shape `row_limit` uses: this document is copied whole on every
1844
+ #: read, so a mark set that can outgrow the rows it describes is a store problem wearing a
1845
+ #: feature's name. One mark per (row, column) is the natural size; the cap is the guard against a
1846
+ #: column deleted without its marks.
1847
+ AI_ENRICH_MAX_MARKS = MAX_ROWS
1848
+ #
1849
+ # ⚠ THERE IS DELIBERATELY NO `drop_marks(column)`, AND THE ABSENCE IS THE DECISION. `delete_field`
1850
+ # leaves a deleted column's CELLS in the rows on purpose β€” an accidental delete stays recoverable,
1851
+ # and re-adding the column under the same key brings the values back. Marks must follow the same
1852
+ # rule or a restored column's agent-written values would come back looking human-authored, which
1853
+ # is the one distinction this whole stratum exists to make. A table that is deleted takes its
1854
+ # document (and therefore its marks) with it, so neither lifecycle leaks.
1855
+
1856
+
1857
+ def ai_enrich_input_hash(prompt, row, refs):
1858
+ """The fingerprint a stored value was produced FROM: the prompt plus the referenced cells.
1859
+
1860
+ ⚠ `refs` is passed in rather than re-parsed here, so the token vocabulary is owned by one
1861
+ module (`api/ai_enrich.py::prompt_refs`) instead of two. A hash built from a second parse
1862
+ would disagree with the runner's the first time the token syntax gains a form.
1863
+ """
1864
+ payload = [str(prompt or '')]
1865
+ for key in sorted(str(r) for r in (refs or ())):
1866
+ payload.append(f'{key}={(row or {}).get(key, "")}')
1867
+ return hashlib.sha256('\x1f'.join(payload).encode('utf-8')).hexdigest()[:32]
1868
+
1869
+
1870
+ def ai_enrich_marks(table_key, col_id, st=None):
1871
+ """`{row_id: mark}` for ONE column. Absent = a cell nobody has written yet."""
1872
+ stratum = (get(table_key, st) or {}).get(AI_ENRICH_MARK_KEY) or {}
1873
+ got = stratum.get(str(col_id))
1874
+ return dict(got) if isinstance(got, dict) else {}
1875
+
1876
+
1877
+ def ai_enrich_is_stale(mark, fresh_hash):
1878
+ """Was this value produced from inputs that have since changed?
1879
+
1880
+ A human edit is never stale β€” a person's own words do not go out of date because a
1881
+ neighbouring cell moved. Only an agent-written value can be.
1882
+ """
1883
+ if not isinstance(mark, dict) or mark.get('state') != 'agent':
1884
+ return False
1885
+ return bool(fresh_hash) and str(mark.get('hash') or '') != str(fresh_hash)
1886
+
1887
+
1888
+ def ai_enrich_human_authored(mark, has_value=False):
1889
+ """Did somebody other than the agent put this value here?
1890
+
1891
+ β›” TWO CASES, AND THE SECOND IS THE ONE THAT GETS MISSED. An explicit `human` mark is obvious.
1892
+ The other is a cell that holds a value with NO mark at all: a value typed, pasted or imported
1893
+ before the column became an enrichment column. The agent did not write it, so the agent does
1894
+ not own it. Without this arm, `overwrite: 'always'` would eat exactly the pre-existing work
1895
+ R8's *"we are helping the team put their work into the system"* is about.
1896
+
1897
+ ⚠ ONE PREDICATE, because `ai_enrich_may_write` and `api/ai_enrich.py::cell_state` both answer
1898
+ "is this the human's?" and a cell the UI labels human-written while the runner overwrites it
1899
+ is the worst of both ([[one-question-two-normalizers]]).
1900
+ """
1901
+ if isinstance(mark, dict) and mark.get('state') == 'human':
1902
+ return True
1903
+ return bool(has_value) and not (isinstance(mark, dict) and mark.get('state') == 'agent')
1904
+
1905
+
1906
+ def ai_enrich_may_write(mark, policy, fresh_hash, has_value=False):
1907
+ """May an AUTOMATIC run write this cell? The law, in one place.
1908
+
1909
+ β›” THE FIRST CLAUSE IS NOT CONFIGURABLE. A cell a person typed into is never overwritten by an
1910
+ agent, whatever `overwrite` says β€” that is why `human` is not one of the `AI_ENRICH_OVERWRITE`
1911
+ values, and why `always` means "always refresh what the AGENT owns" rather than "overwrite
1912
+ everything". A policy that could turn it off would make the column unsafe to hand to a team.
1913
+ ⚠ A MANUAL, per-cell run is a different act with a different door; this predicate governs the
1914
+ automatic paths (`on_change`, `schedule`, and a bulk "all rows" sweep).
1915
+ """
1916
+ if ai_enrich_human_authored(mark, has_value):
1917
+ return False
1918
+ if not has_value:
1919
+ return True # a blank cell is fillable under every policy
1920
+ if policy == 'always':
1921
+ return True
1922
+ if policy == 'stale':
1923
+ return ai_enrich_is_stale(mark, fresh_hash)
1924
+ return False # 'blank': an agent never replaces a value it wrote
1925
+
1926
+
1927
+ def stamp_ai_enrich(table_key, col_id, marks, st=None):
1928
+ """Write MANY marks for one column in ONE store update. Returns the number stored.
1929
+
1930
+ β›” BULK BY CONSTRUCTION, and the signature is what enforces it. Every write here is a
1931
+ read-modify-write of the whole tenant document, so a per-row stamp inside a run loop is the
1932
+ same shape as the 2,000-full-document-copies problem `add_rows` exists to avoid. There is
1933
+ deliberately no `stamp_one`.
1934
+ """
1935
+ col_id = str(col_id)
1936
+ clean = {}
1937
+ for row_id, mark in (marks or {}).items():
1938
+ if not isinstance(mark, dict):
1939
+ continue
1940
+ state = str(mark.get('state') or '')
1941
+ if state not in AI_ENRICH_STATES:
1942
+ continue
1943
+ entry = {'state': state, 'at': _dt.datetime.now(_dt.timezone.utc)
1944
+ .replace(microsecond=0).isoformat()}
1945
+ for key, cap in (('hash', 64), ('model', 60), ('error', 300)):
1946
+ got = str(mark.get(key) or '').strip()[:cap]
1947
+ if got:
1948
+ entry[key] = got
1949
+ tokens = mark.get('tokens')
1950
+ if isinstance(tokens, int) and not isinstance(tokens, bool) and tokens >= 0:
1951
+ entry['tokens'] = tokens
1952
+ clean[str(row_id)] = entry
1953
+ if not clean:
1954
+ return 0
1955
+
1956
+ def _set(cur):
1957
+ t = cur.get(str(table_key))
1958
+ if t is not None:
1959
+ col = t.setdefault(AI_ENRICH_MARK_KEY, {}).setdefault(col_id, {})
1960
+ col.update(clean)
1961
+ # The cap is enforced by DROPPING THE OLDEST, never by refusing the write: a run that
1962
+ # produced values and could not record where they came from would leave the cells
1963
+ # looking human-authored, which is the one state this stratum exists to distinguish.
1964
+ if len(col) > AI_ENRICH_MAX_MARKS:
1965
+ for rid in sorted(col, key=lambda r: str(col[r].get('at') or ''))[
1966
+ :len(col) - AI_ENRICH_MAX_MARKS]:
1967
+ col.pop(rid, None)
1968
+ return cur
1969
+
1970
+ _st(st).update(STORE_KEY, _set, flush='async')
1971
+ return len(clean)
1972
+
1973
+
1974
+ def note_human_edit(table_key, values, row_id, st=None):
1975
+ """A person typed into these cells: mark any `ai_enrich` column among them `human`.
1976
+
1977
+ β›” THE STAMP HAS TO HAPPEN AT THE WRITE DOOR, not at read time, because "did a human write
1978
+ this" is not recoverable from the value afterwards. Called by `routes_tables.patch_row`;
1979
+ a no-op (and cheap) for the overwhelming case where the edited columns are ordinary.
1980
+ """
1981
+ defn = get(table_key, st) or {}
1982
+ touched = {f['key'] for f in ai_enrich_fields(defn)} & {str(k) for k in (values or {})}
1983
+ for col_id in touched:
1984
+ stamp_ai_enrich(table_key, col_id, {str(row_id): {'state': 'human'}}, st=st)
1985
+ return sorted(touched)
1986
+
1987
+
1988
  def _clean_field(raw, previous=None):
1989
  """One field dict β†’ the stored shape, or None. The single validator for create AND patch, so
1990
  a column cannot be typed one way on the way in and another on the way back."""
 
2220
  out['formula'] = fx
2221
  elif ftype == 'formula':
2222
  return None
2223
+ # ⭐⭐ WAVE-34 (owner ruling R13) β€” THE AI ENRICHMENT BAG. Posture and inheritance copied from
2224
+ # `formula` two lines up, deliberately and for the same reason: the bag defines the column, so
2225
+ # the pairing is enforced both ways, and an inherited bag is only inherited while the type
2226
+ # still wants it (retyping an enrichment column to text must not resurrect its prompt through
2227
+ # `prev` and then refuse the whole write, which is the bug the `prev_type == ftype` guard on
2228
+ # `link`/`rollup`/`formula` exists to record).
2229
+ enrich_raw = (raw.get('aiEnrich') if 'aiEnrich' in raw
2230
+ else (prev.get('aiEnrich') if prev_type == ftype else None))
2231
+ if enrich_raw is not None:
2232
+ ae = _clean_ai_enrich(enrich_raw)
2233
+ if ae is None or ftype != 'ai_enrich':
2234
+ return None
2235
+ out['aiEnrich'] = ae
2236
+ elif ftype == 'ai_enrich':
2237
+ return None
2238
+ # C3 (wave 34): the Agents module reads field agents off this bag. DERIVED here, inside the
2239
+ # one validator both write doors run, so the binding can never drift from the config it
2240
+ # describes β€” see `_field_agent_binding` for why it is computed rather than stored beside it.
2241
+ # β›” It is written AFTER the generic `automation` passthrough above, so a caller cannot hand us
2242
+ # a `field_agent` bag that disagrees with the column. Every OTHER automation bag is untouched.
2243
+ if ftype == 'ai_enrich':
2244
+ out['automation'] = _field_agent_binding(out)
2245
  return out
2246
 
2247
 
 
2437
  lives in `aios_grid._clean_automation` β€” this guards the WRITE doors only.)"""
2438
  if not isinstance(bag, dict):
2439
  return True # no bag, no law β€” an ordinary column
2440
+ # ⭐⭐ WAVE-34 (R13 / contract C3) β€” A FIELD AGENT IS BOUND BY ITS OWN COLUMN, NOT BY A FLOW,
2441
+ # and without this arm the whole feature is unsavable. `_clean_field` stamps
2442
+ # `{kind:'field_agent', ...}` on every `ai_enrich` column; `add_field` then asks this
2443
+ # predicate, which read `flowId` and only `flowId` β€” so every AI enrichment column would have
2444
+ # been refused at the create door by the guard for a different feature that happens to share
2445
+ # the `automation` key. ⚠ The binding is REAL, not waived: the bag names the column it belongs
2446
+ # to, and `_clean_field` derives that name from the field itself, so it cannot name a column
2447
+ # that does not exist the way a stale `flowId` can name a deleted flow.
2448
+ if bag.get('kind') == 'field_agent':
2449
+ return bool(str(bag.get('field') or '').strip())
2450
  flow = str(bag.get('flowId') or '').strip()
2451
  if not flow:
2452
  return False
 
2752
  return True
2753
 
2754
 
2755
+ def patch_many_cells(table_key, updates, st=None):
2756
+ """Write cells across MANY rows in ONE store update. Returns the number of rows touched.
2757
+
2758
+ β›” WHY THIS EXISTS RATHER THAN A LOOP OVER `patch_cells`, which is the same argument
2759
+ `add_rows` makes one screen up: every write here is a read-modify-write of the WHOLE tenant
2760
+ document, so filling a 1,000-row enrichment column row by row is 1,000 full-document copies
2761
+ under one lock on the single uvicorn process this product runs. One `update()` writes them
2762
+ all.
2763
+
2764
+ ⚠ `updates` is `{row_id: {field_key: value}}`, and a row id that does not exist is SKIPPED
2765
+ rather than created: this is a cell writer, not an insert door, and inventing a row here would
2766
+ let a stale plan resurrect a record somebody deleted mid-run.
2767
+ """
2768
+ if not is_user_table(table_key, st):
2769
+ return 0
2770
+ have = set((get(table_key, st) or {}).get('rows') or {})
2771
+ clean = {str(r): {str(k): str(v) for k, v in (cells or {}).items()}
2772
+ for r, cells in (updates or {}).items()
2773
+ if str(r) in have and isinstance(cells, dict) and cells}
2774
+ if not clean:
2775
+ return 0
2776
+
2777
+ def _set(cur):
2778
+ t = cur.get(str(table_key))
2779
+ if t is not None:
2780
+ rows = t.setdefault('rows', {})
2781
+ for rid, cells in clean.items():
2782
+ if rid in rows:
2783
+ rows[rid].update(cells)
2784
+ return cur
2785
+
2786
+ _st(st).update(STORE_KEY, _set, flush='sync')
2787
+ return len(clean)
2788
+
2789
+
2790
  def patch_link_cell(table_key, row_id, fkey, value, st=None):
2791
  """Persist one user-picked Link cell in the shared row and return its canonical id string.
2792
 
 
3048
  return _Lent(_st(st), {k: v for k, v in docs.items() if v is not None})
3049
 
3050
 
3051
+ def lend_defs(st=None):
3052
+ """`lend`, but the `user_tables` document it serves is the PROJECTION β€” definitions, no rows.
3053
+
3054
+ ⭐⭐ W33-T01 (D-213). `nav_entries` has done exactly this inline since W32-T02
3055
+ (`all_defs` then `lend(st, user_tables=tables)`); this is that pair named, so the TABLE WALL
3056
+ can take it too. `routes_tables._defn_or_refuse` is the wall a database SWITCH goes through and
3057
+ it read the whole 28.6 MB document to answer two questions β€” "does this key exist" and
3058
+ "may this session open it" β€” neither of which has ever looked at a row.
3059
+
3060
+ β›” **HAND THIS TO A READ, NEVER TO A WRITE.** Every value it serves is a `_Projected`, so a
3061
+ caller that reaches `defn['rows']` gets a `KeyError` naming the projection β€” loud, which is the
3062
+ point β€” and a read-modify-write handed this document would upload a tenant with its rows
3063
+ removed. `store.update`/`_read_strict` are unaffected (they never project), and the six write
3064
+ walls in `routes_tables` deliberately do not opt in. C5's second clause is the same rule from
3065
+ the other end: a projected snapshot is never handed to a post-write read-back.
3066
+
3067
+ ⚠ It is not free β€” it is ~0.1% of the bytes, not zero. A caller that will need rows ANYWAY
3068
+ should take the whole read once rather than this plus that.
3069
+ """
3070
+ return lend(st, **{STORE_KEY: all_defs(st)})
3071
+
3072
+
3073
  def nav_entries(viewer=None, is_admin=False, st=None):
3074
  """Registry-SHAPED dicts so the nav can render user tables beside real modules with no
3075
  special-casing: the same keys the flat nav reads (`key`, `label`, `source`).
platform/core/users.py CHANGED
@@ -445,21 +445,20 @@ def assignable_people(tenant=None):
445
 
446
  Wave 18 (C1-TENANT): pass `tenant` to scope the choices to ONE company β€” the user registry
447
  is a global control-plane bucket, and a Nurilab picker offering Royal's staff is a
448
- cross-tenant name leak. None = unscoped (the Streamlit host, tenant #0's process).
 
 
 
 
 
 
 
 
 
 
 
449
  """
450
- try:
451
- reg = registry() or {}
452
- except Exception:
453
- return []
454
- want = str(tenant or '').strip().lower()
455
- out = []
456
- for username, u in reg.items():
457
- if not isinstance(u, dict) or u.get('active') is False:
458
- continue
459
- if want and str(u.get('tenant') or 'royal-imports').strip().lower() != want:
460
- continue
461
- out.append(str(u.get('name') or username))
462
- return sorted(set(out))
463
 
464
 
465
  def set_avatar(username, data_url):
@@ -486,20 +485,41 @@ def avatar_map(tenant=None):
486
  name, so the display name is the only join a renderer has. Two active accounts sharing a
487
  display name share one option; the first WITH a photo wins the key rather than a coin
488
  flip deciding whether the option has a face. `tenant` scopes it exactly as
489
- `assignable_people(tenant)` does, and for the same leak."""
490
- try:
491
- reg = registry() or {}
492
- except Exception:
493
- return {}
494
- want = str(tenant or '').strip().lower()
495
  out = {}
496
- for username, u in sorted(reg.items()):
497
- if not isinstance(u, dict) or u.get('active') is False:
498
- continue
499
- if want and str(u.get('tenant') or 'royal-imports').strip().lower() != want:
500
- continue
501
  av = u.get('avatar')
502
  nm = str(u.get('name') or username)
503
  if av and nm not in out:
504
  out[nm] = str(av)
505
  return out
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
445
 
446
  Wave 18 (C1-TENANT): pass `tenant` to scope the choices to ONE company β€” the user registry
447
  is a global control-plane bucket, and a Nurilab picker offering Royal's staff is a
448
+ cross-tenant name leak.
449
+
450
+ β›”β›” WAVE 33 (W33-T37) β€” A BLANK `tenant` NOW RETURNS NOTHING. It used to skip the filter and
451
+ return EVERY tenant's staff: `if want and …` is structurally fail-OPEN, and the exemption was
452
+ written for "None = unscoped (the Streamlit host, tenant #0's process)" β€” a host DELETED at
453
+ EXIT-6. So the sanctioned caller no longer exists, and what was left is a whole-platform
454
+ roster one forgotten kwarg away, with no error to notice
455
+ ([[gate-must-go-red-not-crash]]'s sibling: a wall that answers instead of refusing).
456
+ ⚠ The direction is the safety: this can only ever NARROW. All six live call sites pass
457
+ `session.tenant`, which `aios_session.read` refuses to admit blank, so nothing legitimate
458
+ changes β€” and a future caller that forgets gets an empty picker somebody notices instead of a
459
+ leak nobody does.
460
  """
461
+ return sorted({str(u.get('name') or n) for n, u in _tenant_accounts(tenant)})
 
 
 
 
 
 
 
 
 
 
 
 
462
 
463
 
464
  def set_avatar(username, data_url):
 
485
  name, so the display name is the only join a renderer has. Two active accounts sharing a
486
  display name share one option; the first WITH a photo wins the key rather than a coin
487
  flip deciding whether the option has a face. `tenant` scopes it exactly as
488
+ `assignable_people(tenant)` does, and for the same leak β€” including wave 33's fail-closed
489
+ blank, which both take from the ONE resolver below rather than each writing `if want and …`.
490
+ """
 
 
 
491
  out = {}
492
+ for username, u in sorted(_tenant_accounts(tenant), key=lambda kv: kv[0]):
 
 
 
 
493
  av = u.get('avatar')
494
  nm = str(u.get('name') or username)
495
  if av and nm not in out:
496
  out[nm] = str(av)
497
  return out
498
+
499
+
500
+ def _tenant_accounts(tenant):
501
+ """`[(username, record), …]` β€” the ACTIVE accounts of exactly ONE tenant. FAIL-CLOSED.
502
+
503
+ β›” THE ONE PLACE THE TENANT FILTER FOR THE USER REGISTRY IS WRITTEN. It was written twice,
504
+ identically, in `assignable_people` and `avatar_map`, and both copies were fail-OPEN on a
505
+ blank slug in the same way (`if want and …`). Two copies of a wall is two places for one of
506
+ them to be fixed [[one-question-two-normalizers]]; `routes_shares._people` is a THIRD copy of
507
+ the same predicate and is lane C's to fold in.
508
+
509
+ ⚠ `str(u.get('tenant') or 'royal-imports')` is kept on the RECORD side deliberately: a
510
+ pre-wave-18 account genuinely has no `tenant` key and IS tenant #0's, and `users._public`
511
+ normalises it the same way. What changed is the WANT side β€” a blank want is now a refusal
512
+ rather than a wildcard.
513
+ """
514
+ want = str(tenant or '').strip().lower()
515
+ if not want:
516
+ # Not an error and not everything: nobody. See `assignable_people`'s note β€” the one
517
+ # caller this exemption was written for (the Streamlit host) was deleted at EXIT-6.
518
+ return []
519
+ try:
520
+ reg = registry() or {}
521
+ except Exception:
522
+ return []
523
+ return [(n, u) for n, u in reg.items()
524
+ if isinstance(u, dict) and u.get('active') is not False
525
+ and str(u.get('tenant') or 'royal-imports').strip().lower() == want]
platform/harness/datastore.py CHANGED
@@ -188,14 +188,35 @@ ENTITIES = {
188
  # backfill below has walked every partner (a partner's `write_date` did not move because
189
  # WE changed the schema). Every projection degrades a missing/NULL column to blank rather
190
  # than raising, so declaring early is safe β€” it is just not yet true.
 
 
 
 
 
 
 
 
 
 
 
 
191
  "fields": ["name", "team_id", "city", "state_id", "country_id", "active",
192
  "agent", "salesman_as_agent", "customer_rank",
193
- "street", "street2", "zip", "write_date"],
 
 
194
  # `backfill_fields` carries it too, so mirrors that already exist pick it up on the next
195
  # sync_all() instead of needing a reseed.
 
 
 
 
 
196
  "backfill_fields": ["agent", "salesman_as_agent", # added 2026-07-28
197
  "customer_rank", # added 2026-08-11 (W29-T54)
198
- "street", "street2", "zip"], # added 2026-08-12 (W30-T33)
 
 
199
  # m2m fields land as <base>_id (the FIRST linked id β€” the Customers-module convention:
200
  # a customer's assigned agent is agent_ids[0], ~one per customer, MECE) + <field> as a
201
  # JSON list of ALL ids. Declared explicitly: a 2-element m2m read would otherwise be
 
188
  # backfill below has walked every partner (a partner's `write_date` did not move because
189
  # WE changed the schema). Every projection degrades a missing/NULL column to blank rather
190
  # than raising, so declaring early is safe β€” it is just not yet true.
191
+ # ⭐⭐ WAVE 33 (W33-T48, owner item 13) β€” THE CONTACT/IDENTITY HALF, MEASURED BEFORE IT WAS
192
+ # ADDED. The owner: *"I could have sworn we have more relevant Fields under 'Odoo vendors'
193
+ # etc."* He was right, and the census
194
+ # (`.claude/wiki/waves/wave33/odoo-field-census.md`, live read-only, 2026-08-14) says by how
195
+ # much: `res.partner` declares 198 fields, **77 are populated on all 3,621 customers and 76
196
+ # on all 658 vendors**, and this mirror carried THIRTEEN. Every one of the six added here
197
+ # was measured non-empty on the live population first β€” none is added on the strength of
198
+ # "Odoo probably has that", which is how a schema grows columns nobody can fill.
199
+ # ⚠ THE MIRROR IS THE CEILING, NOT THE GRID. Every reader in `odoo_relational.py` reads
200
+ # THIS file, so a column added to a `*_fields()` contract with no entry here renders as a
201
+ # blank column forever β€” which is D-165's shape exactly, and the reason the widening starts
202
+ # on this line rather than on the grid.
203
  "fields": ["name", "team_id", "city", "state_id", "country_id", "active",
204
  "agent", "salesman_as_agent", "customer_rank",
205
+ "street", "street2", "zip",
206
+ "email", "phone", "mobile", "website", "vat", "ref",
207
+ "write_date"],
208
  # `backfill_fields` carries it too, so mirrors that already exist pick it up on the next
209
  # sync_all() instead of needing a reseed.
210
+ # β›” D-165 IS ABOUT THIS LIST, AND THE CENSUS SETTLED WHICH HALF IS WRONG: `street`,
211
+ # `street2` and `zip` are `store=True` on `res.partner` and POPULATED in live Odoo β€” they
212
+ # are declared here and still NULL in the mirror purely because a backfill pass has not
213
+ # walked every partner (changing OUR schema does not move a partner's `write_date`, so an
214
+ # incremental sync never revisits them). Declaring is not filling; the fix is a run.
215
  "backfill_fields": ["agent", "salesman_as_agent", # added 2026-07-28
216
  "customer_rank", # added 2026-08-11 (W29-T54)
217
+ "street", "street2", "zip", # added 2026-08-12 (W30-T33)
218
+ "email", "phone", "mobile", # added 2026-08-14 (W33-T48)
219
+ "website", "vat", "ref"],
220
  # m2m fields land as <base>_id (the FIRST linked id β€” the Customers-module convention:
221
  # a customer's assigned agent is agent_ids[0], ~one per customer, MECE) + <field> as a
222
  # JSON list of ALL ids. Declared explicitly: a 2-element m2m read would otherwise be
platform/harness/tools.py CHANGED
@@ -65,13 +65,22 @@ def _err(msg):
65
  # ------------------------------------------------------------------ schema tools
66
 
67
  def list_topics():
68
- """The 'what data exists' tool."""
 
 
 
 
 
 
69
  out = []
70
  for k, t in SEM.topics().items():
71
- out.append({"topic": k, "label": t.get("label"), "entity": t.get("entity"),
72
- "grain": t.get("grain"),
73
- "dims": list((t.get("store") or {}).get("dims") or {}),
74
- "metrics": [m for m, d in SEM.metrics().items() if d["topic"] == k]})
 
 
 
75
  return out
76
 
77
 
@@ -83,10 +92,30 @@ def describe_topic(topic):
83
  mets = {k: {"label": m.get("label"), "description": m.get("description"),
84
  "format": m.get("format"), "ai_context": m.get("ai_context")}
85
  for k, m in SEM.metrics().items() if m["topic"] == topic}
86
- return {"topic": topic, "label": t.get("label"), "scope": t.get("scope"),
87
- "grain": t.get("grain"), "ai_context": t.get("ai_context"),
88
- "dims": {k: v.get("label") for k, v in ((t.get("store") or {}).get("dims") or {}).items()},
89
- "metrics": mets}
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
90
 
91
 
92
  # ------------------------------------------------------------------ query tools
 
65
  # ------------------------------------------------------------------ schema tools
66
 
67
  def list_topics():
68
+ """The 'what data exists' tool.
69
+
70
+ W33-T49 adds `grid` β€” the store key of the database this topic describes, when it describes
71
+ one. It is what lets the agent connect "the thing the person is looking at" to "the topic I
72
+ can query", which is the whole point of the entity topics: before this, the semantic layer
73
+ could see `sale_order_line` and `account_move` and had never heard of `ut_odoo_customers`.
74
+ """
75
  out = []
76
  for k, t in SEM.topics().items():
77
+ row = {"topic": k, "label": t.get("label"), "entity": t.get("entity"),
78
+ "grain": t.get("grain"),
79
+ "dims": list((t.get("store") or {}).get("dims") or {}),
80
+ "metrics": [m for m, d in SEM.metrics().items() if d["topic"] == k]}
81
+ if t.get("grid"):
82
+ row["grid"] = t["grid"]
83
+ out.append(row)
84
  return out
85
 
86
 
 
92
  mets = {k: {"label": m.get("label"), "description": m.get("description"),
93
  "format": m.get("format"), "ai_context": m.get("ai_context")}
94
  for k, m in SEM.metrics().items() if m["topic"] == topic}
95
+ out = {"topic": topic, "label": t.get("label"), "scope": t.get("scope"),
96
+ "grain": t.get("grain"), "ai_context": t.get("ai_context"),
97
+ "dims": {k: v.get("label") for k, v in ((t.get("store") or {}).get("dims") or {}).items()},
98
+ "metrics": mets}
99
+ # ⭐⭐ W33-T49 β€” THE FIELD SEMANTICS, which this tool did not carry AT ALL.
100
+ #
101
+ # β›” THE GAP WAS NOT THE TOPIC FILES, IT WAS THIS RETURN. Before this, `describe_topic`
102
+ # answered with scope, grain, dims, metrics and `ai_context` β€” and **no vocabulary of
103
+ # COLUMNS**. So an agent told to describe a database could name what it MEASURES and never
104
+ # what it HOLDS: the owner's item 13 asks for "ALL populated fields … including its semantic
105
+ # layer", and adding eight topic files without this would have left `describe_topic` returning
106
+ # the same field-free shape it always did, one screen away from the fields.
107
+ #
108
+ # ⚠ `kind` is the part worth reading, and it is the difference between a column an agent can
109
+ # FILTER and one it cannot: `data` is stored on the row; `rollup` is computed from another
110
+ # topic (`from_topic` names it, and that is where a windowed or reconciled number must come
111
+ # from); `link` points at another database. An agent that treats a rollup as a stored column
112
+ # will ask this topic for a number this topic does not hold.
113
+ if t.get("fields"):
114
+ out["fields"] = [dict(f) for f in t["fields"]]
115
+ for k in ("grid", "subject", "entity"):
116
+ if t.get(k):
117
+ out[k] = t[k]
118
+ return out
119
 
120
 
121
  # ------------------------------------------------------------------ query tools
platform/model/topics/odoo_accounts.yml ADDED
@@ -0,0 +1,60 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # ⭐⭐ W33-T49 (owner item 13) β€” an ENTITY topic: the semantic layer can finally see
2
+ # the databases a person actually opens, not only the line and document grains.
3
+ #
4
+ # β›” THE `fields:` BLOCK IS GENERATED FROM THE GRID'S OWN FIELD CONTRACT
5
+ # and is held to it by `verify_query.py::section_entity_topics`. Do not hand-edit a
6
+ # field row: add the column to the grid contract and re-emit, or the agent is being
7
+ # trained on a schema the product does not have.
8
+ key: odoo_accounts
9
+ label: "Odoo GL accounts"
10
+ entity: account.account
11
+ # The database this topic DESCRIBES β€” the same store key the nav opens (W33-T46).
12
+ grid: ut_odoo_accounts
13
+ subject: "odoo:account.account"
14
+ grain: "one row per GL account in the chart of accounts"
15
+ scope:
16
+ population: "every account.account record (192 measured)"
17
+ store:
18
+ table: account_account
19
+ alias: a
20
+ # NO date_col β€” a registry is not a dated event stream. Stated rather than
21
+ # omitted, so its absence reads as a fact and not as an unfinished file.
22
+ dims:
23
+ account_type: {label: "Account type"}
24
+
25
+ # key / label / type / kind, derived from the grid contract. `kind` says where the
26
+ # value COMES FROM: a `data` column is stored on the row; a `rollup` is computed
27
+ # from another topic; a `link` points at another database.
28
+ fields:
29
+ - key: account_code
30
+ label: "Code"
31
+ type: text
32
+ kind: data
33
+ - key: account_name
34
+ label: "Account"
35
+ type: text
36
+ kind: data
37
+ - key: odoo_id
38
+ label: "Odoo ID"
39
+ type: int
40
+ kind: data
41
+ means: "The `account.account` id. Also this row's id."
42
+ - key: account_type
43
+ label: "Type"
44
+ type: select
45
+ kind: data
46
+ - key: is_expense
47
+ label: "Expense account"
48
+ type: checkbox
49
+ kind: data
50
+ means: "Ticked for the expense family - the same predicate the semantic layer's gl_lines topic uses, so this column and that topic cannot disagree."
51
+ - key: refreshed
52
+ label: "Refreshed"
53
+ type: date
54
+ kind: data
55
+ means: "When this row was last reconciled against Odoo."
56
+
57
+ ai_context: >
58
+ The chart of accounts β€” the registry gl_lines posts against.
59
+ Use it to resolve an account code to a name or a type.
60
+ ⚠ `is_expense` is a DERIVED boolean standing in for Odoo's five-value `internal_group` (asset/liability/equity/income/expense), which the mirror does not yet carry.
platform/model/topics/odoo_agents.yml ADDED
@@ -0,0 +1,59 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # ⭐⭐ W33-T49 (owner item 13) β€” an ENTITY topic: the semantic layer can finally see
2
+ # the databases a person actually opens, not only the line and document grains.
3
+ #
4
+ # β›” THE `fields:` BLOCK IS GENERATED FROM THE GRID'S OWN FIELD CONTRACT
5
+ # and is held to it by `verify_query.py::section_entity_topics`. Do not hand-edit a
6
+ # field row: add the column to the grid contract and re-emit, or the agent is being
7
+ # trained on a schema the product does not have.
8
+ key: odoo_agents
9
+ label: "Odoo agents"
10
+ entity: res.partner
11
+ # The database this topic DESCRIBES β€” the same store key the nav opens (W33-T46).
12
+ grid: ut_odoo_agents
13
+ subject: "odoo:res.partner.agent"
14
+ grain: "one row per sales agent"
15
+ scope:
16
+ population: "a UNION of two disagreeing sources β€” partners carrying commission lines, and partners flagged res_partner.agent = TRUE"
17
+ why_union: "measured 2026-08-09: 16 carry commission lines, 17 carry the flag, and the union is 19 β€” either source alone silently drops real agents"
18
+ store:
19
+ table: res_partner
20
+ alias: p
21
+ # NO date_col β€” a registry is not a dated event stream. Stated rather than
22
+ # omitted, so its absence reads as a fact and not as an unfinished file.
23
+
24
+ # key / label / type / kind, derived from the grid contract. `kind` says where the
25
+ # value COMES FROM: a `data` column is stored on the row; a `rollup` is computed
26
+ # from another topic; a `link` points at another database.
27
+ fields:
28
+ - key: agent
29
+ label: "Agent"
30
+ type: text
31
+ kind: data
32
+ - key: odoo_id
33
+ label: "Odoo ID"
34
+ type: int
35
+ kind: data
36
+ means: "The `res.partner` id. Also this row's id."
37
+ - key: agent_id
38
+ label: "Odoo agent id"
39
+ type: int
40
+ kind: data
41
+ - key: flagged
42
+ label: "Flagged in Odoo"
43
+ type: checkbox
44
+ kind: data
45
+ means: "Ticked = `res.partner.agent` is set. Unticked agents were found by their commission lines instead - both are real, which is why this table is the union of the two."
46
+ - key: commissioned
47
+ label: "Has commission lines"
48
+ type: checkbox
49
+ kind: data
50
+ - key: refreshed
51
+ label: "Refreshed"
52
+ type: date
53
+ kind: data
54
+ means: "When this row was last reconciled against Odoo."
55
+
56
+ ai_context: >
57
+ The sales-agent registry.
58
+ `flagged` and `commissioned` are the two SOURCES, kept as separate columns rather than merged, because they disagree and the disagreement is information.
59
+ For an agent's BOOK use sales_lines/commission_lines with the agent dim.