fsanyoto commited on
Commit
4748aae
·
verified ·
1 Parent(s): 35d0dbe

Deploy AIOS web (React glide grid + FastAPI slice)

Browse files
This view is limited to 50 files because it contains too many changes.   See raw diff
Files changed (50) hide show
  1. api/automation_engine.py +1377 -0
  2. api/deps.py +29 -4
  3. api/main.py +8 -0
  4. api/routes_admin.py +28 -6
  5. api/routes_assets.py +165 -0
  6. api/routes_auth.py +23 -11
  7. api/routes_automation.py +217 -0
  8. api/routes_grid.py +69 -12
  9. api/routes_keychain.py +143 -0
  10. api/routes_nav.py +47 -2
  11. api/routes_tables.py +239 -0
  12. requirements.txt +4 -0
  13. royalimports_os/aios_grid.py +193 -2
  14. royalimports_os/core/assets.py +73 -0
  15. royalimports_os/core/grid_events.py +14 -0
  16. royalimports_os/core/keychain.py +182 -0
  17. royalimports_os/core/registry.py +6 -0
  18. royalimports_os/core/store.py +369 -307
  19. royalimports_os/core/table_store.py +19 -11
  20. royalimports_os/core/user_tables.py +143 -30
  21. royalimports_os/core/users.py +36 -7
  22. royalimports_os/harness/runtime.py +63 -9
  23. web/dist-embed/index.html +0 -0
  24. web/src/apiContract.ts +6 -0
  25. web/src/automation/AutomationEditor.tsx +662 -0
  26. web/src/automation/AutomationSurface.tsx +287 -0
  27. web/src/automation/automationApi.ts +208 -0
  28. web/src/customer-grid/CatalogView.tsx +1147 -0
  29. web/src/customer-grid/ColumnMenu.tsx +168 -6
  30. web/src/customer-grid/CustomerGrid.tsx +236 -1
  31. web/src/customer-grid/RecordDetail.tsx +24 -1
  32. web/src/customer-grid/apiBridge.ts +1 -1
  33. web/src/customer-grid/catalogData.ts +350 -0
  34. web/src/customer-grid/catalogPrint.ts +155 -0
  35. web/src/customer-grid/cells.ts +59 -4
  36. web/src/customer-grid/clipboard.ts +6 -0
  37. web/src/customer-grid/display.ts +41 -0
  38. web/src/customer-grid/iconShapes.ts +33 -0
  39. web/src/customer-grid/theme.ts +17 -0
  40. web/src/customer-grid/types.ts +297 -5
  41. web/src/customer-grid/useCustomerData.ts +15 -3
  42. web/src/customer-grid/useVisibleRows.ts +7 -0
  43. web/src/filter-kit/ops.ts +6 -0
  44. web/src/index.css +0 -0
  45. web/src/settings/SettingsModal.tsx +332 -1
  46. web/src/settings/permsModel.ts +12 -2
  47. web/src/settings/settingsApi.ts +73 -0
  48. web/src/shell/Shell.tsx +271 -29
  49. web/src/shell/nav.ts +19 -4
  50. web/src/viz/seriesData.ts +4 -0
api/automation_engine.py ADDED
@@ -0,0 +1,1377 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """automation_engine.py — wave-18 item 5 (contract C4-AUTO): the automation RUNTIME.
2
+
3
+ THE SHAPE, and why it is this shape. An automation is a DEFINITION that lives in the tenant
4
+ store and a RUN that lives in this process. The definition is durable, small and rarely written;
5
+ the run is hot, chatty and worthless five minutes later. Conflating them is the failure this file
6
+ is built to avoid, and it has two halves:
7
+
8
+ * **Run state is NEVER persisted while it is running.** A `state: 'running'` written to the
9
+ store outlives the process that wrote it, so a container restart mid-run leaves an automation
10
+ that can never run again — the 409 sees a `running` nothing will ever clear. Hot state lives
11
+ in `_RUNNING` (a process dict, cleared by definition on restart) and the store is written
12
+ exactly ONCE per run, at the end.
13
+ * **One coalesced store update per run per bucket.** `core/store.py` coalesces on a 20 s
14
+ floor per key against a 256-commits/hour repo budget, so a per-ROW write is the wrong shape
15
+ by two orders of magnitude — the S&P demo alone is ~500 rows. Every runner below computes
16
+ its whole result first and commits it in a single `update` callback.
17
+
18
+ WHAT IS VENDORED HERE AND WHY. `.claude/skills/browse/scrape.py` is not on `sys.path` and is not
19
+ deployed, so its extraction core is copied into this file (the wave doc's instruction), ported
20
+ httpx → requests (the only HTTP dependency this API already carries). The SSRF rail comes with
21
+ it, HARDENED rather than merely preserved — see `guard`/`fetch`. That hardening is the point:
22
+ the skill version takes a URL from a developer on a CLI; this takes one from a request body.
23
+ """
24
+ from __future__ import annotations
25
+
26
+ import datetime as _dt
27
+ import ipaddress
28
+ import json
29
+ import os
30
+ import re
31
+ import socket
32
+ import threading
33
+ import time
34
+ from urllib.parse import urljoin, urlparse
35
+
36
+ import requests
37
+
38
+ try: # the parser the extraction half needs
39
+ from bs4 import BeautifulSoup
40
+ except Exception: # pragma: no cover — deploy lag; see mailbox ②
41
+ BeautifulSoup = None
42
+
43
+ # ---------------------------------------------------------------------------------------------
44
+ # THE BUCKET (C4-AUTO)
45
+ # ---------------------------------------------------------------------------------------------
46
+
47
+ #: The per-tenant store key. Colocated with its reader, the `routes_nav._NAV_PREFS_KEY` pattern.
48
+ STORE_KEY = "automations"
49
+ #: The user-table bucket. ⚠ Read/written HERE through `TenantRuntime`, never through
50
+ #: `core.user_tables` — that module writes the UNPREFIXED module-global key, which is correct for
51
+ #: tenant #0 (empty prefix) and silently cross-tenant for every R2 tenant after it. Booked in the
52
+ #: session-D mailbox as a finding against A's file rather than fixed from here.
53
+ UT_STORE_KEY = "user_tables"
54
+ UT_PREFIX = "ut_"
55
+
56
+ MAX_AUTOMATIONS = 40
57
+ MAX_RUNS = 20 # trimmed history per automation
58
+ MAX_NAME = 80
59
+ MAX_UT_ROWS = 5000 # mirrors core.user_tables.MAX_ROWS
60
+ MAX_UT_TABLES = 40
61
+
62
+ KINDS = ("scrape_db", "field_instagram")
63
+ EXTRACTS = ("table", "jsonld")
64
+ STATES = ("idle", "running", "ok", "error", "partial")
65
+
66
+ #: Schedule presets the editor offers. Kept here (not in the client) so the vocabulary the UI
67
+ #: shows and the vocabulary the parser accepts cannot drift.
68
+ CRON_PRESETS = [
69
+ {"cron": "*/15 * * * *", "label": "Every 15 minutes"},
70
+ {"cron": "0 * * * *", "label": "Hourly"},
71
+ {"cron": "0 6 * * *", "label": "Daily at 06:00"},
72
+ {"cron": "0 6 * * 1", "label": "Weekly (Monday 06:00)"},
73
+ {"cron": "0 6 1 * *", "label": "Monthly (1st, 06:00)"},
74
+ ]
75
+
76
+ UA = "Mozilla/5.0 (compatible; AIOS-automation/1.0; +https://aios.local/automation)"
77
+
78
+
79
+ def _now():
80
+ return _dt.datetime.now()
81
+
82
+
83
+ def _stamp(dt=None):
84
+ return (dt or _now()).strftime("%Y-%m-%d %H:%M")
85
+
86
+
87
+ def _iso(dt=None):
88
+ return (dt or _now()).strftime("%Y-%m-%dT%H:%M:%S")
89
+
90
+
91
+ def _parse_iso(s):
92
+ try:
93
+ return _dt.datetime.strptime(str(s)[:19], "%Y-%m-%dT%H:%M:%S")
94
+ except Exception:
95
+ return None
96
+
97
+
98
+ # ---------------------------------------------------------------------------------------------
99
+ # THE SSRF RAIL — vendored from scrape.py `guard`, then hardened for a SERVER-SIDE fetcher
100
+ # ---------------------------------------------------------------------------------------------
101
+ # The skill version guards the URL a developer typed. This one guards a URL that arrived in a
102
+ # request body, which changes the threat model in two ways the original does not cover:
103
+ #
104
+ # 1. REDIRECTS. `httpx.Client(follow_redirects=True)` / `requests.get(allow_redirects=True)`
105
+ # never re-enter the guard, so `http://evil.example/x` → 302 → `http://169.254.169.254/…`
106
+ # sails straight past a guard that only ever saw the first URL. The negative control ("a
107
+ # localhost URL is refused") would still pass while the rail was wide open. `fetch` below
108
+ # therefore takes the hops MANUALLY and re-guards every one.
109
+ # 2. DNS. A perfectly public hostname may resolve to a private address. `guard` resolves the
110
+ # host and checks EVERY answer, not just the literal.
111
+ #
112
+ # ⚠ STATED, NOT HIDDEN: this is check-then-connect, so a DNS-rebinding attacker who flips the
113
+ # record between the guard and the socket is not stopped by it. Closing that needs a pinned
114
+ # connection to the validated IP (a custom adapter). Out of scope for v1 and recorded here rather
115
+ # than implied away — the rail refuses the realistic cases and says what it does not cover.
116
+
117
+ class Refused(ValueError):
118
+ """The rail refused a URL. A distinct type so a refusal is never logged as a fetch error."""
119
+
120
+
121
+ def _ip_public(ip):
122
+ return not (ip.is_private or ip.is_loopback or ip.is_link_local
123
+ or ip.is_reserved or ip.is_unspecified or ip.is_multicast)
124
+
125
+
126
+ def guard(url, allowed=()):
127
+ """PUBLIC-WEB-ONLY rail: http(s) only, no loopback/private/link-local host, DNS answers
128
+ checked too, plus the optional `--allowed` domain rail scrape.py carries."""
129
+ p = urlparse(url or "")
130
+ if p.scheme not in ("http", "https"):
131
+ raise Refused(f"only http(s) allowed, not {p.scheme!r}")
132
+ host = (p.hostname or "").strip()
133
+ if not host:
134
+ raise Refused("no host in the URL")
135
+ low = host.lower()
136
+ if (low in ("localhost", "localhost.localdomain")
137
+ or low.endswith(".local") or low.endswith(".internal")
138
+ or low.endswith(".localhost")):
139
+ raise Refused(f"refused non-public host {host!r} (public web only)")
140
+ try: # a bare-IP host is decided on the literal
141
+ ip = ipaddress.ip_address(low)
142
+ except ValueError:
143
+ ip = None
144
+ if ip is not None and not _ip_public(ip):
145
+ raise Refused(f"refused non-public IP {host} (public web only)")
146
+ if allowed and not any(low == d.lower() or low.endswith("." + d.lower()) for d in allowed):
147
+ raise Refused(f"host {host!r} not in the allowed domains {list(allowed)}")
148
+ if ip is None:
149
+ try:
150
+ infos = socket.getaddrinfo(host, None)
151
+ except Exception as e:
152
+ raise Refused(f"could not resolve {host!r}: {type(e).__name__}")
153
+ for info in infos:
154
+ addr = info[4][0]
155
+ try:
156
+ resolved = ipaddress.ip_address(addr.split("%")[0])
157
+ except ValueError:
158
+ continue
159
+ if not _ip_public(resolved):
160
+ raise Refused(
161
+ f"refused {host!r}: it resolves to the non-public address {resolved}")
162
+ return True
163
+
164
+
165
+ def fetch(url, timeout=20.0, max_kb=2048, allowed=(), max_hops=5, headers=None):
166
+ """GET `url`, taking redirects BY HAND so every hop passes `guard`.
167
+
168
+ Returns `(status, final_url, body_bytes)`. A hop chain longer than `max_hops` is a refusal,
169
+ not a silent truncation — an endless redirect is indistinguishable from an attempt to walk
170
+ the fetcher somewhere it was told not to go.
171
+ """
172
+ current = url
173
+ hdrs = {"User-Agent": UA, "Accept-Language": "en-US,en;q=0.9"}
174
+ hdrs.update(headers or {})
175
+ for _hop in range(max_hops + 1):
176
+ guard(current, allowed)
177
+ r = requests.get(current, timeout=timeout, headers=hdrs,
178
+ allow_redirects=False, stream=True)
179
+ if r.status_code in (301, 302, 303, 307, 308):
180
+ loc = r.headers.get("Location")
181
+ r.close()
182
+ if not loc:
183
+ raise Refused(f"redirect {r.status_code} with no Location header")
184
+ current = urljoin(current, loc)
185
+ continue
186
+ body = r.raw.read(max_kb * 1024, decode_content=True) or b""
187
+ status, final = r.status_code, str(r.url)
188
+ r.close()
189
+ return status, final, body
190
+ raise Refused(f"more than {max_hops} redirects — refusing to follow further")
191
+
192
+
193
+ # ---------------------------------------------------------------------------------------------
194
+ # EXTRACTION — vendored from scrape.py, unchanged in behaviour
195
+ # ---------------------------------------------------------------------------------------------
196
+
197
+ def _soup(body):
198
+ if BeautifulSoup is None:
199
+ raise RuntimeError(
200
+ "beautifulsoup4 is not installed in this environment — the automation engine's "
201
+ "extraction half needs it (see the wave-18 mailbox: add beautifulsoup4 + lxml to "
202
+ "aios-web/requirements.txt).")
203
+ try:
204
+ return BeautifulSoup(body, "lxml")
205
+ except Exception:
206
+ return BeautifulSoup(body, "html.parser")
207
+
208
+
209
+ def tables(soup):
210
+ """Every HTML table as rows. Dict rows when the first row looks like a header."""
211
+ out = []
212
+ for t in soup.find_all("table"):
213
+ rows = []
214
+ for tr in t.find_all("tr"):
215
+ cells = [c.get_text(" ", strip=True) for c in tr.find_all(["th", "td"])]
216
+ if cells:
217
+ rows.append(cells)
218
+ if not rows:
219
+ continue
220
+ head, body = rows[0], rows[1:]
221
+ if body and len(head) == len(body[0]) and all(head):
222
+ out.append([dict(zip(head, r)) for r in body if len(r) == len(head)])
223
+ else:
224
+ out.append(rows)
225
+ return out
226
+
227
+
228
+ def jsonld(soup):
229
+ out = []
230
+ for s in soup.find_all("script", attrs={"type": "application/ld+json"}):
231
+ try:
232
+ out.append(json.loads(s.string or s.get_text()))
233
+ except Exception:
234
+ pass
235
+ return out
236
+
237
+
238
+ def meta(soup):
239
+ m = {}
240
+ if soup.title and soup.title.string:
241
+ m["title"] = soup.title.string.strip()
242
+ for tag in soup.find_all("meta"):
243
+ k = tag.get("name") or tag.get("property")
244
+ v = tag.get("content")
245
+ if k and v and (k in ("description", "keywords", "author")
246
+ or k.startswith(("og:", "twitter:"))):
247
+ m[k] = v.strip()
248
+ return m
249
+
250
+
251
+ def preview(url, extract="table", table_index=0, allowed=()):
252
+ """The field-map preview the editor calls BEFORE anything is created: what columns does this
253
+ page actually offer, and what do the first rows look like? Never writes."""
254
+ status, final, body = fetch(url, allowed=allowed)
255
+ if not (200 <= status < 300):
256
+ return {"ok": False, "status": status, "url": final,
257
+ "note": f"the page answered {status} — it may be bot-gated or gone",
258
+ "columns": [], "sample": [], "rowCount": 0}
259
+ soup = _soup(body)
260
+ rows = []
261
+ if extract == "jsonld":
262
+ blocks = jsonld(soup)
263
+ flat = []
264
+ for b in blocks:
265
+ if isinstance(b, list):
266
+ flat.extend([x for x in b if isinstance(x, dict)])
267
+ elif isinstance(b, dict):
268
+ items = b.get("itemListElement")
269
+ flat.extend([x for x in items if isinstance(x, dict)] if isinstance(items, list)
270
+ else [b])
271
+ rows = [{k: _scalar(v) for k, v in d.items()} for d in flat]
272
+ else:
273
+ found = tables(soup)
274
+ idx = max(0, min(int(table_index or 0), len(found) - 1)) if found else 0
275
+ picked = found[idx] if found else []
276
+ rows = [r for r in picked if isinstance(r, dict)]
277
+ cols, seen = [], set()
278
+ for r in rows[:50]:
279
+ for k in r:
280
+ if k not in seen:
281
+ seen.add(k)
282
+ cols.append(k)
283
+ return {"ok": True, "status": status, "url": final, "columns": cols,
284
+ "sample": rows[:8], "rowCount": len(rows),
285
+ "tableCount": len(tables(soup)) if extract != "jsonld" else 0,
286
+ "title": (meta(soup) or {}).get("title", "")}
287
+
288
+
289
+ def _scalar(v):
290
+ if isinstance(v, (str, int, float)) and not isinstance(v, bool):
291
+ return str(v)
292
+ if isinstance(v, bool):
293
+ return "1" if v else ""
294
+ if isinstance(v, dict):
295
+ return str(v.get("name") or v.get("@id") or "")
296
+ if isinstance(v, list):
297
+ return ", ".join(_scalar(x) for x in v[:8])
298
+ return ""
299
+
300
+
301
+ # ---------------------------------------------------------------------------------------------
302
+ # CRON — a 5-field parser and a "what was the last fire time" walk
303
+ # ---------------------------------------------------------------------------------------------
304
+ # Deliberately NOT `croniter` (a dependency for ~60 lines) and deliberately NOT a forward
305
+ # scheduler. The question a tick asks is backwards-looking — *"was there a scheduled minute
306
+ # between the last run and now?"* — and answering it that way is what makes a missed tick
307
+ # self-healing: a container that was asleep for two hours runs once on wake, not eleven times and
308
+ # not never.
309
+
310
+ _FIELD_RANGES = [(0, 59), (0, 23), (1, 31), (1, 12), (0, 6)]
311
+
312
+
313
+ def _parse_field(spec, lo, hi):
314
+ out = set()
315
+ for part in str(spec).split(","):
316
+ part = part.strip()
317
+ if not part:
318
+ raise ValueError("empty cron field part")
319
+ step = 1
320
+ if "/" in part:
321
+ part, _, s = part.partition("/")
322
+ step = int(s)
323
+ if step < 1:
324
+ raise ValueError("cron step must be >= 1")
325
+ if part in ("*", "?"):
326
+ a, b = lo, hi
327
+ elif "-" in part.lstrip("-"):
328
+ a_s, _, b_s = part.partition("-")
329
+ a, b = int(a_s), int(b_s)
330
+ else:
331
+ a = b = int(part)
332
+ if a < lo or b > hi or a > b:
333
+ raise ValueError(f"cron value {part!r} out of range {lo}..{hi}")
334
+ out.update(range(a, b + 1, step))
335
+ return out
336
+
337
+
338
+ def parse_cron(expr):
339
+ """`'m h dom mon dow'` → `(minutes, hours, doms, months, dows, dom_restricted, dow_restricted)`.
340
+
341
+ Raises ValueError on anything malformed — a schedule that cannot be parsed must be refused at
342
+ WRITE time, because a cron string nobody can evaluate is an automation that silently never
343
+ runs and reports no error anywhere.
344
+ """
345
+ parts = str(expr or "").split()
346
+ if len(parts) != 5:
347
+ raise ValueError("a cron schedule has exactly 5 fields: minute hour day month weekday")
348
+ sets = [_parse_field(p, lo, hi) for p, (lo, hi) in zip(parts, _FIELD_RANGES)]
349
+ dom_r = parts[2].strip() not in ("*", "?")
350
+ dow_r = parts[4].strip() not in ("*", "?")
351
+ return (sets[0], sets[1], sets[2], sets[3], sets[4] | ({0} if 7 in sets[4] else set()),
352
+ dom_r, dow_r)
353
+
354
+
355
+ def _day_matches(day, doms, months, dows, dom_r, dow_r):
356
+ if day.month not in months:
357
+ return False
358
+ # POSIX rule: with BOTH day-of-month and weekday restricted the match is the UNION, not the
359
+ # intersection. `0 6 1 * 1` means "the 1st, and every Monday" — getting this backwards makes
360
+ # a schedule that looks right fire almost never.
361
+ dow = (day.weekday() + 1) % 7 # python Mon=0 -> cron Sun=0
362
+ if dom_r and dow_r:
363
+ return day.day in doms or dow in dows
364
+ if dom_r:
365
+ return day.day in doms
366
+ if dow_r:
367
+ return dow in dows
368
+ return True
369
+
370
+
371
+ def prev_fire(expr, now=None, lookback_days=400):
372
+ """The most recent scheduled minute at or before `now`, or None inside the lookback.
373
+
374
+ Walks by DAY (≤400 iterations) rather than by minute (≥500k) — the day fields decide first,
375
+ and only a matching day needs its hours/minutes searched.
376
+ """
377
+ minutes, hours, doms, months, dows, dom_r, dow_r = parse_cron(expr)
378
+ now = (now or _now()).replace(second=0, microsecond=0)
379
+ for back in range(lookback_days + 1):
380
+ day = (now - _dt.timedelta(days=back)).date()
381
+ if not _day_matches(day, doms, months, dows, dom_r, dow_r):
382
+ continue
383
+ same_day = back == 0
384
+ for h in sorted(hours, reverse=True):
385
+ if same_day and h > now.hour:
386
+ continue
387
+ for m in sorted(minutes, reverse=True):
388
+ if same_day and h == now.hour and m > now.minute:
389
+ continue
390
+ return _dt.datetime(day.year, day.month, day.day, h, m)
391
+ return None
392
+
393
+
394
+ def is_due(defn, now=None):
395
+ """Should the scheduler run this automation right now?
396
+
397
+ Due when a scheduled minute exists strictly AFTER the reference point (the last run, else the
398
+ moment the schedule was enabled, else creation) and at or before now. Anchoring on
399
+ `enabledAt` rather than firing on the first tick is what stops "enable a daily 06:00 job at
400
+ 14:00" from running immediately and looking like a bug.
401
+ """
402
+ if not isinstance(defn, dict):
403
+ return False
404
+ sched = defn.get("schedule") or {}
405
+ if not sched.get("enabled"):
406
+ return False
407
+ try:
408
+ fire = prev_fire(sched.get("cron"), now=now)
409
+ except ValueError:
410
+ return False
411
+ if fire is None:
412
+ return False
413
+ since = (_parse_iso((defn.get("status") or {}).get("lastRunAt"))
414
+ or _parse_iso(sched.get("enabledAt"))
415
+ or _parse_iso(defn.get("created")))
416
+ return since is None or fire > since
417
+
418
+
419
+ def next_fire(expr, now=None, lookahead_days=400):
420
+ """The next scheduled minute strictly after `now` — display only (the rail is `is_due`)."""
421
+ try:
422
+ minutes, hours, doms, months, dows, dom_r, dow_r = parse_cron(expr)
423
+ except ValueError:
424
+ return None
425
+ now = (now or _now()).replace(second=0, microsecond=0)
426
+ for ahead in range(lookahead_days + 1):
427
+ day = (now + _dt.timedelta(days=ahead)).date()
428
+ if not _day_matches(day, doms, months, dows, dom_r, dow_r):
429
+ continue
430
+ same_day = ahead == 0
431
+ for h in sorted(hours):
432
+ if same_day and h < now.hour:
433
+ continue
434
+ for m in sorted(minutes):
435
+ if same_day and h == now.hour and m <= now.minute:
436
+ continue
437
+ return _dt.datetime(day.year, day.month, day.day, h, m)
438
+ return None
439
+
440
+
441
+ # ---------------------------------------------------------------------------------------------
442
+ # THE UPSERT — pure, so the arithmetic is testable without a store or a network
443
+ # ---------------------------------------------------------------------------------------------
444
+
445
+ def upsert_rows(existing, incoming, key_field):
446
+ """Merge scraped rows into a user table's rows BY KEY. Returns `(rows, counts)`.
447
+
448
+ THE RULE THAT MATTERS: **an orphan is COUNTED, NEVER DELETED.** A row that has stopped
449
+ appearing on the source page has not necessarily stopped existing — the page changed its
450
+ filter, the fetch was partial, the site paginated. Deleting on absence turns any upstream
451
+ hiccup into silent data loss, and the row may be carrying user-typed overlay values in
452
+ columns the scrape never touches. So the run reports `orphans: N` and leaves them alone.
453
+
454
+ Only MAPPED keys are written: a re-run never clobbers a column a user added by hand.
455
+ """
456
+ rows = {str(k): dict(v or {}) for k, v in (existing or {}).items()}
457
+ counts = {"inserted": 0, "updated": 0, "unchanged": 0,
458
+ "skipped": 0, "duplicates": 0, "orphans": 0}
459
+ by_key, dupe_ids = {}, set()
460
+ for rid, row in rows.items():
461
+ kv = str(row.get(key_field, "") or "").strip()
462
+ if not kv:
463
+ continue
464
+ if kv in by_key:
465
+ dupe_ids.add(rid) # a pre-existing duplicate: first id wins, second left
466
+ continue
467
+ by_key[kv] = rid
468
+ next_id = max((int(r) for r in rows if str(r).isdigit()), default=0) + 1
469
+ seen_keys, incoming_dupes = set(), 0
470
+ for src in incoming or []:
471
+ kv = str((src or {}).get(key_field, "") or "").strip()
472
+ if not kv:
473
+ counts["skipped"] += 1 # no key -> cannot be upserted; never guessed
474
+ continue
475
+ if kv in seen_keys:
476
+ incoming_dupes += 1 # the SOURCE listed it twice; first wins
477
+ continue
478
+ seen_keys.add(kv)
479
+ rid = by_key.get(kv)
480
+ if rid is None:
481
+ if len(rows) >= MAX_UT_ROWS:
482
+ counts["skipped"] += 1
483
+ continue
484
+ rid = str(next_id)
485
+ next_id += 1
486
+ rows[rid] = dict(src)
487
+ by_key[kv] = rid
488
+ counts["inserted"] += 1
489
+ continue
490
+ before = rows[rid]
491
+ changed = {k: v for k, v in src.items() if str(before.get(k, "")) != str(v)}
492
+ if changed:
493
+ before.update(src)
494
+ counts["updated"] += 1
495
+ else:
496
+ counts["unchanged"] += 1
497
+ counts["duplicates"] = incoming_dupes + len(dupe_ids)
498
+ counts["orphans"] = sum(
499
+ 1 for kv, rid in by_key.items() if kv not in seen_keys and rid not in dupe_ids)
500
+ return rows, counts
501
+
502
+
503
+ # ---------------------------------------------------------------------------------------------
504
+ # USER TABLES — read/write through the RUNTIME (never core.user_tables' module-global key)
505
+ # ---------------------------------------------------------------------------------------------
506
+
507
+ def _ut_slug(label):
508
+ s = re.sub(r"[^a-z0-9]+", "_", str(label or "").strip().lower()).strip("_")
509
+ return (s or "table")[:40]
510
+
511
+
512
+ def ut_all(rt):
513
+ try:
514
+ return dict(rt.get(UT_STORE_KEY) or {})
515
+ except Exception:
516
+ return {}
517
+
518
+
519
+ def ut_get(rt, key):
520
+ return ut_all(rt).get(str(key))
521
+
522
+
523
+ def ut_ensure(rt, label, fields, username="automation", key=None):
524
+ """Create the table if it is missing; return its key. Idempotent — a re-run of an automation
525
+ that owns a table must not spawn `ut_x_2`, so the key is DERIVED from the label (or given)
526
+ and an existing table with that key is adopted, not duplicated.
527
+
528
+ Fields are MERGED, never replaced: a user who added a column to an automation's table keeps
529
+ it, and a new source column joins on the next run.
530
+ """
531
+ key = str(key or (UT_PREFIX + _ut_slug(label)))
532
+ if not key.startswith(UT_PREFIX):
533
+ key = UT_PREFIX + key
534
+ wanted = list(fields or [])
535
+ created = _iso()
536
+
537
+ # ⚠ SKIP THE WRITE WHEN NOTHING WOULD CHANGE. Without this, every re-run spends a store
538
+ # commit re-writing an identical definition — against a 20 s flush floor and a 256/hr repo
539
+ # budget, an idempotent helper that always writes is the same defect as a per-row insert.
540
+ have = ut_get(rt, key)
541
+ if have is not None and not [f for f in wanted
542
+ if f.get("key") not in {g.get("key")
543
+ for g in (have.get("fields") or [])}]:
544
+ return key
545
+
546
+ def _up(cur):
547
+ cur = cur if isinstance(cur, dict) else {}
548
+ t = cur.get(key)
549
+ if t is None:
550
+ if len(cur) >= MAX_UT_TABLES:
551
+ return cur
552
+ cur[key] = {"key": key, "label": str(label)[:60], "source": "Automation",
553
+ "createdBy": username, "created": created,
554
+ "fields": wanted, "rows": {}}
555
+ return cur
556
+ have = {f.get("key") for f in (t.get("fields") or [])}
557
+ for f in wanted:
558
+ if f.get("key") not in have:
559
+ t.setdefault("fields", []).append(f)
560
+ have.add(f.get("key"))
561
+ # An automation's table SAYS an automation owns it — the nav badge reads from this.
562
+ t["source"] = t.get("source") or "Automation"
563
+ return cur
564
+
565
+ rt.update(UT_STORE_KEY, _up, flush="sync")
566
+ return key
567
+
568
+
569
+ def ut_write_rows(rt, key, rows):
570
+ """ONE store update for the WHOLE row set — the flush-ceiling rule (see the module header)."""
571
+ def _up(cur):
572
+ cur = cur if isinstance(cur, dict) else {}
573
+ t = cur.get(key)
574
+ if t is not None:
575
+ t["rows"] = rows
576
+ return cur
577
+ rt.update(UT_STORE_KEY, _up, flush="sync")
578
+
579
+
580
+ def ut_set_cell(rt, key, row_id, field_key, value, extra_rows=None):
581
+ """Write one automation-owned cell (+ optional whole extra tables) in ONE update."""
582
+ def _up(cur):
583
+ cur = cur if isinstance(cur, dict) else {}
584
+ t = cur.get(key)
585
+ if t is not None:
586
+ t.setdefault("rows", {}).setdefault(str(row_id), {})[field_key] = value
587
+ for k, rws in (extra_rows or {}).items():
588
+ tt = cur.get(k)
589
+ if tt is not None:
590
+ tt["rows"] = rws
591
+ return cur
592
+ rt.update(UT_STORE_KEY, _up, flush="sync")
593
+
594
+
595
+ def field_def(text_key, label, ftype="text"):
596
+ return {"key": text_key, "label": label, "type": ftype, "source": "overlay"}
597
+
598
+
599
+ # ---------------------------------------------------------------------------------------------
600
+ # DEFINITIONS — validation and the bucket's read/write half
601
+ # ---------------------------------------------------------------------------------------------
602
+
603
+ def _s(v, n=200):
604
+ return str(v if v is not None else "")[:n]
605
+
606
+
607
+ def clean_config(kind, raw):
608
+ """Validate a kind's config. Returns `(config, error)`; error is a user-facing sentence."""
609
+ raw = raw if isinstance(raw, dict) else {}
610
+ if kind == "scrape_db":
611
+ url = _s(raw.get("url"), 2000).strip()
612
+ if not url:
613
+ return None, "a source URL is required"
614
+ try:
615
+ guard(url)
616
+ except Refused as e:
617
+ return None, str(e)
618
+ extract = raw.get("extract") if raw.get("extract") in EXTRACTS else "table"
619
+ fmap = {}
620
+ for k, v in list((raw.get("fieldMap") or {}).items())[:60]:
621
+ tk = re.sub(r"[^a-z0-9_]+", "_", _s(v, 60).strip().lower()).strip("_")
622
+ if tk:
623
+ fmap[_s(k, 120)] = tk[:60]
624
+ if not fmap:
625
+ return None, "map at least one source column to a field"
626
+ key_field = _s(raw.get("keyField"), 60).strip()
627
+ if key_field not in fmap.values():
628
+ return None, "the key field must be one of the mapped fields"
629
+ target = _s(raw.get("targetTable"), 60).strip()
630
+ if target and not target.startswith(UT_PREFIX):
631
+ target = UT_PREFIX + target
632
+ return {"url": url, "extract": extract,
633
+ "tableIndex": max(0, min(int(raw.get("tableIndex") or 0), 50)),
634
+ "fieldMap": fmap, "keyField": key_field,
635
+ "targetTable": target,
636
+ "targetLabel": _s(raw.get("targetLabel"), 60).strip() or "Scraped table"}, None
637
+ if kind == "field_instagram":
638
+ table = _s(raw.get("targetTable"), 60).strip()
639
+ if not table.startswith(UT_PREFIX):
640
+ return None, "an Instagram automation runs against a blank database (ut_*)"
641
+ fkey = _s(raw.get("fieldKey"), 80).strip()
642
+ if not fkey:
643
+ return None, "pick the automation column this run writes into"
644
+ return {"targetTable": table, "fieldKey": fkey,
645
+ "urlField": _s(raw.get("urlField"), 80).strip(),
646
+ "maxPosts": max(1, min(int(raw.get("maxPosts") or 24), 200))}, None
647
+ return None, f"unknown automation kind {kind!r}"
648
+
649
+
650
+ def clean_schedule(raw, previous=None):
651
+ """Validate `{cron, enabled}` and stamp `enabledAt` on the OFF→ON edge (the anchor `is_due`
652
+ measures from — without it, enabling a daily job fires it immediately)."""
653
+ raw = raw if isinstance(raw, dict) else {}
654
+ cron = _s(raw.get("cron"), 120).strip() or "0 6 * * *"
655
+ parse_cron(cron) # raises -> the route answers 400
656
+ enabled = bool(raw.get("enabled"))
657
+ prev = previous or {}
658
+ out = {"cron": cron, "enabled": enabled}
659
+ if enabled:
660
+ out["enabledAt"] = (prev.get("enabledAt") if prev.get("enabled") else None) or _iso()
661
+ return out
662
+
663
+
664
+ def clean_definition(raw, previous=None, username=""):
665
+ """Whole-definition validation. Returns `(defn, error)`."""
666
+ raw = raw if isinstance(raw, dict) else {}
667
+ prev = previous or {}
668
+ kind = _s(raw.get("kind") or prev.get("kind"), 40)
669
+ if kind not in KINDS:
670
+ return None, f"unknown automation kind {kind!r}"
671
+ name = " ".join(_s(raw.get("name") or prev.get("name") or "", MAX_NAME).split())
672
+ if not name:
673
+ return None, "name the automation"
674
+ cfg_raw = raw.get("config") if isinstance(raw.get("config"), dict) else prev.get("config")
675
+ config, err = clean_config(kind, cfg_raw)
676
+ if err:
677
+ return None, err
678
+ try:
679
+ schedule = clean_schedule(
680
+ raw.get("schedule") if isinstance(raw.get("schedule"), dict)
681
+ else prev.get("schedule"), prev.get("schedule"))
682
+ except ValueError as e:
683
+ return None, str(e)
684
+ return {
685
+ "id": _s(prev.get("id") or raw.get("id"), 40),
686
+ "name": name, "kind": kind, "config": config, "schedule": schedule,
687
+ "status": prev.get("status") or {"state": "idle", "lastRunAt": "", "lastSummary": ""},
688
+ "runs": list(prev.get("runs") or [])[:MAX_RUNS],
689
+ "created": prev.get("created") or _iso(),
690
+ "createdBy": prev.get("createdBy") or username,
691
+ }, None
692
+
693
+
694
+ def all_definitions(rt):
695
+ try:
696
+ return dict(rt.get(STORE_KEY) or {})
697
+ except Exception:
698
+ return {}
699
+
700
+
701
+ def _new_id(existing):
702
+ n = 1
703
+ while f"auto_{n}" in existing:
704
+ n += 1
705
+ return f"auto_{n}"
706
+
707
+
708
+ def create(rt, raw, username=""):
709
+ existing = all_definitions(rt)
710
+ if len(existing) >= MAX_AUTOMATIONS:
711
+ return None, f"this workspace is at the {MAX_AUTOMATIONS}-automation limit"
712
+ defn, err = clean_definition(raw, None, username)
713
+ if err:
714
+ return None, err
715
+ defn["id"] = _new_id(existing)
716
+
717
+ def _up(cur):
718
+ cur = cur if isinstance(cur, dict) else {}
719
+ cur[defn["id"]] = defn
720
+ return cur
721
+
722
+ rt.update(STORE_KEY, _up, flush="sync")
723
+ return defn, None
724
+
725
+
726
+ def patch(rt, auto_id, raw, username=""):
727
+ existing = all_definitions(rt)
728
+ prev = existing.get(str(auto_id))
729
+ if prev is None:
730
+ return None, "no such automation"
731
+ merged = dict(prev)
732
+ for k in ("name", "config", "schedule", "kind"):
733
+ if k in (raw or {}):
734
+ merged[k] = raw[k]
735
+ defn, err = clean_definition(merged, prev, username)
736
+ if err:
737
+ return None, err
738
+ defn["id"] = str(auto_id)
739
+
740
+ def _up(cur):
741
+ cur = cur if isinstance(cur, dict) else {}
742
+ cur[defn["id"]] = defn
743
+ return cur
744
+
745
+ rt.update(STORE_KEY, _up, flush="sync")
746
+ return defn, None
747
+
748
+
749
+ def remove(rt, auto_id):
750
+ def _up(cur):
751
+ cur = cur if isinstance(cur, dict) else {}
752
+ cur.pop(str(auto_id), None)
753
+ return cur
754
+ rt.update(STORE_KEY, _up, flush="sync")
755
+
756
+
757
+ # ---------------------------------------------------------------------------------------------
758
+ # RUN STATE — process memory only (see the module header for why this may never be persisted)
759
+ # ---------------------------------------------------------------------------------------------
760
+
761
+ _RUN_LOCK = threading.RLock()
762
+ _RUNNING = {} # (tenant, id) -> {'startedAt', 'step'}
763
+
764
+
765
+ def running(tenant, auto_id):
766
+ with _RUN_LOCK:
767
+ return dict(_RUNNING.get((tenant, str(auto_id))) or {}) or None
768
+
769
+
770
+ def _claim(tenant, auto_id):
771
+ with _RUN_LOCK:
772
+ if (tenant, str(auto_id)) in _RUNNING:
773
+ return False
774
+ _RUNNING[(tenant, str(auto_id))] = {"startedAt": _iso(), "step": "starting"}
775
+ return True
776
+
777
+
778
+ def _step(tenant, auto_id, text):
779
+ with _RUN_LOCK:
780
+ cur = _RUNNING.get((tenant, str(auto_id)))
781
+ if cur is not None:
782
+ cur["step"] = text
783
+
784
+
785
+ def _release(tenant, auto_id):
786
+ with _RUN_LOCK:
787
+ _RUNNING.pop((tenant, str(auto_id)), None)
788
+
789
+
790
+ def _commit_run(rt, auto_id, state, summary, counts, ok, affected=None):
791
+ """THE ONE store write a run performs on the `automations` bucket."""
792
+ entry = {"ts": _iso(), "ok": bool(ok), "summary": _s(summary, 400),
793
+ "counts": {k: int(v) for k, v in (counts or {}).items()
794
+ if isinstance(v, (int, float))},
795
+ "affected": list(affected or [])[:200]}
796
+
797
+ def _up(cur):
798
+ cur = cur if isinstance(cur, dict) else {}
799
+ d = cur.get(str(auto_id))
800
+ if d is None:
801
+ return cur
802
+ d["status"] = {"state": state, "lastRunAt": entry["ts"],
803
+ "lastSummary": entry["summary"]}
804
+ d["runs"] = ([entry] + list(d.get("runs") or []))[:MAX_RUNS]
805
+ return cur
806
+
807
+ rt.update(STORE_KEY, _up, flush="sync")
808
+ return entry
809
+
810
+
811
+ # ---------------------------------------------------------------------------------------------
812
+ # INSTAGRAM (R7) — ANONYMOUS PUBLIC ENDPOINTS ONLY
813
+ # ---------------------------------------------------------------------------------------------
814
+ # The rails, restated because they are the whole reason this is allowed to exist: **no login, no
815
+ # credentials, no session cookie, public data only.** Nothing here authenticates, and nothing
816
+ # here should ever be given something to authenticate WITH.
817
+ #
818
+ # ⚠ HONEST STATUSES ARE THE FEATURE. Instagram rate-limits and outright blocks datacenter egress
819
+ # (the HF Space and any AWS Lambda are both in that class), and its anonymous surface has been
820
+ # progressively closed for years. So this returns a STATE — ok / partial / blocked / error — and
821
+ # the cell says which. A pull that quietly wrote zero posts and reported success would be worse
822
+ # than no automation at all: the table would look maintained and be empty.
823
+ #
824
+ # THE LADDER, tried in order with pacing between rungs. Each rung records how it did in `via`,
825
+ # so a run history answers "what still works anonymously?" from data rather than from memory.
826
+
827
+ PACE_SECONDS = float(os.environ.get("AIOS_IG_PACE_SECONDS") or 2.5)
828
+ IG_APP_ID = "936619743392459" # the public web client id the browser itself sends
829
+ IG_MAX_RETRIES = 2
830
+
831
+ SNAPSHOT_FIELDS = [
832
+ field_def("snapshot_key", "Snapshot"), field_def("influencer_key", "Influencer"),
833
+ field_def("pulled_at", "Pulled at"), field_def("followers", "Followers"),
834
+ field_def("following", "Following"), field_def("posts_count", "Posts"),
835
+ field_def("full_name", "Name"), field_def("bio", "Bio"),
836
+ field_def("verified", "Verified"), field_def("source", "Read via"),
837
+ field_def("approx", "Counts are approximate"),
838
+ ]
839
+ POST_FIELDS = [
840
+ field_def("shortcode", "Shortcode"), field_def("influencer_key", "Influencer"),
841
+ field_def("posted_at", "Posted at"), field_def("type", "Type"),
842
+ field_def("caption", "Caption"), field_def("url", "URL", "url"),
843
+ ]
844
+ POST_SNAPSHOT_FIELDS = [
845
+ field_def("post_snapshot_key", "Snapshot"), field_def("shortcode", "Shortcode"),
846
+ field_def("pulled_at", "Pulled at"), field_def("likes", "Likes"),
847
+ field_def("comments", "Comments"),
848
+ ]
849
+
850
+
851
+ def ig_handle(url):
852
+ """The handle out of a profile URL (or a bare handle). '' when it is not one."""
853
+ raw = str(url or "").strip()
854
+ if not raw:
855
+ return ""
856
+ if raw.startswith("@"):
857
+ return re.sub(r"[^A-Za-z0-9._]", "", raw[1:])[:40]
858
+ if "instagram.com" not in raw:
859
+ return re.sub(r"[^A-Za-z0-9._]", "", raw)[:40] if "/" not in raw else ""
860
+ path = urlparse(raw if "//" in raw else "https://" + raw).path.strip("/")
861
+ first = (path.split("/") or [""])[0]
862
+ if first in ("p", "reel", "reels", "explore", "accounts", "stories", ""):
863
+ return ""
864
+ return re.sub(r"[^A-Za-z0-9._]", "", first)[:40]
865
+
866
+
867
+ def _ig_get(url, headers=None, timeout=20.0):
868
+ """One anonymous GET with 429 backoff. Returns `(status, body_bytes, note)`."""
869
+ delay = PACE_SECONDS
870
+ for attempt in range(IG_MAX_RETRIES + 1):
871
+ try:
872
+ status, _final, body = fetch(url, timeout=timeout, headers=headers, max_kb=4096)
873
+ except Refused:
874
+ raise
875
+ except Exception as e: # noqa: BLE001
876
+ return 0, b"", f"{type(e).__name__}: {str(e)[:120]}"
877
+ if status == 429 and attempt < IG_MAX_RETRIES:
878
+ time.sleep(delay)
879
+ delay *= 2 # exponential, never a tight retry
880
+ continue
881
+ return status, body, ""
882
+ return 429, b"", "rate limited after retries"
883
+
884
+
885
+ def _ig_int(v):
886
+ try:
887
+ return int(v)
888
+ except Exception:
889
+ return None
890
+
891
+
892
+ def _profile_from_web_api(node, handle):
893
+ return {
894
+ "username": node.get("username") or handle,
895
+ "full_name": node.get("full_name") or "",
896
+ "bio": node.get("biography") or "",
897
+ "followers": _ig_int((node.get("edge_followed_by") or {}).get("count")),
898
+ "following": _ig_int((node.get("edge_follow") or {}).get("count")),
899
+ "posts_count": _ig_int((node.get("edge_owner_to_timeline_media") or {}).get("count")),
900
+ "verified": "1" if node.get("is_verified") else "",
901
+ }
902
+
903
+
904
+ def _posts_from_edges(edges):
905
+ out = []
906
+ for e in edges or []:
907
+ n = (e or {}).get("node") or {}
908
+ code = n.get("shortcode")
909
+ if not code:
910
+ continue
911
+ caption = ""
912
+ for ce in ((n.get("edge_media_to_caption") or {}).get("edges") or []):
913
+ caption = ((ce or {}).get("node") or {}).get("text") or ""
914
+ break
915
+ ts = n.get("taken_at_timestamp")
916
+ out.append({
917
+ "shortcode": code,
918
+ "posted_at": (_dt.datetime.fromtimestamp(ts).strftime("%Y-%m-%d %H:%M")
919
+ if isinstance(ts, (int, float)) else ""),
920
+ "type": ("video" if n.get("is_video") else
921
+ "carousel" if n.get("__typename") == "GraphSidecar" else "image"),
922
+ "caption": caption,
923
+ "url": f"https://www.instagram.com/p/{code}/",
924
+ "likes": _ig_int((n.get("edge_liked_by") or n.get("edge_media_preview_like")
925
+ or {}).get("count")),
926
+ "comments": _ig_int((n.get("edge_media_to_comment") or {}).get("count")),
927
+ })
928
+ return out
929
+
930
+
931
+ _OG_COUNTS = re.compile(
932
+ r"([\d.,KMkm]+)\s+Followers?,\s*([\d.,KMkm]+)\s+Following,\s*([\d.,KMkm]+)\s+Posts?")
933
+
934
+
935
+ def _loose_count(txt):
936
+ """'1.2M' / '12,345' -> int. IG's og:description abbreviates, so the snapshot is honest
937
+ about being approximate rather than pretending to a precision it does not have."""
938
+ t = str(txt or "").strip().replace(",", "")
939
+ mult = 1
940
+ if t[-1:].lower() == "k":
941
+ mult, t = 1_000, t[:-1]
942
+ elif t[-1:].lower() == "m":
943
+ mult, t = 1_000_000, t[:-1]
944
+ try:
945
+ return int(float(t) * mult)
946
+ except Exception:
947
+ return None
948
+
949
+
950
+ def pull_profile(url, max_posts=24, log=print):
951
+ """Everything anonymously readable about one public profile.
952
+
953
+ Returns `{state, profile, posts, via, note}` with state ∈ ok | partial | blocked | error.
954
+ `partial` means the identity was read but the media was not — which is the outcome the
955
+ anonymous surface most often gives, and saying so is the point.
956
+ """
957
+ handle = ig_handle(url)
958
+ if not handle:
959
+ return {"state": "error", "note": f"{url!r} is not an Instagram profile URL",
960
+ "profile": {}, "posts": [], "via": ""}
961
+ attempts = []
962
+
963
+ # RUNG 1 — the web client's own profile endpoint. Public, no cookie; the one that used to
964
+ # carry both the profile AND the first page of media in a single answer.
965
+ api = f"https://www.instagram.com/api/v1/users/web_profile_info/?username={handle}"
966
+ status, body, note = _ig_get(api, headers={
967
+ "X-IG-App-ID": IG_APP_ID, "Accept": "*/*",
968
+ "Referer": f"https://www.instagram.com/{handle}/"})
969
+ attempts.append(f"web_profile_info:{status or note}")
970
+ if 200 <= status < 300 and body:
971
+ try:
972
+ node = ((json.loads(body.decode("utf-8", "replace")) or {})
973
+ .get("data") or {}).get("user") or {}
974
+ except Exception:
975
+ node = {}
976
+ if node:
977
+ profile = _profile_from_web_api(node, handle)
978
+ media = node.get("edge_owner_to_timeline_media") or {}
979
+ posts = _posts_from_edges(media.get("edges"))[:max_posts]
980
+ page = media.get("page_info") or {}
981
+ if page.get("has_next_page") and page.get("end_cursor") and len(posts) < max_posts:
982
+ more, pnote = _paginate(node.get("id"), page.get("end_cursor"),
983
+ max_posts - len(posts), log)
984
+ posts.extend(more)
985
+ if pnote:
986
+ attempts.append(pnote)
987
+ state = "ok" if posts else "partial"
988
+ return {"state": state, "profile": profile, "posts": posts,
989
+ "via": "web_profile_info",
990
+ "note": "" if posts else "profile read; media edges empty ("
991
+ + ", ".join(attempts) + ")"}
992
+
993
+ time.sleep(PACE_SECONDS)
994
+
995
+ # RUNG 2 — the profile HTML. Its og:description carries the three counts even when every
996
+ # JSON surface refuses, so it is the honest floor: identity without media.
997
+ status, body, note = _ig_get(f"https://www.instagram.com/{handle}/")
998
+ attempts.append(f"html:{status or note}")
999
+ if 200 <= status < 300 and body:
1000
+ text = body.decode("utf-8", "replace")
1001
+ if "/accounts/login" in text[:4000] and "og:description" not in text:
1002
+ return {"state": "blocked", "profile": {}, "posts": [], "via": "html",
1003
+ "note": "Instagram served the login wall (" + ", ".join(attempts) + ")"}
1004
+ desc = ""
1005
+ m = re.search(r'<meta[^>]+property="og:description"[^>]+content="([^"]*)"', text)
1006
+ if m:
1007
+ desc = m.group(1)
1008
+ counts = _OG_COUNTS.search(desc or "")
1009
+ title = ""
1010
+ mt = re.search(r'<meta[^>]+property="og:title"[^>]+content="([^"]*)"', text)
1011
+ if mt:
1012
+ title = mt.group(1)
1013
+ if counts:
1014
+ profile = {
1015
+ "username": handle, "full_name": title.split("(")[0].strip(),
1016
+ # ⚠ NO BIO. `og:description` on a profile page is
1017
+ # "204K Followers, 5,245 Following, 750 Posts - See Instagram photos and videos
1018
+ # from Inayma (@inayma)" — the tail is Instagram's own boilerplate, not the
1019
+ # person's biography. Storing it would fill a Bio column with the same sentence
1020
+ # for every influencer, which reads as data and is not. Measured 2026-08-04.
1021
+ "bio": "",
1022
+ "followers": _loose_count(counts.group(1)),
1023
+ "following": _loose_count(counts.group(2)),
1024
+ "posts_count": _loose_count(counts.group(3)),
1025
+ "verified": "",
1026
+ # The page abbreviates ("204K"), so the number is the page's rounding, not a
1027
+ # count we read. Recorded on the row rather than left for someone to discover
1028
+ # when 204,000 fails to reconcile with anything.
1029
+ "approx": "1",
1030
+ }
1031
+ return {"state": "partial", "profile": profile, "posts": [], "via": "og:description",
1032
+ "note": "counts are the page's own abbreviations; media is not anonymously "
1033
+ "readable (" + ", ".join(attempts) + ")"}
1034
+ if status in (401, 403, 429) or status == 0:
1035
+ return {"state": "blocked", "profile": {}, "posts": [], "via": "",
1036
+ "note": f"Instagram refused the anonymous read ({', '.join(attempts)})"}
1037
+ return {"state": "error", "profile": {}, "posts": [], "via": "",
1038
+ "note": f"nothing anonymously readable ({', '.join(attempts)})"}
1039
+
1040
+
1041
+ #: The historical anonymous media query. Kept because trying it and RECORDING the refusal is
1042
+ #: what makes "as deep as anonymously accessible" (R7) a measurement instead of an assumption.
1043
+ IG_MEDIA_QUERY_HASH = "e769aa130647d2354c40ea6a439bfc08"
1044
+
1045
+
1046
+ def _paginate(user_id, cursor, want, log=print):
1047
+ """Walk further back through the media edges while the anonymous surface allows it."""
1048
+ out, note = [], ""
1049
+ while user_id and cursor and len(out) < want:
1050
+ time.sleep(PACE_SECONDS)
1051
+ variables = json.dumps({"id": str(user_id), "first": min(50, want - len(out)),
1052
+ "after": cursor}, separators=(",", ":"))
1053
+ url = (f"https://www.instagram.com/graphql/query/?query_hash={IG_MEDIA_QUERY_HASH}"
1054
+ f"&variables={requests.utils.quote(variables)}")
1055
+ status, body, err = _ig_get(url, headers={"X-IG-App-ID": IG_APP_ID})
1056
+ if not (200 <= status < 300) or not body:
1057
+ note = f"graphql:{status or err}"
1058
+ break
1059
+ try:
1060
+ media = (((json.loads(body.decode("utf-8", "replace")) or {}).get("data") or {})
1061
+ .get("user") or {}).get("edge_owner_to_timeline_media") or {}
1062
+ except Exception:
1063
+ note = "graphql:unparseable"
1064
+ break
1065
+ edges = media.get("edges") or []
1066
+ if not edges:
1067
+ note = "graphql:empty"
1068
+ break
1069
+ out.extend(_posts_from_edges(edges))
1070
+ page = media.get("page_info") or {}
1071
+ if not page.get("has_next_page"):
1072
+ break
1073
+ cursor = page.get("end_cursor")
1074
+ return out[:want], note
1075
+
1076
+
1077
+ # ---------------------------------------------------------------------------------------------
1078
+ # RUNNERS
1079
+ # ---------------------------------------------------------------------------------------------
1080
+
1081
+ def run_scrape_db(rt, defn, username="automation", log=print):
1082
+ """Automation #1 (R6): a public page → a blank database, re-runnable, UPSERT by key."""
1083
+ cfg = defn.get("config") or {}
1084
+ fmap, key_field = cfg.get("fieldMap") or {}, cfg.get("keyField")
1085
+ status, final, body = fetch(cfg["url"])
1086
+ if not (200 <= status < 300):
1087
+ return ("error", f"{cfg['url']} answered {status} — nothing was written",
1088
+ {"status": status}, [])
1089
+ soup = _soup(body)
1090
+ if cfg.get("extract") == "jsonld":
1091
+ blocks = jsonld(soup)
1092
+ flat = []
1093
+ for b in blocks:
1094
+ if isinstance(b, list):
1095
+ flat.extend([x for x in b if isinstance(x, dict)])
1096
+ elif isinstance(b, dict):
1097
+ items = b.get("itemListElement")
1098
+ flat.extend([x for x in items if isinstance(x, dict)] if isinstance(items, list)
1099
+ else [b])
1100
+ raw_rows = [{k: _scalar(v) for k, v in d.items()} for d in flat]
1101
+ else:
1102
+ found = tables(soup)
1103
+ idx = max(0, min(int(cfg.get("tableIndex") or 0), len(found) - 1)) if found else 0
1104
+ raw_rows = [r for r in (found[idx] if found else []) if isinstance(r, dict)]
1105
+ if not raw_rows:
1106
+ return ("error", "the page parsed but the selected table had no rows", {"rows": 0}, [])
1107
+
1108
+ incoming = []
1109
+ for r in raw_rows:
1110
+ mapped = {tgt: _s(r.get(src, ""), 500) for src, tgt in fmap.items()}
1111
+ if any(v for v in mapped.values()):
1112
+ incoming.append(mapped)
1113
+
1114
+ fields = [field_def(tgt, src if len(src) <= 60 else src[:60])
1115
+ for src, tgt in fmap.items()]
1116
+ # the key column leads, so the table opens on the identity it is upserted by
1117
+ fields.sort(key=lambda f: 0 if f["key"] == key_field else 1)
1118
+ table_key = ut_ensure(rt, cfg.get("targetLabel") or defn.get("name") or "Scraped table",
1119
+ fields, username, key=cfg.get("targetTable") or None)
1120
+ t = ut_get(rt, table_key) or {}
1121
+ rows, counts = upsert_rows(t.get("rows") or {}, incoming, key_field)
1122
+ ut_write_rows(rt, table_key, rows)
1123
+ counts["source_rows"] = len(incoming)
1124
+ affected = [rid for rid, row in rows.items()
1125
+ if str(row.get(key_field, "")) in {str(i.get(key_field)) for i in incoming}][:200]
1126
+ summary = (f"{counts['inserted']} new, {counts['updated']} updated, "
1127
+ f"{counts['unchanged']} unchanged, {counts['orphans']} no longer on the page "
1128
+ f"(kept)")
1129
+ state = "partial" if counts["skipped"] else "ok"
1130
+ log(f"[aios-auto] scrape_db {defn.get('id')} -> {table_key}: {summary}")
1131
+ return (state, summary, counts, affected)
1132
+
1133
+
1134
+ def run_field_instagram(rt, defn, username="automation", log=print):
1135
+ """Automation #2 (R7): for every row of a blank database that carries a profile URL, pull
1136
+ the public profile anonymously and write a status string into the automation column."""
1137
+ cfg = defn.get("config") or {}
1138
+ table_key, fkey = cfg.get("targetTable"), cfg.get("fieldKey")
1139
+ t = ut_get(rt, table_key)
1140
+ if not t:
1141
+ return ("error", f"{table_key} is not a database in this workspace", {}, [])
1142
+ url_field = cfg.get("urlField") or _auto_url_field(t, fkey)
1143
+ if not url_field:
1144
+ return ("error", "the automation column has no URL field bound to it", {}, [])
1145
+ rows = dict(t.get("rows") or {})
1146
+ # the relational tables the pull lands in (R7): every row timestamped for time-range filters
1147
+ snap_key = ut_ensure(rt, "IG snapshots", SNAPSHOT_FIELDS, username, key="ut_ig_snapshots")
1148
+ post_key = ut_ensure(rt, "IG posts", POST_FIELDS, username, key="ut_ig_posts")
1149
+ ps_key = ut_ensure(rt, "IG post snapshots", POST_SNAPSHOT_FIELDS, username,
1150
+ key="ut_ig_post_snapshots")
1151
+ snaps = dict((ut_get(rt, snap_key) or {}).get("rows") or {})
1152
+ posts = dict((ut_get(rt, post_key) or {}).get("rows") or {})
1153
+ psnaps = dict((ut_get(rt, ps_key) or {}).get("rows") or {})
1154
+
1155
+ counts = {"profiles": 0, "ok": 0, "partial": 0, "blocked": 0, "error": 0,
1156
+ "posts": 0, "new_posts": 0}
1157
+ cells, affected, notes = {}, [], []
1158
+ targets = [(rid, str(r.get(url_field, "") or "").strip())
1159
+ for rid, r in rows.items() if str(r.get(url_field, "") or "").strip()]
1160
+ for i, (rid, url) in enumerate(targets):
1161
+ if i:
1162
+ time.sleep(PACE_SECONDS) # ≥2 s between profiles (R7)
1163
+ counts["profiles"] += 1
1164
+ res = pull_profile(url, max_posts=cfg.get("maxPosts") or 24, log=log)
1165
+ pulled = _iso()
1166
+ if res["state"] == "ok" or res["state"] == "partial":
1167
+ counts["ok" if res["state"] == "ok" else "partial"] += 1
1168
+ prof = res["profile"]
1169
+ snaps, _c = upsert_rows(
1170
+ snaps, [{"snapshot_key": f"{prof['username']}@{pulled}",
1171
+ "influencer_key": prof["username"], "pulled_at": pulled,
1172
+ "followers": _s(prof.get("followers")), "following": _s(prof.get("following")),
1173
+ "posts_count": _s(prof.get("posts_count")), "full_name": _s(prof.get("full_name")),
1174
+ "bio": _s(prof.get("bio"), 500), "verified": _s(prof.get("verified")),
1175
+ "source": _s(res.get("via")), "approx": _s(prof.get("approx"))}],
1176
+ "snapshot_key")
1177
+ new_posts = 0
1178
+ for p in res.get("posts") or []:
1179
+ posts, pc = upsert_rows(posts, [{
1180
+ "shortcode": p["shortcode"], "influencer_key": prof["username"],
1181
+ "posted_at": _s(p.get("posted_at")), "type": _s(p.get("type")),
1182
+ "caption": _s(p.get("caption"), 800), "url": _s(p.get("url"), 300),
1183
+ }], "shortcode")
1184
+ new_posts += pc["inserted"]
1185
+ psnaps, _pc = upsert_rows(psnaps, [{
1186
+ "post_snapshot_key": f"{p['shortcode']}@{pulled}",
1187
+ "shortcode": p["shortcode"], "pulled_at": pulled,
1188
+ "likes": _s(p.get("likes")), "comments": _s(p.get("comments")),
1189
+ }], "post_snapshot_key")
1190
+ counts["posts"] += len(res.get("posts") or [])
1191
+ counts["new_posts"] += new_posts
1192
+ detail = f"{prof.get('followers') or '?'} followers"
1193
+ if res.get("posts"):
1194
+ detail += f", {len(res['posts'])} posts"
1195
+ elif res["state"] == "partial":
1196
+ detail += ", posts not anonymously readable"
1197
+ cells[rid] = f"{res['state']} · {_stamp()} · {detail}"
1198
+ elif res["state"] == "blocked":
1199
+ counts["blocked"] += 1
1200
+ cells[rid] = f"blocked · {_stamp()} · {_s(res.get('note'), 90)}"
1201
+ notes.append(res.get("note") or "blocked")
1202
+ else:
1203
+ counts["error"] += 1
1204
+ cells[rid] = f"error · {_stamp()} · {_s(res.get('note'), 90)}"
1205
+ notes.append(res.get("note") or "error")
1206
+ affected.append(rid)
1207
+
1208
+ def _up(cur):
1209
+ cur = cur if isinstance(cur, dict) else {}
1210
+ tt = cur.get(table_key)
1211
+ if tt is not None:
1212
+ for rid, val in cells.items():
1213
+ tt.setdefault("rows", {}).setdefault(str(rid), {})[fkey] = val
1214
+ for k, rws in ((snap_key, snaps), (post_key, posts), (ps_key, psnaps)):
1215
+ tgt = cur.get(k)
1216
+ if tgt is not None:
1217
+ tgt["rows"] = rws
1218
+ return cur
1219
+
1220
+ rt.update(UT_STORE_KEY, _up, flush="sync") # ONE coalesced update for all four tables
1221
+ # ⚠ A RUN IS ONLY 'ok' IF EVERY PROFILE WAS. The first live run rolled up to 'ok' while
1222
+ # every CELL said `partial`, because the rollup only asked about blocked/error — so the rail
1223
+ # showed a green dot over a table with no posts in it. A summary that disagrees with the
1224
+ # cells it summarises is the failure this whole module's honest-status rule exists to
1225
+ # prevent, so `partial` now propagates. Measured 2026-08-04.
1226
+ read = counts["ok"] + counts["partial"]
1227
+ if not read:
1228
+ state = "error" if counts["error"] else "partial" if counts["blocked"] else "ok"
1229
+ elif counts["blocked"] or counts["error"] or counts["partial"]:
1230
+ state = "partial"
1231
+ else:
1232
+ state = "ok"
1233
+ summary = (f"{read}/{counts['profiles']} profiles read, {counts['posts']} posts "
1234
+ f"({counts['new_posts']} new)")
1235
+ if counts["partial"]:
1236
+ summary += f", {counts['partial']} profile-only (posts not anonymously readable)"
1237
+ if counts["blocked"]:
1238
+ summary += f", {counts['blocked']} blocked by Instagram"
1239
+ if notes:
1240
+ summary += f" — {notes[0][:120]}"
1241
+ return (state, summary, counts, affected)
1242
+
1243
+
1244
+ def _auto_url_field(table, fkey):
1245
+ """The URL field an automation column is bound to: its own `automation.urlField`, else the
1246
+ first url-typed column on the table. Never guessed silently — the caller reports which."""
1247
+ for f in table.get("fields") or []:
1248
+ if f.get("key") == fkey:
1249
+ bound = ((f.get("automation") or {}).get("urlField") or "").strip()
1250
+ if bound:
1251
+ return bound
1252
+ for f in table.get("fields") or []:
1253
+ if f.get("type") == "url":
1254
+ return f.get("key")
1255
+ return ""
1256
+
1257
+
1258
+ RUNNERS = {"scrape_db": run_scrape_db, "field_instagram": run_field_instagram}
1259
+
1260
+
1261
+ def run_now(rt, tenant, auto_id, username="automation", log=print):
1262
+ """Execute one automation SYNCHRONOUSLY. The route wraps this in a thread; the tick calls it
1263
+ directly. Returns the run entry, or None when it was already running (the 409)."""
1264
+ defn = all_definitions(rt).get(str(auto_id))
1265
+ if defn is None:
1266
+ return None
1267
+ if not _claim(tenant, auto_id):
1268
+ return None
1269
+ try:
1270
+ _step(tenant, auto_id, "running")
1271
+ runner = RUNNERS.get(defn.get("kind"))
1272
+ if runner is None:
1273
+ return _commit_run(rt, auto_id, "error",
1274
+ f"unknown automation kind {defn.get('kind')!r}", {}, False)
1275
+ try:
1276
+ state, summary, counts, affected = runner(rt, defn, username=username, log=log)
1277
+ except Refused as e:
1278
+ return _commit_run(rt, auto_id, "error", f"refused: {e}", {}, False)
1279
+ except Exception as e: # noqa: BLE001
1280
+ log(f"[aios-auto] run {auto_id} failed: {type(e).__name__}: {e}")
1281
+ return _commit_run(rt, auto_id, "error",
1282
+ f"{type(e).__name__}: {str(e)[:200]}", {}, False)
1283
+ return _commit_run(rt, auto_id, state, summary, counts, state != "error", affected)
1284
+ finally:
1285
+ _release(tenant, auto_id)
1286
+
1287
+
1288
+ def run_async(rt, tenant, auto_id, username="automation", log=print):
1289
+ """Start a run on a background thread. False when one is already in flight (→ 409)."""
1290
+ if running(tenant, auto_id):
1291
+ return False
1292
+ th = threading.Thread(target=run_now, args=(rt, tenant, auto_id, username, log),
1293
+ daemon=True, name=f"automation-{tenant}-{auto_id}")
1294
+ th.start()
1295
+ return True
1296
+
1297
+
1298
+ # ---------------------------------------------------------------------------------------------
1299
+ # THE TICK + the in-process scheduler
1300
+ # ---------------------------------------------------------------------------------------------
1301
+
1302
+ def due_ids(rt, now=None):
1303
+ """Which of this tenant's automations a tick at `now` should start. Pure over the store."""
1304
+ return sorted(aid for aid, d in all_definitions(rt).items() if is_due(d, now))
1305
+
1306
+
1307
+ def tick(rt, tenant, now=None, log=print):
1308
+ """Fire every due automation for ONE tenant. Returns the ids started."""
1309
+ started = []
1310
+ for aid in due_ids(rt, now):
1311
+ if run_async(rt, tenant, aid, username="scheduler", log=log):
1312
+ started.append(aid)
1313
+ if started:
1314
+ log(f"[aios-auto] tick {tenant}: started {', '.join(started)}")
1315
+ return started
1316
+
1317
+
1318
+ def tick_all(now=None, log=print):
1319
+ """Every registered tenant. Fail-quiet per tenant: one tenant's broken store must not stop
1320
+ the others' schedules."""
1321
+ from harness import runtime as _rt
1322
+ out = {}
1323
+ for slug in _rt.known_tenants():
1324
+ try:
1325
+ out[slug] = tick(_rt.get_runtime(slug), slug, now=now, log=log)
1326
+ except Exception as e: # noqa: BLE001
1327
+ log(f"[aios-auto] tick {slug} skipped: {type(e).__name__}: {e}")
1328
+ return out
1329
+
1330
+
1331
+ #: How often the in-process scheduler wakes. A minute is the cron resolution; anything finer
1332
+ #: would be a busy loop against a vocabulary that cannot express it.
1333
+ TICK_SECONDS = int(os.environ.get("AIOS_AUTOMATION_TICK_SECONDS") or 60)
1334
+ _SCHEDULER = [None]
1335
+
1336
+
1337
+ def scheduler_loop(log=print):
1338
+ """The resync-daemon pattern (`api/main.py:313`): sleep FIRST, then work.
1339
+
1340
+ Sleeping first is deliberate and load-bearing for the gate battery — `verify_api` and
1341
+ `verify_seam` import this module through `main.py`, run against fake stores in seconds and
1342
+ exit. A loop that ticked on entry would fire inside them.
1343
+ """
1344
+ while True:
1345
+ time.sleep(TICK_SECONDS)
1346
+ try:
1347
+ tick_all(log=log)
1348
+ except Exception as e: # noqa: BLE001
1349
+ log(f"[aios-auto] scheduler tick failed: {type(e).__name__}: {e}")
1350
+
1351
+
1352
+ def start_scheduler(log=print):
1353
+ """Start the loop once per process. **OPT-IN: `AIOS_AUTOMATIONS=1`.**
1354
+
1355
+ ⚠ AMENDED 2026-08-03 (D), and the reason is a measurement rather than a preference. The wave
1356
+ brief specified this thread DEFAULT-ON for the API. Then `verify_api.py` was timed: **196
1357
+ seconds**, i.e. more than three 60 s tick intervals. A default-on scheduler therefore fires
1358
+ two or three times inside a gate that imports `main.py` — and while a tick over a fake store
1359
+ finds nothing due, `tick_all` reaches `runtime.get_runtime()`, which BUILDS tenants and
1360
+ mutates the LRU cache that `verify_api`'s own isolation assertions read. A background thread
1361
+ that can move a gate's subject is a flaky suite waiting to happen.
1362
+
1363
+ So it takes the convention `main.py:353` already established for exactly this hazard —
1364
+ `AIOS_PREWARM=1` — for exactly the reason stated there: env-gated rather than a startup
1365
+ event, so importing this module in a gate can never fire anything. The DEPLOY sets it; the
1366
+ external EventBridge tick (R5) does not depend on it either way, since that POSTs the
1367
+ endpoint rather than riding this thread.
1368
+ """
1369
+ if os.environ.get("AIOS_AUTOMATIONS") != "1":
1370
+ return False
1371
+ if _SCHEDULER[0] is not None:
1372
+ return False
1373
+ th = threading.Thread(target=scheduler_loop, kwargs={"log": log},
1374
+ daemon=True, name="automation-scheduler")
1375
+ _SCHEDULER[0] = th
1376
+ th.start()
1377
+ return True
api/deps.py CHANGED
@@ -50,9 +50,20 @@ class Session:
50
  return perms.is_admin(self.user)
51
 
52
  def require(self, module_key):
53
- """403 unless this session may open `module_key`. Returns nothing — it is a gate."""
 
 
 
 
 
 
54
  if not perms.may_open(self.user, module_key):
55
  raise err(403, "forbidden", f"your account does not have access to {module_key}")
 
 
 
 
 
56
 
57
 
58
  def _user_for(claims):
@@ -83,6 +94,7 @@ def _user_for(claims):
83
  uname = (claims.get("u") or "").strip().lower()
84
  if not uname:
85
  return None
 
86
  try:
87
  reg = users.registry() or {}
88
  except Exception:
@@ -90,10 +102,23 @@ def _user_for(claims):
90
  rec = reg.get(uname)
91
  if (rec and rec.get("active", True)
92
  and int(rec.get("epoch") or 0) == int(claims.get("e") or 0)):
93
- return users._public(uname, rec)
94
- if uname == "admin" and os.environ.get("APP_PASSWORD", ""):
 
 
 
 
 
 
 
 
 
 
 
 
95
  return {"username": "admin", "name": "Administrator", "role": "admin",
96
- "bus": "all", "modules": "all", "epoch": int(claims.get("e") or 0)}
 
97
  return None
98
 
99
 
 
50
  return perms.is_admin(self.user)
51
 
52
  def require(self, module_key):
53
+ """403 unless this session may open `module_key`. Returns nothing — it is a gate.
54
+
55
+ TWO walls, both fail-closed (wave 18, C1-TENANT): the ACCOUNT grant (`perms.may_open`)
56
+ and the TENANT catalogue — a registry module a tenant has not enabled is 403 even for
57
+ that tenant's admin, or a freshly provisioned Nurilab admin (role=admin ⇒ may_open
58
+ everything) would open Royal's Odoo-backed surfaces through any module route. One
59
+ chokepoint: `module_gate` and every inline `session.require` pass through here."""
60
  if not perms.may_open(self.user, module_key):
61
  raise err(403, "forbidden", f"your account does not have access to {module_key}")
62
+ tcfg = getattr(self.runtime.tenant, "config", None) or {}
63
+ tmods = tcfg.get("modules", "all")
64
+ if tmods != "all" and str(module_key) not in {str(k) for k in (tmods or [])}:
65
+ raise err(403, "forbidden",
66
+ f"this workspace does not include {module_key}")
67
 
68
 
69
  def _user_for(claims):
 
94
  uname = (claims.get("u") or "").strip().lower()
95
  if not uname:
96
  return None
97
+ claim_tenant = str(claims.get("t") or "").strip().lower()
98
  try:
99
  reg = users.registry() or {}
100
  except Exception:
 
102
  rec = reg.get(uname)
103
  if (rec and rec.get("active", True)
104
  and int(rec.get("epoch") or 0) == int(claims.get("e") or 0)):
105
+ pub = users._public(uname, rec)
106
+ # ⛔ Wave 18 (C1-TENANT, R1): the cookie's tenant must be THE ACCOUNT'S tenant. Before
107
+ # this line, any valid credential could mint a session for any registered slug
108
+ # (login validated the slug's existence, never the membership) — with
109
+ # AIOS_ENABLE_QA_TENANT=1 that was a working cross-tenant session. A signature proves
110
+ # the cookie is ours; THIS proves the account belongs where the cookie says it does.
111
+ if claim_tenant != pub.get("tenant", "royal-imports"):
112
+ return None
113
+ return pub
114
+ if (uname == "admin" and os.environ.get("APP_PASSWORD", "")
115
+ and claim_tenant == "royal-imports"):
116
+ # The emergency master is TENANT #0's break-glass, not the platform's: a synthetic
117
+ # all-access identity minted into another tenant's session would be the exact
118
+ # cross-tenant widening the rule above closes.
119
  return {"username": "admin", "name": "Administrator", "role": "admin",
120
+ "bus": "all", "modules": "all", "tenant": "royal-imports",
121
+ "epoch": int(claims.get("e") or 0)}
122
  return None
123
 
124
 
api/main.py CHANGED
@@ -55,13 +55,17 @@ from starlette.exceptions import HTTPException as StarletteHTTPException # noqa
55
 
56
  import aios_session # noqa: E402
57
  import routes_admin # noqa: E402
 
58
  import routes_auth # noqa: E402
 
59
  import routes_customers # noqa: E402
60
  import routes_grid # noqa: E402
 
61
  import routes_nav # noqa: E402
62
  import routes_pages # noqa: E402
63
  import routes_products # noqa: E402 (wave 15 C-TOPIC — gated, not yet in the nav)
64
  import routes_records # noqa: E402
 
65
  from core import grid_events # noqa: E402
66
  from deps import Session, module_gate # noqa: E402
67
 
@@ -163,6 +167,8 @@ app.include_router(routes_auth.router)
163
  app.include_router(routes_nav.router)
164
  app.include_router(routes_customers.router)
165
  app.include_router(routes_products.router)
 
 
166
  app.include_router(routes_grid.router)
167
  app.include_router(routes_records.router)
168
  # EXIT wave 2: the Y1 page-data envelope (one route for every ported dashboard) and Y4's user
@@ -170,6 +176,8 @@ app.include_router(routes_records.router)
170
  # a new page is a builder module plus one registry line, and nothing here changes.
171
  app.include_router(routes_pages.router)
172
  app.include_router(routes_admin.router)
 
 
173
 
174
 
175
  # --- DEPRECATED ALIASES (removed when S2's shell flips; kept so the current bundle keeps working)
 
55
 
56
  import aios_session # noqa: E402
57
  import routes_admin # noqa: E402
58
+ import routes_assets # noqa: E402 (wave 18 C2-ASSET — catalog product imagery)
59
  import routes_auth # noqa: E402
60
+ import routes_automation # noqa: E402 (wave 18 C4-AUTO — SESSION D's router, mounted by A)
61
  import routes_customers # noqa: E402
62
  import routes_grid # noqa: E402
63
+ import routes_keychain # noqa: E402 (wave 18 C7 — keychain + connectors admin surfaces)
64
  import routes_nav # noqa: E402
65
  import routes_pages # noqa: E402
66
  import routes_products # noqa: E402 (wave 15 C-TOPIC — gated, not yet in the nav)
67
  import routes_records # noqa: E402
68
+ import routes_tables # noqa: E402 (wave 18 C3-UT — user-created databases over the wire)
69
  from core import grid_events # noqa: E402
70
  from deps import Session, module_gate # noqa: E402
71
 
 
167
  app.include_router(routes_nav.router)
168
  app.include_router(routes_customers.router)
169
  app.include_router(routes_products.router)
170
+ app.include_router(routes_assets.router)
171
+ app.include_router(routes_tables.router)
172
  app.include_router(routes_grid.router)
173
  app.include_router(routes_records.router)
174
  # EXIT wave 2: the Y1 page-data envelope (one route for every ported dashboard) and Y4's user
 
176
  # a new page is a builder module plus one registry line, and nothing here changes.
177
  app.include_router(routes_pages.router)
178
  app.include_router(routes_admin.router)
179
+ app.include_router(routes_automation.router)
180
+ app.include_router(routes_keychain.router)
181
 
182
 
183
  # --- DEPRECATED ALIASES (removed when S2's shell flips; kept so the current bundle keeps working)
api/routes_admin.py CHANGED
@@ -90,6 +90,7 @@ def _view(uname, rec):
90
  "modules": rec.get("modules", "all"),
91
  "agent": rec.get("agent") or None,
92
  "email": rec.get("email") or None,
 
93
  "active": bool(rec.get("active", True)),
94
  # The session-revocation handle. An admin needs it: it is the only visible evidence
95
  # that "sign them out everywhere" actually happened.
@@ -202,11 +203,22 @@ def _clean_modules(v):
202
  return sorted(set(out))
203
 
204
 
 
 
 
 
 
 
205
  # ── the routes ───────────────────────────────────────────────────────────────────────────────────
206
  @router.get("/admin/users")
207
  def list_users(session: Session = Depends(admin_gate)):
 
 
 
208
  reg = _registry()
209
- return {"users": [_view(u, reg[u]) for u in sorted(reg)]}
 
 
210
 
211
 
212
  @router.post("/admin/users", status_code=201)
@@ -234,8 +246,10 @@ def create_user(body: dict = Body(default=None), session: Session = Depends(admi
234
 
235
  if uname in _registry():
236
  raise err(409, "user_exists", f"{uname} already exists — PATCH it to change it")
 
 
237
  users.create_user(uname, pw, name, role=role, bus=bus, modules=modules,
238
- agent=agent, email=email)
239
  reg = _registry()
240
  if uname not in reg:
241
  # The store accepted the write and does not have it. Reporting 201 here would be the
@@ -258,7 +272,9 @@ def update_user(username: str, body: dict = Body(default=None),
258
  body = body if isinstance(body, dict) else {}
259
  reg = _registry()
260
  uname = str(username or "").strip().lower()
261
- if uname not in reg:
 
 
262
  raise err(404, "no_such_user", f"no account named {uname!r}")
263
  if not body:
264
  raise err(400, "empty_patch", "no fields to update")
@@ -314,7 +330,9 @@ def set_password(username: str, body: dict = Body(default=None),
314
  """
315
  reg = _registry()
316
  uname = str(username or "").strip().lower()
317
- if uname not in reg:
 
 
318
  raise err(404, "no_such_user", f"no account named {uname!r}")
319
  pw = _clean_password((body or {}).get("password"))
320
  before = int(reg[uname].get("epoch") or 0)
@@ -643,7 +661,9 @@ def get_perms(username: str, session: Session = Depends(admin_gate)):
643
  """This account's permission block plus the field schema the editor needs to render it."""
644
  reg = _registry()
645
  uname = str(username or "").strip().lower()
646
- if uname not in reg:
 
 
647
  raise err(404, "no_such_user", f"no account named {uname!r}")
648
  import core.perm_scope as perm_scope
649
 
@@ -691,7 +711,9 @@ def put_perms(username: str, body: dict = Body(default=None),
691
  body = body if isinstance(body, dict) else {}
692
  reg = _registry()
693
  uname = str(username or "").strip().lower()
694
- if uname not in reg:
 
 
695
  raise err(404, "no_such_user", f"no account named {uname!r}")
696
  if "perms" not in body:
697
  raise err(400, "empty_patch", "no perms to save")
 
90
  "modules": rec.get("modules", "all"),
91
  "agent": rec.get("agent") or None,
92
  "email": rec.get("email") or None,
93
+ "tenant": _tenant_of(rec),
94
  "active": bool(rec.get("active", True)),
95
  # The session-revocation handle. An admin needs it: it is the only visible evidence
96
  # that "sign them out everywhere" actually happened.
 
203
  return sorted(set(out))
204
 
205
 
206
+ def _tenant_of(rec):
207
+ """The account's company, with the pre-wave default. ONE spelling of the default, used by
208
+ every admin route — two spellings is how a tenant wall grows a gap."""
209
+ return str((rec or {}).get("tenant") or "royal-imports").strip().lower()
210
+
211
+
212
  # ── the routes ───────────────────────────────────────────────────────────────────────────────────
213
  @router.get("/admin/users")
214
  def list_users(session: Session = Depends(admin_gate)):
215
+ """Wave 18 (C1-TENANT): scoped to the CALLER'S tenant. The user registry is a global
216
+ control-plane bucket, so without this filter a Nurilab admin would read Royal's whole
217
+ roster — names, emails, agents — from one URL."""
218
  reg = _registry()
219
+ mine = _tenant_of(session.user)
220
+ return {"users": [_view(u, reg[u]) for u in sorted(reg)
221
+ if _tenant_of(reg[u]) == mine]}
222
 
223
 
224
  @router.post("/admin/users", status_code=201)
 
246
 
247
  if uname in _registry():
248
  raise err(409, "user_exists", f"{uname} already exists — PATCH it to change it")
249
+ # Wave 18 (C1-TENANT): an admin creates accounts in their OWN company, never another's —
250
+ # the tenant is stamped from the session, not taken from the body.
251
  users.create_user(uname, pw, name, role=role, bus=bus, modules=modules,
252
+ agent=agent, email=email, tenant=_tenant_of(session.user))
253
  reg = _registry()
254
  if uname not in reg:
255
  # The store accepted the write and does not have it. Reporting 201 here would be the
 
272
  body = body if isinstance(body, dict) else {}
273
  reg = _registry()
274
  uname = str(username or "").strip().lower()
275
+ if uname not in reg or _tenant_of(reg[uname]) != _tenant_of(session.user):
276
+ # Wave 18 (C1-TENANT): a cross-tenant username answers exactly like a missing one —
277
+ # 404, never 403, because "that account exists in another company" is itself a leak.
278
  raise err(404, "no_such_user", f"no account named {uname!r}")
279
  if not body:
280
  raise err(400, "empty_patch", "no fields to update")
 
330
  """
331
  reg = _registry()
332
  uname = str(username or "").strip().lower()
333
+ if uname not in reg or _tenant_of(reg[uname]) != _tenant_of(session.user):
334
+ # Wave 18 (C1-TENANT): a cross-tenant username answers exactly like a missing one —
335
+ # 404, never 403, because "that account exists in another company" is itself a leak.
336
  raise err(404, "no_such_user", f"no account named {uname!r}")
337
  pw = _clean_password((body or {}).get("password"))
338
  before = int(reg[uname].get("epoch") or 0)
 
661
  """This account's permission block plus the field schema the editor needs to render it."""
662
  reg = _registry()
663
  uname = str(username or "").strip().lower()
664
+ if uname not in reg or _tenant_of(reg[uname]) != _tenant_of(session.user):
665
+ # Wave 18 (C1-TENANT): a cross-tenant username answers exactly like a missing one —
666
+ # 404, never 403, because "that account exists in another company" is itself a leak.
667
  raise err(404, "no_such_user", f"no account named {uname!r}")
668
  import core.perm_scope as perm_scope
669
 
 
711
  body = body if isinstance(body, dict) else {}
712
  reg = _registry()
713
  uname = str(username or "").strip().lower()
714
+ if uname not in reg or _tenant_of(reg[uname]) != _tenant_of(session.user):
715
+ # Wave 18 (C1-TENANT): a cross-tenant username answers exactly like a missing one —
716
+ # 404, never 403, because "that account exists in another company" is itself a leak.
717
  raise err(404, "no_such_user", f"no account named {uname!r}")
718
  if "perms" not in body:
719
  raise err(400, "empty_patch", "no perms to save")
api/routes_assets.py ADDED
@@ -0,0 +1,165 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """routes_assets.py — product image service (wave 18, contract C2-ASSET).
2
+
3
+ Serves the catalog's product imagery. Two sources, checked in order:
4
+
5
+ 1. `AIOS_ASSET_DIR` — a local directory of master images whose FILENAME STEM is the product
6
+ `code` (Odoo default_code, case-insensitive). Dev mode: the owner's master folder on this
7
+ machine. Derivatives are generated on demand into a disk cache.
8
+ 2. `AIOS_ASSET_REPO` — a private HF dataset (default `royal-imports/product-assets`) laid out
9
+ by `royalimports_os/ingest_product_assets.py`:
10
+ assets/products/orig/<CODE><ext> originals, archived verbatim (R9)
11
+ assets/products/web/<CODE>.png ~800px alpha-preserving derivative
12
+ manifest.json {"codes": {"<CODE>": {"ext": ".png"}}, ...}
13
+ Files download once per container into the HF cache; responses carry immutable cache
14
+ headers so the browser re-fetches nothing.
15
+
16
+ QUALITY (R9): `q=web` = the low-res derivative the app displays; `q=print` and `q=orig` BOTH
17
+ serve the ORIGINAL bytes — v1's "high quality" IS the master file (dated amendment in the wave
18
+ doc). The gate is the PRODUCT grant: images decorate the product surface, so whoever may open
19
+ `product_data` may fetch them; nobody else may.
20
+
21
+ Codes are validated against a closed charset before they touch a filesystem or a repo path —
22
+ a path separator can never reach the lookup, so traversal is refused at the door (400), and an
23
+ unknown code is 404 `asset_not_found`, never an empty 200.
24
+ """
25
+ import os
26
+ import re
27
+ import tempfile
28
+ import time
29
+ from pathlib import Path
30
+
31
+ from fastapi import Depends
32
+ from fastapi import APIRouter
33
+ from fastapi.responses import FileResponse
34
+
35
+ from deps import Session, err, module_gate
36
+
37
+ router = APIRouter(prefix="/api/v1")
38
+
39
+ #: Same grant as the product surface these images decorate (routes_products.MODULE, restated
40
+ #: here to avoid importing that module for one string).
41
+ MODULE = "product_data"
42
+
43
+ _WEB_EDGE = 800 # max edge (px) of the in-app derivative
44
+ _QUALITIES = ("web", "print", "orig")
45
+ _EXT_MEDIA = {".png": "image/png", ".jpg": "image/jpeg", ".jpeg": "image/jpeg"}
46
+ #: closed charset — no slashes, no backslashes, no colons, so a code can never be a path.
47
+ _CODE_RE = re.compile(r"^[A-Za-z0-9][A-Za-z0-9 ._+&()\-]{0,63}$")
48
+ _IMMUTABLE = "public, max-age=31536000, immutable"
49
+
50
+ #: per-process index of the local master dir: rebuilt when the dir path or its mtime changes.
51
+ _DIR_INDEX = {"path": None, "mtime": None, "map": {}}
52
+
53
+
54
+ def _norm(code):
55
+ return str(code or "").strip().upper()
56
+
57
+
58
+ def _local_dir():
59
+ d = os.environ.get("AIOS_ASSET_DIR", "").strip()
60
+ if not d:
61
+ return None
62
+ p = Path(d)
63
+ return p if p.is_dir() else None
64
+
65
+
66
+ def _cache_root():
67
+ root = Path(os.environ.get("AIOS_ASSET_CACHE", "").strip()
68
+ or Path(tempfile.gettempdir()) / "aios_asset_cache")
69
+ root.mkdir(parents=True, exist_ok=True)
70
+ return root
71
+
72
+
73
+ def _dir_index(d):
74
+ """{CODE: Path} over the master dir. Two files sharing an upper-cased stem keep the
75
+ lexicographically first — the same rule the ingest script applies, so dir mode and repo
76
+ mode resolve identically."""
77
+ try:
78
+ mtime = d.stat().st_mtime_ns
79
+ except OSError:
80
+ return {}
81
+ if _DIR_INDEX["path"] == str(d) and _DIR_INDEX["mtime"] == mtime:
82
+ return _DIR_INDEX["map"]
83
+ idx = {}
84
+ for p in sorted(d.iterdir()):
85
+ if not p.is_file() or p.name.startswith("."):
86
+ continue
87
+ if p.suffix.lower() not in _EXT_MEDIA:
88
+ continue
89
+ idx.setdefault(p.stem.upper(), p)
90
+ _DIR_INDEX.update(path=str(d), mtime=mtime, map=idx)
91
+ return idx
92
+
93
+
94
+ def _web_derivative(src, code):
95
+ """The ~800px alpha-preserving PNG for one master file, built lazily into the disk cache.
96
+ Rebuilt when the master is newer than the cached copy."""
97
+ out = _cache_root() / "web"
98
+ out.mkdir(parents=True, exist_ok=True)
99
+ dst = out / f"{code}.png"
100
+ try:
101
+ if dst.is_file() and dst.stat().st_mtime >= src.stat().st_mtime:
102
+ return dst
103
+ except OSError:
104
+ pass
105
+ try:
106
+ from PIL import Image
107
+ except ImportError:
108
+ raise err(503, "pillow_missing",
109
+ "image derivatives need Pillow — `pip install pillow` on this host")
110
+ with Image.open(src) as im:
111
+ im.load()
112
+ if im.mode not in ("RGBA", "RGB", "LA", "L"):
113
+ im = im.convert("RGBA")
114
+ im.thumbnail((_WEB_EDGE, _WEB_EDGE))
115
+ im.save(dst, format="PNG", optimize=True)
116
+ return dst
117
+
118
+
119
+ def _assets():
120
+ """The asset SOURCE lives in `royalimports_os/core/assets.py`, not here.
121
+
122
+ ⛔ NOT TIDINESS — a CONTRACT. `ops/verify_portability.py` B1/B2 assert the HuggingFace SDK
123
+ appears only in its declared homes and NEVER under `aios-web/api/`, because this process is
124
+ meant to lift to Hetzner/EC2 without a hub client. The first version of this file imported
125
+ `huggingface_hub` directly and the gate caught it."""
126
+ import core.assets as assets
127
+ return assets
128
+
129
+
130
+ @router.get("/assets/products")
131
+ def asset_manifest(session: Session = Depends(module_gate(MODULE))):
132
+ """The codes that HAVE an image — so the catalog designer can mark gaps instead of
133
+ rendering broken tiles. `source` names which backend answered."""
134
+ d = _local_dir()
135
+ if d is not None:
136
+ return {"codes": sorted(_dir_index(d).keys()), "source": "dir"}
137
+ codes = _assets().manifest()
138
+ if codes:
139
+ return {"codes": sorted(codes.keys()), "source": "repo"}
140
+ return {"codes": [], "source": "none"}
141
+
142
+
143
+ @router.get("/assets/products/{code}")
144
+ def product_asset(code: str, q: str = "web",
145
+ session: Session = Depends(module_gate(MODULE))):
146
+ if q not in _QUALITIES:
147
+ raise err(400, "bad_quality", f"q must be one of {', '.join(_QUALITIES)}")
148
+ norm = _norm(code)
149
+ if not _CODE_RE.match(norm):
150
+ raise err(400, "bad_code", "that is not a product code")
151
+
152
+ path = None
153
+ d = _local_dir()
154
+ if d is not None:
155
+ src = _dir_index(d).get(norm)
156
+ if src is not None:
157
+ path = _web_derivative(src, norm) if q == "web" else src
158
+ if path is None:
159
+ path = _assets().fetch(norm, q)
160
+ if path is None or not Path(path).is_file():
161
+ raise err(404, "asset_not_found", f"no image for {norm}")
162
+
163
+ media = _EXT_MEDIA.get(Path(path).suffix.lower(), "application/octet-stream")
164
+ return FileResponse(str(path), media_type=media,
165
+ headers={"Cache-Control": _IMMUTABLE})
api/routes_auth.py CHANGED
@@ -70,6 +70,9 @@ def _public_user(user):
70
  the epoch is a server-side revocation handle and a client has no use for it."""
71
  return {"username": user.get("username", ""), "name": user.get("name", ""),
72
  "role": user.get("role", "user"),
 
 
 
73
  "bus": perms.allowed_bu_labels(user),
74
  "team_id": perms.scope_team_id(user),
75
  "agent": perms.scope_agent(user),
@@ -128,21 +131,22 @@ def clear_avatar(session: Session = Depends(require_session)):
128
 
129
  @router.post("/login")
130
  def login(request: Request, response: Response, body: dict = Body(default=None)):
 
 
 
 
 
 
 
 
 
131
  body = body or {}
132
  uname = str(body.get("username") or "").strip().lower()
133
  pw = str(body.get("password") or "")
134
- tenant = str(body.get("tenant") or "royal-imports").strip().lower()
135
-
136
- # The tenant is validated BEFORE the credential check so an unknown slug can never mint a
137
- # cookie — but it answers with the SAME 401 as a bad password, because "that tenant does not
138
- # exist" is a fact about our customer list.
139
- from harness import runtime
140
- try:
141
- runtime.get_runtime(tenant)
142
- except KeyError:
143
- raise err(401, "invalid_credentials", "that username and password do not match")
144
 
145
- throttle_key = (tenant, uname)
 
 
146
  wait = _throttled(throttle_key)
147
  if wait:
148
  raise err(429, "too_many_attempts",
@@ -161,6 +165,14 @@ def login(request: Request, response: Response, body: dict = Body(default=None))
161
  _note_failure(throttle_key)
162
  raise err(401, "invalid_credentials", "that username and password do not match")
163
 
 
 
 
 
 
 
 
 
164
  _clear_failures(throttle_key)
165
  value, _claims = aios_session.mint(tenant, user["username"], int(user.get("epoch") or 0))
166
  aios_session.set_cookie(response, request, value)
 
70
  the epoch is a server-side revocation handle and a client has no use for it."""
71
  return {"username": user.get("username", ""), "name": user.get("name", ""),
72
  "role": user.get("role", "user"),
73
+ # Wave 18 (C1-TENANT, R1): the company this session is bound to — the client shows
74
+ # it and tests assert the binding; the SERVER-side truth stays in the signed cookie.
75
+ "tenant": str(user.get("tenant") or "royal-imports").strip().lower(),
76
  "bus": perms.allowed_bu_labels(user),
77
  "team_id": perms.scope_team_id(user),
78
  "agent": perms.scope_agent(user),
 
131
 
132
  @router.post("/login")
133
  def login(request: Request, response: Response, body: dict = Body(default=None)):
134
+ """Wave 18 (C1-TENANT, R1): ONE login box — the ACCOUNT decides the tenant.
135
+
136
+ The pre-wave flow validated a caller-posted tenant slug and minted the session for it, so
137
+ the same credential could sign in to any registered tenant. Now the credential resolves
138
+ FIRST (username or email — `users.verify` takes both) and the session binds to the tenant
139
+ ON THE RECORD; the posted `tenant` field is accepted for wire-compat and ignored. An
140
+ account whose tenant no longer resolves (deleted / suspended record) gets the same 401 as
141
+ a bad password — "which tenants exist" is not the login form's question to answer.
142
+ """
143
  body = body or {}
144
  uname = str(body.get("username") or "").strip().lower()
145
  pw = str(body.get("password") or "")
 
 
 
 
 
 
 
 
 
 
146
 
147
+ # Throttle on the IDENTIFIER alone: the tenant is not known until the credential resolves,
148
+ # and a per-(tenant, uname) key would let an attacker reset their budget by rotating slugs.
149
+ throttle_key = ("*", uname)
150
  wait = _throttled(throttle_key)
151
  if wait:
152
  raise err(429, "too_many_attempts",
 
165
  _note_failure(throttle_key)
166
  raise err(401, "invalid_credentials", "that username and password do not match")
167
 
168
+ tenant = str(user.get("tenant") or "royal-imports").strip().lower()
169
+ from harness import runtime
170
+ try:
171
+ runtime.get_runtime(tenant)
172
+ except KeyError:
173
+ _note_failure(throttle_key)
174
+ raise err(401, "invalid_credentials", "that username and password do not match")
175
+
176
  _clear_failures(throttle_key)
177
  value, _claims = aios_session.mint(tenant, user["username"], int(user.get("epoch") or 0))
178
  aios_session.set_cookie(response, request, value)
api/routes_automation.py ADDED
@@ -0,0 +1,217 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """routes_automation.py — wave-18 item 5 (contract C4-AUTO): the automation surface's API.
2
+
3
+ The router is thin on purpose: everything that can be wrong about an automation — a malformed
4
+ cron, a URL the SSRF rail refuses, a key field that is not one of the mapped columns — is decided
5
+ in `automation_engine`, which is a pure-ish module a gate can drive without a server. This file
6
+ does auth, shape and status codes.
7
+
8
+ ⛔ THE TICK ENDPOINT IS THE ONE UNAUTHENTICATED ROUTE, AND IT IS FAIL-CLOSED TWICE OVER. It takes
9
+ no session (an external cron has no cookie), so it is gated on a shared secret in
10
+ `X-AIOS-TICK-TOKEN`; and when `AIOS_AUTOMATION_TICK_TOKEN` is UNSET the route refuses everything
11
+ rather than admitting everyone. A "no token configured means no check" default is how an internal
12
+ trigger becomes a public one — the same class of mistake as an empty-200 permission answer.
13
+ """
14
+ import os
15
+ import time
16
+
17
+ from fastapi import APIRouter, Body, Depends, Header, Request
18
+
19
+ import automation_engine as engine
20
+ from deps import Session, err, module_gate, require_session
21
+
22
+ router = APIRouter(prefix="/api/v1")
23
+
24
+ #: The registry key this surface carries (C-AUTONAV — A adds the row; the gate is live now, so
25
+ #: the day the row lands the wall is already the one that was tested).
26
+ MODULE = "automation"
27
+
28
+ _GATE = module_gate(MODULE)
29
+
30
+
31
+ def _wire(defn, tenant):
32
+ """One automation, as the client reads it. `running` is PROCESS state, never store state —
33
+ see the engine header on why a persisted 'running' is a permanent lock."""
34
+ live = engine.running(tenant, defn.get("id"))
35
+ sched = defn.get("schedule") or {}
36
+ nxt = engine.next_fire(sched.get("cron")) if sched.get("enabled") else None
37
+ status = dict(defn.get("status") or {})
38
+ if live:
39
+ status = {**status, "state": "running", "startedAt": live.get("startedAt"),
40
+ "step": live.get("step")}
41
+ return {
42
+ "id": defn.get("id"), "name": defn.get("name"), "kind": defn.get("kind"),
43
+ "config": defn.get("config") or {}, "schedule": sched, "status": status,
44
+ "runs": list(defn.get("runs") or [])[:engine.MAX_RUNS],
45
+ "created": defn.get("created"), "createdBy": defn.get("createdBy"),
46
+ "nextRunAt": nxt.strftime("%Y-%m-%d %H:%M") if nxt else "",
47
+ "running": bool(live),
48
+ }
49
+
50
+
51
+ @router.get("/automations")
52
+ def list_automations(session: Session = Depends(_GATE)):
53
+ """`{automations: [...], kinds: [...], cronPresets: [...]}` — the rail's whole payload.
54
+
55
+ The vocabularies ride WITH the list rather than sitting in a client constant: the cron
56
+ presets and the kind list are the server's, and a client copy of either is a thing that goes
57
+ stale silently (the editor would offer a preset the parser rejects)."""
58
+ defs = engine.all_definitions(session.runtime)
59
+ items = [_wire(d, session.tenant) for _, d in
60
+ sorted(defs.items(), key=lambda kv: (kv[1].get("name") or "").lower())]
61
+ return {"automations": items,
62
+ "kinds": [{"key": "scrape_db", "label": "Web page to database"},
63
+ {"key": "field_instagram", "label": "Instagram profile column"}],
64
+ "cronPresets": engine.CRON_PRESETS,
65
+ "storeAvailable": bool(session.runtime.available())}
66
+
67
+
68
+ @router.get("/automations/tables")
69
+ def automation_tables(session: Session = Depends(_GATE)):
70
+ """The blank databases an automation can target, with their fields.
71
+
72
+ ⚠ A READ-ONLY MIRROR of C3-UT's `GET /api/v1/tables`, under this router's own prefix so the
73
+ two can never collide. It exists because the automation editor needs the table+field list to
74
+ build a config at all, and D must not block on A's route landing. When C3-UT is live this
75
+ keeps working (same bucket) — it is a duplicate reader, never a second writer.
76
+ """
77
+ out = []
78
+ for key, t in sorted(engine.ut_all(session.runtime).items(),
79
+ key=lambda kv: (kv[1].get("label") or "").lower()):
80
+ if not (session.admin or t.get("createdBy") in (session.uname, "automation",
81
+ "scheduler")):
82
+ continue # user_tables.may_open's rule, mirrored fail-closed
83
+ out.append({"key": key, "label": t.get("label") or key,
84
+ "source": t.get("source") or "Blank",
85
+ "rowCount": len(t.get("rows") or {}),
86
+ "fields": [{"key": f.get("key"), "label": f.get("label"),
87
+ "type": f.get("type") or "text",
88
+ "automation": f.get("automation") or None}
89
+ for f in (t.get("fields") or [])]})
90
+ return {"tables": out}
91
+
92
+
93
+ @router.get("/automations/{auto_id}")
94
+ def get_automation(auto_id: str, session: Session = Depends(_GATE)):
95
+ defn = engine.all_definitions(session.runtime).get(str(auto_id))
96
+ if defn is None:
97
+ raise err(404, "unknown_automation", "no automation with that id")
98
+ return {"automation": _wire(defn, session.tenant)}
99
+
100
+
101
+ @router.post("/automations")
102
+ def create_automation(body: dict = Body(default=None), session: Session = Depends(_GATE)):
103
+ if not session.runtime.available():
104
+ raise err(503, "store_unavailable", "the tenant store is unavailable — nothing was saved")
105
+ defn, error = engine.create(session.runtime, body or {}, username=session.uname)
106
+ if error:
107
+ raise err(400, "invalid_automation", error)
108
+ return {"automation": _wire(defn, session.tenant)}
109
+
110
+
111
+ @router.patch("/automations/{auto_id}")
112
+ def patch_automation(auto_id: str, body: dict = Body(default=None),
113
+ session: Session = Depends(_GATE)):
114
+ if not session.runtime.available():
115
+ raise err(503, "store_unavailable", "the tenant store is unavailable — nothing was saved")
116
+ defn, error = engine.patch(session.runtime, auto_id, body or {}, username=session.uname)
117
+ if error:
118
+ raise err(400 if error != "no such automation" else 404,
119
+ "invalid_automation" if error != "no such automation" else "unknown_automation",
120
+ error)
121
+ return {"automation": _wire(defn, session.tenant)}
122
+
123
+
124
+ @router.delete("/automations/{auto_id}")
125
+ def delete_automation(auto_id: str, session: Session = Depends(_GATE)):
126
+ """Delete the DEFINITION. ⚠ The database it filled is NOT touched — an automation is the
127
+ thing that writes rows, not the thing that owns them, and deleting a job must never be a way
128
+ to lose data (the same rule the orphan count encodes)."""
129
+ if engine.running(session.tenant, auto_id):
130
+ raise err(409, "automation_running", "it is running — wait for it to finish")
131
+ engine.remove(session.runtime, auto_id)
132
+ return {"deleted": str(auto_id)}
133
+
134
+
135
+ @router.post("/automations/preview")
136
+ def preview_source(body: dict = Body(default=None), session: Session = Depends(_GATE)):
137
+ """What does this URL actually offer? The field-map step — never writes anything."""
138
+ body = body or {}
139
+ url = str(body.get("url") or "").strip()
140
+ if not url:
141
+ raise err(400, "no_url", "give a page URL to read")
142
+ try:
143
+ return engine.preview(url, str(body.get("extract") or "table"),
144
+ int(body.get("tableIndex") or 0))
145
+ except engine.Refused as e:
146
+ raise err(400, "refused_url", str(e))
147
+ except Exception as e: # noqa: BLE001
148
+ raise err(502, "fetch_failed", f"could not read that page — {type(e).__name__}: "
149
+ f"{str(e)[:160]}")
150
+
151
+
152
+ @router.post("/automations/{auto_id}/run")
153
+ def run_automation(auto_id: str, session: Session = Depends(_GATE)):
154
+ """Start a run on a background thread. 409 when one is already in flight."""
155
+ defn = engine.all_definitions(session.runtime).get(str(auto_id))
156
+ if defn is None:
157
+ raise err(404, "unknown_automation", "no automation with that id")
158
+ if not engine.run_async(session.runtime, session.tenant, auto_id, username=session.uname):
159
+ raise err(409, "automation_running", "that automation is already running")
160
+ return {"started": str(auto_id), "startedAt": time.strftime("%Y-%m-%dT%H:%M:%S")}
161
+
162
+
163
+ @router.post("/automations/tick")
164
+ def tick(request: Request, x_aios_tick_token: str = Header(default="")):
165
+ """Fire every due schedule, for every tenant. The durable-cron entry point (R5).
166
+
167
+ Unauthenticated BY DESIGN and gated on a shared secret instead — an EventBridge rule has no
168
+ cookie. Refuses when the secret is not configured (see the module header)."""
169
+ want = os.environ.get("AIOS_AUTOMATION_TICK_TOKEN") or ""
170
+ if not want:
171
+ raise err(403, "tick_disabled",
172
+ "AIOS_AUTOMATION_TICK_TOKEN is not configured — the tick endpoint is closed")
173
+ got = x_aios_tick_token or request.headers.get("X-AIOS-TICK-TOKEN") or ""
174
+ if got != want:
175
+ raise err(403, "bad_tick_token", "that token is not valid for this deployment")
176
+ started = engine.tick_all()
177
+ return {"started": started, "at": time.strftime("%Y-%m-%dT%H:%M:%S")}
178
+
179
+
180
+ @router.get("/automations/{auto_id}/rows")
181
+ def run_rows(auto_id: str, session: Session = Depends(_GATE)):
182
+ """The rows the LAST run touched — the drill-down behind a run's counts.
183
+
184
+ Every count in this product drills to the exact rows behind it ([[no-unverifiable-aggregates]]);
185
+ a run history that said "412 updated" and could not show which would be the thing that rule
186
+ exists to forbid.
187
+ """
188
+ defn = engine.all_definitions(session.runtime).get(str(auto_id))
189
+ if defn is None:
190
+ raise err(404, "unknown_automation", "no automation with that id")
191
+ last = (defn.get("runs") or [{}])[0]
192
+ table_key = (defn.get("config") or {}).get("targetTable") or ""
193
+ t = engine.ut_get(session.runtime, table_key) or {}
194
+ rows = t.get("rows") or {}
195
+ ids = [str(i) for i in (last.get("affected") or [])]
196
+ return {"table": table_key, "label": t.get("label") or table_key,
197
+ "fields": [{"key": f.get("key"), "label": f.get("label")}
198
+ for f in (t.get("fields") or [])],
199
+ "rows": [{"id": i, **(rows.get(i) or {})} for i in ids if i in rows],
200
+ "truncated": len(ids) >= 200}
201
+
202
+
203
+ # --- the in-process scheduler ------------------------------------------------------------
204
+ # ⚠ OPT-IN (`AIOS_AUTOMATIONS=1`), and that is an AMENDMENT to the wave brief's "default-on",
205
+ # made on a measurement: `verify_api.py` runs for 196 s, i.e. longer than three tick intervals,
206
+ # and `tick_all` reaches `runtime.get_runtime()` — which builds tenants and moves the LRU cache
207
+ # that verify_api's own isolation checks read. Default-on would put a background thread inside
208
+ # the subject of another session's gate. `AIOS_PREWARM=1` at `main.py:353` is the same decision
209
+ # for the same reason, so this follows it rather than inventing a second convention.
210
+ #
211
+ # The loop also SLEEPS FIRST (`engine.scheduler_loop`) — defence in depth, proven in
212
+ # verify_automation.py section T rather than assumed.
213
+ #
214
+ # ⛔ THE DEPLOY MUST SET `AIOS_AUTOMATIONS=1` (with `AIOS_AUTOMATION_TICK_TOKEN`) or schedules
215
+ # only ever fire from the external cron POSTing /automations/tick. Both paths work; neither is
216
+ # implicit. Booked in the session-D mailbox.
217
+ engine.start_scheduler()
api/routes_grid.py CHANGED
@@ -47,20 +47,32 @@ def _ctx(session: Session, fields, pids, **kw):
47
  # contract the hidden-field closure runs over, and the TABLE OPS the write lands in.
48
  # Getting any one of them from the other topic is the "validated against the wrong field
49
  # contract" near-miss the wave-15 routes_products header warned about.
50
- if kw.get("scope_key") == "product":
 
51
  import modules.product_data as pd
52
  from routes_products import MODULE as _PMOD, pd_fields
53
 
54
  module, canonical = _PMOD, pd_fields(consolidated=True)
55
  kw.setdefault("table", pd.TABLE_OPS)
 
 
 
 
 
 
 
 
 
 
56
  else:
57
  module, canonical = MODULE, aios_grid.FIELDS
 
58
  return grid_events.EventCtx(
59
  uname=session.uname, allowed_pids=pids, fields=fields, admin=session.admin,
60
  # C-PERM: the write wall's field half, on the EVENTS transport too. This route takes the
61
  # component's event objects verbatim, so a hidden key would otherwise arrive here with
62
  # nothing between it and the store.
63
- hidden_keys=perm_scope.hidden_keys(session.user, module, canonical),
64
  fallback_ws=None, seen_ids={}, **kw)
65
 
66
 
@@ -74,8 +86,15 @@ _SCOPES = ("customer", "cohort", "product")
74
  def _scope_or_400(raw):
75
  """⛔ REFUSE AN UNKNOWN SCOPE, never default it. A typo silently served as `customer` would
76
  hand a user the whole book on a page they opened to see one cohort — the widening direction,
77
- which is the one that must fail closed."""
 
 
 
 
 
78
  scope = (raw or "customer").strip().lower()
 
 
79
  if scope not in _SCOPES:
80
  raise err(400, "bad_scope",
81
  f"scope must be one of {', '.join(_SCOPES)} — refusing to guess")
@@ -102,6 +121,35 @@ def workspace(scope: str = "customer",
102
  from routes_customers import grid_assembly
103
 
104
  scope = _scope_or_400(scope)
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
105
  # ── Wave 16 C-TOPIC: the PRODUCT surface has its own assembly (own pool, own field
106
  # contract, own workspace BUCKET) and the contract's own storage key. It branches before
107
  # the customer derivation because the customer storage key embeds (bu, agent) while the
@@ -129,8 +177,9 @@ def workspace(scope: str = "customer",
129
  workspace["viewer"] = {"name": session.uname, "isAdmin": bool(session.admin)}
130
  try:
131
  from core import users as _users
132
- workspace["userOptions"] = _users.assignable_people()
133
- workspace["userAvatars"] = {k: v for k, v in _users.avatar_map().items()
 
134
  if str(v).startswith("data:image/")}
135
  except Exception:
136
  workspace["userOptions"] = []
@@ -184,11 +233,12 @@ def workspace(scope: str = "customer",
184
  workspace["viewer"] = {"name": session.uname, "isAdmin": bool(session.admin)}
185
  try:
186
  from core import users as _users
187
- workspace["userOptions"] = _users.assignable_people()
188
  # Wave 14 C-AVATAR — the options vocabulary's companion: display name -> data URL.
189
  # Absent entries fall back to the client's initials disc; a non-data: value is
190
  # dropped (defence in depth beside the write-side wall in routes_auth).
191
- workspace["userAvatars"] = {k: v for k, v in _users.avatar_map().items()
 
192
  if str(v).startswith("data:image/")}
193
  except Exception:
194
  workspace["userOptions"] = []
@@ -291,10 +341,10 @@ def grid_timeseries(body: dict = Body(default=None), scope: str = "customer",
291
  # product pids are CRC32 hashes and would mostly fall outside the customer book anyway,
292
  # but "mostly" is not a wall. The client hides the mode for this topic; this is the
293
  # server's half of the same refusal.
294
- if _scope_or_400(scope) == "product":
 
295
  raise err(400, "bad_timeseries",
296
- "the product surface has no time-series channel yet — measures are "
297
- "customer-grain")
298
 
299
  body = body or {}
300
  bucket = body.get("bucket")
@@ -492,7 +542,8 @@ def grid_calendar_metrics(body: dict = Body(default=None), scope: str = "custome
492
 
493
  # The same refusal the TS channel makes, for the same reason: measures are customer-grain
494
  # and product pids are CRC32 hashes of SKU codes. "Mostly outside the book" is not a wall.
495
- if _scope_or_400(scope) == "product":
 
496
  raise err(400, "bad_calendar_metrics",
497
  "the product surface has no measure channel yet — measures are customer-grain")
498
 
@@ -645,6 +696,12 @@ def grid_events_route(body: dict = Body(default=None),
645
 
646
  session.require(PRODUCT_MODULE) # the route's own dependency asserts only customer_data
647
  g = product_assembly(session, consume_corrections=False)
 
 
 
 
 
 
648
  else:
649
  g = grid_assembly(session, scope=scope, consume_corrections=False)
650
  # ⚠ THE MEASURE CONTEXT IS NOT OPTIONAL (2026-07-31). Without `measure_offer`,
@@ -698,7 +755,7 @@ def grid_events_route(body: dict = Body(default=None),
698
  # ⚠ AND IT IS A SHORTCUT, NOT A PATH. Any failure is swallowed: `WORKSPACE_STALE` still
699
  # fires from `rerender`, and the re-read still delivers these values exactly as it does
700
  # today. Nothing depends on this having worked.
701
- if out["rerender"] and scope != "product" and any(
702
  isinstance(e, dict) and e.get("type") == "field_upsert"
703
  and str(((e.get("field") or {}) if isinstance(e.get("field"), dict) else {})
704
  .get("key") or "").startswith("measure_")
 
47
  # contract the hidden-field closure runs over, and the TABLE OPS the write lands in.
48
  # Getting any one of them from the other topic is the "validated against the wrong field
49
  # contract" near-miss the wave-15 routes_products header warned about.
50
+ scope_key = str(kw.get("scope_key") or "")
51
+ if scope_key == "product":
52
  import modules.product_data as pd
53
  from routes_products import MODULE as _PMOD, pd_fields
54
 
55
  module, canonical = _PMOD, pd_fields(consolidated=True)
56
  kw.setdefault("table", pd.TABLE_OPS)
57
+ hidden = perm_scope.hidden_keys(session.user, module, canonical)
58
+ elif scope_key.startswith("ut_"):
59
+ # Wave 18 C3-UT: a user table has no module in the permission wall (its wall is
60
+ # `user_tables.may_open`, already applied by the assembly this route ran first), so
61
+ # the hidden-field closure is EMPTY rather than borrowed from another topic's contract.
62
+ import core.table_store as table_store
63
+
64
+ kw.setdefault("table",
65
+ table_store.make(f"{scope_key}_table_workspace", st=session.runtime))
66
+ hidden = frozenset()
67
  else:
68
  module, canonical = MODULE, aios_grid.FIELDS
69
+ hidden = perm_scope.hidden_keys(session.user, module, canonical)
70
  return grid_events.EventCtx(
71
  uname=session.uname, allowed_pids=pids, fields=fields, admin=session.admin,
72
  # C-PERM: the write wall's field half, on the EVENTS transport too. This route takes the
73
  # component's event objects verbatim, so a hidden key would otherwise arrive here with
74
  # nothing between it and the store.
75
+ hidden_keys=hidden,
76
  fallback_ws=None, seen_ids={}, **kw)
77
 
78
 
 
86
  def _scope_or_400(raw):
87
  """⛔ REFUSE AN UNKNOWN SCOPE, never default it. A typo silently served as `customer` would
88
  hand a user the whole book on a page they opened to see one cohort — the widening direction,
89
+ which is the one that must fail closed.
90
+
91
+ Wave 18 (C3-UT): a `ut_`-prefixed scope names a USER TABLE and passes through here —
92
+ existence and the per-table wall are enforced by `routes_tables.ut_assembly` (404/403),
93
+ which every consumer of such a scope goes through. Passing an unknown ut key therefore
94
+ still fails closed, just one layer down where the store can actually be consulted."""
95
  scope = (raw or "customer").strip().lower()
96
+ if scope.startswith("ut_"):
97
+ return scope
98
  if scope not in _SCOPES:
99
  raise err(400, "bad_scope",
100
  f"scope must be one of {', '.join(_SCOPES)} — refusing to guess")
 
121
  from routes_customers import grid_assembly
122
 
123
  scope = _scope_or_400(scope)
124
+ # ── Wave 18 C3-UT: a USER TABLE'S workspace — the third topic through the one wire. The
125
+ # storage key has no (bu, agent) because a user table has no Odoo scope; per-user by the
126
+ # table's own wall (creator or admin, `user_tables.may_open`).
127
+ if scope.startswith("ut_"):
128
+ from routes_tables import ut_assembly
129
+
130
+ storage_key = f"{session.tenant}:{scope}:{session.uname}"
131
+ try:
132
+ g = ut_assembly(session, scope, storage_key=storage_key)
133
+ except grid_events.StoreUnavailable:
134
+ raise err(503, "store_unavailable", "the tenant store is unavailable")
135
+ workspace = g["workspace"]
136
+ workspace["overlays"] = g["ws"].get("overlays") or {}
137
+ workspace["measures"] = g["measures"]
138
+ workspace["measureSets"] = g["measure_sets"]
139
+ workspace["derived"] = {}
140
+ workspace["viewer"] = {"name": session.uname, "isAdmin": bool(session.admin)}
141
+ try:
142
+ from core import users as _users
143
+ workspace["userOptions"] = _users.assignable_people(tenant=session.tenant)
144
+ workspace["userAvatars"] = {k: v for k, v in
145
+ _users.avatar_map(tenant=session.tenant).items()
146
+ if str(v).startswith("data:image/")}
147
+ except Exception:
148
+ workspace["userOptions"] = []
149
+ workspace["userAvatars"] = {}
150
+ workspace["scopeKey"] = scope
151
+ return {"workspace": workspace}
152
+
153
  # ── Wave 16 C-TOPIC: the PRODUCT surface has its own assembly (own pool, own field
154
  # contract, own workspace BUCKET) and the contract's own storage key. It branches before
155
  # the customer derivation because the customer storage key embeds (bu, agent) while the
 
177
  workspace["viewer"] = {"name": session.uname, "isAdmin": bool(session.admin)}
178
  try:
179
  from core import users as _users
180
+ workspace["userOptions"] = _users.assignable_people(tenant=session.tenant)
181
+ workspace["userAvatars"] = {k: v for k, v in
182
+ _users.avatar_map(tenant=session.tenant).items()
183
  if str(v).startswith("data:image/")}
184
  except Exception:
185
  workspace["userOptions"] = []
 
233
  workspace["viewer"] = {"name": session.uname, "isAdmin": bool(session.admin)}
234
  try:
235
  from core import users as _users
236
+ workspace["userOptions"] = _users.assignable_people(tenant=session.tenant)
237
  # Wave 14 C-AVATAR — the options vocabulary's companion: display name -> data URL.
238
  # Absent entries fall back to the client's initials disc; a non-data: value is
239
  # dropped (defence in depth beside the write-side wall in routes_auth).
240
+ workspace["userAvatars"] = {k: v for k, v in
241
+ _users.avatar_map(tenant=session.tenant).items()
242
  if str(v).startswith("data:image/")}
243
  except Exception:
244
  workspace["userOptions"] = []
 
341
  # product pids are CRC32 hashes and would mostly fall outside the customer book anyway,
342
  # but "mostly" is not a wall. The client hides the mode for this topic; this is the
343
  # server's half of the same refusal.
344
+ _sc = _scope_or_400(scope)
345
+ if _sc == "product" or _sc.startswith("ut_"):
346
  raise err(400, "bad_timeseries",
347
+ "this surface has no time-series channel — measures are customer-grain")
 
348
 
349
  body = body or {}
350
  bucket = body.get("bucket")
 
542
 
543
  # The same refusal the TS channel makes, for the same reason: measures are customer-grain
544
  # and product pids are CRC32 hashes of SKU codes. "Mostly outside the book" is not a wall.
545
+ _sc_cal = _scope_or_400(scope)
546
+ if _sc_cal == "product" or _sc_cal.startswith("ut_"):
547
  raise err(400, "bad_calendar_metrics",
548
  "the product surface has no measure channel yet — measures are customer-grain")
549
 
 
696
 
697
  session.require(PRODUCT_MODULE) # the route's own dependency asserts only customer_data
698
  g = product_assembly(session, consume_corrections=False)
699
+ elif scope.startswith("ut_"):
700
+ # Wave 18 C3-UT — the user-table wall (creator/admin) is inside the assembly; the
701
+ # measure/cohort context is honestly EMPTY (customer-grain machinery, no meaning here).
702
+ from routes_tables import ut_assembly
703
+
704
+ g = ut_assembly(session, scope, consume_corrections=False)
705
  else:
706
  g = grid_assembly(session, scope=scope, consume_corrections=False)
707
  # ⚠ THE MEASURE CONTEXT IS NOT OPTIONAL (2026-07-31). Without `measure_offer`,
 
755
  # ⚠ AND IT IS A SHORTCUT, NOT A PATH. Any failure is swallowed: `WORKSPACE_STALE` still
756
  # fires from `rerender`, and the re-read still delivers these values exactly as it does
757
  # today. Nothing depends on this having worked.
758
+ if out["rerender"] and scope in ("customer", "cohort") and any(
759
  isinstance(e, dict) and e.get("type") == "field_upsert"
760
  and str(((e.get("field") or {}) if isinstance(e.get("field"), dict) else {})
761
  .get("key") or "").startswith("measure_")
api/routes_keychain.py ADDED
@@ -0,0 +1,143 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """routes_keychain.py — Keychains + Connectors admin surfaces (wave 18, C7 / R3).
2
+
3
+ Keychain: encrypted per-tenant credential entries (`core/keychain.py`). The routes NEVER
4
+ return a decrypted field — list rows carry a masked preview, and the decrypt function is a
5
+ connector-layer internal. Connectors: the tenant's data sources as STATUS rows — Royal's
6
+ env-configured Odoo, keychain-held sources — plus R3's guardrail: the **Unsynced records**
7
+ count (rows holding overlay data whose pids the current pool no longer serves; counted and
8
+ drillable, never silently dropped) and a pause toggle whose v1 semantics are stated honestly
9
+ in the payload (`pausedNote`): pausing marks intent and warns; the source cutover ships with
10
+ the keychain cutover wave R3 staged.
11
+ """
12
+ import os
13
+
14
+ from fastapi import Body, Depends
15
+ from fastapi import APIRouter
16
+
17
+ from deps import Session, err
18
+ from routes_admin import admin_gate
19
+
20
+ router = APIRouter(prefix="/api/v1")
21
+
22
+ _CONNECTOR_FLAGS_KEY = "connectors"
23
+
24
+
25
+ def _kc():
26
+ import core.keychain as keychain
27
+ return keychain
28
+
29
+
30
+ @router.get("/admin/keychain")
31
+ def list_keychain(session: Session = Depends(admin_gate)):
32
+ kc = _kc()
33
+ return {"entries": kc.list_entries(session.runtime), "locked": not kc.unlocked()}
34
+
35
+
36
+ @router.post("/admin/keychain", status_code=201)
37
+ def add_key(body: dict = Body(default=None), session: Session = Depends(admin_gate)):
38
+ kc = _kc()
39
+ body = body or {}
40
+ if not session.runtime.available():
41
+ raise err(503, "store_unavailable", "the tenant store is unavailable")
42
+ try:
43
+ row = kc.add_entry(session.runtime, body.get("label"), body.get("type"),
44
+ body.get("fields"), session.uname)
45
+ except kc.KeychainLocked as e:
46
+ raise err(503, "keychain_locked",
47
+ f"the keychain is locked — {e}. A secret is never stored unencrypted.")
48
+ except ValueError as e:
49
+ raise err(400, "bad_entry", str(e))
50
+ except Exception:
51
+ raise err(503, "store_unavailable", "the entry was not saved — try again")
52
+ return {"entry": row}
53
+
54
+
55
+ @router.delete("/admin/keychain/{entry_id}")
56
+ def delete_key(entry_id: str, session: Session = Depends(admin_gate)):
57
+ try:
58
+ _kc().delete_entry(session.runtime, entry_id)
59
+ except Exception:
60
+ raise err(503, "store_unavailable", "the delete did not land — try again")
61
+ return {"ok": True}
62
+
63
+
64
+ @router.post("/admin/keychain/{entry_id}/test")
65
+ def test_key(entry_id: str, session: Session = Depends(admin_gate)):
66
+ return _kc().test_entry(session.runtime, entry_id)
67
+
68
+
69
+ def _unsynced_customer_records(session):
70
+ """R3's guardrail, tenant #0's customer topic: overlay-holding pids the CURRENT pool no
71
+ longer serves. Overlays are unioned across EVERY user of the table (the guardrail is a
72
+ tenant fact, not a per-user one). Honest degradation: when the pool cannot be built the
73
+ answer is `known: False`, never a fabricated zero."""
74
+ try:
75
+ import core.table_store as table_store
76
+ bucket = session.runtime.get("customer_table_workspace") or {}
77
+ overlay_pids = {}
78
+ for uname, ws in bucket.items():
79
+ if uname == table_store.SHARED_KEY or not isinstance(ws, dict):
80
+ continue
81
+ for pid, cells in (ws.get("overlays") or {}).items():
82
+ if isinstance(cells, dict) and cells:
83
+ overlay_pids.setdefault(str(pid), cells)
84
+ if not overlay_pids:
85
+ return {"known": True, "count": 0, "rows": []}
86
+ from routes_customers import allowed_pids
87
+ pool = {str(p) for p in allowed_pids(session)}
88
+ orphans = sorted((p for p in overlay_pids if p not in pool), key=lambda x: int(x)
89
+ if str(x).isdigit() else 0)
90
+ rows = []
91
+ for p in orphans[:50]:
92
+ cells = overlay_pids[p]
93
+ hint = next((str(v) for v in cells.values() if str(v).strip()), "")
94
+ rows.append({"pid": int(p) if str(p).isdigit() else p,
95
+ "fields": len(cells), "hint": hint[:80]})
96
+ return {"known": True, "count": len(orphans), "rows": rows,
97
+ "shown": min(len(orphans), 50)}
98
+ except Exception as e:
99
+ return {"known": False, "count": None, "rows": [],
100
+ "note": f"pool unavailable — {type(e).__name__}"}
101
+
102
+
103
+ @router.get("/admin/connectors")
104
+ def connectors(session: Session = Depends(admin_gate)):
105
+ kc = _kc()
106
+ flags = session.runtime.get(_CONNECTOR_FLAGS_KEY) or {}
107
+ rows = []
108
+ if session.tenant == "royal-imports" and os.environ.get("ODOO_URL"):
109
+ rows.append({"key": "odoo-env", "label": "Odoo (environment)", "type": "odoo",
110
+ "source": "env",
111
+ "paused": bool((flags.get("odoo-env") or {}).get("paused"))})
112
+ for e in kc.list_entries(session.runtime):
113
+ rows.append({"key": e["id"], "label": e["label"], "type": e["type"],
114
+ "source": "keychain", "preview": e["preview"],
115
+ "paused": bool((flags.get(e["id"]) or {}).get("paused"))})
116
+ out = {"connectors": rows, "locked": not kc.unlocked(),
117
+ # R3, stated where the UI reads it: pausing never deletes anything — overlays,
118
+ # notes and custom fields stay; serving from the last snapshot + automatic resync
119
+ # arrives with the cutover wave. The pause flag records intent TODAY.
120
+ "pausedNote": ("Pausing a connector never deletes data — notes, custom fields and "
121
+ "views stay. Snapshot serving while paused arrives with the "
122
+ "connector cutover; until then Royal's surfaces keep reading live.")}
123
+ if session.tenant == "royal-imports":
124
+ out["unsynced"] = _unsynced_customer_records(session)
125
+ return out
126
+
127
+
128
+ @router.post("/admin/connectors/{key}/pause")
129
+ def pause_connector(key: str, body: dict = Body(default=None),
130
+ session: Session = Depends(admin_gate)):
131
+ paused = bool((body or {}).get("paused"))
132
+ if not session.runtime.available():
133
+ raise err(503, "store_unavailable", "the tenant store is unavailable")
134
+
135
+ def _up(cur):
136
+ cur[str(key)] = {"paused": paused}
137
+ return cur
138
+
139
+ try:
140
+ session.runtime.update(_CONNECTOR_FLAGS_KEY, _up)
141
+ except Exception:
142
+ raise err(503, "store_unavailable", "the change was not saved — try again")
143
+ return {"key": key, "paused": paused}
api/routes_nav.py CHANGED
@@ -67,11 +67,41 @@ def nav(session: Session = Depends(require_session)):
67
  gets an explicit 403 — an empty 200 is indistinguishable from "the registry is empty" and is
68
  how a permission bug hides in plain sight (X2's never-an-empty-200 rule).
69
  """
70
- pages = perms.nav_pages(session.user)
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
71
  if not pages:
 
 
 
 
 
 
72
  raise err(403, "no_surfaces",
73
  "your account has no dashboards assigned — ask an administrator")
74
- return {"pages": pages, "landing": perms.landing_page(session.user)}
 
 
 
75
 
76
 
77
  @router.get("/nav/prefs")
@@ -116,6 +146,21 @@ def nav_schema(key: str, session: Session = Depends(require_session)):
116
  """The database's schema drawer payload: its field contract + the semantic measures this
117
  session may build with. Fail-closed on the SAME predicate as the nav — a key the session
118
  may not open answers 403, never a redacted schema."""
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
119
  session.require(key)
120
  pages = perms.nav_pages(session.user) or []
121
  page = next((p for p in pages if p.get("key") == key), None)
 
67
  gets an explicit 403 — an empty 200 is indistinguishable from "the registry is empty" and is
68
  how a permission bug hides in plain sight (X2's never-an-empty-200 rule).
69
  """
70
+ pages = list(perms.nav_pages(session.user) or [])
71
+ # Wave 18 C1-TENANT: the REGISTRY is the product's module catalogue — tenant #0's world.
72
+ # A tenant record may enable a subset (`modules: [...]`); absent/'all' means everything
73
+ # (royal-imports and the compiled builders). A blank tenant enables NOTHING: its nav is
74
+ # its own databases, which is what "different set of databases at later waves" means.
75
+ tcfg = getattr(session.runtime.tenant, "config", None) or {}
76
+ tmods = tcfg.get("modules", "all")
77
+ if tmods != "all":
78
+ allowed = {str(k) for k in (tmods or [])}
79
+ pages = [p for p in pages
80
+ if p.get("key") in allowed or (p.get("parent") or "") in allowed]
81
+ # Wave 18 C3-UT: this tenant's user-created databases, merged AFTER the registry rows —
82
+ # the host does the same at app.py:7796. Filtered by the per-table wall (`may_open`), so
83
+ # the nav cannot offer a row the table routes would refuse.
84
+ try:
85
+ import core.user_tables as user_tables
86
+ for e in user_tables.nav_entries(viewer=session.uname, is_admin=session.admin,
87
+ st=session.runtime):
88
+ pages.append({"key": e["key"], "label": e["label"],
89
+ "source": e.get("source") or "Blank", "chrome": "main"})
90
+ except Exception:
91
+ pass # a store blip must not take the whole nav down with it
92
  if not pages:
93
+ if tmods != "all":
94
+ # A provisioned tenant with no modules and no databases YET is a legitimate empty
95
+ # state, not a misconfigured account — the client renders "create your first
96
+ # database", and X2's never-an-empty-200 rule is honoured by saying WHY it is
97
+ # empty rather than leaving 200-[] ambiguous.
98
+ return {"pages": [], "landing": None, "empty": "no_databases"}
99
  raise err(403, "no_surfaces",
100
  "your account has no dashboards assigned — ask an administrator")
101
+ landing = perms.landing_page(session.user)
102
+ if landing and not any(p.get("key") == landing for p in pages):
103
+ landing = pages[0].get("key")
104
+ return {"pages": pages, "landing": landing}
105
 
106
 
107
  @router.get("/nav/prefs")
 
146
  """The database's schema drawer payload: its field contract + the semantic measures this
147
  session may build with. Fail-closed on the SAME predicate as the nav — a key the session
148
  may not open answers 403, never a redacted schema."""
149
+ # Wave 18 C3-UT: a user table's schema is its own definition, walled by ITS predicate
150
+ # (`may_open`) rather than the module grant machinery — `session.require` would 403 every
151
+ # ut key because a user table is deliberately not a module.
152
+ if key.startswith("ut_"):
153
+ import core.user_tables as user_tables
154
+ defn = user_tables.get(key, st=session.runtime)
155
+ if not defn or not user_tables.may_open(key, session.uname, session.admin,
156
+ st=session.runtime):
157
+ raise err(403, "forbidden", "that database belongs to another user")
158
+ return {"key": key, "label": defn.get("label") or key,
159
+ "source": defn.get("source") or "Blank",
160
+ "fields": [{"key": f["key"], "label": f["label"], "type": f["type"],
161
+ "source": f.get("source") or "overlay", "description": ""}
162
+ for f in (defn.get("fields") or [])],
163
+ "measures": []}
164
  session.require(key)
165
  pages = perms.nav_pages(session.user) or []
166
  page = next((p for p in pages if p.get("key") == key), None)
api/routes_tables.py ADDED
@@ -0,0 +1,239 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """routes_tables.py — USER TABLES over the wire (wave 18, contract C3-UT).
2
+
3
+ The runtime-database primitive: `core/user_tables.py` (wave-9 C6, host-only until now) served
4
+ through the SAME grid machinery every other topic rides — `table_store` for the per-user
5
+ workspace strata, `aios_grid.workspace_wire` for the wire shape, `core.grid_events` for every
6
+ durable write except rows. Rows are the one genuinely new channel: the events seam has no row
7
+ event types (the user_tables docstring's `row_add` gate was described, never built), so row
8
+ add/delete/patch are REST endpoints here, walled by `user_tables.is_user_table` +
9
+ `user_tables.may_open` — a connector-backed table can never accept an invented row.
10
+
11
+ TENANCY: every store touch goes through `session.runtime` (the tenant's store handle), so a
12
+ Nurilab admin's tables live under Nurilab's prefix/repo, and the isolation gate's proof #3
13
+ covers them for free.
14
+
15
+ VISIBILITY is `user_tables.may_open` — creator or admin, fail-closed. There is no module grant
16
+ to check because a user table is not a module; the per-table wall is the whole wall, and it is
17
+ applied in `_defn_or_refuse` before any payload is built.
18
+ """
19
+ import time
20
+
21
+ from fastapi import Body, Depends
22
+ from fastapi import APIRouter
23
+
24
+ from deps import Session, err, require_session
25
+
26
+ router = APIRouter(prefix="/api/v1")
27
+
28
+
29
+ def _ut():
30
+ import core.user_tables as user_tables
31
+ return user_tables
32
+
33
+
34
+ def _ops(session, table_key):
35
+ import core.table_store as table_store
36
+ return table_store.make(f"{table_key}_table_workspace", st=session.runtime)
37
+
38
+
39
+ def _defn_or_refuse(session, table_key):
40
+ """The per-table wall: 404 for a key that does not exist, 403 for one this session may not
41
+ open. 404-before-403 leaks nothing useful — ut keys are guessable slugs, and 'exists but
42
+ not yours' is exactly what may_open is for."""
43
+ ut = _ut()
44
+ defn = ut.get(table_key, st=session.runtime)
45
+ if not defn:
46
+ raise err(404, "unknown_table", "that database does not exist")
47
+ if not ut.may_open(table_key, session.uname, session.admin, st=session.runtime):
48
+ raise err(403, "forbidden", "that database belongs to another user")
49
+ return defn
50
+
51
+
52
+ def ut_assembly(session: Session, table_key: str, storage_key: str = "",
53
+ consume_corrections: bool = True):
54
+ """The user-table mirror of `grid_assembly` / `product_assembly` — SAME g-dict keys, so
55
+ `/workspace` and the events route consume any of the three interchangeably.
56
+
57
+ Honest absences: `measures`/`measure_sets`/`lists`/`derived` are EMPTY — measures are
58
+ customer-grain and cohorts store partner pids; neither means anything over user rows."""
59
+ import aios_grid
60
+ from core import grid_events
61
+
62
+ defn = _defn_or_refuse(session, table_key)
63
+ fields_base = [dict(f) for f in (defn.get("fields") or [])]
64
+ field_keys = {f["key"] for f in fields_base}
65
+ rows_src = []
66
+ for rid, row in (defn.get("rows") or {}).items():
67
+ if not str(rid).isdigit():
68
+ continue
69
+ r = {k: v for k, v in (row or {}).items() if k in field_keys}
70
+ r["pid"] = int(rid)
71
+ rows_src.append(r)
72
+ rows_src.sort(key=lambda r: r["pid"])
73
+ pids = frozenset(r["pid"] for r in rows_src)
74
+
75
+ ops = _ops(session, table_key)
76
+ ctx = grid_events.EventCtx(
77
+ uname=session.uname, allowed_pids=pids, fields=[], hidden_keys=frozenset(),
78
+ admin=session.admin, fallback_ws=None, seen_ids={},
79
+ scope_key=table_key, table=ops)
80
+ ws = grid_events.table_workspace(ctx, allowed_pids=pids,
81
+ consume_corrections=consume_corrections)
82
+ workspace, fields, views, _lists = aios_grid.workspace_wire(
83
+ ws, session.uname, set(pids), defs={}, scope_key=table_key,
84
+ storage_key=storage_key, fields_base=fields_base, with_cohorts=False)
85
+
86
+ return {"rows_src": rows_src, "pids": pids, "ws": ws, "workspace": workspace,
87
+ "fields": fields, "views": views, "lists": [], "derived": {},
88
+ "measures": [], "measure_sets": {}, "today": time.strftime("%Y-%m-%d"),
89
+ "defn": defn}
90
+
91
+
92
+ @router.get("/tables")
93
+ def list_tables(session: Session = Depends(require_session)):
94
+ """This session's user tables — the list the '+ New database' surface renders."""
95
+ ut = _ut()
96
+ out = []
97
+ for key, t in sorted(ut.all_tables(st=session.runtime).items(),
98
+ key=lambda kv: (kv[1].get("label") or "").lower()):
99
+ if not ut.may_open(key, session.uname, session.admin, st=session.runtime):
100
+ continue
101
+ out.append({"key": key, "label": t.get("label") or key,
102
+ "source": t.get("source") or "Blank",
103
+ "createdBy": t.get("createdBy") or "",
104
+ "created": t.get("created") or "",
105
+ "fields": [dict(f) for f in (t.get("fields") or [])],
106
+ "rowCount": len(t.get("rows") or {})})
107
+ return {"tables": out}
108
+
109
+
110
+ @router.post("/tables", status_code=201)
111
+ def create_table(body: dict = Body(default=None),
112
+ session: Session = Depends(require_session)):
113
+ ut = _ut()
114
+ body = body or {}
115
+ label = str(body.get("label") or "").strip()
116
+ if not label:
117
+ raise err(400, "bad_label", "give the database a name")
118
+ if not session.runtime.available():
119
+ raise err(503, "store_unavailable",
120
+ "the tenant store is unavailable — nothing was created")
121
+ source = body.get("source")
122
+ try:
123
+ key = ut.create(label, session.uname, fields=body.get("fields"),
124
+ source=source, st=session.runtime)
125
+ except Exception:
126
+ raise err(503, "store_unavailable",
127
+ "the tenant store refused the write — nothing was created")
128
+ if not key:
129
+ raise err(400, "refused",
130
+ f"could not create it — the name may be empty or this tenant already has "
131
+ f"{ut.MAX_TABLES} databases")
132
+ return {"key": key}
133
+
134
+
135
+ @router.delete("/tables/{table_key}")
136
+ def delete_table(table_key: str, session: Session = Depends(require_session)):
137
+ """Creator or admin only — the same actors `may_open` admits, and deletion is the one
138
+ operation the client must confirm explicitly (the server cannot tell a click from a plan)."""
139
+ _defn_or_refuse(session, table_key)
140
+ try:
141
+ _ut().delete(table_key, st=session.runtime)
142
+ except Exception:
143
+ raise err(503, "store_unavailable", "the delete did not land — try again")
144
+ # The workspace bucket (views/custom fields/overlays) is left in place: a delete that also
145
+ # destroyed per-user state would make an accidental delete unrecoverable, and the bucket is
146
+ # invisible without its definition. Booked, not hidden.
147
+ return {"ok": True}
148
+
149
+
150
+ @router.get("/tables/{table_key}/rows")
151
+ def table_rows(table_key: str, session: Session = Depends(require_session)):
152
+ """The `/customers`-shaped envelope for one user table: `{fields, rows, today, pulled_at,
153
+ identity}` — so the client's generic topic fetch consumes it with zero new parsing.
154
+
155
+ ⚠ THE MERGE ORDER IS THE CONTRACT. `rows_from_pool` sources an overlay-typed field's cell
156
+ from the OVERLAY stratum only (that is what makes a custom column render standalone) — a
157
+ user table's base values live in its DEFINITION rows, so they are layered UNDER the user's
158
+ overlay edits here: base first, overlay wins. Without this every base cell reads empty
159
+ (found by this route's own gate check, not by luck)."""
160
+ import aios_grid
161
+
162
+ g = ut_assembly(session, table_key)
163
+ merged = {str(r["pid"]): {k: v for k, v in r.items() if k != "pid"}
164
+ for r in g["rows_src"]}
165
+ for pid, cells in (g["ws"].get("overlays") or {}).items():
166
+ if isinstance(cells, dict):
167
+ merged.setdefault(str(pid), {}).update(cells)
168
+ rows = aios_grid.rows_from_pool(
169
+ g["rows_src"], g["fields"], merged, derived=g["derived"])
170
+ return {"fields": g["fields"], "rows": rows, "today": g["today"],
171
+ "pulled_at": time.strftime("%Y-%m-%d %H:%M"),
172
+ "identity": {"pid": "pid"},
173
+ "scope": {"table": table_key, "rowCount": len(rows)}}
174
+
175
+
176
+ @router.post("/tables/{table_key}/rows", status_code=201)
177
+ def add_row(table_key: str, body: dict = Body(default=None),
178
+ session: Session = Depends(require_session)):
179
+ _defn_or_refuse(session, table_key)
180
+ ut = _ut()
181
+ values = (body or {}).get("values") or {}
182
+ if not isinstance(values, dict):
183
+ raise err(400, "bad_values", "values must be an object of {fieldKey: value}")
184
+ try:
185
+ rid = ut.add_row(table_key, values, session.uname, st=session.runtime)
186
+ except Exception:
187
+ raise err(503, "store_unavailable", "the row was not saved — the store refused")
188
+ if rid is None:
189
+ raise err(400, "refused",
190
+ f"row refused — the table may be at its {ut.MAX_ROWS}-row cap")
191
+ return {"rid": rid, "pid": int(rid)}
192
+
193
+
194
+ @router.delete("/tables/{table_key}/rows/{rid}")
195
+ def delete_row(table_key: str, rid: str, session: Session = Depends(require_session)):
196
+ _defn_or_refuse(session, table_key)
197
+ try:
198
+ ok = _ut().delete_row(table_key, rid, st=session.runtime)
199
+ except Exception:
200
+ raise err(503, "store_unavailable", "the delete did not land — try again")
201
+ if not ok:
202
+ raise err(400, "refused", "rows can only be deleted from user-created databases")
203
+ return {"ok": True}
204
+
205
+
206
+ @router.patch("/tables/{table_key}/rows/{pid}")
207
+ def patch_row(table_key: str, pid: int, body: dict = Body(default=None),
208
+ session: Session = Depends(require_session)):
209
+ """Cell edits — the products PATCH on the user-table ctx. Routed through
210
+ `core.grid_events.handle_one` so truncation and permission rules stay ONE implementation;
211
+ the accepted values are read BACK from the bucket, never echoed from the request."""
212
+ from core import grid_events
213
+
214
+ updates = dict(body or {})
215
+ if not updates:
216
+ raise err(400, "empty_patch", "no fields to update")
217
+ g = ut_assembly(session, table_key, consume_corrections=False)
218
+ if pid not in g["pids"]:
219
+ raise err(403, "out_of_scope", "that row is not in this database")
220
+ ctx = grid_events.EventCtx(
221
+ uname=session.uname, allowed_pids=g["pids"], fields=g["fields"],
222
+ admin=session.admin, fallback_ws=None, seen_ids={}, hidden_keys=frozenset(),
223
+ scope_key=table_key, table=_ops(session, table_key))
224
+ try:
225
+ grid_events.handle_one(
226
+ {"id": f"patch:{table_key}:{pid}:{time.time_ns()}", "type": "overlay_patch",
227
+ "pid": pid, "updates": updates}, ctx)
228
+ except grid_events.StoreUnavailable:
229
+ raise err(503, "store_unavailable",
230
+ "the tenant store is unavailable — your change was not saved")
231
+ stored = (grid_events.table_workspace(ctx, allowed_pids=None)
232
+ .get("overlays") or {}).get(str(pid)) or {}
233
+ accepted = {k: stored.get(k) for k in updates if k in stored}
234
+ refused = sorted(k for k in updates
235
+ if k not in accepted or stored.get(k) != str(updates[k]))
236
+ out = {"pid": pid, "updates": accepted}
237
+ if refused:
238
+ out["refused"] = refused
239
+ return out
requirements.txt CHANGED
@@ -10,3 +10,7 @@ huggingface_hub>=0.20
10
  duckdb>=1.0
11
  pyyaml>=6.0
12
  streamlit==1.58.0
 
 
 
 
 
10
  duckdb>=1.0
11
  pyyaml>=6.0
12
  streamlit==1.58.0
13
+ pillow>=10.0
14
+ beautifulsoup4>=4.12
15
+ lxml>=5.0
16
+ cryptography>=42.0
royalimports_os/aios_grid.py CHANGED
@@ -102,7 +102,8 @@ def _round(v):
102
  #: `max`, SVG stars client-side — never emoji) · `created_time` (READ-ONLY, renders the row's
103
  #: `_created`) · `formula` (READ-ONLY, client-computed from the row's other cells).
104
  CUSTOM_FIELD_TYPES = {"text", "select", "multiselect", "user", "int", "currency", "pct", "date",
105
- "checkbox", "phone", "email", "url", "rating", "created_time", "formula"}
 
106
  #: User-created types whose CELLS are read-only: their values are computed (formula — client
107
  #: side, any error degrades to BLANK) or system-owned (created_time = the row's `_created`).
108
  #: Emitted with the cohort column's read-only mechanism — `source: 'odoo'` + `derived` — so
@@ -253,6 +254,40 @@ def _quotes_balanced(s):
253
  return s.count('"') % 2 == 0
254
 
255
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
256
  def _clean_formula(raw, valid_keys=None):
257
  """STRUCTURAL passthrough for a formula field's expression (wave-5 item 9).
258
 
@@ -576,6 +611,8 @@ def fields_from_workspace(workspace=None, cohorts=False, scope_key=None, fields_
576
  # every member, groups count it under each, the toolbar count stays distinct.
577
  **({"multi": True} if ftype == "multiselect" else {}),
578
  **({"max": _clean_rating_max(field.get("max"))} if ftype == "rating" else {}),
 
 
579
  **_field_extras(field, ftype),
580
  })
581
  if cohorts:
@@ -756,7 +793,7 @@ def _clean_rhs(raw, valid_keys):
756
  #: rename is therefore a stored-value MIGRATION, not a constant rename: accept 'dashboard' on
757
  #: READ forever, only ever WRITE 'chart'.
758
  DISPLAY_MODES = {'grid', 'list', 'calendar', 'kanban', 'map', 'dashboard', 'chart',
759
- 'timeseries'}
760
 
761
  #: C-DISP (wave 2026-08-02): the time-series view's bucket vocabulary and caps, plus the
762
  #: calendar-summary metric cap. types.ts mirrors these as TS_BUCKETS / TS_MAX_LAST_N /
@@ -768,6 +805,22 @@ TS_MAX_FIELDS = 12
768
  MAX_CALENDAR_METRICS = 4
769
  _ISO_DAY = re.compile(r'^\d{4}-\d{2}-\d{2}$')
770
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
771
  #: Wave 14 C-ACC ([[loopable-wave14-split]]; rulings R2/R3). Mirrored by types.ts
772
  #: TS_DELTA_KINDS / TS_MAX_CUSTOM_ROWS / TS_MAX_STYLES — the C-DISP byte-identical law.
773
  TS_DELTA_KINDS = ('abs', 'pct', 'yoy', 'ytd')
@@ -908,6 +961,140 @@ def _clean_chart(raw, valid_keys):
908
  return out
909
 
910
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
911
  def _clean_display(raw, valid_keys):
912
  """Structural passthrough for a view's `config.display` (wave-6 item 10), fail-closed.
913
 
@@ -1066,6 +1253,10 @@ def _clean_display(raw, valid_keys):
1066
  uniq.append(c)
1067
  if uniq:
1068
  out['charts'] = uniq
 
 
 
 
1069
  return out
1070
 
1071
 
 
102
  #: `max`, SVG stars client-side — never emoji) · `created_time` (READ-ONLY, renders the row's
103
  #: `_created`) · `formula` (READ-ONLY, client-computed from the row's other cells).
104
  CUSTOM_FIELD_TYPES = {"text", "select", "multiselect", "user", "int", "currency", "pct", "date",
105
+ "checkbox", "phone", "email", "url", "rating", "created_time", "formula",
106
+ "automation"}
107
  #: User-created types whose CELLS are read-only: their values are computed (formula — client
108
  #: side, any error degrades to BLANK) or system-owned (created_time = the row's `_created`).
109
  #: Emitted with the cohort column's read-only mechanism — `source: 'odoo'` + `derived` — so
 
254
  return s.count('"') % 2 == 0
255
 
256
 
257
+ #: Wave-18 C5-AUTOFIELD. `kind` is a whitelist because an unknown kind would be a column that
258
+ #: silently never runs; `source` names where the run's subject comes from.
259
+ AUTOMATION_KINDS = {"instagram_profile"}
260
+ AUTOMATION_SOURCES = {"record_url_field", "self"}
261
+ MAX_AUTOMATION_SETTINGS = 12
262
+
263
+
264
+ def _clean_automation(raw, valid_keys=None):
265
+ """Validate an `automation` config bag. Returns the clean dict, or None when there is
266
+ nothing valid to store (the column then renders as unconfigured — never invented)."""
267
+ if not isinstance(raw, dict):
268
+ return None
269
+ kind = str(raw.get('kind') or '').strip()
270
+ if kind not in AUTOMATION_KINDS:
271
+ return None
272
+ source = str(raw.get('source') or 'record_url_field').strip()
273
+ if source not in AUTOMATION_SOURCES:
274
+ source = 'record_url_field'
275
+ out = {'kind': kind, 'source': source}
276
+ url_field = str(raw.get('urlField') or '').strip()[:80]
277
+ # fail closed on a ref that does not exist, exactly as _clean_formula does at WRITE time
278
+ if url_field and (valid_keys is None or url_field in valid_keys):
279
+ out['urlField'] = url_field
280
+ settings = {}
281
+ for k, v in list((raw.get('settings') or {}).items())[:MAX_AUTOMATION_SETTINGS]:
282
+ if isinstance(v, bool) or isinstance(v, (int, float)):
283
+ settings[str(k)[:40]] = v
284
+ elif isinstance(v, str):
285
+ settings[str(k)[:40]] = v[:200]
286
+ if settings:
287
+ out['settings'] = settings
288
+ return out
289
+
290
+
291
  def _clean_formula(raw, valid_keys=None):
292
  """STRUCTURAL passthrough for a formula field's expression (wave-5 item 9).
293
 
 
611
  # every member, groups count it under each, the toolbar count stays distinct.
612
  **({"multi": True} if ftype == "multiselect" else {}),
613
  **({"max": _clean_rating_max(field.get("max"))} if ftype == "rating" else {}),
614
+ **({"automation": _clean_automation(field.get("automation"))}
615
+ if ftype == "automation" and _clean_automation(field.get("automation")) else {}),
616
  **_field_extras(field, ftype),
617
  })
618
  if cohorts:
 
793
  #: rename is therefore a stored-value MIGRATION, not a constant rename: accept 'dashboard' on
794
  #: READ forever, only ever WRITE 'chart'.
795
  DISPLAY_MODES = {'grid', 'list', 'calendar', 'kanban', 'map', 'dashboard', 'chart',
796
+ 'timeseries', 'catalog'}
797
 
798
  #: C-DISP (wave 2026-08-02): the time-series view's bucket vocabulary and caps, plus the
799
  #: calendar-summary metric cap. types.ts mirrors these as TS_BUCKETS / TS_MAX_LAST_N /
 
805
  MAX_CALENDAR_METRICS = 4
806
  _ISO_DAY = re.compile(r'^\d{4}-\d{2}-\d{2}$')
807
 
808
+ #: C6-CATALOG (wave 18) — the catalog view's vocabulary and caps. types.ts mirrors every name
809
+ #: below, and `cleanDisplay` applies the same drops in the SAME ORDER, so an accepted save reads
810
+ #: back identically on both engines. The code budget is the order-sensitive one — see
811
+ #: `_clean_catalogs`.
812
+ MAX_CATALOGS = 12
813
+ MAX_CATALOG_PAGES = 40
814
+ MAX_CATALOG_CODES = 500 # cumulative across ONE catalog's pages, spent in PAGE ORDER
815
+ CATALOG_PAPERS = {'letter', 'a4', 'tabloid'}
816
+ CATALOG_ORIENTATIONS = {'portrait', 'landscape'}
817
+ CATALOG_QUALITIES = {'web', 'print'}
818
+ CATALOG_PAGE_KINDS = {'cover', 'intro', 'section', 'gallery'}
819
+ CATALOG_COLS = (2, 3, 4)
820
+ CATALOG_ID_MAX, CATALOG_NAME_MAX = 40, 80
821
+ CATALOG_TITLE_MAX, CATALOG_BODY_MAX, CATALOG_CODE_MAX = 120, 2000, 60
822
+ _HEX6 = re.compile(r'^#[0-9A-Fa-f]{6}$')
823
+
824
  #: Wave 14 C-ACC ([[loopable-wave14-split]]; rulings R2/R3). Mirrored by types.ts
825
  #: TS_DELTA_KINDS / TS_MAX_CUSTOM_ROWS / TS_MAX_STYLES — the C-DISP byte-identical law.
826
  TS_DELTA_KINDS = ('abs', 'pct', 'yoy', 'ytd')
 
961
  return out
962
 
963
 
964
+ def _clean_catalog_page(raw, budget):
965
+ """C6-CATALOG — one page of a catalog. Returns `(page | None, codes_spent)`.
966
+
967
+ `budget` is what is LEFT of the catalog's 500-code allowance. Codes are deduped WITHIN a
968
+ page and not across the catalog: a product legitimately appears on a gallery page and again
969
+ in its section listing, and de-duplicating globally would silently delete the second
970
+ appearance. The budget is spent in page order, so a catalog that runs out loses the TAIL of
971
+ its last pages — never a random scatter, and never a page (a page with no products is a
972
+ heading the user can still see and fix).
973
+ """
974
+ if not isinstance(raw, dict):
975
+ return None, 0
976
+ page_id = str(raw.get('id') or '')[:CATALOG_ID_MAX]
977
+ kind = raw.get('kind')
978
+ if not page_id or kind not in CATALOG_PAGE_KINDS:
979
+ return None, 0
980
+ out = {'id': page_id, 'kind': kind}
981
+ for key, cap in (('title', CATALOG_TITLE_MAX), ('body', CATALOG_BODY_MAX),
982
+ ('imageCode', CATALOG_CODE_MAX)):
983
+ v = raw.get(key)
984
+ if isinstance(v, str) and v:
985
+ out[key] = v[:cap]
986
+ products = raw.get('products')
987
+ if isinstance(products, list) and budget > 0:
988
+ clean_p, seen_p = [], set()
989
+ for c in products:
990
+ if not isinstance(c, str) or not c:
991
+ continue
992
+ c = c[:CATALOG_CODE_MAX]
993
+ if c in seen_p:
994
+ continue
995
+ seen_p.add(c)
996
+ clean_p.append(c)
997
+ if len(clean_p) >= budget:
998
+ break
999
+ if clean_p:
1000
+ out['products'] = clean_p
1001
+ layout = raw.get('layout')
1002
+ if isinstance(layout, dict):
1003
+ clean_l = {}
1004
+ cols = layout.get('cols')
1005
+ if isinstance(cols, int) and not isinstance(cols, bool) and cols in CATALOG_COLS:
1006
+ clean_l['cols'] = cols
1007
+ # The kanbanClamp/tsSparkline asymmetry, one per direction: pack and colour SHOW by
1008
+ # default (the 2027 catalogue shows both), price does NOT (it shows no prices at all).
1009
+ # So only the opt-OUT is storable for the first two and only the opt-IN for the third —
1010
+ # a second spelling of a default is how a round trip starts churning.
1011
+ if layout.get('showPack') is False:
1012
+ clean_l['showPack'] = False
1013
+ if layout.get('showColor') is False:
1014
+ clean_l['showColor'] = False
1015
+ if layout.get('showPrice') is True:
1016
+ clean_l['showPrice'] = True
1017
+ if clean_l:
1018
+ out['layout'] = clean_l
1019
+ return out, len(out.get('products') or ())
1020
+
1021
+
1022
+ def _clean_catalogs(raw, valid_keys):
1023
+ """C6-CATALOG (wave 18) — `display.catalogs`, fail-closed. Returns a list or None.
1024
+
1025
+ A catalog is a PRINT artifact, so the two structural keys that decide how it paginates
1026
+ (`paper`, `orientation`) are NORMALISED WITH A DEFAULT rather than dropped: a page box with
1027
+ no size is not a smaller catalog, it is an unrenderable one. Everything else follows the
1028
+ house rules — unknown keys dropped, per-entry drops never cost the neighbours, empty
1029
+ sub-objects omitted entirely (`brand`, `fields`, `layout`, `products`) so an absent key and
1030
+ an empty one are not two spellings of the same nothing.
1031
+
1032
+ `fields` binds the listing lines to real columns (the 2027 listing prints description / SKU /
1033
+ pack / colour, and `product_data` carries no pack or colour of its own — the user binds
1034
+ custom fields). Refs are checked against `valid_keys` HERE and not on the client, the same
1035
+ split `dateField`/`stackField` already run.
1036
+ """
1037
+ if not isinstance(raw, list):
1038
+ return None
1039
+ out = []
1040
+ for c in raw:
1041
+ if len(out) >= MAX_CATALOGS:
1042
+ break
1043
+ if not isinstance(c, dict):
1044
+ continue
1045
+ cat_id = str(c.get('id') or '')[:CATALOG_ID_MAX]
1046
+ name = c.get('name')
1047
+ # An EMPTY name is legal (the user cleared the box and will type again) — an ABSENT one
1048
+ # is a malformed record. The `tsRows` label rule, same reasoning.
1049
+ if not cat_id or not isinstance(name, str):
1050
+ continue
1051
+ cat = {'id': cat_id, 'name': name[:CATALOG_NAME_MAX]}
1052
+ cat['paper'] = c['paper'] if c.get('paper') in CATALOG_PAPERS else 'letter'
1053
+ cat['orientation'] = (c['orientation']
1054
+ if c.get('orientation') in CATALOG_ORIENTATIONS else 'portrait')
1055
+ if c.get('quality') in CATALOG_QUALITIES:
1056
+ cat['quality'] = c['quality']
1057
+ brand = c.get('brand')
1058
+ if isinstance(brand, dict):
1059
+ clean_b = {}
1060
+ for k in ('primary', 'accent'):
1061
+ v = brand.get(k)
1062
+ if isinstance(v, str) and _HEX6.match(v):
1063
+ clean_b[k] = v
1064
+ company = brand.get('company')
1065
+ if isinstance(company, str) and company:
1066
+ clean_b['company'] = company[:CATALOG_NAME_MAX]
1067
+ # An asset CODE (resolved through C2-ASSET), never a URL: an arbitrary host inside
1068
+ # print CSS is exactly the tokens-not-values rule this contract carries.
1069
+ logo = brand.get('logo')
1070
+ if isinstance(logo, str) and logo:
1071
+ clean_b['logo'] = logo[:CATALOG_CODE_MAX]
1072
+ if clean_b:
1073
+ cat['brand'] = clean_b
1074
+ binds = c.get('fields')
1075
+ if isinstance(binds, dict):
1076
+ clean_bind = {k: binds[k] for k in ('name', 'pack', 'color', 'price')
1077
+ if binds.get(k) in valid_keys}
1078
+ if clean_bind:
1079
+ cat['fields'] = clean_bind
1080
+ pages, budget = [], MAX_CATALOG_CODES
1081
+ raw_pages = c.get('pages')
1082
+ if isinstance(raw_pages, list):
1083
+ for p in raw_pages:
1084
+ if len(pages) >= MAX_CATALOG_PAGES:
1085
+ break
1086
+ page, spent = _clean_catalog_page(p, budget)
1087
+ if page is None:
1088
+ continue
1089
+ budget -= spent
1090
+ pages.append(page)
1091
+ # ALWAYS emitted, even empty: a catalog with no pages yet is the state every catalog
1092
+ # starts in, and dropping the key would make "new" and "corrupt" the same wire value.
1093
+ cat['pages'] = pages
1094
+ out.append(cat)
1095
+ return out or None
1096
+
1097
+
1098
  def _clean_display(raw, valid_keys):
1099
  """Structural passthrough for a view's `config.display` (wave-6 item 10), fail-closed.
1100
 
 
1253
  uniq.append(c)
1254
  if uniq:
1255
  out['charts'] = uniq
1256
+ # ── C6-CATALOG (wave 18) ─────────────────────────────────────────────────────────────
1257
+ catalogs = _clean_catalogs(raw.get('catalogs'), valid_keys)
1258
+ if catalogs:
1259
+ out['catalogs'] = catalogs
1260
  return out
1261
 
1262
 
royalimports_os/core/assets.py ADDED
@@ -0,0 +1,73 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """core/assets.py — the product-image ASSET SOURCE (wave 18, C2-ASSET).
2
+
3
+ WHY THIS FILE EXISTS AT ALL, given the route is three functions long: the API process is
4
+ HOST-AGNOSTIC by contract (`ops/verify_portability.py` checks B1/B2 — the HuggingFace SDK may
5
+ appear ONLY in its declared homes, all of which live under `royalimports_os/`, so the FastAPI
6
+ service can be lifted to Hetzner/EC2 without dragging a hub client with it). The first version
7
+ of `routes_assets.py` imported `huggingface_hub` directly and the gate caught it. The fetch is
8
+ a STORE-shaped concern anyway, so it belongs beside `core/store.py`.
9
+
10
+ Two sources, checked by the caller in order:
11
+ * a local master directory (`AIOS_ASSET_DIR`) — dev, and the fastest path on the owner's box;
12
+ * the private HF dataset `AIOS_ASSET_REPO` (default `royal-imports/product-assets`), laid out
13
+ by `ingest_product_assets.py`:
14
+ assets/products/orig/<CODE><ext> masters, archived verbatim (owner ruling R9)
15
+ assets/products/web/<CODE>.png ~800px derivatives (what the app displays)
16
+ manifest.json {"codes": {"<CODE>": {"ext": ".png"}}, ...}
17
+
18
+ Every failure returns None/{} rather than raising: a missing image is "no picture for that SKU",
19
+ never a broken catalog page.
20
+ """
21
+ import os
22
+ import time
23
+ from pathlib import Path
24
+
25
+ DEFAULT_REPO = 'royal-imports/product-assets'
26
+ _MANIFEST_TTL = 300
27
+ #: (fetched_at, {"CODE": {"ext": ".png"}}) — one manifest read per process per TTL.
28
+ _MANIFEST = {'at': 0.0, 'codes': None}
29
+
30
+
31
+ def repo_id():
32
+ return os.environ.get('AIOS_ASSET_REPO', '').strip() or DEFAULT_REPO
33
+
34
+
35
+ def _token():
36
+ return os.environ.get('HF_TOKEN') or None
37
+
38
+
39
+ def manifest():
40
+ """{CODE: {'ext': '.png'}} for the configured asset repo, or {} when unreachable."""
41
+ now = time.time()
42
+ if _MANIFEST['codes'] is not None and now - _MANIFEST['at'] < _MANIFEST_TTL:
43
+ return _MANIFEST['codes']
44
+ try:
45
+ import json
46
+
47
+ from huggingface_hub import hf_hub_download
48
+ p = hf_hub_download(repo_id(), 'manifest.json', repo_type='dataset', token=_token())
49
+ codes = (json.loads(Path(p).read_text(encoding='utf-8')) or {}).get('codes') or {}
50
+ except Exception:
51
+ codes = {}
52
+ _MANIFEST.update(at=now, codes=codes)
53
+ return codes
54
+
55
+
56
+ def fetch(code, quality='web'):
57
+ """A local Path to one asset, or None. The hub cache dedupes across requests, so a container
58
+ downloads each file at most once."""
59
+ meta = manifest().get(code)
60
+ if not meta:
61
+ return None
62
+ rel = (f'assets/products/web/{code}.png' if quality == 'web'
63
+ else f"assets/products/orig/{code}{meta.get('ext', '.png')}")
64
+ try:
65
+ from huggingface_hub import hf_hub_download
66
+ return Path(hf_hub_download(repo_id(), rel, repo_type='dataset', token=_token()))
67
+ except Exception:
68
+ return None
69
+
70
+
71
+ def reset_cache():
72
+ """Drop the memoized manifest — for gates and for a post-ingest refresh."""
73
+ _MANIFEST.update(at=0.0, codes=None)
royalimports_os/core/grid_events.py CHANGED
@@ -807,6 +807,13 @@ def handle_one(event, ctx):
807
  if formula is None:
808
  return False
809
  field['formula'] = formula
 
 
 
 
 
 
 
810
  fmt = _ag2._clean_format(raw.get('format'), ftype)
811
  if fmt:
812
  field['format'] = fmt
@@ -1273,6 +1280,13 @@ def handle_one(event, ctx):
1273
  admin or (field_by_key.get(key) or {}).get('createdBy') == uname):
1274
  refused = True
1275
  continue
 
 
 
 
 
 
 
1276
  clean = str(value)[:10000]
1277
  if isinstance(value, str) and clean != value:
1278
  # Truncated: the stored truth is shorter than the client's copy. (An
 
807
  if formula is None:
808
  return False
809
  field['formula'] = formula
810
+ if ftype == 'automation':
811
+ # C5-AUTOFIELD (wave 18): persist the validated config bag. No `return False`
812
+ # when absent — an unconfigured automation column is a legitimate intermediate
813
+ # state (create the column, then configure it in the gear).
814
+ auto = _ag2._clean_automation(raw.get('automation'), valid_keys)
815
+ if auto:
816
+ field['automation'] = auto
817
  fmt = _ag2._clean_format(raw.get('format'), ftype)
818
  if fmt:
819
  field['format'] = fmt
 
1280
  admin or (field_by_key.get(key) or {}).get('createdBy') == uname):
1281
  refused = True
1282
  continue
1283
+ # C5-AUTOFIELD (wave 18): an automation cell is MACHINE-WRITTEN. The client marks
1284
+ # the type read-only (READONLY_CELL_TYPES) but that is courtesy — this is the wall.
1285
+ # Without it the UI hides the editor while the API still takes a PATCH, which is
1286
+ # exactly the shape a "read-only" type must not have.
1287
+ if (field_by_key.get(key) or {}).get('type') == 'automation':
1288
+ refused = True
1289
+ continue
1290
  clean = str(value)[:10000]
1291
  if isinstance(value, str) and clean != value:
1292
  # Truncated: the stored truth is shorter than the client's copy. (An
royalimports_os/core/keychain.py ADDED
@@ -0,0 +1,182 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """core/keychain.py — per-tenant CREDENTIAL STORE (wave 18, contract C7 / ruling R3).
2
+
3
+ An admin stores a data source's secrets (Royal's Odoo API key today; any tenant's sources
4
+ tomorrow) encrypted at rest in the TENANT's store bucket. The runtime posture is R3's
5
+ "live with env fallback": the connector layer asks `odoo_creds(rt)` FIRST and falls back to
6
+ the process environment (tenant #0's `.env` / Space secrets), so Royal keeps working with an
7
+ empty keychain and a new tenant needs no code to bring keys. The FULL cutover (env retired)
8
+ is staged for a later wave — R3, verbatim.
9
+
10
+ CRYPTO: Fernet (AES128-CBC + HMAC, the `cryptography` package) under ONE platform key,
11
+ `AIOS_KEYCHAIN_KEY` (a `Fernet.generate_key()` value; a Space secret, never in the store).
12
+ No key ⇒ the keychain is LOCKED: metadata still lists (labels are not secrets), but nothing
13
+ decrypts and writes are refused — fail closed, never a plaintext fallback.
14
+
15
+ SHAPE (bucket `keychain`, per-tenant via the runtime handle):
16
+
17
+ {entries: {id: {id, label, type, enc, preview, created, createdBy}}}
18
+
19
+ `enc` = Fernet token over the JSON fields dict. `preview` = a masked hint ("····1234")
20
+ computed ONCE at write from the entry's most secret-looking field — the ONLY part of a
21
+ secret that ever leaves this module through a route. `read_fields` exists for the CONNECTOR
22
+ LAYER; no route returns its output.
23
+ """
24
+ import datetime as _dt
25
+ import json
26
+ import os
27
+ import secrets as _secrets
28
+
29
+ KEY = 'keychain'
30
+ ENTRY_TYPES = {'odoo', 'generic'}
31
+ #: which stored field feeds the masked preview, per type (first present wins)
32
+ _PREVIEW_FIELDS = ('api_key', 'value', 'password', 'token')
33
+ MAX_ENTRIES = 40
34
+ MAX_FIELD_LEN = 500
35
+
36
+
37
+ class KeychainLocked(Exception):
38
+ """Raised when AIOS_KEYCHAIN_KEY is absent/wrong — callers answer 503, never {}."""
39
+
40
+
41
+ def _fernet():
42
+ key = (os.environ.get('AIOS_KEYCHAIN_KEY') or '').strip()
43
+ if not key:
44
+ raise KeychainLocked('AIOS_KEYCHAIN_KEY is not configured')
45
+ try:
46
+ from cryptography.fernet import Fernet
47
+ return Fernet(key.encode('ascii'))
48
+ except Exception as e:
49
+ raise KeychainLocked(f'keychain key unusable: {type(e).__name__}')
50
+
51
+
52
+ def unlocked():
53
+ try:
54
+ _fernet()
55
+ return True
56
+ except KeychainLocked:
57
+ return False
58
+
59
+
60
+ def _bucket(rt):
61
+ return rt.get(KEY) or {}
62
+
63
+
64
+ def list_entries(rt):
65
+ """Metadata ONLY — never a decrypted field, never `enc` itself."""
66
+ out = []
67
+ for eid, e in sorted((_bucket(rt).get('entries') or {}).items()):
68
+ if not isinstance(e, dict):
69
+ continue
70
+ out.append({'id': eid, 'label': e.get('label') or eid,
71
+ 'type': e.get('type') or 'generic',
72
+ 'preview': e.get('preview') or '',
73
+ 'created': e.get('created') or '',
74
+ 'createdBy': e.get('createdBy') or ''})
75
+ return out
76
+
77
+
78
+ def add_entry(rt, label, etype, fields, username):
79
+ """Encrypt + store. Returns the metadata row, or raises ValueError on shape problems and
80
+ KeychainLocked when there is no key (a secret must never be stored in the clear)."""
81
+ label = ' '.join(str(label or '').split())[:80]
82
+ etype = str(etype or '').strip().lower()
83
+ if not label:
84
+ raise ValueError('give the key a label')
85
+ if etype not in ENTRY_TYPES:
86
+ raise ValueError(f"type must be one of {', '.join(sorted(ENTRY_TYPES))}")
87
+ if not isinstance(fields, dict) or not fields:
88
+ raise ValueError('fields must be a non-empty object')
89
+ clean = {}
90
+ for k, v in list(fields.items())[:12]:
91
+ k = str(k).strip()[:40]
92
+ if k and isinstance(v, (str, int, float, bool)):
93
+ clean[k] = str(v)[:MAX_FIELD_LEN]
94
+ if not clean:
95
+ raise ValueError('no usable fields')
96
+ token = _fernet().encrypt(json.dumps(clean, ensure_ascii=False).encode('utf-8'))
97
+ secretish = next((clean[k] for k in _PREVIEW_FIELDS if clean.get(k)), '')
98
+ preview = ('····' + secretish[-4:]) if len(secretish) >= 4 else ('····' if secretish else '')
99
+ eid = f'k_{_secrets.token_hex(6)}'
100
+ row = {'id': eid, 'label': label, 'type': etype, 'enc': token.decode('ascii'),
101
+ 'preview': preview,
102
+ 'created': _dt.datetime.now().strftime('%Y-%m-%dT%H:%M:%S'),
103
+ 'createdBy': str(username or '')}
104
+
105
+ def _up(cur):
106
+ entries = cur.setdefault('entries', {})
107
+ if len(entries) >= MAX_ENTRIES:
108
+ raise ValueError(f'this keychain is at its {MAX_ENTRIES}-entry cap')
109
+ entries[eid] = row
110
+ return cur
111
+
112
+ rt.update(KEY, _up, flush='sync')
113
+ return {'id': eid, 'label': label, 'type': etype, 'preview': preview,
114
+ 'created': row['created'], 'createdBy': row['createdBy']}
115
+
116
+
117
+ def delete_entry(rt, entry_id):
118
+ def _drop(cur):
119
+ (cur.get('entries') or {}).pop(str(entry_id), None)
120
+ return cur
121
+ rt.update(KEY, _drop, flush='sync')
122
+ return True
123
+
124
+
125
+ def read_fields(rt, entry_id):
126
+ """Decrypt ONE entry's fields — for the CONNECTOR LAYER only; no route returns this.
127
+ None when the entry does not exist; KeychainLocked when the key is absent/wrong (a wrong
128
+ key must be loud — silently empty creds would read as 'not configured')."""
129
+ e = (_bucket(rt).get('entries') or {}).get(str(entry_id))
130
+ if not isinstance(e, dict) or not e.get('enc'):
131
+ return None
132
+ try:
133
+ raw = _fernet().decrypt(str(e['enc']).encode('ascii'))
134
+ except KeychainLocked:
135
+ raise
136
+ except Exception:
137
+ raise KeychainLocked('this entry does not decrypt under the configured key')
138
+ try:
139
+ data = json.loads(raw.decode('utf-8'))
140
+ return data if isinstance(data, dict) else None
141
+ except Exception:
142
+ return None
143
+
144
+
145
+ def odoo_creds(rt):
146
+ """R3's resolver seam: the FIRST odoo-type entry's fields, else None (the caller falls
147
+ back to the environment). Deterministic order = insertion-id sort, so 'first' is stable."""
148
+ for row in list_entries(rt):
149
+ if row['type'] == 'odoo':
150
+ return read_fields(rt, row['id'])
151
+ return None
152
+
153
+
154
+ def test_entry(rt, entry_id, timeout=8):
155
+ """A cheap, READ-ONLY liveness probe. odoo: xmlrpc `common.version()` (no auth, no data);
156
+ generic: shape-only. Always answers {ok, message} — an exception here is a result, not a
157
+ crash."""
158
+ try:
159
+ e = (_bucket(rt).get('entries') or {}).get(str(entry_id))
160
+ if not isinstance(e, dict):
161
+ return {'ok': False, 'message': 'no such entry'}
162
+ fields = read_fields(rt, entry_id) or {}
163
+ if e.get('type') != 'odoo':
164
+ return {'ok': bool(fields), 'message': f'{len(fields)} field(s) stored'}
165
+ url = str(fields.get('url') or '').rstrip('/')
166
+ if not url.startswith(('http://', 'https://')):
167
+ return {'ok': False, 'message': 'url must start with http(s)://'}
168
+ import socket
169
+ import xmlrpc.client
170
+ old = socket.getdefaulttimeout()
171
+ socket.setdefaulttimeout(timeout)
172
+ try:
173
+ common = xmlrpc.client.ServerProxy(f'{url}/xmlrpc/2/common', allow_none=True)
174
+ ver = common.version()
175
+ finally:
176
+ socket.setdefaulttimeout(old)
177
+ sv = (ver or {}).get('server_version') if isinstance(ver, dict) else None
178
+ return {'ok': True, 'message': f'Odoo answered (server {sv or "unknown"})'}
179
+ except KeychainLocked as e:
180
+ return {'ok': False, 'message': str(e)}
181
+ except Exception as e:
182
+ return {'ok': False, 'message': f'{type(e).__name__}: {str(e)[:120]}'}
royalimports_os/core/registry.py CHANGED
@@ -177,6 +177,12 @@ REGISTRY = [
177
  'proof). No nav, no page. Wave 17 R8 ("I don\'t even know what it does — delete '
178
  'them") made that literal: `validate_only` takes it out of the account menu too, '
179
  'which is the last place it was still visible. The proof survives; the door does not.'},
 
 
 
 
 
 
180
  {'key': 'settings', 'label': 'Settings', 'brand': False, 'hq': True, 'validate': False,
181
  'nav': False, # sidebar sentinel (account group) — owner IA 2026-07-12
182
  'note': 'User scope settings: the Business Unit toggle (strict isolation) and the Data basis '
 
177
  'proof). No nav, no page. Wave 17 R8 ("I don\'t even know what it does — delete '
178
  'them") made that literal: `validate_only` takes it out of the account menu too, '
179
  'which is the last place it was still visible. The proof survives; the door does not.'},
180
+ {'key': 'automation', 'label': 'Automation', 'brand': False, 'hq': True, 'validate': False,
181
+ 'nav': True,
182
+ 'note': 'Wave 18 (C-AUTONAV): the Automation surface — scheduled jobs that create and '
183
+ 'refresh user databases (website scrape-to-DB, the Instagram field). React-only '
184
+ 'surface (no PAGE_FUNCS entry, the no-new-Streamlit rule); admins hold it via '
185
+ '"all", other users need the explicit grant — fail-closed default.'},
186
  {'key': 'settings', 'label': 'Settings', 'brand': False, 'hq': True, 'validate': False,
187
  'nav': False, # sidebar sentinel (account group) — owner IA 2026-07-12
188
  'note': 'User scope settings: the Business Unit toggle (strict isolation) and the Data basis '
royalimports_os/core/store.py CHANGED
@@ -1,307 +1,369 @@
1
- """Tiny JSON persistence on a PRIVATE Hugging Face Dataset repo — the platform's ONLY writable store
2
- (Odoo stays strictly read-only). Holds the user registry, customer notes and per-user tags.
3
-
4
- Auth uses HF_TOKEN (a Space secret / .env value with write access to the royal-imports org). The
5
- repo is created private on first write. Reads use a short in-process cache that writes refresh, so
6
- within a running instance reads are consistent with writes; a fresh instance re-fetches the latest.
7
- """
8
- import io
9
- import os
10
- import json
11
- import threading
12
- import time as _time
13
- from pathlib import Path
14
-
15
- try:
16
- from dotenv import load_dotenv
17
- load_dotenv(Path(__file__).resolve().parents[1] / '.env')
18
- except Exception:
19
- pass
20
-
21
- from huggingface_hub import HfApi, hf_hub_download
22
- from huggingface_hub.utils import EntryNotFoundError, RepositoryNotFoundError
23
-
24
- REPO = os.environ.get('OS_DATA_REPO', 'royal-imports/cfo-os-data')
25
- _LOCK = threading.RLock()
26
- _CACHE = {}
27
- # --- async flush state (wave-7 W3, 2026-07-28) --------------------------------------------
28
- # One typed character in a Notes cell used to cost THREE synchronous network calls inside the
29
- # component round-trip (strict re-download + create_repo + full re-upload = one hub commit per
30
- # edit — measured seconds each, and a commit-budget hazard). update(..., flush='async') now
31
- # applies the mutation to the in-process cache synchronously (read-your-writes: every reader
32
- # already consults _CACHE first) and a background worker coalesces uploads. The restart-wipe
33
- # protection is UNCHANGED where it matters: the FIRST update of a key in a fresh process still
34
- # strict-reads (a transient read error aborts the write, never merges into {}); after that the
35
- # cache we last wrote IS the latest known state, so re-downloading it per keystroke bought
36
- # nothing but latency. Known, accepted: with a second concurrent writer (local QA against the
37
- # shared store) the stale window widens from ~one round-trip to ~the flush gap.
38
- _OWNED = {} # name -> True once _CACHE[name] reflects our own last accepted write
39
- _DIRTY = {} # name -> cache is ahead of the hub
40
- _FLUSHERS = {} # name -> live worker thread
41
- _REPO_OK = [False] # create_repo(exist_ok=True) confirmed once per process
42
- _FLUSH_DELAY = 2.0 # coalesce a burst of edits into one commit
43
- _FLUSH_MIN_GAP = 20.0 # floor between commits of one key (the 256-commits/hr budget)
44
- _LAST_FLUSH = {} # name -> monotonic time of the last completed upload
45
-
46
-
47
- def _token():
48
- return os.environ.get('HF_TOKEN') or None
49
-
50
-
51
- def available():
52
- """True when a token is configured — callers degrade gracefully (read-only UI) without one."""
53
- return bool(_token())
54
-
55
-
56
- def _download(name):
57
- """Parse {name}.json into a dict. Returns {} ONLY when the file is genuinely absent. Any
58
- other failure (network, auth, rate-limit, corrupt/partial JSON) RAISES — a transient read
59
- error must never be mistaken for an empty store, or a caller could overwrite real data with
60
- an empty dict (this is what wiped the user registry on restart)."""
61
- try:
62
- path = hf_hub_download(REPO, f'{name}.json', repo_type='dataset', token=_token(),
63
- force_download=True)
64
- with open(path, encoding='utf-8') as f:
65
- return json.load(f)
66
- except (EntryNotFoundError, RepositoryNotFoundError, FileNotFoundError):
67
- return {}
68
-
69
-
70
- def get(name, fresh=False):
71
- """Load {name}.json as a dict. LENIENT (for display reads): on a transient fetch error, hand
72
- back the last known-good cached value if we have one, else {}. Callers that MUTATE the store
73
- must go through update()/_read_strict, which RAISE on such errors so they never clobber real
74
- data with a phantom-empty read."""
75
- with _LOCK:
76
- if not fresh and name in _CACHE:
77
- return json.loads(json.dumps(_CACHE[name])) # hand back a copy
78
- try:
79
- data = _download(name)
80
- except Exception as e:
81
- try: # lazy import — core stays layer-clean
82
- import harness.telemetry as _tel # noqa: PLC0415
83
- _tel.error(f'store:get:{name}', e,
84
- fallback='cached' if name in _CACHE else 'empty')
85
- except Exception:
86
- pass
87
- if name in _CACHE:
88
- return json.loads(json.dumps(_CACHE[name]))
89
- return {}
90
- _CACHE[name] = data
91
- return json.loads(json.dumps(data))
92
-
93
-
94
- def _read_strict(name):
95
- """Like get(fresh=True) but RAISES on any transient read error ({} only for a genuinely
96
- absent file). Read-modify-write callers use this so a failed read ABORTS the write instead
97
- of merging changes into an empty dict and overwriting the store."""
98
- with _LOCK:
99
- data = _download(name)
100
- _CACHE[name] = data
101
- return json.loads(json.dumps(data))
102
-
103
-
104
- def exists(name):
105
- """True if {name}.json is confirmed present, False if confirmed absent, and True on ANY
106
- uncertainty — so a caller that would seed or overwrite the store never clobbers a registry
107
- we merely failed to reach."""
108
- try:
109
- files = HfApi(token=_token()).list_repo_files(REPO, repo_type='dataset')
110
- return f'{name}.json' in files
111
- except RepositoryNotFoundError:
112
- return False
113
- except Exception:
114
- return True
115
-
116
-
117
- def _upload(name, data):
118
- """The raw hub write. create_repo is confirmed ONCE per process — it was a full extra API
119
- round-trip on every put(), pure latency after the first call ever succeeded."""
120
- api = HfApi(token=_token())
121
- if not _REPO_OK[0]:
122
- api.create_repo(REPO, repo_type='dataset', private=True, exist_ok=True)
123
- _REPO_OK[0] = True
124
- buf = io.BytesIO(json.dumps(data, ensure_ascii=False, indent=2).encode('utf-8'))
125
- api.upload_file(path_or_fileobj=buf, path_in_repo=f'{name}.json', repo_id=REPO,
126
- repo_type='dataset', commit_message=f'update {name}')
127
-
128
-
129
- def upload_bytes(path_in_repo, data, message=None):
130
- """Persist RAW BYTES at an arbitrary path in the tenant dataset (wave-8 C5, documents).
131
-
132
- The rest of this module is JSON-per-key; a customer's attachment is neither JSON nor
133
- per-key, so it gets its own path rather than being base64'd into somebody's blob. That
134
- matters for a reason beyond tidiness: the workspace blob is READ ON EVERY PAGE RENDER, and
135
- a few megabytes of base64 inside it would be paid on every keystroke by every user of that
136
- table, not just by whoever opened the document.
137
-
138
- Bytes are never cached in-process: they are large, cold, and fetched one at a time on an
139
- explicit user action.
140
- """
141
- if not available():
142
- raise RuntimeError('No HF_TOKEN configured — persistence is unavailable.')
143
- with _LOCK:
144
- api = HfApi(token=_token())
145
- if not _REPO_OK[0]:
146
- api.create_repo(REPO, repo_type='dataset', private=True, exist_ok=True)
147
- _REPO_OK[0] = True
148
- api.upload_file(path_or_fileobj=io.BytesIO(data), path_in_repo=path_in_repo,
149
- repo_id=REPO, repo_type='dataset',
150
- commit_message=message or f'add {path_in_repo}')
151
-
152
-
153
- def download_bytes(path_in_repo):
154
- """Raw bytes back, or None when absent/unreachable.
155
-
156
- None rather than an exception: a missing attachment must degrade to "that file is gone",
157
- never to a broken page — the same contract every other read here follows.
158
- """
159
- if not available():
160
- return None
161
- try:
162
- local = hf_hub_download(REPO, path_in_repo, repo_type='dataset', token=_token())
163
- return Path(local).read_bytes()
164
- except (EntryNotFoundError, RepositoryNotFoundError):
165
- return None
166
- except Exception:
167
- return None
168
-
169
-
170
- def delete_path(path_in_repo):
171
- """Remove a stored file. Absent is SUCCESS — deleting what is already gone is the goal."""
172
- if not available():
173
- return False
174
- try:
175
- HfApi(token=_token()).delete_file(path_in_repo=path_in_repo, repo_id=REPO,
176
- repo_type='dataset',
177
- commit_message=f'delete {path_in_repo}')
178
- return True
179
- except (EntryNotFoundError, RepositoryNotFoundError):
180
- return True
181
- except Exception:
182
- return False
183
-
184
-
185
- def put(name, data):
186
- """Persist {name}.json (creates the private repo on first write). Updates the in-process cache."""
187
- if not available():
188
- raise RuntimeError('No HF_TOKEN configured — persistence is unavailable.')
189
- with _LOCK:
190
- _upload(name, data)
191
- _CACHE[name] = json.loads(json.dumps(data))
192
- _OWNED[name] = True
193
- _LAST_FLUSH[name] = _time.monotonic()
194
- return _CACHE[name]
195
-
196
-
197
- def _flush_worker(name):
198
- """Coalescing uploader for one key. Runs while dirty; retries with backoff on failure —
199
- the cache stays the truth and the dirty mark survives, so nothing is ever lost silently."""
200
- backoff = 2.0
201
- while True:
202
- _time.sleep(_FLUSH_DELAY)
203
- with _LOCK:
204
- gap = _time.monotonic() - _LAST_FLUSH.get(name, 0.0)
205
- if gap < _FLUSH_MIN_GAP:
206
- wait = _FLUSH_MIN_GAP - gap
207
- elif not _DIRTY.get(name):
208
- _FLUSHERS.pop(name, None)
209
- return
210
- else:
211
- _DIRTY[name] = False
212
- snapshot = json.loads(json.dumps(_CACHE.get(name, {})))
213
- wait = None
214
- if wait is not None:
215
- _time.sleep(wait)
216
- continue
217
- try:
218
- _upload(name, snapshot)
219
- with _LOCK:
220
- _LAST_FLUSH[name] = _time.monotonic()
221
- backoff = 2.0
222
- except Exception as e:
223
- with _LOCK:
224
- _DIRTY[name] = True # keep it queued; cache is still the truth
225
- try:
226
- import harness.telemetry as _tel # noqa: PLC0415
227
- _tel.error(f'store:flush:{name}', e)
228
- except Exception:
229
- pass
230
- _time.sleep(backoff)
231
- backoff = min(backoff * 2, 60.0)
232
-
233
-
234
- def _schedule_flush(name):
235
- """Mark dirty + ensure exactly one live worker for the key. Caller holds _LOCK."""
236
- _DIRTY[name] = True
237
- t = _FLUSHERS.get(name)
238
- if t is not None and t.is_alive():
239
- return
240
- t = threading.Thread(target=_flush_worker, args=(name,), daemon=True,
241
- name=f'store-flush-{name}')
242
- _FLUSHERS[name] = t
243
- t.start()
244
-
245
-
246
- def flush(name=None, timeout=30.0):
247
- """Block until pending async writes for `name` (or ALL keys) reach the hub — QA gates and
248
- shutdown hooks use this; the app itself never needs to."""
249
- deadline = _time.monotonic() + timeout
250
- names = [name] if name else None
251
- while _time.monotonic() < deadline:
252
- with _LOCK:
253
- pending = {k for k, v in _DIRTY.items() if v} | {
254
- k for k, t in _FLUSHERS.items() if t.is_alive()}
255
- if names is not None:
256
- pending &= set(names)
257
- if not pending:
258
- return True
259
- _time.sleep(0.2)
260
- return False
261
-
262
-
263
- def _flush_now_at_exit():
264
- """Process-exit safety net: push any dirty keys synchronously, bypassing the coalescing
265
- gap — a Space restart must not eat the last seconds of the async window."""
266
- with _LOCK:
267
- dirty = [k for k, v in _DIRTY.items() if v]
268
- snaps = {k: json.loads(json.dumps(_CACHE.get(k, {}))) for k in dirty}
269
- for k in dirty:
270
- _DIRTY[k] = False
271
- for k, d in snaps.items():
272
- try:
273
- _upload(k, d)
274
- except Exception:
275
- pass # exiting anyway — nothing left to reschedule
276
-
277
-
278
- import atexit # noqa: E402 (registered after the helpers it needs)
279
- atexit.register(_flush_now_at_exit)
280
-
281
-
282
- def update(name, fn, flush='sync'):
283
- """Read-modify-write. STRICT first read (raises on a transient error rather than starting
284
- from an empty dict — a network blip during 'add user' can never wipe the accounts), apply
285
- fn(data)->data, persist.
286
-
287
- flush='sync' (default) = exactly the historical behavior: fresh strict read + blocking
288
- upload. flush='async' (the table-workspace hot path, wave-7 W3) = the mutation applies to
289
- the in-process cache and returns immediately; a coalescing worker commits to the hub. The
290
- strict read still guards the FIRST async update of a key per process; after we own the
291
- cache, it IS the latest known state and re-downloading it per edit was pure latency."""
292
- with _LOCK:
293
- if flush == 'async' and _OWNED.get(name) and name in _CACHE:
294
- data = json.loads(json.dumps(_CACHE[name]))
295
- else:
296
- data = _read_strict(name)
297
- result = fn(data)
298
- data = result if result is not None else data
299
- if flush == 'async':
300
- if not available():
301
- raise RuntimeError('No HF_TOKEN configured — persistence is unavailable.')
302
- _CACHE[name] = json.loads(json.dumps(data))
303
- _OWNED[name] = True
304
- _schedule_flush(name)
305
- return data
306
- put(name, data)
307
- return data
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """Tiny JSON persistence on a PRIVATE Hugging Face Dataset repo — the platform's ONLY writable store
2
+ (Odoo stays strictly read-only). Holds the user registry, customer notes and per-user tags.
3
+
4
+ Auth uses HF_TOKEN (a Space secret / .env value with write access to the royal-imports org). The
5
+ repo is created private on first write. Reads use a short in-process cache that writes refresh, so
6
+ within a running instance reads are consistent with writes; a fresh instance re-fetches the latest.
7
+
8
+ WAVE 18 (C1-TENANT, R2): the store is now a CLASS, one instance per repo. The module-level
9
+ functions below delegate to the default instance (`OS_DATA_REPO`, tenant #0) — every existing
10
+ `import core.store as store` caller is byte-for-byte unaffected, which the gate battery proves.
11
+ `for_repo(repo_id)` hands back the bound instance for a tenant that owns its OWN dataset repo
12
+ (nurilab, gtmlab, …); `harness.runtime.TenantRuntime` binds it from the tenant record. Each
13
+ instance carries its own cache / dirty-marks / flush workers / commit-gap clock, so one tenant's
14
+ write burst can never starve another's flush window.
15
+ """
16
+ import io
17
+ import os
18
+ import json
19
+ import threading
20
+ import time as _time
21
+ from pathlib import Path
22
+
23
+ try:
24
+ from dotenv import load_dotenv
25
+ load_dotenv(Path(__file__).resolve().parents[1] / '.env')
26
+ except Exception:
27
+ pass
28
+
29
+ from huggingface_hub import HfApi, hf_hub_download
30
+ from huggingface_hub.utils import EntryNotFoundError, RepositoryNotFoundError
31
+
32
+ REPO = os.environ.get('OS_DATA_REPO', 'royal-imports/cfo-os-data')
33
+ _FLUSH_DELAY = 2.0 # coalesce a burst of edits into one commit
34
+ _FLUSH_MIN_GAP = 20.0 # floor between commits of one key (the 256-commits/hr budget)
35
+
36
+
37
+ def _token():
38
+ return os.environ.get('HF_TOKEN') or None
39
+
40
+
41
+ class Store:
42
+ """One repo's persistence: JSON-per-key + raw byte paths, with the wave-7 async flush.
43
+
44
+ Every method body is the pre-wave-18 module function, verbatim except globals→fields.
45
+ The behaviour contracts that matter are stated on each method and are UNCHANGED:
46
+ strict-read-guards-the-write, exists()-is-True-on-uncertainty, lenient display reads.
47
+ """
48
+
49
+ def __init__(self, repo):
50
+ self.repo = str(repo)
51
+ self._lock = threading.RLock()
52
+ self._cache = {}
53
+ # --- async flush state (wave-7 W3, 2026-07-28) ------------------------------------
54
+ # One typed character in a Notes cell used to cost THREE synchronous network calls
55
+ # inside the component round-trip. update(..., flush='async') applies the mutation to
56
+ # the in-process cache synchronously (read-your-writes: every reader consults the
57
+ # cache first) and a background worker coalesces uploads. The restart-wipe protection
58
+ # is UNCHANGED where it matters: the FIRST update of a key in a fresh process still
59
+ # strict-reads. Known, accepted: with a second concurrent writer the stale window
60
+ # widens from ~one round-trip to ~the flush gap.
61
+ self._owned = {} # name -> True once cache reflects our own last accepted write
62
+ self._dirty = {} # name -> cache is ahead of the hub
63
+ self._flushers = {} # name -> live worker thread
64
+ self._repo_ok = False # create_repo(exist_ok=True) confirmed once per instance
65
+ self._last_flush = {} # name -> monotonic time of the last completed upload
66
+
67
+ def available(self):
68
+ """True when a token is configured — callers degrade gracefully without one."""
69
+ return bool(_token())
70
+
71
+ def _download(self, name):
72
+ """Parse {name}.json into a dict. Returns {} ONLY when the file is genuinely absent.
73
+ Any other failure (network, auth, rate-limit, corrupt JSON) RAISES — a transient read
74
+ error must never be mistaken for an empty store, or a caller could overwrite real data
75
+ with an empty dict (this is what wiped the user registry on restart)."""
76
+ try:
77
+ path = hf_hub_download(self.repo, f'{name}.json', repo_type='dataset',
78
+ token=_token(), force_download=True)
79
+ with open(path, encoding='utf-8') as f:
80
+ return json.load(f)
81
+ except (EntryNotFoundError, RepositoryNotFoundError, FileNotFoundError):
82
+ return {}
83
+ except ValueError as e:
84
+ # Wave 18 (R2, found by SESSION D): current huggingface_hub wraps a missing REPO in
85
+ # ValueError("Force download failed…") with the 404 as __cause__ — so a brand-new
86
+ # tenant repo's very first strict read raised here and aborted its very first write.
87
+ # A genuinely-absent repo is the same fact as a genuinely-absent file: {}.
88
+ cause = getattr(e, '__cause__', None)
89
+ if isinstance(cause, (EntryNotFoundError, RepositoryNotFoundError)):
90
+ return {}
91
+ raise
92
+
93
+ def get(self, name, fresh=False):
94
+ """Load {name}.json as a dict. LENIENT (for display reads): on a transient fetch error,
95
+ hand back the last known-good cached value if we have one, else {}. Mutating callers go
96
+ through update()/_read_strict, which RAISE instead."""
97
+ with self._lock:
98
+ if not fresh and name in self._cache:
99
+ return json.loads(json.dumps(self._cache[name])) # hand back a copy
100
+ try:
101
+ data = self._download(name)
102
+ except Exception as e:
103
+ try: # lazy import — core stays layer-clean
104
+ import harness.telemetry as _tel # noqa: PLC0415
105
+ _tel.error(f'store:get:{name}', e,
106
+ fallback='cached' if name in self._cache else 'empty')
107
+ except Exception:
108
+ pass
109
+ if name in self._cache:
110
+ return json.loads(json.dumps(self._cache[name]))
111
+ return {}
112
+ self._cache[name] = data
113
+ return json.loads(json.dumps(data))
114
+
115
+ def _read_strict(self, name):
116
+ """Like get(fresh=True) but RAISES on any transient read error ({} only for a genuinely
117
+ absent file), so a failed read ABORTS a read-modify-write instead of merging into {}."""
118
+ with self._lock:
119
+ data = self._download(name)
120
+ self._cache[name] = data
121
+ return json.loads(json.dumps(data))
122
+
123
+ def exists(self, name):
124
+ """True if {name}.json is confirmed present, False if confirmed absent, and True on ANY
125
+ uncertainty — a seeder must never clobber a registry we merely failed to reach."""
126
+ try:
127
+ files = HfApi(token=_token()).list_repo_files(self.repo, repo_type='dataset')
128
+ return f'{name}.json' in files
129
+ except RepositoryNotFoundError:
130
+ return False
131
+ except Exception:
132
+ return True
133
+
134
+ def _upload(self, name, data):
135
+ """The raw hub write. create_repo is confirmed ONCE per instance — it was a full extra
136
+ API round-trip on every put(), pure latency after the first success."""
137
+ api = HfApi(token=_token())
138
+ if not self._repo_ok:
139
+ api.create_repo(self.repo, repo_type='dataset', private=True, exist_ok=True)
140
+ self._repo_ok = True
141
+ buf = io.BytesIO(json.dumps(data, ensure_ascii=False, indent=2).encode('utf-8'))
142
+ api.upload_file(path_or_fileobj=buf, path_in_repo=f'{name}.json', repo_id=self.repo,
143
+ repo_type='dataset', commit_message=f'update {name}')
144
+
145
+ def upload_bytes(self, path_in_repo, data, message=None):
146
+ """Persist RAW BYTES at an arbitrary path in the tenant dataset (wave-8 C5, documents).
147
+ Bytes are never cached in-process: large, cold, fetched on explicit user action."""
148
+ if not self.available():
149
+ raise RuntimeError('No HF_TOKEN configured — persistence is unavailable.')
150
+ with self._lock:
151
+ api = HfApi(token=_token())
152
+ if not self._repo_ok:
153
+ api.create_repo(self.repo, repo_type='dataset', private=True, exist_ok=True)
154
+ self._repo_ok = True
155
+ api.upload_file(path_or_fileobj=io.BytesIO(data), path_in_repo=path_in_repo,
156
+ repo_id=self.repo, repo_type='dataset',
157
+ commit_message=message or f'add {path_in_repo}')
158
+
159
+ def download_bytes(self, path_in_repo):
160
+ """Raw bytes back, or None when absent/unreachable — a missing attachment degrades to
161
+ "that file is gone", never to a broken page."""
162
+ if not self.available():
163
+ return None
164
+ try:
165
+ local = hf_hub_download(self.repo, path_in_repo, repo_type='dataset',
166
+ token=_token())
167
+ return Path(local).read_bytes()
168
+ except (EntryNotFoundError, RepositoryNotFoundError):
169
+ return None
170
+ except Exception:
171
+ return None
172
+
173
+ def delete_path(self, path_in_repo):
174
+ """Remove a stored file. Absent is SUCCESS — deleting what is already gone is the goal."""
175
+ if not self.available():
176
+ return False
177
+ try:
178
+ HfApi(token=_token()).delete_file(path_in_repo=path_in_repo, repo_id=self.repo,
179
+ repo_type='dataset',
180
+ commit_message=f'delete {path_in_repo}')
181
+ return True
182
+ except (EntryNotFoundError, RepositoryNotFoundError):
183
+ return True
184
+ except Exception:
185
+ return False
186
+
187
+ def put(self, name, data):
188
+ """Persist {name}.json (creates the private repo on first write). Updates the cache."""
189
+ if not self.available():
190
+ raise RuntimeError('No HF_TOKEN configured — persistence is unavailable.')
191
+ with self._lock:
192
+ self._upload(name, data)
193
+ self._cache[name] = json.loads(json.dumps(data))
194
+ self._owned[name] = True
195
+ self._last_flush[name] = _time.monotonic()
196
+ return self._cache[name]
197
+
198
+ def _flush_worker(self, name):
199
+ """Coalescing uploader for one key. Runs while dirty; retries with backoff on failure —
200
+ the cache stays the truth and the dirty mark survives, so nothing is lost silently."""
201
+ backoff = 2.0
202
+ while True:
203
+ _time.sleep(_FLUSH_DELAY)
204
+ with self._lock:
205
+ gap = _time.monotonic() - self._last_flush.get(name, 0.0)
206
+ if gap < _FLUSH_MIN_GAP:
207
+ wait = _FLUSH_MIN_GAP - gap
208
+ elif not self._dirty.get(name):
209
+ self._flushers.pop(name, None)
210
+ return
211
+ else:
212
+ self._dirty[name] = False
213
+ snapshot = json.loads(json.dumps(self._cache.get(name, {})))
214
+ wait = None
215
+ if wait is not None:
216
+ _time.sleep(wait)
217
+ continue
218
+ try:
219
+ self._upload(name, snapshot)
220
+ with self._lock:
221
+ self._last_flush[name] = _time.monotonic()
222
+ backoff = 2.0
223
+ except Exception as e:
224
+ with self._lock:
225
+ self._dirty[name] = True # keep it queued; cache is still the truth
226
+ try:
227
+ import harness.telemetry as _tel # noqa: PLC0415
228
+ _tel.error(f'store:flush:{name}', e)
229
+ except Exception:
230
+ pass
231
+ _time.sleep(backoff)
232
+ backoff = min(backoff * 2, 60.0)
233
+
234
+ def _schedule_flush(self, name):
235
+ """Mark dirty + ensure exactly one live worker for the key. Caller holds the lock."""
236
+ self._dirty[name] = True
237
+ t = self._flushers.get(name)
238
+ if t is not None and t.is_alive():
239
+ return
240
+ t = threading.Thread(target=self._flush_worker, args=(name,), daemon=True,
241
+ name=f'store-flush-{self.repo}:{name}')
242
+ self._flushers[name] = t
243
+ t.start()
244
+
245
+ def flush(self, name=None, timeout=30.0):
246
+ """Block until pending async writes for `name` (or ALL keys) reach the hub — QA gates
247
+ and shutdown hooks use this; the app itself never needs to."""
248
+ deadline = _time.monotonic() + timeout
249
+ names = [name] if name else None
250
+ while _time.monotonic() < deadline:
251
+ with self._lock:
252
+ pending = {k for k, v in self._dirty.items() if v} | {
253
+ k for k, t in self._flushers.items() if t.is_alive()}
254
+ if names is not None:
255
+ pending &= set(names)
256
+ if not pending:
257
+ return True
258
+ _time.sleep(0.2)
259
+ return False
260
+
261
+ def _flush_now_at_exit(self):
262
+ """Process-exit safety net: push any dirty keys synchronously, bypassing the coalescing
263
+ gap — a Space restart must not eat the last seconds of the async window."""
264
+ with self._lock:
265
+ dirty = [k for k, v in self._dirty.items() if v]
266
+ snaps = {k: json.loads(json.dumps(self._cache.get(k, {}))) for k in dirty}
267
+ for k in dirty:
268
+ self._dirty[k] = False
269
+ for k, d in snaps.items():
270
+ try:
271
+ self._upload(k, d)
272
+ except Exception:
273
+ pass # exiting anyway — nothing left to reschedule
274
+
275
+ def update(self, name, fn, flush='sync'):
276
+ """Read-modify-write. STRICT first read (raises on a transient error rather than
277
+ starting from an empty dict), apply fn(data)->data, persist.
278
+
279
+ flush='sync' (default) = fresh strict read + blocking upload. flush='async' (the
280
+ table-workspace hot path, wave-7 W3) = the mutation applies to the in-process cache and
281
+ returns immediately; a coalescing worker commits. The strict read still guards the
282
+ FIRST async update of a key per process."""
283
+ with self._lock:
284
+ if flush == 'async' and self._owned.get(name) and name in self._cache:
285
+ data = json.loads(json.dumps(self._cache[name]))
286
+ else:
287
+ data = self._read_strict(name)
288
+ result = fn(data)
289
+ data = result if result is not None else data
290
+ if flush == 'async':
291
+ if not self.available():
292
+ raise RuntimeError('No HF_TOKEN configured — persistence is unavailable.')
293
+ self._cache[name] = json.loads(json.dumps(data))
294
+ self._owned[name] = True
295
+ self._schedule_flush(name)
296
+ return data
297
+ self.put(name, data)
298
+ return data
299
+
300
+
301
+ # ------------------------------------------------------------------- instances + module API
302
+ _INSTANCES = {}
303
+ _INSTANCES_LOCK = threading.RLock()
304
+
305
+
306
+ def for_repo(repo_id):
307
+ """The Store bound to `repo_id` — one instance per repo per process (each with its own
308
+ cache and flush state). The default instance IS `for_repo(REPO)`."""
309
+ rid = str(repo_id or '').strip() or REPO
310
+ with _INSTANCES_LOCK:
311
+ inst = _INSTANCES.get(rid)
312
+ if inst is None:
313
+ inst = Store(rid)
314
+ _INSTANCES[rid] = inst
315
+ return inst
316
+
317
+
318
+ _DEFAULT = for_repo(REPO)
319
+
320
+
321
+ def available():
322
+ return _DEFAULT.available()
323
+
324
+
325
+ def get(name, fresh=False):
326
+ return _DEFAULT.get(name, fresh=fresh)
327
+
328
+
329
+ def _read_strict(name):
330
+ return _DEFAULT._read_strict(name)
331
+
332
+
333
+ def exists(name):
334
+ return _DEFAULT.exists(name)
335
+
336
+
337
+ def upload_bytes(path_in_repo, data, message=None):
338
+ return _DEFAULT.upload_bytes(path_in_repo, data, message=message)
339
+
340
+
341
+ def download_bytes(path_in_repo):
342
+ return _DEFAULT.download_bytes(path_in_repo)
343
+
344
+
345
+ def delete_path(path_in_repo):
346
+ return _DEFAULT.delete_path(path_in_repo)
347
+
348
+
349
+ def put(name, data):
350
+ return _DEFAULT.put(name, data)
351
+
352
+
353
+ def flush(name=None, timeout=30.0):
354
+ return _DEFAULT.flush(name=name, timeout=timeout)
355
+
356
+
357
+ def update(name, fn, flush='sync'):
358
+ return _DEFAULT.update(name, fn, flush=flush)
359
+
360
+
361
+ def _flush_all_at_exit():
362
+ with _INSTANCES_LOCK:
363
+ instances = list(_INSTANCES.values())
364
+ for inst in instances:
365
+ inst._flush_now_at_exit()
366
+
367
+
368
+ import atexit # noqa: E402 (registered after the class it needs)
369
+ atexit.register(_flush_all_at_exit)
royalimports_os/core/table_store.py CHANGED
@@ -100,16 +100,24 @@ def _unique_name(wanted, existing, *, fallback='Untitled', max_len=120):
100
 
101
 
102
  class TableStore:
103
- """The six store operations for one table object's workspace, closed over its store key."""
104
 
105
- def __init__(self, table_key):
 
 
 
 
 
 
 
106
  self.table_key = table_key
 
107
 
108
  # ---------------------------------------------------------------- read
109
  def workspace(self, username, consume_corrections=True):
110
  """One user's durable workspace: always the full three-strata shape."""
111
  try:
112
- data = store.get(self.table_key) or {}
113
  ws = data.get(username, {}) or {}
114
  # A collision acknowledgement is protocol state, not part of a field definition.
115
  # Consume it with the first fresh workspace payload after the correcting write,
@@ -128,7 +136,7 @@ class TableStore:
128
  current_ws.pop('fieldCorrections', None)
129
  return current
130
 
131
- data = store.update(self.table_key, _take, flush='async')
132
  ws = (data or {}).get(username, {}) or {}
133
  except Exception:
134
  ws = {}
@@ -189,7 +197,7 @@ class TableStore:
189
  # the historical synchronous hub commit cost seconds per edit. The mutation applies to
190
  # the in-process cache (read-your-writes for every subsequent render); the hub write
191
  # coalesces in the background. Registry/auth writes elsewhere stay flush='sync'.
192
- return store.update(self.table_key, _up, flush='async')
193
 
194
  def save_active_view(self, username, view_id):
195
  """Remember which view this user last opened (owner item 3, 2026-07-31).
@@ -254,7 +262,7 @@ class TableStore:
254
  personal view by forgetting to filter.
255
  """
256
  try:
257
- bucket = ((store.get(self.table_key) or {}).get(SHARED_KEY) or {}).get('views') or {}
258
  except Exception:
259
  return {}
260
  return {vid: dict(v) for vid, v in bucket.items()
@@ -265,7 +273,7 @@ class TableStore:
265
  caller must know a view exists and who owns it before it can decide whether the actor
266
  may touch it. Never hand the result to a renderer without checking `_may_see`."""
267
  try:
268
- return ((store.get(self.table_key) or {}).get(SHARED_KEY) or {}
269
  ).get('views', {}).get(str(view_id))
270
  except Exception:
271
  return None
@@ -319,7 +327,7 @@ class TableStore:
319
  (data.get(SHARED_KEY, {}).get('views') or {}).pop(view_id, None)
320
  return data
321
 
322
- store.update(self.table_key, _up, flush='async')
323
  return dict(accepted)
324
 
325
  def delete_view(self, username, view_id):
@@ -335,7 +343,7 @@ class TableStore:
335
  (data.get(SHARED_KEY, {}).get('views') or {}).pop(vid, None)
336
  return data
337
 
338
- store.update(self.table_key, _up, flush='async')
339
 
340
  def save_field(self, username, field, reserved_names=(), correction_id=None):
341
  """Persist a column note or a user-created (custom_/measure_) field definition."""
@@ -467,5 +475,5 @@ class TableStore:
467
  self._update(username, _patch)
468
 
469
 
470
- def make(table_key):
471
- return TableStore(table_key)
 
100
 
101
 
102
  class TableStore:
103
+ """The six store operations for one table object's workspace, closed over its store key.
104
 
105
+ `st` (wave 18, C3-UT) is the STORE HANDLE — anything exposing `get(name)` /
106
+ `update(name, fn, flush=)`. Default = `core.store` (tenant #0, every existing caller,
107
+ zero behaviour change). The API passes the session's `TenantRuntime`, whose accessors
108
+ apply the tenant prefix / repo binding — which is what makes a user table created by a
109
+ Nurilab admin land in Nurilab's store instead of Royal's.
110
+ """
111
+
112
+ def __init__(self, table_key, st=None):
113
  self.table_key = table_key
114
+ self._st = st if st is not None else store
115
 
116
  # ---------------------------------------------------------------- read
117
  def workspace(self, username, consume_corrections=True):
118
  """One user's durable workspace: always the full three-strata shape."""
119
  try:
120
+ data = self._st.get(self.table_key) or {}
121
  ws = data.get(username, {}) or {}
122
  # A collision acknowledgement is protocol state, not part of a field definition.
123
  # Consume it with the first fresh workspace payload after the correcting write,
 
136
  current_ws.pop('fieldCorrections', None)
137
  return current
138
 
139
+ data = self._st.update(self.table_key, _take, flush='async')
140
  ws = (data or {}).get(username, {}) or {}
141
  except Exception:
142
  ws = {}
 
197
  # the historical synchronous hub commit cost seconds per edit. The mutation applies to
198
  # the in-process cache (read-your-writes for every subsequent render); the hub write
199
  # coalesces in the background. Registry/auth writes elsewhere stay flush='sync'.
200
+ return self._st.update(self.table_key, _up, flush='async')
201
 
202
  def save_active_view(self, username, view_id):
203
  """Remember which view this user last opened (owner item 3, 2026-07-31).
 
262
  personal view by forgetting to filter.
263
  """
264
  try:
265
+ bucket = ((self._st.get(self.table_key) or {}).get(SHARED_KEY) or {}).get('views') or {}
266
  except Exception:
267
  return {}
268
  return {vid: dict(v) for vid, v in bucket.items()
 
273
  caller must know a view exists and who owns it before it can decide whether the actor
274
  may touch it. Never hand the result to a renderer without checking `_may_see`."""
275
  try:
276
+ return ((self._st.get(self.table_key) or {}).get(SHARED_KEY) or {}
277
  ).get('views', {}).get(str(view_id))
278
  except Exception:
279
  return None
 
327
  (data.get(SHARED_KEY, {}).get('views') or {}).pop(view_id, None)
328
  return data
329
 
330
+ self._st.update(self.table_key, _up, flush='async')
331
  return dict(accepted)
332
 
333
  def delete_view(self, username, view_id):
 
343
  (data.get(SHARED_KEY, {}).get('views') or {}).pop(vid, None)
344
  return data
345
 
346
+ self._st.update(self.table_key, _up, flush='async')
347
 
348
  def save_field(self, username, field, reserved_names=(), correction_id=None):
349
  """Persist a column note or a user-created (custom_/measure_) field definition."""
 
475
  self._update(username, _patch)
476
 
477
 
478
+ def make(table_key, st=None):
479
+ return TableStore(table_key, st=st)
royalimports_os/core/user_tables.py CHANGED
@@ -20,9 +20,16 @@ SHAPE (one store key, `user_tables`):
20
  which is the same overlay shape `TableStore` already uses, so the grid's write path needs no new
21
  storage concept.
22
 
23
- ⚠ ROW WRITES ARE ONLY EVER LEGAL HERE. An Odoo-sourced table is READ-ONLY at the source and
24
- `_cl_handle_one`'s row_add / row_delete refuse unless `is_user_table(key)` — a user must not be
25
- able to invent a customer in Odoo by typing into a grid.
 
 
 
 
 
 
 
26
  """
27
  import datetime as _dt
28
  import re
@@ -33,36 +40,79 @@ STORE_KEY = 'user_tables'
33
  MAX_TABLES = 40
34
  MAX_LABEL = 60
35
  MAX_ROWS = 5000
 
36
  #: the badge the nav paints for a table with no connector behind it
37
  BLANK_SOURCE = 'Blank'
 
 
38
  #: every user table's key is prefixed, so it can never collide with a registry module key
39
  KEY_PREFIX = 'ut_'
40
 
 
 
 
 
 
 
 
 
 
 
 
 
41
 
42
  def _slug(label):
43
  s = re.sub(r'[^a-z0-9]+', '_', str(label or '').strip().lower()).strip('_')
44
  return (s or 'table')[:40]
45
 
46
 
47
- def all_tables():
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
48
  """{key: definition} — every user-created table in this tenant."""
49
  try:
50
- return dict(store.get(STORE_KEY) or {})
51
  except Exception:
52
  return {}
53
 
54
 
55
- def get(table_key):
56
- return all_tables().get(str(table_key))
57
 
58
 
59
- def is_user_table(table_key):
60
  """THE permission predicate for row writes. Cheap and total: a key that is not in this store
61
  is not a user table, so it cannot accept invented rows."""
62
- return str(table_key or '').startswith(KEY_PREFIX) and get(table_key) is not None
63
 
64
 
65
- def create(label, username, fields=None):
66
  """Create a blank table. Returns its key, or None if refused.
67
 
68
  Refuses rather than raises on the ordinary conditions (no name, duplicate, cap reached) —
@@ -71,7 +121,7 @@ def create(label, username, fields=None):
71
  label = str(label or '').strip()[:MAX_LABEL]
72
  if not label:
73
  return None
74
- existing = all_tables()
75
  if len(existing) >= MAX_TABLES:
76
  return None
77
  key = KEY_PREFIX + _slug(label)
@@ -88,11 +138,11 @@ def create(label, username, fields=None):
88
  defn = {
89
  'key': key,
90
  'label': label,
91
- 'source': BLANK_SOURCE,
92
  'createdBy': username,
93
  'created': _dt.datetime.now().strftime('%Y-%m-%dT%H:%M:%S'),
94
- 'fields': list(fields or [{'key': 'name', 'label': 'Name', 'type': 'text',
95
- 'source': 'overlay', 'default': True}]),
96
  'rows': {},
97
  }
98
 
@@ -100,28 +150,45 @@ def create(label, username, fields=None):
100
  cur[key] = defn
101
  return cur
102
 
103
- store.update(STORE_KEY, _add, flush='sync') # a creation is not a hot path; commit it
104
  return key
105
 
106
 
107
- def delete(table_key):
108
  def _drop(cur):
109
  cur.pop(str(table_key), None)
110
  return cur
111
- store.update(STORE_KEY, _drop, flush='sync')
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
112
 
113
 
114
- def add_row(table_key, values=None, username=None):
115
  """Append one row. Returns the new row id, or None if refused."""
116
- if not is_user_table(table_key):
117
  return None # ⚠ never on a connector-backed table
118
- defn = get(table_key) or {}
119
  rows = defn.get('rows') or {}
120
  if len(rows) >= MAX_ROWS:
121
  return None
122
  rid = str(max((int(r) for r in rows if str(r).isdigit()), default=0) + 1)
123
  valid = {f['key'] for f in (defn.get('fields') or [])}
124
- clean = {k: v for k, v in (values or {}).items() if k in valid}
125
 
126
  def _add(cur):
127
  t = cur.get(str(table_key))
@@ -129,12 +196,57 @@ def add_row(table_key, values=None, username=None):
129
  t.setdefault('rows', {})[rid] = clean
130
  return cur
131
 
132
- store.update(STORE_KEY, _add, flush='async')
133
  return rid
134
 
135
 
136
- def delete_row(table_key, row_id):
137
- if not is_user_table(table_key):
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
138
  return False
139
 
140
  def _drop(cur):
@@ -143,11 +255,11 @@ def delete_row(table_key, row_id):
143
  (t.get('rows') or {}).pop(str(row_id), None)
144
  return cur
145
 
146
- store.update(STORE_KEY, _drop, flush='async')
147
  return True
148
 
149
 
150
- def may_open(table_key, viewer, is_admin=False):
151
  """FAIL-CLOSED visibility: the creator, or an admin. Nothing else.
152
 
153
  A user table cannot be gated by `allowed_modules` — a module grant written last month cannot
@@ -157,21 +269,22 @@ def may_open(table_key, viewer, is_admin=False):
157
  Sharing a user table with named colleagues is a follow-on, and it should reuse the shared-
158
  VIEW vocabulary rather than inventing a second one.
159
  """
160
- t = get(table_key)
161
  if not t:
162
  return False
163
  return bool(is_admin) or t.get('createdBy') == viewer
164
 
165
 
166
- def nav_entries(viewer=None, is_admin=False):
167
  """Registry-SHAPED dicts so the nav can render user tables beside real modules with no
168
  special-casing: the same keys the flat nav reads (`key`, `label`, `source`).
169
 
170
  Filtered by `may_open`, so the nav cannot offer a row the dispatcher would refuse.
171
  """
172
  out = []
173
- for key, t in sorted(all_tables().items(), key=lambda kv: (kv[1].get('label') or '').lower()):
174
- if viewer is not None and not may_open(key, viewer, is_admin):
 
175
  continue
176
  out.append({'key': key, 'label': t.get('label') or key,
177
  'source': t.get('source') or BLANK_SOURCE,
 
20
  which is the same overlay shape `TableStore` already uses, so the grid's write path needs no new
21
  storage concept.
22
 
23
+ ⚠ ROW WRITES ARE ONLY EVER LEGAL HERE. An Odoo-sourced table is READ-ONLY at the source and the
24
+ row endpoints refuse unless `is_user_table(key)` — a user must not be able to invent a customer
25
+ in Odoo by typing into a grid. (Wave 18 note: the events seam has NO row event types; row
26
+ add/delete are REST endpoints on `aios-web/api/routes_tables.py`, and this predicate is their
27
+ wall.)
28
+
29
+ WAVE 18 (C3-UT): every function takes an optional `st` STORE HANDLE — anything exposing
30
+ `get(name)` / `update(name, fn, flush=)` / `exists(name)`. Default = `core.store` (tenant #0,
31
+ the Streamlit host, unchanged). The API passes the session's `TenantRuntime`, whose accessors
32
+ apply the tenant prefix / repo binding, so a Nurilab table lands in Nurilab's store.
33
  """
34
  import datetime as _dt
35
  import re
 
40
  MAX_TABLES = 40
41
  MAX_LABEL = 60
42
  MAX_ROWS = 5000
43
+ MAX_FIELDS = 60
44
  #: the badge the nav paints for a table with no connector behind it
45
  BLANK_SOURCE = 'Blank'
46
+ #: automation-created tables carry their maker instead (wave 18 C4-AUTO)
47
+ AUTOMATION_SOURCE = 'Automation'
48
  #: every user table's key is prefixed, so it can never collide with a registry module key
49
  KEY_PREFIX = 'ut_'
50
 
51
+ #: The field types a user table may declare. ⚠ Kept a SUBSET of
52
+ #: `aios_grid.CUSTOM_FIELD_TYPES` (gated in verify_api's W18-UT section) — minus the two
53
+ #: machine-written kinds (`formula` computes client-side over a workspace stratum; a base
54
+ #: column of that type would have no author). Local literal rather than an aios_grid import so
55
+ #: this module stays dependency-light for the API's boot path.
56
+ UT_FIELD_TYPES = {'text', 'select', 'multiselect', 'user', 'int', 'currency', 'pct', 'date',
57
+ 'checkbox', 'phone', 'email', 'url', 'rating'}
58
+
59
+
60
+ def _st(st):
61
+ return st if st is not None else store
62
+
63
 
64
  def _slug(label):
65
  s = re.sub(r'[^a-z0-9]+', '_', str(label or '').strip().lower()).strip('_')
66
  return (s or 'table')[:40]
67
 
68
 
69
+ def clean_fields(raw):
70
+ """Validate a caller-supplied field list into the stored shape — refuse junk, never store
71
+ it. Returns a list of `{key,label,type,source}` dicts (source is always 'overlay': every
72
+ base column of a user table is user-editable by construction), or None when nothing
73
+ survives. Unknown types are DROPPED, not coerced — a column silently retyped is a lie."""
74
+ out, seen = [], set()
75
+ for f in (raw or [])[:MAX_FIELDS]:
76
+ if not isinstance(f, dict):
77
+ continue
78
+ label = ' '.join(str(f.get('label') or f.get('key') or '').split())[:80]
79
+ key = re.sub(r'[^a-z0-9_]+', '_', str(f.get('key') or _slug(label)).strip().lower())
80
+ key = key.strip('_')[:60]
81
+ ftype = str(f.get('type') or 'text').strip().lower()
82
+ if not key or key in seen or ftype not in UT_FIELD_TYPES or not label:
83
+ continue
84
+ seen.add(key)
85
+ entry = {'key': key, 'label': label, 'type': ftype, 'source': 'overlay'}
86
+ if ftype in ('select', 'multiselect'):
87
+ opts = [' '.join(str(o).split())[:60] for o in (f.get('options') or [])
88
+ if str(o).strip()][:50]
89
+ if opts:
90
+ entry['options'] = opts
91
+ if f.get('default') is True:
92
+ entry['default'] = True
93
+ out.append(entry)
94
+ return out or None
95
+
96
+
97
+ def all_tables(st=None):
98
  """{key: definition} — every user-created table in this tenant."""
99
  try:
100
+ return dict(_st(st).get(STORE_KEY) or {})
101
  except Exception:
102
  return {}
103
 
104
 
105
+ def get(table_key, st=None):
106
+ return all_tables(st).get(str(table_key))
107
 
108
 
109
+ def is_user_table(table_key, st=None):
110
  """THE permission predicate for row writes. Cheap and total: a key that is not in this store
111
  is not a user table, so it cannot accept invented rows."""
112
+ return str(table_key or '').startswith(KEY_PREFIX) and get(table_key, st) is not None
113
 
114
 
115
+ def create(label, username, fields=None, source=None, st=None):
116
  """Create a blank table. Returns its key, or None if refused.
117
 
118
  Refuses rather than raises on the ordinary conditions (no name, duplicate, cap reached) —
 
121
  label = str(label or '').strip()[:MAX_LABEL]
122
  if not label:
123
  return None
124
+ existing = all_tables(st)
125
  if len(existing) >= MAX_TABLES:
126
  return None
127
  key = KEY_PREFIX + _slug(label)
 
138
  defn = {
139
  'key': key,
140
  'label': label,
141
+ 'source': (source if source in (BLANK_SOURCE, AUTOMATION_SOURCE) else BLANK_SOURCE),
142
  'createdBy': username,
143
  'created': _dt.datetime.now().strftime('%Y-%m-%dT%H:%M:%S'),
144
+ 'fields': clean_fields(fields) or [{'key': 'name', 'label': 'Name', 'type': 'text',
145
+ 'source': 'overlay', 'default': True}],
146
  'rows': {},
147
  }
148
 
 
150
  cur[key] = defn
151
  return cur
152
 
153
+ _st(st).update(STORE_KEY, _add, flush='sync') # a creation is not a hot path; commit it
154
  return key
155
 
156
 
157
+ def delete(table_key, st=None):
158
  def _drop(cur):
159
  cur.pop(str(table_key), None)
160
  return cur
161
+ _st(st).update(STORE_KEY, _drop, flush='sync')
162
+
163
+
164
+ def set_fields(table_key, fields, st=None):
165
+ """Replace a table's base-field contract (wave 18 — the automation engine grows tables it
166
+ creates). Cleaned with the same validator as `create`; refuses to leave a table fieldless."""
167
+ clean = clean_fields(fields)
168
+ if not clean or not is_user_table(table_key, st):
169
+ return False
170
+
171
+ def _set(cur):
172
+ t = cur.get(str(table_key))
173
+ if t is not None:
174
+ t['fields'] = clean
175
+ return cur
176
+
177
+ _st(st).update(STORE_KEY, _set, flush='sync')
178
+ return True
179
 
180
 
181
+ def add_row(table_key, values=None, username=None, st=None):
182
  """Append one row. Returns the new row id, or None if refused."""
183
+ if not is_user_table(table_key, st):
184
  return None # ⚠ never on a connector-backed table
185
+ defn = get(table_key, st) or {}
186
  rows = defn.get('rows') or {}
187
  if len(rows) >= MAX_ROWS:
188
  return None
189
  rid = str(max((int(r) for r in rows if str(r).isdigit()), default=0) + 1)
190
  valid = {f['key'] for f in (defn.get('fields') or [])}
191
+ clean = {k: str(v) for k, v in (values or {}).items() if k in valid}
192
 
193
  def _add(cur):
194
  t = cur.get(str(table_key))
 
196
  t.setdefault('rows', {})[rid] = clean
197
  return cur
198
 
199
+ _st(st).update(STORE_KEY, _add, flush='async')
200
  return rid
201
 
202
 
203
+ def upsert_rows(table_key, rows_by_key, key_field, username=None, st=None):
204
+ """Bulk UPSERT for the automation engine (wave 18 C4-AUTO): match on `key_field`'s value,
205
+ update matched rows, insert the rest. Returns `{'updated': n, 'inserted': n, 'orphans': n}`
206
+ where orphans = existing rows whose key value did not appear this run — COUNTED, never
207
+ deleted (the contract's no-silent-delete rule). Refuses on a non-user table."""
208
+ if not is_user_table(table_key, st):
209
+ return None
210
+ defn = get(table_key, st) or {}
211
+ valid = {f['key'] for f in (defn.get('fields') or [])}
212
+ if key_field not in valid:
213
+ return None
214
+ incoming = {}
215
+ for kv, vals in (rows_by_key or {}).items():
216
+ kv = str(kv).strip()
217
+ if kv:
218
+ incoming[kv] = {k: str(v) for k, v in (vals or {}).items() if k in valid}
219
+ counts = {'updated': 0, 'inserted': 0, 'orphans': 0}
220
+
221
+ def _apply(cur):
222
+ t = cur.get(str(table_key))
223
+ if t is None:
224
+ return cur
225
+ rows = t.setdefault('rows', {})
226
+ by_val = {}
227
+ for rid, row in rows.items():
228
+ v = str((row or {}).get(key_field) or '').strip()
229
+ if v and v not in by_val:
230
+ by_val[v] = rid
231
+ next_id = max((int(r) for r in rows if str(r).isdigit()), default=0)
232
+ for kv, vals in incoming.items():
233
+ rid = by_val.get(kv)
234
+ if rid is not None:
235
+ rows[rid].update(vals)
236
+ counts['updated'] += 1
237
+ elif len(rows) < MAX_ROWS:
238
+ next_id += 1
239
+ rows[str(next_id)] = vals
240
+ counts['inserted'] += 1
241
+ counts['orphans'] = sum(1 for v in by_val if v not in incoming)
242
+ return cur
243
+
244
+ _st(st).update(STORE_KEY, _apply, flush='sync')
245
+ return counts
246
+
247
+
248
+ def delete_row(table_key, row_id, st=None):
249
+ if not is_user_table(table_key, st):
250
  return False
251
 
252
  def _drop(cur):
 
255
  (t.get('rows') or {}).pop(str(row_id), None)
256
  return cur
257
 
258
+ _st(st).update(STORE_KEY, _drop, flush='async')
259
  return True
260
 
261
 
262
+ def may_open(table_key, viewer, is_admin=False, st=None):
263
  """FAIL-CLOSED visibility: the creator, or an admin. Nothing else.
264
 
265
  A user table cannot be gated by `allowed_modules` — a module grant written last month cannot
 
269
  Sharing a user table with named colleagues is a follow-on, and it should reuse the shared-
270
  VIEW vocabulary rather than inventing a second one.
271
  """
272
+ t = get(table_key, st)
273
  if not t:
274
  return False
275
  return bool(is_admin) or t.get('createdBy') == viewer
276
 
277
 
278
+ def nav_entries(viewer=None, is_admin=False, st=None):
279
  """Registry-SHAPED dicts so the nav can render user tables beside real modules with no
280
  special-casing: the same keys the flat nav reads (`key`, `label`, `source`).
281
 
282
  Filtered by `may_open`, so the nav cannot offer a row the dispatcher would refuse.
283
  """
284
  out = []
285
+ for key, t in sorted(all_tables(st).items(),
286
+ key=lambda kv: (kv[1].get('label') or '').lower()):
287
+ if viewer is not None and not may_open(key, viewer, is_admin, st=st):
288
  continue
289
  out.append({'key': key, 'label': t.get('label') or key,
290
  'source': t.get('source') or BLANK_SOURCE,
royalimports_os/core/users.py CHANGED
@@ -27,11 +27,15 @@ def _hash(pw, salt):
27
 
28
 
29
  def _record(pw, name, role, bus, active=True, modules='all', agent=None, email=None,
30
- perms=None):
31
  salt = secrets.token_hex(16)
32
  rec = {'salt': salt, 'hash': _hash(pw, salt), 'name': name, 'role': role,
33
  'bus': bus, 'active': active, 'modules': modules,
34
- 'agent': agent or None, 'email': email or None}
 
 
 
 
35
  if perms is not None:
36
  # Wave 15 C-PERM. A record written WITH perms is migrated by construction — the marker
37
  # and the block are set together, here, so no writer can create one without the other.
@@ -78,6 +82,9 @@ def _public(username, u):
78
  'role': u.get('role', 'user'), 'bus': u.get('bus', 'all'),
79
  'modules': u.get('modules', 'all'),
80
  'agent': u.get('agent'), 'email': u.get('email'),
 
 
 
81
  # Wave 14 C-AVATAR: the profile photo is public-safe by definition (it is served
82
  # to every grid session via the workspace map); without it here the API session's
83
  # user record silently drops it and /me can never show your own photo.
@@ -143,6 +150,14 @@ def verify(username, pw):
143
  except Exception:
144
  reg = {}
145
  u = reg.get(username)
 
 
 
 
 
 
 
 
146
  if u and u.get('active', True) and hmac.compare_digest(_hash(pw, u['salt']), u['hash']):
147
  return _public(username, u)
148
  # emergency master: admin + APP_PASSWORD always works (covers first run / store outage)
@@ -165,7 +180,7 @@ def verify(username, pw):
165
 
166
 
167
  def create_user(username, pw, name, role='user', bus='all', modules='all',
168
- agent=None, email=None):
169
  """Create — or, from app.py's dialog, OVERWRITE — an account.
170
 
171
  ⛔ X3: OVERWRITING AN ACCOUNT MUST NOT RESURRECT ITS OLD SESSIONS. `_record()` builds a fresh
@@ -185,7 +200,10 @@ def create_user(username, pw, name, role='user', bus='all', modules='all',
185
  def _add(reg):
186
  prior = reg.get(username) or {}
187
  rec = _record(pw, name or username, role, bus, modules=modules,
188
- agent=agent, email=email)
 
 
 
189
  if prior:
190
  rec['epoch'] = int(prior.get('epoch') or 0) + 1
191
  reg[username] = rec
@@ -273,22 +291,29 @@ def allowed_bus_labels(user):
273
  return (['All'] + labels) if len(labels) > 1 else labels
274
 
275
 
276
- def assignable_people():
277
  """Display names for `user`-typed overlay columns — the tenant's ACTIVE accounts.
278
 
279
  Moved from app.py (2026-07-31) so both hosts serve the same choices. Resolved on every
280
  call rather than persisted with the column: a snapshot would keep offering people who
281
  have left and never offer people who joined. Deactivated accounts are excluded; a value
282
  already stored on a row is untouched — history should still say who owned something.
 
 
 
 
283
  """
284
  try:
285
  reg = registry() or {}
286
  except Exception:
287
  return []
 
288
  out = []
289
  for username, u in reg.items():
290
  if not isinstance(u, dict) or u.get('active') is False:
291
  continue
 
 
292
  out.append(str(u.get('name') or username))
293
  return sorted(set(out))
294
 
@@ -311,20 +336,24 @@ def set_avatar(username, data_url):
311
  store.update('users', _set)
312
 
313
 
314
- def avatar_map():
315
  """{display name -> avatar data URL} for ACTIVE accounts with a photo — the companion of
316
  `assignable_people()`, keyed by the SAME vocabulary: a `user` cell stores the display
317
  name, so the display name is the only join a renderer has. Two active accounts sharing a
318
  display name share one option; the first WITH a photo wins the key rather than a coin
319
- flip deciding whether the option has a face."""
 
320
  try:
321
  reg = registry() or {}
322
  except Exception:
323
  return {}
 
324
  out = {}
325
  for username, u in sorted(reg.items()):
326
  if not isinstance(u, dict) or u.get('active') is False:
327
  continue
 
 
328
  av = u.get('avatar')
329
  nm = str(u.get('name') or username)
330
  if av and nm not in out:
 
27
 
28
 
29
  def _record(pw, name, role, bus, active=True, modules='all', agent=None, email=None,
30
+ perms=None, tenant='royal-imports'):
31
  salt = secrets.token_hex(16)
32
  rec = {'salt': salt, 'hash': _hash(pw, salt), 'name': name, 'role': role,
33
  'bus': bus, 'active': active, 'modules': modules,
34
+ 'agent': agent or None, 'email': email or None,
35
+ # Wave 18 (C1-TENANT, R1): the account's COMPANY. Absent == 'royal-imports' on every
36
+ # pre-wave record — no migration. Login binds the session to THIS value; the posted
37
+ # tenant field can hint but never override it.
38
+ 'tenant': str(tenant or 'royal-imports').strip().lower()}
39
  if perms is not None:
40
  # Wave 15 C-PERM. A record written WITH perms is migrated by construction — the marker
41
  # and the block are set together, here, so no writer can create one without the other.
 
82
  'role': u.get('role', 'user'), 'bus': u.get('bus', 'all'),
83
  'modules': u.get('modules', 'all'),
84
  'agent': u.get('agent'), 'email': u.get('email'),
85
+ # Wave 18 (C1-TENANT): the session's tenant binding travels on the projection or it
86
+ # does not travel — the same rule the perms block states below.
87
+ 'tenant': str(u.get('tenant') or 'royal-imports').strip().lower(),
88
  # Wave 14 C-AVATAR: the profile photo is public-safe by definition (it is served
89
  # to every grid session via the workspace map); without it here the API session's
90
  # user record silently drops it and /me can never show your own photo.
 
150
  except Exception:
151
  reg = {}
152
  u = reg.get(username)
153
+ if u is None and '@' in username:
154
+ # Wave 18 (R1): the login box takes a username OR an email — admin@nurilab.id signs in
155
+ # without knowing the slug an admin chose. First case-insensitive email match wins;
156
+ # ambiguity is an admin data problem, not a login feature.
157
+ for k, r in reg.items():
158
+ if isinstance(r, dict) and str(r.get('email') or '').strip().lower() == username:
159
+ username, u = k, r
160
+ break
161
  if u and u.get('active', True) and hmac.compare_digest(_hash(pw, u['salt']), u['hash']):
162
  return _public(username, u)
163
  # emergency master: admin + APP_PASSWORD always works (covers first run / store outage)
 
180
 
181
 
182
  def create_user(username, pw, name, role='user', bus='all', modules='all',
183
+ agent=None, email=None, tenant=None):
184
  """Create — or, from app.py's dialog, OVERWRITE — an account.
185
 
186
  ⛔ X3: OVERWRITING AN ACCOUNT MUST NOT RESURRECT ITS OLD SESSIONS. `_record()` builds a fresh
 
200
  def _add(reg):
201
  prior = reg.get(username) or {}
202
  rec = _record(pw, name or username, role, bus, modules=modules,
203
+ agent=agent, email=email,
204
+ # An overwrite that names no tenant KEEPS the account's company — a
205
+ # rename must never quietly move a user between tenants.
206
+ tenant=(tenant or prior.get('tenant') or 'royal-imports'))
207
  if prior:
208
  rec['epoch'] = int(prior.get('epoch') or 0) + 1
209
  reg[username] = rec
 
291
  return (['All'] + labels) if len(labels) > 1 else labels
292
 
293
 
294
+ def assignable_people(tenant=None):
295
  """Display names for `user`-typed overlay columns — the tenant's ACTIVE accounts.
296
 
297
  Moved from app.py (2026-07-31) so both hosts serve the same choices. Resolved on every
298
  call rather than persisted with the column: a snapshot would keep offering people who
299
  have left and never offer people who joined. Deactivated accounts are excluded; a value
300
  already stored on a row is untouched — history should still say who owned something.
301
+
302
+ Wave 18 (C1-TENANT): pass `tenant` to scope the choices to ONE company — the user registry
303
+ is a global control-plane bucket, and a Nurilab picker offering Royal's staff is a
304
+ cross-tenant name leak. None = unscoped (the Streamlit host, tenant #0's process).
305
  """
306
  try:
307
  reg = registry() or {}
308
  except Exception:
309
  return []
310
+ want = str(tenant or '').strip().lower()
311
  out = []
312
  for username, u in reg.items():
313
  if not isinstance(u, dict) or u.get('active') is False:
314
  continue
315
+ if want and str(u.get('tenant') or 'royal-imports').strip().lower() != want:
316
+ continue
317
  out.append(str(u.get('name') or username))
318
  return sorted(set(out))
319
 
 
336
  store.update('users', _set)
337
 
338
 
339
+ def avatar_map(tenant=None):
340
  """{display name -> avatar data URL} for ACTIVE accounts with a photo — the companion of
341
  `assignable_people()`, keyed by the SAME vocabulary: a `user` cell stores the display
342
  name, so the display name is the only join a renderer has. Two active accounts sharing a
343
  display name share one option; the first WITH a photo wins the key rather than a coin
344
+ flip deciding whether the option has a face. `tenant` scopes it exactly as
345
+ `assignable_people(tenant)` does, and for the same leak."""
346
  try:
347
  reg = registry() or {}
348
  except Exception:
349
  return {}
350
+ want = str(tenant or '').strip().lower()
351
  out = {}
352
  for username, u in sorted(reg.items()):
353
  if not isinstance(u, dict) or u.get('active') is False:
354
  continue
355
+ if want and str(u.get('tenant') or 'royal-imports').strip().lower() != want:
356
+ continue
357
  av = u.get('avatar')
358
  nm = str(u.get('name') or username)
359
  if av and nm not in out:
royalimports_os/harness/runtime.py CHANGED
@@ -96,24 +96,34 @@ class TenantRuntime:
96
  return _ds.path_for(self.key)
97
 
98
  # ---------------------------------------------------------------- the store handle
 
 
 
 
 
 
 
 
 
 
99
  def store_key(self, name):
100
  """Namespace a product-data key for this tenant. The ONE place the prefix is applied."""
101
  return f"{self.store_namespace}{name}" if self.store_namespace else str(name)
102
 
103
  def get(self, name, fresh=False):
104
- return store.get(self.store_key(name), fresh=fresh)
105
 
106
  def put(self, name, data):
107
- return store.put(self.store_key(name), data)
108
 
109
  def update(self, name, fn, flush="sync"):
110
- return store.update(self.store_key(name), fn, flush=flush)
111
 
112
  def exists(self, name):
113
- return store.exists(self.store_key(name))
114
 
115
  def available(self):
116
- return store.available()
117
 
118
  def assert_datastore_matches(self):
119
  """RAISE if the process-wide DuckDB file is not this tenant's.
@@ -140,7 +150,39 @@ def register(key, builder):
140
 
141
 
142
  def known_tenants():
143
- return sorted(_BUILDERS)
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
144
 
145
 
146
  def get_runtime(tenant_key):
@@ -159,8 +201,14 @@ def get_runtime(tenant_key):
159
  _CACHE.move_to_end(key) # LRU: touch on read
160
  return rt
161
  builder = _BUILDERS.get(key)
162
- if builder is None:
 
 
 
 
 
163
  raise KeyError(key)
 
164
  # Built OUTSIDE the lock: a builder may do real work (a connector handshake), and holding the
165
  # registry lock through it would serialise every other tenant's requests behind it.
166
  tenant = builder()
@@ -169,8 +217,14 @@ def get_runtime(tenant_key):
169
  if existing is not None: # another thread won the race — keep ONE runtime
170
  _CACHE.move_to_end(key)
171
  return existing
172
- rt = TenantRuntime(key=key, tenant=tenant,
173
- store_namespace="" if key == "royal-imports" else f"t/{key}/")
 
 
 
 
 
 
174
  _CACHE[key] = rt
175
  while len(_CACHE) > _MAX_RUNTIMES:
176
  _CACHE.popitem(last=False) # evict least recently used
 
96
  return _ds.path_for(self.key)
97
 
98
  # ---------------------------------------------------------------- the store handle
99
+ #: Wave 18 (C1-TENANT, R2): a tenant that OWNS its dataset repo carries its bound Store
100
+ #: instance here (namespace stays EMPTY — a whole repo needs no prefix). None = the shared
101
+ #: default repo through the MODULE functions, which is what keeps the gates' fake-store
102
+ #: installation effective (the fake patches module attributes, and royal-imports + the
103
+ #: prefix tenants resolve through exactly those).
104
+ store_handle: object = None
105
+
106
+ def _store(self):
107
+ return self.store_handle if self.store_handle is not None else store
108
+
109
  def store_key(self, name):
110
  """Namespace a product-data key for this tenant. The ONE place the prefix is applied."""
111
  return f"{self.store_namespace}{name}" if self.store_namespace else str(name)
112
 
113
  def get(self, name, fresh=False):
114
+ return self._store().get(self.store_key(name), fresh=fresh)
115
 
116
  def put(self, name, data):
117
+ return self._store().put(self.store_key(name), data)
118
 
119
  def update(self, name, fn, flush="sync"):
120
+ return self._store().update(self.store_key(name), fn, flush=flush)
121
 
122
  def exists(self, name):
123
+ return self._store().exists(self.store_key(name))
124
 
125
  def available(self):
126
+ return self._store().available()
127
 
128
  def assert_datastore_matches(self):
129
  """RAISE if the process-wide DuckDB file is not this tenant's.
 
150
 
151
 
152
  def known_tenants():
153
+ keys = set(_BUILDERS)
154
+ try:
155
+ keys |= set(_tenant_records())
156
+ except Exception:
157
+ pass
158
+ return sorted(keys)
159
+
160
+
161
+ #: Wave 18 (C1-TENANT) — the CONTROL-PLANE bucket: tenants provisioned at runtime rather than
162
+ #: compiled in. Lives in the DEFAULT store (tenant #0's repo) deliberately: it answers "which
163
+ #: tenants exist", which is a platform fact, not any one tenant's data.
164
+ TENANTS_KEY = "tenants"
165
+
166
+
167
+ def _tenant_records():
168
+ """{slug: record} from the control-plane bucket. {} on any failure — an unreachable bucket
169
+ must degrade to "only the compiled-in tenants exist", never to an exception at login."""
170
+ try:
171
+ recs = store.get(TENANTS_KEY) or {}
172
+ return {str(k).strip().lower(): v for k, v in recs.items() if isinstance(v, dict)}
173
+ except Exception:
174
+ return {}
175
+
176
+
177
+ def _builder_from_record(key, rec):
178
+ """A Tenant for a bucket-provisioned client: no sources yet (connectors arrive via the
179
+ Keychain flow), config carries the platform fields the request path reads."""
180
+ def _build():
181
+ from harness.base import Tenant
182
+ return Tenant(key=key, name=str(rec.get("name") or key),
183
+ config={"modules": rec.get("modules", []),
184
+ "store_repo": rec.get("store_repo") or None})
185
+ return _build
186
 
187
 
188
  def get_runtime(tenant_key):
 
201
  _CACHE.move_to_end(key) # LRU: touch on read
202
  return rt
203
  builder = _BUILDERS.get(key)
204
+ # Wave 18 (C1-TENANT): a slug the compiled registry does not know may be a PROVISIONED
205
+ # tenant — consult the control-plane bucket before failing closed. A suspended record is
206
+ # the same KeyError as an unknown one: "which tenants exist" is not this caller's question.
207
+ if builder is None:
208
+ rec = _tenant_records().get(key)
209
+ if not rec or rec.get("status", "active") != "active":
210
  raise KeyError(key)
211
+ builder = _builder_from_record(key, rec)
212
  # Built OUTSIDE the lock: a builder may do real work (a connector handshake), and holding the
213
  # registry lock through it would serialise every other tenant's requests behind it.
214
  tenant = builder()
 
217
  if existing is not None: # another thread won the race — keep ONE runtime
218
  _CACHE.move_to_end(key)
219
  return existing
220
+ # R2: a tenant with its OWN repo gets a bound Store instance and an EMPTY prefix (the
221
+ # repo boundary is the namespace); everyone else keeps the t/<slug>/ prefix in the
222
+ # shared repo. Tenant #0 keeps both defaults — empty prefix, shared repo.
223
+ repo = (getattr(tenant, "config", {}) or {}).get("store_repo")
224
+ rt = TenantRuntime(
225
+ key=key, tenant=tenant,
226
+ store_namespace=("" if (key == "royal-imports" or repo) else f"t/{key}/"),
227
+ store_handle=(store.for_repo(repo) if repo else None))
228
  _CACHE[key] = rt
229
  while len(_CACHE) > _MAX_RUNTIMES:
230
  _CACHE.popitem(last=False) # evict least recently used
web/dist-embed/index.html CHANGED
The diff for this file is too large to render. See raw diff
 
web/src/apiContract.ts CHANGED
@@ -76,6 +76,12 @@ export const WORKSPACE_STALE_EVENT = "aios:workspace-stale";
76
  * which is exactly the host-neutral contract the other signals follow. */
77
  export const NAV_MINIMIZE_EVENT = "aios:nav-minimize";
78
 
 
 
 
 
 
 
79
  export function signal(name: string, detail?: unknown): void {
80
  if (typeof window === "undefined") return;
81
  window.dispatchEvent(new CustomEvent(name, { detail }));
 
76
  * which is exactly the host-neutral contract the other signals follow. */
77
  export const NAV_MINIMIZE_EVENT = "aios:nav-minimize";
78
 
79
+ /** Wave 18 (C3-UT): rows changed OUTSIDE the grid's own write path — a shell "Add record",
80
+ * an automation run — and the current topic's rows should be refetched. The grid clears its
81
+ * rows cache and reloads; senders call `clearCustomersCache()` first so the refetch cannot
82
+ * be served from the 5-minute memo. */
83
+ export const ROWS_STALE_EVENT = "aios:rows-stale";
84
+
85
  export function signal(name: string, detail?: unknown): void {
86
  if (typeof window === "undefined") return;
87
  window.dispatchEvent(new CustomEvent(name, { detail }));
web/src/automation/AutomationEditor.tsx ADDED
@@ -0,0 +1,662 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ // ---------------------------------------------------------------------------
2
+ // automation/AutomationEditor.tsx — one automation: what it does, when it runs,
3
+ // and what it did last time.
4
+ //
5
+ // THE SHAPE OF THE SCRAPE FLOW IS THE POINT. You cannot map columns you have not
6
+ // seen, so "Read the page" comes BEFORE the field map and the preview is real
7
+ // rows off the real URL — not a schema the user is asked to describe from
8
+ // memory. The server's `/automations/preview` never writes, so the read is free
9
+ // to be taken as many times as it takes to get the mapping right.
10
+ //
11
+ // EVERY COUNT DRILLS TO ROWS. A run that says "412 updated" opens the 412
12
+ // ([[no-unverifiable-aggregates]]) — a number with no way back to its records is
13
+ // exactly what this codebase treats as a defect rather than a summary.
14
+ // ---------------------------------------------------------------------------
15
+ import { useCallback, useEffect, useMemo, useState } from "react";
16
+
17
+ import type {
18
+ Automation,
19
+ AutomationKind,
20
+ RunEntry,
21
+ RunRows,
22
+ SourcePreview,
23
+ UserTable,
24
+ } from "./automationApi";
25
+ import {
26
+ AutomationError,
27
+ COUNT_LABELS,
28
+ createAutomation,
29
+ deleteAutomation,
30
+ fieldKeyFor,
31
+ listTables,
32
+ patchAutomation,
33
+ previewSource,
34
+ runAutomation,
35
+ runRows,
36
+ } from "./automationApi";
37
+
38
+ interface Props {
39
+ automation: Automation | null;
40
+ kinds: { key: AutomationKind; label: string }[];
41
+ cronPresets: { cron: string; label: string }[];
42
+ onSaved: (id?: string) => void | Promise<void>;
43
+ onDeleted: () => void | Promise<void>;
44
+ onCancel: () => void;
45
+ }
46
+
47
+ interface ColumnPlan {
48
+ column: string;
49
+ include: boolean;
50
+ key: string;
51
+ }
52
+
53
+ const DEFAULT_CRON = "0 6 * * *";
54
+
55
+ export default function AutomationEditor({
56
+ automation,
57
+ kinds,
58
+ cronPresets,
59
+ onSaved,
60
+ onDeleted,
61
+ onCancel,
62
+ }: Props) {
63
+ const isNew = !automation;
64
+ const cfg = (automation?.config || {}) as Record<string, string | number>;
65
+
66
+ const [name, setName] = useState(automation?.name || "");
67
+ const [kind, setKind] = useState<AutomationKind>(automation?.kind || "scrape_db");
68
+ const [cron, setCron] = useState(automation?.schedule?.cron || DEFAULT_CRON);
69
+ const [enabled, setEnabled] = useState(!!automation?.schedule?.enabled);
70
+ const [message, setMessage] = useState("");
71
+ const [problem, setProblem] = useState("");
72
+ const [saving, setSaving] = useState(false);
73
+
74
+ // --- scrape_db half
75
+ const [url, setUrl] = useState(String(cfg.url || ""));
76
+ const [extract, setExtract] = useState(String(cfg.extract || "table"));
77
+ const [tableIndex, setTableIndex] = useState(Number(cfg.tableIndex || 0));
78
+ const [targetLabel, setTargetLabel] = useState(String(cfg.targetLabel || ""));
79
+ const [preview, setPreview] = useState<SourcePreview | null>(null);
80
+ const [reading, setReading] = useState(false);
81
+ const [plan, setPlan] = useState<ColumnPlan[]>(() =>
82
+ Object.entries((automation?.config as { fieldMap?: Record<string, string> })?.fieldMap || {})
83
+ .map(([column, key]) => ({ column, include: true, key }))
84
+ );
85
+ const [keyField, setKeyField] = useState(String(cfg.keyField || ""));
86
+
87
+ // --- field_instagram half
88
+ const [tables, setTables] = useState<UserTable[]>([]);
89
+ const [targetTable, setTargetTable] = useState(String(cfg.targetTable || ""));
90
+ const [fieldKey, setFieldKey] = useState(String(cfg.fieldKey || ""));
91
+ const [urlField, setUrlField] = useState(String(cfg.urlField || ""));
92
+ const [maxPosts, setMaxPosts] = useState(Number(cfg.maxPosts || 24));
93
+
94
+ // --- run history drill
95
+ const [drill, setDrill] = useState<RunRows | null>(null);
96
+ const [drillFor, setDrillFor] = useState("");
97
+
98
+ useEffect(() => {
99
+ const ac = new AbortController();
100
+ listTables(ac.signal)
101
+ .then((r) => setTables(r.tables))
102
+ .catch(() => setTables([]));
103
+ return () => ac.abort();
104
+ }, []);
105
+
106
+ const table = tables.find((t) => t.key === targetTable) || null;
107
+
108
+ const readPage = useCallback(async () => {
109
+ setReading(true);
110
+ setProblem("");
111
+ try {
112
+ const p = await previewSource(url, extract, tableIndex);
113
+ setPreview(p);
114
+ if (!p.ok) {
115
+ setProblem(p.note || `That page answered ${p.status}.`);
116
+ return;
117
+ }
118
+ // Seed the plan from what the page ACTUALLY offers, keeping any mapping the user
119
+ // already made for a column that is still there — re-reading a page must not throw
120
+ // away the work of mapping it.
121
+ setPlan((prev) => {
122
+ const had = new Map(prev.map((c) => [c.column, c]));
123
+ return p.columns.map((column) => {
124
+ const before = had.get(column);
125
+ return {
126
+ column,
127
+ include: before ? before.include : true,
128
+ key: before ? before.key : fieldKeyFor(column),
129
+ };
130
+ });
131
+ });
132
+ setKeyField((k) =>
133
+ k && p.columns.some((c) => fieldKeyFor(c) === k) ? k : fieldKeyFor(p.columns[0] || "")
134
+ );
135
+ if (!targetLabel) setTargetLabel(p.title || "Scraped table");
136
+ setMessage(
137
+ `Read ${p.rowCount} rows and ${p.columns.length} columns` +
138
+ (p.tableCount && p.tableCount > 1 ? ` (of ${p.tableCount} tables on the page)` : "")
139
+ );
140
+ } catch (e) {
141
+ setProblem(
142
+ e instanceof AutomationError ? e.message : "That page could not be read."
143
+ );
144
+ } finally {
145
+ setReading(false);
146
+ }
147
+ }, [url, extract, tableIndex, targetLabel]);
148
+
149
+ const buildConfig = (): Record<string, unknown> => {
150
+ if (kind === "scrape_db") {
151
+ const fieldMap: Record<string, string> = {};
152
+ for (const c of plan) if (c.include && c.key) fieldMap[c.column] = c.key;
153
+ return {
154
+ url,
155
+ extract,
156
+ tableIndex,
157
+ fieldMap,
158
+ keyField,
159
+ targetTable: String(cfg.targetTable || ""),
160
+ targetLabel,
161
+ };
162
+ }
163
+ return { targetTable, fieldKey, urlField, maxPosts };
164
+ };
165
+
166
+ const save = async () => {
167
+ setSaving(true);
168
+ setProblem("");
169
+ setMessage("");
170
+ const body = {
171
+ name,
172
+ kind,
173
+ config: buildConfig(),
174
+ schedule: { cron, enabled },
175
+ };
176
+ try {
177
+ const res = automation
178
+ ? await patchAutomation(automation.id, body)
179
+ : await createAutomation(body);
180
+ setMessage("Saved.");
181
+ await onSaved(res.automation.id);
182
+ } catch (e) {
183
+ setProblem(e instanceof AutomationError ? e.message : "That could not be saved.");
184
+ } finally {
185
+ setSaving(false);
186
+ }
187
+ };
188
+
189
+ const start = async () => {
190
+ if (!automation) return;
191
+ setProblem("");
192
+ try {
193
+ await runAutomation(automation.id);
194
+ setMessage("Started. The rail's status dot follows it.");
195
+ await onSaved(automation.id);
196
+ } catch (e) {
197
+ setProblem(e instanceof AutomationError ? e.message : "It did not start.");
198
+ }
199
+ };
200
+
201
+ const remove = async () => {
202
+ if (!automation) return;
203
+ setProblem("");
204
+ try {
205
+ await deleteAutomation(automation.id);
206
+ await onDeleted();
207
+ } catch (e) {
208
+ setProblem(e instanceof AutomationError ? e.message : "It was not deleted.");
209
+ }
210
+ };
211
+
212
+ const openDrill = async (entry: RunEntry) => {
213
+ if (!automation) return;
214
+ if (drillFor === entry.ts) {
215
+ setDrill(null);
216
+ setDrillFor("");
217
+ return;
218
+ }
219
+ try {
220
+ const rows = await runRows(automation.id);
221
+ setDrill(rows);
222
+ setDrillFor(entry.ts);
223
+ } catch (e) {
224
+ setProblem(e instanceof AutomationError ? e.message : "Those rows could not be read.");
225
+ }
226
+ };
227
+
228
+ const presetMatch = useMemo(
229
+ () => cronPresets.find((p) => p.cron === cron)?.cron || "",
230
+ [cronPresets, cron]
231
+ );
232
+
233
+ const canSave =
234
+ !!name.trim() &&
235
+ (kind === "scrape_db"
236
+ ? !!url.trim() && plan.some((c) => c.include && c.key === keyField)
237
+ : !!targetTable && !!fieldKey);
238
+
239
+ return (
240
+ <div className="auto-editor">
241
+ <header className="auto-head">
242
+ <div className="auto-head-title">
243
+ <input
244
+ className="auto-name"
245
+ value={name}
246
+ placeholder="Name this automation"
247
+ aria-label="Automation name"
248
+ onChange={(e) => setName(e.target.value)}
249
+ />
250
+ {automation ? (
251
+ <span className={`auto-chip is-${automation.running ? "running"
252
+ : automation.status?.state || "idle"}`}>
253
+ {automation.running ? "Running" : automation.status?.state || "idle"}
254
+ </span>
255
+ ) : (
256
+ <span className="auto-chip is-idle">New</span>
257
+ )}
258
+ </div>
259
+ <div className="auto-head-actions">
260
+ {automation ? (
261
+ <button
262
+ type="button"
263
+ className="auto-btn"
264
+ disabled={!!automation.running}
265
+ onClick={() => void start()}
266
+ >
267
+ {automation.running ? "Running…" : "Run now"}
268
+ </button>
269
+ ) : null}
270
+ <button
271
+ type="button"
272
+ className="auto-btn is-primary"
273
+ disabled={!canSave || saving}
274
+ onClick={() => void save()}
275
+ >
276
+ {saving ? "Saving…" : automation ? "Save" : "Create"}
277
+ </button>
278
+ {automation ? (
279
+ <button type="button" className="auto-btn is-danger" onClick={() => void remove()}>
280
+ Delete
281
+ </button>
282
+ ) : (
283
+ <button type="button" className="auto-btn" onClick={onCancel}>
284
+ Cancel
285
+ </button>
286
+ )}
287
+ </div>
288
+ </header>
289
+
290
+ {problem ? (
291
+ <div className="auto-banner is-error" role="alert">
292
+ {problem}
293
+ </div>
294
+ ) : null}
295
+ {message ? (
296
+ <div className="auto-banner is-ok" role="status">
297
+ {message}
298
+ </div>
299
+ ) : null}
300
+
301
+ <div className="auto-body">
302
+ <section className="auto-card">
303
+ <h2>What it does</h2>
304
+ {isNew ? (
305
+ <div className="auto-kinds">
306
+ {kinds.map((k) => (
307
+ <label key={k.key} className={"auto-kind" + (kind === k.key ? " is-on" : "")}>
308
+ <input
309
+ type="radio"
310
+ name="auto-kind"
311
+ checked={kind === k.key}
312
+ onChange={() => setKind(k.key)}
313
+ />
314
+ <span>{k.label}</span>
315
+ </label>
316
+ ))}
317
+ </div>
318
+ ) : (
319
+ <p className="auto-note">
320
+ {kinds.find((k) => k.key === kind)?.label || kind} — the kind is fixed once an
321
+ automation exists, because its configuration and the database it fills are
322
+ shaped by it.
323
+ </p>
324
+ )}
325
+
326
+ {kind === "scrape_db" ? (
327
+ <>
328
+ <div className="auto-field">
329
+ <label htmlFor="auto-url">Page URL</label>
330
+ <div className="auto-row-inline">
331
+ <input
332
+ id="auto-url"
333
+ className="auto-input"
334
+ value={url}
335
+ placeholder="https://en.wikipedia.org/wiki/List_of_S%26P_500_companies"
336
+ onChange={(e) => setUrl(e.target.value)}
337
+ />
338
+ <button
339
+ type="button"
340
+ className="auto-btn"
341
+ disabled={!url.trim() || reading}
342
+ onClick={() => void readPage()}
343
+ >
344
+ {reading ? "Reading…" : "Read the page"}
345
+ </button>
346
+ </div>
347
+ <p className="auto-hint">
348
+ Public web pages only. Private, loopback and link-local addresses are
349
+ refused — including a public URL that redirects to one.
350
+ </p>
351
+ </div>
352
+
353
+ <div className="auto-field-row">
354
+ <div className="auto-field">
355
+ <label htmlFor="auto-extract">Read</label>
356
+ <select
357
+ id="auto-extract"
358
+ className="auto-input"
359
+ value={extract}
360
+ onChange={(e) => setExtract(e.target.value)}
361
+ >
362
+ <option value="table">An HTML table</option>
363
+ <option value="jsonld">Structured data (JSON-LD)</option>
364
+ </select>
365
+ </div>
366
+ {extract === "table" ? (
367
+ <div className="auto-field">
368
+ <label htmlFor="auto-tidx">Which table</label>
369
+ <input
370
+ id="auto-tidx"
371
+ className="auto-input"
372
+ type="number"
373
+ min={0}
374
+ value={tableIndex}
375
+ onChange={(e) => setTableIndex(Number(e.target.value) || 0)}
376
+ />
377
+ </div>
378
+ ) : null}
379
+ <div className="auto-field">
380
+ <label htmlFor="auto-tlabel">Database name</label>
381
+ <input
382
+ id="auto-tlabel"
383
+ className="auto-input"
384
+ value={targetLabel}
385
+ placeholder="S&P 500 companies"
386
+ onChange={(e) => setTargetLabel(e.target.value)}
387
+ />
388
+ </div>
389
+ </div>
390
+
391
+ {plan.length ? (
392
+ <div className="auto-map">
393
+ <h3>Columns</h3>
394
+ <p className="auto-hint">
395
+ Pick the key — the column that identifies a row. On every re-run, a row
396
+ whose key already exists is updated; a new key is inserted; a key that
397
+ has stopped appearing is counted and kept.
398
+ </p>
399
+ <table className="auto-table">
400
+ <thead>
401
+ <tr>
402
+ <th>Use</th>
403
+ <th>Source column</th>
404
+ <th>Field key</th>
405
+ <th>Key</th>
406
+ {preview?.sample?.length ? <th>First value</th> : null}
407
+ </tr>
408
+ </thead>
409
+ <tbody>
410
+ {plan.map((c, i) => (
411
+ <tr key={c.column}>
412
+ <td>
413
+ <input
414
+ type="checkbox"
415
+ checked={c.include}
416
+ aria-label={`Include ${c.column}`}
417
+ onChange={(e) =>
418
+ setPlan((p) =>
419
+ p.map((x, j) =>
420
+ j === i ? { ...x, include: e.target.checked } : x
421
+ )
422
+ )
423
+ }
424
+ />
425
+ </td>
426
+ <td className="auto-cell-name">{c.column}</td>
427
+ <td>
428
+ <input
429
+ className="auto-input is-small"
430
+ value={c.key}
431
+ aria-label={`Field key for ${c.column}`}
432
+ onChange={(e) =>
433
+ setPlan((p) =>
434
+ p.map((x, j) =>
435
+ j === i ? { ...x, key: fieldKeyFor(e.target.value) } : x
436
+ )
437
+ )
438
+ }
439
+ />
440
+ </td>
441
+ <td>
442
+ <input
443
+ type="radio"
444
+ name="auto-keyfield"
445
+ checked={keyField === c.key && c.include}
446
+ disabled={!c.include}
447
+ aria-label={`Use ${c.column} as the key`}
448
+ onChange={() => setKeyField(c.key)}
449
+ />
450
+ </td>
451
+ {preview?.sample?.length ? (
452
+ <td className="auto-cell-sample">
453
+ {String(preview.sample[0]?.[c.column] ?? "")}
454
+ </td>
455
+ ) : null}
456
+ </tr>
457
+ ))}
458
+ </tbody>
459
+ </table>
460
+ </div>
461
+ ) : (
462
+ <p className="auto-note">
463
+ Read the page to see what columns it offers.
464
+ </p>
465
+ )}
466
+ </>
467
+ ) : (
468
+ <>
469
+ <div className="auto-field-row">
470
+ <div className="auto-field">
471
+ <label htmlFor="auto-table">Database</label>
472
+ <select
473
+ id="auto-table"
474
+ className="auto-input"
475
+ value={targetTable}
476
+ onChange={(e) => {
477
+ setTargetTable(e.target.value);
478
+ setFieldKey("");
479
+ setUrlField("");
480
+ }}
481
+ >
482
+ <option value="">Choose a database…</option>
483
+ {tables.map((t) => (
484
+ <option key={t.key} value={t.key}>
485
+ {t.label} ({t.rowCount} {t.rowCount === 1 ? "row" : "rows"})
486
+ </option>
487
+ ))}
488
+ </select>
489
+ </div>
490
+ <div className="auto-field">
491
+ <label htmlFor="auto-fieldkey">Automation column</label>
492
+ <select
493
+ id="auto-fieldkey"
494
+ className="auto-input"
495
+ value={fieldKey}
496
+ onChange={(e) => setFieldKey(e.target.value)}
497
+ >
498
+ <option value="">Choose a column…</option>
499
+ {(table?.fields || [])
500
+ .filter((f) => f.type === "automation")
501
+ .map((f) => (
502
+ <option key={f.key} value={f.key}>
503
+ {f.label}
504
+ </option>
505
+ ))}
506
+ </select>
507
+ </div>
508
+ <div className="auto-field">
509
+ <label htmlFor="auto-urlfield">Profile URL column</label>
510
+ <select
511
+ id="auto-urlfield"
512
+ className="auto-input"
513
+ value={urlField}
514
+ onChange={(e) => setUrlField(e.target.value)}
515
+ >
516
+ <option value="">The column the field is bound to</option>
517
+ {(table?.fields || []).map((f) => (
518
+ <option key={f.key} value={f.key}>
519
+ {f.label}
520
+ </option>
521
+ ))}
522
+ </select>
523
+ </div>
524
+ <div className="auto-field">
525
+ <label htmlFor="auto-maxposts">Posts per pull</label>
526
+ <input
527
+ id="auto-maxposts"
528
+ className="auto-input"
529
+ type="number"
530
+ min={1}
531
+ max={200}
532
+ value={maxPosts}
533
+ onChange={(e) => setMaxPosts(Number(e.target.value) || 24)}
534
+ />
535
+ </div>
536
+ </div>
537
+ <p className="auto-hint">
538
+ Anonymous public endpoints only — no login and no credentials. Instagram
539
+ blocks datacenter addresses and has closed most of its anonymous surface,
540
+ so a pull reports what it actually got: <strong>ok</strong> (profile and
541
+ posts), <strong>partial</strong> (profile only), or <strong>blocked</strong>.
542
+ It never reports success on nothing.
543
+ </p>
544
+ {table && !(table.fields || []).some((f) => f.type === "automation") ? (
545
+ <p className="auto-note">
546
+ That database has no automation column yet. Add one from the grid&rsquo;s
547
+ column menu, then choose it here.
548
+ </p>
549
+ ) : null}
550
+ </>
551
+ )}
552
+ </section>
553
+
554
+ <section className="auto-card">
555
+ <h2>When it runs</h2>
556
+ <div className="auto-field-row">
557
+ <div className="auto-field">
558
+ <label htmlFor="auto-preset">Schedule</label>
559
+ <select
560
+ id="auto-preset"
561
+ className="auto-input"
562
+ value={presetMatch}
563
+ onChange={(e) => e.target.value && setCron(e.target.value)}
564
+ >
565
+ <option value="">Custom…</option>
566
+ {cronPresets.map((p) => (
567
+ <option key={p.cron} value={p.cron}>
568
+ {p.label}
569
+ </option>
570
+ ))}
571
+ </select>
572
+ </div>
573
+ <div className="auto-field">
574
+ <label htmlFor="auto-cron">Cron (minute hour day month weekday)</label>
575
+ <input
576
+ id="auto-cron"
577
+ className="auto-input is-mono"
578
+ value={cron}
579
+ onChange={(e) => setCron(e.target.value)}
580
+ />
581
+ </div>
582
+ <label className="auto-check">
583
+ <input
584
+ type="checkbox"
585
+ checked={enabled}
586
+ onChange={(e) => setEnabled(e.target.checked)}
587
+ />
588
+ <span>Run on this schedule</span>
589
+ </label>
590
+ </div>
591
+ <p className="auto-hint">
592
+ {automation?.nextRunAt
593
+ ? `Next run ${automation.nextRunAt}.`
594
+ : enabled
595
+ ? "Saved schedules start from the moment they are enabled — enabling a daily job after today's time does not fire it immediately."
596
+ : "Disabled. It still runs when you press Run now."}
597
+ </p>
598
+ </section>
599
+
600
+ <section className="auto-card">
601
+ <h2>Run history</h2>
602
+ {!automation?.runs?.length ? (
603
+ <p className="auto-note">It has not run yet.</p>
604
+ ) : (
605
+ <ol className="auto-runs">
606
+ {automation.runs.map((r) => (
607
+ <li key={r.ts} className={"auto-run is-" + (r.ok ? "ok" : "error")}>
608
+ <button
609
+ type="button"
610
+ className="auto-run-head"
611
+ onClick={() => void openDrill(r)}
612
+ aria-expanded={drillFor === r.ts}
613
+ >
614
+ <span className="auto-run-ts">{r.ts.replace("T", " ")}</span>
615
+ <span className="auto-run-summary">{r.summary}</span>
616
+ </button>
617
+ <div className="auto-counts">
618
+ {COUNT_LABELS.filter(([k]) => typeof r.counts?.[k] === "number").map(
619
+ ([k, label]) => (
620
+ <span key={k} className="auto-count">
621
+ <b>{r.counts[k]}</b> {label}
622
+ </span>
623
+ )
624
+ )}
625
+ </div>
626
+ {drillFor === r.ts && drill ? (
627
+ <div className="auto-drill">
628
+ <p className="auto-hint">
629
+ The rows this run touched in {drill.label}
630
+ {drill.truncated ? " (first 200)" : ""}.
631
+ </p>
632
+ <div className="auto-drill-scroll">
633
+ <table className="auto-table">
634
+ <thead>
635
+ <tr>
636
+ {drill.fields.map((f) => (
637
+ <th key={f.key}>{f.label}</th>
638
+ ))}
639
+ </tr>
640
+ </thead>
641
+ <tbody>
642
+ {drill.rows.slice(0, 50).map((row) => (
643
+ <tr key={String(row.id)}>
644
+ {drill.fields.map((f) => (
645
+ <td key={f.key}>{String(row[f.key] ?? "")}</td>
646
+ ))}
647
+ </tr>
648
+ ))}
649
+ </tbody>
650
+ </table>
651
+ </div>
652
+ </div>
653
+ ) : null}
654
+ </li>
655
+ ))}
656
+ </ol>
657
+ )}
658
+ </section>
659
+ </div>
660
+ </div>
661
+ );
662
+ }
web/src/automation/AutomationSurface.tsx ADDED
@@ -0,0 +1,287 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ // ---------------------------------------------------------------------------
2
+ // automation/AutomationSurface.tsx — the Automation surface (contract C-AUTONAV).
3
+ //
4
+ // ⛔ THE CONTRACT THIS FILE GUARANTEES, verbatim: this module path, a DEFAULT
5
+ // export, and NO PROPS. The shell mounts `<AutomationSurface />` and hands it
6
+ // nothing; it fetches its own data from `/api/v1/automations`. That is the whole
7
+ // interface, and it is stated here because the mount line lives in a file this
8
+ // session does not own — a named export would cost a cross-session round trip.
9
+ //
10
+ // "Automation replaces the Views rail" (owner) is implemented the way the wave
11
+ // scout established: the secondary rail is NOT a shell concept — each surface
12
+ // owns its own. So this renders a sibling of `.cg-views` built from the same
13
+ // `--lp-rail-w` tokens and the same fold behaviour, and the Views rail is
14
+ // untouched.
15
+ // ---------------------------------------------------------------------------
16
+ import type { KeyboardEvent as ReactKeyboardEvent } from "react";
17
+ import { useCallback, useEffect, useRef, useState } from "react";
18
+
19
+ import AutomationEditor from "./AutomationEditor";
20
+ import type { Automation, AutomationList } from "./automationApi";
21
+ import {
22
+ AutomationError,
23
+ listAutomations,
24
+ patchAutomation,
25
+ stateOf,
26
+ } from "./automationApi";
27
+
28
+ /** The status dot's title — the rail says WHY it is that colour, not just that it is. */
29
+ function stateTitle(a: Automation): string {
30
+ const s = stateOf(a);
31
+ if (s === "running") return "Running now";
32
+ if (s === "ok") return `Last run succeeded — ${a.status?.lastSummary || ""}`;
33
+ if (s === "partial") return `Last run was partial — ${a.status?.lastSummary || ""}`;
34
+ if (s === "error") return `Last run failed — ${a.status?.lastSummary || ""}`;
35
+ return "Has not run yet";
36
+ }
37
+
38
+ /**
39
+ * The rail's second line — ONE fact, not two.
40
+ *
41
+ * ⚠ It carried "Next 2026-08-04 06:00 · last 2026-08-04 00:20", and the rail is `--lp-rail-w`
42
+ * wide, so it rendered as "Next 2026-08-04 06:00 · l…" — a second line whose only complete word
43
+ * was "Next". Two facts that both truncate are worth less than one that fits. The forward-looking
44
+ * one wins when a schedule exists (it is the question the rail answers: does this run itself?),
45
+ * the last run otherwise. The full picture is one click away in the editor's run history, and the
46
+ * row's `title` already says both.
47
+ *
48
+ * Caught by READING the screenshot: the DOM was correct and the CSS ellipsis was doing exactly
49
+ * its job. [[ui-invisible-to-assertions]] — judge the pixels even when everything is green.
50
+ */
51
+ function railSubtitle(a: Automation): string {
52
+ if (a.schedule?.enabled) return a.nextRunAt ? `Next ${a.nextRunAt}` : "Scheduled";
53
+ const last = a.status?.lastRunAt ? a.status.lastRunAt.replace("T", " ").slice(0, 16) : "";
54
+ return last ? `Last run ${last}` : "Manual only";
55
+ }
56
+
57
+ export default function AutomationSurface() {
58
+ const [data, setData] = useState<AutomationList | null>(null);
59
+ const [error, setError] = useState("");
60
+ const [activeId, setActiveId] = useState<string>("");
61
+ const [creating, setCreating] = useState(false);
62
+ const [railShut, setRailShut] = useState(false);
63
+ const [busy, setBusy] = useState("");
64
+ const listRef = useRef<HTMLDivElement | null>(null);
65
+
66
+ const load = useCallback(async (abort?: AbortSignal) => {
67
+ try {
68
+ const next = await listAutomations(abort);
69
+ setData(next);
70
+ setError("");
71
+ return next;
72
+ } catch (e) {
73
+ if ((e as Error)?.name === "AbortError") return null;
74
+ setError(
75
+ e instanceof AutomationError
76
+ ? e.message
77
+ : "The automation service did not answer."
78
+ );
79
+ return null;
80
+ }
81
+ }, []);
82
+
83
+ useEffect(() => {
84
+ const ac = new AbortController();
85
+ void load(ac.signal);
86
+ return () => ac.abort();
87
+ }, [load]);
88
+
89
+ // A run is a background thread on the server, so the surface has to ASK whether it
90
+ // finished. Polling only while something is actually running keeps an idle surface
91
+ // silent — a fixed interval would be a request every few seconds forever, for a page
92
+ // whose contents change a handful of times a day.
93
+ const anyRunning = (data?.automations || []).some((a) => a.running);
94
+ useEffect(() => {
95
+ if (!anyRunning) return undefined;
96
+ const t = window.setInterval(() => void load(), 2500);
97
+ return () => window.clearInterval(t);
98
+ }, [anyRunning, load]);
99
+
100
+ const items = data?.automations || [];
101
+ const active = items.find((a) => a.id === activeId) || null;
102
+
103
+ const toggleEnabled = async (a: Automation) => {
104
+ setBusy(a.id);
105
+ try {
106
+ await patchAutomation(a.id, {
107
+ schedule: { cron: a.schedule.cron, enabled: !a.schedule.enabled },
108
+ });
109
+ await load();
110
+ } catch (e) {
111
+ setError(e instanceof AutomationError ? e.message : "That change was not saved.");
112
+ } finally {
113
+ setBusy("");
114
+ }
115
+ };
116
+
117
+ const onRailKey = (event: ReactKeyboardEvent<HTMLDivElement>) => {
118
+ if (event.key !== "ArrowDown" && event.key !== "ArrowUp") return;
119
+ const rows = Array.from(
120
+ listRef.current?.querySelectorAll<HTMLButtonElement>(".auto-row-main") || []
121
+ );
122
+ if (!rows.length) return;
123
+ event.preventDefault();
124
+ const at = rows.indexOf(document.activeElement as HTMLButtonElement);
125
+ const next = event.key === "ArrowDown" ? (at + 1) % rows.length
126
+ : (at - 1 + rows.length) % rows.length;
127
+ rows[next < 0 ? 0 : next]?.focus();
128
+ };
129
+
130
+ return (
131
+ <div className="auto-surface">
132
+ <aside
133
+ className={"auto-rail" + (railShut ? " is-collapsed" : "")}
134
+ aria-label="Automations"
135
+ >
136
+ {/* The same three-bars fold control the Views rail carries — the two rails are
137
+ one idea, so they must not fold with two different affordances. */}
138
+ <div className="auto-rail-top">
139
+ <button
140
+ type="button"
141
+ className="cg-rail-toggle"
142
+ aria-label={railShut ? "Expand automations" : "Minimize automations"}
143
+ aria-expanded={!railShut}
144
+ title={railShut ? "Expand automations" : "Minimize automations"}
145
+ onClick={() => setRailShut((v) => !v)}
146
+ >
147
+ <svg width="16" height="16" viewBox="0 0 16 16" fill="none" aria-hidden="true">
148
+ <path
149
+ d="M2.5 4.4h11M2.5 8h11M2.5 11.6h11"
150
+ stroke="currentColor"
151
+ strokeWidth="1.35"
152
+ strokeLinecap="round"
153
+ />
154
+ </svg>
155
+ </button>
156
+ </div>
157
+
158
+ <div className="auto-create">
159
+ <button
160
+ type="button"
161
+ className="auto-create-btn"
162
+ onClick={() => {
163
+ setCreating(true);
164
+ setActiveId("");
165
+ }}
166
+ >
167
+ <svg width="14" height="14" viewBox="0 0 16 16" fill="none" aria-hidden="true">
168
+ <path
169
+ d="M8 3.4v9.2M3.4 8h9.2"
170
+ stroke="currentColor"
171
+ strokeWidth="1.5"
172
+ strokeLinecap="round"
173
+ />
174
+ </svg>
175
+ New automation
176
+ </button>
177
+ </div>
178
+
179
+ <div className="auto-list" ref={listRef} onKeyDown={onRailKey}>
180
+ {items.map((a) => {
181
+ const s = stateOf(a);
182
+ return (
183
+ <div
184
+ key={a.id}
185
+ className={"auto-row" + (a.id === activeId ? " is-active" : "")}
186
+ >
187
+ <button
188
+ type="button"
189
+ className="auto-row-main"
190
+ onClick={() => {
191
+ setActiveId(a.id);
192
+ setCreating(false);
193
+ }}
194
+ >
195
+ <span
196
+ className={`auto-dot is-${s}`}
197
+ title={stateTitle(a)}
198
+ aria-label={stateTitle(a)}
199
+ />
200
+ <span className="auto-row-text">
201
+ <span className="auto-row-name">{a.name}</span>
202
+ <span className="auto-row-desc">{railSubtitle(a)}</span>
203
+ </span>
204
+ </button>
205
+ <button
206
+ type="button"
207
+ className={"auto-row-switch" + (a.schedule?.enabled ? " is-on" : "")}
208
+ disabled={busy === a.id}
209
+ aria-pressed={!!a.schedule?.enabled}
210
+ title={
211
+ a.schedule?.enabled
212
+ ? `Scheduled: ${a.schedule.cron}. Click to pause.`
213
+ : "Not scheduled. Click to enable."
214
+ }
215
+ onClick={() => void toggleEnabled(a)}
216
+ >
217
+ <span className="auto-row-switch-knob" />
218
+ </button>
219
+ </div>
220
+ );
221
+ })}
222
+ {!items.length && !error ? (
223
+ <p className="auto-rail-empty">
224
+ No automations yet. One reads a public web page into a database on a
225
+ schedule; the other fills an automation column from a public profile.
226
+ </p>
227
+ ) : null}
228
+ </div>
229
+ </aside>
230
+
231
+ <section className="auto-main">
232
+ {error ? (
233
+ <div className="auto-banner is-error" role="alert">
234
+ {error}
235
+ </div>
236
+ ) : null}
237
+ {data && !data.storeAvailable ? (
238
+ <div className="auto-banner is-warn" role="status">
239
+ The tenant store is unavailable, so nothing can be saved right now.
240
+ </div>
241
+ ) : null}
242
+
243
+ {creating || active ? (
244
+ <AutomationEditor
245
+ key={active?.id || "new"}
246
+ automation={active}
247
+ kinds={data?.kinds || []}
248
+ cronPresets={data?.cronPresets || []}
249
+ onSaved={async (id) => {
250
+ setCreating(false);
251
+ const next = await load();
252
+ if (id && next) setActiveId(id);
253
+ }}
254
+ onDeleted={async () => {
255
+ setActiveId("");
256
+ setCreating(false);
257
+ await load();
258
+ }}
259
+ onCancel={() => setCreating(false)}
260
+ />
261
+ ) : (
262
+ <div className="auto-empty">
263
+ <h1>Automations</h1>
264
+ <p>
265
+ An automation is a job this workspace runs for you — on demand, or on a
266
+ schedule. Pick one on the left, or create a new one.
267
+ </p>
268
+ <ul className="auto-empty-kinds">
269
+ <li>
270
+ <strong>Web page to database.</strong> Read a public page, map its columns
271
+ to fields, and keep a database in step with it. Re-runs update matched rows
272
+ by key and insert new ones; a row that stops appearing is counted and kept,
273
+ never deleted.
274
+ </li>
275
+ <li>
276
+ <strong>Instagram profile column.</strong> For every record carrying a
277
+ public profile URL, read what is anonymously available and write the result
278
+ into an automation column, with each pull timestamped so the history is
279
+ filterable by date.
280
+ </li>
281
+ </ul>
282
+ </div>
283
+ )}
284
+ </section>
285
+ </div>
286
+ );
287
+ }
web/src/automation/automationApi.ts ADDED
@@ -0,0 +1,208 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ // ---------------------------------------------------------------------------
2
+ // automation/automationApi.ts — the Automation surface's whole data layer.
3
+ //
4
+ // Wave-18 item 5 (contracts C4-AUTO + C-AUTONAV). Deliberately its own leaf
5
+ // rather than an addition to `customer-grid/apiBridge.ts`: the grid tree is the
6
+ // ASSET that also ships inside the Streamlit embed, and an automation surface
7
+ // that only the standalone shell mounts must not grow an edge into it. It
8
+ // imports `apiContract` (the shared leaf both trees already read) and nothing
9
+ // else.
10
+ //
11
+ // ⚠ EVERY VOCABULARY HERE COMES FROM THE SERVER. The kind list and the cron
12
+ // presets ride on `GET /automations` rather than sitting in a client constant,
13
+ // because the parser that accepts a cron string lives in Python — a client copy
14
+ // of the preset list is a list that can offer a schedule the server rejects,
15
+ // and the user would see the refusal as "it just doesn't save".
16
+ // ---------------------------------------------------------------------------
17
+ import { API_V1, CREDENTIALS, UNAUTHORIZED_EVENT, signal } from "../apiContract";
18
+
19
+ export type AutomationKind = "scrape_db" | "field_instagram";
20
+ export type RunState = "idle" | "running" | "ok" | "error" | "partial";
21
+
22
+ export interface RunEntry {
23
+ ts: string;
24
+ ok: boolean;
25
+ summary: string;
26
+ counts: Record<string, number>;
27
+ affected?: string[];
28
+ }
29
+
30
+ export interface Automation {
31
+ id: string;
32
+ name: string;
33
+ kind: AutomationKind;
34
+ config: Record<string, unknown>;
35
+ schedule: { cron: string; enabled: boolean; enabledAt?: string };
36
+ status: { state: RunState; lastRunAt?: string; lastSummary?: string; step?: string };
37
+ runs: RunEntry[];
38
+ created?: string;
39
+ createdBy?: string;
40
+ nextRunAt?: string;
41
+ running?: boolean;
42
+ }
43
+
44
+ export interface AutomationList {
45
+ automations: Automation[];
46
+ kinds: { key: AutomationKind; label: string }[];
47
+ cronPresets: { cron: string; label: string }[];
48
+ storeAvailable: boolean;
49
+ }
50
+
51
+ export interface UserTable {
52
+ key: string;
53
+ label: string;
54
+ source: string;
55
+ rowCount: number;
56
+ fields: { key: string; label: string; type: string; automation?: unknown }[];
57
+ }
58
+
59
+ export interface SourcePreview {
60
+ ok: boolean;
61
+ status: number;
62
+ url: string;
63
+ title?: string;
64
+ note?: string;
65
+ columns: string[];
66
+ sample: Record<string, string>[];
67
+ rowCount: number;
68
+ tableCount?: number;
69
+ }
70
+
71
+ export interface RunRows {
72
+ table: string;
73
+ label: string;
74
+ fields: { key: string; label: string }[];
75
+ rows: Record<string, string>[];
76
+ truncated: boolean;
77
+ }
78
+
79
+ export class AutomationError extends Error {
80
+ status: number;
81
+
82
+ constructor(message: string, status: number) {
83
+ super(message);
84
+ this.name = "AutomationError";
85
+ this.status = status;
86
+ }
87
+ }
88
+
89
+ async function result<T>(response: Response): Promise<T> {
90
+ let body: unknown = null;
91
+ try {
92
+ body = await response.json();
93
+ } catch {
94
+ // The status is still an honest answer; the message below says so.
95
+ }
96
+ if (response.status === 401) signal(UNAUTHORIZED_EVENT);
97
+ if (!response.ok) {
98
+ const shaped = body as { error?: { message?: string } } | null;
99
+ throw new AutomationError(
100
+ shaped?.error?.message || `The server answered ${response.status}.`,
101
+ response.status
102
+ );
103
+ }
104
+ return body as T;
105
+ }
106
+
107
+ function send<T>(path: string, init?: RequestInit): Promise<T> {
108
+ return fetch(`${API_V1}${path}`, {
109
+ credentials: CREDENTIALS,
110
+ headers: init?.body ? { "Content-Type": "application/json" } : undefined,
111
+ ...init,
112
+ }).then((r) => result<T>(r));
113
+ }
114
+
115
+ export function listAutomations(abort?: AbortSignal): Promise<AutomationList> {
116
+ return send<AutomationList>("/automations", { signal: abort });
117
+ }
118
+
119
+ export function listTables(abort?: AbortSignal): Promise<{ tables: UserTable[] }> {
120
+ return send<{ tables: UserTable[] }>("/automations/tables", { signal: abort });
121
+ }
122
+
123
+ export function createAutomation(body: unknown): Promise<{ automation: Automation }> {
124
+ return send("/automations", { method: "POST", body: JSON.stringify(body) });
125
+ }
126
+
127
+ export function patchAutomation(
128
+ id: string,
129
+ body: unknown
130
+ ): Promise<{ automation: Automation }> {
131
+ return send(`/automations/${encodeURIComponent(id)}`, {
132
+ method: "PATCH",
133
+ body: JSON.stringify(body),
134
+ });
135
+ }
136
+
137
+ export function deleteAutomation(id: string): Promise<{ deleted: string }> {
138
+ return send(`/automations/${encodeURIComponent(id)}`, { method: "DELETE" });
139
+ }
140
+
141
+ export function runAutomation(id: string): Promise<{ started: string }> {
142
+ return send(`/automations/${encodeURIComponent(id)}/run`, { method: "POST" });
143
+ }
144
+
145
+ export function getAutomation(id: string): Promise<{ automation: Automation }> {
146
+ return send(`/automations/${encodeURIComponent(id)}`);
147
+ }
148
+
149
+ export function runRows(id: string): Promise<RunRows> {
150
+ return send(`/automations/${encodeURIComponent(id)}/rows`);
151
+ }
152
+
153
+ export function previewSource(
154
+ url: string,
155
+ extract: string,
156
+ tableIndex: number
157
+ ): Promise<SourcePreview> {
158
+ return send("/automations/preview", {
159
+ method: "POST",
160
+ body: JSON.stringify({ url, extract, tableIndex }),
161
+ });
162
+ }
163
+
164
+ /**
165
+ * A source column name → a storable field key. Mirrors the server's own
166
+ * normalisation (`clean_config`) so the key the UI shows is the key that gets
167
+ * stored — a preview that promised `gics_sector` and a store that wrote
168
+ * `gics__sector` would look like data loss.
169
+ */
170
+ export function fieldKeyFor(column: string): string {
171
+ return (column || "")
172
+ .toLowerCase()
173
+ .replace(/[^a-z0-9_]+/g, "_")
174
+ .replace(/^_+|_+$/g, "")
175
+ .slice(0, 60);
176
+ }
177
+
178
+ /** The state word a status dot / chip paints from. */
179
+ export function stateOf(a: Automation): RunState {
180
+ return a.running ? "running" : (a.status?.state as RunState) || "idle";
181
+ }
182
+
183
+ /**
184
+ * A run's counts as ordered label/value pairs. The ORDER is the sentence the
185
+ * numbers make — new, updated, unchanged, then the ones that need explaining —
186
+ * and `orphans` is spelled out rather than abbreviated because a bare number
187
+ * next to the word "orphans" is the one count a user must not have to guess at.
188
+ */
189
+ export const COUNT_LABELS: [string, string][] = [
190
+ ["inserted", "New"],
191
+ ["updated", "Updated"],
192
+ ["unchanged", "Unchanged"],
193
+ ["orphans", "No longer on the page (kept)"],
194
+ ["duplicates", "Duplicate keys"],
195
+ ["skipped", "Skipped"],
196
+ ["source_rows", "Rows read"],
197
+ ["profiles", "Profiles"],
198
+ ["ok", "Read in full"],
199
+ // ⚠ ADDED after reading the first live screenshot. The engine counts `partial` separately
200
+ // (profile read, media not anonymously readable) but this table did not, so a run whose
201
+ // summary said "1/1 profiles read" showed a chip row reading "0 Read" — the chips
202
+ // contradicting the sentence directly above them. A count the summary names must have a chip.
203
+ ["partial", "Profile only"],
204
+ ["blocked", "Blocked"],
205
+ ["error", "Errors"],
206
+ ["posts", "Posts"],
207
+ ["new_posts", "New posts"],
208
+ ];
web/src/customer-grid/CatalogView.tsx ADDED
@@ -0,0 +1,1147 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ // ---------------------------------------------------------------------------
2
+ // customer-grid / CatalogView.tsx
3
+ // Wave-18 owner item 4 (contract C6-CATALOG) — the CATALOG view.
4
+ //
5
+ // A view holds several catalogs; a catalog is an ordered list of pages; a
6
+ // section page names product CODES and joins them to the live pool at paint
7
+ // time. Three columns: the rail (catalogs, then that catalog's pages), the
8
+ // stage (the pages themselves, at true paper size, scaled to fit), and the
9
+ // inspector (everything about the selected page, plus the catalog's brand,
10
+ // paper and column bindings).
11
+ //
12
+ // FIDELITY. The default templates reproduce `Royal Collection 2027.pdf`, which
13
+ // was rendered and read before this file was written:
14
+ // · section header — letterspaced caps, navy, light weight, top-left
15
+ // · product renders on WHITE (the C2-ASSET web derivative is alpha-preserving
16
+ // PNG precisely so this works)
17
+ // · description in caps, centred, two lines
18
+ // · a full-width navy PILL carrying the pack line
19
+ // · SKU in terracotta bold on the left, colour name on the right
20
+ // · a full-bleed lifestyle strip under the grid
21
+ // · crown mark bottom-left, folio bottom-right
22
+ // · gallery pages full-bleed with a letterspaced caps title
23
+ //
24
+ // TYPE. Catalog pages use a display SERIF for titles (`--cat-serif`, a system
25
+ // Didone/old-style stack — Didot, Bodoni, Playfair, Cormorant, Georgia). That is
26
+ // deliberate and it is the ONLY place it appears: app chrome stays Inter, per
27
+ // the owner constant. Listing text is Inter with wide tracking, which is what
28
+ // the reference's geometric sans reads as at listing size. NO EMOJIS anywhere —
29
+ // every mark here is a drawn path.
30
+ // ---------------------------------------------------------------------------
31
+
32
+ import { memo, useCallback, useEffect, useLayoutEffect, useMemo, useRef, useState } from "react";
33
+ import { createPortal } from "react-dom";
34
+ import {
35
+ CATALOG_COLS,
36
+ CATALOG_NAME_MAX,
37
+ CATALOG_ORIENTATIONS,
38
+ CATALOG_PAGE_KINDS,
39
+ CATALOG_PAPERS,
40
+ MAX_CATALOGS,
41
+ MAX_CATALOG_PAGES,
42
+ MAX_CATALOG_CODES,
43
+ } from "./types";
44
+ import type {
45
+ CatalogOrientation,
46
+ CatalogPage,
47
+ CatalogPageKind,
48
+ CatalogPaper,
49
+ CatalogQuality,
50
+ CatalogSpec,
51
+ Field,
52
+ Row,
53
+ } from "./types";
54
+ import {
55
+ CATALOG_CODE_FIELD,
56
+ DEFAULT_BRAND,
57
+ PAGE_KIND_LABELS,
58
+ PAPER_LABELS,
59
+ addCodes,
60
+ assetUrl,
61
+ autoBind,
62
+ codesUsed,
63
+ fetchAssetCodes,
64
+ indexByCode,
65
+ newCatalog,
66
+ newPage,
67
+ pageBox,
68
+ resolveItems,
69
+ sectionLayout,
70
+ } from "./catalogData";
71
+ import type { CatalogItem } from "./catalogData";
72
+ import { applyPrintCss, awaitImages, printOnce } from "./catalogPrint";
73
+ import { formatDisplay } from "./display";
74
+
75
+ // ------------------------------------------------------------------- marks
76
+
77
+ const STROKE = {
78
+ stroke: "currentColor",
79
+ strokeWidth: 1.5,
80
+ strokeLinecap: "round" as const,
81
+ strokeLinejoin: "round" as const,
82
+ fill: "none",
83
+ };
84
+
85
+ function IconPlus() {
86
+ return (
87
+ <svg width="13" height="13" viewBox="0 0 16 16" aria-hidden>
88
+ <path d="M8 3.4v9.2M3.4 8h9.2" {...STROKE} />
89
+ </svg>
90
+ );
91
+ }
92
+ function IconTrash() {
93
+ return (
94
+ <svg width="13" height="13" viewBox="0 0 16 16" aria-hidden>
95
+ <path d="M3.4 4.6h9.2M6.4 4.6V3.2h3.2v1.4M4.8 4.6l.6 8h5.2l.6-8" {...STROKE} />
96
+ </svg>
97
+ );
98
+ }
99
+ function IconUp() {
100
+ return (
101
+ <svg width="13" height="13" viewBox="0 0 16 16" aria-hidden>
102
+ <path d="M4.4 9.6L8 6l3.6 3.6" {...STROKE} />
103
+ </svg>
104
+ );
105
+ }
106
+ function IconDown() {
107
+ return (
108
+ <svg width="13" height="13" viewBox="0 0 16 16" aria-hidden>
109
+ <path d="M4.4 6.4L8 10l3.6-3.6" {...STROKE} />
110
+ </svg>
111
+ );
112
+ }
113
+ function IconPrint() {
114
+ return (
115
+ <svg width="14" height="14" viewBox="0 0 16 16" aria-hidden>
116
+ <path d="M4.6 6.4V2.8h6.8v3.6" {...STROKE} />
117
+ <path d="M2.8 6.4h10.4v4.4H11v2.4H5v-2.4H2.8z" {...STROKE} />
118
+ <path d="M5 10.8h6" {...STROKE} />
119
+ </svg>
120
+ );
121
+ }
122
+
123
+ /**
124
+ * The Royal crown-diamond mark, drawn as paths.
125
+ *
126
+ * A row of three diamond outlines over a solid bar — the silhouette the 2027
127
+ * catalogue carries on its cover, its footers and its back page. Drawn rather
128
+ * than uploaded because a catalog has to print the mark at whatever size the
129
+ * page gives it, and because an image dependency for the ONE element on every
130
+ * page would make an unreachable asset route into a blank catalogue.
131
+ */
132
+ function CrownMark({ className }: { className?: string }) {
133
+ return (
134
+ <svg className={className} viewBox="0 0 44 26" aria-hidden>
135
+ <g fill="none" stroke="currentColor" strokeWidth="1.7" strokeLinejoin="round">
136
+ <path d="M11 4.5l6.5 6.5L11 17.5 4.5 11z" />
137
+ <path d="M22 2.5l7.5 7.5L22 17.5 14.5 10z" />
138
+ <path d="M33 4.5L39.5 11 33 17.5 26.5 11z" />
139
+ </g>
140
+ <rect x="4" y="20" width="36" height="2.6" fill="currentColor" />
141
+ </svg>
142
+ );
143
+ }
144
+
145
+ // ------------------------------------------------------------ page templates
146
+
147
+ /** The image slot every template shares: the asset, or a STATED gap.
148
+ *
149
+ * A missing master is drawn as a labelled frame naming the code, never as
150
+ * silence — the user has to be able to see, in the designer, that this page
151
+ * will print an empty box, and which SKU caused it. */
152
+ function Shot({
153
+ code,
154
+ quality,
155
+ missing,
156
+ cover,
157
+ }: {
158
+ code?: string;
159
+ quality: CatalogQuality;
160
+ missing?: boolean;
161
+ cover?: boolean;
162
+ }) {
163
+ if (!code)
164
+ return (
165
+ <div className="cg-cat-shot cg-cat-shot-empty">
166
+ <span>No image</span>
167
+ </div>
168
+ );
169
+ if (missing)
170
+ return (
171
+ <div className="cg-cat-shot cg-cat-shot-empty">
172
+ <span>No master for {code}</span>
173
+ </div>
174
+ );
175
+ return (
176
+ <div className={cover ? "cg-cat-shot cg-cat-shot-cover" : "cg-cat-shot"}>
177
+ <img src={assetUrl(code, quality)} alt="" loading="eager" />
178
+ </div>
179
+ );
180
+ }
181
+
182
+ function ProductCard({
183
+ item,
184
+ quality,
185
+ layout,
186
+ hasAsset,
187
+ }: {
188
+ item: CatalogItem;
189
+ quality: CatalogQuality;
190
+ layout: ReturnType<typeof sectionLayout>;
191
+ hasAsset: (code: string) => boolean | null;
192
+ }) {
193
+ const known = hasAsset(item.code);
194
+ return (
195
+ <div className="cg-cat-card">
196
+ <Shot code={item.code} quality={quality} missing={known === false} />
197
+ {/* CAPS in CSS (`text-transform`), never by upper-casing the string: the
198
+ value stays the value, so a copy of it is not shouting. */}
199
+ <div className="cg-cat-desc">{item.name ?? (item.known ? "" : "Not in this table")}</div>
200
+ {layout.showPack && item.pack ? <div className="cg-cat-pill">{item.pack}</div> : null}
201
+ <div className="cg-cat-meta">
202
+ <span className="cg-cat-sku">{item.code}</span>
203
+ {layout.showColor && item.color ? (
204
+ <span className="cg-cat-color">{item.color}</span>
205
+ ) : null}
206
+ </div>
207
+ {layout.showPrice && item.price ? <div className="cg-cat-price">{item.price}</div> : null}
208
+ </div>
209
+ );
210
+ }
211
+
212
+ /** Crown mark left, folio right — exactly the reference's footer, which carries no
213
+ * company line (the wordmark's place is the cover). `light` inverts it for the
214
+ * pages whose field is dark.
215
+ *
216
+ * `mark={false}` drops the crown: the intro page's terracotta lattice bleeds through the
217
+ * bottom-left corner, and a second terracotta mark sitting inside it reads as a smudge on the
218
+ * lattice rather than as the brand. The reference's own letter spread carries no crown there
219
+ * either — judged from the rendered page, not assumed. */
220
+ function PageFooter({
221
+ folio,
222
+ light,
223
+ mark = true,
224
+ }: {
225
+ folio: number;
226
+ light?: boolean;
227
+ mark?: boolean;
228
+ }) {
229
+ return (
230
+ <div className={light ? "cg-cat-foot cg-cat-foot-light" : "cg-cat-foot"}>
231
+ {mark ? <CrownMark className="cg-cat-foot-mark" /> : null}
232
+ <span className="cg-cat-folio">{folio}</span>
233
+ </div>
234
+ );
235
+ }
236
+
237
+ /** ONE page, at true paper size. The same component paints the stage and the
238
+ * print tree — there is no second renderer that could drift from what the
239
+ * screenshot showed. */
240
+ const CatalogPageView = memo(function CatalogPageView({
241
+ cat,
242
+ page,
243
+ index,
244
+ items,
245
+ quality,
246
+ hasAsset,
247
+ }: {
248
+ cat: CatalogSpec;
249
+ page: CatalogPage;
250
+ index: number;
251
+ items: CatalogItem[];
252
+ quality: CatalogQuality;
253
+ hasAsset: (code: string) => boolean | null;
254
+ }) {
255
+ const box = pageBox(cat.paper, cat.orientation);
256
+ const px = box.unit === "in" ? 96 : 96 / 25.4;
257
+ const brand = cat.brand ?? {};
258
+ const primary = brand.primary ?? DEFAULT_BRAND.primary;
259
+ const accent = brand.accent ?? DEFAULT_BRAND.accent;
260
+ const company = brand.company ?? DEFAULT_BRAND.company;
261
+ const style = {
262
+ "--cat-w": `${box.w * px}px`,
263
+ "--cat-h": `${box.h * px}px`,
264
+ "--cat-primary": primary,
265
+ "--cat-accent": accent,
266
+ } as React.CSSProperties;
267
+ const folio = index + 1;
268
+ const logoKnown = brand.logo ? hasAsset(brand.logo) : null;
269
+
270
+ const Wordmark = (
271
+ <div className="cg-cat-mark">
272
+ {brand.logo && logoKnown !== false ? (
273
+ <img src={assetUrl(brand.logo, quality)} alt="" />
274
+ ) : (
275
+ <CrownMark className="cg-cat-mark-crown" />
276
+ )}
277
+ <span className="cg-cat-mark-co">{company}</span>
278
+ </div>
279
+ );
280
+
281
+ if (page.kind === "cover") {
282
+ const known = page.imageCode ? hasAsset(page.imageCode) : null;
283
+ return (
284
+ <div className="cg-cat-page cg-cat-p-cover" style={style} data-kind="cover">
285
+ <Shot code={page.imageCode} quality={quality} missing={known === false} cover />
286
+ <div className="cg-cat-cover-scrim" />
287
+ {Wordmark}
288
+ <div className="cg-cat-cover-ttl">
289
+ <span className="cg-cat-cover-a">{page.title}</span>
290
+ {page.body ? <span className="cg-cat-cover-b">{page.body}</span> : null}
291
+ </div>
292
+ </div>
293
+ );
294
+ }
295
+
296
+ if (page.kind === "intro") {
297
+ return (
298
+ <div className="cg-cat-page cg-cat-p-intro" style={style} data-kind="intro">
299
+ <div className="cg-cat-intro-field">
300
+ {/* The terracotta lattice bleeding off the left edge — the reference's
301
+ own device for the letter spread, drawn from the crown's diamond. */}
302
+ <svg className="cg-cat-lattice" viewBox="0 0 200 260" aria-hidden>
303
+ <g fill="none" stroke="currentColor" strokeWidth="7">
304
+ <path d="M-40 130l60-60 60 60-60 60z" />
305
+ <path d="M40 190l60-60 60 60-60 60z" />
306
+ <path d="M100 130l40 40 40-90" />
307
+ </g>
308
+ </svg>
309
+ </div>
310
+ <div className="cg-cat-intro-body">
311
+ <h2 className="cg-cat-intro-ttl">{page.title}</h2>
312
+ {/* Paragraphs, not a wall: the reference's letter is set in short
313
+ blocks and reads as one because of the spacing between them. */}
314
+ {(page.body ?? "").split(/\n{2,}/).map((para, i) =>
315
+ para.trim() ? (
316
+ <p key={i} className="cg-cat-intro-p">
317
+ {para.trim()}
318
+ </p>
319
+ ) : null
320
+ )}
321
+ <div className="cg-cat-intro-sign">The {company} Team</div>
322
+ </div>
323
+ <PageFooter folio={folio} light mark={false} />
324
+ </div>
325
+ );
326
+ }
327
+
328
+ if (page.kind === "gallery") {
329
+ const known = page.imageCode ? hasAsset(page.imageCode) : null;
330
+ return (
331
+ <div className="cg-cat-page cg-cat-p-gallery" style={style} data-kind="gallery">
332
+ <Shot code={page.imageCode} quality={quality} missing={known === false} cover />
333
+ <div className="cg-cat-gal-wash" />
334
+ {page.title ? <div className="cg-cat-gal-ttl">{page.title}</div> : null}
335
+ <PageFooter folio={folio} light />
336
+ </div>
337
+ );
338
+ }
339
+
340
+ const layout = sectionLayout(page);
341
+ const stripKnown = page.imageCode ? hasAsset(page.imageCode) : null;
342
+ return (
343
+ <div
344
+ className={`cg-cat-page cg-cat-p-section${page.imageCode ? " has-strip" : ""}`}
345
+ style={style}
346
+ data-kind="section"
347
+ >
348
+ <h2 className="cg-cat-sec-ttl">{page.title}</h2>
349
+ <div className="cg-cat-grid" style={{ "--cat-cols": layout.cols } as React.CSSProperties}>
350
+ {items.map((item) => (
351
+ <ProductCard
352
+ key={item.code}
353
+ item={item}
354
+ quality={quality}
355
+ layout={layout}
356
+ hasAsset={hasAsset}
357
+ />
358
+ ))}
359
+ </div>
360
+ {page.imageCode ? (
361
+ <div className="cg-cat-strip">
362
+ <Shot code={page.imageCode} quality={quality} missing={stripKnown === false} cover />
363
+ </div>
364
+ ) : null}
365
+ <PageFooter folio={folio} />
366
+ </div>
367
+ );
368
+ });
369
+
370
+ // ------------------------------------------------------------------- stage
371
+
372
+ function CatalogPages({
373
+ cat,
374
+ itemsFor,
375
+ quality,
376
+ hasAsset,
377
+ scale,
378
+ selectedId,
379
+ onSelect,
380
+ }: {
381
+ cat: CatalogSpec;
382
+ itemsFor: (page: CatalogPage) => CatalogItem[];
383
+ quality: CatalogQuality;
384
+ hasAsset: (code: string) => boolean | null;
385
+ /** Absent = paint at true size (the PRINT tree). */
386
+ scale?: number;
387
+ selectedId?: string;
388
+ onSelect?: (id: string) => void;
389
+ }) {
390
+ const box = pageBox(cat.paper, cat.orientation);
391
+ const px = box.unit === "in" ? 96 : 96 / 25.4;
392
+ return (
393
+ <>
394
+ {cat.pages.map((page, i) => {
395
+ const body = (
396
+ <CatalogPageView
397
+ cat={cat}
398
+ page={page}
399
+ index={i}
400
+ items={itemsFor(page)}
401
+ quality={quality}
402
+ hasAsset={hasAsset}
403
+ />
404
+ );
405
+ if (scale === undefined) return <div key={page.id}>{body}</div>;
406
+ return (
407
+ <div
408
+ key={page.id}
409
+ className={`cg-cat-slot${selectedId === page.id ? " is-sel" : ""}`}
410
+ style={{
411
+ width: box.w * px * scale,
412
+ height: box.h * px * scale,
413
+ "--cat-scale": scale,
414
+ } as React.CSSProperties}
415
+ onMouseDown={() => onSelect?.(page.id)}
416
+ >
417
+ {body}
418
+ </div>
419
+ );
420
+ })}
421
+ </>
422
+ );
423
+ }
424
+
425
+ // ------------------------------------------------------------------- view
426
+
427
+ export const CatalogView = memo(function CatalogView({
428
+ catalogs,
429
+ onCatalogs,
430
+ fields,
431
+ fieldByKey,
432
+ poolRows,
433
+ filteredCodes,
434
+ }: {
435
+ catalogs: CatalogSpec[];
436
+ onCatalogs: (next: CatalogSpec[]) => void;
437
+ fields: Field[];
438
+ fieldByKey: Map<string, Field>;
439
+ /** The WHOLE pool — a catalogue keeps painting what it names after the view
440
+ * is filtered, so the join must not run over the filtered rows. */
441
+ poolRows: Row[];
442
+ /** The codes the CURRENT filter kept — the "add what the view shows" door. */
443
+ filteredCodes: string[];
444
+ }) {
445
+ const [openId, setOpenId] = useState<string | null>(catalogs[0]?.id ?? null);
446
+ const [pageId, setPageId] = useState<string | null>(null);
447
+ const [search, setSearch] = useState("");
448
+ const [picked, setPicked] = useState<Set<string>>(() => new Set());
449
+ const [assetCodes, setAssetCodes] = useState<Set<string> | null>(null);
450
+ const [printing, setPrinting] = useState<{ cat: CatalogSpec; quality: CatalogQuality } | null>(
451
+ null
452
+ );
453
+ const [askQuality, setAskQuality] = useState<CatalogSpec | null>(null);
454
+ const stageRef = useRef<HTMLDivElement | null>(null);
455
+ const printRef = useRef<HTMLDivElement | null>(null);
456
+ const [stageW, setStageW] = useState(0);
457
+
458
+ const cat = catalogs.find((c) => c.id === openId) ?? null;
459
+ const page = cat?.pages.find((p) => p.id === pageId) ?? null;
460
+
461
+ // The manifest, once. `null` means "we do not know" and the designer stays
462
+ // quiet — a false gap report sends the user hunting for images that are there.
463
+ useEffect(() => {
464
+ let live = true;
465
+ void fetchAssetCodes().then((codes) => {
466
+ if (live) setAssetCodes(codes);
467
+ });
468
+ return () => {
469
+ live = false;
470
+ };
471
+ }, []);
472
+
473
+ const hasAsset = useCallback(
474
+ (code: string): boolean | null => (assetCodes ? assetCodes.has(code) : null),
475
+ [assetCodes]
476
+ );
477
+
478
+ useLayoutEffect(() => {
479
+ const el = stageRef.current;
480
+ if (!el || typeof ResizeObserver === "undefined") return;
481
+ const ro = new ResizeObserver(() => setStageW(el.clientWidth));
482
+ ro.observe(el);
483
+ setStageW(el.clientWidth);
484
+ return () => ro.disconnect();
485
+ }, []);
486
+
487
+ const byCode = useMemo(() => indexByCode(poolRows), [poolRows]);
488
+ const itemsFor = useCallback(
489
+ (p: CatalogPage) =>
490
+ resolveItems(p.products ?? [], byCode, cat?.fields, formatDisplay, fieldByKey),
491
+ [byCode, cat?.fields, fieldByKey]
492
+ );
493
+
494
+ const scale = useMemo(() => {
495
+ if (!cat || !stageW) return 0.5;
496
+ const box = pageBox(cat.paper, cat.orientation);
497
+ const px = box.unit === "in" ? 96 : 96 / 25.4;
498
+ return Math.min(1, Math.max(0.2, (stageW - 56) / (box.w * px)));
499
+ }, [cat, stageW]);
500
+
501
+ // ------------------------------------------------------------- mutation
502
+ const write = useCallback(
503
+ (next: CatalogSpec[]) => {
504
+ onCatalogs(next);
505
+ },
506
+ [onCatalogs]
507
+ );
508
+ const patchCat = useCallback(
509
+ (id: string, patch: Partial<CatalogSpec>) => {
510
+ write(catalogs.map((c) => (c.id === id ? { ...c, ...patch } : c)));
511
+ },
512
+ [catalogs, write]
513
+ );
514
+ const patchPage = useCallback(
515
+ (catId: string, pgId: string, patch: Partial<CatalogPage>) => {
516
+ write(
517
+ catalogs.map((c) =>
518
+ c.id !== catId
519
+ ? c
520
+ : { ...c, pages: c.pages.map((p) => (p.id === pgId ? { ...p, ...patch } : p)) }
521
+ )
522
+ );
523
+ },
524
+ [catalogs, write]
525
+ );
526
+
527
+ const addCatalog = () => {
528
+ if (catalogs.length >= MAX_CATALOGS) return;
529
+ const made = newCatalog(`Catalog ${catalogs.length + 1}`, fields);
530
+ write([...catalogs, made]);
531
+ setOpenId(made.id);
532
+ setPageId(made.pages[0]?.id ?? null);
533
+ };
534
+
535
+ const addPage = (kind: CatalogPageKind) => {
536
+ if (!cat || cat.pages.length >= MAX_CATALOG_PAGES) return;
537
+ const made = newPage(kind);
538
+ patchCat(cat.id, { pages: [...cat.pages, made] });
539
+ setPageId(made.id);
540
+ };
541
+
542
+ const movePage = (delta: number) => {
543
+ if (!cat || !page) return;
544
+ const i = cat.pages.findIndex((p) => p.id === page.id);
545
+ const j = i + delta;
546
+ if (i < 0 || j < 0 || j >= cat.pages.length) return;
547
+ const pages = [...cat.pages];
548
+ [pages[i], pages[j]] = [pages[j], pages[i]];
549
+ patchCat(cat.id, { pages });
550
+ };
551
+
552
+ // ------------------------------------------------------------ the picker
553
+ const matches = useMemo(() => {
554
+ if (!cat) return [];
555
+ const q = search.trim().toLowerCase();
556
+ const nameKey = cat.fields?.name;
557
+ const out: { code: string; label: string }[] = [];
558
+ for (const r of poolRows) {
559
+ const code = r[CATALOG_CODE_FIELD];
560
+ if (typeof code !== "string" || !code) continue;
561
+ const nameField = nameKey ? fieldByKey.get(nameKey) : undefined;
562
+ const label = nameField ? formatDisplay(nameField, r[nameKey as string]) : "";
563
+ if (q && !code.toLowerCase().includes(q) && !label.toLowerCase().includes(q)) continue;
564
+ out.push({ code, label });
565
+ if (out.length >= 200) break;
566
+ }
567
+ return out;
568
+ }, [cat, poolRows, search, fieldByKey]);
569
+
570
+ const commitCodes = (codes: string[]) => {
571
+ if (!cat || !page) return;
572
+ const res = addCodes(cat, page.id, codes);
573
+ patchCat(cat.id, { pages: res.pages });
574
+ setPicked(new Set());
575
+ if (res.skippedCap || res.skippedDuplicate) {
576
+ const parts: string[] = [];
577
+ if (res.skippedDuplicate) parts.push(`${res.skippedDuplicate} already on the page`);
578
+ if (res.skippedCap)
579
+ parts.push(`${res.skippedCap} over the ${MAX_CATALOG_CODES}-product catalog limit`);
580
+ setNote(`Added ${res.added}. Skipped ${parts.join("; ")}.`);
581
+ } else {
582
+ setNote(`Added ${res.added}.`);
583
+ }
584
+ };
585
+ const [note, setNote] = useState("");
586
+
587
+ // ------------------------------------------------------------- printing
588
+ useEffect(() => {
589
+ if (!printing) return;
590
+ let cancelled = false;
591
+ void (async () => {
592
+ // One frame so the print tree is in the DOM before its images are awaited.
593
+ await new Promise<void>((res) => requestAnimationFrame(() => res()));
594
+ await awaitImages(printRef.current);
595
+ if (cancelled) return;
596
+ const drop = applyPrintCss(printing.cat);
597
+ await printOnce();
598
+ drop();
599
+ if (!cancelled) setPrinting(null);
600
+ })();
601
+ return () => {
602
+ cancelled = true;
603
+ };
604
+ }, [printing]);
605
+
606
+ // ----------------------------------------------------------------- paint
607
+ if (!catalogs.length)
608
+ return (
609
+ <div className="cg-cat cg-cat-blank">
610
+ <div className="cg-cat-blank-in">
611
+ <CrownMark className="cg-cat-blank-mark" />
612
+ <h3>No catalogs yet</h3>
613
+ <p>
614
+ A catalog is a printable document built from this table&rsquo;s products — cover,
615
+ introduction, product sections and full-page images. Print it to PDF from the browser
616
+ when it is ready.
617
+ </p>
618
+ <button type="button" className="cg-btn cg-btn--primary" onClick={addCatalog}>
619
+ <IconPlus /> New catalog
620
+ </button>
621
+ </div>
622
+ </div>
623
+ );
624
+
625
+ const used = cat ? codesUsed(cat) : 0;
626
+
627
+ return (
628
+ <div className="cg-cat">
629
+ {/* ------------------------------------------------------------ rail */}
630
+ <aside className="cg-cat-rail">
631
+ <div className="cg-cat-rail-head">
632
+ <span>Catalogs</span>
633
+ <button
634
+ type="button"
635
+ className="cg-cat-icon-btn"
636
+ title={
637
+ catalogs.length >= MAX_CATALOGS
638
+ ? `A view holds ${MAX_CATALOGS} catalogs.`
639
+ : "New catalog"
640
+ }
641
+ aria-label="New catalog"
642
+ disabled={catalogs.length >= MAX_CATALOGS}
643
+ onClick={addCatalog}
644
+ >
645
+ <IconPlus />
646
+ </button>
647
+ </div>
648
+ <ul className="cg-cat-list">
649
+ {catalogs.map((c) => (
650
+ <li key={c.id}>
651
+ <button
652
+ type="button"
653
+ className={`cg-cat-item${c.id === openId ? " is-open" : ""}`}
654
+ onClick={() => {
655
+ setOpenId(c.id);
656
+ setPageId(c.pages[0]?.id ?? null);
657
+ }}
658
+ >
659
+ <span className="cg-cat-item-name">{c.name || "Untitled catalog"}</span>
660
+ <span className="cg-cat-item-meta">
661
+ {c.pages.length} {c.pages.length === 1 ? "page" : "pages"}
662
+ </span>
663
+ </button>
664
+ </li>
665
+ ))}
666
+ </ul>
667
+ {cat ? (
668
+ <>
669
+ <div className="cg-cat-rail-head cg-cat-rail-sub">
670
+ <span>Pages</span>
671
+ <span className="cg-cat-count">
672
+ {cat.pages.length}/{MAX_CATALOG_PAGES}
673
+ </span>
674
+ </div>
675
+ <ul className="cg-cat-pagelist">
676
+ {cat.pages.map((p, i) => (
677
+ <li key={p.id}>
678
+ <button
679
+ type="button"
680
+ className={`cg-cat-pageitem${p.id === pageId ? " is-sel" : ""}`}
681
+ onClick={() => setPageId(p.id)}
682
+ >
683
+ <span className="cg-cat-pageno">{i + 1}</span>
684
+ <span className="cg-cat-pagettl">
685
+ {p.title || PAGE_KIND_LABELS[p.kind]}
686
+ </span>
687
+ <span className="cg-cat-pagekind">{PAGE_KIND_LABELS[p.kind]}</span>
688
+ </button>
689
+ </li>
690
+ ))}
691
+ </ul>
692
+ <div className="cg-cat-addrow">
693
+ {CATALOG_PAGE_KINDS.map((k) => (
694
+ <button
695
+ key={k}
696
+ type="button"
697
+ className="cg-btn"
698
+ disabled={cat.pages.length >= MAX_CATALOG_PAGES}
699
+ onClick={() => addPage(k)}
700
+ >
701
+ <IconPlus /> {PAGE_KIND_LABELS[k]}
702
+ </button>
703
+ ))}
704
+ </div>
705
+ </>
706
+ ) : null}
707
+ </aside>
708
+
709
+ {/* ----------------------------------------------------------- stage */}
710
+ <div className="cg-cat-stage" ref={stageRef}>
711
+ {cat ? (
712
+ <>
713
+ <div className="cg-cat-bar">
714
+ <input
715
+ className="cg-input cg-cat-name"
716
+ value={cat.name}
717
+ maxLength={CATALOG_NAME_MAX}
718
+ aria-label="Catalog name"
719
+ onChange={(e) => patchCat(cat.id, { name: e.target.value })}
720
+ />
721
+ <select
722
+ className="cg-select"
723
+ aria-label="Paper size"
724
+ value={cat.paper}
725
+ onChange={(e) => patchCat(cat.id, { paper: e.target.value as CatalogPaper })}
726
+ >
727
+ {CATALOG_PAPERS.map((p) => (
728
+ <option key={p} value={p}>
729
+ {PAPER_LABELS[p]}
730
+ </option>
731
+ ))}
732
+ </select>
733
+ <select
734
+ className="cg-select"
735
+ aria-label="Orientation"
736
+ value={cat.orientation}
737
+ onChange={(e) =>
738
+ patchCat(cat.id, { orientation: e.target.value as CatalogOrientation })
739
+ }
740
+ >
741
+ {CATALOG_ORIENTATIONS.map((o) => (
742
+ <option key={o} value={o}>
743
+ {o === "portrait" ? "Portrait" : "Landscape"}
744
+ </option>
745
+ ))}
746
+ </select>
747
+ <span className="cg-cat-budget" title="Products placed across this catalog's pages">
748
+ {used}/{MAX_CATALOG_CODES} products
749
+ </span>
750
+ <button
751
+ type="button"
752
+ className="cg-btn cg-btn--primary cg-cat-print"
753
+ onClick={() => setAskQuality(cat)}
754
+ >
755
+ <IconPrint /> Print
756
+ </button>
757
+ </div>
758
+ <div className="cg-cat-scroll">
759
+ <CatalogPages
760
+ cat={cat}
761
+ itemsFor={itemsFor}
762
+ quality="web"
763
+ hasAsset={hasAsset}
764
+ scale={scale}
765
+ selectedId={pageId ?? undefined}
766
+ onSelect={setPageId}
767
+ />
768
+ </div>
769
+ </>
770
+ ) : (
771
+ <div className="cg-mode-empty">Pick a catalog to open it.</div>
772
+ )}
773
+ </div>
774
+
775
+ {/* ------------------------------------------------------- inspector */}
776
+ <aside className="cg-cat-insp">
777
+ {cat && page ? (
778
+ <>
779
+ <div className="cg-cat-insp-head">
780
+ <span>{PAGE_KIND_LABELS[page.kind]}</span>
781
+ <span className="cg-cat-insp-tools">
782
+ <button
783
+ type="button"
784
+ className="cg-cat-icon-btn"
785
+ aria-label="Move page up"
786
+ onClick={() => movePage(-1)}
787
+ >
788
+ <IconUp />
789
+ </button>
790
+ <button
791
+ type="button"
792
+ className="cg-cat-icon-btn"
793
+ aria-label="Move page down"
794
+ onClick={() => movePage(1)}
795
+ >
796
+ <IconDown />
797
+ </button>
798
+ <button
799
+ type="button"
800
+ className="cg-cat-icon-btn cg-cat-del"
801
+ aria-label="Delete page"
802
+ onClick={() =>
803
+ patchCat(cat.id, { pages: cat.pages.filter((p) => p.id !== page.id) })
804
+ }
805
+ >
806
+ <IconTrash />
807
+ </button>
808
+ </span>
809
+ </div>
810
+
811
+ <label className="cg-cat-fld">
812
+ <span>Title</span>
813
+ <input
814
+ className="cg-input"
815
+ value={page.title ?? ""}
816
+ onChange={(e) => patchPage(cat.id, page.id, { title: e.target.value })}
817
+ />
818
+ </label>
819
+
820
+ {page.kind === "cover" || page.kind === "intro" ? (
821
+ <label className="cg-cat-fld">
822
+ <span>{page.kind === "cover" ? "Second line" : "Body"}</span>
823
+ <textarea
824
+ className="cg-input cg-cat-area"
825
+ rows={page.kind === "cover" ? 2 : 9}
826
+ value={page.body ?? ""}
827
+ onChange={(e) => patchPage(cat.id, page.id, { body: e.target.value })}
828
+ />
829
+ </label>
830
+ ) : null}
831
+
832
+ {page.kind !== "intro" ? (
833
+ <label className="cg-cat-fld">
834
+ <span>{page.kind === "section" ? "Footer image (code)" : "Image (code)"}</span>
835
+ <input
836
+ className="cg-input"
837
+ value={page.imageCode ?? ""}
838
+ placeholder="e.g. CRY-2003"
839
+ onChange={(e) => patchPage(cat.id, page.id, { imageCode: e.target.value })}
840
+ />
841
+ {page.imageCode && hasAsset(page.imageCode) === false ? (
842
+ <em className="cg-cat-warn">
843
+ No image is archived under this code — the page will print an empty frame.
844
+ </em>
845
+ ) : (
846
+ <em className="cg-cat-hint">
847
+ Fills the width edge to edge, so a wide photograph reads best here. A cut-out
848
+ product render will be cropped and enlarged to fill it.
849
+ </em>
850
+ )}
851
+ </label>
852
+ ) : null}
853
+
854
+ {page.kind === "section" ? (
855
+ <>
856
+ <div className="cg-cat-insp-sub">Layout</div>
857
+ <div className="cg-cat-row">
858
+ <label className="cg-cat-fld cg-cat-fld-inline">
859
+ <span>Columns</span>
860
+ <select
861
+ className="cg-select"
862
+ value={sectionLayout(page).cols}
863
+ onChange={(e) =>
864
+ patchPage(cat.id, page.id, {
865
+ layout: { ...page.layout, cols: Number(e.target.value) },
866
+ })
867
+ }
868
+ >
869
+ {CATALOG_COLS.map((c) => (
870
+ <option key={c} value={c}>
871
+ {c}
872
+ </option>
873
+ ))}
874
+ </select>
875
+ </label>
876
+ </div>
877
+ {(
878
+ [
879
+ ["showPack", "Pack line", false],
880
+ ["showColor", "Colour", false],
881
+ ["showPrice", "Price", true],
882
+ ] as const
883
+ ).map(([key, label, optIn]) => {
884
+ const l = sectionLayout(page);
885
+ const on = key === "showPrice" ? l.showPrice : key === "showPack" ? l.showPack : l.showColor;
886
+ return (
887
+ <label key={key} className="cg-check-row">
888
+ <input
889
+ type="checkbox"
890
+ checked={on}
891
+ onChange={() => {
892
+ // Literal-only storage: the toggle writes the opt-out (or the
893
+ // opt-in) and DELETES the key to go back to the default, so the
894
+ // default has exactly one spelling on the wire.
895
+ const next = { ...page.layout };
896
+ if (optIn) {
897
+ if (on) delete next.showPrice;
898
+ else next.showPrice = true;
899
+ } else if (key === "showPack") {
900
+ if (on) next.showPack = false;
901
+ else delete next.showPack;
902
+ } else {
903
+ if (on) next.showColor = false;
904
+ else delete next.showColor;
905
+ }
906
+ patchPage(cat.id, page.id, {
907
+ layout: Object.keys(next).length ? next : undefined,
908
+ });
909
+ }}
910
+ />
911
+ <span>{label}</span>
912
+ </label>
913
+ );
914
+ })}
915
+
916
+ <div className="cg-cat-insp-sub">
917
+ Products
918
+ <span className="cg-cat-count">{page.products?.length ?? 0} on this page</span>
919
+ </div>
920
+ <div className="cg-cat-picked">
921
+ {(page.products ?? []).map((code) => (
922
+ <span key={code} className="cg-cat-chip">
923
+ {code}
924
+ <button
925
+ type="button"
926
+ aria-label={`Remove ${code}`}
927
+ onClick={() =>
928
+ patchPage(cat.id, page.id, {
929
+ products: (page.products ?? []).filter((c) => c !== code),
930
+ })
931
+ }
932
+ >
933
+ &times;
934
+ </button>
935
+ </span>
936
+ ))}
937
+ </div>
938
+ <button
939
+ type="button"
940
+ className="cg-btn cg-cat-addfiltered"
941
+ disabled={!filteredCodes.length}
942
+ onClick={() => commitCodes(filteredCodes)}
943
+ >
944
+ <IconPlus /> Add the {filteredCodes.length} products this view shows
945
+ </button>
946
+ <input
947
+ className="cg-input"
948
+ placeholder="Search products by SKU or name"
949
+ value={search}
950
+ onChange={(e) => setSearch(e.target.value)}
951
+ />
952
+ <div className="cg-cat-results">
953
+ {matches.map((m) => (
954
+ <label key={m.code} className="cg-check-row">
955
+ <input
956
+ type="checkbox"
957
+ checked={picked.has(m.code)}
958
+ onChange={() =>
959
+ setPicked((prev) => {
960
+ const next = new Set(prev);
961
+ if (next.has(m.code)) next.delete(m.code);
962
+ else next.add(m.code);
963
+ return next;
964
+ })
965
+ }
966
+ />
967
+ <span className="cg-cat-res-code">{m.code}</span>
968
+ <span className="cg-cat-res-name">{m.label}</span>
969
+ </label>
970
+ ))}
971
+ {/* The 200 is STATED, never silent: a search that stops early has
972
+ to say so or the user reads absence as "we do not stock it". */}
973
+ {matches.length >= 200 ? (
974
+ <div className="cg-cat-more">
975
+ Showing the first 200 matches — narrow the search to see the rest.
976
+ </div>
977
+ ) : null}
978
+ </div>
979
+ <button
980
+ type="button"
981
+ className="cg-btn cg-btn--primary"
982
+ disabled={!picked.size}
983
+ onClick={() => commitCodes([...picked])}
984
+ >
985
+ Add {picked.size} selected
986
+ </button>
987
+ {note ? <div className="cg-cat-note">{note}</div> : null}
988
+ </>
989
+ ) : null}
990
+
991
+ <div className="cg-cat-insp-sub">Brand</div>
992
+ <div className="cg-cat-row">
993
+ <label className="cg-cat-fld cg-cat-fld-inline">
994
+ <span>Primary</span>
995
+ <input
996
+ type="color"
997
+ className="cg-cat-swatch"
998
+ value={cat.brand?.primary ?? DEFAULT_BRAND.primary}
999
+ onChange={(e) =>
1000
+ patchCat(cat.id, { brand: { ...cat.brand, primary: e.target.value } })
1001
+ }
1002
+ />
1003
+ </label>
1004
+ <label className="cg-cat-fld cg-cat-fld-inline">
1005
+ <span>Accent</span>
1006
+ <input
1007
+ type="color"
1008
+ className="cg-cat-swatch"
1009
+ value={cat.brand?.accent ?? DEFAULT_BRAND.accent}
1010
+ onChange={(e) =>
1011
+ patchCat(cat.id, { brand: { ...cat.brand, accent: e.target.value } })
1012
+ }
1013
+ />
1014
+ </label>
1015
+ </div>
1016
+ <label className="cg-cat-fld">
1017
+ <span>Company</span>
1018
+ <input
1019
+ className="cg-input"
1020
+ value={cat.brand?.company ?? ""}
1021
+ placeholder={DEFAULT_BRAND.company}
1022
+ onChange={(e) =>
1023
+ patchCat(cat.id, { brand: { ...cat.brand, company: e.target.value } })
1024
+ }
1025
+ />
1026
+ </label>
1027
+
1028
+ <div className="cg-cat-insp-sub">Listing columns</div>
1029
+ {(
1030
+ [
1031
+ ["name", "Description"],
1032
+ ["pack", "Pack line"],
1033
+ ["color", "Colour"],
1034
+ ["price", "Price"],
1035
+ ] as const
1036
+ ).map(([slot, label]) => (
1037
+ <label key={slot} className="cg-cat-fld cg-cat-fld-inline">
1038
+ <span>{label}</span>
1039
+ <select
1040
+ className="cg-select"
1041
+ value={cat.fields?.[slot] ?? ""}
1042
+ onChange={(e) => {
1043
+ const next = { ...cat.fields };
1044
+ if (e.target.value) next[slot] = e.target.value;
1045
+ else delete next[slot];
1046
+ patchCat(cat.id, { fields: Object.keys(next).length ? next : undefined });
1047
+ }}
1048
+ >
1049
+ {/* An explicit empty option — the wave-7 trap: a <select> whose value
1050
+ is not among its options renders the FIRST one, so "unbound" needs a
1051
+ row of its own rather than being the absence of one. */}
1052
+ <option value="">Not shown</option>
1053
+ {fields.map((f) => (
1054
+ <option key={f.key} value={f.key}>
1055
+ {f.label}
1056
+ </option>
1057
+ ))}
1058
+ </select>
1059
+ </label>
1060
+ ))}
1061
+ <button
1062
+ type="button"
1063
+ className="cg-btn"
1064
+ onClick={() => {
1065
+ const guess = autoBind(fields);
1066
+ patchCat(cat.id, { fields: Object.keys(guess).length ? guess : undefined });
1067
+ }}
1068
+ >
1069
+ Guess from this table
1070
+ </button>
1071
+ <div className="cg-cat-insp-foot">
1072
+ <button
1073
+ type="button"
1074
+ className="cg-btn cg-btn--danger"
1075
+ onClick={() => {
1076
+ const next = catalogs.filter((c) => c.id !== cat.id);
1077
+ write(next);
1078
+ setOpenId(next[0]?.id ?? null);
1079
+ setPageId(null);
1080
+ }}
1081
+ >
1082
+ <IconTrash /> Delete this catalog
1083
+ </button>
1084
+ </div>
1085
+ </>
1086
+ ) : (
1087
+ <div className="cg-cat-insp-blank">Select a page to edit it.</div>
1088
+ )}
1089
+ </aside>
1090
+
1091
+ {/* --------------------------------------------------- quality prompt */}
1092
+ {askQuality ? (
1093
+ <div className="cg-cat-modal-wrap" role="dialog" aria-modal="true" aria-label="Print quality">
1094
+ <div className="cg-cat-modal">
1095
+ <h3>Print &ldquo;{askQuality.name || "Untitled catalog"}&rdquo;</h3>
1096
+ <p>
1097
+ Your browser&rsquo;s print dialog opens next — choose &ldquo;Save as PDF&rdquo; there.
1098
+ Pick the image quality first.
1099
+ </p>
1100
+ <button
1101
+ type="button"
1102
+ className="cg-cat-choice"
1103
+ onClick={() => {
1104
+ setPrinting({ cat: askQuality, quality: "print" });
1105
+ setAskQuality(null);
1106
+ }}
1107
+ >
1108
+ <strong>High quality</strong>
1109
+ <em>Original master images. Slower, and a much larger PDF — use this for print.</em>
1110
+ </button>
1111
+ <button
1112
+ type="button"
1113
+ className="cg-cat-choice"
1114
+ onClick={() => {
1115
+ setPrinting({ cat: askQuality, quality: "web" });
1116
+ setAskQuality(null);
1117
+ }}
1118
+ >
1119
+ <strong>Standard</strong>
1120
+ <em>The same images you see here. Fast, small file — fine for a proof or email.</em>
1121
+ </button>
1122
+ <button type="button" className="cg-btn" onClick={() => setAskQuality(null)}>
1123
+ Cancel
1124
+ </button>
1125
+ </div>
1126
+ </div>
1127
+ ) : null}
1128
+
1129
+ {/* ------------------------------------------------------ print tree */}
1130
+ {printing
1131
+ ? createPortal(
1132
+ <div className="cg-cat-print-portal" ref={printRef}>
1133
+ <CatalogPages
1134
+ cat={printing.cat}
1135
+ itemsFor={itemsFor}
1136
+ quality={printing.quality}
1137
+ hasAsset={hasAsset}
1138
+ />
1139
+ </div>,
1140
+ document.body
1141
+ )
1142
+ : null}
1143
+ </div>
1144
+ );
1145
+ });
1146
+
1147
+ export default CatalogView;
web/src/customer-grid/ColumnMenu.tsx CHANGED
@@ -26,6 +26,9 @@ interface FieldConfigExtra {
26
  label?: string;
27
  colorCodeOptions?: boolean;
28
  optionColors?: Record<string, string>;
 
 
 
29
  }
30
 
31
  interface ColumnMenuProps {
@@ -557,10 +560,26 @@ const NEW_PREFIX = "new:";
557
  * (source 'odoo', host-computed shape) — turning an overlay column into one is not a retype,
558
  * and the host accepts in-place rebuilds on `custom_` overlay defs only.
559
  */
 
 
 
 
 
560
  const RETYPE_TYPES = CREATABLE_TYPES.filter(
561
- (t) => t !== "formula" && t !== "created_time"
562
  );
563
 
 
 
 
 
 
 
 
 
 
 
 
564
  const DEFAULT_MEASURE_WINDOW: WindowSpec = { kind: "ltm" };
565
 
566
  /** The head every pane shares: title line, context line, close X. */
@@ -647,6 +666,10 @@ function ExtraTypeEditor({
647
  onMaxValue,
648
  idPrefix,
649
  showFormulaHelp = true,
 
 
 
 
650
  }: {
651
  kind: FieldType | "measure";
652
  fields: Field[];
@@ -657,6 +680,12 @@ function ExtraTypeEditor({
657
  onMaxValue: (n: number) => void;
658
  idPrefix: string;
659
  showFormulaHelp?: boolean;
 
 
 
 
 
 
660
  }) {
661
  if (kind === "formula") {
662
  // 2026-07-31 (owner item 2): other FORMULA fields are referencable now — evaluation is
@@ -733,6 +762,67 @@ function ExtraTypeEditor({
733
  </div>
734
  );
735
  }
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
736
  return null;
737
  }
738
 
@@ -794,6 +884,11 @@ export default function ColumnMenu({
794
  /** Wave-5 item 9/11 — the create form's formula source and rating star count. */
795
  const [formulaText, setFormulaText] = useState("");
796
  const [maxValue, setMaxValue] = useState(5);
 
 
 
 
 
797
  const [swapTo, setSwapTo] = useState("");
798
  /** The "New field" half of Change-field keeps its own name/options — a half-typed insert
799
  * form must not leak into a swap and vice versa. */
@@ -834,6 +929,13 @@ export default function ColumnMenu({
834
  const [editFormulaText, setEditFormulaText] = useState(() =>
835
  typeof field.formula === "string" ? field.formula : ""
836
  );
 
 
 
 
 
 
 
837
 
838
  const swapToNewType = swapTo.startsWith(NEW_PREFIX)
839
  ? (swapTo.slice(NEW_PREFIX.length) as FieldType)
@@ -898,7 +1000,11 @@ export default function ColumnMenu({
898
  effectiveLabel.trim() !== "" &&
899
  (!needsOptions(kind) || options.length > 0) &&
900
  (kind !== "measure" || !!chosenMeasure) &&
901
- (kind !== "formula" || formulaCheck.ok);
 
 
 
 
902
 
903
  const extraFor = (
904
  t: CreateKind,
@@ -907,6 +1013,14 @@ export default function ColumnMenu({
907
  ): FieldConfigExtra | undefined => {
908
  if (t === "formula") return { formula: src.trim() };
909
  if (t === "rating") return { max: starCount };
 
 
 
 
 
 
 
 
910
  return undefined;
911
  };
912
  const optionExtraFor = (
@@ -981,10 +1095,20 @@ export default function ColumnMenu({
981
  const editStarsChanged = editType === "rating" && retypeMaxValue !== ratingMax(field);
982
  /** The type half was touched: a different type, or the SAME type with edited choices/stars
983
  * (the owner's named case — "change the list of the single selections"). */
 
 
 
 
 
 
 
984
  const editRetypeTouched =
985
  onRetype != null &&
986
- (editType !== field.type || editOptionsChanged || editAppearanceChanged || editStarsChanged);
987
- const editRetypeValid = !needsOptions(editType) || editOptions.length > 0;
 
 
 
988
  const editFormulaCheck = useMemo(
989
  () =>
990
  editFormulaText.trim() === ""
@@ -1032,6 +1156,15 @@ export default function ColumnMenu({
1032
  : {}),
1033
  }
1034
  : {}),
 
 
 
 
 
 
 
 
 
1035
  ...(editNameDirty ? { label: cleanRename } : {}),
1036
  }
1037
  );
@@ -1190,6 +1323,10 @@ export default function ColumnMenu({
1190
  maxValue={maxValue}
1191
  onMaxValue={setMaxValue}
1192
  idPrefix="cg-create"
 
 
 
 
1193
  />
1194
  {needsOptions(kind) && (
1195
  <OptionsEditor
@@ -1276,7 +1413,32 @@ export default function ColumnMenu({
1276
  </span>
1277
  </label>
1278
  )}
1279
- {onRetype && (
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1280
  <>
1281
  <div className="cg-type-block">
1282
  <span className="cg-type-title">Field type</span>
@@ -1326,7 +1488,7 @@ export default function ColumnMenu({
1326
  </div>
1327
  )}
1328
  </>
1329
- )}
1330
  {/* Item 21 (owner, 2026-08-02) — the PERIOD lives here, with the field's other
1331
  properties, instead of floating above the action list on the menu pane. A metric
1332
  column IS "a number over a period": the window is as much a part of its
 
26
  label?: string;
27
  colorCodeOptions?: boolean;
28
  optionColors?: Record<string, string>;
29
+ /** Wave-18 C5-AUTOFIELD — an automation column's configuration, the same way `formula`
30
+ * carries an expression. Validated host-side (`aios_grid._clean_automation`). */
31
+ automation?: { kind: string; source: "record_url_field" | "self"; urlField?: string };
32
  }
33
 
34
  interface ColumnMenuProps {
 
560
  * (source 'odoo', host-computed shape) — turning an overlay column into one is not a retype,
561
  * and the host accepts in-place rebuilds on `custom_` overlay defs only.
562
  */
563
+ // Wave-18 C5-AUTOFIELD: `automation` joins the exclusion, and for a reason the two above do not
564
+ // share. It IS an overlay type, so the host would accept the retype — which is exactly the
565
+ // hazard. Retyping a Notes column to Automation hands every row's typed value to the next run to
566
+ // overwrite; retyping the other way strands machine-written statuses in a column a person can
567
+ // now edit. Neither is a change of TYPE, both are data loss with a tidy name.
568
  const RETYPE_TYPES = CREATABLE_TYPES.filter(
569
+ (t) => t !== "formula" && t !== "created_time" && t !== "automation"
570
  );
571
 
572
+ /** The automation kinds a column can carry. v1 is one; the shape is a list so the second is a
573
+ * line rather than a refactor. Mirrors `aios_grid.AUTOMATION_KINDS` (host-whitelisted). */
574
+ const AUTOMATION_KINDS: { key: string; label: string; hint: string }[] = [
575
+ {
576
+ key: "instagram_profile",
577
+ label: "Instagram profile",
578
+ hint: "Reads what is anonymously public about the profile in the chosen URL column, and "
579
+ + "writes the result here. Runs from the Automation surface — on demand or on a schedule.",
580
+ },
581
+ ];
582
+
583
  const DEFAULT_MEASURE_WINDOW: WindowSpec = { kind: "ltm" };
584
 
585
  /** The head every pane shares: title line, context line, close X. */
 
666
  onMaxValue,
667
  idPrefix,
668
  showFormulaHelp = true,
669
+ automationKind = "instagram_profile",
670
+ onAutomationKind,
671
+ automationUrlField = "",
672
+ onAutomationUrlField,
673
  }: {
674
  kind: FieldType | "measure";
675
  fields: Field[];
 
680
  onMaxValue: (n: number) => void;
681
  idPrefix: string;
682
  showFormulaHelp?: boolean;
683
+ /** C5-AUTOFIELD — the automation column's config drafts. Optional so the call sites that
684
+ * cannot reach an automation type (the Change-field "new field" form) need no change. */
685
+ automationKind?: string;
686
+ onAutomationKind?: (v: string) => void;
687
+ automationUrlField?: string;
688
+ onAutomationUrlField?: (v: string) => void;
689
  }) {
690
  if (kind === "formula") {
691
  // 2026-07-31 (owner item 2): other FORMULA fields are referencable now — evaluation is
 
762
  </div>
763
  );
764
  }
765
+ if (kind === "automation") {
766
+ // C5-AUTOFIELD — the field's GEAR. An automation column has no value the user types, so
767
+ // this pane configures what a RUN does instead: which job, and which column carries the
768
+ // thing it reads. The schedule and the Run-now button are NOT here — they belong to the
769
+ // automation itself, on the Automation surface, because one job can fill a column across
770
+ // many records and its cadence is a property of the job, not of the column.
771
+ const urlish = fields.filter((f) => f.type === "url" || f.type === "text");
772
+ const chosen = AUTOMATION_KINDS.find((k) => k.key === automationKind)
773
+ ?? AUTOMATION_KINDS[0];
774
+ return (
775
+ <>
776
+ <label>
777
+ <span>What it reads</span>
778
+ <select
779
+ id={`${idPrefix}-automation-kind`}
780
+ className="cg-input"
781
+ value={automationKind}
782
+ aria-label="Automation kind"
783
+ onChange={(event) => onAutomationKind?.(event.target.value)}
784
+ >
785
+ {AUTOMATION_KINDS.map((k) => (
786
+ <option key={k.key} value={k.key}>
787
+ {k.label}
788
+ </option>
789
+ ))}
790
+ </select>
791
+ </label>
792
+ <label>
793
+ <span>URL column</span>
794
+ <select
795
+ id={`${idPrefix}-automation-url`}
796
+ className="cg-input"
797
+ value={automationUrlField}
798
+ aria-label="URL column"
799
+ onChange={(event) => onAutomationUrlField?.(event.target.value)}
800
+ >
801
+ <option value="">Choose a column…</option>
802
+ {urlish.map((f) => (
803
+ <option key={f.key} value={f.key}>
804
+ {f.label}
805
+ </option>
806
+ ))}
807
+ </select>
808
+ </label>
809
+ <div className="cg-field-hint">{chosen.hint}</div>
810
+ <div className="cg-field-hint">
811
+ Cells here are written by the run and cannot be typed into — a value you entered
812
+ would be replaced by the next run without saying so.
813
+ </div>
814
+ {/* A v1 LIMIT, said out loud rather than discovered. The column can be created on any
815
+ table, but an Instagram automation only accepts a blank database as its target
816
+ (`clean_config('field_instagram')`), because the run writes rows and an Odoo-sourced
817
+ table is read-only at the source. Without this line a user creates the column on
818
+ Customer, goes to Automation, and finds no way to point anything at it. */}
819
+ <div className="cg-field-hint">
820
+ Runs target a blank database. On a connector-backed table this column can be created
821
+ but no automation can write to it yet.
822
+ </div>
823
+ </>
824
+ );
825
+ }
826
  return null;
827
  }
828
 
 
884
  /** Wave-5 item 9/11 — the create form's formula source and rating star count. */
885
  const [formulaText, setFormulaText] = useState("");
886
  const [maxValue, setMaxValue] = useState(5);
887
+ /** C5-AUTOFIELD — the create form's automation config. */
888
+ const [automationKind, setAutomationKind] = useState(AUTOMATION_KINDS[0].key);
889
+ const [automationUrlField, setAutomationUrlField] = useState(
890
+ () => fields.find((f) => f.type === "url")?.key ?? ""
891
+ );
892
  const [swapTo, setSwapTo] = useState("");
893
  /** The "New field" half of Change-field keeps its own name/options — a half-typed insert
894
  * form must not leak into a swap and vice versa. */
 
929
  const [editFormulaText, setEditFormulaText] = useState(() =>
930
  typeof field.formula === "string" ? field.formula : ""
931
  );
932
+ /** C5-AUTOFIELD — the Edit pane's automation drafts, seeded from the field's own config. */
933
+ const [editAutomationKind, setEditAutomationKind] = useState(
934
+ () => field.automation?.kind ?? AUTOMATION_KINDS[0].key
935
+ );
936
+ const [editAutomationUrlField, setEditAutomationUrlField] = useState(
937
+ () => field.automation?.urlField ?? ""
938
+ );
939
 
940
  const swapToNewType = swapTo.startsWith(NEW_PREFIX)
941
  ? (swapTo.slice(NEW_PREFIX.length) as FieldType)
 
1000
  effectiveLabel.trim() !== "" &&
1001
  (!needsOptions(kind) || options.length > 0) &&
1002
  (kind !== "measure" || !!chosenMeasure) &&
1003
+ (kind !== "formula" || formulaCheck.ok) &&
1004
+ // C5-AUTOFIELD: an automation column with no URL column to read is a column no run can
1005
+ // ever fill — the same "permanently blank column" the formula rule above refuses, so it
1006
+ // is refused here rather than created and then discovered.
1007
+ (kind !== "automation" || automationUrlField !== "");
1008
 
1009
  const extraFor = (
1010
  t: CreateKind,
 
1013
  ): FieldConfigExtra | undefined => {
1014
  if (t === "formula") return { formula: src.trim() };
1015
  if (t === "rating") return { max: starCount };
1016
+ if (t === "automation")
1017
+ return {
1018
+ automation: {
1019
+ kind: automationKind,
1020
+ source: "record_url_field",
1021
+ urlField: automationUrlField,
1022
+ },
1023
+ };
1024
  return undefined;
1025
  };
1026
  const optionExtraFor = (
 
1095
  const editStarsChanged = editType === "rating" && retypeMaxValue !== ratingMax(field);
1096
  /** The type half was touched: a different type, or the SAME type with edited choices/stars
1097
  * (the owner's named case — "change the list of the single selections"). */
1098
+ /** C5-AUTOFIELD — the automation config is a property of the column exactly as the choice
1099
+ * list is, so a change to it makes the type half DIRTY and rides the same one upsert. */
1100
+ const editAutomationChanged =
1101
+ editType === "automation" &&
1102
+ field.type === "automation" &&
1103
+ (editAutomationKind !== (field.automation?.kind ?? AUTOMATION_KINDS[0].key) ||
1104
+ editAutomationUrlField !== (field.automation?.urlField ?? ""));
1105
  const editRetypeTouched =
1106
  onRetype != null &&
1107
+ (editType !== field.type || editOptionsChanged || editAppearanceChanged ||
1108
+ editStarsChanged || editAutomationChanged);
1109
+ const editRetypeValid =
1110
+ (!needsOptions(editType) || editOptions.length > 0) &&
1111
+ (editType !== "automation" || editAutomationUrlField !== "");
1112
  const editFormulaCheck = useMemo(
1113
  () =>
1114
  editFormulaText.trim() === ""
 
1156
  : {}),
1157
  }
1158
  : {}),
1159
+ ...(editType === "automation"
1160
+ ? {
1161
+ automation: {
1162
+ kind: editAutomationKind,
1163
+ source: "record_url_field" as const,
1164
+ urlField: editAutomationUrlField,
1165
+ },
1166
+ }
1167
+ : {}),
1168
  ...(editNameDirty ? { label: cleanRename } : {}),
1169
  }
1170
  );
 
1323
  maxValue={maxValue}
1324
  onMaxValue={setMaxValue}
1325
  idPrefix="cg-create"
1326
+ automationKind={automationKind}
1327
+ onAutomationKind={setAutomationKind}
1328
+ automationUrlField={automationUrlField}
1329
+ onAutomationUrlField={setAutomationUrlField}
1330
  />
1331
  {needsOptions(kind) && (
1332
  <OptionsEditor
 
1413
  </span>
1414
  </label>
1415
  )}
1416
+ {/* C5-AUTOFIELD — an EXISTING automation column edits its CONFIG, not its type.
1417
+ ⚠ This guard is load-bearing, not tidiness. `automation` is excluded from
1418
+ RETYPE_TYPES (see its note), so rendering the picker for one would show a
1419
+ listbox whose current value is not among its options — the failure recorded in
1420
+ [[cg-condition-builder-items]], where a control missing its own value silently
1421
+ presents the FIRST option as if it were the selection. One Save later the column
1422
+ is a different type and its statuses are editable text. */}
1423
+ {onRetype && field.type === "automation" ? (
1424
+ <div className="cg-type-block">
1425
+ <span className="cg-type-title">Automation</span>
1426
+ <ExtraTypeEditor
1427
+ kind="automation"
1428
+ fields={fields}
1429
+ formulaText={editFormulaText}
1430
+ onFormulaText={setEditFormulaText}
1431
+ formulaError={null}
1432
+ maxValue={retypeMaxValue}
1433
+ onMaxValue={setRetypeMaxValue}
1434
+ idPrefix="cg-edit"
1435
+ automationKind={editAutomationKind}
1436
+ onAutomationKind={setEditAutomationKind}
1437
+ automationUrlField={editAutomationUrlField}
1438
+ onAutomationUrlField={setEditAutomationUrlField}
1439
+ />
1440
+ </div>
1441
+ ) : onRetype ? (
1442
  <>
1443
  <div className="cg-type-block">
1444
  <span className="cg-type-title">Field type</span>
 
1488
  </div>
1489
  )}
1490
  </>
1491
+ ) : null}
1492
  {/* Item 21 (owner, 2026-08-02) — the PERIOD lives here, with the field's other
1493
  properties, instead of floating above the action list on the menu pane. A metric
1494
  column IS "a number over a period": the window is as much a part of its
web/src/customer-grid/CustomerGrid.tsx CHANGED
@@ -59,6 +59,11 @@ import { DashboardView } from "../viz/DashboardView";
59
  import { cleanCharts } from "../viz/chartData";
60
  import type { ChartSpec } from "../viz/chartData";
61
  import { MapView } from "./MapView";
 
 
 
 
 
62
  import { lightTheme, STATUS_ROW_THEME, HOVER_ROW_THEME, HOVER_NEUTRAL,
63
  ACTIVE_ROW_NEUTRAL, CUSTOM_FIELD_MARK } from "./theme";
64
  import { avatarInitials, formatDisplay, ratingCellRenderer, setAvatarRepaint, userCellRenderer }
@@ -184,6 +189,54 @@ function measureGroupText(text: string): number {
184
  const DISPLAY_PAGE = 50;
185
  const DISPLAY_STEP = 250;
186
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
187
  interface LocalWorkspace {
188
  fields: Field[];
189
  views: SavedView[];
@@ -1197,6 +1250,53 @@ function CustomerGrid({ scope = "customer" }: { scope?: SurfaceScope } = {}) {
1197
  [config.groupBy, rowPx, displayRows]
1198
  );
1199
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1200
  // The record drawer resolves positions against what the MODE paints: the display slice for
1201
  // grid/list (the cap is real there), the FULL pipeline for calendar/kanban/map (a month, a
1202
  // stack or a pin reaches past the cap by design — a drawer that refused those pids would
@@ -2143,6 +2243,22 @@ function CustomerGrid({ scope = "customer" }: { scope?: SurfaceScope } = {}) {
2143
  return;
2144
  }
2145
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
2146
  if (spec?.mode === "calendar" && spec.calendarMode === "summary") {
2147
  const sheet = isActive ? calSheetRef.current : null;
2148
  if (!sheet) return refuse("a calendar summary");
@@ -2762,7 +2878,13 @@ function CustomerGrid({ scope = "customer" }: { scope?: SurfaceScope } = {}) {
2762
  // C-TS close-out stitch: the panel's pids come from these rows; without this line the
2763
  // time-series view asked its question about ZERO customers (caught by the mounted
2764
  // judge — the node battery is blind here).
2765
- displayMode !== "timeseries"
 
 
 
 
 
 
2766
  )
2767
  return [];
2768
  const seen = new Set<number>();
@@ -2816,6 +2938,41 @@ function CustomerGrid({ scope = "customer" }: { scope?: SurfaceScope } = {}) {
2816
  () => (displayMode === "timeseries" ? modeDataRows.map((r) => r.pid) : []),
2817
  [displayMode, modeDataRows]
2818
  );
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
2819
  /** Persist a bucket/span/metric pick onto the view, the way every other display key rides. */
2820
  const onTimeseriesDisplay = useCallback(
2821
  (next: Partial<DisplaySpec>) => {
@@ -3151,6 +3308,7 @@ function CustomerGrid({ scope = "customer" }: { scope?: SurfaceScope } = {}) {
3151
  onMouseLeave={() => setExpandAt((prev) => (prev === null ? prev : null))}
3152
  >
3153
  {displayMode === "grid" && (
 
3154
  <DataEditor
3155
  ref={gridRef}
3156
  columns={visibleCols}
@@ -3173,6 +3331,14 @@ function CustomerGrid({ scope = "customer" }: { scope?: SurfaceScope } = {}) {
3173
  rowHeight={rowHeight}
3174
  headerHeight={36}
3175
  rowMarkers="both"
 
 
 
 
 
 
 
 
3176
  gridSelection={gridSelection}
3177
  onGridSelectionChange={onGridSelectionChange}
3178
  getRowThemeOverride={getRowThemeOverride}
@@ -3225,6 +3391,46 @@ function CustomerGrid({ scope = "customer" }: { scope?: SurfaceScope } = {}) {
3225
  }
3226
  rightElementProps={{ sticky: true, fill: false }}
3227
  />
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
3228
  )}
3229
  {displayMode === "list" && (
3230
  <ListView
@@ -3313,6 +3519,35 @@ function CustomerGrid({ scope = "customer" }: { scope?: SurfaceScope } = {}) {
3313
  onSheet={onTsSheet}
3314
  />
3315
  )}
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
3316
  {/* Wave-7 item W11c (C3) — Map is a VIEW: same pipeline rows, pins for rows
3317
  with host-supplied coords, click-through to the same record detail, and
3318
  an honest no-location chip. */}
 
59
  import { cleanCharts } from "../viz/chartData";
60
  import type { ChartSpec } from "../viz/chartData";
61
  import { MapView } from "./MapView";
62
+ /* ═══ W18-C CATALOG ═══ (owner item 4, contract C6) */
63
+ import { CatalogView } from "./CatalogView";
64
+ import { CATALOG_CODE_FIELD } from "./catalogData";
65
+ import type { CatalogSpec } from "./types";
66
+ /* ═══ end W18-C CATALOG ═══ */
67
  import { lightTheme, STATUS_ROW_THEME, HOVER_ROW_THEME, HOVER_NEUTRAL,
68
  ACTIVE_ROW_NEUTRAL, CUSTOM_FIELD_MARK } from "./theme";
69
  import { avatarInitials, formatDisplay, ratingCellRenderer, setAvatarRepaint, userCellRenderer }
 
189
  const DISPLAY_PAGE = 50;
190
  const DISPLAY_STEP = 250;
191
 
192
+ /* ═══════════════════════════════════════════════════════════════════════════════════════════
193
+ ═══ W18-B VOID ═══ (wave 18, owner item 1b) — the two constants the void geometry needs
194
+ that belong to the BROWSER and to GLIDE rather than to us. Both are mirrors, and both say
195
+ here what they are mirroring, because a mirror that does not name its original is how the
196
+ two copies stop agreeing.
197
+ ═══════════════════════════════════════════════════════════════════════════════════════════ */
198
+
199
+ /** The header band. `headerHeight={36}` at the DataEditor mount is the original.
200
+ * Not `+ groupHeaderHeight`: glide adds that only when `enableGroups` is on, which it turns on
201
+ * when a COLUMN carries a `group` — `useGridColumns` never sets one (scrolling-data-grid.js:13). */
202
+ const HEADER_PX = 36;
203
+
204
+ /** glide's own row-marker width ladder, `data-editor.js:103`, verbatim.
205
+ *
206
+ * ⚠ It is passed BACK to glide as `rowMarkerWidth` rather than merely predicted here. The
207
+ * marker column is part of the content width the void's left edge is measured from, so a
208
+ * four-pixel disagreement between what glide draws and what we compute is a four-pixel seam of
209
+ * white against the tint — visible, and invisible to every gate. Pinning makes the two the same
210
+ * number by construction; mirroring the ladder (rather than pinning one constant) is what keeps
211
+ * the pin from changing the marker column's width on tables of more than 100 rows. */
212
+ function rowMarkerPx(rows: number): number {
213
+ return rows > 10_000 ? 48 : rows > 1000 ? 44 : rows > 100 ? 36 : 32;
214
+ }
215
+
216
+ /** The scrollbar gutter this browser steals from a scrollable box, measured once.
217
+ *
218
+ * ⛔ WHY THE VOID HAS TO KNOW. The void rectangles sit ON TOP of glide's scroller, so a
219
+ * rectangle drawn to the box's own edge paints over the scrollbar and hides it. The two cases
220
+ * are CROSSED, which is the part that is easy to get backwards: the BOTTOM void has to stop
221
+ * short when the columns overflow (a horizontal scrollbar), and the RIGHT void has to stop
222
+ * short when the rows overflow (a vertical one). Both happen in ordinary use — twelve columns
223
+ * filtered to five rows is exactly the owner's case.
224
+ *
225
+ * 0 on overlay-scrollbar platforms (macOS, touch), which is correct: nothing is stolen there. */
226
+ let scrollbarGutterPx: number | null = null;
227
+ function scrollbarGutter(): number {
228
+ if (scrollbarGutterPx !== null) return scrollbarGutterPx;
229
+ if (typeof document === "undefined") return 0;
230
+ const probe = document.createElement("div");
231
+ probe.style.cssText =
232
+ "position:absolute;top:-9999px;width:100px;height:100px;overflow:scroll";
233
+ document.body.append(probe);
234
+ scrollbarGutterPx = probe.offsetWidth - probe.clientWidth;
235
+ probe.remove();
236
+ return scrollbarGutterPx;
237
+ }
238
+ /* ═══ end W18-B VOID (module scope) ═══ */
239
+
240
  interface LocalWorkspace {
241
  fields: Field[];
242
  views: SavedView[];
 
1250
  [config.groupBy, rowPx, displayRows]
1251
  );
1252
 
1253
+ /* ═══ W18-B VOID ═══ (wave 18, owner item 1b) — WHERE THE TABLE ENDS, in canvas pixels.
1254
+ Consumed by the two `.cg-grid-void` rectangles at the DataEditor mount; see the CSS region
1255
+ of the same name for why the void is painted in the DOM rather than in the glide theme.
1256
+
1257
+ ⭐ THE PROPERTY THAT MAKES THIS ARITHMETIC AND NOT SCROLL-TRACKING, and the reason there is
1258
+ no `onVisibleRegionChanged` handler anywhere near it: **a void can only exist in an axis the
1259
+ content does not overflow.** glide's scroll extent is EXACTLY the content — `scrollWidth =
1260
+ nonGrowWidth (+ overscrollX)`, `scrollHeight = header + rows (+ overscrollY)`, and we pass
1261
+ neither overscroll prop (scrolling-data-grid.js:12-24) — so at maximum scroll the last
1262
+ row/column lands flush on the client edge. Wherever there is something to paint, the scroll
1263
+ offset in that axis is 0 and cannot become anything else. The rectangles therefore depend on
1264
+ the columns, the rows and the box, and on nothing that moves while the user drags.
1265
+
1266
+ Every column carries an explicit `width` (`useGridColumns` sets `config.widths[key] ??
1267
+ DEFAULT_WIDTH`) and none carries `grow`, so glide's column sizer passes them through
1268
+ untouched and this sum is the width it actually lays out. */
1269
+ const gridVoid = useMemo(() => {
1270
+ const gutter = scrollbarGutter();
1271
+ let colsPx = rowMarkerPx(displayRows.length);
1272
+ for (const c of visibleCols) colsPx += (c as { width?: number }).width ?? 0;
1273
+ let rowsPx = HEADER_PX;
1274
+ if (typeof rowHeight === "number") rowsPx += displayRows.length * rowHeight;
1275
+ else for (let i = 0; i < displayRows.length; i++) rowsPx += rowHeight(i);
1276
+
1277
+ /* Fit is tested against the client box the OTHER axis's scrollbar leaves behind — the same
1278
+ `clientWidth`/`clientHeight` glide's own scroll handler reads (infinite-scroller.js:
1279
+ 110-116), so this branch and glide's cannot disagree about whether a bar is there. The
1280
+ two `if`s resolve the circularity in the only direction it can run: a bar in one axis can
1281
+ CREATE one in the other, but two bars can never un-create each other. */
1282
+ let vBar = rowsPx > gridSize.height;
1283
+ let hBar = colsPx > gridSize.width;
1284
+ if (vBar && !hBar) hBar = colsPx > gridSize.width - gutter;
1285
+ if (hBar && !vBar) vBar = rowsPx > gridSize.height - gutter;
1286
+ const clientW = gridSize.width - (vBar ? gutter : 0);
1287
+ const clientH = gridSize.height - (hBar ? gutter : 0);
1288
+
1289
+ /* `null` = the content reaches that edge, so there is no void and no rectangle. The
1290
+ comparison is strict: a table ending exactly on the edge has nothing past it. */
1291
+ return {
1292
+ clientW,
1293
+ clientH,
1294
+ below: rowsPx < clientH ? rowsPx : null,
1295
+ right: colsPx < clientW ? colsPx : null,
1296
+ };
1297
+ }, [visibleCols, displayRows, rowHeight, gridSize]);
1298
+ /* ═══ end W18-B VOID (geometry) ═══ */
1299
+
1300
  // The record drawer resolves positions against what the MODE paints: the display slice for
1301
  // grid/list (the cap is real there), the FULL pipeline for calendar/kanban/map (a month, a
1302
  // stack or a pin reaches past the cap by design — a drawer that refused those pids would
 
2243
  return;
2244
  }
2245
 
2246
+ /* ═══ W18-C CATALOG ═══ A catalog is the THIRD mode that is not an arrangement of rows,
2247
+ and it is the furthest from one: its pages are authored content, and its product codes
2248
+ are a fraction of the pool in an order the user chose. Handing over the matched rows
2249
+ under a catalogue's name would be the same substitution the two branches around this
2250
+ one exist to stop. It exports as a PDF, through the browser's own print dialog (R10),
2251
+ so this branch refuses the file and names the door that works. */
2252
+ if (spec?.mode === "catalog") {
2253
+ signal(
2254
+ TOAST_EVENT,
2255
+ `"${view.name}" is a catalog — open it and use Print to save it as a PDF. ` +
2256
+ `A spreadsheet of its products is not what it draws.`
2257
+ );
2258
+ return;
2259
+ }
2260
+ /* ═══ end W18-C CATALOG ═══ */
2261
+
2262
  if (spec?.mode === "calendar" && spec.calendarMode === "summary") {
2263
  const sheet = isActive ? calSheetRef.current : null;
2264
  if (!sheet) return refuse("a calendar summary");
 
2878
  // C-TS close-out stitch: the panel's pids come from these rows; without this line the
2879
  // time-series view asked its question about ZERO customers (caught by the mounted
2880
  // judge — the node battery is blind here).
2881
+ displayMode !== "timeseries" &&
2882
+ /* ═══ W18-C CATALOG ═══ The designer's "add the products this view shows" door reads
2883
+ these rows — the same rows the toolbar counts, so the two can never disagree about
2884
+ which products the filter kept. The catalog's PAINT does not: it joins its stored
2885
+ codes against the whole pool, so a filter narrows what you can ADD and never what a
2886
+ finished page prints. ═══ end W18-C CATALOG ═══ */
2887
+ displayMode !== "catalog"
2888
  )
2889
  return [];
2890
  const seen = new Set<number>();
 
2938
  () => (displayMode === "timeseries" ? modeDataRows.map((r) => r.pid) : []),
2939
  [displayMode, modeDataRows]
2940
  );
2941
+ /* ═══ W18-C CATALOG ═══ (owner item 4, contract C6)
2942
+ The catalogs this view holds, and the door that persists an edit to them.
2943
+
2944
+ `displaySpec.catalogs` is ALREADY validated — `cleanDisplay` ran `cleanCatalogs` over it on
2945
+ read — so there is no second normalisation here and no chance of the two disagreeing.
2946
+
2947
+ ⚠ The write goes through `updateConfig` exactly like `onTimeseriesDisplay` below, which
2948
+ means a catalog edit is a VIEW-CONFIG save. That is the point: a catalogue is authored
2949
+ content and has to survive a reload, and the display spec is the only per-view store the
2950
+ grid has. It also means the 500-code / 40-page / 12-catalog caps are enforced twice on the
2951
+ round trip (here on read, and by the host's `_clean_catalogs` on write) — the designer
2952
+ states its budget in the toolbar so the cap is never the user's first news of it. */
2953
+ const catalogs = useMemo<CatalogSpec[]>(
2954
+ () => (displayMode === "catalog" ? (displaySpec?.catalogs ?? []) : []),
2955
+ [displayMode, displaySpec]
2956
+ );
2957
+ /** The codes the current filter kept — the designer's "add what the view shows" shortcut.
2958
+ * Derived from `modeDataRows`, so it counts what the toolbar counts. */
2959
+ const catalogFilteredCodes = useMemo(
2960
+ () =>
2961
+ displayMode === "catalog"
2962
+ ? modeDataRows
2963
+ .map((r) => r[CATALOG_CODE_FIELD])
2964
+ .filter((c): c is string => typeof c === "string" && !!c)
2965
+ : [],
2966
+ [displayMode, modeDataRows]
2967
+ );
2968
+ const onCatalogs = useCallback(
2969
+ (next: CatalogSpec[]) => {
2970
+ const spec = cleanDisplay(config.display) ?? { mode: "catalog" as const };
2971
+ updateConfig({ ...config, display: { ...spec, catalogs: next } });
2972
+ },
2973
+ [config, updateConfig]
2974
+ );
2975
+ /* ═══ end W18-C CATALOG ═══ */
2976
  /** Persist a bucket/span/metric pick onto the view, the way every other display key rides. */
2977
  const onTimeseriesDisplay = useCallback(
2978
  (next: Partial<DisplaySpec>) => {
 
3308
  onMouseLeave={() => setExpandAt((prev) => (prev === null ? prev : null))}
3309
  >
3310
  {displayMode === "grid" && (
3311
+ <>
3312
  <DataEditor
3313
  ref={gridRef}
3314
  columns={visibleCols}
 
3331
  rowHeight={rowHeight}
3332
  headerHeight={36}
3333
  rowMarkers="both"
3334
+ // ═══ W18-B VOID ═══ PINNED, not left to default. The marker column is part of
3335
+ // the content width the void's right-hand edge is measured from, and a width we
3336
+ // merely predicted could disagree with the one glide drew — four pixels of white
3337
+ // against the tint, visible on screen and invisible to every gate. `rowMarkerPx`
3338
+ // reproduces glide's own ladder verbatim (data-editor.js:103), so this changes no
3339
+ // pixel of the marker column; it only makes its width a number we own.
3340
+ // ═══ end W18-B VOID ═══
3341
+ rowMarkerWidth={rowMarkerPx(displayRows.length)}
3342
  gridSelection={gridSelection}
3343
  onGridSelectionChange={onGridSelectionChange}
3344
  getRowThemeOverride={getRowThemeOverride}
 
3391
  }
3392
  rightElementProps={{ sticky: true, fill: false }}
3393
  />
3394
+ {/* ═══ W18-B VOID ═══ (wave 18, owner item 1b / ruling R12) — the two rectangles.
3395
+
3396
+ They are SIBLINGS of the canvas, immediately after it and before everything else
3397
+ `.cg-grid-box` holds, which is the whole of their stacking story: `z-index: auto`
3398
+ plus tree order puts them over the canvas, and `.cg-selbar` / the popovers carry
3399
+ explicit `--cg-z-*` and stay over them. `pointer-events: none` keeps glide's
3400
+ scroller — which is UNDER this and owns every mouse event the grid reacts to —
3401
+ hit-testable straight through them.
3402
+
3403
+ The right-hand one starts BELOW the header rather than at 0: in `Airtable 7.png`
3404
+ the header band continues past the last column in its own near-white (#FBFCFE at
3405
+ (1250,141)), and it is where the add-field "+" lives. The two rectangles overlap
3406
+ in the bottom-right corner; they are the same flat colour, so the overlap is not
3407
+ visible and neither one needs to know about the other. */}
3408
+ {gridVoid.below !== null && (
3409
+ <div
3410
+ className="cg-grid-void"
3411
+ aria-hidden
3412
+ style={{
3413
+ top: gridVoid.below,
3414
+ left: 0,
3415
+ width: gridVoid.clientW,
3416
+ height: gridVoid.clientH - gridVoid.below,
3417
+ }}
3418
+ />
3419
+ )}
3420
+ {gridVoid.right !== null && (
3421
+ <div
3422
+ className="cg-grid-void"
3423
+ aria-hidden
3424
+ style={{
3425
+ top: HEADER_PX,
3426
+ left: gridVoid.right,
3427
+ width: gridVoid.clientW - gridVoid.right,
3428
+ height: gridVoid.clientH - HEADER_PX,
3429
+ }}
3430
+ />
3431
+ )}
3432
+ {/* ═══ end W18-B VOID ═══ */}
3433
+ </>
3434
  )}
3435
  {displayMode === "list" && (
3436
  <ListView
 
3519
  onSheet={onTsSheet}
3520
  />
3521
  )}
3522
+ {/* ═══ W18-C CATALOG ═══ Owner item 4 (C6) — Catalog is a VIEW, and the only one
3523
+ whose payload is authored CONTENT rather than an arrangement of the rows. It
3524
+ joins its stored product codes against the WHOLE pool (`rawRows`), not the
3525
+ filtered rows: a finished page must keep printing the products it names after
3526
+ somebody narrows the view. The filter still has a job — `catalogFilteredCodes`
3527
+ is the "add what this view shows" door — but it narrows what you can ADD, never
3528
+ what a page prints.
3529
+
3530
+ ⚠ Gated on the product surface. A catalog joins products to their images by
3531
+ `code`, which is `product_data`'s business key AND the C2-ASSET key; on the
3532
+ customer table the primary field is a customer NAME, and joining on it would ask
3533
+ the asset route for a picture of a person. The refusal says so in words rather
3534
+ than rendering an empty designer. */}
3535
+ {displayMode === "catalog" &&
3536
+ (scope === "product" ? (
3537
+ <CatalogView
3538
+ catalogs={catalogs}
3539
+ onCatalogs={onCatalogs}
3540
+ fields={fields}
3541
+ fieldByKey={fieldByKey}
3542
+ poolRows={rawRows}
3543
+ filteredCodes={catalogFilteredCodes}
3544
+ />
3545
+ ) : (
3546
+ <div className="cg-mode-empty">
3547
+ A catalog prints products. Open the Products table to build one.
3548
+ </div>
3549
+ ))}
3550
+ {/* ═══ end W18-C CATALOG ═══ */}
3551
  {/* Wave-7 item W11c (C3) — Map is a VIEW: same pipeline rows, pins for rows
3552
  with host-supplied coords, click-through to the same record detail, and
3553
  an honest no-location chip. */}
web/src/customer-grid/RecordDetail.tsx CHANGED
@@ -1,7 +1,13 @@
1
  import { useEffect, useId, useLayoutEffect, useMemo, useRef, useState } from "react";
2
  import type { ReactNode, TextareaHTMLAttributes } from "react";
3
  import { BodyPortal, useOverlayLayer } from "./OverlaySurface";
4
- import { checkboxOn, formatDisplay } from "./cells";
 
 
 
 
 
 
5
  import { StarRow, StarIcon } from "./Stars";
6
  import { isPickType, mayEditField, ratingMax } from "./types";
7
  import type { CustomerDoc, DisplaySpec, Field, Row, Viewer } from "./types";
@@ -703,6 +709,23 @@ export default function RecordDetail({
703
  <StarRow value={ratingValue(field, record[field.key])} max={ratingMax(field)} />
704
  </span>
705
  );
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
706
  } else {
707
  // Everything read-only, with the reason where there is one. After item 9 a picked field
708
  // is no longer read-only BY DESIGN, so the only reasons left are real ones: no
 
1
  import { useEffect, useId, useLayoutEffect, useMemo, useRef, useState } from "react";
2
  import type { ReactNode, TextareaHTMLAttributes } from "react";
3
  import { BodyPortal, useOverlayLayer } from "./OverlaySurface";
4
+ import {
5
+ automationDetail,
6
+ automationState,
7
+ automationStateLabel,
8
+ checkboxOn,
9
+ formatDisplay,
10
+ } from "./cells";
11
  import { StarRow, StarIcon } from "./Stars";
12
  import { isPickType, mayEditField, ratingMax } from "./types";
13
  import type { CustomerDoc, DisplaySpec, Field, Row, Viewer } from "./types";
 
709
  <StarRow value={ratingValue(field, record[field.key])} max={ratingMax(field)} />
710
  </span>
711
  );
712
+ } else if (field.type === "automation") {
713
+ // C5-AUTOFIELD (wave 18). It has to be its OWN branch and not the read-only fallback
714
+ // below, because that branch explains read-only-ness as a PERMISSION ("You do not have
715
+ // permission to edit this field") — and for an automation column that sentence is simply
716
+ // false. Nobody may type here, admins included: the value is what the last run wrote.
717
+ const raw = rawText(record[field.key]);
718
+ const state = automationState(raw);
719
+ slot = (
720
+ <span className="cg-detail-value cg-detail-auto" title={raw || "Not run yet"}>
721
+ <span className={`cg-detail-auto-chip is-${state}`}>
722
+ {automationStateLabel(raw)}
723
+ </span>
724
+ {automationDetail(raw) ? (
725
+ <span className="cg-detail-auto-detail">{automationDetail(raw)}</span>
726
+ ) : null}
727
+ </span>
728
+ );
729
  } else {
730
  // Everything read-only, with the reason where there is one. After item 9 a picked field
731
  // is no longer read-only BY DESIGN, so the only reasons left are real ones: no
web/src/customer-grid/apiBridge.ts CHANGED
@@ -336,7 +336,7 @@ let inFlight = false;
336
  * never a free-text field. Read and write must agree on what a scope is: the
337
  * grid asks `/workspace?scope=X` and every event it then emits says `X`.
338
  */
339
- export type SurfaceScope = "customer" | "cohort" | "product";
340
  let surfaceScope: SurfaceScope = "customer";
341
 
342
  export function setSurfaceScope(scope: SurfaceScope): void {
 
336
  * never a free-text field. Read and write must agree on what a scope is: the
337
  * grid asks `/workspace?scope=X` and every event it then emits says `X`.
338
  */
339
+ export type SurfaceScope = "customer" | "cohort" | "product" | `ut_${string}`;
340
  let surfaceScope: SurfaceScope = "customer";
341
 
342
  export function setSurfaceScope(scope: SurfaceScope): void {
web/src/customer-grid/catalogData.ts ADDED
@@ -0,0 +1,350 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ // ---------------------------------------------------------------------------
2
+ // customer-grid / catalogData.ts
3
+ // Wave-18 owner item 4 (contract C6-CATALOG) — the CATALOG view's pure layer:
4
+ // paper geometry, page templates, product resolution and the C2-ASSET image
5
+ // URLs. No React, no DOM: everything here is drivable by the gate.
6
+ //
7
+ // What a catalog IS, stated once so the rest of the code can be short: a view
8
+ // holds up to MAX_CATALOGS print artifacts; each is an ordered list of PAGES;
9
+ // a `section` page names PRODUCT CODES and the renderer joins them to the live
10
+ // pool at paint time. Codes, never pids — a catalogue outlives any one fetch,
11
+ // and the code is the thing actually printed on the page.
12
+ //
13
+ // Fidelity target is `reference/Royal Imports/Royal Collection 2027.pdf`
14
+ // (rendered and read before this file was written): letterspaced caps section
15
+ // headers, product renders on white, caps description, a navy pill carrying the
16
+ // pack line, SKU in accent bold beside the colour name, a full-bleed lifestyle
17
+ // strip under the grid, page number + crown mark in the footer.
18
+ // ---------------------------------------------------------------------------
19
+
20
+ import { API_V1 } from "../apiContract";
21
+ import {
22
+ CATALOG_CODE_MAX,
23
+ MAX_CATALOG_CODES,
24
+ MAX_CATALOG_PAGES,
25
+ } from "./types";
26
+ import type {
27
+ CatalogOrientation,
28
+ CatalogPage,
29
+ CatalogPageKind,
30
+ CatalogPaper,
31
+ CatalogQuality,
32
+ CatalogSpec,
33
+ Field,
34
+ Row,
35
+ } from "./types";
36
+
37
+ // ----------------------------------------------------------------- the pool
38
+
39
+ /**
40
+ * The column a catalog joins on. `product_data`'s business key
41
+ * (`aios_grid_fields.json: business_key = "code"`) and ALSO the C2-ASSET key —
42
+ * the 1,142 masters are named for `default_code`, so one string is both the
43
+ * SKU printed on the page and the image that goes above it. That coincidence is
44
+ * why the catalog binds to a named column instead of to the table's primary
45
+ * field: on `customer_data` the primary field is a customer NAME, and joining
46
+ * assets on it would ask the asset route for an image of a person.
47
+ */
48
+ export const CATALOG_CODE_FIELD = "code";
49
+
50
+ /** Code → row, over the WHOLE pool (not the filtered rows) so a catalogue keeps
51
+ * painting the products it names after the user narrows the view. */
52
+ export function indexByCode(rows: Row[]): Map<string, Row> {
53
+ const out = new Map<string, Row>();
54
+ for (const r of rows) {
55
+ const code = r[CATALOG_CODE_FIELD];
56
+ if (typeof code === "string" && code && !out.has(code)) out.set(code, r);
57
+ }
58
+ return out;
59
+ }
60
+
61
+ /**
62
+ * The four listing lines, resolved for ONE product.
63
+ *
64
+ * `code` is always present (it is the identity); everything else is absent when
65
+ * the catalog has not bound a column for it, or when the row carries no value.
66
+ * ⚠ `pack` is the bound field's value VERBATIM — never composed from a number.
67
+ * The reference prints "6-Piece per Pack" on p7 and "6-Sets Per Case" on p121,
68
+ * so any composed rule is wrong on one of those pages.
69
+ */
70
+ export interface CatalogItem {
71
+ code: string;
72
+ name?: string;
73
+ pack?: string;
74
+ color?: string;
75
+ price?: string;
76
+ /** False when the row is not in the pool at all — drawn as a stated gap. */
77
+ known: boolean;
78
+ }
79
+
80
+ export function resolveItems(
81
+ codes: string[],
82
+ byCode: Map<string, Row>,
83
+ binds: CatalogSpec["fields"] | undefined,
84
+ format: (field: Field, value: Row[string]) => string,
85
+ fieldByKey: Map<string, Field>
86
+ ): CatalogItem[] {
87
+ const line = (row: Row, key: string | undefined): string | undefined => {
88
+ if (!key) return undefined;
89
+ const field = fieldByKey.get(key);
90
+ if (!field) return undefined;
91
+ const text = format(field, row[key]);
92
+ return text ? text : undefined;
93
+ };
94
+ return codes.map((code) => {
95
+ const row = byCode.get(code);
96
+ if (!row) return { code, known: false };
97
+ return {
98
+ code,
99
+ known: true,
100
+ name: line(row, binds?.name),
101
+ pack: line(row, binds?.pack),
102
+ color: line(row, binds?.color),
103
+ price: line(row, binds?.price),
104
+ };
105
+ });
106
+ }
107
+
108
+ /**
109
+ * The default column bindings for a NEW catalog, guessed from the table's own
110
+ * fields. A guess, not a contract: every one is overridable in the designer, and
111
+ * a wrong guess renders a wrong LINE rather than a wrong number.
112
+ *
113
+ * `name` deliberately skips the code column — printing the SKU twice is the one
114
+ * outcome that always looks broken.
115
+ */
116
+ export function autoBind(fields: Field[]): NonNullable<CatalogSpec["fields"]> {
117
+ const out: NonNullable<CatalogSpec["fields"]> = {};
118
+ const find = (test: RegExp, types?: string[]): string | undefined =>
119
+ fields.find(
120
+ (f) =>
121
+ f.key !== CATALOG_CODE_FIELD &&
122
+ (test.test(f.key) || test.test(f.label)) &&
123
+ (!types || types.includes(f.type))
124
+ )?.key;
125
+ out.name =
126
+ find(/^(product|name|title|description)$/i) ??
127
+ fields.find((f) => f.key !== CATALOG_CODE_FIELD && f.type === "text")?.key;
128
+ out.pack = find(/pack|case|carton|inner/i);
129
+ out.color = find(/colou?r/i);
130
+ out.price = find(/price|msrp|list/i, ["currency", "int", "pct"]);
131
+ for (const k of ["name", "pack", "color", "price"] as const)
132
+ if (!out[k]) delete out[k];
133
+ return out;
134
+ }
135
+
136
+ // -------------------------------------------------------------- the images
137
+
138
+ /**
139
+ * C2-ASSET, consumed as a SHAPE (the contract's client rule: tokens, not
140
+ * values). `API_V1` is the app's own base — this file never names a host.
141
+ *
142
+ * R9: `web` is an ~800px alpha-preserving PNG (transparency matters — the 2027
143
+ * listing sits product renders directly on white), `print` serves the ORIGINAL
144
+ * master bytes. The response is `immutable`-cached, so swapping quality for an
145
+ * export costs one fetch per image and nothing after that.
146
+ */
147
+ export function assetUrl(code: string, quality: CatalogQuality): string {
148
+ return `${API_V1}/assets/products/${encodeURIComponent(code)}?q=${quality}`;
149
+ }
150
+
151
+ /** C2-ASSET's manifest — which codes HAVE an image, so the designer can mark
152
+ * the ones that will print an empty frame instead of discovering it at print. */
153
+ export const ASSET_MANIFEST_URL = `${API_V1}/assets/products`;
154
+
155
+ export async function fetchAssetCodes(): Promise<Set<string> | null> {
156
+ try {
157
+ const res = await fetch(ASSET_MANIFEST_URL, { credentials: "same-origin" });
158
+ if (!res.ok) return null;
159
+ const body = (await res.json()) as { codes?: unknown };
160
+ if (!Array.isArray(body.codes)) return null;
161
+ return new Set(body.codes.filter((c): c is string => typeof c === "string"));
162
+ } catch {
163
+ // A manifest we cannot reach means "we do not know", NOT "nothing exists".
164
+ // Returning null keeps the designer silent instead of marking every product
165
+ // missing — a false gap report would send the user hunting for images that
166
+ // are sitting right there.
167
+ return null;
168
+ }
169
+ }
170
+
171
+ // -------------------------------------------------------------- the paper
172
+
173
+ /**
174
+ * Physical page sizes. Emitted into `@page { size: … }` as EXPLICIT dimensions
175
+ * rather than the CSS keywords (`letter`, `A4`, `ledger`): the keyword set has
176
+ * no unambiguous name for 11x17 — `ledger` and `tabloid` differ by rotation
177
+ * depending on who you read — and an explicit pair is what the print gate can
178
+ * actually assert against the PDF it gets back.
179
+ */
180
+ export const PAPER_SIZES: Record<CatalogPaper, { w: number; h: number; unit: "in" | "mm" }> = {
181
+ letter: { w: 8.5, h: 11, unit: "in" },
182
+ a4: { w: 210, h: 297, unit: "mm" },
183
+ tabloid: { w: 11, h: 17, unit: "in" },
184
+ };
185
+
186
+ export const PAPER_LABELS: Record<CatalogPaper, string> = {
187
+ letter: 'Letter · 8.5 × 11"',
188
+ a4: "A4 · 210 × 297 mm",
189
+ tabloid: 'Tabloid · 11 × 17"',
190
+ };
191
+
192
+ /** The page box, with orientation applied. Landscape SWAPS the pair — it does
193
+ * not rotate the content, which is what `@page`'s own keyword would do. */
194
+ export function pageBox(
195
+ paper: CatalogPaper,
196
+ orientation: CatalogOrientation
197
+ ): { w: number; h: number; unit: "in" | "mm" } {
198
+ const size = PAPER_SIZES[paper];
199
+ return orientation === "landscape"
200
+ ? { w: size.h, h: size.w, unit: size.unit }
201
+ : { ...size };
202
+ }
203
+
204
+ /** `8.5in` / `297mm` — one place that decides how a dimension is spelled, so the
205
+ * screen box and the `@page` rule can never drift apart by a unit. */
206
+ export function cssLength(value: number, unit: "in" | "mm"): string {
207
+ return `${Number.isInteger(value) ? value : value.toFixed(2).replace(/0+$/, "")}${unit}`;
208
+ }
209
+
210
+ // ------------------------------------------------------- catalogs & pages
211
+
212
+ /** Royal Imports' own palette, measured off the 2027 cover: deep navy field,
213
+ * terracotta accent. The defaults for a new catalog; every catalog can differ. */
214
+ export const DEFAULT_BRAND = {
215
+ primary: "#16233A",
216
+ accent: "#C47B5A",
217
+ company: "Royal Imports",
218
+ } as const;
219
+
220
+ function mintId(prefix: string): string {
221
+ const rand =
222
+ typeof crypto !== "undefined" && "randomUUID" in crypto
223
+ ? crypto.randomUUID().slice(0, 8)
224
+ : Math.random().toString(36).slice(2, 10);
225
+ return `${prefix}_${rand}`;
226
+ }
227
+
228
+ export const newCatalogId = (): string => mintId("cat");
229
+ export const newPageId = (): string => mintId("pg");
230
+
231
+ export const PAGE_KIND_LABELS: Record<CatalogPageKind, string> = {
232
+ cover: "Cover",
233
+ intro: "Introduction",
234
+ section: "Product section",
235
+ gallery: "Full-page image",
236
+ };
237
+
238
+ /** A fresh page of a kind, carrying the defaults its template reads best at. */
239
+ export function newPage(kind: CatalogPageKind): CatalogPage {
240
+ const page: CatalogPage = { id: newPageId(), kind };
241
+ if (kind === "cover") page.title = "Collection";
242
+ if (kind === "intro") page.title = "About us";
243
+ if (kind === "section") page.title = "New section";
244
+ return page;
245
+ }
246
+
247
+ /**
248
+ * A new catalog, opening on the reference's own running order: cover, letter,
249
+ * one product section. Three pages rather than an empty list because an empty
250
+ * designer teaches nothing — the user should see the artifact they are editing.
251
+ */
252
+ export function newCatalog(name: string, fields: Field[]): CatalogSpec {
253
+ const binds = autoBind(fields);
254
+ const cat: CatalogSpec = {
255
+ id: newCatalogId(),
256
+ name,
257
+ paper: "letter",
258
+ orientation: "portrait",
259
+ brand: { ...DEFAULT_BRAND },
260
+ pages: [newPage("cover"), newPage("intro"), newPage("section")],
261
+ };
262
+ if (Object.keys(binds).length) cat.fields = binds;
263
+ return cat;
264
+ }
265
+
266
+ /** How much of the 500-code allowance this catalog has spent. Shown in the
267
+ * designer BEFORE the cap bites, so a full catalogue is a fact the user can see
268
+ * rather than a silent truncation on the next save ([[no-unverifiable-aggregates]]). */
269
+ export function codesUsed(cat: CatalogSpec): number {
270
+ let n = 0;
271
+ for (const p of cat.pages) n += p.products?.length ?? 0;
272
+ return n;
273
+ }
274
+
275
+ export function codesRemaining(cat: CatalogSpec): number {
276
+ return Math.max(0, MAX_CATALOG_CODES - codesUsed(cat));
277
+ }
278
+
279
+ export function pagesRemaining(cat: CatalogSpec): number {
280
+ return Math.max(0, MAX_CATALOG_PAGES - cat.pages.length);
281
+ }
282
+
283
+ /**
284
+ * Add codes to a page, honouring BOTH caps and reporting what it refused.
285
+ *
286
+ * The report is the point: "added 40 of 60, the catalogue is full" is a fact the
287
+ * user can act on. Silently keeping 40 is the defect this house calls a silent
288
+ * `[:N]` cap.
289
+ */
290
+ export function addCodes(
291
+ cat: CatalogSpec,
292
+ pageId: string,
293
+ codes: string[]
294
+ ): { pages: CatalogPage[]; added: number; skippedDuplicate: number; skippedCap: number } {
295
+ const budget = codesRemaining(cat);
296
+ let added = 0;
297
+ let skippedDuplicate = 0;
298
+ let skippedCap = 0;
299
+ const pages = cat.pages.map((p) => {
300
+ if (p.id !== pageId) return p;
301
+ const have = new Set(p.products ?? []);
302
+ const next = [...(p.products ?? [])];
303
+ for (const raw of codes) {
304
+ const code = raw.slice(0, CATALOG_CODE_MAX);
305
+ if (!code) continue;
306
+ if (have.has(code)) {
307
+ skippedDuplicate += 1;
308
+ continue;
309
+ }
310
+ if (added >= budget) {
311
+ skippedCap += 1;
312
+ continue;
313
+ }
314
+ have.add(code);
315
+ next.push(code);
316
+ added += 1;
317
+ }
318
+ return next.length ? { ...p, products: next } : p;
319
+ });
320
+ return { pages, added, skippedDuplicate, skippedCap };
321
+ }
322
+
323
+ /** The layout a section page renders at, defaults applied. The three toggles are
324
+ * stored literal-only (see `CatalogPage.layout`), so "absent" is where every
325
+ * default lives and this is the ONE place that knows what they are. */
326
+ export function sectionLayout(page: CatalogPage): {
327
+ cols: number;
328
+ showPack: boolean;
329
+ showColor: boolean;
330
+ showPrice: boolean;
331
+ } {
332
+ const l = page.layout;
333
+ return {
334
+ cols: l?.cols ?? 3,
335
+ showPack: l?.showPack !== false,
336
+ showColor: l?.showColor !== false,
337
+ showPrice: l?.showPrice === true,
338
+ };
339
+ }
340
+
341
+ /**
342
+ * The printed page NUMBER for each page, and the total.
343
+ *
344
+ * The cover is page 1 and carries no folio — exactly the reference, where the
345
+ * first numeral to appear on paper is on the intro spread. So the number is the
346
+ * INDEX, and whether it is drawn is the template's business.
347
+ */
348
+ export function folio(pages: CatalogPage[]): { number: number; total: number }[] {
349
+ return pages.map((_, i) => ({ number: i + 1, total: pages.length }));
350
+ }
web/src/customer-grid/catalogPrint.ts ADDED
@@ -0,0 +1,155 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ // ---------------------------------------------------------------------------
2
+ // customer-grid / catalogPrint.ts
3
+ // Wave-18 owner item 4, ruling R10 — catalog EXPORT is browser print-to-PDF:
4
+ // true paginated pages, a real `@page` sized from the catalog's own paper
5
+ // setting, and page-break rules so ONE catalog prints and nothing else does.
6
+ //
7
+ // Why a generated stylesheet instead of CSS in index.css: `@page { size: … }`
8
+ // cannot read a custom property. The page size is DATA (per catalog, per
9
+ // orientation), so the only way to state it is to write the rule at print time.
10
+ // index.css carries everything static; this module carries the one rule that
11
+ // changes with the artifact.
12
+ //
13
+ // ⚠ The gate that proves this passes `preferCSSPageSize: True` to Playwright's
14
+ // `page.pdf()`. Without it Playwright IGNORES `@page { size }` and emits Letter
15
+ // whatever the rule says — a green check that proves the default, not the
16
+ // contract ([[gate-can-report-green-on-nothing]]).
17
+ // ---------------------------------------------------------------------------
18
+
19
+ import { cssLength, pageBox } from "./catalogData";
20
+ import type { CatalogSpec } from "./types";
21
+
22
+ /** The `<style>` element's id — one per document, replaced rather than stacked. */
23
+ export const PRINT_STYLE_ID = "cg-cat-print-style";
24
+ /** Set on `<html>` for the duration of a print, and removed after. The static
25
+ * half of the isolation lives in index.css keyed off this class. */
26
+ export const PRINTING_CLASS = "cg-cat-printing";
27
+
28
+ /**
29
+ * The catalog's print stylesheet.
30
+ *
31
+ * Dimensions are EXPLICIT (`8.5in 11in`), never the CSS page-size keywords: the
32
+ * keyword vocabulary has no unambiguous name for 11x17 (`ledger` vs `tabloid`
33
+ * differ by rotation depending on the engine), and an explicit pair is what a
34
+ * PDF can be measured against.
35
+ *
36
+ * `print-color-adjust: exact` is load-bearing, not a nicety — without it every
37
+ * engine drops the navy pack pill, the navy cover field and the terracotta rules
38
+ * to white, which is the entire visual language of the reference catalogue.
39
+ */
40
+ export function printCss(cat: CatalogSpec): string {
41
+ const box = pageBox(cat.paper, cat.orientation);
42
+ const w = cssLength(box.w, box.unit);
43
+ const h = cssLength(box.h, box.unit);
44
+ const R = `html.${PRINTING_CLASS}`;
45
+ return [
46
+ `@page { size: ${w} ${h}; margin: 0; }`,
47
+ `@media print {`,
48
+ ` ${R}, ${R} body {`,
49
+ ` width: ${w}; margin: 0; padding: 0; background: #fff;`,
50
+ ` -webkit-print-color-adjust: exact; print-color-adjust: exact;`,
51
+ ` }`,
52
+ ` ${R} .cg-cat-page {`,
53
+ ` width: ${w}; height: ${h};`,
54
+ ` margin: 0; box-shadow: none; border: 0; border-radius: 0;`,
55
+ ` transform: none; break-after: page; break-inside: avoid;`,
56
+ ` }`,
57
+ // A trailing `break-after: page` is how a print job grows one blank sheet at
58
+ // the end — the break is honoured, then there is nothing to put on it.
59
+ ` ${R} .cg-cat-page:last-child { break-after: auto; }`,
60
+ `}`,
61
+ ].join("\n");
62
+ }
63
+
64
+ /** Write (or rewrite) the catalog's print rule into the document. Returns a
65
+ * remover, so a cancelled print leaves no sizing behind for the next one. */
66
+ export function applyPrintCss(cat: CatalogSpec, doc: Document = document): () => void {
67
+ let el = doc.getElementById(PRINT_STYLE_ID) as HTMLStyleElement | null;
68
+ if (!el) {
69
+ el = doc.createElement("style");
70
+ el.id = PRINT_STYLE_ID;
71
+ doc.head.appendChild(el);
72
+ }
73
+ el.textContent = printCss(cat);
74
+ return () => {
75
+ el?.parentNode?.removeChild(el);
76
+ };
77
+ }
78
+
79
+ /**
80
+ * Wait for every image under `root` to be decoded.
81
+ *
82
+ * R9's quality swap changes every `src` in the tree; calling `window.print()` on
83
+ * the next line prints the pages with the NEW urls still loading, i.e. blank
84
+ * frames — the browser does not wait for images before opening the dialog. This
85
+ * is the step that makes "high quality" mean anything.
86
+ *
87
+ * Never rejects: a 404 image (a product with no master) must not stop the print.
88
+ * `decode()` is what rejects on those, so each is caught individually.
89
+ */
90
+ export async function awaitImages(root: ParentNode | null, timeoutMs = 20_000): Promise<void> {
91
+ if (!root) return;
92
+ const imgs = Array.from(root.querySelectorAll("img"));
93
+ if (!imgs.length) return;
94
+ const all = Promise.all(
95
+ imgs.map((img) =>
96
+ img.decode
97
+ ? img.decode().catch(() => undefined)
98
+ : new Promise<void>((res) => {
99
+ if (img.complete) return res();
100
+ img.addEventListener("load", () => res(), { once: true });
101
+ img.addEventListener("error", () => res(), { once: true });
102
+ })
103
+ )
104
+ ).then(() => undefined);
105
+ // The ceiling is a real one: a stalled asset route must delay the dialog, not
106
+ // cancel the export.
107
+ await Promise.race([all, new Promise<void>((res) => setTimeout(res, timeoutMs))]);
108
+ }
109
+
110
+ /**
111
+ * Add the printing class, open the dialog, and take the class back off however
112
+ * the dialog ends.
113
+ *
114
+ * THREE ways out, because none of them is reliable alone: `afterprint` (does not
115
+ * fire on a cancelled dialog in every engine), the `print` media query going
116
+ * false (Safari's route), and a timeout (the honest backstop). `finish` is
117
+ * idempotent, so whichever arrives first wins and the rest are no-ops — leaving
118
+ * the class on would hide the entire app behind a print rule.
119
+ */
120
+ export function printOnce(
121
+ doc: Document = document,
122
+ print?: () => void
123
+ ): Promise<void> {
124
+ const win = doc.defaultView;
125
+ const root = doc.documentElement;
126
+ return new Promise<void>((resolve) => {
127
+ let done = false;
128
+ let timer: ReturnType<typeof setTimeout> | undefined;
129
+ const mql = win?.matchMedia ? win.matchMedia("print") : null;
130
+ const onMql = (e: MediaQueryListEvent) => {
131
+ if (!e.matches) finish();
132
+ };
133
+ function finish() {
134
+ if (done) return;
135
+ done = true;
136
+ root.classList.remove(PRINTING_CLASS);
137
+ win?.removeEventListener("afterprint", finish);
138
+ mql?.removeEventListener?.("change", onMql);
139
+ if (timer !== undefined) clearTimeout(timer);
140
+ resolve();
141
+ }
142
+ root.classList.add(PRINTING_CLASS);
143
+ win?.addEventListener("afterprint", finish);
144
+ mql?.addEventListener?.("change", onMql);
145
+ timer = setTimeout(finish, 120_000);
146
+ try {
147
+ if (print) print();
148
+ else win?.print();
149
+ } catch {
150
+ finish();
151
+ }
152
+ // Chrome's `window.print()` blocks until the dialog closes and only THEN
153
+ // fires afterprint; Firefox returns immediately. Both are handled above.
154
+ });
155
+ }
web/src/customer-grid/cells.ts CHANGED
@@ -30,17 +30,28 @@ import type {
30
  CustomCell,
31
  CustomRenderer,
32
  GridCell,
 
33
  } from "@glideapps/glide-data-grid";
34
  import type { Field } from "./types";
35
  import { ratingMax } from "./types";
36
- import { STATUS_BUBBLE } from "./theme";
 
 
 
 
 
 
 
 
 
 
37
  // ⚠ ALIASED: this module has its own `pickTint` (the bubble table above) and choiceColors has
38
  // another. C-AVATAR names choiceColors' as the avatar fallback's source, so the two stay
39
  // distinguishable at the call site rather than one silently shadowing the other.
40
  import { optionTint, pickTint as choicePickTint } from "./choiceColors";
41
- import { avatarInitials, avatarSize, checkboxOn, dateTimeText, formulaIsBlank, formulaIsText,
42
- num, numberText, userCellPayload } from "./display";
43
- import type { CellValue, UserCellData } from "./display";
44
 
45
  // Wave-7 (item W2): the pure display-string half moved to display.ts so the
46
  // export builders can run under node without dragging glide along. Re-exported
@@ -50,6 +61,29 @@ export { checkboxOn, dateTimeText, formatDisplay } from "./display";
50
  // re-exported here so callers keep importing "the cell module" for everything avatar-shaped.
51
  export { avatarInitials, avatarSize, userCellPayload } from "./display";
52
  export type { UserCellData } from "./display";
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
53
 
54
  /** Stable pill tint for a user-defined choice. Hashed from the VALUE, so the same choice keeps
55
  * its colour across rows, sessions and users without anyone picking one — and a renamed option
@@ -375,6 +409,27 @@ export function makeCell(
375
  allowOverlay: false,
376
  };
377
  }
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
378
  case "status":
379
  return {
380
  kind: GridCellKind.Bubble,
 
30
  CustomCell,
31
  CustomRenderer,
32
  GridCell,
33
+ Theme,
34
  } from "@glideapps/glide-data-grid";
35
  import type { Field } from "./types";
36
  import { ratingMax } from "./types";
37
+ import {
38
+ LP_BLUE_TEXT,
39
+ LP_BLUE_TINT,
40
+ LP_GREEN_DEEP,
41
+ LP_GREEN_TINT,
42
+ LP_RED_DEEP,
43
+ LP_RED_TINT,
44
+ LP_YELLOW_DEEP,
45
+ LP_YELLOW_TINT,
46
+ STATUS_BUBBLE,
47
+ } from "./theme";
48
  // ⚠ ALIASED: this module has its own `pickTint` (the bubble table above) and choiceColors has
49
  // another. C-AVATAR names choiceColors' as the avatar fallback's source, so the two stay
50
  // distinguishable at the call site rather than one silently shadowing the other.
51
  import { optionTint, pickTint as choicePickTint } from "./choiceColors";
52
+ import { automationState, avatarInitials, avatarSize, checkboxOn, dateTimeText, formulaIsBlank,
53
+ formulaIsText, num, numberText, userCellPayload } from "./display";
54
+ import type { AutomationState, CellValue, UserCellData } from "./display";
55
 
56
  // Wave-7 (item W2): the pure display-string half moved to display.ts so the
57
  // export builders can run under node without dragging glide along. Re-exported
 
61
  // re-exported here so callers keep importing "the cell module" for everything avatar-shaped.
62
  export { avatarInitials, avatarSize, userCellPayload } from "./display";
63
  export type { UserCellData } from "./display";
64
+ // C5-AUTOFIELD — same arrangement: the string parsing is pure and lives in display.ts (a node
65
+ // gate can hold it); only the canvas tint table below needs this module.
66
+ export { automationDetail, automationState, automationStateLabel } from "./display";
67
+ export type { AutomationState } from "./display";
68
+
69
+ /**
70
+ * Wave-18 C5-AUTOFIELD — the cell tint per automation state.
71
+ *
72
+ * Standing brand rule: the PASTEL is the fill and the measured `-deep` variant carries the ink.
73
+ * `bgCell` + `textDark` is that pairing at cell scale, and every combination below is one of the
74
+ * pairs already measured for `STATUS_BUBBLE` (≥ 4.9:1), so nothing new needed measuring.
75
+ *
76
+ * `none` deliberately has NO override: a cell that has never run reads as an ordinary empty
77
+ * cell, because a tint would claim the automation had produced something.
78
+ */
79
+ const AUTOMATION_TINT: Record<AutomationState, Partial<Theme> | undefined> = {
80
+ ok: { bgCell: LP_GREEN_TINT, textDark: LP_GREEN_DEEP },
81
+ partial: { bgCell: LP_YELLOW_TINT, textDark: LP_YELLOW_DEEP },
82
+ blocked: { bgCell: LP_YELLOW_TINT, textDark: LP_YELLOW_DEEP },
83
+ error: { bgCell: LP_RED_TINT, textDark: LP_RED_DEEP },
84
+ queued: { bgCell: LP_BLUE_TINT, textDark: LP_BLUE_TEXT },
85
+ none: undefined,
86
+ };
87
 
88
  /** Stable pill tint for a user-defined choice. Hashed from the VALUE, so the same choice keeps
89
  * its colour across rows, sessions and users without anyone picking one — and a renamed option
 
409
  allowOverlay: false,
410
  };
411
  }
412
+ case "automation": {
413
+ // Wave-18 C5-AUTOFIELD. The cell is what the last RUN wrote — `ok · 2026-08-03 14:10 ·
414
+ // 12 posts` — so it is read-only by NATURE, not by policy: a value typed here would be
415
+ // overwritten by the next run with nothing anywhere saying so. The column's behaviour is
416
+ // configured through its gear, the way a formula's expression is.
417
+ //
418
+ // A tinted CELL rather than a bubble, and that is the whole design: the state and the
419
+ // detail are one sentence, and splitting them into two pills would put "2026-08-03 14:10
420
+ // · 12 posts" in a bubble, which is not what a bubble is for. `bgCell` is the one theme
421
+ // key glide alpha-blends (theme.ts:172), so the tints below are the flat pastels rather
422
+ // than anything semi-transparent.
423
+ const text = String(v ?? "");
424
+ return {
425
+ kind: GridCellKind.Text,
426
+ data: text,
427
+ displayData: text,
428
+ allowOverlay: false,
429
+ readonly: true,
430
+ themeOverride: AUTOMATION_TINT[automationState(text)],
431
+ };
432
+ }
433
  case "status":
434
  return {
435
  kind: GridCellKind.Bubble,
web/src/customer-grid/clipboard.ts CHANGED
@@ -29,6 +29,12 @@ export function coerceClipboardValue(
29
  case "formula":
30
  case "created_time":
31
  case "status":
 
 
 
 
 
 
32
  return undefined;
33
  case "checkbox": {
34
  if (trimmed === "") return "";
 
29
  case "formula":
30
  case "created_time":
31
  case "status":
32
+ // C5-AUTOFIELD (wave 18) — an automation cell is what the last RUN wrote, so a paste into
33
+ // one is refused HERE and not only by the renderer. `READONLY_CELL_TYPES` decides what
34
+ // gets an editor; a paste never opens one, so a type that is read-only by nature and
35
+ // absent from this list would be uneditable by keyboard and overwritable by Ctrl+V —
36
+ // and the next run would silently replace whatever landed.
37
+ case "automation":
38
  return undefined;
39
  case "checkbox": {
40
  if (trimmed === "") return "";
web/src/customer-grid/display.ts CHANGED
@@ -193,6 +193,41 @@ export function dateTimeText(field: Field, v: CellValue): string {
193
  * a value reads identically on the canvas, in the panel and in a file. Returns
194
  * "" for empty so callers can substitute their own placeholder.
195
  */
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
196
  export function formatDisplay(field: Field, v: CellValue): string {
197
  switch (field.type) {
198
  case "currency":
@@ -226,6 +261,12 @@ export function formatDisplay(field: Field, v: CellValue): string {
226
  const n = num(v);
227
  return n >= 1 ? `${Math.round(n)} of ${ratingMax(field)}` : "";
228
  }
 
 
 
 
 
 
229
  case "multiselect":
230
  // The comma-joined SET, read back with breathing room ("A, B" not "A,B").
231
  return String(v ?? "")
 
193
  * a value reads identically on the canvas, in the panel and in a file. Returns
194
  * "" for empty so callers can substitute their own placeholder.
195
  */
196
+ /**
197
+ * Wave-18 C5-AUTOFIELD — the STATE word of an automation cell.
198
+ *
199
+ * An automation cell holds one machine-written line: `state · when · detail`, e.g.
200
+ * `ok · 2026-08-03 14:10 · 12 posts`. The state is the first token, and every surface that
201
+ * paints the cell — canvas tint, record modal, the rail — reads it through HERE rather than
202
+ * re-splitting the string, because two parsers of one format is how the canvas and the panel
203
+ * end up disagreeing about whether a run succeeded (the exact way `formula` broke above).
204
+ *
205
+ * `"none"` is the honest answer for a cell nothing has written yet — NOT "ok". A column that
206
+ * has never run must not look like a column that ran and found nothing.
207
+ */
208
+ export type AutomationState = "ok" | "partial" | "error" | "blocked" | "queued" | "none";
209
+
210
+ const AUTOMATION_STATES: AutomationState[] = ["ok", "partial", "error", "blocked", "queued"];
211
+
212
+ export function automationState(v: CellValue): AutomationState {
213
+ const head = String(v ?? "").split("·")[0].trim().toLowerCase();
214
+ return (AUTOMATION_STATES as string[]).includes(head)
215
+ ? (head as AutomationState)
216
+ : "none";
217
+ }
218
+
219
+ /** Everything after the state word — the timestamp and the run's own detail. */
220
+ export function automationDetail(v: CellValue): string {
221
+ const parts = String(v ?? "").split("·");
222
+ return parts.length > 1 ? parts.slice(1).join("·").trim() : "";
223
+ }
224
+
225
+ /** Sentence-cased state, for anywhere a word reads better than a token. */
226
+ export function automationStateLabel(v: CellValue): string {
227
+ const s = automationState(v);
228
+ return s === "none" ? "Not run yet" : s[0].toUpperCase() + s.slice(1);
229
+ }
230
+
231
  export function formatDisplay(field: Field, v: CellValue): string {
232
  switch (field.type) {
233
  case "currency":
 
261
  const n = num(v);
262
  return n >= 1 ? `${Math.round(n)} of ${ratingMax(field)}` : "";
263
  }
264
+ case "automation":
265
+ // The machine-written line, verbatim. Explicit rather than left to the `default` branch
266
+ // below: `formula` fell through a default once and printed every text result as `0` for
267
+ // months, and the lesson recorded there is that a type whose rendering is deliberate
268
+ // should SAY so where a reader looks for it.
269
+ return String(v ?? "");
270
  case "multiselect":
271
  // The comma-joined SET, read back with breathing room ("A, B" not "A,B").
272
  return String(v ?? "")
web/src/customer-grid/iconShapes.ts CHANGED
@@ -96,6 +96,15 @@ export const TYPE_SHAPES: Record<FieldType, IconShape[]> = {
96
  { d: "M4.2 7.6h4.6" },
97
  { d: "M10.2 8.4l3 3.4M13.2 8.4l-3 3.4" },
98
  ],
 
 
 
 
 
 
 
 
 
99
  };
100
 
101
  /**
@@ -135,6 +144,16 @@ export const MODE_SHAPES: Record<DisplayMode, IconShape[]> = {
135
  { d: "M2.6 6.6h10.8M6.2 3.4v9.2" },
136
  { d: "M7.2 10.8l2-2.4 1.6 1.2 1.8-2.6" },
137
  ],
 
 
 
 
 
 
 
 
 
 
138
  };
139
 
140
  /**
@@ -296,6 +315,7 @@ export const MODE_LABELS: Record<DisplayMode, string> = {
296
  kanban: "Kanban",
297
  map: "Map",
298
  timeseries: "Time series",
 
299
  };
300
 
301
  /**
@@ -323,6 +343,11 @@ export const MODE_LABELS: Record<DisplayMode, string> = {
323
  // as the other record-shaped mode. The owner put Time series FIFTH and List LAST. An order is
324
  // a product decision, so it is asserted in `verify_icons` rather than left to a comment —
325
  // nothing else on screen would go red if a future edit re-sorted it "sensibly".
 
 
 
 
 
326
  export const CREATABLE_MODES: DisplayMode[] = [
327
  "grid",
328
  "chart",
@@ -331,6 +356,7 @@ export const CREATABLE_MODES: DisplayMode[] = [
331
  "timeseries",
332
  "map",
333
  "list",
 
334
  ];
335
 
336
  export const MODE_TONE: Record<DisplayMode, FolderTone> = {
@@ -343,6 +369,11 @@ export const MODE_TONE: Record<DisplayMode, FolderTone> = {
343
  map: "red",
344
  // Green with `chart`: it is the other analytical mode, and the two belong to one family.
345
  timeseries: "green",
 
 
 
 
 
346
  };
347
 
348
  /**
@@ -366,6 +397,8 @@ export const TYPE_LABELS: Record<FieldType, string> = {
366
  rating: "Rating",
367
  created_time: "Created time",
368
  formula: "Formula",
 
 
369
  status: "Lifecycle status (Odoo)", // never creatable; present so the map stays total
370
  };
371
 
 
96
  { d: "M4.2 7.6h4.6" },
97
  { d: "M10.2 8.4l3 3.4M13.2 8.4l-3 3.4" },
98
  ],
99
+ // Wave-18 C5-AUTOFIELD (D's spec, applied by C as client-vocab registrar). A 290° cycle ring
100
+ // with an arrowhead, wrapped around a solid run-triangle: a job that runs, repeatedly.
101
+ // Deliberately NOT a bolt (`FOLDER_SHAPE_PATHS.bolt` already means "Priority") and not a clock
102
+ // (`created_time` owns the closed rim + hands).
103
+ automation: [
104
+ { d: "M10.8 4.1A4.8 4.8 0 1 1 5.3 4.1" },
105
+ { d: "M4.2 6L5.3 4.1 3.1 4.5" },
106
+ { d: "M6.9 6.1L9.8 8 6.9 9.9z", fill: true },
107
+ ],
108
  };
109
 
110
  /**
 
144
  { d: "M2.6 6.6h10.8M6.2 3.4v9.2" },
145
  { d: "M7.2 10.8l2-2.4 1.6 1.2 1.8-2.6" },
146
  ],
147
+ // Wave-18 C6-CATALOG — an OPEN BOOK: two facing leaves with a spine between them, and a
148
+ // product block sitting on the left one. Every other mode in this table draws a way of
149
+ // arranging RECORDS; this one has to read as a printed artifact, so it is the only mark here
150
+ // with a spine and a gutter. Deliberately not a page-with-lines (`list` owns that reading) and
151
+ // not a framed grid (`grid`/`chart`/`timeseries` share the frame).
152
+ catalog: [
153
+ { d: "M2.4 4.2h4.6a1.6 1.6 0 0 1 1 .4l0 8a1.6 1.6 0 0 0-1-.4H2.4z" },
154
+ { d: "M13.6 4.2H9a1.6 1.6 0 0 0-1 .4l0 8a1.6 1.6 0 0 1 1-.4h4.6z" },
155
+ { d: "M3.9 6.4h2.3v2.8H3.9z" },
156
+ ],
157
  };
158
 
159
  /**
 
315
  kanban: "Kanban",
316
  map: "Map",
317
  timeseries: "Time series",
318
+ catalog: "Catalog",
319
  };
320
 
321
  /**
 
343
  // as the other record-shaped mode. The owner put Time series FIFTH and List LAST. An order is
344
  // a product decision, so it is asserted in `verify_icons` rather than left to a comment —
345
  // nothing else on screen would go red if a future edit re-sorted it "sensibly".
346
+ // Wave-18 C6-CATALOG — `catalog` was held out of this list until `aios_grid.DISPLAY_MODES`
347
+ // accepted the name, the same hold `timeseries` and `chart` served before it. SESSION A posted
348
+ // "C6 HOST MIRROR APPLIED — you may flip CREATABLE_MODES now" (2026-08-03), so it is offerable.
349
+ // It lands LAST by contract: the owner's seven-mode order above is a product decision and the
350
+ // new mode joins the end of it rather than being sorted into it.
351
  export const CREATABLE_MODES: DisplayMode[] = [
352
  "grid",
353
  "chart",
 
356
  "timeseries",
357
  "map",
358
  "list",
359
+ "catalog",
360
  ];
361
 
362
  export const MODE_TONE: Record<DisplayMode, FolderTone> = {
 
369
  map: "red",
370
  // Green with `chart`: it is the other analytical mode, and the two belong to one family.
371
  timeseries: "green",
372
+ // Wave-18 C6-CATALOG — NEUTRAL, and it is the honest pick rather than the leftover one. The
373
+ // four colour tones each name a family of ways to arrange records (blue = tabular, green =
374
+ // analytical, yellow = board/date, red = spatial); a catalog arranges nothing — it is a
375
+ // published artifact. Giving it a colour would file it under a family it is not in.
376
+ catalog: "neutral",
377
  };
378
 
379
  /**
 
397
  rating: "Rating",
398
  created_time: "Created time",
399
  formula: "Formula",
400
+ // Wave-18 C5-AUTOFIELD (D's spec, applied by C).
401
+ automation: "Automation",
402
  status: "Lifecycle status (Odoo)", // never creatable; present so the map stays total
403
  };
404
 
web/src/customer-grid/theme.ts CHANGED
@@ -56,6 +56,23 @@ export const LP_PURPLE = "#C3B3EE"; // FILL only (1.91:1)
56
  export const LP_PURPLE_TINT = "#F1EEFB"; // the group chip wash + its column header
57
  export const LP_PURPLE_DEEP = "#6B57A8"; // 5.17:1 on its own tint
58
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
59
  export const LP_BLUE_DEEP = "#768FB6"; // 3.29:1 on white — rings, control borders
60
  /* THE PRODUCT PRIMARY — the same #5B8FD9 the host's `.streamlit/config.toml`
61
  paints on every primary button, so embed and standalone stop disagreeing
 
56
  export const LP_PURPLE_TINT = "#F1EEFB"; // the group chip wash + its column header
57
  export const LP_PURPLE_DEEP = "#6B57A8"; // 5.17:1 on its own tint
58
 
59
+ /**
60
+ * ⭐ Wave-18 item 1b — THE VOID: everything past the last row and the last column.
61
+ *
62
+ * SAMPLED, not chosen — `reference/Airtable 7.png` at (700,400), (1250,170) and (900,520) all
63
+ * read #F6F8FC, with the rows themselves pure #FFFFFF. It is a hair COOLER than `LP_WASH`
64
+ * (#FAFBFC, the header band) rather than a shade of it, which is the point: the header is part
65
+ * of the table and reads as chrome; the void is the absence of table and reads as paper.
66
+ *
67
+ * ⚠ MIRRORED, NOT USED, on this side. The void is painted by two DOM elements over the canvas
68
+ * (`.cg-grid-void`, see `CustomerGrid`'s W18-B VOID block for why it is not a theme key), so
69
+ * the live value is `--lp-void` in index.css. This constant exists to put the hex under
70
+ * `verify_icons`' I7c parity leg, which diffs the `LP_*` exports here against the `--lp-*`
71
+ * declarations there — a colour that lives in only one of the two files is a colour that can
72
+ * drift without any gate noticing.
73
+ */
74
+ export const LP_VOID = "#F6F8FC";
75
+
76
  export const LP_BLUE_DEEP = "#768FB6"; // 3.29:1 on white — rings, control borders
77
  /* THE PRODUCT PRIMARY — the same #5B8FD9 the host's `.streamlit/config.toml`
78
  paints on every primary button, so embed and standalone stop disagreeing
web/src/customer-grid/types.ts CHANGED
@@ -33,7 +33,11 @@ export type FieldType =
33
  // `rating` is an integer 1..max drawn as stars (SVG/canvas paths, NEVER emoji — owner
34
  // constant); `created_time` is READ-ONLY and renders the row's `_created`; `formula`
35
  // (item 9) computes client-side over the row via formulaEngine.ts.
36
- | "checkbox" | "phone" | "email" | "url" | "rating" | "created_time" | "formula";
 
 
 
 
37
 
38
  export type FieldSource = "odoo" | "overlay";
39
 
@@ -42,7 +46,7 @@ export type FieldSource = "odoo" | "overlay";
42
  * user-defined set. */
43
  export const CREATABLE_TYPES: readonly FieldType[] = [
44
  "text", "select", "multiselect", "user", "int", "currency", "pct", "date",
45
- "checkbox", "phone", "email", "url", "rating", "created_time", "formula",
46
  ];
47
 
48
  /**
@@ -51,7 +55,7 @@ export const CREATABLE_TYPES: readonly FieldType[] = [
51
  * about the row. Distinct from permissions — these are read-only by NATURE, not by policy.
52
  */
53
  export const READONLY_CELL_TYPES: ReadonlySet<FieldType> = new Set<FieldType>([
54
- "formula", "created_time",
55
  ]);
56
 
57
  /** Types whose value is chosen from a fixed list rather than typed. */
@@ -188,8 +192,14 @@ export type FieldScope = "cohort" | "global";
188
  // is the whole point: `_clean_display` DROPS an unknown mode, so offering the mode first would
189
  // let a user create a time-series view that silently reverts to a grid on the next read, with
190
  // nothing anywhere going red — the wave-9 silent-drop failure, replayed.
 
 
 
 
 
 
191
  export const DISPLAY_MODES = [
192
- "grid", "chart", "dashboard", "list", "calendar", "kanban", "map", "timeseries",
193
  ] as const;
194
  export type DisplayMode = (typeof DISPLAY_MODES)[number];
195
 
@@ -311,6 +321,17 @@ export interface DisplaySpec {
311
  * migration.
312
  */
313
  tsStyles?: Record<string, TsCellStyle>;
 
 
 
 
 
 
 
 
 
 
 
314
  }
315
 
316
  /**
@@ -336,6 +357,88 @@ export const TS_MAX_FIELDS = 12;
336
  /** C-DISP — the cap on a calendar view's summary metrics. */
337
  export const MAX_CALENDAR_METRICS = 4;
338
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
339
  // ── C-TYPES2 (wave 14) — the time-series surface's vocabulary ────────────────────────────────
340
  // Same construction as TS_BUCKETS above: const arrays rather than bare unions, so the validator
341
  // below and the pickers TSREC builds read ONE list. Mirrored by the host's `_clean_display`
@@ -441,6 +544,142 @@ function cleanChartPeriod(c: ChartSpecLike): ChartSpecLike {
441
  return next;
442
  }
443
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
444
  export function cleanDisplay(raw: unknown): DisplaySpec | undefined {
445
  if (!raw || typeof raw !== "object" || Array.isArray(raw)) return undefined;
446
  const d = raw as Record<string, unknown>;
@@ -587,6 +826,9 @@ export function cleanDisplay(raw: unknown): DisplaySpec | undefined {
587
  }
588
  if (kept) out.tsStyles = styles;
589
  }
 
 
 
590
  if (
591
  out.mode === "grid" &&
592
  !out.dateField &&
@@ -612,7 +854,12 @@ export function cleanDisplay(raw: unknown): DisplaySpec | undefined {
612
  out.tsGridlines === undefined &&
613
  out.tsSparkline === undefined &&
614
  !out.tsRows &&
615
- !out.tsStyles
 
 
 
 
 
616
  )
617
  return undefined;
618
  return out;
@@ -779,6 +1026,21 @@ export interface Field {
779
  * right, which is why `verify_formula_engine` now asserts it (and drives it red both ways).
780
  */
781
  formula?: string;
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
782
  /**
783
  * Wave-5 item 11 (same amendment): a RATING field's star count, TOP-LEVEL. 2..10, default 5
784
  * — read through `ratingMax`, never directly.
@@ -1511,10 +1773,40 @@ export const PRODUCT_TOPIC: TopicConfig = {
1511
  measures: false,
1512
  };
1513
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1514
  /** The topic a surface scope draws — `cohort` IS the customer topic over fixed sets. */
1515
  export function topicForScope(scope: string): TopicConfig {
 
1516
  return scope === "product" ? PRODUCT_TOPIC : CUSTOMER_TOPIC;
1517
  }
 
1518
  export function isUndeletableView(view: Pick<SavedView, "id">): boolean {
1519
  return UNDELETABLE_VIEW_IDS.has(view.id);
1520
  }
 
33
  // `rating` is an integer 1..max drawn as stars (SVG/canvas paths, NEVER emoji — owner
34
  // constant); `created_time` is READ-ONLY and renders the row's `_created`; `formula`
35
  // (item 9) computes client-side over the row via formulaEngine.ts.
36
+ | "checkbox" | "phone" | "email" | "url" | "rating" | "created_time" | "formula"
37
+ // Wave-18 C5-AUTOFIELD — `automation` is a column that a scheduled job WRITES. The cell holds
38
+ // a machine-written status string (never user input); the column's behaviour is configured
39
+ // through the field's gear, like `formula` carries its expression.
40
+ | "automation";
41
 
42
  export type FieldSource = "odoo" | "overlay";
43
 
 
46
  * user-defined set. */
47
  export const CREATABLE_TYPES: readonly FieldType[] = [
48
  "text", "select", "multiselect", "user", "int", "currency", "pct", "date",
49
+ "checkbox", "phone", "email", "url", "rating", "created_time", "formula", "automation",
50
  ];
51
 
52
  /**
 
55
  * about the row. Distinct from permissions — these are read-only by NATURE, not by policy.
56
  */
57
  export const READONLY_CELL_TYPES: ReadonlySet<FieldType> = new Set<FieldType>([
58
+ "formula", "created_time", "automation",
59
  ]);
60
 
61
  /** Types whose value is chosen from a fixed list rather than typed. */
 
192
  // is the whole point: `_clean_display` DROPS an unknown mode, so offering the mode first would
193
  // let a user create a time-series view that silently reverts to a grid on the next read, with
194
  // nothing anywhere going red — the wave-9 silent-drop failure, replayed.
195
+ // Wave-18 C6-CATALOG (owner item 4): `catalog` joins the union — a view that holds PRINT
196
+ // artifacts. Same split the two modes above it were landed with, and for the same reason: it
197
+ // is in DISPLAY_MODES (so the type, the icon table and the label table are total over it) but
198
+ // held OUT of CREATABLE_MODES until `aios_grid.DISPLAY_MODES` accepts the name, because
199
+ // `_clean_display` drops an unknown mode and offering it first would let a user build a whole
200
+ // catalogue that silently reverts to a grid on the next read with nothing going red.
201
  export const DISPLAY_MODES = [
202
+ "grid", "chart", "dashboard", "list", "calendar", "kanban", "map", "timeseries", "catalog",
203
  ] as const;
204
  export type DisplayMode = (typeof DISPLAY_MODES)[number];
205
 
 
321
  * migration.
322
  */
323
  tsStyles?: Record<string, TsCellStyle>;
324
+ /**
325
+ * Wave-18 C6-CATALOG (owner item 4) — the PRINT artifacts this view holds, ≤ MAX_CATALOGS.
326
+ *
327
+ * Typed as `CatalogSpec[]` here rather than structurally (the `charts` posture) because the
328
+ * shape has no home further down the import graph: `CatalogView.tsx` imports types.ts, not the
329
+ * reverse, and the validator that enforces the shape lives in this file beside every other
330
+ * `cleanDisplay` clause. The catalogs are the only display key whose payload is CONTENT rather
331
+ * than a projection of the rows — everything else here says how to arrange records, and these
332
+ * say what to print.
333
+ */
334
+ catalogs?: CatalogSpec[];
335
  }
336
 
337
  /**
 
357
  /** C-DISP — the cap on a calendar view's summary metrics. */
358
  export const MAX_CALENDAR_METRICS = 4;
359
 
360
+ // ── C6-CATALOG (wave 18, owner item 4) — the catalog view's vocabulary and caps ──────────────
361
+ // Every name below is mirrored by `aios_grid.py` under the SAME name, and `cleanCatalogs`
362
+ // applies the same drops in the same ORDER as `_clean_catalogs`, so an accepted save reads back
363
+ // identically on both engines. Const arrays rather than bare unions, the TS_BUCKETS
364
+ // construction: the validator and the designer's pickers read ONE list.
365
+ export const CATALOG_PAPERS = ["letter", "a4", "tabloid"] as const;
366
+ export type CatalogPaper = (typeof CATALOG_PAPERS)[number];
367
+ export const CATALOG_ORIENTATIONS = ["portrait", "landscape"] as const;
368
+ export type CatalogOrientation = (typeof CATALOG_ORIENTATIONS)[number];
369
+ /** R9 — which derivative the images render at. Absent = `web` (the in-app default). */
370
+ export const CATALOG_QUALITIES = ["web", "print"] as const;
371
+ export type CatalogQuality = (typeof CATALOG_QUALITIES)[number];
372
+ export const CATALOG_PAGE_KINDS = ["cover", "intro", "section", "gallery"] as const;
373
+ export type CatalogPageKind = (typeof CATALOG_PAGE_KINDS)[number];
374
+ /** The product-grid widths a section page offers. */
375
+ export const CATALOG_COLS = [2, 3, 4] as const;
376
+
377
+ export const MAX_CATALOGS = 12;
378
+ export const MAX_CATALOG_PAGES = 40;
379
+ /** Cumulative across ONE catalog's pages, spent in PAGE ORDER — see `cleanCatalogs`. */
380
+ export const MAX_CATALOG_CODES = 500;
381
+ export const CATALOG_ID_MAX = 40;
382
+ export const CATALOG_NAME_MAX = 80;
383
+ export const CATALOG_TITLE_MAX = 120;
384
+ export const CATALOG_BODY_MAX = 2000;
385
+ export const CATALOG_CODE_MAX = 60;
386
+ /** Six-digit CSS hex, the same shape `optionColors` is validated as at both boundaries. */
387
+ const CATALOG_HEX6 = /^#[0-9A-Fa-f]{6}$/;
388
+
389
+ /**
390
+ * One page of a catalog. `kind` decides which template paints it:
391
+ * · `cover` — full-bleed image, wordmark, display-serif title
392
+ * · `intro` — brand field + body copy (the 2027 catalogue's letter spread)
393
+ * · `section` — the workhorse: a titled product grid, optional full-bleed footer strip
394
+ * · `gallery` — one full-bleed image with an optional overlaid title
395
+ *
396
+ * `products` holds product CODES (the `product_data` business key), never pids: a catalogue
397
+ * outlives any one pool fetch, and a code is the thing printed on the page.
398
+ */
399
+ export interface CatalogPage {
400
+ id: string;
401
+ kind: CatalogPageKind;
402
+ title?: string;
403
+ body?: string;
404
+ /** An ASSET code resolved through C2-ASSET (`/assets/products/{code}`), never a URL. */
405
+ imageCode?: string;
406
+ products?: string[];
407
+ /**
408
+ * ⚠ The three toggles are LITERAL-ONLY, and in opposite directions — the
409
+ * `kanbanClamp`/`tsSparkline` asymmetry, one per direction. Pack and colour SHOW by default
410
+ * (the 2027 catalogue prints both), price does NOT (it prints no prices at all), so only the
411
+ * opt-OUT is storable for the first two and only the opt-IN for the third. A second spelling
412
+ * of a default is how a round trip starts churning, and the literal types make that a compile
413
+ * error rather than a convention.
414
+ */
415
+ layout?: { cols?: number; showPack?: false; showColor?: false; showPrice?: true };
416
+ }
417
+
418
+ /**
419
+ * One catalog. A view holds up to `MAX_CATALOGS` of them.
420
+ *
421
+ * `paper`/`orientation` are the two keys a print artifact cannot be missing — a page box with
422
+ * no size is not a smaller catalogue, it is an unrenderable one — so they are NORMALISED WITH A
423
+ * DEFAULT rather than dropped, unlike every other optional key here.
424
+ *
425
+ * `fields` binds the listing lines to real columns. `product_data` carries no pack, colour or
426
+ * price of its own (measured), so the user binds custom fields; an unbound line does not render.
427
+ * The pack pill prints its bound value VERBATIM ("6-Piece per Pack") rather than composing one
428
+ * from a number — p121 of the reference says "6-Sets Per Case", and any composed rule is wrong
429
+ * there.
430
+ */
431
+ export interface CatalogSpec {
432
+ id: string;
433
+ name: string;
434
+ paper: CatalogPaper;
435
+ orientation: CatalogOrientation;
436
+ quality?: CatalogQuality;
437
+ brand?: { primary?: string; accent?: string; company?: string; logo?: string };
438
+ fields?: { name?: string; pack?: string; color?: string; price?: string };
439
+ pages: CatalogPage[];
440
+ }
441
+
442
  // ── C-TYPES2 (wave 14) — the time-series surface's vocabulary ────────────────────────────────
443
  // Same construction as TS_BUCKETS above: const arrays rather than bare unions, so the validator
444
  // below and the pickers TSREC builds read ONE list. Mirrored by the host's `_clean_display`
 
544
  return next;
545
  }
546
 
547
+ /**
548
+ * C6-CATALOG — one page of a catalog. Returns the page and what it SPENT of the catalog's code
549
+ * allowance, because the 500-code cap is cumulative and order-sensitive (see `cleanCatalogs`).
550
+ *
551
+ * Codes are deduped WITHIN a page and not across the catalog: a product legitimately appears on
552
+ * a gallery page and again in its section listing, and de-duplicating globally would silently
553
+ * delete the second appearance.
554
+ *
555
+ * ⚠ Byte-for-byte the host's `_clean_catalog_page`, including the key EMISSION ORDER
556
+ * (`id, kind, title?, body?, imageCode?, products?, layout?`) — an accepted save has to read
557
+ * back the same on both engines, and this pair is the only place that is decided.
558
+ */
559
+ function cleanCatalogPage(raw: unknown, budget: number): { page: CatalogPage; spent: number } | null {
560
+ if (!raw || typeof raw !== "object" || Array.isArray(raw)) return null;
561
+ const p = raw as Record<string, unknown>;
562
+ const id = typeof p.id === "string" ? p.id.slice(0, CATALOG_ID_MAX) : "";
563
+ if (!id) return null;
564
+ if (
565
+ typeof p.kind !== "string" ||
566
+ !(CATALOG_PAGE_KINDS as readonly string[]).includes(p.kind)
567
+ )
568
+ return null;
569
+ const page: CatalogPage = { id, kind: p.kind as CatalogPageKind };
570
+ for (const [key, cap] of [
571
+ ["title", CATALOG_TITLE_MAX],
572
+ ["body", CATALOG_BODY_MAX],
573
+ ["imageCode", CATALOG_CODE_MAX],
574
+ ] as const) {
575
+ const v = p[key];
576
+ if (typeof v === "string" && v) page[key] = v.slice(0, cap);
577
+ }
578
+ if (Array.isArray(p.products) && budget > 0) {
579
+ const codes: string[] = [];
580
+ const seen = new Set<string>();
581
+ for (const c of p.products) {
582
+ if (typeof c !== "string" || !c) continue;
583
+ const code = c.slice(0, CATALOG_CODE_MAX);
584
+ if (seen.has(code)) continue;
585
+ seen.add(code);
586
+ codes.push(code);
587
+ if (codes.length >= budget) break;
588
+ }
589
+ if (codes.length) page.products = codes;
590
+ }
591
+ if (p.layout && typeof p.layout === "object" && !Array.isArray(p.layout)) {
592
+ const l = p.layout as Record<string, unknown>;
593
+ const layout: NonNullable<CatalogPage["layout"]> = {};
594
+ if (
595
+ typeof l.cols === "number" &&
596
+ Number.isInteger(l.cols) &&
597
+ (CATALOG_COLS as readonly number[]).includes(l.cols)
598
+ )
599
+ layout.cols = l.cols;
600
+ // ⚠ Literal-only, and in OPPOSITE directions — see the member's doc comment. Truthiness
601
+ // here would normalise the user's pick away in both directions at once.
602
+ if (l.showPack === false) layout.showPack = false;
603
+ if (l.showColor === false) layout.showColor = false;
604
+ if (l.showPrice === true) layout.showPrice = true;
605
+ if (layout.cols !== undefined || layout.showPack !== undefined ||
606
+ layout.showColor !== undefined || layout.showPrice !== undefined)
607
+ page.layout = layout;
608
+ }
609
+ return { page, spent: page.products?.length ?? 0 };
610
+ }
611
+
612
+ /**
613
+ * C6-CATALOG (wave 18) — `display.catalogs`, fail-closed. Exported so the gate can drive it
614
+ * directly rather than only through `cleanDisplay`.
615
+ *
616
+ * Field REFS (`fields.*`) are not resolved here — the same split `dateField`/`stackField`
617
+ * already run: the host owns `valid_keys`, and the designer only offers real columns, so a
618
+ * stale ref degrades to "that line does not render" rather than to an error.
619
+ */
620
+ export function cleanCatalogs(raw: unknown): CatalogSpec[] | undefined {
621
+ if (!Array.isArray(raw)) return undefined;
622
+ const out: CatalogSpec[] = [];
623
+ for (const item of raw) {
624
+ if (out.length >= MAX_CATALOGS) break;
625
+ if (!item || typeof item !== "object" || Array.isArray(item)) continue;
626
+ const c = item as Record<string, unknown>;
627
+ const id = typeof c.id === "string" ? c.id.slice(0, CATALOG_ID_MAX) : "";
628
+ // An EMPTY name is legal (the user cleared the box and will type again) — an ABSENT one is
629
+ // a malformed record. The `TsCustomRow` label rule, same reasoning.
630
+ if (!id || typeof c.name !== "string") continue;
631
+ const cat: CatalogSpec = {
632
+ id,
633
+ name: c.name.slice(0, CATALOG_NAME_MAX),
634
+ // NORMALISED WITH A DEFAULT, not dropped: a page box with no size is not a smaller
635
+ // catalogue, it is an unrenderable one.
636
+ paper: (CATALOG_PAPERS as readonly string[]).includes(c.paper as string)
637
+ ? (c.paper as CatalogPaper)
638
+ : "letter",
639
+ orientation: (CATALOG_ORIENTATIONS as readonly string[]).includes(c.orientation as string)
640
+ ? (c.orientation as CatalogOrientation)
641
+ : "portrait",
642
+ pages: [],
643
+ };
644
+ if ((CATALOG_QUALITIES as readonly string[]).includes(c.quality as string))
645
+ cat.quality = c.quality as CatalogQuality;
646
+ if (c.brand && typeof c.brand === "object" && !Array.isArray(c.brand)) {
647
+ const b = c.brand as Record<string, unknown>;
648
+ const brand: NonNullable<CatalogSpec["brand"]> = {};
649
+ for (const k of ["primary", "accent"] as const)
650
+ if (typeof b[k] === "string" && CATALOG_HEX6.test(b[k] as string))
651
+ brand[k] = b[k] as string;
652
+ if (typeof b.company === "string" && b.company)
653
+ brand.company = b.company.slice(0, CATALOG_NAME_MAX);
654
+ // An asset CODE, never a URL: an arbitrary host inside print CSS is exactly what the
655
+ // tokens-not-values rule exists to stop.
656
+ if (typeof b.logo === "string" && b.logo) brand.logo = b.logo.slice(0, CATALOG_CODE_MAX);
657
+ if (Object.keys(brand).length) cat.brand = brand;
658
+ }
659
+ if (c.fields && typeof c.fields === "object" && !Array.isArray(c.fields)) {
660
+ const f = c.fields as Record<string, unknown>;
661
+ const binds: NonNullable<CatalogSpec["fields"]> = {};
662
+ for (const k of ["name", "pack", "color", "price"] as const)
663
+ if (typeof f[k] === "string" && f[k]) binds[k] = f[k] as string;
664
+ if (Object.keys(binds).length) cat.fields = binds;
665
+ }
666
+ let budget = MAX_CATALOG_CODES;
667
+ if (Array.isArray(c.pages)) {
668
+ for (const rawPage of c.pages) {
669
+ if (cat.pages.length >= MAX_CATALOG_PAGES) break;
670
+ const cleaned = cleanCatalogPage(rawPage, budget);
671
+ if (!cleaned) continue;
672
+ budget -= cleaned.spent;
673
+ cat.pages.push(cleaned.page);
674
+ }
675
+ }
676
+ // `pages` is ALWAYS emitted, even empty: a catalogue with no pages yet is the state every
677
+ // catalogue starts in, and dropping the key would make "new" and "corrupt" one wire value.
678
+ out.push(cat);
679
+ }
680
+ return out.length ? out : undefined;
681
+ }
682
+
683
  export function cleanDisplay(raw: unknown): DisplaySpec | undefined {
684
  if (!raw || typeof raw !== "object" || Array.isArray(raw)) return undefined;
685
  const d = raw as Record<string, unknown>;
 
826
  }
827
  if (kept) out.tsStyles = styles;
828
  }
829
+ // ── C6-CATALOG (wave 18) ────────────────────────────────────────────────────────────────
830
+ const catalogs = cleanCatalogs(d.catalogs);
831
+ if (catalogs) out.catalogs = catalogs;
832
  if (
833
  out.mode === "grid" &&
834
  !out.dateField &&
 
854
  out.tsGridlines === undefined &&
855
  out.tsSparkline === undefined &&
856
  !out.tsRows &&
857
+ !out.tsStyles &&
858
+ // C6-CATALOG — catalogs ride the W13 carry like every key above, and this line is the one
859
+ // that makes that true. Without it, switching a catalog view back to Grid reduces the whole
860
+ // spec to `undefined` and the user's catalogues are GONE on the next save — the only key
861
+ // here whose payload is authored content rather than a re-derivable projection.
862
+ !out.catalogs
863
  )
864
  return undefined;
865
  return out;
 
1026
  * right, which is why `verify_formula_engine` now asserts it (and drives it red both ways).
1027
  */
1028
  formula?: string;
1029
+ /**
1030
+ * Wave-18 C5-AUTOFIELD — what this column's automation DOES. Absent = unconfigured (the cells
1031
+ * render "Not configured" and no run ever targets the column). Written through the field's
1032
+ * gear; validated host-side (`aios_grid._clean_automation`) exactly as `options`/`formula` are.
1033
+ */
1034
+ automation?: {
1035
+ /** v1 vocabulary: 'instagram_profile'. Host-whitelisted; an unknown kind is dropped. */
1036
+ kind: string;
1037
+ /** where the run gets its subject. 'record_url_field' reads `urlField` off the row. */
1038
+ source: "record_url_field" | "self";
1039
+ /** the row field holding the profile URL when source = 'record_url_field'. */
1040
+ urlField?: string;
1041
+ /** kind-specific knobs. Scalars only — the host clamps keys and value lengths. */
1042
+ settings?: Record<string, string | number | boolean>;
1043
+ };
1044
  /**
1045
  * Wave-5 item 11 (same amendment): a RATING field's star count, TOP-LEVEL. 2..10, default 5
1046
  * — read through `ratingMax`, never directly.
 
1773
  measures: false,
1774
  };
1775
 
1776
+ /* ═══ W18-A UT-TOPIC region (INTEGRATOR-owned inside this C-owned file — dated ownership
1777
+ amendment in the wave-18 doc's HARD RULES; SESSION C: edit nothing between these anchors) */
1778
+ /**
1779
+ * Wave 18 (C3-UT): a `ut_`-prefixed scope is a USER TABLE — the third topic through the one
1780
+ * wire. Its rows endpoint is the /customers-shaped envelope at `tables/<key>/rows`.
1781
+ *
1782
+ * ⚠ MEMOIZED PER KEY, deliberately: `useCustomerData`'s load effect depends on the topic's
1783
+ * OBJECT IDENTITY (`[scope, topic]`). CUSTOMER_TOPIC/PRODUCT_TOPIC are module constants; a
1784
+ * fresh object per call here would re-run that effect every render — an infinite refetch.
1785
+ */
1786
+ const UT_TOPICS = new Map<string, TopicConfig>();
1787
+
1788
+ export function utTopic(scope: string): TopicConfig {
1789
+ let t = UT_TOPICS.get(scope);
1790
+ if (!t) {
1791
+ t = {
1792
+ key: scope,
1793
+ scope,
1794
+ rowsPath: `tables/${scope}/rows`,
1795
+ noun: "record",
1796
+ nouns: "records",
1797
+ measures: false,
1798
+ };
1799
+ UT_TOPICS.set(scope, t);
1800
+ }
1801
+ return t;
1802
+ }
1803
+
1804
  /** The topic a surface scope draws — `cohort` IS the customer topic over fixed sets. */
1805
  export function topicForScope(scope: string): TopicConfig {
1806
+ if (scope.startsWith("ut_")) return utTopic(scope);
1807
  return scope === "product" ? PRODUCT_TOPIC : CUSTOMER_TOPIC;
1808
  }
1809
+ /* ═══ end W18-A UT-TOPIC region ═══ */
1810
  export function isUndeletableView(view: Pick<SavedView, "id">): boolean {
1811
  return UNDELETABLE_VIEW_IDS.has(view.id);
1812
  }
web/src/customer-grid/useCustomerData.ts CHANGED
@@ -27,7 +27,7 @@ import type { Dispatch, SetStateAction } from "react";
27
  import { topicForScope } from "./types";
28
  import type { CustomersPayload, Field, GridWorkspace, Row } from "./types";
29
  import { fetchTopicRows, fetchWorkspace, patchTopicRow, setSurfaceScope } from "./apiBridge";
30
- import { DERIVED_CELLS_EVENT, WORKSPACE_STALE_EVENT } from "../apiContract";
31
  import type { SurfaceScope } from "./apiBridge";
32
  import {
33
  emitHostEvent,
@@ -169,6 +169,16 @@ export function useCustomerData(scope: SurfaceScope = "customer"): CustomerData
169
  );
170
  const dataRef = useRef<CustomersPayload | null>(data);
171
  dataRef.current = data;
 
 
 
 
 
 
 
 
 
 
172
 
173
  useEffect(() => {
174
  if (STREAMLIT_COMPONENT) {
@@ -282,8 +292,10 @@ export function useCustomerData(scope: SurfaceScope = "customer"): CustomerData
282
  window.removeEventListener(DERIVED_CELLS_EVENT, onDerived);
283
  }
284
  };
285
- // `topic` is a stable module constant per scope (topicForScope), so this cannot loop.
286
- }, [scope, topic]);
 
 
287
 
288
  // Optimistic overlay write: apply the edit immediately (so the grid cell and
289
  // the detail panel reflect it at once), snapshot the pre-edit entry, then
 
27
  import { topicForScope } from "./types";
28
  import type { CustomersPayload, Field, GridWorkspace, Row } from "./types";
29
  import { fetchTopicRows, fetchWorkspace, patchTopicRow, setSurfaceScope } from "./apiBridge";
30
+ import { DERIVED_CELLS_EVENT, ROWS_STALE_EVENT, WORKSPACE_STALE_EVENT } from "../apiContract";
31
  import type { SurfaceScope } from "./apiBridge";
32
  import {
33
  emitHostEvent,
 
169
  );
170
  const dataRef = useRef<CustomersPayload | null>(data);
171
  dataRef.current = data;
172
+ // Wave 18 (C3-UT): a bump re-runs the load effect below — fired by ROWS_STALE_EVENT when
173
+ // something OUTSIDE the grid's own write path changed rows (shell Add-record, an automation
174
+ // run). Senders clear the rows cache first, so the refetch cannot be served from the memo.
175
+ const [rowsBump, setRowsBump] = useState(0);
176
+ useEffect(() => {
177
+ if (STREAMLIT_COMPONENT) return; // embed rows arrive from the host, not this path
178
+ const onStale = () => setRowsBump((b) => b + 1);
179
+ window.addEventListener(ROWS_STALE_EVENT, onStale);
180
+ return () => window.removeEventListener(ROWS_STALE_EVENT, onStale);
181
+ }, []);
182
 
183
  useEffect(() => {
184
  if (STREAMLIT_COMPONENT) {
 
292
  window.removeEventListener(DERIVED_CELLS_EVENT, onDerived);
293
  }
294
  };
295
+ // `topic` is a stable module constant per scope (topicForScope — ut topics are memoized
296
+ // per key for exactly this dependency), so this cannot loop. `rowsBump` re-runs it on the
297
+ // rows-stale signal only.
298
+ }, [scope, topic, rowsBump]);
299
 
300
  // Optimistic overlay write: apply the edit immediately (so the grid cell and
301
  // the detail panel reflect it at once), snapshot the pre-edit entry, then
web/src/customer-grid/useVisibleRows.ts CHANGED
@@ -67,6 +67,13 @@ function isDateType(t: Field["type"]): boolean {
67
  return t === "date" || t === "created_time";
68
  }
69
 
 
 
 
 
 
 
 
70
  /**
71
  * Narrowing search: true if any human-readable (text/status) column contains
72
  * `q`. `q` is already lowercased/trimmed by the caller. Numeric columns are
 
67
  return t === "date" || t === "created_time";
68
  }
69
 
70
+ // Wave-18 C5-AUTOFIELD — `automation` is deliberately in NEITHER predicate, so the engine
71
+ // treats it as TEXT everywhere: string compare for filters, lexical sort, group-by on the raw
72
+ // value. That is the correct behaviour for `ok · 2026-08-03 14:10 · 12 posts`, and it is
73
+ // stated rather than left to be inferred because THIS FILE is the reference the Python engine
74
+ // mirrors — the next person porting a predicate needs to know the omission was a decision.
75
+ // It also means `filter_sql.py` needs no change: no new op, no new type branch.
76
+
77
  /**
78
  * Narrowing search: true if any human-readable (text/status) column contains
79
  * `q`. `q` is already lowercased/trimmed by the caller. Numeric columns are
web/src/filter-kit/ops.ts CHANGED
@@ -83,6 +83,12 @@ export function opsForType(t: FieldType): FilterOp[] {
83
  // by the user's own choice list. `eq` reads "is exactly": the whole set is that one choice.
84
  if (t === "multiselect")
85
  return ["contains", "doesNotContain", "eq", "neq", "isEmpty", "isNotEmpty"];
 
 
 
 
 
 
86
  return ["contains", "doesNotContain", "eq", "neq", "isEmpty", "isNotEmpty"]; // text
87
  }
88
 
 
83
  // by the user's own choice list. `eq` reads "is exactly": the whole set is that one choice.
84
  if (t === "multiselect")
85
  return ["contains", "doesNotContain", "eq", "neq", "isEmpty", "isNotEmpty"];
86
+ // Wave-18 C5-AUTOFIELD — `automation` lands HERE, on the text ops, and that is the whole
87
+ // v1 answer. Its cell is one machine-written line (`ok · 2026-08-03 14:10 · 12 posts`), so
88
+ // `contains ok` and `contains error` are the two questions anyone actually asks of the
89
+ // column, and both are already sound over a string. Nothing new enters the operator
90
+ // vocabulary, so the `filter_sql.py` lock-step is untouched — a new op for "state is" would
91
+ // have to be mirrored in the Python engine, and the substring answer is the same answer.
92
  return ["contains", "doesNotContain", "eq", "neq", "isEmpty", "isNotEmpty"]; // text
93
  }
94
 
web/src/index.css CHANGED
The diff for this file is too large to render. See raw diff
 
web/src/settings/SettingsModal.tsx CHANGED
@@ -36,6 +36,15 @@ import type { AdminUser, SettingsPayload } from "./settingsApi";
36
  // `UserPatch` separately. `noUnusedLocals` is what walks a move like this down to
37
  // the last orphaned import, which is the tsconfig earning its keep again.
38
  import { clearAvatar, createUser, getSettings, listUsers, setAvatar } from "./settingsApi";
 
 
 
 
 
 
 
 
 
39
  import { PermsEditor } from "./PermsEditor";
40
  import { moduleSummary, parseEntry, reachableSection } from "./permsModel";
41
 
@@ -615,7 +624,14 @@ export function SettingsModal({
615
  const rail: Array<[SettingsSection, string]> = [
616
  ["account", "Account"],
617
  ["scope", "Your access"],
618
- ...(admin ? ([["users", "Manage users"]] as Array<[SettingsSection, string]>) : []),
 
 
 
 
 
 
 
619
  ];
620
  // A non-admin deep-linked to the users pane lands on Account rather than on
621
  // an empty frame. The server would refuse the data anyway; this just avoids
@@ -655,6 +671,9 @@ export function SettingsModal({
655
  ×
656
  </button>
657
 
 
 
 
658
  {active === "account" ? (
659
  <div className="set-pane">
660
  <h3 className="set-h">Account</h3>
@@ -738,3 +757,315 @@ export function SettingsModal({
738
  </div>
739
  );
740
  }
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
36
  // `UserPatch` separately. `noUnusedLocals` is what walks a move like this down to
37
  // the last orphaned import, which is the tsconfig earning its keep again.
38
  import { clearAvatar, createUser, getSettings, listUsers, setAvatar } from "./settingsApi";
39
+ import {
40
+ addKeychainEntry,
41
+ deleteKeychainEntry,
42
+ getConnectors,
43
+ listKeychain,
44
+ pauseConnector,
45
+ testKeychainEntry,
46
+ } from "./settingsApi";
47
+ import type { ConnectorRow, KeyEntry, UnsyncedInfo } from "./settingsApi";
48
  import { PermsEditor } from "./PermsEditor";
49
  import { moduleSummary, parseEntry, reachableSection } from "./permsModel";
50
 
 
624
  const rail: Array<[SettingsSection, string]> = [
625
  ["account", "Account"],
626
  ["scope", "Your access"],
627
+ ...(admin
628
+ ? ([
629
+ ["users", "Manage users"],
630
+ // Wave 18 (C7): the tenant's credential store + data-source status board.
631
+ ["keychains", "Keychains"],
632
+ ["connectors", "Connectors"],
633
+ ] as Array<[SettingsSection, string]>)
634
+ : []),
635
  ];
636
  // A non-admin deep-linked to the users pane lands on Account rather than on
637
  // an empty frame. The server would refuse the data anyway; this just avoids
 
671
  ×
672
  </button>
673
 
674
+ {active === "keychains" ? <KeychainPane /> : null}
675
+ {active === "connectors" ? <ConnectorsPane /> : null}
676
+
677
  {active === "account" ? (
678
  <div className="set-pane">
679
  <h3 className="set-h">Account</h3>
 
757
  </div>
758
  );
759
  }
760
+
761
+ // ── Wave 18 (C7): the Keychain pane ─────────────────────────────────────────────────────────
762
+ // Secrets go IN and never come back out: rows carry a masked preview, the add form clears on
763
+ // success, and there is no reveal affordance by design (the server has no route for one).
764
+ function KeychainPane() {
765
+ const [entries, setEntries] = useState<KeyEntry[]>([]);
766
+ const [locked, setLocked] = useState(false);
767
+ const [loaded, setLoaded] = useState(false);
768
+ const [err, setErr] = useState("");
769
+ const [form, setForm] = useState<{
770
+ type: "odoo" | "generic";
771
+ label: string;
772
+ fields: Record<string, string>;
773
+ busy: boolean;
774
+ } | null>(null);
775
+ const [testResult, setTestResult] = useState<Record<string, string>>({});
776
+
777
+ const reload = useCallback(async () => {
778
+ const r = await listKeychain();
779
+ if (r.ok) {
780
+ setEntries(r.data.entries);
781
+ setLocked(r.data.locked);
782
+ setErr("");
783
+ } else {
784
+ setErr(r.message);
785
+ }
786
+ setLoaded(true);
787
+ }, []);
788
+ useEffect(() => {
789
+ void reload();
790
+ }, [reload]);
791
+
792
+ const submit = async () => {
793
+ if (!form || form.busy) return;
794
+ setForm({ ...form, busy: true });
795
+ const fields = Object.fromEntries(
796
+ Object.entries(form.fields).filter(([, v]) => v.trim() !== "")
797
+ );
798
+ const r = await addKeychainEntry(form.label, form.type, fields);
799
+ if (r.ok) {
800
+ setForm(null);
801
+ void reload();
802
+ } else {
803
+ setErr(r.message);
804
+ setForm({ ...form, busy: false });
805
+ }
806
+ };
807
+
808
+ const odooFields: Array<[string, string]> = [
809
+ ["url", "Server URL (https://...)"],
810
+ ["db", "Database"],
811
+ ["user", "API user"],
812
+ ["api_key", "API key"],
813
+ ];
814
+
815
+ return (
816
+ <div className="set-pane">
817
+ <h3 className="set-h">Keychains</h3>
818
+ <p className="set-help set-pane-intro">
819
+ Credentials for this workspace's data sources, encrypted at rest. A stored secret is
820
+ never shown again — only its last four characters identify it.
821
+ </p>
822
+ {locked ? (
823
+ <p className="set-error">
824
+ The keychain is locked: the platform's AIOS_KEYCHAIN_KEY is not configured on this
825
+ server. Entries can be listed but nothing can be added or decrypted.
826
+ </p>
827
+ ) : null}
828
+ {err ? <p className="set-error">{err}</p> : null}
829
+ {!loaded ? (
830
+ <p className="set-help">Loading…</p>
831
+ ) : entries.length === 0 ? (
832
+ <p className="set-help">No keys stored yet.</p>
833
+ ) : (
834
+ <div className="set-card">
835
+ {entries.map((e) => (
836
+ <div key={e.id} className="set-kv-row kc-row">
837
+ <div className="kc-main">
838
+ <strong>{e.label}</strong>
839
+ <span className="kc-meta">
840
+ {e.type}
841
+ {e.preview ? " · " + e.preview : ""} · added by {e.createdBy || "?"}
842
+ </span>
843
+ {testResult[e.id] ? (
844
+ <span className="kc-test">{testResult[e.id]}</span>
845
+ ) : null}
846
+ </div>
847
+ <div className="kc-actions">
848
+ <button
849
+ type="button"
850
+ className="set-btn"
851
+ onClick={async () => {
852
+ setTestResult((t) => ({ ...t, [e.id]: "Testing…" }));
853
+ const r = await testKeychainEntry(e.id);
854
+ setTestResult((t) => ({
855
+ ...t,
856
+ [e.id]: r.ok
857
+ ? (r.data.ok ? "OK — " : "Failed — ") + r.data.message
858
+ : r.message,
859
+ }));
860
+ }}
861
+ >
862
+ Test
863
+ </button>
864
+ <button
865
+ type="button"
866
+ className="set-btn kc-danger"
867
+ onClick={async () => {
868
+ const sure = window.confirm(
869
+ 'Delete the key "' + e.label + '"? This cannot be undone.'
870
+ );
871
+ if (!sure) return;
872
+ const r = await deleteKeychainEntry(e.id);
873
+ if (r.ok) void reload();
874
+ else setErr(r.message);
875
+ }}
876
+ >
877
+ Delete
878
+ </button>
879
+ </div>
880
+ </div>
881
+ ))}
882
+ </div>
883
+ )}
884
+ {form ? (
885
+ <div className="set-card kc-form">
886
+ <div className="set-field">
887
+ <label>Type</label>
888
+ <select
889
+ value={form.type}
890
+ onChange={(e) =>
891
+ setForm({
892
+ ...form,
893
+ type: e.target.value as "odoo" | "generic",
894
+ fields: {},
895
+ })
896
+ }
897
+ >
898
+ <option value="odoo">Odoo</option>
899
+ <option value="generic">Generic secret</option>
900
+ </select>
901
+ </div>
902
+ <div className="set-field">
903
+ <label>Label</label>
904
+ <input
905
+ value={form.label}
906
+ maxLength={80}
907
+ placeholder={form.type === "odoo" ? "Production Odoo" : "e.g. Shipping API"}
908
+ onChange={(e) => setForm({ ...form, label: e.target.value })}
909
+ />
910
+ </div>
911
+ {(form.type === "odoo"
912
+ ? odooFields
913
+ : ([
914
+ ["name", "Name"],
915
+ ["value", "Secret value"],
916
+ ] as Array<[string, string]>)
917
+ ).map(([k, label]) => (
918
+ <div className="set-field" key={k}>
919
+ <label>{label}</label>
920
+ <input
921
+ value={form.fields[k] ?? ""}
922
+ type={k === "api_key" || k === "value" ? "password" : "text"}
923
+ onChange={(e) =>
924
+ setForm({ ...form, fields: { ...form.fields, [k]: e.target.value } })
925
+ }
926
+ />
927
+ </div>
928
+ ))}
929
+ <div className="set-actions">
930
+ <button type="button" className="set-btn" onClick={() => setForm(null)}>
931
+ Cancel
932
+ </button>
933
+ <button
934
+ type="button"
935
+ className="set-btn set-btn-primary"
936
+ disabled={form.busy || !form.label.trim() || locked}
937
+ onClick={() => void submit()}
938
+ >
939
+ {form.busy ? "Saving…" : "Save key"}
940
+ </button>
941
+ </div>
942
+ </div>
943
+ ) : (
944
+ <button
945
+ type="button"
946
+ className="set-btn set-btn-primary"
947
+ disabled={locked}
948
+ onClick={() => setForm({ type: "odoo", label: "", fields: {}, busy: false })}
949
+ >
950
+ Add a key
951
+ </button>
952
+ )}
953
+ </div>
954
+ );
955
+ }
956
+
957
+ // ── Wave 18 (C7): the Connectors pane — sources + R3's Unsynced-records guardrail ───────────
958
+ function ConnectorsPane() {
959
+ const [rows, setRows] = useState<ConnectorRow[]>([]);
960
+ const [note, setNote] = useState("");
961
+ const [unsynced, setUnsynced] = useState<UnsyncedInfo | null>(null);
962
+ const [showOrphans, setShowOrphans] = useState(false);
963
+ const [loaded, setLoaded] = useState(false);
964
+ const [err, setErr] = useState("");
965
+
966
+ const reload = useCallback(async () => {
967
+ const r = await getConnectors();
968
+ if (r.ok) {
969
+ setRows(r.data.connectors);
970
+ setNote(r.data.pausedNote || "");
971
+ setUnsynced(r.data.unsynced ?? null);
972
+ setErr("");
973
+ } else {
974
+ setErr(r.message);
975
+ }
976
+ setLoaded(true);
977
+ }, []);
978
+ useEffect(() => {
979
+ void reload();
980
+ }, [reload]);
981
+
982
+ return (
983
+ <div className="set-pane">
984
+ <h3 className="set-h">Connectors</h3>
985
+ <p className="set-help set-pane-intro">
986
+ Where this workspace's data comes from. Add credentials under Keychains; sources appear
987
+ here with their status.
988
+ </p>
989
+ {err ? <p className="set-error">{err}</p> : null}
990
+ {!loaded ? (
991
+ <p className="set-help">Loading…</p>
992
+ ) : rows.length === 0 ? (
993
+ <p className="set-help">
994
+ No connectors yet. Store a key under Keychains to register a source — nothing syncs
995
+ until a later step turns it on.
996
+ </p>
997
+ ) : (
998
+ <div className="set-card">
999
+ {rows.map((r) => (
1000
+ <div key={r.key} className="set-kv-row kc-row">
1001
+ <div className="kc-main">
1002
+ <strong>{r.label}</strong>
1003
+ <span className="kc-meta">
1004
+ {r.type} · {r.source === "env" ? "environment credentials" : "keychain"}
1005
+ {r.paused ? " · paused" : " · active"}
1006
+ </span>
1007
+ </div>
1008
+ <div className="kc-actions">
1009
+ <button
1010
+ type="button"
1011
+ className="set-btn"
1012
+ onClick={async () => {
1013
+ const res = await pauseConnector(r.key, !r.paused);
1014
+ if (res.ok) void reload();
1015
+ else setErr(res.message);
1016
+ }}
1017
+ >
1018
+ {r.paused ? "Resume" : "Pause"}
1019
+ </button>
1020
+ </div>
1021
+ </div>
1022
+ ))}
1023
+ </div>
1024
+ )}
1025
+ {note ? <p className="set-help kc-note">{note}</p> : null}
1026
+ {unsynced ? (
1027
+ <div className="set-card kc-unsynced">
1028
+ <strong>Unsynced records</strong>
1029
+ {unsynced.known ? (
1030
+ unsynced.count === 0 ? (
1031
+ <p className="set-help">
1032
+ None — every record holding notes or custom-field data matches a row the
1033
+ connector currently serves.
1034
+ </p>
1035
+ ) : (
1036
+ <>
1037
+ <p className="set-help">
1038
+ {unsynced.count} record{unsynced.count === 1 ? "" : "s"} carry saved data
1039
+ (notes, custom fields) but no longer match a synced row. Nothing is deleted —
1040
+ they reattach by ID if the source serves them again.
1041
+ </p>
1042
+ <button
1043
+ type="button"
1044
+ className="set-btn"
1045
+ onClick={() => setShowOrphans((s) => !s)}
1046
+ >
1047
+ {showOrphans ? "Hide" : "Show " + String(unsynced.shown ?? unsynced.rows.length)}
1048
+ </button>
1049
+ {showOrphans ? (
1050
+ <ul className="kc-orphans">
1051
+ {unsynced.rows.map((o) => (
1052
+ <li key={String(o.pid)}>
1053
+ #{o.pid} — {o.fields} saved field{o.fields === 1 ? "" : "s"}
1054
+ {o.hint ? " · " + o.hint : ""}
1055
+ </li>
1056
+ ))}
1057
+ </ul>
1058
+ ) : null}
1059
+ </>
1060
+ )
1061
+ ) : (
1062
+ <p className="set-help">
1063
+ Count unavailable right now{unsynced.note ? " (" + unsynced.note + ")" : ""} — the
1064
+ pool could not be consulted; nothing was assumed.
1065
+ </p>
1066
+ )}
1067
+ </div>
1068
+ ) : null}
1069
+ </div>
1070
+ );
1071
+ }
web/src/settings/permsModel.ts CHANGED
@@ -37,7 +37,13 @@ import type { Field, FieldType, FilterTree } from "../customer-grid/types";
37
  /** The modal's rail entries. Declared HERE, beside the rule that gates them, so
38
  * the component and the gate read one definition of the vocabulary rather than
39
  * two that can drift by one member. `SettingsModal` re-exports it. */
40
- export type SettingsSection = "account" | "scope" | "users";
 
 
 
 
 
 
41
 
42
  /**
43
  * A non-admin never LANDS on the users pane — and therefore never reaches the
@@ -53,7 +59,10 @@ export type SettingsSection = "account" | "scope" | "users";
53
  * someone who does not know it is load-bearing.
54
  */
55
  export function reachableSection(section: SettingsSection, admin: boolean): SettingsSection {
56
- return section === "users" && !admin ? "account" : section;
 
 
 
57
  }
58
 
59
  // --- the wire (C-PERM) ------------------------------------------------------
@@ -111,6 +120,7 @@ const FIELD_TYPE_TABLE: Record<FieldType, true> = {
111
  text: true, status: true, currency: true, int: true, date: true, pct: true,
112
  select: true, user: true, multiselect: true, checkbox: true, phone: true,
113
  email: true, url: true, rating: true, created_time: true, formula: true,
 
114
  };
115
 
116
  export const KNOWN_FIELD_TYPES: ReadonlySet<string> = new Set(Object.keys(FIELD_TYPE_TABLE));
 
37
  /** The modal's rail entries. Declared HERE, beside the rule that gates them, so
38
  * the component and the gate read one definition of the vocabulary rather than
39
  * two that can drift by one member. `SettingsModal` re-exports it. */
40
+ export type SettingsSection =
41
+ | "account"
42
+ | "scope"
43
+ | "users"
44
+ // Wave 18 (C7): the tenant's credential store and its data-source status board.
45
+ | "keychains"
46
+ | "connectors";
47
 
48
  /**
49
  * A non-admin never LANDS on the users pane — and therefore never reaches the
 
59
  * someone who does not know it is load-bearing.
60
  */
61
  export function reachableSection(section: SettingsSection, admin: boolean): SettingsSection {
62
+ // Wave 18 (C7): keychains + connectors are admin rooms for the same reason users is —
63
+ // every control inside them would 403 a member.
64
+ const adminOnly = section === "users" || section === "keychains" || section === "connectors";
65
+ return adminOnly && !admin ? "account" : section;
66
  }
67
 
68
  // --- the wire (C-PERM) ------------------------------------------------------
 
120
  text: true, status: true, currency: true, int: true, date: true, pct: true,
121
  select: true, user: true, multiselect: true, checkbox: true, phone: true,
122
  email: true, url: true, rating: true, created_time: true, formula: true,
123
+ automation: true,
124
  };
125
 
126
  export const KNOWN_FIELD_TYPES: ReadonlySet<string> = new Set(Object.keys(FIELD_TYPE_TABLE));
web/src/settings/settingsApi.ts CHANGED
@@ -192,3 +192,76 @@ export function setAvatar(dataUrl: string): Promise<ApiResult<{ user: unknown }>
192
  export function clearAvatar(): Promise<ApiResult<{ user: unknown }>> {
193
  return call<{ user: unknown }>("/auth/me/avatar", { method: "DELETE" });
194
  }
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
192
  export function clearAvatar(): Promise<ApiResult<{ user: unknown }>> {
193
  return call<{ user: unknown }>("/auth/me/avatar", { method: "DELETE" });
194
  }
195
+
196
+ // ── Wave 18 (C7): Keychains + Connectors ────────────────────────────────────────────────────
197
+ export interface KeyEntry {
198
+ id: string;
199
+ label: string;
200
+ type: string;
201
+ preview: string;
202
+ created: string;
203
+ createdBy: string;
204
+ }
205
+
206
+ export interface ConnectorRow {
207
+ key: string;
208
+ label: string;
209
+ type: string;
210
+ source: "env" | "keychain";
211
+ preview?: string;
212
+ paused: boolean;
213
+ }
214
+
215
+ export interface UnsyncedInfo {
216
+ known: boolean;
217
+ count: number | null;
218
+ rows: Array<{ pid: number | string; fields: number; hint: string }>;
219
+ shown?: number;
220
+ note?: string;
221
+ }
222
+
223
+ export function listKeychain(): Promise<ApiResult<{ entries: KeyEntry[]; locked: boolean }>> {
224
+ return call("/admin/keychain");
225
+ }
226
+
227
+ export function addKeychainEntry(
228
+ label: string,
229
+ type: string,
230
+ fields: Record<string, string>
231
+ ): Promise<ApiResult<{ entry: KeyEntry }>> {
232
+ return call("/admin/keychain", {
233
+ method: "POST",
234
+ body: JSON.stringify({ label, type, fields }),
235
+ });
236
+ }
237
+
238
+ export function deleteKeychainEntry(id: string): Promise<ApiResult<{ ok: boolean }>> {
239
+ return call(`/admin/keychain/${encodeURIComponent(id)}`, { method: "DELETE" });
240
+ }
241
+
242
+ export function testKeychainEntry(
243
+ id: string
244
+ ): Promise<ApiResult<{ ok: boolean; message: string }>> {
245
+ return call(`/admin/keychain/${encodeURIComponent(id)}/test`, { method: "POST" });
246
+ }
247
+
248
+ export function getConnectors(): Promise<
249
+ ApiResult<{
250
+ connectors: ConnectorRow[];
251
+ locked: boolean;
252
+ pausedNote: string;
253
+ unsynced?: UnsyncedInfo;
254
+ }>
255
+ > {
256
+ return call("/admin/connectors");
257
+ }
258
+
259
+ export function pauseConnector(
260
+ key: string,
261
+ paused: boolean
262
+ ): Promise<ApiResult<{ key: string; paused: boolean }>> {
263
+ return call(`/admin/connectors/${encodeURIComponent(key)}/pause`, {
264
+ method: "POST",
265
+ body: JSON.stringify({ paused }),
266
+ });
267
+ }
web/src/shell/Shell.tsx CHANGED
@@ -32,10 +32,11 @@
32
 
33
  import { useCallback, useEffect, useRef, useState } from "react";
34
  import type { MouseEvent as ReactMouseEvent } from "react";
 
35
  import CustomerGrid from "../customer-grid/CustomerGrid";
36
  import { clearCustomersCache } from "../customer-grid/apiBridge";
37
  import { OverlayProvider } from "../customer-grid/OverlaySurface";
38
- import { DATA_ERROR_EVENT, NAV_MINIMIZE_EVENT, TOAST_EVENT, UNAUTHORIZED_EVENT } from "../apiContract";
39
  import { PageSurface } from "../pages/PageSurface";
40
  import { SettingsModal } from "../settings/SettingsModal";
41
  import type { SettingsSection } from "../settings/SettingsModal";
@@ -72,7 +73,7 @@ type Session =
72
  type Nav =
73
  | { phase: "idle" }
74
  | { phase: "loading" }
75
- | { phase: "ready"; entries: NavEntry[]; utility: NavPage[] }
76
  | { phase: "error" };
77
 
78
  const NO_UTILITY: NavPage[] = [];
@@ -316,6 +317,9 @@ function AccountMenu({
316
  export default function Shell() {
317
  const [session, setSession] = useState<Session>({ phase: "checking" });
318
  const [nav, setNav] = useState<Nav>({ phase: "idle" });
 
 
 
319
  const [dataError, setDataError] = useState("");
320
  const [toast, setToast] = useState("");
321
  // `null` = closed. The SECTION is the open state, so "Settings" and
@@ -421,7 +425,8 @@ export default function Shell() {
421
  // so since X6; the Analyst slot and the account menu place them where
422
  // the host does (`core.perms.nav_pages`' own contract).
423
  const { main, utility } = splitChrome(r.pages);
424
- setNav({ phase: "ready", entries: shapeNav(main, APP_BASE), utility });
 
425
  }
426
  // 401 is not "the nav is broken", it is "the session died under us" —
427
  // conflating them would leave a signed-out user staring at a frame.
@@ -431,11 +436,14 @@ export default function Shell() {
431
  return () => {
432
  dead = true;
433
  };
434
- }, [who]);
435
 
436
  // NO_ENTRIES, not a fresh `[]`: a new array identity on every render would
437
  // re-run the hash effect below on every render for no reason.
438
  const entries = nav.phase === "ready" ? nav.entries : NO_ENTRIES;
 
 
 
439
 
440
  // ── C-SCHEMA (2026-08-02): per-user folders over the database list + the
441
  // schema drawer. Placement is cosmetic per-user state — the server-filtered
@@ -547,6 +555,91 @@ export default function Shell() {
547
  void logout().then(() => setSession({ phase: "anon" }));
548
  }, []);
549
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
550
  if (session.phase === "checking") {
551
  // Deliberately wordless. A "Loading…" line here would flash for one round
552
  // trip on every load, and the mark says everything the moment needs.
@@ -611,21 +704,22 @@ export default function Shell() {
611
  <nav className="shell-nav">
612
  {/* The Analyst slot, above the database list — the host's own IA
613
  (app.py:8166 pins "AI assistant" over the nav tree, same label,
614
- same sparkle). A hand-off until the Analyst is ported, which the
615
- owner has deliberately not prioritised. */}
 
 
 
616
  {analyst ? (
617
- <a
 
618
  className="shell-nav-item shell-nav-assist"
619
- href={appLink(APP_BASE, "analyst")}
620
- target="_blank"
621
- rel="noreferrer"
622
  onMouseEnter={navCollapsed ? tipEnter("AI assistant") : undefined}
623
  onMouseLeave={navCollapsed ? tipLeave : undefined}
624
  >
625
  <SparkIcon />
626
  <span className="shell-nav-label">AI assistant</span>
627
- <ExtIcon />
628
- </a>
629
  ) : null}
630
 
631
  <div
@@ -795,9 +889,34 @@ export default function Shell() {
795
  {nav.phase === "error" ? (
796
  <div className="shell-nav-note">Navigation unavailable. Reload to retry.</div>
797
  ) : null}
798
- {nav.phase === "ready" && entries.length === 0 ? (
 
 
 
 
 
 
799
  <div className="shell-nav-note">No surfaces are available to this account.</div>
800
  ) : null}
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
801
  </nav>
802
 
803
  <div
@@ -856,20 +975,47 @@ export default function Shell() {
856
  // varies: wave 3's Collections and Procurement come through this same
857
  // branch, which is the whole return on the envelope contract.
858
  <PageSurface key={active.key} pageKey={active.key} label={active.label} />
 
 
 
 
859
  ) : active?.kind === "native" ? (
860
  // The native grid surfaces — ONE component, topic decided by the route. Wave 16:
861
  // the `#/cohort` route left with the cohort registry row (cohorts are LOCKED VIEWS
862
  // in the Customer rail, C-LOCK; `scope=cohort` stays a working server surface for
863
  // stored artifacts), and `#/product_data` arrived (C-TOPIC) — the same tree over
864
  // the SKU catalogue. The `key` forces a remount on a route change so one topic's
865
- // view state never bleeds into the other's.
866
- <div className="shell-grid-host">
867
- <OverlayProvider>
868
- <CustomerGrid
869
- key={active.key}
870
- scope={active.key === "product_data" ? "product" : "customer"}
871
- />
872
- </OverlayProvider>
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
873
  </div>
874
  ) : active ? (
875
  <StranglerPage entry={active} />
@@ -882,14 +1028,31 @@ export default function Shell() {
882
  // new user, for as long as the round trip took. The sentence is
883
  // RIGHT for a nav that came back empty and WRONG for one that has not
884
  // come back, and those two were indistinguishable. They are not now.
885
- <div className="shell-placeholder">
886
- <h1>Nothing to show</h1>
887
- <p>
888
- {nav.phase === "error"
889
- ? "The navigation service did not answer. Reload to retry."
890
- : "This account has access to no surfaces yet."}
891
- </p>
892
- </div>
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
893
  ) : (
894
  // Still asking (`loading`, or the `idle` beat before the effect runs).
895
  <div className="shell-loading">
@@ -901,6 +1064,85 @@ export default function Shell() {
901
  {/* The server's own confirmation of a write (X2's `toast`). In standalone
902
  it is the ONLY feedback for the events whose effect the payload does
903
  not yet echo back — cohort membership, list adds, folder moves. */}
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
904
  {settings ? (
905
  <SettingsModal
906
  user={session.user}
 
32
 
33
  import { useCallback, useEffect, useRef, useState } from "react";
34
  import type { MouseEvent as ReactMouseEvent } from "react";
35
+ import AutomationSurface from "../automation/AutomationSurface";
36
  import CustomerGrid from "../customer-grid/CustomerGrid";
37
  import { clearCustomersCache } from "../customer-grid/apiBridge";
38
  import { OverlayProvider } from "../customer-grid/OverlaySurface";
39
+ import { API_V1, CREDENTIALS, DATA_ERROR_EVENT, NAV_MINIMIZE_EVENT, ROWS_STALE_EVENT, TOAST_EVENT, UNAUTHORIZED_EVENT } from "../apiContract";
40
  import { PageSurface } from "../pages/PageSurface";
41
  import { SettingsModal } from "../settings/SettingsModal";
42
  import type { SettingsSection } from "../settings/SettingsModal";
 
73
  type Nav =
74
  | { phase: "idle" }
75
  | { phase: "loading" }
76
+ | { phase: "ready"; entries: NavEntry[]; utility: NavPage[]; empty?: string }
77
  | { phase: "error" };
78
 
79
  const NO_UTILITY: NavPage[] = [];
 
317
  export default function Shell() {
318
  const [session, setSession] = useState<Session>({ phase: "checking" });
319
  const [nav, setNav] = useState<Nav>({ phase: "idle" });
320
+ // Wave 18 (C3-UT): bumped after "+ New database" so the rail refetches and the new ut_ key
321
+ // appears without a reload. Nothing else may bump it — the nav stays server-driven.
322
+ const [navEpoch, setNavEpoch] = useState(0);
323
  const [dataError, setDataError] = useState("");
324
  const [toast, setToast] = useState("");
325
  // `null` = closed. The SECTION is the open state, so "Settings" and
 
425
  // so since X6; the Analyst slot and the account menu place them where
426
  // the host does (`core.perms.nav_pages`' own contract).
427
  const { main, utility } = splitChrome(r.pages);
428
+ setNav({ phase: "ready", entries: shapeNav(main, APP_BASE), utility,
429
+ ...(r.empty ? { empty: r.empty } : {}) });
430
  }
431
  // 401 is not "the nav is broken", it is "the session died under us" —
432
  // conflating them would leave a signed-out user staring at a frame.
 
436
  return () => {
437
  dead = true;
438
  };
439
+ }, [who, navEpoch]);
440
 
441
  // NO_ENTRIES, not a fresh `[]`: a new array identity on every render would
442
  // re-run the hash effect below on every render for no reason.
443
  const entries = nav.phase === "ready" ? nav.entries : NO_ENTRIES;
444
+ // Wave 18: a fresh TENANT (server says `no_databases`) is welcomed, not warned.
445
+ const tenantEmptyState =
446
+ nav.phase === "ready" && entries.length === 0 && nav.empty === "no_databases";
447
 
448
  // ── C-SCHEMA (2026-08-02): per-user folders over the database list + the
449
  // schema drawer. Placement is cosmetic per-user state — the server-filtered
 
555
  void logout().then(() => setSession({ phase: "anon" }));
556
  }, []);
557
 
558
+ // ── Wave 18 hooks — ⛔ ABOVE the early returns, or they run conditionally. The first
559
+ // version of this block sat below the `anon` return and blanked the whole app with React
560
+ // error #310 on the first authed render (found by SESSION B's probe, booked in the wave
561
+ // doc): the anon render mounted N hooks, the authed render mounted N+5, and React refuses
562
+ // a component whose hook count changes between renders. Every hook in this component stays
563
+ // above `if (session.phase …) return`, whatever lands here next wave.
564
+ // (owner item 3): the assistant's under-construction note.
565
+ const [assistOpen, setAssistOpen] = useState(false);
566
+ // ⛔ ESCAPE CLOSES A MODAL, or its scrim becomes a trap. The assistant note shipped with only
567
+ // a scrim-click and a Close button, and the first close-out probe to press Escape found every
568
+ // subsequent click swallowed by an invisible full-screen div — the
569
+ // painted-but-unclickable failure [[ui-invisible-to-assertions]] names, in reverse. The
570
+ // new-database dialog had Escape only while its INPUT held focus, which is not the same thing.
571
+ useEffect(() => {
572
+ if (!assistOpen && !newDbOpenRef.current) return;
573
+ const onKey = (e: KeyboardEvent) => {
574
+ if (e.key !== "Escape") return;
575
+ setAssistOpen(false);
576
+ setNewDb((cur) => (cur && cur.busy ? cur : null));
577
+ };
578
+ window.addEventListener("keydown", onKey);
579
+ return () => window.removeEventListener("keydown", onKey);
580
+ });
581
+ // (C3-UT): "+ New database" + the user-table Add-record bar.
582
+ const [newDb, setNewDb] = useState<null | { name: string; busy: boolean; err: string }>(null);
583
+ const newDbOpenRef = useRef(false);
584
+ newDbOpenRef.current = newDb !== null;
585
+ const [utBusy, setUtBusy] = useState(false);
586
+ const createDb = useCallback(async () => {
587
+ setNewDb((cur) => {
588
+ if (!cur || cur.busy || !cur.name.trim()) return cur;
589
+ const name = cur.name;
590
+ void (async () => {
591
+ try {
592
+ const res = await fetch(`${API_V1}/tables`, {
593
+ method: "POST",
594
+ credentials: CREDENTIALS,
595
+ headers: { "Content-Type": "application/json" },
596
+ body: JSON.stringify({ label: name.trim() }),
597
+ });
598
+ const body = (await res.json().catch(() => null)) as
599
+ | { key?: string; error?: { message?: string } }
600
+ | null;
601
+ if (!res.ok || !body?.key) {
602
+ setNewDb({ name, busy: false,
603
+ err: body?.error?.message || `The server answered ${res.status}.` });
604
+ return;
605
+ }
606
+ setNewDb(null);
607
+ setNavEpoch((e) => e + 1);
608
+ window.location.hash = `#/${body.key}`;
609
+ } catch {
610
+ setNewDb({ name, busy: false, err: "Cannot reach the server." });
611
+ }
612
+ })();
613
+ return { ...cur, busy: true, err: "" };
614
+ });
615
+ }, []);
616
+ const addRecord = useCallback(async (key: string) => {
617
+ setUtBusy(true);
618
+ try {
619
+ const res = await fetch(`${API_V1}/tables/${encodeURIComponent(key)}/rows`, {
620
+ method: "POST",
621
+ credentials: CREDENTIALS,
622
+ headers: { "Content-Type": "application/json" },
623
+ body: JSON.stringify({ values: {} }),
624
+ });
625
+ if (res.ok) {
626
+ // The grid's rows cache would happily serve the pre-add pull for 5 minutes —
627
+ // clear it, then tell the current topic to refetch (the C3-UT amendment's event).
628
+ clearCustomersCache();
629
+ window.dispatchEvent(new Event(ROWS_STALE_EVENT));
630
+ } else {
631
+ const body = (await res.json().catch(() => null)) as
632
+ | { error?: { message?: string } }
633
+ | null;
634
+ setToast(body?.error?.message || "The row was not added.");
635
+ }
636
+ } catch {
637
+ setToast("Cannot reach the server.");
638
+ } finally {
639
+ setUtBusy(false);
640
+ }
641
+ }, []);
642
+
643
  if (session.phase === "checking") {
644
  // Deliberately wordless. A "Loading…" line here would flash for one round
645
  // trip on every load, and the mark says everything the moment needs.
 
704
  <nav className="shell-nav">
705
  {/* The Analyst slot, above the database list — the host's own IA
706
  (app.py:8166 pins "AI assistant" over the nav tree, same label,
707
+ same sparkle). Wave 18 (owner item 3): the hand-off into Streamlit
708
+ is PAUSED — the click opens an in-app "under construction" note
709
+ instead. The nav item, the registry row and the Streamlit Analyst
710
+ page all stay (kept-not-ported, never delete); only the door is
711
+ boarded until a later wave expands it. */}
712
  {analyst ? (
713
+ <button
714
+ type="button"
715
  className="shell-nav-item shell-nav-assist"
716
+ onClick={() => setAssistOpen(true)}
 
 
717
  onMouseEnter={navCollapsed ? tipEnter("AI assistant") : undefined}
718
  onMouseLeave={navCollapsed ? tipLeave : undefined}
719
  >
720
  <SparkIcon />
721
  <span className="shell-nav-label">AI assistant</span>
722
+ </button>
 
723
  ) : null}
724
 
725
  <div
 
889
  {nav.phase === "error" ? (
890
  <div className="shell-nav-note">Navigation unavailable. Reload to retry.</div>
891
  ) : null}
892
+ {/* ⚠ Wave 18: this line is for a MISCONFIGURED ACCOUNT — one whose grants give it
893
+ nothing — and it must not fire for a freshly provisioned TENANT, which has no
894
+ modules by design and is being welcomed by the hero in the main pane. Both at
895
+ once said "something is wrong here" and "welcome, start here" in one screen
896
+ (caught in the close-out visual pass, not by any gate). The rail's own "New
897
+ database" button below is the honest affordance in the tenant case. */}
898
+ {nav.phase === "ready" && entries.length === 0 && !tenantEmptyState ? (
899
  <div className="shell-nav-note">No surfaces are available to this account.</div>
900
  ) : null}
901
+ {/* Wave 18 (C3-UT / R4): a database the USER creates — the second branch of the
902
+ owner's "+ New" flow (connectors are Settings' half). Collapsed rail expands
903
+ first: a modal opened from a 56px strip reads as coming from nowhere. */}
904
+ <button
905
+ type="button"
906
+ className="shell-nav-item shell-nav-newdb"
907
+ onClick={() => {
908
+ if (navCollapsed) setNavCollapsed(false);
909
+ setNewDb({ name: "", busy: false, err: "" });
910
+ }}
911
+ onMouseEnter={navCollapsed ? tipEnter("New database") : undefined}
912
+ onMouseLeave={navCollapsed ? tipLeave : undefined}
913
+ >
914
+ <svg width="16" height="16" viewBox="0 0 16 16" aria-hidden="true">
915
+ <path d="M8 3.4v9.2M3.4 8h9.2" stroke="currentColor" strokeWidth="1.5"
916
+ strokeLinecap="round" fill="none" />
917
+ </svg>
918
+ <span className="shell-nav-label">New database</span>
919
+ </button>
920
  </nav>
921
 
922
  <div
 
975
  // varies: wave 3's Collections and Procurement come through this same
976
  // branch, which is the whole return on the envelope contract.
977
  <PageSurface key={active.key} pageKey={active.key} label={active.label} />
978
+ ) : active?.kind === "native" && active.key === "automation" ? (
979
+ // Wave 18 (C-AUTONAV): the Automation surface — its OWN secondary rail + editor,
980
+ // props-free by contract (it fetches /api/v1/automations itself). SESSION D's tree.
981
+ <AutomationSurface />
982
  ) : active?.kind === "native" ? (
983
  // The native grid surfaces — ONE component, topic decided by the route. Wave 16:
984
  // the `#/cohort` route left with the cohort registry row (cohorts are LOCKED VIEWS
985
  // in the Customer rail, C-LOCK; `scope=cohort` stays a working server surface for
986
  // stored artifacts), and `#/product_data` arrived (C-TOPIC) — the same tree over
987
  // the SKU catalogue. The `key` forces a remount on a route change so one topic's
988
+ // view state never bleeds into the other's. Wave 18 (C3-UT): a `ut_` route is a
989
+ // USER TABLE through the same tree — scope IS the key — with the shell-owned
990
+ // Add-record bar above it (the doc's amendment: zero CustomerGrid edits).
991
+ <div className={active.key.startsWith("ut_") ? "shell-ut-frame" : "shell-grid-host"}>
992
+ {active.key.startsWith("ut_") ? (
993
+ <div className="shell-ut-bar">
994
+ <span className="shell-ut-name">{active.label}</span>
995
+ <button
996
+ type="button"
997
+ className="shell-ut-add"
998
+ onClick={() => void addRecord(active.key)}
999
+ disabled={utBusy}
1000
+ >
1001
+ {utBusy ? "Adding…" : "Add record"}
1002
+ </button>
1003
+ </div>
1004
+ ) : null}
1005
+ <div className="shell-grid-host">
1006
+ <OverlayProvider>
1007
+ <CustomerGrid
1008
+ key={active.key}
1009
+ scope={
1010
+ active.key === "product_data"
1011
+ ? "product"
1012
+ : active.key.startsWith("ut_")
1013
+ ? (active.key as `ut_${string}`)
1014
+ : "customer"
1015
+ }
1016
+ />
1017
+ </OverlayProvider>
1018
+ </div>
1019
  </div>
1020
  ) : active ? (
1021
  <StranglerPage entry={active} />
 
1028
  // new user, for as long as the round trip took. The sentence is
1029
  // RIGHT for a nav that came back empty and WRONG for one that has not
1030
  // come back, and those two were indistinguishable. They are not now.
1031
+ nav.phase === "ready" ? (
1032
+ // Wave 18 (C1-TENANT, R4): a READY nav with zero entries is a freshly provisioned
1033
+ // tenant (the server says so explicitly — a misconfigured account 403s and lands
1034
+ // on the error branch instead). The honest next step is the empty state's whole
1035
+ // content: create the first database.
1036
+ <div className="shell-placeholder shell-tenant-hero">
1037
+ <h1>Welcome</h1>
1038
+ <p>
1039
+ This workspace has no databases yet. Create your first one below, or store a
1040
+ data source's keys under Profile → Keychains to connect one later.
1041
+ </p>
1042
+ <button
1043
+ className="login-submit shell-hero-create"
1044
+ type="button"
1045
+ onClick={() => setNewDb({ name: "", busy: false, err: "" })}
1046
+ >
1047
+ Create your first database
1048
+ </button>
1049
+ </div>
1050
+ ) : (
1051
+ <div className="shell-placeholder">
1052
+ <h1>Nothing to show</h1>
1053
+ <p>The navigation service did not answer. Reload to retry.</p>
1054
+ </div>
1055
+ )
1056
  ) : (
1057
  // Still asking (`loading`, or the `idle` beat before the effect runs).
1058
  <div className="shell-loading">
 
1064
  {/* The server's own confirmation of a write (X2's `toast`). In standalone
1065
  it is the ONLY feedback for the events whose effect the payload does
1066
  not yet echo back — cohort membership, list adds, folder moves. */}
1067
+ {/* Wave 18 (owner item 3): the assistant's boarded door. */}
1068
+ {assistOpen ? (
1069
+ <div className="shell-newdb-scrim" onClick={() => setAssistOpen(false)}>
1070
+ <div
1071
+ className="shell-newdb shell-assist-note"
1072
+ role="dialog"
1073
+ aria-label="AI assistant"
1074
+ onClick={(e) => e.stopPropagation()}
1075
+ >
1076
+ <h2>
1077
+ <SparkIcon /> AI assistant
1078
+ </h2>
1079
+ <p className="shell-newdb-sub">
1080
+ Under construction. The assistant is being trained on this workspace's data
1081
+ model and will open here in a later release.
1082
+ </p>
1083
+ <div className="shell-newdb-actions">
1084
+ <button type="button" className="login-submit" onClick={() => setAssistOpen(false)}>
1085
+ Close
1086
+ </button>
1087
+ </div>
1088
+ </div>
1089
+ </div>
1090
+ ) : null}
1091
+
1092
+ {/* Wave 18 (C3-UT): the New-database dialog. Scrim click cancels unless mid-create. */}
1093
+ {newDb ? (
1094
+ <div
1095
+ className="shell-newdb-scrim"
1096
+ onClick={() => {
1097
+ if (!newDb.busy) setNewDb(null);
1098
+ }}
1099
+ >
1100
+ <div
1101
+ className="shell-newdb"
1102
+ role="dialog"
1103
+ aria-label="New database"
1104
+ onClick={(e) => e.stopPropagation()}
1105
+ >
1106
+ <h2>New database</h2>
1107
+ <p className="shell-newdb-sub">
1108
+ A blank database in this workspace. Add fields and records once it opens —
1109
+ nothing here connects to a source.
1110
+ </p>
1111
+ <input
1112
+ autoFocus
1113
+ className="shell-newdb-input"
1114
+ placeholder="Database name"
1115
+ value={newDb.name}
1116
+ maxLength={60}
1117
+ disabled={newDb.busy}
1118
+ onChange={(e) => setNewDb({ ...newDb, name: e.target.value })}
1119
+ onKeyDown={(e) => {
1120
+ if (e.key === "Enter") void createDb();
1121
+ if (e.key === "Escape" && !newDb.busy) setNewDb(null);
1122
+ }}
1123
+ />
1124
+ {newDb.err ? <p className="shell-newdb-err">{newDb.err}</p> : null}
1125
+ <div className="shell-newdb-actions">
1126
+ <button
1127
+ type="button"
1128
+ onClick={() => setNewDb(null)}
1129
+ disabled={newDb.busy}
1130
+ >
1131
+ Cancel
1132
+ </button>
1133
+ <button
1134
+ type="button"
1135
+ className="login-submit"
1136
+ onClick={() => void createDb()}
1137
+ disabled={newDb.busy || !newDb.name.trim()}
1138
+ >
1139
+ {newDb.busy ? "Creating…" : "Create database"}
1140
+ </button>
1141
+ </div>
1142
+ </div>
1143
+ </div>
1144
+ ) : null}
1145
+
1146
  {settings ? (
1147
  <SettingsModal
1148
  user={session.user}
web/src/shell/nav.ts CHANGED
@@ -88,7 +88,14 @@ export function splitChrome(pages: NavPage[]): { main: NavPage[]; utility: NavPa
88
  return { main, utility };
89
  }
90
 
91
- export type NavResult = { ok: true; pages: NavPage[] } | { ok: false; status: number };
 
 
 
 
 
 
 
92
 
93
  /**
94
  * X2 `GET /api/v1/nav`. `status` rides the failure so the caller can tell the
@@ -101,7 +108,9 @@ export async function fetchNav(): Promise<NavResult> {
101
  try {
102
  const res = await fetch(`${API_V1}/nav`, { credentials: CREDENTIALS });
103
  if (!res.ok) return { ok: false, status: res.status };
104
- return { ok: true, pages: parsePages(await res.json()) };
 
 
105
  } catch {
106
  return { ok: false, status: 0 };
107
  }
@@ -348,7 +357,9 @@ export const NATIVE_KEYS: ReadonlySet<string> = new Set<string>(
348
  // `workspace.cohortMode`, the events — reachable from the Customer surface.
349
  // `product_data` joined wave 16 (C-TOPIC, owner items 9+10): the SAME grid tree over the
350
  // SKU catalogue — Shell maps it to `scope="product"` and everything else is the topic's.
351
- ["customer_data", "product_data"]);
 
 
352
 
353
  /** The native keys the Y1 page envelope drives (as opposed to `customer_data`,
354
  * which is its own component tree). Kept beside NATIVE_KEYS so the two cannot
@@ -389,7 +400,11 @@ export function shapeNav(pages: NavPage[], appBase: string): NavEntry[] {
389
  }
390
 
391
  const entry = (p: NavPage, depth: number): NavEntry => {
392
- const isNative = NATIVE_KEYS.has(p.key);
 
 
 
 
393
  // A native surface stays native even with children — the shell can draw it,
394
  // so it is a destination first and a family head second. Otherwise: the
395
  // server's flag wins, and has-children is the fallback for a payload that
 
88
  return { main, utility };
89
  }
90
 
91
+ /** `empty` (wave 18): the SERVER's reason for an empty page list. `"no_databases"` means a
92
+ * freshly provisioned tenant with no modules and no databases YET — a legitimate starting
93
+ * state, not a broken account. Absent means "no reason given", which the shell reads as the
94
+ * misconfigured-account case it always did. Distinguishing them is what stops the rail saying
95
+ * "no surfaces are available" beside a hero saying "welcome, create your first database". */
96
+ export type NavResult =
97
+ | { ok: true; pages: NavPage[]; empty?: string }
98
+ | { ok: false; status: number };
99
 
100
  /**
101
  * X2 `GET /api/v1/nav`. `status` rides the failure so the caller can tell the
 
108
  try {
109
  const res = await fetch(`${API_V1}/nav`, { credentials: CREDENTIALS });
110
  if (!res.ok) return { ok: false, status: res.status };
111
+ const body = await res.json();
112
+ const empty = typeof body?.empty === "string" ? body.empty : undefined;
113
+ return { ok: true, pages: parsePages(body), ...(empty ? { empty } : {}) };
114
  } catch {
115
  return { ok: false, status: 0 };
116
  }
 
357
  // `workspace.cohortMode`, the events — reachable from the Customer surface.
358
  // `product_data` joined wave 16 (C-TOPIC, owner items 9+10): the SAME grid tree over the
359
  // SKU catalogue — Shell maps it to `scope="product"` and everything else is the topic's.
360
+ // `automation` joined wave 18 (C-AUTONAV): SESSION D's surface, its own rail + editor.
361
+ // User tables are native BY PREFIX (`ut_`), decided in `shapeNav`, not listed here.
362
+ ["customer_data", "product_data", "automation"]);
363
 
364
  /** The native keys the Y1 page envelope drives (as opposed to `customer_data`,
365
  * which is its own component tree). Kept beside NATIVE_KEYS so the two cannot
 
400
  }
401
 
402
  const entry = (p: NavPage, depth: number): NavEntry => {
403
+ // Wave 18 (C3-UT): any `ut_`-prefixed key is a USER TABLE — drawable by the same grid
404
+ // tree (Shell passes the key through as the scope), so it is native BY PREFIX rather
405
+ // than by membership. The server only emits ut rows this session may open, so this is a
406
+ // capability rule, never a permission one.
407
+ const isNative = NATIVE_KEYS.has(p.key) || p.key.startsWith("ut_");
408
  // A native surface stays native even with children — the shell can draw it,
409
  // so it is a destination first and a family head second. Otherwise: the
410
  // server's flag wins, and has-children is the fallback for a payload that
web/src/viz/seriesData.ts CHANGED
@@ -28,6 +28,10 @@ export interface WireField {
28
  export const VIZ_FIELD_TYPES: ReadonlySet<string> = new Set([
29
  "text", "status", "currency", "int", "date", "pct", "select", "user",
30
  "multiselect", "checkbox", "phone", "email", "url", "rating", "created_time", "formula",
 
 
 
 
31
  ]);
32
 
33
  /**
 
28
  export const VIZ_FIELD_TYPES: ReadonlySet<string> = new Set([
29
  "text", "status", "currency", "int", "date", "pct", "select", "user",
30
  "multiselect", "checkbox", "phone", "email", "url", "rating", "created_time", "formula",
31
+ // Wave 18 C5 ④b — the RUNTIME copy of the vocabulary. A `Set` is not a `Record`: the
32
+ // compiler stays green when this goes stale and `asFields` silently downgrades the column
33
+ // to text. When a `FieldType` joins the union, this line moves WITH the batch.
34
+ "automation",
35
  ]);
36
 
37
  /**