Spaces:
Running on CPU Upgrade
Running on CPU Upgrade
Download platform/core/data_binding.py from fsanyoto/runloopable: direct link, hf CLI and curl.
- Browser
- Download file 20.4 kB
-
https://huggingface.co/spaces/fsanyoto/runloopable/resolve/main/platform/core/data_binding.py
- Command line
-
hf download hf://spaces/fsanyoto/runloopable/platform/core/data_binding.py
-
curl -L -o data_binding.py https://huggingface.co/spaces/fsanyoto/runloopable/resolve/main/platform/core/data_binding.py
20.4 kB
| """WHICH STORE THIS DEPLOYMENT IS ALLOWED TO WRITE β the fail-closed binding (D-315 / D-298). | |
| ββ THE SENTENCE THIS FILE EXISTS TO MAKE IMPOSSIBLE. Owner, 2026-08-18, verbatim: *"this is so | |
| critical that whenever we do staging, we don't fuck up anyone's data. EVER."* It had already | |
| happened once: `fsanyoto/loopable` became STAGING on 2026-08-14 having never been given a store of | |
| its own, so two different builds wrote tenant #0's real business data. Compose that with D-305 | |
| (the store is last-write-wins at whole-document granularity) and a routine staging deploy does not | |
| merely add test rows β it can erase a real user's views, fields and typed cell values. | |
| β A DEPLOY FLAG IS NOT THE FIX; THE FLAG IS THE THING THAT ALREADY FAILED. The isolation that | |
| existed was bound to a ROLE ("the staging Space"), and a guard bound to a role stops guarding the | |
| moment the role moves β which these two Spaces have done twice. So the binding here is on the | |
| container's OWN IDENTITY, which no deploy can forget to pass and no role swap can invalidate. | |
| ββ THE RULE, AND IT CLASSIFIES THE DEPLOYMENT RATHER THAN THE REPO: | |
| 1. the PRODUCTION deployment (identity in `PRODUCTION_SPACES`) -> may write anything | |
| 2. any other deployment writing a KNOWN production store -> REFUSED | |
| 3. any other deployment writing a repo on its OWN allowlist -> allowed | |
| 4. anything else -> REFUSED | |
| β WHY NOT CLASSIFY THE REPO, which is the design everybody reaches for first. A DENYLIST of | |
| production stores fails OPEN for the next tenant nobody remembered to add. A SUFFIX RULE | |
| (`*-staging-data`) pins a SPELLING rather than a claim β see [[gate-pins-a-spelling-not-a-claim]] β | |
| and a store named without the suffix is unprotected while reading as protected. Rule 4 is the one | |
| that survives a tenant nobody has provisioned yet: an unknown repo is refused, so a store this | |
| deployment was never handed cannot be written by accident. | |
| ββ AND RULE 4 IS NOT THEORETICAL β IT IS THE HALF OF D-315 THE REGISTER ROW DID NOT KNOW ABOUT. | |
| Measured 2026-08-18 by reading both control planes: `royal-imports/cfo-os-staging-data`'s own | |
| `tenants.json` is byte-identical to live's and names **`royal-imports/aios-nurilab-data`, | |
| `aios-gtmlab-data` and `aios-loopable-data`** β the three other tenants' REAL stores. So | |
| `--data-repo=` only ever isolated tenant #0; `harness/runtime.py`'s `store.for_repo(repo)` path | |
| resolved three PRODUCTION repos out of the staging store and wrote them, and no flag has ever | |
| touched that path. The exposure is FOUR production stores, not one. | |
| β READS ARE NOT REFUSED, AND THAT IS A DELIBERATE, STATED SCOPE DECISION. `Store.get` is lenient | |
| by contract: a refused read would hand back `{}`, a caller would paint an empty grid, and the whole | |
| Space would read as "no data" rather than as "refused" β the exact | |
| [[empty-answer-vs-unfinished-answer]] shape this codebase has shipped before. A refused WRITE is | |
| loud (503, with the reason). So staging still SEES production data when pointed at it; it can no | |
| longer CHANGE it. Narrowing the read is a separate decision and is booked, not smuggled in here. | |
| β ABSENT IDENTITY FAILS CLOSED, AND THE FAILURE LOOKS LIKE AN OUTAGE. If a container exposes none | |
| of the identity variables, it is treated as "not the production deployment" and every write to a | |
| production store refuses. That is the safe direction, but on a live Space it would read as a total | |
| product outage β which is why `describe()` reports the RAW observations and not just the verdict: | |
| per D-160 the Space environment cannot be read from outside, so the container has to say what it | |
| saw. One deploy then answers "does SPACE_ID exist here", instead of three. | |
| """ | |
| import os | |
| from core import freeze_notice | |
| #: The deployment that OWNS production data. A frozenset rather than a string because the product | |
| #: has had two public addresses before and will again. | |
| #: | |
| #: β THIS SET IS DUPLICATED IN `aios-web/deploy_web.py` AS `PUBLIC_LIVE`, on purpose and with a | |
| #: gate: `deploy_web` is a standalone CLI that must run without the platform package importable, so | |
| #: it cannot import this. `verify_store_binding.py::section_constants` asserts the two literals | |
| #: agree β a constant two features share is a constant that drifts ([[constant-two-features-share]]). | |
| PRODUCTION_SPACES = frozenset({'fsanyoto/runloopable'}) | |
| # Provisioning pins this exact identity into the scale-to-zero ECS task definition. It is the | |
| # only non-Space process allowed to write production tenant schemas; unknown identities remain | |
| # refused by the same default-deny guard. | |
| PRODUCTION_WORKERS = frozenset({'aws:aios-automation-worker'}) | |
| #: The stores that hold REAL customer business data. Enumerated 2026-08-18 from the live control | |
| #: plane (`royal-imports/cfo-os-data::tenants.json`), not guessed. | |
| #: | |
| #: β THIS LIST IS RULE 2, NOT THE WHOLE GUARD. It exists to make a DELIBERATE mis-pointing loud β | |
| #: `OS_DATA_REPO=royal-imports/cfo-os-data` typed on a laptop, or a production repo added to a | |
| #: sandbox allowlist by hand. Rule 4 is what covers a tenant provisioned after this line was | |
| #: written, which is why the file does not depend on this staying complete. | |
| PRODUCTION_STORES = frozenset({ | |
| 'royal-imports/cfo-os-data', # tenant #0 β Royal Imports | |
| 'royal-imports/aios-nurilab-data', # Nurilab | |
| 'royal-imports/aios-gtmlab-data', # GTM Lab | |
| 'royal-imports/aios-loopable-data', # Loopable | |
| }) | |
| #: The historical default of `OS_DATA_REPO`. Kept as a name so the resolution below can say what it | |
| #: is refusing to assume, rather than repeating a literal. | |
| PRODUCTION_DEFAULT_STORE = 'royal-imports/cfo-os-data' | |
| #: The env vars a Hugging Face Space container sets to describe itself, in the order they are | |
| #: trusted. `SPACE_ID` is the canonical one; the pair is the documented fallback. Read as a LIST so | |
| #: `describe()` can report every one of them and what it held. | |
| _IDENTITY_VARS = ('AIOS_DEPLOYMENT_ID', 'SPACE_ID', 'SPACE_AUTHOR_NAME', | |
| 'SPACE_REPO_NAME', 'SPACE_HOST') | |
| #: The env a NON-production deployment uses to name the stores it may write, beyond `OS_DATA_REPO`. | |
| #: Comma-separated. | |
| #: | |
| #: β THE POLARITY IS THE POINT, and it is what keeps trap 1 satisfied. Forgetting to set this | |
| #: makes a deployment write LESS, never more β so an omitted deploy argument is a refusal, not a | |
| #: silent grant. That is the exact inverse of `--data-repo`, whose omission printed | |
| #: "OS_DATA_REPO unchanged" and left a Space on production. | |
| _SANDBOX_STORES_VAR = 'AIOS_SANDBOX_STORES' | |
| #: The local (no-identity) escape hatch. Present β writes to production stores are permitted from a | |
| #: machine with no Space identity, AND every commit this process makes is stamped distinguishably | |
| #: (see `local_commit_suffix`). Deliberately does NOT widen a Space: a container that HAS an | |
| #: identity and is not production is refused with no override, because that is the case the owner | |
| #: said "EVER" about. | |
| _LOCAL_OPT_IN_VAR = 'AIOS_ALLOW_PRODUCTION_DATA' | |
| class StoreWriteRefused(RuntimeError): | |
| """A write this deployment is not allowed to make. | |
| β DELIBERATELY NOT A SUBCLASS OF `grid_events.StoreUnavailable`, and the reason is the routes. | |
| Several of them carry `except StoreUnavailable:` blocks that DEGRADE to a session-scoped | |
| fallback workspace β correct for an outage, catastrophic for a refusal, because the user would | |
| be told the change was saved somewhere. A distinct type falls through all of them to the | |
| app-level handler, which answers 503 with the reason and never claims a save. | |
| """ | |
| def _env(name): | |
| return str(os.environ.get(name) or '').strip() | |
| def deployment_id(): | |
| """This container's own identity, or `''` when it is not a Space (i.e. local dev). | |
| β NOT DERIVED FROM ANY DEPLOY ARGUMENT. That is the whole design: `SPACE_ID` is set by the | |
| platform inside the container, so it cannot be forgotten, mistyped, or left behind by a Space | |
| that changed role without being redeployed. | |
| """ | |
| explicit = _env('AIOS_DEPLOYMENT_ID') | |
| if explicit: | |
| return explicit | |
| sid = _env('SPACE_ID') | |
| if sid: | |
| return sid | |
| author, repo = _env('SPACE_AUTHOR_NAME'), _env('SPACE_REPO_NAME') | |
| if author and repo: | |
| return f'{author}/{repo}' | |
| return '' | |
| def is_production_deployment(): | |
| """True only on the deployment that owns production data. Unknown β False (fail closed).""" | |
| return deployment_id() in (PRODUCTION_SPACES | PRODUCTION_WORKERS) | |
| def is_space(): | |
| """True when this process is running inside a Space at all. | |
| The distinction matters exactly once: a Space that is not production gets NO override, while a | |
| laptop does. `deployment_id()` conflates them into `''`, so ask separately. | |
| """ | |
| return bool(deployment_id()) | |
| def sandbox_allowlist(): | |
| """The repos a non-production deployment may write: `OS_DATA_REPO` + `AIOS_SANDBOX_STORES`.""" | |
| allowed = {r.strip() for r in _env(_SANDBOX_STORES_VAR).split(',') if r.strip()} | |
| own = _env('OS_DATA_REPO') | |
| if own: | |
| allowed.add(own) | |
| return frozenset(allowed) | |
| def local_override(): | |
| """True when a machine with no Space identity has explicitly opted in to production data.""" | |
| return (not is_space()) and _env(_LOCAL_OPT_IN_VAR) == '1' | |
| #: W-219. Inherited by every child a gate process starts, so a subprocess a gate spawns is a gate too. | |
| _GATE_PROCESS_VAR = 'AIOS_GATE_PROCESS' | |
| #: The directories the product gate battery lives in (`aios-web/api`, `aios-web/web`, `aios-web`, | |
| #: `platform`). `ops/verify_*.py` are OPERATOR tools, some of which read or probe a live store on | |
| #: purpose, so they are not gates here. | |
| _GATE_DIRS = frozenset({'api', 'web', 'aios-web', 'platform'}) | |
| def _argv_gate(): | |
| """The `verify_*.py` gate script this process was started as, or `''`.""" | |
| import sys | |
| argv0 = str((getattr(sys, 'argv', None) or [''])[0] or '') | |
| name = os.path.basename(argv0).lower() | |
| if not (name.startswith('verify_') and name.endswith('.py')): | |
| return '' | |
| parent = os.path.basename(os.path.dirname(os.path.abspath(argv0))).lower() | |
| return name if parent in _GATE_DIRS else '' | |
| def gate_process(): | |
| """The gate this process is, or `''`. W-219: a gate writes FIXTURE stores only, never a real one. | |
| ββ WHY THIS LIVES AT THE STORE DOOR AND NOT IN EACH GATE. A lane container carries the exact | |
| isolated staging posture `store_pg.write_refusal` is written to ALLOW (`STORE_BACKEND=pg`, the | |
| staging `DATABASE_URL`, `AIOS_DEPLOYMENT_ID=fsanyoto/loopable`, `AIOS_SANDBOX_TENANTS= | |
| royal-imports`), and `_fake_store.installed` swaps module functions a bound `PgStore` never | |
| calls. So `verify_meta.py` run in SDLC 61 wrote staging's `t_royal_imports` through the real | |
| app, and a pin at the top of each gate (W-219, W-240) only covers the gates somebody pinned: | |
| `verify_amazon` and `verify_odoo_relational` carry none. Asking the PROCESS covers every gate, | |
| including one written tomorrow. | |
| Two ways to be one: started as `verify_*.py` from a battery directory (every battery row and | |
| every lane runs `python <dir>/verify_<x>.py`), or `AIOS_GATE_PROCESS=1` inherited from one. | |
| β There is no way OUT through the environment, because a lane container exports whatever it | |
| was handed: the only off switch is an in-process patch of this function, which is what a gate | |
| testing the PRODUCT rules does. | |
| """ | |
| name = _argv_gate() | |
| if name: | |
| return name | |
| return 'a process started by a gate' if _env(_GATE_PROCESS_VAR) == '1' else '' | |
| if _argv_gate(): | |
| os.environ[_GATE_PROCESS_VAR] = '1' # W-219: a child this gate spawns is a gate too | |
| def gate_write_refusal(repo): | |
| """`None` unless this is a gate process writing a PRODUCTION store, else why not. W-219. | |
| Consulted by `check_write` BEFORE `write_refusal`, so neither `AIOS_ALLOW_PRODUCTION_DATA=1` | |
| on a laptop nor a production identity opens a production store to a gate. | |
| """ | |
| who = gate_process() | |
| rid = str(repo or '').strip() | |
| if not who or rid not in PRODUCTION_STORES: | |
| return None | |
| return (f'this process is a verification gate ({who}), and a gate never writes {rid}, which ' | |
| f'holds real customer data. A gate writes fixture stores only: empty HF_TOKEN and ' | |
| f'point OS_DATA_REPO at a fixture store (W-219).') | |
| def default_store(): | |
| """What `OS_DATA_REPO` resolves to when it is unset. | |
| ββ FACT (1) OF D-315 DIES HERE: `core/store.py` used to read | |
| `os.environ.get('OS_DATA_REPO', 'royal-imports/cfo-os-data')`, so a Space that was never given | |
| the key β every NEW Space, by construction β came up bound to tenant #0's real business data. | |
| The default now exists ONLY on the deployment that owns it. | |
| β IT STILL RETURNS THE PRODUCTION ID ELSEWHERE, RATHER THAN `None` OR `''`, AND THAT IS | |
| DELIBERATE. A `None` repo would make `Store.__init__` β which runs at IMPORT time via | |
| `_DEFAULT = for_repo(REPO)` β raise or bind something nonsensical, killing the app and every | |
| gate on any non-production machine. Returning the real id keeps binding total and lets | |
| `write_refusal` refuse the WRITE with a message that names the actual problem. Reads still | |
| work, so a developer sees a populated app that will not let them damage it. | |
| """ | |
| return PRODUCTION_DEFAULT_STORE | |
| def write_refusal(repo): | |
| """`None` if this deployment may write `repo`, else the sentence explaining why not. | |
| β RETURNS A REASON RATHER THAN A BOOL, because [[report-the-cause-before-you-fix-it]]: a | |
| silent refusal makes every theory about it unfalsifiable, and this one fires on a path where | |
| the operator cannot read the container's environment (D-160). | |
| """ | |
| rid = str(repo or '').strip() | |
| if not rid: | |
| return ('no store repo is bound. Refusing to guess: an unnamed store is how a deployment ' | |
| 'ends up on somebody else\'s data.') | |
| if is_production_deployment(): | |
| return None # rule 1 β the live Space owns its data | |
| if rid in PRODUCTION_STORES: # rule 2 β never, from anywhere else | |
| if local_override(): | |
| return None | |
| who = deployment_id() or 'this machine (no Space identity)' | |
| if is_space(): | |
| return (f'{who} is not the production deployment, so it may not write {rid}, which ' | |
| f'holds real customer data. Point this Space at its own store with ' | |
| f'--data-repo=, or add it to {_SANDBOX_STORES_VAR}. There is no override for ' | |
| f'a Space: a build that is not live never writes live data.') | |
| return (f'{who} may not write {rid}, which holds real customer data. Set OS_DATA_REPO to a ' | |
| f'sandbox store, or set {_LOCAL_OPT_IN_VAR}=1 to opt in deliberately (every commit ' | |
| f'is then stamped as a local write so it can be told apart afterwards).') | |
| allow = sandbox_allowlist() # rule 3 β what it was explicitly handed | |
| if rid in allow: | |
| return None | |
| who = deployment_id() or 'this machine (no Space identity)' | |
| listed = ', '.join(sorted(allow)) or '(nothing)' | |
| return (f'{who} was not given {rid}. A non-production deployment may write only the stores it ' | |
| f'was handed: {listed}. Add it to {_SANDBOX_STORES_VAR} if that is intended.') | |
| def check_write(repo, operation='write'): | |
| """Raise `StoreWriteRefused` unless this deployment may write `repo`. The one enforcement door.""" | |
| freeze_notice.check_write() | |
| reason = gate_write_refusal(repo) or write_refusal(repo) | |
| if reason: | |
| raise StoreWriteRefused(f'{operation} to {repo!r} refused: {reason}') | |
| def local_commit_suffix(): | |
| """A marker appended to commit messages made under the local override, else `''`. | |
| β THIS IS THE SECOND HALF OF D-298's EXIT CONDITION, satisfied rather than argued around: *"a | |
| local run either cannot reach the live store without an explicit opt-in, OR stamps its commits | |
| distinguishably"*. The measured complaint was that a laptop's commits were titled | |
| `update user_tables` β byte-identical to what the live Space writes every few minutes β so a | |
| local run and production were indistinguishable in the store's git history, which is the ONLY | |
| audit trail this product has. Both clauses now hold: the opt-in is required AND it stamps. | |
| """ | |
| if not local_override(): | |
| return '' | |
| import getpass # noqa: PLC0415 β only on the opt-in path | |
| import socket # noqa: PLC0415 | |
| try: | |
| who = getpass.getuser() | |
| except Exception: # noqa: BLE001 | |
| who = 'unknown' | |
| try: | |
| host = socket.gethostname() | |
| except Exception: # noqa: BLE001 | |
| host = 'unknown' | |
| return f' [LOCAL DEV {who}@{host}]' | |
| def describe(repo=None): | |
| """What this container sees about itself β the payload behind the status field. | |
| β IT REPORTS THE RAW OBSERVATIONS, NOT ONLY THE VERDICT, and that is the point rather than | |
| debug noise. D-160 established that a Space's environment cannot be read from outside, so | |
| "is staging on the right store?" is a question only the container can answer. A verdict alone | |
| ("production: false") cannot distinguish "SPACE_ID says staging" from "SPACE_ID does not exist | |
| on this platform at all" β two states with the same verdict and completely different fixes. | |
| Reporting the variables makes ONE deploy answer it. | |
| β Carries no secret: a Space id and a dataset repo id are public names. The DATA is not, and | |
| none of it is here. The field is session-gated anyway (see `routes_admin.settings`). | |
| ββ W40-T13 (D-333) β IT ALSO NAMES THE STORE BACKEND, WHICH IS THE OTHER HALF OF THE SAME | |
| QUESTION AND THE HALF THAT WENT MISSING. `repo` answers "WHICH STORE am I bound to"; after the | |
| Postgres cutover the sharper question is "am I READING that store from HF or from Neon", and | |
| the payload could not answer it at all. W36-T62's `done-when` said *"/settings reports the pg | |
| backend"* and was therefore unmeasurable, which is how a whole cutover shipped with the | |
| verification clause pointed at a field that did not exist. It exists now. | |
| """ | |
| rid = str(repo or '').strip() | |
| # ββ THE IMPORT IS FUNCTION-LOCAL AND MUST STAY THAT WAY. `core/store.py` imports THIS module | |
| # at its own module scope, so pulling `core.store` up to the top of this file is a hard import | |
| # cycle that kills the app at boot rather than at a test. Do not "tidy" it upward. | |
| # | |
| # β AND AN UNRECOGNISED BACKEND REPORTS `unknown`, NEVER `hf`. `store.backend()` RAISES on a | |
| # value it does not know, and this function is called from `routes_admin::_store_binding` | |
| # inside a bare `except` whose fallback dict has no `backend` key at all β so a raise here | |
| # would delete the field from the payload on exactly the deployment that most needs it. And | |
| # defaulting to `hf` would be worse than raising: a misconfigured container would report the | |
| # store it is NOT reading, which is the guess this field exists to replace. ImportError rides | |
| # the same branch for the same reason: an honest "I could not tell" beats a confident wrong | |
| # answer on a surface nobody can inspect from outside (D-160). | |
| try: | |
| import core.store as store # noqa: PLC0415 β cycle, see above | |
| store_backend = store.backend() | |
| except (RuntimeError, ImportError): | |
| store_backend = 'unknown' | |
| return { | |
| 'backend': store_backend, | |
| 'deployment': deployment_id(), | |
| 'production': is_production_deployment(), | |
| 'repo': rid, | |
| 'writable': write_refusal(rid) is None if rid else False, | |
| 'refusal': write_refusal(rid) if rid else None, | |
| 'localOverride': local_override(), | |
| 'allowlist': sorted(sandbox_allowlist()), | |
| # The raw half. `''` means "the variable is absent", which is itself the answer on a | |
| # platform that does not set it. | |
| 'observed': {name: _env(name) for name in _IDENTITY_VARS}, | |
| } | |