runloopable / platform /core /keychain.py
fsanyoto's picture
Deploy AIOS web (React glide grid + FastAPI slice)
e9619ab verified
Raw History Blame Contribute Delete
55 kB
"""core/keychain.py β€” per-tenant CREDENTIAL STORE (wave 18, contract C7 / ruling R3). An admin stores a data source's secrets (Royal's Odoo API key today; any tenant's sources tomorrow) encrypted at rest in the TENANT's store bucket. The runtime posture is R3's "live with env fallback": the connector layer asks `odoo_creds(rt, connection_id)` FIRST and falls back to the process environment (tenant #0's `.env` / Space secrets), so Royal keeps working with an empty keychain and a new tenant needs no code to bring keys. The FULL cutover (env retired) is staged for a later wave β€” R3, verbatim. CRYPTO: v2 uses AES-256-GCM under the tenant's keychain subkey with tenant and entry AAD. V1 and untagged rows use Fernet under `AIOS_KEYCHAIN_KEY` until migrated. An unavailable tenant key refuses writes without falling back to Fernet. SHAPE (bucket `keychain`, per-tenant via the runtime handle): {entries: {id: {id, connection_id, slot, label, type, enc, envelope_version, preview, created, createdBy}}} Amazon rows also carry `client_secret_rotation_deadline`, empty when unknown, and may carry the ciphertext-bound `probe` proof. `enc` holds base64 ciphertext over the JSON fields dict. `preview` = a masked hint ("Β·Β·Β·Β·1234") computed ONCE at write from the entry's most secret-looking field β€” the ONLY part of a secret that ever leaves this module through a route. `read_fields` exists for the CONNECTOR LAYER; no route returns its output. """ import base64 import datetime as _dt import hashlib import json import os import secrets as _secrets from core import key_shapes, tenant_keys KEY = 'keychain' ORIGINAL_ENVELOPE_VERSION = 1 ENVELOPE_VERSION = 2 AMAZON_ROTATION_DEADLINE_FIELD = 'client_secret_rotation_deadline' # Required Amazon fields belong below the clients that consume credentials. _AMAZON_OAUTH_FIELDS = ('client_id', 'client_secret', 'refresh_token') AMAZON_CREDENTIAL_FIELDS = { 'amazon_sp': _AMAZON_OAUTH_FIELDS + ('seller_id', 'marketplace_id'), 'amazon_ads': _AMAZON_OAUTH_FIELDS + ('profile_id',), } #: ⭐ WAVE 23 (C11/R8) β€” `stripe` and `shopify` joined. They are TOKEN connectors, not OAuth #: ones, and that is the whole reason they could ship in this wave: both issue a long-lived #: restricted key / custom-app token an admin pastes in, so they need no OAuth client, no #: consent screen and no app review β€” the three things that make every other provider on the #: standing queue a multi-week errand. A type here is what lets the connectors directory show #: them as CONNECTABLE rather than as another faded to-do. #: ⭐⭐ WAVE 31 (R2/R5) β€” `meta_ads` JOINS, AND IT IS THE FIRST END-TO-END KEYCHAINβ†’HTTP CONNECTOR. #: Every type above it is either read by nothing (`generic`, `stripe`, `shopify` store a key that #: no data path consumes yet) or read by ONE resolver that talks XML-RPC (`odoo`). So `meta_ads` #: is the first entry whose stored secret actually reaches an HTTP API in production, through #: `meta_creds` below β€” which is why W31-T48's `how:` says *build the seam, do not assume it*. #: β›” A TYPE MISSING FROM THIS SET 400s IN `add_entry`, which is the entire failure: the Connectors #: directory would offer Meta Ads, an admin would paste a token, and the write would be refused #: with "type must be one of odoo, generic, stripe, shopify" β€” a message about our data structure, #: on a screen about their advertising account. #: ⚠ R8 keeps the two doors separate on purpose: this is the TENANT's own token (GTM Lab pastes #: theirs), while R5's "Connect Facebook" is the delegated-OAuth door Nango owns. Neither replaces #: the other, and the schema work is blocked on neither. ENTRY_TYPES = {'odoo', 'generic', 'stripe', 'shopify', 'meta_ads', 'amazon_sp', 'amazon_ads', 'slack'} #: which stored field feeds the masked preview, per type (first present wins) #: ⚠ `access_token` JOINED IN WAVE 31, AND ITS ABSENCE WAS A REAL DEFECT FOUND BY BUILDING THE #: SEAM RATHER THAN BY READING IT. Meta's credential is conventionally an `access_token`, and with #: only `token` on this list `add_entry` stored the secret correctly and computed an EMPTY preview #: β€” so the keychain table would show a Meta row with no hint at all, and an admin holding two #: tokens could not tell which one is stored or whether the paste worked. Nothing raises; the #: entry is fine; the only symptom is a blank cell on the screen the feature exists for. #: β›” `refresh_token` JOINED FOR AMAZON SP-API, AND ITS ABSENCE WOULD HAVE BEEN THE WAVE-31 #: DEFECT A SECOND TIME. An `amazon_sp` entry's only secret is its refresh token; with the #: tuple as it stood, `add_entry` stored the credential correctly and computed an EMPTY #: preview, so the keychain table would show an Amazon row with no hint at all. Nothing #: raises; the entry is fine; the only symptom is a blank cell on the screen the feature #: exists for. Matching is EXACT-KEY (`clean.get(k)` below), so a near-miss name silently #: yields the same blank. _PREVIEW_FIELDS = ('api_key', 'access_token', 'value', 'password', 'token', 'refresh_token', 'bot_token') MAX_ENTRIES = 40 MAX_FIELD_LEN = 500 class KeychainLocked(Exception): """A required encryption key or stored ciphertext is unavailable to the reader.""" class KeychainEnvelopeError(ValueError): """A stored credential envelope is malformed or unsupported.""" class KeychainEnvelopeVersionError(KeychainEnvelopeError): """An unsupported envelope is distinct from an unavailable encryption key.""" def _check_envelope_version(row): # Untagged rows use the original Fernet envelope without rewriting storage. version = row.get('envelope_version', ORIGINAL_ENVELOPE_VERSION) if type(version) is not int or version not in (ORIGINAL_ENVELOPE_VERSION, ENVELOPE_VERSION): raise KeychainEnvelopeVersionError('This credential uses an unsupported encryption envelope version.') return version def _decoded_fields(raw): try: data = json.loads(raw.decode('utf-8')) except Exception: raise KeychainEnvelopeError('The stored credential payload is invalid.') from None if not isinstance(data, dict) or not data: raise KeychainEnvelopeError('The stored credential payload is invalid.') return data def _amazon_rotation_deadline(value): """Validate an Amazon-issued timestamp supplied by the admin, never infer an expiry.""" if value is None or value == '': return '' message = 'Enter the Amazon-issued secret rotation deadline as an ISO 8601 timestamp with a timezone.' if not isinstance(value, str) or len(value) > 64: raise ValueError(message) value = value.strip() if not value: return '' try: stamp = _dt.datetime.fromisoformat(value.replace('Z', '+00:00')) if stamp.tzinfo is None or stamp.utcoffset() is None: raise ValueError() return stamp.astimezone(_dt.timezone.utc).isoformat().replace('+00:00', 'Z') except (ValueError, OverflowError): raise ValueError(message) from None def _fernet(): key = (os.environ.get('AIOS_KEYCHAIN_KEY') or '').strip() if not key: raise KeychainLocked('AIOS_KEYCHAIN_KEY is not configured') try: from cryptography.fernet import Fernet return Fernet(key.encode('ascii')) except Exception as e: raise KeychainLocked(f'keychain key unusable: {type(e).__name__}') def seal(payload: bytes) -> str: """Seal legacy v1 bytes with the configured Fernet key.""" return _fernet().encrypt(payload).decode('ascii') def unseal(token: str) -> bytes: """Open legacy v1 bytes or refuse a locked or unreadable entry.""" try: return _fernet().decrypt(token.encode('ascii')) except KeychainLocked: raise except Exception: raise KeychainLocked('this entry does not decrypt under the configured key') from None def unlocked(): try: if tenant_keys.backend_name() is not None: return True _fernet() return True except (KeychainLocked, tenant_keys.TenantKeyLocked): return False def _tenant(rt): try: return tenant_keys.context(getattr(rt, 'key', None))['tenant'] except tenant_keys.TenantKeyLocked as exc: raise KeychainLocked(str(exc)) from None def _envelope_aad(tenant, entry_id): return json.dumps([tenant, str(entry_id), 'keychain'], ensure_ascii=False, separators=(',', ':')).encode('utf-8') def seal_fields(tenant, entry_id, fields): """Seal fields for their final tenant and entry identity, retaining the legacy path.""" try: tenant = tenant_keys.context(tenant)['tenant'] key = tenant_keys.subkey(tenant, 'keychain') except tenant_keys.TenantKeyLocked as exc: raise KeychainLocked(str(exc)) from None raw = json.dumps(fields, ensure_ascii=False).encode('utf-8') if key is None: return {'enc': seal(raw), 'envelope_version': ORIGINAL_ENVELOPE_VERSION} blob = tenant_keys.aead_seal(key, raw, _envelope_aad(tenant, entry_id)) return {'enc': base64.b64encode(blob).decode('ascii'), 'envelope_version': ENVELOPE_VERSION} def open_fields(tenant, entry_id, row): """Open an envelope with its requested identity, never an identity supplied by the row.""" version = _check_envelope_version(row) if not isinstance(row.get('enc'), str) or not row['enc']: raise KeychainEnvelopeError('The stored credential ciphertext is missing.') if version == ORIGINAL_ENVELOPE_VERSION: return _decoded_fields(unseal(row['enc'])) try: tenant = tenant_keys.context(tenant)['tenant'] key = tenant_keys.subkey(tenant, 'keychain') if key is None: raise KeychainLocked('The tenant data key is unavailable.') blob = base64.b64decode(row['enc'], validate=True) raw = tenant_keys.aead_open(key, blob, _envelope_aad(tenant, entry_id)) except tenant_keys.TenantKeyLocked as exc: raise KeychainLocked(str(exc)) from None except KeychainLocked: raise except Exception: raise KeychainLocked('this entry does not decrypt under the tenant data key') from None return _decoded_fields(raw) def _bucket(rt): return rt.get(KEY) or {} # ═════════════════════════════════════════════════════════════════════════════════════════════ # ⭐⭐ W54-T44 / OWNER RULING R4 / CONTRACT C4 β€” A CONNECTION HAS AN IDENTITY. # ═════════════════════════════════════════════════════════════════════════════════════════════ # # R4, verbatim: *"If a tenant is an agency that helps with clients that each have Odoo, then the # connector for Odoo can be many depending on the API. This is why its critical."* So MANY # connections of ONE kind coexist, and each one has to be addressable on its own. # # ⭐ THE STORAGE ALREADY ALLOWED THIS AND THE READER DID NOT. `add_entry` has always minted a # fresh `k_<hex>` per call with no uniqueness constraint, so a second `odoo` entry was already # storable β€” it was simply never READ (every resolver takes the first of its type). This block # adds the two facts a reader needs in order to tell two connections of one kind apart: # # `connection_id` the entry's own id, stamped EXPLICITLY rather than left implicit. It is # stable, unique, never reused and never re-derived by sorting. # `slot` an IMMUTABLE per-(tenant, type) ordinal, minted HIGH-WATER, which is what # makes the default label `Odoo 1` / `Odoo 2` stable and what a namespaced # table can be keyed on. Deleting a connection must never let a later one # reuse a retired ordinal, or two different connections would answer to one # address at two different times. # # β›” THE READ DEFAULTS ARE READ DEFAULTS AND NEVER A MIGRATION, the same posture `SCOPE_KEY` # takes below. Rows written before this wave carry neither field: they read back as # `connection_id = id` and as their 1-based position among entries of their type in the existing # insertion-id sort. Materialising that on the next read would rewrite a tenant's encrypted # store to record something already derivable ([[a-migration-that-runs-on-the-next-write]]) β€” # and this store's rows are Fernet tokens, so a rewrite is the one operation that can lose a # credential. Nothing here decrypts anything. #: type -> the human word the default label is built from. `generic` has no product name behind #: it, so it says what it is rather than inventing one. _KIND_LABELS = {'odoo': 'Odoo', 'meta_ads': 'Meta Ads', 'amazon_sp': 'Amazon', 'amazon_ads': 'Amazon Ads', 'stripe': 'Stripe', 'shopify': 'Shopify', 'generic': 'Connection', 'slack': 'Slack'} def kind_label(etype): """The human word for a connection type, for a default label. Never blank.""" return _KIND_LABELS.get(str(etype or '').strip().lower()) or 'Connection' def connection_id_of(entry, entry_id): """A stored row's stable connection id β€” its own `connection_id`, else its entry id. ⚠ A NAMED FUNCTION RATHER THAN AN INLINE `or`, so that the identity has ONE derivation a gate can break in memory: the negative control for "two connections stay two" needs somewhere to collapse them. """ return str((entry or {}).get('connection_id') or entry_id) def slot_of(entry, ordinal): """A stored row's slot β€” its own `slot` when it has one, else the read default `ordinal`. ⚠ `ordinal` is the caller's 1-based position among entries of the SAME type, in the same insertion-id sort every resolver already uses. A stored slot always wins, which is what makes the field immutable once written. """ try: n = int((entry or {}).get('slot') or 0) except (TypeError, ValueError): n = 0 return n if n >= 1 else int(ordinal) def list_entries(rt, etype=None): """Metadata ONLY β€” never a decrypted field, never `enc` itself. Optional `etype` filters metadata without changing the unfiltered reader or slot identities. Every row carries `connection_id` and `slot` (W54-T44 / C4). Rows may GAIN keys here and must never LOSE one: `verify_api` asserts the listing has no `enc` / `url` / `api_key` / `fields`, and several callers read these rows by key. """ out = [] seen = {} for eid, e in sorted((_bucket(rt).get('entries') or {}).items()): if not isinstance(e, dict): continue entry_type = e.get('type') or 'generic' seen[entry_type] = ordinal = seen.get(entry_type, 0) + 1 out.append({'id': eid, 'connection_id': connection_id_of(e, eid), 'slot': slot_of(e, ordinal), 'label': e.get('label') or eid, 'type': entry_type, 'preview': e.get('preview') or '', 'created': e.get('created') or '', 'createdBy': e.get('createdBy') or ''}) if etype is not None: wanted = str(etype or '').strip().lower() return [row for row in out if row['type'] == wanted] return out def next_slot(rt, etype): """The slot a NEW connection of `etype` takes: one above the highest this tenant has ever shown for that type. Never decrypts anything. β›” HIGH-WATER, NOT A COUNT. `len(entries of this type) + 1` would hand a retired ordinal to a new connection the moment one is deleted β€” two different credentials answering to `Odoo 2`, a month apart, with every table W54-T46 namespaces on that ordinal pointing at the wrong one. ⚠ It reads through `list_entries`, so a row written before this wave counts at its READ DEFAULT rather than at zero: a tenant with two un-stamped Odoo entries gets 3, not 1. """ etype = str(etype or '').strip().lower() top = int((_bucket(rt).get('slot_high_water') or {}).get(etype) or 0) for row in list_entries(rt): if row.get('type') == etype: try: top = max(top, int(row.get('slot') or 0)) except (TypeError, ValueError): continue return top + 1 def add_entry(rt, label, etype, fields, username, *, scope=None, write_scope=None): """Encrypt + store. Returns the metadata row, or raises ValueError on shape problems and KeychainLocked when there is no key (a secret must never be stored in the clear). A scoped setup writer supplies `scope` and the API's `write_scope` callback. Personal ownership commits before the credential; callers omitting these keep their existing path. ⭐ W54-T44 / C4 β€” the row is stamped with its `connection_id` and its `slot`, and a caller that supplies NO label gets the default the slot names: the tenant's first Odoo connection is `Odoo 1`, the second `Odoo 2`. An explicitly supplied label always wins, and the label stays the user's to change afterwards (`rename_entry`). ⚠ THE TYPE IS VALIDATED FIRST NOW, because the default label is built from it: a blank label on an unknown type must still be told about the type, which is the fixable half. """ label = ' '.join(str(label or '').split())[:80] etype = str(etype or '').strip().lower() if etype not in ENTRY_TYPES: raise ValueError(f"type must be one of {', '.join(sorted(ENTRY_TYPES))}") if scope is not None and scope not in ('business', 'personal'): raise ValueError('invalid credential scope') if scope == 'personal': if etype in TENANT_WIDE_TYPES: raise ValueError('this connector requires business scope') if not username or not callable(write_scope): raise ValueError('personal scope requires an owner and a scoped writer') if etype == 'amazon_sp' and not isinstance(fields, dict): fields = {} if not isinstance(fields, dict) or (not fields and etype != 'amazon_sp'): raise ValueError('fields must be a non-empty object') deadline = (_amazon_rotation_deadline(fields.get(AMAZON_ROTATION_DEADLINE_FIELD)) if etype == 'amazon_sp' else '') clean = {} for k, v in list(fields.items())[:12]: k = str(k).strip()[:40] if etype == 'amazon_sp' and k == AMAZON_ROTATION_DEADLINE_FIELD: continue if k and isinstance(v, (str, int, float, bool)): clean[k] = str(v)[:MAX_FIELD_LEN] if etype in AMAZON_CREDENTIAL_FIELDS: missing = [key for key in AMAZON_CREDENTIAL_FIELDS[etype] if not isinstance(fields.get(key), str) or not clean.get(key, '').strip()] if missing: raise ValueError('Missing ' + ', '.join(missing) + '. Enter every required field.') refusal = key_shapes.check(etype, clean) if refusal: raise ValueError(refusal) if not clean: raise ValueError('no usable fields') secretish = next((clean[k] for k in _PREVIEW_FIELDS if clean.get(k)), '') preview = ('Β·Β·Β·Β·' + secretish[-4:]) if len(secretish) >= 4 else ('Β·Β·Β·Β·' if secretish else '') eid = f'k_{_secrets.token_hex(6)}' envelope = seal_fields(_tenant(rt), eid, clean) slot = next_slot(rt, etype) if not label: label = f'{kind_label(etype)} {slot}' row = {'id': eid, 'connection_id': eid, 'slot': slot, 'label': label, 'type': etype, **envelope, 'preview': preview, 'created': _dt.datetime.now().strftime('%Y-%m-%dT%H:%M:%S'), 'createdBy': str(username or '')} if etype == 'amazon_sp': row[AMAZON_ROTATION_DEADLINE_FIELD] = deadline # Setup links publish a personal entry only AFTER its existing scope writer has # committed the ownership row. If either write fails, keep that reservation: # deleting it could turn an uncertain credential commit into a business entry. if scope == 'personal': if not write_scope(rt, eid, scope, username): raise ValueError('credential scope could not be saved') scopes = rt.get(SCOPE_KEY) saved = scopes.get(eid) if isinstance(scopes, dict) else None if saved != {'scope': 'personal', 'owner': str(username)}: raise ValueError('credential scope could not be saved') def _up(cur): entries = cur.setdefault('entries', {}) if eid in entries: if entries[eid] == row: return cur raise ValueError('credential identity already exists') if len(entries) >= MAX_ENTRIES: raise ValueError(f'this keychain is at its {MAX_ENTRIES}-entry cap') entries[eid] = row high_water = dict(cur.get('slot_high_water') or {}) high_water[etype] = max(int(high_water.get(etype) or 0), slot) cur['slot_high_water'] = high_water return cur stored = rt.update(KEY, _up, flush='sync') if scope is not None: saved_entries = stored.get('entries') if isinstance(stored, dict) else None if not isinstance(saved_entries, dict) or saved_entries.get(eid) != row: raise RuntimeError('credential write was not confirmed') return {'id': eid, 'connection_id': eid, 'slot': slot, 'label': label, 'type': etype, 'preview': preview, 'created': row['created'], 'createdBy': row['createdBy']} def rename_entry(rt, entry_id, label): """Give ONE connection a new name. Returns its metadata row, or None when there is no such entry. Raises ValueError on a blank name. β›” THE LABEL AND NOTHING ELSE. `enc`, `connection_id`, `slot`, `preview`, `created` and `createdBy` are left exactly as they were, and no other row is touched β€” that is the whole contract of a rename under R4, where two connections of one kind sit side by side and renaming one must not disturb the other. β›” AND IT DECRYPTS NOTHING. A stored secret is never re-openable and a rename has no business asking for the key: this works on a LOCKED keychain, where re-encrypting a row to change a display string would refuse (or, worse, rewrite a Fernet token to alter a label). """ label = ' '.join(str(label or '').split())[:80] if not label: raise ValueError('give this connection a name') eid = str(entry_id) found = [] def _up(cur): row = (cur.get('entries') or {}).get(eid) if isinstance(row, dict): row['label'] = label found.append(True) return cur rt.update(KEY, _up, flush='sync') if not found: return None return next((e for e in list_entries(rt) if e['id'] == eid), None) def delete_entry(rt, entry_id): retired = next((row for row in list_entries(rt) if row['id'] == str(entry_id)), None) def _drop(cur): if retired: high_water = dict(cur.get('slot_high_water') or {}) kind = retired['type'] high_water[kind] = max(int(high_water.get(kind) or 0), int(retired['slot'])) cur['slot_high_water'] = high_water (cur.get('entries') or {}).pop(str(entry_id), None) return cur rt.update(KEY, _drop, flush='sync') return True def read_fields(rt, entry_id): """Decrypt ONE entry's fields β€” for the CONNECTOR LAYER only; no route returns this. None when the entry does not exist; KeychainLocked when the key is absent/wrong (a wrong key must be loud β€” silently empty creds would read as 'not configured').""" e = (_bucket(rt).get('entries') or {}).get(str(entry_id)) if e is None: return None if not isinstance(e, dict): raise KeychainEnvelopeError('The stored credential envelope is invalid.') return open_fields(getattr(rt, 'key', None), str(entry_id), e) def first_entry_of_type(rt, etype): """DOES THIS TENANT HAVE ANY ENTRY OF `etype` AT ALL? The first such metadata row, or None. Never decrypts anything. β›”β›” W54-T45 β€” THIS IS AN EXISTENCE QUESTION AND NOTHING ELSE, AND THE NAME IS THE ONLY THING LEFT THAT SUGGESTS OTHERWISE. It used to be the identity twin of `_first_creds`: "which entry serves" and "does one exist" were the same walk, which is exactly the coupling contract C4 exists to break. Under R4 a tenant may hold MANY Odoo connections, so "the first one" is no longer an answer to "which credential serves this connector" β€” that question is now `default_connection_id` / `_creds_for`, which resolve on a `connection_id` and order on the immutable `slot`. β›” Do not route any credential path back through here. What survives is the migration/idempotency question `ensure_entry_of_type` asks below, and `verify_api` asks in three places as a "nothing was written" probe: both of those genuinely mean *is there one*, not *which one*. """ return next((e for e in list_entries(rt) if e.get('type') == str(etype or '')), None) def ensure_entry_of_type(rt, etype, label, fields, username): """Create ONE entry of `etype` only if the tenant has none. `(row, why)`; `row` is None on skip. ⭐ W35-T45 / R11 β€” the primitive tenant #0's env-to-keychain migration is built from, and the reason it lives here is that only this module knows what "already has one" means. ⚠ W54-T45 SOFTENED WHAT IS AT STAKE WITHOUT CHANGING THIS FUNCTION. A second entry of a type is no longer invisible: under R4 it is a second CONNECTION with its own `connection_id`, and every resolver can address it. What this still prevents is a MIGRATION running twice and quietly minting `Odoo 2` out of the same environment variables `Odoo 1` came from. β›” IDEMPOTENT BY TYPE, NOT BY A FLAG. A migration that keys idempotency on a marker it writes itself is one lost marker away from running twice ([[a-migration-that-runs-on-the-next-write]]); keying it on the thing it would create cannot double-apply, whatever else fails. β›” AND IT REFUSES ON A LOCKED KEYCHAIN RATHER THAN STORING A SECRET IN THE CLEAR. `add_entry` would raise `KeychainLocked`; this answers with a REASON instead, because "there is no Fernet key on this deployment" is an operator fact to report, not an exception to propagate out of a boot thread where nobody reads it. ⚠ `why` IS RETURNED EVEN ON SUCCESS-BY-SKIP. "Already migrated" and "could not migrate" are different operator actions and a bare None cannot tell them apart. """ etype = str(etype or '').strip().lower() if etype not in ENTRY_TYPES: return None, f'{etype!r} is not a keychain entry type' existing = first_entry_of_type(rt, etype) if existing: return None, f'this tenant already has a {etype} entry ({existing["id"]})' if not unlocked(): return None, ('this deployment has no keychain key, and a credential must never be stored ' 'in the clear') try: return add_entry(rt, label, etype, fields, username), '' except Exception as exc: # noqa: BLE001 return None, f'{type(exc).__name__}: {exc}' def odoo_creds(rt, connection_id=None): """R3's resolver seam: the fields of the odoo connection NAMED by `connection_id`, else None. ⭐⭐ W54-T45 / R4 / C4 β€” IT TAKES A CONNECTION NOW, AND THAT IS THE WHOLE TICKET. An agency tenant holds one Odoo connection per client, so "the tenant's Odoo credential" is not a question with one answer any more. `connection_id` names WHICH one; a `connection_id` that names no entry of this type resolves to None and never to a sibling. ⚠ NO ID MEANS THE DESIGNATED DEFAULT (`default_connection_id`), which is the lowest-`slot` connection of the type. Every existing one-argument call site therefore keeps working and now means exactly that, rather than "whichever row happened to sort first". β›” None here still means "not configured", and the CALLER decides what that entails: `harness/runtime.py::odoo_source` falls back to tenant #0's compiled environment only when no connection was NAMED. """ return _creds_for(rt, 'odoo', connection_id) def meta_creds(rt, connection_id=None): """The named or designated default `meta_ads` connection's fields, else None. ⭐⭐ W49-T13 / D-202 β€” IT HAS A PRODUCTION CALLER NOW, AND THAT WAS THE WHOLE DEFECT. `harness/meta_store.py::token` read `META_ADS_ACCESS_TOKEN` unconditionally, so this resolver existed, was correct, and was reached by nothing: a tenant's stored Meta credential was accepted, previewed and testable while every sync ran on the deployment's own token. The loader now asks here, and no tenant falls back to the environment. Deleting this function because "nothing calls it" would have been the wrong repair. β›” IT IS DELIBERATELY THE SAME SHAPE AS `odoo_creds` AND SHARES ITS IMPLEMENTATION, because "which stored credential serves this connector?" is ONE question and this module was one copy-paste away from having two answers to it ([[one-question-two-normalizers]]). The difference between the two connectors belongs in the CONNECTOR, not in the lookup. ⚠ AND IT FAILS CLOSED WITH NO ENVIRONMENT FALLBACK, which is where it deliberately DIFFERS from its caller: environment fallback is a Royal-only connector decision, never a keychain lookup decision. `META_ADS_ACCESS_TOKEN` is the OWNER's measurement token (R8), and handing it to another tenant is precisely the leak the R3 rule exists to prevent. A tenant with no entry gets None here. ⭐ W54-T45 / R4 β€” `connection_id` NAMES WHICH Meta connection, and no id means the tenant's DESIGNATED DEFAULT (the lowest-`slot` `meta_ads` connection), not "whichever row sorts first". An id that names no `meta_ads` entry resolves to None, never to a sibling. """ return _creds_for(rt, 'meta_ads', connection_id) def amazon_creds(rt, connection_id=None): """W52: the FIRST `amazon_sp` entry's fields, else None β€” the Amazon connector's resolver. β›” THE SAME SHAPE AND THE SAME IMPLEMENTATION AS `odoo_creds` / `meta_creds`, deliberately. "Which stored credential serves this connector?" is ONE question, and this module has already been one copy-paste away from having several answers to it. What differs between connectors belongs in the connector. ⚠ FAILS CLOSED WITH NO ENVIRONMENT FALLBACK, like Meta and unlike Odoo. `AMAZON_SP_*` in the environment is an OPERATOR's proving credential (`connectors_amazon.py --check`), not a tenant's, and serving it to a workspace whose admin has stored nothing would hand one tenant another's Amazon seller account. `amazon_store.sync` therefore refuses rather than guessing. ⚠ AND IT RETURNS THE WHOLE FIELD SET, not one secret: an Amazon credential is five values (`client_id`, `client_secret`, `refresh_token`, `seller_id`, `marketplace_id`) and the client needs three of them just to mint an access token. ⭐ W54-T45 / R4 β€” `connection_id` NAMES WHICH Amazon connection, and no id means the tenant's DESIGNATED DEFAULT (the lowest-`slot` `amazon_sp` connection), not "whichever row sorts first". An id that names no `amazon_sp` entry resolves to None, never to a sibling. ⚠ `amazon_creds(None)` β€” a MISSING RUNTIME, not a missing id β€” still answers None rather than raising; `verify_amazon` asserts exactly that, and the guard lives in `_creds_for`. """ return _creds_for(rt, 'amazon_sp', connection_id) def amazon_ads_creds(rt, connection_id=None): """S79-T13: the designated `amazon_ads` entry's four fields, else None. No environment fallback.""" return _creds_for(rt, 'amazon_ads', connection_id) #: ⭐⭐ WAVE 32 (R4 / contract C1) β€” THE SCOPE SIDE-BUCKET, READ HERE, WRITTEN NOWHERE IN THIS FILE. #: #: `{entry_id: {"scope": "personal", "owner": "<username>"}}`. A `business` entry writes NO row, so #: an absent row and a `business` row mean the same thing β€” which is why this is a READ DEFAULT and #: never a migration ([[a-migration-that-runs-on-the-next-write]]). #: #: β›” DECLARED TWICE ON PURPOSE, AND GATED. The vocabulary's home is `api/routes_keychain.py` #: (SESSION B's file, C1 is explicit that it is declared there). `core/` may not import from `api/` #: β€” the layer contract points one way β€” so this reads the same store key by name. Two spellings of #: one constant is [[a-constant-two-features-share]], so `verify_api` asserts the two agree, with an #: NC. It is the same posture `store.backend()` takes toward `store_backend.name()`. SCOPE_KEY = 'keychain_scopes' #: The credential types a WHOLE WORKSPACE reads through, which therefore cannot be personal. #: β›” THE ELEVATION THIS PREVENTS, in B's words: `odoo_creds`/`meta_creds` resolve to *a connection #: of that type for the TENANT* (W54-T45: the one a caller NAMES, or the tenant's designated #: default when none is named) β€” that is what spawns every `ut_odoo_*` / `ut_meta_*` grid and #: what every measure column is answered from. So a member storing a personal Odoo key would not get #: "their own Odoo": they would silently become the credential the entire workspace's databases are #: built from. **A credential elevation wearing a scope picker.** #: β›”β›” `amazon_sp` JOINED IN WAVE 52, AND IT IS THE SAME ELEVATION FOR THE SAME REASON. An #: `amazon_sp` credential is what `amazon_store.sync` resolves to, and that sync spawns every #: `ut_amzn_*` grid the workspace reads. A member storing a personal Amazon key would not get #: "their own Amazon": they would become the seller account the entire workspace's Amazon databases #: are built from. The list was written to prevent exactly this and a new tenant-wide connector has #: to join it in the same change that adds the connector, or the guard silently does not cover it. TENANT_WIDE_TYPES = ('odoo', 'meta_ads', 'amazon_sp', 'amazon_ads') def _personal_ids(rt): """Entry ids marked `personal`. Absent bucket β‡’ empty β‡’ today's behaviour, unchanged.""" try: rows = rt.get(SCOPE_KEY) or {} except Exception: # noqa: BLE001 return frozenset() if not isinstance(rows, dict): return frozenset() return frozenset(str(k) for k, v in rows.items() if isinstance(v, dict) and str(v.get('scope') or '') == 'personal') # ═════════════════════════════════════════════════════════════════════════════════════════════ # ⭐⭐ W54-T45 / R4 / CONTRACT C4 β€” EVERY READER NAMES WHICH CONNECTION IT WANTS. # ═════════════════════════════════════════════════════════════════════════════════════════════ # # What stood here was `_first_creds(rt, etype)`, and one line of it β€” "return the first entry of # this type" β€” was the whole ceiling on R4. Storage already held many connections of one kind # (W54-T44 gave each of them an id and a slot); every READER collapsed them back to one. An # agency with two clients on Odoo could store both credentials, see both rows, rename both, and # be served the first one for every question either client asked. That is not a missing feature, # it is R7's isolation rule failing silently: one client's numbers answering another's screen. # # β›” THE HELPER IS NOT NAMED "FIRST" ANY MORE, ON PURPOSE. A shared body called `_first_creds` is # how the defect would grow back one connector at a time: the next resolver copies the name and # inherits the rule. `_creds_for` asks the question the contract asks β€” *which stored credential # serves this connector, for this connection?* # # β›” RESOLVE ON IDS, ORDER ON SLOTS, and the distinction is MEASURED, not stylistic: a pre-wave # row's read-default `slot` MOVES when a sibling of its type is deleted (2 -> 1) while its # `connection_id` does not. So `connection_id` is the ADDRESS a caller passes and holds, and # `slot` is only how the tenant's DESIGNATED DEFAULT is picked and how connections are ordered # on a screen. A caller that stored a slot as an address would eventually address another # client's credential. def default_connection_id(rt, etype): """THE TENANT'S DESIGNATED DEFAULT connection of `etype` β€” its `connection_id`, or None. Never decrypts anything. β›”β›” A DESIGNATED DEFAULT PINNED BY AN IMMUTABLE ORDINAL, NOT "WHICHEVER ROW HAPPENS TO SORT FIRST", AND THAT DISTINCTION IS THE POINT OF THIS WHOLE TICKET. The rule is the LOWEST `slot` β€” the per-(tenant, type) ordinal `add_entry` mints high-water and never rewrites β€” with ties broken by the existing insertion-id order so the answer is total and stable. "First of its type" was an accident of dict ordering that a delete, a re-key or a store rewrite could move under a caller's feet; the lowest slot is a decision the tenant made when it added its first connection. ⚠ Personal entries of a tenant-wide type are skipped exactly as `_creds_for` skips them, so the default and the credential can never disagree about which connection serves. ⚠ AND IT IS A DEFAULT, NEVER A FALLBACK. It answers "no id was named"; it is not consulted when an id WAS named and failed to resolve, because serving a sibling to a caller that asked for a specific client is the leak R7 forbids. """ if rt is None: return None etype = str(etype or '').strip().lower() skip = _personal_ids(rt) if etype in TENANT_WIDE_TYPES else frozenset() best = None for pos, row in enumerate(list_entries(rt)): if row['type'] != etype or str(row['id']) in skip: continue rank = (int(row['slot']), pos) if best is None or rank < best[0]: best = (rank, str(row['connection_id'])) return best[1] if best is not None else None def _row_for(rt, connection_id, etype=None): """The metadata row `connection_id` addresses, or None. Never decrypts anything. Matches on the row's `connection_id` and, for rows written before W54-T44, on its entry id β€” which `connection_id_of` makes the same value anyway. When `etype` is asserted, a row of a DIFFERENT type is None rather than a match: an id is only an address within the connector that was asked for. β›” A personal entry of a tenant-wide type is None here too, even when named EXPLICITLY. The wave-32 skip is a credential-elevation guard, not a tie-breaker, and letting an explicit id walk past it would reopen the door from the one side that stayed shut. """ if rt is None: return None want = str(connection_id or '') if not want: return None etype = str(etype or '').strip().lower() for row in list_entries(rt): if str(row['connection_id']) != want and str(row['id']) != want: continue rtype = str(row['type'] or '') if etype and rtype != etype: return None if rtype in TENANT_WIDE_TYPES and str(row['id']) in _personal_ids(rt): return None return row return None def resolve_connection_id(rt, etype, connection_id=None): """WHICH connection of `etype` would serve β€” an IDENTITY, decrypting nothing. None when nothing would. β›” IT EXISTS SO THERE IS GENUINELY ONE RESOLVER. `harness/runtime.py::odoo_flag_key` needs the identity of the serving connection without opening it (a pause flag must still resolve on a LOCKED keychain), and its docstring has always claimed to be that one resolver while carrying an inline first-entry walk of its own. This is the walk it now calls instead. ⚠ The answer is a `connection_id`, which for every stored row equals that row's entry id (`add_entry` stamps them the same and `connection_id_of` defaults the older rows to it), so the pause flags written against the old resolution keep resolving. """ if connection_id is None: return default_connection_id(rt, etype) row = _row_for(rt, connection_id, etype) return str(row['connection_id']) if row else None def connection_creds(rt, connection_id, etype=None): """THE fields of the ONE connection `connection_id` addresses, decrypted β€” or None. ⭐⭐ W54-T45 / C4 β€” the reader the contract asks for: *"`keychain.odoo_creds` and every other first-entry reader must take a `connection_id`."* Two connections of one kind therefore return two different credential sets for two different ids, which is the agency case R4 names. ⚠ A MISSING RUNTIME ANSWERS None RATHER THAN RAISING, AND THE GUARD IS HERE ON PURPOSE. Every caller's contract is already "the tenant's credential, or None", and `None` means exactly "not configured" everywhere it is consumed β€” but `rt=None` used to reach `rt.get(KEY)` and raise `AttributeError`, so the three resolvers had a documented contract their shared implementation did not keep. Found by `verify_amazon` asserting the contract rather than the behaviour. β›” FIXED HERE AND NOT IN ONE RESOLVER, because this module's own rule is that "which stored credential serves this connector" has ONE implementation: patching `amazon_creds` alone would have created the second normalizer this file warns about. ⭐⭐ WAVE 32 (R4/C1) β€” A PERSONAL ENTRY OF A TENANT-WIDE TYPE IS SKIPPED, NOT SERVED. `routes_keychain` REFUSES to create one, and that closes the door from the only side B owns; this is the other side, and it is the one that matters for anything already stored β€” an entry written before this wave, by an older client, or by any path that does not go through that route. B booked it for A explicitly ("the resolver-side guard is booked for A"). ⚠ SKIP, NOT RAISE. The caller's contract is "the tenant's credential, or None", and None already means "not configured" everywhere it is consumed: `odoo_source` falls back to the environment for tenant #0, and Meta fails closed with no fallback. Raising would turn a mis-scoped entry into an outage for a workspace that has a perfectly good business entry two rows down. ⚠ AND THE SKIP IS SCOPED TO `TENANT_WIDE_TYPES`. A personal entry of any OTHER type is a normal personal credential and this resolver is not how it is reached. ⚠ A LOCKED KEYCHAIN STILL RAISES `KeychainLocked`, from `read_fields`. "No key on this deployment" is an operator fact and silently empty creds would read as "not configured". """ row = _row_for(rt, connection_id, etype) return read_fields(rt, row['id']) if row else None def _creds_for(rt, etype, connection_id=None): """THE ONE INTERNAL HELPER behind `odoo_creds` / `meta_creds` / `amazon_creds`: the fields of the `etype` connection named by `connection_id`, or of the tenant's designated default when no id is named. None when nothing serves. β›” IT REPLACES `_first_creds`, NAME INCLUDED. "Which stored credential serves this connector" has ONE implementation in this module, and the old one answered a question C4 forbids. β›” A NAMED CONNECTION THAT DOES NOT RESOLVE IS None, FULL STOP β€” never the default, never a sibling, and (at the caller) never the environment. Falling back after an explicit ask is how one client's Odoo comes to answer another client's question under R7's strict isolation. """ if rt is None: return None if connection_id is None: connection_id = default_connection_id(rt, etype) if connection_id is None: return None return connection_creds(rt, connection_id, etype) # --------------------------------------------------------------------------------------------- # PER-USER SECRET SLOTS (wave 22, contract C5 β€” the R7 seam adopted from QM) # --------------------------------------------------------------------------------------------- # A connector credential that belongs to a PERSON, not to the tenant: *my* Gmail refresh token, # never "the workspace's". Stored in a SEPARATE sub-bucket (`user_secrets`) beside `entries`, # deliberately: `list_entries` walks `entries` only, so a user's OAuth tokens never show up in # the admin keychain table β€” they are identity, not shared infrastructure β€” and the addition is # migration-free. Each envelope binds the tenant, username and provider when a data key exists. #: Slot names are SHAPE-validated rather than enumerated (wave 22, C5 amendment A2): the OAuth #: provider registry lives API-side and grows one entry per provider β€” a platform-side copy of #: its keys would be the client-union defect one layer down. The shape is the law instead. import re as _re_slots USER_SECRET_SLOT = _re_slots.compile(r'^oauth_[a-z0-9_]{1,24}$') def put_user_secret(rt, username, provider, fields): """Encrypt + store ONE user's credential slot for `provider` (upsert β€” a reconnect replaces). Raises ValueError on shape problems, KeychainLocked when there is no key.""" username = str(username or '').strip() provider = str(provider or '').strip().lower() if not username: raise ValueError('no username for the secret slot') if not USER_SECRET_SLOT.fullmatch(provider): raise ValueError('a secret slot is named oauth_<provider>') if not isinstance(fields, dict) or not fields: raise ValueError('fields must be a non-empty object') clean = {} for k, v in list(fields.items())[:16]: k = str(k).strip()[:40] if k and isinstance(v, (str, int, float, bool)): clean[k] = str(v)[:2000] # a Google refresh token is ~100 chars; JWTs more if not clean: raise ValueError('no usable fields') row = {**seal_fields(_tenant(rt), _user_secret_id(username, provider), clean), 'updated': _dt.datetime.now().strftime('%Y-%m-%dT%H:%M:%S')} def _up(cur): slots = dict(cur.get('user_secrets') or {}) slots[username] = {**(slots.get(username) or {}), provider: row} return {**cur, 'user_secrets': slots} rt.update(KEY, _up, flush='sync') return True def _user_secret_id(username, provider): return json.dumps(['user_secrets', str(username), str(provider).lower()], separators=(',', ':')) def read_user_secret(rt, username, provider): """Decrypt ONE user's slot β€” None when absent; KeychainLocked stays LOUD (a wrong key must never read as 'not connected').""" slot = ((_bucket(rt).get('user_secrets') or {}).get(str(username)) or {}).get( str(provider or '').lower()) if slot is None: return None if not isinstance(slot, dict): raise KeychainEnvelopeError('The stored credential envelope is invalid.') return open_fields(getattr(rt, 'key', None), _user_secret_id(username, provider), slot) def drop_user_secret(rt, username, provider): def _drop(cur): slots = (cur.get('user_secrets') or {}).get(str(username)) if isinstance(slots, dict): slots.pop(str(provider or '').lower(), None) return cur rt.update(KEY, _drop, flush='sync') return True # ═════════════════════════════════════════════════════════════════════════════════════════════ # ⭐⭐ W31 β€” THE LIVENESS-PROBE SLOT, AND WHY IT IS A SLOT AND NOT AN IMPORT. # ═════════════════════════════════════════════════════════════════════════════════════════════ # # `test_entry` below wants to ASK each connector's API whether a stored credential works. The # knowledge of HOW to ask lives in the connector modules β€” and those live in the API layer, which # imports `core`, never the other way round (ARCHITECTURE.md's layering rule, restated in # CLAUDE.md: *"core never imports up"*). A first draft of the Meta arm did the tempting thing β€” # `sys.path.insert` into `aios-web/api` and `import connectors_meta` from inside `core` β€” and that # is an architecture inversion with no gate on it, i.e. the kind that survives. # # β›” THE FIX IS THIS REPO'S OWN IDIOM, NOT A NEW ONE. `harness/datastore.py` has exactly this # problem with the connector-pause flag and solves it with `set_paused_probe(fn)`: the lower layer # exposes a SLOT and the upper layer fills it at import, with the comment *"an import back would # be a cycle"*. Same shape, same reason. `verify_meta` asserts `core/` imports nothing from the # API layer, so the inversion cannot come back quietly. # #: entry type -> `fn(fields: dict, timeout: int) -> {ok, message}`. Filled by the connector layer. _PROBES = {} def register_prober(etype, fn): """Let the connector layer answer `test_entry` for ONE entry type. ⚠ Additive and idempotent by design β€” a re-import must not double-register or raise, because module import order is not something a caller controls. """ etype = str(etype or '').strip().lower() if etype and callable(fn): _PROBES[etype] = fn return etype in _PROBES def entry_verified(rt, entry_id): """Read proof for the current ciphertext without decrypting or making a network call.""" entry = (_bucket(rt).get('entries') or {}).get(str(entry_id)) if not isinstance(entry, dict) or not entry.get('enc'): return False proof = entry.get('probe') return (isinstance(proof, dict) and proof.get('ok') is True and proof.get('encrypted_digest') == hashlib.sha256( str(entry['enc']).encode('utf-8')).hexdigest()) #: S79-T13: the fixed messages per Amazon type. Provider text never crosses this boundary. _AMAZON_TEST_MESSAGES = { 'amazon_sp': ('Amazon confirmed marketplace access.', 'Amazon did not confirm marketplace access. Check the five fields and test again.'), 'amazon_ads': ('Amazon Ads confirmed profile access.', 'Amazon Ads did not confirm profile access. Check the four fields and connect again.'), } def _test_amazon_entry(rt, entry_id, entry, timeout): """Only an explicit test calls the registered probe; storage and listing stay offline. Bind the result to ciphertext so a replacement cannot inherit a previous credential's proof. Neither provider messages nor exception text cross this boundary: both can echo input. """ etype = entry.get('type') if entry.get('type') in _AMAZON_TEST_MESSAGES else 'amazon_sp' passed, failed = _AMAZON_TEST_MESSAGES[etype] result = {'ok': False, 'message': failed} try: fields = read_fields(rt, entry_id) or {} prober = _PROBES.get(etype) got = prober(dict(fields), timeout) if prober is not None else None if isinstance(got, dict) and got.get('ok') is True: result = {'ok': True, 'message': passed} except Exception: # noqa: BLE001 pass def _record(cur): current = (cur.get('entries') or {}).get(str(entry_id)) if not isinstance(current, dict) or current.get('enc') != entry.get('enc'): raise ValueError('The credential changed during the test. Reload and test again.') current['probe'] = { 'ok': result['ok'], 'encrypted_digest': hashlib.sha256(str(entry.get('enc') or '').encode('utf-8')).hexdigest(), } return cur try: rt.update(KEY, _record, flush='sync') except Exception: # noqa: BLE001 return {'ok': False, 'message': 'The test result could not be saved. Reload and test again.'} return result def test_entry(rt, entry_id, timeout=8): """A cheap, READ-ONLY liveness probe. odoo: xmlrpc `common.version()` (no auth, no data); generic: shape-only. Always answers {ok, message} β€” an exception here is a result, not a crash.""" try: e = (_bucket(rt).get('entries') or {}).get(str(entry_id)) if not isinstance(e, dict): return {'ok': False, 'message': 'no such entry'} if e.get('type') in _AMAZON_TEST_MESSAGES: return _test_amazon_entry(rt, entry_id, dict(e), timeout) fields = read_fields(rt, entry_id) or {} # ⭐ W31 β€” A REGISTERED PROBER ANSWERS FOR ITS OWN TYPE, if the connector layer filled the # slot. A shape check says "you pasted something", which is the least useful thing an # admin can be told about a credential: a revoked or wrong-scoped token is the same SHAPE # as a working one, so a connector that can ASK its API should. prober = _PROBES.get(str(e.get('type') or '')) if prober is not None: try: got = prober(dict(fields), timeout) return got if isinstance(got, dict) else {'ok': False, 'message': 'the prober said nothing'} except Exception as ex: # noqa: BLE001 return {'ok': False, 'message': f'{type(ex).__name__}: {str(ex)[:160]}'} if e.get('type') != 'odoo': return {'ok': bool(fields), 'message': f'{len(fields)} field(s) stored'} url = str(fields.get('url') or '').rstrip('/') if not url.startswith(('http://', 'https://')): return {'ok': False, 'message': 'url must start with http(s)://'} import socket import xmlrpc.client old = socket.getdefaulttimeout() socket.setdefaulttimeout(timeout) try: common = xmlrpc.client.ServerProxy(f'{url}/xmlrpc/2/common', allow_none=True) ver = common.version() finally: socket.setdefaulttimeout(old) sv = (ver or {}).get('server_version') if isinstance(ver, dict) else None return {'ok': True, 'message': f'Odoo answered (server {sv or "unknown"})'} except KeychainLocked as e: return {'ok': False, 'message': str(e)} except Exception as e: return {'ok': False, 'message': f'{type(e).__name__}: {str(e)[:120]}'}
"""core/keychain.py β€” per-tenant CREDENTIAL STORE (wave 18, contract C7 / ruling R3).
An admin stores a data source's secrets (Royal's Odoo API key today; any tenant's sources
tomorrow) encrypted at rest in the TENANT's store bucket. The runtime posture is R3's
"live with env fallback": the connector layer asks `odoo_creds(rt, connection_id)` FIRST and falls back to
the process environment (tenant #0's `.env` / Space secrets), so Royal keeps working with an
empty keychain and a new tenant needs no code to bring keys. The FULL cutover (env retired)
is staged for a later wave β€” R3, verbatim.
CRYPTO: v2 uses AES-256-GCM under the tenant's keychain subkey with tenant and entry AAD.
V1 and untagged rows use Fernet under `AIOS_KEYCHAIN_KEY` until migrated.
An unavailable tenant key refuses writes without falling back to Fernet.
SHAPE (bucket `keychain`, per-tenant via the runtime handle):
{entries: {id: {id, connection_id, slot, label, type, enc, envelope_version,
preview, created, createdBy}}}
Amazon rows also carry `client_secret_rotation_deadline`, empty when unknown, and may carry
the ciphertext-bound `probe` proof. `enc` holds base64 ciphertext over the JSON fields dict.
`preview` = a masked hint ("Β·Β·Β·Β·1234")
computed ONCE at write from the entry's most secret-looking field β€” the ONLY part of a
secret that ever leaves this module through a route. `read_fields` exists for the CONNECTOR
LAYER; no route returns its output.
"""
import base64
import datetime as _dt
import hashlib
import json
import os
import secrets as _secrets
from core import key_shapes, tenant_keys
KEY = 'keychain'
ORIGINAL_ENVELOPE_VERSION = 1
ENVELOPE_VERSION = 2
AMAZON_ROTATION_DEADLINE_FIELD = 'client_secret_rotation_deadline'
# Required Amazon fields belong below the clients that consume credentials.
_AMAZON_OAUTH_FIELDS = ('client_id', 'client_secret', 'refresh_token')
AMAZON_CREDENTIAL_FIELDS = {
'amazon_sp': _AMAZON_OAUTH_FIELDS + ('seller_id', 'marketplace_id'),
'amazon_ads': _AMAZON_OAUTH_FIELDS + ('profile_id',),
}
#: ⭐ WAVE 23 (C11/R8) β€” `stripe` and `shopify` joined. They are TOKEN connectors, not OAuth
#: ones, and that is the whole reason they could ship in this wave: both issue a long-lived
#: restricted key / custom-app token an admin pastes in, so they need no OAuth client, no
#: consent screen and no app review β€” the three things that make every other provider on the
#: standing queue a multi-week errand. A type here is what lets the connectors directory show
#: them as CONNECTABLE rather than as another faded to-do.
#: ⭐⭐ WAVE 31 (R2/R5) β€” `meta_ads` JOINS, AND IT IS THE FIRST END-TO-END KEYCHAINβ†’HTTP CONNECTOR.
#: Every type above it is either read by nothing (`generic`, `stripe`, `shopify` store a key that
#: no data path consumes yet) or read by ONE resolver that talks XML-RPC (`odoo`). So `meta_ads`
#: is the first entry whose stored secret actually reaches an HTTP API in production, through
#: `meta_creds` below β€” which is why W31-T48's `how:` says *build the seam, do not assume it*.
#: β›” A TYPE MISSING FROM THIS SET 400s IN `add_entry`, which is the entire failure: the Connectors
#: directory would offer Meta Ads, an admin would paste a token, and the write would be refused
#: with "type must be one of odoo, generic, stripe, shopify" β€” a message about our data structure,
#: on a screen about their advertising account.
#: ⚠ R8 keeps the two doors separate on purpose: this is the TENANT's own token (GTM Lab pastes
#: theirs), while R5's "Connect Facebook" is the delegated-OAuth door Nango owns. Neither replaces
#: the other, and the schema work is blocked on neither.
ENTRY_TYPES = {'odoo', 'generic', 'stripe', 'shopify', 'meta_ads', 'amazon_sp', 'amazon_ads', 'slack'}
#: which stored field feeds the masked preview, per type (first present wins)
#: ⚠ `access_token` JOINED IN WAVE 31, AND ITS ABSENCE WAS A REAL DEFECT FOUND BY BUILDING THE
#: SEAM RATHER THAN BY READING IT. Meta's credential is conventionally an `access_token`, and with
#: only `token` on this list `add_entry` stored the secret correctly and computed an EMPTY preview
#: β€” so the keychain table would show a Meta row with no hint at all, and an admin holding two
#: tokens could not tell which one is stored or whether the paste worked. Nothing raises; the
#: entry is fine; the only symptom is a blank cell on the screen the feature exists for.
#: β›” `refresh_token` JOINED FOR AMAZON SP-API, AND ITS ABSENCE WOULD HAVE BEEN THE WAVE-31
#: DEFECT A SECOND TIME. An `amazon_sp` entry's only secret is its refresh token; with the
#: tuple as it stood, `add_entry` stored the credential correctly and computed an EMPTY
#: preview, so the keychain table would show an Amazon row with no hint at all. Nothing
#: raises; the entry is fine; the only symptom is a blank cell on the screen the feature
#: exists for. Matching is EXACT-KEY (`clean.get(k)` below), so a near-miss name silently
#: yields the same blank.
_PREVIEW_FIELDS = ('api_key', 'access_token', 'value', 'password', 'token', 'refresh_token', 'bot_token')
MAX_ENTRIES = 40
MAX_FIELD_LEN = 500
class KeychainLocked(Exception):
"""A required encryption key or stored ciphertext is unavailable to the reader."""
class KeychainEnvelopeError(ValueError):
"""A stored credential envelope is malformed or unsupported."""
class KeychainEnvelopeVersionError(KeychainEnvelopeError):
"""An unsupported envelope is distinct from an unavailable encryption key."""
def _check_envelope_version(row):
# Untagged rows use the original Fernet envelope without rewriting storage.
version = row.get('envelope_version', ORIGINAL_ENVELOPE_VERSION)
if type(version) is not int or version not in (ORIGINAL_ENVELOPE_VERSION, ENVELOPE_VERSION):
raise KeychainEnvelopeVersionError('This credential uses an unsupported encryption envelope version.')
return version