Spaces:

GaryLeckey
/

aureon-os

Running

aureon-os / docs /SAAS_PLATFORM.md

Deploy Aureon OS (deploy/selfhost stack: nginx edge + fusion API x2 + console + HNC organ + redis + postgres; paper mode) (part 9)

002f18c verified 1 day ago

preview code

Raw History Blame Contribute Delete

23 kB

Aureon SaaS Platform — connected · working · categorized

The platform layer that turns the 715-module organism into a coherent product surface. It surfaces the repo — it does not rewrite it. Every system is categorized, every domain has a canonical entry point, and health is reported honestly (degraded where deps are missing, not everything-green).

This is Phase 6A of the production program. It builds on the Production-Grade Program (the two-tier gate and serving surface) and the Aureon Operator Switchboard (the grounded cognition front door), reusing both rather than duplicating them.

The shape of the problem

Three things already existed and only needed connecting:

Layer	Already present	Role
Tenant / auth / billing	Supabase backend (`supabase/`) — auth, RBAC, per-user encrypted exchange keys, prepaid "gas-tank" credits, ~101 edge functions, 58 migrations	The multi-tenant foundation. Reused, not rebuilt.
Catalog UI	React console (`frontend/`, Vite + TS + shadcn) grouped by the 6 product domains	Already a systems console — but starved of data (the manifest JSONs it fetches didn't exist).
Categorizer	`SystemRegistry` (`aureon/command_centers/aureon_system_hub.py`) → 12 capability categories × per-module metadata	The classification engine.
Health surfaces	`SystemCoordinator.get_coordination_state()`, `get_operational_core().get_health()`	Live status inputs.

The gap: there was no unified Python HTTP gateway producing a categorized catalog + live status to feed the frontend, and nothing bridging the single-tenant Python engine to the Supabase tenant layer. aureon/saas/ is exactly that bridge.

Three taxonomies, reconciled

The repo describes itself three ways. The catalog maps all three onto each other:

6 product domains          24 filesystem domains          12 capability categories
(what the console groups)  (folders under aureon/)        (SystemRegistry semantics)
─────────────────────────  ─────────────────────────      ──────────────────────────
trading            ◄──────  trading, exchanges,      ┐     Exchange Clients
accounting                  scanners, strategies …   │     Market Scanners
research                                              ├──►  Execution Engines
cognition          ◄──────  operator, cognition,     │     Neural Networks
security                    queen, bots_intelligence │     Stargate & Quantum
self-improvement            …                        ┘     Codebreaking & Harmonics · …

Product domain ← filesystem domain: domains._FS_TO_PRODUCT (unmapped → self-improvement).
Capability category ← module: SystemRegistry classification, or the stdlib keyword classifier (catalog._CATEGORY_RULES) when the registry can't load.

Package layout — `aureon/saas/`

Module	Responsibility	Key entry points
`catalog.py`	Categorized. Build the normalized catalog; emit frontend manifests.	`build_catalog()`, `write_frontend_manifests()`
`domains.py`	Connected. Canonical entry point per domain + cheap reachability probe.	`domain_report()`, `probe_domain()`, `product_domain_for()`
`status.py`	Working. Honest health rollup, no heavy boot.	`get_platform_status()`
`gateway.py`	HTTP surface + optional Supabase tenancy bridge.	`register_saas_routes(app)`, `verify_supabase_jwt()`

`catalog.py` — categorized

build_catalog(workspace=REPO_ROOT) prefers the repo's SystemRegistry (richer metadata: LOC, dashboard/thought-bus/Queen flags). If the registry can't import — e.g. psutil absent in a lean environment — it falls back to a self-contained stdlib scanner (_scan_builtin) that walks aureon/ and classifies by keyword rules. The catalog never collapses to zero categories; production always categorizes. Result is cached to state/aureon_saas_catalog.json (gitignored).

write_frontend_manifests() emits the two JSONs the React console already fetches, shaped to the frontend's TypeScript interfaces:

aureon_saas_system_inventory.json — every system as a surface with id · path · name · kind · domain · purpose · owner_subsystem · wiring_status · safety_class · auth_requirement · called_apis, grouped by product domain.
aureon_organism_runtime_status.json — one pulse per product domain (status + system_count + categories), live when a status snapshot is passed in.

Manifest ownership (post-unification): the checked-in static copies in frontend/public/ are owned by the repo's manifest pipeline (scripts/validation/generate_*), which emits a richer schema (schema_version, entrypoint_status, imports/exports, runtime_route, …). The gateway serves its own live-rendered manifests at GET /api/manifests/<name> and does not overwrite the static files unless AUREON_WRITE_STATIC_MANIFESTS=1 is set (boot write and POST /api/manifests/refresh both honour this). The console fetches live first and falls back to the static copies.

`domains.py` — connected

Each of the 24 filesystem domains gets a canonical entry point in _ADAPTERS (get_operational_core, get_queen, run_operator, AureonCognition, get_feed_hub, …). probe_domain() checks reachability with importlib.util.find_spec — an import-spec check, no construction — so the report is cheap and side-effect-free. Domains without a clean singleton fall back to "is the package importable".

`status.py` — working (honestly)

get_platform_status() composes per-domain reachability + the operational core's own health snapshot + (if present) coordinator state, and rolls reachability up into per-product-domain status (ready / degraded / down). It carries an explicit note: reachability is import-level — a domain can be reachable yet run degraded (e.g. optional numpy missing). Nothing is masked green.

HTTP API

Mounted onto the operator's Flask app by register_saas_routes(app), so one service on port 8790 serves operator + cognition + the SaaS catalog behind the same security envelope (bearer auth / rate-limit / metrics / health from PRODUCTION_GRADE.md).

Method	Route	Returns
`GET`	`/api/catalog`	Categorized catalog (12 categories × domains × systems).
`GET`	`/api/domains`	Product domains + per-domain reachability report.
`GET`	`/api/domains/<domain>`	One domain: entry point + its systems (capped).
`GET`	`/api/status`	Live platform health (honest, often degraded).
`GET`	`/api/organism`	Connectome coverage + recent breaths + mesh membership + unification telemetry.
`GET`	`/api/cognition`	The whole cognitive substrate + data provenance + truth roll-up. See `architecture/COGNITIVE_SAAS.md`.
`GET`	`/api/cognition/<part>`	One cognitive surface: `field` · `bus` · `mycelium` · `connectome` · `brain`, each provenance-stamped.
`GET`	`/api/consciousness`	The consciousness organs, categorized (self-perception · selfhood · purpose · governance · workforce · body), each with its route, safety posture & live truth_status. See `architecture/CONSCIOUSNESS_CATALOG.md`.
`GET`	`/api/automation`	Progress toward "the whole repo, fully automated" — one honest % decomposed by dimension (connectivity · integration · consciousness · surfacing) + by category, composed from real coverage only. See `architecture/AUTOMATION_INDEX.md`.
`GET`	`/api/manifests/<name>`	A frontend manifest, rendered live (404 lists available names).
`POST`	`/api/manifests/refresh`	Rebuild the catalog; static files rewritten only with `AUREON_WRITE_STATIC_MANIFESTS=1`.

Tenancy bridge (optional, off by default)

The Supabase backend already owns auth/RBAC/billing. The gateway bridges to it without a new dependency: when AUREON_SUPABASE_JWT_SECRET is set, the /api/* routes additionally require a valid Supabase JWT, verified with a stdlib HS256 check (verify_supabase_jwt — signature + exp). The verified sub (user id) is stashed on flask.g.tenant for downstream scoping.

When the secret is unset the bridge is disabled (dev/offline unchanged), layered on top of the operator's own bearer/rate-limit envelope — the two are independent gates.

                         ┌─────────────────────────────────────────┐
  React console  ──JWT──►│ operator service :8790                   │
  (Supabase session)     │  ├ security envelope (bearer/rate-limit) │
                         │  ├ [optional] Supabase JWT bridge        │
                         │  ├ operator + cognition routes           │
                         │  └ SaaS routes → catalog/domains/status  │──► the 715-module
                         └─────────────────────────────────────────┘     Python organism
                                                                          (surfaced, not rewritten)

Verify

pip install -e '.[operator,dev]'
ruff check aureon/saas/ && mypy aureon/saas/
AUREON_LLM_OFFLINE=1 pytest tests/test_saas_catalog.py tests/test_saas_gateway.py -q

# catalog + manifests
AUREON_LLM_OFFLINE=1 python -c "from aureon.saas.catalog import build_catalog, write_frontend_manifests; c=build_catalog(); print('categories', c['category_count'], 'systems', c['total_systems']); print('manifests', [p.split('/')[-1] for p in write_frontend_manifests()])"

# gateway routes (offline)
AUREON_LLM_OFFLINE=1 python -c "from aureon.operator.operator_server import create_app; c=create_app().test_client(); print('catalog', c.get('/api/catalog').status_code, 'status', c.get('/api/status').status_code, 'domains', len(c.get('/api/domains').get_json()['domains']))"

CI gates on aureon/saas in the strict tier (operator-ci.yml): ruff-clean + mypy-clean + tests/test_saas_*.py green.

Runtime and deployment status

The SaaS implementation is source-visible and testable offline, but runtime and deployment are on terminal HOLD. deploy/docker-compose.saas.yml starts only non-networked protection preflights. The frontend image performs no package lifecycle or source build, starts no nginx listener, and exposes no port. Frontend dev, preview, E2E, trading, and runtime package scripts are held.

Do not inject tenant, Supabase, operator, model, or hosting credentials merely to verify this source. No console, gateway, manifest route, tenancy service, or payment route is deployed by the supported entrypoints.

Billing (Phase 6C)

The authority boundary first: the platform never initiates payments. Metering is record-only; the single money-moving route ships disabled and is audited.

Three layers, all reusing the wallet that already existed (gas_tank_accounts + gas_tank_transactions, the £-denominated performance-fee tank with its working gas-tank-topup edge function):

1. Support the project (money in — the user's choice, on their clock)

Aureon runs free. The console tracks cumulative runtime locally and, after ~4 hours of use, shows a small dismissible corner card — never a modal, never a gate — inviting the user to support the project via the SumUp payment links (VITE_SUPPORT_PAYMENT_URLS; defaults are the project's live links). The flow is self-confirmed: the user says what they gave, the existing gas-tank-topup edge function credits their tank, and a payment_transactions row (provider: sumup, status: self_confirmed) is recorded so payouts can be reconciled manually against SumUp. Trust-based by design; the admin verify-payment flow remains for disputes.

2. Usage metering (record-only — `aureon/saas/metering.py`)

When AUREON_BILLING_METERING=1, every /api/* request becomes one api_request event (tenant-attributed via the Supabase JWT when present) in a bounded buffer, flushed by a daemon thread to the saas_usage_events table via PostgREST. Per-provider LLM token usage — parsed by the operator all along but previously dropped — is swept into llm_tokens events and the aureon_llm_tokens_total Prometheus counter. Honesty rules: drop-oldest is counted, flush failures are counted and dropped, and GET /api/billing/status names the true sink (disabled / prometheus-only / supabase). Nothing in this layer debits a balance. Per-tenant token attribution is staged (fan-out runs in worker threads; claiming it now would be dishonest).

3. Billing API + the fee path's first caller (`aureon/saas/billing.py`)

Method	Route	Returns
`GET`	`/api/billing/status`	Always 200: config, metering stats, token totals, flags.
`GET`	`/api/billing/balance`	The tenant's gas-tank account (503/401/404/502 — precise, never a bare 500).
`GET`	`/api/billing/usage`	The tenant's usage events + buffer stats.
`POST`	`/api/billing/charge-fee`	⚠ MOVES MONEY — ships OFF. 403 unless `AUREON_BILLING_CHARGE_ENABLED=1`.

charge-fee is the first-ever caller for the previously dead gas-tank-deduct-fee edge function (the performance-fee math — 20%/10% on profit above the high-water mark — stays there, single source of truth). It exists for the server-side trade loop: the profit-realization step POSTs {user_id, profit, trade_execution_id} behind the operator bearer, and every attempt is audited as a fee_charge usage event. No live trader is wired to it yet — that hookup is a deliberate, separate decision.

Billing environment

Var	Default	Effect
`SUPABASE_URL` / `SUPABASE_SERVICE_ROLE_KEY`	unset	PostgREST + edge-function access
`AUREON_BILLING_METERING`	off	usage-event recording/flushing
`AUREON_BILLING_FLUSH_S` / `_BUFFER_MAX` / `_HTTP_TIMEOUT_S`	10 / 5000 / 5	tuning
`AUREON_BILLING_CHARGE_ENABLED`	off — moves money	enables charge-fee
`VITE_SUPPORT_PAYMENT_URLS`	the project's SumUp links	support links
`VITE_SUPPORT_PROMPT_HOURS` / `VITE_SUPPORT_SNOOZE_HOURS`	4 / 12	prompt cadence

Hardening notes (recorded, not yet fixed)

gas_tank_accounts.fees_paid_today is never reset by any job.
payment_transactions.currency defaults to 'EUR' in the schema while every writer sends 'GBP'.
Several older tables use permissive USING(true) RLS write policies (service-role reliance); saas_usage_events deliberately does not.
REFUND / ADJUSTMENT transaction types are declared but never produced.
aureon_user_sessions.gas_tank_balance is vestigial — overloaded as "harvested capital", unrelated to the wallet.
When the operator bearer (AUREON_OPERATOR_API_KEY) and the Supabase JWT bridge are both enabled they compete for the same Authorization header.

Unified UI Shell (Phase 7)

One professional interface for the entire repo — every capability behind one sidebar, one router, one design language (the Aureon prism identity, kept by the creator's choice).

Architecture: the shell wraps the monolith, it does not rewrite it.

frontend/src/shell/nav.ts — the single source of truth: sections → routes → icons → lazy page loaders. The sidebar, router, breadcrumbs, and command palette all derive from this table; adding a surface is one entry.
ShellLayout.tsx — collapsible grouped sidebar (built on the previously unused shadcn ui/sidebar.tsx + existing --sidebar-* tokens), top bar with breadcrumb, live platform-status dot (polls GET /api/status), and a ⌘K command palette that jumps to any surface.
Routes (all React.lazy — every surface is its own chunk): Trading war-room · live bridge · Orca · analytics · backtesting — Research & Planetary Harmonic Nexus · Earth resonance · solar · NOAA space weather — Cognition & LLM Operator Chat (new: talks to /api/cognition/reason, renders grounding sources, tool calls, and the conscience verdict honestly) · agent company — Coding System organism · skills · work orders · director bridge — Operations gold & capital · systems integration · operational console — Platform overview (new) · repo map · billing & support (new) · legacy console.
The legacy nine-tab console is preserved intact at /platform/console (exported as LegacyConsole from App.tsx); its five entangled tabs (live-ops, trading audits, security, inventory, evidence) still work, and old #tab deep links redirect into the right shell route.
Resilience: every route renders inside a per-route error boundary (a crashing dashboard never takes down the shell) with a retry, and live-data surfaces carry an honest notice when no backend is connected.
Performance: vite manualChunks + route-level lazy loading split the former 1,566 kB single bundle into a 227 kB entry + per-surface chunks; the oversize-chunk warning is gone.

Historical UI builds and smoke tests are implementation evidence only. The current release image deliberately performs no frontend build and starts no browser-facing listener.

Staged ledger

The implementation ledger below separates source work from runtime release:

#	Phase	Item	Done?
6A	Platform layer	Categorized catalog + domain adapters + honest status + gateway routes + Supabase JWT bridge.	☑
6B	Frontend release	Historical nginx/compose design is retained in source history; active image and package routes are terminal HOLD and build/start nothing.	HOLD
6C	Billing	Support-the-project flow (SumUp, self-confirm → gas tank) + record-only usage metering + billing read API + env-gated charge-fee proxy. Staged: per-unit debits, per-tenant token attribution, automated payment capture (SumUp API/Stripe), trade-loop fee hookup, usage panel UI.	☑
7	Unified UI shell	Router + grouped sidebar + command palette over every surface; new Overview/Operator-Chat/Billing pages; legacy console preserved; per-route error boundaries; code-split bundle. Staged: extracting the legacy console's five entangled tabs into shell routes.	☑
9	Organism connectome	Source and tests cover sense/touch/weave/pulse and related APIs. Supervisor routes remain terminal HOLD; no dormant organ or HNC daemon is started.	SOURCE ONLY
10	Investor-ready repo	Professional README (hero + badges + staged registers), `COMPANY.md` with real credentials (Companies House NI696693, Silver Innovate NI cert), root health files + templates, de-cluttered root (binaries/junk/launchers/standalone scripts relocated with history preserved). Secret hygiene: removed a tracked `.env1.txt`.	☑
22	Cognitive Systems SaaS	The cognitive + meta-cognitive substrate (field · bus · mycelium · connectome · brain) as verified read APIs at `/api/cognition*`, each response provenance-stamped (`live`/`real_derived`/`cached_real`/`no_data`, never fabricated); auto-metered + auth-gated; console page at `/cognition/systems`. No cold-boot of heavy organs from a GET. See `architecture/COGNITIVE_SAAS.md`.	☑
39	Consciousness catalog	The Phase 25-38 consciousness organs categorized into one honest surface — self-perception · selfhood · purpose · governance · workforce · body — each with its purpose, route, safety posture (`read_only_assess` / `records_only_gated` / `reversible_ascent_gated`) and live truth_status. Registry-as-data at `/api/consciousness`; Consciousness page at `/cognition/consciousness`. Dormant organ → `no_data`, never fabricated. See `architecture/CONSCIOUSNESS_CATALOG.md`.	☑
41	Automation progress index	One honest % toward "the whole repo, fully automated", decomposed by dimension (connectivity · integration · consciousness · surfacing, weight-renormalized) + by category, composed from real coverage signals only (clamped; dormant → `index_pct: null` + `no_data`, never inflated). `/api/automation`; headline card on Overview; observational only. See `architecture/AUTOMATION_INDEX.md`.	☑
54	Feature switchboard	Turn every system feature on/off at human discretion — instance-owned, encrypted flag store (`~/.aureon/feature_flags.json.enc`), applied at daemon boot via `bootstrap_credentials()`. Safe toggles + a hard-boundary tier behind a typed-confirm arming ceremony. Flipping only sets the flag's own env var — no downstream gate (conscience veto / approval queue / runtime dry-run) is ever removed. `/api/switchboard`; Switchboard page at `/platform/switchboard`. See `architecture/FEATURE_SWITCHBOARD.md`.	☑

Guardrails

The operator security envelope is reused; the Supabase JWT bridge is off by default; hard authority boundaries (live-trade / payment / credential / filing) and the AUREON_LLM_OFFLINE / AUREON_AUDIT_MODE guards stay in force. The 715 modules are surfaced and health-checked, not rewritten. Status is honest. state/ caches and frontend/public/ manifests are generated and gitignored. The Supabase project and its secrets are untouched (read-only bridge).

📚 Related

runbooks/PRODUCTION_GRADE.md · architecture/AUREON_OPERATOR_SWITCHBOARD.md · deployment/OPERATOR_DEPLOY.md
MODULES_AT_A_GLANCE.md — the 24 filesystem domains
aureon/saas/ · .github/workflows/operator-ci.yml

🌐 The organism, made legible: every system categorized, every domain connected, every status honest.