|
Download cloud-forge/docs/CLOUD_ARCHITECTURE.md from Snapkitty/tlm-jxcl-forge: direct link, hf CLI and curl.
- Browser
- Download file 67 kB
-
https://huggingface.co/Snapkitty/tlm-jxcl-forge/resolve/main/cloud-forge/docs/CLOUD_ARCHITECTURE.md
- Command line
-
hf download hf://Snapkitty/tlm-jxcl-forge/cloud-forge/docs/CLOUD_ARCHITECTURE.md
-
curl -L -o CLOUD_ARCHITECTURE.md https://huggingface.co/Snapkitty/tlm-jxcl-forge/resolve/main/cloud-forge/docs/CLOUD_ARCHITECTURE.md
67 kB
| # cloud-forge architecture | |
| ## The non-negotiable rule | |
| > Do not create one crate per AWS service. Build the primitives once, | |
| > then compose services from those primitives. | |
| `cloud-forge` is a from-first-principles attempt at the substrate | |
| underneath an AWS-shaped cloud platform: resource identity, lifecycle, | |
| ownership, policy, tagging, events, and quota — the handful of concepts | |
| that every cloud service (compute, storage, database, messaging, …) | |
| reuses rather than reinvents. No crate in this workspace is named after | |
| an AWS product, and none will be until it is a thin composition of | |
| already-real primitive crates, not a container for logic that belongs | |
| in one of them. | |
| This mandate carries the same weight as the root workspace's "100 | |
| crates, none of them fake" rule (see the repository root | |
| [`docs/CRATE_ARCHITECTURE.md`](../../docs/CRATE_ARCHITECTURE.md)) and | |
| `verification-forge`'s hard invariants: a crate exists here only when | |
| it owns a real, independently-testable invariant. | |
| ## Why a third workspace | |
| `tlm-jxcl-forge` already separates two independent Cargo workspaces: | |
| the root workspace (the TLM JXCL ISA, post-quantum crypto/storage, one | |
| reference service) and `verification-forge` (a from-scratch proof | |
| kernel). Neither is a natural home for cloud-resource modeling: | |
| - The root workspace's `docs/crates.toml` registry is machine-checked | |
| at exactly 100 crates with its own dependency-DAG rules | |
| (`tools/check_workspace.py`); folding an open-ended, many-phase cloud | |
| platform into it would either blow past that validated count or | |
| force cloud crates to misrepresent themselves as ISA/crypto crates. | |
| - `verification-forge` is a general-purpose proof kernel with no | |
| cloud-specific concept in it at all — it will eventually *verify* | |
| invariants about `cloud-forge` (see "Relationship to | |
| verification-forge" below), which means it must stay independent of | |
| it, not host it. | |
| So `cloud-forge/` is a third, independent top-level Cargo workspace, | |
| with its own `Cargo.toml`, `crates/`, and quality gates — exactly the | |
| same shape as `verification-forge/`. | |
| ## Layering | |
| ```mermaid | |
| flowchart TB | |
| subgraph existing["Already real, in the root workspace"] | |
| isa["jxcl* — ISA, execution engine"] | |
| pq["pq* — PQ crypto, storage, ZK proofs"] | |
| end | |
| subgraph kernel["cloud-forge Phase 1: the primitive kernel (this pass)"] | |
| types["cloud-types<br/>ResourceId, Arn, AccountId, RegionId"] | |
| errors["cloud-errors<br/>shared CloudError"] | |
| tags["cloud-tags"] | |
| lifecycle["cloud-lifecycle<br/>state machine"] | |
| resource["cloud-resource<br/>Resource<T> wrapper"] | |
| region["cloud-region"] | |
| account["cloud-account"] | |
| identity["cloud-identity<br/>Principal (minimal)"] | |
| policy["cloud-policy<br/>Allow/Deny, deny dominates"] | |
| events["cloud-events"] | |
| quota["cloud-quota"] | |
| core["cloud-core<br/>facade"] | |
| types --> errors | |
| errors --> tags | |
| errors --> lifecycle | |
| types --> resource | |
| tags --> resource | |
| lifecycle --> resource | |
| types --> region | |
| types --> account | |
| types --> identity | |
| identity --> policy | |
| types --> policy | |
| errors --> policy | |
| types --> events | |
| types --> quota | |
| resource --> core | |
| region --> core | |
| account --> core | |
| identity --> core | |
| policy --> core | |
| events --> core | |
| quota --> core | |
| end | |
| subgraph future["Planned, later phases — not built yet"] | |
| control["Phase 2: control plane<br/>(provisioning, reconciliation, scheduler)"] | |
| services["Phase 4+: service layer<br/>(compute, storage, database, …)"] | |
| end | |
| existing -.underlies.-> kernel | |
| kernel --> future | |
| ``` | |
| Every arrow above is a real `[dependencies]` edge in a crate's own | |
| `Cargo.toml` — there is no crate in this phase that depends on | |
| something not yet built, and no crate that exists only to be depended | |
| on later without doing anything itself today. | |
| ## The resource model (Phase 0 / Phase 1) | |
| Every cloud resource, regardless of which future service creates it, | |
| carries the same fields — this is the one model every service composes | |
| from rather than redefining: | |
| | Field | Type | Owning crate | | |
| |---|---|---| | |
| | `id` | `ResourceId` | `cloud-types` | | |
| | `resource_type` | `ResourceType` | `cloud-types` | | |
| | `region` | `RegionId` | `cloud-types` / `cloud-region` | | |
| | `account` | `AccountId` | `cloud-types` / `cloud-account` | | |
| | `owner` | `AccountId` | `cloud-types` | | |
| | `lifecycle` | `Lifecycle` | `cloud-lifecycle` | | |
| | `version` | `u64` | `cloud-resource` | | |
| | `tags` | `Tags` | `cloud-tags` | | |
| | `created_at` / `updated_at` | `u64` (unix millis) | `cloud-resource` | | |
| `owner` here means *account ownership* (who a resource bills to and | |
| belongs to) — distinct from `cloud-identity::Principal`, which | |
| `cloud-policy` evaluates access for. A resource-based policy attaching | |
| a `Principal` to a specific resource is a Phase 13 (IAM) concept, not | |
| pulled forward into this phase's `Resource<T>`. | |
| `cloud-resource::Resource<T>` combines all of these plus a | |
| service-specific payload `T`, so a future `Instance` or `Bucket` type | |
| is `Resource<InstanceSpec>` / `Resource<BucketSpec>`, not a | |
| hand-rolled struct that has to remember to include `tags` and | |
| `lifecycle` correctly on its own. | |
| The internal ARN format `cloud-types::Arn` implements is exactly the | |
| one specified in the roadmap: | |
| ``` | |
| jxcl:cloud:<partition>:<service>:<region>:<account>:<resource> | |
| ``` | |
| ## What Phase 1 deliberately does not include | |
| - **`cloud-metadata` is not a separate crate.** Its proposed fields | |
| (`created_at`/`updated_at`/tags/version) are exactly `cloud-resource` | |
| and `cloud-tags`'s fields; a separate crate would own nothing a | |
| caller couldn't already get from those two. | |
| - **`cloud-runtime` is deferred past Phase 2 too, now to Phase 4 | |
| (compute primitives).** Phase 2 does add a scheduler, but what it | |
| schedules is *placement* (which `AzId` a resource is assigned to) — | |
| a decision `cloud-scheduler` owns directly using `cloud-types`'s | |
| existing `AzId`, with no new wrapper type needed. A "runtime" only | |
| becomes a real, distinct primitive once something is actually | |
| executing (a VM, a container, a function invocation) — Phase 4's | |
| concern, not Phase 2's. | |
| - **`cloud-identity` is intentionally minimal in this phase**: a | |
| `Principal` enum (`User`/`Role`/`Service`, each carrying an id) — | |
| enough for `cloud-policy` to evaluate against. The full IAM surface | |
| (credentials, sessions, federation, policy documents-as-JSON) is | |
| Phase 13 in the roadmap and is not pulled forward. | |
| - **No AWS-named crate exists anywhere in this phase.** `services/ec2`, | |
| `services/s3`, etc. cannot be started honestly until the primitives | |
| they'd compose (compute, storage, network primitives — later phases) | |
| are themselves real. | |
| ## Phase 2: the control plane | |
| Phase 2 is where Phase 1's primitives stop being independent data | |
| types and start composing into an actual pipeline: a request to create | |
| a resource gets authorized, checked against quota, placed somewhere, | |
| constructed, and recorded — with the whole operation succeeding | |
| atomically or rolling back cleanly. This is also where the roadmap's | |
| deterministic state machine | |
| ``` | |
| REQUEST -> AUTHENTICATE -> AUTHORIZE -> VALIDATE -> PLAN -> APPLY -> VERIFY -> COMMIT -> AUDIT | |
| ``` | |
| becomes a real, tested code path rather than a diagram. (`AUTHENTICATE` | |
| is not separately modeled here: this phase takes an already-resolved | |
| `Principal` as input, since credential/session verification is Phase | |
| 13's IAM surface, not Phase 2's control-plane concern.) | |
| ### What's real in Phase 2 | |
| | Crate | Owns | | |
| |---|---| | |
| | `cloud-scheduler` | Placement: picks the least-loaded `AzId` in a region, from `cloud-region`'s registry and a caller-supplied usage count per AZ | | |
| | `cloud-service-registry` | A registry of named control-plane services and their health (`Healthy`/`Unhealthy`/`Unknown`) | | |
| | `cloud-reconciler` | Given a current and a desired `Lifecycle`, computes the shortest valid transition path through `cloud-lifecycle`'s state graph (via BFS), or reports the desired state as unreachable | | |
| | `cloud-provisioner` | The `AUTHORIZE -> VALIDATE -> PLAN -> APPLY -> VERIFY -> COMMIT -> AUDIT` pipeline itself, composing `cloud-policy` + `cloud-quota` + `cloud-scheduler` + `cloud-resource` + `cloud-events`, with each stage's failure distinguishable and every reservation made by an earlier stage rolled back if a later stage fails | | |
| | `cloud-control-plane` | `ControlPlane<T>`: holds the account/region registries, policy, quota tracker, event log, and a resource store; exposes `create` (via `cloud-provisioner`), `get`, `list`, `delete` | | |
| ### What Phase 2 deliberately skips or defers | |
| - **`resource-manager`, `lifecycle-manager`, `quota-manager`, | |
| `policy-engine`, `account-manager`, `region-manager` are not separate | |
| crates.** Each would be a thin wrapper doing nothing but forwarding | |
| to `cloud-resource`, `cloud-lifecycle`, `cloud-quota`, `cloud-policy`, | |
| `cloud-account`, or `cloud-region` — exactly the one-crate-per-name | |
| anti-pattern this workspace's non-negotiable rule exists to prevent. | |
| `cloud-control-plane` holds these Phase 1 primitives directly instead | |
| of through an extra indirection layer that owns nothing new. | |
| - **`control-api` (a REST/gRPC layer) is deferred.** The roadmap's own | |
| execution order puts the API layer (Phase 30) well after the control | |
| plane it exposes; building an HTTP surface before the control plane | |
| it would serve is stable is building a service, not a primitive. | |
| - **`cloud-runtime` is deferred to Phase 4** — see above. | |
| ### The rollback invariant | |
| `cloud-provisioner`'s pipeline makes a real claim worth stating | |
| explicitly: if `AUTHORIZE` succeeds but a later stage (`VALIDATE`, | |
| `PLAN`, `APPLY`, or `COMMIT`) fails, every resource reservation an | |
| earlier stage made (currently: `cloud-quota` usage) is released before | |
| the pipeline returns its error. This is tested directly — not inferred | |
| from "the individual stages are each atomic" — since a multi-stage | |
| pipeline built from atomic parts is not itself atomic unless someone | |
| wires the rollback path and verifies it. | |
| ## Phase 3: AWS-like resource identification | |
| The roadmap names four crates for this phase: `cloud-arn`, | |
| `cloud-resource-id`, `cloud-resource-parser`, `cloud-resource-registry`. | |
| Three of the four already exist — just earlier than the roadmap's own | |
| ordering expected, because the resource model itself needed them from | |
| day one: | |
| | Roadmap name | Where it actually lives | Since | | |
| |---|---|---| | |
| | `cloud-arn` | `cloud_types::Arn` — parses and formats `jxcl:cloud:<partition>:<service>:<region>:<account>:<resource>` | Phase 1 | | |
| | `cloud-resource-id` | `cloud_types::ResourceId` | Phase 1 | | |
| | `cloud-resource-parser` | `Arn`'s own `FromStr` impl — a separate parser crate would parse into the exact same `Arn` struct `cloud-types` already owns, with nothing new to own | Phase 1 (not a separate crate — see below) | | |
| | `cloud-resource-registry` | **New this phase** | Phase 3 | | |
| `Resource<T>` (Phase 1) has an id, a type, a region, and an owning | |
| account, but nothing before Phase 3 ever assembles those into the | |
| *canonical, fully-qualified name* for a provisioned resource, or lets | |
| anyone look a resource up by that name rather than by its bare | |
| `ResourceId`. That's what this phase adds: | |
| - **`cloud-resource-registry`** owns exactly one thing: an `Arn` ↔ | |
| `ResourceId` mapping. It rejects a second registration of an | |
| already-used ARN, and (a stricter, deliberate choice beyond what the | |
| roadmap specifies) rejects registering the same resource under a | |
| second ARN too — a provisioned resource's canonical name is chosen | |
| once, at creation, and does not change. | |
| - **`cloud-control-plane`** now knows its own `partition` and | |
| `service` (constructor arguments), and computes each resource's | |
| `Arn` from `partition` + `service` + the resource's own | |
| region/account/id at `create()` time, registering it in an | |
| internally-owned `ResourceRegistry` — with the same rollback | |
| discipline as every other `create()` failure mode: if ARN | |
| construction or registration fails, the quota reservation and | |
| placement `create()` already made through `cloud-provisioner` are | |
| released before the error returns, exactly like an `APPLY` failure. | |
| `resolve_arn()` and `arn_of()` expose the new lookup direction. | |
| `cloud-provisioner` itself is untouched by this phase: ARN identity is | |
| a *naming* concern that belongs to whatever owns a `partition`/ | |
| `service` (the control plane), not to the provisioning pipeline, which | |
| stays deliberately ignorant of naming schemes. | |
| ## Phase 4: compute primitives | |
| Phase 1 deferred `cloud-runtime` twice (first past Phase 1, then again | |
| past Phase 2) on the same stated grounds: "a runtime only becomes a | |
| real, distinct primitive once something is actually executing." This | |
| phase is where that becomes true, and where the two other primitives a | |
| compute-shaped resource needs alongside a runtime -- a supply of | |
| capacity to run on, and something to boot from -- are built: | |
| | Crate | Owns | | |
| |---|---| | |
| | [`cloud-runtime`](./crates/cloud-runtime) | `RuntimeState`: the execution-state machine (`Pending`/`Running`/`Stopping`/`Stopped`/`Terminating`/`Terminated`) for something actually running, kept deliberately separate from `cloud-lifecycle::Lifecycle` (see below) | | |
| | [`cloud-capacity`](./crates/cloud-capacity) | Per-AZ vCPU/memory-MiB capacity: reservations fail closed against an AZ's registered total | | |
| | [`cloud-image`](./crates/cloud-image) | A registry of machine images to launch from: validated id, name, size, architecture; immutable once registered | | |
| ### `cloud-runtime` vs. `cloud-lifecycle` | |
| These are not the same state machine wearing two names. `Lifecycle` | |
| answers "does this resource *record* exist, and in what stage of | |
| being provisioned/updated/torn down?" -- a question every resource in | |
| this workspace has needed an answer to since Phase 1, whether or not | |
| anything about it ever executes. `RuntimeState` answers "is the thing | |
| that record represents currently running?" -- a question that is | |
| meaningless for a bucket or a policy, and that varies *independently* | |
| of `Lifecycle` for the resources it does apply to: a compute instance | |
| can sit `Lifecycle::Active` for its entire existence while cycling | |
| between `RuntimeState::Running` and `RuntimeState::Stopped` many times | |
| over. Collapsing the two into one enum would either force every | |
| non-compute resource to carry meaningless `Running`/`Stopped` states, | |
| or force compute resources to smuggle their execution state through | |
| `Lifecycle::Updating` -- both of which misrepresent what actually | |
| changed. Two small, single-purpose state machines is the primitive | |
| answer; one overloaded one is the shortcut this workspace's rule | |
| exists to reject. | |
| ### Why capacity is a different shape from quota | |
| `cloud-quota` (Phase 1) and `cloud-capacity` (this phase) look similar | |
| -- both track a limit and a running usage -- but encode an opposite | |
| default on purpose. `cloud-quota` is an *account cap*: a resource | |
| category nobody has configured a limit for is unlimited, because the | |
| absence of a quota policy should not itself block anything. `cloud-capacity` | |
| is a *physical(-ish) pool*: an availability zone nobody has told its | |
| own size has nothing to schedule onto, so `CapacityTracker::try_reserve` | |
| fails closed with `NotFound` against an unregistered AZ rather than | |
| inventing an unlimited pool that doesn't exist. Same shape of code, | |
| opposite failure mode, because they model opposite kinds of limit. | |
| ### The one real composition this phase adds | |
| `cloud-scheduler` (Phase 2) gains `place_least_loaded_with_capacity`, | |
| which filters `cloud-region`'s candidate AZs down to the ones | |
| `cloud-capacity` reports enough spare room in, picks the least-loaded | |
| survivor exactly as `place_least_loaded` already did, and reserves | |
| that capacity on the winner in the same call -- all-or-nothing, per | |
| `cloud-capacity::CapacityTracker::try_reserve`'s own atomicity, and | |
| touching neither `usage` nor `capacity` for any AZ if none qualifies. | |
| This is a deliberately generic extension (any resource type can ask | |
| for capacity-aware placement, not just compute), which is why it lives | |
| in `cloud-scheduler` rather than in a new crate: capacity-aware | |
| placement is a placement-policy concern `cloud-scheduler` already | |
| owns, not a compute-specific one. | |
| ### What Phase 4 deliberately does not include | |
| - **No `cloud-compute` (or similarly named) crate exists yet.** Wiring | |
| `cloud-runtime` + `cloud-capacity` + `cloud-image` into an actual | |
| "launch an instance" operation -- a compute-specific `ProvisionRequest` | |
| payload, start/stop/terminate operations driving `RuntimeState` | |
| transitions -- is the first genuinely compute-shaped service this | |
| workspace would build, and per the non-negotiable rule, that | |
| composition doesn't get built (or named) until it is one: three | |
| standalone, independently real primitives today, not a shell crate | |
| waiting for them. | |
| - **`cloud-provisioner` and `cloud-control-plane` are untouched by this | |
| phase.** Reserving vCPU/memory at provisioning time is a | |
| compute-specific requirement, not a property every resource | |
| `cloud-provisioner`'s generic pipeline creates has (a bucket or a | |
| policy has no vCPU count) -- baking it into the generic pipeline | |
| would be exactly the kind of one-off special case this workspace's | |
| primitives are supposed to prevent. The capacity-aware scheduler | |
| function above is available to whatever later composes a compute | |
| service on top of the existing pipeline. | |
| - **Image deregistration is not implemented.** Whether an image that's | |
| still referenced by a launched (but not yet modeled) instance can be | |
| safely removed is a question `cloud-image` alone cannot answer -- | |
| it would need to see resource-to-image references that don't exist | |
| in this workspace until compute resources themselves do. | |
| ## Phase 5: storage primitives | |
| Compute (Phase 4) needed something to execute, a pool to run it in, | |
| and something to boot it from. Storage needs a different set of | |
| concepts entirely -- nothing about vCPUs or images applies -- but the | |
| same discipline: build what every storage-shaped resource (an object, | |
| a volume, a database's write-ahead log) would otherwise redefine for | |
| itself, and nothing that presupposes a specific service shape: | |
| | Crate | Owns | | |
| |---|---| | |
| | [`cloud-checksum`](./crates/cloud-checksum) | `Checksum`: a from-scratch CRC-32/ISO-HDLC content-integrity checksum, streaming or one-shot | | |
| | [`cloud-redundancy`](./crates/cloud-redundancy) | `RedundancyScheme`: replication or erasure coding, shard counts, reconstruction threshold, max tolerable loss | | |
| | [`cloud-attachment`](./crates/cloud-attachment) | `AttachmentState`: the attach/detach state machine for a volume-like resource | | |
| ### Why `cloud-checksum` has zero dependencies and isn't a wrapper around a crates.io hash crate | |
| Every other crate in this workspace either has zero dependencies or | |
| depends only on other `cloud-forge` crates -- there has never been an | |
| external (crates.io) dependency anywhere in this workspace, and this | |
| phase doesn't start now. CRC-32/ISO-HDLC is implemented directly, | |
| bit-by-bit rather than via a precomputed table, favoring an | |
| obviously-correct implementation over a fast one. Its correctness is | |
| anchored to the algorithm's own published standard check value | |
| (`crc32(b"123456789") == 0xCBF4_3926`, the same value every real | |
| implementation -- zlib, gzip, PNG -- uses to confirm itself), not just | |
| internal self-consistency between `of()` and a hand-rolled | |
| alternative. This is a deliberate application of this workspace's | |
| anti-hallucination discipline to code, not just prose: a checksum | |
| algorithm invented for this crate, with no external value to check it | |
| against, would be exactly the kind of unverifiable claim this project | |
| avoids elsewhere. | |
| ### Why redundancy and attachment are their own primitives | |
| `RedundancyScheme` models an arithmetic property (how many of N shards | |
| must survive to reconstruct the original), not an encoder -- this | |
| crate never touches actual bytes, which is why it composes with any | |
| future storage service regardless of whether that service replicates | |
| whole objects or erasure-codes them. `AttachmentState` is a third | |
| state machine alongside `cloud-lifecycle::Lifecycle` and | |
| `cloud-runtime::RuntimeState` (Phase 4), and is worth contrasting with | |
| both explicitly: `Lifecycle` tracks whether a resource *record* | |
| exists, `RuntimeState` tracks whether a *compute* resource is | |
| executing (meaningless for a volume), and `AttachmentState` tracks | |
| whether *this* volume is currently connected to something -- three | |
| questions that vary independently, not one overloaded enum wearing | |
| three names. Unlike the other two, `AttachmentState` has **no terminal | |
| state**: every state has a path back to `Detached`, because attachment | |
| cycles for the volume's whole life; ending that life is `Lifecycle`'s | |
| job, not this crate's. This is tested directly (`every_state_has_a_path_back_to_detached`), | |
| not just asserted from the transition table's shape. | |
| ### What Phase 5 deliberately does not include | |
| - **No `cloud-storage-capacity` crate.** `cloud-capacity` (Phase 4) | |
| tracks vCPU/memory, a two-dimensional quantity meaningful to | |
| compute placement; a byte-capacity pool for storage nodes is a | |
| real, different primitive, but building it now with no concrete | |
| allocator or storage control-plane phase to feed would produce | |
| exactly the kind of crate that exists only to be depended on later | |
| without doing anything itself today -- the thing this workspace's | |
| layering rule explicitly prohibits (see "Layering" above). It's | |
| deferred until a storage provisioning phase actually needs it. | |
| - **No object/blob storage crate and no `cloud-volume` crate.** | |
| Composing `cloud-checksum` + `cloud-redundancy` + `cloud-attachment` | |
| (and `cloud-provisioner`/`cloud-control-plane`) into an actual | |
| storage service is the same kind of composition Phase 4 deferred | |
| for compute (`cloud-compute`): it doesn't get built, or named, until | |
| it's a real composition of already-real primitives. | |
| - **No erasure-coding encoder/decoder.** `cloud-redundancy` models the | |
| arithmetic a real erasure code (e.g. Reed-Solomon) guarantees -- | |
| `data_shards` of `data_shards + parity_shards` shards reconstruct | |
| the original -- without implementing the encoding itself, which is | |
| a substantial cryptographic/mathematical undertaking out of scope | |
| for a placement/durability-accounting primitive. | |
| ## Phase 6: database primitives | |
| Compute (Phase 4) needed execution state, capacity, and images. | |
| Storage (Phase 5) needed integrity, durability, and attachment state. | |
| Database-shaped resources need a third, disjoint set of concepts -- | |
| none of the prior nine crates from those two phases apply here: | |
| | Crate | Owns | | |
| |---|---| | |
| | [`cloud-consistency`](./crates/cloud-consistency) | `ConsistencyLevel` (`Eventual`/`BoundedStaleness`/`Strong`): a total order plus `satisfies()` against a caller's minimum requirement | | |
| | [`cloud-migration`](./crates/cloud-migration) | `MigrationLedger`: enforces that schema migrations apply strictly sequentially and without gaps | | |
| | [`cloud-retention`](./crates/cloud-retention) | `RetentionPolicy`: which backups/snapshots are eligible for deletion, by age, with a floor protecting the most recent N regardless of age | | |
| ### Why the ordering is derived, not hand-rolled | |
| `ConsistencyLevel`'s three variants are declared weakest-to-strongest | |
| specifically so `#[derive(PartialOrd, Ord)]` produces the correct | |
| total order for free -- `Eventual < BoundedStaleness < Strong` falls | |
| out of the derive macro's own documented rule (variants compare by | |
| declaration position) rather than a hand-written `match` that could | |
| silently drift out of sync with the enum if a variant were ever | |
| reordered. `satisfies()` is then just `self >= required`: the derive | |
| does the real work, the method exists only to name the operation. | |
| ### Why `cloud-migration` and `cloud-retention` don't touch each other | |
| Both crates could plausibly want to know about each other -- a | |
| retention policy might seem like it should care whether a migration | |
| happened -- but they answer genuinely unrelated questions: | |
| `MigrationLedger` is about schema shape over time (has version `N`'s | |
| change been applied), `RetentionPolicy` is about data lifetime (is | |
| this snapshot's age past a threshold, and is it protected by the | |
| floor). Coupling them would mean forcing every caller of one to drag | |
| in a concept the other owns. `RetentionPolicy` depends on `cloud-types` | |
| (for `ResourceId` and `Timestamp`) precisely because those are the | |
| workspace's own shared value types, not because it needs anything | |
| migration- or consistency-specific. | |
| ### What Phase 6 deliberately does not include | |
| - **No `cloud-database` (or similarly named) crate, and no | |
| `cloud-table`/`cloud-index`/`cloud-query` crates.** Composing | |
| `cloud-consistency` + `cloud-migration` + `cloud-retention` (and | |
| earlier phases' `cloud-provisioner`/`cloud-control-plane`) into an | |
| actual database service is the same kind of composition Phases 4 | |
| and 5 deferred for compute and storage: it doesn't get built, or | |
| named, until it's a real composition of already-real primitives. | |
| - **No query language, execution engine, or storage engine.** This | |
| phase's crates answer "what guarantee is this database offering," | |
| "is the schema in a consistent state," and "what data may be | |
| discarded" -- not "how is data stored or queried," which is a | |
| vastly larger undertaking out of scope for a primitives phase. | |
| - **`cloud-migration` does not run migrations.** It has no concept of | |
| SQL, DDL, or any specific transformation -- only the version-ordering | |
| invariant. Actually executing a migration's contents is a concern of | |
| whatever future service calls this ledger, exactly as `cloud-reconciler` | |
| (Phase 2) computes a lifecycle transition path without executing | |
| whatever makes a resource actually reach the next state. | |
| ## Phase 7: messaging primitives | |
| Compute, storage, and database (Phases 4-6) each needed their own | |
| disjoint set of concepts. Messaging-shaped resources (queues, topics, | |
| pub/sub) need a fourth, equally disjoint set -- the last of the four | |
| service categories this repository's own README names before AWS-shaped | |
| service composition begins: | |
| | Crate | Owns | | |
| |---|---| | |
| | [`cloud-delivery`](./crates/cloud-delivery) | `DeliverySemantics` (`AtMostOnce`/`AtLeastOnce`/`ExactlyOnce`): a genuine **partial order**, not a total one | | |
| | [`cloud-visibility`](./crates/cloud-visibility) | `MessageLease`: the visibility-timeout mechanism that actually implements at-least-once delivery, plus a dead-letter threshold on receive count | | |
| | [`cloud-fanout`](./crates/cloud-fanout) | `FanoutRegistry`: the topic-to-subscriber pub/sub topology | | |
| ### Why `cloud-delivery` doesn't derive `Ord`, unlike `cloud-consistency` | |
| Phase 6's `cloud-consistency::ConsistencyLevel` is a genuine total | |
| order: every pair of levels is comparable, and `#[derive(Ord)]`'s | |
| declaration-order rule gives the correct answer for free. Delivery | |
| semantics are different in kind, not just in domain: they vary along | |
| **two independent axes** -- whether loss is tolerated, and whether | |
| duplication is tolerated. `AtMostOnce` (loss allowed, duplication | |
| forbidden) and `AtLeastOnce` (loss forbidden, duplication allowed) are | |
| **incomparable**: neither is "stronger" than the other, because each | |
| permits something the other categorically forbids. A derived (or | |
| hand-rolled linear) ordering would have to put one before the other | |
| arbitrarily, and `satisfies()` would then give a wrong answer for that | |
| pair. `cloud-delivery` instead checks both axes explicitly, which | |
| correctly reports `AtMostOnce.satisfies(AtLeastOnce)` and its reverse | |
| as both `false`, while `ExactlyOnce` (which forbids both) satisfies | |
| every requirement. This is worth stating explicitly because it would | |
| have been easy, and wrong, to copy Phase 6's pattern here without | |
| checking whether the underlying relationship was actually total. | |
| ### Why `cloud-visibility` is a separate crate from `cloud-delivery` | |
| `DeliverySemantics` *declares* a guarantee; `MessageLease` is the | |
| actual mechanism (a hide-until-deadline lease plus a receive counter) | |
| that makes `AtLeastOnce` true in practice -- a message that isn't | |
| acknowledged in time becomes visible again automatically, which is how | |
| "never loses, may duplicate" is achieved mechanically rather than | |
| merely claimed. Keeping the declaration and the mechanism in separate | |
| crates mirrors `cloud-lifecycle` (Phase 1, a state machine) staying | |
| separate from `cloud-provisioner` (Phase 2, what actually drives state | |
| transitions) -- a semantics type and the engine that upholds it are | |
| different kinds of thing even when tightly related. | |
| ### What Phase 7 deliberately does not include | |
| - **No `cloud-queue`/`cloud-topic` (or similarly named) crate.** | |
| Composing `cloud-delivery` + `cloud-visibility` + `cloud-fanout` (and | |
| earlier phases' `cloud-provisioner`/`cloud-control-plane`) into an | |
| actual messaging service is the same composition every prior phase | |
| deferred for its own service category: it doesn't get built, or | |
| named, until it's real. | |
| - **`cloud-fanout` does not deliver anything.** It owns the | |
| topic-to-subscriber mapping only -- no idea what a message is, no | |
| retry, no per-subscriber `MessageLease`. Actually fanning a | |
| published message out (and tracking each subscriber's own lease) is | |
| a future service's job, exactly as the Phase 3 `Arn` registry owns a | |
| name-to-resource mapping without touching provisioning. | |
| - **No message ordering (FIFO) primitive.** Ordering guarantees | |
| interact with partitioning and fanout in ways that need a concrete | |
| queue/topic shape to be meaningful; modeling it against primitives | |
| alone risked producing an untestable abstraction. | |
| ## Phase 8: `cloud-compute`, the first real composed service | |
| Phases 4-7 built sixteen primitive crates across four service | |
| categories (compute, storage, database, messaging) and, at every | |
| single one, explicitly deferred composing them into anything -- "no | |
| `cloud-compute` (or similarly named) crate exists yet," repeated | |
| almost verbatim in each phase's own section above. Phase 8 is where | |
| that deferral ends for compute, the category with the most complete | |
| primitive set: `cloud-runtime` and `cloud-capacity` and `cloud-image` | |
| (Phase 4), plus `cloud-resource-registry` (Phase 3) and the Phase 1/2 | |
| resource model, are enough to build a real "launch an instance" | |
| operation without inventing a single new primitive. | |
| | Crate | Owns | | |
| |---|---| | |
| | [`cloud-compute`](./crates/cloud-compute) | `ComputeService`: `launch`/`transition_runtime`/`terminate`, composing eleven existing crates | | |
| ### Why this doesn't reuse `cloud-provisioner`/`cloud-control-plane` | |
| Phase 2's pipeline is deliberately generic: it works for any resource | |
| type and places with `cloud_scheduler::place_least_loaded`, which | |
| takes no capacity into account because most resources (a policy, a | |
| bucket in later phases) have none to check. `ComputeService` needs | |
| `place_least_loaded_with_capacity` (Phase 4) instead, since an | |
| instance genuinely cannot be placed somewhere without room for it. | |
| Two ways existed to get there: teach the generic pipeline about | |
| vCPU/memory (a one-off special case baked into a crate every other | |
| resource type also goes through -- exactly the anti-pattern the | |
| non-negotiable rule exists to prevent), or have `cloud-compute` run | |
| its own pipeline directly against the same underlying primitives. | |
| `cloud-compute` does the latter: it reimplements the shape of | |
| `AUTHORIZE -> VALIDATE -> PLAN -> APPLY` (policy check, quota | |
| reservation, capacity-aware placement, resource construction plus ARN | |
| registration) with the same rollback discipline `cloud-provisioner` | |
| established, but calling `cloud-capacity`/`cloud-image`/`cloud-runtime` | |
| directly rather than through a generic layer that doesn't know they | |
| exist. | |
| ### The rollback chain `launch` upholds | |
| `launch` reserves, in order: a quota unit, then placement capacity | |
| (which `place_least_loaded_with_capacity` reserves atomically with | |
| its own AZ-usage increment), then an ARN registration. A failure at | |
| any later stage releases every earlier reservation before returning -- | |
| tested directly for the capacity-exhaustion case (`QuotaExceeded` from | |
| `PLAN` rolls back the `VALIDATE`-stage quota unit) exactly as | |
| `cloud-provisioner`'s own tests do for its stages. | |
| ### `RuntimeState` lives in its own store, deliberately | |
| `ComputeService` keeps `RuntimeState` in a separate map from | |
| `Resource<InstanceSpec>`, not as a field inside the resource or its | |
| payload. This is not an implementation shortcut: `cloud-runtime`'s own | |
| docs (Phase 4) describe execution state and the resource record as two | |
| things that vary independently, and storing them together would | |
| re-couple what Phase 4 deliberately kept apart. `transition_runtime` | |
| touches only the runtime store; a resource's own `version`/`updated_at` | |
| never changes just because the instance was started or stopped. | |
| ### What Phase 8 deliberately does not include | |
| - **No image deregistration, no instance resize, no attached | |
| volumes.** `terminate` releases exactly what `launch` reserved | |
| (quota, capacity); nothing else about an instance's lifecycle beyond | |
| launch/run/stop/terminate is modeled yet. | |
| - **`cloud-storage`, `cloud-database`, `cloud-messaging` (or similarly | |
| named) services do not exist yet.** Phase 8 is compute's turn only -- | |
| the other three service categories built in Phases 5-7 remain | |
| primitives without a composed service, for exactly the reason this | |
| phase just finished demonstrating takes real, deliberate design work | |
| rather than a copy-paste of `cloud-compute`'s shape. | |
| - **No HTTP/gRPC API surface.** `control-api` remains deferred from | |
| Phase 2's own reasoning: an API layer belongs after the service it | |
| exposes is itself real, and `cloud-compute` is the first phase where | |
| that precondition starts to hold for any one category. | |
| ## Phase 9: `cloud-storage`, the second composed service | |
| Phase 8's closing note named exactly this: "the other three primitive | |
| categories built in Phases 5-7 remain uncomposed." Phase 9 composes | |
| the second -- storage -- following `cloud-compute`'s own shape | |
| deliberately rather than inventing a new one: a service composes | |
| already-real primitives, adds no new primitive of its own, and keeps | |
| the same `AUTHORIZE -> VALIDATE -> APPLY`-shaped pipeline with full | |
| rollback. | |
| | Crate | Owns | | |
| |---|---| | |
| | [`cloud-storage`](./crates/cloud-storage) | `StorageService`: `create_volume`/`transition_attachment`/`delete_volume`, composing nine existing crates | | |
| ### What's different from `cloud-compute`, and why | |
| - **No AZ placement.** `cloud-compute` places instances via | |
| `cloud_scheduler::place_least_loaded_with_capacity` because | |
| `cloud-capacity` (Phase 4) gives it a real per-AZ vCPU/memory pool to | |
| place against. Phase 5 deliberately never built a `cloud-storage-capacity` | |
| equivalent -- "no concrete allocator to feed" -- and Phase 9 doesn't | |
| retroactively invent one just to mirror `cloud-compute`'s shape. A | |
| volume's size is tracked the same way `cloud-compute` tracks instance | |
| count: an account-level `cloud-quota` reservation (`"storage_bytes"`), | |
| real and tested, but not AZ-scoped. | |
| - **A real, new business rule**: `delete_volume` refuses to proceed | |
| unless the volume's `AttachmentState` is `Detached`, checked before | |
| any lifecycle reconciliation or quota release happens. This is the | |
| first place in the workspace a service-level operation is gated by a | |
| *different* primitive's current state (attachment) rather than just | |
| its own resource's lifecycle -- a real cross-primitive invariant, not | |
| a restatement of something `cloud-attachment` or `cloud-lifecycle` | |
| already enforces on its own. | |
| - **`transition_attachment` mirrors `cloud-compute::transition_runtime` | |
| exactly**: a thin, single-step wrapper around the primitive's own | |
| `transition_to`, leaving the higher-level operation (`delete_volume`, | |
| parallel to `terminate`) to own the multi-step, side-effecting case. | |
| ### What Phase 9 deliberately does not include | |
| - **No attachment target validation.** `transition_attachment` moves a | |
| volume's own state machine; it does not check that some real compute | |
| instance exists to attach to, or call into `cloud-compute` to verify | |
| one. Two service crates directly depending on each other's internal | |
| stores would be a tighter coupling than either has needed so far -- | |
| reconciling "does the thing I'm attaching to actually exist" across | |
| two independent services is an orchestration concern for whatever | |
| future layer calls both, exactly as `cloud-fanout` (Phase 7) doesn't | |
| verify a subscriber id names a real resource either. | |
| - **No `cloud-database`/`cloud-messaging` (or similarly named) | |
| services yet.** Phase 9 is storage's turn only. | |
| - **No snapshots, no `cloud-checksum`/`cloud-retention` integration.** | |
| A volume in this phase has no content-integrity check and no backup | |
| lifecycle; wiring those Phase 5/6 primitives in is future work once a | |
| snapshot concept exists to attach them to. | |
| ## Phase 10: `cloud-database`, the third composed service | |
| The third of the four service categories (Phases 5-7) gets composed, | |
| following exactly the shape `cloud-compute` and `cloud-storage` | |
| established. | |
| | Crate | Owns | | |
| |---|---| | |
| | [`cloud-database`](./crates/cloud-database) | `DatabaseService`: `create_database`/`apply_migration`/`create_snapshot`/`expire_snapshots`/`delete_database`, composing ten existing crates | | |
| ### `cloud-retention` finally gets a stateful caller | |
| Phase 6 built `RetentionPolicy::eligible_for_deletion` as a pure | |
| function: given "now" and a list of `(id, created_at)` pairs, it | |
| computes which ids a policy would allow deleting. Nothing before this | |
| phase actually *kept* such a list anywhere. `cloud-database` is the | |
| first thing that does: `create_snapshot` appends to a real, | |
| per-database snapshot list, and `expire_snapshots` calls | |
| `eligible_for_deletion` against it and then actually removes what | |
| comes back -- the stateful half of a primitive Phase 6 deliberately | |
| left as pure computation. This also resolves Phase 9's own deferral | |
| note on `cloud-storage` ("wiring \[retention\] in is future work once a | |
| snapshot concept exists to attach them to"): the snapshot concept now | |
| exists, here, in the service where it was always going to make more | |
| sense (a database's point-in-time backups, not a raw block volume). | |
| ### The delete-time gate, and how it differs from `cloud-storage`'s | |
| `cloud-storage::delete_volume` refuses based on another primitive's | |
| *current state* (`AttachmentState != Detached`). `cloud-database::delete_database` | |
| refuses based on **this service's own recorded state** (a non-empty | |
| snapshot list) rather than a different primitive's state machine -- | |
| a different kind of cross-cutting check, but the same underlying | |
| discipline: a service-level delete operation must look past its own | |
| resource's `Lifecycle` before deciding it's safe to proceed. | |
| ### Why `apply_migration` is a thin pass-through | |
| Exactly like `cloud-compute::transition_runtime` and | |
| `cloud-storage::transition_attachment`, `apply_migration` does nothing | |
| beyond delegating to the primitive's own method | |
| (`MigrationLedger::apply`) and keeping the per-database map in sync. | |
| The pattern across all three services is now well-established: a | |
| primitive's own state-changing method is never reimplemented at the | |
| service layer, only looked up, called, and stored back. | |
| ### What Phase 10 deliberately does not include | |
| - **No query execution, no schema DDL.** `apply_migration` records | |
| that a version was applied; it has no idea what SQL or | |
| transformation that version represents, exactly as `cloud-migration` | |
| itself (Phase 6) does not run migrations. | |
| - **No `cloud-messaging` (or similarly named) service yet.** Phase 10 | |
| is database's turn only; messaging (Phase 7's primitives) remains | |
| the one service category not yet composed. | |
| - **`create_snapshot` does not use `cloud-checksum`.** A snapshot here | |
| is just an id and a creation time -- verifying a snapshot's actual | |
| content integrity would need the snapshot to contain real bytes, | |
| which this phase's model doesn't have. | |
| ## Phase 11: `cloud-messaging`, the fourth and last composed service | |
| The fourth and final service category (Phases 5-7) gets composed, | |
| completing the set `cloud-compute`, `cloud-storage`, and | |
| `cloud-database` began. Every service category a primitive phase left | |
| deferred is now real. | |
| | Crate | Owns | | |
| |---|---| | |
| | [`cloud-messaging`](./crates/cloud-messaging) | `MessagingService`: `create_queue`/`create_topic`/`subscribe`/`publish`/`enqueue`/`receive`/`delete_message`/`delete_queue`/`delete_topic`, composing eleven existing crates | | |
| ### Two resource kinds, not one | |
| `cloud-compute`, `cloud-storage`, and `cloud-database` each had exactly | |
| one top-level resource (an instance, a volume, a database). | |
| `cloud-messaging` has two: a queue, which holds messages directly, and | |
| a topic, which holds none of its own -- only a fan-out list of | |
| subscriber queues via `cloud-fanout::FanoutRegistry`. This is precisely | |
| the composition `cloud-fanout`'s own Phase 7 doc comment named as | |
| deferred: "actually fanning a published message out to each subscriber | |
| ... is a concern for whatever future service composes all three | |
| \[`cloud-delivery`, `cloud-visibility`, `cloud-fanout`\]." `publish` | |
| is that composition: it fans a message out to every subscriber queue, | |
| inserting a fresh `cloud_visibility::MessageLease` into each one, so | |
| `receive` inherits real at-least-once mechanics (in-flight messages | |
| can't be redelivered until their visibility timeout elapses, and a | |
| message received too many times is dead-lettered) whether the message | |
| arrived by direct `enqueue` or by fan-out from a topic. | |
| ### `subscribe`'s gate: a new operation, an established gate shape | |
| `cloud-storage::delete_volume` (Phase 9) refused based on *another | |
| primitive's* current state (a volume's own `AttachmentState`). | |
| `cloud-messaging::subscribe` reuses that exact shape on a different | |
| operation: it refuses to link a queue to a topic unless the queue's own | |
| `cloud_delivery::DeliverySemantics` satisfies the topic's required | |
| semantics (`DeliverySemantics::satisfies`, Phase 7) -- an `AtMostOnce` | |
| queue cannot subscribe to a topic that requires `ExactlyOnce`. The | |
| cross-primitive check that Phase 9 first applied at delete time now | |
| applies at subscribe time instead. | |
| ### `delete_queue`/`delete_topic`'s gate: the `cloud-database` shape, twice | |
| `cloud-database::delete_database` (Phase 10) refused based on the | |
| service's *own* recorded state (a non-empty snapshot list) rather than | |
| another primitive's state machine. Both of `cloud-messaging`'s delete | |
| operations reuse that shape: `delete_queue` refuses while its inbox is | |
| non-empty, and `delete_topic` refuses while it still has subscribers -- | |
| two independent applications of the same "look past your own | |
| `Lifecycle` at your own recorded state before deleting" discipline, | |
| one per resource kind. | |
| ### `publish` validates every subscriber before delivering to any | |
| A duplicate message id colliding with one subscriber's existing inbox | |
| entry, discovered partway through a fan-out, must not leave some | |
| subscribers holding the message and others without it. `publish` checks | |
| every subscriber has room for the new id in a first pass, and only | |
| inserts into any of them in a second pass once every check has | |
| succeeded -- the same validate-before-apply discipline every pipeline | |
| in this workspace follows, applied here to a fan-out instead of a | |
| single resource's creation. | |
| ### What Phase 11 deliberately does not include | |
| - **No message body.** A message is just a `ResourceId`, exactly as a | |
| `cloud-database` snapshot (Phase 10) was just an id and a creation | |
| time -- there is no payload to store, checksum, or serialize. | |
| - **No ordering guarantee.** `BTreeMap`'s key order is used for | |
| determinism in tests and listings, not as a delivery-order promise; | |
| nothing in this phase models FIFO queues. | |
| - **No per-account message-count quota.** Exactly as `cloud-database` | |
| did not quota snapshots (only `"databases"` itself), this phase | |
| quotas `"queues"` and `"topics"` at creation time but not the | |
| messages flowing through them. | |
| - **Every service category originally identified back in Phases 4-7 is | |
| now composed.** A future phase's "what's still deferred" is no | |
| longer a service category; it is whatever composing these four | |
| services *together* (e.g. a compute instance's logs delivered | |
| through a queue) would require, which is deliberately out of scope | |
| here. | |
| ## Phase 12: `cloud-orchestration`, the first cross-service composition | |
| Phases 8-11 each composed primitives *within* one service category. | |
| Phase 12 is a different kind of composition: it composes two already-real | |
| **services** together, directly resolving a deferral `cloud-storage` | |
| (Phase 9) stated in its own words. | |
| | Crate | Owns | | |
| |---|---| | |
| | [`cloud-orchestration`](./crates/cloud-orchestration) | `attach_volume`/`detach_volume`: free functions coordinating `ComputeService` and `StorageService`, composing two existing services | | |
| ### Resolving `cloud-storage`'s own deferral, verbatim | |
| Phase 9's doc comment said this explicitly: "`cloud-storage` doesn't | |
| validate attachment targets -- attaching a volume moves only its own | |
| `AttachmentState`; whether the id it's attached to names a real | |
| `cloud-compute` instance is an orchestration concern for whatever | |
| future layer calls both services." `cloud-orchestration` is that layer. | |
| `attach_volume` takes a `&ComputeService` and a `&mut StorageService` | |
| together, checks the instance exists and is not `Terminating`/ | |
| `Terminated`, and only then drives the volume's attachment transitions. | |
| An unknown or terminated instance leaves the volume's `AttachmentState` | |
| completely untouched -- validate before mutate, the same discipline | |
| every pipeline in this workspace already follows, just applied across | |
| a service boundary instead of within one service's own stages. | |
| ### Free functions, not a third stateful service | |
| Unlike `cloud-compute`/`cloud-storage`/`cloud-database`/`cloud-messaging`, | |
| `cloud-orchestration` owns no resource store, no quota, no policy, no | |
| event log -- it has nothing to persist between calls. Its two functions | |
| take `&ComputeService`/`&mut StorageService` directly and return, | |
| exactly the shape `cloud-provisioner` (Phase 2) already established for | |
| a pipeline with nothing of its own to hold onto. A caller keeps owning | |
| both services; `cloud-orchestration` only ever borrows them for the | |
| duration of one call. | |
| ### Why there is no rollback to write | |
| Every earlier multi-stage pipeline in this workspace (`cloud-provisioner`, | |
| `cloud-compute::launch`, `cloud-database::create_database`, ...) needed | |
| rollback because a later stage could fail after an earlier stage had | |
| already reserved something real (quota, capacity, a name). `attach_volume` | |
| has no such stage: once the instance check passes, `Attaching -> Attached` | |
| is unconditionally valid in `cloud-attachment`'s own transition table | |
| (Phase 5) from the exact state `attach_volume` itself just placed the | |
| volume into. There is no failure mode between those two calls to roll | |
| back from, and the doc comment says so explicitly rather than leaving a | |
| reader to wonder why this phase, alone among the pipeline-shaped crates, | |
| has no rollback path. | |
| ### Why `detach_volume` doesn't check `compute` at all | |
| Detaching is presumed always safe, regardless of the instance's state -- | |
| including an instance that has since been terminated, or was deleted | |
| entirely. This mirrors real block-storage services, which support a | |
| force-detach specifically for the case where the thing a volume was | |
| attached to is already gone: the operation exists to free the volume, | |
| not to protect an instance that no longer needs protecting. | |
| ### What Phase 12 deliberately does not include | |
| - **No orchestration for `cloud-database` or `cloud-messaging`.** This | |
| phase resolves exactly the one deferral `cloud-storage` stated; | |
| wiring a database or a queue to a compute instance is a different | |
| cross-service question with no equivalent stated deferral yet. | |
| - **No compute-instance-to-queue log delivery**, the example Phase 11's | |
| own closing note raised as one of many possible future | |
| cross-service compositions. This phase picks the one with an | |
| existing, explicit deferral behind it rather than inventing a new | |
| composition with no prior art in this workspace to resolve. | |
| - **No new state machine.** `cloud-orchestration` reads | |
| `cloud-runtime::RuntimeState` and drives `cloud-attachment::AttachmentState` | |
| -- both already exist; this phase adds no new states and no new | |
| transitions to either. | |
| ## Phase 13: the IAM surface -- credentials, sessions, policy documents | |
| `cloud-identity`'s own Phase 1 doc comment named this explicitly: "the | |
| full IAM surface (credentials, sessions, federation, policy | |
| documents-as-JSON) is Phase 13 in the roadmap and is not pulled | |
| forward." Phase 13 is that surface -- three new primitive crates, the | |
| same shape Phases 4-7 each used (multiple disjoint primitives, no | |
| service composition yet), rather than a service. A future IAM service | |
| composing them is deferred exactly as `cloud-compute` was deferred | |
| until Phase 8, well after Phase 4 built its primitives. | |
| | Crate | Owns | | |
| |---|---| | |
| | [`cloud-credentials`](./crates/cloud-credentials) | `CredentialStore`: at most two `Active` credentials per principal, the real IAM rule AWS itself enforces | | |
| | [`cloud-session`](./crates/cloud-session) | `SessionStore`: assumed-role and federated sessions, valid only while un-revoked and unexpired | | |
| | [`cloud-policy-document`](./crates/cloud-policy-document) | `to_json`/`from_json`: a `cloud-policy::Policy` serialized to and parsed from a JSON-shaped document, via a hand-rolled parser restricted to exactly this schema | | |
| ### Why no real secret material | |
| `cloud-credentials` models a credential's *lifecycle* -- its id, the | |
| principal it belongs to, and whether it's `Active` or `Inactive` -- and | |
| nothing about its actual secret bytes. Generating a cryptographically | |
| secure secret needs a real source of randomness, which this | |
| zero-external-dependency workspace deliberately doesn't pull in (every | |
| crate here depends only on other `cloud-*` crates). This mirrors | |
| `cloud-lifecycle` modeling a resource's lifecycle without knowing what | |
| the resource actually does: the state machine is real; the payload it | |
| governs is out of scope for the crate that owns the state machine. | |
| ### The real IAM rule: at most two active credentials | |
| Rather than invent a business rule, `cloud-credentials` implements one | |
| AWS itself enforces on its own IAM users: a principal may hold at most | |
| `MAX_ACTIVE_CREDENTIALS_PER_PRINCIPAL` (2) `Active` credentials at | |
| once. `Inactive` credentials don't count against the cap, so a | |
| principal can rotate safely -- create a new key, deactivate the old | |
| one, delete it later -- without ever exceeding the limit at any point | |
| in between. Reactivating a credential re-checks the same cap `create` | |
| does, since reactivation can push a principal back over the limit just | |
| as creation can. | |
| ### `cloud-session`'s two independent invalidity paths | |
| A [`Session`] is valid only while it is **both** un-revoked and | |
| unexpired -- deliberately the same two-independent-axis shape | |
| `cloud_visibility::MessageLease` (Phase 7) already established for | |
| at-least-once redelivery, where a message became undeliverable either | |
| by being in flight or by exceeding its receive count. Here, a session | |
| becomes invalid either by outliving its `expires_at` or by an explicit | |
| `SessionStore::revoke` call ahead of that natural expiry; neither | |
| condition implies the other, and `Session::is_valid` checks both. | |
| `SessionSource` captures the two ways Phase 13's own scope named a | |
| session can come to exist -- `AssumedRole` and `Federated` -- covering | |
| both "sessions" and "federation" from `cloud-identity`'s original | |
| deferral in one type, rather than a separate `cloud-federation` crate | |
| for what is, underneath, still just a session with a different origin. | |
| ### `cloud-policy-document`'s hand-rolled, deliberately narrow parser | |
| Serializing a `Policy` to JSON and back needs *something* that | |
| understands JSON syntax, and pulling in a general-purpose JSON crate | |
| would break this workspace's zero-external-dependency posture -- | |
| exactly the same posture `cloud-checksum` (Phase 5) upheld by | |
| implementing CRC-32 from scratch, and `cloud-types::Arn` (Phase 1) | |
| upheld with its own hand-rolled `Display`/`FromStr` pair instead of a | |
| generic serialization framework. `cloud-policy-document::from_json` is | |
| a parser restricted to exactly the fixed schema a policy document | |
| needs -- strings, arrays, and objects, no numbers, booleans, or `null` | |
| -- not a general JSON value type. It also deliberately does not | |
| replicate AWS's own polymorphism, where `Principal`/`Action`/`Resource` | |
| may each be either a bare string or an array interchangeably: here | |
| `Action`/`Resource` are always arrays, and `Principal` is either the | |
| literal string `"*"` or an array, never a bare non-`"*"` string. This | |
| is the same kind of deliberate scope-narrowing `cloud-policy`'s own | |
| `matches_pattern` already applies (a single trailing-wildcard position, | |
| not a general glob engine) -- one shape per field is all this phase | |
| needs. | |
| ### `Policy::statements`, a small accessor added to an earlier crate | |
| `cloud-policy-document` needs to read a `Policy`'s statements to | |
| serialize them, and `Policy` (Phase 1) never exposed a way to do that | |
| from outside its own crate. Rather than duplicate that data somewhere | |
| else, `cloud-policy` gained one narrow, backward-compatible accessor, | |
| `statements(&self) -> &[Statement]` -- the same precedent | |
| `cloud-scheduler` (Phase 2) already set when it gained | |
| `place_least_loaded_with_capacity` in Phase 4: a later phase's | |
| composition need is satisfied by adding to the crate that already owns | |
| the data, not by duplicating it or routing around it. | |
| ### What Phase 13 deliberately does not include | |
| - **No IAM service composing these three crates.** Exactly as Phases | |
| 4-7 each built primitives with service composition deferred to | |
| Phases 8-11, Phase 13 is primitives only; a service that creates | |
| credentials/sessions/policy documents together, with quota, events, | |
| and an `Arn`-resolvable identity, is a future phase's concern. | |
| - **No real cryptographic secret material, signing, or verification** | |
| for `cloud-credentials` -- see above. | |
| - **No `cloud-federation` crate.** Federation is modeled as one | |
| `SessionSource` variant in `cloud-session`, not a fourth crate, since | |
| a federated identity's whole lifecycle *is* a session with a | |
| different origin. | |
| - **`cloud-policy-document` does not support AWS's bare-string-or-array | |
| polymorphism** for `Action`/`Resource`, and supports only the literal | |
| `"*"` (not an arbitrary bare string) for `Principal` -- see above. | |
| - **No resource-based policies** (a policy document attached to a | |
| specific resource rather than a principal) -- `cloud-resource::Resource<T>`'s | |
| own Phase 1 doc comment already named this as a Phase 13 concept not | |
| pulled forward into `Resource<T>` itself, and this phase's policy | |
| documents remain principal-facing, the same shape `cloud-policy` | |
| already evaluates against. | |
| ## Phase 14: `cloud-iam`, the fifth composed service | |
| Phase 13 built the IAM surface as three disjoint primitives, deliberately | |
| without a service, the same shape Phases 4-7 used before their own | |
| service phases arrived later. Phase 14 is that service: it composes | |
| `cloud-credentials`, `cloud-session`, and `cloud-policy-document` | |
| together with `cloud-policy` and `cloud-events`, following exactly the | |
| "compose already-real primitives, introduce none" discipline | |
| `cloud-compute`/`cloud-storage`/`cloud-database`/`cloud-messaging` | |
| (Phases 8-11) each established. | |
| | Crate | Owns | | |
| |---|---| | |
| | [`cloud-iam`](./crates/cloud-iam) | `IamService`: `create_credential`/`activate_credential`/`deactivate_credential`/`delete_credential`/`assume_role`/`federate`/`revoke_session`/`authorize`/`load_policy_document`/`export_policy_document`, composing five existing crates | | |
| ### The one real business rule: policy gates session creation | |
| Every composed service so far has called `Policy::authorize` before | |
| mutating -- but always to gate *another* service's own resource | |
| operation (`launch`, `create_volume`, `create_database`, `create_queue`). | |
| `IamService::assume_role` and `IamService::federate` are the first | |
| operations in this workspace where `cloud-policy` (Phase 1) governs | |
| something *within the IAM surface itself*: a principal now needs | |
| permission to assume a role or federate in at all, checked before any | |
| session is created, so a denied attempt leaves `cloud-session`'s store | |
| completely untouched. The mechanism is unchanged from every earlier | |
| phase; what's new is that IAM's own operations are finally subject to | |
| the same authorization discipline as everyone else's. | |
| ### `load_policy_document`/`export_policy_document`: the live policy, round-trippable | |
| `IamService` holds exactly one active `Policy`, and `load_policy_document` | |
| replaces it immediately -- every subsequent `authorize`/`assume_role`/ | |
| `federate` call is governed by the newly loaded policy, not just parsed | |
| data sitting inertly. `export_policy_document` is the inverse, so a | |
| policy can be extracted, edited as JSON, and reloaded. A malformed | |
| document leaves the previous policy in place: `self.policy = ...from_json(json)?` | |
| only assigns once the parse has already succeeded, so a failed | |
| `load_policy_document` call is a pure no-op on the active policy, | |
| consistent with every other fallible mutation in this workspace leaving | |
| state untouched on error. | |
| ### No accounts, regions, quota, or `Arn` -- deliberately | |
| Every prior composed service validated an `AccountId`/`RegionId` and | |
| reserved a `cloud-quota` unit before creating anything, and registered | |
| a resolvable `Arn` for what it created. `IamService` does none of | |
| these: real IAM systems (AWS's own included) are inherently global, | |
| not region-scoped, and neither a credential nor a session is | |
| provisioned infrastructure with a finite regional capacity or a | |
| canonical external name -- mirroring real IAM's own inconsistent | |
| resource model, where a role or user has an ARN but an access key or | |
| an STS session token does not. Omitting all four here is a deliberate | |
| design choice mirroring what IAM actually is, not an oversight or a | |
| gap to fill later. | |
| ### What Phase 14 deliberately does not include | |
| - **No resource-based policy attachment.** `IamService` holds one | |
| policy governing every principal; attaching a distinct policy to a | |
| specific resource remains the Phase 13 (IAM) concept `cloud-resource`'s | |
| own Phase 1 doc comment named as not pulled forward into `Resource<T>`. | |
| - **No credential-to-session linkage.** A `Credential` and a `Session` | |
| are tracked independently; `IamService` does not model long-term | |
| credentials being exchanged for a short-term session the way real | |
| STS `GetSessionToken` does, since `cloud-credentials` has no secret | |
| material for such an exchange to authenticate against in the first | |
| place (Phase 13's own deferral). | |
| - **No policy versioning or multiple named policies.** `load_policy_document` | |
| replaces the single active policy outright; there is no history of | |
| prior policies, no policy identifiers, and no way to attach different | |
| policies to different principals within one `IamService`. | |
| ## Phase 15: `cloud-orchestration` gets a second composition, and its first rollback | |
| Phase 12 gave `cloud-orchestration` its first cross-service | |
| composition (`cloud-compute` + `cloud-storage`, resolving | |
| `cloud-storage`'s own Phase 9 deferral). Phase 15 gives it a second, | |
| in the same crate: `cloud-compute` + `cloud-messaging`, resolving a | |
| deferral `cloud-messaging` itself named in its own Phase 11 closing | |
| note -- "what remains deferred is composing these services *together* | |
| (e.g. a compute instance's logs delivered through a queue)." | |
| | Crate | Owns | | |
| |---|---| | |
| | [`cloud-orchestration`](./crates/cloud-orchestration) | Adds `transition_runtime_and_notify`/`terminate_and_notify` alongside Phase 12's `attach_volume`/`detach_volume` | | |
| ### Why this composition needs a rollback and Phase 12's didn't | |
| Phase 12's doc comment explained, at some length, why `attach_volume` | |
| needed no rollback: once its instance check passed, `cloud-attachment`'s | |
| own transition table made the second step (`Attaching -> Attached`) | |
| unconditionally valid, so there was no failure mode between the two | |
| steps to roll back from. `transition_runtime_and_notify` and | |
| `terminate_and_notify` are different: a `RuntimeState` transition | |
| genuinely can fail (an invalid jump, an unknown instance), and unlike | |
| `cloud-attachment`'s state machine, `RuntimeState` transitions are not | |
| generally reversible -- there is no transition that undoes an arbitrary | |
| runtime-state change the way `Detaching -> Detached` undoes | |
| `Attaching`. That asymmetry decides the ordering: both functions | |
| enqueue the notification message into `cloud-messaging` *first*, then | |
| attempt the compute mutation, then -- only if that mutation fails -- | |
| delete the message they just enqueued via `MessagingService::delete_message`. | |
| A failed transition or termination therefore never leaves a stray | |
| notification sitting in a queue, and this is the first rollback | |
| anywhere in this workspace that spans two independent services rather | |
| than undoing steps within one. | |
| ### What this composition still doesn't do | |
| - **No message body describing the event.** Exactly as `cloud-messaging` | |
| (Phase 11) itself deferred, a message is just a `ResourceId`; the | |
| caller chooses a `message_id` that means whatever it needs to mean | |
| (an event id, a correlation id), and `cloud-orchestration` does not | |
| interpret it. | |
| - **No notification on `launch` or on volume attach/detach.** This | |
| phase covers exactly the two operations its motivating deferral | |
| named (an instance's own lifecycle events); extending notification | |
| to other operations or other services is a separate question with | |
| no equivalent stated deferral yet. | |
| - **No fan-out to multiple queues.** Both functions take exactly one | |
| `queue_id`; delivering the same notification to several queues would | |
| need `cloud-messaging`'s own topic/subscriber model (`publish`), not | |
| a direct `enqueue`, and composing that is left for whenever a | |
| concrete need for it exists. | |
| ## Phase 16: `cloud-orchestration` gets a third composition, with database operations | |
| Phase 15 composed `cloud-compute` with `cloud-messaging`. Phase 16 | |
| adds a third composition in the same crate: `cloud-database` + | |
| `cloud-messaging`. Both [`apply_migration_and_notify`] and | |
| [`delete_database_and_notify`] follow the same enqueue-first, | |
| rollback-on-failure discipline, since database operations like schema | |
| migrations can fail (an invalid version jump, incompatible changes) and | |
| are typically not reversible (schema changes are one-way; deletions are | |
| final). | |
| | Crate | Owns | | |
| |---|---| | |
| | [`cloud-orchestration`](./crates/cloud-orchestration) | Adds `apply_migration_and_notify`/`delete_database_and_notify` alongside Phase 12's attach/detach and Phase 15's compute+messaging | | |
| ### Why database operations need the same rollback pattern | |
| Unlike [`attach_volume`]'s second step (which `cloud-attachment`'s own | |
| transition table made unconditionally valid once the first succeeded), | |
| and matching Phase 15's reasoning for `RuntimeState` transitions: | |
| database mutations can fail independently. Applying a migration version | |
| requires the prior version to exist; deleting a database requires all | |
| snapshots to have expired. These operations are also not generally | |
| reversible -- there is no `unapply_migration` that undoes an arbitrary | |
| schema change, and deletion is final. So both functions enqueue the | |
| notification message first, then attempt the database mutation, then | |
| delete the message if the mutation fails. This mirrors Phase 15 exactly, | |
| and is the second (and second-and-third) composition in this workspace | |
| spanning two independent services. | |
| ### What this composition still doesn't do | |
| - **No message body describing the event.** Exactly as Phase 15 deferred | |
| this, a message is just a `ResourceId`. | |
| - **No notification on `create_database` or other database operations.** | |
| This phase covers exactly the two operations with clear, stated use | |
| cases (migration and deletion); extending to other operations or other | |
| services is a separate question. | |
| - **No fan-out to multiple queues.** Both functions take exactly one | |
| `queue_id`, matching Phase 15's scope limitation. | |
| ## Definition of done, per crate | |
| Reusing the roadmap's own maturity levels, scoped to what a primitive | |
| crate (not a service) can claim: | |
| - **L1 (data model)** — the types exist, are validated at construction, | |
| and round-trip through their own `Display`/`FromStr` (or equivalent). | |
| - **L2 (deterministic local implementation)** — the crate's actual | |
| logic (a state-machine transition, a policy evaluation, a quota | |
| check) is implemented and produces the same result for the same | |
| input every time, with no hidden clock/randomness/iteration-order | |
| dependency. | |
| - **L8 (tests)** — every invariant claimed above has a passing test | |
| that would fail if the invariant were violated (not just a | |
| compiles-and-returns-something test). | |
| Every crate in this phase targets L1+L2+L8. Persistence (L5), | |
| distribution (L4), and formal verification (L9/L10, eventually via | |
| `verification-forge`) are explicitly out of scope until the control | |
| plane (Phase 2) gives them something real to attach to. | |
| ## Relationship to `verification-forge` | |
| Per the roadmap: `verification-forge` should eventually verify | |
| `cloud-forge`'s invariants (resource-id uniqueness, deny-dominates | |
| policy evaluation, lifecycle transitions being total functions, quota | |
| violations never committing), not become a runtime dependency of it. | |
| That work is not started in this phase — it requires `cloud-forge`'s | |
| own resource/lifecycle/policy model to exist and stabilize first, which | |
| is exactly what this phase builds. | |
| ## Quality gates | |
| Same discipline as the other two workspaces in this repository: no | |
| crate is considered done until `cargo test -p <crate> --release` | |
| passes, `cargo clippy -p <crate> --all-targets -- -D warnings` is | |
| clean, and `cargo fmt -p <crate> -- --check` is clean — checked after | |
| every crate, not batched at the end. `#![forbid(unsafe_code)]` in every | |
| crate. No `unwrap()`/`expect()` on anything derived from a caller's | |
| input -- a malformed resource id or an invalid policy input is exactly | |
| the kind of thing this substrate has to reject cleanly (a `CloudError`) | |
| rather than panic on. The only `expect()` calls in this phase | |
| (`cloud-region`'s AZ-name construction, `cloud-types::AzId::region()`) | |
| are on values built from a *already-validated* piece of the same type's | |
| own state -- provably unreachable to fail, and each carries a comment | |
| saying so -- never on anything a caller supplied that hasn't already | |
| gone through that type's constructor. | |