docs/DEPLOYMENT.md

# Deployment

**Status tags:** `IMPLEMENTED` · `VERIFIED` · `MEASURED` · `OPEN` · `DEFERRED` · `BLOCKED`.

The live system is deployed across three **private** repositories plus one **public** umbrella repo.
The monorepo working copy is **not** the deployed source. This document records the live topology,
the environment variables, the deploy mechanics, and the traps that cost real debugging time.

---

## 1. Live revisions (VERIFIED)

| Component | Repository | Visibility | Branch | Revision | Host |
|---|---|---|---|---|---|
| Frontend | `Anish-lab-blip/SatQuery-Frontend` | private | `main` | **`2d7ae53b482d`** | Cloudflare Pages → `satquery.pages.dev` |
| Backend / orchestrator | `Anish-lab-blip/SatQuery-Backend` | private | `main` | **`89d80eaddec5`** | Render → `satquery-backend-m4yv.onrender.com` |
| Inference | `Anish-lab-blip/SatQuery-Inference` | private | `main` | **`5a0936ace491`** | Codespace `potential-space-trout-r4ppw969w45j2pvvw`, port 8000 |
| Public umbrella | `Anish-lab-blip/SatQuery-AI` | **public** | `main` | `3dcabd32da41` | this release home |
| Monorepo (working copy) | `C:/Users/anish/satquery-ai` | local only | `master` | `9d57aed` | **no remote**; 334 dirty entries |

> **Trap.** `deploy/` inside the monorepo is **stale and untracked**. It is **not** the deployed
> source. Edits must go to the three real repositories.

## 2. Live health (VERIFIED, probed)

```json
{"status":"ok","service":"satquery-orchestrator",
 "tunnel":{"agent_connected":true,"agent_id":"codespaces-fd1038","pending":0,"completed":97},
 "config":{"codespace_name":"potential-space-trout-r4ppw969w45j2pvvw\n","codespace_port":8000,
           "transport_mode":"auto","tunnel_timeout_s":150.0,"wake_timeout_s":120.0,
           "upstream_timeout_s":90.0,"device":"cpu","has_github_token":true}}
```

Note `codespace_name` still carries a trailing `\n` — the cosmetic B-02 patch is **not** deployed.

## 3. Environment variables

### 3.1 Render (gateway) — measured live

| Variable | Value (live) |
|---|---|
| `CODESPACE_NAME` | `potential-space-trout-r4ppw969w45j2pvvw` |
| `CODESPACE_PORT` | `8000` |
| `SATQUERY_ALLOWED_ORIGINS` | `https://satquery.pages.dev` |
| `SATQUERY_DEVICE` | `cpu` |
| `SATQUERY_TRANSPORT` | `auto` |
| `SATQUERY_TUNNEL_TIMEOUT_S` | `150` |
| `SATQUERY_WAKE_TIMEOUT_S` | `120` |
| `SATQUERY_UPSTREAM_TIMEOUT_S` | `90` |
| `GITHUB_TOKEN` | present |

> There is **no** `SATQUERY_UPSTREAM_URL` and **no** `HF_TOKEN` in the live config — the transport is
> the outbound tunnel, not a forwarded port.

### 3.2 Codespace (inference)

| Variable | Purpose |
|---|---|
| `PORT` | platform-assigned; **must be read** |
| `SATQUERY_DEVICE` | `cpu` \| `cuda` \| `mps` \| `null`; read **without importing torch** |
| `SATQUERY_MAX_FILE_BYTES` | per-file cap (shared with Render) |
| `SATQUERY_ASSET_ENABLED` / `SATQUERY_ASSET_DIR` | both required for `/v1/assets`; fails closed (503) otherwise |
| `SATQUERY_ASSET_MAX_FILES` / `SATQUERY_ASSET_TTL_S` | optional handle capacity / lifetime |

## 4. Deploy mechanics

- **Frontend → Cloudflare Pages.** Staged by `scripts/stage_pages.mjs`, deployed with
  `npx wrangler pages deploy`.
- **Backend → Render.** `render.yaml` blueprint; `main.py` exposes `app`.
- **Inference → Codespace.** `deploy/codespace/serve.py` serves `build_space_app()` on `$PORT`;
  `.devcontainer/` forwards port 8000 and runs the tunnel agent on start via `postStartCommand`.
- **Repository writes** are performed through the **GitHub Git Data API** (blob → tree → commit →
  `PATCH` ref) with **sha256 byte-verification** of every uploaded blob. Deletions are expressed as
  `sha: null` tree entries. This is used instead of `git push` so each deployed file is verified by
  content hash.

### 4.1 Deployed-artifact integrity

Deployed files were re-read from the GitHub API and compared byte-for-byte against the local copies:
**9 files sha256 byte-identical**, and the deployed HEAD re-read from the API. See
`verify_deployed_head.py`.

## 5. Cold start (documented, not hidden)

Render's free tier sleeps when idle and the Codespace may be stopped. The frontend shows
*"Waking inference engine…"* while Render starts the Codespace and polls health. Cold start is
therefore **tens of seconds**, and this is documented rather than papered over.

## 6. Known deployment issues

| ID | Issue | State |
|---|---|---|
| **B-07** | Transient tunnel-agent gaps → a request can hang or return 504. Patch prepared, **NOT deployed**. | **OPEN** |
| — | B-07 root shape: in `auto` mode a tunnel timeout **falls through** to the forward path (`main.py:546`), burning `wake_timeout_s = 120` on a `302` (~249 s ≈ 150 + 120). | measured |
| **B-02** | `/api/health` `codespace_name` trailing `\n`. Cosmetic; the wake path strips it. | **OPEN (cosmetic)** |
| — | Frontend/Backend/Inference repos are **private**, so their links 404 for an outside audience. | by design |

## 7. Platform traps (recorded so they are not rediscovered)

- **Cloudflare `_headers` rules CONCATENATE, they do not override.** Two matching rules are merged;
  Chromium takes the **first** `max-age` it encounters. A later rule cannot "fix" an earlier one.
- **Cloudflare 308-redirects `X.html` → `/X`.** Reference the extensionless path.
- **A forwarded Codespace port returns `302` for a private repo** — which is *why* the tunnel exists.
- **The tunnel agent must be started by the devcontainer `postStartCommand`**, or a restarted
  Codespace comes up with `agent_connected:false`.
- **Never retry `POST /api/infer` at the gateway** — a retry consumes inference twice.

## 8. Preconditions for a fresh deploy (open items)

1. Cloudflare Pages project name / domain (needed for the deploy command and `robots.txt` sitemap).
2. Artifacts present, or capabilities honestly ship `available: false` (the service **degrades rather
   than crashes** on absent artifacts).
3. `HF_TOKEN` set on Render **if** the HF proxy path is used (not used in the live config).
4. Codespace `.devcontainer/` forwarding `:8000` and starting the tunnel agent.
5. The five historical backend blockers closed and verified on the first live run (ASGI stack in
   requirements; `$PORT` read; `OPTIONS` preflight; fail-fast app construction; adapter integrity
   compared against an expected digest).

## 9. Historical context

The superseded design ran inference on an **HF Space with ZeroGPU** behind a **Railway** gateway. The
active design moves to **Render + Codespace**, CPU-first, with an **outbound tunnel**. The four-endpoint
contract, the gateway responsibility table, the env-var vocabulary and the config freeze are
unchanged — only host names moved. `configs/deploy.yaml` still describes the old HF-Space/ZeroGPU
target and is left **undisturbed as frozen paperwork** (editing it would move the config hash); no
Gradio runtime exists in code.