Spaces:
Running on CPU Upgrade
Running on CPU Upgrade
| # Architecture | |
| ## High-level | |
| ``` | |
| HuggingFace Docker Space (single container, port 7860) | |
| ┌──────────────────────────────────────────────────────┐ | |
| Browser ──HTTPS──▶ │ Next.js 15 ── /api/* rewrite ──▶ FastAPI (:8000) │ | |
| │ (port 7860) │ │ | |
| │ ├─ repos/ ◀──┐ │ | |
| │ supervisord runs: ├─ services/ │ │ | |
| │ ┌ frontend (node server.js) └─ routers/ │ │ | |
| │ ├ backend (uvicorn) │ │ | |
| │ └ scheduler (weekly cron) │ │ | |
| └──────────────────────────────────────────────────┼───┘ | |
| │ | |
| HuggingFace Dataset (source of truth) | |
| VAILL/Legislation-Tracker-Data | |
| │ | |
| On container start: sync ◀──────┘ | |
| On pipeline run: upload ────────▶ | |
| ``` | |
| ## Components | |
| ### Frontend — `frontend/` | |
| - Next.js 15 App Router, TypeScript, Tailwind CSS. | |
| - Each public page has a matching `loading.tsx` that renders skeletons while | |
| the server component fetches (Next.js 15 Suspense route conventions). | |
| - Client components (Bills table, Calendar, Newsletter, State Spotlight, | |
| USMap) fetch from the browser to `/api/*`, which is same-origin — the | |
| Next.js rewrite in [`next.config.ts`](../frontend/next.config.ts) proxies | |
| those to the internal FastAPI port. | |
| - Server components use an absolute internal URL (`http://127.0.0.1:8000`) | |
| because Node's `fetch` can't resolve relative paths. | |
| ### Backend — `backend/app/` | |
| - FastAPI, Pydantic v2. | |
| - Layered: | |
| - `routers/` — HTTP endpoints, thin. | |
| - `repositories/` — data access. Reads from `data/*.json` (mirrored from | |
| the HF Dataset). Designed so a future Postgres swap only touches this | |
| layer. | |
| - `services/` — external side-effects (`hf_sync.py`, `eu_vectorstore.py`). | |
| - `schemas/` — Pydantic response models. | |
| - `auth/` — file-based bcrypt, JWT sessions. | |
| ### Pipeline — `backend/data_updating_scripts/` + `backend/update_data.py` | |
| - 14 sequential Python scripts. See [PIPELINE.md](PIPELINE.md). | |
| - `update_data.py` orchestrates: reads args, invokes each step as a | |
| subprocess, tracks status in `data/pipeline_status.json`, uploads back to | |
| HF Dataset when done (unless `--skip-upload`). | |
| ### Scheduler — `backend/scheduler.py` | |
| - APScheduler `BlockingScheduler` on `America/Chicago`. | |
| - Runs `update_data.py --pull --overwrite-pdf --continue-on-error` every | |
| Monday at 7:00 AM CT. | |
| - Only starts inside the Docker container (via supervisord). Local dev does | |
| not spawn a scheduler by default. | |
| ## Request lifecycle | |
| A browser request to https://vaill-legislation-tracker.hf.space: | |
| 1. HF's edge routes to the Space's container on port 7860 (Next.js). | |
| 2. If the URL is a page (e.g. `/bills`), Next.js renders it — the server | |
| component fetches data from `http://127.0.0.1:8000/api/*` inside the | |
| container. | |
| 3. If the URL is `/api/*`, the Next.js server rewrite proxies to FastAPI | |
| internally. | |
| 4. FastAPI hits the in-memory repository, which was populated from `data/*` | |
| at startup (which was synced from the HF Dataset). | |
| 5. Response goes back through Next.js → HF edge → browser. | |
| ## Startup sequence (container boot) | |
| 1. supervisord launches three programs: backend, frontend, scheduler. | |
| 2. Backend runs `app.main` on import: | |
| - Loads settings from environment (HF Space secrets). | |
| - If `HF_REPO_ID` and `HUGGINGFACE_HUB_TOKEN` are set, syncs all core | |
| JSONs + `weekly_changes/` + `newsletter_drafts/` from the HF Dataset. | |
| `snapshots/` is intentionally excluded (large + only needed at | |
| pipeline time). | |
| - If `OPENAI_API_KEY` is set and no EU vectorstore exists on disk, | |
| builds it from the bundled `eu-ai-act.pdf` (~30s, ~$0.001). | |
| - Eagerly loads bills into memory. | |
| - Registers routers and starts serving. | |
| 3. Frontend runs `node server.js` on port 7860 (Next.js standalone). | |
| 4. Scheduler stays idle until its cron fires. | |
| ## Data storage | |
| - **HF Dataset** — source of truth. Any code path that persists data | |
| ultimately writes here. See [DATA.md](DATA.md). | |
| - **Container disk** (`data/`) — a cached mirror. Ephemeral on HF; wiped on | |
| every container restart and refilled from HF. | |
| - **Local `.env` files** — dev secrets only. Git-ignored. | |
| - **HF Space secrets** — production values for API keys, HF token, | |
| SendGrid, etc. Set in the Space Settings, exposed as environment | |
| variables to the container. | |
| ## Authentication | |
| - Users in `users.json` (synced from HF). | |
| - Login POSTs `username + password` to `/api/auth/login`. Backend | |
| bcrypt-checks against the hash and returns a signed JWT. | |
| - Frontend stores the token in `localStorage` (`admin_token`) and sends it | |
| as `Authorization: Bearer …` on `/api/admin/*` requests. | |
| - Admin routes gate on `require_admin` FastAPI dependency. | |