docs/ARCHITECTURE.md

Architecture

High-level

                    HuggingFace Docker Space (single container, port 7860)
                    ┌──────────────────────────────────────────────────────┐
Browser ──HTTPS──▶  │  Next.js 15  ── /api/* rewrite ──▶  FastAPI (:8000) │
                    │  (port 7860)                        │                │
                    │                                     ├─ repos/ ◀──┐   │
                    │  supervisord runs:                  ├─ services/ │   │
                    │    ┌ frontend (node server.js)      └─ routers/  │   │
                    │    ├ backend  (uvicorn)                          │   │
                    │    └ scheduler (weekly cron)                     │   │
                    └──────────────────────────────────────────────────┼───┘
                                                                       │
                                                    HuggingFace Dataset (source of truth)
                                                    VAILL/Legislation-Tracker-Data
                                                    │
                    On container start: sync ◀──────┘
                    On pipeline run: upload ────────▶

Components

Frontend — frontend/

• Next.js 15 App Router, TypeScript, Tailwind CSS.
• Each public page has a matching loading.tsx that renders skeletons while the server component fetches (Next.js 15 Suspense route conventions).
• Client components (Bills table, Calendar, Newsletter, State Spotlight, USMap) fetch from the browser to /api/*, which is same-origin — the Next.js rewrite in next.config.ts proxies those to the internal FastAPI port.
• Server components use an absolute internal URL (http://127.0.0.1:8000) because Node's fetch can't resolve relative paths.

Backend — backend/app/

• FastAPI, Pydantic v2.
• Layered:
  • routers/ — HTTP endpoints, thin.
  • repositories/ — data access. Reads from data/*.json (mirrored from the HF Dataset). Designed so a future Postgres swap only touches this layer.
  • services/ — external side-effects (hf_sync.py, eu_vectorstore.py).
  • schemas/ — Pydantic response models.
  • auth/ — file-based bcrypt, JWT sessions.

Pipeline — backend/data_updating_scripts/ + backend/update_data.py

• 14 sequential Python scripts. See PIPELINE.md.
• update_data.py orchestrates: reads args, invokes each step as a subprocess, tracks status in data/pipeline_status.json, uploads back to HF Dataset when done (unless --skip-upload).

Scheduler — backend/scheduler.py

• APScheduler BlockingScheduler on America/Chicago.
• Runs update_data.py --pull --overwrite-pdf --continue-on-error every Monday at 7:00 AM CT.
• Only starts inside the Docker container (via supervisord). Local dev does not spawn a scheduler by default.

Request lifecycle

A browser request to https://vaill-legislation-tracker.hf.space:

1. HF's edge routes to the Space's container on port 7860 (Next.js).
2. If the URL is a page (e.g. /bills), Next.js renders it — the server component fetches data from http://127.0.0.1:8000/api/* inside the container.
3. If the URL is /api/*, the Next.js server rewrite proxies to FastAPI internally.
4. FastAPI hits the in-memory repository, which was populated from data/* at startup (which was synced from the HF Dataset).
5. Response goes back through Next.js → HF edge → browser.

Startup sequence (container boot)

1. supervisord launches three programs: backend, frontend, scheduler.
2. Backend runs app.main on import:
   • Loads settings from environment (HF Space secrets).
   • If HF_REPO_ID and HUGGINGFACE_HUB_TOKEN are set, syncs all core JSONs + weekly_changes/ + newsletter_drafts/ from the HF Dataset. snapshots/ is intentionally excluded (large + only needed at pipeline time).
   • If OPENAI_API_KEY is set and no EU vectorstore exists on disk, builds it from the bundled eu-ai-act.pdf (~30s, ~$0.001).
   • Eagerly loads bills into memory.
   • Registers routers and starts serving.
3. Frontend runs node server.js on port 7860 (Next.js standalone).
4. Scheduler stays idle until its cron fires.

Data storage

• HF Dataset — source of truth. Any code path that persists data ultimately writes here. See DATA.md.
• Container disk (data/) — a cached mirror. Ephemeral on HF; wiped on every container restart and refilled from HF.
• Local .env files — dev secrets only. Git-ignored.
• HF Space secrets — production values for API keys, HF token, SendGrid, etc. Set in the Space Settings, exposed as environment variables to the container.

Authentication

• Users in users.json (synced from HF).
• Login POSTs username + password to /api/auth/login. Backend bcrypt-checks against the hash and returns a signed JWT.
• Frontend stores the token in localStorage (admin_token) and sends it as Authorization: Bearer … on /api/admin/* requests.
• Admin routes gate on require_admin FastAPI dependency.