docs: humanize codebase, add architecture guide and clean docstrings

ARCHITECTURE.md

@@ -0,0 +1,188 @@
+# Architecture Guide
+
+This document describes the design, execution lifecycle, and operational contracts of `hf-backend`.
+
+## System Overview
+
+`hf-backend` is an aggregation gateway deployed on a Hugging Face Docker Space. Rather than hosting business logic directly, it mounts domain-specific routers from separate repositories under dedicated URL prefixes.
+
+```
+Internet Request
+       │
+       ▼
+[ Hugging Face Edge Proxy ] (:443)
+       │
+       ▼
+[ Uvicorn Worker ] (:7860)
+       │
+       ▼
+[ RequestLimitMiddleware ] (Body size cap + request timeout)
+       │
+       ├── /health ─────────► [ Host Health Monitor ]
+       ├── /semantic/* ─────► [ Mounted Subproject: Semantic-Comment-Analyze ]
+       └── /<prefix>/* ─────► [ Future Mounted Services ]
+```
+
+The gateway handles shared infrastructure requirements:
+- Dynamic discovery and mounting of standalone service packages.
+- Early input validation (payload length enforcement before buffering).
+- Global request timeouts preventing worker stalls.
+- Zero-cold-start initialization through baked image weights and startup warmups.
+- Centralized health checks reflecting mount status and model readiness.
+
+---
+
+## The Cold-Start Problem and Solution
+
+Machine learning services on serverless or restart-prone container hosts typically suffer from two cold-start stages:
+
+1. **Storage delay**: Weights must be pulled from Hugging Face Hub over the network at first boot, taking anywhere from 30 seconds to several minutes.
+2. **Compute delay**: Models must be deserialized, moved to memory, and compiled or JIT-traced on the first inference call.
+
+`hf-backend` eliminates both delays across container restarts.
+
+```
+┌─────────────────────────────────────────────────────────────┐
+│ 1. BUILD TIME (Docker Build via BuildKit)                   │
+│                                                             │
+│  pip install requirements.txt                               │
+│        │                                                    │
+│  COPY registry.py prefetch.py                               │
+│        │                                                    │
+│  RUN python prefetch.py                                     │
+│        └─► Imports mounted packages                         │
+│        └─► Calls warmup()                                   │
+│        └─► Models download into /home/user/hf-cache         │
+│        │                                                    │
+│  ENV HF_HUB_OFFLINE=1 TRANSFORMERS_OFFLINE=1                │
+│        │                                                    │
+│  COPY . . (App code changes never bust prefetch cache)      │
+└─────────────────────────────────────────────────────────────┘
+                               │
+                               ▼
+┌─────────────────────────────────────────────────────────────┐
+│ 2. CONTAINER STARTUP (Lifespan Hook)                        │
+│                                                             │
+│  Uvicorn starts process [1]                                 │
+│        │                                                    │
+│  load_routers() mounts endpoints from registry.py           │
+│        │                                                    │
+│  Lifespan executes:                                         │
+│        ├─► Iterates over loaded projects                    │
+│        ├─► Dispatches warmup() in a background thread       │
+│        └─► Applies WARMUP_TIMEOUT_SECONDS per project       │
+│        │                                                    │
+│  All warmups finish / weights load into memory              │
+│        │                                                    │
+│  Server opens socket :7860 and accepts incoming traffic     │
+└─────────────────────────────────────────────────────────────┘
+                               │
+                               ▼
+┌─────────────────────────────────────────────────────────────┐
+│ 3. SERVING RUNTIME                                          │
+│                                                             │
+│  First incoming inference request (e.g. /semantic/analyze)  │
+│        └─�� Models already resident in memory                │
+│        └─► Response latency < 100 ms                        │
+└─────────────────────────────────────────────────────────────┘
+```
+
+### Layer Caching Details
+
+The build uses a two-phase copy strategy:
+1. `requirements.txt` is installed first.
+2. `registry.py` and `prefetch.py` are copied, then `prefetch.py` executes. Model weights are stored in `/home/user/hf-cache`.
+3. `HF_HUB_OFFLINE=1` and `TRANSFORMERS_OFFLINE=1` are exported to lock down the filesystem at runtime.
+4. Application code (`main.py`, tests, documentation) is copied last.
+
+When you modify routing logic or update documentation, Docker reuses the cached model layers. The Space rebuilds and restarts in seconds rather than downloading multi-gigabyte checkpoints repeatedly.
+
+---
+
+## Service Contracts
+
+Every service mounted into `hf-backend` follows a standard contract.
+
+### 1. Package Structure
+A subproject must be installable as a Python package (e.g., via git tag in `requirements.txt`) and expose a FastAPI `APIRouter` instance:
+
+```python
+# target: "semantic_backend.router:router"
+from fastapi import APIRouter
+
+router = APIRouter()
+
+@router.get("/health")
+def project_health():
+    ...
+```
+
+### 2. Startup Warmup
+If the top-level package defines a callable named `warmup()`, the host lifespan executes it at boot time:
+
+```python
+# semantic_backend/__init__.py
+def warmup():
+    from .nlp_engine import warmup as _warmup
+    return _warmup()
+```
+
+Warmup conventions:
+- Must load lazy checkpoints (embeddings, classification heads, tokenizers).
+- Runs one dummy inference pass through the primary execution path.
+- Runs synchronously; the host wraps execution in `asyncio.to_thread`.
+- If a project does not define `warmup()`, the host records `"no warmup"` and continues without error.
+
+---
+
+## Guard Middleware
+
+The host places `RequestLimitMiddleware` in front of all endpoints:
+
+1. **Streaming Body Guard**:
+   Inspects the `Content-Length` header upfront. If the declared size exceeds `MAX_BODY_SIZE_BYTES` (default 10 MB), the connection returns HTTP 413 immediately. For chunked or undeclared streams, chunks are counted on arrival, raising HTTP 413 before memory exhaustion occurs.
+
+2. **Request Timeout**:
+   Wraps downstream route evaluation with `asyncio.wait_for`. If route processing exceeds `REQUEST_TIMEOUT_SECONDS` (default 60.0s), the gateway returns an HTTP 504 Gateway Timeout.
+
+---
+
+## Health and Degradation Model
+
+The root `/health` and `/` endpoints report the aggregate status of all registered projects:
+
+```json
+{
+  "status": "healthy",
+  "loaded": {
+    "/semantic": "semantic_backend.router:router"
+  },
+  "failed": {},
+  "warm_status": {
+    "/semantic": "warmed"
+  },
+  "total_warmup_seconds": 6.484
+}
+```
+
+### Evaluation Logic
+
+```
+   ┌──────────────────────────────────────────────┐
+   │ Any projects failed to mount?                │
+   │                OR                            │───► "degraded"
+   │ Any project warmup failed / timed out?       │
+   └──────────────────────────────────────────────┘
+                          │ No
+                          ▼
+   ┌──────────────────────────────────────────────┐
+   │ Are there any loaded projects?               │───► No: "unhealthy"
+   └──────────────────────────────────────────────┘
+                          │ Yes
+                          ▼
+                     "healthy"
+```
+
+- A project without a `warmup()` reports `"no warmup"` and remains `"healthy"`.
+- If a warmup throws an exception or exceeds `WARMUP_TIMEOUT_SECONDS`, the host logs the error, marks the project as `"failed: <reason>"`, and marks overall status as `"degraded"`.
+- Surviving projects continue serving traffic even if an adjacent service is degraded.

README.md

@@ -9,64 +9,65 @@ app_port: 7860
 
 # HF Backend
 
-FastAPI host running on a Hugging Face Docker Space. It mounts backend routers from separate project repositories under specific URL prefixes.
+FastAPI gateway running on a Hugging Face Docker Space. The gateway mounts backend routers from standalone project repositories under designated URL prefixes.
 
-## How it works
+For in-depth lifecycle mechanics, prefetch caching, and degradation semantics, see [ARCHITECTURE.md](file:///c:/Users/manga/Projects/github/rachit-mangawa/hf-backend/ARCHITECTURE.md).
 
-The host application does not define business logic. Each subproject lives in its own repository, exposes a router in `<package>/router.py`, and is installed as a Python package.
+## How It Works
 
-The host imports each router dynamically from `registry.py` and mounts it under a prefix. If an individual router fails to import, the host logs the error, records the failure, and keeps running for the other services.
+The host application does not contain domain business logic. Each subproject is maintained in its own repository, exposes a router in `<package>/router.py`, and installs as a standard Python package.
+
+The host imports each router from `registry.py` and mounts it under its configured prefix. If a router fails to import, the host logs the error, records the failure, and keeps running for the remaining services.
 
 ```
-/project-1/* -> HEARTFLOW_OS backend router
-/project-2/* -> Semantic Comment Analyzer backend router
-/health      -> Host health status
+/project-1/* -> HEARTFLOW_OS router
+/semantic/*  -> Semantic Comment Analyzer router
+/health      -> Host health and readiness status
 ```
 
-## Adding a project
+## Adding a Service
 
 Adding a new service takes two steps:
 
-1. Add the prefix and router path to `registry.py`:
+1. Register the URL prefix and target router in `registry.py`:
 ```python
 PROJECT_REGISTRY = {
-    "/project-1": "heartflow_backend.router:router",
-    "/project-2": "semantic_backend.router:router",
-    "/project-3": "new_project.router:router",
+    "/semantic": "semantic_backend.router:router",
+    "/new-service": "new_service.router:router",
 }
 ```
 
-2. Add the pinned Git tag to `requirements.txt`:
+2. Pin the Git tag or release in `requirements.in`, then compile `requirements.txt`:
 ```text
-git+https://github.com/<user>/<repo>.git@v1.0#subdirectory=backend
+git+https://github.com/<owner>/<repo>.git@v1.0#subdirectory=backend
 ```
 
-Pushing to `main` syncs the change to Hugging Face and rebuilds the Space container.
+Pushing to `main` deploys the changes to Hugging Face and rebuilds the Space container.
 
 ## Configuration
 
-Set these in the Space settings under **Variables and secrets**:
+Set these variables in the Space settings under **Variables and secrets**:
 
-| Variable | Required | Default | Notes |
+| Variable | Required | Default | Description |
 | :--- | :--- | :--- | :--- |
 | `CORS_ALLOWED_ORIGINS` | No | None | Comma-separated allowed origins. Wildcards (`*`) are ignored. |
-| `DISABLE_PROJECTS` | No | None | Comma-separated prefixes to skip loading (e.g. `project-2`). |
-| `MAX_BODY_SIZE_BYTES` | No | 10485760 (10 MB) | Max request body size before returning 413. |
-| `REQUEST_TIMEOUT_SECONDS` | No | 60.0 | Request timeout before returning 504. |
-| `WARMUP_TIMEOUT_SECONDS` | No | 300.0 | Warmup timeout per project in worker thread before timing out. |
+| `DISABLE_PROJECTS` | No | None | Comma-separated prefixes to skip loading (e.g. `semantic`). |
+| `MAX_BODY_SIZE_BYTES` | No | 10485760 (10 MB) | Maximum request body size before returning HTTP 413. |
+| `REQUEST_TIMEOUT_SECONDS` | No | 60.0 | Request timeout before returning HTTP 504. |
+| `WARMUP_TIMEOUT_SECONDS` | No | 300.0 | Warmup timeout per project during container boot. |
 | `GITHUB_TOKEN` | Only for private repos | None | GitHub token passed as a build secret to clone private dependencies. |
 
-GitHub repository secrets (used by GitHub Actions):
+Repository secrets for GitHub Actions:
 
-| Secret | Notes |
+| Secret | Description |
 | :--- | :--- |
-| `HF_BACKEND_SPACE` | Hugging Face write token used to push code and trigger Space restart. |
-| `HF_SPACE_URL` | Optional. Health URL for the keep-alive workflow. |
+| `HF_BACKEND_SPACE` | Hugging Face token with write permissions to push code and restart the Space. |
+| `HF_SPACE_URL` | Optional health URL for the keep-alive workflow. |
 
 ## Endpoints
 
 ### `GET /health`
-Returns the status of the host and each registered project.
+Returns the status of the host and each registered project:
 
 ```json
 {
@@ -78,17 +79,21 @@ Returns the status of the host and each registered project.
   "warm_status": {
     "/semantic": "warmed"
   },
-  "total_warmup_seconds": 6.35
+  "total_warmup_seconds": 6.48
 }
 ```
 
-If any service fails to load or warm up, `status` changes to `"degraded"` or `"unhealthy"` and the error is listed under `"failed"` or `"warm_status"`.
+Status outcomes:
+- `healthy`: All registered projects mounted successfully and no warmup failed.
+- `degraded`: One or more services failed to mount, or a warmup failed or timed out.
+- `unhealthy`: No services loaded successfully.
 
-Interactive documentation (`/docs`, `/redoc`, `/openapi.json`) is disabled.
+Interactive documentation (`/docs`, `/redoc`, `/openapi.json`) is disabled for production hardening.
 
-## Release workflow
+## Release Workflow
 
-1. Test changes in the project repository using its `dev.py`.
+1. Test changes in the subproject repository.
 2. Tag the release (`git tag v1.0`, `git push origin v1.0`).
-3. Update the tag in this repo's `requirements.txt`.
-4. Push to `main`. The GitHub Actions workflow pushes to Hugging Face and restarts the Space.
+3. Update the tag reference in `requirements.in` and recompile `requirements.txt`.
+4. Run `pytest` locally to verify routing and warmups.
+5. Push to `main`. The GitHub Actions workflow syncs the commit to Hugging Face and triggers a restart.

main.py

@@ -1,3 +1,10 @@
+"""HF Backend Gateway.
+
+Aggregates subproject routers mounted on a Hugging Face Space. Handles
+model warmup during lifespan startup, streaming payload size limits,
+and centralized health tracking.
+"""
+
 import asyncio
 import importlib
 import logging
@@ -5,9 +12,11 @@ import os
 import time
 from contextlib import asynccontextmanager
 from typing import Callable
+
 from fastapi import FastAPI, HTTPException, Request, Response
 from fastapi.middleware.cors import CORSMiddleware
 from starlette.middleware.base import BaseHTTPMiddleware
+
 from registry import PROJECT_REGISTRY
 
 logging.basicConfig(level=logging.INFO, format="%(asctime)s [%(levelname)s] %(message)s")
@@ -24,6 +33,8 @@ default_warmup_timeout = float(os.getenv("WARMUP_TIMEOUT_SECONDS", "300.0"))
 
 
 class RequestLimitMiddleware(BaseHTTPMiddleware):
+    """Enforces incoming request payload limits and execution timeouts."""
+
     def __init__(self, app, max_size: int, timeout: float):
         super().__init__(app)
         self.max_size = max_size
@@ -55,6 +66,7 @@ class RequestLimitMiddleware(BaseHTTPMiddleware):
 
 
 async def run_warmup_for_project(prefix: str, target: str, timeout: float):
+    """Executes a mounted package's warmup function in a worker thread."""
     logger.info(f"[{prefix}] Checking warmup for target '{target}'...")
     try:
         module_name = target.split(":")[0]
@@ -120,6 +132,7 @@ if allowed_origins:
 
 
 def load_routers():
+    """Mounts routers dynamically from registry.py with isolated failure handling."""
     loaded_projects.clear()
     failed_projects.clear()
     warm_status.clear()

prefetch.py

@@ -1,3 +1,9 @@
+"""Build-time model prefetch script.
+
+Reads project routes from registry.py and invokes warmup() on each
+top-level package to bake weights into the Docker image filesystem.
+"""
+
 import importlib
 import logging
 import sys