api/README.md

api/ -- FastAPI backend

Wraps redactor/ (the same engine scripts/redact.py uses) behind an upload-a-file, poll-for-progress, download-the-result HTTP API, plus a minimal static page to try it by hand.

Run it

pip install -r requirements.txt          # includes fastapi/uvicorn
python -m spacy download en_core_web_trf # if you haven't already
uvicorn api.main:app --reload

Open http://127.0.0.1:8000 for the upload page, or http://127.0.0.1:8000/docs for interactive API docs (auto-generated by FastAPI).

Why this is a job queue, not a synchronous request

Redacting a large document can take real time (see EVALUATION.md's runtime numbers with en_core_web_trf). Holding an HTTP request open for minutes is fragile -- browsers, proxies, and most hosts time out long-idle connections regardless of which model is running underneath. So the API is async by design: submit a job, get an id back immediately, poll a status endpoint for progress, fetch the result once it's done. The upload page just wraps this in a progress bar via setInterval polling every second (see static/index.html).

Endpoints

Method	Path	What
POST	/api/jobs	Upload a .docx (multipart form field file). Returns {job_id, status} immediately -- the actual redaction runs on a background thread.
GET	/api/jobs/{job_id}	Poll status: {status, units_done, units_total, progress_percent, estimated_seconds_remaining, redactions_made, error}. status is one of queued/running/done/failed.
GET	/api/jobs/{job_id}/download	The redacted .docx, once status == "done" (409 otherwise).
GET	/api/jobs/{job_id}/log	The (category, original_value, fake_value) CSV audit trail for that job.
GET/POST/DELETE	/api/config/categories[/{name}]	List / add a regex-based category / remove a category. See README.md's "Customizing" section.
GET/POST/DELETE	/api/config/structural-columns[/{index}]	List / add / remove a personnel-table column rule.
GET/POST/DELETE	/api/config/exclude-list[/{term}]	List / add / remove an ORGANIZATION deny-list entry.

Every /api/config/* mutation calls RedactionEngine.reload_categories() on the shared engine instance afterward, so it takes effect on the next job immediately -- without restarting the server or reloading the transformer model.

Module map

api/
  main.py              FastAPI app + lifespan (loads RedactionEngine ONCE at startup, not per-job)
  settings.py          env-tunable knobs: JOBS_DIR, MAX_UPLOAD_BYTES, JOB_RETENTION_HOURS
  schemas.py            Pydantic request/response models (the wire format)
  job_store.py          in-memory Job record + thread-safe JobStore (see its docstring for scaling notes)
  redaction_worker.py   runs one job to completion; the only file here that touches redactor/
  config_editor.py      reads/writes config/pii_categories.yaml + org_exclude_list.txt (ruamel.yaml, comment-preserving)
  routers/jobs.py        the job endpoints
  routers/config.py      the settings endpoints
  static/index.html      upload form + progress bar
  static/settings.html   Categories / Table layouts / Exclude list editor
  static/style.css        shared styling for both pages
  static/app.js           index.html's logic
  static/settings.js      settings.html's logic

Known limitations (demo-scoped, on purpose)

• In-memory job store. Job state lives in a Python dict in one process. Restart the server and every job's status is gone (the files on disk under api_jobs/ survive, but nothing tracks them anymore). Fine for trying this out or a single instance; swap job_store.py for something Redis/Postgres-backed if you need multi-worker or restart-safe job tracking -- routers only talk to JOB_STORE, so that's the one place that would need to change.
• No auth. Anyone who can reach the server can submit jobs and, if they know or guess a job_id (a UUID4, so not practically guessable, but not access-controlled either), download that job's result.
• No cleanup job yet. JOB_RETENTION_HOURS is defined in settings.py but nothing currently sweeps old api_jobs/* directories -- add a scheduled task if this runs somewhere long-lived.
• Config edits aren't file-locked. Two people hitting /api/config/* at the exact same moment could race on the same YAML write. Fine for one person customizing settings through the UI; add a lock in config_editor.py if this is ever multi-user.

Hosting

This needs real memory -- en_core_web_trf is a transformer model, not the older en_core_web_lg. A 512MB free-tier box isn't enough (confirmed by an actual OOM kill on Render's free tier, not just estimated). See DEPLOYMENT.md at the repo root for concrete, cost-compared steps for both Hugging Face Spaces and Render's paid tiers -- the Dockerfile here works unmodified on either.