ser-inference / docs /ARCHITECTURE.md
spacey-cadet's picture
docs: overhaul README and Dockerfile for production pipeline
dddb6a8
|
Raw History Blame Contribute Delete
7.58 kB
# Architecture
## Request flow (`POST /predict`)
```
β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
audio file, session_id, β”‚ FastAPI /predict β”‚
consent flag ────────► β”‚ (app/api/routes.py) β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
β”‚
β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β–Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚ Canary routing β”‚ Track 3.2
β”‚ (app/routing/canary.py) β”‚ champion or challenger
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
β”‚
β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β–Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚ decode + resample β”‚ app/audio/decode.py
β”‚ (PyAV, any codec β†’ 16k mono)β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
β”‚
β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β–Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚ validation gate β”‚ Track 2.1
β”‚ duration / RMS silence floor β”‚ app/audio/validate.py
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
rejectβ”‚ pass
β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β”‚
β”‚ β–Ό
β”‚ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚ β”‚ VAD trim (silero) β”‚ Track 2.1
β”‚ β”‚ app/audio/vad.py β”‚
β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
β”‚ β”‚
β”‚ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β–Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚ β”‚ WavLM + pooling + MLP β”‚ app/inference/predict.py
β”‚ β”‚ classifier forward pass β”‚ (run_inference)
β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
β”‚ β”‚
β”‚ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β–Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚ β”‚ calibration (Platt/isotonic)β”‚ Track 1.2
β”‚ β”‚ app/calibration/calibrate.pyβ”‚
β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
β”‚ β”‚
β”‚ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β–Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚ β”‚ confidence cascade β”‚ Track 1.3
β”‚ β”‚ app/inference/cascade.py β”‚
β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
β”‚ β”‚
β–Ό β–Ό
response_emotion=neutral response_emotion=<label or neutral>
(chat-safe default) cascade_tier attached
β”‚ β”‚
β”‚ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β–Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚ β”‚ session state update β”‚ Track 3.3
β”‚ β”‚ app/session/state.py β”‚
β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
β”‚ β”‚
β””β”€β”€β”€β”€β”€β”€β–Ίβ”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β–Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚ consent check β”‚ Track 2.4
β”‚ app/logging_pipeline/consent β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
can_log β”‚ can't log
β–Ό β”‚
β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”β”‚
β”‚ feature/drift log β”‚β”‚ Track 2.2/2.3
β”‚ SQLite or HF Dataset β”‚β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”˜β”‚
β”‚β—„β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
β–Ό
JSON response to caller
```
## Offline / scheduled paths (not on the request critical path)
```
scripts/calibration_fit.py ──► data/calibration/calibration.json ──► loaded at app startup
scripts/build_drift_reference.py ──► data/drift_reference/train_features.json
.github/workflows/drift-check.yml (cron) ──► scripts/drift_check.py
reads live features from feature log, compares to reference via ks_2samp,
fires app/alerts/webhook.py on drift or high low-confidence rate
scripts/champion_challenger_eval.py ──► promotion recommendation across all corpora
.github/workflows/ci-eval-gate.yml ──► tests + eval gate on every merge to main
```
## Component responsibilities
- **`app/model/`** β€” loading and sanity-checking checkpoints from HF Hub
revisions. `registry.py` is the seam where a self-hosted MLflow instance
could replace "the Hub repo is my registry" later without touching
anything else.
- **`app/audio/`** β€” everything that happens to a clip before it reaches the
model. Fail-safe by design: VAD fails open (scores raw audio if unavailable
rather than crashing), validation fails closed (rejects and returns
neutral rather than scoring garbage).
- **`app/calibration/` + `app/inference/`** β€” the actual modeling-rigor
layer. Calibration is a pure function of an offline-fit JSON file; the
cascade is pure logic with no I/O β€” both are unit-testable without a model.
- **`app/session/`** β€” the one piece of state that spans requests. Kept
behind a `SessionStore` interface specifically so swapping memory β†’ Redis
is a config change, not a rewrite.
- **`app/logging_pipeline/`** β€” the data flywheel. `consent.py` sits in
front of everything else in this package on purpose β€” nothing else in the
package should be reachable from the request path without going through it
first.
- **`app/routing/`** β€” single-purpose: pick champion or challenger. Kept
separate from `predict.py` so the canary logic can be tested/changed
without touching the inference pipeline.
- **`app/alerts/`** β€” intentionally the simplest module in the repo. Two
lines of actual logic. Resist the urge to grow it in place β€” an on-call
paging system is a different tool, not a bigger webhook module.