ser-inference / docs /ARCHITECTURE.md
spacey-cadet's picture
docs: overhaul README and Dockerfile for production pipeline
dddb6a8
|
Raw History Blame Contribute Delete
7.58 kB

Architecture

Request flow (POST /predict)

                          β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
  audio file, session_id, β”‚   FastAPI /predict       β”‚
  consent flag  ────────► β”‚   (app/api/routes.py)   β”‚
                          β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
                                       β”‚
                          β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β–Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
                          β”‚ Canary routing            β”‚  Track 3.2
                          β”‚ (app/routing/canary.py)   β”‚  champion or challenger
                          β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
                                       β”‚
                          β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β–Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
                          β”‚ decode + resample          β”‚  app/audio/decode.py
                          β”‚ (PyAV, any codec β†’ 16k mono)β”‚
                          β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
                                       β”‚
                          β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β–Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
                          β”‚ validation gate             β”‚  Track 2.1
                          β”‚ duration / RMS silence floor β”‚  app/audio/validate.py
                          β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
                                  rejectβ”‚  pass
                       β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜   β”‚
                       β”‚                   β–Ό
                       β”‚      β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
                       β”‚      β”‚ VAD trim (silero)         β”‚  Track 2.1
                       β”‚      β”‚ app/audio/vad.py          β”‚
                       β”‚      β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
                       β”‚                   β”‚
                       β”‚      β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β–Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
                       β”‚      β”‚ WavLM + pooling + MLP      β”‚  app/inference/predict.py
                       β”‚      β”‚ classifier forward pass    β”‚  (run_inference)
                       β”‚      β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
                       β”‚                   β”‚
                       β”‚      β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β–Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
                       β”‚      β”‚ calibration (Platt/isotonic)β”‚ Track 1.2
                       β”‚      β”‚ app/calibration/calibrate.pyβ”‚
                       β”‚      β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
                       β”‚                   β”‚
                       β”‚      β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β–Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
                       β”‚      β”‚ confidence cascade           β”‚ Track 1.3
                       β”‚      β”‚ app/inference/cascade.py     β”‚
                       β”‚      β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
                       β”‚                   β”‚
                       β–Ό                   β–Ό
             response_emotion=neutral   response_emotion=<label or neutral>
             (chat-safe default)        cascade_tier attached
                       β”‚                   β”‚
                       β”‚      β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β–Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
                       β”‚      β”‚ session state update        β”‚ Track 3.3
                       β”‚      β”‚ app/session/state.py        β”‚
                       β”‚      β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
                       β”‚                   β”‚
                       β””β”€β”€β”€β”€β”€β”€β–Ίβ”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β–Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
                                β”‚ consent check                β”‚ Track 2.4
                                β”‚ app/logging_pipeline/consent β”‚
                                β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
                                        can_log β”‚ can't log
                                             β–Ό        β”‚
                                β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”β”‚
                                β”‚ feature/drift log     β”‚β”‚ Track 2.2/2.3
                                β”‚ SQLite or HF Dataset   β”‚β”‚
                                β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”˜β”‚
                                              β”‚β—„β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
                                              β–Ό
                                    JSON response to caller

Offline / scheduled paths (not on the request critical path)

scripts/calibration_fit.py  ──► data/calibration/calibration.json  ──► loaded at app startup
scripts/build_drift_reference.py ──► data/drift_reference/train_features.json
.github/workflows/drift-check.yml (cron) ──► scripts/drift_check.py
    reads live features from feature log, compares to reference via ks_2samp,
    fires app/alerts/webhook.py on drift or high low-confidence rate
scripts/champion_challenger_eval.py ──► promotion recommendation across all corpora
.github/workflows/ci-eval-gate.yml ──► tests + eval gate on every merge to main

Component responsibilities

  • app/model/ β€” loading and sanity-checking checkpoints from HF Hub revisions. registry.py is the seam where a self-hosted MLflow instance could replace "the Hub repo is my registry" later without touching anything else.
  • app/audio/ β€” everything that happens to a clip before it reaches the model. Fail-safe by design: VAD fails open (scores raw audio if unavailable rather than crashing), validation fails closed (rejects and returns neutral rather than scoring garbage).
  • app/calibration/ + app/inference/ β€” the actual modeling-rigor layer. Calibration is a pure function of an offline-fit JSON file; the cascade is pure logic with no I/O β€” both are unit-testable without a model.
  • app/session/ β€” the one piece of state that spans requests. Kept behind a SessionStore interface specifically so swapping memory β†’ Redis is a config change, not a rewrite.
  • app/logging_pipeline/ β€” the data flywheel. consent.py sits in front of everything else in this package on purpose β€” nothing else in the package should be reachable from the request path without going through it first.
  • app/routing/ β€” single-purpose: pick champion or challenger. Kept separate from predict.py so the canary logic can be tested/changed without touching the inference pipeline.
  • app/alerts/ β€” intentionally the simplest module in the repo. Two lines of actual logic. Resist the urge to grow it in place β€” an on-call paging system is a different tool, not a bigger webhook module.