feat: add nfcorpus quality eval for fp32 vs INT8Running
Aditya Kulkarni
c745c50
Measure cosine drift, rank agreement, and nDCG/recall so INT8 is judged on embedding quality, not speed alone.
chore: add pyright CI and use get_running_loop
Aditya Kulkarni
14b0898
Replace deprecated get_event_loop, drop the tutorial Settings docstring, and
fail CI if pyright reports type errors.
feat: landing server-status panel and embed ttft/ttfr
Aditya Kulkarni
5e96f99
Render GET /health on the landing as server status so the App tab does
not jump to raw JSON. POST /embed includes ttft_ms (enqueue to encode
start) and ttfr_ms (enqueue to embeddings ready). Keep GET /metrics as
an unused always-200 JSON alias.
feat: add HTML landing page for the Spaces App tab
Aditya Kulkarni
c4fe3e6
Serve GET / as a static page that lists the public endpoints and links
to Swagger. Restore the live-demo README framing now that the Space
responds.
feat: add /metrics snapshot and demote dead live-demo badge
Aditya Kulkarni
84098e5
Reuse the /health body on GET /metrics and always return 200. Drop the
"metrics not exposed" limitation. The HF Space timed out on /health and
/embed, so the README no longer frames it as a live demo.
feat: land nfcorpus benches, encode batch_size, and INT8-negative docs
Aditya Kulkarni
dfdc75f
Pass max_batch_size through model.encode so 128/256/512 batches are one
forward pass, not silently chunked at 32. Add file-sourced bench harness,
prepare_dataset.py, and the 24-run latency/throughput matrix including
dynamic INT8.
Document the two-regime result and the INT8 slowdown. Keep nfcorpus text
pools gitignored; they regenerate with a fixed seed.
feat: add readiness /health and structured request logging
Aditya Kulkarni
3e47cef
- /health is now a readiness probe (200/503) reporting model_loaded,
worker_alive, queue_depth, and inflight; previously a static "ok" that
inspected nothing
- DynamicBatcher exposes is_running()/queue_depth()/inflight() with an
in-flight counter tracked in submit()
- JSON logging (app/logging_config.py) with a per-request X-Request-ID
correlation id via middleware; structured access logs (method, path,
status, duration_ms); inference errors logged server-side with a
sanitized 500 returned to the client
- drop the unused request param from /embed; add LOG_LEVEL setting
- tests for readiness-503 and request-id generate/echo; README /health
+ logging docs
feat: add input validation, backpressure, and request timeouts
Aditya Kulkarni
3c0d20e
Harden the /embed serving contract:
- Validate request body (non-empty texts, max texts/request, max chars/text) -> 422
- Bound the request queue; reject with 503 when full instead of growing unbounded
- Enforce a per-request deadline; return 504 on timeout
- Map batcher errors to clean HTTP statuses; sanitize unexpected errors to 500
- Guard future resolution so a timed-out request can't crash the worker
Limits are env-configurable (MAX_TEXTS_PER_REQUEST, MAX_CHARS_PER_TEXT,
MAX_QUEUE_SIZE, REQUEST_TIMEOUT_S). Add tests for queue-full, timeout, and the
422/503/504 endpoint paths; update README failure-handling and API docs.
test: add pytest suite and CI
Aditya Kulkarni
299567b
- DynamicBatcher tests: size/time flush triggers, request->response mapping across
mixed batch sizes, exception propagation, and shutdown queue-drain, using a
deterministic fake model (no MiniLM download, CPU-only)
- endpoint tests for /health and /embed via FastAPI TestClient with load_model stubbed
- pytest + pytest-asyncio + ruff as a dev dependency group; pytest/ruff config
- GitHub Actions workflow: uv sync, ruff check, pytest
- Dockerfile: uv sync --no-dev so dev tools and the plot extra stay out of the image
- add CI badge to README
feat: add POST /predict endpoint with MiniLM embedding generation
