Spaces:
Sleeping
Sleeping
Commit History
-
chore: add pyright CI and use get_running_loop
Aditya KulkarniReplace deprecated get_event_loop, drop the tutorial Settings docstring, and fail CI if pyright reports type errors.14b0898 -
feat: landing server-status panel and embed ttft/ttfr
Aditya KulkarniRender GET /health on the landing as server status so the App tab does not jump to raw JSON. POST /embed includes ttft_ms (enqueue to encode start) and ttfr_ms (enqueue to embeddings ready). Keep GET /metrics as an unused always-200 JSON alias.5e96f99 -
feat: add HTML landing page for the Spaces App tab
Aditya KulkarniServe GET / as a static page that lists the public endpoints and links to Swagger. Restore the live-demo README framing now that the Space responds.c4fe3e6 -
feat: add /metrics snapshot and demote dead live-demo badge
Aditya KulkarniReuse the /health body on GET /metrics and always return 200. Drop the "metrics not exposed" limitation. The HF Space timed out on /health and /embed, so the README no longer frames it as a live demo.84098e5
-
feat: land nfcorpus benches, encode batch_size, and INT8-negative docs
Aditya KulkarniPass max_batch_size through model.encode so 128/256/512 batches are one forward pass, not silently chunked at 32. Add file-sourced bench harness, prepare_dataset.py, and the 24-run latency/throughput matrix including dynamic INT8. Document the two-regime result and the INT8 slowdown. Keep nfcorpus text pools gitignored; they regenerate with a fixed seed.dfdc75f
-
feat: add INT8 ONNX quantization with configurable backend file
Aditya KulkarniAdd scripts/quantize_onnx.py for data-free dynamic INT8 quantization of the exported ONNX model (90.4 -> 22.8 MB, 4x smaller). Run shape-inference preprocessing inside a temp dir (best-effort, with a direct-quant fallback) so it doesn't litter the working tree. Thread a configurable ONNX_FILE_NAME (app/config.py) through load_model(..., onnx_file_name=...) so the server can serve either the fp32 O3 export or the quantized model without code changes. Benchmark subset + fp32-vs-int8 quality delta (STS Spearman) pending a real test dataset.275b8e5
-
feat: add readiness /health and structured request logging
Aditya Kulkarni- /health is now a readiness probe (200/503) reporting model_loaded, worker_alive, queue_depth, and inflight; previously a static "ok" that inspected nothing - DynamicBatcher exposes is_running()/queue_depth()/inflight() with an in-flight counter tracked in submit() - JSON logging (app/logging_config.py) with a per-request X-Request-ID correlation id via middleware; structured access logs (method, path, status, duration_ms); inference errors logged server-side with a sanitized 500 returned to the client - drop the unused request param from /embed; add LOG_LEVEL setting - tests for readiness-503 and request-id generate/echo; README /health + logging docs3e47cef -
feat: add input validation, backpressure, and request timeouts
Aditya KulkarniHarden the /embed serving contract: - Validate request body (non-empty texts, max texts/request, max chars/text) -> 422 - Bound the request queue; reject with 503 when full instead of growing unbounded - Enforce a per-request deadline; return 504 on timeout - Map batcher errors to clean HTTP statuses; sanitize unexpected errors to 500 - Guard future resolution so a timed-out request can't crash the worker Limits are env-configurable (MAX_TEXTS_PER_REQUEST, MAX_CHARS_PER_TEXT, MAX_QUEUE_SIZE, REQUEST_TIMEOUT_S). Add tests for queue-full, timeout, and the 422/503/504 endpoint paths; update README failure-handling and API docs.3c0d20e
-
feat: add ONNX backend support
Aditya Kulkarnif864c14 -
fix: interpret batch wait timeout in milliseconds
Aditya Kulkarni57e38d0
-
cleanup: renamed api endpoint /predict to /embed
Aditya Kulkarnid690983
-
feat: add dynamic batching and benchmark script
Aditya Kulkarni046db3f -
feat: add POST /predict endpoint with MiniLM embedding generation
Aditya Kulkarni0b7903a