Spaces:
Sleeping
Sleeping
Commit History
-
chore: add pyright CI and use get_running_loop
Aditya KulkarniReplace deprecated get_event_loop, drop the tutorial Settings docstring, and fail CI if pyright reports type errors.14b0898 -
feat: landing server-status panel and embed ttft/ttfr
Aditya KulkarniRender GET /health on the landing as server status so the App tab does not jump to raw JSON. POST /embed includes ttft_ms (enqueue to encode start) and ttfr_ms (enqueue to embeddings ready). Keep GET /metrics as an unused always-200 JSON alias.5e96f99
-
feat: land nfcorpus benches, encode batch_size, and INT8-negative docs
Aditya KulkarniPass max_batch_size through model.encode so 128/256/512 batches are one forward pass, not silently chunked at 32. Add file-sourced bench harness, prepare_dataset.py, and the 24-run latency/throughput matrix including dynamic INT8. Document the two-regime result and the INT8 slowdown. Keep nfcorpus text pools gitignored; they regenerate with a fixed seed.dfdc75f
-
feat: add input validation, backpressure, and request timeouts
Aditya KulkarniHarden the /embed serving contract: - Validate request body (non-empty texts, max texts/request, max chars/text) -> 422 - Bound the request queue; reject with 503 when full instead of growing unbounded - Enforce a per-request deadline; return 504 on timeout - Map batcher errors to clean HTTP statuses; sanitize unexpected errors to 500 - Guard future resolution so a timed-out request can't crash the worker Limits are env-configurable (MAX_TEXTS_PER_REQUEST, MAX_CHARS_PER_TEXT, MAX_QUEUE_SIZE, REQUEST_TIMEOUT_S). Add tests for queue-full, timeout, and the 422/503/504 endpoint paths; update README failure-handling and API docs.3c0d20e
-
test: add pytest suite and CI
Aditya Kulkarni- DynamicBatcher tests: size/time flush triggers, request->response mapping across mixed batch sizes, exception propagation, and shutdown queue-drain, using a deterministic fake model (no MiniLM download, CPU-only) - endpoint tests for /health and /embed via FastAPI TestClient with load_model stubbed - pytest + pytest-asyncio + ruff as a dev dependency group; pytest/ruff config - GitHub Actions workflow: uv sync, ruff check, pytest - Dockerfile: uv sync --no-dev so dev tools and the plot extra stay out of the image - add CI badge to README299567b