Spaces:
Sleeping
Sleeping
Commit History
-
feat: land nfcorpus benches, encode batch_size, and INT8-negative docs
Aditya KulkarniPass max_batch_size through model.encode so 128/256/512 batches are one forward pass, not silently chunked at 32. Add file-sourced bench harness, prepare_dataset.py, and the 24-run latency/throughput matrix including dynamic INT8. Document the two-regime result and the INT8 slowdown. Keep nfcorpus text pools gitignored; they regenerate with a fixed seed.dfdc75f
-
feat: add INT8 ONNX quantization with configurable backend file
Aditya KulkarniAdd scripts/quantize_onnx.py for data-free dynamic INT8 quantization of the exported ONNX model (90.4 -> 22.8 MB, 4x smaller). Run shape-inference preprocessing inside a temp dir (best-effort, with a direct-quant fallback) so it doesn't litter the working tree. Thread a configurable ONNX_FILE_NAME (app/config.py) through load_model(..., onnx_file_name=...) so the server can serve either the fp32 O3 export or the quantized model without code changes. Benchmark subset + fp32-vs-int8 quality delta (STS Spearman) pending a real test dataset.275b8e5
-
feat: add ONNX backend support
Aditya Kulkarnif864c14
-
feat: add POST /predict endpoint with MiniLM embedding generation
Aditya Kulkarni0b7903a