Spaces:
Sleeping
Sleeping
Commit History
-
v0.4.0-alpha: cross-encoder reranking + eval harness with CI gate
Phase 0 — fixes the v0.3.x "head of Screen Ireland → wrong person" vocab-mismatch bug on the live HF deployment. - Cross-encoder reranker (flashrank + ms-marco-MiniLM-L-12-v2, ~22 MB ONNX, no torch). Dense top-20 → rerank → top-5. Relevance gate still runs on the dense score so OOC negatives are still rejected. - Targeted query expansion at rerank time only (head/runs/leads/boss → chief executive/ceo). Surgical, not generative. - Reranker pre-warmed in the Docker image alongside BGE so HF cold starts have no network fetch. - RERANK_ENABLED / RERANK_TOP_N / RERANK_MODEL settings; /health reports reranker state. Phase 1 — eval harness so quality is measurable. - 30-question stratified eval set (vocab-mismatch / entity-lookup / fact-lookup / multi-hop / OOC-negative). - Recall@5, Recall@10, MRR, nDCG@10 metrics (binary relevance). - LLM-as-judge faithfulness scorer (NVIDIA Llama 70B). - GitHub Action eval gate: blocks merge on Recall@5 < 0.85 or OOC refusal rate < 0.90. R…310c2f8