Spaces:
Running
Running
Commit History
-
research: Candidate 4 pre-build validation probe (LLM explanation/citation quality) -- YELLOW, conditional go
Before building the full Candidate 4 hybrid (ADR-026: classical model decides presence, LLM only explains/cites its positives), a small, self-contained probe (validation/candidate4_probe/) checked whether an LLM can actually do that narrowed job well. 24 real (contract, category) items, classical decision held fixed, BM25-localized excerpts (not full documents) via the existing src/retrieval.py retriever. Result: 100% citation verbatim accuracy (16/16, zero hallucinated spans) and 93.8% explanation accuracy (15/16) on items where a citation was produced; the other 8 were honest declines (5 correct classical-FP recognitions, 3 real retrieval misses), not fabrications. Not a clean GREEN: the probe's generation step used the frontier model running this session, not the project's actual $0/local production model (Qwen2.5-7B-Instruct) -- an explicit, stated constraint of this specific probe, and an upper-bound limitation on these numbers given this project's own prior finding that the sam…596c106