Commit History
-
[NOTICKET] fix(eval/knowledge): stop the scorer treating `_` as a word character (X22)
Rifqi Hafizuddin +1: Claude Opus 5`norm()` flattened every punctuation mark to a space except the underscore, because `_` is inside `\w`. The gold writes notation bare (`QtyA`) while the renderer folds the LaTeX subscript and emits `Qty_A`. Those normalised to `qtya` and `qty_a`, the substring test failed, and a correct entry was scored as a miss — the instrument was wrong, not the extraction. Underscores are deleted rather than spaced. Spacing only turns `qty_a` into `qty a`, which still does not contain `qtya`; deletion is what makes the two notations meet. The gold is NOT edited. Verified: none of the 15 rules contains an underscore, so no gold fragment changes under the new normaliser and matches can only move from miss to hit. The alternative — rewriting the gold to `Qty_A` — was rejected because it binds the frozen instrument to one renderer's current output, and rendering has already changed once this week and will change again with the backend decision. Two rules use the affected notation, GAINLOSS_QTY_ZERO_…abae344
-
[NOTICKET] feat(eval): knowledge-extraction gold set, scorer + frozen prototype baseline
Rifqi Hafizuddin +1: Claude Fable 5P2 + P3 + P4 from the knowledge-pipeline plan. Lifts the two things out of the kex prototype that could not be recreated if the machine died — the calibration knowledge and the measurement baseline — without porting any pipeline code. eval/knowledge/ (new, follows the eval/{intent,help,readiness,planner} convention): - knowledge_gold.yaml 41 terms + 15 rules, verbatim from the prototype. Still provisional (not expert-reviewed); accepted as good enough so the rebuild is not blocked on expert time. - score.py ported; only change is dropping the prototype's sys.path bootstrap. Pipeline-independent: scores plain lists, so it works against the prototype's artifacts and v2 alike. - results/baseline_prototype_2026-08-13_145132.json FROZEN. The numbers v2 must match or beat. - README.md provenance, what the scorer refuses to conflate, and how …60cf4a8