Commit History
-
[NOTICKET] feat: assert on invariants, report a band, classify the genre (Steps 0 + 5)
Rifqi Hafizuddin +1: Claude Opus 5Step 0 -- measurement hardening. `temperature=0` is argmax over logits a shared inference server does not hand back identically twice, and this corpus has a recorded case of identical input scoring 6/7 one day and 0/12 the next. Three changes follow from that: * `eval/knowledge/invariants.py` asserts on FIELDS AND INVARIANTS, never on exact text: a span that cannot be located verbatim, a missing or duplicate entity id, a rejected field that still carries a value (rejected-never-repaired is only checkable from outside), a page out of range, queue ranks that are not contiguous, a leaked sort key. A violation means a control leaked, so it fails the run (exit 5) regardless of how the scores looked. * `--runs N` repeats ONLY the paid stage -- the free stages are deterministic -- and reports min/median/max with the MEDIAN as the value, because one failed run drags a mean into a region no run occupied. It also checks every run minted identical entity ids, which is the K11…f1d64b1 -
[NOTICKET] fix: retire the fuzzy clustering pass; add E1c post-cluster recall
Rifqi Hafizuddin +1: Claude Opus 5`rapidfuzz.token_set_ratio` returns exactly 100 whenever one string's token set is a SUBSET of the other's, so a head noun always absorbed its own compounds and FUZZY_THRESHOLD=92 was inert. Measured across three documents (BUMA standard, Komatsu shop manual, Open Pit textbook Ch.1), the pass made 140 merges: 131 token-subset matches -> all wrong 6 singular/plural pairs -> correct, now folded deterministically 3 gain/loss pty|qty|ua -> wrong; three distinct metrics 94% of its output was wrong and it produced no correct merge that plural folding does not now do. Matching is exact (plural-folded) -> abbreviation<->expansion. token-subset merges 15/18/97 -> 0/0/0 E1c_strict (BUMA) 0.8293 -> 0.8537 (`EWH`, absorbed by `Working Hours`, recovered) After the fix BUMA's remaining unowned gold terms are all ones the FILTER never proposed, so clustering destroys nothing. `normalize(lemma=True)` is scoped to the clustering key path only. rank/evidence.py imports `norm…2cc21c9
-
/feat knowledge management (#20)
- [NOTICKET] docs: knowledge pipeline test integration — context, audited task checklist, DEV_PLAN §0.8 (8e69dc6c638509f4dd7b6775d520fb4aa4efbd3e) - [NOTICKET] docs: knowledge pipeline — reframe kex as prototype, not port target (7cce5ffc9ab88791ac139dbef8e88e124a3d049e) - [NOTICKET] feat(eval): knowledge-extraction gold set, scorer + frozen prototype baseline (60cf4a885ef31a8aa64916f5a70ac5a0e8062c5f) - [NOTICKET] chore: add missing planner and readiness eval results (bd6659b3bf523597e4482b04ab1fcff5e1284d25) - [NOTICKET] feat(knowledge_extraction): free stages — models, seam adapter, filters, cluster, ranking (024c30abd1189119d33508eded334fe5a3ed5011) - [NOTICKET] fix(knowledge_extraction): move runner into the package; fix two ranking defects (72605ff6876a7936ab685d417c766ce820f6214b) - [NOTICKET] feat(knowledge_extraction): span filter — E1 recall 0.8537, matches the frozen baseline (3a016343398753cf730d50548c0e92ff47c6716c) - [NOTICKET] feat(knowledge_extraction): paid extraction …b68816f
