Commit History
-
[NOTICKET] eval(knowledge): X20 measures net -1; close X23 on expert ruling; instrument X25
Rifqi HafizuddinThree things, all from the same session. X20 -- FIRST CONTROLLED A/B IN THIS PROJECT, AND IT SAYS X20 COSTS A RULE. Arms alternated inside one session so drift hit both equally. Arm B reverts X20's three edits and KEEPS X23's synthetic examples, so the only variable is X20 itself. Rule branch only: 15 calls per run, ~35s, five rounds each. X20 in 5/14, every one of 5 runs X20 reverted 6/14, every one of 5 runs Zero variance in both arms, and exactly one rule differs: GAINLOSS_QTY_ZERO_PLAN, 0% with X20 and 100% without. The cause is X20's own final clause. That gold rule is a genuine conditional whose CONDITION is written symbolically -- "Jika (Qty_P(M) = 0, Qty_A(M) > 0), maka:" -- and X20 ends with "a statement that is mostly symbols, = and numbers is an equation". The heuristic cannot tell a computation from a symbolic condition, so it nulls a real rule. X20 is also not buying what it was written to buy. GAINLOSS_UNITS, the rule it targets, fires 0% in BOTH arms …e6ed414
-
[NOTICKET] feat(eval): knowledge-extraction gold set, scorer + frozen prototype baseline
Rifqi Hafizuddin +1: Claude Fable 5P2 + P3 + P4 from the knowledge-pipeline plan. Lifts the two things out of the kex prototype that could not be recreated if the machine died — the calibration knowledge and the measurement baseline — without porting any pipeline code. eval/knowledge/ (new, follows the eval/{intent,help,readiness,planner} convention): - knowledge_gold.yaml 41 terms + 15 rules, verbatim from the prototype. Still provisional (not expert-reviewed); accepted as good enough so the rebuild is not blocked on expert time. - score.py ported; only change is dropping the prototype's sys.path bootstrap. Pipeline-independent: scores plain lists, so it works against the prototype's artifacts and v2 alike. - results/baseline_prototype_2026-08-13_145132.json FROZEN. The numbers v2 must match or beat. - README.md provenance, what the scorer refuses to conflate, and how …60cf4a8