Commit History
-
[NOTICKET] eval(knowledge): X20 measures net -1; close X23 on expert ruling; instrument X25
Rifqi HafizuddinThree things, all from the same session. X20 -- FIRST CONTROLLED A/B IN THIS PROJECT, AND IT SAYS X20 COSTS A RULE. Arms alternated inside one session so drift hit both equally. Arm B reverts X20's three edits and KEEPS X23's synthetic examples, so the only variable is X20 itself. Rule branch only: 15 calls per run, ~35s, five rounds each. X20 in 5/14, every one of 5 runs X20 reverted 6/14, every one of 5 runs Zero variance in both arms, and exactly one rule differs: GAINLOSS_QTY_ZERO_PLAN, 0% with X20 and 100% without. The cause is X20's own final clause. That gold rule is a genuine conditional whose CONDITION is written symbolically -- "Jika (Qty_P(M) = 0, Qty_A(M) > 0), maka:" -- and X20 ends with "a statement that is mostly symbols, = and numbers is an equation". The heuristic cannot tell a computation from a symbolic condition, so it nulls a real rule. X20 is also not buying what it was written to buy. GAINLOSS_UNITS, the rule it targets, fires 0% in BOTH arms …e6ed414
-
[NOTICKET] eval(knowledge): the runner the harness has been missing
Rifqi Hafizuddin +1: Claude Opus 5The 2026-08-24 figures were assembled by hand. That is why the rule-branch number in the committed result file is 1/15 measured with a scorer that has since been fixed, and why nobody could re-derive it. This is the reproducible version, following the house convention the other four harnesses already use: module mode, timestamped results, never overwritten. It takes a ParsedDocument artifact and never a PDF. MinerU is not invoked and is not required -- that is the seam doing its job, and it is what makes the pending X20 and X23 measurements possible on a machine with no GPU and no parser installed. Free by default: E1 term-filter recall and E2 compression, plus the funnel, with zero API calls. --extract adds the paid branches, E3 schema fill and rule recall. --dry-run prints the token estimate and the per-branch cache-floor status. --mock exercises every branch with no spend, and stamps the result NOT SCOREABLE so a fabricated number can never be quoted as a measurement. Two refusal…1edf61c