[NOTICKET] docs(knowledge): diagnose the rule-branch regression — three causes, one real

The full run scored the rule branch 1/15 against the prototype's 4/15. Diagnosed
to three separate causes; only one is an actual quality loss. Written up in
KNOWLEDGE_PIPELINE_CALIBRATION.md §12, tracked as X19-X22.

(a) TWO OF THE THREE LOSSES ARE A SCORING ARTIFACT. The gold set's
statement_contains strings were written against PyMuPDF-era notation, which ran
subscripts together (QtyA, QtyP). Rendered LaTeX writes them with an underscore
(Qty_A). The scorer normalises punctuation but "_" is a WORD character, so
"qty_a" never contains "qtya" and the match fails on notation alone. Collapsing
"_" takes the score 1/15 -> 3/15. GAINLOSS_QTY_ZERO_PLAN and
QTY_ACTUAL_ZERO_PLAN_ALT are correct extractions scored as misses.

The true comparison is 4/15 -> 3/15: a loss of ONE rule, not three.

(b) THE REAL LOSS: rendered formulas now compete with prose. 4 of 8 v2 rule
statements are bare equations; in the prototype 0 of 7 were. Before rendering,
formulas reached the model as mangled character-spaced LaTeX and did not read as
statements; rendered cleanly they read like prescriptive rules, and rule.txt
never excludes them. The lost rule shows it exactly: chunk ::0015 contains BOTH
the units sentence (gold GAINLOSS_UNITS) and a Gain/Loss equation, was a cue
candidate, and the model returned the equation.

This is pure loss, because the FORMULA BRANCH ALREADY CAPTURES those
equations — "Qty = (I NPRHour s) / (T otalHour s)" appears in both branches
of the same run. The rule branch is duplicating the formula branch and
dropping the operational prose it exists for.

(c) A RENDERING BUG, owner Sofhia: render_latex eats the leading character of
each character-spaced run. "Q t y" -> "ty", "P r o d u c t i o n" ->
"roduction". Three of 31 chunks affected on the reference document, and it
carries into formula_latex. Corrupted formula text in Chunk.text also means a
span quoting it cannot be verified. Three-line reproduction in §12.

Nothing fixed here on purpose. The scorer and gold set are the frozen measuring
instrument (X22), rule.txt is a prompt change that owes an eval run per §7B
(X20), and render.py is the parsing half (X21).

Verification: docs only; ruff clean; import main OK; 497 passed, 7 skipped.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

KNOWLEDGE_PIPELINE_CALIBRATION.md

@@ -305,3 +305,76 @@ blank lines and closing the block on two consecutive blanks.
 **The lesson generalises:** the two halves agree on the *contract*, and this still slipped through —
 a formatting choice that is completely reasonable on one side silently broke a regex on the other.
 Whitespace is part of the interface whether or not the schema says so.
+
+---
+
+## 12. Rule-branch regression — diagnosed (2026-08-24)
+
+The full run scored the rule branch at **1/15**, against the prototype's 4/15. Diagnosed to three
+separate causes, only one of which is an actual quality loss.
+
+### Cause 1 — two of the three losses are a scoring artifact, not a regression
+
+The gold set's `statement_contains` strings were written against the PyMuPDF-era notation, which ran
+subscripts together: `QtyA`, `QtyP`. The rendered LaTeX writes them with an underscore: `Qty_A`,
+`Qty_P`. The scorer normalises punctuation but **`_` is a word character**, so `qty_a` never contains
+`qtya` and the match fails on notation alone.
+
+| Scoring | Rules matched |
+|---|---|
+| As committed | **1/15** |
+| Collapsing `_` to nothing | **3/15** |
+
+`GAINLOSS_QTY_ZERO_PLAN` and `QTY_ACTUAL_ZERO_PLAN_ALT` are both correct extractions scored as
+misses. **The true comparison is 4/15 → 3/15, a loss of one rule.** Fixing this means changing either
+the scorer or the gold set — both are the frozen measuring instrument, so it is Rifqi's call, not a
+silent edit.
+
+### Cause 2 — the real loss: rendered formulas now compete with prose
+
+**4 of 8 v2 rule statements are bare equations. In the prototype, 0 of 7 were.**
+
+| Run | Formula statements | Prose statements |
+|---|---|---|
+| v2 | 4 | 4 |
+| Prototype | 0 | 7 |
+
+Before rendering, formulas reached the model as mangled character-spaced LaTeX and did not read as
+statements. Rendered to clean prose they read exactly like prescriptive rules, and the model takes
+them. `rule.txt` never excludes them — it defines a rule as *"a statement that constrains or directs
+behaviour"*, which a formula arguably is.
+
+The lost rule shows the mechanism exactly: chunk `::0015` contains **both** the units sentence
+(*"Gain/Loss menggunakan satuan BCM atau ton…"* — gold `GAINLOSS_UNITS`) and a Gain/Loss equation.
+The chunk **was** a cue candidate, so the model saw it, and returned the equation.
+
+This is pure loss, because **the formula branch already captures those equations** — `Qty = (I
+NPRHour s) / (T otalHour s)` appears in *both* branches of the same run. The rule branch is spending
+calls duplicating the formula branch and dropping the operational prose it exists for.
+
+Fix direction: `rule.txt` should exclude bare equations explicitly ("an equation is the formula
+branch's job; a rule is prose that constrains behaviour"). That is a prompt change and therefore owes
+an eval run per CLAUDE.md §7B — **not** to be done on vibes.
+
+### Cause 3 — a rendering bug: `render_latex` eats the first character
+
+Reproducible in three lines against `src.knowledge_parsing.render`:
+
+```python
+render_latex(r"$$ Q t y = \frac {I N P R H o u r s}{T o t a l H o u r s} $$")
+# → 'ty = (I NPRHour s) / (T otalHour s)'          expected: 'Qty = (INPRHours) / (TotalHours)'
+
+render_latex(r"$$ P r o d u c t i o n = M O H H \times Q t y $$")
+# → 'roduction = MOHH × Qty'                       expected: 'Production = MOHH × Qty'
+```
+
+The character-spacing collapse consumes the **leading letter** of each spaced run, and re-splits the
+remainder in the wrong place. Present in **3 of 31 chunks** of the reference document (`I NPRHour`,
+`T otalHour`, `P roductio`, `E ffectiveWorkingHour`) and visible in the committed formula entries.
+
+Consequences: formula text in `Chunk.text` is corrupted, so a span quoting it is unverifiable; terms
+that occur *only* inside a formula lose their first letter and can be missed by the term filter; and
+`formula_latex` values carry the damage into the Skill Registry.
+
+**Owner: Sofhia** — `src/knowledge_parsing/render.py`. Not fixed here: it is the parsing half, and
+the reproduction above is enough to act on.

KNOWLEDGE_PIPELINE_TODO.md

@@ -167,7 +167,10 @@ envelope shape proposed in §3 — so S1 can settle either way without touching
 | **X11** | Verbatim-span validation | ✅ **1.00** | ✅ | `validate/span_check.py`. Whitespace-only normalisation; unlocatable provenance rejects every guarded field; never repairs. **Extended:** `full_name`/`source_wording` are checked against the source themselves, so a silent normalisation is caught rather than stored |
 | **X12** | Null-definition escalation | 🔎 | ✅ | `validate/escalate.py` + the glossary loop. Still **unexercised on this document** — most clusters have too little evidence to escalate to, which is a property of a 9-page corpus, not a defect |
 | **X18** | Full-document run through both halves | Rifqi | ✅ | **Done 2026-08-24** → `eval/knowledge/results/v2_full_document_2026-08-24_093051.json`. Parsing module built the artifact (31 chunks), extraction ran all four branches: 77 calls, 80% cache hit. **E3 = 0.90, PASS** — the prototype's failing experiment (0.75 vs a 0.80 line) now clears it, on a 10-entry scoreable base |
-| **X19** | Rule-branch recall regression | Rifqi | ⛔ | **Found 2026-08-24: 1/15, down from the prototype's 4/15.** The cue filter proposed 8 candidate passages vs 9, so the loss is downstream of candidate generation — not diagnosed. The rule branch also runs on nano, the tier its failure mode is least detectable on |
+| **X19** | Rule-branch recall regression | Rifqi | 🔎 | **Diagnosed 2026-08-24 — calibration §12.** Three causes: (a) 2 of 3 losses are a *scoring artifact* (gold written `QtyA`, rendered output `Qty_A`; `_` is a word char so the substring match fails) — true comparison is 4/15 → 3/15; (b) the one real loss is rendered formulas competing with prose — 4 of 8 v2 rule statements are bare equations vs 0 of 7 in the prototype, and the formula branch already captures them; (c) a `render_latex` bug (X21) |
+| **X20** | Exclude bare equations in `rule.txt` | Rifqi | ⬜ | The fix for X19(b). A prompt change → owes an eval run per §7B; do not ship on vibes. The formula branch already covers equations, so this is duplicated spend as well as a miss |
+| **X21** | `render_latex` eats the leading character | Sofhia | ⛔ | **Found 2026-08-24.** `Q t y` → `ty`, `P r o d u c t i o n` → `roduction`. 3 of 31 chunks affected on the reference document; corrupts formula text in `Chunk.text` (so spans quoting it are unverifiable) and carries into `formula_latex`. Three-line reproduction in calibration §12 |
+| **X22** | Gold/scorer notation decision | Rifqi | ⬜ | Whether to normalise `_` in the scorer or rewrite the gold's `statement_contains`. Either edits the frozen measuring instrument, so it needs a deliberate call rather than a silent fix |
 | **X13** | Conflict detection | 🔎 | ✅ | `validate/conflict.py`, token overlap (explainable to the reviewer, unlike embeddings); never picks a winner. Unit-tested both ways; still **0 conflicts on real data** — one consistent standard gives it nothing to find. Works only because clustering puts all evidence in one call |
 | **X14** | Diff vs. active glossary version | 🔎 | ✅ | `diff/glossary_diff.py`. The baseline is passed in explicitly (`--active-glossary`) rather than read from wherever the last run wrote, so the duplicate and conflicting paths are reachable — all three verified by test |
 | **X15** | Frequency-sorted review queue | ✅ | ✅ | `queue/review_queue.py`. Each row carries page, section and the verbatim span so review is quote-vs-page. Gained a review reason for **wording discrepancies**, which is how the literal-wording decision reaches the expert |