[NOTICKET] feat(knowledge): review diff + the rule/domain boundary (r4, o2)
Rifqi Hafizuddin+1: Claude Opus 5
5cb2e2d
r4 `changed_fields()` and `review_diff()`. The queue can now say WHAT moved
rather than only that something did - which was the ask: show before /
after, do not send the expert back to read the entry from the start.
`changed` is empty when the approval still stands, so the screen can say
"nothing to re-read" instead of rendering two identical panes. `before` is
what the expert actually had in front of them; for an `edited` decision
that is their own corrected text, not the model's claim. A test pins
changed_fields() and has_mutated() to the same answer - two notions of
"has this changed" that can disagree is the same defect as two notions of
readiness.
o2 The rule/domain boundary, written down in the output contract. The two
blurred on screen during the 8 Sep review and a boundary nobody has
recorded is one the prompts cannot hold. The test: if removing it would
change how a NUMBER IS COMPUTED OR SOURCED it is a rule; if it would only
…
The 2026-09-08 "ingest bertahap" items, against four rulings taken 2026-09-14.
i5 Identity is `doc_id`; the same document arriving with different bytes is a
NEW VERSION of it, not a new document. That is what keeps an amended file's
approvals attached - entity_id derives from doc_id, which does not move.
The reuse check moves to find_document_by_hash (any version): computing the
next version first and then looking it up would miss the row that already
holds those bytes and re-parse a document we already have, turning the
idempotency guarantee into a paid no-op. The artifact's own `version` is
ignored - parsing stamps 1 on every parse and cannot know what the tenant
already holds.
i2 A term this company has already approved is no longer re-extracted. One LLM
call per cluster is the entire cost of the glossary branch, so every
recognised term is a call not made - and it is also what stops one concept
arriving as term 1.1 and term 1.2. Ids s…
From the 2026-09-08 discussion. `label` now carries a TYPE_SUBJECT key -
GLOSSARY_PA - so an agent that meets an unknown term writes the key and hits
the row: no scan, no embedding, no model call. Searching a list is O(N),
searching a dictionary is O(1), which was the ask.
m1 `src/knowledge_extraction/keys.py`. Prefixes come from the `kind` column
vocabulary (English, singular) so the system holds one spelling per branch
rather than two to keep in sync. Subjects cap at 40 chars, cut on a word
boundary. A rule has no canonical surface of its own, so it borrows the
term it governs plus a short digest (RULE_PA_C07BE4); asking the model to
name it is the option ids.py already rules out, and a model-written
rule_id was a real defect here.
m2 A key is a LABEL, never an identity. `entity_id` stays the hash an
expert's approval hangs on; a key is built from a surface and moves when
clustering canonicalises differently. Pinned by a test: one term from two
…
[NOTICKET] fix: add a fault-code label, measured; re-diagnose the Komatsu tables
Rifqi Hafizuddin+1: Claude Opus 5
366697c
The Komatsu manual prints 27 service codes ([CA559], [CB2249]) in plain
prose and the pipeline returned ZERO mentions for them. Not a parsing
problem -- the codes are in `Chunk.text` verbatim. No label was asking:
a code is not a component, a symptom, a check or an action, so neither
`broad` nor the parked `diagnostic` set proposed one. GLiNER finds them
readily once asked, 6 of 6 on the probe chunk at 0.54-0.69 against a
0.25 floor.
`fault code` added to `diagnostic`, plus a `broad_codes` A/B arm. It is
deliberately NOT in `broad`, because a label is not additive -- GLiNER
runs flat NER and a new label competes for spans:
mentions clusters code clusters E1c_strict
BUMA 251 -> 200 86 -> 67 0 -> 0 0.8537 -> 0.7805
Komatsu 449 -> 548 93 -> 161 0 -> 17 --
Open Pit 787 -> 711 427 -> 377 0 -> 0 --
BUMA loses 51 mentions, 19 clusters and three gold terms; Open Pit loses
76 mentions…
[NOTICKET] feat: pull one linked partner chunk into the evidence window
Rifqi Hafizuddin+1: Claude Opus 5
5d51025
0.4.0 pairs a table chunk with the prose that introduces it
(`referenced_by`, both directions) and nothing consumed the pointer.
Measured what it would buy, ranking only on the three v2 artifacts --
free, no GLiNER, no spend:
clusters top-5 touches a pair BOTH sides only ONE
BUMA 83 31 0 31
Komatsu 93 55 0 55
Open Pit 427 70 0 70
Zero of 603 clusters saw both sides. The ranker never co-selects a table
and the prose that names it, so the partner is unseen content rather than
a duplicate -- and for a bare grid that prose is often its only title
("Troubleshooting of engine (S-mode)" above "S-1 | 6"). Cost at the cap
of 1: evidence read +14.5%/+24.5%/+9.9%, i.e. +1.6%/+4.1%/+4.2% of total
prompt spend; Open Pit moves 71% -> ~74% of the 2M ceiling.
The expansion lives in `top_k`, not `rank_evidence`, for two rea…
[NOTICKET] fix: retire escalation, budget windows by subword (Finding 6 + the over-cap window)
Rifqi Hafizuddin+1: Claude Opus 5
1faca81
Two measurements, two constants.
Finding 6 asked for a number either way on escalation. Counted off all
six persisted paid runs (v1 K=3 and v2 K=5 x three documents):
glossary calls clusters extra definitions from escalation
BUMA v1 69 66 3 0
BUMA v2 83 83 0 0
Komatsu v1 94 75 19 0
Komatsu v2 102 93 9 0
Open Pit v1 361 349 12 0
Open Pit v2 430 425 5 0
48 escalation calls, and not one produced a definition the first round
had missed. `extraction_status == "escalated"` is set only when a later
round succeeds where round 0 failed, and it appears zero times in every
file. Why it cannot work as built: escalation hands the model the NEX…
[NOTICKET] fix: gate spend in cluster units and surface the estimate (Step 6)
Rifqi Hafizuddin+1: Claude Opus 5
cdb5fe8
Found on the way in, and it is the bigger half of this commit: the
pre-flight ceiling was gating on a PARTIAL number. `estimate_cost`
summed the glossary branch alone while reporting a field named
`estimated_prompt_tokens`, so every consumer read a part as a whole.
Measured on the three v2 artifacts, free:
clusters calls old (glossary) new (all four) undercount
BUMA 83 106 170,466 201,751 18.4%
Komatsu 93 95 208,993 227,329 8.8%
Open Pit 427 494 1,354,283 1,560,548 15.2%
The Open Pit chapter is at 78.0% of the 2M ceiling, not 71%.
Adds a SECOND ceiling on cluster count, which is what actually drives
spend -- one LLM call per term cluster -- and unlike the token figure is
exact and known for free. Two things the token ceiling cannot see: the
cache hit rate degrades as a run grows, so the real bill rises faster
than the estimate; and part of that estimate is…
[NOTICKET] feat: assert on invariants, report a band, classify the genre (Steps 0 + 5)
Rifqi Hafizuddin+1: Claude Opus 5
f1d64b1
Step 0 -- measurement hardening. `temperature=0` is argmax over logits a
shared inference server does not hand back identically twice, and this
corpus has a recorded case of identical input scoring 6/7 one day and
0/12 the next. Three changes follow from that:
* `eval/knowledge/invariants.py` asserts on FIELDS AND INVARIANTS, never
on exact text: a span that cannot be located verbatim, a missing or
duplicate entity id, a rejected field that still carries a value
(rejected-never-repaired is only checkable from outside), a page out
of range, queue ranks that are not contiguous, a leaked sort key. A
violation means a control leaked, so it fails the run (exit 5)
regardless of how the scores looked.
* `--runs N` repeats ONLY the paid stage -- the free stages are
deterministic -- and reports min/median/max with the MEDIAN as the
value, because one failed run drags a mean into a region no run
occupied. It also checks every run minted identical entity ids, which
is the K11…
[NOTICKET] fix: cap key-parameter pre-emption in the review queue (Step 2)
Rifqi Hafizuddin+1: Claude Opus 5
4e3a6ae
Unresolved key parameters held their own sort tier beneath conflicts, so
EVERY one of them outranked EVERY routine term regardless of how many
there were. The branch emits them in bulk on exactly the documents it
understands least, so the promotion fired hardest where it was least
deserved: on the Komatsu shop manual that is 24 junk rows standing in
front of `Engine`, a term with 108 mentions, at rank 25.
The original reason for promoting them is preserved and still sound --
that branch has no span check, so an unresolvable surface is the one
hallucination signal it can raise and it must not sink beneath sixty
routine rows. A CAP expresses that without letting the tail take the
queue over: at most one parameter per five term rows.
Also heads the parameter name. The branch picks the right CONCEPTS and
returns them wrapped in their whole defining sentence (185 chars in one
measured case); `term` now carries the head and the new `surface_full`
carries the untruncated text. Split only pa…
[NOTICKET] fix: test the caption hypothesis; ship the signal at weight 0.0
Rifqi Hafizuddin+1: Claude Opus 5
36a166e
The first paid v2 run produced `definitions_citing_a_caption = 0` on all three
documents. Two candidate explanations. Both are now measured and BOTH ARE FALSE.
1. "Captions never reach the term filter."
FALSE. A caption is part of Chunk.text, so GLiNER already scans it and
already produces mentions from it: 2 of 2 captions on BUMA, 3 of 8 on Open
Pit. Feeding captions to the filter would have been a no-op.
2. "Figure-bearing chunks are never selected as evidence."
FALSE. A captioned chunk is already inside the top-5 for 22 of 83 BUMA
clusters and 64 of 427 Open Pit clusters.
To test (2) properly this adds the lever that genuinely was missing: an
`asset_caption_names_term` ranker signal. A/B at 2.5 vs 0.0 across all three
documents changed the top-5 of ONE cluster out of 603, and moved zero clusters
from "no figure" to "has a figure".
The real reason is visible by reading them: not one caption in the corpus is a
definition. They are figure titles --
"Figure 1.3. Re…
[NOTICKET] feat: parsing schema 0.4.0 — asset pointers for figures and tables
Rifqi Hafizuddin+1: Claude Opus 5
e662ac0
A figure used to become its own chunk with an EMPTY `text`, which made it
unreachable by construction:
no text -> no term mentions -> joins no cluster -> never selected as evidence
Measured across three documents: 13 figures, every one already described by a
vision model we pay for, and ZERO of them appearing in any cluster's evidence.
Meanwhile the prose that said "Figure 1.1" had its  reference stripped
and the caption landed in a NEIGHBOURING chunk, so nothing connected the
sentence to the picture.
Now the reference is folded into the prose chunk at its reading-order position
and the figure travels as an `Asset` on that chunk.
image chunks 2/3/8 -> 0/0/0
dangling asset:// refs 0
description leaks 0
captions kept in text 17/17 (Open Pit) — unchanged
E1 / E1c identical v1 -> v2; recall did not move
Tables deliberately DO NOT change: unlike a figure, a table's linearised text
earns its evidence slot (74.6% of the Komatsu manual's t…
`rapidfuzz.token_set_ratio` returns exactly 100 whenever one string's token set
is a SUBSET of the other's, so a head noun always absorbed its own compounds and
FUZZY_THRESHOLD=92 was inert. Measured across three documents (BUMA standard,
Komatsu shop manual, Open Pit textbook Ch.1), the pass made 140 merges:
131 token-subset matches -> all wrong
6 singular/plural pairs -> correct, now folded deterministically
3 gain/loss pty|qty|ua -> wrong; three distinct metrics
94% of its output was wrong and it produced no correct merge that plural folding
does not now do. Matching is exact (plural-folded) -> abbreviation<->expansion.
token-subset merges 15/18/97 -> 0/0/0
E1c_strict (BUMA) 0.8293 -> 0.8537 (`EWH`, absorbed by `Working Hours`, recovered)
After the fix BUMA's remaining unowned gold terms are all ones the FILTER never
proposed, so clustering destroys nothing.
`normalize(lemma=True)` is scoped to the clustering key path only. rank/evidence.py
imports `norm…
