[NOTICKET] feat(knowledge): two review modes, recorded against a person (r1, r2, r3)
Rifqi Hafizuddin+1: Claude Opus 5
6ef7990
Closes Sistem review. Section 7 landed in dedorch, so the columns are declared
and written; verified read-only through the ORM against live before committing.
r1 `review_mode` on the batch, chosen at ingest. "expert" leaves entries
waiting for a human; "llm" approves them on arrival with NO reviewer step.
Worth being plain about what the second one is: there is no second model
call and no second opinion. The entries were written by a model already,
and this mode means accepting them without a human looking. Default is
"expert" - nobody gets LLM approval by accident.
r2/r3 What makes that offerable rather than reckless is the record.
`decided_by` says whether a human looked; `authorised_by` says on whose
authority it happened. They are separate columns because the admin who
clicks ingest and the expert who reviews can be different people, and in
"llm" mode there is no reviewer at all - so without the clicker's id
reaching the review row the de…
[NOTICKET] feat(knowledge): review diff + the rule/domain boundary (r4, o2)
Rifqi Hafizuddin+1: Claude Opus 5
5cb2e2d
r4 `changed_fields()` and `review_diff()`. The queue can now say WHAT moved
rather than only that something did - which was the ask: show before /
after, do not send the expert back to read the entry from the start.
`changed` is empty when the approval still stands, so the screen can say
"nothing to re-read" instead of rendering two identical panes. `before` is
what the expert actually had in front of them; for an `edited` decision
that is their own corrected text, not the model's claim. A test pins
changed_fields() and has_mutated() to the same answer - two notions of
"has this changed" that can disagree is the same defect as two notions of
readiness.
o2 The rule/domain boundary, written down in the output contract. The two
blurred on screen during the 8 Sep review and a boundary nobody has
recorded is one the prompts cannot hold. The test: if removing it would
change how a NUMBER IS COMPUTED OR SOURCED it is a rule; if it would only
…
The section-6 migration landed in dedorch, so the columns are now declared and
written. Verified read-only through the ORM against live before committing:
knowledge_entries.doc_ids/content/provenance and knowledge_documents.withdrawn_at
all resolve.
m4 `content` - the entry's body, promoted out of payload so a consumer does
not have to open jsonb. Per kind, first non-empty wins, so a kind that
abstained on its primary field still promotes something usable.
m5 `provenance` - a LIST of {doc_id, page, page_no, section_no, chunk_id}.
The extraction contract emits one object; it is wrapped rather than
reshaped, because a list is what a fused entry needs and one shape means
every consumer reads the same thing. The span is deliberately NOT promoted:
it is evidence, not a locator, and duplicating it out of payload would give
the span check two copies to disagree about.
m3 `doc_ids` - derived from doc_id + provenance rather than declared, so it
cannot drift…
The 2026-09-08 "ingest bertahap" items, against four rulings taken 2026-09-14.
i5 Identity is `doc_id`; the same document arriving with different bytes is a
NEW VERSION of it, not a new document. That is what keeps an amended file's
approvals attached - entity_id derives from doc_id, which does not move.
The reuse check moves to find_document_by_hash (any version): computing the
next version first and then looking it up would miss the row that already
holds those bytes and re-parse a document we already have, turning the
idempotency guarantee into a paid no-op. The artifact's own `version` is
ignored - parsing stamps 1 on every parse and cannot know what the tenant
already holds.
i2 A term this company has already approved is no longer re-extracted. One LLM
call per cluster is the entire cost of the glossary branch, so every
recognised term is a call not made - and it is also what stops one concept
arriving as term 1.1 and term 1.2. Ids s…
From the 2026-09-08 discussion. `label` now carries a TYPE_SUBJECT key -
GLOSSARY_PA - so an agent that meets an unknown term writes the key and hits
the row: no scan, no embedding, no model call. Searching a list is O(N),
searching a dictionary is O(1), which was the ask.
m1 `src/knowledge_extraction/keys.py`. Prefixes come from the `kind` column
vocabulary (English, singular) so the system holds one spelling per branch
rather than two to keep in sync. Subjects cap at 40 chars, cut on a word
boundary. A rule has no canonical surface of its own, so it borrows the
term it governs plus a short digest (RULE_PA_C07BE4); asking the model to
name it is the option ids.py already rules out, and a model-written
rule_id was a real defect here.
m2 A key is a LABEL, never an identity. `entity_id` stays the hash an
expert's approval hangs on; a key is built from a surface and moves when
clustering canonicalises differently. Pinned by a test: one term from two
…
[NOTICKET] fix: gate spend in cluster units and surface the estimate (Step 6)
Rifqi Hafizuddin+1: Claude Opus 5
cdb5fe8
Found on the way in, and it is the bigger half of this commit: the
pre-flight ceiling was gating on a PARTIAL number. `estimate_cost`
summed the glossary branch alone while reporting a field named
`estimated_prompt_tokens`, so every consumer read a part as a whole.
Measured on the three v2 artifacts, free:
clusters calls old (glossary) new (all four) undercount
BUMA 83 106 170,466 201,751 18.4%
Komatsu 93 95 208,993 227,329 8.8%
Open Pit 427 494 1,354,283 1,560,548 15.2%
The Open Pit chapter is at 78.0% of the 2M ceiling, not 71%.
Adds a SECOND ceiling on cluster count, which is what actually drives
spend -- one LLM call per term cluster -- and unlike the token figure is
exact and known for free. Two things the token ceiling cannot see: the
cache hit rate degrades as a run grows, so the real bill rises faster
than the estimate; and part of that estimate is…
