Rifqi Hafizuddin
Claude Opus 5
[NOTICKET] fix: add a fault-code label, measured; re-diagnose the Komatsu tables
366697c | # Table branch — design proposal (Step 8) | |
| <!-- doc-status --> | |
| > 💡 **PROPOSAL — nothing is built from this.** Written 2026-09-11 against the open queue in | |
| > `knowledge_pipeline_runs/SESSION_HANDOFF.md` §4.6. Needs Rifqi's sign-off before any code. | |
| > Roll-up: [DEV_PLAN.md](../../DEV_PLAN.md) §0.8 · built state: [REPO_STATUS.md](../../REPO_STATUS.md) §14. | |
| **Owner:** Rifqi · **Size:** weeks · **Depends on:** parsing 0.4.0 (shipped), genre classification | |
| (shipped 2026-09-11, advisory) · **Blocks:** any useful result on diagnostic documents. | |
| --- | |
| ## 1 · The problem, stated as a number | |
| The Komatsu HD785-7 troubleshooting manual is **74% tabular** — 59 tables across 80 chunks — and the | |
| pipeline extracts **zero** knowledge entries from it. Two paid runs, v1 and v2, both returned **0 | |
| definitions** from 75 and 93 clusters respectively. That is not a tuning failure and no prompt change | |
| reaches it: a shop manual does not define its terms, it **asserts relationships between them**, and | |
| it does so in a matrix. | |
| The genre classifier (`src/knowledge_extraction/genre.py`) now names this document `diagnostic` and | |
| reports `well_served = False` before anything is spent. This proposal is what would make that answer | |
| change. | |
| ## 2 · What the document actually contains | |
| A troubleshooting table is a **symptom × cause matrix**. Each marked cell is one assertion: | |
| | Symptom \ Cause | Fuel filter clogged | Injector worn | Air cleaner blocked | | |
| |---|---|---|---| | |
| | Engine does not start | ● | ● | | | |
| | Black exhaust smoke | | ● | ● | | |
| | Output drops | ● | | ● | | |
| Three assertions come out of the first row alone, and **none of them is a definition**: | |
| > *"Engine does not start" may be caused by "Fuel filter clogged".* | |
| The unit of knowledge here is the **(row header, column header, cell mark) triple**, and it is | |
| recoverable **deterministically** — the mark is in the parsed HTML, the headers are in the parsed | |
| HTML, nothing needs to be inferred. | |
| ## 3 · The shape of the branch | |
| **Deterministic extraction, LLM only to phrase.** This is the inversion of every existing branch, and | |
| it is the whole point: the assertion's *existence* is read off the table, so the model cannot | |
| fabricate a relationship that is not in the document — it can only phrase one that is. | |
| ``` | |
| table chunk ──► parse table_html ──► (row header, col header, cell mark) triples | |
| │ | |
| ├── deterministic: the assertion EXISTS | |
| │ | |
| ▼ | |
| one LLM call per TABLE (not per cell) | |
| │ | |
| ▼ | |
| natural-language phrasing + typing | |
| │ | |
| ▼ | |
| span check against the cell's own headers | |
| ``` | |
| **Provenance points at the cell**, which is stronger than anything the pipeline produces today: the | |
| span is the row header plus the column header, both verbatim document text, both locatable by the | |
| existing `span_present`. | |
| ### 3.1 · New entity | |
| ```python | |
| class RelationEntry(BaseModel): | |
| relation_id: str # sha256 over (doc_id, table asset_id, row, col) — positional | |
| # within a table, but the table's asset_id is content-addressed, | |
| # so a re-parse of identical bytes re-mints identical ids. | |
| kind: Literal["symptom_cause", "component_action", "condition_effect"] | |
| subject: str # verbatim row header | |
| object: str # verbatim column header | |
| statement: str | None # model-phrased; span-checked against subject + object + caption | |
| table_asset_id: str | |
| provenance: Provenance # span = the two headers; page from the chunk | |
| term_ids: list[str] # links into the glossary where headers match known terms | |
| ``` | |
| ### 3.2 · Why one call per table, not per cell | |
| Komatsu's 59 tables carry on the order of 10³ marked cells. One call per cell is a different cost | |
| class entirely and would blow the pre-flight ceiling by itself. One call per table hands the model | |
| the linearised table plus the extracted triples and asks it to phrase each one — 59 calls, which is | |
| **less than the 93 the glossary branch currently spends to abstain 93 times.** | |
| ## 4 · What has to be decided before coding | |
| | # | Question | Why it blocks | | |
| |---|---|---| | |
| | T1 | Is a relation a first-class **entry kind** (a fifth `kind` in `knowledge_entries`) or a new table? | Persistence contract + Harry's migration. A fifth kind is additive; a table is a DDL handoff | | |
| | T2 | What does a reviewer **do** with 1,000 relations? | The review queue is frequency-sorted and finishable by construction. 1,000 relations is not finishable. Likely answer: review per TABLE, not per relation — one approve covers the matrix | | |
| | T3 | Does a mark type carry meaning (`●` vs `○` vs `▲`)? | Komatsu uses more than one glyph. If they mean "primary/secondary cause" the triple is really a quad, and getting this wrong silently inverts assertions | | |
| | T4 | Which `table_html` shapes does the parser actually emit — merged cells, multi-row headers? | Determines whether header resolution is 20 lines or a small parser. **Measure before designing** | | |
| | T5 | Does this ship behind the genre router (`diagnostic` only) or for every document? | BUMA is 25% tabular and its tables are *data*, not relation matrices. Running this on them produces noise | | |
| **T4 is the one to do first** and it is free: inspect the 59 Komatsu tables and count the shapes. | |
| ## 5 · Kill lines, agreed before the first paid run | |
| Copying the discipline that made the clustering defect findable — a metric that moves the *wrong* way | |
| is worse than no metric (E2), so these are stated as recall against a **human-read sample**: | |
| | Metric | What it measures | Kill line | | |
| |---|---|---| | |
| | **T-E1** | Triples extracted ÷ triples a human reads off the same 5 tables | ≥ 0.90 — it is deterministic; anything less is a parser bug | | |
| | **T-E2** | Phrasings whose span check passes | ≥ 0.95 | | |
| | **T-E3** | Phrasings a human judges to *invert or distort* the relation | 0 — this is the one that matters, and it is the reason the mark glyph question (T3) is not cosmetic | | |
| | **T-cost** | Calls per document | ≤ 1 per table | | |
| **T-E3 has no automated proxy.** It needs Mas Beta or an equivalent reader on a sample, which makes | |
| the R4 conversation (does "found but undefined" help an expert?) worth having in the same sitting. | |
| ## 6 · What this does NOT solve | |
| A relation matrix is not a glossary. After this branch ships, the Komatsu manual still yields **zero | |
| definitions** — correctly, because it contains none. If the product's promise is "a glossary from | |
| your documents", a diagnostic document will always disappoint it, and the honest place to say so is | |
| the ingest response, which the genre classifier now makes possible. | |
| --- | |
| ## 7 · Groundwork already landed (2026-09-11) | |
| Two pieces of this branch's plumbing exist before the branch does, and both are measured: | |
| **The pointer is populated and durable.** 0.4.0 gives every table chunk an `Asset` whose id comes | |
| from `synthetic_asset_id(doc_id, chunk_id, item_index)` — stable document coordinates, so it survives | |
| a re-parse — plus `referenced_by` naming the prose chunk that introduces it, and the reverse pointer | |
| on that prose chunk's `assets`. | |
| **One linked partner now joins the evidence window** (`EVIDENCE_LINK_PARTNERS = 1`, in `rank/evidence.py`). | |
| The measurement that motivated it: across 603 clusters on the three test documents, **zero** had both | |
| sides of a linked pair in their top-5, so a table reached the model without the prose naming it, and | |
| vice versa. 31/83, 55/93 and 70/427 clusters now gain a partner, at +1.6%/+4.1%/+4.2% of prompt spend. | |
| **What this does *not* do, and the branch must:** `table_html` crosses the seam into | |
| `Chunk.table_html` and **nothing reads it** — the prompt shows the linearised `render_table` text | |
| from `Chunk.text` instead. That is deliberate: the span check validates against `heading + text` | |
| only, so putting HTML in front of the model would let it quote markup that is then nulled as | |
| unlocatable. A branch that wants the real cell structure has to extend the span-checked text too, or | |
| carry provenance that points at a cell rather than a character span (§3.1's `cell_ref` does the | |
| latter, which is the cleaner of the two). | |
| --- | |
| ## 8 · Correction and re-diagnosis of the Komatsu tables (2026-09-11) | |
| > **Correction.** An earlier reading of this corpus (same day, recorded in the session | |
| > notes) said the troubleshooting charts had **lost their column headers**, on the | |
| > evidence that only 4 of 59 tabular chunks contain a `<th>`. That count is accurate | |
| > and the conclusion drawn from it was **wrong**. The headers are present; they are | |
| > emitted as ordinary `<td>` cells in a **separate chunk**. | |
| ### What Mistral actually produces | |
| Each chart is split into three or four sibling tables, and the first cell of each names | |
| the fragment. Across the 59 tabular chunks: | |
| | first cell | count | what it holds | | |
| |---|---|---| | |
| | `Causes` | 17 | **the column headers** — the cause list | | |
| | `Troubleshooting` | 16 | check rows | | |
| | `Questions` | 15 | interview/symptom rows | | |
| | `Remedy` | 10 | the action row | | |
| So the (row header, column header, cell mark) triple is recoverable, but its three parts | |
| live in **different chunks**. The branch's first job is re-assembly, not OCR recovery — | |
| the fragments are adjacent, same-page, and regularly labelled. | |
| ### The defect that is real | |
| **Marks survive in only 20 of 59 tabular chunks.** Doc-wide there are 278 marks | |
| (18 ◎, 137 ○, 25 △, 98 ●), and they cluster on the pages where the matrix fragment | |
| kept them; on pages 9-10 the `Questions` and `Troubleshooting` fragments come through | |
| with row labels and **zero marks**. Column structure and `rowspan`/`colspan` are | |
| preserved where the marks are. | |
| A chart whose marks were dropped carries no symptom-to-cause link at all, so roughly | |
| two thirds of the document's assertions are currently unrecoverable from the artifact. | |
| **That, not the headers, is what gates this branch**, and it has to be fixed at the | |
| parsing layer. | |
| ### Free evidence that the content is reachable | |
| Adding a `fault code` label to the span filter (see `config/labels.yaml`) surfaced | |
| **66 cause phrases** as clusters on this document — *"Clogged fuel filter and/or | |
| strainer"*, *"Improper injection timing"*, *"Defective alternator (regulator section)"* — | |
| plus 17 service codes. Those are the column headers, read out of the `Causes` | |
| fragments. The raw material for the branch is already in the artifact and already | |
| findable; what is missing is the mark matrix that connects it to the symptoms. | |