[NOTICKET] fix(knowledge_extraction): close the notation rules against real backend markup
Rifqi Hafizuddin+1: Claude Opus 5
01c76dd
Sofhia supplied the same 11 equations as MinerU transcribed them under four
backends -- 44 real strings, committed as a fixture. Running the shipped
normaliser over them reproduced her figures exactly: 11/11 for vlm/medium,
10/11 for hybrid/medium, 8/11 for pipeline, against hybrid/high as reference.
Her independent check and this code agree on both the counts and which
equations differ, which is the point of her having run it separately.
The remaining gaps were in the command inventory, and the inventory is now
evidence rather than guesswork.
\mathsf and \displaystyle occur in this corpus and were missing -- \mathsf sets
the whole of equation 7 under pipeline, including its multiplication sign, and
\displaystyle appears in equation 4. Both are handled now, and the wrapped-x
rule generalises across every presentational wrapper rather than being \mathrm
specific.
Equation 4 needed two more, and one of them was a decision Sofhia deliberately
left open. The array ALIGNMENT SPEC arrives…
[NOTICKET] feat(knowledge_extraction): record which guarantee formula_latex carries
Rifqi Hafizuddin+1: Claude Opus 5
f81a227
Three paths leave formula_latex populated and they are not equally strong:
located in the source markup, no markup to check against, or an operator the
source cannot express. Until now the output could not tell them apart, so a
verified formula and an unverifiable one reached the expert as identical rows.
That gap is not theoretical. MinerU's pipeline backend writes multiplication as
a bare letter x, so every product formula from such an artifact is unverifiable
-- including the Production formula this contract uses as its own worked
example. Every artifact on Sofhia's machine is pipeline, because she has no GPU.
FormulaEntry gains latex_verification: verified | unverified_no_markup |
unverified_operator, null when there is no claim to characterise. Set by the
span check, never by the model -- the model cannot know which guard ran, and a
field it could write would be a claim rather than a record.
_mark_latex is a never-throw seam like the rest of validation: an entry type
without th…
[NOTICKET] fix(knowledge_extraction): compare formula_latex by notation, not identity
Rifqi Hafizuddin+1: Claude Opus 5
a998ea4
9c8a6e4 gave formula_latex the right haystack and the wrong test. Sofhia ran it
against real artifacts and found `Production = MOHH \times Qty \times ...`
rejected: the `pipeline` backend transcribes the multiplication sign as a bare
letter x, so the claim had nothing to match.
Probing further, the problem was wider than the case she hit. Against a source
that DOES carry \times, the guard also rejected a model that dropped \mathrm{},
one that used \cdot, and one that wrote idiomatic LaTeX -- 4 of 5 plausible
correct outputs. It passed only when the model mirrored MinerU's notation
character for character. The test that shipped green passed only because I
hand-wrote the fixture to mirror the source.
That is the same category error as the original defect, one level up. First the
haystack was the wrong FORMAT; then it was the wrong NOTION OF CORRECTNESS. The
prompt asks the model to transcribe INTO LaTeX, which is a translation, and
identity is not how you check a translation.
normalis…
[NOTICKET] fix(knowledge_extraction): formula_latex is checked against Chunk.latex
Rifqi Hafizuddin+1: Claude Opus 5
9c8a6e4
The field was guarded in name only. It sat in GUARDED_FIELDS but not in
_TRANSCRIBED, so its value was never searched in the source -- only its
provenance.span was. A structurally wrong transcription with a valid prose span
passed, which is exactly the shape X21 produced: a \frac that lost its division
reads as a product and states something the document does not.
Adding the field to _TRANSCRIBED was not the fix. `text` carries rendered prose
and can never contain LaTeX, so that would have nulled every formula entry and
presented as a model failure rather than a validation bug. The haystack was the
problem, not the check.
Chunk.latex is now the haystack. The parsing half has always emitted it;
extraction's adapter dropped it on the floor, which is what left the claim
unprovable. It now crosses the seam and is compared with whitespace removed
ENTIRELY rather than collapsed.
That is a different notion of equality, not a weaker one. In LaTeX, whitespace
outside \text{} is not significa…
Harry's review of the output contract asked for four things: an id on every
entity, a formula referenced rather than restated, rules related to formulas,
and a smaller schema. All four are adopted.
Two defects surfaced while adding the ids, and they matter more than the ids do.
Neither existing identifier survived a re-run: `cluster_id` was a frequency-rank
ordinal (`#c000` over a list sorted by mention count), so one changed count
reshuffled every id after it, and `rule_id` was written by the model, so the same
rule could return a different slug next run. The product is an expert working a
review queue — if Mas Beta approves 40 entries and a prompt is retuned, unstable
ids lose which 40. Approval continuity, not linking, is why these exist.
New `ids.py` mints term/formula/rule/brief ids as sha256 over content: never
positional, never model-supplied. New `link.py` resolves every reference after
the branches run — deterministic, no LLM, no spend — and returns dangle counts
rather than…
Completes every stage that does not depend on the seam. The pipeline now runs
end to end: artifact -> filters -> cluster -> rank -> LLM -> validate -> diff ->
frequency-sorted review queue.
extract/ client on the nano __54n quad; structured output is PROBED not assumed
(json_schema, falling back to json_object + validate-retry, recording
which applied); cached tokens read from the API, never modelled. Four
branches, one call per TERM CLUSTER. Prompts stay as files so the
fixed prefix is byte-identical and caching keeps engaging.
validate/ span check (whitespace-only normalisation, never repairs, unlocatable
provenance rejects every guarded field), escalation, conflict
detection by token overlap — explainable to the reviewer, and it
never picks a winner.
diff/ new / duplicate / conflicting against an explicitly supplied active
glossary, so the duplicate and conflicting paths are reachable. The
…
