Commit History
-
[NOTICKET] docs(knowledge-parsing): record that Chunk.latex is now load-bearing
Until a998ea48 this field was carried but unread — a parser change touching it would have broken nothing. Extraction's formula guard now validates `formula_latex` against it, so that is no longer true, and the contract has to say so before someone tidies it in good faith. The failure mode is the dangerous kind and gets the emphasis: reflowing or dropping this field does not error. The guard simply has less to check against, and claims it can no longer verify are recorded as unverifiable rather than caught. Fewer things fail and nothing looks wrong. The two obligations run OPPOSITE to the rule for `text`, which is why they are stated rather than left to inference. `text` must stay verbatim because the span check searches it literally. `latex` must stay UNCLEANED because extraction canonicalises before comparing — pre-cleaning removes information the canonicaliser was built to absorb. Character-spacing, `\mathrm{}` wrappers and `~` are all expected input, not damage. Also draws the li…a0df6d3
-
[NOTICKET] refactor(knowledge-parsing): English throughout the module
The parsing half was written in Indonesian — identifiers, docstrings, comments, manifest keys and console output. Everything else in the repo is English, and this module is the newest work, so it is the one to bring into line rather than the older code around it. Renamed across the module: parse_dokumen -> parse_document, normalisasi -> normalise, pulihkan_batas_kata -> restore_word_boundaries, kosakata_layak -> trustworthy_vocabulary, periksa_items -> check_items, CatatanDokumen -> DocumentRecord, and the locals with them. Docstrings and comments were REWRITTEN rather than translated word for word. Several carry the reasoning behind a decision — why `\frac` is unwrapped before the braced wrappers, why a heading never enters `text`, why the digit threshold is 3 — and a mechanical translation would have flattened exactly the part worth keeping. Manifest keys move too: detik_per_halaman -> seconds_per_page, peringatan -> warnings, jenis -> kind, galat -> error. That is a data change, …7f153d6
-
[NOTICKET] docs(knowledge-parsing): contract 0.3.0 — image kind + generated_description
Records what the code change added: `image` joins ChunkKind, and `generated_description` carries MinerU's model-written description of a picture. Two things a consumer would otherwise get wrong. Backends disagree on the label — a figure `pipeline` calls a chart, `vlm` calls an image — so branching on one of them silently misses half the cases. And whether descriptions exist at all depends on the backend: never on `pipeline`, disabled outright on `hybrid` medium, on for `hybrid` high and always for `vlm`. An artifact carrying nulls here is not a parsing failure, it is the backend, which is one more reason `parser_backend` has to travel with the data. Also states why the field is kept out of `text`: that field is what the extraction guardrail searches to prove a quote is verbatim, and generated prose there would let a hallucination pass the check built to catch it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>f0e54e9 -
[NOTICKET] docs: reorganise into docs/ and give every doc a status banner
Rifqi Hafizuddin +1: Claude Opus 5The root carried 14 markdown files and no way to tell which were authoritative. That knowledge lived in CLAUDE.md §1 and in Rifqi's head, which works for us and fails for anyone else — and one of the stale ones already has a named failure mode ("The Stale-Doc Believer") for being implemented from. Root now holds four: REPO_STATUS, DEV_PLAN, API_CONTRACT_BE_PYTHON, README. Everything else moves under docs/, by git mv so history follows: docs/knowledge/ context, TODO, calibration, parsing output contract docs/plans/ SPINE_V2_PLAN (approved, W3 deferred), DS_TOOLS_PLAN (proposal) docs/archive/ API_ENDPOINTS_RESTRUCTURE, CODE_REVIEW_2026-07-23, PROJECT_BRD docs/reference/ API_CONTRACT_BE_GOLANG (the Go repo's, a copy that can drift) Every doc gains a one-line status banner under its H1, marked <!-- doc-status -->: LIVE / LIVE PLAN / PROPOSAL / ARCHIVED / REFERENCE. Archived ones name what superseded them — API_ENDPOINTS_RESTRUCTURE now opens with "DO NOT IMPLEMENT FROM…f1342ce