Commit History
-
[NOTICKET] refactor(knowledge-parsing): English throughout the module
The parsing half was written in Indonesian — identifiers, docstrings, comments, manifest keys and console output. Everything else in the repo is English, and this module is the newest work, so it is the one to bring into line rather than the older code around it. Renamed across the module: parse_dokumen -> parse_document, normalisasi -> normalise, pulihkan_batas_kata -> restore_word_boundaries, kosakata_layak -> trustworthy_vocabulary, periksa_items -> check_items, CatatanDokumen -> DocumentRecord, and the locals with them. Docstrings and comments were REWRITTEN rather than translated word for word. Several carry the reasoning behind a decision — why `\frac` is unwrapped before the braced wrappers, why a heading never enters `text`, why the digit threshold is 3 — and a mechanical translation would have flattened exactly the part worth keeping. Manifest keys move too: detik_per_halaman -> seconds_per_page, peringatan -> warnings, jenis -> kind, galat -> error. That is a data change, …7f153d6
-
[NOTICKET] fix(knowledge-parsing): send MinerU a backend name it accepts
The GPU flip config.py advertises did not work. MinerU dispatches on `backend == "pipeline"`, then `startswith("vlm-")` and `startswith("hybrid-")`. A bare "vlm" matches none of them, so MinerU did nothing at all, silently — and the failure surfaced here as "content_list not found", which points at the wrong place entirely. Valid names are pipeline / vlm-engine / hybrid-engine. The short names stay in the config because they are part of the cache key and read better; the translation happens at the boundary, just before MinerU is touched. Nobody would have caught this until the day a GPU arrived and everyone was waiting on the result. Also: the temp directory used with_suffix(), which REPLACES the last suffix rather than appending. Cache folders end in the MinerU version ("...-3.4.4"), so ".4" was thrown away and two versions collided on one temp directory that the next line deletes without asking. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>8e93c60
-
[NOTICKET] feat(knowledge-parsing): document parsing half of the knowledge pipeline
Implements §4 B2/B5/B6 and closes the four open seam items (S2, S3, S4, S6b/c/d) against the contracts.py draft circulated 2026-08-19. Additive and flag-gated: src/knowledge/ (Tesseract -> pgvector) is untouched. Seam — src/knowledge_parsing/contracts.py, pydantic per S6e: - ParsedDocument envelope: content_hash, n_pages, version, parser_name/version/ backend/config, schema_version, raw_output_dir. parser_backend is read from MinerU's _middle.json, so it records what actually ran, not what was configured — a MinerU change and a prompt change stay distinguishable (S2). - Heading structure is READ, not derived. Verified in MinerU's source: pipeline and vlm run identical text_level logic, and bbox is emitted by both. Heading availability is document-dependent, not backend-dependent — the earlier claim came from sampling the McGraw-Hill handbook, which has no numbered headings (S3). - source_wording on TermRecord; heading carried verbatim (S4). - page_idx/page_idxs, 0-based, n…5840d20