Commit History
-
[NOTICKET] docs(knowledge-parsing): settle determinism; finish the English pass
Determinism was the one thing left open on the backend decision: the VLM family is not deterministic in general, so a frozen baseline might drift and no eval number could be trusted. It does not drift. Two runs over the same documents, with the parse cache moved aside so both genuinely re-parsed (from_cache: false on all four records), produced 0 differences across 50 chunks. All four image descriptions were present in both runs and character-for-character identical (192, 256, 77, 146 chars) — those are the only model-written content in the artifact and were the likeliest to wobble. Existing baselines therefore stay valid and need no re-measurement, and hybrid/high is final rather than provisional. Marked settled in config.py, calibration §13, and REPO_STATUS §14. The parsing README was still Indonesian in the how-to-use sections written last week, while the .py files had already moved to English; it is now English throughout. Its console example was also stale — it showed the old I…d5ed50c
-
[NOTICKET] refactor(knowledge-parsing): English throughout the module
The parsing half was written in Indonesian — identifiers, docstrings, comments, manifest keys and console output. Everything else in the repo is English, and this module is the newest work, so it is the one to bring into line rather than the older code around it. Renamed across the module: parse_dokumen -> parse_document, normalisasi -> normalise, pulihkan_batas_kata -> restore_word_boundaries, kosakata_layak -> trustworthy_vocabulary, periksa_items -> check_items, CatatanDokumen -> DocumentRecord, and the locals with them. Docstrings and comments were REWRITTEN rather than translated word for word. Several carry the reasoning behind a decision — why `\frac` is unwrapped before the braced wrappers, why a heading never enters `text`, why the digit threshold is 3 — and a mechanical translation would have flattened exactly the part worth keeping. Manifest keys move too: detik_per_halaman -> seconds_per_page, peringatan -> warnings, jenis -> kind, galat -> error. That is a data change, …7f153d6
-
[NOTICKET] feat(knowledge-parsing): document parsing half of the knowledge pipeline
Implements §4 B2/B5/B6 and closes the four open seam items (S2, S3, S4, S6b/c/d) against the contracts.py draft circulated 2026-08-19. Additive and flag-gated: src/knowledge/ (Tesseract -> pgvector) is untouched. Seam — src/knowledge_parsing/contracts.py, pydantic per S6e: - ParsedDocument envelope: content_hash, n_pages, version, parser_name/version/ backend/config, schema_version, raw_output_dir. parser_backend is read from MinerU's _middle.json, so it records what actually ran, not what was configured — a MinerU change and a prompt change stay distinguishable (S2). - Heading structure is READ, not derived. Verified in MinerU's source: pipeline and vlm run identical text_level logic, and bbox is emitted by both. Heading availability is document-dependent, not backend-dependent — the earlier claim came from sampling the McGraw-Hill handbook, which has no numbered headings (S3). - source_wording on TermRecord; heading carried verbatim (S4). - page_idx/page_idxs, 0-based, n…5840d20