Commit History
-
[NOTICKET] refactor(knowledge-extraction): BriefContext -> DomainContext
Rifqi Hafizuddin +1: Claude Opus 5The workstream diagram has always called this box "Domain Knowledge", while the model and its fields described a short orientation card. Rename the entity to match the intent before anything is persisted: once an expert has approved rows keyed on these ids, changing the `b_` prefix is an id migration that loses the approvals - the exact failure the deterministic-id scheme exists to prevent. BriefContext -> DomainContext brief_id -> domain_id (prefix b_ -> d_) ExtractionResult.brief -> .domain brief_context.json -> domain_context.json Deliberately unchanged: - `Branch = "summary"` and GUARDED_FIELDS["summary"]. The branch literal is the measurement key in CallUsage.branch / RejectedField.branch; renaming it would break comparability with the committed eval baseline. - prompts/summary.txt. Editing prompt text needs the eval path, and the rewrite belongs with the v3 schema change, not with a rename. No field, prompt or guarding change - the …7f47c61
-
[NOTICKET] docs(knowledge-parsing): naming convention, and why it is a stopgap
The file name is currently a primary key, not a label: `doc_id` is the file stem, `chunk_id` is built from it, and `term_id` / `formula_id` / `rule_id` / `brief_id` are all digests of `doc_id` in `knowledge_extraction/ids.py`. Renaming a file after ingestion therefore re-keys every identifier an expert's approve/reject decisions hang on. That was worth writing down, because nothing in the code says it. The convention itself is ordinary — stable doc code, machine-readable revision suffix, no spaces, never `(1)`/`(2)`, never rename after ingestion. `(2)` gets its own line because it means "downloaded twice", not "version 2", and ingested as-is it becomes a second legitimate document with a full set of identifiers. What matters more is the section saying it is a STOPGAP. It holds only because we control the file names while parsing is an admin-triggered batch. Behind the upload endpoint that is planned after eval and the playground it stops holding: people will upload `Copy of SOP final…942f14e
-
[NOTICKET] docs(knowledge-parsing): settle determinism; finish the English pass
Determinism was the one thing left open on the backend decision: the VLM family is not deterministic in general, so a frozen baseline might drift and no eval number could be trusted. It does not drift. Two runs over the same documents, with the parse cache moved aside so both genuinely re-parsed (from_cache: false on all four records), produced 0 differences across 50 chunks. All four image descriptions were present in both runs and character-for-character identical (192, 256, 77, 146 chars) — those are the only model-written content in the artifact and were the likeliest to wobble. Existing baselines therefore stay valid and need no re-measurement, and hybrid/high is final rather than provisional. Marked settled in config.py, calibration §13, and REPO_STATUS §14. The parsing README was still Indonesian in the how-to-use sections written last week, while the .py files had already moved to English; it is now English throughout. Its console example was also stale — it showed the old I…d5ed50c
-
[NOTICKET] docs(knowledge-parsing): how to run it, and how to read what comes back
The README still said `python -m src.knowledge_parsing.run` with no environment set, and pointed at `uv sync --extra knowledge-parsing` — a command that fails. Four sections that answer questions actually asked this week. **How to run it**, and what a successful run looks like, with real console output. First-time readers cannot tell whether `⚠ 3 peringatan` is routine or alarming, and that is the one line worth reading: warnings are normal, `ok (cache)` means nothing was re-parsed, and only `ok / gagal` needs attention. **Backend**, with the measured table and why hybrid/high won — including that `vlm` is identical but slower and `pipeline` fuses the multiplication sign. Stability is marked open, not settled. **Running without a GPU**: the runner drops to `pipeline` on its own, and the result is valid but LOWER FIDELITY — no image descriptions, multiplication copied as a letter. Easy to forget, so it is stated where it will be seen. **Installing MinerU**, with an explicit do-not-…b3eae0f -
[NOTICKET] docs: reorganise into docs/ and give every doc a status banner
Rifqi Hafizuddin +1: Claude Opus 5The root carried 14 markdown files and no way to tell which were authoritative. That knowledge lived in CLAUDE.md §1 and in Rifqi's head, which works for us and fails for anyone else — and one of the stale ones already has a named failure mode ("The Stale-Doc Believer") for being implemented from. Root now holds four: REPO_STATUS, DEV_PLAN, API_CONTRACT_BE_PYTHON, README. Everything else moves under docs/, by git mv so history follows: docs/knowledge/ context, TODO, calibration, parsing output contract docs/plans/ SPINE_V2_PLAN (approved, W3 deferred), DS_TOOLS_PLAN (proposal) docs/archive/ API_ENDPOINTS_RESTRUCTURE, CODE_REVIEW_2026-07-23, PROJECT_BRD docs/reference/ API_CONTRACT_BE_GOLANG (the Go repo's, a copy that can drift) Every doc gains a one-line status banner under its H1, marked <!-- doc-status -->: LIVE / LIVE PLAN / PROPOSAL / ARCHIVED / REFERENCE. Archived ones name what superseded them — API_ENDPOINTS_RESTRUCTURE now opens with "DO NOT IMPLEMENT FROM…f1342ce
-
[NOTICKET] docs(knowledge-parsing): module README + refresh the parsing/seam statuses
README follows the per-module convention already used by src/catalog, src/query, src/retrieval and src/security, and records the three things that are easy to get wrong and expensive to discover: text stays verbatim, page_idx is 0-based, and markup never goes in text (with the recall numbers behind that last one). KNOWLEDGE_PIPELINE_TODO.md — parsing (B) and seam (S) rows only; P/X/D/E/F and S7/S8 are untouched: - B6, B8, S2, S3, S4 -> done. Contract landed, envelope carries parser identity read back from MinerU's _middle.json, section-structure claim corrected against MinerU's source, source_wording added. - B4, B5, S1, S5, S6 -> in progress, deliberately NOT marked done: B5's acceptance test needs the Open Pit textbook (S7) before the NSR cross-page case can be claimed; S5 is satisfied on the parsing side but is Rifqi's to close; B4's language question is answered but text quality is not yet judged. Marking unproven work done is what makes a checklist useless. - S6: (c)…f0cf168