| # Docs index |
|
|
| <!-- doc-status --> |
| > ✅ **LIVE — the map.** Reorganised 2026-08-24. Every doc in this repo carries a **status banner |
| > under its H1**; read it before trusting the body. |
|
|
| **Status legend:** ✅ LIVE (canonical) · 🔄 LIVE PLAN (in flight) · 💡 PROPOSAL (not approved) · |
| ⛔ ARCHIVED (superseded, kept for the record) · 📎 REFERENCE (another repo's, can drift) |
|
|
| --- |
|
|
| ## Start here — repo root |
|
|
| The root deliberately holds only four docs. If you read nothing else, read the first three. |
|
|
| | Doc | Status | Is | |
| |---|---|---| |
| | [../REPO_STATUS.md](../REPO_STATUS.md) | ✅ | **What is built.** Canonical current state | |
| | [../DEV_PLAN.md](../DEV_PLAN.md) | ✅ | **What is in flight.** Sprint plan, task table, locked decisions | |
| | [../API_CONTRACT_BE_PYTHON.md](../API_CONTRACT_BE_PYTHON.md) | ✅ | **The contract of record** for the FE-facing surface | |
| | [../README.md](../README.md) | — | HuggingFace Space card. The YAML frontmatter is load-bearing for deployment — don't restructure it | |
|
|
| `CLAUDE.md` also sits in the root but is **gitignored** (local operating manual, not shipped). |
|
|
| **Trust order when sources disagree:** code > REPO_STATUS / DEV_PLAN > everything else. |
|
|
| --- |
|
|
| ## `knowledge/` — the knowledge-pipeline workstream |
|
|
| Client documents → candidate knowledge entries an expert reviews rather than authors. Two halves |
| (parsing, Sofhia · extraction, Rifqi) meeting at one versioned artifact. |
|
|
| | Doc | Status | Read it when | |
| |---|---|---| |
| | [knowledge/knowledge_pipeline_context.md](knowledge/knowledge_pipeline_context.md) | 🔄 | You need the *why* — ownership split, settled decisions, what not to reopen | |
| | [knowledge/KNOWLEDGE_PIPELINE_TODO.md](knowledge/KNOWLEDGE_PIPELINE_TODO.md) | 🔄 | You want per-task status and owners. **Start here** | |
| | [knowledge/KNOWLEDGE_PIPELINE_CALIBRATION.md](knowledge/KNOWLEDGE_PIPELINE_CALIBRATION.md) | 🔄 | **Before changing any constant.** Every tuned value with the reason it has that value | |
| | [knowledge/KNOWLEDGE_PARSING_OUTPUT_CONTRACT.md](knowledge/KNOWLEDGE_PARSING_OUTPUT_CONTRACT.md) | 🔄 | You're working either side of the seam | |
| | [knowledge/KNOWLEDGE_SURFACE_PLAN.md](knowledge/KNOWLEDGE_SURFACE_PLAN.md) | 🔄 | Persistence + endpoints: who owns what, phased tasks, blockers. Approved 2026-09-01 | |
| | [knowledge/KNOWLEDGE_CONSUMPTION_PLAN.md](knowledge/KNOWLEDGE_CONSUMPTION_PLAN.md) | 💡 | How agents READ the entries (N14). Which agent gets what, and why. **Proposal, nothing built** | |
| | [knowledge/TABLE_BRANCH_PROPOSAL.md](knowledge/TABLE_BRANCH_PROPOSAL.md) | 💡 | Step 8 — the symptom x cause matrix a shop manual actually holds. Deterministic triples, LLM only to phrase. **Proposal, nothing built** | |
| | [knowledge/TERM_HIERARCHY_PROPOSAL.md](knowledge/TERM_HIERARCHY_PROPOSAL.md) | 💡 | Step 7 + R7 — parent/child terms and cross-document identity, designed together. **Touches `entity_id`; read before building** | |
| | [knowledge/KNOWLEDGE_DOMAIN_CONTEXT_V4.md](knowledge/KNOWLEDGE_DOMAIN_CONTEXT_V4.md) | 💡 | What a *real* domain context is. v3's object is a document brief wearing the name. **Proposal, nothing built** | |
| | [knowledge/KNOWLEDGE_PERSISTENCE_CONTRACT.md](knowledge/KNOWLEDGE_PERSISTENCE_CONTRACT.md) | 💡 | The four-table DDL sent to Harry 2026-09-01. **Pending — no ORM written against it yet** | |
| | [knowledge/KNOWLEDGE_DOMAIN_CONTEXT_V3.md](knowledge/KNOWLEDGE_DOMAIN_CONTEXT_V3.md) | ✅ | `DomainContext` design reasoning. **Shipped in full 2026-09-02** — the live shape is the output contract §8 | |
| |
| Roll-up status: DEV_PLAN §0.8 · built state: REPO_STATUS §14 · eval harness: [../eval/knowledge/README.md](../eval/knowledge/README.md) |
| |
| --- |
| |
| ## `plans/` — design plans |
| |
| | Doc | Status | Note | |
| |---|---|---| |
| | [plans/SPINE_V2_PLAN.md](plans/SPINE_V2_PLAN.md) | 🔄 | Approved 2026-07-13; W2+W1 shipped. **W3 deferred — do not start.** Holds the `message_charts` DDL (§4.4) | |
| | [plans/DS_TOOLS_PLAN.md](plans/DS_TOOLS_PLAN.md) | 💡 | Design study only. **Nothing was built from it**; don't implement without sign-off | |
| | [plans/PR0_PR1_TOOL_FIXES.md](plans/PR0_PR1_TOOL_FIXES.md) | 🔄 | Sofhia's six tool fixes. PR-0 merged; PR-1 open. Every claim re-verified against the code, and two of the review's findings narrowed | |
| | [plans/PR2_EVALUATION.md](plans/PR2_EVALUATION.md) | 🔄 | **Read before PR-3.** Four layers, three of them free. Establishes that **14 of 14** tools have zero output-correctness coverage | |
| | [plans/PR3_REPAIR_EDGE.md](plans/PR3_REPAIR_EDGE.md) | 🔄 | The feedback edge (trigger B, budget 1) **gated on measured hit-rate**, plus the determinism fixes that ship regardless | |
| | [plans/PR23_PART2_PLANNER_EVAL.md](plans/PR23_PART2_PLANNER_EVAL.md) | 🔄 | **pr/23 part 2 (2026-09-23).** Planner, router and eval work re-checked against the code: measurement first, then router, retrieve-then-plan, planner structure. **A–E done 2026-09-23** (D4 reverted, F1 optional) | |
| | [plans/UNSTRUCTURED_V2_PLAN.md](plans/UNSTRUCTURED_V2_PLAN.md) | 🔄 | **The demo track (2026-09-23).** Parsing v2 → embeddings, private images via 1h signed URLs, analysis-scoped retrieval, the mixed structured+unstructured path **without a new intent**. Pre-empts the prompt-architecture work | |
|
|
| --- |
|
|
| ## `tools/` — the analytical-tools workstream |
|
|
| The `analyze_*` tool surface: forecast, importance, anomaly, cluster, A/B testing, segmentation, |
| pareto. Tools are **pure functions, no base class** — the "framework" is `spec_builder` + |
| `ml/executor` + the shared chart kit. |
|
|
| | Doc | Status | Read it when | |
| |---|---|---| |
| | [tools/ANALYTICAL_TOOLS_CONVENTIONS.md](tools/ANALYTICAL_TOOLS_CONVENTIONS.md) | ✅ | **Before writing any tool.** The InputModel → ToolSpec flow, the auto-chart convention, `run_fit`, the three registrations, and the rules that aren't negotiable | |
|
|
| Roll-up status: DEV_PLAN §0.9 · built state: REPO_STATUS §9 · chart envelope: [plans/SPINE_V2_PLAN.md](plans/SPINE_V2_PLAN.md) §4.2 |
|
|
| --- |
|
|
| ## `reviews/` — point-in-time assessments |
|
|
| Read-only reviews. Nothing in them is a decision; findings are cited by id elsewhere. |
|
|
| | Doc | Status | Note | |
| |---|---|---| |
| | [reviews/AGENTIC_DESIGN_REVIEW_2026-09-18.md](reviews/AGENTIC_DESIGN_REVIEW_2026-09-18.md) | 📎 | The DS agent against the 2026 state of the art. Findings **D-1…D-12**; the source of PR-0…PR-3 | |
| | [reviews/PROMPT_ARCHITECTURE_2026-09-22.md](reviews/PROMPT_ARCHITECTURE_2026-09-22.md) | 📎 | `planner.md` + `intent_router.md` measured against industry practice. Findings **P-1…P-7 / R-1…R-4**, candidates **C-1…C-9** — **paused behind the unstructured v2 demo work** (added 2026-09-23) | |
|
|
| --- |
|
|
| ## `testing/` — what running the thing actually found |
|
|
| End-to-end dry-runs of the `structured_flow` path against real sources. These runs are |
| expensive (~170k-290k tokens each) and found twelve bugs that 881 passing unit tests did |
| not, because each was an interaction between the model's choices and a real data shape. |
|
|
| | Doc | Status | Read it when | |
| |---|---|---| |
| | [testing/AGENTIC_DRY_RUN_LOG.md](testing/AGENTIC_DRY_RUN_LOG.md) | 📎 | **Before running a dry-run** (it costs real money, and §6 halves it), and before assuming a green run means a working turn — seven of the twelve bugs produced a confident WRONG answer rather than an error | |
|
|
| Suites + shapes: `eval/chat_sim/run_chat.py` · per-run answers: `eval/chat_sim/results/` |
|
|
| --- |
|
|
| ## `archive/` — superseded, kept for the record |
|
|
| Nothing here is safe to implement from. Each says what replaced it. |
|
|
| | Doc | Superseded by | |
| |---|---| |
| | [archive/API_ENDPOINTS_RESTRUCTURE.md](archive/API_ENDPOINTS_RESTRUCTURE.md) | [../API_CONTRACT_BE_PYTHON.md](../API_CONTRACT_BE_PYTHON.md). The frozen 2026-06-30 snapshot sent to Harry; says "observability" (now traceability) and cites the removed `ENABLE_SLOW_PATH` | |
| | [archive/CODE_REVIEW_2026-07-23.md](archive/CODE_REVIEW_2026-07-23.md) | DEV_PLAN §0.7 for live **F-n** status. The review body still explains *why* each fix exists | |
| | [archive/PROJECT_BRD.md](archive/PROJECT_BRD.md) | REPO_STATUS (built state) + API_CONTRACT_BE_PYTHON (contract). Written against `pr/4` | |
| |
| --- |
| |
| ## `reference/` — other repos |
| |
| | Doc | Note | |
| |---|---| |
| | [reference/API_CONTRACT_BE_GOLANG.md](reference/API_CONTRACT_BE_GOLANG.md) | The **Go** Orchestrator's contract. We don't edit that repo; this is a copy and can drift. Verify against the Go source before relying on it. Last synced 2026-07-01 | |
| |
| --- |
| |
| ## Subsystem docs (live beside the code) |
| |
| `src/*/README.md` — catalog · knowledge_parsing · query · retrieval · security. |
| `eval/*/README.md` — chat_sim · help · intent · knowledge · planner · readiness. |
| |
| Eval READMEs are reliable; follow their run and commit conventions exactly. Subsystem READMEs can |
| lag — code wins. |
| |
| --- |
| |
| ## Conventions |
| |
| - **Every doc gets a status banner** under its H1, marked `<!-- doc-status -->`. New docs too. |
| - **Archive, don't delete.** A deleted doc leaves a dead link and no explanation; an archived one |
| says what replaced it. (Repo convention: deletion needs Rifqi's explicit sign-off.) |
| - **Docs are part of the change** — a behaviour, surface or schema change updates the affected doc |
| in the same unit of work. |
| |
