File size: 9,367 Bytes
b68816f f07443e b68816f 00ca133 b68816f e201ae4 7124acf e201ae4 7124acf e201ae4 b68816f | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 | # Docs index
<!-- doc-status -->
> ✅ **LIVE — the map.** Reorganised 2026-08-24. Every doc in this repo carries a **status banner
> under its H1**; read it before trusting the body.
**Status legend:** ✅ LIVE (canonical) · 🔄 LIVE PLAN (in flight) · 💡 PROPOSAL (not approved) ·
⛔ ARCHIVED (superseded, kept for the record) · 📎 REFERENCE (another repo's, can drift)
---
## Start here — repo root
The root deliberately holds only four docs. If you read nothing else, read the first three.
| Doc | Status | Is |
|---|---|---|
| [../REPO_STATUS.md](../REPO_STATUS.md) | ✅ | **What is built.** Canonical current state |
| [../DEV_PLAN.md](../DEV_PLAN.md) | ✅ | **What is in flight.** Sprint plan, task table, locked decisions |
| [../API_CONTRACT_BE_PYTHON.md](../API_CONTRACT_BE_PYTHON.md) | ✅ | **The contract of record** for the FE-facing surface |
| [../README.md](../README.md) | — | HuggingFace Space card. The YAML frontmatter is load-bearing for deployment — don't restructure it |
`CLAUDE.md` also sits in the root but is **gitignored** (local operating manual, not shipped).
**Trust order when sources disagree:** code > REPO_STATUS / DEV_PLAN > everything else.
---
## `knowledge/` — the knowledge-pipeline workstream
Client documents → candidate knowledge entries an expert reviews rather than authors. Two halves
(parsing, Sofhia · extraction, Rifqi) meeting at one versioned artifact.
| Doc | Status | Read it when |
|---|---|---|
| [knowledge/knowledge_pipeline_context.md](knowledge/knowledge_pipeline_context.md) | 🔄 | You need the *why* — ownership split, settled decisions, what not to reopen |
| [knowledge/KNOWLEDGE_PIPELINE_TODO.md](knowledge/KNOWLEDGE_PIPELINE_TODO.md) | 🔄 | You want per-task status and owners. **Start here** |
| [knowledge/KNOWLEDGE_PIPELINE_CALIBRATION.md](knowledge/KNOWLEDGE_PIPELINE_CALIBRATION.md) | 🔄 | **Before changing any constant.** Every tuned value with the reason it has that value |
| [knowledge/KNOWLEDGE_PARSING_OUTPUT_CONTRACT.md](knowledge/KNOWLEDGE_PARSING_OUTPUT_CONTRACT.md) | 🔄 | You're working either side of the seam |
| [knowledge/KNOWLEDGE_SURFACE_PLAN.md](knowledge/KNOWLEDGE_SURFACE_PLAN.md) | 🔄 | Persistence + endpoints: who owns what, phased tasks, blockers. Approved 2026-09-01 |
| [knowledge/KNOWLEDGE_CONSUMPTION_PLAN.md](knowledge/KNOWLEDGE_CONSUMPTION_PLAN.md) | 💡 | How agents READ the entries (N14). Which agent gets what, and why. **Proposal, nothing built** |
| [knowledge/TABLE_BRANCH_PROPOSAL.md](knowledge/TABLE_BRANCH_PROPOSAL.md) | 💡 | Step 8 — the symptom x cause matrix a shop manual actually holds. Deterministic triples, LLM only to phrase. **Proposal, nothing built** |
| [knowledge/TERM_HIERARCHY_PROPOSAL.md](knowledge/TERM_HIERARCHY_PROPOSAL.md) | 💡 | Step 7 + R7 — parent/child terms and cross-document identity, designed together. **Touches `entity_id`; read before building** |
| [knowledge/KNOWLEDGE_DOMAIN_CONTEXT_V4.md](knowledge/KNOWLEDGE_DOMAIN_CONTEXT_V4.md) | 💡 | What a *real* domain context is. v3's object is a document brief wearing the name. **Proposal, nothing built** |
| [knowledge/KNOWLEDGE_PERSISTENCE_CONTRACT.md](knowledge/KNOWLEDGE_PERSISTENCE_CONTRACT.md) | ✅ | The DDL handed to Harry (sent 2026-09-01, five tables since). ~~**Pending — no ORM written against it yet**~~ **Adopted by Go migration `0008` (2026-09-22; recorded 2026-09-30)**, ORM live since 2026-09-07 |
| [knowledge/KNOWLEDGE_DOMAIN_CONTEXT_V3.md](knowledge/KNOWLEDGE_DOMAIN_CONTEXT_V3.md) | ✅ | `DomainContext` design reasoning. **Shipped in full 2026-09-02** — the live shape is the output contract §8 |
Roll-up status: DEV_PLAN §0.8 · built state: REPO_STATUS §14 · eval harness: [../eval/knowledge/README.md](../eval/knowledge/README.md)
---
## `plans/` — design plans
| Doc | Status | Note |
|---|---|---|
| [plans/SPINE_V2_PLAN.md](plans/SPINE_V2_PLAN.md) | 🔄 | Approved 2026-07-13; W2+W1 shipped. **W3 deferred — do not start.** Holds the `message_charts` DDL (§4.4) |
| [plans/DS_TOOLS_PLAN.md](plans/DS_TOOLS_PLAN.md) | 💡 | Design study only. **Nothing was built from it**; don't implement without sign-off |
| [plans/PR0_PR1_TOOL_FIXES.md](plans/PR0_PR1_TOOL_FIXES.md) | 🔄 | Sofhia's six tool fixes. PR-0 merged; PR-1 open. Every claim re-verified against the code, and two of the review's findings narrowed |
| [plans/PR2_EVALUATION.md](plans/PR2_EVALUATION.md) | 🔄 | **Read before PR-3.** Four layers, three of them free. Establishes that **14 of 14** tools have zero output-correctness coverage |
| [plans/PR3_REPAIR_EDGE.md](plans/PR3_REPAIR_EDGE.md) | 🔄 | The feedback edge (trigger B, budget 1) **gated on measured hit-rate**, plus the determinism fixes that ship regardless |
| [plans/PR23_PART2_PLANNER_EVAL.md](plans/PR23_PART2_PLANNER_EVAL.md) | 🔄 | **pr/23 part 2 (2026-09-23).** Planner, router and eval work re-checked against the code: measurement first, then router, retrieve-then-plan, planner structure. **A–E done 2026-09-23** (D4 reverted, F1 optional) |
| [plans/UNSTRUCTURED_V2_PLAN.md](plans/UNSTRUCTURED_V2_PLAN.md) | 🔄 | **The demo track (2026-09-23).** Parsing v2 → embeddings, private images via 1h signed URLs, analysis-scoped retrieval, the mixed structured+unstructured path **without a new intent**. Pre-empts the prompt-architecture work |
---
## `tools/` — the analytical-tools workstream
The `analyze_*` tool surface: forecast, importance, anomaly, cluster, A/B testing, segmentation,
pareto. Tools are **pure functions, no base class** — the "framework" is `spec_builder` +
`ml/executor` + the shared chart kit.
| Doc | Status | Read it when |
|---|---|---|
| [tools/ANALYTICAL_TOOLS_CONVENTIONS.md](tools/ANALYTICAL_TOOLS_CONVENTIONS.md) | ✅ | **Before writing any tool.** The InputModel → ToolSpec flow, the auto-chart convention, `run_fit`, the three registrations, and the rules that aren't negotiable |
Roll-up status: DEV_PLAN §0.9 · built state: REPO_STATUS §9 · chart envelope: [plans/SPINE_V2_PLAN.md](plans/SPINE_V2_PLAN.md) §4.2
---
## `reviews/` — point-in-time assessments
Read-only reviews. Nothing in them is a decision; findings are cited by id elsewhere.
| Doc | Status | Note |
|---|---|---|
| [reviews/AGENTIC_DESIGN_REVIEW_2026-09-18.md](reviews/AGENTIC_DESIGN_REVIEW_2026-09-18.md) | 📎 | The DS agent against the 2026 state of the art. Findings **D-1…D-12**; the source of PR-0…PR-3 |
| [reviews/PROMPT_ARCHITECTURE_2026-09-22.md](reviews/PROMPT_ARCHITECTURE_2026-09-22.md) | 📎 | `planner.md` + `intent_router.md` measured against industry practice. Findings **P-1…P-7 / R-1…R-4**, candidates **C-1…C-9** — **paused behind the unstructured v2 demo work** (added 2026-09-23) |
---
## `testing/` — what running the thing actually found
End-to-end dry-runs of the `structured_flow` path against real sources. These runs are
expensive (~170k-290k tokens each) and found twelve bugs that 881 passing unit tests did
not, because each was an interaction between the model's choices and a real data shape.
| Doc | Status | Read it when |
|---|---|---|
| [testing/AGENTIC_DRY_RUN_LOG.md](testing/AGENTIC_DRY_RUN_LOG.md) | 📎 | **Before running a dry-run** (it costs real money, and §6 halves it), and before assuming a green run means a working turn — seven of the twelve bugs produced a confident WRONG answer rather than an error |
Suites + shapes: `eval/chat_sim/run_chat.py` · per-run answers: `eval/chat_sim/results/`
---
## `archive/` — superseded, kept for the record
Nothing here is safe to implement from. Each says what replaced it.
| Doc | Superseded by |
|---|---|
| [archive/API_ENDPOINTS_RESTRUCTURE.md](archive/API_ENDPOINTS_RESTRUCTURE.md) | [../API_CONTRACT_BE_PYTHON.md](../API_CONTRACT_BE_PYTHON.md). The frozen 2026-06-30 snapshot sent to Harry; says "observability" (now traceability) and cites the removed `ENABLE_SLOW_PATH` |
| [archive/CODE_REVIEW_2026-07-23.md](archive/CODE_REVIEW_2026-07-23.md) | DEV_PLAN §0.7 for live **F-n** status. The review body still explains *why* each fix exists |
| [archive/PROJECT_BRD.md](archive/PROJECT_BRD.md) | REPO_STATUS (built state) + API_CONTRACT_BE_PYTHON (contract). Written against `pr/4` |
---
## `reference/` — other repos
| Doc | Note |
|---|---|
| [reference/API_CONTRACT_BE_GOLANG.md](reference/API_CONTRACT_BE_GOLANG.md) | The **Go** Orchestrator's contract. We don't edit that repo; this is a copy and can drift. Verify against the Go source before relying on it. Last synced 2026-07-01 |
---
## Subsystem docs (live beside the code)
`src/*/README.md` — catalog · knowledge_parsing · query · retrieval · security.
`eval/*/README.md` — chat_sim · help · intent · knowledge · planner · readiness.
Eval READMEs are reliable; follow their run and commit conventions exactly. Subsystem READMEs can
lag — code wins.
---
## Conventions
- **Every doc gets a status banner** under its H1, marked `<!-- doc-status -->`. New docs too.
- **Archive, don't delete.** A deleted doc leaves a dead link and no explanation; an archived one
says what replaced it. (Repo convention: deletion needs Rifqi's explicit sign-off.)
- **Docs are part of the change** — a behaviour, surface or schema change updates the affected doc
in the same unit of work.
|