# Project Specification Document (PSD) **Project:** Spare-Parts Diagram Locator for the BQ188 Warp Tying Machine catalogue **Source document:** `1.pdf` (project root) **Status:** Phase 1 — specification complete, implementation not started **Last updated:** 2026-08-30 > This document is the permanent source of truth for the project. Every statement in > section 4 ("PDF / document structure discovered") was verified programmatically > against `1.pdf`. Nothing in that section is assumed. --- ## 1. Project overview A small, entirely local desktop tool. The user types a machine spare-part number (e.g. `30642`). The tool finds the catalogue page that contains that part, renders the exploded-view diagram from that page, and draws a **red rectangle around the physical component** the number refers to — not merely around the printed number — with the part number and name shown next to it. The catalogue is a fixed, known, 55-page PDF. The document never changes at runtime. Therefore all expensive work (text extraction, OCR, geometry) happens **once, offline, in a build step**. The interactive application only performs a dictionary lookup and draws an overlay. Lookup is deterministic and instant. This is a **document-processing / computer-vision** project, not a retrieval or question-answering project. See section 19 for what is explicitly out of scope. --- ## 2. Exact problem being solved Given a 5-digit part number `P`: | Step | Requirement | |---|---| | R1 | Determine every catalogue page on which `P` appears. | | R2 | Retrieve the part's name (German / English / French) and the section it belongs to. | | R3 | Render that page's exploded-view diagram as an image. | | R4 | Find every callout label reading `P` inside that diagram. | | R5 | For each label, resolve the **component** the label points to, and produce a bounding box around that component's drawn geometry. | | R6 | Draw a red rectangle on the component, and print `P` (plus name) next to it. | | R7 | Report page number, section number, section title, and part name. | **The hard requirement is R5.** R4 (finding the printed text) is easy and already proven. R5 requires following the drawing's leader line from the label to the part. ### Why this is not a text-search problem The number `30642` appears twice on page 34: once as a row in the machine-readable parts table on the left half of the page, and once as a raster callout inside the diagram on the right half. Boxing either of those occurrences boxes *text*. The component itself — a worm gear (`Schnecke`) — carries no text at all. It is identified only by a thin line drawn from the callout to the part outline. --- ## 3. User workflow ``` 1. User launches the app → streamlit run app.py 2. User types a part number → "30642" 3. App validates / autocompletes → suggests from 705 known numbers 4. App shows result header → 30642 · Schnecke,kpl. / worm,cpl. / vis sans fin,cpl Section 07 "Motor-Antrieb / Motor-drive" · PDF page 34 5. App shows the diagram → full-page diagram image · RED rectangle on the physical component · part number drawn next to the rectangle · thin blue rectangle on the printed callout label (so the user can verify) · confidence badge: "located" / "approximate" / "label only" 6. If the part appears on several pages, a page selector appears (61 parts do). 7. If the part has several instances on one page, all instances are boxed. 8. User can zoom / download the annotated PNG. ``` Failure is always visible, never silent: if the component cannot be resolved, the app still shows the page and boxes the label, and says so. --- ## 4. PDF / document structure discovered All facts below were extracted from `1.pdf` with PyMuPDF 1.28.2. ### 4.1 File-level | Property | Value | |---|---| | Pages | 55 | | Page size | 1190 × 842 pt (A4 landscape), rotation 0, uniform on every page | | PDF version | 1.7 | | Producer / Creator | Microsoft® Excel® 2019 | | Created | 2022-08-21 | | Encrypted | No | | Bookmarks / outline | None (0 entries) | | File size | ~39.8 MB | | Machine | BQ188 Warp Tying Machine — Jingong Industrial Co., Ltd | The PDF was exported from an Excel workbook. This matters: the parts tables are **real vector text in a real table grid**, while every diagram is a **flat embedded JPEG**. There is no vector geometry for the drawings. ### 4.2 Page map | PDF pages | Content | Text layer | Images | |---|---|---|---| | 1 | Title page (BQ188, 4 languages, company address) | Yes (376 chars) | 3 | | 2 – 23 | Scanned **operation manual** — prose + photographs | No real text; only the running footer (98 chars/page) | 2 per page | | 24 | `NEEDLE TABLE` — yarn-count / needle-number table | Yes (vector text, 74 vector drawings) | 0 | | 25 | **Spare-parts catalogue cover** — ordering instructions in DE/EN/FR/ES | Yes | 2 | | 26 – 55 | **Spare-parts catalogue — 30 section pages** | Yes (table) | 1 visible diagram + decoys | **The spare-parts catalogue begins at PDF page 25 (cover); the first section page with parts and a diagram is PDF page 26.** Pages 1–24 contain no part numbers and are out of scope for indexing (they may still be viewable in the UI). ### 4.3 Section structure (pages 26–55) Sections are numbered `01`–`27` with three sub-sections. The mapping is complete and was read from the first table row of each page: | Page | Sec | German | English | Models | |---|---|---|---|---| | 26 | 01 | Abdeckhaube | Cover | PU-ELA-Z | | 27 | 02 | Lampe+Gebläse Kurze Knoten | Lamp+fan short knot | | | 28 | 03 | Abdeckblech,hinten | Cover plate,rear | | | 29 | 04 | Abdeckblech,links+Abschlußblende Kurbeldeckel | Cover plate,left+Crank end cover cap | | | 30 | 04-1 | Abdeckblech,link. für kurze knoten | Cover plate,left+for short knot | | | 31 | 04-2 | Kurze Knoten | Short knot | | | 32 | 05 | Gehäuseplatte,oben,unten | Housing plate,above,below | | | 33 | 06 | Handantrieb+Fadenabstreifer+Schutzblech | Hand drive+Thread wipers+Fender | PU-ELA/PU-ELA-Z/PU-S/PU-DK/PU-DKU | | 34 | 07 | Motor-Antrieb | Motor-drive | | | 35 | 08 | Tastfühler+Tastfühler steuerung | Feeler+Feeler control | PU | | 36 | 09 | Fadenschieber+Bürstenhalter | Thread spool+Brush holder | PU | | 37 | 10 | Kreuzhebel | Cross lever | PU-ELA/PU-ELA-Z/PU-DK/PU-C/PU-CC/PU-S | | 38 | 11 | Führungsplatte | Guide plate | PU | | 39 | 12 | Nadelanschlag+Nadelhebelwelle | Needle+Needle stop lever shaft | | | 40 | 13 | Mittelgruppe-Kipper | Central group-tipper | PU-ELA | | 41 | 13-1 | Mittelgruppe Kipper | Middle group tipper | PU-ELA | | 42 | 14 | Kurze Knoten Schere | Short knot scissors | | | 43 | 15 | Fadenklemme+Klemmenhebellager | Thread clamp+clamping lever bearing | | | 44 | 16 | Knotenhelfer+Knüpferlager+Knüpfernadel | Lever hinge+Knotter support+Knotter needle | | | 45 | 17 | Fadenhalter+Knotenauszieher+Fadenkanal | Thread holder+Knot extractor+Thread channel | | | 46 | 18 | Antriebslager+Transport | Drive bearing+Transport | | | 47 | 19 | Kipperwelle | Tipper Shaft | PU-ELA/PU-ELA-Z/PU-S/PU-DK/PU-DKU | | 48 | 20 | Gehäuseplatte,re. li. | Housing plate,left,right | | | 49 | 21 | Antriebs+Mittel+Untere+Welle | Drive+Medium+Low+Shaft | | | 50 | 22 | Kreuzkurvenscheibe | Cross cam | PU | | 51 | 23 | Knüpfrahmen | Knotting frame | PU | | 52 | 24 | Zubehör für Knüpfrahmen | Accessories for knotting frame | | | 53 | 25 | Innentraverse-Knüpfrahmen | Knotting frame inner traverse | | | 54 | 26 | Maschinenwagen | Machine cart | | | 55 | 27 | Ölwanne | Oil pan | | ### 4.4 Layout of a catalogue page Every one of the 30 pages has the identical two-column layout: ``` (0,0) (1190,0) +-------------------------------+-------------------------------+ | PARTS TABLE (vector text) | EXPLODED DIAGRAM (JPEG) | | x ~ 17 ... 607 | x ~ 607 ... 1140 | | y ~ 28 ... (varies) | y ~ 28 ... 800 | | | | | row 0 : section header | header: model codes, sec no | | row 1 : Art.Nr | Bezeichnung | body: line-art drawing | | | Description | + 5-digit callouts | | | Denomination | + leader lines | | rows 2+: one part per row | footer: drawing path, Seite | +-------------------------------+-------------------------------+ page footer: company / URL (1190,842) ``` ### 4.5 The parts table (left column) — machine-readable * `page.find_tables()` returns exactly **one** table on every catalogue page, always with **4 columns**, and the `Art. Nr.` header is always at row index 1 (row 0 is the section header). This was verified on all 30 pages. * Columns: `Art. Nr.` | `Bezeichnung` (DE) | `Description` (EN) | `Dénomination` (FR). * **861 part rows** in total, **705 distinct part numbers**. * Part numbers are **exactly 5 digits**, no letters, no separators. Verified length distribution: `{5 digits: 860, 6 digits: 1, 2 digits: 27}`. The 27 two-digit values are the section-header cells (`01`…`27`). The single 6-digit value is a typo in the source document: page 47, `351159 / Kipperhebel / tipper / culbuteur` — and the diagram on that same page prints it as `35159`, confirming the table cell is wrong. * Observed number prefixes: `73` (281), `74` (239), `75` (73), `30` (48), `33` (44), `70`, `76`, `78`, `77`, `32`, `35`, `72`. * **61 part numbers appear on more than one page** (62 have more than one occurrence; the extra one is `33275`, listed twice on page 43 — see below). Examples: `73290` on 9 pages (31, 35, 36, 37, 39, 42, 45, 47, 49); `73289` on 5 pages; `70507` on 4. A lookup therefore returns a *set* of pages, not one page. * Only one within-page duplicate table row exists: `33275` on page 43. * **Non-part rows must be filtered.** Pages 42, 51 and 52 end with footnote rows whose `Art. Nr.` cell is empty or contains `*`, `**`, `***`: `* = "nur bei Ausführung kurze Knoten" (only for short knot)`, `** = "Länge angeben" (indicate length)`, `*** = "Länge und Durchmesser (18 oder 17) angeben"`, `S = Sonderzubehör (special accessories)`. Some rows appear *after* the footnotes and are still real parts (e.g. page 52 `30343`–`30346`, page 51 `73013`), so filtering must be per-row, not "stop at first footnote". * The description columns already contain OCR-era typos inherited from whoever built the Excel file: `lsolierplatte` (lowercase L for I), `Kreuzsohnur` for `Kreuzschnur`, `Acoesories`, and full-width `．` instead of `.`. **These are cosmetic. The `Art. Nr.` column is clean.** Descriptions are displayed as-is; they are never used for matching. * A leading `*`, `**` or `***` inside the `Bezeichnung` cell is a footnote marker (e.g. `*Handkamm`, `**Kluppe`) and should be parsed off into a `footnote` field. ### 4.6 The diagram (right column) — raster only * Each catalogue page has **exactly one visible diagram image**, embedded as a **DCTDecode (JPEG), DeviceRGB** object. * Native resolution is **1689 px wide × 2248–2367 px tall** on all 30 pages (1689 × 2352 for page 34). Placed at ≈ 533 × 770 pt → **effective ≈ 228 dpi**. Rendering the *page* at 300 dpi would upsample past native; the correct move is to **extract the embedded image at native resolution** and OCR that. * Despite the RGB container, content is essentially bilevel line art: dark-pixel fraction (`gray < 128`) is **2.2 % – 10.6 %** per page, mid-tones 0.8 % – 3.0 %. There are **no halftone photographs** in the catalogue section (unlike pages 2–23 of the manual). * **Decoy images.** Excel left extra image objects on some pages whose placement rectangles fall partly or wholly **outside** the page box — e.g. page 26 has a second image at `y = 802 … 1576` (page height is 842), pages 29 and 54 have one at negative `y`. Additionally the **same xref is reused across consecutive pages** (xref 152 is the second image of page 36 and the visible diagram of page 37). → **The diagram must be selected as the image rect with the largest intersection area with `page.rect`**, and cached per *page*, not per xref. ### 4.7 How part numbers are represented inside the diagram Verified by rendering and by OCR: * Callouts are the **actual 5-digit part numbers** (`30642`, `74506`, `73631`), drawn in a clean ISO-style technical font, roughly 90 px wide × 30 px tall at native resolution. There is **no separate item-number scheme** — no "item 1, 2, 3" that would need a second lookup table. This is the single most favourable property of this document. * Some callouts carry a leading **asterisk** (`*30638`, `*74012`, `*76352`). The asterisk is the footnote marker described in 4.5 and must be stripped before matching. * Some callouts are **enclosed in a drawn rectangle** (`31269`, `32357`, `33333`, `35037`, `36962`, `30907`, `34983`, `30168`, `30169`). Cross-checking these against the table shows they are consistently the **`kpl.` / `cpl.` (complete assembly)** entries — e.g. `31269 Abdeckhaube,kpl. / covering hood cpl.`, `33333 Mittelgruppe,kpl. / middle group cpl.`. A boxed callout therefore denotes a **sub-assembly**, and its "component" is the whole group shown, not one item. The box also means the label glyphs are *not* isolated components — the drawn rectangle around them must be recognised and excluded, or it will be mistaken for the leader line. * The same part number can appear **several times on one page** when the part is used in several places — page 26 has `73060` ×2, `74014` ×3, `74025` ×2, `79038` ×2; page 34 has `74506` ×4. Every instance must be indexed and boxed. * Non-part text also appears in the diagram and must be rejected: * top-left: model codes — `PU-ELA`, `PU-ELA-Z`, `PU-JET` * top-right: section number — `07`, `K 13` * bottom-left: drawing path — `D / PU-S-12 / PU-ELA / ET / ET- 07 Motor-Antrieb` * bottom-right: the manufacturer's internal page number — `Seite 18` (this is **not** the PDF page number: PDF page 34 is "Seite 18") * small revision/date codes such as `061101`, `051201`, `110702` ### 4.8 How labels relate spatially to components — leader lines This was investigated directly on the pixel data of page 34. * Callouts sit in white space and are joined to their component by a **thin solid leader line**, typically 3–5 px wide at native resolution. Most are a single straight segment; some are kinked polylines; a few are **dashed** (page 40: `73158`, `71058`). * The line starts a few pixels away from the label's bounding box (it does **not** touch the glyphs — verified: the digits of `30642` are their own connected components, distinct from the line) and **terminates touching the outline of the target component**. * **Consequence, and the key finding of Phase 1:** because the leader line touches its target and nothing else, the 8-connected component containing the leader line *is* the target component. Running `cv2.connectedComponentsWithStats` on `gray < 150` for page 34 gave 586 components; the component containing the `30642` leader line has bbox `x[310:520] y[1472:1671]`, which is an exact, tight fit around the worm gear. No template matching, no segmentation model, no training data required. * **Some callouts have no leader line at all** and simply sit adjacent to their part (visible on page 40). These need a nearest-component fallback. * Nothing in the diagram uses arrowheads in the catalogue section (arrowheads do appear in the *manual*, pages 2–23, which is out of scope). So the relationship is confirmed as: ``` part number (table) -> identical printed callout (raster) -> leader line -> component outline ``` ### 4.9 What makes localization difficult (measured, not assumed) Running the prototype over all 34 validated labels on page 34 gave roughly 80 % tight, correct component boxes. The failures are of four kinds: 1. **Leader-line bleed.** If a leader line *crosses* a large neighbouring outline on its way to a small part, the line, the neighbour and the target all merge into one connected component and the box balloons. On page 34, labels `74506`, `74903` and `79105` all resolved to the large gearbox housing component (347 × 533 px, 4.7 % of the image) instead of their small rings. 2. **Frame merge.** Any component that touches the diagram's outer border merges with it. Page 34's border component has a bbox covering 64 % of the image. A size guard is mandatory. 3. **Nearest-neighbour capture.** When another part's outline happens to be closer to the label than the label's own leader line, the wrong component wins. On page 34, `74025` (a screw) captured the bracket that belongs to `76352`. 4. **Assembly callouts.** A boxed `kpl.` callout points at a whole group; the "right" answer is a group of components, and any single-component box is arguably wrong. None of these need machine learning to fix — they need a leader-line-aware tracing step (section 10) plus a small manual-override file. ### 4.10 OCR feasibility — measured OCR was actually run, not assumed. Engine: **RapidOCR (ONNX Runtime, PP-OCRv4 weights)**, CPU only, on the natively extracted JPEG, for all 30 catalogue pages. * Runtime ≈ **11 s per diagram** on this machine; ≈ **6 minutes for all 30 pages**, once, offline. * Page 34: 39 text boxes returned, **every one of the 34 real callouts recognised**, confidence ≥ 0.92 for all of them and 1.00 for most. Only two character-level errors, both trivially repaired by stripping non-digits: `*7.4012` → `74012` (spurious dot), `-74506` → `74506` (leader-line dash absorbed). * **Full-catalogue recall** — distinct part numbers OCR'd from each diagram, matched against that page's own table, summed over all 30 pages: | | | |---|---| | Table part rows, pages 26–55 | **861** | | Matched by OCR in the diagram | **817** | | **Recall** | **94.9 %** | | Pages at 100 % | **9 of 30** | | Worst page | **39 → 40/47 = 85 %** | | Unrecognised tokens, all 30 pages | **69 total (~2.3/page)** | Per-page examples: p28 14/14, p30 19/19, p31 27/27, p36 21/21, p41 25/25, p45 37/37, p50 15/15, p26 33/36, p34 27/29, p40 59/61. The residual "missing" entries are largely parts that are genuinely **listed in the table but not drawn** in that diagram, not OCR misses — the catalogue does this deliberately for hardware variants. **94.9 % is therefore a lower bound on true OCR recall.** * **The reverse case also exists:** callouts appear in a diagram that are absent from that page's table — page 40 shows `73182`, `73636`, `74167`; page 36 shows `33134`, `33141`; page 33 shows `73007`, `70507`. So validating OCR output against the *page's* table alone would silently drop real labels. Validation must be against the **global** 705-number set, with the page's own table used only as a confidence bonus. * The 69 unrecognised tokens are almost entirely the known furniture from 4.7 — `061101`-style revision codes and the `Seite NN` + section-number pair merging into things like `1207` — plus a handful of truncated reads (`4639`, `7497`, `4401`) that recover under edit-distance-1 snapping. **Conclusion: OCR works directly on this document and needs no deskewing, no binarization tuning and no custom training.** Pages do not need to be rendered — the embedded image is extracted at native resolution instead, which is both faster and sharper. --- ## 5. Core technical approach **Build once, look up instantly.** 1. **Offline indexer** (`python -m spareparts.build_index`) processes `1.pdf` end to end and writes a SQLite database plus a PNG cache. Runs in ~10 minutes. Re-runnable, idempotent, and each stage is independently re-runnable so OCR is not repeated when only the localization tuning changes. 2. **Streamlit app** reads the database, does a dictionary lookup, loads the cached PNG, draws the overlay with OpenCV, and displays it. No OCR, no PDF parsing at runtime. Three deliberately deterministic decisions: * **The parts table is the authority.** Names, sections and the canonical part list all come from the PDF's real text layer, never from OCR. * **OCR only ever produces coordinates**, never facts. Every OCR token is snapped to a known part number or discarded. * **Localization is classical CV** — binarize, connected components, leader-line trace. No detector is trained, no model is fine-tuned, nothing is learned from data. --- ## 6. Proposed architecture ``` +-------------------------- BUILD TIME (offline, once) --------------------------+ | | | 1.pdf | | | | | +--> pdf_loader.py classify pages, pick the visible diagram per page, | | | extract it at native resolution -> data/pages/p##.png| | | | | +--> table_parser.py find_tables() -> section header + part rows | | | -> parts, occurrences | | | | | +--> ocr.py RapidOCR on each diagram -> raw text boxes | | | | | +--> matcher.py normalize, reject non-part text, snap to the 705- | | | number set (exact -> edit-distance-1) -> labels | | | | | +--> localizer.py binarize -> connected components -> leader-line trace| | | -> component box + method + confidence -> components | | | | | +--> overrides.json hand corrections, applied last, always win | | | | | +--> index_db.py -> data/catalog.db (SQLite) | | | +--------------------------------------------------------------------------------+ | +---------------------------- RUN TIME (Streamlit) ------------------------------+ | | | app.py --> lookup.py (SQLite queries) --> overlay.py (OpenCV drawing) | | | +--------------------------------------------------------------------------------+ ``` There is no server, no API, no queue, no container. Two entry points: one script, one Streamlit app. --- ## 7. Data flow ### Build ``` 1.pdf +- for page in 26..55: diagram_rect = argmax over image rects of area(rect ∩ page.rect) # 4.6 png = extract_image(xref) -> data/pages/p34.png table = page.find_tables()[0].extract() row0 -> section code / DE / EN / model row1 -> header, skipped rows2.. -> part rows, footnote rows filtered -> parts, occurrences ocr_boxes = RapidOCR(png) strip non-digits; drop len != 5 drop boxes in header/footer strips (top 4 %, bottom 6 %) snap to global part set (exact, then edit-distance 1) -> labels bw = (gray(png) < 150) cc = connectedComponentsWithStats(bw, 8) glyph_ids = components fully inside any label box (+ their enclosing box, if boxed) for each label: anchor = trace_leader_line(label, bw, cc, glyph_ids) box, method, conf = resolve_component(anchor, cc) -> components +- apply overrides.json +- write catalog.db ``` ### Query ``` "30642" +- SELECT part -> name_de/en/fr +- SELECT occurrences -> [page 34, section 07 "Motor-Antrieb"] +- SELECT labels + components -> [{label_box, component_box, conf}] +- load data/pages/p34.png +- draw red rect + caption per instance, blue rect on label +- st.image(annotated) ``` --- ## 8. OCR strategy **Engine: RapidOCR (`rapidocr-onnxruntime`).** Rationale, and why *not* the stack originally guessed: * **PaddleOCR** uses the same PP-OCR models but drags in `paddlepaddle`, which is a large, frequently painful wheel on Windows and pins specific Python versions. RapidOCR ships the identical PP-OCRv4 detection + recognition models as ONNX and runs on `onnxruntime` CPU. It installed cleanly here in one `pip install` with no system dependencies, and was **verified working on this exact document at 94.9 % recall**. If PaddleOCR is later preferred, `ocr.py` exposes one function and the engine is swappable. * **Tesseract** was not chosen: it needs a system binary outside pip, and it is weaker on sparse, isolated numerals scattered on a mostly-empty canvas, which is precisely this document's layout. Settings and rules: * OCR the **natively extracted JPEG** (1689 px wide, ~228 dpi effective), **never** a re-rendered page. No upscaling — measured accuracy is already ≈ 100 % on callouts. * No deskew, no rotation correction: every diagram is upright and axis-aligned. * Run **once per page** at build time; results are cached to `data/ocr_cache/p##.json` so that re-tuning localization never re-runs OCR. * Post-processing pipeline per detected box: 1. `re.sub(r'\D', '', text)` — removes the `*` footnote marker, the spurious `.` from `*7.4012`, and the leading `-` from `-74506`. 2. Reject if length ≠ 5. 3. Reject if the box lies in the header strip (top 4 % of image height) or footer strip (bottom 6 %) — this removes `Seite 18`, drawing paths and revision codes. 4. Snap to the global part set (section 9). * Confidence from the engine is stored per label and surfaced in the UI. --- ## 9. Part-number extraction strategy Two independent sources, with clearly separated authority: **A. Canonical list — from the PDF text layer (authoritative).** `page.find_tables()` on pages 26–55. Per row: `Art. Nr.` must match `^\d{5}$` after stripping whitespace; rows failing that (empty, `*`, `**`, `***`) are recorded as footnotes and excluded from the part list. The `Bezeichnung` cell's leading `*`s are split off into a `footnote` field. This yields **705 distinct part numbers / 861 occurrences**, and is the *only* source of names and sections. One known data defect to carry as-is with a flag: page 47's `351159` is 6 digits and the diagram prints `35159`, so it is a source typo. It is stored, marked `is_malformed = 1`, and excluded from OCR snapping. **B. Label positions — from OCR (positional only).** Every OCR token is snapped to the canonical list before it is allowed to become a label: ``` 1. exact match against the 705-number set -> accept, conf = ocr_conf 2. else Levenshtein distance 1 to exactly one number -> accept, conf = ocr_conf * 0.8 (single candidate only — ambiguous ties are dropped) 3. else -> discard, log to build report ``` A token that also appears in **its own page's table** gets a confidence bonus; a token that appears only in the global list is still accepted, because section 4.10 proved that diagrams legitimately show parts absent from their page's table. Because part numbers are exactly 5 digits with no letters, no checksum and no separators, no other normalization is needed. --- ## 10. Component localization strategy This is the heart of the project. Four tiers, tried in order, each recording which one fired so the UI can be honest about it. ### Preparation (once per page) ``` gray = cv2.cvtColor(img, COLOR_BGR2GRAY) bw = (gray < 150).astype(uint8) # verified threshold; art is bilevel n, cc, stats, _ = cv2.connectedComponentsWithStats(bw, 8) glyph_ids = components whose bbox lies inside a label bbox (+ small pad) frame_ids = components whose bbox area > 25 % of the image (border, page frame) box_ids = for boxed assembly callouts: the thin rectangle enclosing the glyphs ``` ### Tier A — leader-line trace (primary) 1. Cast rays from the label's bounding box outward in 5° increments, from 4 px to ~40 px away from the box edge, looking for dark pixels that belong to a component that is neither a glyph nor the frame. This finds the leader line's near end. 2. Verify the candidate is *line-like*: skeletonize the local neighbourhood and check the stroke width is ≤ ~6 px and locally elongated. This is what distinguishes the real leader line from a neighbouring part's outline that happens to be closer (failure mode 3 in 4.9). 3. **Walk the line.** Follow the stroke pixel by pixel away from the label, allowing direction changes for kinks and a small pixel gap for dashed lines, until the stroke either ends or opens out into a wide (non-line-like) region. That transition point is the **anchor**. 4. Take the connected component containing the anchor. Its bbox is the component box. `method = "leader"`. ### Tier B — bounded fallback (when Tier A's component is implausible) Triggered when the Tier A component's bbox exceeds ~8 % of the image area (leader-line bleed / frame merge, failure modes 1 and 2 in 4.9), or when no line-like stroke is found within 40 px. * If an anchor exists: emit a fixed-radius box centred on the anchor, sized from the page's median component size (roughly 120 × 120 px at native resolution). * If no anchor exists: take the nearest non-glyph, non-frame component to the label, with the same size cap. * `method = "anchor_radius"` / `"nearest"`, and confidence is lowered. The UI shows these as **"approximate"**. ### Tier C — assembly callouts A callout enclosed in a drawn rectangle (`kpl.` / `cpl.` parts, section 4.7) points at a whole group. For these: trace the leader line as in Tier A, then take the **union** of all non-glyph components whose bbox centre lies within a generous radius of the anchor, capped at 30 % of the image. `method = "assembly"`. The UI labels these **"assembly (group)"**. ### Tier D — manual override `data/overrides.json`, keyed `"::"`, holding a box in native-image pixel coordinates plus a note. Applied after everything else and always wins. This is the pragmatic escape hatch for the residual ~10–15 % — a small build-time review tool (section 18, Phase 5) renders every auto box so they can be eyeballed and corrected quickly. ### If a label has no resolvable component at all The record is kept with `component_box = NULL`. The app still shows the page and boxes the **label** in blue, stating "label located, component not resolved". The tool is never allowed to draw a red box it cannot justify. ### Why not other approaches * **Hough transform for leader lines** — considered and rejected as the *primary* method: it finds line segments globally but does not tell you which end is the label end, handles kinked polylines badly, and still needs the component-lookup step. It remains a reasonable Tier-A alternative if pixel-walking proves fragile. * **Training an object detector** — needs annotated data that does not exist, and would be non-deterministic and unexplainable for a 30-page fixed document. * **Full manual annotation of all 861 occurrences** — reliable but ~15 hours of work and defeats the point. The tiered approach automates the ~85 % and leaves a reviewable remainder. --- ## 11. Data / schema design SQLite, single file `data/catalog.db`. **All geometry is stored in native diagram-image pixel coordinates** (origin top-left of the extracted JPEG). PDF-page coordinates are derivable from `pages.rect_*` when needed; keeping one coordinate system avoids an entire class of bugs. ```sql CREATE TABLE pages ( pdf_page INTEGER PRIMARY KEY, -- 1-based, as the user sees it section_code TEXT, -- '07', '13-1' section_de TEXT, section_en TEXT, models TEXT, -- 'PU-ELA/PU-ELA-Z/...' img_xref INTEGER, img_w INTEGER, -- native px, e.g. 1689 img_h INTEGER, -- native px, e.g. 2352 rect_x0 REAL, rect_y0 REAL, -- placement on the PDF page, in points rect_x1 REAL, rect_y1 REAL, png_path TEXT -- 'data/pages/p34.png' ); CREATE TABLE parts ( part_no TEXT PRIMARY KEY, -- '30642' name_de TEXT, name_en TEXT, name_fr TEXT, is_malformed INTEGER DEFAULT 0 -- the 6-digit '351159' ); CREATE TABLE occurrences ( -- part listed in a page's table id INTEGER PRIMARY KEY, part_no TEXT REFERENCES parts(part_no), pdf_page INTEGER REFERENCES pages(pdf_page), row_index INTEGER, footnote TEXT, -- '', '*', '**', '***' UNIQUE(part_no, pdf_page, row_index) ); CREATE TABLE labels ( -- callout found in the diagram raster id INTEGER PRIMARY KEY, pdf_page INTEGER REFERENCES pages(pdf_page), part_no TEXT REFERENCES parts(part_no), instance INTEGER, -- 0,1,2... for repeated callouts on one page lx0 INTEGER, ly0 INTEGER, lx1 INTEGER, ly1 INTEGER, -- label bbox, native px has_asterisk INTEGER, -- callout was drawn as '*30638' is_boxed INTEGER, -- callout enclosed in a rectangle -> assembly ocr_text TEXT, -- raw, before normalization ocr_conf REAL, match_kind TEXT -- 'exact' | 'fuzzy1' ); CREATE TABLE components ( -- resolved physical component label_id INTEGER PRIMARY KEY REFERENCES labels(id), cx0 INTEGER, cy0 INTEGER, cx1 INTEGER, cy1 INTEGER, -- may be NULL anchor_x INTEGER, anchor_y INTEGER, -- leader-line endpoint method TEXT, -- 'leader'|'anchor_radius'|'nearest'|'assembly'|'override' confidence REAL, note TEXT ); CREATE INDEX ix_occ_part ON occurrences(part_no); CREATE INDEX ix_lab_part ON labels(part_no); CREATE INDEX ix_lab_page ON labels(pdf_page); ``` `data/overrides.json`: ```json { "34:74506:1": { "box": [1001, 1448, 1106, 1540], "note": "was capturing the housing" } } ``` `data/build_report.json` — per page: OCR token count, accepted/rejected labels, table parts with no label, labels with no table row, per-method counts. Written on every build; this is the project's own quality dashboard. --- ## 12. Proposed project folder structure ``` ImageOCR/ ├── 1.pdf ├── PSD.md <- this document ├── README.md ├── requirements.txt ├── app.py <- Streamlit entry point ├── spareparts/ │ ├── __init__.py │ ├── config.py <- paths, thresholds, page ranges (one place) │ ├── pdf_loader.py <- page classification, diagram extraction │ ├── table_parser.py <- parts tables -> parts / occurrences │ ├── ocr.py <- RapidOCR wrapper + cache │ ├── matcher.py <- token normalization + snap to part list │ ├── localizer.py <- binarize, CC, leader-line trace, tiers A-D │ ├── index_db.py <- schema, writes, migrations │ ├── lookup.py <- runtime queries │ ├── overlay.py <- OpenCV drawing of boxes + captions │ └── build_index.py <- CLI: python -m spareparts.build_index ├── tools/ │ └── review.py <- build-time QA: render every box, write overrides ├── data/ <- generated, git-ignored except overrides.json │ ├── catalog.db │ ├── pages/p26.png ... p55.png │ ├── ocr_cache/p26.json ... p55.json │ ├── build_report.json │ └── overrides.json <- committed; it is hand-authored input └── tests/ ├── test_table_parser.py ├── test_matcher.py ├── test_localizer.py ├── test_index_integrity.py └── fixtures/ ├── page26_expected.json ├── page34_expected.json └── gold_boxes.json <- manual ground truth for evaluation ``` Flat, one package, two entry points. No `src/` indirection, no plugin system, no service boundaries. --- ## 13. Dependencies `requirements.txt`: ``` pymupdf>=1.24 # PDF parsing, table extraction, image extraction rapidocr-onnxruntime>=1.3 # OCR (PP-OCRv4 via ONNX Runtime, CPU, no system deps) opencv-python-headless>=4.9 numpy>=1.26 pillow>=10 streamlit>=1.30 ``` Dev only: `pytest`. Notes: * **Python 3.11** (the interpreter present in this workspace; 3.11.9 verified). * `opencv-python-headless` rather than `opencv-python`: no GUI windows are ever opened, and headless avoids Qt/GTK baggage. * `scipy` is **not** required — `cv2.connectedComponentsWithStats` covers it. * No `pandas`: the data volumes are trivial and SQLite is the store. * Everything runs **offline on CPU**. No network calls at build or run time. Total install footprint is a few hundred MB, dominated by the ONNX models and OpenCV. * `pymupdf` version note: `find_tables()` prints a suggestion to install `pymupdf-layout`. It is not needed here — the plain extractor produced correct 4-column output on all 30 pages. --- ## 14. UI design Single-page Streamlit app, no navigation, no login. ``` +----------------------------------------------------------------------+ | BQ188 Spare-Parts Locator | | | | Part number [ 30642 v ] [ Find ] | | (autocomplete over 705 known numbers) | +----------------------------------------------------------------------+ | 30642 · Schnecke,kpl. | | EN worm,cpl. FR vis sans fin,cpl | | Section 07 — Motor-Antrieb / Motor-drive PDF page 34 | | 1 instance on this page * located | +----------------------------------------------------------------------+ | [ page 34 ] <- tabs if part is on many | | | | +---------------------------------------------+ | | | | | | | +---------+ | | | | | #######<--- red box on the component | | | | +---------+ | | | | 30642 <-- caption drawn on the image | | | | +------+ | | | | |30642 |<-- blue box on the callout | | | | +------+ | | | +---------------------------------------------+ | | | | [ Download annotated PNG ] [ ] show label boxes [ ] show all | +----------------------------------------------------------------------+ ``` Details: * **Red rectangle**, 4–6 px at native resolution, on the component. **Blue thin rectangle** on the printed callout label (toggleable, on by default) so the user can see *why* the tool chose that component. These two colours are never swapped. * The part number is drawn **on the image** next to the red box, on a white plate for legibility against line art. * **Confidence badge** next to the header, driven by `components.method`: `located` (leader) / `approximate` (anchor_radius, nearest) / `assembly (group)` / `label only` (no component box). * **Multi-page parts** (61 of them) → one Streamlit tab per page. * **Multi-instance parts** → every instance boxed and numbered `#1`, `#2`, … in the caption; a count is shown in the header. * **"Show all parts on this page"** checkbox renders every label on the page at once — genuinely useful for browsing, and it doubles as the QA view. * Unknown input → "No part `xxxxx` in this catalogue" plus the nearest 5 numbers by edit distance. * Images are served from the PNG cache; drawing is a few OpenCV calls, so response is effectively instant. --- ## 15. Error and edge cases | # | Case | Evidence | Handling | |---|---|---|---| | E1 | Part appears on several pages | 61 parts; `73290` on 9 pages | Return all; one UI tab per page | | E2 | Part appears several times on one page | `74506` ×4 on p34 | Index every instance; box all | | E3 | Part in the table but not drawn | p39: 7 of 47 | Show the page, box nothing, say "listed in table; not drawn on this diagram" | | E4 | Callout drawn but not in that page's table | p40 `73182`, `73636`, `74167` | Accept via global part list; flag in build report | | E5 | Callout prefixed `*` | `*30638`, `*74012` | Strip in normalization; store `has_asterisk`; show the footnote text | | E6 | Boxed assembly callout | `31269`, `33333`, `34983` | Tier C; badge "assembly (group)" | | E7 | Footnote rows in the table | p42, p51, p52 (`*`, `**`, `***`, blank) | Filtered out of `parts`; text kept for E5 display | | E8 | Real parts listed *after* the footnote rows | p52 `30343`–`30346`, p51 `73013` | Filter per row, never "stop at first footnote" | | E9 | Malformed part number in source | p47 table `351159` vs diagram `35159` | Stored with `is_malformed`; excluded from OCR snapping | | E10 | Off-page / decoy images | p26 second image at y 802–1576; p29, p54 negative y | Select by max intersection with `page.rect` | | E11 | Same image xref on two pages | xref 152 on p36 and p37 | Cache per page, never per xref | | E12 | OCR noise: `Seite 18`, `061101`, `D / PU-S-12 / …`, `07` | Every page | Reject by header/footer strip + 5-digit rule + part-list snap | | E13 | OCR char errors: `*7.4012`, `-74506`, `4639`, `7497` | p34, p27, p48 | `re.sub(r'\D','')` then exact/fuzzy-1 snap | | E14 | Leader line merges component with the frame | p34 border component = 64 % of image | Frame components (>25 % area) excluded; Tier B size cap | | E15 | Leader line crosses a big neighbour | p34 `74506`/`74903`/`79105` → housing | Line-likeness check; >8 % area triggers Tier B | | E16 | Wrong nearest component wins | p34 `74025` captured `76352`'s bracket | Line-likeness check; then override | | E17 | Callout with no leader line | p40 | Tier B "nearest" | | E18 | Dashed leader line | p40 `73158`, `71058` | Gap-tolerant walk in Tier A | | E19 | User searches a number in the manual pages (1–24) | — | Those pages are not indexed; report "not found" | | E20 | User enters non-numeric / wrong length | — | Validate `^\d{5}$`; suggest nearest known numbers | | E21 | `catalog.db` missing or stale | — | App checks for the DB on start and prints the exact build command | --- ## 16. Testing strategy `pytest`, run against the real `1.pdf` — it is fixed and committed, so it *is* the fixture. **Unit** * `test_table_parser.py` — pages 26, 34 and 52 parsed against committed golden JSON: exact row counts, exact part numbers, correct footnote filtering (52), correct handling of real rows after footnotes (52). Asserts the global totals: 30 pages, 861 rows, 705 distinct numbers. * `test_matcher.py` — `*7.4012 → 74012`, `-74506 → 74506`, `Seite 18 → rejected`, `061101 → rejected`, `07 → rejected`; fuzzy-1 accepts a single candidate and rejects ambiguous ties. * `test_localizer.py` — on synthetic images (a label, a leader line, a target square, a decoy square): the target is chosen, the decoy is not; the frame guard fires; the size cap triggers Tier B; a dashed line is still followed. * `test_pdf_loader.py` — the diagram picked for pages 26, 29, 36, 37 and 54 is the on-page one, not a decoy or a shared xref (this pins E10 and E11). **Integration** * `test_index_integrity.py` — over the built `catalog.db`: * every `labels.part_no` exists in `parts` * every `occurrences.part_no` exists in `parts` * every label's bbox lies inside its page's image bounds * every component box, when present, lies inside the image bounds and is smaller than the Tier-B cap * overall label recall stays ≥ 93 % and no page drops below 85 % — a regression guard on OCR settings, anchored to the Phase-1 measurement of 94.9 % / worst 85 % **Regression** * `test_known_parts.py` — a handful of hand-verified pins that must never break: `30642` → page 34, section 07, `Schnecke,kpl.` / `worm,cpl.`, component box overlapping the worm; `31269` → page 26, boxed assembly; `73290` → 9 pages. **Manual** * `tools/review.py` renders every page with all boxes drawn into `data/review/`, which is how localization is actually judged (section 17) and how `overrides.json` gets written. --- ## 17. Accuracy and evaluation strategy Three metrics, each measured against something real. **M1 — Label detection recall (automatic, no manual work).** For each page: distinct part numbers OCR'd from the diagram ÷ part numbers in that page's table. **Already measured in Phase 1 over all 30 pages: 817 / 861 = 94.9 %, 9 pages at 100 %, worst page 85 % (page 39).** It over-counts misses, because some listed parts are genuinely not drawn, so it is a **lower bound**. Target: **≥ 95 % overall, ≥ 85 % on every page** — i.e. do not regress from the Phase-1 baseline. **M2 — Label precision (automatic).** Fraction of accepted labels that snap exactly to a known part number, and the count of rejected tokens per page (Phase-1 baseline: 69 rejects over 30 pages ≈ 2.3/page, nearly all of them known page furniture). Target: **≥ 98 % exact matches; < 3 rejected tokens/page.** **M3 — Component localization accuracy (manual gold set).** Build a ground-truth set of **80 (page, part, instance) triples** across 8 pages chosen for varied difficulty — 26 (dense hardware), 34 (clean), 39 and 40 (very dense), 45, 49, 51 (accessories), 55 (nearly empty) — by hand-drawing the correct component box in `tests/fixtures/gold_boxes.json` using `tools/review.py`. Scored two ways: * **Strict:** IoU(predicted, gold) ≥ 0.5. * **Useful:** the predicted box's centre lies inside the gold box *and* the predicted area is ≤ 3× the gold area. This is the metric that matters — a slightly loose box that unambiguously indicates the right part still answers the user's question. Phase-1 prototype baseline on page 34, judged by eye: **~80 % correct**, with the four failure modes catalogued in 4.9. Targets after Tier A/B are properly implemented: | Metric | Target | |---|---| | M3 useful, Tier A+B automatic | ≥ 85 % | | M3 strict (IoU ≥ 0.5), automatic | ≥ 70 % | | After `overrides.json` review | ≥ 97 % useful | **Honesty requirement:** the build report records the per-method distribution (`leader` / `anchor_radius` / `nearest` / `assembly` / `override` / `none`), and the UI shows the method for the box the user is looking at. The tool never presents a Tier-B guess as if it were a Tier-A result. --- ## 18. Implementation phases | Phase | Deliverable | Definition of done | |---|---|---| | **1 (done)** | **This PSD** | PDF inspected, structure documented, OCR and localization prototyped and measured | | **2** | Index skeleton: `config`, `pdf_loader`, `table_parser`, `index_db`, `build_index` | `catalog.db` holds 30 pages, 705 parts, 861 occurrences; 30 PNGs cached; `test_table_parser` and `test_pdf_loader` pass | | **3** | `ocr.py` + `matcher.py` + OCR cache | `labels` populated for all 30 pages; M1 ≥ 95 %; build report written | | **4** | `localizer.py` Tiers A–C | `components` populated; per-method counts in the report; no box exceeds the size cap | | **5** | `tools/review.py`, gold set, `overrides.json` | 80-box gold set exists; M3 measured; overrides written for the residual failures | | **6** | `app.py` + `overlay.py` + `lookup.py` | `30642` → page 34 with a red box on the worm; multi-page and multi-instance paths work | | **7** | Polish: README, autocomplete, download, "show all", error states | Fresh clone → `pip install -r requirements.txt` → `python -m spareparts.build_index` → `streamlit run app.py` works end to end | Phases 2–4 are the substance; 5 is what turns "mostly works" into "trustworthy"; 6–7 are a day's work each. --- ## 19. Explicitly excluded technologies and features Excluded by requirement, and none of them would help here — the catalogue is a fixed, fully enumerable 705-row lookup table. * **RAG in any form** — no retrieval-augmented generation. * **LangChain / LlamaIndex** or any agent framework. * **Embeddings, vector databases, semantic or fuzzy "meaning" search** (FAISS, Chroma, Pinecone, pgvector, …). Lookup is an exact 5-digit key. * **LLMs anywhere in the pipeline**, including for OCR correction or description cleanup. * **Cloud services** — no OCR APIs, no storage buckets, no hosted inference. The tool runs offline. * **Docker / Kubernetes / CI infrastructure.** Single developer, single machine, one `pip install`. There is no deployment target that would justify it. * **Microservices, REST APIs, message queues, background workers.** The build is a script; the app is a Streamlit process. * **Training or fine-tuning any model.** No annotated data exists, and a fixed 30-page document does not warrant creating it. * **A general-purpose PDF catalogue engine.** This is scoped to `1.pdf` and its exact layout. Generalization is explicitly a non-goal; the code should read as document-specific, with the document's quirks named in `config.py`. * **User accounts, multi-tenancy, telemetry, persistence of user state.** * **PostgreSQL / MongoDB.** SQLite holds ~1600 rows. * **Indexing pages 1–24** (title, operation manual, needle table). They contain no part numbers. They may optionally be *viewable*, never searchable. --- ## 20. Known uncertainties and risks | # | Risk | Severity | Evidence / mitigation | |---|---|---|---| | **R1** | **Leader-line tracing is less reliable than the page-34 sample suggests.** Denser pages (39, 40, 51 — up to 10.6 % ink, 61 parts) have crossing lines and touching parts; the "line touches only its target" property may break down. | **High — this is the project's central risk** | Measured on one page only (~80 %). Mitigation: Tier B size cap keeps failures *conservative* rather than wrong-and-confident; Tier D overrides; the UI always shows the label box so the user can self-correct. **Validate on pages 39/40/51 first in Phase 4, before investing in Tier A refinements.** | | R2 | Component boxes are systematically too large where the leader line crosses a big outline | Medium | Observed on p34 (`74506`, `74903`, `79105`). Line-likeness check in Tier A; 8 % area cap → Tier B | | R3 | Assembly (`kpl.`) callouts have no single correct box | Medium | ~9 seen on p26/p34/p40. Tier C draws a group box and the UI says "assembly"; accepted as inherently approximate | | R4 | The ~15 % residual needs manual review; a fresh reviewer may find it tedious | Medium | 861 occurrences total; the review tool renders a page at a time. Budget ~3–4 hours in Phase 5 | | R5 | OCR misses on the densest pages | Low | Measured 94.9 % overall, worst page 39 at 40/47, and several of those 7 are genuinely undrawn parts | | R6 | RapidOCR model download on first run | Low | Models ship with the wheel; verified — the Phase-1 run needed no network. Pin the version in `requirements.txt` | | R7 | Binarization threshold 150 is tuned on one page | Low | JPEG artifacts vary slightly. Verified ink fraction 2.2–10.6 % across all 30 pages, all bilevel line art. Make the threshold a `config.py` constant and sanity-check the resulting component count per page in the build report | | R8 | Part numbers appearing on up to 9 pages could confuse a user | Low | UI shows one tab per page with the section title, so the user picks by assembly | | R9 | Source-data defects (`351159`, `lsolierplatte`, `Kreuzsohnur`) | Low | Displayed as-is; flagged, never silently "corrected". Descriptions are never used for matching | | R10 | `pymupdf.find_tables()` behaviour could change across versions | Low | Pin `pymupdf`; `test_table_parser` asserts exact golden output on 3 pages and the global 861/705 totals, so a regression fails loudly | | R11 | Build time (~10 min) is annoying during development | Low | OCR results cached to `data/ocr_cache/`; localization can be re-run alone. Stages are independently invocable | ### Open questions to settle during implementation 1. **Which Tier-A line-following implementation wins** — pixel walk vs. probabilistic Hough restricted to a window around the label. Decide empirically on pages 39/40/51 in Phase 4; do not commit to either now. 2. **The right Tier-B fallback box size.** "Median component bbox on the page" is a guess; tune it against the gold set. 3. **Whether boxed-assembly callouts should draw a union box or simply highlight the whole diagram region** with an explanatory caption. Decide after seeing Tier C on real pages. 4. **Whether to expose the operation manual (pages 2–23) as browsable images** in the UI. Nice-to-have, zero indexing cost, and out of scope until Phase 7.