Spaces:
Running
Running
File size: 4,073 Bytes
5110b99 e4a8b06 5110b99 e4a8b06 5110b99 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 | # DEVLOG
Append-only log of design decisions, what changed, what didn't work, and why.
Newest entries at the bottom. Don't rewrite history here β commit fixes go in git.
---
## 2026-05-14 β v0.1 β initial scaffold
**Goal:** Stand up an end-to-end pipeline that takes any scientific PDF and
produces structured per-figure / per-table output, deployable as a Gradio app.
**Decisions:**
- Switched the extraction backend from raw PyMuPDF (notebook v0) to **docling**.
Reason: PyMuPDF's `doc.extract_image()` only yields raster images, which
silently skips vector figures (most plots in modern scientific PDFs). docling
uses a layout model + region rendering, so it gets both.
- Caption-to-figure pairing now comes from docling's layout (not center-distance
matching). More robust on multi-column papers.
- Added a `linker.py` step that scans body text for "Fig N" / "Table N"
references and attaches surrounding paragraphs to each figure/table. This is
the "figure context" the project brief calls out β wasn't in the notebook v0.
- Summarization uses Claude Sonnet 4.6 by default with vision input. Schema is
structured JSON so it's downstream-indexable for the LENR dashboard.
**Stack:**
- `miner.py` (docling) β `linker.py` (regex on body text) β `summarizer.py`
(Anthropic vision API) β `app.py` (Gradio).
**Open questions:**
- Quality on LENR-conference-proceedings PDFs (older scans) β might need OCR
enabled. Test before deploy.
- Whether multi-panel figures (a/b/c) need panel-level splitting.
---
## 2026-05-14 β v0.1.1 β Gradio type-mismatch fix
**Bug:** First click on "Mine the paper" crashed with
`AttributeError: 'list' object has no attribute 'expandtabs'` from inside
`gradio/components/markdown.py`.
**Cause:** Progress `yield` statements were passing `[]` to the `tables_view`
slot, but that's a `gr.Markdown` component which needs `str`. Same for
`json_download` which needs `None` not `""`.
**Fix:** Corrected types in all intermediate yields and the no-PDF early return.
---
## 2026-05-14 β v0.1.2 β transformers version pin
**Bug:** `ValueError: ... model type `rt_detr_v2` but Transformers does not
recognize this architecture`.
**Cause:** docling's layout model uses RT-DETR v2, which needs
`transformers >= 4.49`. Local env had an older version.
**Fix:** Pinned `transformers>=4.49`, `docling>=2.40`,
`docling-ibm-models>=3.4` in `requirements.txt`. Reinstalled.
---
## 2026-05-15 β v0.2 β first deployment + evaluation
**Deployment:**
- Live on Hugging Face Spaces: https://huggingface.co/spaces/deepesh-goel/figure-table-miner
- Pinned `gradio_sdk_version: 5.29.0` in README YAML to fix dark-mode rendering bug (5.0.0 default had broken theme switching)
- Pinned `huggingface_hub<1.0` to fix `HfFolder` import error
- Patched `gradio_client.utils._json_schema_to_python_type` to skip bool schemas (HF Spaces "No API found" bug)
- Wrapped pipeline in try/except so backend errors surface in the Status box instead of generic "Error"
**Evaluation:**
- Tested on 5 papers spanning ML, ACM/SIGKDD, Elsevier medical, and ICCF/LENR proceedings (55 pages, 27 figures, 16 tables ground truth)
- Tables: 16/16 detected (perfect across all five papers)
- Figures: 28 detected vs 27 ground truth, but heterogeneous β 3 papers near-perfect, 1 over-detected (P1: nested example figures inside a meta paper), 1 under-detected (P4: six panel figures merged into one composite)
- Caption number parsing succeeded 16/28 figures, 14/16 tables; failures concentrate in P1 and P4
- Discussion-paragraph linking attached context to 22/28 figures, 13/16 tables; ~100% when caption number parsed
- All summaries populated successfully; no LLM errors after Anthropic credits topped up
- Added two limitations to the report: "Nested figures (meta-content)" from P1, expanded "Multi-panel figures" with P4 as concrete case
## (next entry goes here)
<!--
Template for new entries:
## YYYY-MM-DD β vX.Y β short title
**Goal / Bug:**
**What was tried:**
**What worked / didn't:**
**Decision:**
-->
|