kiyer Claude Fable 5 commited on
Commit
3e4867e
·
1 Parent(s): 7ff862e

docs(plan): Task 5 must preserve blank lines in strip_page_artifacts

Browse files
docs/superpowers/plans/2026-06-10-astroparse.md CHANGED
@@ -346,6 +346,29 @@ curl -L -o backend/tests/fixtures/mowla_iyer_2024.pdf https://arxiv.org/pdf/2402
346
  ```
347
  Verify it's a PDF: `file backend/tests/fixtures/mowla_iyer_2024.pdf` → "PDF document".
348
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
349
  - [ ] **Step 2: Write failing tests**
350
 
351
  ```python
 
346
  ```
347
  Verify it's a PDF: `file backend/tests/fixtures/mowla_iyer_2024.pdf` → "PDF document".
348
 
349
+ - [ ] **Step 1b: Fix `strip_page_artifacts` to preserve blank lines** (integration fix found in Task 4 review: `parse_pdf` rejoins lines and segments on blank lines, so artifact-stripping must NOT drop them). TDD: add to `backend/tests/test_parse_clean.py`:
350
+
351
+ ```python
352
+ def test_strip_page_artifacts_preserves_blank_lines():
353
+ lines = ["First para.", "", "12", "Second para."]
354
+ assert strip_page_artifacts(lines) == ["First para.", "", "Second para."]
355
+ ```
356
+
357
+ Run (FAIL), then change the filter in `strip_page_artifacts` so blank lines pass through (drop only artifacts):
358
+
359
+ ```python
360
+ def strip_page_artifacts(lines: list[str]) -> list[str]:
361
+ return [
362
+ l for l in lines
363
+ if not l.strip()
364
+ or (not _PAGE_NUM.match(l.strip())
365
+ and not _ARXIV_STAMP.search(l)
366
+ and not _DOI_STAMP.match(l.strip()))
367
+ ]
368
+ ```
369
+
370
+ Run (PASS — the existing 3 tests must also still pass; their inputs contain no blank lines).
371
+
372
  - [ ] **Step 2: Write failing tests**
373
 
374
  ```python