LiangLabUMB commited on
Commit
1c7ded6
·
verified ·
1 Parent(s): 1118e3e

Upload 4 files

Browse files
Files changed (3) hide show
  1. README.md +117 -17
  2. app.py +78 -23
  3. grid_seeded.py +443 -0
README.md CHANGED
@@ -1,5 +1,5 @@
1
  ---
2
- title: CellposeCellCounter_Mobile_v2
3
  emoji: 🚀
4
  colorFrom: purple
5
  colorTo: pink
@@ -8,24 +8,124 @@ sdk_version: 5.35.0
8
  app_file: app.py
9
  pinned: false
10
  license: apache-2.0
11
- short_description: Development build — optimized for phone use
12
  ---
13
 
14
- ## What's new in v2
15
 
16
- Based on Matthew's mobile version, with:
 
 
17
 
18
- - **Tap-to-zoom corner selection** — tap roughly near a corner, then precisely
19
- in the zoomed view. No more fighting to place points under your fingertip.
20
- - **Faster crop picker** — the preview no longer re-encodes a 12 MP image on
21
- every tap (~3 s → ~7 ms per tap).
22
- - **Removed duplicate functions** that silently renumbered cell IDs and could
23
- mismatch live/dead labels after adjusting the exclusion sliders.
24
- - **New defaults** — minimum size filter on, stereological counting on with
25
- the hemocytometer model, 1% exclusion width.
26
- - **Viability runs automatically** after segmentation.
27
- - **Cell concentration** at 1:1 (2x) and 1:10 (10x) trypan blue dilutions.
28
- - **Summary tab** now shows per-tab counts, their mean, concentrations for
29
- live/dead/total, and a CSV export covering all tabs plus the summary.
30
 
31
- ⚠️ Development build — not for manuscript data collection.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
+ title: CellposeCellCounter_Mobile
3
  emoji: 🚀
4
  colorFrom: purple
5
  colorTo: pink
 
8
  app_file: app.py
9
  pinned: false
10
  license: apache-2.0
11
+ short_description: Hemocytometer cell counter with one-tap grid detection
12
  ---
13
 
14
+ # CellposeCellCounter (mobile)
15
 
16
+ Open-source, low-cost cell counting from a phone photo of a hemocytometer.
17
+ Segmentation uses a fine-tuned Cellpose model; viability uses a separate
18
+ classifier. Everything runs in this Space — no local install, no instrument.
19
 
20
+ ## What changed in this version
 
 
 
 
 
 
 
 
 
 
 
21
 
22
+ **Setting the counting square now takes one tap instead of eight.**
23
+
24
+ Previously each of the four squares needed four corners, and each corner needed
25
+ two taps (tap roughly, then tap precisely in the zoom view) — 8 taps per image,
26
+ 32 for a full run of four squares.
27
+
28
+ Now the default is **Auto**: tap once near the centre of the 4×4 block and the
29
+ app finds all four corners, then shows them for you to confirm. If it looks
30
+ wrong, press **✎ Tap corners myself** and the original two-tap-per-corner flow
31
+ comes back exactly as it was. A failed detection drops you there automatically
32
+ and says why.
33
+
34
+ Detection is CPU-only OpenCV (`grid_seeded.py`). It never touches the GPU, so
35
+ it costs no ZeroGPU quota — and because it crops to the block, Cellpose then
36
+ sees roughly 7× fewer pixels than a full frame.
37
+
38
+ ### What it reports, and why you should read it
39
+
40
+ After each auto-detection the status line shows how far the proposed edges sit
41
+ from the printed rulings, measured against the image itself:
42
+
43
+ > **Square placed** — confidence 100% • edges sit 4 px (1.1% of a small square)
44
+ > from the printed rulings • tilt +1.5° • fit: similarity
45
+
46
+ That number is not the fit's own residual — it is an independent check that
47
+ re-finds the darkest line near each of the 10 fitted grid lines and reports the
48
+ disagreement. On the validation set it runs 4–16 px, i.e. 1–5% of one small
49
+ square. If it is large, confidence drops and the app hands you the manual flow
50
+ rather than cropping badly.
51
+
52
+ ## How the detection works
53
+
54
+ The hard part is that in a phone-through-eyepiece photo the rulings are only
55
+ about 5–10 grey levels darker than the background, sitting inside a circular
56
+ illuminated field, covered in bright cells. Thresholding, Canny and Hough all
57
+ discard them — three such prototypes each resolved only 1 of 4 test images.
58
+
59
+ What works is averaging *along* a ruling, so the line signal adds coherently
60
+ while cells do not:
61
+
62
+ 1. **Seed.** Your tap anchors the phase, because the centre of a 4×4 block sits
63
+ on the middle ruling intersection. Working in a window around the tap also
64
+ triples the effective resolution versus whole-frame detection.
65
+ 2. **Rotation** from a profile-contrast sweep. This matters: a 2° tilt smears a
66
+ 1-px ruling across ~15 px over a 450-px averaging span.
67
+ 3. **Period** from autocorrelation; **phase** from a 5-tooth comb fit scored by
68
+ its *weakest* tooth, so all five rulings must really be present.
69
+ 4. **Refine.** Relocate all 25 ruling intersections, then fit similarity (4
70
+ parameters), affine (6) and homography (8) and pick between them — a richer
71
+ model is only accepted if it beats the simpler one by more than its extra
72
+ parameters would gain by chance.
73
+ 5. **Verify.** The independent line-offset check described above.
74
+
75
+ ### On perspective distortion
76
+
77
+ The block is a projective quadrilateral when the phone is not square-on, and
78
+ the code fits that. But be careful reading the numbers: bootstrapping the
79
+ homography gives a keystone standard deviation of 0.011–0.020, so on a single
80
+ image **any keystone below about 4% is not distinguishable from noise.** On the
81
+ four validation photos the model selection therefore settles on a rotated
82
+ square, which is the honest answer — letting 8 free parameters chase that noise
83
+ is precisely how an auto-crop ends up wider on one side than the grid really is.
84
+
85
+ Tested against synthetic tilt, the model escalates as it should: mild tilt
86
+ (4% keystone) selects affine and lands within ~19 px of ground truth; strong
87
+ tilt (10%) selects the homography, detects the keystone as significant, but
88
+ correctly reports low confidence and hands over to manual rather than returning
89
+ a bad crop. Photograph reasonably square-on and it will place the square; tilt
90
+ hard and it will tell you to do it yourself.
91
+
92
+ ## Validation
93
+
94
+ On `test1_hemocytometer_Q1..Q4.jpg` (4080×3060, Improved Neubauer):
95
+
96
+ | | |
97
+ |---|---|
98
+ | ruling period | 329.7–338.7 px, CV 1.1% across images |
99
+ | tilt recovered | −1.0° to +4.5° |
100
+ | line-offset QC | 4–16 px mean = 1–5% of one small square |
101
+ | block size | 1283–1324 px |
102
+ | runtime | ~0.4 s per image, CPU |
103
+
104
+ Tap robustness — 36 taps jittered ±67 px (±0.20 of a square) around centre:
105
+ 36/36 detected, 36/36 locked onto the same block, corner agreement median 5 px.
106
+ Taps displaced a full half-square (onto the boundary with the neighbouring
107
+ block) drop to ~82% detected, which is ambiguous by construction and the reason
108
+ the flow asks you to confirm.
109
+
110
+ ## Files
111
+
112
+ | file | purpose |
113
+ |---|---|
114
+ | `app.py` | Gradio app |
115
+ | `grid_seeded.py` | one-tap grid detection (CPU, no GPU, no model weights) |
116
+ | `requirements.txt` | dependencies |
117
+
118
+ Models are pulled from the Hub at startup: the fine-tuned Cellpose weights and
119
+ `LiangLabUMB/viability_model`.
120
+
121
+ ## Known limits
122
+
123
+ - Tuned on one microscope / phone / eyepiece combination at ~333 px per 0.25 mm
124
+ square. The period search brackets roughly a 3× magnification range; a very
125
+ different setup means rechecking `PER_LO_FR` / `PER_HI_FR` in
126
+ `grid_seeded.py`, not new code.
127
+ - Assumes your tap is inside the intended block and the block is substantially
128
+ within frame.
129
+ - The `confidence` score is a heuristic calibrated on four images. The
130
+ line-offset QC is the number to trust; the accept threshold is exposed as
131
+ `ACCEPT_CONF`.
app.py CHANGED
@@ -14,6 +14,7 @@ import csv
14
  import joblib
15
  import os
16
  import time
 
17
 
18
  HF_REPO_ID = "myang4218/cellposemodel"
19
  HF_REPO_ID2 = "LiangLabUMB/viability_model"
@@ -1602,27 +1603,42 @@ HEMO_MODEL = "Hemocytometer Model"
1602
  GENERAL_MODEL = "General Model"
1603
 
1604
 
 
 
 
 
1605
  def _crop_picker(label):
1606
- """Upload + tap-to-zoom corner picker. Returns the components and states.
 
 
 
 
1607
 
1608
- Factored out because Tab 1 needs four of these and Tab 2 needs one; the
1609
- wiring is identical, only the label differs.
 
 
1610
  """
1611
  img_input = gr.Image(type="pil", label=label, image_mode="RGB", height=220)
1612
  crop_display = gr.Image(
1613
- type="pil", label="Tap roughly, then precisely in the zoom",
 
1614
  interactive=True, height=340, format="jpeg",
1615
  show_download_button=False,
1616
  )
1617
  crop_status = gr.Markdown("*Upload an image to set the counting square*")
 
 
1618
  with gr.Row():
1619
- clear_btn = gr.Button("✕ Clear", size="sm")
1620
- cancel_btn = gr.Button("↩ Back to full view", size="sm", visible=False)
 
1621
 
1622
  st = {
1623
  "img": img_input,
1624
  "display": crop_display,
1625
  "status": crop_status,
 
1626
  "points": gr.State(value=[]),
1627
  "base": gr.State(value=None),
1628
  "scale": gr.State(value=1.0),
@@ -1633,7 +1649,7 @@ def _crop_picker(label):
1633
  if img is None:
1634
  return None, None, 1.0, "*Upload an image to set the counting square*"
1635
  preview, scale = make_preview(img)
1636
- return preview, preview, scale, "*Tap roughly near corner 1*"
1637
 
1638
  img_input.change(
1639
  fn=on_upload, inputs=[img_input],
@@ -1644,54 +1660,93 @@ def _crop_picker(label):
1644
  return draw_polygon_overlay(
1645
  preview, [(int(x * scale), int(y * scale)) for x, y in points])
1646
 
1647
- def on_click(full_img, preview, points, scale, zoom, evt: gr.SelectData):
 
 
 
 
 
 
 
 
 
 
 
 
1648
  points = list(points or [])
1649
  if preview is None or full_img is None:
1650
- return gr.update(), points, zoom, gr.update(), gr.update()
1651
  if evt is None or evt.index is None or evt.index[0] is None:
1652
  return (gr.update(), points, zoom,
1653
- "*Tap not registered — try again inside the image*", gr.update())
 
1654
  tx, ty = int(evt.index[0]), int(evt.index[1])
1655
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1656
  if zoom is None:
1657
  if len(points) >= 4:
1658
  return (_overview(preview, points, scale), points, None,
1659
- "*4 corners set ✓ — ✕ Clear to redo*",
1660
- gr.update(visible=False))
1661
  view, z = make_zoom_view(full_img, tx / scale, ty / scale)
1662
  return (view, points, z,
1663
- f"*Zoomed — tap corner {len(points) + 1} precisely*",
1664
- gr.update(visible=True))
1665
 
1666
  x = zoom["x0"] + tx / zoom["zscale"]
1667
  y = zoom["y0"] + ty / zoom["zscale"]
1668
  W, H = full_img.size
1669
  pts = points + [(int(min(max(0, x), W - 1)), int(min(max(0, y), H - 1)))]
1670
  n = len(pts)
1671
- msg = (f"*{n} / 4 corners — tap roughly near corner {n + 1}*" if n < 4
1672
- else "*4 corners set ✓*")
1673
  return (_overview(preview, pts, scale), pts, None, msg,
1674
- gr.update(visible=False))
1675
 
1676
  crop_display.select(
1677
  fn=on_click,
1678
- inputs=[img_input, st["base"], st["points"], st["scale"], st["zoom"]],
1679
- outputs=[crop_display, st["points"], st["zoom"], crop_status, cancel_btn])
 
1680
 
1681
  cancel_btn.click(
1682
  fn=lambda preview, pts, sc: (
1683
  _overview(preview, pts or [], sc), None,
1684
- f"*Back to full view — {len(pts or [])} / 4 corners*",
1685
  gr.update(visible=False)),
1686
  inputs=[st["base"], st["points"], st["scale"]],
1687
  outputs=[crop_display, st["zoom"], crop_status, cancel_btn])
1688
 
 
 
 
 
 
1689
  clear_btn.click(
1690
- fn=lambda base: (base, [], None, "*Cleared — tap roughly near corner 1*",
1691
- gr.update(visible=False)),
1692
- inputs=[st["base"]],
1693
  outputs=[crop_display, st["points"], st["zoom"], crop_status, cancel_btn])
1694
 
 
 
 
 
 
 
 
1695
  return st
1696
 
1697
 
 
14
  import joblib
15
  import os
16
  import time
17
+ from grid_seeded import detect_from_seed
18
 
19
  HF_REPO_ID = "myang4218/cellposemodel"
20
  HF_REPO_ID2 = "LiangLabUMB/viability_model"
 
1603
  GENERAL_MODEL = "General Model"
1604
 
1605
 
1606
+ AUTO_LABEL = "Auto \u2014 one tap in the centre"
1607
+ MANUAL_LABEL = "Manual \u2014 tap the 4 corners"
1608
+
1609
+
1610
  def _crop_picker(label):
1611
+ """Upload + corner picker, with one-tap grid auto-detection.
1612
+
1613
+ Auto mode: one tap near the centre of the 4x4 block proposes all four
1614
+ corners via grid_seeded.detect_from_seed. The user accepts by moving on, or
1615
+ presses "Tap corners myself" to fall back.
1616
 
1617
+ Manual mode is the original tap-roughly-then-tap-in-the-zoom flow, byte for
1618
+ byte unchanged, and is also where a failed detection drops the user.
1619
+
1620
+ Detection is CPU-only and deliberately NOT behind @spaces.GPU.
1621
  """
1622
  img_input = gr.Image(type="pil", label=label, image_mode="RGB", height=220)
1623
  crop_display = gr.Image(
1624
+ type="pil",
1625
+ label="Auto: tap the centre of the block \u2014 Manual: tap roughly, then precisely",
1626
  interactive=True, height=340, format="jpeg",
1627
  show_download_button=False,
1628
  )
1629
  crop_status = gr.Markdown("*Upload an image to set the counting square*")
1630
+ crop_mode = gr.Radio([AUTO_LABEL, MANUAL_LABEL], value=AUTO_LABEL,
1631
+ label="How to set the counting square", interactive=True)
1632
  with gr.Row():
1633
+ clear_btn = gr.Button("\u2715 Clear", size="sm")
1634
+ manual_btn = gr.Button("\u270e Tap corners myself", size="sm")
1635
+ cancel_btn = gr.Button("\u21a9 Back to full view", size="sm", visible=False)
1636
 
1637
  st = {
1638
  "img": img_input,
1639
  "display": crop_display,
1640
  "status": crop_status,
1641
+ "mode": crop_mode,
1642
  "points": gr.State(value=[]),
1643
  "base": gr.State(value=None),
1644
  "scale": gr.State(value=1.0),
 
1649
  if img is None:
1650
  return None, None, 1.0, "*Upload an image to set the counting square*"
1651
  preview, scale = make_preview(img)
1652
+ return preview, preview, scale, "*Tap once in the centre of the 4\u00d74 block*"
1653
 
1654
  img_input.change(
1655
  fn=on_upload, inputs=[img_input],
 
1660
  return draw_polygon_overlay(
1661
  preview, [(int(x * scale), int(y * scale)) for x, y in points])
1662
 
1663
+ def _auto_status(res):
1664
+ qc = res.get("qc") or {}
1665
+ bits = ["**Square placed** \u2014 confidence {:.0%}".format(res["confidence"])]
1666
+ if qc:
1667
+ bits.append("edges sit {:.0f} px ({:.1f}% of a small square) from the "
1668
+ "printed rulings".format(qc["line_offset_mean_px"],
1669
+ 100 * qc["line_offset_mean_frac"]))
1670
+ bits.append("tilt {:+.1f}\u00b0".format(res["angle_deg"]))
1671
+ bits.append("fit: " + res["model"])
1672
+ return (" \u2022 ".join(bits) + " \n*Correct? Move on to the next image. "
1673
+ "If not, press **\u270e Tap corners myself**.*")
1674
+
1675
+ def on_click(full_img, preview, points, scale, zoom, mode, evt: gr.SelectData):
1676
  points = list(points or [])
1677
  if preview is None or full_img is None:
1678
+ return gr.update(), points, zoom, gr.update(), gr.update(), mode
1679
  if evt is None or evt.index is None or evt.index[0] is None:
1680
  return (gr.update(), points, zoom,
1681
+ "*Tap not registered \u2014 try again inside the image*",
1682
+ gr.update(), mode)
1683
  tx, ty = int(evt.index[0]), int(evt.index[1])
1684
 
1685
+ # ---- auto: one tap in the centre proposes all four corners -------
1686
+ if mode == AUTO_LABEL and zoom is None and not points:
1687
+ res = detect_from_seed(np.array(full_img.convert("RGB")),
1688
+ (tx / scale, ty / scale))
1689
+ if res.get("ok"):
1690
+ pts = [(int(x), int(y)) for x, y in res["corners"]]
1691
+ return (_overview(preview, pts, scale), pts, None,
1692
+ _auto_status(res), gr.update(visible=False), mode)
1693
+ return (_overview(preview, [], scale), [], None,
1694
+ "\u26a0\ufe0f *Auto-detect could not place the square reliably "
1695
+ "({}). Falling back \u2014 tap roughly near corner 1.*".format(
1696
+ res.get("reason", "no fit")),
1697
+ gr.update(visible=False), MANUAL_LABEL)
1698
+
1699
+ # ---- manual: original tap-roughly-then-tap-in-the-zoom flow ------
1700
  if zoom is None:
1701
  if len(points) >= 4:
1702
  return (_overview(preview, points, scale), points, None,
1703
+ "*4 corners set \u2713 \u2014 \u2715 Clear to redo*",
1704
+ gr.update(visible=False), mode)
1705
  view, z = make_zoom_view(full_img, tx / scale, ty / scale)
1706
  return (view, points, z,
1707
+ "*Zoomed \u2014 tap corner {} precisely*".format(len(points) + 1),
1708
+ gr.update(visible=True), mode)
1709
 
1710
  x = zoom["x0"] + tx / zoom["zscale"]
1711
  y = zoom["y0"] + ty / zoom["zscale"]
1712
  W, H = full_img.size
1713
  pts = points + [(int(min(max(0, x), W - 1)), int(min(max(0, y), H - 1)))]
1714
  n = len(pts)
1715
+ msg = ("*{} / 4 corners \u2014 tap roughly near corner {}*".format(n, n + 1)
1716
+ if n < 4 else "*4 corners set \u2713*")
1717
  return (_overview(preview, pts, scale), pts, None, msg,
1718
+ gr.update(visible=False), mode)
1719
 
1720
  crop_display.select(
1721
  fn=on_click,
1722
+ inputs=[img_input, st["base"], st["points"], st["scale"], st["zoom"], crop_mode],
1723
+ outputs=[crop_display, st["points"], st["zoom"], crop_status, cancel_btn,
1724
+ crop_mode])
1725
 
1726
  cancel_btn.click(
1727
  fn=lambda preview, pts, sc: (
1728
  _overview(preview, pts or [], sc), None,
1729
+ "*Back to full view \u2014 {} / 4 corners*".format(len(pts or [])),
1730
  gr.update(visible=False)),
1731
  inputs=[st["base"], st["points"], st["scale"]],
1732
  outputs=[crop_display, st["zoom"], crop_status, cancel_btn])
1733
 
1734
+ def on_clear(base, mode):
1735
+ msg = ("*Cleared \u2014 tap once in the centre of the 4\u00d74 block*"
1736
+ if mode == AUTO_LABEL else "*Cleared \u2014 tap roughly near corner 1*")
1737
+ return base, [], None, msg, gr.update(visible=False)
1738
+
1739
  clear_btn.click(
1740
+ fn=on_clear, inputs=[st["base"], crop_mode],
 
 
1741
  outputs=[crop_display, st["points"], st["zoom"], crop_status, cancel_btn])
1742
 
1743
+ manual_btn.click(
1744
+ fn=lambda base: (base, [], None, "*Tap roughly near corner 1*",
1745
+ gr.update(visible=False), MANUAL_LABEL),
1746
+ inputs=[st["base"]],
1747
+ outputs=[crop_display, st["points"], st["zoom"], crop_status, cancel_btn,
1748
+ crop_mode])
1749
+
1750
  return st
1751
 
1752
 
grid_seeded.py ADDED
@@ -0,0 +1,443 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """Seeded 4x4 hemocytometer block detection: one tap -> four corners.
2
+
3
+ Drop-in for CellposeCellCounter v2.5. Replaces the tap-roughly-then-tap-in-zoom
4
+ corner picker (8 taps per image, 32 for a run of four) with one tap in the
5
+ centre of each block, which the user then accepts or overrides.
6
+
7
+ Pure OpenCV/NumPy. CPU only -- do NOT decorate the caller with @spaces.GPU,
8
+ this must not spend ZeroGPU quota. ~0.35 s detect, ~0.65 s with verify=True,
9
+ on a 4080x3060 image.
10
+
11
+
12
+ Why this is hard, and what actually works
13
+ -----------------------------------------
14
+ In these phone-through-eyepiece photos the rulings are only ~5-10 grey levels
15
+ darker than background (centre-crop intensity std ~13), inside a circular
16
+ illuminated field, with hundreds of BRIGHT cells scattered over them. Every
17
+ threshold / Canny / Hough pipeline throws the rulings away -- measured: three
18
+ such prototypes each resolved only 1 of 4 test images.
19
+
20
+ What survives is averaging ALONG a ruling: signal adds coherently, cells and
21
+ debris do not. Everything here is built on that one idea.
22
+
23
+ Stage 1 -- similarity init.
24
+ A tap at the block centre anchors phase, because the centre of a 4x4 block
25
+ coincides with the middle ruling intersection. Windowing on the tap also
26
+ raises effective resolution ~3x over whole-frame detection. Rotation from a
27
+ profile-contrast sweep (de-rotation is not optional: a 2 deg tilt smears a
28
+ 1-px ruling across ~15 px over a 450-px averaging span), period from
29
+ autocorrelation, phase from a 5-tooth comb fit scored by its WEAKEST tooth
30
+ -- so a comb landing on four rulings and one gap loses to one landing on
31
+ five.
32
+
33
+ Stage 2 -- relocate 25 intersections, then choose a model.
34
+ Predict all 5x5 ruling intersections, relocate each locally, then fit
35
+ similarity (4 DOF), affine (6) and homography (8) and PICK BETWEEN THEM.
36
+
37
+ The model choice matters and is easy to get wrong. Bootstrapping the
38
+ homography on 60% subsets of the relocated points gives a keystone SD of
39
+ 0.011-0.020 against keystone estimates of 1.006-1.045 -- i.e. on a single
40
+ image, perspective below ~4% is NOT distinguishable from noise, and 8 free
41
+ parameters will happily absorb that noise into a skewed quad. So a more
42
+ complex model is only accepted when it beats the simpler one by more than
43
+ the improvement expected from the extra degrees of freedom alone
44
+ (sqrt((n-k_simple)/(n-k_complex))). On the four test images this selects the
45
+ homography for three and falls back for the one where it was not earning
46
+ its parameters.
47
+
48
+ Stage 3 (verify=True) -- independent QC.
49
+ Scan each of the 10 fitted lattice lines perpendicular and find where the
50
+ truly darkest line is, taking the MEDIAN intensity along the line (the
51
+ median is what makes this work -- the bright cells destroy a mean). Report
52
+ mean/max offset in pixels. This is a real accuracy number, measured against
53
+ the image rather than against the model's own fit, and it is what `ok`
54
+ is gated on. Measured 7-18 px mean offset = 2-5% of one square.
55
+
56
+ Do not try to iterate this into the fit: at ~5 grey levels of contrast the
57
+ darkest-offset estimate itself carries ~10-20 px of noise, so refitting on
58
+ it oscillates rather than converging. It is a check, not a correction.
59
+
60
+ The returned quad is exactly what warp_polygon_to_square() wants -- it calls
61
+ cv2.getPerspectiveTransform on 4 corners.
62
+
63
+ Corners are returned in the SAME pixel space as the image you pass in, ordered
64
+ top-left, top-right, bottom-right, bottom-left.
65
+
66
+
67
+ Measured on 20260817_test/test1_hemocytometer_Q1..Q4.jpg (4080x3060)
68
+ -------------------------------------------------------------------
69
+ ruling period 329.7 - 338.7 px (CV 1.1% across images)
70
+ tilt -1.0 to +4.5 deg
71
+ line offset (QC) 7 - 18 px mean = 2-5% of one square
72
+ ruling contrast 3 - 9 grey levels
73
+ runtime ~0.35 s, +0.3 s with verify
74
+
75
+ Tap robustness, 36 taps jittered +/-67 px (+/-0.20 period) about centre:
76
+ 36/36 detected, 36/36 locked onto the same block, corner agreement
77
+ median 5.0 px, p95 33.7 px.
78
+
79
+ Worst case, taps jittered the full +/-0.5 period (onto the boundary between
80
+ two blocks): ~82% detected, ~75% agree on the block. Ambiguous by
81
+ construction -- a tap halfway to the neighbour legitimately selects the
82
+ neighbour. It is why the flow proposes a quad for confirmation instead of
83
+ cropping silently.
84
+ """
85
+ import numpy as np
86
+ import cv2
87
+
88
+ WIN_FRAC = 0.55 # window side, as a fraction of min(H, W)
89
+ WORK = 1000 # window resampled to this many px on the long side
90
+ ANG_RANGE = 14.0 # deg, coarse rotation half-range
91
+ PER_LO_FR = 0.10 # period search window, as a fraction of WORK
92
+ PER_HI_FR = 0.32
93
+ N = 5 # a 4x4 block has exactly 5 rulings per axis
94
+
95
+ # Relocation patch half-size in periods, per refine iteration: coarse first
96
+ # (tolerate a poor initial guess), then tight. Always < 0.5 so the neighbouring
97
+ # rulings stay outside the patch.
98
+ PATCH_SCHED = (0.45, 0.30, 0.22)
99
+ MIN_STRENGTH = 1.0 # reject a relocation weaker than this multiple of the
100
+ # patch's own profile noise
101
+
102
+ MAX_LINE_OFF_FR = 0.09 # QC: mean |line offset| above this fraction of a
103
+ # period means the quad is not sitting on the rulings
104
+ ACCEPT_CONF = 0.45
105
+
106
+
107
+ # --------------------------------------------------------------- profiles
108
+ def _hp(p, w=41):
109
+ """High-pass a projection profile. Positive => darker than local mean."""
110
+ b = cv2.blur(p.reshape(-1, 1).astype(np.float32), (1, w)).ravel()
111
+ return b - p
112
+
113
+
114
+ def _profiles(a):
115
+ return _hp(a.mean(1)), _hp(a.mean(0))
116
+
117
+
118
+ def _rot(a, deg, ctr):
119
+ M = cv2.getRotationMatrix2D(ctr, deg, 1.0)
120
+ out = cv2.warpAffine(a, M, (a.shape[1], a.shape[0]),
121
+ flags=cv2.INTER_LINEAR, borderMode=cv2.BORDER_REFLECT)
122
+ return out, M
123
+
124
+
125
+ # ---------------------------------------------------- stage 1: similarity
126
+ def _find_angle(sq, ctr):
127
+ def score(th):
128
+ r, _ = _rot(sq, th, ctr)
129
+ c = int(r.shape[0] * 0.12)
130
+ r = r[c:-c, c:-c]
131
+ vy, vx = _profiles(r)
132
+ return vy.std() + vx.std()
133
+
134
+ coarse = max(np.arange(-ANG_RANGE, ANG_RANGE + 1e-9, 1.0), key=score)
135
+ return float(max(np.arange(coarse - 1.0, coarse + 1.0 + 1e-9, 0.1), key=score))
136
+
137
+
138
+ def _period(v, lo, hi):
139
+ x = v - v.mean()
140
+ ac = np.correlate(x, x, 'full')[len(x) - 1:]
141
+ if ac[0] <= 0:
142
+ return None, 0.0
143
+ ac = ac / ac[0]
144
+ hi = min(hi, len(ac) - 1)
145
+ if hi <= lo:
146
+ return None, 0.0
147
+ seg = ac[lo:hi]
148
+ k = int(np.argmax(seg))
149
+ return lo + k, float(seg[k])
150
+
151
+
152
+ def _fit_axis(v, seed, per0):
153
+ """Joint (period, phase) for a 5-tooth comb centred near `seed`."""
154
+ best = None
155
+ k = np.arange(N) - (N - 1) / 2.0
156
+ for per in np.arange(per0 * 0.85, per0 * 1.15 + 1e-9, 0.5):
157
+ for d in np.arange(-per / 2.0, per / 2.0 + 1e-9, 1.0):
158
+ pos = seed + d + per * k
159
+ if pos[0] < 0 or pos[-1] > len(v) - 1:
160
+ continue
161
+ t = v[np.round(pos).astype(int)]
162
+ sc = float(t.min())
163
+ if best is None or sc > best[0]:
164
+ best = (sc, float(seed + d), float(per), t.copy())
165
+ return best
166
+
167
+
168
+ # --------------------------------------------- stage 2: relocate and model
169
+ def _lattice():
170
+ j, i = np.meshgrid(np.arange(N), np.arange(N), indexing='ij')
171
+ return np.stack([i.ravel(), j.ravel()], 1).astype(np.float32)
172
+
173
+
174
+ def _apply(Hm, pts):
175
+ p = np.hstack([pts, np.ones((len(pts), 1), np.float32)])
176
+ q = (Hm @ p.T).T
177
+ return (q[:, :2] / q[:, 2:3]).astype(np.float32)
178
+
179
+
180
+ def _to_h(M):
181
+ """3x2 affine -> 3x3 homography."""
182
+ return np.vstack([M, [0.0, 0.0, 1.0]]).astype(np.float64)
183
+
184
+
185
+ def _relocate(g, p, du, dv, r):
186
+ """Find the true ruling intersection near predicted point `p`."""
187
+ r = int(max(8, r))
188
+ M = np.array([[du[0], dv[0], p[0] - r * du[0] - r * dv[0]],
189
+ [du[1], dv[1], p[1] - r * du[1] - r * dv[1]]], np.float32)
190
+ patch = cv2.warpAffine(g, M, (2 * r, 2 * r),
191
+ flags=cv2.INTER_LINEAR | cv2.WARP_INVERSE_MAP,
192
+ borderMode=cv2.BORDER_REFLECT).astype(np.float32)
193
+ w = max(11, (r | 1))
194
+ vb = _hp(patch.mean(1), w) # varies along dv -> locates the du-ruling
195
+ va = _hp(patch.mean(0), w) # varies along du -> locates the dv-ruling
196
+ m = max(1, int(r * 0.15))
197
+ if len(vb) - 2 * m < 3:
198
+ return None
199
+ # Prior: the true intersection should be near the prediction. Without this
200
+ # a strong cell edge or the neighbouring ruling can outvote the real line.
201
+ idx = np.arange(len(vb), dtype=np.float32)
202
+ prior = np.exp(-0.5 * ((idx - r) / (0.40 * r)) ** 2)
203
+ nb = max(float(vb.std()), 1e-6)
204
+ na = max(float(va.std()), 1e-6)
205
+ b = m + int(np.argmax((vb * prior)[m:-m]))
206
+ a = m + int(np.argmax((va * prior)[m:-m]))
207
+ s = min(float(vb[b]) / nb, float(va[a]) / na)
208
+ if s < MIN_STRENGTH:
209
+ return None
210
+ q = np.array(p, np.float32) + (a - r) * np.asarray(du) + (b - r) * np.asarray(dv)
211
+ return q.astype(np.float32), s
212
+
213
+
214
+ def _correspondences(g, Hm, per_px, patch_fr):
215
+ """Predict 25 intersections and relocate each. Returns (src, dst, strength)."""
216
+ lat = _lattice()
217
+ pred = _apply(Hm, lat)
218
+ dx = _apply(Hm, lat + np.array([[0.06, 0]], np.float32)) - pred
219
+ dy = _apply(Hm, lat + np.array([[0, 0.06]], np.float32)) - pred
220
+ src, dst, st = [], [], []
221
+ for k in range(len(lat)):
222
+ nu, nv = np.linalg.norm(dx[k]), np.linalg.norm(dy[k])
223
+ if nu < 1e-6 or nv < 1e-6:
224
+ continue
225
+ out = _relocate(g, pred[k], dx[k] / nu, dy[k] / nv, patch_fr * per_px)
226
+ if out is None:
227
+ continue
228
+ q, s = out
229
+ if not (0 <= q[0] < g.shape[1] and 0 <= q[1] < g.shape[0]):
230
+ continue
231
+ src.append(lat[k]); dst.append(q); st.append(s)
232
+ if len(src) < 8:
233
+ return None
234
+ return np.array(src, np.float32), np.array(dst, np.float32), float(np.mean(st))
235
+
236
+
237
+ def _select_model(src, dst, per_px):
238
+ """Fit similarity / affine / homography and pick the justified one.
239
+
240
+ A richer model is accepted only if it beats the simpler fit by more than
241
+ the RMS reduction expected from its extra free parameters alone. Otherwise
242
+ 8 parameters silently absorb relocation noise into a skewed quad -- which
243
+ is exactly how an auto-crop ends up wider on one side than the grid is.
244
+ """
245
+ n = len(src)
246
+ cands = []
247
+
248
+ Ms, _ = cv2.estimateAffinePartial2D(src, dst, method=cv2.RANSAC,
249
+ ransacReprojThreshold=0.07 * per_px)
250
+ if Ms is not None:
251
+ cands.append(("similarity", 4, _to_h(Ms)))
252
+ Ma, _ = cv2.estimateAffine2D(src, dst, method=cv2.RANSAC,
253
+ ransacReprojThreshold=0.07 * per_px)
254
+ if Ma is not None:
255
+ cands.append(("affine", 6, _to_h(Ma)))
256
+ Hh, _ = cv2.findHomography(src, dst, cv2.RANSAC, 0.07 * per_px)
257
+ if Hh is not None:
258
+ cands.append(("homography", 8, Hh.astype(np.float64)))
259
+ if not cands:
260
+ return None
261
+
262
+ def rms(Hm):
263
+ return float(np.sqrt((np.linalg.norm(_apply(Hm, src) - dst, axis=1) ** 2).mean()))
264
+
265
+ scored = [(name, k, Hm, rms(Hm)) for name, k, Hm in cands]
266
+ best = scored[0]
267
+ for name, k, Hm, r in scored[1:]:
268
+ if k <= best[1]:
269
+ continue
270
+ if n - k <= 1:
271
+ continue
272
+ expected = np.sqrt((n - best[1]) / float(n - k)) # chance improvement
273
+ if best[3] / max(r, 1e-6) > expected * 1.05: # 5% margin
274
+ best = (name, k, Hm, r)
275
+ return {"model": best[0], "dof": best[1], "H": best[2], "rms": best[3],
276
+ "rms_all": {s[0]: round(s[3], 2) for s in scored}}
277
+
278
+
279
+ # ------------------------------------------------- stage 3: independent QC
280
+ def _scan_line(g, Hm, axis, t, per, n=140, span=0.22, step=0.01):
281
+ """Offset (lattice units) of the truly darkest line near fitted line t."""
282
+ s = np.linspace(0.08, 3.92, n)
283
+ offs = np.arange(-span, span + 1e-9, step)
284
+ vals = np.empty(len(offs), np.float32)
285
+ for k, dt in enumerate(offs):
286
+ L = (np.stack([np.full(n, t + dt), s], 1) if axis == 0
287
+ else np.stack([s, np.full(n, t + dt)], 1))
288
+ P = _apply(Hm, L.astype(np.float32))
289
+ x = np.clip(P[:, 0], 0, g.shape[1] - 1).astype(int)
290
+ y = np.clip(P[:, 1], 0, g.shape[0] - 1).astype(int)
291
+ vals[k] = np.median(g[y, x].astype(np.float32)) # median kills cells
292
+ k = int(np.argmin(vals))
293
+ return float(offs[k]), float(np.median(vals) - vals[k])
294
+
295
+
296
+ def verify_lines(g, Hm, per):
297
+ """How far the 10 fitted lattice lines sit from the real rulings, in px."""
298
+ o, c = [], []
299
+ for axis in (0, 1):
300
+ for t in range(N):
301
+ dt, con = _scan_line(g, Hm, axis, t, per)
302
+ o.append(abs(dt) * per)
303
+ c.append(con)
304
+ return (float(np.mean(o)), float(np.max(o)), float(np.mean(c)),
305
+ float(np.min(c)))
306
+
307
+
308
+ # ------------------------------------------------------------------ public
309
+ def detect_from_seed(image, seed_xy, verify=True, want_debug=False):
310
+ """One tap -> the four corners of the surrounding 4x4 counting block.
311
+
312
+ Parameters
313
+ ----------
314
+ image : ndarray, HxW grayscale or HxWx3 RGB. Use the FULL-resolution image;
315
+ corners come back in that same pixel space.
316
+ seed_xy : (x, y) tap position in that same pixel space.
317
+ verify : run the independent line-offset QC (adds ~0.3 s). Strongly
318
+ recommended -- it is the only check that looks at the image rather than
319
+ at the model's own residual.
320
+
321
+ Returns a JSON-safe dict. Always check ``ok`` before using ``corners``.
322
+ """
323
+ g = cv2.cvtColor(image, cv2.COLOR_RGB2GRAY) if image.ndim == 3 else image
324
+ g = np.ascontiguousarray(g)
325
+ H, W = g.shape[:2]
326
+ sx, sy = float(seed_xy[0]), float(seed_xy[1])
327
+ if not (0 <= sx < W and 0 <= sy < H):
328
+ return {"ok": False, "reason": "tap was outside the image"}
329
+
330
+ half = int(WIN_FRAC * min(H, W) / 2)
331
+ x0 = int(np.clip(sx - half, 0, max(0, W - 2 * half)))
332
+ y0 = int(np.clip(sy - half, 0, max(0, H - 2 * half)))
333
+ win = g[y0:min(H, y0 + 2 * half), x0:min(W, x0 + 2 * half)]
334
+ if min(win.shape) < 200:
335
+ return {"ok": False, "reason": "image too small for grid detection"}
336
+
337
+ scale = WORK / float(max(win.shape))
338
+ sq = cv2.resize(win, (int(win.shape[1] * scale), int(win.shape[0] * scale)),
339
+ interpolation=cv2.INTER_AREA).astype(np.float32)
340
+ su, sv = (sx - x0) * scale, (sy - y0) * scale
341
+ ctr = (sq.shape[1] / 2.0, sq.shape[0] / 2.0)
342
+
343
+ # ---- stage 1 --------------------------------------------------------
344
+ ang = _find_angle(sq, ctr)
345
+ rot, M = _rot(sq, ang, ctr)
346
+ su_r, sv_r = M @ np.array([su, sv, 1.0])
347
+ vy, vx = _profiles(rot)
348
+ lo, hi = int(PER_LO_FR * WORK), int(PER_HI_FR * WORK)
349
+ py, acy = _period(vy, lo, hi)
350
+ px, acx = _period(vx, lo, hi)
351
+ if py is None or px is None:
352
+ return {"ok": False, "reason": "no repeating grid found near the tap"}
353
+ per0 = 0.5 * (py + px)
354
+ fy = _fit_axis(vy, sv_r, per0)
355
+ fx = _fit_axis(vx, su_r, per0)
356
+ if fy is None or fx is None:
357
+ return {"ok": False, "reason": "the 4x4 block does not fit around the tap"}
358
+ cy_r, pery, ty = fy[1], fy[2], fy[3]
359
+ cx_r, perx, tx = fx[1], fx[2], fx[3]
360
+
361
+ quad_r = np.array([[cx_r - 2 * perx, cy_r - 2 * pery],
362
+ [cx_r + 2 * perx, cy_r - 2 * pery],
363
+ [cx_r + 2 * perx, cy_r + 2 * pery],
364
+ [cx_r - 2 * perx, cy_r + 2 * pery]], np.float32)
365
+ Minv = cv2.invertAffineTransform(M)
366
+ q1 = cv2.transform(quad_r.reshape(-1, 1, 2), Minv).reshape(-1, 2) / scale \
367
+ + np.array([x0, y0], np.float32)
368
+
369
+ noise = 0.5 * (vy.std() + vx.std())
370
+ snr = min(float(ty.min()), float(tx.min())) / max(noise, 1e-6)
371
+ drift = float(np.hypot(cx_r - su_r, cy_r - sv_r)) / per0
372
+ per_orig = 0.5 * (perx + pery) / scale
373
+
374
+ # ---- stage 2 --------------------------------------------------------
375
+ ideal = np.array([[0, 0], [4, 0], [4, 4], [0, 4]], np.float32)
376
+ Hm = cv2.getPerspectiveTransform(ideal, q1.astype(np.float32))
377
+ sel, strength = None, 0.0
378
+ for pf in PATCH_SCHED:
379
+ co = _correspondences(g, Hm, per_orig, pf)
380
+ if co is None:
381
+ break
382
+ src, dst, strength = co
383
+ s = _select_model(src, dst, per_orig)
384
+ if s is None:
385
+ break
386
+ sel, Hm = s, s["H"]
387
+ quad = _apply(Hm, ideal) if sel else q1
388
+
389
+ # ---- stage 3: independent QC ----------------------------------------
390
+ qc = None
391
+ if verify and sel:
392
+ mo, mx, mc, minc = verify_lines(g, Hm, per_orig)
393
+ qc = {"line_offset_mean_px": round(mo, 1), "line_offset_max_px": round(mx, 1),
394
+ "line_offset_mean_frac": round(mo / per_orig, 4),
395
+ "ruling_contrast_mean": round(mc, 1),
396
+ "ruling_contrast_min": round(minc, 1)}
397
+
398
+ # ---- geometry, confidence -------------------------------------------
399
+ def side(a, b):
400
+ return float(np.linalg.norm(quad[a] - quad[b]))
401
+
402
+ top, rgt, bot, lft = side(0, 1), side(1, 2), side(3, 2), side(0, 3)
403
+ keystone = max(max(top, bot) / max(min(top, bot), 1e-6),
404
+ max(lft, rgt) / max(min(lft, rgt), 1e-6))
405
+
406
+ conf = float(np.clip(snr / 0.8, 0, 1)
407
+ * np.clip((0.75 - drift) / 0.35, 0, 1)
408
+ * np.clip(min(acx, acy) / 0.15, 0, 1))
409
+ if sel:
410
+ conf *= float(np.clip((0.16 * per_orig - sel["rms"]) / (0.10 * per_orig), 0, 1))
411
+ else:
412
+ conf *= 0.5
413
+ if qc:
414
+ # the QC term dominates on purpose: this is the only number measured
415
+ # against the image rather than against the fit's own residual
416
+ conf *= float(np.clip(
417
+ (MAX_LINE_OFF_FR - qc["line_offset_mean_frac"]) / (0.6 * MAX_LINE_OFF_FR),
418
+ 0, 1))
419
+
420
+ out = {
421
+ "ok": bool(conf >= ACCEPT_CONF),
422
+ "confidence": round(conf, 3),
423
+ "corners": [[int(round(float(x))), int(round(float(y)))] for x, y in quad],
424
+ "model": sel["model"] if sel else "similarity(comb only)",
425
+ "model_rms_px": None if not sel else round(sel["rms"], 2),
426
+ "model_rms_all": None if not sel else sel["rms_all"],
427
+ "angle_deg": round(float(ang), 2),
428
+ "period_px": round(float(per_orig), 1),
429
+ "block_px": [int(round(0.5 * (top + bot))), int(round(0.5 * (lft + rgt)))],
430
+ "keystone_ratio": round(float(keystone), 4),
431
+ # Honest about this one: bootstrapping gives a keystone SD of
432
+ # 0.011-0.020, so a ratio under ~1.04 on a single image is not
433
+ # distinguishable from relocation noise. Do not report it as tilt.
434
+ "keystone_significant": bool(keystone > 1.04),
435
+ "tap_offset_periods": round(float(drift), 3),
436
+ "tooth_snr": round(float(snr), 2),
437
+ "mean_line_strength": round(float(strength), 2) if sel else None,
438
+ "qc": qc,
439
+ "reason": "ok" if conf >= ACCEPT_CONF else "low confidence",
440
+ }
441
+ if want_debug:
442
+ out["_H"], out["_q1"] = Hm, q1
443
+ return out