| # HLT Geometric Vision — technical documentation | |
| Reference document for **anyone implementing the inference model**: input contract, tensor | |
| preprocessing, output decoding, and the algorithmic specification of rectangle extraction plus the | |
| geometric transforms. Training figures live in [README.md](README.md); everything here is defined | |
| precisely enough to be reimplemented in a language other than Python. | |
| Contents: [1. Two ways to use the model](#1-two-ways-to-use-the-model) · | |
| [2. Input: exact specification](#2-input-exact-specification) · | |
| [3. Output: contract of the ONNX artifact](#3-output-contract-of-the-onnx-artifact) · | |
| [4. Preprocessing, step by step](#4-preprocessing-step-by-step) · | |
| [5. Decoding the output](#5-decoding-the-output) · | |
| [6. Rectangle extraction — algorithmic specification](#6-rectangle-extraction--algorithmic-specification) · | |
| [7. Geometric transforms and coordinate mapping](#7-geometric-transforms-and-coordinate-mapping) · | |
| [8. Extractor parameter table](#8-extractor-parameter-table) · | |
| [9. Known limitations](#9-known-limitations-what-does-not-work) · | |
| [10. Examples in the repository](#10-examples-in-the-repository) | |
| --- | |
| ## 1. Two ways to use the model | |
| The ONNX artifact (`model.onnx`) does **not** do everything on its own: it produces heatmaps and the | |
| vertices of **one single** polygon. Which path to use depends on the domain. | |
| | Path | What runs | Result | When to use it | | |
| |---|---|---|---| | |
| | **A — full pipeline** | ONNX `image -> vertices` (differentiable Hough + keypoint initialization + neural refiner all **inside** the graph) | 1 polygon, up to 8 vertices, sub-pixel | Training domain: i.e. a **clean, well-framed rectangle**, one object per frame. Measured vertex RMSE: 0.81 px (val), 0.38 px (test) at 512² | | |
| | **B — heatmaps + external geometry** | ONNX (uses only `edge_map`) **+** the geometric extractor `hlt.postproc` (numpy/OpenCV, **no weights**) | **N** rectangles/quadrilaterals, nested and cropped-at-border ones included | Real photographs. This is the path used by the [examples](#10-examples-in-the-repository) in this card | | |
| Why path B exists: on real imagery the `edge_map` transfers, the `keypoint_map` **does not** (domain | |
| gap, documented below), and pipeline A — trained with `max_polygons=1` — merges the corners of | |
| different objects into one cloud and outputs a "star-shaped" polygon. On real photos A is discarded | |
| up front; B keeps only the geometry downstream of the network. | |
| > **Practical rule:** if the target is a well-framed rectangle on a clean background use A (sub-pixel | |
| > accuracy, a single run). For indoor/outdoor photos with several frames or windows use B and ignore | |
| > `vertices`. | |
| --- | |
| ## 2. Input: exact specification | |
| The artifact exposes **one single** input tensor: | |
| | Name | Shape | Type | Range | Notes | | |
| |---|---|---|---|---| | |
| | `input_image` | `(B, 3, 512, 512)` | `float32` | `[0.0, 1.0]` | **RGB** channels, **not** normalized (no ImageNet mean/std), **not** BGR | | |
| Non-negotiable constraints (checked at export time by `export_pipeline`): | |
| 1. **Fixed 512×512 resolution.** The Hough accumulator inside the graph is sized on `H,W`: a | |
| rectangular or differently sized input is **not** supported (no dynamic spatial axis is declared). | |
| This export was produced at 512² from `runs/g3_keypoint/best.pt` (`data.img_size: 512`). For a | |
| different resolution re-export the model (`--img-size`, a multiple of 32) and re-calibrate the | |
| extractor thresholds. | |
| 2. **Dynamic batch axis** (`dynamic_axes` on axis 0 of the input and of every output). `B` can be any | |
| integer ≥ 1; the export parity check runs at `B = 2` precisely to exercise the declared axis. | |
| 3. **Contiguous NCHW layout.** The ONNXRuntime binding needs a contiguous buffer: in C/C++ no | |
| unusual strides; in numpy use `np.ascontiguousarray`. | |
| 4. **No state or mask tensor is fed in.** The padding and the valid-area mask used by path B live | |
| **outside** the model: they are needed by the geometric extractor, not by the network. | |
| Training used a **procedurally rendered synthetic** dataset (convex polygons with 4–6 vertices over a | |
| background bank, `radius_range: [0.16, 0.38]`, `center_margin: 0.24`, geometric and photometric | |
| augmenting). That is what defines what the model "knows". | |
| --- | |
| ## 3. Output: contract of the ONNX artifact | |
| Fixed order of the output bindings (7 tensors), shapes shown for `B = 1` (a larger batch replaces | |
| the leading `1`): | |
| | # | Name | Shape | Type | Content | | |
| |---|---|---|---|---| | |
| | 0 | `vertices` | `(B, 1, 8, 2)` | float32 | **refined** vertices, in pixels **of the input tensor** (the 512² frame) | | |
| | 1 | `vertex_mask` | `(B, 1, 8)` | float32 | `1` = active vertex, `0` = empty slot. **This is the field to use** to slice `vertices` | | |
| | 2 | `confidence` | `(B, 1, 8)` | float32 | **uncalibrated** head (≈0.49 for this run): informational, **never** use as an acceptance threshold | | |
| | 3 | `coarse_vertices` | `(B, 1, 8, 2)` | float32 | `P₀` before the neural refiner: useful for diagnostics | | |
| | 4 | `coarse_mask` | `(B, 1, 8)` | float32 | mask of `coarse_vertices` | | |
| | 5 | `edge_map` | `(B, 1, 512, 512)` | float32 | per-pixel edge probability in `[0,1]` ← **input of path B** | | |
| | 6 | `keypoint_map` | `(B, 1, 512, 512)` | float32 | per-pixel corner probability in `[0,1]` (transfers poorly to real photos) | | |
| Implementation caveats: | |
| - **Slice by mask, never by a fixed count**: read `vertex_mask > 0.5` and keep only the active slots. | |
| Here `max_vertices = 8` but the actual number of active vertices is typically 4–6. | |
| - **One polygon per sample** (`max_polygons = 1` in the configuration): the "object" dimension of the | |
| output is a single polygon per input image. | |
| - No softmax/sigmoid to apply: the tensors already contain probabilities. | |
| - If `vertex_mask` is **all zeros** (no Hough peak above `hough_min_strength = 0.5`) the output is not | |
| "a rectangle with zeroed vertices": it is a *not detected*. | |
| - Numerical PyTorch↔ONNXRuntime parity was verified at export: `vertex_max_diff_px` ≈ 2·10⁻³ px, | |
| `edge_map` 7.7·10⁻⁶, `keypoint_map` 2.8·10⁻⁶ (see the build-generated appendix for this artifact). | |
| - **Do not test the export with random inputs.** With noise input the edge map is flat, the Hough | |
| peaks are numerically tied but `TopK/Sort/argmax` pick different candidates (hard indices, no | |
| guaranteed tie-break across runtimes) → tens of px of *fake* difference. The parity check uses a | |
| crisp drawn quadrilateral (see `reference_input`). | |
| --- | |
| ## 4. Preprocessing, step by step | |
| The model performs no preprocessing inside the graph. This is the exact sequence — identical in | |
| training, prediction and this release: | |
| 1. **Decode** the image as BGR (OpenCV) or RGB; if it has an alpha channel, composite it over white. | |
| 2. **Letterbox to a 512² square, aspect preserved, with a replicated border** | |
| (`cv2.BORDER_REPLICATE`) — **not** a constant gray fill: | |
| ```text | |
| scale = 512 / max(h0, w0) | |
| nw = round(w0 * scale); nh = round(h0 * scale) # cv2.INTER_AREA | |
| ox = (512 - nw) // 2 ; oy = (512 - nh) // 2 # offset of the valid window | |
| ``` | |
| Measured reason: a constant gray fill creates **two artificial lines as long as the image** in the | |
| edge map, which saturate the Hough line pool and shift the NMS. | |
| 3. **Valid-area mask** `valid` (512×512, `True` inside the real-image window), used **by the extractor | |
| only**: eroding it by 5 px (`np.ones((5,5))`) stops the four artificial frame borders from | |
| contributing votes. | |
| 4. **Tensor conversion**: `RGB`, `float32`, `/255.0`, `HWC -> CHW`, `ascontiguousarray`, add the batch | |
| axis. | |
| Nothing else: no non-uniform resize, no CLAHE, no denoising, no per-channel normalization. | |
| --- | |
| ## 5. Decoding the output | |
| Path A (full pipeline): | |
| ```python | |
| outs = sess.run(None, {"input_image": x}) | |
| vertices, vmask, conf, coarse, cmask, edge_map, kp_map = outs | |
| v = vertices[0][0][vmask[0][0] > 0.5] # (K,2) in 512-frame coordinates | |
| ``` | |
| When model vertices are used on a non-square image, map them back to the original frame with the | |
| inverse mapping in §7. Path B uses **only** `edge_map`. | |
| --- | |
| ## 6. Rectangle extraction — algorithmic specification | |
| Reference implementation: `hlt/postproc/quads.py` (`extract_quads`, `hybrid_edge_map`, | |
| `extract_quads_hybrid`). Dependencies: numpy + OpenCV (headless). **No weights, no training.** | |
| Input: edge map `float32 (H,W)` in `[0,1]` + optional grayscale image `(H,W)` for the contrast test. | |
| Output: list of `{corners(4,2), score, edge_cov(4), contrast_cov(4), polarity, area, | |
| refine_shift_px}`, sorted by score. | |
| Steps, in order: | |
| 1. **Hybrid edge map** (path B). `edge = max(edge_map, Canny(blur(gray, σ=1.2), 20, 53)/255)`, zeroed | |
| outside `valid`. The network misses low-contrast sides; with no line for a side the candidate is | |
| never born, so classic Canny fills the holes. Canny thresholds: lower 20, upper 53 (ratio ≈ 1:2.6). | |
| 2. **Threshold and dilation**: `near = dilate(edge > edge_thresh)` with a disk of radius `band_px` | |
| (spatial tolerance of the support, 3 px). | |
| 3. **Weighted, oriented Hough**: accumulator with `n_theta = 360` bins over `[0, π)` (0.5°) and `rho` | |
| at `rho_step = 1.0` px. Each edge pixel votes **only** for the `theta` values within | |
| ±`vote_halfwidth_deg` (6°) of its own local normal, and **only** if local orientation coherence is | |
| ≥ `vote_coherence` (0.5). Reason: without oriented voting, textural curves (wood grain, brush | |
| strokes, railings) accumulate votes like a straight line and saturate the pool — measured: 200 | |
| lines were not enough on a real photo with 29 horizontal background lines against 9 useful | |
| vertical ones. | |
| 4. **Line NMS**: bins ±`nms_theta_bins` (3 bins = ±3°) and ±`nms_rho_bins` (6 bins = ±6 px), then the | |
| `min_line_votes` filter (25, the sum of `edge_prob` along the line) **and** `min_rel_votes` | |
| (5% of the strongest line). Surviving lines (≤ `n_lines = 500`) are **refitted** to the edge pixels | |
| within `refit_band_px` (3 px) by weighted least squares, 2 iterations. | |
| 5. **Candidate generation**: take **all** line quadruples `(a,b,c,d)` in cyclic order; the 4 corners | |
| are the consecutive intersections `(a∩b, b∩c, c∩d, d∩a)`, closed form. **Vectorized block-wise** | |
| enumeration (`chunk = 80000` combinations per block: bounded memory, never Python lists). Cost: | |
| `C(N,4) × 3 cyclic orders` with `N ≤ n_lines`; this is why the filter that matters is line | |
| **quality** (step 4), not line count. | |
| 6. **Side coverage in O(1)**: cumulative sums along each line; the coverage of a segment is the | |
| fraction of samples with edge inside the `band_px` band. The `trim = 6%` fractions at each end of | |
| the side are ignored (corners are the region most damaged by the refit). Thresholds: `min_side_cov` | |
| (the **worst** side) and `min_mean_cov` (mean over the perimeter) — see §8. | |
| **Sides flush with the image border are not penalized**: outside the canvas there is no pixel, and | |
| measuring them used to count as "no edge" (the cause of false rejections on cropped objects). | |
| 7. **Shape test**: `shape="rect"` requires the 4 interior angles to lie inside | |
| `corner_angle_deg = [50°, 130°]` → **rectangles in perspective too** (a frame shot obliquely | |
| passes, a degenerate trapezoid does not). `shape="quad"` accepts any convex quadrilateral. | |
| Geometric filters: `min_side_px = 30`, area within | |
| `[min_area_frac, max_area_frac] = [1%, 60%]` of the image, `max_aspect = 6`, corners tolerated up | |
| to `border_margin_px = 8` outside the canvas. | |
| 8. **Gap test** (`max_gap_frac = 0.20`): if along a side there is a stretch longer than 20% of the | |
| side length with no aligned edge, the candidate is a **stitched group** (several neighbouring | |
| objects faking one rectangle) and is rejected. | |
| 9. **Coherent-polarity contrast test** (only with `gray`): for each side, measure the gray-level | |
| difference between an inner band and an outer band (`contrast_px = (3,8)`), and require the sign to | |
| be **coherent on all 4 sides** (`contrast_min = 12` levels). A rectangle invented by joining | |
| uncorrelated edges has mixed polarity. Non-measurable samples (band outside the canvas) count as | |
| passed. With `min_contrast_cov = 0` the test does not filter (default); with | |
| `min_contrast_cov > 0` it rejects below threshold, and the score becomes | |
| `mean(contrast_cov) + 0.5 * mean(edge_cov)`. | |
| 10. **Rectangle NMS**: IoU `nms_iou = 0.30`, containment `nms_contain = 0.70`. With | |
| `prefer_outer = True`, near-duplicates keep the **outer** contour (the outer edge of the frame, | |
| not the mount board). With `nested = True` nested rectangles are kept as well (glass inside a | |
| frame, sash inside a window); a box that "extends" a child (`stretch_margin`) or that is covered | |
| by ≥ `group_cover` (60%) by ≥ 2 disjoint children is dropped. At most `max_quads = 64` outputs. | |
| 11. **Final refinement**: every side is refitted to the edge pixels inside the `refine_band_px = 4` | |
| band (different from `refit_band_px = 3`, used in step 4 for the individual lines) and corners are | |
| recomputed as the intersections of the sides; a shift larger than `refine_max_shift_px = 10` px is | |
| refused (the corner keeps its original position). `refine_shift_px` reports the mean shift | |
| applied, useful as a quality diagnostic. | |
| **Corner order in the output** (`canonical_order`): clockwise in image coordinates (y downwards), | |
| first corner = the vertex with the smallest `x + y`. This is a canonicalization: it lets predictions | |
| and ground truth be compared without rotation/direction ambiguity. | |
| ### Complexity and memory | |
| `Hough`: O(edge pixels × voted bins per pixel). `Enumeration`: dominated by `C(n_lines,4)`, while | |
| the `max_eval = 6000` candidate cut (sorted by coverage) bounds the number of expensive checks; the | |
| expensive checks are contrast + gap. **The real cost depends on the density of valid lines in the | |
| scene**, not on resolution alone: for the two examples in this release, measured in build on a 4 | |
| thread CPU, extraction cost 0.57 s (framed picture) and 6.6 s (window with many panes and muntins, | |
| hence many competing lines). Per-image numbers are in each example JSON and in `BUILD_REPORT.json`. | |
| If bounded time is required, reduce `n_lines` (enumeration grows like ~N⁴ up to where the `max_eval` | |
| cut kicks in) and/or `max_eval` (expensive checks grow linearly with it): both cut cost, at the | |
| price of coverage on the weaker sides. | |
| --- | |
| ## 7. Geometric transforms and coordinate mapping | |
| Conventions: pixel centre (0,0) = top-left corner of pixel 0,0; continuous coordinates with a **half | |
| pixel** offset between index and centre (`cv2.LINE_AA` and `INTER_AREA` follow this convention). | |
| **Forward** (§4): `p_512 = p_orig * scale + [ox, oy] - 0.5 + 0.5`. | |
| **Inverse** (mapping extracted vertices from the 512 frame back to the original image): | |
| ```python | |
| p_orig = (p_512 + 0.5 - np.array([ox, oy])) / scale - 0.5 | |
| ``` | |
| The half-pixel correction is **not cosmetic**: on 4000×3000 photos the mapping error reaches ~1.6 px, | |
| above the extractor's own signature (refit ±0.5 px). Skipping it makes it impossible to tell whether | |
| a 1.5 px residual is an algorithm error or a rectangle-axis error. | |
| **Rectification**: extracted vertices are **quadrilaterals in perspective**, not axis-aligned | |
| rectangles. To get the metric rectangle, use a homography estimated from the 4 `canonical_order` | |
| vertices → the 4 target corners `(0,0), (w,0), (w,h), (0,h)`, where `w`/`h` are the mean lengths of | |
| the opposite sides (`cv2.getPerspectiveTransform` + `cv2.warpPerspective`). For a frontal | |
| rectification of a window/frame the canonical order (§6, end) is already consistent with this target. | |
| **Estimated side rotation** (to label "horizontal/vertical"): mean of the arguments of the 4 sides, | |
| modulo 90°. | |
| --- | |
| ## 8. Extractor parameter table | |
| `QuadConfig` defaults and the recommended `QuadConfig.hybrid()` set (used by the examples in this | |
| release). `hybrid()` raises the coverage thresholds **because** missing sides are filled in by Canny: | |
| | Parameter | Default | `hybrid()` | Meaning | | |
| |---|---|---|---| | |
| | `edge_thresh` | 0.5 | 0.5 | edge pixel = `edge > threshold` | | |
| | `n_theta` / `rho_step` | 360 / 1.0 px | = | Hough grid (0.5° / 1 px) | | |
| | `n_lines` | 500 | = | lines kept after NMS | | |
| | `min_line_votes` / `min_rel_votes` | 25 / 0.05 | = | lines that are too weak are dropped | | |
| | `min_side_cov` | 0.25 | **0.60** | minimum coverage of the worst side | | |
| | `min_mean_cov` | 0.50 | **0.75** | mean coverage of the perimeter | | |
| | `shape` | `rect` | `rect` | `rect` = angles inside `[50,130]°` | | |
| | `band_px` / `trim` | 3 px / 6% | = | support band and corner trim | | |
| | `min_area_frac` / `max_area_frac` | 1% / 60% | = | usable area | | |
| | `min_side_px` / `max_aspect` | 30 / 6 | = | minimum side, maximum aspect | | |
| | `border_margin_px` | 8 | = | corners tolerated outside the canvas | | |
| | `min_contrast_cov` | 0 (off) | 0 (off) | 0 = polarity does not filter (fragile on paintings) | | |
| | `max_gap_frac` | 0.20 | = | maximum gap: "stitched group" | | |
| | `prefer_outer` / `nested` | True / False | True / **True in the examples with frames** | outer contour vs nested ones | | |
| | `max_quads` | 64 | = | maximum number of outputs | | |
| The `hybrid()` thresholds are **calibrated on a small set of real photographs**: before using them in | |
| production, re-tune them on a set with your own ground truth (see `scripts/evaluate.py` in the source | |
| repository). They are the most delicate point of the whole chain: a `min_side_cov` that is too high | |
| loses backlit sides, one that is too low re-opens the "stitched group" false positives. | |
| Full example (ONNXRuntime + extractor, ~30 lines): | |
| ```python | |
| import cv2, numpy as np, onnxruntime as ort | |
| sess = ort.InferenceSession("model.onnx", providers=["CPUExecutionProvider"]) | |
| inp = sess.get_inputs()[0].name | |
| outs = [o.name for o in sess.get_outputs()] | |
| def pad_square(img, size=512): | |
| scale = size / max(img.shape[:2]) | |
| nw, nh = round(img.shape[1] * scale), round(img.shape[0] * scale) | |
| small = cv2.resize(img, (nw, nh), interpolation=cv2.INTER_AREA) | |
| ox, oy = (size - nw) // 2, (size - nh) // 2 | |
| disp = cv2.copyMakeBorder(small, oy, size-nh-oy, ox, size-nw-ox, cv2.BORDER_REPLICATE) | |
| valid = np.zeros((size, size), bool); valid[oy:oy+nh, ox:ox+nw] = True | |
| return disp, valid, scale, ox, oy | |
| disp, valid, scale, ox, oy = pad_square(cv2.imread("photo.jpg")) | |
| x = np.ascontiguousarray(cv2.cvtColor(disp, cv2.COLOR_BGR2RGB).astype(np.float32) | |
| .transpose(2, 0, 1) / 255.0)[None] | |
| out = dict(zip(outs, sess.run(outs, {inp: x}))) | |
| gray = cv2.cvtColor(disp, cv2.COLOR_BGR2GRAY) | |
| vmask = cv2.erode(valid.astype(np.uint8), np.ones((5, 5), np.uint8)).astype(bool) | |
| edge = hybrid_edge_map(out["edge_map"][0, 0], gray, valid=vmask) # §6 step 1 | |
| quads = extract_quads(edge, QuadConfig.hybrid(nested=True), gray=gray.astype(np.float32)) | |
| for q in quads: | |
| corners = (q["corners"] + 0.5 - np.array([ox, oy])) / scale - 0.5 # §7 inverse | |
| print(np.round(corners, 1).tolist(), round(q["score"], 3), q["edge_cov"]) | |
| ``` | |
| --- | |
| ## 9. Known limitations (what **does not** work) | |
| Read these before integrating: they are measured limits, not hypothetical ones. | |
| 1. **Domain gap.** The model is trained on **synthetic data** (rendered convex polygons over an | |
| image bank background, photometric augmenting). On real photographs the `edge_map` transfers, the | |
| `keypoint_map` does not. **Do not expect the model vertices (output 0) to be correct on real | |
| images**: on real imagery path B uses the edge map only. | |
| 2. **One polygon per frame** in path A: several objects → "star-shaped" polygon. | |
| 3. **Uncalibrated `confidence`** (≈0.49 on every vertex): it is not a correctness probability. | |
| 4. **Fixed 512² resolution**: small objects in a large image are shrunk by the letterbox; below | |
| ~30 px of side the extractor discards them (`min_side_px`). | |
| 5. **`hybrid()` thresholds tuned on a handful of photos**: scene-to-scene variability is the main | |
| source of false negatives (backlit sides, low-frequency edges). | |
| 6. **No perspective model**: in scenes rich in parallel lines (the window example shows it) one | |
| proposal in four comes out warped or on the wrong sash — the `corner_angle_deg` angle test accepts | |
| very oblique quadrilaterals too, so a side running across a neighbouring sash can pass. Available | |
| mitigations: `prefer_outer`, `nms_contain` and, application-side, a mask on the object of interest. | |
| 7. **The reported metrics belong to the synthetic domain**: vertex RMSE 0.81 px (val, epoch 43) and | |
| 0.38 px (test) hold **only** on the synthetic dataset drawn from the same training distribution. | |
| 8. Cropping at the canvas border: handled (`border_margin_px`, samples outside the canvas are not | |
| penalized), but a rectangle with two sides outside the canvas is inferred, not measured. | |
| --- | |
| ## 10. Examples in the repository | |
| `examples/` contains public-domain photographs (source metadata re-read at build time) together with | |
| the pipeline debug files: | |
| - `example_1_picture_frame.*` — framed pictures on a plain wall: the outer frame and the mount-board | |
| edge of each picture are separate rectangles; | |
| - `example_2_window.*` — a framed window with many nested rectangles: panes, muntins and the frame | |
| **are** rectangles, so the legitimate candidates are in the dozens; the release keeps the | |
| best-scoring `max_quads = 12` (a value declared in the example JSON); | |
| - `_debug.png` suffix = mosaic `network edge_map | hybrid map | 512² frame with the extracted sides`; | |
| `.json` suffix = license/attribution, preprocessing parameters, vertices in **original** pixels, | |
| scores and per-side coverage. | |
| The number of extracted rectangles and the scores in the examples **are measured on this artifact** | |
| (see the build-generated appendix); the JSON files also record the thresholds used, so you can | |
| reproduce the chain without having to guess it. | |
| --- | |
| # Appendix A — facts measured on **this** artifact | |
| Generated by `scripts/build_hf_release.py` on 2026-11-02 16:11 | |
| (builder: `python -u scripts/build_hf_release.py`). Sections 1–9 are the **specification**, the tables | |
| below are **measurements**: regenerate this appendix whenever you regenerate the model. | |
| ## A.1 The artifact | |
| | Field | Value | | |
| |---|---| | |
| | File | `model.onnx` | | |
| | Bytes | 58451191 (58.5 MB) | | |
| | SHA-256 | `a351dc1ff555a9e8e6fb2dacffadb2b92f91aa7c7e16fb06eed6aceb949e2e89` | | |
| | Opset | 18 / effective `default:18` | | |
| | Single file (external data inlined) | true (57520896 bytes of external weights recovered) | | |
| | Input | `input_image` ('batch', 3, 512, 512) `tensor(float)` | | |
| | Resolution | fixed 512×512, dynamic batch | | |
| | onnxruntime (build check) | ok=true, active vertices on the reference input=4, edge map in [0.0000, 1.0000] | | |
| | B=1 vs B=2 consistency | 0.000000 px (the declared batch axis is correct) | | |
| | ONNX CPU latency | median 137.9 ms, p95 155.9 ms (4 threads, batch 1, real input) | | |
| | Export parity PyTorch→ONNX | vertices 0.00198 px, `edge_map` 7.75e-06, `keypoint_map` 2.80e-06 | | |
| ## A.2 Actual output binding shapes (verified with onnxruntime) | |
| | Output | Shape | | |
| |---|---| | |
| | `vertices` | `(1, 1, 8, 2)` | | |
| | `vertex_mask` | `(1, 1, 8)` | | |
| | `confidence` | `(1, 1, 8)` | | |
| | `coarse_vertices` | `(1, 1, 8, 2)` | | |
| | `coarse_mask` | `(1, 1, 8)` | | |
| | `edge_map` | `(1, 1, 512, 512)` | | |
| | `keypoint_map` | `(1, 1, 512, 512)` | | |
| ## A.3 Numbers of the published examples | |
| The 3 highest-scoring rectangles of each example (example_1_picture_frame: 2 released, example_2_window: 12 released); the complete per-side coordinates are in each example JSON. | |
| | Example | Rectangle | score | side coverage | area [original px] | refit [px] | | |
| |---|---|---|---|---|---| | |
| | example_1_picture_frame | #1 | 1.006 | 1.00, 1.00, 1.00, 1.00 | 60776 | 0.968 | | |
| | example_1_picture_frame | #2 | 1.003 | 1.00, 1.00, 1.00, 1.00 | 34589 | 1.762 | | |
| | example_2_window | #1 | 1.003 | 1.00, 1.00, 1.00, 1.00 | 33953 | 0.619 | | |
| | example_2_window | #2 | 1.002 | 1.00, 1.00, 1.00, 1.00 | 19530 | 2.879 | | |
| | example_2_window | #3 | 1.0 | 0.98, 1.00, 1.00, 1.00 | 44091 | 1.726 | | |
| Extractor cost measured during the build: example_1_picture_frame: ONNX 169.9 ms + extraction 3638.0 ms, example_2_window: ONNX 148.8 ms + extraction 2534.2 ms. | |
| ## A.4 Ablation (run data, not hypotheses) | |
| | Initialization | best val RMSE [px] | best epoch | | |
| |---|---|---| | |
| | keypoint (the released model) | 0.81 | 43 | | |
| | hybrid (Hough + keypoint) | 39.64 | 20 | | |
| | keypoint, no structural losses | 39.90 | 26 | | |
| | pure differentiable Hough | 71.26 | 45 | | |
| **test** split of the released model (0.384 px, IoU 0.9719, | |
| recall 0.9762, precision 1.0000): | |
| `runs/g3_keypoint/report_test.json`, 50 epochs of train history in the CSV, | |
| 20 epochs of val history in `summary.json` (restored run). | |
| ## A.5 Build environment | |
| Python 3.14.5 · Windows 11 · numpy 2.5.3 · | |
| onnxruntime 1.31.0 · onnx 1.23.2 · torch 2.14.1+cu130 · | |
| cv2 5.0.0 · huggingface_hub 2.3.0 · matplotlib 3.11.2 | |