| --- |
| license: mit |
| tags: [robotics, surface-roughness, grinding, contrastive-learning, RNC, segmentation] |
| --- |
| |
| # grind — visual_grind_grading checkpoints |
|
|
| Trained artifacts for the `visual_grind_grading` approach of the |
| [grind](https://github.com/BabaYaga840/grind) repo. Gitignored there; hydrate a |
| fresh clone with `./pull_checkpoints.sh`. |
|
|
| Two independent models, answering two different questions about a weld bead being |
| ground off a metal plate: |
|
|
| | model | question | file | |
| |---|---|---| |
| | ROI detector | **where** must the robot grind? | `outputs/roi_cnn/roi_cnn.pt` | |
| | RNC grader | **how rough** is this surface? | `outputs/rnc_grader/rnc_sandpaper_grader.pt` | |
|
|
| ## ROI detector (seam masking) |
|
|
| `outputs/roi_cnn/roi_cnn.pt` — masks which blocks of the plate are weld seam that |
| needs grinding, vs bare plate. A plain `state_dict` for `TinyNet` |
| (`scripts/p7_cnn_train.py`): 3 × (conv3×3 → BN → ReLU) at 16/32/64 channels with 2 |
| max-pools, global-avg-pool, `Linear(64→1)`, sigmoid. **23,873 parameters.** |
|
|
|  |
|
|
| Green ∝ P(seam), red = annotated ROI polygon, blue = plate. Neutral-lit passes |
| (top) give a tight dense band; the blue-lit passes are sparser — that lighting |
| change is the model's hardest case. |
|
|
| Operating config — the detector is **fully convolutional over blocks, not over the |
| image**: slide a 24 px context window over the plate and paint the 12 px block at |
| its centre. |
|
|
| | | | |
| |---|---| |
| | input | 32×32 BGR, `/255`, = the 24 px context window resized | |
| | paint block | 12 px | |
| | threshold | **0.80** (best F1 0.703, P 0.68 / R 0.72); 0.5 → recall 0.91 | |
| | plate bbox | `[500, 412, 713, 519]` | |
|
|
| ```python |
| import torch, torch.nn as nn |
| |
| class TinyNet(nn.Module): |
| def __init__(s): |
| super().__init__() |
| s.f = nn.Sequential( |
| nn.Conv2d(3, 16, 3, padding=1), nn.BatchNorm2d(16), nn.ReLU(), nn.MaxPool2d(2), |
| nn.Conv2d(16, 32, 3, padding=1), nn.BatchNorm2d(32), nn.ReLU(), nn.MaxPool2d(2), |
| nn.Conv2d(32, 64, 3, padding=1), nn.BatchNorm2d(64), nn.ReLU(), |
| nn.AdaptiveAvgPool2d(1)) |
| s.head = nn.Linear(64, 1) |
| def forward(s, x): |
| return s.head(s.f(x).flatten(1)).squeeze(1) |
| |
| net = TinyNet() |
| net.load_state_dict(torch.load("roi_cnn.pt", map_location="cpu")) |
| net.eval() |
| prob = torch.sigmoid(net(x)) # x: (N,3,32,32) in [0,1], BGR |
| ``` |
|
|
| Leave-one-pass-out (train on 11 passes, test the held-out one): pooled F1 0.632 |
| @0.5, neutral passes F1 0.701, blue-lit passes F1 0.580. Cross-lighting robustness |
| comes from plate-restricted sampling, context windows, and colour/brightness |
| jitter augmentation. Full write-up: |
| [`ROI_DETECTOR.md`](https://github.com/BabaYaga840/grind/blob/main/approaches/visual_grind_grading/ROI_DETECTOR.md). |
|
|
| ## Grading model (RNC) |
| `outputs/rnc_grader/rnc_sandpaper_grader.pt` — the RNC sandpaper Ra grader. |
| `torch.load` gives a dict: `Enc` encoder `state_dict` (3-conv CNN, dim 32) + |
| per-grit `anchors` + `grit_classes` + `Ra_by_grit`. Inference: |
|
|
| ```python |
| import torch, torch.nn.functional as F, numpy as np |
| c = torch.load("rnc_sandpaper_grader.pt", weights_only=False) |
| # rebuild Enc (see scripts/shared_grading.py: class Enc), load c["state_dict"] |
| def illum_norm(x): m=x.mean((2,3),keepdim=True); s=x.std((2,3),keepdim=True)+1e-4; return (x-m)/s |
| z = F.normalize(net(illum_norm(x)), dim=1).cpu().numpy() # x: (N,3,64,64) [0,1] |
| w = np.exp(z @ c["anchors"].T / c["tau"]); w /= w.sum(1, keepdims=True) |
| grade = w @ c["grit_classes"].astype("float32") # soft ordinal, 0..6 |
| ``` |
| Validated leave-angle-out rho ~0.93 vs true grit (real, texture-grounded). |
|
|
| ## Other artifacts |
| - `cache/*.npz` — p27–p38 embedding caches (multisession / SupCon / RNC |
| experiments) for reproducing analyses without recompute. |
|
|
| Not hosted: `outputs/p7_samples.npz`, the ROI detector's training patches. Rebuild |
| with `scripts/p7_build_samples.py` (needs the raw rosbags) if you want to retrain. |
|
|
| ## Scope note |
|
|
| The ROI detector works. The broader hypothesis this approach set out to test — |
| that image appearance encodes *cumulative* grinding, so pass index is recoverable |
| from a photo of the plate — came out **negative** on this dataset; the strong |
| correlations were a lighting/equipment confound. The sandpaper RNC grader above is |
| the texture-grounded replacement. See the repo README for that story. |
|
|
