| --- |
| library_name: pytorch |
| tags: |
| - pose-estimation |
| - 6dof |
| - pointnet |
| - point-cloud |
| - linemod |
| - robotics |
| datasets: |
| - linemod |
| --- |
| |
| # dlrv-ppointnet-exp6 |
|
|
| PointNet++ 6DoF object pose estimator for the fifteen LineMOD objects, |
| trained on masked RGB-D point clouds. This is the **exp6** checkpoint, the |
| variant trained on a **diameter-normalised ADD objective**. It is the most |
| accurate configuration on synthetic validation and the most evenly balanced |
| across object classes. |
|
|
| Produced for a Deep Learning for Robot Vision project at Hochschule |
| Bonn-Rhein-Sieg. |
|
|
| ## The idea behind it |
|
|
| The ADD benchmark accepts a pose when the average model point distance falls |
| below 0.1 x the object's diameter. The threshold therefore scales with the |
| object, but a loss computed on absolute ADD distance does not. Training on |
| raw distance consequently favours physically large objects, whose thresholds |
| are the most forgiving: under that objective the lamp reaches 92.5% and the |
| ape 7.0%. |
|
|
| Dividing the loss by the object diameter aligns the objective's scale with |
| the metric's. Across the twelve asymmetric objects this leaves mean accuracy |
| essentially unchanged while cutting per-class standard deviation from |
| 30.4 to 6.3 percentage points, a factor of 4.8, and lifting the weakest class |
| from 7.0% to 45.2%. |
|
|
| The mechanism is confirmed by a counterintuitive observation: this model has |
| a **higher** mean absolute ADD error than the unnormalised one (69 mm against |
| 34.6 mm) while being more accurate. That is only possible if error moved from |
| objects with tight thresholds to objects with loose ones. |
|
|
| ## Architecture |
|
|
| | | | |
| |---|---| |
| | Backbone | `PointNetPlusPlus`, SA1 to SA3, 1024 input points | |
| | Head | `PoseEstimationHead`, 512-d feature, separate rotation and translation branches | |
| | Parameters | 0.86 M | |
| | Input | 1024 points, metres, centroid-centred | |
|
|
| Rotation is regressed in the continuous 6D representation of Zhou et al.; |
| translation as a residual from the observed point cloud centroid. |
|
|
| ## Results |
|
|
| Trained on the synthetic PBR split only, with no domain adaptation. |
|
|
| | Split | ADD(-S)@0.05d | ADD(-S)@0.1d | ADD(-S)@0.2d | Mean | Median | |
| |---|---|---|---|---|---| |
| | Synthetic validation | | **58.41%** | | | | |
| | Real BOP test, 3,000 images | 6.37% | 33.33% | 57.33% | 45.99 mm | 30.13 mm | |
|
|
| ## Read this before choosing between exp6 and exp5 |
|
|
| Normalisation wins on synthetic data by 1.05 points and **loses on real data |
| by 2.90 points**. Directing capacity toward small objects helps in |
| distribution, but small objects are exactly the ones whose real depth |
| observations are sparsest, so the gain does not survive the domain shift. |
|
|
| Reducing per-class variance in distribution is not the same thing as |
| robustness. If you want the better model on real images, use |
| [jan024/dlrv-pointnet-exp5](https://huggingface.co/jan024/dlrv-pointnet-exp5) |
| at 36.23%. Use this one if you care about balanced per-class behaviour on |
| data that matches the training distribution. |
|
|
| ## Training |
|
|
| ``` |
| --experiment exp6_normadd --loss_type normalized_add --epochs 30 |
| --batch_size 32 --lr 0.001 --backbone_lr 0.0005 |
| ``` |
|
|
| The backbone learning rate matches exp5 so that the objective is the only |
| variable between the two. |
|
|
| ## Usage |
|
|
| The checkpoint is a dict with keys `epoch`, `val_add`, `backbone` and |
| `pose_head`. Load it with the model definitions from the project repository: |
|
|
| ```python |
| import torch |
| from pointnet.model import PointNetPlusPlus |
| from pose.head import PoseEstimationHead |
| |
| ckpt = torch.load('exp6_normadd_best.pth', map_location='cuda') |
| |
| backbone = PointNetPlusPlus(n_classes=15, n_points=1024).cuda() |
| backbone.load_state_dict(ckpt['backbone']) |
| |
| pose_head = PoseEstimationHead(feature_dim=512).cuda() |
| pose_head.load_state_dict(ckpt['pose_head']) |
| ``` |
|
|
| To reproduce the 33.33% figure end to end: |
|
|
| ```bash |
| python -u src/pose/eval_test_set.py \ |
| --checkpoint exp6_normadd_best.pth \ |
| --data_root /path/to/lm \ |
| --output_json results.json |
| ``` |
|
|
| Roughly two minutes on an RTX A5000. LineMOD in BOP format is required and is |
| not distributed here; get it from https://bop.felk.cvut.cz/datasets/. |
|
|
| ## Limitations |
|
|
| - Trained on synthetic data only, and loses more to the domain shift than the |
| surrogate-trained exp5 does. |
| - Requires a ground-truth object mask. No detection or segmentation stage is |
| included. |
| - Evaluation subsamples the point cloud and the CAD vertices without a seed, |
| so per-object figures move by a few points between runs. |
|
|
