dlrv-pointnet-exp5

PointNet++ 6DoF object pose estimator for the fifteen LineMOD objects, trained on masked RGB-D point clouds. This is the exp5 checkpoint: the strongest configuration under the surrogate training objective, and the one used for the head-to-head comparison against the MVCNN track in the report.

Produced for a Deep Learning for Robot Vision project at Hochschule Bonn-Rhein-Sieg.

What it does

Given a masked point cloud of a single object, it regresses the object's full 6DoF pose:

• Rotation in the continuous 6D representation of Zhou et al., which avoids the discontinuities of quaternions and Euler angles.
• Translation as a residual from the observed point cloud centroid, not as an absolute position. This matters: PointNet++ set abstraction re-centres every local neighbourhood on its own centroid, so absolute position cannot survive to the global descriptor. Predicting the residual is what makes the translation learnable at all.

Architecture

Backbone	PointNetPlusPlus, SA1 to SA3, 1024 input points
Head	PoseEstimationHead, 512-d feature, separate rotation and translation branches
Parameters	0.86 M
Input	1024 points, metres, centroid-centred

Results

Trained on the synthetic PBR split only, with no domain adaptation.

Split	ADD(-S)@0.05d	ADD(-S)@0.1d	ADD(-S)@0.2d	Mean	Median
Synthetic validation		57.36%
Real BOP test, 3,000 images	8.17%	36.23%	60.20%	43.82 mm	26.95 mm

Accuracy is strongly correlated with object diameter (Pearson r = +0.92 on real data): the lamp reaches 61.0% and the holepuncher 5.0%. The ADD threshold is 0.1 x diameter while this model's precision is absolute rather than relative, which is the mechanism behind that spread.

Training

--experiment exp5_combined --epochs 30 --batch_size 32
--lr 0.001 --backbone_lr 0.0005 --rot_weight 1.0 --trans_weight 0.05

The objective is a surrogate: a geodesic rotation loss plus an L1 translation loss, weighted by trans_weight. Raising that weight fivefold was the single strongest lever found in the ablation, worth +8.3 points, and it composes with a fivefold higher backbone learning rate for +17.1 over the baseline.

For the variant trained on diameter-normalised ADD instead, see jan024/dlrv-ppointnet-exp6. It is more accurate on synthetic data and less accurate on real data.

Usage

The checkpoint is a dict with keys epoch, val_add, backbone and pose_head. Load it with the model definitions from the project repository:

import torch
from pointnet.model import PointNetPlusPlus
from pose.head import PoseEstimationHead

ckpt = torch.load('exp5_combined_best.pth', map_location='cuda')

backbone = PointNetPlusPlus(n_classes=15, n_points=1024).cuda()
backbone.load_state_dict(ckpt['backbone'])

pose_head = PoseEstimationHead(feature_dim=512).cuda()
pose_head.load_state_dict(ckpt['pose_head'])

To reproduce the 36.23% figure end to end:

python -u src/pose/eval_test_set.py \
    --checkpoint exp5_combined_best.pth \
    --data_root /path/to/lm \
    --output_json results.json

Roughly two minutes on an RTX A5000. LineMOD in BOP format is required and is not distributed here; get it from https://bop.felk.cvut.cz/datasets/.

Limitations

• Trained on synthetic data only. Accuracy falls 21.1 points from synthetic validation to real images, because real depth observations are sparser and more fragmented than rendered ones.
• Small objects fail badly on real data. The ape scores 14.5%, and its failures are gross rather than marginal.
• Requires a ground-truth object mask. No detection or segmentation stage is included.
• Evaluation subsamples the point cloud and the CAD vertices without a seed, so per-object figures move by a few points between runs.