Score checkpoints end to end and keep the results
Aryan Sethi+1: Claude Opus 5 (1M context)
a1e201c
accuracy.py runs a model over the val split, pairs every detection with its
ground-truth distance, and writes both the per-detection rows and the aggregates
to results/depth_accuracy/ keyed by tag -- so a later checkpoint can be compared
against this one without re-running it. Collection happens at a low confidence
and every threshold in the report is applied afterwards, so one pass covers all
operating points.
Matching is greedy by confidence and each ground-truth object can be claimed
once. Without that, two overlapping detections of the same cone both counted and
"objects found" came out at 102%, which is how the bug was noticed. A detection
counts only if it overlaps at IoU >= 0.5 and carries the right class.
On the fine-tuned checkpoint at 759 val objects: keeping detections above 0.50
gives 73% real, 43% of objects found, and a 2.1 m median distance error. The
unexpected part is that confidence predicts distance accuracy -- 1.1 m median
error above 0.75 against 5.6 m between 0.…
