Drop barrel, add held-out stop-sign distance, add the GPU runbook
Aryan Sethi+1: Claude Opus 5 (1M context)
5d449ff
Barrel is out of the pipeline: schema carries three classes, the Argoverse 2
extractor no longer maps CONSTRUCTION_BARREL, and depth routing follows. AV2
barrels are still read to locate the road surface for the per-log camera-height
fit -- they rest on the road and there are many of them -- but they never become
labels.
Distance is now verified on data the model never saw: 20 further Argoverse 2
logs, 2,431 frames, 6,332 objects, scored with the existing checkpoint rather
than retraining. The pinhole estimator comes out far better there than on
nuScenes -- 0.21 m median error under 10 m, 0.79 m overall at 5% off, 98% within
25% -- because AV2 calibration is derived per log and its camera has higher
angular resolution, while the nuScenes figure was dominated by one scene with a
road gradient.
Stop-sign distance is measured for the first time: 21 detections, 9.22 m median,
18% off. Thin, and reported as such. The count is a data limit rather than a
threshold one -- AV2 stop signs sit …
Fix two cross-dataset convention bugs found by scoring against ground truth
Aryan Sethi+1: Claude Opus 5 (1M context)
b9ff8f9
Argoverse 2 cameras are not pinhole. They ship k1,k2,k3 of roughly
-0.24/-0.22/+0.34 and the images are not rectified, so projecting through K
alone put boxes tens of pixels off the objects they label. Calibration now
carries the coefficients and owns both directions, project() and ray();
nuScenes passes zeros, for which the model is the identity, so one code path
serves both.
The datasets also disagree about the ego frame. nuScenes puts its origin on the
ground, so camera height is the sensor z translation. AV2 puts its origin at the
rear axle centre, about 0.26 m up, and taking z translation as camera height
made every AV2 ground-plane distance read ~18% short at every range. Rather
than hardcode a vehicle constant, measure it: cones and barrels rest on the
road, so the median underside of their near-field cuboids is the road surface.
Per log, so it absorbs road grade too. Deliberately not applied to nuScenes,
where it would make a calibration that already checks out at 0.986 worse.…
Verify the pipeline on nuScenes v1.0-mini
Aryan Sethi+1: Claude Opus 5 (1M context)
3309cfc
404 CAM_FRONT keyframes, 1119 objects, 799 after filters. cam_height_m reads
1.511 m and the horizon correlation is -0.62, so the camera-frame transform is
sound.
Pin numpy<2 alongside nuscenes-devkit: the devkit caps matplotlib below 3.6,
which predates numpy 2 and dies on import against it.
Report per-class val instance counts in merge, not just cones. A scene-level
split over mini's ten scenes can land zero instances of a class in val, and a
silent zero means phase 2 has nothing to score that class against.
Two findings recorded in the README. The derived class priors contradict the
plan's looked-up table -- nuScenes cones are 0.78 m and barriers 1.06 m, so
hardcoding 1.07 m for a cone puts it ~37% too far. And a dry run of the
ground-plane estimator against gt_distance_m gives 0.42 m MAE under 10 m rising
to 13.4 m at 30-45 m, with the outliers coming from box bottoms within a few
pixels of the horizon row rather than from bad labels.
Co-Authored-By: Claude Opus 5 (1M context) …
Phase 1: unified data pipeline, QA and training
Aryan Sethi+1: Claude Opus 5 (1M context)
9274a8d
Extract cones, barriers, barrels and stop signs from nuScenes, Argoverse 2,
COCO and BDD100K into one manifest carrying camera intrinsics and ground-truth
distance, then train YOLO11n and YOLO11s on the derived YOLO export.
The manifest is the artifact, not the .txt files: .txt cannot hold intrinsics
or gt distance, and phase 2's geometric distance estimator needs both. 2D boxes
are projected from the 3D cuboids rather than taken from a pre-made 2D
annotation file, so boxes and distances come from one convention.
merge.py asserts the cross-dataset geometry rather than leaving it to be
eyeballed -- distance positive, distance and box-bottom row negatively
correlated within a camera, median cone distance plausible. A coordinate
convention mismatch otherwise produces a manifest that trains fine and makes
the depth numbers nonsense a day later.
Extraction and export run locally; only training moves to Colab, so the dataset
is published to the Hugging Face Hub as tar shards with the mani…
