| --- |
| license: cc-by-nc-sa-4.0 |
| tags: |
| - depth-estimation |
| - nuscenes |
| --- |
| |
| # confhead |
|
|
| Per-pixel confidence heads for a frozen Depth Anything V2 on nuScenes, trained by |
| [confhead](https://github.com/kgqiao/Vision-Project-CSCI-566/tree/main/confhead). |
| A head predicts `p_error`, the probability that the depth of a grid cell is wrong |
| against lidar. Each folder is one training run: |
|
|
| | File | What it holds | |
| |---|---| |
| | `best.pt` | the head of the epoch with the lowest dev loss | |
| | `last.pt` | the head of the last epoch, with the optimizer and the schedule | |
| | `config.json` | every setting, the dev scenes, the class weight, the cache fingerprint, the git commit | |
| | `history.json` | the train and dev loss of each epoch | |
| | `steps.json` | the mean train loss five times per epoch, in runs that have it | |
|
|
| The heads train on labels from [nuScenes](https://www.nuscenes.org/), which is |
| under CC BY-NC-SA 4.0 and the nuScenes terms of use. The same terms apply here. |
|
|
| ## Load a Head |
|
|
| ```sh |
| hf download nadayusc/confhead <run>/best.pt --local-dir . |
| ``` |
|
|
| ```python |
| from confhead import load_head |
| head, inputs, fields = load_head("<run>/best.pt") |
| ``` |
|
|
| ## Training |
|
|
| `confhead/src/confhead/train.py` trains every head with AdamW and a cosine learning |
| rate schedule over all steps, against a binary cross-entropy loss on the cells that |
| carry a lidar label, with the class weight below on the positive class. A cell is |
| positive when the absolute relative error of its depth is above `tau`. The sections |
| below come from the `config.json` and `history.json` of each run. |
|
|
| ## Runs |
|
|
| | Run | Group | Cache | Inputs | Epochs | Best epoch | Best dev loss | |
| |---|---|---|---|---|---|---| |
| | [run1](#run1) | | `nuscenes-trainval` | features, depth, temporal, radar (68 channels) | 8 of 8 | 8 | 0.5562 | |
|
|
| ### run1 |
|
|
| Best epoch 8 of 8, dev loss 0.5562. Training took 3.5 hours. |
|
|
| | Setting | Value | |
| |---|---| |
| | Cache | `nuscenes-trainval`, fingerprint `99d44ff8c71c5355`, grid 132 × 74 | |
| | Depth model | Depth Anything V2 `vitb`, short side 518, tap `output_conv1` | |
| | Inputs | features, depth, temporal, radar (68 channels) | |
| | Labels | lidar-aligned, `tau` 0.1, depth 1 to 80 m | |
| | Radar | radar-aligned, radius 1 grid cells, sweeps: every cached sweep, depth 1 to 80 m | |
| | Head widths | [64, 64, 32] | |
| | Epochs | 8 | |
| | Batch size | 8 | |
| | Learning rate | 0.001 | |
| | Weight decay | 0.0001 | |
| | Train images per epoch | 110,929 of 110,929 | |
| | Dev set | 69 scenes, 3,000 images (`dev_fraction` 0.1, `dev_limit` 3,000) | |
| | Positive rate | 37.6% of labeled cells | |
| | Class weight | 1.658 | |
| | Stats sample | 256 images | |
| | Seed | 0 | |
| | Software | git `e25882c`, torch 2.14.1+cu126, device `cuda` | |
|
|
| | Epoch | Train loss | Dev loss | Learning rate at end | Minutes | |
| |---|---|---|---|---| |
| | 1 | 0.5836 | 0.5818 | 9.62e-04 | 34.7 | |
| | 2 | 0.5680 | 0.5786 | 8.54e-04 | 30.0 | |
| | 3 | 0.5617 | 0.5668 | 6.91e-04 | 30.2 | |
| | 4 | 0.5570 | 0.5607 | 5.00e-04 | 28.5 | |
| | 5 | 0.5531 | 0.5598 | 3.09e-04 | 23.9 | |
| | 6 | 0.5499 | 0.5573 | 1.46e-04 | 23.7 | |
| | 7 | 0.5476 | 0.5567 | 3.81e-05 | 20.0 | |
| | 8 (best) | 0.5463 | 0.5562 | 0.00e+00 | 18.9 | |
|
|
