File size: 3,102 Bytes
b929007 8ae2c7a b929007 8ae2c7a 0388202 8ae2c7a 0388202 8ae2c7a | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 | ---
license: cc-by-nc-sa-4.0
tags:
- depth-estimation
- nuscenes
---
# confhead
Per-pixel confidence heads for a frozen Depth Anything V2 on nuScenes, trained by
[confhead](https://github.com/kgqiao/Vision-Project-CSCI-566/tree/main/confhead).
A head predicts `p_error`, the probability that the depth of a grid cell is wrong
against lidar. Each folder is one training run:
| File | What it holds |
|---|---|
| `best.pt` | the head of the epoch with the lowest dev loss |
| `last.pt` | the head of the last epoch, with the optimizer and the schedule |
| `config.json` | every setting, the dev scenes, the class weight, the cache fingerprint, the git commit |
| `history.json` | the train and dev loss of each epoch |
| `steps.json` | the mean train loss five times per epoch, in runs that have it |
The heads train on labels from [nuScenes](https://www.nuscenes.org/), which is
under CC BY-NC-SA 4.0 and the nuScenes terms of use. The same terms apply here.
## Load a Head
```sh
hf download nadayusc/confhead <run>/best.pt --local-dir .
```
```python
from confhead import load_head
head, inputs, fields = load_head("<run>/best.pt")
```
## Training
`confhead/src/confhead/train.py` trains every head with AdamW and a cosine learning
rate schedule over all steps, against a binary cross-entropy loss on the cells that
carry a lidar label, with the class weight below on the positive class. A cell is
positive when the absolute relative error of its depth is above `tau`. The sections
below come from the `config.json` and `history.json` of each run.
## Runs
| Run | Group | Cache | Inputs | Epochs | Best epoch | Best dev loss |
|---|---|---|---|---|---|---|
| [run1](#run1) | | `nuscenes-trainval` | features, depth, temporal, radar (68 channels) | 8 of 8 | 8 | 0.5562 |
### run1
Best epoch 8 of 8, dev loss 0.5562. Training took 3.5 hours.
| Setting | Value |
|---|---|
| Cache | `nuscenes-trainval`, fingerprint `99d44ff8c71c5355`, grid 132 × 74 |
| Depth model | Depth Anything V2 `vitb`, short side 518, tap `output_conv1` |
| Inputs | features, depth, temporal, radar (68 channels) |
| Labels | lidar-aligned, `tau` 0.1, depth 1 to 80 m |
| Radar | radar-aligned, radius 1 grid cells, sweeps: every cached sweep, depth 1 to 80 m |
| Head widths | [64, 64, 32] |
| Epochs | 8 |
| Batch size | 8 |
| Learning rate | 0.001 |
| Weight decay | 0.0001 |
| Train images per epoch | 110,929 of 110,929 |
| Dev set | 69 scenes, 3,000 images (`dev_fraction` 0.1, `dev_limit` 3,000) |
| Positive rate | 37.6% of labeled cells |
| Class weight | 1.658 |
| Stats sample | 256 images |
| Seed | 0 |
| Software | git `e25882c`, torch 2.14.1+cu126, device `cuda` |
| Epoch | Train loss | Dev loss | Learning rate at end | Minutes |
|---|---|---|---|---|
| 1 | 0.5836 | 0.5818 | 9.62e-04 | 34.7 |
| 2 | 0.5680 | 0.5786 | 8.54e-04 | 30.0 |
| 3 | 0.5617 | 0.5668 | 6.91e-04 | 30.2 |
| 4 | 0.5570 | 0.5607 | 5.00e-04 | 28.5 |
| 5 | 0.5531 | 0.5598 | 3.09e-04 | 23.9 |
| 6 | 0.5499 | 0.5573 | 1.46e-04 | 23.7 |
| 7 | 0.5476 | 0.5567 | 3.81e-05 | 20.0 |
| 8 (best) | 0.5463 | 0.5562 | 0.00e+00 | 18.9 |
|