Rewrite the model card
Browse files
README.md
CHANGED
|
@@ -1,3 +1,83 @@
|
|
| 1 |
---
|
| 2 |
-
license:
|
|
|
|
|
|
|
|
|
|
| 3 |
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
---
|
| 2 |
+
license: cc-by-nc-sa-4.0
|
| 3 |
+
tags:
|
| 4 |
+
- depth-estimation
|
| 5 |
+
- nuscenes
|
| 6 |
---
|
| 7 |
+
|
| 8 |
+
# confhead
|
| 9 |
+
|
| 10 |
+
Per-pixel confidence heads for a frozen Depth Anything V2 on nuScenes, trained by
|
| 11 |
+
[confhead](https://github.com/kgqiao/Vision-Project-CSCI-566/tree/main/confhead).
|
| 12 |
+
A head predicts `p_error`, the probability that the depth of a grid cell is wrong
|
| 13 |
+
against lidar. Each folder is one training run:
|
| 14 |
+
|
| 15 |
+
| File | What it holds |
|
| 16 |
+
|---|---|
|
| 17 |
+
| `best.pt` | the head of the epoch with the lowest dev loss |
|
| 18 |
+
| `last.pt` | the head of the last epoch, with the optimizer and the schedule |
|
| 19 |
+
| `config.json` | every setting, the dev scenes, the class weight, the cache fingerprint, the git commit |
|
| 20 |
+
| `history.json` | the train and dev loss of each epoch |
|
| 21 |
+
|
| 22 |
+
The heads train on labels from [nuScenes](https://www.nuscenes.org/), which is
|
| 23 |
+
under CC BY-NC-SA 4.0 and the nuScenes terms of use. The same terms apply here.
|
| 24 |
+
|
| 25 |
+
## Load a Head
|
| 26 |
+
|
| 27 |
+
```sh
|
| 28 |
+
hf download nadayusc/confhead <run>/best.pt --local-dir .
|
| 29 |
+
```
|
| 30 |
+
|
| 31 |
+
```python
|
| 32 |
+
from confhead import load_head
|
| 33 |
+
head, inputs, fields = load_head("<run>/best.pt")
|
| 34 |
+
```
|
| 35 |
+
|
| 36 |
+
## Training
|
| 37 |
+
|
| 38 |
+
`confhead/src/confhead/train.py` trains every head with AdamW and a cosine learning
|
| 39 |
+
rate schedule over all steps, against a binary cross-entropy loss on the cells that
|
| 40 |
+
carry a lidar label, with the class weight below on the positive class. A cell is
|
| 41 |
+
positive when the absolute relative error of its depth is above `tau`. The sections
|
| 42 |
+
below come from the `config.json` and `history.json` of each run.
|
| 43 |
+
|
| 44 |
+
## Runs
|
| 45 |
+
|
| 46 |
+
| Run | Cache | Inputs | Epochs | Best epoch | Best dev loss |
|
| 47 |
+
|---|---|---|---|---|---|
|
| 48 |
+
| [run1](#run1) | `nuscenes-trainval` | features, depth, temporal, radar (68 channels) | 8 of 8 | 8 | 0.5562 |
|
| 49 |
+
|
| 50 |
+
### run1
|
| 51 |
+
|
| 52 |
+
Best epoch 8 of 8, dev loss 0.5562. Training took 3.5 hours.
|
| 53 |
+
|
| 54 |
+
| Setting | Value |
|
| 55 |
+
|---|---|
|
| 56 |
+
| Cache | `nuscenes-trainval`, fingerprint `99d44ff8c71c5355`, grid 132 × 74 |
|
| 57 |
+
| Depth model | Depth Anything V2 `vitb`, short side 518, tap `output_conv1` |
|
| 58 |
+
| Inputs | features, depth, temporal, radar (68 channels) |
|
| 59 |
+
| Labels | lidar-aligned, `tau` 0.1, depth 1 to 80 m |
|
| 60 |
+
| Radar | radar-aligned, radius 1 grid cells, sweeps: every cached sweep, depth 1 to 80 m |
|
| 61 |
+
| Head widths | [64, 64, 32] |
|
| 62 |
+
| Epochs | 8 |
|
| 63 |
+
| Batch size | 8 |
|
| 64 |
+
| Learning rate | 0.001 |
|
| 65 |
+
| Weight decay | 0.0001 |
|
| 66 |
+
| Train images per epoch | 110,929 of 110,929 |
|
| 67 |
+
| Dev set | 69 scenes, 3,000 images (`dev_fraction` 0.1, `dev_limit` 3,000) |
|
| 68 |
+
| Positive rate | 37.6% of labeled cells |
|
| 69 |
+
| Class weight | 1.658 |
|
| 70 |
+
| Stats sample | 256 images |
|
| 71 |
+
| Seed | 0 |
|
| 72 |
+
| Software | git `e25882c`, torch 2.14.1+cu126, device `cuda` |
|
| 73 |
+
|
| 74 |
+
| Epoch | Train loss | Dev loss | Learning rate at end | Minutes |
|
| 75 |
+
|---|---|---|---|---|
|
| 76 |
+
| 1 | 0.5836 | 0.5818 | 9.62e-04 | 34.7 |
|
| 77 |
+
| 2 | 0.5680 | 0.5786 | 8.54e-04 | 30.0 |
|
| 78 |
+
| 3 | 0.5617 | 0.5668 | 6.91e-04 | 30.2 |
|
| 79 |
+
| 4 | 0.5570 | 0.5607 | 5.00e-04 | 28.5 |
|
| 80 |
+
| 5 | 0.5531 | 0.5598 | 3.09e-04 | 23.9 |
|
| 81 |
+
| 6 | 0.5499 | 0.5573 | 1.46e-04 | 23.7 |
|
| 82 |
+
| 7 | 0.5476 | 0.5567 | 3.81e-05 | 20.0 |
|
| 83 |
+
| 8 (best) | 0.5463 | 0.5562 | 0.00e+00 | 18.9 |
|