tkclam commited on
Commit
8426580
·
verified ·
1 Parent(s): c614e29

List the round 14 octacam detector as the current one

Browse files
Files changed (1) hide show
  1. README.md +8 -7
README.md CHANGED
@@ -17,11 +17,12 @@ tags:
17
 
18
  # deeperfly detectors
19
 
20
- Pretrained 2D keypoint detectors for [deeperfly](https://github.com/NeLy-EPFL/deeperfly), markerless 3D pose estimation of *Drosophila* from multi-camera rigs. They are the detectors the example run configs in the deeperfly repository name, trained in the [Ramdya lab](https://www.epfl.ch/labs/ramdya-lab/) at EPFL on hand labels from two rigs.
21
 
22
  | Folder | Rig | What it is | Backbone | Trained on |
23
  | --- | --- | --- | --- | --- |
24
- | `260927_mvt_dinov3b_384_octacam_fly38` | octacam: eight cameras around a tethered fly on a ball | multiview transformer, all 38 `fly38` keypoints in every view, 384x768 window per view | DINOv3 ViT-B/16 | 56 recordings, 4,488 labeled items |
 
25
  | `260927_mvt_dinov3b_512_hexaview_fly38` | hexaview: two cameras, each seeing a freely walking fly directly and through two prisms (six views) | multiview transformer, 38 keypoints, 512x512 window centered on the fly in each frame | DINOv3 ViT-B/16 | 16 recordings, 3,018 labeled items |
26
  | `260920_fast_efficientvit_b1_352_hexaview_locator` | hexaview | full-frame locator that places the hexaview detector's windows | EfficientViT-B1 | 9 recordings, 824 labeled items |
27
 
@@ -35,11 +36,11 @@ From the root of a deeperfly clone, where the example configs expect them:
35
  uv run hf download tkclam/deeperfly-models --local-dir models
36
  ```
37
 
38
- One model only:
39
 
40
  ```bash
41
  uv run hf download tkclam/deeperfly-models --local-dir models \
42
- --include "260927_mvt_dinov3b_384_octacam_fly38/*"
43
  ```
44
 
45
  Check a download with `sha256sum -c SHA256SUMS` inside its folder.
@@ -50,18 +51,18 @@ Name the checkpoint in a run config's `[pose2d]` table:
50
 
51
  ```toml
52
  [pose2d]
53
- weights = ["models/260927_mvt_dinov3b_384_octacam_fly38/mvt_dinov3b_384_octacam_fly38.pth"]
54
  ```
55
 
56
  Paths resolve against the config file. See the [deeperfly quickstart](https://nely-epfl.github.io/deeperfly/stable/quickstart/) for a full run on the bundled example clips.
57
 
58
  ## Scope and limitations
59
 
60
- The detectors were trained on recordings from two rigs in one lab, with every label used for training (no held-out set). On the bundled example clips, the octacam detector reprojects at 1.49 px median after bundle adjustment and the hexaview detector at 3.15 px median through the recording's calibration; that is agreement across views, not error against labels. They are not expected to transfer to other rigs, magnifications, or lighting without retraining. `deeperfly train` fits a detector on your own labels, and these checkpoints can serve as a starting point.
61
 
62
  ## License
63
 
64
- - The two DINOv3-based detectors are derivatives of Meta's DINOv3 weights and are distributed under the [DINOv3 License](LICENSE.md), including its use restrictions.
65
  - The EfficientViT-based locator is distributed under the [Apache License 2.0](LICENSE-apache-2.0), the license of its ImageNet-pretrained backbone.
66
 
67
  ## Citation
 
17
 
18
  # deeperfly detectors
19
 
20
+ Pretrained 2D keypoint detectors for [deeperfly](https://github.com/NeLy-EPFL/deeperfly), markerless 3D pose estimation of *Drosophila* from multi-camera rigs. They include the detectors the example run configs in the deeperfly repository name, all trained in the [Ramdya lab](https://www.epfl.ch/labs/ramdya-lab/) at EPFL on hand labels from two rigs.
21
 
22
  | Folder | Rig | What it is | Backbone | Trained on |
23
  | --- | --- | --- | --- | --- |
24
+ | `261008_mvt_dinov3b_384_octacam_fly38` | octacam: eight cameras around a tethered fly on a ball | **the current octacam detector** (round 14): multiview transformer, all 38 `fly38` keypoints in every view, 384x768 window per view | DINOv3 ViT-B/16 | 62 recordings, 4,703 labeled items |
25
+ | `260927_mvt_dinov3b_384_octacam_fly38` | octacam | the previous octacam detector (round 11), same architecture; the deeperfly repository's example config names it | DINOv3 ViT-B/16 | 56 recordings, 4,488 labeled items |
26
  | `260927_mvt_dinov3b_512_hexaview_fly38` | hexaview: two cameras, each seeing a freely walking fly directly and through two prisms (six views) | multiview transformer, 38 keypoints, 512x512 window centered on the fly in each frame | DINOv3 ViT-B/16 | 16 recordings, 3,018 labeled items |
27
  | `260920_fast_efficientvit_b1_352_hexaview_locator` | hexaview | full-frame locator that places the hexaview detector's windows | EfficientViT-B1 | 9 recordings, 824 labeled items |
28
 
 
36
  uv run hf download tkclam/deeperfly-models --local-dir models
37
  ```
38
 
39
+ One model only, such as the current octacam detector:
40
 
41
  ```bash
42
  uv run hf download tkclam/deeperfly-models --local-dir models \
43
+ --include "261008_mvt_dinov3b_384_octacam_fly38/*"
44
  ```
45
 
46
  Check a download with `sha256sum -c SHA256SUMS` inside its folder.
 
51
 
52
  ```toml
53
  [pose2d]
54
+ weights = ["models/261008_mvt_dinov3b_384_octacam_fly38/mvt_dinov3b_384_octacam_fly38.pth"]
55
  ```
56
 
57
  Paths resolve against the config file. See the [deeperfly quickstart](https://nely-epfl.github.io/deeperfly/stable/quickstart/) for a full run on the bundled example clips.
58
 
59
  ## Scope and limitations
60
 
61
+ The detectors were trained on recordings from two rigs in one lab, with every label used for training (no held-out set). The current octacam detector's twin, trained with four recordings held out, scores 3.25 px median and 4.49 px mean on their 9,269 labeled cells (93.7% within 10 px). On the bundled example clips, the octacam detector reprojects at 1.46 px median after bundle adjustment and the hexaview detector at 3.15 px median through the recording's calibration; that is agreement across views, not error against labels. They are not expected to transfer to other rigs, magnifications, or lighting without retraining. `deeperfly train` fits a detector on your own labels, and these checkpoints can serve as a starting point.
62
 
63
  ## License
64
 
65
+ - The DINOv3-based detectors are derivatives of Meta's DINOv3 weights and are distributed under the [DINOv3 License](LICENSE.md), including its use restrictions.
66
  - The EfficientViT-based locator is distributed under the [Apache License 2.0](LICENSE-apache-2.0), the license of its ImageNet-pretrained backbone.
67
 
68
  ## Citation