Add the two-fly hexaview detector (round x4) and its locator
Browse files- .gitattributes +1 -0
- 261007_fast_efficientvit_b1_352_hexaview_pair_locator/README.md +18 -0
- 261007_fast_efficientvit_b1_352_hexaview_pair_locator/SHA256SUMS +3 -0
- 261007_fast_efficientvit_b1_352_hexaview_pair_locator/config.toml +77 -0
- 261007_fast_efficientvit_b1_352_hexaview_pair_locator/fast_efficientvit_b1_352_hexaview_pair_locator.pth +3 -0
- 261007_fast_efficientvit_b1_352_hexaview_pair_locator/report.json +213 -0
- 261007_mvt_dinov3b_512_hexaview_pair_fly38/README.md +43 -0
- 261007_mvt_dinov3b_512_hexaview_pair_fly38/SHA256SUMS +4 -0
- 261007_mvt_dinov3b_512_hexaview_pair_fly38/config.toml +116 -0
- 261007_mvt_dinov3b_512_hexaview_pair_fly38/fly38.toml +129 -0
- 261007_mvt_dinov3b_512_hexaview_pair_fly38/mvt_dinov3b_512_hexaview_pair_fly38.pth +3 -0
- 261007_mvt_dinov3b_512_hexaview_pair_fly38/report.json +247 -0
- README.md +11 -5
- assets/hexaview-pair.jpg +3 -0
.gitattributes
CHANGED
|
@@ -35,3 +35,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
|
| 35 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
| 36 |
assets/octacam.jpg filter=lfs diff=lfs merge=lfs -text
|
| 37 |
assets/hexaview.jpg filter=lfs diff=lfs merge=lfs -text
|
|
|
|
|
|
| 35 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
| 36 |
assets/octacam.jpg filter=lfs diff=lfs merge=lfs -text
|
| 37 |
assets/hexaview.jpg filter=lfs diff=lfs merge=lfs -text
|
| 38 |
+
assets/hexaview-pair.jpg filter=lfs diff=lfs merge=lfs -text
|
261007_fast_efficientvit_b1_352_hexaview_pair_locator/README.md
ADDED
|
@@ -0,0 +1,18 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# `fast_efficientvit_b1_352_hexaview_pair_locator` -- the two-fly hexaview locator
|
| 2 |
+
|
| 3 |
+
A deeperfly 2 locator (family `fast`, EfficientViT-B1): it reads a whole hexaview source frame, 2048x1408 scaled by exactly 1/4 to 512x352, and returns where each of two flies is in each of the three views that frame carries, one channel per view with a peak per fly (`animals = 2` in the artifact), so a per-frame run can center each fly's window on it. A run names it as `[pose2d] locator`, beside the two-fly detector `261007_mvt_dinov3b_512_hexaview_pair_fly38`.
|
| 4 |
+
|
| 5 |
+
Exported 2026-10-09 from `hexaview/runs/x4loc`, trained 2026-10-07: from ImageNet weights, lr 3e-4, 3,000 steps, on the 17 labeled single-fly recordings, 28 two-fly recordings (12 real, 16 synthetic composites), 31 touching composites, and pseudo-labels in 9 more. The file is x4loc's, byte for byte.
|
| 6 |
+
|
| 7 |
+
| | |
|
| 8 |
+
|---|---|
|
| 9 |
+
| file | `fast_efficientvit_b1_352_hexaview_pair_locator.pth`, 18,733,207 bytes |
|
| 10 |
+
| sha256 | `1726f5fe6f0964e56edbe9e1026e25594a45b173964283653e7609b08498ec33` |
|
| 11 |
+
| backbone | `efficientvit_b1` (timm) |
|
| 12 |
+
| input | 352x512, the whole source frame; one output channel per view, a peak per fly |
|
| 13 |
+
| trained on | 85 hexaview projects, 28,334 items (`report.json`); no holdout |
|
| 14 |
+
| recipe | `config.toml` beside this file |
|
| 15 |
+
|
| 16 |
+
## Using it
|
| 17 |
+
|
| 18 |
+
See the detector's README for the download and the run config. Verify a download with `sha256sum -c SHA256SUMS` in this folder.
|
261007_fast_efficientvit_b1_352_hexaview_pair_locator/SHA256SUMS
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
2156379721042d6ea90e7991b1baef9583d3189487abd75eca2a0cb161391027 ./config.toml
|
| 2 |
+
1726f5fe6f0964e56edbe9e1026e25594a45b173964283653e7609b08498ec33 ./fast_efficientvit_b1_352_hexaview_pair_locator.pth
|
| 3 |
+
f26d8be4bae51903e8941c080a1f50d0a0208acdbf9bb038f059dcf2b8d52651 ./report.json
|
261007_fast_efficientvit_b1_352_hexaview_pair_locator/config.toml
ADDED
|
@@ -0,0 +1,77 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# x4loc (2026-10-07 evening): b11loc's recipe on the 99 labeled frames, split r16_all.
|
| 2 |
+
format_version = 3
|
| 3 |
+
|
| 4 |
+
[data]
|
| 5 |
+
kind = "project"
|
| 6 |
+
task = "locate"
|
| 7 |
+
projects = [
|
| 8 |
+
"../recordings/*",
|
| 9 |
+
"../../hexaview-pair/recordings/pairs_*",
|
| 10 |
+
"../../hexaview-pair/pseudo4/*",
|
| 11 |
+
"../../hexaview-pair/recordings/reaching_*",
|
| 12 |
+
"../../hexaview-pair/recordings/m06-*",
|
| 13 |
+
"../../hexaview-pair/touch/*",
|
| 14 |
+
"../../hexaview-pair/touch2/*",
|
| 15 |
+
]
|
| 16 |
+
split = "../splits/r16_all.toml"
|
| 17 |
+
cache = "../cache"
|
| 18 |
+
|
| 19 |
+
[data.fill]
|
| 20 |
+
min_views = 2
|
| 21 |
+
max_residual_px = 10.0
|
| 22 |
+
|
| 23 |
+
[model]
|
| 24 |
+
family = "fast"
|
| 25 |
+
backbone = "efficientvit_b1"
|
| 26 |
+
# 2048x1408 scaled by exactly 1/4: the letterbox adds no padding at all, because
|
| 27 |
+
# 512/352 is 2048/1408 to the digit.
|
| 28 |
+
input_hw = [352, 512]
|
| 29 |
+
downsample_factor = 2 # a field cell is 4 input px, so 16 source px
|
| 30 |
+
margin = 0 # a center is always on the sensor; nothing to extrapolate
|
| 31 |
+
reach = 0
|
| 32 |
+
pretrained = true
|
| 33 |
+
|
| 34 |
+
[augment]
|
| 35 |
+
# No mirror and barely any rotation: a locator learns the rig's fixed layout, and a
|
| 36 |
+
# flipped source frame is a rig this one never builds. The photometric range is the
|
| 37 |
+
# pose arms'.
|
| 38 |
+
degrees = 3.0
|
| 39 |
+
scale = [0.95, 1.05]
|
| 40 |
+
aspect = [1.0, 1.0]
|
| 41 |
+
translate = [0.02, 0.02]
|
| 42 |
+
flip_p = 0.0
|
| 43 |
+
brightness = 0.3
|
| 44 |
+
contrast = 0.3
|
| 45 |
+
gamma = [0.7, 1.4]
|
| 46 |
+
blur_sigma = [0.5, 1.5]
|
| 47 |
+
noise_std = 0.03
|
| 48 |
+
photometric_p = 0.5
|
| 49 |
+
|
| 50 |
+
[train]
|
| 51 |
+
steps = 3000
|
| 52 |
+
batch_size = 8 # groups; a group is one moment's two source frames
|
| 53 |
+
grad_clip = 1.0
|
| 54 |
+
amp = true
|
| 55 |
+
ema_decay = 0.999
|
| 56 |
+
view_drop_p = 0.0 # the two sources do not inform each other
|
| 57 |
+
view_min = 1
|
| 58 |
+
log_every = 100
|
| 59 |
+
seed = 0
|
| 60 |
+
out = "../runs/x4loc"
|
| 61 |
+
|
| 62 |
+
[train.optimizer]
|
| 63 |
+
lr = 3e-4
|
| 64 |
+
weight_decay = 0.05
|
| 65 |
+
warmup_steps = 200
|
| 66 |
+
|
| 67 |
+
[train.loss]
|
| 68 |
+
sigma = 2.0
|
| 69 |
+
field_weight = 1.0
|
| 70 |
+
coord_weight = 0.0
|
| 71 |
+
|
| 72 |
+
[train.eval]
|
| 73 |
+
every = 0
|
| 74 |
+
pck_px = 20.0 # source pixels: a center this close draws the right window
|
| 75 |
+
|
| 76 |
+
[compute]
|
| 77 |
+
workers = 6
|
261007_fast_efficientvit_b1_352_hexaview_pair_locator/fast_efficientvit_b1_352_hexaview_pair_locator.pth
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:1726f5fe6f0964e56edbe9e1026e25594a45b173964283653e7609b08498ec33
|
| 3 |
+
size 18733207
|
261007_fast_efficientvit_b1_352_hexaview_pair_locator/report.json
ADDED
|
@@ -0,0 +1,213 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"projects": {
|
| 3 |
+
"260730_001": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview/recordings/260730_001/deeperfly",
|
| 4 |
+
"260804_001": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview/recordings/260804_001/deeperfly",
|
| 5 |
+
"260804_002": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview/recordings/260804_002/deeperfly",
|
| 6 |
+
"260804_003": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview/recordings/260804_003/deeperfly",
|
| 7 |
+
"260804_004": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview/recordings/260804_004/deeperfly",
|
| 8 |
+
"260818_001": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview/recordings/260818_001/deeperfly",
|
| 9 |
+
"260819_001": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview/recordings/260819_001/deeperfly",
|
| 10 |
+
"260819_002": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview/recordings/260819_002/deeperfly",
|
| 11 |
+
"260819_003": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview/recordings/260819_003/deeperfly",
|
| 12 |
+
"260821_002": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview/recordings/260821_002/deeperfly",
|
| 13 |
+
"260916_Fly10_001": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview/recordings/260916_Fly10_001/deeperfly",
|
| 14 |
+
"260916_Fly1_001": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview/recordings/260916_Fly1_001/deeperfly",
|
| 15 |
+
"260916_Fly3_001": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview/recordings/260916_Fly3_001/deeperfly",
|
| 16 |
+
"260916_Fly5_001": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview/recordings/260916_Fly5_001/deeperfly",
|
| 17 |
+
"260916_Fly6_001": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview/recordings/260916_Fly6_001/deeperfly",
|
| 18 |
+
"260916_Fly9_001": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview/recordings/260916_Fly9_001/deeperfly",
|
| 19 |
+
"260917_Fly8_001": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview/recordings/260917_Fly8_001/deeperfly",
|
| 20 |
+
"pairs_260730_001": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview-pair/recordings/pairs_260730_001/deeperfly",
|
| 21 |
+
"pairs_260804_001": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview-pair/recordings/pairs_260804_001/deeperfly",
|
| 22 |
+
"pairs_260804_002": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview-pair/recordings/pairs_260804_002/deeperfly",
|
| 23 |
+
"pairs_260804_003": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview-pair/recordings/pairs_260804_003/deeperfly",
|
| 24 |
+
"pairs_260804_004": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview-pair/recordings/pairs_260804_004/deeperfly",
|
| 25 |
+
"pairs_260818_001": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview-pair/recordings/pairs_260818_001/deeperfly",
|
| 26 |
+
"pairs_260819_001": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview-pair/recordings/pairs_260819_001/deeperfly",
|
| 27 |
+
"pairs_260819_002": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview-pair/recordings/pairs_260819_002/deeperfly",
|
| 28 |
+
"pairs_260819_003": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview-pair/recordings/pairs_260819_003/deeperfly",
|
| 29 |
+
"pairs_260821_002": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview-pair/recordings/pairs_260821_002/deeperfly",
|
| 30 |
+
"pairs_260916_Fly10_001": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview-pair/recordings/pairs_260916_Fly10_001/deeperfly",
|
| 31 |
+
"pairs_260916_Fly1_001": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview-pair/recordings/pairs_260916_Fly1_001/deeperfly",
|
| 32 |
+
"pairs_260916_Fly5_001": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview-pair/recordings/pairs_260916_Fly5_001/deeperfly",
|
| 33 |
+
"pairs_260916_Fly6_001": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview-pair/recordings/pairs_260916_Fly6_001/deeperfly",
|
| 34 |
+
"pairs_260916_Fly9_001": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview-pair/recordings/pairs_260916_Fly9_001/deeperfly",
|
| 35 |
+
"pairs_260917_Fly8_001": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview-pair/recordings/pairs_260917_Fly8_001/deeperfly",
|
| 36 |
+
"pseudo_m06-ff-bright-1h_260803_Fly1_001": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview-pair/pseudo4/pseudo_m06-ff-bright-1h_260803_Fly1_001/deeperfly",
|
| 37 |
+
"pseudo_m06-ff-bright-1h_260807_Fly1_001": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview-pair/pseudo4/pseudo_m06-ff-bright-1h_260807_Fly1_001/deeperfly",
|
| 38 |
+
"pseudo_m06-mm-bright-1h_260804_Fly1_001": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview-pair/pseudo4/pseudo_m06-mm-bright-1h_260804_Fly1_001/deeperfly",
|
| 39 |
+
"pseudo_reaching_260925_Fly1_001": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview-pair/pseudo4/pseudo_reaching_260925_Fly1_001/deeperfly",
|
| 40 |
+
"pseudo_reaching_260928_Fly1_001": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview-pair/pseudo4/pseudo_reaching_260928_Fly1_001/deeperfly",
|
| 41 |
+
"pseudo_reaching_260928_Fly2_001": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview-pair/pseudo4/pseudo_reaching_260928_Fly2_001/deeperfly",
|
| 42 |
+
"pseudo_reaching_261001_Fly1_001": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview-pair/pseudo4/pseudo_reaching_261001_Fly1_001/deeperfly",
|
| 43 |
+
"pseudo_reaching_261001_Fly2_001": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview-pair/pseudo4/pseudo_reaching_261001_Fly2_001/deeperfly",
|
| 44 |
+
"pseudo_two-flies-test-bright_260731_Fly1_001": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview-pair/pseudo4/pseudo_two-flies-test-bright_260731_Fly1_001/deeperfly",
|
| 45 |
+
"reaching_260925_Fly1_001": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview-pair/recordings/reaching_260925_Fly1_001/deeperfly",
|
| 46 |
+
"reaching_260928_Fly1_001": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview-pair/recordings/reaching_260928_Fly1_001/deeperfly",
|
| 47 |
+
"reaching_260928_Fly2_001": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview-pair/recordings/reaching_260928_Fly2_001/deeperfly",
|
| 48 |
+
"reaching_261001_Fly1_001": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview-pair/recordings/reaching_261001_Fly1_001/deeperfly",
|
| 49 |
+
"reaching_261001_Fly2_001": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview-pair/recordings/reaching_261001_Fly2_001/deeperfly",
|
| 50 |
+
"reaching_261005_Fly1_001": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview-pair/recordings/reaching_261005_Fly1_001/deeperfly",
|
| 51 |
+
"reaching_261005_Fly2_001": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview-pair/recordings/reaching_261005_Fly2_001/deeperfly",
|
| 52 |
+
"reaching_261006_Fly1_001": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview-pair/recordings/reaching_261006_Fly1_001/deeperfly",
|
| 53 |
+
"reaching_261006_Fly2_001": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview-pair/recordings/reaching_261006_Fly2_001/deeperfly",
|
| 54 |
+
"m06-ff-bright-1h_260803_Fly1_001": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview-pair/recordings/m06-ff-bright-1h_260803_Fly1_001/deeperfly",
|
| 55 |
+
"m06-ff-bright-1h_260807_Fly1_001": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview-pair/recordings/m06-ff-bright-1h_260807_Fly1_001/deeperfly",
|
| 56 |
+
"m06-mm-bright-1h_260804_Fly1_001": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview-pair/recordings/m06-mm-bright-1h_260804_Fly1_001/deeperfly",
|
| 57 |
+
"touch_260730_001": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview-pair/touch/touch_260730_001/deeperfly",
|
| 58 |
+
"touch_260804_001": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview-pair/touch/touch_260804_001/deeperfly",
|
| 59 |
+
"touch_260804_002": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview-pair/touch/touch_260804_002/deeperfly",
|
| 60 |
+
"touch_260804_003": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview-pair/touch/touch_260804_003/deeperfly",
|
| 61 |
+
"touch_260804_004": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview-pair/touch/touch_260804_004/deeperfly",
|
| 62 |
+
"touch_260818_001": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview-pair/touch/touch_260818_001/deeperfly",
|
| 63 |
+
"touch_260819_001": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview-pair/touch/touch_260819_001/deeperfly",
|
| 64 |
+
"touch_260819_002": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview-pair/touch/touch_260819_002/deeperfly",
|
| 65 |
+
"touch_260819_003": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview-pair/touch/touch_260819_003/deeperfly",
|
| 66 |
+
"touch_260821_002": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview-pair/touch/touch_260821_002/deeperfly",
|
| 67 |
+
"touch_260916_Fly10_001": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview-pair/touch/touch_260916_Fly10_001/deeperfly",
|
| 68 |
+
"touch_260916_Fly1_001": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview-pair/touch/touch_260916_Fly1_001/deeperfly",
|
| 69 |
+
"touch_260916_Fly3_001": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview-pair/touch/touch_260916_Fly3_001/deeperfly",
|
| 70 |
+
"touch_260916_Fly5_001": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview-pair/touch/touch_260916_Fly5_001/deeperfly",
|
| 71 |
+
"touch_260916_Fly6_001": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview-pair/touch/touch_260916_Fly6_001/deeperfly",
|
| 72 |
+
"touch_260916_Fly9_001": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview-pair/touch/touch_260916_Fly9_001/deeperfly",
|
| 73 |
+
"touch_260917_Fly8_001": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview-pair/touch/touch_260917_Fly8_001/deeperfly",
|
| 74 |
+
"touch2_pseudo_260730_001": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview-pair/touch2/touch2_pseudo_260730_001/deeperfly",
|
| 75 |
+
"touch2_pseudo_260804_003": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview-pair/touch2/touch2_pseudo_260804_003/deeperfly",
|
| 76 |
+
"touch2_pseudo_260804_004": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview-pair/touch2/touch2_pseudo_260804_004/deeperfly",
|
| 77 |
+
"touch2_pseudo_260818_001": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview-pair/touch2/touch2_pseudo_260818_001/deeperfly",
|
| 78 |
+
"touch2_pseudo_260819_001": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview-pair/touch2/touch2_pseudo_260819_001/deeperfly",
|
| 79 |
+
"touch2_pseudo_260819_002": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview-pair/touch2/touch2_pseudo_260819_002/deeperfly",
|
| 80 |
+
"touch2_pseudo_260821_002": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview-pair/touch2/touch2_pseudo_260821_002/deeperfly",
|
| 81 |
+
"touch2_pseudo_260916_Fly10_001": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview-pair/touch2/touch2_pseudo_260916_Fly10_001/deeperfly",
|
| 82 |
+
"touch2_pseudo_260916_Fly1_001": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview-pair/touch2/touch2_pseudo_260916_Fly1_001/deeperfly",
|
| 83 |
+
"touch2_pseudo_260916_Fly3_001": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview-pair/touch2/touch2_pseudo_260916_Fly3_001/deeperfly",
|
| 84 |
+
"touch2_pseudo_260916_Fly5_001": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview-pair/touch2/touch2_pseudo_260916_Fly5_001/deeperfly",
|
| 85 |
+
"touch2_pseudo_260916_Fly6_001": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview-pair/touch2/touch2_pseudo_260916_Fly6_001/deeperfly",
|
| 86 |
+
"touch2_pseudo_260916_Fly9_001": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview-pair/touch2/touch2_pseudo_260916_Fly9_001/deeperfly",
|
| 87 |
+
"touch2_pseudo_260917_Fly8_001": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview-pair/touch2/touch2_pseudo_260917_Fly8_001/deeperfly"
|
| 88 |
+
},
|
| 89 |
+
"train_projects": [
|
| 90 |
+
"260730_001",
|
| 91 |
+
"260804_001",
|
| 92 |
+
"260804_002",
|
| 93 |
+
"260804_003",
|
| 94 |
+
"260804_004",
|
| 95 |
+
"260818_001",
|
| 96 |
+
"260819_001",
|
| 97 |
+
"260819_002",
|
| 98 |
+
"260819_003",
|
| 99 |
+
"260821_002",
|
| 100 |
+
"260916_Fly10_001",
|
| 101 |
+
"260916_Fly1_001",
|
| 102 |
+
"260916_Fly3_001",
|
| 103 |
+
"260916_Fly5_001",
|
| 104 |
+
"260916_Fly6_001",
|
| 105 |
+
"260916_Fly9_001",
|
| 106 |
+
"260917_Fly8_001",
|
| 107 |
+
"m06-ff-bright-1h_260803_Fly1_001",
|
| 108 |
+
"m06-ff-bright-1h_260807_Fly1_001",
|
| 109 |
+
"m06-mm-bright-1h_260804_Fly1_001",
|
| 110 |
+
"pairs_260730_001",
|
| 111 |
+
"pairs_260804_001",
|
| 112 |
+
"pairs_260804_002",
|
| 113 |
+
"pairs_260804_003",
|
| 114 |
+
"pairs_260804_004",
|
| 115 |
+
"pairs_260818_001",
|
| 116 |
+
"pairs_260819_001",
|
| 117 |
+
"pairs_260819_002",
|
| 118 |
+
"pairs_260819_003",
|
| 119 |
+
"pairs_260821_002",
|
| 120 |
+
"pairs_260916_Fly10_001",
|
| 121 |
+
"pairs_260916_Fly1_001",
|
| 122 |
+
"pairs_260916_Fly5_001",
|
| 123 |
+
"pairs_260916_Fly6_001",
|
| 124 |
+
"pairs_260916_Fly9_001",
|
| 125 |
+
"pairs_260917_Fly8_001",
|
| 126 |
+
"pseudo_m06-ff-bright-1h_260803_Fly1_001",
|
| 127 |
+
"pseudo_m06-ff-bright-1h_260807_Fly1_001",
|
| 128 |
+
"pseudo_m06-mm-bright-1h_260804_Fly1_001",
|
| 129 |
+
"pseudo_reaching_260925_Fly1_001",
|
| 130 |
+
"pseudo_reaching_260928_Fly1_001",
|
| 131 |
+
"pseudo_reaching_260928_Fly2_001",
|
| 132 |
+
"pseudo_reaching_261001_Fly1_001",
|
| 133 |
+
"pseudo_reaching_261001_Fly2_001",
|
| 134 |
+
"pseudo_two-flies-test-bright_260731_Fly1_001",
|
| 135 |
+
"reaching_260925_Fly1_001",
|
| 136 |
+
"reaching_260928_Fly1_001",
|
| 137 |
+
"reaching_260928_Fly2_001",
|
| 138 |
+
"reaching_261001_Fly1_001",
|
| 139 |
+
"reaching_261001_Fly2_001",
|
| 140 |
+
"reaching_261005_Fly1_001",
|
| 141 |
+
"reaching_261005_Fly2_001",
|
| 142 |
+
"reaching_261006_Fly1_001",
|
| 143 |
+
"reaching_261006_Fly2_001",
|
| 144 |
+
"touch2_pseudo_260730_001",
|
| 145 |
+
"touch2_pseudo_260804_003",
|
| 146 |
+
"touch2_pseudo_260804_004",
|
| 147 |
+
"touch2_pseudo_260818_001",
|
| 148 |
+
"touch2_pseudo_260819_001",
|
| 149 |
+
"touch2_pseudo_260819_002",
|
| 150 |
+
"touch2_pseudo_260821_002",
|
| 151 |
+
"touch2_pseudo_260916_Fly10_001",
|
| 152 |
+
"touch2_pseudo_260916_Fly1_001",
|
| 153 |
+
"touch2_pseudo_260916_Fly3_001",
|
| 154 |
+
"touch2_pseudo_260916_Fly5_001",
|
| 155 |
+
"touch2_pseudo_260916_Fly6_001",
|
| 156 |
+
"touch2_pseudo_260916_Fly9_001",
|
| 157 |
+
"touch2_pseudo_260917_Fly8_001",
|
| 158 |
+
"touch_260730_001",
|
| 159 |
+
"touch_260804_001",
|
| 160 |
+
"touch_260804_002",
|
| 161 |
+
"touch_260804_003",
|
| 162 |
+
"touch_260804_004",
|
| 163 |
+
"touch_260818_001",
|
| 164 |
+
"touch_260819_001",
|
| 165 |
+
"touch_260819_002",
|
| 166 |
+
"touch_260819_003",
|
| 167 |
+
"touch_260821_002",
|
| 168 |
+
"touch_260916_Fly10_001",
|
| 169 |
+
"touch_260916_Fly1_001",
|
| 170 |
+
"touch_260916_Fly3_001",
|
| 171 |
+
"touch_260916_Fly5_001",
|
| 172 |
+
"touch_260916_Fly6_001",
|
| 173 |
+
"touch_260916_Fly9_001",
|
| 174 |
+
"touch_260917_Fly8_001"
|
| 175 |
+
],
|
| 176 |
+
"holdout": [],
|
| 177 |
+
"holdout_frames": {},
|
| 178 |
+
"n_train_items": 28334,
|
| 179 |
+
"augment": {
|
| 180 |
+
"degrees": 3.0,
|
| 181 |
+
"scale": [
|
| 182 |
+
0.95,
|
| 183 |
+
1.05
|
| 184 |
+
],
|
| 185 |
+
"aspect": [
|
| 186 |
+
1.0,
|
| 187 |
+
1.0
|
| 188 |
+
],
|
| 189 |
+
"translate": [
|
| 190 |
+
0.02,
|
| 191 |
+
0.02
|
| 192 |
+
],
|
| 193 |
+
"flip_p": 0.0,
|
| 194 |
+
"rig_symmetry": false,
|
| 195 |
+
"brightness": 0.3,
|
| 196 |
+
"contrast": 0.3,
|
| 197 |
+
"gamma": [
|
| 198 |
+
0.7,
|
| 199 |
+
1.4
|
| 200 |
+
],
|
| 201 |
+
"blur_sigma": [
|
| 202 |
+
0.5,
|
| 203 |
+
1.5
|
| 204 |
+
],
|
| 205 |
+
"noise_std": 0.03,
|
| 206 |
+
"photometric_p": 0.5
|
| 207 |
+
},
|
| 208 |
+
"final_loss": 4.932494640350342,
|
| 209 |
+
"holdout_report": null,
|
| 210 |
+
"select_best": null,
|
| 211 |
+
"best_step": null,
|
| 212 |
+
"best_score": null
|
| 213 |
+
}
|
261007_mvt_dinov3b_512_hexaview_pair_fly38/README.md
ADDED
|
@@ -0,0 +1,43 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# `mvt_dinov3b_512_hexaview_pair_fly38` -- the two-fly hexaview detector, round x4
|
| 2 |
+
|
| 3 |
+
A deeperfly 2 detector (`deeperfly-detector-1`, family `mvt`): a multiview transformer on a DINOv3 ViT-B/16 that predicts all 38 points of the `fly38` skeleton in the six views of the hexaview rig for each of two flies, with one window per fly and view, centered on that fly frame by frame. Each window marks where the other fly is, a white disk in a black ring, so a window on two touching flies says which one it is about. The artifact records this (`mark_others`), and deeperfly draws the mark itself (ADR-0148). It runs with its own locator, `261007_fast_efficientvit_b1_352_hexaview_pair_locator`, which finds both flies in every view. For a single fly, use `261001_mvt_dinov3b_512_hexaview_fly38`.
|
| 4 |
+
|
| 5 |
+
Exported 2026-10-09 from `hexaview/runs/x4pose`, trained 2026-10-07: warm-started from a pseudo-labeled ViT-B, lr 5e-5, 3,000 steps, with a transductive distillation stream on unlabeled frames and the rig's symmetries as augmentation. It trained on 99 hand-labeled frames of 12 real two-fly recordings, the 17 labeled single-fly recordings, 64 synthetic two-fly composites built from the single-fly labels (side by side, touching and stacked), and pseudo-labels in 9 more recordings. The weights are x4's; the export added `mark_others = true` to the artifact and `mark_others`, `mark_jitter = 25.0` to `config.toml`, which x4 trained with as environment variables.
|
| 6 |
+
|
| 7 |
+
| | |
|
| 8 |
+
|---|---|
|
| 9 |
+
| file | `mvt_dinov3b_512_hexaview_pair_fly38.pth`, 383,806,315 bytes |
|
| 10 |
+
| sha256 | `24a0d4839d4af6b963192d23611e6066a1a71254b0c8a8df569ba447b3bf7496` |
|
| 11 |
+
| skeleton | `fly38` -- `fly38.toml` beside this file |
|
| 12 |
+
| backbone | `vit_base_patch16_dinov3.lvd1689m` (timm) |
|
| 13 |
+
| input | 512x512 per view and fly: a square window centered on the fly, the other fly's center marked |
|
| 14 |
+
| trained on | 102 hexaview projects, 186,330 items (`report.json`); no holdout |
|
| 15 |
+
| recipe | `config.toml` beside this file |
|
| 16 |
+
|
| 17 |
+
## Running it
|
| 18 |
+
|
| 19 |
+
It needs a deeperfly that draws the mark (ADR-0148, 2026-10-09 on). An older one loads it without a word and runs it on unmarked windows, which moved 14.6% of its points by more than 5 px on frames where the flies are close. The project's manifest says `animals = 2`, and the run config names both files with windows 1.2 times the fly's size:
|
| 20 |
+
|
| 21 |
+
```toml
|
| 22 |
+
[pose2d]
|
| 23 |
+
weights = ["../models/261007_mvt_dinov3b_512_hexaview_pair_fly38/mvt_dinov3b_512_hexaview_pair_fly38.pth"]
|
| 24 |
+
locator = "../models/261007_fast_efficientvit_b1_352_hexaview_pair_locator/fast_efficientvit_b1_352_hexaview_pair_locator.pth"
|
| 25 |
+
window = "per_frame"
|
| 26 |
+
tightness = 1.2
|
| 27 |
+
```
|
| 28 |
+
|
| 29 |
+
## What it scores
|
| 30 |
+
|
| 31 |
+
A shipping model trains on every label, so it has no held-out number of its own. Its twin from the round before, x3cv (the same recipe on 82 of the labeled frames, with recordings held out), was scored on 35 held-out two-fly frames at these windows. The share of keypoints more than 15 px from the label was 4.0% in the direct views and 9.2% in the prism views in 2D, and 3.1% and 6.3% after triangulation. The single-fly round 11 detector's twin scored 5.8%, 14.7%, 5.2% and 8.2% on the same frames.
|
| 32 |
+
|
| 33 |
+
## Using it
|
| 34 |
+
|
| 35 |
+
Download it into `models/` at the root of a deeperfly clone, with its locator:
|
| 36 |
+
|
| 37 |
+
```bash
|
| 38 |
+
uv run hf download tkclam/deeperfly-models --local-dir models \
|
| 39 |
+
--include "261007_mvt_dinov3b_512_hexaview_pair_fly38/*" \
|
| 40 |
+
--include "261007_fast_efficientvit_b1_352_hexaview_pair_locator/*"
|
| 41 |
+
```
|
| 42 |
+
|
| 43 |
+
Verify a download with `sha256sum -c SHA256SUMS` in this folder.
|
261007_mvt_dinov3b_512_hexaview_pair_fly38/SHA256SUMS
ADDED
|
@@ -0,0 +1,4 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
34b52a15bf9720dbe1441614fa396038ad8026afd9aad44f24619b960bc5ce7c ./config.toml
|
| 2 |
+
56f9a33ea5c286bda06e599602e7b7adaefb7ba29a74a2516548b7cbb7776e52 ./fly38.toml
|
| 3 |
+
24a0d4839d4af6b963192d23611e6066a1a71254b0c8a8df569ba447b3bf7496 ./mvt_dinov3b_512_hexaview_pair_fly38.pth
|
| 4 |
+
451cb69b099ca5ca650148813f71d271fe3f9a5b66291d3fe9103c543956919c ./report.json
|
261007_mvt_dinov3b_512_hexaview_pair_fly38/config.toml
ADDED
|
@@ -0,0 +1,116 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Shipped as 261007_mvt_dinov3b_512_hexaview_pair_fly38 (2026-10-09): the environment
|
| 2 |
+
# variables named below are the [data] keys mark_others and mark_jitter now.
|
| 3 |
+
# x4pose (2026-10-07 evening): x3pose's recipe on all 99 labeled real frames (82 at x3,
|
| 4 |
+
# new: 260925, 261001_Fly1, 261005, 261006, m06-ff 260807). Train with
|
| 5 |
+
# DEEPERFLY_OTHER_MARKER=1 DEEPERFLY_OTHER_MARKER_JITTER=25 from the frozen worktree
|
| 6 |
+
# ~/deeperfly-work/b1/repo. Predict with jobs/predict_x4.toml and
|
| 7 |
+
# DEEPERFLY_WINDOW_TIGHTNESS=1.2 DEEPERFLY_OTHER_MARKER=1. Split r16_all: pseudo4 guarded.
|
| 8 |
+
format_version = 3
|
| 9 |
+
|
| 10 |
+
[data]
|
| 11 |
+
kind = "project"
|
| 12 |
+
projects = [
|
| 13 |
+
"../recordings/*",
|
| 14 |
+
"../../hexaview-pair/edge/pairs/*",
|
| 15 |
+
"../../hexaview-pair/pseudo4/*",
|
| 16 |
+
"../../hexaview-pair/recordings/reaching_*",
|
| 17 |
+
"../../hexaview-pair/recordings/m06-*",
|
| 18 |
+
"../../hexaview-pair/edge/touch/*",
|
| 19 |
+
"../../hexaview-pair/edge/touch2/*",
|
| 20 |
+
"../../hexaview-pair/edge/stack/*",
|
| 21 |
+
]
|
| 22 |
+
split = "../splits/r16_all.toml"
|
| 23 |
+
cache = "../cache"
|
| 24 |
+
# The other-animal mark, which x4 trained with as the environment variables
|
| 25 |
+
# DEEPERFLY_OTHER_MARKER=1 DEEPERFLY_OTHER_MARKER_JITTER=25 before they were these
|
| 26 |
+
# keys (ADR-0148).
|
| 27 |
+
mark_others = true
|
| 28 |
+
mark_jitter = 25.0
|
| 29 |
+
|
| 30 |
+
[data.crops]
|
| 31 |
+
tightness = [0.75, 2.2] # per-sample crop around the animal, log-uniform
|
| 32 |
+
jitter = 0.1
|
| 33 |
+
eval_tightness = 1.25
|
| 34 |
+
full_p = 0.0 # see above: a whole video frame is not one view's
|
| 35 |
+
|
| 36 |
+
[data.fill]
|
| 37 |
+
weight = 0.25
|
| 38 |
+
max_residual_px = 10.0
|
| 39 |
+
|
| 40 |
+
[model]
|
| 41 |
+
input_hw = [512, 512]
|
| 42 |
+
family = "mvt"
|
| 43 |
+
backbone = "vit_base_patch16_dinov3.lvd1689m"
|
| 44 |
+
pretrained = true
|
| 45 |
+
attn_scopes = "llllllllglgl"
|
| 46 |
+
margin = 64
|
| 47 |
+
reach = 2
|
| 48 |
+
drop_path = 0.0
|
| 49 |
+
token_mask = 0.0
|
| 50 |
+
token_pad = 2
|
| 51 |
+
init_from = "../runs/b2pseudo"
|
| 52 |
+
|
| 53 |
+
[augment]
|
| 54 |
+
degrees = 15.0 # a jitter: the symmetry moves the views together
|
| 55 |
+
rig_symmetry = true
|
| 56 |
+
scale = [1.0, 1.0] # the crop sampler supplies scale and shift
|
| 57 |
+
aspect = [0.85, 1.18]
|
| 58 |
+
translate = [0.0, 0.0]
|
| 59 |
+
flip_p = 0.5
|
| 60 |
+
brightness = 0.3
|
| 61 |
+
contrast = 0.3
|
| 62 |
+
gamma = [0.7, 1.4]
|
| 63 |
+
blur_sigma = [0.5, 1.5]
|
| 64 |
+
noise_std = 0.03
|
| 65 |
+
photometric_p = 0.5
|
| 66 |
+
|
| 67 |
+
[train]
|
| 68 |
+
grad_checkpoint = true
|
| 69 |
+
steps = 3000
|
| 70 |
+
images_per_batch = 24 # four 6-view moments per step
|
| 71 |
+
grad_clip = 1.0
|
| 72 |
+
amp = true
|
| 73 |
+
ema_decay = 0.999
|
| 74 |
+
view_drop_p = 0.3
|
| 75 |
+
view_min = 1
|
| 76 |
+
log_every = 100
|
| 77 |
+
seed = 0
|
| 78 |
+
out = "../runs/x4pose"
|
| 79 |
+
|
| 80 |
+
[train.optimizer]
|
| 81 |
+
lr = 5e-5
|
| 82 |
+
weight_decay = 0.05
|
| 83 |
+
warmup_steps = 200
|
| 84 |
+
|
| 85 |
+
[train.loss]
|
| 86 |
+
sigma = 2.0
|
| 87 |
+
field_weight = 1.0
|
| 88 |
+
coord_weight = 0.0
|
| 89 |
+
tri_weight = 0.0 # the calibration's units are arbitrary
|
| 90 |
+
|
| 91 |
+
[train.eval]
|
| 92 |
+
every = 0
|
| 93 |
+
|
| 94 |
+
[compute]
|
| 95 |
+
workers = 8
|
| 96 |
+
|
| 97 |
+
[unlabeled]
|
| 98 |
+
exclude = ["reaching_*", "m06-*", "touche_*", "touch2e_*", "stacke_*"]
|
| 99 |
+
# No `teacher`: the teacher's points, confidences and windows come from each
|
| 100 |
+
# project's merged detections (ADR-0083): r12ship_all.
|
| 101 |
+
n_per_project = 250
|
| 102 |
+
transductive = true
|
| 103 |
+
agree_px = 6.0
|
| 104 |
+
min_views = 3
|
| 105 |
+
conf_min = 0.3
|
| 106 |
+
geom_weight = 0.3
|
| 107 |
+
weight = 0.5
|
| 108 |
+
mask_ratios = [0.0, 0.5, 0.9]
|
| 109 |
+
masked_weight = 1.0
|
| 110 |
+
unmasked_weight = 0.5
|
| 111 |
+
crop_tightness = [0.75, 1.6]
|
| 112 |
+
cache = "../cache/unlabeled"
|
| 113 |
+
|
| 114 |
+
[data.boost]
|
| 115 |
+
projects = ["reaching_*", "m06-*"]
|
| 116 |
+
share = 0.25
|
261007_mvt_dinov3b_512_hexaview_pair_fly38/fly38.toml
ADDED
|
@@ -0,0 +1,129 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# fly38 -- the 38 tracked points of a tethered Drosophila: six 5-point legs, two
|
| 2 |
+
# antennae, the neck, and a 5-point dorsal-midline abdomen chain (the tergite stripes).
|
| 3 |
+
#
|
| 4 |
+
# A skeleton is points, edges and point_symmetries -- the same decomposition as SLEAP
|
| 5 |
+
# and DeepLabCut -- plus an optional palette (point_colors / edge_colors). The palette
|
| 6 |
+
# is decoration, not identity: it draws the points but is kept out of the digest, so
|
| 7 |
+
# recoloring never reports a different skeleton.
|
| 8 |
+
|
| 9 |
+
[skeleton]
|
| 10 |
+
name = "fly38"
|
| 11 |
+
|
| 12 |
+
# The points IN ORDER: channel i of the detector is points[i]. This is the contract
|
| 13 |
+
# with the weights.
|
| 14 |
+
points = [
|
| 15 |
+
"lf_thorax_coxa", "lf_coxa_trochanter", "lf_femur_tibia", "lf_tibia_tarsus", "lf_pretarsus",
|
| 16 |
+
"lm_thorax_coxa", "lm_coxa_trochanter", "lm_femur_tibia", "lm_tibia_tarsus", "lm_pretarsus",
|
| 17 |
+
"lh_thorax_coxa", "lh_coxa_trochanter", "lh_femur_tibia", "lh_tibia_tarsus", "lh_pretarsus",
|
| 18 |
+
"l_antenna",
|
| 19 |
+
"rf_thorax_coxa", "rf_coxa_trochanter", "rf_femur_tibia", "rf_tibia_tarsus", "rf_pretarsus",
|
| 20 |
+
"rm_thorax_coxa", "rm_coxa_trochanter", "rm_femur_tibia", "rm_tibia_tarsus", "rm_pretarsus",
|
| 21 |
+
"rh_thorax_coxa", "rh_coxa_trochanter", "rh_femur_tibia", "rh_tibia_tarsus", "rh_pretarsus",
|
| 22 |
+
"r_antenna",
|
| 23 |
+
"neck",
|
| 24 |
+
"abdomen0", "abdomen1", "abdomen2", "abdomen3", "abdomen4",
|
| 25 |
+
]
|
| 26 |
+
|
| 27 |
+
# The edges, as pairs of points. The antennae and neck appear in no edge -- tracked
|
| 28 |
+
# points with no bone.
|
| 29 |
+
edges = [
|
| 30 |
+
["lf_thorax_coxa", "lf_coxa_trochanter"], ["lf_coxa_trochanter", "lf_femur_tibia"],
|
| 31 |
+
["lf_femur_tibia", "lf_tibia_tarsus"], ["lf_tibia_tarsus", "lf_pretarsus"],
|
| 32 |
+
["lm_thorax_coxa", "lm_coxa_trochanter"], ["lm_coxa_trochanter", "lm_femur_tibia"],
|
| 33 |
+
["lm_femur_tibia", "lm_tibia_tarsus"], ["lm_tibia_tarsus", "lm_pretarsus"],
|
| 34 |
+
["lh_thorax_coxa", "lh_coxa_trochanter"], ["lh_coxa_trochanter", "lh_femur_tibia"],
|
| 35 |
+
["lh_femur_tibia", "lh_tibia_tarsus"], ["lh_tibia_tarsus", "lh_pretarsus"],
|
| 36 |
+
["rf_thorax_coxa", "rf_coxa_trochanter"], ["rf_coxa_trochanter", "rf_femur_tibia"],
|
| 37 |
+
["rf_femur_tibia", "rf_tibia_tarsus"], ["rf_tibia_tarsus", "rf_pretarsus"],
|
| 38 |
+
["rm_thorax_coxa", "rm_coxa_trochanter"], ["rm_coxa_trochanter", "rm_femur_tibia"],
|
| 39 |
+
["rm_femur_tibia", "rm_tibia_tarsus"], ["rm_tibia_tarsus", "rm_pretarsus"],
|
| 40 |
+
["rh_thorax_coxa", "rh_coxa_trochanter"], ["rh_coxa_trochanter", "rh_femur_tibia"],
|
| 41 |
+
["rh_femur_tibia", "rh_tibia_tarsus"], ["rh_tibia_tarsus", "rh_pretarsus"],
|
| 42 |
+
["abdomen0", "abdomen1"], ["abdomen1", "abdomen2"], ["abdomen2", "abdomen3"], ["abdomen3", "abdomen4"],
|
| 43 |
+
]
|
| 44 |
+
|
| 45 |
+
# What each point becomes under a left/right flip. Write each pair once; anything absent
|
| 46 |
+
# is on the midline and maps to itself (hence no neck, no abdomen). The loader checks
|
| 47 |
+
# the implied mirror against `edges`: applying it must give the edge set back, so a
|
| 48 |
+
# wrong side or two joints of one leg exchanged is named rather than trained on silently.
|
| 49 |
+
point_symmetries = [
|
| 50 |
+
["lf_thorax_coxa", "rf_thorax_coxa"],
|
| 51 |
+
["lf_coxa_trochanter", "rf_coxa_trochanter"],
|
| 52 |
+
["lf_femur_tibia", "rf_femur_tibia"],
|
| 53 |
+
["lf_tibia_tarsus", "rf_tibia_tarsus"],
|
| 54 |
+
["lf_pretarsus", "rf_pretarsus"],
|
| 55 |
+
["lm_thorax_coxa", "rm_thorax_coxa"],
|
| 56 |
+
["lm_coxa_trochanter", "rm_coxa_trochanter"],
|
| 57 |
+
["lm_femur_tibia", "rm_femur_tibia"],
|
| 58 |
+
["lm_tibia_tarsus", "rm_tibia_tarsus"],
|
| 59 |
+
["lm_pretarsus", "rm_pretarsus"],
|
| 60 |
+
["lh_thorax_coxa", "rh_thorax_coxa"],
|
| 61 |
+
["lh_coxa_trochanter", "rh_coxa_trochanter"],
|
| 62 |
+
["lh_femur_tibia", "rh_femur_tibia"],
|
| 63 |
+
["lh_tibia_tarsus", "rh_tibia_tarsus"],
|
| 64 |
+
["lh_pretarsus", "rh_pretarsus"],
|
| 65 |
+
["l_antenna", "r_antenna"],
|
| 66 |
+
]
|
| 67 |
+
|
| 68 |
+
# The palette. One #rrggbb per point; a key is a point name or a `*` pattern (an exact
|
| 69 |
+
# name beats a pattern). Optional and may be partial -- an unnamed point takes tab10 by
|
| 70 |
+
# index. The scheme is anatomical, so it reads at a glance: left = blue, right = red,
|
| 71 |
+
# front -> hind lightening within a side; the midline neck and abdomen green. Edges take
|
| 72 |
+
# the average of their endpoints, so every leg bone comes out its own limb's color.
|
| 73 |
+
[skeleton.point_colors]
|
| 74 |
+
"lf_*" = "#0f7399"
|
| 75 |
+
"lm_*" = "#1a8daf"
|
| 76 |
+
"lh_*" = "#75becb"
|
| 77 |
+
l_antenna = "#0a4f6b"
|
| 78 |
+
"rf_*" = "#ba1e31"
|
| 79 |
+
"rm_*" = "#c9564f"
|
| 80 |
+
"rh_*" = "#d58579"
|
| 81 |
+
r_antenna = "#8c1525"
|
| 82 |
+
neck = "#15a315"
|
| 83 |
+
"abdomen*" = "#61e47b"
|
| 84 |
+
|
| 85 |
+
# A canonical neutral posture (NeuroMechFly's rest pose), one [x, y, z] per point
|
| 86 |
+
# in millimeters, with the body center of mass at the origin (x anterior, y the
|
| 87 |
+
# left-right axis, z dorsal). It is NOT part of the skeleton's identity (out of the
|
| 88 |
+
# digest); it draws the skeleton in 3D and SEEDS labeling: projected through a
|
| 89 |
+
# recording's calibration it drops a rough draggable instance into each view when
|
| 90 |
+
# nothing else (a prior labeled frame, another view) is available to seed from.
|
| 91 |
+
[skeleton.default_pose]
|
| 92 |
+
lf_thorax_coxa = [ 0.42966, 0.17200, 0.25226]
|
| 93 |
+
lf_coxa_trochanter = [ 0.31028, 0.06883, -0.07688]
|
| 94 |
+
lf_femur_tibia = [ 0.85227, 0.42043, 0.20537]
|
| 95 |
+
lf_tibia_tarsus = [ 1.05606, 0.65936, -0.20658]
|
| 96 |
+
lf_pretarsus = [ 1.45133, 1.02885, -0.58856]
|
| 97 |
+
lm_thorax_coxa = [ 0.03366, 0.12500, 0.00026]
|
| 98 |
+
lm_coxa_trochanter = [ 0.00831, 0.09367, -0.17620]
|
| 99 |
+
lm_femur_tibia = [ 0.11746, 0.70120, 0.30719]
|
| 100 |
+
lm_tibia_tarsus = [ 0.23055, 0.96680, -0.29411]
|
| 101 |
+
lm_pretarsus = [ 0.38484, 1.49119, -0.73358]
|
| 102 |
+
lh_thorax_coxa = [-0.19334, 0.08700, -0.01474]
|
| 103 |
+
lh_coxa_trochanter = [-0.27402, 0.09655, -0.19640]
|
| 104 |
+
lh_femur_tibia = [-0.64760, 0.63110, 0.32665]
|
| 105 |
+
lh_tibia_tarsus = [-0.96496, 0.77982, -0.26073]
|
| 106 |
+
lh_pretarsus = [-1.40791, 1.03885, -0.86325]
|
| 107 |
+
l_antenna = [ 1.04836, 0.08600, 0.56596]
|
| 108 |
+
rf_thorax_coxa = [ 0.42966, -0.17200, 0.25226]
|
| 109 |
+
rf_coxa_trochanter = [ 0.31028, -0.06883, -0.07688]
|
| 110 |
+
rf_femur_tibia = [ 0.85227, -0.42043, 0.20537]
|
| 111 |
+
rf_tibia_tarsus = [ 1.05606, -0.65936, -0.20658]
|
| 112 |
+
rf_pretarsus = [ 1.45133, -1.02885, -0.58856]
|
| 113 |
+
rm_thorax_coxa = [ 0.03366, -0.12500, 0.00026]
|
| 114 |
+
rm_coxa_trochanter = [ 0.00831, -0.09367, -0.17620]
|
| 115 |
+
rm_femur_tibia = [ 0.11746, -0.70120, 0.30719]
|
| 116 |
+
rm_tibia_tarsus = [ 0.23055, -0.96680, -0.29411]
|
| 117 |
+
rm_pretarsus = [ 0.38484, -1.49119, -0.73358]
|
| 118 |
+
rh_thorax_coxa = [-0.19334, -0.08700, -0.01474]
|
| 119 |
+
rh_coxa_trochanter = [-0.27402, -0.09655, -0.19640]
|
| 120 |
+
rh_femur_tibia = [-0.64760, -0.63110, 0.32665]
|
| 121 |
+
rh_tibia_tarsus = [-0.96496, -0.77982, -0.26073]
|
| 122 |
+
rh_pretarsus = [-1.40791, -1.03885, -0.86325]
|
| 123 |
+
r_antenna = [ 1.04836, -0.08600, 0.56596]
|
| 124 |
+
neck = [ 0.61936, 0.00000, 0.48296]
|
| 125 |
+
abdomen0 = [-0.71698, 0.00000, 0.71444]
|
| 126 |
+
abdomen1 = [-0.91268, 0.00000, 0.66169]
|
| 127 |
+
abdomen2 = [-1.10060, 0.00000, 0.61850]
|
| 128 |
+
abdomen3 = [-1.27163, 0.00000, 0.56551]
|
| 129 |
+
abdomen4 = [-1.48737, 0.00000, 0.46364]
|
261007_mvt_dinov3b_512_hexaview_pair_fly38/mvt_dinov3b_512_hexaview_pair_fly38.pth
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:24a0d4839d4af6b963192d23611e6066a1a71254b0c8a8df569ba447b3bf7496
|
| 3 |
+
size 383806315
|
261007_mvt_dinov3b_512_hexaview_pair_fly38/report.json
ADDED
|
@@ -0,0 +1,247 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"projects": {
|
| 3 |
+
"260730_001": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview/recordings/260730_001/deeperfly",
|
| 4 |
+
"260804_001": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview/recordings/260804_001/deeperfly",
|
| 5 |
+
"260804_002": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview/recordings/260804_002/deeperfly",
|
| 6 |
+
"260804_003": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview/recordings/260804_003/deeperfly",
|
| 7 |
+
"260804_004": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview/recordings/260804_004/deeperfly",
|
| 8 |
+
"260818_001": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview/recordings/260818_001/deeperfly",
|
| 9 |
+
"260819_001": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview/recordings/260819_001/deeperfly",
|
| 10 |
+
"260819_002": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview/recordings/260819_002/deeperfly",
|
| 11 |
+
"260819_003": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview/recordings/260819_003/deeperfly",
|
| 12 |
+
"260821_002": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview/recordings/260821_002/deeperfly",
|
| 13 |
+
"260916_Fly10_001": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview/recordings/260916_Fly10_001/deeperfly",
|
| 14 |
+
"260916_Fly1_001": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview/recordings/260916_Fly1_001/deeperfly",
|
| 15 |
+
"260916_Fly3_001": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview/recordings/260916_Fly3_001/deeperfly",
|
| 16 |
+
"260916_Fly5_001": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview/recordings/260916_Fly5_001/deeperfly",
|
| 17 |
+
"260916_Fly6_001": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview/recordings/260916_Fly6_001/deeperfly",
|
| 18 |
+
"260916_Fly9_001": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview/recordings/260916_Fly9_001/deeperfly",
|
| 19 |
+
"260917_Fly8_001": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview/recordings/260917_Fly8_001/deeperfly",
|
| 20 |
+
"pairse_260730_001": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview-pair/edge/pairs/pairse_260730_001/deeperfly",
|
| 21 |
+
"pairse_260804_001": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview-pair/edge/pairs/pairse_260804_001/deeperfly",
|
| 22 |
+
"pairse_260804_002": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview-pair/edge/pairs/pairse_260804_002/deeperfly",
|
| 23 |
+
"pairse_260804_003": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview-pair/edge/pairs/pairse_260804_003/deeperfly",
|
| 24 |
+
"pairse_260804_004": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview-pair/edge/pairs/pairse_260804_004/deeperfly",
|
| 25 |
+
"pairse_260818_001": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview-pair/edge/pairs/pairse_260818_001/deeperfly",
|
| 26 |
+
"pairse_260819_001": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview-pair/edge/pairs/pairse_260819_001/deeperfly",
|
| 27 |
+
"pairse_260819_002": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview-pair/edge/pairs/pairse_260819_002/deeperfly",
|
| 28 |
+
"pairse_260819_003": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview-pair/edge/pairs/pairse_260819_003/deeperfly",
|
| 29 |
+
"pairse_260821_002": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview-pair/edge/pairs/pairse_260821_002/deeperfly",
|
| 30 |
+
"pairse_260916_Fly10_001": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview-pair/edge/pairs/pairse_260916_Fly10_001/deeperfly",
|
| 31 |
+
"pairse_260916_Fly1_001": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview-pair/edge/pairs/pairse_260916_Fly1_001/deeperfly",
|
| 32 |
+
"pairse_260916_Fly5_001": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview-pair/edge/pairs/pairse_260916_Fly5_001/deeperfly",
|
| 33 |
+
"pairse_260916_Fly6_001": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview-pair/edge/pairs/pairse_260916_Fly6_001/deeperfly",
|
| 34 |
+
"pairse_260916_Fly9_001": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview-pair/edge/pairs/pairse_260916_Fly9_001/deeperfly",
|
| 35 |
+
"pairse_260917_Fly8_001": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview-pair/edge/pairs/pairse_260917_Fly8_001/deeperfly",
|
| 36 |
+
"pseudo_m06-ff-bright-1h_260803_Fly1_001": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview-pair/pseudo4/pseudo_m06-ff-bright-1h_260803_Fly1_001/deeperfly",
|
| 37 |
+
"pseudo_m06-ff-bright-1h_260807_Fly1_001": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview-pair/pseudo4/pseudo_m06-ff-bright-1h_260807_Fly1_001/deeperfly",
|
| 38 |
+
"pseudo_m06-mm-bright-1h_260804_Fly1_001": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview-pair/pseudo4/pseudo_m06-mm-bright-1h_260804_Fly1_001/deeperfly",
|
| 39 |
+
"pseudo_reaching_260925_Fly1_001": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview-pair/pseudo4/pseudo_reaching_260925_Fly1_001/deeperfly",
|
| 40 |
+
"pseudo_reaching_260928_Fly1_001": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview-pair/pseudo4/pseudo_reaching_260928_Fly1_001/deeperfly",
|
| 41 |
+
"pseudo_reaching_260928_Fly2_001": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview-pair/pseudo4/pseudo_reaching_260928_Fly2_001/deeperfly",
|
| 42 |
+
"pseudo_reaching_261001_Fly1_001": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview-pair/pseudo4/pseudo_reaching_261001_Fly1_001/deeperfly",
|
| 43 |
+
"pseudo_reaching_261001_Fly2_001": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview-pair/pseudo4/pseudo_reaching_261001_Fly2_001/deeperfly",
|
| 44 |
+
"pseudo_two-flies-test-bright_260731_Fly1_001": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview-pair/pseudo4/pseudo_two-flies-test-bright_260731_Fly1_001/deeperfly",
|
| 45 |
+
"reaching_260925_Fly1_001": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview-pair/recordings/reaching_260925_Fly1_001/deeperfly",
|
| 46 |
+
"reaching_260928_Fly1_001": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview-pair/recordings/reaching_260928_Fly1_001/deeperfly",
|
| 47 |
+
"reaching_260928_Fly2_001": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview-pair/recordings/reaching_260928_Fly2_001/deeperfly",
|
| 48 |
+
"reaching_261001_Fly1_001": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview-pair/recordings/reaching_261001_Fly1_001/deeperfly",
|
| 49 |
+
"reaching_261001_Fly2_001": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview-pair/recordings/reaching_261001_Fly2_001/deeperfly",
|
| 50 |
+
"reaching_261005_Fly1_001": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview-pair/recordings/reaching_261005_Fly1_001/deeperfly",
|
| 51 |
+
"reaching_261005_Fly2_001": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview-pair/recordings/reaching_261005_Fly2_001/deeperfly",
|
| 52 |
+
"reaching_261006_Fly1_001": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview-pair/recordings/reaching_261006_Fly1_001/deeperfly",
|
| 53 |
+
"reaching_261006_Fly2_001": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview-pair/recordings/reaching_261006_Fly2_001/deeperfly",
|
| 54 |
+
"m06-ff-bright-1h_260803_Fly1_001": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview-pair/recordings/m06-ff-bright-1h_260803_Fly1_001/deeperfly",
|
| 55 |
+
"m06-ff-bright-1h_260807_Fly1_001": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview-pair/recordings/m06-ff-bright-1h_260807_Fly1_001/deeperfly",
|
| 56 |
+
"m06-mm-bright-1h_260804_Fly1_001": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview-pair/recordings/m06-mm-bright-1h_260804_Fly1_001/deeperfly",
|
| 57 |
+
"touche_260730_001": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview-pair/edge/touch/touche_260730_001/deeperfly",
|
| 58 |
+
"touche_260804_001": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview-pair/edge/touch/touche_260804_001/deeperfly",
|
| 59 |
+
"touche_260804_002": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview-pair/edge/touch/touche_260804_002/deeperfly",
|
| 60 |
+
"touche_260804_003": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview-pair/edge/touch/touche_260804_003/deeperfly",
|
| 61 |
+
"touche_260804_004": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview-pair/edge/touch/touche_260804_004/deeperfly",
|
| 62 |
+
"touche_260818_001": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview-pair/edge/touch/touche_260818_001/deeperfly",
|
| 63 |
+
"touche_260819_001": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview-pair/edge/touch/touche_260819_001/deeperfly",
|
| 64 |
+
"touche_260819_002": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview-pair/edge/touch/touche_260819_002/deeperfly",
|
| 65 |
+
"touche_260819_003": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview-pair/edge/touch/touche_260819_003/deeperfly",
|
| 66 |
+
"touche_260821_002": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview-pair/edge/touch/touche_260821_002/deeperfly",
|
| 67 |
+
"touche_260916_Fly10_001": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview-pair/edge/touch/touche_260916_Fly10_001/deeperfly",
|
| 68 |
+
"touche_260916_Fly1_001": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview-pair/edge/touch/touche_260916_Fly1_001/deeperfly",
|
| 69 |
+
"touche_260916_Fly3_001": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview-pair/edge/touch/touche_260916_Fly3_001/deeperfly",
|
| 70 |
+
"touche_260916_Fly5_001": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview-pair/edge/touch/touche_260916_Fly5_001/deeperfly",
|
| 71 |
+
"touche_260916_Fly6_001": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview-pair/edge/touch/touche_260916_Fly6_001/deeperfly",
|
| 72 |
+
"touche_260916_Fly9_001": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview-pair/edge/touch/touche_260916_Fly9_001/deeperfly",
|
| 73 |
+
"touche_260917_Fly8_001": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview-pair/edge/touch/touche_260917_Fly8_001/deeperfly",
|
| 74 |
+
"touch2e_pseudo_260730_001": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview-pair/edge/touch2/touch2e_pseudo_260730_001/deeperfly",
|
| 75 |
+
"touch2e_pseudo_260804_003": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview-pair/edge/touch2/touch2e_pseudo_260804_003/deeperfly",
|
| 76 |
+
"touch2e_pseudo_260804_004": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview-pair/edge/touch2/touch2e_pseudo_260804_004/deeperfly",
|
| 77 |
+
"touch2e_pseudo_260818_001": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview-pair/edge/touch2/touch2e_pseudo_260818_001/deeperfly",
|
| 78 |
+
"touch2e_pseudo_260819_001": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview-pair/edge/touch2/touch2e_pseudo_260819_001/deeperfly",
|
| 79 |
+
"touch2e_pseudo_260819_002": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview-pair/edge/touch2/touch2e_pseudo_260819_002/deeperfly",
|
| 80 |
+
"touch2e_pseudo_260821_002": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview-pair/edge/touch2/touch2e_pseudo_260821_002/deeperfly",
|
| 81 |
+
"touch2e_pseudo_260916_Fly10_001": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview-pair/edge/touch2/touch2e_pseudo_260916_Fly10_001/deeperfly",
|
| 82 |
+
"touch2e_pseudo_260916_Fly1_001": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview-pair/edge/touch2/touch2e_pseudo_260916_Fly1_001/deeperfly",
|
| 83 |
+
"touch2e_pseudo_260916_Fly3_001": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview-pair/edge/touch2/touch2e_pseudo_260916_Fly3_001/deeperfly",
|
| 84 |
+
"touch2e_pseudo_260916_Fly5_001": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview-pair/edge/touch2/touch2e_pseudo_260916_Fly5_001/deeperfly",
|
| 85 |
+
"touch2e_pseudo_260916_Fly6_001": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview-pair/edge/touch2/touch2e_pseudo_260916_Fly6_001/deeperfly",
|
| 86 |
+
"touch2e_pseudo_260916_Fly9_001": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview-pair/edge/touch2/touch2e_pseudo_260916_Fly9_001/deeperfly",
|
| 87 |
+
"touch2e_pseudo_260917_Fly8_001": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview-pair/edge/touch2/touch2e_pseudo_260917_Fly8_001/deeperfly",
|
| 88 |
+
"stacke_260730_001": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview-pair/edge/stack/stacke_260730_001/deeperfly",
|
| 89 |
+
"stacke_260804_001": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview-pair/edge/stack/stacke_260804_001/deeperfly",
|
| 90 |
+
"stacke_260804_002": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview-pair/edge/stack/stacke_260804_002/deeperfly",
|
| 91 |
+
"stacke_260804_003": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview-pair/edge/stack/stacke_260804_003/deeperfly",
|
| 92 |
+
"stacke_260804_004": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview-pair/edge/stack/stacke_260804_004/deeperfly",
|
| 93 |
+
"stacke_260818_001": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview-pair/edge/stack/stacke_260818_001/deeperfly",
|
| 94 |
+
"stacke_260819_001": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview-pair/edge/stack/stacke_260819_001/deeperfly",
|
| 95 |
+
"stacke_260819_002": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview-pair/edge/stack/stacke_260819_002/deeperfly",
|
| 96 |
+
"stacke_260819_003": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview-pair/edge/stack/stacke_260819_003/deeperfly",
|
| 97 |
+
"stacke_260821_002": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview-pair/edge/stack/stacke_260821_002/deeperfly",
|
| 98 |
+
"stacke_260916_Fly10_001": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview-pair/edge/stack/stacke_260916_Fly10_001/deeperfly",
|
| 99 |
+
"stacke_260916_Fly1_001": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview-pair/edge/stack/stacke_260916_Fly1_001/deeperfly",
|
| 100 |
+
"stacke_260916_Fly3_001": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview-pair/edge/stack/stacke_260916_Fly3_001/deeperfly",
|
| 101 |
+
"stacke_260916_Fly5_001": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview-pair/edge/stack/stacke_260916_Fly5_001/deeperfly",
|
| 102 |
+
"stacke_260916_Fly6_001": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview-pair/edge/stack/stacke_260916_Fly6_001/deeperfly",
|
| 103 |
+
"stacke_260916_Fly9_001": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview-pair/edge/stack/stacke_260916_Fly9_001/deeperfly",
|
| 104 |
+
"stacke_260917_Fly8_001": "/mnt/upramdya/data/TL/deeperfly-projects/hexaview-pair/edge/stack/stacke_260917_Fly8_001/deeperfly"
|
| 105 |
+
},
|
| 106 |
+
"train_projects": [
|
| 107 |
+
"260730_001",
|
| 108 |
+
"260804_001",
|
| 109 |
+
"260804_002",
|
| 110 |
+
"260804_003",
|
| 111 |
+
"260804_004",
|
| 112 |
+
"260818_001",
|
| 113 |
+
"260819_001",
|
| 114 |
+
"260819_002",
|
| 115 |
+
"260819_003",
|
| 116 |
+
"260821_002",
|
| 117 |
+
"260916_Fly10_001",
|
| 118 |
+
"260916_Fly1_001",
|
| 119 |
+
"260916_Fly3_001",
|
| 120 |
+
"260916_Fly5_001",
|
| 121 |
+
"260916_Fly6_001",
|
| 122 |
+
"260916_Fly9_001",
|
| 123 |
+
"260917_Fly8_001",
|
| 124 |
+
"m06-ff-bright-1h_260803_Fly1_001",
|
| 125 |
+
"m06-ff-bright-1h_260807_Fly1_001",
|
| 126 |
+
"m06-mm-bright-1h_260804_Fly1_001",
|
| 127 |
+
"pairse_260730_001",
|
| 128 |
+
"pairse_260804_001",
|
| 129 |
+
"pairse_260804_002",
|
| 130 |
+
"pairse_260804_003",
|
| 131 |
+
"pairse_260804_004",
|
| 132 |
+
"pairse_260818_001",
|
| 133 |
+
"pairse_260819_001",
|
| 134 |
+
"pairse_260819_002",
|
| 135 |
+
"pairse_260819_003",
|
| 136 |
+
"pairse_260821_002",
|
| 137 |
+
"pairse_260916_Fly10_001",
|
| 138 |
+
"pairse_260916_Fly1_001",
|
| 139 |
+
"pairse_260916_Fly5_001",
|
| 140 |
+
"pairse_260916_Fly6_001",
|
| 141 |
+
"pairse_260916_Fly9_001",
|
| 142 |
+
"pairse_260917_Fly8_001",
|
| 143 |
+
"pseudo_m06-ff-bright-1h_260803_Fly1_001",
|
| 144 |
+
"pseudo_m06-ff-bright-1h_260807_Fly1_001",
|
| 145 |
+
"pseudo_m06-mm-bright-1h_260804_Fly1_001",
|
| 146 |
+
"pseudo_reaching_260925_Fly1_001",
|
| 147 |
+
"pseudo_reaching_260928_Fly1_001",
|
| 148 |
+
"pseudo_reaching_260928_Fly2_001",
|
| 149 |
+
"pseudo_reaching_261001_Fly1_001",
|
| 150 |
+
"pseudo_reaching_261001_Fly2_001",
|
| 151 |
+
"pseudo_two-flies-test-bright_260731_Fly1_001",
|
| 152 |
+
"reaching_260925_Fly1_001",
|
| 153 |
+
"reaching_260928_Fly1_001",
|
| 154 |
+
"reaching_260928_Fly2_001",
|
| 155 |
+
"reaching_261001_Fly1_001",
|
| 156 |
+
"reaching_261001_Fly2_001",
|
| 157 |
+
"reaching_261005_Fly1_001",
|
| 158 |
+
"reaching_261005_Fly2_001",
|
| 159 |
+
"reaching_261006_Fly1_001",
|
| 160 |
+
"reaching_261006_Fly2_001",
|
| 161 |
+
"stacke_260730_001",
|
| 162 |
+
"stacke_260804_001",
|
| 163 |
+
"stacke_260804_002",
|
| 164 |
+
"stacke_260804_003",
|
| 165 |
+
"stacke_260804_004",
|
| 166 |
+
"stacke_260818_001",
|
| 167 |
+
"stacke_260819_001",
|
| 168 |
+
"stacke_260819_002",
|
| 169 |
+
"stacke_260819_003",
|
| 170 |
+
"stacke_260821_002",
|
| 171 |
+
"stacke_260916_Fly10_001",
|
| 172 |
+
"stacke_260916_Fly1_001",
|
| 173 |
+
"stacke_260916_Fly3_001",
|
| 174 |
+
"stacke_260916_Fly5_001",
|
| 175 |
+
"stacke_260916_Fly6_001",
|
| 176 |
+
"stacke_260916_Fly9_001",
|
| 177 |
+
"stacke_260917_Fly8_001",
|
| 178 |
+
"touch2e_pseudo_260730_001",
|
| 179 |
+
"touch2e_pseudo_260804_003",
|
| 180 |
+
"touch2e_pseudo_260804_004",
|
| 181 |
+
"touch2e_pseudo_260818_001",
|
| 182 |
+
"touch2e_pseudo_260819_001",
|
| 183 |
+
"touch2e_pseudo_260819_002",
|
| 184 |
+
"touch2e_pseudo_260821_002",
|
| 185 |
+
"touch2e_pseudo_260916_Fly10_001",
|
| 186 |
+
"touch2e_pseudo_260916_Fly1_001",
|
| 187 |
+
"touch2e_pseudo_260916_Fly3_001",
|
| 188 |
+
"touch2e_pseudo_260916_Fly5_001",
|
| 189 |
+
"touch2e_pseudo_260916_Fly6_001",
|
| 190 |
+
"touch2e_pseudo_260916_Fly9_001",
|
| 191 |
+
"touch2e_pseudo_260917_Fly8_001",
|
| 192 |
+
"touche_260730_001",
|
| 193 |
+
"touche_260804_001",
|
| 194 |
+
"touche_260804_002",
|
| 195 |
+
"touche_260804_003",
|
| 196 |
+
"touche_260804_004",
|
| 197 |
+
"touche_260818_001",
|
| 198 |
+
"touche_260819_001",
|
| 199 |
+
"touche_260819_002",
|
| 200 |
+
"touche_260819_003",
|
| 201 |
+
"touche_260821_002",
|
| 202 |
+
"touche_260916_Fly10_001",
|
| 203 |
+
"touche_260916_Fly1_001",
|
| 204 |
+
"touche_260916_Fly3_001",
|
| 205 |
+
"touche_260916_Fly5_001",
|
| 206 |
+
"touche_260916_Fly6_001",
|
| 207 |
+
"touche_260916_Fly9_001",
|
| 208 |
+
"touche_260917_Fly8_001"
|
| 209 |
+
],
|
| 210 |
+
"holdout": [],
|
| 211 |
+
"holdout_frames": {},
|
| 212 |
+
"n_train_items": 186330,
|
| 213 |
+
"augment": {
|
| 214 |
+
"degrees": 15.0,
|
| 215 |
+
"scale": [
|
| 216 |
+
1.0,
|
| 217 |
+
1.0
|
| 218 |
+
],
|
| 219 |
+
"aspect": [
|
| 220 |
+
0.85,
|
| 221 |
+
1.18
|
| 222 |
+
],
|
| 223 |
+
"translate": [
|
| 224 |
+
0.0,
|
| 225 |
+
0.0
|
| 226 |
+
],
|
| 227 |
+
"flip_p": 0.5,
|
| 228 |
+
"rig_symmetry": true,
|
| 229 |
+
"brightness": 0.3,
|
| 230 |
+
"contrast": 0.3,
|
| 231 |
+
"gamma": [
|
| 232 |
+
0.7,
|
| 233 |
+
1.4
|
| 234 |
+
],
|
| 235 |
+
"blur_sigma": [
|
| 236 |
+
0.5,
|
| 237 |
+
1.5
|
| 238 |
+
],
|
| 239 |
+
"noise_std": 0.03,
|
| 240 |
+
"photometric_p": 0.5
|
| 241 |
+
},
|
| 242 |
+
"final_loss": 6.937206745147705,
|
| 243 |
+
"holdout_report": null,
|
| 244 |
+
"select_best": null,
|
| 245 |
+
"best_step": null,
|
| 246 |
+
"best_score": null
|
| 247 |
+
}
|
README.md
CHANGED
|
@@ -17,28 +17,34 @@ tags:
|
|
| 17 |
|
| 18 |
# deeperfly detectors
|
| 19 |
|
| 20 |
-
Pretrained 2D keypoint detectors for [deeperfly](https://github.com/NeLy-EPFL/deeperfly), markerless 3D pose estimation of *Drosophila* from multi-camera rigs. They
|
| 21 |
|
| 22 |
| Folder | Rig | What it is | Backbone | Trained on |
|
| 23 |
| --- | --- | --- | --- | --- |
|
| 24 |
| `261008_mvt_dinov3b_384_octacam_fly38` | octacam: eight cameras around a tethered fly on a ball | the octacam detector (round 14): multiview transformer, all 38 `fly38` keypoints in every view, 384x768 window per view | DINOv3 ViT-B/16 | 62 recordings, 4,703 labeled items |
|
| 25 |
| `261001_mvt_dinov3b_512_hexaview_fly38` | hexaview: two cameras, each seeing a freely walking fly directly and through two prisms (six views) | the hexaview detector (round 13), for one fly: multiview transformer, 38 keypoints, 512x512 window centered on the fly in each frame; run it at `[pose2d] tightness = 1.2` | DINOv3 ViT-B/16 | 17 recordings, 3,240 labeled items |
|
| 26 |
| `260920_fast_efficientvit_b1_352_hexaview_locator` | hexaview | full-frame locator that places the hexaview detector's windows | EfficientViT-B1 | 9 recordings, 824 labeled items |
|
|
|
|
|
|
|
| 27 |
|
| 28 |
A real-time octacam detector is in training and will be added here.
|
| 29 |
|
| 30 |
### Expected input
|
| 31 |
|
| 32 |
-
Each detector expects frames like these
|
| 33 |
|
| 34 |
**octacam**
|
| 35 |
|
| 36 |
|
| 37 |
|
| 38 |
-
**hexaview**
|
| 39 |
|
| 40 |
|
| 41 |
|
|
|
|
|
|
|
|
|
|
|
|
|
| 42 |
Each folder holds the checkpoint (`.pth`), the skeleton it predicts (`fly38.toml`), the training recipe (`config.toml`), the training report (`report.json`), `SHA256SUMS`, and a `README.md` written when the model was exported. The paths in those READMEs are the lab's; use the instructions below instead.
|
| 43 |
|
| 44 |
## Download
|
|
@@ -71,12 +77,12 @@ Paths resolve against the config file. See the [deeperfly quickstart](https://ne
|
|
| 71 |
|
| 72 |
## Scope and limitations
|
| 73 |
|
| 74 |
-
The detectors were trained on recordings from two rigs in one lab, with every label used for training (no held-out set). The octacam detector's twin, trained with four recordings held out, scores 3.25 px median and 4.49 px mean on their 9,269 labeled cells (93.7% within 10 px). On the bundled example clips, the octacam detector reprojects at 1.46 px median after bundle adjustment and the hexaview detector at 3.00 px median through the recording's calibration; that is agreement across views, not error against labels. They are not expected to transfer to other rigs, magnifications, or lighting without retraining. `deeperfly train` fits a detector on your own labels, and these checkpoints can serve as a starting point.
|
| 75 |
|
| 76 |
## License
|
| 77 |
|
| 78 |
- The DINOv3-based detectors are derivatives of Meta's DINOv3 weights and are distributed under the [DINOv3 License](LICENSE.md), including its use restrictions.
|
| 79 |
-
- The EfficientViT-based
|
| 80 |
|
| 81 |
## Citation
|
| 82 |
|
|
|
|
| 17 |
|
| 18 |
# deeperfly detectors
|
| 19 |
|
| 20 |
+
Pretrained 2D keypoint detectors for [deeperfly](https://github.com/NeLy-EPFL/deeperfly), markerless 3D pose estimation of *Drosophila* from multi-camera rigs. They include the detectors the example run configs in the deeperfly repository name, all trained in the [Ramdya lab](https://www.epfl.ch/labs/ramdya-lab/) at EPFL on hand labels from two rigs.
|
| 21 |
|
| 22 |
| Folder | Rig | What it is | Backbone | Trained on |
|
| 23 |
| --- | --- | --- | --- | --- |
|
| 24 |
| `261008_mvt_dinov3b_384_octacam_fly38` | octacam: eight cameras around a tethered fly on a ball | the octacam detector (round 14): multiview transformer, all 38 `fly38` keypoints in every view, 384x768 window per view | DINOv3 ViT-B/16 | 62 recordings, 4,703 labeled items |
|
| 25 |
| `261001_mvt_dinov3b_512_hexaview_fly38` | hexaview: two cameras, each seeing a freely walking fly directly and through two prisms (six views) | the hexaview detector (round 13), for one fly: multiview transformer, 38 keypoints, 512x512 window centered on the fly in each frame; run it at `[pose2d] tightness = 1.2` | DINOv3 ViT-B/16 | 17 recordings, 3,240 labeled items |
|
| 26 |
| `260920_fast_efficientvit_b1_352_hexaview_locator` | hexaview | full-frame locator that places the hexaview detector's windows | EfficientViT-B1 | 9 recordings, 824 labeled items |
|
| 27 |
+
| `261007_mvt_dinov3b_512_hexaview_pair_fly38` | hexaview, two flies | the two-fly hexaview detector (round x4): one 512x512 window per fly and view, each marking the other fly's center; needs deeperfly from 2026-10-09 on, which draws the mark (ADR-0148); run it at `[pose2d] tightness = 1.2` | DINOv3 ViT-B/16 | 102 projects (12 real two-fly recordings with 99 labeled frames, single-fly recordings, synthetic composites, pseudo-labels), 186,330 items |
|
| 28 |
+
| `261007_fast_efficientvit_b1_352_hexaview_pair_locator` | hexaview, two flies | full-frame locator that finds both flies in every view, for the two-fly detector | EfficientViT-B1 | 85 projects, 28,334 items |
|
| 29 |
|
| 30 |
A real-time octacam detector is in training and will be added here.
|
| 31 |
|
| 32 |
### Expected input
|
| 33 |
|
| 34 |
+
Each detector expects frames like these (the first two are the example clips in the deeperfly repository); the dashed boxes are its windows, and the skeleton is what it predicts. [Pretrained models](https://nely-epfl.github.io/deeperfly/stable/models/) in the deeperfly docs describes both rigs.
|
| 35 |
|
| 36 |
**octacam**
|
| 37 |
|
| 38 |
|
| 39 |
|
| 40 |
+
**hexaview, one fly**
|
| 41 |
|
| 42 |
|
| 43 |
|
| 44 |
+
**hexaview, two flies**
|
| 45 |
+
|
| 46 |
+
|
| 47 |
+
|
| 48 |
Each folder holds the checkpoint (`.pth`), the skeleton it predicts (`fly38.toml`), the training recipe (`config.toml`), the training report (`report.json`), `SHA256SUMS`, and a `README.md` written when the model was exported. The paths in those READMEs are the lab's; use the instructions below instead.
|
| 49 |
|
| 50 |
## Download
|
|
|
|
| 77 |
|
| 78 |
## Scope and limitations
|
| 79 |
|
| 80 |
+
The detectors were trained on recordings from two rigs in one lab, with every label used for training (no held-out set). The octacam detector's twin, trained with four recordings held out, scores 3.25 px median and 4.49 px mean on their 9,269 labeled cells (93.7% within 10 px). On the bundled example clips, the octacam detector reprojects at 1.46 px median after bundle adjustment and the hexaview detector at 3.00 px median through the recording's calibration; that is agreement across views, not error against labels. The two-fly detector's twin from the round before, trained with recordings held out, put 4.0% of keypoints more than 15 px from the label in the direct views and 9.2% in the prism views on 35 held-out two-fly frames (5.8% and 14.7% for the single-fly detector's twin). They are not expected to transfer to other rigs, magnifications, or lighting without retraining. `deeperfly train` fits a detector on your own labels, and these checkpoints can serve as a starting point.
|
| 81 |
|
| 82 |
## License
|
| 83 |
|
| 84 |
- The DINOv3-based detectors are derivatives of Meta's DINOv3 weights and are distributed under the [DINOv3 License](LICENSE.md), including its use restrictions.
|
| 85 |
+
- The EfficientViT-based locators are distributed under the [Apache License 2.0](LICENSE-apache-2.0), the license of its ImageNet-pretrained backbone.
|
| 86 |
|
| 87 |
## Citation
|
| 88 |
|
assets/hexaview-pair.jpg
ADDED
|
Git LFS Details
|