dwei commited on
Commit
1ecc974
Β·
verified Β·
1 Parent(s): b678176

Add supervised from-scratch j0126 affinity checkpoint (MedNeXt-L, [48,96,96]) + model card

Browse files
Files changed (2) hide show
  1. README.md +85 -0
  2. affinity_scratch_48x96x96.ckpt +3 -0
README.md CHANGED
@@ -1,3 +1,88 @@
1
  ---
2
  license: mit
 
 
 
 
 
 
 
3
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  license: mit
3
+ tags:
4
+ - connectomics
5
+ - electron-microscopy
6
+ - 3d-segmentation
7
+ - affinity-prediction
8
+ - pytorch-connectomics
9
+ - mednext
10
  ---
11
+
12
+ # j0126 affinity model (supervised, from scratch)
13
+
14
+ 3-channel voxel affinity prediction for the **j0126 zebra finch** EM volume, for use with
15
+ [PyTorch Connectomics](https://github.com/PytorchConnectomics/pytorch_connectomics)
16
+ (`tutorials/neuron_j0126`).
17
+
18
+ | | |
19
+ |---|---|
20
+ | File | `affinity_scratch_48x96x96.ckpt` (247 MB) |
21
+ | Architecture | MedNeXt-L, kernel 3 (61.8 M parameters) |
22
+ | Input / output | 1 channel EM β†’ 6 channel affinity (`banis+`; channels 0–2 are the nearest-neighbour affinity used downstream) |
23
+ | Training patch | `[48, 96, 96]` (ZYX) β€” anisotropic, β‰ˆ isotropic in nm for the 9Γ—9Γ—20 nm grid |
24
+ | Training data | the 33 dense-GT j0126 cubes, heavy `aug_em_neuron` augmentation |
25
+ | Schedule | 200 k steps from scratch (no pretrained init), AdamW, lr 1e-3, batch 8 Γ— 4 GPUs |
26
+
27
+ > **This model is not ground-truth-free.** It was trained on labelled j0126 tissue and is
28
+ > the *supervised upper reference* for the zero-shot pipeline in the tutorial, not a
29
+ > drop-in replacement for it. (The dense cubes come from a different alignment than the
30
+ > 50 evaluation skeletons, so it is not voxel-level test contamination β€” but it is
31
+ > supervision on the target domain.)
32
+
33
+ ## Inference window must be `[48, 96, 96]`
34
+
35
+ MedNeXt normalizes with `GroupNorm(num_groups=C, num_channels=C)` β€” per-sample,
36
+ per-channel instance norm with **no running statistics** β€” so normalization statistics are
37
+ computed over the sliding window's spatial extent at every block and **the forward pass is
38
+ window-size dependent**. Running this checkpoint at the tutorial's inherited
39
+ `[144, 144, 144]` is a 3Γ— Z / 1.5Γ— XY mismatch that inverts the trained Z-thin anisotropy.
40
+
41
+ `tutorials/neuron_j0126/1_affinity_supervised.yaml` already sets both
42
+ `model.input_size` and `inference.window.window_size` to `[48, 96, 96]`. Both must be set
43
+ in YAML, never on the CLI: the stage merge can clobber a pre-resolution CLI override.
44
+
45
+ ## Usage
46
+
47
+ ```bash
48
+ huggingface-cli download pytc/j0126 affinity_scratch_48x96x96.ckpt --local-dir ckpt/
49
+
50
+ python scripts/main.py --config tutorials/neuron_j0126/1_affinity_supervised.yaml \
51
+ --mode test --checkpoint ckpt/affinity_scratch_48x96x96.ckpt
52
+ ```
53
+
54
+ Whole volume (726 chunks of 1008Β³ with a 72-voxel halo), one GPU per shard:
55
+
56
+ ```bash
57
+ python scripts/main.py --config tutorials/neuron_j0126/1_affinity_supervised.yaml \
58
+ --mode test --checkpoint ckpt/affinity_scratch_48x96x96.ckpt \
59
+ --shard-id "$SLURM_ARRAY_TASK_ID" --num-shards 726
60
+ ```
61
+
62
+ Output is one float16 CZYX HDF5 per chunk. Affinities are written with the `scale_sigmoid`
63
+ activation, which the tutorial's ABISS watershed thresholds are calibrated against
64
+ (`scale_sigmoid` 0.70 == plain sigmoid 0.9857) β€” do not substitute a plain sigmoid without
65
+ recalibrating step 2.
66
+
67
+ ## Results
68
+
69
+ Full-volume j0126, 50 evaluation skeletons, from `tutorials/neuron_j0126/README.md`. This
70
+ checkpoint is the "scratch" affinity in that table:
71
+
72
+ | Affinity | Decoding | Error correction | NERL mt=0 ↑ | NERL mt=5 ↑ | VOI ↓ |
73
+ |---|---|---|---:|---:|---:|
74
+ | FFN reference | β€” | β€” | **0.526** | 0.538 | **1.856** |
75
+ | scratch | ABISS, exclusion mask | β€” | 0.268 | 0.470 | 2.584 |
76
+ | scratch | + nucleus instance certificate | β€” | 0.287 | 0.482 | 2.562 |
77
+ | scratch | + nucleus instance certificate | morphology-guided branch linking | 0.301 | **0.539** | 2.374 |
78
+ | scratch | + nucleus instance certificate | + 3Γ—3Γ—3 inter-object erosion | **0.441** | 0.528 | 2.440 |
79
+
80
+ Affinity quality alone is not what these numbers measure β€” see the tutorial for the
81
+ decode and error-correction steps that produce them.
82
+
83
+ ## Checkpoint format
84
+
85
+ A PyTorch Lightning checkpoint (`state_dict` keys are prefixed `model.model.`). Optimizer
86
+ state has been stripped for download size; the weights are bit-identical to the reference
87
+ run's `step=00200000.ckpt` (all 517 tensors, float32). It is loadable for inference/finetuning,
88
+ not for resuming the original optimizer trajectory.
affinity_scratch_48x96x96.ckpt ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a9902f49c0b0d15f51b03e6545220a880ffe80a9fb67fa5977a19e3e68c12410
3
+ size 247325642