Add supervised from-scratch j0126 affinity checkpoint (MedNeXt-L, [48,96,96]) + model card
Browse files- README.md +85 -0
- affinity_scratch_48x96x96.ckpt +3 -0
README.md
CHANGED
|
@@ -1,3 +1,88 @@
|
|
| 1 |
---
|
| 2 |
license: mit
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 3 |
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
---
|
| 2 |
license: mit
|
| 3 |
+
tags:
|
| 4 |
+
- connectomics
|
| 5 |
+
- electron-microscopy
|
| 6 |
+
- 3d-segmentation
|
| 7 |
+
- affinity-prediction
|
| 8 |
+
- pytorch-connectomics
|
| 9 |
+
- mednext
|
| 10 |
---
|
| 11 |
+
|
| 12 |
+
# j0126 affinity model (supervised, from scratch)
|
| 13 |
+
|
| 14 |
+
3-channel voxel affinity prediction for the **j0126 zebra finch** EM volume, for use with
|
| 15 |
+
[PyTorch Connectomics](https://github.com/PytorchConnectomics/pytorch_connectomics)
|
| 16 |
+
(`tutorials/neuron_j0126`).
|
| 17 |
+
|
| 18 |
+
| | |
|
| 19 |
+
|---|---|
|
| 20 |
+
| File | `affinity_scratch_48x96x96.ckpt` (247 MB) |
|
| 21 |
+
| Architecture | MedNeXt-L, kernel 3 (61.8 M parameters) |
|
| 22 |
+
| Input / output | 1 channel EM β 6 channel affinity (`banis+`; channels 0β2 are the nearest-neighbour affinity used downstream) |
|
| 23 |
+
| Training patch | `[48, 96, 96]` (ZYX) β anisotropic, β isotropic in nm for the 9Γ9Γ20 nm grid |
|
| 24 |
+
| Training data | the 33 dense-GT j0126 cubes, heavy `aug_em_neuron` augmentation |
|
| 25 |
+
| Schedule | 200 k steps from scratch (no pretrained init), AdamW, lr 1e-3, batch 8 Γ 4 GPUs |
|
| 26 |
+
|
| 27 |
+
> **This model is not ground-truth-free.** It was trained on labelled j0126 tissue and is
|
| 28 |
+
> the *supervised upper reference* for the zero-shot pipeline in the tutorial, not a
|
| 29 |
+
> drop-in replacement for it. (The dense cubes come from a different alignment than the
|
| 30 |
+
> 50 evaluation skeletons, so it is not voxel-level test contamination β but it is
|
| 31 |
+
> supervision on the target domain.)
|
| 32 |
+
|
| 33 |
+
## Inference window must be `[48, 96, 96]`
|
| 34 |
+
|
| 35 |
+
MedNeXt normalizes with `GroupNorm(num_groups=C, num_channels=C)` β per-sample,
|
| 36 |
+
per-channel instance norm with **no running statistics** β so normalization statistics are
|
| 37 |
+
computed over the sliding window's spatial extent at every block and **the forward pass is
|
| 38 |
+
window-size dependent**. Running this checkpoint at the tutorial's inherited
|
| 39 |
+
`[144, 144, 144]` is a 3Γ Z / 1.5Γ XY mismatch that inverts the trained Z-thin anisotropy.
|
| 40 |
+
|
| 41 |
+
`tutorials/neuron_j0126/1_affinity_supervised.yaml` already sets both
|
| 42 |
+
`model.input_size` and `inference.window.window_size` to `[48, 96, 96]`. Both must be set
|
| 43 |
+
in YAML, never on the CLI: the stage merge can clobber a pre-resolution CLI override.
|
| 44 |
+
|
| 45 |
+
## Usage
|
| 46 |
+
|
| 47 |
+
```bash
|
| 48 |
+
huggingface-cli download pytc/j0126 affinity_scratch_48x96x96.ckpt --local-dir ckpt/
|
| 49 |
+
|
| 50 |
+
python scripts/main.py --config tutorials/neuron_j0126/1_affinity_supervised.yaml \
|
| 51 |
+
--mode test --checkpoint ckpt/affinity_scratch_48x96x96.ckpt
|
| 52 |
+
```
|
| 53 |
+
|
| 54 |
+
Whole volume (726 chunks of 1008Β³ with a 72-voxel halo), one GPU per shard:
|
| 55 |
+
|
| 56 |
+
```bash
|
| 57 |
+
python scripts/main.py --config tutorials/neuron_j0126/1_affinity_supervised.yaml \
|
| 58 |
+
--mode test --checkpoint ckpt/affinity_scratch_48x96x96.ckpt \
|
| 59 |
+
--shard-id "$SLURM_ARRAY_TASK_ID" --num-shards 726
|
| 60 |
+
```
|
| 61 |
+
|
| 62 |
+
Output is one float16 CZYX HDF5 per chunk. Affinities are written with the `scale_sigmoid`
|
| 63 |
+
activation, which the tutorial's ABISS watershed thresholds are calibrated against
|
| 64 |
+
(`scale_sigmoid` 0.70 == plain sigmoid 0.9857) β do not substitute a plain sigmoid without
|
| 65 |
+
recalibrating step 2.
|
| 66 |
+
|
| 67 |
+
## Results
|
| 68 |
+
|
| 69 |
+
Full-volume j0126, 50 evaluation skeletons, from `tutorials/neuron_j0126/README.md`. This
|
| 70 |
+
checkpoint is the "scratch" affinity in that table:
|
| 71 |
+
|
| 72 |
+
| Affinity | Decoding | Error correction | NERL mt=0 β | NERL mt=5 β | VOI β |
|
| 73 |
+
|---|---|---|---:|---:|---:|
|
| 74 |
+
| FFN reference | β | β | **0.526** | 0.538 | **1.856** |
|
| 75 |
+
| scratch | ABISS, exclusion mask | β | 0.268 | 0.470 | 2.584 |
|
| 76 |
+
| scratch | + nucleus instance certificate | β | 0.287 | 0.482 | 2.562 |
|
| 77 |
+
| scratch | + nucleus instance certificate | morphology-guided branch linking | 0.301 | **0.539** | 2.374 |
|
| 78 |
+
| scratch | + nucleus instance certificate | + 3Γ3Γ3 inter-object erosion | **0.441** | 0.528 | 2.440 |
|
| 79 |
+
|
| 80 |
+
Affinity quality alone is not what these numbers measure β see the tutorial for the
|
| 81 |
+
decode and error-correction steps that produce them.
|
| 82 |
+
|
| 83 |
+
## Checkpoint format
|
| 84 |
+
|
| 85 |
+
A PyTorch Lightning checkpoint (`state_dict` keys are prefixed `model.model.`). Optimizer
|
| 86 |
+
state has been stripped for download size; the weights are bit-identical to the reference
|
| 87 |
+
run's `step=00200000.ckpt` (all 517 tensors, float32). It is loadable for inference/finetuning,
|
| 88 |
+
not for resuming the original optimizer trajectory.
|
affinity_scratch_48x96x96.ckpt
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:a9902f49c0b0d15f51b03e6545220a880ffe80a9fb67fa5977a19e3e68c12410
|
| 3 |
+
size 247325642
|