File size: 3,197 Bytes
4b948cf a16b69a 4b948cf | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 | ---
license: mit
library_name: pytorch-connectomics
pipeline_tag: image-segmentation
tags:
- connectomics
- electron-microscopy
- em
- neuron-segmentation
- instance-segmentation
- 3d
- affinity
- mednext
- biology
- neuroscience
- pytorch
datasets:
- yanchaoz/EMNeuron
---
# PyTorch Connectomics pretrained models
Pretrained checkpoints for [PyTorch Connectomics](https://github.com/PytorchConnectomics/pytorch_connectomics)
(PyTC) tutorials: 3D electron-microscopy (EM) segmentation of neurons and mitochondria. Each
folder holds one model; its PyTC tutorial has the config, the commands and the full results.
| folder | task | model | data | result | tutorial |
|---|---|---|---|---|---|
| [`neuron_segneuron/`](neuron_segneuron) | **neuron instance segmentation, any EM volume** (zero-shot or fine-tune) | BANIS+ (MedNeXt-L affinities + ABISS watershed) | EMNeuron, 10 labeled EM sources | mean VI 0.869 on SegNeuron's 8 evaluation volumes | [`neuron_segneuron`](https://github.com/PytorchConnectomics/pytorch_connectomics/tree/master/tutorials/neuron_segneuron) |
| [`neuron_nisb/`](neuron_nisb) | neuron instance segmentation | BANIS+ (MedNeXt-L + cc3d) | NISB base | NERL 0.601 | [`neuron_nisb`](https://github.com/PytorchConnectomics/pytorch_connectomics/tree/master/tutorials/neuron_nisb) |
| [`mito_lucchi++/`](mito_lucchi%2B%2B) | mitochondria semantic segmentation | MedNeXt-S | Lucchi++ | Jaccard 0.935 | [`mito_lucchi++`](https://github.com/PytorchConnectomics/pytorch_connectomics/tree/master/tutorials/mito_lucchi%2B%2B) |
## Segment neurons in your EM volume
`neuron_segneuron/banis_emneuron_200k.ckpt` is the general-purpose neuron model: trained on
EMNeuron, the EM database of [SegNeuron](https://github.com/yanchaoz/SegNeuron): 10 labeled
sources (AxonEM-H, AxonEM-M, CREMI, FIB-25, Hemibrain, J0126, Kasthuri, three Mira volumes),
6-9 nm in xy and 8-50 nm in z. Start here for a new EM volume; fine-tune if you have labels.
```bash
git clone https://github.com/PytorchConnectomics/pytorch_connectomics && cd pytorch_connectomics
# install: see QUICKSTART.md; ABISS (github.com/PytorchConnectomics/ABISS) decodes affinities
# input: a directory of <name>_img.h5 (one 3D uint8 dataset, ZYX)
ABISS_HOME=/path/to/abiss tutorials/neuron_segneuron/predict.sh /path/to/volumes /path/to/output
# -> /path/to/output/<name>_seg.h5 (neuron ids) and <name>_aff.h5 (affinities)
```
`predict.sh` downloads the checkpoint on first use. `TTA=1` adds 16-view test-time
augmentation; `MERGE=0.7` (default 0.6) splits more if neurons are merged.
Fine-tune on labeled volumes (`<name>_img.h5` + `<name>_seg.h5` instance labels):
```bash
tutorials/neuron_segneuron/finetune.sh /path/to/train /path/to/val /path/to/run
ABISS_HOME=/path/to/abiss tutorials/neuron_segneuron/predict.sh <printed checkpoint> /path/to/volumes /path/to/output
```
Download only the checkpoint:
```bash
hf download pytc/tutorial neuron_segneuron/banis_emneuron_200k.ckpt --local-dir checkpoints
```
## Citation
If you use these models, cite PyTorch Connectomics (arXiv:2112.05754); for the EMNeuron
model also cite SegNeuron (Zhang et al., MICCAI 2024), whose data it was trained on.
|