File size: 4,641 Bytes
75d75e4
 
e94e0a3
9fc3bc2
e94e0a3
9fc3bc2
 
7a37aaf
9fc3bc2
7a37aaf
f29267a
 
9fc3bc2
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
122a1d6
9fc3bc2
 
 
122a1d6
9fc3bc2
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
---
license: cc-by-nc-sa-4.0
datasets:
  - Dnq2025/Mask2former_Finetune
base_model:
  - facebook/mask2former-swin-large-ade-semantic
  - Dnq2025/swin-large-mask2former-finetuned-ER-Mito-LD8
language:
  - en
pipeline_tag: image-segmentation
library_name: transformers
tags:
  - Electron-Microscopy
  - Ultrastructure-Segmentation
  - Semantic-Segmentation
---

# MicroStructFormer (Swin-Large)

MicroStructFormer is a high-capacity Mask2Former model built on a Swin-Large backbone, designed for the semantic segmentation of ultrastructures in Electron Microscopy (EM) images.

## Model Description

This model has been trained to accurately segment the following critical ultrastructural elements:
- Abnormalities
- Mitochondria (and mitochondria-like organelles)
- Periaxonal Space

*Note: While the model architecture theoretically supports axon and myelin segmentation, this specific release is optimized and calibrated for the classes listed below due to dataset annotations.*

## Usage

You can use this model easily via the `transformers` library.

### 1. Using Pipeline (Recommended)

```python
from transformers import pipeline
from PIL import Image

# Initialize the segmentation pipeline
pipe = pipeline("image-segmentation", model="Dnq2025/MicroStructFormer")

# Load your EM image
image = Image.open("examples/example_tem.png").convert("RGB")

# Run inference
results = pipe(image)
print(results)
```

### 2. Manual Inference with AutoImageProcessor

```python
import torch
import numpy as np
from PIL import Image
from transformers import AutoImageProcessor, Mask2FormerForUniversalSegmentation

repo_id = "Dnq2025/MicroStructFormer"

# Load processor and model
processor = AutoImageProcessor.from_pretrained(repo_id)
model = Mask2FormerForUniversalSegmentation.from_pretrained(repo_id)
model.eval()

# Load image
image = Image.open("examples/example_tem.png").convert("RGB")
inputs = processor(images=image, return_tensors="pt")

with torch.no_grad():
    outputs = model(**inputs)

# Post-process to get the semantic segmentation map
pred = processor.post_process_semantic_segmentation(
    outputs,
    target_sizes=[image.size[::-1]]
)[0].cpu().numpy().astype(np.uint8)

print(f"Prediction shape: {pred.shape}")
print(f"Unique classes found: {np.unique(pred)}")
```

## Label Map

| ID | Class |
|---:|---|
| 0 | background |
| 1 | abnormality |
| 2 | mitochondria-like organelles |
| 3 | mitochondria |
| 4 | mitochondria outside axon |
| 5 | periaxonal space |

## Example

**Input EM image:**

![Example input](sample/sample_image.png)

**Predicted segmentation overlay:**

![Example prediction](sample/sample_prediction.png)

## Model Use, License, and Citation

This Hugging Face model repository is associated with the following bioRxiv preprint:

> Deng N, Miao G, Khadra A, Peterson AC, Bagheri H. Deep learning-based decoding of axonal ultrastructure in gene-edited mice using electron microscopy imaging. bioRxiv. 2026. DOI: 10.64898/2026.05.26.727755.

These model weights and associated architectures are provided for non-commercial academic and research use under the **CC-BY-NC-SA 4.0** license selected for this Hugging Face repository. Users must provide appropriate attribution to the original authors and cite the associated preprint when using these weights, architectures, derived representations, or any models fine-tuned from this project in publications, preprints, theses, presentations, software repositories, benchmarks, or derivative analyses.

Use of this model for commercial purposes is not permitted without prior written permission from the authors. Any redistributed or modified version of these weights or derived model variants must preserve attribution to the original authors and must be shared under the same or a compatible non-commercial share-alike license, unless separate written permission is obtained from the authors.

Users must not claim ownership of the original architectures, pretrained weights, Hugging Face model versions, or associated derived resources.

**Please cite:**

```bibtex
@article {Deng2026.05.26.727755,
	author = {Deng, Nianqi and Miao, Guillaume and Khadra, Anmar and Peterson, Alan C and Bagheri, Hooman},
	title = {Deep learning-based decoding of axonal ultrastructure in gene-edited mice using electron microscopy imaging},
	elocation-id = {2026.05.26.727755},
	year = {2026},
	doi = {10.64898/2026.05.26.727755},
	publisher = {Cold Spring Harbor Laboratory},
	URL = {https://www.biorxiv.org/content/early/2026/05/26/2026.05.26.727755},
	eprint = {https://www.biorxiv.org/content/early/2026/05/26/2026.05.26.727755.full.pdf},
	journal = {bioRxiv}
}
```