Image Segmentation
Transformers
Safetensors
English
mask2former
Electron-Microscopy
Ultrastructure-Segmentation
Semantic-Segmentation
Instructions to use Dnq2025/MicroStructFormer with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use Dnq2025/MicroStructFormer with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("image-segmentation", model="Dnq2025/MicroStructFormer")# pip install -U transformers accelerate # Load model directly from transformers import AutoImageProcessor, Mask2FormerForUniversalSegmentation processor = AutoImageProcessor.from_pretrained("Dnq2025/MicroStructFormer") model = Mask2FormerForUniversalSegmentation.from_pretrained("Dnq2025/MicroStructFormer", device_map="auto") - Notebooks
- Google Colab
- Kaggle
File size: 4,641 Bytes
75d75e4 e94e0a3 9fc3bc2 e94e0a3 9fc3bc2 7a37aaf 9fc3bc2 7a37aaf f29267a 9fc3bc2 122a1d6 9fc3bc2 122a1d6 9fc3bc2 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 | ---
license: cc-by-nc-sa-4.0
datasets:
- Dnq2025/Mask2former_Finetune
base_model:
- facebook/mask2former-swin-large-ade-semantic
- Dnq2025/swin-large-mask2former-finetuned-ER-Mito-LD8
language:
- en
pipeline_tag: image-segmentation
library_name: transformers
tags:
- Electron-Microscopy
- Ultrastructure-Segmentation
- Semantic-Segmentation
---
# MicroStructFormer (Swin-Large)
MicroStructFormer is a high-capacity Mask2Former model built on a Swin-Large backbone, designed for the semantic segmentation of ultrastructures in Electron Microscopy (EM) images.
## Model Description
This model has been trained to accurately segment the following critical ultrastructural elements:
- Abnormalities
- Mitochondria (and mitochondria-like organelles)
- Periaxonal Space
*Note: While the model architecture theoretically supports axon and myelin segmentation, this specific release is optimized and calibrated for the classes listed below due to dataset annotations.*
## Usage
You can use this model easily via the `transformers` library.
### 1. Using Pipeline (Recommended)
```python
from transformers import pipeline
from PIL import Image
# Initialize the segmentation pipeline
pipe = pipeline("image-segmentation", model="Dnq2025/MicroStructFormer")
# Load your EM image
image = Image.open("examples/example_tem.png").convert("RGB")
# Run inference
results = pipe(image)
print(results)
```
### 2. Manual Inference with AutoImageProcessor
```python
import torch
import numpy as np
from PIL import Image
from transformers import AutoImageProcessor, Mask2FormerForUniversalSegmentation
repo_id = "Dnq2025/MicroStructFormer"
# Load processor and model
processor = AutoImageProcessor.from_pretrained(repo_id)
model = Mask2FormerForUniversalSegmentation.from_pretrained(repo_id)
model.eval()
# Load image
image = Image.open("examples/example_tem.png").convert("RGB")
inputs = processor(images=image, return_tensors="pt")
with torch.no_grad():
outputs = model(**inputs)
# Post-process to get the semantic segmentation map
pred = processor.post_process_semantic_segmentation(
outputs,
target_sizes=[image.size[::-1]]
)[0].cpu().numpy().astype(np.uint8)
print(f"Prediction shape: {pred.shape}")
print(f"Unique classes found: {np.unique(pred)}")
```
## Label Map
| ID | Class |
|---:|---|
| 0 | background |
| 1 | abnormality |
| 2 | mitochondria-like organelles |
| 3 | mitochondria |
| 4 | mitochondria outside axon |
| 5 | periaxonal space |
## Example
**Input EM image:**

**Predicted segmentation overlay:**

## Model Use, License, and Citation
This Hugging Face model repository is associated with the following bioRxiv preprint:
> Deng N, Miao G, Khadra A, Peterson AC, Bagheri H. Deep learning-based decoding of axonal ultrastructure in gene-edited mice using electron microscopy imaging. bioRxiv. 2026. DOI: 10.64898/2026.05.26.727755.
These model weights and associated architectures are provided for non-commercial academic and research use under the **CC-BY-NC-SA 4.0** license selected for this Hugging Face repository. Users must provide appropriate attribution to the original authors and cite the associated preprint when using these weights, architectures, derived representations, or any models fine-tuned from this project in publications, preprints, theses, presentations, software repositories, benchmarks, or derivative analyses.
Use of this model for commercial purposes is not permitted without prior written permission from the authors. Any redistributed or modified version of these weights or derived model variants must preserve attribution to the original authors and must be shared under the same or a compatible non-commercial share-alike license, unless separate written permission is obtained from the authors.
Users must not claim ownership of the original architectures, pretrained weights, Hugging Face model versions, or associated derived resources.
**Please cite:**
```bibtex
@article {Deng2026.05.26.727755,
author = {Deng, Nianqi and Miao, Guillaume and Khadra, Anmar and Peterson, Alan C and Bagheri, Hooman},
title = {Deep learning-based decoding of axonal ultrastructure in gene-edited mice using electron microscopy imaging},
elocation-id = {2026.05.26.727755},
year = {2026},
doi = {10.64898/2026.05.26.727755},
publisher = {Cold Spring Harbor Laboratory},
URL = {https://www.biorxiv.org/content/early/2026/05/26/2026.05.26.727755},
eprint = {https://www.biorxiv.org/content/early/2026/05/26/2026.05.26.727755.full.pdf},
journal = {bioRxiv}
}
```
|