File size: 10,710 Bytes
692f33c a1c1f0c 08bf2f6 a1c1f0c 692f33c a1c1f0c 08bf2f6 39d6c29 08bf2f6 a1c1f0c 394a649 66aff93 394a649 66aff93 394a649 692f33c 00988f1 5f80394 39b3761 22ebc38 b4ff3a2 22ebc38 b4ff3a2 22ebc38 b4ff3a2 22ebc38 b4ff3a2 22ebc38 b4ff3a2 22ebc38 39b3761 22ebc38 b4ff3a2 22ebc38 b4ff3a2 22ebc38 b4ff3a2 22ebc38 39b3761 aeb9a89 39b3761 8b2be07 22ebc38 39b3761 8b2be07 39b3761 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 | ---
tags:
- pemola
- panoptic-segmentation
- occlusion
- icme-2026
datasets:
- weiwb/COCO-OLAC
- weiwb/Cityscapes-OLAC
---
# PEMOLA (ICME 2026)
Official checkpoints for **[Occlusion-Aware Panoptic Segmentation with Joint Position Embedding and Occlusion-Level Attention](https://huggingface.co/papers/2607.18112)** (ICME 2026).
## Related resources
**Research overview:** [Occlusion-Aware Visual Understanding](https://huggingface.co/collections/weiwb/occlusion-aware-visual-understanding-6ac212311f2153fa719c9f95). Explore the papers, datasets, model checkpoints, and how they relate.
| Resource | What it provides |
|:---------|:-----------------|
| [COCO-OLAC dataset](https://huggingface.co/datasets/weiwb/COCO-OLAC) | Image-level occlusion annotations for 30,000 COCO training images and 5,000 validation images, including validation subsets by occlusion level. |
| [Cityscapes-OLAC dataset](https://huggingface.co/datasets/weiwb/Cityscapes-OLAC) | Image-level occlusion annotations for 2,975 Cityscapes training images and 500 validation images, following the same annotation protocol. |
Both dataset repositories provide additional occlusion annotations. Obtain the original images and segmentation annotations separately from the [official COCO website](https://cocodataset.org/#download) and the [official Cityscapes website](https://www.cityscapes-dataset.com/downloads/).
## Download
Download all models from the PEMOLA source repository root:
```bash
python -m pip install huggingface_hub
hf download weiwb/PEMOLA --local-dir checkpoints
```
To download one model, specify its checkpoint filename:
```bash
hf download weiwb/PEMOLA mask2former_pemola_coco_olac.pth --local-dir checkpoints
```
For source code, [installation](https://github.com/wenbo-wei/PEMOLA#installation), and [inference](https://github.com/wenbo-wei/PEMOLA#inference--visualization), see the [PEMOLA GitHub repository](https://github.com/wenbo-wei/PEMOLA).
### Panoptic Segmentation on COCO-OLAC
<table>
<thead>
<tr>
<th align="left" valign="middle" style="text-align: left; vertical-align: middle;">Method</th>
<th align="left" valign="middle" style="text-align: left; vertical-align: middle;">Backbone</th>
<th align="center" valign="middle" style="text-align: center; vertical-align: middle;">PQ</th>
<th align="center" valign="middle" style="text-align: center; vertical-align: middle;">PQ<sup>Th</sup></th>
<th align="center" valign="middle" style="text-align: center; vertical-align: middle;">PQ<sup>St</sup></th>
<th align="center" valign="middle" style="text-align: center; vertical-align: middle;">AP<sup>Th</sup><sub>pan</sub></th>
<th align="center" valign="middle" style="text-align: center; vertical-align: middle;">mIoU<sub>pan</sub></th>
<th align="center" valign="middle" style="text-align: center; vertical-align: middle;">Weights</th>
</tr>
</thead>
<tbody>
<tr>
<td align="left" valign="middle" style="text-align: left; vertical-align: middle;">Mask2Former</td>
<td align="left" valign="middle" style="text-align: left; vertical-align: middle;">ResNet-50</td>
<td align="center" valign="middle" style="text-align: center; vertical-align: middle;">40.7</td>
<td align="center" valign="middle" style="text-align: center; vertical-align: middle;">44.5</td>
<td align="center" valign="middle" style="text-align: center; vertical-align: middle;">35.0</td>
<td align="center" valign="middle" style="text-align: center; vertical-align: middle;">30.0</td>
<td align="center" valign="middle" style="text-align: center; vertical-align: middle;">54.2</td>
<td align="center" valign="middle" style="text-align: center; vertical-align: middle;"><a href="https://huggingface.co/weiwb/PEMOLA/resolve/main/mask2former_coco_olac.pth">download</a></td>
</tr>
<tr>
<td align="left" valign="middle" style="text-align: left; vertical-align: middle;"><strong>+ PEMOLA</strong></td>
<td align="left" valign="middle" style="text-align: left; vertical-align: middle;">ResNet-50</td>
<td align="center" valign="middle" style="text-align: center; vertical-align: middle;"><strong>41.5</strong></td>
<td align="center" valign="middle" style="text-align: center; vertical-align: middle;"><strong>45.2</strong></td>
<td align="center" valign="middle" style="text-align: center; vertical-align: middle;"><strong>35.9</strong></td>
<td align="center" valign="middle" style="text-align: center; vertical-align: middle;"><strong>30.4</strong></td>
<td align="center" valign="middle" style="text-align: center; vertical-align: middle;"><strong>54.8</strong></td>
<td align="center" valign="middle" style="text-align: center; vertical-align: middle;"><a href="https://huggingface.co/weiwb/PEMOLA/resolve/main/mask2former_pemola_coco_olac.pth">download</a></td>
</tr>
<tr>
<td align="left" valign="middle" style="text-align: left; vertical-align: middle;">Mask DINO</td>
<td align="left" valign="middle" style="text-align: left; vertical-align: middle;">ResNet-50</td>
<td align="center" valign="middle" style="text-align: center; vertical-align: middle;">44.0</td>
<td align="center" valign="middle" style="text-align: center; vertical-align: middle;">48.5</td>
<td align="center" valign="middle" style="text-align: center; vertical-align: middle;">37.3</td>
<td align="center" valign="middle" style="text-align: center; vertical-align: middle;">33.5</td>
<td align="center" valign="middle" style="text-align: center; vertical-align: middle;">53.4</td>
<td align="center" valign="middle" style="text-align: center; vertical-align: middle;"><a href="https://huggingface.co/weiwb/PEMOLA/resolve/main/maskdino_coco_olac.pth">download</a></td>
</tr>
<tr>
<td align="left" valign="middle" style="text-align: left; vertical-align: middle;"><strong>+ PEMOLA</strong></td>
<td align="left" valign="middle" style="text-align: left; vertical-align: middle;">ResNet-50</td>
<td align="center" valign="middle" style="text-align: center; vertical-align: middle;"><strong>44.8</strong></td>
<td align="center" valign="middle" style="text-align: center; vertical-align: middle;"><strong>49.4</strong></td>
<td align="center" valign="middle" style="text-align: center; vertical-align: middle;"><strong>37.8</strong></td>
<td align="center" valign="middle" style="text-align: center; vertical-align: middle;"><strong>34.2</strong></td>
<td align="center" valign="middle" style="text-align: center; vertical-align: middle;"><strong>55.3</strong></td>
<td align="center" valign="middle" style="text-align: center; vertical-align: middle;"><a href="https://huggingface.co/weiwb/PEMOLA/resolve/main/maskdino_pemola_coco_olac.pth">download</a></td>
</tr>
</tbody>
</table>
### Panoptic Segmentation on Cityscapes-OLAC
<table>
<thead>
<tr>
<th align="left" valign="middle" style="text-align: left; vertical-align: middle;">Method</th>
<th align="left" valign="middle" style="text-align: left; vertical-align: middle;">Backbone</th>
<th align="center" valign="middle" style="text-align: center; vertical-align: middle;">PQ</th>
<th align="center" valign="middle" style="text-align: center; vertical-align: middle;">PQ<sup>Th</sup></th>
<th align="center" valign="middle" style="text-align: center; vertical-align: middle;">PQ<sup>St</sup></th>
<th align="center" valign="middle" style="text-align: center; vertical-align: middle;">AP<sup>Th</sup><sub>pan</sub></th>
<th align="center" valign="middle" style="text-align: center; vertical-align: middle;">mIoU<sub>pan</sub></th>
<th align="center" valign="middle" style="text-align: center; vertical-align: middle;">Weights</th>
</tr>
</thead>
<tbody>
<tr>
<td align="left" valign="middle" style="text-align: left; vertical-align: middle;">Mask2Former</td>
<td align="left" valign="middle" style="text-align: left; vertical-align: middle;">ResNet-50</td>
<td align="center" valign="middle" style="text-align: center; vertical-align: middle;">61.5</td>
<td align="center" valign="middle" style="text-align: center; vertical-align: middle;">54.0</td>
<td align="center" valign="middle" style="text-align: center; vertical-align: middle;">66.9</td>
<td align="center" valign="middle" style="text-align: center; vertical-align: middle;">35.2</td>
<td align="center" valign="middle" style="text-align: center; vertical-align: middle;">76.1</td>
<td align="center" valign="middle" style="text-align: center; vertical-align: middle;"><a href="https://huggingface.co/weiwb/PEMOLA/resolve/main/mask2former_cityscapes_olac.pth">download</a></td>
</tr>
<tr>
<td align="left" valign="middle" style="text-align: left; vertical-align: middle;"><strong>+ PEMOLA</strong></td>
<td align="left" valign="middle" style="text-align: left; vertical-align: middle;">ResNet-50</td>
<td align="center" valign="middle" style="text-align: center; vertical-align: middle;"><strong>62.3</strong></td>
<td align="center" valign="middle" style="text-align: center; vertical-align: middle;"><strong>55.4</strong></td>
<td align="center" valign="middle" style="text-align: center; vertical-align: middle;"><strong>67.2</strong></td>
<td align="center" valign="middle" style="text-align: center; vertical-align: middle;"><strong>38.5</strong></td>
<td align="center" valign="middle" style="text-align: center; vertical-align: middle;"><strong>77.4</strong></td>
<td align="center" valign="middle" style="text-align: center; vertical-align: middle;"><a href="https://huggingface.co/weiwb/PEMOLA/resolve/main/mask2former_pemola_cityscapes_olac.pth">download</a></td>
</tr>
</tbody>
</table>
### Occlusion Classifier
Top-1 accuracies (%) reported in the paper on the COCO-OLAC test split with background-blackened inputs, for occlusion classification.
<table>
<thead>
<tr>
<th align="left">Backbone</th>
<th align="center">Pretraining</th>
<th align="center" valign="middle" style="text-align: center; vertical-align: middle;">Input</th>
<th align="center" valign="middle" style="text-align: center; vertical-align: middle;">Top-1 Acc (%)</th>
<th align="center" valign="middle" style="text-align: center; vertical-align: middle;">Weights</th>
</tr>
</thead>
<tbody>
<tr><td align="left"><b>Swin-L</b></td> <td align="center">ImageNet-22K</td> <td align="center" valign="middle" style="text-align: center; vertical-align: middle;">384</td> <td align="center" valign="middle" style="text-align: center; vertical-align: middle;"><b>75.3</b></td> <td align="center" valign="middle" style="text-align: center; vertical-align: middle;"><a href="https://huggingface.co/weiwb/PEMOLA/resolve/main/occl_classifier_swin_large_window12_384_coco_olac.pth">download</a></td></tr>
</tbody>
</table>
|