| --- |
| tags: |
| - pemola |
| - panoptic-segmentation |
| - occlusion |
| - icme-2026 |
| datasets: |
| - weiwb/COCO-OLAC |
| - weiwb/Cityscapes-OLAC |
| --- |
| |
| # PEMOLA (ICME 2026) |
|
|
| Official checkpoints for **[Occlusion-Aware Panoptic Segmentation with Joint Position Embedding and Occlusion-Level Attention](https://huggingface.co/papers/2607.18112)** (ICME 2026). |
|
|
| ## Related resources |
|
|
| **Research overview:** [Occlusion-Aware Visual Understanding](https://huggingface.co/collections/weiwb/occlusion-aware-visual-understanding-6ac212311f2153fa719c9f95). Explore the papers, datasets, model checkpoints, and how they relate. |
|
|
| | Resource | What it provides | |
| |:---------|:-----------------| |
| | [COCO-OLAC dataset](https://huggingface.co/datasets/weiwb/COCO-OLAC) | Image-level occlusion annotations for 30,000 COCO training images and 5,000 validation images, including validation subsets by occlusion level. | |
| | [Cityscapes-OLAC dataset](https://huggingface.co/datasets/weiwb/Cityscapes-OLAC) | Image-level occlusion annotations for 2,975 Cityscapes training images and 500 validation images, following the same annotation protocol. | |
|
|
| Both dataset repositories provide additional occlusion annotations. Obtain the original images and segmentation annotations separately from the [official COCO website](https://cocodataset.org/#download) and the [official Cityscapes website](https://www.cityscapes-dataset.com/downloads/). |
|
|
| ## Download |
|
|
| Download all models from the PEMOLA source repository root: |
|
|
| ```bash |
| python -m pip install huggingface_hub |
| hf download weiwb/PEMOLA --local-dir checkpoints |
| ``` |
|
|
| To download one model, specify its checkpoint filename: |
|
|
| ```bash |
| hf download weiwb/PEMOLA mask2former_pemola_coco_olac.pth --local-dir checkpoints |
| ``` |
|
|
| For source code, [installation](https://github.com/wenbo-wei/PEMOLA#installation), and [inference](https://github.com/wenbo-wei/PEMOLA#inference--visualization), see the [PEMOLA GitHub repository](https://github.com/wenbo-wei/PEMOLA). |
|
|
| ### Panoptic Segmentation on COCO-OLAC |
|
|
| <table> |
| <thead> |
| <tr> |
| <th align="left" valign="middle" style="text-align: left; vertical-align: middle;">Method</th> |
| <th align="left" valign="middle" style="text-align: left; vertical-align: middle;">Backbone</th> |
| <th align="center" valign="middle" style="text-align: center; vertical-align: middle;">PQ</th> |
| <th align="center" valign="middle" style="text-align: center; vertical-align: middle;">PQ<sup>Th</sup></th> |
| <th align="center" valign="middle" style="text-align: center; vertical-align: middle;">PQ<sup>St</sup></th> |
| <th align="center" valign="middle" style="text-align: center; vertical-align: middle;">AP<sup>Th</sup><sub>pan</sub></th> |
| <th align="center" valign="middle" style="text-align: center; vertical-align: middle;">mIoU<sub>pan</sub></th> |
| <th align="center" valign="middle" style="text-align: center; vertical-align: middle;">Weights</th> |
| </tr> |
| </thead> |
| <tbody> |
| <tr> |
| <td align="left" valign="middle" style="text-align: left; vertical-align: middle;">Mask2Former</td> |
| <td align="left" valign="middle" style="text-align: left; vertical-align: middle;">ResNet-50</td> |
| <td align="center" valign="middle" style="text-align: center; vertical-align: middle;">40.7</td> |
| <td align="center" valign="middle" style="text-align: center; vertical-align: middle;">44.5</td> |
| <td align="center" valign="middle" style="text-align: center; vertical-align: middle;">35.0</td> |
| <td align="center" valign="middle" style="text-align: center; vertical-align: middle;">30.0</td> |
| <td align="center" valign="middle" style="text-align: center; vertical-align: middle;">54.2</td> |
| <td align="center" valign="middle" style="text-align: center; vertical-align: middle;"><a href="https://huggingface.co/weiwb/PEMOLA/resolve/main/mask2former_coco_olac.pth">download</a></td> |
| </tr> |
| <tr> |
| <td align="left" valign="middle" style="text-align: left; vertical-align: middle;"><strong>+ PEMOLA</strong></td> |
| <td align="left" valign="middle" style="text-align: left; vertical-align: middle;">ResNet-50</td> |
| <td align="center" valign="middle" style="text-align: center; vertical-align: middle;"><strong>41.5</strong></td> |
| <td align="center" valign="middle" style="text-align: center; vertical-align: middle;"><strong>45.2</strong></td> |
| <td align="center" valign="middle" style="text-align: center; vertical-align: middle;"><strong>35.9</strong></td> |
| <td align="center" valign="middle" style="text-align: center; vertical-align: middle;"><strong>30.4</strong></td> |
| <td align="center" valign="middle" style="text-align: center; vertical-align: middle;"><strong>54.8</strong></td> |
| <td align="center" valign="middle" style="text-align: center; vertical-align: middle;"><a href="https://huggingface.co/weiwb/PEMOLA/resolve/main/mask2former_pemola_coco_olac.pth">download</a></td> |
| </tr> |
| <tr> |
| <td align="left" valign="middle" style="text-align: left; vertical-align: middle;">Mask DINO</td> |
| <td align="left" valign="middle" style="text-align: left; vertical-align: middle;">ResNet-50</td> |
| <td align="center" valign="middle" style="text-align: center; vertical-align: middle;">44.0</td> |
| <td align="center" valign="middle" style="text-align: center; vertical-align: middle;">48.5</td> |
| <td align="center" valign="middle" style="text-align: center; vertical-align: middle;">37.3</td> |
| <td align="center" valign="middle" style="text-align: center; vertical-align: middle;">33.5</td> |
| <td align="center" valign="middle" style="text-align: center; vertical-align: middle;">53.4</td> |
| <td align="center" valign="middle" style="text-align: center; vertical-align: middle;"><a href="https://huggingface.co/weiwb/PEMOLA/resolve/main/maskdino_coco_olac.pth">download</a></td> |
| </tr> |
| <tr> |
| <td align="left" valign="middle" style="text-align: left; vertical-align: middle;"><strong>+ PEMOLA</strong></td> |
| <td align="left" valign="middle" style="text-align: left; vertical-align: middle;">ResNet-50</td> |
| <td align="center" valign="middle" style="text-align: center; vertical-align: middle;"><strong>44.8</strong></td> |
| <td align="center" valign="middle" style="text-align: center; vertical-align: middle;"><strong>49.4</strong></td> |
| <td align="center" valign="middle" style="text-align: center; vertical-align: middle;"><strong>37.8</strong></td> |
| <td align="center" valign="middle" style="text-align: center; vertical-align: middle;"><strong>34.2</strong></td> |
| <td align="center" valign="middle" style="text-align: center; vertical-align: middle;"><strong>55.3</strong></td> |
| <td align="center" valign="middle" style="text-align: center; vertical-align: middle;"><a href="https://huggingface.co/weiwb/PEMOLA/resolve/main/maskdino_pemola_coco_olac.pth">download</a></td> |
| </tr> |
| </tbody> |
| </table> |
|
|
| ### Panoptic Segmentation on Cityscapes-OLAC |
|
|
| <table> |
| <thead> |
| <tr> |
| <th align="left" valign="middle" style="text-align: left; vertical-align: middle;">Method</th> |
| <th align="left" valign="middle" style="text-align: left; vertical-align: middle;">Backbone</th> |
| <th align="center" valign="middle" style="text-align: center; vertical-align: middle;">PQ</th> |
| <th align="center" valign="middle" style="text-align: center; vertical-align: middle;">PQ<sup>Th</sup></th> |
| <th align="center" valign="middle" style="text-align: center; vertical-align: middle;">PQ<sup>St</sup></th> |
| <th align="center" valign="middle" style="text-align: center; vertical-align: middle;">AP<sup>Th</sup><sub>pan</sub></th> |
| <th align="center" valign="middle" style="text-align: center; vertical-align: middle;">mIoU<sub>pan</sub></th> |
| <th align="center" valign="middle" style="text-align: center; vertical-align: middle;">Weights</th> |
| </tr> |
| </thead> |
| <tbody> |
| <tr> |
| <td align="left" valign="middle" style="text-align: left; vertical-align: middle;">Mask2Former</td> |
| <td align="left" valign="middle" style="text-align: left; vertical-align: middle;">ResNet-50</td> |
| <td align="center" valign="middle" style="text-align: center; vertical-align: middle;">61.5</td> |
| <td align="center" valign="middle" style="text-align: center; vertical-align: middle;">54.0</td> |
| <td align="center" valign="middle" style="text-align: center; vertical-align: middle;">66.9</td> |
| <td align="center" valign="middle" style="text-align: center; vertical-align: middle;">35.2</td> |
| <td align="center" valign="middle" style="text-align: center; vertical-align: middle;">76.1</td> |
| <td align="center" valign="middle" style="text-align: center; vertical-align: middle;"><a href="https://huggingface.co/weiwb/PEMOLA/resolve/main/mask2former_cityscapes_olac.pth">download</a></td> |
| </tr> |
| <tr> |
| <td align="left" valign="middle" style="text-align: left; vertical-align: middle;"><strong>+ PEMOLA</strong></td> |
| <td align="left" valign="middle" style="text-align: left; vertical-align: middle;">ResNet-50</td> |
| <td align="center" valign="middle" style="text-align: center; vertical-align: middle;"><strong>62.3</strong></td> |
| <td align="center" valign="middle" style="text-align: center; vertical-align: middle;"><strong>55.4</strong></td> |
| <td align="center" valign="middle" style="text-align: center; vertical-align: middle;"><strong>67.2</strong></td> |
| <td align="center" valign="middle" style="text-align: center; vertical-align: middle;"><strong>38.5</strong></td> |
| <td align="center" valign="middle" style="text-align: center; vertical-align: middle;"><strong>77.4</strong></td> |
| <td align="center" valign="middle" style="text-align: center; vertical-align: middle;"><a href="https://huggingface.co/weiwb/PEMOLA/resolve/main/mask2former_pemola_cityscapes_olac.pth">download</a></td> |
| </tr> |
| </tbody> |
| </table> |
|
|
| ### Occlusion Classifier |
|
|
| Top-1 accuracies (%) reported in the paper on the COCO-OLAC test split with background-blackened inputs, for occlusion classification. |
|
|
| <table> |
| <thead> |
| <tr> |
| <th align="left">Backbone</th> |
| <th align="center">Pretraining</th> |
| <th align="center" valign="middle" style="text-align: center; vertical-align: middle;">Input</th> |
| <th align="center" valign="middle" style="text-align: center; vertical-align: middle;">Top-1 Acc (%)</th> |
| <th align="center" valign="middle" style="text-align: center; vertical-align: middle;">Weights</th> |
| </tr> |
| </thead> |
| <tbody> |
| <tr><td align="left"><b>Swin-L</b></td> <td align="center">ImageNet-22K</td> <td align="center" valign="middle" style="text-align: center; vertical-align: middle;">384</td> <td align="center" valign="middle" style="text-align: center; vertical-align: middle;"><b>75.3</b></td> <td align="center" valign="middle" style="text-align: center; vertical-align: middle;"><a href="https://huggingface.co/weiwb/PEMOLA/resolve/main/occl_classifier_swin_large_window12_384_coco_olac.pth">download</a></td></tr> |
| </tbody> |
| </table> |
|
|
