Hugging Face's logo

InSAI-Lab
/
Trans4PASS

Image Segmentation
PyTorch
panoramic
vision-transformer
semantic-segmentation
domain-adaptation
trans4pass
trans4pass-plus
cvpr-2022
Model card Files
xet
 Community
Trans4PASS
File size: 3,487 Bytes
5bf049c
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
---
license: apache-2.0
library_name: pytorch
tags:
- panoramic
- vision-transformer
- semantic-segmentation
- domain-adaptation
- trans4pass
- trans4pass-plus
- cvpr-2022
pipeline_tag: image-segmentation
---

# Trans4PASS & Trans4PASS+: Transformers for Panoramic Semantic Segmentation

Official Model Checkpoints Repository for **Trans4PASS** and **Trans4PASS+**.

- **GitHub Repository:** https://github.com/InSAI-Lab/Trans4PASS
- **Paper (Trans4PASS, CVPR 2022):** [arXiv:2203.01452](https://arxiv.org/abs/2203.01452)
- **Paper (Trans4PASS+, arXiv 2022):** [arXiv:2207.11860](https://arxiv.org/abs/2207.11860)
- **Dataset Repository:** [InSAI-Lab/SynPASS](https://huggingface.co/datasets/InSAI-Lab/SynPASS)

---

## Checkpoints Organization

This repository contains all pretrained backbones, supervised weights, and domain adaptation checkpoints matching the Trans4PASS project directory structure:

```text
├── pretrained/
│   ├── mit_b1.pth
│   └── mit_b2.pth
├── adaptations/
│   └── snapshots/
│       ├── CS2DensePASS_Trans4PASS_v1_MPA/BestCS2DensePASS_G.pth
│       ├── CS2DensePASS_Trans4PASS_v2_MPA/BestCS2DensePASS_G.pth
│       ├── CS132CS132DP13_Trans4PASS_plus_v2_MPA/BestCS132DP13_G.pth
│       ├── CS2DP_Trans4PASS_plus_v1_MPA/BestCS2DensePASS_G.pth
│       └── CS2DP_Trans4PASS_plus_v2_MPA/BestCS2DensePASS_G.pth
└── workdirs/
    ├── cityscapes/ (trans4pass & plus, tiny & small)
    ├── cityscapes13/ (trans4pass & plus, tiny & small)
    ├── stanford2d3d/ (trans4pass & plus, tiny & small)
    ├── stanford2d3d8/ (trans4pass & plus, tiny & small)
    ├── stanford2d3d_pan/ (folds F1, F2, F3)
    ├── structured3d8/ (trans4pass & plus, tiny & small)
    ├── synpass/ (trans4pass & plus, tiny & small)
    └── synpass13/ (trans4pass & plus, tiny & small)
```

## Benchmark Results

### Cityscapes -> DensePASS (Domain Adaptation)
| Model | CS mIoU (%) | DP mIoU (%) |
| :--- | :---: | :---: |
| Trans4PASS (Tiny) | 72.49 | 45.89 |
| Trans4PASS (Small) | 72.84 | 51.38 |
| Trans4PASS+ (Tiny) | 72.67 | 50.23 |
| Trans4PASS+ (Small) | **75.24** | **51.41** |

### SynPASS Benchmark
| Model | Val mIoU (%) | Test mIoU (%) |
| :--- | :---: | :---: |
| Trans4PASS (Tiny) | 43.68 | 38.53 |
| Trans4PASS (Small) | 44.80 | 38.57 |
| Trans4PASS+ (Tiny) | 45.21 | 38.85 |
| Trans4PASS+ (Small) | **46.47** | **39.16** |

## Download & Usage

Using `hf`:

```bash
# Download all models to the local directory
hf download InSAI-Lab/Trans4PASS --local-dir .
```

Evaluation example:

```bash
# In Trans4PASS repository
python tools/eval_sp.py --config-file configs/synpass/trans4pass_plus_tiny_512x512.yaml
```

## Citation

```bibtex
@inproceedings{zhang2022bending,
  title={Bending Reality: Distortion-aware Transformers for Adapting to Panoramic Semantic Segmentation},
  author={Zhang, Jiaming and Yang, Kailun and Ma, Chaoxiang and Rei{\ss}, Simon and Peng, Kunyu and Stiefelhagen, Rainer},
  booktitle={CVPR},
  pages={16917--16927},
  year={2022}
}

@article{zhang2022behind,
  title={Behind Every Domain There is a Shift: Adapting Distortion-aware Vision Transformers for Panoramic Semantic Segmentation},
  author={Zhang, Jiaming and Yang, Kailun and Shi, Hao and Rei{\ss}, Simon and Peng, Kunyu and Ma, Chaoxiang and Fu, Haodong and Wang, Kaiwei and Stiefelhagen, Rainer},
  journal={arXiv preprint arXiv:2207.11860},
  year={2022}
}
```