Trans4PASS & Trans4PASS+: Transformers for Panoramic Semantic Segmentation

Official Model Checkpoints Repository for Trans4PASS and Trans4PASS+.

• GitHub Repository: https://github.com/InSAI-Lab/Trans4PASS
• Paper (Trans4PASS, CVPR 2022): arXiv:2203.01452
• Paper (Trans4PASS+, arXiv 2022): arXiv:2207.11860
• Dataset Repository: InSAI-Lab/SynPASS

---

Checkpoints Organization

This repository contains all pretrained backbones, supervised weights, and domain adaptation checkpoints matching the Trans4PASS project directory structure:

├── pretrained/
│   ├── mit_b1.pth
│   └── mit_b2.pth
├── adaptations/
│   └── snapshots/
│       ├── CS2DensePASS_Trans4PASS_v1_MPA/BestCS2DensePASS_G.pth
│       ├── CS2DensePASS_Trans4PASS_v2_MPA/BestCS2DensePASS_G.pth
│       ├── CS132CS132DP13_Trans4PASS_plus_v2_MPA/BestCS132DP13_G.pth
│       ├── CS2DP_Trans4PASS_plus_v1_MPA/BestCS2DensePASS_G.pth
│       └── CS2DP_Trans4PASS_plus_v2_MPA/BestCS2DensePASS_G.pth
└── workdirs/
    ├── cityscapes/ (trans4pass & plus, tiny & small)
    ├── cityscapes13/ (trans4pass & plus, tiny & small)
    ├── stanford2d3d/ (trans4pass & plus, tiny & small)
    ├── stanford2d3d8/ (trans4pass & plus, tiny & small)
    ├── stanford2d3d_pan/ (folds F1, F2, F3)
    ├── structured3d8/ (trans4pass & plus, tiny & small)
    ├── synpass/ (trans4pass & plus, tiny & small)
    └── synpass13/ (trans4pass & plus, tiny & small)

Benchmark Results

Cityscapes -> DensePASS (Domain Adaptation)

Model	CS mIoU (%)	DP mIoU (%)
Trans4PASS (Tiny)	72.49	45.89
Trans4PASS (Small)	72.84	51.38
Trans4PASS+ (Tiny)	72.67	50.23
Trans4PASS+ (Small)	75.24	51.41

SynPASS Benchmark

Model	Val mIoU (%)	Test mIoU (%)
Trans4PASS (Tiny)	43.68	38.53
Trans4PASS (Small)	44.80	38.57
Trans4PASS+ (Tiny)	45.21	38.85
Trans4PASS+ (Small)	46.47	39.16

Download & Usage

Using hf:

# Download all models to the local directory
hf download InSAI-Lab/Trans4PASS --local-dir .

Evaluation example:

# In Trans4PASS repository
python tools/eval_sp.py --config-file configs/synpass/trans4pass_plus_tiny_512x512.yaml

Citation

@inproceedings{zhang2022bending,
  title={Bending Reality: Distortion-aware Transformers for Adapting to Panoramic Semantic Segmentation},
  author={Zhang, Jiaming and Yang, Kailun and Ma, Chaoxiang and Rei{\ss}, Simon and Peng, Kunyu and Stiefelhagen, Rainer},
  booktitle={CVPR},
  pages={16917--16927},
  year={2022}
}

@article{zhang2022behind,
  title={Behind Every Domain There is a Shift: Adapting Distortion-aware Vision Transformers for Panoramic Semantic Segmentation},
  author={Zhang, Jiaming and Yang, Kailun and Shi, Hao and Rei{\ss}, Simon and Peng, Kunyu and Ma, Chaoxiang and Fu, Haodong and Wang, Kaiwei and Stiefelhagen, Rainer},
  journal={arXiv preprint arXiv:2207.11860},
  year={2022}
}