Trans4Trans: Efficient Transformer for Transparent Object and Semantic Scene Segmentation

Official model checkpoints for Trans4Trans: Efficient Transformer for Transparent Object Segmentation to Help Visually Impaired People Navigate in the Real World (ICCVW 2021) and Trans4Trans: Efficient Transformer for Transparent Object and Semantic Scene Segmentation in Real-World Navigation Assistance (IEEE T-ITS 2022).

---

📌 Introduction

Trans4Trans is an efficient Transformer-based system designed for transparent object and semantic scene segmentation, targeting real-world wearable navigation assistance for visually impaired people. The architecture couples a lightweight hierarchical transformer backbone with a Frequency-aware Parsing Transformer (FPT) decoder and an assistive system integrating Intel RealSense depth sensing and acoustic feedback.

---

📂 Model Checkpoints

The repository contains pretrained weights organized by dataset benchmark:

InSAI-Lab/Trans4trans
├── cocostuff/
│   └── model_cocostuff.pth      # General scene context (COCO-Stuff, 162.6 MB)
└── trans10kv2/
    └── model_trans.pth          # Transparent object segmentation (Trans10K-v2, 150.3 MB)

Checkpoint	Target Task / Dataset	Size	Direct Download
cocostuff/model_cocostuff.pth	COCO-Stuff General Context	162.6 MB	Download
trans10kv2/model_trans.pth	Trans10K-v2 Transparent Objects	150.3 MB	Download

---

🚀 Quick Start

Download via Hugging Face CLI

# Download all checkpoints into workdirs/
hf download InSAI-Lab/Trans4trans --local-dir workdirs

Download via Python (huggingface_hub)

from huggingface_hub import hf_hub_download

# Download transparent object segmentation model
ckpt_path = hf_hub_download(
    repo_id="InSAI-Lab/Trans4trans",
    filename="trans10kv2/model_trans.pth",
    local_dir="workdirs"
)
print(f"Downloaded model to {ckpt_path}")

Assistive System Demo

# Save weights under workdirs/cocostuff and workdirs/trans10kv2
bash demo.sh

---

📑 Citation

@inproceedings{zhang2021trans4trans,
  title={Trans4Trans: Efficient transformer for transparent object segmentation to help visually impaired people navigate in the real world},
  author={Zhang, Jiaming and Yang, Kailun and Constantinescu, Angela and Peng, Kunyu and M{"u}ller, Karin and Stiefelhagen, Rainer},
  booktitle={Proceedings of the IEEE/CVF International Conference on Computer Vision Workshops},
  pages={1760--1770},
  year={2021}
}

@article{zhang2022trans4trans,
  title={Trans4Trans: Efficient transformer for transparent object and semantic scene segmentation in real-world navigation assistance},
  author={Zhang, Jiaming and Yang, Kailun and Constantinescu, Angela and Peng, Kunyu and M{"u}ller, Karin and Stiefelhagen, Rainer},
  journal={IEEE Transactions on Intelligent Transportation Systems},
  year={2022},
  publisher={IEEE}
}