---
pipeline_tag: image-segmentation
library_name: pytorch
license: apache-2.0
tags:
- transformer
- transparent-object-segmentation
- semantic-segmentation
- navigation-assistance
- iccvw-2021
- t-its-2022
- trans10k
- cocostuff
datasets:
- cocostuff
---

# Trans4Trans: Efficient Transformer for Transparent Object and Semantic Scene Segmentation

Official model checkpoints for **Trans4Trans: Efficient Transformer for Transparent Object Segmentation to Help Visually Impaired People Navigate in the Real World** (ICCVW 2021) and **Trans4Trans: Efficient Transformer for Transparent Object and Semantic Scene Segmentation in Real-World Navigation Assistance** (IEEE T-ITS 2022).

<div align="center">

[![Paper (ICCVW)](https://img.shields.io/badge/arXiv-2107.03172-B31B1B.svg)](https://arxiv.org/abs/2107.03172)
[![Paper (T-ITS)](https://img.shields.io/badge/arXiv-2108.09174-B31B1B.svg)](https://arxiv.org/abs/2108.09174)
[![GitHub](https://img.shields.io/badge/GitHub-Trans4trans-181717.svg?logo=github)](https://github.com/InSAI-Lab/Trans4trans)
[![Collection](https://img.shields.io/badge/HF-Collection-FFD21E.svg)](https://huggingface.co/collections/InSAI-Lab)

</div>

---

## 📌 Introduction

Trans4Trans is an efficient Transformer-based system designed for transparent object and semantic scene segmentation, targeting real-world wearable navigation assistance for visually impaired people. The architecture couples a lightweight hierarchical transformer backbone with a Frequency-aware Parsing Transformer (FPT) decoder and an assistive system integrating Intel RealSense depth sensing and acoustic feedback.

![Trans4Trans Overview](https://raw.githubusercontent.com/InSAI-Lab/Trans4trans/main/trans4trans_fig_1.jpg)

---

## 📂 Model Checkpoints

The repository contains pretrained weights organized by dataset benchmark:

```text
InSAI-Lab/Trans4trans
├── cocostuff/
│   └── model_cocostuff.pth      # General scene context (COCO-Stuff, 162.6 MB)
└── trans10kv2/
    └── model_trans.pth          # Transparent object segmentation (Trans10K-v2, 150.3 MB)
```

| Checkpoint | Target Task / Dataset | Size | Direct Download |
| :--- | :--- | :---: | :--- |
| `cocostuff/model_cocostuff.pth` | COCO-Stuff General Context | 162.6 MB | [Download](https://huggingface.co/InSAI-Lab/Trans4trans/resolve/main/cocostuff/model_cocostuff.pth) |
| `trans10kv2/model_trans.pth` | Trans10K-v2 Transparent Objects | 150.3 MB | [Download](https://huggingface.co/InSAI-Lab/Trans4trans/resolve/main/trans10kv2/model_trans.pth) |

---

## 🚀 Quick Start

### Download via Hugging Face CLI

```bash
# Download all checkpoints into workdirs/
hf download InSAI-Lab/Trans4trans --local-dir workdirs
```

### Download via Python (`huggingface_hub`)

```python
from huggingface_hub import hf_hub_download

# Download transparent object segmentation model
ckpt_path = hf_hub_download(
    repo_id="InSAI-Lab/Trans4trans",
    filename="trans10kv2/model_trans.pth",
    local_dir="workdirs"
)
print(f"Downloaded model to {ckpt_path}")
```

### Assistive System Demo

```bash
# Save weights under workdirs/cocostuff and workdirs/trans10kv2
bash demo.sh
```

---

## 📑 Citation

```bibtex
@inproceedings{zhang2021trans4trans,
  title={Trans4Trans: Efficient transformer for transparent object segmentation to help visually impaired people navigate in the real world},
  author={Zhang, Jiaming and Yang, Kailun and Constantinescu, Angela and Peng, Kunyu and M{"u}ller, Karin and Stiefelhagen, Rainer},
  booktitle={Proceedings of the IEEE/CVF International Conference on Computer Vision Workshops},
  pages={1760--1770},
  year={2021}
}

@article{zhang2022trans4trans,
  title={Trans4Trans: Efficient transformer for transparent object and semantic scene segmentation in real-world navigation assistance},
  author={Zhang, Jiaming and Yang, Kailun and Constantinescu, Angela and Peng, Kunyu and M{"u}ller, Karin and Stiefelhagen, Rainer},
  journal={IEEE Transactions on Intelligent Transportation Systems},
  year={2022},
  publisher={IEEE}
}
```