| --- |
| pipeline_tag: image-segmentation |
| library_name: pytorch |
| license: apache-2.0 |
| tags: |
| - transformer |
| - transparent-object-segmentation |
| - semantic-segmentation |
| - navigation-assistance |
| - iccvw-2021 |
| - t-its-2022 |
| - trans10k |
| - cocostuff |
| datasets: |
| - cocostuff |
| --- |
| |
| # Trans4Trans: Efficient Transformer for Transparent Object and Semantic Scene Segmentation |
|
|
| Official model checkpoints for **Trans4Trans: Efficient Transformer for Transparent Object Segmentation to Help Visually Impaired People Navigate in the Real World** (ICCVW 2021) and **Trans4Trans: Efficient Transformer for Transparent Object and Semantic Scene Segmentation in Real-World Navigation Assistance** (IEEE T-ITS 2022). |
|
|
| <div align="center"> |
|
|
| [](https://arxiv.org/abs/2107.03172) |
| [](https://arxiv.org/abs/2108.09174) |
| [](https://github.com/InSAI-Lab/Trans4trans) |
| [](https://huggingface.co/collections/InSAI-Lab) |
|
|
| </div> |
|
|
| --- |
|
|
| ## 📌 Introduction |
|
|
| Trans4Trans is an efficient Transformer-based system designed for transparent object and semantic scene segmentation, targeting real-world wearable navigation assistance for visually impaired people. The architecture couples a lightweight hierarchical transformer backbone with a Frequency-aware Parsing Transformer (FPT) decoder and an assistive system integrating Intel RealSense depth sensing and acoustic feedback. |
|
|
|  |
|
|
| --- |
|
|
| ## 📂 Model Checkpoints |
|
|
| The repository contains pretrained weights organized by dataset benchmark: |
|
|
| ```text |
| InSAI-Lab/Trans4trans |
| ├── cocostuff/ |
| │ └── model_cocostuff.pth # General scene context (COCO-Stuff, 162.6 MB) |
| └── trans10kv2/ |
| └── model_trans.pth # Transparent object segmentation (Trans10K-v2, 150.3 MB) |
| ``` |
|
|
| | Checkpoint | Target Task / Dataset | Size | Direct Download | |
| | :--- | :--- | :---: | :--- | |
| | `cocostuff/model_cocostuff.pth` | COCO-Stuff General Context | 162.6 MB | [Download](https://huggingface.co/InSAI-Lab/Trans4trans/resolve/main/cocostuff/model_cocostuff.pth) | |
| | `trans10kv2/model_trans.pth` | Trans10K-v2 Transparent Objects | 150.3 MB | [Download](https://huggingface.co/InSAI-Lab/Trans4trans/resolve/main/trans10kv2/model_trans.pth) | |
|
|
| --- |
|
|
| ## 🚀 Quick Start |
|
|
| ### Download via Hugging Face CLI |
|
|
| ```bash |
| # Download all checkpoints into workdirs/ |
| hf download InSAI-Lab/Trans4trans --local-dir workdirs |
| ``` |
|
|
| ### Download via Python (`huggingface_hub`) |
| |
| ```python |
| from huggingface_hub import hf_hub_download |
|
|
| # Download transparent object segmentation model |
| ckpt_path = hf_hub_download( |
| repo_id="InSAI-Lab/Trans4trans", |
| filename="trans10kv2/model_trans.pth", |
| local_dir="workdirs" |
| ) |
| print(f"Downloaded model to {ckpt_path}") |
| ``` |
| |
| ### Assistive System Demo |
|
|
| ```bash |
| # Save weights under workdirs/cocostuff and workdirs/trans10kv2 |
| bash demo.sh |
| ``` |
|
|
| --- |
|
|
| ## 📑 Citation |
|
|
| ```bibtex |
| @inproceedings{zhang2021trans4trans, |
| title={Trans4Trans: Efficient transformer for transparent object segmentation to help visually impaired people navigate in the real world}, |
| author={Zhang, Jiaming and Yang, Kailun and Constantinescu, Angela and Peng, Kunyu and M{"u}ller, Karin and Stiefelhagen, Rainer}, |
| booktitle={Proceedings of the IEEE/CVF International Conference on Computer Vision Workshops}, |
| pages={1760--1770}, |
| year={2021} |
| } |
| |
| @article{zhang2022trans4trans, |
| title={Trans4Trans: Efficient transformer for transparent object and semantic scene segmentation in real-world navigation assistance}, |
| author={Zhang, Jiaming and Yang, Kailun and Constantinescu, Angela and Peng, Kunyu and M{"u}ller, Karin and Stiefelhagen, Rainer}, |
| journal={IEEE Transactions on Intelligent Transportation Systems}, |
| year={2022}, |
| publisher={IEEE} |
| } |
| ``` |
|
|
