docs: add ISSAFE model card
Browse files
README.md
ADDED
|
@@ -0,0 +1,107 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: apache-2.0
|
| 3 |
+
library_name: pytorch
|
| 4 |
+
tags:
|
| 5 |
+
- multimodal
|
| 6 |
+
- semantic-segmentation
|
| 7 |
+
- vision
|
| 8 |
+
- event-camera
|
| 9 |
+
- accident-scene
|
| 10 |
+
- iros-2021
|
| 11 |
+
- tits-2021
|
| 12 |
+
- edcnet
|
| 13 |
+
- issafe
|
| 14 |
+
pipeline_tag: image-segmentation
|
| 15 |
+
---
|
| 16 |
+
|
| 17 |
+
# ISSAFE & EDCNet: Event-Driven Scene Segmentation in Accidents
|
| 18 |
+
|
| 19 |
+
[IROS 2021 & IEEE T-ITS 2021] Official Model Checkpoints Repository for **ISSAFE** and **EDCNet**.
|
| 20 |
+
|
| 21 |
+
- **GitHub Repository:** https://github.com/InSAI-Lab/ISSAFE
|
| 22 |
+
- **Dataset Repository:** [InSAI-Lab/DADA-seg](https://huggingface.co/datasets/InSAI-Lab/DADA-seg)
|
| 23 |
+
- **Collection:** [ISSAFE & EDCNet (IROS 2021 & T-ITS 2021)](https://huggingface.co/collections/InSAI-Lab/issafe-and-edcnet-iros-2021-and-t-its-2021-6ad1b2429101a4c06509e4cb)
|
| 24 |
+
- **Paper (IROS 2021):** [ISSAFE: Improving Semantic Segmentation in Accidents by Fusing Event-based Data](https://arxiv.org/abs/2008.08974)
|
| 25 |
+
- **Paper (IEEE T-ITS 2021):** [Exploring Event-Driven Dynamic Context for Accident Scene Segmentation](https://arxiv.org/abs/2112.05006)
|
| 26 |
+
|
| 27 |
+
---
|
| 28 |
+
|
| 29 |
+
## Model Description
|
| 30 |
+
|
| 31 |
+
**EDCNet** (Event-Driven dynamic Context Network) and **ISSAFE** address semantic segmentation in extreme traffic scenarios and accident scenes. By leveraging event cameras with microsecond temporal resolution, the architecture extracts dynamic motion cues to compensate for motion blur, severe exposure shifts, and sudden collisions where traditional static RGB cameras fail.
|
| 32 |
+
|
| 33 |
+
Key Features:
|
| 34 |
+
- **Asymmetric Multimodal Architecture:** Combines an RGB stream and an Event Volume stream.
|
| 35 |
+
- **Dynamic Context Mining:** Preserves fine-grained motion of crashing foreground objects.
|
| 36 |
+
- **Domain-to-Scene (D2S) Adaptation:** Trained on synthetic/real event volumes and generalized to critical accident scenarios.
|
| 37 |
+
|
| 38 |
+
## Checkpoints Organization
|
| 39 |
+
|
| 40 |
+
The repository provides trained model weights matching the official ISSAFE codebase evaluation structure:
|
| 41 |
+
|
| 42 |
+
```text
|
| 43 |
+
βββ run/
|
| 44 |
+
β βββ cityscapesevent/
|
| 45 |
+
β βββ test_EDCNet_r18/
|
| 46 |
+
β βββ model_best.pth # EDCNet (ResNet-18, B=2 event time bins)
|
| 47 |
+
βββ test_EDCNet_r18/
|
| 48 |
+
βββ model_best.pth # Convenience direct path copy
|
| 49 |
+
```
|
| 50 |
+
|
| 51 |
+
## Quick Start & Evaluation
|
| 52 |
+
|
| 53 |
+
### 1. Download Model Weights
|
| 54 |
+
|
| 55 |
+
Using Hugging Face CLI:
|
| 56 |
+
|
| 57 |
+
```bash
|
| 58 |
+
# Download model weight to the local workspace
|
| 59 |
+
hf download InSAI-Lab/ISSAFE run/cityscapesevent/test_EDCNet_r18/model_best.pth --local-dir .
|
| 60 |
+
```
|
| 61 |
+
|
| 62 |
+
Using Python (`huggingface_hub`):
|
| 63 |
+
|
| 64 |
+
```python
|
| 65 |
+
from huggingface_hub import hf_hub_download
|
| 66 |
+
|
| 67 |
+
model_path = hf_hub_download(
|
| 68 |
+
repo_id="InSAI-Lab/ISSAFE",
|
| 69 |
+
filename="run/cityscapesevent/test_EDCNet_r18/model_best.pth",
|
| 70 |
+
local_dir="."
|
| 71 |
+
)
|
| 72 |
+
```
|
| 73 |
+
|
| 74 |
+
### 2. Evaluation
|
| 75 |
+
|
| 76 |
+
In the [ISSAFE repository](https://github.com/InSAI-Lab/ISSAFE):
|
| 77 |
+
|
| 78 |
+
```bash
|
| 79 |
+
# Evaluate EDCNet on DADA-seg validation set
|
| 80 |
+
python eval.py --dataset dadaevent --weight-path run/cityscapesevent/test_EDCNet_r18/model_best.pth
|
| 81 |
+
```
|
| 82 |
+
|
| 83 |
+
## Citation
|
| 84 |
+
|
| 85 |
+
```bibtex
|
| 86 |
+
@inproceedings{zhang2021issafe,
|
| 87 |
+
author={Zhang, Jiaming and Yang, Kailun and Stiefelhagen, Rainer},
|
| 88 |
+
booktitle={2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)},
|
| 89 |
+
title={ISSAFE: Improving Semantic Segmentation in Accidents by Fusing Event-based Data},
|
| 90 |
+
year={2021},
|
| 91 |
+
pages={1132-1139},
|
| 92 |
+
doi={10.1109/IROS51168.2021.9636109}
|
| 93 |
+
}
|
| 94 |
+
|
| 95 |
+
@article{zhang2021edcnet,
|
| 96 |
+
author={Zhang, Jiaming and Yang, Kailun and Stiefelhagen, Rainer},
|
| 97 |
+
journal={IEEE Transactions on Intelligent Transportation Systems},
|
| 98 |
+
title={Exploring Event-Driven Dynamic Context for Accident Scene Segmentation},
|
| 99 |
+
year={2021},
|
| 100 |
+
pages={1-17},
|
| 101 |
+
doi={10.1109/TITS.2021.3134828}
|
| 102 |
+
}
|
| 103 |
+
```
|
| 104 |
+
|
| 105 |
+
## License
|
| 106 |
+
|
| 107 |
+
This repository is licensed under the [Apache-2.0 License](https://www.apache.org/licenses/LICENSE-2.0).
|