Add model card and metadata

#1
by nielsr HF Staff - opened
Files changed (1) hide show
  1. README.md +58 -0
README.md ADDED
@@ -0,0 +1,58 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ library_name: transformers
3
+ pipeline_tag: video-text-to-text
4
+ tags:
5
+ - traffic-safety
6
+ - video-understanding
7
+ - autonomous-driving
8
+ ---
9
+
10
+ # CrashChat: A Multimodal Large Language Model for Multitask Traffic Crash Video Analysis
11
+
12
+ CrashChat is a Multimodal Large Language Model (MLLM) specifically designed for traffic crash video analysis. Built upon VideoLLaMA3, it provides a unified framework for tasks such as crash recognition, temporal grounding, and high-level understanding (description and reasoning) across diverse driving scenarios.
13
+
14
+ - **Paper:** [CrashChat: A Multimodal Large Language Model for Multitask Traffic Crash Video Analysis](https://arxiv.org/abs/2512.18878)
15
+ - **Repository:** [https://github.com/Liangkd/CrashChat](https://github.com/Liangkd/CrashChat)
16
+
17
+ ## Introduction
18
+
19
+ CrashChat addresses the challenges of automating crash video analysis by incorporating three key contributions:
20
+ 1. **Multitask Learning Strategy**: Employs task decoupling and grouping to maximize joint learning benefits while mitigating negative transfer between linguistic-centric and perception-centric tasks.
21
+ 2. **Unified Framework**: Capable of performing crash recognition, temporal grounding, and reasoning within a single model.
22
+ 3. **Domain-Specific Fine-tuning**: Trained on a comprehensive instruction-tuning dataset comprising 18,385 videos and over 96,000 video-QA pairs.
23
+
24
+ Numerical experiments demonstrate that CrashChat achieves state-of-the-art performance, reaching near-perfect accuracy in crash recognition and significant improvements in both crash localization and causal reasoning compared to general-purpose MLLMs.
25
+
26
+ ## Model Zoo
27
+
28
+ The project includes several fine-tuned checkpoints based on different training strategies:
29
+
30
+ | Checkpoint | MLLM backbone | Training Strategy - Task |
31
+ |----------|-------------|-------------|
32
+ | VideoLLaMA-3 | VideoLLaMA-3 7B | VideoLLaMA3 - baseline |
33
+ | CrashChat-7B-Finetuned | VideoLLaMA-3 7B | Independent monotask models - crash recognition |
34
+ | CrashChat-7B-Finetuned | VideoLLaMA-3 7B | Independent monotask models - crash description |
35
+ | CrashChat-7B-Finetuned | VideoLLaMA-3 7B | Independent monotask models - causal reasoning |
36
+ | CrashChat-7B-Finetuned | VideoLLaMA-3 7B | Independent monotask models - prevention reasoning |
37
+ | CrashChat-7B-Finetuned | VideoLLaMA-3 7B | Independent monotask models - pre-crash localization |
38
+ | CrashChat-7B-Finetuned | VideoLLaMA-3 7B | Independent monotask models - crash localization |
39
+ | CrashChat-7B-Finetuned | VideoLLaMA-3 7B | Homogeneous multitask models - linguistic-centric tasks |
40
+ | CrashChat-7B-Finetuned | VideoLLaMA-3 7B | Homogeneous multitask models - perception-centric tasks |
41
+ | CrashChat-7B-Finetuned | VideoLLaMA-3 7B | Heterogeneous multitask models - all tasks |
42
+
43
+ ## Citation
44
+
45
+ If you find CrashChat useful for your research, please cite:
46
+
47
+ ```bibtex
48
+ @article{liang2025crashchat,
49
+ title = {CrashChat: A Multimodal Large Language Model for Multitask Traffic Crash Video Analysis},
50
+ author = {Liang, Kaidi and Li, Ke and Hu, Xianbiao and Qin, Ruwen},
51
+ journal = {arXiv preprint arXiv:2512.18878},
52
+ year = {2025},
53
+ archivePrefix = {arXiv},
54
+ eprint = {2512.18878},
55
+ primaryClass = {cs.CV},
56
+ url = {https://arxiv.org/abs/2512.18878}
57
+ }
58
+ ```