| # Data Curation |
|
|
| In Stage IV, we curate a customized dataset to make LMM-Det excel in object detection while preserving its inherent capabilities like caption generation and VQA. |
|
|
| ## Step 1 |
|
|
| We generate pesudo labels on the trainset of COCO using [Salience-DETR](https://github.com/xiuqhou/Salience-DETR) (FocalNet-L backone), and re-organize them into a instruction format. Note that the re-organization data consists of ground-truth labels and pesudo labels. |
| (In practice, this data is aslo used in Stage III.) |
|
|
| ## Step 2 |
|
|
| We remove the textcaps data in the LLaVA-665K instruction data. |
|
|
| ## Step 3 |
|
|
| We concat the the re-organization data and the LLaVA-665K instruction data (without textcaps) as the training data in Stage IV. |
