procaffenator's picture
Document models, configurations and evaluation results
06d7059 verified
|
Raw History Blame Contribute Delete
2.6 kB
# Data preparation
## Dataset 2
The public images and original XML annotations are described in [An open dataset of annotated metaphase cell images for chromosome identification](https://doi.org/10.1038/s41597-023-02003-7). Download the submitted dataset from [CIL:54816](https://doi.org/10.7295/W9CIL54816), then extract the archive. The required folder is `24_chromosomes_object/`, containing `annotations/` and `JEPG/` (the spelling in the original archive).
Use the included split index to reconstruct the exact COCO files used for evaluation:
```bash
python tools/prepare_dataset2.py \
--source /path/to/24_chromosomes_object \
--output /path/to/dataset2/coco
python tools/check_dataset.py /path/to/dataset2/coco
```
The output directory must be new or empty. The tool copies the original images, preserves the XML box coordinates without a one-pixel shift, and assigns the image and category IDs in `dataset2_split.json`. No random splitting, resizing or augmentation is performed. Reconstructed image records, categories and annotations were compared with the evaluated COCO files and matched exactly for all three splits.
```text
coco/
β”œβ”€β”€ train/
β”‚ β”œβ”€β”€ _annotations.coco.json
β”‚ └── image files
β”œβ”€β”€ valid/
β”‚ β”œβ”€β”€ _annotations.coco.json
β”‚ └── image files
└── test/
β”œβ”€β”€ _annotations.coco.json
└── image files
```
| Split | Images | Annotations |
|---|---:|---:|
| Train | 3,500 | 160,888 |
| Validation | 500 | 22,984 |
| Test | 1,000 | 45,980 |
All splits use the same 24 chromosome classes. Keep the provided split, image IDs and category ordering when comparing with the reported results. `check_dataset.py` verifies image IDs, filenames, categories, annotation counts and image availability. The preparation check used the archived original dataset; it was not a new download from the hosting service.
The small image set under `figures/data/` and `demo/data/` contains Dataset 2 examples used in the manuscript and demonstrations. Attribution: J.-J. Tseng and colleagues, the dataset publication above and CIL:54816. The original dataset's terms apply to these images; the software license does not replace them.
## Dataset 1
Dataset 1 is an in-house cohort. Images are not distributed with this repository. Its test set contains 340 images; use the source-group split and annotations supplied by the data custodian. The configurations document transforms and class ordering. `--data-root` selects the local dataset directory. The public repository alone does not provide access to this cohort.