code/docs/datasets.md

# M4SEG data setup

The [M⁴Seg dataset](https://huggingface.co/datasets/PLAN-Lab/M4Seg) contains exactly two splits: 743,463 training examples and the original 1,000 test examples. It provides annotations, compressed COCO RLE masks, and relative image paths. Images must be obtained from their upstream datasets.

Download the native annotations, source metadata, and preparation tool:

```bash
huggingface-cli download PLAN-Lab/M4Seg \
  --repo-type dataset --local-dir ./data/m4seg \
  --include "annotations/*" "metadata/*" "prepare_images.py" "PREPARATION.md"
```

Obtain the images from the original [source projects](https://huggingface.co/datasets/PLAN-Lab/M4Seg#source-images), then create the expected layout with the downloaded tool:

```bash
python ./data/m4seg/prepare_images.py \
  --dataset-dir ./data \
  --coco /path/to/coco2017 \
  --ade20k /path/to/ADEChallengeData2016 \
  --ade20k-part /path/to/ADE20KPart234 \
  --pascal /path/to/VOCdevkit/VOC2010 \
  --verify-sha256
```

The tool links the source image directories and checks them against the released manifest. Add `--split test` to prepare only evaluation images. See the [source preparation guide](https://huggingface.co/datasets/PLAN-Lab/M4Seg/blob/main/PREPARATION.md) for extraction layouts and source-specific details. ADEChallengeData2016 object images and ADE20KPart234 part images must use their respective image versions; matching JPEG filenames can have different pixels.

Set `dataset_dir` to the prepared image root and `m4seg_annpath` to the desired compressed annotation file. For the layout above:

```yaml
dataset_dir: ./data
m4seg_annpath: ./data/m4seg/annotations/m4seg_train.json.gz
```

For evaluation, use the test annotation file instead. The loader reads compressed JSON directly. Native dimensions and masks are included, so separate source annotation databases are unnecessary with the released format.

To explore the dataset through its Parquet interface, install `datasets` and call `datasets.load_dataset("PLAN-Lab/M4Seg")`. That interface uses one row per example, with ordered image records and a flat list of grounding annotations. The native JSON files preserve the dictionary layout expected by PRIMA. The dataset card documents both representations.

`granularity` describes the source annotation family: `object` or `part`. Examples from part sources can ground objects, parts, or both. Grounding targets may occur in a single input image. Object entity codes use an image index followed by a two- or three-digit object index; part codes retain the five-digit image/object/part encoding. Stored answer offsets bind each mention to its grounding, and repeated mentions do not create duplicate training masks.

The only training and evaluation dataset adapter is M4SEG. Its shared utility layer also supports the original annotation format and constituent image-source databases for existing local workflows; these do not add separate training tasks.

Preserve image order, grounded phrases, and masks. `example_id` uniquely identifies each released row; `pair_id` retains the original identifier, including repeated training IDs. Test IDs remain unchanged. Keep train and test separate, and score predictions against the same annotation version used for generation.

For original-format annotation files, the loader uses the original source tables when available and otherwise finds the released compact image/category catalogs under `dataset_dir/m4seg/metadata`. The ordered ADE20KPart234 class map is also included in the dataset metadata; its copy in this codebase supports the original-format loader.