M4SEG data setup
The M⁴Seg dataset contains exactly two splits: 743,463 training examples and the original 1,000 test examples. It provides annotations, compressed COCO RLE masks, and relative image paths. Images must be obtained from their upstream datasets.
Download the native annotations, source metadata, and preparation tool:
huggingface-cli download PLAN-Lab/M4Seg \
--repo-type dataset --local-dir ./data/m4seg \
--include "annotations/*" "metadata/*" "prepare_images.py" "PREPARATION.md"
Obtain the images from the original source projects, then create the expected layout with the downloaded tool:
python ./data/m4seg/prepare_images.py \
--dataset-dir ./data \
--coco /path/to/coco2017 \
--ade20k /path/to/ADEChallengeData2016 \
--ade20k-part /path/to/ADE20KPart234 \
--pascal /path/to/VOCdevkit/VOC2010 \
--verify-sha256
The tool links the source image directories and checks them against the released manifest. Add --split test to prepare only evaluation images. See the source preparation guide for extraction layouts and source-specific details. ADEChallengeData2016 object images and ADE20KPart234 part images must use their respective image versions; matching JPEG filenames can have different pixels.
Set dataset_dir to the prepared image root and m4seg_annpath to the desired compressed annotation file. For the layout above:
dataset_dir: ./data
m4seg_annpath: ./data/m4seg/annotations/m4seg_train.json.gz
For evaluation, use the test annotation file instead. The loader reads compressed JSON directly. Native dimensions and masks are included, so separate source annotation databases are unnecessary with the released format.
To explore the dataset through its Parquet interface, install datasets and call datasets.load_dataset("PLAN-Lab/M4Seg"). That interface uses one row per example, with ordered image records and a flat list of grounding annotations. The native JSON files preserve the dictionary layout expected by PRIMA. The dataset card documents both representations.
granularity describes the source annotation family: object or part. Examples from part sources can ground objects, parts, or both. Grounding targets may occur in a single input image. Object entity codes use an image index followed by a two- or three-digit object index; part codes retain the five-digit image/object/part encoding. Stored answer offsets bind each mention to its grounding, and repeated mentions do not create duplicate training masks.
The only training and evaluation dataset adapter is M4SEG. Its shared utility layer also supports the original annotation format and constituent image-source databases for existing local workflows; these do not add separate training tasks.
Preserve image order, grounded phrases, and masks. example_id uniquely identifies each released row; pair_id retains the original identifier, including repeated training IDs. Test IDs remain unchanged. Keep train and test separate, and score predictions against the same annotation version used for generation.
For original-format annotation files, the loader uses the original source tables when available and otherwise finds the released compact image/category catalogs under dataset_dir/m4seg/metadata. The ordered ADE20KPart234 class map is also included in the dataset metadata; its copy in this codebase supports the original-format loader.