| <p>Artifex v0.1 uses <strong>no manual labels</strong>. Every label is derived deterministically from museum metadata by code and | |
| configuration that can be reviewed. This document describes how those derivations are reviewed and how manual review | |
| should be done if it is added.</p> | |
| <h2>Automatic labelling (current)</h2> | |
| <table> | |
| <thead> | |
| <tr> | |
| <th>Label</th> | |
| <th>Source fields</th> | |
| <th>Rule</th> | |
| <th>Review artefact</th> | |
| </tr> | |
| </thead> | |
| <tbody> | |
| <tr> | |
| <td><code>date_start</code>, <code>date_end</code></td> | |
| <td>Met <code>objectBeginDate</code>/<code>objectEndDate</code>; Cleveland <code>creation_date_earliest</code>/<code>latest</code></td> | |
| <td>historical → astronomical years; unusable values recorded in <code>date_issue</code></td> | |
| <td><code>manifest.csv</code> (<code>original_date_text</code> beside the range)</td> | |
| </tr> | |
| <tr> | |
| <td><code>period_label</code></td> | |
| <td>date range</td> | |
| <td>≥ 75 % of range inside one bin</td> | |
| <td><code>period_status</code> column</td> | |
| </tr> | |
| <tr> | |
| <td><code>region_normalized</code>, <code>culture_normalized</code></td> | |
| <td>culture, place fields, artist nationality, Egyptian Art department</td> | |
| <td>ordered regex rules; multi-region matches → ambiguous</td> | |
| <td><code>region_mapping_report.csv</code></td> | |
| </tr> | |
| <tr> | |
| <td>eligibility</td> | |
| <td>licence flags, classification, object name, title</td> | |
| <td>see <code>configs/dataset.yaml</code> → <code>filters</code></td> | |
| <td><code>exclusions.csv</code></td> | |
| </tr> | |
| <tr> | |
| <td>duplicates</td> | |
| <td>image URL, image bytes</td> | |
| <td>first artwork id is canonical</td> | |
| <td><code>duplicate_of</code> column</td> | |
| </tr> | |
| </tbody> | |
| </table> | |
| <h2>Reviewing the region mapping</h2> | |
| <ol> | |
| <li>Open <code>data/manifests/region_mapping_report.csv</code> (one row per distinct source string per field, with counts).</li> | |
| <li>Sort by <code>count</code>. Check every <code>unmapped</code> and <code>ambiguous</code> string with ≥ 3 records, and a sample of <code>assigned</code> strings | |
| per broad region.</li> | |
| <li>If a mapping is wrong, change <code>configs/region_mapping.yaml</code> (never the manifest), bump its <code>version</code>, rebuild: | |
| <code>python scripts/build_dataset.py</code>. The dataset version changes, which invalidates the trained model and evaluation, | |
| forcing re-training and re-evaluation.</li> | |
| <li>Record the reason for each rule change in the commit message.</li> | |
| </ol> | |
| <p>Decisions taken so far (see <code>label_taxonomy.md</code>): ambiguous strings stay unlabelled; explicit "Painting: X" clauses win; | |
| compound terms such as "Roman Egypt" use override rules; dynasty names are not used as place indicators except where | |
| the dynasty is unambiguous about place (Safavid, Qajar).</p> | |
| <h2>If manual annotation is added</h2> | |
| <ul> | |
| <li>Annotators label from the <strong>museum record and scholarly sources</strong>, never from the model's prediction (the prediction | |
| must be hidden during annotation to avoid anchoring).</li> | |
| <li>Each manual label stores: annotator id, date, source consulted, confidence (certain / probable / uncertain), and free-text | |
| justification. Manual labels live in a separate file keyed by <code>artwork_id</code>; the original museum fields are never edited.</li> | |
| <li>At least 10 % of manually labelled records are double-annotated; agreement (Cohen's κ) is reported.</li> | |
| <li>Manual corrections are evaluated as a separate label set, so results with and without them can be compared.</li> | |
| </ul> |